In the mobile scenario, DNS resolution overhead is a non negligible part of the whole network request. In the weak network environment, based on UDP The local DNS resolution of is very prone to the problem of resolution timeout, and even if the resolution is successful, it will consume hundreds of milliseconds or even more, which is very unfavorable to our whole business request, and it directly affects the customer's experience.
For a popular application, the optimization of DNS has a great weight on the network optimization of the whole application. Next, we have a comprehensive understanding of DNS from the following aspects. I believe it will be of great help to the network optimization in our development.
1. DNS
1.1 understanding DNS
DNS(Domain Name System) is the English abbreviation of domain name "system". It is a computer and network service naming system organized into domain hierarchy. It is used in TCP/IP networks. The service it provides is used to convert host names and domain names into IP addresses. For more information: DNS domain name resolution service under Linux environment
As the top link of network communication, its importance in the whole process of network communication is self-evident.
After ios10, the native http request method provided by apple can return the time indicators of each stage of http request, including DNS resolution time.
1.2 DNS resolution related concepts
1.2.1 DNS domain name hierarchy
DNS is a hierarchical structure. It forms a tree system in the whole Internet. The top layer is the root domain name of the system, and the lower layer is TLD and secondary domain names. The leaves constitute the so-called FQDN (Fully Qualified Domain Names). The root domain name is usually represented by ".". In fact, it is also composed of domain names. There are 13 groups of domain name root nodes all over the world, It is managed by a few countries, while there are only a few root node images in China.
1.2.2 authoritative DNS
Authoritative DNS is a server authorized by the upper level to resolve domain names. At the same time, it can delegate the resolution authorization to others. For example, COM top-level server can authorize xxorg.com. The authoritative server of this domain name is NS.ABC.COM. At the same time, NS.ABC.COM can also delegate the authorization to NS.DDD.COM. In this way, NS.DDD.COM has become the actual authoritative server of ABC.COM. Usually, the results of domain name resolution come from authoritative DNS. eg: Alibaba cloud analysis https://wanwang.aliyun.com/domain/dns
1.2.3 recursive DNS
Recursive DNS, also known as Local DNS, does not have the right to determine the domain name resolution results, but acts as an agent for users to obtain the domain name resolution results from authoritative DNS. There is a cache module on recursive DNS. When the target domain name has cached resolution results and the TTL has not expired (each domain name has TTL time, i.e. effective lifetime. If the cache time of domain name resolution results exceeds TTL, it is necessary to obtain the resolution results from authoritative DNS again), recursive DNS will return the cache results. Otherwise, Recursive DNS will query the authoritative DNS of domain names at all levels level by level until the final resolution result of the complete domain name is obtained. eg: our own computers, DNS servers provided by operators, etc.
1.2.4 public DNS
Public DNS is a special case of recursive DNS. It is an open recursive DNS service in the whole network. The traditional recursive DNS information is generally distributed to users by operators.
In the process of DNS resolution, the user initiates a request to recursive DNS, and recursive DNS requests the resolution result from authoritative DNS. It can be said that recursive DNS plays a role of forwarding. ISP DNS is recursive DNS; At the same time, some individuals or Internet service providers will set up their own recursive DNS for everyone to use, which is called public DNS.
| Operator | anycast | Official website | DNS |
|---|---|---|---|
| Nanjing trade wind public park | Nanjing, Jinan, Chicago | www.114dns.com | 114.114.114..114 114.114.115.115 |
| Alibaba cloud public DNS | Chengdu, Shenzhen, Hangzhou | http://www.alidns.com | 233.5.5.5 233.6.6.6 |
| Google Public DNS | Google 36 data centers | developers.google.com/speed/publi... | 8.8.8.8 8.8.4.4 |
National DNS summary: www.114dns.com/DNS_List.ht… ipip: tools.ipip.net/dns.php
1.2.5 forwarding DNS
It can be understood as a transit station between recursive DNS and users. It does not provide the service of directly resolving domain names. It forwards requests to recursive DNS, and then forwards the results of recursive DNS. It also provides caching. For example, the DNS server of a daily home router is generally 192.168.1.1, which is only forwarded to recursive DNS. For more information: Super clear DNS principle getting started guide!
1.3 domain name resolution record method
Domain name resolution records are mainly divided into A records, MIX records, CNAME records, NS records and TXT records:
-
A record: a stands for Address and is used to specify the ip corresponding to the domain name. For example, www.hello.com is assigned to 113.112.3.xxx. A record can resolve multiple domain names to one ip Address, but it cannot resolve one domain name to multiple ip addresses
-
MIX record: Mail Exchange means that you can send mail servers under a domain name directly to your Mail Server
-
CNAME record: Canonical Name, that is, alias resolution. The so-called alias resolution is to set one or more aliases for a domain name, such as aaa.com to bbb.net and ccc.com to bbb.net, where bbb.net is the alias of aaa.com and ccc.com respectively
-
NS record: specifies a DNS resolution server for a domain name, that is, the domain name is resolved by the DNS server with the specified IP address
-
TXT record: set a description for a host name or domain name. For example, ucloud.cn means that the TXT record is "trusted by the neutral security department"
1.4 domain name resolution process
The following is the specific process of dig Baidu display in the terminal:
macdeiMac:~ ethan$ dig +trace www.baidu.com ; <<>> DiG 9.10.6 <<>> +trace www.baidu.com ;; global options: +cmd . 1615 IN NS a.root-servers.net. . 1615 IN NS g.root-servers.net. . 1615 IN NS f.root-servers.net. . 1615 IN NS l.root-servers.net. . 1615 IN NS m.root-servers.net. . 1615 IN NS j.root-servers.net. . 1615 IN NS k.root-servers.net. . 1615 IN NS c.root-servers.net. . 1615 IN NS b.root-servers.net. . 1615 IN NS d.root-servers.net. . 1615 IN NS i.root-servers.net. . 1615 IN NS e.root-servers.net. . 1615 IN NS h.root-servers.net. ;; Received 239 bytes from 114.114.114.114#53(114.114.114.114) in 10 ms com. 172800 IN NS a.gtld-servers.net. com. 172800 IN NS b.gtld-servers.net. com. 172800 IN NS c.gtld-servers.net. com. 172800 IN NS d.gtld-servers.net. com. 172800 IN NS e.gtld-servers.net. com. 172800 IN NS f.gtld-servers.net. com. 172800 IN NS g.gtld-servers.net. com. 172800 IN NS h.gtld-servers.net. com. 172800 IN NS i.gtld-servers.net. com. 172800 IN NS j.gtld-servers.net. com. 172800 IN NS k.gtld-servers.net. com. 172800 IN NS l.gtld-servers.net. com. 172800 IN NS m.gtld-servers.net. com. 86400 IN DS 30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766 com. 86400 IN RRSIG DS 8 1 86400 20191112050000 20191030040000 22545 . Sr1g7h+DSqi+ekBQQS2ZBc/jt0zL+IR+Od/R9TnMjcy8Mw9RxLrMY2pm 1VYNqL5cAME1stSAfRUKwjD/vixnCeVLoJ6idCFOZeB+t/tTFQF/jfk1 td66pW9V/WgLIvslAwEZjidVeUFYERc7hZXr10BgzryZthizHISimuiQ qBjoIQN/uYULTnmePkIW07mnJXc9/AVZrjeI1AmvYC7wE0uR7DWNg1Ig dL4DaLDOM30qN7FBAD7K091uEgctpdxd/8G5XoYclSqroN4G6RibvkWT /vPCFRUzoaxembT5tR7gIz7gxdhN1r8NBD468JTG180MNUvb16Z/87U6 7UkMrg== ;; Received 1173 bytes from 192.58.128.30#53(j.root-servers.net) in 77 ms baidu.com. 172800 IN NS ns2.baidu.com. baidu.com. 172800 IN NS ns3.baidu.com. baidu.com. 172800 IN NS ns4.baidu.com. baidu.com. 172800 IN NS ns1.baidu.com. baidu.com. 172800 IN NS ns7.baidu.com. CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 - CK0Q1GIN43N1ARRC9OSM6QPQR81H5M9A NS SOA RRSIG DNSKEY NSEC3PARAM CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 8 2 86400 20191103044815 20191027033815 12163 com. U/FwNWeuKzR/uT2X/8Cf9TQmnaMdWf6XwBrFIIOCQ/kfKaOExEiT8LSQ 13OaEjtvFOOlIPK0XIbsL+dgGPYb/UV6sipBeQ1n8KuK18m3bYk47Ely oe+3VVp0zaiXt9DZrmRRenBB13o0DPqCbRHAHq1pj5zG5VkMufu9L/TT g80XlNukAMcu4GrV4VP8OimOQxz7HJbadci2oYn3beiHqQ== HPVV2B5N85O7HJJRB7690IB5UVF9O9UA.com. 86400 IN NSEC3 1 1 0 - HPVVN3Q5E5GOQP2QFE2LEM4SVB9C0SJ6 NS DS RRSIG HPVV2B5N85O7HJJRB7690IB5UVF9O9UA.com. 86400 IN RRSIG NSEC3 8 2 86400 20191105055359 20191029034359 12163 com. J5Dq0lGkcejjg1vPqWDBvNYaAhqFF3Ck8trKj4tgW5Z1bmoXsHGU6/Cl y3GlLfzb49xjiXzxVLCuAQJ9uLuKSX5cn+kesc8rwYqcVXU4nXbD5jzo u3CK2yHD3FqPDCOKlMSNy3KKkL03bB3DfmvAae/qQs7xSe6VTpCkR6v/ lo3UA/pMfTYBjSIOvR2KmpM9yFLmN5LXAQW3rNH8sW91BA== ;; Received 761 bytes from 192.26.92.30#53(c.gtld-servers.net) in 241 ms www.baidu.com. 1200 IN CNAME www.a.shifen.com. a.shifen.com. 1200 IN NS ns2.a.shifen.com. a.shifen.com. 1200 IN NS ns3.a.shifen.com. a.shifen.com. 1200 IN NS ns4.a.shifen.com. a.shifen.com. 1200 IN NS ns5.a.shifen.com. a.shifen.com. 1200 IN NS ns1.a.shifen.com.
;; Received 239 bytes from 180.76.76.92#53(ns7.baidu.com) in 10 ms
Through the above command, we can see the step-by-step process of DNS resolution. In the last step, we can see that www.baidu.com is CNAME to www.a.shifen.com, so we can check www.a.shifen.com to see its ip
;; Received 215 bytes from 220.181.33.31#53(ns2.baidu.com) in 28 ms www.a.shifen.com. 300 IN A 180.101.49.12 www.a.shifen.com. 300 IN A 180.101.49.11 a.shifen.com. 1200 IN NS ns4.a.shifen.com. a.shifen.com. 1200 IN NS ns5.a.shifen.com. a.shifen.com. 1200 IN NS ns1.a.shifen.com. a.shifen.com. 1200 IN NS ns2.a.shifen.com. a.shifen.com. 1200 IN NS ns3.a.shifen.com. ;; Received 236 bytes from 180.76.76.95#53(ns5.a.shifen.com) in 9 ms
Then we use the nslookup command to check whether Baidu's ip addresses are the two shown above:
macdeiMac:~ ethan$ nslookup www.baidu.com Server: 114.114.114.114 Address: 114.114.114.114#53 Non-authoritative answer: www.baidu.com canonical name = www.a.shifen.com. Name: www.a.shifen.com Address: 180.101.49.11 Name: www.a.shifen.com Address: 180.101.49.12
Figure the process of DNS resolving baidu:
-
The terminal sends a domain name resolution request to Local DNS
-
After obtaining the domain name request, Local DNS first obtains the address of the root domain name server from Root hins (Root hins contains the address information of the Internet DNS root server)
-
After obtaining the root domain name server address, Local DNS sends a DNS resolution request to the root domain name server, and the root domain name server returns the top-level domain name server address (com or cn or other)
-
Then, Local DNS sends a resolution request to the COM domain name server and obtains the secondary domain name server address of baidu.com
-
Local DNS initiated a resolution request to the secondary domain name server of baidu.com, and finally delivered the ip address information of www.baidu.com
-
Local DNS caches the IP address information obtained by recursive query and returns it to the client
2. Global load balancing GSLB
GSLB is the abbreviation of Global Server load Blance, that is, global load balancing. The purpose is to realize the traffic allocation between servers in different regions on the Internet, so as to ensure that users' requests can be processed by servers closest to users or with better service quality. So as to ensure the quality of service.
Can judge the load of the server, including CPU Occupancy, bandwidth occupancy and other data determine the availability of the server, judge the link status between the user (Visitor) and the server, and select the server with the best link status. Therefore, GSLB makes a comprehensive judgment on the server and link to determine which server to provide services, so as to ensure the service quality of remote server groups.
Intelligent DNS
Intelligent DNS is an application of GSLB.
3. Problems in DNS resolution
Sometimes when we visit Baidu or send an http request in the application, if the DNS resolution is hijacked, we may eventually access the server we want to access.
3.1 operator hijacking
DNS hijacking is to hijack the DNS server, obtain the resolution record control of a domain name through some means, and then modify the resolution result of the domain name, resulting in the transfer of access to the domain name from the original IP address to the modified specified IP. The result is that the specific web address cannot be accessed or the access is a fake Web address, So as to achieve the purpose of stealing data or destroying the original normal service.
3.2 cache the resolution results when DNS resolves the domain name
We sometimes encounter this situation in development: you are a Unicom user. You enter baidu.com in the mobile browser. A local DNS server, like Baidu authoritative server, checks which server to visit. The authority returns the result to the local DNS server, and the local DNS server returns the result to the user.
If LocalDNS caches the results corresponding to baidu.com, it will not query its corresponding ip like Baidu's authoritative server, but directly return the results in the cache. If the ip corresponding to baidu.com in the authoritative server changes and the LocalDNS is not updated in time, users will not be able to access the server. For more information: Understand these 9 steps and the principle of DNS access will be clear
3.3 forwarding analysis
When you visit baidu.com with your mobile phone, you will check the DNS server of the current operator, and then the DNS service of the operator will check the baidu authoritative server, and finally return the correct ip in the authoritative server.
The above is a normal situation, but if the local DNS of the current operator (such as China Unicom) does not access the authoritative Baidu DNS server, but directly accesses the local DNS server of other operators (such as telecom), some small operators reduce costs by doing so. If the access speed of Telecom's LocalDNS to non home ip is limited, it will obviously affect your DNS resolution time. If some services in your application need operator information isp(eg: only dns, cdn and other services)
The following screenshot shows the network environment of my mobile phone (NetPing open source address: github.com/mediaios/ne...)
I ping Baidu's address directly, and then use wireshark to capture the packet results. The normal results are as follows:
If forwarding occurs, the logic is as follows:
4. HTTPDNS
4.1 what is HTTP DNS
HTTP DNS usage HTTP And DNS Server interaction, instead of traditional based UDP of DNS Protocol, the domain name resolution request is directly sent to the HTTP DNS server, so as to bypass the operator's Local DNS
4.2 features of HTTP DNS
4.2.1 preventing domain name hijacking
Since HttpDns is requested directly through IP HTTP To obtain the recorded address of server A, there is no process of asking the local operator for domain resolution, so the hijacking problem is fundamentally avoided.
4.2.2 precise dispatching
HTTP DNS can directly obtain the user's IP address, so as to achieve accurate positioning and navigation
4.2.3 decrease in user connection failure rate
The algorithm reduces the server sorting with high failure rate in the past, improves the server sorting through the recently accessed data, and improves the server sorting through the historical access success record.
The original request address was "apis.juhe.cn/simpleWeath...". After replacing the domain name through HTTP DNS, the final result is as follows:
4.3 HTTPS IP Content
To send an HTTPS request, you must first SSL/TLS Handshake. The handshake process is roughly as follows:
-
The client initiates a handshake request, carrying parameters such as random number and list of supported algorithms.
-
The server receives the request, selects the appropriate algorithm, and issues the public key certificate and random number.
-
The client verifies the server certificate and sends random number information, which is encrypted by public key.
-
The server obtains the random number information through the private key.
-
Both parties generate a session ticket according to the above interactive information and use it as the encryption key for subsequent data transmission of the connection.
In the above process, step 3 is related to HTTP DNS. The client needs to verify the certificate issued by the server. The verification process has the following two points:
-
The client unlocks the certificate chain with the locally saved root certificate and confirms that the certificate issued by the server is issued by a trusted organization.
-
The client needs to check the domain and extended domain of the certificate to see if it contains the host of this request.
If the above two points pass the verification, it proves that the current server is trusted. Otherwise, it is untrusted and the current connection should be interrupted.
When the client uses HTTP DNS to resolve the domain name, the host in the request URL will be replaced with the IP resolved by HTTP DNS. Therefore, in step 2 of certificate verification, there will be a domain mismatch, resulting in SSL/TLS The handshake was unsuccessful.
5. Problems
5.1 how does the host know the IP address of the DNS server?
adopt DHCP Dynamically obtained or manually configured.
DHCP Protocol: DHCP (Dynamic Host Configuration Protocol) is usually used in large local area network environment. Its main function is centralized management and distribution IP Address, so that the host in the network environment can obtain the address dynamically IP Address, Gateway address DNS Server address and other information, and can improve the utilization rate of the address.
5.2 why does DNS adopt UDP protocol?
TCP The communication process is too complex and expensive, one time TCP The exchange requires nine packets: three connection packets, four disconnection packets, one request packet and one response packet.
UDP The communication process is simple, only one query packet and one response packet are needed.
Source: Zhan Xiaolang's blog