18 Jul Why are we waiting .. for DNS
We host a considerable amount of domains, websites and look after DNS for our client base and as such we have to make changes at times to create links to new sites and services.
One of the questions that I get asked is why are these changes not instant?
I thought it would be useful to explain the basics of how these changes happen and how the Domain Name System or DNS works. This is not a definitive guide to DNS as it is a very complex subject and there are a lot of excellent books and resources that will explain in great detail the bits and bytes of what is an amazing system of data transfer and update of records that probably shouldn’t really work!
DNS – The Very Basics
For the non-technical, a few words on what DNS is there for:
If you consider that the internet works on numbers, all the computers, servers and machines connected to the internet have a unique number ( or IP Address ), let me be clear, when you are connected at your office or at your home, then you will have a local address, your router or modem which connects your internal network to the internet has one of these unique numbers as until recently they have been in short supply.
It is a fact that the vast majority of human beings are not good at remembering numbers. Imagine if you had to remember that the website for your favorite shop was at an address that looked like 22.214.171.124 (just made up!), then I suspect that you would fail to remember and you would need to write it down, this is why we have books of telephone numbers or we have to save them in our digital phone books, most people can’t remember the numbers but they can remember the names.
If you take this idea of a phone book, the DNS System works in a similar way by taking a name e.g. www.abcompany.com and turning this into a number 126.96.36.199 which is then used to make the connection, so you can, therefore, think of DNS in simple terms as a “phonebook” of websites and their unique IP Addresses.
Having got the basic concept of what DNS is. the obvious thing would be to think that there is one big “phone book” of names and addresses that everyone can access, but as there are now expected to be over 1 billion websites online in mid 2016 (source: NetCraft Web Server Survey – [and its data compiled by Pingdom]. Netcraft Ltd. ) and as such a single database of entries is not sustainable and of course with all your eggs in one basket like this the system would not be robust enough.
The way that the DNS system covers this is to use a distributed system of thousands of servers running DNS software and each providing details for a smaller number of domains.
The next stage is to understand how a computer makes a query, again this is at a very high level and not meant to be anything else!
DNS – A simple query
If we quickly go through a query for www.abcompany.co.uk and see how this makes a connection
- You type into your web browser www.abcompany.com
– The first thing that happens is that your computer looks at which DNS server is assigned, this is set to “obtain automatically” in the vast majority of cases so will use the server that your ISP provides, this is called the “recursive resolver” as it knows which other servers to contact in order to get the information required for this domain.
- This first server sees that you are looking for a .com site and it therefore knows that the first place to go for information on a .com is one of the “root servers” that are located across the world and can be considered the ultimate place to start looking. Again in very simple terms, all that is initially asked is about “.com” , the query is then referred to a Top Level Domain server (know as a TLD Server) which then looks up the domain name itself , in our case this is abcompany.com this TLD Server then returns the IP Address (number) of the Nameserver that is assigned to that domain name, for the sake of argument, lets say this is IP Address 188.8.131.52 (again this is not real!) and in fact if possible it will return both the older IPv4 address shown and a newer IPv6 Address which is being used to allow room for all the new systems that are connecting to the internet every day ( but that’s another post !)
- So we have now been directed to the Nameserver for the domain, and it is here that the final IP address of the website you are looking for will be provided, the query is passed through and the ip corresponding to the request is identified, in this case we have not got a prefix, in a domain records you can have just abcompany.com or you can have www.abcompany.com or xyz.abcompany.com and these are called “A” Records and each can have a different address.
You now have the final piece of the puzzle and your browser can now go to the ip address and show you the information you have requested.
This all happens in a few milliseconds ( ideally ) or can be a second or two which is why you can sometimes get a delay before a site is shown.
DNS – Making Changes
So you now know that there can be lots of systems around the world that need to be updated with any changes to your DNS settings, for example, you add a new “A” Record or make a change to a corresponding IP Address, all the relevant systems have to be updated with this new information, this is called “propagation” and this is the bit that takes time, but not generally in the way that you would think.
Each DNS record is actually a Text File containing the information on the domain that you need, and example of a DNS File is shown below:
; ; Database file abcompany.dns for abcompany.com zone. ; Zone version: 2017071801; @ IN SOA nameserver1.i7net.co.uk. root.i7net.co.uk. ( 2017071801 ; serial number 900 ; refresh 600 ; retry 86400 ; expire 3600 ) ; default TTL ; ; Zone NS records ; @ NS nameserver3.i7net.co.uk. nameserver3.i7net.co.uk. A 184.108.40.206 @ NS nameserver2.i7net.co.uk. nameserver2.i7net.co.uk. A 220.127.116.11 @ NS nameserver1.i7net.co.uk. nameserver1.i7net.co.uk. A 18.104.22.168 ; ; Zone records ; @ A 22.214.171.124 ftp A 126.96.36.199 www A 188.8.131.52
We can see that at the top of the file, there is a serial number and this is the basis on which a DNS record is updated.
When any change is made to the Zone Records then this number is incremented so in this case the next change will make the serial 2017071802 then 03 at the end and so on.
The Primary DNS server for the domain which is the one that you make changes on, will then notify any secondary servers that a change has been made to the record and they in turn will access this primary domain server and update their records accordingly. This usually happens within a few minutes, so why does it take so long to see the change everywhere?
Propagation and Caching – the long wait
The function of delivering a DNS Entry from a file is quite laborious so to speed up the process, the vast majority of ISP’s use a system of caching where they only read from the file every few hours or in some cases days and it is this one thing that delays the appearance of your change.
If you change your nameservers, for example, you move to a new providers, then the TLD servers need to be updated with this information and as these are all over the world and there are further lower levels of domain servers which also help out the main TLD servers, this can take time for all of them to be updated.
The default expectation is that all Domain servers should be updated within 72 hours of a change, in reality, we find that the majority of changes to a .uk ( .co.uk / .org.uk etc ) take a couple of hours for the majority of updates, but this is not guaranteed and when you consider the caching at the ISP then the updates can appear much longer.
Can I speed things up?
We often make changes and need to see a fast update to DNS to check that everything is working, there are several ways of achieving this.
If you set your local DNS Server setting to use the name server allocated to your domain then you should see any changes immediately, however you must set a secondary to one of the global DNS servers, a common one would be at Google (IP 184.108.40.206 or 220.127.116.11) , or if you are a bit old school, then you can update your hosts file on your local machine with the entry you want to check, but please do not attempt this if you don’t know the potential effect of any hosts file change !
I hope that this explanation assists with your understanding of why DNS can take time to change, but if you want any further information then feel free to get in touch.