Problem
DNS resolution time can lead to a significant amount of user perceived
latency. The time that DNS resolution takes is highly variable.
Latency delays range from around 1ms (locally cached results) to
commonly reported times of several seconds.
Solution
DNS prefetching is an attempt to resolve domain names before a user tries to follow a link. This is done using the computer's normal DNS resolution mechanism; no connection to Google is used. Once
a domain name has been resolved, if the user does navigate to that
domain, there will be no effective delay due to DNS resolution time.
The most obvious example where DNS prefetching can help is when a user
is looking at a page with many links to various domains, such as a
search results page. When we encounter hyperlinks in pages, we extract
the domain name from each one and resolving each domain to an IP
address. All this work is done in parallel with the user's reading of
the page, using minimal CPU and network resources. When a user clicks
on any of these pre-resolved names, they will on average save about 200
milliseconds in their navigation (assuming the user hadn't already
visited the domain recently). More importantly than the average
savings, users won't tend to experience the "worst case" delays for DNS
resolution, which are regularly over 1 second.
Architecture
Chromium's
implementation of DNS prefetching does not use the browser's network
stack at all. Instead, it relies on external threads to resolve the
names, thereby warming the DNS cache of the operating system
(completely ignoring any cache in the application network stack). The
advantage of this approach was that it was completely compatible with
all network stacks (it is external), and prevented accidental
regressions on the main network stack.
Since some DNS resolutions can take a long time, it is paramount that
such delays in one resolution should not cause delays in other
resolutions. Toward this end (on Windows, where there is no native
support for asynchronous DNS resolution), Chromium currently employs 8
completely asynchronous worker threads to do nothing but perform DNS
prefetch resolution. Each worker thread simply waits on a queue, gets
the next requested domain name, and then blocks on a synchronous
Windows resolution function. Eventually the operating system responds
with a DNS resolution, the thread then discards it (leaving the OS
cache warmed!), and waits for the next prefetch request. With 8
threads, it is rare than more than one or two threads will block
extensively, and most resolution proceed rather quickly (or as quickly
as DNS can service them!). On Debug builds, the
"about:histograms/DNS.PrefetchQueue" has current stats on the queueing
delay.
Manual Prefetch
Chromium
uses the "href" attribute of hyperlinks to find host names to prefetch.
However, some of those hyperlinks may be redirects, for example if the
site is trying to count how many times the link is clicked. In those
situations, the "true" targeted domain is not necessarily discernible
by examining the content of a web page, and so Chromium not able to
prefetch the final targeted domain.
To improve the speed of redirects, content authors can add the following tag to their page:
The
above "link rel" tag has no impact on the visual rendering of the page,
but causes Chromium to prefetch the DNS resolution of
"host_name_to_prefetch.com" as though there was an actual href targeted
at a path in that domain. The double slashes indicate that the URL
starts with a host name (as specified in RFC 1808). It is equivalent (but unnecessary) to use a full URL such as "http://host_name_to_prefetch.com/".
DNS Prefetch Control
By default, Chromium does not prefetch host names in hyperlinks that
appear in HTTPS pages. This restriction helps prevent an eavesdropper
from inferring the host names of hyperlinks that appear in HTTPS pages
based on DNS prefetch traffic. The one exception is that Chromium may
periodically re-resolve the domain of the HTTPS page itself.
An inquisitive content author (for example, a commenter on a blog) may
abuse DNS prefetching to attempt to monitor viewing of content
containing links. For example, links with novel subdomains, when
resolved during a prefetch, may notify a domain's resolver that a link
was viewed, even if it was not clicked. In some such cases, the
authority serving the content (such as a blog owner, or webmail server)
may wish to preclude such abusive monitoring.
To allow webmasters to control whether DNS prefetch is enabled or
disabled, Chromium includes a DNS Prefetch Control mechanism. It can be
used to turn DNS prefetch on for HTTPS pages, or turn it off for HTTP
pages.
Chromium watches for an HTTP header of the form "X-DNS-Prefetch-Control"
(case insensitive) with a value of either "on" or "off." This setting changes the default behavior for the rendered content. A "meta
http-equiv" tag of the same name can be used to make policy changes within a
page. If a page explicitly opts out of DNS prefetch, further
attempts to opt in are ignored.
For example, the following page from https://content_author.com/ would cause Chromium to prefetch "b.com" but not "a.com", "c.com", or "d.com".
<a href="http://a.com"> A) Default HTTPS: No prefetching </a>
<meta http-equiv="x-dns-prefetch-control" content="on">
<a href="http://b.com"> B) Manual opt-in: Prefetch domain resolution. </a>
<meta http-equiv="x-dns-prefetch-control" content="off">
<a href="http://c.com"> C) Manual opt-out: Don't prefetch domain resolution </a>
<meta http-equiv="x-dns-prefetch-control" content="on">
<a href="http://d.com"> D) Already opted out: Don't prefetch domain resolution. </a>
Child frames also inherit the DNS prefetch control opt-out setting from their parent. The DNS prefetch control setting applies only to hyperlinks and not to the manual prefetch mechanism.
Browser Startup
Chromium automatically remembers the first 10 domains that were
resolved the last time the Chromium was started, and automatically
starts to resolve these names very early in the startup process. As
a result, the domains for a user's home page(s), along with any
embedded domains (or anything the user "always" visits just after
startup), are generally resolved before much of Chromium has ever
loaded. When Chromium finally starts to try to load and render those
pages, there is typically no DNS induced latency, and the application
effectively "starts up" (becoming usable) faster. Average startup
savings are 200ms or more, with common acceleration over 1 second.
Omnibox
Prefetching is also used in Chromium's omnibox, where URL and/or search
queries are entered. The omnibox automatically proposes an action,
either of the form of a search query, or a URL navigation, as the user
types in text. This proposed action is considered the autocompletion
of what the user is typing. Each time the omnibox makes a proposal
(suggests an autocompletion), the domain for the underlying URL is
automatically pre-resolved. This means that when a user is entering a
search query, while they type the query (typically when they enter a
space between words), Chromium will automatically prefetch the
resolution of the domain in their search provider's URL. Assuming they
haven't done a search in a while, this can save a user an average of
over 100ms in getting a search result, if not more. Similarly, if the
URL is a "common" URL that they've typed in the past (e.g., their
favorite web site; bank; email provider; their company; etc.), the
savings in resolution time can be quite significant.
Effectiveness
Typical Usage
If a user resolved a domain name to an IP address recently, their
operating system will remember (cache) the answer, and then resolution
time can be as low as 0-1ms (a thousandth of a second). If the
resolution is not locally cached and needs to "go out over the
network," then a bare minimum for resolution time is about 15 ms,
assuming a nearby firewall (home router?) has a cached answer to the
question. Most common names like google.com and yahoo.com are resolved
so often that most local ISP's name resolvers can answer in closer to
80-120ms. If the domain name in question is an uncommon name, then a
query may have to go through numerous resolvers up and down the
hierarchy, and the delay can average closer to 200-300ms. More
interestingly, for any of these queries that access the internet,
dropped packets, and overworked (under provisioned) name resolvers,
regularly increases the total resolution time to between 1 and 10
seconds.
Network Usage
DNS resolutions use a hierarchical system, where each level in the
hierarchy has a cache, to remember previous resolutions. As a result,
extra resolutions don't generally speaking cause end-to-end internet
usage. The resolutions only go as far as needed toward the
"authoritative" resolver, stopping when they reach a resolver that
already has the resolution in a cache. In addition, DNS resolution
requests are very light weight. Each request typically involves
sending a single UDP packet that is under 100 bytes out, and getting
back a response that is around 100 bytes. This minimal impact on
network usage is compensated by a significant improvement in user
experience.
Cache Eviction
The local machine's DNS cache is pretty limited. Current estimates for
the number of resolutions remembered on a Windows XP box are in the
range of 50-200 domain names. As a result, if "too many" resolutions
are made, then some "necessary" resolutions might be "evicted" from the
cache to make room for the new prefetches. Chromium tries to model the
underlying cache, and guess when there is a chance that a "soon to be
needed" domain resolution has been evicted. When Chromium decides an
eviction may have taken place, it can automatically resolve the domain
name again, ensuring it is either reloaded into the cache, or marked as
"recently used" so that it won't be evicted for a "while." Cache
evictions caused by Chromium can have a negative impact on other
applications that don't (yet) use prefetching techniques to keep the
underlying cache "warm."
Many large internet sites (google.com, yahoo.com, etc.) commonly mark
the domain names resolutions with an expiration time in the
neighborhood of 5 minutes. They probably set the expiration time to be
short so that they can better respond to changes in supply and demand
for their services. This in turn tends cause cache evictions. As a
result, most applications tend to face the problem of cache eviction,
and already employ various methods to reduce the impact.
The "about:histograms/DNS" and "about:dns" pages contain more information about DNS prefetch activity.
|
