Subject: [dnsext] Chairs statement on "client-option" debate

From: Olafur Gudmundsson <[email-masked]>
Date: Tue, 01 Jun 2010 14:20:55 -0400

Dear colleagues,

The chairs have been following this discussion with great interest.
At this point we would like to make the following comments.

#0 Culture clash
The thread has demonstrated that "traditional" DNS people and 
"traditional" CDN people do not think the same way or have the same 
"values". We would like to ask all parties to respect each others'
point of view even when you do not agree with it. Please do not use
ancient history as a reason to push/reject proposals.

#1 Terminology disconnect:
When talking to a large group of people do not assume that generic
terms will be understood the same way. I.e. "Optimal" needs context
for productive discussion, CDN people think it related to TCP RTT,
while DNS people think it about making queries resolve faster.

#2 Problem statement:
We think we are close to understanding what the problem statement is. 
But there is need for a concise one that does not
intermingle "misunderstanding" and problem. What we mean here is
that CDN people have made an assumption that resolvers are
"close" to "host  making query", DNS people have to large extent
never agreed with this assumption.
The problem statement needs to be explicit about what the issues are.

#3 Solution spaces:
What we have on the table is proposed solution that does not explain why 
DNS is the best solution space and why.
The question on the table is why can this not be done with HTTP command 
or the http page returned to connection be served up with domain names 
that reflect the location of the source address?

#4 Cost and implementation considerations:
No solution is "free" but different solutions allocate the cost 
differently, the solutions that have best chance of succeeding are the 
ones that allocate the cost to the parties that have most to gain.

#5 Respect:
It is not productive to have people, stick fingers in their ears and 
keep saying "lalalalalalalalalalalalalalalala".
If you have nothing new to contribute please resist posting.

#6 Adoption plans:
We are planning to ask the working group formally if the WG wants
to adopt this work item, but there are two other items in our queue that
we will ask about first.

	Olafur & Andrew

---

From: Colm MacCárthaigh <[email-masked]>
Date: Wed, 2 Jun 2010 09:56:37 -0700

On Tue, Jun 1, 2010 at 11:20 AM, Olafur Gudmundsson <[email-masked]> wrote:
> #2 Problem statement:
> We think we are close to understanding what the problem statement is. But
> there is need for a concise one that does not

I've been trying to come up with a problem statement candidate, but am
having some difficulty with the advice here.

Firstly, I don't agree that CDN people have made such an assumption -
in my experience of dealing with several CDNs and their operators,
they have directly measured the correlation between resolvers and
client hosts and have some confidence in it. Naturally each CDN keeps
that data to itself, which doesn't help convince anyone else. But
experiments are runnable to gather this data, for example - it is
relatively straightforward to figure out what resolver a client is
using;

   http://whatsmyresolver.stdlib.net/

wrapping that up as a simple image render and purchasing a few million
advertising impressions (for a charitable cause) spread across the
internet would be one way to gather the specific data. If this kind of
data is useful, and would help convince anyone - that's one option. Of
course it's time, and money.

However, I'm confused as to why the assumption matters. Even if the
above were not the case, and CDN operators were making assumptions,
and those assumptions are false - why does that matter for this
option? This option is about forwarding client subnet information, the
issue of correlation should be moot. Hence, I don't think it's
possible to frame a correct problem statement that references an issue
that doesn't really matter for the problem at hand. My version of the
problem statement is;

  "Some CDN and Resolver operators would like to cooperate to provide
client network information to authoritative servers in
   an open and inter-operable manner that is also respectful of
end-user privacy".

I don't think this group will ever agree on the justification or
non-justification of DNS tricks, with any amount of data - there are
obviously more fundamental differences of opinion. I don't think
adoption of the option would encourage, or discourage the use of DNS
tricks where they are not already - it's simply about making one
use-case better for end-users. I think it's possible to be tolerant of
this difference of opinion within the group, and still engineer. Just
as many disagree on the DNSCurve/DNSSEC boundaries, that shouldn't
prevent technical progress on both.

-- 
Colm

---

From: Paul Vixie <[email-masked]>
Date: Wed, 02 Jun 2010 19:28:48 +0000

> Date: Wed, 2 Jun 2010 09:56:37 -0700
> From: Colm MacCárthaigh <[email-masked]>
> 
> ... in my experience of dealing with several CDNs and their operators,
> client hosts and have some confidence in it. ...

confidence expressed by those without a possible business motivation in the
outcome would convince me moreso.

---

From: Colm MacCárthaigh <[email-masked]>
Date: Wed, 2 Jun 2010 12:59:49 -0700

> outcome would convince me moreso.

Me too, I would love to see an independent party run - and if not run;
audit - a series of experiments designed to settle these issues. I
think it would make an important research topic too, worthy of CAIDA
or OARC participation - but to the best of my knowledge, noone has
performed it.

If any readers know of how to go about making such a proposal to
bodies such as CAIDA, OARC or know of any other research groups who
might be interested, speak up.

-- 
Colm

---

From: Nicholas Weaver <[email-masked]>
Date: Wed, 2 Jun 2010 13:10:54 -0700

> outcome would convince me moreso.

Paul, there are plenty of measurements you can conduct yourself to show that they do get this right, or at least right enough of the time:

The impact of these tricks can easily be gauged by comparing the ping time to the server returned as the A record for a CDN'ed domain when queried from your default resolver and/or your own IP, compared with the same name looked up through a random open DNS resolver in a different geographical location (there are plenty of those too, just scan for them!).

You can easily see 20-100ms of difference depending on conditions and your location.  I've posted citations and self experiments.

If you want to create your OWN database, its also easy to do (as someone else mentioned in this thread): just pay for a flashy web ad that does a unique DNS lookup along with reporting to the server what IP its behind.  Blend with a GeoIP database and Whois information and voila.

Probably not as good as Akamai's database, but certainly good enough for the "direct the user to the right subcontinent" which alone is responsible for the biggest portion of the performance gains.

And I certainly have NO business motivation in CDN behavior (yet...  I have thought a FreeCDN might be a good deployment model for PlanetLab on Steroids...)

---

From: Paul Vixie <[email-masked]>
Date: Wed, 02 Jun 2010 20:15:28 +0000

> Date: Wed, 2 Jun 2010 12:59:49 -0700
> > confidence expressed by those without a possible business motivation in
> > the outcome would convince me moreso.
> 
> audit - a series of experiments designed to settle these issues. ...

note that while i can imagine becoming convinced that changes to dns would
support the web industry better, i don't think i could agree that the web's
desires warrant the complexity now being proposed. the web is smaller and
younger than the internet, and is only one of many internet applications
the internet will likely outlast "the web" by many centuries. just sayin'.

---

From: Nicholas Weaver <[email-masked]>
Date: Wed, 2 Jun 2010 13:42:57 -0700

> might be interested, speak up.

We could do it in Netalyzr if we were interested enough in the results, its not hard:

Compare the TCP connect time for connecting to a major CDN'ed site, with the IP looked up through the local resolver, the NAT's resolver (if present), looked up through a third party resolver service, and looked up through a resolver we host (so from a known constant geographic location).

We already get client IP and external resolver IP, so if we wanted to blend together a CDN-type mapping we could.

The problem is, IMO, it would be VERY low down on our priority list:  When we or others do such tests manually, the results are pretty plainly obvious: the DNS tricks work, and are deployed because they already work in most cases.  Additionally, it generally doesn't affect the user unless they are foolish enough to use a third party resolver (which we will often warn about in the name lookup latency test anyway).

And given limited coding effort, there are much more important tests (such as potential IPv6 problems, and checking for DNS support for DNSSEC RRs, AAAA RRs, unknown RRs, unknown EDNS0 commands, etc, both on queries to the NAT's DNS proxy and directly to our server, potential issue with various gateways/NATs which may have vulnerabilities, etc) which would be worth conducting first, as these detect user-affecting problems.

---

From: Olafur Gudmundsson <[email-masked]>
Date: Thu, 03 Jun 2010 15:24:16 -0400

On 02/06/2010 12:56 PM, Colm MacCárthaigh wrote:
> On Tue, Jun 1, 2010 at 11:20 AM, Olafur Gudmundsson<[email-masked]>  wrote:
>> #2 Problem statement:

These are two different things, yes we know that large majority of
clients has used resolvers that are close to them, but that does
not mean it has to be that way.

The number of resolvers that were "distant" from the clients was a small
group, but it is a growing group ===> thus the storm in a teapot that
DNS needs to be fixed.

As David Ulevitch said last night, OpenDNS has millions of clients ==>
this is big enough population that CDN vendors needed to suddenly
pay attention. (combine that with UltraDNS and Google offering Recursive 
resolvers for anyone).

> relatively straightforward to figure out what resolver a client is

I'm not sure any experiment will resolve the issue as this is a moving
target.
I forget who said on this list few days ago something like:
"We have been pushing ISP's to use many smaller recursive resolvers"
This to me indicates that CDN providers have been aware of the issue
and trying to minimize it, but the rise in usage of fourth party
recursive resolvers (not CDN, not user, not ISP) is not amenable to the 
same "fixes".

> option?

The point that Andrew and I were trying to get to is:
	what is the basic core problem
	and what are the consequences of the problem
NOT 	what the proposed solution is and its implications.

Once we have a rough consensus what the problem/issue is, it is
much easier to talk about solution spaces.
The push back from the DNS crowd can be paraphrased as:
"This is the way DNS is supposed to work, making any assumptions
about IP address of 'requestor' by looking at the address of the
Resolver will be wrong some of the time."

> This option is about forwarding client subnet information, the

This is partial statement it does not give any background. i.e. you are
justifying the proposed solution w/o saying what the basic problem is.
Why do they want the information, i.e. what are the benefits.

> prevent technical progress on both.

I agree there will be "religious" disagreements between the different
interpretations of what DNS is and what it is supposed to be. We have
different branches of the DNS-religion and each one thinks their 
interpretation is the correct one.

For that reason we go by rough-consensus, but to get to that
point we need to know what the problem is.

As for DNScurve and DNSSEC I expect progress to be made on that
front.  DNScurve is IMHO (not as chair) an attractive
alternate to TSIG/TKEY and has potential to protect the last
mile as well as referrals.

	Olafur
PS: I

---

From: John Payne <[email-masked]>
Date: Wed, 16 Jun 2010 16:21:03 -0400

> the internet will likely outlast "the web" by many centuries. just sayin'.

CDN technology is not limited to "the web".  That's where it started, thats where most CDN bits are today, but CDN technology can be applied to almost any IP application.