Subject: Re: I-D ACTION:draft-lindberg-dnsop-isp-root-server-00.txt

From: "Matt Crawford" <[email-masked]>
Date: Wed, 18 Aug 1999 11:46:23 -0500

> This does increase complexity, but
> the increased complexity is kept entirely within the DNS itself.

This is a *feature* ???

---

From: Tom Limoncelli <[email-masked]>
Date: Wed, 18 Aug 1999 15:37:28 -0400

> the increased complexity is kept entirely within the DNS itself.

This is an empty statement.  Where else could it add complexity?  At my
hair salon?

--tal

P.S.  For those that just tuned in: complexity bad, simplicity good.

---

From: Gunnar Lindberg <[email-masked]>
Date: Thu, 19 Aug 1999 09:06:49 +0200 (MET DST)

Hm. Did any of you expect Free Lunch? Of course not!

Increasing the number of Root Servers from "13" to "enough" will not
come free, like it or not. Complexity will increase and question is
"how much" and "where". The latter is important since it tells whether
complexity in controlled by DNS people or it shows up elsewhere, far
from DNS control.

Saying that clearly is not good/bad, it simply means being honest.

If you look at most of the other proposals they say "anycast" and hope
it's a done deal. I'm concerned:

    1)	It expects a routing behavior that is not well documented.

	What happens if someone (errounously) injects a Root Server's
	IP prefix originating from the wrong AS ("inconsistant AS
	path")? Today, i.e. cisco implementation, it still works,
	although there is a warning message on the console. What will
	happen tomorrow? Maybe that prefix gets dropped? To the best
	of my abilities I've not found that documented.

	If we're going to base the DNS, the very heart of the Internet,
	on "anycast", that default behavior at least needs to be
	documented in an RFC (carved in stone, rather). Btw, the BGP4
	draft has just expired.

    2)	How do you trace erronous information?

	Assume your DNS has cached bad root data. You know NS(.) by
	name and address, but none of those instances you reach when
	you go looking for it contains the error. Does this mean the
	the error is corrected? Or, is it still there, in some of the
	other "[a-m].root-servers.net" instances?

	How do you know which "[a-m].root-servers.net" there are?
	
My proposal is:

    1)	Don't. Use today's unicast routing as is. Simplicity. Good.

    2)	Let ISPs run RSs and let their customers be aware of reality.
	Tell customers that NS(.) =
	    rs1.their.provider [1.2.3.45]
	    rs2.their.provider [1.2.4.56]
	    rs3.their.provider [1.2.5.67]

	Less confusion when bad data shows up. Simplicity. Good.

To summarize: I'm concerned by the web of implicit dependencies that
the "anycast" stuff is likely to create. I think "the hair salon" is
still fairly safe :-) but other things/places are not.

		Gunnar Linbdberg

-----------------------------------------------------------------------
>From [email-masked]  Wed Aug 18 18:51:25 1999
>Message-Id: <[email-masked]>
>To: [email-masked]
>From: "Matt Crawford" <[email-masked]>

>This is a *feature* ???

-----------------------------------------------------------------------
>From [email-masked]  Wed Aug 18 21:39:54 1999
>Message-ID: <[email-masked]>
>Date: Wed, 18 Aug 1999 15:37:28 -0400
>From: Tom Limoncelli <[email-masked]>
>To: [email-masked]

---

From: [email-masked]
Date: Thu, 19 Aug 1999 15:48:31 -0700 (PDT)

Hi Gunnar,

> complexity in controlled by DNS people or it shows up elsewhere, far

I find your comment about controlling complexity interesting,
particularly in that it indicates a split between "DNS people" and
some other group (presumably "network people" or "routing people").  I
think many folks have seen this problem in terms of network
topology-related performance issues and so have focused the solutions
in those arenas.  Latency from the current root servers to some places
in the network is certainly a problem; redundant paths to some set of
root servers can also also a problem.

> 	documented in an RFC (carved in stone, rather). Btw, the BGP4

In the draft I put forward (and will soon revise), the idea is that
some organizations running root servers would make multiple boxes
available in multiple locations.  One of the reasons I prefer to have
a single organization responsible for the AS announcing the route to a
specific root server is that one group remains on the hook for
noticing/responding to/fixing problems like this.  

In that context, the problem you describe is not a lot different from
a bad route being injected now.  The organization serving the route
has to notice it, respond, and fix the problem.  Note that the
injected route being fixed affects how traffic gets to the
organization, where the "shared unicast" portion of the delivery takes
place internal to the organization.

In a context where multiple organizations manage a special AS, fixing
problems like that might have other characteristics.  Masataka may
want to respond to that idea.

> 	other "[a-m].root-servers.net" instances?

One of the reasons I focused on synchronization of the various
instances of a specific root server is this problem.  If you got bad
data from a root server all the copies of it would have bad data; if
you got good data, all of the copies would have good data.  You also
have one organization to contact to correct the problem.

> 	    rs3.their.provider [1.2.5.67]

The problem here is that they are not really roots.  They derive their
data from what you call "Real Root Servers" and they act as a new
level in the hierarchy which is not reflected in the notation.  If I
understand your proposal correctly, rs1.their.provider would have to
respond to an SOA request by claiming to be authoritative for . to
avoid having internal servers just query up the chain to the Real Root
Servers. That seems to imply that they would have to re-write the data
they get from the Real Root Servers to claim that authority.  That
pretty much makes them an active man in the middle attack and open to
all sorts of problems, including a pretty easy form of splintering.

If I misunderstood your proposal, please help me see how you would
avoid this.
			regards,
				Ted Hardie

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Sun, 22 Aug 1999 03:09:57 +0200

>all sorts of problems, including a pretty easy form of splintering.

I don't get this - is any software anywhere actually routing queries
to the nameserver named in the SOA record?

I know for a fact that some zones have a nameserver in their SOA that
isn't in their NS list (the case where I know why, it's because the
Real Master is behind a dialup link, but still with a fixed IP addr)

If not, I'd say that this distinction is part of the solution, not
part of the problem; where the info comes from is SOA, where you get it
from is NS.

The problem of who signs the NS set is a nasty one anyway.

                          Harald

--
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

---

From: Gunnar Lindberg <[email-masked]>
Date: Mon, 23 Aug 1999 15:40:23 +0200 (MET DST)

Ted,

Let me make a half-technical/non-technical remark first and then do
some more comments in-line:

I agree that one single Root Server organization would be a good
thing. However, I see no reasonable way to get that in place, to get
funding for it (from users or from ISPs), etc, etc, and to make it
scale to "enough" number of Root Servers.

As I've written in my draft I think the need is one or more Root
Servers per ISP - maybe not all for pure technical reasons, but no
ISP in their right mind would ever admit (or allow) that their users
totally depend on Root Servers located at, and controlled by, their
main competitor. I know there are several ISP people on this list -
please correct me if I'm wrong...

Therefore, we're into several organizations operating instances of
Root Server(s) (instances with - hopefully - identical content).

>From [email-masked]  Fri Aug 20 00:50:26 1999
>Message-Id: <[email-masked]>
>Subject: Re: I-D ACTION:draft-lindberg-dnsop-isp-root-server-00.txt
>To: [email-masked] (Gunnar Lindberg)
>Date: Thu, 19 Aug 1999 15:48:31 -0700 (PDT)
>Cc: [email-masked]
>From: [email-masked]

>root servers can also also a problem.

If I was to provide a vital service for a large community I would be
careful to base it on things closely under my control and for every-
thing else have that be as simple as possible. That way it's less
likely that things fail, or change under my feet. Then, at times it
pays off to violate this "layered approach"; fine, but then one should
document what behavior is expected from other layers and not try to
hide that under the carpet.

My split between "DNS people" and "routing people" comes from this
observation and a concern that the <idr> WG people might not always
know what undocumented behavior other people expect and depend on
(e.g. at unintended events like "inconsistent AS path"). I put my
draft together in an effort to minimize that dependency and a hope
that existing dependencies/expectations be documented.

I know "anycast" has been discussed for use with Multicast RPs and
I am attracted by the idea, but I also think most of us agree it's
some difference between Multicast RPs and the DNS Root Servers...

>have one organization to contact to correct the problem.

As I said above, I agree that one single org would be good. But, for
the resons I presented I think we have the multi-org situation and
that we must plan for it.

As for synchroniztion it is of course the intent that all instances
are identical at all times. However, in a real world with real people
sh*t happens and my concern is how to act when sh*t has happened;
e.g. someone finds he has bad cache data about equinix.com and now
you're trying to verify what went wrong and get the error corrected.
In a situation where there is an unknown number of [198.41.0.4] spread
out over the entire Internet, I expect that to be a hard task.

>all sorts of problems, including a pretty easy form of splintering.

I think there are 3 different issues:

    1)	SOA content.
	As <[email-masked]> already responded, the fields that
	you might want to change are not interpreted by computers,
	so 'MNAME' & 'RNAME' (RFC1035, pp 19) can either be left as
	is or be changed. I don't have a strong opinion.

    2)	List of NS(.).
	This of course has to be modified to point at the provider's
	IRSs; to yield rs1.their.provider, rs2.their.provider, etc.

    3)	DNSSEC signing.
	As Harald also said in his response, this is hard and nasty.
	I don't have an answer right now.

Hm, I didn't make your "1.2 Zone file delivery" point; will add that.

	    Gunnar Lindberg

---

From: Masataka Ohta <[email-masked]>
Date: Tue, 24 Aug 99 0:37:50 JST

Gunner;

> thing.

The reality is that USG, or Ira, used private secter falacy of
creating ICANN to make it impossible.

> However, I see no reasonable way to get that in place, to get

So, let ISPs, which necessarily is controlled by surrounding countries,
choose the root servers.

							Masataka Ohta

---

From: [email-masked]
Date: Mon, 23 Aug 1999 11:48:04 -0700 (PDT)

<Snipped discussion of SOA info being changed under Gunnar's scheme>

> Real Master is behind a dialup link, but still with a fixed IP addr)

I hadn't thought of this case.  I do know of situations where folks
run script checks on caching nameservers that match up SOA reported by
the roots and NS lists in the cache to see if someone has tried to hijack
domains.  I believe that it would report a false positive in your case
(and it doesn't catch everything in any case--it's just a flagging
mechanism).

Do you know of any cases now where there is no overlap between SOA
and NS?

> part of the problem; where the info comes from is SOA, where you get it

Agreed.

> The problem of who signs the NS set is a nasty one anyway.

And the need to have it signed is even more clear under this scheme.
			
				Ted Hardie

---

From: [email-masked]
Date: Mon, 23 Aug 1999 12:02:54 -0700 (PDT)

Hi Gunnar,
	I think the term "single organization" has tripped me up here;
sorry for any lack of clarity.  I am not proposing that there be one
single Root Server organization.  I actually believe that the
individual roots being run by different organizations helps preserve
the overall system, because it means that different eyes, box types,
and architectures are being used to deliver the data.  One Root Server
organization could mean a single point of failure; it doesn't have to,
of course, but it might.  What I have proposed is allowing
organizations running root servers to do so in a way that distributes
the service to multiple places in the network topology.  A nutshell
might be: one organization runs one root in many places.

There are some further comments in line below.

> please correct me if I'm wrong...

Given the current architecture, they won't ever totally depend on their
competitor; at most, they lowest-latency response could be retrieved
from a server at a competitor.  I would also like to hear how much
that would influence ISPs choices about what to implement in this
space.

> some difference between Multicast RPs and the DNS Root Servers...

I absolutely agree that good documentation and careful examination of
the dependencies is required here.  I also agree that deploying any of
these schemes to the roots would depend on successful experiments
being conducted and reviewed extensively prior to full deployment.

> out over the entire Internet, I expect that to be a hard task.

It's an easier task if you know you call XXNET no matter which 198.41.0.4
it is, and they know they have to be able to check them all at any time.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 23 Aug 1999 23:14:36 +0200

>Do you know of any cases now where there is no overlap between SOA

li.org, slashdot.org, microsoft.com (!), for instance.

more bizarre things happen - altavista.com uses a name in the SOA that
has the same A as one of its NSes, but different name.
microsoft looks as if it's round-robining its NS set, and my copy just
happened not to include ns 1.

the world is a strange place.

                            Harald A

---

From: Gunnar Lindberg <[email-masked]>
Date: Wed, 8 Sep 1999 17:39:36 +0200 (MET DST)

Ted,

Sorry for the long delay, but I had hoped someone else would step in.

Anyway, it seems to me we agree on what can be agreed on, given rather
different proposals. Sometime in the future the WG will make some
decision, I guess.

Finally, a comment on "anycast" that I haven't seen yet (but which
may have been beaten to death before I joined the list). What about:

    "anycast"
    "TCP lookup" (lookup - not AXFR)
    "equal-cost multi-path, per-packet, load sharing"

E.g: at some site they have two equal-cost routes to [198.41.0.4]. We
know it's anycast - routes to *different instances* of [198.41.0.4]
and not two routes to the same host - but the routers don't. Instead
the routers send 50% of the packets one direction and 50% the other.

No big deal for UDP queries that are self contained, but TCP has a
connection and keeps state and 50% TCP packets to each [198.41.0.4]
instance is not likely to work. And, at least for UNIX the choice of
UDP or TCP is set per application, "_res.options |= RES_USEVC", which
is hard enough to handle - what BillGatesWare does is unknown to me.

Am I the only person worried?

	Gunnar

>From [email-masked]  Mon Aug 23 21:03:19 1999
>Message-Id: <[email-masked]>
>Date: Mon, 23 Aug 1999 12:02:54 -0700 (PDT)
>Cc: [email-masked], [email-masked]
>In-Reply-To: <[email-masked]> from "Gunnar Lindberg" at Aug 23, 1999 03:40:23 PM
>From: [email-masked]

---

From: Jerry Scharf <[email-masked]>
Date: Wed, 8 Sep 1999 10:15:31 -0700 (PDT)

> Am I the only person worried?

There are ways of doing load sharing that do not screw up TCP (Cisco CEF
being one example.) If someone is using routing tricks, we have to require
them to not be stupid about how to do that. I think you could add language
that points out to the person implementing anycast routing that this kind
of situation must be avoided.

I don't see it as a specific reason to toss the concept out of hand. It is
a risk and needs to be evaluated against benefits.

jerry

---

From: Gunnar Lindberg <[email-masked]>
Date: Thu, 9 Sep 1999 18:07:46 +0200 (MET DST)

>From [email-masked]  Wed Sep  8 19:16:16 1999
>Date: Wed, 8 Sep 1999 10:15:31 -0700 (PDT)
>From: Jerry Scharf <[email-masked]>
>To: Gunnar Lindberg <[email-masked]>
>cc: [email-masked]
>Message-ID: <[email-masked]>

>of situation must be avoided.

The sad news is that it's not the person implementing anycast routing
who gets hit, but the persons that already have unicast routing up
running (most of us :-), and happened to end up with multiple paths.
I'm not sure it's convinient that the IETF comes up with a "use Cisco
CEF" recommendation to them.

>a risk and needs to be evaluated against benefits.

Agree 100% on the latter.

	Gunnar

---

From: [email-masked]
Date: Thu, 9 Sep 1999 16:15:53 -0700 (PDT)

Gunnar,
	Some notes in-line; sorry for my own delay in
replying.
			Ted

> >> Am I the only person worried?

As you note, the situation with equal-cost routes to two different
instances doesn't effect UDP, which is a big chunk of the traffic to
the roots.  It is a concern for TCP lookups, but only when you
have both equal cost routes and per-packet load sharing.  Note that
for ciscos, fast switching does per-destination load balancing rather
than per packet load balancing; fast switching is also on by default
and preferred for line speeds over 56 Kbps.  There are some cisco
specific docs at:

http://cco.cisco.com/univercd/cc/td/doc/cisintwk/idg4/nd2003.htm#xtocid133834

that describe their behavior.

Of course, ciscos aren't the only boxes in the world, and people do
turn fast switching off.  Two things additional factors mitigate the
severity of the problem.  The first is that the aim of this proposal
is to diversify the topological placement of the roots; for most users
that means that any "new" squiggle.root-servers.net will be a
significantly different cost metric from the "old"
squiggle.root-server.net, because it will be topologically distant
from the original.  Someone may end up in the middle, but that should
be rare if the primary aim is being followed.  Even in that case, any
implementation which measures the reponse performance of the roots to
select a preferred server will quickly drop that server; performance
might subsequently degrade for its users, but the system should still
work.  So even for users who do happen to be
equal-cost-routing/per-packet-load-sharing/tcp-using kind of folks,
the DNS will still work.

> Agree 100% on the latter.

As you both note, it's a risk vs. benefit evaluation.  For something
which is largely UDP, a correct implementation of this scheme should
have a pretty good rewards to risks ratio.  I also don't believe that
there is any case in which implementing this will cause the *overall*
system to fail; at worst, it seems that some number of users might
have degraded performance in the face of certain routing conditions.
If the placement was right and traffic patterns remain as they are,
that number should be pretty small.*
			regards,
				Ted

*Your mileage may vary.  Offer void in Missouri, Quebec, and wherever
prohibited by law.  Standard disclaimers apply.  Asses should be
covered before buckshot fired.