Subject: [dnsext] root DNSKEY response is 1749 bytes

From: William Allen Simpson <[email-masked]>
Date: Sat, 22 Aug 2009 19:12:45 -0400

Paul Vixie wrote:
 >> From: David Conrad <[email-masked]>
 >> However let's posit an alternative reality in which people actually turn
 >> on validation.  A root DNSKEY response is 1749 bytes.  What does this
 >> imply?
 >
 > yipe!
 >
Earlier, we've learned that a NSEC3 NXDOMAIN won't fit within 512 bytes.

Now, we've learned that a basic root query won't fit within 1500 bytes,
let alone 1220 bytes.  Of course, during key rollover, that'll be bigger.

Unless somebody has a magic wand, a heck of a lot of queries are going to
fall over to TCP.

Could we please consider my earlier suggestion, breaking the problem into
two parts?

  1. Send a EDNS0 two part query for A and/or NS, with DO off, with the
Algorithm Understood (AU) option.  I've suggested that this be slightly
extended to return the signature algorithms list with DO off.

  2. If (and only if) a validating resolver, use TCP to query for the
DNSsec records.

  3. If some dastardly intermediate caching resolver doesn't return the
DNSsec answers because it's faking answers, send the TCP query to the
listed NS (cleverly saved above).

Do we really want to perform 2 or 3 more RTT of UDP queries before TCP,
just in case one of them works?

As Weaver suggests above, step 2 SHOULD be cached.  MY first DNS
implementation cached information about server RTT and such, there's no
reason that even the most mundane non-caching validating stub resolver
would have much trouble keeping that tiny bit of state in memory.

---

From: Paul Vixie <[email-masked]>
Date: Sun, 23 Aug 2009 05:53:32 +0000

> Date: Sat, 22 Aug 2009 19:12:45 -0400
> From: William Allen Simpson <[email-masked]>
> ...
> ...

if we're tearing the covers off again and declaring that dnssec still is
not deployable and needs another round of changes before it's safe to use,
then i have a small stack of late changes that i'd like to put in as well.

---

From: bert hubert <[email-masked]>
Date: Sun, 23 Aug 2009 11:47:54 +0200

> then i have a small stack of late changes that i'd like to put in as well.

Since I'm a DNSSEC implementor these days, I'd support this motion.

>From what I make of it right now, we have two futures.

1) we stick with what we have (large keys, 5-15% of resolvers unable
to receive fragmented packets), and prepare for large scale TCP use to
compensate. The losers in this scenario are those who have to support
the TCP traffic, and those that can't do outgoing TCP.

2) we implement workarounds, like 'paging', ecc cryptography and
algorithm selection. The losers in this case are the poor implementors
who have to implement all that. In addition, any workarounds will have
to look like very normal <512 byte queries/responses, so we aren't
caught by another round of middle boxes interfering. DJB pioneered
this concept in DNSCurve.

What people are still pondering is:

3) do nothing, and hope for the best.

I would personally not be happy to promote '3' as is - any
'PowerDNSSEC' efforts would still not end up being used in large
scale, since people will quickly turn off DNSSEC if it 'smells like
downtime'.

In addition, even if 3 is marginally workable, I'm pretty sure it
won't be over (emergency) key rollovers, since that makes packet sizes
balloon. '3' is basically '1' without preparing for increased TCP
usage.

Option 1 requires some guts, and for many servers, a lot of work to
bring TCP performance up to scratch. EDNS0@512 is taking us in this
direction already, if we want it or not.

Option 2 may not be so easy either - 'paging' will also cause packet
counts to go up, and require further maintenance of state. Because of
proxies, each response 'page' will require a new query, causing packet
counts to go up anyhow.

In addition, Richard Stevens had some wise words to say about when you
find yourself adding sequencing and retransmit logic to a UDP based
protocol 'you are rebuilding TCP, and you are not going to do a good
job'. I can't find the exact words, but a lot of wisdom on UDP versus
TCP can be found in section 20.4 of Unix Network Programming, second
edition.

      Bert

---

From: "George Barwood" <[email-masked]>
Date: Sun, 23 Aug 2009 12:27:48 +0100

My take:

(1) I don't think waiting for middleboxes to be upgraded is good strategy, 
so I think
we really do need some kind of standard solution for stub <-> cache via a 
proxy
that truncates responses at 512 bytes.

If that's SCTP/TCP via a DNS@512 tunnel, that may be good, but I haven't
seen a detailed concrete proposal. I think my page option is simpler,
and as this is a kludge, I think simpler is probably better.

(2) I do have serious security concerns regarding UDP fragmentation.

IP fragments are completely unprotected against spoofing - no port number, 
no DNS ID,
and a server IP ID that is highly predictable. Even the timing window is 
wide.

So fetching the root (or any) DNSKEY RRset is highly dangerous ( even for 
non-open caches)
- careful cache policy may help, but has BIND implemented this?

Using TCP for responses > ethernet MTU seems just about ok, but sub-optimal, 
with the usual
worries about the scalability of TCP.

Issue (1) need not delay the deployment of the root.
Issue (2) does I think need very careful consideration.
Has signing .gov actually left it wide open to attack?

"First of all, do no harm" should be the guiding principle here.

George

----- Original Message ----- 
From: "bert hubert" <[email-masked]>
To: "Paul Vixie" <[email-masked]>
Cc: <[email-masked]>
Sent: Sunday, August 23, 2009 10:47 AM
Subject: Re: [dnsext] root DNSKEY response is 1749 bytes

      Bert

---

From: Jim Reid <[email-masked]>
Date: Sun, 23 Aug 2009 12:50:59 +0100

On 23 Aug 2009, at 00:12, William Allen Simpson wrote:

> Unless somebody has a magic wand, a heck of a lot of queries are  
> going to fall over to TCP.

I'm troubled at the lack of meaningful data in this discussion. And  
maybe it's better to move a debate about operational considerations to  
dnsop?

What is meant by "a heck of a lot of queries"? And what is the  
expected operational impact of that increased amount of TCP queries to  
important DNS servers: ie the root and busy TLD name servers?

The statistics published at [hkm].root-servers.org indicate a root  
server gets somewhere between 10-16K qps in steady state. David Conrad  
said 70% of the queries to L have the DO bit set. Worst case, that  
suggests around 7-10K qps will go over TCP once the root is signed. It  
may be a lot, lot less than that assuming EDNS0 payloads are big  
enough and UDP fragmentation doesn't hurt.

Based on these crude numbers, I can think of the following questions:

[1] Can the existing servers cope with that level of TCP traffic?

[2] If the answer to [1] is "yes", then we're done. Though it would be  
good to know where the upper bounds on TCP traffic rates were and the  
constraints on them.

[3] If the answer to [2] is "no", then what needs to be done to fix  
that?
Is it a matter of throwing more hardware and/or anycasting at the  
problem?
Are there protocol or implementation tweaks which can be deployed? [eg  
Don't do EDNS with a 512 byte buffer limit when the DO bit is on.]

---

From: bert hubert <[email-masked]>
Date: Sun, 23 Aug 2009 14:47:16 +0200

> [1] Can the existing servers cope with that level of TCP traffic?

Unless we perform some standards action, and get rid of:

   - If the server needs to close a dormant connection to reclaim
     resources, it should wait until the connection has been idle
     for a period on the order of two minutes.

>From RFC 1035, section 4.2.2, it will definitely not be doable.

    Bert

---

From: Paul Hoffman <[email-masked]>
Date: Sun, 23 Aug 2009 08:34:31 -0700

> >From RFC 1035, section 4.2.2, it will definitely not be doable.

Your reading of the "needs" and "should" in that sentence doesn't feel right. From an operational standpoint, that sentence reads:
  If a server needs to do X, it should not panic and just blindly
  do X; it should first consider the very important operational
  consideration Y. If the server is doing Y but it still is not
  operationally stable, go ahead and do X.

We do not need to "perform some standards action" to allow server vendors to make their systems operationally sound.

--Paul Hoffman, Director
--VPN Consortium

---

From: Paul Vixie <[email-masked]>
Date: Sun, 23 Aug 2009 16:18:41 +0000

> > if we're tearing the covers off again and declaring that dnssec still
> > is not deployable and needs another round of changes before it's safe
> > to use, then i have a small stack of late changes that i'd like to put
> > in as well.

i guess i'd better be specific.  we need subclass names for metadata so
that (1) we have no special rules about DS sets, and (2) we can query for
individual RRSIG sets.  crypto that's based on djb's curve but uses dnssec
semantics (so, NS names aren't keys, and the security is end to end, but
the signatures aren't so damned big).  these by themselves would solve most
of the "messages are too large" problem for the current middlebox generation.

however, i threw the request out there mostly to scare folks.  we're close
to deployment of dnssec as it is.  if we back off for another run at it, we
will never be this close again.  so, irritating as i find the tradeoffs we
made to get this far, i am NOT advocating that we back off and try again.  i
will be one of the people looking for miracles when the TCP storm hits.

---

From: Donald Eastlake <[email-masked]>
Date: Sun, 23 Aug 2009 13:04:33 -0400

I have pretty much always thought that, due to the size
considerations, some form of elliptic curve cryptography was the right
thing for DNSSEC.

Donald
=============================
 Donald E. Eastlake 3rd   +1-508-634-2066 (home)
 155 Beaver Street
 Milford, MA 01757 USA
 [email-masked]

---

From: Paul Vixie <[email-masked]>
Date: Sun, 23 Aug 2009 17:22:44 +0000

> Date: Sun, 23 Aug 2009 13:04:33 -0400
> From: Donald Eastlake <[email-masked]>
> 
> I have pretty much always thought that, due to the size considerations,
> some form of elliptic curve cryptography was the right thing for DNSSEC.

have you evaluated djb's curve for suitability and correctness for DNSSEC?
you (donald) would be an ideal author for a draft in this area, in my view.

i think that since algo roll is already contemplated in the DNSSEC design,
and we're already looking into adding one for use in russia, that we ought
to be looking at new algo's if they're fast enough and secure enough but
also much smaller.

---

From: Paul Hoffman <[email-masked]>
Date: Sun, 23 Aug 2009 10:56:13 -0700

>considerations, some form of elliptic curve cryptography was the right

Indeed, that was part of the motivation for draft-hoffman-dnssec-ecdsa-00. The size and speed issues are discussed in the introduction of that document.

--Paul Hoffman, Director
--VPN Consortium

---

From: Andrew Sullivan <[email-masked]>
Date: Sun, 23 Aug 2009 16:07:38 -0400

On Sun, Aug 23, 2009 at 08:34:31AM -0700, Paul Hoffman wrote:
> 
> We do not need to "perform some standards action" to allow server
> vendors to make their systems operationally sound.

Without wearing any hat, I think that is a really sensible principle
to hang on to, in the DNS or anywhere else.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: bert hubert <[email-masked]>
Date: Sun, 23 Aug 2009 22:16:29 +0200

> will be one of the people looking for miracles when the TCP storm hits.

We might as well move to TCP then anyhow and be rid of the spoofing
risks today :-)

However, if you feel this way, perhaps we should add something to
dnssec-updates that deprecates RFC 1035, section 4.2.2 (the two minute
timeout).

    Bert

---

From: Paul Vixie <[email-masked]>
Date: Sun, 23 Aug 2009 20:25:27 +0000

> From: bert hubert <[email-masked]>
> Date: Sun, 23 Aug 2009 22:16:29 +0200
> ...
> ..., perhaps we should add something to dnssec-updates that deprecates
> RFC 1035, section 4.2.2 (the two minute timeout).

it's not dnssec-specific, so we need a broader document scope for it.  we
also have to cope with the implementations who correctly syslog today when
the responder closes a tcp connection.  this calls for a phased
implementation where in year 1-2 we adjust the initiators and in year 3-on
we adjust the responders.

---

From: William Allen Simpson <[email-masked]>
Date: Sun, 23 Aug 2009 18:27:33 -0400

> then i have a small stack of late changes that i'd like to put in as well.
(Aware of your later posts)  I wasn't suggesting a change to the DNSsec
standard itself, merely a strategy that could be implemented on a case by
case basis (in this case, a validating resolver), distributed as an
application without major structural changes.

If there are late changes, bring them up!  It seems there's always going to
be yet another round....

---

From: William Allen Simpson <[email-masked]>
Date: Sun, 23 Aug 2009 18:31:38 -0400

> we adjust the responders.
As Paul notes, the 2 minute TCP TIME-WAIT isn't DNSsec specific.  But there
are numerous old drafts and papers on how to change that, and nothing has
been done in 18 years....

I'll send Bert my latest draft on it.

---

From: Paul Vixie <[email-masked]>
Date: Mon, 24 Aug 2009 03:37:03 +0000

> Date: Sun, 23 Aug 2009 18:27:33 -0400
> (Aware of [vixie's] later posts) I wasn't suggesting a change to the
> DNSsec standard itself, merely a strategy that could be implemented on a
> case by case basis (in this case, a validating resolver), distributed as
> an application without major structural changes.

if it's not a requirement it will not happen at all since the pain will be
felt by responders and the change you're recommending would have to happen
at initiators.

if it's made a requirement it could happen a little bit slowly over years.

> If there are late changes, bring them up!  It seems there's always going
> to be yet another round....

i'd prefer to let sleeping ideas lay, until we have deployment experience
to feed back into the next round.  so, several years off, at a minimum.

---

From: Michael Graff <[email-masked]>
Date: Mon, 24 Aug 2009 16:08:14 -0500

> TCP can be found in section 20.4 of Unix Network Programming, second

This is exactly my argument against the page option.  I've fought
against adding TCP-like features to UDP protocols for years now, back
when IRC servers had connection problems.

Now they can accept thousands of TCP sessions per second and not flinch.
 So, I say again, why are people actually scared of TCP?  It's come a
very long way since 1980.

- --Michael

---

From: [email-masked]
Date: Mon, 24 Aug 2009 22:04:51 +0000

On Mon, Aug 24, 2009 at 04:08:14PM -0500, Michael Graff wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> bert hubert wrote:
> 

	sure you can -accept- thousands of TCP sessions per second.
	how many can you keep open?
	what is your criteria for dropping one?
	how many TCP sessions lose state partway through the handshake?

	remember that DNS sez you -MUST- keep a TCP session open for 
	120 seconds.

	my numbers indicate about 5% state loss for TCP (any TCP)

	-IF- one migrates the 25,000 qps over UDP on some servers, my rough
	calculations seem to indicate we wouldneed to keep 175,000 qps over TCP
	for the exact same query loading...  150,000 of those are half-open TCP
	sessions that got lost.

	again, how many can you keep open and what is the criteria for closing a TCP 
	session.

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 24 Aug 2009 15:30:37 -0700

> 	remember that DNS sez you -MUST- keep a TCP session open for

Why?  That number is a holdover from when DNS over TCP was used  
primarily for zone transfers.  IF it ends up being a common fallback  
mechanism for ~10% of the queries, thats a completely different story  
and a much more agressive timeout is far more appropriate.

In fact, if its being used primarily as a fallback for normal queries,  
a keepalive at all is probably inappropriate, and simply closing the  
connection after each query would probably serve things almost as well.

> 	again, how many can you keep open and what is the criteria for  

Whats wrong with "When state exceeds LRU, close", or heck, "If a  
'normal' query, close connection when complete".

(You can even close with a RST so you don't need to keep the half-open  
state)

Dealing with a 2,500 simple queries over TCP SHOULD NOT be a major  
issue.  Its stuff that the web server community and related solved a  
decade ago.

---

From: Doug Barton <[email-masked]>
Date: Mon, 24 Aug 2009 15:41:00 -0700

> 	how many TCP sessions lose state partway through the handshake?

Back in 1996 we had over 5,000 simultaneous clients on an IRC server
(which carries a LOT more traffic per client than DNS) attached to
DALnet. We had what was at the time "top of the line" hardware, but
apply Moore's law 4 times to the hardware and the multiplier of your
choice for the software and extrapolate from there.

Someone else already said it, TCP has come a long way in 30 years.

Doug

---

From: Matthew Dempsky <[email-masked]>
Date: Mon, 24 Aug 2009 15:53:37 -0700

> Someone else already said it, TCP has come a long way in 30 years.

Do we have any sense for how many concurrent TCP connections name
servers actually need to support in the worst case?  E.g., has anyone
performed analysis of the DITL 2009 data to see how many unique source
IP addresses are seen in rolling two minute windows?

---

From: [email-masked]
Date: Mon, 24 Aug 2009 22:52:39 +0000

> and a much more agressive timeout is far more appropriate.

	its in the spec Nicholas.  thats why.
 
> 'normal' query, close connection when complete".

	NP with close when complete...  the huge number is based on a 5% state loss
	of TCP transactions - in which case they will never close bythemselves.

> (You can even close with a RST so you don't need to keep the half-open  

	sure.  you can. whats your criteria for dropping a half-open 
	TCP connection for DNS service?

> issue.  Its stuff that the web server community and related solved a  

	its 25,000 qps over UDP.   move that to TCP, assume a 5% drop
	and then your pushing 175,000 qps over TCP for the same goodput.

	175,000 open TCP sessions per second is slightly tougher nut to 
	crack - and no, i don't think that extrapolating HTTP over TCP is
	quite the same as DNS over TCP.

--bill

---

From: [email-masked]
Date: Mon, 24 Aug 2009 23:03:03 +0000

> IP addresses are seen in rolling two minute windows?

why yes..

On Aug 12, 2009, at 2:51 PM, William Allen Simpson wrote:

>With the advent of more widespread DNSsec deployment, more UDP                                      
>sessions                                                                                            
>are likely to fallover into TCP sessions.                                                           
>                                                                                                    
>I've been informed that even today, with a more limited TCP activity,                               
>busy servers cannot wait 2MSL to finish closing.                                                    

One wonders how a web server survives...

>Also, busy caching servers run out of port numbers, and cycle quickly.                              
>So there's ample opportunity for seemingly duplicate transmissions.                                 

Presuming a single source and destination IP address and a single well-
known server port number, the caching server could run through ~60000/
MSL connections per second before attempting to reuse a four-tuple
still in TIME_WAIT.  If one were holding strictly to a four minute MSL
that is ~250 connections per second.  60 seconds seems to be a rather
more common MSL (well, length of TIME_WAIT) so that would be ~1000
connections per second.

--bill

---

From: Matthew Dempsky <[email-masked]>
Date: Mon, 24 Aug 2009 16:07:02 -0700

>> IP addresses are seen in rolling two minute windows?

... um, and the conclusion is... ?

---

From: [email-masked]
Date: Mon, 24 Aug 2009 23:15:19 +0000

> ... um, and the conclusion is... ?

	my sense (there is some debate on this)

	or sense of an analysis of the DITL data?  (no, have not done that,
		i'm chunking some root server logs for slightly other reasons
		at the moment - will get to this titbit in a couple weeks)

--bill

---

From: Michael Graff <[email-masked]>
Date: Mon, 24 Aug 2009 18:41:24 -0500

> 	TCP connection for DNS service?

2 second timeout.  It's what is in use on at least one DNS
implementation I believe.

> 	and then your pushing 175,000 qps over TCP for the same goodput.

Right now .org has what percentage of queries fall back to TCP?  It's
not like we need them all to suddenly go over TCP.  We just need large
responses to.  DNSKEYs with reasonable TTLs should not cause a flood;
it's not like a DNSKEY can't be cached.  Setting a 0 TTL on a DNSKEY
hurt ORG initially -- what is the percentage now that they have a IMHO
still too short 900 second TTL?  What would setting that TTL to 1 hour,
or 12 hours, do to the TCP stats?

> 	quite the same as DNS over TCP.

IRC servers have had long running TCP sessions open in the order of
30,000.  If you run a zone that gets that many queries on a single
machine, you might be in a world of hurt, but for TLDs and roots with
ample equipment and ample horsepower, would this really pose a problem?

- --Michael

---

From: Michael Graff <[email-masked]>
Date: Mon, 24 Aug 2009 18:50:52 -0500

> 	how many can you keep open?

IRC servers can maintain 10,000 to 20,000 TCP sessions open on modest
(by modern definition) hardware using modern OS features like /dev/poll,
epoll, and kqueue.  It does not seem difficult for these server
operators to handle the load.

This whole issue (TCP server capacity) has been studied and handled in
many other forums.  Indeed, one of the discussion threads was about what
OTHER protocols require (fragmented) UDP.  Indeed, few do.  Many use TCP
in fact and I have yet to hear them screaming about a global meltdown.

Perhaps I've gone through several of these myself; I used to run a
fairly large (8,000 or to) connection IRC server back in the day.
Perhaps that is why I'm not so scared.

- --Michael

---

From: Paul Vixie <[email-masked]>
Date: Tue, 25 Aug 2009 00:14:30 +0000

> Date: Mon, 24 Aug 2009 16:08:14 -0500
> From: Michael Graff <[email-masked]>
> 

they do flinch.  maybe try some dnsperf work in the nsf test lab and find out?

---

From: Michael Graff <[email-masked]>
Date: Mon, 24 Aug 2009 19:46:55 -0500

> they do flinch.  maybe try some dnsperf work in the nsf test lab and find out?

Apples to oranges.  DNS severs have, I suspect, never worried about TCP
path performance until now.  Thus, dnsperf against an unoptimized
implementaion is pointless.

- --Michael

---

From: Matthew Dempsky <[email-masked]>
Date: Mon, 24 Aug 2009 17:48:34 -0700

>        remember that DNS sez you -MUST- keep a TCP session open for

This seems like a bogus argument.  I took a bunch of popular name
servers, opened a TCP connection to port 53, and timed how long they
took before closing the socket themselves, and repeated the experiment
for each one 5 times for good measure.  They all closed the connection
after about 30 seconds of idle time.

M.ROOT-SERVERS.NET. 30.02 30.01 30.02 30.01 30.02
A.ROOT-SERVERS.NET. 30.04 30.03 30.02 30.03 30.02
C.ROOT-SERVERS.NET. 30.05 33.03 30.03 30.03 30.02
D.ROOT-SERVERS.NET. 30.04 33.02 30.03 30.03 30.03
G.ROOT-SERVERS.NET. 33.01 30.04 30.03 30.02 30.03
E.ROOT-SERVERS.NET. 30.04 33.02 30.03 30.95 30.04
F.ROOT-SERVERS.NET. 33.00 32.00 32.02 30.03 33.03
B.ROOT-SERVERS.NET. 32.03 32.01 30.03 30.03 30.99
I.ROOT-SERVERS.NET. 30.04 30.03 30.03 30.02 30.03
H.ROOT-SERVERS.NET. 31.99 30.03 31.01 30.92 30.03
J.ROOT-SERVERS.NET. 30.03 31.99 30.03 30.04 30.04
K.ROOT-SERVERS.NET. 30.03 30.99 31.99 33.02 30.94
L.ROOT-SERVERS.NET. 30.04 30.03 30.03 30.03 30.94
b.gtld-servers.net. 31.95 30.03 30.03 33.04 30.02
a.gtld-servers.net. 30.04 30.03 30.98 30.04 30.02
c.gtld-servers.net. 30.94 30.96 33.03 30.04 32.02
e.gtld-servers.net. 30.94 30.04 33.02 30.03 30.03
d.gtld-servers.net. 30.95 30.95 30.03 31.98 30.03
g.gtld-servers.net. 30.02 33.01 30.95 31.98 30.04
h.gtld-servers.net. 30.02 30.03 30.03 30.97 33.03
f.gtld-servers.net. 30.03 30.04 30.03 30.98 30.03
i.gtld-servers.net. 33.02 30.03 30.04 30.02 32.00
j.gtld-servers.net. 33.02 32.01 30.04 30.03 33.03
k.gtld-servers.net. 30.04 32.01 33.04 30.03 31.98
l.gtld-servers.net. 30.04 33.02 33.01 33.02 30.03
D0.ORG.AFILIAS-NST.org. 30.99 30.03 31.02 33.03 30.03
m.gtld-servers.net. 31.00 30.03 32.01 30.03 30.03
B2.ORG.AFILIAS-NST.org. 30.03 30.99 32.01 32.02 30.96
A0.ORG.AFILIAS-NST.INFO. 30.02 30.03 30.03 30.92 30.95
C0.ORG.AFILIAS-NST.INFO. 33.02 30.98 30.03 30.04 30.03
A2.ORG.AFILIAS-NST.INFO. 30.03 30.98 30.02 32.01 30.95
ns.isc.afilias-nst.info. 30.95 30.02 31.98 31.03 30.03
B0.ORG.AFILIAS-NST.org. 30.03 30.01 30.98 30.04 30.03
sfba.sns-pb.isc.org. 30.03 30.03 30.03 30.04 31.05
ord.sns-pb.isc.org. 30.03 33.02 30.02 30.03 30.03
ams.sns-pb.isc.org. 30.03 30.03 30.02 30.98 30.03
dot.ep.net. 31.02 30.02 30.95 30.02 30.03
ns.isi.edu. 30.05 33.02 30.03 30.03 30.03
ns0.ietf.org. 31.00 31.01 30.03 33.02 30.03
ns1.mia1.afilias-nst.info. 30.05 33.02 30.04 30.03 30.03
ns1.ams1.afilias-nst.info. 32.01 30.03 31.01 30.02 30.97
flag.ep.net. 30.05 30.03 31.02 30.95 30.03
ns1.yyz1.afilias-nst.info. 31.97 30.99 30.03 30.02 30.03
ns1.sea1.afilias-nst.info. 31.98 30.03 31.00 30.03 30.04
ns1.hkg1.afilias-nst.info. 30.97 30.98 30.03 30.03 30.95

(The list of hosts came from "dig +short -t ns . com org isc.org
karoshi.com ietf.org".)

---

From: Paul Vixie <[email-masked]>
Date: Tue, 25 Aug 2009 00:55:52 +0000

> Date: Mon, 24 Aug 2009 19:46:55 -0500

i reckon that best-case QPS for UDP/53 will always be 2..100X higher than
best-case QPS for TCP/53, for a typical offered (possibly simulated) load
and a given topology (any determinate mix of LAN and WAN) and a given
server, no matter what host OS or DNS application tuning is done.  assume
an authority server whose answer processing is always synchronous.

the footprint of TCP/53 (7 packets, 3 round trips) vs. the footprint of
UDP/53 (2 to 5 packets, one round trip) does not leave me a lot of room
for speculation.  i can probbaly be re-educated if somebody does some
science.

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 24 Aug 2009 18:07:21 -0700

> 	its in the spec Nicholas.  thats why.

So?  The DNS specs are hardly a sacred document, especially if  
effectively wronged and obsoleted.

> 	and then your pushing 175,000 qps over TCP for the same goodput.

Uhh, WTF?  25,000 qps over UDP, 10% of the resolvers can't hack UDP,  
gives you 2,500 TCP queries.

And how does a 5% drop-rate, even assuming 25k QPS, push you to 175K?   
5% drop does not mean you need to do 5X the # of queries...

> 	quite the same as DNS over TCP.

Yeah.  HTTP has keepalives that matter, and far more resources on the  
server side needed for processing.  And people already doing low  
traffic rate interactive DOSes (which don't work on ISS, BTW), syn  
floods, and everything else.  And it seems to work just fine.

HTTP is a much much HARDER target for high-throughput over TCP than DNS.

---

From: Matthew Dempsky <[email-masked]>
Date: Mon, 24 Aug 2009 18:27:11 -0700

> after about 30 seconds of idle time.

Embarrassingly, I realized I left f.root-servers.net hard-coded in my
scripts, so all of the tests ran against the same server... oops.

Below are the timings against the correct name servers.

A.ROOT-SERVERS.NET. 5.18 5.16 5.16 5.16 5.16
F.GTLD-SERVERS.NET. 5.44 4.38 5.52 5.22 5.66
A.GTLD-SERVERS.NET. 5.40 6.01 5.17 4.27 6.04
G.GTLD-SERVERS.NET. 5.92 5.56 5.82 5.06 5.57
I.GTLD-SERVERS.NET. 5.75 5.84 5.10 6.04 5.41
K.GTLD-SERVERS.NET. 4.93 5.97 5.39 5.49 6.37
M.GTLD-SERVERS.NET. 5.68 5.58 5.62 5.81 5.49
C.GTLD-SERVERS.NET. 5.96 5.40 6.14 5.45 5.45
J.GTLD-SERVERS.NET. 5.66 6.06 5.39 5.94 5.75
D.GTLD-SERVERS.NET. 5.68 5.97 5.43 5.86 5.92
H.GTLD-SERVERS.NET. 6.35 5.42 5.79 5.83 5.90
E.GTLD-SERVERS.NET. 6.21 5.81 5.45 6.16 6.35
B.GTLD-SERVERS.NET. 6.06 6.00 6.00 6.01 5.99
A0.ORG.AFILIAS-NST.INFO. 17.79 15.20 17.69 9.78 15.79
B0.ORG.AFILIAS-NST.org. 20.18 17.29 14.71 8.43 17.15
D0.ORG.AFILIAS-NST.org. 17.93 15.30 15.69 14.96 15.99
ns.isc.afilias-nst.info. 15.37 15.29 16.70 19.20 15.79
ns1.ams1.afilias-nst.info. 17.24 16.79 15.19 18.30 17.67
C0.ORG.AFILIAS-NST.INFO. 17.93 19.19 18.98 15.29 16.88
ns1.mia1.afilias-nst.info. 16.94 20.09 18.69 16.68 15.80
ns1.yyz1.afilias-nst.info. 18.63 19.17 17.39 14.59 19.19
ns1.sea1.afilias-nst.info. 18.73 19.20 15.76 16.18 19.19
F.ROOT-SERVERS.NET. 30.03 30.03 30.01 30.03 30.02
E.ROOT-SERVERS.NET. 30.03 30.02 30.02 30.03 30.01
sfba.sns-pb.isc.org. 30.01 30.03 30.01 30.01 30.01
C.ROOT-SERVERS.NET. 30.05 30.05 30.04 30.04 30.04
ns0.ietf.org. 30.02 30.01 30.02 30.01 30.01
dot.ep.net. 30.05 30.06 30.05 30.06 30.05
flag.ep.net. 30.05 30.05 30.05 30.05 30.05
G.ROOT-SERVERS.NET. 30.12 30.11 30.12 30.14 30.12
B.ROOT-SERVERS.NET. 30.69 30.04 30.03 30.03 30.03
D.ROOT-SERVERS.NET. 30.22 30.19 30.19 30.17 30.18
ord.sns-pb.isc.org. 30.17 30.17 30.17 30.17 30.17
I.ROOT-SERVERS.NET. 30.29 30.29 30.29 30.29 30.30
M.ROOT-SERVERS.NET. 30.33 30.34 30.34 30.34 30.33
J.ROOT-SERVERS.NET. 30.33 30.32 30.32 30.33 30.33
ams.sns-pb.isc.org. 30.43 30.44 30.44 30.44 30.46
ns.isi.edu. 30.04 30.94 31.01 30.99 30.04
L.GTLD-SERVERS.NET. 5.44 5.52 60.02 60.03 60.02
ns1.hkg1.afilias-nst.info. 60.02 60.02 60.02 60.03 60.03
A2.ORG.AFILIAS-NST.INFO. 74.03 59.74 65.32 90.95 64.74
B2.ORG.AFILIAS-NST.org. 74.06 60.16 65.87 90.67 64.76
L.ROOT-SERVERS.NET. 120.16 120.16 120.15 120.15 120.16
K.ROOT-SERVERS.NET. 122.20 120.21 120.21 120.20 120.20
H.ROOT-SERVERS.NET. 125.07 120.16 120.15 120.17 120.16

---

From: Mark Andrews <[email-masked]>
Date: Tue, 25 Aug 2009 11:35:25 +1000

> or 12 hours, do to the TCP stats?

And the fact that at least some of the servers just set TC on DNSKEY
queries.  If you don't let fragmented answers go out then the broken
middleware will not be made visible and it won't be fixed.  All you
do is penalise the sites that have done the correct thing by forcing
them to use TCP when they don't need to.

drugs:9.7.x 11:24 {566} % dig dnskey org @a0.org.afilias-nst.info. +ignore +dnssec

; <<>> DiG 9.3.6-P1 <<>> dnskey org @a0.org.afilias-nst.info. +ignore +dnssec
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31422
;; flags: qr aa tc rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1280
;; QUESTION SECTION:
;org.                           IN      DNSKEY

;; Query time: 261 msec
;; SERVER: 2001:500:e::1#53(2001:500:e::1)
;; WHEN: Tue Aug 25 11:24:21 2009
;; MSG SIZE  rcvd: 32

drugs:9.7.x 11:24 {567} % 

> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.8 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iEYEARECAAYFAkqTJSMACgkQ+NNi0s9NRJ3KOgCglNRCaRVXlc163VXOCEEyktBv
> PV8An0sva7H5XmaWT0dJu5fNXdPfrcyH
> =3PeR
> -----END PGP SIGNATURE-----
> 
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Joe Abley <[email-masked]>
Date: Mon, 24 Aug 2009 21:38:22 -0400

On 24-Aug-2009, at 19:41, Michael Graff wrote:

> hurt ORG initially

... and just on that, hurt how? My observation was that even though  
the increased amount of TCP state was not necessarily expected (or  
else perhaps the TTL 0 bug might have been fixed before signing) in  
fact the infrastructure held up just fine, and there was no pain.

I am somewhat sympathetic to the argument that if ORG could be signed,  
running open source software which implements the DNS standards with  
no special magic, with an unexpected TTL 0 problem, with no other  
elaborate precautions and with a zone several orders of magnitude  
larger than what most people need to handle, then almost all other  
zone administrators have nothing to worry about.

[I'll note in passing that I have conceded in the past that the root  
zone is at least potentially different, since its servers have the  
unusual property of having to respond to priming queries from every  
DNS client on the Internet, old and new. But the concern in this  
thread seems to be more about TCP in general, for the DNS in general,  
than for the root in particular.]

Joe

---

From: Michael Graff <[email-masked]>
Date: Mon, 24 Aug 2009 20:54:19 -0500

Joe Abley wrote:
> ... and just on that, hurt how? My observation was that even though the
> increased amount of TCP state was not necessarily expected (or else
> perhaps the TTL 0 bug might have been fixed before signing) in fact the
> infrastructure held up just fine, and there was no pain.

"hurt" was a bad choice here.  I meant 'caused the big scare we are all
talking about."

That is, the huge increase graph I saw presented.  I'm wondering if the
600x increase or whatever was artificially increased due to the 0 TTL thing.

Your argument is stronger though -- if .org did not face-plant, and that
with the worst choice of a TTL value which disabled caching and negative
caching -- what are we all worried about again?

I hold to my guns that TCP is not a problem here.  Perhaps TCP code in
the current DNS implementations just needs to be fixed up to be closer
to http performance is all.

- --Michael

---

From: Michael Graff <[email-masked]>
Date: Mon, 24 Aug 2009 20:59:05 -0500

> for speculation.  i can probbaly be re-educated if somebody does some

The packet round trip affects a single transaction.  It does not affect
answers per second unless the wire or switches are saturated with packet
limits.

That is, sure, a badly written client that sends one request over tcp,
waits for the response, then sends another over a different stream would
be hurting in your analysis.  But that isn't what we have here.  We have
thousands of clients doing the handshake, each one takes a penalty for
the RTT but the server should still be able to maintain a reasonable QPS
measurement.

- --Michael

---

From: Douglas Otis <[email-masked]>
Date: Mon, 24 Aug 2009 19:23:32 -0700

On 8/24/09 5:55 PM, Paul Vixie wrote:
>> From: Michael Graff<[email-masked]>
>>

Once DNS UDP packets commonly exceed typical PMTUs, TCP will then become 
relied upon.  Although .se TLD is signed, few domains below .se are 
signed.  The same appears true for .org.

It would be interesting to analyze perhaps an hours worth of DNS logs of 
an ISP's resolver.  Two aspects need review.  One would be the nature of 
traffic to between customers and the recursive resolver.  The other 
would be the nature of traffic between recursive resolver and 
authoritative servers.

It seems unlikely the path to the authoritative servers represent a 
significant factor in overhead or latency since most of the records will 
be published with long TTLs.  The results obtained could then indicate 
an ideal Auto-Close duration for SCTP.  From this, one could then 
estimate the expected overhead increase when using SCTP instead of UDP.

Rather than relying upon TCP, SCTP offers important advantages.  TCP 
requires ordered delivery, where DNS messages might not be packet 
aligned.  In addition, any packet loss will hang subsequent processing. 
  On the other hand, SCTP can establish DNS message alignment within the 
packet, and support out-of-order delivery to avoid buffering 
out-of-order information and the delay created by maliciously 
intentional or congestion related packet loss.

The SCTP tag validation, in conjunction with DNS transaction-IDs, 
provide adequate protection from blind spoofing, regardless of the 
data-rate or the receive window size.  That is not as true for TCP.  TCP 
that encodes the byte sequence, also means it can not use options that 
are able to scale the 64KB receive window.  SCTP does not suffer from 
these limitations.

Rather than the DNS application being expected to handling large amounts 
of message reassembly and buffering, as well as timing the duration of 
connections, SCTP provides this functionality within the kernel.  Unlike 
TCP, there is also no half-closed SCTP state.

-Doug

---

From: Paul Vixie <[email-masked]>
Date: Tue, 25 Aug 2009 04:53:33 +0000

> Date: Mon, 24 Aug 2009 20:59:05 -0500

sounds like you're not going to have any problem proving this and thus
advancing the debate beyond tcp-fear vs. tcp-nofear.  (and note that the
comparisons with http aren't useful unless you're comparing to http/0.9
since we're not going to see very many back to back queries in any flow.)

> the RTT but the server should still be able to maintain a reasonable QPS

i know the theory of windowing.  i explicit think it does not apply here.
in any case a proper simulation would (as i said earlier) be of a mix of
lan and wan clients, not of a single client doing a deep queue.

let me be clear.  until someone does the test and shows that this can be
done with the same general class of equipment that does high volume udp/53
today and gets the same order of magnitude of performance with tcp/53, the
onus of proof will remain on those who assert that it can be done.

---

From: Jim Reid <[email-masked]>
Date: Tue, 25 Aug 2009 08:24:23 +0100

> implementaion is pointless.

Not really Michael. Even if the platforms in the lab environment are  
not optimised, a benchmark would provide real data: something that's  
missing from the discussion until now. Besides, the servers and TCP  
stacks in that lab should be representative of the sort of kit that's  
in use today. Even if the results show that the TCP throughput isn't  
good enough, at least there would be some data points for those who  
might work on optimising the name server and/or TCP software.

---

From: Michael Graff <[email-masked]>
Date: Tue, 25 Aug 2009 15:50:24 -0500

Jim Reid wrote:
> Not really Michael. Even if the platforms in the lab environment are not
> optimised, a benchmark would provide real data: something that's missing
> from the discussion until now. Besides, the servers and TCP stacks in
> that lab should be representative of the sort of kit that's in use
> today. Even if the results show that the TCP throughput isn't good
> enough, at least there would be some data points for those who might
> work on optimising the name server and/or TCP software.

Very true.

All I'm saying is that I don't want someone to benchmark current DNS
implementations (which are likely optimized only for UDP) and then use
this as proof that the sky is falling.

Knowing two data points -- what a well tuned TCP based system can do, vs
what we can do out of the box with current implementations, is useful.

- --Michael

---

From: Tony Finch <[email-masked]>
Date: Mon, 31 Aug 2009 15:49:33 +0100

On Mon, 24 Aug 2009, Nicholas Weaver wrote:
>
> Whats wrong with "When state exceeds LRU, close", or heck, "If a 'normal'
> query, close connection when complete".

Remember you should accommodate clients that pipeline multiple queries
down the same TCP connection.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

---

From: Alex Bligh <[email-masked]>
Date: Mon, 31 Aug 2009 18:38:36 +0100

> down the same TCP connection.

Hmmm... Elsewhere Mark Andrews suggested that in extremis clients behind
broken middleboxen could simply have their nameservers manually coded
to resolvers outside the broken middlebox (rather than use the middlebox
itself).

For these, plus middleboxes which just pass through the external
nameserver's address, there is still a problem if they drop
UDP fragments or otherwise prevent long answers that aren't
addressed to the middlebox, causing TCP fallback.

But in such circumstance, if TCP is consistently pipelined, the
adverse effect is at least somewhat reduced, as there will
be a single caching server being queried by the stub resolver,
so at least there won't be a constant stream of setups and
tear downs.

Question: do stub resolvers on popular operating systems actually
fall back to TCP, and if so, do they pipeline?

-- 
Alex Bligh

---

From: bert hubert <[email-masked]>
Date: Mon, 31 Aug 2009 20:04:14 +0200

> fall back to TCP, and if so, do they pipeline?

Some do. Sadly, it turns out there are multiple stubs in most popular
operating systems, with some heavy DNS consumers choosing to roll
their own.

Some versions of Exchange are known to keep open TCP/IP sessions to a
resolver for multiple queries. So it does happen, but it is not common
practice.

   Bert

---

From: Alex Bligh <[email-masked]>
Date: Mon, 31 Aug 2009 19:15:24 +0100

> resolver for multiple queries. So it does happen, but it is not common

Is it a fair assumption that to deploy DNSSEC a fair number of these
stubs will need changing anyway? (and could thus be made to pipeline
TCP requests)

-- 
Alex Bligh

---

From: Douglas Otis <[email-masked]>
Date: Mon, 31 Aug 2009 12:04:14 -0700

>> down the same TCP connection.
[]
> fall back to TCP, and if so, do they pipeline?

DNS message pipe-lining can significantly reduce response latency. 
However, for TCP, DNS message pipe-lining represents an expensive and 
complex mode of operation.  Without transport level framing, TCP 
pipelines carrying DNS messages will need to buffer the entire stream 
subsequent to packet loss.

However, SCTP processing of DNS messages can continue subsequent to 
unrecovered packets.  It should be reasonable for DNS servers to limit 
pipeline operations to the SCTP transport, rather than being handled by 
the DNS application over TCP.  SCTP can also be configured to require 
clients to re-issue DNS requests, rather than having DNS servers retain 
images of the DNS messages in transit.  This too reduces the required 
buffering.

The complexities related to TCP packet-loss represents a significant 
impediment to cost effective implementations, largely due to the 
significant amounts of memory involved.  For TCP, buffering would be a 
function of the RTT and the network bandwidth.  Alternatively, SCTP is 
able to avoid much this buffering requirement.

SCTP could be employed to avoid having to upgrade DNS servers to support 
TCP pipelining.  Once TCP is deemed essential, then the expense related 
to TCP buffering can not be avoided, and it would then represent a 
vulnerability when it necessitates significant increases in the number 
of servers.  Transport modes that might be either unsafe or too 
expensive from a resource perspective should not be made essential.

-Doug

---

From: Alex Bligh <[email-masked]>
Date: Mon, 31 Aug 2009 20:22:02 +0100

> subsequent to packet loss.

Right, but unless I'm missing something, that complexity is already
handled by the TCP stacks at either end. As to expense, I would have
thought handling n queries pipelined was less expensive than
n setups and teardowns; clearly UDP and (for all I know) SCTP
would be cheaper than either.

> unrecovered packets.

I will take you word for it re SCTP, but I don't see how that helps
here; by assumption, the middlebox is broken and not easily fixed/replaced,
and thus extremely unlikely to be permeable to SCTP. Using SCTP requires
not only a stub resolver change, but also a client IP stack change,
a middlebox change, a nameserver s/w change, and a nameserver IP
stack change. That is, I think, changing just about every component
in the system, in order to address a problem with one component.

I'm not saying TCP fallback is fine. I'm saying that where we might
actually see it in practice because of broken middleboxen, it might
be less bad than we thought because of pipelining (assuming the
stub resolver that sets DO implements this).

-- 
Alex Bligh

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 31 Aug 2009 12:43:08 -0700

On Aug 31, 2009, at 12:22 PM, Alex Bligh wrote:

>
>
> --On 31 August 2009 12:04:14 -0700 Douglas Otis <dotis@mail- 
> abuse.org> wrote:
>

THere are two resources and one property here: compute, memory, and  
setup latency.

Dropping each TCP connection after a single query will require  
trivially more compute, but will be massive in terms of saving  
memory.  Assume that each TCP connection requires 50KB in state to  
manage in the OS/kernel/etc.  If you are handling 1000 new hosts/ 
second, and you maintain each TCP connection for 10 seconds, thats  
500MB of state.  Owch.

But if you tear down each connection after the query is received, its  
almost NO state, as you don't need to allocate it until you get the  
last packet in the handshake, and you can immediately deallocate it  
once the query is responded (if you close with a RST so that you don't  
leave a half-open connection.  Many webservers do this when closing  
open connections.  Once you've decided to close a connection, leaving  
it half-open is pointless anyway.)

The only disadvantage is now all TCP queries will see an additional  
1RTT in latency for the handshake.

In general, it is both far simpler and far less resource intensive to  
do "one query per TCP connection".  The major reason why this is NOT  
done for HTTP is that additional 1-RTT latency, and, that for HTTP,  
you'd be forever stuck in slowstart.

---

From: Paul Vixie <[email-masked]>
Date: Mon, 31 Aug 2009 20:00:56 +0000

> From: Nicholas Weaver <[email-masked]>
> Date: Mon, 31 Aug 2009 12:43:08 -0700
> ...
> Dropping each TCP connection after a single query will require trivially
> more compute, but will be massive in terms of saving memory.  Assume that
> each TCP connection requires 50KB in state to manage in the
> OS/kernel/etc.  If you are handling 1000 new hosts/ second, and you
> maintain each TCP connection for 10 seconds, thats 500MB of state.  Owch.
> 
> almost NO state, as you don't need to allocate it until you get the last
> packet in the handshake, and you can immediately deallocate it once the
> query is responded (if you close with a RST so that you don't leave a
> half-open connection.  Many webservers do this when closing open
> connections.  Once you've decided to close a connection, leaving it
> half-open is pointless anyway.)
> 
> The only disadvantage is now all TCP queries will see an additional  1RTT
> in latency for the handshake.

i propose that if we adopt a responder-close attitude toward TCP/53, that
responders be permitted to close but not required to, and that a specific
strategy mentioned in that specification be that if there is another query
in the pipeline, already in the responder's input socket buffers at the
time it has transmitted a response, that it consider answering the new
query.  that way if someone wants a lot of queries answered they can just
pipeline.  responders should also be allowed to use short timeouts (50ms?)
or adaptive timeouts (shorter when there are more concurrent tcp sessions
open or more QPS recently handled).

something like this may lead to a situation where busy caching servers keep
TCP sessions open to popular authority servers, and a session state DoS just
slows that down to one query per session.

> In general, it is both far simpler and far less resource intensive to do
> "one query per TCP connection".  The major reason why this is NOT done
> for HTTP is that additional 1-RTT latency, and, that for HTTP, you'd be
> forever stuck in slowstart.

it's not just the slow start, it's all those web pages with 500 images on
them, and the inability of both servers and browsers to open 500 tcp
sessions at once to get them all in parallel.  the couple of RTT's involved
in TCP setup and teardown are a terrifying spectacle when doing 500 of them
in serial.

---

From: Alex Bligh <[email-masked]>
Date: Mon, 31 Aug 2009 21:13:09 +0100

> OS/kernel/etc.  If you are handling 1000 new hosts/second, and you

Mmmm... it's been a while since I looked at this in a live kernel, but
I thought the amount of STATE recorded per TCP session in the kernel
was closer to 2.5kb (certainly less than a page). In addition to this,
you need buffers, the size of which it is possible to control on
a per socket basis. Assuming sensible values of buffering, you are
well under half the above amount. Reduce the TCP hold time to 5
seconds, and you're at about 100Mb of memory, which these days is
in the "so what" zone, I would have thought, for a dedicated
caching resolver.

> in latency for the handshake.

Plus additional CPU overhead in setting up and tearing down the connection.

I would bet handling 10 queries in one TCP session is substantially
more lightweight than handling 10 separate TCP sessions.

> forever stuck in slowstart.

Why don't the same arguments apply to DNS? If your average DNS packet
is smaller, the extra 1 x RTT latency is proportionately going to be
a more significant part of the transaction.

-- 
Alex Bligh

---

From: Tony Finch <[email-masked]>
Date: Mon, 31 Aug 2009 21:19:04 +0100

> fall back to TCP, and if so, do they pipeline?

The usual high-level resolver APIs don't allow pipelining, but GNU adns
(for example) makes it easy. You can pipeline using the low-level
resolver API but it's much more fiddly.

Tony.

---

From: Alex Bligh <[email-masked]>
Date: Mon, 31 Aug 2009 21:19:28 +0100

> in the "so what" zone, I would have thought, for a dedicated

& the above would assume that the socket buffers are fully allocated
all the time. It would be sensible for OSs to free socket buffer
pages when they are empty, which they would be during the vast
majority of the 10 second period you mention.

-- 
Alex Bligh

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 31 Aug 2009 13:20:58 -0700

On Aug 31, 2009, at 1:13 PM, Alex Bligh wrote:
>> trivially

Not necessarily.  EG, until a packet is ACKed' the TCP stack needs to  
keep full sending state, which means it can easily be full-window.

But I agree, with a short timeout, it should be "whatever".  But you  
have those who insist the DNS spec is sancrosanct in saying you should  
keep these connections open for 2 minutes (which, thank god, nobody  
actually does).
> Plus additional CPU overhead in setting up and tearing down the  

This really is trivial.  With SYN-cookies, its a trivial amount of  
compute to do.

Remember, we are living in a very interesting world:  Compute is  
"Free".  State that isn't accessed is almost "free" too, but state  
which IS accessed is frightfully expensive.

> more lightweight than handling 10 separate TCP sessions.

I'd bet the difference is far smaller than you think.  Anyone have any  
benchmarks?

---

From: Tony Finch <[email-masked]>
Date: Mon, 31 Aug 2009 21:25:17 +0100

On Mon, 31 Aug 2009, Douglas Otis wrote:
>
> DNS message pipe-lining can significantly reduce response latency. However,
> for TCP, DNS message pipe-lining represents an expensive and complex mode of
> operation.  Without transport level framing, TCP pipelines carrying DNS
> messages will need to buffer the entire stream subsequent to packet loss.

It isn't really expensive or complex, at least as currently implemented.
AFAIK neither the stubs nor the recursors worry about head-of-line
blocking and it's common for recursors to process TCP requests in order.
Even so, it works well as a fallback for that proportion of requests whose
UDP responses get truncated (so the stub only needs two sockets open).

Tony.

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 31 Aug 2009 13:28:28 -0700

> slows that down to one query per session.

This makes a lot of sense.  You can avoid the overhead in the case  
where things are persistent, but not hold the state.

However, simpler heuristics would work just as well.

EG, "When under state pressure, close in an Least Recently Used"  
fashion: close most-idle first, and close with a RST to free all state  
immediately.

This would have the same effect, really: you can always answer a new  
TCP query (which requires near-zero state), and old persistent  
connections can stick around for as long as possible.

> in TCP setup and teardown are a terrifying spectacle when doing 500  

The 1 RTT latency...

Latency is a shocking killer in the Web world, its part of the reason  
HTTPS isn't ubiquitous: https requires several more RTTs to set up a  
connection.

---

From: Alex Bligh <[email-masked]>
Date: Mon, 31 Aug 2009 21:27:56 +0100

> This really is trivial.  With SYN-cookies, its a trivial amount of

It wasn't so much that bit I was worried about, as the
allocating/deallocating of a new FD etc.

A /dis/advantage of pipelining is going to be that you are potentially
more prone to FD exhaustion.

> I'd bet the difference is far smaller than you think.  Anyone have any

This would be a better idea than both of us speculating :-)

-- 
Alex Bligh

---

From: Tony Finch <[email-masked]>
Date: Mon, 31 Aug 2009 21:31:52 +0100

On Mon, 31 Aug 2009, Paul Vixie wrote:
> Nicholas Weaver <[email-masked]> wrote:
> >

Especially when the round trip includes a POTS modem :-)

The other big reason is that TCP congestion control takes a while to get
going. In the mid-1990s many people were predicting the imminent death of
the net due to congestion collapse caused by most TCP connections being
too short-lived.

Tony.

---

From: Mark Andrews <[email-masked]>
Date: Tue, 01 Sep 2009 07:11:11 +1000

> fall back to TCP, and if so, do they pipeline?

Yes and yes if requested by the application.  This has been the
case since BIND 4.8 days.  This is also one of the reason how I
can't understand how a DNS proxy can fail to support TCP since there
have been stub resolvers that fallback to TCP the entire time.

No real testing was ever done to see what enviroment they were going
into.  TCP queries, due to TC=1 being set, were rare but not
non-existant.
 
Mark

---

From: bert hubert <[email-masked]>
Date: Mon, 31 Aug 2009 23:31:40 +0200

> into.  TCP queries, due to TC=1 being set, were rare but not

I estimate that your average Windows based end-user ('client') PC
spends its whole life without ever doing a single TCP/IP DNS request.

I would not even be very surprised if it turns out the default Windows
stub can't do it.

     Bert

---

From: Douglas Otis <[email-masked]>
Date: Mon, 31 Aug 2009 15:04:06 -0700

> UDP responses get truncated (so the stub only needs two sockets open).

DNS message handling for TCP is done by the DNS application, even when 
enduring the expensive head-of-line blocking.  Full message handling, 
and options to avoid the expense caused by packet loss or errors is 
provided by SCTP, but not by TCP.

DNS messages pipe-lined over the TCP byte-stream use a 16 bit length 
value prefixed to each.  Over TCP, DNS messages should be considered 
independent of packet boundaries, and once a packet is lost, locations 
of message boundaries within subsequent packets becomes unknown.  This 
is not the case with SCTP.  SCTP is not a byte stream transport. 
Objects handled by SCTP can represent full DNS messages.

To support TCP re-transmissions of lost portions of DNS messages, the 
entire outstanding transmitted byte stream must be retained.  This is 
also not required for SCTP.

During times of stress, such as attacks, packet-loss will place 
non-linearly increasing demands upon server resources.  SCTP is able to 
avoid these run-away demands dependent upon external RTT, data rates, 
and packet-loss rates, which can lead to TCP services collapsing.

-Doug