Subject: [dnsext] Enforcing correct behavior: EDNS0 OK bit set, buffer < 1024

From: Olafur Gudmundsson <[email-masked]>
Date: Wed, 03 Jun 2009 17:35:37 -0400

[speaking as RFC3226 editor with encouragement from RFC3225 editor]

Strictly speaking based on RFC3225 and RFC3226 this is a protocol
violation. Unfortunately there is no explicit statement on how to
treat this mis-behavior.

People claim this is somewhat common in the wild, these
implementations/configurations will get hurt when DNSSEC is deployed at
the root and they ask for ". NS" or now when asking for "org. DNSKEY"

In most cases this is caused by people configuring DNS implementations
to use small buffers, mis-guided firewall, etc.

Can we make these misbehaviors suffer sooner by changing/configuring
implementations to return "REFUSED" to any query that has OPT record
with DO bit set and buffer size less than 1024 or 1220.

In addition I propose to write an update to RFC3226 to include a
error handling section:
-----
When DNS responder sees a DNS message with the DO bit set and
buffer size < 1220 the responder SHOULD return a message with the
RCODE = Refused, and include in the OPT record a new option
REASON that contains following string "DO requires EDNS buffer size > 1220"
----

The REASON EDNS0 Option will be defined in the same document.

	Olafur

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/namedroppers/>

---

From: Mark Andrews <[email-masked]>
Date: Thu, 04 Jun 2009 11:08:58 +1000

In message <[email-masked]>, Olafur Gudmundsson wri
> the root and they ask for ". NS" or now when asking for "org. DNSKEY"

	I believe this is just excesive worry.

	EDNS referrals from the root servers to COM and NET already
	exceed 512 bytes.  If this was going to be a real problem
	it would already have happened.
 
> to use small buffers, mis-guided firewall, etc.

	Or broken routers in the middle of the net.  Not all of the
	sources of problems are under the control of client.

	Some of the problems may actually be the authoritative
	servers, or more correctly their host OS's, which set DF
	on outgoing UDP responses.  BIND now turns DF off on a per
	socket basis when the OS supports it and we have discovered
	the OS specific magic to do it.

> with DO bit set and buffer size less than 1024 or 1220.

	Please NO.  Falling back to EDNS@512 keeps DNSSEC working
	and in many cases just trims off some additional records
	which are causing the response to be dropped.
 
	I've worried about this issue for years.  BIND's gone from
	silently fixing the issue to being noisy when it has to
	fallback.  Excessively noisy initially.

	Mark

-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Paul Vixie <[email-masked]>
Date: Thu, 04 Jun 2009 03:06:59 +0000

> From: Mark Andrews <[email-masked]>
> Date: Thu, 04 Jun 2009 11:08:58 +1000
> ...
> 	...  BIND now turns DF off on a per socket basis when the OS
> 	supports it and we have discovered the OS specific magic to do it.
> ...
> 	I've worried about this issue for years.  BIND's gone from silently
> 	fixing the issue to being noisy when it has to fallback.
> 	Excessively noisy initially.

can you ensure that 2671bis (about to go into production) covers our
implementation experience and makes recommendations about how fallback
ought to work?

---

From: David Conrad <[email-masked]>
Date: Thu, 4 Jun 2009 08:54:26 -0700

Mark,

On Jun 3, 2009, at 6:08 PM, Mark Andrews wrote:
> 	exceed 512 bytes.

Indeed they do.

> If this was going to be a real problem it would already have happened.

What percentage of queries to the root result in NXDOMAIN?

How big are signed NXDOMAIN responses?

How many of those signed NXDOMAIN responses are going to come back  
with TC set?

Is that number going to go up or down over time?

> 	which are causing the response to be dropped.

There is a reason the minimum buffer size for DNSSEC signed responses  
was chosen to be 1220.  If you do not believe those reasons were  
justified, then you should provide revisions to 3226.  Simply ignoring  
3226 is just wrong.

Regards,
-drc

---

From: Matt Larson <[email-masked]>
Date: Thu, 4 Jun 2009 18:45:24 -0400

> justified, then you should provide revisions to 3226.  Simply ignoring  

I'm in total agreement with David.  BIND should not be setting DO if
it's also advertising a maximum buffer size of 512.

Matt

---

From: Mark Andrews <[email-masked]>
Date: Fri, 05 Jun 2009 12:05:29 +1000

In message <[email-masked]>, David Conrad
> How big are signed NXDOMAIN responses?

	What sized ZSK's are you going to be using?  Are you going
	to double sign or just replace when re-signing?
 
> How many of those signed NXDOMAIN responses are going to come back  

	To EDNS@512 queries, most.  To EDNS@1024 queries, most.  To
	EDNS@1200 at the root almost none, for other zones a higher
	percentage as the NSEC records are bigger.  To EDNS@1460
	queries, almost none.

	How many of those NXDOMAIN queries should not need to be
	generated in the first place?
 
> Is that number going to go up or down over time?

	When DNSSEC is enabled there will be a jump.   The long term
	trend depends on a lot of things.

	Default firewall setting.  We should encourage firewall
	vendors to SERVFAIL EDNS queries if they are not going to
	to allow large replies back in.  Work with the protocol
	rather than just stuff the protocol up.  Better still they
	should recognise EDNS queries and allow adjust their filters
	to accept EDNS responses.  Allowing non-standard queries
	to go out and not accepting the replies is bad firewall
	design.

	Encouraging ISP's to be stealth secondary of the root would
	stop a great percentage of these bogus queries reaching the
	root servers.

	Have people recognise that single label hostnames are not
	supposed to work in the global namespace would stop a large
	percentage as we could kill these in the stub resolver.
	There was a reason we went to <foo>.ARPA when heirachial
	names were introduced.

	Do all the root server fragment packets so that the smallest
	fragment contains the initial segment so that the probability
	of out of order fragement is reduced?  This helps firewalls
	and NAT's that only process fragmented responses correctly
	when the initial fragment arrives first.

> justified, then you should provide revisions to 3226.  Simply ignoring  

	And there are also reasons to fallback to a smaller number
	when the initial setting fails.  Unfortunately 512 it the
	obvious choise when falling back as it passes through almost
	all devices.  1220/1460 as a fallback only handles fragmentation
	problems.
 
	Do we have any published stats on the number of queries,
	which result in a referral that would exceed 512 bytes if
	records no record are dropped from the additional section,
	that the have a EDNS UDP size set to 512 compared to those
	we have a size >= 1200?  That should give you a percentage
	figure on how many queries will be affected.

	This thread started with a unsupported supposition that
	there are lots of queries that will cause problems.  I've
	seen no figures to back that up.  I know that there will
	be some queries come in with EDNS UDP size set to 512.

	Mark

---

From: Florian Weimer <[email-masked]>
Date: Fri, 05 Jun 2009 12:01:32 +0200

> 	when the initial fragment arrives first.

Last time I checked, the roots (except one) did not perform PMTUD, so
fragmentation is not under their control.  Many DNS servers send the
final fragment first anyway.  This is not very likely to change.

Anyway, I doubt you'll find serious stateful packet filters which do
not reassemble all packets traversing the device (and fragment again,
if needed).  Stateless filters are a different story, of course, but
to them, the order of fragments doesn't matter, either.

To be honest, I have serious doubts if it makes sense to worry about
this.  Chances are that DNS traffic moves to IPv6 faster than we can
reach consensus. 8-/

-- 
Florian Weimer                <[email-masked]>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstraße 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99

---

From: David Conrad <[email-masked]>
Date: Fri, 5 Jun 2009 09:49:18 -0700

Florian,

On Jun 5, 2009, at 3:01 AM, Florian Weimer wrote:
> this.

How many TCP sessions are the root servers prepared to handle?

> Chances are that DNS traffic moves to IPv6 faster than we can
> reach consensus. 8-/

We reached consensus.  See RFC 3226.

Regards,
-drc

---

From: David Conrad <[email-masked]>
Date: Fri, 5 Jun 2009 10:02:11 -0700

Mark,

> 	to double sign or just replace when re-signing?

Final decision on this hasn't been made (at least to my knowledge) and  
I'm the wrong person to ask.  However, an example using what I believe  
are best common practices can be found at ns.iana.org.

> 	generated in the first place?

Not sure how this is relevant unless you've created a tool that makes  
people stop miconfiguring stuff/ being stupid.

> 	trend depends on a lot of things.
[bunch of changes of how things are deleted]

I'd suggest we discuss the Internet as it is and will likely continue  
to be the day after the root is signed, not as we might like it to be.

> 	all devices.

Lovely.  Turn off DO if you fall back to a buffer size less than 1220.

Regards,
-drc

---

From: [email-masked]
Date: Fri, 5 Jun 2009 17:06:16 +0000

> How many TCP sessions are the root servers prepared to handle?

	to be fair, your should likely recast this as:

	how many concurrent TCP sessions are DNS servers prepared to handle?

> -drc

--bill

---

From: Olafur Gudmundsson <[email-masked]>
Date: Fri, 05 Jun 2009 13:11:14 -0400

>         how many concurrent TCP sessions are DNS servers prepared to handle?

Bill,
you are a root server operator,
what order of magnitude of concurrent TCP sessions are YOUR DNS servers
prepared to handle?
          10^x  x = ?

         Olafur

---

From: Paul Vixie <[email-masked]>
Date: Fri, 05 Jun 2009 17:29:35 +0000

> From: David Conrad <[email-masked]>
> Date: Fri, 5 Jun 2009 09:49:18 -0700
> 

very few.  a fraction of a fraction of a fraction of current udp volume.
this is not a fault in the root server design or provisioning, but in the
dns protocol itself.

---

From: Paul Vixie <[email-masked]>
Date: Fri, 05 Jun 2009 17:30:46 +0000

> Date: Fri, 5 Jun 2009 10:02:11 -0700
> ...

without intending this as instructions to marka, i have to say, i agree.

---

From: Paul Vixie <[email-masked]>
Date: Fri, 05 Jun 2009 17:30:01 +0000

> Date: Fri, 5 Jun 2009 17:06:16 +0000
> From: [email-masked]
> 

very, very, few.

---

From: [email-masked]
Date: Fri, 5 Jun 2009 17:47:59 +0000

> what order of magnitude of concurrent TCP sessions are YOUR DNS servers

	before or after we tune the OS and the DNS server software?
	defaults are quite small, since the expectation is UDP...

--bill

---

From: Olafur Gudmundsson <[email-masked]>
Date: Fri, 05 Jun 2009 15:00:59 -0400

>without intending this as instructions to marka, i have to say, i agree.

This is not just Bind at least: drill, unbound, Net::DNS, and dnsjava
have the same issue.
I can see that tools like drill that are used for testing can overwrite
the RFC3226 recommendation, but recursive resolvers should by default 
not do that.

At the same time if the recommendation in RFC3226 is out-of-date then we should
update it.

         Olafur

---

From: Paul Vixie <[email-masked]>
Date: Fri, 05 Jun 2009 21:38:31 +0000

> Date: Fri, 5 Jun 2009 17:47:59 +0000

the speed of light being a constant, but with the number of connected
endsystems and networks increasing, and with the density of interconnects
gradually declining... tuning does not matter.  it's those darn round trips.

---

From: [email-masked]
Date: Fri, 5 Jun 2009 21:52:51 +0000

> gradually declining... tuning does not matter.  it's those darn round trips.

	well that and the half-open problems... but thats not what
	Olafur asked about...  

	he asked how many... and I can't answer him w/o knowing if he wants
	the defaults or the tuned numbers. I'm not going to give him both

--bill

---

From: Mark Andrews <[email-masked]>
Date: Tue, 09 Jun 2009 10:26:59 +1000

> final fragment first anyway.  This is not very likely to change.

I beg to differ.  8 of the roots are performing PMTU discovery over IPv4/UDP.
Whether the operators are aware of this or not is another matter.  This can
usually be disabled at the the application layer.

09:57:54.387361 198.41.0.4.53 > 211.30.172.21.64982:  24664*- 1/13/11 SOA (497) (DF)
09:58:19.419990 192.228.79.201.53 > 211.30.172.21.54074:  11917*- 1/13/13 SOA (493) (DF)
09:58:56.196852 192.112.36.4.53 > 211.30.172.21.63951:  55465*- 1/13/13 SOA (493) (DF)
09:59:17.432535 128.63.2.53.53 > 211.30.172.21.51513:  47857*- 1/13/13 SOA (493) (DF)
09:59:30.359185 192.36.148.17.53 > 211.30.172.21.60259:  52319*- 1/13/11 SOA (497) (DF)
09:59:42.443443 192.58.128.30.53 > 211.30.172.21.54445:  20147*- 1/13/11 SOA (497) (DF)
10:00:33.782430 193.0.14.129.53 > 211.30.172.21.61326:  48813*- 1/13/13 SOA (493) (DF)
10:00:47.262075 199.7.83.42.53 > 211.30.172.21.51447:  37089*- 1/13/13 SOA (493) (DF)

Also the root servers will almost certainly do the initial fragmentation
of the DNS responses, whether it is IPv4 or IPv6.  99.9% of the
time no further fragmentation will occur.  How the roots choose to
fragement is important to minimising problems caused by misimplementations
in other network elements.  How the roots choose to fragment also
dictates the probability of additional fragmentation occuring.

> to them, the order of fragments doesn't matter, either.

I'm aware of PNAT's which fail to handle fragmented responses where
the initial fragement doesn't arrive first.
 
> this.  Chances are that DNS traffic moves to IPv6 faster than we can

The issues are applicable to IPv6 as they are to IPv4.  IPv6 has
the advantage of a API to control fragmention in the sender.

> --=20
> Kriegsstra=DFe 100              tel: +49-721-96201-1

---

From: Florian Weimer <[email-masked]>
Date: Thu, 18 Jun 2009 18:45:14 +0200

> usually be disabled at the the application layer.

AFAICT, it can't.  You might not send out PMTUD probe messages, but
this doesn't prevent the stack from processing unsolicited answers and
applying the result to the packets you send.

> 09:57:54.387361 198.41.0.4.53 > 211.30.172.21.64982:  24664*- 1/13/11 SOA (497) (DF)

You also need to check if the ICMP "fragmentation needed but DF bit
set" message is processed by the server.

---

From: Mark Andrews <[email-masked]>
Date: Fri, 19 Jun 2009 10:41:21 +1000

In message <[email-masked]>, Florian Weimer writes:
> > I beg to differ.  8 of the roots are performing PMTU discovery over IPv4/=
> UDP.
> > Whether the operators are aware of this or not is another matter.  This c=
> an

	"actively performing PMTU discovery over IPv4" then if you
	want to be picky.

> > 09:57:54.387361 198.41.0.4.53 > 211.30.172.21.64982:  24664*- 1/13/11 SOA=
>  (497) (DF)
> 

	Given the OS chooses to set DS and not the application it is
	reasonable to assume it does process the messages.
	
	Mark