Subject: [dnsext] Question on EDNS MTU...

From: Nicholas Weaver <[email-masked]>
Date: Thu, 3 Dec 2009 13:31:35 -0800

Does anyone know of DNS resolver software other than Google's new recursive resolver service that uses a 1280B EDNS MTU?

---

From: [email-masked]
Date: Thu, 3 Dec 2009 22:24:15 +0000

> Does anyone know of DNS resolver software other than Google's new 
> recursive resolver service that uses a 1280B EDNS MTU?

I can't tell you what brand of server it is, but when I checked the EDNS0 
buffer sizes seen from clients of our authoritative servers 1280B was 
relatively common.

Admittedly it was only about 1.5% of the total EDNS0, but 2048B and 512B 
were about the same volume.  4096B was by far the most common, accounting 
for over 90% of EDNS0 packets.

What is notable is that almost all of those 1280B EDNS0 packets have DO=0.

Ray

---

From: Duane Wessels <[email-masked]>
Date: Thu, 3 Dec 2009 22:50:46 +0000 (UTC)

On Thu, 3 Dec 2009, Nicholas Weaver wrote:

> recursive resolver service that uses a 1280B EDNS MTU?  > > >

The Measurement Factory recently did a survey that measured the
advertised buffer size and the fpdns fingerprint.  For those with
1280 bytes fpdns reported one of:

    JHSOFT simple DNS plus
    Microsoft Windows DNS 2003
    No Match Found
    (timeout)

Duane W.

---

From: Joe Abley <[email-masked]>
Date: Thu, 3 Dec 2009 22:31:04 -0500

> Does anyone know of DNS resolver software other than Google's new recursive resolver service that uses a 1280B EDNS MTU?

I haven't done enough thinking about this, but perhaps 1280 bytes is a good number to choose if you want to provide DNS service over UDP/IPv6, since it avoids the need to keep state to fragment UDP packets out from the sending side (DNS server) to destinations (DNS client) whose path MTU is too small -- 1280 is required to work without fragmentation.

The reason this is more relevant to IPv6 than IPv4 is that in v4 the sending side doesn't keep state so long as the first outbound interface has a large enough MTU; the sending host expects intermediate routers to fragment if necessary.

It seems reasonable to imagine scenarios where a busy DNS server might run into resource problems if there was a large diversity of clients for whom a path MTU was required to be retained.

I have done no measurements that would tell me how well the 1280 MUST is observed in real networks.

I also don't know that Google intends to provide their DNS service over IPv6, but it seems plausible that they would keep their options open.

It has occurred to me that if this thinking has any merit, it might be worth writing up a BCP to document appropriate measures to take to avoid v6/UDP/path MTU state explosions on DNS servers.

Joe

---

From: Mark Andrews <[email-masked]>
Date: Fri, 04 Dec 2009 16:03:42 +1100

In message <[email-masked]>, Joe Abley writes
:
> > Does anyone know of DNS resolver software other than Google's new =
> I haven't done enough thinking about this, but perhaps 1280 bytes is a =
> good number to choose if you want to provide DNS service over UDP/IPv6, =
> since it avoids the need to keep state to fragment UDP packets out from =
> the sending side (DNS server) to destinations (DNS client) whose path =
> MTU is too small -- 1280 is required to work without fragmentation.
> 
> The reason this is more relevant to IPv6 than IPv4 is that in v4 the =
> sending side doesn't keep state so long as the first outbound interface =
> has a large enough MTU; the sending host expects intermediate routers to =
> fragment if necessary.
> 
> It seems reasonable to imagine scenarios where a busy DNS server might =
> run into resource problems if there was a large diversity of clients for =
> whom a path MTU was required to be retained.
> 
> I have done no measurements that would tell me how well the 1280 MUST is =
> observed in real networks.
> 
> I also don't know that Google intends to provide their DNS service over =
> IPv6, but it seems plausible that they would keep their options open.
> 
> It has occurred to me that if this thinking has any merit, it might be =
> worth writing up a BCP to document appropriate measures to take to avoid =
> v6/UDP/path MTU state explosions on DNS servers.
>
> Joe

	IPV6_USE_MIN_MTU was added to the IPv6 Sockets API over a
	decade ago so that nameservers didn't need to keep PMTU
	state for every client.  If your nameserver doesn't use it
	complain.  If your OS doesn't support it complain.

	I suspect this will have more to do with limitations of the
	firewalls Google are using than anything else.

	Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Nicholas Weaver <[email-masked]>
Date: Fri, 4 Dec 2009 05:27:20 -0800

> I haven't done enough thinking about this, but perhaps 1280 bytes is a good number to choose if you want to provide DNS service over UDP/IPv6, since it avoids the need to keep state to fragment UDP…

Except that 1280B DNS MTU != 1280B path MTU, as the DNS MTU doesn't include the UDP header.

Also, Google's server is IPv4 only currently.

---

From: Joe Abley <[email-masked]>
Date: Fri, 4 Dec 2009 09:29:47 -0500

> Except that 1280B DNS MTU != 1280B path MTU, as the DNS MTU doesn't include the UDP header.

Good point :-)

---

From: Nicholas Weaver <[email-masked]>
Date: Fri, 4 Dec 2009 07:12:58 -0800

On Dec 4, 2009, at 7:09 AM, [email-masked] wrote:

> On Fri, Dec 04, 2009 at 09:29:47AM -0500, Joe Abley wrote:
>> 
> 	hence the argument for 1220...

Even then the argument doesn't make sense.  It only makes sense to have the EDNS MTU be small enough to avoid fragmentation if

a) The network can't handle fragmentation at ALL
or
b) The stateholding requirements for fragmentation are WORSE than the stateholding requirements for TCP.

a the resolver can discover for itself and memoize.

b doesn't seem to make sense.

---

From: [email-masked]
Date: Fri, 4 Dec 2009 15:09:53 +0000

--bill

---

From: Joe Abley <[email-masked]>
Date: Fri, 4 Dec 2009 11:02:34 -0500

> a) The network can't handle fragmentation at ALL

With IPv6 the network *doesn't* handle fragmentation. The sending host does it. Intermediate routers either pass packets or drop them.

> b) The stateholding requirements for fragmentation are WORSE than the stateholding requirements for TCP.

The stateholding requirements for fragmentation are potentially much worse than those for TCP if you consider the (usual? inevitable?) case where a nameserver's performance might be limited to a thousand times more queries per second over UDP than over TCP.

For example, BIND9's default setting for the tcp-clients option is 100, I believe. It's easy to imagine retaining path MTU state for 100,000 UDP clients on a busy authority-only server (well, if you can imagine there being 100,000 resolvers in the world that are v6 capable :-)

> a the resolver can discover for itself and memoize.

The resolver MUST discover for itself and record corresponding state, with IPv6.

> b doesn't seem to make sense.

Perhaps if the socket option that Mark mentioned is widely available and widely used, there is no problem here anyway. I don't know how to tell whether that's the case though.

Joe

---

From: Nicholas Weaver <[email-masked]>
Date: Fri, 4 Dec 2009 08:06:34 -0800

This still doesn't make sense to me, even in IPv6.

In IPv6, if the response is < path MTU, no fragmentation or stateholding is required.

If the response is > path MTU but less than the EDNS MTU, you need the fragmentation related stateholding.

But if the EDNS MTU == Path MTU, now it just automatically falls over to TCP AFTER getting a truncated response.  So you STILL have the big stateholding problem (probably worse, given its the full TCP stack), PLUS additional latency because you first have the failover to TCP.

---

From: Florian Weimer <[email-masked]>
Date: Fri, 04 Dec 2009 16:37:34 +0000

> recursive resolver service that uses a 1280B EDNS MTU?

Recent versions of the PowerDNS recursor, if I recall correctly.

-- 
Florian Weimer                <[email-masked]>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstraße 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99

---

From: Paul Vixie <[email-masked]>
Date: Fri, 04 Dec 2009 17:16:21 +0000

> From: Nicholas Weaver <[email-masked]>
> Date: Fri, 4 Dec 2009 07:12:58 -0800
> 
> Even then the argument doesn't make sense.  It only makes sense to have
> the EDNS MTU be small enough to avoid fragmentation if
> 

+1.

new recommended reading on this topic:

"Improving TCP Security with Robust Cookies by Perry Metzger,
William Allen Simpson, and Paul Vixie"

<http://www.usenix.org/publications/login/2009-12/openpdfs/metzger.pdf>

---

From: Florian Weimer <[email-masked]>
Date: Mon, 07 Dec 2009 08:03:27 +0000

> b doesn't seem to make sense.

Typical load balancers apparently don't do UDP properly, breaking
PMTUD if it's active on the nodes.

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 7 Dec 2009 05:25:58 -0800

> PMTUD if it's active on the nodes.

This would be 
a:  "Fragmentation is broken", which IMO, can and should be handled automatically on a resolver by resolver basis, and should not instead be handled by a dumb default (which, incidentally, won't guarentee lack of fragmentation even on IPv6).

---

From: Douglas Otis <[email-masked]>
Date: Mon, 7 Dec 2009 17:43:45 -0800

>>> 	hence the argument for 1220...

1280 should be specified as the desired PMTU instead.

> "Improving TCP Security with Robust Cookies by Perry Metzger, William Allen Simpson, and Paul Vixie"
> 

Paul,

Thanks for posting the link.  I will be waiting for William Simpson's I-D. 

This paper alludes to deployment issues for SCTP, but then references draft-karn-photuris-01.txt, which is not about these issues.  IMHO, SCTP deployment issues can be avoided with reasonable practices and avoiding multi-homing in some cases.  The TCPCT paper also suggests the SCTP's cookie exchange represents an amplification attack concern, which again alludes to multi-homing extremes.  In practice, there are many exchanges having greater amplifications, where this concern seems unwarranted, especially when compared to EDNS0/UDP typical DNSSEC exchanges.  With respect to new transport concerns, SCTP can also be carried over UDP.

I talked to Joe Touch about TCP's deplorable undetected error rate for bit specific errors (~2%).  Joe's response was that TCP is not responsible for detecting these types of errors, although they are common and are beyond the control of Internet users.  While yet-another-option could also bolster TCP's inadequate checksum algorithm, but it took SCTP nearly 7 years to obtain hardware support for the superior error detection algorithm, which is offered as an option in iSCSI.  When error detection boundaries do not correspond with acknowledgement boundaries, the connection state gets ugly. 

I also assume TCPCT will not offer a different transport designation with which to differentiate clients.  Simultaneous support of TCP and TCPCT will incur the same TCP TIME-WAIT constraints needed to avoid duplicate segment (data or ACKs) from arriving in TCP SYN-SENT state.   One would then assume TCP will be ignored beyond some level of resource consumption.

IMHO, progress is not well served by adding another option to TCP, rather than promoting a fully engineered solution that is better at retaining data integrity and framing, while also ensuring the state of the transaction remains known.  Rapidly discarding TCP connections to limit TCP related state during packet loss may lead to instability and collapse.  Perhaps Java will soon support SCTP, where Google-like  approaches to DNS could be well served with SCTP as the transport.  Keep in mind, the CPE equipment declared as problematic for SCTP, is creating similar problems for DNS over TCP.  A solution needling less resources more suitable for CPE equipment could be obtained by using unordered, unreliable SCTP.

-Doug

---

From: [email-masked]
Date: Tue, 8 Dec 2009 15:41:47 +0000

> 1280 should be specified as the desired PMTU instead.

	not so much - not as long as VPNs exist for moving IPv6 packets around.

--bill

---

From: Joe Abley <[email-masked]>
Date: Tue, 8 Dec 2009 16:14:34 +0000

> 	not so much - not as long as VPNs exist for moving IPv6 packets around.

I've forgotten to ask for clarification on this when I've seen you ask it before.

My understanding of the standard is that 1280 is the minimum link MTU for any interface with a v6 address.

My further understanding is that this minimum applies regardless of whether the interface is a physical interface like an ethernet port, or a logical interface whose frames will be encapsulated (802.1q, PPPoE, GRE, whatever).

A VPN which moves IPv6 packets around must be capable of moving around a 1280 byte packet, or it's failing to comply with the standard.

Are you suggesting that my understanding of the standard (per above) is wrong?

Or are you rather suggesting that this aspect of the standard is so widely ignored that it cannot be relied upon in practice?

Joe

---

From: [email-masked]
Date: Tue, 8 Dec 2009 18:52:02 +0000

> Or are you rather suggesting that this aspect of the standard is so widely ignored that it cannot be relied upon in practice?

	i can not suggest anything about your undertanding of the spec or your interpretation
	of same.

	I can say that in my limited (12years) experience with IPv6, generally in tunnels of
	some form or other, than 1220 has been the common path MTU - with the e2e MTU being set
	to 1280 - and hte link layer doing the fragmentation/reassembly.

	YMMV of course.  

--bill

---

From: Paul Vixie <[email-masked]>
Date: Tue, 08 Dec 2009 18:57:49 +0000

> Date: Tue, 8 Dec 2009 18:52:02 +0000
> From: [email-masked]
> 
> 	I can say that in my limited (12years) experience with IPv6,
> 	generally in tunnels of some form or other, than 1220 has been the
> 	common path MTU - with the e2e MTU being set to 1280 - and hte link
> 	layer doing the fragmentation/reassembly.

i think that rfc limits should be based on theory and validated by practice,
but not based on practice directly.

---

From: Joe Abley <[email-masked]>
Date: Tue, 8 Dec 2009 19:17:26 +0000

> 	to 1280 - and hte link layer doing the fragmentation/reassembly.

By "e2e MTU" you mean "MTU" as far as the IPv6 interface is concerned, right? So you're talking about a 1280 byte IPv6 MTU, not a 1220 byte IPv6 MTU?

Joe

---

From: William Allen Simpson <[email-masked]>
Date: Tue, 08 Dec 2009 14:31:09 -0500

> 1280 should be specified as the desired PMTU instead.
Please show your work.

>> <http://www.usenix.org/publications/login/2009-12/openpdfs/metzger.pdf>
There's a problem with your MUA, it sends very long lines without folding.
I'll respond to the initial 80 bytes.

> This paper alludes to deployment issues for SCTP, but then references
> 
Thank you for noticing this trivial error that passed by 3 levels of editor.
The correct reference is [14] (not [15]).

> I talked to Joe Touch about TCP's deplorable undetected error rate for bit 
> 
TCP is not a link protocol, and *bit* error rates are not within the scope
of the specification.  That's failed (TPC-style) thinking.  For example,
Internauts run TCP over IP over PPP over X.25 -- we don't rely on the
transport layer for hop by hop link error detection.

TCP has a good transport error detection rate.  There are a number of papers
on this topic with regard to the flawed (TPC) ATM slicing and dicing.  We've
long had an option to negotiate other checksums for specialty environments.

  * NB: TPC is shorthand for "The Phone Company".  It's a movie reference.
    SCTP is a TPC product, with all the usual TPC failings.

> I also assume TCPCT will not offer a different transport designation with
> 
I have no idea what this means.  This is about a TCP option.  Perhaps you
haven't read *all* the references?

> IMHO, progress is not well served by adding another option to TCP, rather
> 
Admittedly, lately I've been ignoring your militant SCTP advocacy.  There is
no reason to expect that SCTP will pass through the 1999 Apple Airport, the
2004 Airport Express, or even this year's 2009 Airport Extreme Dual Band, as
well as Linksys BEFSR41 routers.  Thus far, TCPCT has been tested passing
through these successfully.

This *is* "a fully engineered solution"....

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 09 Dec 2009 08:23:50 +0900

[email-masked] wrote:

> i can not suggest anything about your undertanding of the
> spec or your interpretation of same.

Historically, 1280B of IPv6 minimum MTU was chosen considering 1500B
MTU of Ethernet and doubly nested IPsec tunneling. I was at an IPng
WG meeting when Seteve Deering stated so.

However, it was also assumed that PMTUD had worked. That is, it was
expected that in an exceptional case of PMTU of a tunnel is 1280B or
so, link fragmentation within the tunnel could be used. RFC2473 says:

   Prepending a tunnel header increases the size of a packet, therefore
   a tunnel packet resulting from the encapsulation of an IPv6 original
   packet may require fragmentation.

So far, it is a pure fantasy of IPv6.

The reality, however, is that PMTUD does not always work and
tunnel MTU must be assumed to be just 1280B, which means MTU of
tunnel payload should be smaller than 1280B or fragmentation
is used within the tunnel. Thanks to poor specificaiton of
ICMPv6, ISPs have to filter some of ICMPv6 messages, which means
many ISPs will filter all of them.

Note that, in the RFC2473, wisdom of IPv4 [RFC791]:

>      In the above procedure each fragment (except the last) was made
>      the maximum allowable size.  An alternative might produce less
>      than the maximum size datagrams.  For example, one could implement
>      a fragmentation procedure that repeatly divided large datagrams in
>      half until the resulting fragments were less than the maximum
>      transmission unit size.

is forgotten, which causes a lot of unnecessary fragmentation.

Finally, note that 1280B of MTU does not guarantee 1024B message size
without fragmentation, not even 512B, because IPv6 header option can
be infinitely lengthy.

The question, then, is "Do you still want to use IPv6?" and my answer
is "No, I don't".

						Masataka Ohta

PS

Of course, there is no point of using DNSSEC, partly because, only a
single key (256B for 2048bit key) needs to be carried in a message
even during key role over if DNS extension for public key cryptography
was properly specified and partly because PKI is merely weakly secure.

---

From: Douglas Otis <[email-masked]>
Date: Tue, 8 Dec 2009 16:58:15 -0800

>> 1280 should be specified as the desired PMTU instead.

See RFC2460 Section 5.
(dealing with IPv6/1280 -> IPv4/ <1280)
 ,---
 In response to an IPv6 packet that is sent to an IPv4 destination
 (i.e., a packet that undergoes translation from IPv6 to IPv4), the
 originating IPv6 node may receive an ICMP Packet Too Big message
 reporting a Next-Hop MTU less than 1280.  In that case, the IPv6 node
 is not required to reduce the size of subsequent packets to less than
 1280, but must include a Fragment header in those packets so that the
 IPv6-to-IPv4 translating router can obtain a suitable Identification
 value to use in resulting IPv4 fragments.  Note that this means the
 payload may have to be reduced to 1232 octets (1280 minus 40 for the
 IPv6 header and 8 for the Fragment header), and smaller still if
 additional extension headers are used.
'---
Additional headers will impact a maximal DNS message size, which
could guide a recommended DNS message size fallback of say
1232 or less, depending upon other extension headers being used.  

> The correct reference is [14] (not [15]).

Was the concern in regard to dealing with IP address reassignments within
a mobile environment?

> transport layer for hop by hop link error detection.

You cut this at bit specific errors.  Don't blame the telcos. The concern was 
not about bit error rates, but was in regard to common error sources not
detected by TCP's checksum algorithm.  This algorithm is highly susceptible
to not detecting common errors occurring on the same bus-specific bit
location.  Likely sources could be due to weak drivers within parallel
interfaces for memory or network devices found within typical Internet
related components.  

In addition, the Ethernet link error detection rate degrades when applied
over jumbo frames. The SCTP polynomial restores error detection rates 
for jumbo frames as well.  As more data is exchange and stored over the
Internet, error detection becomes increasingly  important.  An algorithm 
that fails to detect 1 in 50 common bus related errors should not be 
considered acceptable, and not everything within a DNSSEC message
will have been signed either.  
  
>   SCTP is a TPC product, with all the usual TPC failings.

SCTP is an IETF product designed to meet reliability and DDoS requirements
necessary to support telco related applications as well as others.  SCTP
originally was intended to be carried over UDP, but was given its own transport
designation by the IETF to help in breaking out of the TCP/UDP box.  SCTP
includes modes that benefit applications like DNS as well.

> haven't read *all* the references?

An option working with legacy equipment is good.  The concern was about 
mixed TCP and TCPCT support.  The intended use of this extension is to
allow rapid dropping of TCP connections without enduring a TIME-WAIT
period (and likely not always responding to retries.)  The eventual behavior
provided has not been fully realized or considered.  However, it seems 
unlikely to improve data integrity or stability compared against SCTP.

As a side note:

Advocating for SCTP should not cause one to be described as militant.
Review data presented by Jonathan Stone and Craig Partridge regarding
the high number of undetected Internet errors titled "When The CRC and
TCP Checksum Disagree" available at portal.acm.org.  

-Doug

---

From: Joe Abley <[email-masked]>
Date: Wed, 9 Dec 2009 09:23:01 +0000

On 2009-12-09, at 00:58, Douglas Otis wrote:

> value to use in resulting IPv4 fragments.

"IPv6-to-IPv4 translating router" is fiction though, right?

---

From: Stefan Schmidt <[email-masked]>
Date: Wed, 9 Dec 2009 11:52:22 +0100

> "IPv6-to-IPv4 translating router" is fiction though, right?

I fear it is not.
-> http://tools.ietf.org/html/draft-ietf-softwire-dual-stack-lite-02
-> http://www.techworld.com.au/article/253932/comcast_pitches_ipv6_strategy_standards_body
-> http://gogoware.gogo6.com/4105/description.asp?product_id=178

	Stefan
-- 
There is hopeful symbolism in the fact that flags don't wave in a vacuum.
- Arthur C. Clark

---

From: [email-masked]
Date: Wed, 9 Dec 2009 11:56:42 +0000

> "IPv6-to-IPv4 translating router" is fiction though, right?

	nope - see IVI

--bill