Subject: Re: Reopening jumbo frames in IS-IS

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Jul 2005 09:53:41 -0400

Sorry for the late response.  Been away from email.

In message <015a01c57a75$4e1e7b70$6801a8c0@stephen>
"Stephen Sprunk" writes:
>  
> Thus spake "Curtis Villamizar" <[email-masked]>
> > It probably would not hurt to have an informational draft describing
> > jumbo frames, the reason for needing a them, when to use them and
> > importantly when not to use them.
> >
> > Jumbo frames exists only to avoid the performance drop that would
> > otherwise potentially cripple core networks using GbE or 10GbE
> > and aggregating higher MTU links.  Since jumbo frames could
> > otherwise be harmful, it might make sense to ask that equipment be
> > capable of forwarding jumbo frames but originate standard MTU
> > frames (except when verifying MTU such as in ISIS) and that end
> > user devices such as NIC cards never originate jumbo frames.
>  
> That seems excessively restrictive.  There are many environments today using
> jumbo frames (8k-10k MTU) on end hosts without problems.
>  
> This also begs for a definition of what exactly a "jumbo" frame is; does an
> MPLS label (or several) on the front of a 1500-byte Ethernet frame count?
> Does a packet on media with a >1500 byte native MTU count?  Should a host on
> FDDI or a direct HDLC/PPP link be limited to 1500 byte packets -- even
> though its native MTU is 4470 -- because we fear it might need to talk to an
> Ethernet-connected host?

Jumbo frames specificly refers to ethernet.  FDDI, HIPPI, HDLC, PPP,
etc are not affected.  It was these interfaces at end systems with
100baseT, GbE, and 10GbE in provider POPs that created the requirement
for ethernet jumbo frames.

No one in their right mind (note that I this is a qualified "no one")
would bridge a PPP link to an ethernet link.  Standards should
discourage doing incredibly stupid things rather than try to
accomodate people who do them with no good reason for doing so.

TCP and most other modern protocols handle the end system MTU quite
nicely and TCP path MTU discovery is widely implemented and easy
enough to enable though I think it is still disabled by default in
most implementations.

> > It should also be clearly stated that jumbo frames be disabled by
> > default and any equipment supporting them at the very least put
> > harsh warnings in the equipment's user documentation about the
> > feature being non-standard and has a potential to cause
> > interoperability problem if just casually enabled.
>  
> I certainly agree that jumbo frames should be disabled by default, and the
> warning should explicitly note that severe problems will result if hosts on
> the same subnet have different ideas of what the MTU is.  I personally have
> never seen a problem with jumbos when every device on a subnet supports them
> and is configured with the same MTU, but perhaps someone else can offer some
> enlightening anecdotes.
>  
> We might consider adding a means for hosts configured with a "jumbo" MTU to
> determine if all of its neighbors (and intermediate L2 devices) can indeed
> handle such packets.  If such a mechansim existed, we could allow jumbos to
> be on by default since hosts could detect when it was necessary to fall back
> to a non-jumbo MTU.
>  
> S

If used in a SP network, the SP generally knows what is connected much
more so than in an enterprise environment where anyone could connect
anything.

> Stephen Sprunk      "Those people who think they know everything
> CCIE #3723         are a great annoyance to those of us who do."
> K5SSS                                             --Isaac Asimov

Curtis

_______________________________________________
routing-discussion mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/routing-discussion

---

From: "Stephen Sprunk" <[email-masked]>
Date: Fri, 15 Jul 2005 12:59:35 -0500

>> That seems excessively restrictive.  There are many environments
>> today using jumbo frames (8k-10k MTU) on end hosts without
>> problems.
>>
>> This also begs for a definition of what exactly a "jumbo" frame is;
>> does an MPLS label (or several) on the front of a 1500-byte Ethernet
>> frame count?  Does a packet on media with a >1500 byte native
>> MTU count?  Should a host on FDDI or a direct HDLC/PPP link be
>> limited to 1500 byte packets -- even though its native MTU is 4470
>> -- because we fear it might need to talk to an Ethernet-connected
>> host?
>
> 100baseT, GbE, and 10GbE in provider POPs that created the
> requirement for ethernet jumbo frames.

Only as it relates to IS-IS; jumbos existed long before in other contexts 
despite the IEEE's efforts to the contrary.

> accomodate people who do them with no good reason for doing so.

Then a large number of ISPs are out of their minds...  The two main ways to 
deploy ADSL service are (a) Ethernet bridged to PPPoATM, and (b) PPPoE 
directly to the host.  I think the latter is also commonly used for DOCSIS.

If bridging POS and Ethernet were safer (i.e. jumbos could be assumed), we'd 
probably see that become common in ISPs as well.

> enough to enable though I think it is still disabled by default in

Win2k and later have PMTUD on by default, but unfortunately not PMTU black 
hole detection.  I'm not sure about earlier Windows versions or 
MacOS/Linux/etc.

However, there's no current means for MTU detection for hosts on the same 
subnet.  I can envision a number of ways to fix this for IP traffic, but I 
don't know enough about IS-IS to know if any of them are applicable.

S

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Jul 2005 17:01:38 -0400

In message <023c01c5896a$0ede1020$6501a8c0@dax>
> directly to the host.  I think the latter is also commonly used for DOCSIS.

Agreed.  :-)

These were more the choices of DSLAM vendors and to a less extent the
RBOCs/ILECs who were latecomers.  The traditional ISPs at the time
preferred that the DSLAMs just do IP over PPP on the customer router
or host and IP over PPP or ATM on the backhaul side.  The DSLAM
vendors thought that by matching the ATM parameters to the DSL rate
they could all but eliminate buffering in the DSLAM.

> probably see that become common in ISPs as well.

Has it become that bad already? :-)

Who was it that said the Internet is growing exponentially but the
amount of clue is growing linearly at best so the clue density is
shrinking exponentially?  I think it was Randy Bush in the early 90s.

> CCIE #3723         are a great annoyance to those of us who do."

ISIS sends hellos at full MTU.  If the other end can't handle them,
then no neighbor adjacency comes up.  There is no L2 DF bit and ICMP
would fragment reply as there is in IP so TCP doesn't handle the MTU
mismatch on the same subnet.  Something like that (an L2 DF and ICMP)
would be needed to handle two devices with an MTU mismatch on the same
subnet or some probe packet of the type ISIS uses.  The diagnostic
tool available to today's operator is "ping".  Use the -s option.

Until something changes there is no automatic means to detect and
correct an MTU mismatch on the same subnet.  If running ISIS there is
the means to detect the error but I don't think there is an automatic
correction (no MTU fallback).  I'd have to check the expired draft.

Curtis

---

From: Tony Li <[email-masked]>
Date: Fri, 15 Jul 2005 14:39:01 -0700

> correction (no MTU fallback).  I'd have to check the expired draft.

There is currently no such mechanism.

Note that correction between routers is less of an issue.  If all end
systems on the (V)LAN are not agreed on the MTU, then there is a more
general problem.  If all end systems are fixed, then presumably all
routers are as well.

Tony