Subject: [Int-area] Alissa Cooper's No Objection on draft-ietf-intarea-frag-fragile-16: (with COMMENT)

From: Alissa Cooper via Datatracker <[email-masked]>
Date: Tue, 03 Sep 2019 05:59:45 -0700

Alissa Cooper has entered the following ballot position for
draft-ietf-intarea-frag-fragile-16: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-intarea-frag-fragile/

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Thanks for addressing my DISCUSS.

---

From: Joe Touch <[email-masked]>
Date: Tue, 3 Sep 2019 07:06:20 -0700

Hi, all,

So let me see if I understand:

Alissa issues a comment.

We discuss this on the list and come to a rare consensus on a way forward.

The new draft is issued that:

a) ignores the list consensus
b) removes a paragraph not under the DISCUSS (1.1)
c) now refers to vague “other documents” without citation
d) most importantly:

	REMOVES a key recommendation that we MAY use frag where it works
		
	Asserts the false claim that IP fragmentation “will fail” in the Internet,
	despite citing evidence that the *majority of the time* it does work
		e.g., for IPv6, sec 3.9

What happened? Why is a change this substantial not reflecting the *list consensus*?

Joe

> On Sep 3, 2019, at 5:59 AM, Alissa Cooper via Datatracker <[email-masked]> wrote:
> 
> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Tue, 3 Sep 2019 14:33:33 +0000

Why was this section taken out:
 
> 1.1.  IP-in-IP Tunnels 
>     
>    This document acknowledges that in some cases, packets must be     
>    fragmented within IP-in-IP tunnels [I-D.ietf-intarea-tunnels].     
>    Therefore, this document makes no additional recommendations     
>    regarding IP-in-IP tunnels.

Tunnels always inflate the size of packets to the point that they may exceed
the path MTU even if the original packet is no larger than the path MTU. And,
for IPv6 the only guarantee is 1280. Therefore, in order to robustly support
the minimum IPv6 MTU tunnels MUST employ fragmentation.

Please put this section of text back in the document where it belongs.

Thanks - Fred

> -----Original Message-----
> From: Int-area [mailto:[email-masked]] On Behalf Of Joe Touch
> Sent: Tuesday, September 03, 2019 7:06 AM
> To: Alissa Cooper <[email-masked]>
> Cc: Joel Halpern <[email-masked]>; [email-masked]; [email-masked]; The IESG <[email-masked]>;
> [email-masked]
> Subject: Re: [Int-area] Alissa Cooper's No Objection on draft-ietf-intarea-frag-fragile-16: (with COMMENT)
> 
> Hi, all,
>

---

From: Fernando Gont <[email-masked]>
Date: Tue, 3 Sep 2019 17:49:53 +0300

> the minimum IPv6 MTU tunnels MUST employ fragmentation.

Isn't that an oxymoron? If fragmentation is fragile, if you need
something robust, you need to rely on something else....

-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Tom Herbert <[email-masked]>
Date: Tue, 3 Sep 2019 08:39:11 -0700

> something robust, you need to rely on something else....

Not really, to say fragmentation is fragile is a subjective statement, not
a quantifiable fact. As discussed on the list, fragmentation is
productively in use in many networks that employ tunneling. In those cases
fragmentation is not fragile and so there's nothing to fix.

I don't understand this change nor the others in the latest draft either.

Tom

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Tue, 3 Sep 2019 15:41:35 +0000

> From: Fernando Gont [mailto:[email-masked]]
> Sent: Tuesday, September 03, 2019 7:50 AM
> To: Templin (US), Fred L <[email-masked]>; Joe Touch <[email-masked]>; Alissa Cooper <[email-masked]>

IPv6 fragmentation is not fragile - only IPv4 fragmentation is fragile.

---

From: Fernando Gont <[email-masked]>
Date: Tue, 3 Sep 2019 18:53:54 +0300

On 3/9/19 18:39, Tom Herbert wrote:
> 
> 
> On Tue, Sep 3, 2019, 8:31 AM Fernando Gont <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
> Not really, to say fragmentation is fragile is a subjective statement,
> not a quantifiable fact.

RFC7872. IIRC, Geoff did his own independent measurements.

> As discussed on the list, fragmentation is
> productively in use in many networks that employ tunneling. In those
> cases fragmentation is not fragile and so there's nothing to fix.

The measurements I've seen about fragmentation on the big-I internet
seem to indicate that it is fragile. Yes, you can always build your own
private network where it is not.

---

From: Fernando Gont <[email-masked]>
Date: Tue, 3 Sep 2019 18:57:24 +0300

> IPv6 fragmentation is not fragile - only IPv4 fragmentation is fragile.

* RFC7872.
* https://blog.apnic.net/2017/08/22/dealing-ipv6-fragmentation-dns/

These seems pretty fragile to me. YMMV, though.

Thanks,

---

From: Tom Herbert <[email-masked]>
Date: Tue, 3 Sep 2019 09:07:15 -0700

On Tue, Sep 3, 2019, 8:58 AM Fernando Gont <[email-masked]> wrote:

> >> To: Templin (US), Fred L <[email-masked]>; Joe Touch <
> [email-masked]>; Alissa Cooper <[email-masked]>
> >> Cc: Joel Halpern <[email-masked]>;
> [email-masked]; [email-masked]; The IESG <
> [email-masked]>;
> >> [email-masked]
> >> Subject: Re: [Int-area] Alissa Cooper's No Objection on
> draft-ietf-intarea-frag-fragile-16: (with COMMENT)
> >>

Fernando,

That's fragmentation in DNS, not tunneling which is what Fred's question is
about. Besides that, Fred's point is still correct-- fragmentation is not
fragile for IPv6, it's broken, non-conformant implementations that are
fragile.

Tom

---

From: Bob Hinden <[email-masked]>
Date: Tue, 3 Sep 2019 09:09:55 -0700

Fred,

> On Sep 3, 2019, at 7:33 AM, Templin (US), Fred L <[email-masked]> wrote:
> 

This text in the Introduction was removed because, as noted in Warren Kumari
Comment (2019-08-07 for -15), this didn’t need to be in the introduction, and it didn’t say very much that isn’t described later in the document.

The normative text in Section 5.3. "Packet-in-Packet Encapsulations” is unchanged.  I think Section 5.3 covers the topic.  It includes the reference to [I-D.ietf-intarea-tunnels].

Bob

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Tue, 3 Sep 2019 16:19:29 +0000

Fernando, RFC7872 seems to document the state of the Internet at a given point
of time. I do not know if things have gotten better or worse for extension headers
in the Internet since that time, but I believe that support for extension headers in
the Internet must become more robust. Fragmentation is one reason. Another is
the proposed MTU extension header, and there are still others. Let's not ossify
the IPv6 Internet yet the way the IPv4 Internet is ossified.

Fred

> Sent: Tuesday, September 03, 2019 8:57 AM

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Tue, 3 Sep 2019 19:45:36 +0000

Bob,

> From: Bob Hinden [mailto:[email-masked]]
> Sent: Tuesday, September 03, 2019 9:10 AM
> To: Templin (US), Fred L <[email-masked]>
> Cc: Bob Hinden <[email-masked]>; Joe Touch <[email-masked]>; Alissa Cooper <[email-masked]>; Joel Halpern
> <[email-masked]>; [email-masked]; [email-masked]; IESG <[email-masked]>; intarea-
> [email-masked]

While I agree that both passages supply a working vector to 'intarea-tunnels',
the two strike very different tones. The former gives a balanced citation, while
the latter calls it a "corner case" - twice!

Whether we like it or not, fragmentation and encapsulation will continue to
be associated with each other no matter what gets documented here. So,
a respectful handoff to 'intarea-tunnels' would be appreciated.

Fred

---

From: Bob Hinden <[email-masked]>
Date: Tue, 3 Sep 2019 13:00:57 -0700

Fred,

> On Sep 3, 2019, at 12:45 PM, Templin (US), Fred L <[email-masked]> wrote:
> 
> Bob,
> 

You are talking about text in the Introduction of the document.

The important substance relating to tunnels is in Section 5.3.   The text is:

   5.3.  Packet-in-Packet Encapsulations

   In this document, packet-in-packet encapsulations include IP-in-IP
   [RFC2003], Generic Routing Encapsulation (GRE) [RFC2784], GRE-in-UDP
   [RFC8086] and Generic Packet Tunneling in IPv6 [RFC2473].  [RFC4459]
   describes fragmentation issues associated with all of the above-
   mentioned encapsulations.

   The fragmentation strategy described for GRE in [RFC7588] has been
   deployed for all of the above-mentioned encapsulations.  This
   strategy does not rely on IP fragmentation except in one corner case.
   (see Section 3.3.2.2 of RFC 7588 and Section 7.1 of RFC 2473).
   Section 3.3 of [RFC7676] further describes this corner case.

   See [I-D.ietf-intarea-tunnels] for further discussion.

Seems fine to me, in tone and substance.

Bob

---

From: Tom Herbert <[email-masked]>
Date: Tue, 3 Sep 2019 13:33:54 -0700

Bob,

I agree with Fred. Note, the very first line of the introduction:

"Operational experience [Kent] [Huston] [RFC7872] reveals that IP
fragmentation introduces fragility to Internet communication".

This attempts to frame fragmentation as being generally fragile with
supporting references. However, there was much discussion on the list
about operational experience that demonstrates fragmentation is not
fragile. In particular, we know that fragmentation with tunnels is
productively deployed and has been for quite some time. So that is the
counter argument to the general statement that fragmentation is
fragile. With the text about tunneling included in the introduction I
believe that was sufficient balance of the arguments, but without the
text the reader could be led to believe that fragmentation is fragile
for everyone all the time which is simply not true and would be
misleading.

Speaking of balance, the introduction also mentions that:

"this document recommends that upper-layer protocols address the
problem of fragmentation at their layer"

But the "problem" of fragmentation is in intermediate devices that
don't properly handle it as the draft highlights. So it seems like
part of addressing the problem should also be to fix the problem! That
is implementations should be fixed to deal with fragmentation. IMO,
this should be another high level recommendation that is mentioned in
the introduction.

Tom

Tom

---

From: Fernando Gont <[email-masked]>
Date: Tue, 3 Sep 2019 23:49:02 +0300

> about operational experience that demonstrates fragmentation is not

Discussion is not measurements. Do you have measurements that suggest
otherwise?

We did separate measurements, with different methodologies, and they
suggest the same thing. You can discuss as much as you want. But that
will not make fragmentation work.

> In particular, we know that fragmentation with tunnels is

"fragile" means that it fails in an uncceptably large number of cases.
~30 failure rate is not acceptable. ~20% isn't, either.

> is implementations should be fixed to deal with fragmentation.

The same logic would solve the problem of widespread famine, and others.
I don't think that logic has solved any real problems in the real world.

---

From: Bob Hinden <[email-masked]>
Date: Tue, 3 Sep 2019 13:56:42 -0700

Tom,

> On Sep 3, 2019, at 1:33 PM, Tom Herbert <[email-masked]> wrote:
> 
> Bob,
> 
> fragmentation introduces fragility to Internet communication”.

Yes, that text in in the first paragraph of the Introduction
> for everyone all the time which is simply not true and would be

Yes, but we are discussing some text from the Introduction that to my read didn’t say anything useful so I removed it.  The substantive text about tunneling in in Section 3.5.  The Introduction, is just the introduction.  The text was:

Why is that more useful than what is in 3.5?  If it’s not making a recommendation, why call this out in the introduction.  There are lot of other things it doesn’t make recommendations about that aren’t in the Introduction either.

Bob

> this should be another high level recommendation that is mentioned in

I am serving as document editor.  This to my understanding has been through w.g. last call and now IESG review.

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Tue, 3 Sep 2019 21:02:27 +0000

Fernando,

> Sent: Tuesday, September 03, 2019 1:49 PM
> To: Tom Herbert <[email-masked]>; Bob Hinden <[email-masked]>
> Cc: Templin (US), Fred L <[email-masked]>; [email-masked]; IESG <[email-masked]>; Joel Halpern
> <[email-masked]>; [email-masked]; [email-masked]

What if we fragment the payload packet instead of the delivery packet?
Wouldn't that give a 0% failure rate?

Fred

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Tue, 3 Sep 2019 21:10:26 +0000

Bob,

> Sent: Tuesday, September 03, 2019 1:57 PM
> To: Tom Herbert <[email-masked]>
> Cc: Bob Hinden <[email-masked]>; Templin (US), Fred L <[email-masked]>; [email-masked]; IESG
> <[email-masked]>; Joel Halpern <[email-masked]>; [email-masked]; [email-masked]
> Yes, but we are discussing some text from the Introduction that to my read didn’t say anything useful so I removed it.  The substantive
> text about tunneling in in Section 3.5.  The Introduction, is just the introduction.  The text was:
> 

Yes - good text that should be retained.

> Why is that more useful than what is in 3.5? If it’s not making a recommendation, why call this out in the introduction.  There are lot of
> other things it doesn’t make recommendations about that aren’t in the Introduction either.

Because it sets a more appropriate tone and lets the reader know from the onset that
fragmentation and encapsulation go hand in hand. And tunnel fragmentation avoids the
issues raised by others in this thread.

Thanks - Fred

> >>>> Comment (2019-08-07 for -15), this didn’t need to be in the introduction, and it didn’t say very much that isn’t described later in
> the
> >>>> document.
> >>>>
> >>>> The normative text in Section 5.3. "Packet-in-Packet Encapsulations” is unchanged.  I think Section 5.3 covers the topic.  It
> includes the
> >>>> reference to [I-D.ietf-intarea-tunnels].
> >>>

---

From: Joe Touch <[email-masked]>
Date: Tue, 3 Sep 2019 14:15:29 -0700

Remember that IPsec uses IP in IP too. Shim layers won’t help it.  

Joe

> On Sep 3, 2019, at 2:10 PM, Templin (US), Fred L <[email-masked]> wrote:
> 
> Bob,
>

---

From: Ole Troan <[email-masked]>
Date: Tue, 3 Sep 2019 23:22:00 +0200

Fred,

> issues raised by others in this thread.

While inner fragmentation ensures the fragment will reach the tunnel tail end, a tunnel endpoint will typically not reassemble that fragment, so will generate fragments after the tunnel hop.
Inner fragmentation is only available on IPv4.
Outer fragmentation will look like any other fragmented packet, albeit that the tunnel tail now has to reassemble. At speeds typically much higher than a typical end host.
Tunnels within a controlled domain may use fragmentation, although it still will have problems.
Which is why you see most tunnel specifications for controlled domains, state that the network MTU must be "well managed".

In summary, I don't think the text can say very much more than what it already does.

Cheers,
Ole

---

From: Tom Herbert <[email-masked]>
Date: Tue, 3 Sep 2019 14:34:15 -0700

> ~30 failure rate is not acceptable. ~20% isn't, either.
Okay, so then the definition of a fragile protocol is one that has a
failure rate greater than 20%? If so, then that should really be
stated in the draft. But, then I'd point out that per
https://www.google.com/intl/en/ipv6/statistics.html, IPv6 is only
supported by 30% of the Internet at best, so it fails 70% of the time.
So by that same definition, isn't IPv6 fragile?

Tom

---

From: Fernando Gont <[email-masked]>
Date: Wed, 4 Sep 2019 00:44:43 +0300

> Wouldn't that give a 0% failure rate?

Sure. At which point you are using ip fragmentation in a limited domain,
and that's *not* the case this document is addressing, right?

---

From: "Black, David" <[email-masked]>
Date: Tue, 3 Sep 2019 21:58:15 +0000

> >    regarding IP-in-IP tunnels.

... for the reasons described below - e.g., "tone" matters in the introduction to this sort of BCP.

Thanks, --David

> From: Int-area <[email-masked]> On Behalf Of Templin (US), Fred
> L
> Sent: Tuesday, September 3, 2019 5:10 PM
> To: Bob Hinden; Tom Herbert
> Cc: Joel Halpern; [email-masked]; [email-masked];
> IESG; [email-masked]
> [EXTERNAL EMAIL]
> > Cc: Bob Hinden <[email-masked]>; Templin (US), Fred L
> <[email-masked]>; [email-masked]; IESG
> > <[email-masked]>; Joel Halpern <[email-masked]>; draft-ietf-
> [email-masked]; [email-masked]
> > Yes, but we are discussing some text from the Introduction that to my read
> didn’t say anything useful so I removed it.  The substantive
> > Why is that more useful than what is in 3.5? If it’s not making a
> recommendation, why call this out in the introduction.  There are lot of
> > I am serving as document editor.  This to my understanding has been
> through w.g. last call and now IESG review.
> > >
> > > Tom
> > >
> > >
> > >
> > > Tom
> > >
> > >
> > >
> > >
> > >
> > >>>> Cc: Bob Hinden <[email-masked]>; Joe Touch
> <[email-masked]>; Alissa Cooper <[email-masked]>; Joel
> > Halpern
> > >>>> <[email-masked]>; draft-ietf-intarea-frag-
> [email-masked]; [email-masked]; IESG <[email-masked]>; intarea-
> > >>>> [email-masked]
> > >>>> Comment (2019-08-07 for -15), this didn’t need to be in the
> introduction, and it didn’t say very much that isn’t described later in
> > the
> > >>>> document.
> > >>>>
> > >>>> The normative text in Section 5.3. "Packet-in-Packet Encapsulations”
> is unchanged.  I think Section 5.3 covers the topic.  It
> > includes the
> > >>>>>> Cc: Joel Halpern <[email-masked]>; draft-ietf-intarea-
> [email-masked]; [email-masked]; The IESG
> > >>>> <[email-masked]>;
> > >>>>>> [email-masked]

---

From: Bob Hinden <[email-masked]>
Date: Tue, 3 Sep 2019 17:07:57 -0700

Fred,

> issues raised by others in this thread.

I don’t know how to evaluate “tone” in an IETF specification.

How about if I move this text to section 5.3?  I think that’s better than in the Introduction.

The section would be:

   This document acknowledges that in some cases, packets must be
   fragmented within IP-in-IP tunnels.  Therefore, this document makes no
   additional recommendations regarding IP-in-IP tunnels.

Bob

---

From: Fernando Gont <[email-masked]>
Date: Wed, 4 Sep 2019 04:56:51 +0300

>> On 3/9/19 23:33, Tom Herbert wrote:
[....]
> So by that same definition, isn't IPv6 fragile?

That's of course a bad analogy. Where there's no IPv6, you just don't
use IPv6. -- IPv6 is not broken, because you simply don't use it.

IPv6 broken-ness existed when hosts would try it, but it would fail.
This was indeed the reason why content providers didn't enable v6 at the
time, and the reason why Happy Eyeballs was introduced -- to overcome
the <10% IPv6 brokenness at the time, which was unacceptable to content
providers.

What's the specific number that's unacceptable? -- I'd personally that
at the very least, anything over 10%. I wonder if you'd go on of
vacation with your car if you new it had a 30% chance of breaking apart.

That said, Bob is addressing the comments that were made on this thread,
so this discussion is rather spurious, I think

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Wed, 4 Sep 2019 13:43:44 +0000

Hi Ole,

> From: Ole Troan [mailto:[email-masked]]
> Sent: Tuesday, September 03, 2019 2:22 PM
> Cc: Bob Hinden <[email-masked]>; Tom Herbert <[email-masked]>; Joel Halpern <[email-masked]>; draft-
> [email-masked]; [email-masked]; IESG <[email-masked]>; [email-masked]
> While inner fragmentation ensures the fragment will reach the tunnel tail end, a tunnel endpoint will typically not reassemble that
> fragment, so will generate fragments after the tunnel hop.

Not true. For IPv6 packets, simply insert a GUE header or an RFC2473 header and
fragment on that. The fragments will be reassembled by the tunnel tail end, then
passed to the next hop as a whole IPv6 packet. The fragmentation footprint is
therefore the same as the tunnel footprint.

> Outer fragmentation will look like any other fragmented packet,

I am not talking about outer fragmentation.

> albeit that the tunnel tail now has to reassemble. At speeds typically
> much higher than a typical end host.

Using iperf3, I can show fragmentation and reassembly at near line-rate on 10Gbps
Ethernet gear. That seems pretty good to me. Which shows that implementers
have taken IP fragmentation seriously and put in the hard work necessary to
optimize the performance.

> Which is why you see most tunnel specifications for controlled domains, state that the network MTU must be "well managed".

We should be able to tunnel within any domain, be it controlled or over the open Internet.
Inner fragmentation (with nested encapsulation if necessary) accomplishes that.

Thanks - Fred
[email-masked]

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Wed, 4 Sep 2019 13:46:00 +0000

Hi Fernando,

> Sent: Tuesday, September 03, 2019 2:45 PM
> To: Templin (US), Fred L <[email-masked]>; Tom Herbert <[email-masked]>; Bob Hinden
> <[email-masked]>
> Cc: [email-masked]; IESG <[email-masked]>; Joel Halpern <[email-masked]>; [email-masked];
> [email-masked]

As I just answered to Ole, it is not only for limited domains but also for over
the open Internet. The fragmentation footprint is the same as the tunnel
footprint.

Fred

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Wed, 4 Sep 2019 14:23:29 +0000

Bob,

> Sent: Tuesday, September 03, 2019 5:08 PM
> Cc: Bob Hinden <[email-masked]>; Tom Herbert <[email-masked]>; [email-masked]; IESG <[email-masked]>; Joel
> Halpern <[email-masked]>; [email-masked]; [email-masked]

Paragraph #1 beginning "This document acknowledges" looks good, but then
why include paragraphs #2 and #3 since 'intarea-tunnels' is the place to discuss
IP-in-IP encapsulation. So, why not shorten Section 5.3 and have it as simply:

Fred

---

From: Ole Troan <[email-masked]>
Date: Wed, 4 Sep 2019 16:37:08 +0200

Fred,

I removed the IESG from this list, as we seem to have drifted into a more general fragmentation discussion as opposed to discussing the exact changes to this draft.

> therefore the same as the tunnel footprint.

Is that not the exact definition of outer fragmentation?

> I am not talking about outer fragmentation.

Ehm?

> optimize the performance.

On what hardware?
10G is not at all very much, and given fragmentation you have large packets anyway.
You need to compare the forwarding performance unfragmented versus fragmented.
Then impact of out of order fragments. Fragment chains that don't reassemble, etc etc.
Proving that it performs (or not as I would claim) does require quite a bit of work.

> Inner fragmentation (with nested encapsulation if necessary) accomplishes that.

Sure, you can just slap a UDP header in between. It still be outer fragmentation, but you have hidden it from the intermediate network.
Then you might as well invent your own shim tunnel fragment header, and we could deprecate the IPv6 one. (slight smiley on that idea:)

Cheers,
Ole

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Wed, 4 Sep 2019 15:26:53 +0000

Hi Ole,

> Sent: Wednesday, September 04, 2019 7:37 AM
> [email-masked]; [email-masked]
> >>>> Why is that more useful than what is in 3.5? If it’s not making a recommendation, why call this out in the introduction.  There are
> lot
> >> of

No. I am talking about outer header (OH) followed by tunnel header (TH) followed
by inner packet (IP). Recipe:

  1) wrap the IP in a TH to create a tunnel packet (TP)
  2) fragment the TP
  3) encapsulate each tunnel fragment in an independent OH
  4) send each outer packet (OP). These will look like ordinary
       unfragmented IP packets, but will contain a tunnel fragment

> > I am not talking about outer fragmentation.

Above.

> > optimize the performance.

Nothing special. I get good performance even in network emulations where the
(virtual) links do not have a maximum bandwidth restriction.

> Proving that it performs (or not as I would claim) does require quite a bit of work.

I have shown applications that get better performance by sending fragmented
large IP packets than by sending unfragmented small IP packets.

> Then you might as well invent your own shim tunnel fragment header, and we could deprecate the IPv6 one. (slight smiley on that

To make it clearer, I should be calling it tunnel fragmentation (not inner
fragmentation) - but it is clearly not outer fragmentation. All the network
sees are whole (outer) IP packets with no indication that a tunnel fragment
is being carried inside.

Fred

---

From: Bob Hinden <[email-masked]>
Date: Wed, 4 Sep 2019 08:29:20 -0700

Fred,

> On Sep 4, 2019, at 7:23 AM, Templin (US), Fred L <[email-masked]> wrote:
> 
> Bob,
> 

This text has been through IETF last call and IESG review, and as far as I can tell is factually correct.   Unless we want to repeat that, better to not remove it in my view.

I will plan to include the new first paragraph in the next version.

Bob

---

From: Ole Troan <[email-masked]>
Date: Wed, 4 Sep 2019 17:39:31 +0200

Fred,

>       unfragmented IP packets, but will contain a tunnel fragment

Right you might as well define your own tunnel fragmentation then.
And it will have a lot less overhead. Doing "segmentation" at what would be layer 4 (or 3.5) isn't what we are trying to discourage here.

Cheers,
Ole

---

From: Joe Touch <[email-masked]>
Date: Wed, 04 Sep 2019 08:43:06 -0700

On 2019-09-04 07:23, Templin (US), Fred L wrote:

> ...

There was a reason this text was included up front - despite the
implications of this draft, IP-IP tunnels are not going anywhere anytime
soon. IPsec tunnel mode has not been deprecated. 

To Bob's other point, this was approved by the WG and the IETF last call
as up front. 

Why are we changing it now? 

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 04 Sep 2019 08:45:11 -0700

On 2019-09-04 08:29, Bob Hinden wrote: 

...

> This text has been through IETF last call and IESG review, and as far as I can tell is factually correct.   Unless we want to repeat that, better to not remove it in my view.

Repeating it up front was through WG and IETF last call. 

If IESG review has a problem with a given part of the doc, it's fair
game to open up other parts to address the change. There are no truly
local changes. 

Joe

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Wed, 4 Sep 2019 15:52:00 +0000

Hi Bob,

> Sent: Wednesday, September 04, 2019 8:29 AM

The problem with traditional approaches to tunnel MTU is that they all try very
hard to avoid exceeding the MTU of the smallest link in the path between
ingress and egress. Some guess that the smallest link is 1500 minus encaps
(e.g., 1480), some get more conservative and call it 1400 "just in case", etc.
But, trying to sleuth out the size of the restricting link on the path is just
guesswork when we all know that the *only* guarantee is 1280. So, why
not cut right to the heart of the matter and proactively fragment to a
fragment size of 1280-encaps.

If you are willing to allow for fragmentation to a known safe size, then it frees
you from the paradox of how to plumb the path for the restricting link MTU
(which remember can change at any time if the path changes). It also allows
you to specify larger sizes for the tunnel MTU. For example, AERO sets the
tunnel MTU to 9180, and that size works no matter what path fluctuations
may occur thanks to tunnel fragmentation.

So, I say set the tunnel MTU to 9180 (same as ATM) and forget about the
legacy approaches that think the tunnel MTU has to be limited by the
MTU of the smallest link in the path.

Fred

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Wed, 4 Sep 2019 15:53:51 +0000

> Sent: Wednesday, September 04, 2019 8:40 AM

With GUE encapsulation and GUE extensions, it would be a special tunnel
fragmentation yes. But, with RFC2473 encapsulation, it would be good old
IPv6 fragmentation per RFC8200.

This is a valid application of IPv6 fragmentation, hence in scope for this
discussion. The only difference is that the fragments will not be seen
by the network.

Fred

---

From: Ron Bonica <[email-masked]>
Date: Wed, 4 Sep 2019 17:34:39 +0000

Folks,

We appear to be rehashing issues that have been debated passionately before. And I think that we have the following options:

1)  Continue to debate, as if we might someday converge on consensus.
2)  Agree to disagree.

If we follow the first path, we will need a continuity plan. As those who are participating in the debate age, retire, and pass on to the next life, they will need to be replaced by new participants. Otherwise, the debate will subside without concluding.

If we follow the second path, we will need to figure out what to do with this document. Options are:

- Abandon it
- Progress it as it was approved by the IESG and ignoring all contrary opinions
- Progress it as it was approved by the IESG and add an appendix recording contrary opinions and identifying them as such
- Progress it as it was approved by the IESG, deleting controversial sections and remaining silent on these issues

Does anybody see any other options?

                        Mischievously yours,
                                 Ron

Juniper Business Use Only

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Wed, 4 Sep 2019 17:48:17 +0000

Hi Ron,

No need to throw out the baby with the bathwater. Simply shorten Section 5.3 to

>   See [I-D.ietf-intarea-tunnels] for further discussion.

and then update 'intarea-tunnels' to include the new information on 9180 MTU
fragmenting tunnels that I posted in my previous list post.

But, I agree with your observation that there is tribal knowledge here contained
in a body whose average age is pushing retirement (self included) so all the more
reason to capture it now.

Fred

> From: Int-area [mailto:[email-masked]] On Behalf Of Ron Bonica
> Sent: Wednesday, September 04, 2019 10:35 AM
> To: Fernando Gont <[email-masked]>; Tom Herbert <[email-masked]>; Joe Touch <[email-masked]>
> Cc: Joel Halpern <[email-masked]>; [email-masked]; [email-masked]; IESG <[email-masked]>;
> [email-masked]
> If we follow the first path, we will need a continuity plan. As those who are participating in the debate age, retire, and pass on to the
> next life, they will need to be replaced by new participants. Otherwise, the debate will subside without concluding.
>

---

From: Fred Baker <[email-masked]>
Date: Wed, 4 Sep 2019 20:57:30 +0300

Sent using a machine that autocorrects in interesting ways...

It was taken out in response to Warren Kumari’s comment that it was out of place and already covered in a section later in the document. If it is added back in, it probably belongs in that section, not the introduction.

> On Sep 3, 2019, at 5:33 PM, Templin (US), Fred L <[email-masked]> wrote:
>

---

From: Fred Baker <[email-masked]>
Date: Wed, 4 Sep 2019 21:05:32 +0300

I did some of the edits, so I’ll respond. Alissa issues a comment. We also got several other comments over the summer. This update responds to a set of comments. I’m on vacation and mostly out of contact, so my co-authors will need to proceed as appropriate. But this was NOT a matter of one person commenting and unrelated edits happening.

For the record, I agree that fragmentation will in fact be used by applications and transports that are using it now, unless and until they are changed. Expecting otherwise is a fool’s errand. Hence, while there are known problems with it, fragmentation and reassembly MAY be used. I don’t think anyone is disputing that.

> On Sep 3, 2019, at 5:06 PM, Joe Touch <[email-masked]> wrote:
> 
> Hi, all,
>

---

From: Joe Touch <[email-masked]>
Date: Wed, 04 Sep 2019 11:12:31 -0700

On 2019-09-04 11:05, Fred Baker wrote:

> I did some of the edits, so I'll respond. Alissa issues a comment. We also got several other comments over the summer. This update responds to a set of comments. I'm on vacation and mostly out of contact, so my co-authors will need to proceed as appropriate. But this was NOT a matter of one person commenting and unrelated edits happening.
> 
> For the record, I agree that fragmentation will in fact be used by applications and transports that are using it now, unless and until they are changed. Expecting otherwise is a fool's errand. Hence, while there are known problems with it, fragmentation and reassembly MAY be used. I don't think anyone is disputing that.

And yet the text that says exactly that has been removed and replaced
with text that says exactly the opposite (that it will fail). 

So whose consensus is this currently representing? 

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 04 Sep 2019 11:13:55 -0700

IMO, it belongs back with a forward reference to the detail. 

To do otherwise buries at least part of the lead here - as you note in
another response - to not mistake this document for a step towards
deprecation. 

Joe 

On 2019-09-04 10:57, Fred Baker wrote:

> It was taken out in response to Warren Kumari's comment that it was out of place and already covered in a section later in the document. If it is added back in, it probably belongs in that section, not the introduction.
> 
> c) now refers to vague "other documents" without citation
> d) most importantly:
> 
> Asserts the false claim that IP fragmentation "will fail" in the Internet,

---

From: Fernando Gont <[email-masked]>
Date: Thu, 5 Sep 2019 01:54:32 +0300

On 4/9/19 16:43, Templin (US), Fred L wrote:
[...]
> optimize the performance.

Have you ever checked the number of CVEs related to fragmentation?

---

From: Fernando Gont <[email-masked]>
Date: Thu, 5 Sep 2019 02:04:01 +0300

> the open Internet. The fragmentation footprint is the same as the tunnel

Sorry, maybe I'm missing something: If you fragment the outeer packet,
you better don't because fragmentation is fragile. If you fragment the
inner packet, the the outer packet is not fragmented, so from the pov of
the Internet you're not generating fragmented traffic.

Am I missing something?

Thanks,

---

From: Joe Touch <[email-masked]>
Date: Wed, 4 Sep 2019 21:43:08 -0700

> On Sep 4, 2019, at 10:34 AM, Ron Bonica <[email-masked]> wrote:
> 

You seem to be overlooking:

- progress it as it was approved by the WG and IETF LC and add an appendix recording contrary IESG opinions and identifying them as such
- progress it based on the text I suggested as was agreed by consensus in this discussion and add an appendix recording contrary IESG opinions and identifying them as such

This isn’t an IESG opinion. They get to write those themselves.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 5 Sep 2019 07:31:19 -0700

On Wed, Sep 4, 2019 at 5:34 PM Fernando Gont <[email-masked]> wrote:
>
> On 4/9/19 18:26, Templin (US), Fred L wrote:
> > Hi Ole,
> >
> This looks like a recent claim from the IAB that QUIC has shown that it
> is possible to deploy new transport protocols on the Internet. -- when
> quick employs UDP, and hence from the pov of the Internet, UDP is the
> transport protocol.
>
> Same with your example: you are tunneling your fragments. The Internet
> will not see fragments. *That* is why it would work.
>
> i.e., if you want fragmentation to not be fragile on the Internet, the
> trick is that packets shouldn't look like fragmens ;-)
>
Fernando,

Sure, if we dress up protocols in UDP there is a greater probability
they can traverse the Internet. But this doesn't always work and it's
fallacious to think that unbiquitously solves the problem of protocol
ossification. I really wish the IAB would look at the issues of wide
spread middlebox non-conformance-- this is the root problem and a
fundamental deterrent to evolving the Internet IMO.

Tom

---

From: Fernando Gont <[email-masked]>
Date: Thu, 5 Sep 2019 17:37:07 +0300

On 5/9/19 17:31, Tom Herbert wrote:
> ossification.

I fully agree with that. In fact, encapsulating transport protocols in
UDP *circumvents* the problem -- it doesn't actually fix it.

>  I really wish the IAB would look at the issues of wide

FWIW, I did ask about this in the context of EH-insertion on the IAB's
architecture list.

Thanks,

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Thu, 5 Sep 2019 15:17:26 +0000

Fernando,

The ingress uses IPv6 fragmentation per the letter of RFC8200 and conveys those
fragments over the underlying internetwork (e.g., the Internet) to the egress which
then gets to reassemble. If there are multiple independent internetworks lined up
in succession, there may be many re-encapsulating "waypoints" on the path. But,
only the final egress reassembles - again, per RFC8200.

The appearance of the fragments on the underlying internetwork(s) does not
matter - as long as the letter of RFC8200 is upheld, it is none other than good old
IPv6 fragmentation/reassembly.

Fred

> Sent: Wednesday, September 04, 2019 4:04 PM

---

From: Warren Kumari <[email-masked]>
Date: Thu, 5 Sep 2019 13:47:18 -0400

> It was taken out in response to Warren Kumari’s comment that it was out of place and already covered in a section later in the document. If it is added back in, it probably belongs in that section,…

I was summoned...

My comment was:
"2: I'm unclear why IP-in-IP tunnels are called out at the top / in
the Introduction. There is a whole section (Packet-in-Packet
Encapsulations) where I think it would go better -- I see no harm in
having people have to read down to there to note this."

It would be a disservice if someone looked at this document, saw the
text in the introduction, and then stopped reading....

W

> >> We discuss this on the list and come to a rare consensus on a way forward..
> >>
> >>>> On Sep 3, 2019, at 5:59 AM, Alissa Cooper via Datatracker <noreply@ietf..org> wrote:
> >>>

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Bob Hinden <[email-masked]>
Date: Thu, 5 Sep 2019 11:28:57 -0700

Hi,

Based on the discussion, I would like to propose to see if this will resolve the issues raised.   It attempts to cover the issues raised.

The full section 6.1 is included below, but only the last sentence in the second paragraph changed.

Please review and comment.

Thanks,
Bob

6.1.  For Application and Protocol Developers

   Developers SHOULD NOT develop new protocols or applications that rely
   on IP fragmentation.  When a new protocol or application is deployed
   in an environment that does not fully support IP fragmentation, it
   SHOULD operate correctly, either in its default configuration or in a
   specified alternative configuration.

   While there may be controlled environments where IP fragmentation
   works reliably, this is a deployment issue and can not be known to
   someone developing a new protocol or application.  It is not
   recommended that new protocols or applications be developed that rely
   on IP fragmentation.  Protocols and applications that rely on IP
   fragmentation will work less reliably on the Internet unless they
   also include mechanisms to detect that IP fragmentation isn't working
   reliably.

   Legacy protocols that depend upon IP fragmentation SHOULD be updated
   to break that dependency.  However, in some cases, there may be no
   viable alternative to IP fragmentation (e.g., IPSEC tunnel mode, IP-
   in-IP encapsulation).  In these cases, the protocol will continue to
   rely on IP fragmentation but should only be used in environments
   where IP fragmentation is known to be supported.

   Protocols may be able to avoid IP fragmentation by using a
   sufficiently small MTU (e.g.  The protocol minimum link MTU),
   disabling IP fragmentation, and ensuring that the transport protocol
   in use adapts its segment size to the MTU.  Other protocols may
   deploy a sufficiently reliable PMTU discovery mechanism
   (e.g.,PLMPTUD).

   UDP applications SHOULD abide by the recommendations stated in
   Section 3.2 of [RFC8085].

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Thu, 5 Sep 2019 18:57:19 +0000

Bob,

Your effort is appreciated, but IMHO does not quite go far enough. Here is
a proposed edit:

OLD:
   Protocols and applications that rely on IP
   fragmentation will work less reliably on the Internet unless they
   also include mechanisms to detect that IP fragmentation isn't working
   reliably.

NEW:
   Protocols and applications that rely on IP
   fragmentation will work less reliably on the Internet unless they
   also include mechanisms to detect that IP fragmentation isn't working
   reliably, or encapsulate their fragments in protocol headers that can
   traverse fragment-dropping middleboxes.

Thanks - Fred

> From: Int-area [mailto:[email-masked]] On Behalf Of Bob Hinden
> Sent: Thursday, September 05, 2019 11:29 AM
> To: [email-masked]
> Cc: IESG <[email-masked]>; Joel Halpern <[email-masked]>; [email-masked]; Suresh Krishnan
> <[email-masked]>; [email-masked]
> Subject: [Int-area] Discussion about Section 6.1 in draft-ietf-intarea-frag-fragile
> 
> Hi,
>

---

From: Bob Hinden <[email-masked]>
Date: Thu, 5 Sep 2019 12:33:06 -0700

Fred,

> On Sep 5, 2019, at 11:57 AM, Templin (US), Fred L <[email-masked]> wrote:
> 
> Bob,
> 

Thanks!

>   traverse fragment-dropping middleboxes.

I am not sure we want or should add specific mechanisms here.  Encapsulation is one approach, but there are others.

Bob

---

From: Joel Halpern <[email-masked]>
Date: Thu, 5 Sep 2019 19:52:10 +0000

It seems to me that Fred's proposal takes us into the area that the text says we are not dealing with.
Yours,
Joel

Sent from Workspace ONE Boxer

On Sep 5, 2019 21:33, Bob Hinden <[email-masked]> wrote:

Fred,

Thanks!

Bob

---

From: Tom Herbert <[email-masked]>
Date: Thu, 5 Sep 2019 12:53:01 -0700

Bob,

These two paragraphs seem somewhat contradicatory. For new protocols
the recommendation is not to use fragmentation because we can't know
whether the deployment allows that, but for legacy protocols we're
allowed to use fragmentation if we know the deployment allows that.

I think it's a lot simpler to just say that if you know fragmentation
works in your environment or to some destination then you can use it,
if not then you'll need to do something else. This applies equally to
legacy protocols and new protocols, on the open Internet as well as
limited domains. For that matter the rule applies pretty much to any
protocol that might be considered "fragile" (note, this might just be
a rewording of Joe's proposed text).

Tom

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Thu, 5 Sep 2019 19:57:18 +0000

Bob,

> Sent: Thursday, September 05, 2019 12:33 PM
> Cc: Bob Hinden <[email-masked]>; [email-masked]; IESG <[email-masked]>; Joel Halpern <[email-masked]>; draft-
> [email-masked]; Suresh Krishnan <[email-masked]>; [email-masked]
> Subject: Re: [Int-area] Discussion about Section 6.1 in draft-ietf-intarea-frag-fragile
> 
> Fred,
> 

s/encapsulate/disguise

?

Fred

---

From: Bob Hinden <[email-masked]>
Date: Thu, 5 Sep 2019 15:30:10 -0700

Fred,

> On Sep 5, 2019, at 12:57 PM, Templin (US), Fred L <[email-masked]> wrote:
> 
>>> 

That’s still the same issue.   I also agree with Joel’s comment.

Bob

---

From: Bob Hinden <[email-masked]>
Date: Thu, 5 Sep 2019 15:40:06 -0700

Tom,

> On Sep 5, 2019, at 12:53 PM, Tom Herbert <[email-masked]> wrote:
> 

I think the distinction is that it's hard to change legacy protocols and applications, but for new protocols and application, there is a choice to made and here we recommend that they not rely on fragmentation.   That seems clear to me.

> a rewording of Joe's proposed text).

I think that an implementation can’t know fragmentation works in its environment, with out including some sort of mechanism to test it.   Legacy protocols and applications do what they do now, new protocols and application can either decide to not use fragmentation or include a mechanism in their design.

Bob

---

From: Warren Kumari <[email-masked]>
Date: Thu, 5 Sep 2019 18:51:53 -0400

On Thu, Sep 5, 2019 at 3:53 PM Tom Herbert <[email-masked]> wrote:
> if not then you'll need to do something else.

<no hats>
Advice like this always feels like a cop-out to me[0].

Let's say that I do (somehow) know that fragmentation works in my
environment -- where is the knob that I turn in [ Linux | OS X | iOS |
Android | Windows | IOS | JunOS | <etc> ] which says "These set of
prefixes are within my environment and you can fragment when sending
to them?" And how do I also configure <insert long list of protocols>
to also know this?

It is a *very* small number of cases where my environment is
sufficiently well defined, homogeneous, and not connected to the
Internet that "environment" can be properly defined, *and* where all
nodes know that they are part of it -- off the top if my head, DTN is
the only example that comes to mind...

The "you can do xxx in your environment" usually feels like / results in:
a: "I'm tired of fighting, go do whatever you want"
b: weasel words which later get abuse to justify incorrect behavior
c: additional complexity
or d: all of the above.

What we've built is an Internet -- protocols should "just work", not
only work in specific parts of it, especially if there isn't an
obvious border (e.g: L2 domain, OSPF domain, etc)

W

[0]: An exception to this is the standard boilerplate which BMWG
attaches to its drafts, which goes along the lines of:
" Benchmarking activities as described in this memo are limited to
   technology characterization using controlled stimuli in a laboratory
   environment, with dedicated address space and the constraints
   specified in the sections above.

   The benchmarking network topology will be an independent test setup
   and MUST NOT be connected to devices that may forward the test
   traffic into a production network or misroute traffic to the test
   management network.

   Further, benchmarking is performed on a "black-box" basis, relying
   solely on measurements observable external to the DUT or System Under
   Test (SUT).  Special capabilities SHOULD NOT exist in the DUT/SUT
   specifically for benchmarking purposes.

   Any implications for network security arising from the DUT/SUT SHOULD
   be identical in the lab and in production networks."

> This applies equally to

---

From: Ron Bonica <[email-masked]>
Date: Thu, 5 Sep 2019 23:45:58 +0000

Bob,

I think that this is a close to consensus as we are going to get.

                                           Ron

-----Original Message-----
From: Bob Hinden <[email-masked]> 
Sent: Thursday, September 5, 2019 2:29 PM
To: [email-masked]
Cc: Bob Hinden <[email-masked]>; Alissa Cooper <[email-masked]>; IESG <[email-masked]>; Joel Halpern <[email-masked]>; [email-masked]; [email-masked]; Suresh Krishnan <[email-masked]>
Subject: Discussion about Section 6.1 in draft-ietf-intarea-frag-fragile

Hi,

Thanks,
Bob

---

From: Tom Herbert <[email-masked]>
Date: Thu, 5 Sep 2019 17:20:36 -0700

> the only example that comes to mind...

That's exactly how fragmentation is productively used today,
particularly in those use cases of IPIP tunneling where fragmentation.
There was a lot of discussion about this on the list. Telling someone
who has been happily using fragmentation for years that they shouldn't
use in their new applications they want to deploy seems trite to me.

> obvious border (e.g: L2 domain, OSPF domain, etc)

Yes, protocols should "just work". Fragmentation which has been
defined for more than thirty years is supposed to work reliably, but
it doesn't. Not because the protocol is broken, but because of some
non-conformant middlebox implementations. IMO, the recommendation of
this draft to avoid fragmentation is tantamount to a pragmatic
workaround for a bug and doesn't actually fix anything. As I've
already stated, I believe this draft should have been more direct in
requesting that buggy implementations are fixed.

Tom

---

From: Joe Touch <[email-masked]>
Date: Thu, 5 Sep 2019 18:16:52 -0700

> use in their new applications they want to deploy seems trite to me.

It's worth noting that apps don't always have any control over whether
they use such tunnels or not, or whether they can override the OS's keen
desire in using the largest MTUs possible (e.g., in TCP).

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 5 Sep 2019 18:18:28 -0700

Although this is close, it misses the mark a little on the issue that
the app may not actually have any control here - or know how or when to
reduce its MTU. That might be a minor point to add, but is worth adding.
This isn't just an app layer issue.

Joe

---

From: Bob Hinden <[email-masked]>
Date: Thu, 5 Sep 2019 21:05:26 -0700

Hi,

Joe and I talked off list.   The result is below.  Changes were to add a sentence in the forth and fifth paragraphs.

Please review.

Bob

----------

   Legacy protocols that depend upon IP fragmentation SHOULD be updated
   to break that dependency.  However, in some cases, there may be no
   viable alternative to IP fragmentation (e.g., IPSEC tunnel mode, IP-
   in-IP encapsulation).  Applications and protocols cannot necessarily
   know or control whether they use lower layers or network paths that
   rely on such fragmentation.  In these cases, the protocol will
   continue to rely on IP fragmentation but should only be used in
   environments where IP fragmentation is known to be supported.

   Protocols may be able to avoid IP fragmentation by using a
   sufficiently small MTU (e.g.  The protocol minimum link MTU),
   disabling IP fragmentation, and ensuring that the transport protocol
   in use adapts its segment size to the MTU.  Other protocols may
   deploy a sufficiently reliable PMTU discovery mechanism
   (e.g.,PLMPTUD).  The risks of IP fragmentation can also be mitigated
   through the use of encapsulation, e.g., by transmitting IP fragments
   as payloads.

—————

> On Sep 5, 2019, at 6:18 PM, Joe Touch <[email-masked]> wrote:
>

---

From: Ron Bonica <[email-masked]>
Date: Fri, 6 Sep 2019 04:16:04 +0000

Looks good to me

-----Original Message-----
From: Bob Hinden <[email-masked]>
Sent: Friday, September 6, 2019 12:05 AM
To: [email-masked]
Cc: Bob Hinden <[email-masked]>; Ron Bonica <[email-masked]>; IESG <[email-masked]>; Joel Halpern <[email-masked]>; [email-masked]; Suresh Krishnan <[email-masked]>; [email-masked]; Joe Touch <[email-masked]>
Subject: Re: [Int-area] Discussion about Section 6.1 in draft-ietf-intarea-frag-fragile

Hi,

Please review.

Bob

----------

—————

> the app may not actually have any control here - or know how or when 
> to reduce its MTU. That might be a minor point to add, but is worth adding.
>> Cc: Bob Hinden <[email-masked]>; Alissa Cooper 
>> <[email-masked]>; IESG <[email-masked]>; Joel Halpern 
>> <[email-masked]>; 
>> [email-masked]; [email-masked]; 
>> Suresh Krishnan <[email-masked]>

---

From: Ole Troan <[email-masked]>
Date: Fri, 6 Sep 2019 08:33:31 +0200

Bob, et al,

I have two issues with this text.

1) It introduces something new and undescribed in paragraph 2.
   "unless they also include mechanisms to detect that IP fragmentation isn't working
  reliably."
   That seems like hand-waving to me. Suggest deleting.

2) Paragraph 4:
   "The risks of IP fragmentation can also be mitigated
   through the use of encapsulation, e.g., by transmitting IP fragments
   as payloads."

   This seems like proposing new unspecified solutions with it's own set
   of considerations. IP fragmentation is a general solution to all hosts,
   encapsulation is certainly not in every host, and has different
   properties with regards to NAT traversal etc. Also if encapsulation
   was the answer, other segmentation / reassembly that were tunnel
   specific could be developed. Regardless this also amounts of hand-waving
   and doesn't seem to offer any advice that can be heeded now.
   And of course encapsulation can also exacerbate the problem
   by increasing packet size.
   Suggest deletion.

New text:

  fragmentation will work less reliably on the Internet.

Cheers,
Ole

> On 6 Sep 2019, at 06:05, Bob Hinden <[email-masked]> wrote:
> 
> Hi,
>

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Fri, 6 Sep 2019 14:13:09 +0000

This new text from Bob and Joe's discussion looks good to me.

> Sent: Thursday, September 05, 2019 9:05 PM
> To: [email-masked]
> Cc: Ron Bonica <rbonica=[email-masked]>; IESG <[email-masked]>; Joel Halpern <[email-masked]>; draft-ietf-
> [email-masked]; Suresh Krishnan <[email-masked]>; [email-masked]
> >> Cc: Bob Hinden <[email-masked]>; Alissa Cooper <[email-masked]>; IESG <[email-masked]>; Joel Halpern
> <[email-masked]>; [email-masked]; [email-masked]; Suresh Krishnan
> <[email-masked]>

---

From: Joe Touch <[email-masked]>
Date: Fri, 6 Sep 2019 07:14:29 -0700

Comments below. 

> On Sep 5, 2019, at 11:33 PM, Ole Troan <[email-masked]> wrote:
> 
> Bob, et al,
> 

Fragmentation success or failure is directly testable. Any feedback mechanism will work and specific ones are mentioned elsewhere (PLPMTUD).

This differs from ICMP black-holing in path MTU detection.

>   of considerations.

Specific widely-deployed methods are mentioned elsewhere in the doc, including GRE and UDP.

>   IP fragmentation is a general solution to all hosts,
>   encapsulation is certainly not in every host,

Actually, it is - unless you’re claiming hosts don’t support UDP.

> and has different
>   properties with regards to NAT traversal etc.

If by “different” you mean “much more likely to succeed”, agreed.

> vAlso if encapsulation
>   specific could be developed.

It is and is widely used - IPsec tunnels over UDP, e.g.

>  Regardless this also amounts of hand-waving

Yes, it makes the fragments smaller, which may be additional effort/performance impact, but it completely hides its impact on successful forwarding.

---

---

From: Ole Troan <[email-masked]>
Date: Fri, 6 Sep 2019 16:50:21 +0200

Joe,

> This differs from ICMP black-holing in path MTU detection.

Can you please point me to where in the PLPMTUD document testing for IP fragmentation is described?
I thought PLPMTUD found the largest unfragmented size of a datagram.

> Specific widely-deployed methods are mentioned elsewhere in the doc, including GRE and UDP.

Sorry, I couldn't find those either.
Inner fragmentation, firstly is only applicable to IPv4. And only applicable to tunnels.
And both those specs go to great length in avoiding fragmentation.

> Actually, it is - unless you’re claiming hosts don’t support UDP.

Sorry, I don't understand what you mean.
Are you saying that a new UDP applications should support the following stack:

IPv6 + UDP + IPv6 + FH + UDP + Applcation data

So to be able to hide IP fragments from the network?
While still having to do the full PLPMTUD to function correctly?

> If by “different” you mean “much more likely to succeed”, agreed.

I need to see numbers of that. But regardless I don't see the relevance to this document.

> It is and is widely used - IPsec tunnels over UDP, e.g.

That's a encapsulated solution to start with.

> Yes, it makes the fragments smaller, which may be additional effort/performance impact, but it completely hides its impact on successful forwarding.

You may be making a point. I'm afraid I don't get it.

Cheers,
Ole

---

From: Tom Herbert <[email-masked]>
Date: Fri, 6 Sep 2019 08:24:39 -0700

Bob,

Per the statement "It is not recommended that new protocols or
applications be developed that rely on IP fragmentation." I believe
this advice will have little if any impact on applications developers.

Consider that thousands of UDP applications are developed each year.
The vast majority of these are not presented in IETF, and conversely
most application developers don't participate in IETF and many
probably have never even read an RFC. Not everyone is going to get the
memo. Another problem is that this is a layering violation. A
fundamental design point of fragmentation is that it's a networking
layer function that is supposed to be transparent to the application.
The application is not required to care rather it's packets are
fragmented. I expect that the vast of UDP applications written don't
pay any heed whatsoever to fragmentation, as long as their application
is working to their satisfaction they don't care rather their packets
are being fragmented (i.e. they wrote the application for their own
purposes in their specific environment). That is protocols and
implementation working by design.

Conceivably, the host stack itself could modified to advise
applications that their packets are being fragmented and they might
want to reconsider. While host stacks already take several actions to
mitigate known problems related fragmentation (like transmitting first
fragment first and otherwise in order, ensuring flow label is
consistent set for both fragments and non-fragments, various security
and DOS mitigations), I don't believe it is reasonable to ask the host
stack to do anything to enforce the recommendation that new
applications don't rely on fragmentation-- fundamentally,
fragmentation being fragile is a problem in network devices, it's not
the obligation of host stack to "fix" their problems. If a packet is
sent and it's length exceeds MTU, or path MTU, and DONT_FRAGMENT is
not set, then the packet _will_ be fragmented. That's the way it works
today, and the way it will continue to work.

So regardless of what this draft says, there will be new applications
that rely on fragmentation for the foreseeable future (this is why the
previous text was appropriate). Grant it, the recommendation is not
normative, so I suppose that's already a statement on its expected
relevance. My real concern with this recommendation is that middlebox
developers, who do participate in IETF and do read RFCs, might
interpret this to mean that new applications won't be relying on
fragmentation and therefore they assume they don't need to fix bugs or
continue support, thereby making support for fragmentation even worse
than it currently is.

Tom

---

From: Joe Touch <[email-masked]>
Date: Fri, 6 Sep 2019 08:32:02 -0700

> On Sep 6, 2019, at 7:50 AM, Ole Troan <[email-masked]> wrote:
> 
> Joe,
> 
>> Comments below. 
>> 

Any feedback mechanism will detect when fragmentation - or anything else - prevents delivery.

> I thought PLPMTUD found the largest unfragmented size of a datagram.

It does - by trying larger sizes and telling the next layer down not to fragment. But if there are layers below that either ignore that or aren’t directly controlled that do fragment and those fragments don’t get through, it will back off.

> And both those specs go to great length in avoiding fragmentation.

If you fragment and put the result inside GRE or UDP, the impact of the fragment (interfering with flow-based multi path routing, nat traversal, etc) is overcome.

> IPv6 + UDP + IPv6 + FH + UDP + Applcation data

Yes - if they need FH and can’t do it any other way.

> So to be able to hide IP fragments from the network?

Yes - and again, this already happens for things like IPsec tunnels over UDP.

> While still having to do the full PLPMTUD to function correctly?

No; those would be used in different contexts. PLPMTUD is intended to avoid fragmentation but was given above as one of many examples of ways to determine when fragmentation caused a problem. E.g. (not that anyone needs to do this, but it’s reasonable):

	- test using PLPMTUD
	- if that fails, the app can decide what to do next
		a) use a smaller MTU
		b) fragment and encapsulate

Others have pointed out that there are measurements that indicate that (b) can be more efficient; regardless, it’s a choice.

> I need to see numbers of that. But regardless I don't see the relevance to this document.

It’s the defining reason that IPsec over UDP was developed and is widely deployed. Just look at RFCs or the IANA list of assigned ports for how many protocols are “over UDP” simply to traverse NATs.

> That's a encapsulated solution to start with.

IPsec wasn’t encapsulated. Adding encapsulation is the way it traverses tunnels.

> You may be making a point. I'm afraid I don't get it.

You keep saying this doesn’t help but it’s widely deployed and used already. 

joe

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Fri, 6 Sep 2019 15:47:16 +0000

Joe/Ole,

> >>> by increasing packet size.

All this means is that the fragmentation layer needs to take into account the
size of the outer encapsulation layers that will be added and make sure its
fragments do not exceed 1280 bytes *after* encapsulation. So, e.g., if the
encapsulation layer adds an IPv6 header and a UDP header the fragmentation
layer should not produce fragments larger than 1280 - 40 - 8 = 1232. If the
fragmentation layer does not know the size of the outer encapsulations to
be added, it can overestimate and set a safe smaller value (e.g., 1024).

Fred

---

From: Ole Troan <[email-masked]>
Date: Fri, 6 Sep 2019 18:03:11 +0200

Joe,

edited to focus on the two added "recommendation sentences".

> Any feedback mechanism will detect when fragmentation - or anything else - prevents delivery.

Any pointer to such a mechanism in any IETF protocol?
Would be interesting to get transport/application perspective on this.
But unless there is a reference I would claim this is hand-waving.

[...]

> If you fragment and put the result inside GRE or UDP, the impact of the fragment (interfering with flow-based multi path routing, nat traversal, etc) is overcome.

Sure, but that only applies to tunnels that go end to end. And any development of new tunnel mechanisms don't need to depend on IP fragmentation.
This is essentially link-layer (the tunnel provides a new link layer) fragmentation and reassembly.
It would anyway have to do that to deal with IPv4 DF=1 and IPv6.

This document should not recommend IP in UDP in IP encapsulation to achieve end to end IP fragmentation for new applications.

If this paragraph has to be there it would be more accepting to have it in the "Legacy protocols" parapgraph above.

  The risks of IP fragmentation can also be mitigated	

Cheers,
Ole

---

From: Ole Troan <[email-masked]>
Date: Fri, 6 Sep 2019 18:34:33 +0200

> be added, it can overestimate and set a safe smaller value (e.g., 1024).

Yes, absolutely. But I don't think we are talking about IP fragmentation any more.

Cheers,
Ole

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Fri, 6 Sep 2019 16:46:03 +0000

Ole,

> Sent: Friday, September 06, 2019 9:35 AM
> Cc: Joe Touch <[email-masked]>; Ron Bonica <rbonica=[email-masked]>; [email-masked]; IESG
> <[email-masked]>; Joel Halpern <[email-masked]>; [email-masked]; Suresh Krishnan

No - it is simply RFC8200-standard IPv6 fragmentation - not something else.

Fred

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Fri, 6 Sep 2019 17:43:41 +0000

Ole,

> Sure, but that only applies to tunnels that go end to end. And any development of new tunnel mechanisms don't need to depend on

All existing and future tunneling mechanisms that do not use IP fragmentation work
only due to the good luck that most link sizes in the Internet are on the order of 1500.
But, the only true guarantee is 1280 so the only true way to guarantee that the tunnel
can support 1280 after encapsulation is to apply fragmentation before encapsulation.

> It would anyway have to do that to deal with IPv4 DF=1 and IPv6.

It is RFC8200-standard IPv6 fragmentation.

> This document should not recommend IP in UDP in IP encapsulation to achieve end to end IP fragmentation for new applications.

"Encapsulation" does not necessarily mean IP-in-UDP-in-IP - it could be IP-in-foo-in-IP.
And again, the only true way to ensure that packets will traverse a path is to limit their
size to no more than 1280, and the only true way to achieve that when the original
packet size cannot be reduced is through fragmentation.

> If this paragraph has to be there it would be more accepting to have it in the "Legacy protocols" parapgraph above.

Not for legacy protocols - for the future of the Internet.

Thanks - Fred

---

From: Tom Herbert <[email-masked]>
Date: Fri, 6 Sep 2019 11:20:54 -0700

Ole,

This also makes the implicit and seemingly unproven assumption that
encapsulation is less fragile than IP fragmentation on the Internet.
Without the data that shows otherwise, I don't believe there is any
basis to say that encapsulation is generally a viable alternative.
Also, I would note that there was similar text in early versions of
RFC8200 that encapsulation could be used as a method of allowing
extension header insertion. That text was taken out because
encapsulation is not sufficiently standardized or ubiquitous to be
considered a direct replacement for an IP network layer protocol
function. The same is true here, encapsulation is not a drop-in
replacement for IP fragmentation.

Tom

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Fri, 6 Sep 2019 21:03:24 +0000

Hi Tom,

> From: Int-area [mailto:[email-masked]] On Behalf Of Tom Herbert
> Sent: Friday, September 06, 2019 11:21 AM
> To: Ole Troan <[email-masked]>
> Cc: Ron Bonica <rbonica=[email-masked]>; [email-masked]; IESG <[email-masked]>; Joel Halpern
> <[email-masked]>; [email-masked]; Suresh Krishnan <[email-masked]>; intarea-
> [email-masked]

With the rise of the middle and network-based (deep) packet inspection, I
think the statement could be considered that all things on the Internet are
potentially fragile or could become so in the future. Certainly we know that
naked IP fragments are filtered over many paths, and we have a strong
intuition that if we clothe them in "innocent-looking" clothing that they will
be far more likely to get through. For example, I don't think UDP port 6080
(or even 8060) are on many middlebox blacklists yet.

> basis to say that encapsulation is generally a viable alternative.

I don't think gathering data in a snapshot of time today could serve as a
predictor of the state of affairs a few weeks/months/years down the
road from now. But, what we believe is that right here; right now we
are far more likely to get a whole IP/UDP packet that contains an IP
fragment through than we are to get a naked IP fragment through.

I think if we deployed on the Internet today in innocuous headers like
UDP port 6080/8060 we could start getting the word out that IP fragments
carried in these protocols are good - not evil - and the proper way to
transport them. It's like anything else in the Internet in terms of what
middleboxes will pass vs. block; reputation is everything.

> replacement for IP fragmentation.

No; not a replacement for IP fragmentation but rather a vehicle for
carrying IP fragments. It is still good old IP fragmentation.

Fred

---

From: Tom Herbert <[email-masked]>
Date: Fri, 6 Sep 2019 14:44:03 -0700

Fred,

I suspect a lot of firewalls whitelist UDP ports (e.g. DNS and maybe
QUIC might be the only UDP applications that commonly pass firewalls).

> fragment through than we are to get a naked IP fragment through.
Where is the data to show that? It's a bit paradoxical that we have
declared IP fragmentation to be fragile based because the data "shows"
that and we assume that is a predictor of the future for IP
fragmentation, but yet in the same breath the draft would suggest
alternatives that with no deployment experience on the Internet and no
data to support that they will fare any better than fragmentation.

> carrying IP fragments. It is still good old IP fragmentation.

But, it's NOT IP fragmentation. Support for IP reassembly is a host
requirement. From RFC1122: "The Internet model requires that every
host support reassembly". The implication of this requirement is that
if I send a fragmented packet to any host on the Internet the
expectation is that the receiving host can reassemble the packet.
Support for IP reassembly has long been is ubiquitous in host
implementations, and as long as the network doesn't interfere
fragmentation will work and packets will be reassembled. This is true
for IPv4 and IPv6. There is no requirement that any encapsulation must
be processed by a receiving host. So if someone sends an encapsulated
packet (via GUE, IPIP, GRE, etc.) with fragments to some random
destination, there's no guarantee at all that the receiving host will
do anything with. Encapsulation requires coordination between senders
and receivers which does not exist in any general sense in the
Internet.

This discussion underscores a flaw of this draft I believe. While
there is a recommendation to avoid fragmentation, there really is no
robust substitute offered to replace the functionality. All the
alternatives either only work in narrow circumstances, may or may not
themselves be fragile, or place undue burden on application and
protocol developers to deal with something they have previously been
told they weren't supposed to worry about.

Tom

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Fri, 6 Sep 2019 22:28:09 +0000

Tom,

> From: Tom Herbert [mailto:[email-masked]]
> Sent: Friday, September 06, 2019 2:44 PM
> Cc: Ole Troan <[email-masked]>; Ron Bonica <rbonica=[email-masked]>; [email-masked]; IESG

But, how that came about is through some sort of successful marketing of
those protocols - a good reputation. The current draft at hand smacks of
spreading an unqualified bad reputation and we need to make sure that
the good applications of fragmentation don't get swept up in that. And,
IP fragmentation for tunnels is a good application. 

> data to support that they will fare any better than fragmentation.

It's like I said both above and below - all data can show is a snapshot in time,
but what really matters going forward is word-of-mouth reputation. Declare
X as evil, and everyone will block it; declare the opposite and everyone will
accept it. And word-of-mouth reputation in our field comes from RFCs.

> and receivers which does not exist in any general sense in the

I don't understand the point you are trying to make. The tunnel ingress
certainly needs to know that the egress is capable of removing the outer
encapsulations and reassemble - no one is suggesting otherwise. But, it
is still IP fragmentation and reassembly.

> told they weren't supposed to worry about.

Fragmentation is needed for tunnels. Else, a tunnel that traverses a 1280 link
will black hole. But, we should do better than that; call the tunnel MTU 9180
and use IP fragmentation in a link adaptation fashion, i.e., just like AAL5
except with a larger frame size.

Fred

---

From: Joe Touch <[email-masked]>
Date: Fri, 6 Sep 2019 20:47:12 -0700

> On Sep 6, 2019, at 9:03 AM, Ole Troan <[email-masked]> wrote:
> 
> Joe,
> 

As I said, PLPMTUD. It doesn’t have to detect if there’s a layer of fragmentation it can’t detect; it figures out for itself that a given MTU doesn’t succeed and sends a smaller MTU.

> Sure, but that only applies to tunnels that go end to end.

It applies to tunnels that traverse wherever fragmentation is both needed and doesn’t work other ways.

Speaking of deployed mechanisms, it’s also how IPsec tunnels over UDP already work and are widely deployed.

> ...This document should not recommend IP in UDP in IP encapsulation to achieve end to end IP fragmentation for new applications.

Why not exactly not?

Frag and reassembly outside of an app can be, in some cases, more efficient - even with additional layers of headers.

> If this paragraph has to be there it would be more accepting to have it in the "Legacy protocols" parapgraph above.

It’s unnecessarily limiting to mention it only there.

Joe

---

From: Ole Troan <[email-masked]>
Date: Sat, 7 Sep 2019 11:09:10 +0200

> As I said, PLPMTUD. It doesn’t have to detect if there’s a layer of fragmentation it can’t detect; it figures out for itself that a given MTU doesn’t succeed and sends a smaller MTU.

Sorry, I still don't understand how PLPMTUD detects if fragmentation works reliably or not.

The paragraph we are disccusing is:
  "Protocols and applications that rely on IP
  fragmentation will work less reliably on the Internet unless they
  also include mechanisms to detect that IP fragmentation isn't working
  reliably."

My view is that unless we can point to a reference or specification for how to do this, then it amounts of hand waving.
And possibly sending application developers out on a wild goose chase.
And given that applications would have to work in the cases where fragmentation was found to not work reliably...

> Speaking of deployed mechanisms, it’s also how IPsec tunnels over UDP already work and are widely deployed.

It does not work.
Host A -- ipsec tunnel -- SGW -- Host B

1) If host A sends fragments inside of the tunnel to host B, the fragments are equally fragile between the security gateway and host B.
2) If the PMTUD between Host A and security gateway is 1280, then the tunnel would have to support link-layer fragmentation anyway, because of the minimum MTU requirement.

There are lots of considerations here.
Which is why I do not think adding the new sentence that Bob suggested is a good idea.

>> ...This document should not recommend IP in UDP in IP encapsulation to achieve end to end IP fragmentation for new applications.

Firstly, a fundamental architectural change of how we would recommend application transport to work is not appropriate for this document.
Secondly, the goal of this document is not to "rescue IP fragmentation" at any cost.
In your above scheme it is arguable if it is even network layer fragmentation.
Tunnels as link-layers can define their own link-layer specific fragmentation mechanism. That is a lot more efficient than carrying a superfluous extra header with you.
But I don't think any of this can be in the scope of the document.

> Frag and reassembly outside of an app can be, in some cases, more efficient - even with additional layers of headers.

Yes, in _some_ cases. Try to introduce 1% of packet drop and that number of cases likely drop to 0.
The large scale deployments of this kind of thing is for example TSO/GSO. As a way to bypass bottlenecks in packet processing in the kernel.
But there segmentation is done at the transport layer, which will not suffer from the issues of a single packet drop in a 40 fragment chain.
(64K packet / 1500).

> It’s unnecessarily limiting to mention it only there.

The whole purpose of this document is to limit use of network layer fragmentation.

Section 6.1 with the two sentences deleted:

Cheers,
Ole

---

From: Joe Touch <[email-masked]>
Date: Sat, 7 Sep 2019 08:16:13 -0700

> On Sep 7, 2019, at 2:09 AM, Ole Troan <[email-masked]> wrote:
> 

It’s a homework assignment in a 1st yr net class.

*Any* two-way protocol gets feedback as to whether it works.

If you send large packets and they work, you’re done.

If not, you have to backoff and fragment at the app layer - which is less efficient, but will still get through.

—

ANY protocol that can adjust to an MTU based on feedback CAN try larger MTUs than the first hop link reports and see if they work.

> And given that applications would have to work in the cases where fragmentation was found to not work reliably…

They have to do this anyway, but this way they don’t have to be stuck with app-layer fragmentation when the network can to it instead.

> 1) If host A sends fragments inside of the tunnel to host B, the fragments are equally fragile between the security gateway and host B.

It’s not equally fragile. Fragments inside UDP overcome NATs and ECMP that (incorrectly) refuse to support packets without transport headers.

….
> Firstly, a fundamental architectural change of how we would recommend application transport to work is not appropriate for this document.

There is no architectural change in using UDP for tunnels. There’s a section in RFC 8085 on this if you want a citation.

> Yes, in _some_ cases. Try to introduce 1% of packet drop and that number of cases likely drop to 0.

(speaking of hand waving…)

> The large scale deployments of this kind of thing is for example TSO/GSO.

Oh, now we’re off into implementation mechanisms (not standards) that don’t follow existing standards either in many cases.

> As a way to bypass bottlenecks in packet processing in the kernel.

Fragmentation is fragmentation. If you send 64K messages and lose one fragment - at any layer of a protocol - you lose the message. Unless you send those fragments over a reliable channel that retransmits (e.g., TCP) anyway. 

> The whole purpose of this document is to limit use of network layer fragmentation.

NO IT IS NOT.

It is to reduce the impact of the FAILURE of network layer fragmentation - which IMO is best done through a *variety* of approaches that app and protocol designers can choose.

Joe

---

From: Bob Hinden <[email-masked]>
Date: Sat, 7 Sep 2019 10:25:22 -0700

Ole,

> This document should not recommend IP in UDP in IP encapsulation to achieve end to end IP fragmentation for new applications.

The document doesn’t say that, nor is the document recommending this.  The current text Joe and I proposed to the list was:

If this was recommended it would have included the word “recommended” or SHOULD/MUST terminology.

It’s only mentioning this, along with other approaches, as a possible approach for mitigating the problems.

Bob

---

From: Joe Touch <[email-masked]>
Date: Sat, 7 Sep 2019 10:35:37 -0700

FWIW, in general:

With all the concern not detecting when frag fails, I’d like to point out that it’s equally impossible to detect when it works, e.g., when it happens on tunnels that start more than one hop away or more than one layer of intermediate headers.

E.g, PLPMTUD turns of frag *on the connected interface*. There’s no way to disable source fragmentation that happens later in the network (as it would at tunnel ingresses) or deeper in the stack (when what you think is your interface is locally tunneled over a layer you don’t even know about).

So *all* systems that try to backoff and use smaller MTUs are actually *already* testing whether fragmentation already works in those cases. Even if your app sends a 1-byte packet you have no idea that some set of layers inflates the headers (e.g., with signatures or key exchanges) beyond the MTU somewhere.

Thus claiming that “fragmentation must be avoided at all costs” is simply not even possible anyway. That’s why it’s reasonable to bring encapsulation up. It works - even when you don’t know.

Joe

> On Sep 7, 2019, at 10:25 AM, Bob Hinden <[email-masked]> wrote:
> 
> Ole,
>

---

From: Fernando Gont <[email-masked]>
Date: Sat, 7 Sep 2019 05:25:03 +0300

> But unless there is a reference I would claim this is hand-waving.

FWIW, I don't remember seeing anything like this at the network layer.
(One might guess that in theory you *could* get feedback from upper
layers, but...)

This kind of thing is easier at *higher* layers -- such as trying to do
the 3WHS with the ECN bits set, and if you don't hear from the other
end, try with those bits cleared.

But IMO for IP it is easier said than done...

---

From: Bob Hinden <[email-masked]>
Date: Sat, 7 Sep 2019 10:21:07 -0700

Ole,

Bob

---

From: Fred Baker <[email-masked]>
Date: Sun, 8 Sep 2019 16:16:17 +0300

> On Sep 5, 2019, at 5:31 PM, Tom Herbert <[email-masked]> wrote:
> 
> spread middlebox non-conformance

Wouldn't that require the middle box to become an architectural element?

---

From: Joe Touch <[email-masked]>
Date: Sun, 8 Sep 2019 11:06:35 -0700

> On Sep 8, 2019, at 6:16 AM, Fred Baker <[email-masked]> wrote:
> 
> 
> 

Yes, but not just “an” (one):

Touch, J: Middlebox Models Compatible with the Internet <>. USC/ISI (ISI-TR-711), 2016. (Type: Technical Report | Links <> | BibTeX <>)
https://www.strayalpha.com/pubs/isi-tr-711.pdf <https://www.strayalpha.com/pubs/isi-tr-711.pdf>
Joe

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 9 Sep 2019 08:26:13 +1200

On 09-Sep-19 06:06, Joe Touch wrote:
> 
> 
>> On Sep 8, 2019, at 6:16 AM, Fred Baker <[email-masked] <mailto:[email-masked]>> wrote:
>>
>>
>>
>>> On Sep 5, 2019, at 5:31 PM, Tom Herbert <[email-masked] <mailto:[email-masked]>> wrote:
>>>
> Touch, J: Middlebox Models Compatible with the Internet. USC/ISI (ISI-TR-711), 2016. (Type: Technical Report | Links | BibTeX)
> 
>   * https://www.strayalpha.com/pubs/isi-tr-711.pdf

I'll take the liberty of pointing out that we've known for many years that
there are multiple types of middleboxes and they have multiple facets:
https://tools.ietf.org/html/rfc3234

So far, we haven't added them to any formal architectural description of
the Internet, probably because we don't have one.

    Brian

---

From: Joe Touch <[email-masked]>
Date: Sun, 8 Sep 2019 17:15:12 -0700

> On Sep 8, 2019, at 1:26 PM, Brian E Carpenter <[email-masked]> wrote:
> 
>>> 
>>  * https://www.strayalpha.com/pubs/isi-tr-711.pdf <https://www.strayalpha.com/pubs/isi-tr-711.pdf>
> 
> https://tools.ietf.org/html/rfc3234 <https://tools.ietf.org/html/rfc3234>

Facets are just properties. That doc makes no attempt to describe them as architectural roles.

> the Internet, probably because we don't have one.

RFC1122, RFC1123, RFC1812 as standards.

I (and IMO those RFCs) disagree with the position you took in RFC1958, FWIW.

Joe

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 9 Sep 2019 15:50:19 +1200

On 09-Sep-19 12:15, Joe Touch wrote:
> 
> 
>> On Sep 8, 2019, at 1:26 PM, Brian E Carpenter <[email-masked] <mailto:[email-masked]>> wrote:
>>
>>>>

Agreed. My point is that the issue has been lying on the IETF table for many years, and we've collectively chosen to ignore it.
 
> I (and IMO those RFCs) disagree with the position you took in RFC1958, FWIW.

"you" = the IAB in 1996; I was only the document editor. But IMHO those RFCs are not architectural as such. They describe functions of nodes, not how the nodes work together as a system.

    Brian

---

From: Joe Touch <[email-masked]>
Date: Sun, 8 Sep 2019 21:11:32 -0700

> On Sep 8, 2019, at 8:50 PM, Brian E Carpenter <[email-masked]> wrote:
> 

By “we”, I’m not sure who you mean. There have been plenty of us complaining about this gap for a very long time in the IETF.

> "you" = the IAB in 1996; I was only the document editor.

Fair point, but presumably you were on the IAB at the time?

> But IMHO those RFCs are not architectural as such. They describe functions of nodes, not how the nodes work together as a system.

An architecture is (IMO) the behavior of a system that is the consequence of the behavior of its components and the ways in which they interact.

Arguably, those RFCs - adding 791 and 3819, and a few others, do exactly that. In a nutshell: routers relay; subnets (as links) interconnect), and hosts source, sink, and do the rest (including reliability), all based on globally-unique addresses in that can be aggregated by bit prefix. No, there’s no “roadmap” doc that provides the top-level description, but the rest is definitely there in those and other docs.

IMO, the issue is that middleboxes can’t simply be added to that list consistently - they aren’t a third thing that can be peers to routers or hosts. There is a way to describe them that’s consistent, though - but only if they are different elements when viewed from different points in the network (as I described in the tech report). 

Joe

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 10 Sep 2019 08:40:47 +1200

On 09-Sep-19 16:11, Joe Touch wrote:
> 
> 
>> On Sep 8, 2019, at 8:50 PM, Brian E Carpenter <[email-masked] <mailto:[email-masked]>> wrote:
>>
>>>> On Sep 8, 2019, at 1:26 PM, Brian E Carpenter <[email-masked] <mailto:[email-masked]> <mailto:[email-masked]>> wrote:
>>>>
>>>>>>

Yes, but the IAB hasn't really taken up the issue and we haven't ever had a WG that seriously tackled it, IMHO.

> Fair point, but presumably you were on the IAB at the time?

Yes indeed. But although it's almost my most-cited RFC, I don't deserve sole credit or sole blame.

> Arguably, those RFCs - adding 791 and 3819, and a few others, do exactly that. In a nutshell: routers relay; subnets (as links) interconnect), and hosts source, sink, and do the rest (including rel…

Yes, we agree about that. It's the roadmap that is missing.

> IMO, the issue is that middleboxes can’t simply be added to that list consistently - they aren’t a third thing that can be peers to routers or hosts. There is a way to describe them that’s consiste…

I can't disagree.

   Brian

---

From: Fernando Gont <[email-masked]>
Date: Mon, 9 Sep 2019 23:46:30 +0300

Hi, Joe,

Just one nit:

> So *all* systems that try to backoff and use smaller MTUs are actually *already* testing whether fragmentation already works in those cases. Even if your app sends a 1-byte packet you have no idea…

This would seem to be incorrect. IP has a minimum MTU of 68 bytes, and
IPv6 has a minimum MTU of 1280. Hence if you send packets smaller than
or equal to the minimum MTU, the packets should go through.

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Tue, 10 Sep 2019 13:29:23 +0000

Fernando,

> From: Int-area [mailto:[email-masked]] On Behalf Of Fernando Gont
> Sent: Monday, September 09, 2019 1:47 PM
> To: Joe Touch <[email-masked]>; Bob Hinden <[email-masked]>
> Cc: [email-masked]; [email-masked]; IESG <[email-masked]>; Suresh Krishnan <[email-masked]>
> > With all the concern not detecting when frag fails, I’d like to point out that it’s equally impossible to detect when it works, e.g., when
> it happens on tunnels that start more than one hop away or more than one layer of intermediate headers.
> >
> > E.g, PLPMTUD turns of frag *on the connected interface*. There’s no way to disable source fragmentation that happens later in the
> network (as it would at tunnel ingresses) or deeper in the stack (when what you think is your interface is locally tunneled over a layer
> you don’t even know about).
> >
> > So *all* systems that try to backoff and use smaller MTUs are actually *already* testing whether fragmentation already works in
> those cases. Even if your app sends a 1-byte packet you have no idea that some set of layers inflates the headers (e.g., with
> signatures or key exchanges) beyond the MTU somewhere.
> 

Even if the original source uses the IPv6 minimum MTU of 1280, a tunnel somewhere
further down the path could add encapsulations that would cause the (encapsulated)
packet to exceed 1280 bytes. The tunnel therefore has to apply fragmentation.

Fred

---

From: Geoff Huston <[email-masked]>
Date: Tue, 10 Sep 2019 22:53:56 +0000

> packet to exceed 1280 bytes. The tunnel therefore has to apply fragmentation.

I hesitate to venture into this thread but I did a bit of digging around in the mail archives some time ago to figure out “why 1280?” as the IPv6 MTU. The desire was to lift the minimum unfragmented packet from 64 bytes (IPv4) to something that would reflect what was possible that would all but eliminate the need for fragmentation in IPv6. But at the same time there was the awareness of various forms of encapsulation and the possibility of multiple levels. 1500 octets was taken as the stating point and in the end 1280 was proposed. Why 1280? Because its the number you get when you add 1024 and 256. However, it expressed a basic idea that 1480 (IPv6 in IPv4), 1460 (IPv6 in IPv6), or any other number ‘close’ to 1500 could not. It allowed for almost any form of encapsulation of an IPv6 packet that we would be likely to see and the result would still  be within the 1500 ethernet framing limit and hence avoid a path MTU mismatch. From this starting point it is odd odd to see an argument about packet size that _starts_ with 1280 as some lower level media-related packet limit (it isn’t) and then applies encapsulation models on this. If we really are going to go through such an exercise then it would be more realistic to start with the number 1500 and apply encapsulation to that number. 

Secondly, it is interesting to look at what IPv6 stacks actually do with local MTU values. Do they all use 1280? nope! The most common value is 1430. (see http://www.potaroo.net/ispcol/2019-07/mss.html) 

So I personally don't see any practical value in this line of logic that says: "start with a source using a MTU of 1280 and apply encapsulation”

But I’ve said enough - I’m heading back back to lurking in this rather protracted and messy thread.

g

---

From: Joe Touch <[email-masked]>
Date: Tue, 10 Sep 2019 21:45:04 -0700

+1

You have no  way of knowing how many tunnels are being traversed.

There is no packet size that *guarantees* you have avoided fragmentation somewhere along an Internet path.

Joe

> On Sep 10, 2019, at 6:29 AM, Templin (US), Fred L <[email-masked]> wrote:
> 
> Fernando,
>

---

From: Joe Touch <[email-masked]>
Date: Tue, 10 Sep 2019 21:50:51 -0700

First, IPv4’s minimum is 68, not 64. The header can be up to 60 octets and the smallest fragment is 8 bytes.

Second, the problem with the logic that “bigger avoids fragmentation” is that the very specification of ANY minimum MTU, coupled with IP-in-IP tunnels (for their own sake, or as part of IPsec tunnel mode), ends up then requiring fragmentation. There’s no way around that - once there’s a required minimum and once it’s recursive, the game is over.

(BTW, the only thing that bails you out is to then ensure you have fragmentation and that the receiver reassembly minimum is large enough to cover two fragments, one as large as you can fit as a payload and the other that fits a payload at least as large as the rest of the MTU).

Joe

> On Sep 10, 2019, at 3:53 PM, Geoff Huston <[email-masked]> wrote:
> 
> 
>>>

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Wed, 11 Sep 2019 14:48:08 +0000

Geoff, the 1280 MTU came from Steve Deering's November 13, 1997 proposal to
the ipngwg. The exact message from the ipng archives is reproduced below.

1280 isn't just a recommendation - it's *the law*. Any link that cannot do 1280
(tunnels included) is not an IPv6 link.

Fred

---
From [email-masked]  Thu Nov 13 16:41:01 1997
Date: Thu, 13 Nov 1997 16:37:00 -0800
To: IPng Working Group <[email-masked]>
From: Steve Deering <[email-masked]>
Subject: (IPng 4802) increasing the IPv6 minimum MTU
Cc: [email-masked]
Sender: [email-masked]

In the ipngwg meeting in Munich, I proposed increasing the IPv6 minimum MTU
from 576 bytes to something closer to the Ethernet MTU of 1500 bytes, (i.e.,
1500 minus room for a couple layers of encapsulating headers, so that min-
MTU-size packets that are tunneled across 1500-byte-MTU paths won't be
subject to fragmentation/reassembly on ingress/egress from the tunnels,
in most cases).

After the short discussion in the Munich meeting, I called for a show of
hands, and of those who raised their hands (about half the attendees, if
I recall correctly), the vast majority were in favor of this change --
there were only two or three people opposed.  However, we recognized that
a fundamental change of this nature requires thoughtful discussion and
analysis on the mailing list, to allow those who were not at the meeting
and those who were there but who have since had second thoughts, to express
their opinions.  A couple of people have already, in private conversation,
raised some concerns that were not identified in the discussion at the
meeting, which I report below.  We would like to get this issue settled as
soon as possible, since this is the only thing holding up the publication
of the updated Proposed Standard IPv6 spec (the version we expect to advance
to Draft Standard), so let's see if we can come to a decision before the ID
deadline at the end of next week (hoping there isn't any conflict between
"thoughtful analysis" and "let's decide quickly" :-).

The reason I would like to increase the minimum MTU is that there are some
applications for which Path MTU Discovery just won't work very well, and
which will therefore limit themselves to sending packets no larger than
the minimum MTU.  Increasing the minimum MTU would improve the bandwidth
efficiency, i.e., reduce the header overhead (ratio of header bytes to
payload bytes), for those applications.  Some examples of such applications
are:

    (1) Large-fanout, high-volume multicast apps, such as multicast video
	("Internet TV"), multicast netnews, and multicast software
	distribution.  I believe these applications will end up limiting
	themselves to packets no large than the min MTU in order to avoid
	the danger of incurring  an "implosion" of ICMP Packet-Too-Big
	messages in response.  Even though we have specified that router
	implementations must carefully rate-limit the emission of ICMP
	error messages, I am nervous about how well this will work in
	practice, especially once there is a lot of high-speed, bulk
	multicasting happening.  An appropriate choice of rate or
	probability of emission of Packet-Too-Big responses to multicasts
	really depends on the fan-out of the multicast trees and the MTUs of
	all the branches in that tree, which is unknown and unknowable to
	the routers.  Being sensibly conservative by choosing a very low
	rate could, in many cases, significantly increase the delay before
	the multicast source learns the right MTU for the tree and, hence,
	before receivers on smaller-MTU branches can start receiving the
	data.

    (2) DNS servers, or other similar apps that have the requirement of
	sending a small amount of data (a few packets at most) to a very
	large and transient set of clients.  Such servers often reside on
	links, such as Ethernet, that have an MTU bigger than the links on
	which many of their clients may reside, such as dial-up links.  If
	those servers were to send many reply messages of the size of their
	own links (as required by PMTU Discovery), they could incur very
	many ICMP packet-too-big messages and consequent retransmissions of
	the replies -- in the worse case, multiplying the total bandwidth
	consumption (and delivery delay) by 2 or 3 times that of the
	alternative approach of just using the min MTU always.  Furthermore,
	the use of PMTU Discovery could result in such servers filling up
	lots of memory withed cached PMTU information that will never be
	used again (at least, not before it gets garbage-collected).

The number I propose for the new minimum MTU is 1280 bytes (1024 + 256,
as compared to the classic 576 value which is 512 + 64).  That would
leave generous room for encapsulating/tunnel headers within the Ethernet
MTU of 1500, e.g., enough for two layers of secure tunneling including
both ESP and AUTH headers.

For medium-to-high speed links, this change would reduce the IPv6 header
overhead for min MTU packets from 7% to 3% (a little less than the IPv4
header overhead for 576-byte IPv4 packets).  For low-speed links such as
analog dial-up or low-speed wireless, I assume that header compression will
be employed, which compresses out the IPv6 header completely, so the IPv6
header overhead on such links is effectively zero in any case.

Here is a list of *disadvantages* to increasing the IPv6 minimum MTU that
have been raised, either publically or privately:

    (1) This change would require the specification of link-specific
	fragmentation and reassembly protocols for those link-layers
	that can support 576-byte packets but not 1280-byte packets,
	e.g., AppleTalk.  I think such a protocol could be very simple,
	and I briefly sketch such a protocol in Appendix I of this
	message, as an example.

	Often, those links that have a small native MTU are also the ones
	that have low bandwidth.  On low-bandwidth links, it is often
	desirable to locally fragment and reassemble IPv6 packets anyway
	(even 576-byte ones) in order to avoid having small, interactive
	packets (e.g., keystrokes, character echoes, or voice samples)
	be delayed excessively behind bigger packets (e.g., file transfers);
	the small packets can be interleaved with the fragments of the
	big packets.  Someone mentioned in the meeting in Munich that the
	ISSLL WG was working on a PPP-specific fragmentation and
	reassembly protocol for precisely this reason, so maybe the job
	of specifying such a protocol is already being taken care of.

    (2) Someone raised the concern that, if we make the minimum MTU close
	to Ethernet size, implementors might never bother to implement PMTU
	Discovery.  That would be regrettable, especially if the Internet
	evolves to much more widespread use of links with MTUs bigger
	than Ethernet's, since IPv6 would then fail to take advantage of
	the bandwidth efficiencies possible on larger MTU paths.

    (3) Peter Curran pointed out to me that using a larger minimum MTU for
	IPv6 may result in much greater reliance on *IPv4* fragmentation and
	reassembly during the transition phase while much of the IPv6
	traffic is being tunneled over IPv4.  This could incur unfortunate
	performance penalties for tunneled IPv6 traffic (disasterous
	penalties if there is non-negligible loss of IPv4 fragments).
	I have included Peter's message, describing his concern in more
	detail, in Appendix II of this message.

    (4) Someone expressed the opinion that the requirement for link-layer
	fragmentation and reassembly of IPv6 over low-cost, low-MTU links
	like Firewire, would doom the potential use of IPv6 in cheap
	consumer devices in which minimizing code size is important --
	implementors of cheap Firewire devices would choose IPv4 instead,
	since it would not need a fragmenting "shim" layer.  This may well
	be true, though I suspect the code required for local frag/reasm
	would be negligible compared to the code required for Neighbor
	Discovery.

Personally, I am not convinced by the above concerns that increasing the
minimum MTU would be a mistake, but I'd like to hear what the rest of the
WG thinks.  Are there other problems that anyone can think of?  As I
mentioned earlier, the clear consensus of the Munich attendees was to
increase the minimum MTU, so we need to find out if these newly-identified
problems are enough to swing the consensus in the other direction.  Your
feedback is heartily requested.

Steve

----------

Appendix I

Here is a sketch of a fragmentation and reassembly protocol (call it FRP)
to be employed between the IP layer and the link layer of a link with native
(or configured) MTU less than 1280 bytes.

Identify a Block Size, B, which is the lesser of (a) the native MTU of the
link or (b) a value related to the bandwidth of the link, chosen to bound
the latency that one block can impose on a subsequent block.  For example,
to stay within a latency of 200 ms on a 9600 bps link, choose a block size
of .2 * 9600 = 2400 bits = 240 bytes.

IPv6 packets of length <= B are transmitted directly on the link.
IPv6 packets of length > B are fragmented into blocks of size B
(the last block possibly being shorter than B), and those fragments
are transmitted on the link with an FRP header containing the following
fields:

	[packet ID, block number, end flag]

where:

	packet ID is the same for all fragments of the same packet,
	and is incremented for each new fragmented packet.  The size of
	the packet ID field limits how many packets can be in flight or
	interleaved on the link at any one time.

	block number identifies the blocks within a packet, starting at
	block zero.  The block number field must be large enough to
	identify 1280/B blocks.

	end flag is a one-bit flag which is used to mark the last block
	of a packet.

For example, on a 9600 bps serial link, one might use a block size of
240 bytes and an 8-bit FRP header of the following format:

	4-bit packet ID, which allows interleaving of up to 16 packets.
	3-bit block number, to identify blocks numbered 0 through 5.
	1-bit end flag.

On a 256 kpbs AppleTalk link, one might use the AppleTalk-imposed block
size of ~580 bytes and an 8-bit FRP header of the following format:

	5-bit packet ID, which allows for up to 32 fragmented packets in
		   flight from each source across the AppleTalk internet.
	2-bit block number, to identify blocks numbered 0 through 2.
	1-bit end flag.

On a multi-access link, like AppleTalk, the receiver uses the link-level
source address as well as the packet ID to identify blocks belonging to
the same packet.

If a receiver fails to receive all of the blocks of a packet by the time
the packet number wraps around, it discards the incompletely-reassembled
packet.  Taking this approach, no timers should be needed at the receiver
to detect fragment loss.  We expect the transport layer (e.g., TCP) checksum
at the final IPv6 destination to detect mis-assembly that might be caused by
extreme misordering/delay during transit across the link.

On links on which IPv6 header compression is being used, compression is
performed before fragmentation, and reassembly is done before decompression.

----------

Appendix II

From: Peter Curran <[email-masked]>
Subject: Re: IPv6 MTU issue
To: [email-masked] (Steve Deering)
Date: Mon, 22 Sep 1997 11:50:34 +0100 (BST)

Steve

My problem was that moving the MTU close to 1500 would have an adverse
effect on the transition strategy.  The current strategy assumes that the
typical Internet MTU is >576, and that sending an IPv6 packet close to the
minimum MTU will not require any IPv4 fragmentation to support the tunnel
transparently.  The PMTU discovery mechanism will 'tune' IPv6 to use a
suitable MTU.

If the IPv4 MTU is <= 576 then IPv4 fragmentation will be required to
provide a tunnel with a minimum MTU of 576 for IPv6.  This clearly places
a significant strain on the tunnelling nodes - as these will normally be
routers then there will be a demand for memory (for reassembly buffers)
as well as CPU (for the frag/reassembly process) that will have an overall
impact on performance.

This is an acceptable risk, as Internet MTU's of <= 576 are not too common.

However, if the minimum MTU of IPv6 is increased to something of the order
of 1200-1500 octets then the likelihood of finding an IPv4 path with an
MTU lower than this value increases (I think significantly) and this will
have a performance impact on these devices.

During the brief discussion of this matter in the IPNG session at Munich
you stated that MTU's less than 1500 where rare.  I don't agree with this
completely - it seems to be pretty common practise for smaller 2nd and 3rd
tier ISP's in the UK to use an MTU of 576 for connection to their transit
provider.  Their objective, I believe, is to 'normalize' the packet sizes
on relatively low bandwidth circuits (typically <1Mbps) to provide better
performance for interactive sessions compared to bulk-file transfer users.

I think that before we go ahead and make a decision on an increased minimum
MTU for IPv6 then we should discuss the issues a little more.

Incidentally, I am not convinced of the benefits of doing this anyway
(ignoring the issue raised above).  With a properly setup stack the PMTU
discovery mechanism seems to be able to select a good MTU for use on the
path - at least that is my experience on our test network and the 6Bone.

I appreciate that you are trying to address the issues of PMTU for multi-
casting but I don't see how raising the minumum MTU is going to help much.
PMTU discovery will still be required irrespective of the minimum MTU
adopted, unless we adopt a value that can be used on all link-layer technolo-
gies.

I would welcome wider discussion of these issues before pressing ahead
with a change.

Best regards

Peter Curran
TICL

--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: Bob Hinden <[email-masked]>
Date: Wed, 11 Sep 2019 15:59:24 -0700

Fred,

> On Sep 11, 2019, at 7:48 AM, Templin (US), Fred L <[email-masked]> wrote:
> 

Yes from IPv6’s view, but you can make a link that can’t do 1280 work if it has its own local L2 fragmentation / reassembly as noted in Steve’s email.  ATM with is 53 byte cells comes to mind.

Bob

> Received: (from majordomo@localhost)
> 	by sunroof.eng.sun.com (8.8.8+Sun.Beta.0/8.8.8) id QAA14339
> 	for ipng-dist; Thu, 13 Nov 1997 16:38:00 -0800 (PST)
> Received: from Eng.Sun.COM (engmail1 [129.146.1.13])
> 	by sunroof.eng.sun.com (8.8.8+Sun.Beta.0/8.8.8) with SMTP id QAA14332
> 	for <ipng@sunroof>; Thu, 13 Nov 1997 16:37:51 -0800 (PST)
> Received: from saturn.sun.com (saturn.EBay.Sun.COM [129.150.69.2])
> 	by Eng.Sun.COM (SMI-8.6/SMI-5.3) with SMTP id QAA28654
> 	for <[email-masked]>; Thu, 13 Nov 1997 16:37:48 -0800
> Received: from postoffice.cisco.com (postoffice.cisco.com [171.69.200.88])
> 	by saturn.sun.com (8.8.8/8.8.8) with ESMTP id QAA28706
> 	for <[email-masked]>; Thu, 13 Nov 1997 16:37:49 -0800 (PST)
> Received: from [171.69.199.124] (deering-mac.cisco.com [171.69.199.124]) by postoffice.cisco.com (8.8.5-Cisco.1/8.6.5) with ESMTP id QAA20862; Thu, 13 Nov 1997 16:37:48 -0800 (PST)
> X-Sender: [email-masked]
> Message-Id: <v03110702b0598e80008d@[171.69.199.124]>
> Mime-Version: 1.0
> Content-Type: text/plain; charset="us-ascii"
> Precedence: bulk

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 12 Sep 2019 11:14:16 +1200

> Yes from IPv6’s view, but you can make a link that can’t do 1280 work if it has its own local L2 fragmentation / reassembly as noted in Steve’s email.  ATM with is 53 byte cells comes to mind.

IPv4 with a small PMTU also comes to mind, as discussed in Section 3.2.2 of RFC 4213:

   In this case, the IPv6 layer has to "see" a link
   layer with an MTU of 1280 bytes and the encapsulator has to use IPv4
   fragmentation in order to forward the 1280 byte IPv6 packets.

      Brian

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Thu, 12 Sep 2019 13:30:23 +0000

Bob,

> Sent: Wednesday, September 11, 2019 3:59 PM
> Cc: Bob Hinden <[email-masked]>; Geoff Huston <[email-masked]>; Joe Touch <[email-masked]>; [email-masked];
> Yes from IPv6’s view, but you can make a link that can’t do 1280 work if it has its own local L2 fragmentation / reassembly as noted in
> Steve’s email.  ATM with is 53 byte cells comes to mind.

Yes, link adaptation at a layer below IPv6 is necessary for links that can't provide
a 1280 MTU natively - I think 6lowpan is another example. But, for IPv6 tunnels
the layer below IPv6 is also IP, and the link adaptation is IP fragmentation.

Thanks - Fred

> > Received: from [171.69.199.124] (deering-mac.cisco.com [171.69.199.124]) by postoffice.cisco.com (8.8.5-Cisco.1/8.6.5) with ESMTP
> id QAA20862; Thu, 13 Nov 1997 16:37:48 -0800 (PST)

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Thu, 12 Sep 2019 13:31:00 +0000

Brian,

> From: Brian E Carpenter [mailto:[email-masked]]
> Sent: Wednesday, September 11, 2019 4:14 PM
> To: Bob Hinden <[email-masked]>; Templin (US), Fred L <[email-masked]>
> Cc: [email-masked]; Suresh Krishnan <[email-masked]>

Yes, IP fragmentation - exactly.

Fred

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Thu, 12 Sep 2019 13:57:03 +0000

> From: Int-area [mailto:[email-masked]] On Behalf Of Templin (US), Fred L
> Sent: Thursday, September 12, 2019 6:31 AM
> To: Brian E Carpenter <[email-masked]>; Bob Hinden <[email-masked]>

See also Section 7 of RFC2473.

---

From: Joe Touch <[email-masked]>
Date: Thu, 12 Sep 2019 07:22:25 -0700

> On Sep 12, 2019, at 6:57 AM, Templin (US), Fred L <[email-masked]> wrote:
> 

There are a number of issues with that RFC, as noted in intarea-tunnels.

E.g., it won’t properly do what you say for IPv4 DF=0 (i.e., treat the tunnel as a true link layer by allowing frag at the tunnel header), nor will it correctly interconnect two hosts for packets with TTL/hopcounts fo 0 (which should work - only routers should decrement those).

Joe

---

From: Fred Baker <[email-masked]>
Date: Fri, 13 Sep 2019 22:19:52 +0300

> On Sep 11, 2019, at 7:50 AM, Joe Touch <[email-masked]> wrote:
> 

Indeed.

My perception is that the larger the difference between the packet size sent and the smallest link MTU the packet will encounter, the lower the probability that the packet will get fragmented in flight - in IPv4 - or will result in an ICMP PTB message. The only size that completely prevents those outcomes is to not send the packet.