Subject: [Int-area] intarea-tunnels meta-comment: tunnel fragmentation

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 15 Jul 2016 17:22:11 +0000

intarea-tunnels needs to introduce and discuss the notion of Tunnel
Fragmentation (RFC2764, Section 3.1.7). Use of tunnel fragmentation
was specified in SEAL [RFC5320], and it is now also specified in
(draft-herbert-gue-extensions) and (draft-templin-intarea-grefrag).

Tunnel fragmentation uses fragmentation control fields in a
shim header between the inner and outer IP headers, and it is
neither inner fragmentation nor outer fragmentation. Like outer
fragmentation, only the first fragment includes the inner IP
header and the other fragments include the remaining portions
of the inner IP packet body. Unlike outer fragmentation,
however, the shim header appears in all fragments and not just
the first fragment. It is also possible that both tunnel
fragmentation and outer fragmentation can be applied to the
same tunneled packet, e.g. if an IPv4 tunneled packet that
has undergone tunnel fragmentation is fragmented by a router
within the tunnel.

Tunnel fragmentation is very important, because it overcomes the
16bit IPv4 and 32bit IPv6 ID field limitations. For example, GUE
includes a 40bit ID field, which should be sufficient to ensure there
will be no reassembly errors at any realistic data rate. The tunnel
fragments also appear as whole IP packets on the wire, and not
as IP fragments (i.e., unless network fragmentation occurs) which
means that they will still get through on network paths where
IP fragments may be dropped.

IMHO, the document needs to discuss this.

Fred

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Jul 2016 11:16:16 -0700

Fred,

I agree that the doc should discuss the use of "outer fragmentation" as
using the entire set of protocols involved in the encapsulation - which
could mean IP + GUE, or could mean IP + TCP, etc.

Don't think there's a good "one size fits all" recommendation of how to
handle fragmentation across those layers - it really depends on the layers.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 15 Jul 2016 19:05:26 +0000

Hi Joe,

> -----Original Message-----
> From: Joe Touch [mailto:[email-masked]]
> Sent: Friday, July 15, 2016 11:16 AM
> To: Templin, Fred L <[email-masked]>; [email-masked]
> Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation
> 
> Fred,
> 

Any encapsulation that inserts shim headers between the inner and outer IP
headers can employ tunnel fragmentation by also inserting a fragmentation
shim header. But, again, this is different than both outer and inner
fragmentation and should be brought into the document.

> handle fragmentation across those layers - it really depends on the layers.

Tunnel fragmentation applies to any encapsulation of IP-in-[shim]-in-IP
for any value of "[shim]", e.g., GUE, GRE, SEAL, others. So, one size
fits all text can be added by just referring to the [shim] headers as
a generic middle layer where fragmentation can be applied.

Historical note - when I was developing SEAL I stumbled on the idea
of using tunnel fragmentation and thought I was dealing with something
that was brand-new. It was Bob Braden who brought RFC2764 to my
attention as the original source for this technique.

Thanks - Fred
[email-masked]

---

From: Tom Herbert <[email-masked]>
Date: Fri, 15 Jul 2016 12:29:31 -0700

> a generic middle layer where fragmentation can be applied.
I'm not sure it's quite that easy. There are several nuances to really
do tunnel fragmentation right. First , the encapsulation protocol
needs to be extensible to allow the fragmentation headers-- this
eliminates GRE as a candidate since there are only five bits that can
be used for indicating fields and that are all reserved. VXLAN can't
be extended and I'm pretty it will be hard to make this work in
VXLAN-GPE (maybe there could be a fragmentation nsh header?).

Geneve could be extended with a fragmentation TLV, but then
fragmentation creates a problem in middleboxes that want to look at
Geneve payload. Geneve has a header length and protocol field with an
Ethertype that allows DPI without inspecting the TLVs. The question is
what should the Ethertype be set to for fragments? If the fragment is
for IPv4 packet and IPV4 is set in Ethertype, middelboxes would try to
incorrectly parse the fragment as an IPv4 packet. We get around this
in GUE by setting IP proto field in the GUE header to "no next header"
for fragments, I don't know if there is an equivalent Ethertype that
could be used for the same purpose.

Tom

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 15 Jul 2016 19:43:01 +0000

Hi Tom,

> From: Tom Herbert [mailto:[email-masked]]
> Sent: Friday, July 15, 2016 12:30 PM
> To: Templin, Fred L <[email-masked]>
> Cc: Joe Touch <[email-masked]>; [email-masked]
> Subject: Re: [Int-area] intarea-tunnels meta-comment: tunnel fragmentation
> 
> do tunnel fragmentation right.

I didn't say that it was easy, nor that it could be applied to any and
all encapsulation shim headers.

> First , the encapsulation protocol
> be used for indicating fields and that are all reserved.

Regarding GRE, see 'draft-templin-intarea-grefrag'.

> VXLAN can't

There are many tunneling protocols that cannot be extended in this
way, that is true, but there are undeniable benefits for those that can. 

> could be used for the same purpose.

I didn't notice the "no next header" for fragments, but it makes sense.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Jul 2016 12:51:36 -0700

> fragmentation and should be brought into the document.
It is outer fragmentation.

Inner fragmentation occurs inside the node as a host sourcing traffic or
as a router relaying IPv4 DF=0 on the arriving TTP packet.

Outer fragmentation occurs in the ingress in the layer(s) used for
encapsulation. To the node, this is all "inside the link layer".

Yes, there are good and bad ways to do this, but it's doesn't
necessitate a different interaction between the node and the ingress.

The reason I noted there's no "one size fits all" solution is that the
answer depends on the layer(s) available, their capabilities (e.g.,
whether IPv4 has a 16-bit ID and GUE has a 40-bit ID) and the impact of
the mechanism on the rest of the communication (e.g., does it destroy or
preserve info that might be useful for multipath between ingress and
egress).

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 15 Jul 2016 19:58:10 +0000

Hi Joe,

> Sent: Friday, July 15, 2016 12:52 PM

That is incorrect. Tunnel fragmentation is apart from outer fragmentation,
and it is possible for both tunnel fragmentation and outer fragmentation
to be applied on the same packet. Both use different IDs and fragment
offsets that are managed independently of one another.

It is even possible for a single packet to undergo all three of inner,
tunnel and outer fragmentation - the three levels of fragmentation
are distinct, and all have a place in the Internet architecture.

Fred

---

From: Tom Herbert <[email-masked]>
Date: Fri, 15 Jul 2016 13:04:04 -0700

> Regarding GRE, see 'draft-templin-intarea-grefrag'.
Unfortunately, the bits that you're using are already defined by
RFC1701. GRE is well deployed and there is a lot of middlebox support
that easily break when trying to extend GRE (this is precisely why we
created GUE!).

> way, that is true, but there are undeniable benefits for those that can.
Agreed, it seems to be a much better solution that doing IP
fragmentation of of inner and outer headers.

> I didn't notice the "no next header" for fragments, but it makes sense.
We also use this with DTLS in GUE.

Tom

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Jul 2016 13:06:35 -0700

Fred,

On 7/15/2016 12:58 PM, Templin, Fred L wrote:
> ...
I am claiming that outer fragmentation is defined (or soon to be) as
"fragmentation that occurs inside the ingress due to encapsulation(s)
performed by the ingress", not merely at the outermost protocol.

> to be applied on the same packet. 

Tunnel fragmentation (as you call it) is really just one part of what I
call outer fragmentation.

I don't like the term tunnel fragmentation because that could also mean
all of the fragmentation that occurs to support tunneling.

Yes, though - I do agree that ANY layer of that ingress encapsulation
might be involved in fragmentation, and perhaps multiples of such layers.

Again, IMO, that's internal to the "link protocol" as seen to the node,
so I don't think it's important to go into detail.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 15 Jul 2016 20:14:00 +0000

Hi Tom,

> Sent: Friday, July 15, 2016 1:04 PM

I don't mind citing RFC1701 and using one of the flags bits for what
I want to accomplish. I also don't mind it breaks along some paths.

> fragmentation of of inner and outer headers.

It doesn't have to be all one way or the other. All three of inner, outer
and tunnel layer fragmentation can be employed even on the same
packet. Not that that would be desirable, but certainly possible.

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 15 Jul 2016 20:19:00 +0000

Hi Joe,

> Sent: Friday, July 15, 2016 1:07 PM

Tunnel fragmentation is apart from and separate from outer fragmentation.
This is a fact, and not an opinion.

> call outer fragmentation.

Not as *I* call it, but as RFC2764 calls it.

> all of the fragmentation that occurs to support tunneling.

I don't care much for the term either, but when Bob pointed out the
RFC2764 text I assumed that he meant for me to adopt the terminology
also. I hope Bob is doing well; please tell him I appreciate his guidance.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Jul 2016 16:06:16 -0700

> This is a fact, and not an opinion.

The model in intarea-tunnels is that a tunnel is a link.

Link layers often have multiple layers of encapsulation - e.g., AAL5 and
ATM.

We don't call those multiple link layers - we treat them as one as far
as the attaching host is concerned.

Thus intarea-tunnels differentiates only the two kinds of encapsulation
as seen by the Internet node:

    - TTP-layer fragmentation (source frag by hosts, relay frag of IPv4
DF=0 by routers)
        intarea-tunnels calls this "inner" fragmentation

    - link fragmentation, which includes *everything* that happens
inside what the node considers a link
        intarea-tunnels calls this "outer" fragmentation

I agree that inner and outer are probably not the most useful terms. It
might be better to call these:

    existing Internet fragmentation

    tunnel fragmentation

However, tunnel fragmentation includes everything that a tunnel does to
get from ingress to egress.
> Not as *I* call it, but as RFC2764 calls it.
That document never uses the term "tunnel fragmentation".

I uses "mid-tunnel fragmentation" (Sec 3.1.7), which would be on-path
IPv4 refragmentation inside the tunnel, which isn't what you mean either.

Regarding what you call tunnel fragmentation, it says (same section):

   An alternative approach is for the tunneling protocol itself to
   incorporate a segmentation and reassembly capability that operates at
   the tunnel level, perhaps using the tunnel sequence number and an
   end-of-message marker of some sort.  (Note that multilink PPP uses a
   mechanism similar to this to fragment packets).  This avoids IP level
   fragmentation within the tunnel itself. None of the existing
   tunneling protocols support such a mechanism.

> also. 
See above; I don't think the terms in that doc are particularly useful
given our modern understanding.
> I hope Bob is doing well; please tell him I appreciate his guidance.
(he is and I will...)

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Jul 2016 16:16:04 -0700

Hi, Fred (et al.),

To clean up this part of the thread:

TUN/TAP are basically software versions of the *opposite* of a tunnel as
explained in intarea-tunnels.

They provide virtual interfaces *behind which you can implement a
host/router*, not over which to communicate to a remote node (i.e., a link).

TAPs provide that VIF for L2 - i.e., the attached process needs to
implement a bridge (L2 router) or L2 host.

TUNs provide that VIF for L3 - i.e., the attached process needs to
implement an entire host/router OS.

These devices *can be used* to implement a tunnel by treating them as
the entry point to an "ingress system" implemented in user-space.
However, they can be used to implement things that aren't tunnels at
all, such as L2 bridges, L2 hosts, L3 hosts, and L3 routers.

The user process needs to implement ICMP processing when these devices
are used to implement these devices - but NOT when they're used to
actually implement encapsulation tunnels. For encapsulation tunnels, the
user process can (and should) only encapsulate traffic and write it back
into the OS as a raw L2/L3 packet (via the TAP/TUN interface) for
further processing.

---

So there are two cases:

    - TUN/TAP used to implement a tunnel, which can be done entirely
consistent with intarea-tunnels

    - TUN/TAP used to implement an endsystem, which requires ICMP support

I don't think it's useful to treat these two cases as at all related.

Joe

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Sat, 16 Jul 2016 04:49:31 +0000

Fred,

> On Jul 15, 2016, at 3:43 PM, Templin, Fred L <[email-masked]> wrote:
> 
> Hi Tom,
> 
> Regarding GRE, see 'draft-templin-intarea-grefrag’.

Could you please add an Implementation Status Section [RFC 6982] to this I-D?

> way, that is true, but there are undeniable benefits for those that can. 

There’s more than one way …

RFC 4623 presents yet another way.

I’m not too sure these are generalizable — nor really practical, at all.

Thanks,

— Carlos.

> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 18 Jul 2016 18:06:09 +0000

Hi Joe,

We have:

  1) inner fragmentation
  2) mid-layer fragmentation (what I have been calling "tunnel fragmentation")
  3) outer fragmentation

All three can be applied *on the same packet*; hence, they are three different
things altogether, although potentially complimentary.

The RFC2764 mid-layer/tunnel-layer fragmentation is very important as it
avoids the very pitfalls of outer fragmentation you have been telling us
about since RFC6864. Again, GUE for example uses a 40bit ID field - that
should be enough, shouldn't it?

Thanks - Fred

> Sent: Friday, July 15, 2016 4:06 PM

---

From: Joe Touch <[email-masked]>
Date: Mon, 18 Jul 2016 11:13:57 -0700

On 7/18/2016 11:06 AM, Templin, Fred L wrote:
> All three can be applied *on the same packet*;
What you call "mid-layer" and outer are both outer as far as the entire
system is concerned.

E.g., there can be zero or more of what you call "mid layer". The
management and coordination of mid and outer frag is the responsibility
of the ingress/egress pair, whereas inner fragmentation occurs in the
host (source frag) or router (on-path IPv4 DF=0 refrag) only.

That's why it's not particularly useful to treat outer and the zero or
more mid-layers as independent. Outer includes possibly many layers of
encapsulation, any of which might support frag/reassy.

> The RFC2764 mid-layer/tunnel-layer fragmentation 

That RFC's notion of mid-tunnel (it never says mid-layer) is on-path
frag of the outer layer.

> is very important as it
> about since RFC6864.
It is certainly useful that the *set* of layers used for what I call
outer fragmentation should support a variety of requirements. That's not
the point of this document, though.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 18 Jul 2016 18:19:22 +0000

Hi Joe,

The main point of this thread had to do with ICMP message source addresses.
With TUN/TAP interfaces used for point-to-multipoint purposes, the tun
interface has an MTU which is greater than or equal to the largest MTU of any
of the multipoint egresses (AERO calls these "neighbors"). So, for example, if
the tun interface has a 4K MTU, and neighbor N only has a 1500 MTU, the local
code within the interface code can produce a self-generated PTB using the
link-local address of N as the source address. Upper levels will then believe
that this PTB was generated by N, and not by the local interface.

This is the case for when the node that configures the TUN/TAP interface
is also the source of the original packet. When the node that configures
the TUN/TAP interface is not the original source, but is instead a router,
it instead uses one of its global addresses as the source address.

Thanks - Fred

> Sent: Friday, July 15, 2016 4:16 PM
> Subject: TUN/TAP vs tunnels
> 
> Hi, Fred (et al.),
>

---

From: Joe Touch <[email-masked]>
Date: Mon, 18 Jul 2016 11:25:21 -0700

Fred,

TUN and TAP are OS mechanisms, not network architecture components.

They may or may not be implemented according to any given spec.
Intarea-tunnels focuses on specs, not implementations.

How a given implementation generate ICMPs internally is not the subject
of this doc.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 18 Jul 2016 18:36:21 +0000

Hi Carlos,

> From: Carlos Pignataro (cpignata) [mailto:[email-masked]]
> Sent: Friday, July 15, 2016 9:50 PM
> Cc: Tom Herbert <[email-masked]>; [email-masked]

Sure, I can do that.

> RFC 4623 presents yet another way.

Thanks for the reference. It is interesting to see that the lead author of
that document was also co-author of RFC2764.

> I’m not too sure these are generalizable — nor really practical, at all.

The fragmentation/reassembly routines can and should be based on the
widely-deployed and well-tested fragmentation code available in all
implementations of the Internet Protocol.

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 18 Jul 2016 19:29:04 +0000

Hi Joe,

> Sent: Monday, July 18, 2016 11:14 AM

It is a real and tangible thing, and so needs a name - I don't care what it is
called as long as we can come up with something you won't complain about.
But, it is *not* outer fragmentation.

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 18 Jul 2016 19:29:30 +0000

Hi Joe,

> Sent: Monday, July 18, 2016 11:25 AM
> Subject: Re: TUN/TAP vs tunnels
> 
> Fred,
> 

Well, you asked a question and I gave an answer, and it seems that you are
now agreeing that an implementation can generate ICMPs internally.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Jul 2016 10:47:08 -0700

> But, it is *not* outer fragmentation.

We certainly can refine the terms here.

There are a few places where fragmentation occurs as far as tunnels are
concerned:

    A- before the message gets into the tunnel (intarea-tunnels calls
this "inner", but
        we aren't introducing a new mechanism. This is just "existing
fragmentation" in
        the layer in which the tunnel participates as a link.

    B- as part of the tunnel traversal during ingress encapsulation. We
can call this
        "tunneling fragmentation".

    C- inside IPv4 tunnels (that's a rare case that we do need to
mention as part of the
    IPv6 over IPv4 case only) - not sure this needs a separate name

Would those those terms (existing/tunneling fragmentation) be more clear?

Note - what 2764 "mid-layer" corresponds to C above.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Jul 2016 10:55:21 -0700

> now agreeing that an implementation can generate ICMPs internally.

A TAP enables a user process to emulate an L2 network. A TUN enables a
user process to emulate an L3 network.

Neither TUN nor TAP are necessarily tunnels in the intarea-tunnels
sense. How they behave depends on the user code associated with them -
and that code may or may not follow a given spec.

Your claim that these are counterexamples to the requirements in
intarea-tunnels is similar to claiming that TCP isn't implemented as a
"protocol laye"r because there exist implementations that use offload
devices.  Both are implementation conveniences, not architectural
inconsistencies.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 19 Jul 2016 20:04:39 +0000

Hi Joe,

?  Note - what 2764 "mid-layer" corresponds to C above.

This came as a total surprise; I would have been OK if you said it
corresponds to "B" above, but not "C".

Repeating myself yet a third time, a packet that is admitted into a tunnel could
incur three different levels of fragmentation:

1)      First, for IPv4 packets with DF=0, the packet could incur inner fragmentation

before any mid-layer encapsulations are applied

2)      Next, each inner packet or fragment is encapsulated in mid-layer encapsulations

and could then incur tunnel fragmentation.

3)      Finally, each mid-layer packet or fragment is encapsulated in outer encapsulations

and shipped to the egress. For IPv4, the outer packet could be fragmented by

an IPv4 router on the path to the egress.

So, the three layers of fragmentation are complimentary. You don't seem to
be disagreeing with that so much as you seem to be concerned with what it
is called. To my mind, we can and should say that we have inner fragmentation,
tunnel fragmentation and outer fragmentation.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Tuesday, July 19, 2016 10:47 AM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

There are a few places where fragmentation occurs as far as tunnels are concerned:

    A- before the message gets into the tunnel (intarea-tunnels calls this "inner", but
        we aren't introducing a new mechanism. This is just "existing fragmentation" in
        the layer in which the tunnel participates as a link.

    B- as part of the tunnel traversal during ingress encapsulation. We can call this
        "tunneling fragmentation".

    C- inside IPv4 tunnels (that's a rare case that we do need to mention as part of the
    IPv6 over IPv4 case only) - not sure this needs a separate name

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Jul 2016 13:14:46 -0700

On 7/19/2016 1:04 PM, Templin, Fred L wrote:
>
> Hi Joe,
>
>  
>
> Ø  Note - what 2764 "mid-layer" corresponds to C above.
>
>  
>
> corresponds to “B” above, but not “C”.
>
from that RFC, the only place where mid-tunnel is mentioned (mid-layer
is not):

   If the frame to be transferred is mapped into one IP datagram, normal
   IP fragmentation will occur when the IP datagram reaches a hop with
   an MTU smaller than the IP tunnel's MTU. This can have undesirable
   performance implications at the router performing such mid-tunnel
   fragmentation.

I.e., the tunnel is slower because the on-path router inside the  tunnel
is fragmenting.

> an IPv4 router on the path to the egress.
Yes, we agree that these are places where fragmentation could occur.
However, to the tunnel, both 2 and 3 are part of ingress encapsulation.

> So, the three layers of fragmentation are complimentary.
>

I agree they all can occur.

Each of the zero or more layers in #2 above is equally distinct, though
- so there are possibly many other layers to itemize.

To the Internet seen by the TTP (the IP packet arriving at the node
which might have a tunnel interface), there are only two places:

    - fragmentation within the TTP layer (source frag, on-path IPv4 DF=0
frag)

    - fragmentation within the link layer

        The TTP doesn't know or care about how many layers this takes.
It's all link to the TTP.

> You don’t seem to

Tunnel fragmentation IS outer (#3) and additional layer (however many
#2s there are). The tunnel mechanism may care about the difference, but
the TTP does not and should not.

I'd really like to know if this is as confusing to others, though.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 19 Jul 2016 20:17:55 +0000

Hi Joe,

Forget "TAP" for the moment, and let's just think about TUN. The TUN interface
is seen as an ordinary network interface to the IP layer. When IP injects a packet
into the TUN interface, however, the packet is intercepted before it is admitted
to L2, and can then be manipulated in all sorts of ways before it is finally released
to L2. The code inside the TUN interface could then do things like check the packet
size against the egress MTU and drop the packet and return a PTB if necessary.

So, I will agree that the *IP layer* could see the MTU value for the TUN interface
as a constant value for all egresses when the interface is NBMA and/or multipoint.
But, the TUN interface could configure an MTU that is as large as the largest MTU
among all egresses and let the code inside the TUN interface return a PTB if
necessary.

Thanks - Fred

> Sent: Tuesday, July 19, 2016 10:55 AM

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 19 Jul 2016 20:21:43 +0000

I am sorry Joe; that's not right. Tunnel fragmentation is tunnel fragmentation;
it is not outer fragmentation. Outer fragmentation is outer fragmentation.

Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Tuesday, July 19, 2016 1:15 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

This came as a total surprise; I would have been OK if you said it
corresponds to "B" above, but not "C".
from that RFC, the only place where mid-tunnel is mentioned (mid-layer is not):

   fragmentation.

I.e., the tunnel is slower because the on-path router inside the  tunnel is fragmenting.

an IPv4 router on the path to the egress.
Yes, we agree that these are places where fragmentation could occur. However, to the tunnel, both 2 and 3 are part of ingress encapsulation.

Each of the zero or more layers in #2 above is equally distinct, though - so there are possibly many other layers to itemize.

To the Internet seen by the TTP (the IP packet arriving at the node which might have a tunnel interface), there are only two places:

    - fragmentation within the TTP layer (source frag, on-path IPv4 DF=0 frag)

        The TTP doesn't know or care about how many layers this takes. It's all link to the TTP.

You don't seem to

Tunnel fragmentation IS outer (#3) and additional layer (however many #2s there are). The tunnel mechanism may care about the difference, but the TTP does not and should not.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Jul 2016 13:36:05 -0700

On 7/19/2016 1:17 PM, Templin, Fred L wrote:
> is seen as an ordinary network interface to the IP layer.

It should be, if implemented correctly.

>  When IP injects a packet
> to L2. 
It is sent to a user process. What happens after that is up to the user
process.

> The code inside the TUN interface could then do things like check the packet

The only reason why the code inside the TUN would do this is because the
code is emulating a network, and the PTB is coming from a
software-emulated router inside that network. I.e., it's not the TUN
"network interface" that's sending the PTB; it's the network to which
the TUN attaches, as implemented in the user process. See below for detail.

> among all egresses and let the code inside the TUN interface return a PTB if

That's the equivalent of this:

TUN-----VR1-----VR2-----....
          L1          L2

Where L1 and L2 have different MTUs.

I.e., the PTB you would get back would be from VR1, reporting that it's
interface to L2 has a smaller MTU than can be accommodated by L1.

At this point, you're inside the complexity of the fact that the code
behind a TUN device implements a L3 network, and that your first-hop MTU
isn't correct.

That has very little to do with the idea of an MTU varying within a
given link layer, and only occurs because a TUN device allows user code
to emulate a full network.

I.e., the ICMP here isn't really "generated" by the TUN "network
interface"; it's generated by the software emulated the virtual routers
inside the virtual network.

There's nothing per se wrong with that, but that has nothing to do with
the situation of a single L2 link being expected to have a single MTU,
which IMO is a existing assumption of the Internet.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Jul 2016 13:39:25 -0700

Fred,

Let's start with a few fundamentals...

First, do you at least agree that 2764 mid-tunnel isn't what you thought?

Do you also agree that - to the TTP - both what you call "tunnel frag"
and "outer frag" are both part of what happens invisibly inside what the
TTP considers L2?

If you agree to the two above, I don't see what the benefit of getting
into the possibly large number of mechanisms inside that "L2" serves
*for this doc*.

Joe

On 7/19/2016 1:21 PM, Templin, Fred L wrote:
>
> I am sorry Joe; that’s not right. Tunnel fragmentation is tunnel
> fragmentation;
>
> *From:*Joe Touch [mailto:[email-masked]]
> *Sent:* Tuesday, July 19, 2016 1:15 PM
> *To:* Templin, Fred L <[email-masked]>; [email-masked]
> *Subject:* Re: intarea-tunnels meta-comment: tunnel fragmentation
>
>  
>
>  
>
>  
>
> Each of the zero or more layers in #2 above is equally distinct,
> though - so there are possibly many other layers to itemize.
>
> #2s there are). The tunnel mechanism may care about the difference,
> but the TTP does not and should not.
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 19 Jul 2016 20:57:30 +0000

Hi Joe,

?  First, do you at least agree that 2764 mid-tunnel isn't what you thought?

If it isn't what I thought, then it isn't what Bob B. told me it was and I took
Bob's word as gospel.

If it isn't what I thought, then what is in AERO, GRE extensions and GUE

extensions is something new and better than the way things have been done

in the past.

If it isn't what I thought, then it is still a type of fragmentation that is distinct

from and part from inner and outer fragmentation - call it "tunnel fragmentation".

But, looking at RFC2764 it looks like it *is* what I thought where it says:

At least, that is what AERO and the GRE/GUE extensions are doing.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Tuesday, July 19, 2016 1:39 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

Fred,

Do you also agree that - to the TTP - both what you call "tunnel frag" and "outer frag" are both part of what happens invisibly inside what the TTP considers L2?

If you agree to the two above, I don't see what the benefit of getting into the possibly large number of mechanisms inside that "L2" serves *for this doc*.

Joe

Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Tuesday, July 19, 2016 1:15 PM
To: Templin, Fred L <[email-masked]><mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

   fragmentation.

You don't seem to

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Jul 2016 14:06:50 -0700

On 7/19/2016 1:57 PM, Templin, Fred L wrote:
>
> Hi Joe,
>
> Ø  First, do you at least agree that 2764 mid-tunnel isn't what you
> thought?
>
> If it isn’t what I thought, then it isn’t what Bob B. told me it was
> and I took
> Bob’s word as gospel.
>
> If it isn’t what I thought, then what is in AERO, GRE extensions and GUE
>
> If it isn’t what I thought, then it is still a type of fragmentation
> that is distinct
>
> from and part from inner and outer fragmentation – call it “tunnel
> fragmentation”.
>
>
> But, looking at RFC2764 it looks like it **is** what I thought where
> it says:
>
>  
>
Here's the difference. The full text is:

This means that the "alternative approach" is NOT "mid-tunnel
fragmentation".

The "mid-tunnel" paragraph thus assumes IPv4 on-path fragmentation
(i.e., mid-tunnel), not source fragmentation.

The third paragraph is entirely consistent with IPv4 or IPv6 source
fragmentation at the "outer layer". It does not imply the need for an
additional protocol layer beyond what IP provides. I agree that IPv4 is
weak in this regard due to its small ID space and that both IPv4 and
IPv6 are weak in terms of providing additional validity checks, but a
tunnel that uses UDP inside IPv6 might provide as good protection as is
necessary. There's no strict need that either a "shim" (i.e., not
traditional L3 or L4) layer is needed nor that the fragmentation occur
at any particular layer (e.g., in the shim) rather than in the outermost
IP layer.

The doc is inaccurate in this regard even for its day; IP source
fragmentation was used at the time.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 19 Jul 2016 21:23:38 +0000

Hi Joe,

>From RFC2764:

   end-of-message marker of some sort.

"tunnel sequence number and an end-of-message marker of some sort" imply

some sort of shim header between the inner and outer IP layers. For GUE,

that would be the UDP header, the GUE header and the GUE fragment header.

For GRE, it would be the GRE header and the GRE fragment header.

So, RFC2764 is not a specification of an exact encapsulation format, but

it articulates the spirit of what we are trying to capture here.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Tuesday, July 19, 2016 2:07 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

in the past.

   fragmentation.

This means that the "alternative approach" is NOT "mid-tunnel fragmentation".

The "mid-tunnel" paragraph thus assumes IPv4 on-path fragmentation (i.e., mid-tunnel), not source fragmentation.

The third paragraph is entirely consistent with IPv4 or IPv6 source fragmentation at the "outer layer". It does not imply the need for an additional protocol layer beyond what IP provides. I agree that IPv4 is weak in this regard due to its small ID space and that both IPv4 and IPv6 are weak in terms of providing additional validity checks, but a tunnel that uses UDP inside IPv6 might provide as good protection as is necessary. There's no strict need that either a "shim" (i.e., not traditional L3 or L4) layer is needed nor that the fragmentation occur at any particular layer (e.g., in the shim) rather than in the outermost IP layer.

The doc is inaccurate in this regard even for its day; IP source fragmentation was used at the time.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Jul 2016 14:42:08 -0700

On 7/19/2016 2:23 PM, Templin, Fred L wrote:
> “tunnel sequence number and an end-of-message marker of some sort” imply
> some sort of shim header between the inner and outer IP layers. 
For IP, these values are represented by the offset value and MF field value.

> For GUE,

It indicates that you need to support frag/reassembly.

It also gets into how, but that's not relevant for intarea-tunnels.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 19 Jul 2016 21:48:13 +0000

Hi Joe,

?  For IP, these values are represented by the offset value and MF field value.

GUE extensions and GRE extensions also use offset and MF. But, those fields appear
in an encapsulation header and not an IP header.

?  It indicates that you need to support frag/reassembly.

?  It also gets into how, but that's not relevant for intarea-tunnels.

Fragmentation and reassembly at the tunnel layer - not the inner or outer IP
layers. That is certainly relevant.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Tuesday, July 19, 2016 2:42 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

For GUE,

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 19 Jul 2016 23:56:19 +0200

On Tue, Jul 19, 2016 at 11:48 PM, Templin, Fred L
<[email-masked]> wrote:
> Hi Joe,
>
>
>
> Ø  For IP, these values are represented by the offset value and MF field
> value.
>
>
>
> Ø  It indicates that you need to support frag/reassembly.
>
> Ø  It also gets into how, but that's not relevant for intarea-tunnels.
>
>
>
> Fragmentation and reassembly at the tunnel layer – not the inner or outer IP
>
I wonder if this just another item that needs to be considered in each
encapsulation protocol, sort of like the way congestion control and
UDP checksum handling for UDP encapsulation need to described for each
protocol. Those seems to be handled is by a fair amount of cut and
paste from earlier RFCs. In the case of tunnel fragmentation, maybe
GUE could be the prototype (or guinea pig if you prefer :-) ) and
future encaps can just adapt the mechanisms.

Tom

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 19 Jul 2016 22:15:01 +0000

Hi Tom,

> Sent: Tuesday, July 19, 2016 2:56 PM

SEAL [RFC5320] also specifies tunnel fragmentation, and I have published
experimental code for an earlier version of the spec:

  http://isatap.com/seal/

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 19 Jul 2016 22:28:28 +0000

Hi again Tom,

> From: Int-area [mailto:[email-masked]] On Behalf Of Templin, Fred L
> Sent: Tuesday, July 19, 2016 3:15 PM
> To: Tom Herbert <[email-masked]>
> Cc: [email-masked]

Sorry; that link was old. You can find better code for RFC5320(bis) here:

http://linkupnetworks.com/seal/sealv2-1.0.tgz

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Jul 2016 16:28:43 -0700

On 7/19/2016 2:48 PM, Templin, Fred L wrote:
>
> Hi Joe,
>
>  
>
> Ø  For IP, these values are represented by the offset value and MF
> field value.
>
>  
>

Yes. You can use frag/reassembly in either or both layers.

> layers. That is certainly relevant.
Both layers are L2 to the TTP. The TTP doesn't know or care about the
difference.

The difference involves how the tunnel designer wants to implement the
tunnel. This doc sets requirements for that (such as that frag/reassy is
required) but doesn't indicate a specific solution.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Jul 2016 16:30:20 -0700

> future encaps can just adapt the mechanisms.
I don't see how intermediate encapsulation details are relevant as
distinguished from the outer layer, at least for this doc. All are
"tunnel encapsulation" - encapsulation that gets you across the tunnel.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jul 2016 15:10:00 +0000

Hi Joe,

Inner fragmentation is what happens *before* encapsulation.
Tunnel fragmentation is what happens *during* encapsulation.
Outer fragmentation is what happens *after* encapsulation.

The document needs to say something about this for completion.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Tuesday, July 19, 2016 4:29 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

Fragmentation and reassembly at the tunnel layer - not the inner or outer IP
layers. That is certainly relevant.
Both layers are L2 to the TTP. The TTP doesn't know or care about the difference.

The difference involves how the tunnel designer wants to implement the tunnel. This doc sets requirements for that (such as that frag/reassy is required) but doesn't indicate a specific solution.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 08:43:35 -0700

On 7/20/2016 8:10 AM, Templin, Fred L wrote:
>
> Hi Joe,
>
>  
>
> Inner fragmentation is what happens **before** encapsulation.
>
> Tunnel fragmentation is what happens **during** encapsulation.
>
> Outer fragmentation is what happens **after** encapsulation.
>

It's very clear that inner and outer are confusing terms.

Inner fragmentation happens in the base IP network in which the tunnel
participates (I think this is agreed).

The current document does not describe mid-tunnel fragmentation (yet -
it will for the IPv6 over IPv4 case).

Outer fragmentation as described in intarea-tunnels happens as part of
encapsulation. Encapsulation involves adding one or more headers and
processing each. To the tunnel system, there is no difference between
the fragmentation that might occur in the GUE encapsulation step vs. the
final IP encapsulation step.

Let's now take this as two steps:

    1. why do you believe it is important for tunnels-intarea to
differentiate intermediate vs the last encapsulation and source
fragmentation step performed by a tunnel ingress?

    2. (which we really can't answer until there's convergence on 1)
what name should we use for each step?
           I agree that inner/outer isn't correct, but we can't call
intermediate layers "tunnel" layers because there are plenty of tunnels
that would then have no tunnel layer, which doesn't make sense.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jul 2016 16:09:20 +0000

Hi Joe,

Inner fragmentation, tunnel fragmentation and outer fragmentation are the
three layers at which fragmentation can be applied. Outer fragmentation is
understood by all to be IP fragmentation applied at the outer IP layer. We
know that this kind of fragmentation is broken due to IP ID limitations and
middleboxes that unconditionally drop IP fragments. Tunnel fragmentation
fixes both of these problems, and is why all tunnels that can do this should.
I think that bears mention in the document.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 8:44 AM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

Inner fragmentation happens in the base IP network in which the tunnel participates (I think this is agreed).

The current document does not describe mid-tunnel fragmentation (yet - it will for the IPv6 over IPv4 case).

Outer fragmentation as described in intarea-tunnels happens as part of encapsulation. Encapsulation involves adding one or more headers and processing each. To the tunnel system, there is no difference between the fragmentation that might occur in the GUE encapsulation step vs. the final IP encapsulation step.

    1. why do you believe it is important for tunnels-intarea to differentiate intermediate vs the last encapsulation and source fragmentation step performed by a tunnel ingress?

    2. (which we really can't answer until there's convergence on 1) what name should we use for each step?
           I agree that inner/outer isn't correct, but we can't call intermediate layers "tunnel" layers because there are plenty of tunnels that would then have no tunnel layer, which doesn't make sense.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 10:01:49 -0700

> I think that bears mention in the document.

There are a variety of factors in considering "ingress encapsulation and
fragmentation" (let's call that IETF for brevity, though that might
still not be the final name), which encompasses both intermediate and
what you call "outer" (but not what the doc calls "outer").

There are a variety of hazards with IEF, which include whether tunnel
transit devices or egresses support the protocol, the performance limits
thereof (e.g., fragmentation ID space), etc. What you call "tunnel
fragmentation" causes as many problems as it fixes, e.g, it might not
transit middleboxes or it might not be supported at the egress, so it's
not appropriate to merely claim this is a magic solution.

And I agree that there are many boxes that claim to support Internet
protocols that are incomplete or incorrect. The tide is turning on that
issue in many places - e.g., V6OPS is no longer calling for the
deprecation of IPv6 extension headers and similar attempts in various
venues to deprecate fragmentation have also failed. So we should not
base conclusions in this document on those assumptions.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jul 2016 17:22:52 +0000

Hi Joe,

There are other reasons for using tunnel fragmentation, but let's talk
only about the IP ID for now:

IPv4 has a 16bit IP ID and a 120sec MSL, and we know that that tunnels
would need to go way too slow for modern technologies in order to
avoid reassembly misassociations.

IPv6 has a 32bit IP ID and a 60sec MSL, which is better but even that is
not enough at data rates that are approachable by modern technologies.

GUE/GRE fragmentation use a 40bit ID and a 60sec MSL, which should
be enough to accommodate any conceivable data rates.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 10:02 AM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

There are a variety of factors in considering "ingress encapsulation and fragmentation" (let's call that IETF for brevity, though that might still not be the final name), which encompasses both intermediate and what you call "outer" (but not what the doc calls "outer").

There are a variety of hazards with IEF, which include whether tunnel transit devices or egresses support the protocol, the performance limits thereof (e.g., fragmentation ID space), etc. What you call "tunnel fragmentation" causes as many problems as it fixes, e.g, it might not transit middleboxes or it might not be supported at the egress, so it's not appropriate to merely claim this is a magic solution.

And I agree that there are many boxes that claim to support Internet protocols that are incomplete or incorrect. The tide is turning on that issue in many places - e.g., V6OPS is no longer calling for the deprecation of IPv6 extension headers and similar attempts in various venues to deprecate fragmentation have also failed. So we should not base conclusions in this document on those assumptions.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 10:37:08 -0700

On 7/20/2016 10:22 AM, Templin, Fred L wrote:
>
> Hi Joe,
>
>  
>
> There are other reasons for using tunnel fragmentation, but let’s talk
>
Only for non-atomic datagrams.RFC6864 relaxes the uniqueness requirement
for atomic datagrams.

> not enough at data rates that are approachable by modern technologies.
First, that would apply only to non-atomic (for IPv6, just non
source-fragmented) datagrams.

Second, that's a 4G ID space which would apply only for datagrams over
1280B. In the worst case, that would be 1281B, or over 10K bits. The
result is 683Gbps over a single channel, and is a limit only if all the
datagrams are too large. We're only starting to approach 400Gbps and I
doubt we expect tunneling system to support that using a single
ingress/egress pair. Merely assigning a handful of IP addresses to the
ingress/egress pair alleviates this problem completely, and given IPv6
address spaces that should be trivial.

> be enough to accommodate any conceivable data rates.

And might not traverse a NAT. Remember too that IP in IP tunneling is
the default for IPsec tunnel mode.

Yes, it's worth mentioning that the *entire set of protocols* used for
egress encapsulation and fragmentation should support frag/reassembly at
the expected capacity of the tunnel given the expected load of incoming
TTP packets that it might need to fragment.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jul 2016 17:47:35 +0000

Hi Joe,

We are talking now about fragmentation, and not atomic datagrams. And, it
must be agreed that IPv4 fragmentation at high data rates is unacceptable
per RFC4963.

Your math for IPv6 fragmentation does show a potential RFC4963 scenario
for conceivable future data rates. That would not happen with a 40bit ID.

Your comment about NATs is not accurate since GUE uses UDP encapsulation.
True that IP/GRE/IP encapsulation would not necessarily traverse a NAT, but
IP/UDP/GRE/IP would.

Again, we have inner fragmentation *before* encapsulation, tunnel
fragmentation *during* encapsulation and outer fragmentation *after*
encapsulation. It should be very straightforward to note this in your
document.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 10:37 AM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

IPv4 has a 16bit IP ID and a 120sec MSL, and we know that that tunnels
would need to go way too slow for modern technologies in order to
avoid reassembly misassociations.
Only for non-atomic datagrams.RFC6864 relaxes the uniqueness requirement for atomic datagrams.

IPv6 has a 32bit IP ID and a 60sec MSL, which is better but even that is
not enough at data rates that are approachable by modern technologies.
First, that would apply only to non-atomic (for IPv6, just non source-fragmented) datagrams.

Second, that's a 4G ID space which would apply only for datagrams over 1280B. In the worst case, that would be 1281B, or over 10K bits. The result is 683Gbps over a single channel, and is a limit only if all the datagrams are too large. We're only starting to approach 400Gbps and I doubt we expect tunneling system to support that using a single ingress/egress pair. Merely assigning a handful of IP addresses to the ingress/egress pair alleviates this problem completely, and given IPv6 address spaces that should be trivial.

And might not traverse a NAT. Remember too that IP in IP tunneling is the default for IPsec tunnel mode.

Yes, it's worth mentioning that the *entire set of protocols* used for egress encapsulation and fragmentation should support frag/reassembly at the expected capacity of the tunnel given the expected load of incoming TTP packets that it might need to fragment.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 11:20:18 -0700

On 7/20/2016 10:47 AM, Templin, Fred L wrote:
>
> Hi Joe,
>
>  
>
> We are talking now about fragmentation, and not atomic datagrams.
>

It matters because atomic datagrams can be transmitted without consuming
ID space.

> And, it

Agreed.

> for conceivable future data rates. That would not happen with a 40bit ID.
That scenario makes assumptions about the entire channel being used for
a single tunnel, which is nowhere near the typical use for links that
are that fast.

> True that IP/GRE/IP encapsulation would not necessarily traverse a
UDP doesn't traverse all NATs.

> Again, we have inner fragmentation **before** encapsulation, tunnel
>
> fragmentation **during** encapsulation and outer fragmentation **after**
>
Outer fragmentation (as you're using the term, which again is not
consistent with the current doc) happens as part of encapsulation too.
It's not "after" anything.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jul 2016 18:35:11 +0000

Hi Joe,

?  Outer fragmentation (as you're using the term, which again is not consistent with the current doc) happens as part of encapsulation too. It's not "after" anything.

I disagree with that. With outer fragmentation, you first encapsulate in an outer IP
header *and then* apply fragmentation. And, that is only for source fragmentation;
for network fragmentation, outer fragmentation is performed by a router on the
path which may be many hops down the line from the ingress. But, both types
occur *after* encapsulation.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 11:20 AM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

It matters because atomic datagrams can be transmitted without consuming ID space.

And, it

Agreed.

Your math for IPv6 fragmentation does show a potential RFC4963 scenario
for conceivable future data rates. That would not happen with a 40bit ID.
That scenario makes assumptions about the entire channel being used for a single tunnel, which is nowhere near the typical use for links that are that fast.

Again, we have inner fragmentation *before* encapsulation, tunnel
fragmentation *during* encapsulation and outer fragmentation *after*
encapsulation. It should be very straightforward to note this in your
document.
Outer fragmentation (as you're using the term, which again is not consistent with the current doc) happens as part of encapsulation too. It's not "after" anything.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 11:44:06 -0700

On 7/20/2016 11:35 AM, Templin, Fred L wrote:
>
> Hi Joe,
>
>  
>
> Ø  Outer fragmentation (as you're using the term, which again is not
> header **and then** apply fragmentation.
>
There's no sense in which the packet needs to be fully formed first
before being fragmented - this applies to all fragmentation, whether
GUE, AERO, or IP.

> And, that is only for source fragmentation;
>
> occur **after** encapsulation.
>

Encapsulation that happens downstream is mid-tunnel.

Again, that's a separate case. That's why I think its useful to
distinguish tunnel ingress encapsulation and fragmentation processing
from other processing and give them two different names.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jul 2016 19:53:35 +0000

Hi Joe,

This is all much simpler than you are making it out to be. The steps are:

1)      Perform inner fragmentation on the original IP packet if necessary

2)      Encapsulate the original IP packet (or fragments) in a tunnel

encapsulation header (e.g., GUE, GRE, etc.)

3)      Perform tunnel fragmentation on each such tunnel packet if necessary

4)      Encapsulate each tunnel packet in an outer IP/* header

5)      Perform outer fragmentation on each such outer IP packet

intarea-tunnels currently talks only about steps 1), 4) and 5). It also needs to
talk about steps 2) and 3).

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 11:44 AM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

I disagree with that. With outer fragmentation, you first encapsulate in an outer IP
header *and then* apply fragmentation.
There's no sense in which the packet needs to be fully formed first before being fragmented - this applies to all fragmentation, whether GUE, AERO, or IP.

Again, that's a separate case. That's why I think its useful to distinguish tunnel ingress encapsulation and fragmentation processing from other processing and give them two different names.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 13:18:08 -0700

> 1)      Perform inner fragmentation on the original IP packet if necessary
That is already in the doc.

> 5)      Perform outer fragmentation on each such outer IP packet

That is already in the doc as the integrated process it appears as far
as the tunnel is concerned.

Yes, the "outer encapsulation" steps need to be more clear that this may
involve multiple levels of protocol, but that's all part of the tunnel
ingress processing.

And, to be complete, there's the "exception" case of IPv6 over IPv4,
which involves both the tunnel ingress processing AND relying on the
tunnel path components to refragment the IPv4 datagrams as needed.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jul 2016 21:01:51 +0000

Joe,

Your draft is missing discussion of tunnel fragmentation, which is necessary to
avoid the hazards of IP fragmentation - you need to add a discussion of steps
2-3 in my list below.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 1:18 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

That is already in the doc as the integrated process it appears as far as the tunnel is concerned.

Yes, the "outer encapsulation" steps need to be more clear that this may involve multiple levels of protocol, but that's all part of the tunnel ingress processing.

And, to be complete, there's the "exception" case of IPv6 over IPv4, which involves both the tunnel ingress processing AND relying on the tunnel path components to refragment the IPv4 datagrams as needed.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 14:17:24 -0700

Fred,

I'm going to wait for someone else on the list to confirm that they too
do not understand that egress processing includes as many levels of
fragmentation as are needed to implement the "link" of a tunnel.

Joe

On 7/20/2016 2:01 PM, Templin, Fred L wrote:
> avoid the hazards of IP fragmentation – you need to add a discussion
> of steps
>
> 2-3 in my list below.
>
>  
>
> Thanks - Fred
>
>  
>
> *Sent:* Wednesday, July 20, 2016 1:18 PM
> Yes, the "outer encapsulation" steps need to be more clear that this
> may involve multiple levels of protocol, but that's all part of the
> tunnel ingress processing.
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jul 2016 23:38:44 +0000

Hi Joe,

For completion, your document needs to discuss tunnel fragmentation,
and needs to cite RFC2764, Section 3.1.7.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 2:17 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

Fred,

I'm going to wait for someone else on the list to confirm that they too do not understand that egress processing includes as many levels of fragmentation as are needed to implement the "link" of a tunnel.

Joe

Thanks - Fred

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 16:43:38 -0700

Fred,

RFC2764 is inconsistent with intarea-tunnels. There are plenty of
tunnels that do not use any of what you call tunnel fragmentation, yet
they are tunnels and they do fragment and will work just fine for
existing network technologies.

I have already confirmed that the doc will be updated to indicate that
egress processing may involve more than one level of encapsulation and
fragmentation, but there is insufficient justification for this document
to detail that issue further.

Joe

On 7/20/2016 4:38 PM, Templin, Fred L wrote:
> *Sent:* Wednesday, July 20, 2016 2:17 PM
> I'm going to wait for someone else on the list to confirm that they
> too do not understand that egress processing includes as many levels
> of fragmentation as are needed to implement the "link" of a tunnel.
>
> Joe
>
>     avoid the hazards of IP fragmentation – you need to add a
>     discussion of steps
>
>     2-3 in my list below.
>
>      
>
>     Thanks - Fred
>
>      
>
>     *To:* Templin, Fred L <[email-masked]>
>     <mailto:[email-masked]>; [email-masked]
>     <mailto:[email-masked]>
>     Yes, the "outer encapsulation" steps need to be more clear that
>     this may involve multiple levels of protocol, but that's all part
>     of the tunnel ingress processing.
>
>     And, to be complete, there's the "exception" case of IPv6 over
>     IPv4, which involves both the tunnel ingress processing AND
>     relying on the tunnel path components to refragment the IPv4
>     datagrams as needed.
>
>     Joe
>
>  
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jul 2016 23:56:01 +0000

Hi Joe,

?  RFC2764 is inconsistent with intarea-tunnels. There are plenty of tunnels that do not use any of what you call tunnel fragmentation, yet they are tunnels and they do fragment and will work just fine for existing network technologies.

For tunnels that cannot do RFC2764 tunnel fragmentation, we simply say that tunnel
fragmentation cannot be used and that steady state fragmentation of those tunnel
types could cause problems. That would not be inconsistent with intarea-tunnels.

Besides, what tunnels over IPv4 do you know of that make steady state use of outer
IPv4 fragmentation and work just fine?

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 4:44 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

Fred,

RFC2764 is inconsistent with intarea-tunnels. There are plenty of tunnels that do not use any of what you call tunnel fragmentation, yet they are tunnels and they do fragment and will work just fine for existing network technologies.

I have already confirmed that the doc will be updated to indicate that egress processing may involve more than one level of encapsulation and fragmentation, but there is insufficient justification for this document to detail that issue further.

Joe

Thanks - Fred

Fred,

Joe

Thanks - Fred

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 17:11:51 -0700

On 7/20/2016 4:56 PM, Templin, Fred L wrote:
>
> Hi Joe,
>
>  
>
> Ø  RFC2764 is inconsistent with intarea-tunnels. There are plenty of

In intarea-tunnels, a tunnel uses encapsulation to enable a network to
act as a link. That means that this encapsulation *is* a tunneling
protocol, which means that fragmentation at that layer *is* tunnel
fragmentation.

And steady-state fragmentation is a problem for *any* set of tunneling
protocols whose *combined* fragment identity space is insufficient to
support the maximum tunnel capacity. That's currently a problem only for
IP in IPv4, and yes, that should be noted.

> IPv4 fragmentation and work just fine?

Every IPv4 IPsec tunnel mode tunnel. IPsec tunnel mode is just transport
mode over an IP in IP tunnel (with some exceptions in the details of
forwarding vs. SPI lookup, as noted in RFC 3884.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 00:32:32 +0000

Hi Joe,

The RFC2764, Section 3.1.7 reference needs to be in there, as well as
discussion of inner, outer, *and* tunnel fragmentation. I have made
it very clear what is meant by "tunnel fragmentation".

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 5:12 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

In intarea-tunnels, a tunnel uses encapsulation to enable a network to act as a link. That means that this encapsulation *is* a tunneling protocol, which means that fragmentation at that layer *is* tunnel fragmentation.

And steady-state fragmentation is a problem for *any* set of tunneling protocols whose *combined* fragment identity space is insufficient to support the maximum tunnel capacity. That's currently a problem only for IP in IPv4, and yes, that should be noted.

Every IPv4 IPsec tunnel mode tunnel. IPsec tunnel mode is just transport mode over an IP in IP tunnel (with some exceptions in the details of forwarding vs. SPI lookup, as noted in RFC 3884.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 17:37:31 -0700

Fred,

I have made it clear why this is inconsistent with this this document.

For this topic as well, I need to hear someone else who confirms that
this is critical, AND we need to then resolve the inconsistency in
terminology of that outdated document.

Joe

On 7/20/2016 5:32 PM, Templin, Fred L wrote:
> discussion of inner, outer, **and** tunnel fragmentation. I have made
>
> it very clear what is meant by “tunnel fragmentation”.
>
>  
>
> Thanks - Fred
>
>  
>
> *Sent:* Wednesday, July 20, 2016 5:12 PM
>     Ø  RFC2764 is inconsistent with intarea-tunnels. There are plenty
>     of tunnels that do not use any of what you call tunnel
>     fragmentation, yet they are tunnels and they do fragment and will
>     work just fine for existing network technologies.
>
>      
>
> support the maximum tunnel capacity. That's currently a problem only
> for IP in IPv4, and yes, that should be noted.
>
>
>      
>
> Every IPv4 IPsec tunnel mode tunnel. IPsec tunnel mode is just
> transport mode over an IP in IP tunnel (with some exceptions in the
> details of forwarding vs. SPI lookup, as noted in RFC 3884.
>
> Joe
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 01:12:34 +0000

Hi Joe,

?  I have made it clear why this is inconsistent with this this document.

Your document needs to be made consistent with RFC2764; not the other way around.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 5:38 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

Fred,

For this topic as well, I need to hear someone else who confirms that this is critical, AND we need to then resolve the inconsistency in terminology of that outdated document.

Joe

Thanks - Fred

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 19:33:12 -0700

Please see the end sections of intarea-tunnels. This doc is intended to resolve inconsistencies, not propagate them. Besides, that doc was only ever informational anyway and we should be aiming for BCP. 

Joe 

> On Jul 20, 2016, at 6:12 PM, Templin, Fred L <[email-masked]> wrote:
> 
> Hi Joe,
>  
> Ø  I have made it clear why this is inconsistent with this this document.
>  
> Ø  RFC2764 is inconsistent with intarea-tunnels. There are plenty of tunnels that do not use any of what you call tunnel fragmentation, yet they are tunnels and they do fragment and will work just fine for existing network technologies.
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 18:02:32 +0000

Joe,

Tunnel fragmentation is essential for tunnels that cannot safely use outer fragmentation.
The two are *different* and both need to be discussed in intarea-tunnels, with a reference
to RFC2764.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 7:33 PM
To: Templin, Fred L <[email-masked]>
Cc: [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

Joe

On Jul 20, 2016, at 6:12 PM, Templin, Fred L <[email-masked]<mailto:[email-masked]>> wrote:
Hi Joe,

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 5:38 PM
To: Templin, Fred L <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

Fred,

Joe

Thanks - Fred

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 21 Jul 2016 11:20:39 -0700

> Tunnel fragmentation is essential for tunnels that cannot safely use

And outer fragmentation is essential for tunnels that do not support
intermediate layers of egress fragmentation.

And both are required when only the combination suffices.

> The two are **different** and both need to be discussed in
> intarea-tunnels, with a reference
>
> to RFC2764.
>

I've already said why they are not for this document. This document does
need to explain that egress encapsulation and fragmentation MAY involve
more than one layer of encapsulation and fragmentation and that the
behavior of the COMBINATION of those layers has some requirements, but
there is no reason IN THIS DOC to get into any further details. This doc
does not indicate a specific solution.

Again, though, THIS DOCUMENT will not use the term "tunnel
fragmentation" the way you have been doing *because* it is incorrect.
The outermost level of encapsulation and fragmentation is a much a part
of "tunneling" as any other egress encapsulation layer, whether other
layers exist of not. The entire set is really what ought to be called
"tunnel encapsulation and fragmentation", but RFC2764 uses the term
incorrectly and thus we should not propagate the confusion by using the
same term.

THIS DOC is about cleaning up the confusion that issues in a variety of
docs have created.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 20:57:53 +0000

Ø  This document does need to explain that egress encapsulation and fragmentation MAY involve more than one layer of encapsulation and fragmentation and that the behavior of the COMBINATION of those layers has some requirements

What would you propose as text?

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Thursday, July 21, 2016 11:21 AM
To: Templin, Fred L <[email-masked]>
Cc: [email-masked]
Subject: Re: intarea-tunnels meta-comment: tunnel fragmentation

And outer fragmentation is essential for tunnels that do not support intermediate layers of egress fragmentation.

I've already said why they are not for this document. This document does need to explain that egress encapsulation and fragmentation MAY involve more than one layer of encapsulation and fragmentation and that the behavior of the COMBINATION of those layers has some requirements, but there is no reason IN THIS DOC to get into any further details. This doc does not indicate a specific solution.

Again, though, THIS DOCUMENT will not use the term "tunnel fragmentation" the way you have been doing *because* it is incorrect. The outermost level of encapsulation and fragmentation is a much a part of "tunneling" as any other egress encapsulation layer, whether other layers exist of not. The entire set is really what ought to be called "tunnel encapsulation and fragmentation", but RFC2764 uses the term incorrectly and thus we should not propagate the confusion by using the same term.

THIS DOC is about cleaning up the confusion that issues in a variety of docs have created.

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 21 Jul 2016 14:05:02 -0700

On 7/21/2016 1:57 PM, Templin, Fred L wrote:
>
> Ø  This document does need to explain that egress encapsulation and
> fragmentation MAY involve more than one layer of encapsulation and
> fragmentation and that the behavior of the COMBINATION of those layers
> has some requirements
>
>  
>

I'll propose that in the next rev, when I have cycles.

Joe