Subject: [Int-area] I-D Action: draft-ietf-intarea-tunnels-03.txt

From: [email-masked]
Date: Wed, 06 Jul 2016 23:28:05 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Internet Area Working Group of the IETF.

        Title           : IP Tunnels in the Internet Architecture
        Authors         : Joe Touch
                          Mark Townsley
	Filename        : draft-ietf-intarea-tunnels-03.txt
	Pages           : 47
	Date            : 2016-07-06

Abstract:
   This document discusses the role of IP tunnels in the Internet
   architecture, in which IP datagrams are carried as payloads in non-
   link layer protocols. It explains their relationship to existing
   protocol layers and the challenges in supporting IP tunneling based
   on the equivalence of tunnels to links.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-intarea-tunnels/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-intarea-tunnels-03

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-intarea-tunnels-03

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Joe Touch <[email-masked]>
Date: Thu, 7 Jul 2016 08:29:13 -0700

Hi, all,

This update incorporates all pending changes, notably from detailed
reviews and discussions with Fred Templin, Lucy Yong, Toerless Eckert,
and Tom Herbert.

There's still a bit to do - notably to wrangle out what we want to say
regarding other RFCs in Section 5. This, and the doc's evolution,
suggest that it might be useful to consider shifting the intended track
from Informational to BCP or beyond, depending on whether we need to be
higher than BCP to "update" some of the problems outlined with
standards-track docs (most notably RFC2003).

NOTE: A brief summary will be presented by the chairs in Berlin, but I
will not be attending. Please use the list as the primary venue for
discussion.

I am currently hoping we can decide how to proceed on this doc in the
next few months so we can either remove or complete any "pending"
sections and get to WGLC early this fall.

Joe

---------

Summary of changes:
- now "updates" 4459 (informational too)
- revised MTU terminology based on list discussions (all MTUs indicate
"link" payload sizes)
- revised figures to indicate proximity of ingress/egress "interfaces"
to nodes
- moved Appendix A to new Section 3.6, as outer/inner fragmentation
issue is core
- revised Sec 4.2 (was 4.1) to explain why two different frag algs are
presented
    - one is implicit in all hosts/forwarders, irrespective of link
technology
    - one is contained "within" the link technology of a tunnel
- updated recommendations throughout section 5

Summary of additions:
- Sec 4.1 on the variety of MTU values
- Sec 4.12 on multipoint tunnels
- Sec 5.4 on diagnostics- Sec 5.5.1 on GUE
- Sec 5.5.15 on RTGWG-DT-ENCAP
- Security Considerations text on inner/outer tunnel vulnerabilities
- Recommendation text for Sec 5.5.3 on RFC2003 (IPv4 in IPv4)

----

On 7/6/2016 11:28 PM, [email-masked] wrote:
> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 8 Jul 2016 20:00:03 +0000

Hi Joe,

Sorry to say, but I still disagree with most aspects of the text on fragmentation
and MTU, which are largely the same as what appeared in the previous draft
version. I will say that we agree on one fact, however, in that fragmentation
is ultimately unavoidable.

When we discussed this last year, I thought we had established some things
that got reflected in Section 3.13 of 'draft-templin-aerolink'. I'd like to invite
you and the working group to review that text which IMHO presents a
better MTU and fragmentation consideration for tunnels.

Thanks - Fred
[email-masked]

> -----Original Message-----
> From: Int-area [mailto:[email-masked]] On Behalf Of Joe Touch
> Sent: Thursday, July 07, 2016 8:29 AM
> To: [email-masked]
> Subject: Re: [Int-area] I-D Action: draft-ietf-intarea-tunnels-03.txt
> 
> Hi, all,
>

---

From: Joe Touch <[email-masked]>
Date: Fri, 8 Jul 2016 13:23:05 -0700

Fred,

The text in draft-ietf-intarea-tunnels reflects the current IPv4 and
IPv6 transit and endpoint MTU requirements. It also separates the
existing host and gateway fragment processing functions from that of the
tunnel.

Can you be more specific where you think the algorithms presented are
incorrect? The basis of the text in the current draft was vetted by
several other parties.

----

I don't consider draft-templin-aerolink a useful starting point,
however. The following is a PARTIAL list of why:

- It imports RFC4459 by reference, which is demonstrably incorrect on
several points (see draft-ietf-intarea-tunnels Sec 5.5.5). 

- It is inconsistent - claiming that Aero links support minimum MTUs of
1280 B, but claiming (3.13.2) that the link MTU of the tunnel is
"(initially set to the MTU of the underlying link to be used for
tunneling minus ENCAPS)" and endorsing the notion that the path MTU of a
tunnel can be lower than the requirement of IPv6 ("These procedures
apply when the path MTU for this egress is no smaller than (1280+ENCAPS)
bytes -- which, for most Internet paths, cannot be assumed to be larger
than 1280. This completely ignores the requirement that the tunnel
egress be capable of reassembling a 1500 B datagram, so the correct
limit should be 1500-ENCAPS (as explained in draft-ietf-intarea-tunnels
Sec 4.1).

- it recommends that the Aero interface send PTB packets (routers, not
interfaces, generate PTBs).

- it admits packets up to 1500 B without considering that this exceeds
the requirement of IPv6 reassembly in Sec 3.1.13 (i.e., IPv6 endpoints,
such as the egress, must support reassembling a 1500 B IP datagram, but
this yields a tunnel capacity of 1500-encaps, not 1500).

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 11 Jul 2016 15:47:17 +0000

Hi Joe,

I will respond to your post in two parts - first, I will respond to your critique
of AERO (see below):

> From: Joe Touch [mailto:[email-masked]]
> Sent: Friday, July 08, 2016 1:23 PM
> To: Templin, Fred L <[email-masked]>; [email-masked]

You know that I have been developing the text in AERO since 2002 (you
know because you have been watching me do it). The AERO text is what
I arrived at when many other approaches failed.

> several points (see draft-ietf-intarea-tunnels Sec 5.5.5).

Your critique of RFC4459 seems to be based on the assertions you are making
in your document (e.g., egress reassembly size as tunnel MTU). I am saying
that I do not agree with those assertions.

> apply when the path MTU for this egress is no smaller than (1280+ENCAPS)

1280+ENCAPS refers to the path MTU *within the tunnel*, and not the
MTU of the tunnel interface. There are two levels of MTUs to consider;
the MTU of the tunnel interface (which must be a minimum of 1280)
and the path MTU within the tunnel interface. The text is consistent
in that way.

> limit should be 1500-ENCAPS (as explained in draft-ietf-intarea-tunnels

No, it says that the tunnel egress must be capable of reassembling at
least 1500+ENCAPS and therefore recommends that the egress be
capable of reassembling at least 2KB.

About your 1500-ENACPS, that would leave us in no better condition than
where things stand today. If the tunnel ingress sets an MTU of 1500-ENCAPS,
and if it returns a PTB listing an MTU less than 1500, the PTB could be lost and
the original source would never learn that there was a path MTU restriction.
That is whey AERO enforces a minimum MTU of 1500 bytes for situations
where the original source might not receive PTBs.

> interfaces, generate PTBs).

RFC1981(bis) says:

  "Note that Path MTU Discovery must be performed even in cases where a
   node "thinks" a destination is attached to the same link as itself."

and:

   "If any of the packets sent on that path are too large to be forwarded
   by some node (regardless of whether it decrements the Hop Limit)
   along the path, that node will discard them and return ICMPv6 Packet
   Too Big messages [ICMPv6]."

Forwarding nodes that do not decrement the hop count are exactly what
happens within the AERO interface.

> this yields a tunnel capacity of 1500-encaps, not 1500).

Just as tunnels over IPv4 assume a larger reassembly unit than the RFC1122
guarantee of 576B, we are now asking the egress to set a reassembly unit
of at least 1500+ENCAPS. Setting a minimum reassembly unit that exceeds
the IPv6 1500  should be in scope for this document.

Remember - we are trying to get away from tunnels that set an MTU
smaller than 1500 so that hosts that send packets that are no larger
than 1500 will not get black-holed.

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 11 Jul 2016 17:43:52 +0000

Hi Joe,

In this second part, I will address some of my concerns with intarea-tunnels:

I understand that intarea-tunnels wants to view the tunnel as if it were an
ATM link that fragments packets larger than the cell size and no larger than
the egress reassembly unit. That being the case, the document would need
to say what the cell size is, and afaict that size should be 1280-ENCAPS
(resulting in an encapsulated packet no larger than 1280). I understand the
attractiveness of this model, because it makes the tunnel look like a link,
but there are several important differences.

First, unlike ATM the tunnel may be carried over paths with links that have
highly asymmetric MTUs, link bandwidths, and packet loss ratios. That means
that (again unlike ATM) one or more fragments in a fragmented packet could
be lost resulting in loss of an entire packet which the original source would
then have to retransmit. I agree that if the original source is using RFC4821
that it would then start sending smaller packets, which is good.

However, sending 1280b fragments over paths that support much larger packets
(e.g., 8KB) would be inefficient and would nullify the advantage of setting a larger
MTU on the tunnel interface in the first place. This is why AERO says that only
packets up to 1500 bytes in length are fragmented (when necessary), and larger
packets should be sent in a single piece.

By sending large packets in a single piece, there is no need for the ingress to
have to discover the egress' reassembly buffer size. This becomes important
when there is one ingress and many egresses such as on an NBMA link.

The document also does not state the circumstances under which fragmentation
can be avoided (see: AERO Sections 13.3.1 and 13.3.2). In general, fragmentation
is undesirable when the source is capable of reducing the size of the messages
it sends. Text such as that appears in AERO sections 13.3.1 and 13.3.2 should
be added to intarea-tunnels.

The document also talks about outer fragmentation only, and does not talk about
tunnel fragmentation (see RFC2764). Tunnel fragmentation is different than outer
fragmentation, and it is possible for both to be employed on the same packet.
See: draft-herbert-gue-extensions for an application of tunnel fragmentation.

Finally, I am not sure what you mean by this:

   "When ongoing ingress fragmentation and egress reassembly would be
   prohibitive or costly, larger MTUs can be supported by design and
   confirmed either out-of-band (by design) or in-band (e.g., using
   PLPMTUD [RFC4821], as done in SEAL [RFC5320] and AERO [Te16])."

By PLPMTUD, do you mean sending an unfragmented large packet to see if it
reaches the egress? Or, do you mean sending a *fragmented* large packet to
see if it reaches the egress? The former does not work due to multipath, as I
explained in a recent list message and (I thought) confirmed by you. The
latter could be useful to test the egress' reassembly buffer size no matter
whether there is multipath within the tunnel. Which one did you mean?

We agree that fragmentation and reassembly is a necessary consideration for
tunnels in the Internet architecture. We are not clear yet on the advantages of
sending whole packets vs. fragmented packets for sizes larger than 1500. I
believe that operation over tunnels with a single ingress and potentially many
egresses (e.g., such as for NBMA) needs more clarification. And, I have stated
above a number of omissions in intarea-tunnels that are covered by AERO and
should be brought into your document.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Mon, 11 Jul 2016 14:13:41 -0700

Corrections below as needed...

On 7/11/2016 10:43 AM, Templin, Fred L wrote:
> to say what the cell size is,
intarea-tunnels allows that to be a property of a tunnel and does not
mandate it.

>  and afaict that size should be 1280-ENCAPS
> (resulting in an encapsulated packet no larger than 1280). 

  For IPv6 in IPv6, the TIMTU (what you call 'cell size', which is
really the cell payload size, i.e., the native payload of the link) is
1280-encaps. The size of "encaps" depends on the configuration if the
tunnel ingress, e.g., whether it uses IPsec or other IP extension headers.

That means the unfragmented TTP would need to be smaller than 1240
(assuming the minimum IPv6 header).

> I understand the
I assume the Internet link model (e.g., as discussed in RFC3819), which
assumes that a link is defined by a uniform set of such properties.
Where this varies, the "least common" values could be used (the smallest
PMTU). Where that's not desired, it would be more appropriate (and
sufficient) to model these as separate tunnels.

> MTU on the tunnel interface in the first place. 
Tthis is where our two docs disagree. IMO,  PMTU is intended to ensure
that packets can traverse a net, NOT that the source can find the
optimal largest such size.

I.e., IMO PMTU is not about efficiency; it's about reachability.

That's why we reach different conclusions below.

> This is why AERO says that only

The MTU of a tunnel is defined by the ERMTU, not the PMTU, because ERMTU
packets *can* traverse the tunnel.

> be added to intarea-tunnels.

In all Internet networks, fragmentation is avoided by sending packets
smaller than the required min MTU.

There's no part of Internet protocols that are designed to avoid link
fragementation; only network.

> tunnel fragmentation (see RFC2764).
It specifically discusses both inner and outer fragmentation.

It also discusses inner fragmentation in the context of source
fragmentation as one of the two key fragmentation algorithms.

>  Tunnel fragmentation is different than outer

The tunnel ingress can fragment at any layer it wants - i.e., if the
ingress accepts IP and sends them over GUE, it can fragment at GUE or
the final IP. That's up to the way the link is specified, and
intarea-tunnels considers that link-layer fragmentation.

> see if it reaches the egress? 
PLPMTUD would involve sending some larger packets.

Other methods may involve directly querying the egress, e.g., "what is
your ERMTU?"

> The former does not work due to multipath, as I
> latter could be useful to test the egress' reassembly buffer size no matt

Multipath breaks PLPMTUD everywhere, not just for tunnels. That's why
other methods, e.g., directly querying the egress, may be useful.

> er
I disagree that any are relevant because they assume the need to
transfer link payloads larger than are required by either IPv6 links or
IPv6 endpoints.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 11 Jul 2016 21:45:06 +0000

Hi Joe,

I was unable to parse most of what you said, but one point that requires

> other methods, e.g., directly querying the egress, may be useful.

That is not true to my understanding. When the original source uses PLPMTUD,
the probes take exactly the same path as the data packets so PLPMTUD works
even if there is multipath.

This is different than for tunnels, because the tunnel ingress is not the source
of the original packets - it is only the source of the encapsulated packets. And,
the ingress may need to encapsulate pacekts produced by many original
sources which may take very different routes due to multipath within the tunnel.

Thanks - Fred

> Sent: Monday, July 11, 2016 2:14 PM

---

From: Joe Touch <[email-masked]>
Date: Mon, 11 Jul 2016 14:58:30 -0700

> even if there is multipath.
Only if PLPMTUD is performed within each flow as differentiated by
multipath routing.

There is no solution if the mutlpathing uses other packet info for context.

> of the original packets - it is only the source of the encapsulated packets. 

It is identical in that the ingress is responsible for fragmentation,
which the layer at which PLPMTUD is supposed to happen.

> And,

Then the tunnel should do what it can to ensure that they do not take
different routes.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 11 Jul 2016 22:10:09 +0000

Hi Joe,

> Sent: Monday, July 11, 2016 2:59 PM

Right; PLPMTUD has to be applied to each flow the source produces
individually - there is no way to share the PMTU information among
different flows even though they may have the same destination.

> There is no solution if the mutlpathing uses other packet info for context.

My understanding of multipath is that it needs to ensure that all packets
of the same flow follow the same path. Any multipath equipment in the
network that does not honor that requirement is broken, at least to my
understanding.

> which the layer at which PLPMTUD is supposed to happen.

This statement makes no sense afaict.

> Then the tunnel should do what it can to ensure that they do not take

The tunnel has no way of controlling which routes the network will select.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Mon, 11 Jul 2016 16:14:40 -0700

For clarification

> network that does not honor that requirement is broken, at least to my

If that's the case, then PLPMTUD will work just fine with tunneled
traffic because the multipath and tunnel will both be operating only on
the flow information in the outermost IP header.

> This statement makes no sense afaict.

To clarify:

PLPMTUD operating at the ingress would need to operate on the flows of
the IP packets emitted by the ingress into the tunnel. Those flows are
defined by the outer header. The fragmentation is defined by how the
ingress source fragments (either at the outer IP layer or any other
layer within the added encapsulation headers).

That would make the ingress no different from any other traffic source.
It emits one or more flows, and as long as the flows it emits are
respected by multipath routing, PLPMTUD at the ingress on each flow will
work correctly.
> The tunnel has no way of controlling which routes the network will select.

Agreed, but if the network uses routes that DPI beyond the encapsulation
headers, then *by your claim* that routing is violating multipath
routing and all bets are off anyway.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 12 Jul 2016 14:37:32 +0000

Hi Joe,

> Sent: Monday, July 11, 2016 4:15 PM

Not necessarily. The tunnel ingress may need to encapsulate original
packets with many different flow labels that are all destined to the
same egress. Unless the ingress probes the path *for each distinct
flow* multipath can still cause the probes to take different paths
than the corresponding data packets.

I am also told that some networking gear looks more deeply into
the original packet than just the flow label, i.e., even after the
original packet has been encapsulated. So, it is not necessarily
just the flow label that multipath will operate on.

> respected by multipath routing, PLPMTUD at the ingress on each flow will

One alternative would be for the ingress to set the same flow label in
the encapsulation headers of all packets regardless of the flow labels
in the original packets (i.e., the "pipe" model). But this would either
defeat multipath (undesirable) or still not solve the problem if
multipath looks more deeply into the packet than just the outer
flow label (unfixable).

> routing and all bets are off anyway.

My understanding is that the network may use information more deeply
embedded in the packet than just flow label for multipath decisions.
So, yes, it is DPI but if the ingress does it it needs to be very sure that
all packets of the same flow get the same treatment. This does not
negate my claim.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Tue, 12 Jul 2016 07:48:20 -0700

On 7/12/2016 7:37 AM, Templin, Fred L wrote:
> ...
A flow is defined by source, destination, and flow ID when the flow ID
is set.

(otherwise it's defined by whatever DPI is used, which typically
involves source, destination, protocol, source port, and dest port).

So different flow IDs between the same ingress/egress pairs are
different flows. I've already noted that PLPMUTD would need to run
independently for each flow between ingress and egress, *exactly as it
would for different flows between a given pair of endpoints that don't
use tunnels*.

> just the flow label that multipath will operate on.

That could interfere with multipath between two endpoints as well, e.g.,
if it tries to differentiate between access to different app layer content.

All bets are off of multipoint ignores flow ID and its defining context
- for PLPMTUD for both tunnels and the rest of the Internet.

> multipath looks more deeply into the packet than just the outer

The former may be desirable or not, and it's up to the ingress to decide
(or whomever configures it).

The latter is a problem in multipath that can affect everyone, not just
tunnels, as noted above.

> embedded in the packet than just flow label for multipath decisions.
Multipath can interfere with any PLPMTUD if this is the case.

> all packets of the same flow get the same treatment. This does not

No, but it renders it a bit moot if it isn't specific to tunnels.

Joe