Subject: [Int-area] I-D Action: draft-ietf-intarea-tunnels-05.txt

From: [email-masked]
Date: Mon, 27 Mar 2017 08:34:41 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Internet Area Working Group of the IETF.

        Title           : IP Tunnels in the Internet Architecture
        Authors         : Joe Touch
                          Mark Townsley
	Filename        : draft-ietf-intarea-tunnels-05.txt
	Pages           : 51
	Date            : 2017-03-27

Abstract:
   This document discusses the role of IP tunnels in the Internet
   architecture. An IP tunnel transits IP datagrams as payloads in non-
   link layer protocols. This document explains the relationship of IP
   tunnels to existing protocol layers and the challenges in supporting
   IP tunneling, based on the equivalence of tunnels to links. The
   implications of this document are used to derive recommendations that
   update MTU and fragment issues in RFC 4459.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-intarea-tunnels/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-intarea-tunnels-05
https://datatracker.ietf.org/doc/html/draft-ietf-intarea-tunnels-05

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-intarea-tunnels-05

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Joe Touch <[email-masked]>
Date: Mon, 27 Mar 2017 08:39:15 -0700

Hi, all,

This update catches some typos that are important for understanding the
terminology changes.

Joe

On 3/27/2017 8:34 AM, [email-masked] wrote:
> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 28 Mar 2017 16:36:17 +0000

Here are my comments for this draft. Mostly editorial, but some substantial.
I also noticed several indications that further work was needed in some
sections. Will those sections be worked before publication?

Thanks - Fred
[email-masked]

1) P. 5, change "in which packet sizes ... mismatch" to
   "in which packet size ... mismatches"

2) P. 8, definition of Path MTU, the trailing phrase "(if supported)
   seems awkward and does not seem necessary. Can this be dropped?

3) Section 3.2, "Tis MTU" should be "This MTU".

4) Section 3.4, "egress receives packets to decapsulates". Change
   "decapsulates" to "decapsulate".

5) Section 4.1, "Tunneling uses encapsulation uses a non-link
   protocol" looks strange. Reword to make a coherent sentence.

6) Section 4.1.4, "Although fragmentation is somewhat rare in the
   current Internet at large, but it can be" - eliminate "but".

7) Section 4.2.1, first paragraph after Figure 10, "The minimum
   EMTU_R (c) is 1500 bytes", should be: "The minimum EMTU_R (d)
   is 1500 bytes".

8) Section 4.2.1, third paragraph after Figure 10, "The tunnel
   EMTU_R is EMFS_R minus the link (encapsulation) headers
   includes the encapsulation headers of the link layer." - I
   tried, but could not parse this sentence.

9) Section 4.2.1, third paragraph after Figure 10, should cite
   MFS, Path MFS and EMFS_R by including their letter designators
   (i.e., (h), (i), (j)) as was done in the previous two paragraphs.

10) Section 4.2.1, bulleted list toward bottom of section, tunnel
   "atom" is a very strange word to me. Tunnel "cell"?

11) Section 4.2.2, second paragraph after Figure 12, "Transit
    packets larger than 1460-EHsize will be dropped" does not match
    with two sentences earlier where it says that the tunnel path
    MTU will be *at least* 1500 - (40 + EHsize). It should just say
    "Transit packets larger than the tunnel MTU will be dropped".

12) Section 4.2.2, paragraph beginning "An IPv6 tunnel supports
    IPv6 transit only if EHsize is 180 bytes or less". I know what
    this paragraph is trying to say, but the 180 only pertains to
    cases where the EMTU_R is constrained to the IPv6 minimum of
    1500. When EMTU_R is larger, EHsize could also be larger. In
    fact, I think the document would do well to simply omit this
    paragraph because the relationship between EHsize and EMTU_R
    is already established in the previous paragraphs.

13) Section 4.2.2, final sentence is incorrect. RFC4821-style MTU
    probing cannot be used by tunnels due to ECMP because the probe
    packets may take a different path than the data packets. That
    is why AERO no longer uses RFC4821 probing.

14) Section 4.2.2, should include discussion of how common tunnel
    specs regard EMTU_R and the tunnel atom (cell) size. Namely,
    many common specs assume larger EMTU_R and tunnel cell sizes
    than are guaranteed by RFC1122 and RFC2460. That means that
    these tunnel specs are relying on operational assurances of
    larger values than the IPv4 and IPv6 minimums. Since they are
    already published as standards, this document needs to explain
    how they are able to operate without fragmentation.

15) Section 4.2.3, second paragraph "too bit to for" should be
    "too big for".

16) Section 4.2.3, final paragraph, "message too big" should be
    "packet too big".

17) Section 4.2.3 cites RFC4821, but PLPMTUD cannot be used by
    tunnels due to ECMP. The tunnel ingress is only the source
    of the tunnel packets; it is not the source of the transit
    packets. Since the transit packets of many different flows
    may be subject to the same tunnel packet encapsulation, the
    RFC4821 probes could travel over very different paths than
    the ordinary data packets and could experience very different
    path MTUs over those paths.

18) Section 4.3.3, For Multipoint Tunnels, please cite AERO, as
    well as ISATAP [RFC5214] and 6over4 [RFC2529]. They define
    an NBMA multipoint tunnel framework that has been around a
    lot longer than the other examples cited.

19) Section 4.3.3, third paragraph, I thought it was said earlier
    that all ingress/egress pairs must support the same MTU. I
    thought we agreed earlier on that that multi-MTU subnets
    don't work. So, I think it should say that all ingress pairs
    MUST support a uniform MTU.

20) Section 4.3.3, fourth paragraph, "A multipoint tunnel MUST
    have support for broadcast and multicast" - I think this
    would be better as a "SHOULD". RFC2529 and AERO support
    multicast, but RFC5214 does not yet it is widely deployed.

21) Section 4.3.4 second paragraph illustrates why RFC4821 cannot
    be used over tunnels.

22) Section 5.1, second sub-bullet under "Tunnel endpoints are
    network interfaces", change: "they not generate" to "they do
    not generate"

23) Section 5.1, first sub-bullet under "Tunnels must obey core IP
    requirements", Are you meaning to talk about IPv4 DF=1?

24) Appendix A.2 - this section should say something about
    implications for ECMP when a multi-encapsulation like this
    includes multiple packets that, if transmitted individually,
    would take different paths within the tunnel. Also, in the
    figure the insertion of the block labeled "iHa | iDa" should
    show up as "iHa | iDa" within the packed packet (it currently
    shows up as "iHa | iHa".

> -----Original Message-----
> From: Int-area [mailto:[email-masked]] On Behalf Of Joe Touch
> Sent: Monday, March 27, 2017 8:39 AM
> Cc: [email-masked]
> Subject: Re: [Int-area] I-D Action: draft-ietf-intarea-tunnels-05.txt
> 
> Hi, all,
>

---

From: Joe Touch <[email-masked]>
Date: Tue, 28 Mar 2017 12:38:05 -0700

Hi, Fred,

Thanks for the feedback. Let's assume I incorporate the nits/bugs and I
address any clarifications when I get to them in detail, so I'll jump
only to the deeper issues below...

Joe

> sections. Will those sections be worked before publication?
As per Mark's presentation, it depends on whether this doc stays
informational (as was adopted) or becomes BCP (as we think it probably
should).

If informational, much of Section 5 will be removed. If not, then that
will be fleshed out.

>    "atom" is a very strange word to me. Tunnel "cell"?

The concept of an atomic packet was defined in RFC 6864. This is derived
from that. Cell would be introducing a new term, one that is overloaded
with ATM, which we want to avoid.

>     is why AERO no longer uses RFC4821 probing.

Regarding ECMP issues, I think we need to wrap this issue up. Here's
what I propose:

- point out that ECMP causes problems with PMTUD (and can cause problems
with PLMTUD).
- an interface has two choices:
    - keep track of PMTU based on other packet context (flowID or next
header info)
    - merge PMTU feedback, taking the MIN of reported values

There's no magic here. It's a lot like multicast - either keep track in
a way that you *think* correlates to the different PMTU feedback or take
the MIN.

The current doc does need a scrub to make this point clearly and
consistently.

>     how they are able to operate without fragmentation.
I'll see what I can do to add that point; I'm hoping to not need to cite
dozens of examples, though.

>     tunnels due to ECMP.
I disagree; it can, but the system needs to either take the MIN or have
a way to decouple discovered PMTUs in way that can be trusted to
reasonably correspond to the ECMP splitting.

>     multicast, but RFC5214 does not yet it is widely deployed.

Multicast or its equivalent. Otherwise, you can't support IPv6
multicast, which is a required capability of IPv6.
>     requirements", Are you meaning to talk about IPv4 DF=1?

Yes, and that should be made more explicit. Also honoring the EMTU_R
limits until told otherwise.

-----

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 29 Mar 2017 21:18:36 +0000

Hi Joe,

See below:

> From: Joe Touch [mailto:[email-masked]]
> Sent: Tuesday, March 28, 2017 12:38 PM
> To: Templin, Fred L <[email-masked]>
> Cc: [email-masked]

OK, but then please find a way to call it an "atomic packet". Calling it
an "atom" would be introducing a new term here, and we could
argue whether the term would have more or less baggage than
the term "cell".

>     - merge PMTU feedback, taking the MIN of reported values

The problem is that some paths in the multipath may fail to deliver
the ICMPs. So there is no way to know whether the MIN has been
determined. Also the ingress may be handling transit packets sourced
by a very large number of original sources each of which produce
a very large number of distinct flows. So, there is no way for the
ingress to cache all of the flow information it handles.

> a way that you *think* correlates to the different PMTU feedback or take

It only works if all paths on the mutlipath can be counted on to deliver
the ICMPs. If any paths in the multipath fail to deliver the ICMPs, it
black holes. And, this is a known problem.

> The current doc does need a scrub to make this point clearly and

It doesn't work, regardless of the amount of scrubbing.

> dozens of examples, though.

OK, and I agree that you shouldn't need to cite the multitude of
standards that do this.

> reasonably correspond to the ECMP splitting.

It doesn't work in the generalized case. The ECMP might split into a
multitude of distinct paths, and there is no way for the ingress to
known which of the paths have been tested. And, all it takes is
one un-tested path in the multipath and there is potential for a
black hole. 

> multicast, which is a required capability of IPv6.

Large NBMA links can connect many nodes - thousands or more.
So, for link-scoped multicast, serialized multicast (i.e., multicast
via iterative unicast) would not scale.

That is why some large NBMA links (e.g., RFC5214, AERO) use unicast
NS/NA/RS/RA instead of link-scoped multicast, as permitted by RFC4861.
Link-scoped multicast service discovery (e.g., DHCPv6 discovery) is
supported via multicast mapping to a unicast link-layer address. 

> limits until told otherwise.

OK.

One other comment. I agree with figures 12 and 13 but (and I think this is
a crucial point) I think they need a supporting sentence or two explaining
why the procedure is "fragment then encapsulate" and not "encapsulate
then fragment". This is the difference between tunnel fragmentation
and ordinary outer fragmentation, where your document is correctly
advocating tunnel fragmentation. To the best of my knowledge, this was
first documented in Section 3.1.7 of RFC2764 and should be cited as such.
At least, that is what Bob B. suggested to me about 10yrs ago.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Wed, 29 Mar 2017 15:06:10 -0700

Hi, Fred,

I think we're agreed except for PMTUD and PLMTUD. See below about the
latter (AFAICT, if it black holes, then the PLMTUD would detect a
failure to make forward progress). Black-holing in PMTUD is known and
not something this (or any other doc) is fixing, but also a known standard.

Joe

On 3/29/2017 2:18 PM, Templin, Fred L wrote:
> ...
> OK, but then please find a way to call it an "atomic packet".

Agreed.
> ...
> determined.

That's no different from the multicast case, though.

>  Also the ingress may be handling transit packets sourced
Min requires maintaining the same information as any interface would keep.

> black holes. And, this is a known problem.
Again, same as multicast, and frankly also the same as unicast when the
ICMPs are blocked. That's a known problem with PMTUD.

> It doesn't work, regardless of the amount of scrubbing.

If your point is that PMTUD doesn't work and should never be used,
that's clearly not accurate and unlikely to get WG consensus. You're
welcome to try, though. However, 1981bis is on its way to increased
standards maturity as we speak.

> ...
If PLPMTUD thinks the protocol is making forward progress, then it is
not a black hole.

> via iterative unicast) would not scale.

Serial multicast is not the only equivalent. LANE pushed broadcast ARPs
to a unicast ARP server.

And yes, that won't scale to millions of links, but then if/when it
doesn't scale, then you cease to be able to claim this is a valid IP
link. Multicast is not an optional protocol for IPv6 in particular.

> supported via multicast mapping to a unicast link-layer address. 
Essentially like LANE.

> then fragment". 
Agreed.

> This is the difference between tunnel fragmentation
I'll check that...

----