Subject: [Int-area] intarea-tunnels meta-comment: multipoint tunnels

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 15 Jul 2016 17:48:43 +0000

In intarea-tunnels, section 4.2, it says:

  "(we assume that all tunnels, even multipoint tunnels, have
   a single, uniform TIMTU)"

But, that does not seem to allow for multi-MTU multipoint tunnels.
It should be OK, for example, if not all egresses reached over the
multipoint tunnel configure the same ERMTU.

So, the ingress should configure a TMTU that is no larger than the
largest egress ERMTU and allow path MTU discovery to determine
the correct MTU for each egress.

This is no different than in RFC1981(bis), where it says:

  "Note that Path MTU Discovery must be performed even in cases
    where a node "thinks" a destination is attached to the same link
    as itself."

and:

   "If any of the packets sent on that path are too large to be forwarded
   by some node (regardless of whether it decrements the Hop Limit)
   along the path, that node will discard them and return ICMPv6 Packet
   Too Big messages [ICMPv6]."

IMHO, the document needs to discuss this.

Fred

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Jul 2016 11:14:13 -0700

Hi, Fred,

> But, that does not seem to allow for multi-MTU multipoint tunnels.
The doc currently assumes that a single multi-MTU tunnel would be
treated as separate tunnels.

There's no specific reason to treat how a tunnel handles multi-MTU paths
vs. any other link. Any approach that supports the latter should support
the former.

> the correct MTU for each egress.
That's not safe.

The safe solution is to use either ERMTU and TIMTUs on a per-egress
basis (effectively modeling the multipoint tunnel as independent
point-to-point tunnels) OR to use the minimum of each of these values
across all egresses and internal paths.

>     where a node "thinks" a destination is attached to the same link
This case is used in that doc in the context of multicast, not mulipoint.

The specific example of 'same link as itself' is used only because of
proxy relaying.

>    Too Big messages [ICMPv6]."
That doc assumes PMTUD rather than PLPMTU, and the former is often
blocked and blackholes if so. As a result, I don't think the
recommendation is correct.

However, I don't see this as affecting intarea-tunnels other than "don't
trust what you see or are told" when it comes to ERMTU or TIMTU - "trust
but verify". I think that's sage advice to add for all tunnels, but I
don't see that it has any relation to multipoint per se.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 15 Jul 2016 19:22:05 +0000

Hi Joe,

> -----Original Message-----
> From: Joe Touch [mailto:[email-masked]]
> Sent: Friday, July 15, 2016 11:14 AM
> To: Templin, Fred L <[email-masked]>; [email-masked]
> Subject: Re: intarea-tunnels meta-comment: multipoint tunnels
> 
> Hi, Fred,
> 

In OpenVPN, there is a single tunnel ingress (the tun0 interface) which
has a fixed MTU that determines the largest packet that can be admitted
into the interface. Once inside the interface, the interface may need to
determine which among many potential egresses the tunneled packet
needs to be delivered to, and it is possible for the potential egresses
to have different MTUs. This model is adopted also by AERO.

So, the tunnel interface MTU determines whether the packet can
be admitted into the interface, and the per-egress MTUs determine
whether the packet (once inside the interface) can be encapsulated
and sent to the egress.

So, for example, if the tunnel interface has an MTU of 4KB and the egress
determined from within the tunnel interface has an MTU of 2KB, the
interface drops the packet and returns a PTB, i.e., a self-generated PTB.
This is consistent with the RFC1981(bis) text I quoted below. And, broken
or not, RFC1981 is the standard.

Thanks - Fred
[email-masked]

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Jul 2016 13:02:40 -0700

> to have different MTUs. This model is adopted also by AERO.
That can be an implementation decision, which can still be modeled as
separate ingresses for each egress set with a different TIMTU or ERMTU.

> whether the packet (once inside the interface) can be encapsulated

ERMTU defines the tunnel MTU as reported to the host (i.e., as an
"interface MTU"). If that varies, the smallest value must be reported
AFAICT.

TIMTUs can vary without such impact outside the tunnel mechanism.

> determined from within the tunnel interface has an MTU of 2KB, 

If that's the case, the tunnel is misconfigured. The TMTU needs to be
set no larger than ERMTU-encaps.

> the

Interfaces don't return PTBs. Routers and hosts do. A router/host with
an interface that reports a 4K MTU would never return a PTB for a packet
that is 4K or less.

> or not, RFC1981 is the standard.
1981 doesn't talk about multipoint links; it talks about multicast.

Regardless, it does what I already suggested - it reports the minimum
when the MTU varies.

When it talks about doing MTU even when you think the link is direct,
that just means that the *receiver* (notably the intermediate acting as
a link-layer proxy for the receiver) can say that the packet is too big
- that signal cannot come from within the node that emitted the packet
that was too big.

This is all consistent with what I have stated:

- when we treat a group of egresses as part of a single tunnel, the PMTU
of that tunnel is the min of the PMTU of each ingress-egress pair.

- the receiver can always complain that a packet is too big to be
received, but that would happen inside the node where the packet was
being reassembled, not inside the tunnel itself. link layers (and thus
tunnels, IMO) don't generate PTBs.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 15 Jul 2016 20:24:39 +0000

Hi Joe,

There is nothing wrong with a self-generated PTB. If the ingress is
on a router, the PTB will be delivered to the original source. If the
ingress is on a host, the application will receive the PTB and adjust
the sizes of packets it sends.

Or, if you don't like the latter, I really don't mind if the host sets
the tunnel interface MTU to the minimum of all egress MTUs.
But, for a router, it should set to the maximum.

Thanks - Fred

> Sent: Friday, July 15, 2016 1:03 PM

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Jul 2016 13:38:14 -0700

On 7/15/2016 1:24 PM, Templin, Fred L wrote:
> Hi Joe,
>
> There is nothing wrong with a self-generated PTB. 
A router/host generates a PTB when a packet is too large for its
outgoing interface.

A host might generate a time-exceeded message that might be interpreted
as a PTB hint (see RFC 1122, Sec 3.2.2.4).

All ICMPs are generated by a router/host at layer 3, not the interface
at layer 2.

> If the ingress is
> on a router, the PTB will be delivered to the original source.

That ought to happen before the packet ever gets into the ingress, e.g.,
based on the TMTU.

> If the

Sure, same thing though - before it ever gets to the ingress.

> But, for a router, it should set to the maximum.
It has to be minimum for both. Setting it larger for a router only ends
up creating a blackhole opportunity.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 15 Jul 2016 21:03:53 +0000

Hi Joe,

> Sent: Friday, July 15, 2016 1:38 PM

It is true that the IP layer sees the interface MTU. But, after the packet
has been admitted into the interface, there is no reason the interface
cannot decide for itself that the packet is too small and return a PTB.
It is just as if the PTB were generated by a router somewhere further
down the line.

> All ICMPs are generated by a router/host at layer 3, not the interface

It is not at layer 2; it is at some no-man's-land middle layer between
L3 and L2. 

> That ought to happen before the packet ever gets into the ingress, e.g.,

It can happen anywhere on the path.

> Sure, same thing though - before it ever gets to the ingress.

I believe you are being too dogmatic about this.

> up creating a blackhole opportunity.

No, there is no new opportunity for black-holing introduced by this.
The interface will deterministically generate the necessary PTB and
it will not get black-holed before it leaves the router (i.e., even if it
does end up getting black-holed *after* it leaves the router).

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Jul 2016 14:25:47 -0700

On 7/15/2016 2:03 PM, Templin, Fred L wrote:
> ...
That would only  need to happen if you have a misconfigured interface
MTU, however...
> It is just as if the PTB were generated by a router somewhere further
Routers generate PTBs, not links. Interfaces are part of links.

If an ingress discovers that it needs to adjust its MTU, it should do so
- and affect subsequent arriving packets. There is no reason that the
ingress ever needs to generate a PTB.

> It is not at layer 2; it is at some no-man's-land middle layer between
There's no need for such a mythical beast.

In a system with tunnels, to the router/host, a tunnel is L2.

> It can happen anywhere on the path.

You're just indicating that the tunnel is misconfigured then.

This would be as if your Ethernet interface said its MTU was 1500 and
found out internally that this is not the case.

Nothing needs to happen at L2 other than changing the interface MTU. The
rest should happen outside L2.
> I believe you are being too dogmatic about this.

IMO, you're again trying to find a way to enable a tunnel to try to use
a larger path than might be guaranteed. That opens a can of worms for
which there is no reliable model.

> The interface will deterministically generate the necessary PTB 

How does an interface generate the proper ICMP message? The source IP
address might legitimately be that of the router's canonical IP, not
that of the interface - e.g., if the interface uses RFC1918 addresses.
That's why the router - not the interface - generates ICMPs.

There's no PTB generation requirement in RFC1122. There is in RFC1812.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 15 Jul 2016 22:20:51 +0000

Hi Joe,

> Sent: Friday, July 15, 2016 2:26 PM

An interface is "a node's attachment to a link" [RFC2460], but a lot can
go on inside of an interface before the packet is actually presented
to the link layer. This happens all the time today in implementations
that use TUN/TAP interfaces, for example. Here is an example
implementation:

http://linkupnetworks.com/aero/aero-3.0.1.tgz

> ingress ever needs to generate a PTB.

Taking an implementation that uses TUN/TAP for example (and there are
lots of those), the IP layer sees the TUN/TAP interface and admits the
packet into the interface if it is no larger than the interface MTU. Once
inside the interface, any number of operations can be performed on
the packet long before it is admitted into the tunnel - including drop
for various reasons such as too large for the specific egress among
potentially many egresses.

> There's no need for such a mythical beast.

It isn't mythical; tons of implementations do exactly this every day.
Anything that uses a TUN/TAP interface does exactly this.

> found out internally that this is not the case.

For hosts, local applications would receive locally-generated PTBs and
would operate on those exactly the same as if they came from a router
somewhere down the line. So, the application would simply believe that
there is a link somewhere down the line from its 1500 MTU interface that
configures a smaller MTU - it would not believe that its interface MTU
was less than 1500.

For routers, the PTBs would appear to the original source the same as
any ordinary PTB that came from a router somewhere inside the network.

> which there is no reliable model.

The tunnel generates the necessary PTB messages, which is the best
any node can do when there is an MTU restriction.

> that of the interface - e.g., if the interface uses RFC1918 addresses.

I don't see a problem with an RFC1918 address; the PTB could be generated
from within a NATed network just the same as it could be generated from
outside the NATed network. The source will accept the PTB just the same.

There are source address selection rules for what source address a node
should use for ICMP messages. I can't remember the exact RFC number,
but there is no reason this could not be employed by the interface.

> There's no PTB generation requirement in RFC1122. There is in RFC1812.

A TUN/TAP interface acts very much like a router as it forwards packets
between interfaces.  A lot goes on inside of the TUN/TAP interface
between the time that a packet enters from the inner IP layer until
it is presented to the outer IP layer. And, that includes generating PTBs.

I don't see why you are pushing back on this except for being dogmatic
that the tunnel has to behave *exactly* like a link. All seemingly perfect
analogies have exceptions, and this is one of them.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Jul 2016 16:05:17 -0700

Hi, Fred,

On 7/15/2016 3:20 PM, Templin, Fred L wrote:
...
> http://linkupnetworks.com/aero/aero-3.0.1.tgz

Interface *implementations* can do all sorts of things, including
terminate Layer 4 or above.

However, we need to differentiate what we expect happens at each layer
if possible (and it is possible).

> potentially many egresses.

Sure - but TUN/TAP interfaces don't typically emit ICMPs.

Here's a good example - it uses TUN/TAP and implements ICMP *over* that
- which is equivalent to implementing ICMP in the host/router with
TUN/TAP as an interface.

> Anything that uses a TUN/TAP interface does exactly this.
TUN/TAP is layer 2 to IP in the node (host/router). The examples I see
on the net generate ICMP outside of the TUN/TAP mechanism, not inside.

> any ordinary PTB that came from a router somewhere inside the network.
"somewhere" doesn't exist per se - the ICMP needs a source address. It
can't be that of the ingress because that might not be externally
routable on a router.

> any node can do when there is an MTU restriction.

I disagree that any of this is necessary. Tunnels don't need to generate
ICMPs any more than any other links do.

> outside the NATed network. The source will accept the PTB just the same.
Sure, but see RFC1812 Section  4.3.2.4 for what I'm talking about.

The address DOES NOT come from the interface on which the error
occurred. It comes from a valid source address for that router, which
might be a different interface or an address that is associated with the
router but no interface per se.

> but there is no reason this could not be employed by the interface.
The interface *code* can reach up and look into the router tables and
figure this out, but at that point we're talking about an implementation
hack that supports a router function inside the interface code.

> between interfaces. 

They're merely virtual versions of interfaces, which - to the
host/router - look like layer 2. TAP runs over a real layer 2 and TUN
runs over Layer 3 (i.e., more like the tunnels in intarea-tunnel).

These both are software versions of interfaces that just recycle the
data back up to a user process, where anyone *can* implement whatever
they want.

>  A lot goes on inside of the TUN/TAP interface

I'm pushing back exactly because it's not an exception. Sure, lots of
people have tried to bury ICMP optimizations inside software built over
TUN/TAP interfaces, but that doesn't make it either correct or useful.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jul 2016 23:34:25 +0000

Joe,

For multipoint tunnels, it should be OK if not all egresses configure the same
reassembly buffer size, so the ingress may have an MTU that is greater than
the reassembly buffer size of one or more egresses. Your document needs
to say this.

This is supported by RFC1981(bis), where it says:

   "Note that Path MTU Discovery must be performed even in cases where a
   node "thinks" a destination is attached to the same link as itself."

and:

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 16:36:38 -0700

On 7/20/2016 4:34 PM, Templin, Fred L wrote:
> the reassembly buffer size of one or more egresses. 
Why is that not effectively modeled as separate tunnels?

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jul 2016 23:47:35 +0000

Hi Joe,

> Sent: Wednesday, July 20, 2016 4:37 PM

Because in some cases there may be hundreds/thousands/millions of egresses
for each ingress, and having one tunnel interface for each egress would be untenable.

Egresses may also come and go, e.g., due to mobility. It would therefore lead to
hopeless instability if the node created a new tunnel interface and tore down an
old interface every time an egress appears or disappears.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 16:51:34 -0700

> for each ingress, and having one tunnel interface for each egress would be untenable.
That is an implementation issue.

> old interface every time an egress appears or disappears.
That is common in many networks.

The alternative is to model the subnet the way the rest of the Internet
assumes - as a uniform link layer, which would then have a single TMTU
which would be the min of the egress MTUs.

--
Note - all this need for egress MTU variation concern convinces me
further that you are focused on optimizing MTUs. That way lies madness.
MTUs need to be considered as minimums for communication, not limits to
*try* to approach.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 00:03:03 +0000

Hi Joe,

> Sent: Wednesday, July 20, 2016 4:52 PM

No its not - it is a scaling limitation issue no matter what the implementation.

> That is common in many networks.

It doesn't need to be this way for multipoint tunnels. The tunnel interface of the
ingress can remain stable no matter what chaos ensues with the dynamically
changing collection of egresses.

> which would be the min of the egress MTUs.

Not at all; RFC1981(bis) makes it clear that PMTUD must be used even if the
ingress thinks the egress is connected to the same link because there may
be a node in the path that does not decrement the MTU but returns a
PTB. There is no madness here.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 17:21:39 -0700

> PTB. There is no madness here.
That quote is (still) intended for the case where the destination is
actually more than one hop away. It is not intended to cover a
multipoint link that itself has varying MTUs.

However, whether you run PMTUD or not has no bearing on whether the link
has a single MTU or not. Note that the next section even states that:

   As discussed in section 1
<https://tools.ietf.org/html/draft-ietf-6man-rfc1981bis-02#section-1>, IPv6 nodes are not required to implement
   Path MTU Discovery. 

The real problem is that PMTUD is already known to be untenable due to
ICMP blocking. It's not clear why this doc is being revised without
replacing PMTUD with PMPMTUD.

So this remains a red herring and not relevant to the issue of how a
multipoint tunnel is modeled as a multipoint link.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 00:41:13 +0000

Hi Joe,

?  That quote is (still) intended for the case where the destination is actually more than one hop away. It is not intended to cover a multipoint link that itself has varying MTUs.

It also says:

       If any of the packets sent on that path are too large to be forwarded
   by some node (regardless of whether it decrements the Hop Limit)
   along the path, that node will discard them and return ICMPv6 Packet
   Too Big messages [ICMPv6].

It means that even nodes that appear to be neighbors on the same link need to have

their MTUs verified with PMTUD.

?  The real problem is that PMTUD is already known to be untenable due to ICMP blocking. It's not clear why this doc is being revised without replacing PMTUD with PMPMTUD.

That is a point you would need to go argue in 6man. I tried, and was told that RFC1981

is the proposed standard and RFC1981(bis) will become the full standard.

?  So this remains a red herring and not relevant to the issue of how a multipoint tunnel is modeled as a multipoint link.

Not at all. A multipoint tunnel is a multipoint link where the endpoints may have varying

MTUs. This is not different from an ordinary NBMA link.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 5:22 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: multipoint tunnels

PTB. There is no madness here.
That quote is (still) intended for the case where the destination is actually more than one hop away. It is not intended to cover a multipoint link that itself has varying MTUs.

However, whether you run PMTUD or not has no bearing on whether the link has a single MTU or not. Note that the next section even states that:

   As discussed in section 1<https://tools.ietf.org/html/draft-ietf-6man-rfc1981bis-02#section-1>, IPv6 nodes are not required to implement

   Path MTU Discovery.
The real problem is that PMTUD is already known to be untenable due to ICMP blocking. It's not clear why this doc is being revised without replacing PMTUD with PMPMTUD.

So this remains a red herring and not relevant to the issue of how a multipoint tunnel is modeled as a multipoint link.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 17:51:29 -0700

On 7/20/2016 5:41 PM, Templin, Fred L wrote:
>
> Hi Joe,
>
>  
>
> Ø  That quote is (still) intended for the case where the destination
> is actually more than one hop away. It is not intended to cover a
Sure - because they might have been relayed, but that also sets up a
black hole potential when the relay ICMP is blocked.

>  
> Ø  The real problem is that PMTUD is already known to be untenable due to ICMP blocking. It's not clear why this doc is being revised without replacing PMTUD with PMPMTUD.
>  
I really don't mind if they want to ensure that yet another IETF
document is outdated when published.

>  
> Ø  So this remains a red herring and not relevant to the issue of how a multipoint tunnel is modeled as a multipoint link.
>  
Ordinary NBMA L2 subnets in the Internet largely assume a single MTU.

There are only two reasons to care about PMTU inside this document:

1) we expect the ERMTU to vary. sure - just like RFC1981 says (we don't
need to go to the bis), the IP side of the house can track the ICMPs
coming back and adjust the TMTU as needed. There's no fundamental reason
not to say it's sufficient to take the min of such reported MTUs. Doing
more is fine too, but NOT REQUIRED.

2) we're trying to track the TIMTU on a per-egress basis, which can be
done but again we can also do the same thing here. it's fine to take the
min.

We really don't need to get wrapped around hyperoptimziation of MTUs
using a mechanism that blackholes anyway.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 01:10:47 +0000

Hi Joe,

PTB messages coming from the egress will be encapsulated and returned to the
ingress w/o them being dropped by an ICMP-filtering middlebox because they
will not look like ICMPs to the network - they will look like ordinary tunneled
packets. So, the ingress will receive the PTBs coming from the egress, i.e., they
will not black hole.

The ingress can also (or instead) proactively query the egress to determine the
egress' reassembly buffer size. That is what happens in the "multi-mtu" draft.
I am not saying that I endorse that draft, however.

PLPMTUD is not without its own set of issues. For example, we know that it is
hard to implement, and we have seen that PLPMTUD only works when the
probe packets take the same path as the data packets, which means that every
application would need to employ PLPMTUD independently of all other
applications due to multipath, LAG, etc.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 5:51 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: multipoint tunnels

It also says:

their MTUs verified with PMTUD.
Sure - because they might have been relayed, but that also sets up a black hole potential when the relay ICMP is blocked.

is the proposed standard and RFC1981(bis) will become the full standard.
I really don't mind if they want to ensure that yet another IETF document is outdated when published.

1) we expect the ERMTU to vary. sure - just like RFC1981 says (we don't need to go to the bis), the IP side of the house can track the ICMPs coming back and adjust the TMTU as needed. There's no fundamental reason not to say it's sufficient to take the min of such reported MTUs. Doing more is fine too, but NOT REQUIRED.

2) we're trying to track the TIMTU on a per-egress basis, which can be done but again we can also do the same thing here. it's fine to take the min.

We really don't need to get wrapped around hyperoptimziation of MTUs using a mechanism that blackholes anyway.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 01:22:42 +0000

Joe,

Another high-level point - will you agree that an egress needs to be able to discard
any packet that is too large to be reassembled? So, if the egress receives fragments
that would reassemble to, e.g., 4KB but the egress is only capable of reassembling
up to 2KB, the egress needs to take measures to drop those fragments.

Given that being the case, why can't the egress also send a PTB message back to
the source of the fragments?

Thanks - Fred

From: Int-area [mailto:[email-masked]] On Behalf Of Templin, Fred L
Sent: Wednesday, July 20, 2016 6:11 PM
To: Joe Touch <[email-masked]>; [email-masked]
Subject: Re: [Int-area] intarea-tunnels meta-comment: multipoint tunnels

Hi Joe,

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 5:51 PM
To: Templin, Fred L <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: intarea-tunnels meta-comment: multipoint tunnels

It also says:

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jul 2016 20:59:49 -0700

On 7/20/2016 6:22 PM, Templin, Fred L wrote:
>
> Joe,
>
>  
>
> Another high-level point – will you agree that an egress needs to be
> able to discard
>
> any packet that is too large to be reassembled?
>
Yes.

> So, if the egress receives fragments
>

Certainly.

>  
>
> Given that being the case, why can’t the egress also send a PTB
> message back to
>

IPv4 sends only 'frag needed and DF set' (which is relevant only at
routers) or 'time exceeded' (which applies to reassembly that fails
because a portion is missing when a timer goes off but only if frag 0
has been received).

IPv6 specifies "too big" only from routers based on the next-hop link MTU.

Neither doc ever indicates using PTB or its IPv4 equivalent for this
purpose.

So I really don't know where you're going with this thread at all. An
egress would never send PTB; the only party that sends PTB is an
internal relay link, *again* as per the example in RFC 2764.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 16:18:29 +0000

Hi Joe,

?  Given that being the case, why can't the egress also send a PTB message back to

?  the source of the fragments?

It occurred to me last night that this would not be a good idea; an egress could
be swamped with spoofed fragmented packets coming from adversaries and
would be ill advised to respond by sending PTBs. So, let's forget this idea; if
the egress needs to discard fragments that would be too large to reassemble
it should do so silently.

What it means is that the ingress needs to know how much the egress is capable
of reassembling before sending a large packet. This can be accomplished in AERO
at least by having each egress include an MTU option in its neighbor discovery
messages.

But, in terms of implications for intarea-tunnels, this means that the AERO link at
least is a "variable MTU, NBMA" link, which is a hybrid of the link types specified
in Section 3.2 of RFC4861. It means that multipoint tunnels should allow variable
MTUs determined on a per-egress basis and not necessarily a single MTU for
all egresses.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Wednesday, July 20, 2016 9:00 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: multipoint tunnels

Certainly.

IPv4 sends only 'frag needed and DF set' (which is relevant only at routers) or 'time exceeded' (which applies to reassembly that fails because a portion is missing when a timer goes off but only if frag 0 has been received).

Neither doc ever indicates using PTB or its IPv4 equivalent for this purpose.

So I really don't know where you're going with this thread at all. An egress would never send PTB; the only party that sends PTB is an internal relay link, *again* as per the example in RFC 2764.

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 21 Jul 2016 10:18:29 -0700

On 7/21/2016 9:18 AM, Templin, Fred L wrote:
>
> Hi Joe,
>
>  
>
> Ø  Given that being the case, why can’t the egress also send a PTB
> message back to
>
> Ø  the source of the fragments?
>
>  
>
> would be ill advised to respond by sending PTBs.
>

As I noted, there's no process for end systems to send PTBs. There
doesn't appear to be any way in which an endsystem is required to inform
the source via ICMP that the RMTU has been exceeded.

> So, let’s forget this idea; if
>
AFAICT, it already does.

> least is a “variable MTU, NBMA” link, which is a hybrid of the link
> types specified
>
> in Section 3.2 of RFC4861.
>

That section states quite directly the following, which is reiterated in
4.6.4 (cited below).

     variable MTU   - Neighbor Discovery allows routers to specify an
                      MTU for the link, which all nodes then use.  All
                      nodes on a link must use the same MTU (or Maximum
                      Receive Unit) in order for multicast to work
                      properly.  Otherwise, when multicasting, a sender,
                      which can not know which nodes will receive the
                      packet, could not determine a minimum packet size
                      that all receivers can process (or Maximum Receive
                      Unit).

> It means that multipoint tunnels should allow variable
>

See 4.6.4 of that doc:

        4.6.4 <https://tools.ietf.org/html/rfc4861#section-4.6.4>. MTU

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |     Type      |    Length     |           Reserved            |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                              MTU                              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Fields:

      Type           5

      Length         1

      Reserved       This field is unused.  It MUST be initialized to
                     zero by the sender and MUST be ignored by the
                     receiver.

      MTU            32-bit unsigned integer.  The recommended MTU for
                     the link.

   Description
                     The MTU option is used in Router Advertisement
                     messages to ensure that all nodes on a link use the
                     same MTU value in those cases where the link MTU is
                     not well known.

                     This option MUST be silently ignored for other
                     Neighbor Discovery messages.

                     In configurations in which heterogeneous
                     technologies are bridged together, the maximum
                     supported MTU may differ from one segment to
                     another.  If the bridges do not generate ICMP
                     Packet Too Big messages, communicating nodes will
                     be unable to use Path MTU to dynamically determine
                     the appropriate MTU on a per-neighbor basis.  In
                     such cases, routers can be configured to use the
                     MTU option to specify the maximum MTU value that is
                     supported by all segments.

This is what I have been saying all along. In a single subnet, even an
NMBA, there is one MTU to rule them all, except for *redirects* per
2764. But redirect ICMPs would affect TIMTU, not ERMTU.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 17:56:05 +0000

Hi Joe,

?  his is what I have been saying all along. In a single subnet, even an NMBA, there is one MTU to rule them all

Not on a variable MTU tunnel, which is what happens if not all egresses configure
the same reassembly buffer size. In that case, the MTU of the tunnel interface
should be no larger than the largest MTU of all egresses.

?  except for *redirects* per 2764. But redirect ICMPs would affect TIMTU, not ERMTU.

Redirects are not mentioned in RFC2764. They are at the very heart of AERO,
however, which is what I was referring to when I talked about ND messages
carrying MTU options.

Copying the text you quoted from RFC4861:

                     "In configurations in which heterogeneous
                     technologies are bridged together, the maximum
                     supported MTU may differ from one segment to
                     another."

This is the sense in which I am meaning variable MTU tunnel.

                     "If the bridges do not generate ICMP
                     Packet Too Big messages, communicating nodes will
                     be unable to use Path MTU to dynamically determine
                     the appropriate MTU on a per-neighbor basis."

With AERO at least, the "bridges" (i.e., the L2 code within the tunnel interface)
do generate ICMP PTB messages. So, it should be OK that the tunnel interface
has, e.g., a 4KB MTU but the code within the interface generates a PTB for an
egress that has a 2KB reassembly buffer.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Thursday, July 21, 2016 10:18 AM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: multipoint tunnels

As I noted, there's no process for end systems to send PTBs. There doesn't appear to be any way in which an endsystem is required to inform the source via ICMP that the RMTU has been exceeded.

So, let's forget this idea; if
the egress needs to discard fragments that would be too large to reassemble
it should do so silently.
AFAICT, it already does.

That section states quite directly the following, which is reiterated in 4.6.4 (cited below).

                      Unit).

See 4.6.4 of that doc:
4.6.4<https://tools.ietf.org/html/rfc4861#section-4.6.4>.  MTU

       0                   1                   2                   3

      |                              MTU                              |

   Fields:

      Type           5

      Length         1

                     receiver.

                     the link.

   Description

                     not well known.

This is what I have been saying all along. In a single subnet, even an NMBA, there is one MTU to rule them all, except for *redirects* per 2764. But redirect ICMPs would affect TIMTU, not ERMTU.

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 21 Jul 2016 11:16:03 -0700

On 7/21/2016 10:56 AM, Templin, Fred L wrote:
>
> Hi Joe,
>
>  
>
> Ø  his is what I have been saying all along. In a single subnet, even
> an NMBA, there is one MTU to rule them all
>
>  
>
The text in 4861 addresses this case directly and says the smallest MTU.

>  
>
> Ø  except for *redirects* per 2764. But redirect ICMPs would affect
> TIMTU, not ERMTU.
>
>  
>

That's specifically addressed in RFC4861.

>                      “In configurations in which heterogeneous
>
>                      another.”
>                      “If the bridges do not generate ICMP
>
>                      the appropriate MTU on a per-neighbor basis.”
>

Yes - but you need to KEEP READING:

                     In 
                     ^^
                     such cases, routers can be configured to use the
                     ^^^^^^^^^^
                     MTU option to specify the maximum MTU value that is
                     supported by all segments.
                     ^^^^^^^^^^^^^^^^^^^^^^^^^

ONE MTU. The largest supported by ALL. That's the MIN of the MTUs.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 19:47:51 +0000

Joe,

If the multipoint tunnel interface is capable of delivering internally-generated PTB messages
as if they had been generated by a bridge on the path to the egress, then the interface MTU
can be set to the largest of the reassembly buffer sizes among all egresses; otherwise, it must
be set to the smallest. AERO allows for the former.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Thursday, July 21, 2016 11:16 AM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: multipoint tunnels

                     In

                     ^^

                     ^^^^^^^^^^

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 21 Jul 2016 13:24:59 -0700

> be set to the smallest. AERO allows for the former.

PTBs generated by forwarding nodes inside the tunnel should affect
TIMTU, but have no bearing on ERMTU.

Nodes that do not forward have no business sending PTBs, as per the ICMP
specs.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 20:53:42 +0000

?  Nodes that do not forward have no business sending PTBs, as per the ICMP specs.

The tunnel interface is indeed seeking to forward the packet to the egress but
cannot do so because the egress reassembly buffer is too small. So, it drops the
packet and returns a PTB as if it were  a forwarding node on the path.

So, it should be fine that the interface configures, e.g., a 4KB MTU while the
egress reassembly size is only 2KB - PTB messages will report an MTU of 2KB
for addresses that are reachable via this egress.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Thursday, July 21, 2016 1:25 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: multipoint tunnels

PTBs generated by forwarding nodes inside the tunnel should affect TIMTU, but have no bearing on ERMTU.

Nodes that do not forward have no business sending PTBs, as per the ICMP specs.

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 21 Jul 2016 14:05:17 -0700

On 7/21/2016 1:53 PM, Templin, Fred L wrote:
>
> Ø  Nodes that do not forward have no business sending PTBs, as per the
> ICMP specs.
>
>  
>

There are two cases:

a) the egress reassembly is smaller than the ingress thinks
        we've already discussed that this is a silent drop

b) the packet arrives at the ingress interface but is larger than the
ingress interface
        this should never happen. the ingress has an MTU, and the node
where the ingress is attached would send back the PTB (if it were a router)

There's no case where the tunnel interface should ever see a packet
larger than its configured MTU.
 
> egress reassembly size is only 2KB – PTB messages will report an MTU
> of 2KB
>

An interface should never report an MTU it cannot carry *across the link
it attaches to*.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 21:11:03 +0000

Hi Joe,

I disagree; the interface can advertise an MTU as large as the largest reassembly unit
of all potential egresses as long as it can report PTB for egresses that have smaller
reassembly units. It can report PTB messages because it can use the link-local address
of the egress as the source address.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Thursday, July 21, 2016 2:05 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: multipoint tunnels

There are two cases:

b) the packet arrives at the ingress interface but is larger than the ingress interface
        this should never happen. the ingress has an MTU, and the node where the ingress is attached would send back the PTB (if it were a router)

There's no case where the tunnel interface should ever see a packet larger than its configured MTU.

An interface should never report an MTU it cannot carry *across the link it attaches to*.

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 21 Jul 2016 14:26:11 -0700

On 7/21/2016 2:11 PM, Templin, Fred L wrote:
> reassembly units.

Fred,

We've been over this multiple times.

RFC 4861 says exactly the opposite - that we should use the smallest RMTU.

There is no RFC that defines PTB as coming from anything other than a
router or relay. An egress is not a relay; it is the receiver within the
tunnel. The ingress may be attached to a router, but also may be
attached to a host - in which case it is nonsensical to emit a PTB.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jul 2016 21:34:00 +0000

Hi Joe,

RFC4861 says:

I am talking about a case where the "bridges" *do* generate ICMP PTB
messages. It works, and it is not inconsistent with this text.

RFC1918(bis) also says:

   and:

These are exactly the cases I am referring to.

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Thursday, July 21, 2016 2:26 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: multipoint tunnels

Fred,

There is no RFC that defines PTB as coming from anything other than a router or relay. An egress is not a relay; it is the receiver within the tunnel. The ingress may be attached to a router, but also may be attached to a host - in which case it is nonsensical to emit a PTB.

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 21 Jul 2016 14:41:37 -0700

I am not going around this block again.

Joe

On 7/21/2016 2:34 PM, Templin, Fred L wrote:
> I am talking about a case where the “bridges” **do** generate ICMP PTB
>
>    “If any of the packets sent on that path are too large to be forwarded
>
>    Too Big messages [ICMPv6].”
>
>  
>
>    and:
>
>  
>
>    “Note that Path MTU Discovery must be performed even in cases where a
>
>    node "thinks" a destination is attached to the same link as itself.”
>
>  
>
> *From:*Joe Touch [mailto:[email-masked]]
> *Sent:* Thursday, July 21, 2016 2:26 PM
> *To:* Templin, Fred L <[email-masked]>; [email-masked]
> *Subject:* Re: intarea-tunnels meta-comment: multipoint tunnels
>
>  
>
>  
>
> router or relay. An egress is not a relay; it is the receiver within
> the tunnel. The ingress may be attached to a router, but also may be

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 22 Jul 2016 15:52:48 +0000

OK, I am going to back down from my position because RFC4861 makes
it clear that the reason for all nodes on the link to configure the same
MTU is due to multicast. When an application sees an MTU on an
interface, it has reasonable expectation that a multicast sent on
that link will be received by all group members on that same link.

If not for multicast, the variable MTU link model I have been
representing would be viable - but, unfortunately, we have to
deal with multicast. I think the multi-mtu proposal might have
the same issue. Does anyone see it differently?

Thanks Joe,

Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Thursday, July 21, 2016 2:42 PM
To: Templin, Fred L <[email-masked]>; [email-masked]
Subject: Re: intarea-tunnels meta-comment: multipoint tunnels

Joe

RFC4861 says:

RFC1918(bis) also says:

   and:

Thanks - Fred

From: Joe Touch [mailto:[email-masked]]
Sent: Thursday, July 21, 2016 2:26 PM
To: Templin, Fred L <[email-masked]><mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: intarea-tunnels meta-comment: multipoint tunnels

Fred,

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 22 Jul 2016 10:25:57 -0700

OK - thanks for the discussion.

Joe

On 7/22/2016 8:52 AM, Templin, Fred L wrote:
> representing would be viable – but, unfortunately, we have to
>
> *Sent:* Thursday, July 21, 2016 2:42 PM
>     *To:* Templin, Fred L <[email-masked]>
>     <mailto:[email-masked]>; [email-masked]
>     <mailto:[email-masked]>
>     There is no RFC that defines PTB as coming from anything other
>     than a router or relay. An egress is not a relay; it is the
>     receiver within the tunnel. The ingress may be attached to a
>     router, but also may be attached to a host - in which case it is
>     nonsensical to emit a PTB.
>
>     Joe
>
>
>  
>