Subject: [Bier] PMTUD requirement in BIER OAM Requirements

From: Gregory Mirsky <[email-masked]>
Date: Fri, 6 Nov 2015 01:02:10 +0000

Hi Toerless,
I'd like to bring requirement #9 in the BIER WG document OAM Requirements for BIER<https://tools.ietf.org/html/draft-ietf-bier-oam-requirements-00>:
BIER OAM MUST support Path Maximum Transmission Unit discovery.
Hope that addresses your question why are we doing PMTUD.

                Regards,
                                Greg

---

From: "[email-masked]" <[email-masked]>
Date: Thu, 5 Nov 2015 20:28:54 -0800

Thanks, Greg.

I was looking just for PMTU or MTU in the text and could not find it.

I do not agree to this requirement in the OAM requirement draft as of now.

Why is it needed for BIER ? AFAIK, there is no PMTUD for OAM
in any of the prior IP multicast transport options - native IP or MPLS. True ?
Any reason why BIER would need this more than those prior options ?
Any reason why BIER would make it a lot easier  than the prior options ?

This should IMHO be discussed in the draft.

If we get PMTUD, would it only be for OAM and only for the BIER path, eg:
not the end-to-end path ? If so, it should be called different (BP-MTUD or
the like). 

If someone proposes that it should be end-to-end, then this needs to go
back to MBoned, because it would ask to introduce PMTUD into eg: PIM path
segments.

Wrt. the protocols proposed in your PMTUD draft proposal: I would like
to understand why we would not want to proactively signal MTU intead
of probing it. There seems to have been some work in the past around
this in unicast, not sure how successful.

Cheers
    Toerless

---

From: Gregory Mirsky <[email-masked]>
Date: Fri, 6 Nov 2015 10:48:08 +0000

Hi Toerless,
perhaps I wasn't clear at the mike today. PMTUD is clearly not for OAM but it is OAM service to determine OAM on particular path (the same as PMTUD in p2p, right?). I believe that the fact that PMTUD over multicast in MPLS or IPv4/IPv6 doesn't scale is another indication of advantage BIER technology presents us with.
I'll be glad to continue our discussion on the list.

	Regards,
		Greg

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Friday, November 06, 2015 1:29 PM
To: Gregory Mirsky
Cc: BIER ([email-masked])
Subject: draft-ietf-bier-oam-requirements-00: PMTUD requirement in BIER OAM Requirements

Thanks, Greg.

Why is it needed for BIER ? AFAIK, there is no PMTUD for OAM in any of the prior IP multicast transport options - native IP or MPLS. True ?
Any reason why BIER would need this more than those prior options ?
Any reason why BIER would make it a lot easier  than the prior options ?

If we get PMTUD, would it only be for OAM and only for the BIER path, eg:
not the end-to-end path ? If so, it should be called different (BP-MTUD or the like). 

If someone proposes that it should be end-to-end, then this needs to go back to MBoned, because it would ask to introduce PMTUD into eg: PIM path segments.

Wrt. the protocols proposed in your PMTUD draft proposal: I would like to understand why we would not want to proactively signal MTU intead of probing it. There seems to have been some work in the past around this in unicast, not sure how successful.

Cheers
    Toerless

---

From: "[email-masked]" <[email-masked]>
Date: Sat, 7 Nov 2015 09:16:18 -0800

Thanks, Greg

To repeat and refine my points made at the mike @yokohama:

If we intend to support PMTUD for BIER, it should be accordingly
described, but not as an OAM operation. If it's meant as an
offline OAM operation, thats a different discussion.

There is no agreed upon PMTUD mechanism for IPv4/IPv6 multicast
today, the extended socket API explicitly recommend to stick to min-MTU
for IPv4 multicast (1280) to avoid PMTUD issues with IP multicast.

As long as you do probing to do PMTUD, it does not matter what transport
(native, MPLS, BIER, FOOBAR) you use, you will always have to deal with
the problem of potentially getting a humunguous amount of PMTUD
replies (eg: ICMP) back, and this does not scale, and network operators
are worried about the signaling overhead created by badly written
multicast apps. Remember that most PMTUD in unicast is done by few
TCP stacks in OS, but not in apps (little UDP apps trying to use
PMTUD AFAIK).

Just having some segment of an end-to-end IP(v4v6) multicast path be
BIER does not change that probing problem for the end-to-end path.

If we ignore IP multicast and just think about end-to-end BIER, the
picture changes somewhat. We do know explicitly know the set of receivers,
and its < 64k. But: One key benefit of BIER is the ability to change
the set of receivers WITHOUT SIGALING. If we where to do probing based
PMTUD in BIER, we would loose this benefit of BIER.

Aka: Technically i'd be interested to evaluate if we could do proactive
PMTUD for end-to-end BIER. Aka: via signaling of link-MTU in the IGP
that BIER uses - and calculating the path MTU to each BFER from it.
Or just calculating network-wide min-MTU from link-MTU in the IGP.

But: Use-case wise i really don't know how important this is. Ultimately,
this is not a different problem to MPLS: You do not want to do fragmentation,
so you want to send with a sufficiently small MTU, so well controlled
networks (MPLS, BIER) will get configured with the right MTU. And
applications written for IP in mind will not use MTU > 1280 anyhow.

The best case i can think of for BIER is that we get a strong use-case
that shows well improved throughput for jumbo-packets, and we see
sufficient benefits for auto-configuration of that jumbo-packet MTU
for end-to-end BIER.

The use case Caitlin Bessler told me about might fit this profile.
To make it reliable in a DC it would probably want to discover MTU,
and to be highest throughput it might want to use jumbo packets (8192
at least).

Aka: As long as i don't see different use-case descriptions, i'd
primarily be only interested in MTU > 1280 for end-to-end BIER in
this DC network storage use-case environment.

Cheers
    Toerless

-- 
---
Toerless Eckert, [email-masked]

---

From: Antoni Przygienda <[email-masked]>
Date: Sun, 8 Nov 2015 01:59:00 +0000

> 
> Just having some segment of an end-to-end IP(v4v6) multicast path be BIER does
> not change that probing problem for the end-to-end path.
> 
> If we ignore IP multicast and just think about end-to-end BIER, the picture
> changes somewhat. We do know explicitly know the set of receivers, and its <
> 64k. But: One key benefit of BIER is the ability to change the set of receivers
> WITHOUT SIGALING. If we where to do probing based PMTUD in BIER, we
> would loose this benefit of BIER.

 [Tony saiz:]  yes, because of smaller # receivers & this extension, we can prune the amount of traffic significantly. Observe that a BFR does not have to fwd necessarily the ACK'd responses of the BFERs behind him, it can be all summarized in a mask. 

> 
> Aka: Technically i'd be interested to evaluate if we could do proactive PMTUD
> for end-to-end BIER. Aka: via signaling of link-MTU in the IGP that BIER uses -
> and calculating the path MTU to each BFER from it.

 [Tony saiz:]  feasible but it relies on the riuoting underlay propagating this information.  Not sure that's a good assumption, especially since the end-hosts are not necessarily expected to run the IGP (it's not clearly spelled out in the architecture  today but I can easily imagine a BIER default-GW for an end-host and then this information does not exist in the originator of the PMTU.)

> 
> But: Use-case wise i really don't know how important this is. Ultimately, this is
> not a different problem to MPLS: You do not want to do fragmentation, so you
> want to send with a sufficiently small MTU, so well controlled networks (MPLS,
> BIER) will get configured with the right MTU. And applications written for IP in
> mind will not use MTU > 1280 anyhow.
> 
> The best case i can think of for BIER is that we get a strong use-case that shows
> well improved throughput for jumbo-packets, and we see sufficient benefits for
> auto-configuration of that jumbo-packet MTU for end-to-end BIER.
> 
> To make it reliable in a DC it would probably want to discover MTU, and to be
> highest throughput it might want to use jumbo packets (8192 at least).
> 
> Aka: As long as i don't see different use-case descriptions, i'd primarily be only
> interested in MTU > 1280 for end-to-end BIER in this DC network storage use-
> case environment.

 [Tony saiz:]  hmm, interesting angle

---

From: Gregory Mirsky <[email-masked]>
Date: Sun, 8 Nov 2015 06:04:04 +0000

Hi Toerless,
thank you for your detailed response. Couple questions to clarify and notes in-line and tagged GIM>>.

	Regards,
		Greg

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Sunday, November 08, 2015 2:16 AM
To: Gregory Mirsky
Cc: BIER ([email-masked])
Subject: Re: draft-ietf-bier-oam-requirements-00: PMTUD requirement in BIER OAM Requirements

Thanks, Greg

If we intend to support PMTUD for BIER, it should be accordingly described, but not as an OAM operation. If it's meant as an offline OAM operation, thats a different discussion.
GIM>> I don't know what you mean by "offline OAM". The PMTUD proposed as on-demand OAM operation that can be triggered by an operator or central controller. I don't see the need or benefit of running PMTUD as proactive OAM.

There is no agreed upon PMTUD mechanism for IPv4/IPv6 multicast today, the extended socket API explicitly recommend to stick to min-MTU for IPv4 multicast (1280) to avoid PMTUD issues with IP multicast.

As long as you do probing to do PMTUD, it does not matter what transport (native, MPLS, BIER, FOOBAR) you use, you will always have to deal with the problem of potentially getting a humunguous amount of PMTUD replies (eg: ICMP) back, and this does not scale, and network operators are worried about the signaling overhead created by badly written multicast apps. Remember that most PMTUD in unicast is done by few TCP stacks in OS, but not in apps (little UDP apps trying to use PMTUD AFAIK).

Just having some segment of an end-to-end IP(v4v6) multicast path be BIER does not change that probing problem for the end-to-end path.

If we ignore IP multicast and just think about end-to-end BIER, the picture changes somewhat. We do know explicitly know the set of receivers, and its < 64k. But: One key benefit of BIER is the ability to change the set of receivers WITHOUT SIGALING. If we where to do probing based PMTUD in BIER, we would loose this benefit of BIER.
GIM>> As discussed at the meeting, one may list any subset of BFIRs in the given domain. I don't see what gets lost with such flexibility.

Aka: Technically i'd be interested to evaluate if we could do proactive PMTUD for end-to-end BIER. Aka: via signaling of link-MTU in the IGP that BIER uses - and calculating the path MTU to each BFER from it.
GIM>> AFAIK, "proactive OAM" usually identifies an operation that is continuous. Do you suggest to run the PMTUD continuously with some intervals? Haven't thought about benefit of this mode though the PMTUD method would not preclude one from using it in this manner.

But: Use-case wise i really don't know how important this is. Ultimately, this is not a different problem to MPLS: You do not want to do fragmentation, so you want to send with a sufficiently small MTU, so well controlled networks (MPLS, BIER) will get configured with the right MTU. And applications written for IP in mind will not use MTU > 1280 anyhow.

The best case i can think of for BIER is that we get a strong use-case that shows well improved throughput for jumbo-packets, and we see sufficient benefits for auto-configuration of that jumbo-packet MTU for end-to-end BIER.

The use case Caitlin Bessler told me about might fit this profile.
To make it reliable in a DC it would probably want to discover MTU, and to be highest throughput it might want to use jumbo packets (8192 at least).

Aka: As long as i don't see different use-case descriptions, i'd primarily be only interested in MTU > 1280 for end-to-end BIER in this DC network storage use-case environment.

Cheers
    Toerless

---

From: "[email-masked]" <[email-masked]>
Date: Mon, 9 Nov 2015 14:02:37 -0800

Inline.

> GIM>> I don't know what you mean by "offline OAM". The PMTUD proposed as on-demand OAM operation that can be triggered by an operator or central controller. I don't see the need or benefit of runni…

Yes, that's what i meant with the somewhat bad term "offline". Thats well in scope for OAM.
But i would really like some better use-case justification for this. How does it compare
to eg: MPLS. 

> GIM>> As discussed at the meeting, one may list any subset of BFIRs in the given domain. I don't see what gets lost with such flexibility.

I was referring to the case where we would do MTUD for the application itself as
opposed to doing it purely as an OAM operation from somee operator console.
Eg: if we ever consider MTUD automatic for apps, then it should be proactive as
eg: via IGP so that a sender tht changes receiver bits does not have do do 
any signaling aat that time but immediately knows the MTU for that new receiver set.

> GIM>> AFAIK, "proactive OAM" usually identifies an operation that is continuous. Do you suggest to run the PMTUD continuously with some intervals? Haven't thought about benefit of this mode though…

I was thinking of just announcing per-link MTU into the IGP (or whatever routing protocol
BIER routing underlay relies on). Of course, the per-hop MTU could be discovered
through per-hop probing if there is reason not to simply believe interface configured MTU.

Cheers
    Toerless

---

From: "[email-masked]" <[email-masked]>
Date: Mon, 9 Nov 2015 14:23:19 -0800

>  [Tony saiz:]  feasible but it relies on the riuoting underlay propagating this information.  Not sure that's a good assumption, especially since the end-hosts are not necessarily expected to run t…

Right. At this point in time it seems to me as if BFRs must paerticipate in eg:
an IGP, so can't be "stupid hosts". I would like "hosts" to partake in BIER, but
i would like to avoid loosing information in the process.

[rant] Just look at IoT where every host partakes in the routing protocol. Much simpler
model than what we have done OSI and IETF for the last decades: come up with separate
"ES-IS" signaling, and for every piece of info that a router has, we have to come up
with an often equivalent but different way to let the hosts know about them. We already see
IMHO in homenet how this will not nearly work as well as if the host would just partake
in routing.  And MIF PVD ... pass back routing information to some DHCP server 
to then end it back to the host... *yikes* [/rant]

Cheers
    Toerless