Subject: [pim] Call for adoption of sr-mpls-multicast-framework

From: Michael McBride <[email-masked]>
Date: Mon, 6 Aug 2018 18:24:51 +0000

Hello good people,

Today begins a two week call for adoption of draft-allan-pim-sr-mpls-multicast-framework-00 which was presented in Montreal. There was a good discussion in the meeting and now it's time to gauge interest in adoption on the list. Please respond and let us know what you think about adopting this draft.

https://tools.ietf.org/id/draft-allan-pim-sr-mpls-multicast-framework-00.txt

thanks,
mike

---

From: David Allan I <[email-masked]>
Date: Mon, 6 Aug 2018 20:05:25 +0000

Support (as co-author)...

Cheers
Dave

From: pim <[email-masked]> On Behalf Of Michael McBride
Sent: Monday, August 06, 2018 11:25 AM
To: [email-masked]
Subject: [pim] Call for adoption of sr-mpls-multicast-framework

Hello good people,

thanks,
mike

---

From: Eric C Rosen <[email-masked]>
Date: Thu, 9 Aug 2018 09:38:30 -0400

I am opposed to the adoption of this draft.

The document is not very clear, and it is not always possible to tell 
what is actually being proposed.  The parts I think I understand have 
numerous problems which do not appear to have been properly considered.

The document appears to specify a way of setting up P2MP LSPs (let's 
call them "trees") using IGP as the control plane. Every tree is 
associated with an IPv4 or IPv6 group address.  Through some unspecified 
magic, each node determines whether it has "transmit interest" or 
"receive interest" for a given group address.  This interest is 
advertised via the IGP.

It is not clear whether these group addresses have anything to do with 
the group addresses that appear in IP multicast packets.  If they don't, 
it is not clear why a tree has to be associated with an IPv4/IPv6 
address at all.  If they do, it is not clear how multiple user flows can 
be aggregated on a single tree.

The draft seems to explicitly prohibit a tree from having parallel links 
between a given pair of nodes; this means that ECMP load balancing is 
not possible, which suggests that a tree is to carry only one flow. (I'm 
not sure why this prohibition is made, but it does seem to be there in 
rule 2 of section 5.2.2.)  On the other hand, there are a few sentences 
suggesting that these trees can be used to instantiate MVPN/EVPN PMSIs, 
which suggests that the trees aggregate flows and thus could benefit 
from ECMP.

The idea of using IGP to advertise interest in sending or receiving 
particular multicast flows is not new.  It recurs every few years, and 
generally disappears when people start to examine the evident scaling 
problems.  The number of groups, transmitters, and receivers is not 
bounded, and their rate of change is not bounded.  The last thing one 
wants to do is create an unbounded amount of IGP overhead.  
Surprisingly, the impact on IGP scaling is not mentioned at all in the 
draft.  (Imagine if one had to kick the IGP every time MVPN/EVPN 
initiates a new S-PMSI!)

Each <transmitter, group address> pair seems to be assigned a 
domain-wide unique label, or SID.  I think the intention is that the 
transmitter gets configured with this SID, and then advertises it in the 
IGP messages.  Assigning these domain-wide unique identifiers to trees 
that may appear and disappear with some frequency seems like an 
interesting problem.  However, the authors seem to regard this problem 
as out of scope for the document; that's a bit strange for a document 
that purports to be describing a framework.

Whether a given node has transmit and/or receive interest for a given 
group address is, according to the document, determined by management 
configuration.  There is no consideration of having the edge nodes 
support PIM/IGMP/MLD/mLDP, etc.  There is some mention of MVPN/EVPN, but 
only a mention, not an actual proposal.  That also seems strange for a 
framework document.  Surely a framework document should clearly 
delineate the boundary between overlay and underlay, and address the 
interactions between them.

Once the IGP distributes information about who has transmit and/or 
receive interest in each group address, a computation is performed to 
figure out the best tree for a particular <transmitter, group> pair.  
Some rules are sketched out for figuring this out, but the rules are not 
clear or precise enough to result in interoperable implementations.

Once the tree is determined, per-tree state needs to be established at 
each node that is either a transmitter, a receiver, or a replication 
point.  If a node doesn't fall into one of these categories for a 
particular tree, that node will not need the per-tree state.  If a 
particular replication point on a tree is not directly attached to the 
next replication point downstream, unicast tunneling is used to get the 
packets from one replication point to the next.

Much is made of the fact that per-tree data plane state exists only at 
nodes that are transmitters, replication points, or receivers for the 
tree.  This is presented as significant reduction in data plane state 
when compared to, say, PIM or mLDP.  Of course, whether there is any 
signficant reduction in state depends on the size and shape of the 
trees.  No analysis is given to show that the state reduction is 
significant, just a lot of handwaving.

On the other hand, there is an enormous increase in control plane state 
resulting from the fact that every single node in the network is aware 
of every tree and all its members.  Existing multicast protocols do not 
require a node to know about a tree unless the tree passes through it, 
and do not require all nodes along a tree to know about all the leaf 
nodes of the tree.  The large increase in the amount of control plane 
state is not considered at all, for some reason.

To send a data packet along a particular tree, the SID for that tree is 
pushed onto the packet's label stack.  Each node that receives a packet 
and sees a particular SID at the top of the label stack will replicate 
the packet and send (or tunnel) each copy to the next replication point, 
where the same SID will be seen.  Since unicast tunneling is generally 
used to get a packet from one replication point to the next, packets 
traveling along a particular tree may arrive over any interface.  This 
makes it impossible to do a data plane RPF check (such as PIM would 
do).  Since the tree has a domain-wide unique SID, you can't tell from 
the label who the previous node was, so you can't reject packets from 
the "wrong previous node" (such as mLDP would do).  This may lead to 
potentially disastrous multicast loops during periods when a tree is 
being modified, especially if node A was previously upstream of node B, 
but the change puts it downstream of node B.  (Modifications can be 
caused by topology changes or simply by a change in the set of 
receivers.)  These issues do not appear to have been considered.

I do not see how this document can be considered for WG adoption. Given 
the set of signficant issues that have not received proper 
consideration, it is not possible to say that this document is a good 
basis for future work.

---

From: David Allan I <[email-masked]>
Date: Thu, 9 Aug 2018 19:38:24 +0000

Hi Eric:

Replies in line, prefaced with DA>

The document is not very clear, and it is not always possible to tell what is actually being proposed.  The parts I think I understand have numerous problems which do not appear to have been properly considered.

DA> Then I'll try to respond to the comments you have, and perhaps we can work our way through them.

The document appears to specify a way of setting up P2MP LSPs (let's call them "trees") using IGP as the control plane. Every tree is associated with an IPv4 or IPv6 group address.  Through some unspecified magic, each node determines whether it has "transmit interest" or "receive interest" for a given group address.  This interest is advertised via the IGP.

DA> If it is magic then clearly you understand the trick and where the wires are. 😊

It is not clear whether these group addresses have anything to do with the group addresses that appear in IP multicast packets.  If they don't, it is not clear why a tree has to be associated with an IPv4/IPv6 address at all.  If they do, it is not clear how multiple user flows can be aggregated on a single tree.

DA> Good feedback. Personally I do not see adding support for them as being a particularly big deal. Explicit mention of this will be in the next version.

The draft seems to explicitly prohibit a tree from having parallel links between a given pair of nodes; this means that ECMP load balancing is not possible, which suggests that a tree is to carry only one flow. (I'm not sure why this prohibition is made, but it does seem to be there in rule 2 of section 5.2.2.)  
DA> There I think you have misunderstood the pruning algorithm. If at some point you have parallel paths you can consider them to be a single path from the point of view of resolving where state is going to end up.  It is certainly not a prohibition against ECMP. The algorithm is specifically designed to make ECMP as painless as possible. What you are commenting on is a simplification of the topology model as an intermediate step in determining where state will need to be installed.

On the other hand, there are a few sentences suggesting that these trees can be used to instantiate MVPN/EVPN PMSIs, which suggests that the trees aggregate flows and thus could benefit from ECMP.
DA> See replies above.

The idea of using IGP to advertise interest in sending or receiving particular multicast flows is not new.  It recurs every few years, and generally disappears when people start to examine the evident scaling problems.  The number of groups, transmitters, and receivers is not bounded, and their rate of change is not bounded.  The last thing one wants to do is create an unbounded amount of IGP overhead. Surprisingly, the impact on IGP scaling is not mentioned at all in the draft.  (Imagine if one had to kick the IGP every time MVPN/EVPN initiates a new S-PMSI!)
DA> Deployments of 802.1aq shortest path bridging would provide an existence proof for using the IGP without detrimental effects and I'm aware of some sizeable networks using it. It does have a virtuous circle in that all multicast state exchange is not tied to topology changes, so when a topology change does occur the control plane can simply get on with it with a minimum of messaging.  

Each <transmitter, group address> pair seems to be assigned a domain-wide unique label, or SID.  I think the intention is that the transmitter gets configured with this SID, and then advertises it in the IGP messages.  Assigning these domain-wide unique identifiers to trees that may appear and disappear with some frequency seems like an interesting problem.  However, the authors seem to regard this problem as out of scope for the document; that's a bit strange for a document that purports to be describing a framework.
DA> Again that is good feedback. We'll address this in the next version.

Whether a given node has transmit and/or receive interest for a given group address is, according to the document, determined by management configuration.  There is no consideration of having the edge nodes support PIM/IGMP/MLD/mLDP, etc.  There is some mention of MVPN/EVPN, but only a mention, not an actual proposal.  That also seems strange for a framework document.  Surely a framework document should clearly delineate the boundary between overlay and underlay, and address the interactions between them.
DA> That is an oversight in the content itself, interactions with IGMP, PIM etc. have been discussed outside the draft content. For the next version.

Once the IGP distributes information about who has transmit and/or receive interest in each group address, a computation is performed to figure out the best tree for a particular <transmitter, group> pair. Some rules are sketched out for figuring this out, but the rules are not clear or precise enough to result in interoperable implementations.
DA> If you could enumerate where you think it is not clear, that would be very useful input.

Once the tree is determined, per-tree state needs to be established at each node that is either a transmitter, a receiver, or a replication point.  If a node doesn't fall into one of these categories for a particular tree, that node will not need the per-tree state.  If a particular replication point on a tree is not directly attached to the next replication point downstream, unicast tunneling is used to get the packets from one replication point to the next.
DA> Correct

Much is made of the fact that per-tree data plane state exists only at nodes that are transmitters, replication points, or receivers for the tree.  This is presented as significant reduction in data plane state when compared to, say, PIM or mLDP.  Of course, whether there is any signficant reduction in state depends on the size and shape of the trees.  No analysis is given to show that the state reduction is significant, just a lot of handwaving.
DA> I would have thought it was self evident as described in the PIM and RTGWG presentations where the worst case is that (2xleaf count) nodes need to install state per tree irrespective of network size, and a fairly simple example provided. I don’t recall seeing you there. Paraphrasing a comment of yours from years ago you said that the role of a draft or RFC was to describe "what" and not "why", but if now you think some more "why" is needed we're happy to oblige.

On the other hand, there is an enormous increase in control plane state resulting from the fact that every single node in the network is aware of every tree and all its members.  Existing multicast protocols do not require a node to know about a tree unless the tree passes through it, and do not require all nodes along a tree to know about all the leaf nodes of the tree.  The large increase in the amount of control plane state is not considered at all, for some reason.
DA> I guess that is a bias where control plane memory is cheaper, and therefore more plentiful than dataplane state, making DP state something to optimize around.  We'll think about how to handle this.

To send a data packet along a particular tree, the SID for that tree is pushed onto the packet's label stack.  Each node that receives a packet and sees a particular SID at the top of the label stack will replicate the packet and send (or tunnel) each copy to the next replication point, where the same SID will be seen.  Since unicast tunneling is generally used to get a packet from one replication point to the next, packets traveling along a particular tree may arrive over any interface.  This makes it impossible to do a data plane RPF check (such as PIM would do).  
DA> If PHP is employed, then the multicast SID would be top label, and split horizon filtering would be possible as a form of an RPF check. We'll add this to the draft.   However this is a useful insight to be reminded of when considering what technologies the approach is applicable to as suggested by Toerless's post.  

Since the tree has a domain-wide unique SID, you can't tell from the label who the previous node was, so you can't reject packets from the "wrong previous node" (such as mLDP would do).  This may lead to potentially disastrous multicast loops during periods when a tree is being modified, especially if node A was previously upstream of node B, but the change puts it downstream of node B.  (Modifications can be caused by topology changes or simply by a change in the set of
receivers.)  These issues do not appear to have been considered.
DA> They were, just not elucidated in the draft itself.

I do not see how this document can be considered for WG adoption. Given the set of signficant issues that have not received proper consideration, it is not possible to say that this document is a good basis for future work.
DA> Personally I’m not seeing any show stoppers in your list of concerns. But that is my opinion.

Thanks for the feedback
Dave

_______________________________________________
pim mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pim

---

From: IJsbrand Wijnands <[email-masked]>
Date: Fri, 10 Aug 2018 00:56:24 +0200

Hi,

I'm apposed to the adoption of this draft.

I share the concerns that Eric and Toerless have expressed.

I also think a more high level discussion is needed to determine whether we need an other state-full protocol to build multicast trees. This draft proposes to create multicast state by using the IGP. Are the benefits of that so significant, compared to existing protocols, to justify building it? There will still be state, which will lead to convergence issues and complexity in operating it. Claiming that this idea follows the SR architecture because state is carried in the IGP is not a convincing argument.

Thx,

Ice.

> On 6 Aug 2018, at 20:24, Michael McBride <[email-masked]> wrote:
> 
> Hello good people,
>  
> Today begins a two week call for adoption of draft-allan-pim-sr-mpls-multicast-framework-00 which was presented in Montreal. There was a good discussion in the meeting and now it’s time to gauge interest in adoption on the list. Please respond and let us know what you think about adopting this draft.
>

---

From: John E Drake <[email-masked]>
Date: Thu, 9 Aug 2018 23:34:02 +0000

Dave,

Snipped, comment inline

Yours Irrespectively,

John

> -----Original Message-----
> From: pim <[email-masked]> On Behalf Of David Allan I
> Sent: Thursday, August 9, 2018 3:38 PM
> To: Eric Rosen <[email-masked]>; Michael McBride
> <[email-masked]>; [email-masked]
> Subject: Re: [pim] Call for adoption of sr-mpls-multicast-framework
> 
> Hi Eric:
> 
> the set of signficant issues that have not received proper consideration, it is
> not possible to say that this document is a good basis for future work.

[JD]  I think a willing suspension of disbelief is required to accept the premise that this draft is a good idea.

> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_mailman_listinfo_pim&d=DwIGaQ&c=HAkYuh63rsuhr6Sc
> bfh0UjBXeMK-ndb3voDTXcWzoCI&r=CRB2tJiQePk0cT-h5LGhEWH-
> s_xXXup3HzvBSMRj5VE&m=FmAN5tVJ4ecCeSKs0qRBQxq68UrC3yTDhm1qLL
> Q58d4&s=LV_CD3rAlzfEMJw66TyJB8oGJjH2SXTTa4MCDMvIElU&e=
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_mailman_listinfo_pim&d=DwIGaQ&c=HAkYuh63rsuhr6Sc
> bfh0UjBXeMK-ndb3voDTXcWzoCI&r=CRB2tJiQePk0cT-h5LGhEWH-
> s_xXXup3HzvBSMRj5VE&m=FmAN5tVJ4ecCeSKs0qRBQxq68UrC3yTDhm1qLL
> Q58d4&s=LV_CD3rAlzfEMJw66TyJB8oGJjH2SXTTa4MCDMvIElU&e=

---

From: Greg Shepherd <[email-masked]>
Date: Fri, 10 Aug 2018 10:29:38 -0700

I also do not support adoption. All that needs to be said has been said.
Thanks.

Cheers,
Greg

On Thu, Aug 9, 2018 at 4:34 PM, John E Drake <[email-masked]> wrote:

> consideration, it is
> > DA> Personally I’m not seeing any show stoppers in your list of
> concerns. But
> > that is my opinion.
>
>

---

From: "Jeffrey (Zhaohui) Zhang" <[email-masked]>
Date: Wed, 15 Aug 2018 13:24:05 +0000

I also share the concerns that have been said by others, and I object to the adoption of this document.

As I spoke at the mic in our Montreal session, this is similar to MOSPF and still maintains per-tree state in the core on roots, leaves and branch points (the only saving compared to MOSPF is that there is no state on other nodes on the tree, i.e. nodes that simply forward but not replicate an incoming packet). Compared to PIM or existing P2MP technologies, there is way more control plane state throughout the network in that it floods membership information everywhere.

A few of us prepared an informational draft "Multicast in SR Networks" (draft-zzhang-pim-sr-multicast - see attached email) that exams all options that are available or have been proposed (including this draft-allan-pim-sr-mpls-multicast-framework). I think we should have a good review and discussion on various options before we adopt this.

Thanks.
Jeffrey

> From: pim <[email-masked]> On Behalf Of IJsbrand Wijnands
> Sent: Thursday, August 9, 2018 6:56 PM
> To: Michael McBride <[email-masked]>
> Cc: [email-masked]
> I also think a more high level discussion is needed to determine whether we
> need an other state-full protocol to build multicast trees. This draft proposes to
> create multicast state by using the IGP. Are the benefits of that so significant,
> compared to existing protocols, to justify building it? There will still be state,
> which will lead to convergence issues and complexity in operating it. Claiming
> that this idea follows the SR architecture because state is carried in the IGP is
> not a convincing argument.
> 
> Thx,
> 
> Ice.
> 
> 
> > Today begins a two week call for adoption of draft-allan-pim-sr-mpls-
> multicast-framework-00 which was presented in Montreal. There was a good
> discussion in the meeting and now it’s time to gauge interest in adoption on
> the list. Please respond and let us know what you think about adopting this
> draft.
> >
> 3A__tools.ietf.org_id_draft-2Dallan-2Dpim-2Dsr-2Dmpls-2Dmulticast-
> 2Dframework-2D00.txt&d=DwIGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-
> ndb3voDTXcWzoCI&r=f7wsLGcfzAWDNS6XNTBZwj_OLAOsZZqdrR2IDAzeZqE&
> m=NoOAWfggpxm4roCZiMhp3okMVUF5wxSwu-
> DLHT1XUwo&s=fxb5b29a_7aafJd-3qsPbz_77rcBxXMuRkRp_cIKBRY&e=
> >
> > thanks,
> > mike
> 3A__www.ietf.org_mailman_listinfo_pim&d=DwIGaQ&c=HAkYuh63rsuhr6Scbf
> h0UjBXeMK-
> ndb3voDTXcWzoCI&r=f7wsLGcfzAWDNS6XNTBZwj_OLAOsZZqdrR2IDAzeZqE&
> m=NoOAWfggpxm4roCZiMhp3okMVUF5wxSwu-DLHT1XUwo&s=J1gx6t-cG_-
> aUSzxW8MxQwyuS1yglnQhBohO_1oLEzs&e=
> 
> 3A__www.ietf.org_mailman_listinfo_pim&d=DwIGaQ&c=HAkYuh63rsuhr6Scbf
> h0UjBXeMK-
> ndb3voDTXcWzoCI&r=f7wsLGcfzAWDNS6XNTBZwj_OLAOsZZqdrR2IDAzeZqE&
> m=NoOAWfggpxm4roCZiMhp3okMVUF5wxSwu-DLHT1XUwo&s=J1gx6t-cG_-
> aUSzxW8MxQwyuS1yglnQhBohO_1oLEzs&e=

Subject: New Version Notification for draft-zzhang-pim-sr-multicast-00.txt
From: "[email-masked]" <[email-masked]>
To: "Jeffrey (Zhaohui) Zhang" <[email-masked]>, Andrew Dolganow <[email-masked]>, IJsbrand Wijnands <[email-masked]>, Liang Geng <[email-masked]>
Date: Wed, 15 Aug 2018 13:14:48 +0000

A new version of I-D, draft-zzhang-pim-sr-multicast-00.txt
has been successfully submitted by Zhaohui Zhang and posted to the
IETF repository.

Name:		draft-zzhang-pim-sr-multicast
Revision:	00
Title:		Multicast in SR Networks
Document date:	2018-08-15
Group:		Individual Submission
Pages:		8
URL:            https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_internet-2Ddrafts_draft-2Dzzhang-2Dpim-2Dsr-2Dmulticast-2D00.txt&d=DwICaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=f7wsLGcfzAWDNS6XNTBZwj_OLAOsZZqdrR2IDAzeZqE&m=8jo9o94w8A8EYyBO9Ymv8tt4GtjWUllvPlZajxHPV58&s=T5XwI5JV07xnGVikpEnTJhY9bMg-cSLXO53qCVfXYBM&e=
Status:         https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dzzhang-2Dpim-2Dsr-2Dmulticast_&d=DwICaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=f7wsLGcfzAWDNS6XNTBZwj_OLAOsZZqdrR2IDAzeZqE&m=8jo9o94w8A8EYyBO9Ymv8tt4GtjWUllvPlZajxHPV58&s=cJB2FH8PMFDWjrAF7fxXA3B4OtUuREJ7X16D_M_Z3Oc&e=
Htmlized:       https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dzzhang-2Dpim-2Dsr-2Dmulticast-2D00&d=DwICaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=f7wsLGcfzAWDNS6XNTBZwj_OLAOsZZqdrR2IDAzeZqE&m=8jo9o94w8A8EYyBO9Ymv8tt4GtjWUllvPlZajxHPV58&s=C7UZ8PCvUHMHizFwlAlO34CByVOW4hr3UjTGiZNTX1s&e=
Htmlized:       https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_html_draft-2Dzzhang-2Dpim-2Dsr-2Dmulticast&d=DwICaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=f7wsLGcfzAWDNS6XNTBZwj_OLAOsZZqdrR2IDAzeZqE&m=8jo9o94w8A8EYyBO9Ymv8tt4GtjWUllvPlZajxHPV58&s=C4YFPXbkIhy36l501IvtEsDde971fUJksPcb4cDOwko&e=

Abstract:
   With Segment Routing (SR) architecture [rfc8402], a unicast flow can
   be routed through an SR network following an explicit path specified
   in the packet, w/o the need for per-flow state in the network.  As a
   result, the otherwise needed protocols to signal the per-flow unicast
   state can also be removed from the network.  This document discusses
   options for multicast in an SR network.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat