Subject: Comments on draft-morin-l3vpn-mvpn-considerations-01.txt

From: Eric Rosen <[email-masked]>
Date: Wed, 12 Dec 2007 09:34:01 -0500

I have carefully  reread draft-morin-l3vpn-mvpn-considerations-01.txt, and I
have a number of comments.

In  general, I  think the  authors have  done  a good  job in  trying to  be
even-handed, but  I think there are some  errors in the details,  and I also
think that some of  the risks and overheads involved in the  use of BGP have
not been fully considered.  I also don't think that the different flavors of
MPLS multicast  LSPs have  been carefully considered.   I would like  to see
some of these issues addressed before adopting it as a WG draft.

On the general issue of comparing PIM vs. BGP, while the draft does not make
a recommendation, it  is heavy on PIM's disadvantages  and BGP's advantages.
So I  think it is  worth listing some  of the risk areas  of BGP.  I  do not
believe that  any of  these are "showstoppers"  for BGP, but  they certainly
should  be articulated  in  any document  that  purports to  be providing  a
balanced view. 

- As BGP  is generally  used, BGP churn  is a  function of real  or apparent
  topology changes.  BGP churn is NOT generally caused by enduser events.

  This changes when  BGP is used to distribute  multicast routes.  BGP churn
  can now  be initiated by PIM  Joins, and a PIM  Join may be  the result of
  some enduser clicking  a button on his PC (e.g.,  "receive video").  it is
  not clear that there is any limit on the rate at which such enduser events
  can occur.  We do not appear to  have adequate data from which we can draw
  conclusions as to the impact of enduser actions on BGP churn.

  It is easy to offer opinions about this, but facts are scarce.

- In some  multicast applications, "join  latency" is considered to  be very
  important.  No  one has ever claimed  BGP will provide  lower join latency
  than  PIM, the only  real question  is whether  latency will  be increased
  beyond  the  limits  that   some  applications  find  acceptable.   Again,
  different people have  different opinions about how much  this will matter
  in practice, but hard data is hard to come by.

- Churn  vs. latency:  To  control the  BGP  churn, we  have discussed  such
  features as  Join Dampening.   However, that can't  be expected to  have a
  salutary effect on the latency!

  There really should  be some recognition in the draft  that the effects on
  latency and churn are as yet poorly understood.

- Any time a new high volume address family is added to BGP. it is important
  to consider  the impact on  the route reflectors,  and whether it  will be
  necessary to add  new route reflectors, perhaps ones  dedicated to the new
  family.  One should also consider the impact (especially on the RRs) which
  one address family  may have on another, e.g.,  will an unexpectedly large
  amount  of churn  due  to PIM  cause  a slowdown  in  the unicast  routing
  convergence.  There should be some discussion of this.

- Transactional nature of PIM

  At first glance,  it may seem that having  PIM distribute multicast routes
  to BGP is no different than  having OSPF distribute unicast routes to BGP,
  i.e., that it is really  nothing new.  However, this isn't quite accurate.
  The issue  is that PIM isn't really  a "routing protocol" in  the sense of
  choosing  routes  based   on  topology.   It  is  a   protocol  that  uses
  router-to-router  transactions  to construct  paths  and  assign flows  to
  paths. In  some cases, especially when  Sparse Mode flows  are involved, a
  number of  transactions involving  multiple nodes may  need to  take place
  before the multicast "routing" stabilizes.  Each one of these transactions
  must  pass through  the RR,  placing more  load on  the RR  and increasing
  latency.

  In  some  cases, where  PIM-SM  has  data-driven  state changes,  we  have
  attempted  to avoid  putting the  data-driven state  changes into  BGP, by
  having BGP send extra  messages, including timer-driven messages.  While I
  tend to think that this is the right tradeoff, it does modify the dynamics
  of path construction,  and we don't have any hard data  to tell us whether
  we've made the right tradeoff,  or whether the "right tradeoff" depends on
  just how the applications are using multicast.
  
- Strange uses of PIM

  Talk  to  anyone  with a  lot  of  experience  in  PIM deployment  in  the
  enterprise, and you'll hear lots of  strange stories about how PIM is used
  in some enterprises.   Although PIM is optimized for  "many receivers, few
  sources", it isn't always used  that way.  I've also heard anecdotes about
  applications that make assumptions about the dynamic nature of PIM and may
  break if  anything changes.   As usual,  hard data is  hard to  find.  But
  there's always a risk that when you change the infrastructure underneath a
  crufty old application you will break the application.  There are people I
  respect, with  much more experience  in multicast deployment than  I have,
  who think it's  just foolish to do anything that changes  the way in which
  the infrastructure behaves for sparse mode.  While I am not convinced that
  they are  right, I think that  this does have  to be recognized as  a risk
  element.

Now let's look at some of the topics covered in the draft.

1. From section 3.1, "the operational burden of setting up multicast on a PE
   or for a VR/VRF SHOULD be as low as possible".

   I certainly  agree with  this!  However, I  don't think it  combines well
   with the  later recommendation to support outsourced  RPs.  Obviously the
   use of outsourced RPs increases the operational burden for the provider.

   I  note that  there doesn't  seem to  be any  requirement for  making the
   operational burden  of MVPN on the  SP's customer be as  low as possible,
   though such a requirement is  prominent in another SP requirements draft,
   viz.,  draft-mnapierala-mvpn-part-reqt-00.txt.  Such a  requirement would
   see to  militate against  the use  of the "outsourced  RP", at  least for
   MVPN customers who already use  multicast.  (Which I imagine would be the
   vast majority of them.)

2. Auto-discovery

   "BGP-based  auto-discovery is the  preferred solution  for auto-discovery
   ...  while  PIM/shared-tree  based  auto-discovery should  be  optionally
   considered for migration purposes only".

   There isn't  really a  dichotomy here.  The  facts are that  if PIM-based
   shared  trees  are  used,  BGP-based auto-discovery  doesn't  really  add
   anything,  but in  any  other circumstance,  BGP-based auto-discovery  is
   absolutely necessary.

   But  if the recommendation  here is  to provide  BGP-based auto-discovery
   even in the one case where  it isn't strictly needed, I certainly support
   that recommendation.  This  is, in fact, already true  of the majority of
   existing MVPN  deployments.  (Though  I have been  told that there  is at
   lest one vendor which does not support it in existing deployments.)

   There is a suggestion in the  draft that if PIM-based shared trees are in
   use, BGP auto-discovery can  help detect misconfigurations.  I'm not sure
   about that, as I can imagine  transition scenarios in which more than one
   shared tree is in use at some time.

3. Number of protocols

   "if any additional protocols are introduced compared with the unicast VPN
   service,  the  balance between  their  advantage  and operational  burden
   SHOULD be examined thoroughly"

   Well, who  could disagree with that?   The pros/cons of  any new protocol
   should always be  examined thoroughly.  In fact, the  pros/cons of adding
   new  functionality  to  existing  protocols  should  always  be  examined
   thoroughly  as well.   As  should  the pros/cons  of  adding new  address
   families, new procedures,  new dynamics, new events and  states, etc., to
   existing protocols.

   It is better to focus on the overall complexity of the system, and on the
   overall  behavior  of  the system,  than  just  to  count the  number  of
   protocols. 

4. S-PMSI Signaling

   The BGP-based S-PMSI  signaling mechanism does have a  risk factor that I
   have mentioned above, but that is not considered in the draft

   In current practice,  BGP changes are caused by  topology changes, not by
   end  user  activity.   However,   it  is  enduser  activity  that  causes
   invocation of the S-PMSI signaling  mechanism.  We simply do not have the
   data at  the present time  to determine how  much this will  increase the
   amount of churn  in BGP.  The S-PMSI signaling is  also something that is
   best done through a low latency  mechanism, and we do not at present have
   the data to determine whether BGP will significantly increase the latency
   when compared with the UDP-based proposal.

   In theory,  the use of BGP  S-PMSI signaling has a  number of advantages,
   but I  don't think  I would  recommend against the  use of  the UDP-based
   procedure unless and until I had some data about these pragmatic issues.

   By the  way, when I  say "data", I  don't mean "strongly held  and loudly
   voiced  opinions", even  though the  latter are  sometimes  confused with
   data.

   "The  UDP-based protocol  is  restricted  to use  within  MVPNs using  an
   MI-PMSI".

   This fact seems to me to be  neither here nor there.  It is true that the
   UDP-based protocol is unlikely to be useful unless PIM is the C-multicast
   routing control protocol, in which case  you have an MI-PMSI.  If you are
   using  BGP as  the C-multicast  routing protocol,  then you  have already
   determined that you don't care if BGP churn is caused by enduser events.

   The UDP-based S-PMSIs signaling is best considered as an optional part of
   the  PIM C-multicast  routing control  plane, rather  than as  a separate
   protocol with its own independent set of pros and cons.

   "the  use  of  the  UDP-based  protocol  does  not  preserve  AS  routing
    independence  when  used in  an  inter-AS  option  B context  (i.e.  the
    decision by  a PE in an  AS to use an  S-PMSI for a  given customer flow
    will impact routing state in other ASes)

   I  have no  idea what  this means,  so I  hope one  of the  authors will
   explain.   (I also  don't  know what  is  meant by  the  claim that  the
   BGP-based S-PMSI signaling  mechanism doesn't have whatever disadvantage
   this is.)

5. Off-loading processing onto route reflector

   The claim is made that, when using BGP to carry C-multicast routes, "some
   of  the processing burden  associated with  client multicast  routing [is
   offloaded] onto BGP route reflectors"

   I  would  like the  authors  to  specify  precisely what  this  offloaded
   processing burden is.  Some consideration of its effect on existing L3VPN
   route reflector systems would also be worthwhile.

   As the  text is now, it is  very difficult to understand  or evaluate the
   claim.

6. MI-PMSI issues

   "Moreover, mechanisms one and two are restricted to use within MVPNs
   using an MI-PMSI, thereby necessitating:

   a.  The use of a P-multicast tree technique that allows shared trees
       (for example PIM-SM in ASM mode or MP2MP LDP).

   b.  The use of one P-multicast tree per PE per VPN, even for PEs that
       do not have sources in their directly attached sites for that
       VPN."

   First,  claim b  is  not true,  as  explained in  draft-rosen-l3vpn-mvpn-
   profiles-00.txt, section 3.3.

   Second, the fact that a control  protocol requires that it be possible to
   instantiate a P-tunnel as a shared tree  of some sort does not seem to be
   to be  a "restriction" or disadvantage.   You might as well  say that the
   use  of  BGP  for  C-multicast  routing has  the  disadvantage  of  being
   "restricted"  to those networks  whose route  reflectors support  the BGP
   C-multicast routing address family.

   There are however some other things  that do seem to require the presence
   of an MI-PMSI:   autoRP and BSR.  Since many  SP customers use autoRP/BSR
   to discover  RPs, and since  it is not  clear how to provide  support for
   these without an  MI-PMSI, I am not convinced that one  can do without an
   MI-PMSI even if BGP is used for distributing C-multicast routes.

7. BGP doesn't eliminate PIM anyway
   
   "using BGP for customer routing distribution within multicast VPNs avoids
   the introduction of an  additional protocol that would require additional
   OAM processes and tools."

   It's not as if  PIM is eliminated by BGP.  As long as  PIM is used on the
   PE/CE links, the SP is still operating a PIM environment.  In fact, there
   may be significant  scaling problems due to PE-CE  PIM.  Some people with
   years of PIM  experience believe that the PE-CE  PIM scaling problems are
   actually worse  than the PE-PE PIM  scaling problems, and  hence that the
   attempt  to improve PE-PE  scaling by  using BGP  just attacks  the wrong
   problem. 

   In the absence of  actual data, the risk is that you  end up adding a lot
   of NEW stuff and don't even address the scaling bottlenecks.
 
   In  any  event, I'm  not  sure  I understand  how  adding  a  lot of  new
   functionality  to BGP  (multicast  path  creation) is  going  to be  done
   without the need for additional OAM processes and tools.

8. Consistency with unicast control mechanisms.

   "An illustrative example of the benefit brought by consistency with
   unicast design is how the "extranet" feature can be implemented :
   when BGP-based mechanisms are used, the well defined and well
   understood BGP route target import/export semantics are just reused."

   This seems like a red  herring.  Extranet capability is already supported
   in  the field  today  by existing  MVPN  deployments based  on PIM.   The
   primary impact is really on the auto-discovery phase, not the C-multicast
   routing phase.

9. Encapsulation techniques

    I notice that  there is one particular data plane  technique that is not
    addressed  by the  document, viz.,  the use  of ingress  replication and
    transmission through  unicast tunnels.  It would be  interesting to know
    whether SPs  have interest in this  technology; if not, maybe  it can be
    removed from the  drafts.  (I think it adds  complexity and frankly, I'm
    not sure it's fully and properly specified.)

    I think section 3.4 can be taken as a requirement for both a "BGP + GRE"
    profile  and a  "PIM +  MPLS" profile,  since it  seems to  advocate for
    independence between the the control  plane and the data plane.  But I'm
    not sure if that is what is intended, and would like clarification.

    I would like  to see a specific recommendation for  the support of MP2MP
    LSPs,  given their  importance in  C-bidir support  (for  either control
    plane, see draft-ietf-l3vpn-2547bis-  mcast-05.txt, section 12.2) and in
    general MPLS  multicast support  for the PIM  control plane  (see draft-
    rosen-l3vpn-mvpn-profiles-00.txt, section 3.3).

10. Upstream-Assigned Labels, MI-PMSI, and Scalability
   
    The  WG drafts frequently  discuss aggregation,  and the  possibility of
    aggregation is frequently touted as a major scalability advantage.

    What isn't always clear is  that all forms of S-PMSI aggregation require
    a  major   new  MPLS  feature,   upstream-assigned  labels.   Supporting
    upstream-assigned labels requires significant  changes to the data plane
    processing  of  many  different  platforms,  and is  not  likely  to  be
    available for some period of time.

    So realistically, for  the foreseeable future, the use  of an MI-PMSI is
    going  to remain  the  only  method of  aggregation.   Unless a  service
    provider does not care if their  P-routers have to maintain an amount of
    multicast state  which is proportional to  the sum of the  states of all
    their customers, the service provider should be requiring support for an
    MI-PMSI. 

    As  a further  note,  it should  be  pointed out  that inter-AS  segment
    S-PMSIs are on a per-PE basis, not a per-AS basis.  Therefore without the
    aggregation that  can be obtained  from upstream-assigned labels,  it is
    not clear that segmented inter-AS  S-PMSIs are scalable at all.  This is
    an issue which should be mentioned.

    I think the draft should have a clear recommendation that in the absence
    of upstream-assigned MPLS labels, MI-PMSIs should be used.

11. Outsourced RPs

    I  would like  to see  the  draft offer  much stronger  support for  its
    recommendation to  implement outsourced RPs,  or else to  eliminate that
    recommendation. 

    Outsourced RPs  (i.e., PE  as RP) have  a number of  disadvantages which
    need to be given more emphasis:

    - More work  for the PE (do  many SPs really want  to give a  lot of new
      work to their PEs)?

    - Not  likely to  be  of much  interest  to customers  with an  existing
      multicast infrastructure

    - Require  increased coordination  between SP  and  enterprise customer,
      such as agreement on RP (or RPL) addresses.
    
    - Not completely clear  how this works in scenarios  where an enterprise
      gets VPN  service from two different providers,  particularly if there
      is a site multihomed to two different providers.

    I presume  that the extra  operational work for  SP and customer  is not
    going to be justified by the  fact that it simplifies things for some of
    the SP's vendors.

12. The draft's conclusion

   "Consequently, at the present time and until there is experience with
   all of the proposed mechanisms it is not clear which of the above
   mechanisms should be recommended as the preferred solution to
   implementers.  However, it would appear prudent for implementations
   to consider supporting both the fourth (BGP-based) and first (full
   per-MPVN PIM peering) mechanisms.  Further experience on both
   implementation is likely to be required before some best practice can
   be defined."

   I certainly agree with this  conclusion.  Perhaps it should be given more
   prominence. 

13. My conclusion

    While  I do  think this  draft may  be a  good start  on the  project of
    comparing the  different MVPN options, I  don't think it is  ready to be
    adopted as a WG document, and it  needs a lot more work to deal with the
    issues I have raised in this note.

    I  also hope  it  is clearly  understood  that this  document should  be
    evaluated by the same criteria as any other internet-draft, i.e., by its
    technical quality.

---

From: Yakov Rekhter <[email-masked]>
Date: Fri, 14 Dec 2007 06:01:23 -0800

Eric,

> not been fully considered.
>

You fail to provide any specific facts below on the subject of risks
and overhead. Hence this can hardly be used as an objection against
progressing this document as a WG draft (see below for more...).

>  I also don't think that the different flavors of
> MPLS multicast  LSPs have  been carefully considered.
>

Agreed on this. However, considering different flavors of MPLS
multicast LSPs can be done after the document becomes a WG draft.
Remember that for a document to become a WG draft it doesn't have
to be perfect.

>    I would like  to see

See the previous comment.

> should  be articulated  in  any document  that  purports to  be providing  a
 
The viewpoints should be based on rational *technical* arguments
and not FUD, and/or exercise in political correctness.

>   It is easy to offer opinions about this, but facts are scarce.

Precisely. So why should the draft talk about this since facts are
scarce ?

Moreover, what we are talking about here are two things (a) changes
in the multicast state on PEs due to receiving PIM Join/Prune from
CEs, and (b) a particular mechanism to propagate such changes among
PEs. The issue here is not "BGP churn", but the control plane load
due to (b). You failed to explain why control plane load due to (b)
would be less with PIM than with BGP. 

>   in practice, but hard data is hard to come by.

Again you fail to provide facts. This is just FUD. Moreover since PIM is
based on unreliable message exchange, "join latency" with PIM could be in
10s of seconds.

>   latency and churn are as yet poorly understood.

Section 16 of draft-ietf-l3vpn-2547bis-mcast-bgp discusses this in
sufficient detail. Moreover, Join Dampening is one, but not the
only one way to control control plane load. Another one is dampening
of withdrawls, which is described in Section 16.1. And quoting from
that section:

  "Dampening of withdrawals of C-multicast routes does not impede the
   multicast join latency observed by MVPN customers,"

Since you are a co-author on that draft, you should be aware of this.

>   convergence.  There should be some discussion of this.

Section 15 of draft-ietf-l3vpn-2547bis-mcast-bgp addresses this.
Since you are a co-author on that draft you should be aware of this.
And if you think something is missing, you as a co-author should
be able to add the appropriate text to that draft.

>   must  pass through  the RR,  placing more  load on  the RR  and increasing

Can you give a precise example ?

>   just how the applications are using multicast.

As worded above this is yet another FUD.

>   they are  right, I think that  this does have  to be recognized as  a risk

This is yet another FUD.

>    MVPN customers who already use  multicast.  (Which I imagine would be the

First of all, it is for SPs to decide whether they want this option
or not,  as they are in a much better position than the vendors to
figure out whether this supposed additional burden offers a benefit.

Second, if you argue in favor of reducing operational burden of
MVPN on the SP's customers, then outsourcing RP does reduce such
burden, as it moves the burden of managing customer's RP from
customers to the service provider. In fact, this is fairly consistent
with the main point of 2547 VPNs - outsourcing routing to the service
provider. Outsourced RP is just an example of doing this in the
context of multicast.

Finally, the draft also requires to support non-outsourced RP as
well.

>    shared tree is in use at some time.

I don't see any strong objections here. But again, reflecting this
in the draft could be done after the draft gets accepted as an L3VPN
WG document.

>    overall  behavior  of  the system,  than  just  to  count the  number  of

While there is nothing wrong in focusing "on the overall complexity
of the system", one should not ignore the fact that the number of 
protocols required does impact the complexity.

>    invocation of the S-PMSI signaling  mechanism.  We simply do not have the

This is not exactly accurate, as S-PMSI signaling is triggered not
by the PEs connected to the receivers, but by the PE connected to the
source.

>    amount of churn  in BGP.

Again this is FUD, and not based on any facts. Moreover, it ignores
the fact that what we really talking about here is the control
plane load due to S-PMSI signaling. Please explain why the control
plane load due to S-PMSI signaling with BGP be any more than with
UDP-based signaling. 

>    The S-PMSI signaling is  also something that is

Perhaps you forgot that the UDP-based S-PMSI signaling doesn't have
a predictable bound on latency as its non-reliable. So a loss of a
message results in 60 secs latency.

And while we on this topic, could please quantify "significantly"
as you assert that "BGP will significantly increase the latency".

>    procedure unless and until I had some data about these pragmatic issues.

This is yet another FUD.

>    protocol with its own independent set of pros and cons.

This could be made after the draft is accepted as a WG document
(and it should be *based on the WG consensus*).

>    BGP-based S-PMSI signaling  mechanism doesn't have whatever disadvantage

It means couple of things. First of all, UDP based S-PMSI forces
all ASes to switch to the S-PMSI. In contrast with BGP based segmented
inter-AS S-PMSIs, each AS can decide on its own whether (a) to
propagate the S-PMSI or (b) aggregate S-PMSI into an I-PMSI.

Second, with UDP-based S-PMSI all the ASes have to use the same
tunneling technology. In contrast BGP-based S-PMSI supports segmented
inter-AS tunnels, which enables each AS to make its own choice with
respect to tunneling technology (something which is already supported
for 2547 unicast).

>    route reflector systems would also be worthwhile.

As for the last "effect on existing L3VPN route reflector systems",
please note the the following text in Section 15 of
draft-ietf-l3vpn-2547bis-mcast-bgp:

   Moreover, Route Reflectors used for MVPN do not have to be 
   used for VPN-IP routes

And as a reminder, you are a co-author on that draft !

>    profiles-00.txt, section 3.3.

What is described in section 3.3 of the sited draft restricts
tunneling technology to just *one* - mp2mp LSP with mLDP. As such 
what is described in section 3.3 of the sited draft violates one
of the requirements from rfc4834:

    the control plane SHALL NOT depend on which forwarding plane is used

>    MI-PMSI even if BGP is used for distributing C-multicast routes.

First of all, autoRP is a cisco proprietary technology. Asking
standars to support cisco proprietary technology is inappropriate.

Second, just because you are "not convinced", does not mean it can
not be done.  In fact, just to remind you, supporting BSR without
MI-PMSI has been explained in the presentation at the L3VPN WG
meeting (with you as one of the co-authors on that presentation).

>    PE/CE links, the SP is still operating a PIM environment.
>

It is not the same as using PIM in the backbone.

Moreover, it is not just about eliminating PIM, but also about
having the same architecture for both unicast and multicast vs
requiring two fundamentally different architectures, one for unicast
(aggregated routing), and another for multicast (virtual routers).

>    In fact, there

Let the SPs speak up and tell us that !

>    without the need for additional OAM processes and tools.

Without any specifics this statement is yet another FUD.

>    primary impact is really on the auto-discovery phase, not the C-multicast

What is "in the field today" is hardly based on reuse of 2547 unicast
mechanisms. And moreover, it uses a completely different architecture
(Virtual Routers).

In fact, can you explain why having one architecture for unicast
VPN (aggregated routing) and another architecture for multicast VPN
(virtual routers) is any better than having the same architecture
for both unicast and multicast VPN ?

>     not sure it's fully and properly specified.)

As for the former, I will let SPs decide. As for the latter can you point
out *precisely* what is not "fully and properly specified" ?

>     rosen-l3vpn-mvpn-profiles-00.txt, section 3.3).

The flavor of MPLS multicast LSPs can be clarified after adopting this 
as a WG draft.

>     going  to remain  the  only  method of  aggregation.
>

Do you speak just for cisco, or for all other vendors as well ?

>     Unless a  service
>     not clear that segmented inter-AS  S-PMSIs are scalable at all.
>

Are they less scalable than non-segmented inter-AS S-PMSIs ?

>     This is an issue which should be mentioned.
>

I-PMSI vs S-PMSI offers service providers a tradeoff between state and
b/w. Each service provider should be able to make such a tradeoff on its
own, without standards mandating a particular tradeoff (in your own words
"one size doesn't fit all")

With this in mind the recommendations should reflect position of
SPs, not vendors (and you are part of the latter, not the former). 

>     - More work  for the PE (do  many SPs really want  to give a  lot of new

If less work for the PEs is the goal, then one should not run 2547 VPNs,
but rather L2 VPN, or CE-based VPNs.

>     - Not  likely to  be  of much  interest  to customers  with an  existing

Do you speak for the service providers ? As if not, then this is
to be determined by the service providers, not by vendors.

>       such as agreement on RP (or RPL) addresses.

Let's put this in a proper perspective (and strip the FUD).  For
all CE-PE links there has to be coordination between SP and customers
in terms of addresses used for these links. Requiring coordination
for *one more address* (and that is all what you need for outsourced
RP or RPL) does not seem like a significant increase.

>       is a site multihomed to two different providers.

What precisely is not "clear" ?

>     issues I have raised in this note.

Of the issues raised by you in the above the only legitimate ones
seems to be (a) more analysis on different flavor of multicast LSPs,
(b) whether there is a need for BGP-based auto-discovery with
PIM-based shared trees, and (c) whether the UDP-based S-PMSIs
signaling should be best considered as an optional part of the  PIM
C-multicast  routing control  plane.  All of these issues can be
addressed after the document gets adopted as a WG draft.

The rest of your issues fall in the category of either (a) FUD (not
subsantiated by facts), or (b) asking for editorial clarifications,
or (c) telling the SPs what choices they have to make, rather than
letting SPs to make these choices on their own.  None of these are
legitimate objections to accepting the draft as an L3VPN WG
document.

With this in min I support accepting 
draft-morin-l3vpn-mvpn-considerations-01.txt as an L3VPN WG document.

Yakov.

---

From: Eric Rosen <[email-masked]>
Date: Fri, 14 Dec 2007 11:19:43 -0500

I think Yakov's  reply makes it pretty clear that  adoption of this document
as a WG document at the current time is premature.

I was  prepared for someone to  say that the  issues I raised can  be worked
after the  document is accepted.  However,  we see from  Yakov's answer that
he,  at least,  wants to  suppress discussion  of all  the  remaining issues
(after all, they're  all FUD ;-)).  I didn't think my  reputation in this WG
was that of a FUD-provider, but YMMV ;-)

For the kind of document that draft-morin seems to be, I think it is the job
of the authors to investigate all  the issues, not to say "we won't consider
any issue until some  else proves beyond a shadow of a  doubt that there's a
real problem".  Of  course, I'll be happy to work with  the authors to flesh
out  all the  arguments  pro and  con.  I'm  not  trying to  prove that  the
BGP-based scheme won't  work, but I'd have thought  that everyone knows that
all proposals have  pros and cons; a "considerations"  document should bring
out all the issues.

In fairness, Yakov isn't one of the  authors of the document, so it would be
interesting to know whether the authors  of the document agree that there is
no need to discuss any of the issues I've raised.

A few of the more outrageous comments do require some reply:

> why should the draft talk about this [variety of factors for which we lack
> hard data] since facts are scarce ?

Any choice  which is based  on opinion  rather than facts  is a risk,  and a
"considerations"  document should certainly  make it  clear when  such risks
exist.

> Moreover since PIM is based on unreliable message exchange, "join latency"
> with PIM could be in 10s of seconds.

The worst  case join latency in  either case could be  arbitrarily long, but
I'm sure everyone knows ((well, almost everyone ;-)) that's it's the average
join latency that is important.  Since packets don't usually get lost in the
backbone, that's  not much of  an issue.   I don't think  it takes a  lot of
fancy analysis  to determine  that it's  faster to send  a data  packet from
point A  to point B then  to send it from  point A to  an intermediate point
where it  is processed and then afterwards  sent along to point  C.  One can
argue  that the  latency increase  isn't significant,  or that  it's  a good
tradeoff  against something  else,  but it's  just  silly to  deny that  the
latency increase exists!

> could  please  quantify  "significantly"  as  you assert  that  "BGP  will
> significantly increase the latency".

I did  not state that  BGP will significantly  increase the latency,  I said
that we  do not have  the data to  determine whether BGP  will significantly
increase the latency.

A "significant  increase" would be one  that causes a visible  effect to the
users of a multicast-based application.

>  "Dampening of  withdrawals [as opposed to updates]  of C-multicast routes
>   does not impede the multicast join latency observed by MVPN customers,"

Sure, but  it doesn't have the same  effect on churn reduction  as does join
dampening, and it has other side-effects as well.  This is certainly an area
that would benefit from more analysis in any "considerations" document.

> Since you are a co-author on that draft, you should be aware of this.

I'm glad  you remembered that I'm  a co-conspirator (err,  co-author) of the
BGP-based multicast control plane.  As  a co-author, I am familiar with lots
of  issues  that  might  not  necessarily  be  obvious  to  the  authors  of
draft-morin.  Further,  I appear  to be the  only co-author who  thinks that
both  BGP and  PIM have  merits, and  that it  is important  to do  a pretty
thorough and unbiased comparison.

> And as a reminder, you are a co-author on that draft !

Yes, I am well aware that you  can add new RRs expressly to handle multicast
routes.  In  a "considerations" document, the  need (or lack of  need) to do
this is one of the considerations  that should be discussed.  Unless the SPs
are willing to say up front "not to worry, we don't care how many additional
RRs we need, they come for free and do not increase our operational burden".

As you know, I am a big fan of the 2547 technology :-), but the issue of how
the VPN-IP routes affect BGP generally (and how much "BGP churn" they cause)
has  always  been with  us.   At  least VPN-IP  routes  do  not behave  much
differently than  IP routes, since  updates are still triggered  by topology
changes and  not by  enduser activity.  MVPN  introduces a different  set of
dynamics to BGP and the impact on  the rest of the BGP system is certainly a
consideration that needs to be better understood.

It is fair  to point out that additional BGP activity  is at least partially
compensated for by reduced PIM activity.  On the other hand, PIM activity 
doesn't impact the BGP system, so this comparison is not straightforward.

> for both unicast and multicast VPN ?

I  think the  choice between  the two  schemes should  be based  not  on the
elegance of  the architecture but on  the multitude of  facts that determine
(a) how well the multicast applications in the enterprise will be supported,
and (b) how much operational impact they have on the SP.  

Further, multicast  routing is  really a very  different thing  than unicast
routing, and  there is  no history of  success in  combining the two  into a
single architecture (e.g., MOSPF).

This is  not to say  that BGP is  a poor choice, but  just to explain  why I
don't  think that  the  uniformity  of the  architecture  is the  overriding
considerations. 

> standards to support cisco proprietary technology is inappropriate.

If  an enterprise  is using  proprietary technology,  and if  a SP  wants to
provide MVPN service to that enterprise,  then I think the SP will certainly
want to know whether he has to  tell the enterprise "no MVPN service for you
until you change your deployed technologies" ;-)

A large number of enterprises use Cisco technology, and I imagine that other
vendors would like to be able to provide PEs that support those enterprises.

> supporting BSR without  MI-PMSI has been explained in  the presentation at
> the  L3VPN  WG  meeting  (with  you  as one  of  the  co-authors  on  that
> presentation).

Many of  the presentations  we've given over  the years  presented solutions
which  turned out  to  be  unsatisfactory.  Certainly  the  support or  lack
thereof  for  BSR  is  something  that a  "considerations"  document  should
examine.

>> realistically, for  the foreseeable future, the use  of an MI-PMSI is

> Do you speak just for cisco, or for all other vendors as well ?

I would highly recommend that before eliminating the MI-PMSI, the SPs should
get solid  delivery dates from  their vendors for  upstream-assigned labels!
;-)

>> it is not clear that segmented inter-AS  S-PMSIs are scalable at all.

> Are they less scalable than non-segmented inter-AS S-PMSIs ?

In my mind, the issue is whether there's a real good inter-provider solution
at all. 

> If less work for the PEs is the goal, then one should not run 2547 VPNs,

Well, I'd like to hear the SPs  say that they don't care how much additional
work  the PEs  have to  do!  Or  that they  don't care  how  much additional
coordination they have  to do with their customers, or  that they don't care
how many  changes the customers  have to make  before they can buy  the SPs'
service. 

> the recommendations should  reflect position of SPs, not  vendors (and you
> are part of the latter, not the former).

I've yet to hear any SP say that they don't want my input because I work for
a  vendor.  If  any  do  feel that  way,  I'm sure  they  can  speak up  for
themselves.

---

From: Thomas Morin <[email-masked]>
Date: Fri, 14 Dec 2007 17:43:40 +0100

Hi Eric,

I'm surprised to read such a long email from you, given that you didn't
make any comment on the presentation last Wednesday.

I think we would be, all co-authors of this document, happy to make
clarifications to address your comments.  There are definitely some
areas where content can be improved to be more precise on the pros/cons
of the different approaches. It is precisely the kind of feedback we
asked. I will go more in depth in a further email on items that I see
can be worked on.

However, among the items mentioned in your comments, there are also
points which are a bit "philosophical", or vague, or confusing, not to
mention a few ones which seem to me as erroneous or clearly misleading.
I'll explain in another email why I think some of your comments fall in
these categories, but to get to the point : I don't think diluting the
real content into risk-related worries or philosophical aspects would be
a good idea. On the contrary, we should focus on facts-driven
considerations that help distinguish relative merits on different
options between which we need to make a recommandation.  In light of
this, I don't think we can promise to say "yes sure we'll cover it" to
all your worries...

But in the end, given the support of the working group I would not
expect the improvements to be made as requiring delaying the adoption of
the draft by the working group, since it seems that we agree on the
intent, structure and general direction of the document. After all, if
adoption of a work by a working group was requiring having finished the
doc, then adoption as WG item and Last Call could be made at the same
time, right ?  ;-)

So, night is coming and I'll get back to thinking about buying christmas
gifts and follow up on this next week.

Cheers,

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Fri, 14 Dec 2007 17:52:49 +0100

Hi Eric,

Eric Rosen :
> (after all, they're  all FUD ;-)).  
[...]
> In fairness, Yakov isn't one of the  authors of the document, so it
> would be interesting to know whether the authors  of the document
> agree that there is no need to discuss any of the issues I've raised.

Please let me restate that the intent indeed is to rationally treat all
issues that can help make recommandations on the preferred approaches.

> I didn't think my reputation in this WG was that of a FUD-provider, but YMMV ;-)

What is sure is that, for some of the points you raised, it will be
useful to have more facts and/or explanations so that we can cover them
in our document.

Have a good week-end,

-Thomas

---

From: Eric Rosen <[email-masked]>
Date: Fri, 14 Dec 2007 12:56:41 -0500

> if adoption of a work by a working group was requiring having finished the

Absolutely.  But it  would be nice if there were some  agreement on just how
much more work it needs.

Until we've had  a fuller discussion on  which of my concerns do  and do not
need to  be addressed,  it's hard  to judge whether  the authors  are really
willing to  take this document in  the right direction.  That's  why I would
like to delay adopting it as a WG document.  

> we  should  focus on  facts-driven  considerations  that help  distinguish
> relative merits on different options

I thought that was the point I was making.  There are just too many areas in
which  the  relevant facts  are  either not  known  or  not considered.   If
decisions are being made based on assumptions or opinions rather than facts,
that should certainly be called out.

> gifts and follow up on this next week.

Fair enough!

---

From: Rahul Aggarwal <[email-masked]>
Date: Fri, 14 Dec 2007 11:43:20 -0800 (PST)

Hi Eric,

First, to me the goal of draft-morin is to provide a set of
mandatory and optional, options (from the set of options in
draft-ietf-l3vpn-2547bis-mcast) that would allow SPs to deploy
inter-operable implementations using technology/options that the *SPs
want*.

This is a very important excercise for the L3VPN WG to progress on the
MVPN subject. draft-morin IMO is an excellent start towards this
excercise. It should be adopted as a WG document and can evolve to address
*valid concerns*.

And valid concerns do not include philosophical arguments.
Unfortunately several of your concerns were philosphical in nature and do
not help with constructive progress.

More below:

> as a WG document at the current time is premature.

On the contrary it shows that draft-morin is quite ready to become a WG
document ! I think his reply separates your valid concerns from the ones that are not
valid. And proves that the set of valid concerns is a) addressable and b)
can be addressed after WG adoption.

Let me repeat here what is the set of valid concerns that you seem to
have:

"Of the issues raised by you in the above the only legitimate ones
seems to be (a) more analysis on different flavor of multicast LSPs,
(b) whether there is a need for BGP-based auto-discovery with
PIM-based shared trees, and (c) whether the UDP-based S-PMSIs
signaling should be best considered as an optional part of the  PIM
C-multicast  routing control  plane.  All of these issues can be
addressed after the document gets adopted as a WG draft."

> (after all, they're  all FUD ;-)).

Yakov in his email asked you several specific questions. I don't
call that "supress discussion". If you have reasonable answers to have a
constructive discussion on those subjects, please speak up. Else it just
demonstrates that several of your points called out as FUD were indeed FUD
(or to put more politely philosophical) !

> real problem".

The discussion of the issues has to be grounded in *rational technical
arguments* put forth by SPs to guide the set of mandatory/optional
options.

> join latency that is important.

Try telling that to a SP who has an important financial feed that needs
*predictable* join latency.

>  Since packets don't usually get lost in the

There doesn't seem to be any technical argument on why the average join
latency will increase. The average has to take into account the worst case
latency, and the latency of first join and subsequent joins (which
is lesser than the latency of the first join).

> thereof  for  BSR  is  something  that a  "considerations"  document  should

To me this falls in the category of which multicast LSP flavor to choose.
And this can be done after the document becomes a WG document.

rahul

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Mon, 17 Dec 2007 12:17:23 +0000

On 14/12/07 16:43, "Thomas Morin" <[email-masked]> wrote:
> asked.

Fully agree, as co-author I welcome any comments on the draft and we can
work towards incorporating those.
 
> intent, structure and general direction of the document.

I also agree that the comments so far should not prevent the draft from
becoming a WG draft as I believe the draft to provide useful guidance to the
WG and the current solutions draft(s).

Ben

Sent using the Microsoft Entourage 2004 for Mac Test Drive.

---

From: Thomas Morin <[email-masked]>
Date: Mon, 17 Dec 2007 22:38:43 +0100

Hi Eric, working group,

I already extensively replied to Eric's comments, some of which seem to
me indeed as calling for relevant updates/additions to our draft, but I
want to highlight an answer to one specific comment...

Eric Rosen :
>
> 11. Outsourced RPs
>
>     I  would like  to see  the  draft offer  much stronger  support
>     for  its recommendation to  implement outsourced RPs,  or else to 
>     eliminate that recommendation.
[snip]
>     need to be given more emphasis: [...]
>

Everybody having read the draft can see that it actually not only say
that the RP-colocation feature is useful but also that an implementation
should _not_ require using this feature.  So, Eric, your underlying
suggestion that co-authors of mvpn-considerations state that "the
outsourced-RP feature should be implemented" to to simplify things for
some particular vendor(s) is really ridiculous.

If this is the kind of plot you have in mind, the amount of opposition
you have to mvpn-considerations becoming a working group document is in
fact not surprising !  ;)

But I think the working group support to this work shows that not
everybody has such conspiracy theories in mind, and that many people
understand that operators have just a sincere interest in trying to
identify a best candidate for becoming a standard (vs. a set of
"profiles").   If you happen to disagree with this perspective, then it
would be nice to hear you say that it is the main reason for you
opposition, that will spare a few photos and electrons of l3vpn mailing
list traffic !

Thanks,

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Mon, 17 Dec 2007 22:27:17 +0100

Hi Eric,

Following my reply on yesterday, I'll try to go in detail through your
comments and see what are the points that I think could be better
covered in draft-mvpn-considerations.

* First, some of your comments shows that there is much confusion
related to the theme of "what new is put into BGP vs. what PIM does /
what BGP does new that PIM was/wasn't doing". I admit I would have
expected a co-author of the solution drafts to actually clear up that
kind of confusion, but let's focus on what can be done in
mvpn-considerations to improve everybody's understanding : I propose we
add a section detailing what is done by PIM in the Full PIM peering
approach (both the case where explicit tracking is done by upstream PEs,
and the case where Join suppression is made) and what is done by BGP in
the BGP approach, on the PE and on the RRs. We won't try to compare
things quantitatively, but I think spelling out the details can help
shed light on the general debate of the relative merits of PIM and BGP
to carry C-multicast routing information.  (see below, in reply to your
"transactional nature of PIM", a rough cut of the things we can explain
to help evaluate how the multicast routing processing is spread onto the
different entities).

* On the general theme of BGP routing activity due to C-multicast route
and route reflectors : you are right in saying the document could talk
more about it. I think we can incorporate content explaining that moving
some processing from PIM to BGP will require adjusting the route
reflectors infrastructure by whether by increasing route reflector
processing resources and taking care of not badly impacting unicast
routes (e.g. distinct BGP sessions), or more likely by introducing
distinct route reflectors for mVPN.  We can also talk about the tools to
control the impact of end user activity to the amount of processing done
by the control plane (leave dampening, join dampening, impact of join
dampening on join latency).

* About join latency : for sure minimizing join latency can be important
for some specific applications and I agree we should talk about it. As
said earlier, I think the draft should use fact-based arguments and here
the facts that can be talked about is that the C-multicast routing join
latency relative performance in the BGP and PIM cases, will depend on
BGP performance of the PE and RR(s) relatively compared to PE PIM
processing performance on the PEs, and also on the network latency that
will be added at each processing "hop" (one in the case of PIM, one, two
or more in the case of BGP). All these performance aspects being to
consider in both the low scale space and the higher scale space, since
both PIM and BGP approaches can be expected to be differently impacted
by higher scale scenarios, for join-latency. Last, I think we should
state that absolute performance figure in the end depend on
implementations.

* On outsourcing the RP : the draft currently basically says that the
feature has applicability and should be implemented, but that in no way
it should be required to enable it (and we do mention drawbacks similar
to those you insisted upon).  This section needs some improvement : as
suggested by Yiqun Cai during the meeting, we don't exactly distinguish
between the two role associated with the "RP" label in such a context --
PIM register decapsulation and RP routing function vs.
Anycast-RP-related functions.  We plan to update this section in this
sense, then if you have precise comments on stuff that we
should/shouldn't talk about, please make them (see below inlined, I
didn't really understood what you disagree with in the current text).

* Remark about using BGP autodiscovery to detect misconfigurations, and
the fact it may not be helpful in transitions scenarios : I agree, and I
think we can definitely mention this in an updated version.

* On Extranet : the current text we propose is only saying that the BGP
approach for C-multicast routing reuses the RT import/export semantics
to define Extranet, to illustrate the benefits brought by using the same
protocol/semantics for unicast and multicast. You insist on reminding us
that an early implementation of the Full PIM peering approach do also
support this : if it also is a good illustration of reusing existing RT
import/export semantics, then we should also mention it for sure. The
fact is that Extranet for the Full PIM peering approach is not (yet?)
described in any spec (AFAIK).  So what I propose is that you first
point us toward a description of such a feature and (a) tell us if that
would be relevant to include in that section mentioning consistency
brought by the reuse of RT import/export semantics, and (b) we provide a
section trying to compare the relative merits of the different
approaches to provide an Extranet feature.

* On encapsulation techniques : as you suggest, we can highlight the
benefits of using MP2MP MPLS trees for facilitating the implementation
of some of the different proposed ways of supporting C-bidir-PIM. We can
certainly also explain how it would facilitate the support of the Full
PIM peering approach, but (see below inlined) I'm not sure I do yet
understand well what you suggest.  Another point (that you didn't
mention) is the expected scalability benefits of using MP2MP trees with
one MP2MP tree per VPN, we can also mention this.

* You highlight the fact that the scalability benefits brought by
aggregation of the traffic multiple VPNs in a PSMI-tunnel, requires the
support of upstream assigned labels : this is something that I'm not
against mentioning in the document, but I'm unsure how this will help
see which control plane approach is better than another one, since such
aggregation is supported whatever control plane approach you take...
(any idea ?)

* You make some comments related to S-PSMI scalability in inter-AS in
the segmented inter-AS model : there indeed were recent discussions
about among authors of draft-ietf-l3vpn-2547bis-mcast-bgp, which you and
me happen to be in, on possible improvements needed to make the
segmented approach better in this respect.  We will need to update
mvpn-considerations accordingly (this was in fact planned before your
comments, as mentioned in the slides presented in Vancouver). 

* BSR/auto-rp support : we can definitely tackle this point. It is true
that current specs for the BGP approach doesn't explain in detail how to
cover this.

Please tell us if you feel that the above proposed updates are relevant.
It seems to me that it integrates a fair amount of useful comments that
you made, and that can indeed make the document at the same time more
balanced and precise.

See inlined below, a few reactions I have on your comments.

>   conclusions as to the impact of enduser actions on BGP churn.

Yes, as said above, I propose to add content in the draft to talk about
new routing to be taken care of by BGP. It also has to be said, to be
fair, that (a) BGP updates can be caused by routing events in customer
sites and that BGP has tools to cope with such things (such as
dampening) which can be reused for C-multicast routes and that (b) the
"input" to this processing problem is the same whatever control plane
approach is chosen.

[...]
>   latency and churn are as yet poorly understood.

Well... do you deliberately chose to omit that what is first suggested
by the mvpn bgp solution draft (which you co-author!) is leave-dampening
and has no effect on join latency ? do you really honestly "poorly
understand" the effects of join-dampening on churn and latency ?

You are in fact spreading FUD through strawman here :
- "strawman" because nobody is saying "join dampening" should be
extensively be used, and "join dampening" is in no way a key argument in
the rest of the debate (_leave_ dampening is an important tool) 
- "FUD" because you use vague terms ("poorly understood") to suggest
that something is proposed without having well been thought out ; "join
dampening" is in fact not in this case. It has actually well identified
effects: good effect on "churn" (higher dampening => lower churn) and
impacts latency ("higher dampening" => "higher join latency for the
first receiver"). The solution drafts already formulate this limitation
of join dampening, and I don't think mvpn-consideration has to repeat
this.

So as formulated in the head of this mail: the key thing is "how control
plane copes with join/leave events ? what impact on latency ?", and we
can definitely talk about this.  But we will focus on constructive
points like looking at what tools are proposed for the different
approaches to cope/reduce this load. We can even go further and suggest
that, even though the Full PIM peering approach doesn't
document/proposes such features, join/leave dampening could also be
supported with slight changes to PIM implementations. 

>   i.e., that it is really  nothing new.  However, this isn't quite accurate.

(let me point your use of a "strawman" argument here again : when did
anyone suggest it was the same ? doesn the mvpn-considerations draft
state anything like that ?)

>   just how the applications are using multicast.

You are formulating things in a "PIM is transactional / BGP is not", but
I think that this formulation has two drawbacks:

- talking about the "transactional nature" of PIM, and "router to router
transactions" is in fact unprecise/bogus:
  * first, because on a LAN, PIM message are not send from "router to
router", but rather received AND processed by all PIM routers on the LAN
(they are sent to a multicast group address) -- this is true for Hellos,
Join, Leave, and assert messages -- in many case the processing made
will then be different depending on if the router is the upstream PE or
not for the said Join.
  * second, "translation" is a surprising term in fact : PIM messages
are not acknowledged in any way; they can be lost or unprocessed,
without the router at the origin of a message being aware ; that is, one
of the entity taking part into the "transaction" could be unaware that
the "transaction" didn't happen -- weird for a "transaction", right ?
(BGP being TCP-based, depending on what you mean by "transaction", you
could very much argue than BGP is in fact more "transactional"...)

- this formulation is really hiding one very important thing in how PIM
behaves on a LAN, to resolve the one question important for an upstream
router "is there still somebody needing such (C-s,C-g) across the
mvpn ?"

  * by default, PIM uses the "join suppression/prune override"mechanism,
    on a LAN : this mode help scaling PIM by reducing messaging, by two
    means:
      => when a router sends a Join for a said (s,g), other routers will
         omit sending theirs for some time, sparing each time a PIM Join
      => on reception of a Prune, the upstream router relies on other
         routers on the LAN to know if there are some PE left that need
         the steam 
     The downside of the above is that a PE router has to process all
   Joins sent on the LAN to know if it has to reset a timer or not for 
   Join suppression, and to all Prunes sent on the LAN to know whether
   or not it shall override the Prune.   This really not a "router to
   router" conversation.

[  * optionally, on a LAN, PIM can desactivate these mechanism, but only
  if all routers on a LAN are able to do so (detected through a Hello
  option), and obviously the benefits are lost : more messages will be
  exchanged, and more state has to be maintained (explicit tracking) by
  the upstream PE  // this approach is not pushed by anybody AFAIK  ]

- now, let's look at what BGP does (Eric, I know you know that
already!):

  * it finds a right trade-off between the two extremes possible with
PIM (between "everybody participates to see if there is any downstream
PE left" and "upstream PE does all the work by explicitly tracking who
are the downstream PEs") : the route reflectors are introduced, the set
of route reflectors carrying C-multicast routes for a said VPN virtually
collaborate to maintain the "count", the upstream PE is 100% relieved
from this processing/state effort, which burden is put on the route
reflector(s).  This means you have the tooling to adapt your
architecture to the work load : you can choose to use zero route
reflector (to minimize latency e.g. if you happen to have a specific use
case), or one route reflector, or a hierarchy of route reflectors... you
can choose to you partition the different VPNs to have them by treated
by specific route reflectors... you have _already_ existing tools to
handle the load in high scale scenarios.  None of this you can do with
PIM LAN procedures !

  * by nature (contrarily to PIM exchanges on a LAN by default) the
semantics of C-multicast route processing with the proposed BGP
extension are "router to router" : C-multicast routes are addressed to
one specific upstream router and BGP has the tooling required to respect
this semantic with RT-contraint (reused from unicast).

>   they are  right, I think that  this does have  to be recognized as  a risk

Interesting. You will certainly agree that there is not much we can say
based on so scarce information. Given the above, I would myself even
fear that the specifics of the use of PIM LAN procedures in the context
of draft-ietf-l3vpn-2547bis-mcast would have any side effect...

-> Why wasn't this suggested for incorporation in what is now RFC4834
(mvpn requirements) ?
-> Can you provide more data points and details on what are these
strange use of PIM ? (Or can (the) provider(s) having such use case give
more details ?)  It would be useful to let "the community" and
co-authors of mvpn-considerations evaluate what the limitations of the
BGP C-multicast routing proposal would be. 

>    MVPN customers who already use  multicast.  (Which I imagine would be the

I already mentioned this in the head of my answer: the draft precisely
says that an implementation should not require activating an RP function
on the PE.   

        "...support for a co-located RP model within an implementation
        should not restrict deployments to using a co-located RP
        model,..."
        
What do you exactly disagree with ??

>    shared tree is in use at some time.

I see that we agree here. We can add consideration about limitations of
misconfiguration detection.

>    invocation of the S-PMSI signaling  mechanism.

( Come on Eric, why do you chose a formulation that suggest a dynamic
activity triggered by end-users, while we are talking about an optional
optimisation that can be fully controlled/enabled/disabled by the
provider without impacting the service ?  )

> [...]The S-PMSI signaling is  also something that is

I disagree : the time between "the condition for triggering a Data-MDT"
triggering Data-MDT being true" and "traffic being put on the Data-MDT"
is something that currently already takes a fair among of time with some
implementations (many 10s of seconds).  And this only temporarily
reduces the bandwidth optimisation gain, and -again- with no impact on
the service provided to the user.

That said, comparing the signaling latency of S-PMSI with BGP and the
UDP protocol, is a bit like comparing apples and oranges (yes, Pink
Floyd's fans in the crowd, it is normal if you start hearing a little
singing voice) because BGP signaling ensures the delivery of this
information. The actual signaling minimal latency in both cases will
then depend partly on implementation, and on the number of processing
"hops" and on the network-related latency.

>    procedure unless and until I had some data about these pragmatic issues.

( You know that we do have recent facts about pragmatic issues related
to Data-MDT signaling with the UDP scheme, that would not have arised
with the BGP signaling approach. )

>    routing control protocol, in which case  you have an MI-PMSI.   [...]

Hmmm... I don't read any such thing in draft-ietf-l3vpn-2547bis-mcast.

As I see it, there are two proposed control plane approaches for S-PMSI
signaling. We are trying to focus the solution draft on only providing
one (which will help produce a "standard"), and we happen to see that
there is one of the two that require a specific type of data plane. 

>    protocol with its own independent set of pros and cons.

Hmmm... can you explain why the BGP signaling for S-PMSI would not apply
if PIM is used for C-multicast routing ? it seems to me that it is
totally applicable.

Said differently: why BGP-based autodiscovery of I-PMSI would be a good
fit for the Full PIM peering approach, and BGP-based S-PMSI signaling
would not?

>    BGP-based S-PMSI signaling  mechanism doesn't have whatever disadvantage

The thing is that the BGP segmented model provide means so that an AS
can trigger an S-PMSI without forcing neighbor ASes to create more state
in their backbone.  The drawback of not allowing this is I think fairly
clear : bandwidth optimization decisions made by one provider won't
impact the amount of state another provider will have to cope with.

But, you knew that already, so it really seems like a matter of wording.
We will try to improve this.

>    As the  text is now, it is  very difficult to understand  or evaluate the

See above (answer to your "Transactional nature of PIM" item).

> 6. MI-PMSI issues
> 
>    "Moreover, mechanisms one and two are restricted to use within
> MVPNs using an MI-PMSI, thereby necessitating [...] (b) The use of one
> P-multicast tree per PE per VPN, even for PEs that do not have sources
> in their directly attached sites for that VPN."
> 

Eric, you know very well that mvpn-considerations-01  (from which you
extract these statement) was submitted in early October, but
draft-rosen-l3vpn-mvpn-profiles-00, which is the first proposal of the
use of PIM across a set of M2PMP tunnels, was submitted one month later,
in early November...

So:
- we can make all the effort to be as balanced as possible, but we can't
read into the future !  ;)
- are we expected to cover all mvpn architecture proposal that are not
yet WG documents ?
- even, after reading -profiles section 3.3, it doesn't really appear to
me that this a use of the Full or Lightweight PIM peering approaches,
but rather a fifth proposal to adapt PIM to a specific dynamic mesh of
MP2MP trees 

>    to be  a "restriction" or disadvantage.

I would disagree : if one control plane option was restricting the
choice of the possible underlying encapsulation technique, that would
certainly be a restriction and thus a disadvantage.

But...rereading our text, I see that we don't say the right thing.
The full and lightweight PIM peering approaches do _not_ require a
provider protocol supporting shared trees, but _do_ require setting up
an MI-PMSI (and a set of trees rooted at each PE would thus obviously
allow to support PIM for C-multicast routing).  We will definitely
correct this text.

>    MI-PMSI even if BGP is used for distributing C-multicast routes.

On the other hand, when talking about higher scales, you have to look a
few years down the road. And these days, two things are preferred:
anycast-RP for ASM support, and SSM.  These don't require any
autoRP/BSR.  So, for me, we should ask ourselves : shall deployments or
services/applications to come (SSM) incur the penalty of past choices
(ASM related complexity) ?

>    PE/CE links, the SP is still operating a PIM environment.  

Well, it is true (and I'll propose we update this statement), but it is
also true that much of the complex things to operate with PIM relate to
PIM LAN procedures that are not necessarily used on PE-CE links (but
that are used when you use the Full PIM peering approach). Operation
people would need more expertise on PIM, to work with the Full PIM
peering approach, than when they only have to work on PIM on point to
point PE-CE links.

>    without the need for additional OAM processes and tools.

You have existing tools to observe/handle MP-BGP routes, extending them
to understand/troubleshoot BGP mVPN routing looks easier to me than
making tools to understand/troubleshoot multicast routing event with the
Full PIM peering approach.

>     not sure it's fully and properly specified.)

( As I said earlier the goal is to identify what can be MANDATORY
options to implement. Removing one option wouldn't help in this
respect... )

>     not sure if that is what is intended, and would like clarification.

I don't really see where your interpretation comes from. Please quote
specific text.   In my views, we aim in fact at quit the opposite : we
want to privileges an choice of control plane independent of the data
plane, to not be in a "profile" situation where control plane and data
plane engineering choices are coupled. 

>     their customers, the service provider should be requiring support for an

There is much confusion in the above statement, between aggregation of
traffic from different VPNs into a PMSI tunnel, and aggregation of
distinct (c-s,c-g) flows of a VPN into a same PSMI tunnel:
- aggregation of traffic from different VPNs into a PMSI tunnel does
require upstream assigned labels
- aggregation of distinct (c-s,c-g) flows of a VPN into a same PSMI
tunnel does _not_ require upstream assigned labels

So you can aggregate traffic of distinct (c-s,c-g) flows of a same VPN
into a same PSMI tunnel, but you _don't_ need it to be an MI-PMSI
tunnel. You can also do this with I-PMSIs, and you can do this with an
S-PMSI tunnel too.    Also, the limitation of requiring
upstream-assigned labels to aggregate traffic from distinct VPNs is not
solved by the use of MI-PMSI, this limitation is common to all PMSI
flavors.

( Really, I don't understand what you are trying to suggest... merely
creating confusion ? )

>     an issue which should be mentioned.

As said above, we plan to update mvpn-considerations to take into
account recent discussions/proposals about S-PMSI and inter-AS.  

But you make a rather interesting statement : "it is not clear that
segmented inter-AS S-PMSIs are scalable at all".  In fact the segmented
model merits are to introduce means to better scale all PMSI-related
state.  If the answer to the question was "no, segmented inter-AS
S-PMSIs are not scalable at all", then S-PMSI would not be scalable in
the non-segmented model either (where S-PMSI is by design per-PE!).

>    I think the draft should have a clear recommendation that in
>    the absence of upstream-assigned MPLS labels, MI-PMSIs should be
>    used.

???
That would be bogus :
* there is an applicability for scenarios where only I-PMSI are used
(where only trees toward PE having sources are instantiated)
* S-PMSI obviously have applicability and can be used

(Again, I apart from generating confusion, I really don't get your
point)

>     recommendation to  implement outsourced RPs,  or else to  eliminate that

Your already stated this in your item "1".
Maybe you wrote too many comments ?  ;-)

[...]
>     going to be justified by the  fact that it simplifies things for some of

Oh....?!??
You definitely wrote too many comments...
Despite all the respect I have for you, this is totally bogus.
- First, you imply that the four operators co-authors of mvpn-considerations 
are adding content in mvpn-considerations to simplify the work of specific 
vendor(s). You must be living in a funky world full of conspiracy theories.
- Second, you have missed one important thing : the draft actually state
that implementing only the "outsourced RP" model is not acceptable. I
don't how this is supposed to "simplify things for some [..] vendor(s)",
and doesn't really fit your conspiracy theory plot.

>    I certainly agree with this  conclusion.  Perhaps it should be given more

It is nice to see you agree. 
A few things though :
- this is not the only conclusion of the draft (this talks about about
C-multicast routing)
- this draft is a work in progress: "final" conclusions will be the one
in whatever final version is produced, taking into account feedback,
comments, and experience gained on all solutions

-Thomas

---

From: Eric Rosen <[email-masked]>
Date: Tue, 18 Dec 2007 10:22:12 -0500

Thomas  raises  a lot  of  issues,  which  deserve discussion  (probably  in
separate threads), and which will take a bit of time to respond to.

But there're two issues I want to respond to right away:

Eric> I would  like to  see the  draft offer much  stronger support  for its
Eric> recommendation to implement outsourced  RPs, or else to eliminate that
Eric> recommendation.

This is addressed at the text from section 4 of the draft:

        "It is therefore the recommendation of the authors
        that implementations should support a co-located RP model"

Thus it seems  to me that the draft recommends that  the co-located RP model
be implemented, and I think I am within my rights to oppose the inclusion of
that recommendation.

Thomas> Everybody having read the draft can see that it actually not only say
Thomas> that the RP-colocation feature is useful but also that an implementation
Thomas> should _not_ require using this feature

That's a  completely different issue, I  never said that  the draft requires
the co-located RP feature to be deployed.

Thomas> your  underlying suggestion  that co-authors  of mvpn-considerations
Thomas> state  that "the  outsourced-RP  feature should  be implemented"  to
Thomas> simplify things for some particular vendor(s)

I apologize if my words led you  to make that inference, but that was not my
intended  implication.   I  do  think  that  the  outsourced-RP  feature  is
primarily a  simplification for the  vendor, and really just  a complication
for the  SP and its customers.   However, my assumption is  that the authors
have  an  honest disagreement  with  me, not  that  they  have any  ulterior
motives.

Thomas> operators have just a sincere  interest in trying to identify a best
Thomas> candidate for becoming a standard (vs. a set of "profiles")

At no time did  I question the sincerity of the authors,  nor have I accused
anyone  of attempting to  spread FUD,  of telling  lies, or  having ulterior
motives.  I hope you will agree that ad hominem attacks of that sort have no
place on this ML.  I suppose  I have accused people of not fully considering
all the issues, but that does seem appropriate for this ML.

Thomas> If you happen to disagree with this perspective

I don't disagree about the sincerity  of the authors, but I don't think it's
either possible  or desirable at this  point to dispense with  the notion of
profiles.

---

From: Thomas Morin <[email-masked]>
Date: Tue, 18 Dec 2007 17:05:04 +0100

Hi Eric,

> be implemented, and I think I am within my rights to oppose the inclusion of

For sure and nobody contested this right.

I'm interested in understanding why you have such a strong feeling about
a statement that only recommends a relatively minor optional feature,
that actually happens to be implemented by numerous vendors in multicast
VRFs, including the one you work for.

The mVPN requirements (RFC4834) actually talk in terms of "MAY" about
this feature. Is it the difference between "MAY" and "should", that
makes you so strongly react ?

> for the  SP and its customers.

To fully clarify what caused this misunderstanding, can you explain how
implementing this optional feature is "primarily a simplification for
the vendor" ?

> However, my assumption is  that the authors have  an  honest
> disagreement  with  me, not  that  they  have any  ulterior
> motives.

Nice to see you clarify this point.

> place on this ML.

Ad hominem attacks indeed are not appropriate. 

You will hopefully agree too, that critics on the statements made (and
not on the people who make them) or the form of reasoning used (and not
on the people who use them), certainly do have a place here. 

You can also agree that it is quite natural to wonder about the
motivations when trying to better understand the issue raised.

-Thomas

---

From: Eric Rosen <[email-masked]>
Date: Tue, 18 Dec 2007 11:46:09 -0500

> a statement that only recommends a relatively minor optional feature,

I am uncomfortable with it for a number of reasons:

- If an SP  wants to offer "outsourced RP" service  to its customers, that's
  fine.  That does  not necessarily require that the PEs  serve as RPs.  The
  "PE  as RP"  model  is a  particular  way of  deploying  an outsourced  RP
  service.   How best  to deploy  an outsourced  RP service  is a  topic for
  discussion between  an SP and  its vendors, and not  necessarily something
  that the IETF even needs to be involved in.

- I  cannot believe  that "PE  as RP"  is really  going to  be  the superior
  "outsourced RP" deployment model at scale.  

- The draft  gives the  appearance of recommending  (even if  not requiring)
  both the "outsourced RP" model and this particular method of deploying it.

- The  model itself  has what  I consider  to be  significant disadvantages,
  especially if  you try to deploy it  with a customer that  has an existing
  multicast infrastructure.

- I  don't think  the impression  you get  from the  draft is  that it  is a
  "relatively minor optional feature".   That sort of minor optional feature
  would typically get a "may" rather than a "should".

> makes you so strongly react ?

No objection to MAY.

> can you  explain how  implementing this optional  feature is  "primarily a
> simplification for the vendor" ?

It allows a vendor to ship product which supports only the "PE as RP" model,
while  still claiming  to have  ASM support  in its  MVPN product.   This is
considerably less  of an  implementation effort than  is required  to really
provide ASM support.

---

From: Thomas Morin <[email-masked]>
Date: Tue, 18 Dec 2007 17:18:33 +0000

>   both the "outsourced RP" model and this particular method of deploying it.

The draft actually has 5 short paragraphs about co-located-RP: 
- 2 paragraphs are quotes of the requirements draft reminding the
context and the benefits of the feature
- 2 paragraphs detail the disadvantages of this approach
- the last paragraphs concludes by saying "should be implemented" but
"must not be required to use"

It seems to me that this is quite balanced...

>   would typically get a "may" rather than a "should".

The use of "should" in the text, and not "SHOULD" precisely shows that
the "level of requirement" wasn't strongly defined, this is typically
the type of wording that gets refined once a document progresses
further...

> > can you  explain how  implementing this optional feature is 
> > "primarily a simplification for the vendor" ?
> 
> It allows a vendor to ship product which supports only the "PE as RP"
> model, while  still claiming  to have  ASM support  in its  MVPN
> product. This is considerably less  of an  implementation effort than
> is required  to really provide ASM support.

It doesn't simplify anything as soon as it is required to support
deployment of ASM without using this feature. Which precisely is what
our draft is suggesting to impose. 

> > The mVPN requirements (RFC4834) actually talk in terms of "MAY"
> > about this feature. Is it the difference between "MAY" and "should",
> > that makes you so strongly react ?
> 
> No objection to MAY.

(Which I wouldn't object either.)

So, in the end, this colocation-RP subject that takes 10% of your
comments seem to be resolved by a "should" being refined into a "MAY". 
Do such type of changes usually justify opposing that a draft becomes a
WG doc ?

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Tue, 18 Dec 2007 18:24:50 +0100

Eric Rosen :
> Thomas  raises  a lot  of  issues,

Replying to your numerous comments does indeed take some time ! ;)

> which  deserve discussion  (probably  in

It would be great if you can tell the working group if the updates I
propose in the head of my reply (not that long) are enough to address
your concerns.

Thank you,

-Thomas

---

From: Yakov Rekhter <[email-masked]>
Date: Tue, 18 Dec 2007 11:50:14 -0800

Eric,

>   that the IETF even needs to be involved in.

For the same reason the IETF should not preclude the ability to co-located 
RP and PE.

>   "outsourced RP" deployment model at scale.  

"I cannot believe" is not a technical argument.

Yakov.

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Thu, 27 Dec 2007 05:50:34 +0000

Eric,

> like to delay adopting it as a WG document.

Do Thomas's responses thus far (which I support & endorse) related to which
of your comments we can start to address & incorporate into draft-morin
satisfy your concerns above?

Ben