Subject: Shepherd review of draft-ietf-rtgwg-bgp-pic

From: Yingzhen Qu <[email-masked]>
Date: Fri, 3 Jan 2020 20:00:12 +0000

Hi authors,

Happy New Year!

I did a review of draft-ietf-rtgwg-bgp-pic-10 for shepherd write-up. Thanks for working on this informational document, and it’s very useful to improve routing convergence.

I have the following comments and would like you to consider.

General:

  *   Throughout the document, both BGP PIC and BGP-PIC are used. I’m ok with either one, please keep it consistent.
  *   Regarding references, idnits is giving the following warnings:

  == Outdated reference: draft-ietf-spring-segment-routing-mpls has been
     published as RFC 8660

  == Outdated reference: A later version (-05) exists of
     draft-bashandy-rtgwg-segment-routing-ti-lfa-02

  *   There are links to references in the document are broken/not working, please go through and fix them.
  *   Other idnits warnings:

  == The copyright year in the IETF Trust and authors Copyright Line does not
     match the current year

  == The document seems to contain a disclaimer for pre-RFC5378 work, but was
     first submitted on or after 10 November 2008.  The disclaimer is usually
     necessary only for documents that revise or obsolete older RFCs, and that
     take significant amounts of text from those RFCs.  If you can contact all
     authors of the source material and they are willing to grant the BCP78
     rights to the IETF Trust, you can and should remove the disclaimer.
     Otherwise, the disclaimer is needed and you can ignore this comment.
     (See the Legal Provisions document at
     https://trustee.ietf.org/license-info for more information.)

  *   Section 2.1.2: some clarification needed here. When the primary next-hop fails, my understanding is that BGP PIC will first use other primary next-hops if available, e.g ECMP before using the pre-computed backup paths. Also “The existence of a secondary next-hop is clear for the following reason:”, this needs some explanations, and this is different from for example pre-computed backup paths using IP FRR.
  *   Section 7 title is “Properties”, and it seems to me this section is more like a summary. I’d suggest combining section 7 and 10, then change the title to summary or something. No strong opinion on this one though.
  *   Throughout the document, lots of paragraphs are missing the ending “.”

Nits:

  *   The following are editorial nits, please consider fixing them. I’m using the line number from idnits.

136        techniques, multiple techniques have been proposed to allow for
137        BGP to advertise more than one path for a given prefix

I’m not sure it should be “allow” or “allow for”.

169        o  Ingress PE, "iPE": A BGP speaker that learns about a prefix
170           through a IBGP peer and chooses an egress PE as the next-hop for
171           the prefix.

Should be “an iBGP peer”. Also this definition is not clear to me. I’d also suggestion add one for “ePE”.

239        o  A shared hierarchical forwarding Chain: It is not uncommon to see

Should be “chain”.

270        This section describes the required functionality in the forwarding
271        and control planes to support BGP-PIC described in this document

“functionalities”, also missing ending “.”.

334        VPN-IP2, respectively. Suppose that BGP-NH1 and BGP-NH2 are resolved
335        via the IGP prefixes IGP-IP1 and IGP-P2, where each happen to have 2
336        ECMP paths with IGP-NH1 and IGP-NH2 reachable via the interfaces I1
337        and I2, respectively. Suppose that local labels (whether LDP [4] or
338        segment routing [13]) on the downstream LSRs for IGP-IP1 are IGP-L11
339        and IGP-L12 while for IGP-P2 are IGP-L21 and IGP-L22. As such, the
340        routing table at iPE is as follows:

I think you meant “IGP-IP2”, instead of “IGP-P2”.

Thanks,

Yingzhen

---

From: Gyan Mishra <[email-masked]>
Date: Sat, 4 Jan 2020 13:21:32 -0500

Hi Yingchen & Authors

Since BGP PIC is an BGP protocol standard update should it also be
referenced and update BGP RFC 4721.

Also since this is a major feature improvement to BGP should this be
“Standards track” and not “Informational”

I noticed in the document there is a mention in the beginning of the draft
regarding “best-external” but the detailed use of that knob is not
documented as part of BGP PIC edge functionality.  When all edge paths are
equal attributes and not prepended or costed an eBGP backup path is not
flagged as “best-external”.  If PE edge backup paths exist that are
prepended then with “best-external” knob enabled the path is flagged as a
backup path and is advertised to the route reflector.

I noticed that the router reflector critical functions of selecting all
paths and advertising additional paths is not mentioned which is critical
for BGP PIC operation.  One aspect of the route reflector function which is
critical is that if the route reflector only advertises the “2 best paths”
the route reflector is performing the best path selection lowest IGP metric
tie breaker based on its lowest IGP cost to PE exit point.  This however
breaks use case with many PEs edges at different locations advertising the
same NLRI and being able to use the local ibgp as best path now since the
route reflector is making the best path decision based it’s closed
proximity PE exit point.  Workaround to this issue is to have the route
reflector advertise all paths to the PEs route-reflector-clients so that
now the PEs can make the lowest IGP metric best path calculation decision
and pick its local PE as the best path during  a failure.

Another critical feature to mention in the draft related to route reflector
and BGP PIC edge functionality is that for L3 VPN vpnv4 and vpnv6 the route
reflector is not able to advertise all paths as with IPv4 and IPv6.
Workaround to this issue is that the RD must be unique for each PE within
the VPN so that the route reflector now reflects and advertises all paths
to all PEs based on unique RD.  I tested this workaround and it worked and
allows BGP PIC to properly chose the local to run the BGP best path
calculation and  pick the closest proximity lowest IGP metric PE exit point.

Another critical feature for BGP pic edge for L3 VPN is that  when you
enable pic edge to install backup path you only have to do it globally and
all the customer VRFs inherit BGP PIC edge.  This maybe vendor specific and
in my case was with Cisco XR code.

In section 2 of the document where it talks about hierarchical FIB can we
mention that is related to PIC core and that PIC edge is when PE edge
installs backup path.  That would help tie the PIC function to the core and
edge PIC feature.

Also for PIC core it should be mentioned that newer hardware has or should
have  pic core enabled by default.

Also mention that PIC edge can be used independently of PIC core however in
most cases PIC core if the platform supports is enabled by default.  PIC
edge can be configured even if PIC core is not supported on the hardware
platform and would still improve convergence.

We should also I think mention NHT next hop tracking FIB watcher and
basically part of using the hierarchal fib is similar to NHT feature
tracking the next hop PE exit points that correlate to all BGP NLRI from
the exit point.  For hardware platforms that don’t support hierarchical fib
that NHT can be used as a workaround to achieve the same convergence.

Also note that for PIC Core to really be beneficial that the IGP timers
must be optimized as well as use of IP LFA, R-LFA for ldp and BFD should be
configured to optimize next hop convergence.

Should also be noted that if exit point PEs edge uses  BGP next-hop-self ;
next hop rewrite to loopback ; the loopback never goes down even if PE-CE
failure occurs and that can impact overall convergence and this issue
exists for both MPLS and  IP core scenario.
I am trying to develop a workaround for this issue but have not found a
solution yet.

Kind regards,

Gyan

On Fri, Jan 3, 2020 at 3:01 PM Yingzhen Qu <[email-masked]>
wrote:

> Hi authors,
>
>
>
> Happy New Year!
>
>
>
> I did a review of draft-ietf-rtgwg-bgp-pic-10 for shepherd write-up.
> Thanks for working on this informational document, and it’s very useful to
> improve routing convergence.
>
>
>
>    - Throughout the document, both BGP PIC and BGP-PIC are used. I’m ok
>    with either one, please keep it consistent.
>    - Regarding references, idnits is giving the following warnings:
>
>
>
>    - There are links to references in the document are broken/not
>    working, please go through and fix them.
>    - Other idnits warnings:
>
>
>
>    - Section 2.1.2: some clarification needed here. When the primary
>    next-hop fails, my understanding is that BGP PIC will first use other
>    primary next-hops if available, e.g ECMP before using the pre-computed
>    backup paths. Also “The existence of a secondary next-hop is clear for the
>    following reason:”, this needs some explanations, and this is different
>    from for example pre-computed backup paths using IP FRR.
>    - Section 7 title is “Properties”, and it seems to me this section is
>    more like a summary. I’d suggest combining section 7 and 10, then change
>    the title to summary or something. No strong opinion on this one though.
>    - Throughout the document, lots of paragraphs are missing the ending
>    “.”
>
>
>
> Nits:
>
>    - The following are editorial nits, please consider fixing them. I’m
>    using the line number from idnits.
>
>
>
> _______________________________________________
> rtgwg mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtgwg
>
-- 

Gyan S. Mishra

IT Network Engineering & Technology

Verizon Communications Inc. (VZ)

13101 Columbia Pike FDC1 3rd Floor

Silver Spring, MD 20904

United States

Phone: 301 502-1347

Email: [email-masked]

www.linkedin.com/in/networking-technologies-consultant

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 4 Jan 2020 20:06:01 +0100

Gyan,

> documented as part of BGP PIC edge functionality.
>

It does not need to be documented. Section 2.1.2 just makes is clear that
to use BGP PIC you need more paths. How you organize your BGP such that
multiple paths are present is beyond the scope of this draft.

> for BGP PIC operation.

Use of route reflection is just a way to distribute your BGP paths. It is
not a necessity. Moreover RRs if deployed well do not select bgp paths
based on their view of IGP metric. Hint: Pls look at Optimal Route
Reflection draft.

> Another critical feature to mention in the draft related to route
> reflector and BGP PIC edge functionality is that for L3 VPN vpnv4 and vpnv6
> the route reflector is not able to advertise all paths as with IPv4 and
> IPv6..  Workaround to this issue is that the RD must be unique for each PE
> within the VPN
>

Unique RD per VRF is not a workaround. It is best practice for years now.
Number of implementations allocate RDs by default (auto-rd) per VRF. Other
deployments do it properly via NMS.

> Also mention that PIC edge can be used independently of PIC core however
> in most cases PIC core if the platform supports is enabled by default.  PIC

That would be a bit stating the obvious :)

> that NHT can be used as a workaround to achieve the same convergence.

NHT and hierarchical FIB are completely orthogonal features. First is
responsible to detect the failure via watching RIB updates (registers next
hops and get's callback when IGP deletes or modifies it) while latter is
part of the repair mechanism. There is nothing "similar" between those two
features.

> configured to optimize next hop convergence.

Sorry - no. PIC can work fine without IGP too just using BGP recursive next
hops. Use of any flavour of LFA helps to quickly restore connectivity to a
given next hop - it is completely orthogonal to BGP PIC feature.

> I am trying to develop a workaround for this issue but have not found a

We solved this problem over 11 years ago by defining new Edge_Discriminator
attribute. However the draft was not progressed due to lack of real
interest and easy workaround of not setting next hop self.

https://tools.ietf.org/html/draft-pmohapat-idr-fast-conn-restore-03

Thx,
Robert.

---

From: Gyan Mishra <[email-masked]>
Date: Sat, 4 Jan 2020 15:14:53 -0500

> multiple paths are present is beyond the scope of this draft.

   I was referring to the best-external flag operation and how that feature
works as far as advertisements of backup paths. Not necessarily the
organization of multiple paths.

> based on their view of IGP metric. Hint: Pls look at Optimal Route

   Glad to see the draft and I agree keeping the state centralized is a
better design strategy then pushing state to the edge.  Also lends itself
to a centralized controller based model.

https://tools.ietf.org/html/draft-ietf-idr-bgp-optimal-route-reflection-19#page-6

> deployments do it properly via NMS.
    I agree RD auto  has been a best practice for years.  I was just
pointing out that if RD auto is not employed that the issue will present
itself.  Something to be cognizant of which is why I mentioned.

> That would be a bit stating the obvious :)

   Understood

> part of the repair mechanism. There is nothing "similar" between those two

  Understood

> Sorry - no. PIC can work fine without IGP too just using BGP recursive
> next hops. Use of any flavour of LFA helps to quickly restore connectivity
> to a given next hop - it is completely orthogonal to BGP PIC feature.
>

   Understood.  I was just stating for “optimal” convergence recommendation
which I thought was relevant since using BGP PIC is an “optimization” to
that end in achieving as close to hitless convergence.

>
>
>> Should also be noted that if exit point PEs edge uses  BGP next-hop-self
>> ; next hop rewrite to loopback ; the loopback never goes down even if PE-CE
> We solved this problem over 11 years ago by defining new
> Edge_Discriminator attribute. However the draft was not progressed due to
> lack of real interest and easy workaround of not setting next hop self.
>

   I found that draft when I was researching and wondered why it didn’t
progress.  I do think it has a lot of merit.  Major point being that if you
don’t do the next hop self and advertise you external links now imagine
with millions of peers your LFIB FEC binding state becomes unmanageable.  I
am all for resurrection of the draft and I would support.

Kind regards,

Gyan

-- 

Gyan S. Mishra

Silver Spring, MD 20904

United States

Phone: 301 502-1347

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 4 Jan 2020 21:46:25 +0100

Hi Gyan,

> organization of multiple paths.

Best external is covered already in
https://tools.ietf.org/html/draft-ietf-idr-best-external-05 Honestly I am
not sure what "flag" are you referring to.

>> https://tools.ietf.org/html/draft-ietf-idr-bgp-optimal-route-reflection-19#page-6
I think it would be good not to mix two things. Optimal reflection chooses
N optimal paths for given client and group of clients. Those for PIC to
work still need to be distributed to the edges in any network which uses
encapsulation.

For networks using IP lookup based routing BGP PIC can be used on ABRs
possibly also acting as physical or virtual RRs.

> itself.  Something to be cognizant of which is why I mentioned.

Great that we seems to agree on RD best practice. Again for this draft ie
BGP PIC to be able to work redundant paths must be present.

/* We considered just sending multiple next hops for non VPN use cases
effectively also allowing to construct bundle rewrites with BGP PIC, but
then add-paths came as "the solution" which overruled alternatives :) */

>    Understood.  I was just stating for “optimal” convergence
> recommendation which I thought was relevant since using BGP PIC is an
> “optimization” to that end in achieving as close to hitless convergence.
>

Your point is sound and indeed many networks (or vendors) today do not even
employ basic IGP or BGP PIC. In fact many folks still care about speed of
IGP or BGP *convergence* instead of focusing on fast connectivity
restoration objective leaving protocols in peace allowing them to converge
at their own pace.

But I don't think that current draft in question is the best place for such
education :)

> am all for resurrection of the draft and I would support.

Well I do not know any network with millions of peers :) But I am always
open to learn !

Said this I do still see some value in moving fwd with this draft. If other
think this is useful we could restart this as a separate IDR/GROW thread.

Thx,
R.

---

From: Gyan Mishra <[email-masked]>
Date: Sat, 4 Jan 2020 16:26:18 -0500

On Sat, Jan 4, 2020 at 3:46 PM Robert Raszuk <[email-masked]> wrote:

> Hi Gyan,
>
>    I was referring to the best-external flag operation and how that
>> feature works as far as advertisements of backup paths. Not necessarily the

    I’m good as it’s already documented in another draft.  Maybe good to
add as reference.

> possibly also acting as physical or virtual RRs.

    Thanks for clarification

> then add-paths came as "the solution" which overruled alternatives :) */

   Makes sense

> Your point is sound and indeed many networks (or vendors) today do not
> even employ basic IGP or BGP PIC. In fact many folks still care about speed
> of IGP or BGP *convergence* instead of focusing on fast connectivity
> But I don't think that current draft in question is the best place for
> such education :)
>

   Agreed

> Said this I do still see some value in moving fwd with this draft. If
> other think this is useful we could restart this as a separate IDR/GROW
> thread.
>

    Let me know I would be happy to collaborate with you on the restart.

Gyan S. Mishra

Silver Spring, MD 20904

United States

Phone: 301 502-1347

---

From: Ahmed Bashandy <[email-masked]>
Date: Tue, 7 Jan 2020 06:14:23 +0200

Thanks a lot for the review and the detailed comments

I will work on addressing them

Ahmed