Subject: Re: [Idr] Request WG adoption call for draft-kaliraj-idr-bgp-classful-transport-planes

From: <[email-masked]>
Date: Wed, 2 Mar 2022 16:22:33 +0800 (CST)

Hi Kaliraj,

Appologize for replying so late.
Please see inline [PSF2] .

(Due to the mail delivery issues, I had to choose plain text format. To avoid confusion, some irrelevant contents have been removed)

Regards,
PSF

------------------原始邮件------------------

Hi Peng Shaofu,

I read draft-zhou-idr-inter-domain-lcu, and understand what you are proposing. This proposal is an informational draft, with no extension to LU encoding, uses Addpath, and just resolves over tunnels with color.

This is the first thing we considered, to avoid any protocol extensions.

But the following problems appear with this model:

1/ resolving service-routes with color, over LU routes with color – is not efficient. Because we cannot do LPM (longest-prefix-match) on the nexthop address to find the matching route. Because the route with  desired color community may exist anywhere in the  route list.

Unless, user configures mechanisms to match on the color community and leak LU routes of a color to a ‘CT like’ transport-RIB database which contains prefixes of that color. This is what CT does, and  LU plugs into the CT architecture in this manner.

[PSF] Your guess is right. There is exactly a 'CT like' RIB, the FEC KEY is <color, prefix> for BGP-LU color LSP. And, the resolution is indeed a local implementation, for example, a tunnel-selection module can support the priority of resolved tunnel types,  e.g, exact matching SR policy > BGP-LU color > Flex-algo > ... > best-effort routes.

KV> Do you mean LU received routes are installed in the LU RIB, and in addition to a new RIB, where FEC key is <color, prefix>? How do best-effort tunnels get represented here, as they don’t have any color?

[PSF2] Yes. The best-effort tunnels, if I understand your meanning under our discussion context, are actually represented as traditional LU entries without color.

KV> How does fallback between two colors work? and fallback to best-effort tunnel work? Does these fallback mechanism not use longest-prefix-match (LPM) lookup for resolution, like regular BGP procedures https://www.rfc-editor.org/rfc/rfc4271#section-9.1.2.1 ?

[PSF2] A tunnel selection module can process this, by a defulat policy or specific policy per instance to rank the priority order of multiple tunnel types, including fallback procedure. RFC4271 section 9.1.2.1 provides a basic resolvability rules, however, it is supplemented in many specific implementations as I know, considering the resolution to tunnels case rather than just IGP reachability.

KV> Pls note that, whatever resolution mechanism you follow needs to be followed by all LU nodes on the path, to satisfy the SLA constraint end-to-end. But there is no signaling/capability-negotiation
KV> of any sort to know whether all nodes on the path support your new mechanisms. So ingress-PE cannot be really sure if it has established a path that satisfies the desired SLA end-to-end.

[PSF2] Agree. So draft-zhou-idr-inter-domain-lcu is submited as a possible simple method for tihs purpose. You found a point about capability negociation that we have discussed but missed from the document. This seems also applicable to BGP-CT.

2/ like you note, this proposal requires addpath for ebgp, and also add-path for non-bgp routes. Which may not be supported by all implementations today. These may not be a show-stopper, but a thing to consider.

[PSF] Only local changes are made to the original conditions for sending multipath. It's easier than defining a new set of standards. And, due to the wide deployment of reflectors, add-path is a basic function, right ? In fact, both BGP-CT and BGP-CAR mention  that they rely on add-path.

KV> do u mean ‘sending addpath’? yes, CT uses addpath at RRs, which is well deployed. But like Jeff noted, ebgp-addpath is not well deployed.

[PSF2] Agree. So we suggest a low-cost function, ebgp-addpath, for the same target.

3/ Add-path ID is good at not hiding paths at path-selection pinch points like RR, but not as an end-to-end generic distinguisher. It may make troubleshooting very hard. More so, when the color-communities  are rewritten across domain borders to accommodate  independently administered community namespaces.

[PSF] Add-path ID is not used as end-to-end distinguisher, but Color. According to SR policy color, we think that color SHOULD be global, considering that egress PE advertised a service route containing color EC to ingress PE and ingress PE must recognize it.  However, if that is not the case, LCM defined in BGP-CAR is a possible method to address that.

KV> I don’t understand this part. You said your approach uses plain LU encoding. Then how can you use Color as distinguisher?

[PSF2] Sorry that the distinguisher you said is route distinguisher but I'm using the distinguisher of the LUC-RIB entry KEY to explain. Yes, ADD-PATH ID is used as BGP route distinguisher without necessary to be end-to-end changeless, and Color is used as LUC-RIB entry KEY also without to be end-to-end changeless.

4/ performing multipath or protection for the ‘LU-routes with a certain color’ is not possible, because all routes belong to the same LU-prefix’s route-list.

Unless again, you have user configured policies to segregate LU routes with a color into separate CT transport RIBs.

[PSF] Your guess is right. There is no problem to create ECMP/FRR items for separate BGP-LU color FEC.

KV> Do you mean the MPLS swap routes derive their nexthop from the <color, ip-prefix> Table?

[PSF2] Yes. The <color, ip-prefix> table entry has incomming label attribute, so that another derived ILM entry can be produced, and these two entries share the same forwarding information

So, considering such issues that came out when we started analyzing it, and considering the user-configuration required
across different vendor implementations, to make things work, we decided it is best to establish well-known protocol procedures,
to achieve ‘nexthop resolution over tunnel routes with color’. And we re-used existing 4364 machinery at the transport-layer to
achieve that, which is well understood by network operators today.

[PSF] One concern about BGP-CT is that, it needs maintain VRF instance (although special instance) at any border nodes, that may be a retrogression to option C.

KV> If you think about it, in your approach, you are doing the similar things like VRF but using special procedures.

[PSF2] Although both approaches need extra work, but still quite different.

KV> viz. installing route in a new RIB beside LU RIB,  based on Color community. Forming multipath, FRR, label-allocation based on entries in that RIB

[PSF2] Yes. It is indeed a new FEC type of MPLS LSP, and for implementation it may be merged with SR-MPLS policy FEC. 

KV> Specifying to use RFC-4364 procedures at Transport layer just standardizes the procedures without any ambiguity. Also we get benefit of RTC.
KV> Please note that in CT, whether you achieve the Transport RIB construct using a VRF or not is up to the implementation. The Transport-RIB is just a logical subset of routes which belong to the same color.

[PSF2] Agree,  I always thought it can work.

Other protocol-extension proposals that carry color in the NLRI (BGP SRTE, BGP LCU, BGP CAR) have the basic problem that neither IP:Color or Color:IP encoding can satisfy all the following requirements:

Supporting all the following usecases simultaneously is not possible:
LPM fallback between tunnels with a color, and best-effort tunnels (that don’t have a color) . This requires IP:Color format
LPM fallback between tunnels with a color, and color-only (SRTE) tunnels.                                   This requires Color:IP format
Carry non-host IP tunnel-prefixes  with Color in the NLRI.                                                                  This requires Color:IP format
Carrying best-effort tunnels (IP-prefix only) in the NLRI.                                                                           This requires IP:Color format

[PSF] Not sure that I have understand these points. It is a local implementation for underlay tunnel resolution and table orgnization, I don't think there is a necessary connection between table orgnization and message format.

KV> Actually, the last two points are related to NLRI message format only, not local implementation.
KV> The first two are local implementation choice indeed, but they affect what kind of resolution and fallback are possible using LPM lookup.

[PSF2] I have no further understanding about these four points, sorry for that. IMO, tunnel selection procedure can visit multiple tables each with different entry KEY, such as LU, LU color, LDP, RSVP-TE, SR policy, etc, Each table origininator has its own message format and has nothing to do with each other. There is no need to adopt a unified message format. For example, for our approach, it is just a member of the mentioned table originators set. 
[PSF2] For the above 3rd point, such format requirement seems not necessary, do <ip, color> and <color, ip> necessarily imply significant processing differences? I am in doubt. 
[PSF2] For the above 4th point, we can advertise traditional LU, which is also just a member of the mentioned table originators set.

Not carrying color in the NLRI enables CT to achieve all the above. It is more inline with nexthop resolution without color that happens in deployed BGP networks today.

IOW, CT is a scale-out version of BGP nexthop-resolution that is deployed today. So there are less surprises and nexthop resolution process scales better, because it is confined to routes with  a certain color.

There are other aspects of CT vs CAR that has been discussed in the Interim meeting. Please go thru the link Jeff provided.

[PSF] Seems that we have not reach consensus according to minutes. IMO, each solution can work, but, still have question: why ? What is the pain point scene that  be covered by one solution and not by other ?

KV> Above we discussed few points, about CAR NLRI encoding (IP:color) not able to support non-host prefix. RTC like mechanisms not applying for CAR. Etc.
KV> The different aspects of CT vs CAR are being discussed in the interim and ongoing email threads with questions from Chairs. These discussions will eventually lead to a conclusion. Thanks.

[PSF2] Thanks for the discussions, will continue to pay attention to it.

---

From: <[email-masked]>
Date: Thu, 3 Mar 2022 15:18:39 +0800 (CST)

Hi Jeff,

Appologize for replying so late. Thanks you for sharing the document.
If I understand correctly, this document introduce additional attributes for route decision procedure for the case that eBGP-learnt path continues to be advertised to iBGP with NEXT-HOP unchanged. I think such scenes exist, for example, service route advertisement with option-B. But for the transport route advertisement we are now discussing, NEXT-HOP should be changed, so that the existing route decision procedure is OK for all approaches.

Regards,
PSF

------------------原始邮件------------------
发件人：JeffreyHaas
收件人：彭少富10053815;
抄送人：Kaliraj Vairavakkalai;idr-chairs;[email-masked];[email-masked];
日 期 ：2022年02月19日 00:16
主 题 ：Re: [Idr] Request WG adoption call for draft-kaliraj-idr-bgp-classful-transport-planes
Peng Shaofu,
A detail important for all proposals of ebgp add-paths is that it's not a fully supported scenario.
The technical issues surrounding that feature is documented in this older internet-draft.  The mechanisms in this I-D are not implemented by any vendor that I'm aware of.
https://datatracker.ietf.org/doc/html/draft-pmohapat-idr-fast-conn-restore-03
-- Jeff