Subject: Re: [Idr] Request WG adoption call for draft-kaliraj-idr-bgp-classful-transport-planes

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Sun, 13 Feb 2022 04:58:54 +0000

+ IDR WG alias.

Thanks
Kaliraj

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Saturday, February 12, 2022 at 7:43 PM
To: [email-masked] <[email-masked]>
Cc: [email-masked] <[email-masked]>
Subject: Request WG adoption call for draft-kaliraj-idr-bgp-classful-transport-planes
Hi IDR Chairs,

On behalf of authors of draft-kaliraj-idr-bgp-classful-transport-planes, I would like to request for adoption call of this draft.

I think we have waited long enough for the CT vs CAR proposals to be discussed in the WG, including the IDR Interim meeting on 24 Jan. Thanks to the Chairs for organizing this interim meeting.

We believe these discussions would have given WG members enough insight into each proposal.

We can continue to have further discussions as required, on the list or in another interim meeting, towards the end result of an adoption call.

Thus providing closure on this matter, and helping customers who are waiting to deploy the solution to solve real world problems.

Thanks,
Kaliraj

Juniper Business Use Only

Juniper Business Use Only

Juniper Business Use Only

---

From: <[email-masked]>
Date: Mon, 14 Feb 2022 15:50:17 +0800 (CST)

Hi Kaliraj,

Thank you for your information. 
I missed a lot of discussions, is there a minutes link about the IDR Interim meeting ? I want to get the comparison conclusion of various schemes, which is benefit and which is not enough, at present, that is not very clear to me, maybe others.
I also draw your attention to https://datatracker.ietf.org/doc/draft-zhou-idr-inter-domain-lcu/, a simple solution based on existing protocol. IMO, the extension of the existing mechanism needs to answer the following questions:
    1) why need extensions ?
    2) what is the additional benefit of the extensions ?

Regards,
PSF

------------------原始邮件------------------
发件人：KalirajVairavakkalai
收件人：[email-masked];
抄送人：[email-masked];[email-masked];
日 期 ：2022年02月13日 12:59
主 题 ：Re: [Idr] Request WG adoption call for draft-kaliraj-idr-bgp-classful-transport-planes
_______________________________________________
Idr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/idr

+ IDR WG alias.

Thanks
Kaliraj

Thanks,
Kaliraj

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 14 Feb 2022 16:18:24 -0500

Peng Shaofu,

You may find the minutes, and a link to the video for the most recent
interim at the following link:

https://datatracker.ietf.org/meeting/interim-2022-idr-02/session/idr

My recommendation is to view the minutes at the notes site.  There seems to
be an issue with the markdown rendering on the official import.

https://notes.ietf.org/notes-ietf-interim-2022-idr-02-idr#

Please also note there was an earlier LCU proposal that was presented at
IETF prior to BGP-CT and BGP-CAR.  (I gave the presentation.)  During that
presentation, a number of issues with the base encoding were mentioned as
issues to be addressed.

https://datatracker.ietf.org/doc/html/draft-szarecki-idr-bgp-lcu-traffic-steering-00

-- Jeff

---

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Tue, 15 Feb 2022 00:49:05 +0000

Hi Peng Shaofu,

I read draft-zhou-idr-inter-domain-lcu, and understand what you are proposing. This proposal is an informational draft, with no extension to LU encoding, uses Addpath, and just resolves over tunnels with color.

This is the first thing we considered, to avoid any protocol extensions.

But the following problems appear with this model:

1/ resolving service-routes with color, over LU routes with color – is not efficient. Because we cannot do LPM (longest-prefix-match) on the nexthop address to find the matching route. Because the route with desired color community may exist anywhere in the route list.

     Unless, user configures mechanisms to match on the color community and leak LU routes of a color to a ‘CT like’ transport-RIB database which contains prefixes of that color. This is what CT does, and LU plugs into the CT architecture in this manner.

2/ like you note, this proposal requires addpath for ebgp, and also add-path for non-bgp routes. Which may not be supported by all implementations today. These may not be a show-stopper, but a thing to consider.

3/ Add-path ID is good at not hiding paths at path-selection pinch points like RR, but not as an end-to-end generic distinguisher. It may make troubleshooting very hard. More so, when the color-communities are rewritten across domain borders to accommodate independently administered community namespaces.

4/ performing multipath or protection for the ‘LU-routes with a certain color’ is not possible, because all routes belong to the same LU-prefix’s route-list.

     Unless again, you have user configured policies to segregate LU routes with a color into separate CT transport RIBs.

So, considering such issues that came out when we started analyzing it, and considering the user-configuration required
across different vendor implementations, to make things work, we decided it is best to establish well-known protocol procedures,
to achieve ‘nexthop resolution over tunnel routes with color’. And we re-used existing 4364 machinery at the transport-layer to
achieve that, which is well understood by network operators today.

Other protocol-extension proposals that carry color in the NLRI (BGP SRTE, BGP LCU, BGP CAR) have the basic problem that neither IP:Color or Color:IP encoding can satisfy all the following requirements:

Supporting all the following usecases simultaneously is not possible:

  *   LPM fallback between tunnels with a color, and best-effort tunnels (that don’t have a color) . This requires IP:Color format
  *   LPM fallback between tunnels with a color, and color-only (SRTE) tunnels.                                   This requires Color:IP format
  *   Carry non-host IP tunnel-prefixes with Color in the NLRI.                                                                  This requires Color:IP format
  *   Carrying best-effort tunnels (IP-prefix only) in the NLRI.                                                                           This requires IP:Color format

Not carrying color in the NLRI enables CT to achieve all the above. It is more inline with nexthop resolution without color that happens in deployed BGP networks today.

IOW, CT is a scale-out version of BGP nexthop-resolution that is deployed today. So there are less surprises and nexthop resolution process scales better, because it is confined to routes with a certain color.

There are other aspects of CT vs CAR that has been discussed in the Interim meeting. Please go thru the link Jeff provided.

Thanks
Kaliraj

From: Jeffrey Haas <[email-masked]>
Date: Monday, February 14, 2022 at 1:18 PM
To: [email-masked] <[email-masked]>
Cc: Kaliraj Vairavakkalai <[email-masked]>, [email-masked] <[email-masked]>, [email-masked] <[email-masked]>, [email-masked] <[email-masked]>
Subject: Re: [Idr] Request WG adoption call for draft-kaliraj-idr-bgp-classful-transport-planes
[External Email. Be cautious of content]

Peng Shaofu,

https://urldefense.com/v3/__https://datatracker.ietf.org/meeting/interim-2022-idr-02/session/idr__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0VG0jdfw$<https://urldefense.com/v3/__https:/datatracker.ietf.org/meeting/interim-2022-idr-02/session/idr__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0VG0jdfw$>

https://urldefense.com/v3/__https://notes.ietf.org/notes-ietf-interim-2022-idr-02-idr*__;Iw!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0V02IePv$<https://urldefense.com/v3/__https:/notes.ietf.org/notes-ietf-interim-2022-idr-02-idr*__;Iw!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0V02IePv$>

https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draft-szarecki-idr-bgp-lcu-traffic-steering-00__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0fkPl9uM$<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-szarecki-idr-bgp-lcu-traffic-steering-00__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0fkPl9uM$>

-- Jeff

On Mon, Feb 14, 2022 at 03:50:17PM +0800, [email-masked] wrote:
> I also draw your attention to https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-zhou-idr-inter-domain-lcu/__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0QzLTbMD$<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-zhou-idr-inter-domain-lcu/__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0QzLTbMD$> , a simple solution based on existing protocol. IMO, the extension of the existing mechanism needs to answer the following questions:
>     1) why need extensions ?
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0TaSuBJY$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0TaSuBJY$>
>
> + IDR WG alias.
>
> Thanks
> Kaliraj
>
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0TaSuBJY$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0TaSuBJY$>

---

From: <[email-masked]>
Date: Tue, 15 Feb 2022 09:56:45 +0800 (CST)

Hi Kaliraj, Jeff,

Thanks for your information. : )
I'll take some time to get it, then come back later.

Regards,
PSF

------------------原始邮件------------------
发件人：KalirajVairavakkalai
收件人：Jeffrey Haas;彭少富10053815;
抄送人：[email-masked];[email-masked];[email-masked];
日 期 ：2022年02月15日 08:51
主 题 ：Re: [Idr] Request WG adoption call for draft-kaliraj-idr-bgp-classful-transport-planes
" _ue_custom_node_="true">
Hi Peng Shaofu,
I read draft-zhou-idr-inter-domain-lcu, and understand what you are proposing. This proposal is an informational draft, with no extension to LU encoding, uses Addpath, and just resolves over tunnels with color.
This is the first thing we considered, to avoid any protocol extensions.
But the following problems appear with this model:
1/ resolving service-routes with color, over LU routes with color – is not efficient. Because we cannot do LPM (longest-prefix-match) on the nexthop address to find the matching route. Because the route with  desired color community may exist anywhere in the route list.
Unless, user configures mechanisms to match on the color community and leak LU routes of a color to a ‘CT like’ transport-RIB database which contains prefixes of that color. This is what CT does, and  LU plugs into the CT architecture in this manner.
2/ like you note, this proposal requires addpath for ebgp, and also add-path for non-bgp routes. Which may not be supported by all implementations today. These may not be a show-stopper, but a thing to consider.
3/ Add-path ID is good at not hiding paths at path-selection pinch points like RR, but not as an end-to-end generic distinguisher. It may make troubleshooting very hard. More so, when the color-communities  are rewritten across domain borders to accommodate independently administered community namespaces.
4/ performing multipath or protection for the ‘LU-routes with a certain color’ is not possible, because all routes belong to the same LU-prefix’s route-list.
Unless again, you have user configured policies to segregate LU routes with a color into separate CT transport RIBs.
So, considering such issues that came out when we started analyzing it, and considering the user-configuration required
across different vendor implementations, to make things work, we decided it is best to establish well-known protocol procedures,
to achieve ‘nexthop resolution over tunnel routes with color’. And we re-used existing 4364 machinery at the transport-layer to
achieve that, which is well understood by network operators today.
Other protocol-extension proposals that carry color in the NLRI (BGP SRTE, BGP LCU, BGP CAR) have the basic problem that neither IP:Color or Color:IP encoding can satisfy all the following requirements:
Supporting all the following usecases simultaneously is not possible:
LPM fallback between tunnels with a color, and best-effort tunnels (that don’t have a color) . This requires IP:Color format
LPM fallback between tunnels with a color, and color-only (SRTE) tunnels.                                   This requires Color:IP format
Carry non-host IP tunnel-prefixes  with Color in the NLRI.                                                                  This requires Color:IP format
Carrying best-effort tunnels (IP-prefix only) in the NLRI.                                                                           This requires IP:Color format
Not carrying color in the NLRI enables CT to achieve all the above. It is more inline with nexthop resolution without color that happens in deployed BGP networks today.
IOW, CT is a scale-out version of BGP nexthop-resolution that is deployed today. So there are less surprises and nexthop resolution process scales better, because it is confined to routes with  a certain color.
There are other aspects of CT vs CAR that has been discussed in the Interim meeting. Please go thru the link Jeff provided.
Thanks
Kaliraj
From: Jeffrey Haas <[email-masked]>
Date: Monday, February 14, 2022 at 1:18 PM
To: [email-masked] <[email-masked]>
Cc: Kaliraj Vairavakkalai <[email-masked]>, [email-masked] <[email-masked]>, [email-masked] <[email-masked]>, [email-masked] <[email-masked]>
Subject: Re: [Idr] Request WG adoption call for draft-kaliraj-idr-bgp-classful-transport-planes
[External Email. Be cautious of content]
Peng Shaofu,
You may find the minutes, and a link to the video for the most recent
interim at the following link:
https://urldefense.com/v3/__https://datatracker.ietf.org/meeting/interim-2022-idr-02/session/idr__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0VG0jdfw$
My recommendation is to view the minutes at the notes site.  There seems to
be an issue with the markdown rendering on the official import.
https://urldefense.com/v3/__https://notes.ietf.org/notes-ietf-interim-2022-idr-02-idr*__;Iw!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0V02IePv$
Please also note there was an earlier LCU proposal that was presented at
IETF prior to BGP-CT and BGP-CAR.  (I gave the presentation.)  During that
presentation, a number of issues with the base encoding were mentioned as
issues to be addressed.
https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draft-szarecki-idr-bgp-lcu-traffic-steering-00__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0fkPl9uM$
-- Jeff
On Mon, Feb 14, 2022 at 03:50:17PM +0800, [email-masked] wrote:
> I also draw your attention to  https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-zhou-idr-inter-domain-lcu/__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0QzLTbMD$ , a simple solution based on existing protocol. IMO, the extension of the existing  mechanism needs to answer the following questions:
>     1) why need extensions ?
>  https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0TaSuBJY$
>
> + IDR WG alias.
>
> Thanks
> Kaliraj
>
>  https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!XFh2ZLyp8qK67AWvAsgELmHXtMLW1LFh1SYKnUlx-dDhuawA0Tb1U-sF0TaSuBJY$

---

From: <[email-masked]>
Date: Fri, 18 Feb 2022 17:42:40 +0800 (CST)

Hi Kaliraj,

Thank you again for the detailed explainations.
please see inline.

PS: In early 2019, when we used BGP-LU to implement network slicing (please refer to draft-peng-lsr-network-slicing-00, section "6.3.  inter-domain network slicing via BGP-LU"), we were faced with two methods: one is color EC + add-path, and the other is to define a new NLRI key = <color, prefix>. Considering the implementation cost and code inheritance, we chose the former. So far, ZTE has supported the network slicing function through BGP-LU color + add-path for 3 years. 

Regards,
PSF

------------------原始邮件------------------
发件人：KalirajVairavakkalai

[PSF] Your guess is right. There is exactly a 'CT like' RIB, the FEC KEY is <color, prefix> for BGP-LU color LSP. And, the resolution is indeed a local implementation, for example, a tunnel-selection module can support the priority of resolved tunnel types, e.g, exact matching SR policy > BGP-LU color > Flex-algo > ... > best-effort routes.

[PSF] Only local changes are made to the original conditions for sending multipath. It's easier than defining a new set of standards. And, due to the wide deployment of reflectors, add-path is a basic function, right ? In fact, both BGP-CT and BGP-CAR mention that they rely on add-path.

[PSF] Add-path ID is not used as end-to-end distinguisher, but Color. According to SR policy color, we think that color SHOULD be global, considering that egress PE advertised a service route containing color EC to ingress PE and ingress PE must recognize it. However, if that is not the case, LCM defined in BGP-CAR is a possible method to address that.

[PSF] Your guess is right. There is no problem to create ECMP/FRR items for separate BGP-LU color FEC.

[PSF] One concern about BGP-CT is that, it needs maintain VRF instance (although special instance) at any border nodes, that may be a retrogression to option C.

[PSF] Not sure that I have understand these points. It is a local implementation for underlay tunnel resolution and table orgnization, I don't think there is a necessary connection between table orgnization and message format.

[PSF] Seems that we have not reach consensus according to minutes. IMO, each solution can work, but, still have question: why ? What is the pain point scene that  be covered by one solution and not by other ?

Thanks
Kaliraj

Peng Shaofu,

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 18 Feb 2022 11:16:14 -0500

Peng Shaofu,

A detail important for all proposals of ebgp add-paths is that it's not a fully supported scenario.

The technical issues surrounding that feature is documented in this older internet-draft.  The mechanisms in this I-D are not implemented by any vendor that I'm aware of.

https://datatracker.ietf.org/doc/html/draft-pmohapat-idr-fast-conn-restore-03 <https://datatracker.ietf.org/doc/html/draft-pmohapat-idr-fast-conn-restore-03>

-- Jeff

> On Feb 18, 2022, at 4:42 AM, [email-masked] wrote:
> 
> Hi Kaliraj,
> 
> 抄送人：[email-masked] <mailto:[email-masked]>;[email-masked] <mailto:[email-masked]>;[email-masked] <mailto:[email-masked]>;
> 日 期 ：2022年02月15日 08:51

---

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Sat, 19 Feb 2022 09:21:03 +0000

Pls see inline KV>

From: [email-masked] <[email-masked]>
Date: Friday, February 18, 2022 at 1:43 AM
To: Kaliraj Vairavakkalai <[email-masked]>
Cc: [email-masked] <[email-masked]>, [email-masked] <[email-masked]>, [email-masked] <[email-masked]>, [email-masked] <[email-masked]>
Subject: Re:[Idr] Request WG adoption call for draft-kaliraj-idr-bgp-classful-transport-planes
[External Email. Be cautious of content]

Hi Kaliraj,

Regards,
PSF

------------------原始邮件------------------
发件人：KalirajVairavakkalai

KV> Do you mean LU received routes are installed in the LU RIB, and in addition to a new RIB, where FEC key is <color, prefix>? How do best-effort tunnels get represented here, as they don’t have any color?
KV> How does fallback between two colors work? and fallback to best-effort tunnel work? Does these fallback mechanism not use longest-prefix-match (LPM) lookup for resolution, like regular BGP procedures https://www.rfc-editor.org/rfc/rfc4271#section-9.1.2.1 ?

KV> Pls note that, whatever resolution mechanism you follow needs to be followed by all LU nodes on the path, to satisfy the SLA constraint end-to-end. But there is no signaling/capability-negotiation
KV> of any sort to know whether all nodes on the path support your new mechanisms. So ingress-PE cannot be really sure if it has established a path that satisfies the desired SLA end-to-end.

KV> do u mean ‘sending addpath’? yes, CT uses addpath at RRs, which is well deployed. But like Jeff noted, ebgp-addpath is not well deployed.

KV> I don’t understand this part. You said your approach uses plain LU encoding. Then how can you use Color as distinguisher?

KV> Do you mean the MPLS swap routes derive their nexthop from the <color, ip-prefix> Table?

KV> If you think about it, in your approach, you are doing the similar things like VRF but using special procedures.
KV> viz. installing route in a new RIB beside LU RIB,  based on Color community. Forming multipath, FRR, label-allocation based on entries in that RIB
KV> Specifying to use RFC-4364 procedures at Transport layer just standardizes the procedures without any ambiguity. Also we get benefit of RTC.
KV> Please note that in CT, whether you achieve the Transport RIB construct using a VRF or not is up to the implementation. The Transport-RIB is just a logical subset of routes which belong to the same color.

KV> Actually, the last two points are related to NLRI message format only, not local implementation.
KV> The first two are local implementation choice indeed, but they affect what kind of resolution and fallback are possible using LPM lookup.

KV> Above we discussed few points, about CAR NLRI encoding (IP:color) not able to support non-host prefix. RTC like mechanisms not applying for CAR. Etc..
KV> The different aspects of CT vs CAR are being discussed in the interim and ongoing email threads with questions from Chairs. These discussions will eventually lead to a conclusion. Thanks.

Thanks
Kaliraj

Peng Shaofu,

-- Jeff