Subject: Re: [Idr] BGP routes with color, Question 1: How does route resolution work with your feature?

From: "Dhananjaya Rao (dhrao)" <[email-masked]>
Date: Wed, 2 Mar 2022 12:46:16 +0000

Hello Jeff, co-chairs,

We've tried to provide succinct descriptions and references to sections in the BGP CAR draft below. Please let us know if any other clarifications would be useful.

BGP route resolution in CAR is recursive and occurs at both service steering and transport routing levels. Your question appears to be related to the former, but the model is consistent across both.
 
A colored service route V/v with Color-EC C, and BGP next-hop E is resolved via BGP-CAR route (E, C) with next-hop N, which is recursively resolved via CAR route (N, C) provided by any color-aware mechanism such as IGP-FA or SR-Policy or even BGP-CAR.
 
BGP CAR route resolution follows the same recursive resolution model that has been used in IP routing and MPLS networks for the past 20+ years (E via N). The only difference is that the route and next-hop are color-aware - (E, C) and (N, C).
 
This model is also present in SR deployments today where color-aware paths are being used, with a colored service route resolves via paths provided by SR-Policy and IGP-FlexAlgo
(draft-ietf-spring-segment-routing-policy-18).
 
The resolution model and it’s associated capabilities are described in sections of the BGP CAR draft as follows:
 
An illustration of the steering and recursive resolution model in a multi-domain topology is in section 1.2.
 
A description of the route validation and next-hop resolution procedures in sec 2.4 and 2.5. This includes fallback to a route of a different color or a best-effort route.
 
Detailed examples with topologies and resulting forwarding label stack for MPLS use-case are described in Appendix A. They cover steering and recursive resolution via IGP-FlexAlgo and SR-Policy, hybrid and brownfield scenarios.

Regards,
-Dhananjaya

On 2/18/22, 9:13 PM, "Idr on behalf of Jeffrey Haas" <[email-masked] on behalf of [email-masked]> wrote:

    As promised during the recent IDR interim, we want to start a set of threads
    to the IDR mailing list to clarify points discussed during that interim
    about the BGP routes with color proposals.  These questions are directed to
    the authors of those proposals.  The responses to those questions will be
    used to drive Working Group discussion about the solution space.

    Question 1: How does BGP route resolution operate with your proposal?

    A BGP route, marked with a Color Extended Community C, and a nexthop NH1
    needs to resolve using the proposal.  

    >From the BGP routing state received from the proposed mechanism, how does
    the resolving route make use of this state to decide its forwarding nexthop?

    Be specific in terms of functional behavior.  There is no need to go into
    implementation details unless they're helpful in explaining the functional
    behavior.

    If possible, please refer to relevant sections in your internet-drafts.

    -- Jeff

    _______________________________________________
    Idr mailing list
    [email-masked]
    https://www.ietf.org/mailman/listinfo/idr

---

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Thu, 3 Mar 2022 00:49:25 +0000

> A description of the route validation and next-hop resolution procedures in sec 2.4 and 2.5. This includes fallback to a route of a different color or a best-effort route.

I don’t see any discussion aboufallback.

Because of using (N,C1) to do the resolution, longest-prefix-match cannot be used to provide fallback to (N, C2).

https://www.rfc-editor.org/rfc/rfc4271#section-9.1.2.1

                                                    If multiple matching
         routes are available, only the longest matching route SHOULD be
         considered.

I wanted to point out that, any mechanism that uses Color in the lookup key cannot satisfy this RFC-4271 recommendation.

Thanks
Kaliraj
From: Idr <[email-masked]> on behalf of Dhananjaya Rao (dhrao) <dhrao=[email-masked]>
Date: Wednesday, March 2, 2022 at 4:46 AM
To: Jeffrey Haas <[email-masked]>, [email-masked] <[email-masked]>
Subject: Re: [Idr] BGP routes with color, Question 1: How does route resolution work with your feature?
[External Email. Be cautious of content]

Hello Jeff, co-chairs,

Regards,
-Dhananjaya

    -- Jeff

    _______________________________________________
    Idr mailing list
    [email-masked]
    https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!XCdws0gL6V-QzjaD-uDvqGWr1NteHznYndQquANwKhKuAVcI99qiCFtzUoptaHr7$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!XCdws0gL6V-QzjaD-uDvqGWr1NteHznYndQquANwKhKuAVcI99qiCFtzUoptaHr7$>

_______________________________________________
Idr mailing list
[email-masked]
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!XCdws0gL6V-QzjaD-uDvqGWr1NteHznYndQquANwKhKuAVcI99qiCFtzUoptaHr7$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!XCdws0gL6V-QzjaD-uDvqGWr1NteHznYndQquANwKhKuAVcI99qiCFtzUoptaHr7$>

Juniper Business Use Only

---

From: "Dhananjaya Rao (dhrao)" <[email-masked]>
Date: Mon, 7 Mar 2022 13:26:02 +0000

Section 2.4 describes resolving the BGP CAR next-hop via default table instead of a color-aware path.

Section 2.5 describes resolving (E2, C1) via (N, C2).

For the service steering case, Section 3 refers to the SR-Policy draft since the steering design is consistent with SR-TE. As mentioned earlier, there are multiple implementations that support fallback with this design that are deployed today.

To the other comment, the CAR NLRI data model does not impose any constraints on the implementation, hence longest match can indeed be supported. That said, since fallback is across colors, I don’t see how the clause referred below is relevant.

Regards,
-Dhananjaya

From: Kaliraj Vairavakkalai <kaliraj=[email-masked]>
Date: Thursday, March 3, 2022 at 6:20 AM
To: "Dhananjaya Rao (dhrao)" <[email-masked]>, Jeffrey Haas <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [Idr] BGP routes with color, Question 1: How does route resolution work with your feature?

                                                    If multiple matching

Thanks
Kaliraj

Hello Jeff, co-chairs,

Regards,
-Dhananjaya

    -- Jeff

---

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Tue, 8 Mar 2022 00:33:23 +0000

>Section 2.5 describes resolving (E2, C1) via (N, C2).

This is recursive resolution. Not fallback C1 over C2.

The problems I mention includes any model that uses Color in the lookup key. That includes SRTE.

Like I mentioned, I am just noting that longest-prefix-match based implementations may not be possible, when include Color in the lookup key. ‘Suboptimal lookup’ implementations may be possible that involve walking multiple tables. And effects of those will show up with scale. It is easy to see this when analyzing it with a data-structure mind-set. LPM is of-course not a mandatory requirement, but it does scale well. That is why, I think RFC-4271 mentions it. Procedures descried in CT allow for LPM and are inline with RFC-4271:9.1.2.1.

Another important aspect: things like carrying non-host transport-prefixes in the NLRI. CT can do that, but CAR cannot. Because of the “EP:C” format in NLRI, the EP cannot be a prefix, it needs to be /32 or /128 always.
Quick ref: https://datatracker.ietf.org/doc/html/draft-dskc-bess-bgp-car-03#section-2.9.2

Where-as, BGP-LU, BGP-L3VPN and BGP-CT can carry those subnet prefixes too.

Thanks
Kaliraj
From: Dhananjaya Rao (dhrao) <dhrao=[email-masked]>
Date: Monday, March 7, 2022 at 5:26 AM
To: Kaliraj Vairavakkalai <[email-masked]>, Jeffrey Haas <[email-masked]>, [email-masked] <[email-masked]>
Subject: Re: [Idr] BGP routes with color, Question 1: How does route resolution work with your feature?
[External Email. Be cautious of content]

Regards,
-Dhananjaya

https://www.rfc-editor.org/rfc/rfc4271#section-9.1.2.1<https://urldefense.com/v3/__https:/www.rfc-editor.org/rfc/rfc4271*section-9.1.2.1__;Iw!!NEt6yMaO-gk!Tzz23SoYWQea7smkPi_NdtqknFP_1mqC9ok_XR0gmyoqJnJmFx940fYEdyvs5izt$>

                                                    If multiple matching

Thanks
Kaliraj

Hello Jeff, co-chairs,

Regards,
-Dhananjaya

    -- Jeff

---

From: "Dhananjaya Rao (dhrao)" <[email-masked]>
Date: Fri, 18 Mar 2022 00:28:25 +0000

It is both, since if (N, C1) is not available, then the resolution will be via (N, C2)

I see you’ve gone from asserting that it is not possible to saying it may not be possible. With a bit of further analysis, you will also see that it is not only possible, but also can be equally efficient. The problem is that you keep conflating the protocol definition with a specific implementation.

You also make a reference to a section in the BGP CAR draft. That section clearly describes the support and specification for an IP prefix. So, not sure how you ended up with the opposite conclusion.

Hope this helps.

Regards,
-Dhananjaya

From: Kaliraj Vairavakkalai <kaliraj=[email-masked]>
Date: Tuesday, March 8, 2022 at 6:03 AM
To: "Dhananjaya Rao (dhrao)" <[email-masked]>, Jeffrey Haas <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [Idr] BGP routes with color, Question 1: How does route resolution work with your feature?

Thanks
Kaliraj

Regards,
-Dhananjaya

                                                    If multiple matching

Thanks
Kaliraj

Hello Jeff, co-chairs,

Regards,
-Dhananjaya

    -- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Sat, 19 Mar 2022 17:06:06 -0400

Dhananjaya,

Thanks for your response.  I have some clarifiation questions:

> BGP route resolution in CAR is recursive and occurs at both service steering and transport routing levels. Your question appears to be related to the former, but the model is consistent across both.

Indeed, the primary motivation for the question was for service steering.
Thanks for clarifying that the described procedures apply to both.

> A colored service route V/v with Color-EC C, and BGP next-hop E is resolved via BGP-CAR route (E, C) with next-hop N, which is recursively resolved via CAR route (N, C) provided by any color-aware…

"via (E,C)".

When a Local-Color-Mapping Extended Community is present on the BGP CAR
route, does this mean that this CAR route contributes to resolution as
(E,LCM) rather than (E,C) where C is the original color present in the NLRI
key?

If so, does that mean that when LCM is in use, the original (E,C) is not
permitted to contribute to resolution?

Can there be more than one LCM present on a BGP CAR route?  If so, how does
that interact with resolution?

> BGP CAR route resolution follows the same recursive resolution model that has been used in IP routing and MPLS networks for the past 20+ years (E via N). The only difference is that the route and n…

It's important to note that a property of this historical recursive model
for existing technologies is that it is longest prefix match. 

> A description of the route validation and next-hop resolution procedures in sec 2.4 and 2.5. This includes fallback to a route of a different color or a best-effort route.

Section 2.5 contains the following text in version -03 of your draft:

:    When multiple resolutions are possible, the default preference should
:    be: IGP Flex-Algo, SR Policy, BGP CAR, BGP LU.

Functionally, this can be interpreted as "iterate through the schemes above
by preference order and try to resolve (N,C) in each".  This is clear.

The next paragraph describes color fallback:
:    Through local policy, a BGP color-aware route (E2, C1) from N may be
:    resolved over a color-aware route (N, C2): i.e. the local policy maps
:    the resolution of C1 over C2.  For example, in a domain where
:    resource R is known to not be present, the inter-domain intent
:    C1="low delay and avoid R" may be resolved over an intra-domain path
:    of intent C2="low delay".

BGP CAR routes are endpoint prefixes with a color binding.

For cases where the only CAR prefixes in the system are of host length,
fall-back schemes for color seem fairly clear.  You would search on (N,C1)
and (N,C2) is permitted to match.

When the CAR routes are not host routes and there are perhaps overlapping
color fallback intents, e.g. if not found in C1 try C2, how should the
lookup functionally occur?  Would this be a longest prefix match vs. N for
C1 first and then fallback to a longest prefix match vs. N for C2?

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Sat, 19 Mar 2022 17:31:35 -0400

Kaliraj,

On Fri, Feb 18, 2022 at 07:42:45PM +0000, Kaliraj Vairavakkalai wrote:
> How transport class mapping and fallback happens:
> 
>                 https://datatracker.ietf.org/doc/html/draft-kaliraj-idr-bgp-classful-transport-planes-13#section-7
> 
>                 This section describes it in detail. In short,
> 
> 
>   *   BGP route marked with Mapping community (e.g. Color Extended Community C, or Transport Target Extended community C).
> 
> The mapping-community maps to a locally defined ‘Resolution scheme’.
> 
>   *   The Resolution scheme consists of a primary transport class. And optionally, ordered list of one or more fallback
> 
> transport classes of other colors, including best-effort (tunnels that don’t have any color associated with them).

The following text is from version -13 of your draft:

:    A BGP route is associated with a resolution scheme during import
:    processing.  The first community on the route that matches a mapping
:    community of a locally configured resolution scheme is considered the
:    effective mapping community for the route.  The resolution scheme
:    thus found is used when resolving the route's PNH.  If a route
:    contains more than one mapping community, it indicates that the route
:    considers these multiple mapping communities as equivalent.  So the
:    first community that maps to a resolution scheme is chosen.

Extended Communities don't have a sense of "first".  Implementations may
re-order these communities as part of their processing.  So, what is "first"
in this context?

While I realize the text above suggests that the resolution schemes should
be considered equivalent, consider the case where multiple transport ribs
service these communities; one for each of them.  Even if two routers have
identical transport RIBs from received BGP-CT routes, if the contents of the
destinations are not identical then you could have inconsistent route
selection if router-1 picks one forwarding behavior and router-2 picks
another.

-- Jeff

---

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Sun, 20 Mar 2022 07:12:24 +0000

Hi Jeff,

‘first’ is in terms of order of parsing attributes on the route, order of parsing contents of a community/extended-community attribute.

And you are right, we cannot rely on the ordering of attributes or communities on a route. Let me clarify the intended behavior:

A route should be attached with one mapping community MC1. Which maps to a resolution scheme RS1.

Multiple mapping communities MC1, MC2 MAY be attached on a route, as long as:

  1.  Either both MC1, MC2 map to the same resolution scheme RS1.
  2.  Or, only one of them map to a resolution-scheme locally on a receiving speaker. Yielding one effective mapping-community for the route.

So the order of the communities on the route actually does not matter.

The renumbering/migration scenarios may make use of scenario a. above.

And example of where b. may be useful is: a supercore domain having more colors (e.g. light-red, dark-red) than edge domains(e.g. red).
A CT route can be attached with two communities, dark-red and red, such that it maps to dark-red tunnels when traversing the supercore
and maps to red tunnels when traversing the edge domain. Still, at a box the route maps to one resolution-scheme only. The other community
is inactive, not acting as a mapping community.

To summarize, if a deployment uses two communities on a route that simultaneously map to two distinct resolution-schemes on a receiving BGP speaker, it is considered a mis-configuration.
A route has only one effective mapping-community, and maps to only one resolution-scheme.

Thanks
Kaliraj
From: Jeffrey Haas <[email-masked]>
Date: Saturday, March 19, 2022 at 2:31 PM
To: Kaliraj Vairavakkalai <[email-masked]>
Cc: [email-masked] <[email-masked]>
Subject: Re: [Idr] BGP routes with color, Question 1: How does route resolution work with your feature?
[External Email. Be cautious of content]

Kaliraj,

On Fri, Feb 18, 2022 at 07:42:45PM +0000, Kaliraj Vairavakkalai wrote:
>                 https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draft-kaliraj-idr-bgp-classful-transport-planes-13*section-7__;Iw!!NEt6yMaO-gk!R_qPL_SrPBboWJpbQ8cGeijgQclvaL147gAZdyDuNII0VksREFQS1bZqUdgC18Bl$<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-kaliraj-idr-bgp-classful-transport-planes-13*section-7__;Iw!!NEt6yMaO-gk!R_qPL_SrPBboWJpbQ8cGeijgQclvaL147gAZdyDuNII0VksREFQS1bZqUdgC18Bl$>
>

-- Jeff

---

From: "Dhananjaya Rao (dhrao)" <[email-masked]>
Date: Mon, 21 Mar 2022 10:35:22 +0000

Hi Jeff,

Thank you for your careful review and comments/questions. Please see inline (DR##)

On 3/20/22, 2:36 AM, "Jeffrey Haas" <[email-masked]> wrote:

    Dhananjaya,

    "via (E,C)".

DR## As you phrased it, LCM color is used when the route is used for resolution. Just to restate, there is no rewrite of the BGP NLRI color as it traverses hops.

DR## I did not understand the meaning of this comment. Can you elaborate ?

DR## Yes, and CAR does not attempt to change it.
DR## However, it is also worth noting that for technologies such as MPLS transport used historically for VPNs and other services, the recursive resolution ultimately has been a match on the host route for the PE/ASBR that’s the BGP service route’s next-hop. There is no aggregation. So, this point to some extent appears academic.

DR## That would be the functional behavior by default.
DR## I’m a bit unclear on whether you’re looking for anything specific in terms of behaviors or differences.

Regards,
-Dhananjaya

    -- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 21 Mar 2022 09:07:03 -0400

Dhananjaya,

> DR## As you phrased it, LCM color is used when the route is used for resolution. Just to restate, there is no rewrite of the BGP NLRI color as it traverses hops.

Thanks.  The property I'm trying to ensure we're clear on is that the RIB
key, (E,C) is not necessarily the resolution key (E, effective color).  The
effective color will be the C from the NLRI unless an LCM is present.

> DR## I did not understand the meaning of this comment. Can you elaborate ?

The detail I'm trying to confirm is that if a BGP Speaker implementing -CAR
has received a BGP route (E,C) with LCM C1 that it contributes to resolution
solely as (E,C1) and not (E,C).

> >     that interact with resolution?

I don't believe this question received an answer?

> DR## However, it is also worth noting that for technologies such as MPLS transport used historically for VPNs and other services, the recursive resolution ultimately has been a match on the host ro…

The motivation for the above points is to illustrate that service route
resolution is not solely (E,C) as received as the NLRI key.  The resolution
key is (E, effective color).

> DR## I’m a bit unclear on whether you’re looking for anything specific in terms of behaviors or differences.

The motivations for the two questions from the interim come from attempting
to understand comparability between the two proposals.

Thanks for helping to clarify this item.

Once we have the missing answer for multiple LCM, I think we've closed the
questions from the interim.  FWIW, Kaliraj indicates in -CT that it is a
misconfiguration.  If the same holds for -CAR, that clarifies the intended
behavior and perhaps suggests need for a section on the topic in each
proposal's Error Handling section.

-- Jeff

---

From: "Dhananjaya Rao (dhrao)" <[email-masked]>
Date: Mon, 21 Mar 2022 13:31:32 +0000

Jeff, 

Please see inline (DR>>)

On 3/21/22, 6:37 PM, "Jeffrey Haas" <[email-masked]> wrote:

    Dhananjaya,

DR>> Yes. Section 2.8 of the BGP CAR draft states this along with the rationale.

DR>> Okay. Hope the above reference clarifies the behavior.

DR>> My bad. There can only be a single LCM present on a BGP CAR route.

DR>> Okay.

DR>> Ack, thanks for the perseverance.

Regards,
-Dhananjaya

    -- Jeff

---

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Mon, 21 Mar 2022 16:13:24 +0000

Jeff, Dhananjay,

> >     and (N,C2) is permitted to match.

In this statement, for a IPv4 prefix N, is the lookup-key /32 or /64? Fallback wouldn’t work with /64. So is it a /32 lookup-key and then one or more O(NumColors) searches for desired Color?

And, what happens when a route R1 wants the intent of: C1 primary, fallback to C2. But another route R2 wants the intent: C2 is primary, fallback to C1?

How do the routes specify/disambiguate their intents?
      E.g. if route R3 wants the intent of: C1 primary, fallback to C3.
                           R4 wants C1 primary, fallback best-effort?
      How do routes R1, R3 and R4 specify their intents?

Thanks
Kaliraj

---

From: Natrajan Venkataraman <[email-masked]>
Date: Mon, 21 Mar 2022 18:41:19 +0000

Hi Dhananjaya,

This is Nats (One of the co-authors of the BGP-CT draft). I have questions on the following (NV)

DR>> Okay.

NV>> From the above,

I see that having the LCM attribute alone in the CAR NLRI universally solves the problem of communicating the color across different border nodes and towards the ingress PE nodes. All receivers of CAR NLRI need to make this local decision as to use the NLRI Key fully or to throw away the color in the key and use the LCM attribute as it creates an ambiguity where the NLRI key may be the same or different from the Resolution key. Are there any strong reasons for having color in the NLRI against having LCM as a sole representation of color?

Regards,
-Nats-

From: Idr <[email-masked]> on behalf of Dhananjaya Rao (dhrao) <dhrao=[email-masked]>
Date: Monday, March 21, 2022 at 6:32 AM
To: Jeffrey Haas <[email-masked]>
Cc: [email-masked] <[email-masked]>
Subject: Re: [Idr] BGP routes with color, Question 1: How does route resolution work with your feature?
[External Email. Be cautious of content]

Jeff,

Please see inline (DR>>)

    Dhananjaya,

DR>> Okay.

Regards,
-Dhananjaya

    -- Jeff

_______________________________________________
Idr mailing list
[email-masked]
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!WY_ovfrLsfg2jcynx1JXcb7GKZ9PPEYNnY1F3ZJqc9V5P9d9wfyXSMIg4fMPVA$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!WY_ovfrLsfg2jcynx1JXcb7GKZ9PPEYNnY1F3ZJqc9V5P9d9wfyXSMIg4fMPVA$>