Subject: [Idr] Color domains in BGP routes with color

From: Jeffrey Haas <[email-masked]>
Date: Tue, 22 Mar 2022 15:26:22 -0400

Working Group,

One of the microphone discussion and debate points was about how each of the
proposals dealt with color domains.  The fact that the similar questions
continue to be raised about this suggest inconsistency in understanding of
what we mean by this.  This results in inconsistent discussion about
comparing the proposals.

To try to take a start to this discussion, consider the following quote from
the CAR document:

"Let us assume that these two domains do not share the same color-to-intent
mapping." -car §2.8

A "color domain" is defined by a set of cooperating BGP Speakers under one
or more administrative controls (thus, single to multiple ASes) wherein a
"color" (a 32-bit number) has the same "intent".  

We'll note that "intent" itself is somewhat inconsistently used.  For
purposes of this discussion, presume that its meaning is locally understood
and consistent across the set of devices that implement it.

The trivial case is that BGP routes from any of the proposals can pass
across the domain and maintain the same intent.  Thus, this is true for
iBGP, as well as eBGP.

When a second or further color domain is under consideration, we use the
quote from -CAR cited above as our discussion point:  The same color to
intent mapping doesn't exist.  The question is what does this mean to the
color that is carried in the route in each of the proposals?

There is the indirect but critical point that since the definition is that
the mapping isn't shared (different namespaces), but there's no namespace
identifier, what happens to the same color value across domains in each of
the proposals?

The following is my perception of the treatment from -car and -ct.  The
authors are requested to correct these perceptions.

---

For -car, the color is by default carried in the (E,C) NLRI key.  (E,C)
serves as the RIB key for that BGP AFI/SAFI.  By default, the resolution
table key is (E,C); when LCM is present, it is (E,LCM).

LCM is added or updated when routes cross between color domains. (Section 2.8)

The NLRI key isn't updated, the route remains (E,C) for BGP propagation
purposes.

If the cooperating color domains cooperate enough, C is a value that is not
re-used across the domains.  In such cases, domain 1 using C1 doesn't have
to worry about domain 2+ wanting to use C1.

The BGP NLRI key is (E,C).  As long as the varying domains do not originate
the same (E,C), for BGP route propagation purposes there is no concern.  The
resolution table key (E,C) for a local domain and (E,C overridden by LCM C2)
have different resolution intents.

The remaining problematic scenario would involve the case where E is anycast
and (E,C) was originated by multiple color domains, C has different intent
across those domains.  In such cases, even though a given (E, C overidden by LCM)
may have a different resolution behavior implementing the intent, BGP route
selection may prune one of the paths from use.

My advice to the authors if these assumptions and their consequences are
correct is to note that for cases where (E,C) may not carry same intent
across different domains that there may be issues.  This may be addressed by
choosing distinct C across the cooperating domains.

---

For -ct, the color is by default carried in the Transport Class extended
community C.  (RD:E) serves as the RIB key for that AFI/SAFI.  In -ct, the
resolution table key is consistently (E,C).

The procedure is described in section 10 under "Coordinating between
domains using different community namespaces".

:      Such deployments may deploy
:      mechanisms to map and rewrite the Route-target values on domain
:      boundaries, using per ASBR import policies.  This is no different
:      than any other BGP VPN family.  Mechanisms employed in inter-AS
:      VPN deployments may be used with the Classful Transport family
:      also.

The text covering the color-domain remapping could use some clarifying.
However, the procedure seems clear: You simply remap the transport class
extended community.

For scenarios using distinct RDs assigned in the style of BGP VPNs,
collisions for color intent can't happen with the encoding.  The effective
color is always the color present and may be swapped at a color domain
boundary.  

Since C is not part of the -CT NLRI, collisions can't happen.

I think that for the "same RD" case we can run into similar issues across
domains when the Transport Class extended community changes meaning at a
color domain boundary.  This is because the domain will now have identical
(RD:E) routes with different colors and thus conflicting intents.  Route
selection will potentially prevent full propagation.

-- Jeff

P.S. Since I started this mail, Bruno has supplied a related question under
the subject "BGP CAR - multiple color domains"

---

From: <[email-masked]>
Date: Wed, 23 Mar 2022 15:31:51 +0800 (CST)

Hi Jeff,

Thank you for your analysis. It's very detailed. 
Does it mean that the common problem faced by BGP-CAR and BGP-CT can be solved by ADD-PATH ?

Regards,
PSF

------------------原始邮件------------------
发件人：JeffreyHaas
收件人：[email-masked];
日 期 ：2022年03月23日 03:27
主 题 ：[Idr] Color domains in BGP routes with color
Working Group,
LCM is added or updated when routes cross between color domains. (Section 2..8)
_______________________________________________
Idr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/idr

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 23 Mar 2022 05:50:42 -0400

Peng Shaofu,

> Does it mean that the common problem faced by BGP-CAR and BGP-CT can be solved by ADD-PATH ?

In the case of color domain, this isn't a simple matter of route propagation
where add-paths helps or doesn't help.

My intention is to illustrate that for each of the two proposals below 
that as routes go from one domain to another that there are situations where
remapping may have issues.

(The newer LCU proposal isn't in the analysis since I've had less time to think
about it.) 

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 23 Mar 2022 13:31:23 +0100

All,

> remapping may have issues.

Practically speaking I am not so sure about the value of complexity
introduced by the notion of remapping colors at domain boundaries. That
goes against the KISS principle.

Instead of remapping I would simply propagate tuples (ASN-Color). Now the
meaning of color in each domain has to be known via some form of publishing
it. Be it a web page or pub-sub or xyz is a different topic.

Why do we need to keep colors flat ? Moreover colors signalled by BGP
interdomain may not be the same as intradomain colors as build by some
constrained protocol.

Here is why I posting the above doubt:

* if things are under same admin control we could presume there is no
issue. When you build policy which takes into account each domain's color
mapped to some anchor points (on the edge or within a domain) and therefor
end to end path will likely be formed and work as expected

* if things are not under same administration each color/class may mean
completely separate thing unless we define in stone what it should mean.
And if we do that there is no need to remap anything any longer.

Thx a lot,
R.

---

From: Natrajan Venkataraman <[email-masked]>
Date: Wed, 23 Mar 2022 22:42:41 +0000

Hi Robert,

Please check replies highlighted with NV> inline.

Regards,
-Nats-

From: Idr <[email-masked]> on behalf of Robert Raszuk <[email-masked]>
Date: Wednesday, March 23, 2022 at 5:32 AM
To: Jeffrey Haas <[email-masked]>
Cc: idr@ietf. org <[email-masked]>
Subject: Re: [Idr] Color domains in BGP routes with color
[External Email. Be cautious of content]

All,

Practically speaking I am not so sure about the value of complexity introduced by the notion of remapping colors at domain boundaries. That goes against the KISS principle.

NV> ------------{ Start…
I agree with you practically. However, “actual” customer use cases are not always practical or ideal. There have been known features that customers (predominantly DC) have requested for end to end connectivity between multiple color domains. One such requirement was MPLS RSVP LSPs with Abstract-hop ERO where the LSP have to navigate end to end through more than one color domains where each abstract hop provides the color to go forward. Here, the customer has specifically defined different color domains within their administrative domain(where the color domains are intra-as). Therefore, what is practical might not be what the customer “actually” wants. This is not a disagreement but purely factual based on the deployments that we see where community rewrites happen even today with any color constructs at the AS boundaries.

https://www.juniper.net/documentation/us/en/software/junos/mpls/topics/ref/statement/abstract-hop-edit-protocols-mpls.html
https://www.juniper.net/documentation/us/en/software/junos/mpls/topics/topic-map/lsp-labels.html#id-example-configuring-abstract-hops-for-mpls-lsps

NV> ---------End.}

Instead of remapping I would simply propagate tuples (ASN-Color). Now the meaning of color in each domain has to be known via some form of publishing it. Be it a web page or pub-sub or xyz is a different topic.

Why do we need to keep colors flat ? Moreover colors signalled by BGP interdomain may not be the same as intradomain colors as build by some constrained protocol.

NV> ------------{ Start…

For Service routes mapping to CT transport routes, such an agreement is possible. I will try to explain how.

Can a color uniquely identify the overall end-to-end service intent? This is where the proposals CAR and CT differ w.r.t expressing intent to multiple color domains. IMHO, color is a function of the transport and intent is a function of the end-to-end service realized as a sequential ordering of colors (, a Color Scheme). Color does not fully represent the service intent.

  1.  In CAR, Color is the Intent. Therefore, the service route carries the exact color in the CAR NLRI is encoded in the form of a color community as part of the Service Route which resolves to the CAR route. However, the fallback decision is strictly local to the ingress domain ingress PE. The Ingress PE needs specific policy (per-service / per-prefix) as to what color the Ingress PE needs to fallback when the EP:Color tunnel is down. i.e, the fallback order needs to be policed.
  2.  In CT, Color (Transport Class) is a collection of transport properties that contributes to the overall intent of the service. This is where the “extended mapping community” comes in. The CT transport routes as well as service routes carry extended mapping community. This maps not just to a color but a “resolution scheme” which can be configured and there is no need for local policy at each service/border node but a re-representation of a “globally unique color resolution scheme” that represents the color sequence and its corresponding fallback order, uniquely identified and shared via the extended mapping community. For example,

     *   MapCOM - Color:0:C1 resolve strictly over Transport Class C1 (used by CT transport routes for Color C1)
     *   MapCOM - Color:0:C2 resolve strictly over Transport Class C2 (used by CT transport routes for Color C2)
     *   MapCOM – Color:0:C1BE resolve to Transport Class C1 fallback resolve to Best Effort (Used by Service routes)  <<<
     *   MapCOM – Color:0:C1C2 resolve to Transport Class C1 fallback resolve to Transport Class C2 (Used by Service routes) <<<

In CT,

the extended mapping community helps in creating the same resolution schemes across color domains identified by the mapping community values C1,C2,C1BE,C1C2. CT Transport routes are attached with extended mapping community values C1 or C2 (depending on the transport class). Service routes are attached either with extended mapping community C1BE or C1C2 (uniquely identifiable values that do not represent individual colors)

CT Transport routes:
Rewriting of the mapping community “has to happen” to transit a different color domain since colors are uniquely identified by C1 and C2 values which represent individual colors. This is also where CAR brings in the Local Color Mapping (LCM) community.

Service routes:
Rewrites to Service route mapping communites are not needed as they are uniquely identifying a resolution scheme (sequential ordering of colors) and not the individual color. That being said, different administrative domains can actually come to an agreement on uniquely identifying resolution schemes than uniquely identifying colors easily.

For example.

Color domain A:- Low-Delay: Gold(100), CBR: Silver(200)
Color domain B:- Low-Delay: Blue(400), CBR: Green(500)

Now, the resolution scheme for color C1 falling back to C2 is identified by value Color:0:C1C2 extended mapping community that the service route maps in Color Domain,

  1.  {Resolve to Gold, Fallback resolve to Silver}
  2.  {Resolve to Blue, Fallback resolve to Green}

As we could see, the service intent is achieved irrespective of the differences in color across color domains under same or different administrative domains where in, the abstract value of the resolution scheme (or the sequential order of colors that the service wants) can be easily agreed upon.

NV> ------------End. }

* if things are under same admin control we could presume there is no issue. When you build policy which takes into account each domain's color mapped to some anchor points (on the edge or within a domain) and therefor end to end path will likely be formed and work as expected

* if things are not under same administration each color/class may mean completely separate thing unless we define in stone what it should mean. And if we do that there is no need to remap anything any longer.

NV> ------------{ Start… Example describing how color is different from intent

This is from one of the CT slides for IETF IDR. With CT, the following differential treatment for the different service prefixes of the SAME service at the SAME Service Node or Border Nodes is possible.

Say,

  *   R1 wants the intent : C1 primary, fallback to best-effort
  *   R2 wants the intent : C1 primary, fallback to C2
  *   R3 wants the intent : C1 primary, fallback to C3

Which is an use case where all service prefixes want C1 as primary but have different fallback order.

This is achieved using Mapping Communities, as below:

  *   R1 advertised with M1 (Color:0:<C1BE>),

     *   maps to Resolution-Scheme1: Transport classes {C1, best-effort}

  *   R2 advertised with M2, (Color:0:<C1C2>) -

     *   maps to Resolution-Scheme2: Transport classes {C1, C2}

  *   R3 advertised with M3, (Color:0:<C1C3>) – Abstract value

     *   maps to Resolution-Scheme3: Transport classes {C1, C3}

Here C1BE, C1C2, C1C3 are abstract values and do not represent C1, C2 or C3 that can be easily agreed between different administrative domains.

NV> ------------End. }

Thx a lot,
R.

Juniper Business Use Only

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 24 Mar 2022 00:28:28 +0100

Hello Natrajan,

Thank you for the very detailed response.

By all means decoupling color from intent I find to be a positive feature.

But you did not comment about BGP interdomain vs IGP intradomain
inter-relation which IMHO is very important to get the full picture. Both
drafts for some reason skip this critical element and we all know that BGP
is always not congruent to IGP and actual transport.  (I do realize that
there are MSDC deployments which use BGP as IGP but I would not worry about
those for now).

- - -

Also what I find a bit disturbing in both solutions is that instead of
allowing applications to measure and choose how their packet should be
transported end to end by exposing multiple paths to it, you assume that
the network itself will take the endless application requirements and offer
dozens of proper paths to carry them. I am personally not very comfortable
with this fundamental assumption/approach and do believe in end to end path
selection in real time.

Again if we look outside DCs colorful transport will likely be supported
and used across single transit providers. And my view is that using single
transit (especially in the era of SDWAN and NaaS) is never a good thing.

Best,
Robert

PS. Apologies for trying to look outside of the box.

---

From: Gyan Mishra <[email-masked]>
Date: Wed, 23 Mar 2022 13:40:58 -0400

Hi Robert

The concept of coloring has existed for decades with MPLS, we just didn't
call it "colors" until now with the advent of Segment Routing SR-TE & Flex
Algo, as we called it different VPN SLO's with various QOS policies for
customer SLA Gold, Bronz, Silver classes.

The major difference as Nat pointed out is that CT now acts at two layers,
both the transport  underlay layer and service layer, whereas CAR acts only
at the service layer with its color resolution recursion schema resolving
the service routes to transport SR-TE, Flex Alog, BGP-LU or CAR.  (I
believe CAR can recurse to itself)

Transport Layer = Color (Transport or Topmost label)
The Underlay transport layer with the CT transport RIB which you can think
of as "inter-as option-b" segmented LSP stitching, and that is where the
transport underlay per domain can be stitched so we can have a different
color per domain and use the CT mapping community for next hop resolution.
So the mapping community as NAT pointed out has the color rewrite has to
happen at the inter-as domain boundaries similar to Inter-as opt-B
segmented LSP next-hop-self which terminates the LSP between domains.
(This is where CAR utilizes the local color mapping LCM community at the
inter-as or domain boundaries)

Service Layer = Intent (VPN Service label - BOS)
With CT the Service Layer is the end to end LSP  Inter-AS option C style
next-hop unchanged, thus no rewrite to the service route mapping community
as they are uniquely identified by the resolution scheme at the transport
layer which is a sequential order of colors and not the individual color.

Kind Regards

Gyan

-- 

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*Email [email-masked] <[email-masked]>*

*M 301 502-1347*

---

From: Natrajan Venkataraman <[email-masked]>
Date: Thu, 24 Mar 2022 04:41:04 +0000

Hi Robert,

Please find my replies marked with NV inline.

-Nats-

From: Robert Raszuk <[email-masked]>
Date: Wednesday, March 23, 2022 at 4:27 PM
To: Natrajan Venkataraman <[email-masked]>
Cc: Jeffrey Haas <[email-masked]>, Kaliraj Vairavakkalai <[email-masked]>, [email-masked] <[email-masked]>
Subject: Re: [Idr] Color domains in BGP routes with color
[External Email. Be cautious of content]

Hello Natrajan,

But you did not comment about BGP interdomain vs IGP intradomain inter-relation which IMHO is very important to get the full picture. Both drafts for some reason skip this critical element and we all know that BGP is always not congruent to IGP and actual transport.  (I do realize that there are MSDC deployments which use BGP as IGP but I would not worry about those for now).

NV-------> { Start …

Example to classify and inter-relate IGP tranport tunnels using CT constructs in intra-as domains
-------------------------------------------------------------------------------------------------

CT Constructs:

Transport-Class - represents a color
Transport-rib (Transport class resolution database) - routes of any protocol belonging to a particular transport class

Best Effort resolutions for service routes happen in - inet.3
Transport class specific resolutions for service routes happen in - <transport-class-name>.inet.3

The transport class resoluton databases CAN be organized for resolution in a rib-like fashion (Implementation Specific)
The below shows how Juniper realizes the transport RIB as a 4271 routing table and performs only resolutions on it.
This is similar to how juniper used to do Best Effort resolution to tunnels in inet.3 for BGP Services.

CT family does not need to be configured for INTRA DOMAIN. The above constructs provided by CT are enough to
achieve transport classification. CT family is required for INTER-DOMAIN use cases ONLY.

Representative example below:

RIB INET.0                              GOLD.INET.3 (100)
INET-RT1, PNH: PE2 Color:0:100-------> PE2 -> RSVP TUNNEL GOLD INGRESS (Active)
                                               FLEX ALGO TUNNEL GOLD INGRESS

INET-RT1, PNH: PE2 Color:0:200
                             |              BRONZE.INET.3 (200)
                             +----------->. PE2 -> RSVP TUNNEL BRONZE INGRESS (Active)
                                                   FLEX ALGO TUNNEL BRONZE INGRESS

INET-RT3, PNH: PE2
                |                         INET.3 (Best Effort)
                +--------------------->   PE2 ->  RSVP TUNNEL INGRESS (Active)
                                                  FLEX ALGO TUNNEL INGRESS

FIB.INET.0

INET-RT1->INDIRECTION->RSVP TUNNEL GOLD INGRESS
INET-RT2->INDIRECTION->RSVP TUNNEL BRONZE INGRESS
INET-RT3->INDIRECTION->RSVP TUNNEL INGRESS

To illustrate the inter-domain to intra-domain inter-relation is achived,
let us consider the following Figure.

                                     Gold = 100 Bronze = 200
                                       |-------RSVP-TE-----|
                            +--------[ASBR2     (AS2)     PE2]--+
    Gold = 100 Bronze = 200 |                                   |
    [PE1      (AS1)     ASBR1]                                 CE1 (1.1.1.1)
      |--COL-SR-TUNNEL---|  |                                   |   Multihomed onto PE2,PE3
                              +--------[ASBR3     (AS3)     PE3]--+
                                         |-----FLEX-ALGO-----|
                                     Gold = 200 Bronze = 100

In the above,

AS1 uses COLOR-SRTE for intra-domain tunnels (colored color:ep)
AS2 uses RSVP-TE for intra-domain tunnels (non-colored only ep)
AS3 uses FLEX-ALGO for intra-domain tunnels (colored: Algo-ID)

where,

AS1 and AS2 share the same color values whereas
AS3 has interchanged its color values.

The nodes of interest are

- PE1 (Service Node in AS1),
- ASBR2 and ASBR3 (Border Nodes in AS2 and AS3 respectively)

Intra-domain CT inter-relation with IGP transport in AS2 and AS3 are shown below.

AS2: (Mapping to Non-Colored RSVP)
----------------------------------

ASBR2:

Resolution Scheme Gold-Strict {
      Mapping community: Color:0:100
      Transport-class: Gold
      Transport-rib(Transport Class Resolution Database]: Gold.inet.3
}

Resolution Scheme Bronze-Strict {
      Mapping community: Color:0:200
      Transport-class: Bronze
      Transport-rib(Transport Class Resolution Database]: Bronze.inet.3
}

Configured RSVP tunnels:
RSVP tunnels are not colored and are always installed in Best Effort Resolution Database (inet.3)
A configuration knob has been added to specify color as a property of the tunnel to install the EP as part of Gold.inet.3.

MPLS LSP Tunnel to_PE2_Gold {
      to PE2;
      transport-class Gold; <<<<<
}

PCE created RSVP tunnels:
A draft with an extention to PCE for carrying color has been published parallely by one of the co authors of BGP-CT for
helping dynamically created RSVP LSPs from PCE.

With the above, The RSVP tunnels become part of the Gold Transport-rib resolution table.
Service routes with extended mapping community Color:0:100 can now resolve over a gold RSVP tunnel.

AS3: (Mapping to FLEX-ALGO colored by AlgoID)
---------------------------------------------

ASBR3:

Resolution Scheme Gold-Strict {
      Mapping community: Color:0:200
      Transport-class: Gold
      Transport-rib(Transport Class Resolution Database]: Gold.inet.3
}

Resolution Scheme Bronze-Strict {
      Mapping community: Color:0:100
      Transport-class: Bronze
      Transport-rib(Transport Class Resolution Database]: Bronze.inet.3
}

Configuring Flex-Algo Tunnels:
Color in Flex-Algo is identified by Algo ID. Each algo ID can be mapped to a specific Transport Class in CT.
For flex algo the value for the extended mapping community can be directly configured as part of flex-algo config

flex-algorithm 130 {
      definition {
          spf
            metric-type <metric-type>
            priority <priority>
      }
      color 200; <<< // <Gold extended mapping community value>
}

The above ensures all Gold Flex Algo tunnel Ingress are installed in Gold transport-rib resolution table
Service routes with extended mapping community Color:0:100 can now resolve over a gold FLEX-ALGO tunnel.

AS1: (Mapping to COLOR SRTE with Color:EP)
------------------------------------------

PE1:

Resolution Scheme Gold-Strict {
      Mapping community: Color:0:100
      Transport-class: Gold
      Transport-rib(Transport Class Resolution Database]: Gold.inet.3
}

Resolution Scheme Bronze-Strict {
      Mapping community: Color:0:200
      Transport-class: Bronze
      Transport-rib(Transport Class Resolution Database]: Bronze.inet.3
}

Configuring Colored SRTE Tunnels:
Color SR tunnels are identified by Color:EP value where Color represents the color of the tunnel. An SRTE color
can be mapped to use CT extended mapping community as follows. Now the color SRTE routes will be installed in
Gold Transport-rib.

source-routing-path to_PE2_SRTE_Gold {
      to PE2
      color 100 <<<< (Gold)
    use-transport-class <<<< (Maps it to CT extended mapping community 100)
}

I have not dicussed interdomain scenarios showcasing CT route origination and propagation as it is outise the scope of this question. However, let me know if you need those inputs as well.

NV-------> End.}

- - -

Also what I find a bit disturbing in both solutions is that instead of allowing applications to measure and choose how their packet should be transported end to end by exposing multiple paths to it, you assume that the network itself will take the endless application requirements and offer dozens of proper paths to carry them. I am personally not very comfortable with this fundamental assumption/approach and do believe in end to end path selection in real time.

NV-------> { Start …

I hope I am understanding your question right and I will try to explain how BGP-CT supports it. BGP-CT provides a resolution table any.inet.3 (which may to the special transport-class “any”) for the service to be able to resolve over any color. BGP-CT supports QoS where a service can be resolved to an indexed set of different colored tunnels + best-effort tunnel based of the class of service bits that a particular application is marking in the packet where each forwarding-class maps to a transport-class. In this case, the tunnel will be chosen at real-time by the application by virtue of the QoS classification it made. This does provide real-time end-to-end path selection. I can describe the details here. But I will save it for another email since there is already too much data to consume for the first topic.

NV-------> End.}

Again if we look outside DCs colorful transport will likely be supported and used across single transit providers. And my view is that using single transit (especially in the era of SDWAN and NaaS) is never a good thing.

NV-------> { Start …

Could you please be more specific on what you mean by “single transit”?

NV-------> End.}

Best,
Robert

On Wed, Mar 23, 2022 at 11:42 PM Natrajan Venkataraman <[email-masked]<mailto:[email-masked]>> wrote:
Hi Robert,

Regards,
-Nats-

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 24 Mar 2022 04:31:42 -0400

Kaliraj:

As Jeff mentioned in his original post, the text from section 10 could use
some clarifying.   
The text below does not focus the reader on remapping color.  
An example of re-mapping in the text would help clarify  
the lack of collisions.  

Thanks, 

Sue

extended community.

collisions for color intent can't happen with the encoding.  The effective
color is always the color present and may be swapped at a >color domain
boundary.  

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Jeffrey Haas
Sent: Tuesday, March 22, 2022 3:26 PM
To: [email-masked]
Subject: [Idr] Color domains in BGP routes with color

Working Group,

---

LCM is added or updated when routes cross between color domains. (Section
2.8)

The remaining problematic scenario would involve the case where E is anycast
and (E,C) was originated by multiple color domains, C has different intent
across those domains.  In such cases, even though a given (E, C overidden by
LCM) may have a different resolution behavior implementing the intent, BGP
route selection may prune one of the paths from use.

---

The procedure is described in section 10 under "Coordinating between domains
using different community namespaces".

-- Jeff

---

From: <[email-masked]>
Date: Thu, 24 Mar 2022 08:54:34 +0000

Hi Jeff,

First of all, thank you for your efforts to analyze the solution space and structure the analysis with key questions.

Please see 1 minor comment inline [Bruno]

Orange Restricted

> From: Idr <[email-masked]> On Behalf Of Jeffrey Haas
> Sent: Tuesday, March 22, 2022 8:26 PM
> To: [email-masked]
> LCM)

[Bruno] I agree with you on this problematic scenario (and also on the previous non-problematic scenarios).

IMHO it may certain happen but using anycast IP across different domains, I would call it a corner case.

I think it's also worth mentioning that E is a public name space (public IP address). So sub-cases are:
- if both domains do not cooperate, this scenario can't happen (you don't use some else IP address)
- if we have a "single" administrative domains (or domains fully cooperate), we would choose coherent colors. 
- if both domains cooperate to provide a coherent inter-domain anycast routing on one agreed upon public address, plus coordinate to provide inter domain color routing, how much of an additional burden to coordinate on the color(s) for this specific anycast route? (I  think forwarding states scales as the number of (E,C), not the total number of colors. Hence if needed, using a specific color for this inter-domain anycast address seems doable)

Again, above is specific to the inter-domain IP anycast example. I'm not trying to dismiss the general problem (multiple color domains)

--Bruno
 
> the subject "BGP CAR - multiple color domains"

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 24 Mar 2022 06:13:52 -0400

Bruno,

> IMHO it may certain happen but using anycast IP across different domains, I would call it a corner case.

I absolutely agree this is a corner case.

My attempt was to be thorough rather than specifically try to call out
the scenario as common or not.  We've both seen cases where the rare corner
case hasn't been accounted for and ends up being someone's Important Thing.
:-)

> I think it's also worth mentioning that E is a public name space (public IP address). So sub-cases are:

E may be fully private.  I think the relevant property is that it's a
routeable space that the participating domains agree on.  

> - if both domains do not cooperate, this scenario can't happen (you don't use some else IP address)

Agreed.

> - if we have a "single" administrative domains (or domains fully cooperate), we would choose coherent colors. 

Agreed.

> Again, above is specific to the inter-domain IP anycast example. I'm not trying to dismiss the general problem (multiple color domains)

One way to look at the issue is that with modest coordination, we have none
of these issues.  However, when you have cooperating parties that built
their networks without such cooperation originally, you have an issue of
resolving conflicts.  

No one likes renumbering.  This is just a flavor of that headache.

-- Jeff

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 24 Mar 2022 07:16:38 -0400

Robert: 

When (E2,C2) adds LCM-EC (C1) – do you consider this flat or a tuple?  

(see page 10 section 2.8 of car] 

I cannot find in the CAR text what happens if multiple LCM-EC are posted.

(See section 2.9.3 and section 2.8) 

For example, (E,C2) has LCM-EC(C1) and LCM-EC(C2).  

Did you find any normative text that restricts to a single LCM-EC? 

Cheers, Sue 

From: Idr [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Wednesday, March 23, 2022 8:31 AM
To: Jeffrey Haas
Cc: idr@ietf. org
Subject: Re: [Idr] Color domains in BGP routes with color

All,

Thx a lot,

R.

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 24 Mar 2022 08:09:43 -0400

> On Mar 24, 2022, at 7:16 AM, Susan Hares <[email-masked]> wrote:

This was replied to by Dhananjaya:
https://mailarchive.ietf.org/arch/msg/idr/734Fj97YIPIeSF2BEnMGtidMoz8/ <https://mailarchive.ietf.org/arch/msg/idr/734Fj97YIPIeSF2BEnMGtidMoz8/>

-- Jeff

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 24 Mar 2022 10:13:36 -0400

Dhananjaya: 

My question posted here: 

https://mailarchive.ietf.org/arch/msg/idr/Rm_pnETWbt1gy2kxoNZMmc8BtGY/

ask about the error handling if LCM Extended community is listed twice in
the PDU. 

My question is not the larger question on route resolution that

Jeff asked you here:  

https://mailarchive.ietf.org/arch/msg/idr/734Fj97YIPIeSF2BEnMGtidMoz8/

Below are excerpts that lead me to this position.  

Please let me know if I missed something in your text. 

Cheers, Sue 

Excerpts from 

  https://mailarchive.ietf.org/arch/msg/idr/734Fj97YIPIeSF2BEnMGtidMoz8/

> DR## As you phrased it, LCM color is used when the route is used for
resolution. Just to restate, there is no rewrite of the BGP NLRI color as it
traverses hops.

[Jeff]    Thanks.  The property I'm trying to ensure we're clear on is that
the RIB

[Jeff]    key, (E,C) is not necessarily the resolution key (E, effective
color).  The

[Jeff]    effective color will be the C from the NLRI unless an LCM is
present.

DR>> Yes. Section 2.8 of the BGP CAR draft states this along with the
rationale.

Excerpts from draft-dskc-bess-bgp-car-03.txt in 

Section 2.8 states a set of procedures: 

   o  The nodes within the receiving domain 1 use the local color
      encoded in the LCM-EC for next-hop resolution and BGP CAR route
      installation
..
   o  In the rare case of color incongruence, the local color encoded in
      LCM-EC takes precedence
 
:"Section 2.9 describes the procedures of this draft with the following
text: 
:
:  When a CAR route crosses the originator color domain's boundary, LCM
:   EC is added.  LCM EC conveys the local color mapping for the intent
:  (e.g. low latency) into transit or remote color domains.
 
:  The LCM EC MAY be used for filtering of BGP CAR routes and/or for
:  applying routing policies for the intent, when present.
 
It does not specify whether the LCM Extended community can be listed twice
in the PDU. 
I also do not find the text in 2.10 on fault management. 

From: Jeffrey Haas [mailto:[email-masked]] 
Sent: Thursday, March 24, 2022 8:10 AM
To: Sue Hares
Cc: Robert Raszuk; idr@ietf. org
Subject: Re: [Idr] Color domains in BGP routes with color

https://mailarchive.ietf.org/arch/msg/idr/734Fj97YIPIeSF2BEnMGtidMoz8/

-- Jeff

---

From: <[email-masked]>
Date: Wed, 30 Mar 2022 13:59:27 +0000

Jeff,

Please see inline 

> From: Jeffrey Haas <[email-masked]>
> Sent: Thursday, March 24, 2022 11:14 AM

Sorry for the delay (IETF week) and having everyone lose context.

> routeable space that the participating domains agree on.

Agreed: E may be private and this does not change the picture (both/all ASes agreed on using that private address)

> > - if both domains cooperate to provide a coherent inter-domain anycast routing
> on one agreed upon public address, plus coordinate to provide inter domain color
> routing, how much of an additional burden to coordinate on the color(s) for this
> specific anycast route? (I  think forwarding states scales as the number of (E,C),
> not the total number of colors. Hence if needed, using a specific color for this
> inter-domain anycast address seems doable)
> >
> > Again, above is specific to the inter-domain IP anycast example. I'm not trying to
> dismiss the general problem (multiple color domains)
> 

I'm not certain to follow what "modest coordination" is intended to mean.
Step 0 (before anycast): Assuming 2 domains with zero coordination, already using color with  two color domains.
Step 1 (enabling anycast): When they choose to provide anycast across those two domains, they need to coordinate for this and agree on an IP prefix for this. At this point, they could also agree on the color to use for this anycast service. If needed they can pick a color value unused in both ASes (and both AS can resolve this common color using their own color schema and their own intra-domain protocol). It looks to me that this is (better) described in https://datatracker.ietf.org/doc/html/draft-dskc-bess-bgp-car-03#appendix-B.2

So even in this corner case (inter-domain anycast service) it looks to me that two paths having the same NLRI key (E,C) are indeed two paths for the same FEC (Forwarding Equivalent Class) hence regular routing applies (ECMP, path hiding, BGP convergence, rerouting, protection...).

To me this is different for VPN where there is no coordination between VPN/customers, and hence RD are introduced to have identical IP prefix becoming different NLRI key/FEC.

Orange Restricted