Subject: [Idr] BGP Classful Transport Planes

From: Robert Raszuk <[email-masked]>
Date: Sat, 17 Oct 2020 17:30:11 +0200

Hi,

I have read
https://tools.ietf.org/html/draft-kaliraj-idr-bgp-classful-transport-planes-01
and have few questions & observations.

*1.*

> A domain boundary is demarcated by a rewrite of BGP nexthop to 'self'
while
> readvertising tunnel routes in BGP.

For advertising tunneling in BGP we already have a number of options.
Starting from RFC5512 through draft-ietf-idr-tunnel-encaps-19 or
https://tools.ietf.org/html/draft-ietf-intarea-tunnels-10

Q: Do we really need more signalling extensions instead of working with
existing protocols to accommodate new applications ?

*2. *

> The path uses MPLS label-switching when crossing domain boundary

So if the network for any reason does not use MPLS in its data plane - what
you are proposing here simply is not going to work. Just for clarity MPLS
can be used for service plane demux while transport can still be pure IP.

In contrast if you are carrying your services with SRv6 across your domains
you actually have Seamless SR for free without yet another BGP SAFI and
stack of tunnels.

*3. *

>  Overlay routes carry sufficient indication of the Transport Classes they
should be encapsulated over,
> in form of BGP community called the "mapping community".

My feedback here is that granularity to color routes and group routes into
transport classes is not sufficient.

If I have a cluster of compute servers which I advertise as a single
address block I must have ability to define and disseminate flows in/out
such cluster on a per application basis (ideally per flow 5 tuple).

At min per dst port granularity would be must have. Just doing the dst
based ip lookup and based on that placing the switching via proper color of
inet.3 RIB will just not cut it.

*4.*

> Based on the mapping community, "route resolution" procedure on the
ingress node selects from the
> corresponding Transport Class an appropriate tunnel whose
destination matches (LPM) the nexthop
> of the overlay route.

Assume I am using inter-as option C across my domians.  So VPNv4 AF carries
service routes with remote PEs next hops.

Can you elaborate or better update the draft with a new section describing
in detail the relation between transport next hops, border nodes and
service next hop for a given service route across 3 domains ?

For example what is not stated in the draft - if LPM match is sufficient
here do you depend on BGP-CT to remove the advertisements to remote overlay
tunnel endpoints when such endpoint fails ?

Then how about not complete failure but partial service degradation (for
example increased fabric drops) ? How do you intend to detect those and
automate say "gold" class bypass around such BNs ?

*5.*

> baby carrier

Nice !

Never used such term when describing CSC to customers  :-)

Cheers,
Robert.

---

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Sat, 17 Oct 2020 23:47:12 +0000

Hi Robert, thanks for the comments. Please see inline.

From: Robert Raszuk <[email-masked]>
Date: Saturday, October 17, 2020 at 8:30 AM
To: Kaliraj Vairavakkalai <[email-masked]>, Natrajan Venkataraman <[email-masked]>, Balaji Rajagopalan <[email-masked]>
Cc: "idr@ietf. org" <[email-masked]>
Subject: BGP Classful Transport Planes

[External Email. Be cautious of content]

Hi,

I have read https://tools.ietf.org/html/draft-kaliraj-idr-bgp-classful-transport-planes-01<https://urldefense.com/v3/__https:/tools.ietf.org/html/draft-kaliraj-idr-bgp-classful-transport-planes-01__;!!NEt6yMaO-gk!Xg6xfSibwETZhsptdOJv2QL0vVF8WVkztqeHhepYnVCyfQQAMILyPEH9JnXnhQDO$>
and have few questions & observations.

1.

> A domain boundary is demarcated by a rewrite of BGP nexthop to 'self' while

For advertising tunneling in BGP we already have a number of options. Starting from RFC5512 through draft-ietf-idr-tunnel-encaps-19 or https://tools.ietf.org/html/draft-ietf-intarea-tunnels-10<https://urldefense.com/v3/__https:/tools.ietf.org/html/draft-ietf-intarea-tunnels-10__;!!NEt6yMaO-gk!Xg6xfSibwETZhsptdOJv2QL0vVF8WVkztqeHhepYnVCyfQQAMILyPEH9Jl8x4R7Q$>

Q: Do we really need more signalling extensions instead of working with existing protocols to accommodate new applications ?

This is not an extension parallel to TEA. The above para means to say, when BGP speaker does nexthop-self on a advertised transport-route, the details of the intra-domain tunnel used in the advertising-domain is hidden from the BGP-speaker in the other domain receiving the route. Because it sees as nexthop only the advertising BGP-speaker.

Such nexthop-self happens in

  *   option-B VPNs, at the Service-ASBRs  (BGP-L3VPN families)
  *   option-C VPNs, at the Transport-ASBRs or BGP ABRs (BGP-LU and BGP-CT are examples of this)

So this act of doing nexthop-self demarcate the boundary between two domains. That boundary either link on an inter-AS link, or an ABR.

May be I should clarify the statement to use “transport-route” instead of tunnel-route. Will that make it clear?

2.

> The path uses MPLS label-switching when crossing domain boundary

So if the network for any reason does not use MPLS in its data plane - what you are proposing here simply is not going to work. Just for clarity MPLS can be used for service plane demux while transport can still be pure IP.

It will work. There are two cases here:

  *   intra-AS non-MPLS-tunnel.

As explained above, details of what type of intra-AS tunnel is used is confined to the tunnel boundary nodes. And we can encapsulate MPLS-traffic on other type of tunnels. So this will work.

  *   Inter-AS non-MPLS data-plane.

Yes, in this document we take the case of inter-AS link running MPLS data-plane. Because that is the most prevalent deployment today. But we have put thought into how the mechanisms prescribed in this document can work with non-MPLS data-planes also. But they are out of scope of this document.

In brief, instead of doing MPLS label-swap on the inter-AS link, we can do plain IPv6 forwarding. We may have a separate document in future, that will explain how mechanisms in this document adapt to SRv6 data-plane.

In contrast if you are carrying your services with SRv6 across your domains you actually have Seamless SR for free without yet another BGP SAFI and stack of tunnels.

   Supporting inter-AS MPLS data-plane has the following advantages:

  *   Uses and builds on top of deployment experience that we have gained thru the years.
  *   Preserves returns on investment made in existing deployments. i.e. Brown-field networks.
  *   Scales better. Because MPLS allows us to do build network hierarchies with nexthop chaining and label-stacks.

   But yes, if a green-field network decides to use IPv6 end-to-end, there are two ways it can be done:

  *   Use IPv6 for control-plane, and MPLS in the data-plane.
  *   Use IPv6 in the data-plane as-well. Even in this case, association of IPv6-tunnel endpoints with appropriate transport-class will be required. Which can be achieved with BGP-CT.

In summary, mechanisms described in BGP-CT are agnostic to

  *   What type of tunneling technology is used in the intra-AS transport-layer
  *   And what type of data-plane is used on the inter-AS links. Though the draft uses MPLS.

I will clarify the same in this draft. That, this draft uses MPLS as the defacto inter-AS data-plane, but the mechanisms will work with non-MPLS data-planes also. Future documents will specify details on how.

3.

>  Overlay routes carry sufficient indication of the Transport Classes they should be encapsulated over,

My feedback here is that granularity to color routes and group routes into transport classes is not sufficient.

If I have a cluster of compute servers which I advertise as a single address block I must have ability to define and disseminate flows in/out such cluster on a per application basis (ideally per flow 5 tuple).

At min per dst port granularity would be must have. Just doing the dst based ip lookup and based on that placing the switching via proper color of inet.3 RIB will just not cut it.

Please note, BGP-Flowspec routes can be colored appropriately to achieve what you describe here. Flowspec becomes just another service-family that can resolve over transport-classes created by BGP-CT.

So, you could have an inet-unicast service-route advertised for the ‘cluster of compute servers’ with desired color mapping-community.

And further, inet-flowspec route with the 5-tuple can be advertised, with redirect-to-ip nexthop and color mapping community. Such that any flow mathing this flowspec route is redirected using the transport-class, that the color mapping community maps to.

Does that address this comment?

If yes, I can add some text to the draft describing the flowspec use-case.

4.

> Based on the mapping community, "route resolution" procedure on the ingress node selects from the
> corresponding Transport Class an appropriate tunnel whose destination matches (LPM) the nexthop
> of the overlay route.

Assume I am using inter-as option C across my domians.  So VPNv4 AF carries service routes with remote PEs next hops.

Can you elaborate or better update the draft with a new section describing in detail the relation between transport next hops, border nodes and service next hop for a given service route across 3 domains ?

Sure. We will add text to the draft describing this. in brief,

At the Transport BNs:

  *   It will have BGP-CT route for remote-PE/32 in gold transport-RIB.
  *   This BGP-CT route will be marked un-usable, if there is no routes in the gold transport-RIB matching remote-PE/32
  *   It Will create mpls swap-routes when readvertising the BGP-CT route.

At the Ingress SN:

  *   It will have the BGP-CT route for remote-PE/32 route in gold transport-RIB, when gold-connectivity exists end to end.
  *   Service-routes carrying gold mapping-community will resolve using this gold transport-RIB route.

For example what is not stated in the draft - if LPM match is sufficient here do you depend on BGP-CT to remove the advertisements to remote overlay tunnel endpoints when such endpoint fails ?

Yes. The BGP-CT routes for a gold transport-class will fail resolving it’s nexthop if no gold tunnels exist. So, BGP-CT will not advertise it further. This is similar to the rules followed by any BGP-family regarding nexthop resolution.

Then how about not complete failure but partial service degradation (for example increased fabric drops) ? How do you intend to detect those and automate say "gold" class bypass around such BNs ?

BGP-CT will depend on the individual intra-domain transport-tunneling technologies to monitor such metrics, and classify whether ‘gold reachabilty’ exists to the nexthop in question. BGP-CT provides a way to collect such information procured from various intra-domain tunnels, and stich it across multiple domains and give an end to end picture to the service routes. How exactly such parameters are monitored will be left to the purview of the individual tunneling technologies.

e.g. RSVP or SRTE can monitor such parameters at nodes along the path, and give feedback to the RSVP/SRTE ingress node, which either adjusts the ingress route in gold transport-RIB, or takes down the gold transport-route. Which will trigger appropriate response in the BGP-CT layer.

5.

Nice !

Thanks! :)

Once we reach consensus on the various points discussed in this thread, we will update the draft.

Balaji, Natz, Shradha, please add if anything.

Thanks,
Kaliraj

Juniper Business Use Only

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 18 Oct 2020 13:38:53 +0200

Hi Kaliraj,

Just a few points as a follow up on your note.

Ad 1 - The point was that you can carry mapping (or if you will stitching
information) by extending with new draft encoding defined in
draft-ietf-idr-tunnel-encaps.

As you know tunnel SAFI (5512) was deprecated and going forward the
recommendation is to use a new encap attribute. That is why I suggest you
consider the use of a new attribute attached to RFC3107 SAFI 4 instead of
defining new SAFI.

Ad 2a - You say "it will work" then just a few lines below you admit the
current draft only addresses MPLS stitching via ASBR/ABRs.

Ad 2b - All three points on value of MPLS Inter-AS or Inter-area
stitching are quite questionable even in brownfield deployment. But that's
not the point to debate any more here. Market will decide.

Ad 3 - No I was not asking for yet one more abuse of flowspec use case. I
was not asking for new ACL based mapping to specific VRF either. If your
draft is about "BGP Classful Transport" it better also define "Classful
RIB" where forwarding decision is done not only based on destination
address and its next hop but also on other fields in the packet. That was
the first thing I was curious to see in your new SAFI definition but there
is none.

Ad 4 - I was asking where exactly is the Longest Prefix Match performed ?
Based on aggregates generated by which network element and which protocol ?
What set of destinations does the aggregate cover ? Assuming option-C don't
you still need to leak all /32s or /128s across the domains - or you assume
RFC5283 is in use ? I think it would help whoever is reviewing this work to
have an end to end illustration of how each SN/BN/RR hop service route and
its next hop changes as well as how underlay transport and overlay tunnels
are advertised at each BN with the use of real IP addresses and labels.
Even Shraddha's blog post does not go into that level of details.

Many thx,
Robert

On Sun, Oct 18, 2020 at 1:47 AM Kaliraj Vairavakkalai <[email-masked]>
wrote:

> *From: *Robert Raszuk <[email-masked]>
> *Date: *Saturday, October 17, 2020 at 8:30 AM
> *To: *Kaliraj Vairavakkalai <[email-masked]>, Natrajan Venkataraman <
> [email-masked]>, Balaji Rajagopalan <[email-masked]>
> *Cc: *"idr@ietf. org" <[email-masked]>
> *Subject: *BGP Classful Transport Planes
>
>
>
> *[External Email. Be cautious of content]*
>
>
>
> Hi,
>
>
>
> I have read
> <https://urldefense.com/v3/__https:/tools.ietf.org/html/draft-kaliraj-idr-bgp-classful-transport-planes-01__;!!NEt6yMaO-gk!Xg6xfSibwETZhsptdOJv2QL0vVF8WVkztqeHhepYnVCyfQQAMILyPEH9JnXnhQDO$>
>
> <https://urldefense.com/v3/__https:/tools.ietf.org/html/draft-ietf-intarea-tunnels-10__;!!NEt6yMaO-gk!Xg6xfSibwETZhsptdOJv2QL0vVF8WVkztqeHhepYnVCyfQQAMILyPEH9Jl8x4R7Q$>
>
>
>
> This is not an extension parallel to TEA. The above para means to say,
> when BGP speaker does nexthop-self on a advertised transport-route, the
> details of the intra-domain tunnel used in the advertising-domain is hidden
> from the BGP-speaker in the other domain receiving the route. Because it
> sees as nexthop only the advertising BGP-speaker.
>
>
>
>    - option-B VPNs, at the Service-ASBRs  (BGP-L3VPN families)
>    - option-C VPNs, at the Transport-ASBRs or BGP ABRs (BGP-LU and BGP-CT
>    are examples of this)
>
>
>
> So this act of doing nexthop-self demarcate the boundary between two
> domains. That boundary either link on an inter-AS link, or an ABR.
>
>
>
> So if the network for any reason does not use MPLS in its data plane -
> what you are proposing here simply is not going to work. Just for clarity
> MPLS can be used for service plane demux while transport can still be pure
> IP.
>
>
>
>    - intra-AS non-MPLS-tunnel.
>
>
>
> As explained above, details of what type of intra-AS tunnel is used is
> confined to the tunnel boundary nodes. And we can encapsulate MPLS-traffic
> on other type of tunnels. So this will work.
>
>
>
>    - Inter-AS non-MPLS data-plane.
>
>
>
> Yes, in this document we take the case of inter-AS link running MPLS
> data-plane. Because that is the most prevalent deployment today. But we
> have put thought into how the mechanisms prescribed in this document can
> work with non-MPLS data-planes also. But they are out of scope of this
> document.
>
>
>
> In brief, instead of doing MPLS label-swap on the inter-AS link, we can do
> plain IPv6 forwarding. We may have a separate document in future, that will
> explain how mechanisms in this document adapt to SRv6 data-plane.
>
>
>
>
>
> In contrast if you are carrying your services with SRv6 across your
> domains you actually have Seamless SR for free without yet another BGP SAFI
> and stack of tunnels.
>
>
>
>    - Uses and builds on top of deployment experience that we have gained
>    thru the years.
>    - Preserves returns on investment made in existing deployments. i.e.
>    Brown-field networks.
>    - Scales better. Because MPLS allows us to do build network
>    hierarchies with nexthop chaining and label-stacks.
>
>
>
>    - Use IPv6 for control-plane, and MPLS in the data-plane.
>    - Use IPv6 in the data-plane as-well. Even in this case, association
>    of IPv6-tunnel endpoints with appropriate transport-class will be required.
>    Which can be achieved with BGP-CT.
>
>
>
>    - What type of tunneling technology is used in the intra-AS
>    transport-layer
>    - And what type of data-plane is used on the inter-AS links. Though
>    the draft uses MPLS.
>
>
>
> I will clarify the same in this draft. That, this draft uses MPLS as the
> defacto inter-AS data-plane, but the mechanisms will work with non-MPLS
> data-planes also. Future documents will specify details on how.
>
>
>
> *3. *
>
>
>
> >  Overlay routes carry sufficient indication of the Transport
> Classes they should be encapsulated over,
>
> Please note, BGP-Flowspec routes can be colored appropriately to achieve
> what you describe here. Flowspec becomes just another service-family that
> can resolve over transport-classes created by BGP-CT.
>
>
>
> So, you could have an inet-unicast service-route advertised for the
> ‘cluster of compute servers’ with desired color mapping-community.
>
>
>
> And further, inet-flowspec route with the 5-tuple can be advertised, with
> redirect-to-ip nexthop and color mapping community. Such that any flow
> mathing this flowspec route is redirected using the transport-class, that
> the color mapping community maps to.
>
>
>
> Assume I am using inter-as option C across my domians.  So VPNv4 AF
> carries service routes with remote PEs next hops.
>
>
>
>    - It will have BGP-CT route for remote-PE/32 in gold transport-RIB.
>    - This BGP-CT route will be marked un-usable, if there is no routes in
>    the gold transport-RIB matching remote-PE/32
>    - It Will create mpls swap-routes when readvertising the BGP-CT route.
>
> At the Ingress SN:
>
>    - It will have the BGP-CT route for remote-PE/32 route in gold
>    transport-RIB, when gold-connectivity exists end to end.
>    - Service-routes carrying gold mapping-community will resolve using
>    this gold transport-RIB route.
>
>
>
> Yes. The BGP-CT routes for a gold transport-class will fail resolving it’s
> nexthop if no gold tunnels exist. So, BGP-CT will not advertise it further.
> This is similar to the rules followed by any BGP-family regarding nexthop
> resolution.
>
>
>
> BGP-CT will depend on the individual intra-domain transport-tunneling
> technologies to monitor such metrics, and classify whether ‘gold
> reachabilty’ exists to the nexthop in question. BGP-CT provides a way to
> collect such information procured from various intra-domain tunnels, and
> stich it across multiple domains and give an end to end picture to the
> service routes. How exactly such parameters are monitored will be left to
> the purview of the individual tunneling technologies.
>
>
>
> e.g. RSVP or SRTE can monitor such parameters at nodes along the path, and
> give feedback to the RSVP/SRTE ingress node, which either adjusts the
> ingress route in gold transport-RIB, or takes down the gold
> transport-route. Which will trigger appropriate response in the BGP-CT
> layer.
>
>
>
> *5.*
>
>
>
> > baby carrier
>
>
>
> Nice !
>
>
>

---

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Mon, 19 Oct 2020 02:26:32 +0000

Hi Robert,

Please see inline.

From: Robert Raszuk <[email-masked]>
Date: Sunday, October 18, 2020 at 4:39 AM
To: Kaliraj Vairavakkalai <[email-masked]>
Cc: Natrajan Venkataraman <[email-masked]>, Balaji Rajagopalan <[email-masked]>, "idr@ietf. org" <[email-masked]>, Shraddha Hegde <[email-masked]>
Subject: Re: BGP Classful Transport Planes

Hi Kaliraj,

Ad 1 - The point was that you can carry mapping (or if you will stitching information) by extending with new draft encoding defined in draft-ietf-idr-tunnel-encaps.

As you know tunnel SAFI (5512) was deprecated and going forward the recommendation is to use a new encap attribute. That is why I suggest you consider the use of a new attribute attached to RFC3107 SAFI 4 instead of defining new SAFI.

The new attribute attached to SAFI 76 (BGP-CT) route is a Route-Target extended-community. Ref: https://tools.ietf.org/html/draft-kaliraj-idr-bgp-classful-transport-planes-01#section-4

This RT helps this SAFI in following RFC-4364 mechanisms. I think it doesn’t make sense to use/extend TEA to perform RFC-4364 mechanisms (route leaking in BGP VPNs).

The ‘mapping community’ is just a new role for a bgp-community. Any community/extended-community can be used as a mapping-community. We re-use existing ext-community “Color extended community” as the mapping-community on service-routes.

A new SAFI is required because it carries RD and RT. RD is required to distinguish the NLRI of gold/silver reachability info for a PNH PE1/32. RT is required to associate the PNH route with a specific Transport-class and leak it to the associated Transport-VRF RIB (its just a Transport-layer BGP VPN).

These cannot be achieved using SAFI 4. Addpath-ID can help with path-hiding on a per-BGP-session scope, but it does not provide a proper end-to-end end-point ‘distinguisher’. RD is good for debugging, since looking at a RD1:PE1 route, one can figure out which node in the multi-domain option-C network originated this advertisement. AddPathID doesn’t help with that. Also, instead of redefining SAFI-4 to follow RFC-4364 and allow leaking using RT, it appeared much cleaner to define a new SAFI that uses RD, RT.

We could have overloaded SAFI-128 to carry transport-routes as-well. But separating it into a new SAFI made sense with respect to operational simplicity.

This is explained in https://tools.ietf.org/html/draft-kaliraj-idr-bgp-classful-transport-planes-01#section-9

Yes, defining a new SAFI means other vendors need to come on-board to implement this mechanism. We are hoping customers and fellow-vendors will see the value this proposal brings to brown-field deployments.

Ad 2a - You say "it will work" then just a few lines below you admit the current draft only addresses MPLS stitching via ASBR/ABRs.

Sorry, I must have said it ‘can be made to work’, as the architecture maintains clear separation between layers, and has room for extension. But yes, current draft only addresses MPLS forwarding at the border-nodes.

Ad 2b - All three points on value of MPLS Inter-AS or Inter-area stitching are quite questionable even in brownfield deployment. But that's not the point to debate any more here. Market will decide.

Agree, market will decide.

Ad 3 - No I was not asking for yet one more abuse of flowspec use case. I was not asking for new ACL based mapping to specific VRF either. If your draft is about "BGP Classful Transport" it better also define "Classful RIB" where forwarding decision is done not only based on destination address and its next hop but also on other fields in the packet. That was the first thing I was curious to see in your new SAFI definition but there is none.

OK. I think you are asking about CBF (Class based forwarding). IMHO, CBF is local implementation matter. So we didn’t specify anything in the draft about it. E.g. Junos will support mapping a ‘forwarding-class’ to a ‘transport-class’ by local policy configuration.

SAFI-76 itself is at the Transport-layer, so I feel it should not have any CBF related details, which are service-layer level information.

Ad 4 - I was asking where exactly is the Longest Prefix Match performed ? Based on aggregates generated by which network element and which protocol ? What set of destinations does the aggregate cover ? Assuming option-C don't you still need to leak all /32s or /128s across the domains - or you assume RFC5283 is in use ?

The LPM is performed on the transport-class specific transport-RIBs (e.g. gold.inet.3), which can contain routes from various transport-protocols.

Yes, the RD:PNH/32 or RD:PNH/128 routes will be leaked across all domains. There is no aggregation. And they will be collected in respective transport-ribs, e.g. gold.inet.3, bronze.inet.3 at the SN/BN nodes. Thus, e.g. When a route with mapping-community ‘gold’(Color:0:100) is received, to resolve it’s nexthop, we do the LPM in transport-RIB (gold.inet.3) associated with that transport-class. Routes in best-effort transport-rib (inet.3) will not be used. The RD:PNH/32 routes would have been leaked into this gold.inet.3, by virtue of carrying a RT: transport-target:0:100.

And, these transport-ribs can contain routes from various transport-protocols (RSVP, SRTE, BGP-CT, others). I hope I got your question right.

I think it would help whoever is reviewing this work to have an end to end illustration of how each SN/BN/RR hop service route and its next hop changes as well as how underlay transport and overlay tunnels are advertised at each BN with the use of real IP addresses and labels. Even Shraddha's blog post does not go into that level of details.

Sure, I will add details to the draft with specific example topology.

Thanks a bunch for taking time to review, and give comments.

Kaliraj

---

From: Gyan Mishra <[email-masked]>
Date: Mon, 19 Oct 2020 01:51:47 -0400

Hi Kaliraj,

I read the draft and had a few comments.

This draft creates a new SAFI 76 BGP-CT that parallels BGP-LU that is being
used to fix a deficiency with CSC as far as the transport label being used
as customer carrier Hierarchical VRF to the customer carrier customer VRF
and not in a separate transport rib.

The framework behind CSC is that the backbone carrier PE runs BGP LU to
customer carrier CE, and the transport routes are within SAFI 128 Customer
carrier VRF. The PE-CE backbone to customer carrier must MPLS enabled. The
Customer Carrier
would have a it’s own hierarchical customer VRFs that would be tunneled
across the backbone carrier.

The customer carrier hierarchical VRF to its customers are tunneled over
the backbone carrier.

The customer carrier transport routes are in a VRF SAFI 128 and don’t touch
the backbone carrier global table.  BGP LU is enabled on the eBGP
connection PE-CE for the transport label binding and is identical to
inter-as Opt B usage of BGP-LU and then the customer carrier transport
prefixes  routes sit in SAFI 128 eBGP VPN IPV4 VPN IPV6.

You mention in the draft inter-as opt B interchangeably with CSC which they
both are similar but very different.  However, I believe you are trying to
address deficiencies with only CSC.

As RFC 4364 CSC had been deployed for years by many carriers and it scales
very well.  I am trying to understand the issues and gap related to CSC.

So with this new SAFI the intent is to not have a hierarchical VRF and
instead this new BGP-CT Transport rib.

What is the impact to existing CSC deployments using hierarchical VRF.  How
would you address backwards compatibility.

Kind Regards

Gyan

On Sun, Oct 18, 2020 at 7:39 AM Robert Raszuk <[email-masked]> wrote:

>> *To: *Kaliraj Vairavakkalai <[email-masked]>, Natrajan Venkataraman
>> <[email-masked]>, Balaji Rajagopalan <[email-masked]>
>>    - option-C VPNs, at the Transport-ASBRs or BGP ABRs (BGP-LU and
>>    BGP-CT are examples of this)
>>
>>
>>
>> In brief, instead of doing MPLS label-swap on the inter-AS link, we can
>> do plain IPv6 forwarding. We may have a separate document in future, that
>> will explain how mechanisms in this document adapt to SRv6 data-plane.
>>
>>
>>
>>
>>
>> My feedback here is that granularity to color routes and group routes
>> into transport classes is not sufficient.
>>
>>
>>
>> Can you elaborate or better update the draft with a new section
>> describing in detail the relation between transport next hops, border nodes
>> and service next hop for a given service route across 3 domains ?
>>
>>
>>
>>    - This BGP-CT route will be marked un-usable, if there is no routes
>>    in the gold transport-RIB matching remote-PE/32
>> Yes. The BGP-CT routes for a gold transport-class will fail resolving
>> it’s nexthop if no gold tunnels exist. So, BGP-CT will not advertise it
>> further. This is similar to the rules followed by any BGP-family regarding
>> nexthop resolution.
>>
>>
>>
>> e.g. RSVP or SRTE can monitor such parameters at nodes along the path,
>> and give feedback to the RSVP/SRTE ingress node, which either adjusts the
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr
>
-- 

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*M 301 502-134713101 Columbia Pike *Silver Spring, MD

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 19 Oct 2020 09:27:06 +0200

Hi Kaliraj,

Thank you for more comments. Just to finish this little thread:

The new attribute attached to SAFI 76 (BGP-CT) route is a Route-Target
> extended-community.

Actually I am not sure if overloading RT is the best idea here. Have you
considered defining a new extended community instead ?

Sure you would have to update RTC in such a case, but this seems the only
issue.

Note on RTC ... today RTC dynamics are pretty static. You advertise RTs
which are locally configured for import. Here we are completely
changing this to also advertise RTs which are learned via VPNv4/v6 AF to
receive transport overlays. While I am not immediately saying this will
trigger more instability I think the dynamics between VPN routes itself and
signalling the need for requested colors for tunnels should be
differentiated. Since the draft does mention RTC it should at least give a
hint to implementers about this.

The ‘mapping community’ is just a new role for a bgp-community. Any
> community/extended-community can be used as a mapping-community. We re-use
> existing ext-community “Color extended community” as the mapping-community
> on service-routes.
>

+

SAFI-76 itself is at the Transport-layer, so I feel it should not have any
> CBF related details, which are service-layer level information.
>

My feedback is that per route coloring is not enough. At least per dst port
is needed.

Therefore if you are going via the effort to define a new classful
forwarding signalling my suggestion is to make it a bit more useful
in practical deployments.

Yes, the RD:PNH/32 or RD:PNH/128 routes will be leaked across all domains.
> There is no aggregation. And they will be collected in respective
> transport-ribs, e.g. gold.inet.3, bronze.inet.3 at the SN/BN nodes. Thus,
> e.g. When a route with mapping-community ‘gold’(Color:0:100) is received,
> to resolve it’s nexthop, we do the LPM in transport-RIB (gold.inet.3)
> associated with that transport-class.
>

Let's see if I understand your proposal here.

Today VPN next hops are advertised across all domains by OSPF. You are you
saying that not only those will be distributed by OSPF ... now 1000s of
them will also be duplicated and distributed by SAFI-76 as well ?

Today all of those advertised by OSPF along with labels carried by LDP or
3107 will be installed into inet.3 (mpls RIB + LFIB) to resolve VPN next
hops.

Now for transport class you construct (logically or physically - does not
matter) a new class.inet.3 RIBs and LFIBs and "leaking" VPN next hops from
SAFI-76 based on their RT membership ?

Is that right ? If so I am still not seeing much room for aggregation and
LPM there.

And, these transport-ribs can contain routes from various
> transport-protocols (RSVP, SRTE, BGP-CT, others). I hope I got your
> question right.
>

Feeding transport-ribs is based on the mapping-community. So now do you
expect all of those listed transport-protocols to also carry such
route target communities ?  If so IMO we clearly should not overload RT
semantics with this completely new role.

Many thx,
Robert.

---

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Mon, 19 Oct 2020 18:06:19 +0000

Hi Gyan,

No, there is no impact to existing CsC deployments.

This draft creates a new SAFI that parallels BGP-LU in option-C deployments. I will clarify this with an explicit mention to option-C in the introduction.

The deficiency being addressed is: In today’s option-C deployments, if multiple classes(gold, bronze, etc) of tunnels exist to a destination PE1 in one domain, they can not be extended out to the adjacent domains by using BGP-LU, such that service-traffic in those domains is able to choose a certain class of tunnel.

The new SAFI 76 carries NLRI of the form RDx:PE1, such that route for each class is carried using a distinct RD and doesn’t hide the other classes. And a new type of RT is used to denote the transport-class of the tunnel. This RT is used to leak the transport-routes into transport-RIBs of the same class on the receiving nodes – just like what L3VPN family does.

The note about similarity with CsC was just that: CsC uses SAFI 128 (L3VN) family to carry transport-prefixes in the mother carrier. Similarly, this SAFI 76(BGP-CT) also carries transport-routes in option-C deployments.

Hope that clears the confusion. Thanks for the comments.

Kaliraj

From: Gyan Mishra <[email-masked]>
Date: Sunday, October 18, 2020 at 10:52 PM
To: Robert Raszuk <[email-masked]>
Cc: Kaliraj Vairavakkalai <[email-masked]>, Natrajan Venkataraman <[email-masked]>, "idr@ietf. org" <[email-masked]>
Subject: Re: [Idr] BGP Classful Transport Planes

Hi Kaliraj,

This draft creates a new SAFI 76 BGP-CT that parallels BGP-LU that is being used to fix a deficiency with CSC as far as the transport label being used as customer carrier Hierarchical VRF to the customer carrier customer VRF and not in a separate transport rib.

The framework behind CSC is that the backbone carrier PE runs BGP LU to customer carrier CE, and the transport routes are within SAFI 128 Customer carrier VRF. The PE-CE backbone to customer carrier must MPLS enabled. The Customer Carrier
would have a it’s own hierarchical customer VRFs that would be tunneled across the backbone carrier.

The customer carrier hierarchical VRF to its customers are tunneled over the backbone carrier.

The customer carrier transport routes are in a VRF SAFI 128 and don’t touch the backbone carrier global table.  BGP LU is enabled on the eBGP connection PE-CE for the transport label binding and is identical to inter-as Opt B usage of BGP-LU and then the customer carrier transport prefixes  routes sit in SAFI 128 eBGP VPN IPV4 VPN IPV6.

You mention in the draft inter-as opt B interchangeably with CSC which they both are similar but very different.  However, I believe you are trying to address deficiencies with only CSC.

As RFC 4364 CSC had been deployed for years by many carriers and it scales very well.  I am trying to understand the issues and gap related to CSC.

So with this new SAFI the intent is to not have a hierarchical VRF and instead this new BGP-CT Transport rib.

What is the impact to existing CSC deployments using hierarchical VRF.  How would you address backwards compatibility.

Kind Regards

Gyan

On Sun, Oct 18, 2020 at 7:39 AM Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:
Hi Kaliraj,

Ad 4 - I was asking where exactly is the Longest Prefix Match performed ? Based on aggregates generated by which network element and which protocol ? What set of destinations does the aggregate cover ? Assuming option-C don't you still need to leak all /32s or /128s across the domains - or you assume RFC5283 is in use ? I think it would help whoever is reviewing this work to have an end to end illustration of how each SN/BN/RR hop service route and its next hop changes as well as how underlay transport and overlay tunnels are advertised at each BN with the use of real IP addresses and labels. Even Shraddha's blog post does not go into that level of details.

Many thx,
Robert

On Sun, Oct 18, 2020 at 1:47 AM Kaliraj Vairavakkalai <[email-masked]<mailto:[email-masked]>> wrote:
Hi Robert, thanks for the comments. Please see inline.

From: Robert Raszuk <[email-masked]<mailto:[email-masked]>>
Date: Saturday, October 17, 2020 at 8:30 AM
To: Kaliraj Vairavakkalai <[email-masked]<mailto:[email-masked]>>, Natrajan Venkataraman <[email-masked]<mailto:[email-masked]>>, Balaji Rajagopalan <[email-masked]<mailto:[email-masked]>>
Cc: "idr@ietf. org" <[email-masked]<mailto:[email-masked]>>
Subject: BGP Classful Transport Planes

Hi,

1.

Such nexthop-self happens in
·   option-B VPNs, at the Service-ASBRs  (BGP-L3VPN families)
·   option-C VPNs, at the Transport-ASBRs or BGP ABRs (BGP-LU and BGP-CT are examples of this)

2.

It will work. There are two cases here:
·   intra-AS non-MPLS-tunnel.

·   Inter-AS non-MPLS data-plane.

   Supporting inter-AS MPLS data-plane has the following advantages:
·   Uses and builds on top of deployment experience that we have gained thru the years.
·   Preserves returns on investment made in existing deployments. i.e. Brown-field networks.
·   Scales better. Because MPLS allows us to do build network hierarchies with nexthop chaining and label-stacks.

   But yes, if a green-field network decides to use IPv6 end-to-end, there are two ways it can be done:
·   Use IPv6 for control-plane, and MPLS in the data-plane.
·   Use IPv6 in the data-plane as-well. Even in this case, association of IPv6-tunnel endpoints with appropriate transport-class will be required. Which can be achieved with BGP-CT.

In summary, mechanisms described in BGP-CT are agnostic to
·   What type of tunneling technology is used in the intra-AS transport-layer
·   And what type of data-plane is used on the inter-AS links. Though the draft uses MPLS.

3.

4.

At the Transport BNs:
·   It will have BGP-CT route for remote-PE/32 in gold transport-RIB.
·   This BGP-CT route will be marked un-usable, if there is no routes in the gold transport-RIB matching remote-PE/32
·   It Will create mpls swap-routes when readvertising the BGP-CT route.

At the Ingress SN:
·   It will have the BGP-CT route for remote-PE/32 route in gold transport-RIB, when gold-connectivity exists end to end.
·   Service-routes carrying gold mapping-community will resolve using this gold transport-RIB route.

5.

Nice !

Thanks! :)

Thanks,
Kaliraj

Juniper Business Use Only
_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!U8HbQ5jZBlsxEu8405jmF-287HZ0Rs-PwJCc2URxQqPyf59kvSU9h1YXFcFZxEh2$>
--

[Image removed by sender.]<https://urldefense.com/v3/__http:/www.verizon.com/__;!!NEt6yMaO-gk!U8HbQ5jZBlsxEu8405jmF-287HZ0Rs-PwJCc2URxQqPyf59kvSU9h1YXFRulXXkX$>

Gyan Mishra

Network Solutions Architect

M 301 502-1347
13101 Columbia Pike
Silver Spring, MD

---

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Mon, 19 Oct 2020 22:06:47 +0000

Robert,

Please see inline for some responses.

Date: Monday, October 19, 2020 at 12:27 AM

Hi Kaliraj,

The new attribute attached to SAFI 76 (BGP-CT) route is a Route-Target extended-community.

Actually I am not sure if overloading RT is the best idea here. Have you considered defining a new extended community instead ?

Yes it is a new extended-community Format/Type. Section 4 describes it:

"Transport Class" Route Target extended community is a transitive
extended community EXT-COMM [RFC4360] of extended-type, with a new
Format (Type high = 0xa) and SubType as 0x2 (Route Target).

We did this, so that there is no collision of these Transport-plane RTs with currently deployed service-plane RTs. If I understand you right, this is what you are suggesting as-well.

Sure you would have to update RTC in such a case, but this seems the only issue.

RTC seems to work with any Route-Target (extended-community with SubType 0x2). We can update RTC to specify that if required.

Note on RTC ... today RTC dynamics are pretty static. You advertise RTs which are locally configured for import. Here we are completely changing this to also advertise RTs which are learned via VPNv4/v6 AF to receive transport overlays. While I am not immediately saying this will trigger more instability I think the dynamics between VPN routes itself and signalling the need for requested colors for tunnels should be differentiated. Since the draft does mention RTC it should at least give a hint to implementers about this.

About RTC dynamics, agree – it could be either via local-configuration or a feedback from VPN service-families, to request SAFI-76 routes for the required PNH, Transport-class. This provides a ‘On Demand Nexthop’ mechanism for the ingress SNs.

Page 11, section “Constrained distribution of PNHs to SNs.” tries to explain this. I will add a note about the change in dynamics.
The ‘mapping community’ is just a new role for a bgp-community. Any community/extended-community can be used as a mapping-community. We re-use existing ext-community “Color extended community” as the mapping-community on service-routes.

+

My feedback is that per route coloring is not enough. At least per dst port is needed.

Therefore if you are going via the effort to define a new classful forwarding signalling my suggestion is to make it a bit more useful in practical deployments.

This is a new classful Transport signaling. I think there is a difference. Service-routes, used for Forwarding resolve over Transport-routes. So I still think service-plane information should not be carried in SAFI-76. But if such per-dest-port service/forwarding-routes is carried via some other mechanism, it can resolve over transport-classes built using SAFI-76. I will discuss your suggestion internally to check if I am missing something.
Yes, the RD:PNH/32 or RD:PNH/128 routes will be leaked across all domains. There is no aggregation. And they will be collected in respective transport-ribs, e.g. gold.inet.3, bronze.inet.3 at the SN/BN nodes. Thus, e.g. When a route with mapping-community ‘gold’(Color:0:100) is received, to resolve it’s nexthop, we do the LPM in transport-RIB (gold.inet.3) associated with that transport-class.

Today VPN next hops are advertised across all domains by OSPF. You are you saying that not only those will be distributed by OSPF ... now 1000s of them will also be duplicated and distributed by SAFI-76 as well ?

No. intra-AS distribution will not change. Intra-AS IGP or tunneling-protocol(e.g. RSVP) routes will remain in one domain. They don’t need to be advertised everywhere. That provides better resiliency, by decoupling the domains and confining blast-radius. We do BGP nexthop-self between these loosely coupled domains. That’s the environment we are addressing/presuming in this draft. I will perhaps mention that.

Only the intra-AS tunnel end-points which want to be advertised on inter-domain boundaries, they will be advertised in SAFI-76 with appropriate transport route-target to indicate their class. It is just like redistributing intra-AS tunnels into SAFI-4, here we redistribute them in SAFI-76. And they are received, re-advertised by BNs on adjacent domains, until they reach the ingress-node.

Today all of those advertised by OSPF along with labels carried by LDP or 3107 will be installed into inet.3 (mpls RIB + LFIB) to resolve VPN next hops.

Now for transport class you construct (logically or physically - does not matter) a new class.inet.3 RIBs and LFIBs and "leaking" VPN next hops from SAFI-76 based on their RT membership ?

That is right. However not only SAFI-76 routes, but the intra-AS tunneling-protocol routes (e.g. RSVP/SRTE) will also install ingress routes in class.inet.3. Those intra-AS tunneling protocols are transport-class aware at the ingress-node.

Is that right ? If so I am still not seeing much room for aggregation and LPM there.

OK, there will typically not be anything less specific than /32s in these Transport-RIBs – if that’s what you meant. Because they are tunnel ingress routes, they will be /32s. I just mean to say the nexthop lookup will happen in class.inet.3 instead of inet.3, such that the LPM will pick only a route that belongs to the desired class.
And, these transport-ribs can contain routes from various transport-protocols (RSVP, SRTE, BGP-CT, others). I hope I got your question right.

Feeding transport-ribs is based on the mapping-community. So now do you expect all of those listed transport-protocols to also carry such route target communities ?  If so IMO we clearly should not overload RT semantics with this completely new role.

Feeding transport-ribs is based on Transport Class Route-target. And no, non-BGP transport-protocols don’t carry this RT. They install routes in class.inet.3 by either local configuration (e.g. in RSVP case), or by virtue of mapping the color in their NLRI to a transport-class (SRTE case). E.g. SRTE color 100 maps to Transport-class ID 100. These RSVP/SRTE routes will be installed in class100.inet.3. When they are advertised out in SAFI-76, they will be advertised with Transport-Target:0:100, and on the receiving node, imported into Transport-RIBs class100.inet.3 which is provisioned to import this RT value.

Further, the SAFI-76 routes received in color100.inet.3 will be resolved using intra-AS RSVP/SRTE routes in color100.inet.3 in that receiving domain. Such that end to end, the transport-class 100 SLA is preserved.

Thanks
Kaliraj

---

From: Gyan Mishra <[email-masked]>
Date: Tue, 20 Oct 2020 11:00:03 -0400

Hi Kaliraj

In-line

Thanks

Gyan
On Mon, Oct 19, 2020 at 2:06 PM Kaliraj Vairavakkalai <[email-masked]>

> No, there is no impact to existing CsC deployments.
    Gyan> Understood as a new CT RIB is created per new SAFI 76.  So this
draft defines a new SAFI RD/RT that could be used instead of the current
BGP-LU Labeled Unicast rib in global table used for Opt C which would now
be carried in a separate CT RIB within the domain.  Today with service VPN
their is an automatic underpinning with label stack topmost encapsulation
so no recursion with VPN routes.  With BGP LU,  it is has a BGP RIB for
transport prefixes injected between domains as it’s a BGP RIB and not a
separate VRF RIB as being proposed with CT RIB new extended community,  and
so BGP LU sit in the global table so has next hop recursion.  In contrast
the next hop underpinning to topmost label tunnel PNH would be a special
mapping community mapped it sounds like to a primary and failover tunnel or
a set of tunnels  to map CT transport service to underlay tunnel.  This
feature could also be used as stated in the draft with CSC customer carrier
VRF new CT SAFI for the topmost labeled transport prefixes.

Section 4 details the new extended community.  I believe this is a typo as
the CT community would follow RFC 4360 and 5668.  You mention 8 byte RT and
RD but I believe you meant 6 bytes community with new IANA codepoint for
new Hi Order and low Order sub type setting.  I think you would want to
stay with standard type 0 2 byte global 4 byte local for 2 byte AS, type 1
4 byte global 2 byte local for IPv4 Global, end type 2 4 byte global 2 byte
local for 4 byte AS.

> This draft creates a new SAFI that parallels BGP-LU in option-C
> deployments. I will clarify this with an explicit mention to option-C in
> the introduction.
>
>
>
   Gyan> Understood

> The deficiency being addressed is: In today’s option-C deployments, if
> multiple classes(gold, bronze, etc) of tunnels exist to a destination PE1
> in one domain, they can not be extended out to the adjacent domains by
> using BGP-LU, such that service-traffic in those domains is able to choose
> a certain class of tunnel.
>

    Gyan> In general Opt C is not generally used between ASs with operators
in separate administrative domains end to end LSP.  The other down side of
Opt C is the RR-RR eBGP SAFI 128 129 next hop unchanged peering which makes
it impossible to use between operators in separate admin domains.  The last
major issue with Opt C being addressed here is the import of loopbacks
between domains has to be carried in the global table.  So Opt C in general
is used only for M&A or internal inter AS peering where their is a trust
relationship and the loopbacks FECs can be imported between domains.  So
with this SAFI 76 if addresses the loopback import into the global table
issue but does not address the eBGP SAFI 128 129 RR-RR peering for VPN and
MVPN NLRI.  So in this option as the LSP is end to end the next hop self
rewrite is not done in ingress egress NNI inter-as peering as the LSP
builds end to end from ingress PE in domain A to egress PE in domain B.  So
the major advantage of Opt B is that BGP LU is only for connected transport
label on the inter-as peer and no loopbacks are injected between domains
and as next hop self is utilized to terminate each segment of the LSP into
3 segments.  In Opt B the 1st segment is Ingress PE to egress PE domain A,
2nd segment inter as link, 3rd segment ingress PE to egress PE domain B.
So here I don’t see any benefit of using SAFI 76 for Opt B which scales
well.  With both Opt B SAFI 128 and new 76 you would have to do the RT
rewrite as well so no benefit there.  RTC is a RT membership optimization
that can be used     RR-PE only RTs that the PE has membership explicit
import of RT is what is sent to PE.  By default all PEs have RT filtering
enabled by default so if their is not an interesting explicit import of the
RT the RT is dropped.  So RTC is an optimizations for incongruent VPNs on
operator PEs to cut down on flooding of all RTs. So for CT are we changing
the behavior of RTC RFC 4684.  Are we making RTC part of CT architecture
and what is the requirement that it must be used as RTC is only an
optimization.
Also for CSC we here we are changing the VPN rib for customer carrier from
SAFI 128 to 76.  With CSC hierarchical VRFs VPNs the transport prefix in
the customer carrier VRF sit in the separate SAFI 128 RIB and so do the
have the issue as exists with Opt C and you are just shifting from one VPN
to another SAFI 128 to 76.  I still don’t see the benefit.

With this draft overall I do see the benefit with Opt C to assist with the
BGP LU carrying imported loopbacks between domains in the global table but
don’t see benefit in any other inter as solution that exists today.

With SRv6 LPM IPv6 data plane and no BGP-LU and use of SR-TE binding sid
policy inter domain I don’t see any benefit in using SAFI 76.

With SR-MPLS as can use legacy inter as options would have same opt-c
benefit but I think most customers using SR-MPLS would prefer to use SR-TE
binding sid policy instead of MPLS based inter as options.

> The new SAFI 76 carries NLRI of the form RDx:PE1, such that route for each
> class is carried using a distinct RD and doesn’t hide the other classes.
> And a new type of RT is used to denote the transport-class of the tunnel.
> This RT is used to leak the transport-routes into transport-RIBs of the
> same class on the receiving nodes – just like what L3VPN family does.
>
>
>
> The note about similarity with CsC was just that: CsC uses SAFI 128 (L3VN)
> family to carry transport-prefixes in the mother carrier. Similarly, this
> SAFI 76(BGP-CT) also carries transport-routes in option-C deployments.
>
>
>
> *From: *Gyan Mishra <[email-masked]>
> *Date: *Sunday, October 18, 2020 at 10:52 PM
> *To: *Robert Raszuk <[email-masked]>
> *Cc: *Kaliraj Vairavakkalai <[email-masked]>, Natrajan Venkataraman <
> [email-masked]>, "idr@ietf. org" <[email-masked]>
> *Subject: *Re: [Idr] BGP Classful Transport Planes
>
>
>
> This draft creates a new SAFI 76 BGP-CT that parallels BGP-LU that is
> being used to fix a deficiency with CSC as far as the transport label being
> used as customer carrier Hierarchical VRF to the customer carrier customer
> VRF and not in a separate transport rib.
>
>
>
> The customer carrier transport routes are in a VRF SAFI 128 and don’t
> touch the backbone carrier global table.  BGP LU is enabled on the eBGP
> You mention in the draft inter-as opt B interchangeably with CSC which
> they both are similar but very different.  However, I believe you are
> trying to address deficiencies with only CSC.
>
>
>
> What is the impact to existing CSC deployments using hierarchical VRF.
> How would you address backwards compatibility.
>
>
>
>
>
> Kind Regards
>
>
>
> Gyan
>
>
>
> ·   Use IPv6 in the data-plane as-well. Even in this case, association of
> IPv6-tunnel endpoints with appropriate transport-class will be required.
> ·   This BGP-CT route will be marked un-usable, if there is no routes in
> ·   It will have the BGP-CT route for remote-PE/32 route in gold
> <https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!U8HbQ5jZBlsxEu8405jmF-287HZ0Rs-PwJCc2URxQqPyf59kvSU9h1YXFcFZxEh2$>
>
> --
>
> <https://urldefense.com/v3/__http:/www.verizon.com/__;!!NEt6yMaO-gk!U8HbQ5jZBlsxEu8405jmF-287HZ0Rs-PwJCc2URxQqPyf59kvSU9h1YXFRulXXkX$>
>
> <https://www.google.com/maps/search/13101+Columbia+Pike+%0D%0A+Silver+Spring,+MD?entry=gmail&source=g>
>
> *Gyan Mishra*
>
> *Network Solutions Architect *
>
>
>
> *M 301 502-1347 13101 Columbia Pike
> <https://www.google.com/maps/search/13101+Columbia+Pike+%0D%0A+Silver+Spring,+MD?entry=gmail&source=g>
> *Silver Spring, MD
> <https://www.google.com/maps/search/13101+Columbia+Pike+%0D%0A+Silver+Spring,+MD?entry=gmail&source=g>
>
>
>
-- 

*Gyan Mishra*

---

From: Kaliraj Vairavakkalai <[email-masked]>
Date: Tue, 20 Oct 2020 20:56:13 +0000

Hi Gyan, please see inline..

From: Gyan Mishra <[email-masked]>
Date: Tuesday, October 20, 2020 at 8:00 AM
To: Kaliraj Vairavakkalai <[email-masked]>
Cc: Natrajan Venkataraman <[email-masked]>, Robert Raszuk <[email-masked]>, "idr@ietf. org" <[email-masked]>
Subject: Re: [Idr] BGP Classful Transport Planes

Hi Kaliraj

In-line

Thanks

Gyan
On Mon, Oct 19, 2020 at 2:06 PM Kaliraj Vairavakkalai <[email-masked]<mailto:[email-masked]>> wrote:
Hi Gyan,

    Gyan> Understood as a new CT RIB is created per new SAFI 76.  So this draft defines a new SAFI RD/RT that could be used instead of the current BGP-LU Labeled Unicast rib in global table used for Opt C which would now be carried in a separate CT RIB within the domain.

Today with service VPN their is an automatic underpinning with label stack topmost encapsulation so no recursion with VPN routes.

I suppose you are mentioning option-B inter-ASBR link here. Other-wise at an ingress-node Service-VPN routes do use nexthop-resolution of their PNH, which can be resolved via either an intra-AS tunnel(RSVP, SRTE) or an inter-AS tunnel (LU/CT).

With BGP LU,  it is has a BGP RIB for transport prefixes injected between domains as it’s a BGP RIB and not a separate VRF RIB as being proposed with CT RIB new extended community,

Please consider the BGP-LU global RIB as the best-effort Transport-rib. And with CT, we create per-class instances of Transport-RIBs (e.g. gold, silver) each having a Transport-class ID (e.g. 100, 200 resp). And yes, we use the Transport-Class RT carried on the BGP-CT routes to leak them into appropriate Transport-RIBs. It is really a transport layer BGP-VPN. Origination of BGP-CT route happens at a tunnel-ingress node, where protocols like RSVP/SRTE publish their tunnel ingress routes into these per-class Transport-RIBs, by virtue of local config.

Once the routes are collected in per-class Transport-ribs, they can now be used by other BGP-routes to resolve their nexthop. That is where the ‘mapping-community’ comes in.

The presence/absence of ‘mapping community’ on a service/transport plane route indicates which of these transport-ribs (best-effort, gold, silver) the route’s PNH should be resolved in.

and so BGP LU sit in the global table so has next hop recursion.

BGP-LU has nexthop resolution at an ingress-node. But it doesn’t do nexthop-resolution at inter-ASBR link. Whether we do nexthop-resolution or not actually depends on whether the peer is directly connected or not. It is similar logic for all families: Inet-Uni, L3VPN, LU, CT. No changes there.

In contrast the next hop underpinning to topmost label tunnel PNH would be a special mapping community mapped it sounds like to a primary and failover tunnel or a set of tunnels  to map CT transport service to underlay tunnel

The mapping-community just indicates which Transport-RIB to resolve the route’s PNH on. And it is just a ‘role’ that any BGP-community can play. Local (auto)configuration at ingress/border node identifies mapping-communities, and resolution schemes they map to.

Different routes can use different communities as mapping-communities. Because they have different requirements with respect to fallback. To elaborate:

  *   Service-routes (e.g. L3VPN, Inet-Uni) use Color extended community as mapping community. These map to a resolution-scheme which includes fallback to best-effort tunnels.

E.g. routes received with Color:0:100 will resolve using gold Transport-RIB (Transport-Class-ID:100), and fallback to best-effort transport-rib which may contain BGP-LU and other class-less tunnels (RSVP, LDP).

  - Transport-routes (BGP-CT routes) use the Transport-Class RT as the mapping community. This maps to a resolution-scheme which resolves strictly over specified transport-class, and doesn’t use any fallback.
    E.g. a BGP-CT route for RD1:PE1 route received with Transport-Class RT = transport-target:0:100 will resolve using “gold” Transport-RIB only(Transport-Class-ID:100). It will not fallback to best-effort Transport-RIB if gold transport-rib does not have any matching routes. This will result in sending a withdrawal for the “gold” BGP-CT route, thus providing a feedback mechanism to ingress-node that end to end gold path doesn’t exist now.

This feature could also be used as stated in the draft with CSC customer carrier VRF new CT SAFI for the topmost labeled transport prefixes.

That is right. But it is not the main intent. The main intent is to be able to do provide an architecture for Transport-class aware nexthop-resolution of BGP nexthops.

Section 4 details the new extended community.  I believe this is a typo as the CT community would follow RFC 4360 and 5668.  You mention 8 byte RT and RD but I believe you meant 6 bytes community with new IANA codepoint for new Hi Order and low Order sub type setting.  I think you would want to stay with standard type 0 2 byte global 4 byte local for 2 byte AS, type 1 4 byte global 2 byte local for IPv4 Global, end type 2 4 byte global 2 byte local for 4 byte AS.

rfc4360 Extended communities are 8 bytes. I didn’t get why you think it is a typo. Transport-Class identifier needed to be compatible with the Color value carried by SRTE (one of the classful transport protocols that exist today), that is the reason we defined the transport-class ID as 32-bits, in order to interoperate with existing technology. Hence defined a new format of RT instead of overloading the 4byte-AS format.

   Gyan> Understood

    Gyan> In general Opt C is not generally used between ASs with operators in separate administrative domains end to end LSP.  The other down side of Opt C is the RR-RR eBGP SAFI 128 129 next hop unchanged peering which makes it impossible to use between operators in separate admin domains.  The last major issue with Opt C being addressed here is the import of loopbacks between domains has to be carried in the global table.  So Opt C in general is used only for M&A or internal inter AS peering where their is a trust relationship and the loopbacks FECs can be imported between domains.  So with this SAFI 76 if addresses the loopback import into the global table issue but does not address the eBGP SAFI 128 129 RR-RR peering for VPN and MVPN NLRI.  So in this option as the LSP is end to end the next hop self rewrite is not done in ingress egress NNI inter-as peering as the LSP builds end to end from ingress PE in domain A to egress PE in domain B.  So the major advantage of Opt B is that BGP LU is only for connected transport label on the inter-as peer and no loopbacks are injected between domains and as next hop self is utilized to terminate each segment of the LSP into 3 segments.  In Opt B the 1st segment is Ingress PE to egress PE domain A, 2nd segment inter as link, 3rd segment ingress PE to egress PE domain B.

Option-B or intra-AS VPNs will also see benefit from this architecture. Because,

  *   Even without SAFI-76, intra-AS tunneling protocols (RSVP, SRTE) publish their routes in Transport-RIBs created for the transport-classes that exist in the network. And Service-routes can map to these Transport-classes using appropriate mapping-community.

When information in these transport-ribs needs to be exported out to BGP-speakers in other domains, SAFI-76 comes into play.

  *   When such an intra-AS service-route which has resolved over gold transport-class, gets readvertised across inter-AS link for option-B, the mpls swap-label will stitch to the gold transport-class tunnel. Thus the option-B traffic can also get gold transport-class end-to-end, while traversing any of the ASes participating in the option-B deployment.

  *   Also, when an AS is too big that full-mesh of tunnels between PEs is hard to manage, the network can be segmented into regions. And SAFI-76 can be used to carry Transport-class aware PE reachability across regions (within one AS) in such cases, to stitch together gold-class tunnels in each region, for e.g. to provide end-to-end gold tunnel in the AS.

does not address the eBGP SAFI 128 129 RR-RR peering for VPN and MVPN NLRI

I agree the Multihop EBGP session between domains to exchange service-routes is required. This proposal doesn’t change that. I don’t see any way to avoid that in option-C. Yes Option-B avoids that, but it pushes additional service-routes state at the ASBRs, and increasing the convergence hops in the network. Both Option-B and Option-C have some benefits and disadvantages. This BGP-CT proposal doesn’t alter that. But there is a new proposal “MPLS-namespaces signaled by BGP” which can potentially bring benefits of both these options together. I may present it in the oncoming IETF, in BESS WG. Though it doesn’t avoid the ebgp-multihop session, it brings the benefit of option-B to option-C deployments, without increasing service-route scale at the ASBRs. Details of that mechanism will be out of scope of this discussion.

So here I don’t see any benefit of using SAFI 76 for Opt B which scales well. With both Opt B SAFI 128 and new 76 you would have to do the RT rewrite as well so no benefit there.

There is no intent to mix SAFI-128(service-routes) and SAFI-76. The idea is to maintain clear separation between service-routes and transport-routes. SAFI-76 carries only transport-prefixes.

RTC is a RT membership optimization that can be used     RR-PE only RTs that the PE has membership explicit import of RT is what is sent to PE.  By default all PEs have RT filtering enabled by default so if their is not an interesting explicit import of the RT the RT is dropped.  So RTC is an optimizations for incongruent VPNs on operator PEs to cut down on flooding of all RTs. So for CT are we changing the behavior of RTC RFC 4684.  Are we making RTC part of CT architecture and what is the requirement that it must be used as RTC is only an optimization.

No, we are not changing the behavior. RTC is still optional with this family as-well.

Also for CSC we here we are changing the VPN rib for customer carrier from SAFI 128 to 76.  With CSC hierarchical VRFs VPNs the transport prefix in the customer carrier VRF sit in the separate SAFI 128 RIB and so do the have the issue as exists with Opt C and you are just shifting from one VPN to another SAFI 128 to 76.  I still don’t see the benefit.

There are a few benefits I can see:

  *   It makes it easier to give different treatment, like per-prefix-label, to transport-prefixes carried in SAFI-76. It provides stability by localizing churn. We don’t want to give this treatment to routes advertised in SAFI-128. Because it carries service-prefixes as-well, so it is a scaling problem.
  *   The transport-RIBs proposed in this solution are recommended to be implemented as Control-Plane only RIBs. That don’t get pushed to forwarding. Unlike the CsC VRFs today, which by default, do get pushed to Forwarding as ip-prefixes and consume forwarding resources.
  *   More importantly, debugging. Looking at a SAFI-128 route, it is difficult to tell immediately whether it is a service-prefix or a Csc transport-prefix. If we use SAFI-76, it gives better visibility to humans and tools managing the network.

With this draft overall I do see the benefit with Opt C to assist with the BGP LU carrying imported loopbacks between domains in the global table but don’t see benefit in any other inter as solution that exists today.

Thank you for the note on Option-C. I have tried to clarify above the usefulness in the other use-cases.

With SRv6 LPM IPv6 data plane and no BGP-LU and use of SR-TE binding sid policy inter domain I don’t see any benefit in using SAFI 76.

With SR-MPLS as can use legacy inter as options would have same opt-c benefit but I think most customers using SR-MPLS would prefer to use SR-TE binding sid policy instead of MPLS based inter as options.

IMO, SRTE will have the problem as one of our previous proposal (BGP-LCU). Because it carries Color in the NLRI, co-ordinating between different domains will be difficult. It is not as easy as rewriting RT.

Thanks for the engaging discussion.

Kaliraj

---

From: Gyan Mishra <[email-masked]>
Date: Thu, 22 Oct 2020 02:32:15 -0400

On Tue, Oct 20, 2020 at 4:56 PM Kaliraj Vairavakkalai <[email-masked]>
wrote:

> *Date: *Tuesday, October 20, 2020 at 8:00 AM
> *To: *Kaliraj Vairavakkalai <[email-masked]>
> *Cc: *Natrajan Venkataraman <[email-masked]>, Robert Raszuk <
> [email-masked]>, "idr@ietf. org" <[email-masked]>
> be carried in a separate CT RIB within the domain.
>
>
>
> Today with service VPN their is an automatic underpinning with label stack
> topmost encapsulation so no recursion with VPN routes.
>
>
>
> I suppose you are mentioning option-B inter-ASBR link here. Other-wise at
> an ingress-node Service-VPN routes do use nexthop-resolution of their PNH,
> which can be resolved via either an intra-AS tunnel(RSVP, SRTE) or an
> inter-AS tunnel (LU/CT).
>
> Gyan> I was mentioning in general Intra AS VPN SAFI 128 that the service
> label is bottom of stack sits under the transport label LDP or TE under
> pinning.  In case of VPN mapped TE or per VRF TE next hop rewrite or SR-TE
> the service VPN label BOS would be placed underpinned to whichever
> transport label tunnel ends up being the preferred next hop is set to
> egress PE LSP endpoint.
>
With BGP LU,  it is has a BGP RIB for transport prefixes injected between
> domains as it’s a BGP RIB and not a separate VRF RIB as being proposed with
> CT RIB new extended community,
>
>
>
> Please consider the BGP-LU global RIB as the best-effort Transport-rib.
> And with CT, we create per-class instances of Transport-RIBs (e.g. gold,
> silver) each having a Transport-class ID (e.g. 100, 200 resp). And yes, we
> use the Transport-Class RT carried on the BGP-CT routes to leak them into
> appropriate Transport-RIBs. It is really a transport layer BGP-VPN.
> Origination of BGP-CT route happens at a tunnel-ingress node, where
> protocols like RSVP/SRTE publish their tunnel ingress routes into these
> per-class Transport-RIBs, by virtue of local config.
>
> Gyan> Understood. I think I am getting a better picture of the solution.
> So basically you can have any flavor of transports tunnels topmost label
> outer header you desire but now each transport tunnel type corresponding
> transport prefixes can be placed into a corresponding special SAFI 76 CT
> RIB.
>

    Gyan> Have you looked at BGP Tunnel encapsulation attribute draft
updates RFC 5512.
This does have some similarities to that draft and in some ways can provide
some of the same functionality.

https://datatracker.ietf.org/doc/draft-ietf-idr-tunnel-encaps/

Once the routes are collected in per-class Transport-ribs, they can now be
> used by other BGP-routes to resolve their nexthop. That is where the
> ‘mapping-community’ comes in.
>
>  Gyan> That’s a bit confusing.  So the CT rib contains the transport
> prefixes but then also contains the next hop recursion routes which maybe
> IGP routes.  How is that possible if the next hop is known in IGP and not
> BGP which is almost always the case.
>
> The presence/absence of ‘mapping community’ on a service/transport plane
> route indicates which of these transport-ribs (best-effort, gold, silver)
> the route’s PNH should be resolved in.
>
>
>
>
>
> BGP-LU has nexthop resolution at an ingress-node. But it doesn’t do
> nexthop-resolution at inter-ASBR link. Whether we do nexthop-resolution or
> not actually depends on whether the peer is directly connected or not. It
> is similar logic for all families: Inet-Uni, L3VPN, LU, CT. No changes
> there.
>
>  Gyan> Understood standard next hop recursion.
>
> In contrast the next hop underpinning to topmost label tunnel PNH would be
> a special mapping community mapped it sounds like to a primary and failover
> tunnel or a set of tunnels  to map CT transport service to underlay tunnel
>
>
>
> The mapping-community just indicates which Transport-RIB to resolve the
> route’s PNH on. And it is just a ‘role’ that any BGP-community can play.
> Local (auto)configuration at ingress/border node identifies
> mapping-communities, and resolution schemes they map to.
>
> Gyan> An example would be really helpful.  A picture is with a thousand
> words. 😀
>
    Gyan> I think I finally got it.  So all next hops are sorted by
transport tunnel type gold silver bronze and placed into discrete Transport
RIBs based on their mapping community setting.  The transport prefixes are
the recursed prefixes next hop prefixes one and the same.  Basically the
same BGP-LU next hop loopbacks from all PEs advertised inter-as opt now
have a variety of CT ribs they can be mapped into based on mapping
community.

> Different routes can use different communities as mapping-communities.
> Because they have different requirements with respect to fallback. To
> elaborate:
>
>
>
>    - Service-routes (e.g. L3VPN, Inet-Uni) use Color extended community
>    as mapping community. These map to a resolution-scheme which includes
>    fallback to best-effort tunnels.
>
> E.g. routes received with Color:0:100 will resolve using gold
> Transport-RIB (Transport-Class-ID:100), and fallback to best-effort
> transport-rib which may contain BGP-LU and other class-less tunnels (RSVP,
> LDP).
>
>
>
>   - Transport-routes (BGP-CT routes) use the Transport-Class RT as the
> mapping community. This maps to a resolution-scheme which resolves strictly
> over specified transport-class, and doesn’t use any fallback.
>
>     E.g. a BGP-CT route for RD1:PE1 route received with Transport-Class RT
> = transport-target:0:100 will resolve using “gold” Transport-RIB
> only(Transport-Class-ID:100). It will not fallback to best-effort
> Transport-RIB if gold transport-rib does not have any matching routes. This
> will result in sending a withdrawal for the “gold” BGP-CT route, thus
> providing a feedback mechanism to ingress-node that end to end gold path
> doesn’t exist now.
>
>
>
> This feature could also be used as stated in the draft with CSC customer
> carrier VRF new CT SAFI for the topmost labeled transport prefixes.
>
>
>
> That is right. But it is not the main intent. The main intent is to be
> able to do provide an architecture for Transport-class aware
> nexthop-resolution of BGP nexthops.
>
>
>
> rfc4360 Extended communities are 8 bytes. I didn’t get why you think it is
> a typo. Transport-Class identifier needed to be compatible with the Color
> value carried by SRTE (one of the classful transport protocols that exist
> today), that is the reason we defined the transport-class ID as 32-bits, in
> order to interoperate with existing technology. Hence defined a new format
> of RT instead of overloading the 4byte-AS format.
> Gyan> The size of the entire field is 8 bytes including the high and low 2
> bytes however what is administratively set is the global and local which is
> a total of 6 bytes which is the RD RT extended community.
>
>
>
>
>
>     Gyan> In general Opt C is not generally used between ASs with
> operators in separate administrative domains end to end LSP.  The other
> down side of Opt C is the RR-RR eBGP SAFI 128 129 next hop unchanged
> peering which makes it impossible to use between operators in separate
> admin domains.  The last major issue with Opt C being addressed here is the
> import of loopbacks between domains has to be carried in the global table.
> So Opt C in general is used only for M&A or internal inter AS peering where
> their is a trust relationship and the loopbacks FECs can be imported
> between domains.  So with this SAFI 76 if addresses the loopback import
> into the global table issue but does not address the eBGP SAFI 128 129
> RR-RR peering for VPN and MVPN NLRI.  So in this option as the LSP is end
> to end the next hop self rewrite is not done in ingress egress NNI inter-as
> peering as the LSP builds end to end from ingress PE in domain A to egress
> PE in domain B.  So the major advantage of Opt B is that BGP LU is only for
> connected transport label on the inter-as peer and no loopbacks are
> injected between domains and as next hop self is utilized to terminate each
> segment of the LSP into 3 segments.  In Opt B the 1st segment is Ingress PE
> to egress PE domain A, 2nd segment inter as link, 3rd segment ingress PE to
> egress PE domain B.
>
>
>
>    - Even without SAFI-76, intra-AS tunneling protocols (RSVP, SRTE)
>    publish their routes in Transport-RIBs created for the transport-classes
>    that exist in the network. And Service-routes can map to these
>    Transport-classes using appropriate mapping-community.
>
> When information in these transport-ribs needs to be exported out to
> BGP-speakers in other domains, SAFI-76 comes into play.
>
>
>
>    - When such an intra-AS service-route which has resolved over gold
>    transport-class, gets readvertised across inter-AS link for option-B, the
>    mpls swap-label will stitch to the gold transport-class tunnel. Thus the
>    option-B traffic can also get gold transport-class end-to-end, while
>    traversing any of the ASes participating in the option-B deployment.
>
>
>
>    - Also, when an AS is too big that full-mesh of tunnels between PEs is
>    hard to manage, the network can be segmented into regions. And SAFI-76 can
>    be used to carry Transport-class aware PE reachability across regions
>    (within one AS) in such cases, to stitch together gold-class tunnels in
>    each region, for e.g. to provide end-to-end gold tunnel in the AS.
>
>
>
>  Gyan> In way applying option C to option B in a way.  So with opt B you
> are stitching topmost transport label LDP lsp intra AS with BGP LU SAFI 4
> inter-as to build end to end lsp.  So on the intra-as side with opt b
> instead of stitching ldp to BGP LU you could to SAFI 76 end to end CT rib
> provisioned.
>
> I agree the Multihop EBGP session between domains to exchange
> service-routes is required. This proposal doesn’t change that. I don’t see
> any way to avoid that in option-C. Yes Option-B avoids that, but it pushes
> additional service-routes state at the ASBRs, and increasing the
> convergence hops in the network. Both Option-B and Option-C have some
> benefits and disadvantages. This BGP-CT proposal doesn’t alter that. But
> there is a new proposal “MPLS-namespaces signaled by BGP” which can
> potentially bring benefits of both these options together. I may present it
> in the oncoming IETF, in BESS WG. Though it doesn’t avoid the ebgp-multihop
> session, it brings the benefit of option-B to option-C deployments, without
> increasing service-route scale at the ASBRs. Details of that mechanism will
> be out of scope of this discussion.
>
> Gyan> Understood
>
>
>
> rewrite as well so no benefit there.
>
>
>
> There is no intent to mix SAFI-128(service-routes) and SAFI-76. The idea
> is to maintain clear separation between service-routes and
> transport-routes. SAFI-76 carries only transport-prefixes.
>
> Gyan> Understood
>
> RTC is a RT membership optimization that can be used     RR-PE only RTs
> that the PE has membership explicit import of RT is what is sent to PE.  By
> default all PEs have RT filtering enabled by default so if their is not an
> interesting explicit import of the RT the RT is dropped.  So RTC is an
> optimizations for incongruent VPNs on operator PEs to cut down on flooding
> of all RTs. So for CT are we changing the behavior of RTC RFC 4684.  Are we
> making RTC part of CT architecture and what is the requirement that it must
> be used as RTC is only an optimization.
>
>
>
> Gyan> Good
>    - It makes it easier to give different treatment, like
>    per-prefix-label, to transport-prefixes carried in SAFI-76. It provides
>    stability by localizing churn. We don’t want to give this treatment to
>    routes advertised in SAFI-128. Because it carries service-prefixes as-well,
>    so it is a scaling problem.
>    - The transport-RIBs proposed in this solution are recommended to be
>    implemented as Control-Plane only RIBs. That don’t get pushed to
>    forwarding. Unlike the CsC VRFs today, which by default, do get pushed to
>    Forwarding as ip-prefixes and consume forwarding resources.
>    - More importantly, debugging. Looking at a SAFI-128 route, it is
>    difficult to tell immediately whether it is a service-prefix or a Csc
>    transport-prefix. If we use SAFI-76, it gives better visibility to humans
>    and tools managing the network.
>
>  Gyan> Understood
>
> IMO, SRTE will have the problem as one of our previous proposal (BGP-LCU).
> Because it carries Color in the NLRI, co-ordinating between different
> domains will be difficult. It is not as easy as rewriting RT.
>
> Gyan> Understood
>
> Gyan> Welcome!
-- 

*Gyan Mishra*