Subject: Re: [Idr] rd-orf problem clarification at the local level

From: Aijun Wang <[email-masked]>
Date: Tue, 16 Feb 2021 13:06:20 +0800

Hi, Jeff:
This is the reply to your comments on the local peer that initiated the RD-ORF message.

Wish these explanations can help you know RD-ORF mechanism more throughly and can refine and forward it.

Aijun Wang
China Telecom

> On Feb 16, 2021, at 01:10, Jeffrey Haas <[email-masked]> wrote:
> 
> Authors,
> 
> The IDR chairs were discussing how best to move forward the discussion about
> Working Group adoption for rd-orf.  As part of that discussion, we thought
> it would be helpful to have the discussion split to cover specific points.
> The goal is primarily to clarify the functional requirements.  As part of
> this discussion, there will be some technical observations on the proposal.
> 
> This e-mail is structured with the perceived issue, and a set of questions
> to be commented on as Issues at the end.
> 
> This e-mail focuses on the rd-orf problem statement from the receiving PE's
> perspective.  
> 
> ---
> 
> Presume we have a PE router with some number of attached VRFs.
> For this example, we have a VRF-A that uses a single route target.
> In this VRF, routes from two remote PEs with distinct route distinguishers,
> RD1 and RD2, contribute to that VRF.
> 
> ---
> 
> Presumed problem statement:
> It is possible for this VRF, and perhaps the PE itself, to be overwhelmed by
> an excess number of VPN routes.

> 
> Common implementation mitigations available today:
> 1. A VRF may implement a prefix limit that prevents an excess number of
>  routes from being redistributed from the VPN's address family into the VRF.
> 
>  However, this does not keep the PE from retaining an excess number of VPN
>  routes in its global VPN table.
[WAJ]Yes
> 
> 2. The PE may apply BGP policy to restrict the VPN routes that it will
>  accept and store.  An example of this may be policy based on BGP route
>  properties in Path Attributes such as Extended Communities.  Another
>  example may be policy based on BGP NLRI properties, such as route
>  distinguisher.  (The implementation may require configuration to cause
>  VPN routes rejected by policy to be discarded rather than stored in the
>  Adj-Ribs-In as ineligible routes.)
> 
> It is thus possible for an implementation to mitigate the impact storage
> requirements for an excess number of VPN routes with no new features.
[WAJ] As stated in previous mail, if we can know the attributes of offending routes in advance, such solution is possible. But for the scenarios that described in the draft, the overwhelming VPN routes are interested and permitted, but accept all of them can influence the capabilities for the PE to service other VRFs on it.
> 
> ---
> 
> Reducing the work for the receiving PE for unwanted routes:
> 
> It can be observed that a core benefit of RT-Constrain (RFC 4684) is that it
> permits a receiving PE to avoid receiving routes it is not interested in.
> This may also reduce the work of the sending BGP Speaker by avoiding the I/O
> overhead of the unwanted routes.
> 
> However, RT-Constrain will not help you reduce the work if you want a subset
> of the routes covered by a specific route target.

[WAJ]Yes

> 
> The desired property in rd-orf is that a BGP Speaker will exchange an
> outbound route filter that provides a list of route distinguishers that will
> be excluded from the routes sent on a session.  
> 
> Essentially, a "do not send these RD routes list".
> 
> Such an ORF can reduce the amount of work on a receiving PE, and reduce the
> storage of unwanted routes.

[WAJ] Yes

> 
> ---
> 
> Operational considerations for PE filtering using route distinguishers:
> 
> 1. A VPN route may be targeted for more than one receiving VRF.  Excluding
>  VPN routes by route distinguisher may only be done if it is operationally
>  acceptable that no VRFs on the receiving router will receiving those
>  routes, regardless of VPN membership.  
> 
>  (That is, collateral damage is a possibility that must be understood.)

[WAJ] Yes. Section https://datatracker.ietf.org/doc/html/draft-wang-idr-rd-orf-05#section-5.1.1 in Figure 3 has described such situations, as also the example that Robert mentioned. More clear description should be clarified in the later version to achieve the determined behavior in such situations.

> 
> 2. A PE may have downstream BGP peers.  It must be operationally acceptable
>  that no downstream BGP peers receive VPN routes whose route distinguisher
>  is being filtered.
[WAJ] Yes. This is more like the behavior of RR/ASBR. More descriptions and criteria will be added for the trigger of RD-ORF message on these intermediate/proxy/leader devices.
> 
> 3. VRFs that have prefix limits may have its limit exceeded by routes from
>  multiple remote PE resources.  What is the mechanism that will permit an
>  implementation to choose what route distinguisher will be filtered,
>  presuming the collateral damage in points 1 and 2 is acceptable?
[WAJ] Currently, we consider to select the RD whose associated VPN routes occupy most part of offending VPN routes.
> 
> ---
> 
> rd-orf protocol issues:
> 
> 1. The rd-orf type specific part includes a Sequence field.  Its use in the
>  ORF filtering mechanism is unclear in the draft's text.
> 
>  It is my speculation that this comes from the Prefix ORF (RFC 5292)
>  semantics.  However, it is improperly specified, if so.  A Prefix ORF
>  requires ordering semantics in order to properly execute prefix-match
>  behavior, especially given the need for more-specific/less-specific
>  portions of the match.  In the case of an rd-orf, order is not relevant,
>  only the route-distinguisher.
[WAJ] Yes, it refers the semantic of RFC5292. But I think this field can also be useful for the list/add/remove of the filter policy, not the compared order. 
And, one correction to the previous mail for the ORF mechanism: there are permit/deny actions for the filter policy. Is it possible we also utilize this field to achieve the following effects as that you mentioned in previous mail:
1: “Permit some, deny the rest”
2:”Deny some, permit the rest”
If so, the “sequence” field can be utilized.
Anyway, we can further refine the semantic of these fields later.

> 
> 2. The Route Distinguisher field is a fixed length of 8 octets.  Using ORF
>  procedures (RFC 5291), once an ORF has been received for a given
>  AFI/SAFI, the entries of the ORF must be matched or the routes will not
>  be advertised to the remote BGP speaker.  
> 
>  This means that once there is a desire to specify a "do not send" rd-orf,
>  it becomes required that all desired RDs must be sent.  Knowing the full
>  set of possible RDs a priori is likely impossible.
> 
>  The feature thus needs a "default" entry of some sort.

[WAJ] Yes. One possible way is that adding the following RD-ORF entry in last:
“Action/permit, RD/all-zero”
If acceptable, will add such description later.

> 
> ---
> 
> Encoding rd-orf behaviors using Address-Prefix-Based ORF (RFC 5292):
> 
> Jakob Heitz, among others, has mentioned this solution.
> 
> A VPN route is made up of an RD + Prefix, with RD being a fixed 8 octet
> value from known types.
> 
> A rd-only ORF may be represented as:
> - The AFI/SAFI of the VPN address family to be filtered.
> - An appropriate Sequence number
> - A minlen of 64 bits
> - A maxlen appropriate to the AFI/SAFI of the VPN route in question (as long
> as its bit-length is <= 255).
> - Length is 64 bits
> - Prefix is the encoded route distinguisher
> 
> The default VPN Address Family ORF may be represented as:
> - The AFI/SAFI of the VPN address family to be filtered.
> - The last sequence number in the known series.
> - A minlen of 0 bits
> - A maxlen appropriate to the AFI/SAFI of the VPN route in question (as long
> as its bit-length is <= 255).

[WAJ] We should state clearly the maxlen value in every AFI/SAFI, as that done in RFC7543 https://tools.ietf.org/html/rfc7543.
Is it more prune to error?

> - Length is 0 bits
> - Prefix will not be encoded due to the Length.
> 
> The Sequence number semantics for such Prefix ORFs containing only route
> distinguishers is less important than those for Address prefixes.  The main
> requirement is that the individual members are distinctly numbered, and that
> the default entry, if present, is last.
> 
> In cases where Prefix filtering is desired for VPN routes in addition to
> route distinguisher filtering, the implementation will need to make
> appropriate adjustments to the Sequence number.
> 
> That said, aside from the popularity of ORFs in modern BGP implementations
> being low, VPN route filtering for ORFs is likely uncommon due to the need
> to known the route distinguisher a priori.  (See discussion above.)
[WAJ] Yes, it is not easy to know the RD of offending VPN routes, and it is different from the initial aim of Prefix-Based ORF, which is always planned in advance. This is the main reason that we prefer to using one distinct encoding.

> ---
> 
> Issues for discussion:
> 1. Is this analysis of the issue from the receiving PE's perspective
>  correct?  If not, please comment in the appropriate text above.
[WAJ] Basically correct. Have commented in the related parts above.
> 
> 2. What procedure would an implementation use to apply a prefix-limit to
>  trigger this filtering mechanism given that a VRF may have routes
>  contributed to by several route distinguishers?
[WAJ] Will select the RD whose associated VPN routes occupy the most part of the offending VPN routes. Will clarify this point later if necessary.
> 
> 3. Can the procedure above for using RFC 5292 address the use case?  If not,
>  please comment on the specific use case and encoding difficulty.
[WAJ] It is possible, but not very concise. We must define clearly most of used fields for every AFI/SAFI. And it is contrary in some extent the knowledge of prefix-list usage/implementation in several vendors including Cisco/Huawei and Juniper, as I have explained to Jakob in https://mailarchive.ietf.org/arch/msg/idr/sSrjOAoLZCEd2oYfJAYeryp9i4Y/
> -- Jeff

---

From: "Aijun Wang" <[email-masked]>
Date: Mon, 22 Feb 2021 18:39:10 +0800

Hi, All:

We have upload one new draft to describe the problems that related to the VPN routes control in shared BGP session https://datatracker.ietf.org/doc/html/draft-wang-idr-vpn-routes-control-analysis

Wish to hear your considerations and comments on such analysis.

If we can consensus on the scenarios analysis and the solution requirements, we can focus on the potential solutions in next step.

Thanks in advance.

Best Regards

Aijun Wang

China Telecom

From: [email-masked] <[email-masked]> On Behalf Of Aijun Wang
Sent: Tuesday, February 16, 2021 1:06 PM
To: Jeffrey Haas <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [Idr] rd-orf problem clarification at the local level

Hi, Jeff:

Aijun Wang
China Telecom

On Feb 16, 2021, at 01:10, Jeffrey Haas <[email-masked] <mailto:[email-masked]> > wrote:

Authors,

perspective.  

---

---

[WAJ]Yes

 

---

[WAJ]Yes

[WAJ] Yes

---

 

 is being filtered.

 

 

---

rd-orf protocol issues:

 

 

---

value from known types.

- A minlen of 64 bits

- Length is 64 bits

- A minlen of 0 bits

- Length is 0 bits

---

Issues for discussion:

 

 

 

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 24 Feb 2021 15:59:51 -0500

Aijun,

> [WAJ] Currently, we consider to select the RD whose associated VPN routes occupy most part of offending VPN routes.

That may be operationally acceptable for your deployment.  For others, that
may not work.  

As an example, consider RD1 contributes a large number of prefixes that are
expected.  RD2, which is normally a small number of prefixes, causes the
table to exceed its limit.  RD1 routes would be removed even though they are
desirable ones.

I do not suggest a single correct behavior.  However, your operational
considerations should discuss how prioritization may need to vary depending
on the operator desires and what this might look like.  One example is this
may turn into a "per-RD prefix limit" feature.  See the grow Working Group
for some additional discussions on prefix limits.

> [WAJ] Yes, it refers the semantic of RFC5292. But I think this field can also be useful for the list/add/remove of the filter policy, not the compared order. 

If there are no prefixes associated with the RD (it's the entire set of
bytes), then the Sequence field isn't helpful.  It actually makes it
ambiguous.

For example, Sequence = 1, RD = A - if you withdraw Sequence = 2 RD = A, it
wouldn't match.  All that is needed to add or withdraw without some sort of
prefix length is the RD.

That said, prefix length for purposes of wildcarding would work.  For
example, "permit by default".

> Anyway, we can further refine the semantic of these fields later.

This is an important protocol detail, particularly how you say "match all
RD, permit" for the wildcard rule.

> If acceptable, will add such description later.

All zero may be an acceptable encoding.

> Is it more prune to error?

Stating the expected maxlen values would be okay.

The decision would be that if the existing Prefix-ORF is acceptable for your
use case, as we have discussed for this point, then perhaps there is no new
ORF work to be done. You would be documenting a use case for the existing
feature.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 24 Feb 2021 16:21:38 -0500

Aijun,

> If we can consensus on the scenarios analysis and the solution requirements, we can focus on the potential solutions in next step.

Thanks for separating out the scenarios.  It may help focus the conversation
on what may be able to be functionally done via BGP without also working
through the specific details of a proposed protocol change.

I have two broad comments on this draft:

1. 
The use cases are helpful in describing route distribution scenarios for VPN
routes for this problem of "excess routes".  However, the scenarios are not
equally clear about identifying routes for a given VPN that can be
remediated.

In the prior discussions, it was clear that remediation was trying to be
done on a per-RD basis.  The text in this draft seems to be a larger scope
than that; perhaps per VPN (and thus route-target?).  Is that your
intention?

2.
In section 8, "Requirements for the solutions", b) (second one, you need to
renumber!), you state:

:  b) For RR and ASBR devices, such control message should be only
:  flooded to its upstream BGP neighbor when all its downstream BGP
:  peers can't or don't want to process it, or the process of such
:  excessive routes has exceed its own capability.

Previous discussion regarding rd-orf showed that there is a strong challenge
to determining that there are no interested downstream receivers.  You will
need to document the scenarios where this can be clearly demonstrated.

A few examples:
For a PE, all CE instances do not want this RD or RT.  If there is no
inter-as VPN, this is a clear condition.

All remaining scenarios introduce ambiguity, I think.

For a route reflector, it would require that all possible receivers for a
RD/RT have no interest in the routes, and that the source of the routes is
clearly directional.  BGP does not provide such a distribution graph.

In a network using RT-Constrain without a default RT-Constrain route, such a
graph potentially exists.

-- Jeff

---

From: "Aijun Wang" <[email-masked]>
Date: Thu, 25 Feb 2021 11:09:13 +0800

Hi, Jeffrey:

-----Original Message-----
From: [email-masked] <[email-masked]> On Behalf Of Jeffrey Haas
Sent: Thursday, February 25, 2021 5:00 AM
To: Aijun Wang <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [Idr] rd-orf problem clarification at the local level

Aijun,

On Tue, Feb 16, 2021 at 01:06:20PM +0800, Aijun Wang wrote:
> > 3. VRFs that have prefix limits may have its limit exceeded by 
> > routes from  multiple remote PE resources.  What is the mechanism 
> > that will permit an  implementation to choose what route 
> > distinguisher will be filtered,  presuming the collateral damage in points 1 and 2 is acceptable?

That may be operationally acceptable for your deployment.  For others, that may not work.  

As an example, consider RD1 contributes a large number of prefixes that are expected.  RD2, which is normally a small number of prefixes, causes the table to exceed its limit.  RD1 routes would be removed even though they are desirable ones.
[WAJ] I think such situation is acceptable. We should understand first that the VPN routes associated with RD1 and RD2 are all legitimate, that is to say, all are desirable one from the beginning of deployment.
Once the RD-ORF is triggered, the operator should intervene to find the real reason for it.  It may be the PE should be upgraded, or there are too many VPN routes with RD1. Anyway, remove the VPN routes with RD2 is not reasonable. 

I do not suggest a single correct behavior.  However, your operational considerations should discuss how prioritization may need to vary depending on the operator desires and what this might look like.  One example is this may turn into a "per-RD prefix limit" feature.
[WAJ] Set the "per-RD prefix limit" to one common value is acceptable(that is to say, such threshold is RD-independent) and can be used to trigger the RD-ORF message in future. But set different value for different RD is not possible, because normally the receive PE don't know the RD that assigned by the remote peer in advance.
  See the grow Working Group for some additional discussions on prefix limits.
[WAJ] Will try to follow the related discussions.

> > rd-orf protocol issues:
> > 
> > 1. The rd-orf type specific part includes a Sequence field.  Its use 
> > in the  ORF filtering mechanism is unclear in the draft's text.
> > 
> > semantics.  However, it is improperly specified, if so.  A Prefix 
> > ORF  requires ordering semantics in order to properly execute 
> > prefix-match  behavior, especially given the need for 
> > more-specific/less-specific  portions of the match.  In the case of 
> > an rd-orf, order is not relevant,  only the route-distinguisher.

If there are no prefixes associated with the RD (it's the entire set of bytes), then the Sequence field isn't helpful.  It actually makes it ambiguous.

For example, Sequence = 1, RD = A - if you withdraw Sequence = 2 RD = A, it wouldn't match.  All that is needed to add or withdraw without some sort of prefix length is the RD.

That said, prefix length for purposes of wildcarding would work.  For example, "permit by default".
[WAJ] Will reconsider the related design to accomplish the effect of "permit by default"

> Anyway, we can further refine the semantic of these fields later.

This is an important protocol detail, particularly how you say "match all RD, permit" for the wildcard rule.
[WAJ] Yes

> > 
> > 2. The Route Distinguisher field is a fixed length of 8 octets.  
> > Using ORF  procedures (RFC 5291), once an ORF has been received for 
> > a given  AFI/SAFI, the entries of the ORF must be matched or the 
> > routes will not  be advertised to the remote BGP speaker.
> > 
> >  This means that once there is a desire to specify a "do not send" 
> > rd-orf,  it becomes required that all desired RDs must be sent.  
> > Knowing the full  set of possible RDs a priori is likely impossible.
> > 

All zero may be an acceptable encoding
[WAJ] Yes, will consider this later.

> > A VPN route is made up of an RD + Prefix, with RD being a fixed 8 
> > octet value from known types.
> > 
> > - A maxlen appropriate to the AFI/SAFI of the VPN route in question 
> > (as long as its bit-length is <= 255).
> > - Length is 64 bits
> > - A maxlen appropriate to the AFI/SAFI of the VPN route in question 
> > (as long as its bit-length is <= 255).
> 

Stating the expected maxlen values would be okay.
[WAJ] My consideration is that we don't touch the prefix-ORF to accomplish the RD-ORF effects. Because the prefix part(considering the current implement status and the normal understandings of its semantic) and the RD part will not be changed in different address families.
Focus on the standardization for the fix parts of the address can keep the RFC stable, or else, we must update it once there are new address families emerged. 

The decision would be that if the existing Prefix-ORF is acceptable for your use case, as we have discussed for this point, then perhaps there is no new ORF work to be done. You would be documenting a use case for the existing feature.
[WAJ] Except the above reason, and as discussed with Robert previously, we are considering to add the RT information together with current RD-ORF semantics. Then reusing the existing Prefix-ORF is not the right way to forward.

-- Jeff

_______________________________________________
Idr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/idr

---

From: "Aijun Wang" <[email-masked]>
Date: Thu, 25 Feb 2021 12:07:49 +0800

Hi, Jeffrey:

Thanks for your review of this draft. 
More comments on the scenarios described in this document and additional
scenarios are welcome.

Best Regards

Aijun Wang
China Telecom

-----Original Message-----
From: [email-masked] <[email-masked]> On Behalf Of Jeffrey Haas
Sent: Thursday, February 25, 2021 5:22 AM
To: Aijun Wang <[email-masked]>
Cc: 'Aijun Wang' <[email-masked]>; [email-masked];
[email-masked]
Subject: draft-wang-idr-vpn-routes-control-analysis (was Re: [Idr] rd-orf
problem clarification at the local level)

Aijun,

On Mon, Feb 22, 2021 at 06:39:10PM +0800, Aijun Wang wrote:
> We have upload one new draft to describe the problems that related to 
> the VPN routes control in shared BGP session 
> https://datatracker.ietf.org/doc/html/draft-wang-idr-vpn-routes-contro
> l-analysis
> 
> If we can consensus on the scenarios analysis and the solution
requirements, we can focus on the potential solutions in next step.

1. 

In the prior discussions, it was clear that remediation was trying to be
done on a per-RD basis.  The text in this draft seems to be a larger scope
than that; perhaps per VPN (and thus route-target?).  Is that your
intention?
[WAJ] Yes. We are considering to add RT information to the current RD-ORF
semantic, to identify/control the "excessive routes" in more granular. 
In figure 1 of this draft, if RD is allocated per VRF per PE(for example for
VPN1), then the RD only is enough to identify the "excessive routes". 
But there are situations that RD is not unique across different VRFs on one
PE(although not common, but may exist, will try to add such illustration in
Figure 1 in update version). In such situation, additional information is
help and necessary.

2.

A few examples:

For a route reflector, it would require that all possible receivers for a
RD/RT have no interest in the routes, and that the source of the routes is
clearly directional.  BGP does not provide such a distribution graph.
[WAJ] " all its downstream BGP peers can't or don't want to process it"
means RR receives the RD-ORF messages for the same set of VPN routes from
all of its clients. This can be achieved by RR

In a network using RT-Constrain without a default RT-Constrain route, such a
graph potentially exists.
[WAJ] RT-Constrain can only express explicitly "what I want" and the
distributed graph. It can't express explicitly "what I don't want" now, also
the distribution-block graph. Is that right?

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 25 Feb 2021 09:21:55 -0500

Aijun,

> semantic, to identify/control the "excessive routes" in more granular. 

Thanks for the clarification.  That expands the scope of the original
discussion, but might be helpful for the solution.

> all of its clients. This can be achieved by RR

For my point, consider a trivial case:

PE1 --- RR --- PE2

PE2 is the source of the excess routes.
PE1 is the impacted router.
RR has only PE1 and PE2 as peers.

By your current text, RR can only do something if PE1 and PE2 say they're
not interested.

It can be observed here that if we have exactly one peer left, we could
signal to that one peer.

I suspect your intent is to cover situations where you can distinguish PE
routers from solely internal route distributuion infrastructure, such as
route reflectors.  So, using a slighly more interesting topology:

PE1 --- RR1 --- RR2 --- PE2

PE2 is the source of the excess routes.
PE1 is the impacted router.
RR1 has only PE1 as a PE device.

If PE1 signals to RR1 that it isn't interested in the offending routes, RR1
may propagate the filter.

The issue here is that BGP does not specifically distinguish whether an
attached BGP Speaker is a PE or not.  The protocol doesn't help you make
this determination.

> the distribution-block graph. Is that right?

That is correct.  But consider the following example:

        PE3
         |
PE1 --- RR1 --- RR2 --- PE2

PE2 is the source of the excess routes for VPN RT-A.
PE1 is the impacted router.
RR1 has PE1 and PE3 as attached PE routers.
All routers are configured to use RT-Constrain
PE1 and PE2 source RT-Constrain routes for RT-A.  PE3 does not.

If PE1 signals to RR1 that it isn't interested in the offending routes, and
that signal not only includes the offending RD, but also the matching RTs,
RR1 can determine that there are no further interested receivers for that
RD+RT and propagate that to RR2.

RT-Constrain provides the ability to figure out what the interested
receivers are for a VPN defined by a route-target.  This provides a
restricted subset of the attached routers for propagation purposes.

-- Jeff

---

From: Aijun Wang <[email-masked]>
Date: Fri, 26 Feb 2021 07:20:45 +0800

Hi, Jeff:
Thanks for your suggestions!
I think combine RTC and RD-ORF information can accomplish the fine control for the upstream propagation of unwanted VPN routes.
We will also try to find other ways to achieve the same effect.

Aijun Wang
China Telecom

> On Feb 25, 2021, at 22:01, Jeffrey Haas <[email-masked]> wrote:
> 
> Aijun,
>

---

From: Gyan Mishra <[email-masked]>
Date: Thu, 25 Feb 2021 22:52:25 -0500

Hi Jeff

Keep in mind that RTC is an optimization that does not apply to every
scenario especially in cases where all PEs have all the same VPNs defined
with explicit RT import.  The special cases were RTC does apply is for
incongruent VRFs in cases of a sparse RT distribution graph where PEs don’t
all import the same RTs.  By default all PE have RT filtering enabled by
default meaning if their is not an explicit VRF definition to match and
import the RT, the RT is dropped.  With RTC the distribution graph of what
RTs are advertised RR-PE is now optimized with RT membership, and now  only
RTs explicitly imported by the PEs are now advertised by the RR to PE for
processing.  In the case of RD-ORF, it provides a layer of needed
granularity as the RD PE originator of the flood is a subset of all the VPN
prefixes represented by the RT permitted by RTC to be advertised to the
PE.  The RD-ORF now dynamically blocks the offending PE prefixes that would
otherwise have been advertised by RTC RR to PE and accepted by the PE for
processing and added to the VRF RIB being overloaded.

Kind Regards

Gyan

On Thu, Feb 25, 2021 at 6:21 PM Aijun Wang <[email-masked]> wrote:

> I think combine RTC and RD-ORF information can accomplish the fine control
> for the upstream propagation of unwanted VPN routes.
-- 

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*M 301 502-134713101 Columbia Pike *Silver Spring, MD

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 26 Feb 2021 11:47:32 +0100

Gyan,

What you said is pretty obvious I think to everyone here.

But I think the point of bringing RTC into this picture was a bit
different.

Just imagine RTC is running between PE and RR and signals interest in set
of RTs to the RR. Let's say those RTs are RT100, RT200, RT300, RT400, RT500

Further imagine one VRF importing RT200 overflows on said PE with incoming
(pre-import routes of RD_FOO and RT100, RT200 and RT 300).

So what if now PE via RTC will simply readvertise interest to get RT200
with new BGP attribute say called BLOCK_RD and lists in such attribute body
Route Distinguisher of RD_FOO ?

The overall mission accomplished. No need for any new ORF type. It can be
even transitive if we apply a little hack and adjust the way RR will select
the best path for RTC (meaning include this attribute in advertisements
only if all peers sending RT200 attached it with such RD_FOO).

To me this is very simply RTC extension and it seems solves this entire
"problem".

And no - by proposing the above I am not changing my mind that it is simply
wrong network operation to even allow such problem to happen in the first
place.

Best,
R.

On Fri, Feb 26, 2021 at 4:52 AM Gyan Mishra <[email-masked]> wrote:

>> I think combine RTC and RD-ORF information can accomplish the fine
>> control for the upstream propagation of unwanted VPN routes.

---

From: Aijun Wang <[email-masked]>
Date: Fri, 26 Feb 2021 19:26:48 +0800

Hi, Robert:

Your proposal just let RTC act as the behavior of RD-ORF. 
I think no one will agree with you.

I think what Jeff’s proposal is to utilize the information from RTC to let the RR decide when to send out the RD-ORF message on behalf of its peers, not to change the behavior of RTC.

Aijun Wang
China Telecom

> On Feb 26, 2021, at 18:48, Robert Raszuk <[email-masked]> wrote:
> 
> Gyan,
> 

[WAJ] The action on RR then will be advertised VPN routes that with RT200, but the RD part is not RD_FOO?

> only if all peers sending RT200 attached it with such RD_FOO).

[WAJ] Normally RTC message will be flooded within the network immediately . Your proposal here just let it wait responses from all its peers. If so, why using BGP UPDATE message?

>>>>> semantic, to identify/control the "excessive routes" in more granular

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 26 Feb 2021 12:40:38 +0100

Aijun,

> I think no one will agree with you.

I think you did not understand it. But that's ok.

And if you think that it is better to make any routing protocol decision
(in regards to route distribution) by collecting information from more then
one protocol source then indeed no point to discuss this  any further.

Kind regards,
R.

> I think what Jeff’s proposal is to utilize the information from RTC to let
> the RR decide when to send out the RD-ORF message on behalf of its peers,
> not to change the behavior of RTC.
>
> Aijun Wang
> China Telecom
>
> [WAJ] Normally RTC message will be flooded within the network immediately
> . Your proposal here just let it wait responses from all its peers. If so,
> why using BGP UPDATE message?
>
> >
> >>>>>> In the prior discussions, it was clear that remediation was trying
> to
> >>> be

---

From: Gyan Mishra <[email-masked]>
Date: Fri, 26 Feb 2021 10:49:20 -0500

Hi Robert

In the first comment in the thread as a requirement to identify the
Prefixes based on RD to be blocked in the case of not unique RD requires an
AND of both RD+RT in the logic.  I am in complete agreement that is
required.

So I can see Jeffrey’s idea as a possible solution to modify RTC to include
offending RD update to RTC spec.

However, the RD ORF uses existing ORF machinery with add of new ORF type
which to me seems a very minor change.  The nice thing about RD ORF is that
it is controlled and requires manual intervention to unblock where RTC
would be automatic and that could cause undesirable instability.

I am not sure if modifying RTC to include RD block logic would not cause
added instability with RTC as RTC acts only on RT, as I this is a
significant change to RTC membership logic.  Also any existing
implementation of RTC would all be impacted by this change.  With RD ORF as
this is a new type ORF, the ORF policy would have to be explicitly defined
so this would only effect new deployments wanting to take advantage of RD
ORF granularity of filtering.

Gyan

On Fri, Feb 26, 2021 at 5:47 AM Robert Raszuk <[email-masked]> wrote:

> And no - by proposing the above I am not changing my mind that it is
> simply wrong network operation to even allow such problem to happen in the
> first place.
>
> Best,
> R.
>
>
>>> to be
>> *M 301 502-134713101 Columbia Pike
>> <https://www.google.com/maps/search/13101+Columbia+Pike?entry=gmail&source=g> *Silver
>> Spring, MD
>>

*Gyan Mishra*

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 26 Feb 2021 17:27:55 +0100

On Fri, Feb 26, 2021 at 4:49 PM Gyan Mishra <[email-masked]> wrote:

> So I can see Jeffrey’s idea as a possible solution to modify RTC to
> include offending RD update to RTC spec.
>

Just to be clear ... Jeff did not suggest that. This suggestion I wrote.

> would be automatic and that could cause undesirable instability.

That can be manual even for RTC. Consider advertising an RD attribute as a
static route. It will be advertised till operator removes it.

> ORF granularity of filtering.

The attribute wouild be optional transitive attribute. If you do not
recognize you do not use it.

Thx,
R.

>> Further imagine one VRF importing RT200 overflows on said PE with
>> incoming (pre-import routes of RD_FOO and RT100, RT200 and RT 300).
>>

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 26 Feb 2021 16:02:47 -0500

Aijun,

> We will also try to find other ways to achieve the same effect.

The important thing to take from my comments is that a necessary function is
determining when and how you can determine that there are no more interested
receivers in a set of VPN routes by property.  RD-ORF originally had this in
the context of RD.  This draft potentially widens the problem space.

I think you will find that it's difficult to do this. 

The RT-Constrain example mostly is there to provide a more tractable case
than your base case.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 26 Feb 2021 16:08:23 -0500

Gyan,

> processing and added to the VRF RIB being overloaded.

I agree that RT-Constrain does not solve all of the problems, nor is my
intent to suggest that it does.  It does suggest that a hard problem has
some possible answers.

There are two issues here:
1. How does a receiving PE signal that it wants to STOP receiving some
routes?  RD-ORF is an example of that.  

2. How does something that receives the signal from the first item decide
whether it can perform the same procedure "upstream"?  

I believe that the second point will be very hard to solve and urge the
authors to spend time documenting the scenarios.

Alternatively, if the problem to be solved is restricted to the first case,
I think you have a constrained problem that can be readily solved either
through Prefix ORF or even RD-ORF as special case of Prefix-ORF.

-- Jeff

---

From: Gyan Mishra <[email-masked]>
Date: Fri, 26 Feb 2021 18:20:22 -0500

Hi Jeffrey

> intent to suggest that it does.  It does suggest that a hard problem has

   Gyan> Good we are on the same page.

> routes?  RD-ORF is an example of that.

  Gyan> Agreed on the two issues first being how receiving PE signals

> whether it can perform the same procedure "upstream"?

   Gyan> So as you stated their are two main parts to the RD-ORF solution.

Goal #1 Mitigation of all receiver PEs VPNs from overflow.  So RD-ORF
solution resolves all PEs that are signaling RD-ORF for overflow VRF.  The
receiver driven signaling completely mitigates all PEs having an overflow
issue.  So this solves #1 and #2.  We can update the draft to clarify the
receiver driven solution  signaling so the RR does not have to perform the
same procedure upstream.

Goal #2 Stop the source from sending the flood of routes.  This is more
complicated agreed and would require signaling upstream by the RR to find
the source PE based on RD+RT.  In a typical service provider core the PEs
RRC all peer to multiple RRs so the so the source PE would be one of the
PEs peering directly to the RRs so it would not be too difficult I think to
find the source PE based on RD+RT.   We will work on documenting this
scenario better.

> authors to spend time documenting the scenarios.

   Gyan> Agreed.  The second point I think can be solved with receiver
driven signaling so the RR does not have to signal upstream.  We will work
on better documenting Goal #2 to stop the source from flooding.

> through Prefix ORF or even RD-ORF as special case of Prefix-ORF.

    Gyan> The major caveat with Prefix ORF is if the number of prefixes is
a high order of magnitude would make it not feasible such as a 100k route
flood.  Since the RD ORF is meant to mitigate the overflow PE the use case
would always be a very high number of prefixes that could cause resources
exhaustion.

-- 

*Gyan Mishra*

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 27 Feb 2021 00:26:39 +0100

> would always be a very high number of prefixes that could cause resources

RD is nothing else then /64 prefix.

---

From: Gyan Mishra <[email-masked]>
Date: Fri, 26 Feb 2021 18:40:19 -0500

Robert,

Gyan> Anything can be coded but it would be a serious hack to manually
withdraw the RD which is a complete change to current RTC implementation
which is simply enabling SAFI 132 MP Reach capability exchange between
peers.  Nothing manual.

Gyan> Optional transitive attributes are expected in theory not to be
recognized by all implementations, however the optional transitive
attributes that exist today such as community and aggregator for example
are recognized by most all implementations.  So as this is an extended
community, this would be recognized by all implementations.

>>> Just imagine RTC is running between PE and RR and signals interest in
>>> set of RTs to the RR. Let's say those RTs are RT100, RT200, RT300, RT400,
>>> RT500
>>>
>>> The overall mission accomplished. No need for any new ORF type. It can
>>> be even transitive if we apply a little hack and adjust the way RR will
>>> select the best path for RTC (meaning include this attribute in
>>> advertisements only if all peers sending RT200 attached it with such
>>> RD_FOO).
>>>

*Gyan Mishra*

---

From: Aijun Wang <[email-masked]>
Date: Sat, 27 Feb 2021 07:50:44 +0800

Hi, Robert:

We should keep mind that the message “What I want(expressed by RTC)” can be flooded automatically throughout the network, but “What’s I don’t want(expressed by RD-ORF)” should not be flooded out unconditionally.
Try to use RTC to express “What I don’t want” within the network(not just in one device) will change the RTC mechanism dramatically, not just the unrecognized transitive attributes.

On the contrary, ORF based solutions can cover the situations what RTC aimed. Don’t you think so?

Aijun Wang
China Telecom

> On Feb 27, 2021, at 00:28, Robert Raszuk <[email-masked]> wrote:
> 
> 
> 
> 
>> In the first comment in the thread as a requirement to identify the Prefixes based on RD to be blocked in the case of not unique RD requires an AND of both RD+RT in the logic.  I am in complete agreement that is required.
>> 
>> So I can see Jeffrey’s idea as a possible solution to modify RTC to include offending RD update to RTC spec.  
> 
>> However, the RD ORF uses existing ORF machinery with add of new ORF type which to me seems a very minor change.  The nice thing about RD ORF is that it is controlled and requires manual intervention to unblock where RTC would be automatic and that could cause undesirable instability. 
> 
> That can be manual even for RTC. Consider advertising an RD attribute as a static route. It will be advertised till operator removes it. 
> 
> 
>  
>> I am not sure if modifying RTC to include RD block logic would not cause added instability with RTC as RTC acts only on RT, as I this is a significant change to RTC membership logic.  Also any existing implementation of RTC would all be impacted by this change.  With RD ORF as this is a new type ORF, the ORF policy would have to be explicitly defined so this would only effect new deployments wanting to take advantage of RD ORF granularity of filtering. 
> 
> The attribute wouild be optional transitive attribute. If you do not recognize you do not use it. 
> 
> Thx,
> R.
> 
> 
> 
>  
>> 
>> Gyan
>> 
>>> But I think the point of bringing RTC into this picture was a bit different. 
>>> 
>>> Just imagine RTC is running between PE and RR and signals interest in set of RTs to the RR. Let's say those RTs are RT100, RT200, RT300, RT400, RT500
>>> 
>>> Further imagine one VRF importing RT200 overflows on said PE with incoming (pre-import routes of RD_FOO and RT100, RT200 and RT 300). 
>>> 
>>> So what if now PE via RTC will simply readvertise interest to get RT200 with new BGP attribute say called BLOCK_RD and lists in such attribute body Route Distinguisher of RD_FOO ? 
>>> 
>>> The overall mission accomplished. No need for any new ORF type. It can be even transitive if we apply a little hack and adjust the way RR will select the best path for RTC (meaning include this attribute in advertisements only if all peers sending RT200 attached it with such RD_FOO). 
>>> 
>>> To me this is very simply RTC extension and it seems solves this entire "problem". 
>>> 
>>> And no - by proposing the above I am not changing my mind that it is simply wrong network operation to even allow such problem to happen in the first place. 
>>> 
>>> Best,
>>> R.
>>> 
>>> 
>>>> Keep in mind that RTC is an optimization that does not apply to every scenario especially in cases where all PEs have all the same VPNs defined with explicit RT import.  The special cases were RTC does apply is for incongruent VRFs in cases of a sparse RT distribution graph where PEs don’t all import the same RTs.  By default all PE have RT filtering enabled by default meaning if their is not an explicit VRF definition to match and import the RT, the RT is dropped.  With RTC the distribution graph of what RTs are advertised RR-PE is now optimized with RT membership, and now  only RTs explicitly imported by the PEs are now advertised by the RR to PE for processing.  In the case of RD-ORF, it provides a layer of needed granularity as the RD PE originator of the flood is a subset of all the VPN prefixes represented by the RT permitted by RTC to be advertised to the PE.  The RD-ORF now dynamically blocks the offending PE prefixes that would otherwise have been advertised by RTC RR to PE and accepted by the PE for processing and added to the VRF RIB being overloaded.
>>>> 
>>>> 
>>>> Kind Regards 
>>>> 
>>>> Gyan
>>>> 
>>>> Gyan Mishra
>>>> Network Solutions Architect 
>>>> M 301 502-1347
>>>> 13101 Columbia Pike 
>>>> Silver Spring, MD
>>>> 
>> Gyan Mishra
>> Network Solutions Architect 
>> M 301 502-1347
>> 13101 Columbia Pike 
>> Silver Spring, MD
>>

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 27 Feb 2021 01:07:03 +0100

>  So as this is an extended community, this would be recognized by all
> implementations.
>

Nope sorry.

The suggestion was to use new attribute which would include list of RDs
applicable to RTs. RTC NLRI would not change. BGP dynamics remain as is.

It is just the same model as max prefix. Session get's torn down till
operator's intervention unless you configure otherwise so all claims that
we do not do this in BGP are false.

And RD is not an extended community anyway.

Many thx,
R

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 27 Feb 2021 01:16:58 +0100

I am not proposing anything new nor change to RTC dynamics or current
fundamental operation model of RTC.

RTC NLRIs would be flooded just like today however it is just like with
some attributes or non transitive at domain border communities that routers
drop them before propagating over ebgp here we would not propagate by
default this new attribute. Actually it should be non transitive - I am
self correcting myself here.

Here the definition of handling this attribute would be not to propagate
within it such RDs to other ibgp or ebgp peers unless all requesters
indicating interest with given RT also signalled such filtering request for
those RDs. Obviously there should be no need to also propagate such
attribute with an empty list of RDs.

Sure as pointed by Jakob when new peer comes up and he sends interest in
those RT(s) without RD such routes must be sent to him or requested
upstream. All easily doable if and only if there is real need for it.

Thx
R.

On Sat, Feb 27, 2021 at 12:51 AM Aijun Wang <[email-masked]> wrote:

> Hi, Robert:
>
> We should keep mind that the message “What I want(expressed by RTC)” can
> be flooded automatically throughout the network, but “What’s I don’t
> want(expressed by RD-ORF)” should not be flooded out unconditionally.
> Try to use RTC to express “What I don’t want” within the network(not just
> in one device) will change the RTC mechanism dramatically, not just the
> unrecognized transitive attributes.
>

---

From: Aijun Wang <[email-masked]>
Date: Sat, 27 Feb 2021 08:40:33 +0800

Hi, Robert:
If you use non-transitive attributes, every in-path router must decide and add/delete it based on its justification. 
If so, why bother RTC and why use the BGP-Update message? ORF/Route-refresh message is the right mechanism to utilize.

Aijun Wang
China Telecom

> On Feb 27, 2021, at 08:18, Robert Raszuk <[email-masked]> wrote:
> 
> 
> 
> I am not proposing anything new nor change to RTC dynamics or current fundamental operation model of RTC. 
> 
> RTC NLRIs would be flooded just like today however it is just like with some attributes or non transitive at domain border communities that routers drop them before propagating over ebgp here we would not propagate by default this new attribute. Actually it should be non transitive - I am self correcting myself here. 
> 
> Here the definition of handling this attribute would be not to propagate within it such RDs to other ibgp or ebgp peers unless all requesters indicating interest with given RT also signalled such filtering request for those RDs. Obviously there should be no need to also propagate such attribute with an empty list of RDs. 
> 
> Sure as pointed by Jakob when new peer comes up and he sends interest in those RT(s) without RD such routes must be sent to him or requested upstream. All easily doable if and only if there is real need for it. 
> 
> Thx
> R.
>