Subject: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

From: "Susan Hares" <[email-masked]>
Date: Thu, 4 Feb 2021 10:38:23 -0500

This begins a 2 week WG adoption call for draft-wang-idr-rd-orf-05.txt (from
2/4/2021 to 2/18/2021) 

This draft defines a new Outbound Route Filter (ORF) type, called the

Route Distinguisher ORF (RD-ORF).  RD-ORF is applicable when the

routers do not exchange VPN routing information directly (e.g.

routers in single-domain connect via Route Reflector, or routers in

Option B/Option AB/Option C cross-domain scenario).

Please be aware that this draft has one IPR statement attached. 

 <https://datatracker.ietf.org/ipr/4579/>
https://datatracker.ietf.org/ipr/4579/..

Please consider the following questions in your review and comments: 

1) Will this new ORF filter reduce routing information at key points? 

2) Should the WG consider this draft given it has an IPR claim or 

    Would the IDR WG prefer another approach? 

3) Is this draft ready to be adopted and refined as WG draft? 

Cheerily, Susan Hares

---

From: "Aijun Wang" <[email-masked]>
Date: Mon, 8 Feb 2021 10:07:39 +0800

Hi, Susan and All:

As co-author of this draft, I support its adoption. 

The related questions are answered below inline[WAJ].

Thanks in advance for your efforts.

Best Regards

Aijun Wang

China Telecom

From: [email-masked] <[email-masked]> On Behalf Of Susan Hares
Sent: Thursday, February 4, 2021 11:38 PM
To: [email-masked]
Subject: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021
to 2/18/2021)

[WAJ] Yes, it will filter dynamically the overwhelming VPN routing
information at the other side of the BGP peer, preserve the serving
capabilities for other VPNs on the local router.

[WAJ] Yes.

[WAJ] Other approaches are welcome, except the approaches that has been
discussed on the mail list, or mentioned in the draft. 

We are also welcome IDR experts to make contributions to refine the current
solution.

[WAJ] Yes. The draft has been presented twice on the IETF meetings,
discussed intensely in the mail list and updated several versions to address
the comments from WG mail list.  It will be refined after becoming the WG
draft.

Cheerily, Susan Hares

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Wed, 10 Feb 2021 07:34:49 +0000

Hi Sue,

I support the adoption as a coauthor. Please see my replies to the questions inline:

From: Idr [mailto:[email-masked]] On Behalf Of Susan Hares
Sent: Thursday, February 4, 2021 11:38 PM
To: [email-masked]
Subject: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

This begins a 2 week WG adoption call for draft-wang-idr-rd-orf-05.txt (from 2/4/2021 to 2/18/2021)

Yes

Yes

2)      Is this draft ready to be adopted and refined as WG draft?

Yes, it defines a generic RD based filtering mechanism, which may be used in use cases not limited to the ones described in the current draft. More refinement could be done after the adoption.

Best regards,
Jie

Cheerily, Susan Hares

---

From: Gyan Mishra <[email-masked]>
Date: Wed, 10 Feb 2021 02:46:42 -0500

I support WG Adoption as co-author.

Yes this feature filters dynamically and provides a  granular approach to
filter by RD originator instead of by RT.

2) Should the WG consider this draft given it has an IPR claim

Yes

or

We have reviewed other approaches and this provides a finer grain approach
to dynamically mitigate VPN being overwhelmed by source PE RD offender.  We
have discussed other approaches and best practice such as VPN maximum
prefix as well as peer maximum prefix, however this feature resolves a gap
directly related to offending PE RD originator.

3) Is this draft ready to be adopted and refined as WG draft?
This draft is well written and solves a critical problem and I believe is
mature for adoption.

Thanks

Gyan
On Wed, Feb 10, 2021 at 2:35 AM Dongjie (Jimmy) <[email-masked]> wrote:

> *From:* Idr [mailto:[email-masked]] *On Behalf Of *Susan Hares
> *Sent:* Thursday, February 4, 2021 11:38 PM
> *To:* [email-masked]
> *Subject:* [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt
> (2/4/2021 to 2/18/2021)
>
>
>
> Yes, it defines a generic RD based filtering mechanism, which may be used
> in use cases not limited to the ones described in the current draft. More
> refinement could be done after the adoption.
>
>
>
> Best regards,
>
> Jie
>
>
>
> Cheerily, Susan Hares
>
>
>
>
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr
>
-- 

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*M 301 502-134713101 Columbia Pike *Silver Spring, MD

---

From: Zhuangshunwan <[email-masked]>
Date: Wed, 10 Feb 2021 07:54:01 +0000

Hi Sue and All,

I support the adoption as a co-author.

[Shunwan] Yes.

2)      Should the WG consider this draft given it has an IPR claim or
[Shunwan] Yes.
  Would the IDR WG prefer another approach?
[Shunwan] We've already discussed various options in IDR list, the current solution is an optimization solution based on the community's input. We welcome the IDR community's more comments to improve the solution.

3)      Is this draft ready to be adopted and refined as WG draft?
[Shunwan] Yes. I think this draft is in a good shape, It describes the practical problems and provides a reasonable solution, and we could continue to improve this solution.

Best Regards,
Shunwan

Cheerily, Susan Hares

---

From: "Wanghaibo (Rainsword)" <[email-masked]>
Date: Wed, 10 Feb 2021 08:28:14 +0000

Hi Sue and All,

2)      Should the WG consider this draft given it has an IPR claim or
Would the IDR WG prefer another approach?
Yes,
3) Is this draft ready to be adopted and refined as WG draft?
Yes, the extension is useful for this scenario, and we may explore more scenes with it.

Best Regards,
Haibo

Cheerily, Susan Hares

---

From: Lizhenbin <[email-masked]>
Date: Wed, 10 Feb 2021 09:47:03 +0000

Hi All,

I support the adoption.
1) Yes
2) Yes
3) Yes

Best Regards,
Zhenbin (Robin)

From: Idr [mailto:[email-masked]] On Behalf Of Susan Hares
Sent: Thursday, February 4, 2021 11:38 PM
To: [email-masked]<mailto:[email-masked]>
Subject: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Cheerily, Susan Hares

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Wed, 10 Feb 2021 12:34:37 +0000

Do Not Support.

Thanks,
              Jim Uttaro

From: Idr <[email-masked]> On Behalf Of Lizhenbin
Sent: Wednesday, February 10, 2021 4:47 AM
To: Susan Hares <[email-masked]>; [email-masked]
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Hi All,

I support the adoption.
1) Yes
2) Yes
3) Yes

Best Regards,
Zhenbin (Robin)

https://datatracker.ietf.org/ipr/4579/<https://urldefense.com/v3/__https:/datatracker.ietf.org/ipr/4579/__;!!BhdT!3JFUDaX3WxfXuq0-jDevFeHNv4xKPBpOZ-cktrZ8Hg9b1Ipaj0PIHJh3ZFw$>..

Cheerily, Susan Hares

---

From: Aijun Wang <[email-masked]>
Date: Wed, 10 Feb 2021 22:17:00 +0800

Hi, Jim:

Would you like to elaborate your considerations in detail?
We can try to address your concerns via the mail list or updating the draft if you have reasonable comments.

Thanks in advance.

Aijun Wang
China Telecom

> On Feb 10, 2021, at 20:35, UTTARO, JAMES <[email-masked]> wrote:
> 
> 
> Do Not Support.
>  
> Thanks,
>               Jim Uttaro
>

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Wed, 10 Feb 2021 15:01:56 +0000

TBH looks like a solution in search of a problem.

The tools used for the last 20+ yrs. to prevent a mis-behaving CE or peer are proven.

An example:

“When the VRF of VPN1 in PE1 overflows, due to PE1 and other PEs are
   not iBGP neighbors, BGP Maximum Prefix Features cannot work, so the
   problem on PE2 cannot be known.”

My take on this very first premise is that a VRF ( VPN1 ) on PE1 is being overwhelmed with routes. These routes are not coming from the RR topology. So what is the scenario ? Is it a redistribution from VRF ( VPN2 )->VRF( VPN1 )? If so, you have a serious security issue.. Is it redistribute from another protocol, the GRT?

I disagree with your premise that the current suite of mitigations are not sufficient..  There are certain assertions which are subjective and not appropriate. AN example:

“4) Configure the Maximum Prefix for each VRF on edge nodes

   When a VRF overflows, it stops the import of routes and log the extra
   VPN routes into its RIB.  However, PEs should parse the BGP updates.
   These processes will cost CPU cycles and further burden the
   overflowing PE.”

What does this mean? There are networks that have been parsing millions of BGP Updates and routes. I have seen VRF overflows and they have been specific to said customer’s VRF. Where are you coming up with this and other broad assertions.

From a solution point of view I do not believe in overloading the object. IOW RD is meant to convey uniqueness of a given path through a topology, overloading it with this semantic means what to the existing semantic.. As an ex, some operators use unique RD so there is no path hiding, eibgp load balancing etc.. Generally speaking I do not want my RRs making a best path decision for a given path. Some do not use unique RD for there own reasons including scale, possible alignment of RR topology to underlying best path etc…

In order to use this technology what are the requirements in terms of assigning RDs?

Thanks,
              Jim Uttaro

From: Aijun Wang <[email-masked]>
Sent: Wednesday, February 10, 2021 9:17 AM
To: UTTARO, JAMES <[email-masked]>
Cc: Lizhenbin <[email-masked]>; Susan Hares <[email-masked]>; [email-masked]
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Hi, Jim:

Thanks in advance.

Aijun Wang
China Telecom

On Feb 10, 2021, at 20:35, UTTARO, JAMES <[email-masked]<mailto:[email-masked]>> wrote:

Do Not Support.

Thanks,
              Jim Uttaro

From: Idr <[email-masked]<mailto:[email-masked]>> On Behalf Of Lizhenbin
Sent: Wednesday, February 10, 2021 4:47 AM
To: Susan Hares <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Hi All,

I support the adoption.
1) Yes
2) Yes
3) Yes

Best Regards,
Zhenbin (Robin)

Cheerily, Susan Hares

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!BhdT!yysdkGCNAgoHWvY47NKedcAHI_Yf6MRpOT4zgCSp7HcB9m4Q73F8OU36k5sDVys$>

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 10 Feb 2021 17:22:36 +0000

I support the WG adoption.
Answers to the questions posted by the Chair:

1)      Will this new ORF filter reduce routing information at key points?
Yes, the proposed approach prevents more routes sent to a PE whose VRF overflows.

2)      Should the WG consider this draft given it has an IPR claim or Would the IDR WG prefer another approach?
Yes,  because the IPR has  FRAND royalty-bearing licenses

Yes.

Some questions to the Author:

1)      On Page 3 (right below Figure 1), you have:
"When the VRF of VPN1 in PE1 overflows, due to PE1 and other PEs are
not iBGP neighbors, BGP Maximum Prefix Features cannot work, so the
problem on PE2 cannot be known"

Do you mean that "the problem on PE1 cannot be known" to others?

2)      2 Does PE1 have preconfigured "BGP Maximum prefixes" expected from others? If more than expected prefixes are received, is it because other PEs' fault in sending trashing prefixes? Or is it because the PE1 configuration error?

3)      RD-ORF ROUTE-REFRESH message is propagated to all other PEs, Do all the PEs receiving the message stop sending "that VPN route" (as described in 5.1.3?

Linda Dunbar

From: Idr <[email-masked]> On Behalf Of Wanghaibo (Rainsword)
Sent: Wednesday, February 10, 2021 2:28 AM
To: Susan Hares <[email-masked]>; [email-masked]
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Hi Sue and All,

Best Regards,
Haibo

https://datatracker.ietf.org/ipr/4579/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fipr%2F4579%2F&data=04%7C01%7Clinda.dunbar%40futurewei.com%7Cf05240296188408116d608d8cd9ddfdb%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637485425256503393%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z0HESRyWp%2FKF2Jn7mJe6V8yqMgwYnnt1YhfmKPXR6cw%3D&reserved=0>..

Cheerily, Susan Hares

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 10 Feb 2021 17:58:14 +0000

Jim,

You said
“Generally speaking I do not want my RRs making a best path decision for a given path.”
Isn’t RFC4684 (Constrained Route Distribution) doing the same? i.e. allowing RR to constraint the route distribution to some nodes?

Linda Dunbar
From: Idr <[email-masked]> On Behalf Of UTTARO, JAMES
Sent: Wednesday, February 10, 2021 9:02 AM
To: Aijun Wang <[email-masked]>
Cc: [email-masked]; Susan Hares <[email-masked]>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

An example:

Thanks,
              Jim Uttaro

From: Aijun Wang <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, February 10, 2021 9:17 AM
To: UTTARO, JAMES <[email-masked]<mailto:[email-masked]>>
Cc: Lizhenbin <[email-masked]<mailto:[email-masked]>>; Susan Hares <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Hi, Jim:

Thanks in advance.

Aijun Wang
China Telecom

Do Not Support.

Thanks,
              Jim Uttaro

Hi All,

I support the adoption.
1) Yes
2) Yes
3) Yes

Best Regards,
Zhenbin (Robin)

https://datatracker.ietf.org/ipr/4579/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fdatatracker.ietf.org%2Fipr%2F4579%2F__%3B!!BhdT!3JFUDaX3WxfXuq0-jDevFeHNv4xKPBpOZ-cktrZ8Hg9b1Ipaj0PIHJh3ZFw%24&data=04%7C01%7Clinda.dunbar%40futurewei.com%7Cf60eff6157cf443002f808d8cdd4ea11%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637485661658520629%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=V8BLv2ehjuO5B5gwcVNl8GfE8GjAoXfN7Vjy0huOrLA%3D&reserved=0>..

Cheerily, Susan Hares

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fidr__%3B!!BhdT!yysdkGCNAgoHWvY47NKedcAHI_Yf6MRpOT4zgCSp7HcB9m4Q73F8OU36k5sDVys%24&data=04%7C01%7Clinda.dunbar%40futurewei.com%7Cf60eff6157cf443002f808d8cdd4ea11%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637485661658520629%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=Lh3QmctDwsAWCcPVxUcsPwfkS2VhpRsGz5X2hjfajQk%3D&reserved=0>

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Wed, 10 Feb 2021 18:37:27 +0000

RT Constrain constrains route distribution to the part of the RR topology needed to service a given VPN. RR not making a best path decision means every path for a [  PE,VRF,Path ] has a unique RD.

Thanks,
              Jim Uttaro

From: Linda Dunbar <[email-masked]>
Sent: Wednesday, February 10, 2021 12:58 PM
To: UTTARO, JAMES <[email-masked]>; Aijun Wang <[email-masked]>
Cc: [email-masked]; Susan Hares <[email-masked]>
Subject: RE: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Jim,

You said

Linda Dunbar
From: Idr <[email-masked]<mailto:[email-masked]>> On Behalf Of UTTARO, JAMES
Sent: Wednesday, February 10, 2021 9:02 AM
To: Aijun Wang <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Susan Hares <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

An example:

Thanks,
              Jim Uttaro

Hi, Jim:

Thanks in advance.

Aijun Wang
China Telecom

Do Not Support.

Thanks,
              Jim Uttaro

Hi All,

I support the adoption.
1) Yes
2) Yes
3) Yes

Best Regards,
Zhenbin (Robin)

https://datatracker.ietf.org/ipr/4579/<https://urldefense.com/v3/__https:/nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fdatatracker.ietf.org*2Fipr*2F4579*2F__*3B!!BhdT!3JFUDaX3WxfXuq0-jDevFeHNv4xKPBpOZ-cktrZ8Hg9b1Ipaj0PIHJh3ZFw*24&data=04*7C01*7Clinda.dunbar*40futurewei.com*7Cf60eff6157cf443002f808d8cdd4ea11*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637485661658520629*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000&sdata=V8BLv2ehjuO5B5gwcVNl8GfE8GjAoXfN7Vjy0huOrLA*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!BhdT!2N_GSVDW-NhGTatlTFrOi542rp-Kd5jn4PLVovOJAevqm3bOFixcrOU4ZBxA7g4$>..

Cheerily, Susan Hares

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr<https://urldefense.com/v3/__https:/nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fidr__*3B!!BhdT!yysdkGCNAgoHWvY47NKedcAHI_Yf6MRpOT4zgCSp7HcB9m4Q73F8OU36k5sDVys*24&data=04*7C01*7Clinda.dunbar*40futurewei.com*7Cf60eff6157cf443002f808d8cdd4ea11*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C637485661658520629*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C2000&sdata=Lh3QmctDwsAWCcPVxUcsPwfkS2VhpRsGz5X2hjfajQk*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!BhdT!2N_GSVDW-NhGTatlTFrOi542rp-Kd5jn4PLVovOJAevqm3bOFixcrOU4wqNnStI$>

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Wed, 10 Feb 2021 19:24:07 +0000

Hi Sue, IDR WG,

I agree with Jim Uttaro with respect to the use case being weak and already solved with other mechanisms.

Also, there was much opposition to changing the RD semantics and using it for route filtering. See:

https://mailarchive.ietf.org/arch/browse/idr/?q=%22wang-idr-rd-orf%22

I don’t see that this has changed and, additionally, this will add further complexity to BGP route filtering dynamics.
Thanks,
Acee

Cheerily, Susan Hares

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 10 Feb 2021 21:51:31 +0100

Dear Sue & WG,

Technically the solution is broken - you can't filter based on RD when
single VRF overflows due to simple fact that arriving routes at a PE with
one RD are typically imported locally to many different VRFs which may be
running just fine. That to me is sufficient to dismiss this proposal.

I am not even going to point out that for multihomed sites injecting both
aggregates and more specifics + doing eiBGP load balancing or sharing
similar mixed reachability with Inter-AS option B could  form a rather ugly
data plane behaviour.

But putting those (one could say subjective claims) aside procedurally what
is important here is that *entire*  protocol extension this draft is
attempting to define is already defined for a long time in a RFC ... namely
RFC5292 -> https://tools.ietf.org/html/rfc5292

So it would be pretty bad precedence to now define subset of it in other
RFC. And what if both ORF types are used together ? Hint: VPNv4/v6
PREFIX==RD+NET

At best this draft could be turned into an informational document titled:
"How to shoot yourself in the foot while using prefix ORF to filter VPN
routes".  which I would support adoption of.

Kind regards,
Robert

On Wed, Feb 10, 2021 at 8:24 PM Acee Lindem (acee) <acee=
[email-masked]> wrote:

> I don’t see that this has changed and, additionally, this will add further
> complexity to BGP route filtering dynamics.
>
> Thanks,
>
> Acee
>
>
>

---

From: Aijun Wang <[email-masked]>
Date: Thu, 11 Feb 2021 08:00:19 +0800

Hi, Jim:

Aijun Wang
China Telecom

> On Feb 10, 2021, at 23:02, UTTARO, JAMES <[email-masked]> wrote:
> 
> 
[WAJ] I think you may not get the key scenarios that the draft aims to solve.
> The tools used for the last 20+ yrs. to prevent a mis-behaving CE or peer are proven.
[WAJ]Yes, we also analyze them in the introduction parts of this document, they don’t fit the described scenarios.
> My take on this very first premise is that a VRF ( VPN1 ) on PE1 is being overwhelmed with routes. These routes are not coming from the RR topology.
[WAJ] Yes, RR is just reflector.
> So what is the scenario ? Is it a redistribution from VRF ( VPN2 )->VRF( VPN1 )? If so, you have a serious security issue.. Is it redistribute from another protocol, the GRT?
[WAJ] There are many reasons for the misbehaving PE to send overwhelming VPN routes. The RD-ORF mechanism just want to limit its bad influences to the least.
> What does this mean? There are networks that have been parsing millions of BGP Updates and routes.
[WAJ] Controlling the excess BGP updates is the primary aim of ORF mechanism. RD-ORF is just one new type of ORF.

> I have seen VRF overflows and they have been specific to said customer’s VRF. Where are you coming up with this and other broad assertions.
[WAJ] The overflow of customer VRF can be solved via the BGP Maximum Prefix mechanism. RD-ORF is not for such situation.
> In order to use this technology what are the requirements in terms of assigning RDs?
[WAJ]RD-ORF does not overloading the semantics of RD. It has not specific requirements for assigning RD. The RD can be unique for a VPN on each PE, or keep same for the same VPN on different PEs.

---

From: Aijun Wang <[email-masked]>
Date: Thu, 11 Feb 2021 08:16:20 +0800

Hi, Acee:
Would you also to refer to my replies to Jim also based on you have same opinions with him.
Some additional replies are inline below[WAJ].

Aijun Wang
China Telecom

> On Feb 11, 2021, at 03:24, Acee Lindem (acee) <acee=[email-masked]> wrote:
> 
> 
> Hi Sue, IDR WG,
>  
[WAJ] Please refer my replies to Jim.
>  
> Also, there was much opposition to changing the RD semantics and using it for route filtering.
[WAJ] The RD semantic is not changed.
> See:
[WAJ] These are just technical discussions.

> I don’t see that this has changed and,

[WAJ] The document has updated several versions upon the comments from mail list. Would you like to point out which comments is still yet addressed?

> additionally, this will add further complexity to BGP route filtering dynamics.
[WAJ]The RD-ORF mechanism is straightforward to be understood. Your complex feeling may come from it can apply several complex scenarios? I think this is its technical advantage.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 11 Feb 2021 00:22:50 +0000

I agree with Robert.
The draft is not necessary.
Existing ORF from RF5292 can be used by using an address-prefix ORF in the VPN address family with prefix length 64.
The length 64 covers the RD portion of the prefix.
The elements of procedure in the draft are new. However, an RFC is not needed for that.
It is a local feature on each router.

But really, the overwhelmed router can just drop its own routes.
What difference does it make to send an ORF?
ORF seems to be a very little used feature.
The only question I remember getting about ORFs is "How can we limit the number of ORFs we accept from a neighbor?".
Operators don't like getting ORFs. They worry that it stresses their box.

Regards,
Jakob.

From: Idr <[email-masked]> On Behalf Of Robert Raszuk
Sent: Wednesday, February 10, 2021 12:52 PM
To: Susan Hares <[email-masked]>
Cc: [email-masked]; Acee Lindem (acee) <acee=[email-masked]>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Dear Sue & WG,

Technically the solution is broken - you can't filter based on RD when single VRF overflows due to simple fact that arriving routes at a PE with one RD are typically imported locally to many different VRFs which may be running just fine. That to me is sufficient to dismiss this proposal.

I am not even going to point out that for multihomed sites injecting both aggregates and more specifics + doing eiBGP load balancing or sharing similar mixed reachability with Inter-AS option B could  form a rather ugly data plane behaviour.

But putting those (one could say subjective claims) aside procedurally what is important here is that *entire*  protocol extension this draft is attempting to define is already defined for a long time in a RFC ... namely RFC5292 -> https://tools.ietf.org/html/rfc5292

So it would be pretty bad precedence to now define subset of it in other RFC. And what if both ORF types are used together ? Hint: VPNv4/v6 PREFIX==RD+NET

At best this draft could be turned into an informational document titled: "How to shoot yourself in the foot while using prefix ORF to filter VPN routes".  which I would support adoption of.

Kind regards,
Robert

On Wed, Feb 10, 2021 at 8:24 PM Acee Lindem (acee) <acee=[email-masked]<mailto:[email-masked]>> wrote:
Hi Sue, IDR WG,

Cheerily, Susan Hares

---

From: Aijun Wang <[email-masked]>
Date: Thu, 11 Feb 2021 08:38:45 +0800

Hi, Linda:
Thanks for your support!
The answer to your questions are inline below.[WAJ]

Aijun Wang
China Telecom

> On Feb 11, 2021, at 01:23, Linda Dunbar <[email-masked]> wrote:
> 
> 
> problem on PE2 cannot be known”
>  
> Do you mean that “the problem on PE1 cannot be known” to others?  
[WAJ] You are right. I think this is a typo. Will update in next version.
>  
> 2)      2 Does PE1 have preconfigured “BGP Maximum prefixes” expected from others?
[WAJ] “BGP Maximum prefixes can only be configured between BGP peers. In the RR topology(Figure 1) scenario, such mechanism can only be applied between PE1 and RR, which can’t prevent the overflow of specified VPN routes.
> If more than expected prefixes are received, is it because other PEs’ fault in sending trashing prefixes? Or is it because the PE1 configuration error?
[WAJ] We are considering mainly the other PE’s fault in sending trash prefixing.

> 3)      RD-ORF ROUTE-REFRESH message is propagated to all other PEs, Do all the PEs receiving the message stop sending “that VPN route” (as described in 5.1.3?
[WAJ] No, such message is only transferred only to the overflowing PE, that is, the PE that emit the most part of VPN routes.

---

From: Aijun Wang <[email-masked]>
Date: Thu, 11 Feb 2021 09:36:08 +0800

Hi, Jakob and Robert:

Aijun Wang
China Telecom

> On Feb 11, 2021, at 08:23, Jakob Heitz (jheitz) <jheitz=[email-masked]> wrote:
> 
> 
> I agree with Robert.
[WAJ]  Would you like to see https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/15-mt/irg-15-mt-book/irg-oubound-route-filtering.html for the usage of BGP prefixes based ORF that provided by Cisco. In this example, we focus mainly on the real portion of the VPN routes(the prefix-list itself), there is no RD portion mentioned.
Theoretically, the RD is part of the prefix, but don’t you think reuse the prefix based ORF to accomplish the RD-ORF effects changes the semantics of its main usage and arise chaos in implementation?
We should also know that even we reuse the semantic of prefix based ORF, there is no procedure to achieve the same effects that described in this draft. The usage of prefix based ORF is coming from the static, in advance configuration, which is not the aim of RD-ORF.

> It is a local feature on each router.
[WAJ] ORF mechanism is not only the local feature on each router. It is the negotiation feature between peer router.
> What difference does it make to send an ORF?
[WAJ] Once sent, the overwhelmed router need not do the unnecessary BGP updates parsing then.

> Operators don't like getting ORFs. They worry that it stresses their box.
[WAJ] Actually, the solution come mainly from the Option B/C inter-AS VPN deployment scenarios. For MPLS, the Option B/C inter-AS VPN are not deployed widespread, one reason I think is the control of VPN routes propagation within each VPN.(there are maybe other reasons, for example the distribution of MPLS label).
But for EVPN/VXLAN or EVPN/SRv6, the flexibility of  inter-AS deployment is its advantage. Based on such judgement, we think along the widespread of Option B/C deployment, the control mechanism for the restraining of overwhelm VPN routes should be emergency.

Additional replies to Robert are inline below[WAJ]

> Technically the solution is broken - you can't filter based on RD when single VRF overflows due to simple fact that arriving routes at a PE with one RD are typically imported locally to many differ…
[WAJ] As Jim also mentioned, the redistribution between different VRFs is poor design and we should avoid it in first place.
Even for your situation, the document has already mentioned possible solution(in the end of section 5.1.1) which can be further investigated if necessary. We are also welcome your contribution to the case that you are concerning.
> I am not even going to point out that for multihomed sites injecting both aggregates and more specifics + doing eiBGP load balancing or sharing similar mixed reachability with Inter-AS option B cou…
[WAJ] This is related to the complex traffic engineering and need other tools or AI mechanism to solve. The IETF is just providing the tool to assist the aim achievement. Don’t you think so?
> So it would be pretty bad precedence to now define subset of it in other RFC. And what if both ORF types are used together ? Hint: VPNv4/v6 PREFIX==RD+NET
[WAJ] Please see the above explanation to Jakob.

> At best this draft could be turned into an informational document titled: "How to shoot yourself in the foot while using prefix ORF to filter VPN routes".  which I would support adoption of.  
[WAJ] Can consider adding one section to describe the consequences when using prefix ORF to filter VPN routes if necessary.
Or, is the responses for Jakob addressing your concern?
>  
> Kind regards,
> Robert
>  
> On Wed, Feb 10, 2021 at 8:24 PM Acee Lindem (acee) <acee=[email-masked]> wrote:
> Hi Sue, IDR WG,
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 11 Feb 2021 02:48:55 +0000

The Cisco implementation is not the only possible embodiment of an RFC.
Cisco does not prevent you from using an encoding that an RFC allows.

The RFC does not specify what is to trigger the emission of the ORF.
It may be by static configuration or as a result of your procedure.

In your draft, each BGP speaker makes an independent decision:
If a speaker has no use for a set of routes prefixed by a specific RD
in a specific address family, it sends an ORF to the originators of
that set of routes. "use for a set of routes" means either for its own
use or to propagate to other neighbors. The speaker can decide
that it has no use for a set of routes based upon either style of ORF.
The ORF needs no special semantic to make that work.

Regards,
Jakob.

From: Idr <[email-masked]> On Behalf Of Aijun Wang
Sent: Wednesday, February 10, 2021 5:36 PM
To: Jakob Heitz (jheitz) <jheitz=[email-masked]>
Cc: [email-masked]; Susan Hares <[email-masked]>; Robert Raszuk <[email-masked]>; Acee Lindem (acee) <acee=[email-masked]>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Hi, Jakob and Robert:

Aijun Wang
China Telecom

On Feb 11, 2021, at 08:23, Jakob Heitz (jheitz) <jheitz=[email-masked]<mailto:jheitz=[email-masked]>> wrote:

I agree with Robert.

Regards,
Jakob.

From: Idr <[email-masked]<mailto:[email-masked]>> On Behalf Of Robert Raszuk
Sent: Wednesday, February 10, 2021 12:52 PM
To: Susan Hares <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Acee Lindem (acee) <acee=[email-masked]<mailto:acee=[email-masked]>>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Dear Sue & WG,

Kind regards,
Robert

Cheerily, Susan Hares

---

From: Gyan Mishra <[email-masked]>
Date: Thu, 11 Feb 2021 00:53:44 -0500

Hi Jim

Comments below that may help explain the gap and problem this draft solves.

BGP peer Maximum prefix can be used to limit prefixes accepted by PE on a
PE-CE *peer to help with overwhelming a VRF in theory.  The problem with
maximum prefix is it is hard to set a limit on the maximum due to resources
statistical multiplexing that all PE-CE links  of VPNs won’t flood
simultaneously thus maximum prefix is either set very high or not set at
all.*

VRF maximum prefix can be used as well to control resource exhaustion on a
PE.  However when the VRF maximum is reached and prefixes are clipped and
dropped from the RIB their is CPU processing impact.   On the flip side - T*he
problem with VRF maximum prefix as well is it is hard to set a limit on the
maximum due to resources statistical multiplexing high customer SLAs that
all VPNs won’t flood and overflow simultaneously the ** maximum prefix is
set very high for customer SLA and as a result could still result in
resource exhaustion on the PE,*

RD Discussion:

The main purpose of the RD is to provide uniqueness to a prefixes imported
into a VRF.
RD also provides the ability to have all paths to be reflected by the RR
using unique RD.  Also unique RD allows for load balancing.  Non unique RD
is a very rare deployment and in general is not used on all PEs that import
prefixes for a VRF as that prevents the RR from advertising all unique
paths for the same prefix redundant paths, resulting in all paths being
reflected to the PEs to not be reflected to other PEs breaking redundancy.

The general best practice by all operators is to use a unique RD per PE
that is importing prefixes for a VRF for redundancy if redundancy exists in
the topology with multiple exit points multi homed CEs.

As per the best practice rule of thumb RD type 1, 6 byte extended community
X:Y value is used which is a 4 byte IP:2 byte AS which can be accomplished
with Auto RD feature which sets the X value to 4 byte IP to the router
loopback and Y value to sequence number digit that increments 0-65535 for
each VRF defined on the PE.

So when RD Auto is used each PE has a unique RD within the same VPN and for
each VPN the Y value is incremented so the RD is unique per VPN on a PE as
well.

RFC 4684 RTC is an optimization used for incongruent VRFs defined on PEs
where all PEs don’t have the same set of VPNs or VRFs defined.
By Default - all PEs have RT filtering enabled by Default so if a PE does
not have an explicit import of an RT the PE will automatically drop all RTs
not explicitly imported called “RT Filtering”.

RTC uses a concept of VRF membership advertisement sent by PE to RR to tell
the RR what VRFs it is a member of so that only those interesting RTs are
sent by the RR reflected to the PE.  RTC filters on uninteresting VRFs RTs
that would get dropped anyway and does not filter on interesting VRF or RT
as the interesting VRF is part of the VRF membership and that is allowed
and not filtered.

So what this does is save on RR to PE reflection of all RTs waste of
bandwidth and so not the RTs of interest to the PE but the uninteresting
RTs which result in being dropped.

As RTC is a RR to PE reflection optimization it is completely orthogonal
and not related to prevention method of VRF overloading issue.

So if a PE VRF is overflowing we cannot filter prefixes based on RT for the
overflowing VRF as that would filter all routes.  We need a a better
granularity in filtering.

So with RD ORF the idea is that to prevent VRF VPN overflow on a PE that
now if a rouge offending  PE is overflowing sending flood of prefixes
causing other PEs to overflow we can now filter that PE for that particular
VPN or VPN causing the overflow by filtering on the PE RD originator of the
VRF prefixes using the Type 1 RD ORF filter.  I mention type 1 RD as that
is the most common.  Any RD type can be used as long as unique.

As pertains to this draft we may want to update and say that the RD has to
be unique on each PE and cannot use a non unique RD as the RD ORF filter
would act like an RT import block and drop all routes.

On Wed, Feb 10, 2021 at 10:02 AM UTTARO, JAMES <[email-masked]> wrote:

> *TBH looks like a solution in search of a problem. *
>
>
>
> *The tools used for the last 20+ yrs. to prevent a mis-behaving CE or peer
> are proven. *
>
>
>
> *An example:*
>
>
>
> *“When the VRF of VPN1 in PE1 overflows, due to PE1 and other PEs are*
>
> *   not iBGP neighbors, BGP Maximum Prefix Features cannot work, so the*
>
> *   problem on PE2 cannot be known.”*
>
>
>
> *My take on this very first premise is that a VRF ( VPN1 ) on PE1 is being
> overwhelmed with routes. These routes are not coming from the RR topology.
> So what is the scenario ? Is it a redistribution from VRF ( VPN2 )->VRF(
> VPN1 )? If so, you have a serious security issue.. Is it redistribute from
> another protocol, the GRT? *
>
>
>
> *I disagree with your premise that the current suite of mitigations are
> not sufficient..  There are certain assertions which are subjective and not
> appropriate. AN example:*
>
>
>
> *“4) Configure the Maximum Prefix for each VRF on edge nodes*
>
>
>
> *   When a VRF overflows, it stops the import of routes and log the extra*
>
> *   VPN routes into its RIB.  However, PEs should parse the BGP updates.*
>
> *   These processes will cost CPU cycles and further burden the*
>
> *   overflowing PE.”*
>
>
>
> *What does this mean? There are networks that have been parsing millions
> of BGP Updates and routes. I have seen VRF overflows and they have been
> specific to said customer’s VRF. Where are you coming up with this and
> other broad assertions.*
>
>
>
> *From a solution point of view I do not believe in overloading the object.
> IOW RD is meant to convey uniqueness of a given path through a topology,
> overloading it with this semantic means what to the existing semantic.. As
> an ex, some operators use unique RD so there is no path hiding, eibgp load
> balancing etc.. Generally speaking I do not want my RRs making a best path
> decision for a given path. Some do not use unique RD for there own reasons
> including scale, possible alignment of RR topology to underlying best path
> etc…*
>
>
>
> *In order to use this technology what are the requirements in terms of
> assigning RDs? *
>
>
>
> *Thanks,*
>
> *              Jim Uttaro*
>
>
>
>
>
>
>
>
>
> *From:* Aijun Wang <[email-masked]>
> *Sent:* Wednesday, February 10, 2021 9:17 AM
> *To:* UTTARO, JAMES <[email-masked]>
> *Cc:* Lizhenbin <[email-masked]>; Susan Hares <[email-masked]>;
> [email-masked]
> *Subject:* Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt
> (2/4/2021 to 2/18/2021)
>
>
>
> Hi, Jim:
>
>
>
> *Do Not Support.*
> *Thanks,*
> *              Jim Uttaro*
> *From:* Idr <[email-masked]> *On Behalf Of *Lizhenbin
> *Sent:* Wednesday, February 10, 2021 4:47 AM
> *To:* Susan Hares <[email-masked]>; [email-masked]
> *Subject:* Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt
> (2/4/2021 to 2/18/2021)
>
>
>
> Hi All,
>
>
>
> I support the adoption.
>
> 1) Yes
>
> 2) Yes
>
> 3) Yes
>
>
>
> Best Regards,
>
> Zhenbin (Robin)
>
>
>
>
>
>
>
>
>
> *From:* Idr [mailto:[email-masked] <[email-masked]>] *On
> Behalf Of *Susan Hares
> https://datatracker.ietf.org/ipr/4579/ <https://urldefense.com/v3/__https:/datatracker.ietf.org/ipr/4579/__;!!BhdT!3JFUDaX3WxfXuq0-jDevFeHNv4xKPBpOZ-cktrZ8Hg9b1Ipaj0PIHJh3ZFw$>..
>
>
>
> <https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!BhdT!yysdkGCNAgoHWvY47NKedcAHI_Yf6MRpOT4zgCSp7HcB9m4Q73F8OU36k5sDVys$>
>
-- 

*Gyan Mishra*

---

From: Aijun Wang <[email-masked]>
Date: Thu, 11 Feb 2021 14:16:29 +0800

Hi, Jakob:

Aijun Wang
China Telecom

> On Feb 11, 2021, at 10:49, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 
> 

[WAJ] Not only Cisco, but also Huawei and Juniper, take similar approach to implement Prefix-base ORF. The prefix is the portion of user interested, which is described via Prefix-List.
> It may be by static configuration or as a result of your procedure.

[WAJ] Yeah. Trigger and procedures of the new ORF type is the major part of this document because we want to solve the situation that we can’t know in advance.

What you and Robert want to reuse is the semantic of prefix-based ORF type encoding. If we reuse it, most part of its fields has no meaningful or must set one special value , and to keep it in more general, we must make clear statements about the value of each filed in various address families, as that done in RFC7543.
Don’t you think RD-ORF is one clear slate to achieve the goal? And we have almost finished it?

> The ORF needs no special semantic to make that work.

[WAJ] Current draft is not only for semantic encoding, but also the trigger and procedures, which is lack in RFC 5292. 
For encoding itself, as explained above, reusing the existing one has no direct gain, but only confusion.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 11 Feb 2021 10:19:16 +0100

Hi,

> [WAJ] As Jim also mentioned, the redistribution between different VRFs is
> poor design and we should avoid it in first place.
>

What Jim pointed out is the scenario where your proposal misses to address.
I do not see there claim that vrf to vrf route propagation is a poor
design.

But this is just to prove your proposal is incomplete to protect PE from
being overwhelmed with VPN routes. Even basic import to more then one VRF
every operator (example mgmt VRF) is using can not be addressed by your
proposal.

No. Mechanism is defined if you want to break the VPN service use it.

*> The problem with maximum prefix is it is hard to set a limit on the
maximum due to resources statistical multiplexing that all PE-CE links  of
VPNs won’t flood simultaneously thus maximum prefix is either set very high
or not set at all.*

This point one needs to put things in perspective. If your capacity of PE
is 10 M VPN routes and your VPN is giving you 10 routes per site then even
if you set 100 max prefix to 100 no harm will happen. That is how max
prefix is deployed with proper warning threshold and then session drop as
last resort. The point is to protect from accidental mistakes - that's
all.

> So if a PE VRF is overflowing we cannot filter prefixes based on RT for
the overflowing VRF as that would filter all routes.

If VRF is "overflowing" you apply vrf-limit feature which is completely
different then max-prefix on BGP sessions. You may even shut down such VRF
completely which would be way better then partially drop the routes in it.
See in IP networks partially broken reachability is much worse to
troubleshoot then completely broken. Just imagine hours of troubleshooting
when customer calling your NOC complains his app is not working when ping
and trace are perfectly fine. In most cases NOC will say it is application
issue Mr customer pls go home.

We really here in IETF should not be endorsing such completely broken
operational models.

Kind regards,
Robert.

---

From: Aijun Wang <[email-masked]>
Date: Thu, 11 Feb 2021 19:42:35 +0800

Hi, Robert:

Aijun Wang
China Telecom

> On Feb 11, 2021, at 18:19, Robert Raszuk <[email-masked]> wrote:
> 
> 
> Hi,
>  
> What Jim pointed out is the scenario where your proposal misses to address. I do not see there claim that vrf to vrf route propagation is a poor design. 
> 
> But this is just to prove your proposal is incomplete to protect PE from being overwhelmed with VPN routes. Even basic import to more then one VRF every operator (example mgmt VRF) is using can not be addressed by your proposal. 
[WAJ] As mentioned in https://mailarchive.ietf.org/arch/msg/idr/_1vhNMPlyA7CHrdau15PjMAeDPM/, we have described the solution that you concerned. 
Why you trimmed the already mentioned solutions?

> No. Mechanism is defined if you want to break the VPN service use it. 

[WAJ] Would you like to keep the context of the responses? My above question is just for your mentioned prefix-based ORF.

> 
> > The problem with maximum prefix is it is hard to set a limit on the maximum due to resources statistical multiplexing that all PE-CE links  of VPNs won’t flood simultaneously thus maximum prefix is either set very high or not set at all.
> 
> This point one needs to put things in perspective. If your capacity of PE is 10 M VPN routes and your VPN is giving you 10 routes per site then even if you set 100 max prefix to 100 no harm will happen. That is how max prefix is deployed with proper warning threshold and then session drop as last resort. The point is to protect from accidental mistakes - that's all.  
[WAJ] RD-ORF has also the same aim. It is used when the session be multiplexed by several VPN and can’t be dropped scenarios.
> 
> > So if a PE VRF is overflowing we cannot filter prefixes based on RT for the overflowing VRF as that would filter all routes.
> 
> If VRF is "overflowing" you apply vrf-limit feature which is completely different then max-prefix on BGP sessions. You may even shut down such VRF completely which would be way better then partially drop the routes in it. See in IP networks partially broken reachability is much worse to troubleshoot then completely broken. Just imagine hours of troubleshooting when customer calling your NOC complains his app is not working when ping and trace are perfectly fine. In most cases NOC will say it is application issue Mr customer pls go home. 
[[WAJ] When the VRF-limit is exceeded, it will trigger of RD-ORF message in the discussed scenarios. It adds another tool to control the  propagation of overwhelmed VPN routes.

> 
> We really here in IETF should not be endorsing such completely broken operational models. 

[WAJ] Only the communications between overwhelmed PE and other PEs will be influenced, there will be log/alarm sent to NMS when the RD-ORF message is sent, which is similar with other existing mechanisms.
What will you do in the scenarios that described in this draft? Break the shared BGP session completely? That’s definitely unacceptable. Sending warning messages but can’t give the clues which VPN or RD arises this issue? It’s still useless.
The potential problem is existing, we should find one proper solution. Wish to hear more constructive proposal from you for this.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 11 Feb 2021 13:29:08 +0100

> What will you do in the scenarios that described in this draft ?

As already stated the solution which is deployed in production networks to
prevent scenario described in the draft is not to allow "overwhelming"
amount of routes to enter your network from any VPN.

When you sell VPN service you agree with customers how many routes they
will be injecting into your network. In many cases you even ask for
specific routes which you apply ingress policy to make sure only those
prefixes as expected enter your network.

Same on PE-CE some on Inter-AS option A, B or C or mix of any of the above.

Now if all of the above is not done or done with mistakes and you indeed
experience to many routes to be handled by data plane you stop locally
importing those routes to local VRFs by VRF shutdown. The good thing here
is that this will be noticed by all attached customers as their dynamic
routing will propagate withdraws.

If you are concern that the control plane is not going to keep up you apply
locally inbound drop based on whatever criteria you like (RT, RD etc ...)
and raise alarms and again propagate withdraws to locally attached CEs or
peer's ASBR.

So I am pretty sure none of the suggested by WG tools which have been in
place are going to be sufficient for you. So at this point I will let IDR
chairs and AD to decide on the next steps fwd.

Many thx,
Robert

---

From: Aijun Wang <[email-masked]>
Date: Thu, 11 Feb 2021 23:22:36 +0800

Hi, Robert:

Aijun Wang
China Telecom

> On Feb 11, 2021, at 20:29, Robert Raszuk <[email-masked]> wrote:
> 
> 
> 
> As already stated the solution which is deployed in production networks to prevent scenario described in the draft is not to allow "overwhelming" amount of routes to enter your network from any VPN. 
> 
> When you sell VPN service you agree with customers how many routes they will be injecting into your network. In many cases you even ask for specific routes which you apply ingress policy to make sure only those prefixes as expected enter your network. 
> 
> Now if all of the above is not done or done with mistakes and you indeed experience to many routes to be handled by data plane you stop locally importing those routes to local VRFs by VRF shutdown. The good thing here is that this will be noticed by all attached customers as their dynamic routing will propagate withdraws. 
[WAJ] The BGP session is shared by several VPNs and cannot be shutdown in such situations. Procedures that you expect would not happen.
> 
> If you are concern that the control plane is not going to keep up you apply locally inbound drop based on whatever criteria you like (RT, RD etc ...) and raise alarms and again propagate withdraws to locally attached CEs or peer's ASBR. 
[WAJ] Locally drop is not enough, as that described in https://tools.ietf.org/html/rfc5291#section-1, as the following:

“Currently, it is not uncommon for a BGP speaker [BGP-4] to receive, and then filter out some unwanted routes from its peers based on its local routing policy.  Since the generation and transmission of routing updates by the sender, as well as the processing of routing updates by the receiver consume resources, it may be beneficial if the generation of such unwanted routing updates can be avoided in the first place.”

> 
> So I am pretty sure none of the suggested by WG tools which have been in place are going to be sufficient for you. So at this point I will let IDR chairs and AD to decide on the next steps fwd. 

[WAJ] Let’s discuss first on the mail list. We just want the network can respond quickly and automatically for some abnormal situations. RD-ORF is one feedback mechanism that can be utilized.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 11 Feb 2021 16:40:28 +0100

> [WAJ] The BGP session is shared by several VPNs and cannot be shutdown in
> such situations. Procedures that you expect would not happen.
>

Where did I say to shutdown BGP session ?

I said VRF shutdown ... nothing to do with BGP sessions shutdown.

- - -

Let me provide the analogy here ...

You are presenting a solution on what to do when we fly low and start
hitting the top of the trees.

WG tells you that flying low is bad practice and should not take place
providing what to do to mitigate it well - in the analogy train your ground
crew not to overload the plane.

You say - Oh well we have different idea instead - when we throw away
during the flight some passengers we can keep flying (on the edge of
safety).

That's here we are with this.

- - -

Quite honestly when we started to deploy 2547 back in nearly 2000s we were
preparing much more user friendly solution (a flavor of CSC - not CSC as
defined in the RFC) where SP network would only deal with next hops and
never with customer routes ... yet customer would have the same type of
service. Quite honestly we have had even OSPF reflector ready for it.
Problem was that at that time SPs said oh we want to take millions of
customer routes - no problem. We just buy bigger RRs and bigger routers. So
you can guess what happened with such idea - got shelved as if deployed
would result in revenue loss :).

Best,
R.

---

From: Aijun Wang <[email-masked]>
Date: Fri, 12 Feb 2021 00:11:33 +0800

Hi, Robert:

Aijun Wang
China Telecom

> On Feb 11, 2021, at 23:40, Robert Raszuk <[email-masked]> wrote:
> 
> 
> 
[WAJ] Don’t you think VRF shutdown has the more broader influences? RD-ORF influences only the newly advertised VPN routes. VRF shutdown will influence the existing VPN routes and its existing forwarding traffic.

> You are presenting a solution on what to do when we fly low and start hitting the top of the trees. 
> 
> WG tells you that flying low is bad practice and should not take place providing what to do to mitigate it well - in the analogy train your ground crew not to overload the plane. 
> 
> You say - Oh well we have different idea instead - when we throw away during the flight some passengers we can keep flying (on the edge of safety). 
[WAJ] Just deny the onboard of new passengers, not throwing the existing passengers.
> Quite honestly when we started to deploy 2547 back in nearly 2000s we were preparing much more user friendly solution (a flavor of CSC - not CSC as defined in the RFC) where SP network would only deal with next hops and never with customer routes ... yet customer would have the same type of service. Quite honestly we have had even OSPF reflector ready for it. Problem was that at that time SPs said oh we want to take millions of customer routes - no problem. We just buy bigger RRs and bigger routers. So you can guess what happened with such idea - got shelved as if deployed would result in revenue loss :). 
> 
[WAJ] RD-ORF mechanism can encourage the provider to deploy inter-as VPN solutions because it gives them the granular control over the VPN routes propagation between ASBRs or RRs, which can certainly broader the VPN service coverage and the revenue.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 11 Feb 2021 20:19:12 +0000

You are trying to have a VRF that is receiving routes to notify a VRF that is sending routes that it is sending too many routes.
The receiving VRF and the sending VRF are separated by several or many intermediate BGP speakers.
Bouncing ORFs back through these intermediate speakers is not a reliable way to achieve this notification.
This is because each intermediate speaker has multiple downstream speakers.
One speaker needs to receive the ORF from all of its downstream speakers before it can bounce the ORF upstream.
Thus, it's unlikely that the ORF will make it all the way to the source of the routes.
You are asking for something like a reverse BGP.
That can't scale.
The way that we find out whether a far away speaker got our route correctly is with looking glasses.
Looking glasses aren't implemented by BGP, because BGP operators don't want to store that kind of information.

Regards,
Jakob.

From: Aijun Wang <[email-masked]>
Sent: Thursday, February 11, 2021 8:12 AM
To: Robert Raszuk <[email-masked]>
Cc: Jakob Heitz (jheitz) <[email-masked]>; Susan Hares <[email-masked]>; [email-masked]; Acee Lindem (acee) <[email-masked]>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Hi, Robert:
Aijun Wang
China Telecom

On Feb 11, 2021, at 23:40, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

- - -

- - -

Best,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 11 Feb 2021 21:30:43 +0100

> You are trying to have a VRF that is receiving routes to notify a VRF
that is sending routes that it is sending too many routes.

Well if that was true maybe we could try to make it work. But they are not
doing this.

See receiving VRF of RD:X can import routes from many other PEs/VRFs with
zoo of RDs. So what they say is to be selective and say Oh this RD:FAT is
sending way too many routes so we want to get rid of those routes. Moreover
some PEs may be just weak and can trigger such dislike msg and some may be
taking the routes with no problem.

What is being cooked here to me looks like a complete mess - that's all.

Cheers,
Robert

On Thu, Feb 11, 2021 at 9:19 PM Jakob Heitz (jheitz) <[email-masked]>
wrote:

> You are trying to have a VRF that is receiving routes to notify a VRF that
> is sending routes that it is sending too many routes.
>
> *Sent:* Thursday, February 11, 2021 8:12 AM
> *To:* Robert Raszuk <[email-masked]>
> *Cc:* Jakob Heitz (jheitz) <[email-masked]>; Susan Hares <
> [email-masked]>; [email-masked]; Acee Lindem (acee) <[email-masked]>
> [WAJ] Don’t you think VRF shutdown has the more broader influences? RD-ORF
> influences only the newly advertised VPN routes. VRF shutdown will
> influence the existing VPN routes and its existing forwarding traffic.
>
>
>
> - - -
>
>
>
> [WAJ] RD-ORF mechanism can encourage the provider to deploy inter-as VPN
> solutions because it gives them the granular control over the VPN routes
> propagation between ASBRs or RRs, which can certainly broader the VPN
> service coverage and the revenue.
>
>
>
> Best,
>
> R.
>
>
>
>

---

From: john heasley <[email-masked]>
Date: Thu, 11 Feb 2021 22:35:43 +0000

> 2) Should the WG consider this draft given it has an IPR claim

Why?  The IPR appears to indicate that if any entity that asserts any patent
claim against CT, presumably due to infringement, should expect CT to assert
a patent claim for this process.  Retaliatory patent enforcement.  Awesome.
FRAND does not mean settlement-free, nor reasonable.

---

From: Aijun Wang <[email-masked]>
Date: Fri, 12 Feb 2021 08:01:11 +0800

Hi, Robert and Jakob:
I think you all misunderstood the procedures of RD-ORF mechanism.
The RD-ORF message doesn’t need to find its way back to the source to control the excessive VPN routes, the intermediate nodes, for example, the RR, or the ASBR can accomplish this work instead. 
That is to say, if one PE can’t receive more routes from the RR, it just send the RD-ORF message to the RR. If RR has the capability to process the upcoming BGP UPDATES, it will not send another RD-ORF message to the source.
Only the specified VPN communications between the weak PE and other health PEs will be influenced. The communications among the health PEs will be kept in good state.

Whether and when to send the RD-ORF message is determined independently by each involved nodes.

Let me also explain such behavior to you based your statements inline below.[WAJ]

Wish these explains can correct your understanding of RD-ORF mechanism.

Aijun Wang
China Telecom

> On Feb 12, 2021, at 04:39, Robert Raszuk <[email-masked]> wrote:
> 
> 
> 
> Well if that was true maybe we could try to make it work. But they are not doing this. 
> 
> See receiving VRF of RD:X can import routes from many other PEs/VRFs with zoo of RDs. So what they say is to be selective and say Oh this RD:FAT is sending way too many routes so we want to get rid of those routes. Moreover some PEs may be just weak and can trigger such dislike msg and some may be taking the routes with no problem. 

[WAJ] If the weak PE trigger RD-ORF message to RR, but other health PEs under this RR can process it, then when RR receives such message, it just filter the RD-specified VPN BGP updates to the weak PE, other health  PEs can still receive the specified VPN BGP updates and then got the RD-specified VPN routes(VPN:FAT in your example)

> What is being cooked here to me looks like a complete mess - that's all. 
[WAJ] Wish the above explanations can help you reorganize the imaginal but unreal mess.

> Cheers,
> Robert
> 
> 
> 
> 
> 
> 
>> On Thu, Feb 11, 2021 at 9:19 PM Jakob Heitz (jheitz) <[email-masked]> wrote:
[WAJ] Yes, but controlling the VPN routes needn’t relay the RD-ORF messages back until to the source. There are many valves between them.
>> Bouncing ORFs back through these intermediate speakers is not a reliable way to achieve this notification.
[WAJ] Yes, we have avoided this approach.
>> One speaker needs to receive the ORF from all of its downstream speakers before it can bounce the ORF upstream.
[WAJ] Each intermediate speaker can control the valves to each RD-ORF message initiator.
Whether and when send out its own RD-ORF message is determined by itself process capability, not related to the portion of downstream speakers emitted RD-ORF message. This is more flexible and deployable. 
>> Thus, it's unlikely that the ORF will make it all the way to the source of the routes.
[WAJ] Yes, we have abandoned this approach.
>> Looking glasses aren't implemented by BGP, because BGP operators don't want to store that kind of information.
[WAJ] RD-ORF needs also not store that kind of information.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 12 Feb 2021 00:16:29 +0000

Ajun,

Well, if that's all it is, then you should just drop the routes at the weak PE.
You say that receiving and dropping is burdensome on the receiving router.
Frankly, I don't believe you.
Can you back that statement up and demonstrate how it has actually caused outages?

Regards,
Jakob.

Sent: Thursday, February 11, 2021 4:01 PM

Hi, Robert and Jakob:

Cheers,
Robert

On Thu, Feb 11, 2021 at 9:19 PM Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>> wrote:
You are trying to have a VRF that is receiving routes to notify a VRF that is sending routes that it is sending too many routes.
The receiving VRF and the sending VRF are separated by several or many intermediate BGP speakers.
[WAJ] Yes, but controlling the VPN routes needn’t relay the RD-ORF messages back until to the source. There are many valves between them.

Regards,
Jakob.

From: Aijun Wang <[email-masked]<mailto:[email-masked]>>
Sent: Thursday, February 11, 2021 8:12 AM
To: Robert Raszuk <[email-masked]<mailto:[email-masked]>>
Cc: Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>>; Susan Hares <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; Acee Lindem (acee) <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Hi, Robert:
Aijun Wang
China Telecom

- - -

- - -

Best,
R.

---

From: Aijun Wang <[email-masked]>
Date: Fri, 12 Feb 2021 08:18:51 +0800

Hi, John:

I am not the IPR experts, but it seems such clauses is the normal declaration within IETF community. Would you like to give other examples in IETF that may be more moderate?
If so, I can seek the opinions from our IPR experts and feedback.

Aijun Wang
China Telecom

> On Feb 12, 2021, at 06:36, john heasley <[email-masked]> wrote:
>

---

From: Aijun Wang <[email-masked]>
Date: Fri, 12 Feb 2021 08:30:57 +0800

Hi, Jakob:

This is the start point of ORF mechanism, not only for RD-ORF. I think we can refer the description in https://tools.ietf.org/html/rfc5291#section-1.
Locally discard is not enough. The excessive junk BGP Updates can certainly lead the high CPU usages of the affected PE, and let them react slowly to process the other normal BGP Updates, and then the communications of the other normal VPNs that on the shared BGP connection.

Aijun Wang
China Telecom

> On Feb 12, 2021, at 08:16, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 
> 
> Ajun,
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 12 Feb 2021 01:22:52 +0000

"can" is not good enough. "can certainly" is not much better. "did" with backup is good.

Regards,
Jakob.

On Feb 11, 2021, at 4:31 PM, Aijun Wang <[email-masked]> wrote:

 Hi, Jakob:

Aijun Wang
China Telecom

Ajun,

Regards,
Jakob.

Hi, Robert and Jakob:

Cheers,
Robert

Regards,
Jakob.

Hi, Robert:
Aijun Wang
China Telecom

- - -

- - -

Best,
R.

---

From: Aijun Wang <[email-masked]>
Date: Fri, 12 Feb 2021 10:03:14 +0800

Hi, Jakob:

I think we can evaluate this mechanism frim other viewpoints: 
We will definitely deploy BGP maximum-prefixes feature between two BGP peer, and will break the session if the receiver can’t tolerate. Right?
The RD-ORF mechanism just wants to cope with such scenarios in more graceful/gradual way. 

Aijun Wang
China Telecom

> On Feb 12, 2021, at 09:23, Jakob Heitz (jheitz) <[email-masked]> wrote:
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 12 Feb 2021 02:56:23 +0000

The maximum-prefix configured on a BGP session is not designed to protect the router from too many routes.
When you add up all the maximum-prefix amounts for each session, the total is much higher than the router can tolerate.
There are other configuration knobs to protect the router from too many routes.
The purpose of maximum-prefix is to guard against route leaks and other configuration errors from customer networks.
It is not the only tool for that and it's not very good. But it does tell the customer in no uncertain terms that something went wrong. It's like a large hammer for a small screw. Or 杀鸡用牛刀.
Now, you said before that you don't want to tell the source, so there's little point.

Regards,
Jakob.

From: Aijun Wang <[email-masked]>
Sent: Thursday, February 11, 2021 6:03 PM
To: Jakob Heitz (jheitz) <[email-masked]>
Cc: Robert Raszuk <[email-masked]>; Susan Hares <[email-masked]>; [email-masked]; Acee Lindem (acee) <[email-masked]>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Hi, Jakob:

Aijun Wang
China Telecom

On Feb 12, 2021, at 09:23, Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>> wrote:
 "can" is not good enough. "can certainly" is not much better. "did" with backup is good.

Regards,
Jakob.

On Feb 11, 2021, at 4:31 PM, Aijun Wang <[email-masked]<mailto:[email-masked]>> wrote:
 Hi, Jakob:

Aijun Wang
China Telecom

On Feb 12, 2021, at 08:16, Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>> wrote:

Ajun,

Regards,
Jakob.

Hi, Robert and Jakob:

Cheers,
Robert

Regards,
Jakob.

Hi, Robert:
Aijun Wang
China Telecom

- - -

- - -

Best,
R.

---

From: Aijun Wang <[email-masked]>
Date: Fri, 12 Feb 2021 11:48:25 +0800

Hi, Jakob:

Aijun Wang
China Telecom

> On Feb 12, 2021, at 10:56, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 
> 

[WAJ] In some situations, the message can back to the source, although it is not always. This is also the gradual feature of RD-ORF mechanism 

Happy New Year to you or “新春快乐！”

Aijun Wang
China Telecom

---

From: Gyan Mishra <[email-masked]>
Date: Fri, 12 Feb 2021 02:59:30 -0500

Jacob

As Aijun me mentioned this draft falls along the same premises as RFC 4684
RTC RT membership to create a distribution flood graph or RFC 5291 ORF
filter as Aijun mentioned to control flooding based on distribution graph
as the goal is to avoid unnecessary sending and processing of routing
updates and as well as receiving and processing of routing updates that end
up being dropped in the case of RTC RT membership flood graph.

RTC limits flooding via RT membership from RR to PE to only send to the PE
only interesting RTs being explicitly imported, in comparison to RD-ORF it
provides the same RTC style optimization to avoid flood of routes where the
overloaded PE can now signal via ORF in the same direction of RTC flood
scope from RR to PE adj-rib-out filter, but now at the RD level instead of
RTC RT level adj-rib-out based on the offending RD origin PE flooding the
prefixes using existing BGP route refresh RFC 5291 machinery for the RD-ORF.

As to why RTC does not work for this problem statement of an offending PE
or multiple PEs is that the membership is based on RT layer interesting RT
so the adj-rib-out would match the interesting RT imported by the PE only
so all prefixes for the VPN would be imported it the PEs VRF.  Thus the
offending PE or PEs flooding the VPN issue cannot be mitigated with RTC
since the RDs for the offending PEs would still be imported.

Just as well as you mentioned BGP peer maximum prefix is set to a high
water mark value as is the VPN maximum prefix knob, to prevent a customer
typo or errored configuration causing a flood of routes, but does not
protect the PE resources as if all peers were flooding and hit the high
water mark peer maximum prefix or if all VPNs hit the VPN maximum prefix
simultaneously the PE would be choked out of resources including the newly
upgraded FAT PE with plenty of horse power.

Unfortunately their is no solution that exists today that can circumvent
such issue with an offending PE or PEs plural flooding prefixes impacting
an operators core.

RT has been historically been used as the method of filtering as that is
the match method to import prefixes into a VRF, however the granularity has
never been used to filter based on RD which now I think with this draft and
future drafts and development in this arena, RD based filtering will become
more common as it allows for per PE originator granularity for filtering as
opposed to RT based filtering which represents all VPN prefixes to be
imported, which from a filtering perspective ends be being a sledge hammer
filter approach as opposed to a tiny screw driver RD based filtering
approach.

Kind Regards

Gyan

On Thu, Feb 11, 2021 at 10:49 PM Aijun Wang <[email-masked]>
wrote:

> When you add up all the maximum-prefix amounts for each session, the total
> is much higher than the router can tolerate.
>
> The purpose of maximum-prefix is to guard against route leaks and other
> configuration errors from customer networks.
>
> It is not the only tool for that and it's not very good. But it does tell
> the customer in no uncertain terms that something went wrong. It's like a
> large hammer for a small screw. Or 杀鸡用牛刀.
>
> [WAJ] In some situations, the message can back to the source, although it
> is not always. This is also the gradual feature of RD-ORF mechanism
>
> *Sent:* Thursday, February 11, 2021 6:03 PM
> *To:* Jakob Heitz (jheitz) <[email-masked]>
> *Cc:* Robert Raszuk <[email-masked]>; Susan Hares <[email-masked]>;
> [email-masked]; Acee Lindem (acee) <[email-masked]>
> We will definitely deploy BGP maximum-prefixes feature between two BGP
> peer, and will break the session if the receiver can’t tolerate. Right?
>
> This is the start point of ORF mechanism, not only for RD-ORF. I think we
> can refer the description in https://tools.ietf.org/html/rfc5291#section-1
> .
>
> Locally discard is not enough. The excessive junk BGP Updates can
> certainly lead the high CPU usages of the affected PE, and let them react
> slowly to process the other normal BGP Updates, and then the communications
> of the other normal VPNs that on the shared BGP connection.
>
>
>
> Aijun Wang
>
> China Telecom
>
>
>
> *Sent:* Thursday, February 11, 2021 4:01 PM
> The RD-ORF message doesn’t need to find its way back to the source to
> control the excessive VPN routes, the intermediate nodes, for example, the
> RR, or the ASBR can accomplish this work instead.
>
> That is to say, if one PE can’t receive more routes from the RR, it just
> send the RD-ORF message to the RR. If RR has the capability to process the
> upcoming BGP UPDATES, it will not send another RD-ORF message to the source.
>
> Only the specified VPN communications between the weak PE and other health
> PEs will be influenced. The communications among the health PEs will be
> kept in good state.
>
>
>
> [WAJ] If the weak PE trigger RD-ORF message to RR, but other health PEs
> under this RR can process it, then when RR receives such message, it just
> filter the RD-specified VPN BGP updates to the weak PE, other health  PEs
> can still receive the specified VPN BGP updates and then got the
> RD-specified VPN routes(VPN:FAT in your example)
>
>
>
>
>
>
> [WAJ] Yes, but controlling the VPN routes needn’t relay the RD-ORF
> messages back until to the source. There are many valves between them.
>
>
> Whether and when send out its own RD-ORF message is determined by itself
> process capability, not related to the portion of downstream speakers
> emitted RD-ORF message. This is more flexible and deployable.
>
>
-- 

*Gyan Mishra*

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 12 Feb 2021 12:30:27 +0100

Aijun & Gyan,

Let me try one more (hopefully last time) to explain to both of you - and
for that matter to anyone how supported this adoption.

Let's consider very typical Hub and Spoke scenario as illustrated below:

HQ1 is advertising two routes:

- one default with RDX1 with RT TO_SPOKE
- one or more specifics with RDX1 to the other HUBs

Now imagine HQ1 bought a new BGP "Optimizer" and suddenly is starting to
advertise 100000 /32 routes just to the other HUB with RT: TO_HUB.

So PE2 detects this as VRF with RDX2 on it got overwhelmed during import
with RT TO_HUB and starts pushing RDX1 (original RD) to RR to stop getting
those routes.

Well all great except now you are throwing baby with the water as all
spokes attached to PE2 which just import default route to HUB HQ1 also can
no longer reach their hub site as their default route will be removed.
Therefor they will have nothing to import with RT:TO_SPOKE

Further if RR "independently" decided ... oh let's push this ORF to PE1
then all of the spokes attached to perhaps even much more powerful PE3 can
also no longer reach their headquarters.

- - -

*Summary: *

The above clearly illustrates why the proposed solution to use RD for
filtering is in fact harmful.

See when you design new protocol extensions the difficulty is to not break
any existing protocols and deployments.

Hope this puts this long thread to rest now.

Thx,
Robert

---

From: Aijun Wang <[email-masked]>
Date: Fri, 12 Feb 2021 22:47:47 +0800

Hi, Robert:

https://datatracker.ietf.org/doc/html/draft-wang-idr-rd-orf-05#section-5.1.1 has described such situations, which will require the additional local decisions of PE2 to judge whether to send the RD-ORF message out.
In your example, if only the HUB VRF exceed but the resources of PE2 is not exhausted, then the PE2 will not send the RD-ORF message. It may just discard the excessive 100000/32 routes.
If the resources of PE2 is nearly exhausted, it must send the RD-ORF message out. Or else not only the Spoke VRF, but also other VPNs on this device can’t be used.

Regarding to RR, it is the same principle: if RR can cope with such flooding, it need not send out RD-ORF to PE1. If RR can’t cope with, it must send out the RD-ORF message, or else not only the VPN that import RD X1 routes can’t work, but also other VPNs that don’t import RD x1 routes.

RD-ORF mechanism just keep the influences as small as possible.

Wish the above explanation can refresh your review of this draft.

We are also hopeful to invite you join us to make RD-ORF mechanism more robust and meet the critical challenges.

Aijun Wang
China Telecom

> On Feb 12, 2021, at 19:30, Robert Raszuk <[email-masked]> wrote:
> 
> 
> Aijun & Gyan,
> 
> Let me try one more (hopefully last time) to explain to both of you - and for that matter to anyone how supported this adoption. 
> 
> Now imagine HQ1 bought a new BGP "Optimizer" and suddenly is starting to advertise 100000 /32 routes just to the other HUB with RT: TO_HUB. 
> 
> 
> 
> 
> So PE2 detects this as VRF with RDX2 on it got overwhelmed during import with RT TO_HUB and starts pushing RDX1 (original RD) to RR to stop getting those routes. 
> 
> Well all great except now you are throwing baby with the water as all spokes attached to PE2 which just import default route to HUB HQ1 also can no longer reach their hub site as their default route will be removed. Therefor they will have nothing to import with RT:TO_SPOKE
> 
> Further if RR "independently" decided ... oh let's push this ORF to PE1 then all of the spokes attached to perhaps even much more powerful PE3 can also no longer reach their headquarters. 
> 
> - - - 
> 
> Summary: 
> 
> The above clearly illustrates why the proposed solution to use RD for filtering is in fact harmful. 
> 
> See when you design new protocol extensions the difficulty is to not break any existing protocols and deployments.
>

---

From: "Wanghaibo (Rainsword)" <[email-masked]>
Date: Fri, 12 Feb 2021 15:04:18 +0000

Hi Robert,

ORF is used for reduce the pressure on the receiver.
The receiver device, such as the access router, may have weak performance, while the sender device, such as the RR, may have strong performance.
Therefore, it is sometimes not enough to filter packets at the receiver.

The ORF is a tool and does not exist independently. The deployment problem described here occurs only when the RD-ORF mode is used.
Actually, the ORF can be used together with other ORFs, such as Prefixed ORF.
RD-ORF provides a more efficient filtering method, making solution deployment more flexible.

In fact, most vendors' devices support rd-filter, but only support the configuration.

In addition, if the RR decides to push the RD-ORF to its route source only when PE1 and PE3 do not want that RD.

Regards,
Haibo

From: Idr [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Friday, February 12, 2021 7:30 PM
To: Gyan Mishra <[email-masked]>
Cc: idr@ietf. org <[email-masked]>; Susan Hares <[email-masked]>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Aijun & Gyan,

- - -

Summary:

Thx,
Robert

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 12 Feb 2021 16:04:35 +0100

Sorry Aijun,

What you say is just handwaving. There is no room for it in any spec.

When code is written PE must deterministically behave so the RR or any
other network element.

Statements "decisions of PE2 to judge" are not acceptable in protocol
design.

Just imagine that each PE does what it feels like in a distributed network
.... Same for BGP same for IGP etc ....

And all of this is not needed if on ingress between PE1 and HQ1 you apply
max prefix of 2 or even 100. It is also not needed if you enable  RTC to
send RT:TO_HUB from PE2 to RR.

But I understand - no matter what we say or how much we spend time to
explain why this idea is a bad idea you are still going to push this fwd.
Oh well ...   If I were you I would spend this time to redefine L3VPN such
that customer routes are never needed to be sent to SP core routers.

Thx,
R.

On Fri, Feb 12, 2021 at 3:47 PM Aijun Wang <[email-masked]>
wrote:

> Hi, Robert:
>
>
> https://datatracker.ietf.org/doc/html/draft-wang-idr-rd-orf-05#section-5.1.1 has
> described such situations, which will require the additional local
> decisions of PE2 to judge whether to send the RD-ORF message out.
> In your example, if only the HUB VRF exceed but the resources of PE2 is
> not exhausted, then the PE2 will not send the RD-ORF message. It may just
> discard the excessive 100000/32 routes.
> If the resources of PE2 is nearly exhausted, it must send the RD-ORF
> message out. Or else not only the Spoke VRF, but also other VPNs on this
> device can’t be used.
>
> Regarding to RR, it is the same principle: if RR can cope with such
> flooding, it need not send out RD-ORF to PE1. If RR can’t cope with, it
> must send out the RD-ORF message, or else not only the VPN that import RD
> X1 routes can’t work, but also other VPNs that don’t import RD x1 routes.
>

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 12 Feb 2021 16:09:25 +0100

>
> The receiver device, such as the access router, may have weak performance,
> while the sender device, such as the RR, may have strong performance.
>

Yes., That is precisely why we have  https://tools.ietf.org/html/rfc4684

The ORF is a tool and does not exist independently. The deployment problem
> described here occurs only when the RD-ORF mode is used.
>

See this is one more time indication that you have serious gap in
understanding L3VPNs.

PEs which import routes do not understand sender's ORF. They take routes
and based on RT make a local copy to importing VRF.

In RFC4364 RTs are used to build arbitrary VPN topologies not RDs.

Thx

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 12 Feb 2021 16:14:00 +0100

Sorry typo

s/PEs which import routes do not understand sender's ORF./PEs which import
routes do not understand sender's RDs/

On Fri, Feb 12, 2021 at 4:09 PM Robert Raszuk <[email-masked]> wrote:

>> In addition, if the RR decides to push the RD-ORF to its route source
>> only when PE1 and PE3 do not want that RD.
>>
>

---

From: Aijun Wang <[email-masked]>
Date: Fri, 12 Feb 2021 23:26:16 +0800

Hi, Robert:

Aijun Wang
China Telecom

> On Feb 12, 2021, at 23:10, Robert Raszuk <[email-masked]> wrote:
> 
> 
> See this is one more time indication that you have serious gap in understanding L3VPNs. 
> 
> PEs which import routes do not understand sender's ORF. They take routes and based on RT make a local copy to importing VRF. 
> 

[WAJ] Just one correction, whether RR will send the RD-ORF message out to PE1 is not depend on the reaction of PE2 and PE3, it is its own decision.

---

From: Aijun Wang <[email-masked]>
Date: Fri, 12 Feb 2021 23:31:27 +0800

Hi, Robert:
Yes, the behavior of the device should be determined. There maybe several factors to be considered for this local behavior, we should describe it more clearly in this section later.
We have discussed the differences between RTC and RD-ORF a lot. As Haibo mentioned, they are not exclusive to each other, and can be used together in some situations. But they are different and can’t replace each other.

Aijun Wang
China Telecom

> On Feb 12, 2021, at 23:04, Robert Raszuk <[email-masked]> wrote:
> 
> 
> Sorry Aijun,
> 
> When code is written PE must deterministically behave so the RR or any other network element. 
> 
> Statements "decisions of PE2 to judge" are not acceptable in protocol design. 
> 
> Just imagine that each PE does what it feels like in a distributed network .... Same for BGP same for IGP etc .... 
> 
> And all of this is not needed if on ingress between PE1 and HQ1 you apply max prefix of 2 or even 100. It is also not needed if you enable  RTC to send RT:TO_HUB from PE2 to RR.
> 
> But I understand - no matter what we say or how much we spend time to explain why this idea is a bad idea you are still going to push this fwd. Oh well ...   If I were you I would spend this time to redefine L3VPN such that customer routes are never needed to be sent to SP core routers. 
> 
> Thx,
> R.
> 
> 
>> On Fri, Feb 12, 2021 at 3:47 PM Aijun Wang <[email-masked]> wrote:
>> Hi, Robert:
>>

---

From: "Susan Hares" <[email-masked]>
Date: Fri, 12 Feb 2021 12:21:40 -0500

John: 

Thank you for indicating your concern regarding the IPR.   

Susan Hares 

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of john heasley
Sent: Thursday, February 11, 2021 5:36 PM
To: Gyan Mishra
Cc: Susan Hares; [email-masked]
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt
(2/4/2021 to 2/18/2021)

> We have reviewed other approaches and this provides a finer grain 
> approach to dynamically mitigate VPN being overwhelmed by source PE RD 
> offender.  We have discussed other approaches and best practice such 
> as VPN maximum prefix as well as peer maximum prefix, however this 
> feature resolves a gap directly related to offending PE RD originator.

---

From: Gyan Mishra <[email-masked]>
Date: Fri, 12 Feb 2021 22:16:05 -0500

All

>From Susan Hares summary of where we are at with the adoption call let’s
start with the problem this draft is trying to solve and gaining
consensus.  Once we gain consensus we can get back to RD-ORF solution.  See
w

a) the problem this draft is drafting to solve relating to BGP routes,

The problem we are trying to solve is a scenario where you have an
offending PE that is flooding routes and a weak PE that is overwhelmed by a
flood of routes.  This is not a normal situation and is an outage situation
where the weak PE being overwhelmed by a flood of routes.  Do we all agree
to the problem statement?

Why and why not?

b) the need for additional mechanisms to solve the problem,

Do other methods exist that can solve the problem and if not do we need a
new mechanism to solve this?

RTC, Peer maximum prefix, VPN maximum prefix

c) a clear description of the technology to solve the problem.

Do we all agree that in a normal situation we would never filter on RD as
that would partition the VPN which is unwanted and what Robert mentioned.
As this is not a normal situation but a unique situation where a weak PE is
overwhelmed by a flood of routes.  How best can this be solved?

On Fri, Feb 12, 2021 at 10:32 AM Aijun Wang <[email-masked]>
wrote:

> Hi, Robert:
> Yes, the behavior of the device should be determined. There maybe several
> factors to be considered for this local behavior, we should describe it
> more clearly in this section later.
> We have discussed the differences between RTC and RD-ORF a lot. As Haibo
> mentioned, they are not exclusive to each other, and can be used together
> in some situations. But they are different and can’t replace each other.
>
> Aijun Wang
> China Telecom
>
>> See when you design new protocol extensions the difficulty is to not
>> break any existing protocols and deployments.
>>

*Gyan Mishra*

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 13 Feb 2021 12:42:01 +0100

All,

flood of routes.

The problem is a valid problem. But the proposed solution to the problem is
not. And moreover solutions to this very problem are widely known and used
for years.

*The problem as a matter of fact has nothing to do with VPNs of any sort.*

If you are ISP offering Internet transport unless you apply proper
protection you would be badly exposed. Your clients or peers or upstreams
injecting millions of routes and melting your network and perhaps even
global Internet (if they would use a registered block).

And yes BGP ingress policy here is used to filter out junk before it enters
any network. The same policy must be used in VPN cases too.

Indeed years have passed and I think I have only seen in a very few cases
that operators offering L3VPNs are doing prefix ingress filtering ... Max
prefix on ingress is used much more often. Those are the right tools here
to work on. I am not saying we should not invent more ... we should.

Ideas:

* Customise secure BGP to work in VPN cases as example.

* Augment RRs to be a bit more intelligent with pitch of ML - if number of
routes with given RD for time Tx is R moment you receive 100*R you suspend
those and raise NOC alarm before spraying everywhere

* Do not put VPN customer routes into your data plane ... just handle next
hops. Redefine RFC4364 all together and use IP transport for it.

etc ...

Focus on not allowing the meltdown is the proper solution space ... But
here instead we do nothing to prevent the fire to start and instead focus
on tools to extinguish it. Wrong approach. And specifically this tool
(RD-ORF) does way too much damage when used.

Thx,
R.

On Sat, Feb 13, 2021 at 4:16 AM Gyan Mishra <[email-masked]> wrote:

>> Just imagine that each PE does what it feels like in a distributed
>> network .... Same for BGP same for IGP etc ....
>>
>>> Let me try one more (hopefully last time) to explain to both of you -
>>> and for that matter to anyone how supported this adoption.
>>>

---

From: Aijun Wang <[email-masked]>
Date: Sat, 13 Feb 2021 20:46:08 +0800

Hi, Robert:
Let’s discuss based on the facts, not the exaggeration declarations.
Details replying inline below[WAJ]

Aijun Wang
China Telecom

> On Feb 13, 2021, at 19:42, Robert Raszuk <[email-masked]> wrote:
> 
> 
> All,
> 
> >   The problem we are trying to solve is a scenario where you have an offending PE that is flooding routes and a weak PE that is overwhelmed by a flood of routes.
> 
> The problem is a valid problem. But the proposed solution to the problem is not. And moreover solutions to this very problem are widely known and used for years. 
> 
> The problem as a matter of fact has nothing to do with VPNs of any sort. 

[WAJ] We have said clearly RD-ORF mechanism applying to the scenarios that inter-AS option B/C scenarios or the scenario within one AS when RR present. All of these situations have the following characteristics:
1. Several VPNs share one BGP sessions, where the BGP max-prefix is too coarse to do the fine-granularity control.
2.Junk VPN routes can’t be known in advance where the RTC and prefix-based ORF are not enough for the potential threats.
3. The network should react automatically/quickly to alleviate the threats, to give the chance for the operators step in before it leads other VPN services disrupted.

So, how you get the conclusion that it has nothing to do with VPN?

> 
> If you are ISP offering Internet transport unless you apply proper protection you would be badly exposed. Your clients or peers or upstreams injecting millions of routes and melting your network and perhaps even global Internet (if they would use a registered block). 
> 
> And yes BGP ingress policy here is used to filter out junk before it enters any network. The same policy must be used in VPN cases too. 
[WAJ] RD-ORF doesn’t  preclude the usage of other protection methods. I have said this point several time. Please remember this.
> 
> Indeed years have passed and I think I have only seen in a very few cases that operators offering L3VPNs are doing prefix ingress filtering ... Max prefix on ingress is used much more often. Those are the right tools here to work on. I am not saying we should not invent more ... we should.

[WAJ] OK.

> * Augment RRs to be a bit more intelligent with pitch of ML - if number of routes with given RD for time Tx is R moment you receive 100*R you suspend those and raise NOC alarm before spraying everywhere

[WAJ] RD-ORF mechanism is just one kind of ML capabilities, not only on RR, but also on other PE devices. We just want the network more intelligent and can cope with some extra situations automatically, not always static configuration.

> 
> * Do not put VPN customer routes into your data plane ... just handle next hops. Redefine RFC4364 all together and use IP transport for it. 
> 
> etc ... 
> 
> Focus on not allowing the meltdown is the proper solution space ... But here instead we do nothing to prevent the fire to start and instead focus on tools to extinguish it.

[WAJ] As you mentioned, there existing several methods to prevent the fire to start. But there still some chances the fire is ignited. Don’t you agree? RD-ORF can act as the spray system when the fire is beginning.

> Wrong approach. And specifically this tool (RD-ORF) does way too much damage when used. 

[WAJ] How to get this baseless conclusion? We have explained to your scenarios in previous mail. Do you have others or have still concerns the previous one?

>> From Susan Hares summary of where we are at with the adoption call let’s start with the problem this draft is trying to solve and gaining consensus.  Once we gain consensus we can get back to RD-ORF solution.  See w
>> 
>> The problem we are trying to solve is a scenario where you have an offending PE that is flooding routes and a weak PE that is overwhelmed by a flood of routes.  This is not a normal situation and is an outage situation where the weak PE being overwhelmed by a flood of routes.  Do we all agree to the problem statement?
>> 
>> 
>> 
>> Why and why not?
>> 
>> 
>> 
>> Do other methods exist that can solve the problem and if not do we need a new mechanism to solve this?
>> 
>> Do we all agree that in a normal situation we would never filter on RD as that would partition the VPN which is unwanted and what Robert mentioned.  As this is not a normal situation but a unique situation where a weak PE is overwhelmed by a flood of routes.  How best can this be solved?
>> 
>> 
>> 
>> 
>>> On Fri, Feb 12, 2021 at 10:32 AM Aijun Wang <[email-masked]> wrote:
>>> Hi, Robert:
>> Gyan Mishra
>> Network Solutions Architect 
>> M 301 502-1347
>> 13101 Columbia Pike 
>> Silver Spring, MD
>>

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Mon, 15 Feb 2021 12:34:49 +0000

Gyan,

              Looking at your first bullet (a). It would be useful to describe the terms below in a manner that in not interpretative.  TBH we seem to be going around in circles as the language being used by the authors of the draft is imprecise. Below find a number of examples of how your language could be understood ( Not an exhaustive list )..

Offending PE – What does this mean? It seems that is the flooding routes which is a result of what?

  1.  Multiple CEs advertising routes simultaneously causing a wave of paths at the PE
  2.  Redistribution from VRF<->VRF.
  3.  Redistribution from GRT-> VRF.
  4.  Offending PE is actually an Option B ASBR or is dual purposed
  5.  Other ?

Weak PE – How do you characterize a weak PE and how do you define overwhelmed?

  1.  Cannot process a given number of routes in time period X leading to dropping of routes
  2.  Delayed processing that may result in an incomplete number of inputs to the BGP Best Path decision.
  3.  L3VPN customers experiencing an incorrect VPN specification for some time period Y ( Timeliness )
  4.  Control Plane Processing impact Forwarding ( This would be surprising )
  5.  Other services that may be instantiated on the “Weak PE” are impacted i.e VPLS
  6.  Other ?

Can a PE be both “offending” and “weak”? I would assume so. I see no mention of this in your text below. How does your approach deal with said PE?

Thanks,
              Jim Uttaro

From: Idr <[email-masked]> On Behalf Of Gyan Mishra
Sent: Friday, February 12, 2021 10:16 PM
To: Aijun Wang <[email-masked]>
Cc: [email-masked]; Susan Hares <[email-masked]>; Robert Raszuk <[email-masked]>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

All

Why and why not?

On Fri, Feb 12, 2021 at 10:32 AM Aijun Wang <[email-masked]<mailto:[email-masked]>> wrote:
Hi, Robert:
Yes, the behavior of the device should be determined. There maybe several factors to be considered for this local behavior, we should describe it more clearly in this section later.
We have discussed the differences between RTC and RD-ORF a lot. As Haibo mentioned, they are not exclusive to each other, and can be used together in some situations. But they are different and can’t replace each other.
Aijun Wang
China Telecom

On Feb 12, 2021, at 23:04, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

Sorry Aijun,

Thx,
R.

On Fri, Feb 12, 2021 at 3:47 PM Aijun Wang <[email-masked]<mailto:[email-masked]>> wrote:
Hi, Robert:

https://datatracker.ietf.org/doc/html/draft-wang-idr-rd-orf-05#section-5.1.1<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-wang-idr-rd-orf-05*section-5.1.1__;Iw!!BhdT!wFDvJUfijC2hIGiH79yASFcLdf_N2ZgRzGAyZFBQ9qPt6NEnHHW4evpmkDA$> has described such situations, which will require the additional local decisions of PE2 to judge whether to send the RD-ORF message out.
In your example, if only the HUB VRF exceed but the resources of PE2 is not exhausted, then the PE2 will not send the RD-ORF message. It may just discard the excessive 100000/32 routes.
If the resources of PE2 is nearly exhausted, it must send the RD-ORF message out. Or else not only the Spoke VRF, but also other VPNs on this device can’t be used.

On Feb 12, 2021, at 19:30, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

Aijun & Gyan,

- - -

Summary:

Thx,
Robert

--

[Image removed by sender.]<https://urldefense.com/v3/__http:/www.verizon.com/__;!!BhdT!wFDvJUfijC2hIGiH79yASFcLdf_N2ZgRzGAyZFBQ9qPt6NEnHHW4U1-UgK8$>

Gyan Mishra

M 301 502-1347
13101 Columbia Pike
Silver Spring, MD

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 15 Feb 2021 09:47:17 -0500

Gyan: 

Do you feel the problem has been correctly described by Robert?  If not, please comment on problem. 

Cheers, Sue

From: Robert Raszuk [mailto:[email-masked]] 
Sent: Saturday, February 13, 2021 6:42 AM
To: Gyan Mishra
Cc: Aijun Wang; Jakob Heitz (jheitz); Susan Hares; idr@ietf. org
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

All,

Ideas: 

etc ... 

Focus on not allowing the meltdown is the proper solution space ... But here instead we do nothing to prevent the fire to start and instead focus on tools to extinguish it. Wrong approach. And specifically this tool (RD-ORF) does way too much damage when used. 

Thx,

R.

 

All

Why and why not?

 

Hi, Robert:

Aijun Wang

China Telecom

Sorry Aijun,

Thx,
R.

Hi, Robert:

Aijun Wang

China Telecom

Aijun & Gyan,

 

 

- - - 

Summary: 

 

Thx,

Robert

-- 

 <http://www.verizon.com/> Image removed by sender.

Gyan Mishra

M 301 502-1347
13101 Columbia Pike 
Silver Spring, MD

---

From: Gyan Mishra <[email-masked]>
Date: Mon, 15 Feb 2021 15:59:28 -0500

Hi James

Replies in-line

On Mon, Feb 15, 2021 at 7:35 AM UTTARO, JAMES <[email-masked]> wrote:

> *Gyan,*
>
>
>
> *              Looking at your first bullet (a). It would be useful to
> describe the terms below in a manner that in not interpretative.  TBH we
> seem to be going around in circles as the language being used by the
> authors of the draft is imprecise. Below find a number of examples of how
> your language could be understood ( Not an exhaustive list )..*
>
>
>
> *Offending PE – What does this mean? It seems that is the flooding routes
> which is a result of what? *
>
>    1. *Multiple CEs advertising routes simultaneously causing a wave of
>    paths at the PE*
>    2. *Redistribution from VRF<->VRF.*
>    3. *Redistribution from GRT-> VRF. *
>    4. *Offending PE is actually an Option B ASBR or is dual purposed*
>    5. *Other ?*
>
>
>
> *Gyan> a-d are possible.  Any trigger as the ones you mentioned that could
> cause  flooding of routes.   Single or Multiple VRFs being flooded
> simultaneously hitting VPN maximum prefix. Inter-AS B where “retain RT all”
> is specified where all RTs are imported.  *
>
> *Weak PE – How do you characterize a weak PE and how do you define
> overwhelmed?*
>
>    1. *Cannot process a given number of routes in time period X leading
>    to dropping of routes *
>    2. *Delayed processing that may result in an incomplete number of
>    inputs to the BGP Best Path decision.*
>    3. *L3VPN customers experiencing an incorrect VPN specification for
>    some time period Y ( Timeliness )*
>    4. *Control Plane Processing impact Forwarding ( This would be
>    surprising )*
>    5. *Other services that may be instantiated on the “Weak PE” are
>    impacted i.e VPLS *
>    6. *Other ?*
>
>
              Gyan> A-D are possible.  Definitely multiple services is a
big one such as VPLS / EVPN Mac route flood. The control plane issues can
impact the data plane forwarding on the weak PE.

>
> *Can a PE be both “offending” and “weak”? I would assume so. I see no
> mention of this in your text below. How does your approach deal with said
> PE?*
>
> *Gyan> It’s possible.  The  RD-ORF process is locally significant so the
> offending and weak PE could be the same device.  We can add some verbiage
> to the draft to account for this situation.*
>
> *Thanks,*
>
> *              Jim Uttaro*
>
> *From:* Idr <[email-masked]> *On Behalf Of * Gyan Mishra
> *Sent:* Friday, February 12, 2021 10:16 PM
> *To:* Aijun Wang <[email-masked]>
> *Cc:* [email-masked]; Susan Hares <[email-masked]>; Robert Raszuk <
> [email-masked]>
> https://datatracker.ietf.org/doc/html/draft-wang-idr-rd-orf-05#section-5.1.1
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-wang-idr-rd-orf-05*section-5.1.1__;Iw!!BhdT!wFDvJUfijC2hIGiH79yASFcLdf_N2ZgRzGAyZFBQ9qPt6NEnHHW4evpmkDA$> has
> <https://urldefense.com/v3/__http:/www.verizon.com/__;!!BhdT!wFDvJUfijC2hIGiH79yASFcLdf_N2ZgRzGAyZFBQ9qPt6NEnHHW4U1-UgK8$>
>
> *Gyan Mishra*
>
> *Network Solutions Architect *
>
>
>
> *M 301 502-1347 13101 Columbia Pike
> <https://www.google.com/maps/search/13101+Columbia+Pike?entry=gmail&source=g>
> *Silver Spring, MD
>
>
>
-- 

*Gyan Mishra*

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Mon, 15 Feb 2021 21:05:26 +0000

I asked you for the facts Aijun.
Show me an outage caused by a PE having to drop incoming routes.
You have not shown me the facts.

Here's a fact:
Some time ago, I was debugging an issue with a customer of a route that went missing in a VRF.
The feed from a route reflector included over a million routes.
I asked them to do a clear bgp soft in to the route reflector.
They were very hesitant, because it might overload the CPU on the PE.
"Do it", I said. "Check the InQ and when it goes to zero, it's done".
Finally, they did it.
"Check the InQ" I asked them.
"It's zero. When will it start?" they replied.
"Check the route. Is it there now?" I asked.
"Hey, there it is".
It finished the million routes before they could check the InQ.
Why so fast?
Because the BGP receiver had very little inbound policy, did not have to process the incoming routes, except one,
because they already existed in the BGP table.
Where we see convergence issues is:
- when BGP has to download incoming routes to FIB (FIB prioritizes forwarding, not programming of routes)
- Heavy inbound route-policies (several MEGABYTES of configuration). It takes some time, but it works.
- Recomputing and changing bestpath and propagating the new paths out to many RR clients.

Receiving and matching on an RD to drop a route is just so trivial in comparison.

In your case, the PE has ALREADY received the excess routes. Your rd-orf just tells the the source not to send ANY MORE.
You know what happens then? The source sends a million withdraws that take similar processing power on the receiver.

Regards,
Jakob.

From: Aijun Wang <[email-masked]>
Sent: Saturday, February 13, 2021 4:46 AM
To: Robert Raszuk <[email-masked]>
Cc: Gyan Mishra <[email-masked]>; Jakob Heitz (jheitz) <[email-masked]>; Susan Hares <[email-masked]>; [email-masked]
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Hi, Robert:

Aijun Wang
China Telecom

On Feb 13, 2021, at 19:42, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

All,

[WAJ] OK.

Ideas:

etc ...

Thx,
R.

On Sat, Feb 13, 2021 at 4:16 AM Gyan Mishra <[email-masked]<mailto:[email-masked]>> wrote:

All

Why and why not?

Sorry Aijun,

Thx,
R.

Aijun & Gyan,

- - -

Summary:

Thx,
Robert

--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<http://www.verizon.com/>

Gyan Mishra

M 301 502-1347
13101 Columbia Pike
Silver Spring, MD

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Mon, 15 Feb 2021 21:15:07 +0000

Here is another fact..

It is more like 10+ Million without an issue.

Thanks,
              Jim Uttaro

From: Idr <[email-masked]> On Behalf Of Jakob Heitz (jheitz)
Sent: Monday, February 15, 2021 4:05 PM
To: Aijun Wang <[email-masked]>; Robert Raszuk <[email-masked]>
Cc: [email-masked]; Susan Hares <[email-masked]>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Here's a fact:

Regards,
Jakob.

Hi, Robert:

Aijun Wang
China Telecom

All,

[WAJ] OK.

Ideas:

etc ...

Thx,
R.

All

Why and why not?

Sorry Aijun,

Thx,
R.

https://datatracker.ietf.org/doc/html/draft-wang-idr-rd-orf-05#section-5.1.1<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-wang-idr-rd-orf-05*section-5.1.1__;Iw!!BhdT!wARAO2f7msGVpnoCBq-6Cyc-tCz_NS2ySVnQDjVfsdcqki8vALlyuYmtg9k$> has described such situations, which will require the additional local decisions of PE2 to judge whether to send the RD-ORF message out.
In your example, if only the HUB VRF exceed but the resources of PE2 is not exhausted, then the PE2 will not send the RD-ORF message. It may just discard the excessive 100000/32 routes.
If the resources of PE2 is nearly exhausted, it must send the RD-ORF message out. Or else not only the Spoke VRF, but also other VPNs on this device can’t be used.

Aijun & Gyan,

- - -

Summary:

Thx,
Robert

--

[Image removed by sender.]<https://urldefense.com/v3/__http:/www.verizon.com/__;!!BhdT!wARAO2f7msGVpnoCBq-6Cyc-tCz_NS2ySVnQDjVfsdcqki8vALly2s7MIQU$>

Gyan Mishra

M 301 502-1347
13101 Columbia Pike
Silver Spring, MD

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 15 Feb 2021 22:31:12 +0100

Jim,

And quite frankly I think you nailed it.

If one is to provide a service he better purchase the right gear for it.

Not some weak PEs which balance on the cliff of falling under light wind
...

- - -

Mitigations which are deployed in many networks have been suggested.
Protect from issue not allow it to happen then try to patch it.

Alternative deployment models have been proposed too.

Sorry I do not buy the reason stated that mistaken redistribution of the
Internet to the VPN will crash the network. Right gear carries Internet in
a VRF just fine. And if it crashes ... so be it. Next time folks touching
PEs will be more careful.

I also do not accept as a reason Inter-AS option B or C. If you do not
trust your AS peer do A+B or A.

For me this topic is considered closed.

Thx,
Robert.

On Mon, Feb 15, 2021 at 10:15 PM UTTARO, JAMES <[email-masked]> wrote:

> *Here is another fact.. *
>
>
>
> *It is more like 10+ Million without an issue.*
>
>
>
> *Thanks,*
>
> *              Jim Uttaro*
>
>

---

From: Brendon H <[email-masked]>
Date: Mon, 15 Feb 2021 22:48:21 +0000

hi Susan,
i share the same concerns as those raised by Robert, Jim, Jacob etc..  i think they have continued to clearly articulate valid concerns, which have failed to be addressed by the authors.  i was wondering if the nuances in the language being used could be hindering the interpretation to those whose primary language isn't English; however, i don't think this is the case.  Either way this specific discussion is feeling like "Groundhog Day".

Due to the concerns already raised, i cannot support the adoption of the proposal.

regards,

Brendon

Sent from Outlook<http://aka.ms/weboutlook>
________________________________
From: Idr <[email-masked]> on behalf of Robert Raszuk <[email-masked]>
Sent: 16 February 2021 08:31
To: UTTARO, JAMES <[email-masked]>
Cc: Jakob Heitz (jheitz) <jheitz=[email-masked]>; Susan Hares <[email-masked]>; [email-masked] <[email-masked]>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Jim,

Not some weak PEs which balance on the cliff of falling under light wind ...

- - -

Mitigations which are deployed in many networks have been suggested. Protect from issue not allow it to happen then try to patch it.

Sorry I do not buy the reason stated that mistaken redistribution of the Internet to the VPN will crash the network. Right gear carries Internet in a VRF just fine. And if it crashes ... so be it. Next time folks touching PEs will be more careful.

I also do not accept as a reason Inter-AS option B or C. If you do not trust your AS peer do A+B or A.

Thx,
Robert.

On Mon, Feb 15, 2021 at 10:15 PM UTTARO, JAMES <[email-masked]<mailto:[email-masked]>> wrote:

Here is another fact..

Thanks,

              Jim Uttaro

---

From: Aijun Wang <[email-masked]>
Date: Tue, 16 Feb 2021 08:33:09 +0800

Hi, Jakob:

Aijun Wang
China Telecom

> On Feb 16, 2021, at 05:05, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 
> 

[WAJ] As you said, the routes withdraw process power is trivial, because the VPN routes are already there. But what the ORF mechanism want to avoid is not the withdraw action, but the advertised overwhelming VPN routes. Once you deployed, there will no possibility a million routes being advertised by the peer, and may need not do such debug issue?

---

From: Aijun Wang <[email-masked]>
Date: Tue, 16 Feb 2021 08:36:04 +0800

Hi, Jim:
It is clear that 1 million routes has caused an issue that let Jakob to debug.
Isn’t it?

Aijun Wang
China Telecom

> On Feb 16, 2021, at 05:15, UTTARO, JAMES <[email-masked]> wrote:
> 
> 
> Here is another fact..
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Tue, 16 Feb 2021 01:21:23 +0000

I was debugging one route.
My point was that the other 999,999 being received made no difference.

Regards,
Jakob.

From: Idr <[email-masked]> On Behalf Of Aijun Wang
Sent: Monday, February 15, 2021 4:36 PM
To: UTTARO, JAMES <[email-masked]>
Cc: Jakob Heitz (jheitz) <jheitz=[email-masked]>; Susan Hares <[email-masked]>; Robert Raszuk <[email-masked]>; [email-masked]
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Hi, Jim:

Aijun Wang
China Telecom

On Feb 16, 2021, at 05:15, UTTARO, JAMES <[email-masked]<mailto:[email-masked]>> wrote:

Here is another fact..

Thanks,
              Jim Uttaro

From: Idr <[email-masked]<mailto:[email-masked]>> On Behalf Of Jakob Heitz (jheitz)
Sent: Monday, February 15, 2021 4:05 PM
To: Aijun Wang <[email-masked]<mailto:[email-masked]>>; Robert Raszuk <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; Susan Hares <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Idr] WG Adoption call for draft-wang-idr-rd-orf-05.txt (2/4/2021 to 2/18/2021)

Here's a fact:

Regards,
Jakob.

Cc: Gyan Mishra <[email-masked]<mailto:[email-masked]>>; Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>>; Susan Hares <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>

Hi, Robert:

Aijun Wang
China Telecom

All,

[WAJ] OK.

Ideas:

etc ...

Thx,
R.

All

Why and why not?

Sorry Aijun,

Thx,
R.

Aijun & Gyan,

- - -

Summary:

Thx,
Robert

--

<https://urldefense.com/v3/__http:/www.verizon.com/__;!!BhdT!wARAO2f7msGVpnoCBq-6Cyc-tCz_NS2ySVnQDjVfsdcqki8vALly2s7MIQU$>
<https://urldefense.com/v3/__http:/www.verizon.com/__;!!BhdT!wARAO2f7msGVpnoCBq-6Cyc-tCz_NS2ySVnQDjVfsdcqki8vALly2s7MIQU$>

Gyan Mishra

M 301 502-1347
13101 Columbia Pike
Silver Spring, MD

---

From: Gyan Mishra <[email-masked]>
Date: Tue, 16 Feb 2021 15:59:20 -0500

Hi Robert

Responses In-line.  “Operators perspective”

Thanks

Gyan

> offending PE that is flooding routes and a weak PE that is overwhelmed by a

   Gyan> Correct

>
> The problem is a valid problem. But the proposed solution to the problem
> is not. And moreover solutions to this very problem are widely known and
> used for years.
>

    Gyan>  This issue with a weak PE has many variables and permutations
that could exist.  As their are always upgrade cycles the existence of
devices on older code or hardware.  The other variable is if the PE has
multiple services such as  a mix of L2 and L3 for example L3 VPN along with
L2 VPLS or EVPN type 2 Mac route flood or any other services that running
in parallel on the PE could exacerbate the issue.

>
> And yes BGP ingress policy here is used to filter out junk before it
> enters any network. The same policy must be used in VPN cases too.
>

   Gyan> As far as existing mitigation techniques, the problem is that
although they are pro active and not reactive to this particular flood
issue, they don’t solve the problem.  From an operator standpoint knobs
used today as you mentioned Ingres filtering is not user other then a
generic bit wise prefix list to permit a any  “ge le”.  Both PE-CE maximum
prefix and VPN maximum prefix are both set to a very high water mark
statistical multiplexing to protect from a flood on a single or few VRFs
but not a simultaneous flood on all VRFs as resources would be exhausted
even on a newer hardware strong PE.

> those and raise NOC alarm before spraying everywhere

Gyan> Sounds similar to BGP dampening of routes concept. This would require
maybe a new SAFI or codepoint , also the concept of suspend does not exist
in BGP so would require Major updates to BGP protocol itself which to me is
a more risky and much more intrusive a change. The nice thing about RD-ORF
is the existing machinery already exists with ORF and BGP route refresh.
Also that the ORF ask-rib-out filter  can be manually removed without
impact from flaps or any instability in the network.

> hops. Redefine RFC4364 all together and use IP transport for it.

   Gyan> The major reason why RD-ORF from an operators perspective as far
as risk is acceptable is that it is not a MAJOR rewrite or change to
existing BGP protocol as the others mentioned above being MAJOR code hacks
that can be very risky chance of bug from code collateral damage fixing one
thing could break something else.  SAFI 128 does not have separation of
control plane and data plane as does SAFI 129 MVPN or EVPN.  We have seen
that many times on the past with new features breaking existing features or
the protocol itself.  The beauty behind RD-ORF is that it used existing ORF
spec RFC 5291 and route refresh RFC 7313 so very minor change required.

> (RD-ORF) does way too much damage when used.

    Gyan> What you are proposing as the above ideas are major code hacks to
the BGP protocol to prevent  the fire pro actively.  Thus the attention
with this draft on extinguish the fire.  I think with the extinguish
concept reactive trigger to a problem as this uses existing BGP machinery
and very minor code change to implement it is the least risky to solve the
problem.  I think in the future as maybe a holistic future approach looking
at this issue and any other route flood type issues,  I am all for looking
at a possible solution in the future, but I think that should be
independent draft project and not hold up this solution.

>>> Yes, the behavior of the device should be determined. There maybe
>>> several factors to be considered for this local behavior, we should
>>> describe it more clearly in this section later.
>>> And all of this is not needed if on ingress between PE1 and HQ1 you
>>> apply max prefix of 2 or even 100. It is also not needed if you enable  RTC
>>> to send RT:TO_HUB from PE2 to RR.
>>>
>>>> Now imagine HQ1 bought a new BGP "Optimizer" and suddenly is starting
>>>> to advertise 100000 /32 routes just to the other HUB with RT: TO_HUB.
>>>>
>>>>
>>>>
>>>>
>>>> So PE2 detects this as VRF with RDX2 on it got overwhelmed during
>>>> import with RT TO_HUB and starts pushing RDX1 (original RD) to RR to stop
>>>> getting those routes.
>>>>
>> *M 301 502-134713101 Columbia Pike
>> <https://www.google.com/maps/search/13101+Columbia+Pike?entry=gmail&source=g> *Silver
>> Spring, MD
>>
>> --

*Gyan Mishra*

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 16 Feb 2021 22:27:44 +0100

Hi Gyan.

>  From an operator standpoint knobs used today as you mentioned Ingres
filtering is not user other then a generic bit wise prefix list to permit a
any  “ge le”.

The point is that if you use existing shipping implementations and apply
correct ingress policy - with zero code change to any shipping
implementation - you will protect from fire hence no need to invent new
extinguishers. Fire always results in damage so it's worth to spend extra
cfg effort to protect from it.

It works and scales for Internet so will work and scale for VPNs. Yes it
requires perhaps change to operational model of some providers on how
L3VPNs or L2VPNs are being offered - but it can be as simple as web page
for customer to enter what routes he intends to send. If he enters
100K routes while his SLA allows max 100 per site simple web page
validation can provide an error and no policy will be installed on the
PEs.

Thx,
Robert

---

From: Chenshuanglong <[email-masked]>
Date: Thu, 18 Feb 2021 01:45:53 +0000

Hi ALL
I have reviewed the document and it is useful for inter-AS VPN deployment and service provisioning, I support progressing this document to be published as Standard RFC.

Regards
Shuanglong

Cheerily, Susan Hares

---

From: "Chengli (Cheng Li)" <[email-masked]>
Date: Thu, 18 Feb 2021 03:15:40 +0000

Yes, support.

Cheng

Cheerily, Susan Hares

---

From: "[email-masked]" <[email-masked]>
Date: Thu, 18 Feb 2021 12:01:35 +0800

Hello All,

After disscussion and clarification, I believe the problem to be addressed by this doc is clear and valid. The mechanism proposed is useful in some situations although some aspects have to be improved and considered as mentioned in the list during this adotion call.  
So, I support its adoption.

Best Regards,
Zhenqiang Li

[email-masked]
 
From: Susan Hares
Date: 2021-02-04 23:38
To: [email-masked]

 

Cheerily, Susan Hares

---

From: "Weiqiang Cheng" <[email-masked]>
Date: Thu, 18 Feb 2021 17:37:55 +0800

Support the adoption.

B.R.

Weiqiang Cheng

Cheerily, Susan Hares