Subject: [Idr] draft-ietf-idr-bgp-open-policy-20 - error handling

From: <[email-masked]>
Date: Wed, 19 Jan 2022 10:48:05 +0000

Hi all,

Disclaimer: I haven't read the draft in detail. 

Thank you for having considered and defined the error handling for the OTC attribute.
Drafts says:

"   If an eBGP speaker receives an UPDATE with an OTC Attribute with a
   length different from 4 octets, then the UPDATE SHALL be considered
   malformed.  If malformed, the UPDATE message SHALL be handled using
   the approach of "treat-as-withdraw" [RFC7606]."

Q1: Is the OTC attribute error check to be performed only when received from an eBGP session or from any type of BGP session?
Q2: What is the analysis for not using "attribute discard" [7606]?
It's not obvious to me that removing the OTC attribute ("attribute-discard") received from an eBGP session would cause issue. (RFC7606 says " The analysis should take into account the tradeoff between preserving connectivity and potential side effects.")

Thanks,
Regards,
--Bruno

Orange Restricted

> -----Original Message-----
> From: Idr <[email-masked]> On Behalf Of [email-masked]
> Sent: Tuesday, January 18, 2022 12:15 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: [Idr] I-D Action: draft-ietf-idr-bgp-open-policy-20.txt
> 
> 
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> This draft is a work item of the Inter-Domain Routing WG of the IETF.
> 
>         Title           : Route Leak Prevention and Detection using Roles in UPDATE
> and OPEN Messages
>         Authors         : Alexander Azimov
>                           Eugene Bogomazov
>                           Randy Bush
>                           Keyur Patel
>                           Kotikalapudi Sriram
> 	Filename        : draft-ietf-idr-bgp-open-policy-20.txt
> 	Pages           : 13
> 	Date            : 2022-01-18
> 
> Abstract:
>    Route leaks are the propagation of BGP prefixes that violate
>    assumptions of BGP topology relationships, e.g., announcing a route
>    learned from one transit provider to another transit provider or a
>    lateral (i.e., non-transit) peer or announcing a route learned from
>    one lateral peer to another lateral peer or a transit provider.
>    These are usually the result of misconfigured or absent BGP route
>    filtering or lack of coordination between autonomous systems (ASes).
>    Existing approaches to leak prevention rely on marking routes by
>    operator configuration, with no check that the configuration
>    corresponds to that of the eBGP neighbor, or enforcement that the two
>    eBGP speakers agree on the relationship.  This document enhances the
>    BGP OPEN message to establish an agreement of the relationship on
>    each eBGP session between autonomous systems in order to enforce
>    appropriate configuration on both sides.  Propagated routes are then
>    marked according to the agreed relationship, allowing both prevention
>    and detection of route leaks.
> 
> 
> 
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-ietf-idr-bgp-open-policy/
> 
> There is also an htmlized version available at:
> https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-open-policy-20
> 
> A diff from the previous version is available at:
> https://www.ietf.org/rfcdiff?url2=draft-ietf-idr-bgp-open-policy-20
> 
> 
> Internet-Drafts are also available by rsync at rsync.ietf.org::internet-drafts
> 
> 
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 19 Jan 2022 09:45:45 -0500

Bruno,

> Q1: Is the OTC attribute error check to be performed only when received from an eBGP session or from any type of BGP session?

Syntactic checks are generally done for every update received regardless of
source.

The eBGP portion for the validity should likely be removed.

> It's not obvious to me that removing the OTC attribute ("attribute-discard") received from an eBGP session would cause issue. (RFC7606 says " The analysis should take into account the tradeoff betw…

I have opinions here, but this one I'd also like to hear from the authors.

-- Jeff

---

From: "Sriram, Kotikalapudi (Fed)" <[email-masked]>
Date: Thu, 20 Jan 2022 15:58:01 +0000

Hi Bruno and Jeff:

Thank you for the comments/discussion. Responses inline.

>Syntactic checks are generally done for every update received regardless of source.

Jeff:

I think you are suggesting the following. Yes?

s/ If an eBGP speaker receives an UPDATE with an OTC Attribute... / If an UPDATE is received with an OTC Attribute... /

The suggestion seems good to me.

>I have opinions here, but this one I'd also like to hear from the authors.

RFC 7606 says:

   o  Attribute discard: In this approach, the malformed attribute MUST
      be discarded and the UPDATE message continues to be processed.
      This approach MUST NOT be used except in the case of an attribute
      that has no effect on route selection or installation.

So, what we have is:

Attribute discard MUST NOT be used except in the case of an attribute
that has no effect on route selection or installation.

which implies:

Attribute discard can be used in the case of an attribute
that has no effect on route selection or installation.

which implies:

Attribute discard need not be used in the case of an attribute
that has an effect on route selection or installation.

In the draft, the OTC Attribute is meant to have an effect on route selection (if a leak is detected based on OTC, the route is considered ineligible).

If setting the OTC Attribute was intentional but it got corrupted at some point, the unreachability may be better than allowing a route leak.

Please let us know if you see a different interpretation or analysis that is useful.

Thanks.

Sriram

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 20 Jan 2022 11:18:08 -0500

Sriram,

> The suggestion seems good to me.

Thanks for addressing the concern.

> >I have opinions here, but this one I'd also like to hear from the authors.

[...]

> Please let us know if you see a different interpretation or analysis that is useful.

This largely matches my opinion for using treat-as-withdraw.

-- Jeff

---

From: <[email-masked]>
Date: Thu, 20 Jan 2022 17:17:10 +0000

Sriram, Jeff

Thanks for the reply. Much appreciated.
Please see inline [Bruno]

Orange Restricted
From: Sriram, Kotikalapudi (Fed) <[email-masked]>
Sent: Thursday, January 20, 2022 4:58 PM
To: Jeffrey Haas <[email-masked]>; DECRAENE Bruno INNOV/NET <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: RE: [Idr] draft-ietf-idr-bgp-open-policy-20 - error handling

Hi Bruno and Jeff:

Jeff:

[Bruno] +1

RFC 7606 says:

So, what we have is:

which implies:

which implies:

[Bruno] I agree with the reading of RFC 7606. However, my reading of OTC is that OTC is used to for the filtering of routes over eBGP sessions only. OTC has no effect over IBGP sessions.
[RFC7606] says that attribute discard is unsafe in general, and does not distinguish iBGP vs eBGP sessions (probably because most attributes have effects on iBGP sessions). However, I believe that there is a difference (and an early revision of 7606 made that difference [1] )

  *   For iBGP, attribute discard is unsafe as it leads to inconsistencies within the AS and hence potentially leads to forwarding loops (well, treat as withdraw is also risky, but better with some assumptions)
  *   For eBGP, I would argue that it is safe to filter routes advertised/received over eBGP session: this is widely used today by policies and does not create forwarding loops.

[1] https://datatracker.ietf.org/doc/html/draft-chen-ebgp-error-handling-01

Regarding OTC error handling:

  *   Do you have in mind an example where discarding OTC would be unsafe for the routing consistency?
  *   I do have example where treat as withdraw would be problematic for OTC. R1 is an IP transit router in the middle of the AS. If receives 198.51.100.0/24 with OTC and believe the attribute is in error and hence discard it. Its neighbour, R2 does not. The AS also have a less specific route (e.g. 198.51/16 following a different path across the AS). In this case, we may have a persistent forwarding loops (between the /24 and the /8 as R1 and R2 have a inconsistent knowledge of the routing table)

If setting the OTC Attribute was intentional but it got corrupted at some point, the unreachability may be better than allowing a route leak.
[Bruno] Or it may not.
In my experience attribute errors are caused by implementations (errors, or deliberate changes to the standard). In such case, the vast majority of routes would not be leaks, while treat-as-withdraw would withdraw all those routes.

Thanks for the discussion.
--Bruno

Thanks.

Sriram

---

From: Alvaro Retana <[email-masked]>
Date: Fri, 21 Jan 2022 17:12:23 +0000

On January 20, 2022 at 12:18:15 PM, [email-masked] wrote:

Bruno:

Hi!

...
> I do have example where treat as withdraw would be problematic for OTC. R1
> is an IP transit router in the middle of the AS. If receives 198.51.100.0/24
> with OTC and believe the attribute is in error and hence discard it. Its
> neighbour, R2 does not. The AS also have a less specific route (e.g.
> 198.51/16 following a different path across the AS). In this case, we may
> have a persistent forwarding loops (between the /24 and the /8 as R1 and R2
> have a inconsistent knowledge of the routing table)

The result is due to the inconsistent behavior between R1 and R2.  Why
doesn't R2 also consider the attribute in error?  If the behavior is
consistent the I don't see this an an issue.  Am I missing something?

Thanks!

Alvaro.

---

From: <[email-masked]>
Date: Fri, 21 Jan 2022 17:28:53 +0000

Hi Alvaro,

Thanks for engaging.

> From: Alvaro Retana <[email-masked]>
> 

Two possible answers:
1) Let's call R0 the node originating the OTC attribute.
If R0 and R2 are using the same implementation, they are likely to behave the same way, i.e. assume that they are right about the format of the attribute (e.g. the implementor misread the spec, or use a different version of the draft...).

That's not a theoretical consideration. I have a recent real world example with the BGP Entropy Label Capability.
 
2) There is an error when the receiver is not happy with the format received.
Being on the receiver, many people assume that the error was on the sender (the other 'buggy' implementation, not mine for sure as mine is smart). But in fact there is a chance that the bug be in fact on the reader/receiver. So R1 would have this bug and not R2.

--Bruno

Orange Restricted

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 21 Jan 2022 14:14:53 -0500

Alvaro,

Bruno's case about different implementations is one of the core cases.

The other is simply incremental deployment.

A detail I think that may be inconsistently represented in these discussions is expectations about what checks the attribute.

BGP syntactic checks are typically done at EVERY node that understands the feature.  This means what you do about treat-as-withdraw or discard may happen at an iBGP node.

If Bruno is thinking that the syntactic check only happens at an eBGP node, we're in disagreement over expected behavior.

To Bruno's point in the thread (the joys of MS 'quoting'), implementations certainly are free to strip and then "take an action"... but that's not how current implementations behave.  I wouldn't want to predicate a repair operation based on a future change of implementations for a new idea.

-- Jeff

> On Jan 21, 2022, at 12:28 PM, <[email-masked]> <[email-masked]> wrote:
> 
> Hi Alvaro,
> 
> Thanks for engaging.

---

From: "Susan Hares" <[email-masked]>
Date: Tue, 25 Jan 2022 12:16:00 -0500

Bruno: 

I am catching up with old mail.  Just so I understand your example: 
 
   +------------------+
--|  R2---R1---R0  |---- 
   |    |                 |    |
   |    ^                 v    |
   |     R4-------R3  |
   +-----------------+
      AS-1

Is this the scenario You are considering where the /8 follows R3/R4? 

Thanks, Sue 

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Alvaro Retana
Sent: Friday, January 21, 2022 12:12 PM
To: Jeffrey Haas; Sriram, Kotikalapudi (Fed); [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: [Idr] draft-ietf-idr-bgp-open-policy-20 - error handling

Bruno:

Hi!

...

Thanks!

Alvaro.

---

From: <[email-masked]>
Date: Tue, 25 Jan 2022 17:43:44 +0000

Sue,

I'm considering two prefixes, one being less specific. e.g. a /24 and a /16 (*)
Let's use a simple forwarding topology: 
(/16)  R3--R2--R1--R0  (/24)

All nodes within the same AS.

R0 receives from eBGP the /24 with OTC
R3 receives from eBGP the /16 without OTC

R1 sees an error in the OTC and treat as withdraw.

A packet from R2 toward a /32 would loop between R2 and R1 since those two routers don’t have a consistent view of the iBGP routes (R2 RIB: /16, /24; R1 RIB: /16)

That would not happen with the "attribute discard" error handling. And "attribute discard" would not create routing inconsistencies when transiting the AS as the OTC attribute is not used when transiting the AS. (only from/to eBGP sessions so when crossing an AS boundary)

Thanks,
Bruno
(*) in my below email, the /16 and the /8 were meant to be the same covering prefix. My mistake)

Orange Restricted

> From: Susan Hares <[email-masked]>
> Sent: Tuesday, January 25, 2022 6:16 PM
> To: 'Alvaro Retana' <[email-masked]>; 'Jeffrey Haas' <[email-masked]>;
> 'Sriram, Kotikalapudi (Fed)' <[email-masked]>; DECRAENE Bruno
> INNOV/NET <[email-masked]>

---

From: <[email-masked]>
Date: Tue, 25 Jan 2022 18:00:11 +0000

Jeff,

Please see inline

> From: Jeffrey Haas <[email-masked]>
> 
> Alvaro,
> 
> A detail I think that may be inconsistently represented in these discussions is
> expectations about what checks the attribute.
> 
> BGP syntactic checks are typically done at EVERY node that understands the
> feature.  This means what you do about treat-as-withdraw or discard may happen
> at an iBGP node.
> 
> If Bruno is thinking that the syntactic check only happens at an eBGP node,

No, I don't.

That was my first question since draft -20 specifically referred to error handling only on the eBGP speaker
OLD (-20):
   "If an eBGP speaker receives an UPDATE with an OTC Attribute with a
   length different from 4 octets, then the UPDATE SHALL be considered
   malformed."
https://www.ietf.org/rfcdiff?url2=draft-ietf-idr-bgp-open-policy-21.txt

But following my question this point has been clarified in -21

NEW (-21):
"If an UPDATE is received with an OTC Attribute[...]"

So that point is now clear.

> we're
> in disagreement over expected behavior.
> 
> To Bruno's point in the thread (the joys of MS 'quoting'), implementations certainly
> are free to strip and then "take an action"... but that's not how current
> implementations behave.  I wouldn't want to predicate a repair operation based on
> a future change of implementations for a new idea.

I'm probably misunderstanding your point, regarding how current implementation behaves and changing their behavior.
>From the implementation reports, we only have a limited number of implementations (hence deployment at best), plus both do not implement the error handling behavior defined in the draft hence would need to be modified.. So I don't feel that this is to late to change this point in the spec.
https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-bgp-open-policy

--Bruno
> > If R0 and R2 are using the same implementation, they are likely to behave the
> same way, i.e. assume that they are right about the format of the attribute (e.g. the
> implementor misread the spec, or use a different version of the draft...).
> >
> > Being on the receiver, many people assume that the error was on the sender (the
> other 'buggy' implementation, not mine for sure as mine is smart). But in fact there
> is a chance that the bug be in fact on the reader/receiver. So R1 would have this
> bug and not R2.
> >
> > --Bruno
> >
> >> Thanks!
> >>
> >> Alvaro.
> >
> > Orange Restricted
> >
> >
> ______________________________________________________________________
> ___________________________________________________
> >
> > Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc
> > a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration,
> > This message and its attachments may contain confidential or privileged
> information that may be protected by law;

Orange Restricted

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 25 Jan 2022 15:13:36 -0500

Bruno,

> On Jan 25, 2022, at 1:00 PM, <[email-masked]> <[email-masked]> wrote:

Good.

> From the implementation reports, we only have a limited number of implementations (hence deployment at best), plus both do not implement the error handling behavior defined in the draft hence would…

RFC 7606 covers two behaviors of possible use here from a standards perspective.

1. Remove the attribute.
2. Treat as withdraw.

"Take an action" upon doing one of these things isn't a standardized behavior.

If your intent is to do one of the above plus "take an action", that widens the scope of this discussion.

-- Jeff

> https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-bgp-open-policy <https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-bgp-open-policy>
> 
> --Bruno
>> 
>> -- Jeff
>> 
>>

---

From: John Scudder <[email-masked]>
Date: Tue, 25 Jan 2022 20:39:38 +0000

Hi Bruno,

The general case you’ve illustrated is that treat-as-withdraw is a blunt instrument, that can have its own negative effects when used. The scenario you’ve exhibited could apply equally with any attribute that’s subject to treat-as-withdraw treatment, of which there are quite a few. It’s too bad we didn’t capture this scenario in RFC 7606; nonetheless I don’t think we’d have done something different since 7606 was an exercise in compromising between bad and worse.

But, I guess your real point is that the somewhat-ambiguous text of -20 could have been clarified in either of two ways:

1. As was done in -21, remove the reference to EBGP such that the error handling text applies universally. You make the case that this makes things worse, not better.

2. Alternatively the text could have said something like “MUST treat-as-withdraw if received on EBGP, MUST attribute discard if received on IBGP”[*] This kind of differentiated checking would be unusual I think, but I don’t see any negative to doing it that way. I don’t think the loop concern you exhibited exists when you do treat-as-withdraw at an EBGP boundary, does it? 

$0.02,

—John

[*] Confederation EBGP left as an exercise to the reader I suppose.

> On Jan 25, 2022, at 12:43 PM, [email-masked] wrote:
> 
> 
> Sue,
> 
>> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!SQZI53J2vkvVHaKnnt-iNzJ2vVFA93A4xXMg2G-Y1rroXwFYlfZP8Aghv8uk5A$
> 
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO-gk!SQZI53J2vkvVHaKnnt-iNzJ2vVFA93A4xXMg2G-Y1rroXwFYlfZP8Aghv8uk5A$

---

From: Ben Maddison <[email-masked]>
Date: Wed, 26 Jan 2022 11:32:06 +0200

Hi Bruno,

On 01/25, [email-masked] wrote:
> A packet from R2 toward a /32 would loop between R2 and R1 since those
> two routers don’t have a consistent view of the iBGP routes (R2 RIB:
> /16, /24; R1 RIB: /16)
> 
> That would not happen with the "attribute discard" error handling. And
> "attribute discard" would not create routing inconsistencies when
> transiting the AS as the OTC attribute is not used when transiting the
> AS. (only from/to eBGP sessions so when crossing an AS boundary)
> 
Agreed.
However if R1 has its own eBGP peers (or is an RR), the missing OTC
could induce a leak.

It's hard to say ahead of time whether the leak or the loop is more
disruptive. The loop, at least, is capable of being diagnosed and
addressed by only one operator's NOC.

All sorts of bad things happen when there is a bug in an implementation.
I'm disinclined toward accommodating them in the protocol if that
dilutes the effectiveness of the protocol itself (which I think is the
case if we choose "attribute discard").

Mostly, I consider this an(other) argument for tunneling between ASBRs.

Cheers,

Ben

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 26 Jan 2022 08:24:41 -0500

Ben,

> On Jan 26, 2022, at 4:32 AM, Ben Maddison <benm=[email-masked]> wrote:

I have to argue on this point a little.

Current operational procedures are that if you permit something into your iBGP, you expect it to get distributed to the rest of your ASBRs. 

If this doesn't happen, it's a serious issue.  I don't believe most people have operationalized infrastructure to catch such mismatches.

> case if we choose "attribute discard").

I agree that this is the important detail to conclude.  At this point we're debating "how bad" along with whether certain classes of fault can be mitigated.

-- Jeff

---

From: "Sriram, Kotikalapudi (Fed)" <[email-masked]>
Date: Wed, 26 Jan 2022 14:47:55 +0000

Bruno,

Can you please clarify your scenario? I feel that it should be well understood to discuss further. I am not seeing the possibility of data-looping.

If R0 is the only erroneous implementation, then R2 or any other router (besides R0) will not have the /24 installed as a viable route... because they all either don't receive the /24 or apply treat-as-withdraw correctly as specified in the draft. Can you please explain how you envision R2 having the /24 in its forwarding table?

You considered a "simple forwarding topology" topology. Feel free to consider any other topology.

Thank you.
Sriram

-----Original Message-----
From: [email-masked]<mailto:[email-masked]> [email-masked]<mailto:[email-masked]>
Sent: Tuesday, January 25, 2022 12:44 PM
To: Susan Hares [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; 'Alvaro Retana' [email-masked]<mailto:[email-masked]>; 'Jeffrey Haas' [email-masked]<mailto:[email-masked]>; Sriram, Kotikalapudi (Fed) [email-masked]<mailto:[email-masked]>
Subject: RE: [Idr] draft-ietf-idr-bgp-open-policy-20 - error handling
Sue,
I'm considering two prefixes, one being less specific. e.g. a /24 and a /16 (*) Let's use a simple forwarding topology:

A packet from R2 toward a /32 would loop between R2 and R1 since those two routers don't have a consistent view of the iBGP routes (R2 RIB: /16, /24; R1 RIB: /16)

Thanks,
Bruno

---

From: <[email-masked]>
Date: Wed, 26 Jan 2022 16:31:27 +0000

Hi John,

Thanks for the email.

> From: John Scudder <[email-masked]>
> Sent: Tuesday, January 25, 2022 9:40 PM
> 
> Hi Bruno,
> 
> The general case you’ve illustrated is that treat-as-withdraw is a blunt instrument,
> that can have its own negative effects when used. The scenario you’ve exhibited
> could apply equally with any attribute that’s subject to treat-as-withdraw treatment,
> of which there are quite a few. It’s too bad we didn’t capture this scenario in RFC
> 7606; nonetheless I don’t think we’d have done something different since 7606 was
> an exercise in compromising between bad and worse.

[Bruno] Agreed.
Regarding RFC 7606, I believe this is well captured in the document and clear for the WG at the time. e.g. https://datatracker.ietf.org/doc/html/rfc7606#section-6 I would not suggest any specific change.

> 1. As was done in -21, remove the reference to EBGP such that the error handling
> text applies universally. You make the case that this makes things worse, not
> better.

[Bruno] Regarding the reference to EBGP, I wanted a clarification regarding whether the check was only done on AS boundary or also within the AS (IBGP) (as the draft was only referring to EBGP and not mentioning anything on the IBGP case; and hence seemed possibly ambiguous to me).
The reply was that the error handling is made in all cases. As a consequence, reference to "EBGP" was removed.
Indeed, I don't feel that "treat-as-withdraw" within the AS is the "optimal" behavior, so I'm asking for a discussion on this.

> 2. Alternatively the text could have said something like “MUST treat-as-withdraw if
> received on EBGP, MUST attribute discard if received on IBGP”[*] This kind of
> differentiated checking would be unusual I think, but I don’t see any negative to
> doing it that way. I don’t think the loop concern you exhibited exists when you do
> treat-as-withdraw at an EBGP boundary, does it?

[Bruno]
Your proposition works for me. I equally believe that there would be no loop for EBGP (that's akin to a regular filtering/route policy).
Alternatively, in order to avoid the differentiated checking, I believe "attribute discard" for both EBGP and IBGP would work just fine:
- we already agree on IBGP
- for EBGP, essentially we can't use the OTC attribute, so why not handling the route as if the OTC attribute were not there? (plus discard it to remove the error). I feel sorry that the OTC feature be lost, but that's a given in all cases. In the absence of a (readable) OTC,why should be assume that the route is a leak and hence withdraw it?

In all cases, my comment is very late so can be dismissed if needed.
I just don't feel that "treat-as-withdraw" in case of attribute error is best both for the AS transiting the route, and the AS originating the route. (and hence for the deployment of OTC)

Quite valuable to me at least.
Thank you.

--Bruno

> > A packet from R2 toward a /32 would loop between R2 and R1 since those two
> routers don’t have a consistent view of the iBGP routes (R2 RIB: /16, /24; R1 RIB:
> /16)
> >
> >
> > That would not happen with the "attribute discard" error handling. And "attribute
> discard" would not create routing inconsistencies when transiting the AS as the
> OTC attribute is not used when transiting the AS. (only from/to eBGP sessions so
> when crossing an AS boundary)
> >
> > Thanks,
> > Bruno
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO
> -gk!SQZI53J2vkvVHaKnnt-iNzJ2vVFA93A4xXMg2G-Y1rroXwFYlfZP8Aghv8uk5A$
> >
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!NEt6yMaO
> -gk!SQZI53J2vkvVHaKnnt-iNzJ2vVFA93A4xXMg2G-Y1rroXwFYlfZP8Aghv8uk5A$

Orange Restricted

---

From: <[email-masked]>
Date: Wed, 26 Jan 2022 16:35:40 +0000

Jeff,

Orange Restricted
From: Jeffrey Haas <[email-masked]>
Sent: Tuesday, January 25, 2022 9:14 PM
To: DECRAENE Bruno INNOV/NET <[email-masked]>
Cc: Sriram, Kotikalapudi (Fed) <[email-masked]>; [email-masked]; [email-masked]; Alvaro Retana <[email-masked]>
Subject: Re: [Idr] draft-ietf-idr-bgp-open-policy-20 - error handling

Bruno,

[...]

we're

1. Remove the attribute.
2. Treat as withdraw.

[Bruno] Thanks for the clarification as I had mis-understood your response. My proposition is to do RFC 7606 "Attribute discard" (for both EBGP and IBGP).

--Bruno

-- Jeff

--Bruno

-- Jeff

On Jan 21, 2022, at 12:28 PM, <[email-masked]<mailto:[email-masked]>>
<[email-masked]<mailto:[email-masked]>> wrote:

Hi Alvaro,

Thanks for engaging.

From: Alvaro Retana <[email-masked]<mailto:[email-masked]>>

On January 20, 2022 at 12:18:15 PM, [email-masked]<mailto:[email-masked]> wrote:

Bruno:

Hi!

...

Two possible answers:

That's not a theoretical consideration. I have a recent real world example with the
BGP Entropy Label Capability.

--Bruno

Thanks!

Alvaro.

Orange Restricted

pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler

Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.

they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this
message and its attachments.

As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.

Thank you.

Orange Restricted

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 26 Jan 2022 11:39:11 -0500

> On Jan 26, 2022, at 11:31 AM, <[email-masked]> <[email-masked]> wrote:
> 
>> 

Summarizing some discussion John and I are having in a different thread, OTC has some benefit as "one bit of signal" in these cases.  Present, but with invalid contents, many of the procedures can still be done.

This introduces a possibility of doing "attribute overwrite": Replace the malformed attribute with a properly formed entry with something like AS 0.  Afterwards, standard procedures likely solve the problems discussed.

-- Jeff

---

From: <[email-masked]>
Date: Wed, 26 Jan 2022 17:08:16 +0000

Hi Ben,

Thanks for engaging.

> From: Ben Maddison <[email-masked]>
> Sent: Wednesday, January 26, 2022 10:32 AM
> 
> Hi Bruno,
> 

[Bruno] We are discussing the situation where the OTC attribute is unreadable and hence the OTC feature is lost.
Now we have a route with no (valid) OTC attribute. There are many such routes in the Internet today, and probably still in the short to medium term. Why should be withdraw/drop that route?
(Yes, the route could be many things (e.g. a leak, required for an emergency call or for lots of business...) I'm not seeing the value of making hypothesis given that we have zero information: we are dealing with a route with no OTC info)

> disruptive.

There is zero indication of a leak. Why do/would you assume one?

> The loop, at least, is capable of being diagnosed and

RFC 7606 covers this requirement for diagnosis and correction of the bug  https://datatracker.ietf.org/doc/html/rfc7606#section-6 in a way which seems relatively simple and typical to me.
On the other hand, detecting a forwarding loop seems more difficult to me (transient loops happen regularly, during many/all IGP convergence and many IBGP convergence if tunneling is not used, and I'm not sure that this is monitored and tracked by many ASes)

> All sorts of bad things happen when there is a bug in an implementation.

Agreed. We all wish there were none, in which case the discussion would be moot.

> case if we choose "attribute discard").

Is your inclination up to calling for RFC 4271 "session reset" to make sure we are not accommodating them in the protocol?
On my side I'd prefer the bad thing to be a little less bad.

> Mostly, I consider this an(other) argument for tunneling between ASBRs.

+1 for tunneling solves the IBGP inconsistencies.

And in a AS with no tunneling, the same goal (transit across the AS with no routing inconsistency) would mean that the IP node in transit should behave as if it were doing "attribute discard" rather than "treat as withdraw". (IOW, the OTC attribute is best not read by IBGP nodes in transit). That seem to be in line with my proposal.)

In all cases, thanks for the discussion.

Cheers,
--Bruno

Orange Restricted

---

From: <[email-masked]>
Date: Wed, 26 Jan 2022 17:14:43 +0000

Sriram,

Orange Restricted
From: Sriram, Kotikalapudi (Fed) <[email-masked]>
Sent: Wednesday, January 26, 2022 3:48 PM
To: DECRAENE Bruno INNOV/NET <[email-masked]>; Susan Hares <[email-masked]>
Cc: [email-masked]; [email-masked]; 'Alvaro Retana' <[email-masked]>; 'Jeffrey Haas' <[email-masked]>; Ben Maddison <[email-masked]>
Subject: RE: [Idr] draft-ietf-idr-bgp-open-policy-20 - error handling

Bruno,

Can you please clarify your scenario? I feel that it should be well understood to discuss further.
[Bruno] +1

I am not seeing the possibility of data-looping.
If R0 is the only erroneous implementation, then R2 or any other router (besides R0) will not have the /24 installed as a viable route... because they all either don't receive the /24 or apply treat-as-withdraw correctly as specified in the draft.

[Bruno] Agreed for your first hypothesis. The other hypothesis is that R0 is not the erroneous implementation. R1 is the erroneous implementation and sees an error in the OTC attribute while the OTC attribute is per the spec.

Can you please explain how you envision R2 having the /24 in its forwarding table?
[Bruno] Both R0 and R2 are correct implementations. (or alternatively, they have the same implementation/bug and believe the OTC attribute is correct while it is not)
Thank you,
--Bruno

Thank you.
Sriram

-----Original Message-----
From: [email-masked]<mailto:[email-masked]> [email-masked]<mailto:[email-masked]>
Sent: Tuesday, January 25, 2022 12:44 PM
To: Susan Hares [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; 'Alvaro Retana' [email-masked]<mailto:[email-masked]>; 'Jeffrey Haas' [email-masked]<mailto:[email-masked]>; Sriram, Kotikalapudi (Fed) [email-masked]<mailto:[email-masked]>
Subject: RE: [Idr] draft-ietf-idr-bgp-open-policy-20 - error handling
Sue,

Thanks,
Bruno

---

From: John Scudder <[email-masked]>
Date: Wed, 26 Jan 2022 17:20:50 +0000

As one meta-point on this thread, we are at risk of overthinking: the probability of this particular case being exercised seems quite low, since it involves a sender injecting a wrong-length OTC attribute; a fairly egregious bug. So, it may be that any solution where the cure isn’t worse than the disease would probably be fine, even if it’s not theoretically optimal.

Bruno was making a “cure is worse than the disease” case for the version 21 text, is how I see it. Several of the alternatives being discussed all seem fine, though.

—John

---

From: <[email-masked]>
Date: Wed, 26 Jan 2022 17:42:45 +0000

Jeff,

Orange Restricted
From: Jeffrey Haas <[email-masked]>
Sent: Wednesday, January 26, 2022 5:39 PM

On Jan 26, 2022, at 11:31 AM, <[email-masked]<mailto:[email-masked]>> <[email-masked]<mailto:[email-masked]>> wrote:

2. Alternatively the text could have said something like "MUST treat-as-withdraw if
received on EBGP, MUST attribute discard if received on IBGP"[*] This kind of
differentiated checking would be unusual I think, but I don't see any negative to
doing it that way. I don't think the loop concern you exhibited exists when you do
treat-as-withdraw at an EBGP boundary, does it?

[Bruno]

[Bruno] Thank you the extra effort of proposing new resolutions.
That would work for me.
My 2 cents:

  *   Some people are not comfortable with "error correction". I understand that with OTC the situation is indeed simplified
  *   That's a priori not robust to extension to the OTC attribute (e.g. adding new fields). That been said the current error handling is even worst on this aspect.

--Bruno

-- Jeff

---

From: <[email-masked]>
Date: Wed, 26 Jan 2022 17:51:35 +0000

> Sent: Wednesday, January 26, 2022 6:21 PM
> 
> As one meta-point on this thread, we are at risk of overthinking: 

Agreed.

> the probability of
> this particular case being exercised seems quite low, since it involves a sender
> injecting a wrong-length OTC attribute; a fairly egregious bug.

I'm not disagreeing with your text, but I'm not assuming that that probability is negligible.
I'm sorry to say that we recently faced this exact error case with the Entropy Label BGP attribute, with an implementation sending a wrong length. (that was not a bug but seen as an extension, but the black box behavior is the same)

> So, it may be that
> any solution where the cure isn’t worse than the disease would probably be fine,
> even if it’s not theoretically optimal.

+1 

> Bruno was making a “cure is worse than the disease” case for the version 21 text,
> is how I see it. Several of the alternatives being discussed all seem fine, though.

+1

--Bruno

Orange Restricted

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 26 Jan 2022 14:06:32 -0500

Bruno,

> On Jan 26, 2022, at 12:42 PM, <[email-masked]> <[email-masked]> wrote:
> 
> Some people are not comfortable with “error correction”. I understand that with OTC the situation is indeed simplified
> That’s a priori not robust to extension to the OTC attribute (e.g. adding new fields). That been said the current error handling is even worst on this aspect.

I understand error correction may not be viable.  My suggestion is simply one option that helps preserve the intent of the feature in presence of malformed entries.

In the current form, the feature is not extensible. If the intent was to make it extensible, the text would need to be altered to state "at least four octets" and "the first four octets encode an AS number used as part of the receiving procedures" along with "further octets are left for extensibility".

I'm not recommending making this extensible in the current form. 

-- Jeff

---

From: "Sriram, Kotikalapudi (Fed)" <[email-masked]>
Date: Thu, 27 Jan 2022 04:33:41 +0000

Bruno,

Still not convincing that there is the possibility of a data loop. If you assume full mesh as is normal for iBGP within an AS, then data for a /32 address in the /24 prefix from R2 does not go via R1 to R0. It goes directly to R0 which propagated the /24 in iBGP. So, there can be no data looping between R2 and R1.  (There may be RR instead of a full mesh but that does change the analysis.)

Please see the attached pdf with figures to illustrate my point. Fig. 1 and Fig. 2 cover your two hypotheses: (1) OTC is good, and router (R1) is faulty; and (2) OTC is malformed, and routers (R0 and R2) believe the OTC attribute is correct.

Sriram

>From Bruno: Wed, 26 January 2022 17:14 UTC
Bruno,

Can you please explain how you envision R2 having the /24 in its forwarding table?
[Bruno] Both R0 and R2 are correct implementations. (or alternatively, they have the same implementation/bug and believe the OTC attribute is correct while it is not)
Thank you,
--Bruno
-------------------------------------------

---

From: <[email-masked]>
Date: Thu, 27 Jan 2022 09:50:39 +0000

Sriram,

Orange Restricted

Bruno,

Still not convincing that there is the possibility of a data loop. If you assume full mesh as is normal for iBGP within an AS,
[Bruno] I would definitely not say that iBGP full mesh is normal within an AS, quite the contrary. But we seem to agree that the IBGP architecture (iBGP full mesh or Route Reflector) has no impact, so let's just forget about that.

then data for a /32 address in the /24 prefix from R2 does not go via R1 to R0. It goes directly to R0 which propagated the /24 in iBGP. So, there can be no data looping between R2 and R1.  (There may be RR instead of a full mesh but that does change the analysis.)
[Bruno] The physical (links) topology of the AS that I'm discussing is: R3--R2--R1--R0. "-- "represents a physical link.  R2 is not physically connected to R0 so can't physically send the packet directly to R0. It needs to reach R0 via/through R1.

Please see the attached pdf with figures to illustrate my point. Fig. 1 and Fig. 2 cover your two hypotheses: (1) OTC is good, and router (R1) is faulty; and (2) OTC is malformed, and routers (R0 and R2) believe the OTC attribute is correct.
[Bruno] It's not clear to me what the blue lines between routers are representing.

  *   If it's IBGP sessions, I think we agree that this is not relevant so let's forget about IBGP sessions.
  *   If it's physical links, that's a different topology that the one I am discussing. Yes, if hypothetically all ABSR are physically full mesh by links, there is no router in transit within the AS and there is no issue with IBGP inconsistency within the AS. But that this not the topology I am discussing.
--Bruno
Sriram

Bruno,

---

From: "Sriram, Kotikalapudi (Fed)" <[email-masked]>
Date: Fri, 28 Jan 2022 00:00:35 +0000

Bruno,

Thanks. Please see the attached updated pdf. I have added Figs. 3 and 4 on slide 2 where the physical connections are shown corresponding to Figs. 1 and 2. R2 received the /24 route directly from R0 (slide 1). Just for my understanding... so, what informs R2 that the next hop it should set for the /24 is that for (towards) R1?

Sriram

==============================================================
From: [email-masked] <[email-masked]>
Sent: Thursday, January 27, 2022 4:51 AM

Sriram,

Bruno,

[Bruno] It's not clear to me what the blue lines between routers are representing.
-       If it's IBGP sessions, I think we agree that this is not relevant so let's forget about IBGP sessions.
-       If it's physical links, that's a different topology that the one I am discussing. Yes, if hypothetically all ABSR are physically full mesh by links, there is no router in transit within the AS and there is no issue with IBGP inconsistency within the AS. But that this not the topology I am discussing.

--Bruno

---

From: Ben Maddison <[email-masked]>
Date: Fri, 28 Jan 2022 10:28:41 +0200

Hi Bruno,

Sorry for the delayed response - yesterday went to hell!

In-line...

On 01/26, [email-masked] wrote:
> [Bruno] We are discussing the situation where the OTC attribute is
> unreadable and hence the OTC feature is lost.  Now we have a route
> with no (valid) OTC attribute. There are many such routes in the
> Internet today, and probably still in the short to medium term. Why
> should be withdraw/drop that route?  (Yes, the route could be many
> things (e.g. a leak, required for an emergency call or for lots of
> business...) I'm not seeing the value of making hypothesis given that
> we have zero information: we are dealing with a route with no OTC
> info)
> 
I don't agree with this. R1 certainly has no usable information, but we
know that there *was* an OTC attribute when R0 advertised it, which
*should* have indicated to R1 that the route should not be propagated to
non-customer peers. That is not the same as a situation where such
information simply does not exist.

The choice with respect to R1's behaviour then becomes:
- behave as if there really was no OTC attribute (attribute discard),
  and risk violating routing policy; or
- treat-as-withdraw and risk forming a loop.

Which one is more harmful IMO is highly dependent on the actual
situation.

> There is zero indication of a leak. Why do/would you assume one?
I don't assume one. I said "could", not "will".

> RFC 7606 covers this requirement for diagnosis and correction of the
> bug  https://datatracker.ietf.org/doc/html/rfc7606#section-6 in a way
> which seems relatively simple and typical to me.  On the other hand,
> detecting a forwarding loop seems more difficult to me (transient
> loops happen regularly, during many/all IGP convergence and many IBGP
> convergence if tunneling is not used, and I'm not sure that this is
> monitored and tracked by many ASes)
> 
In my experience, most operators lack tooling for *automatic* detection
and resolution of either leaks or loops (unless you count support
tickets as "automatic" :-)).

My point was simply that inter-AS co-operation is usually slower and
harder than troubleshooting one's own network.

Speaking personally, I would rather be woken up in the middle of the
night for the loop.

Speaking of 7606, in section 8:

    "For any malformed attribute that is handled by the "attribute
   discard" instead of the "treat-as-withdraw" approach, it is critical
   to consider the potential impact of doing so.  In particular, if the
   attribute in question has or may have an effect on route selection or
   installation, the presumption is that discarding it is unsafe unless
   careful analysis proves otherwise."

I would include in this guidance an attribute used in determining export
eligibility, such as OTC.

> Is your inclination up to calling for RFC 4271 "session reset" to make
> sure we are not accommodating them in the protocol?
Session reset would create a whole lot of unnecessary collateral damage,
so no.
I agree less bad is a good goal. Unfortunately in this case I think we
are discussing different types of bad that are not generally comparable.
> And in a AS with no tunneling, the same goal (transit across the AS
> with no routing inconsistency) would mean that the IP node in transit
> should behave as if it were doing "attribute discard" rather than
> "treat as withdraw". (IOW, the OTC attribute is best not read by IBGP
> nodes in transit). That seem to be in line with my proposal.)
> 
The same goal with respect to intra-AS routing consistency, but not with
respect to inter-AS routing policy.

I think we could get closer with something like "from iBGP peers use
attribute-discard *and* set no-export", but I can see several objections
to this:
- lots of additional complexity
- surprising behaviour for operators that haven't closely read the
  spec
- how can R1 even be sure that the offending attribute is an OTC!?

Cheers,

Ben

---

From: <[email-masked]>
Date: Fri, 28 Jan 2022 16:31:04 +0000

Sriram,

Thanks for the updated pdf.

  *   Just for my understanding... so, what informs R2 that the next hop it should set for the /24 is that for (towards) R1?

R2 learns:

  *   From iBGP: for /24, BGP NEXT_HOP (BGP attribute) is R0 's loopback or external interface address
  *   From IGP (e.g. IS-IS, OSPF), that in order to reach R0, the shortest path within the AS is via R1

"The immediate next-hop address is determined by performing a
   recursive route lookup operation for the IP address in the NEXT_HOP
   attribute, using the contents of the Routing Table, selecting one
   entry if multiple entries of equal cost exist.  The Routing Table
   entry that resolves the IP address in the NEXT_HOP attribute will
   always specify the outbound interface."

Cf https://datatracker.ietf.org/doc/html/rfc4271#section-5.1.3

Regards,
--Bruno

Orange Restricted
From: Sriram, Kotikalapudi (Fed) <[email-masked]>
Sent: Friday, January 28, 2022 1:01 AM
To: DECRAENE Bruno INNOV/NET <[email-masked]>
Cc: [email-masked]; [email-masked]; 'Alvaro Retana' <[email-masked]>; 'Jeffrey Haas' <[email-masked]>; Susan Hares <[email-masked]>; John Scudder <[email-masked]>
Subject: RE: [Idr] draft-ietf-idr-bgp-open-policy-20 - error handling

Bruno,

Sriram

==============================================================
From: [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Sent: Thursday, January 27, 2022 4:51 AM

Sriram,

Bruno,

--Bruno

---

From: <[email-masked]>
Date: Fri, 28 Jan 2022 17:01:30 +0000

Hi Ben,

Please see inline

Orange Restricted

> Sorry for the delayed response - yesterday went to hell!

[Bruno] Really no problem. Most likely I'm not one introducing the highest delay.

> information simply does not exist.

[Bruno] That's a very correct point. This seems to lead to two options:
1)- knowing the presence of the OTC is enough information. In which case, the error handling could only disregard the content of the attribute but behave according to the presence of the OTC attribute (as you said.) 
2) - the presence of the OTC is not enough. We have proof that the OTC attribute is malformed and hence should be discarded (the attribute or whole route).

As far as I'm concerned, I would be perfectly fine with option 1. (presence of the OTC attribute is what matters. Content is second order).
My understanding is that Jeff and John had also suggested the same option.

If we all agree with this, should we follow that path?

> I don't assume one. I said "could", not "will".

In next sentences, you followed up with " It's hard to say ahead of time whether the leak or the loop is more disruptive." which I understood as assuming that a leak was there. Definitely possibly my bad. Let's close that point.

> tickets as "automatic" :-)).

Agreed. But as  per RFC 7606, the router should raise some alarm to the noc. No need to detect/search for the issue.

Agreed.
 
> Speaking personally, I would rather be woken up in the middle of the

OK.
On my side, I'd rather keep sleeping unless there is an actual leak (for a significant route) 😉

> eligibility, such as OTC.

Yes, but for OTC, the big point IMHO is that the effect is not across in the AS and hence has no risk of introducing routing loops.
OTC is involved on EBGP route filtering, which is safe from a routing loops point of view.

> - how can R1 even be sure that the offending attribute is an OTC!?

Thanks for your active step to get closer.

Given previous emails, may be we could converge on:
- disregards OTC content, but act as per the presence of OTC (as proposed by Jeff, John and yourself)

This seems to:
- minimize the operation risks
- preserve leak detection
- same behavior for both iBGP and eBGP cases

And seems the OTC feature is preserved, if we don't call this an OTC syntax error, this could also be seen as not been a new error handling strategy (as not seem as a big enough error).

Thanks for the useful discussion. Hopefully, we are progressing.

--Bruno

---

From: Ben Maddison <[email-masked]>
Date: Sat, 29 Jan 2022 12:35:30 +0200

Hi Bruno,

[...]
> 1)- knowing the presence of the OTC is enough information. In
> which case, the error handling could only disregard the content of the
> attribute but behave according to the presence of the OTC attribute
> (as you said.)
> 2) - the presence of the OTC is not enough. We have
> proof that the OTC attribute is malformed and hence should be
> discarded (the attribute or whole route).
> 
> As far as I'm concerned, I would be perfectly fine with option 1.
> (presence of the OTC attribute is what matters. Content is second
> order).  My understanding is that Jeff and John had also suggested the
> same option.
> 
I agree that 1) is the most correct/least harmful at iBGP-in.

At eBGP-in, the only sensible assumption that the receiving router could
make about the (unreadable) contents of the OTC is that it contains
something semantically like AS0.
That, I think, would result in behaviour equivalent to
"treat-as-withdraw" for eBGP, inasmuch as we wouldn't risk keeping stale
paths?

If I'm right, then that avoids having the ugliness of different error
handling depending on whether the session is internal or external, which
is nice.

The only part of this approach that gives me pause is that the receiving
speaker has to trust that the OTC codepoint is correct, even in the face
of detecting an error elsewhere in the attribute.
That seems fragile to me. Is there precedent for this kind of handling?

Cheers,

Ben

---

From: "Sriram, Kotikalapudi (Fed)" <[email-masked]>
Date: Sat, 29 Jan 2022 21:52:39 +0000

Bruno,

> order).

Are you thinking here about iBGP or eBGP?

In iBGP, no processing of OTC other than a syntax check is performed.

In eBGP, if the route is received from a Peer, the content matters (see Section 4):

   2.  If a route with the OTC Attribute is received from a Peer and the
       Attribute has a value that is not equal to the remote (i.e.,
       Peer's) AS number, then it is a route leak and MUST be considered
       ineligible.

No comments inline.

Sriram

---

From: "Sriram, Kotikalapudi (Fed)" <[email-masked]>
Date: Sun, 30 Jan 2022 00:30:32 +0000

Bruno,
Ben,

Thanks much for the discussion/insights. I have read all your posts and appreciate it.

Data looping could happen even when there is no specific BGP Attribute involved. Consider RPKI-based route origin validation (ROV). Inconsistent implementations of ROV (with ‘reject Invalid’) within an AS can cause data looping. This is illustrated in Figs. A and B in slide 1 of the attached pdf. There is no ROV-specific path Attribute involved. So, there is no option of ‘Attribute discard’. The only solution that the operator has available is to somehow detect the data-looping and then debug and have the implementation errors fixed.

Further, the ROV implementations may be consistent (bug-free) across the whole AS, but even then data looping can happen due to inconsistent views of the RPKI data at different routers in the AS (see slide 1). Again the operator has to deal with that and make operational improvements.

I think it is because of the variety of ways faulty implementations could happen, Alvaro had suggested earlier (in off-list conversation): “I interpret the problem as multiple/buggy implementations.  In the document all we can do is to be as clear as possible to reduce the probability of misinterpretation -- but we can't anticipate all possible issues…  In any case, I'll leave you to it.  :-)”

I think that is complemented by what Ben observed:

Sriram

P.S.
Bruno: You had asked (off-list) if ROV is performed at each BGP router within an AS. Yes… please see RFC 8097.

---

From: Ben Maddison <[email-masked]>
Date: Mon, 31 Jan 2022 09:19:09 +0200

Hi Sriram,

On 01/30, Sriram, Kotikalapudi (Fed) wrote:
> Data looping could happen even when there is no specific BGP Attribute
> involved. Consider RPKI-based route origin validation (ROV).
> Inconsistent implementations of ROV (with ‘reject Invalid’) within an
> AS can cause data looping. This is illustrated in Figs. A and B in
> slide 1 of the attached pdf. There is no ROV-specific path Attribute
> involved. So, there is no option of ‘Attribute discard’. The only
> solution that the operator has available is to somehow detect the
> data-looping and then debug and have the implementation errors fixed.
> 
> Further, the ROV implementations may be consistent (bug-free) across
> the whole AS, but even then data looping can happen due to
> inconsistent views of the RPKI data at different routers in the AS
> (see slide 1). Again the operator has to deal with that and make
> operational improvements.
> 
That is all perfectly true.

In general, the possibility of inducing breakage of the sort you
describe is best mitigated by performing policy actions at the eBGP
border, rather than in iBGP.

Importantly, for the purposes of this conversation, the operator can
avoid the above issue through the careful design of routing policy.

In the case of error handling, conversely, the behaviour is mandated by
specification. Once in the hands of an operator, they can do little or
nothing to design around the decisions taken here.

> I think it is because of the variety of ways faulty implementations
> could happen, Alvaro had suggested earlier (in off-list conversation):
> “I interpret the problem as multiple/buggy implementations.  In the
> document all we can do is to be as clear as possible to reduce the
> probability of misinterpretation -- but we can't anticipate all
> possible issues…  In any case, I'll leave you to it.  :-)”
> 
As I indicated earlier in this thread, I'm disinclined toward trading
protocol effectiveness for mitigating the consequences of buggy
implementations.

In this case, however, I believe we have converged on a handling that
achieves some additional safety, with few downsides.

[...]

> within an AS. Yes… please see RFC 8097.
IMNSHO the world would be a better place if 8097 had never been written.

RFC 8481 says that the validation state should be evaluated for all
routes (internal, external, originated). It does *not* say that routing
policy should be applied based on that state at every attachment point.

In my view, in the general case, it should be applied at the eBGP border
only, for exactly the reasons you set out.

Cheers,

Ben

---

From: <[email-masked]>
Date: Mon, 31 Jan 2022 10:01:31 +0000

Sriram,

Orange Restricted
From: Sriram, Kotikalapudi (Fed) <[email-masked]>
Sent: Saturday, January 29, 2022 10:53 PM
To: DECRAENE Bruno INNOV/NET <[email-masked]>; Ben Maddison <[email-masked]>
Cc: Susan Hares <[email-masked]>; [email-masked]; [email-masked]
Subject: RE: [Idr] draft-ietf-idr-bgp-open-policy-20 - error handling

Bruno,

[Bruno] I'm primarily concerned with the routing consistency within the AS. So IBGP.

[Bruno] If this is the choice of the WG, so be it. That been said, I may have another comment on this section.

Thanks,
--Bruno

No comments inline.

Sriram

---

From: <[email-masked]>
Date: Mon, 31 Jan 2022 12:30:00 +0000

Hi Ben,

Please see inline [Bruno2]

Orange Restricted

> I agree that 1) is the most correct/least harmful at iBGP-in.

[Bruno2] excellent.
 
> "treat-as-withdraw" for eBGP, inasmuch as we wouldn't risk keeping stale

[Bruno2] I may be missing something, but what is the functional benefit of this AS number inside the OTC?
(can't be for just checking that your eBGP peer is capable of setting the same number in both the OPEN and OTC?)
 
> That seems fragile to me. Is there precedent for this kind of handling?

[Bruno2] To me, "attribute discard" equally assumes that the attribute codepoint is correct. (otherwise, you are incorrectly removing an attribute)

Cheers,
--Bruno

---

From: <[email-masked]>
Date: Mon, 31 Jan 2022 12:41:06 +0000

Hi Sriram, Ben,

> Sent: Monday, January 31, 2022 8:19 AM
> 
> Hi Sriram,
> 
> > involved. 

[Bruno] Sure.
But why adding a new one for little to no reason?

> > Consider RPKI-based route origin validation (ROV).

[Bruno] I have no knowledge of ROV, but FWIW (i.e. 0) a priori I would agree with Ben.

Cheers,
--Bruno

Orange Restricted