Subject: Re: [Idr] I-D Action: draft-wang-idr-bgp-error-enhance-00.txt

From: Enke Chen <[email-masked]>
Date: Wed, 10 Nov 2021 21:08:08 -0800

Hi, Folks:

I have several comments on the draft, and some of them are similar to
Robert's.  More specifically,

---
On 3.1: Prefix Length Error:

   According to [RFC7606], when a NLRI/UNLRI or MP_REACH_NLRI/
   MP_REACH_UNLRI with invalid length, eg, IPv4 Prefix length is more
   than 32, we must drop this Prefix and ignore the following Prefixes.
   We may keep the prefixes we have parsed correctly before.

This is not correct.  Regarding NLRI parsing, RFC 7606 does not relax the
error handling specified in RFC 4271:

   The NLRI field in the UPDATE message is checked for syntactic
   validity.  If the field is syntactically incorrect, then the Error
   Subcode MUST be set to Invalid Network Field.

In RFC 7606, we have clarified the "Syntactic Correctness of NLRI Fields"
in Sect. 5.3, and also emphasized the fundamental requirement or using
"treat-as-withdraw":

    o j. Finally, we observe that in order to use the approach of "treat-
       as-withdraw", the entire NLRI field and/or the MP_REACH_NLRI and
       MP_UNREACH_NLRI attributes need to be successfully parsed -- what
       this entails is discussed in more detail in Section 5.  If this
       is not possible, the procedures of [RFC4271] and/or [RFC4760]
       continue to apply, meaning that the "session reset" approach (or
       the "AFI/SAFI disable" approach) MUST be followed.

---
On 3.2 Appears More Than Once

    Again we want to make sure NLRIs are parsed correctly and fully.  It's
    difficult to know for sure when there are multiple MP_REACH_NLRI
    attributes or the MP_UNREACH_NLRI in one UPDATE message.

---
On 4.1 Nexthop

    As currently defined in RFC 4271, the procedure does not allow for graceful
    recovery from the error conditions, e.g,, after the duplicate
address is removed
    from an interface. There seems to be room for improvement, perhaps in a
    separate document.

---
On 4.2 MP_REACH_NLRI:

    Again, we must make sure the NLRIs are parsed correctly and fully. That
    is fundamental to the error handling specified in RFC 7606.

---
On 4.3 Prefix SID

    *If* there is an issue with the error handling in RFC 8669, then rfc8669bis
     would be more appropriate for making the correction, IMO.

---
On 4.4 Aggregator, AS4_Aggregator:

      Why does it matter?

---
On 4.5 ORIGINATOR_ID

     "Treat-as-withdraw" would be more disruptive, isn't it? That would contrary
      to the goal of the draft. I don't see a need for the change.

---
On 4.6 Cluster-list

     Did we ever define zero as an invalid CLUSER_ID? I do not recall
we did that,
     and I don't see a need for doing so now.
-----

Thanks.  -- Enke

On Sun, Oct 24, 2021 at 6:47 AM Robert Raszuk <[email-masked]> wrote:
>
> Dear Authors,
>
> Below are my comments after review of your proposal.
>
> 1) There is no such thing as "MP_REACH_UNLRI" ... Please swap all occurances to "MP_UNREACH_NLRI". Likewise please adjust all "UNLRI" or define a priori what you mean by this abbreviation.
>
> 2)  Section 3.1 -
>
> You say:
>
> "Then we may also try to continue parse the next update packet if we can correctly find it."
>
> I don't think this is a good suggestion for the Standards track document.
>
> 3)  Section 3.2 -
>
> I disagree with the suggestion. If a speaker is sending more than one MP_REACH_NLRI or MP_UNREACH_NLRI it should be cut out as soon as possible from causing more damage.
>
> 4) Section 4.1 -
>
> I am not sure why there is any need to enumerate the same specific special IP addresses and not to list others. For example if I receive IPv4 next hop as 127.0.0.1 is this cool ?
>
> I recommend editing this:  "f) The IP address is not a invalid ip address." I think you mean to say:  "f) The IP address is not a valid ip address."
>
> 5) Section 4.2 -
>
> I don't think this is a good suggestion.  Likewise in the case  of same prefix being present in both MP_REACH_NLRI and MP_UNREACH_NRLI suggestion to "firstly process the Prefixes in the MP_REACH_NLRI then process the Prefixes in the MP_REACH_UNLRI" is not good.
>
> 6) Section 4.3 -
>
> How can you treat it as withdraw something which is part of the "malformed prefix-SID attribute" ?
>
> 7) Sections 4.4 & 4.5 & 4.6 -
>
> Why are you suggesting again to check and detect special cases of *only* all zero addresses ? There are lot's of special IPv4 or even more IPv6 addresses to detect. I am not sure if we need to educate implementers about those in the BGP spec.
>
> Conclusion:
>
> If WG agrees that there are some valid suggestions in your proposal we should issue a RFC7606bis and not separate draft updating it like you are suggesting. So far to me like there is no substance to even go for -bis version of 7606.
>
> Yes, the BGP-4 protocol running on TCP is not bullet proof when it comes to handling bad implementations or malicious protocol attacks. Your figure 1 illustrates this well. But even with all suggestions listed there still remains lot's of attack vectors if someone has access to inject bad BGP packets to your network.
>
> I think we should consider using new transport which no longer runs on TCP and essentially not only treats all SAFIs as fully independent streams but also cut's interdependencies between all NLRI even with given SAFI.
>
> Good news is that some proposals in this direction are starting to pop-up in this WG, IMO already opening new doors for the new (hopefully much more robust) BGP version.
>
> Many thx,
> Robert
>
>
>
> On Sun, Oct 24, 2021 at 2:06 PM <[email-masked]> wrote:
>>
>>
>> A New Internet-Draft is available from the on-line Internet-Drafts directories.
>>
>>
>>         Title           : Revised Error Handling for BGP Messages
>>         Authors         : Haibo Wang
>>                           Ming Shen
>>                           Jie Dong
>>         Filename        : draft-wang-idr-bgp-error-enhance-00.txt
>>         Pages           : 8
>>         Date            : 2021-10-24
>>
>> Abstract:
>>    This document supplements and revises RFC7606.  According to RFC
>>    7606, when an UPDATE packet received from a neighbor contains an
>>    attribute of incorrect format, the BGP session cannot be reset
>>    directly.  Instead, the BGP session must be reset based on the
>>    specific problem.  Error packets must minimize the impact on routes
>>    and do not affect the correctness of the protocol.  Different error
>>    handling methods are used.  The error handling methods include
>>    discarding attributes, withdrawing routes, disabling the address
>>    family, and resetting sessions.
>>
>>    RFC 7606 specifies the error handling methods of some existing
>>    attributes and provides guidance for error handling of new
>>    attributes.
>>
>>    This document supplements the error handling methods for common
>>    attributes that are not specified in RFC7606, and provides
>>    suggestions for revising the error handling methods for some
>>    attributes.  The general principle remains unchanged: Maintain
>>    established BGP sessions and keep valid routes updated.  However,
>>    discard or delete incorrect attributes or packets to minimize the
>>    impact on the current session.
>>
>>
>>
>> The IETF datatracker status page for this draft is:
>> https://datatracker.ietf.org/doc/draft-wang-idr-bgp-error-enhance/
>>
>> There is also an htmlized version available at:
>> https://datatracker.ietf.org/doc/html/draft-wang-idr-bgp-error-enhance-00
>>
>>
>> Internet-Drafts are also available by anonymous FTP at:
>> ftp://ftp.ietf.org/internet-drafts/
>>
>>
>> _______________________________________________
>> I-D-Announce mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/i-d-announce
>> Internet-Draft directories: http://www.ietf.org/shadow.html
>> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_idr&d=DwICAg&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=OPLTTSu-451-QhDoSINhI2xYdwiMmfF5A2l8luvN11E&m=SI6tanfZVs9gnmPKxxAheSBJdycZtcVWA5R6ub0TPz0&s=Kx7piVPCBN7qQf0IR_Ue8GQC2a_BNjdKR_7ybyNWWBc&e=

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 12 Nov 2021 00:42:36 +0000

There is another consideration.
Black holes.
A router that incorrectly withdraws can be mitigated by redundant routes.
A router that incorrectly advertises cannot be mitigated against.
It will spread a corrupted route around the network and cause black holes.
Redundancy is ineffective against black holes.
This is especially disruptive in a Clos network, because of the dense connectivity.
A rogue route from just one router can disrupt the whole fabric.

For this reason, I say, if you cannot be sure of the prefix, then kill the session.
It is a fallacy to try to keep faulty equipment partially working at all costs.
If in doubt, kill it.

Specifically:
---
3.1.  Prefix Length Error

If the length is wrong, it may be a previous length that was wrong
and the bytes being interpreted as a length field are actually
something else. Bottom line, the whole NLRI field cannot be trusted.

---
3.2.  Appears More Than Once

How do you know which one is wrong?
I could accept treating all of them as withdraw assuming they
can be parsed correctly.

---
4.2.  MP_REACH_NLRI

If the nexthop length is wrong, I could accept to parse the prefixes
and if the MP_REACH_NLRI is otherwise not in error, threat all
the prefixes as withdraw. To ignore would risk advertising incorrect routes.

Regards,
Jakob.

-----Original Message-----
From: Idr <[email-masked]> On Behalf Of Enke Chen
Sent: Wednesday, November 10, 2021 9:08 PM
To: Robert Raszuk <[email-masked]>
Cc: idr@ietf. org <[email-masked]>
Subject: Re: [Idr] I-D Action: draft-wang-idr-bgp-error-enhance-00.txt

Hi, Folks:

---
On 3.1: Prefix Length Error:

---

---
On 4.1 Nexthop

---
On 4.2 MP_REACH_NLRI:

---
On 4.3 Prefix SID

---

      Why does it matter?

---
On 4.5 ORIGINATOR_ID

---
On 4.6 Cluster-list

Thanks.  -- Enke

_______________________________________________
Idr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/idr

---

From: <[email-masked]>
Date: Fri, 12 Nov 2021 10:07:19 +0000

Not speaking about this draft. Not disagreeing with Jakob, but providing another consideration.

Jakob's point is valid _assuming_ that the alternate path is not experiencing exactly the same situation (same perceived BGP error).

We had multiple cases where such assumptions is wrong:
- unrecognized transitive attributes: error are discovered on all downstream nodes
- interop issues including squatting on code points, including different draft/WG squatting on the same code point: the route is propagated while crossing a single vendor and hence all downstream reacts to the same (perceived) error and each downstream see the same error on all redundant paths
- plain bug on a implementation: all nodes of this implementation will see the error, while obviously all upstream nodes saw no error and propagated the route.

So not disagree with Jakob, I also believe that deciding to make the problem bigger (e.g. shutting down the whole BGP session for a single attribute/NLRI) is to be seriously considered. And in addition to plain error/bugs, this can also be used as a DOS attack.

On a side note, the implementer/implementation not been happy with a received route/attribute is quasi always assuming that the error is on the sender side and then he/it is right to take a decision. Well eventually, it's the receiver/you which/who is wrong. So in addition to been cautious (above point) I would also call for been humble.

So thanks again for RFC 7606.

Regards,
--Bruno

Orange Restricted

> From: Idr <[email-masked]> On Behalf Of Jakob Heitz (jheitz)
> Sent: Friday, November 12, 2021 1:43 AM
> To: idr@ietf. org <[email-masked]>
> Cc: Robert Raszuk <[email-masked]>
> > 1) There is no such thing as "MP_REACH_UNLRI" ... Please swap all occurances
> to "MP_UNREACH_NLRI". Likewise please adjust all "UNLRI" or define a priori what
> you mean by this abbreviation.
> >
> > 2)  Section 3.1 -
> >
> > You say:
> >
> > I disagree with the suggestion. If a speaker is sending more than one
> MP_REACH_NLRI or MP_UNREACH_NLRI it should be cut out as soon as
> possible from causing more damage.
> >
> > 4) Section 4.1 -
> >
> > I am not sure why there is any need to enumerate the same specific special IP
> addresses and not to list others. For example if I receive IPv4 next hop as
> 127.0.0.1 is this cool ?
> >
> > I recommend editing this:  "f) The IP address is not a invalid ip address." I think
> you mean to say:  "f) The IP address is not a valid ip address."
> >
> > 5) Section 4.2 -
> >
> > I don't think this is a good suggestion.  Likewise in the case  of same prefix
> being present in both MP_REACH_NLRI and MP_UNREACH_NRLI suggestion to
> "firstly process the Prefixes in the MP_REACH_NLRI then process the Prefixes in
> the MP_REACH_UNLRI" is not good.
> >
> > 6) Section 4.3 -
> >
> > Why are you suggesting again to check and detect special cases of *only* all
> zero addresses ? There are lot's of special IPv4 or even more IPv6 addresses to
> detect. I am not sure if we need to educate implementers about those in the BGP
> spec.
> >
> > Conclusion:
> >
> > If WG agrees that there are some valid suggestions in your proposal we should
> issue a RFC7606bis and not separate draft updating it like you are suggesting. So
> far to me like there is no substance to even go for -bis version of 7606.
> >
> > Yes, the BGP-4 protocol running on TCP is not bullet proof when it comes to
> handling bad implementations or malicious protocol attacks. Your figure 1
> illustrates this well. But even with all suggestions listed there still remains lot's of
> attack vectors if someone has access to inject bad BGP packets to your network.
> >
> > I think we should consider using new transport which no longer runs on TCP and
> essentially not only treats all SAFIs as fully independent streams but also cut's
> interdependencies between all NLRI even with given SAFI.
> >
> > Good news is that some proposals in this direction are starting to pop-up in this
> WG, IMO already opening new doors for the new (hopefully much more robust)
> BGP version.
> >
> > Many thx,
> > Robert
> >
> >
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_mailman_listinfo_idr&d=DwICAg&c=V9IgWpI5PvzTw83UyHGVS
> oW3Uc1MFWe5J8PTfkrzVSo&r=OPLTTSu-451-
> QhDoSINhI2xYdwiMmfF5A2l8luvN11E&m=SI6tanfZVs9gnmPKxxAheSBJdycZtcV
> WA5R6ub0TPz0&s=Kx7piVPCBN7qQf0IR_Ue8GQC2a_BNjdKR_7ybyNWWBc&e=
> 

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Fri, 12 Nov 2021 12:16:32 +0000

Could the authors articulate the higher level goals of this draft?  LLGR covers the goal of maintaining state when there are failures in the BGP control plane. Each operator can determine which services are candidates. My approach is to use this functionality for AFI/SAFIs that convey "state" that is static and internal to my network. Examples include Kompella and FlowSpec. VPNV4 is being considered as maintaining a customers VPN outweighs an incorrect specification that may occur while the control plane is down..

Could you describe the FOUs for this proposal. Is it all AFI/SAFIs? 

Thanks,
	Jim Uttaro

-----Original Message-----
From: Idr <[email-masked]> On Behalf Of [email-masked]
Sent: Friday, November 12, 2021 5:07 AM
To: idr@ietf. org <[email-masked]>
Cc: Jakob Heitz (jheitz) <jheitz=[email-masked]>; Robert Raszuk <[email-masked]>
Subject: Re: [Idr] I-D Action: draft-wang-idr-bgp-error-enhance-00.txt

Regards,
--Bruno

Orange Restricted

> >> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-wang-idr-bgp-error-enhance/__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJNIDCK3A$ 
> >>
> >> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draft-wang-idr-bgp-error-enhance-00__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJOGmWECE$ 
> >>
> >>
> >> https://urldefense.com/v3/__ftp://ftp.ietf.org/internet-drafts/__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJBaeBJzQ$ 
> >>
> >>
> >> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/i-d-announce__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJxaUUpHE$ 
> >> Internet-Draft directories: https://urldefense.com/v3/__http://www.ietf.org/shadow.html__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJjm-9obU$ 
> >> or https://urldefense.com/v3/__ftp://ftp.ietf.org/ietf/1shadow-sites.txt__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJ8-8NHvY$ 
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJT-k3FIg$ 
> 
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJT-k3FIg$ 

_______________________________________________
Idr mailing list
[email-masked]
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJT-k3FIg$

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Sat, 13 Nov 2021 01:04:01 +0000

On the DDOS thing, it's when the malformed update keeps coming each time
the session restarts. up, down, up, down.....
How about a shutdown-and-stay-down option?

Regards,
Jakob.

-----Original Message-----
From: UTTARO, JAMES <[email-masked]> 
Sent: Friday, November 12, 2021 4:17 AM
To: DECRAENE Bruno INNOV/NET <[email-masked]>; idr@ietf. org <[email-masked]>
Cc: Jakob Heitz (jheitz) <[email-masked]>; Robert Raszuk <[email-masked]>
Subject: RE: [Idr] I-D Action: draft-wang-idr-bgp-error-enhance-00.txt

Thanks,
	Jim Uttaro

-----Original Message-----
From: Idr <[email-masked]> On Behalf Of [email-masked]
Sent: Friday, November 12, 2021 5:07 AM
To: idr@ietf. org <[email-masked]>
Cc: Jakob Heitz (jheitz) <jheitz=[email-masked]>; Robert Raszuk <[email-masked]>
Subject: Re: [Idr] I-D Action: draft-wang-idr-bgp-error-enhance-00.txt

Regards,
--Bruno

Orange Restricted

---

From: <[email-masked]>
Date: Mon, 15 Nov 2021 09:02:16 +0000

> From: Jakob Heitz (jheitz) <[email-masked]>
> Sent: Saturday, November 13, 2021 2:04 AM
> 

If you are referring to my email, same comment: may work, or may not work so well. E.g.
- when the malformed BGP update is received from all redundant BGP sessions and the BGP sessions carry important prefixes you ends up removing all paths for many NLRIs. Possibly a larger DOS.
- when the DOS source has equal capability to send the error over the alternate/backup BGP sessions

Again, I'm not trying to say that nothing can be done. I'm trying to say that this may be more complex than saying "my peer is too buggy to be allowed to (have its BGP session) live". At minimum, there is a large chance (50% at first order) that the bug is on the receiver's/your side.

If you are referring to Jim's email, yes LLGR has an opportunity to shutdown-and-stay-down while keeping the routes before the error. It's also not free but at least it tries to mitigate the consequences of its decision to close the session (keep the route and de-preference them).

Regards,
--Bruno

Orange Restricted

> Could the authors articulate the higher level goals of this draft?  LLGR covers the
> goal of maintaining state when there are failures in the BGP control plane. Each
> operator can determine which services are candidates. My approach is to use this
> functionality for AFI/SAFIs that convey "state" that is static and internal to my
> network. Examples include Kompella and FlowSpec. VPNV4 is being considered
> as maintaining a customers VPN outweighs an incorrect specification that may
> occur while the control plane is down..
> 
> Jakob's point is valid _assuming_ that the alternate path is not experiencing
> exactly the same situation (same perceived BGP error).
> 
> - interop issues including squatting on code points, including different draft/WG
> squatting on the same code point: the route is propagated while crossing a single
> vendor and hence all downstream reacts to the same (perceived) error and each
> downstream see the same error on all redundant paths
> - plain bug on a implementation: all nodes of this implementation will see the error,
> while obviously all upstream nodes saw no error and propagated the route.
> 
> So not disagree with Jakob, I also believe that deciding to make the problem
> bigger (e.g. shutting down the whole BGP session for a single attribute/NLRI) is to
> be seriously considered. And in addition to plain error/bugs, this can also be used
> as a DOS attack.
> 
> On a side note, the implementer/implementation not been happy with a received
> route/attribute is quasi always assuming that the error is on the sender side and
> then he/it is right to take a decision. Well eventually, it's the receiver/you which/who
> is wrong. So in addition to been cautious (above point) I would also call for been
> humble.
> 
> > >> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-wang-idr-
> bgp-error-enhance/__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-
> Yyt0Dp5IPcAJNIDCK3A$
> > >>
> > >> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draft-wang-
> idr-bgp-error-enhance-00__;!!BhdT!xVSA3jr29MgQ-
> ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJOGmWECE$
> > >>
> > >>
> > >> https://urldefense.com/v3/__ftp://ftp.ietf.org/internet-
> drafts/__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-
> Yyt0Dp5IPcAJBaeBJzQ$
> > >>
> > >>
> > >> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/i-d-
> announce__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-
> Yyt0Dp5IPcAJxaUUpHE$
> > >> Internet-Draft directories:
> https://urldefense.com/v3/__http://www.ietf.org/shadow.html__;!!BhdT!xVSA3jr29M
> gQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJjm-9obU$
> > >> or https://urldefense.com/v3/__ftp://ftp.ietf.org/ietf/1shadow-
> sites.txt__;!!BhdT!xVSA3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-
> Yyt0Dp5IPcAJ8-8NHvY$
> > >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVS
> A3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJT-k3FIg$
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVS
> A3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJT-k3FIg$
> 
> ______________________________________________________________________
> ___________________________________________________
> 
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration,
> This message and its attachments may contain confidential or privileged
> information that may be protected by law;
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVS
> A3jr29MgQ-ZiztKwbCrN3KpbqpjfaPxEeAXb-Yyt0Dp5IPcAJT-k3FIg$

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 15 Nov 2021 10:17:36 +0100

All,

Question and two observations.

Q: What's better - swiss cheese reachability or no reachability ? There
seems to be a group of folks who think that partial reachability is better
then no reachability. Well as I mentioned earlier I am not one of them.

O1: The fundamental issue with the proposed text is the attempt to ignore
malformed NLRI in the MP_REACH/MP_UNREACH and still keep the TCP session
up. IMO that's not acceptable as we do not even know if we can parse
anything else in a given UPDATE msg or even subsequent UPDATE messages. BGP
sends incremental updates. Maybe we could force full refresh to check if
the issue repeats itself upon such an event and only then send NOTIFICATION
?

O2: The root cause here seems to be running BGP on a TCP session. Imagine
if each UPDATE message came on it's own independent session (TCP or UDP) we
would not have this issue at all. I think if we are to improve BGP
resilience we should be heading that way. And yes this would be a major
change to how BGP operates. It's almost at the BGP session layer comparable
to shift from circuit switching to packet switching in the transport.

Thx,
Robert.

On Mon, Nov 15, 2021 at 10:02 AM <[email-masked]> wrote:

> - when the malformed BGP update is received from all redundant BGP
> sessions and the BGP sessions carry important prefixes you ends up removing
> all paths for many NLRIs. Possibly a larger DOS.
> Again, I'm not trying to say that nothing can be done. I'm trying to say
> that this may be more complex than saying "my peer is too buggy to be
> allowed to (have its BGP session) live". At minimum, there is a large
> chance (50% at first order) that the bug is on the receiver's/your side.
>
> If you are referring to Jim's email, yes LLGR has an opportunity to
> shutdown-and-stay-down while keeping the routes before the error. It's also
> not free but at least it tries to mitigate the consequences of its decision
> to close the session (keep the route and de-preference them).
>
> Regards,
> --Bruno
>
> > Regards,
> > Jakob.
> >
>
> Orange Restricted
>
> > Cc: Jakob Heitz (jheitz) <jheitz=[email-masked]>; Robert
> Raszuk
> > <[email-masked]>
> > Not speaking about this draft. Not disagreeing with Jakob, but providing
> another
> > consideration.
> >
> > > to "MP_UNREACH_NLRI". Likewise please adjust all "UNLRI" or define a
> priori
> > what
> > > > "Then we may also try to continue parse the next update packet if we
> can
> > > correctly find it."
> > > >
> > > > I recommend editing this:  "f) The IP address is not a invalid ip
> address." I
> > think
> > > "firstly process the Prefixes in the MP_REACH_NLRI then process the
> Prefixes
> > in
> > > > How can you treat it as withdraw something which is part of the
> "malformed
> > > prefix-SID attribute" ?
> > > >
> > > > If WG agrees that there are some valid suggestions in your proposal
> we
> > should
> > > issue a RFC7606bis and not separate draft updating it like you are
> suggesting.
> > So
> > > > I think we should consider using new transport which no longer runs
> on TCP
> > and
> > > > Good news is that some proposals in this direction are starting to
> pop-up in
> > this
> > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
> recu ce
> > message par erreur, veuillez le signaler
> > Orange decline toute responsabilite si ce message a ete altere, deforme
> ou
> > falsifie. Merci.
> >
> > If you have received this email in error, please notify the sender and
> delete this
> > message and its attachments.
> > As emails may be altered, Orange is not liable for messages that have
> been
> > modified, changed or falsified.
> > Thank you.
> >
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
> recu ce message par erreur, veuillez le signaler

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Mon, 15 Nov 2021 12:32:44 +0000

Comments In-Line.

Thanks,
              Jim Uttaro

From: Robert Raszuk <[email-masked]>
Sent: Monday, November 15, 2021 4:18 AM
To: idr@ietf. org <[email-masked]>
Cc: Jakob Heitz (jheitz) <[email-masked]>; UTTARO, JAMES <[email-masked]>; Bruno Decraene <[email-masked]>
Subject: Re: [Idr] I-D Action: draft-wang-idr-bgp-error-enhance-00.txt

All,

Q: What's better - swiss cheese reachability or no reachability ? There seems to be a group of folks who think that partial reachability is better then no reachability. Well as I mentioned earlier I am not one of them.
[Jim U>] It is not about partial reachability but the possible change in reachability over some given time period. An AFI/SAFI that may be frequently exchanging state that changes the “correctness” of a given VPN/GRT i.e IPV4, IPV6 are not candidates for LLGR ( My Opinion ). Kompella VPLS VPN that relies on BGP to exchange VE-IDs ( Anchor Points ) for Data Plane learning or Flowspec are candidates for maintaining forwarding state in the event of a control plane failure as the reachability is relatively static thus the “correctness” is also static and controlled by the operator.

Some AFI/SAFIs are clearly not i.e IPV4, IPV6, and then there are those that an operator has to make a judgement call on what is most important to their customers i.e EVPN, VPNV4 etc.. EVPN introduces additional considerations.

My question to the authors is which FOUs does this draft apply to? Do they believe that different AFI/SAFIs should be treated differently? BGP is a general purpose routing protocol and any draft that proposes a blanket approach is not in sync with how BGP is used today.

O1: The fundamental issue with the proposed text is the attempt to ignore malformed NLRI in the MP_REACH/MP_UNREACH and still keep the TCP session up. IMO that's not acceptable as we do not even know if we can parse anything else in a given UPDATE msg or even subsequent UPDATE messages. BGP sends incremental updates. Maybe we could force full refresh to check if the issue repeats itself upon such an event and only then send NOTIFICATION ?

O2: The root cause here seems to be running BGP on a TCP session. Imagine if each UPDATE message came on it's own independent session (TCP or UDP) we would not have this issue at all. I think if we are to improve BGP resilience we should be heading that way. And yes this would be a major change to how BGP operates. It's almost at the BGP session layer comparable to shift from circuit switching to packet switching in the transport.

Thx,
Robert.

On Mon, Nov 15, 2021 at 10:02 AM <[email-masked]<mailto:[email-masked]>> wrote:
> From: Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>>

Regards,
--Bruno

Orange Restricted

> From: UTTARO, JAMES <[email-masked]<mailto:[email-masked]>>
> To: DECRAENE Bruno INNOV/NET <[email-masked]<mailto:[email-masked]>>; idr@ietf. org
> <[email-masked]<mailto:[email-masked]>>
> Cc: Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>>; Robert Raszuk <[email-masked]<mailto:[email-masked]>>
> From: Idr <[email-masked]<mailto:[email-masked]>> On Behalf Of [email-masked]<mailto:[email-masked]>
> To: idr@ietf. org <[email-masked]<mailto:[email-masked]>>
> Cc: Jakob Heitz (jheitz) <jheitz=[email-masked]<mailto:[email-masked]>>; Robert Raszuk
> <[email-masked]<mailto:[email-masked]>>
> > From: Idr <[email-masked]<mailto:[email-masked]>> On Behalf Of Jakob Heitz (jheitz)
> > To: idr@ietf. org <[email-masked]<mailto:[email-masked]>>
> > Cc: Robert Raszuk <[email-masked]<mailto:[email-masked]>>
> > From: Idr <[email-masked]<mailto:[email-masked]>> On Behalf Of Enke Chen
> > To: Robert Raszuk <[email-masked]<mailto:[email-masked]>>
> > Cc: idr@ietf. org <[email-masked]<mailto:[email-masked]>>
> > On Sun, Oct 24, 2021 at 6:47 AM Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:
> > >
> > > Dear Authors,
> > >
> > > On Sun, Oct 24, 2021 at 2:06 PM <[email-masked]<mailto:[email-masked]>> wrote:
> > >>
> > >>
> > >> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-wang-idr-<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-wang-idr->
> > >> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draft-wang-<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-wang->
> > >> https://urldefense.com/v3/__ftp://ftp.ietf.org/internet-<https://urldefense.com/v3/__ftp:/ftp.ietf.org/internet->
> > >> [email-masked]<mailto:[email-masked]>
> > >> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/i-d-<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/i-d->
> https://urldefense.com/v3/__http://www.ietf.org/shadow.html__;!!BhdT!xVSA3jr29M<https://urldefense.com/v3/__http:/www.ietf.org/shadow.html__;!!BhdT!xVSA3jr29M>
> > >> or https://urldefense.com/v3/__ftp://ftp.ietf.org/ietf/1shadow-<https://urldefense.com/v3/__ftp:/ftp.ietf.org/ietf/1shadow->
> > > [email-masked]<mailto:[email-masked]>
> > [email-masked]<mailto:[email-masked]>
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVS<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVS>
> > [email-masked]<mailto:[email-masked]>
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVS<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVS>
> [email-masked]<mailto:[email-masked]>
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVS<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!BhdT!xVS>

---

From: "Wanghaibo (Rainsword)" <[email-masked]>
Date: Mon, 15 Nov 2021 15:47:46 +0000

Hi, All,
Thanks for Robert, Enke,  Jakob, Jim Uttaro, Jakob, bruno, Jeff and Keyur. Thanks for your comments.
Because there are many emails and comments, here are some summary replies.

Our main purpose is to reduce the impact of error messages on the business, also to provide some clear descriptions of what was not clearly defined in the original RFCs.
At present, more discussion is about the NLRI’s parsing. Such as the scope of application raised by Jeff, it is true that the Internet has a wide range of applications, and this processing may not be appropriate. However, for other AFI/SAFIs, such as EVPN and MVPN, this processing can be considered. This also answers Jim's question. We focus on the MP_REACH/UNREACH attribute, which mainly refers to various service AFI/SAFIs and non-Internet services.
In addition, although the GR mode proposed by Jeff and Jim can be used to reduce the impact of a fault, GR is not deployed on all devices on the live network. This is also a problem that we have encountered during fault locating and operation on the live network. In addition, even if GR is deployed on a device, sessions are repeatedly reset due to such parsing errors, and GR may fail due to repeatedly parsed to errors.
The integrity of the NLRI mentioned by Enke can be divided into two parts. First, the length of the NLRI in the MP_REACH is incorrect. This is described as a parsing error in RFC7606 section 5.3. However, the specific proceduring is not described. It can be understood that the definition of RFC4760 is retained, that is, disable the NLRI’s AFI/SAFI or reset the session. It is also worth discussing whether it is appropriate to disable AFI/SAFI or reset session. Currently, some AFI/SAFIs, such as EVPN, contain multiple routing types. Whether an error of one of them should affect the entire AFI/SAFI. The other is a partial error. That is, some consecutive prefixes in the packet may have been correctly parsed and then some prefix errors are found. This part also needs to be discussed. However, we need to reach an agreement on the content of the first part, that is, the prefix length error, and how to handle the problem more elegantly.
Whether a packet is parsed out of bounds depends on the length of the MP_REACH attribute, the total length of the attribute, and the total length of the packet to ensure that the packet does not across the bounds. According to Keyur's comments, TCP is flow-based. If the current packet is discarded, whether the subsequent packet can be parsed depends on the arrival of the next packet. However, this should not affect the processing of the current packet.
However, we cannot ensure that sessions can always be maintained as much as possible. For example, if the packet header is incorrect or the total length is incorrect, such errors may still cause session interruption and peer dampening . According to Bruno’s comment, whether to close the session and keep it Down is considered? In RFC 4271, the processing of peer dampening is defined. We can also optimize the Damp Peer Oscillations behavior, which is not discussed here.
Finally, some mechanism optimizations, such as dynamic capability negotiation and multi-session, may be used to reduce the impact. However, we also need consider how to do better under the current BGP mechanism.

Regards,
Haibo

From: Idr [mailto:[email-masked]] On Behalf Of UTTARO, JAMES
Sent: Monday, November 15, 2021 8:33 PM
To: Robert Raszuk <[email-masked]>; idr@ietf. org <[email-masked]>
Cc: Bruno Decraene <[email-masked]>
Subject: Re: [Idr] I-D Action: draft-wang-idr-bgp-error-enhance-00.txt

Comments In-Line.

Thanks,
              Jim Uttaro

From: Robert Raszuk <[email-masked]<mailto:[email-masked]>>
Sent: Monday, November 15, 2021 4:18 AM
To: idr@ietf. org <[email-masked]<mailto:[email-masked]>>
Cc: Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>>; UTTARO, JAMES <[email-masked]<mailto:[email-masked]>>; Bruno Decraene <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Idr] I-D Action: draft-wang-idr-bgp-error-enhance-00.txt

All,

Thx,
Robert.

Regards,
--Bruno

Orange Restricted

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 15 Nov 2021 14:43:19 -0500

Haibo,

On Mon, Nov 15, 2021 at 03:47:46PM +0000, Wanghaibo (Rainsword) wrote:
> Our main purpose is to reduce the impact of error messages on the
> business, also to provide some clear descriptions of what was not clearly
> defined in the original RFCs.

Please make sure to highlight specifically which things you believe may not
be clearly defined in the RFCs.  If there is consensus about this, filing
Errata is an appropriate thing.

> At present, more discussion is about the NLRI’s parsing. Such as the scope
> of application raised by Jeff, it is true that the Internet has a wide
> range of applications, and this processing may not be appropriate.
> However, for other AFI/SAFIs, such as EVPN and MVPN, this processing can
> be considered. This also answers Jim's question. We focus on the
> MP_REACH/UNREACH attribute, which mainly refers to various service
> AFI/SAFIs and non-Internet services.

Please remember that even "simple" IPv6 Internet service is carried in
MP_REACH/UNREACH.

The characterization about the scope of the address family is really the
detail we will want to discuss here.  I tend to frame the problem in terms
of the "blast radius" - the damage caused by an explosion - that a protocol
fault may have.  While it is usually the case that mvpn and evpn are
"local", evpn type 5 routes may effectively pass through networks and
eventually interact with the Internet.  

Similarly, IPv4/IPv6 unicast may be used solely for private purposes.

> The integrity of the NLRI mentioned by Enke can be divided into two parts.
> First, the length of the NLRI in the MP_REACH is incorrect. This is
> described as a parsing error in RFC7606 section 5.3. However, the specific
> proceduring is not described.

The text in RFC 7606, section 5.3:

:   The NLRI field or Withdrawn Routes field SHALL be considered
:   "syntactically incorrect" if either of the following are true:

The procedure is understood from RFC 4271, section 6.3:

:   The NLRI field in the UPDATE message is checked for syntactic
:   validity.  If the field is syntactically incorrect, then the Error
:   Subcode MUST be set to Invalid Network Field.

>  It can be understood that the definition of
> RFC4760 is retained, that is, disable the NLRI’s AFI/SAFI or reset the
> session. It is also worth discussing whether it is appropriate to disable
> AFI/SAFI or reset session.

I am personally curious what implementations disable a given AFI/SAFI.

The challenge an implementation has if it decides NLRI is malformed is
whether it can find the next BGP message boundary safely.  This is part of
my microphone comments during the IDR presentation of your draft.

In the most optimistic case, a BGP Update contains a single NLRI.  In the
event that the Update is considered malformed because of the NLRI, it might
be possible to look at adjacent memory patterns to find the BGP Marker
field.  If so, an implementation may choose to "resync" the BGP messages and
continue.  I am unaware of whether any implementation does this.

(The BGP Marker is a somewhat vestigial protocol feature.  It was originally
there to cover a previous authentication feature that was not carried
forward in the protocol updates.)

> Currently, some AFI/SAFIs, such as EVPN,
> contain multiple routing types. Whether an error of one of them should
> affect the entire AFI/SAFI. 

This has been a conversational point before with our Area Director, Alvaro.
The typed NLRI has created an unfortunate additional place of fragility in
error handling.  This is just another example.  

> Whether a packet is parsed out of bounds depends on the length of the
> MP_REACH attribute, the total length of the attribute, and the total
> length of the packet to ensure that the packet does not across the bounds.
> According to Keyur's comments, TCP is flow-based. If the current packet is
> discarded, whether the subsequent packet can be parsed depends on the
> arrival of the next packet. However, this should not affect the processing
> of the current packet.

This again indirectly applies toward using the Marker to find the "next
packet".  The scenario I believe you're trying to highlight is if it is the
case that an implementation believes that all damage to an Update is
contained in a single BGP message that it may be one step "safer" to leave
the session up.

While this may be true, the real question is the blast radius of the damage
of the Update that contains malformed state.

RFC 7606 decided to consider malformed NLRI as toxic.  As we discussed
during the IDR session at this recent IETF, it will take significant
motivation to convince the Working Group to change this consensus.

> Finally, some mechanism optimizations, such as dynamic capability
> negotiation and multi-session, may be used to reduce the impact. However,
> we also need consider how to do better under the current BGP mechanism.

I urge you to note that multi-session and BGP QUIC address similar
considerations but by different mechanisms.  multi-session reduces the
"blast radius" to a separate session.  I commonly refer to this property as
"structural separation".

For BGP QUIC, the protocol supplies a missing OSI Layer 5 that has the
opportunity to change the per-message fault model.

-- Jeff