Subject: Re: [Idr] [spring] Error Handling for BGP-LS with Segment Routing

From: Alvaro Retana <[email-masked]>
Date: Thu, 3 Jan 2019 13:40:13 -0800

Rob:

Hi!

I don’t think I said it before:  speaking as a WG participant...

I want to pick up on a point you make below, which I agree with: "any
topology discovery mechanism (whether used in real-time or not) needs to
define how it handles cases where it might end up with missing
information”….

BGP-LS only defines a mechanism through which it may miss information, but
not how to handle it — or maybe it does (?): by using attribute discard it
just accepts that the information might be missing going forward…and
doesn’t attempt to do anything.  Maybe this quote is true: "Doing Nothing
Often Leads to the Very Best Something” — Winnie the Pooh

That action may be ok in the general case…but I think that doing nothing
may not be enough/appropriate for an application like SR, because it is
explicitly calculating paths….

The point I’m trying to bring up is not necessarily treat-as-withdraw vs.
attribute discard…. But, first, is attribute discard
enough/appropriate/good for a BGP-LS application such as SR?  If it isn’t,
second, is there a different approach that would be better?  Maybe we then
come to a point where something can change…or accept the limitations of the
system and be clear about them.  I fully realize that I may be the only one
who thinks there’s an issue…

Thanks!!

Alvaro.

On December 21, 2018 at 11:23:16 AM, Rob Shakir ([email-masked]) wrote:

Alvaro,

I think this is one of the difficulties of overloading a protocol like BGP
with different datasets -- it's not simple to say how particular attributes
are actually going to be used within a protocol deployment. This was one of
the things that was noted in 7606 -- i.e., I can make *any* attribute
really affect forwarding if I write a policy that accepts/rejects some
UPDATE based on the presence of that attribute.

In general, any topology discovery mechanism (whether used in real-time or
not) needs to define how it handles cases where it might end up with
missing information. Let's consider what the different mechanisms for
discovery we have are today:

   - IGP listening -- in this case, if we have some malformed IS-IS TLV,
   then we might end up discarding this information (whether it be at the
   listening node, or a device that didn't flood it earlier in the chain) --
   meaning that we know that we have some potential gap in the topology.
   - Streaming telemetry -- speaking particularly to gNMI for LSDB
   streaming encoded using the OpenConfig model, here, we are tolerant to
   getting as much information as can be parsed, and have a way to carry
   unknown TLVs (which might include those that cannot be successfully parsed)
   as binary data to the external consumer. This means that the approach is
   "as complete data as possible", but has the same characteristic that we can
   also end up having the potential to lose data.
   - BGP-LS with attribute discard -- this has some information loss, since
   we'll have some attributes that could be malformed in the input data, and
   we discard them at the receiver.

It doesn't seem to me that, given the source of the data is the IGP, and we
might have information discarded there -- that we can really guarantee
strong consistency of an off-box view of the network, since we can't
guarantee strong consistency across the IGP domain itself.

Thus, I'm not sure that the issue that is being highlighted here actually
makes a difference when we're considering the overall system design -- we
always need to deal with the fact that the view of the network at the path
computing node might not match exactly the network's current state in the
presence of malformed protocol messages. One motivation for having the LSDB
via streaming telemetry is the ability to provide such validation ("do all
nodes within my IGP domain, including listeners, have a consistent view of
the state of the network?").

If the discussion is "should we adopt treat-as-withdraw vs. attribute
discard?" -- I don't think that from the system perspective there is really
any difference between the two in this situation. We still have the same
potentially inconsistent view of the network.

For these reasons, I'd err on leaving this unchanged in the current
specification(s).

Cheers,
r.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 3 Jan 2019 23:22:41 +0100

Hi Alvaro,

> BGP-LS only defines a mechanism through which it may miss information,
but not how to handle it

I think the point at least some of us are trying to indicate is that the
overall application is responsible for building into it proper redundancy.

BGP-LS reg error handling is not worse then BGP IPv4 or IPv6 routing - do
you agree ? If so unreachability should be handled end to end say by dual
homing, falling back from SR paths to "native IP path" etc ...

In my view SR controller is mainly used as optimization not as critical
element - well at least in the deployment models I would personally
recommend to use.

Regards,
R.

On Thu, Jan 3, 2019 at 10:40 PM Alvaro Retana <[email-masked]>
wrote:

> doesn’t attempt to do anything..  Maybe this quote is true: "Doing Nothing
>    - BGP-LS with attribute discard -- this has some information loss,
>    since we'll have some attributes that could be malformed in the input data,
>    and we discard them at the receiver.
>
> It doesn't seem to me that, given the source of the data is the IGP, and
> we might have information discarded there -- that we can really guarantee
> _______________________________________________
> spring mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spring
>

---

From: Rob Shakir <[email-masked]>
Date: Thu, 3 Jan 2019 14:40:10 -0800

Hi Alvaro,

Also speaking as a WG participant :-)

> Often Leads to the Very Best Something” — Winnie the Pooh

I think that it defines *something*, albeit not explicitly. Essentially, as
I read it, we're saying "when an attribute encoded by the advertising
BGP-LS source is incorrect, then BGP-LS as a system will prefer to use
partial information" (partial information, since we assume that some
information does get through, since the NLRI could be parsed).

> who thinks there’s an issue…

My point was really the same... The question I was trying to raise is "what
is the alternative that you would suggest?". Other technologies that
fulfill the same role as BGP-LS (those that I described) don't take a very
different approach.

Clearly, it's bad to calculate paths with incomplete information about the
topology of the network. It's also bad to calculate zero paths because you
discarded the entire topology based on an error. We're in-between a rock
and a hard place in terms of maintaining system functionality here -- all
systems that do the same as BGP-LS are having to make some form of
compromise about which constraint (correctness, or connectivity) they are
violating.

This is why I was arguing for leaving things unchanged -- the correctness
constraint seems OK to violate by default. If there are deployments where
connectivity is the desirable constraint to violate, then reacting to the
fact that attribute-discard did occur is possible (or not configuring 7606
error handling if the implementation supports this).

Describing these compromises is, of course, a good idea. However, it's not
clear where this description would go -- we don't really have a document
that describes this overall system and how it might be implemented today.

Cheers and HNY!
r.

---

From: Alvaro Retana <[email-masked]>
Date: Fri, 4 Jan 2019 08:48:20 -0800

Very good point! :-)

On January 3, 2019 at 5:22:52 PM, Robert Raszuk ([email-masked]) wrote:

---

From: Alvaro Retana <[email-masked]>
Date: Fri, 4 Jan 2019 11:39:57 -0800

On January 3, 2019 at 5:40:23 PM, Rob Shakir ([email-masked]) wrote:

Describing these compromises is, of course, a good idea. However, it's not
clear where this description would go -- we don't really have a document
that describes this overall system and how it might be implemented today
<http://airmail.calendar/2019-01-04 12:00:00 EST>.

Right…

I started reviewing the documents with BGP-LS extensions for SR…starting
with draft-ietf-idr-bgp-ls-segment-routing-ext, which is the first BGP-LS
extensions document to be sent for Publication where the application is
explicitly to "construct the end-to-end path (with its associated SIDs)
that need to be applied to an incoming packet to achieve the desired
end-to-end forwarding”.  All other BGP-LS extension documents have in
general followed the “informative” tone of rfc7752.

I don’t necessarily think that the description of the system belongs
there…but there’s no other place to put it, at least not currently.  The SR
Problem Statement (rfc7855) and the SR Architecture (rfc8402) both just
make general statements about the need to support centralized and hybrid
(and distributed, of course) control planes — they don’t go into more
specifics…

…

Alvaro.

---

From: Rob Shakir <[email-masked]>
Date: Fri, 4 Jan 2019 12:48:25 -0800

The alternative approach is to have an "Operational Considerations" section
of draft-ietf-idr-bgp-ls-segment-routing-ext that simply points out this
consideration. i.e., something like:

"Since the error handling approach defined in RFC7752 specifies 'attribute
discard' as the error handling mechanism for BGP-LS, systems implemented
using BGP-LS for discovery of topological attributes used for path
calculation MUST consider their mode of operation based on incomplete data
being received (due to attribute discard). If an assumption of strong
consistency between the BGP-LS receiver, and the network's topology is
made, system designers and operators SHOULD consider means to detect
erroneous attributes being discarded on a session and act accordingly."

Taking this approach doesn't say "hey, let's change this", and also doesn't
say "here's what the system should do", it just makes sure designers and
operators are aware of the consideration. That said, this is rather
implementation specific.

r.

On Fri, Jan 4, 2019 at 11:39 AM Alvaro Retana <[email-masked]>
wrote:

> <http://airmail.calendar/2019-01-04%2012:00:00%20EST>.
>
>
> Right…
>

---

From: Alvaro Retana <[email-masked]>
Date: Fri, 4 Jan 2019 14:05:47 -0800

On January 4, 2019 at 3:48:38 PM, Rob Shakir ([email-masked]) wrote:

We would need something like it in every bgp-ls-sr draft…or a pointer back
to draft-ietf-idr-bgp-ls-segment-routing-ext...

[This is probably not the right place to wordsmith, but I don’t know how
“MUST/SHOULD consider” can be normatively enforced.]

Because it is implementation/deployment specific, some examples of the
cases where there could be more (or less) potential issues would be ideal.
As Robert’s example: "In my view SR controller is mainly used as
optimization not as critical element - well at least in the deployment
models I would personally recommend to use.”

This last part could end up making the document significantly longer…and it
doesn’t really give me a warm fuzzy feeling adding it after WGLC…to an idr
document...and without it being properly reviewed by the spring WG.  But
these are details we can take care of as we move forward [*].

Thanks!

Alvaro.

[*] I haven’t finished my AD Review
of draft-ietf-idr-bgp-ls-segment-routing-ext.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Mon, 7 Jan 2019 18:08:05 +0000

A requirements doc would be a place to describe the behavior of the “system”.

Jim Uttaro

From: Idr <[email-masked]> On Behalf Of Alvaro Retana
Sent: Friday, January 04, 2019 2:40 PM
To: Rob Shakir <[email-masked]>
Cc: idr@ietf. org <[email-masked]>; SPRING WG <[email-masked]>; Robert Raszuk <[email-masked]>
Subject: Re: [Idr] [spring] Error Handling for BGP-LS with Segment Routing

On January 3, 2019 at 5:40:23 PM, Rob Shakir ([email-masked]<mailto:[email-masked]>) wrote:

Describing these compromises is, of course, a good idea. However, it's not clear where this description would go -- we don't really have a document that describes this overall system and how it might be implemented today<https://urldefense.proofpoint.com/v2/url?u=http-3A__airmail.calendar_2019-2D01-2D04-252012-3A00-3A00-2520EST&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=s7ZzB4JbPv3nYuoSx5Gy8Q&m=YPkMlAzgEmV6UHI4lGHe5dQ-N93JfeFuHAnSN6GcHlA&s=rOkYxM7fv1EaibU0ZmaZmT9J4zGKG81KNpHItRnP5II&e=>.

Right…

I started reviewing the documents with BGP-LS extensions for SR…starting with draft-ietf-idr-bgp-ls-segment-routing-ext, which is the first BGP-LS extensions document to be sent for Publication where the application is explicitly to "construct the end-to-end path (with its associated SIDs) that need to be applied to an incoming packet to achieve the desired end-to-end forwarding”.  All other BGP-LS extension documents have in general followed the “informative” tone of rfc7752.

I don’t necessarily think that the description of the system belongs there…but there’s no other place to put it, at least not currently.  The SR Problem Statement (rfc7855) and the SR Architecture (rfc8402) both just make general statements about the need to support centralized and hybrid (and distributed, of course) control planes — they don’t go into more specifics…

…

Alvaro.

---

From: <[email-masked]>
Date: Tue, 8 Jan 2019 14:33:48 +0000

Hi Alvaro, all

Also speaking as a WG participant

For what it’s worth, +1 to Rob’s email.

> From: spring [mailto:[email-masked]] On Behalf Of Alvaro Retana
> Sent: Thursday, January 03, 2019 10:40 PM
[…]
> I fully realize that I may be the only one who thinks there’s an issue…

I don’t think so. But at least the specification does define the behavior in this error condition. And I don’t think that there is a perfect solution: RFC 7606 behavior seems good enough to me and IMHO, going further would require discussing alternative technical proposals (solutions) rather than just raising the problem.

But I’m wondering why error handling is that specific to BGP-LS. Why is that point not been raised on, let’s say, draft-ietf-ospf-ospfv3-segment-routing-extensions which is currently under IESG review? I can see that the specific are (a bit) different, but the big picture seems the same: the information is incomplete, how do we handle this?
Then, I’m not sure that the problem is specific/limited to SR/SID information.

Thanks,
Cheers,
--Bruno

From: spring [mailto:[email-masked]] On Behalf Of Rob Shakir
Sent: Thursday, January 03, 2019 11:40 PM
To: Alvaro Retana
Cc: idr@ietf. org; SPRING WG; Robert Raszuk
Subject: Re: [spring] Error Handling for BGP-LS with Segment Routing

Hi Alvaro,

Also speaking as a WG participant :-)
On Thu, Jan 3, 2019 at 1:40 PM Alvaro Retana <[email-masked]<mailto:[email-masked]>> wrote:
BGP-LS only defines a mechanism through which it may miss information, but not how to handle it — or maybe it does (?): by using attribute discard it just accepts that the information might be missing going forward…and doesn’t attempt to do anything.  Maybe this quote is true: "Doing Nothing Often Leads to the Very Best Something” — Winnie the Pooh

I think that it defines *something*, albeit not explicitly. Essentially, as I read it, we're saying "when an attribute encoded by the advertising BGP-LS source is incorrect, then BGP-LS as a system will prefer to use partial information" (partial information, since we assume that some information does get through, since the NLRI could be parsed).

That action may be ok in the general case…but I think that doing nothing may not be enough/appropriate for an application like SR, because it is explicitly calculating paths….

The point I’m trying to bring up is not necessarily treat-as-withdraw vs. attribute discard…. But, first, is attribute discard enough/appropriate/good for a BGP-LS application such as SR?  If it isn’t, second, is there a different approach that would be better?  Maybe we then come to a point where something can change…or accept the limitations of the system and be clear about them.  I fully realize that I may be the only one who thinks there’s an issue…

My point was really the same... The question I was trying to raise is "what is the alternative that you would suggest?". Other technologies that fulfill the same role as BGP-LS (those that I described) don't take a very different approach.

Clearly, it's bad to calculate paths with incomplete information about the topology of the network. It's also bad to calculate zero paths because you discarded the entire topology based on an error. We're in-between a rock and a hard place in terms of maintaining system functionality here -- all systems that do the same as BGP-LS are having to make some form of compromise about which constraint (correctness, or connectivity) they are violating.

This is why I was arguing for leaving things unchanged -- the correctness constraint seems OK to violate by default. If there are deployments where connectivity is the desirable constraint to violate, then reacting to the fact that attribute-discard did occur is possible (or not configuring 7606 error handling if the implementation supports this).

Describing these compromises is, of course, a good idea. However, it's not clear where this description would go -- we don't really have a document that describes this overall system and how it might be implemented today.

Cheers and HNY!
r.

Thanks!!

Alvaro.

On December 21, 2018 at 11:23:16 AM, Rob Shakir ([email-masked]<mailto:[email-masked]>) wrote:
Alvaro,

I think this is one of the difficulties of overloading a protocol like BGP with different datasets -- it's not simple to say how particular attributes are actually going to be used within a protocol deployment. This was one of the things that was noted in 7606 -- i.e., I can make *any* attribute really affect forwarding if I write a policy that accepts/rejects some UPDATE based on the presence of that attribute.

In general, any topology discovery mechanism (whether used in real-time or not) needs to define how it handles cases where it might end up with missing information. Let's consider what the different mechanisms for discovery we have are today:

  *   IGP listening -- in this case, if we have some malformed IS-IS TLV, then we might end up discarding this information (whether it be at the listening node, or a device that didn't flood it earlier in the chain) -- meaning that we know that we have some potential gap in the topology.
  *   Streaming telemetry -- speaking particularly to gNMI for LSDB streaming encoded using the OpenConfig model, here, we are tolerant to getting as much information as can be parsed, and have a way to carry unknown TLVs (which might include those that cannot be successfully parsed) as binary data to the external consumer. This means that the approach is "as complete data as possible", but has the same characteristic that we can also end up having the potential to lose data.
  *   BGP-LS with attribute discard -- this has some information loss, since we'll have some attributes that could be malformed in the input data, and we discard them at the receiver.
It doesn't seem to me that, given the source of the data is the IGP, and we might have information discarded there -- that we can really guarantee strong consistency of an off-box view of the network, since we can't guarantee strong consistency across the IGP domain itself.

Thus, I'm not sure that the issue that is being highlighted here actually makes a difference when we're considering the overall system design -- we always need to deal with the fact that the view of the network at the path computing node might not match exactly the network's current state in the presence of malformed protocol messages. One motivation for having the LSDB via streaming telemetry is the ability to provide such validation ("do all nodes within my IGP domain, including listeners, have a consistent view of the state of the network?").

If the discussion is "should we adopt treat-as-withdraw vs. attribute discard?" -- I don't think that from the system perspective there is really any difference between the two in this situation. We still have the same potentially inconsistent view of the network.

For these reasons, I'd err on leaving this unchanged in the current specification(s).

Cheers,
r.

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Alvaro Retana <[email-masked]>
Date: Tue, 8 Jan 2019 13:15:16 -0800

On January 8, 2019 at 9:33:49 AM, [email-masked] (
[email-masked]) wrote:

Bruno:

Hi!

...

But I’m wondering why error handling is that specific to BGP-LS.

It is just in this thread...

Why is that point not been raised on, let’s say,
draft-ietf-ospf-ospfv3-segment-routing-extensions which is currently under
IESG review? I can see that the specific are (a bit) different, but the big
picture seems the same: the information is incomplete, how do we handle
this?

[AD Note:  draft-ietf-ospf-ospfv3-segment-routing-extensions is pending a
new revision to clear an outstanding DISCUSS.  If you (or anyone) has
specific issues about this document, please let me know as I’m ready to
approve the publication.]

I didn’t specifically raise the question of error handling for BGP-LS with
SR against the current document I’m reviewing
(draft-ietf-idr-bgp-ls-segment-routing-ext) because I don’t think the
solution explicitly belongs there…so I started the discussion on the
idr/spring lists.  If there are issues with the OSPFv3 transport then the
solution probably doesn’t belong in the extensions draft.

[This part of the discussion belongs on a different list…] I think that the
link state protocols have different characteristics (than BGP-LS) when it
comes to try to ensure common information between the nodes: reliability at
the LSA/LSP level, db synchronization, etc..  That difference brought me to
not bring up the question of incomplete information when I was reviewing
the SR extension drafts.  I may of course be missing an important piece.
If this point needs to be discussed, we should move to lsr.

Then, I’m not sure that the problem is specific/limited to SR/SID
information.

Right.

I think it’s easier to discuss specific problems first and then
generalize…than start with the general problem.  That is one of the reasons
for this thread.  In the specific case of BGP-LS, rfc7752 makes assumptions
about  about the nature of the information, which I think are broken with
SR…. I think there’s an SR-specific issue that needs to be addressed
somewhere.

As you mentioned before, other applications, like lsvr, could have the same
issues.

Thanks!

Alvaro.