Subject: [Idr] [[email-masked]: I-D Action: draft-haas-flowspec-capability-bits-02.txt]

From: Jeffrey Haas <[email-masked]>
Date: Fri, 9 Apr 2021 16:10:47 -0400

This version attempts to address the review comments received to date.

-- Jeff

----- Forwarded message from [email-masked] -----

Date: Fri, 09 Apr 2021 12:45:48 -0700
From: [email-masked]
To: [email-masked]
Subject: I-D Action: draft-haas-flowspec-capability-bits-02.txt

A New Internet-Draft is available from the on-line Internet-Drafts directories.

        Title           : BGP Flowspec Capability Bits
        Author          : Jeffrey Haas
	Filename        : draft-haas-flowspec-capability-bits-02.txt
	Pages           : 8
	Date            : 2021-04-09

Abstract:
   BGP Flowspec (RFC 8955) provides the ability to filter traffic using
   various matching components.  The NLRI format currently defined does
   not permit incremental deployment of new BGP Flowspec components.
   This draft defines a new BGP Capability to permit incremental
   deployment of such new Flowspec component types.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-haas-flowspec-capability-bits/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-haas-flowspec-capability-bits-02
https://datatracker.ietf.org/doc/html/draft-haas-flowspec-capability-bits-02

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-haas-flowspec-capability-bits-02

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

----- End forwarded message -----

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 9 Apr 2021 20:59:07 +0000

Works for me.

Regards,
Jakob.

-----Original Message-----
From: Idr <[email-masked]> On Behalf Of Jeffrey Haas
Sent: Friday, April 9, 2021 1:11 PM
To: [email-masked]
Subject: [Idr] [[email-masked]: I-D Action: draft-haas-flowspec-capability-bits-02.txt]

-- Jeff

----- Forwarded message from [email-masked] -----

Abstract:

----- End forwarded message -----

_______________________________________________
Idr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/idr

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Mon, 12 Apr 2021 17:04:32 +0000

Hi Jeff, all, 

I've read the recent discussion and the updated version of this draft, here are some thoughts and comments: 

I fully agree that incremental deployment of new flowspec components is important, and it does not need to wait for Flowspec 2.0.

Regarding a BGP flowspec component, there could be three different capabilities: 

1.	The capability of parsing and implementing the component as a filter

2.	The capability of parsing and propagating the component, but not for local use

3.	The capability of propagating the component further even it is not parsed

The first two capabilities are discussed in this thread and the updated draft, and to me it makes sense to distinguish these two capabilities. 

As sometimes a flowspec rule only needs to be installed on a subset of nodes, a node which is not required to install it or even support it may be on the propagation path of the flowspec rule. If there is some mechanism to limit the propagation scope or indicate the target nodes of the flowspec, the third capability may further help the incremental deployment of new flowspec components.

Then back to the encoding of the flowspec capability bits, if we want to consider the case that a node can propagate unknown flowspec components, one reserved bit in the bit-string may be used to indicate such capability, other mechanism is also possible. 

Thoughts?

Best regards,
Jie

> From: Idr [mailto:[email-masked]] On Behalf Of Jeffrey Haas
> Sent: Saturday, April 10, 2021 4:11 AM
> To: [email-masked]
> Subject: [Idr] [[email-masked]: I-D Action:
> draft-haas-flowspec-capability-bits-02.txt]
> 
> Internet-Draft directories: http://www.ietf.org/shadow.html or
> ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 12 Apr 2021 13:51:21 -0400

Jie,

Thanks for your feedback.

> The first two capabilities are discussed in this thread and the updated draft, and to me it makes sense to distinguish these two capabilities. 

Operationally, how would you expect a BGP Speaker to differentiate these two cases?  

> As sometimes a flowspec rule only needs to be installed on a subset of nodes, a node which is not required to install it or even support it may be on the propagation path of the flowspec rule. If t…

The third case is unfortunately problematic right now.  Deployment
experience has shown implementations aren't willing to deal with fully
"opaque" NLRI.  We've had a number of outages resulting from parsing bugs as
it is.

With flowspec v2, the primary obstacle toward opaque but still able to be
checked for NLRI high level syntactic correctness can be done.  We know this
is still problematic once you hit a node where the inner contents are
validated and found to be problematic, but at least in this case "treat as
withdraw" procedures are capable of being done.

The underlying behavior you're discussing is the scope discussion that I
think we're in general need of, and a separate thread was forked for.
Perhaps once that's been discussed in more detail it makes sense to come
back to this third point?

> Then back to the encoding of the flowspec capability bits, if we want to consider the case that a node can propagate unknown flowspec components, one reserved bit in the bit-string may be used to i…

The encoding is the less exciting bit of the discussion. :-)  Let's revisit
this after you've clarified your first two points.

-- Jeff

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Wed, 14 Apr 2021 15:21:54 +0000

Hi Jeff, 

Thanks for your reply, and please see some further discussion inline:

> From: Jeffrey Haas [mailto:[email-masked]]
> Sent: Tuesday, April 13, 2021 1:51 AM
> To: Dongjie (Jimmy) <[email-masked]>
> Cc: [email-masked]
> Subject: Re: [Idr] [[email-masked]: I-D Action:
> > I fully agree that incremental deployment of new flowspec components is
> important, and it does not need to wait for Flowspec 2.0.
> >
> > The first two capabilities are discussed in this thread and the updated draft,
> and to me it makes sense to distinguish these two capabilities.
> 

The assumption here is that not all the nodes are required to implement the filters (or some just don't want to), while they may be on the path of the flowspec rule propagation. If this is true, it may be useful for the operator to know the set of nodes with the first capability and the set of nodes with the second capability, although the capability of parsing and propagating is more relevant to this document. These two capabilities may be advertised using different mechanisms, as the first capability may be needed by nodes not directly peered. 

> > As sometimes a flowspec rule only needs to be installed on a subset of nodes,
> a node which is not required to install it or even support it may be on the
> propagation path of the flowspec rule. If there is some mechanism to limit the
> propagation scope or indicate the target nodes of the flowspec, the third
> capability may further help the incremental deployment of new flowspec
> components.
> 
> "opaque" NLRI.  We've had a number of outages resulting from parsing bugs
> as it is.

I understand the problems with the "opaque" NLRI, while if the rule specified in the NLRI is not required (or not capable) to be implemented in data plane on a transit node, will it cause less harm to the network? 

> is still problematic once you hit a node where the inner contents are validated
> and found to be problematic, but at least in this case "treat as withdraw"
> procedures are capable of being done.

Agree flowspec v2 can help with the syntax validation to some extent. While if the NLRI is considered malformed, the "treat as withdraw" error handling may not always work. 

> The underlying behavior you're discussing is the scope discussion that I think
> we're in general need of, and a separate thread was forked for.
> Perhaps once that's been discussed in more detail it makes sense to come back
> to this third point?

Sounds reasonable, will take a look at that thread. 

> 
> > Then back to the encoding of the flowspec capability bits, if we want to
> consider the case that a node can propagate unknown flowspec components,
> one reserved bit in the bit-string may be used to indicate such capability, other
> mechanism is also possible.
> 
> The encoding is the less exciting bit of the discussion. :-)  Let's revisit this
> after you've clarified your first two points.

OK. 

Best regards,
Jie

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 14 Apr 2021 20:43:11 -0400

Jie,

On Wed, Apr 14, 2021 at 03:21:54PM +0000, Dongjie (Jimmy) wrote:
> The assumption here is that not all the nodes are required to implement
> the filters (or some just don't want to), while they may be on the path of
> the flowspec rule propagation. If this is true, it may be useful for the
> operator to know the set of nodes with the first capability and the set of
> nodes with the second capability, although the capability of parsing and
> propagating is more relevant to this document. These two capabilities may
> be advertised using different mechanisms, as the first capability may be
> needed by nodes not directly peered. 

Understood.  My question is what should a BGP Speaker that has been told
that the adjacent BGP Speaker can parse a certain set of Flowspec components
but not use some of them do about it?  Is it strictly advisory?

If it's strictly advisory, I would advocate that we not worry about
publishing that part in BGP.  One motivation is that capability space is
still at somewhat of a premium without extended optional parameters.

A second motivation is there may be better mechanisms to publish the network
wide capabilities for using the filters.  I'll likely share a proposal in
the upcoming weeks.

> I understand the problems with the "opaque" NLRI, while if the rule
> specified in the NLRI is not required (or not capable) to be implemented
> in data plane on a transit node, will it cause less harm to the network? 

My draft tries to make clear that any device that filters a rule, either
because it doesn't receive it because of lack of capability of an upstream
node, or lack of local ability, can have potential to result in
mis-forwarding.  Whether it does so or not will depend on the independence
of the rules.

If my use of "independence" is unclear, I'll share an example next round.

> Agree flowspec v2 can help with the syntax validation to some extent.
> While if the NLRI is considered malformed, the "treat as withdraw" error
> handling may not always work. 

I think lengths in structured BGP NLRI will likely give us a hybrid
situation.  

Consider an NLRI that has a known prefix length.
Consider that it has components that have individual sub-lengths that are
clear in the protocol.

It is possible that the length of the individual components summed together
are still a valid "length" from the top level (the prefix-length) from a
high level syntactic level, but individual components may be of invalid
length or syntax.  This is similar to the situation for path attributes.

In such a case, the NLRI is malformed, but may be of enough integrity to
properly handle as an ignore/discard.

The BGP-CAR proposal and perhaps some of the evpn NLRI error handling cases have
some similar overlap to this case.  (RFC 7606 tries to be too generic about
"typed NLRI" and I think got it somewhat wrong.)

-- Jeff

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 15 Apr 2021 03:37:23 +0000

I agree.
The capability is exchanged only with neighbors and thus is
not capable of informing the capabilities of all BGP speakers
in the network. A more capable network management method
is needed for that.

The BGP capability is only useful to avoid sending a flowspec
to a neighbor that it would consider as malformed.

Regards,
Jakob.

-----Original Message-----
From: Idr <[email-masked]> On Behalf Of Jeffrey Haas
Sent: Wednesday, April 14, 2021 5:43 PM
To: Dongjie (Jimmy) <[email-masked]>
Cc: [email-masked]
Subject: Re: [Idr] [[email-masked]: I-D Action: draft-haas-flowspec-capability-bits-02.txt]

Jie,

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Apr 2021 11:42:40 +0200

All,

Yes what Jakob and now Jeff are saying makes sense. If we are to introduce
this capability it should be only about what BGP peer understands as part
of the NLRI. Nothing to do with data plane and filtering. Just observe that
even if data plane supports it and even of operator enables it filter may
still fail to get installed due to no space. So we never know at BGP level
what got installed and what not.

Now - I am still seriously concerned to add this to v1. Well unless we also
add the notion of optional types. That means that when setting up a filter
(manually or programmatically) types have an "Optional" flag - which means
that if a peer does not support this type it is ok to drop it and sent all
remaining types in NLRI instead of not sending the entire NLRI at all.

That would provide some tools to make sure that v1 deployments are intact.

Thx,
R.

On Thu, Apr 15, 2021 at 5:37 AM Jakob Heitz (jheitz) <jheitz=
[email-masked]> wrote:

> > > > I've read the recent discussion and the updated version of this
> draft, here are
> > > some thoughts and comments:
> > > >
> > > > 2.        The capability of parsing and propagating the component,
> but not for
> > > local use
> > > >
> > > Operationally, how would you expect a BGP Speaker to differentiate
> these two
> > > cases?
> >
> validated

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 15 Apr 2021 08:46:39 -0400

Robert,

> remaining types in NLRI instead of not sending the entire NLRI at all.

If I'm understanding this correctly as "strip some parts of the match
components", I think we'll find that problematic.

A useful bit of this conversation is we're getting discussion about what
happens when you have inconsistent filtering behaviors in a flowspec domain.
This was going to be a problem regardless of how we pass around the state
for it; flowspec v1 with new fields, or flowspec v2.

I also suspect this is not as dire as people are fearing.  Even in the route
reflector dissemination model, it's fairly common for deployments to have a
distinct route reflector plane for flowspec.  This avoids the concern that
errors in a flowspec session may have collateral damage for other address
families carried on the same peering session.

What this tends to mean is that such deployments are mostly one step away
from the targeted session deployment that is the alternative option.  This
means that grouping devices into sets that can support common classes of
filtering isn't a massive operational challenge.

That said, it's understood that passing flowspec in-band with a normal
peering session can still happen.  This will still likely be the case for
the providers that accept flowspec from their customers.  (A small number
based on my experience.)  But it does have overlap with inter-domain
deployment for the same administrative group - multiple ASes for same
entity.

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Apr 2021 15:23:39 +0200

Hi Jeff,

I think there is one more interesting observation to this.

Imagine I am running 1000 PEs and want to deploy some new filtering
capability to v1. For simplicity let's assume single flat layer of RRs is
used. I start upgrading say 100 routers

That means that unless the RRs can drop unsupported types on a per peer
basis I need to inject two almost identical NLRIs one with new type and one
without new types to the network.

And now the fun starts - 900 non upgraded PEs will receive the "old" NLRI,
but the new ones will receive both. And if they install the "old" one first
they will never act on matching the new types coming in subsequent NLRI.

If new comes first to some of those PEs and not the others we have mixed
behaviour again.

Only if the new comes first we have at least consistent filtering among
those 100 upgraded PEs.

Thx,
R.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 15 Apr 2021 13:24:43 +0000

Regards,
Jakob.

On Apr 15, 2021, at 2:43 AM, Robert Raszuk <[email-masked]> wrote:

All,

Yes what Jakob and now Jeff are saying makes sense. If we are to introduce this capability it should be only about what BGP peer understands as part of the NLRI. Nothing to do with data plane and filtering. Just observe that even if data plane supports it and even of operator enables it filter may still fail to get installed due to no space. So we never know at BGP level what got installed and what not.

That's being just a little over dramatic. It's the case with all BGP. For any use case where it really matters and the originator must know exactly when and where its originated NLRI got installed, we don't use BGP.

Now - I am still seriously concerned to add this to v1. Well unless we also add the notion of optional types. That means that when setting up a filter (manually or programmatically) types have an "Optional" flag - which means that if a peer does not support this type it is ok to drop it and sent all remaining types in NLRI instead of not sending the entire NLRI at all.

Thx,
R.

On Thu, Apr 15, 2021 at 5:37 AM Jakob Heitz (jheitz) <jheitz=[email-masked]<mailto:[email-masked]>> wrote:
I agree.
The capability is exchanged only with neighbors and thus is
not capable of informing the capabilities of all BGP speakers
in the network. A more capable network management method
is needed for that.

Regards,
Jakob.

-----Original Message-----
From: Idr <[email-masked]<mailto:[email-masked]>> On Behalf Of Jeffrey Haas
Sent: Wednesday, April 14, 2021 5:43 PM
To: Dongjie (Jimmy) <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [Idr] [[email-masked]<mailto:[email-masked]>: I-D Action: draft-haas-flowspec-capability-bits-02.txt]

Jie,

-- Jeff

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Apr 2021 15:34:18 +0200

Jakob,

Pls see my last msg. It is not about originator assurance that something
got installed. It is about operator basic understanding on what is going on
in his network from BGP distribution point of view.

Also pls observe that capabilities today require session reset. Dynamic
capabilities is not there. So if I enable some filtering type on PE I need
to reset my session to RR. If more SAFIs travel over this session this will
get pretty ugly.

Perhaps we should instead define FlowSpec ORF and single new capability
"Wait for FS-ORF". FS ORF would signal to peer what types it supports.
Simple, easy and non disruptive with pretty much the same end effect.

Cheers,
R.

On Thu, Apr 15, 2021 at 3:24 PM Jakob Heitz (jheitz) <[email-masked]>
wrote:

> That's being just a little over dramatic. It's the case with all BGP. For
> any use case where it really matters and the originator must know exactly
> when and where its originated NLRI got installed, we don't use BGP.
>
>
> Now - I am still seriously concerned to add this to v1. Well unless we
> also add the notion of optional types. That means that when setting up a
> filter (manually or programmatically) types have an "Optional" flag - which
> means that if a peer does not support this type it is ok to drop it and
> sent all remaining types in NLRI instead of not sending the entire NLRI at
> all.
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 15 Apr 2021 17:45:06 +0000

Jeff's idea is even simpler.
It allows us to add new types to flowspec.

Regards,
Jakob.

From: Robert Raszuk <[email-masked]>
Sent: Thursday, April 15, 2021 6:34 AM
To: Jakob Heitz (jheitz) <[email-masked]>
Cc: Jeffrey Haas <[email-masked]>; Dongjie (Jimmy) <[email-masked]>; [email-masked]
Subject: Re: [Idr] [[email-masked]: I-D Action: draft-haas-flowspec-capability-bits-02.txt]

Jakob,

Pls see my last msg. It is not about originator assurance that something got installed. It is about operator basic understanding on what is going on in his network from BGP distribution point of view.

Also pls observe that capabilities today require session reset. Dynamic capabilities is not there. So if I enable some filtering type on PE I need to reset my session to RR. If more SAFIs travel over this session this will get pretty ugly.

Perhaps we should instead define FlowSpec ORF and single new capability "Wait for FS-ORF". FS ORF would signal to peer what types it supports. Simple, easy and non disruptive with pretty much the same end effect.

Cheers,
R.

On Thu, Apr 15, 2021 at 3:24 PM Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>> wrote:

Regards,
Jakob.

On Apr 15, 2021, at 2:43 AM, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

All,

Thx,
R.

Regards,
Jakob.

-----Original Message-----
From: Idr <[email-masked]<mailto:[email-masked]>> On Behalf Of Jeffrey Haas
Sent: Wednesday, April 14, 2021 5:43 PM
To: Dongjie (Jimmy) <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [Idr] [[email-masked]<mailto:[email-masked]>: I-D Action: draft-haas-flowspec-capability-bits-02.txt]

Jie,

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Apr 2021 21:24:12 +0200

>  It allows us to add new types to flowspec.

So does ORF without any side effects which come free with using bgp
capabilities.

In any case to restate adding new types to flowspec v1 is not something I
support.

Thx,
R.

On Thu, Apr 15, 2021 at 7:46 PM Jakob Heitz (jheitz) <[email-masked]>
wrote:

> *From:* Robert Raszuk <[email-masked]>
> *Sent:* Thursday, April 15, 2021 6:34 AM
> *To:* Jakob Heitz (jheitz) <[email-masked]>
> *Cc:* Jeffrey Haas <[email-masked]>; Dongjie (Jimmy) <[email-masked]>;
> [email-masked]
> *Subject:* Re: [Idr] [[email-masked]: I-D Action: