Subject: Re: [Idr] Flowspec draft-ietf-idr-rfc5575bis last (known) issue that needs to be resolved

From: Jeffrey Haas <[email-masked]>
Date: Fri, 25 Jan 2019 17:35:07 -0500

Christoph,

On Thu, Dec 27, 2018 at 11:55:46AM +0100, Christoph Loibl wrote:
> During the interim we agreed on a procedure to resolve the last known
> issues of draft-ietf-idr-rfc5575bis. Since then I uploaded some changes
> (as posted on the list previously). The result is, that there is only one
> remaining issue that I want to address now.

Thanks for the recent diffs.  They covered my comments on those sections.

> The draft RFC5557bis contains a section on traffic action interference
> which is (as pointed out already on the list) very restrictive and may
> break current use-cases as pointed out by Jeffery. We had some discussion
> on this amongst the authors of the draft. I try to summarize and would
> love to get advice from the list:
> 
> *) Precondition - RFC5575:
> 
> RFC5575 completely ignores the fact that traffic actions may “interfere”
> with each other. (simplest example: 2 different rate-limits for a given
> flow - which rate-limit should be applied?) This makes the result
> unpredictable.

Arguably, it makes the results "implementation dependent".  

An important observation is that since these conflicts are caused by an
excess of extended communities that inconsistent behaviors can be corrected
by operators.  "Don't do that".

That said, I'm supportive of trying to be clearer in our behaviors here.

> *) Suggested solution A - make action predictable:
> 
> As pointed out above the current specification (RFC5575) is unpredictable
> while the draft RFC5575bis is too restrictive. Suggested solution A
> targets this problem by sorting the traffic action communities (while not
> restricting what is actually propagated):
> 
> + For traffic rate communities: Use the lowest traffic rate (floating
> point number encoded in the ext-community)
> 
> + For redirect-rt commuities: Sort them in ascending order and use the
> lowest feasible for traffic redirection (some RTs may not be feasible
> because there is on import statement for that particular RT in the
> configuration).
> 
> + For the remaining actions: Always use the lowest encoded value (memcmp).
> 
> Using always the “lowest” is only for the sake of predictability,
> “highest” may be as good. (I remember that someone mentioned during the
> interim meeting that some sorting is already part of someones
> implementation).

As has been previously noted in mails on extended community features, it's
quite common for implementations to canonicalize sets of extended
communities through sorting.  So, in the sense that sorting should be done,
that part is covered.

The place I think you may find resistance to this suggestion is sorted
_how_.  I'll air a very minor amount of dirty laundry from our
implementation to offer as an example.

We store the type/subtype in a unsigned 16 bit number.
We store the remaining 6 octets in one of two different 32-bit unsigned
numbers, depending on the community semantic.  (32-bit implementation
originally.)
The sort will consider type/subtype, then each of the two 32-bit numbers.

As such, there's no guarantee in our current implementation that there'd be
a sort congruent a more abstract sort routine in a draft such as this one.  

There's also the issue that another working group may look upon this
discussion and request a different sort mechanism for their mechanisms.
(I'm looking at you, bess!)

What this would lead to is per-application (e.g. flowspec) sort orders,
which means either your extended community collection gets canonicalized for
easy use, or you have to do a sub-sort before taking an action.  Choose
where your CPU goes. :-)

All of that said:

> *) Suggested solution B - only document the behaviour:
> 
> We can add to the operational consideration section that there may be
> interfering traffic actions. FS filters with interfering actions shall
> still be propagated but the actual selection of the appropriate actions is
> up to the implementation. If a network/fs-domain wide predictable
> behaviour is needed for an application this can still be achieved by using
> custom BGP communities and BGP policies (plus rewriting those to the
> appropriated actions).
> 
> This is very “lightweight” and I think that solution B may easily find
> consensus. However the benefit over RFC5575 is only very little.
> 
> Any preferences how we should proceed?

I'd suggest B.

The text I would suggest would roughly say:
Traffic actions are encoded as BGP extended communities.  Traffic actions,
in many cases, are expected to only be encoded once.  For example, only a
single valid traffic-rate-bytes extended community can be processed at a
time.  Operators should take precautions to not introduce conflicting
actions as a result of policy configuration.

...

There may be some worth in adding a similar comment to implementors about
having good policy mechanisms to make this job easy for operators.

-- Jeff