Subject: Re: [GROW] I-D Action: draft-ymbk-grow-wkc-behavior-00.txt

From: Job Snijders <[email-masked]>
Date: Tue, 27 Feb 2018 18:16:03 +0100

Hi authors,

This is an excellent initiative.

Kind regards,

Job

On Tue, Feb 27, 2018 at 09:13:31AM -0800, [email-masked] wrote:
> 
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> 
> 
>         Title           : Well-Known Community Policy Behavior
>         Authors         : Jay Borkenhagen
>                           Randy Bush
>                           Ron Bonica
>                           Serpil Bayraktar
> 	Filename        : draft-ymbk-grow-wkc-behavior-00.txt
> 	Pages           : 6
> 	Date            : 2018-02-27
> 
> Abstract:
>    Well-Known BGP Communities are manipulated inconsistently by current
>    implementations.  This results in difficulties for operators.  It is
>    recommended that removal policies be applied consistently to Well-
>    Known Communities.
> 
> 
> 
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-ymbk-grow-wkc-behavior/
> 
> There are also htmlized versions available at:
> https://tools.ietf.org/html/draft-ymbk-grow-wkc-behavior-00
> https://datatracker.ietf.org/doc/html/draft-ymbk-grow-wkc-behavior-00
> 
> 
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: Randy Bush <[email-masked]>
Date: Thu, 01 Mar 2018 09:52:37 +0900

question for ops:

  did you even know this was happening, that "set community" might not
  replace ALL well known communities?"

the reasom i ask is, as you see in the draft, we are hesitant to propose
changing behavior for existing wks as it might impact ops.  but if we
were not aware of this behavior, then maybe change is ok, in fact
desirable.

i did not realize this was going on.  jay, who discovered it, was
surprised.

randy

---

From: Job Snijders <[email-masked]>
Date: Thu, 1 Mar 2018 01:32:33 +0000

On Thu, Mar 01, 2018 at 09:52:37AM +0900, Randy Bush wrote:
> the reasom i ask is, as you see in the draft, we are hesitant to
> propose changing behavior for existing wks as it might impact ops.
> but if we were not aware of this behavior, then maybe change is ok, in
> fact desirable.
> 

NTT noticed this type of inconsistency between some vendors somewhere in
July 2016. A customer used a combination of NTT-specific BGP communities
to influence traffic, together with NO_ADVERTISE. Unbeknownst to us,
this combination of communities resulted in network behavior that made
the customer happy.

Further down the timeline, the customer was migrated to another vendor's
routing platform (or got a new circuit on a new platform, don't recall)
and suddenly noticed that the previously working combinatory trick
no longer worked. At that point investigation showed that "set
community" does not do the same thing on all platforms, and we had to
figure out a way to make things consistent.

Our currently routing policy is to allow (not delete) some of the
well-known communities, and squash (remove) a bunch of not-so-well-know
communities and unsupported well-known communities, and let
potential-future-well-known communitie pass-through to try to not
stiffle innovation. For each new well-known community we'll have to
decide whether to remove it or let it pass-through.

Kind regards,

Job

---

From: Randy Bush <[email-masked]>
Date: Thu, 01 Mar 2018 10:54:08 +0900

>> but if we were not aware of this behavior, then maybe change is ok,
>> in fact desirable.
>> 
> NTT noticed this type of inconsistency between some vendors somewhere
> in July 2016. A customer used a combination of NTT-specific BGP
> communities to influence traffic, together with
> NO_ADVERTISE. Unbeknownst to us, this combination of communities
> resulted in network behavior that made the customer happy.
> 
> Further down the timeline, the customer was migrated to another
> vendor's routing platform (or got a new circuit on a new platform,
> don't recall) and suddenly noticed that the previously working
> combinatory trick no longer worked. At that point investigation showed
> that "set community" does not do the same thing on all platforms, and
> we had to figure out a way to make things consistent.
> 
> well-known communities, and squash (remove) a bunch of
> not-so-well-know communities and unsupported well-known communities,
> and let potential-future-well-known communitie pass-through to try to
> not stiffle innovation. For each new well-known community we'll have
> to decide whether to remove it or let it pass-through.

not sure if you and/or your customer(s) would be made unhappy if current
behavior in some vendor(s), where "set" does not remove some wkcs, was
changed to remove all.

randy

---

From: Job Snijders <[email-masked]>
Date: Thu, 1 Mar 2018 02:14:20 +0000

> behavior in some vendor(s), where "set" does not remove some wkcs, was

I'm not sure either. NTT's thinking is shifting to "delete as little as
you can, preserve as much of other people's communities", which
precludes the use of "set community" anyhow.

I hope that in the 'large community' era, life will be simpler where you
can just delete "2914:*:*", and perhaps "65535:* except 65535:0" on
sessions where you don't want to honor NTT-specific communities and WKCs
(except GRACEFUL_SHUTDOWN) and leave the rest. Oh.. that's already quite
involved. :-)

I know of other carriers who are not interested in preserving any
community information and heavily rely on "set community" == 'delete'.
But... at the same time those same carriers are interested in honoring
GRACEFUL_SHUTDOWN.

If the concept of "set community X" == "delete everything & set X" had
actually been a thing in the last 10 years it would've prevented that
example I mentioned.

On the other hand, "set community X" == "delete everything" also removes
WKC an operator might want to honor, so perhaps the (unreliable) concept
of "set community X" is outdated and needs to be replaced with a
different construct all together.

Trouble with WKC is that some you want to remove, some you want to
honor, and some you just want to pass-through without taking any action.
"set community" isn't a great fit for cherry-picking.

Kind regards,

Job

---

From: Jay Borkenhagen <[email-masked]>
Date: Wed, 28 Feb 2018 21:52:50 -0500

> > > in July 2016. [...]

 [...]

 > But... at the same time those same carriers are interested in honoring

For what it's worth, I first noticed the special treatment of some
well-known communities after Job announced his GRACEFUL_SHUTDOWN
beacon prefix.  I expected that it would not have passed intact
through a "set community foo" policy, but it did.

Clearly "set community foo" is not appropriate for use on many classes
of connections -- e.g. on connections to one's customers -- but where
it is used it should not cause surprises.

						Jay B.

---

From: Randy Bush <[email-masked]>
Date: Thu, 01 Mar 2018 13:27:40 +0900

> honor, and some you just want to pass-through without taking any

and, if i was of that mind, i would want to decide which, not have the
vendors do so.

> "set community" isn't a great fit for cherry-picking.

nor is

   a = b

but it's something to which we are used.  to find that it actually
means

   a = (b-c) for undocumented values of c

was rather surprising.

randy