Subject: [GROW] draft-ietf-grow-bgp-gshut status?

From: Job Snijders <[email-masked]>
Date: Wed, 15 Mar 2017 14:10:39 +0100

Hi group,

I noticed that https://tools.ietf.org/html/draft-ietf-grow-bgp-gshut-06
expired two years ago. Can anyone offer some insight why it lapsed?

What implementatations exist? A fellow operator told me that IOS, IOS XE
has support for graceful shutdown, are there others?

Kind regards,

Job

---

From: <[email-masked]>
Date: Wed, 15 Mar 2017 14:00:37 +0000

Hi Job, GROW

> From: GROW [mailto:[email-masked]] On Behalf Of Job Snijders  > Sent: Wednesday, March 15, 2017 2:11 PM
> 
 > Hi group,
 > 

In order to signal the graceful shutdown over the EBGP session, gshut uses a "well-know" BGP community. Compared to using a protocol extension, this allows a vanillia sender/receiver to handle the information using a regular BGP policy.
So far so good. This is specified, implemented both with BGP policy and automated by some routers, tested (both options).

Now, for some deployments, the use of a non-transitive community offer a better assurance that the community has indeed be originated by the connected eBGP peer. The issue is that currently there is no implementation of non-transitive well-known communities.
draft-ietf-idr-reserved-extended-communities is a short draft to define non-transitive well-known communities. It proposed to re-use an "existing" non-transitive extended community, defined for four-octets AS: draft-ietf-idr-as4octet-extcomm-generic-subtype. But it turns out that this latter draft did not progress and has recently been replaced by BGP large community. The later do no support non-transitive community. 

So after waiting for some years for draft-ietf-idr-as4octet-extcomm-generic-subtype, this path has just been closed. As of today, the possible options seems:
- forget about non-transitive community. In particular as during the BGP large community discussions, the requirement for non-transitive community has been discussed and explicitly called as not needed. So let's listen to this and do the same.
- have draft-ietf-idr-reserved-extended-communities use a different extended community type. This is easy to write, but if this does not get implemented, the value is limited/null. 

 > has support for graceful shutdown, are there others?
 
Same information on my side.
With the restriction that those implementations only implement the transitive community.
 
Kind regards,
--Bruno

 > Kind regards,
 > 
 > Job
 > 
 > _______________________________________________
 > GROW mailing list
 > [email-masked]
 > https://www.ietf.org/mailman/listinfo/grow

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Job Snijders <[email-masked]>
Date: Wed, 15 Mar 2017 15:54:16 +0100

Hi Bruno,

On Wed, Mar 15, 2017 at 02:00:37PM +0000, [email-masked] wrote:
> In order to signal the graceful shutdown over the EBGP session, gshut
> uses a "well-know" BGP community. Compared to using a protocol
> extension, this allows a vanillia sender/receiver to handle the
> information using a regular BGP policy.
> So far so good. This is specified, implemented both with BGP policy
> and automated by some routers, tested (both options).
> 
> Now, for some deployments, the use of a non-transitive community offer
> a better assurance that the community has indeed be originated by the
> connected eBGP peer. The issue is that currently there is no
> implementation of non-transitive well-known communities.
> draft-ietf-idr-reserved-extended-communities is a short draft to
> define non-transitive well-known communities. It proposed to re-use an
> "existing" non-transitive extended community, defined for four-octets
> AS: draft-ietf-idr-as4octet-extcomm-generic-subtype. But it turns out
> that this latter draft did not progress and has recently been replaced
> by BGP large community. The later do no support non-transitive
> community. 
> 
> So after waiting for some years for
> draft-ietf-idr-as4octet-extcomm-generic-subtype, this path has just
> been closed. As of today, the possible options seems:
> 
> - forget about non-transitive community. In particular as during the
> BGP large community discussions, the requirement for non-transitive
> community has been discussed and explicitly called as not needed. So
> let's listen to this and do the same.

I'd like to add a small nuance: For the use case of large communities,
non-transitivity was considered an undesirable property. 

To be honest, if the 'gshut' community 'escapes' the adjacent ASN for
which it was intended, what is the worst that can happen? That BGP
speakers somewhere in the DFZ consider the path less desirable? This
aligns with what is expected to happen in the near future anyway: the
bgp session will be torn down and the path will cease to exist.

In the case where no shutdown event follows (the gshut community is used
as a traffic engineering trick), it kind of goes in the same category as
intermediat networks prepending ASNs to the AS_PATH to make it less
desirable, or fiddling with origin. If I were to consider "permanent
use" of the gshut community a violation of my agreement with the
adjacent network, this would be easy enough to monitor for and
subsequently resolve at layer-8.

> - have draft-ietf-idr-reserved-extended-communities use a different
> extended community type. This is easy to write, but if this does not
> get implemented, the value is limited/null. 

I concur. A similar consideration could be made whether gshut deserves
its own path attribute or not.  Usually the nice thing about well known
rfc 1997 communities is their rapid implementation and deployability.

> > What implementatations exist? A fellow operator told me that IOS,
> > IOS XE has support for graceful shutdown, are there others?
>  
> Same information on my side.  With the restriction that those
> implementations only implement the transitive community.

ack.

I'm somewhat inclined to proceed with the gshut concept as a well-known
transitive rfc 1997 community. What do others think?

Kind regards,

Job

---

From: <[email-masked]>
Date: Wed, 15 Mar 2017 15:21:18 +0000

Hi Job,

Thanks for the feedback.
More inline

> From: Job Snijders [mailto:[email-masked]]  > Sent: Wednesday, March 15, 2017 3:54 PM
> 
 > Hi Bruno,
 > 

"undesirable property" is a bit beyond the term that I would use, but who cares now; it's not part of it.
 
 > subsequently resolve at layer-8.
 
In sync. In particular in sync with the security considerations of the draft
https://tools.ietf.org/html/draft-ietf-grow-bgp-gshut-06#section-8

 > rfc 1997 communities is their rapid implementation and deployability.

Indeed. That was specifically important for the VPN customers, with 1000s of CE, some with "legacy" software which are difficult to upgrade. So the possibility to use a vanilla CE with a BGP policy was part of the requirements. This may be less of an issue for big SP routers, although incremental deployment is always a plus, especially when more than one AS are involved.
 
 > transitive rfc 1997 community. What do others think?
 
I agree with you.
Until now, in order to capture all the options, waiting for draft-ietf-idr-as4octet-extcomm-generic-subtype seemed reasonable (at the cost of waiting for years, but this was not expected as we though vendors would implement it to accommodate 4-octects AS deployments). It's not anymore, so I guess we'll update the draft to follow this proposed path.

Thanks,
Kind regards,
--Bruno

---

From: Ben Maddison <[email-masked]>
Date: Wed, 15 Mar 2017 15:22:32 +0000

I would be very happy with that outcome.
We've been using this for ages, and would like to see it work more widely in our adjacent networks.

Although not truly a fix for the transitivity problem, when we match on gshut from a peer, in addition to setting a lower-than-usual LP, we also append no-export, which prevents gshut-ed prefixes from leaking at all.
I've never been hugely worried about the security consequences otherwise.

Cheers,

Ben Maddison

Director
Workonline Communications (Pty) Ltd

Office:     +27 (0) 21 200 9000
Mobile:   +27 (0) 82 415 5545
Email:      [email-masked]
SIP:          [email-masked]

-----Original Message-----
From: GROW [mailto:[email-masked]] On Behalf Of Job Snijders
Sent: Wednesday, March 15, 2017 4:54 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [GROW] draft-ietf-grow-bgp-gshut status?

Hi Bruno,

On Wed, Mar 15, 2017 at 02:00:37PM +0000, [email-masked] wrote:
> > From: GROW [mailto:[email-masked]] On Behalf Of Job Snijders  
> > > Sent: Wednesday, March 15, 2017 2:11 PM
> >
> > I noticed that 
> > https://tools.ietf.org/html/draft-ietf-grow-bgp-gshut-06

I'd like to add a small nuance: For the use case of large communities, non-transitivity was considered an undesirable property. 

To be honest, if the 'gshut' community 'escapes' the adjacent ASN for which it was intended, what is the worst that can happen? That BGP speakers somewhere in the DFZ consider the path less desirable? This aligns with what is expected to happen in the near future anyway: the bgp session will be torn down and the path will cease to exist.

In the case where no shutdown event follows (the gshut community is used as a traffic engineering trick), it kind of goes in the same category as intermediat networks prepending ASNs to the AS_PATH to make it less desirable, or fiddling with origin. If I were to consider "permanent use" of the gshut community a violation of my agreement with the adjacent network, this would be easy enough to monitor for and subsequently resolve at layer-8.

> get implemented, the value is limited/null.

I concur. A similar consideration could be made whether gshut deserves its own path attribute or not.  Usually the nice thing about well known rfc 1997 communities is their rapid implementation and deployability.

ack.

I'm somewhat inclined to proceed with the gshut concept as a well-known transitive rfc 1997 community. What do others think?

Kind regards,

Job

---

From: <[email-masked]>
Date: Wed, 15 Mar 2017 15:26:27 +0000

Thanks for the useful feedback.

--Bruno

 > From: Ben Maddison [mailto:[email-masked]]
 > Sent: Wednesday, March 15, 2017 4:23 PM
 > To: Job Snijders; DECRAENE Bruno IMT/OLN
 > Cc: [email-masked]
 > Subject: RE: [GROW] draft-ietf-grow-bgp-gshut status?
 > 
 > Although not truly a fix for the transitivity problem, when we match on gshut from a peer, in
 > addition to setting a lower-than-usual LP, we also append no-export, which prevents gshut-
 > ed prefixes from leaking at all.
 > To be honest, if the 'gshut' community 'escapes' the adjacent ASN for which it was intended,
 > what is the worst that can happen? That BGP speakers somewhere in the DFZ consider the
 > path less desirable? This aligns with what is expected to happen in the near future anyway:
 > the bgp session will be torn down and the path will cease to exist.
 > 
 > In the case where no shutdown event follows (the gshut community is used as a traffic
 > engineering trick), it kind of goes in the same category as intermediat networks prepending
 > ASNs to the AS_PATH to make it less desirable, or fiddling with origin. If I were to consider
 > "permanent use" of the gshut community a violation of my agreement with the adjacent
 > network, this would be easy enough to monitor for and subsequently resolve at layer-8.
 > 
 > I concur. A similar consideration could be made whether gshut deserves its own path
 > attribute or not.  Usually the nice thing about well known rfc 1997 communities is their
 > rapid implementation and deployability.
 >

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 17 Mar 2017 09:41:31 +0100

Bruno,

BGP session can carry multiple messages. One is UPDATE MSG

Why do you want to nest indicator about session going down in the UPDATE
msg rather then just using new NOTIFICATION subcode say "Gracefull
shutdown" ?

Clearly it would not be backwards compatible but I guess this is no longer
a goal is it ?

The issue with sending it in UPDATE MSG is that you effectively need to
re-advertise the entire table just before going down which will in turn
could cause more churn in your peer's ASes and beyond.

Or do you want to came back to concept of defining a beacon prefix acting
as semaphore for entire session :)

Cheers,
R.

On Wed, Mar 15, 2017 at 4:26 PM, <[email-masked]> wrote:

>  > We've been using this for ages, and would like to see it work more
> widely in our adjacent
>  > networks.
>  >
> which it was intended,
> its own path
> ____________________________________________________________
> _____________________________________________________________
>
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
> recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration,
> This message and its attachments may contain confidential or privileged
> information that may be protected by law;

---

From: <[email-masked]>
Date: Fri, 17 Mar 2017 08:58:41 +0000

Hi Robert,

Please see inline [Bruno]

From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Friday, March 17, 2017 9:42 AM
To: DECRAENE Bruno IMT/OLN
Cc: [email-masked]
Subject: Re: [GROW] draft-ietf-grow-bgp-gshut status?

Bruno,

Why do you want to nest indicator about session going down in the UPDATE msg rather then just using new NOTIFICATION subcode say "Gracefull shutdown" ?

Clearly it would not be backwards compatible but I guess this is no longer a goal is it ?

[Bruno] The goal was to be able to use gshut even if both EBGP peer are not enhanced to support it. The benefit of flagging routes with a community is that gshut may be implemented on vanilla routers using a BGP route map/policy. Incremental deployment in a benefit, in particular between AS/organisations, and in particular on small/low end routers which are not replaced every 4 years.

The issue with sending it in UPDATE MSG is that you effectively need to re-advertise the entire table just before going down which will in turn could cause more churn in your peer's ASes and beyond.

[Bruno] There is no issue. The goal _is_ to trigger BGP path hunting in search for the back up path, before shutting down the BGP session:
- If the remote EBGP peer is already aware of the back up route, the message/churn is not propagated in the peer’s ASes not beyond
- If the remote EBGP peer is not aware of the back up route, the message/churn is indeed propagated in the peer’s ASes and possibly beyond, but that is the goal and the feature. Note that this is also the case when you perform a vanilla BGP shut. There is no free lunch: you do need to search for the back up paths.
Plus with gshut, you replace “urgent” BGP message by non-urgent BGP messages.

Or do you want to came back to concept of defining a beacon prefix acting as semaphore for entire session :)
[Bruno] No. This is not what is described in the draft.

Cheers,
--Bruno

Cheers,
R.

On Wed, Mar 15, 2017 at 4:26 PM, <[email-masked]<mailto:[email-masked]>> wrote:
Thanks for the useful feedback.

--Bruno

 > From: Ben Maddison [mailto:[email-masked]<mailto:[email-masked]>]
 > Cc: [email-masked]<mailto:[email-masked]>
 > Office:     +27 (0) 21 200 9000<tel:%2B27%20%280%29%2021%20200%209000>
 > Mobile:   +27 (0) 82 415 5545<tel:%2B27%20%280%29%2082%C2%A0415%205545>
 > Email:      [email-masked]<mailto:[email-masked]>
 > SIP:          [email-masked]<mailto:[email-masked]>
 >
 >
 >
 >
 > From: GROW [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Job Snijders
 > To: [email-masked]<mailto:[email-masked]>
 > Cc: [email-masked]<mailto:[email-masked]>
 > On Wed, Mar 15, 2017 at 02:00:37PM +0000, [email-masked]<mailto:[email-masked]> wrote:
 > > > From: GROW [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Job Snijders
 > [email-masked]<mailto:[email-masked]>

[email-masked]<mailto:[email-masked]>

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 17 Mar 2017 11:39:25 +0100

Hi Bruno,

[Bruno] The goal was to be able to use gshut even if both EBGP peer are not
> enhanced to support it. The benefit of flagging routes with a community is
> that gshut may be implemented on vanilla routers using a BGP route
> map/policy.
>

​Sure thing. However peers need to be consistently configured with ​the
same policy to understand the meaning of the community.

If you think this is easy - great.

> Incremental deployment in a benefit, in particular between
> AS/organisations, and in particular on small/low end routers which are not
> replaced every 4 years.
>

​Well it is not router replacement .. it is software upgrade. But sure ​if
you can convince peers to setup same error free policy that is perfect. No
objections at all :).

​Best,
R.​

---

From: <[email-masked]>
Date: Fri, 17 Mar 2017 11:11:28 +0000

Hi Robert,

From: Robert Raszuk

Hi Bruno,

[Bruno] The goal was to be able to use gshut even if both EBGP peer are not enhanced to support it. The benefit of flagging routes with a community is that gshut may be implemented on vanilla routers using a BGP route map/policy.

​Sure thing. However peers need to be consistently configured with ​the same policy to understand the meaning of the community.

[Bruno2] On the receiver side, this is regular use of BGP community between 2 ASes. I’m not trying to evaluate simplicity or easiness.  That being said, community is a pretty common tool, including between ASes. So definitely doable IMHO.

On the sender side (gshut initiator), there is a need to attach a new community, e.g. though the application of the gshut BGP policy. Difficulty is implementation dependent. This part is clearly easier when implemented by the OS as part of the shutdown of the BGP session.
Incremental deployment in a benefit, in particular between AS/organisations, and in particular on small/low end routers which are not replaced every 4 years.

​Well it is not router replacement .. it is software upgrade.

[Bruno2] I do make the distinction. Indeed, this is a control plane only feature, which can be implemented by a software upgrade. Yet, even software upgrade may be costly when it involves thousands of CE routers in 1000 of locations and 100s of actors. Especially for small CE routers which do not support High Availability features.
Then you have “legacy” plateform, i.e. vendors not providing software upgrade anymore.

But sure ​if you can convince peers to setup same error free policy that is perfect. No objections at all :).
[Bruno2] I do believe that peers are capable of configuring a BGP policy matching one BGP community.
My intention is not to convince unwilling peers, but to provide this as a service for peers willing to reduce the impact of a BGP session shutdown. Actually, the request was coming from my peers which were affected by my maintenance operations requiring the eBGP session shutdown.

Best,
Bruno

​Best,
R.​

---

From: Ben Maddison <[email-masked]>
Date: Fri, 17 Mar 2017 11:14:24 +0000

Hi Robert,

There is no requirement that peers implement anything at all. In the absence of any configured policy, the community gets ignored, and no action is taken prior to shutdown – which is exactly the status-quo without the draft, so no harm done. Each network is free to implement whatever ingress policy is sensible in their own view, which should be to take the same action as when receiving an IXP list mail saying “we’re gonna shut x,y,z down shortly…”. The difference is they can now handle that in BGP policy to avoid the manual NOC activity.

Also, there is no additional code required (assuming support for 1997) in order to implement the bare bones of the functionality – e.g. match on 65535:0, set local-pref.
The automation features around it (such as neighbor shutdown graceful on XE) need new software, but are also trivial to recreate in an in house EMS if people are that pathological about touching their routers.

The only problem scenario that I can see is where someone is using 65535:0 for a different purpose in eBGP. But some people are beyond help!

Cheers,

Ben Maddison

Director

Office:     +27 (0) 21 200 9000
Mobile:   +27 (0) 82 415 5545
Email:      [email-masked]<mailto:[email-masked]>
SIP:          [email-masked]<sip:[email-masked]>

[Workonline Communications]<http://www.workonline.co.za/>

From: GROW [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Friday, March 17, 2017 12:39 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [GROW] draft-ietf-grow-bgp-gshut status?

Hi Bruno,

​Well it is not router replacement .. it is software upgrade. But sure ​if you can convince peers to setup same error free policy that is perfect. No objections at all :).

​Best,
R.​

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 17 Mar 2017 12:29:51 +0100

Hi Ben & Bruno,

If we all agree community approach is the best for signalling to peer that
this prefix should be avoided why folks just do not do it today already ?
Why do we need IETF draft for that :) ?

If this is about publishing a communities which include "warning" to peers
"Hey I am bringing this prefix down soon .. "(maybe even with parameter how
soon :))) what stopping Orange to declare we will attach YZ:MN community
and re-advertise routes when we are to take down the session.

That way any peer of Orange may use it at their own discretion.

- - -

The idea behind putting single message in NOTIFICATION was just to automate
it and make it deterministic. When you send community some peers may
converge quicker some much slower. You never know when your peer is done.

Here only after his best path and rib insertion of new paths around you is
complete he would tear the session. Some timeout would be defined as well.
And it will apply to all AFI/SAFIs as well as current and future BGP
messages send over given session.

Best,
R.

On Fri, Mar 17, 2017 at 12:14 PM, Ben Maddison <[email-masked]>
wrote:

> Hi Robert,
>
>
>
> There is no requirement that peers implement anything at all. In the
> absence of any configured policy, the community gets ignored, and no action
> is taken prior to shutdown – which is exactly the status-quo without the
> draft, so no harm done. Each network is free to implement whatever ingress
> policy is sensible in their own view, which should be to take the same
> action as when receiving an IXP list mail saying “we’re gonna shut x,y,z
> down shortly…”. The difference is they can now handle that in BGP policy to
> avoid the manual NOC activity.
>
>
>
> Also, there is no additional code required (assuming support for 1997) in
> order to implement the bare bones of the functionality – e.g. match on
> 65535:0, set local-pref.
>
> The automation features around it (such as neighbor shutdown graceful on
> XE) need new software, but are also trivial to recreate in an in house EMS
> if people are that pathological about touching their routers.
>
>
>
> The only problem scenario that I can see is where someone is using 65535:0
> for a different purpose in eBGP. But some people are beyond help!
>
>
>
> Cheers,
>
>
>
> Ben Maddison
>
>
>
> *Director*
>
> Office:     +27 (0) 21 200 9000 <+27%2021%20200%209000>
>
> Mobile:   +27 (0) 82 415 5545 <+27%2082%20415%205545>
>
>  <http://www.workonline.co.za/>
>
>
>
> *From:* GROW [mailto:[email-masked]] *On Behalf Of *Robert Raszuk
> *Sent:* Friday, March 17, 2017 12:39 PM
> *To:* [email-masked]
> *Cc:* [email-masked]
> *Subject:* Re: [GROW] draft-ietf-grow-bgp-gshut status?
>
>
>
> Hi Bruno,
>
>
>
> [Bruno] The goal was to be able to use gshut even if both EBGP peer are
> not enhanced to support it. The benefit of flagging routes with a community
> is that gshut may be implemented on vanilla routers using a BGP route
> map/policy.
>
>
>

---

From: Ben Maddison <[email-masked]>
Date: Fri, 17 Mar 2017 12:09:05 +0000

Hi Robert,

If we all agree community approach is the best for signalling to peer that this prefix should be avoided why folks just do not do it today already ? Why do we need IETF draft for that :) ?

If this is about publishing a communities which include "warning" to peers "Hey I am bringing this prefix down soon .. "(maybe even with parameter how soon :))) what stopping Orange to declare we will attach YZ:MN community and re-advertise routes when we are to take down the session.

[BEN] Nothing is stopping them. The primary benefit is the use of a well-known community that can be matched in generically constructed policies without prior knowledge of whether the peer makes use of this procedure, and what community they use for the purpose if they do.

- - -

The idea behind putting single message in NOTIFICATION was just to automate it and make it deterministic. When you send community some peers may converge quicker some much slower. You never know when your peer is done.

Here only after his best path and rib insertion of new paths around you is complete he would tear the session. Some timeout would be defined as well. And it will apply to all AFI/SAFIs as well as current and future BGP messages send over given session.

[BEN] From a quick read of RFC4271:

“A NOTIFICATION message is sent when an error condition is detected.
   The BGP connection is closed immediately after it is sent.”

I can’t find any updates that contradict that.
If I understand your suggestion correctly, you mean:

1.    the peer initiating the shutdown (A) sends its peer (B) a NOTIFICATION with a new error code that means “I’m going away shortly, please start re-converging and let me know when you’re done”.

2.    B attempts to re-converge around the paths learnt from A (possibly needing to initiate a route-refresh in the process?), and once it no longer has any of those routes in its FIB sends A back a further NOTIFICATION saying “I’m finished” and then shuts the session down.

3.    If A hasn’t heard back within a configurable timeout, then it yanks the session anyway.

If so, that sounds like a hell of a lot of new protocol spec and router code, to solve a fairly marginal issue. The determinism that you gain only goes one hop into the remote network (as B’s other peers may still be converging when the session goes down), and is therefore only guaranteed to be useful if that network tunnels between ASBRs.
It is also unclear what a BGP speaker that didn’t understand the new code would do when receiving a NOTIFICATION that wasn’t immediately followed by a session reset. I suspect that would vary widely by implementation and be the source of some fairly nasty bugs!
Compared to a line in an RFC suggesting reasonable timer values between gshut and shutdown, this seems to be a lot more work for very little benefit.

Cheers,
Ben Maddison

Director

From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Friday, March 17, 2017 1:30 PM
To: Ben Maddison <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [GROW] draft-ietf-grow-bgp-gshut status?

Hi Ben & Bruno,

If we all agree community approach is the best for signalling to peer that this prefix should be avoided why folks just do not do it today already ? Why do we need IETF draft for that :) ?

If this is about publishing a communities which include "warning" to peers "Hey I am bringing this prefix down soon .. "(maybe even with parameter how soon :))) what stopping Orange to declare we will attach YZ:MN community and re-advertise routes when we are to take down the session.

- - -

The idea behind putting single message in NOTIFICATION was just to automate it and make it deterministic. When you send community some peers may converge quicker some much slower. You never know when your peer is done.

Here only after his best path and rib insertion of new paths around you is complete he would tear the session. Some timeout would be defined as well. And it will apply to all AFI/SAFIs as well as current and future BGP messages send over given session.

Best,
R.

On Fri, Mar 17, 2017 at 12:14 PM, Ben Maddison <[email-masked]<mailto:[email-masked]>> wrote:
Hi Robert,

Cheers,

Ben Maddison

Director

Office:     +27 (0) 21 200 9000<tel:+27%2021%20200%209000>
Mobile:   +27 (0) 82 415 5545<tel:+27%2082%20415%205545>
Email:      [email-masked]<mailto:[email-masked]>
SIP:          [email-masked]<mailto:[email-masked]>

From: GROW [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Robert Raszuk

Hi Bruno,

​Best,
R.​

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 17 Mar 2017 13:33:34 +0100

> ​> ​
> The primary benefit is the use of a well-known community
>
>
​Not that long ago we went via tsunami ​of IDR on and offline emails when
discussing large communities which contained  "operators" voice stating
*NO* to any well known or predefined meaning to the communities nor
welcomed any predefined actions associated with the communities.

Everyone wants to assign his own and inform interested parties about such
meaning. Has that already changed just few weeks after the RFC was issued
:-) ?

1.    the peer initiating the shutdown (A) sends its peer (B) a
> NOTIFICATION with a new error code that means “I’m going away shortly,
> please start re-converging and let me know when you’re done”.
>
> 2.    B attempts to re-converge around the paths learnt from A (possibly
> needing to initiate a route-refresh in the process?), and once it no longer
> has any of those routes in its FIB sends A back a further NOTIFICATION
> saying “I’m finished” and then shuts the session down.
>
​Yes that's good summary. ​

> If so, that sounds like a hell of a lot of new protocol spec
>

I don't think this is that complex. And use of NOTIFICATION message was
just an example. One could also put it in new OPERATIONAL message.

​Anyhow just a suggestion how to improve protocol if there is real need. If
this however as you said "fairly marginal issue" then let's not bother.

Cheers,
R.

---

From: Gert Doering <[email-masked]>
Date: Fri, 17 Mar 2017 13:47:42 +0100

Hi,

On Fri, Mar 17, 2017 at 01:33:34PM +0100, Robert Raszuk wrote:
> > ???> ???
> ???Not that long ago we went via tsunami ???of IDR on and offline emails when

You're taking this out of context.  We disagreed to having a fixed format
and fixed rules what to do with large communities.

Nobody objected to having a set of well-known communities for a well-known
purpose (as can be seen by the consensus on 7999).  But this is not what
was discussed in large community context.

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 17 Mar 2017 13:52:56 +0100

> and fixed rules what to do with large communities.

​Large communities have fixed format. There is no TLV there. ​Which
proposal was more of fixed format then large comms ???

​And no proposal suggested only fixed rules. Wide Communities supported TLV
and local rules exactly as you would do with current large comms. Also they
proposed in parallel to full operator's flexibility set of well known
values as documented here:

https://tools.ietf.org/html/draft-ietf-idr-registered-wide-bgp-communities-02

Thx,
R.

---

From: <[email-masked]>
Date: Fri, 17 Mar 2017 12:53:01 +0000

Hi Robert,

From: Robert Raszuk
Sent: Friday, March 17, 2017 12:30 PM

Hi Ben & Bruno,

[Bruno] The benefit of using a well-known community is that everyone uses the same. Otherwise, the operator shutting down/reloading a node with 50 EBGP sessions would need to attach 50 different communities. And update this list of communities as peer are configured/unconfigured.
- - -

The idea behind putting single message in NOTIFICATION was just to automate it and make it deterministic. When you send community some peers may converge quicker some much slower. You never know when your peer is done.
[Bruno]
- The sending of community may be automated just like the sending of another message.
- Again, whatever type of message you use over the EBGP session, on the receiving peer it needs to be translated in deprefering each route and sending N updates to peers. And yes, some peers may converge quicker than others, but there is nothing you can do about this. A clever implementation could use heuristic to evaluate the BGP convergence of its peer (e.g. by measuring inbound traffic). But typically a single timer is enough, because in all cases, people will not wait/delay their maintenance forever. So if they can afford to wait 5 minutes, they wait 5 minutes and then close the BGP session regardless the speed of their peer.

Best,
R.

Cheers,

Ben Maddison

Director

Hi Bruno,

​Best,
R.​

---

From: Ben Maddison <[email-masked]>
Date: Fri, 17 Mar 2017 12:54:00 +0000

Hi Robert,

Not that long ago we went via tsunami ​of IDR on and offline emails when discussing large communities which contained  "operators" voice stating *NO* to any well known or predefined meaning to the communities nor welcomed any predefined actions associated with the communities.

Everyone wants to assign his own and inform interested parties about such meaning. Has that already changed just few weeks after the RFC was issued :-) ?

I was one of those operators saying *NO*! By definition, well-known communities have a meaning that is not specific to a particular ASN. The lack of encoding space in std-comms for 32bit ASNs is therefore a non-issue. There is certainly operator value in well-known communities, but they don’t need to be recreated in large-comms – unless you think that there are more than 65535 useful meanings that we can all agree on?!

Cheers,

Ben Maddison

Director

From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Friday, March 17, 2017 2:34 PM
To: Ben Maddison <[email-masked]>
Cc: [email-masked]
Subject: Re: [GROW] draft-ietf-grow-bgp-gshut status?

​> ​

​Not that long ago we went via tsunami ​of IDR on and offline emails when discussing large communities which contained  "operators" voice stating *NO* to any well known or predefined meaning to the communities nor welcomed any predefined actions associated with the communities.

Everyone wants to assign his own and inform interested parties about such meaning. Has that already changed just few weeks after the RFC was issued :-) ?

I don't think this is that complex. And use of NOTIFICATION message was just an example. One could also put it in new OPERATIONAL message.

​Anyhow just a suggestion how to improve protocol if there is real need. If this however as you said "fairly marginal issue" then let's not bother.

Cheers,
R.

---

From: <[email-masked]>
Date: Fri, 17 Mar 2017 12:57:19 +0000

From: Robert Raszuk Sent: Friday, March 17, 2017 1:34 PM

​> ​

[Bruno] You are welcome to write such a draft and have it implemented.
Again, draft-ietf-grow-bgp-gshut is using a BGP community for the signaling as it brings compatibility with vanilla routers.

--Bruno

Cheers,
R.

---

From: Gert Doering <[email-masked]>
Date: Fri, 17 Mar 2017 13:59:55 +0100

Hi,

On Fri, Mar 17, 2017 at 01:52:56PM +0100, Robert Raszuk wrote:
> ???Large communities have fixed format. There is no TLV there. ???Which

"fixed format" and "there is no flexibily in *usage*" is not the same thing.

Gert Doering
        -- NetMaster

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 17 Mar 2017 14:08:54 +0100

> [Bruno] The benefit of using a well-known community is that everyone uses
> the same. Otherwise, the operator shutting down/reloading a node with 50
> EBGP sessions would need to attach 50 different communities.
>

​Nope. You send only one .. the one you told your peers that means your
g-shut.

But to respect your peer's g-shuts you may end up matching on a few ... not
that each VPN site will have it's own but your upstreams or peers sure
could.

​In any case as far as RFC 1997 there was big discussion in July and August
of 2008 in IDR started by Dave on how to reserve new values. I do not
recall what was the final outcome of that.

https://mailarchive.ietf.org/arch/msg/idr/U6mR4hST3ZX-Ai5jyRKabhB9n8o

---

From: <[email-masked]>
Date: Fri, 17 Mar 2017 13:28:25 +0000

From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Friday, March 17, 2017 2:09 PM
To: DECRAENE Bruno IMT/OLN
Cc: Ben Maddison; [email-masked]
Subject: Re: [GROW] draft-ietf-grow-bgp-gshut status?

[Bruno] The benefit of using a well-known community is that everyone uses the same. Otherwise, the operator shutting down/reloading a node with 50 EBGP sessions would need to attach 50 different communities.

​Nope. You send only one .. the one you told your peers that means your g-shut.

But to respect your peer's g-shuts you may end up matching on a few ... not that each VPN site will have it's own but your upstreams or peers sure could.

[Bruno2] I hope that in the end, we’ll agree that using a single community for all EBGP sessions is just easier.
But in the worst case, we don’t have to. Registration policy is FCFS and the g-shut community is already allocated.

​In any case as far as RFC 1997 there was big discussion in July and August of 2008 in IDR started by Dave on how to reserve new values. I do not recall what was the final outcome of that.

[Bruno2]  https://www.iana.org/assignments/bgp-well-known-communities/bgp-well-known-communities.xhtml

Range [https://www.iana.org/assignments/_support/sort_none.gif]

Registration Procedures [https://www.iana.org/assignments/_support/sort_none.gif]

0xFFFF0000-0xFFFF8000

First Come First Served

0xFFFF8001-0xFFFFFFFF

Standards Action

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 20 Mar 2017 17:00:21 -0400

> Until now, in order to capture all the options, waiting for draft-ietf-idr-as4octet-extcomm-generic-subtype seemed reasonable (at the cost of waiting for years, but this was not expected as we thou…

I have no issues with a well known RFC 1997 community.

The generic as4-octet could be used as well.  It's just an extended
community code point.

End of the day, it's whatever gets the feature shipped.[1]

-- Jeff

[1] You've been able to do arbitrary extended communities in Junos for a
while.  They don't necessarily display right in show commands, but the
transitivity flag is respected.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Mon, 20 Mar 2017 21:20:36 +0000

I implemented it in IOS-XR.
It is called graceful maintenance, because it works on bringup as well as shutdown.
It can be configured to send LOCAL_PREF or AS_PATH prepends upon activation 
to support neighbors that do not recognize the GSHUT community.

Thanks,
Jakob.

> Sent: Wednesday, March 15, 2017 6:11 AM
> To: [email-masked]
> Subject: [GROW] draft-ietf-grow-bgp-gshut status?
> 
> Hi group,
>