Subject: [Idr] draft-keyur-bgp-enhanced-rotue-refresh-00t.xt

From: "Susan Hares" <[email-masked]>
Date: Thu, 11 Nov 2010 02:02:09 -0500

WARNING: contains banned part

Subject: draft-keyur-bgp-enhanced-rotue-refresh-00t.xt
From: "Susan Hares" <[email-masked]>
To: <[email-masked]>
Cc: "'John Scudder'" <[email-masked]>, <[email-masked]>, <[email-masked]>
Date: Thu, 11 Nov 2010 02:02:09 -0500

The authors request that: 

draft-keyur-bgp-enhanced-route-refresh-00
<http://tools.ietf.org/id/draft-keyur-bgp-enhanced-route-refresh-00.txt> 

  Enhanced Route Refresh Capability for BGP-4
<http://www.ietf.org/proceedings/79/slides/idr-15.pdf> 

is accepted as a working group item.

This starts a two week call for adoption. It will end on 11/25/10. 

John and Sue

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Thu, 11 Nov 2010 12:24:40 +0100

Yes/support

---

From: <[email-masked]>
Date: Thu, 11 Nov 2010 12:44:51 +0100

Support.

________________________________________
From: Susan Hares [mailto:[email-masked]] 
Sent: Thursday, November 11, 2010 8:02 AM
To: [email-masked]
Cc: 'John Scudder'; [email-masked]; [email-masked]
Subject: draft-keyur-bgp-enhanced-rotue-refresh-00t.xt

---

From: "iLya" <[email-masked]>
Date: Thu, 11 Nov 2010 15:15:47 +0100

support

/iLya

--------------------------------------------------
From: "Susan Hares" <[email-masked]>
Sent: Thursday, November 11, 2010 8:02 AM
To: <[email-masked]>
Cc: "'John Scudder'" <[email-masked]>; <[email-masked]>; 
<[email-masked]>
Subject: draft-keyur-bgp-enhanced-rotue-refresh-00t.xt

---

From: zengqing <[email-masked]>
Date: Mon, 15 Nov 2010 14:38:16 +0800

Support for adoption.

Thanks,
Qing

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Susan Hares
Sent: Thursday, November 11, 2010 3:02 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: [Idr] draft-keyur-bgp-enhanced-rotue-refresh-00t.xt

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 23 Nov 2010 15:16:00 +0100

Hi,

While we are discussing various ways to protect bgp speakers from 
impacting all prefixes at the event of encoding errors in attributes or 
update messages due to implementation issues the below new functionality 
of the enhanced route refresh does trigger one question.

The draft says:

    All the routes that were not re-advertised in the route
    refresh MUST be purged, and SHOULD be logged for further
    analysis.

That really means that if my END_OF_ROUTE_REFRESH is send mistakenly and 
prematurely before really all bgp prefixes have been sent the receiving 
speaker must delete all unreceived routes from their BGP tables, RIB and 
FIB.

That is very service impacting perhaps maybe without a good reason.

Therefor I would recommend to replace "MUST be purged" with "MAY be 
purged" giving operator a chance to diagnose the issue first without 
impacting the actual routing. Logging itself should be enough of the 
warning.

Many thx,
R.

---

From: Jakob Heitz <[email-masked]>
Date: Tue, 23 Nov 2010 12:22:50 -0500

Then don't send it mistakenly. 

--
Jakob Heitz. 510-566-2901

On Nov 23, 2010, at 6:16 AM, "Robert Raszuk" <[email-masked]> wrote:

> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

---

From: Jakob Heitz <[email-masked]>
Date: Tue, 23 Nov 2010 13:46:53 -0500

If you don't delete them, you keep black holes.

BGP is a high service impacting protocol.

Perhaps it's time to standardize a test suite for BGP

--
Jakob Heitz. x25475. 510-566-2901

> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Jakob Heitz
> Sent: Tuesday, November 23, 2010 9:23 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [Idr] draft-keyur-bgp-enhanced-rotue-refresh-00.txt
>

---

From: John Scudder <[email-masked]>
Date: Tue, 23 Nov 2010 11:31:36 -0800

Robert,

I think it really depends on the use case.  If the route refresh is solicited as part of a manual troubleshooting process, then I see your point.  If the route refresh is solicited as a side-effect of the normal operation of the router (for example due to a change in an import policy) then I think the text is best as written.  One way of capturing this would be to change the MUST to a SHOULD, with an explanation that the automatic purge might not be done in a manual troubleshooting situation.

Regards,

--John

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 23 Nov 2010 21:07:11 +0100

Hi John,

 > I think it really depends on the use case.

It really does indeed.

 > If the route refresh is solicited as a side-effect of
 > the normal operation of the router (for example due to a change in an
 > import policy) then I think the text is best as written.

I would not agree. If purging wrong database entries would take place in 
normal operation finding the root cause of those entries to be there in 
the first place will be very hard if not impossible. That will result in 
much less ability to fix the real issue. Postmortem logging will not 
help much I am afraid in this case.

However I think the draft itself brings good value to synchronize 
databases once the cause of de-sync has been determined or just like in 
IGPs as a tool for very periodic LSDB full synchronization.

That's why I would recommend to make the proposed refresh with purge not 
a default behaviour.

Many thx,
R.

> Robert,
>
> I think it really depends on the use case.  If the route refresh is
> solicited as part of a manual troubleshooting process, then I see
> your point.  If the route refresh is solicited as a side-effect of
> the normal operation of the router (for example due to a change in an
> import policy) then I think the text is best as written.  One way of
> capturing this would be to change the MUST to a SHOULD, with an
> explanation that the automatic purge might not be done in a manual
> troubleshooting situation.
>
> Regards,
>
> --John
>
>> impacting all prefixes at the event of encoding errors in
>> attributes or update messages due to implementation issues the
>> below new functionality of the enhanced route refresh does trigger
>> one question.
>>
>> The draft says:
>>
>> All the routes that were not re-advertised in the route refresh
>> MUST be purged, and SHOULD be logged for further analysis.
>>
>> That really means that if my END_OF_ROUTE_REFRESH is send
>> mistakenly and prematurely before really all bgp prefixes have been
>> sent the receiving speaker must delete all unreceived routes from
>> their BGP tables, RIB and FIB.
>>
>> Therefor I would recommend to replace "MUST be purged" with "MAY
>> be purged" giving operator a chance to diagnose the issue first
>> without impacting the actual routing. Logging itself should be
>> enough of the warning.
>>
>> Many thx, R.
>>
>> _______________________________________________ Idr mailing list
>> [email-masked] https://www.ietf.org/mailman/listinfo/idr
>
>

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 23 Nov 2010 21:16:53 +0100

Hi Jakob,

 > Then don't send it mistakenly.

If you would not have mistakes in the protocol operation we would never 
need to check for possible missing withdraws or missing updates as 
stated in section 1 of the draft.

> BGP is a high service impacting protocol.

Indeed .. couldn't agree more. That is why I think we should not mask 
the problems but try hard to find the cause of those and fix them if 
they appear. Resync of the databases may be useful after that.

To be perhaps a bit more descriptive ... instead of making the holes and 
finding a to fill them we should not make the holes in the first place.

> Perhaps it's time to standardize a test suite for BGP

Haven't I just attempted to start this effort ?

http://tools.ietf.org/html/draft-raszuk-bgp-diagnostic-message-00

If you think we need to add more tlv-s or modify those proposed you are 
more then happy to propose them.

Cheers,
R.

---

From: John Scudder <[email-masked]>
Date: Tue, 23 Nov 2010 12:48:11 -0800

Robert,

On Nov 23, 2010, at 3:07 PM, Robert Raszuk wrote:
...
> help much I am afraid in this case.

Is your point that maybe buggy implementations will get out of sync but will inadvertently get re-synced as the result of an enhanced route refresh that is triggered for some unrelated reason, and that instead of having the out-of-sync routers get re-synced, you think it is important for them to stay out-of-sync (that is to say, broken) until operations happens to notice and manually intervene?

> IGPs as a tool for very periodic LSDB full synchronization.

Of course, in the IGPs there is no requirement for manual intervention.  If two IS-IS speakers exchange CSNPs and notice a discrepancy, they automatically fix it, they do not wait for an operator to tell them it is OK to resume correct operation.

> That's why I would recommend to make the proposed refresh with purge not 

Actually, if the MUST is weakened to a SHOULD or less, the draft will also need to say that an implementation sending an enhanced route refresh MUST send withdraws for deleted routes even if the routes have been implicitly withdrawn by not being included in the enhanced route refresh.  This is a not-completely-trivial additional requirement.  One would have to consider whether the benefit justifies it.  For my own part, I'm back to thinking that MUST was right.

Regards,

--John

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 23 Nov 2010 22:22:07 +0100

Hi John,

> Is your point that maybe buggy implementations will get out of sync
> but will inadvertently get re-synced as the result of an enhanced
> route refresh that is triggered for some unrelated reason, and that
> instead of having the out-of-sync routers get re-synced,

Yes.

> you think it
> is important for them to stay out-of-sync (that is to say, broken)
> until operations happens to notice and manually intervene?

Nope .. I think it is important to detect the desync for example by 
periodically comparing the database counters on both BGP speakers, run 
set of event scripts to log as much as possible details then trigger the 
enhanced refresh to clear the problem.

>> That's why I would recommend to make the proposed refresh with
>> purge not a default behaviour.
>
> Actually, if the MUST is weakened to a SHOULD or less, the draft will
> also need to say that an implementation sending an enhanced route
> refresh MUST send withdraws for deleted routes even if the routes
> have been implicitly withdrawn by not being included in the enhanced
> route refresh.  This is a not-completely-trivial additional
> requirement.  One would have to consider whether the benefit
> justifies it.  For my own part, I'm back to thinking that MUST was
> right.

Are you saying that sender of enhanced route refresh may no longer send 
explicit withdraws for some prefixes which are to be removed if withdraw 
for them has been received in the window of START_REFRESH and END_REFRESH ?

That means that you are assuming the dynamic modifications to be 
happening to content which you are sending as part and during enhanced 
refresh reply which may also be not-completely-trivial I am afraid.

Best regards,
R.

---

From: John Scudder <[email-masked]>
Date: Tue, 23 Nov 2010 13:44:20 -0800

On Nov 23, 2010, at 4:22 PM, Robert Raszuk wrote:
...
> for them has been received in the window of START_REFRESH and END_REFRESH ?

Depends on how I bind the "may" in "may no longer send" in your sentence :-)  I'm saying that the sender of the ERR wouldn't *have to* send explicit withdraws.  It still *could*, if the implementor so chose.

> refresh reply which may also be not-completely-trivial I am afraid.

I don't follow your point. 

--John

---

From: Jie Dong <[email-masked]>
Date: Wed, 24 Nov 2010 14:34:17 +0800

> help much I am afraid in this case.

Agree in some cases the root cause needs to be found out before some re-sync
operation, thus some trouble shooting mechanisms would be helpful.

Best Regards,
Jie

---

From: Jie Dong <[email-masked]>
Date: Wed, 24 Nov 2010 14:45:08 +0800

> BGP is a high service impacting protocol.

Yes, and the problem which caused the de-sync needs to be fixed if possible.
Otherwise you may have some other inconsistent routes later.

> Perhaps it's time to standardize a test suite for BGP

Agreed. And One-time ORF could be used as one of the test/debugging tools
for BGP. 

Best regards,
Jie

> > attributes or
> > mistakenly and

---

From: <[email-masked]>
Date: Wed, 24 Nov 2010 11:56:24 +0100

Hi,

> From: Robert Raszuk

+1, and Robert thanks for the clarifying question and initiating the
debate.

so what about 2 tools:
- enhanced-route-refresh for protocol consistency enforcement
- a debug / diag tool (which eventually may not require sending all
routes to check consistency)

>  > import policy) then I think the text is best as written.

+1

For the protocol consistency tool (enhanced-route-refresh) IMHO the
default behavior should be to purge the route. Robert, I agree that this
could be service impacting (hopefully in limited cases) but we're in a
situation which is already service impacting (some routes are wrong).

At least, the spec should make crystal clear to the sender of the
ROUTE-REFRESH-END that the received is expected to remove all the non
refreshed routes. As such the sender should be careful before sending
ROUTE-REFRESH-END.

As a side comment, I'm not fan of the current following wording " Once
the [timer] upper bound is reached, the implementation MAY send a
ROUTE-REFRESH message with the specified message subtype marking End of
the Route Refresh. " IMHO, it should be made clear that the sender MUST
NOT send the ROUTE-REFRESH-END before it has re-advertised all the
routes (at least those which were valid when ROUTE-REFRESH-BEGIN was
sent).

Regarding the MUST/SHOULD/MAY be purged, I would favor a SHOULD as for
the sender of the ROUTE-REFRESH-END it should be considered the default
behavior, while for the receiver, in the end, it's up to him to decide.

" The BGP speaker SHALL use the demarcations of
   the beginning and the ending of a route refresh to perform
   consistency validations of the updates received from the peer.  All
   the routes that were not re-advertised in the route refresh MUST be
   purged, and SHOULD be logged for further analysis. "

SHALL use / MUST be purged. IMO the mandatory behavior is not crystal
clear: it's a MUST purge but if I don't want to purge, I could still
avoid the MUST by claiming not to use the RR demarcations? :-)

----
As an additional comment on the draft, I would propose the following
change:

OLD: " All the routes that were not re-advertised in the route refresh
MUST be
   purged, and SHOULD be logged for further analysis. "

NEW: " All the routes received before the reception of the
ROUTE-REFRESH-BEGIN message and that were not re-advertised in the route
refresh SHOULD be
   purged, and SHOULD be logged for further analysis. "

Thanks,
Bruno

---

From: Jie Dong <[email-masked]>
Date: Wed, 24 Nov 2010 19:36:38 +0800

Hi,

> routes to check consistency)

Agreed.

> routes (at least those which were valid when ROUTE-REFRESH-BEGIN was

Besides the description of the sender side, I would also recommend some
explanation of the receiver side operation, e.g. whether a timer on the
receiver could be used for cases where end-of-refresh is still not received
a long enough time after the receipt of start-of-refresh. And how should the
receiver operate if such timer times out with the end-of-refresh still not
received. IMO this case can be regarded as an incomplete consistency check
and any route should not be purged.

Best regards,
Jie

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 24 Nov 2010 15:14:19 +0100

Hi Bruno,

> routes to check consistency)

I do agree. Once updated I will republish the Diagnostic Message draft 
and then we can verify on the list if this is eligible to be used as 
debug/diag tool.

> situation which is already service impacting (some routes are wrong).

Well I am not that worried about service impact due to the fact that as 
you point out there is problem already impacting service. I am trying to 
picture the sequence of events which are to happen ...

I am worried about few points:

* If Enhanced Route Refresh capability is negotiated - for example being 
the default ON - you will loose ability to even execute couple of output 
command capture scripts in the WRONG state before the state is cleaned up.

* Errors of premature in the presence of continued churn sending of 
END_OF_REFRESH does bring some worry, but even bigger worry would be how 
to repair from it. To send another Enhanced refresh to make sure ?

* The draft is very unclear to confirm or deny that once I negotiate 
enhanced route refresh capability would operator have a way to send 
normal route refresh request without purge any longer. I have asked 
authors to clarify that point.

* Last the point John mentioned that some implementation may choose not 
to send normal withdraws once it makes sure that in the enhanced route 
refresh those nets/paths are missing. I think the draft should discuss 
this case/option explicitly. So far it remains sort of implementation's 
discretion.

Best rgds,
R.

---

From: <[email-masked]>
Date: Wed, 24 Nov 2010 15:18:49 +0100

Hi Keyur, Enke, Balaji, all,

Please find below some proposed additional comments on the draft

4. Error Handling

1)"   When the BGP speaker detects an error while processing a ROUTE-
   REFRESH message with a non-zero "Message Subtype" field, it MUST send
   a NOTIFICATION message with Error Code "ROUTE-REFRESH Message
Error"."

In the light of recent discussions and proposals about avoiding shutting
down a BGP session if this can be avoided, I'm wondering what are the
motivations for sending a NOTIFICATION. If the error is recoverable
(which IMHO is the case for the 2 examples provided), what about simply
logging the issue and ignoring the message?

2) What about specifying the expected & allowed behavior in corner
cases. e.g.:
- is it allowed to send a (new) ROUTE-REFRESH message if following a
previously sent ROUTE-REFRESH, I received a ROUTE-REFRESH-BEGIN but no
ROUTE-REFRESH-END?

- what if two ROUTE-REFRESH-BEGIN are received? Should the first be
ignored? Or the second? (in particular if " In the case of continuous
churn of the BGP table," ROUTE-REFRESH-END is never sent)

- what if I receive a ROUTE-REFRESH-END with no preceding
ROUTE-REFRESH-BEGIN
...

3) " In the case of continuous churn of the BGP table, an implementation
   SHALL impose an upper bound on how long it would take to complete the
   route refresh."

What about a MAY? Indeed setting an upper bound is desirable, but I fear
that everybody will no agree on the solution / cost to achieve this.

And although this could be seen as a local matter, what about adding a
MAY for a timer on the receiving side? (i.e. +1 on Jie's comment)

Thanks,
Regards,
Bruno

---

From: John Leslie <[email-masked]>
Date: Wed, 24 Nov 2010 13:13:39 -0500

> This starts a two week call for adoption. It will end on 11/25/10. 

   I must admit to some reservations about proceeding this way in
extending route-refresh.

   Plain-Old Route-Refresh [RFC 2918] is designed to avoid storing
all routes-as-advertised from a peer. (I haven't paid much attention
to it, being uninterested in programming routers with "insufficent"
memory.)

   The Introduction of draft-keyur... speaks of "consistency
validations"; so I assume that is what drives this work. (I must
admit I don't worry about these much, and when forced to worry I
generally try for a graceful restart -- hardly an elegant solution.)
It might help for some of us to give examples of consistency issues
that would lead to using this tool...

   Regardless, I'd rather see the re-advertisement(s) be out-of-band
from the usual UPDATEs; and if we're enhancing, I'd like to be able
to request smaller blocks of re-advertisement.

   I'd also like to make the out-of-band (re)advertisements capable
of things which have become interesting in the last ten years, such
as alternate routes; and I'd dearly love for (re)advertisements to
be capable of end-running things which now call for session-reset
(or at least "treat-as-withdraw"). This really _does_ seem achievable
to me.

   I'd welcome questions on any of the stuff I'm asking for: but the
"end-run session-reset" is such an obvious question that I'll treat
it briefly in this email:

   The proposal currently suggesting "treat-as-withdraw" instead of
session-reset calls for blindly withdrawing "junk" NLRI. I'd like
to be able to respond (instead) with a "request-readvertisement
because I couldn't make sense of this" flavor of route-refresh,
listing only the CIDR block which I would otherwise treat-as-withdraw.
The offending peer could reasonably re-advertise in simpler form. In
the meantime, I know the peer has claimed to be able to reach that
block, so I needn't discard the route if I have none other; but I'd
want to prefer an alternative "likely" to be more trustworthy; and
of course I'd want to "eventually" disincent the "bad" route more...

   While facing the "junk" route, I know I can't be sure of actual
AS_PATH length, or even whether a routing loop may exist; so using
it entails "some" risk -- it might even prove necessary to rate-limit
traffic using it, though that is an implementation decision, not a
protocol issue.

   If asked to re-advertise a "junk" route, it would be reasonable
to advertise it at a disincentive to reflect "plausible" guesses
at what my peer couldn't decode. Alternatively, I could advertise
a withdraw to say I can't be bothered. So long as these are "out-of-
band" of the usual UPDATEs, there need be no confusion.

   In any case, passing the "I couldn't make sense of this" message
to the maintainers of the code for your router would be an appropriate
way to start resolving the actual issue. (Actually, passing it to
_both_ sender code-maintainers and receiver code-maintainers would
help, though IMHO "be conservative in what you send" works better
than "be liberal in what you accept"...)

--
John Leslie <[email-masked]>

---

From: Paul Jakma <[email-masked]>
Date: Thu, 25 Nov 2010 12:43:17 +0000 (GMT)

On Tue, 23 Nov 2010, John Scudder wrote:

> the normal operation of the router (for example due to a change in 
> an import policy) then I think the text is best as written.  One 
> way of capturing this would be to change the MUST to a SHOULD, with 
> an explanation that the automatic purge might not be done in a 
> manual troubleshooting situation.

TBH, I think it should be a MAY. Detecting unsent prefixes will 
require significant extra state or significant extra work to be done.

Either:

- at start of RIB, a list is made of unsent prefixes, initialised by
   scanning the current RIB, prefixes ticked off as they arrive

or

- at start of RIB, some temporally ordered marker is noted, as each
   prefix is received its marked with same, at end RIB is scanned for
   routes which lack the mark or whose mark orders earlier than the
   RIB scan.

That's a lot to ask implementations to do at the arbitrary whim of 
neighbours (e.g. because some neighbour's admin is live-debugging 
their policies and regularly sending requests).

Whether to do this work should be completely at the discretion of the 
receiving speaker.

regards,
-- 
Paul Jakma	[email-masked]	@pjakma	Key ID: 64A2FF6A
Fortune:
You teach best what you most need to learn.

---

From: Enke Chen <[email-masked]>
Date: Thu, 25 Nov 2010 09:48:58 -0800

Paul,

"MAY" is not what we need nor want.  Remember that the goal of the draft 
is to provide a re-sync mechanism in BGP, and it won't be accomplished 
if the receiver did not purge the stale entries.

Please find more comments below.

> their policies and regularly sending requests).

What you have described is only one way to implement the draft.  There 
are other ways that can be near-zero cost.

> Whether to do this work should be completely at the discretion of the 

Remember that is is already controlled by a capability.

Regards,    -- Enke

---

From: Paul Jakma <[email-masked]>
Date: Thu, 25 Nov 2010 21:49:42 +0000 (GMT)

On Thu, 25 Nov 2010, Enke Chen wrote:

> "MAY" is not what we need nor want.  Remember that the goal of the 
> draft is to provide a re-sync mechanism in BGP, and it won't be 
> accomplished if the receiver did not purge the stale entries.

If the incoming route was originally requested by that speaker (i.e.
it had sent a route-refresh request), then yes fair enough on SHOULD.
The thing is that the draft also specifies "locally initiated" - i.e.
the route-sender unilaterally decides to push this work.

As the draft stands there is no way for a way to speaker to signal it
wants the enhanced RR but does not intend to commit itself to having
extra pushed onto it arbitrarily.

> Remember that is is already controlled by a capability.

Sure, but implementations might want to follow this draft only where 
they requested the incoming dump!

regards,
-- 
Paul Jakma	[email-masked]	@pjakma	Key ID: 64A2FF6A
Fortune:
"OK, so ten out of ten for style, but minus several million
for good thinking, yeah? "