Subject: Re: [Idr] draft-scholl-idr-advisory-00

From: "Elmar K. Bins" <[email-masked]>
Date: Wed, 25 Mar 2009 13:11:40 +0100

(Sorry broke the headers on the first version...)

[email-masked] (Neil J. McRae) wrote:

> Im sure its that useful  Dave but BGP is a routing protocol and
> I think by bending it into this shape we just put most risk into
> routing infrastructure. Personally I'd prefer not to do that.

I concur. Operational sloppiness should not be turned into the
extension of a protocol that serves a defined signalling purpose.

While it seems a nice idea being able to "chat" via BGP, it
clutters the protocol with non BGP-related information _and_
adds another communication channel that operators need to check
(unless the advisory message becomes mandatory attribute to
every withdrawal or session shutdown; and then what happens
when the peer simply goes away? Another channel to check...)

The proposal says that this message "can speed up...by providing
a direct correlation..." Well, this also means that all router
gear has to store these messages. Typical implementation of this
would be to syslog and forget them (memory is limited). The
typical operator will then have that message mailed to her, and
voilà - either you're in our out of the loop, but an additional
BGP attribute/message doesn't change that.

I believe the proposal should be rejected due to not solving
the problems inherent in operating networks. It should rather
be used as a starting point for thinking about some better way
for operators to communicate amongst themselves and towards
interested parties where necessary. (Don't give your password
to the press)

I personally would like operators to use PeeringDB more, and
implement some kind of "this location is down due to..." there.

Yours,
	Elmar.

---

From: "SCHOLL, TOM (ATTLABS)" <[email-masked]>
Date: Wed, 25 Mar 2009 10:55:44 -0400

> extension of a protocol that serves a defined signalling purpose.

What if mail servers are offline/unreachable at either end? What if the
IX operations mailing list is down?

> when the peer simply goes away? Another channel to check...)

For some parties, checking the router is the first place to check. When
you notice traffic change on graphs, some individuals go to check on the
router to find out why. When customers complain about a path change from
observing traceroutes, someone is going to login to a router to
determine why they're not going through the previous path.

> implement some kind of "this location is down due to..." there.

What if PeeringDB has a disk failure or is unreachable?

---

From: "Elmar K. Bins" <[email-masked]>
Date: Wed, 25 Mar 2009 16:12:54 +0100

Re Tom,

> IX operations mailing list is down?

What if the syslog snippet has been mailed to you, but your mail
server is offline/unreachable? What if your logging buffer on
the router is only 4/8/16/...KBytes and that message has scrolled
off?

(Meaning: Where's the difference?)

> determine why they're not going through the previous path.

If my alerting system hasn't already told me which BGP session has
gone down, and if my mailbox doesn't hold anything from the peer
in question, I'll check the router. But: If my NOC works ok/good,
they (or I) know about the failure due to advance notice. If my
NOC doesn't work ok/good, they will not bother checking the router.

So your point is: If I for whatever reason missed the email, _AND_
this is a scheduled outage, _THEN_ the message would help me.
There will be no improvement if this is _not_ a scheduled outage
or if I read my emails (provided the other party can write that
warning email, which is pretty likely if it's a scheduled outage).

Well, as I wrote before: Usually, logging buffers on routers are
kept (too) small and have to contain (too) many messages, so
practicality demands that this message too should go through
syslog and the usual filtering/alerting process. If your alerting
system (usually it is email or email-to-sms, if we're truthful about
it) is down, you won't get either message.

Were this feature implemented in a way that would

  - store the (last) message in the BGP neighbour's info
  - display that message in "show * neighbor xxx" output

_then_ this would be helpful. Otherwise there would be no
improvement. Basing a protocol upon implementation specifics,
because it would be useless if implemented in another way...?

See, I am lazy as well, so if I could just say

"router bgp xxxx
 neighbor * message ---Scheduled outage from 1500 to 1730---"

and I could be certain my peers knew what was about to happen,
I'd like that, too. Still I prefer to write an outage email if
I expect multiple session drops or an extended outage.

Then there's the matter of security issues in implementation and
usage (abuse, buffer overflows, buggy implementation, ...)

> What if PeeringDB has a disk failure or is unreachable?

What if it snows in summer?
It can happen. In that case you're screwed (or rather: uninformed),
just like if you didn't see the message scrolling by in your syslog
or you broke your alerting system.

Honestly, getting operational messages across is a pretty good idea,
yet I don't see this in a standardised routing protocol.

Yours,
	Elmi.

PS: Anyone thought of character sets and encoding? Anyone want to
    go Kanji or chinese script about this?

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 25 Mar 2009 08:29:53 -0700

Tom,

How big buffer of those messages you would like to store at the router ?

When you are logged in you can browse syslog which is most likely at 
NOC. It is just an illusion that the info is local while it is 
physically far away or worse distributed in chunks across 100+ hard drives.

Let's just don't fall into an old grandmamma's fear that if the money 
are not in her purse - deep in the pocket then no matter how much she 
has in her bank account she feels broke & poor.

In fact contrary I think that just keep such info locally and not 
sharing it (at least) intra-domain is an issue .. perhaps your colleague 
from other NOC in Asia is troubleshooting a problem and while logged 
into ASBR in his jurisdiction wants to grep by given timestamp all 
syslog messages. If the info is just local it will not be visible.

Cheers,
R.

> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr
> 
>

---

From: "SCHOLL, TOM (ATTLABS)" <[email-masked]>
Date: Wed, 25 Mar 2009 11:35:35 -0400

> because it would be useless if implemented in another way...?

In the slides I presented, initial feedback included a request to have
the equivalent of a "sticky" message. This would transmit to a neighbor
and would be visible via "show bgp nei <ip>" output.

Are you saying that you'd agree with this functionality?

---

From: "Elmar K. Bins" <[email-masked]>
Date: Wed, 25 Mar 2009 17:48:20 +0100

Re Tom,

> and would be visible via "show bgp nei <ip>" output.

Honestly, if you think this way then this should be in your proposal.
And then, still, I'd object because

  - this isn't routing info being transmitted (anyone want to
    introduce the like in OSPF or ISIS?) and thus doesn't belong
    in BGP transmissions

  - it doesn't really help (because if my peer's router crashes or one
    of either ports at the exchange just goes down, I cannot deduce any
    information from the non-existence of such message) unless I already
    should have been warned through different - SOP - channels (meaning
    prearranged and planned outage), thus is just cluttering BGP.

> Are you saying that you'd agree with this functionality?

It would be the only way for the idea to make sense and thus would need
to be in the proposal. Then you base everything on implementation specifics.

Can you really not think of a better way to communicate planned outages
to your peers? Is this the last straw?

Yours,
	Elmar.

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 25 Mar 2009 10:00:36 -0700

Hi Elmar,

>     introduce the like in OSPF or ISIS?) and thus doesn't belong

Actually on this you are not fully right. The amount of non routing 
information carried in IGPs today is huge :) Look at the mpls traffic 
eng alone ...

Perhaps let me suggest analogy to this solution from the IGP world. In 
IGP just for reuse of good IGP flooding there is OSPF transport instance 
defined draft-ietf-ospf-transport-instance They came with this idea too 
late and have already bunch of non routing stuff carrier in the routing 
instance.

Perhaps let's just do things right in BGP to start with. Why not to 
define a BGP-CI (communication instance) and use it for exchange of such 
advisory information without any impact to routing ?

 From operator's point of view if the need is so strong (in which I 
don't fully believe) it would be fully transparent. For implementations 
as well as for mitigation of the impact much safer yet still allowing 
the very same functionality (or in fact more :) as proposed originally.

Comments ?
R.

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 25 Mar 2009 10:38:49 -0700

Hi Marco,

>> define a BGP-CI (communication instance) and use it for exchange of
>> such advisory information without any impact to routing ?
> 
> Are you suggesting another parallel connection between neighbors
> designed specifically to only carry communication information? Wouldn't
> one want to consider the scaling issues with this approach? 

* Those sessions could be on demand and short lived.

* The timers responsible for communication protocol can be much less 
aggressive then for a routing protocol instance.

* The amount of information would be (hopefully) much smaller then BGP 
table for the sigma of SAFIs today, but I don't this this would be a 
concern as long as we run it separate.

* The process responsible for handling this BGP instance can run at much 
lower CPU priority as compared with the main BGP. Sure both will end up 
using TCP sessions but all above that can be separated. In fact one 
could run it on totally different core of the RP CPU or on different RE.

* Equally it could be a new instance of BGP as it could be just a TCP 
SSL connection most of today's implementations support it today already.

The point is keep it separate from routing and this point is met if we 
would go that path :-)

Cheers,
R.

---

From: Jeff S Wheeler <[email-masked]>
Date: Wed, 25 Mar 2009 14:18:25 -0400

> could run it on totally different core of the RP CPU or on different RE.
Now you want to add more complexity by running a separate BGP session,
suggesting that vendors implement it using a separate process context,
and put it on another CPU core?  I don't know if Force10 can maintain
their aggressive margins if they have to upgrade their platforms from
three CPUs to four.  Perhaps you should solicit more vendor input.

> SSL connection most of today's implementations support it today already.
Are we worried about AMS-IX sniffing the content of the messages and
learning about your secret pending maintenance window, even though you
are advertising it to your BGP neighbors?  Did you intend to put some
secret information in the message, like "please change BGP MD5 to 123456
after session goes down?"  Or maybe you were going to include your
American Express number in case the same person who runs the routers
also bills customers?

> The point is keep it separate from routing and this point is met if we 
I thought you were concerned about adding complexity to the router
software, but your suggestions above seem contrary to that notion.

> Phase II would be to worry about how to distribute those information 
> inter-as. I will still argue that they should not mixed with routing, 
> but I see a need for other ways and the pendulum of choices is wide and 
> I think we could satisfy the need without exposing any risk on current 
> BGP4 as a routing protocol.
Having read your six posts on this topic in the past 12 hours, I am
totally confused about your position.  I won't be surprised if your next
post is a BGP5 draft that requires operators to swap one-time-pads for
securing the protocol messages and sends SMS notices when the OTP data
is nearing exhaustion.  But don't add any simple conveniences to the
protocol, because if something is going to be useful, it should be hard.

In short, my own position on Tom's proposal is that it's a great idea.
I think I said something along the lines of, "why didn't someone invent
this years ago?"  I love the idea that vendors could choose to store a
bit of "sticky" information for display in `show bgp neighbor,` even if
the session was down.  And the very thought of an eBGP neighbor
communicating their maintenance advisory to my syslog in haiku brings
joy to my heart.  My only disappointment is that the proposed buffer
sizes and probable lack of newline characters in the eventual standard
will not allow for ASCII-art, as [email-masked] is a frequent source of
joy when searching the IRR databases.

Kind regards,
-- 
Jeff S Wheeler <[email-masked]> +1-212-981-0607
Sr Network Operator  /  Innovative Network Concepts

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 25 Mar 2009 11:43:14 -0700

Jeff,

> Now you want to add more complexity by running a separate BGP session,

What complexity you are talking about ? It can be as transparent for you 
CLI wise as enabling the new BGP message the original draft talks about.

> three CPUs to four.  Perhaps you should solicit more vendor input.

That's an option not a requirement. If you keep routing and 
informational warnings/automated events together you are loosing that 
option.

> American Express number in case the same person who runs the routers

No .. I was going to keep your customers from getting free text spam.

In the IX case it can be quite likely that some peers want to spoof 
messages and make their business enemy to wake up at 4:00 am Sunday 
morning. If you are proposing to have such massages in plane text on the 
wire, no option for MD5, no option for GTSM, no option for any 
protection you are welcome.

> software, but your suggestions above seem contrary to that notion.

Nope. This is not a forum to discuss any implementation complexity in 
the router.

I was and still are concerned about repeating mistakes made in the past 
of mixing routing and non-routing information in the same protocol just 
because it is there already and it's distribution nature scales.

I am not saying let's forget about the idea all together. All I am 
saying is that achieving your goal of seeing this extra bit of history 
info in "sh bgp neighbor" display can be easily accomplished in dozens 
of ways - many of them with no risk/impact to routing at all.

R.

---

From: Jeff S Wheeler <[email-masked]>
Date: Wed, 25 Mar 2009 16:40:25 -0400

> protection you are welcome.
Did I miss the slide in Tom's presentation that says, "advisory message
will be excluded from TCP MD5?"  Oh, no, there was no such slide.  You
just don't know how BGP with MD5 works.  I suggest you read RFC2385.

> Nope. This is not a forum to discuss any implementation complexity in 
Really?  Because someone suggested separating the advisory messages onto
an independent session run in an isolated process context with a low
scheduler priority on a CPU core distinct from the one running BGP.
That sounds like a lot of implementation complexity to me.  Let's use
The Google to find out whose idea that was ...
   results:
> could run it on totally different core of the RP CPU or on different RE.
Oh, I see, it was your doppelganger.

> because it is there already and it's distribution nature scales.
You have already incorrectly identified traffic engineering information
as irrelevant to routing.  I see nothing intrinsically wrong with
extending an existing protocol or signaling mechanism to carry new
information.  Calling Party Name isn't needed to complete telephone
calls, but we have that on top of SS7.  ANI isn't even needed, but
again, it made sense to include it there.  I doubt that Jon Postel
envisioned DNS being used as a database of spam sources or open proxies.

> of ways - many of them with no risk/impact to routing at all.
Now you think it's valuable to have the information displayed via the
`show bgp neighbor` output, but think there are better ways than by
exchanging the information via the BGP session?  Do you suggest the
router learn to receive, as well as send, email?  What happened to your
conviction that generating email to noc@ was a better idea?

Kind regards,

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 25 Mar 2009 13:47:45 -0700

> exchanging the information via the BGP session?  

Correct.

> Do you suggest the router learn to receive, as well as send, email?  

No.

> What happened to your

It is still the same idea. It just requires an open mind to grasp it 
apparently. Router generates notifications out-of-band of routing 
information, those end up in syslogs of the peers, routers can display 
the info just by retrieving it from syslog.

Done.

Cheers,
R.

---

From: john heasley <[email-masked]>
Date: Wed, 25 Mar 2009 22:04:55 +0000

> to your peers? Is this the last straw?

I agree with Elmar and Seely; this proposal seems vastly silly.  This is not
routing information, it does not belong in the protocol.  If a protocol-
supported identification of planned maintenance is truely necessary,
add a notification(s) more explicit than 'cease'.  Such as 'administrative
cease/shutdown', 'reboot', and so on.  Note that notifications can also
carry optional "data", such as a string:
	router#reload in 1:00 load new ios

And, build some redundancy into your MTAs, or whatever your peer-communication
channels are, if you're concerned that it may be unavailable when a maint.
notification is sent.  And, if you have so much trouble with your peers
communicating planned maintenance information to you, stop peering with them
on the basis of incompetence.

---

From: "Elmar K. Bins" <[email-masked]>
Date: Thu, 26 Mar 2009 01:54:43 +0100

(Yeah, go out for a couple rounds of badminton and people start real heat...)

I have a couple of annotations and one or two questions...

> the info just by retrieving it from syslog.

I would actually prefer the info in the "noc view" of the network. My router
might be unreachable if things get really hairy. And this "noc view" need
not pull that info from some BGP stream on some remote router.

I agree (maybe that wasn't so clear in previous emails) with the general
idea of having a -- ideally standardised -- solution for passing operational
info to my peers and other parties with a valid interest (exchanges, customers,
transits, the press...) Yet, transporting this kind of info is not the job
of the routing protocol itself. I am convinced that this should happen
out-of-band wrt/BGP. This would give the benefit of being able to communicate
an incident even if this doesn't work over that peering anymore ;-)

So, yes, Jeff, we'd all be happy had someone invented this years ago, but
would we really want it inside the routing protocol? What about communicating
available downstream bandwidth to the n-th degree inside named protocol? What
about maintainance windows of customers? Discussions about birthday presents?

I agree, something like Tom envisioned is definitely needed. It should be
drafted and then implemented, and that needs to be done outside IDR, in a
group that's more operations-focused, but at the same time not limited to
the BGP peering view. In that way, not only this one special problem can
be solved, but a lot more could be achieved.

Btw, I also agree with you, Jeff, that a little bit of fun can easen the
pain of a big peer going down for maintainance. I'd love the Haiku bit.
Just not in the router ;-)

To David I have a probably silly question:

[email-masked] (David Freedman) wrote:

> >If a protocol-
> >supported identification of planned maintenance is truely necessary
>
> Can we please stop limiting the scope of this to simply conveying planned maintenance announcement
> I want a non-terminal NOTIFICATION to be carried INSIDE BGP, that is my main care here.
> Give me a nonterminal NOTIFICATION and I'll be happy.

Maybe my grasp of english is not sufficient here, or I just don't get the
term, but what do you mean by "non-terminal notification"?

Yours (and good night),
			Elmar.

---

From: Marco Rodrigues <[email-masked]>
Date: Wed, 25 Mar 2009 13:28:22 -0400

> information carried in IGPs today is huge :) Look at the mpls traffic

Thank you. 

> define a BGP-CI (communication instance) and use it for exchange of
such

-mpr