Subject: [GROW] I-D Action:draft-ietf-grow-bmp-05.txt

From: [email-masked]
Date: Wed, 15 Dec 2010 14:15:02 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Global Routing Operations Working Group of the IETF.

	Title           : BGP Monitoring Protocol
	Author(s)       : J. Scudder, et al.
	Filename        : draft-ietf-grow-bmp-05.txt
	Pages           : 16
	Date            : 2010-12-15

This document proposes a simple protocol, BMP, which can be used to
monitor BGP sessions.  BMP is intended to provide a more convenient
interface for obtaining route views for research purpose than the
screen-scraping approach in common use today.  The design goals are
to keep BMP simple, useful, easily implemented, and minimally
service-affecting.  BMP is not suitable for use as a routing
protocol.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-grow-bmp-05.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 15 Dec 2010 23:16:06 +0000

> service-affecting.  BMP is not suitable for use as a routing

from a conceptual point of view:

1. I can see several immediate problems implementing this on a route server 
in the context of section 5 (e.g. scaling on large systems, multiple 
LocRIBs on same bgp instance, etc)

2. I would much prefer to see support for bgp4mibv2 on regular routers, 
rather than a completely new tool (although I do acknowledge the necessity 
of being able to peek into ribs, and that snmp is woefully inadequate for 
this job).

 From a details point of view:

1. please encode 1970-epoch timestamps as 64 bits, not 32.  32 bits will 
break before I retire.

2. ASCII.  Sigh.

3. maybe TLS/SSL instead of ipsec?  It's not that I loathe ipsec (I don't); 
it's just that ssl is much easier to implement, as there are well 
understood client system APIs for it.

4. address encoding: would it not be wise from an implementation point of 
view to include the protocol number near the address fields, on the basis 
that ipv4 == len(4) and ipv6 == len(16) is, well, a bit hacky?  What if 
there's some other protocol in future which might want to use BMP?

Nick

-- 
Network Ability Ltd. | Head of Operations      | Tel: +353 1 6169698
3 Westland Square    | INEX - Internet Neutral | Fax: +353 1 6041981
Dublin 2, Ireland    | Exchange Association    | Email: [email-masked]

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 16 Dec 2010 00:26:27 +0100

John at all,

> 	Filename        : draft-ietf-grow-bmp-05.txt

If I remember correctly the fundamental reason for bmp was to provide a 
very simple mechanism for replaying content of received updates from the 
peers as well as provide some form of signaling reg their state 
transition. That goal was great.

Unfortunately version -01 and above moved away from this and simplified 
the requirement to reply/send content of Adj_RIB_In instead of those 
coming updates from the peers.

So we are effectively loosing the most crucial part of the original 
proposal as we no longer be able to pass to some observatory linux 
station those prefixes which were dropped on inbound or worse any 
potential malformed updates or attributes if they were not stored in the 
RIB_In.

I think it defeats a lot of use cases. Perhaps this subtle difference 
should be discussed more before we proceed any further with this document.

Many thx,
R.

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 16 Dec 2010 16:35:09 +0000

Nick,

Addressing one specific point:

On Wed, Dec 15, 2010 at 11:16:06PM +0000, Nick Hilliard wrote:
> rather than a completely new tool (although I do acknowledge the 
> necessity of being able to peek into ribs, and that snmp is woefully 
> inadequate for this job).

Speaking as the editor of the MIB and being known for being opinionated
about SNMP: SNMP sucks.  SNMP is *not* a good tool for monitoring routing
state.  It's fine for monitoring peer counters and monitoring specific
prefixes using standard polling techniques.

The MIB also has the ability to let you examine multi-rib scenarios as well,
but the indexing mechanism will have some amount of vendor-specificness to
it.

-- Jeff

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 16 Dec 2010 17:02:43 +0000

Yes, it's a ghastly protocol, and the bgp4mibv2 will cause all sorts of horrible contortions. But the alternative (screen scraping) is worse. Also it's got the advantage that my current monitoring systems support it, so monitoring my bgp sessions is just a matter of changing a few lines in my provisioning system, rather than writing support for a completely new protocol like bmp.  

Nick

Sent from my iWotsit.

---

From: john heasley <[email-masked]>
Date: Thu, 16 Dec 2010 09:03:04 -0800

Thu, Dec 16, 2010 at 04:35:09PM +0000, Jeffrey Haas:
> SNMP is *not* a good tool for monitoring routing

Please repeat this to yourselves every time you think of SNMP and routing.

---

From: john heasley <[email-masked]>
Date: Thu, 16 Dec 2010 09:09:34 -0800

Thu, Dec 16, 2010 at 05:02:43PM +0000, Nick Hilliard:
> But the alternative (screen scraping) is worse. Also it's got the advantage that my current monitoring systems support it, so monitoring my bgp sessions is just a matter of changing a few lines in my provisioning system, rather than writing support for a completely new protocol like bmp.  

Netconf and similar.

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 16 Dec 2010 17:30:35 +0000

> Please repeat this to yourselves every time you think of SNMP and routing.

I think my rant at the OPS and routing area open meetings a few IETFs back
convinced the relevant people that we should never try to push for such
things in SNMP again.

But this does mean that if you care about getting such state with a clean
presentation layer of some sort (i.e. no screen scraping), spend some cycles
in places like netconf/netmod.  Contribute.  And then convince your vendors
to implement. :-)

-- Jeff

---

From: John Scudder <[email-masked]>
Date: Thu, 16 Dec 2010 12:00:40 -0800

> 1. please encode 1970-epoch timestamps as 64 bits, not 32.  32 bits will 

This is not out of the question, but keeping in mind that there's already implementation of the currently specified format, what is the value of making an incompatible change in order to avoid a wrap in 2106?  I'd think that given 96 years to work on it and a grasp of modular arithmetic, it may be possible to work up BMP listeners that deal gracefully with the wrap.  Further feedback from the WG on this point would be welcome.

Also, out of curiosity, do you really expect to still be working in 2106?  

> 2. ASCII.  Sigh.

Colleagues keep threatening to educate me about text internationalization but it hasn't happened yet.  If someone would care to contribute the least-onerous possible revision that would make the spec internationally correct, that would be welcome.

> understood client system APIs for it.

IPSec is exemplary, not normative.  TLS/SSL is another example, we can mention it and list in the Informative References too if folks feel that would be helpful.

> there's some other protocol in future which might want to use BMP?

I won't argue it's not a bit hacky but it's a fairly common idiom. As a practical matter, BMP is pretty tightly bound to BGP and BGP is only specified over v4 and v6.  I won't argue that generality isn't good in principle though, all things being equal.

Feedback from the WG on this point would also be welcome, again bearing in mind that there's already running code.  

Regards,

--John

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 16 Dec 2010 21:28:18 +0000

On 16/12/2010 20:00, John Scudder wrote:
> This is not out of the question, but keeping in mind that there's
> already implementation of the currently specified format, what is the
> value of making an incompatible change in order to avoid a wrap in 2106?
> I'd think that given 96 years to work on it and a grasp of modular
> arithmetic, it may be possible to work up BMP listeners that deal
> gracefully with the wrap.  Further feedback from the WG on this point
> would be welcome.

I'm getting my signage mixed up - the 2038 problems associated with epoch 
timestamps are related to signed 32 bit time_t rather than the unsigned 
quantity used here.  I agree it's less of an issue in this case - although 
electrons are pretty cheap and if this were version 0, I'd be more 
interested in seeing 64 bits on a matter of principle.  As there's running 
code, maybe it would be easier to stay with 32 bits.

Apologies - I'm jumping into this rather late in the proposal, with no 
background context.  Hence the revision 1 style comments on the revision 5 
draft.

>> 2. ASCII.  Sigh.
>
> Colleagues keep threatening to educate me about text
> internationalization but it hasn't happened yet.  If someone would care
> to contribute the least-onerous possible revision that would make the
> spec internationally correct, that would be welcome.

I appreciate the difficulties here, and I'm sure you do too.  But as a 
general comment which is not particularly related to BMP, the ietf is 
rather english/ascii centric, despite the fact that a large portion of its 
global constituency doesn't even use latin based character sets.  Some 
time, as a point of policy, it may be good for the ietf to consider moving 
away from ASCII to e.g. UTF8 for text strings in newly-designed protocols. 
  This isn't anything specific to GROW, though.

> IPSec is exemplary, not normative.  TLS/SSL is another example, we can
> mention it and list in the Informative References too if folks feel that
> would be helpful.

It is exemplary, yes.  However, implementers sometimes believe that 
examples are in fact recommendations.  At least mentioning SSL/TLS would be 
good, as ipsec causes so much stomach acid for both implementers and end-users.

> I won't argue it's not a bit hacky but it's a fairly common idiom. As a
> practical matter, BMP is pretty tightly bound to BGP and BGP is only
> specified over v4 and v6.  I won't argue that generality isn't good in
> principle though, all things being equal.
>
> Feedback from the WG on this point would also be welcome, again bearing
> in mind that there's already running code.

Yeah, again considering the running code issue, it may be more prudent to 
leave this as-is, even if it would have been better on day 1 to formulate 
it more elegantly.

Perhaps if there are future updates to BMP which cause protocol 
incompatibilities, it may be more appropriate to consider these suggestions 
then rather than now.

Nick

---

From: Stephen Stuart <[email-masked]>
Date: Thu, 16 Dec 2010 17:08:13 -0800

On Wed, Dec 15, 2010 at 3:26 PM, Robert Raszuk <[email-masked]> wrote:
> If I remember correctly the fundamental reason for bmp was to provide a very
> simple mechanism for replaying content of received updates from the peers as
> well as provide some form of signaling reg their state transition. That goal
> was great.
>
> Unfortunately version -01 and above moved away from this and simplified the
> requirement to reply/send content of Adj_RIB_In instead of those coming
> updates from the peers.

It allowed an implementation to reconstruct updates, rather than replicate them.

> So we are effectively loosing the most crucial part of the original proposal
> as we no longer be able to pass to some observatory linux station those
> prefixes which were dropped on inbound or worse any potential malformed
> updates or attributes if they were not stored in the RIB_In.

I disagree. The most crucial part of the proposal is to get the
contents of Adj_RIB_In from the router to a monitoring station,
without the filter of best path selection that occurs when you use BGP
peering as a means to monitor what the router sees (and bgp-add-paths
doesn't accomplish this, either). The corner cases of malformed
updates and filtered updates simply aren't interesting for the
application that this protocol was designed to enable - centralized
analysis of the whole of an autonomous system's Adj_RIB_In.

> I think it defeats a lot of use cases. Perhaps this subtle difference should
> be discussed more before we proceed any further with this document.

This was discussed before, if I recall correctly. BMP is proposed to
do one job well, and that's to get the collected Adj_RIB_In out,
globally, to where it can be processed by real computers.

Stephen

---

From: John Scudder <[email-masked]>
Date: Thu, 16 Dec 2010 18:27:54 -0800

On Dec 16, 2010, at 8:08 PM, Stephen Stuart wrote:
> This was discussed before, if I recall correctly.

That's correct.  In fact because of the divergence between -00 and -01, I took pains to make this clear when presenting -01, including at NANOG and GROW and probably other places.  Those interested in the history should be able to find the slides easily enough.

--John

---

From: john heasley <[email-masked]>
Date: Thu, 16 Dec 2010 18:38:23 -0800

> That's correct.  In fact because of the divergence between -00 and -01, I took pains to make this clear when presenting -01, including at NANOG and GROW and probably other places.  Those interested…

why does this matter?  sure, adj-rib-in is MUST, but can that not be
configurable which is (are) exported?

---

From: Stephen Stuart <[email-masked]>
Date: Thu, 16 Dec 2010 21:07:11 -0800

> configurable which is (are) exported?

I'm not sure what you're advocating to be configurable. Loc_RIB isn't
interesting, you can get that by BGP-peering with the router if you're
content to have the data obscured by best path selection. I'm not
interested in turning BMP into a protocol for debugging malformed BGP
updates, syslog already does that adequately. BMP is meant to do one
job: allow a real computer to reproduce the Adj_RIB_In of a router, in
a way that protects the router against conditions such as a bad
receiver implementation or congestion loss between the speaker and the
receiver.

Stephen

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 17 Dec 2010 10:02:27 +0100

Hi Stephen,

Many thx for your reply.

>> Unfortunately version -01 and above moved away from this and
>> simplified the requirement to reply/send content of Adj_RIB_In
>> instead of those coming updates from the peers.
>

Update "reconstruction" to me means to take what is already in 
Adj_RIB_In, place it into the bmp wrapper and send over bmp session. In 
fact spec is pretty clear to say this is normal, local, update 
generation essentially per peer.

Few points:

* What is in Adj_RIB_In is already the product or result past the BGP IO 
processing therefor any issues there will not be visible,

* The information about update format received from the peer is lost - 
example you will not be able to say if MP_(UN)REACH_NLRI were first in 
the update msg nor what was for example the packing ratio

* Many address families (example VPNv4) drop uninteresting routes when 
non intersecting RT is found hence they will not be in Adj_RIB_In .. I 
would not expect that for BMP reasons routers now will store all VPN 
routes permanently

* You loose information about update frequency, inter-update gaps, churn

* You report only accumulated number of invalidate routes without 
listing those NLRIs ..

I think those are enough of reasons to consider adding a new option to 
bmp to actually allow to wrap received TCP strems from each peer like 
Tony has proposed in a bmp header and send it to the observatory linux 
station. In this case non of the above drawback would occur.

Moreover such wrapping and sending does not need to happen inline .. it 
is very easy to buffer in RAM or in HDD copy of incoming streams for 
their push to linux box when CPU has spare cycles to do so.

Also it needs to be noticed that such option would not require to 
increase amount of permanent resource use by the bgp speaker. While 
Adj_RIB_In would need to be kept, keeping above described TCP streams is 
only a transient resource usage.

The timestamp in BMP header in this case may actually reflect the true 
reception of such stream to the router allowing for much broader use case.

And again I am not saying that current message types should be removed 
from the spec. I am just proposing to add one more and let the 
implementer and operator choose which one makes most sense for them.

Many thx,
R.

>> potential malformed updates or attributes if they were not stored
>> in the RIB_In.
>
> without the filter of best path selection that occurs when you use
> BGP peering as a means to monitor what the router sees (and
> bgp-add-paths doesn't accomplish this, either). The corner cases of
> malformed updates and filtered updates simply aren't interesting for
> the application that this protocol was designed to enable -
> centralized analysis of the whole of an autonomous system's
> Adj_RIB_In.
>
>> I think it defeats a lot of use cases. Perhaps this subtle
>> difference should be discussed more before we proceed any further
>> with this document.
>

---

From: john heasley <[email-masked]>
Date: Fri, 17 Dec 2010 15:01:23 -0800

> receiver implementation or congestion loss between the speaker and the

I'm suggesting that BMP can do both as it is defined now; and which is
exported could be user configurable on exporting device.

---

From: Stephen Stuart <[email-masked]>
Date: Fri, 17 Dec 2010 15:38:53 -0800

> exported could be user configurable on exporting device.

As noted, Loc_RIB can already be collected by peering with the device
as a regular BGP peer. I would argue that introducing malformed
updates runs the risk of introducing instability into a BMP receiver
that may potentially cause it to lose state for a large Adj_RIB_In for
no good cause, where the router will have (hopefully) just dumped the
single BGP peer that sent the malformed update - and logging of the
malformed update by the router using an existing mechanism like syslog
is sufficient for what ought to be a rare event in the field. So, yes,
BMP could do both, but there's no reason to add functionality to BMP
to do jobs that are already done quite sufficiently by other
mechanisms.

Stephen

---

From: Stephen Stuart <[email-masked]>
Date: Fri, 17 Dec 2010 15:47:48 -0800

On Fri, Dec 17, 2010 at 1:02 AM, Robert Raszuk <[email-masked]> wrote:
> Update "reconstruction" to me means to take what is already in Adj_RIB_In,
> place it into the bmp wrapper and send over bmp session. In fact spec is
> pretty clear to say this is normal, local, update generation essentially per
> peer.
>
> Few points:
>
> example you will not be able to say if MP_(UN)REACH_NLRI were first in the
> update msg nor what was for example the packing ratio

That level of protocol debugging was not a requirement for BMP, and
isn't of interest for the family of applications that I'm trying to
enable. Note that an implementation of BMP is free to replicate
updates as received rather than generate them, and (arguably) it would
bad from a software complexity standpoint to implement both. If you'd
like to explore this further in, say, IOS, I would be happy to do the
work on the receiver to bring it up to draft -04 and test with you.

> * Many address families (example VPNv4) drop uninteresting routes when non
> intersecting RT is found hence they will not be in Adj_RIB_In .. I would not
> expect that for BMP reasons routers now will store all VPN routes
> permanently
>
> * You report only accumulated number of invalidate routes without listing
> those NLRIs ..
>
> I think those are enough of reasons to consider adding a new option to bmp
> to actually allow to wrap received TCP strems from each peer like Tony has
> proposed in a bmp header and send it to the observatory linux station. In
> this case non of the above drawback would occur.
>
> Moreover such wrapping and sending does not need to happen inline .. it is
> very easy to buffer in RAM or in HDD copy of incoming streams for their push
> to linux box when CPU has spare cycles to do so.
>
> Also it needs to be noticed that such option would not require to increase
> amount of permanent resource use by the bgp speaker. While Adj_RIB_In would
> need to be kept, keeping above described TCP streams is only a transient
> resource usage.
>
> And again I am not saying that current message types should be removed from
> the spec. I am just proposing to add one more and let the implementer and
> operator choose which one makes most sense for them.

As above, that's orthogonal to the type of application that I need to
enable, but if you want to prototype something I'm happy to help on
the receiver side. How soon do you think you can have an
implementation ready?

Thanks,
Stephen

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 18 Dec 2010 01:19:25 +0100

Hi Stephen,

Many thx for your offer to work on the receiving side if the sender 
could replicate incoming TCP streams and encapsulate them into the BMP 
wrapper rather then maintain and generate updates from Adj_RIB_In.

I will get to you offline after discussion with other members of IOS BGP 
team when we could offer prototype/eft of such solution.

However for perhaps obvious reasons it is in everyone's interest to 
avoid any proprietary solutions to surface. That is main reason why I 
proposed to include a new option in the current bmp spec allowing for it 
to be while not mandatory, but still a standards based BMP encapsulation 
option.

Cheers,
R.

> On Fri, Dec 17, 2010 at 1:02 AM, Robert Raszuk<[email-masked]>  wrote:
>> Hi Stephen,
>>
>> Many thx for your reply.
>>