Subject: Open Routing Area meeting in Vienna

From: Bill Fenner <[email-masked]>
Date: Tue, 8 Jul 2003 16:00:09 -0700

Hiya folks,

  Alex and I managed to both assume that the other one was going to
schedule the routing area meeting for Vienna -- oops!  Luckily, we
have very forgiving people in charge of the scheduling, so we were
able to get a room: Hall NO on Thursday afternoon, 1530-1730.  (This
is on the online agenda, but didn't make it into the printed version.)

  We will begin with the usual status reports from the working groups
and BOFs that meet.  We'll start the open mic time with a discussion of
reuse of routing protocols for things that look less and less like
routing, and we'll see where things go from there.  A more complete
agenda will follow.

  Bill

_______________________________________________
routing-discussion mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/routing-discussion

---

From: Danny McPherson <[email-masked]>
Date: Tue, 22 Jul 2003 10:55:19 -0600

I missed the latter part of the panel discussion in the Routing Area meeting
in Vienna.  Although I doubt any conclusions were reached, were there any
action items that came about (e.g., produce problem statement, applicability
notes or other)?

-danny

---

From: David Meyer <[email-masked]>
Date: Tue, 22 Jul 2003 10:23:55 -0700

>> action items that came about (e.g., produce problem statement, applicability

	I had suggested producing a problem statement
	(http://www.1-4-5.net/~dmm/IETF57/routing_area), 
	which is something I'm working on, though I don't know
	how much traction there really was for such a thing. In
	any event, ss soon as I have a strawperson statement I'll
	post it here. 

	Dave

---

From: Danny McPherson <[email-masked]>
Date: Tue, 22 Jul 2003 10:55:19 -0600

-danny

---

From: David Meyer <[email-masked]>
Date: Tue, 22 Jul 2003 10:23:55 -0700

Dave

---

From: Kireeti Kompella <[email-masked]>
Date: Tue, 22 Jul 2003 17:45:17 -0700 (PDT)

> I had suggested producing a problem statement

I second the motion (if this is a motion :-))

> ss soon as I have a strawperson statement I'll
> post it here. 

Wonderful!

Kireeti.

PS: Good X-philosophy!

---

From: Kireeti Kompella <[email-masked]>
Date: Tue, 22 Jul 2003 17:45:17 -0700 (PDT)

Wonderful!

Kireeti.

PS: Good X-philosophy!

---

From: David Meyer <[email-masked]>
Date: Wed, 23 Jul 2003 07:39:45 -0700

>> PS: Good X-philosophy!

Thanks :-)

---

From: David Meyer <[email-masked]>
Date: Wed, 23 Jul 2003 07:39:45 -0700

Thanks :-)

---

From: Alex Zinin <[email-masked]>
Date: Thu, 24 Jul 2003 18:03:41 -0700

Danny, et al-

 There are two main things that I took out of the meeting:

  1. Though we didn't reach consensus that RP overloading is a problem
     (as I said in the beginning, the community is quite polarized on
     this question), the number of concerned people was much more than
     I expected. In other words, we do not have a consensus that RP
     overloading is a non-issue.

  2. At the end of the meeting I asked the room if we should pursue
     creation of a separate protocol that would potentially inherit
     some of the MP-BGP framework and would be used as a generalized
     info distribution protocol. There was a very strong consensus
     in the room that we should do so. This is a very important step,
     I believe, and will be following up on this soon.

  It would also be useful to document what people concerned with RP
  overloading consider as potential problems. This should map nicely
  to Dave's problem statement idea.

  BTW, I'm planning on sending the minutes within a week--need to
  sync up notes taken by Ross and Dimitri.
  
-- 
Alex
http://www.psg.com/~zinin/

---

From: Alex Zinin <[email-masked]>
Date: Thu, 24 Jul 2003 18:03:41 -0700

Danny, et al-

---

From: Pedro Roque Marques <[email-masked]>
Date: Thu, 24 Jul 2003 20:06:05 -0700 (PDT)

Alex Zinin writes:

> creation of a separate protocol that would potentially inherit some
> of the MP-BGP framework and would be used as a generalized info
> distribution protocol. There was a very strong consensus in the room
> that we should do so. This is a very important step, I believe, and
> will be following up on this soon.

The problem w/ the question that you asked is that you made no mention
of any drawbacks associated w/ such a choice...

It is a bit like asking a room full of people if they want to receive
a million dollars. Everybody will raise their hands but in fact the
deal may not be much better than the business oportunities from
Nigeria that one receives daily. :-)

Given that the underlying desire seems to be that SPs that are not
interest in a given service would prefer not to have that
functionality added to BGP (in fact many people mentioned that they
wanted not to have the functionality in the router software, period)
and that there several services that are in consideration, it is not
clear to me that the community will be able to agree on which subsets
are wanted and which unwanted. Either in BGP or in any new protocol.

An (incomplete) list of services would be:
ipv4-unicast, ipv4-multicast, ipv6-unicast, ipv6-multicast, 2547 w/
ipv4 / ipv6 / unicast/multicast variations, labeled-unicast,
vpn auto-discovery, bgp-l2vpn, ip-tunneling-discovery, etc...

One interesting question is how do you go about defining which subsets
of services fit where... given that different providers want different
subsets and that the rational is that the service a SP don't care
about should not be in the protocol that is used by that SP, it would
seem to me that you need one protocol per service/application.

Developing a single additional protocol doesn't seem to fit the bill,
as far as i can see. Or at least not based in the rational presented.

And if the new protocol under discussion is not subject to "overload"
considerations then the question needs to be why not ? Is there some
new technical ability that will be added to it ? If so why not add it
to BGP itself?    

Or is the intention to add a new protocol per service/application ?
I would be very surprised if that where to result in increased
software quality.

I believe that the drawbacks of a given decision have to be
considered. The discussion in Vienna was about the existence of a
problem w/ routing protocol overload. I don't think it is fair game to
attempt to establish consensus on a question that was not discussed
and that you asked in the last second of the meeting immediatly
cutting off discussion.

regards,
  Pedro.

---

From: Pedro Roque Marques <[email-masked]>
Date: Thu, 24 Jul 2003 20:06:05 -0700 (PDT)

regards,
  Pedro.

---

From: Eric Rosen <[email-masked]>
Date: Fri, 25 Jul 2003 10:51:24 -0400

> wanted not to have the functionality in the router software, period)

This  is a  very natural  desire, but  it is  difficult to  draw engineering
conclusions from it. 

Sure, if  someone has no  intention of deploying,  say, IPv6, they  will say
"please don't mess up BGP by putting  IPv6 stuff in it".  On the other hand,
if they  do want  to deploy IPv6  they'll say  "please don't make  me deploy
another  inter-AS routing  protocol just  to  handle IPv6;  can't you  just
extend BGP a little, I already know how to deal with that."  

Obviously  it is  impossible  for a  vendor  to provide  a  single piece  of
software that satisfies both groups.  And since, for any given feature, both
groups will always  exist, it is difficult to  obtain consensus.  The desire
cited above will never provide a good basis for deciding whether to extend a
particular protocol for a particular purpose or not.

---

From: Eric Rosen <[email-masked]>
Date: Fri, 25 Jul 2003 11:17:09 -0400

I wasn't in Vienna, but let me put in my two cents on why it sometimes makes
sense  to use BGP  for distributing  non-routing information.   Basically, I
think you  need to look at  the specific requirements  for distributing that
information.   (Yes,  I  know  this  isn't  as  much  fun  as  making  broad
generalizations ;-))

Consider a set of routers S such that: 

- Members of S know apriori that they are members of S, but they do not know
  apriori which other routers are members of S. 

- S is divided into subsets (not necessarily disjoint) such that all members
  of a subset share a certain attribute.  Thus for attribute A, we can speak
  of the A-subset of S.  

- Each member  of S is configured with  a set of "export  attributes", and a
  set of "import attributes". 

- If a router in S is configured  with export attribute E, it is a member of
  the E-subset. 

- If a router in S is configured  with import attribute I, it needs to learn
  the membership of the I-subset. 

- The  set  of  import and/or  export  attributes  with  which a  router  is
  configured  may change  at  any time;  changes  in the  membership of  the
  various A-subsets should be learned within minutes.

- The changes do not occur with great frequency. 

- The total number of attributes under consideration may be very large. 

- Members of  S cannot be assumed to  be in the same  Autonomous System, nor
  can  they necessarily  be assumed  to  be under  the control  of the  same
  administration.

Now  we  need  a  protocol  which  detects  changes  in  the  A-subsets  and
distributes these changes.  What requirements can be put on such a protocol: 

- It has to  provide a way for a  router to announce that it  has acquired a
  new export attribute. (Call this an "update" operation.)

- It has  to provide  a way for  a router  to announce that  it has  lost an
  export attribute.  (Call this a "withdraw" operation.) 

- For  scalability, we  prefer  that routers  which  are not  in  S are  not
  impacted by any  of this; in particular, they should  not need to maintain
  any state which is specific to S or any of its subsets.

Since the members of S do not know each other apriori, one might expect that
there will be a server involved, where  the members of S talk to the server,
and the  server redistributes the  information as necessary.  Of  course, we
usually want  to have each router  talking to two servers,  with the servers
also updating each other.  And to make sure the whole system scales up, we'd
like to be able to have  servers with different sets of "clients" talking to
each  other.  Since  we may  be passing  large amounts  of  information over
arbitrary  network  paths, we  need  to  have  a reliable,  flow  controlled
transport, and it would be  nice if the transport handled congestion control
as well.  A  point-to-point transport between clients and  servers seems the
best fit,  given that we  don't want to  burden any of S's  non-members with
S-specific state. 

To further  improve scalability, we'd  like to be  able to have  each client
tell its server(s) its set of import attributes, so that a given router only
receives the  information in which  it is interested.  (Call  this "outbound
attribute filtering".)  If  we have servers talking to  servers, it would be
nice to have  the outbound attribute filtering passed  around the servers in
some loop-free manner, but we can leave that for further study. 

If  you  put all  these  requirements together,  and  look  for an  existing
protocol to satisfy them, you are  inevitably led to BGP.  The salient issue
is  not whether  the attributes  that  need to  be distributed  are part  of
"routing",  but  whether the  requirements  for  the  distribution of  these
attributes  are very  similar to  the requirements  for the  distribution of
inter-domain routing information.  We are led to BGP because of a similarity
in the distribution requirements, not necessarily because of a similarity in
the information distributed.

Many alternatives  have been suggested  in the past,  but most of  them just
make BGP look even better ;-)

---

From: Eric Rosen <[email-masked]>
Date: Fri, 25 Jul 2003 11:17:09 -0400

---

From: Eric Rosen <[email-masked]>
Date: Fri, 25 Jul 2003 10:51:24 -0400

---

From: Scott W Brim <[email-masked]>
Date: Thu, 24 Jul 2003 21:27:37 -0400

>      I believe, and will be following up on this soon.

Do you (or anyone) have a summary of the pros and cons?

Thanks ... Scott

---

From: Scott W Brim <[email-masked]>
Date: Thu, 24 Jul 2003 21:27:37 -0400

Thanks ... Scott

---

From: Eric Rosen <[email-masked]>
Date: Mon, 28 Jul 2003 11:25:00 -0400

Jon> its always  attractive to overload "working"  infrastructures (vis DNS,
Jon> SIP,  etc etc)  but personally,  i believe  in choosing  ones  that are
Jon> aliged  with the  performance requirements  of the  application, rather
Jon> than ones that are barely able to meet existing demands

Using BGP as  a distribution protocol does not  necessarily imply hooking it
into the public  inter-domain routing infrastructure.  If BGP  is a suitable
protocol for distributing some  information, an infrastructure for that info
can be set up without impacting any pre-existing BGP infrastructure.

Jon> for  a  start you  create  other reasons  why  BGP  speakers might  get
Jon> overloaded,  might get  attacked,  might get  misconfigured, and  might
Jon> never converge 

Any  time you use  a protocol  to distribute  information, you  create these
possibilities, it's nothing to do with what protocol you're using.

---

From: Eric Rosen <[email-masked]>
Date: Mon, 28 Jul 2003 11:25:00 -0400

---

From: Jon Crowcroft <[email-masked]>
Date: Sat, 26 Jul 2003 07:50:42 +0100

>>I wasn't in Vienna, but let me put in my two cents on why it sometimes makes

unfortunately for the dollar, the cent does not go as far in europe as it used to:-)

 >>information.   (Yes,  I  know  this  isn't  as  much  fun  as  making  broad

despite your abstract argument (which i agree with) the point is that
distributing things that matter in an infrastructure that is severely broken
is a very bad idea - 

for a start you create other reasons why BGP speakers might get overloaded, might get
attacked, might get misconfigured, and might never converge - 

its always attractive to overload "working" infrastructures (vis DNS, SIP, etc etc) but
personally, i believe in choosing ones that are aliged with the performance requirements of
the application, rather than ones that are barely able to meet existing demands

 >>make BGP look even better ;-)

 cheers

   jon

---

From: Jon Crowcroft <[email-masked]>
Date: Sat, 26 Jul 2003 07:50:42 +0100

cheers

   jon

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 12:47:21 -0700

Pedro,

> Nigeria that one receives daily. :-)

[reordering paras a bit]

> and that you asked in the last second of the meeting immediatly

I see no harm at all in taking the sense of the room about the
interest in exploring a certain direction based on the discussion
about the existence of the problem. My reading of this is that we
clearly need an alternative solution to overloaded BGP. We certainly
need to discuss further details.

> vpn auto-discovery, bgp-l2vpn, ip-tunneling-discovery, etc...

IMO, attempts to pull out from MP-BGP what's already there would be a
no-op. I also believe that we have a certain level of agreement (based
on my discussions within the rtg-dir, and in the hall-ways) that there
are certain types of information that should be distributed within the
RP. The major point of contention seems to be distribution of
information that is not really routing-related or tightly coupled with
it, such as VPN discovery and signaling, etc.

What I am planning to do is ask the RTG directorate to document the
discussion we had about this and come up with a document listing those
considerations. We would then discuss this document here.

> I would be very surprised if that where to result in increased

The way I see it, the major concern is overloading of BGP, which is
part of Internet-critical infrastructure, with non-routing
functionality. Hence, the first step would be to give another
(similar) transport to those application and keep them away from BGP.

I could certainly see how people would feel comfortable putting
multiple non-critical network-based applications on the same transport
system. However, I could also see that at some point some critical
functionality needs to be implemented and people don't feel
comfortable putting it on the same transport, and decide to have a
separate protocol for it.

Alex

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 12:51:34 -0700

Eric,

  See my reply to Pedro, please.

  Arguing that we should stop putting _any_ features in BGP would
  seem a non-starter to me. I believe the discussion is about
  non-routing applications of the MP-BGP framework.

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 12:51:34 -0700

Eric,

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 13:07:53 -0700

Eric,

> some loop-free manner, but we can leave that for further study. 

Thanks! Really great summary of the required functionality!
This is at least what I would have in mind.

> make BGP look even better ;-)

And that's why, if we follow the direction of instantiating a separate
protocol, it would most probably inherit a lot from MP-BGP. I.e., the
idea is that it would have a very similar framework, though work over
a different TCP port number, and would live its own life with
independent (from BGP) features, extensions, optimizations, etc.

[from your thread with Jon:]

Jon>> for  a  start you  create  other reasons  why  BGP  speakers might  get
Jon>> overloaded,  might get  attacked,  might get  misconfigured, and  might
Jon>> never converge 
> possibilities, it's nothing to do with what protocol you're using. 

I think the argument here is about risk assessment and management.

Though it is possible to do a fine job of clearly separating different
applications of the same protocol within the system, using a separate
protocol makes it much more likely to happen.

Alex

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 13:22:58 -0700

Scott,

> Do you (or anyone) have a summary of the pros and cons?

I have some in my head, but not in writing.

Should be an interesting exercise. Would be faster if someone took the
first crack at this and we would then discuss it on this list.
Otherwise, we'll have to wait for me :)

Alex

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 13:22:58 -0700

Scott,

Alex

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 12:47:21 -0700

Pedro,

[reordering paras a bit]

Alex

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 13:07:53 -0700

Eric,

Alex

---

From: "John G. Scudder" <[email-masked]>
Date: Tue, 29 Jul 2003 17:20:43 -0400

At 6:03 PM -0700 7/24/03, Alex Zinin wrote:
>      info distribution protocol.

Does a set of (prefix, next hop) information fall under the heading 
of "generalized info"?  It seems only sensible that it does.  Given 
that, wouldn't it be good engineering to use this GIDP you speak of 
for distribution of such information, instead of maintaining two 
protocols one of which is a functional subset of the other?

If not, why not, and why is the GIDP suitable for every other type of 
"generalized" information with only this one limitation?

If so, what makes the imaginary GIDP more suited for so-called 
"overloading" than BGP?  A clear answer to this question would likely 
be usable as requirements for improving BGP... which would be a 
positive outcome.

Let me also say I agree with Pedro and Eric's remarks.

--John

---

From: Pedro Roque Marques <[email-masked]>
Date: Tue, 29 Jul 2003 15:00:26 -0700 (PDT)

Alex Zinin writes:

>> An (incomplete) list of services would be: ipv4-unicast,
>> ipv4-multicast, ipv6-unicast, ipv6-multicast, 2547 w/ ipv4 / ipv6 /
>> unicast/multicast variations, labeled-unicast, vpn auto-discovery,
>> bgp-l2vpn, ip-tunneling-discovery, etc...

> IMO, attempts to pull out from MP-BGP what's already there would be
> a no-op.

I'm glad we agree on something :-) I'm just going to press you a
little bit more and request you to define "already there".

What is the criteria: WG document ? service provider deployements ?
a vendor implementation ? interoperable implementations ?

> multiple non-critical network-based applications on the same
> transport system.

One of the points that people keep ignoring is that the definition of
critical and non-critical is highly subjective and depends a lot on
who is making the call.

I work with customers to whom IP is non-critical while VPN services
are critical given that they make more money out of the latter
than the former (VPN services are also often sold as higher
reliability than IP service). I also work w/ customers that have a
completely opposite point of view.

Thus my definition of critical has necessarily to be the entire set of
services (ip unicast/multicast, 2547, l2vpns).

ipv6 routing is probably the service considered least critical by SPs
but even this one is critical to quite a few of them and a vendor is
force to treat it has such.

Personally i think that the assumption of "non-critical" is one of the
highest reasoning flaws in this discussion.

  Pedro.

---

From: Pedro Roque Marques <[email-masked]>
Date: Tue, 29 Jul 2003 15:00:26 -0700 (PDT)

Pedro.

---

From: "John G. Scudder" <[email-masked]>
Date: Tue, 29 Jul 2003 17:20:43 -0400

--John

---

From: Scott W Brim <[email-masked]>
Date: Tue, 29 Jul 2003 18:39:49 -0400

How about we rename BGP, declare that we're done with the general
information distribution protocol, and work on a new routing protocol?

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 30 Jul 2003 00:53:26 +0200

Alex,

While watching this debate I have a feeling that the problem here is as
old as civilization ;-). 

There has always been an issue if one should extend one city with new
buildings or start over and build new city. My take on this is that if
the architecture of a given city is good, communication is perfect and
can scale adding a new buildings or districts nicely divided from one
another will not harm anyone to get to large sizes accomodating all
newcommers.

On the other hand if architecture is bad, communication is a nightmare
just asking for building another city will do no good at all. In fact I
would say that in most cases it may do more harm then good if another
group of architects will take over in another city and some of the
intercity communication would still need to happen anyway.

So as it was said by others a number of times this discussion is just
about software eng - nothing else. 

A few providers said that they would like to run only a number of
features they use ... AFAIK that should be doable already in most of the
shipping implementations. 

Also it seems to be some interesting assumption that if we separate so
called "non-routing" info from BGP the entire system would behave much
better. I would like to see what facts make anyone feel/think that way
;)

Last lightly assume we go down the path and invent, implement & deploy a
new protocol instance/port would that mean that we will from such a
moment start indefinitely putting all new things on it or will this
topic be comming back every now and then and each time a "new protocol"
or "new port number" will be recommended ??? As clearly definitions of
routing and non routing, critical and non critical information is
different depending who get's to reply my suggestion for routing
directorate would be to have a BGP port number progressing with the
calendar years :) Now we have 179, in 2004 let's add 180, 2006 - 181 etc
... :):). Just one page draft.

Now time for few comments from this thread ...

> > To further  improve scalability, we'd  like to be  able to have  each
> > client tell its server(s) its set of import attributes, so that a 
> > given router only receives the  information in which  it is interested.
> > (Call  this "outbound attribute filtering".)  If we have servers 
> > talking to  servers, it would be nice to have the outbound attribute
> > filtering passed  around the servers in some loop-free manner, 
> > but we can leave that for further study. 
> 

I am missing the point above. BGP provides this already in two planes.
One is per AFI/SAFI information distribution. As far as I see all new
BGP (even non-routing) extensions are nicely separated in their own
AFI/SAFIs most with additional capabilities. Second one is realized in
at least two ways for pushing to your peer inbound filtering so the peer
on outbound already filters (ORF drafts - prefix, ext community, as path
or rt-constrain draft). So what exactly is missing ?

Cheers,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 30 Jul 2003 00:53:26 +0200

Alex,

Cheers,
R.

---

From: Randy Bush <[email-masked]>
Date: Tue, 29 Jul 2003 16:06:03 -0700

> another will not harm anyone to get to large sizes accomodating all

well, so much for bgp, as anyone researcher who actually looks at
how the protocol performs on the net will tell you.

randy

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 16:18:25 -0700

Hi John, good to hear from you!

> "generalized" information with only this one limitation?

There are protocol-related and operations-related considerations here.

Protocol-wise, prefix distribution may or may not fit in depending on
the design decisions that would be made. I.e., if all BGP mechanisms
needed for IP prefix distribution (such as policy-based best-path
selection, aggregation, NH behavior, etc.) are preserved, than the
protocol may be capable of doing so. There would still be the same
operational considerations though, see below pls.

> separate protocol for it.

Regards.
Alex

---

From: Scott W Brim <[email-masked]>
Date: Tue, 29 Jul 2003 18:39:49 -0400

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 30 Jul 2003 01:26:39 +0200

I am sure those opinions must be documented - could you compile a URL
reference list so we can all see how bad it "performs" today ? Also if
those folks are real researchers we will most likely be able to also see
studies on how much time do we have left.

R.

> Randy Bush wrote:

---

From: David Meyer <[email-masked]>
Date: Tue, 29 Jul 2003 16:38:03 -0700

Robert,

>> studies on how much time do we have left.

	While nowhere near exhaustive, you might have a look at 

	http://www.routeviews.org/papers

	for a few (notably, the craig/abha paper isn't on that
	list).

	Dave

---

From: David Meyer <[email-masked]>
Date: Tue, 29 Jul 2003 16:38:03 -0700

Robert,

	Dave

---

From: Randy Bush <[email-masked]>
Date: Tue, 29 Jul 2003 16:45:36 -0700

> reference list so we can all see how bad it "performs" today ?

other than the large literature on the issue, we working on one
particularly ugly piece of it now.  showing it to dr. bug on
thursday, and hope to have the first ppt at apnic next month.

> Also if those folks are real researchers we will most likely be
> able to also see studies on how much time do we have left.

it's a long way from that.  research is still trying to
differentiate between problems caused by configuration, by
implementaion, and by the protocol itself.

but today, i am looking at one single announce creating over
40 events just a few hops away (in a real network).

but the point is that, while we have such outrageous behavior, no
matter that we are still trying to work out all the causes, it is
scarily not appropriate to characterize bgp as

> the architecture of a given city is good, communication is
> perfect and can scale

randy

---

From: Randy Bush <[email-masked]>
Date: Tue, 29 Jul 2003 16:06:03 -0700

randy

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 16:57:51 -0700

Pedro,

> a vendor implementation ? interoperable implementations ?

Let's think about this. On the one hand we want to be reasonable and
avoid recoding/redebugging things that are already widely agreed on
and/or deployed. On the other hand we don't want to fall for the "oh,
but I just coded it yesterday, and here's the draft" trick. How about
something along these lines: "an IETF RFC, or a de-facto multi-vendor
interoperable standard"? Other suggestions?

> critical and non-critical is highly subjective and depends a lot on

Indeed. Note the "Internet-critical" part.

> highest reasoning flaws in this discussion.

We have a couple of considerations here:
  a) level of criticality, and
  b) fate sharing.

While from the perspective of a stand-alone SP, Internet routing and
L2VPNs might be of equal significance, and hence criticality, if we
look at it from the perspective of the Internet as a whole (which is
what we are supposed to at least keep in mind in the IETF), or at
least a set of cooperating service providers, then VPN functionality
may very well depend on routing (in case of a PE-to-PE tunnel, for
example); in other words, VPN mechanisms use services provided by
routing, and hence routing becomes more critical.

The fate sharing argument is bidirectional--we probably don't want
routing to pay the tax for non-routing related features, and the guys
for whom VPNs are much more important may not want to pay the tax for
whatever we do with Internet routing. If that's the case, separating
the two is a good idea.

Alex

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 16:18:25 -0700

Regards.
Alex

---

From: Pedro Roque Marques <[email-masked]>
Date: Tue, 29 Jul 2003 17:13:29 -0700 (PDT)

Alex Zinin writes:

> Pedro,
>>> IMO, attempts to pull out from MP-BGP what's already there would
>>> be a no-op.

> and/or deployed. On the other hand we don't want to fall for the
> "oh, but I just coded it yesterday, and here's the draft" trick. How
> about something along these lines: "an IETF RFC, or a de-facto
> multi-vendor interoperable standard"? Other suggestions?

Alex,
What is published as RFCs doesn't reflect my definition of what is
"already there".

In several areas, the best one can get to is WG document despite years
spent in the process, running code and customer deployments. Without
wanting to reopen unproductive discussions, this doesn't usually
reflect lack of quality of the proposals but "process blocks".

There is an a very high level of entropy which is created when
something stops being an 'idea' and becomes used in production
networks.

Can we expect the IESG to acknowledge that these documents should be
published as RFCs in order to facilitate interoperability ?

  Pedro.

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 30 Jul 2003 01:26:39 +0200

R.

---

From: Randy Bush <[email-masked]>
Date: Tue, 29 Jul 2003 16:45:36 -0700

randy

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 17:35:46 -0700

Robert,

> While watching this debate I have a feeling that the problem here is as

[leaving out the city development part, it seems being discussed
already ;)]

> about software eng - nothing else. 

Yes and no.

Yes, SW engineering is what we constantly keep in mind in this
discussion. However, this is not what we are trying to define here.

As one of the IETF participants told after the area meeting, we need
to admit that the decisions we make at the protocol level within the
IETF _do_ affect the SW architectures and running code.

I'd like to take this further and say that it is perfectly appropriate
for us to keep the engineering aspects in mind and make the right
choices at the protocol level. This is the i_E_tf. Saying "ouch! this
is engineering, we can't be thinking about this" would be wrong.

What we are also talking about here is splitting the protocol
evolution. I.e., making it clear that this is not BGP any more,
checking if any refinements are needed in the base functionality, and
then letting it live its own life with its own extensions and
features, without looking back at backwards compatibility with BGP...

> shipping implementations. 

from my message to Eric:

> better. I would like to see what facts make anyone feel/think that way

again, it's about risk management.

even with a separate protocol, there is some risk that a vendor will
do a poor job, but it's much less likely that the same running
environment, internal system infrastructure will be used. Same way as
I _practically_ don't have to worry about OSPF bugs if I'm running
ISIS in my network, I won't be as much concerned about bugs in
mechanism X if I'm running BGP and X is put on top of a separate
protocol.

> ... :):). Just one page draft.

The topic of where certain functionality should be put is a valid one
and will come back over and over again. This is a normal process.
From my message to Pedro:

> or rt-constrain draft). So what exactly is missing ?

The point is that I agree that the MP-BGP framework is a good match
for the required functionality, however the number of concerns we have
heard suggest that it may not be THE right choice in the long run.

Cheers

Alex

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 16:57:51 -0700

Pedro,

Alex

---

From: Chandrashekhar Appanna <[email-masked]>
Date: Tue, 29 Jul 2003 17:52:07 -0700

> is engineering, we can't be thinking about this" would be wrong.

..
 
> mechanism X if I'm running BGP and X is put on top of a separate

So how do we reach consensus on a measure for 'risk' that is not
tainted by preferences based on what is profitable/convenient for
some given set of participants in any proposal/discussion for
a new 'item' that needs BGP.

A new protocol would raise the same set of concerns. If there
is a reasonably clear requirement for a new protocol for a given 
set of applications, I think it may be a good idea to just define 
what those are and why there is a feeling that they do not belong
in BGP.  Would that be easier than to debate about a future proof 
'new' protocol for 'all the other stuff'?

rgds,
Chandra.

---

From: Pedro Roque Marques <[email-masked]>
Date: Tue, 29 Jul 2003 17:13:29 -0700 (PDT)

Alex,

  Pedro.

---

From: "John G. Scudder" <[email-masked]>
Date: Tue, 29 Jul 2003 21:19:25 -0400

>Hi John, good to hear from you!

Glad to bring a little ray of sunshine to your day. :-)

>  > functionality.

I would be interested to know if you can elaborate on this without 
resort to the ambiguous term "non-routing".

--John

---

From: Alex Zinin <[email-masked]>
Date: Tue, 29 Jul 2003 17:35:46 -0700

Robert,

Yes and no.

from my message to Eric:

Cheers

Alex

---

From: Chandrashekhar Appanna <[email-masked]>
Date: Tue, 29 Jul 2003 17:52:07 -0700

..

rgds,
Chandra.

---

From: "John G. Scudder" <[email-masked]>
Date: Tue, 29 Jul 2003 21:19:25 -0400

--John

---

From: Pedro Roque Marques <[email-masked]>
Date: Tue, 29 Jul 2003 22:59:14 -0700 (PDT)

>> One of the points that people keep ignoring is that the definition
>> of critical and non-critical is highly subjective and depends a lot
>> on who is making the call.

> Indeed. Note the "Internet-critical" part.

This is perhaps a fundamental difference in perception that we have.

I don't see the existence of an "Internet" infrastructure as separate
from the telecomunications industry. Although the later provides
several services to its customers including access to a public network
it does so on an infrastruture which is shared in variable degrees.

For instance, the transmission infrastruture (wdm, sonet, tdm) tends
to be the same for all services the carrier provides. In an increasing
number of cases, network nodes such as routers are also shared
resources, either by providing transport to several applications or by
terminating them.

I confess that i'm not quite sure what your goal is... my assumption
is that you are concerned w/ the availiability of internet service,
which depends on several factors including the stability of router
software.

While the latter is certainly a cause for concern, i would suggest
that you think of this equipment not as "Internet" infrastructure but
as equipment which is bought and operated by a telecomunications
company.

Thus equipment caracteristics and failures are going to play an
important role in "Internet" availability, regardless of the root
cause of the problem. As such, "critical" needs to be defined from the
point of view of a service provider.

Personally, i believe that this issue is being completly blown out of
proportion. I've not seen any evidence to suggest that adding VPN
services, as we are discussing, has higher impact in routing software
stability than to add any other type of functionality.
I'd like to see people that believe otherwise put forward data that
corroborates that view.

While people love to state that the "I in IETF stands for Internet" in
tend to be very sceptic of how realistic is this view. The Internet
doesn't exist in a vacum. The IETF itself doesn't have a vertical
approach in terms of considering all levels and elements used to
provide internet services. It has mostly an horizontal approach to
standardize the IP protocol suite regardless of the application.

Typically when the word "Internet" enters a discussion it doesn't do
much in terms of achieving progress.

  Pedro.

---

From: Pedro Roque Marques <[email-masked]>
Date: Tue, 29 Jul 2003 22:59:14 -0700 (PDT)

Pedro.

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 30 Jul 2003 08:55:36 +0200

>> wanted not to have the functionality in the router software, period)

A number of people said they at least did want this code enabled if 
they where not using the service.

>> are wanted and which unwanted. Either in BGP or in any new protocol.

It's not about what services the community want's. It's how data for 
each of the protocols/services are distributed. The SPs that went to 
the mike said they didn't see the benefit of overloading BGP with all 
this. Or any other routing protocol.

>> vpn auto-discovery, bgp-l2vpn, ip-tunneling-discovery, etc...

These come in two types. Topology reachability information, and others. 
It's the second group that I personally think we need to move away. But 
I agree with Alex comment below that taking something out that is 
already in is a no-op.

> it, such as VPN discovery and signaling, etc.

Agreed.

>> seem to me that you need one protocol per service/application.

Why not? Why do we need a "super-protocol" that distributes it all? 
Will this then speak XML? Can I upload my configurations in it? Where 
is the limit? Let's at least break out signaling from the reachability 
information.

> (similar) transport to those application and keep them away from BGP.

Agreed.

- - kurtis -

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 30 Jul 2003 08:55:36 +0200

Agreed.

Agreed.

- - kurtis -

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 30 Jul 2003 09:12:46 +0200

> While watching this debate I have a feeling that the problem here is as

While watching this debate I realize that most people posting are 
router vendors...:-)

> about software eng - nothing else.

 From the SP POV this is about operational stability. Not about software 
engineering.

> shipping implementations.

I disagree. In the products from the two vendors, I can't today decide 
what features to use or not AFAIK. I can decide what information to 
process or not. But that is not the same thing.

> better. I would like to see what facts make anyone feel/think that way

What are the facts that we need to put everything into BGP? And where 
do we stop? Why not make the NLRI field XML code? Javascript? 
Operational experience tells me that the easier, simpler and less 
information there is - the less the chance of a problem. It's as simple 
as that.

> ... :):). Just one page draft.

I don't see a problem (actually on the contrary) with separating 
reachability information, signaling, etc.

- - kurtis -

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 30 Jul 2003 09:45:08 +0200

> to be the same for all services the carrier provides.

But you do not use SS7 to signal the paths in SONET/SDH, or the OSI 
information from the ADMs to route the WDM channels.

>  In an increasing

Yes. But the routers in this case are the same as the ADMs. And those 
do not signal anything else than the reachability.

> which depends on several factors including the stability of router

Among other things. As Randy said, there i a lot of research that also 
points to the protocol itself.

> as equipment which is bought and operated by a telecomunications

I fail to understand this comment?

> point of view of a service provider.

 From my experience, when a fault happens, the time it takes to recover 
is many times mostly due to the protocol.

> stability than to add any other type of functionality.

I don't think anyone will disagree with you. I am waiting for someone 
to provide the evidence that we gain stability and faster convergence 
by putting all this into BGP....

- - kurtis -

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 30 Jul 2003 09:12:46 +0200

- - kurtis -

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 30 Jul 2003 09:45:08 +0200

- - kurtis -

---

From: Randy Bush <[email-masked]>
Date: Wed, 30 Jul 2003 07:23:34 -0700

> While watching this debate I realize that most people posting are 

bingo!

and they seem to be all for putting it all in the "it's running
perfectly now" bgp.  except people are getting phd theses by
showing how strangely it is actually running, and this should scare
the hell out of us (not because it's a clear disaster, but because
we don't know).

and the operators seem to be a bit more hesitant with what hits the
soft underbelly of their critical infrastructure.  and thanks, i
get to say what is critical to me.

it's pretty strange to keep watching the customer (yes, try saying
that 'customer' word a few times, and then maybe try 'user' a few
times) being told that their concerns are invalid.

randy

---

From: Randy Bush <[email-masked]>
Date: Wed, 30 Jul 2003 07:23:34 -0700

bingo!

randy

---

From: Eric Rosen <[email-masked]>
Date: Wed, 30 Jul 2003 12:30:37 -0400

Randy> it's pretty  strange to keep  watching the customer (yes,  try saying
Randy> that 'customer'  word a few  times, and then  maybe try 'user'  a few
Randy> times) being told that their concerns are invalid. 

One advantage  of working for a  vendor is that you  get to talk  to lots of
customers,  not just the  ones that  are politically  well connected  in the
IETF.  And  I can tell you  that the idea  of extending BGP ad  infinitum is
enormously popular with the customers, MUCH more popular than it deserves to
be; I'm  often in  the position of  trying to  convince a customer  that BGP
should not be extended for some particular function.

Randy> well, so  much for bgp, as  any one researcher who  actually looks at
Randy> how the protocol performs on the net will tell you.

Sure, BGP is the worst possible  protocol, except for all the others.  If we
can come up with a protocol  that is clearly better, let's definitely use it
instead.  In the meantime, however, ... 

Alex> I'd like to take this further and say that it is perfectly appropriate
Alex> for us  to keep  the engineering  aspects in mind  and make  the right
Alex> choices at the protocol level.  This is the i_E_tf. Saying "ouch! this
Alex> is engineering, we can't be thinking about this" would be wrong.

Sure  the IETF  has  to be  concerned  with engineering.   But the  software
engineering of a  particular box is the responsibility  of the box builders,
and not a matter for IETF consensus.

Alex> even with a  separate protocol, there is some risk  that a vendor will
Alex> do  a poor  job,  but it's  much  less likely  that  the same  running
Alex> environment, internal system infrastructure will be used.

If we had a  second protocol which is "like BGP", probably  there would be a
lot of reuse of modules for table maintenance and stuff.  Of course, some of
these modules might need to be  generalized, which would then impact BGP.  I
think this is the kind of stuff that Pedro has been talking about.

---

From: Randy Bush <[email-masked]>
Date: Wed, 30 Jul 2003 09:33:50 -0700

> One advantage of working for a vendor is that you get to talk to
> lots of customers, not just the ones that are politically well
> connected in the IETF.  And I can tell you that the idea of
> extending BGP ad infinitum is enormously popular with the
> customers

one of the advantages of working for an operator is that you get to
talk to lots of vendors, not just the ones that are politically
well connected in the ietf.  and i can tell you that the idea of
extending bgp ad infinitum horrifies most of them.

can we now finish representing hidden others than ourselves,
expecially given that the ietf is supposedly a forum of individual
experts?

randy

---

From: Randy Bush <[email-masked]>
Date: Wed, 30 Jul 2003 09:33:50 -0700

randy

---

From: Eric Rosen <[email-masked]>
Date: Wed, 30 Jul 2003 13:05:16 -0400

Randy> can we now finish representing hidden others than ourselves,
Randy> especially given that the ietf is supposedly a forum of individual
Randy> experts? 

Seems like a fair enough deal:  you stop pretending to speak for all service
providers, and I won't pretend to speak for all vendors ;-)

---

From: Randy Bush <[email-masked]>
Date: Wed, 30 Jul 2003 10:07:21 -0700

> you stop pretending to speak for all service providers, and I won't
> pretend to speak for all vendors ;-)

no problem, when you actually listen to all the operators actually
in the room

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 30 Jul 2003 13:14:58 -0400

>the hell out of us (not because it's a clear disaster, but because

Well, rather than be scared and run around in terror, I would prefer 
it if we'd remain calm and look at said research, learn something 
from it and maybe even (gasp) work to fix the problems identified. 
We don't seem to be doing any of that in this conversation.  If 
anything, we're distracting ourselves.

Is there in fact research that "people are getting phd theses" for 
that indicates that "overloading" BGP is making the routing 
infrastructure "run strangely"?  I'm not aware of it, and I think I'm 
reasonably conversant with the literature.  The research I'm aware of 
tends to have identified what I'd call day-one bugs (or, ahem, 
limitations) in the protocol.  In particular I'm aware of a fair body 
of research broadly related to policy (and why it's hairy), BGP's 
path-exploration behavior (ditto) and flap dampening (ditto ditto). 
None of these relate to "overloading".

I'd be happy to be corrected with relevant references (really).  But 
lacking such correction, it seems to me that the research you allude 
to isn't germane to the present conversation.

--John

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 30 Jul 2003 13:14:58 -0400

--John

---

From: Eric Rosen <[email-masked]>
Date: Wed, 30 Jul 2003 12:30:37 -0400

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 30 Jul 2003 19:17:12 +0200

The only problem with this is that most of those operators who were in
the room are running and using those ugly non pure ipv4 related bgp
extensions in their production networks to sell more services and to
make extra money. 

Furthen most of them just wants their features in the code tomorrow and
not anybody's elses. 

Cheers,
R.

---

From: Alex Zinin <[email-masked]>
Date: Wed, 30 Jul 2003 10:39:29 -0700

Folks-

 I suggest we once and for all agree that each of us speaks as an
 individual. The chest beating matches like "I've talked to so many
 guys and they think..., how many have you talked to?" are NOT going
 to be productive. The topic is _extremely_ important and we want to
 encourage an open discussion here.

 If you believe someone thinks one way or the other--please ask that
 person to speak up on the list and explain, don't "represent" their
 opinion.

 Please, let's also NOT second-guess why people think one way or the
 other, let's NOT try to put reasons behind THEIR words, let THEM
 do that.

 Let's please get back to the very important topic at hand.
 Thanks.

---

From: Alex Zinin <[email-masked]>
Date: Wed, 30 Jul 2003 10:39:29 -0700

Folks-

---

From: Eric Rosen <[email-masked]>
Date: Wed, 30 Jul 2003 13:44:40 -0400

Randy> no problem, when you actually listen to all the operators actually in
Randy> the room 

As individual  technical experts, we all  need to evaluate  and critique the
technical arguments  using our complete  range of knowledge  and experience.
The correctness of  a technical position has nothing to  do with who happens
to be in some room at some particular time. 

In  the case  at  hand, no  one would  deny  that providers  desire to  have
operational stability. Providers also  desire other things, such as delivery
of new services in a timely manner.   What is at issue is the best technical
approach  for achieving  these  goals.   This is  a  legitimate subject  for
discussion, and the "operators actually in the room" are not the only people
who may have something useful to say.

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 30 Jul 2003 19:50:22 +0200

> instead.  In the meantime, however, ...

I think one of the problems with BGP is that it has lured into this "as 
there is nothing better, let's put it all in there". To be honest I 
think that the reason that you are seeing customers being perfectly 
happy to put various features into BGP is not that they don't see a 
problem with it. They don't think there is an alternative.

If we go to the next Nanog/RIPE/APNIC/SANOG/NORDNOG etc and try and get 
a feeling if people would be happy to add a new protocol, I think you 
would have an interesting discussion.

As I said in Vienna, I think a large part of the problem is that we are 
getting complacent (I hope that is the right word; read lazy).

> and not a matter for IETF consensus.

I would say that the problem we are having now is the opposite. In 
order to solve a software problem, people are trying to add things to a 
protocol.

> think this is the kind of stuff that Pedro has been talking about.

This is a theoretical statement until we know what that protocol or the 
requirement for it would be.

- - kurtis -

---

From: Randy Bush <[email-masked]>
Date: Wed, 30 Jul 2003 10:07:21 -0700

---

From: Eric Rosen <[email-masked]>
Date: Wed, 30 Jul 2003 13:05:16 -0400

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 30 Jul 2003 19:17:12 +0200

Cheers,
R.

---

From: Danny McPherson <[email-masked]>
Date: Wed, 30 Jul 2003 12:16:15 -0600

I.e., hearsay is inadmissible..

It's always amused me how folks within the IETF tout it's existence as a
body of individuals when it's convenient, yet those same folks 
explicitly
ask for "operator input" on a consistent basis.  It's always amusing as 
well
to see folks attempt to solidify their own stance by suggesting that 
they're
speaking on behalf of "lots of customers" or "all the big operators" on 
some
important IETF topic.  If it's that important [and the folks they're 
speaking
on behalf of have email and half a clue!], you'd think they might be 
capable
of speaking for themselves (I am aware of some legitimate exceptions).
Then again, the extreme (and probably most entertaining) is when all of
one vendor's "customers" come out of the woodwork and post the "we like 
this
idea" to the WG mailing list, which is typically equally useless.

The reality is that being an operator doesn't make you any more clueful
then the next guy, nor does being a vendor or "developer" at said 
vendor.
Heck, I know lots of vendor folks that used to be good operators, some 
of
which actually maintained their operator mindset!  I know vendors that
have never been an operator that think more about operational issues
than most operators...  Of course, I see *service provider* requirements
documents authored entirely by vendors with only more vendors in the
acknowledgments section (i.e., no service providers or operators) and
I've seen service provider folks listed as co-authors on documents 
they've
neither read nor understand.

My point: each person's arguments should carry weight based on the
content, not on the author's employer...

On the topic of RP overloading...

There are [some] very real problems folks are attempting to address and
given MP-BGP & BGP-CAP, BGP has been the easy answer.  Heck, with IGP
Capabilities being defined we'll be able to do more things even faster
(although I'm kidding, be wary :-)

Seriously though, maybe piggybacking on routing protocols is the right
answer -- or is it?  Most of the problems folks are trying to solve 
certainly
seem very real.

I was mesmerized by the sight of a "panel" at the Routing Area meeting, 
and
as I suspected, only the same set of arguments were repeated to no 
avail.  As
several people have suggested, I think if folks have something more to 
say
and are genuinely concerned with this topic, you should probably 
attempt to
organize and document your thoughts in an Internet-Draft and post it
accordingly, else it'll likely be lost in the obvious rumble of 
political and
commercial agendas and egotistic chest thumping.

I know of several folks that are engaged in this process now, 
hopefully, some
good will come of it.

-danny

---

From: Eric Rosen <[email-masked]>
Date: Wed, 30 Jul 2003 13:44:40 -0400

---

From: Randy Bush <[email-masked]>
Date: Wed, 30 Jul 2003 11:37:32 -0700

interesting question, john.  how would one measure whether and how
adding complexity to the protocol decreases reliability of the
network (of course there is a large body of literature on software
compexity, but that aside)?

an academic research approach might be to measure the change in the
software complexity over a long period of time and, in parallel,
measure the reliability of the network.  but we have not done so,
and getting those measures would be difficult as would controlling
for other variables.

but, as operators, we do have some measures, e.g., how long it
takes to get an image reliable enough to deploy after a new feature
has been added.  but it will be difficult to discuss this without
getting into bashing.

so, how would you suggest we measure this?

randy

---

From: Randy Bush <[email-masked]>
Date: Wed, 30 Jul 2003 11:37:32 -0700

randy

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 30 Jul 2003 19:50:22 +0200

- - kurtis -

---

From: Fred Baker <[email-masked]>
Date: Wed, 30 Jul 2003 11:57:25 -0700

At 12:16 PM 7/30/2003 -0600, Danny McPherson wrote:
>If it's that important [and the folks they're speaking
>on behalf of have email and half a clue!], you'd think they might be capable
>of speaking for themselves

I agree with the sentiment, and at the same time I can tell you of a number 
of cases where they have chosen not to but rather to pull me (and 
presumably others) off in the hallways and say "please do <this> in the 
IETF and we will use it". I get very frustrated with them for not speaking 
for themselves, but I also know that they often feel that there is some 
competitive information that they don't want to leak. They would rather use 
some sort of surrogate.

So, for them, I would hope that this is a call to speak for themselves.

---

From: Danny McPherson <[email-masked]>
Date: Wed, 30 Jul 2003 12:16:15 -0600

-danny

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 30 Jul 2003 21:07:31 +0200

> so, how would you suggest we measure this?

What I can't figure out....

We all know that BGP doesn't scale, convergence is a serious problem, 
etc, etc.

What on earth makes us think that adding data to be distributed and 
processed will not have impact? Or make it worse? Could someone 
enlighten me?

<undue flame - so Alex can tell me to stop :-)>

Router vendors have a hard time as it is to get packets from interface 
A to interface B. The people designing these systems and providing 
feedback (so I include myself, and most people on this list) have been 
at this for more than 10 years. What we have today is the best we have 
done so far. If this would be any other business, we would have the 
government intervene (for example, look at the food scandals in Europe. 
To a large extend that is failure of the standards bodies). We have 
continuously complained of the performance of BGP (I am not as much 
long-timer as some people here, but I have probably been around longer 
than many), and yet we continue to add features. The first thing that 
amazes me is that developers of various features _even want_ them to 
use BGP, the second thing that amazes me is that we haven't started to 
see a pattern.
</undue flame>

I am sorry I can't work this out - and I have had way to much coffee 
for someone who shouldn't drink coffee at all ;-)

Last, but I think Randy have a point, we need to agree on what we are 
trying to measure first.

- - kurtis -

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 30 Jul 2003 21:07:31 +0200

- - kurtis -

---

From: Fred Baker <[email-masked]>
Date: Wed, 30 Jul 2003 11:57:25 -0700

---

From: Eric Rosen <[email-masked]>
Date: Wed, 30 Jul 2003 16:40:43 -0400

Kurt> We all know that BGP doesn't scale

Yes,  that  must be  what  prevented the  Internet  from  ever getting  very
large. ;-)

Kurt> What on  earth makes us think  that adding data to  be distributed and
Kurt> processed  will not  have impact?   Or  make it  worse? Could  someone
Kurt> enlighten me?

Let's look at two cases: 

1. The new  data is going to be  distributed to the same set  of nodes which
   currently speak BGP for the purpose of maintaining their Internet routing
   tables.  

   Assuming that there is some necessity of distributing that new data, then
   however one distributes  it, those nodes are going to  have to handle it.
   Either they  have enough capacity for  it or they don't.   Using a second
   protocol does not reduce the distribution or the capacity load. 

2. The  new data is distributed  to a different  set of nodes than  the ones
   which currently speak  BGP for the purpose of  maintaining their Internet
   routing tables. 

   In this case, the use of BGP by  the nodes which need the new data has no
   impact  on the nodes  which use  BGP to  maintain their  Internet routing
   tables. 

So in neither case do I see what this terrifying impact is. 

Now for a particular kind of data, you may want to argue that BGP is not the
appropriate mechanism  for distributing  that kind of  data.  But  to decide
that you need to look at the details of the particular proposal.

Kurt> The first thing that amazes  me is that developers of various features
Kurt> _even want_ them  to use BGP, the second thing that  amazes me is that
Kurt> we haven't started to see a pattern. 

I guess not everyone agrees with you that

---

From: Eric Rosen <[email-masked]>
Date: Wed, 30 Jul 2003 16:40:43 -0400

Let's look at two cases:

---

From: Geoff Huston <[email-masked]>
Date: Thu, 31 Jul 2003 07:02:14 +1000

>Yes,  that  must be  what  prevented the  Internet  from  ever getting  very

I suspect in this comment that Eric is  skipping across some amazing efforts
by a small pool of folk to reign in some of the more extreme forms of
deployed routing abuse (no, I do not count my own small efforts in this pool
of individuals). Keeping the network vaguely stable at it's current size
is taking increasingly large amounts of manual effort, and no, this
cannot scale indefinitely.

There _are_ problems in BGP deployments, and overloading the running
instance of the protocol to carry more information related to various overlay
signalling applications does not in itself assist in improving the situation.
For me its a counter-intuitive action that increases the level of operational
reliance on a single element of the network. But as Randy and Kurt have
said this already there's no point in my repeating it.

Geoff

---

From: Geoff Huston <[email-masked]>
Date: Thu, 31 Jul 2003 07:02:14 +1000

Geoff

---

From: Dorian Kim <[email-masked]>
Date: Wed, 30 Jul 2003 17:31:16 -0400

On Wed, Jul 30, 2003 at 07:50:22PM +0200, Kurt Erik Lindqvist wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 

Along the same vein, I wonder how many other operators started out with
a strong stance against overloading of BGP, and after years of tilting
at that windmill and observing that vendors continues to pile on new
"feature" after "feature" into BGP, just gave up?

I certainly have. As an operator stability and optimal performance of
BGP implementations are of upmost concern for me, and there was a time
that many others appeared to feel the same. Afterall, MSDP didn't end
up in BGP and became a separate protocol because of that concern.

However, since then I've seen vendors continually ignore our concern
and input and overload other features into BGP. 

There are those who say that it's operators' need to deploy new services
that's been causing this, but from my perspective, when/where we do use
new gadgets in BGP to rollout new service deployments it's because vendors
have already put those gadgets into their BGP implementation despite my 
feedback, and there's only so much time and energy we can spend bashing
our heads against the brick wall. Afterall, we have a network to run and 
services to deploy.

So, is it really that operators all want to pile everything on BGP, or
is it that they don't see any alternatives?

-dorian

---

From: Eric Rosen <[email-masked]>
Date: Wed, 30 Jul 2003 17:22:30 -0400

Geoff> There _are_ problems in  BGP deployments, and overloading the running
Geoff> instance of the protocol to carry more information related to various
Geoff> overlay  signalling  applications  does   not  in  itself  assist  in
Geoff> improving the  situation. For me its a  counter-intuitive action that
Geoff> increases the  level of operational  reliance on a single  element of
Geoff> the network. 

This  is all  true,  but it  seems like  a  deployment issue  rather than  a
protocol issue.

---

From: Eric Rosen <[email-masked]>
Date: Wed, 30 Jul 2003 17:22:30 -0400

---

From: Dorian Kim <[email-masked]>
Date: Wed, 30 Jul 2003 17:31:16 -0400

-dorian

---

From: Henry Kilmer <[email-masked]>
Date: Wed, 30 Jul 2003 13:08:00 -0400

>expecially given that the ietf is supposedly a forum of individual

Amen.  Although I don't think the operator to vendor ratio within the
IETF forums are what they should be to produce the best results.

-Hank

---

From: Henry Kilmer <[email-masked]>
Date: Wed, 30 Jul 2003 13:08:00 -0400

-Hank

---

From: Henry Kilmer <[email-masked]>
Date: Wed, 30 Jul 2003 12:59:41 -0400

You're asking for the URL for all of the NOCs of the major ISPs.

---

From: Henry Kilmer <[email-masked]>
Date: Wed, 30 Jul 2003 12:59:41 -0400

---

From: Alex Zinin <[email-masked]>
Date: Thu, 31 Jul 2003 00:27:50 -0700

I guess I should start with this one:

Tuesday, July 29, 2003, 10:59:14 PM, Pedro Roque Marques wrote:
[...]
> I confess that i'm not quite sure what your goal is...

For a long time I have been hearing concerns about mmmm... ever
increasing popularity of BGP and routing protocols in general as one
and the only answer to any problem involving distribution of
information among routers (whether routing-related or not), a the
potential impact of this on stability of BGP holding the Internet
together.

I've also seen folks refusing to accept the possibility that the above
may cause concerns. I've also been observing vendors hard-pressed by
dead-lines and competition to put more applications on top of BGP. As
you said, there is "a fundamental difference in perception that we
have".

The question, however, is very important, and needs to be discussed in
the open. My goal for this discussion is that we come out with a clear
sense of direction. If we are fine with what's happening to RPs (and
BGP in particular)--we'll keep walking down the same path. However, if
this is not the case, we better have a very good idea where we want to
be long term and what our next steps should be.

---

From: Alex Zinin <[email-masked]>
Date: Thu, 31 Jul 2003 00:27:50 -0700

---

From: Alex Zinin <[email-masked]>
Date: Thu, 31 Jul 2003 00:28:10 -0700

John,

Tuesday, July 29, 2003, 6:19:25 PM, John G. Scudder wrote:
[...]

> resort to the ambiguous term "non-routing".

Sure.
The discussion started with two VPN-related drafts (they had different
names then):

  draft-ietf-l3vpn-bgpvpn-auto-00.txt and
  draft-ietf-l2vpn-vpls-bgp-00.txt

At the IETF meeting in Vienna, we had some more BGP-based proposals,
that are not related to distribution of {prefix, next-hop,...} tuples.

If wanted to formalize when I see concerns being raised, I would
probably use the results of the discussion we had on the RTG
directorate:

   There are types of information that are fair to distribute in
   routing protocols:

      a) information required to calculate the routing tables

      b) route tagging, administrative, and policy-related information

      c) routing security-related information

      d) information closely related to routing, especially when
         synchronization with routing information is needed

   Others would be considered non-routing.

Alex

---

From: Alex Zinin <[email-masked]>
Date: Thu, 31 Jul 2003 00:29:25 -0700

> which depends on several factors including the stability of router

Yes. See my message on "the goal".

> as equipment which is bought and operated by a telecomunications

Both views are valid, and both need to be kept in mind.
We can't look at the routers in a SP network only within the contents
of that given SP. They are also part of the Internet infrastructure,
and we need to keep this picture in mind too.

> point of view of a service provider.

If by "from the POV of a service provide" you mean within the context
of its network only, then I would disagree. This would merely ignore
the bigger picture.

[part on impact on SW taken to a separate thread]

> standardize the IP protocol suite regardless of the application.

However, issues of the scale of the Internet should be kept in mind,
and, frankly, I'm a bit surprised I find myself having to convince
_routing_ folks on this...

Alex

---

From: Alex Zinin <[email-masked]>
Date: Thu, 31 Jul 2003 00:29:25 -0700

Alex

---

From: Alex Zinin <[email-masked]>
Date: Thu, 31 Jul 2003 00:29:38 -0700

Pedro,

> What is published as RFCs doesn't reflect my definition of what is

what about the second part of the suggested definition?

> published as RFCs in order to facilitate interoperability ?

I don't think the IESG could do that, because until the document is
past the WG progress and the IETF Last Call, we can't state that
there's a consensus on it.

> a new 'item' that needs BGP.

One of the suggestions that was being discussed in rtg-dir and ops-dir
is to have the proponents of the protocol extension explain why it
belongs in a particular routing protocol, and require consensus that
putting it there is the right approach.

> in BGP.

Two points here:

1. If there is a feeling that a given set of applications needs
   a separate protocol, we should do so.

2. Note how your logic wrt "belong in BGP" is completely reversed
   compared to the suggestion I mentioned above. I.e., you suggest
   to explain why something does NOT belong in BGP, while the
   suggestion is to have people explain why it should go there in
   the first place. I personally believe (and it seems to be the
   consensus in the rtg-dir and ops-dir) that the latter makes much
   more sense.

> Would that be easier than to debate about a future proof 

It does not seem to me that one excludes the other.

One of the attractive properties of the MP-BGP framework is that you
can quickly distribute new types of information among the routers
without having to specify and implement new protocol machinery. A
generalized transport protocol would provide the same possibility.

Alex

---

From: Alex Zinin <[email-masked]>
Date: Thu, 31 Jul 2003 00:29:38 -0700

Pedro,

Two points here:

Alex

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Thu, 31 Jul 2003 09:33:10 +0200

> Yes,  that  must be  what  prevented the  Internet  from  ever getting 

BGP convergence is slow, due to a number of reasons, the size of the 
routing table being one of them. Attributes is another.  Have you ever 
tried to deploy a VoIP network over several AS boundaries? A VoIP 
network that carries live, real paid for, regulated voice traffic? 
These are real operational problem faced by operators today.

>    protocol does not reduce the distribution or the capacity load.

No. But it will have impact on the other services carried in that 
protocol. What you are saying is that bGP is merely a data distribution 
protocol and that the receiving node will have to hand the data off to 
each process for handling and processing. If this really is what you 
want, why not make it even more generic than what BGP is today? I most 
certainly disagrees with the first approach.

>    impact  on the nodes  which use  BGP to  maintain their  Internet 

I am not arguing that we should not distribute this information. I am 
arguing that we should not mix it all up in one protocol.

> that you need to look at the details of the particular proposal.

So what are the limits in your opinion on what should not go into BGP?

> I guess not everyone agrees with you that

If we all agreed we would not have this debate :-)

- - kurtis -

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Thu, 31 Jul 2003 09:33:10 +0200

- - kurtis -

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Thu, 31 Jul 2003 09:34:50 +0200

> This  is all  true,  but it  seems like  a  deployment issue  rather 

The deployment problems are symptoms of the protocol problems.

- - kurtis -

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Thu, 31 Jul 2003 09:34:50 +0200

- - kurtis -

---

From: Alex Zinin <[email-masked]>
Date: Thu, 31 Jul 2003 00:28:10 -0700

John,

Sure.

Alex

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 31 Jul 2003 10:36:55 +0200

Alex,

Last attempt to switch this tread to the right tracks ...

>    Others would be considered non-routing.

IMHO the best thing that can be done today by RTG directorate/ADs/IETF
is to mandate the clear separation of so called "non-routing"
information being distributed into their own AFI/SAFIs. That for one
will long term provide easy migration into a new transport (when such is
available & deployed) as well kind of "force" better software
engineering and code architecture.

It could be said that it is recommended by each implementation to have
such a new AFI/SAFIs separated into their own RIBs, their own
maintenance & operation routines given AFI/SAFI specific and be
demultiplexed by general BGP inbound process based on the AFI/SAFI
values very early. 

Looking realistically otherwise what we will see soon would an explosion
of extended/flexible communities or new attribiutes use under current
AFI/SAFIs which may in fact have very negative impact on current
Internet stability.

Cheers,
R.

PS. 
> > If I have BGP free core, and I don't type by accident "router bgp ..." I
> > don't really see a good reason to buy routers which do not support BGP
> > just so I don't have it in the core. If I have IPv4 ASBRs and i am not
> > keen on running on them 2547 just don't configure vpnv4 AF there. I am
> > yet to see a one single case where new opaque to existing functionality
> > code presence if not configured or enabled by user caused any trouble to
> > anyone.
>   
> I have to say it's not that simple.
> 
> There are reasons (beyond required flash mem and DRAM size) for
> existence of feature sets in SW of at least one major vendor. There
> were also reasons why one major vendor had to maintain two code
> trains--one with 2547, and another where SPs asked to please "don't
> even think about touching BGP!"

Adding support for MP-BGP required code modifications. And that has
nothing to do with introduction of 2547. For vendors starting later they
just provided MP-BGP support since day one. Unfortunately one major
vendor did not have a crstal ball few years back to see the future and
desing the entire code already fitting 2010 requirements. 

In other words as I think Yakov alreay mentioned in Vienna what is
happening to BGP is just a regular protocol evolution and improvemnt. I
don't see anything wrong with this. In fact I would highly encourage
constructive discussion (piece started above reg AFI/SAFI separation)
what can be done to make the protocol evolution smooth and least painful
for operators. That is what I would call "Engineering" meaning of "E".

What on the other hand I see to be the real goal of this discussion is
to do revolution. While revolution leaders get much more glory and
reputation if revolution suceeds history shows that when revolutions
fail they quickly also become part of the dark sides of the history.
Also the idea of centrally deciding what is good and what is bad should
to at least two of us remind something :).

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 31 Jul 2003 10:36:55 +0200

Alex,

Cheers,
R.

PS.

---

From: Pedro Roque Marques <[email-masked]>
Date: Thu, 31 Jul 2003 03:17:05 -0700 (PDT)

Alex Zinin writes:

> Sure.  The discussion started with two VPN-related drafts (they had
> different names then):

> that are not related to distribution of {prefix, next-hop,...}

draft-ietf-l2vpn-vpls-bgp-00.txt is about distribution of {prefix,
next-hop} tuples at least as far as BGP is concerned.

It uses the same path selection rules than ipv4-unicast and the same
next-hop resolution rules. The forwarding engines get to interpret the
information differently although.

In fact, the forwarding engine control code changes to support such
application are much more extensive than routing ones, and that is not
going to change regardless of what control protocol is used.

Well, i guess you can make it worse by requiring a new control
protocol to be developed for this application... and requiring network
ops people to learn a new protocol, which btw is a much higher cost
than control protocol developement.

Doesn't implementing vpls raise the possibility of introducing
regressions in the code ? It certainly does.

That doesn't make "don't implement vpls" a productive discussion and
doesn't mean that implementing it via a different control protocol is
going to be any less harmful to system stability.

  Pedro.

---

From: Pedro Roque Marques <[email-masked]>
Date: Thu, 31 Jul 2003 03:17:05 -0700 (PDT)

Pedro.

---

From: Russ White <[email-masked]>
Date: Thu, 31 Jul 2003 08:34:51 -0400 (EDT)

> > Kurt> What on earth makes us think that adding data to be distributed
> > and processed will not have impact?  Or make it worse? Could someone
> > enlighten me?

> > Eric> Let's look at two cases:
> >
> > 1. The new data is going to be distributed to the same set of nodes
> > Assuming that there is some necessity of distributing that new data,
> > then however one distributes it, those nodes are going to have to
> > handle it. Either they have enough capacity for it or they don't.
> > Using a second protocol does not reduce the distribution or the
> > capacity load.
>

How is carrying it in another protocol going to keep the new data being
carried and processed from impacting the old data being carried and
processed? It's not. Unless you're going to argue that adding an OSPF
process to a router running BGP doesn't impact BGP processing in any way.

Given infinite bandwidth, memory, and processing, you can have two
protocols running on the same machine that do not impact each other. When
you're willing to buy links with infinite bandwidth, and machines with
infinite processing and memory, let us know. In the mean time, it's a not a
matter of "will this impact that," because it will. It's a matter of best
deciding how to share the resources which actually exist.

We need to examine each case of what to carry inside BGP independantly. The
basic problem is that in each case, the ones who want the new information
to be carried without waiting on a new working group to form up, and a new
protocol to be deployed, will always claim it is critical, while those who
are don't want it, will consider it bloat. Weeds are always just flowers in
the wrong part of the yard, after all.

My general take: It might be nice to have a new protocol to carry stuff
around between routers in. Especially if that stuff doesn't relate to
routing. My questions, though, are:

-- How long will it wait to develop, deploy, and debug, the stuff protocol?

-- Will the providers stop asking for new features that require carrying
non routing information between routers until the stuff protocol is fully
deployed?

I think the answers to these two questions make it apparent that it's
useful to create such a stuff protocol, but we need to think realisitically
about:

-- What do we mean by "non routing stuff?"

-- What are we going to do with "non routing stuff" in the meantime?

-- What do we do when some mission critical app rolls out on the stuff
protocol, and people start calling for us not to change it any more, since
it will introduce stuff instability, and thus we need a "stuff 2" protocol
to break out the "new stuff" from the "old stuff?"

-- Should, if we do a stuff protocol, go back and rip the "nonrouting
stuff" out of BGP, if we can actually determine what that is, and put it in
the stuff protocol?

So, I'm undecided on the issue. I think there are advantages and
disadvantages to both ways of doing things--I hate the mess that HTTP has
become, but I also don't want to see us with stuff 2, and stuff 3, and
stuff 4--and any argument leveled against putting x in BGP can also,
probably, be used as an argument against putting something new in the stuff
protocol, once it's developed and deployed.

Further, I also see Eric's point that all processes on a device interact,
no matter whether we like it or not, and deploying a new protocol is
probably just about as risky as touching old, well known code (which is
going to be touched to fix things, and add things, anyway).

Maybe it's worth splitting off a bof for some time just to talk about this,
and come back with some sort of an answer on these sorts of questions? So
we'd have some concept of the practicality of designing and depoying a
"stuff protocol," and some concept of the risks involved?

:-)

Russ

__________________________________
[email-masked] CCIE <>< Grace Alone

---

From: Russ White <[email-masked]>
Date: Thu, 31 Jul 2003 08:34:51 -0400 (EDT)

:-)

Russ

---

From: Russ White <[email-masked]>
Date: Thu, 31 Jul 2003 08:42:59 -0400 (EDT)

> infinite processing and memory, let us know. In the mean time, it's a not
> a matter of "will this impact that," because it will. It's a matter of
> best deciding how to share the resources which actually exist.

BTW--this isn't meant to be mean, just a realistic assessment of the 'way
things are.'

> useful to create such a stuff protocol, but we need to think
> realisitically about:

It _might_ be useful.

:-)

Russ

---

From: Russ White <[email-masked]>
Date: Thu, 31 Jul 2003 08:42:59 -0400 (EDT)

It _might_ be useful.

:-)

Russ

---

From: Eric Rosen <[email-masked]>
Date: Thu, 31 Jul 2003 10:56:00 -0400

Pedro> draft-ietf-l2vpn-vpls-bgp-00.txt  is about  distribution  of {prefix,
Pedro> next-hop} tuples at least as far as BGP is concerned. 

No it  isn't. The data  which is  proposed there to  be carried in  the NLRI
field is nothing at all like an address prefix. 

L2VPN discovery and L2VPN signaling  (including VPLS) are pretty clear cases
where the  data proposed to be carried  by BGP is "non-routing".   It may or
may not be  appropriate to carry such data in BGP  (personally I think "yes"
in the discovery case  and "no" in the signaling case), but  I don't think a
defense of draft-ietf-l2vpn-vpls-bgp can be  mounted on the basis that it is
just using BGP to distribute routing info.

---

From: Eric Rosen <[email-masked]>
Date: Thu, 31 Jul 2003 10:56:00 -0400

---

From: Chandrashekhar Appanna <[email-masked]>
Date: Thu, 31 Jul 2003 09:42:11 -0700

>    to explain why something does NOT belong in BGP, while the

  I suggest that because this entire topic/discussion seems to
  be primarily driven from the point of view that BGP is being
  used to carry information that it should not be carrying.

>    suggestion is to have people explain why it should go there in

  I agree with the above statement too. To me they are the same
  thing. A draft being put out => the authors think it should belong
  in BGP. IDR does discuss the merits/demerits of every draft anyway.
  
  -
  Chandra.

---

From: Chandrashekhar Appanna <[email-masked]>
Date: Thu, 31 Jul 2003 09:42:11 -0700

-
  Chandra.

---

From: Mark Prior <[email-masked]>
Date: Fri, 01 Aug 2003 08:50:29 +0930

> providers, and I won't pretend to speak for all vendors ;-)
Having Randy speak for me seems to be a terrifying thought :-) so I will
speak for myself and say on this issue I agree with Randy.

Mark.
(also an operator)

---

From: Mark Prior <[email-masked]>
Date: Fri, 01 Aug 2003 08:50:29 +0930

Mark.
(also an operator)

---

From: Alex Zinin <[email-masked]>
Date: Thu, 31 Jul 2003 18:32:18 -0700

Robert,

> Last attempt to switch this tread to the right tracks ...

[I'll take this part to a separate thread together with Russ'es
suggestion, since I see some good suggestions that we should discuss
in them]

> to at least two of us remind something :).

and notice that we are not deciding what's good and what's bad
centrally (pushing down the directives from the IESG/ADs/RTG-DIR,
etc.), but instead are heaving an open discussion first at the routing
area meeting and then here on the list. No desire for a revolution,
but a huge desire to get a closure on this topic and move on.

Thanks.

Alex

---

From: Alex Zinin <[email-masked]>
Date: Thu, 31 Jul 2003 18:32:18 -0700

Robert,

Thanks.

Alex

---

From: Avri Doria <[email-masked]>
Date: Fri, 1 Aug 2003 11:54:21 +0900

> putting it there is the right approach.

This, at a minimum seems a very good start.
Anyone who think that a new feature belongs in the protocol should be
able to present a cogent and convincing argument for that new feature.

This may do nothing to relieve the bloat, but it will hopefully prevent
it from getting worse.

> generalized transport protocol would provide the same possibility.

And this can be expected to be part of someone's presentation on why add
new functionality.

> a.

---

From: Avri Doria <[email-masked]>
Date: Fri, 1 Aug 2003 11:54:21 +0900