Subject: Re: Extending BGP

From: Yakov Rekhter <[email-masked]>
Date: Wed, 06 Aug 2003 07:40:34 -0700

Alex,

> Eric,
> 
> > I wasn't in Vienna, but let me put in my two cents on why it sometimes makes
> > sense  to use BGP  for distributing  non-routing information.   Basically, I
> > think you  need to look at  the specific requirements  for distributing that
> > information.   (Yes,  I  know  this  isn't  as  much  fun  as  making  broad
> > generalizations ;-))
> 
> > Consider a set of routers S such that: 
> 
> > - Members of S know apriori that they are members of S, but they do not know
> >   apriori which other routers are members of S. 
> > - S is divided into subsets (not necessarily disjoint) such that all members
> >   of a subset share a certain attribute.  Thus for attribute A, we can speak
> >   of the A-subset of S.  
> > - Each member  of S is configured with  a set of "export  attributes", and a
> >   set of "import attributes". 
> > - If a router in S is configured  with export attribute E, it is a member of
> >   the E-subset. 
> 
> > - If a router in S is configured  with import attribute I, it needs to learn
> >   the membership of the I-subset. 
> > - The  set  of  import and/or  export  attributes  with  which a  router  is
> >   configured  may change  at  any time;  changes  in the  membership of  the
> >   various A-subsets should be learned within minutes.
> > - The changes do not occur with great frequency. 
> > - The total number of attributes under consideration may be very large. 
> > - Members of  S cannot be assumed to  be in the same  Autonomous System, nor
> >   can  they necessarily  be assumed  to  be under  the control  of the  same
> >   administration.
> >
> > Now  we  need  a  protocol  which  detects  changes  in  the  A-subsets  and
> > distributes these changes.  What requirements can be put on such a protocol:
> > - It has to  provide a way for a  router to announce that it  has acquired a
> >   new export attribute. (Call this an "update" operation.)
> > - It has  to provide  a way for  a router  to announce that  it has  lost an
> >   export attribute.  (Call this a "withdraw" operation.) 
> > - For  scalability, we  prefer  that routers  which  are not  in  S are  not
> >   impacted by any  of this; in particular, they should  not need to maintain
> >   any state which is specific to S or any of its subsets.
> 
> > Since the members of S do not know each other apriori, one might expect that
> > there will be a server involved, where  the members of S talk to the server,
> > and the  server redistributes the  information as necessary.  Of  course, we
> > usually want  to have each router  talking to two servers,  with the servers
> > also updating each other.  And to make sure the whole system scales up, we'd
> > like to be able to have  servers with different sets of "clients" talking to
> > each  other.  Since  we may  be passing  large amounts  of  information over
> > arbitrary  network  paths, we  need  to  have  a reliable,  flow  controlled
> > transport, and it would be  nice if the transport handled congestion control
> > as well.  A  point-to-point transport between clients and  servers seems the
> > best fit,  given that we  don't want to  burden any of S's  non-members with
> > S-specific state. 
> 
> > To further  improve scalability, we'd  like to be  able to have  each client
> > tell its server(s) its set of import attributes, so that a given router only
> > receives the  information in which  it is interested.  (Call  this "outbound
> > attribute filtering".)  If  we have servers talking to  servers, it would be
> > nice to have  the outbound attribute filtering passed  around the servers in
> > some loop-free manner, but we can leave that for further study. 
> 
> Thanks! Really great summary of the required functionality!
> This is at least what I would have in mind.
> 
> > If  you  put all  these  requirements together,  and  look  for an  existing
> > protocol to satisfy them, you are  inevitably led to BGP.  The salient issue
> > is  not whether  the attributes  that  need to  be distributed  are part  of
> > "routing",  but  whether the  requirements  for  the  distribution of  these
> > attributes  are very  similar to  the requirements  for the  distribution of
> > inter-domain routing information.  We are led to BGP because of a similarity
> > in the distribution requirements, not necessarily because of a similarity in
> > the information distributed.
> 
> > Many alternatives  have been suggested  in the past,  but most of  them just
> > make BGP look even better ;-)
> 
> And that's why, if we follow the direction of instantiating a separate
> protocol, it would most probably inherit a lot from MP-BGP. I.e., the
> idea is that it would have a very similar framework, though work over
> a different TCP port number, and would live its own life with
> independent (from BGP) features, extensions, optimizations, etc.
> 
> [from your thread with Jon:]
> 
> Jon>> for  a  start you  create  other reasons  why  BGP  speakers might  get
> Jon>> overloaded,  might get  attacked,  might get  misconfigured, and  might
> Jon>> never converge 
> > 
> > Any  time you use  a protocol  to distribute  information, you  create these
> > possibilities, it's nothing to do with what protocol you're using. 
> 
> I think the argument here is about risk assessment and management.
> 
> Though it is possible to do a fine job of clearly separating different
> applications of the same protocol within the system, using a separate
> protocol makes it much more likely to happen.

Following your logic one should have a separate protocol per
application, each with "its own life with independent (from BGP)
features, extensions optimizations, etc." 

Yakov.

_______________________________________________
routing-discussion mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/routing-discussion

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 06 Aug 2003 07:42:55 -0700

Alex,

[clipped...]

> protocol makes it much more likely to happen.

Since you agree that it is quite possible "to do a fine job of
clearly separating different applications of the same protocol within
the system", let me suggest that the folks who are capable of
doing such "fine job" should continue doing this, while the folks
who need a separate protocol to make this happen should pursue the
avenue of the separate protocol per application.

Yakov.

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 06 Aug 2003 07:42:55 -0700

Alex,

[clipped...]

Yakov.

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 06 Aug 2003 07:40:34 -0700

Alex,

Yakov.

---

From: Alex Zinin <[email-masked]>
Date: Thu, 7 Aug 2003 17:59:27 -0700

Let's keep the original concerns in mind.

> features, extensions optimizations, etc." 

To repeat what I've said already, one of the major concerns is
stability of Internet routing. Separating routing from non-routing
applications is the main thing we are talking about here, but one can
certainly follow any logic to an absurd extreme.

> avenue of the separate protocol per application.

The issue under discussion may affect very wide Internet community,
and I believe that shrugging it off in a manner "if you believe
there's an issue--it's your problem, feel free to solve it" would not
be appropriate and would only further polarize the community. We're
here to discuss the issue in the open and work out the direction for
future efforts.

Alex

---

From: Alex Zinin <[email-masked]>
Date: Thu, 7 Aug 2003 17:59:27 -0700

Alex

---

From: Pedro Roque Marques <[email-masked]>
Date: Fri, 8 Aug 2003 00:42:44 -0700 (PDT)

> stability of Internet routing.

If that is the goal or one of the goals you are trying to pursue, it
would seem to me that you would have to go through the process of
analyzing what it depends on and gather data as to what causes of
instability. Most service providers keep information of which defects
cause faults in their network, for instance, and many times have
information about their root causes.

However, imho, the reasons behind this discussion have nothing to do
with extending routing protocols and much less how to do so. One need
only keep in mind that the characteristics of unextended-BGP are much
more of an issue in operational networks than any extensions.

IMHO, the real issue behind this debate is that services providers are
themselfs internally divided about issues of future direction and
service offerings. There are people that believe in an IP-centric
service offering and/or infrastructure and people that believe that
there is the requirement to converge network infrastructure and/or
provide additional services.

Some in the IP-centric camp are attempting to get a statment from the
IETF that going against their particular view of the world threatens
the stability of the internet.

I personally have no idea which faction is correct in terms of
direction given it is essentially a commercial decision based on data
which i don't have access to.

But it seems to me that it is a rather poor idea to attempt to settle
it in the IETF precisely because it is a commercial decision. And one
that is particular to each service provider.

Lets assume for a moment that the IETF publishes a document which says
that "multi-service offering networks" are a treat to internet
stability or something of the sort. How is that going to fly on a
discussion with management or the marketing department on a given
carrier/provider ? Does anyone really intend to suggest to their
marketing department to be really creative and sell the fact that they
don't provide a given service offering ?

The "stability of the internet" is a scare tactic that simply won't
fly with management on any service provider, imho.

The IETF can and should address engineering problems... in this case
it means make the upmost to make services successful and
deployable. Which is the exact opposite of what the people that don't
"want to see any code changes" are looking for...

Providing fault isolation in a multi-service router is something that
probably most vendors are working on/have products/have powerpoint. It
is however a problem that doesn't start or end in BGP.

This is also an issue that service providers have been working on with
vendors for a considerable amount of time, in a constructive way and
with demonstrable results.

If people want to work on this particular goal i would suggest
starting w/ the requirement lists that have been developed by several
SPs. Some of these are already quite mature by now.

  Pedro.

---

From: Pedro Roque Marques <[email-masked]>
Date: Fri, 8 Aug 2003 00:42:44 -0700 (PDT)

Pedro.

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 8 Aug 2003 10:07:22 -0400

> information about their root causes.

Seeing pointers to some of these would be good.  Pretty much every
paper I've seen at NANOG has been roughly along these lines:

most failures: hardware outages or telecom outages
<20%: operator's twiddling something wrong
<3-5%: software/protocol problems

We can only address the last one.

#include <vijay_gill.h> :-)

> service offerings.

Very polite of you to put it this way. :-)

> which i don't have access to.

IMO, the problem isn't so much whether one faction is more "correct"
than the others.  The issue is that each party has valid reasons for
wanting to do what they want to do.  The problem is that the faction
that wants changes is the one that is *potentially* destabilizing.

Or, to channel Randy and Vijay a little bit, many operators want
their networks to work stably and well first and foremost.  They
are convinced for one reason or another that the software quality
is such that it isn't as good as it should be and we should concentrate
on fixing bugs rather than adding features.  Adding features adds
more bugs.

Push this one level further up towards IETF, these people are convinced
that taking a protocol that has known operational headaches will
get new features and add more operational headaches.

Address this issue and I think we're good.

> that is particular to each service provider.

While we aren't a standards body, we get treated that way by people
who test the hardware operators use.  If we're broken by design
(for some metric of broken), people are going to get cranky for good
reason.

> Does anyone really intend to suggest to their

As the saying goes, "New features sell.  Stable code doesn't."

> "want to see any code changes" are looking for...

I think you're seeing Alex state this as an extreme.  I don't think
this is the case.

What I see Alex calling for is all about deployment.  How can we
deploy new stuff without destabilizing old stuff?

> is however a problem that doesn't start or end in BGP.

As I said in a previous e-mail - all eggs, one basket.

> SPs. Some of these are already quite mature by now.

The trick is pulling those out of NDA enough to get them published
in a sufficiently abstract fashion as to make them happy.

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 8 Aug 2003 10:07:22 -0400

-- 
Jeff Haas 
NextHop Technologies

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 11 Aug 2003 07:46:52 -0700

Alex,

> stability of Internet routing. 

To repeat what I've  (and few other folks) asked already, are we
talking about stability of IPv4 unicast Internet routing ? Or IPv6
unicast Internet routing ? Or IPv4 multicast Internet routing ? Or
IPv6 multicast Internet routing ?

> Separating routing from non-routing
> applications is the main thing we are talking about here, 

This is *one* of the things we are talking about here. But not the
only one.

We've yet to see any *rationale* explanation from the proponents of
"separating routing from non-routing applications" on why such
separation is sufficient, and per application separation is not
needed.

> but one can

I have nothing against discussing "the issue in the open". 

Ditto for "work out the direction for future efforts". 

In fact, the above (from my original message) is my proposal for 
the direction:

   the folks who are capable of doing such "fine job" should continue
   doing this, while the folks who need a separate protocol to make
   this happen should pursue the avenue of the separate protocol
   per application.

Yakov.

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 11 Aug 2003 07:46:52 -0700

Alex,

Yakov.