Subject: Re: [Idr] Progressing draft-bhatia-bgp-multiple-next-hops-01.txt

From: Paul Jakma <[email-masked]>
Date: Fri, 1 Sep 2006 08:52:11 +0100 (IST)

Hi Enke,

On Thu, 31 Aug 2006, Enke Chen wrote:

> In draft-walton-bgp-add-paths-03.txt, there was a "Path Identifier 
> Type" defined. With the Path Identifier Type, a BGP speaker will be 
> able to tell if a path identifier is a natural id (such as 
> router-id), and will not re-generate one for ibgp.

That seems an idea.

Though, its redundant information in many cases (e.g ORIGINATOR_ID, 
NEXT_HOP).

> We will clarify this point in the next revision.

Also, all contiguous paths of add-path speakers would have to be 
configured to use the same add-path-id type.

regards,
-- 
Paul Jakma	[email-masked]	[email-masked]	Key ID: 64A2FF6A
Fortune:
If it has syntax, it isn't user friendly.

_______________________________________________
Idr mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/idr

---

From: Paul Jakma <[email-masked]>
Date: Fri, 1 Sep 2006 08:03:45 +0100 (IST)

On Thu, 31 Aug 2006, Pedro Roque Marques wrote:

> I'd like to suggest that rough consensus has been reached that the 
> next-hop attribute is not a viable path identifier. Except for the 
> authors, is there anyone that doesn't consider this to be a "self 
> evident truth" by now ?

So which identifier would work?

You need to pick a stable identifier..

regards,
-- 
Paul Jakma	[email-masked]	[email-masked]	Key ID: 64A2FF6A
Fortune:
In just seven days, I can make you a man!
 		-- The Rocky Horror Picture Show

---

From: John Leslie <[email-masked]>
Date: Fri, 1 Sep 2006 04:50:50 -0400

> authors, is there anyone that doesn't consider this to be a "self 

   Yes.

--
John Leslie <[email-masked]>

---

From: "Glen Kent" <[email-masked]>
Date: Fri, 1 Sep 2006 21:43:53 +0530

Enke,

> In draft-walton-bgp-add-paths-03.txt, there was a "Path Identifier Type"
> defined. With the Path Identifier Type, a BGP speaker will be able to
> tell if a path identifier is a natural id (such as router-id), and will
> not re-generate one for ibgp.

So, are you saying that you are only going to use Router IDs in that case.

Didnt we see cases earlier where we asserted that using the Router IDs
alone wouldnt help? An implementation would need to generate unique
arbitrary numbers.

Glen

> -- Enke
> Paul Jakma wrote:
> > Hi Curtis,
> > On Wed, 30 Aug 2006, Curtis Villamizar wrote:
> >
> >> You can't propose a protocol extension where routes cannot be
> >> distinguished when a legitimate example is given just because the
> >> example is in your opinion "something no one would want to do".
> >
> >
> > It's not a consideration because the exit-peer can easily peer with
> > the RS in Enke's example.
> >
> > 3-p-n generally is discouraged, it has reliability problems - it's the
> > poor man's substitute for direct peering (usually administrative
> > overhead of setting it up). Certainly at IXes we know of it is
> > generally discouraged.
> >
> > Its use tends to be primarily because of the /lack/ of eBGP RS solutions.
> >
> > We'd be interested in any plausible cases where a peer would /not/
> > want to advertise the path to the RS, but would be happy for other
> > peers to advertise the path to the RS (and hence all RS-client peers).
> > We can't quite think of them.
> >
> >> It is true that most peerings use "next hop self" at an exchange
> >> where a route server is located, however if two peers did not set
> >> next hop self and provided third party routes with the same next hop
> >> to the RS, there would be no way to make these routes unambiguous.
> >
> >
> > There's no ambiguity, the RS just doesn't accept paths from peers
> > which won't forward the path. That's not a problem a route-server is
> > there to solve.
> >
> > However, if the exit peer wants to peer with the RS, and hence
> > *remove* the need for others to advertise path with it as nexthop,
> > then that is exactly what the RS is there for, that's exactly the
> > problem the IX operators /want/ solved (they don't care about 3-p-n at
> > all).
> >
> > If not, then *neither* add-path nor multiple-hop can help the peers
> > involved btw..
> >
> >> It seems that we now have an example where the path identifier in
> >> draft-walton-bgp-add-paths-05.txt is needed.
> >
> >
> > I would tend to disagree.
> >
> >> I don't recall seeing any objections to the use of path identifier in
> >> draft-walton-bgp-add-paths that held up.
> >
> >
> > There are none against multiple-hop, except vague ones about some
> > unspecified, unknown cases (possibly future).
> >
> > E.g. where multiple-hop "restricts" paths advertisements in accordance
> > with (destination,target), the add-path approach on the other hand can
> > make it impossible to reconcile that two paths are identical[2]. In
> > other words, add-path turns BGP into a flooding protocol between
> > contiguous sets of add-path speakers[1].
> >
> > The problem is, in chasing ultimate flexibility, add-path makes itself
> > a quite inefficient flooding mechanism. BGP paths will be flooded
> > along /every/ possible path between a contiguous set of add-path
> > speakers:
> >
> > RR2
> > / | \
> > (P)A | B
> > \ | /
> > RR1
> >
> > Based on Add-Path alone, B would receive:
> >
> > (A->RR2->B): P:a
> > (A->RR1->B): P:b
> > (A->RR2->RR1->B): P:c
> > (A->RR1->RR2->B): P:d
> >
> > Now think about convergence if A withdraws P. B is going to just
> > switch between the different 'ghosts', as they are all equal bar the
> > path-ID, until /all/ are withdrawn[3].
> >
> > And that's just with two hops of add-path.
> >
> > So the flexibility is not quite free.
> >
> > --paulj
> >
> > 1. It took a while to tease out that add-path does indeed amount to
> > random local IDs, rather than distinguishable IDs derived in some
> > way though..
> >
> > 2. Other than by comparing path-attributes, but not having to care
> > about path attributes (inc. nexthop) is part of the attraction of
> > add-path.
> >
> > 3. Course, we could just say that the two RR's must never enable
> > Add-Path between each other (as is the case for reflection today),
> > but doesn't that seem restrictive? :) The Add-Path draft does not
> > disallow 2-way add-path.
> >
> > Maybe Add-Path for inter-RR could be defined to compare
> > router-ID of paths from clients against ORIGINATOR_ID of paths
> > from RRs? But that defeats the point somewhat of using a new
> > opaque ID as discriminator.
> >
> > regards,
>
>

---

From: Enke Chen <[email-masked]>
Date: Fri, 01 Sep 2006 09:24:24 -0700

Glen:

> So, are you saying that you are only going to use Router IDs in that 

It depends on the application.  There are several path-id types, including:

       o router-id
       o neighboring-AS
       o opaque number

> alone wouldnt help? An implementation would need to generate unique

For the RS that advertises all the paths to its clients (ebgp), a 
path-id will need
to be generated. For RR, it can use the router-id or neighboring-AS as 
the path
identifier inside an AS.

-- Enke

---

From: "Glen Kent" <[email-masked]>
Date: Fri, 1 Sep 2006 22:04:26 +0530

Enke,

> It depends on the application.  There are several path-id types, including:

This i understand would get negotiated during the capability exchange.
So a session that negotiates "router-id" cannot switch to opaque
number midway? Right?

> to be generated. For RR, it can use the router-id or neighboring-AS as

What if a client advertises a static route (redistributed into BGP,
for purists) with two different next-hops to the RR? Do you think
using the Router ID alone would work?

Glen

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 1 Sep 2006 13:45:57 -0400

Enke,

> It depends on the application.  There are several path-id types, including:

I think we can agree that we're trying to read the future with
regards to this future.  That makes for a lot of mail list churn.

When might we expect a respin of this document?

-- Jeff

---

From: Enke Chen <[email-masked]>
Date: Fri, 01 Sep 2006 10:37:51 -0700

> So a session that negotiates "router-id" cannot switch to opaque

It is desirable to be able to switch the types without tearing down the 
session. We will cover it in the next version of the draft.

> using the Router ID alone would work?

If the goal is to emulate full-mesh for optimal routing and route 
oscillation elimination, the router-id would be sufficient (following 
the current paradigm that each border router only sources one path). 
Otherwise, the path-id may need to be generated, and that would remain 
as an option.

-- Enke

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 1 Sep 2006 13:45:57 -0400

Enke,

-- Jeff

---

From: Enke Chen <[email-masked]>
Date: Fri, 01 Sep 2006 10:37:51 -0700

DKFrom [email-masked] Fri Sep 01 13:49:34 2006
To: Paul Jakma <[email-masked]>
Subject: Re: [Idr] draft-bhatia-bgp-multiple-next-hops vs
	draft-walton-bgp-add-paths 
Date: Fri, 01 Sep 2006 13:44:05 -0400
From: Curtis Villamizar <[email-masked]>
Cc: [email-masked], Manav Bhatia <[email-masked]>, [email-masked]
Reply-To: [email-masked]

In message <[email-masked]>
Paul Jakma writes:
>  
> > The case is where the route is not a direct customer of the 
> > provider and is not getting transit to the IX through that provider 
> > but is peering with another major provider with agreement to 
> > exchange all routes.
>  
> That sounds an interesting case. Could you provide a diagramme for 
> clarity? Is it:
>  
>  	------------- IX fabric
>            |   |   |
>  	 P1   P2  P3
>             \ /
>              C

Here is the example:

                      AS2 -----------------------
                       |                         \
   dest -- AS1 -IBGP- AS1 --| blocked by policy  - RS => others
   ASe                 |                         /
                      AS3 -----------------------

The AS path from AS1 would be AS1 ASe, 2 AS in the path.  AS1 and AS2
direct peer.  AS1 and AS3 direct peer.  The AS path from AS2 and AS3
area AS2 AS1 ASe and AS3 AS1 ASe, both three hops.  The RS would pick
the shortest AS path.  In this case AS1 does not export this route and
so there is a tie between AS2 and AS3 routes.  They both have the same
next-hop.

This is slightly different from Enke's example in that in Enke's
example AS1 advertises directly to RS and AS3 doesn't exist.  The AS
Path lengths would be different in this case and one route would be
(should be) preferred over the other on that basis.

Joel has been arguing that maybe we want to include the path not taken
and mark it as not taken.  Maybe that would make route changes faster
but I think the cost in terms of added state is too great.

> > You've just dismissed a real case above.
>  
> Apologies, I hadn't realised the argument was operating under the 
> assumption that direct peering would be rendered illegal. ;)

For someone that doesn't understand the examples that we've been
discussing for numerous email messages you sure provide a vigorous
argument.  :-)

Curtis

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 1 Sep 2006 13:45:57 -0400

Enke,

-- Jeff

---

From: Enke Chen <[email-masked]>
Date: Fri, 01 Sep 2006 10:37:51 -0700

-- Enke

IM-Signature: a=rsa-sha1; q=dns; l=1227; t=1157132272; x=1157996272;
	c=relaxed/simple; s=sjdkim2002;
	h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version;
	d=cisco.com; i=[email-masked];
	z=From:Enke=20Chen=20<[email-masked]>
	|Subject:Re=3A=20[Idr]=20Progressing=20draft-bhatia-bgp-multiple-next-hops-01.txt;
	X=v=3Dcisco.com=3B=20h=3DD/xXM8993FtgmwQe1uOqxBX5ju0=3D;
	b=pPADZN8btROYzyfdOWAs+fwzDaKp2axP3yNLSALcXhopW37D0yviSLZSISvSLoKExR6k4gAk
	BHe5BXcUq5rHu/V6nUa7MpoNk1fEPyvir6lZCOvAs5vmqDU9PngEd+BB;
Cc: [email-masked]

-- Enke

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 1 Sep 2006 13:45:57 -0400

Enke,

-- Jeff

---

From: Enke Chen <[email-masked]>
Date: Fri, 01 Sep 2006 10:37:51 -0700

-- Enke

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 01 Sep 2006 14:11:03 -0400

> You need to pick a stable identifier..

The type of identified used in add-paths (<prefix,path-id> with
path-id unique per sender) should remain in consideration.

The next-hop as an identifier used in multiple-next-hops should no
longer be considered by the WG.

The add-path draft as it stands remains too vague and may be
misinterpreted due to omission of important details.

I suggest that NEITHER internet-draft be considered as a WG item at
this point, multiple-next-hops because it won't work and add-paths
because it is too vague as it stands.

I suggest we revisit the question as to whether to make add-paths a WG
document after the next iteration of the internet-draft.  In the mean
time, perhaps advocates of multiple-next-hops can make constructive
suggestions to improve the clarity of the add-paths draft.

Curtis

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 1 Sep 2006 14:06:38 -0400

> because it is too vague as it stands.

I heartily support this.

That said, I think that the discussion has spurred us to talk about
the problem space.  We should continue this discussion.

-- Jeff

---

From: Enke Chen <[email-masked]>
Date: Fri, 01 Sep 2006 11:21:32 -0700

Jeff:

>When might we expect a respin of this document?

Should be this month.  We will ask a few people review it before it is 
posted.

-- Enke

---

From: Enke Chen <[email-masked]>
Date: Fri, 01 Sep 2006 11:25:27 -0700

Curtis:

I agree with your assessment and suggestions.

-- Enke

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 01 Sep 2006 15:07:13 -0400

> It depends on the application.  There are several path-id types, including:

I'm sorry but I didn't see that in the -05 draft.  Where is it?

Enke,

A RR could get a multipath from either or both RR client.  Since there
is only one path-id in the route the RR advertises, using the neighbor
AS or router-id is inadequate as a general case.

It seems that the opaque number is the only identifier that works if
the multipath may consist of learned components that were already
multipaths.

Curtis

---

From: Enke Chen <[email-masked]>
Date: Fri, 01 Sep 2006 14:34:59 -0700

Curtis,

>I'm sorry but I didn't see that in the -05 draft.  Where is it?

Sorry - I did not make it clear.  It was in -03 draft, and we will need 
to resurrect it:

http://www.faqs.org/ftp/pub/pub/internet-drafts/draft-walton-bgp-add-paths-03.txt

>the multipath may consist of learned components that were already multipaths.
>  
>

Agreed -  the router-id would work only within the current paradigm that 
a client advertises its bestpath to ibgp, and that would emulate the 
full-mesh.  If more paths need to be advertised, then the opaque number 
would be necessary.

-- Enke

---

From: "Glen Kent" <[email-masked]>
Date: Sat, 2 Sep 2006 21:13:06 +0530

Enke,

> full-mesh.  If more paths need to be advertised, then the opaque number

Since we've been talking of the hypothetical all this time, lets not
assume anything. These proposals go beyond the current paradigm so
lets not mix the different paradigms - if you will. Moreover, I had
given you a valid scenario (which you ignored) where a client would
want to advertise multiple paths.

That being the case, i think the only ID that you can use is the
opaque number. Lets not run around in circles where one of us has to
remind you every now and then that the Router ID is not a valid path
distinguisher and will _not_ work.

Glen.

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 02 Sep 2006 13:15:21 -0400

> distinguisher and will _not_ work.

I suggest we suspend all discussion of add-paths since we don't know
what the authors intend.  It seems absurd for the authors to tell us
to ignore the -05 and go back to the -03.  We need to wait for -06.

In the mean time discussion of the problem space would be helpful.
It might be helpful or even necessary to write a requirements doc.

Curtis

---

From: Enke Chen <[email-masked]>
Date: Sat, 02 Sep 2006 10:33:22 -0700

Glen:

> distinguisher and will _not_ work.

As I said many times, in some applications the router-id would work 
fine, and in others the opaque numbers will be needed.  It is 
application specific and configuration driven. The spec needs to support 
different types of path-id types.

In my reply to Curtis, I listed the condition under which the router-id 
can be used as the path-id to emulate the full-mesh:

         o client advertise only the bestpath (which is the current 
paradigm). In other words, each speaker sources at most one path to 
ibgp. Therefore (prefix, router-id) would be unique.

I actually think this will satisfy the vast majority (if not all)  of 
the ibgp requirements (optimal routing, and route oscillation 
elimination).  If there is a case that more than the bestpaths need to 
be advertised by a client, then the opaque number can be used (by 
configuration).

As I also mentioned, in the next revision, the recipient will have the 
ability to know the path-id type so that it can make a decision whether 
the path-id can be re-used for an particular application.

Not sure if I missed some of your points.

-- Enke

---

From: "Glen Kent" <[email-masked]>
Date: Sat, 2 Sep 2006 23:06:05 +0530

Curtis,

> It might be helpful or even necessary to write a requirements doc.

Which i am assuming would be add-paths/mulitple-hop agnostic. Can the
(co)authors of the contending proposals co-operate here? :-)

Glen

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 02 Sep 2006 14:43:27 -0400

> (co)authors of the contending proposals co-operate here? :-)

So far I've seen the following uses brought up.

  BGP multipath

    Generally this is interpreted to mean send everything where route
    decision ties at IGP cost.  Any other interpretations?

  RR and AS confed oscillation avoidance

    This usage seems to imply sending everything where route decision
    is broken at MED comparison, sending all routes that made it to
    this comparison.

  RS sending all routes

    Send all routes with no decision process applied by the RS.

Are there any others?  Are any of these considered out of scope?

A separat e question is whether either draft was targeting one usage
but not others.  Maybe we should save that question for later.

Curtis

---

From: Enke Chen <[email-masked]>
Date: Sat, 02 Sep 2006 12:11:47 -0700

Curtis:

>    decision ties at IGP cost.  Any other interpretations?

EBGP multipath (across diff. AS) is a can of worms (-:   The requirement 
itself needs to be justified due to its complexities and implications.

IBGP multipath and RR can result in forwarding loops with certain 
topology. "send everything where route decision ties at IGP cost" by RR 
would resolve it.

-- Enke

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 02 Sep 2006 16:17:46 -0400

> itself needs to be justified due to its complexities and implications.

I should have separated IBGP multipath and EBGP multipath.

Does "can of worms" mean out of scope?  I agree that there seems to be
no serious operator interest in EBGP multipath and therefore the
requirement needs to be justified.  Even if this usage won't occur, it
might be worth producing a solution where it has been considered,
either in case it ever does get used, or just to avoid lengthy IDR
discussion of failure to address it.

If the protocol extension is capable of supporting EBGP multipath,
operators are still free not to use it.

Since IBGP multipath is commonly used, I assume it is within scope.

> topology. "send everything where route decision ties at IGP cost" by RR 

RR can cause loops without multipath.  Isn't the place to put that
"send everything" for RR before MED (RFC4271 9.1.2.2 c) rather than at
IGP (RFC4271 9.1.2.2 e).  Note that b is Origin and d is EBGP/IBGP.

btw - This would mean that a single router would still never send two
LOCAL_PREF since that is 9.1.2.2 a.  :-)

If there was no potential for RR oscillation in a given network (its
already avoidable) then sending a multipath for a tie at IGP would do
the trick.  This sounds like an implementation needs a knob and the
spec (either one) needs a SHOULD in terms of having the right knobs.

Curtis

---

From: Paul Jakma <[email-masked]>
Date: Sun, 3 Sep 2006 00:48:24 +0100 (IST)

Hi Enke,

On Sat, 2 Sep 2006, Enke Chen wrote:

> application specific and configuration driven. The spec needs to 
> support different types of path-id types.

I'm sorry, but you can't have your cake and eat it.

The discussion on problems with add-path /started/ with the problems 
of router-ID. Are we just going to gloss over those? Suggesting they 
can be worked around by allowing a speaker to still generate an 
opaque-ID if a clash arises just brings us back to the problems 
raised with that which lead you to return to router-ID.

You can't configure around it either.

> In my reply to Curtis, I listed the condition under which the 
> router-id can be used as the path-id to emulate the full-mesh:

>        o client advertise only the bestpath (which is the current paradigm). 
> In other words, each speaker sources at most one path to ibgp. Therefore 
> (prefix, router-id) would be unique.

This doesn't work for the "seperate confeds which share one IGP and 
leave external next-hops unchanged" case, which Curtis and you gave 
in an earlier email.

It's an unusual case, but it seemed important..

I am amused that after all the discussions of and objections to the 
'restrictions' of nexthop, we'd then be happy with something with 
even greater restrictions.

Met vriendelijke groeten,
-- 
Paul Jakma	[email-masked]	[email-masked]	Key ID: 64A2FF6A
Fortune:
> Can you explain this behaviour?

Yes
--
Alan

[Oh wait you want to know why...]

 	- Alan Cox on linux-kernel

---

From: Enke Chen <[email-masked]>
Date: Sat, 02 Sep 2006 17:02:27 -0700

Paul:

We will allow different path-id types for different applications. Please 
reserve your judgment until the revised draft is posted.  It will not be 
long considering it's a Saturday and we are working :-)

-- Enke

Paul Jakma wrote:

> opaque-ID if a clash arises just brings us back to the problems raised 
> with that which lead you to return to router-ID.
>
> leave external next-hops unchanged" case, which Curtis and you gave in 
> an earlier email.
>
> 'restrictions' of nexthop, we'd then be happy with something with even 
> greater restrictions.
>

---

From: "Manav Bhatia" <[email-masked]>
Date: Sun, 3 Sep 2006 09:40:28 +0530

Hi Curtis,

> I should have separated IBGP multipath and EBGP multipath.

I think we need to be clear on what each of us means by IBGP and EBGP 
multipath.

IBGP multipath is simple and there isnt much ambiguity there. However, 
different people can have different impression of what is meant by EBGP 
multipath.

B     X
   \   /
    A
   /   \
C     Y

A, B and C are IBGP peers. A peers with X and Y, both of which belong to a 
different, but the same AS number (X and Y are IBGP peers).

A learns a prefix from both X and Y and installs a multipath route with 
next-hops X and Y. Would this be treated as an IBGP multipath or an EBGP 
multipath? I for one, would treat this as a form of EBGP multipath. Someone 
OTOH, could argue (which is not entirely without merit) that this is IBGP 
multipath as A is advertising multipath routes inside its own AS.

My point is we need to be clear on what we mean by each of these terms. 
Because from my point of view, this flavour of EBGP multihop isnt "a can of 
worms" at all.

> no serious operator interest in EBGP multipath and therefore the

I presume you are talking of the case when both X and Y are in different 
ASes. Is this so?

> operators are still free not to use it.

Definitely.

Manav

---

From: Enke Chen <[email-masked]>
Date: Sat, 02 Sep 2006 23:05:55 -0700

Hi, Manav:

Manav Bhatia wrote:

> different people can have different impression of what is meant by 
> EBGP multipath.
>
> B     X
>   \   /
>    A
>   /   \
> C     Y
>
> A, B and C are IBGP peers. A peers with X and Y, both of which belong 
> to a different, but the same AS number (X and Y are IBGP peers).
>
> A learns a prefix from both X and Y and installs a multipath route 
> with next-hops X and Y. Would this be treated as an IBGP multipath or 
> an EBGP multipath? I for one, would treat this as a form of EBGP 
> multipath. Someone OTOH, could argue (which is not entirely without 
> merit) that this is IBGP multipath as A is advertising multipath 
> routes inside its own AS.
>
> My point is we need to be clear on what we mean by each of these 
> terms. Because from my point of view, this flavour of EBGP multihop 
> isnt "a can of worms" at all.

In your example, X and Y are in the same AS, and EBGP multipath (over 
the same AS) is an issue that was resolved ten years ago.  I am fully 
aware of this useful feature.

Pleaes be precise in what your quote. Here is what I said about "a can 
of worms":

              EBGP multipath (across diff. AS) is a can of worms (-:

-- Enke

---

From: "Manav Bhatia" <[email-masked]>
Date: Sun, 3 Sep 2006 12:17:47 +0530

Enke,

>> Because from my point of view, this flavour of EBGP multihop isnt "a can 
>> of worms" at all.
>
>
> In your example, X and Y are in the same AS, and EBGP multipath (over the 
> same AS) is an issue that was resolved ten years ago.  I am fully aware of 
> this useful feature.

I am sure everybody is.

Which is precisely why i said that we need to distinctly define what we mean 
by "EBGP multipath" here. Because I dont find "this flavor" a can of worms, 
and am sure neither do you ! ;-)

>
> Pleaes be precise in what your quote. Here is what I said about "a can of 
> worms":
>

I know this, which is why i said the following:

"I presume you are talking of the case when both X and Y are in different 
ASes."

Manav

---

From: "Manav Bhatia" <[email-masked]>
Date: Mon, 4 Sep 2006 10:40:18 +0530

>itself needs to be justified due to its complexities and implications.

The fact that we've been discussing various scenarios where one would want
to advertise multiple paths across confed boundaries [1] leads me to believe
that there could be considerable interest in doing multipath there (its
after all only one AS that has been split). 

If so, then EBGP multipath across different ASes, *is* a valid requirement.
Again, the point being that we need to be very clear on our use of the
terminology.

Manav

[1] Yes, I am cognizant of applications, other than multipath, that may
require us to advertise multiple paths across confed boundaries.

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 05 Sep 2006 13:41:42 -0400

In message <007601c6cf0e$e956a7e0$0201a8c0@samba>
> > operators are still free not to use it.

Manav,

These are some definitions that may be sufficient.

  IBGP multipath is kept within a single AS even when more than one AS
  path is used.  One AS path is sent to all EBGP peers.

  EBGP multipath involves sending more than one path for the same NLRI
  from one BGP speaker to one of its EBGP peers.  The multiple paths
  may have different AS paths.

We already get a situation similar to EBGP multipath where there are
two peerings between two AS and a different AS path is provided as
each peering.  The recipient of these routes MAY use IBGP multipath or
may pick one.

It may be that EBGP multipath is "a can of worms" only because it has
the potential to significantly increase the amount of routing
information within the global Internet.  Sudden significant increases
in the amount of global routing information has in the past been known
to make the Internet somewhat unstable as providers find that their
routers don't have enough memory to accommodate the increase.

Curtis

---

From: "Glen Kent" <[email-masked]>
Date: Wed, 6 Sep 2006 06:23:39 +0530

Curtis,

>  may have different AS paths.

Why would an  EBGP speaker send multiple paths, each with different
path lengths for the same NLRI?

This isnt btw a general definition of EBGP multipath. What Enke and
Manav suggested was much closer to the reality.

> two peerings between two AS and a different AS path is provided as

Only if the paths have the same path length will these pass for EBGP
multipath routes.

> each peering.  The recipient of these routes MAY use IBGP multipath or

How can you use IBGP multipath across routes learnt from two different ASes?

> information within the global Internet.  Sudden significant increases

So what happens in case of confederations? Do you do EBGP multipath?

> routers don't have enough memory to accommodate the increase.

Agree.

Glen.

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 05 Sep 2006 22:26:10 -0400

> path lengths for the same NLRI?

Strange and inconsistent policy is the only reason I can think of for
different path lengths.

I did say different AS paths not different AS Path lengths.

> Manav suggested was much closer to the reality.

Here's what I find (from Enke):

  In your example, X and Y are in the same AS, and EBGP multipath
  (over the same AS) is an issue that was resolved ten years ago.  I
  am fully aware of this useful feature.

  Pleaes be precise in what your quote. Here is what I said about "a
  can of worms":

It doesn't appear to me that Enke and Manav are in full agreement on a
definition of EBGP multipath.  That's why I suggested a definition.

> Only if the paths have the same path length will these pass for EBGP

Same path length.  If the tie breaker is IGP cost the AS Path length
comparison has already been made and the path lengths are the same.

> How can you use IBGP multipath across routes learnt from two different ASes?

Router A, B, and C are in the same AS.  Router A learns a route with
one AS path.  Router B learns a route with another AS path.  The
decision comes down to IGP cost (AS Path length is the same, etc).
The IGP cost from router C to A and B is the same.  Router C uses IBGP
multipath.

Alternately A, B, and C can be the same router.  That is neither an
IBGP or EBGP case, just a local decision at the router.  If the
different AS paths were advertised into IBGP it would be IBGP
multipath.  Otherwise for the purpose of readvertisement the router
C=A=B would have to pick one to advertise into IBGP though forwarding
to both.

> > routers don't have enough memory to accommodate the increase.

There is no such thing as EBGP multipath today by the definition I
gave.  Today there is no way to advertise two routes with the same
NLRI to one peer.  Consider the following diagram.

        AS1--AS1 - AS3--AS3 - AS4--AS4 ...etc...ASn (router A)
       /                                         \
   Dest                                           ASn (router C)
       \                                         /
        AS2--AS2 - AS3--AS3 - AS4--AS4 ...etc...ASn (router B)

That is the case where ASn+1 is doing IBGP multipath but it looks a
lot like EBGP multipath looking at it from a distance.

Would it be very different if it were as follows?

        AS1
       /   \
   Dest     AS3==AS3 = AS4--AS4 ...etc = ASn
       \   /
        AS2

[Note: Where = is the advertisement of a multipath, "--" or "==" is
IBGP and " - " or " = " is EBGP.]  In both cases ASn has a multipath
with AS4 AS3 AS1 and AS4 AS3 AS2.  In the first case each AS has a
pair of peerings.  The first case is possible but fairly unlikely.  In
the second case there may be a single peering or multiple peerings but
the multipath occurs at AS3 and is propogated by some new signaling
(add-path++ or whatever) that allows an explicit multipath.

This second case is what I think is the biggest can of worms because
it may result in a lot more routing information.

Curtis

---

From: "Manav Bhatia" <[email-masked]>
Date: Wed, 6 Sep 2006 11:26:07 +0530

Curtis,

>  path is used.  One AS path is sent to all EBGP peers.

Yes, the interesting part is to see how an IBGP peer compresses multiple
paths into one, when advertising the routes to its EBGP peers.

>  may have different AS paths.

These definitions are acceptable and good to start with, although the
general meaning of EBGP multihop has always been different from this.

From: "Curtis Villamizar" <[email-masked]>
Subject: Re: [Idr] multiple-next-hops/add-paths proble space

> definition of EBGP multipath.  That's why I suggested a definition.

We are.

Both of us feel that the example that i had given for EBGP multipath is
*not* a "can of worms", while doing a split across multiple ASes, could be
one. We also agree that both are btw valid forms of EBGP multihop.

[clipped]

> The IGP cost from router C to A and B is the same.  Router C uses IBGP

This is an interesting case because C is now doing a load split across A and
B and the traffic will flow via different AS Paths. If there is a 
requirements
document then we must specify the advertisement that C sends out to (i) its
EBGP peers and (ii) non ecmp capable peers (aka ordinary rfc4271 compliant
peers).

> C=A=B would have to pick one to advertise into IBGP though forwarding

I don't think its correct to advertise one while using both for forwarding.
NOTE: I am assuming that the routes share the same path length but have
different AS Paths (as mentioned by you earlier in this mail)

Manav

---

From: Russ White <[email-masked]>
Date: Wed, 06 Sep 2006 08:36:06 -0400

> it may result in a lot more routing information.

Note it is possible to do this, without advertising all the routes--1771
contains this option in section 9.1.4, since equal routes technically
overlap one another. This was taken out in the current RFC, presumably
because no-one has implemented it....

AS3 can advertise the route with an AS SET of [1,2], so the AS Path at
AS4 is {[1,2],3}.

:-)

Russ

- --
[email-masked] CCIE <>< Grace Alone

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 06 Sep 2006 10:56:01 -0400

In message <043701c6d179$2984b9d0$0201a8c0@samba>
"Manav Bhatia" writes:
...snip...

I'll look back at your prior mail and see where the difference is in
definition.

> different AS Paths (as mentioned by you earlier in this mail)

I think this is being done by many providers.

Curtis

---

From: "Manav Bhatia" <[email-masked]>
Date: Wed, 6 Sep 2006 21:34:33 +0530

Curtis,

> I think this is being done by many providers.

Consider two paths for nlri X

as_path 1 {x y z} next_hop n1
as_path 2 {m n z} next_hop n2

Are you suggesting that providers are installing ecmp routes for X with 
next-hops n1 and n2, while advertising only one of the paths to their IBGP 
peers?

Manav

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 06 Sep 2006 12:55:21 -0400

> AS3 can advertise the route with an AS SET of [1,2], so the AS Path at

Some of the examples could yield multipaths of the form AS5 AS3 AS1
and AS4 AS2 AS1.  Should these by combined to [AS4,AS5][AS2.AS3]AS1?
Probably not.

I question whether any of this multipath work is worth it but that's a
discussion for later.

First we have to agree on what set of problems we are discussing.
Then we can look at what problems we are targeting.  Then we can look
at which solutions fit those targets.

If seems the applications of multipath we are discussing are as stated
previously:

  multipath

    IBGP

    EBGP - need to agree on the definition

  RR

  AS confed

  RS

For RS the problem is clear.  We need to pass all routes with no
application of policy or route selection so that the peers have the
same set of routes they would get in a direct peering.

For RR and AS confed we are trying to eliminate a very well documented
problem with route oscillation that is related to MED.

For multipath, both IBGP and EBGP, it is not clear what problem we are
solving.  It is true that the routes being advertised would be more
accurate if the multipath that was used in forwarding was always
advertised.  It doesn't seem to solve any forwarding loop problems or
oscillation.  Perhaps someone could enlighten me on what problem is
solved by correctly advertising the BGP multipath.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 06 Sep 2006 13:00:21 -0400

In message <009701c6d1ce$275c8200$0201a8c0@samba>

Yes.

Curtis

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 06 Sep 2006 13:52:17 -0400

BGP Loop prevention relies on the assumption that advertisements have 
all of their AS Path information.

If folks are really advertising one thing while using something 
different, that seems an invitation to trouble.

In general, this aspect of the multi-path debate causes me 
concern.  We seem to be asking for perfect support for a capability, 
which if used with its full generality, will break BGP.  To the 
degree that these various tricks are used currently, they are used by 
by individuals who have at least tried to determine if that specific 
use is safe.  And they know they are operating outside of the 
specs.  If we write specs which endorse all these various things, we 
need to be clear about how it is safe to use.

Yours,
Joel

PS: Yes, the distinct next-hop is safer.  And when used in 
combination with the ECMP advertisements rules we wrote, it at least 
complies with all current written behavioral requirements.  Even that 
usage probably warrants closer examination for loop and oscillation 
before it is adopted.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 06 Sep 2006 14:00:14 -0400

If we support multi-path, then we need to define what happens when a 
multi-path speaker / user wants to pass on its actual used 
information to a non multi-path speaker.  if we don't want to break 
BGP loop prevention, then the speaker will need to combine the paths 
in a way that preserves all distinct AS values and total AS 
length.  The example you give (which is what we suggested in our 
earlier ID) is one valid way to meet that goal.  Not documenting that 
the goal needs to be met, or not providing an example of how to meet 
it, would seem to be a very bad idea if we are going down this path.

Yours,
Joel M. Halpern

PS: This applies whether we use add-path or the distinct next hop 
proposal.  It is not a difference between them.

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 06 Sep 2006 16:38:12 -0400

In message <[email-masked]>
> different, that seems an invitation to trouble.

But from a practical standpoint if AS Path length is used in the
decision (no use of LOCAL_PREF) then there is no problem.

> need to be clear about how it is safe to use.

Yes.  RFC4271 does not address multipath so any use of multipath is
currently outside the spec.

> usage probably warrants closer examination for loop and oscillation 

BGP multipath and in particular EBGP multipath is on the list of
problems under discussion.  I don't know how you've jumped to the
"distinct next-hop is safer" statement but we are trying not to
discuss solutions at this point.  Still waiting for add-path++.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 06 Sep 2006 16:44:01 -0400

In message <[email-masked]>
> >and AS4 AS2 AS1.  Should these by combined to [AS4,AS5][AS2.AS3]AS1?

I agree completely.  I also agree (with the prior multipath draft)
that the AS Path length must remain the same.  Since an AS Set counts
as one, a possibility for the above example is to pad one's own AS and
make one AS Set as in AS6 AS6 [AS2 AS33 AS4 AS5] AS1 if AS6 was
advertising this outside of the AS to a BGP speaker that doesn't speak
BGP multipath whatever that is.

Curtis

---

From: "Glen Kent" <[email-masked]>
Date: Thu, 7 Sep 2006 05:24:29 +0530

> BGP multipath whatever that is.

I would be least comfortable in proposing this. If implementations
start doing this and we get more (or longer) paths that we are
aggregating then the AS Set can overflow, which would increase the
Path length by 1.

No, not a very elegant solution.

> and AS4 AS2 AS1.  Should these by combined to [AS4,AS5][AS2.AS3]AS1?

This best (given the circumstances) describes the topology that
existed before the ecmp. What problem do you see with this?

---

From: "Glen Kent" <[email-masked]>
Date: Thu, 7 Sep 2006 05:29:51 +0530

> > next-hops n1 and n2, while advertising only one of the paths to their IBGP

This is blasphemous if operators are really doing this - advertising
routes via some AS Path and forwarding traffic via some other AS Path.

I have my reservations if this really happens.

Can the operators on this list contact me in private if they are aware
of instances where this has happened?

Glen

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 06 Sep 2006 22:21:21 -0400

> existed before the ecmp. What problem do you see with this?

Fine.  I appologize for delving into possible solution space.  We seem
to agree that there would be a problem if EBGP multipath of this sort
is done but lets not try to engineer a solution yet.

Curtis

---

From: "Glen Kent" <[email-masked]>
Date: Thu, 7 Sep 2006 23:18:56 +0530

IDR WG by the act of endorsing either of add-paths/multiple_hop would
be facilitating, if not encouraging, providers to do EBGP multipath.
If folks are already doing EBGP multipath in the way suggested by
Curtis then they're already breaking BGP (something which i am unsure
of).

In light of this, it becomes all the more important to specify how
multipath advertisements need to be announced to other speakers - both
for the speakers capable and the ones incapable of understanding
add-path/multiple_hop extensions.

If this means advertising a fancy AS Path then so be it, but the same
should be documented.

Glen

---

From: "Glen Kent" <[email-masked]>
Date: Fri, 8 Sep 2006 00:39:25 +0530

Curtis,

> is done but lets not try to engineer a solution yet.

No, I see a problem if you try to compress the AS Path in the manner
you suggested, that is, by prepending your own AS "n" of times and by
lumping every other AS inside a SET.

As Ever

Glen

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 07 Sep 2006 16:23:30 -0400

> Curtis then they're already breaking BGP (something which i am unsure

In practice they are not breaking anything.  As long as LOCAL_PREF
does not override AS Path length no loops can form.

Playing very loose with the rules at the very least with a potential
for trouble.

> If this means advertising a fancy AS Path then so be it, but the same

If nothing is being broken the urgency isn't there.

A clean way to do this would be preferable as long as the amount of
information held in global routing doesn't blow up.  No rush though.
Better to get it right.

Curtis