Subject: [Idr] draft-walton-bgp-route-oscillation-stop as an IDR WG document

From: John Scudder <[email-masked]>
Date: Mon, 10 May 2010 14:00:51 -0700

Folks,

We have received a request to adopt draft-walton-bgp-route-oscillation-stop as an IDR working group document.  Please send comments to the list.  The deadline for comments is May 25, 2010.

--John

-------- 
Subject:	I-D Action:draft-walton-bgp-route-oscillation-stop-03.txt
Date:	Mon, 10 May 2010 11:30:02 -0700 (PDT)
From:	[email-masked]
Reply-To:	[email-masked]
To:	[email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title           : BGP Persistent Route Oscillation Solutions
	Author(s)       : D. Walton, et al.
	Filename        : draft-walton-bgp-route-oscillation-stop-03.txt
	Pages           : 9
	Date            : 2010-05-10

In this document we present two sets of paths for an address prefix
that can be advertised by a BGP route reflector or confederation ASBR
to eliminate the MED-induced route oscillations in a network.  The
first set involves all the available paths, and would achieve the
same routing consistency as the full IBGP mesh.  The second set,
which is a subset of the first one, involves the neighbor-AS based
Group Best Paths, and would be sufficient to eliminate the MED-
induced route oscillations (subject to certain commonly adopted
topological constrains).

A URL for this Internet-Draft is:

http://www.ietf.org/internet-drafts/draft-walton-bgp-route-oscillation-stop-03.txt

Internet-Drafts are also available by anonymous FTP at:

ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

---

From: Uli Bornhauser <[email-masked]>
Date: Tue, 11 May 2010 10:38:50 +0200

Hi Daniel, other authors, all,

one comment, more precisely a question: What about potential problems the "BGP Persistent Route Oscillation Solutions" may cause? For example, if I did not miss an important aspect, the concept may cause situations where BGP speakers cannot choose a unique best path any more:

                   |
           clusterI|clusterII
p1->  |----|       |
------| C1 |       |
      |----|       |
            \ 1    |
            |----| |1  |----|
            | R1 |-----| R2 |
            |----| |   |----|
            / 1    |
p2->  |----|       |
------| C1 |       |
      |----|       |
 
C1 and C2 both provide their best path (externally learned via different ASs, same LOCAL_PREF, AS_PATH length, etc.) to R1. R1 in turn advertises these paths (p1 and p2) to R2 according to your draft. As both paths are learned via the same session, the common BGP tie breaker process does not work at this point: Even after executing step g) [1], both paths are still in the decision process. Obviously, randomly choosing one path does not work, too. Is there a simple solution for this problem (I missed it in the draft)? I think either a modification of the selection process or new attributes are needed (which seem to be in conflict with the statement that only minor changes are needed at the most BGP speakers, cf. 5. Deployment Considerations). If I did not miss a point, this observation leads to the following question whether advertising "all Paths" / "Group Best Paths" may cause other, not that obvious problems.

Thanks in advance for the clarification and

Best Regards

Uli

[1] RFC4271 - http://www.ietf.org/rfc/rfc4271

Am 10.05.2010 um 23:00 schrieb John Scudder:

> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

-- 
_______________________________________________________
ULI BORNHAUSER
University of Bonn - Institute of Computer Science IV
c/o Bonn-Aachen International Center for Information Technology B-IT 
Dahlmannstr. 2 - D-53113 Bonn - Germany

Web: www.cs.bonn.edu/IV/ub
Email: [email-masked]			
Phone: +49 (228) 2699-154
Fax: +49 (228) 73 - 4571

---

From: John Scudder <[email-masked]>
Date: Tue, 11 May 2010 08:06:34 -0700

Uli,

Doesn't this text from RFC 4456 address your point?

      If a route carries the ORIGINATOR_ID attribute, then in Step f)
      the ORIGINATOR_ID SHOULD be treated as the BGP Identifier of the
      BGP speaker that has advertised the route.

      In addition, the following rule SHOULD be inserted between Steps
      f) and g): a BGP Speaker SHOULD prefer a route with the shorter
      CLUSTER_LIST length.  The CLUSTER_LIST length is zero if a route
      does not carry the CLUSTER_LIST attribute.

Regards,

--John

---

From: Pierre Francois <[email-masked]>
Date: Tue, 11 May 2010 17:29:48 +0200

John,

If paths p1 and p2 were coming from the same ASBR, the problem comes back, no ?

Using the NEXT_HOP attribute will also lead to the same issue when the ASBR sets 
nexthop to self.

Regards,

Pierre.

---

From: Uli Bornhauser <[email-masked]>
Date: Wed, 12 May 2010 12:13:42 +0200

Hi John, Pierre, all,

>>      does not carry the CLUSTER_LIST attribute.

John, yes, you are of course absolutely right. My example was a little bit too simple. 

> If paths p1 and p2 were coming from the same ASBR, the problem comes back, no ?

However, it seems easy to modify the example so that the basic problem remains: 

              |
p1->  |----|  |
------|    |  |
      |----|  |
            \ /
             \
AS1         / \
-----------/  |----|  1  |----|
              | R1 |-----| R2 |
-----------\  |----|     |----|
AS2         \ /
             /
            / \
p2->  |----|  |
------|    |  |
      |----|  |

Let assume that p1 and p2 are advertised from external ASs to the Route Reflector R1. In this case, according to RFC4456, the Originator ID is not set by the Reflector R1:

   ORIGINATOR_ID is a new optional, non-transitive BGP attribute of Type
   code 9.  This attribute is 4 bytes long and it will be created by an
   RR in reflecting a route.  This attribute will carry the BGP
   Identifier of the originator of the route in the local AS.  A BGP
   speaker SHOULD NOT create an ORIGINATOR_ID attribute if one already
   exists.  A router that recognizes the ORIGINATOR_ID attribute SHOULD
   ignore a route received with its BGP Identifier as the ORIGINATOR_ID.

We have no *local* originator and RR1 does not "reflect" a path in the closer sense. Thus, the originator ID is not specified in the path advertisements. So Pierre is right if the ASBR is a Route Reflector. Because the best path cannot be determined uniquely in all configurations, [1] may induce a problematic side-effect. If the ASBR is a common client, it only advertises its (one unique) best path, cf. [1]. In that case, the problem I have in mind should not occur.

Regards

Uli

[1] http://www.ietf.org/id/draft-walton-bgp-route-oscillation-stop-03.txt

> 
> Using the NEXT_HOP attribute will also lead to the same issue when the ASBR sets nexthop to self.
> 
> Regards,
> 
> Pierre.
> 
> 
> John Scudder wrote:
>> 
>>>> -------- Subject:	I-D Action:draft-walton-bgp-route-oscillation-stop-03.txt
>>> c/o Bonn-Aachen International Center for Information Technology B-IT Dahlmannstr. 2 - D-53113 Bonn - Germany
>>>

---

From: Jakob Heitz <[email-masked]>
Date: Wed, 12 May 2010 10:18:32 -0400

I remember an attribute: attr_set in an expired draft that solves this problem
http://tools.ietf.org/id/draft-pmohapat-idr-fast-conn-restore-00.txt
Whatever happened to it?

--
Jakob Heitz.

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Uli Bornhauser
Sent: Wednesday, May 12, 2010 3:14 AM
To: John Scudder
Cc: Inter-Domain Routing List
Subject: Re: [Idr] draft-walton-bgp-route-oscillation-stop as an IDR WG document

Hi John, Pierre, all,

Uli,

John,

              |
p1->  |----|  |
------|    |  |
      |----|  |
            \ /
             \
AS1         / \

Regards

Uli

Regards,

Pierre.

John Scudder wrote:

                  |
          clusterI|clusterII
p1->  |----|       |
------| C1 |       |
     |----|       |
           \ 1    |
           |----| |1  |----|
           | R1 |-----| R2 |
           |----| |   |----|
           / 1    |
p2->  |----|       |
------| C1 |       |
     |----|       |

Best Regards

Uli

Folks,

--John

-------- Subject: I-D Action:draft-walton-bgp-route-oscillation-stop-03.txt
Date: Mon, 10 May 2010 11:30:02 -0700 (PDT)
From: [email-masked]<mailto:[email-masked]>
Reply-To: [email-masked]<mailto:[email-masked]>
To: [email-masked]<mailto:[email-masked]>

[email-masked]<mailto:[email-masked]>

Web: www.cs.bonn.edu/IV/ub<http://www.cs.bonn.edu/IV/ub>
Email: [email-masked]<mailto:[email-masked]>
Phone: +49 (228) 2699-154
Fax: +49 (228) 73 - 4571

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr
_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr

Web: www.cs.bonn.edu/IV/ub<http://www.cs.bonn.edu/IV/ub>
Email: [email-masked]<mailto:[email-masked]>
Phone: +49 (228) 2699-154
Fax: +49 (228) 73 - 4571

---

From: Uli Bornhauser <[email-masked]>
Date: Wed, 12 May 2010 20:22:42 +0200

Hi Jakob,

> http://tools.ietf.org/id/draft-pmohapat-idr-fast-conn-restore-00.txt

yes, applied correctly, the Border Router Attribute Set could solve the problem. However, the modifications sketched in this draft are not completely unproblematic, too. I sketched a problematic example in an email I send to the list on Dec. 04th, 2008.

Best Regards

Uli

---

From: "Shyam Sethuram (shsethur)" <[email-masked]>
Date: Mon, 17 May 2010 01:46:55 -0700

I think we should go ahead and adopt this as a WG draft.

thanks--shyam

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of John Scudder
> Sent: Monday, May 10, 2010 2:01 PM
> To: Inter-Domain Routing List
> Subject: [Idr] draft-walton-bgp-route-oscillation-stop as an 
> IDR WG document
> 
> Folks,
> 
> We have received a request to adopt 
> draft-walton-bgp-route-oscillation-stop as an IDR working 
> group document.  Please send comments to the list.  The 
> deadline for comments is May 25, 2010.
> 
> --John
> 
> -------- 
> Subject:	I-D 
> Action:draft-walton-bgp-route-oscillation-stop-03.txt

---

From: Pierre Francois <[email-masked]>
Date: Mon, 17 May 2010 13:29:41 +0200

Uli,

 > So Pierre is right if the ASBR is a Route Reflector.
> Because the best path cannot be determined uniquely in all 
> configurations, [1] may induce a problematic side-effect. If the ASBR is 
> a common client, it only advertises its (one unique) best path, cf. [1].

Haaa, right.

Note that with this ASBR behavior we miss the nice side gain of solving MED 
Disrespect states although we were almost at it thanks to this draft.

Let us assume that a group best path p1 is not the best path of the ASBR that 
receives it, because it prefers another eBGP path received from another 
neighboring AS, say, based on router-id.

p1 will not be advertised to the RRs and hence will not be reflected to those 
routers who have other paths coming from the same neighboring AS as p1.

These routers, not knowing about p1, could pick as best a path p2 from the same 
neighboring AS, although p2 has an higher MED value than p1, because they don't 
know about p1.

If [1] was suggesting that the ASBR could also propagate (non best) AS Group 
best paths with add-paths, p1 would be advertised to the RRs, which would 
reflect them to those who initially picked p2, so that they will be forced to 
respect the MED and pick another path.

If we do allow non-RR ASBRs to propagate non best AS Group Best path in 
add-paths, we absolutely need the attr_set attribute from
draft-pmohapat-idr-fast-conn-restore-00 . But we need it for all
the good reasons listed in the draft, anyway.

Regards,

Pierre.

> In that case, the problem I have in mind should not occur.
> 
> Regards
> 
> Uli
> 
>>>> one comment, more precisely a question: What about potential 
>>>> problems the "BGP Persistent Route Oscillation Solutions" may cause? 
>>>> For example, if I did not miss an important aspect, the concept may 
>>>> cause situations where BGP speakers cannot choose a unique best path 
>>>> any more:
>>>>
>>>>                   |
>>>>           clusterI|clusterII
>>>> p1->  |----|       |
>>>> ------| C1 |       |
>>>>      |----|       |
>>>>            \ 1    |
>>>>            |----| |1  |----|
>>>>            | R1 |-----| R2 |
>>>>            |----| |   |----|
>>>>            / 1    |
>>>> p2->  |----|       |
>>>> ------| C1 |       |
>>>>      |----|       |
>>>> C1 and C2 both provide their best path (externally learned via 
>>>> different ASs, same LOCAL_PREF, AS_PATH length, etc.) to R1. R1 in 
>>>> turn advertises these paths (p1 and p2) to R2 according to your 
>>>> draft. As both paths are learned via the same session, the common 
>>>> BGP tie breaker process does not work at this point: Even after 
>>>> executing step g) [1], both paths are still in the decision process. 
>>>> Obviously, randomly choosing one path does not work, too. Is there a 
>>>> simple solution for this problem (I missed it in the draft)? I think 
>>>> either a modification of the selection process or new attributes are 
>>>> needed (which seem to be in conflict with the statement that only 
>>>> minor changes are needed at the most BGP speakers, cf. 5. Deployment 
>>>> Considerations). If I did not miss a point, this observation leads 
>>>> to the following question whether advertising "all Paths" / "Group 
>>>> Best Paths" may cause other, not that obvious problems.
>>>>
>>>>> draft-walton-bgp-route-oscillation-stop as an IDR working group 
>>>>> document.  Please send comments to the list.  The deadline for 
>>>>> comments is May 25, 2010.
>>>>>
>>>>> --John
>>>>>
>>>>> -------- Subject: I-D 
>>>>> From: [email-masked] <mailto:[email-masked]>
>>>>> Reply-To: [email-masked] <mailto:[email-masked]>
>>>>> To: [email-masked] <mailto:[email-masked]>
>>>>>
>>>>>
>>>>> [email-masked] <mailto:[email-masked]>
>>>> Web: www.cs.bonn.edu/IV/ub <http://www.cs.bonn.edu/IV/ub>
>>>> Email: [email-masked] <mailto:[email-masked]>
>>>> [email-masked] <mailto:[email-masked]>
>>> [email-masked] <mailto:[email-masked]>
> Web: www.cs.bonn.edu/IV/ub <http://www.cs.bonn.edu/IV/ub>
> Email: [email-masked] <mailto:[email-masked]>

---

From: Pierre Francois <[email-masked]>
Date: Wed, 26 May 2010 10:31:03 +0200

Hi,

There comes a few comments/questions on the draft.

A. Configuration workarounds

In the deployment considerations section, it is stated that simple
configuration workarounds will do the job, once the route oscillations
are detected. It might be interested to list them.
Could you confirm that what you had in mind is the following :

Let's say that we have a set of prefixes for which we have routing oscillations.

The procedure is

0. Detect (assumed solved) the oscillating prefixes.

1. Write a prefix list containing them

2. Reconfigure all the RRs in the AS to now apply AS Group Best for
the prefixes in the list.

then, optionally,

3. Fix the root cause of the oscillation.

4. Reconfigure all the RRs to no longer apply AS Group Best for the
prefix list.

During step 2 (and optional step 4), transiently, some RRs will have been
reconfigured to do AS Group Best, and some not. It might be
interesting to state in the draft that this transient use of different
add-path modes does not put the network into more trouble than what it
is facing.

It might also be interesting to state that fixing the root cause of
the oscillation (if in the plan of operations) typically takes much more time 
than completing steps 1 and 2.

If this is not the case, maybe operators will consider that

- only doing steps 0 and 3 is the way to go...

or

- always apply AS Group Best path propagation, be there an oscillation
   or not, is the way to go. If the additional control-plane state and
   state update is not an issue, this is much easier than the re-active
   approach.

B. Discussing the number of additional paths propagated.

I read "The number of the neighbor-AS's for a particular address
prefix is typically small because of the limited number of upstream
providers for a customer [...]"

Isn't this statement assuming that we are in the context of an ISP
dealing with MED oscillations with MED-tweaked paths received from a provider ?

I would have bet that MED is typically used the other way round, hence
the ISP gets oscillations due to MED tweaking by its customer.

In which case the statement would become "The number of neighbor AS's
for a particular address prefix is typically small because of the
limited number of customer ASes providing a path to a given prefix ?"

In which case this statement might need to be backed up with data
showing that large Tiers do not have a lot of neighboring customer
ASes providing a path to a given prefix to which the same loc-pref is
applied and which have the same shortest AS path length.  This is less
intuitive than the other way round.

[...] "and the nature of advertising only customer routes at the
inter-exchange points."

What do you exactly mean with this ?

Is it that the number of paths received from peers is small so if
there is MED oscillation on peer paths, it won't be on a lot of paths so that
Group best will not let many additional paths be propagated in that scenario ?
?

The fact that we're only talking about control-plane state increase
should be stressed. More paths, yes, but not more entries in the
FIB.

Regards,

Pierre.