Subject: [Idr] Transient loop prevention when using the best external and add-path approaches

From: Xu Xiaohu <[email-masked]>
Date: Wed, 10 Feb 2010 17:39:32 +0800

Hi all,

In the following scenario where the BGP best external approach (see
draft-ietf-idr-best-external-01) is enabled on the BGP speakers of AS#1
(i.e., R1, R2 and R3), assuming R1 and R2 each learns a route to the same
destination P from AS#2 and AS#3 respectively, and the route learnt from
AS#2 is selected as the best route by R1, R2 and R3. According to the best
external approach, R2 will advertise its best external route to P (i.e., the
route learnt from AS#3) to its iBGP peers, R1 and R3. When R1's connection
to AS#2 is broken, a transient forwarding loop will be formed during the
time the network takes to converge to the alternate path (i.e., the route
learnt from AS#3). That is: R3 still forward packets destined to P towards
R1, R1 in turn will forward them to R2 due to fast local repair, and R2 will
then forward them back to R1... therefore, a forwarding loop is formed
between R1 and R2 till the network is converged.

         AS#2        AS#3
           |           |
           |           |
    +------+-----------+------+
    |   +--+--+     +--+--+   |
    |   |  R1 +-----+  R2 |   |
    |   +-\---+     +--/--+   |
    |      \          /       |
    |       \        /        |
    |        *------*         |
    |        |  R3  |         |
    |        +------+         |
    |         AS#1            |
    +-------------------------+

In fact, the above transient loop can be avoided as long as the best
external route is processed according to the approaches defined in Carrying
Label Information in BGP-4 [RFC3107]. For example, R2 will assign a unique
MPLS label for its best external route (i.e., the route to P learnt from
AS#3) and then install it into its MPLS forwarding table. When advertising
the best external route to its iBGP peers, R2 will attach the corresponding
MPLS label in the BGP update. Upon receiving the best external route, R1 and
R3 will install it into their own MPLS forwarding table respectively as a
backup route to destination P.  In this way, R1 will forward packets
destined to P towards R2 with the MPLS label attached. As a result, R2 will
use the corresponding MPLS forwarding entry (i.e., its best external route
to P), rather than the route entry in its Routing table (i.e., its best
route to P), to forward the packets. Of course, if R1 and R2 are not
directly connected, an IP or MPLS tunnel from R1 to R2 is required since the
above MPLS label has local meanings only on R2.

By the way, I think the add-path approach (see draft-ietf-idr-add-paths-03)
may cause the same transient loop problem which can be avoided similarly
with the above approach.

Hence I wonder whether it would be better that the approach defined in
[RFC3107] is STRONGLY recommended in the two WG drafts mentioned above.

Comments?

Best wishes,
Xiaohu

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 10 Feb 2010 11:17:45 +0100

Hi Xu,

I think we need to step back and define what transient loop you may be 
referring to.

 From your description I infer that the loop you are referring to 
happens  from the moment of R1 to AS#2 connection down or AS#2's ASBR 
down and lasts till BGP withdraws the prefix P from R1 ?

Is this true ? If so - yes it would be relatively long and undesired.

But I would say that this is wrong network design.

The alternative network design would be to use IGP within AS#1 for fast 
flooding of AS#2's ASBR next hop down event. That event would be 
propagated within AS#1 in single miliseconds therefor each BGP speaker 
can switch upon reception of this trigger to an alternative path.

Tools like PIC (prefix independent convergence) and preinstallation of 
backup paths in RIB and FIB are available in shipping implementations 
and can improve the actual connectivity restoration times significantly 
from the reception of the IGP next hop down event trigger.

If one would be still concerned about the transient loops for the 
duration of IGP single next hop flooding time which is IMHO 
insignificant and happens today each time IGP converges there is set of 
IPFRR solutions available as well.

There is clearly no need to move entire Internet routing to RFC3107 or 
mpls encapsulation when best external is enabled.

The similar point you make regarding add-paths also can be addressed by 
the proper design. In fact IMHO with the proper network design enabling 
add-paths can be avoided all together - but this is a separate topic not 
for this thread :).

Cheers,
R.

> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr
>

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Fri, 12 Feb 2010 15:19:21 -0600

This is an interesting topic wrt transient loop due to best-external
and/or add-path usage. Good points being brought up.

Quick observation -

> can switch upon reception of this trigger to an alternative path.

This would warrant prescribing no next-hop-self usage at ASBRs in AS#1.
Again, a function of network design, as Robert pointed out.

Cheers
Rajiv

I agree with Robert 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Robert Raszuk (raszuk)
> Sent: Wednesday, February 10, 2010 5:18 AM
> To: Xu Xiaohu
> Cc: [email-masked]
> Subject: Re: [Idr] Transient loop prevention when using the best
external
> and add-path approaches
> > AS#2 is selected as the best route by R1, R2 and R3. According to
the
> best
> > R1, R1 in turn will forward them to R2 due to fast local repair, and
R2
> will
> > MPLS label in the BGP update. Upon receiving the best external
route,
> R1 and
> > R3 will install it into their own MPLS forwarding table respectively
as
> a
> > destined to P towards R2 with the MPLS label attached. As a result,
R2
> will

---

From: Xu Xiaohu <[email-masked]>
Date: Mon, 22 Feb 2010 14:48:23 +0800

Hi Robert,

Sorry for late response due to Chinese New Year holiday. Please see my
response inline.

> -----邮件原件-----
> 发件人: Robert Raszuk [mailto:[email-masked]]
> 发送时间: 2010年2月10日 18:18
> 收件人: Xu Xiaohu
> 抄送: [email-masked]
> 主题: Re: [Idr] Transient loop prevention when using the best external and
> add-path approaches
> 
> Hi Xu,
> 

Yes.

> can switch upon reception of this trigger to an alternative path.

Does that mean the local ASBR (i.e., R1) MUST disable the "next-hop-self"
when advertising iBGP updates to its iBGP peers? 

> from the reception of the IGP next hop down event trigger.

I'm not sure whether all the deployed routers can support the PIC feature or
even be able to support that feature through upgrading their softwires
(i.e., OS) only.

Best wishes,
Xiaohu

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 22 Feb 2010 17:48:02 +0100

Hi Xu,

> Does that mean the local ASBR (i.e., R1) MUST disable the "next-
> hop-self" when advertising iBGP updates to its iBGP peers?

If someone desires to have fast connectivity restoration across two
independent ASBRs I see no other equally fast trigger to use then
flooding next hops events in the IGP.

Even if you distribute multiple paths you need to have a fast way to
determine if they are still valid.

If you have multiple peerings on given ASBR you can set next hop to self
- but you still better to track all atomic next hops you are masking
with the new local one and remove one from the IGP when all of them go down.

The alternative you were proposing to instead build set of cross ASBRs
tunnels, with the label for each path signaled in 3107 only to avoid IP
lookup in exit ASBRs seems completely not compelling to me.

Cheers,
R.

---

From: Xu Xiaohu <[email-masked]>
Date: Sat, 27 Feb 2010 15:40:19 +0800

Hi Robert,

> 发送时间: 2010年2月23日 0:48

> with the new local one and remove one from the IGP when all of them go
down.

Assuming a given ASBR has multiple remote ASBR pears and the routes learnt
from one remote peer are different from those learnt from another peer, if
the ASBR sets self as next-hop when advertising updates to its iBGP peers,
it can not simply use the IGP to trigger the route update on the other iBGP
peers. 

Best wishes,
Xiaohu

> > I'm not sure whether all the deployed routers can support the PIC
feature
> or
> >>> external approach, R2 will advertise its best external route to P
(i.e.,
R2
> >>> MPLS label in the BGP update. Upon receiving the best external route,
R1
> > and
as
> >>> directly connected, an IP or MPLS tunnel from R1 to R2 is required
since

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 27 Feb 2010 09:39:50 +0100

Hi Xu,

> it can not simply use the IGP to trigger the route update on the other iBGP

It sure can. "Tracking all atomic next hops" is the key here.

Cheers,
R.

---

From: Xu Xiaohu <[email-masked]>
Date: Sat, 27 Feb 2010 17:53:27 +0800

> 发送时间: 2010年2月27日 16:40

For example, one remote peer is down while another one is still OK, should
the IGP route to the local ASBR itself be withdrawn?

Best wishes,
Xiaohu

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 27 Feb 2010 12:16:07 +0100

Hi Xu,

> the IGP route to the local ASBR itself be withdrawn?

The simplest (but not only approach) is to remove the local next hop
from the IGP only when all original next hops go down.

That is still better then to attract traffic to such ASBR when all
external peering go down.

But the main question to ask is why to set next hop to self on ASBR for
IPv4 Internet ?

That clearly seems to have the disadvantage of blocking the original
fast next hop down event propagation of peering ASBR or interface to it.

In fact this is not related to best external or add-paths or
diverse-path ways of distributing additional paths. The same issue
happens with simple IBGP full mesh today.

In the past I had one proposal to still signal it in BGP to address the
case you are pointing out:

http://ftp.iptel.org/pub/standards/internet-drafts/draft-raszuk-ext-failure-propagation-00.txt

But it seemed that it is much better and faster to leave the next hop
unchanged.

Cheers,
R.

---

From: xuxiaohu 41208 <[email-masked]>
Date: Sat, 27 Feb 2010 23:18:09 +0800

Hi Robert,

----- 原邮件 -----
发件人: Robert Raszuk <[email-masked]>
日期: 星期六, 二月 27日, 2010 下午7:16
主题: Re: [Idr] Transient loop prevention when using the best external and add-path approaches
收件人: Xu Xiaohu <[email-masked]>
抄送: [email-masked]

> But the main question to ask is why to set next hop to self on 

There does exist some scenarios where the local ASBR should set the next-hops of iBGP routes to self. BGP-free core and Softwire Mesh are two examples. Even in a normal BGP scenario, if a given local ASBR is connected to a remote AS via two remote peers (e.g., P1 and P2), it would be better for the local ASBR to set the next-hops of the iBGP routes to self. In case that the nexthop of a learnt eBGP route is changed from P1 to P2, if all of 
the path attributes except the nexthop attribute are identical, there is no need for the local ASBR to withdraw the previous route update and then advertise a new one. In this way, the route advertisement traffic can be reduced greatly.

Best wishes,
Xiaohu

---

From: Uli Bornhauser <[email-masked]>
Date: Sat, 27 Feb 2010 21:48:28 +0100

Hi Xu,

independent of your discussion about the IGP next-hop down event and the possible corresponding problems, regarding your original issue, I want to point out that the transient loop effect you have described is IMHO no design problem of best-external and add-path, respectively.
In principle, the effect you have described may also appear if classical BGP and Route Reflection are used. Consider your original example and add a fourth reflector "Rnew" in the following way (BGP sessions are of course kept between all reflectors):

         AS#2                   AS#3
          |                       |
          |                       |
   +------+-----------------------+------+
   |   +--+--+  2  +-----+  2  +--+--+   |
   |   |  R1 +-----+ Rnew+-----+  R2 |   |
   |   +-----+     +--+--+     +-----+   |
   |          \       +       /          |
   |           \      +      /           |
   |            \   1 +     /            |
   |           1 \    +    / 3           |
   |              \   +   /              |
   |               *--+--*               |
   |               |  R3 |               |
   |               +-----+               |
   |                              AS#1   |
   +-------------------------------------+

Assume that both paths to P via AS#2 and AS#3 are of equal preference regarding the first path attributes up and including step b) (ORIGIN_CODE).
The MED is not compared, as both paths have different peer-ASs. W.l.o.g., let R1 specify the lower BGP ID. So all routers route via R1, except of R2 - this router of course prefers the externally learned path.
When R1's connection to AS#2 breaks (or the path is withdrawn), the same basic effect you have described temporarily appears:
- R3 forwards traffic to R1. 
- R1 forwards traffic to R2 via Rnew due to fast local repair (as R2 uses the path via AS#3, it is known by all reflectors)
- Rnew forwards traffic back to R1.

Thus, we have the same problem in principle, but this time best-external and add-path are not involved.

Regards

Uli

-- 
_______________________________________________________
ULI BORNHAUSER
University of Bonn - Institute of Computer Science IV
c/o Bonn-Aachen International Center for Information Technology B-IT 
Dahlmannstr. 2 - D-53113 Bonn - Germany

Web: www.cs.bonn.edu/IV/ub
Email: [email-masked]			
Phone: +49 (228) 73 - 4550
Fax: +49 (228) 73 - 4571

---

From: Xu Xiaohu <[email-masked]>
Date: Mon, 01 Mar 2010 10:59:38 +0800

Hi

If the route reflector Rnew chooses the route to P learnt from R1 as the
best one, it will not reflect the route to P learnt from R2 to other clients
such as R1.

Best wishes,

Xiaohu

  _____  

发件人: Uli Bornhauser [mailto:[email-masked]] 
发送时间: 2010年2月28日 4:48
收件人: xuxiaohu 41208
抄送: [email-masked]; [email-masked]
主题: Re: [Idr] Transient loop prevention when using the best external and
add-path approaches

Hi Xu,

independent of your discussion about the IGP next-hop down event and the
possible corresponding problems, regarding your original issue, I want to
point out that the transient loop effect you have described is IMHO no
design problem of best-external and add-path, respectively.

In principle, the effect you have described may also appear if classical BGP
and Route Reflection are used. Consider your original example and add a
fourth reflector "Rnew" in the following way (BGP sessions are of course
kept between all reflectors):

         AS#2                   AS#3

          |                       |
          |                       |

Assume that both paths to P via AS#2 and AS#3 are of equal preference
regarding the first path attributes up and including step b) (ORIGIN_CODE).

The MED is not compared, as both paths have different peer-ASs. W.l.o.g.,
let R1 specify the lower BGP ID. So all routers route via R1, except of R2 -
this router of course prefers the externally learned path.

When R1's connection to AS#2 breaks (or the path is withdrawn), the same
basic effect you have described temporarily appears:

- R1 forwards traffic to R2 via Rnew due to fast local repair (as R2 uses
the path via AS#3, it is known by all reflectors)

Thus, we have the same problem in principle, but this time best-external and
add-path are not involved.

Regards

Uli

Hi Robert,

----- 原邮件 -----
发件人: Robert Raszuk <[email-masked]>
日期: 星期六, 二月 27日, 2010 下午7:16
主题: Re: [Idr] Transient loop prevention when using the best external and
add-path approaches

Hi Xu,

For example, one remote peer is down while another one is still 

OK, should

But the main question to ask is why to set next hop to self on 

ASBR for

IPv4 Internet ?

There does exist some scenarios where the local ASBR should set the
next-hops of iBGP routes to self. BGP-free core and Softwire Mesh are two
examples. Even in a normal BGP scenario, if a given local ASBR is connected
to a remote AS via two remote peers (e.g., P1 and P2), it would be better
for the local ASBR to set the next-hops of the iBGP routes to self. In case
that the nexthop of a learnt eBGP route is changed from P1 to P2, if all of 
the path attributes except the nexthop attribute are identical, there is no
need for the local ASBR to withdraw the previous route update and then
advertise a new one. In this way, the route advertisement traffic can be
reduced greatly.

Best wishes,
Xiaohu

fast next hop down event propagation of peering ASBR or interface 

to it.

In the past I had one proposal to still signal it in BGP to 

address the

http://ftp.iptel.org/pub/standards/internet-drafts/draft-raszuk-

ext-failure-propagation-00.txt

unchanged.

Cheers,

R.