Subject: [Idr] Comments on slides-99-idr-03-idr-rs-bfd_ietf99-00.pdf

From: Enke Chen <[email-masked]>
Date: Fri, 21 Jul 2017 14:20:44 -0700

Hi, Jeff:

I have several comments on slides-99-idr-03-idr-rs-bfd_ietf99-00.pdf,
Page 10 "Why not add-paths?":

•Some IXPs do not want to use add-paths or their customers may not want to  use it.

>Enke: Do we understand the reasons?  Please also note that the add-path can be
>Enke: deployed one a per-neighbor basis.

• Which backup path?  Degenerate case becomes add paths all.

>Enke: The RS uses a route selection algorithm for the bestpath calculation.
>Enke: Is there an issue with using the same algorithm (with the bestpath
>Enke: excluded) for selecting the backup path?

• Add-paths is intended for iBGP See non-deployed work in draft -
  pmohapat-idr-fast-conn-restore for eBGP

>Enke: It is true that in general there are potential issues with using add-path
>Enke: for EBGP and that is not recommended.
>Enke:
>Enke: But that is not true for this special RS case.  The deployment practice
>Enke: recommended by RFC 7947 is to have the clients in "receive-only" mode. That
>Enke: is, each client advertises only its bestpath to the RS.  Thus all the paths
>Enke: advertised by the RS are valid and can be used for forwarding.  It is really
>Enke: equivalent to having full or partial mesh of EBGP peers.  I do not see any
>Enke: issue with "protocol correctness" in this special case.  It is actually a
>Enke: good use case (perhaps the only one) for EBGP add-path.
 
•
Example peer scale from DE
-
CIX at their Frankfort exchange:
•
1069 physical ports, 858 logical
•
This means that in some cases, path diversity may be *very* high.  However, 
connectivity issues may be based on connectivity issues within the exchange.

>Enke: Can we get the actual data on "prefix vs path" as a reference for "path
>Enke: diversity"?

Thanks.  -- Enke

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 22 Jul 2017 01:09:07 +0200

​Hi Enke,​

> >Enke: Can we get the actual data on "prefix vs path" as a reference for

I have written small script to produce actual data you asked for from
quoted in the slides IX:

1 path - 174868 prefixes
2 paths - 36877 prefixes

No prefixes with 3 or more paths.

So to send all paths to all clients in this exchange each client's ASBR
will burn extra 4.22 MB of RAM if b_path size is 120 bytes or 7 MB if
b_path size is 200 bytes.

Kind regards,
//RR.

---

From: Gert Doering <[email-masked]>
Date: Sat, 22 Jul 2017 09:34:22 +0200

Hi,

> No prefixes with 3 or more paths.

Which IX data is this based on?

I'm asking because at DECIX, there are peers that have 3 or 4 routers
from the same AS on the DECIX fabric, so these peers alone would contribute
3+ paths - same AS path(!), but different next-hops...

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 22 Jul 2017 12:23:30 +0200

Hi Gert,

The data I got and used to calculate number of paths indeed *only* assumes
different AS_PATH for each prefix to indicate presence of more then one
path. There are no paths with different next hops in the RS dump I
received. I assumed that the data is complete.

Let me ask them again to get the data which will include same AS_PATH +
different next hops and as soon as I get it I will repost the result.

Cheers,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 22 Jul 2017 18:01:30 +0200

Hi,

Thanks to Gert's comment I now asked for full complete dump which will
include *all* paths on the RS in question including those with identical
AS_PATH, but different next hops.

Here is the distribution from one major RS in Europe:

*prefixes    paths *

 123453 1
  48165 2
  25069 3
  10316 4
   6511 5
   2347 6
   1227 7
    556 8
    560 9
    254 10
     17 11
      4 12
      2 15
      1 18
      1 19
      1 20
      1 24
      1 29

I also did similar count for one RS in US:

  81498 1
  11207 2
    590 3
     71 4
      4 5
      1 6
      1 8

Another RS in US:

  27017 1
    448 2
     52 3

One RS in Japan:

   9214 1
    518 2
     25 3
      7 4

If there are other IX operators and would like to send me offline their
full BGP dump I can provide more summary data to the list.

Hope this helps,
Robert.

- - -

---

From: Ben Maddison <[email-masked]>
Date: Sat, 22 Jul 2017 20:11:56 +0000

Hi Robert,

Thanks for the useful and interesting info.
I'd point out that the addition of ADD_PATH to the RS at a given IXP would remove one (of many) of the operational issues present in the use of an RS by a participant that operates a selective policy, and may therefore increase participation.
The numbers below should therefore be taken as a baseline from which path diversity may grow if implemented. By how much it might grow is of course down to guesswork that I won't get into.

Cheers,

Ben

--
Sent from Hiri<https://www.hiri.com/>

On 2017-07-22 18:02:07+02:00 Robert Raszuk wrote:

Hi,
Thanks to Gert's comment I now asked for full complete dump which will include *all* paths on the RS in question including those with identical AS_PATH, but different next hops.
Here is the distribution from one major RS in Europe:
prefixes    paths
 123453 1
  48165 2
  25069 3
  10316 4
   6511 5
   2347 6
   1227 7
    556 8
    560 9
    254 10
     17 11
      4 12
      2 15
      1 18
      1 19
      1 20
      1 24
      1 29
I also did similar count for one RS in US:
  81498 1
  11207 2
    590 3
     71 4
      4 5
      1 6
      1 8
Another RS in US:
  27017 1
    448 2
     52 3
One RS in Japan:
   9214 1
    518 2
     25 3
      7 4
If there are other IX operators and would like to send me offline their full BGP dump I can provide more summary data to the list.
Hope this helps,
Robert.
- - -
On Sat, Jul 22, 2017 at 12:23 PM, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:
Hi Gert,
The data I got and used to calculate number of paths indeed *only* assumes different AS_PATH for each prefix to indicate presence of more then one path. There are no paths with different next hops in the RS dump I received. I assumed that the data is complete.
Let me ask them again to get the data which will include same AS_PATH + different next hops and as soon as I get it I will repost the result.
Cheers,
R.

On Sat, Jul 22, 2017 at 9:34 AM, Gert Doering <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

Gert Doering
        -- NetMaster

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444<tel:%2B49%20%280%2989%2F32356-444>           USt-IdNr.: DE813185279

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 24 Jul 2017 22:22:11 +0200

Adding one more v4 data point from RS BGP table received from friends in
Moscown Internet Exchange:

*  prefixes paths*

  31201 1
  11613 2
   4379 3
   3628 4
   1900 5
   1451 6
    816 7
    696 8
    317 9
    101 10
     62 11
     62 12
      4 13
      1 16
      1 23

//RR.

On Sat, Jul 22, 2017 at 10:11 PM, Ben Maddison <[email-masked]>
wrote:

> I'd point out that the addition of ADD_PATH to the RS at a given IXP would
> remove one (of many) of the operational issues present in the use of an RS
> by a participant that operates a selective policy, and may therefore
> increase participation.
> The numbers below should therefore be taken as a baseline from which path
> diversity may grow if implemented. By how much it might grow is of course
> down to guesswork that I won't get into.
>
> Cheers,
>
> Ben
>
> --
> Sent from Hiri <https://www.hiri.com/>
>
>
>> The data I got and used to calculate number of paths indeed *only*
>> assumes different AS_PATH for each prefix to indicate presence of more then
>> one path. There are no paths with different next hops in the RS dump I

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 31 Jul 2017 15:13:04 -0400

Enke,

> >Enke: deployed one a per-neighbor basis.

I'll have to leave this motivation to the coauthors, who also are associated
with IXP.  Resource consumption has been offered as an example consideration
by others.

> >Enke: excluded) for selecting the backup path?

The point was intended to address the case where N paths are available and
you send a fixed number, say 2.  So, if 3 paths are available and paths 1
and 2 are not reachable, you're missing the third path that is.

My general experience with customers deploying add-paths is that they select
a number of paths that both matches their resource consumption and their use
case.  This is particularly important since it's not possible to usefully
constrain the paths on *receipt*.

The IXP case is somewhat unusual.  The number of total paths is often low
enough that add-paths might be acceptable to some situations.  However, it
used to be the case that in per-view cases that some customers may choose
what IXP routes are present in their view based not only on policy but also
on resource constraints.

> >Enke: good use case (perhaps the only one) for EBGP add-path.

I agree that the IXP case, again being unusual, does remove a portion of the
motivations.  It does mean, however, that routes would solely tie-break in
many cases on age per RFC 5004.  This is presuming equal length paths.

This thought makes my brain itch.  Given third party nexthops and
inconsistent tie-breaking based on the rib-out dissemination from the RS to
a particular peer... is that safe?  Possibly so. I'm not up for determinism
Sudokku at this part fo the day.

> >Enke: Can we get the actual data on "prefix vs path" as a reference for "path

Robert was kind enough to supply some stats.

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 1 Aug 2017 00:04:27 +0200

> with IXP.  Resource consumption has been offered as an example

​So if resource of ADD-PATHS ALL is a major concern here how does new SAFI
solves this as compared with just sending all eligible next hops from RS to
clients in a new attribute ?

After all client must establish all BFD sessions (or other probing) to
detect liveness of next hops, but client would not need to report back who
is up or down and would not need to wait for RS computing and distributing
new best path. He can simply switch locally forwarding to go to those next
hops which are up.

Likewise RS would not need to become super powerful controller ...

Thx,
R.