Subject: [Idr] I-D Action: draft-ietf-idr-rs-bfd-03.txt

From: [email-masked]
Date: Mon, 03 Jul 2017 08:56:54 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Inter-Domain Routing of the IETF.

        Title           : Making Route Servers Aware of Data Link Failures at IXPs
        Authors         : Randy Bush
                          Jeffrey Haas
                          John G. Scudder
                          Arnold Nipper
                          Thomas King
	Filename        : draft-ietf-idr-rs-bfd-03.txt
	Pages           : 12
	Date            : 2017-07-03

Abstract:
   When BGP route servers are used, the data plane is not congruent with
   the control plane.  Therefore, peers at an Internet exchange can lose
   data connectivity without the control plane being aware of it, and
   packets are lost.  This document proposes the use of a newly defined
   BGP Subsequent Address Family Identifier (SAFI) both to allow the
   route server to request its clients use BFD to track data plane
   connectivity to their peers' addresses, and for the clients to signal
   that connectivity state back to the route server.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-idr-rs-bfd/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-idr-rs-bfd-03
https://datatracker.ietf.org/doc/html/draft-ietf-idr-rs-bfd-03

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-idr-rs-bfd-03

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: "John G. Scudder" <[email-masked]>
Date: Mon, 3 Jul 2017 12:04:23 -0400

Hi All,

This is a significant rewrite from the previous version. Most of the changes are refactoring for clarity and readability, but their extent and volume means rfcdiff isn't so helpful; sorry about that.

There is one substantive change worth calling out: the previous version used a plain IPv4/IPv6 NLRI format, and communicated the reachability state of tracked next hops using an extended community. The new version uses its own NLRI format, and embeds reachability state in the NLRI. In other respects, the intent is that the specified behavior is the same as the previous version.

Thanks and (to those in the U.S. and elsewhere who celebrate it) have a good holiday.

Regards,

--John

> On Jul 3, 2017, at 11:56 AM, [email-masked] wrote:
> 
> 
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

---

From: Job Snijders <[email-masked]>
Date: Mon, 3 Jul 2017 18:08:00 +0200

Hi,

Does this version include anything related to my remarks in this
message?

    https://mailarchive.ietf.org/arch/msg/idr/2LgFxEqjmWSVuAC5PM103pR7buU

Kind regards,

Job

On Mon, Jul 03, 2017 at 08:56:54AM -0700, [email-masked] wrote:
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: "John G. Scudder" <[email-masked]>
Date: Mon, 3 Jul 2017 12:19:08 -0400

>    https://mailarchive.ietf.org/arch/msg/idr/2LgFxEqjmWSVuAC5PM103pR7buU

Yes, at least in some small way:

   Throughout this document, we generally assume that the route server
   being discussed is able to represent different RIBs towards different
   clients, as discussed in section 2.3.2.1 of [RFC7947].  If this is
   not the case, the procedures described here to allow BFD to be
   automatically provisioned between clients still have value; however,
   the procedures for signaling reachability back to the route server
   may not.

My summary of the major points in your your wall-of-text :-) is

a. There is value in the provisioning BFD part,
b. There is not value in the communicating reachability state back to the server part.

The paragraph above hopefully captures (a). As for (b), I (as an author) didn't see WG consensus to get rid of that element of the draft. Since that was exactly the raison d'être of the draft, it would have been pretty cheeky to cut it out without WG consensus to make that kind of major course change (more than just an "optimalisation" as you described it).

A possible middle ground would be to more clearly divide the procedural elements of (a) and (b) in the draft, so that an implementation could clearly say it does (a) but not (b). If the WG wants to do this, I think it may be easier with the refactored version.

HTH,

--John

---

From: Job Snijders <[email-masked]>
Date: Mon, 3 Jul 2017 18:38:46 +0200

On Mon, Jul 03, 2017 at 12:19:08PM -0400, John G. Scudder wrote:
> The paragraph above hopefully captures (a). As for (b), I (as an
> author) didn't see WG consensus to get rid of that element of the
> draft. Since that was exactly the raison d'être of the draft, it would
> have been pretty cheeky to cut it out without WG consensus to make
> that kind of major course change (more than just an "optimalisation"
> as you described it).

Ah, yes of course WG consensus is important. I had assumed that the
authors were expressing their 100% agreement with all my points through
silence. ;-)

I look forward to hearing counterpoints as to why it is useful to
communicate reachability state back to the route server, and how the
authors (or other working group participants) envision the scalability
cost aspect.

Kind regard,

Job

---

From: Gert Doering <[email-masked]>
Date: Mon, 3 Jul 2017 19:38:10 +0200

Hi,

> authors (or other working group participants) envision the scalability

I find this a useful point for telemetry so the IXP operator can know
"there is loss of connectivity between participant".

Also, it enables best-path selection on the route-server to pick other
paths if available.

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Job Snijders <[email-masked]>
Date: Mon, 3 Jul 2017 19:53:08 +0200

> "there is loss of connectivity between participant".

Sure, but there is a cost associated with the distribution of this
information. Additionally, IXP operators seem to be able to also
retrieve this information in other ways: real time analysis of
{net,s}flow data, deploying their own site-to-site monitoring (example:
https://ams-ix.net/technical/statistics/real-time-stats) 

I'd argue that "it is may be interesting to know this" is a weaker
argument then "without this knowledge the technology doesnt work". As it
currently stands, all route server implementations are struggling with
high volume, so adding additional load doesn't seem attractive. "Just
because we can" doesn't go well with a resource constrained construct
like a route server.

> Also, it enables best-path selection on the route-server to pick other

This is addressed through ADD-PATH / RFC 7911. By the time
draft-ietf-idr-rs-bfd gets anywhere I expect that ADD-PATH is
implemented on all common platforms. It would seem unlikely to me that
we'll have BGP implementations that implement draft-ietf-idr-rs-bfd but
not RFC 7911.

Kind regards,

Job

---

From: Randy Bush <[email-masked]>
Date: Mon, 03 Jul 2017 22:07:42 +0200

> authors were expressing their 100% agreement with all my points through

actually, you atacked the style and tone of my response, so i gave up.

we have long experience (can you say DataKit?) with bad failures and
hard debugging of partial data plane connectivity interacting with
lack of knowledge of the control plane.

when my router's packets can not reach you, or vice versa, i *really*
want to learn a better place to send them.

remember, this arose from arnold asking me what it would take for me to
be willing to consider using route servers.  btw, doed ntt base its
forwarding on route server supplied control plane at many exchanges?

randy

---

From: Randy Bush <[email-masked]>
Date: Mon, 03 Jul 2017 22:08:42 +0200

> argument then "without this knowledge the technology doesnt work".

without this knowledge, betting my ass on route serves doesn't work.

randy

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 3 Jul 2017 22:16:57 +0200

The intention here has nothing to do with telemetry ...

//RR

---

From: Gert Doering <[email-masked]>
Date: Mon, 3 Jul 2017 22:20:02 +0200

Hi,

> The intention here has nothing to do with telemetry ...

But it *is* telemetry.  "Hey, Route-Server, I can see something that
you do not see" - so the RS can do better educated guesses, and the
IXP operator can go fix the L2 issue.

If that's not an intentional aspect, it's useful nonetheless.

(So what would you say what the *intention* of this is?)

Gert Doering
        -- NetMaster

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 3 Jul 2017 22:21:21 +0200

Randy,

Route server should not be bothered with any of his 100s clients suffering
from broken mpls base data plane bugs.

It can send all client two (or more) diffetent paths and remain stateless.

At this point I oppose to proceed with this document any further.

Thx
//RR.

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 3 Jul 2017 22:24:39 +0200

Is result of bfd a telemetry ? Interesting ..

---

From: Job Snijders <[email-masked]>
Date: Mon, 03 Jul 2017 20:27:20 +0000

> actually, you atacked the style and tone of my response, so i gave up.

I'm not sure I know what you are referring to. I sent feedback on the draft
to the list (in a fresh thread) and only received a response from Robert
Raszuk. If I discouraged dialogue on this topic in some way, please point
me to it and allow me an opportunity to apologize for doing so.

we have long experience (can you say DataKit?) with bad failures and hard
> debugging of partial data plane connectivity interacting with lack of
> knowledge of the control plane.

Yes. But a route server will never be a full replacement for regular
bilateral sessions, so that will always remain. If the implication is that
the technology (full state available to the RS) is critical in the
maintenance and operation of the IXP itself, it should clearly state so.
This would be new information to me though.

As a generalized mechanism, IXPs could snoop BFD traffic to disseminate the
current state of reachability on the fabric, such BFD snooping would work
for both RS peers as well as bilateral peers. A question would be "why
impose additional reporting duties (load) on the RS peers when the IXP can
glean similar information through snooping"?

when my router's packets can not reach you, or vice versa, i *really* want
> to learn a better place to send them.

Yes. But how does statement show that the route server itself must be kept
appraised on who can reach who?

Kind regards,

Job

---

From: Job Snijders <[email-masked]>
Date: Mon, 03 Jul 2017 20:29:37 +0000

> without this knowledge, betting my ass on route serves doesn't work.

I don't follow. If the route server facilitates you in setting up BFD to
all "nexthops of interest", the route server's job is done, right? Any RS
peer has the ability to locally verify reacahbility across the fabric and
route accordingly.

Kind regards,

Job

---

From: Gert Doering <[email-masked]>
Date: Mon, 3 Jul 2017 22:38:00 +0200

Hi,

> Is result of bfd a telemetry ? Interesting ..

Of course it is, if you tell the results of measurements A->B to C.

Whatever measurement this is.

Gert Doering
        -- NetMaster

---

From: Job Snijders <[email-masked]>
Date: Mon, 03 Jul 2017 20:42:14 +0000

> Whatever measurement this is.

You are right, any observation of a state can be distributed as a
telemetry.

What I question is whether it is useful (performance or scalability wise)
to have route servers take that particular state into consideration when
doing best path (or multiple best paths) selection on behalf of their peers.

Kind regards,

Job

---

From: Randy Bush <[email-masked]>
Date: Mon, 03 Jul 2017 22:53:12 +0200

> peer has the ability to locally verify reacahbility across the fabric and

route servers are for the small folk who can not do full mesh, let alone
add path and other fancy things.  the rs IS their control plane, period.

---

From: Job Snijders <[email-masked]>
Date: Mon, 03 Jul 2017 21:10:11 +0000

> add path and other fancy things.  the rs IS their control plane, period.

If a device can't do add-path, can we really expect these devices to do
fancy things like opportunistic "route server assisted" BFD to all
next-hops of interest?

If we operate under the assumption that these clients can't support
receiving multiple paths through add-path, I envision a somewhat ugly
cadence:

1) Hey RS, can't reach A
2) Ok, here try reaching A via B
3) hey RS, I can't reach B either
Etc, etc

The above scenario might look small, but often problems at IXPs don't
affect (isolate) a single participant, but rather clusters of participants.
A challenging aspect of IXPs are the "ISLs" between sites.

If the RS is to receive and keep track of the state of who can reach who,
it should somehow create a clear benefit that visibly reduces the load on
the participant's devices. If the only benefit is "if the RS keeps track of
this state, the participant doesn't need to support ADD-PATH (but only
draft-ietf-idr-rs-bfd)", I'm skeptical whether I can get behind that
reasoning.

Kind regards,

Job

---

From: "John G. Scudder" <[email-masked]>
Date: Mon, 3 Jul 2017 17:18:37 -0400

> On Jul 3, 2017, at 5:10 PM, Job Snijders <[email-masked]> wrote:
> 
> 
> On Mon, 3 Jul 2017 at 22:53, Randy Bush <[email-masked] <mailto:[email-masked]>> wrote:
> If a device can't do add-path, can we really expect these devices to do fancy things like opportunistic "route server assisted" BFD to all next-hops of interest?
> 
> If we operate under the assumption that these clients can't support receiving multiple paths through add-path, I envision a somewhat ugly cadence:
> 
> 1) Hey RS, can't reach A

If that cadence happens, they haven't implemented what's in rs-bfd, right? So why would even #1 happen?

--John

---

From: Job Snijders <[email-masked]>
Date: Mon, 03 Jul 2017 21:31:49 +0000

On Mon, 3 Jul 2017 at 23:18, John G. Scudder <[email-masked]> wrote:

> So why would even #1 happen?

Perhaps my understanding of the draft is flawed,  but rs-bfd (currently)
specifies the concept of relaying information from RS participant back to
the RS about who the participant can reach or not. So #1 can only happen if
rs-bfd is implemented, and the cadence would be a logical consequence if
add-path is not supported.

Kind regards,

Job

---

From: John Scudder <[email-masked]>
Date: Mon, 3 Jul 2017 17:39:49 -0400

On Jul 3, 2017, at 5:31 PM, Job Snijders <[email-masked]> wrote:
> 
> On Mon, 3 Jul 2017 at 23:18, John G. Scudder <[email-masked] <mailto:[email-masked]>> wrote:
> 
>> On Jul 3, 2017, at 5:10 PM, Job Snijders <[email-masked] <mailto:[email-masked]>> wrote:
> Perhaps my understanding of the draft is flawed,  but rs-bfd (currently) specifies the concept of relaying information from RS participant back to the RS about who the participant can reach or not.

Yes, but I think the part you are overlooking is that the RS asks the client to track connectivity to all candidate next-hops, whether or not the RS is currently sending the client a route with that next hop. 

> So #1 can only happen if rs-bfd is implemented,

Yes, but.

> and the cadence would be a logical consequence if add-path is not supported. 

That's the error. Numbers 2 and 3 wouldn't happen if rs-bfd were implemented as written, because in step zero the RS would have said "hey client, please use BFD towards A and B" and in step 1, the client would say "hey RS, can't reach A, also can't reach B".

--John

---

From: Job Snijders <[email-masked]>
Date: Mon, 03 Jul 2017 21:55:58 +0000

On Mon, 3 Jul 2017 at 23:40, John Scudder <[email-masked]> wrote:

> the RS about who the participant can reach or not.
>
>
> Yes, but I think the part you are overlooking is that the RS asks the
> client to track connectivity to all candidate next-hops, whether or not the
> RS is currently sending the client a route with that next hop.
>

Conceptually there isn't a big difference between exchanging this
information in bulk or as deltas. But thank you for pointing this out, I
now also wonder whether it is efficient for a RS participant to track
next-hops it migh not have any business with. That seems like a waste of
resources.

> That's the error. Numbers 2 and 3 wouldn't happen if rs-bfd were
> implemented as written, because in step zero the RS would have said "hey
> client, please use BFD towards A and B" and in step 1, the client would say
> "hey RS, can't reach A, also can't reach B".
>

I'm not sure whether events will be packed together this neatly.

Questions that remain:

- is it absolutely necessary for the RS to be aware of the entire bfd
session topology state?

- is it reasonable to assume there will be BGP speakers that support
draft-ietf-idr-rs-bfd, but not RFC 7911?

- if the RS is not stateless (in this regard), can we reasonable expect
things to scale? (As it currently stands the industry has trouble scaling
RS services)

If it was not clear from previous messages: I am supportive of the idea of
"assisted BFD" to take away some obvious downsides of route servers. Just
not sure the proposed approach will work out good enough.

Kind regards,

Job

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 3 Jul 2017 18:20:28 -0400

> On Jul 3, 2017, at 5:55 PM, Job Snijders <[email-masked]> wrote:
> 
> 
> - is it absolutely necessary for the RS to be aware of the entire bfd session topology state?

The draft is written with the idea that some/many devices *won't* be aware.  If the state is Unknown, things still work.

> 
> - is it reasonable to assume there will be BGP speakers that support draft-ietf-idr-rs-bfd, but not RFC 7911?
> 
> - if the RS is not stateless (in this regard), can we reasonable expect things to scale? (As it currently stands the industry has trouble scaling RS services)
> 
> If it was not clear from previous messages: I am supportive of the idea of "assisted BFD" to take away some obvious downsides of route servers. Just not sure the proposed approach will work out good enough. 

The way I suggest reading this:
1. BFD to the BGP nexthops is really desirable.  It lets you avoid blackholing in several well known scenarios, including the RS based distribution of routes.
2. Regular BFD needs both sides of the session to be prepared.  In the absence of full mesh BFD configuration, there needs to be a feature that permits the RS to notify the clients what nexthops are of interest on both sides.
3. The feedback mechanism permits a RS to send alternate paths - when it is capable of doing so - when a given nexthop is unreachable by a given client.

The third mechanism is very optional.  Many RSes need not implement any sort of new path selection if they don't want to; points 1 and 2 are still helpful. 

To the points about add-paths, and statefulness, even if the RS clients spoke add-paths, you have to choose what level of path scaling you want to do.  If you're to the point of sending everything, a RS isn't even necessary.  And clearly, people use RSes partially for "remote policy" - effectively path pruning.

Looked at a somewhat different way, choosing the backup paths is no different than add-paths or diverse paths feature choosing the next best path.  But since the RS and the client could have already conveyed the possible set of nexthops for the view, there's no need for stepwise discovery.  Perhaps this point needs to be made clearer in the draft.

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 4 Jul 2017 00:22:55 +0200

On Jul 3, 2017 23:40, "John Scudder"

And that is precisly the fundamental flow of this proposal.

Clients are to track next hops for bgp paths they do not even have ...

What happens when RS gets 100 paths for policy eligible net for a given
client ?

Client will enable waist of 100 bfd sessions and what is much worse will
report state of those 100 next hops to RS ?

This is so bad ....

Cheers
R.

Yes, but.

and the cadence would be a logical consequence if add-path is not
supported.

--John

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 4 Jul 2017 00:31:17 +0200

If client talks add-paths you send them two paths. You may send them more
if they wish so.

It is just broken desing to signal failure between nodes to get new paths.
PIC has proven this long time back.

What this is about is really a new idea of sdn controller with bgp and
distributed bfd for weak clients.

Now maybe we could understand what are those weak clients who can not
handle two paths for a given net ?

Thx
R.

On Jul 4, 2017 00:20, "Jeffrey Haas" <[email-masked]> wrote:

> The draft is written with the idea that some/many devices *won't* be
> aware.  If the state is Unknown, things still work.
>
> >
> > If it was not clear from previous messages: I am supportive of the idea
> of "assisted BFD" to take away some obvious downsides of route servers.
> Just not sure the proposed approach will work out good enough.
>
> 1. BFD to the BGP nexthops is really desirable.  It lets you avoid
> blackholing in several well known scenarios, including the RS based
> distribution of routes.
> 2. Regular BFD needs both sides of the session to be prepared.  In the
> absence of full mesh BFD configuration, there needs to be a feature that
> permits the RS to notify the clients what nexthops are of interest on both
> sides.
> 3. The feedback mechanism permits a RS to send alternate paths - when it
> is capable of doing so - when a given nexthop is unreachable by a given
> client.
>
> The third mechanism is very optional.  Many RSes need not implement any
> sort of new path selection if they don't want to; points 1 and 2 are still
> helpful.
>
> To the points about add-paths, and statefulness, even if the RS clients
> spoke add-paths, you have to choose what level of path scaling you want to
> do.  If you're to the point of sending everything, a RS isn't even
> necessary.  And clearly, people use RSes partially for "remote policy" -
> effectively path pruning.
>
> Looked at a somewhat different way, choosing the backup paths is no
> different than add-paths or diverse paths feature choosing the next best
> path.  But since the RS and the client could have already conveyed the
> possible set of nexthops for the view, there's no need for stepwise
> discovery.  Perhaps this point needs to be made clearer in the draft.
>
> -- Jeff
>

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 3 Jul 2017 18:33:09 -0400

> On Jul 3, 2017, at 6:31 PM, Robert Raszuk <[email-masked]> wrote:
> 
> What this is about is really a new idea of sdn controller with bgp and distributed bfd for weak clients. 
> 

Regardless of what you might think of the idea of signaling back to the RS to permit the RS to bias its own route selection, if you think the connectivity test to the nexthop is of little value, you haven't done nearly enough work with IXPs.

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 4 Jul 2017 00:39:12 +0200

I never questioned connectivity check between clients.

I question  (similar to Job's observation) to put RS in the middle of the
repair chain of events.

Thx,
R.

PS. I do recommend we focus on the problem rather then going personal on
who understands IX env better.

On Jul 4, 2017 00:33, "Jeffrey Haas" <[email-masked]> wrote:

> Regardless of what you might think of the idea of signaling back to the RS
> to permit the RS to bias its own route selection, if you think the
> connectivity test to the nexthop is of little value, you haven't done
> nearly enough work with IXPs.
>
> -- Jeff
>
>

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 4 Jul 2017 00:50:17 +0200

Also Jeff what makes you assume that IX operators and IX env are so bad as
compared with other enterprise or sp igp networks ?

After all I do know a lot of enterprise PEs with such old code which is
never going to support add-paths.

Is next draft going to extend new SAFI to make regular RRs selecting paths
based on client's feedback ?

Is this really the direction we should be all endorsing here ?

Thx
R.

---

From: John Scudder <[email-masked]>
Date: Tue, 4 Jul 2017 01:07:41 +0000

On Jul 3, 2017, at 6:22 PM, Robert Raszuk <[email-masked]> wrote:
> 
> What happens when RS gets 100 paths for policy eligible net for a given client ? 
> 
> Client will enable waist

Waist?

> of 100 bfd sessions and what is much worse will report state of those 100 next hops to RS ?

Unclear to me why reporting 100 next hops is so bad?

> This is so bad .... 

Perhaps you can elaborate on why you think so. 

--John

---

From: Gert Doering <[email-masked]>
Date: Tue, 4 Jul 2017 10:38:53 +0200

Hi,

> handle two paths for a given net ?

What makes you assume that these two paths will *work*?

IXPs are big (like, 600 nodes), and there might be a partial outage that 
kills off everything behind one 100G link in a 4x 100G bundle -> 100+
remotes are gone.

Two paths won't do much good then, if both next-hops are in that set.

Gert Doering
        -- NetMaster

---

From: Gert Doering <[email-masked]>
Date: Tue, 4 Jul 2017 10:41:35 +0200

Hi,

> compared with other enterprise or sp igp networks ?

Nobody said "IX operators are bad".  But IXP fabrics are tricky things
- too large, too many paths, not enough visibility into the end nodes
(= participant routers), critical path for many users.

> never going to support add-paths.

Last time I looked, IDR wasn't really targetting "enterprise internal"

Gert Doering
        -- NetMaster

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 4 Jul 2017 11:02:55 +0200

John,

Today vast majority of OPEN policy clients share single RS view/table so
you do best path and update generation once for all of them.

The moment you introduce per client state you are braking those apart into
individual views.

Even those with policy today have subset of nets in their own tables not
all. So this optimization also goes away.

So if you send clients real paths they locally can repair failures without
any additional state per client on RS. That is why I called it "a waist".

And this is not 100 it is 100 x number of clients say 1000 of dynamic state
which triggers 1000 best paths and update generations on propagated
failures of best path.

Thx
R.

---

From: Job Snijders <[email-masked]>
Date: Tue, 4 Jul 2017 11:26:46 +0200

On Tue, Jul 04, 2017 at 10:38:53AM +0200, Gert Doering wrote:
> > If client talks add-paths you send them two paths. You may send them
> > more if they wish so.
> > 
> > It is just broken desing to signal failure between nodes to get new
> > paths.  PIC has proven this long time back.
> > 
> > Now maybe we could understand what are those weak clients who can
> > not handle two paths for a given net ?
> 
> IXPs are big (like, 600 nodes), and there might be a partial outage
> that kills off everything behind one 100G link in a 4x 100G bundle ->
> 100+ remotes are gone.
> 

Yes, two paths won't help when faced with catastrophic events (where
large clusters of participants of the IXP go offline). But should it?

In that case the IXP participant will have to use other means (IP
transit, private peering) until the IXP has been repaired (usually done
within a few hours?)

We already observed that path _diversity_ isn't that great at IXPs in
the sense that often there is only a single path towards a prefix:
https://mailarchive.ietf.org/arch/msg/idr/qhgG2cCs0UuDHdPRdfDIMs7jeHs

I posit that it is safe to assume that other (non-RS) paths will always
exist (via transit, other peering), and as such in the trade-off between
keeping more state at the RS vs trying to keep the RS lean and mean, the
design parameters should be as following:

    - the main purpose of this draft is to ensure that each RS
      participant sets up BFD sessions to the other participants of
      interest, the main purpose is _not_ to ensure there are always
      paths even if half the IX is on fire. The priority should be quick
      fault detection, not to "make the best use of the IX under any and
      all circumstances".

    - we should work from the assumption that if a client supports
      draft-ietf-idr-rs-bfd, it will also have support for ADD-PATH. If
      we cannot assume this, I'd like to understand why.

    - explicitly consider scalability issues. The bigger internet
      exchanges are struggling with their current route server load, so
      if IDR adds any computational cost to their list of tasks, it'll
      need to properly justify that.

    - The technology facilitate setting up the minimum required set of
      BFD sessions, not the maximum.

Kind regards,

Job

---

From: Job Snijders <[email-masked]>
Date: Tue, 4 Jul 2017 12:48:40 +0200

> aware.  If the state is Unknown, things still work.

Like with today's RS? "This draft doesn't make it worse" is hardly an
impressive statement ;-)

> > - if the RS is not stateless (in this regard), can we reasonable
> > expect things to scale? (As it currently stands the industry has
> > trouble scaling RS services)
> > 
> > If it was not clear from previous messages: I am supportive of the
> > idea of "assisted BFD" to take away some obvious downsides of route
> > servers. Just not sure the proposed approach will work out good
> > enough. 
> 

yes.

> absence of full mesh BFD configuration, there needs to be a feature
> that permits the RS to notify the clients what nexthops are of
> interest on both sides.

yes

> 3. The feedback mechanism permits a RS to send alternate paths - when
> it is capable of doing so - when a given nexthop is unreachable by a
> given client.
> 
> The third mechanism is very optional.  Many RSes need not implement
> any sort of new path selection if they don't want to; points 1 and 2
> are still helpful. 

agreed.

> To the points about add-paths, and statefulness, even if the RS
> clients spoke add-paths, you have to choose what level of path scaling
> you want to do.  If you're to the point of sending everything, a RS
> isn't even necessary.  And clearly, people use RSes partially for
> "remote policy" - effectively path pruning.
> 
> different than add-paths or diverse paths feature choosing the next
> best path.  But since the RS and the client could have already
> conveyed the possible set of nexthops for the view, there's no need
> for stepwise discovery. Perhaps this point needs to be made clearer
> in the draft.

But any churn in the possible set of nexthops, directly results in BGP
churn, where as when the route server is not kept in the loop about who
can reach who, there will not be any churn at the route server level
when 2 RS participants cannot reach each other (temporarily).

Kind regards,

Job

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 4 Jul 2017 16:27:47 +0200

Gert,

The argument that we need controller rather then route server because " IXP
fabrics are tricky things" just does not convince me.

Let's compare your arguments vs MSDCs:

> too large

DCs today have fabrics which are orders of magnitude bigger then any IX on
the planet.

> too many paths

Please explain where for a given net so many paths are coming from. If IX
client is connected to 2 or 3 ports in a given IX it is max. As clients
share their own nets it is hardly that they will share someone elses nets
and volunteer to become free transit.

> not enough visibility into the end nodes
(= participant routers)

Please observe then when in MSDCs compute nodes attach to a fabric such
fabric has zero visibility into their state.

> critical path for many users.

Well since going via RS in IX is a local optimization there is always path
around it. Here pls let's not mix direct peerings via IXes as their control
plane is also direct and never goes via any route server.

Cheers,
R.

---

From: "John G. Scudder" <[email-masked]>
Date: Tue, 4 Jul 2017 11:55:23 -0400

One observation regarding the "RR won't scale if it has to do per-client route selection" concern -- it seems to me it would be entirely feasible for an implementation to associate some figure-of-merit with each next hop, a function of its reported reachability by each of its indirect peers. That figure-of-merit could be used during traditional (not per-client) route selection. The general idea would be "if one other client can't reach you, that's their problem, if dozens of other clients can't reach you, maybe we should choose a different next hop if we've got one" [*]. Think of it as making the binary resolvability condition of RFC 4271 9.1.2.1 fuzzy.

I think specifying the function would be beyond the scope of this (or really, any) draft since it's "just policy". But it demonstrates there's some potential benefit from the proposal without going all the way down to per-client route selection.

--John

---

From: "John G. Scudder" <[email-masked]>
Date: Tue, 4 Jul 2017 11:57:15 -0400

which earlier message says in part:

> AMS-IX RS indicated that one out of three routes have an alternative path
> DE-CIX RS 1 out of 2 routes have an alternative path
> INEX RS seemed to have virtually no alternative paths for the routes

The first two seem to me like non-trivial cases of diversity, although of course well short of full coverage. One thing not taken in by your quick survey is whether there's any correlation between traffic and diversity (e.g., it could be that the 1 out of 2 DE-CIX routes that have alternative paths account for well over 50% of the traffic). I don't necessarily advocate trying to go down the path of generating these statistics, because the "one out of two routes would benefit" point already seems like reasonable motivation to me. I'm merely cautioning against reading the numbers as being more conclusive than they are.

--John

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 4 Jul 2017 13:45:23 -0400

> If that's not an intentional aspect, it's useful nonetheless.

Such a side effect was noted in other offline discussions.  But it didn't
make sense to bloat the document with this information.  

Similarly, in the IPsec "VPN" scenario, it also permits a box to collect
telemetry about the health of the tunnel mesh.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 4 Jul 2017 13:48:20 -0400

>       fault detection, not to "make the best use of the IX under any and
[intentional re-ordering of your text...]
>       BFD sessions, not the maximum.

The draft already accomplishes these.

>       need to properly justify that.

The draft does not require the RS to support additional path selection.  It
does, however, enable it if the RS is capable of such things.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 4 Jul 2017 13:53:34 -0400

> Like with today's RS? "This draft doesn't make it worse" is hardly an

It's BGP.  You have to deal with what incremental deployment looks like.

[...]
> when 2 RS participants cannot reach each other (temporarily).

I believe you are confused about the CPU work associated with providing
add-paths actually is.

Add-paths implementations already run N-best calculations.  They must
already include the work of running policy and BGP loop checks.  Adding an
additional exclusion list based on nexthop viability for the view isn't
significantly more work for a given round of route selection.

If you'd like to argue "send them 2, and if they both don't work, let god
and their backup paths sort it out", fine.  You're welcome to run your RS
that way - if you run a RS.  What I don't understand is why you object to
someone else doing something different.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 4 Jul 2017 13:58:27 -0400

On Mon, Jul 03, 2017 at 08:27:20PM +0000, Job Snijders wrote:
> for both RS peers as well as bilateral peers.

I must admit to being boggled at expecting IXP switch fabric endpoints being
expected to sniff on a chatty protocol to derive state.  Why do you think
this makes sense?

> A question would be "why

There's also the matter that the fabric isn't necessarily aware of the
semantics of a session.  It would have to report up to some central box
anyway.

-- Jeff

---

From: Job Snijders <[email-masked]>
Date: Tue, 4 Jul 2017 20:09:04 +0200

On Tue, Jul 04, 2017 at 01:58:27PM -0400, Jeffrey Haas wrote:
> > As a generalized mechanism, IXPs could snoop BFD traffic to
> > disseminate the current state of reachability on the fabric, such
> > BFD snooping would work for both RS peers as well as bilateral
> > peers.
> 
> I must admit to being boggled at expecting IXP switch fabric endpoints
> being expected to sniff on a chatty protocol to derive state.  Why do
> you think this makes sense?

Because some IXPs already use 'snooping' style methods, this is not my
own idea. For instance VIX offers insight into who peers with who at
https://www.vix.at/vix_peeringmatrix_detail_ipv6.html (page may take up
to 60 seconds to load) based on a form of snooping, some other IXPs
snoop ('intercept') ARP traffic to prevent unnecessary chatter. This
isn't new.

> > A question would be "why impose additional reporting duties (load)
> > on the RS peers when the IXP can glean similar information through
> > snooping"?
> 

Yes, but that box wouldn't be the same box as the route server.

Kind regards,

Job

---

From: Job Snijders <[email-masked]>
Date: Tue, 4 Jul 2017 20:14:54 +0200

On Tue, Jul 04, 2017 at 01:53:34PM -0400, Jeffrey Haas wrote:
> If you'd like to argue "send them 2, and if they both don't work, let
> god and their backup paths sort it out", fine.  You're welcome to run
> your RS that way - if you run a RS.  What I don't understand is why
> you object to someone else doing something different.

I'm not sure what to make of this paragraph?

---

From: Jared Mauch <[email-masked]>
Date: Tue, 4 Jul 2017 14:21:28 -0400

> any additional state per client on RS. That is why I called it "a waist".

	I suspect you mean the homophone waste.

> which triggers 1000 best paths and update generations on propagated

	Something I've observed in the past ~20 years is the relative compute
power has gone up significantly for a BGP speaker, as many platforms may even
do hardware offload of BFD, this part isn't what worries me.

	Tracking the N to N-1 BFD state worries me more, I've seen significant
operational issues on multiple vendors with BFD.  These of course are bugs and
should be fixed.

	While I'm not a big fan of route servers personally, many people do use
them in high cost areas to reduce their operational load.  I'm also concerned
about 3rd parties tasking me with unexpected work without constraints.

	Section 4 comment:

	Any thoughts on rate-limiting for the messages?  Some IX fabrics are
large and I cound envison a lot of results.  I'd suggest limiting ReachAsk to
once every 1,3 or possibly even 6 seconds.  Was any thought given to the min/max
rate of ReachAsk?  (I could also see non-rs peers wanting me to send them ReachTell
in the event of someone selling transit over an IX as well).

	Section 5 text comment:

   "Again, since the
   contents of ReachAsk are simply a set of host routes to be tested,
   route selection over a combined ReachAsk MAY be omitted."

	Suggest dropping Again.

	Section 6 comment:

	Prohibition of fe80:: next-hops may make sense here.  One may
not know what outbound interface is in-use.  I'm always wary when I see fe80
addresses being used.
	
	- jared

-- 
Jared Mauch  | pgp key available via finger from [email-masked]
clue++;      | http://puck.nether.net/~jared/  My statements are only mine.

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 4 Jul 2017 14:36:10 -0400

> On Jul 4, 2017, at 2:14 PM, Job Snijders <[email-masked]> wrote:
> 

As unicast to Job, my phrasing was inappropriate and I apologize to any who might be offended.

My point here is that at no point does this document mandate the RS implement path selection mechanisms in response to the reported state from the clients.  However, this shouldn't preclude implementations that desire to make use of such state.

Thus, I don't understand the proscriptive desires.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 4 Jul 2017 14:44:29 -0400

> On Jul 4, 2017, at 2:21 PM, Jared Mauch <[email-masked]> wrote:
> 

Most of the issues I've seen here tend to be in a modest number of categories:
- Session scaling with regard to high speed timers.  This proposal suggests very sedate timers, really in the same lines as the ones of BFD in its Down/Init state.
- Session scaling for number of sessions.  This might be an issue for lower end hardware implementations with fixed session tables.  For such issues, it's reasonable for an implementation of this proposal to provide filtering (policy) to select which sessions it desires to form BFD adjacencies with.
- Async vs. Echo mode.  Not every implementation supports Echo.
- Inappropriate BGP behaviors with regard to AdminDown transitions.  I'll even own to the fact we've had this class of bug in our implementation.

> about 3rd parties tasking me with unexpected work without constraints.

As noted above, policy/configuration to limit work is quite reasonable.  "If it hurts, don't do that."

> in the event of someone selling transit over an IX as well).

I had considered this a consequence of the hysteresis comment later in the document.  Perhaps this could be spelled out in further detail if necessary.

> not know what outbound interface is in-use.  I'm always wary when I see fe80

I'm supportive of this.  Link scoping is tricky at the best of times.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 4 Jul 2017 14:46:35 -0400

> On Jul 4, 2017, at 2:09 PM, Job Snijders <[email-masked]> wrote:
> 

Traffic matrixing is pretty standard.
ARP filtering with consequent static ARP is pretty common.
IGMP snooping usually relies on first hop upstream device to implement and it's a very quiet and boring protocol.

BFD is chatty.  I don't think it's the same problem space at all.

-- Jeff

---

From: Job Snijders <[email-masked]>
Date: Tue, 4 Jul 2017 21:13:12 +0200

On Tue, Jul 04, 2017 at 02:36:10PM -0400, Jeffrey Haas wrote:
> My point here is that at no point does this document mandate the RS
> implement path selection mechanisms in response to the reported state
> from the clients.  However, this shouldn't preclude implementations
> that desire to make use of such state.
> 

Interesting, my perception of the document was different. From section 4
it is not clear to me that this behaviour is optional:

    "The RS uses the received ReachTell routes as input to the NHIB and
    hence the route selection process it performs on behalf of the
    client."

and in section 4.4:

    "In computing its per-client Loc-RIB, the RS uses the content of the
    related per-client NHIB as input to the route resolvability
    condition (section 9.1.2.1 of [RFC4271]).  The next hop being
    resolved is looked up in the NHIB and its state determined:"

and specifically:

    "Down next hops MUST NOT be considered resolvable."

Kind regards,

Job

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 4 Jul 2017 15:18:28 -0400

> On Jul 4, 2017, at 3:13 PM, Job Snijders <[email-masked]> wrote:
> 

I see the issue now.  Some of the language got simplified out of -03 from -02.  My fault as one of the contributors in not catching this in John's rewrite.  

In -02:

3.  Advertising Client Router Connectivity to the Route Server

   As discussed above, a client router will advertise its Adj-NHIB-Out
   to the route server.  The route server SHOULD update the reachability
   information of next hops in the client's NHIB table accordingly.
   Furthermore, the route server SHOULD use reachability information
   from the NHIB as input to its own decision process when computing the
   Adj-RIB-Out for this client. 

The lack of the SHOULD appears to be the point of contention.

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 5 Jul 2017 02:19:06 +0200

Jeff,

This has nothing to do with the major point of contention, but nice try ;).

Bringing in some cosmetics is in no way helpfull to address fundamental
flows of the proposal.

But I must ask ... this appears in spite of major comments against to be a
wg doc. Well so be it. But  do we have multiple implementations of both RS
and clients supporting this proposal ?

Cheers
R.

On Jul 4, 2017 21:19, "Jeffrey Haas" <[email-masked]> wrote:

> I see the issue now.  Some of the language got simplified out of -03 from
> -02.  My fault as one of the contributors in not catching this in John's
> rewrite.
>
> In -02:
>

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 5 Jul 2017 02:27:54 +0200

Jeff,

Running add paths for common policy paths vs per client best path are
orders of magnitude more work.

I can not believe why you would think otherwise ...

Thx
RR

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 4 Jul 2017 22:47:23 -0400

> On Jul 4, 2017, at 8:27 PM, Robert Raszuk <[email-masked]> wrote:
> 
> Jeff,
> 
> Running add paths for common policy paths vs per client best path are orders of magnitude more work.
> 

At no point did I state add-paths would be per client.

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 5 Jul 2017 10:29:41 +0200

Neither did I.

Add-paths is for all clients while signalled bfd state of remote nhs is per
client.

And this is the issue.

Thx
R.

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 5 Jul 2017 17:05:05 +0200

Hi John,

You are describing the situation where client notices next hop down,
notifies RS but does not get new path till other clients also report the
same next hop as down.

Is this really a good idea ?

Note that due to selective policy only very few clients may get given path
so your heuristic would now also need to account for static or dynamic
(signalled with communities) policies.

Well I have nothing against machine learning or AI, but do we really need
it here on RSes ?

Thx
R.

On Jul 4, 2017 17:55, "John G. Scudder" <[email-masked]> wrote:

> One observation regarding the "RR won't scale if it has to do per-client
> route selection" concern -- it seems to me it would be entirely feasible
> for an implementation to associate some figure-of-merit with each next hop,
> a function of its reported reachability by each of its indirect peers. That
> figure-of-merit could be used during traditional (not per-client) route
> selection. The general idea would be "if one other client can't reach you,
> that's their problem, if dozens of other clients can't reach you, maybe we
> should choose a different next hop if we've got one" [*]. Think of it as
> making the binary resolvability condition of RFC 4271 9.1.2.1 fuzzy.
>
> I think specifying the function would be beyond the scope of this (or
> really, any) draft since it's "just policy". But it demonstrates there's
> some potential benefit from the proposal without going all the way down to
> per-client route selection.
>
> --John

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 5 Jul 2017 17:21:28 +0200

Jeff,

> What I don't understand is why you
> object to someone else doing
> something different.

The draft proposes a solution which does not allow to locally repair paths.
That is my biggest issue with this draft.

Solutions which do allow for local repairs are available so why don't we
encorage their use instead of inventing alternatives ?

As to doing something different by someone else is there a pointer to
existing implementations of this wg proposal ?

Are those implementations able to support add-paths ?

Thx,
R.

---

From: John Scudder <[email-masked]>
Date: Wed, 5 Jul 2017 12:43:49 -0400

On Jul 5, 2017, at 11:05 AM, Robert Raszuk <[email-masked]> wrote:
> 
> Hi John,
> 
> You are describing the situation where client notices next hop down, notifies RS but does not get new path till other clients also report the same next hop as down.
> 

If the RS does implement per-client selection, then it gets a new path right away, but concerns have been raised about the computational and storage costs of per-client. The baseline RS as currently deployed will obviously not provide a new path to the client nor will it even know about the client's connectivity issue. I have merely pointed out that there is a middle ground available. Whether it's "really a good idea" or not is for the implementor and operator of the RS to say (and the WG too, I suppose, if we wanted to bake the idea into the draft, though I think that is not necessary).

> Note that due to selective policy only very few clients may get given path so your heuristic would now also need to account for static or dynamic (signalled with communities) policies.

What I described was per next hop, not per route, so I don't think your critique applies. 

> Well I have nothing against machine learning or AI, but do we really need it here on RSes ?

This is a straw man of course.

--John

> 
> Thx
> R.
> 
> On Jul 4, 2017 17:55, "John G. Scudder" <[email-masked] <mailto:[email-masked]>> wrote:
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/idr <https://www.ietf.org/mailman/listinfo/idr>

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 5 Jul 2017 12:47:28 -0400

> it is not clear to me that this behaviour is optional:

I think it would be reasonable to fix that in -04. As Jeff pointed out, I was pretty aggressive in cutting between -02 and -03.

--John

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 5 Jul 2017 19:18:01 +0200

Hi,

I think it does apply.

Reason is that you can't (well shouldn't) delay to send a client a path
with different next hop when previous next hop is down and you know (by
learning per prefix policies) that no other client got such next hop.

Of course implementation wise there can be per nh global counter
incremented when you advertise given path to the client.

Good luck troubleshooting those cases on RS ;).

Cheers,
R.

What I described was per next hop, not per route, so I don't think your
critique applies.

--John

Thx
R.

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 5 Jul 2017 16:15:18 -0400

On Jul 4, 2017, at 3:13 PM, Job Snijders <[email-masked]> wrote:
> Interesting, my perception of the document was different.

Did you overlook this paragraph in the introduction, or did you see it but find it insufficiently clear?

There are some fairly obvious patches to the text you quoted, if that turns out to be the right way to improve clarity. For example (not necessarily final proposal):

> From section 4

NEW:
   The RS uses the received ReachTell routes as input to the NHIB and
   hence the route selection process it performs on behalf of the
   client, assuming it performs one per section 2.3.2.1 of [RFC7947].

>    resolved is looked up in the NHIB and its state determined:"

NEW:
   In computing its per-client Loc-RIB, assuming it maintains one
   per section 2.3.2.1 of [RFC7947], the RS uses the content of the
   related per-client NHIB as input to the route resolvability
   condition (section 9.1.2.1 of [RFC4271]).  The next hop being
   resolved is looked up in the NHIB and its state determined:"

And probably there are other pieces that would need similar.

It would be nice to find a solution that makes the document clear and unambiguous without requiring little caveats all over the place, but if that's what it takes, fine. 

--John

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Wed, 5 Jul 2017 21:13:07 +0000

Another benefit of add-path is convergence time.

With the nexthop address family, when BGP starts,
before the non-best paths will be sent,
 . the RS has to send all the nexthops,
. the client has to set up all the BFD sessions and
. the client has to send all the nexthop reachability info back to the RS.

Thanks,
Jakob.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 6 Jul 2017 00:54:31 +0200

Hi Jakob,

I assume that new SAFI is "informational" what really means that during
client bring up time it has no dependency to real AFs (1/1 & 2/1) best
paths on the client.

Your point is correct when you assume that the best path RS sends out is a
bad one which is sort of unlike if one follows John's suggestion to based
best path selection for client A based on the feedback from other clients
B-Z. But I already expressed my opinion on such scheme ....

Cheers,
R.

---

From: John Scudder <[email-masked]>
Date: Thu, 6 Jul 2017 03:21:10 +0000

On Jul 5, 2017, at 5:13 PM, Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>> wrote:

If you read the draft carefully you'll see this is not the case. Unreported nexthops are specified as having implicit state "unknown" which is eligible for selection.

--John

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 6 Jul 2017 04:04:41 +0000

The reason you need this nexthop address family is to know
which nexthops are unreachable, no?
So that you can send a path other than the best path, no?

I said non-best.

With add-path, you get the non-best up front, so when
connectivity to the best is broken, you already have the alternative.

Thanks,
Jakob.

From: John Scudder [mailto:[email-masked]]
Sent: Wednesday, July 05, 2017 8:21 PM
To: Jakob Heitz (jheitz) <[email-masked]>
Cc: idr wg <[email-masked]>
Subject: Re: [Idr] I-D Action: draft-ietf-idr-rs-bfd-03.txt

--John

---

From: Randy Bush <[email-masked]>
Date: Thu, 06 Jul 2017 10:35:57 +0200

btw, the massive smokescreen of bs is not worth the time of anyone
serious.  please do not expect serious folk to play further.

randy

---

From: John Scudder <[email-masked]>
Date: Thu, 6 Jul 2017 11:17:20 +0000

On Jul 6, 2017, at 12:04 AM, Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>> wrote:

I said non-best.

Yes, my mistake.

--John

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 6 Jul 2017 20:22:24 +0000

The draft proposes to filter paths from best-path selection
based upon nexthop reachability.

This looks more like a new ORF type than a SAFI.
SAFI is for one-to-many information.
ORF is used for peer-to-peer info.
It would certainly make it easier to write the code for it
if it were an ORF type.
Well, 2: one for ReachTell and another for ReachAsk.
Was this considered?

Thanks,
Jakob.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 6 Jul 2017 20:36:50 +0000

Why stop at reachability?

A client may not wish to accept routes from another client, because
they don't have a contract or whatever reason.
Can we include the ASN of the nexthop with each nexthop?

Other possible reasons for not wanting a route:
. The AS-path contains an undesirable ASN.
. The origin AS does not have the right to advertise the prefix.
. I prefer a customer route over a peer route.
. The route is a leak.
. Etc, Etc.

Can you put the whole attribute set into the ReachAsk/ReachTell
rather than just the nexthop?

Heck, why don't you just send the complete route --> Add-path.

Thanks,
Jakob.

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 6 Jul 2017 16:37:20 -0400

> On Jul 6, 2017, at 4:22 PM, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 

I tend to consider this a bit more like rt-constrain than an ORF.  I accept your comparison for the peer-to-peer being a bit more ORF-like than rt-constrain.

I think my biggest bit of feedback is that ORF seems generally less setup for "chatty" state changes than an rt-constrain like mechanism.  ORF still has a bit of the implication of "do a refresh to change state", even if the results are incremental.

-- jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 6 Jul 2017 16:40:31 -0400

Your strawman is about to file a domestic violence restraining order.

These are not our use cases.  If you'd like one, feel free to write a draft covering those.

The draft simply proxies information related to BGP's reachability condition for a nexthop.  If this was full-mesh, the game would be different.  If this was a RR, this would be different.  The RS is (mostly) exceptional in this case.

-- Jeff

> On Jul 6, 2017, at 4:36 PM, Jakob Heitz (jheitz) <[email-masked]> wrote:
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 6 Jul 2017 20:40:39 +0000

From: Jeffrey Haas [mailto:[email-masked]]
Sent: Thursday, July 06, 2017 1:37 PM
To: Jakob Heitz (jheitz) <[email-masked]>
Cc: John Scudder <[email-masked]>; idr wg <[email-masked]>
Subject: Re: [Idr] I-D Action: draft-ietf-idr-rs-bfd-03.txt

On Jul 6, 2017, at 4:22 PM, Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>> wrote:

[Jakob] That makes ORF even more apt. When reachability changes, a refresh of the diffs is required.

-- jeff

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 6 Jul 2017 22:50:41 +0200

Jeff,

Redirect to "write a new draft" is not an option if I need both
reachability based and client to client IX MTU fluctuations based path
selection. And this is a real use case.

//RR.

On Thu, Jul 6, 2017 at 10:40 PM, Jeffrey Haas <[email-masked]> wrote:

> The draft simply proxies information related to BGP's reachability
> condition for a nexthop.  If this was full-mesh, the game would be
> different.  If this was a RR, this would be different.  The RS is (mostly)
> exceptional in this case.
>
>
> -- Jeff
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 6 Jul 2017 20:51:36 +0000

The game is about the RS hiding paths that the client wants.
Nexthop reachability is but one of many criteria to determine
the path that a client wants.
Focusing on nexthop rechability will result in an incomplete solution.
I don't need a draft. A complete solution to the problem already exists:
https://tools.ietf.org/html/rfc7911

Thanks,
Jakob.

Sent: Thursday, July 06, 2017 1:41 PM

-- Jeff

On Jul 6, 2017, at 4:36 PM, Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>> wrote:

Thanks,
Jakob.
_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr

---

From: Arnold Nipper <[email-masked]>
Date: Thu, 6 Jul 2017 23:10:08 +0200

Robert

On 04.07.2017 11:02, Robert Raszuk wrote:

> The moment you introduce per client state you are braking those apart
> into individual views.
> 

vast majority run BIRD as RS and it is BCP to do per client (ASN) best path

Arnold
-- 
Arnold Nipper
email: [email-masked]
mobile: +49 172 2650958

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 7 Jul 2017 10:17:43 +0200

Hi Arnold,

​Perhaps it is a Common Practice but it is hardly "​Best" one. That must be
why so many folks already complain today about their RS scaling issues.

Copying OPEN policy nets and paths to individual per client views seems
very suboptimal for RS design. Well you get what you paid for :).

To contrast that let me remind everyone that Route Server design in Cisco
IOS XE keep all common policy paths in single global BGP "view" and only
when there is static or dynamic (by communities) policy moves subject to
policy paths to individual views. So clients get sigma of per view + global
updates. And you can run on your existing x86 RS blade too.

The draft under discussion breaks this design and its scaling properties.

Now to someone who mentioned that this is all about weak client routers
let's face the numbers.

The data from various route servers (two in Asia, one in EU and one in US)
I have captured recently show:

IX1 - 250K IPv4 total paths
IX2 - 167K IPv4 total paths
IX3 - 50K IPv4 total paths
IX4 - 30K IPv4 total paths

So with 120 bytes per path (this is size of BGP path data structure in
efficient BGP code base) even if you send all clients all paths with
add-paths you need on client control plane total RAM of:

IX1 -  30 MB RAM
IX2 -  20 MB RAM
IX3 -  6 MB RAM
IX4 - 3.5 MB RAM

​So what problem is this draft trying to solve ? Save max 10s of ​MB of RAM
on the client ?

Note also that even add-paths is optional. Most IX already today have two
Route Servers. So it is trivial to have such one route-server to act as
primary sending overall best path and second one acting as backup sending
2nd best. (The IOS XE definition of second best mandates different next hop
then overall all best).

All what is needed is few implementation twicks in your RS code vendor to
address the problem at hand.

Cheers,
Robert.