Subject: How do we handle 4 billion CRH-FIB entries ¿?

From: "Pablo Camarillo (pcamaril)" <[email-masked]>
Date: Fri, 22 May 2020 11:00:04 +0000

Hi Ron, authors,

I've been reading the example you provided to Robert/Ole, and there's something concerning to me:

> Consider the following network:
>
>      -    Contains 65,000 routers
>      -    Each router has 500 directly connected neighbors or fewer
>      -    Uses 16-bit CRH
>
>In this network, each node might have 65,499 CRH-FIB entries:
>
>     -     64,999 CRH-FIB entries cause packets to follow the least-cost path to another node in the domain
>     -     500 CRH-FIB entries cause packets to traverse a specific link to a specific neighbor.

In addition, you've said several times that:
> Now recall that identifiers have node local significance

In summary:
You have 65000 routers in your network, and each router has 65499 "node-local" CRH-FIB entries.
In other words, this network has in total 4257435000 node-local CRH-FIB entries.

1.- How does it scale for an operator to configure on CLI more than 4 billion node-local CRH-FIB entries for the entire network?
2.- Assuming that some day in the future you would like to advertise this in the IGP: how is it going to deal with 4 billion node-local entries?

Also, you said:
> Somewhere in the network, there is an entity constructs the CRH and the list that it contains. That entity needs access to another data structure, that includes a global view of each node's CRH-FIB.

So, this entity needs a data structure with 4 billion node-local entries, and I guess some path computation algorithms with that scale...

Note that as per your explanation, we could configure the same CRH-FIB entry/tag on several nodes. However this changes nothing: as long as entries are defined as node-local you still have to deal with 4 billion.

In my opinion this is a fatal scalability problem.

Thanks,
Pablo.

PS. Those numbers are only assuming 65.000 routers; however as part of the 16b vs 32b the authors have stated several times the need to support more routers...

---

From: Ron Bonica <[email-masked]>
Date: Fri, 22 May 2020 15:37:24 +0000

Pablo,

The scope of the current draft is limited to CRH forward plane behavior and CRH-FIB scalability. Your question concerns the scalability of a path computation module. While your question is legitimate, it is beyond the scope of this document and the charter of this working group.

Nonetheless, I will try to answer it. Section 7 of this document says that although identifiers have node local significance, they can be assigned as if they had domain-wide significance. In my last email to Robert, you will see that I allocated loosely routed identifiers as if they had node local significance.

In many cases, this strategy is beneficial and facilitates scaling. You will see it in the IS-IS extensions for CRH <https://datatracker.ietf.org/doc/draft-bonica-lsr-crh-isis-extensions/> document. When this strategy is used, control planes that support CRH have roughly the same scaling characteristics as control planes that support SR-MPLS.

                                                             Ron

Juniper Business Use Only
From: Pablo Camarillo (pcamaril) <[email-masked]>
Sent: Friday, May 22, 2020 7:00 AM
To: Ron Bonica <[email-masked]>; [email-masked]
Cc: [email-masked]; Ole Troan <[email-masked]>; Robert Raszuk <[email-masked]>
Subject: How do we handle 4 billion CRH-FIB entries ¿?

[External Email. Be cautious of content]

Hi Ron, authors,

In summary:

Thanks,
Pablo.

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 22 May 2020 17:56:23 +0200

Hi Ron,

In many cases, this strategy is beneficial and facilitates scaling. You
> will see it in the IS-IS extensions for CRH
> <https://datatracker.ietf.org/doc/draft-bonica-lsr-crh-isis-extensions/>document.
> When this strategy is used, control planes that support CRH have roughly
> the same scaling characteristics as control planes that support SR-MPLS.
>

I am afraid the above assertion is highly incorrect.

in SR-MPLS Node_SIDs are globally unique within a domain. That drastically
reduces the scaling issue we are facing with CRH.

Moreover in SR-MPLS when locally significant adj SIDs are used they are
always used coupled with Node SIDs:

   A packet injected anywhere within the SR domain with a segment list
   {SN, SNL} where SN is the Node-SID of node N and SNL is an Adj-SID
   attached by node N to its adjacency over link L will be forwarded
   along the shortest path to N and then be switched by N, without any
   IP shortest-path consideration, towards link L.

So your analogy comparing CRH to SR-MPLS scaling is I am afraid a fake news.

Many thx,
R.

---

From: Sander Steffann <[email-masked]>
Date: Fri, 22 May 2020 22:58:33 +0200

Robert,

> So your analogy comparing CRH to SR-MPLS scaling is I am afraid a fake news.

Calling something fake news has all kinds of negative connotations. Can you please use more constructive wording?

Cheers,
Sander

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 23 May 2020 10:00:15 +1200

Pablo,
On 22-May-20 23:00, Pablo Camarillo (pcamaril) wrote:
...
> 1.- How does it scale for an operator to configure on CLI more than 4 billion node-local CRH-FIB entries for the entire network?

It doesn't. It's clear that the words "configure" and "CLI" must be removed from an operator's vocabulary in such a network. Everything must configure itself, autonomically.

> 2.- Assuming that some day in the future you would like to advertise this in the IGP: how is it going to deal with 4 billion node-local entries?

There will never be such a problem. I'm not sure that RPL is the answer to all questions, but you can forget about traditional IGPs completely in such a network.

> Also, you said:
> 
>> Somewhere in the network, there is an entity constructs the CRH and the list that it contains. That entity needs access to another data structure, that includes a global view of each node’s CRH-FIB.

Indeed, that will not happen at such scale. There needs to be a distributed data structure; there will be no "global view" except at the policy intent level.

This is why some people have been thinking about autonomic networking for some years, and if it sounds like research, some aspects are indeed still research.

That doesn't argue against 32 bits; it just tells you that 32 bits is a strategic, not a tactical, choice. Rather like choosing 128 bits for IPv6, when 64 would have been "enough".

Regards,
    Brian