Subject: Re: [Int-area] Fwd: New Version Notification for draft-li-int-aggregation-00.txt

From: Toerless Eckert <[email-masked]>
Date: Fri, 25 Feb 2022 09:02:24 +0100

Tony,

Thanks for the document. Certainly an interesting topic to discuss.

1. Is there any specific reason to bring this up now, e.g.: urgency to
avoid running out of headroom or the like ? Would be good to add that
to the text for motivation. right now it reads very architectural.

2. There is no distinction made whether/how the scalability issue
affects the routing system differently from the forwarding system.
But i think if we want to step back and take a look at the overal
solution landscape, then it is necessary to make that distinction.

Aka: In one possible universe (where less-stupid router vendors finally
start putting powerful enough control plane CPU/memory into routers),
i may not predominantly have a scalability issue of the routing subsystem,
but only of the forwarding subsystem (hardware forwarding entries). In this
universe i could solve the problem through a combination of two
mechanisms alone:

a) goegraphical aggregateable allocation of address space by registries
b) auto-aggregation within routers from routing-plane to forwarding
   plane. Aka: Just don't populate the poor HW tables with all those
   non-aggregated prefixes, but calculate the minimum number of
   sufficient shorter prefixes.

3. In general, i would suggest to split the asks between that
geographical aggregateable allocation and the technologies that
could then potentially benefit from it. Because: I think we want
the ask for better aggregateable addressing to be pushed into
the registries faster and independent from the individual technical
measures operators will employ to benefit from them. E.g.

- aggregateable addresses first
- auto-aggregation maybe as first optimization vendors could
  deliver, creating minimal operational overhead to operators
- actual aggregation configuration on edge-routers for 
  the control plane whenever/wherever operators can be persuaded
  to do this.

4. It would be great to have a research group in IRTF around addressing,
for various reasons, but in this case of course specifically to have
a community of researchers where one might raise the ask to do more
numerical analysis of possible benefits. Aka: Any of the possible
points going forward (except with the forwarding-plane auto-aggregation
maybe) creates significant operational work for registries / operators,
so there should be sume hard number evidence of how much this cold give
in return.

5. Couple of nits for the first part of the doc appended below.

Cheers
    Toerless

> Abstract
> 
>     Routing and addressing are inexorably tied, and the scalability of

     ^
Nit:
    prepend "In the Internet architecture"

E.g.: If we would have a better architecture, including LISP, we would
arguably have a less than inexorable tie... i think.

...

>    The Internet depends upon the efficiency and scalability of the
>    routing system.

>    Without effective routing, no traffic can be delivered. 

Nit: Why is effectiveness relevant here ? If you think that scaling limits
impact effectiveness, please say so explicitly instead of letting it be
subject to readers guesswork.

> 3.  Routing and Addressing
> 
>    The routing subsystem of the Internet is responsible for discovering
>    paths from any point on the Internet to any other point.

Nit:

s/discovering/announcing/ ?

Arguably if we had a better internet architecture where routes are
discovered on demand, we again might have a different scalability discussion.
Did i mention LISP ?

>    For routing to function, there must be specific names for hosts.  The
>    architecture of the namespace for these names is a critical decision,
>    as these names will have global scope.  When we also use these names
>    as a binding to a location in the network, we call these names
>    'addresses' and the namespace that they are taken from an 'address
>    space'.

Nit:

I don't think its necessary or beneficial for this document to try to
invent how IP addresses are a type of name, and what properties names
have to have to be called addresses. It just creates another instance
of a neverending side discussion. E.g. I don't think we stop calling
IP addresses IP addresses even in instances where they are only identifiers
and not locators. Did i mention LISP ?

E.g. Suggest to leave away mentioning of names, just statig the facts
that in the routing architecture of the Intenet, IP addresses are
used to locate hosts.

> 
>    Scalability is a central concern for routing.  Each item of
>    information that routing must propagate around the network requires
>    processing power and memory for storage throughout the network.  This
>    scales with the size of the network.

Nit: Same number of routes, e.g.: 10,000, case A), small network with
100 routers, case B) large network with 5,000 routers. Q: How is routing
in the B) network 50 times more expensive than in the A) network ?
Are you just adding up the resource requirements across the routers ?
IF so: why is that an interesting metric ? E.g.: IMHO the cost of
routing within a router for example with DV (best case) does not increase
with the size of the network. At best with the number of neighbors.
Aka: Either leave out the argument about the network size or make it more
precise.

>    If routing also scaled linearly
>    with the number of hosts, then the cost of running the routing system
>    would grow as the size of the network times the number of hosts,
>    which is clearly problematic.  For this reason, we cannot have a
>    routing subsystem that just carries individual host routes.

Nit: Its good enough (and IMHO more true), just to look at the cost of
linear growth of routing if we didn't have aggregation. That has been
the problem all along.
> 
> 4.  Abstraction and Aggregation
> 
>    Instead, we seek to define groups of hosts and treat them together as
>    a single abstraction, commonly known as a 'prefix'.  We call the
>    process of combining addresses together into a prefix 'aggregation'.
>    Under some circumstances, prefixes themselves may also be aggregated
>    to form another prefix, resulting in a recursive structure.  If
>    prefix A is a proper subset of prefix B, we say that A is 'more
>    specific' than B and that B is 'less specific' than A.
> 
>    We can then define the routing efficiency of a specific prefix as the
>    cost of carrying that prefix, plus all of its more specifics,
>    integrated across the entire network, and divided by the number of
>    host addresses subsumed by the prefix.

Nit: this looks like a too complex formula. Just say a prefix has an efficiency
of N if it covers N hosts or the like.

>    It is well known that abstraction obscures important detail and that
>    abstraction in routing can cause sub-optimal paths, resulting in
>    extra hops, wasted bandwidth, and managerial difficulties.  As a
>    result, there will always be a trade-off between scalability and
>    optimality when introducing abstractions.

Nit: i'd prefer to stick to the term aggregation instead of bringing
in abstraction as an equivalent. But the text also used prefixes already,
so why not just make the statement about routing for prefixes and maybe
add that the shorter the prefix the greater the risk of such sub-optimalities ?

>    When optimality is paramount and simple reachability is insufficient,
>    the routing subsystem has additional mechanisms that allow network
>    operators to make different path selection choices, sometimes
>    intentionally ignoring or explicitly working against abstraction.  We
>    call this broad set of mechanisms 'traffic engineering'.

Nit: WOuldn't the most simple example be a prefix route and then to get
a more optimum path for some important hosts in that prefix be another
longer-prefix route just for those host ? I don't think we would call that
setup traffic-engineering. Aka: Not sure why we need to bring in yet another
loaded term (traffic engineering). Also: Did i mention LISP (is that
traffic engineering ?) But it would equally provide tools for more optimality.

> 
> 5.  Abstraction Boundaries
> 
>    Abstractions have three different boundaries that we will be
>    concerned with:
> 
>    Abstraction Administrative Boundary
>       An abstraction's administrative boundary occurs at the topological
>       interface between the abstraction's administratively controlled
>       network and other administrations.
> 
topological interface used but not defined.
Would prefer to stick to prefix or aggregate instead of abstration.

> 
>    Abstraction Naming Boundary
>       An abstraction's naming boundary is the topological container of
>       all of the host addresses within that abstraction.

New word for prefix ?

Aka: overall a bit too much abstraction/reinvention of terminology in
the doc for my taste, makes the whole problem look more convoluted as
it is IMHO.

---

From: "[email-masked]" <[email-masked]>
Date: Fri, 25 Feb 2022 07:40:25 -0800

> On Feb 25, 2022, at 12:02 AM, Toerless Eckert <[email-masked]> wrote:
> 
>> Abstract
>> 

There are only so many ways to determine paths, e.g.:
- by the structure of the address itself
- by flooding, at least until you cache a path
- by knowing what flooding would have accomplished, e.g., by ‘global knowledge’ of the network topology

We use all three in the Internet (longest prefix, ARP/LISP, and RIP/OSPF, respectively).

“Better” is relative; structure is more efficient and scalable but harder to manage, flooding is least efficient and scalable but requires no management, and global knowledge is a bit of a balance between the two.

Or is there some other “better” you’re imagining?

Joe

---

From: Tony Li <[email-masked]>
Date: Fri, 25 Feb 2022 08:34:29 -0800

Hi Toerless,

> to the text for motivation. right now it reads very architectural.

Yes. My hair is turning gray. AFAICT, this is not written down elsewhere. If not published, the concept of an abstraction action boundary might be lost. Noel, who deserves 100% of the credit for this has already passed the point of caring.

> but only of the forwarding subsystem (hardware forwarding entries).

I’ll try not to be offended. But I’m failing.

Router vendors already put in very high end CPUs and gobs of memory. And it’s never enough. The routing table continues to grow open loop. And carriers complain about the costs of the extra memory.

> In this

As pointed out, this is already happening.

>   sufficient shorter prefixes.

And some of this is happening, but it is NOT a friendly and rational response to the problem. It has numerous setbacks. Oh, look, a non-aggregatable more-specific showed up. Now I need to expand some of my aggregates. And guess what: it only achieves so much. There are still customers who run out of resources even with this enabled.

Actually doing the aggregation in the control plane is far preferable: it reduces processing and memory requirements for all upstream systems.

>  the control plane whenever/wherever operators can be persuaded

The first two are not asks. They are already complete.  The ask is thus the last point.

> so there should be sume hard number evidence of how much this cold give

That’s your agenda, not mine. I have no interest in ‘research’ in this document. This is purely pragmatic and operational.

> Aka: Either leave out the argument about the network size or make it more

This is correct. It scales with the number of prefixes. Fixed.

> of N if it covers N hosts or the like.

That would not cover the costs of people advertising more-specifics.

> add that the shorter the prefix the greater the risk of such sub-optimalities ?

That is simply not true. /8 is not somehow less optimal than /16. It depends on the topology, not the prefix length.

> traffic engineering ?) But it would equally provide tools for more optimality.

That would definitely be traffic engineering.

> the doc for my taste, makes the whole problem look more convoluted as

Regards,
Tony

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 25 Feb 2022 17:37:40 +0100

I primarily thought that the document wasn't improving by making statements
about routing and addressing in general given how it does specifically seem
to want to improve the situation for the Internet Architecture, so non-Internet
Architecture considerations would be derailments ?!

Aka: Avoid for the document to have to deal with the type of generic discussion
like what you're opening up in your email (which of course is a fun discussion).

For example:

Is LISP really part of the Internet Architecture ? I thought (unfortunately)
not. E.g.: i don't think i can become an Internet transit ISP without participating
in the "native" BGP routing. "Hey, i don't want these gigantic BGP Internet
routing tables, and my customers don't need it. I just want do do LISP".

If we didn't have the need for the global Internet BGP routing table,
but had a requirement that every Internet ISP had to use LISP, would
there be a need for Tony to write this document ? 

Cheers
    Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 25 Feb 2022 18:38:02 +0100

> I’ll try not to be offended. But I’m failing.

Sorry. totally not my intention. Only trying to be helpfull to
strengthen the document.

I just ran against control plane resource limitations in products way more
often during the decades than i felt necessary knowing what control plane
performane would be possible with appropriately scaled CPU/memory. 

> Router vendors already put in very high end CPUs and gobs of memory.
And it’s never enough. The routing table continues to grow open loop.
And carriers complain about the costs of the extra memory.

Maybe its not as bad now as it was in the recent past given how
Moore's law is changing, but my past experience was that dedicated
route processor boards could not compete in price and life-cycle agility
with general purpose data-center servers, but for Internet BGP, there
is IMHO no reason (other than desires for revenue), to NOT use general
purpose data center servers for Internet BGP routing tables. 

So at least we should  take the most price/performance optimized
opex/capex model into account as the reference against which to
vet cost/benefit of change. 

> And some of this is happening, but it is NOT a friendly and rational response to the problem. It has numerous setbacks. Oh, look, a non-aggregatable more-specific showed up. Now I need to expand so…

If some of this is happening but in your opinion not a rational response
that would make it important IMHO to be dicussed in the document. Or
at least add references to wherever this may already be discussed. Without
quantified numbers published to show how this does or does not help,
we can just believe or not believe your assessment.

Also it sounds as if this mechanism is reducing the number of customers
who still have a problem. That too sounds relevant for the community
to make judgements about what you propose.

> Actually doing the aggregation in the control plane is far preferable: it reduces processing and memory requirements for all upstream systems.

The more aggregation you want to support, the more geographic structure
you introduce into the addressing space and the more you will also
run the risk of creating detriments against cross-geographic
shortcut links (oh no, a peering with you would raise the cost of
my routing table undesirably...). Aka we're trying to compare
capex cost for potentially overpriced CPU/RAM in routers with the
cost of operational processes in registries and operators. Thats
a difficult comparison.

> The first two are not asks. They are already complete.  The ask is thus the last point.

Hmm. Then i misread your draft. It did sound as if there is an ask
in your document against registries to improve how they assign address
block such that better geographic aggregation than today was enabled.

> That’s your agenda, not mine. I have no interest in ‘research’ in this document.
> This is purely pragmatic and operational.

My text Wasnt asking for research in this document. Just saying that research
might be one option to create insights in support of this document.

Aka: any numbers either from extrapolating actual deployment/product data
and/or simulation data good enough to stand in for real-world data.
How many routes do customers that have memory issues have today, how much
would they have with your proposal in place. That type of evidence.

> That would not cover the costs of people advertising more-specifics.

Maybe start putting in an example for calculating the routing efficiency.
I have a hard time trying to figure out what formula the text means

> That is simply not true. /8 is not somehow less optimal than /16. It depends on the topology, not the prefix length.

The shorter the prefix length of an aggregate, the more longer aggregates it
could have, right ?

> That would definitely be traffic engineering.

Ok, i wouldn't have guessed that you would consider a longer prefix
traffic engineering.

Let me see if i understand it correctly:

In general we would not want a longer prefix because we want the aggregation
(with your proposal). So now _if_ someone wants a longer perfix to go
on path where it would violate the aggregation, we call it traffic engineering.
But whats the operational mechanism by which one would decide this is
a permitted instance of traffic engineering or a bug / misconfiguration
in the aggregation scheme ?

Cheers
    Toerless

---

From: Tony Li <[email-masked]>
Date: Fri, 25 Feb 2022 13:17:17 -0800

> On Feb 25, 2022, at 9:38 AM, Toerless Eckert <[email-masked]> wrote:
> 

Well, here’s the problem: Internet usage is decoupled from all other applications. Data centers, even some of the hyperscalars, do not have the resource requirements of Internet routing. And yes, this exacerbated by the policies and practices of ISPs.

It’s to the point where the resource and cost differential between ’normal’ products and ISP products have forked into two separate product variants. And even then, continuing to scale the ISP products is challenging.

As a vendor, I should be happy about this: it all goes to better margins. However, as an engineer, I’m aghast. Most of this is waste. If we paid more attention to efficiency, we would be in a much better place. Unfortunately, efficiency is not sexy and cool.

> purpose data center servers for Internet BGP routing tables. 

The volumes are ten orders of magnitude lower, so that’s no surprise. And just having routing without direct coupling to a forwarding plane restricts you to off-box applications, which is missing a key part of the problem.

But while this is interesting, it is also irrelevant to the matter at hand: we can be more efficient in our routing if we want to.

> that would make it important IMHO to be dicussed in the document.

The point is to make routing more efficient, not to talk about FIB compression mechanisms.

> Or

I’m not asking you to believe anything. I’m simply pointing out the capabilities that are available today for anyone to employ should they so choose.

> cost of operational processes in registries and operators. Thats

And I’m not trying to force anyone one way or another. I’m simply documenting the tool.

> block such that better geographic aggregation than today was enabled.

Only for geographic addressing that is not already covered by per-registry block allocation. Largely, that means allocating blocks to Australia. :)

> The shorter the prefix length of an aggregate, the more longer aggregates it

That’s irrelevant. We are not concerned with potential downsides. The question is how can we improve efficiency.

> (with your proposal).

We do not want to carry longer prefixes when they point to the same next hops.  They add no value. And unfortunately, they are all too common.  How about we stop carrying them?

> So now _if_ someone wants a longer perfix to go

First off, this is not about defining laws or rules.  This is not about ‘permitted’ or ‘bugs’.

The point is that a single operator can look and see that all of the next hops of a prefix and its more specifics are aligned. That operator can then choose to aggregate. It’s that simple.  Yes, traffic flow may be affected. The operator needs to decide whether or not that is problematic for them.

Tony

---

From: Dino Farinacci <[email-masked]>
Date: Fri, 25 Feb 2022 15:07:05 -0800

>   sufficient shorter prefixes.

We did that in year 2000. It is/was called FIB compression. It is useful in routers where a large number of routes shared the same next-hops. What I should really say, it benefits routers with less interfaces because the set is smaller. On the edge, like a PE router with DS3s to 100s of customers (circa year 2000) FIB compression didn't help that much when the customer would advertise non-compressible prefixes.

But it IS a useful feature at the expense of clever FIB engineers figuring out how to implement it on their respective company's hardware platform.

> arguably have a less than inexorable tie... i think.

You still have to aggregate locators.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Fri, 25 Feb 2022 15:07:29 -0800

> We use all three in the Internet (longest prefix, ARP/LISP, and RIP/OSPF, respectively).

But we haven't used ML. Wonder what people think about that?

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Fri, 25 Feb 2022 15:08:42 -0800

> Yes. My hair is turning gray. AFAICT, this is not written down elsewhere. If not published, the concept of an abstraction action boundary might be lost. Noel, who deserves 100% of the credit for th…

I will vouch for this since the last networking project Noel worked on was LISP with us back in 2012ish timeframe.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Fri, 25 Feb 2022 15:10:47 -0800

> routing tables, and my customers don't need it. I just want do do LISP".

LISP isn't really tunnels (if you look really really closely) but to quote Noel "Tunnels were never first class citizens of the Internet architecture". Take it for what's it worth.

Dino

---

From: Joe Touch <[email-masked]>
Date: Fri, 25 Feb 2022 15:28:17 -0800

> On Feb 25, 2022, at 3:07 PM, Dino Farinacci <[email-masked]> wrote:
> 
> 
>> 

Machine learning?

As in the failed DARPA Intelligent Nets effort?

If so, ARP is learning - just the simplest kind.  Without name structure that maps to network topology, there’s nothing else to learn. 

Or did you mean something else?

Joe

---

From: Dino Farinacci <[email-masked]>
Date: Fri, 25 Feb 2022 15:31:30 -0800

>> But we haven't used ML. Wonder what people think about that?

Yes.

> As in the failed DARPA Intelligent Nets effort?

I don't know.

> If so, ARP is learning - just the simplest kind.  Without name structure that maps to network topology, there’s nothing else to learn. 

Yes, probably better than constand retraining ML models.

> Or did you mean something else?

Use probabability to know which next-hops to use in a FIB entry. Start with a static topology baseline and then prune paths based on probablility of failure (typical links going up and down, nodes not reliable, etc).

This is very high-level. And this is NOT SDN. All predications are done local to a router. Just like my phone and car do it for other types of applications/use-cases.

Dino

---

From: "[email-masked]" <[email-masked]>
Date: Fri, 25 Feb 2022 20:14:43 -0800

Hi, Dino,

Yes, ML can help deal with unpredictable link issues *if* there are some underlying statistics at work. However, it’s generally more useful to track such links as faulty and replace them than to use AI to “adapt” to their failure patterns.

I looked at ML techniques for predicting connection parameters (e.g., AI meets TCB-sharing) years ago, but it turned out to not find anything we didn’t know going in (diurnal patterns, association based on routing prefix, etc.).

Joe

—
Dr. Joe Touch, temporal epistemologist
www.strayalpha.com

> On Feb 25, 2022, at 3:31 PM, Dino Farinacci <[email-masked]> wrote:
> 
> 
>> 
>>

---

From: "[email-masked]" <[email-masked]>
Date: Fri, 25 Feb 2022 20:19:42 -0800

FWIW...

> On Feb 25, 2022, at 3:10 PM, Dino Farinacci <[email-masked]> wrote:
> 
> LISP isn't really tunnels (if you look really really closely)

Agreed; it’s what I presented to Russ White, et al., in 2006 as a “recursive router”. If done correctly (IMO), it makes a network subnet look like a router to the rest of the network.

> but to quote Noel "Tunnels were never first class citizens of the Internet architecture". Take it for what's it worth.

I disagree; a tunnel (done correctly) is isomorphic to a link. There’s no difference between tunnels and what we already rely on as “L2”.

The flaw is the OSI model assuming layer levels are absolute (they’re relative) from all viewpoints (again, relative). There’s a strong equivalence between a link, a tunnel, and a router (which, in essence, emulates shared link). And, interestingly, forwarding can also be described as recursive tunneling.

Joe

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 26 Feb 2022 12:51:08 -0800

> On Feb 25, 2022, at 8:14 PM, [email-masked] wrote:
> 

Thanks for sharing Joe. 

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 26 Feb 2022 12:59:28 -0800

> Agreed; it’s what I presented to Russ White, et al., in 2006 as a “recursive router”. If done correctly (IMO), it makes a network subnet look like a router to the rest of the network.

Right. 

> I disagree; a tunnel (done correctly) is isomorphic to a link. There’s no difference between tunnels and what we already rely on as “L2”.

Yeah it depends how the tunnel is implemented. It can be a dynamic encapsulation like to do for an IP packet sent on Ethernet or it can be a logical p2p or p2mp interface that has addressing and not only included in routing protocols but you run protocols and ACLs/DSCP over them. 

LISP is the former in the various implementations I have been associated with. 

> The flaw is the OSI model assuming layer levels are absolute (they’re relative) from all viewpoints (again, relative). There’s a strong equivalence between a link, a tunnel, and a router (which, in…

Right. That is, it is architecturally broken to run network layer functionality at the application level. 

We have IaaS, PaaS, and I named my open-source LISP implementation as RaaS (or Routing as an Application). 

Dino

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Feb 2022 18:40:19 +0100

Thanks, Tony,

Inline

> Well, here’s the problem: Internet usage is decoupled from all other applications. Data centers, even some of the hyperscalars, do not have the resource requirements of Internet routing. And yes, t…

Haha. I will keep this paragraph for citation when i am again faced
with arguing how solutions for private networks do NOT have to meet
all the same requirements as those for the Internet. And that it is
NOT appropriate to invalidate this statement by starting that traffic
COULD and SHOULD always be able to leak between the two and hence
private network need to inherit all Internet requirements. This is
official IETF policy as can be seen in TSV review and specs like
RFC8084.

> As a vendor, I should be happy about this: it all goes to better margins. However, as an engineer, I’m aghast. Most of this is waste. If we paid more attention to efficiency, we would be in a much…

I don't think this is true (not being sexy). The whole PE/P structuring is
also about efficiency.  On another thread, Jeff Tentsura just yesterday reminded
of that principle and that expensive packet processing is best done offpath and
that hop-by-hop forwarding should be cost/energy optimized. 

Now, when i listen to Geoff Houstons pitches, such as in last years
DINRG meeting (https://www.youtube.com/watch?v=1kbSbVjb1ZU starting at 14 mins), 
it seems to me as if we might be about to see "peak Internet (transit)",
aka: more and more traffic will be for a limited number of edge-DC services
from big OTT, and an even more declining percentage of traffic will be
to the "whereever else" locations in the Internet. 

In this environment, i would, in the same way as you propose unnecessary
traffic by shorter prefixes, go all the way for the "0.0.0.0/0" (default)
prefix in my high-speed edge-network, and then redesign my
"full routing table" Internet edge/transit devices to be ideally cost
optimized for the special Internet requirements: Highly parallelized
BGP on an off-the-shelf Intel/ADM blade-server plus a forwarding plane
with the aforementioned FIB entry optimizations.

Thats at least IMHO the reference model against which your proposal
should measure up in terms of cost/benefit. Because its clear that
your proposal would reduce routes. But how would we measure the cost/benefit ?

> The volumes are ten orders of magnitude lower, so that’s no surprise. And just having routing without direct coupling to a forwarding plane restricts you to off-box applications, which is missing a…

The off-the-shelf blade-server to run routing on can perfectly well
have a 100Gbps ethernet into the forwarding chassis. I jut don't think
we should expect the price/overhead of that control plane instance to
anything more than commodity CPU/memory pricing.

> But while this is interesting, it is also irrelevant to the matter at hand: we can be more efficient in our routing if we want to.

Sure. And we can be more flexible in our non-Internet addressing and a lot
of other innovation we IMHO should do, but there too we are boing faced
with rightful questions about cost/benefit of comparable/other approaches.

> The point is to make routing more efficient, not to talk about FIB compression mechanisms.

I am sure registries/operators would like to see a quantitative costs/benefit
comparison.

> And I’m not trying to force anyone one way or another. I’m simply documenting the tool.

Btw: It might be useful to go to OPSAWG to present and get feedback
on how to set up such aggregated routes at some particular
location in the Internet, because unfortunately to me it is not
quite clear how easy this would be in the face of this highly meshed
peering we have today.

> Only for geographic addressing that is not already covered by per-registry block allocation. Largely, that means allocating blocks to Australia. :)

See above. Practical examples of how/where it would be applied and how
gain it would give in the face of more and more arbitrary
peering would be nice to see.

> That’s irrelevant. We are not concerned with potential downsides. The question is how can we improve efficiency.
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
???

> We do not want to carry longer prefixes when they point to the same next hops.  They add no value. And unfortunately, they are all too common.  How about we stop carrying them?

Probably best done by practical operartional recommendations for operators
how and when to insert those aggregates, and actually what the pro/cons
are (e.g.: how to exclude longer poison/blackhole routes from being aggregated, because
you probably till want/need those to escape).

Do we even have a YANG model to operationally express these procedures ?
I remember CLI configs, but haven't followed YANG ;-(

In any case, OPSAWG would IMHO be a good place to present and get feedback from.
If not even a better place as a home for such work than INTAREA (no strong opinion).

Public shaming portals are also helpfull, e.g.: where routing tables
are examined and worst offenders are named. Now you've got to find someone
brave enough to do this ;-)

There is also pre-established practice i think to already filter prefixes
longer than some max-length if i am not mistaken. So operators will ask
why/where that is not sufficient.

As an operator i also would love to have features that i think do not exist.
Such as alerts (YANG PUSH or the like) whenever i do see longer routes
"from the Internet" for a prefix i did aggregate. And of course there
are different cases to consider.

> The point is that a single operator can look and see that all of the next hops of a prefix and its more specifics are aligned. That operator can then choose to aggregate. It’s that simple.  Yes, tr…

Sure. And a BCP guidelines RFC  (opsarea ?) would go a lot further than an architectural
definition of new terms... (IMHO...).

But again: i still wish there was some qualitative analysis of possible benefits.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Feb 2022 18:46:51 +0100

Hey wait. I didn't even say tunnel at all ;-)

I just said you can unfortunately not claim to be an Internet ISP and
not carry the whole bloody BGP routing table by just using LISP (unfortunately).

Aka: Joe touch pointed out that something like LISP (on-demand routing information
if thats an appropriate classification in our context here) is part of overall
routing architecture (which i wholeheartedly agree), but its alas not a sufficient
option to become an Internet ISP, and so i argued its not part of the "Internet Architecture"
as it is reelevant to this doc.

Cheers
    Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Feb 2022 18:54:10 +0100

> I disagree; a tunnel (done correctly) is isomorphic to a link. There’s no difference between tunnels and what we already rely on as “L2”.

I guess wrt. routing we (Internet Routing Architecture) started out with
alot of simplifying assumptions about the properties of links. Primarily them
being full-mesh-mp2mp or p2p. And then MPLS gave us arbitrary mesh mp2p LSPs,
 and then MANET came with arbitrary partial-connectivity meshes. I don't this
we ever formalized any of this so that we could really confidently use 
link or tunnels and be clear about what options the reader will assume.

> The flaw is the OSI model assuming layer levels are absolute (they’re relative) from all viewpoints (again, relative). There’s a strong equivalence between a link, a tunnel, and a router (which, in…

I thought you just look whats on the wire and you know
the level. If its a frame it's L2, if it's a packet, it's L3. If it's
anything more complicated, it's a tunnel (sorry, couldn't resist ;-))

Is there any IETF RFC that is actually any better than OSI in this respect ?

And yes. I wish there was better material about recursion as the core
fundamental of building networks. Especially because so far we mostly
have re-invented the wheel for every instance of recursion (frame,
packets, tunnel headers..).

Cheers
    Toerless

---

From: "[email-masked]" <[email-masked]>
Date: Mon, 28 Feb 2022 12:49:53 -0800

FWIW:

> On Feb 28, 2022, at 9:46 AM, Toerless Eckert <[email-masked]> wrote:
> 

Agreed. This is a common misconception; LISP and ‘recursive routers’ (which is what I’ve called them for two decades) hide routing from the core by pushing the work to the edge of an enterprise (in an unscalable manner). But they have no impact on how enterprises interconnect.

So, to me at least, they ARE part of the Internet architecture, but don’t replace the need for backbone routing. They just isolate it from pockets under a single enterprise’s control.

Joe

---

From: "[email-masked]" <[email-masked]>
Date: Mon, 28 Feb 2022 12:54:14 -0800

—

> On Feb 28, 2022, at 9:54 AM, Toerless Eckert <[email-masked]> wrote:
> 

RFC3918 tried to formalize some of it; the rest is pending in draft-ietf-intarea-tunnels.

> anything more complicated, it's a tunnel (sorry, couldn't resist ;-))

There is a base case to the recursion, i.e., where logical information meets fermions and bosons (literally). But that tells you only that base layer; it tells you nothing about the meaning of the headers you see inside, e.g., in OSI, they would be 1,2,3,4,5,6,7, but in an IP tunnel, they could be 1,2,3,3,4,5,6,7, with GRE they could be 1,2,3,2,3,4,5,6,7, etc.

(i.e., referring to protocols by the layer they’re commonly associated with).

> Is there any IETF RFC that is actually any better than OSI in this respect ?

I hope the tunnels doc will be ;-)

> packets, tunnel headers..).

Part of the reason that doc is delayed is the “intro to computer networking from first principles” I’m writing as we speak that does exactly that.

Joe

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 28 Feb 2022 19:54:36 -0800

I didn't say you said tunnel. I was stating for clarity, since people think LISP does tunnels (because we have terminlogy that implies it, ex. Ingress Tunnel Router (ITR)). Tunnels, from the original introduction of them were virtual/logical point-point interfaces in a router/switch implementation. We like to think that LISP has "dynamic encapsulation". Its kinda marketing terminlogy, but there is no long-term logical interface state stored per RLOC in the map-cache.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 28 Feb 2022 19:56:28 -0800

But if LISP is running in CPE routers, the enterprise has a say on which paths are used to get to a destination EID. So I would argue the enterprise does have impact.

Dino

> On Feb 28, 2022, at 12:49 PM, [email-masked] wrote:
> 
> FWIW:
>

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 28 Feb 2022 20:00:33 -0800

> There is a base case to the recursion, i.e., where logical information meets fermions and bosons (literally). But that tells you only that base layer; it tells you nothing about the meaning of the…

An IP tunnel with protocol number 4 is indeed 1,2,3,3,4,5,6,7, but LISP (UDP port 4341) UDP tunnels are 1,2,3,4,3,4,5,6,7 and GRE is 1,2,3,3,4,5,6,7 because it runs directly over IP with protocol number 47.

Dino

---

From: "[email-masked]" <[email-masked]>
Date: Mon, 28 Feb 2022 21:29:34 -0800

> On Feb 28, 2022, at 8:00 PM, Dino Farinacci <[email-masked]> wrote:
> 

If GRE runs over IP, then it would be the same as IP-over-UDP tunnels:

1,2,3(IP), 4(GRE, since it is a protocol number of IP), 3(IP in GRE), 5,6,7, i.e.: 1,2,3,4,3,4,5,6,7

It’s all actually relative, though - to the left IP, GRE is layer 4. To the right IP, GRE is layer 2.

Joe

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 28 Feb 2022 22:15:24 -0800

There is no UDP port number assigned for GRE because it does not run over UDP. It runs DIRECTLY over IP. Check the RFC if you don’t believe me. 

Dino

> On Feb 28, 2022, at 9:29 PM, [email-masked] wrote:
> 
>

---

From: Joe Touch <[email-masked]>
Date: Mon, 28 Feb 2022 22:35:46 -0800

In the example I gave, I was equating GRE *to* UDP, not saying it ran over UDP, though it can (port 4754, per RFC 8086).

Joe

> On Feb 28, 2022, at 10:15 PM, Dino Farinacci <[email-masked]> wrote:
>