Subject: [Int-area] Where to aggregate, where to drop

From: Dino Farinacci <[email-masked]>
Date: Fri, 1 Apr 2022 14:06:00 -0700

Tony, sorry for the delay on this. Thanks for your patience and for poking me about it.

For the list, I had a question about Tony bringing up *where* to do proxy aggregation and what the tradeoffs are. I asked a question about having a discussion about it. Lets use his slide below for reference to list the tradeoffs.

When a provider proxy aggregates, it means they will summarized more specific routes they have stored in their routing table. Like ISP-A above has routes P.1, P.2, and P.3. When ISP-A advertises a P prefix, it is indicating it can reach all more specifics, even though it may not have a full-set of those more specifics that are covered by P.

So here are the options:

(1) ISP-A advertises P to ISP-B (and may also advertise more specifics to other peers for policy reasons).
(2) ISP-A advertises P.1, P.2, and P.3 to ISP-B and ISP-B advertises P to its peers.

The questions is *where is the best place to aggregate*. Its a tradeoff on routing table savings and how far a packet can travel to either (1) get delivered to the destination or (2) get dropped by a router that doesn't have a more-specific for the destination (thereby wasting resources from the source to the drop point).

(1) If ISP-B aggregates P to the links you see above to stub peers, then the stubs can load-split on P and ISP-B can drop packets for say P.4 and deliver packets for P.3. The drop can happen at the PE router relatively soon.

(2) If ISP-B gets P from ISP-A and advertises to the links you see above to stub peers, then P.3 and P.4 packets move through ISP-B where the edge router in ISP-A delivers to P.3 and drops for P.4.

All things not considered and looking at this specific topology, I would vote for (1).

Another issue, which I think Tony brought up is, if P gets sent to peers, hijackers have a better opportunity to hijack routes by injecting more-specifics to direct traffic to a bad acting honey pot. This has been known for a long time though so we are not bringing up a new issue.

Another question is, if all more-specfics are not stored (due to major link failure), is the aggregate withdrawn from the routing system. That is, if you want less route flapping, you may just want to keep P advertised. That optimizes FIB add/delete entropy everywhere that wants to store P. I would rather have hardware routers drop packets fast, then to have route oscillation.

Comments?

Thanks,
Dino

---

From: Tony Li <[email-masked]>
Date: Sat, 2 Apr 2022 08:29:45 -0700

Dino,

Thanks for the question.

> When a provider proxy aggregates, it means they will summarized more specific routes they have stored in their routing table. Like ISP-A above has routes P.1, P.2, and P.3. When ISP-A advertises a…

In the figure, ISP-A is the administrator for P, so when they aggregate it’s not proxy aggregation, it’s just simple aggregation.

> (1) If ISP-B aggregates P to the links you see above to stub peers, then the stubs can load-split on P and ISP-B can drop packets for say P.4 and deliver packets for P.3. The drop can happen at the…

If they are stub peers, then sending them anything more than default is inefficient. Yes, ISP B will drop anything for P.4.  There’s no way to deliver that any way.

If the peers are not stubs and ISP B advertises P, then yes, it will attract P.4 traffic.  Presumably, the peers are not learning alternate paths for this traffic, so there’s also no way of delivering this traffic and ISP B would get to drop it.  

I’m not sure why we’re trying to optimize drop traffic. Hopefully, there isn’t a large volume of it. :)

> (2) If ISP-B gets P from ISP-A and advertises to the links you see above to stub peers, then P.3 and P.4 packets move through ISP-B where the edge router in ISP-A delivers to P.3 and drops for P.4.

And in that case, ISP B has now paid for transit for P.4 drop traffic. That seems painful.

As you point out, there’s a trade-off here: if you move the abstraction action boundary outward, you carry more prefixes but you drop unreachable traffic sooner.  If you move the abstraction action boundary inwards, then you carry fewer prefixes, possibly damage traffic engineering, and cause unreachable traffic to take a longer path before it’s dropped.

> All things not considered and looking at this specific topology, I would vote for (1).

Of course, but what we’ve found is that prefix owners are not willing to just aggregate. They want traffic engineering so they also advertise more specifics.  Thus, the question is (2) or (3) let more specifics propagate throughout the network.

> Another issue, which I think Tony brought up is, if P gets sent to peers, hijackers have a better opportunity to hijack routes by injecting more-specifics to direct traffic to a bad acting honey po…

Actually, the point that I was bringing up was not one of security.  If more specifics are also present when ISP B performs remote aggregation, then the more specifics will tend to draw traffic away from ISP-B.  ISP B might like that.

> Another question is, if all more-specfics are not stored (due to major link failure), is the aggregate withdrawn from the routing system. That is, if you want less route flapping, you may just want…

Yes, if ISP B isn’t getting all of the more specifics and continues to aggregate, it could attract traffic that it can’t deliver. Presumably this is a transient until it can get the more specifics.

Tony

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 2 Apr 2022 15:59:34 -0700

> In the figure, ISP-A is the administrator for P, so when they aggregate it’s not proxy aggregation, it’s just simple aggregation.

Right, but doesn't matter what you call it, the point is where to aggregate.

> If they are stub peers, then sending them anything more than default is inefficient. Yes, ISP B will drop anything for P.4.  There’s no way to deliver that any way.

Well the stub AS could be multi-homed and want to load-spread traffic for different prefixes. The a set of advertised prefixes are exceptions for not following the default path.

> I’m not sure why we’re trying to optimize drop traffic. Hopefully, there isn’t a large volume of it. :)

If you can, you want to drop traffic early as possible so you optimize network resources.

> And in that case, ISP B has now paid for transit for P.4 drop traffic. That seems painful.

Right.

> As you point out, there’s a trade-off here: if you move the abstraction action boundary outward, you carry more prefixes but you drop unreachable traffic sooner.  If you move the abstraction action…

Yep, well said.

> Of course, but what we’ve found is that prefix owners are not willing to just aggregate. They want traffic engineering so they also advertise more specifics.  Thus, the question is (2) or (3) let m…

Right, but you are trying to fix this, right? You are trying to do better.

> Actually, the point that I was bringing up was not one of security.  If more specifics are also present when ISP B performs remote aggregation, then the more specifics will tend to draw traffic awa…

Yes.

> Yes, if ISP B isn’t getting all of the more specifics and continues to aggregate, it could attract traffic that it can’t deliver. Presumably this is a transient until it can get the more specifics.

So what is your conclusion? Is your draft saying to optimize traffic engineering or saving routing table space? Or simply documenting the tradeoffs?

Cheers,
Dino

---

From: Tony Li <[email-masked]>
Date: Tue, 5 Apr 2022 16:12:49 -0700

Hi Dino,

> Well the stub AS could be multi-homed and want to load-spread traffic for different prefixes. The a set of advertised prefixes are exceptions for not following the default path.

??? If the AS is multi-homed, then it’s not a stub.

Yes, if this is a third AS (ISP-C) that has other connectivity, then it would see longer prefixes as well as the aggregate.  Traffic would be diverted to the longer prefixes and away from ISP-B, as noted.

> If you can, you want to drop traffic early as possible so you optimize network resources.

As always, it’s a trade-off.  In this case, do you want to optimize your routing resources or a small amount of bandwidth.  You can either carry more specifics or drop P.4 traffic.  IMHO, that’s an easy call.

> Right, but you are trying to fix this, right? You are trying to do better.

That is indeed the point.  If aggregation happens at the right place, you get traffic engineering close to the destination and you get abstraction farther away. IMHO, that’s ideal.

> So what is your conclusion? Is your draft saying to optimize traffic engineering or saving routing table space? Or simply documenting the tradeoffs?

I’m saying do both: aggregate at the right place and it’s a win-win.  But figuring out where to aggregate does require some thought.

Tony

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 5 Apr 2022 16:40:17 -0700

> ??? If the AS is multi-homed, then it’s not a stub.

It's not transit if it's a multi-homed stub. Like most multi-national enterprise networks. That's what I mean.

> Yes, if this is a third AS (ISP-C) that has other connectivity, then it would see longer prefixes as well as the aggregate.  Traffic would be diverted to the longer prefixes and away from ISP-B, as…

That's what I meant by exceptions.

> As always, it’s a trade-off.  In this case, do you want to optimize your routing resources or a small amount of bandwidth.  You can either carry more specifics or drop P.4 traffic.  IMHO, that’s an…

Not at all clear that its a small amount of bandwidth.

> That is indeed the point.  If aggregation happens at the right place, you get traffic engineering close to the destination and you get abstraction farther away. IMHO, that’s ideal.

That is, if where traffic engineering is required or desired. 

> I’m saying do both: aggregate at the right place and it’s a win-win.  But figuring out where to aggregate does require some thought.

Definitely agree.

Dino

---

From: Tony Li <[email-masked]>
Date: Tue, 5 Apr 2022 16:47:35 -0700

Hi Dino,

> Not at all clear that its a small amount of bandwidth.

Other than a DoS attack, why would it be significant?

>> I’m saying do both: aggregate at the right place and it’s a win-win.  But figuring out where to aggregate does require some thought.

So the draft is supposed to help people think about it. :)

Do you think it’s worth of WG adoption?

T

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 5 Apr 2022 17:14:50 -0700

> On Apr 5, 2022, at 4:47 PM, Tony Li <[email-masked]> wrote:
> 

Good point. It would only be for traffic to more-specifics that didn’t exist. 

> So the draft is supposed to help people think about it. :)

That’s what it did for me. 

> Do you think it’s worth of WG adoption?

I think so yes. 

Dino

---

From: Tony Li <[email-masked]>
Date: Wed, 6 Apr 2022 08:25:03 -0700

> On Apr 5, 2022, at 5:14 PM, Dino Farinacci <[email-masked]> wrote:
> 
>> 

Thank you Dino.

Do others concur? The chairs want to see some indication of interest.

T