Subject: Re: [tsvwg] L4S WGLC

From: Stuart Cheshire <[email-masked]>
Date: Mon, 25 Oct 2021 21:25:53 -0700

It has taken me a while to write this because I have been struggling to work out a viable path forward.

# Executive Summary (TL;DR)

I fully support publishing the L4S Experimental RFCs. I think the L4S approach has a lot of merit.

In parallel, I’d like us to devote energy into working out what we need to do to get L4S deployed.

Over the next 12 months, what do we want servers to do? What do we want clients to do? what do we want network routers to do?

# Introduction

I believe strongly that if the Internet community fails to solve the bufferbloat problem and deliver consistent, reliably low-delay communications, that failure is going to limit continued development of Internet applications.

As network connection throughput rates increase this problem becomes ever more urgent. We are already reaching a point of diminishing returns, where adding yet more throughput without doing anything to reduce delay yields little noticeable improvement to the end user.

In the most extreme case, something better could come along and relegate the Internet to the role of second-class service. That’s not really a short-term concern though, because it’s unlikely right now that anyone else understands bufferbloat better than we do. But eventually, someone might figure it out. We know there are lots of groups vying to build the Internet’s successor.

# Proposal

How do we fix this before it’s too late?

I fully support publishing the L4S Experimental RFCs. (At the end of this email I have attached more detail about how the reasoning that lead me to support L4S rather than SCE.)

In parallel with publishing the L4S Experimental RFCs, I’d like us to devote energy into working out what we need to do to get L4S deployed. Let’s not wait until the L4S experiment is over and only then start thinking about how to deploy it. Let’s start the preparation now, so that if the L4S experiment is a success we’ve already started doing the groundwork to enable us to turn it on.

Over the next 12 months, what do we want clients to do? What do we want servers to do? what do we want network routers to do?

# Background

I believe we should abandon Classic ECN. Classic ECN has already had twenty years to catch on, and it hasn’t. We could wait another twenty years and still see no meaningful change.

In 2015 Apple made a big push to promote Classic ECN, enabling it on the client side on all Apple devices. Even back then, most Linux servers would negotiate ECN if requested. With client and server code now in place, we thought that network operators would finally have a reason to deploy ECN marking, because they could achieve immediate visible benefits, in the form of more responsive network applications.

<https://developer.apple.com/videos/play/wwdc2015/719/?time=1170>

We were wrong. It never happened. Even with all the client and server work in place and ready to be used, the benefit of Classic ECN was not compelling enough for network operators to care.

Another lesson we learned is to beware of heuristics.

We had heuristics to detect ECN bleaching and similar network problems, and then disable ECN when those problems were detected.

These compatibility heuristics worked too well. They were so good at avoiding network problems with ECN, that network operators with bad networks that broke ECN never noticed a problem, so they never fixed their problems and ECN stayed broken on that network.

Also, in an extreme effort to avoid causing even the slightest problem that could make news and incur the disapproval Apple management, our heuristics were very conservative. If in doubt we would choose in the “safe” option. As a result our heuristics would often trigger unnecessarily and disable ECN incorrectly. This meant that sometimes, even on networks where ECN should work, the heuristics would disable ECN and the user wouldn’t see any benefit. This meant that even if network operators did the work to deploy ECN, the benefit was inconsistent and uncertain. ECN would not activate reliably.

In summary, the compatibility heuristics meant that network operators with bad networks had no incentive to fix their networks, and network operators with a desire to provide better customer experience had no incentive to enable ECN in production, because they found they couldn’t depend on it to actually work all the time.

To recap: Beware of compatibility heuristics. Compatibility heuristics can both (a) mask brokenness in the network sufficiently that the network never gets fixed, and (b) can dilute the benefits of a new advancement sufficiently that the new advancement never gets deployed.

# Call to Action

Returning to my questions: What do we want clients to do? What do we want servers to do? what do we want network routers to do? I have some ideas. Certainly I will have missed some things, and I may have some mistakes. Feedback is welcomed.

## Clients

For TCP, I think we want clients to negotiate ECN with Accurate ECN. If the other end supports Accurate ECN the client should mark its outbound packets ECT(1) and control its sending using a scalable congestion controller. If the other end supports ECN without Accurate ECN, the client should mark its outbound packets ECT(0) and control its sending using a classic congestion controller. If the other end does not support ECN at all, the client should mark its outbound packets Non-ECT and control its sending using a classic loss-based congestion controller.

QUIC already defines Accurate ECN, and the QUIC specification [RFC 9000] describes how endpoints test to see if their peer supports ECN. The same guidance as for TCP applies: If the QUIC peer supports ECN (with Accurate ECN), the client should mark its outbound packets ECT(1) and control its sending using a scalable congestion controller. If not, the client should mark its outbound packets Non-ECT and control its sending using a classic loss-based congestion controller.

For other transport protocols running over a datagram layer (IP or UDP), the transport protocol should be upgraded to report Accurate ECN, mark its outbound packets ECT(1), and control its sending using a scalable congestion controller. For transport protocols not yet capable of Accurate ECN reporting and/or scalable congestion control, they should continue to use ECT(0) (or Non-ECT) and a classic congestion controller in the short term until they are upgraded.

## Servers

For TCP, servers should implement Accurate ECN and use it when negotiated by the client. When Accurate ECN is in use, servers should mark outbound outbound packets ECT(1) and control sending using a scalable congestion controller. If the client negotiates ECN without Accurate ECN, the server should mark its outbound packets ECT(0) and control its sending using a classic congestion controller. If the client does not negotiate ECN at all, the server should mark its outbound packets Non-ECT and control its sending using a classic loss-based congestion controller.

For QUIC, servers should implement Accurate ECN for QUIC, and use it when negotiated by the client. As for QUIC clients, if the QUIC peer supports ECN (with Accurate ECN), the server should mark its outbound packets ECT(1) and control its sending using a scalable congestion controller. If not, the server should mark its outbound packets Non-ECT and control its sending using a classic loss-based congestion controller.

For other transport protocols, the same guidance applies as given for clients, above.

## Routers

Today, most routers use only tail-drop FIFO. Many support a smarter AQM like CoDel, but few have it enabled by default. Even fewer routers currently available support L4S-style early congestion signalling.

Effective as soon as possible, we want all routers to be improved, though the ones that matter most in the short term are the ones at bottleneck queues.

In the long term, L4S-style early congestion signalling everywhere would be the best, either with FQ (per-flow queueing) or L4S Dual Queue. Where per-flow queueing is feasible to implement, FQ provides nice properties, like more predictable fairness between flows, and lower latency for sparse flows. Where per-flow queueing is not feasible to implement, or not desired for other reasons, L4S Dual Queue is an attractive alternative.

That may take some time though, and some people may still not be convinced that they support this direction.

In the short term, we’d like current CoDel and FQ_CoDel queues modified to treat ECT(1) differently. They should continue to interpret ECT(0) as they do today, giving cautious sparse congestion marks, only after a persistent queue has formed and remained for some time. But when these AQMs see packets marked ECT(1) they should be more generous with their congestion marks, giving congestion signals early and often. This means both giving congestion marks earlier (when the queue is shorter and/or has persisted for less time) and being more liberal with their marking, for as long as the queue remains above the threshold. They can do this safely for two reasons: 1. ECN marking is less destructive than packet discard, so it can be used more liberally. 2. Knowing that the receiver will not overreact to congestion marks means that they should be used more frequently, to give more fine-grained feedback for the sender to adjust its rate.

A patch has already been created by Eric Dumazet to do something along these lines. I don’t know anything about it, so I cannot comment directly. It may need some polish and tuning, but it seems like a step in the right direction: <https://lore.kernel.org/netdev/[email-masked]/>

Stuart Cheshire

--

Below is my reasoning for supporting (1) early notification of mild congestion and (2) L4S as the mechanism for doing that.

1. To keep queueing delay consistently low we need to keep network queues consistently short. We don’t want them wildly oscillating from empty to enormous.

1a. That means senders need to make smaller rate adjustments, not the big CWND adjustments made by Reno and CUBIC today.

1b. That means routers need to signal congestion earlier and more frequently, so that senders can make frequent small rate corrections, instead of occasional huge rate corrections. Signalling congestion earlier, while it is still mild, depends on knowing that the sender will not overreact to that early congestion signal.

1c. Flow queueing, flow isolation, separate queues for each QoS class, etc., are not, by themselves, a complete solution. Even if we had completely perfect flow separation, we still want a single flow, in a queue by itself, to be able to be capacity-seeking while still getting consistently low delay at the same time. A flow should be able to be capacity-seeking without suffering accidental self-induced queueing delays resulting from bufferbloat. This means that even if we could have perfect FQ, that alone is not a substitute for solving bufferbloat. In order to provide good agility and responsiveness of network applications, we still need early notification of mild congestion, to enable a capacity-seeking flow to achieve low delay. This concern also applies when partially independent flows may share single connection, such as when fetching multiple web page resources over H2, QUIC, or H3, where the resources are somewhat related but still at least partially independent. Large (capacity-seeking) objects and small (delay-sensitive) objects may be transferred concurrently over a single logical transport-layer connection, so we need that connection to be both capacity-seeking and low-delay at the same time.

Item 1 should not be controversial since both SCE and L4S agree on this point. SCE proposes two different congestion signals: a mild congestion signal and an extreme congestion signal. Endpoints that are upgraded to understand the mild congestion signal can recognize that signal and react accordingly to keep the queue short and stable; older endpoints that do not recognize the mild congestion signal will ignore that signal and instead process the extreme congestion signal when it arrives, reacting with an extreme CWND reduction. L4S proposes that, rather than implementing two different parallel congestion signal strategies for every packet, endpoints instead should mark their packets to indicate whether they want early notification of mild congestion, or late notification of extreme congestion. Queues in the network respect this marking and signal congestion for that packet one way or the other, as requested by the sender.

2. We need to assume that different traffic flows will sometimes share a common queue. Some network gear likely will never support FQ, due to hardware architectures, performance requirements, expected flow counts, concerns about fairness to VPNs, inability to examine L4 headers, or other reasons. Even when the network gear does support FQ, the flow hashing may sometimes be imperfect. So, we need a solution that can be implemented without depending on perfect FQ everywhere.

2a. If different logical flows may share a common queue, that implies they must cooperate to keep the shared queue short.

2b. A requirement for cooperation implies, unfortunately, a requirement for policing, to make deliberate abuse ineffective.

2c. Policing is not the same as perfect FQ. Policing is a lightweight statistical sampling process that monitors a small subset of traffic. Just as with road traffic policing, which will not catch every single road traffic violation 100% of the time (or anything even remotely close to that), the penalty when a violation is detected has to be sufficient disincentive to discourage the violation in the first place. To be a sufficient disincentive, the expected penalty for a detected violation has to significantly exceed the expected gain achieved by an undetected violation. Similarly, policing does not have to be performed on every router in the network. Policing simply has to be performed by enough routers, enough of the time, to be a deterrent to bad behavior by transport protocols (TCP, SCTP, QUIC, etc.), and particularly, a deterrent to bad behavior by the home-grown, often simplistic, transport protocols embedded within applications that run over UDP (and often written by people who don’t even realize that if they’re running over UDP datagrams, and not using a mature protocol like QUIC, that means they’re designing their own new transport protocol from scratch).

2d. Only senders that have agreed to follow our new rules in order to get high throughput and consistent low delay should be subject to the associated policing. It would be unreasonable to severely penalize existing traffic for failing to follow our new rules. That implies the need for an input signal (a “queue selector” in L4S terms) to let traffic indicate what treatment the sender wants.

Item 2 tilts the argument decisively in favor of L4S over SCE. If we have to assume that different logical flows will sometimes land in the same queue, it is important that all those flows sharing a queue follow the same rules. Otherwise a single flow that is ignoring the mild congestion signals will fill up the queue to the extreme congestion signal level, thereby negating the efforts of all the other flows cooperating to keep the shared queue short and stable. And this deviant flow that is ignoring the SCE marks is not breaking any Internet rules -- it is simply from a legacy device that has not been updated to understand SCE, and may never be updated. Therefore, following principle (2d) there are no valid grounds to “punish” this flow for being uncooperative, because it never promised to be cooperative in the first place. If we could safely assume perfect semantic flow separation everywhere in the network that a queue may form, then SCE might be sufficient, but that that assumption is impossible, because of imperfect hashing, encrypted VPN tunnels, and other ways that independent traffic might end up being classified as the same flow.

This leads me to two conclusions:

1. We want early notification of mild congestion.
2. We need the queue-selector mechanism of L4S to make that work when flows may share a queue.

If we could go back in time and design the Internet from scratch we could take these requirements into account. But we don’t have that luxury. We have to transition from the Internet we have today to the low-delay Internet we want in the future. I fear that means we might have to be willing to accept some minor short-term pain in exchange for major long-term improvement. Doing anything at the IETF that entails even the slightest downside is always very controversial. The guiding doctrine is “do no harm”. But we have to remember that inaction also causes harm. Many years of stagnation results in a gradual decline into irrelevance. History is full of companies that had an existing profitable business, and didn’t have the courage to risk disrupting that business, so instead they were just overtaken by competitors and slowly went bankrupt. Let us not ignore the danger of inaction.

If we had five code points to use instead of four, then we could keep Classic ECN around as a “backwards compatibility” mechanism for some time, and that might make for an easier transition. But we don’t have that luxury, and Classic ECN marking is still extraordinarily rare, so we don’t even really have much of a backwards compatibility problem in the first place.

Of course it is important to “do no harm”, but at the same time, let us not cling to that principle so fervently that it causes us to do no good either.

Hence my conclusion is that we need to forget about classic “late marking” ECN, and move as swiftly as we can to an Internet using entirely L4S-style “early marking” ECN, before someone else does it first.

--

---

From: Ian Swett <[email-masked]>
Date: Tue, 26 Oct 2021 05:55:33 -0400

I strongly support abandoning classic ECN now as well.  I hope that we can
reclaim the codepoint someday, but I'm not sure when that will be.  Maybe
it'll be ok for limited(ie: only some ASNs) experimental use in the
meantime.

I'd also be interested to see experiences with L4S in QUIC.  We haven't
added the ECN responses to QUIC's BBRv2 yet, but they're available for TCP,
so if there are peers negotiating L4S and middleboxes are marking, then I'd
be happy to run some experiments.  It sounds like a good number of Apple
clients could start negotiating L4S soonish, in which case we could
negotiate it in some fraction of Google Server connections to gather some
data.

Ian

On Tue, Oct 26, 2021 at 12:26 AM Stuart Cheshire <cheshire=
[email-masked]> wrote:

> Over the next 12 months, what do we want servers to do? What do we want
> clients to do? what do we want network routers to do?
>
> # Introduction
>
> I believe strongly that if the Internet community fails to solve the
> bufferbloat problem and deliver consistent, reliably low-delay
> communications, that failure is going to limit continued development of
> Internet applications.
>
> As network connection throughput rates increase this problem becomes ever
> more urgent. We are already reaching a point of diminishing returns, where
> adding yet more throughput without doing anything to reduce delay yields
> little noticeable improvement to the end user.
>
> In the most extreme case, something better could come along and relegate
> the Internet to the role of second-class service. That’s not really a
> short-term concern though, because it’s unlikely right now that anyone else
> understands bufferbloat better than we do. But eventually, someone might
> figure it out. We know there are lots of groups vying to build the
> Internet’s successor.
>
> # Proposal
>
> I fully support publishing the L4S Experimental RFCs. (At the end of this
> email I have attached more detail about how the reasoning that lead me to
> support L4S rather than SCE.)
>
> In parallel with publishing the L4S Experimental RFCs, I’d like us to
> devote energy into working out what we need to do to get L4S deployed.
> Let’s not wait until the L4S experiment is over and only then start
> thinking about how to deploy it. Let’s start the preparation now, so that
> if the L4S experiment is a success we’ve already started doing the
> groundwork to enable us to turn it on.
>
> Over the next 12 months, what do we want clients to do? What do we want
> servers to do? what do we want network routers to do?
>
> # Background
>
> I believe we should abandon Classic ECN. Classic ECN has already had
> twenty years to catch on, and it hasn’t. We could wait another twenty years
> and still see no meaningful change.
>
> In 2015 Apple made a big push to promote Classic ECN, enabling it on the
> client side on all Apple devices. Even back then, most Linux servers would
> negotiate ECN if requested. With client and server code now in place, we
> thought that network operators would finally have a reason to deploy ECN
> marking, because they could achieve immediate visible benefits, in the form
> of more responsive network applications.
>
> We were wrong. It never happened. Even with all the client and server work
> in place and ready to be used, the benefit of Classic ECN was not
> compelling enough for network operators to care.
>
> We had heuristics to detect ECN bleaching and similar network problems,
> and then disable ECN when those problems were detected.
>
> These compatibility heuristics worked too well. They were so good at
> avoiding network problems with ECN, that network operators with bad
> networks that broke ECN never noticed a problem, so they never fixed their
> problems and ECN stayed broken on that network.
>
> Also, in an extreme effort to avoid causing even the slightest problem
> that could make news and incur the disapproval Apple management, our
> heuristics were very conservative. If in doubt we would choose in the
> “safe” option. As a result our heuristics would often trigger unnecessarily
> and disable ECN incorrectly. This meant that sometimes, even on networks
> where ECN should work, the heuristics would disable ECN and the user
> wouldn’t see any benefit. This meant that even if network operators did the
> work to deploy ECN, the benefit was inconsistent and uncertain. ECN would
> not activate reliably.
>
> In summary, the compatibility heuristics meant that network operators with
> bad networks had no incentive to fix their networks, and network operators
> with a desire to provide better customer experience had no incentive to
> enable ECN in production, because they found they couldn’t depend on it to
> actually work all the time.
>
> To recap: Beware of compatibility heuristics. Compatibility heuristics can
> both (a) mask brokenness in the network sufficiently that the network never
> gets fixed, and (b) can dilute the benefits of a new advancement
> sufficiently that the new advancement never gets deployed.
>
> # Call to Action
>
> Returning to my questions: What do we want clients to do? What do we want
> servers to do? what do we want network routers to do? I have some ideas.
> Certainly I will have missed some things, and I may have some mistakes.
> Feedback is welcomed.
>
> ## Clients
>
> For TCP, I think we want clients to negotiate ECN with Accurate ECN. If
> the other end supports Accurate ECN the client should mark its outbound
> packets ECT(1) and control its sending using a scalable congestion
> controller. If the other end supports ECN without Accurate ECN, the client
> should mark its outbound packets ECT(0) and control its sending using a
> classic congestion controller. If the other end does not support ECN at
> all, the client should mark its outbound packets Non-ECT and control its
> sending using a classic loss-based congestion controller.
>
> QUIC already defines Accurate ECN, and the QUIC specification [RFC 9000]
> describes how endpoints test to see if their peer supports ECN. The same
> guidance as for TCP applies: If the QUIC peer supports ECN (with Accurate
> ECN), the client should mark its outbound packets ECT(1) and control its
> sending using a scalable congestion controller. If not, the client should
> mark its outbound packets Non-ECT and control its sending using a classic
> loss-based congestion controller.
>
> For other transport protocols running over a datagram layer (IP or UDP),
> the transport protocol should be upgraded to report Accurate ECN, mark its
> outbound packets ECT(1), and control its sending using a scalable
> congestion controller. For transport protocols not yet capable of Accurate
> ECN reporting and/or scalable congestion control, they should continue to
> use ECT(0) (or Non-ECT) and a classic congestion controller in the short
> term until they are upgraded.
>
> ## Servers
>
> For TCP, servers should implement Accurate ECN and use it when negotiated
> by the client. When Accurate ECN is in use, servers should mark outbound
> outbound packets ECT(1) and control sending using a scalable congestion
> controller. If the client negotiates ECN without Accurate ECN, the server
> should mark its outbound packets ECT(0) and control its sending using a
> classic congestion controller. If the client does not negotiate ECN at all,
> the server should mark its outbound packets Non-ECT and control its sending
> using a classic loss-based congestion controller.
>
> For QUIC, servers should implement Accurate ECN for QUIC, and use it when
> negotiated by the client. As for QUIC clients, if the QUIC peer supports
> ECN (with Accurate ECN), the server should mark its outbound packets ECT(1)
> and control its sending using a scalable congestion controller. If not, the
> server should mark its outbound packets Non-ECT and control its sending
> using a classic loss-based congestion controller.
>
> Today, most routers use only tail-drop FIFO. Many support a smarter AQM
> like CoDel, but few have it enabled by default. Even fewer routers
> currently available support L4S-style early congestion signalling.
>
> Effective as soon as possible, we want all routers to be improved, though
> the ones that matter most in the short term are the ones at bottleneck
> queues.
>
> In the long term, L4S-style early congestion signalling everywhere would
> be the best, either with FQ (per-flow queueing) or L4S Dual Queue. Where
> per-flow queueing is feasible to implement, FQ provides nice properties,
> like more predictable fairness between flows, and lower latency for sparse
> flows. Where per-flow queueing is not feasible to implement, or not desired
> for other reasons, L4S Dual Queue is an attractive alternative.
>
> In the short term, we’d like current CoDel and FQ_CoDel queues modified to
> treat ECT(1) differently. They should continue to interpret ECT(0) as they
> do today, giving cautious sparse congestion marks, only after a persistent
> queue has formed and remained for some time. But when these AQMs see
> packets marked ECT(1) they should be more generous with their congestion
> marks, giving congestion signals early and often. This means both giving
> congestion marks earlier (when the queue is shorter and/or has persisted
> for less time) and being more liberal with their marking, for as long as
> the queue remains above the threshold. They can do this safely for two
> reasons: 1. ECN marking is less destructive than packet discard, so it can
> be used more liberally. 2. Knowing that the receiver will not overreact to
> congestion marks means that they should be used more frequently, to give
> more fine-grained feedback for the sender to adjust its rate.
>
> A patch has already been created by Eric Dumazet to do something along
> these lines. I don’t know anything about it, so I cannot comment directly.
> It may need some polish and tuning, but it seems like a step in the right
> direction: <
> https://lore.kernel.org/netdev/[email-masked]/
> >
>
> Stuart Cheshire
>
> --
>
> Below is my reasoning for supporting (1) early notification of mild
> congestion and (2) L4S as the mechanism for doing that.
>
> 1. To keep queueing delay consistently low we need to keep network queues
> consistently short. We don’t want them wildly oscillating from empty to
> enormous.
>
> 1a. That means senders need to make smaller rate adjustments, not the big
> CWND adjustments made by Reno and CUBIC today.
>
> 1b. That means routers need to signal congestion earlier and more
> frequently, so that senders can make frequent small rate corrections,
> instead of occasional huge rate corrections. Signalling congestion earlier,
> while it is still mild, depends on knowing that the sender will not
> overreact to that early congestion signal.
>
> 1c. Flow queueing, flow isolation, separate queues for each QoS class,
> etc., are not, by themselves, a complete solution. Even if we had
> completely perfect flow separation, we still want a single flow, in a queue
> by itself, to be able to be capacity-seeking while still getting
> consistently low delay at the same time. A flow should be able to be
> capacity-seeking without suffering accidental self-induced queueing delays
> resulting from bufferbloat. This means that even if we could have perfect
> FQ, that alone is not a substitute for solving bufferbloat. In order to
> provide good agility and responsiveness of network applications, we still
> need early notification of mild congestion, to enable a capacity-seeking
> flow to achieve low delay. This concern also applies when partially
> independent flows may share single connection, such as when fetching
> multiple web page resources over H2, QUIC, or H3, where the resources are
> somewhat related but still at least partially independent. Large
> (capacity-seeking) objects and small (delay-sensitive) objects may be
> transferred concurrently over a single logical transport-layer connection,
> so we need that connection to be both capacity-seeking and low-delay at the
> same time.
>
> Item 1 should not be controversial since both SCE and L4S agree on this
> point. SCE proposes two different congestion signals: a mild congestion
> signal and an extreme congestion signal. Endpoints that are upgraded to
> understand the mild congestion signal can recognize that signal and react
> accordingly to keep the queue short and stable; older endpoints that do not
> recognize the mild congestion signal will ignore that signal and instead
> process the extreme congestion signal when it arrives, reacting with an
> extreme CWND reduction. L4S proposes that, rather than implementing two
> different parallel congestion signal strategies for every packet, endpoints
> instead should mark their packets to indicate whether they want early
> notification of mild congestion, or late notification of extreme
> congestion. Queues in the network respect this marking and signal
> congestion for that packet one way or the other, as requested by the sender.
>
> 2. We need to assume that different traffic flows will sometimes share a
> common queue. Some network gear likely will never support FQ, due to
> hardware architectures, performance requirements, expected flow counts,
> concerns about fairness to VPNs, inability to examine L4 headers, or other
> reasons. Even when the network gear does support FQ, the flow hashing may
> sometimes be imperfect. So, we need a solution that can be implemented
> without depending on perfect FQ everywhere.
>
> 2a. If different logical flows may share a common queue, that implies they
> must cooperate to keep the shared queue short.
>
> 2b. A requirement for cooperation implies, unfortunately, a requirement
> for policing, to make deliberate abuse ineffective.
>
> 2c. Policing is not the same as perfect FQ. Policing is a lightweight
> statistical sampling process that monitors a small subset of traffic. Just
> as with road traffic policing, which will not catch every single road
> traffic violation 100% of the time (or anything even remotely close to
> that), the penalty when a violation is detected has to be sufficient
> disincentive to discourage the violation in the first place. To be a
> sufficient disincentive, the expected penalty for a detected violation has
> to significantly exceed the expected gain achieved by an undetected
> violation. Similarly, policing does not have to be performed on every
> router in the network. Policing simply has to be performed by enough
> routers, enough of the time, to be a deterrent to bad behavior by transport
> protocols (TCP, SCTP, QUIC, etc.), and particularly, a deterrent to bad
> behavior by the home-grown, often simplistic, transport protocols embedded
> within applications that run over UDP (and often written by people who
> don’t even realize that if they’re running over UDP datagrams, and not
> using a mature protocol like QUIC, that means they’re designing their own
> new transport protocol from scratch).
>
> 2d. Only senders that have agreed to follow our new rules in order to get
> high throughput and consistent low delay should be subject to the
> associated policing. It would be unreasonable to severely penalize existing
> traffic for failing to follow our new rules. That implies the need for an
> input signal (a “queue selector” in L4S terms) to let traffic indicate what
> treatment the sender wants.
>
> Item 2 tilts the argument decisively in favor of L4S over SCE. If we have
> to assume that different logical flows will sometimes land in the same
> queue, it is important that all those flows sharing a queue follow the same
> rules. Otherwise a single flow that is ignoring the mild congestion signals
> will fill up the queue to the extreme congestion signal level, thereby
> negating the efforts of all the other flows cooperating to keep the shared
> queue short and stable. And this deviant flow that is ignoring the SCE
> marks is not breaking any Internet rules -- it is simply from a legacy
> device that has not been updated to understand SCE, and may never be
> updated. Therefore, following principle (2d) there are no valid grounds to
> “punish” this flow for being uncooperative, because it never promised to be
> cooperative in the first place. If we could safely assume perfect semantic
> flow separation everywhere in the network that a queue may form, then SCE
> might be sufficient, but that that assumption is impossible, because of
> imperfect hashing, encrypted VPN tunnels, and other ways that independent
> traffic might end up being classified as the same flow.
>
> If we could go back in time and design the Internet from scratch we could
> take these requirements into account. But we don’t have that luxury. We
> have to transition from the Internet we have today to the low-delay
> Internet we want in the future. I fear that means we might have to be
> willing to accept some minor short-term pain in exchange for major
> long-term improvement. Doing anything at the IETF that entails even the
> slightest downside is always very controversial. The guiding doctrine is
> “do no harm”. But we have to remember that inaction also causes harm. Many
> years of stagnation results in a gradual decline into irrelevance. History
> is full of companies that had an existing profitable business, and didn’t
> have the courage to risk disrupting that business, so instead they were
> just overtaken by competitors and slowly went bankrupt. Let us not ignore
> the danger of inaction.
>
> If we had five code points to use instead of four, then we could keep
> Classic ECN around as a “backwards compatibility” mechanism for some time,
> and that might make for an easier transition. But we don’t have that
> luxury, and Classic ECN marking is still extraordinarily rare, so we don’t
> even really have much of a backwards compatibility problem in the first
> place.
>
> Of course it is important to “do no harm”, but at the same time, let us
> not cling to that principle so fervently that it causes us to do no good
> either.
>
> Hence my conclusion is that we need to forget about classic “late marking”
> ECN, and move as swiftly as we can to an Internet using entirely L4S-style
> “early marking” ECN, before someone else does it first.
>
> --
>
>