Subject: [tsvwg] L4S utilization tests through a point-to-point wireless link

From: Pete Heist <[email-masked]>
Date: Mon, 23 Aug 2021 22:02:21 +0200

We've done some real-world tests of utilization for two L4S transports
(Prague and BBR2) vs TCP CUBIC, through DualPI2 and fq_codel
bottlenecks, with a wireless link on the path:

https://github.com/heistp/l4s-tests/#underutilization-with-bursty-links

Both TCP Prague and BBR2 see significant underutilization when
responding to signals in the L queue, relative to TCP CUBIC responding
to conventional ECN signals. BBR2 sees significantly higher throughput
when responding to conventional signals in an fq_codel queue, than when
responding to L4S signals in the L queue. This supports our earlier lab
tests which suggested that this would be the case on paths where
burstiness is introduced, e.g. with wireless links.

Pete

(so far, the ECT(1) police have not come knocking...)

---

From: Ingemar Johansson S <[email-masked]>
Date: Tue, 24 Aug 2021 11:25:47 +0000

Hi Pete

If I understand correct, this case mimics an upstream case where a device 
first transmits data over WiFi. The WiFi AP is then connected to the public 
internet over e.g a Cable modem which implements DualQ. The problem you 
describe becomes noticeable because the rate upstreams is limited to 20Mbps 
(or 25Mbps). A higher rate and the problem will be less apparent.
5G access can also give similar jitter in the uplink, the difference is here 
that the traffic is forwarded over high bandwidth aggregation links which 
makes the problem less  likely to occur.
My take is that , given that it is a known limitation in the uplink then the 
most sensible would be to pace the traffic from the WiFi AP.

/Ingemar

> -----Original Message-----
> From: tsvwg <[email-masked]> On Behalf Of Pete Heist
> Sent: den 23 augusti 2021 22:02
> To: tsvwg IETF list <[email-masked]>
> Subject: [tsvwg] L4S utilization tests through a point-to-point wireless 
> link
>
> (Prague and BBR2) vs TCP CUBIC, through DualPI2 and fq_codel bottlenecks,
> with a wireless link on the path:
>
> https://protect2.fireeye.com/v1/url?k=3d7105f3-62ea3fb6-3d714568-
> 867b36d1634c-9dfcd8c0e1850e3d&q=1&e=b665872e-037c-4f6f-a8ac-
> 4ad0474ce401&u=https%3A%2F%2Fgithub.com%2Fheistp%2Fl4s-
> tests%2F%23underutilization-with-bursty-links
>
> Both TCP Prague and BBR2 see significant underutilization when responding
> to signals in the L queue, relative to TCP CUBIC responding to conventional
> ECN signals. BBR2 sees significantly higher throughput when responding to
> conventional signals in an fq_codel queue, than when responding to L4S
> signals in the L queue. This supports our earlier lab tests which suggested
> that this would be the case on paths where burstiness is introduced, e.g.
> with wireless links.
>
> Pete
>

---

From: Jonathan Morton <[email-masked]>
Date: Tue, 24 Aug 2021 20:03:32 +0300

> On 24 Aug, 2021, at 2:25 pm, Ingemar Johansson S <ingemar.s.johansson=[email-masked]> wrote:
> 
> internet over e.g a Cable modem which implements DualQ.

I think that's about right - except that the AQM is at the opposite end of the path to the Wifi AP, not adjacent to it.  So it's more like a peer-to-peer connection between a cable subscriber and someone on a WISP, on different continents to boot.  This is not a rare scenario, given how much video calling between friends and relatives is done these days.

> The problem you 

There's a lot of people out there with this level of capacity or less, especially in the upstream direction.  Any new proposal intended for Internet deployment will potentially have to deal with it.

> makes the problem less  likely to occur.

Actually, higher throughput just causes more packets to be stuffed into each aggregate, thereby keeping the burstiness *in time units* relatively similar.  The effect on L4S goodput is also broadly similar.

> most sensible would be to pace the traffic from the WiFi AP.

So once again, you propose changing something other than the actual cause of the problem, simply so that you and your friends don't have to lift a finger.  The technical term for this is "externalisation", as I have mentioned before.

Here's a question for you: given that L4S clearly requires a path with very low jitter to function effectively, how does an endpoint determine with some confidence that the path is suitable for L4S transport, *in advance* of negotiating such a transport?

 - Jonathan Morton

---

From: Ingemar Johansson S <[email-masked]>
Date: Tue, 24 Aug 2021 17:44:31 +0000

Hi

Please see inline [IJ]

/Ingemar

> From: Jonathan Morton <[email-masked]>
> Sent: den 24 augusti 2021 19:04
> To: Ingemar Johansson S <[email-masked]>
> Cc: Pete Heist <[email-masked]>; tsvwg IETF list <[email-masked]>
> Subject: Re: [tsvwg] L4S utilization tests through a point-to-point
wireless link
> 
> > On 24 Aug, 2021, at 2:25 pm, Ingemar Johansson S
> <ingemar.s.johansson=[email-masked]> wrote:
> >
> > If I understand correct, this case mimics an upstream case where a
> > device first transmits data over WiFi. The WiFi AP is then connected
> > to the public internet over e.g a Cable modem which implements DualQ.
> 
> I think that's about right - except that the AQM is at the opposite end of
the
> path to the Wifi AP, not adjacent to it.  So it's more like a peer-to-peer
> connection between a cable subscriber and someone on a WISP, on different
> continents to boot.  This is not a rare scenario, given how much video
calling
> between friends and relatives is done these days.

[IJ] OK, the behavior should be roughly the same then except that the
traffic would perhaps be slightly more paced out when the AQM is in the far
end. 

> 
> > The problem you
> > describe becomes noticeable because the rate upstreams is limited to
> > 20Mbps (or 25Mbps). A higher rate and the problem will be less apparent.
> 
> There's a lot of people out there with this level of capacity or less,
especially
> in the upstream direction.  Any new proposal intended for Internet
> deployment will potentially have to deal with it.
> 
> > 5G access can also give similar jitter in the uplink, the difference
> > is here that the traffic is forwarded over high bandwidth aggregation
> > links which makes the problem less  likely to occur.
> 
> Actually, higher throughput just causes more packets to be stuffed into
each
> aggregate, thereby keeping the burstiness *in time units* relatively
similar.
> The effect on L4S goodput is also broadly similar.
> 
> > My take is that , given that it is a known limitation in the uplink
> > then the most sensible would be to pace the traffic from the WiFi AP.
> 
> So once again, you propose changing something other than the actual cause
> of the problem, simply so that you and your friends don't have to lift a
finger.
> The technical term for this is "externalisation", as I have mentioned
before.
[IJ] I don't see this as being a solely a DualQ problem. 

> 
> Here's a question for you: given that L4S clearly requires a path with
very low
> jitter to function effectively, how does an endpoint determine with some
> confidence that the path is suitable for L4S transport, *in advance* of
> negotiating such a transport?

[IJ] My guess is as good or bad as yours, perhaps this will become a problem
in these cases, how widespread it will become is nothing I can judge. 
What I do believe though is that nothing is static. Commercial 5G access can
move towards shorter numerologies, already covered in the standards, this
shortens the slot intervals and thus reduces the jitter. I guess there is
bound to be improvements for WiFi as well ?  Meanwhile, waiting for
standards to improve it  can be necessary to handle issues where they occur.

---

From: Pete Heist <[email-masked]>
Date: Tue, 24 Aug 2021 20:13:05 +0200

On Tue, 2021-08-24 at 20:03 +0300, Jonathan Morton wrote:
> > device 
> I think that's about right - except that the AQM is at the opposite
> end of the path to the Wifi AP, not adjacent to it.  So it's more
> like a peer-to-peer connection between a cable subscriber and someone
> on a WISP, on different continents to boot.  This is not a rare
> scenario, given how much video calling between friends and relatives
> is done these days.

Just confirming, yes the first hop transmits over airMAX with flexible
framing (not 802.11 WiFi) then there's a backhaul link, then Ethernet
to the gateway and Internet. We placed DualPI2 on ingress at the
receiver at 20Mbps / 25Mbps. The limit at the sender is typically
higher than that, so was not the bottleneck. Looking at one pcap at the
receiver, I'm also not sure this was an "extreme" case of
aggregation/burstiness, although we'll have to test more link types
under different conditions to know for sure.

I run a web server, as do many of the ISPs other users. So, this
situation could appear if I were serving content to an end user with a
DualPI2 bottleneck at their end. Someone could call it "unusual" from
their perspective, but IMO it's important that when it comes to
congestion control at Internet scale, we settle on a system that works
generally well on all link types, at all RTTs. Otherwise, it's not
clear how users or admins should know which CCA to choose from one
activity to the next.

Pete

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 25 Aug 2021 12:08:22 +0000

Just to put things in perspective: Putting a 1ms threshold in a 20Mbps link allows for just 1 packet in the queue, the second will get marked, for a 25Mbps link the third will get marked. We showed that if the end to end network path is very smooth and burstless, and all L4S applications are smoothly paced, it "can" work. Trying this at home today with non-L4S applications and non-L4S networks and expecting the top performance is a bit over-optimistic... Try to set the threshold to a more realistic setting if you want to support those conditions (4 or 5ms), and the results will be much better, allowing you to benefit from L4S today, even when being constrained as you are.

When deploying technology, it’s a matter of making things work, rather than trying it not to work. When things evolve, it only can get better... Look to the future...

Koen.

-----Original Message-----
From: tsvwg <[email-masked]> On Behalf Of Pete Heist
Sent: Tuesday, August 24, 2021 8:13 PM
To: tsvwg IETF list <[email-masked]>
Subject: Re: [tsvwg] L4S utilization tests through a point-to-point wireless link

On Tue, 2021-08-24 at 20:03 +0300, Jonathan Morton wrote:
> end of the path to the Wifi AP, not adjacent to it.  So it's more like 
> a peer-to-peer connection between a cable subscriber and someone on a 
> WISP, on different continents to boot.  This is not a rare scenario, 
> given how much video calling between friends and relatives is done 
> these days.

Just confirming, yes the first hop transmits over airMAX with flexible framing (not 802.11 WiFi) then there's a backhaul link, then Ethernet to the gateway and Internet. We placed DualPI2 on ingress at the receiver at 20Mbps / 25Mbps. The limit at the sender is typically higher than that, so was not the bottleneck. Looking at one pcap at the receiver, I'm also not sure this was an "extreme" case of aggregation/burstiness, although we'll have to test more link types under different conditions to know for sure.

I run a web server, as do many of the ISPs other users. So, this situation could appear if I were serving content to an end user with a
DualPI2 bottleneck at their end. Someone could call it "unusual" from their perspective, but IMO it's important that when it comes to congestion control at Internet scale, we settle on a system that works generally well on all link types, at all RTTs. Otherwise, it's not clear how users or admins should know which CCA to choose from one activity to the next.

Pete

---

From: Mirja Kuehlewind <[email-masked]>
Date: Wed, 25 Aug 2021 12:32:42 +0000

+1

On 25.08.21, 14:08, "tsvwg on behalf of De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked] on behalf of [email-masked]> wrote:

    Koen.

    -----Original Message-----
    From: tsvwg <[email-masked]> On Behalf Of Pete Heist
    Sent: Tuesday, August 24, 2021 8:13 PM
    To: tsvwg IETF list <[email-masked]>
    Subject: Re: [tsvwg] L4S utilization tests through a point-to-point wireless link

    Pete

---

From: Sebastian Moeller <[email-masked]>
Date: Wed, 25 Aug 2021 14:33:27 +0200

Hi Koen,

> On Aug 25, 2021, at 14:08, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 

	[SM] But at that point the latency improvement over an appropriately configured state of the art AQM like cake or fq_codel is rather modest, and that state of the art AQM will work well with existing protocols and applications...
At that point we are down to most of the pain of introducing a non-backward compatible AQM with little gains. 
	BUT I really believe you are misreading the end-user market here, achieving the advertised/contractual throughput on home links is not only something end-users are keen on/test regularly, but it is also something that, at least in the EU is also (partly) guaranteed by the EU in the "REGULATION (EU) 2015/2120 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL" Article 4.4  (https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32015R2120)... So to not afoul of the regulations deploying ISPs will now have to scale the L-queue target individually for each individual user (or at least as a function of the rate)... On the plus side, this will also slightly ameliorate the still unhandled RTT Bias introduced by the too large latency target for the C-queue.* I wonder how an DualQ instance would perform utilization and latency wise that uses a 5ms target for both queues (it might be necessary to re-introduce the burst tolerance additions from the original PIE as these also help in the case when the true RTT of a flow is above the reference RTT in PIE).

Best Regards
	Sebastian

*) I do not entertain the idea that https://arxiv.org/pdf/2107.01003.pdf gives a convincing rationale for how this number was chosen. It rather looks like a post-hoc attempt at justification of the initially selected number. In any way what seems missing is a data set exploring the consequences of changing that parameter.

---

From: Jonathan Morton <[email-masked]>
Date: Wed, 25 Aug 2021 16:01:26 +0300

> On 25 Aug, 2021, at 3:08 pm, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 

Codel already defaults to a 5ms threshold, so that wouldn't be much of an improvement over current technology, would it?

The key feature of Codel that the current L4S AQMs lack is *burst tolerance*.  This is what allows Codel to outperform L4S on bursty paths and with bursty application traffic, not the higher queue target itself.

 - Jonathan Morton

---

From: Pete Heist <[email-masked]>
Date: Wed, 25 Aug 2021 15:06:42 +0200

Yes, I understand. If we increase the buffer sizes and trade off lower
latency, we can see higher throughput.

I would also point out Section 4.3 of RFC7567 (AQM Algorithm Deployment
SHOULD NOT Require Operational Tuning):

https://datatracker.ietf.org/doc/html/rfc7567#section-4.3

Pete

On Wed, 2021-08-25 at 12:08 +0000, De Schepper, Koen (Nokia -
BE/Antwerp) wrote:
> Just to put things in perspective: Putting a 1ms threshold in a 20Mbps
> link allows for just 1 packet in the queue, the second will get marked,
> for a 25Mbps link the third will get marked. We showed that if the end
> to end network path is very smooth and burstless, and all L4S
> applications are smoothly paced, it "can" work. Trying this at home
> today with non-L4S applications and non-L4S networks and expecting the
> top performance is a bit over-optimistic... Try to set the threshold to
> a more realistic setting if you want to support those conditions (4 or
> 5ms), and the results will be much better, allowing you to benefit from
> L4S today, even when being constrained as you are.
> 
> When deploying technology, it’s a matter of making things work, rather
> than trying it not to work. When things evolve, it only can get
> better... Look to the future...
> 
> Koen.
> 
> > like

---

From: <[email-masked]>
Date: Wed, 25 Aug 2021 14:37:01 +0000

[PH] Yes, I understand. If we increase the buffer sizes and trade off lower latency, we can see higher throughput.

AFAIK, the optimum buffer depth depends on the number of competing connections during congestion and RTT mix. Endemically adding buffer creates bufferbloat.

[PH] I would also point out Section 4.3 of RFC7567 (AQM Algorithm Deployment SHOULD NOT Require Operational Tuning):

[PH] https://datatracker.ietf.org/doc/html/rfc7567#section-4.3

...don't tell that too many people, I could loose my job ;-) .... seriously: sometimes my view on vendor innovation is "...if your AQM design was at least somewhat compatible with that of your last HW generation..."

I think providing a fast and reliable congestion feedback to transport is worth a try. I'm having the impression that some applications benefit from trading a small amount of throughput for decreased jitter and packet loss. And not all of those which are within scope are applicable over 10 mbps access interfaces.

Regards, Ruediger

BE/Antwerp) wrote:
> link allows for just 1 packet in the queue, the second will get 
> marked, for a 25Mbps link the third will get marked. We showed that if 
> the end to end network path is very smooth and burstless, and all L4S 
> top performance is a bit over-optimistic... Try to set the threshold 
> to a more realistic setting if you want to support those conditions (4 
> or 5ms), and the results will be much better, allowing you to benefit 
> from L4S today, even when being constrained as you are.
> 
> > > device first transmits data over WiFi. The WiFi AP is then 
> > > connected to the public internet over e.g a Cable modem which 
> > > implements DualQ.
> > 
> > like a peer-to-peer connection between a cable subscriber and 
> > someone on a WISP, on different continents to boot.  This is not a 
> > rare scenario, given how much video calling between friends and 
> > relatives is done these days.
> 
> higher than that, so was not the bottleneck. Looking at one pcap at 
> the receiver, I'm also not sure this was an "extreme" case of

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 25 Aug 2021 15:34:43 +0000

>>"REGULATION (EU) 2015/2120 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL" Article 4.4  (https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32015R2120)

Interesting literature. I'm not a layer, but to me it seems you are quoting a small paragraph out of context and misinterpreted it. BTW with new technology, new legislation can be produced if needed.

Koen.

-----Original Message-----
From: Sebastian Moeller <[email-masked]> 
Sent: Wednesday, August 25, 2021 2:33 PM
To: De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]>
Cc: Pete Heist <[email-masked]>; tsvwg IETF list <[email-masked]>
Subject: Re: [tsvwg] L4S utilization tests through a point-to-point wireless link

Hi Koen,

Best Regards
	Sebastian

> 
> When deploying technology, it's a matter of making things work, rather than trying it not to work. When things evolve, it only can get better... Look to the future...
> 
> Koen.
>

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 25 Aug 2021 15:36:08 +0000

>>The key feature of Codel that the current L4S AQMs lack is *burst tolerance*.

This is exactly that L4S wants to avoid: bursts resulting in latency spikes of possibly 100's of milliseconds. How tolerant do you want to be? It is up to the AQM design/parameters. Nobody prevents you from developing a more tolerant L4S solution (as long as you believe it is needed, and hopefully not leading into 100's of milliseconds, or is it seconds that Codel can tolerate?).

>> Codel already defaults to a 5ms threshold, so that wouldn't be much of an improvement over current technology, would it?

It definitely would. It eliminates the large latency spikes, and for smoother and higher throughput network technologies (not the border cases you are referring to), it can already be used with the proposed defaults.

Look to the future...
Koen.

-----Original Message-----
From: Jonathan Morton <[email-masked]> 
Sent: Wednesday, August 25, 2021 3:01 PM
To: De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]>
Cc: Pete Heist <[email-masked]>; tsvwg IETF list <[email-masked]>
Subject: Re: [tsvwg] L4S utilization tests through a point-to-point wireless link

 - Jonathan Morton

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 25 Aug 2021 15:40:08 +0000

>>I would also point out Section 4.3 of RFC7567 (AQM Algorithm Deployment SHOULD NOT Require Operational Tuning):

Again a one-liner quote... When reading the full section, I believe L4S is doing a very good job.
Quoting: " AQM algorithm deployment MAY support further manual tuning that
      could improve performance in a specific deployed network.
      Algorithms that lack such variables are acceptable, but, if such
      variables exist, they SHOULD be externalized (made visible to the
      operator)."
I believe the above quote is very applicable in the cases you tested. An L4S STEP only has one parameter, the target, which in this case can be adapted to the burstiness that the network generates. It would be silly trying (read: impossible) to undo latency that happened already on the path (in the end, network caused jitter is just varying latency that was applied to your packets and cannot be reverted anymore). For these network induced bursts, packets get paced out again on the bottleneck, and if paced at the sender, all packets get the same latency as the first packet of the burst already had. From this point onwards, you can only wait until those technologies get improved, which L4S makes worth the effort spend.

So again: Look to the future...
Koen.

-----Original Message-----
From: Pete Heist <[email-masked]> 
Sent: Wednesday, August 25, 2021 3:07 PM
To: De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]>
Cc: tsvwg IETF list <[email-masked]>
Subject: Re: [tsvwg] L4S utilization tests through a point-to-point wireless link

Yes, I understand. If we increase the buffer sizes and trade off lower latency, we can see higher throughput.

I would also point out Section 4.3 of RFC7567 (AQM Algorithm Deployment SHOULD NOT Require Operational Tuning):

Pete

BE/Antwerp) wrote:
> When deploying technology, it's a matter of making things work, rather

---

From: Jonathan Morton <[email-masked]>
Date: Wed, 25 Aug 2021 18:55:44 +0300

> On 25 Aug, 2021, at 6:36 pm, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 

As you should know very well by now, by default Codel tolerates exceeding the 5ms queue target for up to 100ms before signalling begins.  This works just fine with conventional TCP in congestion-avoidance mode, since the cwnd doesn't grow much in that short period of time.  There are acknowledged shortcomings with the slow-start phase of TCP, and with some non-TCP traffic, although for example it also works just fine with existing videoconferencing protocols.

Even these small excursions are hidden from most other traffic by the FQ mechanism usually deployed alongside Codel. This means that a latency-sensitive gaming session is unaffected by the startup of several TCP flows at the same time, even though the latter themselves experience a brief excursion in delay.  The Multiplicative Decrease response of conventional TCP ensures that only a few CE marks are needed to bring these flows into line.

We have also explored the use of Codel with a modified tuning for high-fidelity congestion control signalling.  It seemed to work just fine, under the assumption that a non-ambiguous signalling scheme was used, hence the high-fidelity and conventional signals could be distinguished upon receipt.

 - Jonathan Morton

---

From: Sebastian Moeller <[email-masked]>
Date: Wed, 25 Aug 2021 17:57:14 +0200

Hi Koen,

On 25 August 2021 17:34:43 CEST, "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]> wrote:
>Interesting literature. I'm not a layer, but to me it seems you are quoting a small paragraph out of context and misinterpreted it.

[SM] Sorry, I am not quoting anything here, I referenced one paragraph that is relevant, just read the whole thing and tell me where I am misinterpreting the text. The text is clear in that ISP owe their paying customers access to all of the internet's endpoints (as far as under the ISPs control) at the rates as defined in the contract. The monitoring of the rate requirement is put in the hands of the national regulatory agencies. In that light introducing a new technology that will reduce utilisation has its issues. Since this is not really relevant for the IETF's decision on L4S I will drop this, but reject your so far unsubstantiated claim of misinterpretation.

 BTW with new technology, new legislation can be produced if needed.

[SM] Sure, but to be deployed soon technology better is compliant with existing legislation....  now I am happy to postpone L4S RFCication until the EU changes this regulation....

Again, I will drop this sub-thread as all I wanted to show is that underutilisation can be viewed under different perspectives where it becomes more or less of an issue.
Personally, I am with you on this (team lower latency). And in the past I even shaped my 100/40 link down to 49/35 (the shaping limit of my router) as that resulted in way less latency under load increase, or more RPM, and I personally value reliable low latency over throughput. But that view is not universally shared by all users/regulators.

Also the observed underutilisation runs afoul of the claims in the drafts, and that IMHO is an issue for the WG/IETF to fix.

Regards
           Sebastian

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

---

From: Sebastian Moeller <[email-masked]>
Date: Wed, 25 Aug 2021 19:36:13 +0200

Hi Koen,

> On Aug 25, 2021, at 17:36, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 
> This is exactly that L4S wants to avoid: bursts resulting in latency spikes of possibly 100's of milliseconds. How tolerant do you want to be?

	[SM] Well, higher burst tolerance in Codel and PIE than in dualQ also should leads to considerably less RTT-bias in the AQM, as overshooting responses from longer RTT flows (that still have received a 'slow-down' signal but where the result of that signal is not yet apparent at the AQM) will not be throttled as hard. As Jonathan wrote Codel does quite well with its default interval of 100ms.

> It is up to the AQM design/parameters. Nobody prevents you from developing a more tolerant L4S solution (as long as you believe it is needed, and hopefully not leading into 100's of milliseconds, or is it seconds that Codel can tolerate?).
> 

	Erm, you propose that DualQ with an L/C target set of 5/15 will outperform fq_codel considerably in utilization and low latency? I would like to see data supporting that claim, please.
The problem with Pete's case really is, that any endpoint might introduce bursts due to using WiFi, DSL, to a lesser degree DOCSIS, so any AQM not in a closely policed environment should deal with such traffic patterns gracefully. But that would mean that your 4-5 ms become the default target for dualQ...

Regards
	Sebastian

---

From: Pete Heist <[email-masked]>
Date: Wed, 25 Aug 2021 21:39:45 +0200

On a more technical topic, since I have this link available for
testing, I can help verify that tuning dualpi2 works as expected.

Just to make sure, is it the step_thresh parameter I should change from
1ms to 5ms?

Pete

On Wed, 2021-08-25 at 15:40 +0000, De Schepper, Koen (Nokia -
BE/Antwerp) wrote:
> > > I would also point out Section 4.3 of RFC7567 (AQM Algorithm
> > > Deployment SHOULD NOT Require Operational Tuning):
> 
> I believe the above quote is very applicable in the cases you tested.
> An L4S STEP only has one parameter, the target, which in this case
> can be adapted to the burstiness that the network generates. It would
> be silly trying (read: impossible) to undo latency that happened
> already on the path (in the end, network caused jitter is just
> varying latency that was applied to your packets and cannot be
> reverted anymore). For these network induced bursts, packets get
> paced out again on the bottleneck, and if paced at the sender, all
> packets get the same latency as the first packet of the burst already
> had. From this point onwards, you can only wait until those
> technologies get improved, which L4S makes worth the effort spend.
> 
> Yes, I understand. If we increase the buffer sizes and trade off
> lower latency, we can see higher throughput.
> 
> I would also point out Section 4.3 of RFC7567 (AQM Algorithm
> Deployment SHOULD NOT Require Operational Tuning):
>

---

From: Pete Heist <[email-masked]>
Date: Fri, 27 Aug 2021 13:00:49 +0200

I've added runs to a server at 7ms RTT, and tested changing step_thresh
from 1ms to 5ms. The new results are here:

https://github.com/heistp/l4s-tests/#bursty-link-underutilization-test-czech-7ms-rtt

Summary of the results:

- underutilization also occurs at 7ms RTT
- raising the bottleneck bandwidth from 20 Mbps to 40 Mbps makes the
  underutilization percentage higher (larger aggregates? not sure)
- raising DualPI2's step_thresh to 5ms improves Prague's utilization to
  just under that of CUBIC, but BBR2 still underutilizes somewhat even
  with the increased step_thresh

Pete

---

From: Ingemar Johansson S <[email-masked]>
Date: Tue, 31 Aug 2021 06:53:02 +0000

Hi

I have too observed that BBRv2 gives a slight underutilization with L4S in 5G 
simulations that I have run (L4S marking for UL traffic). I would however 
guess that BBRv2 is  still work in progress so I don't judge that as alarming.

/Ingemar

> Sent: den 27 augusti 2021 13:01
> https://protect2.fireeye.com/v1/url?k=10f6b98e-4f6d808c-10f6f915-
> 869a14f4b08c-1f867293a2157fed&q=1&e=4f787c35-7514-4078-9e51-
> 4e373d8cfe3a&u=https%3A%2F%2Fgithub.com%2Fheistp%2Fl4s-
> tests%2F%23bursty-link-underutilization-test-czech-7ms-rtt
>
> Summary of the results:
>
> > https://protect2.fireeye.com/v1/url?k=1c2b6eef-43b057ed-1c2b2e74-
> 869a1
> > 4f4b08c-d5306c5d24ec8f8d&q=1&e=4f787c35-7514-4078-9e51-
> 4e373d8cfe3a&u=
> > https%3A%2F%2Fgithub.com%2Fheistp%2Fl4s-
> > when responding to conventional signals in an fq_codel queue, than
> > when responding to L4S signals in the L queue. This supports our
> > earlier lab tests which suggested that this would be the case on paths
> > where burstiness is introduced, e.g. with wireless links.
> >
> > Pete
> >