Subject: [tsvwg] tests of Prague utilization with bursty competition in L and C

From: Pete Heist <[email-masked]>
Date: Sun, 08 Aug 2021 18:03:58 +0200

In this test, we used our same 20 Mbps DualPI2 bottleneck to test Skype
video traffic together with Prague traffic. The Skype traffic was first
placed in L, then in C, and pcaps/plots were obtained for each:

https://github.com/heistp/l4s-tests/#underutilization-with-bursty-traffic

With Skype traffic in L, the Prague flow's steady-state throughput was
2.4 Mbps, for a utilization of around 15% (relative to the 18.5 Mbps
measured for a single flow with no competition, minus 2.5 Mbps from the
Skype traffic).

With Skype traffic in C, the Prague flow's steady-state throughput was
12.5 Mbps, for a utilization of around 78%.

This suggests that in addition to bursty traffic in L leading to under-
utilization, bursty traffic in C can lead to under-utilization for
capacity seeking flows in L through the DualPI2 coupling mechanism.

Pete

---

From: Pete Heist <[email-masked]>
Date: Thu, 12 Aug 2021 17:44:50 +0200

We realized that this result needed a more thorough analysis, so it's
been updated with:

- comparisons to the performance of plain CUBIC in DualPI2 and Codel
- a more accurate Skype bitrate, taken from the data direction only
- more accurate TCP goodputs, calculated from seqnos
- use of goodput backoff, instead of utilization percentages
- a note about CUBIC's large goodput dip in DualPI2 C with Skype
- a summary chart

The same general conclusion remains, that there are utilization issues
for L4S flows when mixed with bursty traffic, relative to the current
state of the art.

It's at the same location:
https://github.com/heistp/l4s-tests/#underutilization-with-bursty-traffic

Pete

---

From: Sebastian Moeller <[email-masked]>
Date: Mon, 23 Aug 2021 22:40:46 +0200

Dear Pete, dear list,

I just had a look at G.998.4 : Improved impulse noise protection for digital subscriber line (DSL) transceivers, which specifies a re-transmission system over a DSL link, where the sender keeps data transfer units (DTUs) in its buffers until they are acknowledged as being received correctly by the receiver. The system uses separate bearers for the ACK channels (potentially one per direction).

For profile 17a  the "Aggregate interleaver and de-interleaver delay (octets)" equals 98304, with 100 Mbps down and 40 Mbps up (as e.g. offered by major ISPs in Germany) we can estimate the worst case delay introduced by this as
98304 * 40/100 * 8 / (100 * 1000^2) * 1000 = 3.145728 ms 
or for the other direction:
98304 * (1 - 40/100) * 8 / (100 * 1000^2) * 1000 = 4.718592 ms

IMHO the bigger issue is that now this 3.1 or4.7  ms worth of data (98304 * 40/100 / (1500+38) = 25.6 or 98304 * (1 - 40/100) / (1500+38) = 38.4 full MTU packets) can potentially be released by the respective receiver side in one fell swoop, leading to an L4S unfriendly line-rate bursts (unless the receiver uses a traffic shaper to smooth out the burst somewhat).

I would love to see data how L4S/TCP Prague performs over a DSL link with G.INP active and reasonable amounts of retransmissions (as these are not that rare).

Regards
	Sebastian

> On Aug 12, 2021, at 17:44, Pete Heist <[email-masked]> wrote:
>

---

From: Pete Heist <[email-masked]>
Date: Tue, 24 Aug 2021 17:45:44 +0200

Hi Sebastian, thanks for the reference. In case there's a link
somewhere available to test I'd be willing to try it. There's a
timescale of 250 us suggested at the end of l4s-id section 5.5, but I
didn't notice any further guidance in the drafts:

https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id-19#section-5.5

Pete

On Mon, 2021-08-23 at 22:40 +0200, Sebastian Moeller wrote:
> Dear Pete, dear list,
> 
> 
> I just had a look at G.998.4 : Improved impulse noise protection for
> digital subscriber line (DSL) transceivers, which specifies a re-
> transmission system over a DSL link, where the sender keeps data
> transfer units (DTUs) in its buffers until they are acknowledged as
> being received correctly by the receiver. The system uses separate
> bearers for the ACK channels (potentially one per direction).
> 
> For profile 17a  the "Aggregate interleaver and de-interleaver delay
> (octets)" equals 98304, with 100 Mbps down and 40 Mbps up (as e.g.
> offered by major ISPs in Germany) we can estimate the worst case delay
> introduced by this as
> IMHO the bigger issue is that now this 3.1 or4.7  ms worth of data
> (98304 * 40/100 / (1500+38) = 25.6 or 98304 * (1 - 40/100) / (1500+38)
> = 38.4 full MTU packets) can potentially be released by the respective
> receiver side in one fell swoop, leading to an L4S unfriendly line-rate
> bursts (unless the receiver uses a traffic shaper to smooth out the
> burst somewhat).
> 
> I would love to see data how L4S/TCP Prague performs over a DSL link
> with G.INP active and reasonable amounts of retransmissions (as these
> are not that rare).
> 
> 
> Regards
>         Sebastian
> 
> 
>

---

From: Sebastian Moeller <[email-masked]>
Date: Tue, 24 Aug 2021 21:07:23 +0200

Hi Pete,

> On Aug 24, 2021, at 17:45, Pete Heist <[email-masked]> wrote:
> 
> somewhere available to test I'd be willing to try it.

[SM] I volunteer my private internet access link* for such testing.

*) I spent some time and effort to clean up my apartment's telephony wiring which made my link much more stable, but that also reduced the G.INP retransmission rate considerably...

> There's a

	Well, I fail to see how 250µs is a reasonable permissible burst size, given that most links along a path are faster than the bottleneck.... If a dsl modem will burst out packets for 250µs at its gigabit internet interface, that corresponds to
(0.25 * 1000^3 / 1000) = 250kb or  (0.25 * 1000^3 / 1000)  / (1538 * 8) = 20.3185955787 full MTU packets (with overhead)...
 these will expand to 25ms worth of transmission and hence queuing time at a 10Mbps bottleneck link... 

My point is, 250µs burst is not that helpful as guidance... as IIUC these 250µs would be tolerable in an L4S bottleneck, but none of the upstream nodes (short of the sender0 has any robust estimate of the relevant bottleneck rate.

What am I missing here?

Regards
	Sebastian

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 25 Aug 2021 13:44:46 +0100

Sebastian, Pete, see [BB] inline...

> My point is, 250µs burst is not that helpful as guidance... as IIUC these 250µs would be tolerable in an L4S bottleneck, but none of the upstream nodes (short of the sender0 has any robust estimate…

[BB] What you're missing is that the section you're quoting from is not 
about links in general. It's about the link below where the L4S AQM is 
deployed. Because an RFC can only say what to implement at the link 
where L4S support is added. The section you're quoting from says this:

    This document does not set any quantified requirements for links to
    limit burst delay, primarily because link technologies are outside
    the remit of L4S specifications.  Nonetheless, it would not make
    sense to implement an L4S AQM that feeds into a particular link
    technology without also reviewing opportunities to reduce any form of
    burst delay introduced by that link technology.

This was done (successfully) to the DOCSIS MAC as part of the work to 
introduce Low Latency DOCSIS, which added an L4S AQM feeding this 
improved MAC into the link. This is all a mandatory part of DOCSIS now.

Regarding link-layer retransmission bursts in DSL, the first L4S 
performance tests were all done on Alcatel-Lucent's DSL testbed, using 
real equipment and copper lines, and we verified the results were the 
same as with our ethernet-based testbed. I don't have any record of what 
the degree of interleaving was, and I admit we didn't look into 
deliberately winding up the interleaver to the worst case. With ADSL1, 
ADSL2/2+ and VDSL2, interleaving typically adds 16ms, 8ms and 8ms of 
delay (always) in the downstream, but 0ms in the upstream. This is the 
latency price that telcos pay for sweating a hundred-year old asset.  In 
addition there's the (intermittent) repair delay you talk about, which 
is about 0.5ms for the protection channel to detect each loss and 4ms to 
repair it, as you say.

Any low latency technology would have to deal with these physical 
realities. I can't see how this is a specific critique of L4S. Just 
before the above para, the draft says, in general:

    ...delay
    reduction proceeds by cutting down 'the longest pole in the tent',
    which turns the spotlight on the next longest, and so on.

If a telco's customer asks for low latency over a particularly flaky 
copper pair that cannot easily be remedied physically, so that the 
interleaver has to be wound up to max, a tradeoff will be unavoidable 
between latency and utilization. As with DOCSIS, if a telco was 
introducing L4S over DSL, it would be prepared to go round seeking out 
bad earths, low resistance across the pairs and other causes of 
interference on flaky links. Interleaving is a general sledgehammer to 
crack all these problems without having to physically visit the line. 
This would all proceed in parallel to the telcos' programs to replace 
copper with fibre.

Just because some links are flaky doesn't mean the IETF should give up 
trying to reduce delay as much as possible for everyone else.

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Sebastian Moeller <[email-masked]>
Date: Wed, 25 Aug 2021 15:18:00 +0200

Hi Bob,

thanks and see [SM] below

> On Aug 25, 2021, at 14:44, Bob Briscoe <[email-masked]> wrote:
> 
> [BB] What you're missing is that the section you're quoting from is not about links in general. It's about the link below where the L4S AQM is deployed. Because an RFC can only say what to implement at the link where L4S support is added. The section you're quoting from says this:

	[SM] Okay so that basically is restricted to the egress interface(s) of the L4S node. In that case it would help at least me to spell this out explicitly.

> This was done (successfully) to the DOCSIS MAC as part of the work to introduce Low Latency DOCSIS, which added an L4S AQM feeding this improved MAC into the link. This is all a mandatory part of DOCSIS now. 

	[SM] Seems like this fact and a reference to the relevant DOCSIS spec document could help to clarify this section.

> 
> Regarding link-layer retransmission bursts in DSL, the first L4S performance tests were all done on Alcatel-Lucent's DSL testbed, using real equipment and copper lines, and we verified the results were the same as with our ethernet-based testbed.

	[SM] If I recall correctly, you only applied an L4S AQM for the downstream and that was located upstream of the DSL link, so assuming downstream G.INP was active on that link, your AQM would never have seen the burst introduced...

> I don't have any record of what the degree of interleaving was, and I admit we didn't look into deliberately winding up the interleaver to the worst case.

	[SM] This is not what I am talking about, the interleaver just adds a more or less static delay, it is G.INP and its retransmissions that have the potential to introduce bursts at the line-rate of the egress interface, sorry if I did not explain this sufficiently before. (It is just that the G.INP machinery reuses the same memory that could be used for interleaving (and might still partly be used for interleaving, as G.INP and interleaving are not mutually exclusive) I assume this is because this allowed to enable G.INP on already existing CPE by modifying the firmware instead of requiring a hardware change, but I digress).

>  With ADSL1, ADSL2/2+ and VDSL2, interleaving typically adds 16ms, 8ms and 8ms of delay (always) in the downstream, but 0ms in the upstream.

	[SM] That might be true for BT, but is not generally true, I have seen links with considerable interleaving delay in both up- and downstream direction.

> This is the latency price that telcos pay for sweating a hundred-year old asset.  In addition there's the (intermittent) repair delay you talk about, which is about 0.5ms for the protection channel to detect each loss and 4ms to repair it, as you say.

	[SM] Let's agree to not go to deep into the details, the problem here is that DSL CPE release the Data in DTU sequence, that is, if DTU X is detected as damaged and a retransmit is requested all DTUs Y with Y>X that are delivered until the retransmission is successful or hits the timeout, will potentially be released on the the egress interface in one batch, resulting in a burst of ~(Y-X) packets at line rate. And it is these kinds of bursts that cause over-marking and under-utilisation in L4S AQMs.

> Any low latency technology would have to deal with these physical realities.

	[SM] +1; the goal should be to work well with the existing internet, which already includes a number of burst sources.

> I can't see how this is a specific critique of L4S. Just before the above para, the draft says, in general:
>    ...delay

	[SM] Let me spell it out then, the critique is that the L4S-AQM design/implementation seems to lack the required amount of burst tolerance to deliver the utilization it promises over at least parts of the existing internet.

> 
> If a telco's customer asks for low latency over a particularly flaky copper pair that cannot easily be remedied physically, so that the interleaver has to be wound up to max, a tradeoff will be unavoidable between latency and utilization.

	[SM] Not really, that interleaver will only increase the static RTT by a few (dozens) of milli seconds, which should have a negligible effect on utilization, sure latency will be statically increased, but the typical "killer" for real time applications is not static RTT (as long as that is somewhat sane) but RTT-variance or jitter. Interleaving does not increase jitter noticeably, G.INP however does (when retransmissions are triggered).

> As with DOCSIS, if a telco was introducing L4S over DSL, it would be prepared to go round seeking out bad earths, low resistance across the pairs and other causes of interference on flaky links. Interleaving is a general sledgehammer to crack all these problems without having to physically visit the line. This would all proceed in parallel to the telcos' programs to replace copper with fibre.

	[SM] Well, here is my disconnect, if we expect the world to change for L4S to be useful anyway, why can L4S then not be completely orthogonal to the existing internet and not redefine the CE mark's meaning? This is a rhetorical question...

> Just because some links are flaky doesn't mean the IETF should give up trying to reduce delay as much as possible for everyone else.

	[SM] Well, the point is even if my ISP does not roll out L4S AQM, given the ambitious scope of L4S my bursty upstream packets might hit an L4S AQM along the path and reduce the utilization through that AQM. Now, this is an accidental case where the sender is not even acting maliciously, but there is also clear potential for new attack vectors (that might be hard to detect, if the smoothed attack flow rate is not unreasonable, but bursty enough to cause over-marking). 

Regards
	Sebastian

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 26 Aug 2021 12:11:35 +0100

Pete,

Thanks for this experiment. I've been trying to interpret the results. I 
think I understand what's going on. But before deciding for sure, I need 
to confirm all my assumed answers to these questions:

  * Q1. It's hard to be sure what's going on with a result from just a
    single fairly extreme scenario. 160ms base RTT is fairly unusual.
    Are the results as bad with a more common base RTT for the CCA (e.g.
    40ms)? It would be even more useful to see results under a range of
    link rates and RTTs, to know how much of the effect of these bursts
    is compounded by large RTTs, or compounded by the ratio between the
    sender's interface rate and the bottleneck.
      o Also, it would be useful to see latency as well as throughput -
        to determine whether latency is being compromised at all, or
        solely utilization.
  * Q2. The ECN fields of the Skype packets are weird in the packet
    capture for Skype packets in the L queue. Did you use some other way
    of classifying packets into the L queue? Based on my assumption in
    Q4.1, those in the direction I'm assuming is the same as the Prague
    data are all Not-ECT. While those in the opposite direction are
    ECT(1), but only if packet size >=1025B. Otherwise sizes <1025B are
    Not-ECT.
      o BTW, do you know if Skype takes any notice of ECN markings? I'm
        assuming it doesn't.
  * Q3. How did you make the Skype session repeatable? Did you capture a
    pcap through the same capacity link once then rerun it each time? Or
    did you record a video, and rerun it into Skype's camera input
    interface?
  * Q4. I'm having trouble matching the 'real world test setup' to the
    pcaps.
      o Q4.1. Which of the two (forward and reverse) Skype flows in the
        pcap is running through the bottleneck in the same direction as
        the the TCP Prague data flow? I assume the one with the much
        larger bursts (dst 10.72.0.88).
      o Q4.2. Are the packet captures in both directions taken at the
        same place (so one direction is after the bottleneck while the
        other is before)?
      o Q4.3. Was the 20Mb/s capacity symmetric?

A nitty gripe: The blue bars in your charts are surely incorrectly 
labelled as "Theoretical Max w/ Skype". They are the Skype average bit 
rate subtracted from what the CCA achieves without Skype, which is a 
practical outcome, not a theoretical max, because in theory there ought 
to be plenty of room for multiplexing between the bursts.

For now, here's my record of the Skype burst pattern in the pcap (with 
Skype in either queue - it doesn't seem to make much difference):

  * In TCP Prague's data direction, Skype is sending bursts of irregular
    sizes at widely spaced but irregular intervals that average about
    one burst every 50-80ms.
      o Each burst consists of about 15kB on average in 10-25 datagrams
        all arriving within a few hundred μs (that's >200Mb/s
        instantaneously).
      o The average UDP throughput is about 2.3Mb/s if that means
        anything with such bursty traffic (was your 1.79Mb/s excluding
        headers?)
  * In the other direction Skype sends a regular burst every 100ms.
      o These bursts also have irregular sizes but they definitely
        alternate between large (~80kb) and small (~30kb). The large
        bursts consist of 15-20 packets as well, but over a much longer
        duration burst (30-40ms, that's only >3Mb/s instantaneously).
      o The average UDP rate in this direction is 0.88Mb/s
  * Plus, in both directions, Skype sends regular single small datagrams
    every 20ms.

Thanks again for doing all this.

Bob

---

From: Jonathan Morton <[email-masked]>
Date: Thu, 26 Aug 2021 15:52:23 +0300

> On 26 Aug, 2021, at 2:11 pm, Bob Briscoe <[email-masked]> wrote:
> 
> It's hard to be sure what's going on with a result from just a single fairly extreme scenario. 160ms base RTT is fairly unusual.

Tell that to Australians and New Zealanders.  They will laugh you out of the room.

> Are the results as bad with a more common base RTT for the CCA (e.g. 40ms)?

As it happens, the base RTT doesn't matter very much for this scenario.  The jitter is what influences the L queue AQM's marking, and thus the majority of the impact on TCP Prague.

I'll leave the rest to Pete.

 - Jonathan Morton

---

From: Pete Heist <[email-masked]>
Date: Thu, 26 Aug 2021 15:22:21 +0200

Hi Bob,

On Thu, 2021-08-26 at 12:11 +0100, Bob Briscoe wrote:
> Pete, 
> 
> Thanks for this experiment. I've been trying to interpret the
> results. I think I understand what's going on. But before deciding
> for sure, I need to confirm all my assumed answers to these
> questions:
> 
>     - Also, it would be useful to see latency as well as throughput - to
>       determine whether latency is being compromised at all, or solely
>       utilization.

The pcaps are there for viewing RTT plots, but I'm short on time to
generate those.

>     - BTW, do you know if Skype takes any notice of ECN markings? I'm
>       assuming it doesn't.

My mistake. My tc filter match to mark ECT(1) on UDP packets above port
1024 actually marked UDP packets above length 1024. :) I had checked in
the pcap that the filter was "working", but didn't check enough packets
to notice this. I'll note this in the repo, but given the amount of
time needed to run/analyze the test, I'm not sure if/when we'll get
back to it. I suppose the effect of it is that less Skype traffic was
in L than we thought.

>    did you record a video, and rerun it into Skype's camera input

The source video was always very similar (gameplay from the same scenes
in the same game), but not pre-recorded such that it was exactly the
same.

>     - Q4.1. Which of the two (forward and reverse) Skype flows in the pcap
>       is running through the bottleneck in the same direction as the the
>       TCP Prague data flow? I assume the one with the much larger bursts
>       (dst 10.72.0.88)

Yes.

>     - Q4.2. Are the packet captures in both directions taken at the same
>       place (so one direction is after the bottleneck while the other is
>       before)?
>     - Q4.3. Was the 20Mb/s capacity symmetric?

Yes.

> labelled as "Theoretical Max w/ Skype". They are the Skype average
> bit rate subtracted from what the CCA achieves without Skype, which
> is a practical outcome, not a theoretical max, because in theory
> there ought to be plenty of room for multiplexing between the bursts.
> 
> For now, here's my record of the Skype burst pattern in the pcap
> (with Skype in either queue - it doesn't seem to make much
> difference):
>     - Each burst consists of about 15kB on average in 10-25 datagrams all
>       arriving within a few hundred μs (that's >200Mb/s instantaneously).
>     - The average UDP throughput is about 2.3Mb/s if that means anything
>       with such bursty traffic (was your 1.79Mb/s excluding headers?)

Using skype_codel_cubic_20mbps_160ms.pcap.gz, I took the Cumulative
Bytes column in Wireshark from the last packet (total of frame sizes),
filtered for a destination of 10.72.0.88 (the burstier direction, same
as the download flow), and divided by the time to get bits per second.
I only checked the first couple files though and hadn't noticed that it
differs for two of the files:

skype_codel_cubic_20mbps_160ms.pcap.gz: 1.79 Mbps
skype_dualpi2_c_cubic_20mbps_160ms: 1.78 Mbps
skype_dualpi2_c_prague_20mbps_160ms: 2.26 Mbps
skype_dualpi2_l_prague_20mbps_160ms: 2.32 Mbps
skype_dualpi2_l_cubic_20mbps_160ms: 1.78 Mbps

That probably explains the two numbers we came up with, and the results
could be updated to reflect that.

Thanks,
Pete

>     - These bursts also have irregular sizes but they definitely alternate
>       between large (~80kb) and small (~30kb). The large bursts consist of
>       15-20 packets as well, but over a much longer duration burst (30-
>       40ms, that's only >3Mb/s instantaneously).
>     - The average UDP rate in this direction is 0.88Mb/s

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 31 Aug 2021 12:37:26 +0100

Sebastian,

On 25/08/2021 14:18, Sebastian Moeller wrote:
>> On Aug 25, 2021, at 14:44, Bob Briscoe<[email-masked]>  wrote:
>>
>>>> On Aug 24, 2021, at 17:45, Pete Heist<[email-masked]>
>>>>   wrote:
>>>>

[BB] You're right on both the above. The text is rather abstract, and 
could just say what it really means, with examples.

Overnight, I was thinking that this section could also say sthg like: 
"The initial scope of the L4S experiment is to deploy L4S AQMs at 
bottlenecks and L4S congestion controls at senders. This is expected to 
highlight interactions with the most bursty upstream links and lead 
operators to tune down the burstiness of those links in their network 
that are configurable, or failing that, to have to compromise on the 
delay target of some L4S AQMs. It might also require specific redesign 
work relevant to the most problematic link types. Such knock-on effects 
of initial L4S deployment would all be part of the learning from the L4S 
experiment. The details of changing links other than where L4S is 
deployed are beyond the scope of the present document."

> 	[SM] If I recall correctly, you only applied an L4S AQM for the downstream and that was located upstream of the DSL link, so assuming downstream G.INP was active on that link, your AQM would never…

[BB] You're correct. We would have seen bursts of ACKs releasing a burst 
of data though. Particularly because at that time DCTCP didn't do 
pacing. The first occurrence of ACK bursts I saw was when we started 
working on DOCSIS, which of course releases data discontinuously 
(burstily) itself. This led to data bursts which led us to sort out the 
pacing in the sender (at the time that was in ns3).

> 	[SM] This is not what I am talking about, the interleaver just adds a more or less static delay, it is G.INP and its retransmissions that have the potential to introduce bursts at the line-rate of…

[BB] I did already understand that. But the two are not unrelated 'cos 
more interleaving removes most interference losses, so you trade off 
removing the (intermittent) burst effects of link-layer retransmissions 
by introducing the (continual) delay of interleaving.

> 	[SM] Let me spell it out then, the critique is that the L4S-AQM design/implementation seems to lack the required amount of burst tolerance to deliver the utilization it promises over at least part…

[BB] /If/ links continue to deliver data in order the only solution to 
these bursts is to run time backwards. Otherwise you either get a burst 
in the queue, or you have to underutilize to set aside room for the 
bursts. You are blaming L4S for not being able to run time backwards, as 
if something else can.

Instead of running time backwards, the solution to this problem is to 
remove the need for links to deliver data in order. /Superficially/, 
modern transports like QUIC allow links to deliver data out of order. 
Then, if the app needs data in order, it will still have to self-impose 
the same delay. But if it doesn't, it won't (end-to-end principle).

However, links are too low in the stack to tell which transport is used 
on which packets (if an outer IP header says UDP, it could be 
encapsulating TCP). L4S solves that problem by identifying packets that 
are re-ordering tolerant in the /outer/ IP-ECN field. That creates an 
incremental deployment path to move resequencing solely to end-systems. 
That fundamentally removes the burst delay problem by allowing all the 
PDUs after a gap to be released without delay, then solely the 
retransmitted PDUs pop out late. This has been prototyped and is 
explained in ecn-l4s-id (A.1.8).

Updated link designs are not going to appear any time soon. In the mean 
time, the parameters of any flaky links and the L4S target Q delay can 
be set to levels that trade off between utilization and delay. In these 
cases, of course the L4S performance figures will be compromised. But 
saying this is a design flaw is like complaining that the design of a 
car is fundamentally flawed because on a bumpy dirt track it can't reach 
the top speed promised in the specs.

So please don't accuse the L4S design of being flawed because it doesn't 
solve an impossibility, when it actually offers a way round the 
impossibility in the long term /and/ provides a simple way to be 
configured for any problem links in the short term.

> 	[SM] Not really, that interleaver will only increase the static RTT by a few (dozens) of milli seconds, which should have a negligible effect on utilization, sure latency will be statically increa…

[BB] See above re trade off.

> 	[SM] Well, here is my disconnect, if we expect the world to change for L4S to be useful anyway, why can L4S then not be completely orthogonal to the existing internet and not redefine the CE mark'…

[BB] The world is not having change 'for L4S'. Whichever technology 
enables reduced queuing delay variability, the world would want to 
change to reduce burst delays as well.

DSL-based operators can do this either by i) increasing interleaving, 
which trades off more delay all the time for less burst delay; or by ii) 
removing the root physical causes of the interference that causes the 
bursts.

> 	[SM] Well, the point is even if my ISP does not roll out L4S AQM, given the ambitious scope of L4S my bursty upstream packets might hit an L4S AQM along the path and reduce the utilization through…

[BB] You're generally right (although I disagree on the point about 
bursty behaviour being hard to detect). I am preparing a response about 
Pete's Skype results, which covers malicious addition of burstiness.

Regards

Bob

>>>>>> On Aug 12, 2021, at 17:44, Pete Heist<[email-masked]>
>>>>>>   wrote:
>>>>>>

-- 
________________________________________________________________
Bob Briscoehttp://bobbriscoe.net/