Subject: Re: [tsvwg] Related to "Non-L4S traffic abusing the L-queue" discussion during the interim

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Tue, 1 Mar 2022 12:04:53 +0000

Sebastian,

>> But on a purely logical level, how can a real attack vector for CoDel you might or might not have detected remedy the fact that DualQ essentially established a new fast-lane for unresponsive paced flows that manage to stay below L4S weighted schedulers L4S capacity?

Thank you for confirming that the experiment you/Jonathan did used unresponsive traffic.

There is no remedy required for DualQ, since there is no problem with DualQ. The DualQ is deliberately designed to treat unresponsive flows as similarly as possible to a single queue AQM like PIE. An unresponsive flow will take the BW capacity it sends from the shared link capacity, because responsive flows back-off, and so it will in a DualQ, no matter ECT or not. We've shown that the results for unresponsive traffic below and above the link capacity are very similar for a Single-Q and a DualQ, as intended.

If ok, we can close this discussion here, remove the non-issue from the write-up, and progress with the drafts.

Thanks,
Koen.

-----Original Message-----
From: Sebastian Moeller <[email-masked]> 
Sent: Sunday, February 27, 2022 8:45 PM
To: De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]>
Cc: Black, David <[email-masked]>; Neal Cardwell <[email-masked]>; tsvwg IETF list <[email-masked]>; Bob Briscoe <[email-masked]>
Subject: Re: [tsvwg] Related to "Non-L4S traffic abusing the L-queue" discussion during the interim

Dear Koen,

I have not spent the time to verify that I understand your claims an that I agree with them, (I have a hard time understanding how dropping packets that the head of a queue can be called "taildrop" with a straight face and hence am not willing to give you the benefit of the doubt, sorry).
	But on a purely logical level, how can a real attack vector for CoDel you might or might not have detected remedy the fact that DualQ essentially established a new fast-lane for unresponsive paced flows that manage to stay below L4S weighted schedulers L4S capacity? 

> On Feb 27, 2022, at 18:57, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 
> Hi Sebastian,
> 
> The important plots to check are the throughput plots in figure 1, 
> where PIE and DualPI2 have very similar throughput profiles 
> (confirming a coupled DualQ works like a single Q),

	As I said that is not enough, the whole premise of DualQ is that L4S and traditional traffic can not be mixed, so regressing to single queue behavior indicates that DualQ fails to meet its promise... 

> and figure 3 where PIE and DualPI2 have a queue latency at the target. This can only be achieved if the drop and marks are correctly set by the AQM.

	But is this actually showing more than the known fact that PIE and CoDel have different strictness when interpreting the reference target parameter? To me this looks not really unexpected.

> Only in the CoDel case the ECT(0) non-responding UDP flow takes practically all the link capacity (99.9% when sending unresponsive UDP-ECT(0) at 100Mbps, and really 100% when sending at the higher 140 and 200Mbps rates, probably killing "all" not-ECT traffic; see other mail to Dave Taht).
> These results speak undeniably for themselves.

	That is, as always, a contingent upon interpretation; I am not convinced that your interpretation is necessarily the best objective one available.

> Your hunch and the issue which is now mentioned in the interim slides and the shepherd's writeup is just wrong.

	Excuse me, there is no data indicating that a faked ECT(1) unresponsive-to-CE flow (especially one staying below the L-queues capacity share) will get the same amount of actual drops in the L-queue as it would in a singe-queue CoDel or FIFO. I might not be seeing the forrest for the trees here, so please explain how the existing data shows that my hunch is wrong. But with the additional information about figure 8, I object to your claim, my "hunch [...] is just wrong". The data confirms what I described as my hunch, for cases when when the "illegitimate" flow stays below the L4S AQM's L-queue capacity.
	As I implied before this situation is not as exotic as you might think, given that most access links are << than core links, and L4S apparently is targeted for deployment on core links, no? So no, my flow might not noticeably affect the L-2-C "fairness at a congested L4S AQM but I might still be able to gain an unfair and undeserved throughput advantage over other flows at that link that runs counter to the L4S claims of rough equitable sharing between flows.

> This non-issue should just be removed from the writeup and I suggest you make it an issue for the CoDel/FQ-CoDel RFCs instead. There we detected a "real" attack vector!

	Yeah, you essentially demonstrated that CoDel in itself is not as DOS resilient as it would be desirable. As Dave already mentioned fq_codel partly takes the sting out of this by restricting the fall-out mostly to the hash-bucket housing the offending flow. Also note how the stress-dropper takes care to find the largest queue (which will house the DOS flow) so even batch-emergency dropping does mostly the right thing. A sufficiently motivated attacker obviously will spoof/randomize at least SRC ports or addresses causing more problems for FQ, but my concern really is not so much DOS resistance but simple ways to gain fast-lane access for actual useful data transfer and there address spoofing will make things much harder, than just rate-limiting and fake-ECT(1) marking a flow...
As I see it if I would simply fake ECT(1) mark capacity-seeking non-ECN-honoring flows L4S will grant me a priority lane for data as long as I a) pace packets sufficiently well and b) stay below the L4S bottlenecks L-queue capacity.

> 
> Koen.
> 
> BTW:
>>> the highly interesting drops for Not-ECT UDP (for CoDel) seem to be 
>>> missing (generally the labeling for figure 8 is imprecise, e.g. 
>>> showing Drops ECT(1)-UDP in the first set PIE with ECT(0)-UDP)
> 
> These results are not missing, only the legend is indeed missing

	Which for someone not intimately familiar with the data boils down to the same consequence, thanks for clearing this up.

> the brown colored "Drop Not-ECT" label, but the plots show these 
> values (in brown). Also ECT(1) label in the legend should say ECT(0/1)

	I had figured that out from context ;)

> which depends on whether ECT(0/1) is used in the experiment (as in the labels under the results).

	Ah, okay with that additional information figure 8 now confirms my hunch just perfectly, no drops for unresponsive UDP in L-queue versus drops in C-queue as long as it stays below L-queue capacity. That is in direct conflict with your argument above that my "hunch [...] is just wrong", no?

> 
> -----Original Message-----
> From: Sebastian Moeller <[email-masked]>
> Sent: Friday, February 25, 2022 11:32 PM
> To: De Schepper, Koen (Nokia - BE/Antwerp) 
> <[email-masked]>
> Cc: Black, David <[email-masked]>; Neal Cardwell 
> <[email-masked]>; tsvwg IETF list <[email-masked]>; Bob Briscoe 
> <[email-masked]>
> Subject: Re: [tsvwg] Related to "Non-L4S traffic abusing the L-queue" 
> discussion during the interim
> 
> Hi Koen,
> 
> from your link:
> 
> "Only when non-responsive traffic is below the link capacity it can fully use that share, making the responsive flows share the rest of the capacity (as usual for any AQM on the Internet on a shared queue)." 
> 
> 
> This pretty much confirms what I predicted, except the drop plots look 
> incomplete, the highly interesting drops for Not-ECT UDP (for CoDel) 
> seem to be missing (generally the labeling for figure 8 is imprecise, 
> e.g. showing Drops ECT(1)-UDP in the first set PIE with ECT(0)-UDP). 
> My hunch is that comparing drops ECT(1)-UDP in DualPi2 with the 
> missing drops Not-ECT UDP in Codel, that in the latter we see much 
> more drops than in the former, et voila, exploitable way to gain more 
> throughput by marking traffic ECT(1)... as long as that traffic is 
> reasonably paced and does not exceed the capacity of the L4S AQM (not 
> an unlikely scenario even for capacity seeking traffic, my measly 
> 100Mbps access link will not saturate the BNG uplink of my ISP or the 
> link to the DSLAM)
> 
> Regards
> 	Sebastian
> 
> 
>> On Feb 25, 2022, at 19:30, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
>> 
>> Hi David,
>> 
>> To be sure, we re-did the overload tests recently, confirming the 
>> previous overload results. These results are available at: Overload 
>> results caused by non-responsive UDP traffic for PIE, DualPI2 and 
>> CoDel AQMs | l4steam.github.io
>> 
>> Specifically look at figure 8 at the end which shows that L4S traffic gets marks, up to 100% and appropriate drop if it reaches and exceeds the link capacity.
>> 
>> The test case of Jonathan is approximated by the 70Mbps non-responsive ECT(1) UDP traffic on a 100Mbps link on a DualPI2 (Prague+Cubic) test case. In Jonathan's case it was 40Mbps on a 50Mbps link. We also evaluated in extreme when sending at 100Mbps non-responsive ECT(1) UDP traffic on a 100Mbps link, and even exceeding at 140Mbps and 200Mbps. You will see the results are as if it is on a Single Q PIE AQM. Note also that CoDel which never drops ECT packets, causes actually close to starvation and high tail-drop delay results as shown in figure 1, even with ECT(0). So I guess all the concerns about FQ_CoDel and tunnels/Hash-collisions are equally severe and not related to L4S alone (can just be exploited by ECT(0) traffic today already!!).
>> 
>> Koen.
>> 
>> From: Black, David <[email-masked]>
>> Sent: Friday, February 25, 2022 7:04 PM
>> To: Neal Cardwell <[email-masked]>
>> Cc: De Schepper, Koen (Nokia - BE/Antwerp) 
>> <[email-masked]>; tsvwg IETF list 
>> <[email-masked]>; Jonathan Morton <[email-masked]>; Bob 
>> Briscoe <[email-masked]>; Black, David <[email-masked]>
>> Subject: RE: [tsvwg] Related to "Non-L4S traffic abusing the L-queue" 
>> discussion during the interim
>> 
>> Hi Neal,
>> 
>> So, I saw that explanation - could someone check the "running code" to make sure that the coupling and marking occur even when the L queue is always empty?
>> 
>> Thanks, --David
>> 
>> From: Neal Cardwell <[email-masked]>
>> Sent: Friday, February 25, 2022 12:58 PM
>> To: Black, David
>> Cc: De Schepper, Koen (Nokia - BE/Antwerp); tsvwg IETF list; Jonathan 
>> Morton; Bob Briscoe
>> Subject: Re: [tsvwg] Related to "Non-L4S traffic abusing the L-queue" 
>> discussion during the interim
>> 
>> [EXTERNAL EMAIL]
>> 
>> 
>> 
>> On Fri, Feb 25, 2022 at 11:56 AM Black, David <[email-masked]> wrote:
>> Koen,
>> 
>> I'll observe that "traffic that is not responding at all to CE marks" 
>> is not necessary to achieve the reported results if the experimental 
>> setup "prevents the L queue from seeing any
>> 
>> need to apply congestion signals, because it is always empty" as there would be no CE marks for the traffic in the L queue to respond to.
>> 
>> I think the key part here is "if". :-) The assertion "prevents the L queue from seeing any need to apply congestion signals, because it is always empty" is from:
>>  https://sce.dnsmgr.net/downloads/L4S-WGLC2-objection-details.pdf
>> [sce.dnsmgr.net] That assertion is inconsistent with the functioning of the Dual-Q algorithm, as described in:
>>  https://www.ietf.org/id/draft-ietf-tsvwg-aqm-dualq-coupled-21.html
>> [ietf.org]
>> 
>> As Bob noted: "in the scenario shown, although the L queue is indeed always empty, it will see a high level of congestion signals (~10% in this case) via the coupling."
>> Here's Bob's e-mail for more context/details:
>> 
>> https://mailarchive.ietf.org/arch/msg/tsvwg/joFr3sfOrxxkYhWdYrO2rLlCN
>> U
>> w/ [mailarchive.ietf.org]
>> 
>> thanks,
>> neal
>> 
>> 
>> 
>> Please give that further consideration.
>> 
>> Thanks, --David (as an individual)
>> 
>> From: tsvwg <[email-masked]> On Behalf Of De Schepper, Koen 
>> (Nokia - BE/Antwerp)
>> Sent: Friday, February 25, 2022 4:29 AM
>> To: tsvwg IETF list; Jonathan Morton
>> Subject: Re: [tsvwg] Related to "Non-L4S traffic abusing the L-queue" 
>> discussion during the interim
>> 
>> [EXTERNAL EMAIL]
>> 
>> Hi Jonathan,
>> 
>> Can you confirm that this test is done with "Cubic" traffic that is not responding at all to CE marks? So it is just like any other non-responding traffic (like UDP CBR). We don't see any other way to explain your results. 
>> 
>> If so, we can/should remove this "issue" from the shepherd's write-up, as such unresponsive flows will get the same throughput on any single-Q bottleneck with or without AQM (taildrop/PI2/PIE/CoDel/STEP/RED/...) with a latency that matches the AQM strategy.
>> 
>> Koen.
>> 
>> 
>> From: tsvwg <[email-masked]> On Behalf Of De Schepper, Koen 
>> (Nokia - BE/Antwerp)
>> Sent: Thursday, February 17, 2022 7:01 PM
>> To: tsvwg IETF list <[email-masked]>; Jonathan Morton 
>> <[email-masked]>
>> Subject: [tsvwg] Related to "Non-L4S traffic abusing the L-queue" 
>> discussion during the interim
>> 
>> Hi Jonathan,
>> 
>> It seems that the following open issue identified by the chairs:
>> 
>> Non-L4S traffic abusing the L-queue
>> * 'DualQ gives a large throughput bonus to L queue traffic, ie. a "fast lane"'
>> * Is this a matter specific for DualQ that can be left for experimentation?
>> 
>> is based on the following experiment you performed:
>> 
>>>            simple two-flow competition test on a standard dumbbell 
>>> topology,
>> 
>>>            with the bottleneck running a DualQ qdisc into a 50Mbps shaper.
>> 
>>>            Both flows were configured to use CUBIC congestion 
>>> control with
>> 
>>>            ECN negotiated, but one was additionally tweaked to set
>>> ECT(1)
>> 
>>>            instead of ECT(0) on all data segments, and to pace its 
>>> output at
>> 
>>>            40Mbps. This latter measure prevents the L queue from 
>>> seeing any
>> 
>>>            need to apply congestion signals, because it is always 
>>> empty.  These
>> 
>>>            tweaks allowed that flow to use 80% of the link capacity, 
>>> gaining a
>> 
>>>            fourfold advantage over its competitor,
>> 
>> 
>> If there is capacity seeking traffic in the Classic queue, then it is even desired that the L4S queue does not add extra marks. The L4S marks should come only from the Classic coupling.
>> Before diving into details, can you first explain why in your experiment the coupling from the Classic Q has no effect on your paced and ECT(1) labeled Cubic flow?
>> 
>> I would expect that this ECT(1) labeled Cubic flow would get even less throughput than the Classic Cubic flow, as the first gets the doubled coupled CE marking probability (eg 2*10% = 20%) for L4S flows instead of the squared CE marking probability (10%^2 = 1%) which ECT(0) traffic would get.
>> 
>> Thanks,
>> Koen.
>

---

From: Sebastian Moeller <[email-masked]>
Date: Tue, 1 Mar 2022 13:23:53 +0100

Dear Koen,

> On Mar 1, 2022, at 13:04, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 
> Sebastian,
> 

	[SM] Ah, please let me clarify, I did offer a thought experiment here supported by the data you presented in the overload write-up. As I am about to write to Bob, a flow that does not respond to ECN but to drop is not such a unicorn as you might think... I let Jonathan speak for his real experiments but want to iterate that I was not involved on those and hence can not give authoritative confirmations how that experiment was conducted. 

> There is no remedy required for DualQ, since there is no problem with DualQ.

	[SM] That is a position you can certainly take, I do however object and see a problem.

> The DualQ is deliberately designed to treat unresponsive flows as similarly as possible to a single queue AQM like PIE.

	[SM] Well, what exactly is the reference here, a FIFO (the current internet's default) or any specific AQM, then please define the exact version. It is hard to confirm a "similarly as possible" claim without a stringent description of the reference.

> An unresponsive flow will take the BW capacity it sends from the shared link capacity, because responsive flows back-off, and so it will in a DualQ, no matter ECT or not. We've shown that the results for unresponsive traffic below and above the link capacity are very similar for a Single-Q and a DualQ, as intended.

	[SM] Well if that is how you intended it, my objection is not about the implementation, but about the design instead. With your weighted scheduler you effectively build a fast-lane for any flow that is a) reasonably paced and b) stays below the AQM bottleneck's L-queue capacity and c) ignores CE-marks (given tat the TOS bye is manipulated on-route you better not defer your safety into the fact that abusing ECN marks is "verboten").

> If ok, we can close this discussion here, remove the non-issue from the write-up, and progress with the drafts.

	[SM] I would ask you to take Jonathan's objection up with Jonathan, before you declare it resolved. My objection focuses on something you already admitted as a problem:

I posit that this situation is not a rare unicorn (IFF the L4S goals of getting L4S AQM deployed universally should ever materialize). Since a lot of design decisions in L4S are justified by designing for universality, it seems only fair to also consider issues arising from universal deployment.

Regards
	Sebastian

---

From: Sebastian Moeller <[email-masked]>
Date: Tue, 1 Mar 2022 13:36:28 +0100

Hi Bob,

> On Feb 28, 2022, at 14:17, Bob Briscoe <[email-masked]> wrote:
> 
> Sebastian,
> 
> On 27/02/2022 19:44, Sebastian Moeller wrote:
>> Dear Koen,
>> 
> [BB] Because Jonathan's claim in his write-up that this is a new fast-lane is false.
> Also, in the heading, Jonathan says "This bonus is easily exploited by unscrupulous senders without disabling congestion control."
> But clearly, the flow's congestion control is not responding to the ECN markings coupled across from the C queue. So this statement must also be false.

	[SM] It can be argued whether a flow responsive to drops but not CE has congestion control disabled though. Given the historic use of the TOS byte with potential value changes by all intermediate hops... so ECT(1) might actually be set by some other party then the end-points, and if they did not negotiate ECN on a flow they are not really obliged to honor CE-marks either.

> Reasoning:
> By deliberate design, the DualQ treats an unresponsive flow the same as a FIFO AQM (like PIE) would.

	[SM] That is imprecise, CE-unresponsive traffic in the L-queue will only see actual drops if it squeezes the C-queue below its weighted scheduler share, in a FIFO an unresponsive flow will acquire drops (more or less) commensurate with its capacity share. This is clearly not the identical treatment, and you acknowledge that "conditional fast lane" in the overload document:

This raises the question how likely this situation can/will arise. From my own limited experience the typical congestion prone locations of a typical internet path are:
(1) at (both of) the edges, home networks typically were much narrower than ISP internal networks or typical data centers housing servers(this can include one's own access link, or the uplink of the next aggregation, like DOCSIS segment or OLT/DSLAM uplink)
(2) at transition points between networks (peering, transit, ...)

L4S as currently written clearly has ambitions for deployment in (1) and (2), and that can result in the situation, that the access link (1) limits a capacity seeking flow's max rate such that is stays below a type (2) L-queue capacity (this flow if limited by the access link's capacity will also mostly appear "paced"), but above its promised L4S share (rough equality between flows but allowing for RTT-bias, if I recall correctly). IMHO that is a novel fast-lane.

> If Jonathan ran a similar experiment but classified that unresponsive flow into the C queue, whether it was ECN-capable or not, it would have also got 40Mb/s.

	[SM] Yes, assuming with "unresponsive" we mean does not respond to any congestion signal... but that gets us in DOS territory, because it is somewhat hard to do useful data transfer when arbitrary pieces can go missing. Sure one could design a protocol such that the required retransmissions are performed out of line so that an unrelenting main flow can just keep on sending, but that would be a lot of work for little gain. (fq_codel can be configured to ignore ECN, codel ignores ECN by default)

> So, there is no new fast-lane.  

	[SM] But only if we ignore the unscrupulous ignore-CE (because I can without much cost) but respect drop (because retransmissions otherwise get complicated) flow. Really the issue here is CE-marks versus drops and the respective cost of ignoring those for individual flows.

> There is no new throughput bonus. This is just the same 40Mb/s that any unresponsive 40Mb/s flow has always got in a single queue (AQM or not, ECN or not). 
> 
> This is the oft-stated deliberate design intent of the DualQ. Per-flow rate policing (and/or per-flow latency policing) can be added as an optional extra, but only if the operator wants it. The vast majority of the Internet currently works fine without per-flow policing.

	[SM] Does it? Why then propose L4S if everything is fine in the Internet already? 

> So far, I am solely focusing on Jonathan's scenario (40Mb/s in a 50Mb/s link). Koen's experiments show CoDel falls down when unresponsive flow(s) send more than the link capacity, but I want to take this one scenario at a time.

	[SM] But the 40/50 flow is exactly the situation where the L-queue can demonstrably be abused to gain more throughput then promised in the L4S drafts by flows that do honor drops but ignore CE-marks... 

> Bob
>>> The important plots to check are the throughput plots in figure 1, where PIE and DualPI2 have very similar throughput profiles (confirming a coupled DualQ works like a single Q),
>>> 
>>>>> the highly interesting drops for Not-ECT UDP (for CoDel) seem to be missing (generally the labeling for figure 8 is imprecise, e.g. showing Drops ECT(1)-UDP in the first set PIE with ECT(0)-UDP)
>>>>> 
>>> the brown colored "Drop Not-ECT" label, but the plots show these values (in brown). Also ECT(1) label in the legend should say ECT(0/1)
>>> 
>>> Cc: Black, David 
>>> <[email-masked]>; Neal Cardwell <[email-masked]>; tsvwg IETF list <[email-masked]>; Bob Briscoe <[email-masked]>
>>> 
>>> This pretty much confirms what I predicted, except the drop plots look incomplete, the highly interesting drops for Not-ECT UDP (for CoDel) seem to be missing (generally the labeling for figure 8 is imprecise, e.g. showing Drops ECT(1)-UDP in the first set PIE with ECT(0)-UDP). My hunch is that comparing drops ECT(1)-UDP in DualPi2 with the missing drops Not-ECT UDP in Codel, that in the latter we see much more drops than in the former, et voila, exploitable way to gain more throughput by marking traffic ECT(1)... as long as that traffic is reasonably paced and does not exceed the capacity of the L4S AQM (not an unlikely scenario even for capacity seeking traffic, my measly 100Mbps access link will not saturate the BNG uplink of my ISP or the link to the DSLAM)
>>> 
>>> Regards
>>> 	Sebastian
>>> 
>>> 
>>> 
>>>> ; tsvwg IETF list 
>>>> <[email-masked]>; Jonathan Morton <[email-masked]>
>>>> ; Bob Briscoe 
>>>> 
>>>> <[email-masked]>; Black, David <[email-masked]>
>>>> 
>>>> On Fri, Feb 25, 2022 at 11:56 AM Black, David 
>>>> <[email-masked]>
>>>>  wrote:
>>>> Koen,
>>>> 
>>>> https://mailarchive.ietf.org/arch/msg/tsvwg/joFr3sfOrxxkYhWdYrO2rLlCNU
>>>> 
>>>> From: tsvwg 
>>>> <[email-masked]>
>>>>  On Behalf Of De Schepper, Koen 
>>>> (Nokia - BE/Antwerp)
>>>> From: tsvwg 
>>>> <[email-masked]>
>>>>  On Behalf Of De Schepper, Koen 
>>>> (Nokia - BE/Antwerp)
>>>> To: tsvwg IETF list 
>>>> <[email-masked]>
>>>> ; Jonathan Morton 
>>>>>            tweaks allowed that flow to use 80% of the link 
>>>>> capacity, gaining a
>>>>> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               
> http://bobbriscoe.net/

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 2 Mar 2022 14:34:48 +0000

>>	[SM] That is imprecise, CE-unresponsive traffic in the L-queue will only see actual drops if it squeezes the C-queue below its weighted scheduler share, in a FIFO an unresponsive flow will acquire drops (more or less) commensurate with its capacity share. This is clearly not the identical treatment, 

[K] Your thought experiment is incomplete and for that reason is not matching the results. In PIE, PI2 and DualPI2, an ECT flow (both for L4S or Classic ECN) will only see drop if the PIE/PI2 Classic drop probability is above the drop-ECN threshold. Even for DualPI2 the rate share is independent from the scheduler, only depends on the senders' response to drop, as the drop is equally distributed over all flows.
If there is sufficient congestion in the Classic Q, it can already raise the probability above this threshold, without the L4S share reaching the 90% (for instance if there is also Classic non-responsive traffic or a high number of Classic flows). If there isn't enough Classic traffic, this Classic traffic will even be protected by the scheduler as long as the L4S (non-responsive) traffic is not causing more congestion in the L4S queue as there is in the Classic queue.

>> and you acknowledge that "conditional fast lane" in the overload document:

[K] I don't see how I did. I mentioned the total "link capacity", not the L4S WRR share. I hope your problem is not that non-responsive traffic below the link capacity can have zero queue size? If there is no other traffic and it sends at 80% of the link capacity it will also have 0 queueing delay, and if it is send to a single-Q L4S STEP AQM with other L4S capacity seeking traffic it will get 1ms latency too, and still will not respond to the CE marks, so uses 80% of that capacity too. Just like any non-responsive traffic in any single-Q AQM.

Hope this clarifies, and explains why your thought experiments are incomplete.

Koen.

-----Original Message-----
From: Sebastian Moeller <[email-masked]> 
Sent: Tuesday, March 1, 2022 1:36 PM
To: Bob Briscoe <[email-masked]>
Cc: De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]>; Black, David <[email-masked]>; Neal Cardwell <[email-masked]>; tsvwg IETF list <[email-masked]>
Subject: Re: [tsvwg] Related to "Non-L4S traffic abusing the L-queue" discussion during the interim

Hi Bob,

> 
> 
> Bob
> 
>> 
>>> On Feb 27, 2022, at 18:57, De Schepper, Koen (Nokia - BE/Antwerp) 
>>>>> the highly interesting drops for Not-ECT UDP (for CoDel) seem to 
>>>>> be missing (generally the labeling for figure 8 is imprecise, e.g. 
>>> <[email-masked]>; Neal Cardwell <[email-masked]>; tsvwg 
>>> IETF list <[email-masked]>; Bob Briscoe <[email-masked]>
>>> 
>>> This pretty much confirms what I predicted, except the drop plots 
>>> look incomplete, the highly interesting drops for Not-ECT UDP (for 
>>> CoDel) seem to be missing (generally the labeling for figure 8 is 
>>> imprecise, e.g. showing Drops ECT(1)-UDP in the first set PIE with 
>>> ECT(0)-UDP). My hunch is that comparing drops ECT(1)-UDP in DualPi2 
>>> with the missing drops Not-ECT UDP in Codel, that in the latter we 
>>> see much more drops than in the former, et voila, exploitable way to 
>>> gain more throughput by marking traffic ECT(1)... as long as that 
>>> traffic is reasonably paced and does not exceed the capacity of the 
>>> L4S AQM (not an unlikely scenario even for capacity seeking traffic, 
>>> my measly 100Mbps access link will not saturate the BNG uplink of my 
>>> ISP or the link to the DSLAM)
>>> 
>>> Regards
>>> 	Sebastian
>>> 
>>> 
>>> 
>>>> On Feb 25, 2022, at 19:30, De Schepper, Koen (Nokia - BE/Antwerp) 
>>>> <[email-masked]>; Jonathan Morton <[email-masked]> ; Bob 
>>>> Briscoe
>>>> 
>>>> Cc: De Schepper, Koen (Nokia - BE/Antwerp); tsvwg IETF list; 
>>>> Jonathan Morton; Bob Briscoe
>>>> is not necessary to achieve the reported results if the 
>>>> experimental setup "prevents the L queue from seeing any
>>>>

---

From: Luca Muscariello <[email-masked]>
Date: Thu, 3 Mar 2022 15:52:31 +0100

Bob

Thanks for the clarification which confirms my understanding of the
dynamics of the prioritization.
As a side note, I would assume that we do not know the initial conditions
of the queueing system and we may have a mix of traffic where TCP may be in
different phases. Your analysis is one specific initial condition. But
whatever is this mix, the prioritization will be highly predictable for non
responsive traffic and work as an effective head of the line priority which
is not equivalent to a FIFO AQM.

Luca

On Mon, Feb 28, 2022 at 6:51 PM Bob Briscoe <[email-masked]> wrote:

> Luca,
>
> On 28/02/2022 14:54, Luca Muscariello wrote:
>
>
>
> On Mon, Feb 28, 2022 at 3:35 PM Bob Briscoe <[email-masked]> wrote:
>
>> Luca, Koen's just answered your first email. This is in response to your
>> second on this thread. See [BB]
>>
>> On 28/02/2022 13:32, Luca Muscariello wrote:
>>
>>
>>
>> On Mon, Feb 28, 2022 at 2:18 PM Bob Briscoe <[email-masked]> wrote:
>>
>>> Sebastian,
>>>
>>> Also, in the heading, Jonathan says "This bonus is easily exploited by
>>> unscrupulous senders *without* disabling congestion control."
>>> But clearly, the flow's congestion control is not responding to the ECN
>>> markings coupled across from the C queue. So this statement must also be
>>> false.
>>>
>>> Reasoning:
>>> By deliberate design, the DualQ treats an unresponsive flow the same as
>>> a FIFO AQM (like PIE) would. If Jonathan ran a similar experiment but
>>> classified that unresponsive flow into the C queue, whether it was
>>> ECN-capable or not, it would have also got 40Mb/s.
>>>
>>
>> Beyond the specific experiment, I do not think this is totally accurate.
>> As a stationary regime
>> a FIFO AQM and DualQ may converge to the same bandwidth sharing condition
>> wrt to unresponsive
>> traffic but in the FIFO AQM this would happen through a fight against,
>> say TCP. So the unresponsive flow would get packet losses during that
>> fight. In dualQ the unresponsive flow would get not a single loss to my
>> understanding, thanks to the prioritization behavior either via strict
>> priority or WFQ scheduling cycles.
>>
>>
>> [BB] The DualQ introduces drop of ECT packets when AQM marking exceeds an
>> overload threshold (25% mark or drop in the C queue). It applies drop of
>> ECT packets to /both/ queues with the same probability (IOW, the coupling
>> for drop is equal, whereas for ECN marking it uses the square
>> relationship).
>>
>> See the 'else' at line 8b after the 'if' at line 4b in Fig 7 and the
>> associated explanation in the DualQ draft.
>>
>> https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-aqm-dualq-coupled-21#appendix-A.2
>>
>> If you meant something different by "beyond the specific experiment"
>> please say. I've guessed you meant that they are still both ECN-capable
>> flows but, due to transient events, the AQM marking probability temporarily
>> becomes high enough for the AQM to introduce drop of ECT packets. I think
>> you thought this only happens in the C queue, but the DualQ applies drop to
>> both queues, as just explained above (and recently discussed on this list
>> regarding the MUSTs/SHOULDs around this).
>>
>
> I was thinking about unresponsive traffic.
> In maybe better English: In general, in presence of unresponsive traffic a
> FIFO AQM and dualQ
> may converge to the same stationary bandwidth sharing, i.e. the
> unresponsive traffic gets all input rate while TCP gets the residual
> capacity. Hope this is more explicit.
>
> My remark was that in a FIFO AQM, unresponsive traffic may need to
> generate quite a lot of queuing to make TCP backoff before gaining all the
> share of bandwidth. In the case of dualQ, unresponsive traffic would get
> that share immediately as it is obtained by packet prioritization and not
> by forcing TCP to backoff.
>
>
> [BB] You're right that the dynamics will differ, but...
> The longer it takes the TCP to back off, the /higher/ its share of
> capacity until it does. The priority scheduler serves more of the L packets
> earlier, but it doesn't change capacity shares (how many of each type of
> packet); only the sources can do that. At least, not unless there's unequal
> drop, but there isn't. Drop can only occur if the DualQ enters overload
> mode, and then the DualQ ensures the drop probability is equal in both
> queues, as I've explained. That's the same as a FIFO AQM.
>

>
> Walking through this step-by-step:
>
> ==In the first RTT before any response, ==
> 1. The queue growth in the single FIFO queue case will be the same as the
> growth in the sum of the two Qs in the DualQ case. However, the C queue
> will grow more and the L queue less, because the L queue has higher
> scheduling priority.
> 2. So as the TCP traffic leaves the FIFO AQM it is more mixed-in with the
> unresponsive traffic. Whereas, in the DualQ, more of the TCP traffic leaves
> later.
>
> These effects will lead to two opposing outcomes:
>     a. #1 causes the Classic AQM to increase its mark/drop probability
> faster than a FIFO AQM would
>     b. #2 causes the congestion signals on the Classic TCP packets to
> leave the queue later than they would have from a FIFO AQM, so TCP will
> receive them later.
>
> It's not clear to me which will be the strongest effect. But it's not
> appropriate to say "the unresponsive traffic gets that share immediately"
> because the capacity share depends on what the sources send, and that
> hasn't changed yet.
>
> ==In subsequent rounds==
> Classic TCPs only respond once per RTT and by the same reduction whatever
> the degree of queuing. So, if TCP responds slightly later in the DualQ
> case, it will remain slower to respond, and continue to hold a /higher/
> share of the capacity for longer.
>
> One would have to test the dynamics, but altho' I think the capacity share
> advantage might be the opposite of what you say, the difference will only
> last 2 or 3 rounds, so it's trivial either way.
>
>
> Bob
>
>
>
>>
>> Classic AQMs (like PIE or even RED) don't start dropping ECT packets
>> based on the instantaneous queue - they smooth their response. The DualQ is
>> no different. It determines whether it starts introducing drop to ECT
>> packets (in both queues equally) based on the probability from the Base
>> AQM, which is already smoothed.
>>
>>
>>
>> Bob
>>
>>
>>
>>
>>
>>>
>>> So, there is no new fast-lane.  There is no new throughput bonus. This
>>> is just the same 40Mb/s that any unresponsive 40Mb/s flow has always got in
>>> a single queue (AQM or not, ECN or not).
>>>
>>> This is the oft-stated deliberate design intent of the DualQ. Per-flow
>>> rate policing (and/or per-flow latency policing) can be added as an
>>> optional extra, but only if the operator wants it. The vast majority of the
>>> Internet currently works fine without per-flow policing.
>>>
>>>
>>> So far, I am solely focusing on Jonathan's scenario (40Mb/s in a 50Mb/s
>>> link). Koen's experiments show CoDel falls down when unresponsive flow(s)
>>> send more than the link capacity, but I want to take this one scenario at a
>>> time.
>>>
>>>
>>> Bob
>>>
>>> On Feb 27, 2022, at 18:57, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> <[email-masked]> wrote:
>>>
>>> Hi Sebastian,
>>>
>>> From: Sebastian Moeller <[email-masked]> <[email-masked]>
>>> To: De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> <[email-masked]>
>>> Cc: Black, David <[email-masked]> <[email-masked]>; Neal Cardwell <[email-masked]> <[email-masked]>; tsvwg IETF list <[email-masked]> <[email-masked]>; Bob Briscoe <[email-masked]> <[email-masked]>
>>> On Feb 25, 2022, at 19:30, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> <[email-masked]> wrote:
>>>
>>> Hi David,
>>>
>>> From: Black, David <[email-masked]> <[email-masked]>
>>> To: Neal Cardwell <[email-masked]> <[email-masked]>
>>> Cc: De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> <[email-masked]>; tsvwg IETF list <[email-masked]> <[email-masked]>; Jonathan Morton <[email-masked]> <[email-masked]>; Bob Briscoe <[email-masked]> <[email-masked]>; Black, David <[email-masked]> <[email-masked]>
>>> From: Neal Cardwell <[email-masked]> <[email-masked]>
>>> On Fri, Feb 25, 2022 at 11:56 AM Black, David <[email-masked]> <[email-masked]> wrote:
>>> Koen,
>>>
>>> From: tsvwg <[email-masked]> <[email-masked]> On Behalf Of De Schepper, Koen
>>> (Nokia - BE/Antwerp)
>>> From: tsvwg <[email-masked]> <[email-masked]> On Behalf Of De Schepper, Koen
>>> (Nokia - BE/Antwerp)
>>> To: tsvwg IETF list <[email-masked]> <[email-masked]>; Jonathan Morton <[email-masked]> <[email-masked]>
>>> Bob Briscoe                               http://bobbriscoe.net/
>>>
>>>
>> --
>> Bob Briscoe                               http://bobbriscoe.net/
>>
>>
> --
> Bob Briscoe                               http://bobbriscoe.net/
>
>