Subject: Re: [tsvwg] L4S status: #17 Interaction w/ FQ AQMs

From: Wesley Eddy <[email-masked]>
Date: Mon, 9 Sep 2019 15:01:11 -0400

Since this thread seems to have dwindled, I just wanted to summarize 
that it looks to me like we have agreement that testing as described is 
needed.

I updated the issue tracker with a comment saying as much and pointing 
back to this thread in the archive for reference.

Is anyone planning to perform this testing in a rough timeframe they 
might want to share?

---

From: Wesley Eddy <[email-masked]>
Date: Mon, 9 Sep 2019 15:08:51 -0400

On 8/23/2019 10:01 AM, Scharf, Michael wrote:
>
> Hi Wes,
>
> I’d like to add a smaller item that is mostly editorial and can 
> hopefully be sorted just out by re-wording, albeit it may require a 
> careful analysis of all documents.
>
> As already noted in 
> https://mailarchive.ietf.org/arch/msg/tsvwg/zZkYZKF-hDvWO3I5MudwpNkKyHY 
> , I object to the terms „traditional TCP“ and also „classic TCP“ or 
> „legacy“ TCP when referring to a TCP implementation according to IETF 
> standards-track RFCs.
>
>  ...
>

Hello Michael, thanks for noting this.  I've added a ticket in the issue 
tracker pointing back to this thread.

---

From: Pete Heist <[email-masked]>
Date: Wed, 11 Sep 2019 09:52:03 +0200

> On Sep 9, 2019, at 9:01 PM, Wesley Eddy <[email-masked]> wrote:
> 
> Since this thread seems to have dwindled, I just wanted to summarize that it looks to me like we have agreement that testing as described is needed.
> 
> I updated the issue tracker with a comment saying as much and pointing back to this thread in the archive for reference.
> 
> Is anyone planning to perform this testing in a rough timeframe they might want to share?

Hi Wesley, I’ll share results from relevant testing in the next day or two...

Regards,
Pete

---

From: Greg White <[email-masked]>
Date: Wed, 11 Sep 2019 17:16:48 +0000

I'm planning on doing testing as well, but it will be more than a day or two to get it done.  Rough timeframe would be 2-3 weeks from now.

-Greg

    
    
    Regards,
    Pete

---

From: Pete Heist <[email-masked]>
Date: Mon, 16 Sep 2019 07:58:08 +0200

> On Sep 11, 2019, at 9:52 AM, Pete Heist <[email-masked]> wrote:
> 

Just adding to this thread that the interaction with CoDel is shown in a few different ways in the SCE-L4S “bakeoff” tests in scenarios 2, 3, 5 and 6:

https://github.com/heistp/sce-l4s-bakeoff/

---

From: Sebastian Moeller <[email-masked]>
Date: Mon, 16 Sep 2019 09:27:55 +0200

Hi Pete,

> On Sep 16, 2019, at 07:58, Pete Heist <[email-masked]> wrote:
> 
> 

Excellent, thanks.

So it looks like my fear that L4S-flows would completely crowd out standard compliant flows in the post bottleneck FQ-AQM scenario was unfounded! The intra-L4S latency increases in that situation, however, look terrible, but since they seem restricted to all L4S flows I can certainly live with that. I wonder whether the L4S crowd will consider that acceptable or whether they will work on detecting standard compliant ECN-AQMs and falling back to standard compliantt behavior in that case now? This was/is in the potential todo list for tcp prague anyway, if I understand correctly.

Best Regards
	Sebastian

---

From: Pete Heist <[email-masked]>
Date: Mon, 16 Sep 2019 10:05:39 +0200

> On Sep 16, 2019, at 9:27 AM, Sebastian Moeller <[email-masked]> wrote:
> 
> Hi Pete,
> 
> Excellent, thanks.
> 
> So it looks like my fear that L4S-flows would completely crowd out standard compliant flows in the post bottleneck FQ-AQM scenario was unfounded!

If bottleneck 3 were a single queue (‘flows 1’ parameter to fq_codel), that may lead to unfairness, similar to what happens in two-flow competition in Scenario 3:
https://www.heistp.net/downloads/sce-l4s-bakeoff/bakeoff-2019-09-13T045427-r1/l4s-s3-2/batch-l4s-s3-2-cubic-vs-prague-50Mbit-10ms_fixed.png

Note also that in scenario 5-2 for Cubic vs Prague:
https://www.heistp.net/downloads/sce-l4s-bakeoff/bakeoff-2019-09-13T045427-r1/l4s-s5-2/batch-l4s-s5-2-cubic-vs-prague-50Mbit-80ms_fixed.png

the Prague flow exits slow start early, which is at least party influenced by Linux’s default pacing parameters. The early exit from slow start prevents the latency spike on startup of the Prague flow from occurring. On a run of the same test with ss_ratio=100 and ca_ratio=40, which we typically test with for SCE, the early exit from slow start doesn’t happen and results on startup of the Prague flow look different:

https://www.heistp.net/downloads/sce-l4s-bakeoff/bakeoff-2019-09-12T021200-pacing-100/l4s-s5-2/batch-l4s-s5-2-cubic-vs-prague-50Mbit-80ms_fixed.png

---

From: Jonathan Morton <[email-masked]>
Date: Mon, 16 Sep 2019 11:06:17 +0300

> On 16 Sep, 2019, at 10:27 am, Sebastian Moeller <[email-masked]> wrote:
> 
> So it looks like my fear that L4S-flows would completely crowd out standard compliant flows in the post bottleneck FQ-AQM scenario was unfounded! The intra-L4S latency increases in that situation, however, look terrible, but since they seem restricted to all L4S flows I can certainly live with that.

Except that's not what the data shows.  In Scenarios 5 & 6, the purple traces indicate queuing in the dumb FIFO (which affects *all* traffic), and that is much more serious with L4S traffic in play than with only RFC-3168 compliant (including SCE) traffic.  For direct comparison:

https://www.heistp.net/downloads/sce-l4s-bakeoff/bakeoff-2019-09-13T045427-r1/sce-s6-2/batch-sce-s6-2-cubic-vs-cubic-50Mbit-10ms_fixed.png

https://www.heistp.net/downloads/sce-l4s-bakeoff/bakeoff-2019-09-13T045427-r1/l4s-s6-2/batch-l4s-s6-2-prague-vs-prague-50Mbit-10ms_fixed.png

Note particularly how long the purple latency spikes last.  These are measured by parallel ICMP probes, and represent what a competing, sparse, latency-sensitive flow would see.

CUBIC and Reno-SCE almost avoid latency spikes altogether.  When Reno-SCE starts up, there's a very brief spike that's definitely less than a second wide.  For each Prague startup, the spike is at least one second and up to 5 seconds wide at its base, and much taller (the first one literally disappears off the top of the graph, which is at 110ms).  Given the baseline RTT of 10ms, this is clearly unacceptable.

Note also that in the above comparison, Reno-SCE is running into a completely non-SCE network.  In particular the link FIFO and the FQ-AQM that form the actual bottlenecks are not providing SCE signals, only RFC-3168 CE marks from the FQ-AQM and not the FIFO.  This shows the advantage of maintaining full backwards compatibility.

 - Jonathan Morton

---

From: Sebastian Moeller <[email-masked]>
Date: Mon, 16 Sep 2019 10:12:10 +0200

Hi Jonathan, hi Pete

right you are. I had ignored the "transient spike" (even though at multiple seconds duration it can wreck havoc on any latency sensitive traffic on that AQM). I really only wanted to conceed that my intuition that L4S flows will completely incapacitate a post-bottleneck FQ-AQM. And obviously did not look closely enough at the ICMP traces. Thanks both of you for the great data source and the support in interpretation!

Best Regards & many thanks
	Sebastian

> On Sep 16, 2019, at 10:06, Jonathan Morton <[email-masked]> wrote:
> 

	+1; this backwards compatibility helps SCE flows ot behave like good neicens should do instead of simply bulldozing over everything. ;)