Subject: [iccrg] L4S Issue #26 RFC3168 ECN Fallback: Results site up (finally)

From: Bob Briscoe <[email-masked]>
Date: Fri, 24 Apr 2020 19:05:37 +0100

tsvwg, iccrg,

I'd like to announce a new (and still growing) large set of results from 
testing the Classic ECN AQM Fallback algorithm in TCP Prague.

         http://folk.uio.no/asadsa/ecn-fbk/

This is linked from the usual L4S landing page, so you don't need to 
keep the URL: https://riteproject.eu/dctth/#eval
(indeed, don't store it, 'cos it will be moving before the Autumn)

There are currently 3000 experiments with different combinations of 
AQMs, traffic, and parameter options. Underneath that are 42,000 
time-series plots so far, tracking all the salient variables. We're now 
adding more experiments continually.

However, you don't have to wade through all those plots. We've created 
top level summary plots and clickable heatmaps which not only succinctly 
visualize the success ratings of all the results so far (by the colour 
of each tile), but they also allow you to drill down into any results 
that look interesting or suspect. For those, select the Evaluation 
Results menu once you're on any page.

If you think we should do other tests, let us know. We have a backlog 
already, but we'll get there.

        Tentative Conclusions so far

The tentative conclusion so far is that the algorithm reliably ensures 
that the L4S reference congestion control (TCP Prague) does not 
out-compete TCP Cubic when they share a bottleneck that supports a 
Classic ECN AQM, over a wide range of testbed scenarios.

The figure of merit used is the ratio that compares a Cubic flow's rate 
when competing with TCP Prague versus when competing with otherwise 
identical Cubic-ECN flows. The worst ratio seen so far in all the test 
scenarios on this site is 48%, which was for 9 Prague vs 1 Cubic-ECN 
flow. The worst ratio seen so far in 1v1 flow scenarios is 70%. I 
believe both are well within the bounds of what is considered 
'TCP-Friendly'. The average rate ratio so far is 89.2%.

Although so far evaluation has only been carried out with CoDel as the 
Classic AQM, CoDel was chosen as the likely worst-case; being least 
distinguishable from an L4S AQM. This gives some confidence that planned 
evaluation with other Classic AQMs will be successful.

        Background

The algorithm aims to detect an AQM that supports RFC3168 ECN, and fall 
back to Reno-Friendly behaviour (ABE-Reno to be precise). This is the 
primary outstanding issue to be resolved before L4S can proceed to WGLC 
in tsvwg.

The site includes (or links to) detailed info on the source code, the 
experiment set-up, the algorithm design, planned work, etc

Finally, I'd like to take this opportunity to thank Asad Ahmed for all 
the work he's put into implementing and testing the algorithm and 
producing the visualizations of the results. And for putting up with me.

And really finally, I'd like to apologize for taking much longer than 
promised to get this to you all.

Cheers

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/