Subject: [tsvwg] FCT tests of Prague in shared RFC3168 queues

From: Pete Heist <[email-masked]>
Date: Wed, 10 Mar 2021 13:52:26 +0100

We've run tests of flow completion time (FCT) for CUBIC flows, in
competition with single CUBIC or Prague flows in a shared RFC3168
queue. The results are in the l4s-tests repo, and below.

https://github.com/heistp/l4s-tests/#harm-to-flow-completion-time

TL;DR
In shared RFC3168 queues with the chosen parameters, median and P95 FCT
increase significantly across RTTs with Prague as the competition,
compared to CUBIC.

Flow arrival times use an exponential distribution, and flow lengths a
lognormal distribution, with the parameters below. "Harm" calculations
(https://www.cs.cmu.edu/~rware/assets/pdf/ware-hotnets19.pdf) are added
for each metric. The results are just text, so I'll include them here.

Test Parameters:
----------------
CCAs under test:  cubic, prague
RTTs:             10ms, 20ms, 40ms, 80ms, 160ms
Bandwidth:        50Mbps
Qdisc:            fq_codel flows 1
Slow start delay: 20s

FCT Workload Parameters:
------------------------
Server URL:        localhost:8188
CCA:               cubic
Duration:          3m0s
Flows:             900
Mean arrival time: 200ms
Est. bandwidth:    25.83Mbps
Flow lengths (lognormal distribution):      
|- P5:             65536
|- Mean:           645683
|- P95:            2097152

RTT    CCA     GeoMean (Harm)    Median (Harm)     P95 (Harm)
---    ---     --------------    -------------     ----------
10ms   -       176.2ms           154.3ms           1061.3ms
10ms   cubic   339.3ms (0.481)   314.9ms (0.510)   2002.7ms (0.470)
10ms   prague  1171.3ms (0.850)  1131.7ms (0.864)  7305.2ms (0.855)
20ms   -       243.0ms           211.5ms           1155.3ms
20ms   cubic   376.6ms (0.355)   351.7ms (0.399)   1986.0ms (0.418)
20ms   prague  1394.1ms (0.826)  1385.3ms (0.847)  8454.8ms (0.863)
40ms   -       369.2ms           317.8ms           1634.7ms
40ms   cubic   475.7ms (0.224)   423.9ms (0.250)   2179.9ms (0.250)
40ms   prague  1643.8ms (0.775)  1637.7ms (0.806)  9879.3ms (0.835)
80ms   -       599.9ms           554.5ms           2135.6ms
80ms   cubic   906.7ms (0.338)   846.9ms (0.345)   3709.4ms (0.424)
80ms   prague  2384.8ms (0.748)  2228.4ms (0.751)  12159.9ms (0.824)
160ms  -       976.6ms           976.4ms           3465.1ms
160ms  cubic   1127.7ms (0.134)  1043.2ms (0.064)  4207.8ms (0.177)
160ms  prague  2471.9ms (0.605)  2174.7ms (0.551)  14418.8ms (0.760)