Subject: [tsvwg] L4S more questions about C-queue target selection rationale

From: Sebastian Moeller <[email-masked]>
Date: Wed, 25 Aug 2021 21:01:22 +0200

Dear Bob,

initially I just wanted to avoid looking too deep into your PI2 parameter paper, but unfortunately I looked too deep, so....

Your rationale why 15ms is a reasonable target for dualQ's C-queue PIE instance found at:
https://arxiv.org/pdf/2107.01003.pdf contains the following
contains the following text:

"As a sanity check, 34 ms compares reasonably well with the global averages given on Ookla’s Speedtest Global Index page:

• 20 ms fixed and 37 ms mobile (Apr 2021 data); • 24 ms fixed and 42 ms mobile (Apr 2020 data).

Ookla’s data is collected from self-selecting users who use speedtest’s algorithm to find the closest CDN-based servers [Ook21]. The page gives a sin- gle global figure without details of the method used."

Which seems incorrect to me, Ookla, allows anybody to add speedtest servers to their network that fulfills their requirements (see https://support.ookla.com/hc/en-us/articles/234578568), the >= 1 Gbps connectivity rules out most/many home networks, but quite a number of these machines are hosted by individuals or companies at well connected data centers but outside of content delivery networks (which I take to be the canonical expansion of CDN). 
Ookla's own description under https://www.speedtest.net/global-index/about does not mention CDN-based servers at all, so it seems prudent to clarify either how you know these servers used for the global averages are exclusively housed in CDNs or just remove that qualification completely.

Regarding the ATLAS data, making a few excursions into:
1) https://github.com/emirb/ripe-atlas-cdn-analysis (for the number of probes recruited to get the data)
2) https://atlas.ripe.net/results/maps/network-coverage (filtered for the country in 1) to get a feel for the distribution of ATLAS probes in each country)
convinces me that your dataset requires much more pruning. You certainly need to exclude all countries in which the spatial coverage is poor (most metropoles will have CDN nodes, if all ATLAS probes are located in metropoles this screws your data towards non-representative short RTTs), you probably also should exclude countries with less than X probes completely (the study tried to get 50 probes, so X = 50 seems like a sane start).
I applaud you looking into the relative high RTT for China, but the same diligence seems also due for countries with unexpected low CDN-RTTs like Nigeria (with only four probes in the set, see e.g. https://github.com/emirb/ripe-atlas-cdn-analysis/edit/master/googlecloud/NG.json). Digging deeper into the Nigeria case it looks like results from CDNs larger than ~100ms on average have been excluded from the averaging (not well described in the reference). It seems clear that 3 of the four probes simply where close to google's pop on Lagos... In short the Nigeria data are not representative and should not be included in your estimation of the typical RTT.

Regards
	Sebastian

P.S.: I have not looked deeper into the other section of the paper, but it seems even the first of your three values you use to justify the 15ma value is dubious. If this was a used car, I would stop the tire kicking right there, if that was a paper under peer review I would recommend to reject it.

---

From: Sebastian Moeller <[email-masked]>
Date: Wed, 25 Aug 2021 22:36:29 +0200

Dear Bob,

following up on the fact that the ATLAS data set is inconsistent, if I throw out all countries with less than 50 connected probes (as of this evening, I have not researched the historic state at the time of the 2019 measurements) the weighted average CDN_RTT comes down to 20.7 ms and the product of 20.7 * 0.22 * 2 = 9.108 ms... So according to your own rationale the classic target should be ~10ms, not 15...

But as I said before it seems rather like one of those parameters where a few tests with different settings allow to pick the sweet spot/compromise of utilization, RTT-bias, and low latency conservation under load.

Best Regards
	Sebastian

> On Aug 25, 2021, at 21:01, Sebastian Moeller <[email-masked]> wrote:
> 
> Dear Bob,
>