Subject: [tsvwg] L4S, rationale and data for CDN-RTT questioned

From: Sebastian Moeller <[email-masked]>
Date: Fri, 27 Aug 2021 11:21:46 +0200

Dear list, dear Bob,

in PI2 Parameters (https://arxiv.org/pdf/2107.01003.pdf), it is argued that the "typical RTT" is best approximated by taking the population/internet-user weighted per country average RTT towards the nearest CDN nodes. The CDN RTTs are taken from a Ripe ATLAS study (https://labs.ripe.net/author/emirb/analysing-global-cdn-performance/) that aims at finding the best CDN for a give region/country
This approach has iMHO several issues:

1) the reported data is only from the single "nearest" CDN, but given that different content providers use different CDN providers this is not even a good approximation of access to all of  the "CDN-cached" part of the internet.*

2) CDN networks are distributed in a biased fashion, with a strong over representation of the rich north American and European countries, (see e.g. slides 14-19 of https://www.ripe.net/participate/forms/uploads/fobi_plugins/file/see8/Analysing%20Global%20CDN%20Performance%20using%20RIPE%20Atlas_91c6593a-3aed-4f5f-bbed-499af31374d3.pdf for the CDN node distribution during the time of the study).

3) the internet is more than the collection of CDNs and hence reducing the "public internet" to the set of nearest CDN nodes seems wrong. That is not to say that near by servers (of any kind) are not important, after all locality will be relevant for many users, but reducing the reach of the internet to only those close by data centers seems conceptually at odds with the big promises of the internet of connecting the world over all continents ans oceans.

MHO this whole approach to justify your choice of classic PI2 target on theoretical grounds is considerably worse than simply testing a few different values and to compare the relevant measures (here, utilization at different RTTs, RTT-bias, latency under load increase) and simply pick one with an acceptable set of measures. I also note that between:

May 2017 https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-aqm-dualq-coupled-00#page-16:
target = 20ms

and July 2017 https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-aqm-dualq-coupled-01#page-14's
2:    target = 15 ms              % PI AQM Classic queue delay target

target was cut by 5 ms (IMHO a step in the right direction) without any discussion which data and considerations justified that change.

Best Regards
	Sebastian

*) If we look at the surprising 4ms CDN-RTT result for Nigeria as an example again and dig a little deeper, we will see, that out of the 7 CDNs in the study only two (Google Cloud, and cloudflare) have any nodes in that part of western Africa at all, conveniently both in/near Lagos. And the four ATLAS probes in Nigeria that where included in the study deemed ot have been located relative close to Lagos as well (here showing the average RTTs in milliseconds over the three ICMP echos per probe, for each of the 4 probes to each of the 7 CDNs under study):

https://github.com/emirb/ripe-atlas-cdn-analysis/blob/master/akamai/NG.json:
	(95.4864766667 + 150.3572666667 + 95.5142783333 + 98.6786333333) / 4 = 110.00916375
https://github.com/emirb/ripe-atlas-cdn-analysis/blob/master/azure/NG.json
	(95.54486 + 104.5804183333 + 105.9598483333 + 101.031923) / 4 = 101.779262417
https://github.com/emirb/ripe-atlas-cdn-analysis/blob/master/cachefly/NG.json
	(94.9310933333 + 103.9737216667 + 117.5644233333 + 100.7371176667) / 4 = 104.301589
https://github.com/emirb/ripe-atlas-cdn-analysis/blob/master/cloudflare/NG.json
	(80.4441933333 + 42.5460516667 + 1.2427183333 + 190.1043) / 4 = 78.5843158333
https://github.com/emirb/ripe-atlas-cdn-analysis/blob/master/cloudfront/NG.json
	(101.4728 + 167.7288683333 + 118.0780783333 + 100.4719973333) / 4 = 121.937936
https://github.com/emirb/ripe-atlas-cdn-analysis/blob/master/fastly/NG.json
	(80.5721666667 + 136.23388 + 95.1872616667 + 191.3619853333) / 4 = 125.838823417
https://github.com/emirb/ripe-atlas-cdn-analysis/blob/master/googlecloud/NG.json
	(1.7249766667 + 12.3254366667 + 1.0844533333 + 1.0544553333) / 4 = 4.0473305

across CDN-average: (110.00916375 + 101.779262417 + 104.301589 + 78.5843158333 + 121.937936 + 125.838823417 + 4.0473305) / 7 = 92.4 ms
We can see, if at all Google cloud seems to be the outlier here and not a proxy for "normal" CDN access latency.

If we accept the CDNs are a proxy for internet access (and ignore the fact that 4 probes in Lagos are not representative for the overall CDN-RTT in all of Nigeria), the number to plug into your weighted average would be 92, not 4. We can haggle whether the average should be over all 7 CDNs in the study, but I think it is clear that just taking a single one is not a useful measure for anything but figuring out which CDN was the best to serve different countries in 2019.