Subject: Re: [tsvwg] NQB, what rate is permissable for flows to still "be" NQB

From: Greg White <[email-masked]>
Date: Fri, 11 Feb 2022 16:24:18 +0000

Hi Ruediger,

Comments [GW2] below.

-Greg

On 1/14/22, 12:18 AM, "[email-masked]" <[email-masked]> wrote:

    Hi Greg,

    That's an important issue...just some thoughts

    --snip--

    [GW] Another aspect I'm considering is whether the draft should provide some guidance for "very low rate" links.  Guidance could be that very low rate links (e.g. links that are less than the 10th percentile of global access network rates) should implement a Traffic Protection mechanism that is more tolerant of burstiness (i.e. a temporary queue) than higher rate links might be.

    Draft says: If the application's traffic exceeds more than a few packets per RTT, or exceeds approximately 1 Mbps on an instantaneous (inter-packet) basis, the application SHOULD NOT mark its traffic with the NQB DSCP.

    [RG] Two additional questions, which may be worth a thought:
    Should there be guidance relating the "acceptable standard NQB sender behaviour", e.g., 1 Mbps, to the speed of a provider access link? Like do not support... / add traffic management of kindx,y,z for... NQB, if access speed is less than n*1 Mbps? n = 10 as a first guess, to start by a value (note that this doesn't limit the sum of simultaneous NQB senders). The latter example is based on current text and may have to be adapted, as text changes.

[GW2] I am adding a subsection on "Guidance for Very Low Rate Links".  I've not specifically written it to apply just to access provider links, but rather more generally to operators of network that have low speed links.    I currently don't put it in a formula (since the text about the 1Mbps limit has, intentionally, gotten a bit fuzzier).  Instead I've just used qualitative terms:

***** draft new Section 5.3 Guidance for Very Low Rate Links *******
The NQB sender requirements in Section 4.1 place responsibility in the hands of the application developer to determine the likelihood that the application's sending behavior could result in a queue forming along the path. These requirements rely on application developers having a reasonable sense for the network context in which their application is to be deployed. Even so, there will undoubtedly be networks that contain links having a data rate that is below what is considered "typical", and even potentially below the instantaneous sending rate of certain NQB-marked applications.

To limit the consequences of this scenario, operators of such networks SHOULD utilize a traffic protection function that is more tolerant of burstiness (i.e. a temporary queue). Alternatively, operators of such networks MAY choose to disable NQB support on these low speed links. In particular, for links that are far below "typical" path rates, it is RECOMMENDED that NQB support be disabled.¶
***************

I'm not confident that the language here is right, so would welcome thoughts from you or others on better phrasing and/or guidance. 

[RG]    RTT is mentioned in the draft text. If RTT awareness had an impact, adding some more discussion may help. So far, RTT awareness isn't assumed, perfectly ok. Why may it be important? If the definition of NQB was based on "bits in flight" , it could be independent of the MTU, I think. To get the statement right, a differentiation between NQB senders being RTT aware and those which aren't might make sense and if it does, standards defining acceptable behaviour of both may be useful.

[GW2] Hmm. Well RTT was mentioned in order to include very low rate request-response applications (e.g. the "Low Data-Volume Application" type discussed in RFC8085).  It isn't clear to me how to write more specific guidance for RTT-aware applications.  

    Regards,

    Ruediger

    On 11/13/21, 10:41 AM, "tsvwg on behalf of Sebastian Moeller" <[email-masked] on behalf of [email-masked]> wrote:

        Dear list,

        in last Friday's tsvwg meeting Greg asked what recommendation the NQB draft should give senders about maximally permissible rates for NQB flows. (At least I left the session with the feeling that input on this topic was requested).

        	One proposal, already under discussion on the list was to base this on some statistics of access rate, with the proposal being to take 10% of Ookla's idea of global average access rates (see https://www.speedtest.net/global-index "Speedtest Global Index") which reports global average (and per country sub-averages) of upload, download and latency numbers split into mobile and fixed broadband.

        	These are taken from random user measurement against Ookla's speedtest.net (which in itself is a collection of mostly volunteered measurement nodes that just need to fulfill a number of criteria (see https://support.ookla.com/hc/en-us/articles/234578628) to enter the speedtest network and are mostly operated by entities other than Ookla). The required minimum internet access speed (1 Gbps) makes it very likely that these measurement nodes are located in data-centers. During a typical measurement the system will try to automatically match clients with "close by" servers (but allows clients to manually select servers as an alternative).

        	The global index (from 2019 on) will include data from all countries that "have at least 300 unique user test results for mobile or fixed broadband in the reported month" (https://www.speedtest.net/global-index/about). Neither for the global average, nor for the per country numbers any information about the sample size is given (above the minimum), nor any measure of variance. So this source only gives (somewhat under-qualified) average numbers that are likely to be not really representative (of the spatial distribution of all users within a given country/the world) and  will be biased towards higher speeds (as averages of positively skewed distributions do). If we compare Ookla's current result for Germany https://www.speedtest.net/global-index/germany#fixed
        with the data from the official German speedtest site (lower plot on https://breitbandmessung.de/interaktive-darstellung-jahresvergleich *) it should become clear what a per country, or global average hides: the composition of access speeds in Germany as seen by that CDF clearly is more complicated than an average implies (it is not a smooth curve as there clearly are speed-tiers that ISPs can and do offer with their available technology). 
        	For answering the NQB problem something related to the experience a given number of endusers will have seems more appropriate than an average. So, IMHO, something like the lower 10% or 25%-percentile of the access speed (taken from a per-country CDF of reliable data sources like in ehe EU the national regulators' reports**) might be a much better number to use for the NQB recommendation than Ookla's averages. Qantiles/Median at least allow to estimate which fraction of users can expect rate issues with compliant NQB-senders. (This is not to dismiss Ookla's numbers, it is great that they compile and offer those, it is just that these are not terribly well suited for the NQB purpose).

        	But overall, given that such a recommendation can only ever be approximate and will not in itself guarantee safety for any individual connection but can just be a rough guideline for senders, I would say the current recommendation to not NQB mark flows with instantaneous rates >> 1Mbps seems just as appropriate as any statistics taken from non-official/non-representative numbers like Ookla's.

        	As written this recommendation will however run into "conflict" with gaming traffic (not cloud gaming with its video streaming but client server gaming where mainly control packets are exchanged), as games tend to send/receive packets on a "clock" (often 30 or 60 Hz, but 128 Hz is not unheard of) with one to multiple packets per clock-tick (see e.g. https://www.sciencedirect.com/science/article/abs/pii/S1084804511001731) and since back-2-back packets will create bursts at link speed (assuming the burst travels intact to the bottleneck and the bottleneck does not employ a scheduler that splits bursts up) these can exceed the current 1 Mpbs recommendation (I am grossly simplifying here): "Other types of QB flows include those that frequently send at a high burst rate (e.g. several consecutive packets sent well in excess of 1 Mbps) even if the long-term average data rate is much lower."

        	Assuming NQB is really intended for gaming control packets, this might be remedied as Greg proposed by giving a more permissive definition of "several consecutive packets" (like say expand several to >= 10 or similar). Whether that is justified on a slow connection within NQB's internal logic is a different question.

        Regards
        	Sebastian

        *) These numbers are not as current as Ookla's and hence a direct comparison seems not advised (https://www.speedtest.net/insights/blog/icymi-ookla-data-and-analysis-from-december-2020/ gives Ookla's fixed broadband global average for December 2020 as 96/52). But these graphs show the development over a number of years including the respective N for each curve and allow to see the countries dominant speed tiers (50 and 100 Mbps) and alsi allow to see the median of the 2019-2020 period (51.1 Mbps down, ~10.7 up) and that there was a small fraction of links with considerably higher speeds (500 and 1000 Mbps) which will make sure that the average (which appears purposefully not reported by the national regulator) would be noticeably larger than the median. BTW the 25%-ile was around 22.7 Mbps down 5.2 up...

        **) Unlike Ookla's "mostly-for-fun" speedtest*** (no disrespect intended) the official EU speedtests like the one in Germany that can be used as evidence at court has considerably more stringent specifications than Ookla's and generally seems to be designed and operated carefully, including approaches like having a user report the contracted speed before starting a measurement to make sure the measurement back-end has sufficient resources to saturate that speed.

        ***) To repeat, I do not want to criticize Ookla here, it is just that their numbers are not suitable for some purposes, especially purposes the test was never designed for.

---

From: <[email-masked]>
Date: Mon, 14 Feb 2022 07:35:29 +0000

Hi Greg,

Thanks for the offered text.

Regarding RTT awareness, you so far didn't give a deeper thought to transport protocols using NQB being RTT aware or not, if I got that right. As I wrote, not an issue to me, as there are no RTT related claims or requirements in the text. I'd rather look at that as a field for new ideas, should RTT awareness be helpful for transport protocols using NQB. To be more concise, maybe transport protocols sending traffic above a threshold bandwidth (which I don't want to determine, I just offer that terminology to illustrate where RTT awareness may make sense for NQB senders). I note that e.g. L4S is RTT aware.

Regards,

Rüdiger

-----Ursprüngliche Nachricht-----
Von: Greg White <[email-masked]> 
Gesendet: Freitag, 11. Februar 2022 17:24
An: Geib, Rüdiger <[email-masked]>
Cc: [email-masked]
Betreff: Re: [tsvwg] NQB, what rate is permissable for flows to still "be" NQB

Hi Ruediger,

Comments [GW2] below.

-Greg

    Hi Greg,

    --snip--

***** draft new Section 5.3 Guidance for Very Low Rate Links ******* The NQB sender requirements in Section 4.1 place responsibility in the hands of the application developer to determine the likelihood that the application's sending behavior could result in a queue forming along the path. These requirements rely on application developers having a reasonable sense for the network context in which their application is to be deployed. Even so, there will undoubtedly be networks that contain links having a data rate that is below what is considered "typical", and even potentially below the instantaneous sending rate of certain NQB-marked applications.

    Regards,

    Ruediger

        Dear list,

        Regards
        	Sebastian

---

From: Greg White <[email-masked]>
Date: Mon, 14 Feb 2022 19:49:56 +0000

Yes, you got it right. 

And just to be clear, you are seeing that as a topic for future ideas & discussion, not as something that needs to be added to the NQB draft.  Correct?

Thanks, 
Greg

On 2/14/22, 12:35 AM, "[email-masked]" <[email-masked]> wrote:

    Hi Greg,

    Regards,

    Rüdiger

    -----Ursprüngliche Nachricht-----
    Von: Greg White <[email-masked]> 
    Gesendet: Freitag, 11. Februar 2022 17:24
    An: Geib, Rüdiger <[email-masked]>
    Cc: [email-masked]
    Betreff: Re: [tsvwg] NQB, what rate is permissable for flows to still "be" NQB

    Hi Ruediger,

    Comments [GW2] below.

    -Greg

        Hi Greg,

        --snip--

        Regards,

        Ruediger

            Dear list,

            Regards
            	Sebastian

---

From: <[email-masked]>
Date: Tue, 15 Feb 2022 07:45:36 +0000

Hi Greg,

Correct, no deep NQB RTT discussion required in this version. 

I'd be happy about a note mentioning that RTT awareness could be worth a deeper discussion for transport protocols sending with "higher NQB conform bandwidth" (terrible term, please replace, if you have a better idea) in future versions and mentioning that, e.g., L4S is RTT aware. If L4S based transport may not be looked at as NQB conformant in all cases, because L4S also supports streams which definitely aren't NQB conformant, than that might have to be added too. I leave that open, as from what I think to have understood, L4S and NQB aren't firmly related.

Regards,

Rüdiger

-----Ursprüngliche Nachricht-----
Von: Greg White <[email-masked]> 
Gesendet: Montag, 14. Februar 2022 20:50
An: Geib, Rüdiger <[email-masked]>
Cc: [email-masked]
Betreff: Re: [tsvwg] NQB, what rate is permissable for flows to still "be" NQB

Yes, you got it right. 

Thanks, 
Greg

    Hi Greg,

    Regards,

    Rüdiger

    -----Ursprüngliche Nachricht-----
    Von: Greg White <[email-masked]> 
    Gesendet: Freitag, 11. Februar 2022 17:24
    An: Geib, Rüdiger <[email-masked]>
    Cc: [email-masked]
    Betreff: Re: [tsvwg] NQB, what rate is permissable for flows to still "be" NQB

    Hi Ruediger,

    Comments [GW2] below.

    -Greg

        Hi Greg,

        --snip--

        Regards,

        Ruediger

            Dear list,

            Regards
            	Sebastian