Subject: [tsvwg] PI2 Parameters are still not convincing

From: Sebastian Moeller <[email-masked]>
Date: Wed, 3 Nov 2021 21:11:09 +0100

Dear Bob,

the most recent version of PI2 Parameters (https://arxiv.org/abs/2107.01003#) now contains this sentence:

"As points of interest, the countries at the extremes are labelled in red (Nigeria is both in the top 10 by user population, and it has the lowest latency, tying with South Korea)."

But as I had demonstrated on this list Nigeria's 4ms RTT is a consequence of its only four measurement nodes being located ~1ms, 1ms, 1ms, and 12ms away from the single CDN node on that country in Lagos. What you need to do is remove Nigeria from your list due to lack of sufficient data, and not go and pretend that the internet in Subsaharan Afrika is as advanced as in South Korea. 
After the time I spend looking into your source data and finding gaping holes I would have expected you to really sanity check the input data for your list.
IMHO this demonstrates nicely what is wrong with L4S, the engineering is not based on and guided by objective numbers but numbers are massaged until they fit the pre-existing theory. 

Regards
	Sebastian

P.S.: I really would like ot see numbers on how using classic targets of say: 5, 10, 15, 20 ms affects utilization, RTT bias, and latency under load in a dualQ node. And given your test setup it should have cost much less time to empirically justify your classic target that way instead of post-hoc inventing a theory that shows that the number that initially was decided on is "optimal" (but required to multiply three increasingly dubious numbers).

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Fri, 5 Nov 2021 14:25:02 +0000

Hi Sebastian,

I see other drafts just specifying a default number without any explanation. Bob made an effort in getting some rational behind the number we specified, but seems this is indeed just more food for needless discussion. The draft mentions that this reference can be used as an example to find your own value, based on the situation you are in. So is only useful as an example process, so it should not be used as a solid justification for the default. I don't think the draft claims this.

At the end the Classic and L4S targets are just parameters (even the defaults) that everyone can fill in as they want. The results are on theoretical basis predictable in the non-corner cases. Specific tests depend heavily on the CCs used as well, and on the requirements from its users. As L4S support in end-systems further develops, the classic target can be increased, as latency sensitive apps will use L4S, while applications insensitive to and capable of providing (sometimes useful) network buffers can stay using Classic. These all should be part of further experimentation and evolution.

Koen.

-----Original Message-----
From: tsvwg <[email-masked]> On Behalf Of Sebastian Moeller
Sent: Wednesday, November 3, 2021 9:11 PM
To: TSVWG <[email-masked]>
Subject: [tsvwg] PI2 Parameters are still not convincing

Dear Bob,

Regards
	Sebastian

---

From: Sebastian Moeller <[email-masked]>
Date: Fri, 5 Nov 2021 15:55:56 +0100

Hi Koen,

> On Nov 5, 2021, at 15:25, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 
> Hi Sebastian,
> 
> I see other drafts just specifying a default number without any explanation.

	[SM] I wish you had lefty it at "classic target was selected to match the PIE default recommendations without further research optimisation". Because the retro-explanation you came up with is clearly deficient (both in the theory and in the numbers), which makes your 15ms much less useful.

> Bob made an effort in getting some rational behind the number we specified,

	[SM] That is a decent description, but the question was how did you initially converge on that number, not what post-hoc rationale you can come up with.

> but seems this is indeed just more food for needless discussion.

	[SM] Well, if your theory would have been sound and the numbers well sanitized we would not have a discussion. We do because neither your selection theory nor the numbers you plug into it survived a tiny bit of scrutiny. We are at the point where you should thank the reviewer and fix up your errors, or drop that hole attempt at retro-actively in 2021 figure out why you selected a number several years ago.... 
	But honestly, dropping China, because the RTT got too high after fixing the miscalculation in the geometry factor (while not fixing the Nigeria number) really exposes the whole "theory" as deficient (unless the theory is: find three numbers with a product of 15).

> The draft mentions that this reference can be used as an example to find your own value, based on the situation you are in. So is only useful as an example process, so it should not be used as a solid justification for the default. I don't think the draft claims this.

	[SM] You might have missed the fact that in your DualQ design the ratio of L4S and classic target affects capacity sharing behavior at the intended use-case of short RTTs considerably, something that could be ameliorated a lot by picking a lower classic target, as I have tried pointing out for some time. And DOCSIS PIE, which is supposed to work in exactly the same internet as L4S defaulted to a 10ms target, apparently without any adverse side-effects (according to the RFC this number was based on simulations, that do not seem to be published anywhere).

> 
> At the end the Classic and L4S targets are just parameters (even the defaults) that everyone can fill in as they want.

	[SM] Sure, but the onus is on you to at least propose a set of parameters under which your design performs according to its promises, and for short RTTs is still does not, but reducing the classic target would help there. 

> The results are on theoretical basis predictable in the non-corner cases.

	[SM] Well, your corner-cases might be somebody else's normal internet access link, so I do not think it wise to be too cavalier about this. But humor me and predict, what will be the results of setting the classic target to: 5ms, 10, 15ms, 20ms in regards to RTT fairness inside the classic queue, latency under load increase, and capacity sharing with the L-queue at extremely short path RTTs.

> Specific tests depend heavily on the CCs used as well, and on the requirements from its users. '

	[SM] Sure, but on an internet link a certain level of progress for every flow can be expected, as can be expected that the IETF only endorses designs that do not put such forward progress in jeopardy (I might be in the rough here). In your own network, go wild and deploy what ever you like, but in the wider internet make sure you are coexisting peacefully. As far as I can tell the IETF considers Reno and maybe Cubic as reference TCP CC's newcomers should be "compatible" with, so I am tiotally fine with restricting tests to those CCs.

> As L4S support in end-systems further develops, the classic target can be increased, as latency sensitive apps will use L4S, while applications insensitive to and capable of providing (sometimes useful) network buffers can stay using Classic. These all should be part of further experimentation and evolution.

	[SM] Well, how about we make sure that we can safely start the experiment in the existing internet first? And for the current state a smaller classic target would already help. Especially since you lack and rationale for your 15ms to begin with. I am beginning to assume that the bias at short RTTs is not an accident but a conscious decision to give L4S an advantage.
	
Sebastian

---

From: Martin Duke <[email-masked]>
Date: Fri, 5 Nov 2021 08:14:48 -0700

Sebastian,

It would be more appropriate to suggest that Bob's inclusion of Nigeria is
flawed, and leave it that.

Suggesting that he is an incompetent empiricist and/or operating in bad
faith is not appropriate for an IETF email list.

This is not the first transgression by either side in this debate, but
hopefully it is one of the last.

Martin
(AD Hat on)

On Wed, Nov 3, 2021 at 1:11 PM Sebastian Moeller <[email-masked]> wrote:

> Dear Bob,
>
>
> the most recent version of PI2 Parameters (
> https://arxiv.org/abs/2107.01003#) now contains this sentence:
>
> "As points of interest, the countries at the extremes are labelled in red
> (Nigeria is both in the top 10 by user population, and it has the lowest
> latency, tying with South Korea)."
>
> But as I had demonstrated on this list Nigeria's 4ms RTT is a consequence
> of its only four measurement nodes being located ~1ms, 1ms, 1ms, and 12ms
> away from the single CDN node on that country in Lagos. What you need to do
> is remove Nigeria from your list due to lack of sufficient data, and not go
> and pretend that the internet in Subsaharan Afrika is as advanced as in
> South Korea.
> After the time I spend looking into your source data and finding gaping
> holes I would have expected you to really sanity check the input data for
> your list.
> IMHO this demonstrates nicely what is wrong with L4S, the engineering is
> not based on and guided by objective numbers but numbers are massaged until
> they fit the pre-existing theory.
>
> Regards
>         Sebastian
>
> P.S.: I really would like ot see numbers on how using classic targets of
> say: 5, 10, 15, 20 ms affects utilization, RTT bias, and latency under load
> in a dualQ node. And given your test setup it should have cost much less
> time to empirically justify your classic target that way instead of
> post-hoc inventing a theory that shows that the number that initially was
> decided on is "optimal" (but required to multiply three increasingly
> dubious numbers).
>

---

From: <[email-masked]>
Date: Mon, 8 Nov 2021 07:55:10 +0000

Hi Martin,

thanks for your statement. I share your view.

Regards,

Ruediger

Von: tsvwg <[email-masked]> Im Auftrag von Martin Duke
Gesendet: Freitag, 5. November 2021 16:15
An: Sebastian Moeller <[email-masked]>
Cc: TSVWG <[email-masked]>
Betreff: Re: [tsvwg] PI2 Parameters are still not convincing

Sebastian,

It would be more appropriate to suggest that Bob's inclusion of Nigeria is flawed, and leave it that.

Suggesting that he is an incompetent empiricist and/or operating in bad faith is not appropriate for an IETF email list.

This is not the first transgression by either side in this debate, but hopefully it is one of the last.

Martin
(AD Hat on)

On Wed, Nov 3, 2021 at 1:11 PM Sebastian Moeller <[email-masked]<mailto:[email-masked]>> wrote:
Dear Bob,

the most recent version of PI2 Parameters (https://arxiv.org/abs/2107.01003#<https://arxiv.org/abs/2107.01003>) now contains this sentence:

Regards
        Sebastian