Subject: [tsvwg] L4S drafts: Next Steps

From: "Black, David" <[email-masked]>
Date: Fri, 4 Dec 2020 15:54:28 +0000

The WG chairs have consulted among ourselves and with our AD (Martin Duke).
Our guidance to the WG and the L4S draft authors is that at least the following two things need to be done before WGLC will become possible on the L4S drafts:
[1] Rework the "Prague Requirements" into the following two entities, which will overlap:
a.       Transport-protocol-agnostic normative requirements for all congestion-controlled traffic that uses the L4S low latency queue.
*  The DualQ AQM design relies upon traffic adhering to these requirements.
b.       Design and implementation guidelines for new congestion controls that meet the normative requirements (a).
*  Ultimately, the choice of whether a particular aspect is a guideline (b) or requirement (a) is a WG "rough consensus" decision.
     Success of this exercise will be judged by the WG reaching "rough consensus" that multiple transport protocol implementations
     currently meet and/or are reasonably expected to be able to meet the normative requirements (a).
[2] Build the safety case for L4S experimental Internet-wide deployment in the L4S Ops draft.
a.       This safety case does not rely on endpoints running TCP Prague, though it can assume endpoints are meeting the reworked Prague requirements
b.       Ops draft will provide guidance on how to detect L4S problems on their RFC 3168 network, and how to mitigate them.
c.       Ops draft must consider these scenarios:
*   An unsophisticated user purchases an L4S endpoint and runs it on a service provider's single-queue RFC 3168 network.
*   L4S traffic from another domain enters an RFC 3168 single-queue network (e.g. a peer-to-peer application)
     Success of this exercise will be judged by the WG reaching "rough consensus" that deployment of the L4S experiment
     is unlikely to cause significant damage to the Internet, and that any damage that results is expected to be tolerable.
Thanks, --David [as TSVWG WG co-chair]
----------------------------------------------------------------
David L. Black, Senior Distinguished Engineer
Dell Technologies, Infrastructure Systems Group
176 South St., Hopkinton, MA  01748
+1 (774) 350-9323<tel:+17743509323>           Mobile: +1 (978) 394-7754<tel:+19783947754>
[email-masked]<mailto:[email-masked]>
----------------------------------------------------------------

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 8 Dec 2020 23:00:44 +0000

David and chairs,

On 04/12/2020 15:54, Black, David wrote:
> Our guidance to the WG and the L4S draft authors is that at least the 
> following two things need to be done before WGLC will become possible 
> on the L4S drafts:
>
> [1] Rework the “Prague Requirements” into the following two entities, 
> which will overlap:
>
> a.Transport-protocol-agnostic normative requirements for all 
> congestion-controlled traffic that uses the L4S low latency queue.
>
> ·The DualQ AQM design relies upon traffic adhering to these requirements.
>
> b.Design and implementation guidelines for new congestion controls 
> that meet the normative requirements (a).
>
> ·Ultimately, the choice of whether a particular aspect is a guideline 
> (b) or requirement (a) is a WG “rough consensus” decision.
>

[BB] I believe draft-ietf-tsvwg-ecn-l4s-id is already structured for 
this division between normative requirements and design and 
implementation guidelines.
* [1]a) is Section 4 "Prerequisite Transport Layer Behaviour". Nearly 
every paragraph also refers off to subsections of Appendix A.1 for extra 
non-normative info, headed "Requirements for Scalable Transport 
Protocols". [Actually, it might be useful to swap these two headings]
* [1]b) is Appendix A.2: "Scalable Transport Protocol Optimizations"

The rule that determined which aspect went in which was:
[1]a) if it's about an aspect of one transport's behaviour that has 
potential to impact/harm others
[1]b) if it's solely about how a transport can improve it's own performance.
They have been divided like that from the very first ad hoc meeting in 
Prague that formulated them (in 2015)

Code to address all the requirements and most of the optimizations has 
been implemented. Over the last year, 3 or 4 of the later requirements 
in a) have become SHOULDs not MUSTs. Personally I would have left most 
as MUSTs, but I've tried to reflect what the WG wanted. Those demoted to 
SHOULD have generally been considered a lower risk of harm (either low 
likelihood of occuring or minor harm if they do) - taking into account 
the complexity of implementing them.

We could certainly flag the point where the transition from MUSTs to 
SHOULDs occurs within section 4. But I think we should still group all 
the items with potential for harm together in the same section. Because 
assessment of risk will change as the Internet landscape changes 
(possibly over the duration of the experiment).

Now I've explained, I hope the chairs will all agree that "rework the 
Prague Requirements into two entities" (normative requirements and 
design and implementation guidelines) is already done and dusted.

Nonetheless, if there are specific items in S.4 that someone believes 
should or shouldn't be there, or should be worded differently, let's 
have those discussions now on the list - I'm all ears.

Bob

>      Success of this exercise will be judged by the WG reaching “rough 
> consensus” that multiple transport protocol implementations
> a.This safety case does not rely on endpoints running TCP Prague, 
> though it can assume endpoints are meeting the reworked Prague 
> requirements
>
> b.Ops draft will provide guidance on how to detect L4S problems on 
> their RFC 3168 network, and how to mitigate them.
>
> c.Ops draft must consider these scenarios:
>
> · An unsophisticated user purchases an L4S endpoint and runs it on a 
> service provider's single-queue RFC 3168 network.
>
> · L4S traffic from another domain enters an RFC 3168 single-queue 
> network (e.g. a peer-to-peer application)
>
>      Success of this exercise will be judged by the WG reaching “rough 
> consensus” that deployment of the L4S experiment
>      is unlikely to cause significant damage to the Internet, and that 
> any damage that results is expected to be tolerable.
>
> +1 (774) 350-9323 <tel:+17743509323>           Mobile: +1 (978) 
> 394-7754 <tel:+19783947754>
>
> [email-masked] <mailto:[email-masked]>
>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: "Black, David" <[email-masked]>
Date: Wed, 9 Dec 2020 22:28:56 +0000

Bob,

     >> currently meet and/or are reasonably expected to be able to meet the normative requirements (a).
If the L4S folks believe that this has already been done, then it's time to start bringing forward the multiple transport protocols that demonstrate success of that exercise.

>Nonetheless, if there are specific items in S.4 that someone believes should or shouldn't be there, or should be worded differently, let's have those discussions now on the list - I'm all ears

Speaking only for myself as an individual beyond this point, these two normative requirements struck me as ones that might be better stated as design and implementation guidelines:

   o  A scalable congestion control MUST eliminate RTT bias as much as

      possible in the range between the minimum likely RTT and typical

      RTTs expected in the intended deployment scenario (see

      Appendix A.1.5<https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id-12#appendix-A.1.5> for rationale).

   o  A scalable congestion control SHOULD remain responsive to

      congestion when typical RTTs over the public Internet are

      significantly smaller because they are no longer inflated by

      queuing delay (see Appendix A.1.6<https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id-12#appendix-A.1.6> for rationale).

The use of "as much as possible" in the first bullet makes it hard to figure out whether a specific congestion control meets that mandatory requirement for participation in the L4S experiment.  The second bullet appears to involve a prediction of the future as currently written.

Thanks, --David

From: Bob Briscoe <[email-masked]>
Sent: Tuesday, December 8, 2020 6:01 PM
To: Black, David; [email-masked]
Subject: Re: [tsvwg] L4S drafts: Next Steps

[EXTERNAL EMAIL]
David and chairs,

[BB] I believe draft-ietf-tsvwg-ecn-l4s-id is already structured for this division between normative requirements and design and implementation guidelines.
* [1]a) is Section 4 "Prerequisite Transport Layer Behaviour". Nearly every paragraph also refers off to subsections of Appendix A.1 for extra non-normative info, headed "Requirements for Scalable Transport Protocols". [Actually, it might be useful to swap these two headings]
* [1]b) is Appendix A.2: "Scalable Transport Protocol Optimizations"

The rule that determined which aspect went in which was:
[1]a) if it's about an aspect of one transport's behaviour that has potential to impact/harm others
[1]b) if it's solely about how a transport can improve it's own performance.
They have been divided like that from the very first ad hoc meeting in Prague that formulated them (in 2015)

Code to address all the requirements and most of the optimizations has been implemented. Over the last year, 3 or 4 of the later requirements in a) have become SHOULDs not MUSTs. Personally I would have left most as MUSTs, but I've tried to reflect what the WG wanted. Those demoted to SHOULD have generally been considered a lower risk of harm (either low likelihood of occuring or minor harm if they do) - taking into account the complexity of implementing them.

We could certainly flag the point where the transition from MUSTs to SHOULDs occurs within section 4. But I think we should still group all the items with potential for harm together in the same section. Because assessment of risk will change as the Internet landscape changes (possibly over the duration of the experiment).

Now I've explained, I hope the chairs will all agree that "rework the Prague Requirements into two entities" (normative requirements and design and implementation guidelines) is already done and dusted.

Nonetheless, if there are specific items in S.4 that someone believes should or shouldn't be there, or should be worded differently, let's have those discussions now on the list - I'm all ears.

Bob

--

---

From: Jonathan Morton <[email-masked]>
Date: Thu, 10 Dec 2020 03:55:47 +0200

> On 10 Dec, 2020, at 12:28 am, Black, David <[email-masked]> wrote:
> 
> Reminder of the goal for [1] (Rework the “Prague Requirements” into the following two entities ...):

>      >> Success of this exercise will be judged by the WG reaching “rough consensus” that multiple transport protocol implementations

> If the L4S folks believe that this has already been done, then it’s time to start bringing forward the multiple transport protocols that demonstrate success of that exercise. 

To amplify this point, I would like to emphasise that neither TCP Prague, nor DCTCP, nor the modified version of SCReAM meet the following requirement, presently documented as a MUST:

   o  A scalable congestion control MUST implement monitoring in order
      to detect a likely non-L4S but ECN-capable AQM at the bottleneck.
      On detection of a likely ECN-capable bottleneck it SHOULD be
      capable (dependent on configuration) of automatically adapting its
      congestion response to coexist with TCP Reno congestion controls
      [RFC5681] (see Appendix A.1.4 for rationale and a referenced
      algorithm).

SCReAM, last I checked, has no code to implement the detection heuristic.  Neither does DCTCP.  TCP Prague has it implemented, but disabled by default, and when enabled the cited algorithm proved to not be reliable enough to satisfy a MUST requirement.  That strongly suggests that you have some work to do in this area.

Let me be clear: coexistence with RFC-3168 infrastructure and conventional AIMD traffic is mandatory, not optional, if an Internet-wide experiment is contemplated.

 - Jonathan Morton

---

From: Sebastian Moeller <[email-masked]>
Date: Thu, 10 Dec 2020 09:08:02 +0100

Hi David,

> On Dec 9, 2020, at 23:28, Black, David <[email-masked]> wrote:
> 
> Bob,
>  
>       Appendix A.1.5 for rationale).

	[SM] As argued before, the comparison here should be against the current state of affairs and not against an unspecific potential and the requirement should be something actionable like:

RTT bias MUST not be significantly worse than the RTT bias observed in a reference transport protocol (e.g. Reno, or to better reflect the current internet CUBIC?) over a reference queue (e.g.  dumb FIFO or reference AQM, like RED or PIE). As written it is quite confusing, including the question why the bias removal magnitude seems is contingent upon deployment scenario. How should a transport protocol designer be able to predict where the protocol is going to be used? Most protocols will aim for use over the internet, so what are minimum and typical RTTs over the internet? 
	I have noted before that L4S introduces a specific new meaning to the term "typical RTT", so does the typical RTT in this requirement denote that specific L4S definition or the "intuitive" interpretation of "the mean/median RTT ne expects on a specific link"?

The really really odd thing here is, that TCP Prague plus DualPI2 actually introduce a considerably HIGHER RTT bias compared to the status quo (see e.g. https://camo.githubusercontent.com/0ca81a2fabe48e8fce0f98f8b8347c79d27340684fe0791a3ee6685cf4cdb02e/687474703a2f2f7363652e646e736d67722e6e65742f726573756c74732f6c34732d323032302d31312d3131543132303030302d66696e616c2f73312d6368617274732f727474666169725f63635f71646973635f31306d735f3136306d732e737667 and compare the first 4 pairs of bars with the fifth, L4S clearly increases RTT bias by a lot) and hence both fail completely to meet this requirement.  Comparing pair 1 with pair 5 shows that DualPI2 increases RTT bias even for flows in the cassic queue over a FIFO, and comparing pair 4 with pair 5 shows that TCP Prague takes this to a new starvation-class extreme. Hence both the reference L4S AQM (Dual Queue coupled) and the L4S reference transport protocol fail hard in this requirement.

Best Regards
	Sebastian

>       queuing delay (see Appendix A.1.6 for rationale).
>  
> The use of “as much as possible” in the first bullet makes it hard to figure out whether a specific congestion control meets that mandatory requirement for participation in the L4S experiment.  The second bullet appears to involve a prediction of the future as currently written.
>  
> Thanks, --David
>  
> [1] Rework the “Prague Requirements” into the following two entities, which will overlap:
> ·         The DualQ AQM design relies upon traffic adhering to these requirements.
> ·         Ultimately, the choice of whether a particular aspect is a guideline (b) or requirement (a) is a WG “rough consensus” decision.
> 
> ·          An unsophisticated user purchases an L4S endpoint and runs it on a service provider's single-queue RFC 3168 network.
> ·          L4S traffic from another domain enters an RFC 3168 single-queue network (e.g. a peer-to-peer application)
>      Success of this exercise will be judged by the WG reaching “rough consensus” that deployment of the L4S experiment
> +1 (774) 350-9323           Mobile: +1 (978) 394-7754
> [email-masked]

---

From: Wesley Eddy <[email-masked]>
Date: Fri, 11 Dec 2020 14:23:56 -0500

Just replying to clarify the goal on splitting of the requirements:

> design and implementation guidelines) is already done and dusted.
As I'm seeing it at the moment, the problem is that there seems to be 
disagreement about whether it's possible to create transport algorithms 
or ensembles that meet the requirements.  Some people are fine with 
this, while others are uncomfortable that there isn't a clear "existence 
proof" yet that this can be done.

I believe everyone would agree that there are viable research projects 
(and some results have been shared in the WG) to address the 
requirements in various ways, and that those are of strong interest to 
the IETF and ICCRG now and in the future.

I think it's fine if we can have agreement that additional transport 
work to further this research is part of the experimental scope for L4S, 
but we should be hearing people saying they're comfortable to be 
contributing to that and make transports that will coherently meet all 
of the MUSTs.  Or, this is an opportunity for them to say if some of the 
requirements should be altered/relaxed in the split.