Subject: Re: [tcpm] [tsvwg] draft-ietf-tsvwg-ecn-l4s-id

From: Bob Briscoe <[email-masked]>
Date: Mon, 4 Nov 2019 15:26:57 +0000

Michael,

Thank you for pointing this out, many months ago now.

There is no formal dependency of draft-ietf-tsvwg-ecn-l4s-id on AccECN, 
as explained below.

For any transport protocol to comply with ecn-l4s-id, it needs 
sufficiently accurate feedback of the extent of CE marking on the 
forward path. You pointing this out has made me realize that there is a 
'needs' that should be a 'MUST' in the spec:

4.2.  Prerequisite Transport Feedback

CURRENT:

    In general, a scalable congestion control, needs feedback of the
    extent of CE marking on the forward path.

PROPOSED:

    In general,for a transport protocol to provide  scalable congestion control,
    it MUST provide  feedback of the
    extent of CE marking on the forward path.

However, ecn-l4s-id only specifies the requirements that any L4S 
transport protocol would have to satisfy. It does not define a transport 
protocol itself. So AccECN is not a normative ref. So I am also altering 
the text a little further down, where it refers to AccECN, as follows:

CURRENT:

   TCP:  Support for accurate ECN feedback (AccECN
       [I-D.ietf-tcpm-accurate-ecn  <https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-07#ref-I-D.ietf-tcpm-accurate-ecn>]) by both ends is a prerequisite for
       scalable congestion control.

PROPOSED:

   TCP:  Support for accurate ECN feedback (such as AccECN
       [I-D.ietf-tcpm-accurate-ecn  <https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-07#ref-I-D.ietf-tcpm-accurate-ecn>]) by both ends is a prerequisite for
       scalable congestion control.

Thanks again for pointing this out.

      Dependencies between Experiments

The case of TCP Prague provides a counter-argument:

TCP Prague could be specified in one monolithic experimental spec.
Or it could be broken down into experimental components, and a top level 
experimental spec that pulls them all together.

The latter option would create multiple dependencies of one experiment 
on others. But IMHO such modularity is preferable to a monolithic spec. 
Especially if some of the components might have other uses outside of 
TCP Prague (which is exactly the case with TCP Prague, e.g. AccECN, 
ECN++, RACK are all potentially parts of the base TCP stack, and 
paced-chirping is potentially usable by any other congestion control - 
these are all experimental component parts).

As another example, BBR was written up as an experimental draft, and the 
Linux implementation uses RACK (although not mentioned in the draft) 
which is enabled by default in Linux.

Such modularity is good.

Bob

On 24/03/2019 19:15, Scharf, Michael wrote:
> I have just parsed through draft-ietf-tsvwg-ecn-l4s-id-06 and I run into the following sentence:
>
> TCP:  Support for accurate ECN feedback (AccECN
>        [I-D.ietf-tcpm-accurate-ecn]) by both ends is a prerequisite for
>        scalable congestion control.  Therefore, the presence of ECT(1) in
>        the IP headers even in one direction of a TCP connection will
>        imply that both ends support AccECN.  However, the converse does
>        not apply.  So even if both ends support AccECN, either of the two
>        ends can choose not to use a scalable congestion control, whatever
>        the other end's choice.
>
> Can the authors please clarify whether there is a normative dependency of draft-ietf-tsvwg-ecn-l4s-id on draft-ietf-tcpm-accurate-ecn?
>
> If so, I think the formal dependency on implementing AccECN has to be spelt out explicitly. In that case, I would be looking for a statement of the sort "MUST/SHOULD/MAY implement draft-ietf-tcpm-accurate-ecn" somewhere. If such a requirement does not exist, I think the above paragraph is confusing.
>
> As individual contributor, I want to note that I disagree with mandatory dependencies between experiments. I have mentioned in the past that I believe that different experiments must be specified independently of each other and should be implementable independently of each other, specifically also to be able to fail independently of each other. It is possible that I am in the rough part of the consensus. But I haven't changed my mind.
>
> Michael (as individual)
>
>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 11 Nov 2019 09:39:00 +0000

Michael,

On a similar question of dependency between experiments, I think you 
also expressed concern that ecn-l4s-id depends on the RACK experiment. 
However, ecn-l4s-id only refers to RACK informatively, because it only 
uses the general idea behind RACK. Just as quic-recovery (stds track) 
uses the general idea behind RACK (expt track) by referring to it 
informatively.

Bob

On 04/11/2019 15:26, Bob Briscoe wrote:
> There is no formal dependency of draft-ietf-tsvwg-ecn-l4s-id on 
> AccECN, as explained below.
>
> forward path. You pointing this out has made me realize that there is 
> a 'needs' that should be a 'MUST' in the spec:
>
> transport protocol would have to satisfy. It does not define a 
> transport protocol itself. So AccECN is not a normative ref. So I am 
> also altering the text a little further down, where it refers to 
> AccECN, as follows:
>
> CURRENT:
> Or it could be broken down into experimental components, and a top 
> level experimental spec that pulls them all together.
>
> on others. But IMHO such modularity is preferable to a monolithic 
> spec. Especially if some of the components might have other uses 
> outside of TCP Prague (which is exactly the case with TCP Prague, e.g. 
> AccECN, ECN++, RACK are all potentially parts of the base TCP stack, 
> and paced-chirping is potentially usable by any other congestion 
> control - these are all experimental component parts).
>
> As another example, BBR was written up as an experimental draft, and 
> the Linux implementation uses RACK (although not mentioned in the 
> draft) which is enabled by default in Linux.
>
> Such modularity is good.
>
>
> Bob
>
>
> Bob Briscoehttp://bobbriscoe.net/

---

From: "Scharf, Michael" <[email-masked]>
Date: Mon, 11 Nov 2019 10:16:26 +0000

So can you confirm that a TCP sender can use the L4S service without implementing AccECN? I am asking again since there has been some recent list discussion on that, e.g., given the points raised by Sebastian. I am a bit lost here on what the future plan in L4S actually is.

And I still fail to understand why even the general idea of RACK matters to AQM algorithms, traffic classifiers, or priority schedulers for traffic with 1/p congestion control in a router. RACK has lots of merits, but most of them are IMHO entirely unrelated to the vision of L4S. I’d suggest to focus L4S on solving the key problems for traffic with 1/p congestion control.

Michael (with no hat)

From: Bob Briscoe <[email-masked]>
Sent: Monday, November 11, 2019 10:39 AM
To: Scharf, Michael <[email-masked]>; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: [tsvwg] draft-ietf-tsvwg-ecn-l4s-id

Michael,

On a similar question of dependency between experiments, I think you also expressed concern that ecn-l4s-id depends on the RACK experiment. However, ecn-l4s-id only refers to RACK informatively, because it only uses the general idea behind RACK. Just as quic-recovery (stds track) uses the general idea behind RACK (expt track) by referring to it informatively.

Bob

There is no formal dependency of draft-ietf-tsvwg-ecn-l4s-id on AccECN, as explained below.

For any transport protocol to comply with ecn-l4s-id, it needs sufficiently accurate feedback of the extent of CE marking on the forward path. You pointing this out has made me realize that there is a 'needs' that should be a 'MUST' in the spec:

CURRENT:

   In general, for a transport protocol to provide scalable congestion control,

However, ecn-l4s-id only specifies the requirements that any L4S transport protocol would have to satisfy. It does not define a transport protocol itself. So AccECN is not a normative ref. So I am also altering the text a little further down, where it refers to AccECN, as follows:

CURRENT:

      [I-D.ietf-tcpm-accurate-ecn<https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-07#ref-I-D.ietf-tcpm-accurate-ecn>]) by both ends is a prerequisite for

      [I-D.ietf-tcpm-accurate-ecn<https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-07#ref-I-D.ietf-tcpm-accurate-ecn>]) by both ends is a prerequisite for

TCP Prague could be specified in one monolithic experimental spec.
Or it could be broken down into experimental components, and a top level experimental spec that pulls them all together.

The latter option would create multiple dependencies of one experiment on others. But IMHO such modularity is preferable to a monolithic spec. Especially if some of the components might have other uses outside of TCP Prague (which is exactly the case with TCP Prague, e.g. AccECN, ECN++, RACK are all potentially parts of the base TCP stack, and paced-chirping is potentially usable by any other congestion control - these are all experimental component parts).

As another example, BBR was written up as an experimental draft, and the Linux implementation uses RACK (although not mentioned in the draft) which is enabled by default in Linux.

Such modularity is good.

Bob

      the other end's choice.

Michael (as individual)

--

--

---

From: Wesley Eddy <[email-masked]>
Date: Mon, 11 Nov 2019 09:42:33 -0500

I think both AccECN and RACK are appropriately "informational" 
references in the L4S drafts.

They might be important to TCP Prague code, but the in-network L4S 
pieces don't have a strict dependency.

On 11/11/2019 5:16 AM, Scharf, Michael wrote:
>
> So can you confirm that a TCP sender can use the L4S service without 
> implementing AccECN? I am asking again since there has been some 
> recent list discussion on that, e.g., given the points raised by 
> Sebastian. I am a bit lost here on what the future plan in L4S 
> actually is.
>
> And I still fail to understand why even the general idea of RACK 
> matters to AQM algorithms, traffic classifiers, or priority schedulers 
> for traffic with 1/p congestion control in a router. RACK has lots of 
> merits, but most of them are IMHO entirely unrelated to the vision of 
> L4S. I’d suggest to focus L4S on solving the key problems for traffic 
> with 1/p congestion control.
>

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 20 Nov 2019 17:11:01 +0800

Michael,

> Sebastian. I am a bit lost here on what the future plan in L4S 
L4S needs fine-grained feedback, but not necessarily AccECN.
So, while AccECN is the way to do fine-grained feedback, L4S needs AccECN.

I took your question to be in a context where something else might have 
replaced AccECN in the future for fine-grained feedback. In that case, 
L4S would need that new shiny thing instead, and would not then need AccECN.

> with 1/p congestion control.

I've promised David Black a full answer to this. So pls see that.
But briefly, an important advance of L4S is scalable throughput (it's in 
the name).

It would be pointless to remove one scaling limitation if it was 
replaced by the 3 DupACK as a throughput limitation (Amdahl's law). In 
an ideal world, the two would be done separately. But L4S creates a 
clean-slate opportunity where there's a new queue and new CCs, and there 
is no downside to placing the time-units requirement on these senders 
that have to be updated anyway. Then future expts that might rely on all 
hosts using time-units will not be precluded.

This is all in the latest text in the appendix.

Bob
>
> Michael (with no hat)
>
> *From:*Bob Briscoe <[email-masked]>
> *Sent:* Monday, November 11, 2019 10:39 AM
> *To:* Scharf, Michael <[email-masked]>; 
> [email-masked]
> *Cc:* [email-masked]; [email-masked]
> *Subject:* Re: [tsvwg] draft-ietf-tsvwg-ecn-l4s-id
>
> Michael,
>
>     forward path. You pointing this out has made me realize that there
>     is a 'needs' that should be a 'MUST' in the spec:
>
>        In general,for a transport protocol to providescalable
>     congestion control,
>
>     it MUST providefeedback of the
>
>     transport protocol itself. So AccECN is not a normative ref. So I
>     am also altering the text a little further down, where it refers
>     to AccECN, as follows:
>
>     CURRENT:
>
>     The latter option would create multiple dependencies of one
>     experiment on others. But IMHO such modularity is preferable to a
>     monolithic spec. Especially if some of the components might have
>     other uses outside of TCP Prague (which is exactly the case with
>     TCP Prague, e.g. AccECN, ECN++, RACK are all potentially parts of
>     the base TCP stack, and paced-chirping is potentially usable by
>     any other congestion control - these are all experimental
>     component parts).
>
>     As another example, BBR was written up as an experimental draft,
>     and the Linux implementation uses RACK (although not mentioned in
>     the draft) which is enabled by default in Linux.
>
>     Such modularity is good.
>
>
>     Bob
>
>         Can the authors please clarify whether there is a normative
>         dependency of draft-ietf-tsvwg-ecn-l4s-id on
>         draft-ietf-tcpm-accurate-ecn?
>
>         If so, I think the formal dependency on implementing AccECN
>         has to be spelt out explicitly. In that case, I would be
>         looking for a statement of the sort "MUST/SHOULD/MAY implement
>         draft-ietf-tcpm-accurate-ecn" somewhere. If such a requirement
>         does not exist, I think the above paragraph is confusing.
>
>         As individual contributor, I want to note that I disagree with
>         mandatory dependencies between experiments. I have mentioned
>         in the past that I believe that different experiments must be
>         specified independently of each other and should be
>         implementable independently of each other, specifically also
>         to be able to fail independently of each other. It is possible
>         that I am in the rough part of the consensus. But I haven't
>         changed my mind.
>
>         Michael (as individual)
>
>
>
>     -- 
>
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: "Black, David" <[email-masked]>
Date: Wed, 20 Nov 2019 15:04:39 +0000

Michael Scharf writes:
>> And I still fail to understand why even the general idea of RACK matters to AQM algorithms,
>> traffic classifiers, or priority schedulers for traffic with 1/p congestion control in a router.
>> RACK has lots of merits, but most of them are IMHO entirely unrelated to the vision of L4S
>> I’d suggest to focus L4S on solving the key problems for traffic with 1/p congestion control.

> I've promised David Black a full answer to this. So pls see that.

David Black (as an individual contributor) continues:
I firmly agree with Michael Scharf on this topic, including that RACK has lots of merits.  This is issue #18 in the tracker, and it’s a long-standing issue across multiple IETF meetings …

> But briefly, an important advance of L4S is scalable throughput (it's in the name).
>
> It would be pointless to remove one scaling limitation if it was replaced by the 3 DupACK as a throughput limitation (Amdahl's law).

The full answer will need to address three points to satisfy me …

Point #1 is to explain how and why scalable congestion control requires scalable reordering tolerance, beyond the fact that the word “scalable” has been used twice.  The text in A.1.7 in the -08 version of the draft does not do that, and simply citing Amdahl’s law is not sufficient … the full answer needs to spell out the detailed reasoning.

Point #2 is to explain the scope of the scalable reordering tolerance requirement.  The problem is that the use of “replaced by” in the above sentence appears to be misleading, in that it implies global replacement for the entire Internet.  The text in A.1.7 of the draft indicates that the primary motivation for scalable reordering tolerance is bonded channel links.  Beyond that, this motivation appears to arise only or primarily from RF links (copper or wireless) where the channels that are bonded use different radio frequencies.  In contrast, 3DupACK does not appear to cause problems for optical links, e.g., data center Ethernet, optical WAN and fiber-based ISP technologies.   The upshot is that the scalable reordering tolerance requirement appears to be specific to certain link technologies and hence to certain networks, i.e., is not generally applicable to the Internet as a whole.

Point #3 is to explain why it’s a good thing for the Internet as a whole to bifurcate the relatively new 1/p-class congestion control protocols into those that can be used with L4S low latency service (i.e., TCP Prague, which has scalable reordering tolerance) and those that can’t (everything else, which doesn’t have scalable reordering tolerance).  If the overall goal for the Internet is (or ought to be) to increase use of 1/p-class congestion control, barring most of the current 1/p protocols from the L4S low-latency service seems counterproductive.

> In an ideal world, the two would be done separately. But L4S creates a clean-slate opportunity where
> there's a new queue and new CCs, and there is no downside to placing the time-units requirement
> on these senders that have to be updated anyway.

Hmm – I have severe problems with that “no downside” assertion based on comparing the long period of time that it’s taken to get TCP Prague into it’s almost-done state to how quickly multiple protocols have been modified to support SCE.   Not only is L4S-compatibility of congestion control not a “simple matter of software development” but 1/p-class congestion control functionality also exists in most Ethernet NIC hardware/firmware at 25 Gb/sec and higher speeds.

> Then future expts that might rely on all hosts using time-units will not be precluded.

Future experiments with scalable reordering tolerance ought to be exactly that, future experiments, IMHO.  L4S should be concentrating on scalable Internet congestion control and not be trying to fix other link problems because it is convenient to do so at the same time.

I apologize somewhat for the tone of this message, but my concerns on this topic have been repeatedly dismissed over a number of IETF meetings, and the intent of this message’s tone is to put a stop to that.

Thanks, --David (as an individual contributor)

From: tsvwg <[email-masked]> On Behalf Of Bob Briscoe
Sent: Wednesday, November 20, 2019 4:11 AM
To: Scharf, Michael; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: [tsvwg] [tcpm] draft-ietf-tsvwg-ecn-l4s-id

[EXTERNAL EMAIL]
Michael,

I took your question to be in a context where something else might have replaced AccECN in the future for fine-grained feedback. In that case, L4S would need that new shiny thing instead, and would not then need AccECN.

I've promised David Black a full answer to this. So pls see that.
But briefly, an important advance of L4S is scalable throughput (it's in the name).

It would be pointless to remove one scaling limitation if it was replaced by the 3 DupACK as a throughput limitation (Amdahl's law). In an ideal world, the two would be done separately. But L4S creates a clean-slate opportunity where there's a new queue and new CCs, and there is no downside to placing the time-units requirement on these senders that have to be updated anyway. Then future expts that might rely on all hosts using time-units will not be precluded.

Bob

Michael (with no hat)

From: Bob Briscoe <[email-masked]><mailto:[email-masked]>
Sent: Monday, November 11, 2019 10:39 AM
To: Scharf, Michael <[email-masked]><mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [tsvwg] draft-ietf-tsvwg-ecn-l4s-id

Michael,

Bob

CURRENT:

CURRENT:

Such modularity is good.

Bob

      the other end's choice.

Michael (as individual)

--

--

tcpm mailing list

[email-masked]<mailto:[email-masked]>

--