Subject: [tsvwg] Review comments on a careful read of the L4S ID

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 6 May 2021 07:52:43 +0100

Here are some review comments on a careful read of the L4S ID that I 
hope will help in preparation of a new revision of the draft. More 
details have been sent to the authors.

Best wishes,

Gorry

=================================================================

*1. Abstract needs to be corrected to accommodate ABE (RFC 8511).*

**

Abstract text says:

“ 'Classic' ECN marking is required to be

equivalent to a drop, both when applied in the network and when

responded to by a transport.”

**

⁃That’s no longer completely correct in light of ABE (RFC 8511), 
although the strong connection between this marking and reaction to 
drops is still the case.Perhaps: ‘Classic’ ECN marking is an enhancement 
to drop-based congestion control, both when applied …

=================================================================

*2 Abstract RTT variance*

**

Abstract text says:

“The two changes counterbalance each other so that the

throughput of an L4S flow will be roughly the same as a non-L4S flow

under the same conditions.”

**

⁃Please insert “comparable” before “non-L4S flow” to avoid bogus flow 
comparisons.

**

*3. The Abstract can be seen as marketing. In the past the IESG has 
taken exception to “claims” in an abstract of a standards-track document 
by marketing of “ultra-low”.*

Abstract text says:

so that _ultra-low and consistently low_

queuing delay (typically sub-millisecond on average) becomes possible

for L4S traffic without compromising link utilization.

⁃Please avoid the assertion of “ultra-low” since this seems like a 
marketing brag.

⁃The abstract is still long, and some of the comparison with ‘Classic 
ECN’ could be moved

⁃into the Introduction.

=================================================================

*4. Please be careful with the words here.*

This text:

“It gives an incremental

migration path so that suitably modified network bottlenecks can

distinguish and isolate existing traffic that still follows the

Classic behaviour, to prevent it degrading the low queuing delay and

low loss of L4S traffic.This specification defines the rules that

L4S transports and network elements need to follow_to ensure they_

_neither harm each other's performance nor that of Classic traffic_.“

⁃I suggest removing _"to ensure they_

_neither harm each other's performance nor that of Classic traffic"_

=================================================================

*5. Please be careful with the words here.*

**

This text:

“This specificationdefines _the protocol to be used for_ a new network

service called low latency, low loss and scalable throughput (L4S).”

**

⁃This document does not define a protocol, so the words "_the protocol 
to be used for" _should be removed.

=================================================================

*6. Add text to acknowledge ABE (RFC 8511)*

**

This text:

“RFC 3168 required an ECN mark to be equivalent to a drop, both when 
applied in the

network and when responded to by a transport.”

**

⁃ABE (RFC 8511) has already modified that drop equivalence.Revise this 
text accordingly.

=================================================================

*7. Introduction needs to sidestep RTT variance*

**

This text:

“Thetwo changes counterbalance

each other so that the throughput of an L4S flow will be roughly the

same as a non-L4S flow under the same conditions.”

**

⁃Please insert “comparable” before “non-L4S flow” to avoid bogus flow 
comparisons.

=================================================================

*8. Please be clear throughout that the IETF is NOT endorsing DCTCP 
spec. as an Internet Protocol, even if the underlying basis is important 
to the L4S transport.*

This text:

“An example of a scalable congestion control that would enable the L4S

service is Data Center TCP (DCTCP), _which until now has been_

applicable solely to controlled environments like data centres

[RFC8257], because it is too aggressive to co-exist with existing

TCP-Reno-friendly traffic. “

and later:

“Note that a transport such as DCTCP is

still not safe to deploy on the Internet _unless it satisfies the_

_requirements listed in Section _4.”

and later still:

“cause Classic ECN

congestion controls sharing the same queue to starve themselves,

which is why they have been confined to private data centres or

research testbeds_(until now)_.”

and

“It turns out that a congestion control algorithm like DCTCP that

_solves_ the latency problem also _solves_ the scalability problem of

Classic congestion controls.”

and

“The L4S service is

for more general traffic _than just_ DCTCP--“

The ID later states:

“As with all transport behaviours, a detailed specification (probablyan 
experimental RFC) will need to be defined for each congestion

control, following the guidelines for specifying new congestion

control algorithms in [RFC5033].”

and Annexe A appears to confirm this.

⁃This would be significantly improved by replacing references to DCTCP 
as a protocol with references to the congestion control method/algorithm 
used by DCTCP: RFC8257 is informational and explicitly explained it is 
not EXP.To me this text in the ID provides many contradictions about 
implying DCTCP as a transport for the Internet. That’s something that 
really grates with me and I much prefer the much later statement in the 
IDthat “a detailed specification (probablyan experimental RFC) will need 
to be defined”. If the claim were different, relating to methods based 
on DCTCP, that might be more acceptable.

Making this a reference DCTCP as a CC method would be good to address my 
issue.

=================================================================

*9. More marketing at the end of the Introduction*

This text:

“It gives an incremental migration path so that suitably modified 
network bottlenecks can

distinguish and isolate existing Classic traffic from L4S traffic to 
prevent the former from

degrading the ultra-low delay and loss of the new scalable transports, 
without harming

Classic performance.”

⁃This is better than the corresponding Abstract text (see item 2 above), 
but still contains marketing – “ultra-low” needs to be changed to either 
“low” or “consistently low”. I suggest toi add  “at the modified network 
bottlenecks” after “without harming Classic performance.”to avoid the 
implication that the absence of harm is global (which is known not to be 
the case).

=================================================================

**

*10. Please avoid making a claim that the IETF is NOT making a statement 
about usability of DSCPs in this document.*

This text grates:

“ However, on access links dedicated to individual sites (homes, small

enterprises or mobile devices), often all traffic at any one time

will be latency-sensitive.Then, given nothing to differentiate

from, Diffserv makes no difference.“

⁃To me this is not balanced. It seems to suggest that setting a DSCP is 
useless. I don’t believe that is the consensus of tsvwg, although at 
some additional pain we can debate the merits of setting DSCPs for a 
traffic - such as the implications in enterprise networks; the 
implications on UP in access points, etc. I’m not against this 
discussion, but needing this to progress the draft seems unfortunate to 
me. “Then, given nothing to differentiatefrom, Diffserv makes no 
difference.” is gratuitous and should be removed.The paragraph continues 
with “Instead, we need to remove the causes of any unnecessary delay.” 
which can be combined into the previous sentence as: “ … will be 
latency-sensitive, making it imperative to remove the underlying causes 
of delay.”Nothing is lost here because the crucial comparison with 
Diffserv is picked up two paragraphs later: “Unlike Diffserv, which 
gives low latency to some traffic at the expense of others, AQM controls 
latency for _all_ traffic in a clas

=================================================================

*11.Reference TCP Cubic in addition to TCP Reno*

**

This text:

“ClassicCongestion Control:A congestion control behaviour that can

co-exist with standard TCP Reno [RFC5681] without causing

significantly negative impact on its flow rate [RFC5033].”

**

⁃This is one of a number of references to TCP Reno, which would be 
enhanced by also referencing TCP Cubic to avoid criticisms that 
comparison with TCP Reno is dated and limited.

=================================================================

*12.Remove “Consensus” from Section 2:*

**

This text:

“2._Consensus_ Choice of L4S Packet Identifier: Requirements

This subsection briefly records the process that led to a _consensus_

choice of L4S identifier, selected from all the alternatives in

Appendix B.”

**

⁃Both instances of the word “consensus” are inappropriate and 
incorrect.Both need to be removed, as IETF consensus is a matter for the 
IETF to determine.

**

=================================================================

**

*13. Position with respect to RFC4774 is rather unclear, which motivates 
a case for alternative semantics for carrier experiments*

This text:

“The L4S treatment is an experimental track alternative packet marking

treatment [RFC4774]”

•RFC4774 states: “The assumption of this document is that when alternate 
semantics are defined for the ECN field, a codepoint in the diffserv 
field is used to signal the use of these alternate ECN semantics to the 
router.” This text and position needs to be clarified, and is being 
discussed in a tsvwg email thread.

=================================================================

*14. Replace “network node”  (part 1)*

**

This text:

“ A network node that implements the L4S service always classifies

arriving ECT(1) packets for L4S treatment and by default classifies

CE packets for L4S treatment unless the heuristics described in

Section 5.3 are employed.”

⁃Use of “network node” is excessive and over-constrains implementations, 
e.g., the techniques in Sections 5.4.1.2 and 5.4.1.3 conflict with this 
sentence.Change “A network node that implements” to “An implementation 
of the L4S service” It would be nice to be at least clear this is 
marking behaviour, e.g. “An implementation that marks using L4S”, or 
something similar.

=================================================================

*15. Avoid attempting to predict the future: don’t use will.*

**

This text:

“ As with all transport behaviours, a detailed specification (_probably_

an experimental RFC) _will_ need to be defined for each congestion

control, following the guidelines for specifying new congestion

control algorithms in [RFC5033].In addition it _will_ need to

document these L4S-specific matters, specifically the timescale over

which the proportionality is averaged, and control of burstiness.”

⁃Please say why this is important to document, we have similar 
requirements in other transport specs.

⁃Reword in present tense - remove both instances of “will” and say “needs”

⁃Replace “probably” with “e.g.,”

=================================================================

*16. RFC 3168 coexistence*

**

This text:

“oA scalable congestion control MUST implement monitoring in order

to detect a likely non-L4S but ECN-capable AQM at the bottleneck.

On detection of a likely ECN-capable bottleneck it SHOULD be

capable (dependent on configuration) of automatically adapting its

congestion response to coexist with TCP Reno congestion controls

[RFC5681] (see Appendix A.1.4 for rationale and a referenced

algorithm).”

⁃This will have to be resolved in the WG, including details of what 
“coexist” means.

=================================================================

*17. RTT Bias: **implementation requirements to documentation requirements*

This text:

“A scalable congestion control MUST eliminate RTT bias as much as

possible in the range between the minimum likely RTT and typical

RTTs expected in the intended deployment scenario (see

Appendix A.1.5 for rationale).”

⁃A writing style that says: must... as hard as possible style is very 
hard to accept for me. Use of an RFC 2119 “MUST” is not appropriate 
here.Please do change “MUST” to “is expected to” as used in the last 
bulleted item. One possibility that would work for me is some rephrasing 
to say “needs to eliminate” and “specifications are REQUIRED to include 
methods that limit RTT bias, although I have no problem with saying “is 
expected to …”.

=================================================================

*18. Loss detection in time-based units.*

This text:

“oA scalable congestion control SHOULD detect loss by counting in

time-based units, which is scalable, as opposed to counting in

units of packets (as in the 3 DupACK rule of RFC 5681 TCP), which

is not scalable.As packet rates increase (e.g., due to new and/

or improved technology), congestion controls that detect loss by

counting in units of packets become more likely to incorrectly

treat reordering events as congestion-caused loss events (see

Appendix A.1.7 for further rationale).This requirement does not

apply to congestion controls that are solely used in controlled

environments where the network introduces hardly any reordering.”

⁃Please replace with text agreed to on the list.

=================================================================

*19. SHOULD document?*

In section 4.4:

“Nonetheless, the specification

of a particular L4S congestion control SHOULD describe how it smooths

the L4S ECN signals fed back to it from the receiver.”

⁃Why is this a RFC2119 SHOULD? I might be OK with requiring for a reason 
- or requiring because it is needed to explain how something is 
prevented, etc, but requiring to document seems very odd to me. Please 
justify or make it not RFC2119; or say SHOULD specify? I suggest do not 
use an RFC 2119 keyword herr or could just say “SHOULD smooth the L4S 
ECN signals fed back to it from the receiver” to talk about what an 
algorithm should do rather than what a document should say.

=================================================================

*20.Reduce “network node” scope (part 2)*

**

In section 5.1:

“A network node that implements the L4S service MUST classify arriving

ECT(1) packets for L4S treatment and, other than in the exceptional

case referred to next, it MUST classify arriving CE packets for L4S

treatment as well.”

⁃Please change to “An implementation of the L4S service MUST classify 
…”This removes conflicts with at least Sections 5.4.1.2 and 5.4.1.3.

=================================================================

*21. Reduce “network node” scope (part 3)*

This text:

“ For backward compatibility in uncontrolled environments, a network

node that implements the L4S treatment MUST also implement an AQM

treatment for the Classic service as defined in Section 1.2.”

⁃Change to “an implementation of the L4S service that supports the L4S 
treatment MUST also implement …”This removes conflicts with at least 
Sections 5.4.1.2 and 5.4.1.3.

=================================================================

*22. Update to include ABE (RFC 8511)*

**

This text:

“Note that, contrary to RFC 3168, a Dual Queue Coupled AQM

implementing the L4S and Classic treatments does not mark an ECT(1)

packet under the same conditions that it would have dropped a Not-ECT

packet, as allowed by [RFC8311], which updates RFC 3168.However, if

it marks ECT(0) packets, it does so under the same conditions that it

would have dropped a Not-ECT packet.”

**

⁃ABE (RFC 8511) has modified that drop equivalence.Revise this text 
accordingly.

=================================================================

*23. What about if a sender uses Not-ECT and ECT(0) in combination? 
Also, reduce “network node” scope (part 4)*

This text:

“Nonetheless, if an implementer is

willing to identify transport-layer flows at a network node, and if

the most recent ECT packet in the same flow was ECT(0), the node MAY

classify CE packets for Classic ECN [RFC3168] treatment.”

⁃Please tell us if you have thought about when the previous packet was 
not-ECT. Has this been considered and is it explicitly required to then 
send a CE mark via the L4S queue? I understand the next para to speak 
only about when next packet was ECT(1).

⁃See other note on “at a network node” and change “the node may” to “the 
implementation MAY”.This removes conflicts with at least Sections 
5.4.1.2 and 5.4.1.3.

⁃It would also help to change “if an implementer is willing” to “if an 
implementation is able”.

⁃If the latter is done, change “If an implementer uses” to “If an 
implementation uses” at the start of the next paragraph.

=================================================================

*24. Requirements in 5.4.1.*

“Such

non-ECN-based packet types _MUST be safe to mix with L4S traffic_

_without harming_the low latency service, where 'safe' is explained in

Section 5.4.1.1.1 below.”

⁃It is not clear what the MUST relates to, I think this needs to be more 
like (you’d still need to explain harm): “Non-L4S packets that are mixed 
with L4S traffic MUST NOT harm the low latency service, where 'safe' is 
explained in Section 5.4.1.1.1 below. and… Consider use of “SHOULD” 
here, as the consequences of not following it are stated, but not the 
current “MUST”.

“Therefore, a network element that

implements L4S in a shared queue MAY classify additional packets into

the L queue if they carry certain non-ECN identifiers.”

⁃This later sentence appears to be a requirement on the same thing, but 
isn’t the same

⁃“MAY classify additional packets” -> “MAY classify additional ‘safe’ 
packets” would provide consistency.

=================================================================

*25. Missing drop-only requirement for excluded traffic*

**

Section 5.4.1.2.Exclusion of Traffic From L4S Treatment – this text:

“The operator MUST NOT alter the end-to-end L4S ECN identifier from

L4S to Classic, because its decision to exclude certain traffic from

L4S treatment is local-only.”

**

⁃I think be the word /its/this/ .

⁃Please add a requirement that ECT(1) traffic excluded from L4S 
treatment MUST be handled as non-ECN traffic (e.g., all congestion 
signalling is via drops), as Classic AQM treatment and ECN marking 
produce the wrong results for such traffic.

=================================================================

*26. L4S Diffserv draft is not being pursued*

**

“[I-D.briscoe-tsvwg-l4s-diffserv] gives a number of examples of such

arrangements to address various requirements. 
[I-D.briscoe-tsvwg-l4s-diffserv]

describes how an operator might use L4S to offer low latency for all L4S 
traffic as well as

using Diffserv for bandwidth differentiation.It identifies two main types

of approach, …”

**

⁃If [I-D.briscoe-tsvwg-l4s-diffserv] is not being pursued, this is not a 
useful reference in a spec RFC.

References to it should be removed by rewriting the text, e.g., “There 
are two primary types of approaches, …”

=================================================================

*27. Appendix B and C
*

**

⁃Appendix B is lengthy, at best loosely connected with the rest of the 
draft, definitely out of date, and not aligned with current WG views of 
this area. This appendix can be removed prior to publication. Appendix C 
is unecessary to the final publication.

*These points below are Nits, but if you don’t think so, we can discuss 
more?*

**

=================================================================

**

*A. Applications?*

This text:

“it SHOULD survive end-to-end between source and destination applications”

⁃Unclear what applications mean here?

=================================================================

*B. List style uses semicolon at end of item?*

“oit SHOULD be visible at the IP layer”

⁃add semicolon to match style of list.

=================================================================

*C. Seems very similar, does it need to be repeated?*

This para in section 5.2, seems very similar to the para in section 5.1.

“Note that, contrary to RFC 3168, a Dual Queue Coupled AQM

implementing the L4S and Classic treatments does not mark an ECT(1)

packet under the same conditions that it would have dropped a Not-ECT

packet, as allowed by [RFC8311], which updates RFC 3168.However, if

it marks ECT(0) packets, it does so under the same conditions that it

would have dropped a Not-ECT packet.”

⁃Seems very similar, does it need to be repeated? Can this text be 
reduced/removed?

=================================================================

*D. Seems very similar, does it need to be repeated?*

In sect 5.2, can we remove:

“Also, L4S CE marking needs to be interpreted as an unsmoothed signal,

in contrast to the Classic approach in which AQMs filter out

variations before signalling congestion.“

⁃Could this simply refer to section 4.4?

=================================================================

*E. I don’t understand this:*

“_(or perhaps they comply with L4S behaviour and can respond to ECN_

_feedback, but perhaps cannot set the ECN field for some reason)_”

⁃What is the extra point here in brackets? and how can the flow be 
compliant but not set ECT(1) and so doesn’t respond… This all seems like 
a difficult point to pursue - or maybe it simply does not need to be 
said at all. Can we delete this parenthesis?

=================================================================

*F. NiT: lightweight*

“certain protocols that are usually _lightweight_

“

⁃I think we have elsewhere called these “Low Data-Volume Applications”. 
The term lightweight is used in other places in the IETF for other meanings.

=================================================================

*G. What does this sentence mean, and why say it?*

**

*“*Of course, a packet that carried both the ECT(1) codepoint and a non-

ECN identifier associated with the L queue would be classified into

the L queue.”

⁃This confuses me - is it saying that NQB would be classified to the L 
queue, even if ECT(1)-marked, which isn’t something you talk about. or 
is it implying more. If this is what you intend, I think it would be 
much clearer to say nothing, or explain more.

⁃The following paras are also maybe to exhaustive, maybe it is rather 
worth writing this simply.

=================================================================

*H. List style uses semicolon at end of item?*

In sect 6.2, and6.3. Theitems in the bullet list do not end in 
semi-colons; but do elsewhere.

=================================================================

*I. List style uses semicolon at end of item?*

Editorial Annex 1:

“ But they cannot

easily satisfy this loss recovery requirement._However, it is_

_believed they do not need to. _It is believed that such

implementations solely exist in controlled environments,…“

⁃GF: is the sentence with “however...” strictly required here? I fear it 
could be mistakenly read, and without it, the next sentence is clear 
about what is intended.

=================================================================

*J. Please don’t assume what TCPM will publish.*

“This means these packets

are not protected from congestion loss by ECN, which considerably

harms performance, particularly for short flows.

[I-D.ietf-tcpm-generalized-ecn] _counters each argument in RFC 3168 in_

_turn, showing it was over-cautious.Instead it _proposes experimental

use of ECN on all types of TCP packet as long as AccECN feedback

[I-D.ietf-tcpm-accurate-ecn] is available (which itself satisfies the

accurate feedback requirement in Section 4.2 for using a scalable

congestion control).”

⁃I don’t like this interpretation of what TCPM will finally conclude is 
useful. It’s OK to say this proposes something, if you want to say it 
concludes something then we need to block publication on the prior 
publication of the other ID. Can you please please remove “counters each 
argument in RFC 3168 in turn, showing it was over-cautious.Instead it “

=================================================================

*K. Typo?*

“to 4 4 Mb/s”

⁃is this correct?

---

From: Sebastian Moeller <[email-masked]>
Date: Thu, 6 May 2021 13:47:42 +0200

Hi Gorry,

see [SM] below for a question.

> On May 6, 2021, at 08:52, Gorry Fairhurst <[email-masked]> wrote:
> 
> 
> Here are some review comments on a careful read of the L4S ID that I hope will help in preparation of a new revision of the draft. More details have been sent to the authors.
> 
> Best wishes,
> 
> Gorry
> [...]
> 25. Missing drop-only requirement for excluded traffic
>  
> Section 5.4.1.2.  Exclusion of Traffic From L4S Treatment – this text:
>  
> ⁃           I think be the word /its/this/ .
> ⁃            Please add a requirement that ECT(1) traffic excluded from L4S treatment MUST be handled as non-ECN traffic (e.g., all congestion signalling is via drops), as Classic AQM treatment and ECN marking produce the wrong results for such traffic.
[...]

We actually recommend doing exactly that as a measure of last resorts in https://tools.ietf.org/html/draft-white-tsvwg-l4sops-02#page-11
We could:
a) simply accept that divergence
b) add text to the ECN-ID draft to explicitly allow that for extreme situations (or reduce the MUST to a SHOULD, pointing out the cost of doing so)
c) add text to the OPs draft why that violation of the ECN-ID draft is acceptable as a measure of last resort...

Given that this is the final back-stop in the (unlikely) case that the L4S experiment should cause the next congestion collapse I think it made sense to tackle this explicitly, no?

Best Regards
	Sebastian

---

From: Tom Henderson <[email-masked]>
Date: Thu, 6 May 2021 10:19:15 -0700

Gorry, on this point, I made the original suggestion to call this a 
protocol document during the -13 review (email to the list on March 7); 
please see below my original comment regarding this.

- Tom

 >
 > Title
 >
 > The title of this draft suggests that the scope is narrowly defining
 > the identifier of L4S semantics, but the draft covers much more than
 > this; in fact, it perhaps it could more accurately be described as an
 > L4S protocol specification.  At the end of the abstract, the draft
 > states "This specification defines the rules that L4S transports and
 > network elements need to follow...", i.e. a protocol.  It also gets
 > into operational considerations and open questions for experimentation.
 >
 > Perhaps a broader title such as "Explicit Congestion Notification
 > (ECN) Protocol for Ultra-Low Queuing Delay (L4S)" would better match
 > the contents.

---

From: "Black, David" <[email-masked]>
Date: Thu, 6 May 2021 20:30:23 +0000

Tom,

I'm the source of pushback here.  In my view, L4S is an interesting mix, as the L4S ID draft does not define a complete protocol - rather, it specifies the ECN marking mechanism and places requirements on the endpoint congestion control response without specifying that response in detail (e.g., to implement TCP Prague congestion control based on L4S, one also needs to also go look at a TCP Prague spec).

I'd be happy with "mechanism" or "functionality" but I don't see a fully implementable "protocol" here.  What do you think?

Thanks, --David

-----Original Message-----
From: tsvwg <[email-masked]> On Behalf Of Tom Henderson
Sent: Thursday, May 6, 2021 1:19 PM
To: Gorry Fairhurst
Cc: [email-masked]
Subject: Re: [tsvwg] Review comments on a careful read of the L4S ID

[EXTERNAL EMAIL] 

- Tom

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 6 May 2021 21:39:12 +0100

Tom/David,

Aha - I see now. Another possibility could be something like:

"Explicit Congestion Notification (ECN) Protocol for Ultra-Low Queuing Delay (L4S)" ??

Gorry

On 06/05/2021 21:30, Black, David wrote:
> Explicit Congestion Notification
>   > (ECN) Protocol for Ultra-Low Queuing Delay (L4S)
>
>   >
>   > Title
>   >
>   > Perhaps a broader title such as "" would better match
>   > the contents.
>

---

From: "Black, David" <[email-masked]>
Date: Thu, 6 May 2021 21:25:57 +0000

Hi Sebastian,

Something does indeed need to be done about this, as there's an even bigger problem elsewhere in the L4S ID.

There are (at least) two different reasons for not applying L4S AQM treatment to ECT(1) marked traffic: 
	- Policy-based exclusion (e.g., Section 5.4.1.2 of the L4S ID draft).
	- Operational exclusion (e.g., L4S OPS draft).
The L4S ID needs to allow both reasons for excluding ECT(1)-marked traffic from L4S AQM treatment.

Unfortunately, the L4S ID currently prohibits both exclusions due to an unintended consequence of the first sentence of Section 5.1:

   A network node that implements the L4S service MUST classify arriving ECT(1) packets for L4S treatment ...

My understanding is that the L4S authors are going to rewrite that sentence to correct this oversight, and check for other places where this problem may occur.  That rewrite ought to cover both exclusion cases listed above.

Thanks, --David

-----Original Message-----
From: tsvwg <[email-masked]> On Behalf Of Sebastian Moeller
Sent: Thursday, May 6, 2021 7:48 AM
To: Gorry Fairhurst
Cc: [email-masked]
Subject: Re: [tsvwg] Review comments on a careful read of the L4S ID

[EXTERNAL EMAIL] 

Hi Gorry,

[...]

We actually recommend doing exactly that as a measure of last resorts in https://urldefense.com/v3/__https://tools.ietf.org/html/draft-white-tsvwg-l4sops-02*page-11__;Iw!!LpKI!1yLUhTX-HOwh-wMGgmzGrZngRafqgEyTPQoamHkJO0n_ZR1k6R0cJ2RFGzexjN3L$ [tools[.]ietf[.]org]
We could:
a) simply accept that divergence
b) add text to the ECN-ID draft to explicitly allow that for extreme situations (or reduce the MUST to a SHOULD, pointing out the cost of doing so)
c) add text to the OPs draft why that violation of the ECN-ID draft is acceptable as a measure of last resort...

Best Regards
	Sebastian

---

From: Tom Henderson <[email-masked]>
Date: Thu, 6 May 2021 16:53:39 -0700

> I'd be happy with "mechanism" or "functionality" but I don't see a fully implementable "protocol" here.  What do you think?

David,
While I agree that congestion control is not specified in detail, the 
title doesn't currently claim that it does.  It seems to me that 
sections 4 and 5 define the protocol rules for the use of ECN.  My 
earlier email suggested what more I thought could be included in those 
sections, but I'm not sure whether the authors intend to pick up those 
comments.

- Tom

---

From: Sebastian Moeller <[email-masked]>
Date: Fri, 07 May 2021 08:34:13 +0200

Gorry,

>"Explicit Congestion Notification (ECN) Protocol for Ultra-Low Queuing

[SM] I thought "ultra-low" is going to be replaced with "consistently low"?

Also if the ecn-id draft is considered a specification for protocols I very much would expect to see at least one fully functioning mostly-finished actual protocol being developed from that spec BEFORE standardisation as proof thar the spec itself is sufficiently complete and functional, no?
And TCP Prague with all its unfinished/partly abandoned? attempts at implementing the 'specification's' requirements is not that convincing that the spec is complete and reasonably implementable.
To be explicit, neither RTT-debiasing, nor rfc3168 detection are robustly and reliably tackled which is somewhat sobering given that the requirements exist for IIRC more than 5 years now...

Regards
        P.

>> I'm the source of pushback here.  In my view, L4S is an interesting
>mix, as the L4S ID draft does not define a complete protocol - rather,
>it specifies the ECN marking mechanism and places requirements on the
>endpoint congestion control response without specifying that response
>in detail (e.g., to implement TCP Prague congestion control based on
>L4S, one also needs to also go look at a TCP Prague spec).
>>
>> I'd be happy with "mechanism" or "functionality" but I don't see a
>fully implementable "protocol" here.  What do you think?
>>
>> Thanks, --David
>>

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 7 May 2021 10:32:32 +0100

Thanks Sebastian,

Please se below marked [GF].

> [SM] I thought "ultra-low" is going to be replaced with "consistently low"?

[GF] My comment was to ensure the abstract and title describe the goal 
of this architecture. (In terms of SLAs, operators often don't provide 
hard 100% guarentees, but do generally express the key aspects of their 
intended service class.) In this sense, "Ultra" seems like a marketing 
term, and I would expect that technical specifications avoid this word.

> Also if the ecn-id draft is considered a specification for protocols I very much would expect to see at least one fully functioning mostly-finished actual protocol being developed from that spec BE…

[GF] I'm not myself concerned about the Spec being implemented before it 
is published - but it was mist helpful to see implementor feedback on 
whether they understand the text. The IETF has published specs before 
and then after that the details have developed after the first round of 
documents were RFCs. In particular, any EXP spec would naturally be 
followed by another IETF spec that further details what is needed.

> To be explicit, neither RTT-debiasing, nor rfc3168 detection are robustly and reliably tackled which is somewhat sobering given that the requirements exist for IIRC more than 5 years now...
[GF] I know in this period that support for DualQ has been added to some 
equipment - albeit while waiting for RFCs to finalise details.

I do expect that many larger endpoint implementors also do a lot of 
their initial development based on published RFCs, but also do 
prototyping of the details of algorithms in their various (often 
private) implementations. That appears to be very much the way of 
operation in working groups like QUIC. (QUIC has defined the fundamental 
feedback mechanisms needed for L4S). I'd love to see implementations 
emerge for TCP also.

The topic of RTT-debiasing always has been a really good direction, and 
remains so. I think that TSVWG would welcome more insight on this, and 
surely if people want to bring methods and experience to the WG, I'd 
personally encourage that.

So looking forward after the Specs are publsihed... I think network 
(router) deployment of the original ECN has to date been limited, this 
situation needs to change if L4S techniques are to be widley used. If 
there is wider support for L4S, then the incentives exist to get the 
protocol details correct, and my hope if that the implementors can bring 
their experience to the IETF.

Best wishes,

Gorry

---

From: "Rodney W. Grimes" <[email-masked]>
Date: Fri, 7 May 2021 05:23:03 -0700 (PDT)

Gorry, 

> "Explicit Congestion Notification (ECN) Protocol for Ultra-Low Queuing Delay (L4S)" ??
                                          ^^^^         ^^^^
May I remind you that you have rejected the use of the word/phrase
"Ultra-Low" as marketing hype  and also rejected this draft as a Protocol
spec.

Stealing this from RFC 3168:
	"The Addition of Explicit Congestion Notification (ECN) to IP"

I derive this:

	"The Modification of Explicit Congestion Notification (ECN) in IP"

As far as I descern here L4S signalling is a modification of the
Internet Protocol (IP), as it has been modified by RFC3168, ECN.

Another might be:
	"The Redefinition of the Explicit Congestion Notification (ECN)
	Congestion Experienced (CE) signal"

For me this last one is more precise about what this draft is attempting,
but is a bit long..

Regards,
Rod
> >> ?This specificationdefines _the protocol to be used for_ a new network
> >>
> >> service called low latency, low loss and scalable throughput (L4S).?
> >>
> >> **
> >>
> >> ?This document does not define a protocol, so the words "_the protocol

-- 
Rod Grimes                                                 [email-masked]

---

From: Sebastian Moeller <[email-masked]>
Date: Fri, 7 May 2021 16:45:44 +0200

Hi Gorry,

see [SM2] below.

> On May 7, 2021, at 11:32, Gorry Fairhurst <[email-masked]> wrote:
> 
> Thanks Sebastian,
> 
> [GF] My comment was to ensure the abstract and title describe the goal of this architecture. (In terms of SLAs, operators often don't provide hard 100% guarentees, but do generally express the key aspects of their intended service class.) In this sense, "Ultra" seems like a marketing term, and I would expect that technical specifications avoid this word.
> 
> [GF] I'm not myself concerned about the Spec being implemented before it is published - but it was mist helpful to see implementor feedback on whether they understand the text. The IETF has published specs before and then after that the details have developed after the first round of documents were RFCs. In particular, any EXP spec would naturally be followed by another IETF spec that further details what is needed.

	[SM2] An implementation is the best way to demonstrate that a spec is actually implementable and to assess how much of the expected behavior actually transfers to the real word. The rfc3168 example demonstrates how hard it is to fix/change such behavior once nodes are deployed in the field.

> [GF] I know in this period that support for DualQ has been added to some equipment - albeit while waiting for RFCs to finalise details.

	[SM2] Is an untested implementation really proof of anything, though? I explicitly asked on list whether industry members had private data convincing them of safety and functionality of L4S' design. There was more or less zero response to that question, mind you I did not ask for data to be revealed, just whether people had seen subjectively convincing data.

> I do expect that many larger endpoint implementors also do a lot of their initial development based on published RFCs,

	[SM2] One more reason to make sure that published RFCs are actually implementable before ratification, IMHO. Unless you are convinced the TCP Prague proofs both sufficient safety and functionality we have no indication that the L4S design will be implementable to the advantage of the internet.

> but also do prototyping of the details of algorithms in their various (often private) implementations.

	[SM2] I am fine with private implementations, and also fine for the time being for developers to keep a low profile on the details, but asking "is L4S safe and functional according to your internal assessment" seems respectful of the desire for secrecy, no?

> That appears to be very much the way of operation in working groups like QUIC. (QUIC has defined the fundamental feedback mechanisms needed for L4S).

	[SM2] But QUICK is missing quite a number of the other requirements for a L4S-compatible protocol, so can clearly not be used for open-internet testing at the current time, no?

> I'd love to see implementations emerge for TCP also.

	[SM2] So we agree that neither DCTCP nor TCP Prague count here, eh? ;) Partly kidding...

> The topic of RTT-debiasing always has been a really good direction, and remains so. I think that TSVWG would welcome more insight on this, and surely if people want to bring methods and experience to the WG, I'd personally encourage that.

	[SM2] In theory I agree, yet the fact remains that L4S' AQM increases RTT bias between C and L queue traffic and that the reference L4S protocol TCP Prague displays quite large RTT-bias with itself in the ubiquitous FIFO bottleneck. In other words I agree that working against RTT-bias is a good direction, but L4S does not improve that situation (in fact it makes it worse, as can be seen in e.g. Pete's data).

> So looking forward after the Specs are publsihed... I think network (router) deployment of the original ECN has to date been limited, this situation needs to change if L4S techniques are to be widley used. If there is wider support for L4S, then the incentives exist to get the protocol details correct, and my hope if that the implementors can bring their experience to the IETF.

	[SM2] This assumes that the network bits of L4S as designed right now are sufficiently functional and safe, and that short-comings in their design can be "fixed" on the protocol side. In fact quite a number of these short-comings are already (indirectly) documented in the L4S protocol requirements already, like RTT-bias and burst-intolerance. I fail to see the rush, so let's fix the next ECN-AQM contender before we try to deploy it en-masse, please?

Best Regards
	Sebastian

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 7 May 2021 22:35:46 +0100

Sebastien,

> 	[SM2] But QUICK is missing quite a number of the other requirements for a L4S-compatible protocol, so can clearly not be used for open-internet testing at the current time, no?

[BB] What exactly do you believe is missing?

If you add a scalable congestion control at the sender, by default from 
v1 of the IETF QUIC protocol, it provides the necessary accurate ECN 
feedback (and of course loss feedback) with no alterations required to 
the protocol.

See for instance https://github.com/qdeconinck/picoquic/tree/quic-prague 
where Quenten De Coninck added the Prague CC as it was at the time to 
QUIC in a couple of days at a hackathon (not maintained since, but it at 
least proved that QUIC had all the necessary facilities).

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Sebastian Moeller <[email-masked]>
Date: Sat, 8 May 2021 00:25:59 +0200

Hi Bob,

> On May 7, 2021, at 23:35, Bob Briscoe <[email-masked]> wrote:
> 
> Sebastien,
> 

	Since so far all that QUIC has implemented is a back-channel for the received ECN code points at the receiver, I naively? assume the following laundry list is not confirmed to be operational:

   o  A scalable congestion control MUST be capable of being replaced by
      a Classic congestion control (by application and/or by
      administrative control).  If a Classic congestion control is
      activated, it will not tag its packets with the ECT(1) codepoint
      (see Appendix A.1.3 for rationale).

   o  As well as responding to ECN markings, a scalable congestion
      control MUST react to packet loss in a way that will coexist
      safely with Classic congestion controls such as standard Reno
      [RFC5681], as required by [RFC5033] (see Appendix A.1.4 for
      rationale).

   o  In uncontrolled environments, monitoring MUST be implemented to
      support detection of problems with an ECN-capable AQM at the path
      bottleneck that appears not to support L4S and might be in a
      shared queue.  Such monitoring SHOULD be applied to live traffic
      that is using Scalable congestion control.  Alternatively,
      monitoring need not be applied to live traffic, if monitoring has
      been arranged to cover the paths that live traffic takes through
      uncontrolled environments.

      The detection function SHOULD be capable of making the congestion
      control adapt its ECN-marking response to coexist safely with
      Classic congestion controls such as standard Reno [RFC5681], as
      required by [RFC5033].  Alternatively, if adaptation is not
      implemented and problems with such an AQM are detected, the
      scalable congestion control MUST be replaced by a Classic
      congestion control.

      Note that a scalable congestion control is not expected to change
      to setting ECT(0) while it transiently adapts to coexist with
      Reno.

      See Appendix A.1.5 and [I-D.ietf-tsvwg-l4sops] for rationale.

   o  A scalable congestion control MUST reduce RTT bias as much as
      possible in the range between the minimum likely RTT and typical
      RTTs expected in the intended deployment scenario (see Appendix A.1.6
      for rationale).

   o  A scalable congestion control SHOULD remain responsive to
      congestion when typical RTTs over the public Internet are
      significantly smaller because they are no longer inflated by
      queuing delay.  It would be preferable for the minimum window of a
      scalable congestion control to be lower than 1 segment rather than
      use the timeout approach described for TCP in S.6.1.2 of [RFC3168]
      (or an equivalent for other transports).  However, a lower minimum
      is not set as a formal requirement for L4S experiments (see Appendix A.1.7
      for rationale).

   o  A scalable congestion control's loss detection SHOULD be resilient
      to reordering over an adaptive time interval that scales with
      throughput and adapts to reordering (as in [RFC8985]), as opposed
      to counting only in fixed units of packets (as in the 3 DupACK
      rule of [RFC5681] and [RFC6675], which is not scalable).  As
      packet rates increase (e.g., due to new and/or improved
      technology), congestion controls that detect loss by counting in
      units of packets become more likely to incorrectly treat
      reordering events as congestion-caused loss events (see Appendix A.1.8
      for further rationale).  This requirement does not
      apply to congestion controls that are solely used in controlled
      environments where the network introduces hardly any reordering.

   o  A scalable congestion control is expected to limit the queue
      caused by bursts of packets.  It would not seem necessary to set
      the limit any lower than 10% of the minimum RTT expected in a
      typical deployment (e.g. additional queuing of roughly 250 us for
      the public Internet).  This would be converted to a number of
      packets under the worst-case assumption that the bottleneck link
      capacity equals the current flow rate.  No normative requirement
      to limit bursts is given here and, until there is more industry
      experience from the L4S experiment, it is not even known whether
      one is needed - it seems to be in an L4S sender's self-interest to
      limit bursts.

https://datatracker.ietf.org/doc/html/draft-ietf-quic-recovery-34#section-7 seems to confirm that QUIC currently implements rfc3168 style CE response, with the possibility of exchanging the sender CC algorithm. But I see no L4S "module" for quic. If you know where I can find the implementation of the above laundry list for QUIC,  point to the right UL, please.

> 
> If you add a scalable congestion control at the sender, by default from v1 of the IETF QUIC protocol, it provides the necessary accurate ECN feedback (and of course loss feedback) with no alterations required to the protocol.

	[SM] Potentially a nomenclature mismatch, when I say protocol, I do not speak only about the on-the-wire headers, but also the specification what to do with the received information so I include the CC algorithm as part of the protocol. If protocol is the wrong word, I apologize. But yeah we seem to agree, to fulfill the L4S protocol requirements QUIC needs to grow a compliant L4S-CC, in other words currently that part is missing in QUIC.

> See for instance https://github.com/qdeconinck/picoquic/tree/quic-prague where Quenten De Coninck added the Prague CC as it was at the time to QUIC in a couple of days at a hackathon (not maintained since, but it at least proved that QUIC had all the necessary facilities).

	Well, see, IMHO current TCP Prague does not fully fulfill the L4S protocol-requirements, so I am not convinced, that an abandoned port of an older version of TCP Prague without either the the WIP RTT-debias ot rfc3168 detection/fall-back components, will count as proof that QUIC is an L4S-compatible protocol today. I do not claim that making n L4S-aware QUIC version is impossible, but simply that current QUIC is not that version (yet?).

Best Regards
	Sebastian

P.S.: I must be missing something here...

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 7 May 2021 23:36:03 +0100

Tom, David,

On 07/05/2021 00:53, Tom Henderson wrote:
>> mix, as the L4S ID draft does not define a complete protocol - 
>> rather, it specifies the ECN marking mechanism and places 
>> requirements on the endpoint congestion control response without 
>> specifying that response in detail (e.g., to implement TCP Prague 
>> congestion control based on L4S, one also needs to also go look at a 
>> TCP Prague spec).
>>

[BB] I'll pick up your comments in later email. But keeping to the 
question of whether ecn-l4s-id defines a protocol...

You might not be able to implement a protocol without its algorithms, 
but that doesn't mean the algorithms have to be standardized as part of 
a protocol. It is still a valid protocol if the algorithms are expected 
to be contributed by implementers.

For instance, the 3GPP defines a protocol to request upstream capacity. 
These requests are directed at the scheduling algorithm which gives out 
grants of capacity. Developers are not required to reveal their 
scheduling algorithms if they prefer to keep them as secret sauce, and 
there is no scheduling algorithm defined in the specs. So the spec's are 
not implementable without adding some of your own knowledge. But the 
spec's still define a request-grant protocol.

ecn-l4s-id defines a modification to the ECN protocol for L4S. The wire 
protocol is very simple, but that doesn't mean it's not a protocol. It 
sets requirements for an implementation of CC and of AQM, but doesn't 
define either in detail, although it refers to examples. That's a 
protocol, surely.

I'm not going to push this one. We can go back to the previous title if 
preferred, which was
     Identifying Modified Explicit Congestion Notification (ECN) 
Semantics for *****-Low Queuing Delay (L4S)
         (The ***** is bleeped out, so it doesn't send this thread off 
into a rat-hole - I'll be responding to that point separately.)

I just preferred Tom's suggestion of calling this a protocol, 'cos it's 
accurate and it's less pompous than calling it semantics (which was my 
choice originally, so I'm criticizing only myself).

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Sat, 8 May 2021 00:25:41 +0100

David, Sebastian, Gorry, and adding Greg as author of l4sops for the 
last point inline below, see [BB]...

> My understanding is that the L4S authors are going to rewrite that sentence to correct this oversight, and check for other places where this problem may occur.  That rewrite ought to cover both exc…

[BB] Following you pointing this out, we were indeed going to add that 
the rule to classify ECT(1) as L4S was subject to policy exceptions (as 
in S.5.4.1.2).

For the l4sops draft point, see my response embedded below to Sebastian, 
plus some others while I'm at it...

>> Here are some review comments on a careful read of the L4S ID that I hope will help in preparation of a new revision of the draft. More details have been sent to the authors.
[BB] Possibly, but an operator is an it, and it can make decisions. 
'Its' seems better to my English ear than 'this'. I'll leave it.

>> ⁃            Please add a requirement that ECT(1) traffic excluded from L4S treatment MUST be handled as non-ECN traffic (e.g., all congestion signalling is via drops), as Classic AQM treatment an…

[BB] Good point. Here's the new para I've added at the end of the section:

An operator that excludes traffic carrying the L4S identifier from L4S 
treatment MUST NOT treat such traffic as if it carries the ECT(0) 
codepoint, which could confuse the sender.

> c) add text to the OPs draft why that violation of the ECN-ID draft is acceptable as a measure of last resort...

[BB] Thanks for pointing this out.

ecn-l4s-id is about what L4S nodes do, whereas l4sops is about what 
non-L4S nodes might do. So l4sops is not strictly contrary to 
ecn-l4s-id. However, that's splitting hairs, because l4sops is about 
trying to help 3168 and L4S nodes to be more compatible. So it is not 
particularly compatible to be inconsistent.

When I sent my review of l4sops to the list recently, I said I didn't 
like the mention of bleaching. Quoting:
     "For both 5.5. & 5.6., I think it's necessary to say why we are 
even mentioning these options, given they are not recommended."

An alternative to your a) b) c) would be
d) not to mention this in l4sops. I'd support that.

If 5.6 remains in l4sops, I'd support your c).

Bob

---

From: "Black, David" <[email-masked]>
Date: Sat, 8 May 2021 01:30:52 +0000

Bob,

> I'm not going to push this one. We can go back to the previous title if preferred, which was
>      Identifying Modified Explicit Congestion Notification (ECN) Semantics for *****-Low Queuing Delay (L4S)
>          (The ***** is bleeped out, so it doesn't send this thread off into a rat-hole - I'll be responding to that point separately.)

Except that bringing back "Identifying" discards Tom's valid concern that this draft is doing much more than "Identifying" - dropping that word, and also dropping "Semantics" (to reduce pomposity) yields:
	Modified Explicit Congestion Notification (ECN) for *****-Low Queuing Delay (L4S)
Which can then be reworded to:
	Explicit Congestion Notification (ECN) Modifications for *****-Low Queuing Delay (L4S)
Which seems good, at least to me ;-).

That said, I'm going to agree to disagree with Tom on use of "protocol" and moreover, I'm willing to accept use of "protocol" if necessary, as this is now feeling like much more of an editorial discussion than a technical one.

Thanks, --David

-----Original Message-----
From: Bob Briscoe <[email-masked]> 
Sent: Friday, May 7, 2021 6:36 PM
To: Tom Henderson; Black, David
Cc: [email-masked]; Gorry Fairhurst
Subject: Re: [tsvwg] Review comments on a careful read of the L4S ID

[EXTERNAL EMAIL] 

Tom, David,

Bob

-- 
________________________________________________________________
Bob Briscoe                               https://urldefense.com/v3/__http://bobbriscoe.net/__;!!LpKI!0jGSQESS07WxazAURFPBfqsCWuWhVr7qQQfKVNeXwDrGUXBm6312pu3yg9FaYTGY$ [bobbriscoe[.]net]

---

From: Gorry Fairhurst <[email-masked]>
Date: Sat, 8 May 2021 06:29:26 +0100

> 'Its' seems better to my English ear than 'this'. I'll leave it. 

Ah, so I see I was mis-reading that. I see this was correct, to avoid 
other people's mistakes, could we just replace the word by saying who 
the phrase refers to perhaps?

“The operator MUST NOT alter the end-to-end L4S ECN identifier from
     L4S to Classic, because an operator decision to exclude certain 
traffic from
     L4S treatment is local-only.”

Gorry

---

From: Gorry Fairhurst <[email-masked]>
Date: Sat, 8 May 2021 07:22:16 +0100

SEE below marked [GF2].

> 	[SM2] An implementation is the best way to demonstrate that a spec is actually implementable and to assess how much of the expected behavior actually transfers to the real word. The rfc3168 exampl…
[GF2]  An implementation is indeed one way to demonstrate this, that has 
some weight. Howevere, just because one implementor knows how to 
implement a certain specifcation doesn't mean that this particular 
specifcation is unabiguous for all who may read it, alas. This sort of 
completeness is important for a PS, and will need more attention if ever 
L4S is proposed on the standards track.

I do not think an implementation is required to publish an EXP spec.

>> but also do prototyping of the details of algorithms in their various (often private) implementations
[GF] You seem to keep arguing in a direction that would result in 
obsoleting RFC3168 before we progress L4S, but I don't agree this 
ordering is needed. The proposal I see is that the deployment takes 
place and then the IETF has the option to decide what happens next.
> 	[SM2] But QUICK is missing quite a number of the other requirements for a L4S-compatible protocol, so can clearly not be used for open-internet testing at the current time, no?
[GF] See separate thread.
> 	[SM2] So we agree that neither DCTCP nor TCP Prague count here, eh? ;) Partly kidding...
[GF] :-).  I do see value in the comments received from TCP implementors 
on the spec though.
> 	[SM2] In theory I agree, yet the fact remains that L4S' AQM increases RTT bias between C and L queue traffic and that the reference L4S protocol TCP Prague displays quite large RTT-bias with itsel…
[GF] I'd prefer to see this as cases where L4S results in more bias. I 
expect the WG will conclude to include a goal of seeking to reducing 
bias in the description in the finally published spec. That wording is 
important to get correct.
> 	[SM2] This assumes that the network bits of L4S as designed right now are sufficiently functional and safe, and that short-comings in their design can be "fixed" on the protocol side. In fact quit…
[GF] That's for the WG to decide via a WGLC.
Best wishes,

Gorry

---

From: "C. M. Heard" <[email-masked]>
Date: Sat, 8 May 2021 08:15:03 -0700

> place and then the IETF has the option to decide what happens next.

Seeing the word ***deployment*** instead of the word ***experiment***
in the last sentence above,  I have to agree with Sebastian on this point.

Mike Heard

---

From: Steven Blake <[email-masked]>
Date: Sat, 08 May 2021 12:23:40 -0400

> in the last sentence above,  I have to agree with Sebastian on this

Which is why people keep arguing that the L4S "deployment" needs to be
isolated to networks that are confirmed to not be deploying RFC 3168
ECN, because L4S and RFC 3168 ECN are incompatible.

Regards,

// Steve

---

From: "Gorry (erg)" <[email-masked]>
Date: Sat, 8 May 2021 18:15:21 +0100

See below 

> On 8 May 2021, at 16:15, C. M. Heard <[email-masked]> wrote:
> 
> 

Well, I think that the purpose of an EXP RFC is to allow initial deployment, and to gain useful experience and then to understand any changes that are  needed to the Spec.

Nobody needs a RFC to do an experiment within their own controlled network. Anyway, this is what I will call  the “alternate ECN semantics RFC” already allows using a private DSCP.

I also know that In many cases, EXP specs do not result in any useful deployment, and that also is a risk.

Gorry

---

From: Sebastian Moeller <[email-masked]>
Date: Sat, 8 May 2021 20:04:27 +0200

Hi Gorry,

see below.

> On May 8, 2021, at 19:15, Gorry (erg) <[email-masked]> wrote:
> 
> See below 
> 
> Nobody needs a RFC to do an experiment within their own controlled network.

	[SM] So we are we contemplating to letting L4S onto the wider internet without any robust containment mechanism, sice L4S has not even been demonstrated to be safe and functional in such a "controlled experiment"?

> Anyway, this is what I will call  the “alternate ECN semantics RFC” already allows using a private DSCP.

	[SM] Maybe this should be change to make such testing a MUST...

> I also know that In many cases, EXP specs do not result in any useful deployment, and that also is a risk.

	[SM] In the L4S case that is almost guaranteed, it already does not work robustly and reliably under lab conditions... so why are we expecting it to fare any better under real life conditions? And that is fact not opinion, its failure modes are demonstrated in Pete's data...

Best Regards
	Sebastian

---

From: Steven Blake <[email-masked]>
Date: Sat, 08 May 2021 21:58:43 -0400

On Sat, 2021-05-08 at 18:15 +0100, Gorry (erg) wrote:
> Well, I think that the purpose of an EXP RFC is to allow initial
> deployment, and to gain useful experience and then to understand any
> changes that are  needed to the Spec.
> 
> Nobody needs a RFC to do an experiment within their own controlled
> network. Anyway, this is what I will call  the “alternate ECN
> semantics RFC” already allows using a private DSCP.

As far as we know, that has not happened.

To use an analogy everyone should hopefully understand in 2021, L4S
only completed phase 1 (lab) trials, completely skipped phase 2 & 3
(placebo-controlled safety and efficacy) trials, and now wants an EUA
(Experimental Use Authorization) from the IETF.

I'll see myself out.

Regards,

// Steve

---

From: Bob Briscoe <[email-masked]>
Date: Sun, 9 May 2021 21:40:51 +0100

Gorry,
Done
Bob

---

From: <[email-masked]>
Date: Mon, 10 May 2021 06:45:36 +0000

Gorry,

Thanks. I don’t think that any content or network provider interested in the L4S queue management has to wait for standards bodies. But to see, whether L4S is a useful queue management mechanism, a transport protocol is required, as well as end points correctly interpreting the L4S ECN marking behaviour. And transport then needs to react to that in an appropriate way.

Experimental deployment of L4S without an adapted transport for any kind of pilot looks pointless to me.

Reading RFC3168, I see

      * The sender receives the TCP ACK with ECN-Echo set, and reacts to
        the congestion as if a packet had been dropped.

Looking into draft-ietf-tsvwg-ecn-l4s-id-15, I find:

   As a condition for a host to send packets with the L4S identifier

   (ECT(1)), it SHOULD implement a congestion control behaviour that

   ensures that, in steady state, the average duration between induced

   ECN marks does not increase as flow rate scales up, all other factors

   being equal.  This is termed a scalable congestion control.  This

   invariant duration ensures that, as flow rate scales, the average

   period with no feedback information about capacity does not become

   excessive.  It also ensures that queue variations remain small,

   without having to sacrifice utilization.

   With a congestion control that sawtooths to probe capacity, this

   duration is called the recovery time, because each time the sawtooth

   yields, on average it take this time to recover to its previous high

   point.  A scalable congestion control does not have to sawtooth, but

   it has to coexist with scalable congestion controls that do.

   For instance, for <example Transport> the average recovery time is always half a round

   trip (or half a reference round trip), whatever the flow rate.

   As with all transport behaviours, a detailed specification (probably

   an experimental RFC) will need to be defined for each congestion

   control, following …. [RFC5033<https://datatracker.ietf.org/doc/html/rfc5033>].  In addition it will need to

   which the proportionality is averaged, and control of burstiness.

   The recovery time requirement above is worded as a 'SHOULD' rather

   than a 'MUST' to allow reasonable flexibility when defining these

   specifications…..

It is not appropriate to blame the authors of L4S for the evolution of the Internet which causes some of the constraints, all transport developers are obliged to respect today. I still note that I can’t yet walk away and implement a transport making fair use of L4S in the full Internet environment of today, whereas I am able to request adaptation of one transport protocol at least using RFC3168. If I’m wrong, those operating pilot deployments in productive environments may speak up (I also don’t expect results, the fact that there’s a test is sufficient). I personally am interested in a mechanism delivering the properties promised by L4S, if it allows for a smooth migration.

Regards,

Ruediger

Von: tsvwg <[email-masked]> Im Auftrag von Gorry (erg)
Gesendet: Samstag, 8. Mai 2021 19:15
An: C. M. Heard <[email-masked]>
Cc: TSVWG <[email-masked]>
Betreff: Re: [tsvwg] Review comments on a careful read of the L4S ID

See below

On 8 May 2021, at 16:15, C. M. Heard <[email-masked]<mailto:[email-masked]>> wrote:

Mike Heard

Gorry

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 13 May 2021 18:50:25 +0100

Gorry, (and David and Wes I believe)

Thank you for the extremely diligent review.
Those points that can just be accepted will be, without comment.
Otherwise, you will now start to see responses to them on the list. I'll 
try to group them to minimize confusion, and I'll add the item no's. in 
the subject line.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 13 May 2021 19:04:17 +0100

See [BB]...

On 06/05/2021 07:52, Gorry Fairhurst wrote:
> drops is still the case.Perhaps: ‘Classic’ ECN marking is an 
> enhancement to drop-based congestion control, both when applied …
>

[BB] The quotes from the draft above refer to what the standards track 
[RFC3168] says, not RFC8311 experiments like ABE. I don't think it will 
be appropriate for this draft to specifically call out ABE as if it is 
now accepted IETF practice. That opens us to more controversy and delay 
if someone disagrees with the ABE experiment.

The quote in #22 is purely about marking in the network, which ABE 
doesn't change, so it's definitely not appropriate to cite ABE there.

The draft does refer to RFC 8511 from an informative appendix, which I 
think is appropriate.

Proposed resolution: No text changes.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 13 May 2021 19:09:34 +0100

See [BB]...

[BB] Isn't 'under the same conditions' already a more precise way of 
saying 'comparable'?

---

From: Sebastian Moeller <[email-masked]>
Date: Thu, 13 May 2021 22:23:14 +0200

See [SM]

> On May 13, 2021, at 20:09, Bob Briscoe <[email-masked]> wrote:
> 
> See [BB]...
> 
>> 2 Abstract  RTT variance
>> ⁃ Please insert “comparable” before “non-L4S flow” to avoid bogus flow comparisons.
>>   
>> 7. Introduction needs to sidestep RTT variance
>>   
>> This text:
>>    “The two changes counterbalance
>> ⁃          Please insert “comparable” before “non-L4S flow” to avoid bogus flow comparisons.
>> 
> [BB] Isn't 'under the same conditions' already a more precise way of saying 'comparable'?
> 

	[SM] This is still ignoring the fact that L4S itself actively interferes and turns the "same" external condition, like a path's minimal RTT into unequal conditions, by inflating C-queue traffic by >= the C-queues default 25 ms latency taget, while the L-queue only sees >= the default 1ms target. Describing that as "equal throughput under the same conditions" seems misguided at best. What folks will understand is, L and C queue traffic on the same path will share equitably, but that is not what L4S delivers. Let's be exquisitely honest here and add a sentence like:

"Note that due to DualQ's default configuration, classic traffic experiences approximately 25ms more of RTT under load, hence for en equal external path RTT, L4S traffic will get a higher throughput."

IMHO, dualQ needs to be fixed so this behavior disappears, but until that is done the least the IDs need to do is spell this out explicitly, end-users of L4S should be able to make reasonably accurate predictions about how it is supposed to work.

Regards
	Sebastian

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 13 May 2021 21:31:18 +0100

See [BB]...

Please confirm this one is acceptable.

On 06/05/2021 07:52, Gorry Fairhurst wrote:
> taken exception to “claims” in an abstract of a standards-track 
> document by marketing of “ultra-low”.*
>
> Abstract text says:

[BB see end]

> ⁃The abstract is still long, and some of the comparison with ‘Classic 

[BB] OK. We can take out sentence 2 onwards, until 'Nonetheless' 
(inclusive).

> ⁃This is better than the corresponding Abstract text (see item 2 
> above), but still contains marketing – “ultra-low” needs to be changed 
> to either “low” or “consistently low”. I suggest toi add  “at the 
> modified network bottlenecks” after “without harming Classic 
> performance.”to avoid the implication that the absence of harm is 
> global (which is known not to be the case).
>

We'll change ultra-low to very low throughout, including in the title.

I would like to explain why we had left 'ultra' in when we went through 
de-hyping the document previously. 'Ultra-low delay' was used throughout 
as a defined term. We figured it would be hard for a reader to recognize 
that 'very low delay' is a defined term, because it just looks like some 
vague words. However, this is a price we have to pay to make progress.

We'll also make the additional changes below.

CURRENT:
ultra-low and consistently low queuing delay (typically sub-millisecond 
on average)
PROPOSED:
very low (typically sub-millisecond on average) and consistently low 
queuing delay :
REASON:
There seems to be some confusion whether we're talking about very low 
delay or consistently low delay. The answer is both.

And to the Introduction:
CURRENT:
“It gives an incremental migration path so that suitably modified 
network bottlenecks can
distinguish and isolate existing Classic traffic from L4S traffic to 
prevent the former from
degrading the ultra-low delay and loss of the new scalable transports, 
without harming
Classic performance.”
PROPOSED:
“It gives an incremental migration path so that suitably modified 
network bottlenecks can
distinguish and isolate existing Classic traffic from L4S traffic to 
prevent the former from
degrading the very low delay and loss of the new scalable transports, 
without harming
Classic performance at these bottlenecks.”
REASON:
    As requested but less clunky than suggested.

Bob

---

From: "Black, David" <[email-masked]>
Date: Thu, 13 May 2021 20:30:56 +0000

Bob,

Let's see if we can come up with an intermediate approach that conveys useful info to the reader about ABE without endorsing it as an IETF standard.

Suggestion: No text changes for 1 and 22 (for 22, I agree that ABE does not affect marking), just add a sentence for 6 in the first paragraph of the Introduction that explains what ABE is/does and that ABE is an experiment.  No need to mention ABE again in the body of the draft (there's one mention in Appendix B.1).

Will that work?

Thanks, --David

From: tsvwg <[email-masked]> On Behalf Of Bob Briscoe
Sent: Thursday, May 13, 2021 2:04 PM
To: Gorry Fairhurst
Cc: [email-masked]
Subject: Re: [tsvwg] Review comments on a careful read of the L4S ID (#1: ABE)

[EXTERNAL EMAIL]
See [BB]...
On 06/05/2021 07:52, Gorry Fairhurst wrote:
=================================================================
1. Abstract needs to be corrected to accommodate ABE (RFC 8511).

Abstract text says:

⁃           That’s no longer completely correct in light of ABE (RFC 8511), although the strong connection between this marking and reaction to drops is still the case.  Perhaps: ‘Classic’ ECN marking is an enhancement to drop-based congestion control, both when applied …

=================================================================
6. Add text to acknowledge ABE (RFC 8511)

This text:
     “RFC 3168 required an ECN mark to be equivalent to a drop, both when applied in the
     network and when responded to by a transport.”

⁃            ABE (RFC 8511) has already modified that drop equivalence.  Revise this text accordingly.

=================================================================
22. Update to include ABE (RFC 8511)

This text:
     “   Note that, contrary to RFC 3168, a Dual Queue Coupled AQM
   implementing the L4S and Classic treatments does not mark an ECT(1)
   packet under the same conditions that it would have dropped a Not-ECT
   packet, as allowed by [RFC8311], which updates RFC 3168.  However, if
   it marks ECT(0) packets, it does so under the same conditions that it
   would have dropped a Not-ECT packet.”

⁃            ABE (RFC 8511) has modified that drop equivalence.  Revise this text accordingly.

[BB] The quotes from the draft above refer to what the standards track [RFC3168] says, not RFC8311 experiments like ABE. I don't think it will be appropriate for this draft to specifically call out ABE as if it is now accepted IETF practice. That opens us to more controversy and delay if someone disagrees with the ABE experiment.

The quote in #22 is purely about marking in the network, which ABE doesn't change, so it's definitely not appropriate to cite ABE there.

The draft does refer to RFC 8511 from an informative appendix, which I think is appropriate.

Bob

--

Bob Briscoe                               http://bobbriscoe.net/ [bobbriscoe.net]<https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!LpKI!zfPbqwU20-zdSS8IyP0GZ48zVTkiYuVPeCEerb_tyGCe-JXJAVzENUM6PA-5RaXE$>

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 13 May 2021 21:36:50 +0100

Thanks for doing this. For me, this is all OK.

Gorry

On 13/05/2021 21:31, Bob Briscoe wrote:
>> degrading the ultra-low delay and loss of the new scalable 
>> transports, without harming
>> Classic performance.”
>>
>> above), but still contains marketing – “ultra-low” needs to be 
>> changed to either “low” or “consistently low”. I suggest toi add  “at 
>> the modified network bottlenecks” after “without harming Classic 
> I would like to explain why we had left 'ultra' in when we went 
> through de-hyping the document previously. 'Ultra-low delay' was used 
> throughout as a defined term. We figured it would be hard for a reader 
> to recognize that 'very low delay' is a defined term, because it just 
> looks like some vague words. However, this is a price we have to pay 
> to make progress.
>
> ultra-low and consistently low queuing delay (typically 
> sub-millisecond on average)
> PROPOSED:
> Bob Briscoehttp://bobbriscoe.net/

---

From: "Black, David" <[email-masked]>
Date: Thu, 13 May 2021 20:38:17 +0000

Going back to the original comment ...

> [BB] Isn't 'under the same conditions' already a more precise way of saying 'comparable'?

No, the concern is that "under the same conditions" will be (mis-)read as "under the same network conditions" inviting bogus comparisons between L4S and non-L4S flows with very different RTTs and lifetimes.

Sebastian's concern about DualQ design/behavior is a separate comment, and the appropriate place to address it (if more text is in order) would likely be the dualq-coupled draft.

Thanks, --David

-----Original Message-----
From: tsvwg <[email-masked]> On Behalf Of Sebastian Moeller
Sent: Thursday, May 13, 2021 4:23 PM
To: Bob Briscoe
Cc: Gorry Fairhurst; [email-masked]
Subject: Re: [tsvwg] Review comments on a careful read of the L4S ID (#2: RTT Variance)

[EXTERNAL EMAIL] 

See [SM]

Regards
	Sebastian

> Bob Briscoe                               
> https://urldefense.com/v3/__http://bobbriscoe.net/__;!!LpKI!zwOahng_hy4Ex53Fiy_jJ8a4vpQ5LvP38-VOfRUHTTb6TKCwG3MyF7gtaeQ75ePz$ [bobbriscoe[.]net]

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 13 May 2021 21:49:24 +0100

See [BB]

[BB] What is the concern here? Cutting off the sentence like this 
disconnects its meaning from the previous sentence, which it was 
intended to follow on from.

Surely this is not a concern about making claims, because this sentence 
describes the document's role: setting requirements.
Perhaps you think the document defines more rules than just those to do 
with harm. It does in some minor cases, but the core purpose of the 
document is about preventing harm (mostly to latency).

Proposed resolution: Pending further guidance.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 13 May 2021 23:13:06 +0100

Gorry,

Please advise whether the following edits address this concern.

See [BB]

On 06/05/2021 07:52, Gorry Fairhurst wrote:
> spec. as an Internet Protocol, even if the underlying basis is 
> important to the L4S transport.*
>
> This text:

[BB] PROPOSED:

   L4S relies on 'scalable' congestion controls for these delay
    properties and for preserving low delay as flow rate scales, hence
    the name.  The congestion control used in Data Center TCP (DCTCP) is
    an example of a scalable congestion control, but DCTCP is applicable
    solely to controlled environments like data centres [RFC8257],
    because it is too aggressive to co-exist with existing TCP-Reno-
    friendly traffic.

> _requirements listed in Section _4.”

[BB] PROPOSED:

    Note that a scalable congestion
    control is still not safe to deploy on the Internet unless it
    satisfies the requirements listed in Section 4.

> research testbeds_(until now)_.”

[BB] PROPOSED:

    outcompete Classic ECN congestion controls
    sharing the same queue, which is why they have been confined to
    private data centres or research testbeds.

> Classic congestion controls.”

[BB] PROPOSED:

    It turns out that these scalable congestion control algorithms that
    solve the latency problem also solve the scalability problem of
    Classic congestion controls.

> for more general traffic _than just_ DCTCP--“

[BB] Substituted 'TCP Prague'

> The ID later states:
> “As with all transport behaviours, a detailed specification 
> (probablyan experimental RFC) will need to be defined for each congestion

[BB] Incidentally, as part of other changes requested by implementers 
during the survey, we've changed the following:
s/will need to be defined/is preferable/

> as a protocol with references to the congestion control 
> method/algorithm used by DCTCP: RFC8257 is informational and 
> explicitly explained it is not EXP.To me this text in the ID provides 
> many contradictions about implying DCTCP as a transport for the 
> Internet. That’s something that really grates with me and I much 
> prefer the much later statement in the IDthat “a detailed 
> specification (probablyan experimental RFC) will need to be defined”. 
> If the claim were different, relating to methods based on DCTCP, that 
> might be more acceptable.
>
> Making this a reference DCTCP as a CC method would be good to address 
> my issue.
>

[BB] I certainly sympathize with GF's concerns about causing confusion 
on the status of DCTCP. I thought I'd done well on this, but I can see 
now the concerns that Gorry raises. I hope the above changes are acceptable.

Proposed resolution: See instances above.

Bob

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 14 May 2021 09:55:15 +0100

>     because it is too aggressive to co-exist with existing TCP-Reno-
[GF] Perhaps on thinking more... maybe it would be better to say:

/it is/the current specification is/

... I've no idea if the DCTCP spec will in future be updated.

>    satisfies the requirements listed in Section 4.
[GF] Looks good.
>     private data centres or research testbeds.
[GF] OK
>     Classic congestion controls.
[GF] I'd prefer to change: /also solve/ or /also can solve/ ... but this 
isn't important to me.
> [BB] Substituted 'TCP Prague'
[GF] OK
> s/will need to be defined/is preferable/
[GF] Unsure that I do like "preferable" - because I don't wish to 
prejudge how TSVWG will handle new methods in future, and I would 
probably will need to re-read to check this. Although perhaps, I could 
now suggest:

"As with all transport behaviours, each congestioncontrol will require a 
detailed specification (which could be published asan experimental RFC), 
following the guidelines for specifying new congestioncontrol algorithms 
in [RFC5033]."

>> ⁃This would be significantly improved by replacing references to 
>> DCTCP as a protocol with references to the congestion control 

We seem to be converging,

Gorry

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 10:17:55 +0100

Gorry,

> ... I've no idea if the DCTCP spec will in future be updated.

[BB] Doesn't that introduce more doubt, whereas you wanted unambiguous 
clarity (and CC designs don't tend to become less aggressive anyway)?

> [GF] I'd prefer to change: /also solve/ or /also can solve/ ... but 
> this isn't important to me.

[BB2] OK.

> "As with all transport behaviours, each congestioncontrol will require 
> a detailed specification (which could be published asan experimental 
> RFC), following the guidelines for specifying new congestioncontrol 
> algorithms in [RFC5033]."
>

[BB2] I understood that we had been asked to remove any indications that 
specification of secret sauce (congestion controls) is mandatory. I had 
altered the rest from "requirements" to "recommendations", but I had 
missed this one.

Bob

>>> explicitly explained it is not EXP.To me this text in the ID 
>>> provides many contradictions about implying DCTCP as a transport for 
>>> the Internet. That’s something that really grates with me and I much 
>>> specification (probablyan experimental RFC) will need to be 
>>> defined”. If the claim were different, relating to methods based on 
>>> DCTCP, that might be more acceptable.
>>>
>>> Making this a reference DCTCP as a CC method would be good to 
>>> address my issue.
>>>
>> [BB] I certainly sympathize with GF's concerns about causing 
>> confusion on the status of DCTCP. I thought I'd done well on this, 
>> but I can see now the concerns that Gorry raises. I hope the above 
>> changes are acceptable.
>>

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 14 May 2021 10:23:56 +0100

Bob, please see below:

> clarity (and CC designs don't tend to become less aggressive anyway)?
[GF] I like to avoid doubt, so I agree. Anyway the text says RFC8257, 
then "it" - so I'll withdraw my comment. Your proposal looks good to me.
>>> [BB] Incidentally, as part of other changes requested by 
>>> implementers during the survey, we've changed the following:
>> probably will need to re-read to check this. Although perhaps, I 
>> could now suggest:
>>
>> "As with all transport behaviours, each congestioncontrol will 
>> require a detailed specification (which could be published asan 
>> experimental RFC), following the guidelines for specifying new 
>> congestioncontrol algorithms in [RFC5033]."
>>
>
> [BB2] I understood that we had been asked to remove any indications 
> that specification of secret sauce (congestion controls) is mandatory. 
> I had altered the rest from "requirements" to "recommendations", but I 
> had missed this one.
>
>
[GF] I still think "specification required" is fine - This can be 
suggested as an EXP RFC, or it could be something else, but from the 
IETF WG viewpoint it seems to me that a specification is needed.

Gorry
>>>> provides many contradictions about implying DCTCP as a transport 
>>>> for the Internet. That’s something that really grates with me and I 
>>>> much prefer the much later statement in the IDthat “a detailed

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 10:41:27 +0100

David,

A relevant place to refer to ABE is in S.1.3:

CURRENT:

    This document is intended for experimental status, so it does not
    update any standards track RFCs.  Therefore it depends on [RFC8311  <https://datatracker.ietf.org/doc/html/rfc8311>],
    which is a standards track specification that:

    o  updates the ECN proposed standard [RFC3168  <https://datatracker.ietf.org/doc/html/rfc3168>] to allow experimental
       track RFCs to relax the requirement that an ECN mark must be
       equivalent to a drop (when the network applies markings and/or
       when the sender responds to them);

PROPOSED to append:
      For instance, an experimental ABE sender [RFC8511] responds less 
to ECN marks than to drops.

Regarding adding ABE to the first para of the introduction, I really 
don't want to go off into a side-remark right at the start.

Bob

On 13/05/2021 21:30, Black, David wrote:
>
> Bob,
>
> Let's see if we can come up with an intermediate approach that conveys 
> useful info to the reader about ABE without endorsing it as an IETF 
> standard.
>
> Suggestion: No text changes for 1 and 22 (for 22, I agree that ABE 
> does not affect marking), just add a sentence for 6 in the first 
> paragraph of the Introduction that explains what ABE is/does and that 
> ABE is an experiment.  No need to mention ABE again in the body of the 
> draft (there's one mention in Appendix B.1).
>
> Will that work?
>
> Thanks, --David
>
> *From:* tsvwg <[email-masked]> *On Behalf Of * Bob Briscoe
> *Sent:* Thursday, May 13, 2021 2:04 PM
> *To:* Gorry Fairhurst
> *Cc:* [email-masked]
> *Subject:* Re: [tsvwg] Review comments on a careful read of the L4S ID 
> (#1: ABE)
>
> [EXTERNAL EMAIL]
>
> See [BB]...
>
>     although the strong connection between this marking and reaction
>     to drops is still the case.  Perhaps: ‘Classic’ ECN marking is an
>     ⁃ABE (RFC 8511) has already modified that drop equivalence. 
>     Revise this text accordingly.
>
>     ⁃ABE (RFC 8511) has modified that drop equivalence.  Revise this
>     text accordingly.
>
> [RFC3168] says, not RFC8311 experiments like ABE. I don't think it 
> will be appropriate for this draft to specifically call out ABE as if 
> it is now accepted IETF practice. That opens us to more controversy 
> and delay if someone disagrees with the ABE experiment.
>
> Bob Briscoehttp://bobbriscoe.net/ [bobbriscoe.net]  <https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!LpKI!zfPbqwU20-zdSS8IyP0GZ48zVTkiYuVPeCEerb_tyGCe-JXJAVzENUM6PA-5RaXE$>

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 10:46:50 +0100

David, [Pls ignore previous email - had a better idea - see replacement 
text below]

CURRENT:

PROPOSED to append:
      For instance, an experimental ABE sender [RFC8511] responds less 
to ECN marks than to drops.
      For instance, this permits an ABE sender [RFC8511] to experiment 
with responding less to ECN marks than to drops.

Bob

---

From: Michael Welzl <[email-masked]>
Date: Fri, 14 May 2021 12:07:58 +0200

Hi,

A comment below:

> On May 14, 2021, at 11:46 AM, Bob Briscoe <[email-masked]> wrote:
> 
> David, [Pls ignore previous email - had a better idea - see replacement text below]
> 
>      For instance, an experimental ABE sender [RFC8511] responds less to ECN marks than to drops.
>      For instance, this permits an ABE sender [RFC8511] to experiment with responding less to ECN marks than to drops.

I disagree with “experiment with”. The specification is experimental, but this doesn’t entail that a sender would “experiment with responding less” - this sounds as if ABE would just specify that a reduced back-off is something cool to experiment with, and an ABE sender would then try this or that backoff, depending on their liking. That’s a misrepresentation.

Suggestion:
     For instance, this permits an ABE sender [RFC8511] to respond less to ECN marks than to drops.

I find it unnecessary to stress “experiment” again in this sentence because it describes an example of “updating the ECN proposed … to allow experimental track RFCs to … “ - i.e., it’s very clear from the context here that ABE is experimental.

Cheers,
Michael

> Regarding adding ABE to the first para of the introduction, I really don't want to go off into a side-remark right at the start.
> 
> 
> Bob
> 
> 
>> From: tsvwg <[email-masked]> <mailto:[email-masked]> On Behalf Of Bob Briscoe
>> Cc: [email-masked] <mailto:[email-masked]>
>> Bob Briscoe                               http://bobbriscoe.net/ [bobbriscoe.net] <https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!LpKI!zfPbqwU20-zdSS8IyP0GZ48zVTkiYuVPeCEerb_tyGCe-JXJAVzENUM6PA-5RaXE$>
> 
> -- 
> Bob Briscoe                               http://bobbriscoe.net/ <http://bobbriscoe.net/>

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 11:21:48 +0100

Michael,

On 14/05/2021 11:07, Michael Welzl wrote:
> Hi,
>
> A comment below:
>
>> On May 14, 2021, at 11:46 AM, Bob Briscoe <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
> I disagree with “experiment with”. The specification is experimental, 
> but this doesn’t entail that a sender would “experiment with 
> responding less” - this sounds as if ABE would just specify that a 
> reduced back-off is something cool to experiment with, and an ABE 
> sender would then try this or that backoff, depending on their liking. 
> That’s a misrepresentation.
>
> Suggestion:
> I find it unnecessary to stress “experiment” again in this sentence 
> because it describes an example of “updating the ECN proposed … to 
> allow experimental track RFCs to … “ - i.e., it’s very clear from the 
> context here that ABE is experimental.

[BB] I am being asked to stress the status of experimental drafts.
I agree with your point about "experiment with" tho, so I'll take a 
middle line:

     in the ABE experiment [RFC8511] this permits a sender to respond 
less to ECN marks than to drops

Bob

>>> Let's see if we can come up with an intermediate approach that 
>>> conveys useful info to the reader about ABE without endorsing it as 
>>> an IETF standard.
>>>
>>> paragraph of the Introduction that explains what ABE is/does and 
>>> that ABE is an experiment.  No need to mention ABE again in the body 
>>> of the draft (there's one mention in Appendix B.1).
>>>
>>> Will that work?
>>>
>>> Thanks, --David
>>>
>>> *Subject:* Re: [tsvwg] Review comments on a careful read of the L4S 
>>> ID (#1: ABE)
>>>
>>> [EXTERNAL EMAIL]
>>>
>>> See [BB]...
>>>
>>>     to drops is still the case.  Perhaps: ‘Classic’ ECN marking is
>>>     an enhancement to drop-based congestion control, both when applied …
>>>
>>> [BB] The quotes from the draft above refer to what the standards 
>>> track [RFC3168] says, not RFC8311 experiments like ABE. I don't 
>>> think it will be appropriate for this draft to specifically call out 
>>> ABE as if it is now accepted IETF practice. That opens us to more 
>>> controversy and delay if someone disagrees with the ABE experiment.
>>>

---

From: Michael Welzl <[email-masked]>
Date: Fri, 14 May 2021 12:59:51 +0200

> On May 14, 2021, at 12:21 PM, Bob Briscoe <[email-masked]> wrote:
> 
> Michael,
> 
>>> On May 14, 2021, at 11:46 AM, Bob Briscoe <[email-masked] <mailto:[email-masked]>> wrote:
>>> 
> I agree with your point about "experiment with" tho, so I'll take a middle line:
> 
>     in the ABE experiment [RFC8511] this permits a sender to respond less to ECN marks than to drops

This works for me. Thanks.

Cheers,
Michael

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 12:22:16 +0100

Gorry,
[the last round of postings got cc'd to tsvwg-bounces not tsvwg. For the 
list, I've left it quoted in full below, then added to it. ]

On 14/05/2021 11:03, Bob Briscoe wrote:
> Gorry, [Changed the subject line, and incremented the last issue 
> number you gave]
>
> On 14/05/2021 10:23, Gorry Fairhurst wrote:
>>> that specification of secret sauce (congestion controls) is 
>>> mandatory. I had altered the rest from "requirements" to 
>>> "recommendations", but I had missed this one.
>>>
>>>
> [BB] I have to push back here on behalf of all the implementers who 
> objected to being required to document CCs.
>
> From the tsvwg minutes of IETF-110:
> Koen presents the Prague requirements survey regarding
> draft-ietf-tsvwg-ecn-l4s-id
> Strong objections from implementers to documentation-only requirements.
> An alternative approach in the draft would be to encourage documentation.
>

[BB2] How about:

"As with all transport behaviours, a detailed specification (probably an 
experimental RFC) is expected for each congestion control, following the 
guidelines for specifying new congestion control algorithms in [RFC5033]."

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 12:34:19 +0100

David,

> No, the concern is that "under the same conditions" will be (mis-)read as "under the same network conditions" inviting bogus comparisons between L4S and non-L4S flows with very different RTTs and l…

[BB2] OK.  Done.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 12:48:39 +0100

tsvwg list,
see [BB2]

On 13/05/2021 21:49, Bob Briscoe wrote:
> Surely this is not a concern about making claims, because this 
> sentence describes the document's role: setting requirements.
> Perhaps you think the document defines more rules than just those to 
> do with harm. It does in some minor cases, but the core purpose of the 

[BB] Gorry took this off list to reduce noise, and explained that the 
concern was the word 'ensure' which has been resolved as below.
For the record:

“It gives an incremental
migration path so that suitably modified network bottlenecks can
distinguish and isolate existing traffic that still follows the
Classic behaviour, to prevent it degrading the low queuing delay and
low loss of L4S traffic.This specification defines the rules that
L4S transports and network elements need to follow_with the intention 
that L4S flows_
neither harm each other's performance nor that of Classic traffic."__

[GF] Yes: For me stating an "intention" would be fine. It's actually 
good to know why you should follow something.
"ensuring" is something quite different, and harder to prove.

Gorry

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 12:53:54 +0100

See [BB]

On 06/05/2021 07:52, Gorry Fairhurst wrote:
> *10. Please avoid making a claim that the IETF is NOT making a 
> statement about usability of DSCPs in this document.*
>
> This text grates:
[BB] I've added some context around this quote:
> “ Latency is becoming the critical performance factor for many (most?)
>    applications on the public Internet
> [...]
>    The Diffserv architecture provides Expedited Forwarding [RFC3246], so
>    that low latency traffic can jump the queue of other traffic.
>    However, on access links dedicated to individual sites (homes, small
> from, Diffserv makes no difference. Instead, we need to remove the
>    causes of any unnecessary delay.“
>
> ⁃To me this is not balanced. It seems to suggest that setting a DSCP 
> is useless. I don’t believe that is the consensus of tsvwg, although 
> at some additional pain we can debate the merits of setting DSCPs for 
> a traffic - such as the implications in enterprise networks; the 
> discussion, but needing this to progress the draft seems unfortunate 
> to me. “Then, given nothing to differentiatefrom, Diffserv makes no 
> difference.” is gratuitous and should be removed.The paragraph 
> continues with “Instead, we need to remove the causes of any 
> unnecessary delay.” which can be combined into the previous sentence 
> as: “ … will be latency-sensitive, making it imperative to remove the 
> underlying causes of delay.”Nothing is lost here because the crucial 
> comparison with Diffserv is picked up two paragraphs later: “Unlike 
> Diffserv, which gives low latency to some traffic at the expense of 
> others, AQM controls latency for _all_ traffic in a clas
>

[BB] I appreciate that you are trying to avoid upsetting anyone in the 
IETF who might take exception to criticism of Diffserv. However, I have 
tried to make it clear below that this is about limits to its 
applicability, not criticism of the whole idea.

This whole section is about the limits of other pre-existing low latency 
technologies. So it has something to irritate everyone - inherently. It 
is unfortunately necessary because, when I'm asked to review drafts from 
other areas of the IETF, I often have to ask the authors to say why they 
are proposing another way to do something that can already be done 
(which is the opposite of what a standards body ought to be encouraging 
- unless there's a new problem).

Second attempt...
[...]
    The Diffserv architecture provides Expedited Forwarding [RFC3246], so
    that low latency traffic can jump the queue of other traffic.

Then the following PROPOSED replacement text:

    If growth in high-throughput latency-sensitive applications continues,
    periods with solely latency-sensitive traffic will become increasingly
    common on links where traffic aggregation is low. For instance, on the
    access links dedicated to individual sites (homes, small enterprises
    or mobile devices). These links also tend to become the path bottleneck
    under load.  During these periods, given nothing to differentiate from,
    Diffserv would make no difference, at these bottlenecks. Instead, it
    becomes imperative to remove the underlying causes of any unnecessary
    delay.

Reasoning: Instead of relying on the reader connecting two concepts 
across paragraphs, this spells out the problem. This para's role was to 
explain the problem first. Before the subsequent para with the solution 
(AQMs reduce delay for all traffic).

But to soften the offending sentence, I've conditioned it  on "During 
these periods" and "at these bottlenecks".

The problem may seem obvious or gratuitous to experts in traffic 
control. However, to many networks engineers who are not traffic 
experts, Diffserv is something you take off a pick-list - "if you need 
low latency, you stick a low latency DSCP on the packets".

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 13:04:24 +0100

Gorry, tsvwg,
See [BB]

[BB] For the record, as discussed in the tsvwg meeting, "standard TCP 
Reno" has already been modified (in two normative sentences) to:

"...Classic congestion controls such as standard Reno..."

Bob

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 14 May 2021 13:51:20 +0100

On 14/05/2021 12:53, Bob Briscoe wrote:
> IETF who might take exception to criticism of Diffserv. However, I 
> have tried to make it clear below that this is about limits to its 
> This whole section is about the limits of other pre-existing low 
> latency technologies. So it has something to irritate everyone - 
> inherently. It is unfortunately necessary because, when I'm asked to 
> review drafts from other areas of the IETF, I often have to ask the 
> authors to say why they are proposing another way to do something that 
> can already be done (which is the opposite of what a standards body 
> ought to be encouraging - unless there's a new problem).
>
> Second attempt...
> [...]
> across paragraphs, this spells out the problem. This para's role was 
> to explain the problem first. Before the subsequent para with the 
> solution (AQMs reduce delay for all traffic).
>

To me this helps a lot, but I'd prefer to avoid /given nothing to 
differentiate from/ and explain, is this better?:

/During these periods, given nothing to differentiate from,
    Diffserv would make no difference, at these bottlenecks./

During these periods, ifall the traffic were marked for the same 
treatment at these bottlenecks,
Diffserv would make no difference./

Goirry

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 14 May 2021 13:51:51 +0100

> "...Classic congestion controls such as standard Reno..."

Sounds good to me.

Gorry

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 14:04:46 +0100

Gorry,
See [BB2]

On 14/05/2021 13:51, Gorry Fairhurst wrote:
>>> at some additional pain we can debate the merits of setting DSCPs 
>>> for a traffic - such as the implications in enterprise networks; the 
>>> as: “ … will be latency-sensitive, making it imperative to remove 
>>> the underlying causes of delay.”Nothing is lost here because the 
>>> crucial comparison with Diffserv is picked up two paragraphs later: 
>>> “Unlike Diffserv, which gives low latency to some traffic at the 
>>> expense of others, AQM controls latency for _all_ traffic in a clas
>>>
>> [BB] I appreciate that you are trying to avoid upsetting anyone in 
>> the IETF who might take exception to criticism of Diffserv. However, 
>> I have tried to make it clear below that this is about limits to its 
>> authors to say why they are proposing another way to do something 
>> that can already be done (which is the opposite of what a standards 
>> body ought to be encouraging - unless there's a new problem).
>>
>> Second attempt...
>> [...]
>> experts, Diffserv is something you take off a pick-list - "if you 
>> need low latency, you stick a low latency DSCP on the packets".
>>
>> Bob
>>
>> -- 

[BB2] Done.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 14:32:22 +0100

Gorry, David,

On 06/05/2021 07:52, Gorry Fairhurst wrote:
> ***13. Position with respect to RFC4774 is rather unclear, which 
> motivates a case for alternative semantics for carrier experiments*
>
> This text:
> •RFC4774 states: “The assumption of this document is that when 
> alternate semantics are defined for the ECN field, a codepoint in the 
> diffserv field is used to signal the use of these alternate ECN 
> semantics to the router.” This text and position needs to be 
> clarified, and is being discussed in a tsvwg email thread.

[BB] RFC4774 does not say anything that implies that a DSCP is 
recommended / required / preferred as a way to signal alternative ECN 
semantics. The RFC itself is very clear that DSCP is just an example. At 
the end of Section 3 of RFC4774, after the above quote, it says:

    We note that this discussion does not exclude the possibility of
    using other methods, including out-of-band mechanisms, for signalling
    the use of alternate semantics for the ECN field.  The considerations
    in the rest of this document apply regardless of the method used to
    signal the use of alternate semantics for the ECN field.

Also quoting from Section 2:

        As an example, in most of the proposals for alternate ECN
        semantics, a diffserv field is used to specify the use of
        alternate ECN semantics.

Proposed resolution: No text changes. Although if something else 
underlies this concern, I'd be happy to fix. Please advise.

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 14:46:44 +0100

David, Gorry,

On 06/05/2021 07:52, Gorry Fairhurst wrote:
> ⁃Use of “network node” is excessive and over-constrains 
> implementations, e.g., the techniques in Sections 5.4.1.2 and 5.4.1.3 
> conflict with this sentence.Change “A network node that implements” to 
> “An implementation of the L4S service” It would be nice to be at least 
> clear this is marking behaviour, e.g. “An implementation that marks 
> using L4S”, or something similar.
> =================================================================*
> 20.Reduce “network node” scope (part 2)*
> In section 5.1:
> not-ECT. Has this been considered and is it explicitly required to 
> then send a CE mark via the L4S queue? I understand the next para to 
> speak only about when next packet was ECT(1).
> ⁃See other note on “at a network node” and change “the node may” to 
> “the implementation MAY”.This removes conflicts with at least Sections 
> 5.4.1.2 and 5.4.1.3.

[BB] For the record, I believe these concerns were resolved offlist to 
reduce noise, and the resolutions are already included in draft-16, by 
referring forward to exceptions defined later in S.5.1, and in 5.1 
making those exceptions clearer and more complete.

Please confirm whether the text in -16 is now acceptable.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 14:56:52 +0100

Gorry, David, all,

The text of all these requirements was changed, in some cases 
significantly, at draft-15 & -16.
So this email reinforces my request during the interim for people to 
please check whether the normative wording is now acceptable. If not, 
pls suggest improvements. If it is good, praise is always welcome too ;)

Bob

On 06/05/2021 07:52, Gorry Fairhurst wrote:
> bulleted item. One possibility that would work for me is some 
> rephrasing to say “needs to eliminate” and “specifications are 
> REQUIRED to include methods that limit RTT bias, although I have no 
> problem with saying “is expected to …”.

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 15:47:32 +0100

Gorry, David,

On 06/05/2021 07:52, Gorry Fairhurst wrote:
> ⁃It is not clear what the MUST relates to, I think this needs to be 
> more like (you’d still need to explain harm): “Non-L4S packets that 
> are mixed with L4S traffic MUST NOT harm the low latency service, 
> where 'safe' is explained in Section 5.4.1.1.1 below. and… Consider 
> use of “SHOULD” here, as the consequences of not following it are 
> stated, but not the current “MUST”.

[BB] Actually, I now realize that this has been written as a requirement 
on the operator, not the implementation. See below for fix.

> ⁃This later sentence appears to be a requirement on the same thing, 
> but isn’t the same

[BB] To resolve the problems with both the above normative sentences, I 
have deleted them both. And introduced the bullet list of different 
types of identifier with these words instead: "Examples of relevant 
non-ECN identifiers are:"

Then I've added the following normative sentence just after the bullet list:

    In summary, a network element that implements L4S in a shared queue
    MAY classify additional types of packets into the L queue based on
    identifiers other than the ECN field, but the types SHOULD be 'safe'
    to mix with L4S traffic, where 'safe' is explained in
    Section 5.4.1.1.1.

Please confirm if this resolves the problems here.

---

From: "Black, David" <[email-masked]>
Date: Fri, 14 May 2021 14:49:35 +0000

From: Michael Welzl <[email-masked]>
Sent: Friday, May 14, 2021 7:00 AM
To: Bob Briscoe
Cc: Black, David; <[email-masked]> Fairhurst; [email-masked]
Subject: Re: [tsvwg] Review comments on a careful read of the L4S ID (#1: ABE)

[EXTERNAL EMAIL]

On May 14, 2021, at 12:21 PM, Bob Briscoe <[email-masked]<mailto:[email-masked]>> wrote:

Michael,

A comment below:

On May 14, 2021, at 11:46 AM, Bob Briscoe <[email-masked]<mailto:[email-masked]>> wrote:

CURRENT:

   update any standards track RFCs.  Therefore it depends on [RFC8311 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc8311__;!!LpKI!wpysl1bDzmAB29BK4FM7tv328JmHwJ198YqY8zYiKE2IQARLOe6WKjXdK6gZfe8E$>],

   o  updates the ECN proposed standard [RFC3168 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc3168__;!!LpKI!wpysl1bDzmAB29BK4FM7tv328JmHwJ198YqY8zYiKE2IQARLOe6WKjXdK8LqHmSA$>] to allow experimental

PROPOSED to append:

Suggestion:

Cheers,
Michael
[David>] Works for me also, Thanks, --David

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 14 May 2021 15:51:48 +0100

> pls suggest improvements. If it is good, praise is always welcome too ;)

Yes indeed, thanks! The original comments from me were addressed. I'll 
review the entire document at the next stage.

Gorry

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 14 May 2021 15:53:16 +0100

On 14/05/2021 15:47, Bob Briscoe wrote:
> [BB] Actually, I now realize that this has been written as a 
> requirement on the operator, not the implementation. See below for fix.
>
>>
> [BB] To resolve the problems with both the above normative sentences, 
> I have deleted them both. And introduced the bullet list of different 

For me this works, thanks.

Gorry

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 15:54:19 +0100

Gorry, David, list,
See [BB]

[BB] For the record, these were addressed in draft-16 with:

    "...because an operator decision to exclude certain traffic..."

And this para was added at the end of the section:

    An operator that excludes traffic carrying the L4S identifier from
    L4S treatment MUST NOT treat such traffic as if it carries the ECT(0)
    codepoint, which could confuse the sender.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 16:02:19 +0100

Gorry, David, Wes,
See [BB]

On 06/05/2021 07:52, Gorry Fairhurst wrote:
> describes how an operator might use L4S to offer low latency for all 
> L4S traffic as well as
> ⁃If [I-D.briscoe-tsvwg-l4s-diffserv] is not being pursued, this is not 
> a useful reference in a spec RFC.

[BB] It has since been clarified with the chairs (off list) that the L4S 
Diffserv draft was left in a holding pattern while the main L4S drafts 
are processed, so its fate has not been decided either way. Altho' it is 
an individual draft, this informative reference will be left in, because 
ecn-l4s-id only discusses combining L4S and Diffserv /identifiers/, 
whereas l4s-diffserv discussed when it would make sense to combine L4S 
and Diffserv /services/ (which would motivate combining the identifiers, 
and it also allows combining identifiers to be discussed in more detail 
and in context).

Bob

---

From: "Black, David" <[email-masked]>
Date: Fri, 14 May 2021 15:05:59 +0000

Top posting to make this quick to read – this concern is resolved by the following revised Section 5.1 text (in -16):

   A network node that implements the L4S service:

   o  MUST classify arriving ECT(1) packets for L4S treatment, unless
      overridden by a another classifier (e.g., see Section 5.4.1.2<https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id-16#section-5.4.1.2>);

   o  MUST classify arriving CE packets for L4S treatment as well,
      unless overridden by a another classifier or unless the exception
      referred to next applies;

The use of "another classifier" allows use of other criteria to include/exclude ECT(1) and CE packets for L4S treatment.  The exception that follows is use of a flow-aware mechanism to distinguish CE packets for L4S flows from those for non-L4S flows and only apply L4S treatment to CE packets for L4S flows.

Nit: "a another" -> "another" (twice).

Thanks, --David

From: Bob Briscoe <[email-masked]>
Sent: Friday, May 14, 2021 9:47 AM
To: Gorry Fairhurst; Black, David
Cc: [email-masked]
Subject: Re: [tsvwg] Review comments on a careful read of the L4S ID (#14. network node)

[EXTERNAL EMAIL]
David, Gorry,
On 06/05/2021 07:52, Gorry Fairhurst wrote:
=================================================================
14. Replace “network node”  (part 1)

This text:

⁃            Use of “network node” is excessive and over-constrains implementations, e.g., the techniques in Sections 5.4.1.2 and 5.4.1.3 conflict with this sentence.  Change “A network node that implements” to “An implementation of the L4S service” It would be nice to be at least clear this is marking behaviour, e.g. “An implementation that marks using L4S”, or something similar.
=================================================================
20.  Reduce “network node” scope (part 2)
In section 5.1:
“A network node that implements the L4S service MUST classify arriving
  ECT(1) packets for L4S treatment and, other than in the exceptional
   case referred to next, it MUST classify arriving CE packets for L4S
   treatment as well.”

⁃            Please change to “An implementation of the L4S service MUST classify …”  This removes conflicts with at least Sections 5.4.1.2 and 5.4.1.3.
=================================================================
21. Reduce “network node” scope (part 3)

This text:

⁃      Change to “an implementation of the L4S service that supports the L4S treatment MUST also implement …”  This removes conflicts with at least Sections 5.4.1.2 and 5.4.1.3.
=================================================================
23. What about if a sender uses Not-ECT and ECT(0) in combination? Also, reduce “network node” scope (part 4)
This text:
“Nonetheless, if an implementer is
   willing to identify transport-layer flows at a network node, and if
   the most recent ECT packet in the same flow was ECT(0), the node MAY
   classify CE packets for Classic ECN [RFC3168] treatment.”
⁃            Please tell us if you have thought about when the previous packet was not-ECT. Has this been considered and is it explicitly required to then send a CE mark via the L4S queue? I understand the next para to speak only about when next packet was ECT(1).
⁃            See other note on “at a network node” and change “the node may” to “the implementation MAY”.  This removes conflicts with at least Sections 5.4.1.2 and 5.4.1.3.
⁃            It would also help to change “if an implementer is willing” to “if an implementation is able”.
⁃            If the latter is done, change “If an implementer uses” to “If an implementation uses” at the start of the next paragraph.
=================================================================

[BB] For the record, I believe these concerns were resolved offlist to reduce noise, and the resolutions are already included in draft-16, by referring forward to exceptions defined later in S.5.1, and in 5.1 making those exceptions clearer and more complete.

Bob

--

Bob Briscoe                               http://bobbriscoe.net/ [bobbriscoe.net]<https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!LpKI!1enWVfZ8juwUTPZhR8eDn_Cawae1UGNlkNx5uU75k0CGy_jA23R8zDakSDZR4KwS$>

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 16:09:45 +0100

Chairs, Martin,
See [BB]

On 06/05/2021 07:52, Gorry Fairhurst wrote:
> draft, definitely out of date, and not aligned with current WG views 
> of this area. This appendix can be removed prior to publication. 
> Appendix C is unecessary to the final publication.

[BB] Offlist discussion with the chairs and Martin has come up with the 
solution of deleting the sub-sections of these appendices that are not 
referenced from the rest of the draft, but keeping those that are.

B.1 is referenced from the body of the draft a couple of times, because 
it's the pros and cons of the chosen scheme.
C.1 is referenced from the security considerations.

So I'm planning on promoting B.1 to B, and C.1 to C.
But I'll take out the remainder of B and C, and change the introductory 
context of each.

For reference, here's the end of the ToC as it stands before the 
proposed edits:

   Appendix B  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B>.  Alternative Identifiers  . . . . . . . . . . . . . .46  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-46>
      B.1  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.1>.  ECT(1) and CE codepoints  . . . . . . . . . . . . . . . .47  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-47>
      B.2  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.2>.  ECN-DualQ-SCE1  . . . . . . . . . . . . . . . . . . . . .51  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-51>
      B.3  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.3>.  ECN-DualQ-SCE0  . . . . . . . . . . . . . . . . . . . . .54  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-54>
      B.4  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.4>.  ECN Plus a Diffserv Codepoint (DSCP)  . . . . . . . . . .56  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-56>
      B.5  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.5>.  ECN capability alone  . . . . . . . . . . . . . . . . . .58  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-58>
      B.6  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.6>.  Protocol ID . . . . . . . . . . . . . . . . . . . . . . .58  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-58>
      B.7  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.7>.  Source or destination addressing  . . . . . . . . . . . .59  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-59>
      B.8  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.8>.  Summary: Merits of Alternative Identifiers  . . . . . . .59  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-59>
    Appendix C  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-C>.  Potential Competing Uses for the ECT(1) Codepoint  .  60
      C.1  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-C.1>.  Integrity of Congestion Feedback  . . . . . . . . . . . .60  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-60>
      C.2  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-C.2>.  Notification of Less Severe Congestion than CE  . . . . .61  <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-61>

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 16:17:57 +0100

Gorry, David, List,

I believe I've now started threads on the list about all the points you 
raised, and most are resolved. I see some more responses, which I'll 
deal with later. When/if everything seems to be resolved, I'll finalize 
the edits and submit another draft.

Bob

On 13/05/2021 18:50, Bob Briscoe wrote:
> Otherwise, you will now start to see responses to them on the list. 
> I'll try to group them to minimize confusion, and I'll add the item 
> no's. in the subject line.
>
>
> Bob
>
>

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 14 May 2021 16:46:19 +0100

> See [BB]
ACK.

That seems like a good plan, and was what we discussed.

Gorry

>> ⁃Appendix B is lengthy, at best loosely connected with the rest of 
>> the draft, definitely out of date, and not aligned with current WG 
>> views of this area. This appendix can be removed prior to 
>> publication. Appendix C is unecessary to the final publication.
> [BB] Offlist discussion with the chairs and Martin has come up with 
> the solution of deleting the sub-sections of these appendices that are 
> not referenced from the rest of the draft, but keeping those that are.
>
> B.1 is referenced from the body of the draft a couple of times, 
> because it's the pros and cons of the chosen scheme.
> But I'll take out the remainder of B and C, and change the 
> introductory context of each.
>

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 17:09:25 +0100

David,

On 14/05/2021 16:05, Black, David wrote:
>
> Top posting to make this quick to read – this concern is resolved by 
> the following revised Section 5.1 text (in -16):
>
> overridden by a another classifier (e.g., see Section 5.4.1.2 
> <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id-16#section-5.4.1.2>);
>
> The use of "another classifier" allows use of other criteria to 
> include/exclude ECT(1) and CE packets for L4S treatment. The exception 
> that follows is use of a flow-aware mechanism to distinguish CE 
> packets for L4S flows from those for non-L4S flows and only apply L4S 
> treatment to CE packets for L4S flows.
>

[BB] Done.

Thx

bob

> Thanks, --David
>
> *From:* Bob Briscoe <[email-masked]>
> *Sent:* Friday, May 14, 2021 9:47 AM
> *To:* Gorry Fairhurst; Black, David
> *Cc:* [email-masked]
> (#14. network node)
>     implementations, e.g., the techniques in Sections 5.4.1.2 and
>     5.4.1.3 conflict with this sentence.  Change “A network node that
>     implements” to “An implementation of the L4S service” It would be
>     nice to be at least clear this is marking behaviour, e.g. “An
>     implementation that marks using L4S”, or something similar.
>     20.  Reduce “network node” scope (part 2)*
>
>     In section 5.1:
>     ⁃Please change to “An implementation of the L4S service MUST
>     classify …”  This removes conflicts with at least Sections 5.4.1.2
>     and 5.4.1.3.
>     ⁃Change to “an implementation of the L4S service that supports the
>     L4S treatment MUST also implement …”  This removes conflicts with
>     at least Sections 5.4.1.2 and 5.4.1.3.
>     *23. What about if a sender uses Not-ECT and ECT(0) in
>     combination? Also, reduce “network node” scope (part 4)*
>     This text:
>     ⁃Please tell us if you have thought about when the previous packet
>     was not-ECT. Has this been considered and is it explicitly
>     required to then send a CE mark via the L4S queue? I understand
>     the next para to speak only about when next packet was ECT(1).
>     ⁃See other note on “at a network node” and change “the node may”
>     to “the implementation MAY”.  This removes conflicts with at least
>     ⁃It would also help to change “if an implementer is willing” to
>     “if an implementation is able”.
> Bob Briscoehttp://bobbriscoe.net/ [bobbriscoe.net]  <https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!LpKI!1enWVfZ8juwUTPZhR8eDn_Cawae1UGNlkNx5uU75k0CGy_jA23R8zDakSDZR4KwS$>

---

From: Martin Duke <[email-masked]>
Date: Fri, 14 May 2021 10:32:32 -0700

SGTM

On Fri, May 14, 2021 at 8:46 AM Gorry Fairhurst <[email-masked]>
wrote:

> *27. Appendix B and C *
> ⁃            Appendix B is lengthy, at best loosely connected with the
> rest of the draft, definitely out of date, and not aligned with current WG
> views of this area. This appendix can be removed prior to publication.
> For reference, here's the end of the ToC as it stands before the proposed
> edits:
>
>   Appendix B <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B>.  Alternative Identifiers  . . . . . . . . . . . . . .  46 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-46>
>      B.1 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.1>.  ECT(1) and CE codepoints  . . . . . . . . . . . . . . . .  47 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-47>
>      B.2 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.2>.  ECN-DualQ-SCE1  . . . . . . . . . . . . . . . . . . . . .  51 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-51>
>      B.3 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.3>.  ECN-DualQ-SCE0  . . . . . . . . . . . . . . . . . . . . .  54 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-54>
>      B.4 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.4>.  ECN Plus a Diffserv Codepoint (DSCP)  . . . . . . . . . .  56 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-56>
>      B.5 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.5>.  ECN capability alone  . . . . . . . . . . . . . . . . . .  58 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-58>
>      B.6 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.6>.  Protocol ID . . . . . . . . . . . . . . . . . . . . . . .  58 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-58>
>      B.7 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.7>.  Source or destination addressing  . . . . . . . . . . . .  59 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-59>
>      B.8 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-B.8>.  Summary: Merits of Alternative Identifiers  . . . . . . .  59 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-59>
>      C.1 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-C.1>.  Integrity of Congestion Feedback  . . . . . . . . . . . .  60 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-60>
>      C.2 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#appendix-C.2>.  Notification of Less Severe Congestion than CE  . . . . .  61 <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id#page-61>
>
>
>
> Bob
>
> --

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 14 May 2021 19:08:48 +0100

Gorry, all,

I've recorded here how I dealt with nits, unless I just did as asked.
See [BB]

On 06/05/2021 07:52, Gorry Fairhurst wrote:
>
> *These points below are Nits, but if you don’t think so, we can 
> discuss more?*
>
> **
>

[BB] end-points

In S.5.1, I've referred forward to 5.2 for the relative strengths:

    This
    Classic AQM treatment need not mark ECT(0) packets, but if it does,
    see Section 5.2 for the strengths of the markings relative to drop.

[BB] Deliberate. S.4 is for transports. S.5 is for network AQMs.
A CC developer might solely focus on reading S.4, and an AQM developer 
only S.5

> compliant but not set ECT(1) and so doesn’t respond… This all seems 
> like a difficult point to pursue - or maybe it simply does not need to 
> be said at all. Can we delete this parenthesis?
>

[BB] I've changed it to:
     "address ranges of specific applications or hosts configured to be, 
or known to be, safe, e.g. hard-coded IoT devices sending low intensity 
traffic;"

FYI, the original text was thinking of some consumer device where you 
can modify the congestion control, but packet sending is done with 
unmodifiable hardware. However, that's not a good example of low latency 
but non-L4S, so I've used IoT instead.

> ⁃I think we have elsewhere called these “Low Data-Volume 
> Applications”. The term lightweight is used in other places in the 
> IETF for other meanings.
>

[BB] First, a picky point: NQB does not go into the same queue as L4S 
unless the operator arranges it (which is the case in DOCSIS, but other 
operators can do what they want). Assuming it does then, yes, a packet 
with both markings would be classified into the L queue. I think this is 
so obvious it's taken you unawares. So I've changed it round as follows:

    A packet that carries one of these non-ECN identifiers to classify it
    into the L queue would not be subject to the L4S ECN marking
    treatment, unless it also carried an ECT(1) or CE codepoint.  The
    specification of an L4S AQM MUST define the behaviour for packets
    with unexpected combinations of codepoints, e.g. a non-ECN-based
    classifier for the L queue, but ECT(0) in the ECN field (for examples
    see section 2.5.1.1 of [I-D.ietf-tsvwg-aqm-dualq-coupled]).

> worth writing this simply.

[BB] I can't think how to say these things more simply. Pls suggest text 
if you have ideas.

> ⁃GF: is the sentence with “however...” strictly required here? I fear 
> it could be mistakenly read, and without it, the next sentence is 
> clear about what is intended.
>

[BB] I've run the two together:

"However, it is believed they do not need to, because such 
implementations are believed to solely exist in controlled environments, 
where the network technology keeps reordering extremely low anyway."

> ⁃I don’t like this interpretation of what TCPM will finally conclude 
> is useful. It’s OK to say this proposes something, if you want to say 
> it concludes something then we need to block publication on the prior 
> publication of the other ID. Can you please please remove “counters 
> each argument in RFC 3168 in turn, showing it was 
> over-cautious.Instead it “
>
[BB] Good catch - meant to be just 4 Mb/s

As always many many thanks for catching all these.

Cheers

Bob

---

From: "Black, David" <[email-masked]>
Date: Fri, 14 May 2021 19:11:48 +0000

Ok, leave informative reference in, and the WG can decide what to do with/about the l4s-diffserv draft later.

Thanks, --David

Gorry, David, Wes,
See [BB]
On 06/05/2021 07:52, Gorry Fairhurst wrote:
=================================================================
26. L4S Diffserv draft is not being pursued

   “[I-D.briscoe-tsvwg-l4s-diffserv] gives a number of examples of such
   arrangements to address various requirements. [I-D.briscoe-tsvwg-l4s-diffserv]
   describes how an operator might use L4S to offer low latency for all L4S traffic as well as
   using Diffserv for bandwidth differentiation.  It identifies two main types
   of approach, …”

⁃            If [I-D.briscoe-tsvwg-l4s-diffserv] is not being pursued, this is not a useful reference in a spec RFC.
                  References to it should be removed by rewriting the text, e.g., “There are two primary types of approaches, …”
=================================================================

[BB] It has since been clarified with the chairs (off list) that the L4S Diffserv draft was left in a holding pattern while the main L4S drafts are processed, so its fate has not been decided either way. Altho' it is an individual draft, this informative reference will be left in, because ecn-l4s-id only discusses combining L4S and Diffserv /identifiers/, whereas l4s-diffserv discussed when it would make sense to combine L4S and Diffserv /services/ (which would motivate combining the identifiers, and it also allows combining identifiers to be discussed in more detail and in context).

Bob

--

Bob Briscoe                               http://bobbriscoe.net/ [bobbriscoe.net]<https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!LpKI!1PYnM1BawVJa47Mjd8Qsxz53ATBZg7p9l2Na7l0kq0ozhOLgERa87mfPwDjNzO4a$>

---

From: "Black, David" <[email-masked]>
Date: Fri, 14 May 2021 19:13:22 +0000

On 14/05/2021 15:47, Bob Briscoe wrote:
Gorry, David,
On 06/05/2021 07:52, Gorry Fairhurst wrote:
=================================================================
24. Requirements in 5.4.1.
“Such
   non-ECN-based packet types MUST be safe to mix with L4S traffic
   without harming the low latency service, where 'safe' is explained in
   Section 5.4.1.1.1 below.”

⁃            It is not clear what the MUST relates to, I think this needs to be more like (you’d still need to explain harm): “Non-L4S packets that are mixed with L4S traffic MUST NOT harm the low latency service, where 'safe' is explained in Section 5.4.1.1.1 below. and… Consider use of “SHOULD” here, as the consequences of not following it are stated, but not the current “MUST”.

[BB] Actually, I now realize that this has been written as a requirement on the operator, not the implementation. See below for fix.

“Therefore, a network element that
   implements L4S in a shared queue MAY classify additional packets into
   the L queue if they carry certain non-ECN identifiers.”
⁃           This later sentence appears to be a requirement on the same thing, but isn’t the same
⁃            “MAY classify additional packets” -> “MAY classify additional ‘safe’ packets” would provide consistency.
=================================================================

[BB] To resolve the problems with both the above normative sentences, I have deleted them both. And introduced the bullet list of different types of identifier with these words instead: "Examples of relevant non-ECN identifiers are:"

   Section 5.4.1.1.1.

--

Bob Briscoe                               http://bobbriscoe.net/ [bobbriscoe.net]<https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!LpKI!0jXfCejEct4n89kIgsjIJqII_v6UaoH1vvcfs6MuMNMoyUgrBX83ErRnEpiDVZ1p$>

Gorry

[David>] +1

---

From: "Rodney W. Grimes" <[email-masked]>
Date: Fri, 14 May 2021 13:16:29 -0700 (PDT)

[ Charset UTF-8 unsupported, converting... ]
> >> ?ClassicCongestion Control:A congestion control behaviour that can
> >> significantly negative impact on its flow rate [RFC5033].?
> >> **
> >> ?This is one of a number of references to TCP Reno, which would be 

I know of:
	TCP Tahoe
	TCP Reno
	TCP New Remo

I do not know what:
	"standard Reno"
	"standard TCP Reno"
are.

RFC 5681 does not define any of these as terms.

---

From: "Black, David" <[email-masked]>
Date: Fri, 14 May 2021 20:33:57 +0000

This looks good to me, Thanks, --David

See [BB]...

3. The Abstract can be seen as marketing. In the past the IESG has taken exception to “claims” in an abstract of a standards-track document by marketing of “ultra-low”.

Abstract text says:
   so that ultra-low and consistently low
   queuing delay (typically sub-millisecond on average) becomes possible
   for L4S traffic without compromising link utilization.

⁃     Please avoid the assertion of “ultra-low” since this seems like a marketing brag.

[BB see end]

⁃      The abstract is still long, and some of the comparison with ‘Classic ECN’ could be moved
⁃      into the Introduction.

[BB] OK. We can take out sentence 2 onwards, until 'Nonetheless' (inclusive).

=================================================================
9. More marketing at the end of the Introduction

This text:
“It gives an incremental migration path so that suitably modified network bottlenecks can
   distinguish and isolate existing Classic traffic from L4S traffic to prevent the former from
   degrading the ultra-low delay and loss of the new scalable transports, without harming
   Classic performance.”

⁃            This is better than the corresponding Abstract text (see item 2 above), but still contains marketing – “ultra-low” needs to be changed to either “low” or “consistently low”. I suggest toi add  “at the modified network bottlenecks” after “without harming Classic performance.”  to avoid the implication that the absence of harm is global (which is known not to be the case).

I would like to explain why we had left 'ultra' in when we went through de-hyping the document previously. 'Ultra-low delay' was used throughout as a defined term. We figured it would be hard for a reader to recognize that 'very low delay' is a defined term, because it just looks like some vague words. However, this is a price we have to pay to make progress.

CURRENT:
ultra-low and consistently low queuing delay (typically sub-millisecond on average)
PROPOSED:
very low (typically sub-millisecond on average) and consistently low queuing delay :
REASON:
There seems to be some confusion whether we're talking about very low delay or consistently low delay. The answer is both.

And to the Introduction:
CURRENT:
   “It gives an incremental migration path so that suitably modified network bottlenecks can
   distinguish and isolate existing Classic traffic from L4S traffic to prevent the former from
   degrading the ultra-low delay and loss of the new scalable transports, without harming
   Classic performance.”
PROPOSED:
   “It gives an incremental migration path so that suitably modified network bottlenecks can
   distinguish and isolate existing Classic traffic from L4S traffic to prevent the former from
   degrading the very low delay and loss of the new scalable transports, without harming
   Classic performance at these bottlenecks.”
REASON:
   As requested but less clunky than suggested.

Bob

--

Bob Briscoe                               http://bobbriscoe.net/ [bobbriscoe.net]<https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!LpKI!1kCMEcRP1DmGVPLrjPo4u_4t-OLHvQsJHrom3v9rVElFBV8OvGeT2eOhe1RqotoO$>

---

From: "Black, David" <[email-masked]>
Date: Fri, 14 May 2021 20:38:18 +0000

Gorry,
See [BB2]
10. Please avoid making a claim that the IETF is NOT making a statement about usability of DSCPs in this document.

This text grates:

   will be latency-sensitive.  Then, given nothing to differentiate

⁃            To me this is not balanced. It seems to suggest that setting a DSCP is useless. I don’t believe that is the consensus of tsvwg, although at some additional pain we can debate the merits of setting DSCPs for a traffic - such as the implications in enterprise networks; the implications on UP in access points, etc. I’m not against this discussion, but needing this to progress the draft seems unfortunate to me. “Then, given nothing to differentiate   from, Diffserv makes no difference.” is gratuitous and should be removed.  The paragraph continues with “Instead, we need to remove the causes of any unnecessary delay.” which can be combined into the previous sentence as: “ … will be latency-sensitive, making it imperative to remove the underlying causes of delay.”  Nothing is lost here because the crucial comparison with Diffserv is picked up two paragraphs later: “Unlike Diffserv, which gives low latency to some traffic at the expense of others, AQM controls latency for _all_ traffic in a clas

[BB] I appreciate that you are trying to avoid upsetting anyone in the IETF who might take exception to criticism of Diffserv. However, I have tried to make it clear below that this is about limits to its applicability, not criticism of the whole idea.

This whole section is about the limits of other pre-existing low latency technologies. So it has something to irritate everyone - inherently. It is unfortunately necessary because, when I'm asked to review drafts from other areas of the IETF, I often have to ask the authors to say why they are proposing another way to do something that can already be done (which is the opposite of what a standards body ought to be encouraging - unless there's a new problem).

Second attempt...
[...]

Reasoning: Instead of relying on the reader connecting two concepts across paragraphs, this spells out the problem. This para's role was to explain the problem first. Before the subsequent para with the solution (AQMs reduce delay for all traffic).

But to soften the offending sentence, I've conditioned it  on "During these periods" and "at these bottlenecks".

The problem may seem obvious or gratuitous to experts in traffic control. However, to many networks engineers who are not traffic experts, Diffserv is something you take off a pick-list - "if you need low latency, you stick a low latency DSCP on the packets".

Bob

--

Bob Briscoe                               http://bobbriscoe.net/ [bobbriscoe.net]<https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!LpKI!zF9Ua67olmvIDP5O29IrV6vgiOMEj1C0P-PTxGHQe0x9Ls1UAXsPrZ1Dh4Z8HSfK$>

To me this helps a lot, but I'd prefer to avoid /given nothing to differentiate from/ and explain, is this better?:

During these periods, if all the traffic were marked for the same treatment at these bottlenecks,
Diffserv would make no difference./

Goirry

[BB2] Done.

[David>] That works for me.
Bob

--

Bob Briscoe                               http://bobbriscoe.net/ [bobbriscoe.net]<https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!LpKI!zF9Ua67olmvIDP5O29IrV6vgiOMEj1C0P-PTxGHQe0x9Ls1UAXsPrZ1Dh4Z8HSfK$>

---

From: "Black, David" <[email-masked]>
Date: Fri, 14 May 2021 20:40:46 +0000

Dell Customer Communication - Confidential

This is a good approach - I agree with Gorry and Martin.  Thanks, --David

From: Martin Duke <[email-masked]>
Sent: Friday, May 14, 2021 1:33 PM
To: Gorry Fairhurst
Cc: Bob Briscoe; Black, David; Wesley Eddy; [email-masked]
Subject: Re: [tsvwg] Review comments on a careful read of the L4S ID (27. Appx B & C)

SGTM

On Fri, May 14, 2021 at 8:46 AM Gorry Fairhurst <[email-masked]<mailto:[email-masked]>> wrote:
On 14/05/2021 16:09, Bob Briscoe wrote:
Chairs, Martin,
See [BB]

ACK.

Gorry
27. Appendix B and C

⁃            Appendix B is lengthy, at best loosely connected with the rest of the draft, definitely out of date, and not aligned with current WG views of this area. This appendix can be removed prior to publication. Appendix C is unecessary to the final publication.
 =================================================================

[BB] Offlist discussion with the chairs and Martin has come up with the solution of deleting the sub-sections of these appendices that are not referenced from the rest of the draft, but keeping those that are.

B.1 is referenced from the body of the draft a couple of times, because it's the pros and cons of the chosen scheme.
C.1 is referenced from the security considerations.

So I'm planning on promoting B.1 to B, and C.1 to C.
But I'll take out the remainder of B and C, and change the introductory context of each.

For reference, here's the end of the ToC as it stands before the proposed edits:

  Appendix B [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-B__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ6AHDr5T$>.  Alternative Identifiers  . . . . . . . . . . . . . .  46 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*page-46__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQzd684zq$>

     B.1 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-B.1__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQy6q3Dje$>.  ECT(1) and CE codepoints  . . . . . . . . . . . . . . . .  47 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*page-47__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ-awesXy$>

     B.2 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-B.2__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ2XxQ8wo$>.  ECN-DualQ-SCE1  . . . . . . . . . . . . . . . . . . . . .  51 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*page-51__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQyelDoYl$>

     B.3 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-B.3__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQwfULdT3$>.  ECN-DualQ-SCE0  . . . . . . . . . . . . . . . . . . . . .  54 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*page-54__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQy_br1jM$>

     B.4 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-B.4__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ6MLannR$>.  ECN Plus a Diffserv Codepoint (DSCP)  . . . . . . . . . .  56 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*page-56__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ64Rhzca$>

     B.5 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-B.5__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ0gamkn7$>.  ECN capability alone  . . . . . . . . . . . . . . . . . .  58 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*page-58__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ3Szy7v_$>

     B.6 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-B.6__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ5cRVsMD$>.  Protocol ID . . . . . . . . . . . . . . . . . . . . . . .  58 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*page-58__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ3Szy7v_$>

     B.7 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-B.7__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ8lCv0zk$>.  Source or destination addressing  . . . . . . . . . . . .  59 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*page-59__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQyB5VPQU$>

     B.8 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-B.8__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ3HQm7eU$>.  Summary: Merits of Alternative Identifiers  . . . . . . .  59 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*page-59__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQyB5VPQU$>

   Appendix C [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-C__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ6okbzfI$>.  Potential Competing Uses for the ECT(1) Codepoint  .  60

     C.1 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-C.1__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ_Woa064$>.  Integrity of Congestion Feedback  . . . . . . . . . . . .  60 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*page-60__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ15ZU4t1$>

     C.2 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*appendix-C.2__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ_pHcpIM$>.  Notification of Less Severe Congestion than CE  . . . . .  61 [datatracker.ietf.org]<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id*page-61__;Iw!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ5kdaXEm$>

Bob

--

Bob Briscoe                               http://bobbriscoe.net/ [bobbriscoe.net]<https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!LpKI!yiN9x6gFkQ-F9vMztrvCBAuLhKGnQCG2jTiC5vGE4Bc63eBgTtCbxKpVQ1vMSIKp$>