Subject: Re: [tsvwg] L4S: Guard DSCP

From: Jonathan Morton <[email-masked]>
Date: Sat, 1 May 2021 20:17:49 +0300

> On 1 May, 2021, at 12:21 am, Greg White <[email-masked]> wrote:
> 
> There has been little evidence shown of single queue RFC3168 bottlenecks.  In Pete Heist’s data (one small ISP in Czech) the known instances of RFC3168 were all fq_codel, and the rest of the observed CE markings were agreed to likely be fq_codel or CAKE as well.  Similarly the earlier data from Apple showing some level of ECN marking in France were consistent with the reported fq_codel deployments by one of the ISPs there.  Also, I don’t believe that any ISP has come forward to state that they have single queue RFC3168 bottlenecks in their network (and, given the amount of list traffic on this topic, I would assume that any ISP that monitors TSVWG would have self-identified by now).  

Let's not forget that there are other ways that parts of the 5-tuple can get hidden from an otherwise flow-aware AQM.  When all the flows are of conventional type, this is only of minor consequence.  When an L4S flow gets mixed in, there is a big problem which the conventional flows' endpoints have no way to mitigate (in particular, disabling ECN for themselves doesn't help).

We've shown that this can feasibly happen with existing VPN tunnels, and we saw some evidence of VPN tunnels being used in the Czech data.  Though we couldn't reliably conclude that ECN was being used over them, that does not mean there is no problem.  If L4S endpoints were introduced at both ends of one of those tunnels, there would be L4S traffic over that tunnel, and that would interact with the conventional traffic in that same tunnel, regardless of whether it was RFC-3168 enabled or Not-ECT.

The persistent minimisation of this externalised risk by calling it "rare", rather than tackling it honestly, is a big part of why I, and others I'm aware of, do not trust L4S to conduct their experiment safely or produce an accurate appraisal of the results.

 - Jonathan Morton

---

From: Ingemar Johansson S <[email-masked]>
Date: Mon, 3 May 2021 06:21:58 +0000

Hi Jonathan

This discussion appears to run in circles and each time it delves into all sorts of problems flow aware AQMs will/may see when L4S experiments are run. The obvious solution would be to fix these issues as already pointed out by Greg earlier in this mail thread but that appears to be completely out of the question. 
This gives me the impression that fq-codel is used as a pawn in this endless debate for whatever reason. I don't believe that it will be a particularly successful approach in the long run if L4S proves to be widely picked up. If you want fq-codel to be relevant in the future you should really take possibility that L4S will fly into account and weight the risks/benefits accordingly. 

/Ingemar

> -----Original Message-----
> From: tsvwg <[email-masked]> On Behalf Of Jonathan Morton
> Sent: den 1 maj 2021 19:18
> To: Greg White <[email-masked]>
> Cc: TSVWG <[email-masked]>
> Subject: Re: [tsvwg] L4S: Guard DSCP
> 
> > There has been little evidence shown of single queue RFC3168 bottlenecks.
> In Pete Heist’s data (one small ISP in Czech) the known instances of RFC3168
> were all fq_codel, and the rest of the observed CE markings were agreed to
> likely be fq_codel or CAKE as well.  Similarly the earlier data from Apple
> showing some level of ECN marking in France were consistent with the
> reported fq_codel deployments by one of the ISPs there.  Also, I don’t
> believe that any ISP has come forward to state that they have single queue
> RFC3168 bottlenecks in their network (and, given the amount of list traffic on
> this topic, I would assume that any ISP that monitors TSVWG would have
> self-identified by now).
> 
> Let's not forget that there are other ways that parts of the 5-tuple can get
> hidden from an otherwise flow-aware AQM.  When all the flows are of
> conventional type, this is only of minor consequence.  When an L4S flow gets
> mixed in, there is a big problem which the conventional flows' endpoints
> have no way to mitigate (in particular, disabling ECN for themselves doesn't
> help).
> 
> We've shown that this can feasibly happen with existing VPN tunnels, and
> we saw some evidence of VPN tunnels being used in the Czech data.  Though
> we couldn't reliably conclude that ECN was being used over them, that does
> not mean there is no problem.  If L4S endpoints were introduced at both
> ends of one of those tunnels, there would be L4S traffic over that tunnel,
> and that would interact with the conventional traffic in that same tunnel,
> regardless of whether it was RFC-3168 enabled or Not-ECT.
> 
> The persistent minimisation of this externalised risk by calling it "rare", rather
> than tackling it honestly, is a big part of why I, and others I'm aware of, do not
> trust L4S to conduct their experiment safely or produce an accurate appraisal
> of the results.
> 
>  - Jonathan Morton

---

From: Jonathan Morton <[email-masked]>
Date: Mon, 3 May 2021 09:33:47 +0300

> On 3 May, 2021, at 9:21 am, Ingemar Johansson S <[email-masked]> wrote:
> 

No - the discussion goes in circles because you (in the plural) refuse to accept that L4S is the *cause* of the problem.  I have been consistent in pointing that out, however tiresome it becomes to repeat myself, because it is simply and demonstrably true.

Notice, for example, that just plain Codel - without the FQ - is an example of the "single queue RFC-3168 AQM" that is *guaranteed* to have problems with L4S, while there are merely *some* circumstances where fq_codel is unable to supply a workaround.  So FQ is not the villain in this piece.

The fact is that fq_codel and Cake are out there, and together constitute the majority, at this time, of ECN AQM deployment.  That is the environment which L4S must interact appropriately with if it is to deployed on the Internet.  Even if we started changing them to accommodate L4S today, the existing deployments will not be eliminated for many years - by which time L4S will be a forgotten footnote in the annals of abject failure.

 - Jonathan Morton

---

From: Ingemar Johansson S <[email-masked]>
Date: Mon, 3 May 2021 06:52:07 +0000

Hi

I don't have a desire to take this discussion any further, we have been
there before. You have to weight the risks/benefits with your standpoint and
how/if you wish to adapt to L4S, that's nothing that I (or anybody else?)
can decide for you.

/Ingemar

> From: Jonathan Morton <[email-masked]>
> Sent: den 3 maj 2021 08:34
> To: Ingemar Johansson S <[email-masked]>
> Cc: Greg White <[email-masked]>; TSVWG <[email-masked]>
> > On 3 May, 2021, at 9:21 am, Ingemar Johansson S
> <[email-masked]> wrote:
> >
> > This discussion appears to run in circles and each time it delves into
all sorts
> of problems flow aware AQMs will/may see when L4S experiments are run.
> The obvious solution would be to fix these issues as already pointed out
by
> Greg earlier in this mail thread but that appears to be completely out of
the
> question.
> 
> No - the discussion goes in circles because you (in the plural) refuse to
accept
> that L4S is the *cause* of the problem.  I have been consistent in
pointing
> that out, however tiresome it becomes to repeat myself, because it is
simply
> and demonstrably true.
> 
> Notice, for example, that just plain Codel - without the FQ - is an
example of
> the "single queue RFC-3168 AQM" that is *guaranteed* to have problems
> with L4S, while there are merely *some* circumstances where fq_codel is
> unable to supply a workaround.  So FQ is not the villain in this piece.
> 
> The fact is that fq_codel and Cake are out there, and together constitute
the
> majority, at this time, of ECN AQM deployment.  That is the environment
> which L4S must interact appropriately with if it is to deployed on the
Internet.
> Even if we started changing them to accommodate L4S today, the existing
> deployments will not be eliminated for many years - by which time L4S will
be
> a forgotten footnote in the annals of abject failure.
> 
>  - Jonathan Morton

---

From: Jonathan Morton <[email-masked]>
Date: Mon, 3 May 2021 09:55:39 +0300

> On 3 May, 2021, at 9:52 am, Ingemar Johansson S <[email-masked]> wrote:
> 
> You have to weight the risks/benefits with your standpoint and

But that is the whole problem - why should I, or Joe Random Public, have to "adapt to L4S" in order for L4S to be safely deployable?  We should not.  L4S should adapt to accommodate already-standardised, already-deployed infrastructure - or die.

 - Jonathan Morton

---

From: Pete Heist <[email-masked]>
Date: Mon, 03 May 2021 09:42:11 +0200

On Fri, 2021-04-30 at 21:21 +0000, Greg White wrote:
>  
> 
> [GW] There has been little evidence shown of single queue RFC3168
> bottlenecks.  In Pete Heist’s data (one small ISP in Czech) the known
> instances of RFC3168 were all fq_codel, and the rest of the observed
> CE markings were agreed to likely be fq_codel or CAKE as well.
>  Similarly the earlier data from Apple showing some level of ECN
> marking in France were consistent with the reported fq_codel
> deployments by one of the ISPs there.  Also, I don’t believe that any
> ISP has come forward to state that they have single queue RFC3168
> bottlenecks in their network (and, given the amount of list traffic
> on this topic, I would assume that any ISP that monitors TSVWG would
> have self-identified by now).  

While the known RFC3168 AQM marking in this case was from fq_codel,
it's hard to say what the unknown marking was from
(https://datatracker.ietf.org/doc/draft-heist-tsvwg-ecn-deployment-observations/
). Those could be fq_codel/Cake, WiFi drivers in Linux (afaik including
ath9k, ath10k, mt76 and iwl drivers), or even if uncommon, single queue
AQMs in non-default switch configs. The latter could lead to all non-
L4S flows through a bottleneck, ECN capable or not, being dominated by
a single L4S flow.

Regarding WiFi: In my spare time I help administer a WiFi network that
includes five Open Mesh OM2P-HS APs (different ISP/region from where
the ECN data was collected). All of those APs received Linux's ath9k
drivers with fq_codel via an automatic update several years ago,
without my intervention. It's not possible in the admin interface to
disable it, but it is possible and even reasonable to turn Automatic
Updates off, which would add delay to any patches required by the L4S
experiment. Most users of this and other WiFi products are innocent
bystanders with no knowledge of RFC3168.

Coming back to the Guard DSCP topic, the discussion seems to show a
fault line on whether or not the L4S experiment needs to be contained
to participating networks. In my opinion, it does, and specifically
using DSCP, or an equivalent mechanism not involving the ECN bits
alone, so as to preserve their end-to-end passage. I think the history
of ToS and DSCP suggests that netops folks faced with (or just aware
of) problems will act using the simplest method available to them, i.e.
bleaching. The introduction of a signaling mechanism that isn't
compatible with RFC3168, a proposed standard which is currently in use,
would unfortunately invite that, and could make bleaching of the entire
DS byte routine.

Mike's proposal to use DSCP at the endpoints seems sensible to me,
avoiding any action on the ECN field in the network.

Pete

> > I recall from the lengthy discussion that went on about one year
> > ago (see,
> > e.g., https://mailarchive.ietf.org/arch/msg/tsvwg/ij5BqzNtMxKV8VbHE
> > 8pUevZjx7k/ [mailarchive.ietf.org]) that one of the big objections
> > to the use of a Guard DSCP as a means to contain the experiment is
> > that an L4S experimental domain does not map well to a DSCP domain,
> > and so the Guard DSCP may be inadvertently bleached (e.g. by
> > tunnels) inside the participating network. If correct, that
> > objection is hard to address.
> 
>  
> Mike Heard

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 3 May 2021 21:39:39 -0700

On Tue, Apr 27, 2021 at 3:57 PM Black, David wrote:

> the appropriate L4S experiment scope may be sets of
> cooperating/participating networks rather than the (originally envisioned)
> Internet as a whole.
>

I had certainly understood the intended scope to be sets of
cooperating/participating networks rather than the Internet as a whole. How
could it possibly be otherwise, with a change in the semantics of CE that
is incompatible with existing standards, without at the same time
obsoleting those standards?

When Differentiated Services was introduced by RFC 2474, the existing TOS
standard was obsoleted, based on good evidence that it was unused (there
was only one implementation of TOS routing, IIRC, and negligible
deployment). It didn't get introduced as an experiment.

When the original experimental incarnation of ECN was specified by RFC
2481, it could be introduced as an internet-wide experiment because prior
standardized uses of the field had been obsoleted on good evidence of
non-use, and there was good evidence that harm would not be caused to
deployed systems.

The evidence presented here strongly suggests to me that L4S is not benign
to existing standards. If it is to be used experimentally, it must be
contained. If it is to be used internet-wide, then existing incompatible
uses should be obsoleted.

One or the other, please. Don't deploy L4S internet-wide and call it
experimental.

Mike Heard

---

From: "Black, David" <[email-masked]>
Date: Tue, 4 May 2021 16:35:26 +0000

Mike,

> I had certainly understood the intended scope to be sets of cooperating/participating networks rather than the Internet as a whole.
> How could it possibly be otherwise, with a change in the semantics of CE that is incompatible with existing standards, without at the same time obsoleting those standards?

Short answer (as I understand it) – L4S was originally intended to be compatible.  For details of what compatible means in this context, see Section 4.3 of RFC 4774 (Option 3:  Friendly Coexistence with Competing Traffic, https://datatracker.ietf.org/doc/html/rfc4774#section-4.3).

> The evidence presented here strongly suggests to me that L4S is not benign to existing standards. If it is to be used experimentally, it must be contained.
> If it is to be used internet-wide, then existing incompatible uses should be obsoleted.

You're not the only person who has expressed the view that L4S is not able to proceed under RFC 4774 Option 3.

Everyone - when responding to this message, please keep in mind that RFC 4774 is a BCP (Best Current Practice) document, namely BCP 124.

Thanks, --David

From: C. M. Heard <[email-masked]>
Sent: Tuesday, May 4, 2021 12:40 AM
To: Black, David
Cc: TSVWG
Subject: Scope of the L4S Experiment (was: Guard DSCP)

[EXTERNAL EMAIL]
On Tue, Apr 27, 2021 at 3:57 PM Black, David wrote:
the appropriate L4S experiment scope may be sets of cooperating/participating networks rather than the (originally envisioned) Internet as a whole.

I had certainly understood the intended scope to be sets of cooperating/participating networks rather than the Internet as a whole. How could it possibly be otherwise, with a change in the semantics of CE that is incompatible with existing standards, without at the same time obsoleting those standards?

When Differentiated Services was introduced by RFC 2474, the existing TOS standard was obsoleted, based on good evidence that it was unused (there was only one implementation of TOS routing, IIRC, and negligible deployment). It didn't get introduced as an experiment.

When the original experimental incarnation of ECN was specified by RFC 2481, it could be introduced as an internet-wide experiment because prior standardized uses of the field had been obsoleted on good evidence of non-use, and there was good evidence that harm would not be caused to deployed systems.

The evidence presented here strongly suggests to me that L4S is not benign to existing standards. If it is to be used experimentally, it must be contained. If it is to be used internet-wide, then existing incompatible uses should be obsoleted.

One or the other, please. Don't deploy L4S internet-wide and call it experimental.

Mike Heard

---

From: Jonathan Morton <[email-masked]>
Date: Tue, 4 May 2021 19:44:16 +0300

> On 4 May, 2021, at 7:35 pm, Black, David <[email-masked]> wrote:
> 

That surprises me a bit, since there's nothing in L4S' design that makes it inherently compatible with RFC-3168 traffic.  With a working "Classic ECN detection heuristic" they could reasonably claim Option 2, ie. detecting whether they were in a participating network or not and switching modes.  At present, they don't even have that.

> You're not the only person who has expressed the view that L4S is not able to proceed under RFC 4774 Option 3.

I will remind readers briefly that SCE *is* an RFC-4774 Option 3 proposal.  That is what sets it apart from L4S.

 - Jonathan Morton

---

From: Steven Blake <[email-masked]>
Date: Tue, 04 May 2021 13:05:38 -0400

On Tue, 2021-05-04 at 16:35 +0000, Black, David wrote:
> cooperating/participating networks rather than the Internet as a
> whole.
> > How could it possibly be otherwise, with a change in the
> semantics of CE that is incompatible with existing standards, without
> at the same time obsoleting those standards?
>  
> Short answer (as I understand it) – L4S was originally intended to be
> compatible.  For details of what compatible means in this context,
> see Section 4.3 of RFC 4774 (Option 3:  Friendly Coexistence with
> Competing
>  Traffic, https://datatracker.ietf.org/doc/html/rfc4774#section-4.3).
>  
> > The evidence presented here strongly suggests to me that L4S is not
> benign to existing standards. If it is to be used experimentally, it
> must be contained.
> Everyone - when responding to this message, please keep in mind that
> RFC 4774 is a BCP (Best Current Practice) document, namely BCP 124.
>  
> Thanks, --David
>  
> 
> > cooperating/participating networks rather than the (originally
> > envisioned) Internet as a whole.
> > 
> > 
> 
>  
> 
> 
> cooperating/participating networks rather than the Internet as a
> whole. How could it possibly be otherwise, with a change in the
> semantics of CE that is incompatible with existing
>  standards, without at the same time obsoleting those standards?
> 
> 
>  
> 
> 
> When Differentiated Services was introduced by RFC 2474, the existing
> TOS standard was obsoleted, based on good evidence that it was unused
> (there was only one implementation of TOS routing, IIRC, and
> negligible deployment). It didn't get
>  introduced as an experiment.
> 
> 
>  
> 
> 
> When the original experimental incarnation of ECN was specified by
> RFC 2481, it could be introduced as an internet-wide experiment
> because prior standardized uses of the field had been obsoleted on
> good evidence of non-use, and there was
>  good evidence that harm would not be caused to deployed systems.
> 
> 
>  
> 
> 
> The evidence presented here strongly suggests to me that L4S is not
> benign to existing standards. If it is to be used experimentally, it
> must be contained. If it is to be used internet-wide, then existing
> incompatible uses should be obsoleted.
> 
> 
>  
> 
> 

To reiterate what I wrote on March 26: if there is really so little RFC
3168 ECN deployed in the Internet that it can be safely ignored for an
Internet-wide ECN experiment with incompatible signaling and congestion
response, then the only responsible thing to do is to go through the
process of moving RFC 3168 to historic.

Regards,

// Steve

---

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 5 May 2021 09:24:34 +0000

Hi

My comment, admittedly with too little experience in IETF nomenclature and lingo so it is not too unlikely that I am off here.

I read both RFC4774 and 

https://www.iana.org/assignments/dscp-registry/dscp-registry.xhtml

The latter lists ECT(1) for experimental use only, with a reference to RFC8311

My interpretation is that for future deployment of AQMs (fq-codel, cake) only ECT(0) should be used for classic ECN. This means that this part of the problem goes away gradually.

Then we have already deployed devices, and here opinions differ around everything from upgradeability to expected life time of these devices. And here RFC4774 and in particular option 3 can be applicable.

As mentioned below RFC4774 is a BCP, I had to look up the meaning of this. My understanding of  (https://tools.ietf.org/html/rfc1818) is that it lists a set of guidance rules at the time of writing.  But quote:“The resulting document is viewed as having the technical approval of the IETF, but it is not, and  cannot become an official Internet Standard”. My understanding is then that a BCP gives more slack than a proposed standard. 

Can option 3 in RFC4774 be side stepped ?, if RFC4774 is read as being an official Internet standard, then the answer is definitely no. But RFC4774 is a BCP. 

The question is then, is it safe enough to side step option 3 ?. Opinions differ here. My opinion is that there I (still) little evidence of deployed and enabled RFC3168 AQMs and it is furthermore a limited set of few corner cases where L4S traffic can potentially starve out classic ECN traffic.

Also, internet gear (routers, switches, home gateways) have an economical or physical lifetime of just a few years. So my impression is that a large share of the deployed gear has been upgraded or even replaced by the time that we have standardized a guard DCSP. And then we have allocated a lot of time for very little gain and in the process delayed L4S standardization even more.

/Ingemar

From: tsvwg <[email-masked]> On Behalf Of Steven Blake
Sent: den 4 maj 2021 19:06
To: Black, David <[email-masked]>; C. M. Heard <[email-masked]>
Cc: TSVWG <[email-masked]>
Subject: Re: [tsvwg] Scope of the L4S Experiment (was: Guard DSCP)

Mike,

Thanks, --David

From: C. M. Heard <[email-masked] <mailto:[email-masked]> > 
Sent: Tuesday, May 4, 2021 12:40 AM
To: Black, David
Cc: TSVWG
Subject: Scope of the L4S Experiment (was: Guard DSCP)

[EXTERNAL EMAIL] 

Mike Heard

To reiterate what I wrote on March 26: if there is really so little RFC 3168 ECN deployed in the Internet that it can be safely ignored for an Internet-wide ECN experiment with incompatible signaling and congestion response, then the only responsible thing to do is to go through the process of moving RFC 3168 to historic.

Regards,

// Steve

---

From: Jonathan Morton <[email-masked]>
Date: Wed, 5 May 2021 16:47:03 +0300

> On 5 May, 2021, at 12:24 pm, Ingemar Johansson S <ingemar.s.johansson=[email-masked]> wrote:
> 
> As mentioned below RFC4774 is a BCP, I had to look up the meaning of this. My understanding of  (https://tools.ietf.org/html/rfc1818) is that it lists a set of guidance rules at the time of writing.

RFC-1818 has status HISTORIC.  The document you should probably be reading is RFC-2026, Section 5:

	https://tools.ietf.org/html/rfc2026#page-15

>    The BCP subseries of the RFC series is designed to be a way to
>    standardize practices and the results of community deliberations.  A
>    BCP document is subject to the same basic set of procedures as
>    standards track documents and thus is a vehicle by which the IETF
>    community can define and ratify the community's best current thinking
>    on a statement of principle or on what is believed to be the best way
>    to perform some operations or IETF process function.

My view is that BCPs have approximately the same force as Proposed Standard RFCs, but cover a different class of subject matter - policy rather than technicalities.  It is possible to deviate from what a BCP states, but there had better be a very good reason for doing so.  In this case, RFC-4774 details considerations for the standardisation process in the very specific field of congestion control signalling, which is precisely what we are discussing today.

I think you are also seriously misinterpreting what RFC-4774 states.  It talks very specifically about "old routers" as the context for coexistence between "alternate ECN" traffic (ie. L4S), RFC-3168 and loss-based traffic.  That means fq_codel and Cake as they exist *today*, as well as any single-queue AQMs that may be out there, or which may yet be deployed conforming to RFC-3168 - and, incidentally, the bog-standard dumb FIFO, for completeness' sake.  You cannot sidestep that context merely by proposing that fq_codel and Cake be changed; you must actually show that the change has already taken place.  As of right now, it has not.

> Option 3:  The alternate ECN semantics are defined in such a way as
>    to ensure the fair and peaceful coexistence of the alternate-ECN
>    traffic with best-effort and other traffic, even in environments that
>    include old routers that do not understand the alternate ECN
>    semantics.

RFC-4774 Option 3 requires that the new, alternate ECN proposal is inherently compatible with existing traffic at existing routers.  That is categorically *not* true of L4S.  Option 3 therefore does not apply to L4S.

> Option 2:  All alternate-ECN traffic deploys some mechanism for
>    verifying that all routers on the path understand and agree to use
>    the alternate ECN semantics for this traffic

RFC-4774 Option 2 is a slightly more realistic target for L4S.  The "Classic ECN detection heuristic" algorithm aimed in this direction, as does my "dual DSCP" proposal for retrofitting to L4S.

> Option 1:  Alternate-ECN traffic is clearly understood as unsafe for
>    deployment in the global Internet

If neither Option 2 nor Option 3's requirements are met, then Option 1 is the only viable solution.  That is what requires the use of a containment mechanism to explicitly keep L4S within the participating network(s).

 - Jonathan Morton

---

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 5 May 2021 15:05:56 +0000

Hi Jonathan

The comment round the IANA registry was left without comments, so I repeat
it again

>>My interpretation is that for future deployment of AQMs (fq-codel, cake)
only ECT(0) should be used for classic ECN. This means that this part of the
problem >>goes away gradually.
Is this something that can be agreed upon ?

Please see inline [IJ]

/Ingemar

> Sent: den 5 maj 2021 15:47
> Cc: Steven Blake <[email-masked]>; Black, David
> <[email-masked]>; C. M. Heard <[email-masked]>; TSVWG
> <[email-masked]>
> > On 5 May, 2021, at 12:24 pm, Ingemar Johansson S
> <ingemar.s.johansson=[email-masked]> wrote:
> >
> > As mentioned below RFC4774 is a BCP, I had to look up the meaning of
this.
> My understanding of  (https://tools.ietf.org/html/rfc1818) is that it
lists a set
> of guidance rules at the time of writing.
> 

[IJ] Good point, thanks

> My view is that BCPs have approximately the same force as Proposed
> Standard RFCs, but cover a different class of subject matter - policy
rather
> than technicalities.  It is possible to deviate from what a BCP states,
but there
> had better be a very good reason for doing so.  In this case, RFC-4774
details
> considerations for the standardisation process in the very specific field
of
> congestion control signalling, which is precisely what we are discussing
today.
> 
> I think you are also seriously misinterpreting what RFC-4774 states.  It
talks
> very specifically about "old routers" as the context for coexistence
between
> "alternate ECN" traffic (ie. L4S), RFC-3168 and loss-based traffic.  That
means
> fq_codel and Cake as they exist *today*, as well as any single-queue AQMs
> that may be out there, or which may yet be deployed conforming to RFC-
> 3168 - and, incidentally, the bog-standard dumb FIFO, for completeness'
> sake.  You cannot sidestep that context merely by proposing that fq_codel
> and Cake be changed; you must actually show that the change has already
> taken place.  As of right now, it has not.

[IJ] Point taken. But then again, all network gear have a limited lifetime
(mentioned in my email), either economic (increased Opex, energy cost/bit)
or plain physical (things break down, especially home gateways do).  

> RFC-4774 Option 3 requires that the new, alternate ECN proposal is
> inherently compatible with existing traffic at existing routers.  That is
> categorically *not* true of L4S.  Option 3 therefore does not apply to
L4S.

[IJ] That is subject to interpretation, just because what you say may
happen, does not mean that it will happen. In addition there is work on
fall-back solutions in TCP Prague. Sure these are not bullet proof and you
found a few cases where it fails. But the question is how serious these
issues are?. Sure we can continue and discuss this for a few years more and
even device a guard DSCP. But by the time that we are done, the majority of
these old routers that we discuss today have passed EOL and have been
replaced by devices that treat ECT(1) as experimental (or L4S) 

> RFC-4774 Option 2 is a slightly more realistic target for L4S.  The
"Classic ECN
> detection heuristic" algorithm aimed in this direction, as does my "dual
DSCP"
> proposal for retrofitting to L4S.
> 
> If neither Option 2 nor Option 3's requirements are met, then Option 1 is
the
> only viable solution.  That is what requires the use of a containment
> mechanism to explicitly keep L4S within the participating network(s).
> 
>  - Jonathan Morton

---

From: Jonathan Morton <[email-masked]>
Date: Wed, 5 May 2021 18:50:01 +0300

> On 5 May, 2021, at 6:05 pm, Ingemar Johansson S <[email-masked]> wrote:
> 

> problem goes away gradually.

> Is this something that can be agreed upon ?

No, because there are two aspects to use of ECT(1) at play.  You have found only one of them.

RFC-3168 states that ECT(0) and ECT(1) shall be treated identically by middleboxes.  It is of course possible for a later specification to update that, but unless and until RFC-3168 is obsoleted, middleboxes that treat ECT(1) identically to ECT(0) will continue to exist and be newly deployed on the Internet.  This aspect of RFC-3168 was included *specifically* to provide a consistent baseline behaviour upon which experimental uses of ECT(1) could rely, so it seems unlikely to be discarded without serious justification.

One such experimental use was ECN Nonce.  This emitted an unpredictable pattern of ECT(0) and ECT(1) from the sender as a way to verify whether correct RFC-3168 semantics existed on the network path and in the receiver.  It was never deployed, but it would have relied on that aspect of RFC-3168.

Another such use is by SCE, which also relies on existing ECN-aware middleboxes implementing this aspect of RFC-3168.

If L4S cannot operate safely in an environment where existing middleboxes correctly implement well-established RFCs, then L4S has a problem, not the RFC in question.

>> You cannot sidestep that context merely by proposing that fq_codel

Right now, that process is *increasing* the number of RFC-3168 compliant middleboxes on the Internet, not reducing them.  The reverse will only *begin* to occur if you can convince the IETF to obsolete RFC-3168.  I do not see any chance of that happening.

 - Jonathan Morton

---

From: "Black, David" <[email-masked]>
Date: Wed, 5 May 2021 18:44:33 +0000

As a WG chair, without commenting on the rest of this discussion, I agree with this portion of Jonathan's summary of the role of BCPs at IETF:

	My view is that BCPs have approximately the same force as Proposed Standard RFCs,
	but cover a different class of subject matter - policy rather than technicalities.
	It is possible to deviate from what a BCP states, but there had better be a very good
	reason for doing so.  

In the past, I have seen the IESG apply more rigorous review to drafts intended for Best Current Practice status than to drafts intended for Proposed Standard status, but that can vary from year to year.

Thanks, --David

-----Original Message-----
From: Jonathan Morton <[email-masked]> 
Sent: Wednesday, May 5, 2021 9:47 AM
To: Ingemar Johansson S
Cc: Steven Blake; Black, David; C. M. Heard; TSVWG
Subject: Re: [tsvwg] Scope of the L4S Experiment (was: Guard DSCP)

[EXTERNAL EMAIL] 

> As mentioned below RFC4774 is a BCP, I had to look up the meaning of this. My understanding of  (https://urldefense.com/v3/__https://tools.ietf.org/html/rfc1818__;!!LpKI!1k2aHtuNmS8NqaE6aw6FbJXpRSyyQBXX9-m0yAnCxJRUpaTLhJVSFb9QUO9eEdep$ [tools[.]ietf[.]org]) is that it lists a set of guidance rules at the time of writing.

	https://urldefense.com/v3/__https://tools.ietf.org/html/rfc2026*page-15__;Iw!!LpKI!1k2aHtuNmS8NqaE6aw6FbJXpRSyyQBXX9-m0yAnCxJRUpaTLhJVSFb9QUFSPRMK6$ [tools[.]ietf[.]org]

 - Jonathan Morton

---

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 5 May 2021 20:07:43 +0000

Hi Jonathan
Please see below, marked [IJ2]

/Ingemar

> Sent: den 5 maj 2021 17:50
> > On 5 May, 2021, at 6:05 pm, Ingemar Johansson S
> > The latter lists ECT(1) for experimental use only, with a reference to
> > RFC8311 My interpretation is that for future deployment of AQMs
> > (fq-codel, cake) only ECT(0) should be used for classic ECN. This
> > means that this part of the problem goes away gradually.
> 
> No, because there are two aspects to use of ECT(1) at play.  You have
found
> only one of them.
> 
> RFC-3168 states that ECT(0) and ECT(1) shall be treated identically by
> middleboxes.  It is of course possible for a later specification to update
that,
> but unless and until RFC-3168 is obsoleted, middleboxes that treat ECT(1)
> identically to ECT(0) will continue to exist and be newly deployed on the
> Internet.  This aspect of RFC-3168 was included *specifically* to provide
a
> consistent baseline behaviour upon which experimental uses of ECT(1) could
> rely, so it seems unlikely to be discarded without serious justification.
> 
> One such experimental use was ECN Nonce.  This emitted an unpredictable
> pattern of ECT(0) and ECT(1) from the sender as a way to verify whether
> correct RFC-3168 semantics existed on the network path and in the
receiver.
> It was never deployed, but it would have relied on that aspect of
RFC-3168.
> 
> Another such use is by SCE, which also relies on existing ECN-aware
> middleboxes implementing this aspect of RFC-3168.
> 
> If L4S cannot operate safely in an environment where existing middleboxes
> correctly implement well-established RFCs, then L4S has a problem, not the
> RFC in question.

[IJ2] This does not make sense at all.
RFC8311 updates RFC3168 and RFC3168 is updated by RFC8311 among other... In
addition, the IANA registry lists ECT(1) as for experimental use.. RFC8311
states, "unless otherwise specified by an experimental RFC...". Well that
RFC is https://tools.ietf.org/wg/tsvwg/draft-ietf-tsvwg-ecn-l4s-id/ which
has been waiting for WGLC for some time now. 
Besides this, ECN Nonce is historic and SCE is not of interest in this
context, as we discuss L4S and not SCE

> >> and Cake be changed; you must actually show that the change has
> >> already taken place.  As of right now, it has not.
> >
> > [IJ] Point taken. But then again, all network gear have a limited
> > lifetime (mentioned in my email), either economic (increased Opex,
> > energy cost/bit) or plain physical (things break down, especially home
> gateways do).
> 
> Right now, that process is *increasing* the number of RFC-3168 compliant
> middleboxes on the Internet, not reducing them.  The reverse will only
> *begin* to occur if you can convince the IETF to obsolete RFC-3168.  I do
not
> see any chance of that happening.
> 
>  - Jonathan Morton

---

From: Jonathan Morton <[email-masked]>
Date: Wed, 5 May 2021 23:33:39 +0300

> On 5 May, 2021, at 11:07 pm, Ingemar Johansson S <[email-masked]> wrote:
> 
> states, "unless otherwise specified by an experimental RFC...".

As I have explained before, RFC-8311 does not *obsolete* RFC-3168 - that is a different and more drastic step than updating it.  This means that middlebox implementors can continue to refer to RFC-3168 as the only necessary authority on what an ECN AQM needs to do.  RFC-3168 specifies that as far as a middlebox is concerned, ECT(0) and ECT(1) are synonymous.

A new RFC, satisfying the requirements in RFC-8311, could further update RFC-3168 by specifying alternate behaviour in response to ECT(1), or permitting middleboxes to change ECT(0) to ECT(1) (or vice versa) under certain conditions.  The "new AQMs" proposed as part of the L4S and SCE systems would rely on such allowances.  But implementors would still have a choice as to whether to implement RFC-3168 or the new RFC, and there would be no requirement to change existing middleboxes that already conform to RFC-3168.  We should therefore expect RFC-3168 middleboxes to continue to be a feature of the Internet.

Requiring all existing ECN AQMs, which at present rely on RFC-3168, to be changed to treat ECT(1) differently, would require obsoleting RFC-3168 so that it is no longer an active Internet specification.  There is a process to do so, but I do not think that process would succeed if initiated.

 - Jonathan Morton

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 6 May 2021 07:27:26 +0100

> In the past, I have seen the IESG apply more rigorous review to drafts intended for Best Current Practice status than to drafts intended for Proposed Standard status, but that can vary from year to…

+1 on the status of BCPs.The key question I have is whether this BCP is 
relevent to this proposed use.

* The reclassification of RFC8311 in 2018 - as a PS - has enabled new 
experimental use of the ECT(1) codepoint by an RFC. This was not an 
option open to the IETF when RFC 4774 was published. RFC8311explicitly 
provides ABE (published as an EXP RFC in 2018) and L4S as an example.

* I would expectevidence/rationale in the L4S draft to explain why RFC 
4774 (a BCP) is not relevant. This might clarify whether L4S is an 
update to RFC 3168 (defining both transport and network aspects), or an 
instance of RFC 4774 (which I read as targeting multi-level ECN marking 
and use in an edge-to-edge context – e.g. as  realised in PCN, RFC 5696).

Gorry

---

From: Ingemar Johansson S <[email-masked]>
Date: Thu, 6 May 2021 06:35:39 +0000

Jonathan.

This part of the discussion was around new deployments of AQMs. See
https://mailarchive.ietf.org/arch/msg/tsvwg/Ge7cbzFFQyFzpHEMGGnG6AqnjnY/ .
This appears to get lost in the iterations. In other words, it does not deal
with already existing AQMs out there, that was a separate topic where
RFC4774 comes into the discussion.

Please see more inline [IJ3]

/Ingemar

> Sent: den 5 maj 2021 22:34
> > On 5 May, 2021, at 11:07 pm, Ingemar Johansson S
> > RFC8311 updates RFC3168 and RFC3168 is updated by RFC8311 among
> > other... In addition, the IANA registry lists ECT(1) as for
> > experimental use.. RFC8311 states, "unless otherwise specified by an
> experimental RFC...".
> 
> As I have explained before, RFC-8311 does not *obsolete* RFC-3168 - that
is
> a different and more drastic step than updating it.  This means that
> middlebox implementors can continue to refer to RFC-3168 as the only
> necessary authority on what an ECN AQM needs to do.  RFC-3168 specifies
> that as far as a middlebox is concerned, ECT(0) and ECT(1) are synonymous.

[IJ3] But I don't see why it is necessary, sure it could be a possibility in
the long term, but for L4S it is not really necessary. It is sufficient with
the wordings in RFC8311.

> 
> A new RFC, satisfying the requirements in RFC-8311, could further update
> RFC-3168 by specifying alternate behaviour in response to ECT(1), or
> permitting middleboxes to change ECT(0) to ECT(1) (or vice versa) under
> certain conditions.  The "new AQMs" proposed as part of the L4S and SCE
> systems would rely on such allowances.  But implementors would still have
a
> choice as to whether to implement RFC-3168 or the new RFC, and there
> would be no requirement to change existing middleboxes that already
> conform to RFC-3168.  We should therefore expect RFC-3168 middleboxes to
> continue to be a feature of the Internet.

[IJ3] That RFC would then be L4S-ID (recall RFC8311 ""unless otherwise
specified by an experimental RFC..." , then the only sensible way forwards
is to move it L4S-ID to WGLC. That should give sufficient guidance for
implementors of new network gear as RFC3168 clearly states, updated by
RFC8311. 
And note that SCE is not subject to discussion. This is only about L4S .

> 
> Requiring all existing ECN AQMs, which at present rely on RFC-3168, to be
> changed to treat ECT(1) differently, would require obsoleting RFC-3168 so
> that it is no longer an active Internet specification.  There is a process
to do
> so, but I do not think that process would succeed if initiated.

[IJ3] As mentioned above, already existing AQMs were not part of this
particular discussion, that was a separate part where RFC4774 came into
play.