Subject: [tsvwg] L4S: Guard DSCP

From: "Black, David" <[email-masked]>
Date: Thu, 22 Apr 2021 23:40:52 +0000

[posted as an individual, not as a WG chair]

Some time ago, I wrote:

> in the hope of fostering productive discussion, I'll try to put together a summary (as an individual) of what a Guard DSCP

> could look like whose sole purpose is to better equip operators to deal with things that might go wrong in an L4S

> experiment (without redesigning the experiment).  Patience, please ...

Here's that attempt, but first some process discussion ...

The L4S ops draft currently contains material for operators of L4S hosts (Section 4) and for operators of networks participating in the L4S experiment who experience problems at RFC 3168 FIFO bottlenecks in their own networks).  A Guard DSCP for L4S complements material in the L4S ops draft by providing means for an operator actively participating in the L4S experiment to deal with reports that L4S traffic leaving the operator's network is causing problems downstream in a network that may not be participating in the L4S experiment (including an all-but-unmanaged residential WiFi access point).  I'd suggest that initial discussion focus on what a Guard DSCP for L4S is and how it works with a goal of getting detailed text on that into the L4S ops draft, after which could come the inevitable heated discussion on requirement level (MUST/SHOULD/MAY) for a Guard DSCP.

... and so here is that summary for discussion, phrased as goals/guidelines for specifying the Guard DSCP:

DO:

  1.  Mark all L4S traffic with a Guard DSCP at endpoints and/or network edges.
  2.  Participating operators are able to identify and separately handle Guard DSCP traffic at network boundaries, dropping it as a last resort.  Strongly discourage traffic filtering to drop ECT(1) traffic.
  3.  Plan to remove Guard DSCP requirement if L4S experiment succeeds.

DON'T

  1.  Require AQMs to use Guard DSCP in addition to ECT(1) to identify L4S traffic.   This should be allowed, not required, which is already the case - see Section 5.4 of the ecn-l4s-id draft.
  2.  Rely on DSCPs for signaling or modify transport protocols to have reactions that vary by received DSCP.
  3.  Do anything else that would be a significant obstacle to removing use of the Guard DSCP if the L4S experiment succeeds.

For further discussion: Operator/technology-specific alternate means of accomplishing the functional equivalent of A & B
E.g., I would think that 5G networks have other means at their disposal to handle traffic aggregates at network boundaries in a fashion that could make 5G use of a Guard DSCP pointless.

This is what I had in mind for a Guard DSCP - I'll do my best to promptly answer questions, respond to comments etc.  Please try to keep the discussion calm and technical.

[reminder: posted as an individual, not as a WG chair]

Thanks, --David

David L. Black, Sr. Distinguished Engineer, Technology & Standards
Infrastructure Solutions Group, Dell Technologies
mobile +1 978-394-7754 [email-masked]<mailto:[email-masked]>

---

From: Jonathan Morton <[email-masked]>
Date: Fri, 23 Apr 2021 23:45:10 +0300

> On 23 Apr, 2021, at 2:40 am, Black, David <[email-masked]> wrote:
> 
> A Guard DSCP for L4S complements material in the L4S ops draft by providing means for an operator actively participating in the L4S experiment to deal with reports that L4S traffic leaving the operator's network is causing problems downstream in a network that may not be participating in the L4S experiment (including an all-but-unmanaged residential WiFi access point).

Unfortunately, I don't think this is sufficient, even as a temporary measure, to enable deploying L4S as an Internet Experiment.  This Guard DSCP would be a good fit for a scheme conforming to RFC-4774 Option 2 or 3, but not Option 1, because you specifically exclude the measure of mandating traffic filtering at the network border - only ensuring that such filtering is *possible*.

Without some means of reliably determining whether a path over which an L4S connection is being negotiated is within an L4S participating network, L4S is an Option 1 scheme - one that has to be contained to a participating network.  The current specification suggests the use of a detection heuristic, but does not mandate it; this is reinforced by the reference implementation of L4S containing a detection heuristic that has been shown to be unreliable under lab conditions (never mind the real world), and is also disabled by default.

This raises the spectre of the following scenario:

1: The L4S experiment goes ahead with a Guard DSCP implemented strictly to the minimum standard you outlined, and conforming to the minimum letter of the L4S specification.

2: L4S traffic then spills over into non-participating networks (there are several ways this could happen) and causes a very bad interaction with conventional traffic at an RFC-compliant AQM.

3: The users involved in that affected traffic and the operator of the AQM, being "innocent bystanders", have no idea why this bad interaction is occurring, only that some particular traffic appears to be unresponsive to normal congestion signals.  The more observant ones might notice that ECT(1) is present rather than ECT(0), but it would still take concerted effort to work out who to complain to based on such a meagre clue.

4: The L4S experiment fails to receive information about the bad interactions thus caused, does not implement mitigations at the network border, and eventually considers the experiment a success.  But it is *not* a success.

The two proposals I previously put forward would move L4S from Option 1 status to Option 2 or Option 3.  I contend that this class of modification is necessary to permit L4S deployment without a robust *and permanent* containment mechanism.  At the very least, the development, demonstration, and mandatory use of a reasonably robust detection heuristic is necessary to move L4S into Option 2 status.

 - Jonathan Morton

---

From: Pete Heist <[email-masked]>
Date: Sat, 24 Apr 2021 10:21:54 +0200

One question/comment below...

On Thu, 2021-04-22 at 23:40 +0000, Black, David wrote:
> > in the hope of fostering productive discussion, I'll try to put
> together a summary (as an individual) of what a Guard DSCP
> > could look like whose sole purpose is to better equip operators to
> deal with things that might go wrong in an L4S
> Here's that attempt, but first some process discussion …
>  
> The L4S ops draft currently contains material for operators of L4S
> hosts (Section 4) and for operators of networks participating in the
> L4S experiment who experience problems at RFC 3168 FIFO bottlenecks
> in their own networks).  A Guard DSCP for L4S complements material in
> the L4S ops draft by providing means for an operator actively
> participating in the L4S experiment to deal with reports that L4S
> traffic leaving the operator's network is causing problems downstream
> in a network that may not be participating in the L4S experiment
> (including an all-but-unmanaged residential WiFi access point).  I'd
> suggest that initial discussion focus on what a Guard DSCP for L4S is
> and how it works with a goal of getting detailed text on that into
> the L4S ops draft, after which could come the inevitable heated
> discussion on requirement level (MUST/SHOULD/MAY) for a Guard DSCP.
>  
> … and so here is that summary for discussion, phrased as
> goals/guidelines for specifying the Guard DSCP:
>  
> DO:
> A. Mark all L4S traffic with a Guard DSCP at endpoints and/or
> network edges.
> B. Participating operators are able to identify and separately
> handle Guard DSCP traffic at network boundaries, dropping it as a
> last resort.  Strongly discourage traffic filtering to drop
> ECT(1) traffic.
> C. Plan to remove Guard DSCP requirement if L4S experiment succeeds.

Regarding B, other than dropping Guard DSCP marked traffic at network
boundaries, are there other options to prevent L4S and non-L4S traffic
from meeting at an RFC compliant AQM outside the experimental network?
Bleaching ECT(1) on traffic with the Guard DSCP might be one...

Pete

> DON'T
> A. Require AQMs to use Guard DSCP in addition to ECT(1) to identify
> L4S traffic.   This should be allowed, not required, which is
> already the case - see Section 5.4 of the ecn-l4s-id draft.
> B. Rely on DSCPs for signaling or modify transport protocols to have
> reactions that vary by received DSCP.
> C. Do anything else that would be a significant obstacle to removing
> use of the Guard DSCP if the L4S experiment succeeds.
>  
> For further discussion: Operator/technology-specific alternate means
> of accomplishing the functional equivalent of A & B
> E.g., I would think that 5G networks have other means at their
> disposal to handle traffic aggregates at network boundaries in a
> fashion that could make 5G use of a Guard DSCP pointless.
>  
> This is what I had in mind for a Guard DSCP – I'll do my best to
> promptly answer questions, respond to comments etc.  Please try to
> keep the discussion calm and technical.
>  
> David L. Black,Sr. Distinguished Engineer, Technology & Standards
> mobile +1 [email-masked]
>

---

From: "C. M. Heard" <[email-masked]>
Date: Sat, 24 Apr 2021 19:38:05 -0700

On Sat, Apr 24, 2021 at 1:22 AM Pete Heist wrote:

> … and so here is that summary for discussion, phrased as goals/guidelines
> for specifying the Guard DSCP:
>
> [ ... ]

>    2. Participating operators are able to identify and separately handle
>    Guard DSCP traffic at network boundaries, dropping it as a last resort.
>    Strongly discourage traffic filtering to drop ECT(1) traffic.
> boundaries, are there other options to prevent L4S and non-L4S traffic from
> meeting at an RFC compliant AQM outside the experimental network? Bleaching
> ECT(1) on traffic with the Guard DSCP might be one...
>

When you say "Bleaching ECT(1)" I presume that you mean "Changing ECT(1) to
ECT(0)". If that's not right, please correct me.

Mike Heard

---

From: Pete Heist <[email-masked]>
Date: Sun, 25 Apr 2021 09:29:09 +0200

Hi Mike, response below...

On Sat, 2021-04-24 at 19:38 -0700, C. M. Heard wrote:
> > Regarding B, other than dropping Guard DSCP marked traffic at
> > network boundaries, are there other options to prevent L4S and non-
> > L4S traffic from meeting at an RFC compliant AQM outside the
> > experimental network? Bleaching ECT(1) on traffic with the Guard
> > DSCP might be one...
> > 
> 
> When you say "Bleaching ECT(1)" I presume that you mean "Changing
> ECT(1) to ECT(0)". If that's not right, please correct me.

Actually, here I mean re-marking ECT(1) to Not-ECT. It seems to me that
re-marking ECT(1) to ECT(0) would allow unsafe interaction to occur
between L4S and non-L4S traffic in RFC3168 queues outside the
experimental network. Truthfully, I think that dropping the Guard DSCP
marked traffic would be the safest option.

Pete

---

From: Sebastian Moeller <[email-masked]>
Date: Sun, 25 Apr 2021 11:07:41 +0200

Hi Pete, hi Mike,

I think re-marking ECT(1) to ECT(0) is infeasible, since it is no way guaranteed that the sender will be notified of that change, e.g. for TCP Prague AccECN when not using the TCP option does not seem to transfer information only about the CE count. And since it is not the L4S AQM-behaviour that causes the rfc3168 incompatibility but the end-points mis-interpreting the message a CE-mark contains, re-marking to ECT(0) is not an option if the goal is to make L4S safer. 
	NotECT should work as downstream non-L4S bottlenecks will simply drop packets to signal congestion, to which L4S-compatible protocols are required to respond (but so are general internet wide used protocols, and still BBR does not do so immediately).
	All in all I agree with Pete, dropping is the best option as it will also catch errors in L4S protocols, which seems prudent as no L4S protocol has reached maturity yet, so all should be expected to not be production ready and hence should be treated with caution. I still wonder how wise it is to inflict an experiment onto the existing internet with so much components still untested/in their infancy (and after years in the making, I am still puzzled why this lack of suitable protocols is not held against L4S as a token of lack of sincerity).

Best Regards
	Sebastian

> On Apr 25, 2021, at 09:29, Pete Heist <[email-masked]> wrote:
> 
>>>> … and so here is that summary for discussion, phrased as goals/guidelines for specifying the Guard DSCP:
>>>> 
>>> 
>> [ ... ]  
>>>> DO:
>>>> 
>>>> 	• Mark all L4S traffic with a Guard DSCP at endpoints and/or network edges.
>>>> 	• Participating operators are able to identify and separately handle Guard DSCP traffic at network boundaries, dropping it as a last resort.  Strongly discourage traffic filtering to drop ECT(1) traffic.
>>>> 	• Plan to remove Guard DSCP requirement if L4S experiment succeeds.
>>>> 
>>> 
>>> Regarding B, other than dropping Guard DSCP marked traffic at network boundaries, are there other options to prevent L4S and non-L4S traffic from meeting at an RFC compliant AQM outside the experimental network? Bleaching ECT(1) on traffic with the Guard DSCP might be one...
>>> 
>> 
>> When you say "Bleaching ECT(1)" I presume that you mean "Changing ECT(1) to ECT(0)". If that's not right, please correct me.
> 
> Actually, here I mean re-marking ECT(1) to Not-ECT. It seems to me that re-marking ECT(1) to ECT(0) would allow unsafe interaction to occur between L4S and non-L4S traffic in RFC3168 queues outside the experimental network. Truthfully, I think that dropping the Guard DSCP marked traffic would be the safest option.
> 
> Pete
> 
>> Mike Heard
>

---

From: "C. M. Heard" <[email-masked]>
Date: Sun, 25 Apr 2021 15:46:11 -0700

On Sun, Apr 25, 2021 at 12:29 AM Pete Heist wrote:

> When you say "Bleaching ECT(1)" I presume that you mean "Changing ECT(1)
> to ECT(0)". If that's not right, please correct me.
>
>
> re-marking ECT(1) to ECT(0) would allow unsafe interaction to occur between
> L4S and non-L4S traffic in RFC3168 queues outside the experimental network.
> Truthfully, I think that dropping the Guard DSCP marked traffic would be
> the safest option.
>

... and ...

On Sun, Apr 25, 2021 at 2:07 AM Sebastian Moeller wrote:

> I think re-marking ECT(1) to ECT(0) is infeasible, since it is no way
> guaranteed that the sender will be notified of that change, e.g. for TCP
> Prague AccECN when not using the TCP option does not seem to transfer
> information only about the CE count. And since it is not the L4S
> AQM-behaviour that causes the rfc3168 incompatibility but the end-points
> mis-interpreting the message a CE-mark contains, re-marking to ECT(0) is
> not an option if the goal is to make L4S safer.

Actually, I would like to respectfully disagree with that for the case of
TCP, at least in part.

As mentioned above (and in draft-ietf-tsvwg-ecn-l4s-id-14 Section4), AccECN
is a required part of L4S when the transport is TCP. Note that AccECN
provides a means to echo back to the sender of a SYN whether the SYN
contains not-ECT, ECT(0), ECT(1), or CE, if AccECN is negotiated (see
draft-ietf-tcpm-accurate-ecn-14 Section-3.1.2 and in particular the first
for lines of Table 2). Note further that draft-ietf-tsvwg-ecn-l4s-id-14
Appendix A.2.1 explicitly urges that L4S implementations set ECT(1) on SYN
segments. Based on this, it should be possible for an L4S connection
initiator to determine whether ECT(1) was bleached before its SYN was sent
onward to a remote endpoint that supports AccECN. That is enough to tell it
to fall back to RFC 3168 compatibility mode for that connection,

There is a corner case in the above: what if the initial SYN has ECT(1)
changed to CE? In this case the guardian at the egress of the L4S domain
must, upon finding the guard DSCP, drop the packet.

So, it seems to me that the egress rule for handling packets with the guard
DSCP set, when handing the packet to a domain not known to be participating
in the L4S experiment, would be to bleach the guard DSCP and handle the ECN
field as follows:

IN            OUT
==           ===
not-ECT not-ECT
ECT(0)   ECT(0)
ECT(1)   ECT(0)
CE          drop

This is subject to some constraints on L4S endpoint behavior, in particular
setting ECT(1) on its SYN segments, and falling back to RFC 3168 mode if
ECT(1) is not echoed in the SYN-ACK.

What about simultaneous open? That works too if the ingress rule is the
same as above with respect to the ECN code point and the L4S endpoint falls
back to RFC 3168 mode if it does not see ECT(1) in the "colliding" SYN.

Note that the above requires inspection of the DSCP and action upon the ECN
codepoint only by the egress/ingress "guardian" and not by AQMs or
transport endpoints and so meets David Black's other requirements (or
desiderata):

DON'T

   1. Require AQMs to use Guard DSCP in addition to ECT(1) to identify L4S
   traffic.   This should be allowed, not required, which is already the case
   - see Section 5.4 of the ecn-l4s-id draft.
   2. Rely on DSCPs for signaling or modify transport protocols to have
   reactions that vary by received DSCP.
   3. Do anything else that would be a significant obstacle to removing use
   of the Guard DSCP if the L4S experiment succeeds.

Ok, if this is full of holes, kindly point them out and/or fix them.

Mike Heard

---

From: Jonathan Morton <[email-masked]>
Date: Mon, 26 Apr 2021 02:43:57 +0300

> On 26 Apr, 2021, at 1:46 am, C. M. Heard <[email-masked]> wrote:
> 
> As mentioned above (and in draft-ietf-tsvwg-ecn-l4s-id-14 Section4), AccECN is a required part of L4S when the transport is TCP. Note that AccECN provides a means to echo back to the sender of a SYN whether the SYN contains not-ECT, ECT(0), ECT(1), or CE, if AccECN is negotiated (see draft-ietf-tcpm-accurate-ecn-14 Section-3.1.2 and in particular the first for lines of Table 2). Note further that draft-ietf-tsvwg-ecn-l4s-id-14 Appendix A.2.1 explicitly urges that L4S implementations set ECT(1) on SYN segments. Based on this, it should be possible for an L4S connection initiator to determine whether ECT(1) was bleached before its SYN was sent onward to a remote endpoint that supports AccECN. That is enough to tell it to fall back to RFC 3168 compatibility mode for that connection,
> 
> There is a corner case in the above: what if the initial SYN has ECT(1) changed to CE? In this case the guardian at the egress of the L4S domain must, upon finding the guard DSCP, drop the packet.

There are two big holes in this idea:

1: Some middleboxes discard unknown TCP options.  A path on which this occurs prevents AccECN from giving feedback about anything beyond the count of CE marks received, using the ACE field.  If passage of the option were required for AccECN to work, the ACE field probably wouldn't be in the spec.

2: What happens if the CE mark is applied to the SYN *after* it has passed the border guardian?  The receiver cannot distinguish which ECT codepoint was originally on it in that case, so the information added by the guardian is lost.

 - Jonathan Morton

---

From: "C. M. Heard" <[email-masked]>
Date: Sun, 25 Apr 2021 20:54:32 -0700

On Sun, Apr 25, 2021 at 4:44 PM Jonathan Morton wrote:

> On 26 Apr, 2021, at 1:46 am, C. M. Heard wrote:
>
>>
>> As mentioned above (and in draft-ietf-tsvwg-ecn-l4s-id-14 Section4),
>> AccECN is a required part of L4S when the transport is TCP. Note that
>> AccECN provides a means to echo back to the sender of a SYN whether the SYN
> 1: Some middleboxes discard unknown TCP options.  A path on which this
> occurs prevents AccECN from giving feedback about anything beyond the count
> of CE marks received, using the ACE field.  If passage of the option were
> required for AccECN to work, the ACE field probably wouldn't be in the spec.
>

A middlebox stripping the option is not really a problem, since AccECN
negotiation relies entirely on the ACE field. Indeed, the initial SYN does
not contain the option at all -- see draft-ietf-tcpm-accurate-ecn-14
Section 2.1.

2: What happens if the CE mark is applied to the SYN *after* it has passed
> the border guardian?  The receiver cannot distinguish which ECT codepoint
> was originally on it in that case, so the information added by the guardian
> is lost.
>

I did indeed fail to consider this case. It can be addressed, though there
is an issue with the solution. The originating TCP will get an ACE field of
110 in the SYN-ACK, meaning AccECN / CE on SYN, which is indeed ambiguous. The
safe thing for the originating TCP to do would be either to fall back to
RFC 3168 compatibility mode unconditionally or to look at the guard DSCP to
decide whether or not to do so. Looking at the guard DSCP allows the
originating TCP to determine whether or not it is communicating with
another L4S TCP in the domain and avoid the unnecessary fallback behavior
in that case. Similar remarks apply for the case of a simultaneous open.

The issue with this solution is that inspection of the DSCP by the
transport layer to avoid suboptimal behavior within the L4S domain (i.e.,
fallback to classic when an ECT(1) SYN is marked with CE) violates the
David's desideratum Y:

DON'T
>
>    1. Require AQMs to use Guard DSCP in addition to ECT(1) to identify
>    L4S traffic.   This should be allowed, not required, which is already the
>    case - see Section 5.4 of the ecn-l4s-id draft.
>    3. Do anything else that would be a significant obstacle to removing
> I might also add that the business of falling back to RFC 3168 behavior
based on what is in a SYN-ACK (or SYN) is not needed not needed to
communicate with an endpoint in the L4S that supports AccECN but not L4S;
indeed, in that situation precludes the possibility of L4S congestion
control in one direction and classic RFC 3168 congestion control in the
other direction. It's not clear how important that case is, though.

Mike Heard

---

From: "Black, David" <[email-masked]>
Date: Mon, 26 Apr 2021 20:46:07 +0000

[Still posting as an individual]

If ECT(1) is to be remarked, I agree with Sebastian's earlier comment that the robust remarking would be to Not-ECT:

> Not-ECT should work as downstream non-L4S bottlenecks will simply drop packets to signal congestion, to which L4S-compatible protocols

> are required to respond (but so are general internet wide used protocols, and still BBR does not do so immediately).

However, this entire discussion is disheartening, as the end-to-end usability of the ECN field, and in particular the ECT(1) value was a major L4S rationale for using ECT(1) by itself to indicate the low-latency service.  It would be rather unfortunate (IMHO, as an individual) if the L4S experiment were to result in widespread bleaching of ECT(1) to Not-ECT.

Thanks, --David

From: C. M. Heard <[email-masked]>
Sent: Sunday, April 25, 2021 11:55 PM
To: Jonathan Morton
Cc: Pete Heist; Sebastian Moeller; Black, David; TSVWG
Subject: Re: [tsvwg] L4S: Guard DSCP

[EXTERNAL EMAIL]

A middlebox stripping the option is not really a problem, since AccECN negotiation relies entirely on the ACE field. Indeed, the initial SYN does not contain the option at all -- see draft-ietf-tcpm-accurate-ecn-14 Section 2.1.

I did indeed fail to consider this case. It can be addressed, though there is an issue with the solution. The originating TCP will get an ACE field of 110 in the SYN-ACK, meaning AccECN / CE on SYN, which is indeed ambiguous. The safe thing for the originating TCP to do would be either to fall back to RFC 3168 compatibility mode unconditionally or to look at the guard DSCP to decide whether or not to do so. Looking at the guard DSCP allows the originating TCP to determine whether or not it is communicating with another L4S TCP in the domain and avoid the unnecessary fallback behavior in that case. Similar remarks apply for the case of a simultaneous open.

The issue with this solution is that inspection of the DSCP by the transport layer to avoid suboptimal behavior within the L4S domain (i.e., fallback to classic when an ECT(1) SYN is marked with CE) violates the David's desideratum Y:

DON'T

X.      Require AQMs to use Guard DSCP in addition to ECT(1) to identify L4S traffic.   This should be allowed, not required, which is already the case - see Section 5.4 of the ecn-l4s-id draft.

Y.      Rely on DSCPs for signaling or modify transport protocols to have reactions that vary by received DSCP.

Z.      Do anything else that would be a significant obstacle to removing use of the Guard DSCP if the L4S experiment succeeds.
I might also add that the business of falling back to RFC 3168 behavior based on what is in a SYN-ACK (or SYN) is not needed not needed to communicate with an endpoint in the L4S that supports AccECN but not L4S; indeed, in that situation precludes the possibility of L4S congestion control in one direction and classic RFC 3168 congestion control in the other direction. It's not clear how important that case is, though.

Mike Heard

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 26 Apr 2021 20:59:59 -0700

On Mon, Apr 26, 2021 at 1:46 PM Black, David <[email-masked]> wrote:

> However, this entire discussion is disheartening, as the end-to-end
> usability of the ECN field, and in particular the ECT(1) value was a major
> L4S rationale for using ECT(1) by itself to indicate the low-latency
> service.
>

Fair enough; how about this, for the specific case of TCP:

Within a network actively participating in the L4S experiment, a TCP with
L4S capability will mark all L4S traffic that it originates with the Guard
DSCP. If it receives a SYN or SYN-ACK without the Guard DSCP, it will fall
back to RFC 3168 mode (and will not mark traffic sent while in fallback
mode). At the participating network edge, the Guard DSCP will be bleached
from all traffic leaving that network and entering a network that is not
known to be participating in the experiment AND on all traffic entering
from such a network. There will be no dropping of traffic and no bleaching
of the ECN field based on the Guard DSCP.

Under this proposal, the Guard DSCP is being used as a means for a node
that is actively participating in the L4S experiment to prevent L4S traffic
from escaping into a non-participating network. That is a rather more
ambitious goal than merely using it as a "means for an operator actively
participating in the L4S experiment to deal with reports that L4S traffic
leaving the operator's network is causing problems downstream in a network
that may not be participating in the L4S experiment," but I honestly think
that's what we need.

I recall from the lengthy discussion that went on about one year ago (see,
e.g.,
https://mailarchive.ietf.org/arch/msg/tsvwg/ij5BqzNtMxKV8VbHE8pUevZjx7k/)
that one of the big objections to the use of a Guard DSCP as a means to
contain the experiment is that an L4S experimental domain does not map well
to a DSCP domain, and so the Guard DSCP may be inadvertently bleached (e.g.
by tunnels) inside the participating network. If correct, that objection is
hard to address.

Mike Heard

---

From: <[email-masked]>
Date: Tue, 27 Apr 2021 05:36:33 +0000

Hi David,

maybe I missed an important point, but isn’t the intent of the Guard DSCP to inform the receiver about the meaning of a set ECN bit? My take is, if Domain decides to go for an experiment, Domain should be conservative with what they send to peers and configure suitable re-marking policies to set default DSCP/ECN 00 at interconnection, unless they know their peer is happy to participate in the L4S experiment too. What I also expect is that a few domains are liberal with what they send and forward traffic with a bunch of DSCPs, without setting up any kind of formal or informal agreement with their peers.

If the Guard DSCP is in range 000 xxx, it will pass some networks at least. Maybe that’s a way forward.

Regards,

Ruediger

Von: tsvwg <[email-masked]> Im Auftrag von Black, David
Gesendet: Montag, 26. April 2021 22:46
An: C. M. Heard <[email-masked]>; Jonathan Morton <[email-masked]>
Cc: TSVWG <[email-masked]>
Betreff: Re: [tsvwg] L4S: Guard DSCP

Thanks, --David

From: C. M. Heard <[email-masked]<mailto:[email-masked]>>
Sent: Sunday, April 25, 2021 11:55 PM
To: Jonathan Morton
Cc: Pete Heist; Sebastian Moeller; Black, David; TSVWG
Subject: Re: [tsvwg] L4S: Guard DSCP

[EXTERNAL EMAIL]

DON'T

Mike Heard

---

From: "Black, David" <[email-masked]>
Date: Tue, 27 Apr 2021 22:57:27 +0000

Mike,
[still posting as an individual]

I'm not enthusiastic about a transport protocol reacting to a received DSCP value, but this proposal seems less intrusive/disruptive than some of the other approaches that have been floated:

> Within a network actively participating in the L4S experiment, a TCP with L4S capability will mark all L4S traffic that it originates with the Guard DSCP.
> If it receives a SYN or SYN-ACK without the Guard DSCP, it will fall back to RFC 3168 mode (and will not mark traffic sent while in fallback mode).
> At the participating network edge, the Guard DSCP will be bleached from all traffic leaving that network and entering a network that is not known
> to be participating in the experiment AND on all traffic entering from such a network. There will be no dropping of traffic and no bleaching of the
> ECN field based on the Guard DSCP.

Moving on:

> Under this proposal, the Guard DSCP is being used as a means for a node that is actively participating in the L4S experiment to prevent L4S traffic from escaping into a non-participating network.

Understood – the appropriate L4S experiment scope may be sets of cooperating/participating networks rather than the (originally envisioned) Internet as a whole.

Thanks, --David [as an individual]

From: C. M. Heard <[email-masked]>
Sent: Tuesday, April 27, 2021 12:00 AM
To: Black, David
Cc: Jonathan Morton; Pete Heist; Sebastian Moeller; TSVWG
Subject: Re: [tsvwg] L4S: Guard DSCP

[EXTERNAL EMAIL]
On Mon, Apr 26, 2021 at 1:46 PM Black, David <[email-masked]<mailto:[email-masked]>> wrote:
However, this entire discussion is disheartening, as the end-to-end usability of the ECN field, and in particular the ECT(1) value was a major L4S rationale for using ECT(1) by itself to indicate the low-latency service.

Within a network actively participating in the L4S experiment, a TCP with L4S capability will mark all L4S traffic that it originates with the Guard DSCP. If it receives a SYN or SYN-ACK without the Guard DSCP, it will fall back to RFC 3168 mode (and will not mark traffic sent while in fallback mode). At the participating network edge, the Guard DSCP will be bleached from all traffic leaving that network and entering a network that is not known to be participating in the experiment AND on all traffic entering from such a network. There will be no dropping of traffic and no bleaching of the ECN field based on the Guard DSCP.

Under this proposal, the Guard DSCP is being used as a means for a node that is actively participating in the L4S experiment to prevent L4S traffic from escaping into a non-participating network. That is a rather more ambitious goal than merely using it as a "means for an operator actively participating in the L4S experiment to deal with reports that L4S traffic leaving the operator's network is causing problems downstream in a network that may not be participating in the L4S experiment," but I honestly think that's what we need.

I recall from the lengthy discussion that went on about one year ago (see, e.g., https://mailarchive.ietf.org/arch/msg/tsvwg/ij5BqzNtMxKV8VbHE8pUevZjx7k/ [mailarchive.ietf.org]<https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/tsvwg/ij5BqzNtMxKV8VbHE8pUevZjx7k/__;!!LpKI!3O_yc-c_keXnDzQr5kIsAGHjsdLwrDdwRmywXodJmxuAEvzsG8GA0pNttYbDFl5f$>) that one of the big objections to the use of a Guard DSCP as a means to contain the experiment is that an L4S experimental domain does not map well to a DSCP domain, and so the Guard DSCP may be inadvertently bleached (e.g. by tunnels) inside the participating network. If correct, that objection is hard to address.

Mike Heard

---

From: Greg White <[email-masked]>
Date: Thu, 29 Apr 2021 17:46:40 +0000

Mike, David,

In my opinion this proposal (if mandated) would significantly diminish the L4S experiment.  There is widespread evidence of DSCP bleaching in the Internet, while there is little evidence of RFC3168 bottleneck deployment.  The lack of Guard DSCP at a recipient would almost certainly be a false positive (strong evidence of bleaching, weak evidence of RFC3168 deployment), and the presence of Guard DSCP would not be a guarantee that 3168 doesn’t exist on the path. On top of that, the need to negotiate DSCP passthrough (or coordinated, bi-directional, DSCP remapping) to enable this across boundaries between networks that *are* interested in participating (actively or passively) in L4S deployment creates a huge hurdle that would make it difficult to achieve L4S TCP deployment outside of closed systems.  Furthermore, it seems to rely on having appropriate implementation at several entities in order to work (both endpoints and a network operator).  The “bleach ECT1 to Not-ECT at a network boundary if you are aware of 3168 in the other network” option (which has its own downsides) at least only requires one entity to take action.

I would be wary of any proposal that significantly weakens the likelihood of a successful L4S experiment, particularly if the only benefit is to prevent extremely rare (and fixable) collateral damage.  Presupposing that RFC3168 bottlenecks are widely deployed (contrary to evidence) and disabling L4S on the majority of paths doesn’t seem to me to be a recipe for success.

However, I do see this mechanism possibly being a tool in the toolbox for situations where a sender is concerned about the potential presence of RFC3168 bottlenecks, and both endpoints can be configured to use the mechanism. In those situations, the sender response to seeing SYN or SYN-ACK (or their equivalent) without the Guard DSCP could be to fall back to 3168 mode, or not-ECT mode, or perform more detailed heuristics (e.g. the inband detection algorithms), whichever is more appropriate for the situation.

Also, fast-tracking fixes to Linux/OpenWRT boxes to begin eliminating issues with CAKE deployments would be a good step to take.

-Greg

From: tsvwg <[email-masked]> on behalf of David Black <[email-masked]>
Date: Tuesday, April 27, 2021 at 4:57 PM
To: "C. M. Heard" <[email-masked]>
Cc: TSVWG <[email-masked]>
Subject: Re: [tsvwg] L4S: Guard DSCP

Mike,

[EXTERNAL EMAIL]

Mike Heard

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 29 Apr 2021 20:25:19 -0700

On Thu, Apr 29, 2021 at 10:46 AM Greg White <[email-masked]> wrote:

> In my opinion this proposal (if mandated) would significantly diminish the
> L4S experiment.  There is widespread evidence of DSCP bleaching in the
> Internet,
>

Understood; that comment was anticipated by the reference at the end.

> while there is little evidence of RFC3168 bottleneck deployment.
>

Is that actually true? I got exactly the opposite message from Pete Heist's
data.

> e.g., https://mailarchive.ietf.org/arch/msg/tsvwg/ij5BqzNtMxKV8VbHE8pUevZjx7k/
> [mailarchive.ietf.org]
> <https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/tsvwg/ij5BqzNtMxKV8VbHE8pUevZjx7k/__;!!LpKI!3O_yc-c_keXnDzQr5kIsAGHjsdLwrDdwRmywXodJmxuAEvzsG8GA0pNttYbDFl5f$>)
Mike Heard

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 30 Apr 2021 21:26:45 +0100

[As an individual]

On 30/04/2021 04:25, C. M. Heard wrote:
> On Thu, Apr 29, 2021 at 10:46 AM Greg White <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     In my opinion this proposal (if mandated) would significantly
>     diminish the L4S experiment.  There is widespread evidence of DSCP
>     bleaching in the Internet,
>
>
> Is that actually true? I got exactly the opposite message from Pete 
> Heist's data.

I haven't seen much evidence that pathology of using RFC3168 is common 
(i.e. the behaviour from 2001), if others have better data that has not 
been presented, I would be interested in seeing this.

>
>     I recall from the lengthy discussion that went on about one year
>     ago (see, e.g.,
>     https://mailarchive.ietf.org/arch/msg/tsvwg/ij5BqzNtMxKV8VbHE8pUevZjx7k/
>     [mailarchive.ietf.org]
>     that one of the big objections to the use of a Guard DSCP as a
>     means to contain the experiment is that an L4S experimental domain
>     does not map well to a DSCP domain, and so the Guard DSCP may be
>     inadvertently bleached (e.g. by tunnels) inside the participating
>     network. If correct, that objection is hard to address.
>
>
> Mike Heard
>
I'd personally see a Guard DSCP as an optional mechanism that someone 
can deploy for an operator to contain an experiment. As I recall, the 
original use in RFC 4774, was to allow such a 3-level marking experiment.

Gorry (as an individual)

---

From: Greg White <[email-masked]>
Date: Fri, 30 Apr 2021 21:21:35 +0000

From: "C. M. Heard" <[email-masked]>

On Thu, Apr 29, 2021 at 10:46 AM Greg White <[email-masked]<mailto:[email-masked]>> wrote:
In my opinion this proposal (if mandated) would significantly diminish the L4S experiment.  There is widespread evidence of DSCP bleaching in the Internet,

[GW] You are right, it was.  That reference had been snipped, so I guess it had also been snipped from my short term memory. In any event, I think it is important to not let it get lost in the discussion.

Is that actually true? I got exactly the opposite message from Pete Heist's data.

[GW] There has been little evidence shown of single queue RFC3168 bottlenecks.  In Pete Heist’s data (one small ISP in Czech) the known instances of RFC3168 were all fq_codel, and the rest of the observed CE markings were agreed to likely be fq_codel or CAKE as well.  Similarly the earlier data from Apple showing some level of ECN marking in France were consistent with the reported fq_codel deployments by one of the ISPs there.  Also, I don’t believe that any ISP has come forward to state that they have single queue RFC3168 bottlenecks in their network (and, given the amount of list traffic on this topic, I would assume that any ISP that monitors TSVWG would have self-identified by now).
I recall from the lengthy discussion that went on about one year ago (see, e.g., https://mailarchive.ietf.org/arch/msg/tsvwg/ij5BqzNtMxKV8VbHE8pUevZjx7k/ [mailarchive.ietf.org]<https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/tsvwg/ij5BqzNtMxKV8VbHE8pUevZjx7k/__;!!LpKI!3O_yc-c_keXnDzQr5kIsAGHjsdLwrDdwRmywXodJmxuAEvzsG8GA0pNttYbDFl5f$>) that one of the big objections to the use of a Guard DSCP as a means to contain the experiment is that an L4S experimental domain does not map well to a DSCP domain, and so the Guard DSCP may be inadvertently bleached (e.g. by tunnels) inside the participating network. If correct, that objection is hard to address.

Mike Heard