Subject: [tsvwg] draft-white-tsvwg-l4sops-02.txt proposa for additional sections

From: Sebastian Moeller <[email-masked]>
Date: Fri, 16 Apr 2021 22:57:29 +0200

Hi Greg, hi list,

I think that the L4S ops draft could be improved by saying something explicit about the end of the L4S experiment and what differential actions are recommended for participants at that point in time. I made this argument already at the end of https://mailarchive.ietf.org/arch/msg/tsvwg/NXEACkPX1pFriOtqsoifPzoI8Sk/ but I believe that it might have been overlooked there at the end of a long email.

So, here is my proposal to that regard as a new section 6:

6. What L4S-deploying networks should to do at the end of the L4S experiment

This section gives guidance how L4S-deploying networks should respond to any of the two possible outcomes of the IETF-supported L4S experiment.

[COMMENT: I believe that there needs to be a proper definition how the L4S experiment is supposed to be evaluated at the end to decide between success and failure, but the L4S-ops document seems to be the wrong place for that, so I propose to add this to one of the L4S core IDs, preferably to section 6 of https://datatracker.ietf.org/doc/draft-ietf-tsvwg-ecn-l4s-id/ which already contains discussion about what the experiment is supposed to figure out, so adding a section how to evaluate the outcome there seems a natural fit; 
IMHO the deciding factor here could be whether L4S is promoted to proposed standard status after the experiment, if and only if that happens L4S should be considered a success... I base this on the already known problematic side-effects of L4S and the fact that L4S will consume a rather scarce resource in an IP-level code-point the exists in both IPv4 and IPv6, which IMHO should be released ASAP if L4S fails to succeed]

6.1 Successful termination of the L4S experiment

If the L4S experiment is deemed successful, participating networks are encouraged to continue deploying L4S-aware nodes and if possible replace all non-L4S-aware rfc3168 AQM already deployed as far as feasible, or at least restrict rfc3168 AQM to interpret ECT(1) equal to NotECT. Participants are also expected to track the evolution of the L4S standards and adapt their implementations accordingly (e.g. if as part of switching from experimental to standards track, changes in the L4S RFCs become necessary).

6.2 Unsuccesful termination of the L4S experiment

If the L4S experiment is deemed unsuccessful, it might need to be terminated: L4S network nodes should be un-deployed and the ECT(1) codepoint usage should be released/recycled quickly. 
	To achieve the former, participants of the L4S experiment are expected to configure their L4S-aware network nodes such that either the LL-queue of a dual queue AQM gets completely disabled, or that the LL-queue is switched from issuing CE-marks to pure drops; thereby removing L4S-signaling from the network. To achieve the latter, all endpoint hosts need to stop negotiating L4S-congestion signaling. For nodes under control of participating networks that should only require re-configuration of the endpoint hosts. For nodes not under control of participants of the experiment that will be a considerable challenge. To give a strong signal to such out-of-network endpoints drastic measures might be required, like re-marking ECT(1) packets to NotECT or even dropping all ECT(1) packets on network ingress, as these will give clear and strong incentives to endpoints to stop using ECT(1)/L4S-signaling. But to avoid "burning" the ECT(1) codepoint indefinitely, such measures should be restricted to a limited duration (e.g. 24 month) after an unsuccessful termination of the L4S experiment, to allow the ECT(1) codepoint to be recycled for other uses/experiments afterwards.

I am sure that this is not going to be complete and that there will be differing opinions on these sections, but I believe something similar in scope should be added.

Best Regards
	Sebastian

---

From: Jonathan Morton <[email-masked]>
Date: Sat, 17 Apr 2021 00:43:30 +0300

> On 16 Apr, 2021, at 11:57 pm, Sebastian Moeller <[email-masked]> wrote:
> 

One measure that should seriously be considered is placing an explicit, coordinated sunset date in each implementation, after which the implementation MUST assume that the experiment has concluded unsuccessfully and L4S semantics are no longer permitted on the wire.  This sunset date can be extended if a longer experiment is deemed desirable, by deploying revised implementations to all appropriate nodes.

The advantage of this approach is that *if* the experiment is unsuccessful, there is no long tail of deployed implementations that might inadvertently interfere with future experiments using the same fields and codepoints.  Such new experiments can begin at least as soon as the last sunset date actually deployed.  Given the scarcity of available codepoints in the ECN field, and the poor performance of L4S in lab tests to date, I think this is a prudent precaution to take.

 - Jonathan Morton

---

From: Ingemar Johansson S <[email-masked]>
Date: Sat, 17 Apr 2021 14:21:45 +0000

Hi Sebastian..
Unfortunately the proposed section 6.2 sounds a lot like the famous Mission
Impossible one-liner "This tape will self-destruct in five seconds. Good
luck, Jim."
The ECN Nonce was experimental and was recently declared historic a few
years ago, literary speaking RFC3540 did not self destruct, it just faded
away. I don't see anywhere in RFC3540 about what end hosts and networks
should do when RFC3540 is declared historic. And I guess that the way
forward, in case the whole L4S experiment falls flat, is that the L4S ID RFC
is declared historic, I guess this should be enough.
But lets imagine that detailed information is needed to address the case
that L4S fails. This leads to the question. 
Are there any other examples in IETF's history where an experimental RFC has
been accompanied with a detailed instruction on how to clean up after the
experiment ?

/Ingemar 

> -----Original Message-----
> From: tsvwg <[email-masked]> On Behalf Of Sebastian Moeller
> Sent: den 16 april 2021 22:57
> To: tsvwg IETF list <[email-masked]>; Greg White <[email-masked]>
> Subject: [tsvwg] draft-white-tsvwg-l4sops-02.txt proposa for additional
> sections
> 
> Hi Greg, hi list,
> 
> I think that the L4S ops draft could be improved by saying something
explicit
> about the end of the L4S experiment and what differential actions are
> recommended for participants at that point in time. I made this argument
> already at the end of
> https://mailarchive.ietf.org/arch/msg/tsvwg/NXEACkPX1pFriOtqsoifPzoI8Sk/
> but I believe that it might have been overlooked there at the end of a
long
> email.
> This section gives guidance how L4S-deploying networks should respond to
> any of the two possible outcomes of the IETF-supported L4S experiment.
> 
> [COMMENT: I believe that there needs to be a proper definition how the L4S
> experiment is supposed to be evaluated at the end to decide between
> success and failure, but the L4S-ops document seems to be the wrong place
> for that, so I propose to add this to one of the L4S core IDs, preferably
to
> section 6 of https://datatracker.ietf.org/doc/draft-ietf-tsvwg-ecn-l4s-id/
> which already contains discussion about what the experiment is supposed to
> figure out, so adding a section how to evaluate the outcome there seems a
> natural fit; IMHO the deciding factor here could be whether L4S is
promoted
> to proposed standard status after the experiment, if and only if that
happens
> L4S should be considered a success... I base this on the already known
> problematic side-effects of L4S and the fact that L4S will consume a
rather
> scarce resource in an IP-level code-point the exists in both IPv4 and
IPv6,
> which IMHO should be released ASAP if L4S fails to succeed]
> 
> If the L4S experiment is deemed successful, participating networks are
> encouraged to continue deploying L4S-aware nodes and if possible replace
all
> non-L4S-aware rfc3168 AQM already deployed as far as feasible, or at least
> restrict rfc3168 AQM to interpret ECT(1) equal to NotECT. Participants are
also
> expected to track the evolution of the L4S standards and adapt their
> implementations accordingly (e.g. if as part of switching from
experimental
> to standards track, changes in the L4S RFCs become necessary).
> 
> 
> If the L4S experiment is deemed unsuccessful, it might need to be
> terminated: L4S network nodes should be un-deployed and the ECT(1)
> codepoint usage should be released/recycled quickly.
> 	To achieve the former, participants of the L4S experiment are
> expected to configure their L4S-aware network nodes such that either the
> LL-queue of a dual queue AQM gets completely disabled, or that the LL-
> queue is switched from issuing CE-marks to pure drops; thereby removing
> L4S-signaling from the network. To achieve the latter, all endpoint hosts
> need to stop negotiating L4S-congestion signaling. For nodes under control
of
> participating networks that should only require re-configuration of the
> endpoint hosts. For nodes not under control of participants of the
> experiment that will be a considerable challenge. To give a strong signal
to
> such out-of-network endpoints drastic measures might be required, like re-
> marking ECT(1) packets to NotECT or even dropping all ECT(1) packets on
> network ingress, as these will give clear and strong incentives to
endpoints to
> stop using ECT(1)/L4S-signaling. But to avoid "burning" the ECT(1)
codepoint
> indefinitely, such measures should be restricted to a limited duration
(e.g. 24
> month) after an unsuccessful termination of the L4S experiment, to allow
the
> ECT(1) codepoint to be recycled for other uses/experiments afterwards.
> 
> 
> 
> I am sure that this is not going to be complete and that there will be
differing
> opinions on these sections, but I believe something similar in scope
should
> be added.

---

From: Sebastian Moeller <[email-masked]>
Date: Sat, 17 Apr 2021 16:54:30 +0200

Hi Ingemar,

in essence you seem to argue against adding such a section, do i understand that correctly?

> On Apr 17, 2021, at 16:21, Ingemar Johansson S <[email-masked]> wrote:
> 
> Hi Sebastian..

	[SM] How that? Section 6 is requiring network operators tat deployed L4S during the experiment to take active steps to incentivize end-points to stop using L4S-signaling ASAP, there is no "self-destruction" involved at all; I am not sure whether that is a fitting analogy. How about commenting upon the specifics of my proposal instead?

> away.

	[SM] Sure, but I note that the Nonce had no negative side-effects on standards compliant AQMs (unlike L4S): if after a terminated experiment L4S signaling is still employed by end-points this will still wreck havoc with rfc 3168 AQM and strongly violate the expected sharing behaviour at those nodes. Now at that point in time, rfc3168 will still have PS status and there are no motions to change that I am aware of, while L4S will have failed and the RFC needs to be made historic; I would humbly argue that rfc3168 operators should be able expect no delayed side-effects from a terminated experiment, no?

> I don't see anywhere in RFC3540 about what end hosts and networks
> should do when RFC3540 is declared historic.

	[SM] Given the lack of side-effects there is/was no need to do so. 

> And I guess that the way

	[SM] How is that going to help one iota with the situation that will arose then: end-points will continue to negotiate L4S-signaling and rfc3168 AQM on the path will take a hit from that. 
	Yes,  I agree that this is a consequence of a design decision in L4S (one I consider sub-optimal), but it has become clear, that team L4S is not going to entertain the idea of actually changing the L4S design one bit. So if that dangerous signal stays part of L4S, we need a way to un-deploy L4S signaling from end-points. My proposal does not achieve that by itself, but it will 

> been accompanied with a detailed instruction on how to clean up after the

	[SM] While I am as much a fan of precedence as the next guy, I am not interested in changing/improvinf IETF processes in general. I am interested however in keeping the blast range of the L4S experiment as small as possible. Data so far strongly indicates failure is likely and hence prident engineering requires to take precautions. 

BUT Ingemar, how about you propose an alternative text for the L4S-ops draft how to handle possible actions after the experiment has run its course that acknowledge that L4S is one of the riskier experiments the IETF has ever started? Maybe my choice of wording was perceived as too adversarial, and we agree in principle?
Or is your argument that you reject adding section 6 to the OPs ID?

Best Regards
	Sebastian

---

From: Ingemar Johansson S <[email-masked]>
Date: Sat, 17 Apr 2021 18:55:57 +0000

Hi Sebastian
Please see inline [IJ]
/Ingemar

> From: Sebastian Moeller <[email-masked]>
> Sent: den 17 april 2021 16:55
> To: Ingemar Johansson S <[email-masked]>
> Cc: tsvwg IETF list <[email-masked]>; Greg White <[email-masked]>
> Subject: Re: [tsvwg] draft-white-tsvwg-l4sops-02.txt proposa for
additional
> sections
> 
> Hi Ingemar,
> 
> in essence you seem to argue against adding such a section, do i
understand
> that correctly?

[IJ] Not yet. But I am interested to know if there is any precedence in this
area in IETF, i.e, is there any experimental RFC that dictates in detail of
the experimental RFC is undone. What I have seen is that RFCs are declared
historical but I have a hard time to see how the exit of an RFC is
formulated in detail, especially if other SDOs adopt the RFC ? Should IETF
oversee that endpoints are updated. You have your mentioned how hard it is
to update e.g. RFC3168 edge routers. 
 
> 
> 
> > On Apr 17, 2021, at 16:21, Ingemar Johansson S
> <[email-masked]> wrote:
> >
> > Hi Sebastian..
> > Unfortunately the proposed section 6.2 sounds a lot like the famous
> > Mission Impossible one-liner "This tape will self-destruct in five
> > seconds. Good luck, Jim."
> 
> 	[SM] How that? Section 6 is requiring network operators tat
> deployed L4S during the experiment to take active steps to incentivize
end-
> points to stop using L4S-signaling ASAP, there is no "self-destruction"
> involved at all; I am not sure whether that is a fitting analogy. How
about
> commenting upon the specifics of my proposal instead?
[IJ] I see it as some kind of requirement for a self-destruction mechanism
in the sense that nodes that implement L4S should exit L4S at a given day or
otherwise put some requirement on network operators to terminate it. How
many will follow these guidelines and based what decision, a special IETF
interim that declares "now we close the shop, everybody out!" . What it
other SDOs have picked up L4S ?
The only thing I see that experimental RFCs fade away because or lack of
interest, poor performance or whatever other reason that may exist. 

> 
> 
> > The ECN Nonce was experimental and was recently declared historic a
> > few years ago, literary speaking RFC3540 did not self destruct, it
> > just faded away.
> 
> 	[SM] Sure, but I note that the Nonce had no negative side-effects on
> standards compliant AQMs (unlike L4S): if after a terminated experiment
L4S
> signaling is still employed by end-points this will still wreck havoc with
rfc 3168
> AQM and strongly violate the expected sharing behaviour at those nodes.
> Now at that point in time, rfc3168 will still have PS status and there are
no
> motions to change that I am aware of, while L4S will have failed and the
RFC
> needs to be made historic; I would humbly argue that rfc3168 operators
> should be able expect no delayed side-effects from a terminated
> experiment, no?
> 
> 
> > forward, in case the whole L4S experiment falls flat, is that the L4S
> > ID RFC is declared historic, I guess this should be enough.
> 
> 	[SM] How is that going to help one iota with the situation that will
> arose then: end-points will continue to negotiate L4S-signaling and
rfc3168
> AQM on the path will take a hit from that.
> 	Yes,  I agree that this is a consequence of a design decision in L4S
> (one I consider sub-optimal), but it has become clear, that team L4S is
not
> going to entertain the idea of actually changing the L4S design one bit.
So if
> that dangerous signal stays part of L4S, we need a way to un-deploy L4S
> signaling from end-points. My proposal does not achieve that by itself,
but it
> will
> 
> > But lets imagine that detailed information is needed to address the
> > case that L4S fails. This leads to the question.
> > Are there any other examples in IETF's history where an experimental
> > RFC has been accompanied with a detailed instruction on how to clean
> > up after the experiment ?
> 
> 	[SM] While I am as much a fan of precedence as the next guy, I am
> not interested in changing/improvinf IETF processes in general. I am
> interested however in keeping the blast range of the L4S experiment as
small
> as possible. Data so far strongly indicates failure is likely and hence
prident
> engineering requires to take precautions.
[IJ] Sure, I understand that you believe that L4S will be a big fail, and
you want some exit method other than the declare historical. But perhaps I
misunderstand how IETF works and your intentions?. I have too few years in
the IETF to know how all these processes work 

> 
> BUT Ingemar, how about you propose an alternative text for the L4S-ops
> draft how to handle possible actions after the experiment has run its
course
> that acknowledge that L4S is one of the riskier experiments the IETF has
ever
> started? Maybe my choice of wording was perceived as too adversarial, and
> we agree in principle?
> >> Subject: [tsvwg] draft-white-tsvwg-l4sops-02.txt proposa for
> >> additional sections
> >>
> >> Hi Greg, hi list,
> >>
> >> recommended for participants at that point in time. I made this
> >> argument already at the end of
> >> https://mailarchive.ietf.org/arch/msg/tsvwg/NXEACkPX1pFriOtqsoifPzoI8
> >> Sk/ but I believe that it might have been overlooked there at the end
> >> of a
> > long
> >> email.
> >>
> >> This section gives guidance how L4S-deploying networks should respond
> >> to any of the two possible outcomes of the IETF-supported L4S
> experiment.
> >>
> >> [COMMENT: I believe that there needs to be a proper definition how
> >> the L4S experiment is supposed to be evaluated at the end to decide
> >> between success and failure, but the L4S-ops document seems to be the
> >> wrong place for that, so I propose to add this to one of the L4S core
> >> IDs, preferably
> > to
> >> section 6 of
> >> https://datatracker.ietf.org/doc/draft-ietf-tsvwg-ecn-l4s-id/
> >> which already contains discussion about what the experiment is
> >> supposed to figure out, so adding a section how to evaluate the
> >> outcome there seems a natural fit; IMHO the deciding factor here
> >> could be whether L4S is
> > promoted
> >> L4S should be considered a success... I base this on the already
> >> known problematic side-effects of L4S and the fact that L4S will
> >> consume a
> > rather
> >> If the L4S experiment is deemed successful, participating networks
> >> are encouraged to continue deploying L4S-aware nodes and if possible
> >> replace
> > all
> >> non-L4S-aware rfc3168 AQM already deployed as far as feasible, or at
> >> least restrict rfc3168 AQM to interpret ECT(1) equal to NotECT.
> >> Participants are
> > also
> >> expected to configure their L4S-aware network nodes such that either
> >> the LL-queue of a dual queue AQM gets completely disabled, or that
> >> the LL- queue is switched from issuing CE-marks to pure drops;
> >> thereby removing L4S-signaling from the network. To achieve the
> >> latter, all endpoint hosts need to stop negotiating L4S-congestion
> >> signaling. For nodes under control
> > of
> >> participating networks that should only require re-configuration of
> >> the endpoint hosts. For nodes not under control of participants of
> >> the experiment that will be a considerable challenge. To give a
> >> strong signal
> > to
> >> such out-of-network endpoints drastic measures might be required,
> >> like re- marking ECT(1) packets to NotECT or even dropping all ECT(1)
> >> packets on network ingress, as these will give clear and strong
> >> incentives to
> > endpoints to