Subject: [tsvwg] Call for comments on the suggested publication timeline for draft-white-tsvwg-l4sops

From: Gorry Fairhurst <[email-masked]>
Date: Mon, 21 Mar 2022 15:28:16 +0100

The presentation in TSVWG on Monday 21st March 2022 indicated that the 
authors thought the document was complete and ready for review. The 
chairs previously indicated that publication could be delayed to wait 
for initial experience.  The chairs would appreciate feedback, to help 
decide.

So, does the WG have a  preference regarding when to publish the L4S Ops 
draft:

(1)  If the work is thought complete, there could be value in having a 
frozen draft while experience is gained, this could be updated with 
initial experience, before requesting publication as an RFC.

(2) The WG would not intentionally delay the request for the IESG to 
publish: If the work is thought complete, then we can finish this draft 
and publish in the short-term. The WG retains the option to decide to 
subsequently publish an updated RFC in the future.

Please could send an email to tsvwg, or to the tsvwg chairs by 8th March 
2022.

Before any publication, the ID will be subject to normal review and WGLC 
comments.

Best wishes,

Gorry Fairhurst
(tsvwg co-chair)

---

From: Greg White <[email-masked]>
Date: Mon, 21 Mar 2022 17:53:22 +0000

Hi Gorry,

A few points of clarification.  

The draft in question is actually https://datatracker.ietf.org/doc/draft-ietf-tsvwg-l4sops/

Also, there remains one "ToDo" in the document that I briefly mentioned today, and for which I'm hoping to get input from the WG.  Aside from that one item, I'm not aware of any other areas where it is considered incomplete.

Is the deadline for responses to your question possibly *April* 8?

-Greg

On 3/21/22, 8:28 AM, "tsvwg on behalf of Gorry Fairhurst" <[email-masked] on behalf of [email-masked]> wrote:

    Best wishes,

    Gorry Fairhurst
    (tsvwg co-chair)

---

From: Sebastian Moeller <[email-masked]>
Date: Tue, 22 Mar 2022 10:39:43 +0100

So the one bid issue I see is that the whole length section
 6.  Operator of a Network Employing RFC3168 FIFO Bottlenecks  . .  13
is aimed at an audience which is in no way guaranteed to even notice this draft at all.
All networks employing RFC3168 (FIFO) Bottlenecks will see unexpected behavior once L4S-type ECN response traffic traverses their network and one of the rfc3168 nodes becomes the bottleneck. Those networks that are also actively deploying L4S (either for edge testing or in the core) will likely notice all L4S related drafts/RFCs, but networks not participating in the L4S experiment are IMHO unlikely to do so.

What is our intended way forward on how to make sure we actually reach the affected audience in a timely fashion?

If getting the OPs draft RFC'd will result in a noticeable higher likelihood of being stumbled over by affected networks, then RFC-ing soon seems wise. If the IETF document status has no effect on the L4S OPs text "visibility" then I would argue it does not really matter and keeping it open to avoid a .bis version seems equally productive as "declaring victory now" and start an .bis draft (so I think maybe those who actually will carry the burden of maintaining/updating the L4S-OPs RFC/draft should make that decision).

Regards
	Sebastian

P.S.: My preferred solution to all of this is not to ratify the main L4S drafts any time soon since I consider L4S not ready for internet-wide testing* which makes the decision about the OPs draft also less urgent. 

*) The way forward IMHO is to do _safe_ internet-scale experiments first with willingly participating parties (networks and end-hosts) using network paths engineered to minimize impact on unsuspecting third party traffic. Given the illustrious list of entities interested in L4S I am still puzzled why such tests have not been performed or why results have not been published (or at least a short summary posted to the list).

> On Mar 21, 2022, at 15:28, Gorry Fairhurst <[email-masked]> wrote:
> 
> The presentation in TSVWG on Monday 21st March 2022 indicated that the authors thought the document was complete and ready for review. The chairs previously indicated that publication could be delayed to wait for initial experience.  The chairs would appreciate feedback, to help decide.
> 
> So, does the WG have a  preference regarding when to publish the L4S Ops draft:
> 
> (1)  If the work is thought complete, there could be value in having a frozen draft while experience is gained, this could be updated with initial experience, before requesting publication as an RFC.
> 
> (2) The WG would not intentionally delay the request for the IESG to publish: If the work is thought complete, then we can finish this draft and publish in the short-term. The WG retains the option to decide to subsequently publish an updated RFC in the future.
> 
> Please could send an email to tsvwg, or to the tsvwg chairs by 8th March 2022.
> 
> Before any publication, the ID will be subject to normal review and WGLC comments.
> 
> Best wishes,
> 
> Gorry Fairhurst
> (tsvwg co-chair)
>

---

From: Jonathan Morton <[email-masked]>
Date: Tue, 22 Mar 2022 12:52:35 +0200

> On 22 Mar, 2022, at 11:39 am, Sebastian Moeller <[email-masked]> wrote:
> 
> So the one big issue I see is that the whole length section

> All networks employing RFC3168 (FIFO) Bottlenecks will see unexpected behavior once L4S-type ECN response traffic traverses their network and one of the rfc3168 nodes becomes the bottleneck. Those…

Yes, this is symptomatic of the larger problem with L4S as a whole.

> P.S.: My preferred solution to all of this is not to ratify the main L4S drafts any time soon since I consider L4S not ready for internet-wide testing* which makes the decision about the OPs draft…

This would also be my preferred solution.

However, as I understand it, this is now the decision of the IESG to make, not the WG.  It is crucial that the IESG has all the information necessary to make the appropriate decision.  Some of that is in the drafts themselves, some in the shepherd's writeup, and some in the l4s-ops draft.  But the IESG does not look at drafts.

My considered opinion is that the L4S drafts cannot safely be used without the precautions that l4s-ops is intended to detail.  (The adequacy of these precautions is a matter for separate discussion.)  Therefore, publication of the L4S drafts - and consideration by the IESG - should not proceed until l4s-ops is passed by the WG as ready for publication alongside them, as a complete unit.

 - Jonathan Morton

---

From: "Livingood, Jason" <[email-masked]>
Date: Tue, 22 Mar 2022 14:09:56 +0000

Speaking as an L4S experimenter, I'm fine with either approach - publishing an RFC now and updating based on lessons learned later or keeping it open during experiments. That said, I agree with others that I can certainly see the benefits in terms of external awareness in making it an RFC now along with related L4S docs.

Jason

On 3/21/22, 13:53, "tsvwg on behalf of Greg White" <[email-masked] on behalf of [email-masked]> wrote:

    Hi Gorry,

    The draft in question is actually https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-ietf-tsvwg-l4sops/__;!!CQl3mcHX2A!QJkXc_jbXZq5tYf0Q_r9SYFURjW85F09E0exu-KeshccyJn8YqCokyANQtil7XIT7RVG8A$

    -Greg

        Best wishes,

        Gorry Fairhurst
        (tsvwg co-chair)

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 24 Mar 2022 13:35:06 +0000

Gorry,

I can see the two sides of this.
* Sebastian's point that publication could improve visibility for 
network and server operators is a good one.
* But if I were on the IESG and asked to approve publication of 
operational guidance before there was any operational experience, I 
would be concerned.

Similarly, because some of the points in the draft do not have much real 
experience to back them up, they might have to be removed in order to 
get it approved for publication. For instance, we have been arguing about:
* likely traffic patterns;
* likelihood of shared queue Classic ECN being deployed;
* likelihood of hash collisions
* likelihood that good classic ECN bottleneck detection can be implemented;
* likelihood that network equipment can be configured in certain ways; 
etc. etc.
Without some real-world experience on these matters, I'm not sure what 
the value of this document as an RFC would be.

I think Sebastian's point might be slightly reworded to say this draft 
needs 'visible status'. Visibility comes from Internet discussion 
forums, blogs and articles. Not yet being published as an RFC doesn't 
really change visibility (e.g. anyone interested in the implications of 
QUIC would have read a draft without waiting for RFC publication). 
Rather, publication changes the status of the guidance once it is visible.

How about this compromise:
* We agree now that this draft is intended to give guidance based on 
'early' or 'initial' operational experience (e.g. 1 year), and aim to 
publish once there is some experience, but not wait until experience is 
exhaustive.
* Where l4sops is cited in ecn-l4s-id (from the requirement on 
coexistence in Classic ECN AQMs), we edit to say explicitly that l4sops 
is being kept open as a living document for a short while to capture 
initial operational experience.

Then, if ecn-l4s-id is published as an RFC, the visible status of the 
coexistence issue ought to be no less than if two RFCs are published 
about it.
And this extra explanation of the draftiness of the citation should make 
it more likely that people who need to read l4sops will understand its 
status and not dismiss it as "not worth reading yet".

HTH

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: <[email-masked]>
Date: Thu, 24 Mar 2022 14:09:56 +0000

Gorry,

I agree to Bob's proposal to wait with publication until some experimental experience has been reached. 

Regards,

Ruediger

-----Ursprüngliche Nachricht-----
Von: tsvwg <[email-masked]> Im Auftrag von Bob Briscoe
Gesendet: Donnerstag, 24. März 2022 14:35
An: Gorry Fairhurst <[email-masked]>
Cc: [email-masked]
Betreff: Re: [tsvwg] Call for comments on the suggested publication timeline for draft-white-tsvwg-l4sops

Gorry,

I can see the two sides of this.
* Sebastian's point that publication could improve visibility for network and server operators is a good one.
* But if I were on the IESG and asked to approve publication of operational guidance before there was any operational experience, I would be concerned.

Similarly, because some of the points in the draft do not have much real experience to back them up, they might have to be removed in order to get it approved for publication. For instance, we have been arguing about:
* likely traffic patterns;
* likelihood of shared queue Classic ECN being deployed;
* likelihood of hash collisions
* likelihood that good classic ECN bottleneck detection can be implemented;
* likelihood that network equipment can be configured in certain ways; etc. etc.
Without some real-world experience on these matters, I'm not sure what the value of this document as an RFC would be.

I think Sebastian's point might be slightly reworded to say this draft needs 'visible status'. Visibility comes from Internet discussion forums, blogs and articles. Not yet being published as an RFC doesn't really change visibility (e.g. anyone interested in the implications of QUIC would have read a draft without waiting for RFC publication). 
Rather, publication changes the status of the guidance once it is visible.

How about this compromise:
* We agree now that this draft is intended to give guidance based on 'early' or 'initial' operational experience (e.g. 1 year), and aim to publish once there is some experience, but not wait until experience is exhaustive.
* Where l4sops is cited in ecn-l4s-id (from the requirement on coexistence in Classic ECN AQMs), we edit to say explicitly that l4sops is being kept open as a living document for a short while to capture initial operational experience.

Then, if ecn-l4s-id is published as an RFC, the visible status of the coexistence issue ought to be no less than if two RFCs are published about it.
And this extra explanation of the draftiness of the citation should make it more likely that people who need to read l4sops will understand its status and not dismiss it as "not worth reading yet".

HTH

Bob

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 25 Mar 2022 09:14:49 +0000

+1 

I think it makes much sense to wait with he publication for draft-white-tsvwg-l4sops until we have more data from the experiments.

/Ingemar

> -----Original Message-----
> From: tsvwg <[email-masked]> On Behalf Of
> [email-masked]
> Sent: Thursday, 24 March 2022 15:10
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [tsvwg] Call for comments on the suggested publication timeline
> for draft-white-tsvwg-l4sops
> 
> Gorry,
> 
> * But if I were on the IESG and asked to approve publication of operational
> guidance before there was any operational experience, I would be
> concerned.
> 
> experience to back them up, they might have to be removed in order to get
> it approved for publication. For instance, we have been arguing about:
> I think Sebastian's point might be slightly reworded to say this draft needs
> 'visible status'. Visibility comes from Internet discussion forums, blogs and
> articles. Not yet being published as an RFC doesn't really change visibility (e.g.
> anyone interested in the implications of QUIC would have read a draft
> without waiting for RFC publication).
> * We agree now that this draft is intended to give guidance based on 'early'
> or 'initial' operational experience (e.g. 1 year), and aim to publish once there
> is some experience, but not wait until experience is exhaustive.
> * Where l4sops is cited in ecn-l4s-id (from the requirement on coexistence in
> Classic ECN AQMs), we edit to say explicitly that l4sops is being kept open as
> a living document for a short while to capture initial operational experience.
> 
> Then, if ecn-l4s-id is published as an RFC, the visible status of the coexistence
> issue ought to be no less than if two RFCs are published about it.
> And this extra explanation of the draftiness of the citation should make it
> more likely that people who need to read l4sops will understand its status
> and not dismiss it as "not worth reading yet".
> 
> HTH
> 
> 
> 
> Bob
> 
> > Also, there remains one "ToDo" in the document that I briefly mentioned
> today, and for which I'm hoping to get input from the WG.  Aside from that
> one item, I'm not aware of any other areas where it is considered
> incomplete.
> >
> > On 3/21/22, 8:28 AM, "tsvwg on behalf of Gorry Fairhurst" <tsvwg-
> [email-masked] on behalf of [email-masked]> wrote:
> >
> Bob Briscoe                               https://protect2.fireeye.com/v1/url?k=31323334-
> 501d5122-313273af-454445555731-b0108c2709550a3a&q=1&e=c5e952cd-
> dfdb-4fee-babd-2c04232e13cd&u=http%3A%2F%2Fbobbriscoe.net%2F

---

From: Sebastian Moeller <[email-masked]>
Date: Sun, 27 Mar 2022 12:19:53 +0200

Dear list,

if we agree that the L4S OPs draft is supposed to be published only well after the main drafts' ratification and L4S deployment, I believe that the main drafts should at least carry appendices describing how initial experiments should be made safe within the existing internet. This can well be caveated with something  like "For the initial experimental deployment the following recommendations are given which can/will be superseded by a planned RFC giving L4S operational guidance" to make sure these recommendations do not appear to be set in stone in all perpetuity.

Again, I would very much prefer if we would run well-guarded experiments first before RFC'ing the main drafts to gain sufficient operational experience to allow us to give decent guidance (and assure that the safety and functionality assumptions the drafts seem built upon actually hold in reality). That would seem the prudent and inherently safer path forward (and with L4S ideas/drafts being kicking around for almost a decade, adding say two more years for safe internet-scale experiments would not cost all that much in time)

Regards
	Sebastian

> On Mar 25, 2022, at 10:14, Ingemar Johansson S <ingemar.s.johansson=[email-masked]> wrote:
> 
> +1 
>

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 29 Mar 2022 18:05:16 +0100

Sebastian,

> if we agree that the L4S OPs draft is supposed to be published only well after the main drafts' ratification and L4S deployment, I believe that the main drafts should at least carry appendices desc…

[BB] Given the plan all along was to publish l4sops later, this is how 
ecn-l4s-id has already been written. Appendix A.1.5 gives an outline of 
all the types of test and details of the simplest off-line test. For 
in-band testing it refers off to the tech report, which is as much as 
l4sops does - so nothing more would be written on that topic.

The main info in l4sops that is not in the appendix of ecn-l4s-id (but 
referred to from it) is:
a) the outline of all the measurement studies trying to get a handle on 
the prevalence of the problem.
b) all the potential techniques for modifying Classic ECN AQMs

Omitting both these was deliberate:
* The measurement studies have been useful, but there have been multiple 
opinions on what they imply
* There have been questions about whether the AQM mod's are feasible on 
existing hardware

So, I appreciate you're trying to help, but in summary, there isn't 
really anything to write into ecn-l4s-id that hasn't already been 
considered.

> Again, I would very much prefer if we would run well-guarded experiments first before RFC'ing the main drafts to gain sufficient operational experience to allow us to give decent guidance (and assu…

We will gain experience by doing now.
Kicking it around has been offering diminishing returns for about the 
last 12 months. The codepoint needs to be assigned now, so that Internet 
experimentation can commence.

Cheers

Bob

---

From: Greg White <[email-masked]>
Date: Wed, 30 Mar 2022 17:59:03 +0000

Based on comments received thus far, it seems to me that keeping the draft as a draft for a while longer in order to a) not delay the start of the L4S experiments any further, and b) be able to capture some early experience, would be the best course.

-Greg 

    Best wishes,

    Gorry Fairhurst
    (tsvwg co-chair)