Subject: [tsvwg] FW: New Version Notification for draft-white-tsvwg-l4sops-00.txt

From: Greg White <[email-masked]>
Date: Thu, 30 Jul 2020 11:05:59 +0000

TSVWG members-

I've posted a rough draft of Operational Guidance for L4S deployment.  This is not much more than an outline at this point, and is almost certainly incomplete even at that, so please read it with that in mind. 

-Greg

From: "[email-masked]" <[email-masked]>
Date: Thursday, July 30, 2020 at 4:53 AM
To: Greg White <[email-masked]>
Subject: New Version Notification for draft-white-tsvwg-l4sops-00.txt

A new version of I-D, draft-white-tsvwg-l4sops-00.txt
has been successfully submitted by Greg White and posted to the
IETF repository.

Name:		draft-white-tsvwg-l4sops
Revision:	00
Title:		Operational Guidance for Deployment of L4S in the Internet
Document date:	2020-07-30
Group:		Individual Submission
Pages:		7
URL:            https://www.ietf.org/internet-drafts/draft-white-tsvwg-l4sops-00.txt
Status:         https://datatracker.ietf.org/doc/draft-white-tsvwg-l4sops/
Htmlized:       https://tools.ietf.org/html/draft-white-tsvwg-l4sops-00
Htmlized:       https://datatracker.ietf.org/doc/html/draft-white-tsvwg-l4sops

Abstract:
   This is an early, work-in-progress draft - a start at getting some of
   the ideas from the mailing list and email exchanges on paper.

   This draft is intended to provide guidance to operators of end-
   systems, operators of networks, and researchers in order to ensure
   reasonable fairness between L4S and Classic flows sharing a single-
   queue RFC3168 bottleneck link.  This draft identifies opportunites to
   prevent and/or detect and resolve fairness problems in such networks.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Sebastian Moeller <[email-masked]>
Date: Thu, 30 Jul 2020 23:43:52 +0200

Hi Greg,

my comments for sections 1 & 2 below (more will follow):

"1.  Introduction

   In the majority of network paths, including paths where the
   bottleneck link utilizes packet drops (either due to buffer overrun
   or active queue management) in response to congestion, as well as
   paths that implement a 'flow-queuing' scheduler such as fq_codel or
   Cobalt, and those that implement dual-Q-coupled AQM, L4S traffic
   coexists well with classic congestion controlled traffic."

[SM] This description ignores the birthday paradox issue that Pete Heist demonstrated. A misbehaving L4S will cause less havok there but it is not without side-effects on stochastic flow queueing systems, we might as well acknowledge that, no?

"   On network paths where the bottleneck link implements a shared-queue
   (FIFO) with an Active Queue Management algorithm that provides
   Explicit Congestion Notification signaling according to RFC3168, it
   has been demonstrated that when a set of long-running flows
   comprising both "Classic" congestion controlled flows and L4S-
   compliant congestion controlled flows compete for bandwidth, the
   classic congestion controlled flows may achieve lower throughput when
   compared to the L4S congestion controlled flows.  This 'unfairness'
   between the two classes appears to be more pronounced on longer RTT
   paths (e.g. 50ms and above) and/or at higher link rates (e.g. 50 Mbps
   and above)."

This is rather cautious I would use more drastic terms, the observed unfairness approaches starvation of the non-L4S flows and this text makes it sound like it is a minor almost theoretical concern.

"  The root cause of this unfairness is that RFC3168 does not
   differentiate between packets marked ECT0 (used by classic senders)
   and those marked ECT1 (used by L4S senders), and provides an
   identical congestion signal (CE marks) to both classes, whereas the
   two classes respond differently to that congestion signal."

How about keeping causality intact and frame this as a consequence of L4S redefining what CE means. There are reasons for doing that, but this text leaves it unclear who caused this problem (or rather the text implicates rfc3168, which IMHO violates temporal causality).

"The result is that the
   classic senders respond to the CE marks provided by the bottleneck by
   yielding capacity to the L4S flows.  While this has not been
   demonstrated to cause starvation of the classic flows, the resulting
   rate imbalance can be a cause of concern."

Mmmh, http://sce.dnsmgr.net/results/ect1-2020-04-23-final/l4s-s2-twoflow/l4s-s2-twoflow-ns-cubic-vs-prague-codel1q_20ms_-160ms_tcp_delivery_with_rtt.svg
pretty much demonstrated starvation and that is with TCP-Prague with rfc3168 detection. 

"2.  Per-Flow Fairness

   There are a number of factors that influence the relative rates
   achieved by a set of congestion controlled flows sharing a queue in a
   bottleneck link.

   TODO: discuss startup & convergence times, short flows, RTT-
   unfairness, differences in deployed CC algorithms, etc.

   TODO: also mention that flow sharding is commonplace, so per-flow
   fairness does not imply per-application fairness"

This obviously needs flashing out before one can meaningfully comment, but regarding the last section, you realize that application not the relevant classification for intermediate nodes? I would assume that for an ISP a (potentially weighted) per-end-host fairness would be the exact tool to avoid sharding as work-around... This is obviously not a new idea or comment, so if you opt for elaborating on fairness, please also include known remedies. Or better avoid that discussion here altogether.

Best Regards
	Sebastian

> On Jul 30, 2020, at 13:05, Greg White <[email-masked]> wrote:
> 
> TSVWG members-
>

---

From: Sebastian Moeller <[email-masked]>
Date: Fri, 31 Jul 2020 12:10:59 +0200

Hi Greg,

comments for section 3:

"3.  Operator of an L4S host

   Support for L4S involves both endpoints: ECT1 marking & L4S-
   compatible congestion control on the sender, and ECN feedback on the
   receiver.  Between these two entities, it is incumbent upon the
   sender to evaluate the potential for unfairness and make decisions
   whether or not to use L4S congestion control.  The receiver is not
   expected to perform any testing or monitoring for unfairness, and is
   also not expected to invoke any active response in the case that
   unfairness occurs."

[SM] Maybe worth noting that the receiver will need to play a role in that it is only information send back from the receiver to the sender that will allow the sender to make appropriate decisions. In other words the reciever is still passively involved, which is compatible with your text, but maybe make this explicit?

"3.1.  CDN Servers

   Some hosts (such as CDN leaf nodes and servers internal to an ISP)
   are deployed in environments in which they serve content to a
   constrained set of networks or clients.  The operator of such hosts
   may be able to determine whether there is the possibility of RFC3168
   FIFO bottlenecks being present, and utilize this information to make
   decisions on selectively deploying L4S."

[SM] Mmmh, in reality the ISP needs full control over all the end-hosts in that set, otherwise we are back at the concern that an ISP might make unfortunate choices for the connected machines in its customers leaf networks, no?

   "o  Prior to deploying L4S on servers:

      *  Consult with network operators on presence of RFC3168 FIFO
         bottlenecks"

[SM] As above, these bottlenecks can be actually located at end-hosts, so it might be required to check with all operators of networks and end-hosts. That said, checking with the network operators should be done, independent of the fact that it will not give a full picture, since a positive answer from net ops will already have consequences.

     " *  Perform downstream tests per access network

         +  Tests (TBD) to detect absence of RFC 3168"

[SM] That will require perpetual vigilance though, as it is simply hard to prove a negative, as an AQM might be instantiated not only on intermediary network hops but also on end-systems. 

         "+  Enable AccECN feedback, but enable/disable L4S per access
            network"

[SM] Sounds like decent way to try to asses whether L4S capable endpoints might exist in those networks. ASSUMING that all L4S transports will use AccECN, which I am just not certain of. I think we heard rumblings from Google employees that envision using DCTCP style ECN marking without using TCP-Prague and/or witout following all the principles declared required for using L4S style signaling.

   "o  In-band RFC3168 detection and monitoring:

      *  Real-time response (fallback)

[SM] This is again having all the issues of being an imprecise heuristic... which is worth noting in some part of this document, no?

      "*  Non-real-time response (disable for future connections)"

[SM] Mmmh, how realistic is it to assume that an operator will be willing to maintain that much state?

"3.2.  Other hosts

   o  In-band RFC3168 detection (and possibly fallback)

   o  Per-dst path test:

      *  For a connection capable of L4S feedback

      *  If CE feedback, perform active test (TBD) for RFC3168 presence"

[SM] Remind me again, have we actually seen numbers for that active tests yet, especially regarding convergence time and false positive and false negative rates?

     " *  Could cache result per-dst"

[SM] This again will be quite a lot of state, no?

   o  Query a TBD public whitelist of domains that are participating in
      L4S experiment"

[SM] +1; I very much hope that the initial experiment will be performed with an explicit allow list, that will also ALLOW end users to register/unregister explicitly.

Best Regards
	Sebastian