Subject: [tsvwg] Regarding https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id-14#section-6 and involuntary participation

From: Sebastian Moeller <[email-masked]>
Date: Fri, 30 Apr 2021 10:41:32 +0200

Dear All,

https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id-14#section-6 list the following research goals for the L4S experiment (without assigning any of these to specific parties):
amended with paragraph identifiers

"6.1.  Open Questions

   L4S experiments would be expected to answer the following questions:

   A)  Have all the parts of L4S been deployed, and if so, what
      proportion of paths support it?

   B)  Does use of L4S over the Internet result in significantly improved
      user experience?

   C)  Has L4S enabled novel interactive applications?

   D)  Did use of L4S over the Internet result in improvements to the
      following metrics:

      D.1)  queue delay (mean and 99th percentile) under various loads

      D.2)  utilization

      D.3)  starvation / fairness

      D.4)  scaling range of flow rates and RTTs

   E)  How much does burstiness in the Internet affect L4S performance,
      and how much limitation of bustiness was needed and/or was
      realized - both at senders and at links, especially radio links?

   F)  Was per-flow queue protection typically (un)necessary?

      F.1)  How well did overload protection or queue protection work?

   G)  How well did L4S flows coexist with Classic flows when sharing a
      bottleneck?

      G.1)  How frequently did problems arise?

      G.2)  What caused any coexistence problems, and were any problems due
         to single-queue Classic ECN AQMs (this assumes single-queue
         Classic ECN AQMs can be distinguished from FQ ones)?

   H)  How prevalent were problems with the L4S service due to tunnels /
      encapsulations that do not support ECN decapsulation?

   I)  How easy was it to implement a fully compliant L4S congestion
      control, over various different transport protocols (TCP.  QUIC,
      RMCAT, etc)?

   Monitoring for harm to other traffic, specifically bandwidth
   starvation or excess queuing delay, will need to be conducted
   alongside all early L4S experiments.  It is hard, if not impossible,
   for an individual flow to measure its impact on other traffic.  So
   such monitoring will need to be conducted using bespoke monitoring
   across flows and/or across classes of traffic."

Quite a number of those seem to require active feed-back from both end-points of an L4S path (like D.3, D.4, F, F.1, G, G.1, ...). What is the plan how to get to that data from end-hosts that did not knowingly opt-in to the experiment and hence might not even now that they are supposed to monitor and report such issues (and report to whom)? End points not knowing they were en-rolled in an experiment will not be prepared to monitor anything, let alone "bandwidth starvation or excess queuing delay"; and yet end-point will be the places that will be affected most.

I would be interested in hearing the opinion of the wider list on that issue, so if anybody is willing to offer an opinion I would be delighted.

Best Regards
	Sebastian

P.S.: I might be overly pessimistic, but I fail to see an alternative to run the experiment opt-in to get meaningful feed-back on the open questions. And in all honesty, I consider it pre-mature to run an experiment over the wider internet with questions like "Does use of L4S over the Internet result in significantly improved user experience" still "open" (we already see failure to achieve that goal in lab settings, so expecting any change for the better by running over a less controlled environment seems optimistic.)