Subject: [tsvwg] TCP Prague's RFC 5033 guidelines status?

From: "Holland, Jake" <[email-masked]>
Date: Sun, 10 Nov 2019 20:58:48 +0000

Hi l4s-arch authors,

In a side discussion, Pete mentioned the RFC 5033 guidelines list for
new congestion controls, and looking through the items in Section 3, I
wasn't sure whether or not they were all addressed in the L4S drafts and
references:
https://tools.ietf.org/html/rfc5033#section-3

Would it be possible to add a section that goes through the points from
RFC 5033 section 3 and provides pointers to the text and/or research
results you know of that addresses each of them?

I think it would be very helpful for the review of the proposal to have
those listed out instead of trying to find them by digging through the
documents and references.

I guess it would also be OK to provide such a map on the mailing list,
but I'll suggest that I think it would be more helpful if there's a way
to include it as a section in one of the drafts.

(And if people agree this is a useful addition, should we open a new
issue to track it?)

Thanks and regards,
Jake

---

From: Wesley Eddy <[email-masked]>
Date: Mon, 11 Nov 2019 09:24:53 -0500

> (And if people agree this is a useful addition, should we open a new

Across threads, some of what we're discussing as "L4S" is really TCP 
Prague end-host code or algorithms, and not strictly the L4S in-network 
parts that we're currently chartered for in TSVWG.

I think 5033 is more relevant to the TCP Prague behaviors that use the 
signals coming from L4S queues, than the L4S in-network behavior by itself.

---

From: "Black, David" <[email-masked]>
Date: Mon, 11 Nov 2019 16:06:52 +0000

Posting as an individual, not as a WG chair ...

> signals coming from L4S queues, than the L4S in-network behavior by itself.

Well, L4S imposes strict requirements on end-host algorithms for traffic that uses the low-latency service.  TCP Prague is the only known transport protocol that satisfies those requirements, i.e., no other existing TCP or DCTCP code satisfies those requirements, as far as I am aware (please correct this statement if it's wrong).

For that reason, decisions about deploying L4S and TCP Prague appear to be linked, as it would be silly to deploy a network service (L4S) that cannot be used by end-systems, and end-system usage of L4S appears to require TCP Prague.

Thanks, --David

> -----Original Message-----
> From: tsvwg <[email-masked]> On Behalf Of Wesley Eddy
> Sent: Monday, November 11, 2019 9:25 AM
> To: [email-masked]
> Subject: Re: [tsvwg] TCP Prague's RFC 5033 guidelines status?
> 
> 
> [EXTERNAL EMAIL]
>

---

From: Wesley Eddy <[email-masked]>
Date: Mon, 11 Nov 2019 17:11:43 -0500

> For that reason, decisions about deploying L4S and TCP Prague appear to be linked, as it would be silly to deploy a network service (L4S) that cannot be used by end-systems, and end-system usage of…

I believe you're referring to the congestion response requirements in 
the L4S ID draft.  I think those are a start to requiring an end-host 
algorithm to have good properties with regard to 5033 guidelines, rather 
than requiring some bad behavior (e.g. the first one is "MUST coexist 
safely with Reno" which speaks right to 5033 guideline 1 "Impact on 
Standard TCP, SCTP, and DCCP").  So, I'm not sure why those are a 
concern (i.e. which L4S ID congestion response requirements do we have 
5033 concerns with?).

If really needed, it seems like the 5033 guidelines could be mapped to 
sections of the L4S documents where the topics are discussed, but I'm 
not sure how meaningful it could be without regard to specific scalable 
end-host algorithms (e.g. Prague).

---

From: "Black, David" <[email-masked]>
Date: Tue, 12 Nov 2019 00:55:51 +0000

Wes,
[still as an individual]

> end-host algorithms (e.g. Prague).

My current (individual) view is that the L4S experiment requires TCP Prague, so I view those two activities as inherently linked.  If anyone disagrees and wants to continue the discussion ;-), please identify another congestion-controlled transport protocol that meets the L4S low latency service requirements.  In contrast, the SCE folks appear to have successfully modified a number of congestion-controlled transport protocols.

Thanks, --David

> From: Wesley Eddy <[email-masked]>
> Sent: Monday, November 11, 2019 5:12 PM
> To: Black, David; [email-masked]
> > Well, L4S imposes strict requirements on end-host algorithms for traffic that
> uses the low-latency service.  TCP Prague is the only known transport protocol
> that satisfies those requirements, i.e., no other existing TCP or DCTCP code
> satisfies those requirements, as far as I am aware (please correct this statement
> if it's wrong).
> >
> > For that reason, decisions about deploying L4S and TCP Prague appear to be
> linked, as it would be silly to deploy a network service (L4S) that cannot be used
> by end-systems, and end-system usage of L4S appears to require TCP Prague.
> 
>

---

From: "Rodney W. Grimes" <[email-masked]>
Date: Mon, 11 Nov 2019 18:04:45 -0800 (PST)

David, Wes, etc al,

	Though RFC5033 may of been written at a time when only end node
changes to protocols had been occuring I think its spirit is that of
a "functional system".  L4S as an IP modification makes zero since
without the existance of a transport (TCP Prague) to use with it.

	It is, IMHO, imperative for the sucess of any congestion
mechanism that both the Internet and Transport layers be considered
in unision and that critera such as RFC5033 be applied to the system
as a whole.

	This is not unique to L4S, SCE and even RFC3168 ECN is
in this same sitution.  CE is usless without the likes of ECE,
which doesnt exist until you couple the two layers.

	More in line.
Regards,
Rod
	
> > end-host algorithms (e.g. Prague).

A seperate document that sites the place that each RFC5033 critera is
addressed was offered as an option in Jake's request.

> 
> My current (individual) view is that the L4S experiment requires TCP Prague, so I view those two activities as inherently linked.  If anyone disagrees and wants to continue the discussion ;-), please identify another congestion-controlled transport protocol that meets the L4S low latency service requirements.

+1, there is no way to unlink them.  Some of the current issues in the tracker are probably transport issues already.

> In contrast, the SCE folks appear to have successfully modified a number of congestion-controlled transport protocols.

I find this wording important enough to repeat it "Congestion-controlled transport"i, for me that shows how tight the 2 layers are coupled when working in this area.

> > the L4S ID draft.? I think those are a start to requiring an end-host
> > Standard TCP, SCTP, and DCCP").? So, I'm not sure why those are a

-- 
Rod Grimes                                                 [email-masked]

---

From: "Holland, Jake" <[email-masked]>
Date: Tue, 12 Nov 2019 04:33:41 +0000

> concern (i.e. which L4S ID congestion response requirements do we have 

I don't want to get too bogged down on venue issues, and the point that
TCP Prague is out of scope for tsvwg is fair.

Maybe the issue that's appropriate in the tsv tracker would be something
more like "a reasonably complete candidate definition of at least one
proposed congestion control suitable for deployment on the internet that
makes use of the proposal", and details like asking for an RFC 5033-related
section could be handled separately in reviewing those docs?  After all,
I'm not sure there's much urgency on starting the experiment if there's not
a candidate endpoint system defined...

(This is exacerbated by the fact that DCTCP didn't do this, since it had
disclaimers like "only in controlled environments" and "documents what's
currently deployed", but now that it's proposed for the internet, it seems
to me the evaluation should be written somewhere in order to run this
experiment.)

Anyway, the motivation for the request was because it's hard to go through
the 5033 requirements and check them to decide how well they've been covered.
For example: the difficult environments considered (2) and the range of
environments investigated (3), performance with misbehaving nodes and
outside attackers (6), and responses to sudden or transient events (7).

I was thinking there have been a number of tests along these lines done as
sort of random spot checks, but it's hard to go through the list of tests
looking at responses to sudden or transient events to figure out whether a
case you're worried about is covered, and likewise with the others.

So I was asking for the authors to apply their uniquely wide knowledge of
the tests conducted toward categorizing the references in the way suggested
by RFC 5033, maybe in an appendix or a side doc, so they could be more
readily evaluated.

I agree some of the 5033 requirements are likely covered to some extent
already (#1 I agree seems reasonably well examined, as it's a consistent
theme in many of the reports, though it would still be useful to have a
list of references to areas where it's directly addressed), but I'm less
sure of several of the others.

Even a list of what 5033 requirements are covered and not covered by the
L4S requirements to set expectations on the ones that TCP Prague or other
L4S-compatible congestion controls will have to explicitly address would
be worthwhile, IMO.

Thanks and regards,
Jake

---

From: Wesley Eddy <[email-masked]>
Date: Mon, 11 Nov 2019 23:48:43 -0500

Thanks, you make good points.

I noticed that it was done very clearly for CUBIC in 
https://tools.ietf.org/html/rfc8312 which looks like a great recent 
example of how the 5033 topics were each addressed.

For Prague + L4S, I think we have a lot of material spread out in 
various places, and are pretty much covering all the bases to some 
extent, but lack a single place where it's all collected.

Here are my thoughts on what we have:

(1) Impact on Standard TCP

Parts of Appendix A and Section 4 of l4s-id draft are relevant, e.g. 
A.1.4 "Fall back to Reno-friendly congestion control on classic ECN 
bottlenecks"

Results w/ CUBIC + Prague in github/heistp/sce-l4s-bakeoff and 
l4s.cablelabs.com data are relevant.

Issue 16 results from l4s.cablelabs.com are relevant.

4.1.1 of the dualq doc is relevant.

(2) Difficult Environments

I'm not sure this is super-deeply investigated yet?

Some parts of the dualq draft touch on it, though since dualq is a 
construction where the actual AQM algorithm(s) can vary, it's maybe 
sufficient that those AQMs are suitable for the environments they're 
used in?

Scenario 5 in github/heistp/sce-l4s-bakeoff and l4s.cablelabs.com tests 
may be one data point, though I think the gist of 5033 is wider on 
wireless access, etc.

(3) Investigating a Range of Environments

Section 4 and A.1.6 ("Scaling down to fractional congestion windows") of 
the l4s-id doc partly speak to this.  A.2.2 ("Faster than Additive 
Increase") is also partly relevant.

The scenarios used for github/heistp/sce-l4s-bakeoff and 
l4s.cablelabs.com data have some datapoints and use different base 
delays, but I think the gist of this in 5033 is probably towards wider 
sweeping of the underlying network rates, numbers of flows, and other 
variables.

(4) Protection Against Congestion Collapse

A.1.3 ("Fall back to Reno-friendly congestion control on packet loss") 
pretty much covers this?

(5) Fairness within the Alternate Congestion Control Algorithm

Section 4 and A.1.5 ("Reduce RTT dependence") of l4s-id are relevant.  
Also A.2.3 ("Faster Convergence at Flow Start") is relevant.

The two-flow cases in github/heistp/sce-l4s-bakeoff and 
l4s.cablelabs.com data are a simple case of this (2 flow cases / 
prage-vs-prague).

(6) Performance with Misbehaving Nodes and Outside Attackers

Section 8 of l4s-arch + Section 4 of dualq and covers this?

Scenario 4 results from the github/heistp/sce-l4s-bakeoff and 
l4s.cablelabs.com tests have some relation to this also?

(7) Responses to Sudden or Transient Events

This could be related to parameter tuning of AQMs used in the dualq 
construction and to Prague, but I'm not sure it's applicable to the 
higher-level L4S architecture itself.

(8) Incremental Deployment

Section 6.3 of l4s-arch covers this?

---

From: "Holland, Jake" <[email-masked]>
Date: Tue, 12 Nov 2019 07:19:28 +0000

Thanks Wes, this is a helpful start.

I think the [DualQ-Test] reference from dualq also probably fits
in #2: difficult environments, with an examination of behavior in
the presence of fixed completely unresponsive UDP traffic at
various levels:
https://riteproject.files.wordpress.com/2018/07/thesis-henrste.pdf

What my request was about is that I suspect there's probably more,
it's just hard to comb through it all to see what's missing.

Best,
Jake

On 2019-11-11, 20:49, "Wesley Eddy" <[email-masked]> wrote:

---

From: Pete Heist <[email-masked]>
Date: Tue, 12 Nov 2019 18:14:19 +0100

Thanks, this looks like a good start on an application of 5033 to L4S. 

I wanted to back up and mention that what led me to RFC5033 was a reference to it in section 5 of draft-fairhurst-tsvwg-cc. I’d been looking for guidance on how to evaluate SCE. Even though it’s older, it resonated with me due to its emphasis on the need for scientific rigor when evaluating new CC proposals (thanks to the late Sally Floyd for that).

I admit I'll need to read and re-read some of the L4S literature referred to, as some of the below has likely been addressed in theory. Although there may be other theoretical questions, my interest is in ascertaining what if any more closed experiments are required before public experiments begin? I’ve put a few (*) next to what seems most important to me.

Comments below…

Pete

> On Nov 12, 2019, at 8:19 AM, Holland, Jake <[email-masked]> wrote:
> 

Here’s a non-exhaustive list of difficult environments and their characteristics mentioned or implied in 5033. The two *’d are very common and may present challenges.

- wireless and other bursty links (*)
- asymmetric links and delays (*)
- very high and very low speed links
- high delay links (e.g. GEO sat)
- high BDP links
- intermittent link connectivity

> sweeping of the underlying network rates, numbers of flows, and other 

Some other conditions:

- bi-directional traffic (*)
- compatibility with other existing RFC3168 AQMs, e.g. RED and PIE (CoDel / fq_codel we started on)
- many flows in a bottleneck
- packet loss, corruption and re-ordering
- misbehaving senders, receivers or routers (really (6))

> l4s.cablelabs.com tests have some relation to this also?

Scenario 4 covers the case of what happens if ECT(1) is set on all packets for a “classic” flow, and that seems to work as designed.

Could a flow instead set ECT(1) on only the “right” packets, like only so many per RTT, in order to gain some advantage?

> higher-level L4S architecture itself.

Might include:
- sudden congestion, capacity, and RTT changes (*)
- bottleneck shifts (*)
- multipath routing and routing changes

> Section 6.3 of l4s-arch covers this?

Compatibility with existing RFC3168 AQMs due to the redefinition of CE was one of our main concerns here. Some of its impacts on fairness and TCP RTT have been explored in the “bakeoff” tests.

There’s mention of it in l4s-arch 6.3.3 and it’s addressed further in the recent paper "TCP Prague Fall-back on Detection of a Classic ECN AQM”.