Subject: [tsvwg] dave taht's rationale for option 3 on ect(1) (more testing)

From: Dave Taht <[email-masked]>
Date: Sun, 17 May 2020 11:14:42 -0700

The codebases, simulations, etc, for the l4s and SCE approaches
are still in a very raw state. With these early implementations,
many problems have been identified, and only a few resolved. The
internet drafts and codebases are out of alignment, with new
bugs landing in the codebases and operating systems every time
someone takes them out for a spin.

Nearly all the tests bufferbloat.net considered basic in the
development of the original pie,
codel, and fq_codel aqms in the aqm working group as of 2004 have yet to be run.

Participants may weight the results differently, but adversarial tests
of all kinds
need to be run, regularly, in response to every change to the
specification and code,
in order to retain an accurate picture of the problems found, fixed, or ignored.

(I confess that much of my interest in lossless congestion control
comes from longer
 RTT near earth orbit, geo, and gto - and even more distant space-born
applications,
 which is a from-orbit viewpoint I suspect few here share)

To the greatest extent possible, tests to be performed in simulation
AND on a variety
of real hardware, AND in virtualized environments.

The recent bugs found in encapsulations need to be evaluated against more tunnel
types and operating systems. More shipping switches and hardware based
need to be tested and configured using ect(1) -> CE transitions, in
particular, to
see what happens.

Problems with legacy uses of the IP stack, for private and industrial
and embedded
r/t control use, have not been explored either. Examples include 6lowpan, and
other relatively unknown services in the ietf you can find in all the
other ethernet
packet types known to man.

The potential side effects on classic traffic, have been barely explored,
if at all. The impact on conventional udp and classic gaming, voice,
and DNS traffic
of the pie side of the l4s solution, (with the initial exploration of
the rrul_be test
showing up bugs galore), is largely unknown.

The true effects on the internet of senders such as the increasingly
available BBRv1 with RFC3168 ecn accidentally enabled deployment
needs analysis, as do the effects of a modestly or truly lying senders.

The costs and methods required to mitigate ECN-enabled DDOS attacks, also
is absent from analysis.

The side effects of essentially deprioritizing classic routing
protocols and other essential oam and maintenance packet types
(such as rip v2 (used on cable), ospf, Babel, ISIS, RA, ND and ARP),
needs to be explored.

The effects on existing deployed RFC3168 compliant multicast file
transports such as
uftp[1], needs to be tested.

Recognising “loss and mark” as an even stronger signal than either,
not thought about.
There's a long list of other more trivial things - dynamically
reducing mss size at cwnd 2 or less,
improving packet pacing to cope with sub cwnds, more fully exploring l4s-style
behaviors on unreliable, jittery networks such as wifi and lte, and chirping.

In short:

I see no reason for the IVTF to allocate the ECT(1) codepoint at this
time without
a ton more simulation and testing.

With a lot more robust, repeatable testing, good software engineering practices,
complete with regression tests, and continuous integration, and source code that
matches the internet drafts (and vice versa), perhaps we can all find
a way forward
closer to the goals of low latency and high throughput and the most
viable, safest,
use for this last half a bit in the IP header.

-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

[email-masked] <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

---

From: Pete Heist <[email-masked]>
Date: Mon, 18 May 2020 10:30:11 +0200

Thanks for taking the time to explain your rationale. I’ll address some of it from an SCE standpoint.

> On May 17, 2020, at 8:14 PM, Dave Taht <[email-masked]> wrote:
> 

Any issues found may be submitted at our GitHub repo: https://github.com/chromi/sce

Overall, we are comfortable that our reno-sce and dctcp-sce CC algos are in good shape. cubic-sce has a subtle issue in its response that needs working out, although major problems have been addressed. Much of the future development work is anticipated to be in the middle.

> codel, and fq_codel aqms in the aqm working group as of 2004 have yet to be run.

Our current SCE algos are variants of Reno, DCTCP and CUBIC that have the addition of the linear backoff needed to do HFCC. Therefore, we feel it largely unnecessary to run in-depth tests of the SCE CC algos against these qdiscs, as they should perform nearly identically to the originals, although broad coverage as verification is probably worthwhile. If and when a new class of SCE CC algo is added, more tests like these would be needed.

> need to be tested and configured using ect(1) -> CE transitions, in

All of our prior testing was done on VMs on a physical network, with netns support added in the latest batch, and the ability to run the same tests across both the netns and physical environments. With physical networks, one can see differences in behavior crop up that don’t appear in the idealized netns environment, so this needs more exploration.

Based on recent conversations, a more in-depth exploration of tunnel behavior has been moved up our list.

> showing up bugs galore), is largely unknown.

The SCE design presents no risk to non-SCE traffic, however, any new AQM code will need to be evaluated as would any other.

> needs analysis, as do the effects of a modestly or truly lying senders.

BBRv1’s unresponsiveness to RFC3168’s ECN signals will need to be addressed in our AQMs. We would also like to see the addition of ECN support to BBRv1, so that when it shares a queue with ECN capable flows, either in a single queue or by FQ hash collision, it does not dominate competing ECN capable flows.

> The costs and methods required to mitigate ECN-enabled DDOS attacks, also

We feel this is important any time a short queue is created to isolate HFCC or low latency flows, because this short queue effectively creates a new class of service, and thus a DoS target.

We view an attack on a FIFO with unresponsive traffic differently, because there is no differentiated service available, although that type of attack should also be mitigated over time with the further deployment of AQM (RFC 7567).

> (such as rip v2 (used on cable), ospf, Babel, ISIS, RA, ND and ARP),

We don’t anticipate major problems here with the SCE design.

However, we did have one potential issue raised by a member of the cooperative ISP I belong to who has done some work within the IETF. He is concerned that sparse flow optimization may fool some routing protocols into making the wrong decisions, when they actively measure RTT/loss over routes in order to detect and respond to congestion. That’s an interesting perspective that applies to any AQM that does sparse flow optimization. However, I would point out that those protocols could be modified to additionally monitor the ECN field as an indication of congestion. To that end, having an ECN field with unambiguous signaling would be very important.

> behaviors on unreliable, jittery networks such as wifi and lte, and chirping.

The right marking strategies and parameters need to be chosen for dealing with bursty and jittery traffic. It’s not just wifi, lte, or other wireless tech that introduces burstiness and jitter, but simple cross-flow traffic as well. This will be a reality on the open Internet for the foreseeable future.

---

From: Paul Vixie <[email-masked]>
Date: Mon, 18 May 2020 08:57:55 +0000

On Monday, 18 May 2020 08:30:11 UTC Pete Heist wrote:
> ...
> 
> Overall, we are comfortable that our reno-sce and dctcp-sce CC algos are in
> good shape. ...

i remember thinking the same about TCP itself, until 1988 and "slow start". 
can we be humble as to our comfort levels yet?

-- 
Paul

---

From: Pete Heist <[email-masked]>
Date: Mon, 18 May 2020 11:32:01 +0200

> On May 18, 2020, at 10:57 AM, Paul Vixie <[email-masked]> wrote:
> 

If any overconfidence was projected, I rescind that sentiment. Our efforts to find problems with those implementations have not yet been successful, so we should be open to new ways to do so.