Subject: [tsvwg] Explaining my ect(1) as input to network position

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Tue, 12 May 2020 11:18:43 +0000

Hi all,

A short version of the points I want to make related to the safety issues:

  *   Classic RFC3168 unfairness can be easily mitigated/fixed in the network if it would cause problems, preferably by reusing and reconfiguring exactly the RFC3168 functionality some people think L4S will make useless. It will become on the contrary a driver for better support for L4S.
  *   Vulnerability to non-responsiveness flows exists today in SingleQ AQMs and this is accepted. DualQ behaves exactly the same without benefitting L4S. Like other AQMs, it provides no solution to "save" responsive flows from non-responsive ones, as this is not a requirement for an AQM. So it can't be claimed this is a safety issue of L4S/DualQ.

The big advantage of ECT(1) as the input to the network is that the new traffic can be identified very easily by the network (IP-header only). This is also a big advantage for potential safety issues wrt Classic RFC3168 ECN bottlenecks. A network operator can indeed bleach ECT(1) to prevent the new behavior on its Classic ECN RFC3168 nodes if it would be needed, but even better, an operator can make use of these Classic ECN nodes to support L4S. The identification can be used to separate the 2 traffic types in different queues (or even links/paths), with each its dedicated Classic ECN configuration (a larger target with smoothed response for Classic and a shallow immediate threshold for L4S). The queues can be scheduled with WRR of for example 50/50 for each traffic type. This has been described in early papers for DCTCP deployments in DataCenters. It is actually a NON-Coupled DualQ. This config can be easily configured on existing equipment. It won't have all Coupled features, but is a good starting point, and preserves the low latency for L4S.

I noticed that there are quite some "safety due to non-responsive TCP"-concerns raised again which were solved already. We have shown in the past that the Coupled DualQ responds to non-responsive traffic as a normal Single Q Classic bottleneck (with or without ECN). No fast-lane for L4S non-responsive traffic, only lower latency. Try sending 3 non-responsive UDP/CBR flows using each one of the non-ECT, ECT(0) and ECT(1) with each a sending rate of 100% link capacity into a DualQ. They will each get 33% of the link capacity, while pushing away all TCP traffic (no matter which ECT they are). Note that cumulative, L4S gets even only 33%, while Classic gets 66% capacity in this case! It shows again that the coupled AQM disables the scheduling and fully overrules its claimed "priority"-behavior. This is a very unintuitive concept of DualQ, that seems to be easily overlooked. Depending on the overload strategy, the Classic flows can experience 15ms extra latency compared to the ECT(1), but there won't be throughput benefit. Do the same experiment on a correctly designed Classic ECN bottleneck and the behavior will be exactly the same (except for the latency benefit for ECT(1)). This sensitivity to non-responsiveness has been always the case, even now on the Internet you all are using. It is typically constrained within a single subscriber/user by per user scheduling.

So concluding again, related to the safety-issues:

  *   Classic RFC3168 unfairness can be easily mitigated/fixed in the network if it would cause problems, preferably by reusing and reconfiguring exactly the RFC3168 functionality some people think L4S will make useless. It will become on the contrary a driver for better support for L4S.
  *   Vulnerability to non-responsiveness flows exists today in SingleQ AQMs and this is accepted. DualQ behaves exactly the same without benefitting L4S. Like other AQMs, it provides no solution to "save" responsive flows from non-responsive ones, as this is not a requirement for an AQM. So it can't be claimed this is a safety issue of L4S/DualQ.

So let's go for the forward looking solution with the best opportunity for success...

Koen.

---

From: Sebastian Moeller <[email-masked]>
Date: Wed, 13 May 2020 09:32:00 +0200

Hi Koen,

I thought I would let this slide, but on second thought this raises more questions than it answers, so more below in-line.

> On May 12, 2020, at 13:18, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
> [...]
>  
> I noticed that there are quite some “safety due to non-responsive TCP”-concerns raised again which were solved already. We have shown in the past that the Coupled DualQ responds to non-responsive traffic as a normal Single Q Classic bottleneck (with or without ECN). No fast-lane for L4S non-responsive traffic, only lower latency. Try sending 3 non-responsive UDP/CBR flows using each one of the non-ECT, ECT(0) and ECT(1) with each a sending rate of 100% link capacity into a DualQ. They will each get 33% of the link capacity, while pushing away all TCP traffic (no matter which ECT they are). Note that cumulative, L4S gets even only 33%, while Classic gets 66% capacity in this case! It shows again that the coupled AQM disables the scheduling and fully overrules its claimed “priority”-behavior. This is a very unintuitive concept of DualQ, that seems to be easily overlooked. Depending on the overload strategy, the Classic flows can experience 15ms extra latency compared to the ECT(1), but there won’t be throughput benefit. Do the same experiment on a correctly designed Classic ECN bottleneck and the behavior will be exactly the same (except for the latency benefit for ECT(1)). This sensitivity to non-responsiveness has been always the case, even now on the Internet you all are using. It is typically constrained within a single subscriber/user by per user scheduling.
> [...]

	[SM] I find this argument to be questionable on two accounts. Let me break down my issues:

1) Is this actually relevant? You are basically demonstrating fair sharing between multiple DOS streams. I severely doubt that anybody is considering that a priority goal for achieving fairness at. Just consider that all of your streams loose 66% of their packets, so it is quite doubtful that if those are truly data streams the user is going to be happy with the result.
	But what I really would like to see, is what happens if you add an additional ECT((0)-CUBIC and EXT(1)-TCP Prague stream to the mix. If these streams survive passage though your DOSed gateway, you might have more than a purely academic point. But as you concede TCPs will be severely impacted. 
	As an end-user this is exactly the situation where I would wish my upstream AQM to be flow-fair as that has a much higher probability of still delivering reasonable throughput for responsive flows. And yes, this can be fooled by spreading the DOS traffic over multiple 5-tuples, but that same spreading will also ruin the normal dualQ AQM and will also out-smart your proposed queue-protection design so L4S does just as bad as FQ in that situation and worse in the non-spoofed-sprayed context.
	Worse an enduser, on ingress, I will not be able to fix this problem post-hoc, as the non-responsive flows make L4S already evict the TCP streams and no amount of FQing on my side of the bottleneck can bring these already dropped TCP packets back. 

2) You seem to be relaying on the overload protection here, and the fact that your streams are both unresponsive and CBR (aka paced), if I understand the dual queue coupled AQM draft correctly your AQM basically regresses to being a computationally expensive head-drop system under those circumstances, and the fairness you describe above is a direct consequence if your three DOS sources using the same fixed data rates with an equal probability dropping AQM.

Best Regards
	Sebastian

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 13 May 2020 10:07:18 +0000

Hi Sebastian,

1) I'm not claiming any extra nor less DualQ AQM features related to non-responsive flows than there are on single Q AQMs. The point is it works exactly as a single Q AQM, because of the equal drop to all packets mechanism when the classic probability is higher than a certain threshold. This is a sufficient requirement for any ECN-based AQM to protect it from giving ECT flows any benefits. Such an overall equal distributed drop complete overrules any scheduling (even strict priority). 

On the other hand, I'm not sure if all RFC AQMs have this ECN to drop protection described, and those have indeed potential safety issues. A review and correction on those RFCs would be useful.

2) This mechanism is independent from head- or taildrop. DualQ does not rely on headdrop. The appendices are just examples of possible implementations. You should be free to improve/rearrange and optimize to the conditions and constraints of a specific network node. 

In general we should compare apples with apples. DualQ is the basic AQM, while it is being compared with so called "AQMs" enhanced with flow schedulers and mechanisms doing prioritization of small packets, known to help for instance with bidirectional greedy downloads. Nothing stops you from also using this small packet prioritization in a DualQ implementation (I have no opinion if this should be a requirement, but could be a useful discussion). Taking it further L4S also supports FQ, besides the 2 class aggregated flows treatment, but at least it can do without FQ where FQ is not possible or preferred to be used (for whatever reason). We should evaluate the different features separately and network developers can freely pick their favorite tweaks to define the optimal solution for the case at hand...

If there is consensus in this WG and the rest of the IETF that FQ with small packet prioritization are MUST-haves in future AQMs, then we should immediately discuss about that (do BoF, change the charter, ...). Then we can deprecate all aggregate-Q mechanisms and this discussion can stop. If not, then let's keep things separated, or it will become very confusing for people not deeply involved to understand what's going on, and any further useful TSVWG/AQM work will be blocked forever (it seems)...

Koen.

-----Original Message-----
From: Sebastian Moeller <[email-masked]> 
Sent: Wednesday, May 13, 2020 9:32 AM
To: De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]>
Cc: [email-masked]
Subject: Re: [tsvwg] Explaining my ect(1) as input to network position

Hi Koen,

Best Regards
	Sebastian

---

From: Sebastian Moeller <[email-masked]>
Date: Sat, 16 May 2020 23:07:42 +0200

Hi Koen,

> On May 13, 2020, at 12:07, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 
> Hi Sebastian,
> 
> 1) I'm not claiming any extra nor less DualQ AQM features related to non-responsive flows than there are on single Q AQMs.

	[SM] And I am reporting that for end-users that is too little to offer. You brought an example that vividly demonstrates the short-comings of single-queue AQMs with describing your overload protection scheme.

> The point is it works exactly as a single Q AQM,

	[SM] Sure, but that is not good enough. You need to balance the considerable cost of L4S by some benefits, if all you do is be as good as existing tech, then why should anybody use yours?

> because of the equal drop to all packets mechanism when the classic probability is higher than a certain threshold. This is a sufficient requirement for any ECN-based AQM to protect it from giving ECT flows any benefits.

	[SM] Pardon me? Of course do ECN-based AQM give benefits to ECT-flows, that is their whole reason to exist... Not dropping but sending CE is an advantage. Not doing so under severe loads but go to drops is essentially unavoidable and also covered in rfc3168, IIRC.

> Such an overall equal distributed drop complete overrules any scheduling (even strict priority). 

	[SM] Again, pardon me? Why should a strictly tiered system drop everything just because some tiers encounter an overload? I must be missing something here, what?

> On the other hand, I'm not sure if all RFC AQMs have this ECN to drop protection described, and those have indeed potential safety issues. A review and correction on those RFCs would be useful.

	[SM] How is that relevant for the discussion at hand?

My point is that with your drafts promising:

https://tools.ietf.org/html/draft-ietf-tsvwg-aqm-dualq-coupled-11:
"L4S enables both ultra-low queuing latency (sub-millisecond on
   average) and high throughput at the same time, for ad hoc numbers of
   capacity-seeking applications all sharing the same capacity."

https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-l4s-arch-06:
"This coupling of the congestion signaling between the two
       queues makes the L4S flows slow down to leave the right amount of
       capacity for the Classic traffic (as they would if they were the
       same type of traffic sharing the same queue)"

These clearly imply a goal of equitable sharing between flows, and we have no seen considerable number of cases where your implementation fails to meet that promise including your DOS overload example (where the TCPs are crowded out by the three attack flows, which as you helpfully point out share the bottleneck ~equal amongst themselves). These are IMHO excellent demonstrations why equitable sharing is a sane default to aim for (in your examples the TCP flows would still work to a degree).

> 2) This mechanism is independent from head- or taildrop. DualQ does not rely on headdrop. The appendices are just examples of possible implementations. You should be free to improve/rearrange and o…

	[SM] I agree that it does not really matter that much at which end of the queue dropping happens, but I went back to the dual queue coupled AQM draft to describe your design correctly.

> 
> In general we should compare apples with apples.

	[SM] Well, this a rather peculiar thing to say, because it is only comparison that let's one appreciate the differences between say apples and bananas, but I digress. 

> DualQ is the basic AQM,

	[SM] "The" basic AQM? I would have thought that honor belongs to AQMs that came long before the dual queue coupled AQM, no?

> while it is being compared with so called "AQMs" enhanced with flow schedulers and mechanisms doing prioritization of small packets, known to help for instance with bidirectional greedy downloads.

	[SQM] Why do you call those "AQMs" in quotes? Do you consider these techniques to be verboten for a proper AQM? The point is, your clever "coupling" is also just a heuristic for something, I just happen to think that it is too much of a fair-whether heuristic with insufficiently explored performance under adversarial traffic patterns. 
In regards to my claim of "fair-whether-ness", I note that Destruction Testing: Ultra-Low Delay using Dual Queue Coupled Active Queue Management explicitly remarks "All though overload can be seen similar to a malicious user, such a user is probably able to cause a higher degree of overload and unstability in the system, than what I have investigated. Security concerns such as this has not been investigated." Could you please point me to where you present data for performance under adversarial traffic patterns, please?

> Nothing stops you from also using this small packet prioritization in a DualQ implementation (I have no opinion if this should be a requirement, but could be a useful discussion).
> Taking it further L4S also supports FQ, besides the 2 class aggregated flows treatment, but at least it can do without FQ where FQ is not possible or preferred to be used (for whatever reason).

	[SM] I has become pretty clear that the "can do without" claim is not generally accepted and also not properly demonstrated by your testing data... 

> We should evaluate the different features separately and network developers can freely pick their favorite tweaks to define the optimal solution for the case at hand...

> If there is consensus in this WG and the rest of the IETF that FQ with small packet prioritization are MUST-haves in future AQMs,
> then we should immediately discuss about that (do BoF, change the charter, ...). Then we can deprecate all aggregate-Q mechanisms and this discussion can stop. If not, then let's keep things separated, or it will become very confusing for people not deeply involved to understand what's going on, and any further useful TSVWG/AQM work will be blocked forever (it seems)...

	[SM] Koen, you do realize that this paragraph is a nice as a polemic but but less impressive as a reasoned argument?