Subject: [tsvwg] L4S Issue #26: Q Protection: mandatory to Implement?

From: Bob Briscoe <[email-masked]>
Date: Wed, 28 Oct 2020 23:28:34 +0000

David,

You will recall that I and others have argued against QProt being 
mandatory to implement using arguments about incentives. That is still 
the main argument, which I can summarize if necessary, but that's not 
the purpose of this email.

Here. I want to add another argument that is about modularity. In the 
latest revision of the l4s architecture you will find seven examples 
listed of how queue protection could be implemented (eight if you count 
the null case of no protection):
https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-07#section-8.2
Only two have the queue protection integrated with the AQM. Five use 
distributed arrangements, with characterization of the traffic 
congestion located remote from the bottleneck queues that are being 
protected.

For a distributed solution, when a developer implements an L4S AQM in 
one box, it doesn't even make sense for the IETF to mandate that she 
must implement queue protection with it. In a distributed approach, 
queue protection would be an independent module running on a different box.

The only way that would make sense would be if the IETF was writing a 
normative standard for a complete traffic control system. Then we would 
be telling operators what to do, not implementers. System standards are 
usually the territory of the ITU, or the broadband forum, etc. The IETF 
is usually expected to specify component parts - protocols, algorithms - 
"Do one thing and do it well".

Regards

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 29 Oct 2020 16:50:44 +0000

Sebastian,

I've added tsvwg in cc, cos these questions are probably being asked by 
others, and the answers might help unravel some misconceptions that 
others might share. See [BB] inline...

On 29/10/2020 09:40, Sebastian Moeller wrote:
> Hi Bob,
>
> more below, prefixed [SM].
>   
>   
>   
>
> Gesendet: Donnerstag, 29. Oktober 2020 um 00:28 Uhr
> Von: "Bob Briscoe" <[email-masked]>
> An: "Black, David" <[email-masked]>
> Cc: "tsvwg IETF list" <[email-masked]>
> Betreff: [tsvwg] L4S Issue #26: Q Protection: mandatory to Implement?
> David,
>
>         [SM] in section 8.2 (https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-07#section-8.2) I see:
> "8.2.  'Latency Friendliness'
>
>     Like the Classic service, the L4S service relies on self-constraint -
>     limiting rate in response to congestion.  In addition, the L4S
>     service requires self-constraint in terms of limiting latency
>     (burstiness).  It is hoped that self-interest and guidance on dynamic
>     behaviour (especially flow start-up, which might need to be
>     standardized) will be sufficient to prevent transports from sending
>     excessive bursts of L4S traffic, given the application's own latency
>     will suffer most from such behaviour."
>
> How do you define and measure "suffering" that this assertion holds true?

[BB] First an intuitive explanation: If you are the one sending the most 
bursty traffic, your traffic will be perfectly correlated with the 
biggest bursts (because they are yours), and therefore a higher 
proportion of your traffic than anyone else's will experience the 
highest queuing delay.

I've written a deeper answer at the bottom, so as not to disturb the 
logical flow of this thread.

>
>
> "None of these possible queue protection capabilities are considered a
>     necessary part of the L4S architecture, which works without them (in
>     a similar way to how the Internet works without per-flow rate
>     policing).  Indeed, under normal circumstances, latency policers
>     would not intervene, and if operators found they were not necessary
>     they could disable them.  Part of the L4S experiment will be to see
>     whether such a function is necessary, and which arrangements are most
>     appropriate to the size of the problem."
>
> How is, "let's not mandate the implementation of one queue protection method in the L4S AQM nodes" incompatible with either of the proposed pie-in-the-sky alternatives or the experiment to test whether such a mechanism needs to be activated?

[BB] Perhaps it's so obvious that you're looking for something more 
complicated. Company X implements box x (an AQM). Company Y implements 
box y (e.g. an edge traffic conditioner). The IETF can't mandate that X 
can only implement x if it also modifies y, because X has no involvement 
in y. And anyway, the operator doesn't have to deploy x together with y, 
it can use x and z.

> If the outcome of the experiment is the queue protection is necessary, how are you proposing to change all the L4S AQM instances out in the field by then? If however we are exceedingly lucky and L4S provves to be robust and reliable in the field even with de-activated queue protection, it is going to be a piece of cake (comparatively) to simply drop thar queue protection requirement in any subsequent standards track RFC, no?

[BB] Implementers can decide it's better to provide queue protection 
with L4S (whether as separate distributed modules or integrated).
Operators can ask for queue protection when they deploy L4S.
But it would be infeasible for the IETF to tell X that, if it implements 
x, it MUST modify someone else's independent code (y).

Even with bandwidth sharing (where there /is/ an incentive for one 
sender to take more at the expense of others), per-flow rate policing is 
rarely implemented in the boxes that public ISPs buy, let alone enabled. 
I'm sure the IETF has never even considered making per-flow bandwidth 
policing mandatory to implement.

Bob

PS.
As promised above, here's a more precise definition of 'suffering' in 
the context of delay and congestion:

In the economic theory behind congestion control, the rate at which a 
user (or flow, or whatever) causes 'suffering' (formally termed the 
negative externality) is p*x, where p is the ECN marking probability and 
x is the bit-rate of the thing in question (user, flow, or whatever). 
Both these variables (p and x) can vary rapidly with time in a queue, 
which you would think would make their product difficult to measure. 
Nonetheless fortunately the product of their instantaneous values is the 
output of an AQM. It is the bit-rate of ECN-marked packets, which is 
called the congestion-bit-rate.

It's not just arbitrary to take the product p * x. You might ask, why 
not p^2 * x^3 or something? It has been proven theoretically that 
reflecting the 'cost' of this particular form of the externality (p * x) 
back on the user responsible for it would maximize social welfare across 
the whole network.

I give examples to explain this approach to 'scoring' the harm of 
different traffic patterns in the draft I wrote about queue protection, 
which you really ought to read if you've not come across this before, 
particularly Section 5 "Rationale":
https://tools.ietf.org/html/draft-briscoe-docsis-q-protection-00#section-5
I noticed a few disdainful comments from you about queue protection on 
this list, which I brushed aside. You are probably not aware that a 
number of the companies that implement the Internet's traffic management 
use this approach to 'score' the harm caused by traffic. It was 
originally proposed as congestion pricing, but I translated it into 
policing (in my PhD) given most people don't like the unpredictability 
of dynamic pricing.

>          [SM] That is not a rationale argument against implementation, but against mandating active queue protection on each L4S AQM node. These are clearly and logically two distinct things.
>
>          [SM] I would agree, if team L4S would have made its home-work and demonstrated that L4S will work robustly and reliably over the existing internet (instead of effectively repeating that one test "end-host to close by CDN" ad nauseam"). But we are talking about releasing an RFC in spite of unproven safety (and even core functionality) and in that case, requiring robust circuit-breaker seems like the least the IETF should do. In my opinion the L4S drafts should not be granted (experimental) RFC status before robust and reliable functionality and safety have been demonstrated, but I accept that the working group in general seems to be happy to act on L4S' promises alone rather in spite to the sorry picture that L4S gives in mildly adversarial testing. Queue protection will at least help to mediate some of L4S' inherent sharing issues (where observed behavior and promised behavior differ at the expense of non-L4S traffic) and hence requiring implementation should be completely undisputed, I can sort of understand that you do not want to mandate activating it (which would run a bit against the experimental goal to figure out whether it is needed or not).
>
>
>
>
> Regards
>
>
> Bob
>
>
> --
> Bob Briscoe http://bobbriscoe.net/[http://bobbriscoe.net/]
>

---

From: Jonathan Morton <[email-masked]>
Date: Thu, 29 Oct 2020 19:49:08 +0200

> On 29 Oct, 2020, at 6:50 pm, Bob Briscoe <[email-masked]> wrote:
> 
> [BB] Perhaps it's so obvious that you're looking for something more complicated. Company X implements box x (an AQM). Company Y implements box y (e.g. an edge traffic conditioner). The IETF can't mandate that X can only implement x if it also modifies y, because X has no involvement in y. And anyway, the operator doesn't have to deploy x together with y, it can use x and z.

In which case, what you should do is place a requirement on the network operator, to ensure that if an L4S AQM that does not include queue protection is deployed on their network, that a suitable queue protection is installed elsewhere on every relevant path.  Merely pointing to "oh no it might be possible to do this in two separate boxes" is not sufficient reason to remove the requirement entirely.

You can also specify the requirements of what the queue protection must do qualitatively, so long as a concrete implementation that actually works is provided as a reference.  This is well established in precedent, and leaves implementers the option of substituting equivalent or superior alternatives.

 - Jonathan Morton

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 29 Oct 2020 19:16:52 +0000

Jonathan,

> In which case, what you should do is place a requirement on the network operator, to ensure that if an L4S AQM that does not include queue protection is deployed on their network, that a suitable q…

[BB] It is a common misconception that RFCs are a way for engineers to 
tell companies what to do. They are not - that's what twitter's for. The 
reader of an RFC should always translate "X MUST" into "In order to 
claim compliance with this RFC, X MUST".

X doesn't have to do the MUST if X isn't claiming to comply with the 
RFC. Operators rarely, if ever, claim compliance with IETF RFCs. So 
mandating operators to do things in RFCs is generally viewed as vacuous.

> You can also specify the requirements of what the queue protection must do qualitatively, so long as a concrete implementation that actually works is provided as a reference.  This is well establis…

[BB] Specifying what QProt must do qualitatively would be fine. The 
QProt draft already specifies goals like this, before it specifies an 
actual implementation, with very detailed pseudocode that is very close 
to the code. And Rationale, etc. Have you read it?

Also, I wouldn't object at all to adding a sentence to the DualQ draft 
(e.g. under Security Considerations where it is already discussed), to 
mandate some form of overload protection, in order to be no worse than a 
FIFO, and to reference the algo in the DualQ appendix.

If that's what David meant, no problem. But we need to check he did.

For completeness, I'll say what we have documented:
* We have specified per-class overload protection in the DualQ draft 
(and in the reference Linux code):
https://tools.ietf.org/html/draft-ietf-tsvwg-aqm-dualq-coupled-12#section-4.1
https://tools.ietf.org/html/draft-ietf-tsvwg-aqm-dualq-coupled-12#appendix-A.2
* We have opened up the detailed spec. of the per-flow QProt algorithm 
that the DOCSIS cable industry has implemented.
    Annex P of the DOCSIS 3.1 spec referenced via 
https://tools.ietf.org/html/draft-briscoe-docsis-q-protection-00#ref-DOCSIS3.1
* And we have written up a more human-readable explanation of its 
rationale and pseudocode in an I-D in the independent stream:
    https://tools.ietf.org/html/draft-briscoe-docsis-q-protection-00
* There are also references for the more mature solutions in the list 
I've added in the l4s-arch draft.
https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-07#section-8.2

Bob

---

From: Sebastian Moeller <[email-masked]>
Date: Thu, 29 Oct 2020 20:59:04 +0100

Dear Bob,

more below in-line, prefixed [SM]

Gesendet: Donnerstag, 29. Oktober 2020 um 17:50 Uhr

An: "Sebastian Moeller" <[email-masked]>, "tsvwg IETF list" <[email-masked]>

Betreff: Re: Aw: [tsvwg] L4S Issue #26: Q Protection: mandatory to Implement?

Sebastian,

[SM] Thanks for putting it back on the list, less thanks for labeling my justified objections as misconceptions.

highest queuing delay.

[SM] Well, if my flow really wants low latency that might be a reasonable argument, but in L4S that less-responsive flow will also get the lions share of the bandwidth in the the L4S-queue, and if I understand the numbers coorrectly also absolutely. And at that point any fnot too latency sensitive flow will just win enough throughput t make it worth wile. Heck, if I can game my way into getting effectively all capacity for my flow, I could well implement a little FEC to paper over the expected packet-loss from occasionally triggering overload detection.

it can use x and z.

[SM] This makes no sense. If the IETF mandates that X needs to implement function CP as part of X, then surely Y can add CP as part of Y as well and an operator can, if needed decde wether to use CP functionality of x or y. But not mandating CP simlpe results in the operator not having a decent chance of having that option.

[SM] Well, that is not what is proposed, we really are talking about requiring X to implement CP so that the operator has CP available if he/she uses L4S, but that operator is still free to use CP at different points in his network, assuming equal performance. That choice is in no way reduced, by mandating that X needs to implement CP. There is a clear difference between mandatinf implementation, and mandating implementation and activation.

[SM] Im all for making the L4S AQM robust against abuse and catastrophic sharing failures by other means then. Feel free to propose something.

Bob

PS.

the whole network.

[SM] Could you please povide a link to an actual paper? Thanks.

of dynamic pricing.

--

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 29 Oct 2020 22:06:38 +0000

Sebastian,

On 29/10/2020 19:59, Sebastian Moeller wrote:
> *Gesendet:* Donnerstag, 29. Oktober 2020 um 17:50 Uhr
> *Von:* "Bob Briscoe" <[email-masked]>
> *An:* "Sebastian Moeller" <[email-masked]>, "tsvwg IETF list" 
> <[email-masked]>
> *Betreff:* Re: Aw: [tsvwg] L4S Issue #26: Q Protection: mandatory to 
> Implement?
> Sebastian,
>
>         [SM] Thanks for putting it back on the list, less thanks for 
> labeling my justified objections as misconceptions.
>
> > [SM] in section 8.2 
> (https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-07#section-8.2) 
> I see:
>         [SM] Well, if my flow really wants low latency that might be a 
> reasonable argument, but in L4S that less-responsive flow will also 
> get the lions share of the bandwidth in the the L4S-queue, and if I 
> understand the numbers coorrectly also absolutely. And at that point 
> any fnot too latency sensitive flow will just win enough throughput t 
> make it worth wile. Heck, if I can game my way into getting 
> effectively all capacity for my flow, I could well implement a little 
> FEC to paper over thtry to e expected packet-loss from occasionally 
> triggering overload detection.

[BB] When did anyone mention an unresponsive flow? You're obsessed with 
flow rate equality. You turn every conversation around to it.

In a FIFO queue, any flow can take the lion's share of the bandwidth. 
You could argue against nearly everything the IETF is doing on the basis 
that it wouldn't work if individual flows hogged FIFO capacity. This is 
not specific to L4S, and it's certainly not relevant to this thread.

The thesis referenced in the DualQ draft [Destruction testing...] tried 
to exploit the DualQ to get a greater bandwidth advantage than a FIFO, 
but couldn't. If you have experiments that show otherwise, let's see 
them. Otherwise, stay silent please.

We are trying to work on this ML.

> > How is, "let's not mandate the implementation of one queue 
> protection method in the L4S AQM nodes" incompatible with either of 
> the proposed pie-in-the-sky alternatives or the experiment to test 
> whether such a mechanism needs to be activated?
>
>         [SM] This makes no sense. If the IETF mandates that X needs to 
> implement function CP as part of X, then surely Y can add CP as part 
> of Y as well and an operator can, if needed decde wether to use CP 
> functionality of x or y. But not mandating CP simlpe results in the 
> operator not having a decent chance of having that option.

[BB] I am not going to explain it a third time. Jonathan clearly 
understood the argument quite easily. Please read the original 
explanation (which also requires that you read the relevant section in 
L4S-arch).

>
> > If the outcome of the experiment is the queue protection is 
> necessary, how are you proposing to change all the L4S AQM instances 
> out in the field by then? If however we are exceedingly lucky and L4S 
> provves to be robust and reliable in the field even with de-activated 
> queue protection, it is going to be a piece of cake (comparatively) to 
> simply drop thar queue protection requirement in any subsequent 
> standards track RFC, no?
>
>         [SM] Well, that is not what is proposed, we really are talking 
> about requiring X to implement CP so that the operator has CP 
> available if he/she uses L4S, but that operator is still free to use 
> CP at different points in his network, assuming equal performance. 
> That choice is in no way reduced, by mandating that X needs to 
> implement CP. There is a clear difference between mandatinf 
> implementation, and mandating implementation and activation.

[BB] I am not going to explain it a third time. Jonathan clearly 
understood the argument quite easily. Please read the original 
explanation (which also requires that you read the relevant section in 
L4S-arch).

>         [SM] Im all for making the L4S AQM robust against abuse and 
> catastrophic sharing failures by other means then. Feel free to 
> propose something.

[BB] Not an L4S-specific problem.

Bob

> > [SM] That is not a rationale argument against implementation, but 
> against mandating active queue protection on each L4S AQM node. These 
> are clearly and logically two distinct things.
> >
> > [SM] I would agree, if team L4S would have made its home-work and 
> demonstrated that L4S will work robustly and reliably over the 
> existing internet (instead of effectively repeating that one test 
> "end-host to close by CDN" ad nauseam"). But we are talking about 
> releasing an RFC in spite of unproven safety (and even core 
> functionality) and in that case, requiring robust circuit-breaker 
> seems like the least the IETF should do. In my opinion the L4S drafts 
> should not be granted (experimental) RFC status before robust and 
> reliable functionality and safety have been demonstrated, but I accept 
> that the working group in general seems to be happy to act on L4S' 
> promises alone rather in spite to the sorry picture that L4S gives in 
> mildly adversarial testing. Queue protection will at least help to 
> mediate some of L4S' inherent sharing issues (where observed behavior 
> and promised behavior differ at the expense of non-L4S traffic) and 
> hence requiring implementation should be completely undisputed, I can 
> sort of understand that you do not want to mandate activating it 
> (which would run a bit against the experimental goal to figure out 
> whether it is needed or not).
> >
> >
> >
> >
> > Regards
> >
> >
> > Bob
> >
> >
> > --

---

From: Sebastian Moeller <[email-masked]>
Date: Fri, 30 Oct 2020 09:43:49 +0100

Dear Bob,

more below, prefixed [SM2]

Gesendet: Donnerstag, 29. Oktober 2020 um 23:06 Uhr

An: "Sebastian Moeller" <[email-masked]>

Betreff: Re: Aw: Re: [tsvwg] L4S Issue #26: Q Protection: mandatory to Implement?

Sebastian,

Dear Bob,

Sebastian,

highest queuing delay.

[SM] Well, if my flow really wants low latency that might be a reasonable argument, but in L4S that less-responsive flow will also get the lions share of the bandwidth in the the L4S-queue, and if I understand the numbers coorrectly also absolutely. And at that point any fnot too latency sensitive flow will just win enough throughput t make it worth wile. Heck, if I can game my way into getting effectively all capacity for my flow, I could well implement a little FEC to paper over thtry to e expected packet-loss from occasionally triggering overload detection.

[BB] When did anyone mention an unresponsive flow? You're obsessed with flow rate equality. You turn every conversation around to it.

[SM2] Please keep it civil, I have no obsession, and would apprerciate if you would not characterize my objections with such derogatory terms, thanks on advance. I bring thjis as an argument to highlight one of L4S/dual queue coupled AQM's design short comings, exactly that short coming that queue protection is designed to tackle. Whether you characterize the issue as under-responsive ECT(1) flows destroy low latency in the L4S-shallow queue, or as under-responsive ECT(1) flows gain a signifivcant throughput advantage at the expense of other L4S-shappow queue flows and even more at the expense of L4S-deeper queue flows, is a matter od preferrence. IMHO both effects are observable, and for them to materialize I assume a flow does not need to be fully non-responsive, but I predict a flow that is significantly less responsive (and tolerant against some loss) will suffice. Such a flow is not an unlikely unicorn, but e.g. adaptive CBR flws might qualify if their reaction time is considerably > than their path RTT. (Which is one more reason, why I want to see real worls test data, showing how L4S sharing behaves when pitting say short RTT flows against a trans-atlantic adaptive CBR flow, but I digress).

[SM2] Which is true, but the goal is (at least roughly) equitable sharing and that no flow gets starved selectively (that is if the aggregate bandwidth/number of cooncurrent flows only leaves a trickle than all flows effectively are starving, but that coerner case, I believe is accepted).

You could argue against nearly everything the IETF is doing on the basis that it wouldn't work if individual flows hogged FIFO capacity. This is not specific to L4S, and it's certainly not relevant to this thread.

[SM2] The point is, L4S's design and implementaion will allow to make matters worse, and I would expect at least a "no regression" criterion to be applied.

The thesis referenced in the DualQ draft [Destruction testing...] tried to exploit the DualQ to get a greater bandwidth advantage than a FIFO, but couldn't. If you have experiments that show otherwise, let's see them. Otherwise, stay silent please.

[SM2] Destruction testing, managed to only test the same subset of condirtions as most of your other tests, thereby demonstrating that thee is a set of conditions under which L4S behaves roughly as advertized. Extrapolating from that, that L4S will behave as it promises under all realistic cionditions requires a leap of faith I am not willing to take.

But more to the point, in the thesis the relevant traffic id the UDP load generator, which as can be seen on e.g. figues 12.6-12.9 will always get the lion share of the traffic (but the UDPload was either applied to one of the queues, as so often a relevabt trst, UDP loads in both queues has been mitted), BUT more to the point for quite a number of conditions (12.6, 12.8) the aggregate throughput of the L4S flows exceeds the thoughput of the non-L4S flows by several orders of magnitude. As so often, your interpretaton of xyour own data seems driven by the results you want to see, and not by what is in the data...

And I believe to be the task for the designers of L4S to demonstrate that it behaves at leat acceptably under all conditions, until that is done, ecuse me just brushing off that as an unfriendly attempt at putting your work on my plate, thanks, but no thanks.

[SM2] Could you elaborate, please? What does ML stand for here, mailing list, machine learning, ...?

it can use x and z.

[BB] I am not going to explain it a third time. Jonathan clearly understood the argument quite easily. Please read the original explanation (which also requires that you read the relevant section in L4S-arch).

[SM2] Just because you repeat an assertion several times it does not get any more true or relevant. But humor me, we agree that the IETF will not be able to mandate anything regarding L4S for companies and products not implementing L4S in the first place, the logical conclusion then is, if the IETFs position should come down to that the L4S experiment requires CP to be available, it MUS-t mandate CP to be available in the node implementing the L4S AQM as that surely must be covered by the experimental RFCs.

[BB] I am not going to explain it a third time. Jonathan clearly understood the argument quite easily. Please read the original explanation (which also requires that you read the relevant section in L4S-arch).

[SM2] Not sue that the best response to being shown some piece of questionable logic in one's argument is to give out home-work to the person noting the ligical error.

[SM2] I disagree, L4S makes this considerably worse, and hence needs to try to ameliorate its hopefully undesired side-effects.

Sebastian

Bob

Bob

PS.

the whole network.

of dynamic pricing.

--