Subject: Re: [tsvwg] New Version Notification for draft-white-tsvwg-lld-00.txt

From: Greg White <[email-masked]>
Date: Tue, 12 Mar 2019 02:18:01 +0000

Hi Luca, 

You can find the details in Annex P of the DOCSIS MULPIv3.1 spec.
https://specification-search.cablelabs.com/CM-SP-MULPIv3.1

-Greg

From: Luca Muscariello <[email-masked]>
Date: Monday, March 11, 2019 at 6:26 PM
To: Greg White <[email-masked]>
Cc: Dave Taht <[email-masked]>, Jonathan Morton <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [tsvwg] New Version Notification for draft-morton-taht-tsvwg-sce-00.txt

Hi Greg,

I'm curious about the queue protection function in the LLD document.
It seems to assume that a flow table is maintained to determine if a flow 
has the right to enter the low latency queue.

Can you give more details about that component? Or point me to a reference?

Thanks
Luca

On 3/11/19, 5:29 PM, "tsvwg on behalf of Greg White" <[email-masked] on behalf of [email-masked]> wrote:

    TSVWG,
    
    I've posted a new informative draft that gives an overview of the new Low Latency DOCSIS specification (see links below).  This overview may be interesting to TSVWG participants because it includes support for L4S (https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-aqm-dualq-coupled) and for the NQB PHB (https://datatracker.ietf.org/doc/html/draft-white-tsvwg-nqb).
    
    Best Regards,
    Greg

    On 3/11/19, 5:05 PM, "[email-masked]" <[email-masked]> wrote:

        A new version of I-D, draft-white-tsvwg-lld-00.txt
        has been successfully submitted by Greg White and posted to the
        IETF repository.
        
        Name:		draft-white-tsvwg-lld
        Revision:	00
        Title:		Low Latency DOCSIS - Technology Overview
        Document date:	2019-03-11
        Group:		Individual Submission
        Pages:		25
        URL:            https://www.ietf.org/internet-drafts/draft-white-tsvwg-lld-00.txt 
        Status:         https://datatracker.ietf.org/doc/draft-white-tsvwg-lld/ 
        Htmlized:       https://tools.ietf.org/html/draft-white-tsvwg-lld-00 
        Htmlized:       https://datatracker.ietf.org/doc/html/draft-white-tsvwg-lld 

        Abstract:
           NOTE: This document is a reformatted version of [LLD-white-paper].
        
           The evolution of the bandwidth capabilities - from kilobits per
           second to gigabits - across generations of DOCSIS cable broadband
           technology has paved the way for the applications that today form our
           digital lives.  Along with increased bandwidth, or "speed", the
           latency performance of DOCSIS technology has also improved in recent
           years.  Although it often gets less attention, latency performance
           contributes as much or more to the broadband experience and the
           feasibility of future applications as does speed.
        
           Low Latency DOCSIS technology (LLD) is a specification developed by
           CableLabs in collaboration with DOCSIS vendors and cable operators
           that tackles the two main causes of latency in the network: queuing
           delay and media acquisition delay.  LLD introduces an approach
           wherein data traffic from applications that aren't causing latency
           can take a different logical path through the DOCSIS network without
           getting hung up behind data from applications that are causing
           latency, as is the case in today's Internet architectures.  This
           mechanism doesn't interfere with the way applications share the total
           bandwidth of the connection, and it doesn't reduce one application's
           latency at the expense of others.  In addition, LLD improves the
           DOCSIS upstream media acquisition delay with a faster request-grant
           loop and a new proactive scheduling mechanism.  LLD makes the
           internet experience better for latency sensitive applications without
           any negative impact on other applications.
        
           The latest generation of DOCSIS equipment that has been deployed in
           the field - DOCSIS 3.1 - experiences typical latency performance of
           around 10 milliseconds (ms) on the Access Network link.  However,
           under heavy load, the link can experience delay spikes of 100 ms or
           more.  LLD systems can deliver a consistent 1 ms delay on the DOCSIS
           network for traffic that isn't causing latency, imperceptible for
           nearly all applications.  The experience will be more consistent with
           much smaller delay variation.
        
           LLD can be deployed by field-upgrading DOCSIS 3.1 cable modem and
           cable modem termination system devices with new software.  The
           technology includes tools that enable automatic provisioning of these
           new services, and it also introduces new tools to report statistics
           of latency performance to the operator.
        
           Cable operators, DOCSIS equipment manufacturers, and application
           providers will all have to act in order to take advantage of LLD.
           This white paper explains the technology and describes the role that
           each of these parties plays in making LLD a reality.

        
        Please note that it may take a couple of minutes from the time of submission
        until the htmlized version and diff are available at tools.ietf.org.
        
        The IETF Secretariat

---

From: Luca Muscariello <[email-masked]>
Date: Wed, 13 Mar 2019 13:37:36 -0700

Greg,

Thanks for the reference.
I have a few questions. I read the document just once so, don't be
surprised if I got something wrong.

1) A packet enters the low latency queue if it is marked. Who is
responsible for the marking?
Is the original application to do that?

2) Once a packet enters the low latency queue, the flow, or microflow,
is tracked in a (micro)flow table by reporting a function of the queue
occupancy.  How do updates of the table
take place? Insert is clear unless the flow is banned, update is clear.
Remove is only clear to me when the flow is moved
out of the low latency queue once it  started to get too much resources.
Remove in the general case is not clear to me. Timers?

3)  It seems like the mechanism trusts the packet marker until the queue
protection mechanism takes
a different decision. How can a marker be redeemed and related  packets be
readmitted to the low latency queue?

Thanks
Luca

On Mon, Mar 11, 2019 at 7:18 PM Greg White <[email-masked]> wrote:

> On 3/11/19, 5:29 PM, "tsvwg on behalf of Greg White" <
> [email-masked] on behalf of [email-masked]> wrote:
>
>     TSVWG,
>
>     I've posted a new informative draft that gives an overview of the new
> Low Latency DOCSIS specification (see links below).  This overview may be
> interesting to TSVWG participants because it includes support for L4S (
> https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-aqm-dualq-coupled)
> and for the NQB PHB (
> https://datatracker.ietf.org/doc/html/draft-white-tsvwg-nqb).
>
>     Best Regards,
>     Greg
>
>
>
>         URL:
> https://www.ietf.org/internet-drafts/draft-white-tsvwg-lld-00.txt
>         Status:
> https://datatracker.ietf.org/doc/draft-white-tsvwg-lld/
>         Htmlized:
> https://tools.ietf.org/html/draft-white-tsvwg-lld-00
>         Htmlized:
> https://datatracker.ietf.org/doc/html/draft-white-tsvwg-lld
>
>
>         Abstract:

---

From: Greg White <[email-masked]>
Date: Wed, 13 Mar 2019 21:03:54 +0000

Luca,

We may work on improving the informative text in that section in order to make the behavior more clear without puzzling through the pseudocode.

  1.  A packet enters the QP algorithm if it is classified to the low latency queue.  By default, the intention is to use ((DSCP==NQB) OR (ECN==ECT(1)) OR (ECN==CE)) (see the NQB draft and ECN-L4S-ID draft) to identify likely NQB flows (but this can be changed if needed).  The algorithm of course doesn’t care how the packets get marked, but our view is that the original application (or OS) should be the one that does the marking. For either marking, there is no incentive to lie.
  2.  The algorithm sanctions (redirects to the classic queue) on a per-packet basis.  So, a microflow that only briefly causes queuing will have some of its packets sanctioned during that episode only, and even during that episode, all of its packets are scored.  Note, in some cases this could result in out-of-order delivery, which might be a minor disincentive for persistently QB flows mismarking themselves as NQB.
  3.  See above.

Greg

From: Luca Muscariello <[email-masked]>
Date: Wednesday, March 13, 2019 at 2:37 PM
To: Greg White <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [tsvwg] New Version Notification for draft-white-tsvwg-lld-00.txt

Greg,

Thanks for the reference.
I have a few questions. I read the document just once so, don't be surprised if I got something wrong.

1) A packet enters the low latency queue if it is marked. Who is responsible for the marking?
Is the original application to do that?

2) Once a packet enters the low latency queue, the flow, or microflow,
is tracked in a (micro)flow table by reporting a function of the queue occupancy.  How do updates of the table
take place? Insert is clear unless the flow is banned, update is clear. Remove is only clear to me when the flow is moved
out of the low latency queue once it  started to get too much resources. Remove in the general case is not clear to me. Timers?

3)  It seems like the mechanism trusts the packet marker until the queue protection mechanism takes
a different decision. How can a marker be redeemed and related  packets be readmitted to the low latency queue?

Thanks
Luca

On Mon, Mar 11, 2019 at 7:18 PM Greg White <[email-masked]<mailto:[email-masked]>> wrote:
Hi Luca,

-Greg

From: Luca Muscariello <[email-masked]<mailto:[email-masked]>>
Date: Monday, March 11, 2019 at 6:26 PM
To: Greg White <[email-masked]>
Cc: Dave Taht <[email-masked]<mailto:[email-masked]>>, Jonathan Morton <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [tsvwg] New Version Notification for draft-morton-taht-tsvwg-sce-00.txt

Hi Greg,

Thanks
Luca

On 3/11/19, 5:29 PM, "tsvwg on behalf of Greg White" <[email-masked]<mailto:[email-masked]> on behalf of [email-masked]> wrote:

    TSVWG,

    Best Regards,
    Greg

    On 3/11/19, 5:05 PM, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>> wrote:

        Abstract:

        Please note that it may take a couple of minutes from the time of submission
        until the htmlized version and diff are available at tools.ietf.org<http://tools.ietf.org>.

        The IETF Secretariat

---

From: Luca Muscariello <[email-masked]>
Date: Wed, 13 Mar 2019 15:07:13 -0700

Greg,

Is there any intention to also grant access to the low latency queue to
applications that do not mark?
Also, if I got this whole mechanism right (big IF), it seems like entering
the low latency queue is only based on marking.
This way it looks like the queue is  easy to access and the mechanism
reacts to bad behaviour afterwards.
Does not this risk to create spurious packet enqueues into the low latency
queue? Or worse malicious sources
to make the low latency queue DoSable?

Luca

On Wed, Mar 13, 2019 at 2:03 PM Greg White <[email-masked]> wrote:

> Luca,
>
>
>
> We may work on improving the informative text in that section in order to
> make the behavior more clear without puzzling through the pseudocode.
>
>
>
>    1. A packet enters the QP algorithm if it is classified to the low
>    latency queue.  By default, the intention is to use ((DSCP==NQB) OR
>    (ECN==ECT(1)) OR (ECN==CE)) (see the NQB draft and ECN-L4S-ID draft) to
>    identify likely NQB flows (but this can be changed if needed).  The
>    algorithm of course doesn’t care how the packets get marked, but our view
>    is that the original application (or OS) should be the one that does the
>    marking. For either marking, there is no incentive to lie.
>    2. The algorithm sanctions (redirects to the classic queue) on a
>    per-packet basis.  So, a microflow that only briefly causes queuing will
>    have some of its packets sanctioned during that episode only, and even
>    during that episode, all of its packets are scored.  Note, in some cases
>    this could result in out-of-order delivery, which might be a minor
>    disincentive for persistently QB flows mismarking themselves as NQB.
>    3. See above.
>
>
>
> Greg
>
>
>
> *From: *Luca Muscariello <[email-masked]>
> *Date: *Wednesday, March 13, 2019 at 2:37 PM
> *To: *Greg White <[email-masked]>
> *Cc: *"[email-masked]" <[email-masked]>
> *Subject: *Re: [tsvwg] New Version Notification for
> draft-white-tsvwg-lld-00.txt
>
>
>
> Greg,
>
>
>

---

From: Greg White <[email-masked]>
Date: Thu, 14 Mar 2019 23:42:18 +0000

Yes, it is easy for applications to access the low latency queue, and the job of QP is to redirect those that shouldn’t have been there.  We could instead run all traffic through QP and let it be automatically sorted, but if there is some traffic that can be excluded a priori, then doing so reduces the queue delay (recall that the QP algorithm only sanctions packets if the queue delay is above the 1ms threshold), reduces the potential that a well-behaving application is sanctioned (perhaps due to hash collisions), etc.   Also, since sanctioning could result in out-of-order delivery for a microflow, presuming that this is ok for all applications might not be advisable.  We have some ideas in the works for modifications that might be useful to reduce the need for packet marking.

I don’t believe that this mechanism increases DoSability (is that a word?) compared with single queue or fq systems, but would be interested in some validation of that.  Yes it is possible for a malicious sender to mark its packets as NQB, and send at a high rate with various port numbers (to get unique flow ids), and thus have some percentage of packets avoid sanctioning, and in the worst case drive queue delay up.  The default threshold values are being adjusted from what is shown in the spec currently, so when using the new default values, the worst case with carefully crafted attack streams is ~32 ms of queuing delay (which, using brute force, would require the malicious sender to launch ~400 Mbps of traffic at the QP function).  Similar traffic patterns in a single queue system (or fq) could, depending on egress rate, result in much more queue delay than this.   Also, if the attacker does not mark their packets NQB, then the system protects the NQB flows from the attack, so in this sense it is also slightly less susceptible than single queue (and maybe fq).

Date: Wednesday, March 13, 2019 at 4:07 PM

Greg,

Is there any intention to also grant access to the low latency queue to applications that do not mark?
Also, if I got this whole mechanism right (big IF), it seems like entering the low latency queue is only based on marking.
This way it looks like the queue is  easy to access and the mechanism reacts to bad behaviour afterwards.
Does not this risk to create spurious packet enqueues into the low latency queue? Or worse malicious sources
to make the low latency queue DoSable?

Luca

On Wed, Mar 13, 2019 at 2:03 PM Greg White <[email-masked]<mailto:[email-masked]>> wrote:
Luca,

3.      See above.

Greg

Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>

Greg,

Thanks
Luca

-Greg

Hi Greg,

Thanks
Luca

    TSVWG,

    Best Regards,
    Greg

        Abstract:

        The IETF Secretariat

---

From: Luca Muscariello <[email-masked]>
Date: Sun, 17 Mar 2019 09:02:00 +0100

One point that is not specified anywhere is when a mflow entry is removed
front the flow table.

As the table is used to blacklist mflows my intuition was that it has to be
based on some sort of timer. Which is another parameter to set in the
mechanism.

The table then is tracking flows that are in progress or maybe not. For
sure an entry is maintained even for flows w/o packets in the queues.  This
has poor scalability properties compared to fq where an entry is maintained
for flows with actual data in the queues.

The protection mechanism assumes that one queue has soft priority over  the
other. Strict priority would be stupid, so there must be a wfq weight to
schedule the classic queue less frequently. I did not find the magic number
that  is set in the DOCSIS specs but whatever number is chosen I wonder
which opitimization objective would be the foundation of that.
10%, 20%, 30% or any number would imply that if the priority queue is used
at 100% utilization the other apps would get a small fraction of the link
capacity.

What number is chosen and based on which calculations?

Thanks
Luca

On Fri 15 Mar 2019 at 00:42, Greg White <[email-masked]> wrote:

> Yes, it is easy for applications to access the low latency queue, and the
> job of QP is to redirect those that shouldn’t have been there.  We could
> instead run all traffic through QP and let it be automatically sorted, but
> if there is some traffic that can be excluded a priori, then doing so
> reduces the queue delay (recall that the QP algorithm only sanctions
> packets if the queue delay is above the 1ms threshold), reduces the
> potential that a well-behaving application is sanctioned (perhaps due to
> hash collisions), etc.   Also, since sanctioning could result in
> out-of-order delivery for a microflow, presuming that this is ok for all
> applications might not be advisable.  We have some ideas in the works for
> modifications that might be useful to reduce the need for packet marking.
>
>
>
> I don’t believe that this mechanism increases DoSability (is that a word?)
> compared with single queue or fq systems, but would be interested in some
> validation of that.  Yes it is possible for a malicious sender to mark its
> packets as NQB, and send at a high rate with various port numbers (to get
> unique flow ids), and thus have some percentage of packets avoid
> sanctioning, and in the worst case drive queue delay up.  The default
> threshold values are being adjusted from what is shown in the spec
> currently, so when using the new default values, the worst case with
> carefully crafted attack streams is ~32 ms of queuing delay (which, using
> brute force, would require the malicious sender to launch ~400 Mbps of
> traffic at the QP function).  Similar traffic patterns in a single queue
> system (or fq) could, depending on egress rate, result in much more queue
> delay than this.   Also, if the attacker does not mark their packets NQB,
> then the system protects the NQB flows from the attack, so in this sense it
> is also slightly less susceptible than single queue (and maybe fq).
>
>
>
>
>
>
>
> *Date: *Wednesday, March 13, 2019 at 4:07 PM

---

From: "Tilmans, Olivier (Nokia - BE/Antwerp)" <[email-masked]>
Date: Sun, 17 Mar 2019 17:32:36 +0000

Hi Luca,

> The protection mechanism assumes that one queue has soft priority over
> the other. Strict priority would be stupid, so there must be a wfq weight to

The protection mechanism only affect how competing packets within a RTT gets
dequeued. In this instance, a WRR protects the classic flows from
unresponsive/misclassified L4S flows, at the expense of the other L4S flows.

It does not affect the overall throughput of the flows when measuring it over a 
timeframe of a few Tupdate interval (e.g., 16ms). Indeed, the core of the
DualPI2 AQM is a PIE2 controller computing the random drop/marking probability
based on the classic queue. This probability is then also used as base to mark L4S
flows, albeit with a quadratic relationship (i.e., for the same base probability, the
L4S flows are marked much more often than the classic ones).

Consequently, whenever L4S traffic causes the classic queue to build up (even 
slightly above the PIE2 target), its marking probability drastically increases as the 
PIE2 controller reacts. As the L4S flows have scalable CCs (i.e., understand how to 
leverage multiple CE marks per RTT), they reduce their cwnd accordingly within
 a RTT (which is much smaller than the classic flows' RTT). 

In other words, L4S senders end up slowing down themselves (to a point where 
the L4S queue is empty for most of the transmission slots) to allow the classic 
queue to drain, and this reaction is the result of a much faster control-loop than
the one used by the classic sender.

There are thus no magic number per say for the WRR ratio. §4.1.4 of
 draft-ietf-tsvwg-aqm-dualq-coupled explains the guidelines to pick one.

Note that the exact throughput ratio between L4S and classic flows can be 
tweaked using a coupling factor, as explained in appendix C. of
 draft-ietf-tsvwg-aqm-dualq-coupled. 

I hope this was clear enough.

Best,
Olivier

---

From: Luca Muscariello <[email-masked]>
Date: Sun, 17 Mar 2019 20:32:09 +0100

Is there an assumption that the low latency queue is only used by rate
controlled flows
that are sensitive to ECN feedback?

 This goes back to my question to Greg about serving non marked flows in
the low latency queue.

I would assume that a low latency queue is open to flows that are non
necessarily controlled by ECN but just exogenously rate limited by say a
 codec rate.

But now it seems like this is not the case.

Thanks
Luca

---

From: "Tilmans, Olivier (Nokia - BE/Antwerp)" <[email-masked]>
Date: Mon, 18 Mar 2019 08:56:01 +0000

> that are sensitive to ECN feedback?

The L4S definitions in draft-ietf-tsvwg-l4s-arch state that L4S flow have a 
scalable CC, in order to perform bandwidth pooling (fairly) between the
L4S flows and the classic ones.
This can however be relaxed a bit, see below.

> necessarily controlled by ECN but just exogenously rate limited by say a

This is what draft draft-white-tsvwg-nqb discusses, namely also adding 
non-queue building flows in the L4S queue (e.g., non-capacity seeking
flows such as gaming, DNS traffic, pings, sparse CBR flows).

Note that the presence of such unresponsive flows in the L4S queue 
implicitly decreases the throughput of the scalable L4S flows (but not
their classic counterpart), as they have to back off on behalf of the 
non-responsive flows (but can do so without fiddling with a magic WRR 
number). This can be accounted for in the coupling factor of the queue.

Best,
Olivier

> On Sun 17 Mar 2019 at 18:32, Tilmans, Olivier (Nokia - BE/Antwerp)
> <[email-masked] <mailto:olivier.tilmans@nokia-bell-
> labs.com> > wrote:
> 
> 
> 	Hi Luca,
>

---

From: Luca Muscariello <[email-masked]>
Date: Mon, 18 Mar 2019 12:21:01 +0100

> number). This can be accounted for in the coupling factor of the queue.

Right, this is what I was trying to understand.
I checked draft-white-tsvwg-nqb but the mechanism to share traffic in the
two queues
is not described. This document refers back to draft-ietf-tsvwg-l4s-arch so
I cycled back to
my starting point as there is no description of how the queueing system
works when
non marked ECNed traffic is sent  to the low latency queue.

I thought that that was described in the DOCSIS spec Greg shared, but did
not find an answer there either.

> >       > the other. Strict priority would be stupid, so there must be a
> wfq
> > weight to
> >       > schedule the classic queue less frequently. I did not find the
> magic
> > number
> >       > at 100% utilization the other apps would get a small fraction of
> the
> > link
> >       > capacity.
> >       >
> >       timeframe of a few Tupdate interval (e.g., 16ms). Indeed, the core
> of
> > the
> >       based on the classic queue. This probability is then also used as
> base
> > to mark L4S
> >       flows, albeit with a quadratic relationship (i.e., for the same
> base
> > probability, the
> >       Consequently, whenever L4S traffic causes the classic queue to
> build
> > up (even
> >       slightly above the PIE2 target), its marking probability
> drastically
> > increases as the

---

From: Luca Muscariello <[email-masked]>
Date: Mon, 18 Mar 2019 15:46:54 +0100

On Mon, Mar 18, 2019 at 1:21 PM Tilmans, Olivier (Nokia - BE/Antwerp) <
[email-masked]> wrote:

> Hi Luca,
>
> Just to make sure, did I answer your question?
>

maybe. Let me ask a few more clarification questions below.

>
> If not the behavior of putting NQB flows in the L4S queue is dependent on
> the
> AQM used, hence the answer would be to look at
> draft-ietf-tsvwg-aqm-dualq-coupled for a discussion of the effects of
> doing so.
> DOCSIS adds on top of the AQM a protection scheme to sanction flows in the
> L4S that cause queue build up, and uses a WRR to control how packets get
> dequeued.
>
> The final answer would then be: yes, you can put non-ECN controlled flows
> in
> the L4S queue provided those are sparse enough. Otherwise, they would need
> to be wrapped in a scalable CC and/or interpret the received ECN feedback
> (as
>  the 2015 demo showing live video streaming zooming/zoomout did).
>

This is where things become unclear or just unspecified in the documents I
read so far.
Sometimes maths or code is simpler to read that a thousand words.

My interpretation of what you say is that the time sharing between the two
queues
depends on a closed loop mechanism that relies on classification first and
a compliant sender that
is sensitive to ECN according  to the mechanism described in
draft-ietf-tsvwg-l4s-arch.

Assuming that the control theoretic equilibrium you get requires a given
WRR weight,
anytime non conformant traffic is allowed into the low latency queue this
equilibrium can be
perturbed. Which is fine as long as the perturbation is small enough.

Non controlled traffic can generate a busy period in the low latency queue
that can grow
w/o necessarily generating large standing queues. That should not trigger
the queue protection
mechanism Greg mentioned.
>From what I understand WRR would  then make the equilibrium drift to
another value.
This value should be that the classic queue gets lower service time
compared to what
the original configuration was meant to provide.

Do I get this right?

Thanks
Luca

>
>
> Best,
> Olivier
>
>
> > -----Original Message-----
> > Sent: Monday, March 18, 2019 12:21 PM
> > To: Tilmans, Olivier (Nokia - BE/Antwerp) <olivier.tilmans@nokia-bell-
> > labs.com>
> > Cc: Greg White <[email-masked]>; tsvwg IETF list <[email-masked]>
> > Subject: Re: [tsvwg] Fwd: New Version Notification for
> draft-white-tsvwg-lld-
> > 00.txt
> >
> >
> >
> > On Mon, Mar 18, 2019 at 9:56 AM Tilmans, Olivier (Nokia - BE/Antwerp)
> >       The L4S definitions in draft-ietf-tsvwg-l4s-arch state that L4S
> flow
> > have a
> >       > necessarily controlled by ECN but just exogenously rate limited
> by
> > say a
> >       > codec rate.
> >
> > is not described. This document refers back to draft-ietf-tsvwg-l4s-arch
> so I
> > cycled back to
> >       > <[email-masked]
> > <mailto:[email-masked]>  <mailto:
> olivier.tilmans@nokia-
> > bell- <mailto:olivier.tilmans@nokia-bell->
> >       > labs.com <http://labs.com> > > wrote:
> >       >
> >       >
> >       >       Hi Luca,
> >       >
> >       >       > that  is set in the DOCSIS specs but whatever number is
> chosen
> > I
> >       > wonder
> >       >       > 10%, 20%, 30% or any number would imply that if the
> priority
> >       > queue is used
> >       >       The protection mechanism only affect how competing packets
> > within
> >       > a RTT gets
> >       >       dequeued. In this instance, a WRR protects the classic
> flows
> > from
> >       >       unresponsive/misclassified L4S flows, at the expense of the
> > other
> >       > L4S flows.
> >       >
> >       >       timeframe of a few Tupdate interval (e.g., 16ms). Indeed,
> the
> > core of
> >       > the
> >       >       flows, albeit with a quadratic relationship (i.e., for the
> same base
> >       > probability, the
> >       >       PIE2 controller reacts. As the L4S flows have scalable CCs
> (i.e.,
> >       > understand how to
> >       >       In other words, L4S senders end up slowing down themselves
> > (to a
> >       > point where
> >       >       the L4S queue is empty for most of the transmission slots)
> to
> > allow
> >       > the classic
> >       >       queue to drain, and this reaction is the result of a much
> faster
> >       > control-loop than
> >       >        draft-ietf-tsvwg-aqm-dualq-coupled explains the
> guidelines to
> > pick
> >       > one.
> >       >
> >       >       Note that the exact throughput ratio between L4S and
> classic
> > flows
> >       > can be

---

From: Greg White <[email-masked]>
Date: Tue, 19 Mar 2019 03:31:49 +0000

Hi Luca, see below [GW].

On Mon, Mar 18, 2019 at 1:21 PM Tilmans, Olivier (Nokia - BE/Antwerp) <[email-masked]<mailto:[email-masked]>> wrote:
Hi Luca,

If not the behavior of putting NQB flows in the L4S queue is dependent on the
AQM used, hence the answer would be to look at
draft-ietf-tsvwg-aqm-dualq-coupled for a discussion of the effects of doing so.
DOCSIS adds on top of the AQM a protection scheme to sanction flows in the
L4S that cause queue build up, and uses a WRR to control how packets get
dequeued.

The final answer would then be: yes, you can put non-ECN controlled flows in
the L4S queue provided those are sparse enough. Otherwise, they would need
to be wrapped in a scalable CC and/or interpret the received ECN feedback (as
 the 2015 demo showing live video streaming zooming/zoomout did).

This is where things become unclear or just unspecified in the documents I read so far.
Sometimes maths or code is simpler to read that a thousand words.

My interpretation of what you say is that the time sharing between the two queues
depends on a closed loop mechanism that relies on classification first and a compliant sender that
is sensitive to ECN according  to the mechanism described in draft-ietf-tsvwg-l4s-arch.

Assuming that the control theoretic equilibrium you get requires a given WRR weight,
anytime non conformant traffic is allowed into the low latency queue this equilibrium can be
perturbed. Which is fine as long as the perturbation is small enough.

[GW] When both queues are in use by responsive traffic, the classic queue AQM drives the congestion signal for both queues, and gives them both a congestion signal that the two sender types interpret in a consistent way, leading toward per-flow fairness.  Just as with a single queue AQM, the consistent (flow unaware) application of drop probability (and presumed consistent interpretation of those congestion signals by senders) leads toward per-flow fairness.  The equilibrium is not strongly related to the WRR weight.  As long as the weight is high enough, the relationship between the congestion signals (drop in the case of the classic queue and ECN mark in the case of the low latency queue) drives the equilibrium.  Just as an example of a simple case, if all of the flows happened to be long-running responsive flows in congestion avoidance, then the congestion signals will drive the equilibrium as long as the WRR weight given to the low latency queue is greater than or equal to the share of the flows that are occupying that queue.  In the DOCSIS spec the default value for the weight is 230/256 (about 90%, see section C.2.2.7.17.6 in the DOCSIS spec), so as long as less than 90% of the simultaneous flows are occupying the low latency queue (for example if there are four L4S flows and one CUBIC flow), per-flow fairness will be maintained.   If greater than 90% of the simultaneous flows are in the low latency queue (e.g. twenty-four L4S flows vs. two CUBIC flows), the ~10%  weight given to the classic queue will result in the classic flows each getting somewhat more than their share of bandwidth (e.g. they will each get 5% of the capacity, while the L4S flows will each get 3.75%).  If we could assume that ALL flows are well-behaved responsive flows, then strict priority could be used, and per-flow fairness would be maintained regardless of the mix.  Since we can’t make that assumption, the non-100% weight protects some bandwidth in the Classic queue.

Non controlled traffic can generate a busy period in the low latency queue that can grow
w/o necessarily generating large standing queues. That should not trigger the queue protection
mechanism Greg mentioned.
From what I understand WRR would  then make the equilibrium drift to another value.
This value should be that the classic queue gets lower service time compared to what
the original configuration was meant to provide.

[GW] Non-responsive traffic in the low latency queue will consume some of that queue’s WRR weight (similarly, non-responsive traffic in the classic queue will consume some of that queue’s weight), with the result being that for the remaining congestion controlled traffic the range of flow mixes over which per-flow fairness is achieved is shifted.  As an example, if non-responsive traffic in the low latency queue consumed 50% of the total capacity, and the rest of the traffic was congestion controlled, the system would behave as if (for the remaining 50% of capacity available to the congestion controlled flows) the WRR weight was 80% (90% minus 50%, divided by 50%).   So per-flow fairness would only be achieved when the low latency queue is carrying less than or equal to 80% of the flows, otherwise the classic flows will get somewhat more than their share.    As another example, if non-responsive traffic in the classic queue were consuming 50% of the total capacity, and the rest of the traffic was congestion controlled, the system would provide per-flow fairness (with the remaining 50% of the capacity) across ALL mixes of L4S/Classic.

[GW] Yes, there are pathological cases where high-bandwidth non-responsive flows are consuming the entire channel capacity.  In those cases, the WRR weight will result in the flows in the low latency queue getting more aggregate bandwidth than the ones in the classic queue.

Do I get this right?

Thanks
Luca

Best,
Olivier

> labs.com<http://labs.com>>
> Cc: Greg White <[email-masked]<mailto:[email-masked]>>; tsvwg IETF list <[email-masked]<mailto:[email-masked]>>
> Subject: Re: [tsvwg] Fwd: New Version Notification for draft-white-tsvwg-lld-
> 00.txt
>
>
>
> <[email-masked]<mailto:[email-masked]> <mailto:olivier.tilmans@nokia-bell-<mailto:olivier.tilmans@nokia-bell->
> labs.com<http://labs.com>> > wrote:
>
>
> is not described. This document refers back to draft-ietf-tsvwg-l4s-arch so I
> cycled back to
>       > <[email-masked]<mailto:[email-masked]>
> <mailto:[email-masked]<mailto:[email-masked]>>  <mailto:olivier.tilmans@nokia-<mailto:olivier.tilmans@nokia->
> bell- <mailto:olivier.tilmans@nokia-bell-<mailto:olivier.tilmans@nokia-bell->>
>       > labs.com<http://labs.com> <http://labs.com> > > wrote:
>       >
>       >
>       >       Hi Luca,
>       >
>       >       > that  is set in the DOCSIS specs but whatever number is chosen
> I
>       > wonder
>       >       timeframe of a few Tupdate interval (e.g., 16ms). Indeed, the
> core of
>       > the
>       >       the L4S queue is empty for most of the transmission slots) to
> allow
>       > the classic
>       >        draft-ietf-tsvwg-aqm-dualq-coupled explains the guidelines to
> pick
>       > one.
>       >
>       >       Note that the exact throughput ratio between L4S and classic
> flows
>       > can be

---

From: Luca Muscariello <[email-masked]>
Date: Tue, 19 Mar 2019 09:53:20 +0100

Got it, thanks.

Consider that one concern that I have is on what you Greg consider a
pathological case.
That case is currently a big chunk of Internet traffic that requires low
latency.

The current scheduler makes substantial assumptions about the behaviour and
misbehaviour of data sources.
The thing the I find uncomfortable is that this proposal assumes that every
source conforms to a given profile,
including explicit marking.

The current queueing system instead of creating incentives to well behave
creates a mechanism that punishes
sources that do not conform. Even  sources that well behave can be punished
just because they do not conform.

My understanding is that this might be difficult to accept for many
applications and, most important, people.

Luca

On Tue, Mar 19, 2019 at 4:31 AM Greg White <[email-masked]> wrote:

> [GW] When both queues are in use by responsive traffic, the classic queue
> AQM drives the congestion signal for both queues, and gives them both a
> congestion signal that the two sender types interpret in a consistent way,
> leading toward per-flow fairness.  Just as with a single queue AQM, the
> consistent (flow unaware) application of drop probability (and presumed
> consistent interpretation of those congestion signals by senders) leads
> toward per-flow fairness.  The equilibrium is not strongly related to the
> WRR weight.  As long as the weight is high enough, the relationship between
> the congestion signals (drop in the case of the classic queue and ECN mark
> in the case of the low latency queue) drives the equilibrium.  Just as an
> example of a simple case, if all of the flows happened to be long-running
> responsive flows in congestion avoidance, then the congestion signals will
> drive the equilibrium as long as the WRR weight given to the low latency
> queue is greater than or equal to the share of the flows that are occupying
> that queue.  In the DOCSIS spec the default value for the weight is 230/256
> (about 90%, see section C.2.2.7.17.6 in the DOCSIS spec), so as long as
> less than 90% of the simultaneous flows are occupying the low latency queue
> (for example if there are four L4S flows and one CUBIC flow), per-flow
> fairness will be maintained.   If greater than 90% of the simultaneous
> flows are in the low latency queue (e.g. twenty-four L4S flows vs. two
> CUBIC flows), the ~10%  weight given to the classic queue will result in
> the classic flows each getting somewhat more than their share of bandwidth
> (e.g. they will each get 5% of the capacity, while the L4S flows will each
> get 3.75%).  If we could assume that ALL flows are well-behaved responsive
> flows, then strict priority could be used, and per-flow fairness would be
> maintained regardless of the mix.  Since we can’t make that assumption, the
> non-100% weight protects some bandwidth in the Classic queue.
>
>
>
> [GW] Non-responsive traffic in the low latency queue will consume some of
> that queue’s WRR weight (similarly, non-responsive traffic in the classic
> queue will consume some of that queue’s weight), with the result being that
> for the remaining congestion controlled traffic the range of flow mixes
> over which per-flow fairness is achieved is shifted.  As an example, if
> non-responsive traffic in the low latency queue consumed 50% of the total
> capacity, and the rest of the traffic was congestion controlled, the system
> would behave as if (for the remaining 50% of capacity available to the
> congestion controlled flows) the WRR weight was 80% (90% minus 50%, divided
> by 50%).   So per-flow fairness would only be achieved when the low latency
> queue is carrying less than or equal to 80% of the flows, otherwise the
> classic flows will get somewhat more than their share.    As another
> example, if non-responsive traffic in the classic queue were consuming 50%
> of the total capacity, and the rest of the traffic was congestion
> controlled, the system would provide per-flow fairness (with the remaining
> 50% of the capacity) across ALL mixes of L4S/Classic.
>
>
>
> [GW] Yes, there are pathological cases where high-bandwidth non-responsive
> flows are consuming the entire channel capacity.  In those cases, the WRR
> weight will result in the flows in the low latency queue getting more
> aggregate bandwidth than the ones in the classic queue.
>
>
>
>
>
> Do I get this right?
>
>
>
> Thanks
>
> Luca
>
>
>
>
>
> Best,
> Olivier
>
>

---

From: Greg White <[email-masked]>
Date: Tue, 19 Mar 2019 21:11:59 +0000

I think that is a mischaracterization.

I don’t believe that (at least on cable broadband links) non-responsive, link-saturating traffic is “currently a big chunk of Internet traffic that requires low latency”.   Perhaps on very low speed links this may happen, but what are the applications that are streaming at (e.g.) 10 Mbps without any congestion control?

The current Internet places expectations on data sources that they implement congestion control, and that they respond in a fairly consistent way to congestion signals (usually packet drops).  Not all of them do, but the majority do, and it has not resulted in a tragedy of the commons.  The dual queue mechanism continues that expectation.

I also don’t agree that the dual queue system lacks incentives.  For an application to get the benefit of sub millisecond queuing delay, it needs to ensure that it is not the cause of the queuing delay.  It can do that in a couple of ways.  One is that it sends at a low and smooth enough rate that it is highly unlikely to cause queuing, the other is that it can respond to fast congestion signals (new ECN marks).  The applications that do this can enjoy sub millisecond queuing delay.  If that isn’t an incentive, that’s fine. Some NQB traffic may not care about latency performance, and so would be equally happy in the classic queue.

-Greg

From: Luca Muscariello <[email-masked]>
Date: Tuesday, March 19, 2019 at 2:53 AM
To: Greg White <[email-masked]>
Cc: "Tilmans, Olivier (Nokia - BE/Antwerp)" <[email-masked]>, tsvwg IETF list <[email-masked]>
Subject: Re: [tsvwg] Fwd: New Version Notification for draft-white-tsvwg-lld-00.txt

Got it, thanks.

Consider that one concern that I have is on what you Greg consider a pathological case.
That case is currently a big chunk of Internet traffic that requires low latency.

The current scheduler makes substantial assumptions about the behaviour and misbehaviour of data sources.
The thing the I find uncomfortable is that this proposal assumes that every source conforms to a given profile,
including explicit marking.

The current queueing system instead of creating incentives to well behave creates a mechanism that punishes
sources that do not conform. Even  sources that well behave can be punished just because they do not conform.

My understanding is that this might be difficult to accept for many applications and, most important, people.

Luca

On Tue, Mar 19, 2019 at 4:31 AM Greg White <[email-masked]<mailto:[email-masked]>> wrote:
Hi Luca, see below [GW].

Do I get this right?

Thanks
Luca

Best,
Olivier

---

From: Luca Muscariello <[email-masked]>
Date: Tue, 19 Mar 2019 23:13:14 +0100

Greg,

You may be right. We may be doing different assumptions.
To me the point where we disagree is that you assume that an application,
to get access to the low latency queue, is not only well behaving
but also sending at a very low rate compared to the link capacity.

The point I'm trying to make is that in general, we do not know that rate.
As long as we have no predictable information about the amount of non
classified
traffic that can be admitted to the low latency queue, the coupling factor,
that
determines how time is shared between the two queues, cannot be set.
Any value would break the stable equilibrium assumed by the system and
therefore
that much traffic has to be kicked out of the queue.

This is not a positive incentive but rather policy enforcement.
This is in itself a subject of discussion.
Another concern I see is more about DoS to the low latency queue and
then to both queues.

It seems a priori definitely possible to create a limit cycle to the
control theoretic
equilibrium by injecting background non classified traffic that generates
little queuing.
Such packet process could then disturb all other sources or worse starve
them enough
to force user to abandon.
In this case even a FIFO queue would be better (providing more predictable
performance)
than two coupled queues relying on
a closed loop equilibrium which assumes only compliant traffic is present.

Now, being aware of how difficult it is to compute stability regions of a
closed loop system
with delays, which is the one we are discussing here, my overall impression
is that
this queueing system is fragile and can provide unpredictable performance.

Assuming that you're pretty confident about the cable use case you have in
mind,
with all the assumptions you are making about today applications' rates,
would you also
be confident about future applications or about other network access
technologies?

Luca

On Tue, Mar 19, 2019 at 10:12 PM Greg White <[email-masked]> wrote:

> I don’t believe that (at least on cable broadband links) non-responsive,
> link-saturating traffic is “currently a big chunk of Internet traffic that
> requires low latency”.   Perhaps on very low speed links this may happen,
> but what are the applications that are streaming at (e.g.) 10 Mbps without
> any congestion control?
>
>
>
> The current Internet places expectations on data sources that they
> implement congestion control, and that they respond in a fairly consistent
> way to congestion signals (usually packet drops).  Not all of them do, but
> the majority do, and it has not resulted in a tragedy of the commons.  The
> dual queue mechanism continues that expectation.
>
>
>
> I also don’t agree that the dual queue system lacks incentives.  For an
> application to get the benefit of sub millisecond queuing delay, it needs
> to ensure that it is not the cause of the queuing delay.  It can do that in
> a couple of ways.  One is that it sends at a low and smooth enough rate
> that it is highly unlikely to cause queuing, the other is that it can
> respond to fast congestion signals (new ECN marks).  The applications that
> do this can enjoy sub millisecond queuing delay.  If that isn’t an
> incentive, that’s fine. Some NQB traffic may not care about latency
> performance, and so would be equally happy in the classic queue.
>
>
>
> -Greg
>
>
>
>
>
>
>
>
>
> *Date: *Tuesday, March 19, 2019 at 2:53 AM
> *Cc: *"Tilmans, Olivier (Nokia - BE/Antwerp)" <
> [email-masked]>, tsvwg IETF list <[email-masked]>
> *Subject: *Re: [tsvwg] Fwd: New Version Notification for
> The thing the I find uncomfortable is that this proposal assumes that
> every source conforms to a given profile,
>
> sources that do not conform. Even  sources that well behave can be
> punished just because they do not conform.
>
>
>

---

From: Greg White <[email-masked]>
Date: Wed, 20 Mar 2019 20:03:30 +0000

Hi Luca,

First off, thank you for your review and probing questions.  I very much appreciate it.

Now on the points where you still have concerns, either I am not understanding them, or you aren’t understanding my answers.  I’ve added some responses inline below.  Hopefully we can get to the point that we agree.

-Greg

Date: Tuesday, March 19, 2019 at 4:13 PM

Greg,

[GW]  No, that is not the assumption. The fundamental requirement is that, in order to have 100% of its packets remain in the low latency queue, a flow cannot be contributing appreciably to queuing latency.  Note, we are actually not describing the performance of the low latency queue as being a DetNet guarantee, so the requirement for 100% of packets to have deterministic latency does not exist.  Rather, the focus is on 99th percentile latency.  Nonetheless, in the abstract, a single application flow could send smooth (CBR) non-responsive traffic that fully utilizes the link capacity, and have all of that traffic utilize either queue (low latency or classic, depending on if it marked its packets NQB or not).

The point I'm trying to make is that in general, we do not know that rate.
As long as we have no predictable information about the amount of non classified
traffic that can be admitted to the low latency queue, the coupling factor, that
determines how time is shared between the two queues, cannot be set.
Any value would break the stable equilibrium assumed by the system and therefore
that much traffic has to be kicked out of the queue.

[GW] In practice, as you point out, an application generally cannot know the link rate (nor can it know whether it is the only application).  So, in practice, an application that is interested in low latency has 2 options in order to ensure that it isn’t causing queuing delay.  One option is that it simply does not send traffic that is likely to exceed the available capacity of the link. There are a lot of applications that do just this, and are great candidates for being marked NQB.  (Even without dual-queue, this is a practical constraint for any non-responsive application that cares about latency or loss performance, the only additional factor in the low latency queue is that the constraint applies on a more instantaneous basis than for a deep buffered link.) Given that the application cannot “know” the available capacity, all non-responsive applications take on some risk that they have guessed wrong, and have to accept the consequences (latency or loss) in the case that they have.  Of course, the lower the sending rate, the greater the likelihood that it won’t exceed the link capacity. The other option is to implement a low latency scalable congestion control (e.g. TCP Prague).  TCP Prague can fully utilize its per-flow-fair share of the link capacity across a very wide range of cases without the need to tune any parameters in the dual-queue configuration.  Outside that range of cases, the behavior is not catastrophic.   I’m not certain what you mean by “non-classified traffic” do you mean non-responsive (or non-congestion-controlled)? Also you used the term “coupling factor” which is a specific term used in dual-queue coupled AQM to refer to the multiplicative factor that relates the L4S ECN marking probability to the Classic AQM drop probability.  I don’t think that is what you were referring to.  I think you intended to mean the WRR weight.  In case I am right on both counts, I thought I had explained this in my earlier email, but let me make it clear.  Stability is not in question. What you are arguing is that per-flow-fairness isn’t guaranteed in certain situations.   This is true, but as I pointed out, the range of conditions where fairness is more-or-less achieved is pretty large, and outside that range things aren’t so bad.  I say “more-or-less achieved” because, with the dual-queue, just like a single queue, flows will only tend toward fairness (with the caveats of RTT differences, variations in congestion controllers, etc.).   But, a goal of TCP Prague is to at least reduce this RTT dependence.

[GW] This seems to be a matter of opinion, but I’m actually not sure why you feel the need to make the distinction.  The low latency queue is only for flows that aren’t causing queuing delay.  If a flow is causing queuing delay then the system (and in some cases the application itself) is better off if the application’s packets take the classic queue.  The job of Queue Protection is to enforce this, and ideally create incentives for “correctly” marking packets.  Furthermore, the applications that are the best behaved are the most likely to be able to send all of their packets via the low latency queue, so this is an incentive. Keep in mind, that this isn’t DetNet.

It seems a priori definitely possible to create a limit cycle to the control theoretic
equilibrium by injecting background non classified traffic that generates little queuing.
Such packet process could then disturb all other sources or worse starve them enough
to force user to abandon.
In this case even a FIFO queue would be better (providing more predictable performance)
than two coupled queues relying on
a closed loop equilibrium which assumes only compliant traffic is present.

[GW] I think that I have already dispelled this in an earlier email, but let me know if you would like me to try again.

Now, being aware of how difficult it is to compute stability regions of a closed loop system
with delays, which is the one we are discussing here, my overall impression is that
this queueing system is fragile and can provide unpredictable performance.

[GW] Then you would undoubtedly have the overall impression that the current internet is fragile and can provide unpredictable performance (which, I suppose, is true).  In my opinion, dual-queue reduces the unpredictability, and does not increase fragility.  Analyzing stability is even more difficult given that the internet is not a linear time-invariant system.  As a result, I don’t personally give a lot of credence to that sort of analysis for congestion control systems.  Nonetheless, LTI approximations have been used to give confidence in the stability of PIE (at least for Reno CC), and in the cases that you seem to be concerned about, the PIE controller is in use.  Simulations have confirmed stability for DCTCP in these mixed traffic situations as well.

Assuming that you're pretty confident about the cable use case you have in mind,
with all the assumptions you are making about today applications' rates, would you also
be confident about future applications or about other network access technologies?

[GW] Again, we’re not making any assumptions about today’s application rates.  It is the applications that need to make assumptions based on all of the environments that they expect to be deployed in (and this is 100% not new).  If an application wants low latency and low loss, then it needs to fit in the available capacity along the path, whatever that path may be.  If it doesn’t want to use congestion control for some reason, then it needs to consider how it will send its traffic to ensure that it doesn’t overload the bottleneck link.  That is fundamental (and not specific to dual queue).

Luca

On Tue, Mar 19, 2019 at 10:12 PM Greg White <[email-masked]<mailto:[email-masked]>> wrote:
I think that is a mischaracterization.

-Greg

Cc: "Tilmans, Olivier (Nokia - BE/Antwerp)" <[email-masked]<mailto:[email-masked]>>, tsvwg IETF list <[email-masked]<mailto:[email-masked]>>

Got it, thanks.

Luca

Do I get this right?

Thanks
Luca

Best,
Olivier

---

From: Luca Muscariello <[email-masked]>
Date: Thu, 28 Mar 2019 12:50:19 +0100

Hi Greg,

sorry for my belated follow up on this. More comments below. I cut most of
the history for readability.

Luca

On Wed, Mar 20, 2019 at 9:03 PM Greg White <[email-masked]> wrote:

> Now on the points where you still have concerns, either I am not
> understanding them, or you aren’t understanding my answers.  I’ve added
> some responses inline below.  Hopefully we can get to the point that we
> agree.
>
>
>
> -Greg
>
>
>
>
>
>
>
> *Date: *Tuesday, March 19, 2019 at 4:13 PM
> [GW]  No, that is not the assumption. The fundamental requirement is that,
> in order to have 100% of its packets remain in the low latency queue, a
> flow cannot be contributing appreciably to queuing latency.  Note, we are
> actually not describing the performance of the low latency queue as being a
> DetNet guarantee, so the requirement for 100% of packets to have
> deterministic latency does not exist.  Rather, the focus is on 99th
> percentile latency.  Nonetheless, in the abstract, a single application
> flow could send smooth (CBR) non-responsive traffic that fully utilizes the
> link capacity, and have all of that traffic utilize either queue (low
> latency or classic, depending on if it marked its packets NQB or not).
>

[LM] If a single CBR marked application can absorb say 90% of the link
capacity then what you are describing is equivalent to a strict priority
queue.
I noticed that this is one option in DualQ that is, wisely, dismissed in
the related I-D document too. Even a D/D/1 Queue with the right values can
get very
close to saturation and a single packet in the queue at most. But that is
not the kind of sharing we want because it would also contradict what you
write
in the abstract:
"LLD makes the internet experience better for latency sensitive
applications without any negative impact on other applications."
This example you make has very negative impact on other applications.

> [GW] In practice, as you point out, an application generally cannot know
> the link rate (nor can it know whether it is the only application).  So, in
> practice, an application that is interested in low latency has 2 options in
> order to ensure that it isn’t causing queuing delay.  One option is that it
> simply does not send traffic that is likely to exceed the available
> capacity of the link. There are a lot of applications that do just this,
> and are great candidates for being marked NQB.  (Even without dual-queue,
> this is a practical constraint for any non-responsive application that
> cares about latency or loss performance, the only additional factor in the
> low latency queue is that the constraint applies on a more instantaneous
> basis than for a deep buffered link.) Given that the application cannot
> “know” the available capacity, all non-responsive applications take on some
> risk that they have guessed wrong, and have to accept the consequences
> (latency or loss) in the case that they have.  Of course, the lower the
> sending rate, the greater the likelihood that it won’t exceed the link
> capacity. The other option is to implement a low latency scalable
> congestion control (e.g. TCP Prague).  TCP Prague can fully utilize its
> per-flow-fair share of the link capacity across a very wide range of cases
> without the need to tune any parameters in the dual-queue configuration.
> Outside that range of cases, the behavior is not catastrophic.   I’m not
> certain what you mean by “non-classified traffic” do you mean
> non-responsive (or non-congestion-controlled)? Also you used the term
> “coupling factor” which is a specific term used in dual-queue coupled AQM
> to refer to the multiplicative factor that relates the L4S ECN marking
> probability to the Classic AQM drop probability.  I don’t think that is
> what you were referring to.  I think you intended to mean the WRR weight.
> In case I am right on both counts, I thought I had explained this in my
> earlier email, but let me make it clear.  Stability is not in question.
> What you are arguing is that per-flow-fairness isn’t guaranteed in certain
> situations.   This is true, but as I pointed out, the range of conditions
> where fairness is more-or-less achieved is pretty large, and outside that
> range things aren’t so bad.  I say “more-or-less achieved” because, with
> the dual-queue, just like a single queue, flows will only tend toward
> fairness (with the caveats of RTT differences, variations in congestion
> controllers, etc.).   But, a goal of TCP Prague is to at least reduce this
> RTT dependence.
>

[LM]  Non classified traffic is traffic that is not marked. So it has to go
to the classic queue UNLESS you can classify it locally and then enqueue it
in the low latency queue even if it is not Prague.
Take WebEx, Skype, WebRTC voice as an example, w/o assuming that there is
one single voice call at the same time. Assume I have several voice calls
in progress and maybe video.
But I also do care about DNS traffic and so on that has impact on other
applications' completion time.
I am not arguing for per-flow fairness intended as per-flow rate
equalization. I'm arguing for fairness in the sense that my application is
not killed by this system by starvation.
The queue coupling factor includes also WRR weight which plays a role in
coupling both queue service times together with ECN and packet loss.
Whatever is the set of parameters used in the 2-queue systems, as long as
both queues are backlogged, ECN marking and packet loss rate have no impact
in queue scheduling time, only WRR is responsible for time sharing.
Coupling factors are used to create idle periods in the low latency queue
that are exploited by the classic queue.
Now, my point is that this approach is harmful and you cannot stabilize the
control loop as intended therefore performance will not be as expected.

> [GW] This seems to be a matter of opinion, but I’m actually not sure why
> you feel the need to make the distinction.  The low latency queue is only
> for flows that aren’t causing queuing delay.  If a flow is causing queuing
> delay then the system (and in some cases the application itself) is better
> off if the application’s packets take the classic queue.  The job of Queue
> Protection is to enforce this, and ideally create incentives for
> “correctly” marking packets.  Furthermore, the applications that are the
> best behaved are the most likely to be able to send all of their packets
> via the low latency queue, so this is an incentive. Keep in mind, that this
> isn’t DetNet.
>

[LM] I think that queue protection has to be mandatory. Not only, I think
that queue protection should also be used to classify packets. DualQ as it
is, is harmful and cannot be fixed.
A coupled queueing system where service time and switch over time is
determined by the input traffic is very difficult to stabilize.
And what I have in mind is a stochastic process not a detnet like system.
Very far from that.