Subject: [iccrg] Better QoS for TCP ACK question

From: Toerless Eckert <[email-masked]>
Date: Thu, 12 Apr 2018 22:09:48 +0200

Was wondering folks on ICCRG had any more insight into the
state of research/understanding into this type of simple type of
congestion experience improvement. I somehow fear it might be
a lost simple trick that could help more people if wee would
manage to make it become a BCP, but Mikaels reply/example
showed me already how there are interesting variations, so
i wonder if we even had a simple rule of thump safe
enough to recommend as a BCP (and the constraints under which
it makes sense).

If not, maybe this would be a lovely simple job for a student
thesis for someone wanting to do NS2/NS3 simulation work
in congestion control (primary issue is a good set of
test cases and traffic mixes i guess).

Cheers
    Toerless

On Sat, Apr 07, 2018 at 07:10:17AM +0200, Mikael Abrahamsson wrote:
> On Fri, 6 Apr 2018, Toerless Eckert wrote:
> 
> > A couple of years ago i was told that a range of home routers
> > would do some DPI based diffserv QoS, aka prioritizing TCP ACP
> > messages in the queue before any other packets 9sonded like
> > a strict priority queue for these packets). The goal was to
> > improve performance when you had separate up and downstream TCP
> > connections, especially when the upstream TCP connection delayed
> > the downstream TCP connections ACKs.
> > 
> > I never could find a lot of good details about this, e.g.: quantitiative
> > measurements, so i was wondering if others here on the list
> > are aware of it. Of course, if something like this would
> > be significantly useful even in the face of current TCP congestion
> > control mechanisms (and not only bad older ones).
> 
> I know some people who have run two queues, one for packets less than 200
> byte in size, and one for larger packets. They didn't even do any DPI, the
> setup was purely based on size of the packet. I am not aware of any deeper
> analysis on pros/cons of this setup, but these people were very happy with
> their setups for years and it solved their problems with a single large file
> transfer ruining their interactive performance, plus also "solved" the
> ACKs-get-stuck-behind-other-session-large-packets problem.
> 
> This was used for platforms with otherwise quite limited QoS capabilities,
> so flow based queueing wasn't available.
> 
> -- 
> Mikael Abrahamsson    email: [email-masked]

---

From: Toke Høiland-Jørgensen <[email-masked]>
Date: Fri, 13 Apr 2018 11:49:22 +0200

Toerless Eckert <[email-masked]> writes:

> Was wondering folks on ICCRG had any more insight into the state of
> research/understanding into this type of simple type of congestion
> experience improvement. I somehow fear it might be a lost simple trick
> that could help more people if wee would manage to make it become a
> BCP, but Mikaels reply/example showed me already how there are
> interesting variations, so i wonder if we even had a simple rule of
> thump safe enough to recommend as a BCP (and the constraints under
> which it makes sense).

Another variation is the sparse flow optimisation of FQ-CoDel:
Prioritise flows that don't build a queue. This usually also implies
prioritising small packets, but does not require setting a threshold;
and it dynamically adjusts to the link bandwidth.

Of course this requires some form of flow queueing. Not necessarily as
much as you think, though; we've had quite good performance with as
little as 32 queues...

-Toke

---

From: Luca Muscariello <[email-masked]>
Date: Fri, 13 Apr 2018 13:31:32 +0200

Toerless,

for flow queueing you might want to have a look at these then:

Latency reduction in home access gateways with shortest queue first
<https://scholar.google.com/scholar?oi=bibs&cluster=8307818605842678238&btnI=1&hl=fr>
N Benameur, F Guillemin, L Muscariello - Proc. ISOC Workshop on Reducing
Internet Latency, 2013

Giovanna Carofiglio
<https://dblp.uni-trier.de/pers/hd/c/Carofiglio:Giovanna>, Luca Muscariello:
On the Impact of TCP and Per-Flow Scheduling on Internet Performance.
IEEE/ACM Trans. Netw. 20(2)
<https://dblp.uni-trier.de/db/journals/ton/ton20.html#CarofiglioM12>:
620-633 (2012)

Self-prioritization of audio and video traffic
<https://scholar.google.com/scholar?oi=bibs&cluster=3937175170411326717&btnI=1&hl=fr>
T Bonald, L Muscariello, N Ostallo - … (ICC), 2011 IEEE International
Conference on, 2011

For the first reference we had a hardware implem in a MIPS with 32 buckets
for queues.

On Fri, Apr 13, 2018 at 11:49 AM, Toke Høiland-Jørgensen <[email-masked]>
wrote:

> _______________________________________________
> iccrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/iccrg
>

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 13 Apr 2018 13:46:25 +0200 (CEST)

> much as you think, though; we've had quite good performance with as

None of this is typically available on the ISP side of the customer 
connection.

The Cisco ASR9k is a decently popular device for these kinds of things. 
Here is a list of things that can be matched on:

RP/0/RSP0/CPU0:901-R1-2054(config-cmap)#match ?
   access-group         Match access group
   atm                  Match based on ATM specific criteria
   cac                  Match based on CAC fields
   cos                  Match based on IEEE 802.1Q/ISL Class Of Service value
   dei                  Match based on DEI bit (0/1)
   destination-address  Match based on destination address
   destination-port     Match based on destination port
   discard-class        Match based on discard class (upto 8 Ids)
   dscp                 Match based on IP DSCP value (upto 8 values or ranges)
   ethertype            Match based on ethertype (Upto 8 values or ranges)
   flow-key             Match based on flow keys
   fr-de                Match based on FrameRelay DE bit
   fragment-type        Fragment type for a packet
   frame-relay          Match based on frame-relay specific criteria
   ipv4                 Match based on ipv4 icmp
   ipv6                 Match based on IPv6
   mpls                 Match based on MPLS specific values
   not                  Negate match criteria
   packet               Match based on packet length
   precedence           Match based on IP precedence values
   protocol             Match based on L3 protocol (Upto 8 values or ranges)
   qos-group            Match based on QoS Group (upto 8 Ids or ranges)
   source-address       Match based on source address
   source-port          Match based on source port
   tcp-flag             Match based on TCP flags
   traffic-class        Match based on traffic class field (upto 8 Ids or ranges)
   vlan                 Match based on Vlan Ids (Upto 8 values or ranges)
   vpls                 Match based on VPLS fields

RP/0/RSP0/CPU0:901-R1-2054(config-cmap)#match packet length ?
   <0-65535>   Enter IP Packet length
   <0-65535>-  Lower limit of the packet length to match.
   ipv4        IPV4 Packet Length
   ipv6        IPV6 Packet Length

RP/0/RSP0/CPU0:901-R1-2054(config-cmap)#match source-port 4096-8192 ?
   <0-65535>   A Port Number
   <0-65535>-  Lower limit of the Port range to match.
   <cr>

You can then put whatever into a separate queue. Typically you want to 
have low single digit number of queues per customer, because these types 
of cards don't have huge amount of queues per customer. Typical things to 
match on that makes sense:

src/dst port (ranges)
src/dst addresses
packet size
diffserv bits

So with the above, I guess I could use src/dst ranges to map flows into 
different queues, or I could do packet size. So I guess I could interweave 
based on port numbers into a bunch of flows:

match source-port 1024-2047 8192-9215 12288-13311 16384-17407 .... etc
match source-port 2048-

etc, so I can put ranges of 1024 ports into different queues. This is 
available today. Flow based queueing is not.

So as Toerless asked, is anyone interested in actually doing research on 
solutions that are configurable today on ISP equipment, or do everybody 
just want to look at the shiny new things that won't make it into 
production gear in the near future?

I think it would be interesting to come out with guidelines of "this is a 
lot better than FIFO and can be configured on most gear", so ISPs stop 
doing large bufferbloated FIFOs and do something better. Today. Not in 5 
years.

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 13 Apr 2018 11:51:14 +0000

Hi

I may have missed parts of this discussion. But wouldn't fq-codel solve this ?

/Ingemar

> -----Original Message-----
> From: Toerless Eckert [mailto:[email-masked]]
> Sent: den 12 april 2018 22:10
> To: [email-masked]
> Cc: Mikael Abrahamsson <[email-masked]>
> Subject: [iccrg] Better QoS for TCP ACK question
> 
> experience improvement. I somehow fear it might be a lost simple trick that
> could help more people if wee would manage to make it become a BCP, but
> Mikaels reply/example showed me already how there are interesting
> variations, so i wonder if we even had a simple rule of thump safe enough to
> recommend as a BCP (and the constraints under which it makes sense).
> 
> If not, maybe this would be a lovely simple job for a student thesis for
> someone wanting to do NS2/NS3 simulation work in congestion control
> (primary issue is a good set of test cases and traffic mixes i guess).
> 
> Cheers
>     Toerless
> 
> > > A couple of years ago i was told that a range of home routers would
> > > do some DPI based diffserv QoS, aka prioritizing TCP ACP messages in
> > > the queue before any other packets 9sonded like a strict priority
> > > queue for these packets). The goal was to improve performance when
> > > you had separate up and downstream TCP connections, especially when
> > > the upstream TCP connection delayed the downstream TCP connections
> > > ACKs.
> > >
> > > I never could find a lot of good details about this, e.g.:
> > > quantitiative measurements, so i was wondering if others here on the
> > > list are aware of it. Of course, if something like this would be
> > > significantly useful even in the face of current TCP congestion
> > I know some people who have run two queues, one for packets less than
> > 200 byte in size, and one for larger packets. They didn't even do any
> > DPI, the setup was purely based on size of the packet. I am not aware
> > of any deeper analysis on pros/cons of this setup, but these people
> > were very happy with their setups for years and it solved their
> > problems with a single large file transfer ruining their interactive
> > performance, plus also "solved" the ACKs-get-stuck-behind-other-session-
> large-packets problem.
> >
> > This was used for platforms with otherwise quite limited QoS
> > capabilities, so flow based queueing wasn't available.
> >
> > --

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 13 Apr 2018 13:55:13 +0200 (CEST)

> I may have missed parts of this discussion. But wouldn't fq-codel solve this ?

Because FQ anything isn't available in the ISP CO side equipment.

It doesn't even do PIE. It does (W)RED and has a few queues per customer 
that can be statically configured with some matching criteria. See my 
other email on the topic.

I'm trying to give guidance to ISPs who today will probably have a single 
bufferbloated FIFO, what they can configure in their existing equipment 
with existing software/hardware.

---

From: Emmanuel Lochin <[email-masked]>
Date: Fri, 13 Apr 2018 13:58:13 +0200

Hi Toerless,

May be these contributions wouldalso be of interest:

Anelli, Pascal and Diana, Rémi and Lochin, Emmanuel /FavorQueue: A 
parameterless active queue management to improve TCP traffic 
performance./ (2014) Computer Networks, vol. 60. pp. 171-186. ISSN 
1389-1286 -> http://oatao.univ-toulouse.fr/10282/

and also a similar idea proposed here:

Chen, Jinbang; Heusse, Martin; Urvoy-Keller, Guillaume, /EFD: An 
efficient low-overhead scheduler/, 10th IFIP International Conference on 
Networking, May 9-13, 2011 -> 
http://www.eurecom.fr/fr/publication/3357/download/rs-publi-3357.pdf

EL

On 13/04/2018 13:31, Luca Muscariello wrote:
> N Benameur, F Guillemin, L Muscariello - Proc. ISOC Workshop on 
> Reducing Internet Latency, 2013
>
> Giovanna Carofiglio 
> <https://dblp.uni-trier.de/pers/hd/c/Carofiglio:Giovanna>,Luca 
> Muscariello:
> IEEE/ACM Trans.. Netw.20(2) 
> <https://dblp.uni-trier.de/db/journals/ton/ton20.html#CarofiglioM12>:620-633(2012)
>
>
> For the first reference we had a hardware implem in a MIPS with 32 
> buckets for queues.
>
>
>
>
>
> On Fri, Apr 13, 2018 at 11:49 AM, Toke Høiland-Jørgensen <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Toerless Eckert <[email-masked] <mailto:[email-masked]>> writes:
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.irtf.org/mailman/listinfo/iccrg>
>
>
>
>

-- 
Emmanuel LOCHIN
Professeur ISAE

ISAE SUPAERO - Institut Supérieur de l'Aéronautique et de l'Espace
10 avenue Edouard Belin - BP 54032 - 31055 TOULOUSE CEDEX 4 FRANCE - http://www.isae-supaero.fr
Tel +33 5 61 33 84 85 - Fax (+33) 5 61 33 83 30

---

From: Luca Muscariello <[email-masked]>
Date: Fri, 13 Apr 2018 14:04:42 +0200

In principle the best place to do this should be down to DSLAM/OLT/CMTS.
Doing this in the BNG, while convenient in some cases, could require too
many resources
while scaling up the number of users subscribed there.

There is a tradeoff between the available memory at each slot and the
hardware clocks.
If you have fiber users with Gbps subscription feasible user fan-out goes
really down at the BNG.

Are you looking for a feasible hack in today equipment that also scales?

On Fri, Apr 13, 2018 at 1:46 PM, Mikael Abrahamsson <[email-masked]>
wrote:

> Another variation is the sparse flow optimisation of FQ-CoDel: Prioritise
>> flows that don't build a queue. This usually also implies prioritising
>> small packets, but does not require setting a threshold; and it dynamically
>> adjusts to the link bandwidth.
>>
> have low single digit number of queues per customer, because these types of
> cards don't have huge amount of queues per customer. Typical things to

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 13 Apr 2018 14:21:47 +0200 (CEST)

> Are you looking for a feasible hack in today equipment that also scales?

Yes.

The ask from Toerless (which I agree with), is:

What are the user implications of setting up a two-queue customer access 
QoS profile where small packets (probably defining small packets as 
smaller than around 200 bytes) are sent into one queue, and larger into 
another. What should approximate capacity share be between these two. 
What's the difference on low and high speed access. What should the buffer 
depth be on each? What should the RED profiles be, if any?

There has been a lot of focus on the great next thing (new queueing 
algorithms etc), but I have seen very little in terms of "these are the 
tools we already have in the toolbox, how should we use them for best 
effect?"

---

From: Toke Høiland-Jørgensen <[email-masked]>
Date: Fri, 13 Apr 2018 14:43:17 +0200

> match source-port 1024-2047 8192-9215 12288-13311 16384-17407 .... etc

That might make sense as a sort of 'poor mans FQ' I guess.

If I understand this right you can't do anything other than exact
matching? Modulo or hashing is off the table, for instance?

I think it would make sense to explicitly put TCP SYN and SYNACKs and
DNS lookups into the "small packets" queue; the packet size restriction
would do that automatically for some of them, but not necessarily all. I
would do round-robin between all queues (DRR if possible, but I'm
guessing it isn't?).

> So as Toerless asked, is anyone interested in actually doing research
> on solutions that are configurable today on ISP equipment, or do
> everybody just want to look at the shiny new things that won't make it
> into production gear in the near future?
>
> I think it would be interesting to come out with guidelines of "this
> is a lot better than FIFO and can be configured on most gear", so ISPs
> stop doing large bufferbloated FIFOs and do something better. Today.
> Not in 5 years.

Look, all for improving things in the real world, and I'm as pleased as
anyone that operators are starting to pull their heads of out the sand
on this. But blaming the research community for focusing on the 'R' in
R&D is hardly productive. It used to be the case that
multi-billion-dollar corporations would actually allocate some resources
to the 'development' part themselves... :/

-Toke

---

From: Luca Muscariello <[email-masked]>
Date: Fri, 13 Apr 2018 14:53:08 +0200

For the queue depth, you should also specify, among the requirements, the
maximum RTT you want to support
for full utilisation of the subscribed user as you get the TCP BDP and
minimum queue depth.
A convenient assumption to do the math is that users with the same
subscribed rate are managed by the same BNG.

If you use the current queues as they are in each slot, and you use two per
user, you double the speed of the hardware clock
to visit them asynchronously. This gives you a constraint on user
max-shaping rate and max number of queues.

This is true for all BNGs based on NPUs.
All these constraints can be found in the manuals of ASR9K, 7550 or Juniper
or other vendors.
Last time I checked was three years ago.

About the two queues solution with static rates, I don't think it would
work well for any rate combination. It is DOSable.

On Fri, Apr 13, 2018 at 2:21 PM, Mikael Abrahamsson <[email-masked]>
wrote:

> QoS profile where small packets (probably defining small packets as smaller
> than around 200 bytes) are sent into one queue, and larger into another.
> What should approximate capacity share be between these two. What's the
> difference on low and high speed access. What should the buffer depth be on
> each? What should the RED profiles be, if any?
>

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 13 Apr 2018 15:08:40 +0200 (CEST)

> matching? Modulo or hashing is off the table, for instance?

In current software, it seems I cannot. It might be possible for vendor to 
implement some kind of hashing-group, but it's not there right now.

There is this:

RP/0/RSP0/CPU0:901-R1-2054(config-cmap)#match flow-key ?
   5-tuple     Five tuples to identify a flow
   dst-ip      Destination IP address
   flow-cache  Configure the flow-cache parameters
   src-ip      Source IP address
   <cr>

However, looking at the documentation this seems to be some kind of 
admission control based mechanism where you can let X 5-tuple flows into 
one queue, then it won't match any new flows into that same queue.

> would do round-robin between all queues (DRR if possible, but I'm

The way to solve that on this platform is basically to tell each queue 
that it can have 45% of the bandwidth, and you'll get something like that.

> to the 'development' part themselves... :/

Instead of calling it blaming, perhaps we can call it that I request some 
R based on what's in shipping products and how to configure it?

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 13 Apr 2018 15:10:03 +0200 (CEST)

> work well for any rate combination. It is DOSable.

More DOSable than a single stupid FIFO?

---

From: Luca Muscariello <[email-masked]>
Date: Fri, 13 Apr 2018 15:14:00 +0200

Yes. If I know you deterministically put all SYN, ACK, DNS packets in a
queue one can go hit that queue until it drops.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 13 Apr 2018 15:17:57 +0200 (CEST)

> queue one can go hit that queue until it drops.

Let's say I have a 100 megabit/s connection. I get hit with 5 gigabit/s of 
64 byte packets.

How is a two queue setup here (with small packets going into one, 
large into a second) worse than the single queue setup? For any relevant 
amount of "worse" here?

---

From: Luca Muscariello <[email-masked]>
Date: Fri, 13 Apr 2018 15:25:14 +0200

It's not just a question of total rate.
You just need to increase the drop rate to ACK and SYN to create damage.
And you want to put them all in the same queue.
It's just easier to shoot them as they are all lined up in the same queue.
And you don't need 5Gbps to do that.

In a FIFO you would need to saturate the link, so the attacker has to have
more resources and it is also easier to detect and then mitigate.

On Fri, Apr 13, 2018 at 3:17 PM, Mikael Abrahamsson <[email-masked]>
wrote:

> How is a two queue setup here (with small packets going into one, large
> into a second) worse than the single queue setup? For any relevant amount
> of "worse" here?
>
>
> --

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 13 Apr 2018 15:27:58 +0200 (CEST)

On Fri, 13 Apr 2018, Luca Muscariello wrote:

> In a FIFO you would need to saturate the link, so the attacker has to 
> have more resources and it is also easier to detect and then mitigate.

With the 2 queue setup you need to saturate half the link instead of the 
full link, but I don't see the huge difference in that.

DDOS today is a few USD for magnitude higher speed than anyone typically 
has as access-speed, which will break any access, especially a FIFO.

---

From: Luca Muscariello <[email-masked]>
Date: Fri, 13 Apr 2018 15:37:44 +0200

So you know that you split the capacity 50-50. How do you get to that
number?
There are users with 1Gbps accesses today (like me). I would disagree to
get a cut of 50% to get lower latency.

Even if is 50-50, I just need 50% of the rate to kill all ACK and SYN to
kill the other 50% as no TCP connection would even work w/o the missing SYN
and ACKs.
Suppose you can mitigate attacks, and  you do 50-50 most of the time the
ACK queue is empty. So the cost to get low latency seems rather high.

---

From: Toke Høiland-Jørgensen <[email-masked]>
Date: Fri, 13 Apr 2018 16:11:24 +0200

Mikael Abrahamsson <[email-masked]> writes:

> admission control based mechanism where you can let X 5-tuple flows
> into one queue, then it won't match any new flows into that same
> queue.

Right; that doesn't seem terribly useful, no.

> that it can have 45% of the bandwidth, and you'll get something like

Is that a fixed 45%? I.e., if only one queue is active, is it throttled
to half the bandwidth? You really want queues to be able to borrow
bandwidth from one another...

-Toke

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 13 Apr 2018 16:12:04 +0200 (CEST)

> So you know that you split the capacity 50-50. How do you get to that

That's just a hunch. In my test (you can find email chain in AQM) was to 
set up 45/45/5 between all-the-rest/BE/LE.

https://www.ietf.org/mail-archive/web/aqm/current/msg02143.html

> get a cut of 50% to get lower latency.

That's not what's happening.

> ACK queue is empty. So the cost to get low latency seems rather high.

There is no such cost. If the ACKs use 5% of BW, then the rest (95%) can 
be used by large packets.

It's only that the queues are guaranteed 50% if they need it. Any spare 
unused capacity can be used by the other queue.

We can argue the exact numbers/split, but that's what should be 
researched.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 13 Apr 2018 16:20:40 +0200 (CEST)

> bandwidth from one another...

They can borrow any unused capacity. The 45% is assured part of the total, 
if one doesn't use it, it can be used by others to go over their 
assurances.

---

From: Luca Muscariello <[email-masked]>
Date: Fri, 13 Apr 2018 16:29:35 +0200

I was describing a DoS example were the small packet queue is always at
full utilization so you cannot borrow.
Whatever is the ceil rate for the ACK queue, it is still DoSable. Whatever
the fraction is. The attacker doesn't need to know
the ceil rate but just how to forge packets, sizes in this case and rump up
until it drops.

On Fri, Apr 13, 2018 at 4:20 PM, Mikael Abrahamsson <[email-masked]>
wrote:

> Is that a fixed 45%? I.e., if only one queue is active, is it throttled to
>> half the bandwidth? You really want queues to be able to borrow bandwidth
>> from one another...
>>
>
> if one doesn't use it, it can be used by others to go over their assurances.
>
>
> --

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 13 Apr 2018 17:05:11 +0200

Thanks, Luca

Are you aware if any microcode of home gateways chips ever implemented SQF ?

Traditional CPU forwarded higher end (not home gateway) cpe devices supported WFQ.
>From my understanding WFQ should perform better than DRR, so comparison
SQF/WFQ would have been interesting. SQF sounding in pricniple of course
better, but wouldn't dare to predict actual results. Of course, WFQ is typically not
supported in microcode based gateways.

Independnt of this, the question on the thread boils down to what Mikhael
already said, e.g.: can we get a way with a simple 2 queue system through
appropriate simple classification and get reasonable results.. And if
so, whats the best classification (packet size ?).

Cheers
    Toerless

On Fri, Apr 13, 2018 at 01:31:32PM +0200, Luca Muscariello wrote:
> T Bonald, L Muscariello, N Ostallo - ??? (ICC), 2011 IEEE International
> Conference on, 2011
> 
>

---

From: Luca Muscariello <[email-masked]>
Date: Fri, 13 Apr 2018 17:26:16 +0200

> Are you aware if any microcode of home gateways chips ever implemented SQF

Microcode for Ikanos MIPS (acquired  and closed by Qualcomm I think),
commissioned by and property of  Orange SA.

> so, whats the best classification (packet size ?).

I'm not aware of any solution that can do that w/o modifying dequeue()
or w/o an enqueue() that has knowledge of the current status of virtual
queues, e.g.
Is this flow hash already present in the current queue? If you can verify
this test
you could send traffic in the secondary queue iff there is a match in the
primary queue.
But this is not something you can do with just classification.
This solution is not DoSable.

> buckets

---

From: Luca Muscariello <[email-masked]>
Date: Fri, 13 Apr 2018 18:12:24 +0200

Errata
 If you can verify this test
you could send traffic in the secondary queue iff there isNT a match in the
primary queue.

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 13 Apr 2018 18:14:19 +0200

> This solution is not DoSable.

Well, this would be for interfaces to an indivudal residential
subscriber, so DoS would be one application in te home being
unfair against another app. And if DoS happens, it wouldn't
be worse than without the feature. So if no better solution
can be configured, this is an ok. Limitation. Not much different
from not having no aqm at all in a single BE queue and a badly
behacving flow in that queue (eg: inelastic), right ? 

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Luca Muscariello <[email-masked]>
Date: Fri, 13 Apr 2018 17:11:07 +0000

DoS is an external source sending  traffic to the home in the ACK queue to
induce losses. This is worse than just unfairness.
Spoof the IP address of the gateway.

The point is never enter the secondery queue if you are bottlenecked in the
primary queue.
Otherwise the secondary queue becomes DoSable.

On Fri, Apr 13, 2018 at 6:14 PM, Toerless Eckert <[email-masked]> wrote:

> > > > <https://dblp.uni-trier.de/db/journals/ton/ton20.html#CarofiglioM12
> >:
> > > > 620-633 (2012)
> > > >
> > > >

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 13 Apr 2018 19:30:59 +0200

> Spoof the IP address of the gateway.

Ok. This would apply to upstream traffic from the subscriber whose
downstream ACKs in the queue of the SP router (BRAS etc.) gets DOS'ed.

But so then could any per-flow QoS solution be DoS'ed by forcing the
eg: BRAS to create too many downstream queues, right ? If i can crearte
enough packets to fill a queue, i can create enough packets to make it
look like too many flows ?!

> Otherwise the secondary queue becomes DoSable.

Uhhmmm...... thinking... thinking failed.. ;-(
can you rephrase maybe with example ? What does "never enter" means ?

Thanks
    Toerless

-- 
---
[email-masked]

---

From: Luca Muscariello <[email-masked]>
Date: Fri, 13 Apr 2018 19:19:32 +0000

The attack you describe, if successfull, makes flow-queuing behave globally
as FIFO.

The attack I was referring to,if seccesful, makes the double queue  induce
high loss rate on ACK, SYN DNS etc.  Which is worse.

In an ideal  double queues system, not the one you described, you would
need to enqueue packets in the secondary queue trying to give lower latency
service wrt to the primary queue.
At least this is what I understand from the conversation.

Enqueue to the secondary queue, to be effective and non  DoSable should be
possible only if the given flow hash is not already bottlenecked in the
primary queue. Meaning the flow has several packets in the primary queue.
Flows that fall in that category are ACK, SYN etc. But the hash is still on
a per-flow basis. The hash cannot just be ACK or SYN no matter which flow.
This is an approximation which requires more than just classification
because the packet dispatcher on the enqueue is flow aware and sees the
flow  status of queues.

You need a flow table or a counting bloom filter or another set membership
mechanism with updates.

On Fri 13 Apr 2018 at 19:31, Toerless Eckert <[email-masked]> wrote:

> > > > > > <
> https://dblp.uni-trier.de/db/journals/ton/ton20.html#CarofiglioM12
> > > >:
> > > > > > 620-633 (2012)
> > > > > >
> > > > > >
> state
> > > of
> > > > > > > > experience improvement. I somehow fear it might be a lost
> simple
> > > > > trick
> > > > > > > > interesting variations, so i wonder if we even had a simple
> rule
> > > of

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 14 Apr 2018 07:53:08 +0200 (CEST)

On Fri, 13 Apr 2018, Luca Muscariello wrote:

> The attack I was referring to,if seccesful, makes the double queue 
> induce high loss rate on ACK, SYN DNS etc.  Which is worse.

Agreed, but I don't care if someone shoots me 100 times or 200 times. 200 
times is worse, but I'll be dead by 100 shots anyway.

My point is that it's trivial today to ddos someone with 10-100x their 
normal access speed and then it doesn't matter if you have 1 stupid FIFO 
or 2 stupid FIFOs where selection between these two FIFOs is based on 
packet size.

Or what am I missing?