Subject: Re: [aqm] aqm Digest, Vol 1, Issue 7

From: [email-masked]
Date: Tue, 19 Mar 2013 23:02:20 -0400 (EDT)

A problem with ECN is that it does not drain an already flooded queue, and depends (like RED) on the idea that one has a large queue in normal operation.  It is not obvious that sustaining a deep queue is a desirable state, especially when there are highly variable capacities along any path, as is the case where one link is more than 2x slower than the rest of the links.
 
If the dynamics of user loads are fractal, this will cause draining (and latency reduction) not to work.
 
If user loads are stationary gaussian and so forth, ECN can "tune" to relatively stable situations, but the queueing delay will be quite long end-to-end.
 
Since there is lots of evidence that user loads are fractal, dropping packets during sudden bursts will prevent sustained queueing, whereas ECN will not.
 
So a proper approach (in my opinion) on the "real" Internet (rather than a bunch of competing long-duration FTPs) requires keeping the queues so short that there is no real opportunity to benefit from ECN.
 
Hence - we need to really see what happens with *real* traffic loads, not simplified simulations.
 
Real traffic loads are fractal, and fractal loads do not generally obey the Gaussian style "law of large numbers" - multiple fractal flows are *rougher* not smoother than a single one.
 
So check your mathematical "intuitions" at the door.  Stop writing equations based on tractable analytic models, because actual traffic does not follow those models.
 
Yeah, the Bell Heads believe in never dropping packets.  So what.  The Internet is not about making them comfortable in their misunderstandings.
 
-----Original Message-----
From: [email-masked]
Sent: Tuesday, March 19, 2013 7:43pm
To: [email-masked]
Subject: aqm Digest, Vol 1, Issue 7

If you have received this digest without all the individual message
attachments you will need to update your digest options in your list
subscription.  To do so, go to 

https://www.ietf.org/mailman/listinfo/aqm

Click the 'Unsubscribe or edit options' button, log in, and set "Get
MIME or Plain Text Digests?" to MIME.  You can set this option
globally for all the list digests you receive at this point.

Send aqm mailing list submissions to
 [email-masked]

To subscribe or unsubscribe via the World Wide Web, visit
 https://www.ietf.org/mailman/listinfo/aqm
or, via email, send a message with subject or body 'help' to
 [email-masked]

You can reach the person managing the list at
 [email-masked]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of aqm digest..."
Today's Topics:

 1. Re: [tcpm] ECN support and usage on the Internet (Yuchung Cheng)
 2. Re: ECN support and usage on the Internet (Fred Baker (fred))
 3. Re: ??: [iccrg] [tcpm] ECN support and usage on the Internet
 (Jim Gettys)
 4. Re: ??: [iccrg] [tcpm] ECN support and usage on the Internet
 (Simon Barber)
_______________________________________________
aqm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/aqm

---

From: John Leslie <[email-masked]>
Date: Tue, 19 Mar 2013 23:56:45 -0400

[email-masked] <[email-masked]> wrote:
> 
> A problem with ECN is that it does not drain an already flooded queue,

   I presume you mean that ECN doesn't drop the packet it marks.

> and depends (like RED) on the idea that one has a large queue in normal
> operation.

   I don't follow this at all. ECN is pointless without an AQM which
drops or marks well before tail-drop. Were we to run ECN without AQM,
we'd have to drop each packet anyway after marking it.

   Several flavors of AQM mark/drop well enough in advance of tail-drop
that the queue need never fill. Did you mean to say that ECN requires
being able to queue the ECN-marked packet which would otherwise be
dropped by the AQM?

> It is not obvious that sustaining a deep queue is a desirable state,

   Agreed!

> especially when there are highly variable capacities along any path,
> as is the case where one link is more than 2x slower than the rest of
> the links.

   Isn't that _always_ the case?

> (and latency reduction) not to work.

   I'm guessing you mean that under "fractal" loads there's a tendency
for ECN-marked packets to clump and introduce latency at the bottleneck.
Recall that this is offset by the quicker response to an ECN mark at
RTT speed instead of having to infer a packet loss.

> If user loads are stationary gaussian and so forth, ECN can "tune"
> to relatively stable situations, but the queueing delay will be quite
> long end-to-end.

   I don't follow... Some AQMs may indeed allow queueing delay to grow
large; but this can only happen if the instantaneous percentage of
marked packets is quite a bit higher than TCP can tolerate.

> Since there is lots of evidence that user loads are fractal, dropping
> packets during sudden bursts will prevent sustained queueing, whereas
> ECN will not.

   I believe there is substantial agreement that ECN-marked packets
will need to be dropped anyway under particularly severe congestion.
At the very least, tail-drop can become necessary...

> So a proper approach (in my opinion) on the "real" Internet (rather
> than a bunch of competing long-duration FTPs) requires keeping the
> queues so short that there is no real opportunity to benefit from ECN.

   Here I quite disagree.

   An AQM could be designed to ECN-mark at a 1 millisecond queue depth;
and that can give the full benefit of ECN. Again, recall that ECN will
deliver the congestion signal in one RTT, instead of waiting to infer
a loss. TCP stops delivering useful bandwidth at one or two percent
packet loss: so it's hard to believe that ECN-marked packets will clog
the path significantly.

> not simplified simulations.

   Agreed! (Now if we could just get enough ECN deployment...)

> Real traffic loads are fractal, and fractal loads do not generally
> obey the Gaussian style "law of large numbers" - multiple fractal
> flows are *rougher* not smoother than a single one.

   I would say rather that real traffic loads _contain_ fractal flows.
Aggregation simply wouldn't work if we never experienced gaussian
smoothing.

> So check your mathematical "intuitions" at the door.  Stop writing
> equations based on tractable analytic models, because actual traffic
> does not follow those models.

   I strongly suspect our models _are_ faulty -- but we need research
results for _actual_ flows, with and without ECN, is order to diagnose
their faults.

--
John Leslie <[email-masked]>

---

From: [email-masked]
Date: Wed, 20 Mar 2013 09:46:38 -0400 (EDT)

John - I understand your response, and I understand the idea of doing ECN marking at the 1 msec. level.  That certainly would increase the amount of congestion information delivered to the endpoints.
 
However, if you think about how long it will take for the receiver to reduce its "window" in typical cases, along with the "reluctance" of router designers to ever drop packets, I think this does not solve the problem.
 
Yes, ECN might help a bit in the case where there are *large* numbers of stationary-process TCP flows traversing a node, leading to nearly full capacity.
 
But one should focus on the actually difficult problems: e.g. what went wrong in DOCSIS 2.0 and early DOCSIS 3 deployments, still in the field, that made queueing delays close to 1 second typical from a single home.
 
Because there are relatively small numbers of flows, *way* too much buffering, leading to sustained queues that could not drain fast enough under ECN or anything else, user experience *sucked*.
 
Which is why Comcast tried "killing BitTorrent" with Sandvine DPI-based RST-injection, and started claiming it was to "stop music piracy, which was destroying the Internet".
 
When Comcast was forced to actually look at why the queues were sustained and unfixable, they realized that the problem was that DOCSIS was *designed* to hold packets rather than drop them.  ECN marking would probably not have helped get the queues down under 50 msec, without dropping packets, neither would RED.  The only way to get the queue down is to prevent it from being created in the first place....  And since the RTT from US east coast to US west coast is under 45 msec, it does little good to send early congestion signals.
 
I'm sure you know this.
 
An analogous problem occurs today because of the 3-4 *second* buffering in HSPA and LTE networks under load.  Having measured it in many American and European deployments of such equipment, I would suggest that the fundamental problem is *dropping packets fairly and quickly*.  You can't get there (to sub-50 msec. queueing) with ECN or RED.
 
-----Original Message-----
From: "John Leslie" <[email-masked]>
Sent: Tuesday, March 19, 2013 11:56pm
To: [email-masked]
Cc: [email-masked]
Subject: Draining queues

 Agreed!

 Here I quite disagree.

---

From: Jim Gettys <[email-masked]>
Date: Wed, 20 Mar 2013 12:25:06 -0400

On Wed, Mar 20, 2013 at 9:46 AM, <[email-masked]> wrote:

> John - I understand your response, and I understand the idea of doing ECN
> marking at the 1 msec. level.  That certainly would increase the amount of
> congestion information delivered to the endpoints.
>
>
>
> However, if you think about how long it will take for the receiver to
> reduce its "window" in typical cases, along with the "reluctance" of router
> designers to ever drop packets, I think this does not solve the problem.
>
>
>
> Yes, ECN might help a bit in the case where there are *large* numbers of
> stationary-process TCP flows traversing a node, leading to nearly full
> capacity.
>
>
>
> But one should focus on the actually difficult problems: e.g. what went
> wrong in DOCSIS 2.0 and early DOCSIS 3 deployments, still in the field,
> that made queueing delays close to 1 second typical from a single home.
>
>
>
> Because there are relatively small numbers of flows, *way* too much
> buffering, leading to sustained queues that could not drain fast enough
> under ECN or anything else, user experience *sucked*.
>
>
>
> Which is why Comcast tried "killing BitTorrent" with Sandvine DPI-based
> RST-injection, and started claiming it was to "stop music piracy, which was
> destroying the Internet".
>
>
>
> When Comcast was forced to actually look at why the queues were sustained
> and unfixable, they realized that the problem was that DOCSIS was
> *designed* to hold packets rather than drop them.  ECN marking would
> probably not have helped get the queues down under 50 msec, without
> dropping packets, neither would RED.  The only way to get the queue down is
> to prevent it from being created in the first place....  And since the RTT
> from US east coast to US west coast is under 45 msec, it does little good
> to send early congestion signals.
>
>
>
> I'm sure you know this.
>
>
>
> An analogous problem occurs today because of the 3-4 *second* buffering in
> HSPA and LTE networks under load.  Having measured it in many American and
> European deployments of such equipment, I would suggest that the
> fundamental problem is *dropping packets fairly and quickly*.  You can't
> get there (to sub-50 msec. queueing) with ECN or RED.
>

First off: everyone has made the same mistake: operating system writers,
DSL, and GPON, just to name a few.  Don't single out cable.  We've all
screwed up for a long time; in this sense, we're all "Bell Heads".

With flow queuing such as fq_codel or sfqcodel, the ECN equation changes:
other flows don't get held up the way they do when running a simple AQM
such as RED in a single queue.

And the web's behavior means that any straight AQM such as CoDel, RED or
PIE *cannot* get latency anywhere where it needs to be: too many packets
end up in flight at once to get stuck in any queue; we *have* to do
something to solve this problem: flow queuing makes a huge difference.
 I've measured 100's of milliseconds of transient latency hitting some web
sites as these packet flights hit the broadband edge.  This becomes a
non-issue with fq_codel.

Even so, Dave Taht's experiments (IIRC) were that for low bandwidth, drop
was preferable for ECN as even a single full sized packet takes
considerable time.

Exactly how this is going to play out is really only something we can
figure out by experiment.

I also need to clarify what I mean by "Bell Heads": these people are *not*
restricted to the telecom industry, but everywhere.  In fact, Dave Taht
first encountered "every packet is precious" in working on 802.11, where
the driver writers seemed to think that ever dropping even one packet was
evil, and would retransmit effectively infinitely.  So those of you working
on cellular wireless are in good company, and don't take this personally.
                                                   - Jim

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 20 Mar 2013 16:31:42 +0000

Well,

there are those who believe that marking/dropping should ONLY be done on enqueue into a queue (or at least that there is no difference between doing it at enqueue vs. dequeue),

http://bramcohen.com/2012/05/07/tcp-sucks#comment-669641053

and think a packet once processed deserves delivery – potentially breaking, or at least delaying, the control loop on responsive transport protocols (TCP being the prime example).

As Matt M. pointed out, the control loops of an end-to-end transport are an entangled mess. Therefore it is pointless to speculate what one minor tweak will have, without also considering the feedback effects this has on the overall whole…

ECN by itself is only part of the solution (and there are discussion to achieve more accurate ECN feedback in TCP, for a number of reasons), ECN can not live without a proper AQM, and a responsive (!) transport protocol.

Best regards,

Richard Scheffenegger

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Mittwoch, 20. März 2013 14:47
To: John Leslie
Cc: [email-masked]
Subject: Re: [aqm] Draining queues

I'm sure you know this.

-----Original Message-----
From: "John Leslie" <[email-masked]<mailto:[email-masked]>>
Sent: Tuesday, March 19, 2013 11:56pm
To: [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Draining queues

Agreed!

Here I quite disagree.

--
John Leslie <[email-masked]<mailto:[email-masked]>>

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Mar 2013 09:40:57 -0700

On 3/20/2013 9:31 AM, Scheffenegger, Richard wrote:
> ECN by itself is only part of the solution (and there are discussion to
> achieve more accurate ECN feedback in TCP, for a number of reasons), ECN
> can not live without a proper AQM, and a responsive (!) transport protocol

Don't forget a decent scheduler and polling-based processing too; 
latency reduction is a systems issue, not just a networking one.

Joe

---

From: [email-masked]
Date: Wed, 20 Mar 2013 17:26:12 -0400 (EDT)

Joe's right, at least historically: Once upon a time, one of the problems the "receive window" was thought to solve was operating systems that had very, very long response times and expensive process switching, so there was some benefit for the endpoints in "batching" up lots of data before sending and before receiving.
 
One way to think of this is that the bulk of the queueing delay on an end-to-end basis was spent with the packets sitting in operating system buffers, so one could optimize cost of process switching and/or deal with time quanta on the order of 50-100 msec. in the endpoint schedulers.  It was especially true in the days when the dominant content was "character-at-a-time" remote logins and the user could type much faster than the remote system could echo.
 
Such systems may still exist - multiprocessing systems where processors are kind of slow, memory is relatively free, etc.
 
I've never been sure that making TCP's design include these endpoint-buffering features was a good idea, since it encouraged managing the end-to-end flow with an incredibly slow and variable control response.
 
Put this kind of buffering in the layer above TCP, at least in a logical sense, by not counting these buffers as part of the "window".  Then you never end up letting the TCP window get big just because the recipient needs buffering to cover for its weak process scheduler. It's a *transmission* control protocol, not an Operating Systems workload scheduling protocol.
 
-----Original Message-----
From: "Joe Touch" <[email-masked]>
Sent: Wednesday, March 20, 2013 12:40pm
To: "Scheffenegger, Richard" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>, "John Leslie" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [aqm] Draining queues

Joe

---

From: Mirja Kuehlewind <[email-masked]>
Date: Tue, 26 Mar 2013 16:02:31 +0100

Hi,

it is actually a feature of AQM/ECN that you have small standing queues when 
having persistent flows but short flow are able to fill the queue (for a 
short period only) without getting packet losses immediately!

Mirja

On Wednesday 20 March 2013 04:02:20 [email-masked] wrote:
> A problem with ECN is that it does not drain an already flooded queue, and
> depends (like RED) on the idea that one has a large queue in normal
> operation.  It is not obvious that sustaining a deep queue is a desirable
> state, especially when there are highly variable capacities along any path,
> as is the case where one link is more than 2x slower than the rest of the
> links.
>
> If user loads are stationary gaussian and so forth, ECN can "tune" to
> relatively stable situations, but the queueing delay will be quite long
> end-to-end.
>
> packets during sudden bursts will prevent sustained queueing, whereas ECN
> will not.
>
> So a proper approach (in my opinion) on the "real" Internet (rather than a
> bunch of competing long-duration FTPs) requires keeping the queues so short
> that there is no real opportunity to benefit from ECN.
>
> Real traffic loads are fractal, and fractal loads do not generally obey the
> Gaussian style "law of large numbers" - multiple fractal flows are
> *rougher* not smoother than a single one.
>
> equations based on tractable analytic models, because actual traffic does
> not follow those models.
>
> Yeah, the Bell Heads believe in never dropping packets.  So what.  The
> Internet is not about making them comfortable in their misunderstandings.
>

-- 
-------------------------------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Institute of Communication Networks and Computer Engineering (IKR)
University of Stuttgart, Germany
Pfaffenwaldring 47, D-70569 Stuttgart

tel: +49(0)711/685-67973
email: [email-masked]
web: www.ikr.uni-stuttgart.de
-------------------------------------------------------------------

---

From: Jim Gettys <[email-masked]>
Date: Tue, 26 Mar 2013 12:18:27 -0400

> short period only) without getting packet losses immediately!
Exactly what the tradeoffs we're going to have when we throw flow queuing
into the mix isn't clear to me as yet.  It's one thing when the packet's
latency hurts every flow; different when it's just the flow building the
queue.

I don't think anyone has looked into this at all.
                                    - Jim

> and

---

From: [email-masked]
Date: Tue, 26 Mar 2013 12:46:27 -0400 (EDT)

And I would ask: why is this a feature?  It's a bug, IMHO.   The motivation, I understand.  The idea seems to be that the standing queue allows a "stateless" measurement to have enough information to assess what it might do.   But why not just remember what packets had  passed through and had been dropped.  That would be like a "queue" without a queue's bad side-effect properties of holding up traffic and slowing recovery from the congested state.
 
It's like saying that to manage highway traffic congestion, first we must create a sustained traffic jam that never goes away, but adds significant latency to every driver's trip - solely to measure congestion - before it becomes self-sustaining!
 
The problem with many actual situations (the ones where bufferbloat happens) is that there are processes that create self-sustaining congestion.
 
The best queue is the shortest possible queue, and there are techniques to achieve that - fq_codel with aggressive dropping does not need to build up a standing queue to do its work.  ECN does. RED does.  They are therefore inferior on a key quality of experience metric.
 
In passing, let me point out what many have pointed out:  Optimizing TCP goodput is NOT the correct measure to apply to the Internet.   Yes, ECN is good for that.  But it does NOT fix the real user problems.  This is what Raj Jain didn't seem to understand: the Internet is about latency, above all, because it is about both human-computer interaction and computer-computer (distributed systems) interaction.  It is not a set of "tubes" that are unto themselves and measurable by goodput.
 
The optimal state of a highway system in terms of "cars per hour" (optimum goodput) is, unfortunately, really bad for the cars - because it is a highway system in a state of constant traffic jam, and where the latency of any particular car is not minimized - it is actually far, far from the minimum.
 
That being said, the Internet, though different from the highway system, shares this problem.  If you tune for "goodput", you will have a completely congested system, where users will *walk away* and abandon interactions leaving them in the queues.   We see that in HSPA and LTE networks today, live.  It's not "interference" that causes this on wireless networks, it's been demonstrated that it is queueing delay that is allowed to get too large, because "goodput" is the only metric used.  Users don't want the last 10% of goodput, if it causes them a factor of 100 in response time.  (easily happening if there are 10 handshakes in a protocol step, and each is 10x longer from a latency perspective).
 
-----Original Message-----
From: "Mirja Kuehlewind" <[email-masked]>
Sent: Tuesday, March 26, 2013 11:02am
To: [email-masked]
Cc: [email-masked]
Subject: Re: [aqm] aqm Digest, Vol 1, Issue 7

Hi,

Mirja

tel: +49(0)711/685-67973

---

From: Mirja Kuehlewind <[email-masked]>
Date: Tue, 26 Mar 2013 18:06:25 +0100

Hi,

you claim that the Internet is only about latency. That's is as wrong as to 
say it's only about goodput. It really depends on the service you want to use 
over the Internet. Some might care about latency only and some about goodput 
(as few as possible packet drops only). Today's Internet is mainly optimized 
for goodput though. To realize both requirement at the same time is actually 
hard. What's the solution? Do we need two Internets?

Mirja

On Tuesday 26 March 2013 17:46:27 [email-masked] wrote:
> And I would ask: why is this a feature?  It's a bug, IMHO.   The
> motivation, I understand.  The idea seems to be that the standing queue
> allows a "stateless" measurement to have enough information to assess what
> it might do.   But why not just remember what packets had  passed through
> and had been dropped.  That would be like a "queue" without a queue's bad
> side-effect properties of holding up traffic and slowing recovery from the
> congested state.
>
> It's like saying that to manage highway traffic congestion, first we must
> create a sustained traffic jam that never goes away, but adds significant
> latency to every driver's trip - solely to measure congestion - before it
> becomes self-sustaining!
>
> The problem with many actual situations (the ones where bufferbloat
> happens) is that there are processes that create self-sustaining
> congestion.
>
> The best queue is the shortest possible queue, and there are techniques to
> achieve that - fq_codel with aggressive dropping does not need to build up
> a standing queue to do its work.  ECN does. RED does.  They are therefore
> inferior on a key quality of experience metric.
>
> In passing, let me point out what many have pointed out:  Optimizing TCP
> goodput is NOT the correct measure to apply to the Internet.   Yes, ECN is
> good for that.  But it does NOT fix the real user problems.  This is what
> Raj Jain didn't seem to understand: the Internet is about latency, above
> all, because it is about both human-computer interaction and
> computer-computer (distributed systems) interaction.  It is not a set of
> "tubes" that are unto themselves and measurable by goodput.
>
> The optimal state of a highway system in terms of "cars per hour" (optimum
> goodput) is, unfortunately, really bad for the cars - because it is a
> highway system in a state of constant traffic jam, and where the latency of
> any particular car is not minimized - it is actually far, far from the
> minimum.
>
> That being said, the Internet, though different from the highway system,
> shares this problem.  If you tune for "goodput", you will have a completely
> congested system, where users will *walk away* and abandon interactions
> leaving them in the queues.   We see that in HSPA and LTE networks today,
> live.  It's not "interference" that causes this on wireless networks, it's
> been demonstrated that it is queueing delay that is allowed to get too
> large, because "goodput" is the only metric used.  Users don't want the
> last 10% of goodput, if it causes them a factor of 100 in response time. 
> (easily happening if there are 10 handshakes in a protocol step, and each
> is 10x longer from a latency perspective).
>
> it is actually a feature of AQM/ECN that you have small standing queues
> when having persistent flows but short flow are able to fill the queue (for
> a short period only) without getting packet losses immediately!
>
> Mirja
>
> > state, especially when there are highly variable capacities along any
> > path, as is the case where one link is more than 2x slower than the rest
> > of the links.
> >
> > So a proper approach (in my opinion) on the "real" Internet (rather than
> > a bunch of competing long-duration FTPs) requires keeping the queues so
> > short that there is no real opportunity to benefit from ECN.
> >
> > Real traffic loads are fractal, and fractal loads do not generally obey
> > the Gaussian style "law of large numbers" - multiple fractal flows are

tel: +49(0)711/685-67973

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 26 Mar 2013 18:27:36 +0100 (CET)

On Tue, 26 Mar 2013, [email-masked] wrote:

> The optimal state of a highway system in terms of "cars per hour" 
> (optimum goodput) is, unfortunately, really bad for the cars - because 
> it is a highway system in a state of constant traffic jam, and where the 
> latency of any particular car is not minimized - it is actually far, far 
> from the minimum.

Analogies are usually hard, but I have been told by a traffic system 
professor that the optimal cars/hour speed is ~70 km/h, that's when the 
flow is the highest. Faster than that, and cars increase the inter-car 
distance more than the cars/second increases, slower than that and flow 
suffers. So as traffic increases from an nearly empty road, speed 
decreases until everybody drives ~70 km/h, then any small event causes the 
whole system to break down and traffic starts to back up.

Now, I don't have a good analogy to this and fq_codel but giving every 
flow "fair access" to the medium sounds to me like a good solution. I 
don't really care if I for some combination of flows and latencies can 
only use 80-90% of theoretical maximum, it's better to keep performance 
under high load fairly good than to wreck interactive flows in name of 
100% usage for aggressive flows.

I have 10 years experience in trying to fault find performance problems 
for customers in real life. I have been using fair-queue for 8 years on my 
home connection because I don't like bufferbloat (200ms of buffering was 
quite common even on my fairly high capacity connection). It uses quite a 
lot of CPU on cisco routers, so fq_codel sounds to me to be a great 
compromise between simplicity and still getting the job done. However, I'd 
imagine it's the "fq" part that is of more importance than "codel", but 
together they sound like a great combination.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 26 Mar 2013 17:42:06 +0000

Hi,

> And I would ask: why is this a feature?  It's a bug, IMHO.   

You might have the wrong metaphor in mind to come to this conclusion...

> The motivation, I 
> understand.  The idea seems to be that the standing queue allows a "stateless"
> measurement to have enough information to assess what it might do.   

No, a small standing queue (and small means an average queue length of 1+-1 packet) is necessary to fully utilize a packet network.

> But why not just 
> remember what packets had  passed through and had been dropped.  That would be like a 
> "queue" without a queue's bad side-effect properties of holding up traffic and slowing 
> recovery from the congested state.

I think this concept is called virtual queue - you drain a shadow (virtual) queue at slightly lower rate (ie. 95-99%) than the physical one, thus legacy AQM like RED can react *before* the actual queue starts building. However, this property is bought by less than full utilization (in steady state) of the network...

> 
> It's like saying that to manage highway traffic congestion, first we must create a 
> sustained traffic jam that never goes away, but adds significant latency to every 
> driver's trip - solely to measure congestion - before it becomes self-sustaining!
>

As Van pointed out, cars on a highway are not necessarily the correct metaphor to grasp whats going on in packet networks.

http://www.ietf.org/proceedings/84/slides/slides-84-tsvarea-4.pdf

>
> The problem with many actual situations (the ones where bufferbloat happens) is that 
> there are processes that create self-sustaining congestion.

Well, one can also state, that if these processes are reactive, they don't get timely feedback to avoid this from happening. (There will always be non-responsive UDP blasts that cann't be cured really).

> The best queue is the shortest possible queue, and there are techniques to achieve that
> - fq_codel with aggressive dropping does not need to build up a standing queue to do
> its work.  ECN does. RED does.  They are therefore inferior on a key quality of
> experience metric.

ECN by itself is not a AQM scheme but only a signaling scheme. Also, the AQM scheme doesn't prevent the buildup - it's a reactive sender (which reacts properly to a signal from an AQM) that results in low queue occupancy.

Now, we can argue if forward- (ECN loop) or back-signaling (802.1Q CN) the congestion signal is better. But without a reactive sender (such as TCP, SCTP, DCCP, Ledbat, ...) all that is in vain, and the AQM will have to drop (excessive) amounts of packets.

> In passing, let me point out what many have pointed out:  Optimizing TCP goodput is NOT 
> the correct measure to apply to the Internet.   Yes, ECN is good for that.  But it does 
> NOT fix the real user problems.  This is what Raj Jain didn't seem to understand: the 
> Internet is about latency, above all, because it is about both human-computer interaction
> and computer-computer (distributed systems) interaction.  It is not a set of "tubes" that
> are unto themselves and measurable by goodput.

Agreed - fixing TCP goodput by itself (use of ECN with some legacy "A"QM scheme like RED or "Mark above threshold") doesn't help. But driving down standing queues close to the optimum (1+-1 packet) operating point does... ECN is a means, not an end.

> The optimal state of a highway system in terms of "cars per hour" (optimum goodput) is, 
> unfortunately, really bad for the cars - because it is a highway system in a state of 
> constant traffic jam, and where the latency of any particular car is not minimized - it 
> is actually far, far from the minimum.

Again, wrong metaphor. Also, this picture is wrong insofar, as the same number of cars per unit time can pass a given mark as vastly different speeds (say, 5 kph gridlock, and 80 kph flowing traffic; distance between cars a meager 3m first, and gracious 120m in the other case).

As Fred stated earlier, the optimum state of a packet network is close to the knee (small standing queues), not close to the cliff. It's the job of a good AQM, to signal, by means of loss, delay and ECN, that knee area to a reactive sender, and protect the network from non-reactive senders...

> 
> That being said, the Internet, though different from the highway system, shares this 
> problem.  If you tune for "goodput", you will have a completely congested system, where 
> users will *walk away* and abandon interactions leaving them in the queues.   

I disagree. If you tune for thoughput, you end up with a completely congested system, with effective zero goodput. If you tune for goodput, you also address latency and utilization issues.

> We see that
> in HSPA and LTE networks today, live.  It's not "interference" that causes this on 
> wireless networks, it's been demonstrated that it is queueing delay that is allowed to 
> get too large, because "goodput" is the only metric used.  Users don't want the last 10% 
> of goodput, if it causes them a factor of 100 in response time.  (easily happening if 
> there are 10 handshakes in a protocol step, and each is 10x longer from a latency 
> perspective).

Unfortunately, "Users" use tools like speed test, which are completely agnostic to latency effects and go for throughput (not goodput).

I believe Jim wanted to address the simulatenous measurement of throughput (bits/s) together with latency while testing... All popular tools I know today measure (icmp ping) latency separate from the (tcp/udp) traffic flow...

Regards,
   Richard

-----Original Message-----
From: "Mirja Kuehlewind" <[email-masked]>
Sent: Tuesday, March 26, 2013 11:02am
To: [email-masked]
Cc: [email-masked]
Subject: Re: [aqm] aqm Digest, Vol 1, Issue 7
Hi,

Mirja

tel: +49(0)711/685-67973

---

From: Joe Touch <[email-masked]>
Date: Tue, 26 Mar 2013 11:06:45 -0700

> latency of any particular car is not minimized - it is actually far, far

http://www.paleale.eecs.berkeley.edu/~varaiya/papers_ps.dir/accessF05v2.pdf

Max cars-per-second happens at a variety of speeds, but min driving time 
happens at the highest speed only. See Figure 1.

I'm not sure if it's in this study but I also remember hearing that "max 
cars per second" in most cities is around 45MPH, whereas in LA it's 
closer to 65MPH. LA drivers tend to travel in tighter "packs".

Joe