Subject: [aqm] [FQ_CoDel] "Official" version

From: Nicolas KUHN <[email-masked]>
Date: Mon, 3 Feb 2014 15:30:03 +0100

Dear all, 

We would like to test FQ_CoDel and we wonder which version to evaluate. 

Does anyone know where we could get the "official" version / source code of FQ_CoDel ?
(linux and/or NS2 if possible)

Thanks a lot, 

Kind regards, 

Nicolas

---

From: Jim Gettys <[email-masked]>
Date: Mon, 3 Feb 2014 10:57:45 -0500

> We would like to test FQ_CoDel and we wonder which version to evaluate.

fq_codel has been in Linux since about Linux 3.5, IIRC.  For casual
experiments, a current Linux distro will have it available at the
invocation of the "tc" command.  But...

You should generally be testing the latest Linux release (not that I
think there has been much change in fq_codel, as demonstrated in:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/?id=refs%2Ftags%2Fv3.14-rc1&qt=grep&q=fq_codel

For serious bench marking getting results from an old system can make your
results easily invalid since there has been a large number of improvements
in Linux for bufferbloat everywhere over the last three years, and the
queue discipline is only part of the equation.  For example, many ethernet
drivers in SOC's found in home routers may still lack even BQL support.

You should be testing the fq_codel queue discipline: while Dave Taht has
played with a number of variants on that theme, nothing to date has shown
itself (by testing) with enough better results to be worth merging upstream
into kernel.org.  Pie finally entered kernel.org Linux a month or so ago.

Note that there a large number of "traps" for the unwary in this
benchmarking/testing business, into which multiple groups have come to bad
ends causing questionable results to confuse everyone. For example, the
details of the underlying device driver can matter a lot (actually,
hugely!), and netem has to be used with exquisite care just to name a
couple of the big traps.

Fundamentally, you need to realize that the queue discipline in Linux
(where you can apply aqm) does *not* have control of all of the buffering
in the system; device drivers and even the hardware itself can do
"interesting" things to you.  And a bunch of changes have taken place in
the Linux TCP stack to remove unneeded buffering since around Linux 3.2.
 At one point I think the list was up to a dozen or more significant
improvements  in the Linux networking stack, independent of fq_codel.

We've come to learn that WiFi in particular needs a serious rewrite, and
before venturing far, you'd be wise to get in touch with us further.  ATM,
the ath9k driver is least problematic, but even there, aggregation causes
large amounts of buffering (32 packets!).

Dave Taht and I tried to outline what you should be thinking about when
bench marking in:

http://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Codel

Pretty much all of these issues apply to anything you do, not just fq_codel.

I think that page overs most of the problems  though of course, Linux is a
moving target so I'm sure there will be future problems :-(.

> (linux and/or NS2 if possible)

www.kernel.org for the source.

You can use git to see the exact revision history of the code.

I forget off hand where to find the simulation code...  There is a link
somewhere in the bufferbloat/codel wiki

http://www.bufferbloat.net/projects/codel/wiki/Wiki

Help from people to get the wiki up to date gratefully appreciated!

Jim

> Thanks a lot,
>
> Kind regards,
>
> Nicolas
>
> _______________________________________________
> aqm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/aqm
>

---

From: Dave Taht <[email-masked]>
Date: Mon, 3 Feb 2014 12:52:04 -0800

On Mon, Feb 3, 2014 at 7:57 AM, Jim Gettys <[email-masked]> wrote:
>
>
> On Mon, Feb 3, 2014 at 9:30 AM, Nicolas KUHN
> <[email-masked]> wrote:
>>
>> Dear all,
>>
> experiments, a current Linux distro will have it available at the invocation
> of the "tc" command.  But...

Improvements as of 3.12 is the ability to set a default qdisc across the
entire system via a single sysctl.conf option.

> You should generally be testing the latest Linux release (not that I think
> there has been much change in fq_codel, as demonstrated in:

However there were some improvements to the jhash algorithm that
landed in 3.8 and later to improve ipv6 hashing, and to more deeply
inspect gre and 6in4 tunnels.

> in Linux for bufferbloat everywhere over the last three years, and the queue
> discipline is only part of the equation.  For example, many ethernet drivers
> in SOC's found in home routers may still lack even BQL support.

I note the sanest results are more consistently obtained using a SQM
system like what's in cero, which leverages htb, thus moving control
to htb and away from the drivers deeper in the stack.

that said, htb also underwent improvements in 3.11 to make it faster, and
there are some "dis-improvements" the current SQM system for cero that
speed it up for the low powered hardware it's aimed at, above 20Mbit.
Faster systems don't need those hacks...

(and htb can be improved further, or replaced with hfsc.)

> into kernel.org.

nfq_codel presently does mildly better at very low bandwidths (<2Mbit)
as it is more sfq-like. Actually everything below 4Mbit is kind of
finicy with everything (aqm, sfq, codel, fq_codel).

I have been thinking of bringing back a patch from 3.5 in some form to
make maxpacket handling saner at lower bandwidths (at a price in latency)

> Pie finally entered kernel.org Linux a month or so ago.

I have been backporting that as far back as 3.8.

> hugely!), and netem has to be used with exquisite care just to name a couple
> of the big traps.

netem was hugely buggy prior to 3.11. I still only run it on a dedicated box
to insert delay and watch it carefully.

> Fundamentally, you need to realize that the queue discipline in Linux (where
> you can apply aqm) does *not* have control of all of the buffering in the
> system; device drivers and even the hardware itself can do "interesting"
> things to you.  And a bunch of changes have taken place in the Linux TCP
> stack to remove unneeded buffering since around Linux 3.2.  At one point I
> think the list was up to a dozen or more significant improvements  in the
> Linux networking stack, independent of fq_codel.

Yep.

I look forward to hardware that can run this stuff directly.

> http://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Codel

Don't be too scared...

> www.kernel.org for the source.

patches and backports to older kernel versions are available as part
of the kernel-backports package.

(YMMV, as we note lots of other changes in the stack elsewhere
 matter quite a lot)

> http://www.bufferbloat.net/projects/codel/wiki/Wiki

ns2 and ns3 versions of the patches have been rotting in a repo on github.
Hope someone makes it a google summer of code project to get these mainlined.

-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html