Subject: [tsvwg] how much of a problem is buffer bloat today?

From: Eliot Lear <[email-masked]>
Date: Wed, 13 Mar 2013 10:23:48 -0400

I don't have an answer to that question, but Mark Allman from ICIR did
attempt to characterize buffer bloat on the Internet through an
empirical study that appeared in the January edition of CCR.  You can
find a reference to that paper at the following URL:

http://www.sigcomm.org/ccr/papers/2013/January/2427036.2427041

Eliot

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Wed, 13 Mar 2013 15:28:15 +0000

> http://www.sigcomm.org/ccr/papers/2013/January/2427036.2427041

Well, yes, he says that in his gigabit FTTH network he doesn't see megabit-scale problems.

We know of data center cases; I can't go into listing customers, but think of well-known data center operators and ask them about it. That tends to be about the incast problem and what happens when you ask a whole lot of folks a question and they all answer. 

We know of broadband cases in DSL, Cable Modem, and so on. Ask yourself why Comcast wrote RFC 6057.

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 13 Mar 2013 16:15:26 +0000

Hi,

I recently did some very preliminary measurements using some lab equipment and Rick Jones' bloat.sh script. In their default configuration and with the latest available firmware, here is what I saw:

Cisco Catalyst 2960: at 100Mb/s, 80ms under load
Cisco Catalyst 3750: at 1Gb/s, 8ms under load
Cisco Nexus 5010 & 5548: at 1Gb/s, 8ms under load

(I haven't measured the Nexus switches at 10Gb/s, because the single netperf instance used by Rick Jones' bloat.sh script can only pump out 5G/s on one core. Once Toke's netperf-wrapper works on FreeBSD, I'll re-test using that.)

Lars

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 13 Mar 2013 16:34:16 +0000

(And let me be clear that I'm not riffing on Cisco here. We like their gear, and that's why I had it lying around in the lab.)

---

From: Greg White <[email-masked]>
Date: Wed, 13 Mar 2013 16:39:58 +0000

I would also suggest that in the conclusion of the paper:

		

							However, the magnitude of the problem as manifest in live traffic
is modest. In particular, we find bloat bounded at (i) 100 msec for more
than half the samples and (ii) 250 msec for at least 94% of the samples.

Some may consider 100ms a big number.

-Greg

On 3/13/13 11:28 AM, "Fred Baker (fred)" <[email-masked]> wrote:

>We know of data center cases; I can't go into listing customers, but
>think of well-known data center operators and ask them about it. That
>tends to be about the incast problem and what happens when you ask a
>whole lot of folks a question and they all answer.
>

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 13 Mar 2013 18:32:05 +0100 (CET)

> Some may consider 100ms a big number.

My personal opinion that for sustained congestion, there is very little 
use to ever keep a packet more than ~50 ms in a buffer. There is use to 
have larger buffers for short term smoothing of flows, but if the buffer 
has been at higher depths than 50 ms for several seconds, just start 
dropping/marking aggressively until the buffer is substantially lower than 
this. This example if more in a general WRED scenario. If it's a per-flow 
queueing scenario with fair-queue or alike, then of course there is little 
reason to drop the traffic early.

I liked the numbers presented at ICCRG yesterday which had substantially 
lower numbers than this, because even 10-30ms is felt for interactive 
applications (or badly written ones that request data, waits for the data 
to come back, and then makes new requests based on that data). Speed of 
light in fiber is something we can't do much about, but buffer depths we 
definitely can.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Roland Bless <[email-masked]>
Date: Wed, 13 Mar 2013 20:27:04 +0100

Hi,

> http://www.sigcomm.org/ccr/papers/2013/January/2427036.2427041

I know the paper, but here are my 2 cents:
- more systematic analysis is good
- "bufferbloat" exists at least in many residential broadband access
scenarios and is _reproducible_ there. The effects are pretty annoying
for interactive communications. IMHO this is enough motivation
to change the current braindead tail drop behavior and replace it
by some AQM mechanism.

Regards,
 Roland

---

From: Joe Touch <[email-masked]>
Date: Thu, 14 Mar 2013 13:12:08 -0700

FWIW, not all bufferbloat happens in the routers.

The original example that JimG discussed with me at the start of this 
crusade ended up being poor scheduling and socket configuration inside a 
single OS. These aren't addressed by AQM AFAICT.
Joe

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 15 Mar 2013 04:40:31 +0100 (CET)

On Thu, 14 Mar 2013, Joe Touch wrote:

> The original example that JimG discussed with me at the start of this crusade 
> ended up being poor scheduling and socket configuration inside a single OS. 
> These aren't addressed by AQM AFAICT.

I was under the impression that for instance Linux, would use the same AQM 
code when putting packets from a local TCP socket onto a local ethernet 
interface, as if this packet came in from another interface and was being 
routed out the same interface as the TCP packet. AQM should be used 
regardless whether the packet was locally sourced or not.

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Mar 2013 15:44:14 -0700

Hi, Mikael,

On 3/14/2013 8:40 PM, Mikael Abrahamsson wrote:
>> crusade ended up being poor scheduling and socket configuration inside
>> a single OS. These aren't addressed by AQM AFAICT.
>
> I was under the impression that for instance Linux, would use the same
> AQM code when putting packets from a local TCP socket onto a local
> ethernet interface, as if this packet came in from another interface and
> was being routed out the same interface as the TCP packet. AQM should be
> used regardless whether the packet was locally sourced or not.

Sockets are between the app and the OS. If they're configured too large, 
they act like queues and add delay, except there's no AQM in a socket 
buffer - they don't have drops.

I don't know where Linux implements AQM; it might be just on the ingress 
side of the interfaces. I don't know if there are similar queues or not 
on the egress side.

Joe

---

From: Andrew McGregor <[email-masked]>
Date: Sat, 16 Mar 2013 12:12:08 -0400

Linux has very comprehensive AQM on most possible paths though the system.
Certain device drivers are problematic, as is the WiFi stack. AQM is
available on egress as well as ingress and on the forwarding path. As yet,
AQM is not the default behaviour, but there are people working on that.
On 15 Mar 2013 18:45, "Joe Touch" <[email-masked]> wrote:

> side of the interfaces. I don't know if there are similar queues or not on
> the egress side.
>
> Joe
>

---

From: Joe Touch <[email-masked]>
Date: Sun, 17 Mar 2013 09:43:09 -0700

Hi, Mikael,

On 3/16/2013 9:12 AM, Andrew McGregor wrote:
> Linux has very comprehensive AQM on most possible paths though the
> system. Certain device drivers are problematic, as is the WiFi stack.
> AQM is available on egress as well as ingress and on the forwarding
> path. As yet, AQM is not the default behaviour, but there are people
> working on that.

Sockets are neither ingress packets nor egress packets. If the socket 
buffer is very large and/or the scheduling granularity to refill the 
socket is very large, then apps can experience the same high latency.

AQM isn't relevant for OS sockets. You can't drop stuff in a 
SOCK_STREAM, and you can't send "ECN" up to the app.

Even if you use backpressure to flow-control the app (and you do), a 
large socket buffer to a small egress pipe just creates a lot of app 
delay unless the app reacts to it, and most apps don't.

Joe

>
> On 15 Mar 2013 18:45, "Joe Touch" <[email-masked] <mailto:[email-masked]>>
> wrote:
>
>     Hi, Mikael,
>
>     Sockets are between the app and the OS. If they're configured too
>     large, they act like queues and add delay, except there's no AQM in
>     a socket buffer - they don't have drops.
>
>     I don't know where Linux implements AQM; it might be just on the
>     ingress side of the interfaces. I don't know if there are similar
>     queues or not on the egress side.
>
>     Joe
>

---

From: Rick Jones <[email-masked]>
Date: Mon, 18 Mar 2013 10:05:06 -0700

> delay unless the app reacts to it, and most apps don't.

In that case though at least, it is the application giving latency to 
itself and its large socket buffer backlog is not adding latency to 
unrelated applications.  In other words as it is the application messing 
with itself, the application is in a position to deal with it.

And Linux is not trying to hit anything SOCK_STREAM with AQM.  The AQM 
is at the handoff from IP to the drivers

rick jones

---

From: Joe Touch <[email-masked]>
Date: Mon, 18 Mar 2013 13:54:12 -0700

> with itself, the application is in a position to deal with it.

Depending on the scheduler, the OS, the packet sizes, and whether there 
is pre-emptive packet scheduling, a large socket buffer for one app can 
starve out another.

> is at the handoff from IP to the drivers

Sure, but the first example that I was told was a bad switch 
implementation turned out to be inside a single OS. An OS with a 10ms 
scheduler can't have latency that is consistently lower than 10ms when 
under load.

Joe

---

From: Graham Beneke <[email-masked]>
Date: Thu, 21 Mar 2013 18:06:01 +0200

>> Some may consider 100ms a big number.

I've heard the 100ms figure thrown about quite a bit. In the VoIP world
its considered to be the pain threshold for a reasonable call experience
and for "general Internet" (aka http browsing) it doesn't normally
impact the user experience notably.

> dropping/marking aggressively until the buffer is substantially lower
> than this. This example if more in a general WRED scenario. If it's a
> per-flow queueing scenario with fair-queue or alike, then of course
> there is little reason to drop the traffic early.

I agree. I've been playing around with building RED queues that buffer
only 10-50ms and the user experience seems to be quite good. Anything
more than that and user experience suffers.

The other issue that this research probably missed is the problem of
"dark buffers" - those buffers that only start filling up and causing
latency when some other part of the network fails and the network
experiences unplanned congestion.

I have observed numerous incidents on peer and upstream networks where
latency through a single buffer suddenly goes from ~0 ms to as much as
250 ms during a congestion incident. Most of this will not be solved by
a single test and needs ongoing monitoring and trends analysis to detect
and resolve.

-- 
Graham Beneke

---

From: Greg White <[email-masked]>
Date: Thu, 21 Mar 2013 16:59:03 +0000

....cross posting to aqm.

>impact the user experience notably.

Web browsing *is* sensitive to RTT (though maybe less so than VoIP or
gaming).  Page load time increases linearly with RTT (at approximately
10x-20x).  So adding 100ms of *unnecessary* buffering latency adds ~1-2
seconds of *unnecessary* delay to page loads.

---

From: Oliver Hohlfeld <[email-masked]>
Date: Thu, 21 Mar 2013 18:50:07 +0100

(cross posting to bloat)

> Well, yes, he says that in his gigabit FTTH network he doesn't see megabit-scale problems.

Marks paper is /not/ about measuring buffer bloat in an FTTH network.
While he uses the FTTH network as /vantage point/, the paper actually
measures buffer bloat in various remote networks.

Marks paper is not the only study suggesting the extend of the problem
to be modest. The presented results are in line with recent findings by
Chirichella and Rossi [1]. Based on unpublished work, I can confirm the
low magnitude of the problem. I analyzed passive measurements of
residential users traffic from multiple continents (~60 million IPs
originating from 50\% of all ASes) and rarely find excessive RTTs that,
among other problems, can indicate the presence of buffer bloat.

In summary, bloated buffers exist and buffer bloat can be demonstrated,
but current findings suggest that it rarely occurs in practice. One
potential reason being that users do not often sustainably utilize
their uplink capacity and fill-up their potentially large queues.

Oliver

[1] To the Moon and back: are Internet bufferbloat delays really that large?
http://perso.telecom-paristech.fr/~drossi/paper/rossi13tma-a.pdf

---

From: Oliver Hohlfeld <[email-masked]>
Date: Thu, 21 Mar 2013 19:14:03 +0100

> I believe you are actually measuring the *fraction of the time* your
> measurements show bad latency

Yes.

> The best data I've seen on how widespread the problem is is the ICSI
> Netalyzr scatter plots results

One has to be extremely careful on what to conclude from this data.
The Netalyzr data shows that bloated buffers _exist_, not that they
are _used_ in practice. Or as Mark has put it: "[it] shows that large
delays due to buffering can happen, not that they do happen."
Empirical evidence cited in my previous mail suggests that buffer
bloat does not happen often.

> I encourage everyone to run netalyzr and/or the mlabs tests for
> bufferbloat on your own broadband connections, or do simple copy and
> ping tests inside your own house over wifi to your local file servers....

While this will identify bloated buffers, it does not help in answering
what fraction of Internet users actually experience the problem.

Oliver

---

From: Oliver Hohlfeld <[email-masked]>
Date: Thu, 21 Mar 2013 20:08:56 +0100

> http://www.sigcomm.org/ccr/papers/2013/January/2427036.2427041

The full extend of the answer is still unclear to me. We have recently
attempt to complement the rather technical and QoS centric view on the
buffer bloat problem by estimating the user experience impact:

BufferBloat: How Relevant? A QoE Perspective on Buffer Sizing.
Oliver Hohlfeld, Enric Pujol, Florin Ciucu, Anja Feldmann, Paul Barford
http://downloads.ohohlfeld.com/paper/bufferbloat-qoe-tr.pdf

The paper studies the impact of buffer sizes on VoIP, IPTV, and web
browsing Quality of Experience (QoE). We find that:

- oversized buffers indeed degrade QoE when they are sustainable filled.
- however, large buffers do not always degrade user experience.
- the level of congestion significantly degrades QoE, oftentimes more
  than buffer sizes.

One example discussed in the paper is web browsing. When the level of
congestion is low, HTTP transactions benefit from "large" buffers as
they reduce losses by absorbing transient bursts. When the level of
congestion is high, transfer times become RTT dominated and the queuing
delays start to kick in.

Note that objective QoE metrics used in our paper also do not provide
the full picture:
(i) objective QoE metrics and subjective user experience are not
    always correlated.
(ii) the influence memory effects is still unclear (e.g., for how
     long will a user be influenced by a single degradation and how
     does it alter his behavior?). Psychological insights are only
     available for short-time scales.
(ii) even if service degradations exist that would degrade the user
     experience, the user might not always notice them.

In summary, the question on how much of a problem buffer bloat currently
is cannot be fully answered and still requires further research.

Oliver

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 21 Mar 2013 20:25:35 +0100 (CET)

> is cannot be fully answered and still requires further research.

Buffer bloat is a problem on basically all access forms apart from ETTH. 
Usually ETTH is produced using L2 or L3 switches with very small buffers 
(5 ms or so), and policing is used instead of buffering for limiting 
service rate. This means high speed TCP flows will sawtooth their 
performance over time because of large amount of consecutive drops, 
meaning low bw interactive flows are less impacted.

So it's my belief that your measurements means most people don't actually 
put congestion pressure on their accesses, thus the large buffers are 
seldom used and you're not seeing buffering.

---

From: Oliver Hohlfeld <[email-masked]>
Date: Thu, 21 Mar 2013 21:47:53 +0100

On Thu, Mar 21, 2013 at 01:04:21PM -0700, David Lang wrote:
> Also, if you measure the impact of bufferbloat in terms of how many
> seconds a day the line is impacted, you get a horribly skewed view of
> the impact.

I agree that such a temporal view will skew the results. The approach
taken in Mark Allman's paper is to measure the magnitude by measuring
the number of affected packets relative to the total number of packets.
This implies the amount of time a user can potentially suffer from
buffer bloat when actively using the line. It's not relative to when
the user is idle.

Oliver

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 22 Mar 2013 05:27:31 +0100 (CET)

On Thu, 21 Mar 2013, Jim Gettys wrote:

> Every more modern TCP can easily fill any sized buffer given time with a 
> single TCP connection.

I agree with this, I made this discovery myself back in 2004 or so, and 
had to implement Fairqueue and WRED on my home connection to make it 
bearable to use any interactive application while transferring files.

In IETF75 in Stockholm in 2009, I made proposals in both TCP and at open 
mic in one of the sessions, that I would like to see statistics and 
performance numbers on packet loss, delay variation etc from actual 
traffic. The IP stack has great insight in what the network conditions are 
(especially with TCP Timestamping), but as far as I know it's not really 
exported in any usable format to the user. My idea was to have some kind 
of dashboard for the user to show if currently the network was the 
limiting factor, if the tcp window was maxed out etc. Would also be nice 
if there was output that could be cut/pasted and attached to a fault 
report in case the customer talks to customer support. It would be good if 
this was actually a standard so all OSes did the same.

I am not aware of any such work going on, so I'd like to know if anyone 
else is aware of work in this area?

---

From: Jim Gettys <[email-masked]>
Date: Thu, 21 Mar 2013 13:20:20 -0400

> seconds of *unnecessary* delay to page loads.

This is one very visible result; but far from the most important, as we
work on trying to make applications like WebRTC viable.

One way I think about latency is that it sets the "diameter" in which
particular applications are viable, as well as causing performance problems
in general.  These are set by human factors and the speed of our neurology.

Variance of latency is just about as bad as the latency itself; you have to
add that to jitter buffers and it increases the latency the system can be
used at.

And you have to think of latency in terms of a budget: it takes of order
75ms to get across the United States by a good unloaded Internet connection
(the same as it did in the 1990's; Einstein + the index of refraction of
glass  makes this hard to change).  So you really have much, much, less
than 100ms you can afford to burn in the edge of the network even by this
metric.

For example, human perception to make interactive "rubber
banding" imperceptible is in the 20-30ms range.  As the latency budget
includes screen refresh time, this one is doable, but you have to work
hard, as we did with the X Window System in the 1980's.  I find it
personally very sad that while X11 used to be usable between continents,
it's not typically usable today (though mistakes we made in the X11
protocol made this worse than it should have been).

If you want to play music with a friend, professional musicians might
tolerate up to 100ms total; but duffers can't tolerate even half of that
latency well.  So if you want to play music in Boston with a friend in NYC,
you've lost 20ms before you spend time in any queuing.

Gamers (and day traders) care about latency down to the single millisecond
range as it changes the *probability* of bad outcomes.

So unpredictable 100ms latency not only hurt web performance in easily
measurable ways as Greg and I have shown, but, more insidiously,  kill a
large set of applications, or reduces the geographic network "diameter"
over which they are practical.

Courtesy of the speed of light, therefore, any unnecessary latency *and*
jitter hurts what the Internet can be used for.
                                                  - Jim

---

From: Jim Gettys <[email-masked]>
Date: Thu, 21 Mar 2013 14:01:37 -0400

I believe you are actually measuring the *fraction of the time* your
measurements show bad latency, rather than the fraction of paths that may
suffer from significant bufferbloat at various times due to excessive
buffering.

Buffers only fill when they are being used and that only happens when
saturation occurs.

The best data I've seen on how widespread the problem is is the ICSI
Netalyzr scatter plots results, which (in color) are in my blog.
http://gettys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-throw-stones-at-another/
These must be flavored by understanding that those tests top out at about
20Mbps, and that (to keep the time that netalzyr takes to run sane) stops
at about 5 seconds of buffering.

I encourage everyone to run netalyzr and/or the mlabs tests for bufferbloat
on your own broadband connections, or do simple copy and ping tests inside
your own house over wifi to your local file servers....
                                                               - Jim

> _______________________________________________
> Bloat mailing list
> [email-masked]
> https://lists.bufferbloat.net/listinfo/bloat
>

---

From: Jim Gettys <[email-masked]>
Date: Thu, 21 Mar 2013 16:05:57 -0400

Also, Windows XP is still significantly in use in the Internet
(unfortunately).

On Thu, Mar 21, 2013 at 3:25 PM, Mikael Abrahamsson <[email-masked]>wrote:

> Usually ETTH is produced using L2 or L3 switches with very small buffers (5
> ms or so), and policing is used instead of buffering for limiting service
> rate. This means high speed TCP flows will sawtooth their performance over
> time because of large amount of consecutive drops, meaning low bw
> interactive flows are less impacted.
>

Due to the fact TCP window scaling is off by default in Windows XP, the
most a single TCP connection will have in flight on Windows XP is 64K bytes.

This both limits the ability of a Windows XP system to saturate a link in
the first place (thereby sometimes avoiding filling buffers at the
bottleneck at all), and limits the amount of latency a single TCP
connection will inflict.

> ______________________________**_________________
> Bloat mailing list
> https://lists.bufferbloat.net/**listinfo/bloat<https://lists.bufferbloat.net/listinfo/bloat>
>

---

From: Jim Gettys <[email-masked]>
Date: Thu, 21 Mar 2013 14:28:19 -0400

> bloat does not happen often.

Heh.

6% of the time isn't "often"?  The measurements is much more a measurement
of how much of the time your competing with someone else (or yourself in
background) for the bottleneck than anything else.

Start up any long lived TCP connection (using a modern TCP; that leaves out
Windows XP).

Stand back a few seconds.

Measure latency.

Think uploading/downloading files/videos, or sending email with images
attached, etc.

More insidious is what visiting a web page with lots of embedded images can
do.

The cross product of:
   1) current web browsers using many more than 2 TCP connections.
   2) web site "sharding".
   3) the initial window of data in those TCP connections.

This causes transients of what is up to hundreds of milliseconds of head of
line blocking even
on very high speed broadband/wifi connections.  Again, I encourage you to
do this measurement on your own broadband connection, by visiting any image
heavy web site and running ping....

> what fraction of Internet users actually experience the problem.

We've not come across any broadband equipment buffered "correctly".  If
thought is given at all to buffering, it's been sized to the maximum
buffering the device might ever need for a single TCP flow at its maximum
bandwidth.  Example, plug a current DOCSIS 3 modem (capable up to 300Mbps
these days) and use it at 20Mbps; unless the buffersizing amendment has
been turned on (no ISP I am aware of yet does so), you'll be overbuffered
by a factor of 15.

And all the operating systems have problems, to one degree or another (and
as they are all adjacent to wireless links these days, they also contribute
to the problem.

Bufferbloat just waits in hiding to get you when you try to use the network.
                                        - Jim

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 22 Mar 2013 14:23:56 +0100 (CET)

On Fri, 22 Mar 2013, grenville armitage wrote:

>
>
> On 03/22/2013 15:27, Mikael Abrahamsson wrote:
> 	[..]
>> I am not aware of any such work going on, so I'd like to know if
>> anyone else is aware of work in this area?
>
> Of possible relevance, recent FreeBSDs have SIFTR -- a kernel resident
> module for collecting TCP state-machine stats (like cwnd, rtt estimator,
> etc) as packets arrive and depart.
>
> http://www.freebsd.org/cgi/man.cgi?query=siftr

This seems exactly like what I'm after.

On another note, is there software that can get mirrored traffic from 
another host and do TCP analysis of flows when it comes to buffer bloat, 
packet loss etc. If there was, I am in the position of running this 
against a large number of customers over time and can present the results.

---

From: Mirja Kuehlewind <[email-masked]>
Date: Tue, 26 Mar 2013 18:25:47 +0100

Hi,

+1. Would be nice to have such statictics available for the user in every OS!

Mirja

-- 
-------------------------------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Institute of Communication Networks and Computer Engineering (IKR)
University of Stuttgart, Germany
Pfaffenwaldring 47, D-70569 Stuttgart

tel: +49(0)711/685-67973
email: [email-masked]
web: www.ikr.uni-stuttgart.de
-------------------------------------------------------------------

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 26 Mar 2013 17:49:49 +0000

Hmm...

TCP measures RTT; 

one could create a global histogram of all measured RTTs by TCP (whenever a valid measurement is taken), and export that with "netstat -sp tcp"... Of course, vastly different paths would be gobbled up together, but when investigating specific paths, that should be good enough as a high level starting point.

Richard Scheffenegger

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Mirja Kuehlewind
> Sent: Dienstag, 26. März 2013 18:26
> To: [email-masked]
> Cc: bloat; [email-masked]
> Subject: Re: [tsvwg] [aqm] [Bloat] how much of a problem is buffer bloat
> today?
> 
> Hi,
> 
> > > Every more modern TCP can easily fill any sized buffer given time
> > > with a single TCP connection.
> >
> > I agree with this, I made this discovery myself back in 2004 or so,
> > and had to implement Fairqueue and WRED on my home connection to make
> > it bearable to use any interactive application while transferring files.
> >
> > In IETF75 in Stockholm in 2009, I made proposals in both TCP and at
> > open mic in one of the sessions, that I would like to see statistics
> > and performance numbers on packet loss, delay variation etc from
> > actual traffic. The IP stack has great insight in what the network
> > conditions are (especially with TCP Timestamping), but as far as I
> > know it's not really exported in any usable format to the user. My
> > idea was to have some kind of dashboard for the user to show if
> > currently the network was the limiting factor, if the tcp window was
> > maxed out etc. Would also be nice if there was output that could be
> > cut/pasted and attached to a fault report in case the customer talks
> > to customer support. It would be good if this was actually a standard so
> all OSes did the same.
> >
> University of Stuttgart, Germany Pfaffenwaldring 47, D-70569 Stuttgart
> 
> tel: +49(0)711/685-67973

---

From: Hagen Paul Pfeifer <[email-masked]>
Date: Tue, 26 Mar 2013 21:02:16 +0100

* Scheffenegger, Richard | 2013-03-26 17:49:49 [+0000]:

>TCP measures RTT; 
>
>one could create a global histogram of all measured RTTs by TCP (whenever a
>valid measurement is taken), and export that with "netstat -sp tcp"... Of
>course, vastly different paths would be gobbled up together, but when
>investigating specific paths, that should be good enough as a high level
>starting point.

For Linux tcp-probe[1] is your friend. The last colum is the smoothed round trip
time. The port filter is optional and can be 0 to disable filtering. Not sure
if Linux Distribution ship this module by default. But via tcp-probe you get
accurate per-path rtt statistics without any effort (just a few CPU cycles).

Hagen

[1] http://www.linuxfoundation.org/collaborate/workgroups/networking/tcpprobe