Subject: RE: Latency and load

From: Jim McQuaid <[email-masked]>
Date: Fri, 20 Aug 1999 16:16:31 -0400

So what you are saying, I think, is that most devices will exhibit THREE
regions in this -hypothetical- latency characteristic curve we are
proposing.  And that NO region can be characterized without a time of
"adequate" duration.

Lowest latency - region one
	The device can handle every input without buffering and output is
therefore as "instantaneous" as it can ever be.

Rising latency - region two
	The device can't quite handle every input instantly and begins to
pile up inputs, therefore latency rises. . . .

Maximum latency - region three +
	The device can't quite handle inputs instantly and buffers are now
full.  The additional processing time provided by buffering = high latency.
This should corespond to behavior at max_throughput rate.

Hit the wall - region four?
	The device can't quite keep up; many packets make it through with
high latency but some number have infinite latency, i.e. get dropped.

Jim McQuaid
Ganymede Software

-----Original Message-----
From: John Dawson [mailto:[email-masked]]
Sent: Friday, August 20, 1999 2:43 PM
To: [email-masked]
Subject: Re: Latency and load

David,

My two cents.

Throughput tests are affected by the buffering depth of the DUT.  The
Throughput results for a 30 second test can be different then the results
for a 60 second test versus a 5 minute test.

The goal of the Throughput test is to find the steady-state operating
point.  If the throughput tests is too short, then the steady-state
operating point does not get found and buffering depth can make up the
difference.  If the tests are too short, then a DUT may not actually keep
up with the input but buffering helps prevent dropping any packets.  If the
test is allowed to run long enough then no amount of buffering will
cover-up for a DUT that can not keep up with the input.

So how does this apply to Latency? 

At 99% load, the device may not be able to completely keep up with the
input, but it can be close enough such that it can float the difference in
it's buffers.  This affects the latency measurement.  Each packet received
drives a little deeper into the buffers and hence has a slightly worse
latency.  You can test this theory by making several latency measurements:
one at five seconds, one at ten seconds, and one at twenty seconds.  I
would suspect that if your DUT is not in steady state but relying on
buffers, then plotted the latency on a time line you would see a straight
line curve start developing (latency gets worse with test duration).

When the load is dropped below a certain threshold, then the DUT can keep
up with the input without using it's buffers as a crutch.  In this
situation, the traffic flows freely and hence the latency achieves a steady
state characteristic.  If you repeat the measurement process at several
different time intervals (5 sec, 10 sec, and 20 sec) you would see less
variation and it would be independent of the test duration.

This actually suggests that multiple latency measurements at staggered time
intervals can tell you whether the DUT is relying on buffering depth in
order to pass the throughput test or whether the DUT has actually acheived
steady state performance.

John

At 04:46 PM 8/19/99 -0400, you wrote:
>
>BMWGer's,
>
>I'd like to start a discussion regarding the offered load to be used
>when benchmarking latency of routers and switches.
>
>I'm asking for two reasons:
>
>1. Numerous devices from numerous vendors exhibit relatively high
>latency when offered loads are at line rate, and substantially lower
>latency with lower offered loads.
>
>I believe this occurs because of congestion internal to the DUT/SUT. In
>other words, congestion can still occur even when the offered load
>itself should not produce congestion. In my experience, reducing the
>offered load (to somewhere between 1 percent and 99 percent--it varies
>by device) will clear the internal congestion and produce far lower
>latency measurements.
>
>2. I haven't been able to locate clear documentation on what offered
>load should be used when making latency measurements. Section 26.2 of
>RFC 2544 instructs testers to "first determine throughput" and then
>"send a stream of frames ... at the determined throughput rate." 
>
>This is problematic, since there isn't necessarily a strong correlation
>between throughput and latency. I have encountered devices with
>line-rate throughput that nonetheless exhibit less-than-optimal latency
>when the offered load is 100 percent. Although RFC 1242 makes clear that
>throughput doesn't necessarily mean line rate, there's still a problem:
>I know of at least one device where throughput is 99 percent of the
>theoretical maximum, yet latency doesn't settle down until the offered
>rate is 96 percent or less. 
>
>Maybe this issue should be clarified in the switching methodology. 
>
>I would be very curious to hear what offered loads other BMWGer's use in
>measuring latency.
>
>dn
>
>

---

From: Steve Bell <[email-masked]>
Date: Fri, 20 Aug 1999 17:31:48 -0700

Our test engineers tell me that a 30 second test has been adequate for most
of the 100Mbps switches
being tested today, to avoid skewing the results due to buffering dynamics.
For example, we just tested a
workgroup switch from a far-Eastern manufacturer -- running the latency test
at 30, 60, and 90 seconds.
The results showed the same latency regardless of the test period, to a
fraction of a microsecond.
We checked it for all RFC2544 frame sizes, in store and forward mode.  This
is for a normal latency test,
in it's "linear packet processing" region; that is not saturating the switch
so that it is dropping packets.

There may be some 10Mbps devices, or some WAN devices with huge buffers
where the 30 second test period becomes
an issue so it's hard to generalize, so some calculation on a range of
devices (buffer depth in terms of
time) to verify that 30 seconds is an adequate test period to allow the
buffers to stabilize.

-Steve Bell
 SVNL
 www.svnl.com

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Jim McQuaid
Sent: Friday, August 20, 1999 1:17 PM
To: 'John Dawson'; [email-masked]
Subject: RE: Latency and load

Jim McQuaid
Ganymede Software

-----Original Message-----
From: John Dawson [mailto:[email-masked]]
Sent: Friday, August 20, 1999 2:43 PM
To: [email-masked]
Subject: Re: Latency and load

David,

My two cents.

John