Subject: Re: Latency and load

From: John Dawson <[email-masked]>
Date: Fri, 20 Aug 1999 11:43:05 -0700

David,

My two cents.

Throughput tests are affected by the buffering depth of the DUT.  The
Throughput results for a 30 second test can be different then the results
for a 60 second test versus a 5 minute test.

The goal of the Throughput test is to find the steady-state operating
point.  If the throughput tests is too short, then the steady-state
operating point does not get found and buffering depth can make up the
difference.  If the tests are too short, then a DUT may not actually keep
up with the input but buffering helps prevent dropping any packets.  If the
test is allowed to run long enough then no amount of buffering will
cover-up for a DUT that can not keep up with the input.

So how does this apply to Latency? 

At 99% load, the device may not be able to completely keep up with the
input, but it can be close enough such that it can float the difference in
it's buffers.  This affects the latency measurement.  Each packet received
drives a little deeper into the buffers and hence has a slightly worse
latency.  You can test this theory by making several latency measurements:
one at five seconds, one at ten seconds, and one at twenty seconds.  I
would suspect that if your DUT is not in steady state but relying on
buffers, then plotted the latency on a time line you would see a straight
line curve start developing (latency gets worse with test duration).

When the load is dropped below a certain threshold, then the DUT can keep
up with the input without using it's buffers as a crutch.  In this
situation, the traffic flows freely and hence the latency achieves a steady
state characteristic.  If you repeat the measurement process at several
different time intervals (5 sec, 10 sec, and 20 sec) you would see less
variation and it would be independent of the test duration.

This actually suggests that multiple latency measurements at staggered time
intervals can tell you whether the DUT is relying on buffering depth in
order to pass the throughput test or whether the DUT has actually acheived
steady state performance.

John

At 04:46 PM 8/19/99 -0400, you wrote:
>
>BMWGer's,
>
>I'd like to start a discussion regarding the offered load to be used
>when benchmarking latency of routers and switches.
>
>I'm asking for two reasons:
>
>1. Numerous devices from numerous vendors exhibit relatively high
>latency when offered loads are at line rate, and substantially lower
>latency with lower offered loads.
>
>I believe this occurs because of congestion internal to the DUT/SUT. In
>other words, congestion can still occur even when the offered load
>itself should not produce congestion. In my experience, reducing the
>offered load (to somewhere between 1 percent and 99 percent--it varies
>by device) will clear the internal congestion and produce far lower
>latency measurements.
>
>2. I haven't been able to locate clear documentation on what offered
>load should be used when making latency measurements. Section 26.2 of
>RFC 2544 instructs testers to "first determine throughput" and then
>"send a stream of frames ... at the determined throughput rate." 
>
>This is problematic, since there isn't necessarily a strong correlation
>between throughput and latency. I have encountered devices with
>line-rate throughput that nonetheless exhibit less-than-optimal latency
>when the offered load is 100 percent. Although RFC 1242 makes clear that
>throughput doesn't necessarily mean line rate, there's still a problem:
>I know of at least one device where throughput is 99 percent of the
>theoretical maximum, yet latency doesn't settle down until the offered
>rate is 96 percent or less. 
>
>Maybe this issue should be clarified in the switching methodology. 
>
>I would be very curious to hear what offered loads other BMWGer's use in
>measuring latency.
>
>dn
>
>

---

From: Jeffrey Dunn <[email-masked]>
Date: Mon, 23 Aug 1999 20:26:47 -0400

John,

I agree implicitly with your approach.  Assuming that the user if the
network equipment understands their traffic characteristics (and that' a
big if ;->), this sort of analysis could be used to determine an
"average worst case" scenario.

Jeff

>John Dawson wrote: