Subject: RE: Duration

From: John Dawson <[email-masked]>
Date: Mon, 21 Jun 1999 12:26:10 -0700

> UNH Lab looks for 10E-11 BER

10E-11 BER across 100 ports at 100 Mbps. You do the math -- how long until
1 bit error is expected?

>What your suggesting is that these new technologies can not run error free
>for 30 seconds in a lab environment?  I disagree.  I have seen several 200+
>port DUTs run 60 seconds with no physical errors (one of these was from
>Nortel Networks).

If a test is run at 7 different packet sizes and 8 different operating mode
(10/100 half/full bi/uni) with a couple different port/pair combinations,
is this a 30 second test or a couple hundred 30 second tests?  Getting
something to work for 30 seconds is much different than getting something
to work a couple hundred times for 30 seconds each.

> We can probably agree that if a test fails due to a "physical error" or
>"clock tolerance", we can void the test with no penalty.  This must be
>proven to be the cause.  Not an excuse for poor performance from an over
>zealous marketeer.

How does one prove this to be the cause without a well defined methodology?  

>We verified that the frames when into the DUT error free but were never 
>transmitted (per the DUTs statistics).

Does the test process need to include inspection of DUT statistics?  I
would hate to open this door, it creates an automation and reproducibility
nightmare.

John

---

From: "Steve Bell" <[email-masked]>
Date: Mon, 21 Jun 1999 16:17:01 -0700

> to work a couple hundred times for 30 seconds each.

By definition each throughput measurement is made with a fixed packet size.
During the binary search  each "packet size" iteration in the RFC 1944 is
iterated in
a series of (seperate) 30 second tests, so they can take a while.  It would
certainly be possible to configure each
test iteration for <30secs, however that what's done by convention (I don't
believe that RFC1944 actually calls
it out).  NetCom set the default duration to 30 seconds for this test, and I
believe that Bradner
and McClellan (SNCI) have used 30 sec throughput settings.  It might be
timely to shorten
that for GbE testing.

We have noticed that different test vendors implement the throughput testing
algorithms somewhat
differently.  Also, different labs configure the equipment differently when
conducting
these measurements.  There is plenty of room for refinement's on everyone's
part!
(including the test labs)

> would hate to open this door, it creates an automation and reproducibility

We always check our test results against RMON statistics and analyzer probe
measurements;
It is never a good idea to depend upon one peice of test equipment.  The
problem with including
the DUT statistics is, not all RMON implementations are created equal...
some work better than others.
The various MAC and PMD implementations vary widely, as far as there ability
to accurately collect RT packet info internally ...
so this would need to be (externally) observable (e.g. link probe/analyzer)
information.

-Steve
www.svnl.com