Subject: RE: FW: [bmwg] Some questions/comments on the multicast draft.

From: Jim McQuaid <[email-masked]>
Date: Thu, 18 Oct 2001 13:59:26 -0400

When you say:

"The maximum granularity of the trials MUST be 10% of the maximum
offered load; a finer granularity is encouraged."

I think it might be clearer to the intent to say:

"The maximum granularity of the trials MUST be AT LEAST 10% of the maximum
offered load; a finer granularity is encouraged." (caps added for emphasis
here only)

On one hand, it is clear that a true throughput number must = the maximum
rate at which nothing is dropped.  However, a more coarse granularity will
yield a metric that is a known throughput rate but not necessarily the
Maximum rate.  So testers (or those submitting to tests) have an incentive
to use fine grained testing here, such as the converging binary search Kevin
suggests, in order to get to the MAXIMUM.  That is, if the max throughput
rate is 19% and I only test 20% (fail) and 10% (pass) I can only claim 10%
even though my actual max is nearly double in this case.

It's really a sampling problem and higher frequency sampling  = better data.
Incorporating a bucketized approach does distort the metric but at least the
bias is in the direction of understating performance.

cheers,

Jim McQuaid

NetIQ Raleigh

-----Original Message-----
From: Debby Stopp [mailto:[email-masked]]
Sent: Thursday, October 18, 2001 1:31 PM
To: 'Kevin Dubray'
Cc: Bmwg (E-mail)
Subject: RE: FW: [bmwg] Some questions/comments on the multicast draft.

kevin wrote:
> I have no problem modifying the collection algorithm to reduce
> collection
> time; the issue that I have is that RFC 1242 defines throughput as the
> _maximum_ lossless forwarding rate.  Your proposed algorithm won't 
> necessarily preserve this.  
> 
> A hybrid approach might work here:  use your 10% (or smaller ;-)
> decrements
> to narrow the binary search area.  Then apply the binary-search like
> technique
> to find the maximum, lossless forwarding rate.
> 
> Then we don't recast RFC 1242's meaning of throughput via a 
> methodology 
> statement.  

Hmm.  I don't believe I've actually stepped on RFC 1242's def of throughput
by suggesting a different methodology, from 1242:

"Definition:
                The maximum rate at which none of the offered frames
                are dropped by the device.
 
        Discussion:
                The throughput figure allows vendors to report a
                single value which has proven to have use in the
                marketplace.  Since even the loss of one frame in a
                data stream can cause significant delays while
                waiting for the higher level protocols to time out,
                it is useful to know the actual maximum data
                rate that the device can support.  Measurements should
                be taken over a assortment of frame sizes.  Separate
                measurements for routed and bridged data in those
                devices that can support both.  If there is a checksum
                in the received frame, full checksum processing must
                be done."

There's no mention in there re: binary search or any other specific
algorithm to find the "maximum rate at which none of the offered frames are
dropped".

However, I have no problem compromising, how about this:

"Similar to the Frame loss rate test in RFC 2544, the first trial SHOULD be
run for the frame rate that corresponds to 100% of the maximum offered load
for the frame size on the input media. Repeat the procedure for the rate
that corresponds to 90% of the maximum offered load, then again for 80% of
this load and so on. This sequence SHOULD be continued (at reducing 10%
intervals) until there are two successive trials in which no frames are
lost. The maximum granularity of the trials MUST be 10% of the maximum
offered load; a finer granularity is encouraged.

Alternately, the first trial MAY be run for a frame rate that is well known
to correspond to 0% frame loss; consecutive trials MUST be run at regular
increasing intervals of 10% of the maximum offered load.  When loss is
encountered, perform a binary search between the loss point and the last
zero-loss point to determine the throughput metric. "

Debby

_______________________________________________
bmwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/bmwg

_______________________________________________
bmwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/bmwg

---

From: Debby Stopp <[email-masked]>
Date: Thu, 18 Oct 2001 11:17:28 -0700

Jim wrote:
> Maximum rate.  

I can see your point; however I think the intent of the original statement
was to limit the successive frame rate increases to not more than 10% at a
time, which would in turn limits the maximum number of trials to 10,
assuming frame loss occurred at every trial.  If what you're looking for is
a little more flexibility in the size of the frame rate change, I suggest an
alternate wording:

"The maximum granularity of the trials SHOULD be 10% of the maximum offered
load; a finer graularity is encourage."

This allows the tester to optionally use a coarser granularity (ie., greater
than 10%) but also stay w/in the confines of the methodology.

Debby Stopp 
Principal Software Engineer

IXIA
26601 W. Agoura Rd. 
Calabasas 
CA 91302 
Tel: 818 871 1800 (US) +1 818 871 1800 (international) 

[email-masked]
http://www.ixiacom.com