Subject: FW: [bmwg] Some questions/comments on the multicast draft.

From: Debby Stopp <[email-masked]>
Date: Tue, 16 Oct 2001 10:09:49 -0700

here's my response:

> On <draft-ietf-bmwg-mcastm-07.txt>:
> 
> 1. Section 2. You may wish to review your wording on the usage of
>    RFC 2119.  Please consider adding wording like:
> 2. Section 3, Test Setup.  

no prob, no comment required, cutNpaste worked for me, thanks.

> 3. Section 4.1, Mixed Class Throughput, presents a collection 
>    heuristic that departs from RFC 2544. (I.e., decreasing 10%
>    increments from the Max Offered Load, until no loss is
>    encountered twice consecutively vs. 2544's binary search-like
>    mechanism.) 
> 
>    Is this OK with everyone?

I believe that the general concensus w/the public is that most hate binary
searches - they seem to take a long time if you already have a good idea or
ballpark of where your metric should land, that is why we opted for a more
"frameloss" like method.
 
>    If so, the last sentence in the procedure states , "The maximum 
>    granularity of the trials MUST be 10% of the maximum rate, a 
>    finer granularity is encouraged."  I think I know what you
>    mean, but I wonder if there's a better way to say it.  In
>    any event, I'd suggest employing established terminology 
>    (e.g. Maximum Offered Load?)  Also, wording should be added 
>    to reinforce the degree of error this approach would introduce.

how about this instead:

"Similar to the Frame loss rate test in RFC 2544, the first trial SHOULD be
run for the frame rate that corresponds to 100% of the maximum offered load
for the frame size on the input media. Repeat the procedure for the rate
that corresponds to 90% of the maximum offered load, then again for 80% of
this load and so on. This sequence SHOULD be continued (at reducing 10%
intervals) until there are two successive trials in which no frames are
lost. The maximum granularity of the trials MUST be 10% of the maximum
offered load; a finer granularity is encouraged."

> 
>    Trial duration aren't specified or suggested; recording trial
>    duration is not required in reporting in this or many of 
>    the forwarding tests.  Oversight?

I added this section at the top; I think it's redundant to have duration
specified for every section, especially since the layout of this document
really does specify these details at the beginning in the "Test
Considerations" section:

"3.1.5.	Trial Duration

The duration of the test portion of each trial SHOULD be at least 30
seconds."

I made the minimum duration 30 seconds remember the discussion that we had
re: the lan switch methodology.  I specifically left off a maximum duration
because I don't believe it is pertinent to specify a max time in this case
<I think I could argue that whatever max someone came up w/, someone else
would want to go just that much longer>

> 
> 4. Aggregated Multicast's Throughput, Section 4.3, doesn't specify 
>    whether the "throughput" determination heuristic used is based on 
>    RFC 2544 or the one introduced in section 4.1.  I believe this
>    would be beneficial to clearly articulate which one.  
> 
>    The results paragraph talks about reporting frame loss. (
>    Throughput -should be zero, right?)  The defining RFC talks to 
>    recording measurment units of  (N,t) where N is the number of 
>    destination ports tested and t is the associated throughput.
> 
>    In addition, the notion that "t" is expressed relative to the input
>    source stream is missing in the I-D.
> 

Hmm... I looked this over; still need some help.  Do you want me to remove
the frameloss aspect & make it a true "throughput" as called out in the term
doc, or do you want me to re-define the sense of throughput for this
document?  Here's what I did re: your comments:

"Objective

The maximum rate at which none of the offered frames to be forwarded through
N destination interfaces of the same multicast group is dropped.

Procedure

Multicast traffic is sent at a fixed percent of maximum offered load with a
fixed number of groups at a fixed frame length for a fixed duration of time.

The initial number of receive ports of the tester will join the group(s) and
the sender will transmit to the same groups after a certain delay (a few
seconds).

On each iteration, an incremental number of receive ports will join the same
groups and then the multicast traffic is sent as stated.

The receive ports will continue to be added and multicast traffic sent until
a user defined maximum number of ports is reached or frame loss occurs at
one or more ports.

Results

Parameters to be measured SHOULD include the maximum rate or throughput (as
defined by RFC 2544) per destination port for each multicast group address.

The nature of the traffic stream contributing to the result MUST be
reported.  All required reporting parameters of aggregated throughput MUST
be reflected in the results report, such as the total number of receive
ports, total number of multicast group addresses, the transmitted packet
size(s) and offered load of the packet stream.  Constructing a table from
the measurements might be useful in illustrating the effect of modifying the
number of active egress ports on the tested system."

>    It might be helpful to add wording such as "constructing a table 
>    from the measurements might be useful in illustrating the effect 
>    of modifying the number of active egress ports on the tested 
>    system". 

you'll see I incorporated your wording in the previous paragraph.

> 5. Section 4.4, Encapsulation/Decapsulation (Tunneling) Throughput,
>    doesn't provide substantial insight to the methodological 
> approaches
>    taken.  I'd recommend deleting it.  If it is thought best to
>    keep the dialog, I'd consider discarding Figure 3.  It doesn't add 
>    much value, and it appears to use reference labels for text that 
>    no longer appears. ;-)

It's gone, love to delete stuff that no one finds useful :)

> 
> 6. Section 4.4.1, Encapsulation Throughput. In much of this I-D,
>    the RFC 2432 definition is recast as an "Objective", generally
>    word for word.  Here, RFC 2432's definition is explicitly modified
>    to provide for collection across a SUT.  This is not the intent
>    of RFC 2432's definition for this metric.  This is a single device
>    characterization only.  The wording and subsequent methodology MUST
>    be changed to reflect that.
> 
>    Also, throughput is cited to be calculated a la RFC 2544! This is
>    inconsistent with the throughput methodology of the I-D's other
>    throughput measurements.

Ok, now I'm totally at a loss, again, do you want a redifing of "throughput"
for ipmc or do you want this test to be 0% loss?  My personal pref is to
leave it w/frameloss, as I believe that is more useful for the router/switch
vendors; if you agree, can you suggest where I can reword to make it less
2544-tput like?

Debby Stopp 
Principal Software Engineer

IXIA
26601 W. Agoura Rd. 
Calabasas 
CA 91302 
Tel: 818 871 1800 (US) +1 818 871 1800 (international) 

[email-masked]

_______________________________________________
bmwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/bmwg

---

From: Kevin Dubray <[email-masked]>
Date: Thu, 18 Oct 2001 13:06:27 -0400

> ballpark of where your metric should land, that is why we opted for a more

I have no problem modifying the collection algorithm to reduce
collection
time; the issue that I have is that RFC 1242 defines throughput as the
_maximum_ lossless forwarding rate.  Your proposed algorithm won't 
necessarily preserve this.  

A hybrid approach might work here:  use your 10% (or smaller ;-)
decrements
to narrow the binary search area.  Then apply the binary-search like
technique
to find the maximum, lossless forwarding rate.

Then we don't recast RFC 1242's meaning of throughput via a methodology 
statement.  

-Kevin