Subject: [bmwg] Comments on draft-ietf-bmwg-mcastm-12.txt

From: "Perser, Jerry" <[email-masked]>
Date: Wed, 14 May 2003 10:09:47 -0700

Here are a few comments:

4.1. Mixed Class Throughput

- Suggest changing 'multicast class frames' to either 'multicast class' or
'multicast frames'.  I could not find in RFC2432 (or elsewhere) the use of
'multicast class frames'.  What is the difference between 'multicast class
frames' and 'multicast frames' ?  Same applies to 'unicast class frames'.

- Suggest changing 'The DUT/SUT MUST learn the appropriate unicast
addresses' to 'The DUT/SUT MUST learn the appropriate unicast and multicast
addresses'.  My experience with DUT is that they DO NOT populate the
forwarding table based on IGMP or ARP messages.  They go from slow path to
fast past after reception of a frame that requires forwarding.

- The search algorithm use to determine the Mixed Class Throughput.  Does it
maintain the multicast to unicast ratio, or can you change the load on one
class only?  My opinion is that you MUST change both classes.  Please
clarity in the I-D.

- When the unicast frames 'transmit in a round-robin fashion', are they to
be in the pattern specified in RFC2889, or ANY round-robin fashion?  There
are several different patterns that fall under 'round-robin'.  Mr.
Mandeville used the one in RFC2889.

- Consider changing 'Total number of multicast groups' to 'Total number of
ingress multicast groups'.  In reporting the 'Total number of multicast
groups', do you sum up the ingress address, or the egress address?  I am
concerned that someone may multiple the groups by the number of egress ports
to get a bigger number.  Other test has this same issues.

4.2. Scaled Group Forwarding Matrix 

- Suggest changing 'The recommended granularity of additional groups to join
per iteration is 10, although the tester MAY choose a finer granularity.' to
'The recommended granularity of additional groups to join per iteration is
10, although the tester MAY choose a different granularity.'  If my DUT has
a capacity of 65K to 512K, I do not want to step by 10 or smaller.  I may
want to step by a 'different' amount.

4.4. Encapsulation/Decapsulation (Tunneling) Throughput 

- Suggest changing 'DUT/SUT or a set of DUTs' to 'DUT/SUT'.  A SUT is a set
of DUTs.

5.1. Multicast Latency

- The paragraph starting with 'End-to-end reachability' is missing.  The
paragraph offered some insight on determining the intent of the
characterization.  Instead of deleting this information, can you change the
first sentence to 'The test monitor MUST verify the correct forwarding of
test traffic by the DUT/SUT prior to the engagement of a test trial.'
End-to-end wording sounded like a network test, not a DUT/SUT test. 

5.2. Min/Max Multicast Latency

- In the reporting format, why MUST the 'set of all latencies' be reflected
in the test report?  I can build hardware to determine this number.  There
may not be an actual 'set of all latencies' stored in the FPGA.  The
hardware will discard the latency values as quickly as it does that actual
frames.  The min/max will be to the procedure, but there will be no 'set of
all latencies'.

6.1 Group Join Delay

- Please put the objective in chronological order.  Consider changing to 'To
determine the time duration it takes from when the DUT/SUT receives an IGMP
group membership report on an interface, to when the DUT/SUT starts
forwarding multicast frames out the same interface.'

- The relationship between state and method is not clear.  I am guessing
that state 0 maps to ONLY method A and state 1 maps to ONLY method B.  This
section looks like two tests jammed under one heading.

One suggestion would be to divide this into two different tests.  Put state
0 and method A under '6.1.1 Initial Group Join Delay'.  Then put state 1 and
method B under '6.1.2 Primed Group Join Delay'.  If there are better
prefixes than 'initial' and 'primed', use them.  This is the best I could
come up with in this short amount of time.

6.2. Group Leave Delay

- Please put the objective in chronological order.  Consider changing to 'To
determine the time duration it takes from when the DUT/SUT receives an IGMP
Leave Group message on an interface, to when the DUT/SUT stops forwarding
multicast frames on the same interface.

8.1. Forwarding Burdened Multicast Latency

- Why 'Perform a baseline measurement of Multicast Latency'?  It is not used
in determining the Forwarding Burdened Multicast Latency.  You can run both
tests for comparison.  The burdened test is a separate test.  Suggest to
remove this.

- Suggest removing 'Similar to Section 5.1, ' in the reporting format.  You
do not need to read Section 5.1 to implement this test.  Everything needed
is already documented here.

8.2. Forwarding Burdened Group Join Delay

- Why 'Perform a baseline measurement of Group Join Delay'?  It is not used
in determining the Forwarding Burdened Group Join Delay.  You can run both
tests for comparison.  The burdened test is a separate test.  Suggest to
remove this.

- Suggest removing 'Similar to Section 6.1, ' in the reporting format.  You
do not need to read Section 5.1 to implement this test.  Everything needed
is already documented here.

Jerry.
_______________________________________________________ 
Jerry Perser                         Spirent Communications
Director of Test Methodology         SmartBits Division 
Phone: 818.676.2320                  26750 Agoura Road 
Lab:   818.676.2337                  Calabasas, CA, 91302 
Cell:  818.292.6457                  Corp:  818.676.2300

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: Debby Stopp <[email-masked]>
Date: Fri, 16 May 2003 12:14:57 -0700

Jerry wrote re: 6.1 Group Join Delay

>- The relationship between state and method is not clear.  I am 
>guessing that state 0 maps to ONLY method A and state 1 maps to ONLY 
>method B.  This section looks like two tests jammed under one heading.

I will add clarifying text to Method A & Method B that specifically
references the initial MFDB state.

>One suggestion would be to divide this into two different tests.  Put 
>state 0 and method A under '6.1.1 Initial Group Join Delay'.  Then put 
>state 1 and method B under '6.1.2 Primed Group Join Delay'.

I don't recommend that this section be broken into two tests.  Regardless of
the two test conditions, delay is still determined using the same goalposts,
much like obtaining Latency or Throughput metrics while modifiers are
present.  By stating the method used in the configuration section of the
report, one can easily compare if the delay metric is affected under the two
test conditions.  

Debby Stopp 
Principal Software Engineer
(818) 444-2365
IXIA
26601 W. Agoura Rd. 
Calabasas 
CA 91302 
Tel: 818 871 1800 (US) +1 818 871 1800 (international) 
[email-masked]
http://www.ixiacom.com 
_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: Scott Poretsky <[email-masked]>
Date: Sat, 17 May 2003 09:31:41 -0400

BMWG-ers,

A lot of people have worked hard on this for a long time, including a 
recent team that was assembled to address the final issues to complete the 
draft.   (I have to confess that I was not one of them).  This team and the 
authors have addressed most of the significant technical and documentation 
issues.  My recommendations to progress this draft are as follow:

In 4.1,  change 'The DUT/SUT MUST learn the appropriate unicast
addresses' to 'The DUT/SUT MUST learn the appropriate unicast and multicast
addresses' as Jerry recommended.

In 4.1, state it is "unweighted" round-robin.

In 5.2, report min, max, and average latency for each multicast 
group.  Eliminate "set of all latencies".

In 6.1, create two separate tests as Jerry recommended:  6.1.1 'Initial 
Group Join Delay' and '6.1.2 Primed Group Join Delay'.

I recommend that upon incorporating these changes that this draft be 
forwarded to the Area Directors for consideration in progressing the draft 
to an Informational RFC.

Scott

---

From: "Debby Stopp" <[email-masked]>
Date: Sat, 17 May 2003 15:41:49 -0700

> addresses' as Jerry recommended.

This issue is already dealt w/nicely in global section 3.1 - why do we need
to repeat it again?

> In 4.1, state it is "unweighted" round-robin.

Good suggestion, thanks.

> group.  Eliminate "set of all latencies".

The current suggestion is to make this a 'should', vs. a 'must', not to
eliminate the text. Since we do not provide an average from the set of all
latencies, reporting the set of all latencies can provide valuable insight
into the overall behaviour of the DUT that the min/max metric may omit <ie.,
the min/max may not account for spikes, for example>.

> Group Join Delay' and '6.1.2 Primed Group Join Delay'.

See previous email from friday.

Debby Stopp

---

From: "Hickman, Brooks" <[email-masked]>
Date: Mon, 19 May 2003 12:30:17 -0700

>This issue is already dealt w/nicely in global section 3.1 - why do we need

My take on this is that the current text might make it confusing in regards
to the authors intent
with respect to "priming" the DUT/SUT. Section 3.1 does specify that:

     "Therefore, prior to running any tests that require forwarding of
multicast or unicast
     packets, the test apparatus MUST generate test traffic utilizing the
same addressing
     characteristics to the DUT/SUT that will subsequently be used to
measure the DUT/SUT
     response."

However, the mixed class specifies that:

          o The DUT/SUT MUST learn the appropriate unicast addresses; 
            and 

Basically, prime the DUT/SUT with unicast only. Does this override the
global requirement in section
3.1 not? I think that it should include multicast so there is consistency
between the two sections(Global section and mixed class section).

> In 4.1, state it is "unweighted" round-robin.

I would suggest changing to:

"The unicast class frames MUST be configured to transmit in a round-robin
fashion to all of the destination port as specified in RFC2889."

The traffic pattern is clearly specified in RFC2889, which is an
"unweighted" round-robin traffic pattern.

> In 5.2, report min, max, and average latency for each multicast group.
Eliminate "set of all latencies".

Note that the procedure for the latency test is a "blip" test as described
in RFC2544(One tagged frames in the middle of the burst). There is only one
latency value for each port which is why the wording is a "set of latency
measurements". In regards to the MUST versus a SHOULD. I would not have an
issue with it being a SHOULD for consistency, since the metric for the
Min/Max Latency test is the difference between the minimum and maximum
latencies and not the latencies themselves. However, as an implementer, I
would always want to present the set of latencies along with the Min/Max
value. 

>class only?  My opinion is that you MUST change both classes.  Please

Since we are dealing with two traffic classes(Multicast AND unicast) versus
only one(Multicast) with the other throughput tests, it might be good to add
the following to the last paragraph in the mixed class throughput procedure:

"The ratio of unicast to multicast frames MUST remain the same when varying
the intended load."

---

From: Scott Poretsky <[email-masked]>
Date: Tue, 20 May 2003 10:55:57 -0400

Brooks, Sounds good.  Scott