Subject: [bmwg] Multicast forwarding Benchmark

From: Ron Bonica <[email-masked]>
Date: Wed, 5 May 2010 10:59:12 -0400

Folks,

It occurs to me that we have never produced a multicast forwarding
benchmark document. Is there any interest in producing one?

                                        Ron

---

From: Ron Bonica <[email-masked]>
Date: Wed, 5 May 2010 11:00:42 -0400

Please disregard this message. I just found RFC 3918.

                                 Ron

---

From: "Jan Novak (janovak)" <[email-masked]>
Date: Wed, 5 May 2010 17:09:38 +0200

Hi,

mcast has always been my favorite but what about RFC2432/3918 ??

Jan

The climate of Edinburgh is such that the weak succumb young .... 
and the strong envy them.

                                 Dr. Johnson

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Ron Bonica
Sent: 05 May 2010 15:59
To: [email-masked]
Subject: [bmwg] Multicast forwarding Benchmark

Folks,

                                        Ron
_______________________________________________
bmwg mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/bmwg

---

From: "Gunter Van de Velde (gvandeve)" <[email-masked]>
Date: Wed, 5 May 2010 17:09:55 +0200

I think that makes lots of sense to create... 
I even can volunteer as co-editor for such a paper if desired and if it
hasn't been worked upon before.

G/

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Ron Bonica
Sent: Wednesday, May 05, 2010 4:59 PM
To: [email-masked]
Subject: [bmwg] Multicast forwarding Benchmark

Folks,

                                        Ron

---

From: David Newman <[email-masked]>
Date: Wed, 05 May 2010 08:13:29 -0700

On 5/5/10 7:59 AM, Ron Bonica wrote:
> benchmark document. 

???

I'm not crazy about parts of it, but RFC 3918 includes tests to measure
aggregated multicast throughput and mixed class (mcast + ucast) throughput.

> Is there any interest in producing one?

I would not object to seeing this work extended to cover multiple Rx and
multiple Tx ports, and to include the use of a monitor port to detect
flooding.

dn

---

From: Al Morton <[email-masked]>
Date: Wed, 05 May 2010 11:38:59 -0400

>benchmark document. Is there any interest in producing one?

Did these miss your scan of our literature?
http://tools.ietf.org/html/rfc2432
http://tools.ietf.org/html/rfc3918

Or, were you looking for additional work on the topic?

Al

---

From: Al Morton <[email-masked]>
Date: Wed, 05 May 2010 11:55:06 -0400

Ron,

Sorry for my late message, about 40 minutes of discussion missed.

This topic seems to evoke as much mail as any other recently, or more.

I'm finding myself in a position to possibly try-out parts of
2432 and 3918 in the very near future...

Does anyone else have experiences to share?

Al
bmwg chair

---

From: David Newman <[email-masked]>
Date: Thu, 06 May 2010 17:50:35 -0700

> Does anyone else have experiences to share?

I regularly use 2432 and 3918 in switch and router testing. Here are
some common gotchas (and thanks to 3918 co-editor Brooks Hickman and
also Timmons Player for sanity-checking these):

1. 3918 has no explicit requirement for a monitor port, similar to the
Mport in 2889's address caching capacity test. This is a big omission
since it allows a DUT that floods or broadcasts multicast traffic to
pass a test.

There is a requirement for verifying correct learning via "externally
observable means" in some tests in 3918. But that requirement is not
present in all tests; for example, it's there for MCT and JLD but not
AMT or MGC.

A monitor port is especially useful in MGC testing. An explicit
requirement to check against flooding would be a welcome addition.

I previously noted this issue in this 2007 thread:

http://www.ietf.org/mail-archive/web/bmwg/current/msg01657.html

2. Both 2432 and 3918 assume testing with at least one Tx and one Rx
port but do not describe or require test cases with multiple Tx and/or
multiple Rx ports.

The multiple Tx case is meaningful for applications such as
videoconferencing that connect multiple participants using multicast.

The multiple Rx case is obvious; with only one Rx port (which is a valid
3918 test) a device that broadcasts or floods or unicasts would pass
some cases described in 3918.

3. The MCT test lacks a requirement to state expected egress ratios or
rates for unicast and multicast traffic. Timmons Player notes two
problems with this:

". . .having only a single unicast tx port doesn't provide an
opportunity to actually stress the DUT with traffic.

Also, "load specification for meaningful 3918-like* tests is a
nightmare, since you can have intended loads of < 100% generate expected
egress loads of 100% (or more).  This is always an issue with the Mixed
Class Throughput test and can be an issue for any test if multiple
multicast tx ports are used.  A mechanism to state expected egress loads
would really help clarify what the DUT is actually being asked to forward.

Also, "3918 doesn't say whether you should state the ingress or egress
multicast/unicast ratio for the MCT test. (I think it should be the
egress ratio.)"

*3918-like because they're based on 3918 but use more than 1 tx ports."

4. In the MGC test, 3918 allows the use of two topologies, and reports
the same result for both:

a. Different sets of ports with mutually exclusive traffic. For example,
in a 40-port DUT, there could be 8 sets of ports, each with one Tx and
four Rx ports.

b. One Tx port, and all remaining ports are are Rx ports.

These two topologies may yield vastly different results, yet there is no
requirement to state which setup was used. Personally I would prefer
seeing case (a) go away since it's less stressful and arguably less
meaningful too -- end-users have a reasonable expectation of being able
to send traffic to and from any arbitrary set of ports they buy.

5. Because work on 2432/3918 started much earlier, there are no tests
that describe IGMPv3-specific issues. Most of the bugs described at this
URL have long been addressed, but these are useful test cases for
devices that support IGMPv3:

http://www.columbia.edu/~alan/igmp/

6. Not a problem with either 2432 or 3918, but more of an observation:
The group join and group leave delay tests can be very stressful on
DUTs, especially given the modest control-plane CPUs found in many
smaller Ethernet switches.

dn