Subject: [bmwg] logic problem in RFC 3918

From: David Newman <[email-masked]>
Date: Wed, 19 Dec 2007 14:17:10 -0800

The multicast group capacity test (RFC 3918, section 7.1) says an
iteration passes if "at least one frame for each multicast group is
forwarded properly by the DUT/SUT on each participating egress interface."

That word "properly" is a problem; as the test is currently written,
there's no way to verify it.

Most (though perhaps not all) switches will flood multicast traffic when
the test instrument attempts to join more groups than the DUT/SUT can
handle.

A flooded frame looks exactly the same as a non-flooded one. Since an
RFC-compliant test can be performed with as little equipment as one
ingress and one egress interface (indeed, that's explicitly stated in at
least one commercial implementation), the test instrument cannot detect
that the DUT/SUT is flooding, and will still pass that iteration.

A test with 1000 egress interfaces also will incorrectly pass if
receivers on all 1000 interfaces join the same groups and flooding occurs.

Two possible fixes:

1. Add one or more monitor ports to the test, and fail any iteration in
which the test instrument receives multicast test traffic on the monitor
port(s). This is conceptually similar to the setup in RFC 2889's address
caching capacity test.

2. Configure the test instrument so that each egress interface joins a
different set of multicast groups. Fail any iteration in which multicast
traffic is not forwarded properly (in this case, shows up on the wrong
egress interface).

Of these two, I strongly prefer option 1. It allows for tests with more
fan-out and, in L3 scenarios, more mroutes. Both of these are more
stressful on the DUT/SUT.

dn

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Sat, 22 Dec 2007 09:21:29 -0500

Hi Dave,

You have brought up an interesting issue. 

> the test instrument attempts to join more groups than the DUT/SUT can

Could you please elaborate? A sane multicast implementation should not
let the multicast traffic be forwarded on the interface that didn't
participate on a particular multicast tree.

This, of course, is orthagonal to IGMP snooping capability that is
common on layer2 devices.

But as I read section 7.1, I find it a bit left for interpretation. For
example, 
the below excerpt from procedure -

...
   If at least one frame for each multicast group is forwarded properly
   by the DUT/SUT on each participating egress interface, the iteration
   is said to pass at the current capacity.
...

If the DUT receives 1000 frames for each multicast group from the
source, then one interpretation of the above excerpt is that the receipt
of one or more frames (out of 1000 frames) would qualify the case as
"success". 

AFAIK, the DUT must forward 1000 frames on each participating egress
interface for the success. 

Cheers,
Rajiv

> -----Original Message-----
> From: David Newman [mailto:[email-masked]] 
> Sent: Wednesday, December 19, 2007 5:17 PM
> To: [email-masked]
> Subject: [bmwg] logic problem in RFC 3918
> 
> port(s). This is conceptually similar to the setup in RFC 
> 2889's address caching capacity test.
>

---

From: David Newman <[email-masked]>
Date: Sat, 22 Dec 2007 07:24:50 -0800

> participate on a particular multicast tree.

That's true for a non-overload case -- ie, when the number of groups
involved is less than or equal to the DUT's IGMP table capacity.

In an overload case, where the test instrument attempts to join more
IGMP groups than the DUT can handle and then offers traffic to all those
groups, most DUTs will flood multicast traffic.

I make no judgment as to whether this is sane or not, but I have
observed this behavior in many different vendors' implementations,
including one from a certain San Jose-based networking vendor. (To its
credit, this particular device does filter flooded frames so they won't
be forwarded back out the ingress interface; however, there is flooding
to other interfaces that aren't part of the multicast tree.)

By definition, the multicast group capacity test attempts to find the
maximum number of groups. With any kind of limits testing, there should
be iterations where the tester attempts to go over the DUT's limit.

The logic problem with the test, as currently written, is that it is not
possible to evaulate whether a frame has been flooded.

> interface for the success. 

I would like that too, and argued unsuccessfully for zero loss at the
time bmwg considered this methodology. But the RFC says "at least one
frame", and 1 != 1000. The DUT could drop 999 frames and still pass.

However, this is orthogonal to the logic problem. Regardless of whether
the DUT forwards 1 or 1 million frames, the test instrument needs some
method to determine if they have been flooded. And as currently written,
the RFC allows a result with flooding to be considered a success.

dn