Subject: more comments on draft-ietf-bmwg-mswitch-01.txt

From: "Debby Stopp" <[email-masked]>
Date: Thu, 3 Jun 1999 14:36:54 -0700

I have a few more comments - I broke up my comments into two emails because
they were getting a bit tedious - sorry for the lengthy reply.

***
Section 5.6 Filter illegal frames & Section 5.7 Broadcast frame handling and
latency test-
Will there be a reporting format description for these sections? are they
TBD...?

>From Section 5.9.3/5.10.3 (address cache/address learning):
   This test SHOULD at a minimum be performed in a three-port
   configuration as described below.

   This test MUST consist of a multiple of three ports.  Three ports
   are REQUIRED and MAY be expanded to fully utilized the DUT/SUT in
   increments of three.  Each group of three will contain a test
   block as follows:

Again I ask why a multiple of 3 ports?  I suggest that a better test of the
address cache capacity of a sut/dut is to sprinkle the total addresses to be
learned among a group of ports, rather than from one port.

Debby Stopp
[email-masked]
IXIA Communications

---

From: Jerry Perser <[email-masked]>
Date: Fri, 4 Jun 1999 09:05:03 -0700

>Will there be a reporting format description for these sections? are they

For Section 5.6 Filter illegal frames, I'll add a small paragraph stating
the results should be reported as a PASS/FAIL for each of the illegal
frames.  For Section 5.7, I will copy section 5.1.5 and renumber it.  I was
trying to save paper since the draft is over 30 pages long.  Oh well. 

>learned among a group of ports, rather than from one port.

The test is structured around 3 functional blocks: PUT (Port under Test), TX
port, and a SPY port. The first part of the test is to clear the DUT/SUT
address table and learn the addresses from the PUT.  The next step is to
send frames from the TX port to the PUT with each of the addresses generated
by the PUT.

At this point, the DUT/SUT will do one of three things with each of the
frames offered:
	1. Forward
	2. Flood
	3. Filter

By looking at the frame counts on the PUT and SPY ports, you can determine
what the DUT/SUT did with each of the offered frames.  If the DUT/SUT
Flooded or Filtered the offered frames, the test iteration failed.  There
MUST be a minimum of 3 ports to run this test.

How do we expand beyond this?  Testing 3 ports versus more will give
different results.  Some DUT/SUT use a global address pool for address
lookup, while other don't.  And the next generation of switches need to be
tested per port, per blade, and per switch.

The method described in the draft is the simplest, it works, and will
generate consistent results.  Do you have an algorithm that will sprinkle
the total addresses, and determine the three Fs?  And generate consistent
results?  If you have something that works, I'm listening.

Jerry Perser
SmartLab Manager
Netcom Systems
[email-masked]

---

From: John Dawson <[email-masked]>
Date: Mon, 07 Jun 1999 12:49:41 -0700

>   increments of three.  Each group of three will contain a test

Performing this test in multiples of three ports does not seem logical to
me.  I can see the usefulness of dedicating one port to look for flooding
while the other ports verify forwarding, but it would be overkill to
dedicate one flooding detector port for each pair of ports being tested.  A
single flood detector for each VLAN in the test process seem to make a lot
more sense.

I would also like to restate my objections to the need for reporting only
perfect learning rates.  I believe the learning rate measurement are too
focused on perfect learning characteristics (i.e. the switch is not allowed
to miss an offered address).

The address learning methodology tries to determine the loss-less address
learning rate, rather than the typical address learning rate.  The current
algorithm assumes that even if a single address is not-learned then all
other address that are learned are meaningless.  This has caused problems
in the past, at my company, interpreting results for switches that employ
adaptive algorithms (algorithms that can fail to learn every address in a
perfect fashion).  I would suggest that a more appropriate address learning
methodology would test a quantity of ports at the same time and be
structured in a fashion similar to forwarding rates - in this case
documenting learning rate at a specific offered load.

For instance if a switch is offered 1000 packets in 1 second and learns 900
of these addresses, then the learning rate is 900 packets per second (not
test inconclusive invoke a binary search).

Since this concern was brought up a year ago, how do we go about actually
considering this concern?

Thank you for your consideration in this matter.
John

---

From: David Newman <[email-masked]>
Date: Mon, 07 Jun 1999 17:40:40 -0400

> learning rate, rather than the typical address learning rate.  

Although I'm sympathetic to the point you raise (the current test is a
pain to run), I'm not so sure about the solution proposed. Any metric
other than a perfect learning rate implies that some amount of loss is
permissible. 

This is an approach that is inconsistent with the bmwg's previous work.
For example, RFC 1242, section 3.12, states that loss is what happens in
overload situations, which this isn't. And the goal of RFC 2544's
description of frame loss tests (section 26.3) is to find the highest
offered rate at which *no* loss occurs. 

What is an acceptable amount of loss? This is a slippery slope.

> test inconclusive invoke a binary search).

900 out of what? This approach requires the reporting of two metrics
(addresses offered, addresses learned) while a perfect-learning approach
requires reporting of just one number (addresses learned). 

And if I'm one of the 100 that got dropped I'm going to be a pretty
unhappy camper. . .

dn