Subject: draft-ietf-bmwg-mswitch-01.txt

From: "Debby Stopp" <[email-masked]>
Date: Thu, 3 Jun 1999 14:36:43 -0700

A few comments:
>From section 4:
<!--StartFragment-->
   Devices switching tagged frames of over 1518 bytes will have a lower
maximum forwarding rate than standard untagged frames.
<!--EndFragment-->

Comment:
How is this comment relevant to this document?  We know that as the frame
size increases, the forwarding rates will drop.  Is this suggesting that
devices that handle tagged frames (ie., vlan, isl, etc) will generally have
a lower throughput?  Or is this suggestion that the frame itself just takes
longer because the tag makes it a larger frame.  For example, the max
throughput for a 64-byte frame on a 10mbit interface is about 14881fps.  If
we assume that by tagging the frame, it then becomes a 68-byte frame, the
max throughput then drops to about 14205fps, or about 5% reduction in
throughput.

My opinion is that the total framesize, regardless of any additional
tagging, should be used for the throughput calculation and therefore whether
the frame is tagged or not becomes a non-issue.

>From section 5.1.2:
<!--StartFragment-->
.      Frame Size - Recommended frame sizes are 64, 128, 256, 512, 1024,
      1280 and 1518 bytes, per RFC 2544 section 9.  The four CRC bytes
      are included in the frame size specified.
<!--EndFragment-->

Again, here the framesize is the basic list from rfc1242, no mention of
tagging, etc.

Perhaps we need to consider an additional document that handles the metrics
associated w/devices that are specifically configured to forward tagged
frames?
***

More from sections 5.1.2 & 5.2.2:
<!--StartFragment-->
.      Addresses per port - Represents the number of addresses which
      are being tested for each port.  Number of addresses SHOULD be a
      binary exponential (i.e. 1, 2, 4, 8, 16, 32, 64, 128, 256, ...).
      Recommended values are 1, 16 and 256.
<!--EndFragment-->

Why a binary exponential?  I realize that this is a 'SHOULD', but I believe
we SHOULD make this a little more generic & flexible at the same time.
Perhaps it would be a better test to specify the number of addresses per
port based on the size of the address forwarding table..?

Congestion Tests:
I'm not sure that this test is any more or less meaningful when more than
one address per port is tested.  Is this just a cut & paste exercise, or is
there some additional information we can glean from the sut/dut when
performing congestion tests w/multiple addresses per port?

>From section 5.3.3 (
<!--StartFragment-->
This test MUST consist of a multiple of four ports.  Four ports are
   REQUIRED and MAY be expanded to fully utilize the DUT/SUT in
   increments of four.  Each group of four will contain a test block
   with two of the ports as source transmitters and two of the ports as
   receivers.
<!--EndFragment-->

Why MUST this test consist of multiple of four ports?  why not a minimum of
4 ports?  I suggest the following setup should be allowed:
	tx1	--->	rx1
	tx2	___
		       \    rx2
	tx3	___ /
	tx4       ---->      rx3

Or conversly:
	tx1	--->	rx1
	tx2	___
		       \    rx2
	tx3	___ /
	tx4	___
		       \    rx3
	tx5	___ /

In my opinion, the important bit of the HOL test is that we have *at least
one* congested interface & *at least one* uncongested interface.

One other comment regarding congestion - I'm curious why there isn't a
section describing the methodology for Backpressure (rfc 2285, section
3.7.1).  Has this been rolled into section 5.3?

Debby Stopp
[email-masked]
IXIA Communications

---

From: "Robert Mandeville" <[email-masked]>
Date: Tue, 8 Jun 1999 11:34:56 +0200

----- Original Message -----
From: Debby Stopp <[email-masked]>
To: bmwg <[email-masked]>
Sent: Thursday, June 03, 1999 11:36 PM
Subject: draft-ietf-bmwg-mswitch-01.txt

> the frame is tagged or not becomes a non-issue.

I agree that "We know that as the frame size increases, the forwarding rates
will drop." The purpose of the statement is simply a reminder to the person
testing not to forget that a frame will be longer specifically when tags are
part of the test setup. We can assume that that goes without saying. I think
it does no harm to issue an explicit reminder.

> Again, here the framesize is the basic list from rfc1242, no mention of

Are you suggesting we should put in the reminder here too? I'd be for that.

> associated w/devices that are specifically configured to forward tagged
Best to handle the issue in this document.

> port based on the size of the address forwarding table..?
This would make the result relative to the capacity of the device under
test. Since this can be different for different devices, comparing results
is made very difficult. I never do this for that reason.

> performing congestion tests w/multiple addresses per port?

It is important for the user to know that the device can handle a large
address table when under stress. This is a test I am often asked to perform
in large account environments.

> one* congested interface & *at least one* uncongested interface.

As long as some of the traffic going to the congested port is coming from a
transmit port which is also sending to the uncongested port. That is what
HOLB means. I can't make out from your diagram if this fundamental condition
is met.

> 3.7.1).  Has this been rolled into section 5.3?

yes

---

From: "Debby Stopp" <[email-masked]>
Date: Thu, 17 Jun 1999 13:55:39 -0700

On page 4 of the mswitch draft, I noticed that 'Test Duration' is stated as
such:

      Test Duration - Test duration SHOULD be between 1 and 300 seconds.
      RFC 2285 recommends a duration of at least 10 seconds for each
      test.

First of all, I've scoured RFC 2285 but I can't find the reference, however
I did find a similar reference in RFC 2544:

24. Trial duration

   The aim of these tests is to determine the rate continuously
   supportable by the DUT.  The actual duration of the test trials must
   be a compromise between this aim and the duration of the benchmarking
   test suite.  The duration of the test portion of each trial SHOULD be
   at least 60 seconds.  The tests that involve some form of "binary
   search", for example the throughput test, to determine the exact
   result MAY use a shorter trial duration to minimize the length of the
   search procedure, but  the final determination SHOULD be made with
   full length trials.

Secondly, I don't believe that 10 seconds is long enough.  On a DUT/SUT
w/some fairly sizable buffers, this may not fully exercise the device's
buffer & buffer overflow systems.  Therefore, I believe that we should try
to be consistant w/the previous work, ie something like this:

Test Duration - Test duration SHOULD be between a minimum of 60 seconds.

Debby Stopp
Ixia Communications
[email-masked]

---

From: "Debby Stopp" <[email-masked]>
Date: Fri, 18 Jun 1999 13:28:06 -0700

Sorry for the late response...

Robert Mandeville wrote:
> > 4 ports?
<snip>
> HOLB means. I can't make out from your diagram if this

Sorry for the tab problem.  What I was trying to get at was that I don't see
why it's necessary to restrict the port configuration to *only* 4 ports -
why not have 3 Tx ports transmitting to one overloaded Rx port & 2
uncongested ports... or some other variation.  Why not just use all
available ports on the DUT/SUT... so long as the initial constraint is being
met, that is, "some of the traffic going to the congested port is coming
from a transmit port which is also sending to the uncongested port", I don't
see the need to restrict the configuration to only 4 ports.

> > 3.7.1).  Has this been rolled into section 5.3?

Why?  I seem to recall that there was some discussion regarding terminology
vs. methodology drafts at the last meeting - the general consensus, if
memory serves me, was that if it was in the terminology RFC, then it should
be included in the methodology; if it wasn't in the terminology RFC then it
should not be addressed in the methodology.

RFC2285, section 3.7, describes 3 variations on congestion control testing
techniques: 3.7.1 - Backpressure, 3.7.2 Forward Pressure & 3.7.3 Head of
Line blocking.  I would like to see the methodology address all congestion
control variations listed in RFC2285, not just the popular one.

Debby Stopp
Ixia Communications
[email-masked]

---

From: "Howard C. Berkowitz" <[email-masked]>
Date: Wed, 23 Jun 1999 10:49:28 -0400

>see the need to restrict the configuration to only 4 ports.

I rather like the idea of having a required test of four ports, with
optional tests for larger numbers of ports.  Four ports are the minimum to
create the configuration that tests the behavior in question, so that it
can be considered a baseline test configuration for any switch.  It allows
apples-to-apples comparisons between different implmentations.

I grant that large switches with many ports might show behavior that a
four-port configuration does not, and for those, I could see either
multiples-of-four or all available port configurations.

The idea of having a generally achievable baseline, however, has merit.

Howard Berkowitz
Gett Communications