Subject: Re: [bmwg] I-D ACTION:draft-ietf-bmwg-ipv6-meth-03.txt

From: David Newman <[email-masked]>
Date: Fri, 31 Aug 2007 22:17:12 -0700

On 8/30/07 9:15 AM, [email-masked] wrote:
> A New Internet-Draft is available from the on-line Internet-Drafts 
> directories.
> This draft is a work item of the Benchmarking Methodology Working Group of the IETF.
> 
> 	Title		: IPv6 Benchmarking Methodology for Network Interconnect Devices
> 	Author(s)	: C. Popoviciu, et al.
> 	Filename	: draft-ietf-bmwg-ipv6-meth-03.txt
> 	Pages		: 19
> 	Date		: 2007-8-30
> 	
> The Benchmarking Methodologies defined in RFC2544 [8] are IP version
>    independent.  However, RFC 2544 does not address some of the
>    specificities of IPv6.  This document provides additional
>    benchmarking guidelines, which in conjunction with RFC2544, lead to a
>    more complete and realistic evaluation of the IPv6 performance of
>    network interconnect devices.  IPv6 transition mechanisms are outside
>    the scope of this document.
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-ietf-bmwg-ipv6-meth-03.txt

Is there a diff or better yet summary of changes between -02 and -03?

In -03 appendix A.2 I noticed the reappearance of "maximum throughput."
Since there is no such metric -- RFC 1242 defines throughput as a single
rate -- I would recommend using "maximum Ethernet frame rate reference"
as in RFC 2544, Appendix B.

dn

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: "Chip Popoviciu \(cpopovic\)" <[email-masked]>
Date: Tue, 4 Sep 2007 10:31:33 -0400

Hi David,

This might have been a remnant from an earlier edit. Thank you for
pointing this out, we will fix this per your suggestion.

We have a summary of the items discussed during the Last Call and the
actions taken. I just sent it to Al and he will make the document
available to the working group.

Best Regards,
Chip

-----Original Message-----
From: David Newman [mailto:[email-masked]] 
Sent: Saturday, September 01, 2007 1:17 AM
Cc: [email-masked]
Subject: Re: [bmwg] I-D ACTION:draft-ietf-bmwg-ipv6-meth-03.txt

dn

---

From: Al Morton <[email-masked]>
Date: Tue, 04 Sep 2007 14:52:39 -0400

>available to the working group.

It's posted here:
http://home.comcast.net/%7Eacmacm/BMWG/IPv6-Meth-Last-Call-resolution2.pdf

Thanks for preparing this summary!
Al

---

From: David Newman <[email-masked]>
Date: Tue, 04 Sep 2007 12:49:52 -0700

> Thanks for preparing this summary!

Hi Chip et al,

Here, I hope, is my final comment on frame sizes. Characterizing
minimum-length SONET frames as being of "no practical use" is
problematic, in that practicality is relative to the user.

I know you really mean "no practical use in any conceivable production
setting" but that's beside the point in the context of bmwg docs.
Certainly for any tester asked to determine maximum switch fabric
performance, the use of minimum-length frames is not only practical but
necessary.

I'd suggest deleting this paragraph:

"Note: The 47 bytes SONET frame has no practical use since it can carry
only the 40 bytes
header of an IPv6 packet and no upper layer protocol information.  It
represents however
the smallest frame size for this media type.?

If I were going to offer any comment on minimum-length SONET frames, it
would be to note that the minimum length may be 47 or 49 bytes depending
on the CRC field length in use.

dn

---

From: "Chip Popoviciu \(cpopovic\)" <[email-masked]>
Date: Tue, 4 Sep 2007 16:29:00 -0400

Hi David,

Your perspective on the methodology makes sense, indeed it should be
defined without assumptions of the scope for which it will be used. In
this sense there is no disagreement. The idea of the note was to put
things in context for the reader who might just take the recommendations
as they are and not relate them to his true needs. If the reader is
someone testing the maximum switch fabric performance or wants to
evaluate a limit case, that reader most likely knows what the data means
and what is its use. On the other hand, we might have readers who,
without understanding the context, would take the recommendation ad
literam.

Do you see value in providing some context to the limit case? I still
think of the comments made by Curtis (which I think are realistic):

"The ISPs don't care what BMWG specifies.  They will provide their own
number if they disagree.  The enterprise market does not have that
luxury and may also be more prone to pay attention to meaningless
benchmark figures that marketeers waive around if theirs is better."

The idea is not to discourage the use of that frame size but rather to
encourage proper interpretation of the results. The note might have been
poorly worded but that is what it intended to do. Either way, your
proposal to mention the 49 bytes option should be added.

Best Regards,
Chip  

-----Original Message-----
From: David Newman [mailto:[email-masked]] 
Sent: Tuesday, September 04, 2007 3:50 PM
To: [email-masked]
Subject: Re: [bmwg] I-D ACTION:draft-ietf-bmwg-ipv6-meth-03.txt

Hi Chip et al,

"Note: The 47 bytes SONET frame has no practical use since it can carry
only the 40 bytes header of an IPv6 packet and no upper layer protocol
information.  It represents however the smallest frame size for this
media type.?

dn

---

From: David Newman <[email-masked]>
Date: Tue, 04 Sep 2007 13:55:35 -0700

> proposal to mention the 49 bytes option should be added.

This is a methodology for stimulating a DUT/SUT and measuring the
effects of that stimulation.

It is not a methodology for analyzing those effects based on some
predetermined notion of what the user's "true needs" may be, nor is it a
guide to "proper interpretation."[1]

Those concepts both have many definitions, each in the eyes of their
beholders.

We don't know, can't know, and shouldn't know how different users will
interpret results. Those are subjective judgments, and far out of scope
here IMO.

dn

[1] In my day job I receive many generous offers to define true needs
and proper interpretations, usually from persons with "marketing" in
their titles.

---

From: "Chip Popoviciu \(cpopovic\)" <[email-masked]>
Date: Tue, 4 Sep 2007 17:45:46 -0400

In my opinion the methodology is well defined and clear. The notes
simply clarify aspects of the methodology and put things in context for
non-specialists who still use BMWG documents for reference. If this
commentary is diminishing the value of the document in the context of
the BMWG charter, we remove it and leave only the point about the 49
bytes option for the minimum size.

Thank you for your feedback!

Best Regards,
Chip

-----Original Message-----
From: David Newman [mailto:[email-masked]] 
Sent: Tuesday, September 04, 2007 4:56 PM
To: [email-masked]
Subject: Re: [bmwg] I-D ACTION:draft-ietf-bmwg-ipv6-meth-03.txt

> things in context for the reader who might just take the 
> recommendations as they are and not relate them to his true needs. If 
> the reader is someone testing the maximum switch fabric performance or

> wants to evaluate a limit case, that reader most likely knows what the

> data means and what is its use. On the other hand, we might have 
> readers who, without understanding the context, would take the 
> recommendation ad literam.
> 

> encourage proper interpretation of the results. The note might have 
> been poorly worded but that is what it intended to do. Either way, 
> your proposal to mention the 49 bytes option should be added.

dn