Subject: [bmwg] Third WGLC on draft-ietf-bmwg-traffic-management

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 4 Feb 2015 19:07:02 -0500

BMWG (and AQM):

A WG Last Call period for the Internet-Draft on
Traffic Management Benchmarking:

   http://tools.ietf.org/html/draft-ietf-bmwg-traffic-management

will be open from 4 February 2015 through 19 February 2015.

Please note that the RFC diff for this version indicates the 
introduction of many new instances of RFC 2119 Requirements Language:
http://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-traffic-management-02
Past reviewers should check the current draft and indicate their 
current opinion or support.

The second WGLC (on -01) closed on 6 January, 2015 with substantial 
comments and the authors believe they are now addressed.

The first WGLC (on -00) closed October 21 2014, with substantial 
comments.

This draft is continuing the BMWG Last Call Process. See
http://www1.ietf.org/mail-archive/web/bmwg/current/msg00846.html

Please read and express your opinion on whether or not this
Internet-Draft should be forwarded to the Area Directors for
publication as an Informational RFC.  Send your comments
to this list or [email-masked] and [email-masked]

Al
bmwg co-chair

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Mon, 16 Feb 2015 18:27:59 -0500

Hi Barry and Ramki,

This is getting closer, comments and suggestions below.
(you'll have to text search for locations, sorry)
Al
(as a participant)

BMWG:
The WGLC ends on the 19th.
Al
(as co-chair)

-=-=-=-=-=-=-

Global: Decide how you want to call-out references,
but the text in Brackets must be present:

use
RFC 1242 [RFC1242] defines...
or
[RFC1242] defines...

-=-=-=-=-=-=-=-

   - Lost Packets(LP): For all traffic management tests, the tester will
   transmit the test packets into the DUT ingress port and the number of
   packets received at the egress port will be measured.  The difference
   between packets transmitted into the ingress port and received at the
   egress port is the number of lost packets as measured at the egress
   port.  These packets must have unique identifiers such that only the
   test packets are measured.  For cases where multiple flows are
   transmitted from ingress to egress port (e.g. IP conversations), each
   flow must have sequence numbers within the test packets stream.
ACM:
I think some suggested references got confused...
OLD
   RFC 4737 and RFC 2680 [RFC2680] describe the need to establish the
   time threshold to wait before a packet is declared as lost, and this 
   threshold MUST be reported with the results. This metric shall be 
   reported as an integer number which cannot be negative.
NEW
   [RFC6703] and [RFC2680] describe the need to establish the
   time threshold to wait before a packet is declared as lost, and this 
   threshold MUST be reported with the results. This metric shall be 
   reported as an integer number which cannot be negative.
(see: http://tools.ietf.org/html/rfc6703#section-4.1)
ACM:
This benchmark is a bit confusing now, I suggest to simply measure OOO:
OLD   
   - Out of Sequence (OOS): in additions to the LP metric, the test
   packets must be monitored for sequence and the out-of-sequence (OOS)
   packets. RFC 4689 defines the general function of sequence tracking, 
   as well as definitions for in-sequence and out-of-order packets.  
   Out-of-order packets will be counted per RFC 4737 and RFC 2680 
   [RFC2680]. This metric shall be reported as an integer number which 
   cannot be negative.
NEW
   - Out of Order (OOO): in additions to the LP metric, the test
   packets must be monitored for sequence. [RFC 4689] defines the 
   general function of sequence tracking, 
   as well as definitions for in-sequence and out-of-order packets.  
   Out-of-order packets will be counted per [RFC 4737].
   This metric shall be reported as an integer number which 
   cannot be negative.

   - Packet Delay (PD): the Packet Delay metric is the difference 
   between the timestamp of the received egress port packets and the 
   packets transmitted into the ingress port and specified in RFC 1242 
   [RFC1242].  The transmitting host and receiving host time must be in
   time sync using NTP , GPS, etc.  This metric shall be reported as an
   real number of seconds which cannot be negative, which usually 
   indicates a time synchronization problem.
ACM:
there are too many "which"es in the last sentence. Try this:
NEW
   time sync using NTP , GPS, etc.  This metric SHALL be reported as a
   real number of seconds, where a negative measurement usually 
   indicates a time synchronization problem between test devices.
 
   - Packet Delay Variation (PDV): the Packet Delay Variation metric is
   the variation between the timestamp of the received egress port
   packets and specified in RFC 5481 [RFC5481].  Note that per RFC 5481, 
   this PDV is the variation of one-way delay across many packets in 
   the traffic flow.  Per the measurement formulat in RFC 5481, select 
   the high percentile of 99% and units of measure will be a real 
   number of seconds (negative is not possible for PDV and would 
   indicate a measurement error).   
Constantine                         January 27, 2015           [Page 10]
?
Internet-Draft     Traffic Management Benchmarking        November, 2014
ACM:
The first sentence for SR should give the definition, let's qualify it later. 
OLD   
  - Shaper Rate (SR): the Shaper Rate is only applicable to the 
   traffic shaping tests.  The SR represents the average egress output
   rate (bps) over the test interval.
NEW
  - Shaper Rate (SR): The SR represents the average DUT output
   rate (bps) over the test interval. The Shaper Rate is only applicable to the 
   traffic shaping tests.  
ACM:
Need the same fixes in SBB and SBI below, 
also s/shall/SHALL/
Also "(bytes back to back)" must respect the 
- Shaper Burst Bytes (SBB): the Shaper Burst Bytes is only applicable
   to the traffic shaping tests.  A traffic shaper will emit packets in
   different size "trains" (bytes back-to-back).  This metric
   characterizes the method by which the shaper emits traffic. Some
   shapers transmit larger bursts per interval, and a burst of 1 packet
   would apply to the extreme case of a shaper sending a CBR stream of
   single packets. This metric shall be reported in units of bytes,
   KBytes, or MBytes.
   
   - Shaper Burst Interval(SBI):  the interval is only applicable to the 
   traffic shaping tests and again is the time between shaper emitted
   bursts.  
ACM:  ... measured at the DUT output. (?)
OLD
   ...This metric shall be reported as an real number of
   seconds which cannot be negative, which usually indicates a time
   synchronization problem.
NEW
   ...This metric shall be reported as an real number of
   seconds.
(note: one test device is making this measurement, right? no sync problem...)

   The TCP implementation used by the test host MUST be specified in 
   the test results (e.g. TCP New Reno, TCP options supported, etc.). 
ACM:
(the results could be worthless without it)
OLD
   Additionally, RFC 3148 recommends that specific congestion control
   algorithm details that should also be included in the test results.
NEW
   Additionally, the test results SHALL provide specific congestion control
   algorithm details, as per [RFC3148].

ACM:
OLD  
   Finally, application throughput measurements are based off of Layer 4
NEW
   Finally, application throughput measurements are based on Layer 4

EOT
________________________________________
From: bmwg [[email-masked]] On Behalf Of MORTON, ALFRED C (AL)
Sent: Wednesday, February 04, 2015 7:07 PM
To: [email-masked]; [email-masked]
Subject: [bmwg] Third WGLC on draft-ietf-bmwg-traffic-management

BMWG (and AQM):

Al
bmwg co-chair

_______________________________________________
bmwg mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/bmwg

---

From: Barry Constantine <[email-masked]>
Date: Fri, 20 Feb 2015 23:32:45 +0000

Hi Al,

Thanks for the comments, one question in this area (see my initials BAC):

ACM:
The first sentence for SR should give the definition, let's qualify it later. 
OLD   
  - Shaper Rate (SR): the Shaper Rate is only applicable to the 
   traffic shaping tests.  The SR represents the average egress output
   rate (bps) over the test interval.
NEW
  - Shaper Rate (SR): The SR represents the average DUT output
   rate (bps) over the test interval. The Shaper Rate is only applicable to the 
   traffic shaping tests.  
ACM:
Need the same fixes in SBB and SBI below, also s/shall/SHALL/ Also "(bytes back to back)" must respect the
BAC:
What did you mean by ,,, / Also "(bytes back to back)" must respect the...???

Thank you,
Barry Constantine

JDSU Network and Service Enablement
Principal Member Technical Staff
301-325-7069

-----Original Message-----
From: bmwg [mailto:[email-masked]] On Behalf Of MORTON, ALFRED C (AL)
Sent: Monday, February 16, 2015 6:28 PM
To: [email-masked]
Subject: Re: [bmwg] Third WGLC on draft-ietf-bmwg-traffic-management

Hi Barry and Ramki,

This is getting closer, comments and suggestions below.
(you'll have to text search for locations, sorry) Al (as a participant)

BMWG:

-=-=-=-=-=-=-

Global: Decide how you want to call-out references, but the text in Brackets must be present:

use

-=-=-=-=-=-=-=-

 
Need the same fixes in SBB and SBI below, also s/shall/SHALL/ Also "(bytes back to back)" must respect the
   
ACM:  ... measured at the DUT output. (?) OLD

(the results could be worthless without it) OLD

ACM:
OLD  
   Finally, application throughput measurements are based off of Layer 4 NEW
   Finally, application throughput measurements are based on Layer 4

EOT

BMWG (and AQM):

A WG Last Call period for the Internet-Draft on Traffic Management Benchmarking:

Please note that the RFC diff for this version indicates the introduction of many new instances of RFC 2119 Requirements Language:
http://www.ietf.org/rfcdiff?url2=draft-ietf-bmwg-traffic-management-02
Past reviewers should check the current draft and indicate their current opinion or support.

The second WGLC (on -01) closed on 6 January, 2015 with substantial comments and the authors believe they are now addressed.

The first WGLC (on -00) closed October 21 2014, with substantial comments.

This draft is continuing the BMWG Last Call Process. See http://www1.ietf.org/mail-archive/web/bmwg/current/msg00846.html

Please read and express your opinion on whether or not this Internet-Draft should be forwarded to the Area Directors for publication as an Informational RFC.  Send your comments to this list or [email-masked] and [email-masked]

Al
bmwg co-chair

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Fri, 20 Feb 2015 22:22:37 -0500

Hi Barry,

I must have been interrupted, I meant to say
"respect the mandatory inter-frame gap."

Al

> From: Barry Constantine [mailto:[email-masked]]
> Sent: Friday, February 20, 2015 6:33 PM
> To: MORTON, ALFRED C (AL); [email-masked]
> Subject: RE: Third WGLC on draft-ietf-bmwg-traffic-management
> 
> Hi Al,
> 
> Please note that the RFC diff for this version indicates the introduction
> of many new instances of RFC 2119 Requirements Language:
> Please read and express your opinion on whether or not this Internet-Draft
> should be forwarded to the Area Directors for publication as an
> Informational RFC.  Send your comments to this list or [email-masked]
> and [email-masked]
> 
> Al
> bmwg co-chair
> 
> 
>

---

From: Barry Constantine <[email-masked]>
Date: Mon, 23 Feb 2015 17:45:12 +0000

Hi Al,

We incorporated all of your comments and posted today.

Thanks again for all of your great review on this work,
Barry and Ramki

-----Original Message-----
From: bmwg [mailto:[email-masked]] On Behalf Of MORTON, ALFRED C (AL)
Sent: Monday, February 16, 2015 6:28 PM
To: [email-masked]
Subject: Re: [bmwg] Third WGLC on draft-ietf-bmwg-traffic-management

Hi Barry and Ramki,

BMWG:

-=-=-=-=-=-=-

use

-=-=-=-=-=-=-=-

 

ACM:
OLD  

EOT

BMWG (and AQM):

Al
bmwg co-chair