Subject: [bmwg] AD Review of draft-ietf-bmwg-b2b-frame

From: Warren Kumari <[email-masked]>
Date: Mon, 9 Nov 2020 10:22:10 -0500

Hi there ADs and WG.

Thank you for this document, I found it clear, useful, and an easy read.
I do have a few comments/nits that addressing now should help the IETF
LC and iesg eval go more smoothly. Some of the plenary discussions at
IETF108 brought to light a desire from the community that nits and
similar be better addressed before LC/eval/the RFC Ed, and so I'm
trying to be especially diligent about noting them.

Please SHOUT loudly once you've had a chance to address these. We are
during a posting-blackout, but seeing as the document is already in
"Publication Requested" state, and these are nits/readability, I'm
happy to approve posting a new version anyway...

Comments:
1: RFC2119 was updated/replaced with RFC8174. Replacing RFC2119 with RFC8174 in:
"Thus, the requirements presented in this memo are expressed in
[RFC2119] terms,... " would probably be a good idea (or explain in the
document why not). This will require some text massaging in the prior
sentence[0].

2: "Fortunately, the Open Platform for Network Function Virtualization
(OPNFV) VSPERF project’s Continuous Integration (CI) testing ..."
Is there a link/reference that we can insert for this?

3: "Back-to-back Frame Benchmark was very consistent for some fixed
frame sizes, and somewhat variable for others."
s/for others/for other frame sizes/ ? It might just be me, but this
took me a few tries to understand. This might be a nit...

4: "It’s important that the buffering time was used in this analysis."
This reads like a fragment -- *why* is it important? Perhaps "it's
important to note that...", but I must admit I still don't quite
understand what this sentence is adding - that 30 seconds is
unrealistically long and someone should have asked "...what, what?!"?

 5: Similarly, I'm having a hard time parsing "using results" in:
"It was found that the actual extent of buffer time in the DUT could
be estimated using results to measure the longest burst"
Using *what* results?
Perhaps:
"It was found that the actual extent of buffer time in the DUT could
be estimated by measuring the longest burst in frames without loss and
results from the Throughput tests conducted according to Section 26.1
of [RFC2544]." ? But I actually have no idea, and so am just
guessing...

6: Section 7 - Security Considerations
I *think* that it might be useful to add:
"[RFC Editor, please remove before publication. This is the default
security considerations text which is included in BMWG documents - see
rfc6815 for more info.] "
or similar. This may help having to explain this again....

Nits:
1: "The IETF’s fundamental Benchmarking Methodologies are defined
in[RFC2544],..."
There is a missing space before the reference.

2: "In particular, analysis of the results indicates that buffers size
matters when compensating for disruptions in the software packet
processor,  "
s/buffers/buffer/ (or "that the size of buffers matters", or similar)

3: "The tests described in this memo address the cases where the
maximum frame rate"
s/cases where/cases in which/? (readability)

4: The [RFC8239] methods measure
   buffer latency directly with traffic on multiple ingress ports that
   overload an egress port on the Device Under Test (DUT), and are not ...
s/(DUT),./(DUT)/ spurious comma.

5: To summarize, there are several reasons that
   devices on a network produce bursts of frames at the minimum allowed
   spacing, and it is therefore worthwhile to understand the Device
[O]  spacing, and it is therefore worthwhile
[P] spacing; and it is, therefore, worthwhile

6:  3. Calculation of the extent of buffer time in the DUT helped to
       explain the results observed with all frame sizes (for example,
       some frame sizes cannot exceed the frame header processing rate
       of the DUT and therefore no buffering occurs, therefore the
[O] occurs, therefore
[P] occurs; therefore,

7: The presentation of OPNFV VSPERF evaluation and development of
   enhanced search alogorithms [VSPERF-BSLV] was discussed at IETF-102.

[O] alogorithms
[P] algorithms

   The enhancements are intended to compensate for transient inerrrupts

[O] inerrrupts
[P] interrupts

8: Therefore, sources of packet loss that are un-related

[O] un-related
[P] unrelated

9: Fortunately, the adaptation of Binary
   Search, and an enhanced Binary Search with Loss Verification have
   been specified in clause 12.3 of [TST009].  These alogorithms can

[O] alogorithms
[P] algorithms

   easily be used for Back-to-back Frame benchmarking by replacing the
   Offered Load level with burst length in frames.  [TST009] Annex B
   describes the theory behind the enhanced Binary Search with Loss
   Verification algorithm.

   There is also promising work-in-progress that may prove useful in for

[O]  useful in for
[P] useful in

10: The "Measured Throughput" is the [RFC2544] Throughput Benchmark
       for the frame size tested, as augmented by methods including the
       Binary Search with Loss Verification aglorithm in [TST009] where

[O] aglorithm
[P] algorithm

Thank you again,
W

[0]: Some people take the 2119->8174 change more seriously than
others. I don't really care, but it's a hot-button for some, and so
addressing it now seems cleaner....

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 11 Nov 2020 22:11:08 +0000

Hi Warren,

Thanks for your comments on the b2b-frame draft. Please see replies below, [acm].

Al

> -----Original Message-----
> From: Warren Kumari [mailto:[email-masked]]
> Sent: Monday, November 9, 2020 10:22 AM
> To: [email-masked]; [email-masked]
> Subject: AD Review of draft-ietf-bmwg-b2b-frame
> 
> Hi there ADs and WG.
> 

[acm]
<SHOUTING>
I will send a separate message about this => The .txt and .xml files are ready!
</SHOUTING>

> document why not). This will require some text massaging in the prior
[acm] 
Yes, I have updated both sentences to reflect the arrival of RFC 8174 and its clarifications.

Please see the reply to [0] as well.

> Is there a link/reference that we can insert for this?
[acm] 
Yes, there's a wiki page:
https://wiki.opnfv.org/display/vsperf/VSPERF+CI

> took me a few tries to understand. This might be a nit...
[acm] 
Yes, you are correct, I made the substitution above.

> unrealistically long and someone should have asked "...what, what?!"?
[acm] 
Yes, I noticed this difficult sentence just recently while putting the memo to use :-) in a current study. I think some of it may have arrived by way of welcome comments/ideas, but I didn't do a good job of fitting all together at the first attempt: 

It was important that the buffering time calculations were part the referenced testing and analysis [VSPERF-b2b], because the calculated buffer times of 30 seconds for some frame sizes were clearly wrong or highly suspect.

> of [RFC2544]." ? But I actually have no idea, and so am just
[acm]
Sorry about that.  This version may help:

It was found that a better estimate of the DUT buffer time could be calculated using measurements of both the longest burst in frames without loss and results from the Throughput tests conducted according to Section 26.1 of [RFC2544]. 

> or similar. This may help having to explain this again....
[acm]
Well, if we remove the entire section at publication, then there will be no requirements like:

The benchmarking network topology will be an independent test setup and MUST NOT be connected to devices that may forward the test traffic into a production network, or misroute traffic to the test management network.

which the SEC-DIR reviewers consider a big deal, and rightfully so. BMWG's test conditions often require link utilization of 100% for extended periods of time, and the traffic should not appear in the Internet. In fact, BMWG's dedicated IPv4 and IPv6 address space assignments allow production network devices to discard matching packets when received.

I think we leave this section as-is.

> There is a missing space before the reference.
[acm] 
thanks - my "old" version of XMLMind made it appear as though there was a space ...

> s/buffers/buffer/ (or "that the size of buffers matters", or similar)
[acm]

I went with:
In particular, analysis of the results indicates that buffer size matters when compensating for interruptions of software packet processing,...

> s/cases where/cases in which/? (readability)
[acm]

yeah, that sounds better, thanks.

> s/(DUT),./(DUT)/ spurious comma.
[acm] 
ok

> [P] spacing; and it is, therefore, worthwhile
[acm]
ok

>        of the DUT and therefore no buffering occurs, therefore the
[acm]
I suddenly saw that "therefore" appears twice in close proximity.
Here's what I wrote:
... tests with some frame sizes cannot exceed the frame header processing rate of the DUT and thus no buffering occurs; therefore, the results depended on the test equipment and not the DUT).

>    enhanced search alogorithms [VSPERF-BSLV] was discussed at IETF-102.
[acm]
Sorry about these, it's also a disadvantage that my version of XMLMind lacks spell-check. 
 
>    The enhancements are intended to compensate for transient inerrrupts
[acm]
ok

> 8: Therefore, sources of packet loss that are un-related
[acm]
ok

>    been specified in clause 12.3 of [TST009].  These alogorithms can
[acm]
Hmm, I didn't know I was mis-typing this so often...
>    There is also promising work-in-progress that may prove useful in for
[acm]
ok

>        Binary Search with Loss Verification aglorithm in [TST009] where
[acm] 
... and that someone named "al" would avoid this typo more easily!

> addressing it now seems cleaner....
[acm]
Yes, I know that Barry is one of them. BMWG also takes the Requirements Language seriously (as described in the Intro, Scott Bradner created it for our use, despite our work being limited to IETF-stream Informational RFCs).