Subject: [bmwg] review of draft-ietf-bmwg-mlrsearch-01

From: Vladimir Vassilev <[email-masked]>
Date: Sun, 7 Nov 2021 18:57:40 +0100

Hi,

I reviewed the draft. IMO it proposes solutions for 4 independent problems:

1. RFC2544 throughput search algorithm using shorter trial periods 
initially corresponding to less precise throughput result range and 
increase of trial period and precision up to the final trial sequence 
with standard period.

2. RFC2544 throughput search algorithm with smart initial phase that 
uses maximum transmit rate (MTR) to discover the maximum receive rate 
(MRR) which is used to calculate the effective minimum throughput for 
the initial range searched by the algorithm.

3. Throughput benchmarks where the acceptance criteria is less strict 
then the private case of zero packet drops specified in RFC2544.

4. The information of the packet drop ratio from all trials from all 
preceding runs is used for reduction of tested throughput range for 
following runs with more relaxed acceptance criteria.

(4.) is reflected in the title of the Draft - "Multiple Loss Ratio 
Search for Packet Throughput

I think the algorithm can be implemented/described simpler if the fact 
that these problems are indeed independent is taken into account.

For example instead of one single loop which updates one non-drop rate 
(NDR) and several partial drop rate (PDR) ranges it can be described as 
simpler algorithm that updates a single range. The followup PDR runs can 
reuse the same logic and state machine except they will need to take the 
reduced ranges as input. It will also make the algorithm easier to debug 
and the output will be of several simpler tables.

This is possible since the information from the relaxed packet drop 
acceptance criteria at shorter trial periods does not seem to be helpful 
in optimizing the search for stricter packet drop acceptance criteria 
ranges.

There are 3 nits I would like to point out at this stage all in the 
Terminology section:

1. "Packet size: same as frame size, both terms used interchangeably." 
IMO it is enough to use "frame size" only as RFC2544 does unless there 
is a good reason not to.

2. DUT/SUT is introduced and used. If the text does not differentiate between DUT and SUT I think that keeping with DUT is enough in the spirit of RFC2544. There is nothing that says a DUT can not be a software only based virtual device.

3. The following definition is controversial "Bandwidth throughput rate bw_rate = pkt_rate * (frame_size + L1_overhead) * 8". I think there was a reason RFC2544 did not go into details and just used packets per second. We can do the same.

I intend to implement support for the 4 solutions proposed in this draft as an alternative search algorithm part of our RFC2544 test.

/Vladimir