Subject: [rmcat] Benjamin Kaduk's No Objection on draft-ietf-rmcat-eval-test-09: (with COMMENT)

From: Benjamin Kaduk via Datatracker <[email-masked]>
Date: Thu, 07 Mar 2019 06:52:16 -0800

Benjamin Kaduk has entered the following ballot position for
draft-ietf-rmcat-eval-test-09: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-rmcat-eval-test/

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

We don't really explain the usage of the "variation pattern index" that
I can see.

Section 3

         +  Bottleneck-link capacity: defines minimum capacity of the
            end-to-end path

I'm not sure I'd describe this as the "minimum capacity of the
end-to-end path", since attempts to use anything larger than this value
will pile up at the bottleneck.

Section 5.6, 5.7

Do we want anyone to consider testing with alternative TCP congestion
control algorithms?

Section 6.1

If you're not going to reference what scheme is used for implementing
priority, say how these priority values are interpreted.

Section 6.2

nit: "delay-based" (hyphenated)

Section 6.3

   Expected behavior: The candidate algorithm is expected to achieve
   full utilization at both bottleneck links without starving any of the

I'm not sure how we can get "full utilization" at both bottlenecks, when
the links in question have different capacity ratios.

Section 11.2

Some of these may need to be normative references; e.g., if we require
default TCP congestion control (RFC5681) for the competing TCP flows,
then it's mandatory.

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Thu, 21 Mar 2019 13:41:28 +0000

Hi Benjamin,

Thanks for the comments. Please see inline below with [ZS] prefix.

BR
Zahed

On 2019-03-07, 15:52, "Benjamin Kaduk via Datatracker" <[email-masked]> wrote:

[ZS] right, that column could be removed. I think it is still there for potential reference in the discussion.
    
    Section 3

[ZS] hmm, I guess that is why that link will act as bottleneck link for a respective testcase. If the flows remain below that capacity it should not experience queuing and as you said anything above that capacity will observe queuing. The job of the congestion control algorithm will be to avoid queuing.
    
    Section 5.6, 5.7

[ZS]The text should be read as the minimum one need to do and should encourage other to try with more alternatives. Will add the encouragement text (there is already one for multiple media sources).
    
    Section 6.1

[ZS] not sure I understood the comments here. I don’t think we have any recommendation of which priority schemes one need to use. The testers are free to use any priority scheme and the share of the available bandwidth must reflect the priority imposed on flows. What we can ask is to describe the scheme used in the test so that results can be interpreted clearly. Like a particular SCREAM (https://tools.ietf.org/html/rfc8298#section-4.1.2.8) implementation uses a credit based scheme, then presenting results that scheme needs to be described by the proponents. Will this work?
    
    Section 6.2
    
[ZS] OK
    
    Section 6.3

    I'm not sure how we can get "full utilization" at both bottlenecks, when
    the links in question have different capacity ratios.
[ZS] there are two criteria here, 1) avoid starvation, 2) full utilization with fair share. In this particular test case the bottlenecks are not common for all the flows. The the S1-R1 flow will traverse both of the bottleneck. At time, it will share the bottleneck link with S2-R2 and further down the path it will share the bottleneck link with S3-R3. S1-R1 then obviously it will observe the minimum of the available bandwidth from all the bottlenecks present in the path (in this case C-D). As different flows are on the bottleneck the expectation is S2-R2 will fill the rest of the available bandwidth and none of them should starve. So, to me the text seems OK.
    
    Section 11.2
    
[ZS] OK

---

From: Benjamin Kaduk <[email-masked]>
Date: Sat, 23 Mar 2019 08:42:27 -0500

> On 2019-03-07, 15:52, "Benjamin Kaduk via Datatracker" <[email-masked]> wrote:

I think I was expecting "maximum capacity" to be more appropriate than
"minimum capacity".  Perhaps I'm confused about what is intended here.

> [ZS] not sure I understood the comments here. I don’t think we have any recommendation of which priority schemes one need to use. The testers are free to use any priority scheme and the share of th…

The example has priority values of 1 and 2 -- are higher numbers more
preferred or lower numbers more preferred?

> [ZS] there are two criteria here, 1) avoid starvation, 2) full utilization with fair share. In this particular test case the bottlenecks are not common for all the flows. The the S1-R1 flow will tr…

Thanks for the explanation.

-Benjamin

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Sun, 24 Mar 2019 09:33:33 +0000

On 2019-03-23, 14:42, "Benjamin Kaduk" <[email-masked]> wrote:

(disclaimer. this has been done in the long past and I hope my memory is supporting me on explaining it ☺)

The thinking was that the bottleneck-link capacity was not equals to the maximum link capacity. That means if the maximum link capacity is X then the bottleneck-link capacity is Y , where Y<X and (X-Y) capacity is used by other traffic traversing that link. Hence, in the test if a link is assumed to be bottleneck then the "the bottleneck-link capacity" is the minimum capacity available to the media flow traversing that path. 

Ok Understood, I will explain that in the text. What about the following text?

   "In this test case media flows will have different priority levels imposed by the application.
   This will be an extension of Section 5.4 where the same test will be
   run with different priority levels imposed on each of the media
   flows.  For example, the first flow (S1) is assigned a priority of 2
   whereas the remaining two flows (S2 and S3) are assigned a priority
   of 1.  Here, the higher priority number indicates higher application level priority for bandwidth distribution. The candidate algorithm must reflect the relative priorities
   assigned to each media flow.  In this case, the first flow (S1) must
   arrive at a steady-state rate approximately twice of that of the
   other two flows (S2 and S3).

---

From: Benjamin Kaduk <[email-masked]>
Date: Sun, 24 Mar 2019 08:51:34 -0500

> The thinking was that the bottleneck-link capacity was not equals to the maximum link capacity. That means if the maximum link capacity is X then the bottleneck-link capacity is Y , where Y<X and (…

Ah, that makes sense.  Maybe "minimum guaranteed capacity" or similar would
convey the intent better.

>    other two flows (S2 and S3).

Thanks!

-Benjamin

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Tue, 26 Mar 2019 08:18:00 +0000

I will update the description a bit more elaborative about the intention here.
Thanks for your review.

BR
Zahed