Subject: [rmcat] Proposed text on fairness for draft-ietf-rmcat-eval-criteria

From: "Xiaoqing Zhu (xiaoqzhu)" <[email-masked]>
Date: Mon, 26 Mar 2018 15:35:52 +0000

Hi,

This is to follow up on a discussion we had at the recent IETF-101 RMCAT WG, regarding how to address the notion of "fairness" in the eval-criteria draft.

Brief recap for folks not at the meeting: There was consensus to remove the current open issue (#1) on using Jain's Fairness Index (JFI) as fairness metric since nobody used it in their tests. On the other hand, there were some debates/discussions regarding whether & how to address the issue of fairness in the eval-criteria draft.  I signed up to provide some updated text to cover this topic.

As promised, please find below my proposed revision for #7 in Sec. 3, Metrics:

-------------------------------------

7. Self-Fairness and Fairness with respect to cross traffic:  Experiments testing a given RMCAT proposal must report on relative ratios of the average throughput (measured at coarser time intervals) obtained by each RMCAT stream. In the presence of background cross-traffic such as TCP, the report must also include the relative ratio between average throughput of RMCAT streams and cross-traffic streams.

During static periods of a test (i.e., when bottleneck bandwidth is constant and no arrival/departure of streams),  these report on relative ratios serve as an indicator of how fair the RMCAT streams share bandwidth amongst themselves and against cross-traffic streams. The throughput measurement interval can be set at a few values --- for example, at 1s, 5s, and 20s --- so as to measure fairness across different time scales.

As a general guideline, the relative ratio between RMCAT flows with the same priority level and similar path RTT should be bounded between (0.333 and 3.)

--------------------------------------

The last part of this write-up spells a rough guide-rail on fairness. I've removed the other two criteria in the original text for the following reasons:

* On 1. Does not trigger the circuit breaker:  this is already stated in the intro and should always apply;

* On 3. RTT should not grow by a factor of 3 for the existing flows when a new flow is added: I understand this to be more a measure of low-latency instead of fairness.   Also I am not aware that any eval tests have used this in the past.  Am I missing something here?

Additional feedback and input is welcome.  I'll leave it to the authors on how they want to incorporate this.

Thanks,

Xiaoqing

---

From: Colin Perkins <[email-masked]>
Date: Thu, 29 Mar 2018 17:34:29 +0100

Hi,

Thanks, Xiaoqing!  Along with the minutes, summarising the other decisions (https://datatracker.ietf.org/meeting/101/materials/minutes-101-rmcat-01 <https://datatracker.ietf.org/meeting/101/materials/minutes-101-rmcat-01>), it should now be possible to revise this draft. Jörg, do you have everything you need?

Cheers,
Colin

> On 26 Mar 2018, at 16:35, Xiaoqing Zhu (xiaoqzhu) <[email-masked]> wrote:
> 
> Hi,
> 

-- 
Colin Perkins
https://csperkins.org/

---

From: Joerg Ott <[email-masked]>
Date: Fri, 30 Mar 2018 13:09:59 +0200

Hi Colin and all,

yes, I believe so but I didn't check the details yet.  I will only get
to this work item next week.

Cheers,
Jörg

On 29.03.18 18:34, Colin Perkins wrote:
> Hi,
> 
> Thanks, Xiaoqing!  Along with the minutes, summarising the other 
> decisions 
> (https://datatracker.ietf.org/meeting/101/materials/minutes-101-rmcat-01), 
> it should now be possible to revise this draft. Jörg, do you have 
> everything you need?
> 
> Cheers,
> Colin
> 
> 
> 
> 
>> On 26 Mar 2018, at 16:35, Xiaoqing Zhu (xiaoqzhu) <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>> Hi,
>>
>> This is to follow up on a discussion we had at the recent IETF-101 
>> RMCAT WG, regarding how to address the notion of "fairness" in the 
>> eval-criteria draft.
>>
>> Brief recap for folks not at the meeting: There was consensus to 
>> remove the current open issue (#1) on using Jain's Fairness Index 
>> (JFI) as fairness metric since nobody used it in their tests. On the 
>> other hand, there were somedebates/discussions regarding whether 
>> &how to address the issue of fairness in the eval-criteria draft.  I 
>> signed up to provide some updated text to cover this topic.
>>
>> 7. Self-Fairness and Fairness with respect to cross 
>> traffic:  Experiments testing a given RMCAT proposal must report 
>> on relative ratios of the average throughput (measured at 
>> coarser time intervals) obtained by each RMCAT stream. In the presence 
>> of background cross-traffic such as TCP, the report must also 
>> include the relative ratio between average throughput of RMCAT streams 
>> and cross-traffic streams.
>>
>> During static periods of a test (i.e., when bottleneck bandwidth is 
>> constant and no arrival/departure of streams),  these report 
>> on relative ratios serve as an indicator of how fair the RMCAT streams 
>> share bandwidth amongst themselves and against cross-traffic streams. 
>> The throughput measurement interval can be set at a few values --- for 
>> example, at 1s, 5s, and 20s --- so as to measure fairness 
>> across different time scales.
>>
>> As a general guideline, the relative ratio between RMCAT flows with 
>> the same priority level and similar path RTT should be bounded between 
>> (0.333 and 3.)
>> The last part of this write-up spells a rough guide-rail on fairness. 
>> I've removed the other two criteria in the original text for the 
>> following reasons:
>> * On 3. RTT should not grow by a factor of 3 for the existing flows 
>> whena new flow is added: I understand this to be more a measure of 
>> low-latency instead of fairness.   Also I am not aware that any 
>> eval tests have used this in the past.  Am I missing something here?
>>
>> Additional feedback and input is welcome.  I'll leave it to the 
>> authors on how they want to incorporate this.
>>
>> Thanks,
>> Xiaoqing
> 
> -- 
> Colin Perkins
> https://csperkins.org/
> 
> 
> 
>