Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 5 Nov 2013 23:02:18 +0000

Hi,

this is just a simple remainder that the WG will convene also this Friday;

As the mechanism update presentations took quite a bit longer than we had expected - and we appreciate the discussion these presentations have sparked - we agreed with Fred to have his presentation as the first agenda point on Friday; the second agenda point will be a presentation around AQM evaluation process and criteria.

The Room will be Regency B (3. Floor), and we will start at 12:30.

Best regards,

Richard Scheffenegger

---

From: "Akhtar, Shahid (Shahid)" <[email-masked]>
Date: Wed, 6 Nov 2013 20:38:24 +0000

Is a document on AQM evaluation and criteria available before the meeting?

Thanks. 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Scheffenegger, Richard
Sent: Tuesday, November 05, 2013 5:02 PM
To: [email-masked]
Cc: Fred Baker (fred) ([email-masked]); Naeem Khademi ([email-masked])
Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B

Hi,

Best regards,

Richard Scheffenegger

_______________________________________________
aqm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/aqm

---

From: David Ros <[email-masked]>
Date: Wed, 6 Nov 2013 13:49:24 -0800

Hi,

Not yet, sorry (couldn't make it in time before the deadline), but we're planning to submit it in the coming few weeks.

David

> The Room will be Regency B (3. Floor), and we will start at 12:30.

=================================================================
David ROS
http://www.rennes.enst-bretagne.fr/~dros/

We've heard that a million monkeys at a million keyboards could produce the Complete Works of Shakespeare; now, thanks to the Internet, we know this is not true. -- Robert Wilenski

---

From: "Akhtar, Shahid (Shahid)" <[email-masked]>
Date: Thu, 7 Nov 2013 16:59:52 +0000

Hi All,

Had some comments on Fred's document. I have added the comments as track changes in a word document to easily see them. I used the 02 version. 

Thanks. 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Scheffenegger, Richard
Sent: Tuesday, November 05, 2013 5:02 PM
To: [email-masked]
Cc: Fred Baker (fred) ([email-masked]); Naeem Khademi ([email-masked])
Subject: [aqm] IETF88 Fri 08Nov13 - 12:30 Regency B

Hi,

Best regards,

Richard Scheffenegger

---

From: [email-masked]
Date: Thu, 7 Nov 2013 18:19:15 -0000

Thanks.

Please note the current version of the draft is:
draft-ietf-aqm-recommendation-00

Diff here:
http://www.ietf.org/rfcdiff?url1=draft-baker-aqm-recommendation-02&difftype=--html&submit=Go%21&url2=draft-ietf-aqm-recommendation-00

... and that when we did the last edits we tried to include text that
relates to some of these comments.  Fred & I WILL look at your comments
and how they relate to the new draft revision and get back on the specific
points you raise.

Gorry

> Hi All,
>
> Had some comments on Fred's document. I have added the comments as track
> changes in a word document to easily see them. I used the 02 version.
>
> Thanks.
>
> As the mechanism update presentations took quite a bit longer than we had
> expected - and we appreciate the discussion these presentations have
> sparked - we agreed with Fred to have his presentation as the first agenda
> point on Friday; the second agenda point will be a presentation around AQM
> evaluation process and criteria.
>
>

---

From: "Akhtar, Shahid (Shahid)" <[email-masked]>
Date: Thu, 7 Nov 2013 18:21:05 +0000

Hi All,

I am resending the same comments I had sent in the earlier mail in e-mail format only (for ease of reading) as some may not have access to MS word.

Section 4.3 - Manual tuning

The document states and implies that AQMs should not require manual tuning. In section 4.3 it states "The algorithms that the IETF recommends SHOULD NOT require operational (especially manual) configuration or tuning". I recommend to change this sentence to: "The algorithms that the IETF recommends should minimize tuning or configurations changes for specific traffic or network conditions"

I would further argue that all AQMs will likely require or implicitly assume some type of configuration/tuning.

For example, if we take Codel:
*       For small thin links, such as 1-10Mbs DSL, the 5ms target would increase packet loss significantly and at 2Mbps, a single MTU time may even exceed the 5ms target.
*       If the average RTT of all flows going through a link is more than 500ms, e.g. for satellite, then the 100ms interval would prematurely drop packets before the sources have had a chance to reduce their sending rate. Or if the average RTT is very low - e.g. 10ms - such as for flows between data-center elements, then 100ms interval may be slow to signal congestion back to the sources and significant packet loss may have occurred before such signaling.

One of the the objectives of newer AQMs being defined here should be to minimize tuning, but we should recognize that likely tuning or some configuration cannot be eliminated altogether.

Section 4.7 - Further Research

AQM impact on end-user QoE
Research should also focus on improving end user QoE from AQMs rather than network related metrics only. Often a significant change in a network metric may only make a minimal change in end-user QoE and thus the value of such change may be minimal.

Suggestion on best buffer sizes
Research should make suggestions on how to configure buffer sizes with each type of AQM (e.g. 2xBDP etc) - explaining why/how such buffer sizes improve end-user QoE and network health.

Best way to leverage deployed AQMs
Research should be done on methods or configurations that leverage deployed AQMs such as RED/WRED to reduce delays and lockout for typical traffic which require minimal effort or tuning from the operator.

Best regards,

-Shahid.

-----Original Message-----
From: Scheffenegger, Richard [mailto:[email-masked]]
Sent: Thursday, November 07, 2013 11:44 AM
To: Akhtar, Shahid (Shahid)
Cc: Fred Baker (fred) ([email-masked]); Naeem Khademi ([email-masked])
Subject: Re: IETF88 Fri 08Nov13 - 12:30 Regency B

Hi shahid,

Thanks. Please note that not necessarily everyone has access to full featured ms word; a short summary of the nits sent to the list, or direct communication with the authors is the usual way.

It looks that many of you comments are not so much specifica of the draft but discussion points for the wg. Repeating them on list with some context would be good!

Best regards
   Richard

Von meinem iPhone gesendet

Am 07.11.2013 um 09:01 schrieb "Akhtar, Shahid (Shahid)" <[email-masked]>:

> <draft-baker-aqm-recommendation-02.doc>

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Thu, 7 Nov 2013 18:45:00 +0000

> Had some comments on Fred's document. I have added the comments as track changes in a word document to easily see them. I used the 02 version. 

Permit me to put your comments in email, along with my own views. Also adding my co-author and the other working group chair on the CC line; if he is like me, he receives far too much email, and mail that is explicitly to or copies him bubbles higher in the column.

>> 4.  Conclusions and Recommendations
>>   [snip] 
>>    3.  The algorithms that the IETF recommends SHOULD NOT require
>>        operational (especially manual) configuration or tuning.
> 
> Some tuning may be required or implicitly assumed for virtually all AQMs – please see my comment later.

That's an opinion. One of the objectives of Van and Kathy's work, and separately of Rong Pan et al's work, is to design an algorithm that may have different initial conditions drawn from a table given the interface it finds itself on, but requires no manual tuning. The great failure of RED, recommended in RFC 2309, is not that it doesn't work when properly configured; it's that real humans don't have the time to properly tune it differently for each of the thousands of link endpoints in their networks. There is no point in changing away from RED if that is also true of the replacement.

>>    7.  Research, engineering, and measurement efforts are needed
>>        regarding the design of mechanisms to deal with flows that are
>>        unresponsive to congestion notification or are responsive, but
>>        are more aggressive than present TCP.
>  
>       Do we want to make a suggestion on how to configure buffer sizes with each type of AQM here (e.g. 2xBDP etc) or simply state that research should be conducted on the best buffer sizes to use with AQM.
 
I'm not sure that buffer sizes are specific to AQM algorithms; I'd entertain evidence otherwise. Buffer *thresholds* ("at what point do we start dropping/marking traffic?") may differ between algorithms. Buffer size ("how many bytes/packets do we allow into the queue in the worst case?") is a matter of the characteristics of burst behavior in a given network and the applications it supports. If I have, say, a Map/Reduce application that simultaneously asks thousands of systems a question, the queues in the intervening switches will need to be able to briefly absorb thousands of response packets. The key word here is "briefly". When Van or Kathy talk about "good queue" and "bad queue", they are saying that burst behavior may call for deep queues, but we really want the steady state to achieve 100% utilization with a statistically empty queue if we can possibly achieve that.

>> 4.3.  AQM algorithms deployed SHOULD NOT require operational tuning
>>  
>>    A number of algorithms have been proposed.  Many require some form of
>>    tuning or initial condition.  This can make them difficult to use
>>    operationally.  Hence, self-tuning algorithms are to be preferred.
>>    The algorithms that the IETF recommends SHOULD NOT require
>>    operational (especially manual) configuration or tuning.
>  
> May be better to state that tuning should be minimized. For the second sentence “The algorithms that the IETF recommends should minimize tuning or configurations changes for specific traffic or network conditions”
>  
> I would argue that all AQMs will likely require or assume some type of configuration/tuning.
>  
> ·       For small thin links, such as 1-10Mbs DSL, the 5ms target would increase packet loss significantly and at 2Mbps, a single MTU time may even exceed the 5ms target.
>  
> ·       If the average RTT of all flows going through a link is more than 500ms, e.g. for satellite, then the 100ms interval would prematurely drop packets before the sources have had a chance to reduce their sending rate. Or if the average RTT is very low – e.g. 10ms – such as for flows between data-center elements, then 100ms interval may be slow to signal congestion back to the sources and significant packet loss may have occurred before such signaling.

What you describe is what I referred to as "initial conditions derived from the links the algorithm finds itself on". We may be in violent agreement there. If so, the wording I might suggest would be "SHOULD NOT require operational (especially manual) configuration or tuning apart from automated determination of initial conditions" or some such thing.

>> 4.7.  The need for further research
>>  
>>    The second recommendation of [RFC2309] called for further research in
>>    the interaction between network queues and host applications, and the
>>    means of signaling between them.  This research has occurred, and we
>>    as a community have learned a lot.  However, we are not done.
>>  
>>    We have learned that the problems of congestion, latency and buffer-
>>    sizing have not gone away, and are becoming more important to many
>>    users.  A number of self-tuning AQM algorithms have be found that
>>    offer significant advantages for deployed networks.  There is also
>>    renewed interest in deploying AQM and the potential of ECN.
>>  
>>    An obvious example of further research in 2013 is the need to
>>    consider the use of Map/Reduce applications in data centers; do we
>>    need to extend our taxonomy of TCP/SCTP sessions to include not only
>>    "mice" and "elephants", but "lemmings"?  "Lemmings" are flash crowds
>>    of "mice" that the network inadvertently tries to signal to as if
>>    they were elephant flows, resulting in head of line blocking in data
>>    center applications.
>  
> Such research should also focus on improving end user QoE from AQMs rather than network related metrics only. Often a significant change in a network metric may only make a minimal change in end-user QoE and thus the value of such change may be minimal.

There is also a question of what user is under discussion. If you take a look at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-0.pptx and specifically the third slide, you will see (and tomorrow afternoon I will discuss) a capture I took of pings from my home network to another site overnight. In the evening, we watched a movie (home network), and in the morning I had a video conference (office network). I'll tell you right now that both worked fine, and managing the delay to zero or allowing it to be one hundred ms would not have materially affected the QOE of either. However, my ping is a competing application, and it saw sustained increase in delay, variation in delay, and the possibility of loss as a result of the queuing. The value of AQM is in part to the application being throttled, but in large part to competing applications, and the QOE of both must be considered.

>>    Examples of other required research include:
>>  
>>    o  Research into new AQM and scheduling algorithms.
>>  
>>    o  Research into the use of and deployment of ECN alongside AQM.
>>  
>>    o  Tools for enabling AQM (and ECN) deployment and measuring the
>>       performance.
>>  
>>    o  Methods for mitigating the impact of non-conformant and malicious
>>       flows.
>  
> Methods or configurations that leverage deployed AQMs such as RED/WRED to reduce delays and lockout for typical traffic which require minimal effort or tuning from the operator.
 
Not a complete sentence, but I think I understand what you're getting at. You would like to have research determine how to easily configure existing systems using the tools at hand. I'm all for it in the near term.

---

From: [email-masked]
Date: Thu, 7 Nov 2013 19:01:59 -0000

Gorry - I added a few words to add to those from Fred, see in-line as [gf].

>
> On Nov 7, 2013, at 8:59 AM, "Akhtar, Shahid (Shahid)"
> <[email-masked]> wrote:
>
>> Hi All,
>>
> Permit me to put your comments in email, along with my own views. Also
> adding my co-author and the other working group chair on the CC line; if
> he is like me, he receives far too much email, and mail that is explicitly
> to or copies him bubbles higher in the column.
>
> That's an opinion. One of the objectives of Van and Kathy's work, and
> separately of Rong Pan et al's work, is to design an algorithm that may
> have different initial conditions drawn from a table given the interface
> it finds itself on, but requires no manual tuning. The great failure of
> RED, recommended in RFC 2309, is not that it doesn't work when properly
> configured; it's that real humans don't have the time to properly tune it
> differently for each of the thousands of link endpoints in their networks.
> There is no point in changing away from RED if that is also true of the
> replacement.
>
[gf] I note you commented on the previous individual submission. In the
latest version (WG -00 revision),  we tried to clarify the difference
between the overall goal and practicalities. Do you think the update
accommodate the need/ability for tuning to specific use-cases?

>>       Do we want to make a suggestion on how to configure buffer sizes
>> with each type of AQM here (e.g. 2xBDP etc) or simply state that
>> research should be conducted on the best buffer sizes to use with
>> AQM.
>
> I'm not sure that buffer sizes are specific to AQM algorithms; I'd
> entertain evidence otherwise. Buffer *thresholds* ("at what point do we
> start dropping/marking traffic?") may differ between algorithms. Buffer
> size ("how many bytes/packets do we allow into the queue in the worst
> case?") is a matter of the characteristics of burst behavior in a given
> network and the applications it supports. If I have, say, a Map/Reduce
> application that simultaneously asks thousands of systems a question, the
> queues in the intervening switches will need to be able to briefly absorb
> thousands of response packets. The key word here is "briefly". When Van or
> Kathy talk about "good queue" and "bad queue", they are saying that burst
> behavior may call for deep queues, but we really want the steady state to
> achieve 100% utilization with a statistically empty queue if we can
> possibly achieve that.
>
[gf] I would love to be able start making recommendations in this space,
but I think this is a lot more complicated.

But, "buffer size" in actual router/middlebox design isn't just a single
number, there are multiple buffers, thresholds and scheduling that
interplay and relate to specific methods.

I think this particular draft, as a BCP, should not delve into detail here.

>> May be better to state that tuning should be minimized. For the second
>> sentence “The algorithms that the IETF recommends should minimize tuning
>> or configurations changes for specific traffic or network conditions”
>>
>> ·       For small thin links, such as 1-10Mbs DSL, the 5ms target would
>> increase packet loss significantly and at 2Mbps, a single MTU time may
>> even exceed the 5ms target.
>>
>> ·       If the average RTT of all flows going through a link is more
>> than 500ms, e.g. for satellite, then the 100ms interval would
>> prematurely drop packets before the sources have had a chance to reduce
>> their sending rate. Or if the average RTT is very low – e.g. 10ms – such
>> as for flows between data-center elements, then 100ms interval may be
>> slow to signal congestion back to the sources and significant packet
>> loss may have occurred before such signaling.
>
>
> What you describe is what I referred to as "initial conditions derived
> from the links the algorithm finds itself on". We may be in violent
> agreement there. If so, the wording I might suggest would be "SHOULD NOT
> require operational (especially manual) configuration or tuning apart from
> automated determination of initial conditions" or some such thing.
>
[gf] Again, is the WG -00 draft closer or not to what you are thinking?

>> Such research should also focus on improving end user QoE from AQMs
>> rather than network related metrics only. Often a significant change in
>> a network metric may only make a minimal change in end-user QoE and thus
>> the value of such change may be minimal.
>
>
> There is also a question of what user is under discussion. If you take a
> look at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-0.pptx and
> specifically the third slide, you will see (and tomorrow afternoon I will
> discuss) a capture I took of pings from my home network to another site
> overnight. In the evening, we watched a movie (home network), and in the
> morning I had a video conference (office network). I'll tell you right now
> that both worked fine, and managing the delay to zero or allowing it to be
> one hundred ms would not have materially affected the QOE of either.
> However, my ping is a competing application, and it saw sustained increase
> in delay, variation in delay, and the possibility of loss as a result of
> the queuing. The value of AQM is in part to the application being
> throttled, but in large part to competing applications, and the QOE of
> both must be considered.
>
>> Methods or configurations that leverage deployed AQMs such as RED/WRED
>> to reduce delays and lockout for typical traffic which require minimal
>> effort or tuning from the operator.
>
> Not a complete sentence, but I think I understand what you're getting at.
> You would like to have research determine how to easily configure existing
> systems using the tools at hand. I'm all for it in the near term.
>

[gf] Gorry

---

From: David Collier-Brown <[email-masked]>
Date: Thu, 07 Nov 2013 14:19:34 -0500

It's written in the light of recent work that suggest that tuning is
unnecessary, but is not a proof that it will never be necessary.  If it
were, the wording would arguably be "must not" as opposed to "should not".

I think the emphasis in "especially manual", however, is inappropriate. 
I would prefer to see language that says that they should adapt to
measurable conditions themselves, and not require outside intervention
to inject information they require.  I'd then weaken it for initial
settings that are never changed, like "this is strictly for
IP-over-carrier-pigeons".

--dave

On 11/07/2013 01:21 PM, Akhtar, Shahid (Shahid) wrote:
> Hi All,
>  
> I am resending the same comments I had sent in the earlier mail in
> e-mail format only (for ease of reading) as some may not have access
> to MS word.
>  
> *Section 4.3 - Manual tuning*
>  
> The document states and implies that AQMs should not require manual
> tuning. In section 4.3 it states "The algorithms that the IETF
> recommends SHOULD NOT require operational (especially manual)
> configuration or tuning". I recommend to change this sentence to: "The
> algorithms that the IETF recommends should minimize tuning or
> configurations changes for specific traffic or network conditions"
>  
> I would further argue that all AQMs will likely require or implicitly
> assume some type of configuration/tuning.
>  
> .       For small thin links, such as 1-10Mbs DSL, the 5ms target
> would increase packet loss significantly and at 2Mbps, a single MTU
> time may even exceed the 5ms target.
> .       If the average RTT of all flows going through a link is more
> prematurely drop packets before the sources have had a chance to
> reduce their sending rate. Or if the average RTT is very low -- e.g.
> 10ms -- such as for flows between data-center elements, then 100ms
> interval may be slow to signal congestion back to the sources and
> significant packet loss may have occurred before such signaling.
>  
> One of the the objectives of newer AQMs being defined here should be
> to minimize tuning, but we should recognize that likely tuning or some
> configuration cannot be eliminated altogether.
>  
> *Section 4.7 - Further Research*
>  
> *AQM impact on end-user QoE*
> Research should also focus on improving end user QoE from AQMs rather
> than network related metrics only. Often a significant change in a
> network metric may only make a minimal change in end-user QoE and thus
> *Suggestion on best buffer sizes*
> Research should make suggestions on how to configure buffer sizes with
> each type of AQM (e.g. 2xBDP etc) - explaining why/how such buffer
> sizes improve end-user QoE and network health.
>  
> *Best way to leverage deployed AQMs*
> Research should be done on methods or configurations that leverage
> deployed AQMs such as RED/WRED to reduce delays and lockout for
> typical traffic which require minimal effort or tuning from the operator.
>  
> Best regards,
>  
> -Shahid.
>  
> From: Scheffenegger, Richard [_mailto:[email-masked]]
> Thanks. Please note that not necessarily everyone has access to full
> featured ms word; a short summary of the nits sent to the list, or
> direct communication with the authors is the usual way.
>  
> It looks that many of you comments are not so much specifica of the
> draft but discussion points for the wg. Repeating them on list with
> some context would be good!
>  
> Best regards
>    Richard
>  
> > From: [email-masked] [_mailto:[email-masked]] On Behalf Of
> > Scheffenegger, Richard
> > As the mechanism update presentations took quite a bit longer than we had expected - and we appreciate the discussion
> these presentations have sparked - we agreed with Fred to have his
> presentation as the first agenda point on Friday; the second agenda
> point will be a presentation around AQM evaluation process and criteria.
> > 
> > 
> > _https://www.ietf.org/mailman/listinfo/aqm_

-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
[email-masked]           |                      -- Mark Twain
(416) 223-8968

---

From: "Akhtar, Shahid (Shahid)" <[email-masked]>
Date: Thu, 7 Nov 2013 20:44:28 +0000

Fred, All,

I am traveling today - so should respond by end of day.

- Shahid.

Sent from my iPhone

---

From: Naeem Khademi <[email-masked]>
Date: Thu, 7 Nov 2013 22:38:21 -0800

Hi

Snipping other stuff, I would like to add some specific points inline:

On Thu, Nov 7, 2013 at 10:45 AM, Fred Baker (fred) <[email-masked]> wrote:

> have different initial conditions drawn from a table given the interface it
> finds itself on, but requires no manual tuning. The great failure of RED,
> recommended in RFC 2309, is not that it doesn't work when properly

and "Adaptive RED" was an attempt to address exactly that issue! but it
seems that it has likely to have remained un-noticed since 2001. The draft
on Sally's homepage still says "August 1, 2001, under submission" while
it's been in Linux for a while :-).

> with each type of AQM here (e.g. 2xBDP etc) or simply state that research
> should be conducted on the best buffer sizes to use with AQM.
>

The terms "good queue", "bad queue" or "briefly" are very subjective. How
"brief" is the key. We can possibly know that when we're talking about a
specific application in a specific environment (e.g. Map/Reduce in data
center). On the access link with a mixture of all sort of traffics, and
also RTTs , that's more controversial. A queue that gives a 100 ms burst a
pass, is a good queue from CoDel designer's point of view (!)

Cheers,
Naeem

---

From: "Akhtar, Shahid (Shahid)" <[email-masked]>
Date: Fri, 8 Nov 2013 13:56:46 +0000

Fred,

I have taken the liberty to tie your comments to the e-mail list of comments I had sent and put a "FB" next to your comment. I have added "SA" next to my comments. This  is easier for people to read.

-Shahid.

____________________________________________________

FB: That's an opinion. One of the objectives of Van and Kathy's work, and separately of Rong Pan et al's work, is to design an algorithm that may have different initial conditions drawn from a table given the interface it finds itself on, but requires no manual tuning. The great failure of RED, recommended in RFC 2309, is not that it doesn't work when properly configured; it's that real humans don't have the time to properly tune it differently for each of the thousands of link endpoints in their networks. There is no point in changing away from RED if that is also true of the replacement.

SA: You argue that "initial conditions" determine some of the parameters of newer AQMs (like Codel and PIE), then those same initial conditions would also determine some of the key parameters for RED/WRED.

Can you explain further why "real humans don't have the time to properly tune it differently for each of the thousands of link endpoints in their networks" with a realistic example.

FB: What you describe is what I referred to as "initial conditions derived from the links the algorithm finds itself on". We may be in violent agreement there. If so, the wording I might suggest would be "SHOULD NOT require operational (especially manual) configuration or tuning apart from automated determination of initial conditions" or some such thing.

SA: Looks like we may be agreeing on something here - which is great. It would be valuable to describe the meaning of "initial conditions" somewhere.

FB: There is also a question of what user is under discussion. If you take a look at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-0.pptx and specifically the third slide, you will see (and tomorrow afternoon I will discuss) a capture I took of pings from my home network to another site overnight. In the evening, we watched a movie (home network), and in the morning I had a video conference (office network). I'll tell you right now that both worked fine, and managing the delay to zero or allowing it to be one hundred ms would not have materially affected the QOE of either. However, my ping is a competing application, and it saw sustained increase in delay, variation in delay, and the possibility of loss as a result of the queuing. The value of AQM is in part to the application being throttled, but in large part to competing applications, and the QOE of both must be considered.

SA: My point about bringing up QoE is to tie the impact of a new investment (such as AQM algorithms) to an operator to revenue. By improving QoE of revenue making services, an operator may determine that it makes business sense to invest. I do not understand how "ping" - is linked a revenue making service? The QoE of Web browsing which is a key service to end-users is closely tied to latency as the PLT of a web page is directly linked to RTT. However the QoE of different services such as web browsing, HAS Video, progessive download video and others should be weighed according to their importance to the end user.

FB: I'm not sure that buffer sizes are specific to AQM algorithms; I'd entertain evidence otherwise. Buffer *thresholds* ("at what point do we start dropping/marking traffic?") may differ between algorithms. Buffer size ("how many bytes/packets do we allow into the queue in the worst case?") is a matter of the characteristics of burst behavior in a given network and the applications it supports. If I have, say, a Map/Reduce application that simultaneously asks thousands of systems a question, the queues in the intervening switches will need to be able to briefly absorb thousands of response packets. The key word here is "briefly". When Van or Kathy talk about "good queue" and "bad queue", they are saying that burst behavior may call for deep queues, but we really want the steady state to achieve 100% utilization with a statistically empty queue if we can possibly achieve that.

SA: In access situations (with AQM) buffer sizes can determine the size of a burst that the buffer can withstand and our research shows that it can directly impact end user QoE.

FB: Not a complete sentence, but I think I understand what you're getting at. You would like to have research determine how to easily configure existing systems using the tools at hand. I'm all for it in the near term.

SA: Glad we agree on this - assume some update will be made around this topic.

-----Original Message-----
From: Fred Baker (fred) [mailto:[email-masked]]
Sent: Thursday, November 07, 2013 12:45 PM
To: Akhtar, Shahid (Shahid)
Cc: Richard Scheffenegger; [email-masked]; Naeem Khademi ([email-masked]); Gorry Fairhurst; Wesley Eddy
Subject: Re: IETF88 Fri 08Nov13 - 12:30 Regency B

> Some tuning may be required or implicitly assumed for virtually all AQMs - please see my comment later.

FB: That's an opinion. One of the objectives of Van and Kathy's work, and separately of Rong Pan et al's work, is to design an algorithm that may have different initial conditions drawn from a table given the interface it finds itself on, but requires no manual tuning. The great failure of RED, recommended in RFC 2309, is not that it doesn't work when properly configured; it's that real humans don't have the time to properly tune it differently for each of the thousands of link endpoints in their networks. There is no point in changing away from RED if that is also true of the replacement.

SA: You argue that "initial conditions" determine some of the parameters of newer AQMs (like Codel and PIE), then those same initial conditions would also determine the key parameters for RED/WRED.

Can you explain further why "real humans don't have the time to properly tune it differently for each of the thousands of link endpoints in their networks" with a realistic example.

>       Do we want to make a suggestion on how to configure buffer sizes with each type of AQM here (e.g. 2xBDP etc) or simply state that research should be conducted on the best buffer sizes to use…

FB: I'm not sure that buffer sizes are specific to AQM algorithms; I'd entertain evidence otherwise. Buffer *thresholds* ("at what point do we start dropping/marking traffic?") may differ between algorithms. Buffer size ("how many bytes/packets do we allow into the queue in the worst case?") is a matter of the characteristics of burst behavior in a given network and the applications it supports. If I have, say, a Map/Reduce application that simultaneously asks thousands of systems a question, the queues in the intervening switches will need to be able to briefly absorb thousands of response packets. The key word here is "briefly". When Van or Kathy talk about "good queue" and "bad queue", they are saying that burst behavior may call for deep queues, but we really want the steady state to achieve 100% utilization with a statistically empty queue if we can possibly achieve that.

SA: In access situations (with AQM) buffer sizes can determine the size of a burst that the buffer can withstand and our research shows that it can directly impact end user QoE.

> May be better to state that tuning should be minimized. For the second sentence "The algorithms that the IETF recommends should minimize tuning or configurations changes for specific traffic or network conditions"
>

FB: What you describe is what I referred to as "initial conditions derived from the links the algorithm finds itself on". We may be in violent agreement there. If so, the wording I might suggest would be "SHOULD NOT require operational (especially manual) configuration or tuning apart from automated determination of initial conditions" or some such thing.

SA: Looks like we may be agreeing on something here - which is great. It would be valuable to describe the meaning of "initial conditions" somewhere.

> Such research should also focus on improving end user QoE from AQMs rather than network related metrics only. Often a significant change in a network metric may only make a minimal change in end-us…

FB: There is also a question of what user is under discussion. If you take a look at http://www.ietf.org/proceedings/88/slides/slides-88-aqm-0.pptx and specifically the third slide, you will see (and tomorrow afternoon I will discuss) a capture I took of pings from my home network to another site overnight. In the evening, we watched a movie (home network), and in the morning I had a video conference (office network). I'll tell you right now that both worked fine, and managing the delay to zero or allowing it to be one hundred ms would not have materially affected the QOE of either. However, my ping is a competing application, and it saw sustained increase in delay, variation in delay, and the possibility of loss as a result of the queuing. The value of AQM is in part to the application being throttled, but in large part to competing applications, and the QOE of both must be considered.

SA: My point about bringing up QoE is to tie the impact of a new investment (such as AQM algorithms) to an operator to revenue. By improving QoE of revenue making services, an operator may determine that it makes business sense to invest. I do not understand how "ping" - is linked a revenue making service? The QoE of Web browsing which is a key service to end-users is closely tied to latency as the PLT of a web page is directly linked to RTT. However the QoE of different services such as web browsing, HAS Video, progessive download video and others should be weighed according to their importance to the end user.

> Methods or configurations that leverage deployed AQMs such as RED/WRED to reduce delays and lockout for typical traffic which require minimal effort or tuning from the operator.

FB: Not a complete sentence, but I think I understand what you're getting at. You would like to have research determine how to easily configure existing systems using the tools at hand. I'm all for it in the near term.

SA: Glad we agree on this - assume some update will be made around this topic.

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 8 Nov 2013 16:27:13 +0000

> SA: You argue that "initial conditions" determine some of the parameters of newer AQMs (like Codel and PIE), then those same initial conditions would also determine some of the key parameters for R…

I'm simply going to point out that Van and Kathy spent quite a bit of time and effort trying to do exactly that, and it didn't pan out. Codel is their suggestion of a replacement that is largely auto-tuning within a specified range of situations.

On your other points, please suggest text, and the WG can discuss whether they buy it.

---

From: "Akhtar, Shahid (Shahid)" <[email-masked]>
Date: Fri, 8 Nov 2013 18:38:07 +0000

Fred, all,

Below is the recommneded text as requested.

-Shahid.

______________________________________________

Manual Tuning

Current text

section 4:

"The algorithms that the IETF recommends SHOULD NOT require operational (especially manual) configuration or tuning."

section 4.3:

"Algorithms recommended for general Internet deployment by the IETF need to be designed so that they do not require operational (especially manual)configuration or tuning."

Recommended text

section 4.3:

"Algorithms recommended for general Internet deployment by the IETF need to be designed so that they minimize operational (especially manual) configuration or tuning. An AQM considered by IETF SHOULD provide rules which determine its parameters based on a set of well-known initial conditions."

section 4:

"The algorithms that the IETF recommends SHOULD minimize operational (especially manual) configuration or tuning."

Recommend the following text in section 4.7

Research should focus on improving end user QoE from AQMs rather than network related metrics only. Often a significant change in a network metric may only make a minimal change in end-user QoE and thus the value of such change may be minimal.

Research should make suggestions on how to make good decisions on buffer sizes with each type of AQM (e.g. 2xBDP) - explaining why or how such buffer sizes improve end-user QoE and network health.

Research should be done on methods or good configurations that leverage deployed AQMs such as RED/WRED that reduce delays and prevent lockout with typical traffic and network conditions.

________________________________

Sent: Friday, November 08, 2013 10:27 AM

On Nov 8, 2013, at 5:56 AM, "Akhtar, Shahid (Shahid)" <[email-masked]<mailto:[email-masked]>> wrote:

---

From: David Ros <[email-masked]>
Date: Fri, 8 Nov 2013 11:23:45 -0800

On 8 nov. 2013, at 10:38, Akhtar, Shahid (Shahid) <[email-masked]> wrote:

[snip]
> Research should focus on improving end user QoE from AQMs rather than network related metrics only. Often a significant change in a network metric may only make a minimal change in end-user QoE and…

Can this be said to be true for the QoE of *any* application?

And, if we focus on QoE instead of network metrics, (a) what apps are we going to consider? (b) do we have easy-to-use, readily-available and accurate models for assessing QoE for all those apps?

(Note, I'm not trying to contradict what you say, I just want to better understand what concrete evaluation guidelines could be given on this)

Thanks,

David

Deadlines really start to press a week or two after they pass. -- John Perry

---

From: "Akhtar, Shahid (Shahid)" <[email-masked]>
Date: Fri, 8 Nov 2013 20:08:17 +0000

David,

Please see a quick answer to your questions below.

-Shahid.

Sent from my iPhone

> (Note, I'm not trying to contradict what you say, I just want to better understand what concrete evaluation guidelines could be given on this)

My text says to focus on QoE as well - meaning both network metrics and service QoE.

Key services on the Internet like web browsing have been around for like 20 years - so major services are slow to change. 

Research does exist on translating application performance to QoE - key new example may be HAS where I have mentioned a couple of papers in my talk at ICCRG. Note that since this is a research topic, we do not need to have all the answers now.