Subject: Re: [iccrg] Fwd: Review of draft-khademi-alternativebackoff-ecn-01.txt

From: Bob Briscoe <[email-masked]>
Date: Fri, 20 Nov 2015 16:56:16 +0000

Grenville,

Sorry, I've just had your reply pointed out to me - I hadn't noticed it 
at the time. I hope it's OK to continue...

I am not arguing that there is not a valid logic behind ABE. I am 
arguing that the way it has been written could be seen as disingenuous, 
or at best over-claiming. The proper flow of logic needed to justify ABE 
is very tricky, and the current text does a very poor job. I think you 
are arguing for the logic you believe is written between the lines. I am 
arguing that the logic as written down misleads, making the significance 
of ABE appear greater than it is. This is probably down to the usual 
problem of a set of unwritten assumptions shared by the authors, but not 
by others.

More inline...

On 13/10/15 01:34, grenville armitage wrote:
> Hi Bob,
>
>>> From: Bob Briscoe <[email-masked]>
>>> Date: 7. oktober 2015 kl. 15.19.12 CEST
>     [..]
>
> I'll re-order things a little to address some common ground first:
>
>>> If the authors want to use a higher beta for ECN flows, I have no 
>>> objection.
>
> Nice to start with a point of agreement ;-)
>
>>> But they shouldn't use this misguided logic to justify it.
>
> Perhaps not surprisingly, I disagree. More precisely, I disagree with 
> your version of our reasoning. I'm still happy with the rationale 
> articulated in 
> https://tools.ietf.org/html/draft-khademi-alternativebackoff-ecn-01 
> (and http://caia.swin.edu.au/reports/150710A/CAIA-TR-150710A.pdf).
Admittedly, I did expand the reasoning to show the flaw in the logical 
flow, because the reasoning in both draft and paper didn't separate out 
the different RTT cases - it only went down the branch of the logic tree 
where ABE gave a benefit.

See also the end of this email for specific problems.

>
> To your questions...
>
>     [..]
>>> Why optimize TCP for a greater RTT than that which AQM designers 
>>> have assumed is most likely?
>
> I don't agree with that characterisation. Rather, ABE recognises that 
> an existing tool (ECN) can let TCP detect, and compensate for, the 
> case where an AQM-managed bottleneck would otherwise lead to poor 
> performance.
>
>>> * I think AQM designers chose these low RTTs because most traffic
>>> these days comes from CDNs, and therefore such low base RTTs are
>>> typical.
>
> AQM designers might well consider low base RTTs to be 'typical'. And 
> TCP connections will 'typically' work well when the base RTTs are a 
> consequence of connections to local CDNs. But there's a bigger world 
> out there. ABE is an incremental change to TCP that improves things 
> when an AQM'd bottleneck might otherwise degrade things -- i.e. when 
> connections are not limited to one's local CDNs.
>
>>> * If the authors believe that typical base RTTs are higher, they
>>> ought to produce some evidence
>
> I'm biased -- my work desktop in Melbourne is a long way from most 
> interesting places around the Internet:
>
> [gja@garmitage] ~% ping -c 10 cisco.com
> PING cisco.com (72.163.4.161): 56 data bytes
>     [..]
> 10 packets transmitted, 10 packets received, 0.0% packet loss
> round-trip min/avg/max/stddev = 206.837/208.044/210.559/1.304 ms
> [gja@garmitage] ~%
> [gja@garmitage] ~% ping -c 10 bbc.co.uk
> PING bbc.co.uk (212.58.244.18): 56 data bytes
>     [..]
> 10 packets transmitted, 10 packets received, 0.0% packet loss
> round-trip min/avg/max/stddev = 299.367/299.576/300.040/0.237 ms
> [gja@garmitage] ~%
>
> I know, Australia's a long way from anyone, and we're probably not 
> typical. But the truth is out there ;)
I'm not saying high RTT doesn't exist.

I'm saying that, if ABE uses a more aggressive beta for all RTTs, there 
is only a benefit, without a counter-balancing deficit somewhere else, when:
a) the RTT of the flow(s) at the bottleneck is/are all greater than that 
assumed typical by the AQM
b) the ABE flow is large enough to reach congestion avoidance phase
c) there is an AQM at the bottleneck that causes low utilisation at high 
RTT (e.g. PIE, Codel and variants as they currently stand).

For more typical RTTs, and for large enough flows, the only perceived 
benefits from ABE are at the expense of other non-ABE flows.

Nonetheless, as I said, I have no objection to anyone using a more 
aggressive TCP. This is because I believe it is the network operator's, 
not the IETF's job to arbitrate rates between users. Then messing with 
relative rates between a users' own flows becomes just a silly game.

However, others might object to such lack of flow `fairness' under 
typical RTT conditions. The only hint to such people that they should be 
worried is a rather hedged sentence right near the end:

    If the method is only deployed by
    some TCP senders, and not by others, the senders that use this method
    can gain advantage, possibly at the expense of other flows that do
    not use this updated method.

So I just want the draft to be honest and lay this all out. At present 
it focuses nearly exclusively on the case where it gives a benefit. It 
gives no indication of how unusual this is, and it avoids clearly 
explaining what happens in more typical cases.

>
>>> before asserting that standard TCP
>>> should be re-written around an assumption of higher RTTs.
>
> I agree with your aversion to rewriting TCP. Fortunately, our Linux 
> and FreeBSD implementations of ABE were trivial tweaks to existing 
> code paths.
I was actually objecting to re-writing TCP /around an assumption of 
higher RTT/.
>
>>> * If they really think the RTT that AQM designers chose was too
>>> small, they should just tell them.  Why change TCP to try to correct
>>> an error of judgement in AQMs that are only just starting to be
>>> deployed? Even those that are already deployed can be
>>> reconfigured.
>
> I'd phrase it this way: ABE lets AQMs do what they've been designed to 
> do in the typical case. By simply relaxing TCP backoff when 
> ECN-enabled flows see marks, ABE also produces the nice result that 
> ECN-enabled TCP connections over long RTT paths wont be degraded as 
> much as they would be otherwise.
Fine. And what happens in the case of typical RTTs...?
>
>>> Anyway, TCP doesn't have to pick just one RTT to optimize against.
>>> Unlike an AQM, TCP knows its own RTT. So why not use a higher value
>>> of beta when TCP measures a higher RTT? Why use a higher value of
>>> beta whatever the RTT, when under-utilzation is only a problem with
>>> high RTT?
>
> Adding RTT sensitivity where it doesn't already exist would seem 
> dangerously close to the sort of re-writing of TCP you eschew above.
I only objected to rewriting TCP around an assumption of higher RTT.

I would prefer the draft to be more honest about what happens at lower 
RTTs, and how infrequently there are likely to be benefits.

>
> TL;DR -- ABE introduces a trivial way to enhance the mutual benefit of 
> ECN + AQM using code paths that already exist.

As promised, below are specific problems with the text of the I-D: 
http://tools.ietf.org/html/draft-khademi-alternativebackoff-ecn-01
No point going into detail about the paper - it exists and it won't change.

NOTE: these are problems with what /is/ said.
These are additions to my main objections in the mail above about what 
is /not/ said.

Abstract

    ECN-marking can allow a
    network device to signal congestion at a point before a transport
    experiences congestion loss or additional queueing delay.

Nope.
As you know, ECN marking has to be no earlier than loss otherwise you 
get starvation of ECN flows when they compete with non-ECN.
And anyway, early ECN marking isn't what the draft is proposing.

    The intention is to achieve good throughput when the queue
    at the bottleneck is smaller than the bandwidth-delay-product of the
    connection. This is more likely when an Active Queue Management
    (AQM) mechanism has used ECN to CE-mark a packet, than when a packet
    was lost.

The second sentence needs to be put in terms of how a transport 
implementation sees it:

    An ECN-capable transport can estimate that this is more likely if its
    RTT is large and it is receiving CE marks, because ECN marking can
    only come from an Active Queue Management (AQM) mechanism, whereas
    loss of ECN packets is more likely to come from a tail drop queue.

You ought to add the further rider:

    This is based on the further assumptions that deployment of ECN-capable
    AQMs is not widespread and that the AQM in question suffers from the
    deficiency seen in current designs of PIE or CoDel that lead to under-
    utilization with large RTT flows.

1. Introduction

    Consequently, a non-ECN enabled standard TCP flow using this reaction
    needs significant network queue space:

Remove "non-ECN enabled".

    AQM mechanisms typically strive to maintain a small queue length,
    regardless of the bandwidth-delay product of flows passing through
    them.  Receipt of an ECN CE-mark might therefore reasonably be taken
    to indicate that a small bottleneck queue exists in the path,

You ought to add the further rider:

    This is based on the assumption that the AQM in question suffers from the
    deficiency seen in current designs of PIE or CoDel that lead to under-
    utilization with large RTT flows.

If you mean that this seems to be a problem that will always be 
intrinsically hard to solve, you need to say that. The above text makes 
it clear that this is only a reasonable inference today, with known 
current AQMs. If AQMs are deployed in future that solve this problem, we 
will have deployed ABE to plug a deficiency that no longer exists, so 
instead we will just have contributed to the TCP arms race.

    resulting in reduced completion times for short flows

This needs to explain that the completion time benefits are for flows 
long enough to have entered congestion avoidance and continued for more 
than one saw-tooth.

Regards

Bob

>
> cheers,
> gja
>
>
> _______________________________________________
> iccrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/iccrg

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 20 Nov 2015 17:03:46 +0000

John,

On 14/10/15 19:35, John Leslie wrote:
>     Yes, the TCP sender knows its own RTT. Yes, the AQM doesn't.
>
>     Yes, a higher value of beta_{ecn} makes sense for some values oF RTT,
> and perhaps nonsense for others. This IMHO should become part of the
> experiment.

Precisely. There is no need to preclude experiments that adapt beta 
based on RTT.
IMO, the choice of using the same beta for all RTTs is not the essence 
of ABE.
It was the  implementation choice of the current authors.

Bob

---

From: Michael Welzl <[email-masked]>
Date: Fri, 20 Nov 2015 23:39:11 +0100

As a co-author, I’ll just jump in and answer in a way that I hope reflects what all co-authors think -

> On 20. nov. 2015, at 17.56, Bob Briscoe <[email-masked]> wrote:
> 
> Grenville,
> 
> Sorry, I've just had your reply pointed out to me - I hadn't noticed it at the time. I hope it's OK to continue...
> 
> I am not arguing that there is not a valid logic behind ABE. I am arguing that the way it has been written could be seen as disingenuous, or at best over-claiming.

Wow.

> The proper flow of logic needed to justify ABE is very tricky, and the current text does a very poor job. I think you are arguing for the logic you believe is written between the lines. I am arguing that the logic as written down misleads, making the significance of ABE appear greater than it is. This is probably down to the usual problem of a set of unwritten assumptions shared by the authors, but not by others.

I do find that surprising.
So please be more specific - give us examples of paragraphs of the draft that exhibit that problem please.   (thanks for providing concrete text suggestions at the end of the email - but here I’m asking about the "tricky proper flow of logic needed to justify ABE”).

>>>> If the authors want to use a higher beta for ECN flows, I have no objection.
>> 
>> Perhaps not surprisingly, I disagree. More precisely, I disagree with your version of our reasoning. I'm still happy with the rationale articulated in https://tools.ietf.org/html/draft-khademi-alternativebackoff-ecn-01 (and http://caia.swin.edu.au/reports/150710A/CAIA-TR-150710A.pdf).
> Admittedly, I did expand the reasoning to show the flaw in the logical flow, because the reasoning in both draft and paper didn't separate out the different RTT cases - it only went down the branch of the logic tree where ABE gave a benefit.
> 
>>>> Why optimize TCP for a greater RTT than that which AQM designers have assumed is most likely?
>> 
>> I don't agree with that characterisation. Rather, ABE recognises that an existing tool (ECN) can let TCP detect, and compensate for, the case where an AQM-managed bottleneck would otherwise lead to poor performance.
>> 
>> AQM designers might well consider low base RTTs to be 'typical'. And TCP connections will 'typically' work well when the base RTTs are a consequence of connections to local CDNs. But there's a bigger world out there. ABE is an incremental change to TCP that improves things when an AQM'd bottleneck might otherwise degrade things -- i.e. when connections are not limited to one's local CDNs.
>> 
>> I'm biased -- my work desktop in Melbourne is a long way from most interesting places around the Internet:
>> 
>> I know, Australia's a long way from anyone, and we're probably not typical. But the truth is out there ;)
> I'm saying that, if ABE uses a more aggressive beta for all RTTs, there is only a benefit, without a counter-balancing deficit somewhere else, when:
> a) the RTT of the flow(s) at the bottleneck is/are all greater than that assumed typical by the AQM
> c) there is an AQM at the bottleneck that causes low utilisation at high RTT (e.g. PIE, Codel and variants as they currently stand).

Isn’t b) pretty clear, albeit implicitly, in that we only propose a change to CA? Would you like every draft proposing a change to CA to remind the reader: “oh btw, this doesn’t apply to SS because, as we say, it’s only about CA” ?

Regarding a) and c), see below:

> For more typical RTTs, and for large enough flows, the only perceived benefits from ABE are at the expense of other non-ABE flows.
> 
> Nonetheless, as I said, I have no objection to anyone using a more aggressive TCP. This is because I believe it is the network operator's, not the IETF's job to arbitrate rates between users. Then messing with relative rates between a users' own flows becomes just a silly game.

Honestly, I think you’re making this sound much more complex and worrisome than it really is. Nobody’s trying to play silly games with fairness, and is this REALLY not laid out clearly in the draft?
From the abstract:
"The intention is to achieve good throughput when the queue at the bottleneck is smaller than the bandwidth-delay-product of the connection.”
That’s what this is about - not more, not less. You’ll find the  “when the bottleneck queue is smaller than the BDP of the connection” part mentioned several times in the draft, so I’m surprised by your concern about the draft not being clear regarding the large-RTT (in fact, large-BDP) case.

It’s really very simple:
With Reno’s factor of 0.5, you require a BDP worth of queuing to achieve 100% utilization. This is often quite much queuing - it means that, when the queue is full, the perceived RTT is doubled. Whenever a congestion indication happens at a value below that, a larger factor is a good idea, we think - and that’s what we assume to happen when you get an ECN mark. And we say so. We may be wrong (just like Reno’s 0.5 may be wrong when the queue is larger than a BDP), but then you get another ECN mark, and the impact of this isn’t too bad evaluated in the accompanying paper.

> However, others might object to such lack of flow `fairness' under typical RTT conditions. The only hint to such people that they should be worried is a rather hedged sentence right near the end:
> 
> So I just want the draft to be honest and lay this all out. At present it focuses nearly exclusively on the case where it gives a benefit. It gives no indication of how unusual this is, and it avoids clearly explaining what happens in more typical cases.

I thought the statement you quote above is honest enough. Do you want us to repeat it a few times?  :-)     note that this is a 0.8-factor decrease, as has been used by Cubic for a while - until kernel version 2.6.25 released in 2008. We also say that. Cubic used it in response to loss as well as ECN, which I would argue can be a much more aggressive behavior than *only* doing this in response to ECN.

>> I agree with your aversion to rewriting TCP. Fortunately, our Linux and FreeBSD implementations of ABE were trivial tweaks to existing code paths.
> I was actually objecting to re-writing TCP /around an assumption of higher RTT/.

The assumption is that the queue at the bottleneck is smaller than the connection’s BDP. For the fun of it, I’ll quote the same sentence from the abstract, yet again:
***
The intention is to achieve good throughput when the queue
   at the bottleneck is smaller than the bandwidth-delay-product of the
   connection.
***

>> I'd phrase it this way: ABE lets AQMs do what they've been designed to do in the typical case. By simply relaxing TCP backoff when ECN-enabled flows see marks, ABE also produces the nice result that ECN-enabled TCP connections over long RTT paths wont be degraded as much as they would be otherwise.
> Fine. And what happens in the case of typical RTTs…?

See above, and see the draft: the same as what Cubic did until 2008, but in fact less because it’s only in response to ECN.

>> Adding RTT sensitivity where it doesn't already exist would seem dangerously close to the sort of re-writing of TCP you eschew above.
> I would prefer the draft to be more honest about what happens at lower RTTs, and how infrequently there are likely to be benefits.

Discussed above…

>> TL;DR -- ABE introduces a trivial way to enhance the mutual benefit of ECN + AQM using code paths that already exist.
> 
> As promised, below are specific problems with the text of the I-D: http://tools.ietf.org/html/draft-khademi-alternativebackoff-ecn-01
> These are additions to my main objections in the mail above about what is /not/ said.

Thank you very much for these. They will have to be discussed with my co-authors - so, for now, just one question and one comment below:

> As you know, ECN marking has to be no earlier than loss otherwise you get starvation of ECN flows when they compete with non-ECN.
> The second sentence needs to be put in terms of how a transport implementation sees it:
> 
>   AQMs is not widespread and

Why?

> that the AQM in question suffers from the

You call it a deficiency and I wouldn’t do that, it’s subjective. It could be a design decision - a trade-off that the authors of PIE od CoDel were willing to make.
With Reno (as opposed to ABE), you’re caught in having to make this trade-off. With ABE, you can have PIE or CoDel running like they do now and still get good utilization in the long-RTT case while still getting the low delay that they’re already either configured to provide or do provide as the result of a design deficiency, as you call it.

> If you mean that this seems to be a problem that will always be intrinsically hard to solve, you need to say that. The above text makes it clear that this is only a reasonable inference today, with known current AQMs. If AQMs are deployed in future that solve this problem, we will have deployed ABE to plug a deficiency that no longer exists, so instead we will just have contributed to the TCP arms race.
> 
> This needs to explain that the completion time benefits are for flows long enough to have entered congestion avoidance and continued for more than one saw-tooth.
> 
> Regards
> 
> 
> 
> Bob

Cheers,
Michael

---

From: Michael Welzl <[email-masked]>
Date: Fri, 20 Nov 2015 23:40:51 +0100

> On 20. nov. 2015, at 18.03, Bob Briscoe <[email-masked]> wrote:
> 
> John,
> 
> Precisely. There is no need to preclude experiments that adapt beta based on RTT.

How does a SHOULD in an Experimental RFC (which is what we’d have as the result of what we’re currently proposing) preclude such experiments?

Cheers,
Michael

> IMO, the choice of using the same beta for all RTTs is not the essence of ABE.

---

From: Bob Briscoe <[email-masked]>
Date: Sat, 21 Nov 2015 01:16:12 +0000

Michael,

> How does a SHOULD in an Experimental RFC (which is what we’d have as the result of what we’re currently proposing) preclude such experiments?
The thread was in the context of my previous email with Grenville.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Sat, 21 Nov 2015 04:18:34 +0000

Michael,

> So please be more specific - give us examples of paragraphs of the draft that exhibit that problem please.   (thanks for providing concrete text suggestions at the end of the email - but here I’m a…
As I've said. The problem is primarily omission. The omissions are from 
the places where the rationale and justification is - the introduction, 
the abstract, etc. There are also the cases I have pointed out now where 
the text hedges when describing changes to capacity sharing.
> Isn’t b) pretty clear, albeit implicitly, in that we only propose a change to CA? Would you like every draft proposing a change to CA to remind the reader: “oh btw, this doesn’t apply to SS because…
Every draft that proposes changes to CA ought to quantify how unusual it 
is for flows to reach CA (the research on IW10 and TFO are examples of 
good practice). Then more people would understand the relative 
importance of SS vs CA, and we could spend more time on stuff that is 
more important.

To get significant completion time benefits for short flows (as claimed 
in the ABE draft), a flow has to be on the long end of short. The large 
majority of flows don't reach CA, let alone the period after the first 
downward sawtooth after that. Maybe less than 1% of flows continue 
beyond the first sawtooth (educated guess from my recent work on flow 
size distributions over the last decade). But don't take my guess - I'm 
saying this work is missing from the ABE research.
The ABE paper claims "a substantial increase in the length of short 
flows" and cites an httparchive URL. I don't think you can conclude that 
from the cited data, because it is about the size of object requests, 
which is different from TCP flows (altho it does show #flows has been 
fairly constant recently, but those stats are only available for the 
last 2yrs).

Whatever, whether a flow reaches CA depends on the growth of available 
capacity per flow relative to the growth in flow sizes. Access link 
capacity has been doubling roughly every 1.6 years and the httparchive 
data shows that total Web transfer size has increased roughly linearly 
in the last 5 years by roughly x3, with little change in number of flows 
per transfer (data on #flows is only for the last 2yrs). So, bearing in 
mind we're missing some data (e.g. the trend in simultaneous sessions 
from multiple machines or multiple apps) and we're relying on data that 
might not be relevant, it /seems/ that Web flows are trending to become 
less likely to reach CA, not more.

> That’s what this is about - not more, not less. You’ll find the  “when the bottleneck queue is smaller than the BDP of the connection” part mentioned several times in the draft, so I’m surprised by…
See previous emails. How prevalent is a large RTT?
And to be clear, this means how prevalent is a large /base/ RTT? If 
measurements might include bloated buffer delay, they will not be 
relevant for ABE, because CE marks prove there is an AQM at the bottleneck.

>
> It’s really very simple:
> With Reno’s factor of 0.5, you require a BDP worth of queuing to achieve 100% utilization. This is often quite much queuing - it means that, when the queue is full, the perceived RTT is doubled. Whenever a congestion indication happens at a value below that, a larger factor is a good idea, we think
No, the ID /always/ uses a larger factor. Not just "Whenever a 
congestion indication happens at a value below that".
> - and that’s what we assume to happen when you get an ECN mark. And we say so. We may be wrong (just like Reno’s 0.5 may be wrong when the queue is larger than a BDP), but then you get another ECN mark, and the impact of this isn’t too bad evaluated in the accompanying paper.
Neither the paper nor the I-D  say how often this "not too bad" 
situation is likely to occur, relative to the good outcome.
And the ID doesn't mention how "not too bad" this case is.
> I thought the statement you quote above is honest enough. Do you want us to repeat it a few times?  :-)     note that this is a 0.8-factor decrease, as has been used by Cubic for a while - until ke…
See previous mail about Cubic falling back to Reno at typical BDPs seen 
in today's access networks.
>     at the bottleneck is smaller than the bandwidth-delay-product of the
And perhaps I should ask again: how prevalent is that likely to be?

We all know how buffer-bloat was bigged-up as a significant problem. 
Most people working on AQM had worked out that this problem would be 
intermittent, but believed that the unpredictability of large but 
infrequent delay was still important to remove.

However, to outside observers, not quantifying the low prevalence of the 
problem made the AQM community look like drama queens when someone else 
did the quantification for us (e.g. "A QoE Perspective on Sizing Network 
Buffers" in SIGCOMM'14).

Have you ever criticised someone for overselling the prevalence of the 
buffer-bloat problem?
Now, imagine that person wrote the ABE I-D, and you are reviewing it.

>>    AQMs is not widespread and
Sorry. I meant to take that sentence out of the email - I took it out 
from one place, but forgot it was here as well.
> With Reno (as opposed to ABE), you’re caught in having to make this trade-off. With ABE, you can have PIE or CoDel running like they do now and still get good utilization in the long-RTT case while…
The deficiency I mean is that the AQM doesn't adapt to RTT. I don't mean 
AQMs ought to adapt to RTT. I just mean they don't.

Cheers

Bob