Subject: Re: [iccrg] Review of draft-khademi-alternativebackoff-ecn-01.txt

From: Dave Taht <[email-masked]>
Date: Fri, 20 Nov 2015 09:10:50 -0800

I guess I am confused. Near as I remember, the aqm-ers were mostly
agreed that AQMs should exert CE sooner than drop - which none do (or
only barely), and this draft is proposing that tcp's using CE be
responded to by reducing the rate by .8, rather than half.

what could go wrong?

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 20 Nov 2015 17:34:24 +0000

Dave,

On 20/11/15 17:10, Dave Taht wrote:
> only barely),
Care!
If 'sooner' means 'with less smoothing delay', agreed.
If 'sooner' means 'at a shallower queue depth', no - numerous 
experiments show ECN flows would starve.

Anyway ABE is a sender-only change. It doesn't depend on AQMs like PIE 
and CoDel treating ECN any different to non-ECN.

> and this draft is proposing that tcp's using CE be
A lot of people don't like allowing one TCP to be more aggressive than 
existing TCPs.
I don't mind.
But I am saying this draft hedges that point, and I prefer honesty.

I'm also objecting to over-claiming benefits and not being clear on how 
unusual the circumstances are when benefits occur.

Bob
>
> _______________________________________________
> iccrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/iccrg

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Michael Welzl <[email-masked]>
Date: Sat, 21 Nov 2015 00:02:01 +0100

> On 20. nov. 2015, at 18.34, Bob Briscoe <[email-masked]> wrote:
> 
> Dave,
> 
> If 'sooner' means 'at a shallower queue depth', no - numerous experiments show ECN flows would starve.
> 
> Anyway ABE is a sender-only change. It doesn't depend on AQMs like PIE and CoDel treating ECN any different to non-ECN.
> 
> A lot of people don't like allowing one TCP to be more aggressive than existing TCPs.
> I don't mind.
> I'm also objecting to over-claiming benefits and not being clear on how unusual the circumstances are when benefits occur.

Frankly, I am getting annoyed by this repeated accusation of dishonesty, over-claiming benefits and not being clear on how unusual the circumstances are when benefits occur. You’re basically saying that our draft is wrongly informing the reader about benefits and limitations, and you question our honesty. We can stress things more here and there and make adjustments, but I think you’re going too far here, given how our short draft currently addresses these matters:

1) regarding being more aggressive than existing TCPs, the second paragraph of the security considerations section:

***
   This document describes a change to TCP congestion control that can
   make TCP senders more aggressive than flows using TCP as specified in RFC 3819.
   This could lead to a change in the capacity achieved by
   flows sharing a network bottleneck.  If some flows use this method
   and share capacity with other flows using previous methods this could
   reduce fairness in the capacity allocation.  Similar unfairness is
   also exhibited by other congestion control mechanisms that have been
   in use in the Internet for many years (e.g., CUBIC [ID.CUBIC]).
   Unfairness may also be a result of other factors, including the round
   trip time experienced by a flow.  This advantage applies only to ECN-
   marked packets and not to loss indications, and will therefore can
   not lead to congestion collapse.
***

You may dislike that this appears only once (it’s a short draft) or so late, and we can change that, but questioning honesty is taking this rather far.

2) regarding over-claiming benefits, there is really only one sentence which describes the benefits:

***
Results reported in [ABE2015] show significant benefits (improved
   throughput, resulting in reduced completion times for short flows)
   when reacting to ECN-Echo by multiplying cwnd and sstthresh with a
   value in the range [0.7..0.85].
***
and by the time the reader reaches it, (s)he should already be quite clear about the fact that this is only meant for the case when the ECN-mark was given from a queue that was below the connection’s BDP. So, given the context of where this appears in the text I don’t think it’s fair to call it “overstating”. But I don’t see a problem in adding this caveat to this sentence about results and probably my co-authors won’t, either.

3) regarding “how unusual the circumstances are when benefits occur”:

the draft says, in the abstract:
***
The intention is to achieve good throughput when the queue
at the bottleneck is smaller than the bandwidth-delay-product of the
connection.
***

This describes the circumstances, and as I said in the other email, it appears multiple times in the draft.

Cheers,
Michael

---

From: Bob Briscoe <[email-masked]>
Date: Sat, 21 Nov 2015 01:12:33 +0000

Michael,

Saying I want it to be more honest in no way was meant to imply that any 
of the authors are deliberately being dishonest. As I thought I made 
clear, you are probably not aware that your assumptions are not shared. 
I apologise unreservedly if anyone got the impression I am accusing any 
of the authors of dishonesty. Asking for more honesty means more 
objectivity. Definitely not deliberate deception.

Similarly over-claiming is rarely deliberate deception. It is more often 
self-deception. It is a skill to be able to state the importance of 
one's own work objectively.

More...

> You may dislike that this appears only once (it’s a short draft) or so late, and we can change that, but questioning honesty is taking this rather far.
It doesn't need to appear more than once. But here's what I call honest.

With typical RTTs, ABE /will/ make TCP senders more aggressive. Not /can/.

This /will typically/ lead to a change in the capacity achieved... Not 
/could/.

Similar 'unfairness' is rarely exhibited by Cubic, because Cubic falls 
back to Reno mode, at least for any BDP seen today over a realistic 
public access network (which typically restricts the path at one end at 
least). For instance, with a typical RTT of 20ms, TCP Cubic remains in 
Reno mode up to about 500Mb/s.

I feel dirty having to make these arguments, given for so much of my 
career I have argued about why flow rate is not a measure of fairness 
(without taking account of behaviour over time, and behaviour of users 
and applications, not flows). However, whether or not relative flow rate 
is a useful metric, if you're talking about relative flow rates, you 
need to say it objectively.

> and by the time the reader reaches it, (s)he should already be quite clear about the fact that this is only meant for the case when the ECN-mark was given from a queue that was below the connection…
For instance, compare ABE publications with the work on IW10 or TFO. 
These quantified how prevalent the circumstances were under which the 
benefits occurred. If they hadn't you would have expected them to. 
That's why it shouldn't be such a surprise that I expect the ABE 
research to have done at least a back-of-the-envelope estimate of the 
prevalence of the benefits.

How many readers know how likely it is that the connection's BDP will be 
greater than the queue? Does the paper or the draft cite research on the 
distribution of RTTs in observed traffic (before my very first criticism 
of ABE on this score, I searched for papers on that - unsurprisingly the 
data is out there already - sorry I didn't record them, but a good 
search engine is all you need).

"Is the benefit worth the effort?" is always the most important question 
before embarking on any work, including when someone is trying to decide 
whether to implement ABE, whether to spend time reading ABE papers, 
whether to consume the WG's time, the IESG's time and RFC Editor's time 
standardising ABE.

John Leslie said it is enough that there will be cases when this 
applies. I'm sure he didn't mean it is as simple as that:
* If a larger BDP is fairly prevalent, it says the higher beta might as 
well always apply.
* Otherwise it strengthens the case for linking beta to RTT somehow.

> This describes the circumstances, and as I said in the other email, it appears multiple times in the draft.
Again, I'm not saying anything is dishonest. I'm saying describing the 
circumstances is strictly truthful, but it's not "truth in 
advertising",  i.e. it doesn't quantify how unusual the circumstances are.

HTH

Bob

---

From: Dave Taht <[email-masked]>
Date: Sat, 21 Nov 2015 13:35:30 +0100

> If 'sooner' means 'with less smoothing delay', agreed.

OK.

> show ECN flows would starve.

OK.

>
> Anyway ABE is a sender-only change. It doesn't depend on AQMs like PIE and
> CoDel treating ECN any different to non-ECN.
>

Being the FQ advocate 'round heah, I don't mind in my own contexts,
either! Go ahead Blast away! (add pacing). The only flow you will hurt
is your own.

But: when trying to wear a single queue (or less queues) hat, well,
see "drop and mark" below. Haven't bothered to implement it for single
queued aqms, lacking a tcp to try it on. (yet).

As presently implemented it seems to "do no harm" with a variety of tcps.

> unusual the circumstances are when benefits occur.

OK. I regard modern TCPs as too aggressive in the first place but more
of my objection is to the exorbitant size of the IW (esp when multiple
flows are being used, as in web traffic, and with TSO/GRO/GSO network
offloads), which is somewhat mediated by starting to see pacing
deploy, and having cake do "TSO peeling". I'd grumble about it more if
QUIC didn't have a even bigger IW, effectively, than IW10. (10 IW, 22
packets paced, last I looked)

Anyway, in context with both this draft and the aqm discussion, cake
implements (and fq-codel may one day implement) drop a packet then
mark the next, on extreme overload. No TCP (I know of. (?) yet(?))
responds to these two signals in conjunction (within a RTT), and I
increasingly think they should (backing off by even more than half
when in loss recovery and CE is seen).

so... I would like to see this sort of proposed more aggressive
behavior in this draft be coupled with thorough analysis of pacing,
IWs, the new rack stuff, drop and mark, et al.

Did your sub-packet window idea get any traction?

---

From: Michael Welzl <[email-masked]>
Date: Sun, 22 Nov 2015 00:34:27 +0100

Hi Bob,

I’ll try to answer in more detail soon - but here’s a general note:

I think I now finally understand that your major concern - the fact that we don’t talk much about how aggressive ABE is when the BDP is small - is due to a misunderstanding.

Our draft doesn’t go into a long discussion of the impact that ABE flows with a small BDP could have on other flows.
I now understand that you think that ABE’s aggressive behavior with small BDPs is something unheard of (and would therefore have to be highlighted): flows that, in the presence of ECN feedback, would *always* multiply with 0.8 - while we only justify this by saying “CUBIC did the same”!   (“did” for 0.8, “does” for 0.7)

However, this:

> Similar 'unfairness' is rarely exhibited by Cubic, because Cubic falls back to Reno mode, at least for any BDP seen today over a realistic public access network (which typically restricts the path at one end at least). For instance, with a typical RTT of 20ms, TCP Cubic remains in Reno mode up to about 500Mb/s.

is wrong. Unless I totally misunderstood the code, graphs I've seen are wrong, and/or my memory plays tricks on me …. when it sees a loss or ECN mark, CUBIC *always* multiplies with its beta, which was 0.8 for a while and later became 0.7.

For the CUBIC=0.8 case (and 0.7 can be debated if we really must), ABE is therefore strictly only less aggressive than CUBIC (which has already been evaluated to death):
- it increases exactly like Reno
- it decreases like CUBIC, but only in response to an ECN mark, not loss

Cheers,
Michael

---

From: Michael Welzl <[email-masked]>
Date: Sun, 22 Nov 2015 00:39:16 +0100

> On 21. nov. 2015, at 13.35, Dave Taht <[email-masked]> wrote:
> 

FWIW,
- all real-life tests (which means: almost all tests we did) involving CUBIC were on Linux, which exhibits some form of pacing (I’m not an expert on it - AFAIK it’s not “quite” normal pacing)
- IW doesn’t play much of a role in CA, which is what this draft is proposing to change
- the “drop and mark” scheme that you describe contradicts a basic and very old notion of congestion control, which is that TCP should only react to a single congestion event per RTT. I.e., two, three, four consecutive packet losses within the same RTT should provoke the same TCP reaction. Same for “loss + ECN”. Fine by me if you challenge that old logic, but anyway this is really orthogonal to the ABE draft.

Cheers,
Michael