Subject: [tsvwg] transport protocol feedback overhead

From: Bob Briscoe <[email-masked]>
Date: Thu, 30 Mar 2017 21:17:45 +0100

Ingemar, list,

I seemed to get blocked from talking after I was trying to be polite - 
letting you finish your talk...

You described one problem:
  (A) In mobile (and other networks)  upstream ACK load is a problem.

There is a complementary problem too:
(B) Capacity variation is getting huge as radio wavelength increases. So 
it's useful for the sender to watch the timing of the ACKs to be able to 
track changes in capacity as fast as possible. It's also useful to be 
able to vary the spacing between the packets you send to probe for 
increases in capacity by watching the ACK spacing in response.

These two are in tension - (A) wants less ACKs, (B) wants them all.

A possible solution, is a data structure in a stretch ACK to be able to 
report the time of arrival of all the previous data packets (ie. the 
packets that the stretch ACK is ACKing.

Even tho this info will arrive at the sender later than it would have 
done if the ACKs had not been suppressed, as long as you are still 
getting a reasonable number of ACKs every RTT, it resolves the tension 
between A and B.

Thoughts?
Does this idea already exist?

Bob

PS. Apologies for dropping an innovative idea into a standards meeting. 
This probably ought to be a research paper first.

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Michael Welzl <[email-masked]>
Date: Thu, 30 Mar 2017 16:14:49 -0500

Hi,

Thanks Ingemar for your interesting presentation today:
https://www.ietf.org/proceedings/98/slides/slides-98-tsvwg-sessb-7-transport-protocol-feedback-overhead-issues-and-solutions-00.pdf <https://www.ietf.org/proceedings/98/slides/slides-98-tsvwg-sessb-7-transport-protocol-feedback-overhead-issues-and-solutions-00.pdf>

I wanted to share with you, and the list, that David Ros  (one of the original authors of ACK-CC, RFC 5690) and I once advised a master student to:
- implement ACK-CC
- test it
- analyze all RFC 2119 - style keywords in RFC 5690, to see how they should be changed (if at all) in a possible update

Back then, our thinking was that we could write a paper about it, and try to get this deployed as a way of doing the Experiment and moving this to PS status.
However, we were less and less sure about the real value of this - whether actual networks need this? Then, we had less and less time available and at some point dropped the ball.
I of course found it interesting to hear that it *is* a real problem, even today.

Anyway: this master thesis is available here:  http://heim.ifi.uio.no/michawe/teaching/dipls/marius-olsen/mastersthesis-mariusno.pdf <http://heim.ifi.uio.no/michawe/teaching/dipls/marius-olsen/mastersthesis-mariusno.pdf>
and I think I can safely say on behalf of David, Marius (the master student who did this) and myself that we’re happy if someone picks this up and continues the work.
It would require some digging, but I might also still have the code lying around somewhere - for some outdated version of Linux…

Cheers,
Michael

---

From: Yuchung Cheng <[email-masked]>
Date: Thu, 30 Mar 2017 15:16:37 -0700

regarding the question in the slide 5.
"Is immediate ACK needed for out of order segments if RACK is used ?"

The answer is both yes and no: a timely feedback is appreciated (yes). But
RACK does not need 100 DUPACKs each telling one more packet is delivered
out of order (no). In other words if the ACK is timely and precise about
the delivery process, RACK would work well (so does BBR). However this can
go terribly wrong if the middle-box sends only the last ACK/SACK. It has
happened in one major cloud platform.

On Thu, Mar 30, 2017 at 2:14 PM, Michael Welzl <[email-masked]> wrote:

> https://www.ietf.org/proceedings/98/slides/slides-98-tsvwg-s
> essb-7-transport-protocol-feedback-overhead-issues-and-solutions-00.pdf
>
> I wanted to share with you, and the list, that David Ros  (one of the
> original authors of ACK-CC, RFC 5690) and I once advised a master student
> to:
> - implement ACK-CC
> - test it
> - analyze all RFC 2119 - style keywords in RFC 5690, to see how they
> should be changed (if at all) in a possible update
>
> Back then, our thinking was that we could write a paper about it, and try
> to get this deployed as a way of doing the Experiment and moving this to PS
> status.
> However, we were less and less sure about the real value of this - whether
> actual networks need this? Then, we had less and less time available and at
> some point dropped the ball.
> Anyway: this master thesis is available here:
> http://heim.ifi.uio.no/michawe/teaching/dipls/marius-olsen/
> mastersthesis-mariusno.pdf
> and I think I can safely say on behalf of David, Marius (the master
> student who did this) and myself that we’re happy if someone picks this up
> and continues the work.
> It would require some digging, but I might also still have the code lying
> around somewhere - for some outdated version of Linux…
>
> Cheers,
> Michael
>
>

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Thu, 30 Mar 2017 22:45:14 +0000

I agree that ACKs giving timing details of the last packet arrivals would be very useful. It will also avoid delay noise in the ACK path.

Koen.

> -----Original Message-----
> From: tsvwg [mailto:[email-masked]] On Behalf Of Bob Briscoe
> Sent: donderdag 30 maart 2017 22:18
> To: Ingemar Johansson S <[email-masked]>; tsvwg IETF
> list <[email-masked]>
> Subject: [tsvwg] transport protocol feedback overhead
> 
> Ingemar, list,
>

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 31 Mar 2017 16:40:18 +0000

Hi Michael and others.
Thanks for the very good feedback from all of you.

It is a bit of a challenge to determine how one should pursue this matter, is BCP the best way to go or are there too many differences  between transport protocols ?.
There is a certain risk that we open up a can of worms ?, the latter is of course a certain indication that middlebox meddling with ACKs should be avoided.

Thanks for the reference to the master thesis, I have plenty of time to read it on my way back.

/Ingemar

From: Michael Welzl [mailto:[email-masked]]
Sent: den 30 mars 2017 23:15
To: Ingemar Johansson S <[email-masked]>
Cc: tsvwg IETF list <[email-masked]>; David Ros <[email-masked]>
Subject: TCP ACK-CC (re: Transport protocol feedback overhead )

Hi,

Thanks Ingemar for your interesting presentation today:
https://www.ietf.org/proceedings/98/slides/slides-98-tsvwg-sessb-7-transport-protocol-feedback-overhead-issues-and-solutions-00.pdf

Anyway: this master thesis is available here:  http://heim.ifi.uio.no/michawe/teaching/dipls/marius-olsen/mastersthesis-mariusno.pdf
and I think I can safely say on behalf of David, Marius (the master student who did this) and myself that we’re happy if someone picks this up and continues the work.
It would require some digging, but I might also still have the code lying around somewhere - for some outdated version of Linux…

Cheers,
Michael

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 31 Mar 2017 17:35:12 +0000

Thanks for the details. It may be the case that it can be tricky to get a decent understanding of what is possible in terms of ACK rate reduction for TCP. The middlebox behavior to discard everything except the last ACK has also been proposed in the RAN2 WG in 3GPP.

/Ingemar

From: Yuchung Cheng [mailto:[email-masked]]
Sent: den 31 mars 2017 00:17
To: Michael Welzl <[email-masked]>
Cc: Ingemar Johansson S <[email-masked]>; tsvwg IETF list <[email-masked]>; [email-masked] Extensions <[email-masked]>
Subject: Re: [tsvwg] TCP ACK-CC (re: Transport protocol feedback overhead )

The answer is both yes and no: a timely feedback is appreciated (yes). But RACK does not need 100 DUPACKs each telling one more packet is delivered out of order (no). In other words if the ACK is timely and precise about the delivery process, RACK would work well (so does BBR). However this can go terribly wrong if the middle-box sends only the last ACK/SACK. It has happened in one major cloud platform.

On Thu, Mar 30, 2017 at 2:14 PM, Michael Welzl <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

Cheers,
Michael

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 31 Mar 2017 17:42:54 +0000

Hi

Please see inline..

/Ingemar

> From: Bob Briscoe [mailto:[email-masked]]
> Sent: den 30 mars 2017 22:18
> Subject: transport protocol feedback overhead
> 
> Ingemar, list,
> 
> I seemed to get blocked from talking after I was trying to be polite - letting
> you finish your talk...
[IJ] It never pays off to be polite :-)

>   (A) In mobile (and other networks)  upstream ACK load is a problem.
[IJ] Yes, it can be problematic for instance in TDD access, due to the asymmetry, there are of course different degrees in hell. The main point I wanted to convey (not sure how lucky I was) was to encourage developers to send as much ACKs as a needed, nothing more. 

> (B) Capacity variation is getting huge as radio wavelength increases. So it's
> useful for the sender to watch the timing of the ACKs to be able to track
> changes in capacity as fast as possible. It's also useful to be able to vary the
> spacing between the packets you send to probe for increases in capacity by
> watching the ACK spacing in response.
> 
> A possible solution, is a data structure in a stretch ACK to be able to report
> the time of arrival of all the previous data packets (ie. the packets that the
> stretch ACK is ACKing.
[IJ] Like what is implemented in QUIC, this will allow to get detailed timing of each received packet if wanted. Not sure how feasible it is with TCP.

> 
> Even tho this info will arrive at the sender later than it would have done if the
> ACKs had not been suppressed, as long as you are still getting a reasonable
> number of ACKs every RTT, it resolves the tension between A and B.
> 
> Thoughts?