Subject: [iccrg] Sub-packet congestion window

From: Bob Briscoe <[email-masked]>
Date: Tue, 19 Nov 2019 10:05:38 +0800

iccrg,

You may recall Neal outlined the problem of a sub-packet congestion 
window in his talk on BBRv2 yesterday.

      ==Links==

As promised in the iccrg session yesterday, here's the recent work on 
this he mentioned was expected soon:

  * Ahmed, Asad, S., "Extending TCP for Low Round Trip Delay
    <http://bobbriscoe.net/projects/latency/thesisAsadAhmed-submitted20190815.pdf>"
    Master Thesis Thesis submitted to: University of Oslo (August 2019)
  * Modification to Linux TCP stack
    <https://bitbucket.org/asadsa/kernel420/src/submss/> to work with a
    sub-packet window (research prototype)

I've also linked to Asad's Master's thesis and code off the L4S landing 
page:

  * L4S: Ultra-Low Queuing Delay for All
      o papers <https://riteproject.eu/dctth/#papers>
      o code <https://riteproject.eu/dctth/#code>

A more permanent archival URL for the thesis should appear very soon 
(the Uni's library takes 90 days to make a thesis available, which is 
nearly up).

      ==More Info==

If time, we'll write a paper on it - a thesis isn't the most accessible 
medium. We'll prepare a talk on this too - I've requested a slot in the 
next iccrg, but no promises 'cos Jana wants to make it a BBR special 
session.

Asad is in cc if you have questions, or he's here in Singapore (except 
he was taken ill yesterday, but I hope he'll have recovered soon).

      ==Summary==

It's not specific to TCP Prague: Asad modified the base TCP stack and 
tested it with Reno and DCTCP. It uses a modified AIMD called LS-AIMD 
(logarithmically scaled AIMD), so more work would be needed to modify 
congestion control modules that are not pure AIMD, e.g. Cubic

The patch was fairly small in the end, although the design process we 
went through was tortuous, given we were breaking some fairly basic 
assumptions in TCP's design:

  * First, of course, Asad had to represent sub-packet cwnd. To minimize
    impact on the code, he used the usual Linux variable in segment
    units complemented by an integer part in bytes.
  * When sending less than 1 packet per RTT, one has to pace, of course;
  * Then the additive increase becomes larger than the multiplicative
    decrease - solved by logarithmically scaling the additive increase
    dependent on ssthresh, rather than a constant of 1 segment per RTT.
  * Then we had to fix Proportional Rate Reduction
  * Then we found bursts echoed for multiple round trips, which we
    solved by re-basing the pacing timer whenever we received an ACK,
    instead of always basing send times off each other.
  * Then we had all the implementation problems you get when you don't
    have a frequent enough ACK event to trigger the code to do stuff. So
    we had to adjust certain per-ACK actions mathematically so the
    outcome was as if they had been repeated per RTT (e.g. maintenance
    of DCTCP's EWMA).
  * Also had to produce patches for bugs in the original code on the way

It all worked nicely in the end. Lots more testing to do tho (see 
thesis) - ran out of time, as is traditional .

As I said in iccrg, there are other ways to approach this, but with this 
considerable experience under out belts, we are better placed to judge 
the pros and cons.

Cheers

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/