Subject: Re: [EToSat] TR: New Version Notification for draft-kuhn-quic-4-sat-04.txt

From: Christian Huitema <[email-masked]>
Date: Wed, 22 Apr 2020 20:04:47 -0700

Something related: I have been working for some time on how to
accelerate the "startup" phase of a QUIC connection over satellite. The
main changes in Picoquic are:

1) If the measured RTT is large, scale up the initial Window. We
discussed this in Singapore.

2) Tweak the pacing algorithm so Picoquic can send data bursts
(moderately sized)

3) Implement a peak bandwidth measurement by looking at acknowledgements
over time, and noticing peaks when data bursts are acknowledged

4) During start-up, increase the congestion window if it is smaller than
1/2 peak-bandwidth times RTT.

With that, the test of sending 100MB on a 250/3 Mbps link completes in
5.2 second, of which 1.2 seconds are the connection set up RTT and one
RTT to request and ack the file. That means the actual transfer lasts 4
seconds, average rate of 200 Mbps, 80% of the capacity.

Details on how the experiment progressed at
https://huitema.wordpress.com/2020/04/21/faster-slow-start-for-satellite-links/.

-- Christian Huitema

On 3/6/2020 7:53 AM, Kuhn Nicolas wrote:
> Hi,
>
> We have just pushed an updated version of the "QUIC 4 SAT" document. 
> The main difference with the previous version is that we have proposed baseline values for the regression test for the performance of QUIC on SATCOM systems. 
>
> We plan a side-meeting during IETF107 where we will present the rationale behind these numbers and recent experiments on QUIC over SATCOM systems.
> 	Monday 23rd 
> 	8:30 am 
> 	Room: Regency E
>
> If you are interested by remote participation, please join the ETOSAT mailing list where we will share conf call details. 
> In the meantime, do not hesitate if you have comments on this version of the document. 
>
> Cheers, 
>
> Nico - on the behalf of the authors 
>
> -----Message d'origine-----
> De : [email-masked] <[email-masked]> 
> Envoyé : vendredi 6 mars 2020 16:43
> À : Gorry Fairhurst <[email-masked]>; Kuhn Nicolas <[email-masked]>; John Border <[email-masked]>; Godred Fairhurst <[email-masked]>; Stephan Emile <[email-masked]>; Emile Stephan <[email-masked]>
> Objet : New Version Notification for draft-kuhn-quic-4-sat-04.txt
>
>
> A new version of I-D, draft-kuhn-quic-4-sat-04.txt has been successfully submitted by Nicolas Kuhn and posted to the IETF repository.
>
> Name:		draft-kuhn-quic-4-sat
> Revision:	04
> Title:		QUIC for SATCOM
> Document date:	2020-03-06
> Group:		Individual Submission
> Pages:		13
> URL:            https://www.ietf.org/internet-drafts/draft-kuhn-quic-4-sat-04.txt
> Status:         https://datatracker.ietf.org/doc/draft-kuhn-quic-4-sat/
> Htmlized:       https://tools.ietf.org/html/draft-kuhn-quic-4-sat-04
> Htmlized:       https://datatracker.ietf.org/doc/html/draft-kuhn-quic-4-sat
> Diff:           https://www.ietf.org/rfcdiff?url2=draft-kuhn-quic-4-sat-04
>
> Abstract:
>    QUIC has been designed for use across Internet paths.  Initial
>    designs of QUIC have focussed on common deployment scenarios for web
>    traffic and have not focussed on the performance when using a path
>    with a large Bandwidth-Delay Product (BDP).  A path can combine
>    satellites network segment together with a wide variety of other
>    network technologies (Ethernet, cable modems, WiFi, cellular, radio
>    links, etc): this complicates the characteristics of the end-to-end
>    path.  One example of such a scenario occurs when a satellite
>    communication (SATCOM) system is used to provide all or a part of the
>    end-to-end path.  If this is not addressed, the end-to-end quality of
>    experience can be degraded.
>
>    This memo identifies the characteristics of a SATCOM link that impact
>    the operation of the QUIC transport protocol.  It proposes regression
>    tests to evaluate QUIC over SATCOM links.  It discusses how to ensure
>    acceptable protocol performance.
>
>                                                                                   
>
>
> Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org.
>
> The IETF Secretariat
>
>
> _______________________________________________
> EToSat mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/etosat

---

From: Lars Eggert <[email-masked]>
Date: Thu, 23 Apr 2020 08:09:22 +0300

Hi,

do you have measurements with multiple parallel flows and/or competing TCP traffic?

Thanks,
Lars

-- 
Sent from a mobile device; please excuse typos.

> On Apr 23, 2020, at 06:05, Christian Huitema <[email-masked]> wrote:
>

---

From: Christian Huitema <[email-masked]>
Date: Wed, 22 Apr 2020 22:34:16 -0700

I don't. I could probably do a parallel flow test, but I don't have the TCP simulation ready. Basically, I have been starting with the test cases in the quic4sat draft.

In theory, parallel flows are easier, not harder, because each flow has a smaller BDP than if it were alone. But fairness is much harder to achieve with long delay links. In TCP fairness is a statistical property, measured over some time. But in our cases the whole transfer is over in a half dozen RTT, and that does not leave much time for the statistics to play out. If we do desire fairness, my suggestion would be appropriate AQM in front of the satellite link.

-- Christian Huitema 

> On Apr 22, 2020, at 10:09 PM, Lars Eggert <[email-masked]> wrote:
> 
> Hi,
>

---

From: Lars Eggert <[email-masked]>
Date: Thu, 23 Apr 2020 11:07:32 +0300

I was asking, because it’s pretty simple to come up with a scheme that does ok for a given static and otherwise empty pipe.

The tricky part is how things behave if there is other traffic, which is why in CC research there is a bunch of standard scenarios that are evaluated (dumbbell, parking lot, etc.)

> On Apr 23, 2020, at 08:35, Christian Huitema <[email-masked]> wrote:
>

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 23 Apr 2020 10:22:42 +0100

I think we do need to be quite careful here!

Filling a pipe with a flow could be a goal for some cases, although I 
don't think it is typical, if we consider the higher capacity paths. 
Multiple flows per bottleneck are often more important, and then we have 
the usual CC questions about RTT distribution, randomness of flow start, 
etc.

For startup, it's also really important to think about the arrival of 
new flows into an already congested pipe. I like to start with two very 
specific questions:

* What happens in the first actual path RTT of time? - Anything sent 
before feedback from the remote endpoint can lead to congestion 
collapse, because the sender does not know the situation. So a stupid 
method that says "last time I sent at X Mbps" so this time I will do 
also, inevitably ends in collapse. Initial entry into the pipe has to be 
conservative. We seem to agree that 1 packet/RTT is a safe starting 
point. IW=10 revises this to, "if IW=10 worked last time, you can try 
again" - conversely if it did not, you should not.

* After one RTT, did *everything* that was sent make it through? i.e., 
was the path as expected? Then, and only then, a sender may have reason 
to be optimistic that you are not already observing congestion collapse. 
That means the sender can *then* probe for capacity and retreat from 
congestion. How you do that depends...

Without surprise, TCP does both of these, in it's syn exchange and CC.

I think QUIC currently is a poor citizen with respect to avoiding 
congestion collapse (It has an initial RTO of 100s of ms, while sending 
a large initial segment and aggresively using IW=10) - perhaps because 
it assumes well-provisioned broadband? Anyhow this isn't nice if you are 
stuck for capacity, and we need to think about that initial RTT of data, 
before we think about CC.

Gorry

On 23/04/2020 09:07, Lars Eggert wrote:
> -- Sent from a mobile device; please excuse typos.
>> On Apr 23, 2020, at 08:35, Christian Huitema<[email-masked]>  wrote:
>>
>>> On Apr 22, 2020, at 10:09 PM, Lars Eggert<[email-masked]>  wrote:
>>>
>>> Hi,
>>>
>>>>> On Apr 23, 2020, at 06:05, Christian Huitema<[email-masked]>  wrote:
>>>>> De :[email-masked]  <[email-masked]>  
>>>>> À : Gorry Fairhurst<[email-masked]>; Kuhn Nicolas<[email-masked]>; John Border<[email-masked]>; Godred Fairhurst<[email-masked]>; Stephan Emile<[email-masked]>; Emile Stephan<[email-masked]>
>>>>> URL:https://www.ietf.org/internet-drafts/draft-kuhn-quic-4-sat-04.txt
>>>>> Status:https://datatracker.ietf.org/doc/draft-kuhn-quic-4-sat/
>>>>> Htmlized:https://tools.ietf.org/html/draft-kuhn-quic-4-sat-04
>>>>> Htmlized:https://datatracker.ietf.org/doc/html/draft-kuhn-quic-4-sat
>>>>> Diff:https://www.ietf.org/rfcdiff?url2=draft-kuhn-quic-4-sat-04
>>>>>
>>>>> Abstract:

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Thu, 23 Apr 2020 08:14:28 -0500

Hi, Lars,

On Thu, Apr 23, 2020 at 3:07 AM Lars Eggert <[email-masked]> wrote:

> I was asking, because it’s pretty simple to come up with a scheme that
> does ok for a given static and otherwise empty pipe.
>
> The tricky part is how things behave if there is other traffic, which is
> why in CC research there is a bunch of standard scenarios that are
> evaluated (dumbbell, parking lot, etc.)
>

Just to avoid the "bring us a rock" scenario ("no, that's not the right
rock, please bring us another rock"), is there a better resource for people
to be looking at than
https://tools.ietf.org/html/draft-irtf-iccrg-tcpeval-01?

Best,

Spencer, who has never done congestion control research ...

---

From: Lars Eggert <[email-masked]>
Date: Thu, 23 Apr 2020 16:20:19 +0300

On 2020-4-23, at 16:14, Spencer Dawkins at IETF <[email-masked]> wrote:
> Just to avoid the "bring us a rock" scenario ("no, that's not the right rock, please bring us another rock"), is there a better resource for people to be looking at than https://tools.ietf.org/html/draft-irtf-iccrg-tcpeval-01?

That's probably (and unfortunately) the most recent. The paper (see citations) is even older.

Lars

---

From: Christian Huitema <[email-masked]>
Date: Thu, 23 Apr 2020 16:26:02 -0700

> That's probably (and unfortunately) the most recent. The paper (see citations) is even older.

Clearly my results so far are preliminary -- showing that a combination
of changes could work, but not passing the gauntlet of tests listed in
the TCP Eval draft. Given my limited resource, it will take some time to
actually do that. I need to prioritize, and I have a couple of questions:

1) The tests in the TCP Eval draft are largely focused on long term
equilibrium. In my proposal, I am mostly looking at ramp up time from
the start of the connection. Once the connection has found the initial
sstresh (or equivalent), it will behave just like BBR. Is there specific
guidance on the evaluation of the ramping up scenario?

2) The battery of tests covers a large number of configurations. The
proposal is very specifically aiming at long delay links, defined as min
RTT larger than some threshold. Is there any advantage to also testing
for the low delay configurations?

3) The draft states that the traces uses for calibrating TMIX are
available at http://trac.tools.ietf.org/group/irtf/trac/wiki/ICCRG.
<http://trac.tools.ietf.org/group/irtf/trac/wiki/ICCRG> I cannot find
them at that location. Where should I look?

I also have a puzzling question regarding the purpose of the tests. The
TCP Eval draft follows the classic approach that Internet stability
depends on well behaved transports, and that this good behavior should
be documented by a large battery of tests. I got some fairly consistent
feedback last year, after worrying that free wheeling innovation in
transport protocols might end up having bad consequences for the
Internet. A number of distinguished colleagues lined up at the mic and
explained that my fears were exaggerated, that the Internet was very
robust, and that I should not worry. So, which is which? Should I worry
about consequences for others, or is selfishness OK?

Of course, even assuming that selfishness is OK, I need to do additional
tests to verify that the proposal still results in better performance in
the presence of competing traffic, at various data rates, or in the
presence of delay jitter. I already perform the tests for several data
rates, and with presence or absence of packet losses -- as specified in
the Etosat draft. My first priority is to add the delay jitter tests, as
they have a critical impact on bandwidth measurement.

-- Christian Huitema

---

From: Jeremy Harris <[email-masked]>
Date: Fri, 24 Apr 2020 10:02:53 +0100

> about consequences for others, or is selfishness OK?

Those distinguished colleagues have forgotten, or weren't around,
for the congestion-collapse phase of the Internet (around 1990?)
which drove the development of congestion control.

We only have robustness because of such consideration.  Selfishness
is not ok.
-- 
Cheers,
  Jeremy

---

From: Lars Eggert <[email-masked]>
Date: Fri, 24 Apr 2020 13:56:07 +0300

Hi,

On 2020-4-24, at 2:26, Christian Huitema <[email-masked]> wrote:
> 1) The tests in the TCP Eval draft are largely focused on long term equilibrium. In my proposal, I am mostly looking at ramp up time from the start of the connection. Once the connection has found the initial sstresh (or equivalent), it will behave just like BBR. Is there specific guidance on the evaluation of the ramping up scenario?

what's interesting to look at is what happens if some connections using your scheme ramp up while others (yours or legacy ones) are (1) also ramping up in parallel, (2) are already in steady-state, or a mix of the two scenarios. Both in terms of what happens to the performance of those connections using your scheme, and also what happens to those that are not.

> 2) The battery of tests covers a large number of configurations. The proposal is very specifically aiming at long delay links, defined as min RTT larger than some threshold. Is there any advantage to also testing for the low delay configurations?

Can you know that you will run over a long-delay link ahead of time? If you want to limit yourself to only evaluating those scenarios (because you will use a different scheme for the other scenarios), you need to be able to tell reliably.

> 3) The draft states that the traces uses for calibrating TMIX are available at http://trac.tools.ietf.org/group/irtf/trac/wiki/ICCRG. I cannot find them at that location. Where should I look?

Don't know. Ask the authors?

> I also have a puzzling question regarding the purpose of the tests. The TCP Eval draft follows the classic approach that Internet stability depends on well behaved transports, and that this good behavior should be documented by a large battery of tests. I got some fairly consistent feedback last year, after worrying that free wheeling innovation in transport protocols might end up having bad consequences for the Internet. A number of distinguished colleagues lined up at the mic and explained that my fears were exaggerated, that the Internet was very robust, and that I should not worry. So, which is which? Should I worry about consequences for others, or is selfishness OK?

The *Internet* will very likely be fine these days, even if broken CC gets shipped to some fraction of the nodes. The reason being that the network topology these days (for consumer access networks) is usually such that the bottleneck link is the access hop.

You still want a working CC scheme, however, because you can DDoS your other flows sharing that same access link, i.e., from other machines/users in your own home. (BitTorrent wasn't a problem for the Internet per se, but it did kill parallel VoIP connections out of the same house pretty effectively, which is what motivated LEDBAT.)

Lars

> Of course, even assuming that selfishness is OK, I need to do additional tests to verify that the proposal still results in better performance in the presence of competing traffic, at various data rates, or in the presence of delay jitter. I already perform the tests for several data rates, and with presence or absence of packet losses -- as specified in the Etosat draft. My first priority is to add the delay jitter tests, as they have a critical impact on bandwidth measurement.
> 
> -- Christian Huitema
>

---

From: Lars Eggert <[email-masked]>
Date: Fri, 24 Apr 2020 13:58:00 +0300

Hi,

> We only have robustness because of such consideration.  Selfishness

the Internet is different in many places now. Selfishness is still not OK, but usually your are (only) hurting yourself vs. impacting others. Back then, the core was under-provisioned compared to the access networks, these days it's usually the opposite.

Lars

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Fri, 24 Apr 2020 06:01:15 -0500

Top posting ...

When Christian sent out his paper on implications of pluggable congestion
control in user space, Mirja and I talked about having him talk about it at
TSVAREA. I don't remember why that didn't happen, but ISTM that's still a
really important question, and having that discussion on a non-WG mailing
list for a rather narrow topic isn't going to get the attention it deserves.

Christian, do you want to ask your question on TSVAREA? If not, I'd be
happy to.

Best,

Spencer

---

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 24 Apr 2020 13:59:17 +0200

Christian gave exactly this talk a little later in a tsvarea session. I think that was what he was referring to.

> On 24. Apr 2020, at 13:01, Spencer Dawkins at IETF <[email-masked]> wrote:
> 
> Top posting ...
> 
> When Christian sent out his paper on implications of pluggable congestion control in user space, Mirja and I talked about having him talk about it at TSVAREA. I don't remember why that didn't happen, but ISTM that's still a really important question, and having that discussion on a non-WG mailing list for a rather narrow topic isn't going to get the attention it deserves.
> 
> Christian, do you want to ask your question on TSVAREA? If not, I'd be happy to.
> 
> Best,
> 
> Spencer
>

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Fri, 24 Apr 2020 08:52:07 -0500

Hi, Mirja,

> think that was what he was referring to.

Ah, at https://datatracker.ietf.org/doc/minutes-105-tsvarea/. The IETF
meeting where I had a concussion at 3 AM on Monday morning (not kidding,
it's surprising I remember anything from that week) - thanks for the
pointer!

I think Christian's question about how much we should care about
fairness/selfishness, and what that means, is still open, still
interesting, and still better had on a mailing list with a broader stated
purpose than "encryption over Satellite". Is TSVAREA still the right place
for that discussion? If not, where?

Best,

Spencer

> > When Christian sent out his paper on implications of pluggable
> congestion control in user space, Mirja and I talked about having him talk
> about it at TSVAREA. I don't remember why that didn't happen, but ISTM
> that's still a really important question, and having that discussion on a
> non-WG mailing list for a rather narrow topic isn't going to get the
> attention it deserves.
> >

---

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 24 Apr 2020 15:58:39 +0200

Good that we have recordings then :-)

I think ICCRG is the right place for that discussion. We mainly had this in the tsvarea because this was something brought up by the IAB to get IETF input but part of the conclusion I think is also that ICCRG is the better place for this work! :-)

Mirja

> On 24. Apr 2020, at 15:52, Spencer Dawkins at IETF <[email-masked]> wrote:
> 
> Hi, Mirja, 
> 
> On Fri, Apr 24, 2020 at 6:59 AM Mirja Kuehlewind <[email-masked]> wrote:
> Ah, at https://datatracker.ietf.org/doc/minutes-105-tsvarea/. The IETF meeting where I had a concussion at 3 AM on Monday morning (not kidding, it's surprising I remember anything from that week) - thanks for the pointer!
> 
> I think Christian's question about how much we should care about fairness/selfishness, and what that means, is still open, still interesting, and still better had on a mailing list with a broader stated purpose than "encryption over Satellite". Is TSVAREA still the right place for that discussion? If not, where?
> 
> Best,
> 
> Spencer
>

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Fri, 24 Apr 2020 10:05:15 -0500

Hi, Mirja,

> Good that we have recordings then :-)

And participants without concussions ;-)

> I think ICCRG is the right place for that discussion. We mainly had this
> in the tsvarea because this was something brought up by the IAB to get IETF
> input but part of the conclusion I think is also that ICCRG is the better
> place for this work! :-)
>

That would be a fine answer. Thanks!

Spencer

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 24 Apr 2020 17:47:55 +0100

Yes. By the time traffic leaves the aggregation level, I'd expect most 
packets to travel over a well-provisioned core. Sadly, not true for all.

However, "Yourself" is an interesting term: If one tests an aggressive 
app, I'd have thought the pain was more likely caused to others in your 
house/company/cafe or whatever that share the service. Measuring a 
method to see whether it performs well, is unlikely to show how 
disruptive the tested application is to other possibly quite different, 
traffic.

Gorry