Subject: Re: [EToSat] [QUIC performance and SATCOM]

From: Joerg Deutschmann <[email-masked]>
Date: Wed, 9 Oct 2019 16:55:58 +0200

Dear all,

we continued our QUIC over satellite performance measurements with different operators and QUIC implementations. Unfortunately, these are still only black-box tests. The results were presented at the 25th KaConf. The related paper [1] also gives an introduction to QUIC, therefore you probably want to skip to section 4 and 5.

[1] www7content.cs.fau.de/~deutschmann/KaConf2019/KaConf2019_QUICoverSatellite_paper.pdf

Best regards,
Joerg

On 15.02.19 16:48, Joerg Deutschmann wrote:
> Dear all,
> 
> thanks for the link to this interesting paper. We did some related work, 
> which you might be interested in. It will be presented at the NetSys 
> 2019 conference [1]. A version is available at [2].
> 
> While your "QUIC and SATCOM" paper provides more details regarding 
> HTTP/2 vs. QUIC, we took a broader view. We measured page load times 
> over HTTP(S)/1.1 and HTTP/2, with and without VPNs. Additionally we set 
> up two Quic servers (Chromium QUIC and quic-go). All tests were run with 
> three different satellite internet providers, showing quite some 
> differences...
> 
> Best regards,
> Joerg
> 
> [1] www.netsys2019.org
> [2] www7content.cs.fau.de/~deutschmann/NetSys2019_preprint.pdf
> 
> 
> 
> 
> 
> On 09.11.18 04:35, Kuhn Nicolas wrote:
>> Dear all,
>>
>> We have recently posted on ArXiv an article on QUIC and its 
>> performance over a GEO SATCOM link.
>>
>> https://arxiv.org/pdf/1810.04970.pdf
>>
>> This paper has been presented at IETF 103 @MAPRG 
>> https://datatracker.ietf.org/meeting/103/materials/agenda-103-maprg-00
>>
>> The code used to run these experiments is available, so do not 
>> hesitate to contact us if you want to reproduce our experiments.
>>
>> Kind regards,
>>
>> Ludovic Thomas (ISAE-SUPAERO), Emmanuel Dubois (CNES), Nicolas Kuhn 
>> (CNES), Emmanuel Lochin (ISAE-SUPAERO)
>>
>

---

From: Kuhn Nicolas <[email-masked]>
Date: Thu, 10 Oct 2019 10:06:47 +0000

Dear Joerg, 

Thank you for sharing these results with us. 
This seems to confirm that we have an issue when QUIC is confronted to a long RTT and loss-link. 
We will try to come up with "similar" experiments for the next IETF.

Kind regards,

Nicolas

-----Message d'origine-----
De : EToSat <[email-masked]> De la part de Joerg Deutschmann
Envoyé : mercredi 9 octobre 2019 16:56
À : [email-masked]
Objet : Re: [EToSat] [QUIC performance and SATCOM]

Dear all,

Best regards,
Joerg

On 15.02.19 16:48, Joerg Deutschmann wrote:
> Dear all,
> 
> thanks for the link to this interesting paper. We did some related 
> work, which you might be interested in. It will be presented at the 
> NetSys
> over HTTP(S)/1.1 and HTTP/2, with and without VPNs. Additionally we 
> set up two Quic servers (Chromium QUIC and quic-go). All tests were 
> run with three different satellite internet providers, showing quite 
> some differences...
> 
> Best regards,
> Joerg
> 
> [1] www.netsys2019.org

---

From: Tom Jones <[email-masked]>
Date: Thu, 10 Oct 2019 11:45:39 +0100

On Wed, Oct 09, 2019 at 04:55:58PM +0200, Joerg Deutschmann wrote:
> Dear all,
> 
> we continued our QUIC over satellite performance measurements with
> different operators and QUIC implementations. Unfortunately, these are
> still only black-box tests. The results were presented at the 25th
> KaConf. The related paper [1] also gives an introduction to QUIC,
> therefore you probably want to skip to section 4 and 5.
> 

Hi, 

thanks for sharing these results.

Could you expand, maybe off list, the problems you had with netem and
Linux? 

We have network emulation testbeds here using both dummynet on FreeBSD
and netem on Linux and with some work we managed to get similar network
charactaristics out of both. I do wonder if we might have missed something
that you have seen though.

> _______________________________________________
> EToSat mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/etosat

-- 
- [tj]

---

From: Emmanuel Lochin <[email-masked]>
Date: Thu, 10 Oct 2019 13:52:16 +0200

Dear Joerg, Tom,

If you attempt to mimic a satellite link with Netem or Dummynet, you 
might be interested in having a better emulation system such as OpenSAND 
: http://opensand.org/
If you need to be convinced by the capabilities of OpenSAND, check this 
paper : "Making Trustable Satellite Experiments: an Application to a 
VoIP Scenario" available here : http://oatao.univ-toulouse.fr/21907/

EL

-- 
Emmanuel LOCHIN
Professeur ISAE
ISAE SUPAERO - Institut Supérieur de l'Aéronautique et de l'Espace
10 avenue Edouard Belin - BP 54032 - 31055 TOULOUSE CEDEX 4 FRANCE -
http://www.isae-supaero.fr
Tel +33 5 61 33 84 85 - Fax (+33) 5 61 33 83 45
Web : http://personnel.isae.fr/emmanuel-lochin/

Raconter la science, imaginer l'avenir : Journée Portes Ouvertes de l'ISAE-SUPAERO, rendez-vous le samedi 12 octobre de 10h à 18h! Toutes les infos sur isae-supaero.fr

---

From: "Border, John" <[email-masked]>
Date: Thu, 10 Oct 2019 15:44:17 +0000

The results we are in the process of writing up right now also focus on packet loss impact on bulk throughput.  I have asked for a slot at MAPRG to present the results but we intend to publish an I-D before then...

John

-----Original Message-----
From: EToSat <[email-masked]> On Behalf Of Kuhn Nicolas
Sent: Thursday, October 10, 2019 6:07 AM
To: 'Joerg Deutschmann' <[email-masked]>; [email-masked]
Subject: Re: [EToSat] [QUIC performance and SATCOM]

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Dear Joerg, 

Kind regards,

Nicolas

-----Message d'origine-----
De : EToSat <[email-masked]> De la part de Joerg Deutschmann Envoyé : mercredi 9 octobre 2019 16:56 À : [email-masked] Objet : Re: [EToSat] [QUIC performance and SATCOM]

Dear all,

Best regards,
Joerg

On 15.02.19 16:48, Joerg Deutschmann wrote:
> [1] 
> https://urldefense.com/v3/__http://www.netsys2019.org__;!0rLxnc5oh4s!k
> AlU66iaWLJ6FEuJ0maSW_ZWJKLtUszz3bVm1hE3alEoGN5_V5RS6sNlMCe6cXvXkw$
>> https://urldefense.com/v3/__https://arxiv.org/pdf/1810.04970.pdf__;!0
>> rLxnc5oh4s!kAlU66iaWLJ6FEuJ0maSW_ZWJKLtUszz3bVm1hE3alEoGN5_V5RS6sNlMC
>> cNccHCHw$
>>
>> https://urldefense.com/v3/__https://datatracker.ietf.org/meeting/103/
>> materials/agenda-103-maprg-0__;!0rLxnc5oh4s!kAlU66iaWLJ6FEuJ0maSW_ZWJ
>> KLtUszz3bVm1hE3alEoGN5_V5RS6sNlMCfUIN9rXA$
>> 0
>>

_______________________________________________
EToSat mailing list
[email-masked]
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/etosat__;!0rLxnc5oh4s!kAlU66iaWLJ6FEuJ0maSW_ZWJKLtUszz3bVm1hE3alEoGN5_V5RS6sNlMCcnWN2jnA$

---

From: Joerg Deutschmann <[email-masked]>
Date: Mon, 14 Oct 2019 16:52:00 +0200

Dear Emmanuel,

thank you. I've been aware of OpenSAND for some time, but did not try it yet. The paper certainly is interesting.

When using NetEm, our results were worse than using real links or dummynet. The reason may be that we do not have specific NetEm/tbf settings for satellite links. If anybody has, please tell me. With dummynet the results matched the measurements from real satellite links quite well.

Despite the benefits of OpenSAND, I'm afraid that a lot of developers will stick to the emulation tools they are used to. Therefore, I think it would also be interesting to see how satellite links perform with the ns-3 QUIC benchmarking framework [1].

[1] https://github.com/marten-seemann/quic-network-simulator

Best regards,
Joerg

On 10.10.19 13:52, Emmanuel Lochin wrote:
>>> www7content.cs.fau.de/~deutschmann/KaConf2019/KaConf2019_QUICoverSatellite_paper.pdf 
>>>
>> Hi,
>>
>>>> work,

---

From: Kuhn Nicolas <[email-masked]>
Date: Tue, 15 Oct 2019 06:46:57 +0000

Dear Joerg,

If you have any issues in using OpenSAND, do not hesitate to let me know. 

If you are interested in integrate satellite links in the ns-3 QUIC benchmarking framework, you may want to look at SNS-3 : http://www.sns3.org/content/home.php 

Kind regards, 

Nicolas

-----Message d'origine-----
De : EToSat <[email-masked]> De la part de Joerg Deutschmann
Envoyé : lundi 14 octobre 2019 16:52
À : [email-masked]; Emmanuel Lochin <[email-masked]>
Objet : Re: [EToSat] [QUIC performance and SATCOM]

Dear Emmanuel,

Best regards,
Joerg

On 10.10.19 13:52, Emmanuel Lochin wrote:
> If you need to be convinced by the capabilities of OpenSAND, check 
> this paper : "Making Trustable Satellite Experiments: an Application 
> to a VoIP Scenario" available here : 
> http://oatao.univ-toulouse.fr/21907/
> 
> EL
> 
>>> different operators and QUIC implementations. Unfortunately, these 
>>> are still only black-box tests. The results were presented at the 
>>> 25th KaConf. The related paper [1] also gives an introduction to 
>>> QUIC, therefore you probably want to skip to section 4 and 5.
>>>
>>> [1]
>> We have network emulation testbeds here using both dummynet on 
>> FreeBSD and netem on Linux and with some work we managed to get 
>> similar network charactaristics out of both. I do wonder if we might 
>> have missed something that you have seen though.
>>
>>
>>> Best regards,
>>> Joerg
>>>
>>>
>>>
>>>
>>>> HTTP/2 vs. QUIC, we took a broader view. We measured page load 
>>>> times over HTTP(S)/1.1 and HTTP/2, with and without VPNs. 
>>>> Additionally we set up two Quic servers (Chromium QUIC and 
>>>> quic-go). All tests were run with three different satellite 
>>>> internet providers, showing quite some differences...
>>>>
>>>> Best regards,
>>>> Joerg
>>>>
>>>> [1] www.netsys2019.org
>>>>> Ludovic Thomas (ISAE-SUPAERO), Emmanuel Dubois (CNES), Nicolas 
>>>>> Kuhn (CNES), Emmanuel Lochin (ISAE-SUPAERO)
>>>>>
>>
>>