Subject: [EToSat] What we are looking for from QUIC

From: "Border, John" <[email-masked]>
Date: Tue, 3 Dec 2019 18:43:27 +0000

With TCP spoofing, we can currently achieve very high speeds (e.g. > 200 Mbps).  Mostly this is in the lab but we have shown it over the air at tradeshows.  TCP spoofing also provides minimal degradation to these speeds in the face of packet loss.  Ideally, we would like QUIC to evolve to match this performance without spoofing.  The ideal is probably not achievable but we want to get as close as possible.  We are looking at FEC to help with the packet loss problem and large windows to address the throughput problem.  Since enabling FEC and very large windows (especially initial windows) will not be good defaults for general use, we need some sort of learning capability to know when they should be used.  I think all of this is well known.  But, we are also looking ahead.  More and more, satellite is being paired with terrestrial connectivity.   Multipath QUIC will work when that pairing happens at the client.  But, it doesn't help when the client is not aware of the existence of the multiple paths.  Specifically, if the client is connected via WiFi to some sort of access port and that access port has parallel satellite and terrestrial (e.g. LTE) connectivity, the existence of the multiple paths is hidden from the client.  One solution path is to somehow have the access point interact with the client to make the client aware.  (A proxy comes to mind but there are other options.)  The other solution path is to make QUIC be able to adapt quickly to changes in path characteristics.

     We just wanted to throw the above out there so we can keep it mind when working on solutions to the shorter term problem of making QUIC work well over satellite in the first place....

John

---

From: Christian Huitema <[email-masked]>
Date: Wed, 4 Dec 2019 07:41:54 +0800

On 12/4/2019 2:43 AM, Border, John wrote:
>
>  
>
>      With TCP spoofing, we can currently achieve very high speeds
> (e.g. > 200 Mbps).  Mostly this is in the lab but we have shown it
> over the air at tradeshows.  TCP spoofing also provides minimal
> degradation to these speeds in the face of packet loss.  Ideally, we
> would like QUIC to evolve to match this performance without spoofing. 
> The ideal is probably not achievable but we want to get as close as
> possible.  We are looking at FEC to help with the packet loss problem
> and large windows to address the throughput problem.  Since enabling
> FEC and very large windows (especially initial windows) will not be
> good defaults for general use, we need some sort of learning
> capability to know when they should be used.  I think all of this is
> well known. 
>
Yes, that's what I have been testing for with Picoquic. Once the windows
are established, we can certainly get >200Mbps on the simulator -- we
get 600 Mbps in loopback connections. The congestion control and flow
control algorithms are designed to find the right window values.
Starting with 0 knowledge it takes time, but given time they do find
them. The challenge is to shorten that adaptation time in the general case.

Detecting 1% loss and retransmitting the missing packets is obvious. I
don't think we need FEC, unless we value timeliness over efficiency. The
big challenge is to distinguish between congestion losses happening at
the bottleneck and random losses happening elsewhere on the path. That's
possible if the congestion algorithm uses delay variations and ECN marks
instead of losses to detect congestion.

Then there is a issue of asymmetric links. That requires sending way
fewer ACK than what is specified in the QUIC recovery spec.

> But, we are also looking ahead.  More and more, satellite is being
> paired with terrestrial connectivity.   Multipath QUIC will work when
> that pairing happens at the client.  But, it doesn’t help when the
> client is not aware of the existence of the multiple paths. 
> Specifically, if the client is connected via WiFi to some sort of
> access port and that access port has parallel satellite and
> terrestrial (e.g. LTE) connectivity, the existence of the multiple
> paths is hidden from the client.  One solution path is to somehow have
> the access point interact with the client to make the client aware. 
> (A proxy comes to mind but there are other options.)  The other
> solution path is to make QUIC be able to adapt quickly to changes in
> path characteristics.
>
Let's deal with that later. Multipath is on the work list for QUIC v2.

>  
>
>      We just wanted to throw the above out there so we can keep it
> mind when working on solutions to the shorter term problem of making
> QUIC work well over satellite in the first place….
>
>  
>
The good news is that while satellite links are special, the three
challenges above are also present in other environments. Faster
adaptation, recognizing non congestion losses or sending fewer ACKs is
beneficial in general. So there is hope...
> John
-- Christian Huitema

---

From: Junho Choi <[email-masked]>
Date: Tue, 3 Dec 2019 16:08:05 -0800

On Tue, Dec 3, 2019 at 10:43 AM Border, John <[email-masked]> wrote:

>
>
>      With TCP spoofing, we can currently achieve very high speeds (e.g. >
> 200 Mbps).
>
>
Sorry for newbie question but where can I find more information on TCP
spoofing and how it can improve long-haul network performance?

Thanks,

-- 
Junho Choi <junho dot choi at gmail.com>

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 4 Dec 2019 01:37:27 +0000 (UTC)

John Border wrote

https://tools.ietf.org/html/rfc3135

Lloyd Wood [email-masked] http://about.me/lloydwood 

    On Wednesday, 4 December 2019, 11:08:51 GMT+11, Junho Choi <[email-masked]> wrote:  

 

     With TCP spoofing, we can currently achieve very high speeds (e.g. > 200 Mbps).  

Sorry for newbie question but where can I find more information on TCP spoofing and how it can improve long-haul network performance?
Thanks,

-- 
Junho Choi <junho dot choi at gmail.com

---

From: "Su, Chi-Jiun" <[email-masked]>
Date: Wed, 4 Dec 2019 01:39:48 +0000

This thesis and the references therein should provide basic idea of tcp spoofing (tcp proxies).

https://drum.lib.umd.edu/handle/1903/6106

https://drum.lib.umd.edu/bitstream/handle/1903/6106/MS_99-10.pdf?sequence=1&isAllowed=y

some more math
http://web.eecs.umich.edu/~mingyan/pub/JCC04.pdf

cj
Navid Ehsan*, Mingyan Liu<http://web.eecs.umich.edu/~mingyan/pub/JCC04.pdf>
server–client RTT. Our analysis, therefore only applies to situations where there is a cache miss. 2.2. Assumptions and parameters We assume that a ﬁle contains exactly M segments of the
web.eecs.umich.edu

drum.lib.umd.edu<https://drum.lib.umd.edu/bitstream/handle/1903/6106/MS_99-10.pdf?sequence=1&isAllowed=y>
M P IJ^KI NM ME IKT P IO5fPUxK LNJ hs r U O HNM O%m J^ { Ux nUz n ot ^W'< AO I ^KIJ fD ` P AO I ^ rHNMLNM O otmE r ot MP L P r NM N B s K{ 5Lpm J^ V yUxL M O O Iop NM n J^L opLNJ ] IO%mD IJ^ h NJ opLNMEKI
drum.lib.umd.edu

________________________________
From: EToSat <[email-masked]> on behalf of Junho Choi <[email-masked]>
Sent: Tuesday, December 3, 2019 7:08 PM
To: Border, John <[email-masked]>
Cc: [email-masked] <[email-masked]>
Subject: Re: [EToSat] What we are looking for from QUIC

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

On Tue, Dec 3, 2019 at 10:43 AM Border, John <[email-masked]<mailto:[email-masked]>> wrote:

Thanks,

--
Junho Choi <junho dot choi at gmail.com<https://urldefense.com/v3/__http://gmail.com__;!!Emaut56SYw!gFqcg39rPXq4EDxtSDfvwc6kmTTdXBZ-kFJ_SvE8j1rdhCfaVQiLXom0rV1JmH-9jA$>>

---

From: Junho Choi <[email-masked]>
Date: Tue, 3 Dec 2019 20:53:14 -0800

Thank you guys, it's very helpful! I will take a look.

Best,

On Tue, Dec 3, 2019 at 5:39 PM Su, Chi-Jiun <[email-masked]> wrote:

> Navid Ehsan*, Mingyan Liu
> <http://web.eecs.umich.edu/~mingyan/pub/JCC04.pdf>
> server–client RTT. Our analysis, therefore only applies to situations
> where there is a cache miss. 2.2. Assumptions and parameters We assume that
> a ﬁle contains exactly M segments of the
> web.eecs.umich.edu
>
> drum.lib.umd.edu
> <https://drum.lib.umd.edu/bitstream/handle/1903/6106/MS_99-10.pdf?sequence=1&isAllowed=y>
> M P IJ^KI NM ME IKT P IO5fPUxK LNJ hs r U O HNM O%m J^ { Ux nUz n ot ^W'<
> AO I ^KIJ fD ` P AO I ^ rHNMLNM O otmE r ot MP L P r NM N B s K{ 5Lpm J^ V
> yUxL M O O Iop NM n J^L opLNJ ] IO%mD IJ^ h NJ opLNMEKI
> drum.lib.umd.edu
>
>
>
> ------------------------------
> *From:* EToSat <[email-masked]> on behalf of Junho Choi <
> [email-masked]>
> *Sent:* Tuesday, December 3, 2019 7:08 PM
> *To:* Border, John <[email-masked]>
> *Cc:* [email-masked] <[email-masked]>
> *Subject:* Re: [EToSat] What we are looking for from QUIC
>
> *CAUTION:* This email originated from outside of the organization. Do not
> click links or open attachments unless you recognize the sender and know
> the content is safe.
>
> <https://urldefense.com/v3/__http://gmail.com__;!!Emaut56SYw!gFqcg39rPXq4EDxtSDfvwc6kmTTdXBZ-kFJ_SvE8j1rdhCfaVQiLXom0rV1JmH-9jA$>
> >
>

---

From: Kuhn Nicolas <[email-masked]>
Date: Wed, 4 Dec 2019 07:51:22 +0000

De : EToSat <[email-masked]> De la part de Christian Huitema
Envoyé : mercredi 4 décembre 2019 00:42
À : Border, John <[email-masked]>; [email-masked]
Objet : Re: [EToSat] What we are looking for from QUIC

     With TCP spoofing, we can currently achieve very high speeds (e.g. > 200 Mbps).  Mostly this is in the lab but we have shown it over the air at tradeshows.  TCP spoofing also provides minimal degradation to these speeds in the face of packet loss.  Ideally, we would like QUIC to evolve to match this performance without spoofing.  The ideal is probably not achievable but we want to get as close as possible.  We are looking at FEC to help with the packet loss problem and large windows to address the throughput problem.  Since enabling FEC and very large windows (especially initial windows) will not be good defaults for general use, we need some sort of learning capability to know when they should be used.  I think all of this is well known.

Yes, that's what I have been testing for with Picoquic. Once the windows are established, we can certainly get >200Mbps on the simulator -- we get 600 Mbps in loopback connections. The congestion control and flow control algorithms are designed to find the right window values. Starting with 0 knowledge it takes time, but given time they do find them. The challenge is to shorten that adaptation time in the general case.

[NK] The challenge is to shorten that adaption time indeed - and to let client and server know about the network underneath for 0-RTT connections. This is what we tried to achieve with the 0-RTT BDP draft that we need to update.

Detecting 1% loss and retransmitting the missing packets is obvious. I don't think we need FEC, unless we value timeliness over efficiency. The big challenge is to distinguish between congestion losses happening at the bottleneck and random losses happening elsewhere on the path. That's possible if the congestion algorithm uses delay variations and ECN marks instead of losses to detect congestion.

[NK]  For short flows, 1% losses is not really a big deal. However, when it comes to long files transfers, things are different. 0.1% random losses may halve the goodput. Considering FEC in QUIC has been proposed in NWCRG [https://tools.ietf.org/html/draft-swett-nwcrg-coding-for-quic-03] but it looks like going further in this discussion needs to consider the impacts between FEC and loss-based congestion control [which we tried to initiate in https://tools.ietf.org/html/draft-kuhn-coding-congestion-transport-00]. If the congestion control is delay based, life is easier but otherwise, distinguishing congestion and random losses is a chimera to me, if there is no efficient in network signaling.

Then there is a issue of asymmetric links. That requires sending way fewer ACK than what is specified in the QUIC recovery spec.

[NK] Indeed. Looking at the QUIC transport document [https://github.com/quicwg/base-drafts/blob/master/draft-ietf-quic-transport.md], I can see : "An ACK frame SHOULD be generated for at least every second ack-eliciting packet. This recommendation is in keeping with standard practice for TCP {{?RFC5681}}." However, there do not seem to be a generic transport parameter for changing dynamically this value. Should we file an issue and change the SHOULD for a RECOMMEND and propose other values and the need for a transport parameter?
But, we are also looking ahead.  More and more, satellite is being paired with terrestrial connectivity.   Multipath QUIC will work when that pairing happens at the client.  But, it doesn't help when the client is not aware of the existence of the multiple paths.  Specifically, if the client is connected via WiFi to some sort of access port and that access port has parallel satellite and terrestrial (e.g. LTE) connectivity, the existence of the multiple paths is hidden from the client.  One solution path is to somehow have the access point interact with the client to make the client aware.  (A proxy comes to mind but there are other options.)  The other solution path is to make QUIC be able to adapt quickly to changes in path characteristics.

The good news is that while satellite links are special, the three challenges above are also present in other environments. Faster adaptation, recognizing non congestion losses or sending fewer ACKs is beneficial in general. So there is hope...
[NK] Indeed !

John

-- Christian Huitema

---

From: Emmanuel Lochin <[email-masked]>
Date: Wed, 4 Dec 2019 09:14:07 +0100

You can have a look to the following Marc Allman paper : 
https://www.icir.org/mallman/pubs/IA01/IA01.pdf

EL

Le 04/12/2019 à 01:08, Junho Choi a écrit :
>     (e.g. > 200 Mbps).
> Junho Choi <junho dot choi at gmail.com <http://gmail.com>>
>
> _______________________________________________
> EToSat mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/etosat

-- 
Emmanuel LOCHIN
Professeur ISAE
ISAE SUPAERO - Institut Supérieur de l'Aéronautique et de l'Espace
10 avenue Edouard Belin - BP 54032 - 31055 TOULOUSE CEDEX 4 FRANCE -
http://www.isae-supaero.fr
Tel +33 5 61 33 84 85 - Fax (+33) 5 61 33 83 45
Web : http://personnel.isae.fr/emmanuel-lochin/

---

From: Gorry Fairhurst <[email-masked]>
Date: Wed, 04 Dec 2019 11:00:01 +0000

On 04/12/2019, 07:51, Kuhn Nicolas wrote:
>
> *De :* EToSat <[email-masked]> *De la part de* Christian Huitema
> *Envoyé :* mercredi 4 décembre 2019 00:42
> *À :* Border, John <[email-masked]>; [email-masked]
> *Objet :* Re: [EToSat] What we are looking for from QUIC
>
>     With TCP spoofing, we can currently achieve very high speeds (e.g.
>     > 200 Mbps). Mostly this is in the lab but we have shown it over
>     the air at tradeshows. TCP spoofing also provides minimal
>     degradation to these speeds in the face of packet loss. Ideally,
>     we would like QUIC to evolve to match this performance without
>     spoofing. The ideal is probably not achievable but we want to get
>     as close as possible. We are looking at FEC to help with the
>     packet loss problem and large windows to address the throughput
>     problem. Since enabling FEC and very large windows (especially
>     initial windows) will not be good defaults for general use, we
>     need some sort of learning capability to know when they should be
>     used. I think all of this is well known.
>
> Yes, that's what I have been testing for with Picoquic. Once the 
> windows are established, we can certainly get >200Mbps on the 
> simulator -- we get 600 Mbps in loopback connections. The congestion 
> control and flow control algorithms are designed to find the right 
> window values. Starting with 0 knowledge it takes time, but given time 
> they do find them. The challenge is to shorten that adaptation time in 
> the general case.
>
[GF] The params from John seem fine - but for me I'd also like to ensure 
we test 10/2 Mbps as a baseline - hinted in John's email, because this 
is also a current service datapoint.
>
> */[NK] The challenge is to shorten that adaption time indeed – and to 
> let client and server know about the network underneath for 0-RTT 
> connections. This is what we tried to achieve with the 0-RTT BDP draft 
> that we need to update. /*
>
> don't think we need FEC, unless we value timeliness over efficiency. 
> The big challenge is to distinguish between congestion losses 
> happening at the bottleneck and random losses happening elsewhere on 
> the path. That's possible if the congestion algorithm uses delay 
> variations and ECN marks instead of losses to detect congestion.
>
> */[NK] For short flows, 1% losses is not really a big deal. However, 
> when it comes to long files transfers, things are different. 0.1% 
> random losses may halve the goodput. Considering FEC in QUIC has been 
> proposed in NWCRG 
> [https://tools.ietf.org/html/draft-swett-nwcrg-coding-for-quic-03] but 
> it looks like going further in this discussion needs to consider the 
> impacts between FEC and loss-based congestion control [which we tried 
> to initiate in 
> https://tools.ietf.org/html/draft-kuhn-coding-congestion-transport-00]. If 
> the congestion control is delay based, life is easier but otherwise, 
> distinguishing congestion and random losses is a chimera to me, if 
> there is no efficient in network signaling. /*
>
> */[NK] Indeed. Looking at the QUIC transport document 
> [/*https://github.com/quicwg/base-drafts/blob/master/draft-ietf-quic-transport.md*/], 
> I can see : "An ACK frame SHOULD be generated for at least every 
> second ack-eliciting packet. This recommendation is in keeping with 
> standard practice for TCP {{?RFC5681}}." However, there do not seem to 
> be a generic transport parameter for changing dynamically this value. 
> /**/Should we file an issue and cha/**/nge the SHOULD for a RECOMMEND 
> and propose other values and the need for a transport parameter? /*
>
[GF} We at Aberdeen are digging in this space again. I think what I'd 
like most to see is a recommendation of how to safely adapt 2 to 10. The 
number is important because it impacts CC dynamics and the basic 
smoothness/burstiness of the cwnd growth; and a flow should be more than 
x10 more aggressive than a new flow. I will argue that the number 10 
seems a credible safe number to propose - because we alreday allow IW=10 
as a burst (that should of course be paced if you can), but at least the 
notion exists that 10 buffers are often available on the path.

[GF] If this argument stands, I think we should propose adding a 
decription of why more than 10 needs mroe care, and how you could decide 
between needing 2 or 10 as the number of packets that are cumulatively 
ACKed.
>     paired with terrestrial connectivity. Multipath QUIC will work
>     when that pairing happens at the client. But, it doesn’t help when
>     the client is not aware of the existence of the multiple paths.
>     paths is hidden from the client. One solution path is to somehow
>     have the access point interact with the client to make the client
>     aware. (A proxy comes to mind but there are other options.) The
>     other solution path is to make QUIC be able to adapt quickly to
>     changes in path characteristics.
>
>     We just wanted to throw the above out there so we can keep it mind
>     when working on solutions to the shorter term problem of making
> */[NK] Indeed !/*
Gorry

---

From: Kuhn Nicolas <[email-masked]>
Date: Wed, 4 Dec 2019 16:04:27 +0000

-----Message d'origine-----
De : Gorry Fairhurst <[email-masked]> 
Envoyé : mercredi 4 décembre 2019 12:00
À : Kuhn Nicolas <[email-masked]>
Cc : 'Christian Huitema' <[email-masked]>; Border, John <[email-masked]>; [email-masked]
Objet : Re: [EToSat] What we are looking for from QUIC

On 04/12/2019, 07:51, Kuhn Nicolas wrote:
>
> *De :* EToSat <[email-masked]> *De la part de* Christian 
> Huitema *Envoyé :* mercredi 4 décembre 2019 00:42 *À :* Border, John 
> <[email-masked]>; [email-masked] *Objet :* Re: [EToSat] What 
> we are looking for from QUIC
>
[GF] The params from John seem fine - but for me I'd also like to ensure we test 10/2 Mbps as a baseline - hinted in John's email, because this is also a current service datapoint.
[NK] I guess we should at least keep 3 three scenarios : (10/2 Mbps or 50/10 Mbps) (250 Mbps no loss) (250 Mbps with François loss model). I will push an updated version of the regression tests ASAP. 
>
> */[NK] The challenge is to shorten that adaption time indeed - and to 
> https://tools.ietf.org/html/draft-kuhn-coding-congestion-transport-00]
> . If the congestion control is delay based, life is easier but 
> otherwise, distinguishing congestion and random losses is a chimera to 
> me, if there is no efficient in network signaling. /*
>
> [/*https://github.com/quicwg/base-drafts/blob/master/draft-ietf-quic-t
> ransport.md*/], I can see : "An ACK frame SHOULD be generated for at 
> least every second ack-eliciting packet. This recommendation is in 
> keeping with standard practice for TCP {{?RFC5681}}." However, there 
> do not seem to be a generic transport parameter for changing 
> dynamically this value.
[GF} We at Aberdeen are digging in this space again. I think what I'd like most to see is a recommendation of how to safely adapt 2 to 10. The number is important because it impacts CC dynamics and the basic smoothness/burstiness of the cwnd growth; and a flow should be more than
x10 more aggressive than a new flow. I will argue that the number 10 seems a credible safe number to propose - because we alreday allow IW=10 as a burst (that should of course be paced if you can), but at least the notion exists that 10 buffers are often available on the path.

[GF] If this argument stands, I think we should propose adding a decription of why more than 10 needs mroe care, and how you could decide between needing 2 or 10 as the number of packets that are cumulatively ACKed.
[NK] I totally agree that we should add that description and change the text on the QUIC draft. 
>     when that pairing happens at the client. But, it doesn't help when
>     QUIC work well over satellite in the first place..
>
Gorry

---

From: Ted Hardie <[email-masked]>
Date: Wed, 4 Dec 2019 08:24:34 -0800

On Tue, Dec 3, 2019 at 11:52 PM Kuhn Nicolas <[email-masked]> wrote:

>
>
> *Considering FEC in QUIC has been proposed in NWCRG
> [https://tools.ietf.org/html/draft-swett-nwcrg-coding-for-quic-03
> <https://tools.ietf.org/html/draft-swett-nwcrg-coding-for-quic-03>] but it
> looks like going further in this discussion needs to consider the impacts
> between FEC and loss-based congestion control [which we tried to initiate
> in https://tools.ietf.org/html/draft-kuhn-coding-congestion-transport-00
> <https://tools.ietf.org/html/draft-kuhn-coding-congestion-transport-00>].
> If the congestion control is delay based, life is easier but otherwise,
> distinguishing congestion and random losses is a chimera to me, if there is
> no efficient in network signaling.  *
>
I think it's worth noting here that there are two basic design approaches
possible within the QUIC context.  In one, you add FEC capabilities to QUIC
itself, so that there are standardized transport methods of doing data
recovery without retransmission.  In the second, you layer a FEC-capable
application on top of QUIC's basic acknowledgement mechanisms and allow it
not to acknowledge data loss when FEC has recovered the data for the
application.  The first strategy ensures it is available; the second
ensures the FEC methods chosen match the application's needs as closely as
possibly (including, possibly, no FEC at all).

In the context of real-time communication, we've often seen FEC schemes
vary in suitability by the codecs in use, so we've tended to avoid making
over-arching selections for any general purpose system.  If we're going to
do something similar when those are layered on top of QUIC, I think loss
based congestion control will need what amounts to control frames that
distinguish between "data you should send again" and "data I didn't receive
but there is no need to send again".  From a design perspective my question
is whether those are the same messages you would use to tune the FEC or
different messages.  It would be handy if the same messages could tell the
application to change the K in a K-of-N FEC and tell QUIC something about
how much loss along the path is being absorbed by the FEC data recovery.

I agree that without network signaling there is very little you can do to
directly assess the source of loss.  You can sometimes infer it from jitter
when you have radio networks which do lower-layer retransmission to recover
losses, as the recovery at that layer is seen as jitter at the IP layer,
but it isn't a real good signal as path changes can give the same signal.
I think, though, that this is one of the reasons delay-based congestion
control is seeing more utility as we see more and more paths have at least
one radio link.

regards,

Ted Hardie

---

From: "Morten V. Pedersen" <[email-masked]>
Date: Mon, 9 Dec 2019 12:19:56 +0100

Hi John,
Could you elaborate on the point below:

--------------------------

On 12/3/19 7:43 PM, Border, John wrote:
> We are looking at FEC to help with the packet loss problem and large 
> windows to address the throughput problem.  Since enabling FEC and 
> very large windows (especially initial windows) will not be good 
> defaults for general use, we need some sort of learning capability to 
> know when they should be used.

--------------------------

Is it because of computational complexity or similar?

All the best,
Morten

---

From: "Su, Chi-Jiun" <[email-masked]>
Date: Mon, 9 Dec 2019 13:31:05 +0000

Hi Morten,

John may respond to you when he is available.
I work in the same company as John.
Yes. You're right more or less.

- large window requires allocation of memory.  The number may be significantly huge for a server serving tens/hundreds of thousands of connections.
- FEC adds overhead in transmitted bytes and delay in encoding and decoding of FEC in addition to computational resource requirements.

Hope it answers your questions.
Thanks.
cj

-----Original Message-----
From: EToSat <[email-masked]> On Behalf Of Morten V. Pedersen
Sent: Monday, December 9, 2019 6:20 AM
To: [email-masked]
Subject: Re: [EToSat] What we are looking for from QUIC

Hi John,

All the best,
Morten

_______________________________________________
EToSat mailing list
[email-masked]
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/etosat__;!!Emaut56SYw!jbhJ11p7YT7--EvCJ88bLI55OqJl1kcHQ46EEnS7wGZdk3cJCgFk3JMytIjk4Fi9Cw$

---

From: "Border, John" <[email-masked]>
Date: Mon, 9 Dec 2019 17:39:43 +0000

I guess technically I was making an assumption but...

FEC adds overhead.  The amount of overhead which is tolerable in a given case depends on a lot of factors including the error rate and the latency.  If the link is clean, the overhead is wasted bandwidth which could be used for more goodput.

John

-----Original Message-----
From: EToSat <[email-masked]> On Behalf Of Morten V. Pedersen
Sent: Monday, December 09, 2019 6:20 AM
To: [email-masked]
Subject: Re: [EToSat] What we are looking for from QUIC

Hi John,

All the best,
Morten

_______________________________________________
EToSat mailing list
[email-masked]
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/etosat__;!!Emaut56SYw!gzEsWLNOSGfsmPGleEr0e68WSH3WZviRzCuB3AGdrtc0tB5sRn_0Ir13ykfxS-azlA$

---

From: Christian Huitema <[email-masked]>
Date: Mon, 9 Dec 2019 08:04:35 -1000

This reminds me of the "case for FEC" paper that I wrote a long time ago. The abstract is here: https://link.springer.com/chapter/10.1007%2F978-0-387-34986-2_8

My copy of the text, in Postscript, is here: http://huitema.net/papers/case4fec.ps

-- Christian Huitema 

> On Dec 9, 2019, at 3:31 AM, Su, Chi-Jiun <[email-masked]> wrote:
> 
> Hi Morten,
>

---

From: "Su, Chi-Jiun" <[email-masked]>
Date: Tue, 10 Dec 2019 19:50:18 +0000

From Christian’s paper, the first two cases where use of packet level FEC provides benefits are in satellite communication.

“There are in fact at least three environment where the reduced error rate proves very valuable. When multicasting data toward large groups, even a small individual error rate per recipient may result in large retransmission rates for the whole group and the use of redundancy will result in dramatic efficiency gains. In the case of long transmission delays, the use of redundancy helps maintaining the delivery delays within acceptable limits, even in presence of errors. When the receivers do not have enough memory resources to implement sophisticated retransmission techniques, forward error correction can compensate the relative inefficiency of cheap algorithms of the go-back N family.”

From: EToSat <[email-masked]> On Behalf Of Christian Huitema
Sent: Monday, December 9, 2019 1:05 PM
To: Su, Chi-Jiun <[email-masked]>
Cc: Morten V. Pedersen <[email-masked]>; [email-masked]
Subject: Re: [EToSat] What we are looking for from QUIC

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
This reminds me of the "case for FEC" paper that I wrote a long time ago. The abstract is here: https://link.springer.com/chapter/10.1007%2F978-0-387-34986-2_8<https://urldefense.com/v3/__https:/link.springer.com/chapter/10.1007*2F978-0-387-34986-2_8__;JQ!!Emaut56SYw!ghR40lR1fPlxTxd2FAQ-A_MS23aqVeE3QWHRTwu4oXqBld0WqeiQZEFO8RA2UrI3xw$>

My copy of the text, in Postscript, is here: http://huitema.net/papers/case4fec.ps<https://urldefense.com/v3/__http:/huitema.net/papers/case4fec.ps__;!!Emaut56SYw!ghR40lR1fPlxTxd2FAQ-A_MS23aqVeE3QWHRTwu4oXqBld0WqeiQZEFO8RClCMWs_Q$>
-- Christian Huitema

On Dec 9, 2019, at 3:31 AM, Su, Chi-Jiun <[email-masked]<mailto:[email-masked]>> wrote:
Hi Morten,

-----Original Message-----
From: EToSat <[email-masked]<mailto:[email-masked]>> On Behalf Of Morten V. Pedersen
Sent: Monday, December 9, 2019 6:20 AM
To: [email-masked]<mailto:[email-masked]>
Subject: Re: [EToSat] What we are looking for from QUIC

Hi John,

All the best,
Morten

_______________________________________________
EToSat mailing list
[email-masked]<mailto:[email-masked]>
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/etosat__;!!Emaut56SYw!jbhJ11p7YT7--EvCJ88bLI55OqJl1kcHQ46EEnS7wGZdk3cJCgFk3JMytIjk4Fi9Cw$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/etosat__;!!Emaut56SYw!jbhJ11p7YT7--EvCJ88bLI55OqJl1kcHQ46EEnS7wGZdk3cJCgFk3JMytIjk4Fi9Cw$>

_______________________________________________
EToSat mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/etosat

---

From: "Morten V. Pedersen" <[email-masked]>
Date: Tue, 10 Dec 2019 21:53:45 +0100

Dear all,
Thanks for your input.

@Christian Thanks for the paper - I will carefully go though it. It 
looks like a really nice motivation why coding is useful in multicast / 
broadcast scenarios.

For multicast I would say that the gain of coding should also be present 
even if the receivers have no packet loss, but simply are blocked from 
time to time (this creates a discrepancy between the N receivers) which 
would be costly to fix with a retransmission based system.

I have one question on the encoding you describe, you write that it can 
be done with with XOR and shifts alone. Could you provide some more 
details on that?

@John & CJ: In general I think that makes a lot of sense for 
point-to-point links. You would typically use packet level FEC to avoid 
retransmissions in which case you add redundancy on the link and if you 
don't needed it - well then it is essentially wasted bandwidth.

All the best,
Morten

On 12/10/19 8:50 PM, Su, Chi-Jiun wrote:
>
> From Christian’s paper, the first two cases where use of packet level 
> FEC provides benefits are in satellite communication.
>
>
> “There are in fact at least three environment where the reduced error 
> rate proves very valuable. When multicasting data toward large groups, 
> even a small individual error rate per recipient may result in large 
> retransmission rates for the whole group and the use of redundancy 
> will result in dramatic efficiency gains. In the case of long 
> transmission delays, the use of redundancy helps maintaining the 
> delivery delays within acceptable limits, even in presence of errors. 
> When the receivers do not have enough memory resources to implement 
> sophisticated retransmission techniques, forward error correction can 
> compensate the relative inefficiency of cheap algorithms of the 
> go-back N family.”
>
> *From:* EToSat <[email-masked]> *On Behalf Of *Christian Huitema
> *Sent:* Monday, December 9, 2019 1:05 PM
> *To:* Su, Chi-Jiun <[email-masked]>
> *Cc:* Morten V. Pedersen <[email-masked]>; [email-masked]
> *CAUTION:* This email originated from outside of the organization. Do 
> not click links or open attachments unless you recognize the sender 
> and know the content is safe.
>
> This reminds me of the "case for FEC" paper that I wrote a long time 
> ago. The abstract is here: 
> https://link.springer.com/chapter/10.1007%2F978-0-387-34986-2_8 
> <https://urldefense.com/v3/__https:/link.springer.com/chapter/10.1007*2F978-0-387-34986-2_8__;JQ!!Emaut56SYw!ghR40lR1fPlxTxd2FAQ-A_MS23aqVeE3QWHRTwu4oXqBld0WqeiQZEFO8RA2UrI3xw$>
>
> <https://urldefense.com/v3/__http:/huitema.net/papers/case4fec.ps__;!!Emaut56SYw!ghR40lR1fPlxTxd2FAQ-A_MS23aqVeE3QWHRTwu4oXqBld0WqeiQZEFO8RClCMWs_Q$>
>
> -- Christian Huitema
>
>
>
>     - large window requires allocation of memory.  The number may be
>     significantly huge for a server serving tens/hundreds of thousands
>     of connections.
>     - FEC adds overhead in transmitted bytes and delay in encoding and
>     decoding of FEC in addition to computational resource requirements.
>
>     From: EToSat <[email-masked]
>     <mailto:[email-masked]>> On Behalf Of Morten V. Pedersen
>     To: [email-masked] <mailto:[email-masked]>
>     CAUTION: This email originated from outside of the organization.
>     Do not click links or open attachments unless you recognize the
>     sender and know the content is safe.
>
>     Hi John,
>     [email-masked] <mailto:[email-masked]>
>     <https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/etosat__;!!Emaut56SYw!jbhJ11p7YT7--EvCJ88bLI55OqJl1kcHQ46EEnS7wGZdk3cJCgFk3JMytIjk4Fi9Cw$>
>
>
>     [email-masked] <mailto:[email-masked]>

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 11 Dec 2019 00:46:02 +0000 (UTC)

It's worth pointing out that, to multicast across a satellite link to a group of satellite receivers sharing a forward carrier (say DVB-S2), you need to send using the lowest-common-denominator modulation and coding (modcod) that is usefully accepted by all receivers as closing their individual link budgets. To ensure delivery, you can even lower that commonly-used modcod codepoint even further. (example: with a group of modems that can close their links and receive successfully at 32APSK 7/8, 16APSK 5/6 and 8QPSK 3/4, a shared multicast to the group will be at 8PSK 3/4, because that's the worst case but accepted by all.)

This is link-specific, not requiring network coding redundancy or packet level FEC. For a large diverse group, multicast is likely to already be more heavily and robustly coded across the satellite link than individual unicast transmissions - and you might even split a large multicast group into groups coded separately, based on individual link performances and what coding closes the links for the terminals, so that a couple of poor-performing outliers don't bring everyone else down (and then move the poor performers to another carrier...)

(I'd assume low-density parity-check LDPC use in the forward path - expensive to compute for individual terminals, less so for headends in gateways.)

 

 

From: EToSat <[email-masked]> On Behalf OfChristian Huitema

| 
  |

 
 
-- Christian Huitema 

Hi Morten,

-----Original Message-----
From: EToSat <[email-masked]> On Behalf Of Morten V. Pedersen
Sent: Monday, December 9, 2019 6:20 AM
To: [email-masked]
Subject: Re: [EToSat] What we are looking for from QUIC

Hi John,

 

All the best,
Morten

---

From: "Su, Chi-Jiun" <[email-masked]>
Date: Wed, 11 Dec 2019 14:49:36 +0000

Here are links to packet level FEC, XOR and beyond.

RFC 5109
https://tools.ietf.org/html/rfc5109

Francois Michel’s MS thesis
https://dial.uclouvain.be/memoire/ucl/fr/object/thesis%3A14650/datastream/PDF_01/view
page 29 describe XOR.

From: EToSat <[email-masked]> On Behalf Of Morten V. Pedersen
Sent: Tuesday, December 10, 2019 3:54 PM
To: [email-masked]
Subject: Re: [EToSat] What we are looking for from QUIC

Dear all,
Thanks for your input.

@Christian Thanks for the paper - I will carefully go though it. It looks like a really nice motivation why coding is useful in multicast / broadcast scenarios.

For multicast I would say that the gain of coding should also be present even if the receivers have no packet loss, but simply are blocked from time to time (this creates a discrepancy between the N receivers) which would be costly to fix with a retransmission based system.

I have one question on the encoding you describe, you write that it can be done with with XOR and shifts alone. Could you provide some more details on that?

@John & CJ: In general I think that makes a lot of sense for point-to-point links. You would typically use packet level FEC to avoid retransmissions in which case you add redundancy on the link and if you don't needed it - well then it is essentially wasted bandwidth.

All the best,
Morten

From: EToSat <[email-masked]><mailto:[email-masked]> On Behalf Of Christian Huitema
Sent: Monday, December 9, 2019 1:05 PM
To: Su, Chi-Jiun <[email-masked]><mailto:[email-masked]>
Cc: Morten V. Pedersen <[email-masked]><mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [EToSat] What we are looking for from QUIC

-----Original Message-----
From: EToSat <[email-masked]<mailto:[email-masked]>> On Behalf Of Morten V. Pedersen
Sent: Monday, December 9, 2019 6:20 AM
To: [email-masked]<mailto:[email-masked]>
Subject: Re: [EToSat] What we are looking for from QUIC

Hi John,

All the best,
Morten

_______________________________________________
EToSat mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/etosat<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/etosat__;!!Emaut56SYw!ivs3VnRUz1xYDvhqkbCsC-TQv5gp0ltfQMJ9xMReizTkKC0ZhDQN4lsHG9IdRCTWEg$>

EToSat mailing list

https://www.ietf.org/mailman/listinfo/etosat<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/etosat__;!!Emaut56SYw!ivs3VnRUz1xYDvhqkbCsC-TQv5gp0ltfQMJ9xMReizTkKC0ZhDQN4lsHG9IdRCTWEg$>

---

From: François Michel <[email-masked]>
Date: Wed, 11 Dec 2019 15:12:49 +0000

Hello,

> https://dial.uclouvain.be/memoire/ucl/fr/object/thesis%3A14650/datastream/PDF_01/view

Thank you so much for referencing this. I apologize in advance for the 
imprecisions that may be present in the master thesis. :-) A 
mathematical intuition behind XOR and RLC are present starting from page 
13 if needed.

You can also implement RLC using only XORs (and a pre-computed table) 
and it provides better correcting capabilities. The SWIF codec coming 
from the nwcrg does that: 
https://github.com/irtf-nwcrg/swif-codec/blob/master/src/swif_symbol.c
(the addition is a xor and the multiplication is an access in a 
pre-computed table)

Regards,

François

> *From:* EToSat <[email-masked]> *On Behalf Of *Morten V. Pedersen
> *Sent:* Tuesday, December 10, 2019 3:54 PM
> *To:* [email-masked]
>      From Christian’s paper, the first two cases where use of packet
>     level FEC provides benefits are in satellite communication.
> 
> 
>     “There are in fact at least three environment where the reduced
>     error rate proves very valuable. When multicasting data toward large
>     groups, even a small individual error rate per recipient may result
>     in large retransmission rates for the whole group and the use of
>     redundancy will result in dramatic efficiency gains. In the case of
>     long transmission delays, the use of redundancy helps maintaining
>     the delivery delays within acceptable limits, even in presence of
>     errors. When the receivers do not have enough memory resources to
>     implement sophisticated retransmission techniques, forward error
>     correction can compensate the relative inefficiency of cheap
>     algorithms of the go-back N family.”
> 
>     *From:* EToSat <[email-masked]>
>     <mailto:[email-masked]> *On Behalf Of *Christian Huitema
>     <mailto:[email-masked]>
>     *Cc:* Morten V. Pedersen <[email-masked]>
>     <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>
>         significantly huge for a server serving tens/hundreds of
>         thousands of connections.
>         - FEC adds overhead in transmitted bytes and delay in encoding
>         and decoding of FEC in addition to computational resource
>         requirements.
>

---

From: "Morten V. Pedersen" <[email-masked]>
Date: Fri, 13 Dec 2019 10:29:59 +0100

Dear all,
Thanks for these links, interesting reading. I do know about RLC in the 
binary field (i.e. using XOR) and also some of the 1D and 2D parity 
codes suggested for e.g. RTP. However, in Christian's paper it seemed to 
me that he was not doing RLC, but something different involving also a 
shift (I thought it might be something like a zigzag code). But I 
couldn't quite figure out the notation :) Perhaps some of you can 
enlighten me? :)

All the best,
Morten