Subject: Re: [nwcrg] [LOOPS] BOF co-chairs thinking on LOOPS next steps

From: Marie-Jose Montpetit <[email-masked]>
Date: Thu, 25 Jul 2019 05:34:54 -0700

Adding the nwcrg list to list thread. And with my NWCRG co-chair hat on.

Like I mentioned the WIFI to node hop was one of the original successes of
network coding as Prof. Médard and her team can attest to. A lot of the
LOOPS use cases for per hop network optimization have been not only
*researched* but *implemented* with running code by many of the nwcrg
participants.

NWCRG-ites; there was a LOOPS BOF on Monday to address the issues of
performance in multiple hop networks and Carsten is pointing to the related
material.

mjm
Marie-José Montpetit, Ph.D.
Research Affiliate, MIT Media Laboratory
[email-masked]
[email-masked]

On July 25, 2019 at 12:56:37 AM, Carsten Bormann ([email-masked]) wrote:

On Jul 24, 2019, at 18:17, Marie-Jose Montpetit <[email-masked]>
wrote:
>
> Standardisation of what? The architecture? The protocols (beware there is
already FECFRAME and others)? The proxies?

Hi Marie-José,

details of the thinking that was developed in the side meetings that
happened at the previous two IETF meetings can be found at

https://github.com/loops-wg/charter

The BOF actually added one piece of information I didn’t previously
perceive that way:
People seem to really want to tackle the host-to-node case (as needed,
e.g., for going through WiFi access clouds) right away, not just the
in-network (node-to-node) case. That would be the one thing I would change
based on my recollection of the BOF. But of course there may be other
things; it may be worth to now have another round of discussion of the
proposed charter.

Grüße, Carsten

---

From: Marie-Jose Montpetit <[email-masked]>
Date: Tue, 30 Jul 2019 07:42:14 -0700

Since most of your questions are related to FEC I copy the nwcrg. See <mjm>
below

mjm

On July 30, 2019 at 10:23:04 AM, Spencer Dawkins at IETF (
[email-masked]) wrote:

Hi, Mirja,

On Tue, Jul 30, 2019 at 5:34 AM Mirja Kuehlewind <[email-masked]>
wrote:

> Hi Carsten, hi Spencer,
>
> > On 25. Jul 2019, at 20:59, Spencer Dawkins at IETF <
> [email-masked]> wrote:
> >
> > My impression, and Magnus might say I'm insane, was that Magnus is right
> on the edge of whether this work can be chartered without a second, working
> group-forming BOF. If the LOOPS community can't live without host-to-node,
> the community should add it. If the community can make use of LOOPS without
> host-to-node and add it later, the community might consider whether that
> reduces the scope of the initial proposed charter enough to help Magnus
> approve it :-)
>
> Yes, it would be possible to charter without another BoF a small and
> well-scoped group, however, it also would be possible to work on small and
> well-defined pieces of work without a new working group. This was one of
> the reasons why this was not a non-wg-forming BoF because it is not clear
> yet if a new group is needed or the resulting pieces of standardization
> work are small enough to fit into an existing group.
>
> Therefore, I really recommend focus on defining the standardising work
> items that you think are needed rather than trying to do any word-smithing
> of a potential charter (that eventually is not needed).
>

Thanks for giving me the chance to clarify - I really was talking about
chartering work, not just chartering a working group, but that wasn't clear
from my note. As you said, knowing what the work is, in more detail, will
help you and Magnus know what to do next.

So, my suggestion for the interested community is to nail down

   1. In order to "do LOOPS", what already exists, that can be used without
   changes?
   2. what already exists, but needs to be extended for LOOPS?
   3. what needs to be created, because nothing exists that meets the needs?

 I'm not a LOOPS proponent (the ADs asked me to chair the BOF about a month
before IETF 105), but speaking as someone who hasn't been involved in
depth, I wonder about

   1. How a sender tunes FEC dynamically - is that automatic, based on FEC
   mechanisms people are thinking about, or is there work to do there?

<mjm> There are FEC mechanisms to adapt coding dynamically based on
acknowledgments for example. In the QUICK implementation we will have flag
a RECOVERED packet that could be used to adapt the coding rate. Other
implementations or code-specific protocols are also available.

   1.
   How a sender knows whether to do FEC, retransmission, or both FEC and
   retransmission, dynamically?

<mjm> Well the use of FEC (or not) is a design decision.  Of course you
need the code (and the libraries). And then the decision will be based on
known link condition, goodput, expected performance, type of traffic
(video, time sensitive etc.). And finding a tunnel and PEPs if you need to.
And it can depend on path statistics and not being end to end. I am not
sure you could standardize that decision.

   1.
   How a sender knows that it shouldn't be doing anything, because anything
   it does won't help ("first, do no harm")?

<mjm> Of course in a congested network doing nothing is probably best.
Again there is a lot of work on the interaction of coding and congestion
control.

<mjm> Not everyone knows the things but there has been a lot of work in
FECFRAME (in IETF) and NWCRG. Someone did mention that a lot of the work
was done in a research group not a working group. But real implementations
exist in famous “networks".

<mjm> Since most of your questions are about erasure coding (coding for
packet loss) is that a direction LOOPS wants to take? Somekind of nwcWG?

mjm

Does everyone know how to do those three things, except me? :-)

Pointers to specifications would be awesome ...

Spencer

> Mirja
> --
LOOPS mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/loops

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 30 Jul 2019 17:01:27 +0200

Hi Marie-José,

> <mjm> There are FEC mechanisms to adapt coding dynamically based on acknowledgments for example. In the QUICK implementation we will have flag a RECOVERED packet that could be used to adapt the coding rate. Other implementations or code-specific protocols are also available.

Right.  So a LOOPS variant will still need to be developed.

>> 	• 
>> How a sender knows whether to do FEC, retransmission, or both FEC and retransmission, dynamically?
> <mjm> Well the use of FEC (or not) is a design decision.  Of course you need the code (and the libraries). And then the decision will be based on known link condition, goodput, expected performance, type of traffic (video, time sensitive etc.). And finding a tunnel and PEPs if you need to. And it can depend on path statistics and not being end to end. I am not sure you could standardize that decision.

What can be decided unilaterally does not need to be standardized.
Information that needs to be collected in cooperation between both ends to enable this decision (“measurement information”) may need to be standardized.

In any case, the LOOPS stance to setting up the channel is a “controller model”, i.e. we don’t plan to have negotiation or some such in the protocol, but expect a controller to set up both ends in a coordinated way.  This is probably the biggest item that needs to be added to enable host-to-node mode, as the host is less likely to be involved in that controller model (even though it could be, that part would then also need to be standardized).

>> How a sender knows that it shouldn't be doing anything, because anything it does won't help ("first, do no harm")?
> <mjm> Of course in a congested network doing nothing is probably best. Again there is a lot of work on the interaction of coding and congestion control.

Right, which would need to be worked out into a protocol plus at least some basic best current practice algorithms for LOOPS.

> <mjm> Not everyone knows the things but there has been a lot of work in FECFRAME (in IETF) and NWCRG. Someone did mention that a lot of the work was done in a research group not a working group. But real implementations exist in famous “networks”. 

Great.  So let’s start to identify those documents that are nearly ready to be used and fit well.  E.g., for NWCRG, I’m aware of:

  "Sliding Window Random Linear Code (RLC) Forward Erasure Correction (FEC)
  Schemes for FECFRAME", Vincent Roca, Belkacem Teibi, 2019-06-18,
  <draft-ietf-tsvwg-rlc-fec-scheme-16.txt>

and possibly (I’m not sure yet LOOPS will be a straightforward application of FECFRAME):

  "Forward Error Correction (FEC) Framework Extension to Sliding Window
  Codes", Vincent Roca, Ali Begen, 2019-01-11,
  <draft-ietf-tsvwg-fecframe-ext-08.txt>

> 
> <mjm> Since most of your questions are about erasure coding (coding for packet loss) is that a direction LOOPS wants to take? Somekind of nwcWG?

In a certain way, very much so.  Now LOOPS is intended to have a retransmission mode, too, which probably would go beyond standard fare of NWCRG.  I’m also not sure that the finer points of measurement that are needed (bilaterally) to enable really good LOOPS implementations (mostly unilaterally) are covered in NWCRG, and I’m also not sure about congestion feedback.  But I wouldn’t mind to call the thing NWCWG, as long as we get to ship the retransmission mode first.

Grüße, Carsten

---

From: Emmanuel Lochin <[email-masked]>
Date: Tue, 30 Jul 2019 17:14:37 +0200

Hi Spencer, all,

On 30/07/2019 16:22, Spencer Dawkins at IETF wrote:
>
>  1. How a sender tunes FEC dynamically - is that automatic, based on
>     FEC mechanisms people are thinking about, or is there work to do
>     there?
>  2. How a sender knows whether to do FEC, retransmission, or both FEC
>     and retransmission, dynamically?
>  3. How a sender knows that it shouldn't be doing anything, because
>     anything it does won't help ("first, do no harm")?
>
IMHO, each answer depends on which layer you seek to apply FEC and why. 
Adding and adapting FEC at L2 or L3 does not have the same impact than 
at L4 and does not lead to the same adaptive mechanism. Then, adding FEC 
at L4 also depends on whether you are congestion controlled or not, you 
want to stick to TCP-friendliness, your QoS objectives, and many other 
things...
Finally concerning #3, I would say that monitoring the e2e delay would 
give you a good indication to enable FEC or not. If the delay is 
"small", you won't beat ARQ.

Emmanuel

-- 
Emmanuel LOCHIN
Professeur ISAE
ISAE SUPAERO - Institut Supérieur de l'Aéronautique et de l'Espace
10 avenue Edouard Belin - BP 54032 - 31055 TOULOUSE CEDEX 4 FRANCE -
http://www.isae-supaero.fr
Tel +33 5 61 33 84 85 - Fax (+33) 5 61 33 83 45
Web : http://personnel.isae.fr/emmanuel-lochin/

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 30 Jul 2019 12:43:15 -0500

I'm going to be replying to two or three e-mail in this thread, so I'll try
to be coherent over my replies ...

On Tue, Jul 30, 2019 at 9:42 AM Marie-Jose Montpetit <[email-masked]>
wrote:

> Since most of your questions are related to FEC I copy the nwcrg. See
> <mjm> below
>
> mjm
>
>> > My impression, and Magnus might say I'm insane, was that Magnus is
>> right on the edge of whether this work can be chartered without a second,
>> working group-forming BOF. If the LOOPS community can't live without
>> host-to-node, the community should add it. If the community can make use of
>> LOOPS without host-to-node and add it later, the community might consider
>> whether that reduces the scope of the initial proposed charter enough to
>> help Magnus approve it :-)
>>
>    1. In order to "do LOOPS", what already exists, that can be used
>    without changes?
>  I'm not a LOOPS proponent (the ADs asked me to chair the BOF about a
> month before IETF 105), but speaking as someone who hasn't been involved in
> depth, I wonder about
>
>    FEC mechanisms people are thinking about, or is there work to do there?
>

So, speaking with no dots because I don't have any, I agree with what
Marie-Jose is saying here, but my understanding is that LOOPS endpoints
might turn LOOPS off completely if local optimizations aren't needed. So
I'm thinking about how a sender makes that decision, and how a sender might
do trade-offs.

I think one point in discussions with Marie-Jose might be that she's
thinking in terms of Technical Specifications (
https://tools.ietf.org/html/rfc2026#section-3.1), which are necessary, but
discussions are making me think that proponents might usefully also think
in terms of Applicability Statements (
https://tools.ietf.org/html/rfc2026#section-3.2), which (quoting)

   An Applicability Statement specifies how, and under what
   circumstances, one or more TSs may be applied to support a particular
   Internet capability.  An AS may specify uses for TSs that are not
   Internet Standards, as discussed in Section 7.

   An AS identifies the relevant TSs and the specific way in which they
   are to be combined, and may also specify particular values or ranges
   of TS parameters or subfunctions of a TS protocol that must be
   implemented.  An AS also specifies the circumstances in which the use
   of a particular TS is required, recommended, or elective (see section
   3.3).

   An AS may describe particular methods of using a TS in a restricted
   "domain of applicability", such as Internet routers, terminal
   servers, Internet systems that interface to Ethernets, or datagram-
   based database servers.

So, if Applicability Statements (also standards-track documents) would be
helpful for LOOPS deployment and operation, the proponents might think
about whether they want to do work in that space.

If decisions about what senders do turn out to be entirely up to the
sender, that's great, but if the proponents want to bound the possible
behaviors of implementations to improve deployment and operation, that
might also be appropriate in BCPs (
https://tools.ietf.org/html/rfc2026#section-5).

>
>    1.
>    How a sender knows that it shouldn't be doing anything, because
Right - so one possibility is publishing work from NWCRG in the IETF
stream, and especially standards-track, with appropriate review by the IETF
community. That could take a lot of forms (RGs and WGs can interact in a
lot of different ways).

> packet loss) is that a direction LOOPS wants to take? Somekind of nwcWG?

See Mirja's note about chartering work, and only chartering working groups
when that's needed. But as you note, the FECFRAME extensions happened in
TSVWG, so that would be a reasonable conversation to have with TSVWG chairs
(and I note that Wes Eddy, who popped up further down in this thread, is a
TSVWG co-chair). If it fits, TSVWG could be a plan. If it doesn't fit in an
existing working group, NWCWG could be a plan.

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 30 Jul 2019 12:46:14 -0500

Hi, Emmanuel,

On Tue, Jul 30, 2019 at 10:14 AM Emmanuel Lochin <
[email-masked]> wrote:

> Adding and adapting FEC at L2 or L3 does not have the same impact than at
> L4 and does not lead to the same adaptive mechanism. Then, adding FEC at L4
> also depends on whether you are congestion controlled or not, you want to
> stick to TCP-friendliness, your QoS objectives, and many other things...
> give you a good indication to enable FEC or not. If the delay is "small",
> you won't beat ARQ.
>

My understanding from discussions around the LOOPS BOF is that this is
intended to be transport-agnostic, so most likely L3. I haven't heard
anyone talking about L2 LOOPS, but as you note, that depends on the
networks that you're trying to help out.

Thanks for the insight! (I thought that was true, but I'm not an NWC expert
by any stretch)

Spencer

> 10 avenue Edouard Belin - BP 54032 - 31055 TOULOUSE CEDEX 4 FRANCE -http://www.isae-supaero.fr

---

From: Marie-Jose Montpetit <[email-masked]>
Date: Tue, 30 Jul 2019 10:55:58 -0700

Well if you to encode IP packets and you have encryption you will get into
the problem of the proposed coding scheme for QUIC that recovers  ALL
packets without knowing what the losses were due to and messing with
congestion control above or not knowing what these packets carry to
determine if they need coding or not or to define the adaptive schemes. Our
solution is before encryption because of that. And as Emmanuel said at
layer 4 you also need to deal with TCP congestion control unless you split
the sessions.

There is a lot of accumulated research, development and implementations out
there :) But ok you are talking to the right people!

mjm

On July 30, 2019 at 1:46:51 PM, Spencer Dawkins at IETF (
[email-masked]) wrote:

Hi, Emmanuel,

Spencer

---

From: "Stuart William Card" <[email-masked]>
Date: Tue, 30 Jul 2019 17:56:40 -0400

I realize LOOPS is about neither multicast nor transport, but for likely
useful background, I hope interested people will at least briefly renew
their acquaintance with, inter alia, RFCs 2357, 3048, 3453, 5740, 6584.

> ...
> There is a lot of accumulated research, development and implementations
> out there :) But ok you are talking to the right people!
> ... Marie-Jos Montpetit...

-- 
Stuart W. Card, VP & Chief Scientist
Critical Technologies Inc. (CTI)
<[email-masked]>

---

From: Colin Perkins <[email-masked]>
Date: Wed, 31 Jul 2019 13:17:21 +0100

> On 30 Jul 2019, at 18:43, Spencer Dawkins at IETF <[email-masked]> wrote:
> 
> I'm going to be replying to two or three e-mail in this thread, so I'll try to be coherent over my replies .... 
> 
> On Tue, Jul 30, 2019 at 9:42 AM Marie-Jose Montpetit <[email-masked] <mailto:[email-masked]>> wrote:
> Since most of your questions are related to FEC I copy the nwcrg. See <mjm> below
> 
> mjm
> 
> [email-masked] <mailto:[email-masked]>
> [email-masked] <mailto:[email-masked]>
> On July 30, 2019 at 10:23:04 AM, Spencer Dawkins at IETF ([email-masked] <mailto:[email-masked]>) wrote:
> 
>> Hi, Mirja, 
>> 
>> On Tue, Jul 30, 2019 at 5:34 AM Mirja Kuehlewind <[email-masked] <mailto:[email-masked]>> wrote:
>> Hi Carsten, hi Spencer,
>> 
>> > On 25. Jul 2019, at 20:59, Spencer Dawkins at IETF <[email-masked] <mailto:[email-masked]>> wrote:
>> >
>> > My impression, and Magnus might say I'm insane, was that Magnus is right on the edge of whether this work can be chartered without a second, working group-forming BOF. If the LOOPS community can't live without host-to-node, the community should add it. If the community can make use of LOOPS without host-to-node and add it later, the community might consider whether that reduces the scope of the initial proposed charter enough to help Magnus approve it :-)
>> 
>> Yes, it would be possible to charter without another BoF a small and well-scoped group, however, it also would be possible to work on small and well-defined pieces of work without a new working group. This was one of the reasons why this was not a non-wg-forming BoF because it is not clear yet if a new group is needed or the resulting pieces of standardization work are small enough to fit into an existing group.
>> 
>> Therefore, I really recommend focus on defining the standardising work items that you think are needed rather than trying to do any word-smithing of a potential charter (that eventually is not needed).
>> 
>> Thanks for giving me the chance to clarify - I really was talking about chartering work, not just chartering a working group, but that wasn't clear from my note. As you said, knowing what the work is, in more detail, will help you and Magnus know what to do next. 
>> 
>> In order to "do LOOPS", what already exists, that can be used without changes?
>> what already exists, but needs to be extended for LOOPS?
>> what needs to be created, because nothing exists that meets the needs?
>>  I'm not a LOOPS proponent (the ADs asked me to chair the BOF about a month before IETF 105), but speaking as someone who hasn't been involved in depth, I wonder about
>> How a sender tunes FEC dynamically - is that automatic, based on FEC mechanisms people are thinking about, or is there work to do there?
> 
> So, speaking with no dots because I don't have any, I agree with what Marie-Jose is saying here, but my understanding is that LOOPS endpoints might turn LOOPS off completely if local optimizations aren't needed. So I'm thinking about how a sender makes that decision, and how a sender might do trade-offs. 
> 
> I think one point in discussions with Marie-Jose might be that she's thinking in terms of Technical Specifications (https://tools.ietf.org/html/rfc2026#section-3.1 <https://tools.ietf.org/html/rfc2026#section-3.1>), which are necessary, but discussions are making me think that proponents might usefully also think in terms of Applicability Statements (https://tools.ietf.org/html/rfc2026#section-3.2 <https://tools.ietf.org/html/rfc2026#section-3.2>), which (quoting) 
> 
>    implemented..  An AS also specifies the circumstances in which the use
> So, if Applicability Statements (also standards-track documents) would be helpful for LOOPS deployment and operation, the proponents might think about whether they want to do work in that space. 
> 
> If decisions about what senders do turn out to be entirely up to the sender, that's great, but if the proponents want to bound the possible behaviors of implementations to improve deployment and operation, that might also be appropriate in BCPs (https://tools.ietf.org/html/rfc2026#section-5 <https://tools.ietf.org/html/rfc2026#section-5>).
>  
>> 
> <mjm> Not everyone knows the things but there has been a lot of work in FECFRAME (in IETF) and NWCRG. Someone did mention that a lot of the work was done in a research group not a working group. But real implementations exist in famous “networks". 
> 
> Right - so one possibility is publishing work from NWCRG in the IETF stream, and especially standards-track, with appropriate review by the IETF community. That could take a lot of forms (RGs and WGs can interact in a lot of different ways). 

An IETF WG could have a work item to develop a standards track FEC scheme based on the research previously done in NWCRG, certainly.
> See Mirja's note about chartering work, and only chartering working groups when that's needed. But as you note, the FECFRAME extensions happened in TSVWG, so that would be a reasonable conversation to have with TSVWG chairs (and I note that Wes Eddy, who popped up further down in this thread, is a TSVWG co-chair). If it fits, TSVWG could be a plan. If it doesn't fit in an existing working group, NWCWG could be a plan.  

I perhaps misunderstood LOOPS, but I’d assumed that – to the extent is uses FEC and not retransmission – it’d pull in one of the schemes developed in FECFRAME. Has there been any requirements analysis to show if anything further is needed?

This is separate to the question of whether there's work in NWCRG that’s ready to move into IETF for standardisation, of course. 

Colin

-- 
Colin Perkins
https://csperkins.org/

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Wed, 31 Jul 2019 08:18:32 -0500

Hi, Colin,

On Wed, Jul 31, 2019, 07:36 Colin Perkins <[email-masked]> wrote:

> On Tue, Jul 30, 2019 at 9:42 AM Marie-Jose Montpetit <
> [email-masked]> wrote:
>
> https://tools.ietf.org/html/rfc2026#section-3.1), which are necessary,
> but discussions are making me think that proponents might usefully also
> think in terms of Applicability Statements (
> An IETF WG could have a work item to develop a standards track FEC scheme
> based on the research previously done in NWCRG, certainly.
>
> I perhaps misunderstood LOOPS, but I’d assumed that – to the extent is
> uses FEC and not retransmission – it’d pull in one of the schemes developed
> in FECFRAME. Has there been any requirements analysis to show if anything
> further is needed?
>

That's the analysis that I'm suggesting the proponents work on now. I
didn't talk to all the proponents in Montreal, but at least a couple were
pretty sure they wanted to use FEC, but were still figuring out details on
that usage.

Marie-Jose and some other nice people from NWCRG are providing pointers to
help with that analysis. For which, I'm grateful. I was the responsible AD
for the recent sliding window FECFRAME extensions that migrated through
TSVWG, and I understand FECFRAME at a 10,000-meter level, but the
proponents (and interested parties) need to work through the next level of
details.

This is separate to the question of whether there's work in NWCRG that’s
> ready to move into IETF for standardisation, of course.
>

The nice thing about the existing relationship between NWCRG and TSVWG is
that if there's more stuff from NWCRG that's ready to standardize in the
IETF, we know how to do that, and Wes Eddy (one of the TSVWG co-chairs) is
already helping with congestion discussion on this mailing list.

I think the non-working group-forming BOF did a good job at flushing out
the questions and the resources needed to answer them. Thanks, Magnus and
Mirja, for making that easy.

Spencer

Colin

---

From: Colin Perkins <[email-masked]>
Date: Wed, 31 Jul 2019 15:14:42 +0100

Agree - thanks, Spencer.
Colin

> On 31 Jul 2019, at 14:18, Spencer Dawkins at IETF <[email-masked]> wrote:
> 
> Hi, Colin,
> 
> On Wed, Jul 31, 2019, 07:36 Colin Perkins <[email-masked] <mailto:[email-masked]>> wrote:
>> On 30 Jul 2019, at 18:43, Spencer Dawkins at IETF <[email-masked] <mailto:[email-masked]>> wrote:
>> 
> That's the analysis that I'm suggesting the proponents work on now. I didn't talk to all the proponents in Montreal, but at least a couple were pretty sure they wanted to use FEC, but were still figuring out details on that usage. 
> 
> Marie-Jose and some other nice people from NWCRG are providing pointers to help with that analysis. For which, I'm grateful. I was the responsible AD for the recent sliding window FECFRAME extensions that migrated through TSVWG, and I understand FECFRAME at a 10,000-meter level, but the proponents (and interested parties) need to work through the next level of details.
> 
> The nice thing about the existing relationship between NWCRG and TSVWG is that if there's more stuff from NWCRG that's ready to standardize in the IETF, we know how to do that, and Wes Eddy (one of the TSVWG co-chairs) is already helping with congestion discussion on this mailing list.
> 
> I think the non-working group-forming BOF did a good job at flushing out the questions and the resources needed to answer them. Thanks, Magnus and Mirja, for making that easy.
> 
> Spencer
> 
> Colin
> 
> 
> 
> 
> -- 
> Colin Perkins
> https://csperkins.org/ <https://csperkins.org/>