Subject: Re: Multipath (was: Re: Re-chartering for extension work)

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 9 Jan 2020 10:19:01 +0000

Hi all,

see below.

From: QUIC <[email-masked]> on behalf of Jana Iyengar <[email-masked]>
Date: Saturday, 21. December 2019 at 02:49
To: Tommy Pauly <[email-masked]>
Cc: Olivier Bonaventure <[email-masked]>, Quentin De Coninck <[email-masked]>, Ryan Hamilton <[email-masked]>, Mark Nottingham <[email-masked]>, Christian Huitema <[email-masked]>, Lars Eggert <[email-masked]>, Mikkel Fahnøe Jørgensen <[email-masked]>, IETF QUIC WG <[email-masked]>
Subject: Re: Multipath (was: Re: Re-chartering for extension work)

Fundamentally, the capabilities that “full” multipath enable on top of migration are:
- Bandwidth aggregation
- Separated congestion control
- The ability to split traffic (such as on a stream basis) across paths to take advantage of different link characteristics

These are all good to enable, but these are not the core benefits that devices right now are gaining from multipath, as much as the ability to migrate with agility. We should absolutely work on multipath, but it may be good to see how shipping migration (the first half of multipath) goes first.

Yep. This has been my experience as well.

I appreciate that there are some cases where simple bandwidth aggregation is useful, but there is a real question about the amount of effort required to not just standardize but also build a multipath extension for those specific use cases.

The relevant question I would be looking for answers to is how many implementers will commit to implementing a multipath extension if one were to be standardized.

FWIW: I also don't believe MPTCP has solved all multipath problems, not in the general case anyways. There are real ones that make actual deployment super difficult. Specifically, scheduling is a real issue. Deciding when to use multiple paths and how to use them remains an open question. The answer in my experience ends up being very application-dependent, which means that you need an application to buy in before you develop a scheduler. And without this buy-in, I couldn't justify spending the time and energy to spend time on building multipath into our implementation at least.

As Tommy said, migration is already the first half of multipath support for QUIC. So I expect it will be easier to add it to QUIC than it was for TCP. I would at least be curious to spend some time on figuring out how much (or how less) effort it would be. Maybe there is also another step before going to full multipath support that could support the handover use case better, e.g. sending redundant data on multi paths or something. I think it worth think about it. However, while I would like to see this discussion starting now, I think it’s fine to wait for adoption of any draft until v1 is done.

Mirja

---

From: Olivier Bonaventure <[email-masked]>
Date: Fri, 10 Jan 2020 10:50:57 +0100

Dear Mirja,

>  
> As Tommy said, migration is already the first half of multipath support for QUIC. So I expect it will be easier to add it to QUIC than it was for TCP.

The main benefit of QUIC is that we do not need to take middleboxes into account and can reason about the transport layer directly. 

> I would at least be curious to spend some time on figuring out how much (or how less) effort it would be. Maybe there is also another step before going to full multipath support that could support the handover use case better, e.g. sending redundant data on multi paths or something. I think it worth think about it.

Please don’t do that. With QUIC,  have a unique opportunity to design an architecturally clean and easily deployable multipath transport protocol. We should not pollute this design with hacks and short term solutions that have been included in too many Internet protocols for various reasons. 

> However, while I would like to see this discussion starting now, I think it’s fine to wait for adoption of any draft until v1 is done.

As already explained in previous emails, Quentin De Coninck has leveraged all the lessons learned from Multipath TCP in a Multipath QUIC design that has evolved in parallel with the standardisation work of QUIC v1. The initial design was described in

https://multipath-quic.org/conext17-deconinck.pdf <https://multipath-quic.org/conext17-deconinck.pdf>  and in https://www.ietf.org/archive/id/draft-deconinck-multipath-quic-00.txt <https://www.ietf.org/archive/id/draft-deconinck-multipath-quic-00.txt>

There was a running implementation on top of quic-go that is available from https://multipath-quic.org <https://multipath-quic.org/>

The draft has evolved several times on https://datatracker.ietf.org/doc/draft-deconinck-quic-multipath/ <https://datatracker.ietf.org/doc/draft-deconinck-quic-multipath/> 

The recent draft has been implemented on top of picoquic and was released on pquic.org <http://pquic.org/> 

Quentin has studied these designs in details with many experiments, both in the lab and in the wild on smartphones. He’d be ready to present this in details to the working group if if eventually decides to work on the multipath item of its charter. I would encourage the working group members to start to read the last version of Quentin’s draft and provide comments.

Best regards,

Olivier

-- 

Disclaimer: https://www.tessares.net/mail-disclaimer/ 
<https://www.tessares.net/mail-disclaimer/>

---

From: Lucas Pardue <[email-masked]>
Date: Fri, 10 Jan 2020 11:39:15 +0000

Hi Oliver, all,

Taking a very brief look over the 03 draft, if I grokked it properly I have
an observation that may be interesting in the broader thinking about
extending QUIC.

In draft-deconinck-quic-multipath-03 [1] it states that because there are
new path-related IDs, some frames need to be modified and then lists them:
NEW_CONNECTION_ID, RETIRE_CONNECTION_ID and ACK. IIUC this would keep the
type value but modify the frame contents. My model of thinking has been to
assume that frame definitions are immutable, and that extensions that
require tweaks to frames would define new ones.

For example, draft-huitema-quic-1wd-00 [2] proposes a timestamped ack and
so requires a "Time Stamp" field in the ACK frame. It does so by defining
two new ACK frame types (vanilla, and one to contain ECN marking).

So my observation is not about the proposals themselves but about the
composability of extensions. Say for example someone wanted to combine
mpquic with 1wd, would they modify the ACK frame or define a new one?

>From an implementation perspective, making frame parsing condition on
connection-level characteristics such as extensions seems like a bad idea.
Frame types are cheap so lets use them.

[1] https://tools.ietf.org/html/draft-deconinck-quic-multipath-03
[2] https://tools.ietf.org/html/draft-huitema-quic-1wd-00

On Fri, Jan 10, 2020 at 9:51 AM Olivier Bonaventure <
[email-masked]> wrote:

> Dear Mirja,
>
>
> As Tommy said, migration is already the first half of multipath support
> for QUIC. So I expect it will be easier to add it to QUIC than it was for
> TCP.
>
>
> The main benefit of QUIC is that we do not need to take middleboxes into
> account and can reason about the transport layer directly.
>
> I would at least be curious to spend some time on figuring out how much
> (or how less) effort it would be. Maybe there is also another step before
> going to full multipath support that could support the handover use case
> better, e.g. sending redundant data on multi paths or something. I think it
> worth think about it.
>
>
>
> Please don’t do that. With QUIC,  have a unique opportunity to design an
> architecturally clean and easily deployable multipath transport protocol.
> We should not pollute this design with hacks and short term solutions that
> have been included in too many Internet protocols for various reasons.
>
> However, while I would like to see this discussion starting now, I think
> it’s fine to wait for adoption of any draft until v1 is done.
>
>
> As already explained in previous emails, Quentin De Coninck has leveraged
> all the lessons learned from Multipath TCP in a Multipath QUIC design that
> has evolved in parallel with the standardisation work of QUIC v1. The
> initial design was described in
>
> https://multipath-quic.org/conext17-deconinck.pdf  and in
> https://www.ietf.org/archive/id/draft-deconinck-multipath-quic-00.txt
>
> There was a running implementation on top of quic-go that is available
> from https://multipath-quic..org <https://multipath-quic.org>
>
> The draft has evolved several times on
> https://datatracker.ietf.org/doc/draft-deconinck-quic-multipath/
>
> The recent draft has been implemented on top of picoquic and was released
> on pquic.org
>
> Quentin has studied these designs in details with many experiments, both
> in the lab and in the wild on smartphones. He’d be ready to present this in
> details to the working group if if eventually decides to work on the
> multipath item of its charter. I would encourage the working group members
> to start to read the last version of Quentin’s draft and provide comments.
>
>
> Best regards,
>
>
> Olivier
>
>
> ------------------------------
>

---

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 10 Jan 2020 15:28:58 +0000

Or alternatively, we can define a new version with MP support and „modify” the frames.

From: Lucas Pardue <[email-masked]>
Date: Friday, 10. January 2020 at 12:39
To: Olivier Bonaventure <[email-masked]>
Cc: Mirja Kuehlewind <[email-masked]>, Quentin De Coninck <[email-masked]>, Jana Iyengar <[email-masked]>, Ryan Hamilton <[email-masked]>, Mark Nottingham <[email-masked]>, Christian Huitema <[email-masked]>, Lars Eggert <[email-masked]>, Mikkel Fahnøe Jørgensen <[email-masked]>, IETF QUIC WG <[email-masked]>, Tommy Pauly <[email-masked]>
Subject: Re: Multipath (was: Re: Re-chartering for extension work)

Hi Oliver, all,

Taking a very brief look over the 03 draft, if I grokked it properly I have an observation that may be interesting in the broader thinking about extending QUIC.

In draft-deconinck-quic-multipath-03 [1] it states that because there are new path-related IDs, some frames need to be modified and then lists them: NEW_CONNECTION_ID, RETIRE_CONNECTION_ID and ACK. IIUC this would keep the type value but modify the frame contents. My model of thinking has been to assume that frame definitions are immutable, and that extensions that require tweaks to frames would define new ones.

For example, draft-huitema-quic-1wd-00 [2] proposes a timestamped ack and so requires a "Time Stamp" field in the ACK frame. It does so by defining two new ACK frame types (vanilla, and one to contain ECN marking).

So my observation is not about the proposals themselves but about the composability of extensions. Say for example someone wanted to combine mpquic with 1wd, would they modify the ACK frame or define a new one?

From an implementation perspective, making frame parsing condition on connection-level characteristics such as extensions seems like a bad idea. Frame types are cheap so lets use them.

On Fri, Jan 10, 2020 at 9:51 AM Olivier Bonaventure <[email-masked]<mailto:[email-masked]>> wrote:
Dear Mirja,

https://multipath-quic.org/conext17-deconinck.pdf<https://protect2.fireeye.com/v1/url?k=4963015f-15eaaeee-496341c4-0cc47ad93cac-a3fac04bd65ca537&q=1&e=84207ce9-30c8-4b7d-bfbf-abe56cf50ef4&u=https%3A%2F%2Fmultipath-quic.org%2Fconext17-deconinck.pdf>  and in https://www.ietf.org/archive/id/draft-deconinck-multipath-quic-00.txt

There was a running implementation on top of quic-go that is available from https://multipath-quic..org<https://protect2.fireeye.com/v1/url?k=328f362d-6e06999c-328f76b6-0cc47ad93cac-9fee698078d89320&q=1&e=84207ce9-30c8-4b7d-bfbf-abe56cf50ef4&u=https%3A%2F%2Fmultipath-quic.org%2F>

The draft has evolved several times on https://datatracker.ietf.org/doc/draft-deconinck-quic-multipath/

The recent draft has been implemented on top of picoquic and was released on pquic.org<https://protect2.fireeye.com/v1/url?k=48975e15-141ef1a4-48971e8e-0cc47ad93cac-a7391ac2483be27f&q=1&e=84207ce9-30c8-4b7d-bfbf-abe56cf50ef4&u=http%3A%2F%2Fpquic.org%2F>

Best regards,

Olivier

________________________________

Disclaimer: https://www.tessares.net/mail-disclaimer/<https://protect2.fireeye.com/v1/url?k=c464a2ac-98ed0d1d-c464e237-0cc47ad93cac-c45a73c094cd50c3&q=1&e=84207ce9-30c8-4b7d-bfbf-abe56cf50ef4&u=https%3A%2F%2Fwww.tessares.net%2Fmail-disclaimer%2F>