Subject: More details about multipath quic in Alibaba

From: Yanmei Liu <[email-masked]>
Date: Sun, 25 Oct 2020 14:45:29 +0800

Hi everyone,

After we introduced the use cases and requirements of Multipath QUIC
project at Alibaba in the IETF Interim meeting on Oct. 22rd., we have
received many questions from people who are curious about the details in
the working group, so we would like to give more information in this
mailing list.

- Q1: Do we care about cellular cost?
- Sure. Cellular cost is a very important factor that influences our packet
scheduler design. Some of our customers are sensitive about the cellular
cost and some do not, so they need different scheduling policies. For
customers who are sensitive about cellular cost, Alibaba have collaborated
with several mobile carriers, so customers can get enrolled in a special
data plan, which allows them to enjoy unlimited free cellular data when
using apps from Alibaba eco-system(i.e. alibaba traffic costs the user
nothing). For customers who are not sensitive about the data cost, for
example, professional streamers/Vloggers and customers who want to get high
quality services for business collaborations, they can turn on the
multi-path feature if they want.

- Q2: If someone who wants this could go off an cook up an extension and
get experience with it then come back with that experience?
- We completely agree that a large-scale deployment experience is needed to
draw a conclusion and inspire new innovations. Right now, we are doing the
extension described in
https://tools.ietf.org/html/draft-an-multipath-quic-00, which is integrated
in Taobao for short-form video streaming. The feature is already made
available for certain customers in China and we are working to expand the
scope.

- Q3: How to solve the linkability problem in multipath ?
- QUICv1 solves the linkablity problem with Connection ID’s re-negotiation
when connection migration happens. We reuse the mechanism from QUICv1 in
our multipath quic implementation. For each new sub-connection, client and
server need to exchange available unused Connection IDs before a client
tries to create a new sub-connnection. When client creates a new
sub-connection and sends packets on the new path, it uses the new
Connection IDs. As the process of the negotiation is encrypted, it would
not link the packets sent on new path with the previous path.

- Q4: Difference from early multipath QUIC proposals?
- First of all, we really appreciate the work of early proposals such as
[I-D.deconinck-quic-multipath]. However, in our implementation, we’ve found
that two things are extremely important and should impact the protocol
design. First, we build multi-paths based on the concept bi-directional
“sub-connections” in each new path, and find it’s the easiest way to
implement multipath because it readily fits into the nature of both
cellular and wifi links that cover the majority of multi-path applications.
In doing so, you can almost reuse all the design / implementations for
QUICv1 connections. Second,  the multi-path QUIC design needs dynamic
scheduling strategy. As is pointed in the answer to Q1 above, our customers
who want multipath have very different needs, so making the scheduler
dynamic allows us to fulfill the job.

We would like to know what people think about the draft, and all feedbacks
and suggestions are welcome.

Thanks,
Yanmei Liu

---

From: David Schinazi <[email-masked]>
Date: Sun, 25 Oct 2020 12:34:39 -0700

Thanks for these details Yanmei!

A few followup questions inline:

On Sat, Oct 24, 2020 at 11:45 PM Yanmei Liu <[email-masked]> wrote:

> - Sure. Cellular cost is a very important factor that influences our
> packet scheduler design. Some of our customers are sensitive about the
> cellular cost and some do not, so they need different scheduling policies.
> For customers who are sensitive about cellular cost, Alibaba have
> collaborated with several mobile carriers, so customers can get enrolled in
> a special data plan, which allows them to enjoy unlimited free cellular
> data when using apps from Alibaba eco-system(i.e. alibaba traffic costs the
> user nothing). For customers who are not sensitive about the data cost, for

That's really cool. Out of curiosity, how does your application know that
the user has unlimited free cellular data or not?

> - We completely agree that a large-scale deployment experience is needed
> to draw a conclusion and inspire new innovations. Right now, we are doing
> the extension described in
> https://tools.ietf.org/html/draft-an-multipath-quic-00, which is
> integrated in Taobao for short-form video streaming. The feature is already
> made available for certain customers in China and we are working to expand
> the scope.
>

This is great! If by any chance you have the time to run an experiment
between connection migration and full multipath that would be incredibly
valuable.

> not link the packets sent on new path with the previous path.

That's a great plan.

> and suggestions are welcome.

I'd have to implement this to form a full opinion, but I think I like this
design - placing the complexity on top of QUIC instead of inside QUIC
sounds like a great way to simplify implementations.

David

Thanks,

---

From: Yunfei Ma <[email-masked]>
Date: Sun, 25 Oct 2020 19:05:56 -0700

Hi David,

>> multi-path feature if they want.
David: That's really cool. Out of curiosity, how does your application know
that the user has unlimited free cellular data or not?

Answer: We collaborate with mobile carriers to issue a sim card called
AliCard so users who use the sim card automatically get unlimited free
cellular data. For other users, they can decide if they want to turn on
this option.

>> made available for certain customers in China and we are working to expand
David: This is great! If by any chance you have the time to run an
experiment between connection migration and full multipath that would be
incredibly valuable.

Answer: Thanks, it is a really great suggestion. We have been doing some
experiments in several use cases we care about and have got some early
observations. However, to reach a solid conclusion, we would like to
conduct a more thorough study, so It would still take us sometime to
collect more data as well as our customers' feedback, but we would
definitely love to share our findings later on.

>> - QUICv1 solves the linkablity problem with Connection ID’s
>> re-negotiation when connection migration happens. We reuse the mechanism
>> from QUICv1 in our multipath quic implementation. For each new
>> sub-connection, client and server need to exchange available unused
>> Connection IDs before a client tries to create a new sub-connnection. When
>> client creates a new sub-connection and sends packets on the new path, it
>> uses the new Connection IDs. As the process of the negotiation is
>> encrypted, it would not link the packets sent on new path with the previous
>> path.
>>
>
David: That's a great plan.

Answer: Thanks. We would also like to learn if anyone has a simple way of
doing this.

>> We would like to know what people think about the draft, and all
>> feedbacks and suggestions are welcome.
>>
>
David: I'd have to implement this to form a full opinion, but I think I
like this design - placing the complexity on top of QUIC instead of inside
QUIC sounds like a great way to simplify implementations.

Answer: We really appreciate it if you can try and let us know how you
think. To make things simple to understand and easy to use is always an
important goal for us. Given that QUIC will be used for most of the web
traffic, the ease of maintenance is very important for our  day-to-day
operations.

Cheers,
Yunfei