Subject: [tcpm] comments on draft-kang-tcpm-accurate-data-scheduling-by-server-00

From: Yoshifumi Nishida <[email-masked]>
Date: Sun, 19 Jul 2020 04:26:28 -0700

Hi,
I've read draft-kang-tcpm-accurate-data-scheduling-by-server-00.
I think this is an interesting topic for mptcp, but I think it would be
better to clarify the following points.

1: One thing I'm not very clear is why we cannot use MP_PRIO for the use
cases described in the draft. I believe the draft should describe the cases
where existing features cannot fulfill the requirements more specifically.

2: Clients also have their own constraints. (e.g. policy or routing) So,
even though servers send a navigation request, they might not follow it. I
think this point should be clarified.

3: What's the meaning of 'r', 'E', 'B' flags in Section 4.1?

Thanks,
--
Yoshi

---

From: Kangjiao <[email-masked]>
Date: Mon, 20 Jul 2020 03:54:09 +0000

Hi Yoshi,

Thanks for your suggestions. We clarify the issues as below:

1: One thing I'm not very clear is why we cannot use MP_PRIO for the use cases described in the draft. I believe the draft should describe the cases where existing features cannot fulfill the requirements more specifically.

KJ: The new MP_Navigation Option is used for the server to indicate destination network interface to client for which server wants to use for traffic switching. For my understanding, MP_PRIO is used to signal a change in priority of subflows to the peer. In application, MP_PRIO can reduce the chance of data transmission on a specific subflow but it cannot tell its peer which network interface is the destination from server side. For example, if there are multiple subflows with high priority from difference network interfaces, client receiving MP_PRIO does not know which is the target one.

2: Clients also have their own constraints. (e.g. policy or routing) So, even though servers send a navigation request, they might not follow it. I think this point should be clarified.

KJ: If the mechanism of accurate-data-scheduling-by-server is deployed, the principle is that the server takes precedence.

KJ:  For the protocol design, the definition of ’r’, ‘E’ and ’B’ are as following:
Flag ‘r’: reserved for future usage.
Flag ‘E’: exists to provide reliability for this option (like that in ”ADD_ADDR”).
Flag ’B’: indicates whether the subflow over which the option is received is a backup one (that is compatiable with the value by MP_PRIO).

But we are thinking whether these fields are necessary and should be set as mandatory.

Sincerely,
Jiao
From: tcpm [mailto:[email-masked]] On Behalf Of Yoshifumi Nishida
Sent: Sunday, July 19, 2020 7:26 PM
To: [email-masked] Extensions <[email-masked]>
Subject: [tcpm] comments on draft-kang-tcpm-accurate-data-scheduling-by-server-00

Hi,
I've read draft-kang-tcpm-accurate-data-scheduling-by-server-00.
I think this is an interesting topic for mptcp, but I think it would be better to clarify the following points.

1: One thing I'm not very clear is why we cannot use MP_PRIO for the use cases described in the draft. I believe the draft should describe the cases where existing features cannot fulfill the requirements more specifically.

2: Clients also have their own constraints. (e.g. policy or routing) So, even though servers send a navigation request, they might not follow it. I think this point should be clarified.

Thanks,
--
Yoshi

---

From: Christoph Paasch <[email-masked]>
Date: Tue, 21 Jul 2020 08:59:45 -0700

Hello,

as an overall comment, I agree that one of the difficulties around multipath
is how the hosts can communicate to the peers the desired scheduling
mechanisms. Currently there is no reliable way besides backup-bits which is
a "hard" switch. However, often scheduling is much more fine-grained as
shown in a number of the schedulers described in
https://tools.ietf.org/html/draft-bonaventure-iccrg-schedulers-00.

It's a bit similar to TCP which can't pick the peer's congestion-control and
for example if one wants the peer to use a background-CC the host needs to
resort to tuning the receive-window a la draft-bagnulo-iccrg-rledbat-00.

So, having a way to communicate the scheduling "requirements" to the peer
(and let the peer match these requirements with its local policies), would
be good!

One more comment inline:

On 07/20/20 - 03:54, Kangjiao wrote:
> cases described in the draft. I believe the draft should describe the
> cases where existing features cannot fulfill the requirements more
> specifically.
> 
> KJ: The new MP_Navigation Option is used for the server to indicate
> destination network interface to client for which server wants to use for
> traffic switching. For my understanding, MP_PRIO is used to signal a
> change in priority of subflows to the peer. In application, MP_PRIO can
> reduce the chance of data transmission on a specific subflow but it cannot
> tell its peer which network interface is the destination from server side.
> For example, if there are multiple subflows with high priority from
> difference network interfaces, client receiving MP_PRIO does not know
> which is the target one.

For this particular scenario the server could send the MP_PRIO with backup-bit
set on all subflows that the client should not use. Thus implicitly the
high-priority network-interfaces can then be used by the client, right?

Christoph

> KJ: If the mechanism of accurate-data-scheduling-by-server is deployed,
> the principle is that the server takes precedence.
> 
> KJ:  For the protocol design, the definition of ’r’, ‘E’ and ’B’ are as
> following: Flag ‘r’: reserved for future usage.  Flag ‘E’: exists to
> provide reliability for this option (like that in ”ADD_ADDR”).  Flag ’B’:
> indicates whether the subflow over which the option is received is a
> backup one (that is compatiable with the value by MP_PRIO).
> 
> Sincerely, Jiao From: tcpm [mailto:[email-masked]] On Behalf Of
> Yoshifumi Nishida Sent: Sunday, July 19, 2020 7:26 PM To: [email-masked]
> Extensions <[email-masked]> Subject: [tcpm] comments on
> draft-kang-tcpm-accurate-data-scheduling-by-server-00
> 
> Hi, I've read draft-kang-tcpm-accurate-data-scheduling-by-server-00.  I
> think this is an interesting topic for mptcp, but I think it would be
> cases described in the draft. I believe the draft should describe the
> cases where existing features cannot fulfill the requirements more
> specifically.
> 
> Thanks, -- Yoshi

> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 22 Jul 2020 00:21:25 -0700

Hi Kangjiao,

Thanks for the response.
I put my comments in lines.

On Sun, Jul 19, 2020 at 8:54 PM Kangjiao <[email-masked]> wrote:

> traffic switching. For my understanding, MP_PRIO is used to signal a change
> in priority of subflows to the peer. In application, MP_PRIO can reduce the
> chance of data transmission on a specific subflow but it cannot tell its
> peer which network interface is the destination from server side. For
> example, if there are multiple subflows with high priority from difference
> network interfaces, client receiving MP_PRIO does not know which is the
> target one.
>

I think It would be better if you can show more concrete examples for the
scenario in the draft.
I would like to use the example in Section 3.2 of the draft for this. (BTW,
I personally prefer to use IP_C1 or IP_S1 or C1, S1 rather than IP1, IP2 so
that it will be easy to understand which address belongs to client or
server)
If the server sends MP_PRIO with B flag to the client by using <IP1, IP3>
and <IP2, IP3>, I think the client will send traffic to IP4. Isn't this
what the server wants?

> the principle is that the server takes precedence.

Well, let's say the server wants to use 5G because it's beneficial for
them, but the client doesn't want to use it because it's costly.
The client still should follow the server's request?

> following:
> Flag ’B’: indicates whether the subflow over which the option is received
> is a backup one (that is compatiable with the value by MP_PRIO).
>
>
>

Got it. Thanks for clarification. I think you'd better put these info into
the draft later.

Thanks,
--
Yoshi

---

From: Kangjiao <[email-masked]>
Date: Thu, 23 Jul 2020 02:00:41 +0000

Hi Christoph and Yoshi,

If I understand right, you both point to the same question whether reusing the MP_PRIO can finish accurate data scheduling for server. I think MP_PRIO may be not enough even if setting all the "non-target" subflows to backup ones. I can give a scenario:

Six subflows have been established between client and server that are <IP_C1, IP_S1>, <IP_C2, IP_S1>, <IP_C1, IP_S2>, <IP_C2, IP_S2>, <IP_C1, IP_S3> and <IP_C2, IP_S3>. The client has IP_C1 and IP_C2. The server has IP_S1, IP_S2 and IP_S3.

Server tool detects that KPI of IP_S2 is better now so the server wants to switch data traffic from <IP_C1, IP_S1> to the destination IP_S2. If we set <IP_C1, IP_S1> as "backup", it is not guaranteed that all the load over <IP_C1, IP_S1> can be diverted to IP_S2 because of the ongoing suflows towards IP_S3.

So I think MP_PRIO can reduce the chance of data transmission on a specific subflow but it cannot convey the info of target network interface from server to client.

Thanks
Jiao

-----Original Message-----
From: Christoph Paasch [mailto:[email-masked]] 
Sent: Wednesday, July 22, 2020 12:00 AM
To: Kangjiao <[email-masked]>
Cc: Yoshifumi Nishida <[email-masked]>; [email-masked] Extensions <[email-masked]>; Liangqiandeng <[email-masked]>
Subject: Re: [tcpm] comments on draft-kang-tcpm-accurate-data-scheduling-by-server-00

Hello,

as an overall comment, I agree that one of the difficulties around multipath is how the hosts can communicate to the peers the desired scheduling mechanisms. Currently there is no reliable way besides backup-bits which is a "hard" switch. However, often scheduling is much more fine-grained as shown in a number of the schedulers described in https://tools.ietf.org/html/draft-bonaventure-iccrg-schedulers-00.

It's a bit similar to TCP which can't pick the peer's congestion-control and for example if one wants the peer to use a background-CC the host needs to resort to tuning the receive-window a la draft-bagnulo-iccrg-rledbat-00.

So, having a way to communicate the scheduling "requirements" to the peer (and let the peer match these requirements with its local policies), would be good!

One more comment inline:

On 07/20/20 - 03:54, Kangjiao wrote:
> 1: One thing I'm not very clear is why we cannot use MP_PRIO for the 
> use cases described in the draft. I believe the draft should describe 
> the cases where existing features cannot fulfill the requirements more 
> specifically.
> 
> destination network interface to client for which server wants to use 
> for traffic switching. For my understanding, MP_PRIO is used to signal 
> a change in priority of subflows to the peer. In application, MP_PRIO 
> can reduce the chance of data transmission on a specific subflow but 
> it cannot tell its peer which network interface is the destination from server side.

For this particular scenario the server could send the MP_PRIO with backup-bit set on all subflows that the client should not use. Thus implicitly the high-priority network-interfaces can then be used by the client, right?

Christoph

> 2: Clients also have their own constraints. (e.g. policy or routing) 
> So, even though servers send a navigation request, they might not 
> follow it. I think this point should be clarified.
> 
> KJ: If the mechanism of accurate-data-scheduling-by-server is 
> deployed, the principle is that the server takes precedence.
> 
> Yoshifumi Nishida Sent: Sunday, July 19, 2020 7:26 PM To: 
> [email-masked] Extensions <[email-masked]> Subject: [tcpm] comments on
> Hi, I've read draft-kang-tcpm-accurate-data-scheduling-by-server-00.  
> I think this is an interesting topic for mptcp, but I think it would 
> be better to clarify the following points.
> 
> 1: One thing I'm not very clear is why we cannot use MP_PRIO for the 
> use cases described in the draft. I believe the draft should describe 
> the cases where existing features cannot fulfill the requirements more 
> specifically.
> 
> 2: Clients also have their own constraints. (e.g. policy or routing) 
> So, even though servers send a navigation request, they might not 
> follow it. I think this point should be clarified.
>

---

From: Christoph Paasch <[email-masked]>
Date: Wed, 22 Jul 2020 20:10:41 -0700

Hello,

> On Jul 22, 2020, at 7:01 PM, Kangjiao <[email-masked]> wrote:
> 
> Hi Christoph and Yoshi,
> 

But we could simply also set <IP_C1,IP_S3> ans <IP_C2,IP_S3> as backup as well, no?

Now, all the traffic will be steered towards S2.

Christoph

---

From: Kangjiao <[email-masked]>
Date: Thu, 23 Jul 2020 04:38:31 +0000

Hi Christoph,

From my perspective, I think following are two different requirements:
 
1. Making all the traffic towards S2 (for this case, MP_PRIO may be enough)

2. Making traffic only from <IP_C1, IP_S1> to the destination IP_S2, which means diverting traffic that should be scheduled to <IP_C1, IP_S1> to one subflow belonging to the target network interface (it is "on-demand scheduling", for this case, MP_PRIO is not enough)

What do you think?

Sincerely
Jiao

-----Original Message-----
From: Christoph Paasch [mailto:[email-masked]] 
Sent: Thursday, July 23, 2020 11:11 AM
To: Kangjiao <[email-masked]>
Cc: Yoshifumi Nishida <[email-masked]>; [email-masked] Extensions <[email-masked]>; Liangqiandeng <[email-masked]>
Subject: Re: [tcpm] comments on draft-kang-tcpm-accurate-data-scheduling-by-server-00

Hello,

Christoph

> Cc: Yoshifumi Nishida <[email-masked]>; [email-masked] Extensions 
> <[email-masked]>; Liangqiandeng <[email-masked]>
> Subject: Re: [tcpm] comments on 
>> the cases where existing features cannot fulfill the requirements 
>> more specifically.
>> 
>> for traffic switching. For my understanding, MP_PRIO is used to 
>> signal a change in priority of subflows to the peer. In application, 
>> MP_PRIO can reduce the chance of data transmission on a specific 
>> subflow but it cannot tell its peer which network interface is the destination from server side.
>> Sincerely, Jiao From: tcpm [mailto:[email-masked]] On Behalf 
>> Of Yoshifumi Nishida Sent: Sunday, July 19, 2020 7:26 PM To:
>> the cases where existing features cannot fulfill the requirements 
>> more specifically.
>>

---

From: Christoph Paasch <[email-masked]>
Date: Thu, 23 Jul 2020 07:40:58 -0700

> 2. Making traffic only from <IP_C1, IP_S1> to the destination IP_S2, which means diverting traffic that should be scheduled to <IP_C1, IP_S1> to one subflow belonging to the target network interfac…

What is the use-case for such a scheme?

I don't see how a multipath scheduler could assign the traffic that "would"
have been sent to <C1,S1> to a different subflow. Because for that he would
need to know what *would* have been sent on <C1,S1> which he can only know
once he actually sends traffic on that subflow as that will be driving the
congestion-window which will allow to send more (or less) traffic on <C1,S1>.

Christoph

---

From: Olivier Bonaventure <[email-masked]>
Date: Fri, 24 Jul 2020 16:16:42 +0200

Hello,

> What is the use-case for such a scheme?

Understanding the requirements from the applications would really help
in this discussion. This could also serve as input for Multipath QUIC
where the middlebox problems that are mentioned in the draft do not
exist.

> congestion-window which will allow to send more (or less) traffic on <C1,S1>.

Another point that needs to be taken into account is that when client
moves, the network conditions change dynamically and the server view
might differ from the wireless conditions on the client. In this case,
it is difficult for the server to precisely decide where it wants to
receive data. See http://multipath-tcp.org/multimob/

Olivier

-- 

Disclaimer: https://www.tessares.net/mail-disclaimer/ 
<https://www.tessares.net/mail-disclaimer/>

---

From: Kangjiao <[email-masked]>
Date: Sat, 25 Jul 2020 15:18:28 +0000

Hi,

The scenarios in this draft are not for traffic handover when encountering network failure. It is applied when one peer has an explicit plan on traffic switching. For example, for network migration or expansion on server, operator wants to adjust the traffic on these networks in advance. This is an actual case.

We think packet scheduler knows on which subflow the next packet will be sent. So it may be a good choice to deploy the logic of accurate traffic switching on this component.

Sincerely,
Jiao

-----Original Message-----
From: Olivier Bonaventure [mailto:[email-masked]] 
Sent: Friday, July 24, 2020 10:17 PM
To: Christoph Paasch <cpaasch=[email-masked]>
Cc: Kangjiao <[email-masked]>; [email-masked] Extensions <[email-masked]>; Liangqiandeng <[email-masked]>
Subject: Re: [tcpm] comments on draft-kang-tcpm-accurate-data-scheduling-by-server-00

Hello,

> > 2. Making traffic only from <IP_C1, IP_S1> to the destination IP_S2, 
> > which means diverting traffic that should be scheduled to <IP_C1, 
> > IP_S1> to one subflow belonging to the target network interface (it 
> > is "on-demand scheduling", for this case, MP_PRIO is not enough)
>

Understanding the requirements from the applications would really help in this discussion. This could also serve as input for Multipath QUIC where the middlebox problems that are mentioned in the draft do not exist.

> have been sent to <C1,S1> to a different subflow. Because for that he 
> would need to know what *would* have been sent on <C1,S1> which he can 
> only know once he actually sends traffic on that subflow as that will 
> be driving the congestion-window which will allow to send more (or less) traffic on <C1,S1>.

Another point that needs to be taken into account is that when client moves, the network conditions change dynamically and the server view might differ from the wireless conditions on the client. In this case, it is difficult for the server to precisely decide where it wants to receive data. See http://multipath-tcp.org/multimob/

Olivier

--

---

From: Christoph Paasch <[email-masked]>
Date: Mon, 27 Jul 2020 09:58:22 -0700

On 07/25/20 - 15:18, Kangjiao wrote:
> The scenarios in this draft are not for traffic handover when encountering
> network failure. It is applied when one peer has an explicit plan on
> traffic switching. For example, for network migration or expansion on
> server, operator wants to adjust the traffic on these networks in advance.
> This is an actual case.
> 
> We think packet scheduler knows on which subflow the next packet will be
> sent.
> So it may be a good choice to deploy the logic of accurate traffic
> switching on this component.

I would love to understand more on how one would implement such kind of a
logic. Do you have a pseudo-code example? I have a hard time imagining how the
scheduling estimates the amount of traffic it would have sent on subflow A and
rather sends it on subflow B, because to drive this decision-making it needs
an estimate of RTT, congestion-window,... of subflow A which it only can get
by actually scheduling traffic on A.

While at a macro-level an operator can do its traffic-engineering and decide
what proportion of traffic should be sent on one vs the other path, I don't
see how this translates into the same method at the micro-level where
scheduling happens on a packet-by-packet basis and is hugely driven by
congestion-control and RTT.

Thus, to achieve such traffic-engineering for the operator it looks to me
like it translates rather to adjusting the weights and preferences on the
subflows.

Thanks,
Christoph