Subject: [multipathtcp] mptcp quick Q: MP-TCP for address redundancy

From: Toerless Eckert <[email-masked]>
Date: Thu, 11 Jun 2020 18:29:04 +0200

Could not easily find the answer to the following question:

Can i effectively use MP-TCP just to overcome possible loss of interfaces ?

E.g.: server and client each have two interfaces, connection is established
from client interface 1 address to server interface 1 address, the
additional addresses are exchanged via MP-TCP. Sometime during life-time
of both client interface 1 and server interface 1 fail in short order.

Q1: Can MP-TCP survive that ? That being general case of N addresses
on client/server and N-1 address failures simulateneously.

Q2: If so, does it require full-mesh of subflows or any other setup
of subflows ?

Thanks!
    Toerless

---

From: Olivier Bonaventure <[email-masked]>
Date: Thu, 11 Jun 2020 18:41:00 +0200

Hello,

> of both client interface 1 and server interface 1 fail in short order.

Yes, this works.

> on client/server and N-1 address failures simulateneously.

Since MPTCP exchanges the different addresses, the client can try them 
in sequence to reach the server through the available one.

> Q2: If so, does it require full-mesh of subflows or any other setup

A full-mesh would open all subflows. You might need a reactive path 
manager that reacts to changes in subflows to recreate new ones, see 
e.g. chapter 7 of Viet Honag Tran thesis

https://inl.info.ucl.ac.be/publications/measuring-and-extending-multipath-tcp.html

Olivier

---

From: Matthieu Baerts <[email-masked]>
Date: Thu, 11 Jun 2020 18:53:23 +0200

Hello,

> https://inl.info.ucl.ac.be/publications/measuring-and-extending-multipath-tcp.html

If it is not an issue for you to open subflows but not using them for 
some interfaces, subflows can have the "backup" flag set.

With the out-of-tree Linux kernel, "backup" subflows will only be used 
if there is no other "non-backup" subflow. Maybe this could help you.

http://multipath-tcp.org/pmwiki.php/Users/Tools

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 11 Jun 2020 19:48:10 +0200

Thanks, Olivier

I am trying to understand what overhead and delay in recovery incurred
when i am using MP-TCP not for what i think it was designed for,
but just to deal with address redundancy in IP.

Aka: Imagine two routers each with N interfaces, but
only one active IGP calculated path between them. In practice,
one always uses a loopback interface address and never the
N physical interface addresses. But that is just operational
practice, i have not seen a comparison in actual RFC text.

Aka: instead of the loopback i do use the N interface addresses,
how best would i characterize the redundancy (speed of recovery
from actively used interface loss) and overhead of MP-TCP.

I guess i can try to go with

"higher complexity and per-connection state
due to handling N^2 addresses and potentially slower recovery
loss of actively used failing remote peer addresses,
and/or overhead of maintaining as many as N^2
sub-flows to minimize loss of any address" ?

Would that be a fair statement - specifically of course for the
case when it is known that there is only one path.

Cheers
    Toerless

On Thu, Jun 11, 2020 at 06:41:00PM +0200, Olivier Bonaventure wrote:
> Since MPTCP exchanges the different addresses, the client can try them in
> sequence to reach the server through the available one.
> 
> A full-mesh would open all subflows. You might need a reactive path manager
> that reacts to changes in subflows to recreate new ones, see e.g. chapter 7
> of Viet Honag Tran thesis
>

---

From: Olivier Bonaventure <[email-masked]>
Date: Thu, 11 Jun 2020 21:12:22 +0200

Hello,
> but just to deal with address redundancy in IP.

This mainly depends on how the path manager is tuned. With MPTCP, you 
can do make before break or break before make. As mentionned by Mathieu, 
you can also use the backup bit to only use one primary subflow and 
leave the other as backups.

> practice, i have not seen a comparison in actual RFC text.

Taking the router example, one could run iBGP sessions over MPTCP. The 
session would be established to one of the IP addresses of the routers 
and the others would be learned automatically, enabling a failover in 
case of failure of the initial one. The BGP session could initiate all 
subflows and select the one with the lowest rtt.

> from actively used interface loss) and overhead of MP-TCP.

If there is a second subflow, MPTCP can typically recover from a failure 
within one or a few rtts since it checks for an alternate subflow when 
it retransmits data. This can be tuned and some have proposed schedulers 
that duplicate packets.

> sub-flows to minimize loss of any address" ?

MPTCP does not need to maintain all the N² subflows. It can decide to 
only create a few subflows, say 3 or 4 over very diverse addresses. 
That's a path manager decision.

With MPTCP, once an address fails, it is announced over one subflow 
using rm_addr and all the subflows that use this address fail immediately.

> case when it is known that there is only one path.

It is possible to tune MPTCP depending on the use case that you have. If 
you want fast failover, it is possible to create some additional 
subflows, use a redundant scheduler to reduce the failover time at the 
expense of more overhead

Olivier

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 11 Jun 2020 21:58:23 +0200

Thanks, Olivier,

Not quite sure from your reply what applies to non-backup vs. backup subflows,
you did not say.

Also: Is it necessary to avoid loosing all sub-flows, or does
that only speed up recovery ?

Aka: If i start with one subflow, learn additional addresses, but
not create additional subflows (just announce addresses), and
now one of the peer has an interface failure and this only
sub-flow fails. Would it then still be able to create a new 
subflow from a different address ?

Sorry, even to me these questions sound 101, but really hard to read 
asnwers from the spec to the uninitiated...

Cheers
    Toerless

On Thu, Jun 11, 2020 at 09:12:22PM +0200, Olivier Bonaventure wrote:

> This mainly depends on how the path manager is tuned. With MPTCP, you can do
> make before break or break before make. As mentionned by Mathieu, you can
> also use the backup bit to only use one primary subflow and leave the other
> as backups.
>
> session would be established to one of the IP addresses of the routers and
> the others would be learned automatically, enabling a failover in case of
> failure of the initial one. The BGP session could initiate all subflows and
> select the one with the lowest rtt.
>
> within one or a few rtts since it checks for an alternate subflow when it
> retransmits data. This can be tuned and some have proposed schedulers that
> duplicate packets.
>
> MPTCP does not need to maintain all the N² subflows. It can decide to only
> create a few subflows, say 3 or 4 over very diverse addresses. That's a path
> manager decision.
> 
> With MPTCP, once an address fails, it is announced over one subflow using
> rm_addr and all the subflows that use this address fail immediately.
> 
> It is possible to tune MPTCP depending on the use case that you have. If you
> want fast failover, it is possible to create some additional subflows, use a
> redundant scheduler to reduce the failover time at the expense of more
> overhead
> 
> 
> Olivier

-- 
---
[email-masked]

---

From: Olivier Bonaventure <[email-masked]>
Date: Thu, 11 Jun 2020 22:19:17 +0200

Toerless,
> Not quite sure from your reply what applies to non-backup vs. backup subflows,

MPTCP only uses a backup subflow if there is no available non-backup 
subflow.

> that only speed up recovery ?

That speeds up the recovery but is not required. MPTCP supports both 
make before break and break before make.
> subflow from a different address ?

Yes. Subflows can be created at any time from any local address to any 
of the addresses learned from the peer. Note that the client does not 
need to advertise its addresses to be able to use them to reach a 
server. For example, a smartphone has only Wifi enabled and connects to 
a server. It leaves out of reach of the Wifi, the subflow fails. It then 
enables its cellular interface, obtains an IP address and use it to 
create a new subflow to the server.
> asnwers from the spec to the uninitiated...

There are a few tutorials on MPTCP where this is partially discussed:

https://www.usenix.org/publications/login/october-2012-volume-37-number-5/overview-multipath-tcp
https://dl.acm.org/doi/10.1145/2578508.2591369
http://www.sigcomm.org/content/ebook (chapter 2)
https://www.youtube.com/watch?v=PkvLq_kCv4o

Hope this helps

Olivier