Subject: [multipathtcp] New draft: draft-scharf-mptcp-mctcp-00

From: "SCHARF, Michael" <[email-masked]>
Date: Thu, 1 Jul 2010 18:07:25 +0200

Dear all,

this draft describes an alternative approach that somehow combines
option and TLV-based payload encoding. It does not use TLV-encoding in
the single-path case.

To my understanding, the advantage of using multiple connections is that
one can implement this with some minor changes in the TCP stack, in
combination with a corresponding user-space library that is transparent
to applications. This is actually quite simple and straightforward; I do
have running MCTCP code for Linux kernel 2.6.32.

Of course, any feedback would be welcome

Michael

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of
[email-masked]
Sent: Thursday, July 01, 2010 5:45 PM
To: [email-masked]
Subject: I-D Action:draft-scharf-mptcp-mctcp-00.txt 

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

	Title           : Multi-Connection TCP (MCTCP) Transport
	Author(s)       : M. Scharf
	Filename        : draft-scharf-mptcp-mctcp-00.txt
	Pages           : 31
	Date            : 2010-07-01

Multipath transport over potentially different paths can be realized by
several coupled Transmission Control Protocol (TCP) connections.
Multi-Connection TCP (MCTCP) transport aggregates multiple TCP
connections between potentially different addresses into a single
session that can be accessed by an application like a single TCP
connection.  MCTCP encodes control information, as far as possible, in
the payload of the TCP connections and therefore requires only minor
changes in the TCP implementations, and it is transparent in the
single-path case.  MCTCP is therefore proposed as a simple, modular, and
extensible mechanism for multipath transport.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-scharf-mptcp-mctcp-00.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 7 Jul 2010 16:55:36 -0700

Hello Michael,

Please correct me if I'm wrong.
I think the main difference between the mctcp and the current mptcp is using
option or payload for their signaling method
Other parts are mostly similar except the usage of the mode option. In my
understanding, we can say even the current mptcp uses multiple connections.
Could you elaborate a bit more about how mctcp architecture can be simpler?

Thanks,
--
Yoshifumi Nishida
[email-masked]

2010/7/1 SCHARF, Michael <[email-masked]>

> _______________________________________________
> multipathtcp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/multipathtcp
>

---

From: "SCHARF, Michael" <[email-masked]>
Date: Thu, 8 Jul 2010 09:25:57 +0200

Yoshifumi,

> Other parts are mostly similar except the usage of the mode option.

to some extend, MCTCP is one solution how to transport MPTCP-like signaling in the payload. But there are several further differences to current MPTCP, e. g., how the initial connection is handled. Apparently, MCTCP also differs to the MPTCP payload encoding variant that Costin's analysis at the last IETF meeting was based on.

>From an architectural point of view, MCTCP is a new (shim) protocol layer on top of individual TCP connections, with a lightweight interface to these connections in order to detect and negotiate its usage and to remain backward compatible in the single path case. Thus, it is modular. To my understanding, MPTCP with option encoding is much more tightly integrated in TCP.

Please note that draft-scharf-mptcp-mctcp-00 has been written to show that MCTCP can provide more or less the same functions like draft-ford-mptcp-multiaddressed-03. As explained in the appendix, a session protocol without the limitations of TCP options could of course have further functionality. But before specifying such extensions, it is IMHO worth discussing whether the overall approach of the baseline protocol is promising.

> In my understanding, we can say even the current mptcp uses multiple connections.

The advantages (and also potential drawbacks) of MCTCP are discussed in section 3.3. Here is a brief summary:

- Architecture: MCTCP results in a clear separation of functions between TCP and the handling of multiple paths. MCTCP hardly interacts with the data transport inside a TCP connection. One can also simply add further MCTCP functions in the future by just defining additional optional TLV messages (e. g., MCTCP version 2.0).

- Deployment: For an initial support of MCTCP, one only needs a very small change in the TCP stack, which is pretty straightforward (at least in Linux). The deployment of new protocols in the user-space is typically simpler, given that production network TCP stacks are highly optimized.

- Issues of sporadic TCP options: MCTCP avoids all complexity of sporadically added TCP options (e. g., support by TCP offloading cards, variable MSS). As the transport of all control information is protected by TCP, there is no need to worry about potentially lost control information due to dropped packets. IMHO this simplifies the protocol design.

- Middleboxes: I don't see much issues of MCTCP with potential middleboxes that mess up TCP options. Of course, modifications of the byte stream are still an issue, but one can easily detect content modifications with a checksum after each data chunk, which is pretty straightforward.

As mentioned in the draft, MCTCP also comes at some cost. But the MCTCP protocol design actually just follows the KISS principle.

Does this answer your question?

Michael

---

From: Yoshifumi Nishida <[email-masked]>
Date: Fri, 9 Jul 2010 01:06:56 -0700

Hi Michael,

Thanks for your response.
I've been thinking a bit, but I still feel I missed something. Please let me
explain.
If I were an implementer, probably I would create a new layer on top of
individual TCP connections for both mptcp and mctcp.
The difference would be the interface between the new layer and tcp layers.
In case of mctcp, I would establish one channel between mctcp and individual
tcp layer to send/receive TLV data.
In case of mptcp, I would establish two channels between mptcp and
individual tcp layer to send/receive byte stream and control info in the
options. There might be a synchronizing issue in mptcp approach, however,
the current protocol does not require tight synchronization between data and
control info. So, I think it will be ok.
If I think this way, both approach seems to have the same level of
implementation complexity.

Thanks,
--
Yoshifumi Nishida
[email-masked]

2010/7/8 SCHARF, Michael <[email-masked]>

> to some extend, MCTCP is one solution how to transport MPTCP-like signaling
> in the payload. But there are several further differences to current MPTCP,
> e. g., how the initial connection is handled. Apparently, MCTCP also differs
> to the MPTCP payload encoding variant that Costin's analysis at the last
> IETF meeting was based on.
>
> From an architectural point of view, MCTCP is a new (shim) protocol layer
> on top of individual TCP connections, with a lightweight interface to these
> connections in order to detect and negotiate its usage and to remain
> backward compatible in the single path case. Thus, it is modular. To my
> understanding, MPTCP with option encoding is much more tightly integrated in
> TCP.
>
> Please note that draft-scharf-mptcp-mctcp-00 has been written to show that
> MCTCP can provide more or less the same functions like
> draft-ford-mptcp-multiaddressed-03. As explained in the appendix, a session
> protocol without the limitations of TCP options could of course have further
> functionality. But before specifying such extensions, it is IMHO worth
> discussing whether the overall approach of the baseline protocol is
> promising.
>
>
> - Architecture: MCTCP results in a clear separation of functions between
> TCP and the handling of multiple paths. MCTCP hardly interacts with the data
> transport inside a TCP connection. One can also simply add further MCTCP
> functions in the future by just defining additional optional TLV messages
> (e. g., MCTCP version 2.0).
>
> - Deployment: For an initial support of MCTCP, one only needs a very small
> change in the TCP stack, which is pretty straightforward (at least in
> Linux). The deployment of new protocols in the user-space is typically
> simpler, given that production network TCP stacks are highly optimized.
>
> - Issues of sporadic TCP options: MCTCP avoids all complexity of
> sporadically added TCP options (e. g., support by TCP offloading cards,
> variable MSS). As the transport of all control information is protected by
> TCP, there is no need to worry about potentially lost control information
> due to dropped packets. IMHO this simplifies the protocol design.
>
> - Middleboxes: I don't see much issues of MCTCP with potential middleboxes
> that mess up TCP options. Of course, modifications of the byte stream are
> still an issue, but one can easily detect content modifications with a
> checksum after each data chunk, which is pretty straightforward.
>
> As mentioned in the draft, MCTCP also comes at some cost. But the MCTCP
> protocol design actually just follows the KISS principle.
>

---

From: "SCHARF, Michael" <[email-masked]>
Date: Fri, 9 Jul 2010 11:40:20 +0200

Hi Yoshifumi, 

> If I were an implementer, probably I would create a new layer 
> on top of individual TCP connections for both mptcp and mctcp.

I totally agree to this - this is how I currently implement MCTCP. And,
in that case TLV encoding actually makes a lot of sense, since you have
an additional protocol layer...

> The difference would be the interface between the new layer 

Yes, the interface would indeed be different. But there is also a
difference in the amount of support you need from the TCP stack itself.

> In case of mctcp, I would establish one channel between mctcp 
> and individual tcp layer to send/receive TLV data.

True. For instance, one can just use the standard socket interface for
that, plus some few additional socket options.

> In case of mptcp, I would establish two channels between 
> mptcp and individual tcp layer to send/receive byte stream 
> and control info in the options. There might be a 
> synchronizing issue in mptcp approach, however, the current 
> protocol does not require tight synchronization between data 
> and control info. So, I think it will be ok.

I have not looked into such a solution for option encoding so far, but
it would definitively be interesting if someone wrote a proof-of-concept
for such a combination of kernel and user-space implementation, to
compare the complexity.

However, I do see at least two additional problems: First, you have to
pass the information from the second channel through the segmentation
and reassembly engines of TCP, which is not necessarily trivial (both at
sender and receiver). Synchronization could indeed be tricky.

Second, you need functions to add the corresponding TCP options, which
raises issues with variable MSS sizes, TCP offloading etc. I'd
personally send the options in segments without payload to work around
these issues, but then the overhead increases and one can raise the
question whether those packets have to be congestion- and
flow-controlled, in particular if they are sent frequently.

> If I think this way, both approach seems to have the same 
> level of implementation complexity. 

I am not sure if I agree to that conclusion. In addition, there is also
a question of implementation complexity in the parts that are not
necessarily implemented in the kernel. Checksumming, segmentation, and
reassembly of TLV messages is IMHO straightforward. I have the
impression that reassembly and checksumming is more difficult in case of
MPTCP with option encoding, as you might not necessarily have  valid
data sequence mappings if the packets with these options got somehow
lost, i. e., you have to consider awaiting retranmissions, etc.

Michael

---

From: <[email-masked]>
Date: Fri, 9 Jul 2010 16:06:44 +0100

Michael,
In your draft, thanks for S3.3, where you identify some differences compared with draft-ietf-mptcp-architecture. This is very useful and I think it would be good to have some discussion about this before our ietf meeting, as we havent been given much meeting time.

It would be good to identify what are fundamental differences, and what are related to a detail of the protocol (or implementation) that could be changed. The Options vs Payload decision should be about the first (or mainly so).

In the first category, I think:
- less use of TCP options
- ability to exchange a list of candidate IPv6 addresses or even URI

Whereas I think the others are in the second category?
- eg robustness in face of middleboxes that drop, modify etc TCO options. This is similar, now that the Options approach uses a checksum.

Thanks
phil

> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of SCHARF, Michael
> Sent: 01 July 2010 17:07
> To: [email-masked]
> Subject: [multipathtcp] New draft: draft-scharf-mptcp-mctcp-00
>
>
> Dear all,
>
> this draft describes an alternative approach that somehow
> combines option and TLV-based payload encoding. It does not
> use TLV-encoding in the single-path case.
>
> To my understanding, the advantage of using multiple
> connections is that one can implement this with some minor
> changes in the TCP stack, in combination with a corresponding
> user-space library that is transparent to applications. This
> is actually quite simple and straightforward; I do have
> running MCTCP code for Linux kernel 2.6.32.
>
> [mailto:[email-masked]] On > Behalf Of
> [email-masked]
> A New Internet-Draft is available from the on-line
> Internet-Drafts directories.
>
> Multipath transport over potentially different paths can be
> realized by several coupled Transmission Control Protocol
> (TCP) connections. Multi-Connection TCP (MCTCP) transport
> aggregates multiple TCP connections between potentially
> different addresses into a single session that can be
> accessed by an application like a single TCP connection.
> MCTCP encodes control information, as far as possible, in the
> payload of the TCP connections and therefore requires only
> minor changes in the TCP implementations, and it is
> transparent in the single-path case.  MCTCP is therefore
> proposed as a simple, modular, and extensible mechanism for
> multipath transport.
>

Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. _______________________________________________
multipathtcp mailing list
[email-masked] https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: <[email-masked]>
Date: Fri, 9 Jul 2010 16:14:56 +0100

Michael,

As I understand it:
1. you start with an ordinary TCP connection, with Multipath capable option
2. later you send a Syn + Join on the same path. The ordinary TCP connection is no longer used (is killed?]. Multipath TCP is now in operation, although at this stage only one path is used
3. you send further Syn + Join on other paths, which are added to the first path.

I wasn't quite clear on a couple of points:
- if one of the paths is lost, how is data sent over another path?
- if the first path dies [ie step 2], what happens?

Thanks
phil

---

From: "SCHARF, Michael" <[email-masked]>
Date: Fri, 9 Jul 2010 18:20:52 +0200

Philip, 

> 1. you start with an ordinary TCP connection, with Multipath 
> capable option 2. later you send a Syn + Join on the same 
> path.

true. Note that MCTCP could in principle use another path as well for
the first coupled connection, if the address is already known by some
means (e. g., caching from previous connections). But, for several
reasons, it seems to be more reasonable to use the same path first,
since the initiator knows that it is operational.

> The ordinary TCP connection is no longer used (is 
> killed?].

The spec at the moment mandates to keep the initial connection open.
There are two reasons for this:
- Allowing a fall-back, at least immediately after the setup of the
coupled connections
- Exposure of valid addresses by getpeername() and getsockname(), i. e.,
addresses that are not used by another connection.

If we close the initial connection but still continue to expose the
addresses by getpeername() and getsockname(), an application might see
two different connections with exactly the same addresses. It is
probably better to avoid this and keep the connection established to
"reserve" the addresses (possibly using keepalives).

> Multipath TCP is now in operation, although at this 
> stage only one path is used 3. you send further Syn + Join on 
> other paths, which are added to the first path.

Yes. And, of course, it is one of the obvious drawbacks of MCTCP that it
needs in total 3 connections for the multipath usage of 2 paths.

We could also use the initial connection for TLV transport to work
around this, but this would make the whole approach more complex, and we
would loose the fall-back option.

> - if one of the paths is lost, how is data sent over another path?

The MCTCP sender just retransmits the data on another path by assembling
new TLV messages. It could either try to determine the missing data from
the amount of unacked data in the socket for that path, or await an
MCTCP NACK message from the receiver indicating gaps in the received
global sequence number space (this message is not specified yet) - or
any other reliability mechanism, including a data ACK+retransmission as
ultima ratio. Of course, the MCTCP sender must still have the data
buffered in order to do so, and the retransmission has to wait until the
sending queue on the alternative path drains, at least if MCTCP is
realized in the user space.

> - if the first path dies [ie step 2], what happens?

In this case all multipath variants have a problem anyway, as one
exposes invalid addresses to legacy apps (cf. draft-scharf-mptcp-api).
Currently I'd suggest to close the MCTCP session in that case, which is
in line with opinions in the working group in the past. Keeping the
MCTCP session open should be possible too, but then the exact closing
procedure of the MCTCP session would have to be better defined.

Do you see an open issue here?

Michael