Subject: [multipathtcp] Options vs. payload analysis

From: Costin Raiciu <[email-masked]>
Date: Tue, 9 Feb 2010 23:38:25 +0000

Hi,

I attach my analysis of options vs. payload signalling for multipath TCP. This document is in rough shape, but should be useful as grounds for discussion. 
Talk to you tomorrow!

Thanks,
Costin

---

From: Pasi Sarolahti <[email-masked]>
Date: Tue, 9 Feb 2010 17:01:56 -0800

Hi Costin,

On Feb 9, 2010, at 3:38 PM, Costin Raiciu wrote:

> I attach my analysis of options vs. payload signalling for multipath  
> TCP. This document is in rough shape, but should be useful as  
> grounds for discussion.

Nice document.

If MPTCP is designed as a separate layer on top of TCP (as it  
currently is), I think the payload signaling is the right approach,  
like your document in the end of section 4 hints. In this case the TCP  
subflows would just view these as the opaque payload of their point-to- 
point connection (right?). There is one small architectural flaw,  
though: Join would still need to be an option because it is sent with  
SYN (in addition to MPTCP capable, but there needs to be some  
negotiation). I think it would be more elegant if MPTCP signaling was  
done either entirely in the payload or entirely using the options.

It would be good if this TCP+ - layer framing would be specified in a  
generic way, and in a separate document, because there might be other  
future enhancements that would benefit from this capability, too.

- Pasi

---

From: Costin Raiciu <[email-masked]>
Date: Wed, 10 Feb 2010 01:14:02 +0000

Hi Pasi,

You are right: there are no clean "options only" or "payload only" solutions, as far as I can see. Options only is impossible because of the need to signal security information; it seems silly to spread that over 100 packets in options; using the payload makes sense,  Payload only is impossible because of what you just mentioned :). I have no winner in my mind at this point :).

Cheers,
Costin

On 10 Feb 2010, at 01:01, Pasi Sarolahti wrote:

> If MPTCP is designed as a separate layer on top of TCP (as it currently is), I think the payload signaling is the right approach, like your document in the end of section 4 hints. In this case the TCP subflows would just view these as the opaque payload of their point-to-point connection (right?). There is one small architectural flaw, though: Join would still need to be an option because it is sent with SYN (in addition to MPTCP capable, but there needs to be some negotiation). I think it would be more elegant if MPTCP signaling was done either entirely in the payload or entirely using the options.
> 
> It would be good if this TCP+ - layer framing would be specified in a generic way, and in a separate document, because there might be other future enhancements that would benefit from this capability, too.
> 
> - Pasi
>

---

From: Scott Brim <[email-masked]>
Date: Wed, 10 Feb 2010 10:43:02 -0500

I can't tell from past discussion if you are considering carrying MPTCP
control info in its own separate packets (looking like payload to
intermediaries), or carrying both MPTCP control info and application
info in the same packets.  My instinct is for clean separation.  You
still need an indication to MPTCP as to whether the packet is a MPTCP
packet or an application packet.

---

From: "Ford, Alan" <[email-masked]>
Date: Wed, 10 Feb 2010 16:04:59 -0000

The problem is that you cannot guarantee that a middlebox or similar
will not coalesce packets - what may be a separate packet when you send
it may be appended to data en route: to a middlebox a subflow is just a
TCP stream of data.

There's no great reason not to send control information in separate
packets (indeed if it is being instigated separately from the data it
quite possibly will be), but we must not make assumption on the receiver
about such things.

Therefore we will need to delimit control vs data information in the
payload, via e.g. a Type/Length header before the data.

Cheers,
Alan

> -----Original Message-----
> From: [email-masked]
[mailto:[email-masked]] On
> Behalf Of Scott Brim
> Sent: 10 February 2010 15:43
> To: Costin Raiciu
> Cc: [email-masked] List
> Subject: Re: [multipathtcp] Options vs. payload analysis
> 
> _______________________________________________
> multipathtcp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Scott Brim <[email-masked]>
Date: Wed, 10 Feb 2010 11:16:26 -0500

> payload, via e.g. a Type/Length header before the data.

Very good point.  Thanks.

---

From: "SCHARF, Michael" <[email-masked]>
Date: Wed, 10 Feb 2010 19:17:06 +0100

Costin, 

> You are right: there are no clean "options only" or "payload 
> only" solutions, as far as I can see. Options only is 
> impossible because of the need to signal security 
> information; it seems silly to spread that over 100 packets 
> in options; using the payload makes sense,  Payload only is 
> impossible because of what you just mentioned :). I have no 
> winner in my mind at this point :).

one potential drawback of payload encoding, i. e. stateful middleboxes
that loose synchronization, could perhaps be addressed as follows:

A short, maybe even optional TCP option could mark segments that contain
control data (e. g., TLV encoded), possibly with an offset pointer to
the byte stream number where the in-band control data starts (e. g., 4B
in total). 

This information would "help" middleboxes (and maybe also receivers) to
find and parse the relevant control information in the byte stream, but
it still has the flexibility of payload encoding. Furthermore, a
middlebox could then also distinguish MPTCP flows and TCP flows.

Michael

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 10 Feb 2010 19:18:56 +0100

I apologize for falling off the net just as I was trying to make my point.

As you will have noticed, I'm pretty firmly in the camp that believes that it is increasingly likely there will be some boxes on the path between sender and receiver that understand and act on TCP packets ("middleboxes", "PEPs", "WAN optimizers", "application delivery controllers" etc.), merrily coalescing, splitting and just generally re-arranging data packets and their acknowledgement content but preserving "TCP semantics", for any definition of that term that makes Web apps mostly work.

So any scheme that requires data packets as such, or TCP options on such data packets, to go through in order to make MPTCP work would prevent the use of MPTCP on these paths.  (On the other hand, many of these boxes can be (or already are) configured to let specific TCP options through on SYN packets, as long as they don't have to process them.  But don't count on sequence number spaces to be the same on both ends etc.)

We already said that bundle-level ACKs are sent at the discretion of the receivers, i.e. not necessarily in every data packet (but certainly often enough to let the sender manage its buffer space).  So my proposal was to:

1) have bundle-level ACKs in the payload (as this is the only way I can imagine them to survive the above paths and thus enable MPTCP in the performance-enhanced world), but also 

2) provide a simple TCP option that enables sending an out-of-band data ACK on clear paths to solve the worrying head-of-line blocking that payload-based ACKs incur.

Again, it would be at the discretion of the receiver to make use of method 2 (or method 1, for that matter) as often as it sees fit.

Unfortunately, I couldn't hear any discussion resulting from that half of that proposal I was able to make, so I'm looking forward to the minutes.

Gruesse, Carsten