Subject: [multipathtcp] Options Vs Payload

From: Murari Sridharan <[email-masked]>
Date: Tue, 23 Mar 2010 23:06:53 +0000

I haven't closely read the drafts yet but based on a discussion with Jana and Andrew there were some concerns. Below is a summary of why we think payload is the way to go.

-        Payload, after TLS, Integrity protection and works with HTTPS proxies that are trusted intermediaries

-        Options have several problems. It is one thing to have an option to negotiate in the SYN but every data packet, well space is one issue. But there is more.

-        If there is a terminating proxy it will unlikely transfer the options back and forth. Even if it wants to do there can be no assumptions made about packet boundaries, so if packets get concatenated or split how do the options get merged?

-        Implementing it in payload abstracts the protocol that is carrying MPTCP which seems like a good idea architecturally. This allows it to be run over UDP, bare IP, HIP, TLS, ESP whatever.

-        Allows the possibility of implementing MPTCP in user mode

Thanks

---

From: "SCHARF, Michael" <[email-masked]>
Date: Wed, 24 Mar 2010 02:23:53 +0100

> -        Allows the possibility of implementing MPTCP in user mode

I share these concerns. I am also wondering if the option space would still be sufficient if further information exchange between the endpoints would turn out to be useful, as the spec evolves (e. g., in order to exchange subflow scheduling preferences).

Michael

---

From: Murari Sridharan <[email-masked]>
Date: Wed, 24 Mar 2010 21:17:27 +0000

Based on yesterdays discussion I will add one more reason why I think payload is probably the way to go (although I do agree with Tim that its too premature to make that call).

The number of (cheap) devices that look at options is considerably higher than the number of terminating proxies/DPI devices. Even if we can get through today's network as soon as MPTCP traffic percentages increase several devices will try and do something about it. Having it deeper in the packet means several error prone devices will have no chance of screwing up the options data.

Regarding the possibility of implementing this in user mode, Mark did mention that keeping this in app space breaks the coordinated congestion controller. So that point is no longer valid although it does let apps try this out first before convincing kernels to provide native support for MPTCP.

My 2c.

From: SCHARF, Michael [mailto:[email-masked]]
Sent: Tuesday, March 23, 2010 6:24 PM
To: Murari Sridharan; [email-masked]
Cc: [email-masked]
Subject: Re: [multipathtcp] Options Vs Payload

Michael

---

From: Scott Brim <[email-masked]>
Date: Wed, 24 Mar 2010 14:47:09 -0700

Murari Sridharan allegedly wrote on 03/24/2010 14:17 PDT:
> The number of (cheap) devices that look at options is considerably
> higher than the number of terminating proxies/DPI devices. Even if we
> can get through today’s network as soon as MPTCP traffic percentages
> increase several devices will try and do something about it. Having it
> deeper in the packet means several error prone devices will have no
> chance of screwing up the options data.

Yes but as was mentioned, in the long run if MPTCP takes hold you want
middleboxes to be able to notice (easily, without DPI) that a flow is
multipath, and to do things to "help" it just as they "help" TCP today.
 For this, the options approach is easier.

> Regarding the possibility of implementing this in user mode, Mark did
> mention that keeping this in app space breaks the coordinated congestion
> controller. So that point is no longer valid although it does let apps
> try this out first before convincing kernels to provide native support
> for MPTCP.

Well, you _could_ have a user mode process running that coordinated
congestion for all apps, but anyway ...

I think ultimately we do want to end up with options mode.  How soon can
we get there?  My inclination is to

Run some tests to understand the situation today.  For example, set up a
TCP connection and send out data packets with random "options".  Have
the other end just encapsulate back what it gets.  See what gets through
and what doesn't, including on the SYN.  Run it between various parts of
the Internet and track down problem areas.  How bad is the middlebox
problem, really?  (If you set this up before I do, I'll be willing to
run your code.)

If the situation isn't bad, estimate how much of a backlash there will
be from middlebox operators if MPTCP is used for real traffic.  Will it
be squashed?  (Unlikely)

If options mode has a problem, implement MPTCP twice.  To start with
implement it in payload mode.  Three to five years from now, when it's
established, implement and promote options mode.  Sorry.  I know it's
work but at least it's spread out, and at least MPTCP will have a chance
of getting deployed.  But first test against reality.

?

Scott

---

From: Murari Sridharan <[email-masked]>
Date: Wed, 24 Mar 2010 21:53:11 +0000

Inline. 

-----Original Message-----
From: Scott Brim [mailto:[email-masked]] 
Sent: Wednesday, March 24, 2010 2:47 PM
To: Murari Sridharan
Cc: SCHARF, Michael; [email-masked]; [email-masked]
Subject: Re: [multipathtcp] Options Vs Payload

[MS] Yes but they do see the initial signaling option in the SYN that tells them that this flow is trying to use multipath so nothing stops them from parsing MPTCP specific addresses. 

[MS] sure

[MS] I am on the fence right now leaning towards payload. 

[MS] This test wont be easy in terms of getting coverage. It is not enough to try this with random options in the SYN but random options inside the data packets. Most OS' should ignore options they don't understand so hopefully this test doesn't cause random problems. 

?

[MS] Might be the way to go .

Scott

---

From: Andrew McGregor <[email-masked]>
Date: Wed, 24 Mar 2010 16:33:29 -0700

If you want test points in a fairly remote part of the net, keep me posted.  Two different ISPs and a 3G provider in New Zealand available.

Andrew

---

From: Scott Brim <[email-masked]>
Date: Wed, 24 Mar 2010 16:58:42 -0700

> If you want test points in a fairly remote part of the net, keep me posted.  Two different ISPs and a 3G provider in New Zealand available.

I was hoping you'd speak up.  And if you write the code before I do,
even better.

---

From: Yoshifumi Nishida <[email-masked]>
Date: Thu, 25 Mar 2010 09:45:05 +0900 (JST)

Hi, This is just my indivisual feeling and might be totaly wrong.. 
But, somehow I feel there might be a hybrid approach between option and payload approach.

As I said during the meeting, I feel optimzing mptcp flow would be difficult. 
For exmaple, your middle box might find two mptcp subflows. but it could be 2 of 10 subflows
in the mptcp session. In this case, I'm wondering how optimizing these 2 subflow can be
efficient to the total performance of the mptcp session.
So, I feel DATA mapping info can be stored in payload since if we cannot utlize them well.

On the contrary, I feel JOIN or ADDR option can be stored in option. Because they are not
frequently sent, we don't have to worry about option space very much. 
If you want to signal to middle box that this is mptcp subflow, I think we can send 
new tcp option with a certain interval for this purpose.

--
Yoshifumi Nishida
[email-masked]

---

From: Pasi Sarolahti <[email-masked]>
Date: Thu, 25 Mar 2010 11:42:14 -0700

I'm not in favor of hybrid approach. It could combine the  
disadvantages of both sides.

I have flapping between liking and disliking the payload approach. In  
the end I guess it depends on how we want the architecture to look like:

* If we think MPTCP as a modification to TCP, I think options should  
be used.

* If we think MPTCP as a new (sub)layer on top of the core TCP, doing  
control messaging in payload seems like a natural approach (perhaps  
with one additional TCP option that tells about this encoding)

Additional layers are not going to improve performance. Design-wise it  
seems kind of elegant to me to keep the core TCP simple and have the  
additional functionality on a separate layer. Congestion control would  
be one of the problems, but if a new multipath congestion control  
algorithm is implemented anyway, maybe it is possible to do it on this  
MPTCP layer, while disabling the standard TCP congestion control in  
that case.

- Pasi

On Mar 24, 2010, at 5:45 PM, Yoshifumi Nishida wrote:

> _______________________________________________
> multipathtcp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Yoshifumi Nishida <[email-masked]>
Date: Fri, 26 Mar 2010 14:57:46 +0900 (JST)

Hi Pasi,
you may be right.
But, one point I want to mention is that there is no pure option approach
nor pure payload approach. I mean, for option appaorch, we need to use payload 
for security infomration. for payload approach, we need to use option for 
conection/subflow setup. 
In a sense, every approach is hybrid.

Thanks,
--
Yoshifumi Nishida
[email-masked]

From: Pasi Sarolahti <[email-masked]>
Subject: Re: [multipathtcp] Options Vs Payload
Date: Thu, 25 Mar 2010 11:42:14 -0700
Message-ID: <[email-masked]>

 > * If we think MPTCP as a modification to TCP, I think options should be
 > * used.
 > 
 > * control messaging in payload seems like a natural approach (perhaps
 > * with one additional TCP option that tells about this encoding)
 >