Subject: [multipathtcp] comment about draft-mptcp-multiaddressed - MPC option

From: Sébastien Barré <[email-masked]>
Date: Mon, 01 Mar 2010 15:09:51 +0100

Hi,

About the MPC option, section 4.1contains that sentence:

----------
This option is only present in packets with the SYN flag set.
----------

AFAICT, this will prevent connections with TCP SYN cookie option enabled 
from working in multipath mode.
The reason is that if we send the MPC only in the SYN segment, the 
responder will neither remember that we are multipath-capable, nor will 
it store the token, since no state is recorded upon SYN reception.

So, why not sending the MPC option in the first segment following the 
SYN, that is, when we are sure that our peer is able to store the info ?

regards,

Sébastien.

-- 
Sébastien Barré
Researcher,
CSE department, UCLouvain, Belgium
http://inl.info.ucl.ac.be/sbarre

---

From: Costin Raiciu <[email-masked]>
Date: Mon, 1 Mar 2010 14:49:49 +0000

Hi,

The basic answer is: we need to think how to encode the MPC stuff best in the SYN-ACK packet. 

The difficulty here is to encode both the local token (4B) and the remote token. There is not enough space to encode both tokens in the ISN (this is obious). I think it's possible to encode 
both using timestamps, and this will work most of the time (this techniques are already used in freebsd i think to encode wscale, sack-permitted, etc). 

Sending MPC on subsequent packets as you propose is not enough, as it only provides the client token to the server; the server still needs to be reminded its own token. Unless we change the protocol to have the client include this too, this will not work. I think its an overkill to change the protocol.

Note that this only applies to initial SYN packets, and not JOIN packets; here SYN flooding attacks are much more difficult because they need to include the connection token.

In practice, it seems that SYN cookies are rarely turned on by default; if this is the case, it would also be fine to stick to single-path tcp when syn-cookies are enabled.

Cheers,
Costin

On 1 Mar 2010, at 14:09, Sébastien Barré wrote:

> AFAICT, this will prevent connections with TCP SYN cookie option enabled from working in multipath mode.
> The reason is that if we send the MPC only in the SYN segment, the responder will neither remember that we are multipath-capable, nor will it store the token, since no state is recorded upon SYN reception.
> 
> So, why not sending the MPC option in the first segment following the SYN, that is, when we are sure that our peer is able to store the info ?
> 
> regards,
> 
> Sébastien.
> 
> -- 
> Sébastien Barré
> Researcher,
> _______________________________________________
> multipathtcp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Lars Eggert <[email-masked]>
Date: Mon, 1 Mar 2010 06:58:43 -0800

On 2010-3-1, at 6:49, Costin Raiciu wrote:
> In practice, it seems that SYN cookies are rarely turned on by default;

they seem to be on by default on the two FreeBSD and Linux machines I have access to.

Lars

---

From: Costin Raiciu <[email-masked]>
Date: Mon, 1 Mar 2010 15:06:34 +0000

Hi Lars,

My information was from RFC 4987 "TCP SYN Flooding Attacks and Common Mitigations", on page 9:
"   Although SYN cookie implementations exist and are deployed, the use
   of SYN cookies is often disabled in default configurations, so it is
   unclear how much operational experience actually exists with them or
   if using them opens up new vulnerabilities.  Anecdotes of incidents
   where SYN cookies have been used on typical web servers seem to
   indicate that the added processing burden of computing MD5 sums for
   every SYN packet received is not significant in comparison to the
   loss of application availability when undefended.  For some
   computationally constrained mobile or embedded devices, this
   situation might be different."

If SYN cookies are always turned on as you found, i think it may be worthwhile to change the MPTCP handshake to deal with them more elegantly (e.g. make the protocol asymetric,as sebastien suggests).

Cheers,
Costin

---

From: Sébastien Barré <[email-masked]>
Date: Mon, 01 Mar 2010 16:13:10 +0100

Hi,

> both using timestamps, and this will work most of the time (this techniques are already used in freebsd i think to encode wscale, sack-permitted, etc). 
That might be one option to explore.
> Sending MPC on subsequent packets as you propose is not enough, as it only provides the client token to the server; the server still needs to be reminded its own token.
No. My proposal of "sending the MPC in the first segment following the 
SYN" was also true for the server. That is, I expected that the server 
also send its MPC separately from its SYN+ack.
One such approach would thus be:
client                          server
------------------SYN-------------->
<------------SYN/ACK------------
--------------ACK/MPC----------->
--------------DATA------------------>
<------------ACK/MPC------------

Is there an objection to that approach ? We may indeed need some delay 
before getting the MPC from server. But is it really a problem ?

On the other hand, Joerg's proposal seems fine to me as well.
>  Unless we change the protocol to have the client include this too, this will not work. I think its an overkill to change the protocol.
>   
Anyway, not changing the protocol means probably not supporting Syn cookies
> Note that this only applies to initial SYN packets, and not JOIN packets; here SYN flooding attacks are much more difficult because they need to include the connection token.
agreed.
> In practice, it seems that SYN cookies are rarely turned on by default; if this is the case, it would also be fine to stick to single-path tcp when syn-cookies are enabled.
As Lars says, my Linux Ubuntu has also Syn cookies enabled by default.

cheers,

Sébastien.

---

From: <[email-masked]>
Date: Mon, 1 Mar 2010 15:19:09 -0000

Syn cookies are incompatible with certain other aspects of TCP. ECN
being the one I know about, from my reading of the Linux kernel source.

Alan

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Lars Eggert
Sent: 01 March 2010 14:59
To: Costin Raiciu
Cc: Multipath TCP Mailing List
Subject: Re: [multipathtcp] comment about draft-mptcp-multiaddressed -
MPCoption

> In practice, it seems that SYN cookies are rarely turned on by
default;

they seem to be on by default on the two FreeBSD and Linux machines I
have access to.

Lars

---

From: Costin Raiciu <[email-masked]>
Date: Mon, 1 Mar 2010 15:29:46 +0000

Hi,

What you propose makes sense, but will be difficult to work in practice in my opinion. The main problem is the data sequence number: when do you start using it in the connection? After MPC-enabled is received, or sent? How do you switch from tcp mode to multipath mode? Such an abrupt switch can have weird failure modes, and we really need to document them properly if we will go that way.

I like the clear separation between negotiation (in syn) and use (in data segments) that TCP currently has. I would not like to blur it unless we really have to. 

The question is:  are syn-cookies a goodenough reason? Are deployed enough that we should design for them?

Them being on by default on client ubuntu doesn't mean much. Most web servers will have admins that change the settings anyway, to "enhance performance". Do they leave syn cookies on?

Cheers,
Costin

On 1 Mar 2010, at 15:13, Sébastien Barré wrote:

>> The difficulty here is to encode both the local token (4B) and the remote token. There is not enough space to encode both tokens in the ISN (this is obious). I think it's possible to encode both using timestamps, and this will work most of the time (this techniques are already used in freebsd i think to encode wscale, sack-permitted, etc).   
> No. My proposal of "sending the MPC in the first segment following the SYN" was also true for the server. That is, I expected that the server also send its MPC separately from its SYN+ack.
> Is there an objection to that approach ? We may indeed need some delay before getting the MPC from server. But is it really a problem ?
>

---

From: Sébastien Barré <[email-masked]>
Date: Mon, 01 Mar 2010 17:26:45 +0100

Hi,

> What you propose makes sense, but will be difficult to work in practice in my opinion. The main problem is the data sequence number: when do you start using it in the connection? After MPC-enabled…
We can go from tcp to multipath mode by simply properly mapping the 
current subflow number to the initial data seq number. Of course that 
implies that we no longer use 1 byte in the data seq number space for 
the SYN, but it is not clear to me that it is necessary to eat 1 byte of 
dataseq space for the SYN, we can just start mapping whenever we want. 
Now I agree that from a receiver point of view we must be very careful 
(due to the possibility of the first mapping being lost for example).

Now, all this indeed increases complexity, and there is another much 
more flexible option that I have already proposed in the past: 
separating the MPC from the token. After all the main source of our 
problems is the token. By separating, I mean announce MPC and the token 
as different options.

My initial motivation for proposing to separate MPC from the token, was 
that the token is not necessary for all path management schemes. In my 
current implementation, based on Shim6 for path management, there is 
even no token at all. So defining MPC as separate from the token allows 
for just not sending the token when we do not need it.

Now we have a second reason, with that syn cookie problem. If the MPC is 
sent in the SYN, we start mapping the dataseq from the beginning, and 
things are much easier. regarding syncookies, they can work by just 
repeating the MPC in the first segment following the client SYN, that 
is, that way:
client                                 server
---------------------SYN/MPC----------->
<--------------SYN/ACK/MPC----------
--------------------ACK/MPC------------>

*The server sees the MPC, so it replies with MPC.
*The client has state and can remember that the server supports Multipath.
*The server receives the third message. It "learns" again that the 
client supports multipath, since it had discarded the initial SYN. Also, 
it knows that it always replies with MPC to this category of requests, 
so it can enable multipath.

As for the token, we can send it whenever we want, as far as it is 
before any subflow addition. Makes sense ? At least this is the current 
working of my implementation.
> I like the clear separation between negotiation (in syn) and use (in data segments) that TCP currently has. I would not like to blur it unless we really have to. 
me too :-)
> Them being on by default on client ubuntu doesn't mean much.
ok. I have now checked the server version of ubuntu: syn cookie is 
enabled as well. Of course we should check several distributions and 
operating system to have a better picture of what is enabled by default 
usually.
>  Most web servers will have admins that change the settings anyway, to "enhance performance". Do they leave syn cookies on?
>   
I tend to think that if they disable syn cookie, they just accept the 
risk of syn flooding, while gaining almost nothing in performance. So I 
really wonder why admins would decide to disable this.
Now I don't know how to derive a general trend in syn cookie admin's 
behaviour. The only thing I can tell is that I have not yet found even 
one TCP with syn cookie disabled.
Anyway, if we find a cheap solution, why not to include it ?

cheers,

Sébastien.

---

From: Arnd Hannemann <[email-masked]>
Date: Mon, 01 Mar 2010 17:36:50 +0100

> they seem to be on by default on the two FreeBSD and Linux machines I have access to.

Even if they are "turned on" in Linux they are only activated, if a syn flooding attack is
detected.
FWIW: A few years ago code was merged in Linux to make syn cookies compatible with a lot of TCP options,
by exploiting the Timestamp option...

http://git.kernel.org/?p=linux/kernel/git/davem/net-2.6.git;a=commit;h=4dfc2817025965a2fc78a18c50f540736a6b5c24

Best regards,
Arnd

-- 
Dipl.-Inform. Arnd Hannemann
RWTH Aachen University
Dept. of Computer Science, Informatik 4
Ahornstr. 55, D-52074 Aachen, Germany
Phone: (+49 241) 80-21423 Fax: (+49 241) 80-22220

---

From: Sébastien Barré <[email-masked]>
Date: Tue, 02 Mar 2010 14:17:21 +0100

Hi,

> Even if they are "turned on" in Linux they are only activated, if a syn flooding attack is
Many thanks for pointing this. I see now that indeed, we still store 
state even if syn cookies are turned on, except when under attack.

So based on this, I guess we decide to keep everything as is ?

Just one thing we should remain careful about is that we are still 
increasing the size of a structure we want as small as possible.
In the case of linux, the struct request_sock is the temporary small 
socket used by the server side until the connection is established.
I have been warned by Ilpo Järvinen about the fact that we should really 
avoid increasing the size of that structure. If we keep the token 
together with the MPC option,
we need to store 4bytes of token plus 1bit of "mpc_received" 
information. Maybe this is tolerable, I don't know. Just to be sure...

regards,

Sébastien.

---

From: Sébastien Barré <[email-masked]>
Date: Tue, 02 Mar 2010 15:37:18 +0100

Hi,

sorry, small correction:

Sébastien Barré a écrit :
> I have been warned by Ilpo Järvinen about the fact that we should 
> really avoid increasing the size of that structure. If we keep the 
> token together with the MPC option,
> we need to store 4 bytes of token plus 1bit of "mpc_received" 
we need to store 8 bytes of token (local+remote) plus 1 bit of 
mpc_received. I forgot to mention that if the server answers with 
MPC+token in the syn+ack, it must store its local token in addition to 
the remote token.
Just to make things more precise.

Sébastien.

---

From: <[email-masked]>
Date: Tue, 2 Mar 2010 15:56:11 -0000

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Sébastien Barré
Sent: 02 March 2010 14:37
To: Multipath TCP Mailing List
Subject: Re: [multipathtcp] comment about draft-mptcp-multiaddressed - MPC option

Hi,

sorry, small correction:

Sébastien.
<snip>

Have you tried asking on the Linux development lists as to whether this would be tolerable? I would think that the [email-masked] list would be the most appropriate place to ask even though this is not current Linux kernel development work.

Alan

---

From: <[email-masked]>
Date: Fri, 5 Mar 2010 10:41:54 -0000

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Sébastien Barré
Sent: 02 March 2010 14:37
To: Multipath TCP Mailing List
Subject: Re: [multipathtcp] comment about draft-mptcp-multiaddressed - MPC option

Hi,

sorry, small correction:

Sébastien.
<snip>

I saw my suggestion to email [email-masked] elicited no replies. I uses this is because MPTCP is too far in the future for them. I'm sure it will be of interest to them when/if MPTCP reaches RFC status. As the current development work is not going directly into the released Linux build, I would say make the changes to the struct and test for any performance issues. Any other opinions out there?

Alan

---

From: Sébastien Barré <[email-masked]>
Date: Fri, 05 Mar 2010 12:55:29 +0100

> I saw my suggestion to email [email-masked] elicited no replies. I uses this is because MPTCP is too far in the future for them. I'm sure it will be of interest to them when/if MPTCP reaches RFC st…
Indeed, since you suggested to write a mail to netdev to get opinions 
there, I tried that, and indeed it seems that the mail got lost in the 
dozens of messages per day that list receives. Anyway, at the moment I 
implement the draft as it is. Although I still think that we should 
separate MPC from token into separate options (from both efficiency and 
architectural reasons), until I hear a sufficient number of people 
telling me that they agree, I assume everybody agrees rather with the 
current draft. Thus I stick to the draft.

That means that indeed I store the local and remote token in the 
temporary sock that is created between receiving the SYN and the final 
ACK. At least, this will not hurt in an experimental environment.

regards,

Sébastien.