Subject: [tcpinc] TCP-ENO draft issues

From: Christoph Paasch <[email-masked]>
Date: Sun, 01 May 2016 15:11:14 -0700

Hello,

I have read through the TCP-ENO draft and have a few comments:

First, am wondering whether there might be an issue when the third ACK gets lost.

Specifically, the draft says that a host MUST disable ENO if the following condition is met:

   4.  The first ACK segment received by a host does not contain an ENO
       option.

So, if this ACK gets lost, and the client goes on with sending (encrypted) data, the server won't be able to know whether ENO has been successfully negotiated or not and how to interpret the data.

To overcome this, one possible way would be to include the ENO-option in every data-segment that a host is sending after the 3-way handshake until successful use of ENO has been confirmed by the receiver.

But even then, there might be a middlebox that removes the ENO-option from non-SYN segments (assuming that we want to support such kind of middleboxes). The sender will thus send encrypted data with the ENO-option. The receiver receives this encrypted data without the ENO-option (due to the middlebox stripping the option) and thus will assume that the data is unencrypted. (for the background, this kind of behavior is the reason why MPTCP makes sure to only send in-order data in the first few segments, until middlebox-support has been confirmed as it allows to fallback to regular TCP) 

This kind of means that we would need to do a 4-way handshake where data can only be sent after it has been confirmed on non-SYN segments that ENO is enabled. Maybe there is a better solution to it?

Second, there are loadbalancers out there (used by baidu.com <http://baidu.com/> in particular) that echo unknown TCP options. As far as I can tell, TCP-ENO won't be able to handle this situation gracefully, as an active opener will interpret the ECHO'd option as valid. The P-bit seems to allow to resolve this situation as the client would receive a TCP-ENO option with the P-bit set on a SYN/ACK segment. But, the P-bit is currently not part of the option thus I think it would be good to include it.

Cheers,
Christoph

---

From: David Mazieres <[email-masked]>
Date: Sun, 01 May 2016 16:41:25 -0700

Christoph Paasch <[email-masked]> writes:

> So, if this ACK gets lost, and the client goes on with sending
> (encrypted) data, the server won't be able to know whether ENO has
> been successfully negotiated or not and how to interpret the data.
>
> To overcome this, one possible way would be to include the ENO-option
> in every data-segment that a host is sending after the 3-way handshake
> until successful use of ENO has been confirmed by the receiver.

Indeed, this is a good point.  The idea is really that you MUST include
the ENO option whenever you ACK a SYN segment.  The problem is that the
ACK might be cumulative.  So we would like flows to look like this:

   A -> B:  SYN ENO
   B -> A:  SYN-ACK ENO
   A -> B:  ACK ENO (lost)
   A -> B:  ACK ENO* data

The draft is not at all clear about the need for the ENO marked by *.
Your suggestion is one way to fix this.

> But even then, there might be a middlebox that removes the ENO-option
> from non-SYN segments (assuming that we want to support such kind of
> middleboxes). The sender will thus send encrypted data with the
> ENO-option. The receiver receives this encrypted data without the
> ENO-option (due to the middlebox stripping the option) and thus will
> assume that the data is unencrypted. (for the background, this kind of
> behavior is the reason why MPTCP makes sure to only send in-order data
> in the first few segments, until middlebox-support has been confirmed
> as it allows to fallback to regular TCP)

This is one reason we would love to collaborate with someone who has
very concrete middlebox experience (hint hint), at the very least on a
TCP-ENO Middlebox Probing document, if not on ENO itself.  We have not
observed stripping from SYN but not non-SYN segments, and agree it would
be annoying.

Currently we would deal with this in the same as DPI--probe the
middlebox at DHCP lease time and disable ENO if the middlebox is not
compatible.  However, if such middleboxes are common enough, then we
might need to support them.  Such middleboxes are kind of pernicious,
since a lot of options are first negotiated in SYN segments then used
later.  Hence, if they exist but are not too common, I would be okay
with disabling encryption, so long as connections don't fail entirely or
get garbled.

> This kind of means that we would need to do a 4-way handshake where
> data can only be sent after it has been confirmed on non-SYN segments
> that ENO is enabled. Maybe there is a better solution to it?

There are other solutions, though it's not clear if they are better.
The 4-way handshake could be compressed by sending a SYN-ACK and plain
ACK back-to-back (so at least it doesn't incur an extra 1/2 round trip
in the common case).  Or we could do it probabilistically, including 4-8
bytes of randomness in the ENO options that then get hashed along with
some magic number and included in the TCP payload data.  That's kind of
yucky.

> Second, there are loadbalancers out there (used by baidu.com
> <http://baidu.com/> in particular) that echo unknown TCP options. As
> far as I can tell, TCP-ENO won't be able to handle this situation
> gracefully, as an active opener will interpret the ECHO'd option as
> valid. The P-bit seems to allow to resolve this situation as the
> client would receive a TCP-ENO option with the P-bit set on a SYN/ACK
> segment. But, the P-bit is currently not part of the option thus I
> think it would be good to include it.

That's another very useful point we didn't know about.  The p-bit is
currently implicit, but we could get rid of it and rely entirely on the
explicit b bit.  That would have the added advantage of simplifying role
negotiation at the cost of sometimes consuming an extra byte of option
space in the SYN-ACK segments.

Thanks for the useful feedback.

David

---

From: Mirja Kühlewind <[email-masked]>
Date: Wed, 4 May 2016 10:14:16 +0200

Hi maprg,

there is an on-going thread about tcp option stripping on the tcpinc mailing list (see below). Two problem cases have been identified:

1) Strip of non-SYN option while SYN-options pass correctly

2) Reflection of SYN options in the SYN/ACK (by load-balancers/proxies)

Does anybody have further data here?

Mirja

> Anfang der weitergeleiteten Nachricht:
> 
> Von: David Mazieres <[email-masked]>
> Betreff: Aw: [tcpinc] TCP-ENO draft issues
> Datum: 2. Mai 2016 um 01:41:25 MESZ
> An: Christoph Paasch <[email-masked]>, [email-masked]
> Kopie: [email-masked]
> Antwort an: David Mazieres expires 2016-07-30 PDT <[email-masked]>
> 
> _______________________________________________
> Tcpinc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpinc