Subject: [tcpinc] Call for data and opinions on option reodering

From: David Mazieres <[email-masked]>
Date: Fri, 06 Nov 2015 10:28:07 -0800

Wednesday, Tero brought up the prospect of middleboxes reordering
back-to-back unknown options.  This would currently mess up the ENO
transcript.  Obviously middleboxes can process and manipulate options
they know about.  The question is whether anyone has knowledge of a
middlebox that would flip the order of two, back-to-back, options of the
same, unknown kind.  It's hard to see a rationale for such behavior.
Then again, rationality is not the best predictor middlebox behavior.
Hence, I'm wondering if anyone has any concrete data or knows of a box
that would do this in practice.

Depending on the likelihood of such reordering, I see several ways to
proceed.

 1. If it truly never happens in practice, then we can leave ENO as-is
    and add a clause to the effect that middleboxes MUST NOT reorder ENO
    options with respect to one another.

 2. If it happens extremely rarely, then the transcript can
    lexicographically sort host A's ENO options.  The order of options
    is meaningful in host B, however, so there we don't sort.  Since
    host B SHOULD send only one ENO option except in weird fringe cases
    (such as simultaneous open), we can figure that two unlikely things
    are really unlikely to happen simultaneously.

 3. Use bytes of the form 1000xxxx (general suboptions with v=1) as
    no-ops, and specify that multiple ENO options must be
    lexicographically sorted.  This sacrifices other uses of bytes
    128-143, but has the advantage that in the common case it requires
    no extra bytes.  Another advantage is that if we ever get truly
    large SYN options, this would potentially allow more than 256 bytes
    of ENO options.

 4. We could add a length byte to every variable-length suboption,
    allowing multiple variable-length suboptions to be packed into the
    same ENO option.  Then we limit hosts to sending only one ENO option
    in a SYN segment.  The cost is an extra byte of precious option
    space in the common case, and limitation to 254 ENO bytes if we ever
    get large SYN options.

 5. Use bytes of the form binary 100xxxxx (that's general suboption and
    reserved cs values, with v=1) as markers for a 1-32 byte suboption
    (where the length of variable data is 1 + xxxxx).  This has the
    advantage that you only need the byte if you have more than one
    variable-length suboption, and the disadvantage that bytes 128-159
    might have been useful for something else in the future.  Another
    disadvantage, if we ever get large SYN options, is that all but the
    biggest variable-length suboption is now limited to 32 bytes.

 6. Like the previous, but certain values of xxxxx could mean "a length
    byte follows."  This has the disadvantage that the most likely value
    of xxxxx is 32, yet 32 is a nice number for ECDHE parameters if we
    ever get large SYN options.  Also, again all ENO suboptions are
    limited to 254 total bytes.

My preference, in order of decreasing appeal, is: 1, 2, 3, 6, 5, 4, with
a big distance between the first three (1, 2, 3) that I'm basically okay
with, and the last three (6, 5, 4) for which I would require some
convincing.  However, now that the document belongs to the working
group, I'm soliciting opinions from everyone.

Obviously my preference is heavily motivated by the hope that we someday
get large TCP options.  Though it seems unlikely in the near future,
history is littered with examples of people underestimating maximum
parameter values.  Particularly given that large options would make us
want to embed public-key-like values into SYN segments, and that each
individual ENO option will still be 256 bytes, I'd like to preserve the
ability to have more than one ENO option per SYN segment.

David

---

From: Joe Touch <[email-masked]>
Date: Fri, 6 Nov 2015 11:02:52 -0800

On 11/6/2015 10:28 AM, David Mazieres wrote:
> back-to-back unknown options. 

You can be certain of three things:

	- death
	- taxes
	- that middleboxes will do X, for all X

Someone, somewhere will come up with a justification for sorting,
reordering, reinserting, or otherwise messing things up.

IMO, there are two solutions to this:

	1) run TCP as an application layer protocol inside vanilla
	UDP or TCP (i.e., TCP with options inside TCP without)

		this is basically where MPTCP went, burying some
		signal info inside TCP's application data stream

	2) use security that protects the TCP header

		e.g., IPsec, TCP-AO, etc.

There is no third option. Putting mechanisms inside TCP options to
detect whether middleboxes alter them in unknown ways would be turning
TCP into a "network alteration probing" mechanism, which it is not.

Joe

---

From: Tero Kivinen <[email-masked]>
Date: Sat, 7 Nov 2015 09:04:01 +0200

> same, unknown kind.  It's hard to see a rationale for such behavior.

Do we have any text in any RFC which do say that order of TCP options
is important, and should be preserved? I tried to do quick search
through some RFCs, but could not find any.

If we do not have any such text anywhere, I do not think we can
complain if some middlebox does reordering... 
-- 
[email-masked]

---

From: Bryan Ford <[email-masked]>
Date: Sat, 7 Nov 2015 16:06:13 +0900

On Nov 7, 2015, at 3:28 AM, David Mazieres <[email-masked]> wrote:
> […]

I guess one of the approaches I suggested in my last E-mail (before I had read yours) is basically equivalent to this, only with the pattern being 10xxxxxx (allowing for 1-64-byte sub options but consuming more primary sub-option space).  Some flavor of this approach would be my preference, but I don’t have a strong intuition for whether the 10xxxxxx or 100xxxxx pattern is better.

At any rate, to reiterate, having multiple TCP-ENO options floating around in one TCP header and expecting different implementations to do something sensibly consistent with them sounds terrifying, especially when the common - and thus only well-tested - case will almost certainly be only one option.

B

---

From: David Mazieres <[email-masked]>
Date: Sat, 07 Nov 2015 00:42:46 -0800

Bryan Ford <[email-masked]> writes:

> At any rate, to reiterate, having multiple TCP-ENO options floating
> around in one TCP header and expecting different implementations to do
> something sensibly consistent with them sounds terrifying, especially
> when the common - and thus only well-tested - case will almost
> certainly be only one option.

I'm not sure what you mean by "something" sensibly consistent.  There is
only one sensible thing to do, namely follow the well-defined behavior
prescribed by the ENO draft, for which we will of course construct test
cases.  The main issue in this thread is middleboxes.

That said, regardless of whether the final ENO document allows hosts to
send more than one ENO option, the behavior of *receiving* multiple ENO
options will have to be well-specified and tested in all
implementations.

David