Subject: [tcpm] SYN/ACK Payloads, draft 01

From: "Adam Langley" <[email-masked]>
Date: Mon, 11 Aug 2008 10:35:20 -0700

The comments on SYNACK Payloads[1] seem to have died down, so I'd like
to see how people are feeling about it.

Just to be clear; I'm seeking to publish this as experimental in order
to get an option number assigned to conduct more widespread tests.

I thought that I would dig out a couple of the points that I believe
Joe Touch was still unsure about after the last thread. I don't wish
to misrepresent anyone, but it is a long thread to dig through: thanks
to Joe for being so generous with his time in giving feedback.

> I appreciate the
> idea of optimizing TCP, but there are so many other places where RTTs
> are not under your control, and so many RTTs in a connection, that
> removing (or adding) a single one anywhere seems in the noise."

Many Google users are now using search bars to submit queries. Thus
they type into a text field in the browser, hit enter and the browser
makes a connection, sends the query encoded in a GET and displays the
result. This is one example of a situation where the connection
latency isn't amortised. There are many others, but obviously, this is
the example that usually dominates my thinking. And, in this
situation, it turns out that every 10ms makes a difference.

I might be wrong here. This might be an overcomplex solution. But I
think only time will tell, which is why I wish to have it published as
an experimental spec. If, in a couple of years, it doesn't work the
option number can be released, ready for the next guy.

> Using a transport signal to change the application protocol seems
> very strange to me. Other's can comment on that.

It is strange, I agree. My usual test about such things is if I get an
"ick" reaction and, in this case, I don't. Although, that might just
be me.

Two reasonable questions arise:

"Where will this stop?"  - obviously, the TCP option space shouldn't
be a repository for application data. In this case, at least, the
option is advertising a certain minimum capability of the TCP stack,
as defined in the draft. Also, I'll claim that this is the last such
need in this area:

The SYNACK payload sizes is a decision for the server (within the
advertised window). No more client options could be needed to
advertise support for larger ones.

Since the SYNACK payload is (nearly) constant, and TCP stacks are not
going to differ sufficiently to change this in the next 20 years,
applications don't gain anything else by stuffing anything more in the
SYN option space.

So I don't think this sets an unreasonable precedent.

Thanks all,

[1] http://www.ietf.org/internet-drafts/draft-agl-tcpm-sadata-01.txt

AGL

-- 
Adam Langley [email-masked] http://www.imperialviolet.org
_______________________________________________
tcpm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tcpm

---

From: "Sergio Freire" <[email-masked]>
Date: Mon, 11 Aug 2008 23:03:14 +0100

Hi Adam et all,
I have been on vacations but I have the thread of emails concerning this
subject.
We would like also to propose something about this as Joe Touch mentioned
earlier.
Me and Andre Zúquete have published a paper for proposing a TCP-layer name
service.
http://www.usenix.org/events/usenix08/tech/full_papers/freire/freire.pdf

In this work we "propose" the use of data in the SYN/SYN+ACK segments.
We intend to generalize this and also to go forward on this matter.
I have to see carefully the thread of your earlier discussions and your
draft as of the current version, so we can see if we're going in the same or
in opposite/alternative directions.
Meanwhile, if you can take a look at our work, it would be great for further
discussion (regarding the way of extending tcp options and not the name
service itself).

Note: I think this is an important subject and I'm glad we're pushing this
forward, even if we propose alternate ways for this. Please give me some
days so I can synch.

Regards,
Sergio Freire

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Adam
Langley
Sent: segunda-feira, 11 de Agosto de 2008 18:35
To: [email-masked]
Subject: [tcpm] SYN/ACK Payloads, draft 01

Thanks all,

AGL

---

From: "Adam Langley" <[email-masked]>
Date: Mon, 11 Aug 2008 16:17:08 -0700

> discussion (regarding the way of extending tcp options and not the name

Somehow I've already read your paper, although don't have any idea
where I got the pointer from!

A few questions:

There's no legacy mode for active opens because the port name mode is
singled with a destination port number of 0, right?

You're taking the option number 0x45 for your length indicator. Lars
pointed out to me a few months ago that 254 and 255 are assigned for
this if you're using it over real networks.

Assuming that a SYN frame is sent to port 0, requesting a named TCP
service which the destination knows about: what's the source port
number of the resulting SYN/ACK? I think I'm reading that it's chosen
randomly.

>From the table in 3.3 it doesn't appear that there's any SYNACK
payload, is that correct?

It's unfortunate that there's no way for clients to gradually start
using such a scheme, but you can't extend the option space in SYNs
and, if you used non-0 destination ports, some hosts may enqueue the
payload.

In short, if my understanding is correct, I don't believe these two
schemes interfere, even on a single connection.

Are you intending on trying to get an option number assigned for this?

Cheers

AGL

---

From: "Sergio Freire" <[email-masked]>
Date: Tue, 12 Aug 2008 14:44:33 +0100

Hi Adam,
First let me say that the paper I mentioned presents a possible solution for embedding data in the synchronization packets.
Our work mixes two concepts:
 a) use data in the segments during the three-way handshake, that in this case is somehow part of a TCP option which just signals its length
 b) a TCP-layer name service

What we intend to do is to split this two concepts and propose:
 a) a way to extend the space available for TCP options, by reusing the payload of TCP segments
 b) a generalization of the TCP-layer name service using the functionality provided by a)

See detailed answers to your questions inline, bellow.

Regards,
Sergio

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Adam Langley
Sent: terça-feira, 12 de Agosto de 2008 0:17
To: Sergio Freire
Cc: [email-masked]; [email-masked]
Subject: Re: [tcpm] SYN/ACK Payloads, draft 01

A few questions:

Sergio: I'm not sure if I've understood correctly but I think so. In our model you can establish a connection to a named service. The connection can be either established from a legacy client or from an "enhanced" portname aware client. If the connection is established from a legacy client, then it must be established to the fixed port number where the service was bound (and not 0). If the client is portname aware, then it can either establish a connection to destination port number 0 (and the given portname) or to the specific destination port number (if the service was bound to a specific port number).
I'm forgetting on purpose the port access restrictions that we speak about in the paper, these can limit legacy connection.
So ultimately, you can have legacy and named connections.

Sergio: Well I don't think that is exactly correct. Looking at http://www.iana.org/assignments/tcp-parameters/ and RFC3692 and RFC4727, we can see that the reserved kind values are 253 and 254. These two are used for testing purposes as RFC3692 (I think for general testing purposes, not for a specific purpose). So, a new option would have to be assigned to specify a way of extending the available space for TCP options. In a first phase I guess 253 or 254 could be used for that since they are "generic", if what I've said earlier is correct.

Sergio: It's in the paper but let me clarify: at the server side you can bind a service to a certain name and to a specific port or else you may not specify the port number. In the last case the operating system assigns it a random port. So the SYNACK source port corresponds either to the specific port using during bind or to a random port chosen by the OS at bind time. 
Actually changing ports during a connection raises some problems like in firewalls, NAT boxes. We have to chose another approach like Joe Touch's suggestion of a randomly chosen destination port number at client side (see portnames draft from J.Touch).

Sergio: No. There is payload in the SYNACK, which corresponds to the same payload we used in the SYN. We use it to correlate both packets. There was a typo bug in the initial paper in that table though. I think it's correct as it is now.

Sergio: Well, that is the key point. We must figure out a out for that to not happen. Most OSes and TCP implementations ignore payload in the SYN or SYNACK segments. Payload in the ACK segment of the three-way handshake is generally understood. We used destination port 0 to force a RST on legacy server side TCP implementations. There are several options for not going through with a wrong connection with legacy systems... one is to assign a fixed destination port when using this "TCP option extension mechanism". If you change the sequencing numbers from the standard behavior, many stacks will drop the connection. But this is not 100% safe. The great idea is to have some way that traditional stacks drop the connection while new ones accept it.
We can discuss this latter in more detail, I was just giving a general overview of the problematic here.

Sergio: Well, we would like to write two drafts: one for a generic way to extend the space available for TCP options and another one to specify a "name" that would be used to resolve to a certain service with that given name. I guess this corresponds to two TCP options. As I've mentioned above, the last one reuses the possibility created by our work or some other work like yours that can reuse the TCP payload.

Cheers

AGL

---

From: Lars Eggert <[email-masked]>
Date: Tue, 12 Aug 2008 17:47:39 +0300

On 2008-8-12, at 16:44, ext Sergio Freire wrote:
> Sergio: Well I don't think that is exactly correct. Looking athttp://www.iana.org/assignments/tcp-parameters/ 
>  and RFC3692 and RFC4727, we can see that the reserved kind values  
> are 253 and 254. These two are used for testing purposes as RFC3692  
> (I think for general testing purposes, not for a specific purpose).  
> So, a new option would have to be assigned to specify a way of  
> extending the available space for TCP options. In a first phase I  
> guess 253 or 254 could be used for that since they are "generic", if  
> what I've said earlier is correct.

If you look at the URL you posted, you'll see this bit: "Registration  
Procedures: IESG Approval or Standards Action"

That means that you MUST obtain a number through one of these two  
procedures (see RFC5226). You MUST NOT simply grab an unassigned one.

Adam is correct that for limited experimentation, you should use 253  
or 254, as per RFC4727. To go beyond that, you MUST register an option  
number, either via IESG Approval or a Standards Action.

Lars

---

From: "Sergio Freire" <[email-masked]>
Date: Tue, 12 Aug 2008 16:52:46 +0100

Hi Lars,
Of course, I know that but thanks for clarifying :)
We used a value just for experimentation purposes only.
What Adam was saying is that TCP option kind values 254 and 255 were
reserved to specify a length indicator, that's what I understood. But I
think he meant 253 and 254. Anyway, these kind values seem to not be
reserved for a specific purposed... instead, they are reserved for
experimentation purposes. Also, just to clarify this. :)
Regards,
Sergio

-----Original Message-----
From: Lars Eggert [mailto:[email-masked]] 
Sent: terça-feira, 12 de Agosto de 2008 15:48
To: ext Sergio Freire
Cc: 'Adam Langley'; [email-masked]; [email-masked]
Subject: Re: [tcpm] SYN/ACK Payloads, draft 01

On 2008-8-12, at 16:44, ext Sergio Freire wrote:
> Sergio: Well I don't think that is exactly correct. Looking
athttp://www.iana.org/assignments/tcp-parameters/ 

Lars

---

From: "Adam Langley" <[email-masked]>
Date: Tue, 12 Aug 2008 09:14:28 -0700

>  b) a generalization of the TCP-layer name service using the functionality provided by a)

For a) also see [1], although it doesn't do anything for including
extra data in the SYN frame.

> Sergio: I'm not sure if I've understood correctly but I think so. In our model you can establish a connection to a named service. The connection can be either established from a legacy client or fr…

Right, it's just unfortunate that a client connecting to a portname
cannot fallback to a default portnumber. (For example, connecting to
"http", but also setting the destination port to 80 for server which
don't understand that). However, I don't believe that there's any way
to do that.

> Actually changing ports during a connection raises some problems like in firewalls, NAT boxes.

Indeed, NATs strike again. I believe most of them would fail in this situation.

> We have to chose another approach like Joe Touch's suggestion of a randomly chosen destination port number at client side (see portnames draft from J.Touch).

However, if you aren't setting dport to 0 to signal that the SYN is
special, then hosts may interpret the payload of the SYN frame to be
application level data. Joe's portname draft uses options to avoid
this.

> Sergio: No. There is payload in the SYNACK, which corresponds to the same payload we used in the SYN. We use it to correlate both packets. There was a typo bug in the initial paper in that table th…

In the bottom right cell of the table, the final ACK is only acking
ISN_s + 1, should it not be ISN_s + 1 + L? (Unless the echoed data in
the SYNACK isn't acked). Also, I don't understand why you need this to
correlate the packet - you have the destination port in the SYNACK.

AGL

---

From: "Sergio Freire" <[email-masked]>
Date: Tue, 12 Aug 2008 18:17:30 +0100

Inline..

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Adam Langley
Sent: terça-feira, 12 de Agosto de 2008 17:14
To: Sergio Freire
Cc: [email-masked]; [email-masked]
Subject: Re: [tcpm] SYN/ACK Payloads, draft 01

Sergio: I think it would be great to handle also additional payload. If we are changing things at this level I think it would be of great valor to handle the two cases :) I still have to look carefully at your draft.

Sergio: well, using Joe Touch's portname draft, which is a bit different from our approach, you could. Anyway, you can also think on specifying the destination port number (your default portnumber) instead of 0. In that case we would need to signal the portname and maintain the legacy compatibility using another method.

Sergio: Actually they don't, because in that specific case the destination host would have to understand port names.

Sergio: Right, in my thesis that was corrected. Thanks. Let me explain how segments are correlated: src_IP+src_port+dst_IP+dst_port. Since the destination port in SYN may be 0 and the source port in SYNACK is always the resolved port. So, there must be another field to correlate, which in this case we decided to use the portname.

AGL

---

From: "Adam Langley" <[email-masked]>
Date: Tue, 12 Aug 2008 11:55:06 -0700

> Sergio: I think it would be great to handle also additional payload. If we are changing things at this level I think it would be of great valor to handle the two cases :) I still have to look caref…

Sadly, I don't believe that there is a good way to include extra
options space in a SYN frame in a backwards compatible manner. You're
trick of using port 0 is one such way, but obviously ties it to an
alternative port resolution scheme.

> Sergio: Actually they don't, because in that specific case the destination host would have to understand port names.

I'm suggesting that if you sent a SYN with a payload and a random
destination port to a host, that host, if the port was open, could
interpret the payload to be application level data and reply with a
SYNACK. Although you would notice that it wouldn't echo your option.

> Sergio: Right, in my thesis that was corrected. Thanks. Let me explain how segments are
> correlated: src_IP+src_port+dst_IP+dst_port. Since the destination port in SYN may be 0
> and the source port in SYNACK is always the resolved port. So, there must be another field
> to correlate, which in this case we decided to use the portname.

I understand that you've lost one of the elements of the 4-tuple that
identify a connection, I was suggesting that you could just do without
it if your source ports for conections were random. There might be a
case in which this goes wrong that I'm not thinking of right now. I
guess it is rather against the TCP spirit. Fundamentally, I don't
believe that this conflicts with my SYNACK payloads draft. The option
in the SYNACK can easilly denote the portname from the application
level data.

Also, SYN crossings are interesting with this scheme. I think they
work, but it's complicated. The ACKs, in this case, would also have to
include the portname. I think a SYN crossing where one of the sides
isn't using the portname to resolve the other goes wrong though.

AGL

---

From: "Sergio Freire" <[email-masked]>
Date: Tue, 12 Aug 2008 22:37:09 +0100

Hi Adam.
Again, inline :)
Regards,
Sergio Freire

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Adam Langley
Sent: terça-feira, 12 de Agosto de 2008 19:55
To: Sergio Freire
Cc: [email-masked]; [email-masked]
Subject: Re: [tcpm] SYN/ACK Payloads, draft 01

Sergio: well, we must figure out the best way to achieve this. Using a destination port 0 was a solution we devised in our specific case. Again, we want to generalize this and consider other possibilities like the one depicted on my next comment.

Sergio: Yeap! That is an option we are considering: resetting the connection if the SYNACK does not contain the given TCP option.

Sergio: yes, by maintaining the connection elements (the 4-tuple) during connection solves many problems. Yet, there's a possibility of connecting to wrong services (e.g. imagine a server that is bound to some X port... the client issues a connection and the randomly chosen destination port at client side is X... this will collide with an existing service). But again, this can be "solved" looking at the SYNACK reply, for example.
About your draft, let me take a look first.

Sergio: Nice that you thought about this. In fact, we didn't considered simultaneous connections mainly because nobody seems to be using them. 
Besides that, the paradigm is like a client/server model: the client tries to connect to a server which is bound to a name.
The client itself does not have a name. So, it does not make too much sense to think on this case using this paradigm, unless the connection comes from the socket server but that does not make sense.
If the client and the server both have names, then SYN crossing would make sense.

AGL

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 13 Aug 2008 10:27:52 -0700

$Id: draft-agl-tcpm-sadata-01-rev.txt,v 1.2 2008/08/13 16:15:00 ekr Exp $

I don't really have a strong opinion about whether this extension
should be approved for use with TCP; that's for the TCP experts
to decide. However, I don't find motivational use cases--particularly
the cryptographic use cases listed in S 4 and S 4--particularly
motivating.

S 4. Opportunistic HTTP Encryption

As the draft states, this is intended to provide opportunistic 
security without burning any round trips in the application
layer.

As I understand this mechanism, what's being proposed is this:

      Client                           Server
      ------                           ------
      SYN + PPExt ->
      
          <- SYN/ACK + PPExt + [Crypto stuff]
      
      [Crypto Stuff] ->
      [HTTP Req]    

Do I have that right?

I see a number of issues here:

1. The PPExt payload is overloaded here. First, it's signalling
   a transport message "it's OK to put stuff in the SYN/ACK". Second,
   it's signalling an application layer message "It's OK to send me
   a crypto handshake as the first thing you send." Note that in
   an ordinary HTTP connection, this data would be invalid. 

   This overloading seems fairly unwise, since it would preclude
   any other use of this SYN/ACK payload in the future. If the
   server is going to send non-HTTP compliant data in response to
   this option, then more information needs to be signalled than
   just "I would accept data".

2. It's not clear to me that opportunistic security for HTTP is that
   great. Unlike, for instance, SMTP, HTTP fetches are nearly always
   the result of some URI, so it's very easy to simply indicate 
   in the URI that the server supports HTTPS. This is superior
   from a security perspective for the usual reasons (referential
   integrity, etc...)

3. There are a number of mechanisms for optimizing the performance
   of TLS vis-a-vis round trips that have not yet been explored
   (e.g., Fast-Track [BSR04]). It's not clear if we can shave
   things down to zero RTTs (though I've got some ideas), but 
   it seems that if we're really concerned about crypto setup 
   time it would be worth exploring this before we start with
   a new security layer, especially because of point (2) above.

4. Some of the latency benefit you're getting here is from not
   negotiating the crypto algorithms; this is why in TLS the
   client speaks first. This isn't an issue for the symmetric
   algorithm since the server could provide a list, but it's
   a real issue for the asymmetric algorithm, since it provides
   no agility in the case of upgrade or compromise. If, for
   instance, someone successfully attacks this curve but there
   are other, strong curves available, then we'd be hosed.

S 5. Faster SSH Connections

It's true that SSH latency is really bad, but it's really much
worse than you suggest, since the authentication handshake takes
several exchanges as well. [GutXX] It's not clear to me that removing 
1 RTT helps here. I'm skeptical that there are really that
many applications that involve bringing up and tearing down
a lot of SSH connections. 

Moreover, since SSH involves a lot of connections between the same
pair of nodes, it seems like a TLS style session caching mechanism
would improve the situation a lot more, optimizing out nearly all
round trips for subsequent connections, while also reducing public
key load while again requiring no changes to the kernel.

S 6. Compressed HTTP Headers
I don't know how much value this adds, but note that the server's
indication that it supports compressed headers lives in the same
section of the datastream as the crypto stuff from S 5. So, we
now need some sort of framing mechanism for this data to allow
clients which know about opportunistic crypto but not compressed
headers to ignore that advertisement. 

[BSR04] H. Shacham, D. Boneh, and E. Rescorla. "Client-Side Caching for
TLS." ACM Trans. Info. & Sys. Security, 7(4):553-75,
Nov. 2004.

[GutXX] P. Gutmann, "Performance Characteristics of Application-Level
Security Protocols," Work in Progress.

-Ekr

---

From: "Adam Langley" <[email-masked]>
Date: Wed, 13 Aug 2008 10:47:57 -0700

> As I understand this mechanism, what's being proposed is this:
[snip]
> Do I have that right?

Yep,

>   an ordinary HTTP connection, this data would be invalid.

Actually, it's only signaling the application extension. The SYNACK
payloads are valid in any TCP connection, although this spec does make
some guarantees about weather the stacks will process them.

>   any other use of this SYN/ACK payload in the future.

The format of the SYNACK payload for this scheme is similar to the
option space: i.e. it's a tagged, extensible payload.

>   from a security perspective for the usual reasons (referential

I believe that there's a use for a low-cost cryptographic scheme, such
as this. Despite widespread TLS deployment, most traffic is still in
the clear, including semi-sensitive information. I would rather that
traffic be protected with TLS but, since that isn't happening, I would
rather this than nothing.

Opinions on this vary. Some folks think it's a great idea (including
some working on implementing the TLS Fast-Track you cite below), some,
like you are unimpressed. Fundamentally, I'm only asking for an option
number to try it out.

>   are other, strong curves available, then we'd be hosed.

Firstly, this isn't rigorous crypto, so the primitives here are in the
enviable situation of being good enough if it's infeasible to break
them in near-real time. If you want real security, use TLS.

As mentioned above, the SYNACK payload is a tagged structure. Since
there are very tight space constraints it probably wouldn't be
possible to offer several elliptic curve points, but if curve25519
were totally broken there is a gradual upgrade path by upgrading
clients and then switching servers to offer another tag with a
different curve. Older clients would fall back to no encryption, but
it doesn't require a flag day.

> a lot of SSH connections.

You're correct. This isn't the greatest example, I was just looking
for some diversity to suggest that this isn't limited to HTTP.

> headers to ignore that advertisement.

Yep. Again, the framing mechanism already exists, although it's not
specified in the draft because it's not in scope for a TCPM draft.

The framing mechanism also allows for any other crazy HTTP (or any
other "client transmits first") protocol extensions in the future. I
think that, even if you don't much care for the opportunistic
encryption, that's fairly powerful.

Cheers

AGL

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 13 Aug 2008 11:16:30 -0700

At Wed, 13 Aug 2008 10:47:57 -0700,
> some guarantees about weather the stacks will process them.

I'm srory, I don't understand what you're saying here. Does this
data appear to the application to be part of the data stream
or not? 

> option space: i.e. it's a tagged, extensible payload.

OK, this wasn't entirely clear from reading the draft.

> rather this than nothing.

But the premise of this scheme is that people don't do TLS
because of either (1) latency or (2) lack of opportunistic 
negotiation. As far as I can tell, neither is correct. Rather,
it's concerns about server performance. And server performance
can be improved by a number of mechanisms within TLS that
are (1) backward compatible with TLS and (2) don't require
modifying the kernel. 

That said, if you're willing to burn even very small amounts
of data in the SYN, you can do something TLS-compatible by
passing a compressed ClientHello of some kind + FastTrack.
This only works with static RSA one the server side, but
so does the scheme you propose.

> like you are unimpressed. Fundamentally, I'm only asking for an option

As I indicated earlier, I'm not taking a position on whether you
should be assigned an 

> them in near-real time. If you want real security, use TLS.

Yeah, I'm not a big fan of inventing new things we know to be weaker
than the old things we already have, especially when the benefit
claimed by the newer thing seems marginal.

-Ekr

---

From: "Adam Langley" <[email-masked]>
Date: Wed, 13 Aug 2008 11:45:07 -0700

> data appear to the application to be part of the data stream

Yes, the data is part of the application stream. However, including a
payload in the SYN/ACK frame is permitted by TCP. Thus application
protocols in which the server speaks first could include payloads in
the SYN/ACK today, with no changes. Admittedly many stacks will only
ACK the SYN flag and not the data, so the data has to be
retransmitted, but the changes needed don't need any standard work.

The option proposed here is just for "client speaks first" to escape
that mode and switch to a "server speaks first" mode.

> OK, this wasn't entirely clear from reading the draft.

I didn't include it in the draft because it seemed out of scope, but
maybe I should since attentive readers are obviously picking up on it.

> are (1) backward compatible with TLS and (2) don't require

(1) is certainly a concern today. Even a single round trip. Most
actual numbers on this are considered secrets by the organisations in
a good place to gather them, but here's one soundbite:

"every 100ms of latency costs Amazon 1% of profit"
http://radar.oreilly.com/2008/08/radar-theme-web-ops.html

(2) is also a concern: simply put, users type http by habit.
Additionally, getting TLS certificates is a pain for small sites. As
for server load, curve25519 can be performed in 240us on a Core2 and
Salsa20/8 runs at 2 cycles/byte (same hardware). That reduces server
load.

But, really, I don't really know the reasons why TLS isn't ubiquitous
even with ubiquitous support. But, given that it isn't, I'm looking to
the best that I can with a very low-cost (i.e. computation, latency)
system.

I should also add another point of the opportunistic encryption scheme
that isn't covered in the draft:

When in effect, servers may return an X-ObsTCP: header that specifies
an https URL to a document that contains a list of Diffie-Hellman
public values and validity time periods for each. (The hostname is
required to match.) It's hoped that client's can cache it and validate
keys in the future. (Credit to Markus Gutschke from the Google
security team for that.

However, as you point out, there's another route here: additional TLS.
One of the tags that I have penciled in for the SYNACK payloads is
"TLS upgrade" - e.g. the server can advertise TLS support without a
round trip. With client-side context caching, that can be setup with
only one extra RTT in the optimal case (2 otherwise I believe).

The reason that I didn't include that as an explicit example is
because it can be downgrade attacked (obviously) and so, it might
actually be counter productive because it wouldn't teach users to type
"https".

However, if you have ideas for a zero RTT TLS, please send them my way
(possibly off list in that case). If there's a possibility to use
something standard(ish) with no additional latency I'll use it.

> so does the scheme you propose.

I can reasonably fit 14 bytes in a SYN frame. I did consider trying to
get RSA in the SYN (because then the computation costs could be mostly
put on the server), but I don't think I can fit anything useful in
there. (Not even a good elliptic key).

AGL

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 13 Aug 2008 12:50:27 -0700

At Wed, 13 Aug 2008 11:45:07 -0700,
> that mode and switch to a "server speaks first" mode.

Right, but this has application-level semantics because the content
of the server's data is not what an unmodified client expects. So,
it's not just a timing issue: the server sends different data 
to the client if this option is included or not.

> http://radar.oreilly.com/2008/08/radar-theme-web-ops.html

Well, this is just an unsupported claim in the presentation, so I have
no idea what the underlying data is. However, superficially I find it
deeply implausible. I have a Comcast business line, a fast computer,
and a modern version of Firefox and it takes about 5 seconds to load
Amazon's home page. There are lots of things Amazon could do to
improve that performance (e.g., show me less images on the
main page). Given that they don't, it's pretty hard to take complaints
about a couple of hundred ms latency seriously.

> (2) is also a concern: simply put, users type http by habit.

And the servers could say "use TLS next time" and the browsers could 
remember that. 

> Additionally, getting TLS certificates is a pain for small sites.

Huh? I've done this. It's cheap (< $50/yr) and simple (< 10 minutes).

> As

You're conflating unrelated issues, since TLS is perfectly capable
of using these algorithms, if someone thought it was worth defining
a cipher suite.

> the best that I can with a very low-cost (i.e. computation, latency)

This sounds to me like a classic case of premature optimization.
For all you know the reason TLS isn't ubiquitous is that it
doesn't have a cool name. Maybe we should work on that instead.

> there. (Not even a good elliptic key).

I'm talking about a different dataflow.  your dataflow is:

      Client                           Server
      ------                           ------
      SYN + PPExt ->

      [Crypto Stuff] ->
      [HTTP Req]    

Consider the basic TLS handshake from 4346:

      ClientHello                  -------->
                                                      ServerHello
                                                     Certificate*
                                               ServerKeyExchange*
                                              CertificateRequest*
                                   <--------      ServerHelloDone
      Certificate*
      ClientKeyExchange
      CertificateVerify*
      [ChangeCipherSpec]
      Finished                     -------->
                                               [ChangeCipherSpec]
                                   <--------             Finished
      Application Data             <------->     Application Data

But the client can techncially start sending data as soon as it has
sent Finished, so that leaves us with:

      ClientHello                  -------->
                                                      ServerHello
                                                     Certificate*
                                               ServerKeyExchange*
                                              CertificateRequest*
                                   <--------      ServerHelloDone
      Certificate*
      ClientKeyExchange
      CertificateVerify*
      [ChangeCipherSpec]
      Finished                     -------->
      Application Data

Which is the same number of messages as in your example. The problem
is now sending them before the 3-way handshake completes.

The Server's first message can be stuffed in the SYN/ACK payload, but
the ClientHello cannot be stuffed in a SYN option. But if you were
willing to define compressed versions of the most common ClientHello
options, you could just say in the SYN option "I speak package 9" and
then the server could send the ServerHello right away. You'd need to
move some of the ClientHello (e.g., the Random) to the Client's first
real message, but that's technically feasible.

-Ekr

---

From: "Adam Langley" <[email-masked]>
Date: Wed, 13 Aug 2008 13:07:45 -0700

> to the client if this option is included or not.

Yes, it absolutely has application level semantics, that's its purpose:

"Modifications to take advantage of SYNACK payloads would then require
changes to the application level protocol ... Because of this, we also
describe a TCP option that lets the application layer on both sides
... agree to use an alternative protocol which takes advantage of it."

> deeply implausible.

As I suggested, this data is considered secret by the organisations
who have it. But I can say that, had I suggested an opportunistic
system that required an extra round trip, various teams here (at
Google) would have (rightfully) laughed at me. I know that sucks as an
argument, but I can't release company data. I'm a bit stuck here
sadly.

> And the servers could say "use TLS next time" and the browsers could

And that's not a bad idea either. If I get around to it I might sketch
something like that out and see what the Mozilla folks think of it.

> real message, but that's technically feasible.

At which point, I think we have basically the same scheme, but with
different names!

ClientHello (1 bit)  ---->
<----- ServerHello, ServerHelloDone
ClientRandom, Finished, Application Data ----->

For implementation reasons I'm reducing the "I speak package 9" to a
single bit. The sockets API makes it very painful to have applications
process each SYN frame. One could imagine giving the kernel a list of
different payloads to send in the case of different options in the
SYN, but I didn't for complexity reasons. The major advantage would be
that we could more easily recover from primitive compromise but, as
I've outlined in a previous email, the tagged nature of the payload
makes it doable anyway.

We can't fit any certificates in the ServerHello and (unless we're
moving this into the kernel) the key agreement value has to be
constant. So we define a new cipher spec for a Diffie-Hellman function
that works in the face of constant public values and is very fast
(curve25519).

I, indeed, put the client's random in the first real message. I could
support client certificates with the tagged structure, currently I
don't.

So, given the space constraints, it's largely isomorphic, right? If
there's anything important that I've missed please put me straight.

Cheers

AGL

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 13 Aug 2008 13:36:07 -0700

At Wed, 13 Aug 2008 13:07:45 -0700,
> argument, but I can't release company data. I'm a bit stuck here

You're right, it does suck as an argument. And given that it's
superficially implausible for the reasons you elided, I don't feel the
need to entertain this as a serious design objective.

If/when you can come up with someone who is actually willing to
share this data, I'll be happy to reconsider that position.

-Ekr

---

From: "Adam Langley" <[email-masked]>
Date: Wed, 13 Aug 2008 14:40:02 -0700

> need to entertain this as a serious design objective.

I certainly don't agree that it's superficially implausible. I shall
ask someone senior here if I can release some of our latency affect
data, although I doubt the answer will be positive.

Remeber that > 40% of US homes are still using dial-up[1] and that's
leveling out. Even if you are within the US, the RTT across the
country on a dial-up is 300-400ms[2].

[1] http://www.websiteoptimization.com/bw/0804/
[2] http://www.rohwer.org/Latency/Latency9.html

AGL

---

From: Joe Touch <[email-masked]>
Date: Wed, 13 Aug 2008 15:15:22 -0700

Adam Langley wrote:
| On Wed, Aug 13, 2008 at 12:50 PM, Eric Rescorla
| <[email-masked]> wrote:
|> Right, but this has application-level semantics because the content
|> of the server's data is not what an unmodified client expects. So,
|> it's not just a timing issue: the server sends different data
|> to the client if this option is included or not.

This is, IMO, the most compelling argument that this mechanism ought to
be bound to a TCP port, not a TCP option.

Joe

---

From: "Adam Langley" <[email-masked]>
Date: Wed, 13 Aug 2008 15:25:38 -0700

> be bound to a TCP port, not a TCP option.

Although that removes any ability for it to be opportunistic since the
latency of trying an alternative port is very high. Many sites will
drop SYNs on ports other than 80 or 443. Even if the original port
were to advertise the alternative port for future connections, many
clients are behind firewalls that will restrict their outbound
connections.

AGL

---

From: Joe Touch <[email-masked]>
Date: Wed, 13 Aug 2008 15:32:36 -0700

Adam Langley wrote:
| On Wed, Aug 13, 2008 at 3:15 PM, Joe Touch <[email-masked]> wrote:
|> This is, IMO, the most compelling argument that this mechanism ought to
|> be bound to a TCP port, not a TCP option.
|
| Although that removes any ability for it to be opportunistic since the
| latency of trying an alternative port is very high. Many sites will
| drop SYNs on ports other than 80 or 443.

Sites that understand this option would be listening on the desired
ports. It's trivial to issue two SYNs and wait to see which ones are
returned.

Further, I'm not even sure how to write the application at the server -
one that responds with different data depending on signals in a SYN.
What does that even mean, and how does the TCP API (as specified in
RFC793, e.g.) need to be extended to support that?

| Even if the original port
| were to advertise the alternative port for future connections, many
| clients are behind firewalls that will restrict their outbound
| connections.

Firewalls and NATs also sometimes munge TCP options, especially ones
they don't understand, FWIW.

Joe

---

From: "Adam Langley" <[email-masked]>
Date: Wed, 13 Aug 2008 16:05:22 -0700

> ports. It's trivial to issue two SYNs and wait to see which ones are

If the two connections are made in parallel, and port 80 returns
first, how long do you wait for the other one? In this case, that
additional latency effects all *existing* sites. Hopefully the
difference would be small, but 5ms, 10ms? Also, in the case that both
return, the application probably can't close the socket in time for
the kernel not to ACK it, in which case clients would constantly be
opening very short lived connections to port 80.

I understand that application level changes based on options is a bit
discomforting but, personally, I find racing SYNs to be much worse.

> RFC793, e.g.) need to be extended to support that?

The existing patches make the change fairly easy. From the comments:

"A listening socket is configured with a setsockopt with non-zero payload len
and, optionally, TCP_SADATA_INC_NONCE and tcpsa_nonce_offset if the kernel
should include random data.

On a resulting, accepted socket, a getsockopt reveals:
  TCP_SADATA_REQ - a SYN/ACK payload was requested
  TCP_SADATA_NONCE - the 8 random bytes generated are in tcpsa_payload"

So the code looks like:

int newsocket = accept(...)
getsockopt(newsocket, SOL_TCP, &tcpsa, ...)
if (TCP_SADATA_REQ & tcpsa.tcpsa_flags) {
   // The first data from the socket is the client's draft-agl handshake
   decode_tagged_data(newsocket);
   ...
} else {
   // No special case,
   continue_as_normal();
}

> they don't understand, FWIW.

Certainly, and middleware which drops packets packets with the extra
option would be a concern. (The draft makes provisions for retrying
without the extra option in case implementations wish to do this).
However, hosts with restricted outbound port numbers are downright
common, sadly.

Cheers

AGL

---

From: Joe Touch <[email-masked]>
Date: Wed, 13 Aug 2008 17:51:18 -0700

Adam Langley wrote:
| On Wed, Aug 13, 2008 at 1:36 PM, Eric Rescorla
<[email-masked]> wrote:
|> You're right, it does suck as an argument. And given that it's
|> superficially implausible for the reasons you elided, I don't feel the
|> need to entertain this as a serious design objective.
|
| I certainly don't agree that it's superficially implausible. I shall
| ask someone senior here if I can release some of our latency affect
| data, although I doubt the answer will be positive.
|
| Remeber that > 40% of US homes are still using dial-up[1] and that's
| leveling out. Even if you are within the US, the RTT across the
| country on a dial-up is 300-400ms[2].

I think you're quoting >40% by assuming that anyone who doesn't use
broadband uses a modem (i.e., 100-57% = 43%), which isn't what those
charts indicate. The direct metric from citation [1] is Fig 2, which
shows less than 12% of homes using dialup right now, a figure that has
been steadily falling for the entire timeline shown.

[2] is a VERY old reference using a 10-year-old modem standard that
lasted only two years (1998-1999). V.92 superceded it in 1999.

I measured RTTs from LA to Boston, and found 200-220ms. Pings locally
were 140-150ms; the bulk of the increased ping delay was the modems.
Unless you're going through a satellite or a very inefficient multihop
radio system (e.g., anyone remember Metricom?), delays ought not exceed
around 150ms + broadband delays, e.g., they ought to top out around 250ms.

Joe

| [1] http://www.websiteoptimization.com/bw/0804/
| [2] http://www.rohwer.org/Latency/Latency9.html
|
| AGL
|
_______________________________________________
tcpm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tcpm

---

From: Joe Touch <[email-masked]>
Date: Wed, 13 Aug 2008 18:01:36 -0700

Adam Langley wrote:
...
| The existing patches make the change fairly easy. From the comments:
|
| "A listening socket is configured with a setsockopt with non-zero
payload len
| and, optionally, TCP_SADATA_INC_NONCE and tcpsa_nonce_offset if the kernel
| should include random data.
|
| On a resulting, accepted socket, a getsockopt reveals:
|   TCP_SADATA_REQ - a SYN/ACK payload was requested
|   TCP_SADATA_NONCE - the 8 random bytes generated are in tcpsa_payload"
|
| So the code looks like:
|
| int newsocket = accept(...)
| getsockopt(newsocket, SOL_TCP, &tcpsa, ...)
| if (TCP_SADATA_REQ & tcpsa.tcpsa_flags) {
|    // The first data from the socket is the client's draft-agl handshake
|    decode_tagged_data(newsocket);
|    ...
| } else {
|    // No special case,
|    continue_as_normal();
| }

I'm wondering why an implementation in user space would expect to find
out anything about a TCP connection that hadn't finished handshaking,
i.e., the accept call above (AFAIR) ought to return only after the end
of the three-way handshake (I'll have to dig out my Stephens book to
confirm, though):

	syn ->
		<- syn/ack
	ack ->
				--->accept returns

The above code would do what you intend only if the accept returns on
the initial SYN. At the very least, even if the accept returns when the
initial SYN is received, by the time the accept returns the syn/ack
ought to have been sent. unix doesn't wait for the accept call to
actually return the syn/ack - various discussions confirm this - or are
you also modifying that code?

Joe

---

From: "Adam Langley" <[email-masked]>
Date: Wed, 13 Aug 2008 18:21:08 -0700

> been steadily falling for the entire timeline shown.

I believe that the number of homes with broadband is, indeed, 57%.
(Although it's a fair point that some might be using's others
unsecured wireless connections). On another reading of the report, it
looks like this is the number of homes were broadband is /available/.
There are /two/ figures labeled "Figure 2", so I'll refer to them as
the first and second:

I think the first fig 2 includes the numbers of people who are using
broadband connections at work and the second figure 2 has a much
higher number because it counts active users. Homes with broadband are
more likely to use it, it seems.

But, you're right, I should be using the number of active users rather
than the number of homes with broadband. I knocked up a model for the
losses experienced by an example website based on an extra round trip
time. I'm still trying to get permission to release it.

> lasted only two years (1998-1999). V.92 superceded it in 1999.

Yes, but I didn't find anything very good that was more recent -
people don't seem to study modems much anymore. Also, the V.92
standard increased the upload speed at the determent of the download
speed. It's not clear that V.92 would have have much, if any, effect
on latency.

> around 150ms + broadband delays, e.g., they ought to top out around 250ms.

It's about 75ms from SF to NYC, so that makes sense. However, 250ms
still makes a big difference. Like I said, I'm trying to get the
numbers released.

AGL

---

From: "Adam Langley" <[email-masked]>
Date: Wed, 13 Aug 2008 18:27:04 -0700

> of the three-way handshake (I'll have to dig out my Stephens book to

Absolutely accept() only returns after the 3-way has completed.

Note that the draft only requires that the stack be able to configure
a static payload. Additionally, it suggests that it should be able to
insert some random bytes in there for cryptographic protocols that
need it.

Thus, the kernel is able to send the SYN/ACK (with payload) without
any application involvement. Thus, all the application needs to find
out, after an accept() is weather the kernel echoed the SYNACK Payload
Permitted option. If it did, then the server knows that it's SYNACK
payload was sent and the data from the client will be a tagged
structure that I don't actually define in the draft (since it's not
anything to do with TCP). Otherwise, the option wasn't echoed in which
case the protocol continues as normal.

AGL

---

From: Joe Touch <[email-masked]>
Date: Wed, 13 Aug 2008 22:11:31 -0700

Adam Langley wrote:
| On Wed, Aug 13, 2008 at 5:51 PM, Joe Touch <[email-masked]> wrote:
|> I think you're quoting >40% by assuming that anyone who doesn't use
|> broadband uses a modem (i.e., 100-57% = 43%), which isn't what those
|> charts indicate. The direct metric from citation [1] is Fig 2, which
|> shows less than 12% of homes using dialup right now, a figure that has
|> been steadily falling for the entire timeline shown.
|
| I believe that the number of homes with broadband is, indeed, 57%.

It's 57% of homes having broadband access. That does NOT imply that the
other 43% use dialup; some people still don't use the Internet, or (as
you note below) that all 57% have broadband turned on.

| (Although it's a fair point that some might be using's others
| unsecured wireless connections). On another reading of the report, it
| looks like this is the number of homes were broadband is /available/.
| There are /two/ figures labeled "Figure 2", so I'll refer to them as
| the first and second:
|
| I think the first fig 2 includes the numbers of people who are using
| broadband connections at work and the second figure 2 has a much
| higher number because it counts active users. Homes with broadband are
| more likely to use it, it seems.

Fig 2 is fairly conclusive to the point you're making; the ratio of
broadband to dialup - by house - is 88:12. You've noted that reducing a
single RTT is significant for dialup users; these figures argue that
such users are a small minority (12%), and perhaps this is thus
optimizing for a small, disappearing group.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 13 Aug 2008 22:20:51 -0700

Adam Langley wrote:
| On Wed, Aug 13, 2008 at 6:01 PM, Joe Touch <[email-masked]> wrote:
|> I'm wondering why an implementation in user space would expect to find
|> out anything about a TCP connection that hadn't finished handshaking,
|> i.e., the accept call above (AFAIR) ought to return only after the end
|> of the three-way handshake (I'll have to dig out my Stephens book to
|> confirm, though):
|
| Absolutely accept() only returns after the 3-way has completed.
|
| Note that the draft only requires that the stack be able to configure
| a static payload.

I'm confused now; your code - and your note. Let's go back to the key issue:

	- is the data received by the client dependent on the option?

Here's your code:

- ----

"A listening socket is configured with a setsockopt with non-zero
payload len
and, optionally, TCP_SADATA_INC_NONCE and tcpsa_nonce_offset if the kernel
should include random data.

On a resulting, accepted socket, a getsockopt reveals:
~  TCP_SADATA_REQ - a SYN/ACK payload was requested
~  TCP_SADATA_NONCE - the 8 random bytes generated are in tcpsa_payload"

So the code looks like:

int newsocket = accept(...)
getsockopt(newsocket, SOL_TCP, &tcpsa, ...)
if (TCP_SADATA_REQ & tcpsa.tcpsa_flags) {
~   // The first data from the socket is the client's draft-agl handshake
~   decode_tagged_data(newsocket);
~   ...
} else {
~   // No special case,
~   continue_as_normal();
}

- ---

The SERVER is the one in listen; this means that the server thinks there
is different data on the socket depending on whether the client has the
option set? That would mean that the SYN had the data, not the SYN-ACK.

Or am I misreading the code?

Further, how does the server put different code in the connection
depending on a socket option it can't see until after the listen? It
can't write to the socket until the listen completes, at which point the
SYN-ACK is gone.

It seems like you're going to end up needing a new call to write to the
socket before the listen completes, i.e., a setsockopt that effectively
'writes'. Again, this is --far-- afield from TCP semantics.

| Thus, the kernel is able to send the SYN/ACK (with payload) without
| any application involvement.

This is truly confusing to me. The payload **is** application
communication. You're creating an option to generate that data inside
the TCP protocol?

| Thus, all the application needs to find
| out, after an accept() is weather the kernel echoed the SYNACK Payload
| Permitted option. If it did, then the server knows that it's SYNACK
| payload was sent and the data from the client will be a tagged
| structure that I don't actually define in the draft (since it's not
| anything to do with TCP). Otherwise, the option wasn't echoed in which
| case the protocol continues as normal.

As a final question, if this is experimental, why not just use the
experimental TCP option KIND?

(and how much of this has been implemented, traced, and checked for
things like simultaneous open - I don't recall if you noted that before)

Joe

---

From: "Caitlin Bestler" <[email-masked]>
Date: Thu, 14 Aug 2008 11:27:13 -0400

There are several separable issues here. Each needs to be
considered on its own to determine if they should be addressed
by TCPM.

1) The sockets API forces applications to supply the necessary
   content for the first application message only after connection
   establishment has been signaled to the application layer.
   This typically results in an unneeded round-trip between the
   application domain and the TCP stack domain, which under most
   conventional Operating Systems is relatively expensive.

This is true. In addition to the problem cited, virtually all
client applications could supply the first request message in
the same call/request to initiate the connection.

But optimizing the local TCP stack interface can be done by
any TCP stack developer. It is a local interface. No IETF 
permission is required.

2) There are applications which would benefit from having
   optional application payload during connection setup.
   Such optional application payload is optionally provided
   by the client, and optionally provided to the server if
   the server requests it.

   This allows a "free upgrade" to the upper layer protocol,
   because *both* sender and receiver must be aware of the
   enhanced transport capability. If either is unaware, the
   protocol functions in compatibility mode based on the
   pre-existing definition of the TCP Stream.

While this is an application layer problem that any application
could have solved on its own, there is some merit in providing
this as a generic transport capability precisely because it enables
support of existing clients and servers. This is especially relevant
when the client obtains the server contact info through a URL
whose format and distribution is already fixed (meaning that the
URL cannot effectively be used to tell the client which way to
attempt the connection). As the draft points out, the goal of
the optional data is typically to expedite latency, therefore
trial and error probing for capabilities would undermine the
intended benefit.

The two strongest counter-arguments against supporting such an
option are 1) it's new, all new TCP features should be viewed
with suspicion and avoided unless their benefits are strong,
and 2) Designing a feature that deliberately throws away some
of the TCP payload stream seems contrary to the whole concept
of a reliable transport.

3) The presence of this optional payload should be signaled
   through TCP options.

If this is going to be done, this is the only way to do it.
The existing definition of TCP payloads encompasses all possible
byte streams. Therefore the alternate semantics must come from
outside of the TCP payload stream itself. For ULPs driven by
URLs (HTTP, SMTP, etc.) the TCP options are the only solution.

4) The size of the optional payload is what fits in the SYN ACK
   response.

I have a major problem with this. Having an option to indicate
that the TCP payload stream will start with a special header
that documents its own length would make a lot more sense to me.

The key problem here is that if the active side is not capable of
delivering SYN-ACK payload it will just not ACK that portion.
There is no guarantee that the resulting retransmit will be
segmented the same way.

---

From: "Adam Langley" <[email-masked]>
Date: Thu, 14 Aug 2008 09:40:46 -0700

>        - is the data received by the client dependent on the option?

I do apologise, the confusion is my fault:

There are two extensions interacting here. Firstly, there's an
extension to include data in the SYNACK. Then there's the application
layer signaling with the option. I was outlining the code for the
latter. The former is an extension that could be deployed today, it's
managed with a setsockopt on the server socket:

struct tcpsa tcpsa;
memcpy(tcpsa.tcpsa_payload, mysynackdata, length);
setsockopt(socket, SOL_TCP, TCP_SADATA, &tcpsa, sizeof(tcpsa));

listen(socket);

for (;;) { accept(); ... }

Obviously, this is just an implementation detail. It's nice for
sockets based APIs to do it like this. Since such stacks are dominant,
the idea of a constant payload appears in the draft. I make a couple
of exceptions to this. The ability to include a 64-bit nonce is very
useful and easy for the stack to manage. That's done with:

struct tcpsa tcpsa;
memcpy(tcpsa.tcpsa_payload, mysynackdata, length);
tcpsa.tcpsa_flags = TCP_SADATA_INC_NONCE;
tcpsa.tcpsa_nonce_offset = 0;
setsockopt(socket, SOL_TCP, TCP_SADATA, &tcpsa, sizeof(tcpsa));

Also, the ability to only send this data when the SYN includes the
SAPP option. At the moment, this is the default, although there is
space in the flags set aside for when I get round to it.

Example client side code and stack patches are referenced below.

Again, this is just to appease the sockets API. Stacks like [1] allow
the application to process the SYN directly and construct any SYNACK
payload they like on a per-packet basis.

[1] http://tservice.net.ru/~s0mbre/old/?section=projects&item=unetstack

> things like simultaneous open - I don't recall if you noted that before)

A linux patch[2] has been reviewed by the networking maintainer,
although I haven't asked for it to be applied yet because it's not
quite final. (It's using an experimental KIND number for one). Example
client side code example can be found at [3]. This includes preload
code to support Apache, Firefox etc (screenshot: [4]). It's been
tested by several people over the real Internet. (Note that [3] is
just a dump of my current code, it has many rough edges at the moment.
And ignore the TCP-AO stuff therein, it's #ifdef'ed out).

Also, I believe that I currently have the Diffie-Hellman speed record
with [5] since my implementation is 10% faster than the previously
claimed best[6]. (The version in [3] is much slower, I've yet to
update it.)

I'm am currently using an experimental KIND. However, if I were to,
say, deploy this on all the Google frontend servers (hypothetically,
of course), I suspect that would be in excess of what the experimental
kinds were for.

As for simultaneous open. The linux patch handles it by not doing
anything different. The option may be carried in the SYN, but the ACKs
are as normal. Since all BSD like stacks would, very likely, work the
same, it might be best to specify that behaviour in the draft.

Cheers

AGL

[2] http://marc.info/?l=linux-netdev&m=121857466907992&w=2
[3] http://www.imperialviolet.org/binary/libobstcp-0.2a.tar.bz2
[4] http://bp3.blogger.com/_kRcqvZJgJFs/SH0B_oPSlzI/AAAAAAAAAUM/eg-IxgbyhIs/s1600-h/firefox-obstcp.png
[5] http://code.google.com/p/curve25519-donna
[6] http://cr.yp.to/ecdh/reports.html

---

From: Joe Touch <[email-masked]>
Date: Thu, 14 Aug 2008 09:55:15 -0700

FWIW, IMO it is not wise to deploy code into a widely-used codebase that
uses the experimental KIND values. Experiments should be controlled.

See RFC4727, notably:

~   ... The network scope of support
~   for experimental values should carefully be evaluated before
~   deploying any experiment across extended network domains, such as the
~   public Internet.

Joe

Adam Langley wrote:
| On Wed, Aug 13, 2008 at 10:20 PM, Joe Touch <[email-masked]> wrote:
|> I'm confused now; your code - and your note. Let's go back to the key
issue:
|>
|>        - is the data received by the client dependent on the option?
|
| I do apologise, the confusion is my fault:
|
| There are two extensions interacting here. Firstly, there's an
| extension to include data in the SYNACK. Then there's the application
| layer signaling with the option. I was outlining the code for the
| latter. The former is an extension that could be deployed today, it's
| managed with a setsockopt on the server socket:
|
| struct tcpsa tcpsa;
| memcpy(tcpsa.tcpsa_payload, mysynackdata, length);
| setsockopt(socket, SOL_TCP, TCP_SADATA, &tcpsa, sizeof(tcpsa));
|
| listen(socket);
|
| for (;;) { accept(); ... }
|
| Obviously, this is just an implementation detail. It's nice for
| sockets based APIs to do it like this. Since such stacks are dominant,
| the idea of a constant payload appears in the draft. I make a couple
| of exceptions to this. The ability to include a 64-bit nonce is very
| useful and easy for the stack to manage. That's done with:
|
| struct tcpsa tcpsa;
| memcpy(tcpsa.tcpsa_payload, mysynackdata, length);
| tcpsa.tcpsa_flags = TCP_SADATA_INC_NONCE;
| tcpsa.tcpsa_nonce_offset = 0;
| setsockopt(socket, SOL_TCP, TCP_SADATA, &tcpsa, sizeof(tcpsa));
|
| Also, the ability to only send this data when the SYN includes the
| SAPP option. At the moment, this is the default, although there is
| space in the flags set aside for when I get round to it.
|
| Example client side code and stack patches are referenced below.
|
| Again, this is just to appease the sockets API. Stacks like [1] allow
| the application to process the SYN directly and construct any SYNACK
| payload they like on a per-packet basis.
|
| [1] http://tservice.net.ru/~s0mbre/old/?section=projects&item=unetstack
|
|> As a final question, if this is experimental, why not just use the
|> experimental TCP option KIND?
|>
|> (and how much of this has been implemented, traced, and checked for
|> things like simultaneous open - I don't recall if you noted that before)
|
| A linux patch[2] has been reviewed by the networking maintainer,
| although I haven't asked for it to be applied yet because it's not
| quite final. (It's using an experimental KIND number for one). Example
| client side code example can be found at [3]. This includes preload
| code to support Apache, Firefox etc (screenshot: [4]). It's been
| tested by several people over the real Internet. (Note that [3] is
| just a dump of my current code, it has many rough edges at the moment.
| And ignore the TCP-AO stuff therein, it's #ifdef'ed out).
|
| Also, I believe that I currently have the Diffie-Hellman speed record
| with [5] since my implementation is 10% faster than the previously
| claimed best[6]. (The version in [3] is much slower, I've yet to
| update it.)
|
| I'm am currently using an experimental KIND. However, if I were to,
| say, deploy this on all the Google frontend servers (hypothetically,
| of course), I suspect that would be in excess of what the experimental
| kinds were for.
|
| As for simultaneous open. The linux patch handles it by not doing
| anything different. The option may be carried in the SYN, but the ACKs
| are as normal. Since all BSD like stacks would, very likely, work the
| same, it might be best to specify that behaviour in the draft.
|
|
| Cheers
|
|
| AGL
|
| [2] http://marc.info/?l=linux-netdev&m=121857466907992&w=2
| [3] http://www.imperialviolet.org/binary/libobstcp-0.2a.tar.bz2
| [4]
http://bp3.blogger.com/_kRcqvZJgJFs/SH0B_oPSlzI/AAAAAAAAAUM/eg-IxgbyhIs/s1600-h/firefox-obstcp.png
| [5] http://code.google.com/p/curve25519-donna
| [6] http://cr.yp.to/ecdh/reports.html
|
|
_______________________________________________
tcpm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tcpm

---

From: "Adam Langley" <[email-masked]>
Date: Thu, 14 Aug 2008 09:59:50 -0700

> uses the experimental KIND values. Experiments should be controlled.

Agreed. I've not requested that the Linux patch be applied, primarily
because it's using an experimental kind number. That's also way I'm
requesting a real option number.

Cheers

---

From: Joe Touch <[email-masked]>
Date: Thu, 14 Aug 2008 10:05:16 -0700

Adam Langley wrote:
| Obviously, this is just an implementation detail.

It's a lot more than that; it hints that the socket interface is
insufficient for using the available TCP capability for sending data
with either SYNs or SYN-ACKs. I'm wondering how/if this has been
explored before, and whether there would be a solution that maps to
TCP's model of a stream better.

The socket option above is incorrect, IMO. It implies that the user
should have control over data that ends up in the SYN-ACK directly; IMO,
TCP's stream model should allow only that data be written to the stream
early - which may require a socket option - BUT should *not* imply that
the data maps to the SYN-ACK directly.

E.g., what happens if the data is larger than the MSS? Or if the SYN-ACK
comes back with an ICMP 'too-big'? I.e., this should be called just
TCP_DATA, not TCP_SADATA.

| It's nice for

I do not understand the semantics of a TCP option that changes the TCP
stream. TCP is a stream-oriented protocol, and each connection provides
one stream. If you want a separate channel, you ought to be using a
different protocol, e.g., SCTP.

That's fine as an implementation decision, as Caitlin notes, but has no
business affecting the TCP protocol spec.

Joe

---

From: "Adam Langley" <[email-masked]>
Date: Thu, 14 Aug 2008 10:14:51 -0700

If I might address your concern first:

> There is no guarantee that the resulting retransmit will be

I've not included any details of the application level data in the
draft because I didn't think it to be in scope for TCPM. However, the
payload is self deliminating. It has to be because, if you are a
client application using the sockets API you might not run for an
arbitary amount of time. In that time, the server can have sent any
amount of data and there's no deliminators in the bytestream from the
socket.

If you are interested, I've included by current notes on the format at
the end of this email. (It's certainly not set in stone). However, the
draft does require a minimal level of support in conforming stacks to
allow applications to decide not to use the ability unless they know
that it has certain charactoristics. That minimal level is: the active
open side must be able to enqueue 64 bytes of payload from a SYN/ACK
and that the server be able to include 64-byte of data in a SYN/ACK.

I'll skip the points with which I agree with entirely.

> of the TCP payload stream seems contrary to the whole concept

Here you are thinking about including SYN/ACK payloads when many
clients will ignore them? It does have some inefficiency problems,
I'll agree. However, I mostly expect that SYN/ACK payloads will only
be returned to clients which have explicitly requested it with an
option. There are certainly some server-speaks-first protocols (like
SMTP) which could oppitunistically use SYN/ACK payloads. Then the
inefficiency decision has to be made on a per application basis,
probably a config option somewhere.

Cheers

--- Current notes on the possible format of the start of the
application data streams ---

This is nothing to do with TCP. It describes an extensible, self
deliminating format which is expected to prefix the usual application
data (say HTTP) in both directions:

// The format is like a TCP options structure. We wish to have an extensible
// set of options.
//
// The payload is in two sections. First, a bit-packed list of option numbers
// and payload lengths. Second, byte aligned, the payloads of all those
// options, concatenated.
//
// Numbers in the first part are encoded thus:
//
// The number of bits needed is encoded in uniary form, terminated by a zero
// bit, followed by that many bits.
//
// Thus, 0 is encoded with a single 0 bit.
// One is encoded by specify that one bit is needed with bits 10, followed by
// one bit of content. This bit is actually a zero. Since we already know that
// the number isn't 0, we bias the payload by one.
//
// Likewise, four is encoded with 110 to specify two bits, followed by 01.
// Since the 110 means that the smallest number that it could be is three, we
// only need to add one to make four.
//
// The first part of the payload is then a list of (kind number delta, payload
// length) tuples, where both elements are numbers, encoded as above. The kind
// numbers MUST appear in order, smallest first. The delta of the tuple
// specifies the difference between the current kind number and the next one.
// The current kind number is initially 0. A delta of 0 terminates the list.
// Such a delta doesn't have a payload length following it.
//
// Kinds which are a power of two (1, 2, 4, 8, ...) are defined to be protocol
// agnostic. Other kind numbers are availible for per-protocol options.
//
// Kinds 1 and 2 are special in that they have no payload length encoded. The
// payload length of kind 1 is defined to be 40 bytes and the payload length of
// kind 2 is defined to be 0 bytes.
//
// After all the tuples have been encoded, 0 bits are added to align to the
// next 8-bit boundary. Then the payloads are added, in order.
//
// If the end of the first section and the end of the whole bytestring
lie within the same 4-byte word, 0 bytes are appened to pad to the
next 4 byte word.

---

From: "Adam Langley" <[email-masked]>
Date: Thu, 14 Aug 2008 10:23:31 -0700

> TCP_DATA, not TCP_SADATA.

There are several cases:

If the setsockopt has requested that the data be included in the
stream without exception, then it is enqueued like any other data. If
there's space in the SYN/ACK, it can go in there. If not, it will be
sent as usual. If the client doesn't ACK a SYNACK payload, it will be
retransmitted etc. This requires no standards effort, it's a TODO for
me at the moment.

If the setsockopt flags request that the data only be sent in reply to
SYNs with the SAPP option:

If the payload fits in the SYNACK (given MSS etc) then it is
transmitted and the option is echoed back. Otherwise, a bare SYNACK
(without the option) is sent back. In the latter case, the data is not
enqueued to be sent in the future. The applications on both sides are
aware of what happened and proceed correctly given the situation.

> different protocol, e.g., SCTP.

It's not a separate channel, it's just that the option is an
application level signal. Caitlin probably puts it's better than I do:

"While this is an application layer problem that any application
could have solved on its own, there is some merit in providing
this as a generic transport capability precisely because it enables
support of existing clients and servers."

Cheers

AGL

---

From: Joe Touch <[email-masked]>
Date: Thu, 14 Aug 2008 10:30:59 -0700

Adam Langley wrote:
| On Thu, Aug 14, 2008 at 10:05 AM, Joe Touch <[email-masked]> wrote:
|> It's a lot more than that; it hints that the socket interface is
|> insufficient for using the available TCP capability for sending data
|> with either SYNs or SYN-ACKs. I'm wondering how/if this has been
|> explored before, and whether there would be a solution that maps to
|> TCP's model of a stream better.
|>
|> The socket option above is incorrect, IMO. It implies that the user
|> should have control over data that ends up in the SYN-ACK directly; IMO,
|> TCP's stream model should allow only that data be written to the stream
|> early - which may require a socket option - BUT should *not* imply that
|> the data maps to the SYN-ACK directly.
|>
|> E.g., what happens if the data is larger than the MSS? Or if the SYN-ACK
|> comes back with an ICMP 'too-big'? I.e., this should be called just
|> TCP_DATA, not TCP_SADATA.
|
| There are several cases:
|
| If the setsockopt has requested that the data be included in the
| stream without exception, then it is enqueued like any other data. If
| there's space in the SYN/ACK, it can go in there. If not, it will be
| sent as usual. If the client doesn't ACK a SYNACK payload, it will be
| retransmitted etc. This requires no standards effort, it's a TODO for
| me at the moment.

OK. So why bother with an option at that point? Why does the user data
need to be different if the option is present?

If the user data were not different, then this all boils down to a
change in the implementation of the API. The trouble is that you're
expecting a TCP option to change the *content* of the data stream, and
that seems bizarre to me.

| If the setsockopt flags request that the data only be sent in reply to
| SYNs with the SAPP option:
|
| If the payload fits in the SYNACK (given MSS etc) then it is
| transmitted and the option is echoed back. Otherwise, a bare SYNACK
| (without the option) is sent back. In the latter case, the data is not
| enqueued to be sent in the future. The applications on both sides are
| aware of what happened and proceed correctly given the situation.

This is a big change - you're saying that if the payload fits, it goes
in the SYN-ACK. If not, *none* of it is sent in the SYN-ACK? Why?

TCP provides a *byte stream*. The description above implies that the
application knows/sees segment boundaries, which isn't TCP anymore.

Joe

---

From: "Caitlin Bestler" <[email-masked]>
Date: Thu, 14 Aug 2008 13:44:36 -0400

> probably a config option somewhere.

My reservation is entirely theoretical.

The "overhead" of 64 data bytes on a SYN/ACK frame that is being
sent anyway is very slight or even zero, depending on exactly
what forms of intermediate queuing are in use.

The real concern would be if applications that did not use the
special sockopts were to have this special payload automatically
discarded by the stack.

Automatic discarding of payload, no matter how efficient, just
strikes me as contrary to the definition of what it means to
be a "reliable transport" service.

And experience has taught me that when something looks really
bad from a "purely theoretical" viewpoint it merely means that
I have not yet identified the specific case where it *will*
cause problems. So at the minimum I like to come up with
a strong reason why something is intrinsically safe, not
just "So far, I haven't thought of how it goes wrong."

---

From: "Adam Langley" <[email-masked]>
Date: Thu, 14 Aug 2008 13:03:28 -0700

> that seems bizarre to me.

Because for server-speaks-first application protocols there's no
problem. This is just for client-speaks-first protocols. For these
protocols, had common stacks been able to send data in the SYN/ACK
when they were being designed, then they might have ended up as
server-speaks-first protocols. But they didn't for latency reasons.

In order for them to take advantage of this they need to be able to
signal their compliance somehow. There is, indeed, a cleanliness
argument against such a option, but I'm arguing that it's worthwhile
for backward compatibility reasons.

> in the SYN-ACK. If not, *none* of it is sent in the SYN-ACK? Why?

I'm saying that implementations have latitude here. If the passive
open end wishes to ignore the client's option it may. Equally, it may
echo it, but not include the data in the SYNACK if the MTU makes it
troublesome. In that case, it needs to send it in the next packet as
normal. Since the implementation has the best knowledge of what's
best, I didn't see any reason to constrain it.

AGL

---

From: Joe Touch <[email-masked]>
Date: Thu, 14 Aug 2008 13:12:41 -0700

Adam Langley wrote:
| On Thu, Aug 14, 2008 at 10:30 AM, Joe Touch <[email-masked]> wrote:
|> OK. So why bother with an option at that point? Why does the user data
|> need to be different if the option is present?
|>
|> If the user data were not different, then this all boils down to a
|> change in the implementation of the API. The trouble is that you're
|> expecting a TCP option to change the *content* of the data stream, and
|> that seems bizarre to me.
|
| Because for server-speaks-first application protocols there's no
| problem. This is just for client-speaks-first protocols. For these
| protocols, had common stacks been able to send data in the SYN/ACK
| when they were being designed, then they might have ended up as
| server-speaks-first protocols. But they didn't for latency reasons.

Client-speaks-first puts data in the SYN, not the SYN-ACK. Or have you
got the roles reversed, as with FTP data connections?

| In order for them to take advantage of this they need to be able to
| signal their compliance somehow. There is, indeed, a cleanliness
| argument against such a option, but I'm arguing that it's worthwhile
| for backward compatibility reasons.
|
|> This is a big change - you're saying that if the payload fits, it goes
|> in the SYN-ACK. If not, *none* of it is sent in the SYN-ACK? Why?
|
| I'm saying that implementations have latitude here. If the passive
| open end wishes to ignore the client's option it may. Equally, it may
| echo it, but not include the data in the SYNACK if the MTU makes it
| troublesome. In that case, it needs to send it in the next packet as
| normal.

If the option is ignored, and you don't intend to change TCP semantics,
then the data - i.e., the hash or whatever - needs to be sent in the
next data packet.

Joe

---

From: "Adam Langley" <[email-masked]>
Date: Thu, 14 Aug 2008 13:25:08 -0700

> got the roles reversed, as with FTP data connections?

Sorry, I was speaking of the application layer protocol. Commonly,
client-speaks-first application protocols are putting data in the
final ACK of the 3-way handshake.

> then the data - i.e., the hash or whatever - needs to be sent in the

Ok, I'm fine with that. But there will always be servers which ignore
the option because they don't understand it. In that case, obviously,
they can't send it in the next packet.

AGL

---

From: Joe Touch <[email-masked]>
Date: Thu, 14 Aug 2008 13:36:45 -0700

Adam Langley wrote:
| On Thu, Aug 14, 2008 at 1:12 PM, Joe Touch <[email-masked]> wrote:
|> Client-speaks-first puts data in the SYN, not the SYN-ACK. Or have you
|> got the roles reversed, as with FTP data connections?
|
| Sorry, I was speaking of the application layer protocol. Commonly,
| client-speaks-first application protocols are putting data in the
| final ACK of the 3-way handshake.
|
|> If the option is ignored, and you don't intend to change TCP semantics,
|> then the data - i.e., the hash or whatever - needs to be sent in the
|> next data packet.
|
| Ok, I'm fine with that. But there will always be servers which ignore
| the option because they don't understand it. In that case, obviously,
| they can't send it in the next packet.

Please walk through this again. AFAIK, "client" refers to the initiator
of the first connection. Let's use that convention here:

client sets socket option before connect
		server sets socket option after listen, before accept?
		(again, this is a problem; the socket of the connection
		exists only after the accept returns; before that, the
		'socket' applies to *any* connection that would return
		a subsequent listen call)

1.	SYN->
2.		<- SYN/ACK
3.	ACK->
4.	data->
5.		<- data

The option you're talking is useful, AFAICT, only in #2 (the SYN/ACK).

I'm proposing that the data that the server sends in the socket option
be considered equivalent to a WRITE on the socket - which would
otherwise have been possible only after the ACCEPT call. I.e.:

	- if the SYN (#1) has the option, then the socket option
	data commences in the SYN/ACK (#2) [it could spill over into
	later data segments #5 (or even between steps 3 and 4)]

	- if the SYN (#1) lacks the option, then the socket option
	data commences in the data segments #5 (or between steps
	3 and 4)

At that point, there is no real need for an option. The server can
ALWAYS send the data in the SYN/ACK, and just resend it if the ACK (#3)
is just for the SYN (in the SYN/ACK #2).

Isn't it a *lot* simpler to avoid an option and just send the data anyway?

Joe

---

From: "Adam Langley" <[email-masked]>
Date: Thu, 14 Aug 2008 13:41:20 -0700

> just "So far, I haven't thought of how it goes wrong."

I haven't been able to think of any case where it goes horribly wrong yet ;)

So, we're considering the case where there are no options in play.
This is a pure implementation trick.

To make it concrete, we have an SMTP server which puts "220
example.com ESMTP\r\n" in the SYN/ACK.

If the client ACKs less than the full payload, we retransmit the
remaining bytes in the next packet.

That's not a strong reason, but I'll ponder how it could go wrong
again tonight. If you think of anything, please do say.

Cheers

AGL

---

From: "Adam Langley" <[email-masked]>
Date: Thu, 14 Aug 2008 14:01:05 -0700

> Isn't it a *lot* simpler to avoid an option and just send the data anyway?

Yep! And I intend to patch Linux to support exactly this (no options
involved anywhere) at some point. To reuse an example, I could have an
SMTP server which writes "200 example.com ESMTP\r\n" this way and save
a round trip. (Although SMTP isn't too latency sensitive some busy
servers might be glad of it.)

But many application level protocols have been designed as
client-speaks-first, like HTTP. So, let's define HTTP-ssf (server
speaks first). HTTP-ssf is exactly the same as HTTP, except that the
client waits for a banner from the HTTP server before sending a banner
of it's own. We can use the same trick as the SMTP server to maybe fit
this banner in the SYNACK. If it doesn't fit, no loss, it can go in
later packets. If the client doesn't ACK all the data from the SYNACK,
no loss, again it goes in the next packets.

Now, I can run an HTTP-ssf server, but no web browser on the planet
can talk to me! And the browsers that I modify to speak HTTP-ssf can't
take to any non-ssf HTTP servers. It's a very lonely website.

I could define a whole new URL scheme (httpssf://...) and a new port.
But then there's no gradual deployment path because users with old
browsers don't understand httpssf:// and it confuses the users anyway.
Most type http:// by habit.

My HTTP-ssf server happens to be an embedded device and it handles the
TCP processing directly. So, I have browsers that understand HTTP-ssf
set an option in the SYN packet. If I see this option, I start talking
HTTP-ssf, otherwise I stick with HTTP. When I start talking HTTP-ssf,
I echo the option to let the client know which protocol is in effect.
Now HTTP-ssf can be deployed gradually.

In order to run HTTP-ssf servers on non-embedded machines I define a
couple of (get|set)sockopt's to set and query the option so I know
what protocol to speak. I also want to include the banner in the
SYNACK for latency reasons, so I include another setsockopt to set a
constant banner that's prepended to the beginning of the server's
outgoing data stream. However, the kernel needs to know only to
prepend this banner if the client include the option, so I add that
ability too.

Now my server gets SYN flooded and, to mitigate the problem, it
doesn't want to send large SYNACKs out, even when the client has
requested them. It can send minimal SYNACKs and put the banner in
later packets. However, user's hate it when their websites slow down
so, rather than do that, the kernel can also ignore the requests and
fall back to HTTP.

(Sorry, I don't mean to be condescending in the slightest! I just got
carried away with the personification there. However, I don't think
the clarity suffered for it, so I'm going to stick with it. I hope you
don't mind)

AGL

---

From: Joe Touch <[email-masked]>
Date: Thu, 14 Aug 2008 14:25:50 -0700

Adam Langley wrote:
| On Thu, Aug 14, 2008 at 1:36 PM, Joe Touch <[email-masked]> wrote:
|> Isn't it a *lot* simpler to avoid an option and just send the data
anyway?
|
| Yep! And I intend to patch Linux to support exactly this (no options
| involved anywhere) at some point. To reuse an example, I could have an
| SMTP server which writes "200 example.com ESMTP\r\n" this way and save
| a round trip. (Although SMTP isn't too latency sensitive some busy
| servers might be glad of it.)
|
| But many application level protocols have been designed as
| client-speaks-first, like HTTP.

OK, so the crux of the matter is that:

	- most app protocols are client-speaks first
	- you want server-speaks-first

There's nothing about TCP that prevents that, but I agree that the
current socket interface isn't quite sufficient.

| So, let's define HTTP-ssf (server
| speaks first). HTTP-ssf is exactly the same as HTTP, except that the
| client waits for a banner from the HTTP server before sending a banner
| of it's own. We can use the same trick as the SMTP server to maybe fit
| this banner in the SYNACK. If it doesn't fit, no loss, it can go in
| later packets. If the client doesn't ACK all the data from the SYNACK,
| no loss, again it goes in the next packets.

Yup. That works.

| Now, I can run an HTTP-ssf server, but no web browser on the planet
| can talk to me! And the browsers that I modify to speak HTTP-ssf can't
| take to any non-ssf HTTP servers. It's a very lonely website.

YES. You have a problem because you're redefining HTTP, not because of
anything in TCP. FWIW, there's an HTTPbis WG in which this issue would
be relevant.

| I could define a whole new URL scheme (httpssf://...) and a new port.
| But then there's no gradual deployment path because users with old
| browsers don't understand httpssf:// and it confuses the users anyway.
| Most type http:// by habit.

So let's consider secure HTTP. That could have been done with a TCP
option, but instead it was done in exactly the way you don't think works
- - a new port, and a new URI prefix (https:). Initially, users needed to
know what to type; eventually, that ended up being embedded in URLs on
web pages people click anyway (e.g., search engine results).

OK, so why doesn't that path suffice? It worked once, why won't it work
again?

- ----

What you're *really* asking for is a way to send data from client to
server BEFORE the TWHS completes. You talk about server-talks-first, but
it's really the client speaking - by sending the option. If this were a
flag in the HTTP request, and if the data were available to the server
app when the SYN arrives, you'd be done. You're using the option as a
trick to get around the fact that TCP doesn't want to send user data up
to the app until the end of the TWHS.

I don't think that sort of channel is something TCP needs. IMO, this is
precisely what T/TCP was designed for; T/TCP may need fixing, but that's
a different issue. I'm not familiar enough with SCTP to know if that's
an alternate path.

I don't support the idea that doing this in TCP is appropriate just to
get around deployment issues.

Joe

---

From: "Adam Langley" <[email-masked]>
Date: Thu, 14 Aug 2008 14:43:03 -0700

> web pages people click anyway (e.g., search engine results).

HTTPS was slightly different because it had to be resistant to
downgrade attacks - it couldn't have been done as an option for that
reason.

However, I'd cite HTTPS as an example that new uri schemes don't work.
Despite widespread support, far too few people use it. There are other
user interaction problems with TLS (and all other secure protocols) to
be sure (user's will click past invalid certificate warnings etc), but
having to know what "https" is certainly one of them. (One's mother is
the canonical example here, but you can confirm this by asking random,
non-technical people)

> it's really the client speaking - by sending the option.

Hmm, yea. That's a fair view of matters. I'm sending a single bit from
the client in the SYN.

> get around deployment issues.

Ok. You've certainly understood the issues, so that's reasonable. If
others feel the same way I shall withdrawl the options request and
figure out another path.

Thanks for all your time, Joe.

AGL

---

From: Michael Tüxen <[email-masked]>
Date: Fri, 15 Aug 2008 00:04:35 +0200

On Aug 14, 2008, at 11:25 PM, Joe Touch wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>
> Adam Langley wrote:
SCTP has a four way handshake:
---- INIT ---->
<-- INIT-ACK --
--- COOKIE --->
<- COOKIE-ACK -

You can send user data with the third and fourth message, not
with the first or second.

The socket API supports sending user data on the third message...
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAkikot4ACgkQE5f5cImnZrvyWACgtAlcCP3R5UijYW+IpRc2x3VX
> Rw0AoPiUJhbWhD2cWKliTRFrhtINfYai
> =pX3/
> -----END PGP SIGNATURE-----

---

From: "Caitlin Bestler" <[email-masked]>
Date: Thu, 14 Aug 2008 18:44:11 -0400

> Now HTTP-ssf can be deployed gradually.

This is an excellent summary of the core problem. It also summarizes
why SCTP is not the solution. An httpsctp protocol would be far
superior to an httpssf protocol, but it would be just as lonely.

So to translate your application level problem into transport layer
terms, you want to subdivide the class TCP payload stream into
a finite-sized "banner" that is at the start of the stream and
is followed by the traditional byte stream payload.

Further, you want to establish both bannerless and banner-enabled
connections using the same server-side listen. That is:

- When a banner-unaware client attempts to connect to a banner-capable
  server, everything will work. The server will know that the client 
  did not provide a banner, and the client will never see a banner
  that it does not know what to do with.
- When a banner-aware client attempts to connect to a banner-unaware
  server, everything will work because the client does not send a banner
  until it has seen the server's TCP options. The client will be told
  that the connection was accepted by a banner-unaware server, and the
  server will think it has a traditional client.
- When a banner-aware client attempts to connect to a banner-capable
  server they will actually exchange banners, and the banner payloads
  will be delivered to the client and server applications in a manner
  that is separate from the normal payload.

The rationale for doing this in TCP is that the total application
infrastructure cannot steer banner-aware clients to an alternate port
due to the class chicken-and-egg problem.

   If only 1% of clients understand banner-enabled HTTP then no sites
   will be encoded to use the special URL indicating banner capability.

   Since no sites use the banner indicating URL there is no reason for
   new clients to add that capability.

So there *is* rationale for doing this at the transport layer. Whether
there is *sufficient* rationale is subject to debate. My key reservation
remains that this breaks a fundamental concept of TCP. Instead of the
TCP payload being "a stream of bytes" it is now "an optional banner
message sent only when the peer's TCP options indicate it can be
parsed followed by a stream of bytes".

I've never been a fan of the "stream of bytes" model (I far prefer
the SCTP "stream of messages" model), but it is very simple and it
is what TCP is. I'm not sure the improvements cited here are reaching
a threshold sufficient to justify the required extra conceptual
complexity.

---

From: "Adam Langley" <[email-masked]>
Date: Thu, 14 Aug 2008 16:35:47 -0700

> parsed followed by a stream of bytes".

Just to correct one point - the banners are still seen as part of the
stream of bytes by both ends. The option communicates a single bit,
out of band, to both ends. I think that's the problem.

So it looks like the concenous for this is pretty thin.

(Un)fortunately, I'm persistant as hell, so I shall consider what
other avenues I have. I still firmly believe that a world with
opportunistic encryption would be a better world than we have now.

Cheers all,

AGL

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 14 Aug 2008 19:03:45 -0700

At Thu, 14 Aug 2008 16:35:47 -0700,
> opportunistic encryption would be a better world than we have now.

I believe this too. But there are lots of ways to get opportunistic
encryption that don't involve screwing with TCP. For starters,
there could simply be a header that said "You can use TLS to
talk to me as well" and supporting clients could redirect.
The rationale for this mechanism is opportunistic encryption PLUS
lower latency than currently offered by TLS. It's the second
piece you haven't made the case for.

-Ekr

---

From: Michael Tüxen <[email-masked]>
Date: Fri, 15 Aug 2008 09:14:31 +0200

> talk to me as well" and supporting clients could redirect.
In SCTP, RFC 5061, there is a features called 'Adaptation Layer  
Indication',
which could be used to signal this kind of information during the
handshake...

---

From: "Adam Langley" <[email-masked]>
Date: Fri, 15 Aug 2008 07:57:49 -0700

On Fri, Aug 15, 2008 at 4:09 AM, Anders Waldenborg <[email-masked]> wrote:
> Doesn't SA payloads open up for DoS-amplification (attacker would send SYN
> packets to your SMTP server with spoofed source address to send more data to
> victim then would be possible by sending them directly to victim)?

It's certainly a consideration, which is why the draft recommended
that 64 bytes be the maximum payload size. At this size there are more
juicy targets for amplification, like DNS servers.

Cheers

AGL

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Aug 2008 09:07:33 -0700

Adam Langley wrote:
...
| Thanks for all your time, Joe.

And thanks to you (and others on the list) for taking the time to
explain this in detail.

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Aug 2008 09:34:11 -0700

> juicy targets for amplification, like DNS servers.

FWIW, since this is payload data, the size of the data is an
implementation detail - unless, as I've noted, you're changing the
semantics of TCP from a byte stream to delineated messages.

(yes, you're already changing the semantics from one where connection
info is available only after TWHS to one that allows info to be used by
the app before the TWHS completes, which is a different issue)

Joe

---

From: "Caitlin Bestler" <[email-masked]>
Date: Fri, 15 Aug 2008 12:56:24 -0400

> the app before the TWHS completes, which is a different issue)

If this were to move forward, the Security Considerations would need
to document the amplification attack, and probably have SHOULD language
about a default maximum packet size.

But this is probably not something that the TCP stack should enforce.
There are many situations where the system administrator would have
valid reasons for knowing that a DoS was already blocked by other means.

---

From: "Adam Langley" <[email-masked]>
Date: Fri, 15 Aug 2008 10:09:27 -0700

> about a default maximum packet size.

The 01 draft says this in the security considerations:

"  Any payload in a SYNACK packet must be as frugal as possible since a
   host will be transmitting it to an unconfirmed address.  If a 40 byte
   frame could elicit a 1500 byte reply to an attacker controlled
   address, this would be readily used to hide and amplify distributed
   denial of service attacks."

> valid reasons for knowing that a DoS was already blocked by other means.

You're correct here, the current MUST should be downgraded to a
SHOULD. Implementations can make the best decision here.

As for it moving forward, I don't feel that I would ever get consensus
for the current design. (Although I don't really know how these things
work, I believe that's required.) I shall ponder other means this
weekend, but a quick discussion amongst those who (ab)use TCP stacks
for our internal networks didn't come up with anything.

Cheers

AGL

---

From: "Anantha Ramaiah (ananth)" <[email-masked]>
Date: Thu, 21 Aug 2008 14:02:28 -0700

> | getsockopt(newsocket, SOL_TCP, &tcpsa, ...) if (TCP_SADATA_REQ & 
> | tcpsa.tcpsa_flags) {
> I'm wondering why an implementation in user space would 
> expect to find out anything about a TCP connection that 
> hadn't finished handshaking, i.e., the accept call above 

This has been done in some stacks today (like ours). The incoming SYN is
reported to the application and the application can suggest whether or
not to accept the connection. If the application gives a nod, proceed
with the 3 way handshake else send an RST. This is different from a 3
way handshake happening (which involves the TCB and the associated
baggage creation) and then informing the application and applications
can reject this connection during accept(). Applications wishing to use
this special property would set a socket option indicating the
notification of SYN when the listener is created.

The advantage is that you don't allocate the memory ahead of time if the
application isn't interested in accepting this connection. Other ways of
handling this is to have the application program the access control list
in TCP or below, so that an early discard can be made.

-Anantha

---

From: Joe Touch <[email-masked]>
Date: Thu, 21 Aug 2008 17:01:31 -0700

Hi, Ananth,

Anantha Ramaiah (ananth) wrote:
> with the 3 way handshake else send an RST. 

Does the app have access to the SYN options? If so, that seems outside
the scope of what TCP should be exposing to the app.

> This is different from a 3
> application isn't interested in accepting this connection.

It seems to me the app already sufficiently controls this in how/when it
issues a accept(). On what basis is it making the rejection at the time
of a SYN? (if by IP address, it could have done this via a different
listen() call).

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 22 Aug 2008 10:04:32 -0700

Hi, Ananth,

Anantha Ramaiah (ananth) wrote:
>  
> <snip>
> 
>>>> I'm wondering why an implementation in user space would expect to 
>>>> find out anything about a TCP connection that hadn't finished 
>>>> handshaking, i.e., the accept call above
>>> This has been done in some stacks today (like ours). The 
>> incoming SYN 
>>> is reported to the application and the application can 
>> suggest whether 
>>> or not to accept the connection. If the application gives a nod, 
>>> proceed with the 3 way handshake else send an RST.
>> Does the app have access to the SYN options? If so, that 
>> seems outside the scope of what TCP should be exposing to the app.
> 
> Ok, currently we don't pass TCP options upwards, we just pass the 4
> tuple (may be 5 tuple if you include VRF's) to the application. The
> application can decide whether to move forward with this connection or
> not. This is simply the access control at the application level for the
> incoming SYN.

That's roughly equivalent to having better resolution of listen() calls,
which seems OK.

(what's a VRF?)

>>> can reject this connection during accept(). Applications wishing to 
>>> use this special property would set a socket option indicating the 
>>> The advantage is that you don't allocate the memory ahead 
>> of time if 
>>> the application isn't interested in accepting this connection.
>> It seems to me the app already sufficiently controls this in 
>> how/when it issues a accept(). On what basis is it making the 
>> rejection at the time of a SYN? (if by IP address, it could 
>> have done this via a different
>> listen() call).
> 
> 4 or (5 tuple) mainly IP adresses and VRF since dest port and src port
> aren't of much help.. you could push the access control below, but if
> you want to factor in other application factors which could vary from
> application to application (like say # of connections etc.,) then it
> seems expedient to do the check at the application level. Consider the
> case where TCP is library linked to the application and it could
> register a callback to be invoked with the 5 tuple and accept/reject the
> connection synchronously which is efficient in terms of resource
> utilization, i.e, you drop the SYN once it is determined that it is
> "useless" rather than waiting the 3 way handshake to complete. At the
> least, it comes across as an optmization, but YMMV.

These sound like more sophisticated interfaces to the OS, in terms of
how deep to listen for calls, what patterns of ports/addrs to listen
for, etc. Although they're not implemented in the OS, they could be. The
fact that you're implementing them at the app layer isn't a big issue,
and can certainly be easier or a performance win, depending on the system.

This doesn't sound like the issue I'm concerned about - i.e., using
information that circumvents the TWHS.

Joe