Subject: [tcpinc] TCP-TLS latency & SYN floods: kernel vs user-space

From: Bob Briscoe <[email-masked]>
Date: Mon, 17 Nov 2014 08:27:05 +0000

Ekr,

There needs to be more behind the text on "Implementation Options"
<https://tools.ietf.org/html/draft-rescorla-tcpinc-tls-option-00#section-5>

You say:
"  There are two primary implementation options for TCP-TLS:
    o  Implement all of TCP-TLS in the operating system kernel.
    o  Implement just the TCP-TLS negotiation option in the operating
       system kernel with an interface to tell the application that TCP-
       TLS has been negotiated and therefore that the application must
       negotiate TLS.
"

* If TLS is implemented in user-space, TCP will hold back TLS 
commands from the app until the 3-way handshake has completed, then 
pass it TLS commands via the datastream.
* If TLS is implemented in the kernel, presumably TCP will process 
and remove TLS commands out of the datastream even during the 
handshake, before they are passed to the app.

This has implications both for vulnerability to SYN floods and for latency.

==SYN Floods==
When you have more TCP options than just MSS, Window Scale and 
SACK-OK, the space for a SYN cookie is not enough. So Alex Zimmerman, 
Fernando Gont and I are producing an echo option draft. The idea is 
that it will be mandatory to implement with all new TCP options, so 
pls consider using it for TCP-TLS. The filename will be 
draft-zimmermann-tcpm-echo-option, but an initial placeholder version 
is at draft-briscoe-tcpm-echo-cookie.

It's hard to know whether to recommend that TLS only accepts a 
ClientHello command within the initial SYN. That would minimise the 
server's workload under attack, but it would preclude advances like 
SnapStart, which do all the following in the first shot:
- ClientHello
- Predicted ServerHello
- ChangeCipherSpec
- Finished

==Latency==
On Friday in the tcpinc mtg we agreed that TCP-TLS latency would 
benefit from Inner Space as much as tcpcrypt would. I've realised 
it's not quite that simple.

Two cases:
1) On first time access of a server, Inner Space only cuts out a 
round for TCP-TLS(v2) if TLS is implemented in the kernel. Otherwise, 
the app would have to wait until the 3WHS had completed before 
getting the TLS commands;.
2) On resuming a connection, if TFO can be used, the ClientHello can 
be sent on the SYN, so Inner Space can't improve on that. If TFO 
cannot be used {Note 1}, then Inner Space cuts out a round if TLS is 
implemented in the kernel, but not if it is implemented in 
user-space, for the same reason as with first-time access.

{Note 1}: TFO can only be used where the app is robust to duplicate 
SYNs, i.e. either the app detects duplication, or it doesn't matter. 
This decision has to depend on the actual app - TLS cannot be thought 
of as an app (AFAICT).

BTW, you also need to say something in Sec.5 about what happens if 
the application doesn't know what TLS is.

Cheers

Bob

________________________________________________________________
Bob Briscoe,                                                  BT

---

From: Joe Touch <[email-masked]>
Date: Mon, 17 Nov 2014 10:36:09 -0800

> <https://tools.ietf.org/html/draft-rescorla-tcpinc-tls-option-00#section-5>
...
> This has implications both for vulnerability to SYN floods and for latency.

It's not clear whether SYN floods are in-scope for these solutions.

I.e., they're aimed at privacy, but it's not yet clear whether TCP
robustness is in-scope. It'd be useful to converge on that issue before
dissecting the capabilities of the various approaches.

Joe

---

From: Watson Ladd <[email-masked]>
Date: Mon, 17 Nov 2014 10:46:27 -0800

On Nov 17, 2014 10:42 AM, "Joe Touch" <[email-masked]> wrote:
> > <
https://tools.ietf.org/html/draft-rescorla-tcpinc-tls-option-00#section-5>
> It's not clear whether SYN floods are in-scope for these solutions.

Well, if you expect kernels to implement and users to deploy these
solutions, they shouldn't make security worse. Increased DoS vulnerability
will go a long way to making sure these solutions never see the light of
day.

Sincerely,
Watson Ladd
_______________________________________________
> Tcpinc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpinc

---

From: Nico Williams <[email-masked]>
Date: Mon, 17 Nov 2014 12:46:49 -0600

On Mon, Nov 17, 2014 at 2:27 AM, Bob Briscoe <[email-masked]> wrote:
> * If TLS is implemented in user-space, TCP will hold back TLS commands from
> the app until the 3-way handshake has completed, then pass it TLS commands
> via the datastream.

I don't see why that's implied in the quoted text.  Consider the
traditional scheme used by RPCSEC_GSS (for secure NFS)
implementations: they upcall to process handshake messages (what GSS
calls "security context tokens").

Sure, it would require additional APIs to do this in user-land in a
portable way (and, in particular, in the same process that will be
sending and receiving data on the connected socket).

> * If TLS is implemented in the kernel, presumably TCP will process and
> remove TLS commands out of the datastream even during the handshake, before
> they are passed to the app.

Well, yeah, that would be the point, though at least some TLS state
information has to be exposed to the app if it wants it.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Mon, 17 Nov 2014 12:48:41 -0600

>> It's not clear whether SYN floods are in-scope for these solutions.

I wouldn't expect the charter to explicitly rule them in scope.  Some
things go without saying.

> will go a long way to making sure these solutions never see the light of

+1

Nico
--

---

From: Joe Touch <[email-masked]>
Date: Mon, 17 Nov 2014 10:50:50 -0800

On 11/17/2014 10:46 AM, Watson Ladd wrote:
> 
> On Nov 17, 2014 10:42 AM, "Joe Touch" <[email-masked]
> <mailto:[email-masked]>> wrote:
>>
>>
>>
> solutions, they shouldn't make security worse. Increased DoS
> vulnerability will go a long way to making sure these solutions never
> see the light of day.

I do agree - see my recent post on the header fields I think ought to be
protected.

I'm only suggesting that we have THAT discussion first - i.e., what
protection we expect a TCPINC solution to provide beyond data privacy.

Joe

---

From: Nico Williams <[email-masked]>
Date: Mon, 17 Nov 2014 12:52:10 -0600

On Mon, Nov 17, 2014 at 2:27 AM, Bob Briscoe <[email-masked]> wrote:
> It's hard to know whether to recommend that TLS only accepts a ClientHello
> command within the initial SYN. That would minimise the server's workload
> under attack, but it would preclude advances like SnapStart, which do all
> the following in the first shot:
> - ClientHello
> - Predicted ServerHello
> - ChangeCipherSpec
> - Finished

Well, that seems like a good idea whenever Fast Open is a good idea,
and a bad idea when you can't open fast.

The server should have some idea that the client is live and not an
attacker before spending cycles on processing the client's handshake
messages.  TCP normally does this, but hitching the TLS handshake to
the TCP handshake means losing that protection.  TCP Fast Open type
solutions are clearly indicated.

> ==Latency==
> On Friday in the tcpinc mtg we agreed that TCP-TLS latency would benefit
> from Inner Space as much as tcpcrypt would. I've realised it's not quite
> that simple.
>
> Two cases:
> 1) On first time access of a server, Inner Space only cuts out a round for
> TCP-TLS(v2) if TLS is implemented in the kernel. Otherwise, the app would
> have to wait until the 3WHS had completed before getting the TLS commands;.
> 2) On resuming a connection, if TFO can be used, the ClientHello can be sent
> on the SYN, so Inner Space can't improve on that. If TFO cannot be used
> {Note 1}, then Inner Space cuts out a round if TLS is implemented in the
> kernel, but not if it is implemented in user-space, for the same reason as
> with first-time access.
>
> {Note 1}: TFO can only be used where the app is robust to duplicate SYNs,
> i.e. either the app detects duplication, or it doesn't matter. This decision
> has to depend on the actual app - TLS cannot be thought of as an app
> (AFAICT).
>
>
> BTW, you also need to say something in Sec.5 about what happens if the
> application doesn't know what TLS is.
>
> Cheers
>
>
>
> Bob
>
>
>

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 17 Nov 2014 10:58:07 -0800

On Mon, Nov 17, 2014 at 12:27 AM, Bob Briscoe <[email-masked]> wrote:

> <https://tools.ietf.org/html/draft-rescorla-tcpinc-tls-option-00#section-5
> >
>
> You say:
> * If TLS is implemented in user-space, TCP will hold back TLS commands
> from the app until the 3-way handshake has completed, then pass it TLS
> commands via the datastream.
>

I'm not sure why this has to be the case. There's assumed cooperation
between the kernel and the user-space process, so this could also
include the ability to read and write early.

> they are passed to the app.

Yes, I would imagine so.

> When you have more TCP options than just MSS, Window Scale and SACK-OK,
> the space for a SYN cookie is not enough. So Alex Zimmerman, Fernando Gont
> and I are producing an echo option draft. The idea is that it will be
> mandatory to implement with all new TCP options, so pls consider using it
> for TCP-TLS. The filename will be draft-zimmermann-tcpm-echo-option, but
> an initial placeholder version is at draft-briscoe-tcpm-echo-cookie.
>

Thanks for the link.

> the following in the first shot:

I don't think anyone really does Snap Start but with TLS 1.3, which
will include a similar feature, you are going to want to send more stuff
in the first flight. Note that the server always has the option of
forcing the client to re-send this data if it chooses by simply
refusing 0-RTT.

==Latency==
> have to wait until the 3WHS had completed before getting the TLS commands;.

See above.

> 2) On resuming a connection, if TFO can be used, the ClientHello can be
> sent on the SYN, so Inner Space can't improve on that.

I don't believe that this is correct: the issue is that the client doesn't
know for
sure that the server continues to be willing to do TCP-TLS, so the data
needs to not be in-band from the application's perspective.

> application doesn't know what TLS is.

I don't understand this point. Can you please elaborate.

-Ekr

---

From: Joe Touch <[email-masked]>
Date: Mon, 17 Nov 2014 11:45:04 -0800

On 11/17/2014 10:58 AM, Eric Rescorla wrote:
> 
> 
> On Mon, Nov 17, 2014 at 12:27 AM, Bob Briscoe <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     Ekr,
> 
>     <https://tools.ietf.org/html/__draft-rescorla-tcpinc-tls-__option-00#section-5
>     <https://tools.ietf.org/html/draft-rescorla-tcpinc-tls-option-00#section-5>>
> 
>     You say:

It's always OK to write "early" to a TCP connection; the data just gets
queued up until the protocol handles it.

However, there's should never be "early read". The TCP API is defined as
holding back data from the user until the TWHS completes (or its
equivalent, e.g., for TFO).

"Cooperation" can occur within various components that implement *TCP*
(whether in userspace or kernelspace), but the user-level API should not
need to change for TCPINC.

Joe

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 18 Nov 2014 02:12:39 +0000

Ekr,

>I don't understand this point. Can you please elaborate.

Just a nit. It concerns the second bullet point at 
<https://tools.ietf.org/html/draft-rescorla-tcpinc-tls-option-00#section-5>:

I'm simply pointing out that tcpinc is for any port and therefore any 
app. So when the kernel tells the app that it must negotiate TLS, it 
cannot rely on getting a positive response.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 18 Nov 2014 02:33:46 +0000

Joe,

At 19:45 17/11/2014, Joe Touch wrote:

> <https://tools.ietf.org/html/__draft-rescorla-tcpinc-tls-__option-00#section-5> 
>
> >

I think the point Ekr (and Nico) are making is that TLS is really a 
control channel within the TCP Data, so if an app understands that 
distinction, it can preempt the handshake. Therefore it /can/ read 
and process such control commands.

The app has to take responsibility for the consequences if the 
handshake fails, but I think the point is that, in many scenarios 
involving TLS, there will be no consequences - other than wasted 
work. No 'real' data will have been consumed.

Bob

---

From: Nico Williams <[email-masked]>
Date: Tue, 18 Nov 2014 11:59:51 -0600

> and process such control commands.

Something like that.  My point is that the interfaces used to process
handshake messages in "user-land" would have to be designed for the task
(I've no opinion as to whether they should be designed as an extension
of BSD sockets).

> work. No 'real' data will have been consumed.

Even with Fast Open and resumption (but I repeat myself) there's still
the possibility of clients attempting to DoS a server *after*
establishing Fast Open / resumption state, and so services will need to
be able to measure resumption load and use it to shorten resumption
state lifetime on any of global, per-client, and/or per-session bases.

If I were designing an implementation of this, I'd probably go with a
gssd-like daemon to do the TLS handshake processing in user-land, with a
task queue on the kernel side, and enough metadata in the requests that
the daemon could implement throttling as needed.  This would provide
some separation of privilege (access to private keys or the HSMs
containing them from the processes providing services), which would be
much more than merely convenient (obviously).

('gssd' is a daemon in various secure NFS implementations -some of them
open-source- that implements the GSS equivalent of hanshaking in
user-land, with the kernel doing "upcalls" to handle handshaking, but
with all "app data records" handled in kernel-land.  The IPC mechanisms
for such upcalls are well understood.)

Nico
--

---

From: Joe Touch <[email-masked]>
Date: Tue, 18 Nov 2014 10:16:50 -0800

> (I've no opinion as to whether they should be designed as an extension

AFAICT that would require the specification of a new API over TCP;
otherwise, you have to translate TLS's expectation of a reliable,
byte-ordered stream into what TCP provides within TCP (which is neither).

Joe

---

From: Nico Williams <[email-masked]>
Date: Tue, 18 Nov 2014 12:46:00 -0600

> byte-ordered stream into what TCP provides within TCP (which is neither).

Not in the design I gave, which could be implemented using entirely
private interfaces, with TLS session state exported via socket options.

There is an existence proof of this: the gssd implementations generally
export a pure GSS-API interface in kernel-land (the Illumos one prefixes
a 'k' to the function names, but otherwise the APIs are very close to
RFC2744) but the IPC interfaces for upcalling are non-standard, and the
gssd daemon's implementation is private.

There is a second existence proof of this: IPsec.  (Here PF_KEY is a
standard-ish IPC protocol for KE daemons, which are very similar to gssd
anyways.)  Applications aren't expected to (indeed: can't) implement the
KE protocol in user-land without being privileged enough to act as the
KE daemon (iked) for the system.

TCPINC should be amenable to similar designs as RPCSEC_GSS and IPsec/IKE.

It might be OK to also allow TCPINC proposals to be implemented as you
imagined, but I'd rather not make that the only available implementation
approach.

Nico
--

---

From: Joe Touch <[email-masked]>
Date: Tue, 18 Nov 2014 15:10:58 -0800

> private interfaces, with TLS session state exported via socket options.

I'm speaking of the private interface. Yes, it could be
implementation-specific, but if you are intending to define a complete
spec for using SSL over raw TCP segments you first have to explain what
you expect from raw TCP segments.

Joe

---

From: Nico Williams <[email-masked]>
Date: Tue, 18 Nov 2014 17:51:01 -0600

> you expect from raw TCP segments.

This sub-thread is about your "However, there's should never be "early
read"." comment.  I agreed that "early read", whether through public
interfaces or private ones, creates DoS considerations.  Therefore we're
not really disagreeing, right?

---

From: Joe Touch <[email-masked]>
Date: Tue, 18 Nov 2014 16:02:32 -0800

> not really disagreeing, right?

No.

But I'm making another point - "early read" is a property of the TCP
application interface.

AFAICT, your SSL over TCP proposal can't use the TCP application
interface. You need to define SSL over *raw* TCP segments, not SSL over
ordered, reliable bytestream.

I.e., it's not about early read or not. If you think that's related to
the problem, you're looking at the wrong interface (well-defined, as per
RFC793, or "private"/TBD).

Joe

---

From: Nico Williams <[email-masked]>
Date: Tue, 18 Nov 2014 18:36:32 -0600

> But I'm making another point - "early read" is a property of the TCP

And I'm saying that for this particular protocol (TLS in TCP) the
early data is best handled by the system, with a system interface, not
an application interface.

---

From: Joe Touch <[email-masked]>
Date: Tue, 18 Nov 2014 17:31:54 -0800

> an application interface.

I think we're agreeing, but my concern is how much the early data
handling by the system needs to be spec'd out, not just left to the
implementation.

Joe

---

From: Nico Williams <[email-masked]>
Date: Tue, 18 Nov 2014 20:11:24 -0600

Joe,

No one is disagreeing with this.