Subject: TLS at transport level vs stream multiplexing and aggregation (http "routers")

From: "Adrien de Croy" <[email-masked]>
Date: Sun, 17 Nov 2013 20:59:36 +0000

Hi all

there has been talk in the past about http message routers that forward 
messages relating to multiple concurrent streams over the same 
underlying protocol stream.

I'm a big fan of this idea, but I think requiring http2 to be over TLS 
would effectively prohibit this.

If the TLS is being used to establish credentials between client and 
server, and is connection-associated, then it holds the same set of 
badness that everyone holds against NTLM.

This means that TLS is being applied at the wrong level.

I think we should look into using TLS at the stream level, rather than 
transport.  This would allow a single TCP connection to contain multiple 
streams where each stream can be between different final endpoints, with 
different TLS layers.  And include unencrypted streams as well.

Where it is desired to minimise TLS setup overhead where all streams on 
a connection will use the same TLS context, then allow for that in the 
protocol as well.

That would then allow point to point links to use TLS to secure messages 
that may be themselves secured with TLS at the stream level or not.

Adrien

---

From: Mike Belshe <[email-masked]>
Date: Sun, 17 Nov 2013 13:24:44 -0800

I think this is an interesting concept.  We did consider a number of
scenarios like this for SPDY as well - with a small amount of extra
certificate/meta data changes you could do it.

But you should probably try to garner some implementation experience with
it.  There are a lot of details.  I'm also unclear what the goals would be.

Thinking larger - I agree with you that TLS is at the wrong layer.  We've
wedged it into the application space and transport layers today.  You're
advocating moving it up a layer.   Whereas I would actually move it down a
layer :-)  Today our transport is vulnerable to numerous types of attacks.
 I think Dan Bernstein has it right with his layering in curvecp (
http://curvecp.org/) and the minion guys have some proposals that lean this
way as well.

The problem is that if your control structures are exposed in an
unauthenticated and unencrypted manner, then anyone can hijack/DoS any
connection very easily.  But this is a much larger change than HTTP/2.

Mike

On Sun, Nov 17, 2013 at 12:59 PM, Adrien de Croy <[email-masked]> wrote:

> messages relating to multiple concurrent streams over the same underlying
> protocol stream.
>
> server, and is connection-associated, then it holds the same set of badness
> that everyone holds against NTLM.
>
> Where it is desired to minimise TLS setup overhead where all streams on a
> connection will use the same TLS context, then allow for that in the
> protocol as well.
>

---

From: "Adrien de Croy" <[email-masked]>
Date: Sun, 17 Nov 2013 21:44:49 +0000

------ Original Message ------
From: "Mike Belshe" <[email-masked]>
To: "Adrien de Croy" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Sent: 18/11/2013 10:24:44 a.m.
Subject: Re: TLS at transport level vs stream multiplexing and 
aggregation (http "routers")
>But you should probably try to garner some implementation experience 
>with it.  There are a lot of details.  I'm also unclear what the goals 
>would be.

the main goals would be:

a) allow aggregation of streams
b) allow flexibility around whether crypto is used or not
c) allow point to point (e.g. router to router) authentication and 
privacy (e.g. traversing public networks)
d) allow for fault-tolerant system architectures (e.g. by allowing a 
stream to be handed to another path/router).

All these things would require end-to-end crypto and authentication to 
be at stream layer.  The stream is the only end to end thing, therefore 
the crypto would need to be at that layer.

Transport layer crypto would serve a different purpose and could still 
be useful in some cases.  E.g. 2 routers establishing mutual trust for 
relaying messages.

Sure it's a bigger change, but if we went to port 100, the sky's the 
limit.

I wonder if we need to take a step back and re-evaluate what we are even 
trying to do and why.

If we're building a new and better web, based on the things we have 
learned from http/1.0, http/1.1, https and SPDY then if a new port 
number is on the cards, then we shouldn't arbitrarily limit ourselves.

>
>Thinking larger - I agree with you that TLS is at the wrong layer.  
>We've wedged it into the application space and transport layers today.  
>You're advocating moving it up a layer.   Whereas I would actually move 
>it down a layer :-)  Today our transport is vulnerable to numerous 
>types of attacks.  I think Dan Bernstein has it right with his layering 
>in curvecp (http://curvecp.org/) and the minion guys have some 
>proposals that lean this way as well.
there's already IPSEC.  It can't deal with per-stream auth and crypto 
contexts.

>connection very easily.  But this is a much larger change than HTTP/2.
very easily?  Breaking into a TCP stream?  Maybe then TCP should be 
fixed.

Adrien

>>there has been talk in the past about http message routers that 
>>forward messages relating to multiple concurrent streams over the same 
>>transport.  This would allow a single TCP connection to contain 
>>multiple streams where each stream can be between different final 
>>endpoints, with different TLS layers.  And include unencrypted streams 
>>as well.
>>
>>Where it is desired to minimise TLS setup overhead where all streams 
>>on a connection will use the same TLS context, then allow for that in 
>>the protocol as well.
>>
>>That would then allow point to point links to use TLS to secure 
>>messages that may be themselves secured with TLS at the stream level 
>>or not.
>>
>>Adrien
>

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Sun, 17 Nov 2013 22:58:30 +0100

> stream to be handed to another path/router).

Awesome!

and I'd add

e) reuse a security model everyone is knowledgeable and comfortable with,
physical mail (where routing is public and content -not)

d) clearly separate intermediary auth and messages from endpoint auth and
messages without requiring to stuff them in a single tls channel that
requires intermediaries to open to communicate with the client

Regards,

-- 
Nicolas Mailhot

---

From: Roberto Peon <[email-masked]>
Date: Sun, 17 Nov 2013 14:30:03 -0800

Sounds like an interesting experiment (as Mike already said, we considered
this way back when).
-=R

---

From: "Adrien de Croy" <[email-masked]>
Date: Sun, 17 Nov 2013 22:49:12 +0000

as they keep telling us, http is a _message_ based protocol after all.

------ Original Message ------
From: "Roberto Peon" <[email-masked]>
To: "Nicolas Mailhot" <[email-masked]>
Cc: "Adrien de Croy" <[email-masked]>; "Mike Belshe" <[email-masked]>; 
"[email-masked]" <[email-masked]>
Sent: 18/11/2013 11:30:03 a.m.
Subject: Re: TLS at transport level vs stream multiplexing and 
aggregation (http "routers")
>Sounds like an interesting experiment (as Mike already said, we 
>considered this way back when).
>-=R
>
>
>On Sun, Nov 17, 2013 at 1:58 PM, Nicolas Mailhot 
><[email-masked]> wrote:
>>

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 17 Nov 2013 23:56:37 +0100

> Sounds like an interesting experiment (as Mike already said, we considered

yes, and it's already more or less what's done with WebSocket if my
memory serves me right, as the masking is per message and I believe
per channel when mux is used.

I would personally like to see encryption used only on what *needs*
to be encrypted so that "routing" HTTP doesn't require decrypting
for most standard cases. We're not there yet...

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 17 Nov 2013 23:06:48 +0000

>for most standard cases. We're not there yet...

That would be one of the best ways HTTP/2.0 could improve
performance over HTTP/1.1...

Routing encrypted transactions is perfectly feasible, we just
have to define a non-encrypted routable envelope (Host: + non-query
part of URL).

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Roberto Peon <[email-masked]>
Date: Sun, 17 Nov 2013 15:06:49 -0800

There are interesting security implications of interlacing unencrypted and
encrypted data which I'm fairly sure have not at all been
analyzed/experimented with.
This was one of the reasons why we originally thought about, but did not
implement, encryption as an upper, rather than lower layer.

WS does do masking, but that is a fair bit less involved than TLS, which
requires bidirectional communication and is more involved.

Again, I think it is an interesting thing to experiment with, and think
that it will absolutely require lots of analysis and experience...
-=R

---

From: James M Snell <[email-masked]>
Date: Sun, 17 Nov 2013 15:23:56 -0800

It's definitely worth exploring more.  I've done some experimentation with
it as well.  There are a host of issues that jump out immediately but the
overall concept of http2 stream level encryption is theoretically sound.

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 18 Nov 2013 00:27:15 +0100

> have to define a non-encrypted routable envelope (Host: + non-query

I'm certain it is possible, I've worked for a bank where some
webservices had only their payload encrypted/signed, and since
they were using only POST, there was no query string :-)

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 18 Nov 2013 00:30:28 +0100

> implement, encryption as an upper, rather than lower layer.

I remember another point which was about the number of TLS contexts over
a single connection which would make the memory usage explode. This is
still true. With cryptography, it's not possible to have an "Y" shaped
device which saves both CPU and memory at the same time :
  - either you want to save decrypt/encrypt cycles and you route encrypted
    messages with their own contexts, buffers, etc. Here however for routing,
    mostly the endpoint (origin server) has to deal with that many contexts.

  - or you want have only one context and have to encrypt/decrypt.

> that it will absolutely require lots of analysis and experience...

Yes, as usual. I just hope we won't experience that one-year dance we had
with WS...

Willy

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Nov 2013 12:43:32 +1100

Hi Adrien,

I'm personally *very* interested in this as well; I've been thinking about it for a while, and talking to people about it too. There are a number of significant benefits possible with such an approach (which is very similar to SHTTP of old).

Currently, I'm planning to prototype something once I have a Python implementation going (insert running joke here).

Conceptually, it's pretty simple in HTTP/2; you'd just need a handful of new frame types to negotiate per-origin session keys, a flag to indicate that encryption is in use, and the first few bytes of the payload to indicate which session key is in use (since there can be more than one).

The really interesting stuff comes in when you can selectively sign frames instead of encrypting them; that would allow a HTTP response to look like this:

HEADERS [signed]
HEADERS [encrypted]
DATA [encrypted]
DATA [encrypted]
...

so that you can selectively expose headers (e.g., Cache-Control, Host) to intermediaries to allow them to do interesting things.

Talking to security and other folks about this, the main concerns seem to be:
  a) Security evaluation of a Whole New Protocol is hard. There are lots of new potential attacks
  b) Taking what was a generic security function (TLS) and making it effectively application-specific -- with corresponding loss of value to the rest of the ecosystem
  c) Greater information leakage because an attacker can see message boundaries
  d) Exposing finer gradations of encryption / signature (as above) can lead to very subtle security properties; i.e., it's really easy to mess things up here.

As Mike said, implementation experience will be critical before we can take something this big seriously; sucking TLS into HTTP2 is not a trivial task.

Cheers,

On 18/11/2013, at 7:59 AM, Adrien de Croy <[email-masked]> wrote:

> Hi all
>  
> there has been talk in the past about http message routers that forward messages relating to multiple concurrent streams over the same underlying protocol stream.
>  
> I'm a big fan of this idea, but I think requiring http2 to be over TLS would effectively prohibit this.
>  
> If the TLS is being used to establish credentials between client and server, and is connection-associated, then it holds the same set of badness that everyone holds against NTLM.
>  
> I think we should look into using TLS at the stream level, rather than transport.  This would allow a single TCP connection to contain multiple streams where each stream can be between different final endpoints, with different TLS layers.  And include unencrypted streams as well.
>  
> Where it is desired to minimise TLS setup overhead where all streams on a connection will use the same TLS context, then allow for that in the protocol as well.
>  
> That would then allow point to point links to use TLS to secure messages that may be themselves secured with TLS at the stream level or not.
>  
> Adrien

--
Mark Nottingham   http://www.mnot.net/

---

From: James M Snell <[email-masked]>
Date: Sun, 17 Nov 2013 17:48:50 -0800

Just a memory refresher..  The draft needs to be updated but I'm waiting
until stuff is more stable

http://tools.ietf.org/html/draft-snell-httpbis-keynego-00

- james
On Nov 17, 2013 5:45 PM, "Mark Nottingham" <[email-masked]> wrote:

> Hi Adrien,
>
> I'm personally *very* interested in this as well; I've been thinking about
> it for a while, and talking to people about it too. There are a number of
> significant benefits possible with such an approach (which is very similar
> to SHTTP of old).
>
> Currently, I'm planning to prototype something once I have a Python
> implementation going (insert running joke here).
>
> Conceptually, it's pretty simple in HTTP/2; you'd just need a handful of
> new frame types to negotiate per-origin session keys, a flag to indicate
> that encryption is in use, and the first few bytes of the payload to
> indicate which session key is in use (since there can be more than one).
>
> The really interesting stuff comes in when you can selectively sign frames
> instead of encrypting them; that would allow a HTTP response to look like
> this:
>
> HEADERS [signed]
> HEADERS [encrypted]
> DATA [encrypted]
> DATA [encrypted]
> ...
>
> so that you can selectively expose headers (e.g., Cache-Control, Host) to
> intermediaries to allow them to do interesting things.
>
>   b) Taking what was a generic security function (TLS) and making it
> effectively application-specific -- with corresponding loss of value to the
> rest of the ecosystem
>   d) Exposing finer gradations of encryption / signature (as above) can
> lead to very subtle security properties; i.e., it's really easy to mess
> things up here.
>
> As Mike said, implementation experience will be critical before we can
> take something this big seriously; sucking TLS into HTTP2 is not a trivial
> task.
>
> Cheers,
>
>

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Nov 2013 12:51:15 +1100

I somehow hadn't seen that... great minds :)

I'd encourage folks to look at this draft and have a play if they're inclined.

Cheers,

On 18/11/2013, at 12:48 PM, James M Snell <[email-masked]> wrote:

> Just a memory refresher..  The draft needs to be updated but I'm waiting until stuff is more stable
>

---

From: James M Snell <[email-masked]>
Date: Sun, 17 Nov 2013 17:55:58 -0800

It's very old and needs to be updated.  It was based on some very early
brainstorming so feel free to rip it apart. The whole idea was to just get
something started.

- james
On Nov 17, 2013 5:51 PM, "Mark Nottingham" <[email-masked]> wrote:

> > I'm personally *very* interested in this as well; I've been thinking
> about it for a while, and talking to people about it too. There are a
> number of significant benefits possible with such an approach (which is
> very similar to SHTTP of old).
> >
> > The really interesting stuff comes in when you can selectively sign
> frames instead of encrypting them; that would allow a HTTP response to look
> like this:
> >
> > HEADERS [signed]
> > HEADERS [encrypted]
> > DATA [encrypted]
> > DATA [encrypted]
> > ...
> >
> > so that you can selectively expose headers (e.g., Cache-Control, Host)
> to intermediaries to allow them to do interesting things.
> >
> on a connection will use the same TLS context, then allow for that in the
> protocol as well.
> > >
> messages that may be themselves secured with TLS at the stream level or not.
> > >
> > > Adrien
> >
> > --

---

From: Roberto Peon <[email-masked]>
Date: Sun, 17 Nov 2013 17:58:35 -0800

As a reminder of some of the difficulties, without an integrity MAC or
equivalent, you can ensure nothing :)
Though James' draft includes an integrity frame, it doesn't require its
usage, and thus suffers from downgrade attacks.
Requiring its usage would be helpful... :)
-=R

---

From: James M Snell <[email-masked]>
Date: Sun, 17 Nov 2013 17:59:49 -0800

+1
On Nov 17, 2013 5:58 PM, "Roberto Peon" <[email-masked]> wrote:

>> > Conceptually, it's pretty simple in HTTP/2; you'd just need a handful
>> of new frame types to negotiate per-origin session keys, a flag to indicate
>> > > I think we should look into using TLS at the stream level, rather
>> than transport.  This would allow a single TCP connection to contain
>> endpoints, with different TLS layers.  And include unencrypted streams as
>> well.
>> > >