Subject: [hybi] An alternative design for the web socket handshake

From: Adam Barth <[email-masked]>
Date: Thu, 13 May 2010 12:26:13 -0700

I know there are a lot of pressures on the design of the web socket
protocol, but I thought I would take a few minutes and share my
thoughts on the handshake.  I think bits and pieces of this design
have been proposed by other folks (who I'm failing to credit
properly).

Instead of basing the web socket handshake on HTTP, we should base the
handshake on TLS.  Using TLS gives us better security properties and
more robustness in the face of benign network intermediaries.  Using
TLS has the following important security property:

1) (Security) It is safe to spam TLS connections to arbitrary ports on
arbitrary servers.  (Browsers to this already, so we're either already
screwed or it's ok.)

In particular, we should use the
<http://tools.ietf.org/html/draft-agl-tls-nextprotoneg> extension to
TLS.  Using this extension serves two proposes:

2) (Functionality) Folks can host both their HTTPS server and their
web socket server on the same port (e.g., 443).
3) (Security) We know the receiving server undertands web sockets
because they announced support as part of the TLS handshake.  That
means we don't have to worry about a non-web socket server getting
confused and responding to the web socket handshake.

Finally, using TLS works better with proxies because user agents can
use the CONNECT method of the proxy in the same way we use it today
for HTTPS connections.  This will interoperate with the vast majority
of proxies because the vast majority of proxies accept HTTPS
connections today.

4) (Robustness) Using TLS makes web sockets look like HTTPS to benign
network intermediaries.

Notice that we can still over two kinds of web socket connections (ws
and wss): one that checks the server's certificate and one that
ignores it.  By ignoring the certificate over ws, we make it easy for
hobbyists and small sites to run web socket servers without the need
to purchase a CA-validated certificate.

Adam

---

From: Mike Belshe <[email-masked]>
Date: Thu, 13 May 2010 12:35:06 -0700

On Thu, May 13, 2010 at 12:26 PM, Adam Barth <[email-masked]> wrote:

> +1

> to purchase a CA-validated certificate.

As an added benefit, TLS can also negotiate stream-level compression.  I
believe all browsers today send the null-set of compressors (preventing TLS
based compression from being negotiated), but the protocol does support it
without an extra round trip.

Mike

>
> Adam
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: John Tamplin <[email-masked]>
Date: Thu, 13 May 2010 15:40:15 -0400

> without an extra round trip.

However, I doubt you want to compress framing bytes, control frames, etc as
well.  It seems more likely that putting compression inside WebSocket will
result in better results.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 13 May 2010 12:42:21 -0700

On May 13, 2010, at 12:26 PM, Adam Barth wrote:

> more robustness in the face of benign network intermediaries.  

After thinking about it, this seems like a really good design. It would finesse the HTTP compatibility/compliance issue because it operates below the HTTP layer. And it would have good security properties.

The main downside is that it would be impossible to do non-TLS WebSocket; while the ability to use an unverified cert would leave the protocol open to hobbyist use, connection setup cost would be significantly more expensive in terms of client load, server load and latency. However, since WebSocket is likely to make relatively few long-lived connections, this seems like an acceptable consequence.

It would also require the hypothetical hobbyist implementing from scratch to use a TLS library rather than just a sockets library. That seems like an acceptable consequence to me too.

Regards,
Maciej

> Using

---

From: Ian Hickson <[email-masked]>
Date: Thu, 13 May 2010 21:15:26 +0000 (UTC)

On Thu, 13 May 2010, Adam Barth wrote:
> handshake on TLS.

This would require the hypothetical hobbyist implementing from scratch to 
use a TLS library rather than just using the sockets API directly. I think 
this is an unacceptable increase in complexity.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 13 May 2010 23:26:08 +0200

Maciej Stachowiak wrote:
> The main downside is that it would be impossible to do non-TLS WebSocket; 

It would not be impossible.

If websocket was allocated a dedicated port for non-TLS connections,
then that could be used at least for development and testing.

TLS would only need to be used for actual deployment and the nextprotoneg
extension does allow easy sharing of port 443.  However I'm not sure how
long it will take before nextprotoneg is well supported by TLS libraries?

Note that the possibility of using non-HTTP handshake suggests that even
if we start with HTTP upgrade, it would be worthwhile to not depend too
much on HTTP headers in the handshake.  Having something like the
control frames in Scott Fergusons proposal will work regardless of the
way the connection is made.

cheers

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 13 May 2010 14:40:07 -0700

> this is an unacceptable increase in complexity.

Is using a TLS library more complicated than using a socket library? Can you provide some code samples that show this to be the case?

Regards,
Maciej

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 13 May 2010 14:42:06 -0700

> long it will take before nextprotoneg is well supported by TLS libraries?

Under Adam's proposal, WebSocket would *always* use TLS, and the handshake would be a TLS handshake. In theory we could invent a second handshake for development and testing purposes only, but that seems like a poor use of resources.

Regards,
Maciej

---

From: Ian Hickson <[email-masked]>
Date: Thu, 13 May 2010 21:46:40 +0000 (UTC)

On Thu, 13 May 2010, Maciej Stachowiak wrote:
> >> Instead of basing the web socket handshake on HTTP, we should base 
> >> the handshake on TLS.
> > 
> > This would require the hypothetical hobbyist implementing from scratch 
> > to use a TLS library rather than just using the sockets API directly. 
> > I think this is an unacceptable increase in complexity.
> 
> Is using a TLS library more complicated than using a socket library? Can 
> you provide some code samples that show this to be the case?

You don't need to use a socket library to implement Web Sockets. Just use 
the OS API directly.

Greg's suggestion is an interesting one, though. Instead of constructing 
Web Sockets so that sharing a port with an existing HTTP server works at 
the HTTP level, why not make Web Sockets itself just its own protocol with 
no relationship to HTTP, and have the port-sharing work only for encrypted 
connections, by doing the dispatch at the TLS level?

The amateurs I'm concerned about aren't going to be doing port sharing, so 
they don't need the sharing magic. The experts who _do_ need the sharing 
magic are experts, so they can do it at the more complicated TLS level 
without any trouble.

Would that be satisfactory to everyone?

(This idea would be quite a big change from what we have now, and would 
make a few of the requirements bogus. This is why I'm concerned about our 
requirements document assuming specific solutions.)

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 13 May 2010 14:55:00 -0700

> You don't need to use a socket library to implement Web Sockets. Just use 

The OS API is, in general, provided by a library. Unless you're writing in assembly and making system call traps directly.

If you consider libc not to be a library, then keep in mind that higher-level languages tend to provide a higher-level wrapper over what is exposed to C. And often your best bet is a higher-level framework than the lowest-level wrapper. 

For example, if you are coding a network server in Python, you are much better off using Twisted than the socket module that is part of the Python standard library. Not only is your program more likely to be correct, but your programming is likely to be easier. And it's apparently fairly straightforward to use TLS with Twisted. See here for sample TLS server code: <http://twistedmatrix.com/documents/current/core/howto/ssl.html>. It's 15 lines for an echo server.

Regards,
Maciej

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 13 May 2010 15:17:53 -0700

> Under Adam's proposal, WebSocket would *always* use TLS, and the handshake would be a TLS handshake. In theory we could invent a second handshake for development and testing purposes only, but that…
Since WebSocket still needs to negotiate the /host/, /resource/, and 
/origin/ values, and I don't think we can piggyback those values on the 
TLS handshake, we'd still need some kind of WebSocket-specific initial 
handshake.

I like the general idea. As Greg pointed out, we won't be able to ship 
the 443 TLS sharing for a few years until the TLS libraries catch up, 
but we can always use separate IPs in the short term.

-- Scott

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 13 May 2010 15:27:57 -0700

On May 13, 2010, at 3:17 PM, Scott Ferguson wrote:

> Since WebSocket still needs to negotiate the /host/, /resource/, and /origin/ values, and I don't think we can piggyback those values on the TLS handshake, we'd still need some kind of WebSocket-specific initial handshake.

Good point. However, the second-level handshake you'd design to be used on top of a TLS handshake might not be sufficiently robust or secure to use by itself, without TLS. So we'd have to design a defense against cross-protocol attacks for the non-TLS case that is totally unnecessary in combination with TLS.

Regards,
Maciej

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 14 May 2010 07:01:34 +0800

> Would that be satisfactory to everyone?

WFM
 
> make a few of the requirements bogus. This is why I'm concerned about
> our requirements document assuming specific solutions.)

Hopefully, it's not about being bogus, just really easy to satisfy.  The requirements that become bogus probably were bogus in the first place ;)

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 14 May 2010 07:07:13 +0800

> TLS handshake, we'd still need some kind of WebSocket-specific initial

Maybe that's not a problem.

The host is known through the TLS handshake (that's the server identity from the cert, and maybe SNI).  

Constructing an origin for this might be interesting, since origin includes port.  However, the only entity that actually cares about this (in this case) is the client and they know what port they opened.  The server doesn't necessarily have to know.

That should be enough for a client to enforce a same-origin policy.

It's possible that the resource isn't necessary.
 
> I like the general idea. 

So do I.

---

From: Ian Hickson <[email-masked]>
Date: Thu, 13 May 2010 23:30:58 +0000 (UTC)

On Thu, 13 May 2010, Maciej Stachowiak wrote:
> 
> Good point. However, the second-level handshake you'd design to be used 
> on top of a TLS handshake might not be sufficiently robust or secure to 
> use by itself, without TLS. So we'd have to design a defense against 
> cross-protocol attacks for the non-TLS case that is totally unnecessary 
> in combination with TLS.

We already have such a handshake, so that design work is basically done.

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 13 May 2010 16:47:37 -0700

> We already have such a handshake, so that design work is basically done.

But there's no reason to impose that level of complexity when operating over TLS, or over non-TLS that's not intended to share with HTTP.

Regards,
Maciej

---

From: John Tamplin <[email-masked]>
Date: Thu, 13 May 2010 19:51:18 -0400

One question -- how would switching to TLS affect proxies that alter the
traffic?  For example, for my compression experiments I have a WebSockets
proxy which receives uncompressed frames locally, then compresses them and
sends them to a remote server via compressed WebSockets frames and then
decompresses responses before sending them to the browser.  If TLS is in
play, I am not sure how this could work.

Maybe the answer is we don't care about such uses, but I wanted to bring it
up.

---

From: Adam Barth <[email-masked]>
Date: Thu, 13 May 2010 16:53:31 -0700

You can just use a TLS proxy:

http://crypto.stanford.edu/ssl-mitm/

Adam

---

From: Roberto Peon <[email-masked]>
Date: Thu, 13 May 2010 16:55:09 -0700

The proxy which does any transformation would have to answer the cert for
the site in question. Alternately, the client would need a root cert
installed which would allow a corporate proxy to act on behalf of all users,
in which case it would receive a TLS connection from the client, and then
make another TLS connection to the real server.

It shouldn't affect how you choose to modify you content otherwise.
-=R

---

From: Brad Spencer <[email-masked]>
Date: Thu, 13 May 2010 21:42:36 -0300

On Thu, May 13, 2010 at 02:42:06PM -0700, Maciej Stachowiak wrote:
> Under Adam's proposal, WebSocket would *always* use TLS, and the
  handshake would be a TLS handshake. In theory we could invent a
  second handshake for development and testing purposes only, but that
  seems like a poor use of resources.

If the suggestion is that WebSocket use a handshake that is very
similar to that in TLS, but that it in no way requires any encryption,
authentication, keys, certificates, or other such features of TLS, I
personally think could work, but it sounds like sharing a port would
be hard without an awesome TLS library.

If the suggestion is that WebSocket _depends_ on TLS in some way,
including encryption, etc., and then use TLS-like framing, that seems
to presume that TLS is generally preferable over plain TCP, and I
don't think that is the case (and might still be hard to implement
without an awesome TLS library).  FWIW, there are many applications of
WebSocket, including some I plan to build, that work just fine (and in
many ways better) without authentication and encryption.

I think saying the WebSocket should always be over TLS (not just share
a similar-looking handshake) is not dissimilar from saying there
should only be HTTPS and no HTTP.

Just my thoughts as a (small) implementor.

-- 
----------------------------------------------------------
Brad Spencer - [email-masked] - www.starscale.com

---

From: Adam Barth <[email-masked]>
Date: Thu, 13 May 2010 18:17:38 -0700

> be hard without an awesome TLS library.

Fortunately, TLS already offers this mode via the null cipher suite.
There's no need to reinvent the wheel.

> without an awesome TLS library).

There are two advantages TLS gives us over plain TCP:

1) TLS has an excellent extension mechanism, in particular the
nextprotoneg extension lets us avoid cross-protocol vulnerabilities.
TCP itself has no such mechanism.

2) Using TLS lets us interoperate with HTTPS servers on the same port,
again by leveraging TLS's extension mechanism.  Forking the connection
at the TLS layer lets us avoid the complexities of trying to fork the
connection at the HTTP layer.

> FWIW, there are many applications of

Awesome.

> should only be HTTPS and no HTTP.

Not really.  We're interested in the connection negotiation aspects of
TLS primarily and the encryption/authentication aspects secondarily.

For what it's worth, it's entirely likely that the vast majority of
HTTP traffic will be over TLS in the not-too-distant future if SPDY
takes off the way its proponents hope that it will.

Adam

---

From: Rob Sayre <[email-masked]>
Date: Thu, 13 May 2010 21:22:48 -0400

> takes off the way its proponents hope that it will.

But that's just speculation, isn't it?

- Rob

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 14 May 2010 09:50:43 +0800

> takes off the way its proponents hope that it will.

Let's be realistic.

Using TLS always is nice in theory but it does have these shortcomings:

 - The TLS_NULL_WITH_NULL_NULL ciphersuite isn't a valid product of TLS negotiation.

 - No-one implements TLS unless they want crypto.  So, if you have TLS, you have crypto, as far as I am aware.

 - The complexity of implementing the handshake and record layer is > 0.  (Though the record payload might be a candidate for WS frame.)

For the case where you are looking for a lightweight implementation, these are a problem.

Even with nextprotoneg, it's desirable to have a secured capabilities (re)negotiation.  The nextprotoneg draft doesn't talk about this, but it should: an attacker that is able to rewrite a handshake packet could deny or downgrade service without detection.  [1]

--Martin

[1] Caveat: I've not been following the TLS WG on this subject, the topic might have been covered.

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 14 May 2010 04:49:09 +0100

Maciej Stachowiak wrote:
> However, since WebSocket is likely to make relatively few long-lived
> connections, this seems like an acceptable consequence.

Are you sure?  I don't believe this.  Neither that there are few
connections, nor that they are long-lived.

Number of connections:

Having 100 tabs open (over multiple windows) is no big deal in a
modern browser.  It's normal for some users.  Let's say half of them
have a WebSocket in 5 years time, because advertisements started using
WebSockets to stream updates.  That's 50 open connections

Short-lived:

Ian Hickson has often suggested that its appropriate for pages to open
several WebSockets - most likely one per independent "module" of code
on the page which needs one, primarily when they are implemented by
different authors and use different code bases.  For example one for
an embedded interactive calendar, one for a scrolling ticker, one for
that "special" advertisement...  (That has been used as the main
argument against client-side automatic multiplexing - that outgoing
connections are sufficiently low cost not to need it.)

Moreover, each time a link is followed on a page containing
WebSocket-using code (such as the example above), all the WebSockets
will be closed, and new ones on the next page opened.

In other words, every few seconds, there may be several connections
closed, and several new connections if moving round a site with
complex pages.  If they are simple pages, or smart in their connection
sharing, that's still one connection per link followed, except on
sites which are especially Ajaxy and don't load whole pages.

> It would also require the hypothetical hobbyist implementing from
> scratch to use a TLS library rather than just a sockets
> library. That seems like an acceptable consequence to me too.

It rules out running it on small devices, which can't do TLS at all
but can easily support the current draft and even proposed
multiplexing extensions and other "complex" things.

I'm not sure what the licensing situation is for good, up to date TLS
implementations these days.  I don't think there are any good ones in
the public domain.

TLS also uses quite a lot of memory per connection.  See above for
numbers of connections - most of them idle, but still using TLS memory.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 14 May 2010 04:55:45 +0100

Roberto Peon wrote:
> 
>    The proxy which does any transformation would have to answer the cert
>    for the site in question. Alternately, the client would need a root
>    cert installed which would allow a corporate proxy to act on behalf of
>    all users, in which case it would receive a TLS connection from the
>    client, and then make another TLS connection to the real server.

That seems to rule out transparent, opportunistic proxy
transformations such as multiplexing, compression and keepalive sharing.

I regard multiplexing and keepalive sharing as especially valuable due
to connection overheads and the large number of mostly-idle WebSocket
connections anticipated, even if it's only proxy-to-proxy (the
endpoints don't need to know about it.)

It also seems to rule out firewalls recognising and filtering the
traffic over WebSocket.  (I take no position on whether that's a good
or bad thing.)

-- JAmie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 14 May 2010 05:08:22 +0100

> Would that be satisfactory to everyone?

No to me.

Port 80 == available near-universally; unencrypted == low overhead.

Port 443 == not available universally; encrypted == high overhead;
certificated == impossible to coalesce by intermediaries.

Port other == not available universally.

So port 80 is a legitimate thing to want to use sometimes, and it must
have a HTTP compatible handshake due to intermediaries which intercept
and mess about with port 80 connections.

Otherwise we're back to not using WebSocket and using hanging-GET instead.

(Now, using TLS over port 80 _after_ the HTTP-compatible handshake, for
when you want an encrypted/authenticated connection with near
universal availability, there's a thought...)

-- Jamie

---

From: Roberto Peon <[email-masked]>
Date: Thu, 13 May 2010 21:08:43 -0700

> transformations such as multiplexing, compression and keepalive sharing.

Its a feature! You know all those deployment problems we're having right
now? Guess what you can point your finger at (hint: it is not the server,
nor the client).!
Trying to work around the brokenness of transparent proxies is *tough*. If
you want your protocol to have any chance of being upgraded in the near
future, you should either anticipate lots of problems with transparent
proxies, or you should make it so that they cannot mess up stuff for you in
the future.

> endpoints don't need to know about it.)

Then we should ensure that we can multiplex WebSockets within the browser to
keep OS level overhead to reasonable levels!

> traffic over WebSocket.  (I take no position on whether that's a good

Again, if you want to do that, you install root certs on the clients, and
then the firewall can do whatever it wants to it, since it can pretend to be
the endpoint.

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 14 May 2010 05:10:39 +0100

John Tamplin wrote:
> 
>    On Thu, May 13, 2010 at 3:35 PM, Mike Belshe <[1][email-masked]>
>    wrote:
> 
>    As an added benefit, TLS can also negotiate stream-level compression.
>    Â I believe all browsers today send the null-set of compressors
>    (preventing TLS based compression from being negotiated), but the
>    protocol does support it without an extra round trip.
> 
>    However, I doubt you want to compress framing bytes, control frames,
>    etc as well. It seems more likely that putting compression inside
>    WebSocket will result in better results.

Where there are independent streams multiplexed (even if it's at the
application level), sometimes you'd get better results with multiple compression contexts.

Where some of the data is already compressed, you would not want to
recompress that part of the stream.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 14 May 2010 05:21:04 +0100

Roberto Peon wrote:
> 
>    On Thu, May 13, 2010 at 8:55 PM, Jamie Lokier <[1][email-masked]>
>    wrote:
> 
>    Roberto Peon wrote:
>    >
>    Its a feature! You know all those deployment problems we're having
>    right now? Guess what you can point your finger at (hint: it is not
>    the server, nor the client).!

Agreed.

>    Trying to work around the brokenness of transparent proxies is
>    *tough*. If you want your protocol to have any chance of being
>    upgraded in the near future, you should either anticipate lots of
>    problems with transparent proxies, or you should make it so that they
>    cannot mess up stuff for you in the future.

Sure.  But there's the converse problem: I want to write transparent
proxies that deliberately detect and modify WebSocket traffic, to
improve its network overhead and performance.

>      I regard multiplexing and keepalive sharing as especially
>      valuable due to connection overheads and the large number of
>      mostly-idle WebSocket connections anticipated, even if it's
>      only proxy-to-proxy (the endpoints don't need to know about
>      it.)
> 
>    Then we should ensure that we can multiplex WebSockets within the
>    browser to keep OS level overhead to reasonable levels!

Is that a serious suggestion?  I wouldn't mind that, but I haven't
seen much serious discussion of it.  It kept getting dismissed when I
raised it...

There's a problem, though: The browser does not know enough about the
network hop-by-hop to select good keepalive parameters for every hop,
without help from the proxies which sometimes know better.

Moreover, when you have 10 users sharing a single mobile 3G
connection, there is something to be said for sharing their keepalives
at the 3G connection point, which the browser won't know about.

I anticipate some WebSocket applications will be 99% keepalive traffic
(e.g. all the idle web pages open in background tabs/windows that are
waiting for something to happen that doesn't happen often), making
keepalives by far the dominant bandwidth cost in some applications,
and therefore worth optimising.

>    Again, if you want to do that, you install root certs on the clients,
>    and then the firewall can do whatever it wants to it, since it can
>    pretend to be the endpoint.

No, that application is about _not_ modifying the clients.

-- Jamie

---

From: Mike Belshe <[email-masked]>
Date: Thu, 13 May 2010 21:59:55 -0700

> Port 80 == available near-universally; unencrypted == low overhead.
It seems to be universal, but it is not... see below.

> and mess about with port 80 connections.

As you point out, it seems there are really only 2 ports available - 443 and
80.  Other ports are often firewalled.

But the data has shown that port 80 is so infested with tampering
intermediaries, that it is effectively not universally available for the
purposes of a websocket upgrade.  In fact, our tests show that using any
port other than port 80 is more effective overall for successful websocket
establishment.  I can dig out the numbers if you like, but basically TLS is
best, then a non-standard port, followed by port 80.  They're on this list
somewhere.

> universal availability, there's a thought...)
That solves none of the problems for deploying over port 80.

Mike

---

From: Maciej Stachowiak <[email-masked]>
Date: Thu, 13 May 2010 22:25:35 -0700

> certificated == impossible to coalesce by intermediaries.

Do you have evidence that port 80 is more widely available than 443?

> universal availability, there's a thought...)

Port 80 already has a way to upgrade to TLS, see RFC2817. I don't think we need to create a different way to upgrade HTTP to TLS. Note that you could do the HTTP-to-TLS upgrade followed by the WebSocket upgrade to talk over port 80, though I think that would be a pretty harsh requirement for a standalone server.

Regards,
Maciej

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 14 May 2010 08:58:28 +0200

> We already have such a handshake, so that design work is basically done.

Hmmm,

While having a notice ahead of time that a connection is intended
to be websocket would definitely help, it would still be very
strange to have an I-can't-believe-it's-not-HTTP style handshake.

In a general purpose HTTP server, I would still be confronted with
the choice of re-implementing parts of HTTP or trying to change
existing HTTP components to try to get some code reuse.

Surely if we know something is websocket when we accept the
connection, it would be far simpler for all if that connection
only ever spoke websocket frames.

Besides, as Scott's proposal shows, there are good use-cases
for control frames during a connection as well as at the start.

So instead of a broken HTTP handshake, why can't we just have
a simple control frame mechanism and some required initial
control frames?  (and this would also work over a simplified
HTTP upgrade (with worse RTT) if we wanted port 80 as a fallback).

regards

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 14 May 2010 09:06:13 +0200

> numbers of connections - most of them idle, but still using TLS memory.

Actually this is a very good point!

For us, TLS represents at best a doubling of buffer requirements,
but at worst it's many times more because it is very very difficult
to do async SSL, where the server can manage a recycle unused buffers.

Several server developers have already said that one of our key
scalability hurdles is number of connections.

Thus while I'm very interested in the TLS option, I think it
would only be viable on the server side if we have a protocol
with well defined multiplexing and connection sharing.

regards

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 14 May 2010 09:19:12 +0200

All,

Isn't the obvious solution to come up with a pure websocket
protocol that is not dependent on any parameter exchange in
a HTTP (or HTTP like handshake), but instead uses websocket
control frames.

This wire protocol can then be used over:

  directly over a new dedicated port
  directly over a new dedicated port with TLS upgrade
  directly over 443 with nextproto extension
  over 443 with a simple compliant HTTP upgrade
  over 80 with a simple compliant HTTP upgrade

The simple HTTP upgrade connections would suffer from
an extra RTT, so there would be a good driver towards
using the dedicated port (and thus getting them opened
and supported by intermediaries).

Meanwhile the HTTP upgrade path would provide a moderately
reliable fallback.

Simple clients/servers need only support 1 or 2 connection
mechanisms.

Complex client/servers could support all connection mechanisms
and try them in a fall back strategy.  The RTTs would be
horrendous for the first connection, but the browser could
cache the known working connection types for given destination.

Multiplexing would also need to be supported because
of TLS overheads, and this would also greatly reduce
RTTs.

The really good thing about this approach, is that somebody
else has done most of the work already:

http://www.chromium.org/spdy/spdy-protocol/spdy-protocol-draft2#TOC-Framing-Layer

regards

---

From: Ian Hickson <[email-masked]>
Date: Fri, 14 May 2010 07:20:20 +0000 (UTC)

On Fri, 14 May 2010, Greg Wilkins wrote:
> 
> While having a notice ahead of time that a connection is intended to be 
> websocket would definitely help, it would still be very strange to have 
> an I-can't-believe-it's-not-HTTP style handshake.
> 
> In a general purpose HTTP server, I would still be confronted with the 
> choice of re-implementing parts of HTTP or trying to change existing 
> HTTP components to try to get some code reuse.

I'd be happy with making trivial changes to the handshake to take it 
further away from HTTP if we did this. That would be pretty easy while 
keeping the characteristics that make it relatively safe.

> Surely if we know something is websocket when we accept the connection, 
> it would be far simpler for all if that connection only ever spoke 
> websocket frames.

I don't think it's particularly useful for the handshake to use the same 
framing mechanism as the main Web Socket text (similarly, HTTP doesn't use 
chunking for headers). We should use appropriate syntax for each part. If 
we don't have to be compatible with HTTP, though, we could definitely make 
things simpler, e.g. using LF as a field terminator rather than CRLF, 
removing the spaces between the field names and the field values (i.e. 
separating them with ":" rather than ": "), changing the first line to be 
consistent with the rest (rather than the GET and 200 OK lines), etc.

---

From: Maciej Stachowiak <[email-masked]>
Date: Fri, 14 May 2010 00:31:38 -0700

> with well defined multiplexing and connection sharing.

What is the per-connection memory cost for a TLS connection? Are we talking about server side or client side?

Regards,
Maciej

---

From: Maciej Stachowiak <[email-masked]>
Date: Fri, 14 May 2010 00:36:28 -0700

>  over 80 with a simple compliant HTTP upgrade

The last two options are not "simple" because the HTTP upgrade mechanism would have to be designed to avoid cross-protocol attacks. 

> Simple clients/servers need only support 1 or 2 connection

Browser-hosted clients would have to support all supported connection types. As a browser implementor, I'm not really interested in supporting 5 different ways to set up a WebSocket connection.

> cache the known working connection types for given destination.

Trying to negotiate among multiple ways to do the connection would not only be a performance hit but it would also greatly increase the complexity of the protocol.

I think TLS nextproto extension, plus *maybe* use of TLS upgrade on plain HTTP connections, should be sufficient. I can't think of any new use cases that would be opened up by the other options. That way there is only one form of the WebSocket handshake.

Regards,
Maciej

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 14 May 2010 10:25:38 +0200

> What is the per-connection memory cost for a TLS connection? Are we talking about server side or client side?

Probably both.

But my experience is on server side java and
this problem is probably more a problem of the java
libraries than with TLS itself - but it's still
a problem.

In order to scale HTTP, we do not permanently allocate any
buffers per connection.  We can typically service many thousands
of connections with only a few hundred buffers.

But with SSL in java, you have a choice of using the provided
blocking SSL sockets (which allocate 128k of crypto buffers)
or using the async SSL engine - which is rather over complex and
very difficult to use, specially with the complexities of java
having both kernel space and user space buffers, of JVM kernel
space buffer allocation bugs etc.

So in practice it proves far simpler to just allocated crypto
buffers to the SSL connections.   If all our connections are
SSL, then we will have out-of-memory issues.

This is a solvable problem, but it's not easily solvable.

I suspect that the vast majority of SSL libraries
also take the simple road and allocated buffers
to connections.

regards

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 14 May 2010 10:39:28 +0200

Maciej Stachowiak wrote:
> The last two options are not "simple" because the HTTP upgrade mechanism
> would have to be designed to avoid cross-protocol attacks. 

I thought that we were mostly agreed that a simple nonce
mechanism would solve such cross protocol issues.

Also if the majority of the handshake is done in
websocket framing, then that is also protection against
cross protocol attacks.

> Browser-hosted clients would have to support all supported connection
> types. As a browser implementor, I'm not really interested in supporting
> 5 different ways to set up a WebSocket connection.

Sure.

But there is no silver bullet and no 1 technique is going
to give you 100% connection rate.

Ideally with some more research like that already conducted by
google, we can pick 2 or 3 techniques that will give
us a very high success rate.    The 2 or 3 techniques
used may also change over time, with the preferred
methods of today being replaced by better methods
as they become better supported.

Thus it makes more sense to put as much of the handshake
smarts into something that can be common (eg websocket
control frames) to all connection mechanisms.

The -76 spec already has bytes sent after the HTTP
handshake.  Why not just turn those into websocket
control frames and make them do most of the
heavy lifting of the handshake.

> Trying to negotiate among multiple ways to do the connection would not
> only be a performance hit but it would also greatly increase the
> complexity of the protocol.

Hopefully we could make a cleaner separation between the protocol
and the mechanisms used to open a protocol connection.

> I think TLS nextproto extension, plus *maybe* use of TLS upgrade on
> plain HTTP connections, should be sufficient. I can't think of any new
> use cases that would be opened up by the other options. That way there
> is only one form of the WebSocket handshake.

If the one form of websocket handshake was in a websocket frame,
then it could be used regardless of how the connection was opened.

regards

---

From: Adam Barth <[email-masked]>
Date: Fri, 14 May 2010 07:42:14 -0700

> mechanism would solve such cross protocol issues.

I certainly didn't agree to this proposition.

> websocket framing, then that is also protection against

I'm not sure you fully appreciate the subtitles of cross-protocol
vulnerabilities.  They are very difficult to reason about.  For
example, the reason we can't use the DNS protocol to completely break
the security of HTTP in browsers relies on an esoteric property of how
certain unusual characters are processed by the HTML form element.

Adam

---

From: Scott Ferguson <[email-masked]>
Date: Fri, 14 May 2010 08:16:15 -0700

Mike Belshe wrote:
>
>
> On Thu, May 13, 2010 at 9:08 PM, Jamie Lokier <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Ian Hickson wrote:
>
>
> As you point out, it seems there are really only 2 ports available - 
> 443 and 80.  Other ports are often firewalled.  
>
> intermediaries, that it is effectively not universally available for 
> the purposes of a websocket upgrade.  In fact, our tests show that 
> using any port other than port 80 is more effective overall for 
> successful websocket establishment.  I can dig out the numbers if you 
> like, but basically TLS is best, then a non-standard port, followed by 
> port 80.  They're on this list somewhere.

The firewall issue is only a problem short term, though. If WebSockets 
used 880/443, within a few years the 880 port would be treated like HTTP 
for the firewalls.

-- Scott

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 14 May 2010 16:38:43 +0100

Greg Wilkins wrote:
> Maciej Stachowiak wrote:
> 
> > What is the per-connection memory cost for a TLS connection? Are
> > we talking about server side or client side?
> 
> Probably both.
> 

If you're using blocking SSL sockets, then you will also have a thread
per connection, which adds more memory (for thread stacks and
per-thread storage), prevents using NIO for performance (in Java), and
does not scale as well as event-driven code (on a single machine) to
the large numbers of mostly-idle connections we've predicted.

It can still work provided you don't mind making a larger cluster :-)

> also take the simple road and allocated buffers

I believe OpenSSL allocates about 128k per idle connection.

It is not a ridiculously high overhead, because TCP will use some
amount of kernel send buffers - although those can be limited, can be
handed off quickly to a front-end load balancer (so the originator can
use small buffers), and are subject to some amount of sharing among
TCPs in a single machine.

So as a rough, worst-case back of the envelope guess, with 128k in the
TLS library, 16k TCP buffer in a system with a nearby load balancer
not optimised for sending bulk data per connection, and 128k per-thread
stacks as opposed to 4k event-driven state, let's call it...

12 times as much memory for the TLS version at high scaling :-)

(Or more, if anyone wants to send static resources over it,
such as one might with SPDY, due to inability to use sendfile().)

But that is a worst-case, rough guess, and it's not my field anyway.

-- Jamie

---

From: Mike Belshe <[email-masked]>
Date: Fri, 14 May 2010 08:46:05 -0700

> to do async SSL, where the server can manage a recycle unused buffers.

This sounds like an implementation issue.  There is nothing inherent to the
protocol which causes this.  It is generally true that TLS handshakes are
more expensive than non-TLS handshakes though.

I'm happy to provide data here on scalability.  You do need to be careful on
which TLS implementation you use - I've seen a few that are unbelievably
poor implementations.  Maybe any tests done should compare t least two
implementations.

> scalability hurdles is number of connections.

It would be good to quanitfy here - lots of things effect scalability.  How
much is too much?

Mike

---

From: Mike Belshe <[email-masked]>
Date: Fri, 14 May 2010 09:00:48 -0700

> the large numbers of mostly-idle connections we've predicted.

This is not a protocol issue.  This is an implementation issue.

> 12 times as much memory for the TLS version at high scaling :-)

These are just guesses, a real number would be nice.

I don't believe there is anything inherent to the TLS protocol which
requires 128KB - can you think of anything?  If there is, that is a problem.
 Otherwise, your guess is just wrong or it's just an implementation bug.

> such as one might with SPDY, due to inability to use sendfile().)

This is also implementation, not protocol.

Mike

---

From: Roberto Peon <[email-masked]>
Date: Fri, 14 May 2010 09:07:32 -0700

> I believe OpenSSL allocates about 128k per idle connection.

Our flavor of this uses quite a bit less memory than that. I believe that
the patch which accomplished this is/was public.

> 12 times as much memory for the TLS version at high scaling :-)

The loadbalancer at Google which terminates all HTTP/HTTPS traffic for us
(which my team implemented) has a much better ratio than that. We do use
event driven state machines and OpenSSL in async mode with patches for small
buffers. However, truthfully, the amount of memory saved by using smaller
buffers in OpenSSL was smaller than the amount of memory saved by ensuring
that we never make a copy of the data unless we're doing a transformation of
it, and ensuring that we can stream data with minimal buffering (e.g. don't
read in an entire request and then forward. Read as little as you can and
then forward).
If we're worried about memory scalability, then we should be sure that
proxies have the ability to fragment frames, but that is another topic.

> such as one might with SPDY, due to inability to use sendfile().)

I wish that sendfile worked better than it does. Theoretically it (and
splice()) improves performance...

-=R

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 14 May 2010 21:37:00 +0200

Mike Belshe wrote:

> This sounds like an implementation issue.  There is nothing inherent to
> the protocol which causes this.  It is generally true that TLS
> handshakes are more expensive than non-TLS handshakes though.  

Certainly it's a implementation issue and mostly solvable.

For large scale servers, they need already need to be smart with
their SSL implementations.   I think such memory issues will mean
more medium sized servers will also have to be as smart.

> It would be good to quanitfy here - lots of things effect scalability.
>  How much is too much?

This is hard to quantify and there are no fixed limits and your
mileage may vary etc.

My experience is that we have achieved scalability by having
a focus on minimizing resources allocated per connection.
Others here have expressed similar sentiments.

Currently websocket can significantly increase the number
of connections per client - essentially with an unlimited
number of connections in addition to the already growing
number of HTTP connections.  This was already a worry
without TLS.  If we also use TLS, then that
would be a double whammy of more connections and more
resources per connection. Scalability would suffer.

So if we use TLS, then I think it is even more important
that we consider how we can limit and/or reduce the
usage of connections. Multiplexing would help.
Sharing connections with HTTP traffic is also a good idea.

cheers

---

From: Jamie Lokier <[email-masked]>
Date: Sat, 15 May 2010 01:18:14 +0100

>    proxies have the ability to fragment frames, but that is another

They can't do that if it's TLS, can they?

-- Jamie

---

From: Roberto Peon <[email-masked]>
Date: Fri, 14 May 2010 17:42:42 -0700

Sure, why not? That is/would be a feature of the protocol running over TLS.
-=R

---

From: John Tamplin <[email-masked]>
Date: Sat, 15 May 2010 08:56:36 -0400

> Sure, why not? That is/would be a feature of the protocol running over TLS.

If it is encrypted, the frames of the protocol over TLS are just opaque data
to an intermediary (which is the point of the encryption after all).  If you
want an intermediary to be able to fragment the encrypted frames, that would
have to be supported at the TLS layer.

---

From: Roberto Peon <[email-masked]>
Date: Sat, 15 May 2010 12:18:30 -0700

Yup. The intermediary would have to be able to terminate the TLS session and
create another.
Other than corporate proxies and reverse proxies, can you tell me of a
use-case for making an intermediary for websockets that really makes sense,
given that we're attempting to allow "unstructured" data?

-=R

On Sat, May 15, 2010 at 5:56 AM, John Tamplin <[email-masked]> wrote:

> If it is encrypted, the frames of the protocol over TLS are just opaque
> data to an intermediary (which is the point of the encryption after all).
>  If you want an intermediary to be able to fragment the encrypted frames,
> that would have to be supported at the TLS layer.
>
> --
> John A. Tamplin

---

From: Scott Ferguson <[email-masked]>
Date: Sat, 15 May 2010 14:54:03 -0700

I've written up a draft of what the alternative web socket handshake 
might look like at

http://hessian.caucho.com/websockets/draft-ferg-hybi-websockets-latest.html

(basically so we can make sure we're all talking about the same idea.)

There are some differences in that draft from what some people have been 
discussing:

1. The server can open a plain WebSocket TCP port (at a new port like 
880) that does not use TLS.  That port cannot be shared with HTTP.

2. The TLS port 443 is shared with HTTP (assuming tls-nextprotoneg)

3. The client and server handshake is with a HELLO control frame. The 
first line is "WebSocket/1.0" and the next lines are headers. So a 
request might look like:

 %x80.02.00.85 WebSocket/1.0
               url: ws://example.com/sample
               origin: http://example.com/sample.php
               protocol: text

The response might look like:

 %x80.02.00.85 WebSocket/1.0
               status: 200
               origin: http://example.com/sample.php
               protocol: text

4. Since the resource is identified by the full URL, there's no separate 
host header.

5. "protocol" identifies the application protocol, if specified by the 
client API.  The unspecified protocol is "text", which is UTF-8 encoded 
strings like the JavaScript API.

6. Client pipelining is built into the specification. The client may 
send data immediately following its HELLO; it does not need to wait to 
receive the server's data.

7. The frame header is now a fixed 32 bits for both data frames and 
control frames.  Data frames have the control flag, a "more" flag, 2 
unused bits, and 28-bit length. The high bit is 0 (meaning data):

  0 | M(1) | X(2) | length(28)

8. The control header now includes the control opcode. To make room, the 
length is limited to 16 bits (we can add a More flag if that's a 
program, but I think it's a reasonable limitation.) The high bit is 1 
(meaning control):

 1 | X(7) | OP(8) | length(16)

9. The multiplexing extension was moved out. I'll see if I can get out a 
first draft of a possible mux this weekend.

-- Scott

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 16 May 2010 20:38:50 +0100

>    This is not a protocol issue.  This is an implementation issue.

Yes, I agree.

However, it is a practical issue if scaling is particularly difficult
to implement.  I don't know enough SSL/TLS to say - only that my very
limited experience mirrors Greg's (about 128k per connection), and I
know enough about "blocking" to know that it forces thread overhead
into the mix too.

WebSocket should not be designed without regard for practical
consequences, to the extent those may be difficult to overcome in the
timescale when it's going to be used.

>      amount of kernel send buffers - although those can be limited,
>      can be handed off quickly to a front-end load balancer (so the
>      originator can use small buffers), and are subject to some
>      amount of sharing among TCPs in a single machine.  So as a
>      rough, worst-case back of the envelope guess, with 128k in the
>      TLS library, 16k TCP buffer in a system with a nearby load
>      balancer not optimised for sending bulk data per connection,
>      and 128k per-thread stacks as opposed to 4k event-driven state,
>      let's call it...  12 times as much memory for the TLS version
>      at high scaling :-)
> 
>    These are just guesses, a real number would be nice.  I don't
>    believe there is anything inherent to the TLS protocol which
>    requires 128KB - can you think of anything?  If there is, that is
>    a problem.  Otherwise, your guess is just wrong or it's just an
>    implementation bug.

I don't know enough about SSL/TLS to know if there's a protocol reason.
(But I think you're probably right - no reason).

128kB it is an actual rough measurement of OpenSSL, and 128kb is an
actual thread stack size; they're not guesses (but the stack won't be
fully used).  Maybe it varies between versions; also OpenSSL is not
the only TLS library.  I haven't looked at others.

As a best-case, it's definitely off.  As a worst-case using
off-the-shelf code, compared against an event-driven server optimised
for large numbers of connections, not so much.

It's fair to argue that TLS doesn't need it, or OpenSSL is a poor
implementation choice, on the other hand, TLS is a rather complicated
protocol, and new implementations by those who aren't crypto-security
experts is discouraged.

WebSocket would be almost unique, as a TLS user, in seeming to use
10-50 connections per individual user by design (the current one,
where browsers can't share connections between WebSocket objects).
TLS itself isn't designed with that kind of usage in mind.

>    This is also implementation, not protocol.

To design a protocol with no regard to implementation is
inappropriate.  Regarding sendfile(), the specifics are connected with
currently available network hardware's TCP acceleration methods - it's
not just a software issue.  But it would be fair enough to decide that
WebSocket will simply not be designed to assist that sort of thing.

-- Jamie

---

From: Maciej Stachowiak <[email-masked]>
Date: Sun, 16 May 2010 17:16:48 -0700

> TLS itself isn't designed with that kind of usage in mind.

People keep saying this, but do we have any evidence there would be 10-50 connections per user to the same server? For web apps using the "hanging get" approach, do you get 10-50 connections per user? If not, why would you expect it for WebSocket? None of the typical "hanging get" techniques have a built-in way to do connection sharing. Why do we expect things to be different with WebSocket?

Regards,
Maciej

---

From: Roberto Peon <[email-masked]>
Date: Sun, 16 May 2010 19:39:58 -0700

On Sun, May 16, 2010 at 5:16 PM, Maciej Stachowiak <[email-masked]> wrote:

> People keep saying this, but do we have any evidence there would be 10-50
> connections per user to the same server?

Yes, and that is *without* the hanging gets.
Just try out the Alexa top XXX, and see for yourself!

For web apps using the "hanging get" approach, do you get 10-50 connections
> per user? If not, why would you expect it for WebSocket? None of the typical
> "hanging get" techniques have a built-in way to do connection sharing. Why
> do we expect things to be different with WebSocket?
>

It seems that more and more sites are using collaborative
and/or/notification-based features. It seems natural that the number of
requested concurrent streams will increase as a result. The assumption that
# concurrent streams == # TCP connections does assume that we don't build in
multiplexing (or don't leave ourselves the possibility open for the future).
With multiplexing, the number of connections can go down to the number of
hosts involved. I sure hope for the sake of the poor bottleneck (aka
servers) that we do multiplexing right.

-=R

---

From: Vladimir Katardjiev <[email-masked]>
Date: Mon, 17 May 2010 09:34:00 +0200

On 15 maj 2010, at 21.18, Roberto Peon wrote:

> Yup. The intermediary would have to be able to terminate the TLS session and create another.
> Other than corporate proxies and reverse proxies, can you tell me of a use-case for making an intermediary for websockets that really makes sense, given that we're attempting to allow "unstructured" data?

A mobile operator proxy performing opportunistic protocol optimisation such as, for example, changing the keepalive(s) to a better interval given the connection, applying compression, or buffering frame(s) in order to minimise the uptime of a device's radio electronics. Lab tests have shown that as few as three independent keepalives can drain a smartphone's battery in 16 hours, without the user touching the device at all.

Allowing these two, simple mechanics to be applied by intermediaries on a network path that can make use of such optimisation (e.g. wireless networks), this provides a tremendous benefit for the devices (they are usable; I think that's a benefit) while requiring no additional complexity server-side and only small additional complexity client-side (compression).

Of course, there's a lot more that could be done to improve the efficiency, but these two things are extremely simple to accommodate.

Vladimir

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 17 May 2010 10:48:47 +0200

On 15/05/10 21:18, Roberto Peon wrote:

> > use-case for making an intermediary for websockets that really makes
> > sense, given that we're attempting to allow "unstructured" data?
Roberto,

I agree that other than the cases that you describe, there are few
real use-cases for intermediaries on the public segments of a websocket
connection  (although as an Australian, my government appears to
think it has the right to legislate that every connection is
inspected for censorship purposes - granted this makes no sense).

However, it is not just intermediaries on the public segments
that we need to be concerned with.

Once a connection enters the data centre, it is highly likely that
it will be processed not by a single server, but by a sequence
of machines providing services like: TLS offload, load balancing,
connection aggregation, etc.

Huge organizations like google will write their own custom protocol
to move events between the protocol termination servers and
the application servers.   However most other organizations will
purchase/lease/outsource the infrastructure to do this
with 3rd parties.

So I think it is important that the standard extend to the
establishment of non TLS connections through heterogeneous
multi-vendor infrastructure.  (however, perhaps this part
of the connection need not share a port with HTTP).

regards

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 17 May 2010 10:49:33 +0200

On 17/05/10 02:16, Maciej Stachowiak wrote:
> > People keep saying this, but do we have any evidence there would be 10-50 connections per user to the same server? For web apps using the "hanging get" approach, do you get
10-50 connections per user? If not, why would you expect it for WebSocket? None of the typical "hanging get" techniques have a built-in way to do connection sharing. Why do we
expect things to be different with WebSocket?
Maceij,

we have no hard evidence yet because websockets is not yet
widely deployed.   But there is anecdotal evidenced based
on experience and observation.

My experience is that even within simple web applications developed
by the  same group, there are multiple independent concerns that
require a feed of "push" events from the server or a messaging style
interaction.

Within moderate and larger organizations, their web presence is often
developed almost like a portal sight, with components being developed
entirely independently.

Thus multiple components on these pages will result in multiple
websocket connections.

It is also a frequent occurrence that a user will either inadvertently
or deliberately open many tabs/windows to the same site, so any multiple
of connections on 1 page can easily be multiplied 2 or 3 times more.

These are already issues with hanging get frameworks, even though
they encourage shared connections when used within the same frame.

There a currently speed advantages in using multiple
connections - for an example, see the slow start paper on the
SPDY site.    Developers who realize this will deliberately open
more connections than necessary in order to achieve real or
even imaginary performance advantages.

Finally, there is the convenience aspect.  Sharing a connection
is difficult and can complicate an application.  It is something
ideally done in infrastructure.    Any developers working to
the websocket API are highly unlikely to go to the effort of
sharing a connection, when opening a new one is much easier.

cheers

---

From: James Graham <[email-masked]>
Date: Mon, 17 May 2010 16:11:53 +0200

On 05/13/2010 11:55 PM, Maciej Stachowiak wrote:

> For example, if you are coding a network server in Python, you are
> much better off using Twisted than the socket module that is part of
> the Python standard library. Not only is your program more likely to
> be correct, but your programming is likely to be easier. And it's
> apparently fairly straightforward to use TLS with Twisted. See here
> for sample TLS server
> code:<http://twistedmatrix.com/documents/current/core/howto/ssl.html>.
> It's 15 lines for an echo server.

I think trying to extrapolate complexity based on the LOC required to 
implement an echo server is likely to be highly misleading. For example 
I have the impression that twisted is generally not considered simple, 
although I admit that I have only limited experience with it.

A rather low bar for considering something to be simple in the sense of 
"accessible to the amateur" might be "is possible in python using only 
the standard-library for low-level components". By this metric, I think 
TLS is borderline simple; Python >= 2.6 has the ssl module [1] (note 
that many web hosts will only have older pythons installed). I am not 
expert enough to know if this module provides all the features required 
by Adam's proposal without substantial work from the web-sockets server 
author. In any case it seems obvious that the use of TLS does make the 
protocol more complex, and, if this approach is taken, we should be 
clear that we are trading simplicity for something of greater value.

[1] http://docs.python.org/library/ssl.html

---

From: Roberto Peon <[email-masked]>
Date: Mon, 17 May 2010 09:16:53 -0700

Sure. Note that by removing the choice from the user we end up with cases
where you have tethered your computer to your phone, and the proxy is
helpfully stripping the bits off of the jpg data for the mapping application
that you're attempting to use.
Proxies do benefit some users, but, without the user's ability to choose to
not use them, they can do serious harm.
In the case that you propose here, the phone seller could provide the
appropriate proxy settings. They could install a root cert on the phone
belonging to the provider.
If that sounds scary, well, it is exactly equivalent to what you're
proposing already :)
-=R

On Mon, May 17, 2010 at 12:34 AM, Vladimir Katardjiev <[email-masked]>wrote:

> A mobile operator proxy performing opportunistic protocol optimisation such
> as, for example, changing the keepalive(s) to a better interval given the
> connection, applying compression, or buffering frame(s) in order to minimise
> the uptime of a device's radio electronics. Lab tests have shown that as few
> as three independent keepalives can drain a smartphone's battery in 16
> hours, without the user touching the device at all.
>
> Allowing these two, simple mechanics to be applied by intermediaries on a
> network path that can make use of such optimisation (e.g. wireless
> networks), this provides a tremendous benefit for the devices (they are
> usable; I think that's a benefit) while requiring no additional complexity
> server-side and only small additional complexity client-side (compression).
>
> Of course, there's a lot more that could be done to improve the efficiency,
> but these two things are extremely simple to accommodate.
>
> Vladimir

---

From: Vladimir Katardjiev <[email-masked]>
Date: Mon, 17 May 2010 19:46:24 +0200

On 17 maj 2010, at 18.16, Roberto Peon wrote:

> Sure. Note that by removing the choice from the user we end up with cases where you have tethered your computer to your phone, and the proxy is helpfully stripping the bits off of the jpg data for the mapping application that you're attempting to use.
> Proxies do benefit some users, but, without the user's ability to choose to not use them, they can do serious harm.
> In the case that you propose here, the phone seller could provide the appropriate proxy settings. They could install a root cert on the phone belonging to the provider.
> If that sounds scary, well, it is exactly equivalent to what you're proposing already :)

I'm not really sure where I should start answering that... First off, the statement I replied to was if there was a use-case for an intermediary for websockets other than corporate proxies (which I assume means filtering on content) and reverse proxies (load balancing). This I showed.

Now, the question is user choice. I hold that user choice is equivalent in both encrypted and unencrypted transfers. If a user is capable enough to know what a proxy is I'm sure they can find the settings to turn it off. However, given that a not insignificant proportion of users choose to enter www.google.com into the Firefox google search box to reach google.com, I also hold that requiring users to access esoteric settings such as proxy configuration is about as helpful as telling a thirsty man in a desert that there's water some 500km due west.

Could the phone seller provide the appropriate root CA? Yes, they could. But what happens when you change operator, but keep your phone? (This might not be a US-relevant concern, but it's certainly EU-relevant). Are you going to provide them with an instruction booklet on how to install a root CA, or are you going to tell them to recharge the phone three times a day.

Practically speaking, it's not improbable that operators would just require a root cert belonging to some CA more than willing to allow this proxying to happen on every single handset sold. If you really want to, I can go into detail on why this is so (it's a long story), but the fact is that, for a mobile network, sporadic traffic of occasional 1kb packets (such as keepalives) is orders of magnitude worse for performance than more consistent traffic over a longer period of time. So the incentive would definitely be there; the question is why do we want to give it to them?

I say we spec an appropriate way for WebSockets to be proxied, including the ability to adjust keepalives, apply compression, etc. When the client has a configured WS proxy, use regular unencrypted websockets for ws://. When there's no WS proxy configured, or an HTTP one, use TLS. This gives incentive for WS proxy deployment without breaking the ability to optimise for traffic (or having to decrypt/encrypt in order to perform it), or the installation of root certs. It allows the immediate deployment of websockets without mandating a technological game of cat and mouse. And the user choice is covered. Seems like a perfect solution to me...

Vladimir

> On Mon, May 17, 2010 at 12:34 AM, Vladimir Katardjiev <[email-masked]> wrote:
>

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 17 May 2010 19:28:18 +0100

Roberto Peon wrote:
> 
>    On Sun, May 16, 2010 at 5:16 PM, Maciej Stachowiak <[1][email-masked]>
>    wrote:
> 
>      People keep saying this, but do we have any evidence there would be
>      10-50 connections per user to the same server?

I did explain in an earlier mail where that figure comes from.  In my
estimate, it isn't 10-50 per server, but 10-50 per user among all
servers they are connected to.  But when upstream proxies near the
client get involved (and mobile operators in particular might wish to
do so deliberately), then they are all to the same proxy.

It's interesting that Roberto points out that figure applies even to a
single server, although I expect only the most high profile servers
would see that.

That adds up to a *lot* of keepalive traffic when you consider, say,
an organisation with 100 users behind a 24Mbit/s-down 2Mbit/s-up link.
(When working it out, remember to include packet headers...)

One of the traffic pattern differences from HTTP that we've largely
not talked about is this:

WebSockets remain active when the user has a page open but it is
inactive, in the background, or even as an iconified/minimised window.
>From what I've seen, most pages are kept in that state nearly all the
time.  That is, they're left open in a browser that isn't being used
most of the time.

There is little or no HTTP traffic, but any WebSocket connections will
be relatively active, because they need to continuously transmit
"idle" keepalives, even if there are no data messages.  That's so they
will receive server-initiated messages when those happen.

In other words, there's a real chance that keepalives will be a large
fraction of traffic, or even dominant in some applications.  Moreover,
they are steady, so the traffic from multiple users doesn't
statistically multiplex as nicely as their bursty HTTP access.

I have seen that pattern in some non-web applications using a
publish-subscribe update model, which is one of the use-cases for
WebSocket in web pages.

>    Just try out the Alexa top XXX, and see for yourself!

That's useful, thank you.

(Btw, Imho we could really do with collecting the most useful
pointers, stats and so on together, out of the the many discussion
emails.  Anyone volunteer to help?)

>      For web apps using the "hanging get" approach, do you get 10-50
>      connections per user? If not, why would you expect it for
>      WebSocket? None of the typical "hanging get" techniques have a
>      built-in way to do connection sharing. Why do we expect things to
>      be different with WebSocket?
> 
>    and/or/notification-based features.

It's the notification-based features - or more generally,
server-initiated client updates - which I'm thinking about when I think
of WebSocket usage.

>    It seems natural that the number of requested concurrent streams
>    will increase as a result. The assumption that # concurrent
>    streams == # TCP connections does assume that we don't build in
>    multiplexing (or don't leave ourselves the possibility open for
>    the future). With multiplexing, the number of connections can go
>    down to the number of hosts involved. I sure hope for the sake of
>    the poor bottleneck (aka servers) that we do multiplexing right.

If there was more of a consensus in the direction of doing
multiplexing, or of an extension subprotocol that multiplexes, I'd be
happy to help make sure it is done well.  At the moment, the balance
of views expressed seems to be against multiplexing, and in favour of
one TCP connection per WebSocket object, but only just.

Within a single page, WebSocket browser apps can multiplex now - but
with faults.

It is done at the application layer by an app needing to dispatch
messages to different parts of the page.  The main issues I see in
multiplexing at the browser Javascript level, at present, are:

  - Separate open tabs/windows will have no choice but to use separate
    WebSockets.

  - Parts of a page authored using different frameworks, will
    inevitably use their own WebSocket objects.  (This is where the
    prediction that active ads will tend to add their own per-page
    WebSockets comes from.)

  - Without per-"channel" flow control (without considerable effort),
    if one independent part of the page, such as an iframe or WebWorker
    or whatever is unable to keep up with incoming updates, either the
    rest of the page stops receiving updates (a kind of head of line
    blocking), or the browser must use ever-increasing memory,
    queueing events faster than they are processed, to allow messages
    for the rest of the page to get through.

-- Jamie

---

From: Roberto Peon <[email-masked]>
Date: Mon, 17 May 2010 12:21:09 -0700

On Mon, May 17, 2010 at 10:46 AM, Vladimir Katardjiev <[email-masked]>wrote:

> I'm not really sure where I should start answering that... First off, the
> statement I replied to was if there was a use-case for an intermediary for
> websockets other than corporate proxies (which I assume means filtering on
> content) and reverse proxies (load balancing). This I showed.
>

Agreed.

>
> Now, the question is user choice. I hold that user choice is equivalent in
> both encrypted and unencrypted transfers. If a user is capable enough to
> know what a proxy is I'm sure they can find the settings to turn it off.
> However, given that a not insignificant proportion of users choose to enter
> www.google.com into the Firefox google search box to reach google.com, I
> also hold that requiring users to access esoteric settings such as proxy
> configuration is about as helpful as telling a thirsty man in a desert that
> there's water some 500km due west.
>
> Could the phone seller provide the appropriate root CA? Yes, they could.
> But what happens when you change operator, but keep your phone? (This might
> not be a US-relevant concern, but it's certainly EU-relevant). Are you going
> to provide them with an instruction booklet on how to install a root CA, or
> are you going to tell them to recharge the phone three times a day.
>
> Practically speaking, it's not improbable that operators would just require
> a root cert belonging to some CA more than willing to allow this proxying to
> happen on every single handset sold. If you really want to, I can go into
> detail on why this is so (it's a long story), but the fact is that, for a
> mobile network, sporadic traffic of occasional 1kb packets (such as
> keepalives) is orders of magnitude worse for performance than more
> consistent traffic over a longer period of time. So the incentive would
> definitely be there; the question is why do we want to give it to them?
>

This also argues strongly that we want multiplexing for WS for mobile
devices, to help preserve their CPU, reduce radio-xmit-time, and possibly to
reduce load/costs for the network providers.

>
> I say we spec an appropriate way for WebSockets to be proxied, including
> the ability to adjust keepalives, apply compression, etc. When the client
> has a configured WS proxy, use regular unencrypted websockets for ws://.When there's no WS proxy configured, or an HTTP one, use TLS. This gives
> incentive for WS proxy deployment without breaking the ability to optimise
> for traffic (or having to decrypt/encrypt in order to perform it), or the
> installation of root certs. It allows the immediate deployment of websockets
> without mandating a technological game of cat and mouse. And the user choice
> is covered. Seems like a perfect solution to me...
>

And when the user is using a websockets-enabled site to deal with their
medical history, it should automatically go out over the air with no
encryption?

If we want to make websockets-secure and websockets separate, that is
perfectly fine. It is a potentially valid use-case. Keep in mind that
without the secure option, many applications will not be able to safely use
websockets.

If we must absolutely support proxying because we must cede control of the
channel over to the cell providers, then we should make that a first-class
requirement. It sounds dubious to me. I have a hard time trusting that
network operators will consistently do the right thing with those proxies
for their customers.

-=R

>> A mobile operator proxy performing opportunistic protocol optimisation
>> such as, for example, changing the keepalive(s) to a better interval given
>> the connection, applying compression, or buffering frame(s) in order to
>> minimise the uptime of a device's radio electronics. Lab tests have shown
>> that as few as three independent keepalives can drain a smartphone's battery
>> in 16 hours, without the user touching the device at all.
>>
>> Of course, there's a lot more that could be done to improve the
>> efficiency, but these two things are extremely simple to accommodate.
>>
>> Vladimir

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 17 May 2010 20:24:45 +0100

> on experience and observation.

I agree.  We have current user behaviour and current HTTP messaging
styles to guide us, and a bit of vision about what might become viable
with an easier to use WebSocket object.  That's all we can do prior to
the first big rollout.

If there is data on hanging GET, that would be very helpful.

Data on rapid polling AJAX would also be helpful - because that is
often used today due to hanging GET having implementation difficulties
and also perceived complexity.  If WebSocket takes off, it's likely to
replace some of these rapid polling AJAX uses too.

> require a feed of "push" events from the server or a messaging style

I also think this will increase considerably, as increasingly
sophisticated frameworks and server-side languages are able to
automate event-driven updates from simpler descriptions, and an
increasing trend to web developers being expected to provide "web app"
style delivery gain traction (along with tools to make it easier).

> of connections on 1 page can easily be multiplied 2 or 3 times more.

Yes, easily.

Also, having lots of open but "background" tabs/windows is quite
common, and it's not uncommon for those to be open 24 hours a day.
Those have little HTTP traffic when backgrounded, but their WebSockets
will continuously emit a level of keepalive traffic.

> they encourage shared connections when used within the same frame.

It would be great to have some hard data on the current network/server
impact of hanging get frameworks - and rapid polling AJAX frameworks.

They are both relevant because WebSocket (assuming it works) is a
suitable replacement for both.

(I'm still amazed that there's a crowd who consider polling AJAX every
5 seconds or so to be *inherently* more scalable than hanging GET -
because that is true in common styles of server implementation.)

Do you have any such data?

> even imaginary performance advantages.

Interesting, thanks.

(We could do with gathering the "hard numbers and data" together a
bit, I'm thinking.)

> sharing a connection, when opening a new one is much easier.

More than difficult - it's technically impossible (or extremely
difficult) for Javascript apps to share connections between
tabs/windows in a single browser, between the browser and non-browser
apps (such as phonetop status applets), and between multiple users
behind a shared connection.

There are also client environments where concentrating users'
connections through a local proxy and fanning them out at upstream
proxy on the other side of a limited uplink - to reduce keepalive
costs and aggregate short messages into fewer packets - would be
useful.  I have no data on how common this is going to be with
WebSocket (but I have done it, with other protocols, and it makes
quite a difference.)

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 17 May 2010 20:37:03 +0100

> Sharing connections with HTTP traffic is also a good idea.

Multiplexing over TLS works for me technically as far as scaling at
the client and server.  I'd worry a little about the inability to
fan-in and fan-out at intermediate proxies, which is not likely to be
relevant to the majority of locations, but is potentially useful in
some networks.

My main concern, with TLS, would be the social/political ramifications
of everything being encrypted and end-to-end authenticated.

Maybe that's a good direction to take the web, but are we sure
WebSocket and the Hybi forum is the place to push it, starting from a
technical concern?

If port 443 is needed, why not send non-TLS traffic over it?  Are
proxies which pass TLS bytes unchanged going to recognise unencrypted
data and mangle it?  Is it forbidden to use anything other than TLS
over it?

Thanks,
-- Jamie

---

From: Vladimir Katardjiev <[email-masked]>
Date: Mon, 17 May 2010 22:11:47 +0200

On 17 maj 2010, at 21.21, Roberto Peon wrote:

> On Mon, May 17, 2010 at 10:46 AM, Vladimir Katardjiev <[email-masked]> wrote:
> 
> This also argues strongly that we want multiplexing for WS for mobile devices, to help preserve their CPU, reduce radio-xmit-time, and possibly to reduce load/costs for the network providers.

Yes and no. I've tried to stay out of this discussion, but maybe it's about time. First off, I think we need to clear some assumptions. When you say "multiplex", do you specifically mean "multiplex multiple WebSocket connections to the same remote host" (e.g. "google.com") or "multiplex multiple WebSocket connections to numerous remote hosts"? There's a significant difference. (I'm not suggesting this wg deals with multiplexing to/from different hosts, by the way. That's a hornet's nest I'd rather not delve into)

Earlier, I noted that three independent keepalives (10-min timers) were enough to drain a smartphone's battery over the course of 16 hours. This number will vary based on network conditions, but it's a rough ballpark (and quite Bad™). Even if multiplexing to remote hosts is applied, that still means that as long as there's more than two open connections, a smartphone is toast, so there's still a case for proxy involvement.

But a more fundamental question would be "is it actually WebSockets that is the best candidate for doing the multiplexing"?

In a typical webapp scenario, I'd expect that the webapp would load a number of HTML, CSS and JS files from my remote host. It might then establish a WS connection and retrieve some more application data periodically, which could contain references to images or other media files that are not suited for transfer over WS (right now).

This, to me, strongly suggests that it's not the WS connection (in its current incarnation) that we really want to be the one handling the multiplexing. This is likely better handled by a dedicated multiplexing protocol.
> And when the user is using a websockets-enabled site to deal with their medical history, it should automatically go out over the air with no encryption?

I explicitly went out of my way to name "regular" websockets, and specifically referred to the ws:// prefix (as opposed to the wss:// one), and this is the conclusion you draw? Yeah, I kind of feel misrepresented here.

Having said that, I don't really like TLS in the context of WebSockets because there's no trivial way to present the certificate to the user. Due to the nature of TLS, it is possible (as we discussed before) to craft certificates that will satisfy the PKI check but allow an unexpected (to the user) MITM to listen to unencrypted data. Given that some (if not many/most) ISPs and telcos happen to be or affiliate with a CA, this might even be possible without a root CA.

The "check" on TLS is that the certificate actually says "Your Bank, Inc" and not "Your ISP" when visiting your bank. But if this is never presented to the user, then the certificate can say whatever it wants as long as it is valid cryptographically. But I digress...

Ideally, control and framing should be understandable for intermediaries, but not hint on the contents of the payload (which should be opaque). I'm not holding my breath though. With that in mind, I think we should encourage non-sensitive information (such as the vast amounts of twitter bowel movement updates) to use a non-sensitive transport.

(As an aside: A possible solution is if WS is tunneled over an unencrypted multiplexed protocol, then the encrypted WS could opt out of sending keepalives. Instead, the keepalives would be sent (unencrypted) by the multiplexing protocol thus fulfilling both desires)
> 
> If we must absolutely support proxying because we must cede control of the channel over to the cell providers, then we should make that a first-class requirement. It sounds dubious to me. I have a hard time trusting that network operators will consistently do the right thing with those proxies for their customers.

Maybe they will, maybe they won't. However, that is a business  (and perhaps legislative) problem. I don't think we should try to form technical standards from a business standpoint. What I do think is that we should provide a system where it is possible to make a cooperatively better solution. The alternative – fighting over who "controls the channel" – would just be a virtual turf war where the only casualties are the users.

Vladimir

---

From: Adam Barth <[email-masked]>
Date: Mon, 17 May 2010 15:05:36 -0700

> connections, by doing the dispatch at the TLS level?

If we did that, we'd want to do something to mitigate cross-protocol
attacks for the non-TLS version of the protocol.  One way to do this
is at the application layer, e.g. by restricting web content to
opening non-TLS web sockets only on a single well-known port.

Adam

---

From: Scott Ferguson <[email-masked]>
Date: Mon, 17 May 2010 15:23:13 -0700

> opening non-TLS web sockets only on a single well-known port.
Sure, but it's cleaner to design the handshake to handle those attacks 
directly. For example, in the draft I put together, the WebSocket HELLO 
looks like:

%x80.02.00.86 WebSocket/1.0 LF
                         url=....
                         origin=...

Since that first line is invalid HTTP, using a WebSocket client on a 
HTTP server will be rejected.

And since no HTTP first line is a valid WebSocket HELLO, a HTTP request 
to a WebSocket server will also be rejected.

WebSocket to WebSocket attacks are handled by the origin header in the 
HELLO, as they are in Ian's draft.

-- Scott

---

From: Adam Barth <[email-masked]>
Date: Mon, 17 May 2010 15:42:39 -0700

On Mon, May 17, 2010 at 3:23 PM, Scott Ferguson <[email-masked]> wrote:
> Since that first line is invalid HTTP, using a WebSocket client on a HTTP
> server will be rejected.
>
> And since no HTTP first line is a valid WebSocket HELLO, a HTTP request to a
> WebSocket server will also be rejected.
>

Unfortunately, there are many more protocols in the world than just
HTTP and web sockets.  When considering cross-protocol
vulnerabilities, we need to consider how web sockets interacts with
all of these protocols.  For example, we must be sure that web sockets
cannot be used to send spam by interacting poorly with the SMTP
protocol.

Adam

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 18 May 2010 00:03:53 +0100

Vladimir Katardjiev wrote:
>    Could the phone seller provide the appropriate root CA? Yes, they
>    could. But what happens when you change operator, but keep your phone?
>    (This might not be a US-relevant concern, but it's certainly
>    EU-relevant). Are you going to provide them with an instruction
>    booklet on how to install a root CA,

I've encountered similar (EU).  Having changed provider, my old phone
cannot access the net because it needs a different data setting, and
every place I tried to set it doesn't work.  Working instructions are
not available, and the manual doesn't cover it.

Random users changing root CAs on phones?  Not a chance.

>    or are you going to tell them to recharge the phone three times a day.

Your 16 hours measurement is with three independent 10 minute
keepalives, isn't it?

With 10 apps (news, email, facebook, twitter, MSN, skype etc.) doing
keepalives, the same battery will presumably last about 5 hours if
not cleverly merged.

For some applications, such as status updaters which must show when
they aren't accurate, 10 minute keepalive is *way* too long.  For
those the keepalive needs to reflect how fresh you need the data to be
before it shows as stale.  It's not enough to keep the connection
alive (unless you have a smart link-layer that you can trust -
something to be investigated for mobile connections).

Merging application-required keepalives with different timeout
requirements is a bit tricky.

We can't solve all those, but we can lead the way by providing a
mechanism they might easily adopt which Just Works, rather than adding
to the problem.

>    Practically speaking, it's not improbable that operators would just
>    require a root cert belonging to some CA more than willing to allow
>    this proxying to happen on every single handset sold. If you really
>    want to, I can go into detail on why this is so (it's a long story),
>    but the fact is that, for a mobile network, sporadic traffic of
>    occasional 1kb packets (such as keepalives) is orders of magnitude
>    worse for performance than more consistent traffic over a longer
>    period of time. So the incentive would definitely be there; the
>    question is why do we want to give it to them?
> 
>    I say we spec an appropriate way for WebSockets to be proxied,
>    including the ability to adjust keepalives, apply compression, etc.

I agree.

Technically, to do this, frame encoding *and* precise semantics for
those features needs to be understood.  It's a basic prerequisite.

It's necessary for the proxy to know exactly what timing and frame
changes are acceptable.  If timings are negotiated between endpoints
(and really they need to be), then the proxy must also participate or
at least understand the negotiation.

However, ways to adjust keepalives, etc. don't have to be agreed at
the first deployment, or in the base protocol.  They can be
standardised extensions.

But it's near-essential that the base protocol provides a mechanism to
negotiate extensions.  (The WebSocket subprotocol field has a problem
with this right now: It can't be changed or added to "below the API"
so to speak.)

>    When the client has a configured WS proxy, use regular unencrypted
>    websockets for [3]ws://. When there's no WS proxy configured, or an
>    HTTP one, use TLS. This gives incentive for WS proxy deployment
>    without breaking the ability to optimise for traffic (or having to
>    decrypt/encrypt in order to perform it), or the installation of root
>    certs. It allows the immediate deployment of websockets without
>    mandating a technological game of cat and mouse. And the user choice

It's a bit like the pros and cons of client-configured HTTP proxies
and "intercepting" proxies.

We might see some places blocking TLS when it resembles WebSocket
traffic (it's distinctively unlike HTTP, usually), to force users to
point at the local WS proxy for network efficiency reasons.  If that
happens, it will annoy users who are using "not supported" clients.

-- Jamie

---

From: Scott Ferguson <[email-masked]>
Date: Mon, 17 May 2010 16:19:09 -0700

> cannot be used to send spam by interacting poorly with the SMTP
Yep.  Checking against well-known protocols would be a good idea in any 
case. One reason the "WebSocket/1.0" is a fixed string in a fixed early 
location is to immediately conflict with non-websocket services.

In the case of SMTP, the draft HELLO isn't a valid SMTP HELO header.  
And even if SMTP is configured to allow initial non-HELO garbage lines, 
postfix at least will detect the "url: ..." lines and drop the 
connection (although it would skip the "url=" that I mistyped above.)

So it's certainly true that the HELLO frame format matters. And even 
after designing the handshake properly, the idea of restricting allowed 
ports from JavaScript to 880 (or whatever websockets is) may be a good idea.

-- Scott

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 18 May 2010 00:30:53 +0100

Roberto Peon wrote:
>    And when the user is using a websockets-enabled site to deal with
>    their medical history, it should automatically go out over the air
>    with no encryption?

WebSockets-secure is a good plan and valid use-case, I agree.

I'm surprised it hasn't been discussed earlier.  It's not been
mentioned in any requirements document or discussion that I've noticed.

>    If we must absolutely support proxying because we must cede control of
>    the channel over to the cell providers, then we should make that a
>    first-class requirement. It sounds dubious to me. I have a hard time
>    trusting that network operators will consistently do the right thing
>    with those proxies for their customers.

Not just cell providers.  Some corporate VPNs, office internet
connections (many users with poor upstream bandwidth due to ADSL -
optimised for HTTP download, not a river of upstream keepalives),
places like cafés (providing service to say 30 anonymous wireless
users) may benefit from keepalive aggregation, at least.

Note that things like keepalive merging and multiplexing
*don't* necessarily require insecure connections.

What they require is enough framing information visible for the proxy
to do its job.

Just an example: TLS on top of WebSocket instead of below
would be like that.

-- Jamie

---

From: Ian Hickson <[email-masked]>
Date: Tue, 18 May 2010 00:00:08 +0000 (UTC)

On Mon, 17 May 2010, Adam Barth wrote:
> attacks for the non-TLS version of the protocol.

We've already done that. If we don't need to support connecting to 
servers that also support HTTP, then we can do a much better job (e.g. 
using your idea of making the handshake look random).

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 18 May 2010 01:11:45 +0100

Roberto Peon wrote:
> 
>    Sure. Note that by removing the choice from the user we end up with
>    cases where you have tethered your computer to your phone, and the
>    proxy is helpfully stripping the bits off of the jpg data for the
>    mapping application that you're attempting to use.
> 
>    Proxies do benefit some users, but, without the user's ability to
>    choose to not use them, they can do serious harm.

I agree.  I've had to route around intercepting HTTP proxies a number
of times due to problems they cause, and any WebSocket-intercepting
proxy will no doubt cause problems too.

One reason to consider that proxy actions like keepalive aggregation
and multiplexing/demultiplexing may be necessary despite the possible
difficulties is that WebSocket's traffic profile will differ from HTTP.

The current draft is oriented around the idea of lots of connections
per browser user - one per WebSocket object.  (We've covered why many
WebSocket objects per browser are anticipated.)

WebSockets are active traffic generators even from unused
(backgrounded) web pages, which users may keep open 24 hours, even
though they only actively browse for a few hours, and then only with a
subset of the open ones.  This is because WebSockets used to receive
server-initiated updates must use keepalives all the time - and those
are about equally upstream and downstream.

In contrast, HTTP is more cautious about the number of connections,
most inactive web pages don't use any (or very little) HTTP traffic,
and the traffic is highly asymmetric.

We don't yet know how the new traffic profile will fare on various
networks, except some networks we can make better predictions about.

Actively designing to block proxy acceleration and also make
it difficult to retrofit later, seems unjustifiable because of that,
even if it turns out to be unneeded in most places.

But I agree there are issues we've all seen from "helpful" HTTP
proxies getting in the way.

-- Jamie

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 17 May 2010 18:29:34 -0700

> proxy will no doubt cause problems too.

The best way to "route around" intermediaries is to make it easy
for them to perform their intended functionality without harm.
We should assume that they exist for some reason, even if we happen
to disagree with it, and that more reasons (not less) will appear
with a new protocol.  We should not engage in an arms race over
protocol routing, since that will be interpreted by the same folks
as inserting a backdoor to bypass organizational security and
simply cause them to insert more content-parsing and port-blocking
behavior to compensate.

As I mentioned back in December, if a client can make a TLS connection
(with SNI for same-origin checks) to a server, then why does it need
a Websocket protocol?  It has TCP already.  If you can make the TLS
exchange suitably unique, such that it can be distinguished from other
TLS traffic, then you can have both an intermediary-proof tunnel and
an easy mechanism by which firewalls can block such tunnels without
blocking normal https use.  That will satisfy the intermediaries.

OTOH, if use cases for this hybi protocol are supposed to include
many-to-one long-lived connections over the Internet, then any design
dependent on per-client TCP is doomed to fail.  Even a multiplex
connection per client is not going to handle an Internet-scale
notification system (it is an old research problem).  We should be
focusing on message-based protocols, with optional message-based
encryption, that are independent of the transport mechanism.

It would be far more interesting to me (and beneficial to the net)
if the vendors working on this problem would consider using their
considerable influence on network consumers to deploy a real
solution using a connectionless protocol as transport.  I don't
care which one, and I don't care if it takes years to deploy on
the wider net, just so long as it isn't actively harmful.
Game developers do it all the time, and proxy vendors have
adjusted their firewall options accordingly.

> proxies getting in the way.

I wish we could get a list of proxy product names that
cause problems, since it has been a very long time since I have
seen an HTTP error report concerning them. (since 1996, in fact).
We used to be able to get people to fix them in short order, once
data loss is revealed to their customers.

....Roy

---

From: Erik Möller <[email-masked]>
Date: Tue, 18 May 2010 10:27:01 +0200

> adjusted their firewall options accordingly.

I'd love to see a discussion on a connectionless protocol started. It's  
really the last piece missing before game developers would seriously  
consider HTML5 a viable platform. I know a lot of them are already  
starting to look into it, but being stuck with something built on top TCP  
will make many hesitant to try their hand at it.

-- 
Erik Möller
Core Developer
Opera Software

---

From: John Tamplin <[email-masked]>
Date: Tue, 18 May 2010 11:05:31 -0400

On Tue, May 18, 2010 at 4:27 AM, Erik Möller <[email-masked]> wrote:

> consider HTML5 a viable platform. I know a lot of them are already starting
> to look into it, but being stuck with something built on top TCP will make
> many hesitant to try their hand at it.
>

I can see the value if you allow peer-to-peer datagram communication, but I
don't know how we can allow that safely.  If you have t relay things through
the server anyway, what is the advantage over maintaining one connection to
the server?  GWT Quake, for example, works this way using WebSockets.

---

From: Erik Möller <[email-masked]>
Date: Tue, 18 May 2010 17:41:15 +0200

On Tue, 18 May 2010 17:05:31 +0200, John Tamplin <[email-masked]> wrote:

>> starting

How to safely allow peer-to-peer communication in the browser I really  
can't tell you.

Why UDP even in a client-server model? Even though GWT Quake runs fine on  
a LAN I think you'd quickly find the original was using UDP for a reason  
should you deploy it on the net with some packet loss and resends. When  
you're sending position updates to a client in a real-time game you don't  
want to retry and make sure the position for time t gets delivered ok  
before you send off the position for t+1... if it didn't get there in time  
it's of no use to the client anyway, you're just concerned with the most  
recent position (and if a position is lost it'll interpolate with  
dead-reckoning).

Sure, there are games out there that use TCP as transport, but if it's  
feasible to use comes down to what sort of latency is acceptable in that  
particular game. Though I don't have any source to back it up with I can  
almost guarantee the majority of the shooters use UDP as transport.

-- 
Erik Möller
Core Developer
Opera Software

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 19 May 2010 03:44:55 +0100

> How to safely allow peer-to-peer communication in the browser I really  

I'm fairly sure permitting UDP sends and receives of arbitrary on
UDP port 80 would introduce no security issues to the existing
services.  Though it might start an ecosystem of new services with
their own security issues. :-)

Doing it properly probably involves negotiating a confirmed safe
channel first.  It sounds like SIP territory. 

> almost guarantee the majority of the shooters use UDP as transport.

The problem with TCP in such applications isn't that an occasional
update might arrive a bit too late to be useful.

The problem is one delayed or lost packet delays *everything*
afterwards until it recovers.  Moreover, if there a few unlucky lost
packets, recovery can be quite slow (e.g. 20 seconds).  Interpolation
of occasional lost updates can't be done - it becomes prediction for
umpteen seconds.  Commit-decisions (like kills in a shooter) must
wait, positional accuracy is lost, and gamers get annoyed.

It's usually fine on a fast LAN, if it's not overloaded, because TCP
on a LAN detects the low latency and tends to retransmit things so
quickly that you hardly notice.

But since we tend to code for the internet these days, we tend to use
UDP for that type of traffic whatever the network.  (And in the old
days, we used modems which are somewhat high-latency, so similar
protocols were used even though it didn't go over the internet.)

-- Jamie