Subject: [hybi] More feedback on WebSockets

From: Mark Nottingham <[email-masked]>
Date: Tue, 27 Oct 2009 12:55:38 +1100

Overall, I think this draft looks very good, excepting the issues below:

1) Abstract - AIUI one of the primary motivations for WS (especially  
over just exposing a TCP API in browsers) is the use of the "Web  
security model." This should be mentioned in the Abstract (and  
background?).

2) The handshake protocol looks like HTTP, but is not (e.g., it's case- 
sensitive and whitespace-sensitive). However, you define the default  
ports for WS and WSS to be port 80 and 443, respectively.

These ports are reserved for HTTP, at least initially; while you can  
use HTTP to upgrade to a different protocol, defining a HTTP-like-but- 
not-really-HTTP handshake protocol to run over them initially  
conflicts with the designated use of the port.

Possible remedies;
   a) define the handshake in terms of HTTP, not byte sequences, or
   b) define a different (i.e., non-HTTP) protocol identifier to use  
on the handshake request and response.

3) In section 1.6 (Establishing a Connection), you say

>  On the face of it, the simplest method would seem to be to use port
>    80 to get a direct connection to a Web Socket server.  Port 80
>    traffic, however, will often be intercepted by HTTP proxies, which
>    can lead to the connection failing to be established.

Given this, is it a good idea for the default port for the WS scheme  
to be port 80? Why not use a separate port and fall back to 443 with  
null encryption, for example?

4) In section 1.2, you define the payload of the headers as ASCII, but  
then allow a wide range of Unicode characters. Which is it?

5) Are there any constraints placed upon the field-value of the  
WebSocket-Protocol header? Is there any structure?

If I want to define an intermediary function for WebSockets that  
allows multiplexing, and I wish to unambiguously identify that it's in  
use, is this an appropriate use for the WebSocket-Protocol header?

If so, how do I avoid conflicting with the names of other  
subprotocols, and how do I indicate that multiple subprotocols are in  
use on one connection?

If not, what's the appropriate mechanism?

Cheers,

--
Mark Nottingham     http://www.mnot.net/

---

From: Ian Hickson <[email-masked]>
Date: Tue, 27 Oct 2009 09:37:51 +0000 (UTC)

> security model." This should be mentioned in the Abstract (and 

Added some text to that effect.

> These ports are reserved for HTTP, at least initially; while you can use 
> HTTP to upgrade to a different protocol, defining a 
> HTTP-like-but-not-really-HTTP handshake protocol to run over them 
> initially conflicts with the designated use of the port.

IANA told me to use ports 80 and 443 for WebSocket. See IANA ticket
#257455. (I had originally requested new ports, ideally 81 and 815.)

I disagree that the current handshake isn't like HTTP enough, though. The 
request is fully HTTP-compliant. What value would there be in relaxing the 
rules on what WebSocket clients should send in the request? I don't 
understand the real-world case you are concerned about.

> Given this, is it a good idea for the default port for the WS scheme to 
> be port 80? Why not use a separate port and fall back to 443 with null 
> encryption, for example?

That's what the spec used to require, but IANA requested that I use ports 
80 and 443 directly.

> then allow a wide range of Unicode characters. Which is it?

Section 1.2 doesn't say that the payload of the headers is ASCII, it just 
says that they are ASCII case-insensitive (i.e. that a-z matches A-Z, but 
that U+00C1 doesn't match U+00E1). I've tried to clarify this a bit.

(Also, section 1.2 is non-normative; it doesn't define anything.)

> WebSocket-Protocol header? Is there any structure?

Not currently.

> If I want to define an intermediary function for WebSockets that allows 
> multiplexing, and I wish to unambiguously identify that it's in use, is 
> this an appropriate use for the WebSocket-Protocol header?

Yes.

> If so, how do I avoid conflicting with the names of other subprotocols, 

Use a unique name, e.g. by using your domain name as the protocol name, 
or googling for your protocol name and seeing if anyone else has used it.

> and how do I indicate that multiple subprotocols are in use on one 
> connection?

Only one protocol can be in use per connection. If your protocol is just 
another layer below the final application-level protocol, then your 
protocol would have its own equivalent mechanism.

Cheers,
-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 27 Oct 2009 21:13:37 +1100

> understand the real-world case you are concerned about.

What about a load balancer in front of the server that inserts
a cookie or X-Forwarded-For header into all HTTP requests that
it forwards.

This will probably be harmless with regards to the subsequent
WS connection, but it will break the handshake so there will
not be a subsequent WS connection.

regards

---

From: Ian Hickson <[email-masked]>
Date: Tue, 27 Oct 2009 18:36:44 +0000 (UTC)

On Tue, 27 Oct 2009, Greg Wilkins wrote:
> Ian Hickson wrote:
> > I disagree that the current handshake isn't like HTTP enough, though. 
> > The request is fully HTTP-compliant. What value would there be in 
> > relaxing the rules on what WebSocket clients should send in the 
> > request? I don't understand the real-world case you are concerned 
> > about.
> 
> What about a load balancer in front of the server that inserts a cookie 
> or X-Forwarded-For header into all HTTP requests that it forwards.
> 
> This will probably be harmless with regards to the subsequent WS 
> connection, but it will break the handshake so there will not be a 
> subsequent WS connection.

How do such load balancers handle pipelining?

If they support them, then that means they are almost certainly 
incompatible with WebSocket, as far as I can tell, and we would _want_ the 
connection to fail.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 28 Oct 2009 07:50:26 +1100

> incompatible with WebSocket, as far as I can tell, and we would _want_ the 

They all are different.  But many just look at the first request on
a connection and then just treat the rest of the connection with
packet forwarding.   So that type should work.

Others look at every request and will probably not work.

regards

---

From: Ian Hickson <[email-masked]>
Date: Tue, 27 Oct 2009 20:58:54 +0000 (UTC)

> Others look at every request and will probably not work.

For the packet-forwarding ones, are we talking about inserting a header on 
incoming connections (client to server) or outgoing responses (server back 
to client)?

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 28 Oct 2009 08:02:34 +1100

> incoming connections (client to server) or outgoing responses (server back 

Again - there are a huge number of variations.

But inserting  X-forwarded-for headers on requests and Via headers
on responses is common.
So is setting cookies on responses so that subsequent
connections can be balanced the same.
Also SSL offload will want to set certificate details in the
request header.

regards

---

From: Ian Hickson <[email-masked]>
Date: Tue, 27 Oct 2009 21:06:04 +0000 (UTC)

On Wed, 28 Oct 2009, Greg Wilkins wrote:
> > 
> > For the packet-forwarding ones, are we talking about inserting a 
> > header on incoming connections (client to server) or outgoing 
> > responses (server back to client)?
> 
> But inserting X-forwarded-for headers on requests and Via headers on 
> responses is common. So is setting cookies on responses so that 
> subsequent connections can be balanced the same. Also SSL offload will 
> want to set certificate details in the request header.

It sounds like there are some intermediaries that are harmless and would 
work fine, and others that are harmful and which WebSocket would correctly 
detect and prevent connections through. If someone wants to deploy a 
WebSocket server behind the latter, they'll quickly discover the problem, 
so it doesn't seem like it'd be a barrier to adoption. It's client-side 
intermediaries that are the main concern (and for which TLS-based 
WebSocket is the most obvious solution in most cases).

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 28 Oct 2009 08:12:01 +1100

> WebSocket is the most obvious solution in most cases).

But they need to be able to set cookies and headers in order to work.

Setting these on the upgrade request and response should be perfectly
possible and HTTP legal.  It would allow existing intermediaries to
communicate important information to client (which node they are stuck
to) and to server (ssl and original client details).

These are just a few of many examples of why allowing an intermediary
to add meta data is an important part of a web protocol.

WS does not allow this because it is trying to do an end-run around
intermediaries rather then trying to sensibly work with them.

regards

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 28 Oct 2009 08:23:15 +1100

> I disagree that the current handshake isn't like HTTP enough, though.

I didn't say it isn't like it (in fact, I specifically said it is); I  
said that it isn't HTTP.

Just as Starbucks is like coffee, but not; or a vehicle that only  
allows you to turn the steering wheel left is like a car, but not.  
There are many more examples I could produce, but decency suggests I  
stop.

You've effectively profiled the syntactic conventions of HTTP.  
Implementations that speak HTTP -- whether profiled or not -- should  
be able to do so confidently on the port allocated to it.

> The

My concern is with the precedent it will set.

As far as real-world use cases, I think that your handshake has a  
significant chance of producing false positives with deployed systems;  
many so-called "transparent" (interception) proxies often take great  
pains to stay hidden, so that while the client and server will think  
they have an operative WebSocket connection, as soon as they try to do  
any bi-directional (i.e., not request/response) communication, it will  
fail in a non-obvious, slow way (e.g., hanging) that's hard to recover  
from.

You'd be better off using a handshake that looks as little like HTTP  
as possible, so as to fail more reliably and quickly. Or, preferably,  
another port.

> That's what the spec used to require, but IANA requested that I use  

What was the reasoning behind this request? The ticket number you  
refer to isn't useful to those not involved.

> protocol would have its own equivalent mechanism.

Hmm. This seems like punting the problem to yet another layer.  
However, I do agree that offering a complete solution in WS is  
difficult at best.

---

From: Ian Hickson <[email-masked]>
Date: Tue, 27 Oct 2009 21:23:59 +0000 (UTC)

On Wed, 28 Oct 2009, Greg Wilkins wrote:
> >> subsequent connections can be balanced the same. Also SSL offload 
> >> will want to set certificate details in the request header.
> > 
> > It sounds like there are some intermediaries that are harmless and 
> > would work fine, and others that are harmful and which WebSocket would 
> > correctly detect and prevent connections through. If someone wants to 
> > deploy a WebSocket server behind the latter, they'll quickly discover 
> > the problem, so it doesn't seem like it'd be a barrier to adoption. 
> > It's client-side intermediaries that are the main concern (and for 
> > which TLS-based WebSocket is the most obvious solution in most cases).
> 
> These are just a few of many examples of why allowing an intermediary to 
> add meta data is an important part of a web protocol.
> 

WebSocket doesn't preclude WebSocket-aware intermediaries inserting data 
into the stream. It will have no effect, though; a WebSocket client isn't 
going to change any cookies or change behaviour based on headers set on 
the handshake.

---

From: Ian Hickson <[email-masked]>
Date: Fri, 30 Oct 2009 03:45:43 +0000 (UTC)

On Wed, 28 Oct 2009, Mark Nottingham wrote:
> Implementations that speak HTTP -- whether profiled or not -- should be 
> able to do so confidently on the port allocated to it.

Could you give a concrete example of where this wouldn't work? I don't 
understand the problem here.

> My concern is with the precedent it will set.

Could you elaborate on this?

> fail in a non-obvious, slow way (e.g., hanging) that's hard to recover 

Yes, this is a known risk. The solution is to use TLS. It's not clear that 
there is any way around this without sacrificing some of the other 
requirements (like being able to share a port with an HTTP server).

If anyone can provide any information on any specific MITM proxies that 
have this problem, I would be very grateful. I would love to be able to 
test this in real-world situations against such proxies.

> You'd be better off using a handshake that looks as little like HTTP as 
> possible, so as to fail more reliably and quickly. Or, preferably, 
> another port.

The relevant goal here is to be able to share a port with an HTTP server. 
Not doing an HTTP Upgrade and not using the same port both rather fail to 
achieve this goal.

> What was the reasoning behind this request?

I didn't really understand their reasoning, but they insisted that it was 
the right thing even when I asked for confirmation.

> The ticket number you refer to isn't useful to those not involved.

I haven't checked, but I assume that the number would let you find the 
issue in the IANA's archives... the number was in the subject line of the 
e-mails, if that helps.

> > Only one protocol can be in use per connection. If your protocol is 
> > just another layer below the final application-level protocol, then 
> > your protocol would have its own equivalent mechanism.
> 

WebSocket's entire design is to punt everything to the next layer. It's 
not trying to add any features to the network; it's just trying to provide 
the closest thing to raw TCP as possible to JavaScript, within the 
constraints of the Web (browser) platform: the origin security model, the 
lack of binary support, and the event-based (rather than stream-based) 
programming model.