Subject: [hybi] WebSockets

From: Ian Hickson <[email-masked]>
Date: Mon, 30 Mar 2009 21:34:19 +0000 (UTC)

The goal of the WebSocket work is to provide two-way communication from a 
Web page in a clean way. The requirements that I collected for this effort 
over the past few years are:

 - The ability for Web pages to send arbitrary text to the server from a 
   script in a Web page, in the form of ordered discrete blocks.

 - The ability for Web servers to send arbitrary text to a Web page that 
   has established a connection, in the form of ordered discrete blocks.

 - Text messages should support all of Unicode.

 - The protocol should support running over ports 80 and 443, ideally 
   with the ability to share the port on the server with an HTTP server.

 - It must be possible to implement a fully-conforming server-side 
   component for this in a few dozen lines of scripting code, in the case 
   where sharing the connection with an HTTP server isn't required.

 - The must be extensible so that it can be extended to support binary 
   data and structured data in both directions in the future, without 
   breaking deployments from the first generation.

 - It must not be possible for the mechanism to be used to successfully 
   connect to existing servers and send them arbitrary commands. In 
   particular, SMTP, HTTP, HTTPS, IMAP, POP, and similar protocols must be 
   safe from this.

 - It must be possible to connect to arbitrary remote hosts, but 
   connections to these hosts must only be allowed if the remote host 
   explicitly opts in to allowing that Web page to connect (ideally using 
   the 'origin' convention).

I would be interested in any suggestions people may have along these 
lines. Are there existing protocols that satisfy these requirements? Are 
there requirements I have missed? Are there requirements that people think 
should not be met?

There is a strawman proposal here:

   http://www.ietf.org/internet-drafts/draft-hixie-thewebsocketprotocol
   http://dev.w3.org/html5/websockets/

Comments welcome.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Peter Keane <[email-masked]>
Date: Mon, 30 Mar 2009 16:58:21 -0500

I am not really sure if this fits in with the goals here or not, but
the functionality I am interested in is a method for doing batch
processes, e.g., uploading a directory full of images to a web server.
 I'd like to be able to have the client register a URL with the
server, at which point the server can perform a get at that URL, which
returns a list of items (essentially a directory listing), and GETs
each one.  I have implemented ths with success using the Firefox Plain
Old Webserver (POW)  plugin. I simply configure POW to have a local
directory as Doc root.   It's then easy to do large uploads (really
reverse GET) for many files very quickly (of course I have an HTTP
client like CURL on the actual server to do the GETs).  I could
imagine a standard mechanism to do this sort of thing very handy.

--peter keane

On Mon, Mar 30, 2009 at 4:34 PM, Ian Hickson <[email-masked]> wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: "Edward Z. Yang" <[email-masked]>
Date: Mon, 30 Mar 2009 19:06:30 -0400 (EDT)

Hello,

>   particular, SMTP, HTTP, HTTPS, IMAP, POP, and similar protocols must be

I believe that modern browsers specifically block requests to ports that
are not normally used in web browsing on an ad hoc basis.  Given
WebSocket's handshake, I think this problem is equivalent to HTTP.

> there requirements I have missed?

Plugins such as Flash and Java already allow sockets to programmers on a
limited basis.  Looking forwards, these languages will need client
libraries that implement the WebSocket protocol.  JavaScript will also
need an API designed for it.

Cheers,
Edward

---

From: Ian Hickson <[email-masked]>
Date: Mon, 30 Mar 2009 23:15:57 +0000 (UTC)

> WebSocket's handshake, I think this problem is equivalent to HTTP.

That's the intent, yes.

> > lines. Are there existing protocols that satisfy these requirements? 
> > Are there requirements I have missed?
> 

There is a JavaScript API designed for this protocol here:

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 30 Mar 2009 18:43:55 -0700

On Mar 30, 2009, at 2:34 PM, Ian Hickson wrote:

> Web page in a clean way.

It would be nice to have a list of use cases for why anyone would want
a Web page (as opposed to an installed client or client extension) to
provide two-way communication, and also why that two-way communication
must be tunneled over the Web ports in a way that deliberately intends
to fool security-enforcing intermediaries into thinking it is HTTP.

I know that Netscape and Macromedia/Adobe and MS have provided various
such features (and their associated security holes) over the years, but
I've never found the need to copy them within a Web page.  It would be
nice to put the rationale out there before the protocol.

....Roy

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 30 Mar 2009 19:58:55 -0600

> to fool security-enforcing intermediaries into thinking it is HTTP.

I can't speak for the WebSocket work, but I know that in XMPP we use
BOSH <http://xmpp.org/extensions/xep-0124.html> as a way of emulating
long-lived TCP connections over HTTP. We do this to build browser-based
clients (a "no-download" solution) and to support devices that cannot
maintain long-lived TCP connections (either because of device
restrictions or network conditions). BOSH traffic is not necessarily
tunneled over ports 80 and 443, and the purpose is not to work around
firewalls but to offer an alternative binding for XMPP traffic when the
TCP binding defined in RFC 3920 is simply not feasible.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Ian Hickson <[email-masked]>
Date: Tue, 31 Mar 2009 02:12:13 +0000 (UTC)

On Mon, 30 Mar 2009, Roy T. Fielding wrote:
> 
> It would be nice to have a list of use cases for why anyone would want a 
> Web page (as opposed to an installed client or client extension) to 
> provide two-way communication,

Some sample use cases:

 * GMail's built-in IM client
 * Collaborative editing as seen in tools like Google Apps
 * Multiplayer games
 * Pages that monitor services, e.g. a wifi router's status page, showing 
   up-to-date information about signal:noise ratios, etc.

Personally the reason I want a WebSocket-like mechanism is so that I can 
make my Web-based model railway control panel quicker. Right now every 
time I click anything it has to do a round-trip TCP connection to the Web 
server to tell it to tell the computer running the trains to perform the 
action, whereas what I really want is for the Web page to just have a 
direct connection to that computer. This would also allow the train 
controlling computer to give immediate feedback instead of having to 
tunnel messages back through a long-lived CGI script returning drip-fed 
<script> blocks to an <iframe> over a long-lived HTTP connection.

> and also why that two-way communication must be tunneled over the Web 
> ports in a way that deliberately intends to fool security-enforcing 
> intermediaries into thinking it is HTTP.

For my use case (the trains) I'm fine with not doing any tunneling, 
however it turns out that a lot of sites block all but port 80 and port 
443 and yet people still want to use their IM clients. Right now Web 
applications like GMail are tunneling through ports 80 and 443 by abusing 
HTTP; I think it would be much better to have them Upgrade to another 
protocol and leave HTTP for REST-like purposes.

Note that the way WebSocket is designed, there's no deliberate fooling; 
it's a straight up HTTP Upgrade attempt.

HTH,

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 30 Mar 2009 19:57:30 -0700

> <script> blocks to an <iframe> over a long-lived HTTP connection.

Thanks.  Perhaps we should start a wiki somewhere to list the use cases
and then a protocol comparison table to see what use cases are satisfied
by each protocol.

> protocol and leave HTTP for REST-like purposes.

A lot of offices put locks on the door and only give keys to their
employees and trusted contractors.  That is inconvenient for a reason.

> it's a straight up HTTP Upgrade attempt.

Not quite.  You can't place constraints on specific ordering of bytes
within WebSocket and then claim it is HTTP, because HTTP allows (and
sometimes requires) intermediaries to change or reorder those same
bytes in specific situations.  I don't buy the argument regarding
ease of implementation because it simply isn't robust to parse a
line-delimited protocol as if it were a fixed structure, again because
of the presence of those intermediaries over which you have no control.
Upgrade was specifically designed for this purpose and can just as
easily be implemented with a standard HTTP/1.1 OPTIONS or CONNECT
request.

The good thing is that, by initiating the connection with an HTTP
request to the same-origin server by hostname, WebSocket doesn't
suffer quite as much from the shared-IP security holes that are
present in web-based software that uses raw sockets.  However, it
could still be used to trigger arbitrary requests by intercept
proxies that don't understand that the connection has been upgraded.
A more secure design would require the use of CONNECT and then
forcibly encode the stream such that it can't be misinterpreted.

Alternatively, just use a different port and let the organizations
decide whether they want to allow such connections through their
own firewalls.

....Roy

---

From: Ian Hickson <[email-masked]>
Date: Tue, 31 Mar 2009 03:16:19 +0000 (UTC)

> and then a protocol comparison table to see what use cases are satisfied 

As far as I know, all the use cases are addressed by all the proposed 
protocols. However, please be my guest if you would like to start such a 
page; I'm happy for people to use the WHATWG wiki for that purpose if the 
question is which wiki to use. (http://wiki.whatwg.org/)

> > however it turns out that a lot of sites block all but port 80 and 
> > port 443 and yet people still want to use their IM clients. Right now 
> > Web applications like GMail are tunneling through ports 80 and 443 by 
> > abusing HTTP; I think it would be much better to have them Upgrade to 
> > another protocol and leave HTTP for REST-like purposes.
> 

I'm not making judgements about whether ignoring these IT policies is a 
good idea, or whether the IT policies themselves are good ideas; I'm just 
conveying what requirements people have put forward. There would be no 
point us designing a protocol that doesn't handle cases that we have been 
told are critical, after all! :-)

> > Note that the way WebSocket is designed, there's no deliberate 
> > fooling; it's a straight up HTTP Upgrade attempt.
> 
> within WebSocket and then claim it is HTTP

You're right; my bad. It's not a straight-up HTTP Upgrade attempt. It's a 
WebSocket handshake that happens to act like an HTTP Upgrade attempt in 
practice. There is indeed deliberate fooling at the server-side; my point 
was that there is no deliberate fooling at the level of security policies.

(The handshake is designed that way so that non-HTTP victim servers can't 
be tricked into sending back what looks like a valid response: we have to 
have as strict a handshake as possible so that there is as little room for 
attacks to maneuver in as possible.)

> However, it could still be used to trigger arbitrary requests by 
> intercept proxies that don't understand that the connection has been 
> upgraded.

Could you elaborate on this? An example would be very useful.

> A more secure design would require the use of CONNECT and then forcibly 
> encode the stream such that it can't be misinterpreted.

This is what is done for the SSL variant of WebSocket.

> decide whether they want to allow such connections through their own 
> firewalls.

As designed, WebSocket doesn't default to the HTTP ports. The proposal has 
its own port. To use the HTTP ports you have to explicitly override the 
default, just like HTTP can be tunnelled over ports other than port 80.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 31 Mar 2009 22:27:11 -0700

> question is which wiki to use. (http://wiki.whatwg.org/)

I created an initial page at

   http://trac.tools.ietf.org/bof/trac/wiki/HyBi

which I think is an appropriate place for it.

> told are critical, after all! :-)

On the contrary, ethics are a part of engineering that the IETF follows
better than most SDOs.

> attacks to maneuver in as possible.)

I think the same goal is accomplished by any valid HTTP/1.1 message.

> Could you elaborate on this? An example would be very useful.

I mean that, after the initial constrained Upgrade request is
accepted by the server, there is nothing to prevent the WebSocket
sender from continuing with a sequence of HTTP requests on the
same connection that might be inspected and redirected by an
interception proxy (thinking that they are just pipelined
normal HTTP requests).  For more info, see

http://www.thesecuritypractice.com/the_security_practice/ 
TransparentProxyAbuse.pdf

I don't know if they are worth designing around, but I do know that
they are avoided by using the different TCP ports, encrypting the
stream, or using a more constrained protocol after the upgrade
such that the client API generates the message framing in a way
that can't be misconstrued.

Routing of requests in HTTP over TCP depends on both the IP address/
TCP port and the value in the Host header field.  In particular, the
same-origin requirement of web page script/plugin processing is only
effective with HTTP if both the TCP/IP parts and the Host correspond
to the same-origin host:port.  If the TCP/IP doesn't match, then the
message will be directed to some other server.  If the Host (or the
absolute URI for proxy requests) does not match then any host on
a shared IP address (think large ISPs like dreamhost or clouds
like AWS) can bypass the same-origin policy for any other hostname
on the same ISP/cloud.

WebSockets is much better in that regard than some implementations
that provide raw socket-like access to web page components without
sending an initial HTTP request.

>> forcibly

Yep.

> default, just like HTTP can be tunnelled over ports other than port  

In that case, you really are looking for a different protocol rather
than just reusing an HTTP connection.  I guess I should get back to
working on the waka spec.

....Roy

---

From: Ian Hickson <[email-masked]>
Date: Wed, 1 Apr 2009 06:47:15 +0000 (UTC)

On Tue, 31 Mar 2009, Roy T. Fielding wrote:
> > > Not quite.  You can't place constraints on specific ordering of 
> > > bytes within WebSocket and then claim it is HTTP
> > 
> > You're right; my bad. It's not a straight-up HTTP Upgrade attempt. 
> > It's a WebSocket handshake that happens to act like an HTTP Upgrade 
> > attempt in practice. There is indeed deliberate fooling at the 
> > server-side; my point was that there is no deliberate fooling at the 
> > level of security policies.
> > 
> > (The handshake is designed that way so that non-HTTP victim servers 
> > can't be tricked into sending back what looks like a valid response: 
> > we have to have as strict a handshake as possible so that there is as 
> > little room for attacks to maneuver in as possible.)
>

It seems that the implementation work of checking whether a response is a 
valid HTTP/1.1 message is significantly more complicated than the work of 
determining whether the Web Socket handshake is precisely correct or not.

It also seems far easier to reason about the security aspects of the 
handshake as seen by the servers of other protocols when the handshake is 
a very well-defined series of bytes than when the handshake is an 
open-ended set of possible requests. Similarly, it's a lot less likely 
that an attacker could trick an arbitrary protocol's server into 
outputting the exact sequence of bytes that the Web Socket handshake 
requires than it would be if the handshake consisted of any sequence of 
bytes that matched the HTTP grammar.

> I mean that, after the initial constrained Upgrade request is accepted 
> by the server, there is nothing to prevent the WebSocket sender from 
> continuing with a sequence of HTTP requests on the same connection that 
> might be inspected and redirected by an interception proxy (thinking 
> that they are just pipelined normal HTTP requests).

Surely that would be a bug in the proxy.

> > As designed, WebSocket doesn't default to the HTTP ports. The proposal 
> > has its own port. To use the HTTP ports you have to explicitly 
> > override the default, just like HTTP can be tunnelled over ports other 
> > than port 80.
> 
> than just reusing an HTTP connection.

Yes, the Web Socket protocol is its own protocol. Is that not clear in the 
Web Socket Protocol Internet Draft?

The ability to bootstrap using an HTTP connection, Upgrade:ing to 
WebSockets, is optional, and is only intended for advanced users that need 
to reuse the same port for both purposes.