Subject: Re: [hybi] Providing two-way communication back to a server from a Web page

From: Ian Hickson <[email-masked]>
Date: Mon, 30 Mar 2009 22:02:06 +0000 (UTC)

The e-mails below were sent to [email-masked] a while back; I'm 
responding now that we have a more appropriate forum.

On Fri, 9 Jan 2009, Robert Brewer wrote:
> 
> I've already been getting some CherryPy users asking if the next version 
> of CP will support WebSockets. My initial thoughts:
> 
> [...]
> 
> Python web frameworks at least have pretty much standardized now on the 
> Web Server Gateway Interface [1], which is strictly request/response. 
> Rewriting WSGI, and/or extending the servers, to use WebSockets would be 
> nontrivial. I imagine most other existing web frameworks in other 
> languages are similarly bound to that model.

In general, I wouldn't expect bi-directional communication to be 
shoehorned into existing frameworks -- after all, the whole point of 
bidirectional communication is that it's not a request-response model. 
This is just like I wouldn't recommend using the framework intended for an 
IRC bot to host a Web server. :-)

> [...] I've seen too many custom TCP protocols in my day, and am quite 
> concerned this would be a step backwards into some pretty ugly RPC. Not 
> just hard to read, but lacking any visibility or most of the other 
> network-architectural benefits we've come to rely on in HTTP.

Could you elaborate on this?

> [...] if I were implementing bi-directional communication in an HTTP 
> environment that was machine-to-machine (as opposed to 
> user/agent-to-machine), I would naturally consider making both ends 
> expose an HTTP server interface: the "origin server" would expose a 
> POST'able subscription resource, and the "client" would POST a URL from 
> its own space.

That sounds like a terrible idea. :-) HTTP is a request-response 
mechanism, which is quite different from bi-directional, full-duplex, 
asynchronous communication. I don't think shoehorning the latter into the 
former would lead to a simple protocol or an easy model to write for. In 
particular, one of the requirements for this, IMHO, is that it be possible 
to implement the server-side in a few dozen lines of code, and I don't see 
any way to implement a compliant HTTP server and client in such a small 
code footprint.

> The "server" could then POST updates to that URL; that is:
> 
>     client.user.com
>     ---------------
>     POST server.biz.com/resource/poll_me HTTP/1.1
>     ...
>     myurl=client.user.com/listeners/A
> 
> ...then, at intervals when there is new data to be pushed:
> 
>     server.biz.com
>     ---------------
>     POST client.user.com/listeners/A
>     ...
>     eventtype=update&data=blahblahblah

That would be a lot of overhead for "blahblahblah"!

> Synthesizing those, I'd feel better about WebSockets if the messages 
> passed in both directions were HTTP, not opaque binary formats.

I don't really understand why HTTP would be relevant here.

> I also wonder if this couldn't be better achieved by creating a spec 
> (HTTP 2?) that allowed pipelined HTTP request and response messages in 
> both directions arbitrarily.

That definitely sounds like a lot more complexity than necessary.

On Sat, 10 Jan 2009, Jamie Lokier wrote:
> 
> Can you overlay something like Bayeux 
> (http://svn.cometd.com/trunk/bayeux/bayeux.html) over WSGI?

Bayeux does seem like a better solution for servers that are based on a 
request-response model.

On Mon, 12 Jan 2009, Mark Baker wrote:
> >
> >  - The ability for Web pages to send arbitrary text to the server from 
> >    a script in a Web page, in the form of ordered discrete blocks.
> 
> This one comes for free with XHR/HTTP, no?  Unless you meant something 
> more than what's written there.

I don't think XHR guarantees ordering, but it is certainly possible to 
overlay an ordering mechanism on top of XHR to do this. Using XHR you 
would also have to implement a mechanism to resend data that used failed 
connections.

However, it seems suboptimal to have to design ordering and resending over 
a protocol that itself uses TCP, which has built-in features for ordering 
and resending packets.

> > I would be interested in any suggestions people may have along these 
> > lines. Are there existing protocols that satisfy these requirements?
> 
> Depends what you mean by "existing".  Waka[1] sounds like it would 
> address all those requirements.  It has the advantage of being a full 
> HTTP replacement protocol too, meaning that it's still the Web and so 
> still suitable for use over ports 80 and 443 (whereas Web Sockets is a 
> tunnel for potentially arbitrary applications).  It also means that 
> developers will be starting with something quite feature rich and so 
> will be less prone to reinventing those features poorly (possibly 
> inefficiently, or worse, with security problems).
> 
> On the downside, Roy hasn't published the protocol yet.
> 
>  [1] http://en.wikipedia.org/wiki/Waka_(protocol)

I don't think that ideas that exist only in Roy's head really meet the 
definition of "existing" that I had in mind. :-)

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: "Edward Z. Yang" <[email-masked]>
Date: Mon, 30 Mar 2009 18:04:45 -0400 (EDT)

> any way to implement a compliant HTTP server and client in such a small

It seems to me that you are suggesting sandboxed socket support on the browser
side.

Cheers,
Edward

---

From: Ian Hickson <[email-masked]>
Date: Mon, 30 Mar 2009 22:28:08 +0000 (UTC)

On Mon, 30 Mar 2009, Edward Z. Yang wrote:
> > asynchronous communication. I don't think shoehorning the latter into 
> > the former would lead to a simple protocol or an easy model to write 
> > for. In particular, one of the requirements for this, IMHO, is that it 
> > be possible to implement the server-side in a few dozen lines of code, 
> > and I don't see any way to implement a compliant HTTP server and 
> > client in such a small code footprint.
> 
> It seems to me that you are suggesting sandboxed socket support on the 
> browser side.

Yes, that's basically what the WebSockets proposal is:

   http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol