Subject: [hybi] ws and bwtp thoughts

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 21 Oct 2009 10:21:08 +0800

I've been following the discussions on WS and BWTP (awful name, BTW) with some interest.

The differences are many, but aside from a few they probably make little difference from a technical standpoint.

My current thoughts, from the easy ones to the hard ones.

==Upgrade from HTTP

It seems to me that this is something that could be addressed independent of either submission.  I get the impression that the differences between camps aren't so fervent.

Maybe if this was made a different document, discussing the implications on (and of) intermediaries, discussing the security model that it enables, setting guidelines for its use, then any other submission need only register a protocol tag.

==Readability

The savings in space (and performance) gained by having a binary header (ws) are outweighed by the cost in constructing and reading messages.  Debugging a binary protocol is less easy than a textual one.

==MIME

There's potential gain from using all the existing base here.  Take a leaf from BEEP and set a default that means that basic implementations don't need to be bothered by the existence of the option.  Thus, at most you have maybe four characters (\r\n\r\n) separating protocol header from body.

MIME also provides a channel for extension.

==Channels

This seems a rather fundamental difference.  I was in two minds on this issue until I realized something.  Channels are not the issue - it's doing multiple things with the one session.  Channels are just one way of addressing the problem.  Delegating any responsibility to a higher layer is another way.

HTTP doesn't have channels.  There's no multiplexing.  If you need multiplexing, you open a new connection.  We're somehow managing.  

What does HTTP provide in this regard that (f'rinstance) TCP does not?  Messages are addressed to specific destinations.  So I have a (half-baked) suggestion: Use MIME and allow requests to be addressed.

A server has addressable entities.  Each of those entities might be the source of events.  A server might identify an originating resource in a MIME "From:" header.  Each of those entities might also the target of an event.  A client might identify a target resource with a "To:" header.

The initial concern here is that a client does not have an identity, nor does it conceivably have addressable resources.  If an application wishes to establish the idea that multiple entities exist at a client, then "From:" and "To:" headers can also be used in the reverse.

Now, for those applications that don't care, make addressing optional.  Don't force them to do anything.  After all, for those who truly care, there is an address established on the server end when the upgrade is made (unless the upgrade is made with a resource URI of "*"); and we can assume a default ("*"?) on the client.

==URI

When dealing with this same problem in Geopriv, we flip-flopped on the issue a number of times.  Whether or not a URI is necessary is a trade-off that can be happily made in either direction.

BWTP doesn't define a URI scheme.  It relies on a client having contextual information/prior knowledge to determine that a given URI actually points to a BWTP session.  Sure, there are those who choose to assign semantics to URIs, but that doesn't help a generic processor, like a browser make any sensible decision on how to use it.

To some extent, whether a URI is useful depends on whether there is any benefit to be had from establishing a session without some sort of pre-existing context.  How necessary is the contextual information specific to the application?

Based on this sort of analysis, the choices made in BWTP and WS are opposite to what their protocol definitions facilitate.  WS relies on application context to a greater extent, where BWTP allows for the use of meta-data.  It would seem that BWTP is better equipped to providing any contextual information through MIME headers; therefore, BWTP is more suited to having its own scheme.  Because WS doesn't provide any in-protocol information to a generic processor that would allow it to make use of a WS connection without prior knowledge, a URI scheme is of less benefit for WS.

Of course, having a unique namespace for identifiers has its own value.

==Martin

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 20 Oct 2009 23:26:31 -0700

Martin,

> The differences are many, but aside from a few they probably make little difference from a technical standpoint.

I totally agree.

Most differences between BWTP and Websocket a non-semantic

For example websocket encodes length in a binary integer of arbitrary
length, while BWTP uses a ascii decimal number of arbitrary length.
The difference is pretty much just a style choice.

I think websocket can be improved to have the capabilities
I'm advocating and the other differences are not important.
See my blog on this exact subject:

   http://blogs.webtide.com/gregw/entry/how_to_improve_websocket

> ==Channels
> Channels are not the issue - ...
> Channels are just one way of addressing the problem.

+1

Channels are just a means to an end and not an end in
themselves.

> Delegating any responsibility to a higher layer is
> another way.

That depends what you think is "the problem". :)

I consider the problem is that applications should
not have to manage TCP/IP connections nor have the
ability to directly request them.

I don't think delegating to a higher layer solves
this problem.

> HTTP doesn't have channels.  There's no multiplexing.  
> If you need multiplexing, you open a new connection.

HTTP does support multiplexing - but not with channels.

If you open another XHR in js, the browser will look
for an idle connection, which may have been opened
by another application in another frame/tab/window
(or in same frame, but previous page).

Also there are proxies and other intermediaries that
multiplex many clients onto shared connections.

> So I have a (half-baked) suggestion: Use MIME and 
> allow requests to be addressed.

If by "request" you mean "message", that is indeed
another possible solution abstract away from the
actual TCP/IP connections - and is probably simpler
than channels.

However, like HTTP, you would need some meta data
with each and every message.

If you are concerned about reducing data usage,
then channels (or at least BWTPs implementation of them)
allows meta data to be sent only once per channel
and then each message can be just data and no meta
data.

If you are less concerned about data usage, then
meta-data per message works ok for HTTP

But your  point is well taken.  There is more
than one way to skin a cat - and we should be
prepared compromise on the means we use to
achieve the outcomes that we want.

cheers

---

From: "Thomson, Martin" <[email-masked]>
Date: Thu, 22 Oct 2009 06:24:09 +0800

Thanks Greg,

Just throwing out some other cat skinning tools... trying to be value-neutral in the process.

>    http://blogs.webtide.com/gregw/entry/how_to_improve_websocket

This post was the impetus for my email.  You make some good points there, but then use that as a vehicle for expressing your opinion.  That's a valuable contribution to the debate, but I wanted to approach this a little more objectively; approach from a different angle.

> If you are concerned about reducing data usage,

That's another variable in the mix.  I personally tend to value aspects like maintainability more highly.

Sure, metadata overheads are high, relative to our expected payload, but given the nature of the problem space, I believe that there are better ways to spend our specification/implementation complexity budget than on such optimizations.

--Martin

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 21 Oct 2009 17:17:30 -0700

Thomson, Martin wrote:
> You make some good points there, but then use that as a 
> vehicle for expressing your opinion.

can't help myself :)

> That's another variable in the mix.  I personally tend to 
> value aspects like maintainability more highly.
> 
> Sure, metadata overheads are high, relative to our 
> expected payload, but given the nature of the
> problem space, I believe that there are better ways to
> spend our specification/implementation complexity budget
> than on such optimizations.

I can see both sides of this.

Needless to say that even with per message meta-data, I think we
can do a lot better than HTTP which has a lot of request/response
baggage in the accept-xyz headers.

So I don't think that we should immediately conclude that if
per message meta-data is supported, then we are going to see
hundreds of extra bytes per message.

One suggestion I have made previously is to simply define
another type byte for websocket that would mean the content
is mime encoded content with standard headers terminated by
CRLFCRLF.     To put that into my simplified websocket BNF
without channels is:

    ws1-frame           = frame-length
                          frame-type
                          octet-data

    frame-type          = 0x00    ; utf8 frame
                        / 0x01    ; mime encoded frame
                        / 0x02-FF ; undefined binary frame

    frame-length        = unlimited-integer

    unlimited-integer   = *( %x80-FF ) %x00-7F
                        ; concatenate 7 low order bits from each octet
                        ; to form a binary integer

    octet-data          = *( %x00-FF )
                        ; the number of octets is exactly the length determined
                        ; by the frame-length

But this would probably need either some websocket API changes
or some serious extra header conventions to allow transparent
sharing  of TCP/IP connections between websocket "connections",
but it at least opens the possibility.

In fact, header conventions could also be used to
implement channels and other protocol extensions without
needing to put them into the framing.

I think the cat is feeling very nervous about all
the different types of knives that can be used !

However, transporting mime encoded content is such an
obvious thing to do, that the strenuous rejection of
defining a type value for it made me even more concerned
about the non extensible nature of a fixed type byte
mapping.

cheers

---

From: "Thomson, Martin" <[email-masked]>
Date: Thu, 22 Oct 2009 10:28:52 +0800

Hi Greg,

> hundreds of extra bytes per message.

In many cases no parameters will be needed.  Thus, applications get the best of both worlds.

For many WS-like use cases, I can think of no real benefit that is gained by adding additional parameters.  We've got folks looking for more than that though.

> CRLFCRLF.

I would go further and make everyone pay the 4 byte cost.

> but it at least opens the possibility.

We should not necessarily make protocol decisions based on a draft API for that protocol.  But then, you knew that ;)

> needing to put them into the framing.

That was the essence of my email...  That has certain advantages from a complexity/flexibility standpoint.

> the different types of knives that can be used !

There has to be a lolcats pic for that.  Google fails me.