Subject: [hybi] Requirement: framing

From: Salvatore Loreto <[email-masked]>
Date: Tue, 02 Mar 2010 21:57:30 +0200

folks,

there has been discussions on the mailing list about WebSocket framing
and the opportunity or not to have also a standard mechanism for 
chunking of the message payload.

at present we have in the requirement draft the following requirements:

    REQ. 2:  The Web Socket Protocol MUST layer a minimal framing
       mechanism on top of TCP data stream.

    REQ. 3:  The Web Socket Protocol MUST define the supported frame
       types.

client side:

    REQ. 9:  The Web Socket Client MUST have the ability to send
       arbitrary text content to the server on the established
       communication channel, in the form of ordered discrete blocks.

    REQ. 10:  The Web Socket Client MUST have the ability to send
       arbitrary binary content to the server on the established
       communication channel, in the form of ordered discrete blocks.

server side:

    REQ. 12:  The Web Socket Server MUST have the ability to send
       arbitrary text content to the client on the established
       communication channel, in the form of ordered discrete blocks.

    REQ. 13:  The Web Socket Server MUST have the ability to send
       arbitrary binary content to the client on the established
       communication channel, in the form of ordered discrete blocks.

I would like to see a discussion on the framing requirements
and also check if there is or not consensus on chunking.

all thoughts or comments are very well welcome and appreciated.

cheers
Sal

Salvatore Loreto
www.sloreto.com

---

From: Scott Ferguson <[email-masked]>
Date: Tue, 02 Mar 2010 12:32:45 -0800

> and also check if there is or not consensus on chunking.
The main additional requirement is that the sender and receiver must be 
able to implement the framing using a small-ish fixed-size buffer. 
Chunking itself isn't a requirement, but it's an obvious solution.

For clarification, the "block" refers to the combined message, i.e. 
after any chunks are merged? If so, I think we're just missing the 
fixed-size buffer requirement and it looks fine.

-- Scott
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>
>

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 02 Mar 2010 22:10:47 +0100

> chunking of the message payload.

Sal,

I'm not sure that framing is a requirement.   Isn't that a solution?

The requirement is probably to support the datagram style websocket
API?   The reason might like to discuss some of the advantages of
datagram vs streaming APIs (eg avoids the need to parse/detect
partial content).

With regards to chunking, I can think of several requirements that
if accepted would imply chunking as a solution.

  A client/server MUST be able to commence transmission of a
  datagram before the total size of the datagram is known.
  Reason: This will allow dynamic datagrams to be constructed and sent
  without the need to buffer the entire datagram.

  A client/server MUST be able to send/receive a datagram that
  is larger than fixed size buffer.   Reason: it is a common
  technique to use fixed size buffers.  (This is the
  requirement Scott raised).

  A large message MUST NOT be able to delay the transmission
  of a small message.   Reason: quality of service and fair share
  of resources.

I think this last one is unlikely to have wide support.  It's
not that useful on it's own, unless channels (requirement to
allow only a partial ordering among messages) or multiplexing
are also included.  So I think this is something for sub
protocols - but it it's worth noting that having chunking
in the base protocol will make implementation of interleaving,
channels and multiplexing easier and more standardized in
the subprotocols.  So maybe there is a weaker SHOULD
requirement there to well support these types of features
in a subprotocol?

cheers

---

From: Roberto Peon <[email-masked]>
Date: Tue, 2 Mar 2010 13:13:24 -0800

> requirement there to well support these types of features

Even with multiplexing, this requirement could only be supported (without
requiring infinite buffering..) if flow-control was added for the messages.
-=R

---

From: Scott Ferguson <[email-masked]>
Date: Tue, 02 Mar 2010 14:19:20 -0800

>   requirement Scott raised).
These two are equivalent, I think. I just used the fixed size buffer to 
make the idea concrete. The first requirement by itself would be 
sufficient if people think it's more clear.

-- Scott

---

From: Justin Erenkrantz <[email-masked]>
Date: Wed, 3 Mar 2010 08:12:04 +0100

On Tue, Mar 2, 2010 at 11:19 PM, Scott Ferguson <[email-masked]> wrote:
> These two are equivalent, I think. I just used the fixed size buffer to make
> the idea concrete. The first requirement by itself would be sufficient if
> people think it's more clear.

Agreed - partial frames need to be supported rather than forcing a
datagram to be passed along end-to-end as a fixed item.

IOW, to make it even a touch more concrete (just to make sure we're
all on the same page), if someone sends one 128KB "message", in the
frameworks I work on (httpd and serf), they ain't going to be sending
or receiving it in one go - it will be processed in multiple passes
via multiple non-blocking socket calls...buffering of the entire
"message" before processing is a non-starter - the APIs have to handle
fragments.  -- justin

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 03 Mar 2010 08:24:29 +0100

Scott Ferguson wrote:

> These two are equivalent, I think. 

I guess so, how about this text then:

  A datagram MUST be able to sent when it's total
  size is either unknown or exceeds a fixed buffer size.
  Reason: This will allow dynamic datagrams to be constructed
  and sent without the need to buffer the entire datagram.

Justin Erenkrantz wrote:
> the APIs have to handle fragments.

I think the APIs are a different matter to the protocol
itself and I think there is definitely room for simple APIs
that do not support fragments.

cheers

---

From: Mridul Muralidharan <[email-masked]>
Date: Wed, 3 Mar 2010 15:25:48 +0530 (IST)

----- Original Message ----
> From: Justin Erenkrantz <[email-masked]>
> To: Scott Ferguson <[email-masked]>
> Cc: HyBi <[email-masked]>
> Sent: Wed, 3 March, 2010 12:42:04 PM
> Subject: Re: [hybi] Requirement: framing
> 
> On Tue, Mar 2, 2010 at 11:19 PM, Scott Ferguson wrote:
> > Greg Wilkins wrote:
> >>

Wont this not logically lead to need for multiplexing messages ?

Regards,
Mridul

      The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 03 Mar 2010 14:23:54 +0100

> Wont this not logically lead to need for multiplexing messages ?

It could, but I don't think there is agreement to have that in
the base protocol.

What I'm hoping for is that we will agree a base level of
fragmentation and maybe a smear of meta data.   This can then
be used as a common base for multiple diverse sub protocols
to implement channels, multiplexing, etc.

But I guess we will have to see where the process takes us.

cheers

---

From: Arman Djusupov <[email-masked]>
Date: Wed, 03 Mar 2010 16:18:26 +0200

Processing a single fragmented message would not require multiplexing unless frames that belong to different messages intermix on the wire. Since the latter (intermixing frames from different messages) can be avoided, multiplexing is not required.

When a message is already buffered, our own WebSocket implementation sends messages as a single big frame that starts from 0x80, irrespective of the message size. When a message is not buffered and the payload needs to be streamed, it is sent as a sequence of frames; the first and subsequent frames start with byte 0x81, the final frame that terminates the sequence starts with 0x80.

This approach gives us the ability to entirely avoid framing when the final size of the message is already known, and to use a configurable (not fixed) buffer size when the message needs to be framed (which also permits flushing at will).

It seems that Scott's implementation uses a different approach, so we already have at least 2 proprietary solutions for this very simple problem. IMHO the WG must take a decision on framing rather fast, otherwise the number of proprietary solutions might grow and it will be harder to agree on which one to use.

With best regards,
Arman

---

From: Mridul Muralidharan <[email-masked]>
Date: Wed, 3 Mar 2010 20:13:56 +0530 (IST)

I agree that if the entire message is generated apriori, it can be essentially something similar to - lock channel, send message, unlock channel.
Even something this primitive could possibly be used to remove need for multiplexing (at the cost of memory, etc).

The way Justin described the implementation, and the way most async IO frameworks typically work would preclude this.
At client side it might be easier to guarantee the locking - tougher at server side : a not-so-toy server rather (buffering requirements, etc).

I hope I am not missing something here.

Regards,
Mridul

----- Original Message ----
> From: Arman Djusupov <[email-masked]>
> To: Mridul Muralidharan <[email-masked]>
> Cc: Justin Erenkrantz <[email-masked]>; Scott Ferguson <[email-masked]>; HyBi <[email-masked]>
> Sent: Wed, 3 March, 2010 7:48:26 PM
> Processing a single fragmented message would not require multiplexing unless 
> frames that belong to different messages intermix on the wire. Since the latter 
> (intermixing frames from different messages) can be avoided, multiplexing is not 
> required.
> 
> When a message is already buffered, our own WebSocket implementation sends 
> messages as a single big frame that starts from 0x80, irrespective of the 
> message size. When a message is not buffered and the payload needs to be 
> streamed, it is sent as a sequence of frames; the first and subsequent frames 
> start with byte 0x81, the final frame that terminates the sequence starts with 
> 0x80.
> 
> This approach gives us the ability to entirely avoid framing when the final size 
> of the message is already known, and to use a configurable (not fixed) buffer 
> size when the message needs to be framed (which also permits flushing at will).
> 
> It seems that Scott's implementation uses a different approach, so we already 
> have at least 2 proprietary solutions for this very simple problem. IMHO the WG 
> must take a decision on framing rather fast, otherwise the number of proprietary 
> solutions might grow and it will be harder to agree on which one to use.
> 
> With best regards,
> Arman
> 
> >> From: Justin Erenkrantz 
> >> To: Scott Ferguson 
> >> Cc: HyBi

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Mar 2010 19:24:17 +0000

Arman Djusupov wrote:
> streamed, it is sent as a sequence of frames; the first and subsequent 
> frames start with byte 0x81, the final frame that terminates the sequence 
> starts with 0x80.
...
> It seems that Scott's implementation uses a different approach, so we 
> already have at least 2 proprietary solutions for this very simple problem. 
> IMHO the WG must take a decision on framing rather fast, otherwise the 
> number of proprietary solutions might grow and it will be harder to agree 
> on which one to use.

Since these solutions with binary frames cannot be used with browsers'
WebSocket, they are obviously intended only for non-browser clients, or
for something like Flash clients.

Can I ask why you are using WebSocket at all for these applications,
instead of a simple TCP socket?

It's hard to justify requirements, if the only users requiring it
could easily use something else instead.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Mar 2010 19:28:54 +0000

Roberto Peon wrote:
>    Even with multiplexing, this requirement could only be supported
>    (without requiring infinite buffering..) if flow-control was added for
>    the messages.

Without multiplexing, flow control can be carried by TCP itself, and
that's quite efficient.  It's only with multiplexing that you have to
add explicit flow control messages to avoid problems.

The WebSocket draft doesn't seem to say anything about flow control.

I don't know if that means a browser implementation will queue
WebSocket data messages until it crashes running out of memory if the
server sends too fast, or will stop reading from the TCP port and the
server is expected to cope with blocking.  Either way, things like the
"simple Tic-Tac-Toe" WebSocket example will break in these situations.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Mar 2010 19:31:58 +0000

> fixed-size buffer requirement and it looks fine.

I don't think there's anything in the current specs which prevents
implementation with fixed-size buffers, except for the JavaScript/DOM API.

An example of something which would prevent it would be a checksum at
the end of a message which had to be verified before forwarding or
processing the message.  (The correct thing to do in that case is
permit forwarding the message parts, and make sure there's a way to
signal "abort this message" instead of "clean end".  In WebSocket
that's begging for 0xFE to be given that meaning.)

-- Jamie

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 03 Mar 2010 11:43:11 -0800

> instead of a simple TCP socket?
Then you should argue for ripping out the binary encoding entirely from 
Web Sockets. But if you leave the binary encoding in, it needs to be 
done right. You can't have a binary encoding that requires a sender 
buffer an arbitrarily large message before sending it.

-- Scott

---

From: Ian Hickson <[email-masked]>
Date: Wed, 3 Mar 2010 19:45:20 +0000 (UTC)

> buffer an arbitrarily large message before sending it.

Right now, WebSocket doesn't define any binary frames at all. It has a 
length-encoding mechanism for future use, but it doesn't designate any of 
the frame types that use that mechanism as valid.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 03 Mar 2010 11:47:54 -0800

> implementation with fixed-size buffers, except for the JavaScript/DOM API.
Of course there is.  The binary encoding uses a length prefix to frame 
the messages. The only way you know the length of the complete message 
is to buffer it.

-- Scott

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 03 Mar 2010 11:54:10 -0800

> the frame types that use that mechanism as valid.
If you're not going to do it right, rip it out and reserve the 0x80-0xff 
for future use.

-- Scott

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Mar 2010 20:16:06 +0000

> buffer an arbitrarily large message before sending it.

The length-delimited frame encoding _does not_ require the sender to
buffer an arbitrarily large message.

WebSocket does not require you to make your application's binary
messages correspond exactly with WebSocket length-delimited frames.

That's why I'm asking why are you using WebSocket instead of a TCP
socket?  I'm not trying to get you to reveal trade secrets or tell you
not to use it.  Just want to know what is the use case that isn't well
satisfied by TCP - and see if another I can suggest a solution.

(By the way, I'm writing a generic TCP-over-WebSocket shim that must
work with all TCP applications, so I'm well aware of buffering,
framing, timing and closing issues.)

It's possible that defining a particular way to carry split messages
wouldn't prevent splinting into proprietary variants anyway - because
there's several different types of buffering requirements.

If you think the existing WebSocket spec forces you to have
arbitrarily large buffering, you're mistaken.  However, it does leave
quite a lot of things ambiguous.

One way to use it within spec is to send a *very large* length in each
direction initially, and just send your entire message sequence in a
single WebSocket frame.

That doesn't work in the JavaScript API, but then neither do binary
messages so it doesn't matter.

In your own clients and servers, you can just send a very large length
(e.g. 2^128) with type 0x80 once in each direction, followed by any
amount of binary data.  Then you are already using WebSocket compliant
framing, and you can frame your application messages any way you want
inside the infinite stream.

(The only issue will be what happens when WebSocket-aware
intermediaries emerge and don't know whether to forward the bytes
quickly or not.  And of course break with a 2^128 length, probably
silently corrupting the data. :-) But that problem already occurs with
HTTP streaming; it's not new.)

-- Jamie

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 03 Mar 2010 22:18:37 +0200

> datagram vs streaming APIs (eg avoids the need to parse/detect

RFC1594: http://www.ietf.org/rfc/rfc1594.txt
define Datagram as

    "a self-contained, independent entity of data
    carrying sufficient information to be routed from the source to the
    destination computer without reliance on earlier exchanges between this
    source and destination computer and the transporting network."

so reuse "Datagram" term can be misleading, as the "WebSocket Datagram" 
it will in any case
sent over an established connection and so it will relay on an earlier 
echange!

In the current version of requirement draft 
(http://tools.ietf.org/id/draft-loreto-hybi-requirements-00.txt)
I used and defined Frame (using same text taken from BWTP draft)

    frame:  The basic unit of Web Socket communication, consisting of a
       structured sequence of octets matching the syntax defined in the
       actual protocol and transmitted on the established communication
       channel.

If people do not like Frame term, we have to come up with a new one 
(e.g. message).

/Sal

www.sloreto.com

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Mar 2010 20:18:57 +0000

Scott Ferguson wrote:
> Rip it out and reserve the 0x80-0xff for future use.

That's a fair comment, since the meaning of those frames is
unspecified, why do we even have to define how those frames are
delimited?  Nothing should receive those frames unless it's negotiated
that it understands them.  It should be an error to send them.

However, I'd be inclined to split the reservation into "reserved for
future WebSocket revisions and approved extensions" versus "reserved
for applications to use as they want".

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Mar 2010 20:22:52 +0000

> At client side it might be easier to guarantee the locking - tougher at server side : a not-so-toy server rather (buffering requirements, etc).

Servers must be able to deal with blocked sends somehow.

After all, what happens if the client stops reading their TCP for a
while because it's slow?

Any async IO framework which can't cope with the client not reading
has problems.  And if it can cope, then it can also cope with
local-head-of-line blocking, which is another way of saying lack of
multiplexing.

Even with multiplexing, you really need per-channel flow control to
avoid client resource problems and to preserve channel independence
(especially if the channel users are actually independent processes :-)

So, even with multiplexing in the protocol, an IO framework has to
cope with blocked sends.

Multiplexing is good, but it doesn't solve this particular problem.
(Though it does reduce the amount of buffering needed in some cases.)

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Mar 2010 21:06:21 +0000

> the messages. The only way you know the length of the complete message 

There is no requirement that you encode binary messages as a single
WebSocket length-delimited frame.

You can just as well use length-delimited frames like the way HTTP
uses length-delimited chunks - which is what some people are actually doing.

The WebSocket spec doesn't tell you to use one method or the other,
because it only defines length-delimited framing.  (The reasons why it
bothers are deeply unclear though.)  It doesn't specify that those
frames correspond with your application's messages - that's an
assumption you're making that isn't in the spec.

-- Jamie

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 03 Mar 2010 13:10:19 -0800

> messages correspond exactly with WebSocket length-delimited frames.
Then why have frames at all? If the frames don't delimit a 
datagram/message, they're not doing anything useful. The same is true of 
the text encoding. If you don't want a datagram concept at the heart of 
Web Sockets, kill the framing. If you want a datagram at the heart of 
Web Sockets, you need to make it work for all defined encodings.
> satisfied by TCP - and see if another I can suggest a solution
>   
Non-browser clients use HTTP as well, e.g. for RPC or SOAP-style 
messaging. Those non-browser clients can benefit from WebSockets for the 
exact same reasons as browser clients (including the benefits of 
integrating with their existing applications/server frameworks). 
However, those clients don't have a silly text limitation, so there's no 
sense requiring them to encode their 8-bit data in utf-8.

The issues involved are identical, with the exception of utf-8 encoding, 
and since we'd like to provide a common server and client API 
(eventually to be supported in the next version of the Java Servlet 
spec), it's silly to give primacy to the text encoding.

So we don't exactly know how people will use it, just like HTTP/servlets 
didn't exactly know how it would be used.

Our internal/prototype use is a binary redesign/cleanup of XMPP using 
Hessian to serialize the payloads. Like XMPP, it's packet based, which 
fits perfectly into the current framing of WebSockets. We'd like to make 
that available to anyone to use, like Hessian is today.  Web Sockets 
plus a common Java server API would let that happen easily.

(But creating a general API/protocol for both browser and non-browser 
clients is more important to us than the specific protocol we're 
experimenting with.)
> framing, timing and closing issues.)
Then you also know that using utf-8 to encode binary data is silly. It 
would be cleaner to follow the HTTP model and have the datagram/payload 
be 8-bit binary and let text sub-protocols use utf-8 on top of the clean 
datagram/stream layer, e.g. through content/sub-protocol negotiation, 
just like REST/XML does on top of HTTP.

(That sound like a very cool project, by the way. I'd think you'd want a 
clean binary stream as the primary for that project.)
> quite a lot of things ambiguous.
Is the datagram/message at the heart of Web Sockets or not? If not, then 
you're right, the delimiters in the spec are entirely bogus and can be 
ignored or hacked around or eliminated. If datagrams are at the heart of 
Web Sockets, then it's unreasonable to require the entire datagram to be 
buffered, when it very easy to design the spec to not require infinite 
buffering.

-- Scott

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 03 Mar 2010 22:50:39 +0100

> If you're not going to do it right, rip it out and reserve the 0x80-0xff

If we are going to rip out one of the framing mechanisms---- please
let it be the sentinel framing.

There is no need for 2 framing mechanisms.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 03 Mar 2010 22:57:11 +0100

Sal,

You are completely right that use of the word datagram is confusing.

Frame is sufficient - only if we don't have fragmentation.  If we
have fragmentation, then we also need a word to describe a related
sequence of Frames - "message" work for me.

cheers

Salvatore Loreto wrote:
> ------------------------------------------------------------------------
>

---

From: Mridul Muralidharan <[email-masked]>
Date: Thu, 4 Mar 2010 03:44:45 +0530 (IST)

There is a difference between how servers handle case of "client not reading data"/"client not reading data fast enough"/etc, and data meant for a client (request) getting generated over time : whether cos the data is too large (file transfer ?), is getting streamed (not classic streaming, but streamed from other source like db - which might be 'slow'), etc - while other messages are to be sent in parallel (if you want to avoid head of line blocking).

Regards,
Mridul

----- Original Message ----
> From: Jamie Lokier <[email-masked]>
> Cc: Arman Djusupov <[email-masked]>; HyBi <[email-masked]>
> Sent: Thu, 4 March, 2010 1:52:52 AM
> > I agree that if the entire message is generated apriori, it can be essentially 
> something similar to - lock channel, send message, unlock channel.
> > Even something this primitive could possibly be used to remove need for 
> multiplexing (at the cost of memory, etc).
> > 
> > The way Justin described the implementation, and the way most async IO 
> frameworks typically work would preclude this.
> > At client side it might be easier to guarantee the locking - tougher at server 
> side : a not-so-toy server rather (buffering requirements, etc).
>

---

From: Mridul Muralidharan <[email-masked]>
Date: Thu, 4 Mar 2010 03:45:57 +0530 (IST)

As much as I would prefer binary framing, with textual content being special case of it, I agree that if it is under-/un-specified, it is way better to leave binary framing out until it is thrashed out properly.
It just leads to ambiguity otherwise.

Regards,
Mridul

----- Original Message ----
> Sent: Thu, 4 March, 2010 2:36:21 AM

      Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

---

From: Mridul Muralidharan <[email-masked]>
Date: Thu, 4 Mar 2010 03:54:22 +0530 (IST)

----- Original Message ----
> From: Scott Ferguson <[email-masked]>
> To: Jamie Lokier <[email-masked]>
> Cc: HyBi <[email-masked]>
> Sent: Thu, 4 March, 2010 2:40:19 AM
> Then why have frames at all? If the frames don't delimit a datagram/message, 
> they're not doing anything useful. The same is true of the text encoding. If you 
> don't want a datagram concept at the heart of Web Sockets, kill the framing. If 
> you want a datagram at the heart of Web Sockets, you need to make it work for 
> all defined encodings.

Personally, I would prefer binary framing too - with textual framing a special case of it.

But one thing which comes to mind is, if we have a transparent framing, it is possible to intermediaries (server itself, other proxies if not ssl, etc) to 'break' a frame into smaller parts ... things should hopefully not break in the face of that : which is why a datagram approach (which implicitly does not allow these since it is expected to be self-contained message) is slightly uncomfortable.

> Non-browser clients use HTTP as well, e.g. for RPC or SOAP-style messaging. 
> Those non-browser clients can benefit from WebSockets for the exact same reasons 
> as browser clients (including the benefits of integrating with their existing 
> applications/server frameworks). However, those clients don't have a silly text 
> limitation, so there's no sense requiring them to encode their 8-bit data in 
> utf-8.
> 
> The issues involved are identical, with the exception of utf-8 encoding, and 
> since we'd like to provide a common server and client API (eventually to be 
> supported in the next version of the Java Servlet spec), it's silly to give 
> primacy to the text encoding.
> 
> So we don't exactly know how people will use it, just like HTTP/servlets didn't 
> exactly know how it would be used.
> 
> Our internal/prototype use is a binary redesign/cleanup of XMPP using Hessian to 
> serialize the payloads. Like XMPP, it's packet based, which fits perfectly into 
> the current framing of WebSockets. We'd like to make that available to anyone to 
> use, like Hessian is today.  Web Sockets plus a common Java server API would let 
> that happen easily.
> 
> (But creating a general API/protocol for both browser and non-browser clients is 
> more important to us than the specific protocol we're experimenting with.)

I agree that it makes no sense to encode binary data using utf-8 ! That is just plain silly and extremely inefficient.

> Then you also know that using utf-8 to encode binary data is silly. It would be 
> cleaner to follow the HTTP model and have the datagram/payload be 8-bit binary 
> and let text sub-protocols use utf-8 on top of the clean datagram/stream layer, 
> e.g. through content/sub-protocol negotiation, just like REST/XML does on top of 
> HTTP.
> 
> (That sound like a very cool project, by the way. I'd think you'd want a clean 
> binary stream as the primary for that project.)
> Is the datagram/message at the heart of Web Sockets or not? If not, then you're 
> right, the delimiters in the spec are entirely bogus and can be ignored or 
> hacked around or eliminated. If datagrams are at the heart of Web Sockets, then 
> it's unreasonable to require the entire datagram to be buffered, when it very 
> easy to design the spec to not require infinite buffering.

My understanding from the discussion on the list seems to be that -

a) Framing is more for knowing the limits - not to specify datagrams or fully contained messages.
b) Multiplexing is heavily frowned upon.

In which case, a "http-ish" style approach could be taken too - similar to chunked encoding : start a message, send 1 or more frames, send end of message marker.
I think someone mentioned something like this before too.

This ofcourse prevents future easy addition of multiplexing.

Regards,
Mridul

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Mar 2010 22:47:46 +0000

Mridul Muralidharan wrote:
> There is a difference between how servers handle case of "client not
> reading data"/"client not reading data fast enough"/etc, and data
> meant for a client (request) getting generated over time : whether
> cos the data is too large (file transfer ?), is getting streamed (not
> classic streaming, but streamed from other source like db - which
> might be 'slow'), etc - while other messages are to be sent in
> parallel (if you want to avoid head of line blocking).

What I meant is that in all cases you still have to handle blocking
sends in the server framework, so head of line blocking doesn't add
any additional burden in that respect.  You said that async IO
frameworks generally don't handle it, but they have to no matter what
protocol is used; there's no way to avoid sender blocking, except
unbounded buffering.

(Btw in my own framework, I permit both blocking and unbounded
buffering, with "fence" checks in data generators where they can block
at convenient places instead of on every byte generated.  It works
very well.)

I agree that there are different kinds of blocking and that it's good
to avoid head of line blocking (essential if you want independent
streams to behave independently except for timing).  If you've read my
messages you will know I generally favour a multiplexing capability,
preferably an opportunistic one that's invisible to APIs.

-- Jamie

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 04 Mar 2010 00:02:14 +0100

Mridul Muralidharan wrote:

> But one thing which comes to mind is, if we have a transparent framing, 
> it is possible to intermediaries (server itself, other proxies if not 
> ssl, etc) to 'break' a frame into smaller parts ... things should hopefully
> not break in the face of that : which is why a datagram approach (which
> implicitly does not allow these since it is expected to be self-contained
> message) is slightly uncomfortable.

I think we are using the word Frame for two different things.

I think the basic reasons for handling data in semantic blocks
(rather than streaming) are:

 + it allows simpler handling by the receiver if natural boundaries in
   the data are signalled.  The receive need not dispatch or attempt to
   parse partially data received data.

 + it allows intermediaries to know when any buffers need to be flushed.

There are also reasons to fragment a semantic block into smaller
chunks:

 + start transmitting a dynamically generated block before
   the total size is known.
 + to send large semantic blocks using small buffers.

I think it is possible to allow intermediaries to break or assemble
fragments, but they cannot break or assemble the semantic blocks of
data (else the value of a semantic block of data would be lost).

I think the term "Frame" has been used to mean both a semantic block
of data and a chunk of data.

If we have a protocol that only has Frames, then it is not OK
for intermediaries to break a frame in to smaller frames.  This
would break the receivers that assumed no partial input withing
a frame.

If we have a protocol that has messages and frames, then a message
can be broken into frames and an intermediary can break a frame
in to smaller frames (or assemble frames into larger frames), so
long as the message boundaries are preserved.

> b) Multiplexing is heavily frowned upon.

I would say Multiplexing is "approach with caution" rather than frowned upon.

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 04 Mar 2010 00:24:42 +0100

Mridul Muralidharan wrote:
> As much as I would prefer binary framing, with textual content being special 
> case of it, I agree that if it is under-/un-specified, it is way better to
> leave binary framing out until it is thrashed out properly.

Mridul (well actually most of us participating in this thread including me),

Perhaps we should focus on capturing the requirement before decided if we
need 0, 1 or 2 framing mechanisms in the protocol and if they should
be binary or not.

I think we are all pretty much agreed that there is a requirement
for the protocol to send blocks of related data with
identified boundaries.  I think the reasons for this are:

 + so that receivers can avoid attempting to parse/handle partial
   content.
 + so intermediaries will know that any buffers must be flushed
   if a boundary is received.

Currently the protocol calls these blocks "Frames", but I think
the term "Message" is more appropriate - specially in a requirements
document.

We then have to work out if we have 1 message per frame
or can a message be fragmented into multiple frames, where a
frame is a unit of transmission.

I'm do not think that we can directly require messages to
be fragmented into multiple frames - that's an design decision.
The requirement is more likely to be worded about being able
to send messages with sizes that are either unknown or that
exceed an implementation buffer size.

If binary data is not required, then I think sentinel framing
is a solution that meets the requirements:
  + can carry text data
  + identifies message boundaries
  + can send unknown/large size messages.

If binary data IS required, then sentinel framing is not
enough of a solution as it can't do binary efficiently.

The length framing solution can send binary data
and identify boundaries, but can't to unknown/large
size messages.

A solution that fragments a message into 1 or more frames
would satisfy all requirements:
  + can carry both text or binary data
  + identifies message boundaries
  + can send unknown/large size messages.

cheers

---

From: Salvatore Loreto <[email-masked]>
Date: Thu, 04 Mar 2010 10:28:10 +0200

Hi there,

it is pretty clear from the discussion that the following requirement 
are not so
good to explain the concept

so I propose that we introduce the following terminology:

    Message = user message: a block of related data with identified
    boundaries.

and rephrase the above requirements:

     REQ. 2    The WebSocket Protocol MUST be able to handle (send and 
receive) messages
                    on top of a TCP data stream connection.

     Reason: transfer data as message prevents the receiver to 
parse/handle partial content.

not sure if we need REQ 3. I mean I do not think we need to mandate in 
the requirement draft
the design of specific message types neither what the data types (e.g. 
text, binary ) the message
can contain.

However it is not yet clear if there is or not an agreement on the 
possibility to in some way
chunk the message.

     REQ. X    It MUST be possible to sent a message when the total
                    size is either unknown or exceeds a fixed buffer size.

     Reason: This will allow dynamic messages to be constructed  and 
sent without the need
                   to buffer the entire message.

I would like to see this discussion focusing more on this specific 
requirement
so to better understand if there is or not consensus on *"chunking"*.

I think we also do not need the REQ. 9, 10, 12 and 13 that are listed in 
the current version
of the requirement draft: 
http://tools.ietf.org/id/draft-loreto-hybi-requirements-00.txt

cheers
Sal

Salvatore Loreto
www.sloreto.com

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 04 Mar 2010 09:20:57 -0800

Salvatore Loreto wrote:
> the requirement draft the design of specific message types neither 
> what the data types (e.g. text, binary ) the message can contain.
I don't think having a frame type byte is a requirement. In the current 
draft, it's a mechanism for changing the encoding on a frame by frame 
basis, not any response to a requirement.

However, it is important to specify if the Web Sockets protocol must be 
capable of supporting 8-bit binary or if can be restricted to utf-8 text.

>                   to buffer the entire message.
This is perfect.

-- Scott

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 4 Mar 2010 18:23:25 +0000

> long as the message boundaries are preserved.

That's a nice way of explaining my point of view too.

However, although those are useful concepts, I think from a buffering
point of view they aren't the most effective division.  Take this:

   - Message end boundary implies it MUST now be forwarded to target.
   - Chunk end does not imply any forwarding.
     Can be buffered for any amount of time until a message end.

Note that some applications have no need for semantic message
boundaries.  Take my TCP-over-WebSocket proxy for example (it's a work
in progress to test the technical implications of WebSocket-style
messaging over a range of uses.)

In a byte stream application, it should not split the data into lots of
small semantic *messages*, because those force overly-eager
forwarding, and especially they don't permit splitting and merging in
any kind of intermediary.[1][2]

But on the other hand, such an application cannot split the data into
lots of *chunks* either, because those don't have any guaranteed
forwarding time.

What this means is, under some circumstances you want this combination:

  - The splitting and merging properties of "chunks".
  - Certain forwarding guarantees more akin to "messages".

You could just define chunks to have some forwarding guarantees.

But then that is over-constraining some other situations where it is
significantly more efficient if intermediaries are allowed to buffer
chunks for longer to find chunk merging opportunities.[3][4]

That leads to this:

  - Three types of boundary are useful:

    1. Semantic message boundary with guaranteed forwarding.

    2. Chunk boundary with some kind guaranteed forwarding, for byte
       streams and things like that which need guaranteed progress but
       not semantic boundaries.

    3. Chunk boundary with no bound on forwarding delay.  This
       maximises efficient merging - to reduce byte overhead, packet count,
       and transport inefficiencies.[4]

    Plus two more, which I won't explain but they are important.[5]

And that leads me to this conclusion:

  - Instead of many of boundary type codes, I wonder if it's better to
    have boundary *flags* indicating what combination of properties
    are wanted.

  - From the above, there would be independent "can delay" and
    "non-semantic boundary" flags.  Both clear would be compatible
    with current WebSocket's vague but implied behaviour.

  - "Last chunk" automatically comes for free, because it's the same
    as a message boundary.

(There are some refinements I haven't described to keep it as simple
as possible.)

-- Jamie

[1] Unless the intermediary is aware of the byte stream protocol
    overlaid on top, but it's highly desirable for this to work well over a
    baseline intermediary without special extras.

[2] Note that logical intermediaries and their buffering properties
    applies just the same to client-to-API and server-to-API
    implementations, because they all relay data somehow and have a
    buffering and forwarding strategy.

[3] Merging opportunities are even more useful with multiplexing,
    but they are useful without it too.

[4] You *really* want a small delay for merging over TCP/IP.
    Paradoxically it reduces the time, because some TCP
    implementations add their own relatively long delays (do I need to
    provide references?) when a mixture of full segments and partial
    segments are sent.

[5] "Close" - discussed elsewhere in the context of orderly close.
    "Abort" - an alternate sentinel byte at the end of
    sentinel-delimited messages (I suggest 0xFE), and an other type or
    flag at the end of chunked messages.  This is needed when an error
    occurred in message generation or forwarding.  It is not essential
    without multiplexing because the connection can be abruptly closed
    instead - but that is less informative.  It is *required* with
    multiplexing to prevent forwarders needing infinite buffers.

-- Jamie

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 05 Mar 2010 00:04:31 +0100

>     with current WebSocket's vague but implied behaviour.

Jamie,

I'm stuggling to see the difference between the "can delay" flag
and the "non-semantic boundary" flag.

Isn't a non-semantic boundary just a reason to set the can delay
flag?

Ie. if we just had a no-delay flag, then senders could set that
because a) the frame carried a semantic boundary; or b) the sender
is operating in streaming mode.

What I do like about your suggestion that we can just have
frames with a no-delay flag.  This can be used to build fragmented
message, non-fragmented messages or streams at the whim of
the sub protocols etc.  but all intermediaries need to know
is should they flush or not and a no-delay flag tells them
that.

cheers

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 5 Mar 2010 01:46:54 +0000

> Isn't a non-semantic boundary just a reason to set the can delay

No.  It means chunks which can be merged by removing the boundary.
Basic protocol would be allowed to merge adjacent chunks with this
boundary between; a multiplexing extension would use it to merge
chunks on the same channel.

Note that "can delay" means the receiver is permitted to delay
forwarding forever.  Realistically it will do so as long as it has
buffer space.

=> A byte stream sets "non-semantic boundary" for chunk merging, but
_must not_ set the "can delay" flag, because you're not allowed to
delay received bytes indefinitely.

=> A split message requires _both_ "non-semantic boundary" and "can
delay" for its non-final chunks.

I'm open to better names, by the way.  I think your no-delay name has
plenty of merit - it's easy to understand :-) But also I think that
there's some merit to "a basic implementation is allowed to ignore all
flags and behave as if none of them are set", making each flag an
optional enhancement, which can then be extensions not in the base
protocol, to keep the base really basic.

But perhaps no-delay should be in the base protocol specification to
ensure that forwarders aren't vague about what they have to do, and
don't feel they have to use delaying heuristics either.
In which case, it is a great name.

Actually I can think of lots of better names than "non-semantic
boundary".  For example, "chunk-boundary".

Note: Two flags allows a fourth case which I didn't describe, which
are application messages permitted to be delayed until the last in a
batch.  It's not a requirement, but there's no harm in allowing that
meaning to fall out naturally.

> is operating in streaming mode.

So how does the agent know whether it can remove that boundary,
merging the adjacent frames?  That's the reason for the "non-semantic
boundary" flag.  Byte streaming mode should permit the agent to merge
frames and remove the boundary, but must not permit infinite delay.

There could be an analogous "may-split" flag on a frame, telling
agents that they can introduce "can-delay, non-semantic" boundaries.
Or that could just be implied for all frames when a particular
extension protocol is in use.  It could even be useful to have
"may-split-with-no-delay" variant.  However, splitting is not
something that unaware forwarders need to care about, so this is not
an important flag in the base protocol, in the same way that no-delay is.

> is should they flush or not and a no-delay flag tells them

I agree that the no-delay flag is the most important thing that
framing-aware intermediaries must know about, to build anything useful
that performs well on top of them.

If there isn't that flag, then agents must _always_ guarantee to
forward each frame without waiting for more, to be reliable.  That's
WebSocket's current implied behaviour.

HTTP CONNECT proxies already have to handle that, and it's accepted
for web apps, although not acceptable for some other things, so it's
technically acceptable if "basic" implementations have that same behaviour.

But we should support the option to perform better than CONNECT
proxies, and encourage the use of this very easy to understand flag
even in basic implementations.

Side note (ignore if it complicates things, but it's a useful enhancement):

In my own systems current I actually have two flags: "no-delay":
needed for real-time updates where minimum possible response time is
desired such as games and LAN-speed interactivity, and "small-delay":
which allows a short time for merging subsequent data to make up full
packets, but still guarantees forward progress suitable for
interactive behaviour such as streaming data into a web page.  I find
it's useful to provide both flags in sender APIs, even if it's not
part of a network protocol, to decide how to buffer data from a
data-generating source before sending it onto the network.  The
explicit two flags in the protocol are useful to help proxies make
similarly good decisions.

Interestingly, TCP implementations offer basically those same two
options (no-delay and short-delay), in the form of the TCP_NODELAY
switch.  The "delay" option is just "don't call write() yet".

-- Jamie

---

From: Ian Hickson <[email-masked]>
Date: Fri, 5 Mar 2010 02:29:37 +0000 (UTC)

On Tue, 16 Feb 2010, Dave Cridland wrote:
> On Mon Feb 15 01:12:53 2010, Thomson, Martin wrote:
> >     frame         = text-frame / binary-frame
> >     binary-frame  = %x80 length *%x00-FF
> >     length        = %x00 / %x01-7f / ( %x81-FF *%x80-FF %x00-7F )
> > 
> > This is a canonical length encoding in that it doesn't allow for 
> > leading zeroes.
> 
> Am I alone in seeing two design errors here?
> 
> Firstly, I see no benefit in having two different framing types. I do 
> see a marginal benefit in having multiple frame types, but the idea of 
> distinct framing just strikes me as a bad idea, and prone to error. I 
> think the correct design decision here is that "bytes is bytes", and to 
> the wire, all frames are binary and octet counted. I'm not against an 
> end marker, incidentally, although I'm not convinced it's needed except 
> in early stages of implementation.

Once we send binary data with WebSocket, we'll need length-based framing 
of some kind, because the only way to do sentinel framing with arbitrary 
data is to have an escape mechanism, and that has all kinds of performance 
implications.

However, when a server is just sending UTF-8 text, it is very easy to get 
length measurement wrong without realising it (because of only testing 
ASCII, but outputting the string length instead of the byte length). For 
this reason, and because it is generally much easier, sentinel-based 
framing is used for text frames.

In practice this means that most authors need only implement the sentinel 
framing (which is trivial); only more advanced (and thus competent) 
authors will need to implement the more complicated framing.

> Secondly, is that really a bespoke new method for integer representation 
> on the wire? What is the overriding benefit over a network byte order 
> 32-bit integer? I see that it can be 1-N octets, and therefore will 
> typically save 1-3 octets - I'm assuming we're not expecting to ship 
> more than 256M in a frame, where the trade off is.

It's just a BER compressed integer, as used in various places, e.g. ASN.1 
subidentifiers, or the Ruby and Perl pack() "w" formats, or (almost) 
protobufs (though they have the least significant bits first).

On Wed, 17 Feb 2010, Jamie Lokier wrote:
> 
> But imagine someone writes a trivial WebSocket services which recieves 
> the name of a file (say, it's just an example), and sends back the 
> contents of the file as a message.  Imagine it was written intended for 
> small files, but someone decided to use it for bigger ones one day 
> because it was convenient to do so, and it turned out to work fine.
> 
> Then one day, someone else puts an 8GB file there and expects it to 
> work.  Why should it be fine for 3GB files but fail for an 8GB file?

It's up to the subprotocol or service implementor to define what limits 
are imposed on frame sizes. One of the main reasons for using a variable 
sized integer is to avoid imposing any arbitrary limits, so that when 
we're using Tbps internet, we don't have to rewrite all the protocols to 
send our 8GB files. From the WebSocket point of view, it shouldn't fail 
for any size.

On Thu, 18 Feb 2010, Jamie Lokier wrote:
> 
> But that ignores something: An endpoint shouldn't be sending frame types 
> that the other end doesn't know about.  So there is no need to specify 
> how to discard binary frames, unless there is a general principle of 
> discarding frames with unknown type byte too.

There _is_ a general principle of discarding frames with unknown type 
bytes.

We need to make sure that today's clients don't handle tomorrow's servers 
in unpredicatable ways, because if they do, then we might never be able to 
upgrade the protocol, due to Web pages depending on particular behaviours, 
the same way that, for example, many Web pages depend on browsers doing 
browser sniffing, or on browsers ignoring Content-Location headers.

> One thing that *really* needs to be right early on is making sure future 
> proxies know the frame boundaries, for efficient forwarding.
> 
> If the spec didn't describe binary frame delimiting, we could find, in a 
> few years, that we'd have no choice but to use UTF-8 for everything, 
> even having to encode binary messages into Unicode text, simply because 
> some parts of the net would have proxies parsing WebSocket frames and 
> failing to forward anything else properly or with useful 
> timing/buffering.
> 
> That's not stated in the spec, but it's an implied consequence of the 
> binary frame rule being described that WebSocket-aware proxies will 
> probably apply it.  Perhaps it should be explicit.

I don't see why we'd want any client-side proxies other than SOCKS proxies 
or HTTP proxies (via CONNECT), and on the server-side it seems that 
upgrading dedicated WebSocket-aware proxies would be relatively easy in 
this kind of scenario.

However, whether I'm right and it's a matter for clients or you're right 
and it's a matter for proxies, the effective reason is the same. We need 
to make sure things are defined so that we are forwards-compatible.

On Tue, 23 Feb 2010, Dave Cridland wrote:
> 
> What about a three-way handshake, with client-speaks-first on the 
> WebSocket itself? Something akin to:
> 
> ** HTTP **
> Client: "Give me a websocket session to foo." [HTTP request, probably GET with
> Upgrade]
> Server: "Okay. Your magic token is 'bar'" [HTTP response, followed by switch
> to WS]
> ** WS **
> Client: "My magic token was 'bar'". [WS special message type]
> [... server validates WS session and the subprotocol begins ...]

That's more or less what the spec now does, though with the client doing 
the verification and not the server. (The client is the one that needs to 
be convinced that the server isn't going to be tricked, as far as I can 
tell.)

On Wed, 24 Feb 2010, Arman Djusupov wrote:
> 
> I have a use case which doesn't seem to have been taken into account in 
> the spec. When large binary messages of initially unknown size are 
> getting streamed over a connection, the transport does not know the 
> final size of the binary message when it starts serializing it and so it 
> cannot encode the message's length at the start of the binary frame. In 
> our implementation we handle such cases by buffering data until the 
> buffer is full and then flushing it into a frame of specific length; 
> subsequent data of the same message are sent in the following frames.

Currently there's no way to send binary data with WebSocket at all, but 
going forward, when we add binary support, if one of the use cases is 
sending binary data of unknown size, we can definitely use a chunk-based 
encoding scheme.

Whether it's useful or not depends on what the API for binary data ends up 
looking like. If the API only exposes binary data using Blob objects, then 
there's not really a reason to avoid requiring that the server prebuffer 
all the data ahead of time to determine the length. If we expose the data 
using something akin to the Stream object currently in the HTML spec, then 
we'd probably want to make it chunks of fixed (maximum) size. Since the 
API doesn't have any binary support at all right now, though, it's 
probably premature to worry about this.

> We currently use 0x81 as the identifier of the first and subsequent 
> frames, and 0x80 as the identifier of the last frame. When a message is 
> contained in a single then it starts with 0x80.

Please don't use frame types that aren't defined yet. If you are doing 
something for use unrelated to the WebSocket API, there's really no reason 
not to just use TCP. You can use the same framing as WebSockets (though I 
don't see why, it's not that great for general purposes), but if you use 
WebSocket itself, it's just going to cause problems for you when the API 
is updated to do binary.

On Mon, 1 Mar 2010, Dave Cridland wrote:
> 
> This behavioural flux, though, is an excellent argument for putting 
> keepalive behaviour into the core, as the people designing the 
> frameworks are likely to have good ideas of the defaults, and keepalive 
> frequencies will need to be controlled per-deployment, typically, rather 
> than per-application.

We could reserve a frame byte for control messages (server talking to the 
browser rather than to the script), with the null message being put aside 
for keep-alive purposes, if people think that would be especially useful. 
I presume we would want the server in charge of sending keepalives rather 
than the browser or the script?

On Tue, 2 Mar 2010, Pieter Hintjens wrote:
> On Tue, Mar 2, 2010 at 9:37 PM, Ian Hickson <[email-masked]> wrote:
> >
> > If it's less than an order of magnitude, the benefits gained [by 
> > supporting multiple compression algorithms] would be outweighed by the 
> > damage of having more than one choice.
> 
> Choice = damage?  I've rarely heard such a... singular analysis.

In the specific case of software, offering more options is often a bad 
idea, because it leads to more code, meaning more bugs, more testing cost, 
and more documentation cost. In a platform it is similarly problematic, as 
it leads to a higher cost for users of the platform. If the choices 
offered have non-obvious trade-offs (as is the case with compression 
algorithms, cyphers, sorting algorithms, and so on), it is likely that the 
choices made will often be wrong, with the developer optimising for one 
aspect despite this not being the better choice in the developer's case.

For example, if we require that clients support three 
compression algorithms, one optimised for compression ratio, one optimised 
for CPU consumption, and one optimised for compressing music, then it is 
quite likely that developers will select one over the other without 
testing which is, in practice, actually better. For example, they might 
use the high compression one even though their data is essentially 
incompressible (e.g. they're shipping PNG files), resulting in nearly no 
benefit in terms of saved bytes, yet a huge cost in terms of CPU on both 
the client and the server. This is the kind of problem that is hard to 
spot. IMHO, *we are responsible* for helping authors make the right choice 
here, even if in some cases that means simply not giving them a choice.

(There's quite a wide body of research showing that more choice is not 
automatically a good thing in general, by the way. For some informal 
discussion of the matter see, for instance, "Predictably Irrational" by 
Dan Ariely, or Barry Schwartz's TED talk on the paradox of choice.)

> I thought choice meant, for example, that if an algorithm had flaws, it 
> could be replaced.  Performance is only one of many aspects.  Do you 
> recall something called GIF?

I am not arguing that we shouldn't make it possible to revise the standard 
at some future date if we decide another algorithm is needed. There's a 
huge difference between standardising on a variety of algorithms, 
standardising on an open-ended set of algorithms, and standardising on a 
minimal set of algorithms (typically starting at 0 or 1 and only adding 
further options when a clear need is demonstrated).

> Even in your strange parallel universe where one size fits all, what do 
> you actually do if, ten years later, someone finds an algorithm that is 
> indeed 10x faster?

Update the spec.

On Wed, 3 Mar 2010, Scott Ferguson wrote:
> > length-encoding mechanism for future use, but it doesn't designate any 
> > of the frame types that use that mechanism as valid.
>
> If you're not going to do it right, rip it out and reserve the 
> 0x80-0xff for future use.

That's exactly what the spec does.

On Wed, 3 Mar 2010, Jamie Lokier wrote:
> 
> That's a fair comment, since the meaning of those frames is unspecified, 
> why do we even have to define how those frames are delimited?

As you argued in the e-mail quoted a few pages higher, we need it for 
forwards-compatibility.

> Nothing should receive those frames unless it's negotiated that it 
> understands them.  It should be an error to send them.

It is.

> future WebSocket revisions and approved extensions" versus "reserved for 
> applications to use as they want".

Since the API doesn't expose the frame types at all, I don't understand 
why any would fall into the second category. If you're exposing a 
WebSocket server for non-browser UAs, just make the client send something 
in the handshake that opts in to using a different protocol (that looks 
very similar to WebSocket, but isn't), and then this whole spec becomes 
irrelevant, and you can use whatever mechanisms you want.

On Thu, 4 Mar 2010, Greg Wilkins wrote:
> 
> I hope that this type of mass update is a once off due to the 
> circumstances we find in starting off the WG.

There were multiple updates, one for each part of the spec that was 
updated (opening handshake, closing handshake, etc). You can see the 
latest changes to the WebSocket spec, and the other specs I edit at the 
WHATWG, here:

   http://html5.org/tools/web-apps-tracker

Every non-editorial change is also twittered:

   http://twitter.com/WHATWG

...and sent to the commit-watchers mailing list:

   http://www.whatwg.org/mailing-list#commits

The Subversion repository for this work is public:

   svn://svn.whatwg.org/webapps/source

> In general I think it would be far easier handle changes in smaller 
> increments and individual threads.  Also it would be good to see 
> proposed diffs to the draft

All the proposed diffs are available from the Subversion repository above, 
and are also made available through the aforementioned tracker page. For 
example, here's the closing handshake diff:

   http://html5.org/tools/web-apps-tracker?from=4819&to=4820

> before they are actually just put in the draft.

I'm not really sure what you mean by "the draft".

> If the handshake messages are to contain content, then they MUST have 
> headers to indicate the content length to be legal HTTP: in this case, a 
> Content-Length header would be appropriate

The handshake messages don't contain content. Should I add a 
"Content-Length: 0" field to the handshake to make this clearer?

> Considering that we are already shipping products with websocket 
> implementations from the existing draft, can we specify a transition to 
> the new handshake in an appendix.

Not sure what you mean.

> Ie while the standard is under development connections without websocket 
> keys MAY be accepted, but implementation SHOULD warn that support for 
> such connections is deprecated.

You can implement the old draft if you want, but since it has a security 
vulnerability, I would strongly recommend shipping a security update 
immediately to use the new handshake, if you haven't already.

> The text of 1.3, still kind of implies that the HTTP fields must be 
> ordered as per the spec.  Can we add a sentence to say that header 
> ordering may be different.

Done.

> Also I'm not sure this sentence is really that clear:
> 
>  "Additional fields are used to select options in the WebSocket
>    protocol.  The only option available in this version is the
>    subprotocol selector, |Sec-WebSocket-Protocol|:"
> 
> I thought we were going to allow arbitrary extra headers, but that their 
> interpretation was not defined by the ws spec other than the optional 
> Sec-WebSocket-Protocol.

I'm not sure what you mean by "allow"... the client can't send them. The 
server behaviour with unknown (invalid or from-the-future) headers is 
well-defined, though (do nothing).

> We still need to allow headers for cookies, authentication etc.

Oops, I forgot to mention the Cookie header in the intro. Added.

The HTTP auth headers aren't currently added to the WebSocket connection; 
this was briefly in the spec but was taken out based on implementor 
feedback a few months ago.

On Thu, 4 Mar 2010, Greg Wilkins wrote:
>
> The server needs to consider the bytes of content before formulating the 
> accepting the upgrade request and sending the 101 response, so it can't 
> be considered as data in the stream after the 101.

The bytes sent as part of the WebSocket connection after the GET aren't 
needed to send the 101 response.

On Thu, 4 Mar 2010, Greg Wilkins wrote:
> 
> But the problem with this is it assumes success and that a 101 is the 
> only possible response.

Why is that a problem?

> I think that once we have gone through the process, it is very likely 
> that we will see the need for return types other than 101.

Could you elaborate on this? Why would we need both the secure handshake 
and different Status-Code responses?

> For example, it will be hard to stop some servers sending a 401 
> unauthorized if there are missing credentials.

It seems pretty trivial to just not require credentials.

> It might also be desirable to send 302 responses if a server has moved 
> or wants to redirect to a non HTTP port or a secure https/wss port.

This was in an earlier version of the protocol, but there are some serious 
security implications of following redirects cross-origin, and so it seems 
better to get the basics done before we start adding redirects. In either 
case, redirects wouldn't interfere with the key part of the handshake, so 
I don't think that's a problem here. (They'd come first, and their 
security would be based purely on the Origin model, since nothing would be 
exposed to the API until the redirect was followed completely.)

> If the bytes after the request header are sent as request content, what 
> attack verctor is opened up?

I think Maciej described the cross-protocol attack in detail last month, 
that's probably the best description of the problem I've seen.

On Thu, 4 Mar 2010, Greg Wilkins wrote:
> 
> I think there are plenty of intermediaries that don't understand 
> websocket that will pass these bytes on well enough for the handshake to 
> pass.

That's possible. How many will do this but fail to pass enough for the 
protocol to survive? Is there anything in particular you can think of 
which would cause even intermediaries that allow after-GET text through 
(but that fail to keep going after the handshake) to fail early?

On Thu, 4 Mar 2010, Joe Hildebrand wrote:
> 
> At the same time, particularly as this becomes a WG draft, a quick "I'm 
> about to..." mail to the list before a large change will help build that 
> consensus more quickly.

Indeed. I hope the e-mails I sent to this effect in the weeks prior to 
making the handshake change, discussing options I was examining, were 
satisfactory on this front.

> Of course, as always, whenever proposing a change to the document, it is 
> much more likely to get included in the way you want if you also propose 
> text.

Personally I prefer clear problem descriptions to camera-ready text, for 
what it's worth.

On Thu, 4 Mar 2010, Dave Cridland wrote:
> On Thu Mar  4 03:21:28 2010, Ian Hickson wrote:
> > 
> > It's true that if the server doesn't handle UTF-8 correctly, it can 
> > end up outputting 0xFF bytes. In practice, this would need some 
> > out-of-band erroneous data (e.g. an ISO-8859-1 form submission); you 
> > couldn't trigger this bug easily by sending data to the server and 
> > having it return it later, for instance. The length bug, on the other 
> > hand, could occur with no external data: sending non-ASCII data could 
> > very easily result in the server screwing things up if we use lengths 
> > rather than delimiters.
>
> This is nonsensical - I don't see how the (rare) case of someone failing 
> to distinguish between character counts and octet lengths is going to be 
> more common than broken or invalid UTF-8.

I don't really know of a way to prove this one way or the other, 
unfortunately.

> > For scalability it's probably ideal if we can use lengths, but on the 
> > long run for environments where that matters we'll probably just use 
> > compressed frames which would be binary anyway (and thus 
> > length-encoded), so this will probably become a non-issue in that kind 
> > of environment.
> 
> Except you've already thrown away binary framing.

Not sure what you mean.

> > If a client and a server are speaking a specific sub-protocol, they 
> > don't actually have to even use WebSockets -- they can just use a 
> > protocol that happens to look like WebSockets but defines whatever 
> > frame types they want.
> > 
> > If the client is a browser, and thus they're speaking generic 
> > WebSockets, then the frame types would be just those supported by the 
> > API, and there wouldn't be any custom types. So extensions would just 
> > be "registered" by revving the protocol.
> 
> If we ever need to use frame types as part of the protocol (for 
> chunking, to give one discussed example), then we *will* need to 
> consider standardized frame types.

I don't see how this contradicts what I wrote.

> I'd be happy with, at this point, mere reservation of a range for 
> protocol purposes, and leaving a range clear for subprotocol usage.

I don't understand how a subprotocol would ever make use of these frame 
types. The API doesn't expose them.

> > > > [...] users can't be trusted to always provide valid utf-8 data, 
> > > > so if user data is not validated then sentinel encoding allows 
> > > > frame injection attacks.  After all we have learnt with HTTP, it 
> > > > seams silly to repeat the mistake of a protocol that is exposed to 
> > > > such attacks
> > > 
> > > If you have several bits of code sharing a connection - even if it's 
> > > just by sharing a common Javascript framework on a single page, then 
> > > you have security issues from this.
> > 
> > You certainly have security issues, but you don't have this issue 
> > (user data containing 0xFF), because the protocol requires the browser 
> > to make the data be valid UTF-8, and so you'll never be able to inject 
> > a 0xFF byte from the client.
> > 
> > Of course, if the server has other sources of data, and it doesn't 
> > validate them, then it's possible you'll have this problem in the 
> > other direction.
>
> Up above, you implied this was rare.

I believe this problem would be less common than the problem of people 
misimplementing string measurement, yes.

> > >      frame         = text-frame / binary-frame
> > >      binary-frame  = %x80 length *%x00-FF
> > >      length        = %x00 / %x01-7f / ( %x81-FF *%x80-FF %x00-7F )
> > > 
> > > This is a canonical length encoding in that it doesn't allow for leading
> > > zeroes.
> > 
> > Leading zeros are not disallowed. (There wouldn't be much point 
> > disallowing them as far as I can tell.)
> 
> Yes, because multiple representations of the same values has *never* 
> caused a security issue.

On the contrary, it's often the source of problems. However, in this case, 
I don't really see how it could be. Do you have an attack model in mind?

More importantly, I don't really see what our options are here. We can 
certainly ban the client sending leading nulls -- and indeed we have, 
since the client isn't allowed to send binary data at all currently, let 
alone binary data with leading nulls in the length. But what should the 
server do when faced with a buggy or hostile client? If we don't define 
how to handle this, but say that servers should expect it and close the 
connection if they see it, then it's clear to me that most implementations 
would _not_ check for it (that has been my experience with other languages 
and protocols -- asking implementors to check for something for the 
purposes of failing usually leads to them ignoring the requirement on the 
basis that doing something else is "more reliable"). So that leaves us 
with the choice the spec has: define it in a way that is trivial to 
implement interoperably (it's just an intrinsic part of handling valid 
data), and which is apparently harmless.

> > The point isn't to catch ever error, but that the processing be 
> > defined for all inputs. It doesn't matter how errors are handled, so 
> > long as they are handled.
> 
> That's an awesome quote.

Thank you!

> So, your defined behaviour in the face of a DOS or overflow attack is 
> to, what? Just work?

The spec explicitly that "If the user agent is faced with content that is 
too large to be handled appropriately, runs out of resources for buffering 
incoming data, or hits an artificial resource limit intended to avoid 
resource starvation, then it must fail the WebSocket connection". It 
furthermore says that "Servers may close the WebSocket connection whenever 
desired". I don't see what else we can say with respect to handling DOS or 
overflow attacks.

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 5 Mar 2010 11:56:18 +0800

> For

A lot of accommodation has been made for this mythical brain-dead programmer.  Build an "idiot-proof" system and you'll just breed a more virulent strain of idiot.

And, as Greg continues to point out, easier is subjective.  For some value of easier, you also get injection attacks and other wonderful things.
 
> From the WebSocket point of view, it shouldn't fail
> for any size.

There's a marked difference between "designed for large files" and "suitable for large files".  I personally don't get the use case.  If I want to transfer a large file, it seems obvious to me that indirection is the right solution.  Put a URI in your WS message and let the other end fetch the data.

One reason you don't want to overload protocol use is that you might give WS traffic a higher priority; file transfers usually get lower priority.  Dumping them all in the same stream leaves you with no way to treat these things separately.
 
> I don't really see how it could be. Do you have an attack model in

Establishing a covert channel using leading zeroes on each frame might be fun.

Causing a buffer overrun might also be possible.  Assume that an implementation assumes that it can reject a frame when its size gets above a certain threshold.  That implementation supports 10K frames.  A bad implementation incorrectly assumes that the size is never going to take more than two octets, so they only read a few to start:

Byte[] octets = readSomeOctets();
Int size = 0;
for( int i = 0; octets[i] & 0x80 == 0x80; ++i) {
   size = size << 7 + (octets[i] & 0x7f);
   if (size > 10000) { throw error; }
}

(Yes, that's terrible code.)

> More importantly, I don't really see what our options are here. 

You can stop insisting that implementers are stupid or lazy.  While there's plenty of evidence to the contrary, there's also evidence that stupid or lazy implementations don't live long enough to thrive.

If you are assuming that people are idiots, it might be worth pointing out that calling read() on a TCP socket doesn't return an entire frame always.  I've seen code that assumes that one call to read() corresponds to a write() on the other end.  When two messages are written close enough together that they get bundled into the same packet, the second message is lost; when a write is too big to fit into the same packet, the tail of that message is lost.

How far do you want to go?

> I don't see what else we can say with respect to handling DOS
> or overflow attacks.

We can explain how someone might detect such an attack.  We can explain what they might do to mitigate the damage.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 05 Mar 2010 10:30:05 +0100

Jamie,

thanks for the clarification.  I do like your proposals ability
to batch, as batching is something that I've found very useful in
cometd - where semantically separate messages are still sent in the
same block to reduce overheads etc.     Websocket has less overheads,
but it would still be good have batching if it fell out of other
solutions.

I also hear what you say about keeping the default to be
non-delayed semantic boundary, so that 0x00 frames work
as they do.

So if bit 2 was "can delay" and bit 1 was "non-semantic-boundary",
then we could have the following frame types interpretations

 0x00   no delay message
 0x01   no delay fragment (or streamed data)
 0x02   can delay message
 0x03   can delay fragment (or stream data)

and some examples of frame sequences would be

 0x00 0x00 0x00 0x00   - messages as per current draft
 0x03 0x03 0x03 0x00   - message sent in 4 fragments
 0x02 0x02 0x03 0x02   - match of 3 message, of which the
                         third is in 2 fragments
 0x01 0x01 0x01 0x01   - streamed data

Note that intermediaries will either be just dump
relays and always do no delay, or they might do
some buffering/fragmentation and look at the
no-delay bit.

I think fewer intermediaries that might use
the "non-semantic-boundary" bit. But  can see uses for
it for doing debugging/logging or some other smart
routing.

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 05 Mar 2010 14:41:21 +0100

Ian,

I know you respond to a lot of feedback... but continually merging
it to be under a single "Feedback" thread is really not conducive
to discussion.   If feels more like you are handing down your
decisions from the mountain... and I think contributes somewhat
to the excessive heat on this list.

I'll response to some specific points in new subject threads.

regards

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 05 Mar 2010 15:01:18 +0100

Ian Hickson wrote:

> Please don't use frame types that aren't defined yet.

I thought that a sub protocol was free to use the frame type bytes.
The frame type byte is defined in the protocol, but with a meaning
allocated to only 1 bit - leaving the other 7 available for subprotocols.

> why any would fall into the second category.

Ian, I think you are not considering one of the most likely implementors
of subprotocols - browsers!

A subprotocol that provides one or more of: multiplexing, compression,
fragmentation, etc  would be most useful if implemented transparently
below the websocket API by the browsers themselves.

I certainly do not think that frame bytes should be exposed to the
js application, as they are primarily a transport concern and

> If you're exposing a

We don't want an infinite variety of websocket like protocols.
One of the huge challenges for websocket is to get the intermediaries
and other network appliances to work well with websocket.

Once that is done, then ideally any subprotocol variation can be
transparent to intermediaries and they will not need to be updated
as new and interesting uses for websocket are invented.

I think this is the acid test of what should be in the base
protocol and what should not.   Ie the base protocol should not
have any feature that can be subsequently implemented without
affecting intermediaries.

> types. The API doesn't expose them.

I think it is really important that we all come to an understanding of
how subprotocols are going to work.    It's not sufficient to boot
a whole bunch of requested features "to be implemented in subprotocols",
but then every time somebody proposes how a subprotocol could work
they are told that they can't do that because it can't be implemented
in the javascript API.

I know you think that application programmers will write absolutely
everything except the browser. Well good for you and I wish you well.

But can you also respect the position that many of us want to continue
to stand on the shoulders of giants and reuse software developed by
others.

I think it is entirely reasonable to expect that websocket extensions
and subprotocols will be implemented by browsers and server side
frameworks and provide a whole range of transport features hidden
from the application developer.

regards

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 05 Mar 2010 15:24:37 +0100

> feedback a few months ago.

I think this is short sited and you are disabling an existing
authentication mechanism without providing an alternative.

The upgrade request should be a standard HTTP request and
should be able to be authenticated as any other HTTP request.

>> But the problem with this is it assumes success and that a 101 is the 

I explained the problem in the next paragraph!

This "solution" will prevent the use of HTTP return status codes.
The server will only have the option of sending a 101 or closing the
connection.

Even if you don't accept the example potential uses I've outlined,
it strikes me as short sited to disable yet another standard mechanism
in the name of protecting against some phantom menace.

> that's probably the best description of the problem I've seen.

You are misrepresenting this.

Maciej described an attack vector that is easily addressed simply
by having a unique ID in the headers that the server needs to include
in the response. If the unique ID is generated by the browser, then
it will not even be in existence when any injection attack is formulated
and thus we are safe unless the ID is predicable.

You wanted the extra protection of spaces in the ID so that even if
the attacker somehow forsaw the future, they still could not inject
them.    This was accepted as probably overkill, but harmless.

The issue the random bytes after the GET request and after the
101 response are trying to handle is that of fast fail.   While
fast fail is a desirable attribute, there has been no rigorous
explanation of how this "solution" achieves that - nor was there
any discussion of alternative ways this could be achieved.

There has been no evidence or argument presented that
this "solution" actually provides fail fast semantics.

Yet this speculative "solution" has been added to the
spec at the cost of disabling the  possibility of using
the established HTTP response.

There are a significant number of response codes that may be
of value in the handshake, either now or in the future. While
there may be some problems with using some of them, those
issues should be identified and discussed - not wholesale
disabling of the possibility to use them.

Again, likely candidates of response codes that might
be meaningful in a handshake exchange include:

 301 Moved
 302 Found
 303 See Other
 305 Use Proxy
 400 Bad request
 401 Unauthorized
 402 Payment Required
 406 Not Acceptable
 414 Request URI too long
 415 Unsupported media Type
 500 Internal server error
 501 Not Implemented
 503 Service unavailable

If websockets wants to have some extra message exchange
after the 101 in order to address some security concern,
then it's entitled to do so.  But PLEASE do not try to
re-invent the Upgrade mechanism.  As I have said before,
if you think there is an security issue with HTTP upgrade,
then please raise it with HTTPbis and we can fix it for
all uses of Upgrade and not make websockets a special
case.

regards

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 5 Mar 2010 16:50:46 +0000

> authors will need to implement the more complicated framing.

What about "UTF-8 text" that incorrectly, but in reality, contains
0xFF bytes?  This can happen in most scripting languages
unintentionally, for example a script which reads lines from a file
that "should" be in UTF-8 (but really contains some 0xff bytes) and
sends them as messages, will result in a frame injection error.

That's even more likely for, say, reading a directory from a
filesystem where UTF-8 is the standard name encoding, and sending
those names in a message.  Except... in reality, it's easy for someone
to subvert it by putting something non-UTF-8 in there.

> browser sniffing, or on browsers ignoring Content-Location headers.

But that's what the subprotocol negotiation is for.  We should
encourage it's use, or spell out the circumstances in which it's
useful for an endpoint to speculatively use frame types that the other
end might not recognise.

> upgrading dedicated WebSocket-aware proxies would be relatively easy in 

It's not about what you/we want - it's what we'll get *anyway*,
because (a) other people will insist on firewalling your WebSocket
connections, to make sure you can only send non-pornographic text
messages or whatever, and (b) there are significant transport
optimisations possible using proxies, so they will be deployed.

I'm already finding that I have to write a client-side proxy - to
merge keepalives from multiple WebSocket conections, and reduce the
number of TCPs to a managable level for a slow network.  I anticipate
installing it as a site-wide TCP-intercepting proxy.  That's an
example of transport optimisation using a proxy - invisible to the
actual clients and servers, and they can't avoid it.  By the way, you
mentioned "author-friendly" elsewhere as a criterion.  This is very
author friendly because the endpoints don't know or care.

If I'm already looking at this at this early stage, then I won't be
the only one doing it when it's widely deployed.

> to make sure things are defined so that we are forwards-compatible.

I agree.

However, in this case it probably isn't forwards-compatible.

If I send the frame length 2^128, so that I can follow it with any
amount of binary data (effectively turning the connection into a byte
stream), I am sure that some future implementation, perhaps a proxy,
will treat it as zero and then break.

There is also the open question of what buffering and forwarding
behaviour is expected, both of proxies, and of receivers.  If I send a
large frame incrementally (i.e. by writing bytes), is it permitted to
buffer it with unlimited delay until the end of the frame, or must the
received bytes be passed on to the application, or the next hop in the
case of a proxy?

> probably premature to worry about this.

What does the API have to do with it?

Arman's question was not about web browsers as far as I can tell.  It
was about non-browser clients, using other languages.

> WebSocket itself, it's just going to cause problems for you when the API 

Are frame types free to use for applications and/or protocol
extensions now, or are they reserved for future WebSocket
specifications?

That needs to be made clear, because it's clear people are starting to
use them for protocol extensions.

> than the browser or the script?

Keepalives and timeouts (they go together) are used for three
equally critical things:

   - Keeping TCP open over NATs.

   - Detecting when the connection is broken at the server, so you can
     clean it up instead of running out of memory and crashing the
     server after a few days.  (The Tic-Tac-Toe demo has this problem.)

   - Detecting when the connection is broken at the client, so it
     can initiated another one.  (Interactive applications need this.)

To achieve all three, it's necessary to have keepalive messages sent
from both sides.  It is not enough to send a message from one side,
and rely on the TCP ACK coming back.  (That'll keep the NAT open, but
it won't allow both sides to detect broken connection in a reasonable time.)

This can be done as a "ping" style request+response, or each side can
transmit independently when it's not transmitted anything for a
keepalive interval.

For a given NAT timeout, the "ping" style uses more bandwidth than the
"independent" style if the client and server's broken-connection
timeouts are quite different.  This is because you must reduce the
keepalive interval by more due to the extra jitter from pinging, and
because it causes a message in both directions resulting in 3 TCP
packets, when 2 packets would be enough most of the time.

When the server and client's broken-connection timeout needs are the
same, then the "ping" style uses less bandwidth than "independent" style.

Note that applications like, say, Facebook and Gmail would be better
off with asymmetric broken-connection timeouts and therefore using the
"independent" style.

In some applications, keepalive bandwidth is actually the main
bandwidth consumer, and it gets quite expensive, so minimising it
is quite desirable.

What I conclude from this is that each application must be able to
decide for itself what keepalive strategy it's going to use, and it
must not be assumed that server-initiated pings are always a good choice.

> IMHO, *we are responsible* for helping authors make the right choice 

So why do you want to push things like message dispatch, flow control,
and cooperation between independent components on a page onto the
application authors, instead of helping to make the right choices with
those technical things?

> why any would fall into the second category.

The API is not relevant: because the discussion here is all about
implementations that aren't using the WebSocket API at all.

> If you're exposing a WebSocket server for non-browser UAs, just make
> the client send something in the handshake that opts in to using a
> different protocol (that looks very similar to WebSocket, but
> isn't), and then this whole spec becomes irrelevant, and you can use
> whatever mechanisms you want.

I'm guess the intention is they want to use WebSocket so that,
eventually, browsers will be able to speak to the services they've
deployed without having to rewrite those services.

And to reuse the inevitable WebSocket client APIs that will appear
quite soon for Java, .NET, Python, Perl etc.

Or maybe it's just a herd instinct, using the protocol even though
it's not well suited to their problem? ;-)

> "Content-Length: 0" field to the handshake to make this clearer?

Definitely not, because something will inevitably parse the following
data as the beginning of another HTTP response if you do that.

It might be interesting to see if "Content-Length: bogus" breaks
non-aware proxies and servers properly ;-)

> immediately to use the new handshake, if you haven't already.

Cannot do.  The moment the new one is shipped, it will break
communication with services using the old one.  So it cannot be a
routine security update: Browser security updates must not casually
break deployed, actively used web services.

> I don't really know of a way to prove this one way or the other, 

One argument in favour of length-delimiting is that forwarders
(including the APIs) don't have to repeatedly examine the data at all
to ensure it's integrity, e.g. for things like parsing and inserting
transport control messages (you mentioned them earlier).  Sentinals
force each component to examine every byte that it passes along, which
is a noticable cost under high load - especially for large messages.
(Modern OSes and hardware can move data between files and among
network ports without the CPU ever loading it into it's cache.)

That's a practical efficiency thing.  Google people have said that
SPDY experiments showed length-delimiting is generally faster too.

Regarding the chances of different failure modes, I think both failure
modes (wrong length, wrong UTF-8) are quite likely to occur, and not just
from scripts.

Among IETF protocols generally, sentinel-delimiting and
length-delimting are both common.

For example most of the text-using protocols are sentinel-delimited:
they look for LF or CRLF.  HTTP does this for headers.  SMTP looks for
"." on a line by itself.  Lots of IETF protocols do these things.

Length-delimiting is also very common.  HTTP manages to have both in
the same protocol.  IP obviously uses lengths.

So we can hardly use history as a precedent for deciding that one is
automatically better than the other.

However, I'm not aware of any IETF protocol which uses UTF-8 with an
0xff sentinel.  That's unusual and I doubt if it's been seriously
examined what security holes etc. that may encourage.

It *has* been observed that UTF-8 in general caused other security
holes, which is why the Unicode Consortium now specify that
non-minimal encodings must never be transmitted and should always be
rejected or treated as invalid.  For example, people were sending
UTF-8 encodings of "/" to bypass path filters and things like that.

Realistically, there probably is a class of security hole attributable
to the 0xff sentinel - and we probably don't know what it is yet.

> types. The API doesn't expose them.

Most of the clients and servers under discussion don't use the API.

However, there might be a popular misconception that using WebSocket
in this way is a good idea because it might work better with browsers
in future, or that it might work better over "The Internet" at large
especially the heavily firewalled and proxied internet.

Perhaps it needs to be made Very Clear that WebSocket with
application-specified frame types is a poor choice, because it won't
be future compatible with browsers, and it doesn't offer any
particular advantages on The Internet.

I'm as puzzled as you, Ian, as to why people are actually writing
custom clients and servers to use WebSocket with application-specific
binary frame types at the present time, instead of plain sockets!

The only reason I've seen given is that it's generally useful to build
up reusable, repurposable protocol stacks.  Maybe that is a good enough
reason, because it means building code that will be more useful in future.

> misimplementing string measurement, yes.

I think you're probably right, but I would vote for length-delimiting
on efficiency grounds anyway.

If you wanted to be more secure, it's possible to protect against both
issues at once: Require a leading byte length *and* 0xff sentinel following.

That won't make amateur code magically work with non-ASCII
characters, but it will protect better against injection attacks
taking advantage of amateur code.

> I don't really see how it could be. Do you have an attack model in mind?

How about sending a continuous stream of leading zeros, and causing
denial of service because the parser author didn't think to empty the
input buffer while parsing the number, or uses a quadratic parsing
(repeatedly start from the beginning on receiving more input) because
they expected the number to be a short byte sequence?

Actually I think we should support split-message chunks and then
define a fixed maximum chunk size of 2^31-1.  It wouldn't limit
message size, only chunk size.  It's much more likely to be treated
correctly by real implementations.

> data), and which is apparently harmless.

The something else is only "more reliable" if these things ever occur
in the wild.  If it's very clear from the start that there are no
leading zeros, nothing will need to parse it, and nothing will send it
because they get rejected when they first write code which does so.

> desired". I don't see what else we can say with respect to handling DOS or 

I'm concerned about client APIs, server APIs, and network proxies,
that may pass along data incrementally and have to keep a running
counter based on the frame length they have parsed (or sent).

That's how large length-delimited frames are likely to be handled in
practice inside frameworks.  And WS-parsing proxies must handle any
legitimate frame sized.

Both are quite likely to break when presented with a length which
doesn't fit into 2^32 or 2^64, depending on the variable size they use
internally.  By not specifying a maximum, I think we're encouraging
this as a failure point - in the same way that old HTTP agents crashed
when faced with a Content-Length >= 2^31 - because the authors didn't
know what was a sensible limit to implement, but practically they had
to use some fixed size type for it - a "bignum" implementation would
be ridiculous despite being the only formally correct thing to do.

That's why I'm thinking that an explicit maximum length of 2^31-1 per
chunk, combined with permitting messages composed of multiple chunks
(because that's useful for other reasons anyway), is the safest choice
coming to mind at the moment.  For the length-delimited frames, that is.
Sentinel-delimited frames can already be any length.

Note that this is *different* from running out of resource limits or
protecting against starvation, because this is just about counting, so
it doesn't come under the section which talks about resource limits -
and it shouldn't.

-- Jamie