Subject: [hybi] Chunking of the message payload

From: Arman Djusupov <[email-masked]>
Date: Wed, 24 Feb 2010 18:52:39 +0200

Hello List,

As a starter, I would like to sincerely praise Web Socket for its simplicity and efficiency. It is an excellent protocol for full duplex communications.

I have a use case which doesn't seem to have been taken into account in the spec. When large binary messages of initially unknown size are getting streamed over a connection, the transport does not know the final size of the binary message when it starts serializing it and so it cannot encode the message's length at the start of the binary frame. In our implementation we handle such cases by buffering data until the buffer is full and then flushing it into a frame of specific length; subsequent data of the same message are sent in the following frames.

When the message is chunked in multiple frames in this manner, the receiving side needs to have some means of identifying the first and final frames that belong to the same message so that it knows when a message is completely received and from which frame the next message starts. We currently use 0x81 as the identifier of the first and subsequent frames, and 0x80 as the identifier of the last frame. When a message is contained in a single then it starts with 0x80.

The current version of the spec is flexible enough to support "proprietary" solutions such as ours. But we would like to keep our implementation interoperable with other vendors, and I believe that the case of binary messages that need to be chunked is not uncommon. There are cases when it's impossible (or undesired) for the sender to buffer the complete message irrespective of its size, as well as cases when chunking the message is simply more efficient than sending it in a single chunk.

It would be very nice if the spec could take into account message chunking and provide a standard solution (either ours or another) that ensures interop. Do you think it's possible to standardize message chunking?

With best regards,
Arman

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 24 Feb 2010 09:44:43 -0800

Arman Djusupov wrote:
> Hello List,
>
> As a starter, I would like to sincerely praise Web Socket for its 
> simplicity and efficiency. It is an excellent protocol for full duplex 
> communications.
>
> When the message is chunked in multiple frames in this manner, the 
> receiving side needs to have some means of identifying the first and 
> final frames that belong to the same message so that it knows when a 
> message is completely received and from which frame the next message 
> starts. We currently use 0x81 as the identifier of the first and 
> subsequent frames, and 0x80 as the identifier of the last frame. When 
> a message is contained in a single then it starts with 0x80.

>
> The current version of the spec is flexible enough to support 
> "proprietary" solutions such as ours. But we would like to keep our 
> implementation interoperable with other vendors, and I believe that 
> the case of binary messages that need to be chunked is not uncommon. 
> There are cases when it's impossible (or undesired) for the sender to 
> buffer the complete message irrespective of its size, as well as cases 
> when chunking the message is simply more efficient than sending it in 
> a single chunk.
>
We have the same binary chunking requirement (buffering the entire 
message is not an option), but we used the HTTP chunking system where a 
0-length chunk identifies the end of the message (and we basically 
ignore the identifier byte since it doesn't convey any useful 
information for us.) A standard solution for binary chunking would be 
fantastic.

-- Scott
>
> It would be very nice if the spec could take into account message 
> chunking and provide a standard solution (either ours or another) that 
> ensures interop. Do you think it's possible to standardize message 
> chunking?
>
> With best regards,
> Arman
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>
>

---

From: Tim Bray <[email-masked]>
Date: Wed, 24 Feb 2010 09:56:24 -0800

On Wed, Feb 24, 2010 at 9:44 AM, Scott Ferguson <[email-masked]> wrote:

> We have the same binary chunking requirement (buffering the entire message
> is not an option), but we used the HTTP chunking system where a 0-length
> chunk identifies the end of the message (and we basically ignore the
> identifier byte since it doesn't convey any useful information for us.) A
> standard solution for binary chunking would be fantastic.

The 0-length message expressed in two bytes, 00 FF used as a signal
that a multi-frame message is done, sure seems like the Simplest Thing
That Could Possibly Work.  I like it.   I might argue though, that we
should get Web Sockets shaken down before we start climbing up the
protocol stack.  -T

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 24 Feb 2010 10:57:29 -0800

> should get Web Sockets shaken down before we start climbing up the
It should be the same layer because the main WebSocket entity is the 
message, and the spec defines the framing of that message. It's just 
that the binary encoding needs a chunking system because it's 
length-encoded, while the text encoding doesn't need chunking because 
it's FF-delimited. (I might be missing your point, though.)

So the 0-length sequence here is 80 00, not 00 FF.

-- Scott

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 25 Feb 2010 10:11:21 +1100

Note that once we introduce message fragmentation, the next obvious
thing is fragment interleaving... ie can short messages overtake large
messages.... which leads on to channels... and multiplexing.

So there is indeed a slippery slope of complexity when considering
these issues.  While there is an argument that the base protocol should
cover some/all of these issues, I think the momentum is currently for
keeping the base as simple as possible.

So I think coming up with a layered approach for fragmentation,
interleaving, channels, multiplexing, etc. is the best way.

Given that we have already seen multiple ways to fragment a message,
it certainly would be good to decide on just one to use in most
protocol extensions.  We certainly don't want to repeat the multiple
framing mistake already in websoocket.

I'm working on a proposal that will cover the issues that I've
listed above and hope to post more details in a few days.

cheers

Scott Ferguson wrote:
>>> message

---

From: Roberto Peon <[email-masked]>
Date: Wed, 24 Feb 2010 15:13:29 -0800

Once you get into that level of complexity, you start getting in to
the territory that we're exploring with SPDY.
-=R

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 25 Feb 2010 11:01:43 +1100

> the territory that we're exploring with SPDY.

During the formation of hybi (but before SPDY was revealed), there
was considerable discussion with regards to should we be looking
to standardize websocket, or should we be looking for a new protocol
that would solve basic problems that websockets addresses, plus
some of the other pain points much discussed here.

I for one, advocated a new protocol and would have liked to have
seen us upgrading from HTTP to a better HTTP.    It is a great
pity that SPDY was not on the table at that point, as it would
have been a very good candidate (or at least an example of
what could be done) and perhaps we would now be aiming a little
higher up the semantic food chain.

However, that did not happen and the consensus at the last
IETF meeting was that efforts should focus on standardizing
websocket rather than try to pick another protocol.

But I do think there is a reasonable way forward.  If we can
address the basic concerns raised against websocket, yet keep
it a very simple message framing protocol, then it should be
possible to come up with a version of SPDY that runs over
websocket framing.

I would hope that the goodness of SPDY was mostly independent
of the exact framing used for each message and I would think
it a great test for websocket standard that it should be able
to be extended to support SPDY as a subprotocol without
any loss of capability or efficiency.

If SPDY were to run over websocket framing, then the internet
intermediaries would not need to be updated yet again to support
SPDY and the work done for websocket could be leveraged.

cheers

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 24 Feb 2010 16:36:29 -0800

> messages.... which leads on to channels... and multiplexing.
I'll argue there's a fundamental distinction between chunking and the 
higher-complexity features like channels and multiplexing. Chunking is 
fundamental to the problem that WebSockets is solving, while 
multiplexing is an additional complexity (and therefore best layered on 
top of WebSockets.)

WebSockets defines only two entities:

   1. a character sequence
   2. a byte sequence.

It doesn't define fragments, or channels or multiplexing. The aim of the 
spec is to encode those two entities on the wire, and hopefully make it 
straightforward to implement efficiently.

The character encoding is easy to implement because it's code delimited. 
You can encode and receive arbitrarily large sequences with a small 
fixed buffer.

The binary encoding is incomplete as written because you cannot encode 
an arbitrarily large byte sequence using a small fixed buffer, and 
requiring an arbitrarily large buffer is an unreasonable requirement.

Chunking does not add any additional entities to the WebSockets model. 
It merely defines the encoding of the existing entities in a way that's 
easy to implement.

In contrast, fragment interleaving, multiplexing, etc. all introduce 
additional entities (and complexity) to the model (i.e. they change the 
ontology), because they introduce new entities for fragments, channels, 
etc.  And so those protocol capabilities are more properly layered on 
top of the two basic sequences provided by WebSockets. But you still 
need to completely solve the problem for the two entities that 
WebSockets sets out to solve.

-- Scott

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 25 Feb 2010 12:33:26 +1100

> multiplexing is an additional complexity (and therefore best layered on

Scott,

I certainly can be swayed by such arguments - and I think having them
clearly stated (like you have) is a good way to ensure that adding
one feature does not start us down the slippery slop of complex features.

I'd add to your argument that having chunking supported in the base
protocol would allow a simplification of the length encoding for
length framed messages.    If chunking was provided, then we would
not need an arbitrary length frame size encoding.  2 bytes would be
sufficient to encode the length of a frame.

regards

PS.  I still think that the design of higher level features like
channels, multiplexing should leverage any chunking in the base
protocol and the design of this should be considered when
evaluating potential chunking solutions.

---

From: Martin Tyler <[email-masked]>
Date: Thu, 25 Feb 2010 08:38:04 +0000

> SPDY and the work done for websocket could be leveraged.

But I thought WebSocket itself wasn't intended to (or couldn't) run through
all intermediaries?

---

From: Arman Djusupov <[email-masked]>
Date: Thu, 25 Feb 2010 15:24:15 +0200

Having the experience of implementing two frame multiplexing protocols and following their progress over the years, I have come to consider frame multiplexing to be a flaw rather than a feature.

When the receiving thread encounters chunks of the second message before completely receiving the first message, it is forced to either buffer both messages or span another thread to process the second message -- otherwise neither the first nor the second message can be completely processed. Scale that upwards and you discover that the thread pool easily ends up running short of threads; even under normal conditions (not to mention planned DoS attacks) this leads to deadlocks. In an attempt to avoid such situations, the protocol implementation is called to include TCP-style timeouts and flow control handling, often resulting in allocating significantly more threads and buffers than if these were handled by the lower layer TCP stack. Unless frame multiplexing is the main purpose of the protocol, I personally find it much preferable to leave multiplexing to the network layer.

In contrast to multiplexing, plain message framing (without multiplexing) is practically already in the spec. The Web Socket framing is quite simple and implementable even in scripting languages, which IMHO is a key feature. All that implementers need is just a standardized way to (a) determine whether the message serialized in a single frame, and (b) when the message spans several frames, identify the last frame of the message. The rest can be implemented by higher layer protocols by encoding whatever info is required inside the frame.

With best regards,
Arman

Roberto Peon wrote:
> ------------------------------------------------------------------------
> 
> ------------------------------------------------------------------------
> 
> 
> No virus found in this incoming message.
> Checked by AVG - www.avg.com 
> Version: 9.0.733 / Virus Database: 271.1.1/2708 - Release Date: 02/24/10 21:34:00
>

---

From: Arman Djusupov <[email-masked]>
Date: Thu, 25 Feb 2010 17:52:36 +0200

Using a frame of 00 length to identify the end of the message would definitely work. But since the spec provides 0x80-0xFF as a binary frame identifier, it would be reasonable to make use of it (unless it is reserved for some purpose that I am not aware of). Another aspect is that using the first byte as a "last or only" frame identifier is very comfortable, because the receiving side can just allocate a buffer of the required size and read the complete message into it without having to check whether the next frame is of 00 length. This is a bit simpler (rather than looking for an empty frame) and also faster when a large amount of very short messages is being received.

With best regards,
Arman

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 25 Feb 2010 12:09:06 -0800

Arman Djusupov wrote:
> Using a frame of 00 length to identify the end of the message would 
> definitely work. But since the spec provides 0x80-0xFF as a binary 
> frame identifier, it would be reasonable to make use of it (unless it 
> is reserved for some purpose that I am not aware of). Another aspect 
> is that using the first byte as a "last or only" frame identifier is 
> very comfortable, because the receiving side can just allocate a 
> buffer of the required size and read the complete message into it 
> without having to check whether the next frame is of 00 length. This 
> is a bit simpler (rather than looking for an empty frame) and also 
> faster when a large amount of very short messages is being received.
Yes, that works with a number of advantages. I wasn't sure, though, if 
the frame identifier was intended to be used by applications, so I 
didn't want to step on its values.

It would be great if the encoding was something simple like:

  x80 xNN xNN - terminal chunk of length NNNN
  x81 xNN xNN - non-terminal chunk of length NNNN

And just reserve the rest of the x82-xff range and not allow 
implementations to use it.

(BTW, I've had the same experience as you with channel/multiplexing 
protocols and want to avoid them if at all possible because the 
complexity is generally more than the value gained.)

-- Scott
>>> simplicity and efficiency. It is an excellent protocol for full 
>>> duplex communications.
>>>
>>> subsequent frames, and 0x80 as the identifier of the last frame. 
>>> When a message is contained in a single then it starts with 0x80.
>>
>>>
>>> There are cases when it's impossible (or undesired) for the sender 
>>> to buffer the complete message irrespective of its size, as well as 
>>> cases when chunking the message is simply more efficient than 
>>> sending it in a single chunk.
>>>
>> message is not an option), but we used the HTTP chunking system where 
>> a 0-length chunk identifies the end of the message (and we basically 
>>> chunking and provide a standard solution (either ours or another) 
>>> that ensures interop. Do you think it's possible to standardize 
>>> message chunking?
>>>
>>> With best regards,
>>> Arman
>> Checked by AVG - www.avg.com Version: 9.0.733 / Virus Database: 
>> 271.1.1/2708 - Release Date: 02/24/10 21:34:00
>>
>
>
>
>

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 26 Feb 2010 11:42:10 +0000

Martin Tyler wrote:
> On Thu, Feb 25, 2010 at 12:01 AM, Greg Wilkins <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
> But I thought WebSocket itself wasn't intended to (or couldn't) run
> through all intermediaries? 

upgrade may fail in some intermediaries... in which case it falls back
to using CONNECT.  I think SPDY does the same thing.

Note the second draft of the SPDY spec is written to clearly
separate out their framing transport from their higher level protocol
stuff.   It's well worth reading before considering websocket
requirements and future.

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 26 Feb 2010 11:46:59 +0000

Arman,

Not supporting multiplexing does not avoid these kinds
of issues.  A client can still open multiple websockets
so you can still have multiple partially received messages
that need to be buffered and threading issues that need
to be solved.

If done correctly, then multiplexing does not greatly
increase the complexity of a server, as you simply
move data structures that you have to have anyway, from
being associated with a single TCP/IP connection to
being associated with a virtual channel.

... but I do agree that, multiplexing
should be done by a websocket subprotocol and/or
extension and not supported in the base protocol.
But we do need to consider the design of a higher
layer multiplexing protocol when considering the
design of the framing mechanism.

cheers

Arman Djusupov wrote:
> Having the experience of implementing two frame multiplexing protocols
> and following their progress over the years, I have come to consider
> frame multiplexing to be a flaw rather than a feature.
> 
> When the receiving thread encounters chunks of the second message before
> completely receiving the first message, it is forced to either buffer
> both messages or span another thread to process the second message --
> otherwise neither the first nor the second message can be completely
> processed. Scale that upwards and you discover that the thread pool
> easily ends up running short of threads; even under normal conditions
> (not to mention planned DoS attacks) this leads to deadlocks. In an
> attempt to avoid such situations, the protocol implementation is called
> to include TCP-style timeouts and flow control handling, often resulting
> in allocating significantly more threads and buffers than if these were
> handled by the lower layer TCP stack. Unless frame multiplexing is the
> main purpose of the protocol, I personally find it much preferable to
> leave multiplexing to the network layer.
> 
> In contrast to multiplexing, plain message framing (without
> multiplexing) is practically already in the spec. The Web Socket framing
> is quite simple and implementable even in scripting languages, which
> IMHO is a key feature. All that implementers need is just a standardized
> way to (a) determine whether the message serialized in a single frame,
> and (b) when the message spans several frames, identify the last frame
> of the message. The rest can be implemented by higher layer protocols by
> encoding whatever info is required inside the frame.
> 
> With best regards,
> Arman
> 
> Roberto Peon wrote: