Subject: Re: [hybi] Chunking of the message payload

From: Arman Djusupov <[email-masked]>
Date: Wed, 03 Mar 2010 15:38:36 +0200

Greg,

When one side opens multiple connections in order to send messages A, B, C and D (each of them over a separate TCP connection), the receiving side can read message A from its connection and keep messages B, C and D pending -- even when the receiving side has only one working thread. The TCP flow control mechanism will prevent the sending side from transmitting more data over the pending connections in a very effective manner; the other messages will either eventually get processed or they will timeout automatically by TCP.

There are cases that favor multiplexing, for example when each frame can be consumed separately with no need to wait for the next frame (e.g. sending frames for 4 different real-time video channels or multiplexing 4 files over the same connection), but multiplexing something like XML messages is quite harder.

With best regards,
Arman

Greg Wilkins wrote:
> Arman,
> 
> Not supporting multiplexing does not avoid these kinds
> of issues.  A client can still open multiple websockets
> so you can still have multiple partially received messages
> that need to be buffered and threading issues that need
> to be solved.
> 
> If done correctly, then multiplexing does not greatly
> increase the complexity of a server, as you simply
> move data structures that you have to have anyway, from
> being associated with a single TCP/IP connection to
> being associated with a virtual channel.
> 
> ... but I do agree that, multiplexing
> should be done by a websocket subprotocol and/or
> extension and not supported in the base protocol.
> But we do need to consider the design of a higher
> layer multiplexing protocol when considering the
> design of the framing mechanism.
> 
> cheers
> 
> 
> 
> Arman Djusupov wrote:
>> Having the experience of implementing two frame multiplexing protocols
>> and following their progress over the years, I have come to consider
>> frame multiplexing to be a flaw rather than a feature.
>>
>> When the receiving thread encounters chunks of the second message before
>> completely receiving the first message, it is forced to either buffer
>> both messages or span another thread to process the second message --
>> otherwise neither the first nor the second message can be completely
>> processed. Scale that upwards and you discover that the thread pool
>> easily ends up running short of threads; even under normal conditions
>> (not to mention planned DoS attacks) this leads to deadlocks. In an
>> attempt to avoid such situations, the protocol implementation is called
>> to include TCP-style timeouts and flow control handling, often resulting
>> in allocating significantly more threads and buffers than if these were
>> handled by the lower layer TCP stack. Unless frame multiplexing is the
>> main purpose of the protocol, I personally find it much preferable to
>> leave multiplexing to the network layer.
>>
>> In contrast to multiplexing, plain message framing (without
>> multiplexing) is practically already in the spec. The Web Socket framing
>> is quite simple and implementable even in scripting languages, which
>> IMHO is a key feature. All that implementers need is just a standardized
>> way to (a) determine whether the message serialized in a single frame,
>> and (b) when the message spans several frames, identify the last frame
>> of the message. The rest can be implemented by higher layer protocols by
>> encoding whatever info is required inside the frame.
>>
>> With best regards,
>> Arman
>>
>> Roberto Peon wrote:
>>> Once you get into that level of complexity, you start getting in to
>>> the territory that we're exploring with SPDY.
>>> -=R
>>>
>>> On Wed, Feb 24, 2010 at 3:11 PM, Greg Wilkins <[email-masked]> wrote:
>>>
>>>> Note that once we introduce message fragmentation, the next obvious
>>>> thing is fragment interleaving... ie can short messages overtake large
>>>> messages.... which leads on to channels... and multiplexing.
>>>>
>>>> So there is indeed a slippery slope of complexity when considering
>>>> these issues.  While there is an argument that the base protocol should
>>>> cover some/all of these issues, I think the momentum is currently for
>>>> keeping the base as simple as possible.
>>>>
>>>> So I think coming up with a layered approach for fragmentation,
>>>> interleaving, channels, multiplexing, etc. is the best way.
>>>>
>>>> Given that we have already seen multiple ways to fragment a message,
>>>> it certainly would be good to decide on just one to use in most
>>>> protocol extensions.  We certainly don't want to repeat the multiple
>>>> framing mistake already in websoocket.
>>>>
>>>> I'm working on a proposal that will cover the issues that I've
>>>> listed above and hope to post more details in a few days.
>>>>
>>>> cheers
>>>>
>>>>
>>>> Scott Ferguson wrote:
>>>>> Tim Bray wrote:
>>>>>> On Wed, Feb 24, 2010 at 9:44 AM, Scott Ferguson <[email-masked]>
>>>> wrote:
>>>>>>> We have the same binary chunking requirement (buffering the entire
>>>>>>> message
>>>>>>> is not an option), but we used the HTTP chunking system where a
>>>> 0-length
>>>>>>> chunk identifies the end of the message (and we basically ignore the
>>>>>>> identifier byte since it doesn't convey any useful information for
>>>>>>> us.) A
>>>>>>> standard solution for binary chunking would be fantastic.
>>>>>>>
>>>>>> The 0-length message expressed in two bytes, 00 FF used as a signal
>>>>>> that a multi-frame message is done, sure seems like the Simplest Thing
>>>>>> That Could Possibly Work.  I like it.   I might argue though, that we
>>>>>> should get Web Sockets shaken down before we start climbing up the
>>>>>> protocol stack.  -T
>>>>>>
>>>>> It should be the same layer because the main WebSocket entity is the
>>>>> message, and the spec defines the framing of that message. It's just
>>>>> that the binary encoding needs a chunking system because it's
>>>>> length-encoded, while the text encoding doesn't need chunking because
>>>>> it's FF-delimited. (I might be missing your point, though.)
>>>>>
>>>>> So the 0-length sequence here is 80 00, not 00 FF.
>>>>>
>>>>> -- Scott
>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> hybi mailing list
>>>>> [email-masked]
>>>>> https://www.ietf.org/mailman/listinfo/hybi
>>>> _______________________________________________
>>>> hybi mailing list
>>>> [email-masked]
>>>> https://www.ietf.org/mailman/listinfo/hybi
>>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> hybi mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/hybi
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> No virus found in this incoming message.
>>> Checked by AVG - www.avg.com Version: 9.0.733 / Virus Database:
>>> 271.1.1/2708 - Release Date: 02/24/10 21:34:00
>>>
>> _______________________________________________
>> hybi mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/hybi
> 
> 
>

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 03 Mar 2010 15:12:39 +0100

Arman Djusupov wrote:
> Greg,
> 
> When one side opens multiple connections in order to send messages A, B,
> C and D (each of them over a separate TCP connection), the receiving
> side can read message A from its connection and keep messages B, C and D
> pending -- even when the receiving side has only one working thread. The
> TCP flow control mechanism will prevent the sending side from
> transmitting more data over the pending connections in a very effective
> manner; the other messages will either eventually get processed or they
> will timeout automatically by TCP.

Arman,

Good point.   I think that applications that truly need that type
of flow control are exactly the ones that have good reason for using
multiple connections.

> There are cases that favor multiplexing, for example when each frame can
> be consumed separately with no need to wait for the next frame (e.g.
> sending frames for 4 different real-time video channels or multiplexing
> 4 files over the same connection), but multiplexing something like XML
> messages is quite harder.

It is interesting how often multiplexing is used without problems without
having per channel flow control.   HTTP appliances often multiplex onto
shared connections and this was not a significant problem until such things
as long polling started to be used.

I think this is a good argument for multi-plexing not being in the
base protocol - as a one size fits all solution would probably need
to address flow control.    However, having a simple multiplexing
sub protocol of websocket is likely to be useful for many applications,
and can fill the gap until something like SCTP is widely available.

cheers