Subject: [hybi] An _idea_ on framing

From: Micheil Smith <[email-masked]>
Date: Mon, 16 Aug 2010 18:27:03 +1000

Hi All,

I know that most of the framing for messages is starting to be set in some-what 
fixed state, although, I had a thought last night that may possibly alleviate the 
problems of having to allocate room for features X, Y, and Z in the initial framing 
protocol.

The proposal, as un-thought out as it is, is this:

	You have not just one content-length fragment, but two, say, two 8 bit integers. 
	The first of these is the meta-data length, the second is the actual content length.

	I'm not sure how these would be structured, perhaps something like:

       +-----------------+----------------------+--------------------+-------------------------+
        | 8 bits             | meta-length bits | 8 bits                 | content-length bits |
       +-----------------+----------------------+--------------------+-------------------------+
        | mete-length | meta data            | content-length | content                     |
       +-----------------+----------------------+--------------------+-------------------------+

This would allow us to later add or remove as much meta-data as we like, while staying
consistent the whole time. It'd also add the possibility the servers or clients may ignore 
meta-data, but that's another issue all together.

Take this as the view of an implementor of the protocol / an outsider to the process, this 
is just a thought, I'm not trying to necessarily start an argument / reopen a closed case, 
but it's just an idea nonetheless.

Let me know what you think,

Yours,
Micheil Smith
--
BrandedCode.com

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 16 Aug 2010 22:22:22 +1000

Michael,

your idea is the basis of the recent proposal from Ian Fette (as well
as some others).
If we wish to have meta data per frame, then I think having a
meta-data length within a total frame length is a good way of
achieving that.   However, I think we need to first agree that we need
meta data per frame.

cheers

On 16 August 2010 18:27, Micheil Smith <[email-masked]> wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: John Tamplin <[email-masked]>
Date: Mon, 16 Aug 2010 09:34:52 -0400

> achieving that.   However, I think we need to first agree that we need

My feeling is that if we are going to try and get a base protocol out and
implemented quickly, which means leaving out use cases we know we want to
eventually support because we don't want to try and get consensus on them
now, then we need to make sure we have sufficient extensibility in the base
protocol so that we can add to it later without having to change existing
implementations.  The most recent proposal from Ian Fette has the following
extensibility provisions:

   - if no extensions are negotiated in the handshake, they will not be
   present in the framing
   - extensions may exchange arbitrary information in the handshake
   - two bits in the frame header are available for later definition for
   things that need to be very low overhead
      - if we decide the best way to support compression is on a per-frame
      basis, it seems very likely we would want to use one of these bits to
      indicate the frame has been compressed
   - 9 opcodes are available for definition in a future version of the
   standard
   - 4 opcodes are available for private use, either for experiments for
   future standardization or for permanent use if the same entity controls the
   complete path and both endpoints
   - an arbitrary amount of data may be included in each frame, in a format
   to be determined when the first extension is defined
   - additional control frame types may be defined by an extension

Regarding per-frame extension data, the case seemed to have been made
earlier that if extension data was required to exist outside of the frame,
that would require additional state to be kept for each connection.  For
example, imagine the base frame was not extended for multiplexing, then you
would have to have a concept of the current channel on a muxed connection.
 Alternating frames between two channels would require sending a "change
channel" message between each payload frame.  If instead we have per-frame
extension data, then the channel number of a message can be included in that
frame itself.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 17 Aug 2010 00:16:50 +1000

On 16 August 2010 23:34, John Tamplin <[email-masked]> wrote:
> if no extensions are negotiated in the handshake, they will not be present
> in the framing
> extensions may exchange arbitrary information in the handshake
> two bits in the frame header are available for later definition for things
> that need to be very low overhead
>
> if we decide the best way to support compression is on a per-frame basis, it
> seems very likely we would want to use one of these bits to indicate the
> frame has been compressed
>
> 9 opcodes are available for definition in a future version of the standard
> 4 opcodes are available for private use, either for experiments for future
> standardization or for permanent use if the same entity controls the
> an arbitrary amount of data may be included in each frame, in a format to be
> determined when the first extension is defined
> additional control frame types may be defined by an extension

John,

nice summary.

> extension data, then the channel number of a message can be included in that

There was some earlier talk about an interleaved solution based on a
stateful connection,were an extension frame was sent before a normal
data frame. The extension frame could contain meta data, channel id's
etc.   However I think there was reasonable consensus that such a
mechanism is somewhat complex and fragile.

However the counter proposal that I've been making recently is not for
such a solution. Instead I'm saying that if you want to have ext data
plus normal data, then send a special op-code for that type of frame.
The ext data would only apply to the data in the same frame (or maybe
message with fragmentation).   Messages sent in normal data frames
would have no extension data and would be unaffected by previous data
frames.

This would mean that if you wanted to do something like multiplexing
then you could not use text-data or binary-data op-codes, and would
have to sent all data in an extension-data frame (with a dedicated
op-code) which would contain the channel ID and the binary/text data.
 The op codes defined for ping, pong and final frame could still be
used.

I currently am not convinced of the  benefit in adding extension data
to the frames for  text-data and binary-data op-codes since:

 + if the extension encodes text (eg compression), then the op code
would have to be changed anyway from text to binary.

 + if the data is changed in any way, they any intermediary that does
not know about the extension might break if it ignores the extension
data, but not the real data.  It is safer for intermediaries to ignore
all data for extensions that they do not understand.

With Ian Fettes current per frame extension mechanism, we need to rely
on the good sense of the extenders to avoid problematic combinations
of extension and op-code. That might be OK, but it would be even more
OK if somebody could come up with a good use-case of why text-data and
binary-data opcode frames need to be extended.

cheers