Subject: [hybi] Framing Proposal Take VII

From: John Tamplin <[email-masked]>
Date: Thu, 19 Aug 2010 15:14:31 -0400

Here is my latest proposal written up more formally

   0                   1                   2                   3

>    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

>   +-+-+---+-------+-+-------------+-------------------------------+

>   |M|C| R | opcode|R| Payload len |    Extended payload length    |

>   |O|M| S |  (4)  |S|     (7)     |             (64)              |

>   |R|L| V |       |V|             |     (if payload len==127)     |
  |E|P| 1 |       |2|             |                               |

>   +-+-+---+-------+-+-------------+ - - - - - - - - - - - - - - - +

>   |                                                               |

>   + - - - - - - - - - - - - - - - +-------------------------------+

>   |                               |                               |

>   +-------------------------------+ - - - - - - - - - - - - - - - +

>   |             Complete message length (64) (if CMLP=1)          |

>   + - - - - - - - - - - - - - - - +-------------------------------+

>   |                               |          Payload data         |

>   +-------------------------------+ - - - - - - - - - - - - - - - +

>   |                                                               |

>   +---------------------------------------------------------------+

    ws-frame            = frame-type

>                           frame-reserved

>                           frame-payload-length

>                           frame-complete-message-length

>                           application-data

>     frame-type          = type-more-frames

>                           type-msglen-present

>                           type-extension

>                           type-reserved

>                           type-opcode

    type-initial        = %x0 ; not initial frame

>                         = %x1 ; initial frame

    type-more-frames    = %x0 ; final frame of message

>                         = %x1 ; more frames of this message follow

    type-reserved       = %x0 ; 2 bits

    type-opcode         = %x0 ; continuation frame
                        / %x1 ; control frame

>                         / %x2 ; text data frame

>                         / %x3 ; binary data frame (blob)

>                         / %x4-%xB ; reserved

>                         / %xC-%xF ; user defined

    frame-reserved      = %x0 ; 1 bit

    frame-payload-length = %x00-7E

>                          / %x7F frame-extended-payload-length

    frame-extended-payload-length = %x000000000000007F-7FFFFFFFFFFFFFFF ;
payload length including all fields after this field

    frame-complete-message-length = %x0000000000000001-7FFFFFFFFFFFFFFF ;
complete message length. only application data part will be counted in (iff
CMLP==1)

>                                   / ; none (otherwise)

>     payload-data        = text-data (if type-opcode=%2)

>                         / binary-data (if type-opcode=%3)

>                         / control-data  (if type-opcode=%0)
                        / user-defined-data (if type-opcode in %xC-%xF)

    text-data           = *( %x00-FF )

    binary-data         = *( %x00-FF )

    user-defined-data   = *( %x00-FF )

     control-data        = control-type control-info

    control-type        = %x00-FF ; type of control frame (details TBD)

    control-info        = *( %x00-FF ) ; additional data of control frame

Clarifications:

   - complete-message-length SHOULD NOT be present in a single-frame
   message.
   - Senders may send a message in one or more frames.  They SHOULD include
   a complete-message-length in the earliest fragment of a fragmented message
   where it is known.  They MAY include an estimate of the complete message
   length and MAY provide new complete message length values on subsequent
   frames.  The receiver MUST be prepared to accept fragmented messages, even
   when the original sender sent it as a single frame.
   - All non-initial frames of a message MUST use the continuation frame
   opcode.
   - Intermediaries MAY fragment a single frame into multiple frames and MAY
   combine fragmented messages into a single frame.  An intermediary that
   fragments a single-frame message SHOULD include a complete-message-length on
   the first frame of the fragmented message.  An intermediary which combines
   fragmented messages into a single-frame message SHOULD remove any
   complete-message-length that is present.
   - Senders MAY use the frame-extended-payload-length format even if the
   length being represented is 126 octets or less.
   - Future specifications may define extensions to this protocol.  Such
   extensions will be present only if negotiated during the handshake, so an
   endpoint that refuses to negotiate any extension will never see anything
   beyond this spec.  However, intermediaries may not be able to participate in
   the extension negotiation, and must be prepared to deal with frames
   containing extensions.  Such an intermediary MUST NOT attempt to interpret,
   modify, or fragment/defragment any frame with frame-reserved or
   type-reserved set to non-zero values, or using any reserved/user-defined
   opcode, except as provided in a future specification defining such
   extensions or by private agreement.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: "Anne van Kesteren" <[email-masked]>
Date: Thu, 19 Aug 2010 21:30:11 +0200

I much appreciate you try to keep complexity down. Not entirely sure yet  
whether I like it and whether it is enough, but it seems reasonable.

>     type-initial        = %x0 ; not initial frame

This is not referenced from anywhere. It seems type-more-frames addresses  
this.

>     type-opcode         = %x0 ; continuation frame

Why do we need continuation if we have type-more-frames?

>                         / %x1 ; control frame

What is this for?

Also, Ian's draft has a closing frame that was deemed necessary at some  
point. That is no longer needed?

>    - complete-message-length SHOULD NOT be present in a single-frame

Not sure it makes sense to enforce this so strongly. It's just an  
optimization after all. And does having complete-message-length really  
help if it can be so inaccurate? I.e. it can be an estimate that is  
completely wrong. How does that help?

-- 
Anne van Kesteren
http://annevankesteren.nl/

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 19 Aug 2010 13:09:18 -0700

I am a bit worried about _totally_ punting extensions. You essentially have
a bit reserved for use for extensions, I think it would be better if we
labeled it as such now rather than calling it RSV, and simply state that the
presence of this bit indicates that this frame either contains extension
data, or contains data modified by extensions, and as such an intermediary
not aware of the extensions that have been negotiated or not understanding
the extensions having been negotiated should not attempt to modify this
frame.

I would suggest that if this working group can agree on some form of base
proposal, the next thing we should turn to is better defining extensibility
and what exactly happens when the extension bit is sent. E.g. how do
multiple extensions interact, for an extension frame, how do we indicate the
length of the extension data, etc.

-Ian

On Thu, Aug 19, 2010 at 12:14 PM, John Tamplin <[email-masked]> wrote:

> complete message length. only application data part will be counted in
> (iff CMLP==1)
>
>>                                   / ; none (otherwise)
>
>    - Senders may send a message in one or more frames.  They SHOULD
>    include a complete-message-length in the earliest fragment of a fragmented
>    message where it is known.  They MAY include an estimate of the complete
>    message length and MAY provide new complete message length values on
>    subsequent frames.  The receiver MUST be prepared to accept fragmented
>    messages, even when the original sender sent it as a single frame.
>    - Intermediaries MAY fragment a single frame into multiple frames and
>    MAY combine fragmented messages into a single frame.  An intermediary that
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 19 Aug 2010 13:10:23 -0700

> Here is my latest proposal written up more formally

This is workable.

The 10 byte overhead for a tweet (~ 350 byte payload) is more than it 
needs to be, but it's fine. A fixed 8 or 9 byte header would be an 
improvement because 127 bytes is so short it doesn't even fit a tweet 
(and therefore only works for specialized applications), and using the 
len=127 technique wastes a byte for every message over 127. But neither 
are major problems. It's not a particularly efficient encoding, but it's 
workable.

Also, the ping/pong/close should be taken from the reserved opcodes, not 
part of the control group. That way, non-extension implementations don't 
need to deal with the control group.

(And type-initial is an orphan production?)

-- Scott

>     frame-extended-payload-length = 
> %x000000000000007F-7FFFFFFFFFFFFFFF ; payload length including all 
> fields after this field
>
>
>     frame-complete-message-length = 
> %x0000000000000001-7FFFFFFFFFFFFFFF ; complete message length. only 
> application data part will be counted in (iff CMLP==1)
>
>                                   / ; none (otherwise)
>
>     * complete-message-length SHOULD NOT be present in a single-frame
>       message.
>     * Senders may send a message in one or more frames.  They SHOULD
>       include a complete-message-length in the earliest fragment of a
>       fragmented message where it is known.  They MAY include an
>       estimate of the complete message length and MAY provide new
>       complete message length values on subsequent frames.  The
>       receiver MUST be prepared to accept fragmented messages, even
>     * All non-initial frames of a message MUST use the continuation
>       frame opcode.
>     * Intermediaries MAY fragment a single frame into multiple frames
>       and MAY combine fragmented messages into a single frame.  An
>       intermediary that fragments a single-frame message SHOULD
>       include a complete-message-length on the first frame of the
>       fragmented message.  An intermediary which combines fragmented
>       messages into a single-frame message SHOULD remove any
>     * Senders MAY use the frame-extended-payload-length format even if
>       the length being represented is 126 octets or less.
>     * Future specifications may define extensions to this protocol.
>        Such extensions will be present only if negotiated during the
>       handshake, so an endpoint that refuses to negotiate any
>       extension will never see anything beyond this spec.  However,
>       intermediaries may not be able to participate in the extension
>       negotiation, and must be prepared to deal with frames containing
>       extensions.  Such an intermediary MUST NOT attempt to interpret,
>       type-reserved set to non-zero values, or using any
>       reserved/user-defined opcode, except as provided in a future
>       specification defining such extensions or by private agreement.
>
> -- 
> John A. Tamplin
> ------------------------------------------------------------------------
>

---

From: Salvatore Loreto <[email-masked]>
Date: Thu, 19 Aug 2010 22:18:49 +0200

(commenting as individual)

I like the proposal, is becoming simpler and leave multiple extension 
hooks  so that we can extend it without having to change existing 
implementations of the base protocol.

more comments in line

> Here is my latest proposal written up more formally
type extension is not present in the figure. Where 2 & 3 are RSV1.
I would prefer, as Ian commented in another mail, to explicitly label a 
bit as extension
so that once we have agreed on the base frame the wg can start right 
away to defining extensions

>                         / %x3 ; binary data frame (blob)
it would good have also a closing frame
and some sort of keepalive or a ping and a pong frame as proposed in 
previous proposals.

>                         / control-data  (if type-opcode=%0)
for control-data type-opcode should be %1.
>     * complete-message-length SHOULD NOT be present in a single-frame
+1.
Actually in IMO it would be good define complete-message-length as an 
extension instead of defining it in the basic frame.
but I can live with it in the base frame if the wg feel that is better 
having it here.

-- 
Salvatore Loreto
www.sloreto.com

---

From: John Tamplin <[email-masked]>
Date: Thu, 19 Aug 2010 16:26:19 -0400

On Thu, Aug 19, 2010 at 4:10 PM, Scott Ferguson <[email-masked]> wrote:

> The 10 byte overhead for a tweet (~ 350 byte payload) is more than it needs
> to be, but it's fine. A fixed 8 or 9 byte header would be an improvement
> because 127 bytes is so short it doesn't even fit a tweet (and therefore
> only works for specialized applications), and using the len=127 technique
> wastes a byte for every message over 127. But neither are major problems.
> It's not a particularly efficient encoding, but it's workable.
>

I would think tweets would be highly compressible, and once we have
compression in the spec they will likely all fit in the short frames.  I
would expect that many WebSocket apps are going to be highly interactive,
sending messages on each input, or things like games.  Therefore, I suspect
Wave and GWT Quake will in fact be representative, and those do make heavy
use of short frames as previously detailed.

As the frames get larger, the efficiency of encoding those frames becomes
less important -- I am most concerned, efficiency-wise, about encoding small
messages with minimal overhead.

> need to deal with the control group.

There are a limited number of opcodes available (especially if we decide to
use opcodes to encode extensions rather than separate metadata), so I would
strongly prefer to keep them in the control frame where there is more room
to define them how we like without worrying about every bit.  That said, if
the consensus was that they should be opcodes I wouldn't object strongly to
it (which would also mean that we wouldn't even define a control opcode in
the base protocol).

> (And type-initial is an orphan production?)

Yes, sorry.

---

From: John Tamplin <[email-masked]>
Date: Thu, 19 Aug 2010 16:26:57 -0400

> This is not referenced from anywhere. It seems type-more-frames addresses

Correct, I failed to remove it when editing Ian's proposal.

> Why do we need continuation if we have type-more-frames?

What should happen if you receive a sequence like this:

MORE=1, Opcode=Text, ...
MORE=0, Opcode=Binary, ...

It doesn't seem useful to allow switching the payload type between
fragments.  We could instead specify that they must all have the same
opcode, if that is preferred.

> point. That is no longer needed?

The control frames are used to define any out-of-band communication that may
be needed.  It needs to be nailed down before the base protocol is
finalized, but I would expect to have the following:

   - close connection
   - ping
   - pong
   - with some reserved for private use and the rest reserved for future
   definition

I don't think we need flow control in the base protocol, but I would expect
it to be eventually defined here.

 Clarifications:

> optimization after all.

I wouldn't mind removing this statement, though it would seem silly to
include in a single-frame message since the frame length is the message
length.

> And does having complete-message-length really help if it can be so
> inaccurate? I.e. it can be an estimate that is completely wrong. How does
> that help?

It was previously argued that having an estimate was useful to allow
progress indicators.  The receiver already has to be prepared to reallocate
buffers if the complete message length is not included, so worst case they
still do so when they find the message exceeds what was allocated.

---

From: James Graham <[email-masked]>
Date: Thu, 19 Aug 2010 22:31:07 +0200 (CEST)

>                         / %x3 ; binary data frame (blob)

So there are 12 opcodes of which 8 are free? This doesn't seem like very 
many. Of course it is possible to make an opcode that says "look elsewhere 
for more opcodes".

I don't really understand how user-defined opcodes will work in practice. 
If two clients try to use %xC to mean different things we have an instant 
interoperability problem. Are they supposed to be used only with 
handshake headers to opt-in to extensions?

>  *  complete-message-length SHOULD NOT be present in a single-frame message.
>  *  Senders may send a message in one or more frames.  They SHOULD include a complete-message-length in the earliest
>     fragment of a fragmented message where it is known.  They MAY include an estimate of the complete message length and
>     MAY provide new complete message length values on subsequent frames.  The receiver MUST be prepared to accept
>     fragmented messages, even when the original sender sent it as a single frame.

I think the complete message length stuff is adding a lot of complexity 
for very little gain. For example it seems likely that some clients will 
break if the complete message length is sent multiple times with different 
values. Also, what happens if it is sent with a value shorter than the 
amount of data already recieved? Ignoring multiplexing, we may assume that 
fragmented messages will typically be sent where the size of the message 
is not known a-priori. In that case this feature is useless. If it it 
useful in the multiplexing case we could add it as part of that.

>  *  All non-initial frames of a message MUST use the continuation frame opcode.
>  *  Intermediaries MAY fragment a single frame into multiple frames and MAY combine fragmented messages into a single
>     frame.  An intermediary that fragments a single-frame message SHOULD include a complete-message-length on the first
>     frame of the fragmented message.  An intermediary which combines fragmented messages into a single-frame message
>     SHOULD remove any complete-message-length that is present.

There seems to be a clear deployment failure mode here. If we assume that 
clients will typically not fragment, at least in the case of simple text 
messages, then it is quite likely that some servers will be developed with 
buggy handling of fragments. If an intermediary is later deployed that 
fragments messages, this will break previously working services.

>  *  Senders MAY use the frame-extended-payload-length format even if the length being represented is 126 octets or less.

I am really unconvinced that it is not simpler just to have a variable 
length length rather than have two special cases.

---

From: John Tamplin <[email-masked]>
Date: Thu, 19 Aug 2010 16:32:47 -0400

On Thu, Aug 19, 2010 at 4:18 PM, Salvatore Loreto <
[email-masked]> wrote:

> I would prefer, as Ian commented in another mail, to explicitly label a bit
> as extension
> so that once we have agreed on the base frame the wg can start right away
> to defining extensions
>

What if we decide that we prefer to define extensions by new opcodes?  Would
we later redefine that bit to be part of the opcode field?  Would that
confuse implementations of the initial spec that expect it to indicate
extension data is present?  Or would we just waste that bit?

Until we make that decision, it seems safest to declare it reserved and that
initial implementations must do nothing but forward the frame unmodified or
drop it if any of the reserved bits are non-zero or any reserved opcode is
used.

> and some sort of keepalive or a ping and a pong frame as proposed in

That is in the control-type field -- I wasn't sure we had decided exactly
what control frames were included, so I left it as in Ian's proposal.

>  for control-data type-opcode should be %1.

right, thanks.

> but I can live with it in the base frame if the wg feel that is better

If we are supporting fragmentation in the base protocol, it seems that an
intermediary that fragments the message should be able to provide the
receiver the original message length, without requiring that recipient
support extensions.

---

From: John Tamplin <[email-masked]>
Date: Thu, 19 Aug 2010 16:52:38 -0400

> many. Of course it is possible to make an opcode that says "look elsewhere

There are also three adjacent bits which could be assigned to the opcode if
we need them.  As you mention, there could also be a ExtendedOpcode opcode,
which says grab the first n bytes of the payload as the real opcode.

If we go with extension metadata (using one of the reserved bits to indicate
its presence), then I am not sure how many more opcodes we would need.  If
we go with opcodes as the primary extension mechanism, then we would likely
need more, but we have the reserved bits we can allocate to add more.

> interoperability problem. Are they supposed to be used only with handshake
> headers to opt-in to extensions?
>

Private-use opcodes can only be used by external agreement -- think of it
like private IP address space.  As an example, Google will likely implement
multiplexing before it is standardized due to internal needs and to get
implementation experience to guide the ultimate standard.  We could define
some private-use opcodes to identify MUX frames, and then use them
internally without having to worry about interference from
standards-conforming implementation.

> I think the complete message length stuff is adding a lot of complexity for
>> very little gain. For example it seems likely that some clients will break
>> if the complete message length is sent multiple times with different values.
>> Also, what happens if it is sent with a value shorter than the amount of
>> data already recieved? Ignoring multiplexing, we may assume that fragmented
>> messages will typically be sent where the size of the message is not known
>> a-priori. In that case this feature is useless. If it it useful in the
>> multiplexing case we could add it as part of that.
>
>
S----A=====B----R

The sender sends a single large frame to R.  It passes through a third-party
network, entering at A.  A fragments the message (and may
mutliplex/demultiplex it etc internally), and the fragments eventually leave
that network via B (which doesn't combine fragments due to memory/latency
constraints).  If we don't support CML in the base protocol, R will have to
reallocate/copy its buffer, even though the size was known at S.  If A can
add the CML on the first fragment, R still gets the benefit of being able to
allocate the receive buffer up front.

As mentioned in another message, it was argued that having an estimate was
useful for indicating progress to the end user, and that a reasonable
estimate may save reallocs on the receiver even if it isn't perfect.

All that said, personally I would be fine with leaving CMLP as another
reserved bit, as long as that doesn't bring back the objections we had
earlier on this list about fragmentation complicating receiver
implementations.

>>  *  Intermediaries MAY fragment a single frame into multiple frames and
>> MAY combine fragmented messages into a single
>>    frame.  An intermediary that fragments a single-frame message SHOULD
>> include a complete-message-length on the first
>>    frame of the fragmented message.  An intermediary which combines
>> fragmented messages into a single-frame message

There is only so much you can do to make sure people implement the spec
properly.  Maybe we can provide a testing proxy which fragments messages and
exercises the various corner cases.  Given that supporting fragmentation is
an accepted requirement, I am not sure what else we could do about it.

>   *  Senders MAY use the frame-extended-payload-length format even if the

I feel very strongly about having a special-case for short messages keeping
a small header, which the variable-length length also gives (though at the
expense of one bit over this proposal).  I gave arguments for this approach
over the variable-length length approach in another message (basically you
have 9 steps of lengths rather than 2, with little gain for the complexity).
 However, if the consensus is that a variable length-length is preferred
(which it didn't seem to be from earlier discussions), then I wouldn't
object -- we could just merge RSV2 into the length field and use the
high-bit means more length bytes approach (outlawing leading 0x80's and
restricting the length to 9 bytes).

---

From: Salvatore Loreto <[email-masked]>
Date: Thu, 19 Aug 2010 22:57:29 +0200

(as individual)

On 8/19/10 10:32 PM, John Tamplin wrote:
> On Thu, Aug 19, 2010 at 4:18 PM, Salvatore Loreto 
> <[email-masked] <mailto:[email-masked]>> 
> wrote:
>
>     I would prefer, as Ian commented in another mail, to explicitly
>     label a bit as extension
>     so that once we have agreed on the base frame the wg can start
>     right away to defining extensions
>
>
> What if we decide that we prefer to define extensions by new opcodes? 
>  Would we later redefine that bit to be part of the opcode field? 
>  Would that confuse implementations of the initial spec that expect it 
> to indicate extension data is present?  Or would we just waste that bit?
>
> Until we make that decision, it seems safest to declare it reserved 
> and that initial implementations must do nothing but forward the frame 
> unmodified or drop it if any of the reserved bits are non-zero or any 
> reserved opcode is used.

I see your reason now and I agree that it would be better let it as 
reserved in the based protocol spec.
Actually this is even a better way to keep distinct the 
discussion/design on extensibility from the basic protocol.
IMO extensibility should be defined in a distinct draft from the basic 
protocol at least for the time being, even if it would be better to 
start the discussion
on it right after we have reached an agreement on the base frame.

>     and some sort of keepalive or a ping and a pong frame as proposed
>     in previous proposals.
>
>
> That is in the control-type field -- I wasn't sure we had decided 
> exactly what control frames were included, so I left it as in Ian's 
> proposal.
that was not so clear for me.
using control frame for closing, keepalive (or ping/pong) and other 
similar stuff make sense to me.

>     Actually in IMO it would be good define complete-message-length as
>     an extension instead of defining it in the basic frame.
>     but I can live with it in the base frame if the wg feel that is
>     better having it here.
>
>
> If we are supporting fragmentation in the base protocol, it seems that 
> an intermediary that fragments the message should be able to provide 
> the receiver the original message length, without requiring that 
> recipient support extensions.
explained in this way starts to make more sense to have 
complete-message-length in the basic frame.

thanks for the clarifications
/Sal

-- 
Salvatore Loreto
www.sloreto.com

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 19 Aug 2010 15:57:59 -0700

John Tamplin wrote:
> On Thu, Aug 19, 2010 at 4:10 PM, Scott Ferguson <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     The 10 byte overhead for a tweet (~ 350 byte payload) is more than
>     it needs to be, but it's fine. A fixed 8 or 9 byte header would be
>     an improvement because 127 bytes is so short it doesn't even fit a
>     tweet (and therefore only works for specialized applications), and
>     using the len=127 technique wastes a byte for every message over
>     127. But neither are major problems. It's not a particularly
>     efficient encoding, but it's workable.
>
>
> compression in the spec they will likely all fit in the short frames.  
> I would expect that many WebSocket apps are going to be highly 
> interactive, sending messages on each input, or things like games. 
>  Therefore, I suspect Wave and GWT Quake will in fact be 
> representative, and those do make heavy use of short frames as 
> previously detailed.

The simple Wave over XMPP example stanzas are ~400 bytes, by the way. 
Even the trivial ack example is 118 bytes. The conversation reply blip 
example is 119 bytes (without XMPP overhead and with minimal metadata). 
A tweet is 350 bytes (including its metadata). A Wave implementation 
would probably be better off using the 10-byte frame and ignoring the 
short frame entirely. The 127 byte length just doesn't help those 
applications.

As for compression fitting in 127 bytes: it's probably not worth 
shifting a compressed buffer just to save a few bytes. It doesn't matter 
if a compressed buffer can theoretically fit into a short frame if it's 
not worth the buffer copy overhead or the code complexity overhead to do 
the packing.

I'll give you Quake, since those packets will be handcrafted binary, and 
can be designed to fit into 127 bytes. But the more typical JSON/XML are 
dynamic in length (and tend to accumulate metadata without the authors 
noticing any overrun beyond 127 bytes.)

We'll see. The short frame sounds impressive (we're squeezing every 
byte!), and even if it doesn't work as well as advertised in practice, 
the 10-byte overhead is fine.

-- Scott

>
> As the frames get larger, the efficiency of encoding those frames 
> becomes less important -- I am most concerned, efficiency-wise, about 
> encoding small messages with minimal overhead.
>  
>
>     Also, the ping/pong/close should be taken from the reserved
>     opcodes, not part of the control group. That way, non-extension
>     implementations don't need to deal with the control group.
>
>
> There are a limited number of opcodes available (especially if we 
> decide to use opcodes to encode extensions rather than separate 
> metadata), so I would strongly prefer to keep them in the control 
> frame where there is more room to define them how we like without 
> worrying about every bit.  That said, if the consensus was that they 
> should be opcodes I wouldn't object strongly to it (which would also 
> mean that we wouldn't even define a control opcode in the base protocol).
>  
>

---

From: John Tamplin <[email-masked]>
Date: Thu, 19 Aug 2010 19:09:38 -0400

On Thu, Aug 19, 2010 at 6:57 PM, Scott Ferguson <[email-masked]> wrote:

> The simple Wave over XMPP example stanzas are ~400 bytes, by the way. Even
> the trivial ack example is 118 bytes. The conversation reply blip example is
> 119 bytes (without XMPP overhead and with minimal metadata). A tweet is 350
> bytes (including its metadata). A Wave implementation would probably be
> better off using the 10-byte frame and ignoring the short frame entirely.
> The 127 byte length just doesn't help those applications.
>

I published the results of a compression experiment in April, giving the
packet size percentiles of uncompressed payload and various compression
forms.  As mentioned, a significant portion of the Wave over WebSocket
frames were under 127 bytes and when compressed an overwhelming majority
were.  If you would like the payload size information from the trace I can
dig it up tonight.

> As for compression fitting in 127 bytes: it's probably not worth shifting a
> compressed buffer just to save a few bytes. It doesn't matter if a
> compressed buffer can theoretically fit into a short frame if it's not worth
> the buffer copy overhead or the code complexity overhead to do the packing.
>

If you start writing at byte 10 of the header (or 18 if you have a complete
message length), and then when you flush the buffer you write a 2/10/18 byte
header before it in the buffer, I don't know that is too much more
complexity to take advantage of it.  Or you could use scatter/gather, or
multiple writes.

> noticing any overrun beyond 127 bytes.)

The GWT Quake example just used the standard Quake2 packet structure
designed for Ethernet, other than having to encode binary bytes as Unicode
characters (so bytes 0x80-0xFF were encoded as two UTF8 bytes in the
WebSocket payload) because binary wasn't available.

> We'll see. The short frame sounds impressive (we're squeezing every byte!),
> and even if it doesn't work as well as advertised in practice, the 10-byte
> overhead is fine.
>

So if I understand your position, you are skeptical of the benefits for
small frames, but you have no objection as proposed?

>> There are a limited number of opcodes available (especially if we decide
>> to use opcodes to encode extensions rather than separate metadata), so I
>> would strongly prefer to keep them in the control frame where there is more
>> room to define them how we like without worrying about every bit.  That
>> said, if the consensus was that they should be opcodes I wouldn't object
>> strongly to it (which would also mean that we wouldn't even define a control
>> opcode in the base protocol).
>>
>
Does not response to this mean you agree with the rationale given?

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 19 Aug 2010 18:54:34 -0700

John Tamplin wrote:
> On Thu, Aug 19, 2010 at 6:57 PM, Scott Ferguson <[email-masked] 
>     The simple Wave over XMPP example stanzas are ~400 bytes, by the
>     way. Even the trivial ack example is 118 bytes. The conversation
>     reply blip example is 119 bytes (without XMPP overhead and with
>     minimal metadata). A tweet is 350 bytes (including its metadata).
>     A Wave implementation would probably be better off using the
>     10-byte frame and ignoring the short frame entirely. The 127 byte
>     length just doesn't help those applications.
>
>
> I published the results of a compression experiment in April, giving 
> the packet size percentiles of uncompressed payload and various 
> compression forms.  As mentioned, a significant portion of the Wave 
> over WebSocket frames were under 127 bytes and when compressed an 
> overwhelming majority were.  If you would like the payload size 
> information from the trace I can dig it up tonight.

No need.  That's essentially matches my point.
>     shifting a compressed buffer just to save a few bytes. It doesn't
>     matter if a compressed buffer can theoretically fit into a short
>     frame if it's not worth the buffer copy overhead or the code
>     complexity overhead to do the packing.
>
>
> If you start writing at byte 10 of the header (or 18 if you have a 
> complete message length), and then when you flush the buffer you write 
> a 2/10/18 byte header before it in the buffer, I don't know that is 
> too much more complexity to take advantage of it.  Or you could use 
> scatter/gather, or multiple writes.

Although, that only works when you have a single message per packet.

The point is, the gain of saving a byte (or 8 bytes) is very small, 
especially for a single packet. It doesn't take much to overrun that 
savings. Using writev over write, for example, might be enough by itself 
to lose any gain, even ignoring the added pointer costs of that 
technique. Once you add in the fixed overhead of the system calls, TCP 
stack, etc., for each packet, plus compression overhead, plus the 
JSON/XML serialization costs plus application/framework code, the small 
gain from a byte (or 8) is so small that adding code complexity to save 
the byte is just silly.

>     I'll give you Quake, since those packets will be handcrafted
>     binary, and can be designed to fit into 127 bytes. But the more
>     typical JSON/XML are dynamic in length (and tend to accumulate
>     metadata without the authors noticing any overrun beyond 127 bytes.)
>
>
> designed for Ethernet, other than having to encode binary bytes as 
> Unicode characters (so bytes 0x80-0xFF were encoded as two UTF8 bytes 
> in the WebSocket payload) because binary wasn't available.

Understood. I'd just meant that binary protocols for low-latency 
applications are willing to spend the time counting bytes. Quake2's 
already done that (not for websockets specifically of course.)

>  
>
>     We'll see. The short frame sounds impressive (we're squeezing
>     every byte!), and even if it doesn't work as well as advertised in
>     practice, the 10-byte overhead is fine.
>
>
> So if I understand your position, you are skeptical of the benefits 
> for small frames, but you have no objection as proposed?

Correct.

I've seen worse attempted optimizations in the past. CORBA/IIOP, for 
example, spends 4 bytes per object for a big/little endian flag so the 
sender can pick its optimum endian dynamically. Nothing is as silly as 
that attempted optimization. This proposal is fine.

Technically, I'm not opposed to small frames entirely, just question the 
choice of the 127 boundary because that boundary adds code 
complexity/overhead (however small but the benefit is also small), while 
32k/64k small frames add essentially zero complexity.

>         There are a limited number of opcodes available (especially if
>         we decide to use opcodes to encode extensions rather than
>         separate metadata), so I would strongly prefer to keep them in
>         the control frame where there is more room to define them how
>         we like without worrying about every bit.  That said, if the
>         consensus was that they should be opcodes I wouldn't object
>         strongly to it (which would also mean that we wouldn't even
>         define a control opcode in the base protocol).
>
>

I didn't feel strongly enough about it to argue the point.

The ping/pong/close are just three opcodes. Those plus the 
text/binary/continuation are 6 out of 16, and with the addition of an 
ext/control opcode, those are really the only base opcodes people have 
been talking about. Having 9 free system opcodes is plenty, especially 
if you push extension codes into the control frame, which I think is a 
good idea.  In other words, the control frame is a good idea for the 
extension, but can be avoided for the base opcodes.

There's a solid value in making the basic protocol very simple and 
avoiding concepts like the control frame entirely for the people who 
don't care about extensions.

-- Scott

---

From: Patrick McManus <[email-masked]>
Date: Fri, 20 Aug 2010 10:21:12 -0400

On Thu, 2010-08-19 at 13:09 -0700, Ian Fette (イアンフェッティ) wrote:
> I am a bit worried about _totally_ punting extensions. 

Yes, +1.

If we table the necessary details for even the most basic extension
(i.e. TML on any fragment) that will just lead to a total non-deployment
of extensions. If we define it, then to proceed along the standards path
we will need multiple implementations and that will have a decent shot
at making the framework widely useful.

I don't think defining a basic extension is especially hard. And having
that definition will enable deployment of a harder feature (e.g.
multiplexing) on existing infrastructure.

---

From: John Tamplin <[email-masked]>
Date: Fri, 20 Aug 2010 11:18:46 -0400

> at making the framework widely useful.

In the formal proposal, I tried to avoid this problem by requiring that
intermediaries not try to inspect or modify a frame in any way if any
reserved values are used.  That way, if we decide to add multiplexing as an
extension, a v1.0 intermediary can't screw up the multiplexed frames that it
doesn't understand.  Note that does not preclude an intermediary that
understands the MUX extension from doing something with it.

> multiplexing) on existing infrastructure.

Discussions here about extensions indicate otherwise.  I think there is
support for each of two totally different extension mechanisms (use
extension opcodes vs per-frame/message metadata), and then there is debate
on a lot of smaller issues related to what extensions people think are
valuable which impacts how they think the extension mechanism should work.
 Beyond that, there are lots of details, like how do multiple extensions
interact?  What should intermediaries do when they encounter an extension
they don't understand?  Are there extensions an intermediary could safely
ignore?

I really think we should define the base protocol with sufficient
extensibility hooks, so we can get that off the table and agreed upon.  If
we don't, I think we will go around in circles, as we have been, trying to
adjust the base protocol as we propose different ways of implementing the
extensions.