Subject: [hybi] Framing take III

From: Greg Wilkins <[email-masked]>
Date: Sat, 31 Jul 2010 14:34:44 +1000

Below is an update of the frame proposals.  I've tried to take into account
the consensus from the f2f meeting, plus I've removed all feature for which
there is not yet a clear consensus (or at least some unresolved concerns).
Hopefully  we can agree on a basic frame style before progressing with other
issues.

*Proposal : Websocket framing*

The websocket framing be changed to only support binary framing as defined
by

          ws-frame            = frame-flags
                          frame-length
                          octet-data   ; binary data

    frame-flags         = %x00-FF

    frame-length        = %x00-FF %x00-FF

    octet-data          = *( %x00-FF )
            ; the number of octets is exactly the length determined
            ; by the frame-length

The frame length is a 16 bit unsigned integer sent high octect first. The
maximum frame data length is 64k and larger messages must be sent in
multiple frames.

*Proposal : Frame Flags*

The frame flags octet  is structured as

+--------+--------+--------+--------+--------+--------+--------+--------+
|fragment| TBD    | TBD    | ext    | op-code                           |

+--------+--------+--------+--------+--------+--------+--------+--------+

The op-code values are in the range %x00-0F. Current allocated op-codes are:

%x00 - Text Data frame (utf-8 text content)
%x01 - Binary Data frame
%x02-0D - TBD
%x0E - Ping
%x0F - Final Frame

Frames with the ext (extension) bit clear are for the core part of the
protocol and are either text data frames (op-code==%0x00), binary data
frames (op-code==%0x01) or control frames. The allocation and behaviour of
all non-ext op codes will be specified in the websocket protocol
specification, or reserved for future versions of the protocol.

If the ext (extension) bit is set, then the frame is an extension frame and
the allocation and interpretation of the op-code is dependent on the
negotiated (TBD)  extensions.  Extension op-codes may be defined and
specified outside of the websocket protocol specification.

Two bits of the frame flags are currently unallocated and a reserved for
features that cannot be addressed by op-codes (eg frame compression).

*Proposal : Message fragmentation*

If the highest  order bit of the frame-flags is set (the fragment bit), then
the frame contains an partial message chunk.    A receiver should should not
consider the message complete until a frame is received with the chunk bit
clear.

An example of a chunked message in three frames is:

    %x80 %x00 %x08 Goodbye_
    %x80 %x00 %x06 cruel_
    %x00 %x00 %x05 world

The semantics of a message MUST NOT be affected by fragmentations.
Implementations and intermediaries are free to arbitrarily decompose
messages into fragments or to aggregate frames.

*Proposal : Ping

* A control frame with the op-code of %x0E is a ping frame. An endpoint
receiving a ping frame with content should immediately send another ping in
response with the same content. [ Note we need a mechanism to prevent loops
- this could be in the content of the ping or a separate op code can be
allocated for responses ]

Ping messages may be sent to keep alive a connection.

*Proposal : Final Frame*

A control frame with the op-code of %x0F indicates the final frame to be
sent on the connection. A minimal final frame is:

 %x0F %x00 %x00

A final frame may contain TBD content that will give more information
regarding the reason for the final frame.

An endpoint that has sent a final frame should close the connection only
once it has received a final frame. There is no time limit for how long a
connection may be half closed (other than normal idle time outs).

*

Issues to be considered later:*

   - feature/extension negotiation ?
   - frame compression ?
   - connection compression ?
   - connection meta-data ?
   - message meta-data ?
   - error frames ?

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 31 Jul 2010 07:34:39 +0200

> %x00 - Text Data frame (utf-8 text content)

(...)
> Ping messages may be sent to keep alive a connection.

For that, please use two opcodes :
  - ping request  (0x0E)
  - ping response (0x0D)

An agent MUST respond to a ping request with a ping response.
Unsollicitated ping responses MUST be ignored. An agent MAY send
unsollicitated ping responses to keep a connection alive.

Regards,
Willy

---

From: "Thomson, Martin" <[email-masked]>
Date: Sat, 31 Jul 2010 14:46:06 +0800

I like this.  Add Willy’s suggestion and this will work.

Two more suggestions.

The ext flag is unnecessary.  If the behaviour is opaque, then the entire payload is available to the extension.  Op-codes are only necessary if you are modifying core protocol semantics.  If you need an intermediary to understand the semantics so that they can do something about it.  (The great thing is that we don’t have, and probably don’t want, a great many semantics.  I expect that we wont ever use all these op-codes.)

--Martin

From: [email-masked] [mailto:[email-masked]] On Behalf Of Greg Wilkins
Sent: Saturday, 31 July 2010 6:35 AM
To: [email-masked]
Subject: [hybi] Framing take III

Below is an update of the frame proposals.  I've tried to take into account the consensus from the f2f meeting, plus I've removed all feature for which there is not yet a clear consensus (or at least some unresolved concerns).  Hopefully  we can agree on a basic frame style before progressing with other issues.

Proposal : Websocket framing

The websocket framing be changed to only support binary framing as defined by

          ws-frame            = frame-flags
                          frame-length
                          octet-data   ; binary data

    frame-flags         = %x00-FF

The frame length is a 16 bit unsigned integer sent high octect first. The maximum frame data length is 64k and larger messages must be sent in multiple frames.

Proposal : Frame Flags

Frames with the ext (extension) bit clear are for the core part of the protocol and are either text data frames (op-code==%0x00), binary data frames (op-code==%0x01) or control frames. The allocation and behaviour of all non-ext op codes will be specified in the websocket protocol specification, or reserved for future versions of the protocol.

If the ext (extension) bit is set, then the frame is an extension frame and the allocation and interpretation of the op-code is dependent on the negotiated (TBD)  extensions.  Extension op-codes may be defined and specified outside of the websocket protocol specification.

Two bits of the frame flags are currently unallocated and a reserved for features that cannot be addressed by op-codes (eg frame compression).

Proposal : Message fragmentation

If the highest  order bit of the frame-flags is set (the fragment bit), then the frame contains an partial message chunk.    A receiver should should not consider the message complete until a frame is received with the chunk bit clear.

    %x80 %x00 %x08 Goodbye_
    %x80 %x00 %x06 cruel_
    %x00 %x00 %x05 world

The semantics of a message MUST NOT be affected by fragmentations. Implementations and intermediaries are free to arbitrarily decompose messages into fragments or to aggregate frames.

Proposal : Ping

A control frame with the op-code of %x0E is a ping frame. An endpoint receiving a ping frame with content should immediately send another ping in response with the same content. [ Note we need a mechanism to prevent loops - this could be in the content of the ping or a separate op code can be allocated for responses ]

Proposal : Final Frame

A control frame with the op-code of %x0F indicates the final frame to be sent on the connection. A minimal final frame is:

 %x0F %x00 %x00

A final frame may contain TBD content that will give more information regarding the reason for the final frame.

An endpoint that has sent a final frame should close the connection only once it has received a final frame. There is no time limit for how long a connection may be half closed (other than normal idle time outs).

Issues to be considered later:

 *   feature/extension negotiation ?
 *   frame compression ?
 *   connection compression ?
 *   connection meta-data ?
 *   message meta-data ?
 *   error frames ?

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 31 Jul 2010 17:00:00 +1000

Martin,

I think that an extension should not be able to just override the standard
op codes.

That will force intermediaries into having to understand all extensions.
If the core op-codes are distinct from any extensions, then intermediaries
can do meaningful things with pings and final frames that they could not do
if all op-codes were opaque.

I guess we could just have a range of undefined opcodes that extensions can
use, but that is essentially the same as an ext bit.

cheers

On 31 July 2010 16:46, Thomson, Martin <[email-masked]> wrote:

> The ext flag is unnecessary.  If the behaviour is opaque, then the entire
> payload is available to the extension.  Op-codes are only necessary if you
> are modifying core protocol semantics.  If you need an intermediary to
> understand the semantics so that they can do something about it.  (The great
> thing is that we don’t have, and probably don’t want, a great many
> semantics.  I expect that we wont ever use all these op-codes.)
>
>
>
> --Martin
>
>
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of
> *Greg Wilkins
> *Sent:* Saturday, 31 July 2010 6:35 AM
> *To:* [email-masked]
> *Subject:* [hybi] Framing take III
>
>
>
>
> If the highest  order bit of the frame-flags is set (the fragment bit),
> then the frame contains an partial message chunk.    A receiver should
> should not consider the message complete until a frame is received with the
> chunk bit clear.
>
> *A control frame with the op-code of %x0E is a ping frame. An endpoint

---

From: "Thomson, Martin" <[email-masked]>
Date: Sat, 31 Jul 2010 17:15:18 +0800

I think that you misunderstood the suggestion.

When there are core features that are not understood by intermediaries, there are two options:
  tunnel them through if the intermediary doesn’t understand, or
 halt and catch fire

Feature negotiation is used to avoid these sorts of problems.

From: Greg Wilkins [mailto:[email-masked]]
Sent: Saturday, 31 July 2010 9:00 AM
To: Thomson, Martin
Cc: [email-masked]
Subject: Re: [hybi] Framing take III

Martin,

I think that an extension should not be able to just override the standard op codes.

That will force intermediaries into having to understand all extensions.
If the core op-codes are distinct from any extensions, then intermediaries can do meaningful things with pings and final frames that they could not do if all op-codes were opaque.

I guess we could just have a range of undefined opcodes that extensions can use, but that is essentially the same as an ext bit.

cheers

On 31 July 2010 16:46, Thomson, Martin <[email-masked]<mailto:[email-masked]>> wrote:
I like this.  Add Willy’s suggestion and this will work.

Two more suggestions.

--Martin

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Greg Wilkins
Sent: Saturday, 31 July 2010 6:35 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [hybi] Framing take III

          ws-frame            = frame-flags
                          frame-length
                          octet-data   ; binary data

    frame-flags         = %x00-FF

Proposal : Frame Flags

    %x80 %x00 %x08 Goodbye_
    %x80 %x00 %x06 cruel_
    %x00 %x00 %x05 world

Proposal : Ping

Proposal : Final Frame

 %x0F %x00 %x00

---

From: "Thomson, Martin" <[email-masked]>
Date: Sat, 31 Jul 2010 17:19:22 +0800

Sorry, I’m just being stupid.  Ignore that.

Reserve a few bits, or just reserve the rest of the byte for future use.   It doesn’t matter so much which you choose.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Thomson, Martin
Sent: Saturday, 31 July 2010 11:15 AM
To: Greg Wilkins
Cc: [email-masked]
Subject: Re: [hybi] Framing take III

Martin,

cheers

Two more suggestions.

--Martin

          ws-frame            = frame-flags
                          frame-length
                          octet-data   ; binary data

    frame-flags         = %x00-FF

Proposal : Frame Flags

    %x80 %x00 %x08 Goodbye_
    %x80 %x00 %x06 cruel_
    %x00 %x00 %x05 world

Proposal : Ping

Proposal : Final Frame

 %x0F %x00 %x00

---

From: Patrick McManus <[email-masked]>
Date: Sat, 31 Jul 2010 10:49:02 -0400

It is nice to see consensus forming around a bit of technical syntax!
huzzah.

On Sat, 2010-07-31 at 14:34 +1000, Greg Wilkins wrote:

> The frame length is a 16 bit unsigned integer sent high octect first.
> The maximum frame data length is 64k and larger messages must be sent
> in multiple frames.
> 

Hi, I would really urge the easy changes necessary to keep frames
aligned on reasonable word boundaries. The frame length in this
definition is a 16 bit quantity aligned at a 8 bit offset - that means
you either have to copy and manually align it (instead of just
addressing it in situ) or accept the penalties of misaligned access on
your hardware.(*) It is important to allow efficient implementations for
use on busy servers and even embedding in intermediary hardware down the
line.

You could trivially fix it by swapping the length field with the flag
byte.  I would also expand the flag byte to 16 bits, as those 11 unused
bits are the sensible place to put session ids when multiplexing is
added. That would leave you with a nice clean aligned 32 bit read to get
the whole frame header.

This requires padding the frame to keep the alignment - but the start of
next frame calculation is still trivial: newframe = oldframe +
((frame-header-size + data-length + 3) & ~3)

That's all the change that is needed.

Many other protocols do work to maintain alignment for good reasons
(e.g. tcp and ip!) and the technique is well matched to the message
oriented semantics of web sockets. The overhead here is so small as to
be well worth it.

-Patrick

(*) the downsides of unaligned access are platform dependent - which of
course just adds complexity and is a good reason in itself to be
avoided.

* On vanilla X86, it mostly just works ok. There is some non-optimal l2
cache behavior that can slow things down though.

* X86 SSE instructions insist on alignment to work well. Many will
simply fail with unaligned integers.

* I've worked on a bunch of risc OS's that throw processor level
exceptions on unaligned data accesses.

* On ARM the situation could really be any of the above, depending on
the exact details of chip and OS:
http://www.aleph1.co.uk/chapter-10-arm-structured-alignment-faq

> Issues to be considered later:

+ multiplexing

---

From: Scott Ferguson <[email-masked]>
Date: Sat, 31 Jul 2010 10:08:02 -0700

Greg Wilkins wrote:
>
> Below is an update of the frame proposals.  I've tried to take into 
> account the consensus from the f2f meeting, plus I've removed all 
> feature for which there is not yet a clear consensus (or at least some 
> unresolved concerns).  Hopefully  we can agree on a basic frame style 
> before progressing with other issues.

Looks good!

> %x00 - Text Data frame (utf-8 text content)

Please reserve %x0F for extended opcodes (or use some equivalent opcode 
extension mechanism). It doesn't need to be defined now, just reserved. 
Hopefully we'll never need to use it, but you never know.

Also, Willy's suggestion for separate ping request and response opcodes 
is a good one, because it makes the implementation (and testing) much 
simpler. Actually, his suggestion is probably mandatory, because with 
only a single opcode, a receiver can't distinguish a new peer request 
ping from a peer response ping to its own request. Plus, Willy's 
suggestion allows for unidirectional heartbeats when that's more 
appropriate than a ping.

-- Scott

---

From: Scott Ferguson <[email-masked]>
Date: Sat, 31 Jul 2010 10:56:16 -0700

Scott Ferguson wrote:
> Please reserve %x0F for extended opcodes (or use some equivalent 
> opcode extension mechanism). It doesn't need to be defined now, just 
> reserved. Hopefully we'll never need to use it, but you never know.

Hmm. On rereading, I see that's the idea of the "ext" bit, and that core 
control messages like ping have the "ext" bit clear. So the extension 
could reserve %x0F on its own.

I'm still somewhat uncomfortable with not having an escape hatch, even 
though I can't imagine needing more than 16 core opcodes.

-- Scott

---

From: Roberto Peon <[email-masked]>
Date: Sat, 31 Jul 2010 11:05:50 -0700

The only part of the current proposal that makes me at all uncomfortable is
that we're not yet talking about multiplexing. However, everything else
looks pretty good in this proposal.
-=R
On Sat, Jul 31, 2010 at 10:56 AM, Scott Ferguson <[email-masked]> wrote:

> control messages like ping have the "ext" bit clear. So the extension could
> reserve %x0F on its own.
>
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Scott Ferguson <[email-masked]>
Date: Sat, 31 Jul 2010 15:59:21 -0700

Greg Wilkins wrote:
> _*Proposal : Frame Flags*_
>
> frames (op-code==%0x01) or control frames. The allocation and 
> behaviour of all non-ext op codes will be specified in the websocket 
> protocol specification, or reserved for future versions of the protocol.
>
> If the ext (extension) bit is set, then the frame is an extension 
> frame and the allocation and interpretation of the op-code is 
> dependent on the negotiated (TBD)  extensions.  Extension op-codes may 
> be defined and specified outside of the websocket protocol specification.

This extension opcode model doesn't quite work if we allow multiple 
simultaneous extensions (for example multiplexing and mime-type) because 
two extensions would likely have conflicting opcodes and we would have 
no way of resolving the conflict.

If we had a unified opcode-space, say 64k, then extensions could choose 
unique opcodes to avoid conflicts. In that case, the core opcodes would 
just be %x00-0F in that larger space.

The encoding of that opcode-space could be something like Dave's 
suggestion, where %x00-%x1E  are encoded directly and  %x1F-%xFFFF would 
use the first two bytes of the frame payload. (The frame length would 
increase by 2.)

-- Scott

---

From: Greg Wilkins <[email-masked]>
Date: Sun, 1 Aug 2010 09:13:41 +1000

> looks pretty good in this proposal.

I still think there are many that are unconvinced by the need for
multiplexing (I'm not one of them), so I think it is important to come up
with a consensus of the core framing that is both uncomplicated by the needs
of multiplexing, yet able to support multiplexing without change.

So I'm deliberately leaving out any mention of multiplexing from this
minimal framing proposal, so we can see if it stands on it's own merits.
But those that think multi-plexing is important should be evaluating this
framing against the needs of a multi-plexing extension and I would hope that
proposals would soon result.

I wont update the proposal yet again until those travelling back from the
f2f get a chance to comment and for the hums to be published.    However  my
thoughts on the feedback so far:

I agree a ping request and ping response op-code is the best way to prevent
loops (names leave a little to be desired).

I'm drawn to the long-word alignment of the header and the padding of the
data to a long-word boundary and keen to include it if there are no
strenuous objections.  It would give us several things:

   - better efficiency on some platforms
   - An extra octet in the header that could be used by extension, for extra
   length frames or be left as 0x00 as protection against injection from text
   sources.
   - The padding could be 0xff bytes that act as further protection from
   injection and a good validation that framing has not gone astray.

But I do recognize that it is not essential and that it is a little extra
complexity, so we will have to see how much concern that generates.

I hear/share Scotts concerns about how are multiple extensions going to work
and share a limited op code space. Having an extra opcode octet may decrease
the chance of collisions, but it would not remove it, so we would still need
to have some kind of central register of extensions (which would be a pain
and difficult to administer and a drag on innovation).

But if we have 4 bits for extension op-code and the 8 bits of extra octets
from the long-word alignment, then perhaps we could go with an extension ID
and extension op-code combination.  Extension ID's would not be fixed, but
would be allocated when the extensions are negotiated.  So if we negotiated
both multiplexing and meta-data, then multiplexing would be extension 0x00
and meta-data would be 0x01.  Each extension frame would then be delivered
to the right extension and it's op-codes could then be handled as fixed
values within that extension.

This is kind of complex and it may be far far simpler just to allocated core
op-codes for known things like meta-data and multiplexing.  Maybe we need to
do a thought experiment and list every conceivable extension that might need
an op-code within header rather than body and see how many there actually
are. If the number is small then let's avoid a complex ext op-code
allocation mechanism.

cheers

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 1 Aug 2010 08:15:53 +0200

Hi Greg,

> loops (names leave a little to be desired).

Agreed the names are a bit odd. You could use an "AYT" (Are You There)
message for the request and IAT (I Am There) for the response of the
unsollicited message. I believe it's telnet which uses this terminology
but I may be wrong as I've not worked on it for some time. At least it
clearly states the intent of each messages.

> complexity, so we will have to see how much concern that generates.

Indeed, my feeling is that it will be harder to implement on senders because
they'll have to count data sent + padding and could easily get it wrong in
some languages where counting bytes is not the most common thing we do. In
my opinion, the added value on the receiver is minimal. If we had a very
complex message, it could make send to make it fit a struct, but here we
only have to read two bytes and shift one to get the length, it is almost
as cheap as the ntohs() which would be required on many platforms anyway.

I agree with you to keep the protocol simple first and not suggest
multiplexing from the beginning.

Regards,
Willy