Subject: [hybi] Framing Take VI (a compromise proposal)

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 12 Aug 2010 17:53:23 -0700

>
> There have been a lot of framing proposals coming out lately, that have
generated a lot of good discussion and feedback. A group of us at Google
have tried to sift through the proposals, requirements, and feedback and
tried to come up with a proposal that I'm hoping meets people's needs. No
one is ever going to be 100% happy with every last detail, but I really want
to see us agree on something that we agree is solid and move on. At some
point, we do need to decide.

The below proposal is based largely on Dave's earlier proposal, to give
credit where it's due. Takeshi Yoshino and John Tamplin are largely to
credit for the modifications, my contribution is minimal :) We've tried to
take into account what have come out of various consensus calls as well as
where it seems people aren't likely to budge. We've given in on a number of
points (e.g. multiplexing, giving up fixed 16-bit length, supporting a
64-bit length for large messages) in an attempt to make progress, so I would
also ask that people really think about what they feel strongly about and
what they can be willing to consider to support consensus.

That said, here's the proposal:

   0                   1                   2                   3

>    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

>   +-+-+-+-+-------+-+-------------+-------------------------------+

>   |I|F|E|R| opcode|R| Payload len |    Extended payload length    |

>   |N|I|X|S|  (4)  |S|     (7)     |             (64)              |

>   |I|N|T|V|       |V|             |     (if payload len==127)     |

>   +-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - +

>   |                                                               |

>   + - - - - - - - - - - - - - - - +-------------------------------+

>   |                               |                               |

>   +-------------------------------+ - - - - - - - - - - - - - - - +

>   |       Complete message length (64) (if INI==1 && FIN==0)      |

>   + - - - - - - - - - - - - - - - +-------------------------------+

>   |                               |     Ext len (8) (if EXT==1)   |

>   +-------------------------------+-------------------------------+

>   |   Extended extension length (64) (if EXT=1 && ext len == 0)   |

>   + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +

>   |                                                               |

>   +---------------------------------------------------------------+

>   |                        Extension data                         |

>   |                          (if EXT==1)                          |

>   +---------------------------------------------------------------+

>   |                       Application data                        |

>   |                                                               |

>   +---------------------------------------------------------------+

    ws-frame            = frame-type

>                           frame-reserved

>                           frame-payload-length

>                           frame-complete-message-length

>                           frame-extension-length

>                           extension-data

>                           application-data

    frame-type          = type-initial

>                           type-final

>                           type-extension

>                           type-reserved

>                           type-opcode

    type-initial        = %x0 ; not initial frame

>                         = %x1 ; initial frame

    type-final          = %x0 ; not final frame

>                         = %x1 ; final frame

    type-extension      = %x0 ; no extension

>                         = %x1 ; has extension

    type-reserved       = %x0 ; 1 bit

    type-opcode         = %x0 ; control frame

>                         / %x1 ; text data frame

>                         / %x2 ; binary data frame (blob)

>                         / %x3-%xB ; reserved

>                         / %xC-%xF ; user defined

    frame-reserved      = %x0 ; 1 bit

    frame-payload-length = %x00-7E

>                          / %x7F frame-extended-payload-length

    frame-extended-payload-length = %x000000000000007F-FFFFFFFFFFFFFFFF ;
payload length including all fields after this field

    frame-complete-message-length = %x0000000000000000 ; complete message
length is unknown (iff initial==1 && final==0)

>                                   / %x0000000000000001-FFFFFFFFFFFFFFFF ;
complete message length. only application data part will be counted in (iff
initial==1 && final==0)

>                                   / ; none (otherwise)

    frame-extension-length = ; none (iff type-extension==0)

>                            / %x01-7F

>                            / %x00 frame-extended-extension-length

    frame-extended-extension-length = %x000000000000007F-FFFFFFFFFFFFFFFF ;
extension length

    extension-data      = *( %x00-FF ) ; iff type-extension==1

    application-data    = text-data

>                         / binary-data

>                         / control

    text-data           = *( %x00-FF )

    binary-data         = *( %x00-FF )

    control-data        = control-type control-info

    control-type        = %x00-FF ; type of control frame

    control-info        = *( %x00-FF ) ; additional data of control frame

* terminology

> - frame : WebSocket framing unit. if chunked, each chunk forms one frame

> - message : JavaScript level data unit. JavaScript tells WebSocket boundary
between messages by some manner. may be chunked into multiple frames on wire

> - chunking : cutting single message into multiple frames

* requirements and how this proposal addresses them

> bandwidth/memory consumption

> - small frame size for small message

> -- 3 octet frame for 1 octet payload with no extension

> - if payload is small (<127), we have only one octet length header field

> - turn off EXT flag to have no other overhead for extension

> - if not chunked, we don't have "complete message length field" as it's
redundant

processing efficiency

> - data type is available on initial chunk so that we can start UTF-8
decoding immediately

> -- having a single opcode to start a fragmented message and separate
opcodes to determine if it is a text or binary message means you can't start
to decode UTF8 text until you receive the entire message, which means you
add a buffering requirement of the undecoded message

buffer management

> - complete message length is available on initial chunk so that we can
allocate sufficient buffer. no realloc

> -- many people objected to not having an overall message length to ease
receiver buffer management

> - Question: are endpoints likely enough to use UTF16 for internal
representation of text that it would make sense to send the number of UTF16
characters instead of bytes as the message length or as an additional field
on text frames?

> - Note that there is still the option to not provide the message length for
cases when it is not known, such as dynamically generated content or
compressed content (though it is possible we would define the message length
to be the uncompressed length in the case of compression).

> - Also, the message length is only required in the case the message is
fragmented.

extensibility

> - we'd include room for extension

> -- variable length (we cannot fix the size)

simplicity/range of length header

> - variable length header

> -- we give up the use of fixed length header

> --- there should be small sized header for small data. 16 bit, 32 bit, 64
bit, all not good

> --- 8 bit is also not good. We'll have too much overhead for chunking large
message

> - variable length length header

> -- not so complicated. just two case. 1 octet for 0 - 126 and 9 octet for
127 - 2^64-1

> -- simpler enough than variable length length header defined in -76

> -- big range

> --- we can use extended payload length to send large message as 1 frame
without any header stripe on it
---- various people objected to having to fragment messages at all if they
are already in RAM (no more copy)
-- small range

--- small overhead as explained in "bandwidth/memory consumption" section

> simplicity/intermediates

> - extension transparent to intermediate

> -- payload length counts in extension. intermediates that don't recognize
extension don't have to care about extension length

> - only 1 frame boundary indication scheme (length header based one. no
sentinel based one) as many people prefer

* extension plan

> - we included spaces for future extension

> - for new spec

> -- two bits per frame are available for future definition

> -- reserved opcodes

> - for implementors

> -- private use opcodes

> -- extension data based on extensions negotiated during the handshake

---

From: Ian Hickson <[email-masked]>
Date: Fri, 13 Aug 2010 14:16:14 -0700

> That said, here's the proposal:

This seems overly complex. In particular:

 - I don't see the need for explicit extension lengths.

 - I don't see what having both a short length and a long length gets
us. Saving 7 bytes seems like a very minimal gain, especially when
considered in the context of the rest of the header and the IP and TCP
overhead.

 - Supporting fragmentation today is premature. We have no experience
with how people use this protocol, we shouldn't be adding such
features yet. Six months from now, sure, once we can quantify what it
is people actually need.

 - I don't see why we would need to flag fragmentation frames using
reserved bits rather than just using the frame type byte mechanic.
Just define four frame types for text and four frame types for binary
when we add binary (not fragmented, fragment start, fragment middle,
fragment end).

-- 
Ian Hickson

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Fri, 13 Aug 2010 14:20:42 -0700

>  - I don't see the need for explicit extension lengths.
If you don't want to use any extensions, which I assume you don't, then set
EXT=0 and you don't have to set any extension lengths. Given that you don't
seem to want to use extensions, I'm not sure why you seem to have a
preference for how extension data is stated.

> considered in the context of the rest of the header and the IP and TCP
An alternate would be to use a variable length. I really don't care that
much.

> features yet. Six months from now, sure, once we can quantify what it
This point has been argued extensively enough on the list that I don't think
anything I say here is going to be new.

---

From: John Tamplin <[email-masked]>
Date: Fri, 13 Aug 2010 17:26:33 -0400

On Fri, Aug 13, 2010 at 5:20 PM, Ian Fette (イアンフェッティ) <[email-masked]>wrote:

> This point has been argued extensively enough on the list that I don't
> think anything I say here is going to be new.
>

Not to mention support for fragmentation has been declared a consensus
requirement without objection on this list.

>> when we add binary (not fragmented, fragment start, fragment middle,
Isn't that using two bits to define fragmentation the same way, except now
an implementation has to know opcodes 0, 14, 15, 23, and 25 are fragment
start codes, etc?  It seems that fragmentation should be supported across
all frame types -- why would one type of frame want to omit fragmentation
support?

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 14 Aug 2010 04:24:38 +0200

>add a buffering requirement of the undecoded message

The formatting of your mail and its HTML attachment is somewhat broken
so I am not sure what I am responding to here, but the observation seems
incorrect; http://bjoern.hoehrmann.de/utf-8/decoder/dfa/ for instance
makes it rather easy to decode the bytes as you receive them, so long as
they are not delivered out of order.

>characters instead of bytes as the message length or as an additional field

Sending it .instead. is probably not an option as that would encourage
some implementers to take shortcuts like sending just twice the number
of bytes if they expect to only ever send US-ASCII. And sending it in
addition would still mean the number could be wrong, and there are many
unknowns (length of strings, which code points are in the strings, how
scripts, in case of web browsers receiving text, use the text, how the
recepient implements strings, and so on).

The computer I am using right now is a AMD Athlon II X2 215 with some
very cheap main memory and it can transcode UTF-8 to UTF-16 at about
500 KB per millisecond (using the latest version of my decoder, which
is about the fastest I know of), that's three orders of magnitude re-
moved from the computer's Internet connection's bandwidth. I don't see
a particular indication that knowing the length of the UTF-16 buffer
in advance would have a noticable effect on my browsing experience.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: John Tamplin <[email-masked]>
Date: Fri, 13 Aug 2010 22:47:58 -0400

> they are not delivered out of order.

Dave's original proposal had one opcode to start a frame, and two separate
opcodes to end a frame - one for text, and one for binary.  That meant until
you read the entire message, you didn't know if it was going to be UTF8 or
not.  That is what the paragraph quoted is referring to -- you need to know
the type of frame you are receiving at the first frame so you can decode it
as you receive each fragment.

> in advance would have a noticable effect on my browsing experience.

So imagine you are writing the code to receive a text WebSocket message.
 Ultimately, you want to pass some UTF16-based string to the client code.
 The total message length in bytes is available, but UTF8 characters of 1-5
bytes will convert to 1-2 UTF16 characters.  So, that means that (given
message length of n bytes from the first frame) you need to allocate
wchar_t[n] (or char in Java, etc) in case each character in the message is
US-ASCII and possibly waste storage when some non-ASCII characters are
included.  Another alternative is to allocate a smaller buffer and then
resize it in the event that it is not large enough.  If instead the number
of UTF16 characters is known from the first fragment of the message, you can
simply allocate the correct size and never have to reallocate.  So, it isn't
about the processing speed of converting UTF8->UTF16, but rather buffer
management.

The downside is not all implementations may want to use UTF16 representation
of the text data, in which case the value is useless.  So, I think if it
were useful, it would have to be in addition to the overall message length.

---

From: Takeshi Yoshino <[email-masked]>
Date: Sat, 14 Aug 2010 14:47:11 +0900

> An alternate would be to use a variable length. I really don't care that

I don't have strong preference on this point, too.

But let me provide some metric about this. As you said, people who mind
latency much would turn off NAGLE, so short packets containing, e.g. key
stroke of 1 octet would ultimately forms minimum_gap(8) + ethernet(18) +
ipv4(20) + tcp(20) + type(1) + len(1) + payload(1) = 69 octet on wire.
difference of 7 octets is just 10% increase on wire and 7/43=16% increase on
kernel level, and 3/9=33% increase on WebSocket level. Not much as 33%, 16%
and 10% still seems not so negligible amount.

If a user sends short messages and doesn't urge so much the messages to be
delivered, client can be implemented using some buffering so that they
(multiple WebSocket frames for multiple send() calls) can be combined into
one TCP packet. In such case, the difference of length header size would be
more significant. Actually, this can be also solved on application level by
combining e.g. several key stroke information into one send() call, so it's
not strong support for this choice.

Takeshi

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>

---

From: Takeshi Yoshino <[email-masked]>
Date: Sat, 14 Aug 2010 14:58:48 +0900

2010年8月14日14:47 Takeshi Yoshino <[email-masked]>:

>> If you don't want to use any extensions, which I assume you don't, then
>> set EXT=0 and you don't have to set any extension lengths. Given that you
>> don't seem to want to use extensions, I'm not sure why you seem to have a
> difference of 7 octets is just 10% increase on wire and
>

Pardon me. minimum_gap 8 -> 12 (most modern NICs work with 8, i think, but
takes number on the 802.3). And, I missed preamble, so s/69 octet/81 octet/.
7/81=8.64% increase on occupation on ethernet baseband signal.

Takeshi

> 7/43=16% increase on kernel level, and 3/9=33% increase on WebSocket level.
> Not much as 33%, 16% and 10% still seems not so negligible amount.
>

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 14 Aug 2010 08:12:40 +0200

[ a bit off-topic ]

> 7/81=8.64% increase on occupation on ethernet baseband signal.

The network overhead should be considered at IP level because that's what
most network providers will count. So your 7 more bytes are 50 vs 43, which
means 16%. That said, it's hard to believe that some networks will be loaded
with 1-byte frames to the point of being significant enough. The packet
processing is much more of a concern there.

Regards,
Willy

---

From: "Anne van Kesteren" <[email-masked]>
Date: Sat, 14 Aug 2010 11:09:55 +0200

On Fri, 13 Aug 2010 23:16:14 +0200, Ian Hickson <[email-masked]> wrote:
> This seems overly complex. [...]

Agreed. I would really like to avoid having so much features in the  
initial deployment.

-- 
Anne van Kesteren
http://annevankesteren.nl/

---

From: John Tamplin <[email-masked]>
Date: Sat, 14 Aug 2010 13:50:51 -0400

On Sat, Aug 14, 2010 at 5:09 AM, Anne van Kesteren <[email-masked]> wrote:

> Agreed. I would really like to avoid having so much features in the initial
> deployment.

[The following is quick-and-dirty code I wrote on the spot, so don't pick it
apart -- it is just to illustrate the point that it isn't that complicated]

For a simple sender which doesn't negotiate any extensions and sends only
text in unfragmented frames, it looks like this:

Connection::sendMessage(char* utf8, int utf8len) {
  writeByte(0xC0 | TEXT_FRAME);
  if (utf8len >= 127) {
    writeByte(127);
    writeUnsignedLong(utf8len);
  } else {
    writeByte(utf8len);
  }
  writeBytes(utf8, utf8len);
}

For a receiver, there is additional complication because of the need to
support fragmented messages, but that has been accepted as a requirement.
 It isn't much worse:

Connection::readFrame) {
  int frameType = getByte();
  int opcode = frameType & 0xF;
  if (opcode != TEXT_FRAME) // error, discard, whatever
  long length = getByte();
  if (length == 127) {
    length = getUnsignedLong();
  }
  switch (frameType & 0xC0) {
     case 0x80: // initial frame of fragmented message
       if (state == IN_MESSAGE) // error
       state = IN_MESSAGE;
       int messageLength = getUnsignedLong();
       bufLen = messageLength ? messageLength : DEFAULT_BUFFER_SIZE;
       buf = new char[bufLen];
       bufIdx = 0;
       // fallthrough
     case 0x00: // intermediate frame of fragmented message
       if (state != IN_MESSAGE) // error
       growBufferIfNeeded(length);
       getBytes(buf + bufIdx, length);
       bufIdx += length;
       break;
     case 0x40: // final frame of fragmented message
       if (state != IN_MESSAGE) // error
       // fallthrough
     case 0xC0: // unfragmented message
       growBufferIfNeeded(length);
       getBytes(buf + bufIdx, length);
       deliverUtf8TextMessage(buf, bufIdx + length);  // receiver is
responsible for freeing
       state = NOT_IN_MESSAGE;
       buf = 0;
       bufLen = 0;
       break;
   }
}

Connection::growBufferIfNeeded(int length) {
   if (buf && bufLen >= bufIdx + length) return;
   int newBufLen = (bufLen == 0) ? length : bufLen * 2;
   char* newbuf = new char[newBufLen];
   if (buf) {
     memcpy(newbuf, buf, bufIdx);
     delete[] buf;
   }
   buf = newBuf;
   bufLen = newBufLen;
}

Given the requirement for fragmentation support, it isn't clear how you make
it a lot simpler.  Note that the parts dealing with the frame structure are
just as simple as in the sender case.

---

From: John Tamplin <[email-masked]>
Date: Sat, 14 Aug 2010 14:05:46 -0400

> processing is much more of a concern there.

Besides, Ethernet isn't in place for the case we most care about the number
of bytes -- it is wireless clients.  AFAIK, they don't currently use RFC2507
header compression, but if they did the TCP/IP overhead will typically go
down to under 7 bytes making saving 7 bytes from a length field much more
important.  Wireless PDUs are also much smaller than Ethernet, so small
frames have less chance of being fragmented over the wireless network.

---

From: Scott Ferguson <[email-masked]>
Date: Sat, 14 Aug 2010 13:03:08 -0700

John Tamplin wrote:
> On Sat, Aug 14, 2010 at 5:09 AM, Anne van Kesteren <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     On Fri, 13 Aug 2010 23:16:14 +0200, Ian Hickson <[email-masked]
>     <mailto:[email-masked]>> wrote:
>
>         2010/8/12 Ian Fette (イアンフェッティ) <[email-masked]
>         <mailto:[email-masked]>>:
>
>
>     Agreed. I would really like to avoid having so much features in
>     the initial deployment.
>
>
> [The following is quick-and-dirty code I wrote on the spot, so don't 
> pick it apart -- it is just to illustrate the point that it isn't that 
> complicated]
>
> For a simple sender which doesn't negotiate any extensions and sends 
> only text in unfragmented frames, it looks like this:
>

Well, the complexity is more of a spec issue than a coding one because 
none of these proposals are hugely complicated to implement. 
(Implementing them efficiently is a different matter.)

A Jamie-style proposal less complex because it leaves the basic frame 
clean without needing the extension length or the extension payload and 
without the extra complete-length field. The frame would look like:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+---------+-------------------------------+---------------+
|I|F|B|opcode(5)| length(16)                    |               |
|N|I|I|         |                               |               |
|I|N|G|         |                               |               |
+-+-+-+---------+-------------------------------+ - - - - - - - +
| big-length(48) if BIG = 1                                     |
+ - - - - - - - +-----------------------------------------------+
|               | payload(length)                               |
+---------------+                                               |
|                                                               |
+---------------------------------------------------------------+

As a measure of complexity, compare the features that can vary the 
frame-header size:

  a) short/long payload length
  b) extension data
  c) short/long extension length
  d) complete message-length

A Jamie-style extension only has the single complexity-feature (a) to 
the frame-header, while your proposal adds 3 more (b,c,d). Since the 
Jamie-style extension supports all the extension capabilities that yours 
does, there's no added capabilities with the extra complexity added by 
b,c,d.

As a separate issue, your code's signature doesn't capture the essence 
of the problem, because you really need to consider all the APIs when 
evaluating a framing proposal, not just one that happens to be easy to 
implement for the proposal. Your example was a specialized API that 
really only works for utf-8 encoded 8-bit C-strings.

The basic sender APIs that framing needs to support are:

a)  sendTextMessage(char []buffer, int offset, int length); // 
JavaScript 16-bit text send
a*) sendTextMessage(byte []buffer, int offset, int length); // utf-8 
encoded text (like PHP)

b)  sendBinaryMessage(byte []buffer, int offset, int length); // binary 
single-buffer send

c)  class BinaryMessageOutputStream {
    write(byte []buffer, int offset, int length);
    close();
  }

d)  class TextMessageWriter {
    write(char []buffer, int offset, int length);
    close();
  }

The receiver variations are

a) char []receiveTextMessage()

b) byte []receiveBinaryMessage() // if returning utf-8 8-bit strings, 
this is the text receive

c) class BinaryMessageInputStream {
      int read(byte []buffer, int offset, int length);
    }

d) class TextMessageReader {
      int read(char []buffer, int offset, int length);
    }

A good frame design will support all 5 sender API styles and 4 receiver 
API styles in a straightforward/efficient manner. You can't just pick 
one API that happens to make a frame proposal work.

-- Scott

> For a receiver, there is additional complication because of the need 
> to support fragmented messages, but that has been accepted as a 
> requirement.  It isn't much worse:
>
> Connection::readFrame) {
> Given the requirement for fragmentation support, it isn't clear how 
> you make it a lot simpler.  Note that the parts dealing with the frame 
> structure are just as simple as in the sender case.
>
> -- 
> John A. Tamplin
> ------------------------------------------------------------------------
>

---

From: John Tamplin <[email-masked]>
Date: Sun, 15 Aug 2010 00:03:24 -0400

On Sat, Aug 14, 2010 at 4:03 PM, Scott Ferguson <[email-masked]> wrote:

> Well, the complexity is more of a spec issue than a coding one because none
> of these proposals are hugely complicated to implement. (Implementing them
> efficiently is a different matter.)
>
> A Jamie-style proposal less complex because it leaves the basic frame clean
> without needing the extension length or the extension payload and without
> the extra complete-length field. The frame would look like:

There were objections raised about exactly this sort of thing before:

   - a receiver would prefer to allocate the buffer at once for a message
   rather than having to reallocate/copy it, which means the sender should send
   the complete message length with the first fragment if it is available
   - having a short length that was "mostly enough" would encourage buggy
   implementations that don't support or test the long length case

> A Jamie-style extension only has the single complexity-feature (a) to the
> frame-header, while your proposal adds 3 more (b,c,d). Since the Jamie-style
> extension supports all the extension capabilities that yours does, there's
> no added capabilities with the extra complexity added by b,c,d.
>

The point of having a specified way of passing extension data in the framing
is so intermediaries can know where the payload data is even if they don't
understand the extensions that were negotiated.  I personally don't feel
terribly strongly about this.

Regarding d, as mentioned above those arguing against fragmentation objected
to not having the full message length, and after proposing that addition
they seemed happy with the compromise.  As fragmentation support has been
accepted as a requirement

> As a separate issue, your code's signature doesn't capture the essence of
> the problem, because you really need to consider all the APIs when
> implement for the proposal. Your example was a specialized API that really
> only works for utf-8 encoded 8-bit C-strings.
>
> a)  sendTextMessage(char []buffer, int offset, int length); // JavaScript
> 16-bit text send
>

The complication of UTF16->UTF8 conversion is the same in any other
length-based framing options, though supporting fragmentation means you can
have a fixed-size send buffer rather than having to allocate all of it at
once.

> b)  sendBinaryMessage(byte []buffer, int offset, int length); // binary

Other than the choice of opcode, these are identical, right?

>   write(char []buffer, int offset, int length);

Again, I don't see these as showing any differences between the various
framing proposals, and fragmentation makes it easier to support streams as
you just write a fragment when the buffer fills.

> b) byte []receiveBinaryMessage() // if returning utf-8 8-bit strings, this
> is the text receive
>

I would think there would also be an event-driven API, since that is what
the JS API is going to look like -- where there is an onMessage event called
with a complete message after it has been received.  The only public
server-side API I know of (Jetty7) also operates this way.

> A good frame design will support all 5 sender API styles and 4 receiver API
> styles in a straightforward/efficient manner. You can't just pick one API
> that happens to make a frame proposal work.
>

Can you explain how your proposed framing is any different for any of these
APIs?  The difference is about the size of the short frame length, leaving
out the extension information which isn't going to be used in the base
protocol anyway, and leaving out the overall message length which means you
have to guess a buffer size for fragmented messages and grow it if necessary
(or fail the receive if using a too-small caller-supplied buffer).

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 16 Aug 2010 13:44:59 +1000

Ian,

thanks for the proposal.

It appears that the total message length is mandatory if fragmentation
is used, however I believe that one of the use-cases for fragmentation
is to send dynamic messages where the total length is not known in
advance.   Could we make message length optional somehow?

Alternately, message length (in bytes and/or in characters etc.) is
just one example of meta-data.  Would it not be possible to support
that in a standardized meta data extension?

If we are to have per frame extension data, then I think having the
extension length in the header so simple implementations can skip is a
good approach.  However, I think we probably would also need to goto
extra detail about how data from multiple extensions would be
separated within the extension data.

However, I remain a little concerned that having both op-code and per
frame extension data could result in some contradictory data.  For
example, if we have an op-code for final frame, then it is likely that
intermediaries will know that frame and possibly use it as part of
shutting down the connection (with half closes etc.).  If we then have
an extension that provides multiplexing and allows a channel ID to be
associated with every frame, what does it mean to have a final frame
op code extended with a channel ID?  The obvious interpretation for
the end points is that is the final frame for just that channel, but
intermediaries would not know that and could shutdown the whole
connection.

Thus until such potential contradictions can be worked out, I think
I'm still in favour of an op-code that indicates that the frame
contains extended data plus data.  This could still be used for
something like multiplexing as a total replacement for plain
text/binary frame opcodes. I'll put together a more detailed proposal
for this shortly.

cheers

2010/8/13 Ian Fette (イアンフェッティ) <[email-masked]>:
> -- having a single opcode to start a fragmented message and separate opcodes
> to determine if it is a text or binary message means you can't start to
> decode UTF8 text until you receive the entire message, which means you add a
> buffering requirement of the undecoded message
>
> buffer management
>

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Sun, 15 Aug 2010 21:02:45 -0700

> advance.   Could we make message length optional somehow?
It is optional. %x0000000000000000 ; complete message length is unknown (iff
initial==1 && final==0) -- suggestions as to how to make that clearer most
welcome.

> op code extended with a channel ID?

I would assume it means that a complete message has been sent, but that the
channel remains open. E.g. I have two channels open, the fact that I've sent
a complete message on one channel shouldn't be taken as an indication that
I'm done sending messages on that channel. Even in the non-multiplexing
case, the fact that you've received a full message doesn't mean the
connection should be shut down. That's the point of "persistent" :)

>  The obvious interpretation for
> the end points is that is the final frame for just that channel,

No, it's not the final frame for the channel, it's the final frame for a
message. We said a message can be multiple frames, and multiple messages can
be sent on a websocket connection (or on a channel in the multiplex case).

> but
> opcodes

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 16 Aug 2010 14:28:27 +1000

> initial==1 && final==0) -- suggestions as to how to make that clearer most

Sorry my bad. I see it in the BNF now.

It does feel a little funny to be byte squeezing other lengths and
then to send 8 zeros to say unknown.  To be consistent, we should
either just always send lengths in full or always make an attempt to
squeeze.

>> that in a standardized meta data extension?

any thoughts on this?

> connection should be shut down. That's the point of "persistent" :)

Sorry I think you have misunderstood my scenario.  By Final frame, I
was not meaning the final frame as indicated by the bit in your frame
header, but the final frame op-code that we have previously discussed
as one of the potential op-codes (text-data, biinary-data, ping, pong,
final frame, etc).   The final frame op-code has been proposed to half
close a connection - ie no more frames will be sent after a final
frame.  There is such an op-code in the current -00 proposal already.

So to restate my concern, I'm worried that per frame extension data
and core protocol op-codes may not be entirely orthogonal. Some
extensions may be very meaningful for some op-codes, but not others.

To flip this around to a question - can you say why you need extend
every frame?   if there are ping, pong and final frame op-codes
defined, what is the use-case to extend them?  I can see the good
use-cases for extending frames with text and binary op-codes, but that
could be done with just a extended-text and extended-binary op-code?

cheers

---

From: Takeshi Yoshino <[email-masked]>
Date: Mon, 16 Aug 2010 18:24:40 +0900

> >> that in a standardized meta data extension?
it also sounds ok to me.

> frame.  There is such an op-code in the current -00 proposal already.
I feel like that kind of "logical" half closing should be dealt with by
multiplexing extension level by raising
some bit or some mux op-code in extension bits for mux protocol to tell that
channel X is half closed by
client.

Such final frame op-code in standard WebSocket framing (core protocol)
should not be diverted for indicating closing some of mux-ed channels.

That said, we're saying that complete message length field means the size of
messages in mux-ed channel.
Yes, this is also "logical" information for mux-ed channels.
Letting two meta-data (the followings) both in standard WebSocket framing
(core protocol) have meanings for
different level sounds a bit complicated. Good point.
- complete message length : to tell complete message length of messages
in mux-ed logical channel,
- op-code : to tell encapsulating real WebSocket channel

i'm flexible with combining or separating EXT from op-code. If we have
op-codes for which extension doesn't make sense,
your idea looks better in terms of utilization.

if we define EXT bit separately, when new op-codes are added in the future,
old implementation still can know that there's
extension in that frame. op-code is unknown for that implementation, so i'm
not sure if this is really useful.

Thanks,

---

From: John Tamplin <[email-masked]>
Date: Mon, 16 Aug 2010 09:21:13 -0400

> either just always send lengths in full or always make an attempt to

The idea was that if we are sending a fragmented message, the overall
message size is known to be fairly large and thus 8 bytes is not a
significant overhead.  The frame length and extension data fields however
may be present in very small frames, so it seems more important to conserve
bytes there.

> >> that in a standardized meta data extension?

Yes, though I think that makes the simple implementations more complex since
they now have to parse the extension data.  Also, it means we have to define
it now, which seems like it will take a long time to reach consensus.

> frame.  There is such an op-code in the current -00 proposal already.

In this proposal, it would be a Control opcode with a Close control frame
type.

> extensions may be very meaningful for some op-codes, but not others.

Sure, and it seems clear that when an extension is defined it will indicate
how it will be used on different frame types.  In you example, I would
expect that Control/Close [which I assume is what you are specifically
talking about] means to close the entire connection gracefully, while
add/drop of mux channels would take place entirely within the mux extension
data (which might include definition of new Control opcodes).

> could be done with just a extended-text and extended-binary op-code?

First, there are limited opcodes available.  If you start using them for
every extension, you will run out quickly.  That said, there are some
available for future definition if we feel their allocation is justified,
and there are 4 private-use opcodes which can be used for experimentation or
where both ends of the connection are controlled by the same entity.

Basically, if we are going to be defining the framing format for the base
protocol, leaving out known use cases (compression and multiplexing
specifically come to mind) until we can see how best to implement them after
seeing how the base protocol is used, I think we need to leave multiple
extension hooks in the base protocol so that we can extend it without having
to change existing implementations of the base protocol.

Finally, it seems better for intermediaries if the opcodes are generally
unchanging.  Imagine an intermediary that scans for viruses, important
intellectual property, etc.  If the frame type remains TEXT/BINARY and any
mux data is in an extension header, it can completely ignore the extension
and simply look at the payload data for content it wants to block.  If
instead a MUX opcode was added and buried inside the MUX payload as a new
frame type, the intermediary would have to understand the MUX extension in
order to do anything useful with the message.

In particular, I would think that per-frame extension data would be a good
fit for multiplexing, as you would want to be able to attach a channel id to
each frame.

For example, here is one way multiplexing might be supported (after
negotiating it in the handshake):

Endpoint 1:
INI=1/FIN=1/EXT=0 len=... opcode=CONTROL
  control: MUX control: add channel (details)

E2:
INI=1/FIN=1/EXT=0 len=... opcode=CONTROL
  control: MUX control: channel 1 created

...

INI=1/FIN=1/EXT=1 len=.. opcode=TEXT
  extendsion data: MUX=channel 1
  payload data: text for channel 1

...

E1:
INI=1/FIN=1/EXT=0 len=... opcode=CONTROL
   control: MUX control: drop channel 1

E2:
INI=1/FIN=1/EXT=1 len=... opcode=CONTROL
   control: MUX control: acknowledge drop channel 1

(the client would then treat that as graceful close on the channel, just
like a close on an unmuxed connection)

---

From: Patrick McManus <[email-masked]>
Date: Mon, 16 Aug 2010 10:05:50 -0400

The total message length (TML) field is mandatory on the first chunk of
a "large" message even if that field contains an "unknown" length (0)..
do I read that right?

Furthermore, TML only appears on the first chunk.. yes?

It seems to me that there are cases where TML is known before the bulk
of the message is sent but not on the first chunk.. almost any largish
dynamically created document is going to meet that criteria as send
buffers fill up quickly after the initial bytes have hit the wire. The
TML is still helpful to the client on chunk 5 of 100 (if not as useful
as it would have been on chunk 1). 

Can't TML just be put in an extension/option when it is first known to
the sender? That saves the mandatory run of 0's from ever appearing when
TML is not known, and allows TML to be sent on chunk other than the
first one too.

TML is useful meta information to be sure, but I don't really see the
need to make it first class with a fixed location. A multiplexing ID has
a better claim to that as it would be used by end points and
intermediaries for higher throughput routing and flow matching.

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 17 Aug 2010 00:32:11 +1000

John,

I'm just going to cherry pick your words to respond to... because I
don't particularly disagree with most of what you say... but...

On 16 August 2010 23:21, John Tamplin <[email-masked]> wrote:

> and simply look at the payload data for content it wants to block.

But I don't think we can generally do that.  If the extension changes
the data in anyway (add's headers, compresses it, encodes it etc.)
then the intermediary is probably going to fail in it's attempt to
scan the data.

Also if a text frame is being sent, but a compression extension is
applied, then the contents of the frame would not even be text.
Should the opcode be changed to binary to reflect the content, or left
at text to reflect that it is destined for the text API?

I think if an intermediary does not understand an extension, then it
must ignore all data in an extended frame, so there is no harm sending
it as a extension-data op-coded frame.

> If

It could still forward the message along the connection.

I think that an intermediary that is doing anything with muxed data
has to be MUX aware otherwise it will risk contaminating one channel
with state obtained from another.

> fit for multiplexing, as you would want to be able to attach a channel id to

An extension-data op-code would allow a channel id to be associated
with every extension-data frame, so messages would just have to be
sent using that frame type.  I'm not sure that is a bad restriction
given that I think intermediaries that do not understand MUX should do
nothing with the data other than forward it.

cheers

---

From: John Tamplin <[email-masked]>
Date: Mon, 16 Aug 2010 10:51:09 -0400

> nothing with the data other than forward it.

To make sure I understand what you are proposing, here is a sample
conversation according to my understanding of what you are suggesting:

INI=1/FIN=1 opcode=CONTROL
   control: MUX - create channel (details)

INI=1/FIN=1 opcode=CONTROL
   control: MUX - use channel 1

INI=1/FIN=1 opcode=MUXDATA
   payload: multiplexed data:
      channel 1, 42 bytes: (channel 1 payload)

...

INI=1/FIN=1 opcode=CONTROL
   control: MUX - close channel 1

INI=1/FIN=1 opcode=CONTROL
   control: MUX - acknowledge close channel 1

If my understanding is correct, then I think that can work though I do think
it limits what intermediaries can do if they don't understand the frame
type.  As you say above, some extensions will mean they either blindly pass
the frame or drop it.  I worry that corporate firewalls will default to
dropping anything they can't inspect, which would limit the ability to
actually use extensions defined in the future, or at least greatly delay
their usefulness as such intermediaries will have to upgraded first.

The bigger problem is that this approach of allocating new opcodes means
that we need more opcodes, and we will have to pay that cost up front for
even unextended frames, since we won't be able to make it larger later
without breaking existing implementations.  Having a variable-length
extension header basically means we can allocate it later as we define new
extensions.

Maybe a compromise would be to define an opcode EXTENDED_FRAME, and for that
opcode the n bytes (however many we want to allocate) are an extension ID,
and the rest are extension-defined (which might include data from multiple
logical channels, for example, so it would need multiple frame
types/lengths).

Also note that you were previously suggesting moving the message length
field to the extension data -- that wouldn't be possible if we do away with
extension data and instead make it an opcode.  So, I assume that means we
would keep the message length as defined in the current proposal.

---

From: Dave Cridland <[email-masked]>
Date: Mon, 16 Aug 2010 15:56:25 +0100

> as you receive each fragment.
Ah - yes, I'd thought there was no reason to decode early, but this  
seems like a reasonable one. It also makes debugging easier, so I'm  
quite happy with this flagrant casting aside of my brilliant ideas.

> were useful, it would have to be in addition to the overall message  

This feels like something that's not a requirement, and therefore  
needs to be an extension.

It also feels like the kind of extension nobody will actually  
implement, too.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Dave Cridland <[email-masked]>
Date: Mon, 16 Aug 2010 16:01:00 +0100

On Fri Aug 13 01:53:23 2010, Ian Fette (イアンフェッティ)  
> what they can be willing to consider to support consensus.

I basically like this - nothing that's present gives me cause for  
concern.

I do reiterate my previous suggestion that the group give serious  
consideration to including a multiplex identifier within the header,  
however, or at least space for one via usage of a reserved bit - I  
think this will help intermediaries cope when we actually get  
multiplexing going.

Dave.

---

From: John Tamplin <[email-masked]>
Date: Mon, 16 Aug 2010 11:00:54 -0400

> as it would have been on chunk 1).

If you are already reallocating/copying the buffer it seems like you are
getting little benefit from getting it after the first frame.  It also isn't
clear when dynamically generated content is going to be known ahead of time.
 I would expect a typical implementation of that would be stream-based --
the content generator writes to a stream, and when the stream's buffer is
full it is written to a fragment frame.  When the stream is closed, only
then do you know the total message length but at that point you are writing
the final frame of the message so there is nothing to gain by including it
then.

> intermediaries for higher throughput routing and flow matching.

The main problem with moving it to per-frame extension data is that we have
to define that extension data now if we want to have TML in the base
protocol (which it seemed the people against fragmentation support thought
it had to be to mitigate the receiver's buffer management issues if
fragmented messages are supported).  Given the progress of reaching
consensus, that would likely significantly delay any WS standard -- the
longer we wait, the more entrenched v76-based implementations get and the
less chance we have to get universal adoption of the final spec.

Leaving extension data defined in the framing format but the exact format of
the extension data undefined is a way to get consensus on the base protocol
sooner rather than later.  Including TML where known seems valuable in order
to gain consensus as well.

---

From: John Tamplin <[email-masked]>
Date: Mon, 16 Aug 2010 11:12:23 -0400

On Mon, Aug 16, 2010 at 11:01 AM, Dave Cridland <[email-masked]> wrote:

> however, or at least space for one via usage of a reserved bit - I think
> this will help intermediaries cope when we actually get multiplexing going.
>

The problem is that defining it now basically means deciding how it is going
to be implemented, and I don't think we want to wait until we get such
consensus first.

For example, we might want a multiplexing extension to aggregate multiple
channels' data into a single frame.  So, it might look something like this
(using the idea in the discussion with Greg about extension data):

INI=1/FIN=1 opcode=EXTENDED_FRAME, len=...
  payload: extension type=MUX
      channel 1, FRAME_TEXT, 14 bytes
      channel 14, FRAME_BINARY, 48 bytes
      channel 32, FRAME_TEXT, 1 byte

In that case, you can't have a single channel ID in the header.  So, if we
were to allocate one of the two reserved bits for mux'd frames and defining
the channel id in a fixed place in the header if that bit is set, it would
go unused if we wanted this sort of mux implementation.

The whole reason the agreement to get a base protocol out now and work on
extensions in early 2011 is that we don't have any sort of consensus on how
these extensions should be implemented.  For example, we seem to have pretty
broad agreement that compression should be supported.  However, we have at
least 3 very different ideas for doing it:

   - per-frame compression, using negotiated (and possibly asymmetric)
   algorithms and using a bit per frame to indicate compressed data (and
   possibly exchange compression dictionary information in extension data)
   - mandate zlib compression and manipulate the compression level to avoid
   trying to compress uncompressible data
   - compress the entire stream and flush at each frame boundary, and assume
   uncompressible data won't expand too much

For multiplexing, there appear to be several people on the list who do not
see a need for it at all.  If we want to define mux support in the base
protocol, we not only have to design it now but we have to get consensus
that it should even be included.

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 17 Aug 2010 01:16:58 +1000

John,

you don't exactly have what I'm proposing.

Consider the following two independent ws connections

connection 1:
   op=text data=Hello World
   op=binary data=9ab39129def329432993
   op=test data=goodbye

connection 2:

   op=text data=One
   op=binary data=2222222222222
   op=text data=three

A hypothetical MUX extension might merge those connections as:

   op=extended ext=MUX(channel=1,type=text) data=Hello World
   op=extended ext=MUX(channel=2,type=text) data=One
   op=extended ext=MUX(channel=1,type=binary)  data=9ab39129def329432993
   op=extended ext=MUX(channel=2,type=binary)  data=2222222222222
   op=extended ext=MUX(channel=2,type=text)  data=three
   op=extended ext=MUX(channel=1,type=text)  data=goodbye

> logical channels, for example, so it would need multiple frame

that's more or less what I'm proposing.  Although I think of it more
as using something similar to Ian Fette's proposal for extension data,
but only for 1 op code.

> would keep the message length as defined in the current proposal.

It could still be done as an extension.  Eg the message

   op=text data=Hello World

might be fragmented as

  op=text frag=1 data=Hello
  op=text frag=0 data= World

If you wanted to send the message length, you might send

 op=extension frag=1 ext=META-DATA(byte-length=11, char-length=11,
content-type=text/plain) data=Hello
 op=text frag=0 data= World

Or if you wanted to preserve the ability of intermediaries to see the
data you could send:

 op=extension frag=1 ext=META-DATA(byte-length=11, char-length=11,
content-type=text/plain)
 op=text frag=1 data=Hello
 op=text frag=0 data= World

Or if you wanted to hide the data from intermediaries that didn't know
the extension

 op=extension frag=1 ext=META-DATA(byte-length=11, char-length=11,
content-type=text/plain) data=Hello
 op=extension frag=0 ext= data= World

cheers

---

From: Patrick McManus <[email-masked]>
Date: Mon, 16 Aug 2010 11:21:12 -0400

On Mon, 2010-08-16 at 11:00 -0400, John Tamplin wrote:

> If you are already reallocating/copying the buffer it seems like you
> are getting little benefit from getting it after the first frame. 

a couple significant benefits to having TML at some early chunk vs not
having it at all:

* progress indicators
* abort early if tml is too large
* your next realloc() can be your last.

of course, tml in chunk 1 is optimal.

>  It also isn't clear when dynamically generated content is going to be
> known ahead of time.  

my point exactly! CPU and network speed mismatches create large
buffering all the time, and a write-once stream API isn't the only way
applications manage those buffers.

> I would expect a typical implementation 

but not an optimal implementation. The spec shouldn't restrict an
optimal implementation. A typical implementation could just send in
chunk 1 or not at all and the recvr could just ignore it if not in chunk
1. There is no requirement to send or use it at all.

> The main problem with moving it to per-frame extension data is that we
> have to define that extension data now if we want to have TML in the
> base protocol 

I have a different take. Defining a simple non controversial extension
up front is a very good way to get clients and servers implementing code
that handles the extension framework instead of having them just punt on
the topic all together as a quagmire. 

This has to be the simplest possible extension.

---

From: John Tamplin <[email-masked]>
Date: Mon, 16 Aug 2010 11:27:40 -0400

>  op=extension frag=0 ext= data= World

The problem I have with this approach is when you have multiple extensions
you make a long chain to process.  For example, you want to send total
message length on muxed messages:

op=extension, ext=MUX (channel id=1, type=META_DATA(..., type=text,
data="Hello world"))

As more extensions are defined, finding the data you want gets more
problematic.  Instead, if there is one extension data field, presumably an
implementation could compute pointers to specific the field for a specific
extension during the handshake when the extension is negotiated (there might
need to be another level of indirection if variable-length per-extension
data is supported).

Anyway, if others like your proposal I don't feel strongly enough against it
to object.

---

From: John Tamplin <[email-masked]>
Date: Mon, 16 Aug 2010 11:32:00 -0400

> This has to be the simplest possible extension.

Sure, but it requires defining how all sorts of extensions are going to be
supported.  For example, do we want to allow per-extension data to be
variable length, or will it be fixed up front (which would allow
precomputation of offsets into the extension data)?  How do we identify the
data associated with different extensions - are there registered extension
ids, tied to the order defined in the negotiation in the handshake, or
something else?  Would this be a special-case extension that is required to
be understood even if nothing is negotiated in the handshake, or would
require that all implementations wishing to use it have to negotiate it in
the handshake?

I think having even one extension defined in the base protocol means all
those decisions and more have to be decided now, rather than 6 months from
now.  Given the rate we have been achieving consensus so far, that seems
like it would unnecessarily delay getting the base protocol finalized.

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Mon, 16 Aug 2010 08:33:39 -0700

> >> that in a standardized meta data extension?

I think some on the list (Maciej?) were saying that if they don't know the
entire length up front, it makes buffer management more complex. I'm
personally rather neutral, that's the reason it was in there though.

> frame.  There is such an op-code in the current -00 proposal already.
I think I get it, sorry for misunderstanding. I think John has addressed
this in a follow-up message.

> could be done with just a extended-text and extended-binary op-code?
For things that switch rapidly between frames (e.g. channel/stream ID in
multiplexing), you don't want to have to inject a control message before
each message, and it also complicates things (requires you to store more
state) if the stream ID isn't in each message frame.

---

From: Dave Cridland <[email-masked]>
Date: Mon, 16 Aug 2010 16:44:20 +0100

>    uncompressible data won't expand too much

As an aside, the last two are closely related - that is, the second  
is almost a subset of the first, since if deflate (or zlib, or gzip)  
compression is negotiated, then the receiver needn't know that the  
sender is fiddling with the compression levels at all, as far as I  
understand things.

Dave.

---

From: Douglas Otis <[email-masked]>
Date: Mon, 16 Aug 2010 17:36:17 -0700

On 8/14/10 1:03 PM, Scott Ferguson wrote:
> clean without needing the extension length or the extension payload 
> and without the extra complete-length field. The frame would look like:
>
> 0                   1                   2                   3
> Jamie-style extension supports all the extension capabilities that 
> yours does, there's no added capabilities with the extra complexity 
> added by b,c,d.
>
> b)  sendBinaryMessage(byte []buffer, int offset, int length); // 
> binary single-buffer send
>
> A good frame design will support all 5 sender API styles and 4 
> receiver API styles in a straightforward/efficient manner. You can't 
> just pick one API that happens to make a frame proposal work.
>
> -- Scott

This offers a cleaner definition.  The higher level protocol could also 
define an initial frame as a TLV content declaration where the next 
frame (after the TLV) begins the payload.  Perhaps this is what is meant 
by extension?  Having this information in a defined place helps with 
content filtering. Perhaps one of the 5 opcode bits could signal the 
presence of the declaration.

-Doug

---

From: John Tamplin <[email-masked]>
Date: Tue, 17 Aug 2010 14:10:46 -0400

On Mon, Aug 16, 2010 at 11:27 AM, John Tamplin <[email-masked]> wrote:

> Anyway, if others like your proposal I don't feel strongly enough against
> it to object.
>

How about deferring the decision of exactly how to represent extensions
until later?  That would be

   - INI - 1 bit, indicates initial frame of message
   - FIN - 1 bit, indicates final frame of message
   - Reserved1 - 1 bit - must be 0, except for private use by external
   agreement
   - Opcode - 4 bits, 0=Control, 1=Text, 2=Binary, 3-11=Reserved for future
   definition, 12-15=Reserved for private use
   - Reserved2 - 1 bit - must be 0
   - Length - 7 bits, 127=uses extended length
   - Extended length - 8 bytes [if Length=127]
   - Total message length - 8 bytes [if INI=1 & FIN=0] -- all 0s means
   unknown
   - Payload - (Extended) Length bytes - interpretation defined by opcode

Later, we can decide to allocate one of the reserved bits adjacent to the
opcode as an Extension Data Present flag, as proposed originally in this
thread, or as an extra bit in the Opcode field if we decide that we want all
extensions to be defined with new opcodes (and nesting the final payload
opcode in the extension's payload).  That way we can defer deciding exactly
how extensions should be represented until we actually have an extension to
implement.

I can see rationale for your suggestion, but I still think some extensions
are ignorable (for example, an intermediary that doesn't understand the
MIME-type extension data could safely ignore it) -- maybe at the point we
define extension data formats we can define "critical" extension types which
must be understood or the message is altered vs "optional" extension types
("critical" terminology taken from the PNG spec about chunks).

Note that if we defer defining extension data, we can't move the total
message length to extension data, so I am leaving it where it is.

Would this remove remaining objections to this proposal?

---

From: John Tamplin <[email-masked]>
Date: Tue, 17 Aug 2010 18:13:15 -0400

On Tue, Aug 17, 2010 at 2:10 PM, John Tamplin <[email-masked]> wrote:

>    - Opcode - 4 bits, 0=Control, 1=Text, 2=Binary, 3-11=Reserved for
>    future definition, 12-15=Reserved for private use
> Reserved1 should be 2 bits wide of course.

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 18 Aug 2010 06:53:39 +0800

> Agreed. I would really like to avoid having so much features in the

Just to add another +1.

As I understand it, at the framing layer, all you need is a length count.  Pick a length and run with it, it doesn't really matter how big in the end.  That's all a framing layer really needs.

At the next layer, we can use opcodes to identify frame types:

 0 = this frame contains a message, and that message is binary
 1 = this frame contains a message, and that message is UTF-8 text

It would be possible, technically, to reserve the rest and ship a working protocol.

---

From: gustav trede <[email-masked]>
Date: Wed, 18 Aug 2010 01:05:17 +0200

On 18 August 2010 00:53, Thomson, Martin <[email-masked]> wrote:

> As I understand it, at the framing layer, all you need is a length count.
>  Pick a length and run with it, it doesn't really matter how big in the end.
>  That's all a framing layer really needs.
>

That sounds great, getting the essential stuff out the door and leave the
complex (and needed) but almost endless debates to later.

regards
  gustav

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Tue, 17 Aug 2010 16:11:26 -0700

It would also have been possible for HTTP to say you have an address that
maps to an IP address, you use TCP on port 80, and everything else is left
to a sub protocol. After all, that's all a transport protocol really needs.
However, that wouldn't really be useful because if there's something we
expect all implementations to want to know or do (e.g. have a standard way
to request a resource in HTTP) we want to standardize that in the protocol.
So, there is some tradeoff between saying "just specify a length and ship
it" and shipping something that is useful
for inter-operable implementations.

I think what we're trying to figure out here is where that threshold is.
Length only is too simplistic (e.g. IMO pushing the binary/text into a
sub-protocol would not fly), there's pushback that baking in how the
extensions' data is represented may be too detailed, and so I think people
like John are offering reasonable compromises that we should consider.

On Tue, Aug 17, 2010 at 4:05 PM, gustav trede <[email-masked]>wrote:

>
>
> On 18 August 2010 00:53, Thomson, Martin <[email-masked]>wrote:
>
>> Anne writes:

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 18 Aug 2010 07:50:51 +0800

That wasn’t the suggestion at all.  Binary/text was included.  The proposal was length + opcode.  Two opcodes only…for now.

The other stuff (fragmentation, channels, etc) are well enough understood now to say that they can be built in later. Hence, we reserve the bits and plan to build them later (or not at all).

The difference is that this approach is modular rather than monolithic (HTTP).  You get a working protocol for the use case we know about (the API).

There are some mandatory things for a base specification, and this has all of them (assuming of course that we can get a workable security option, I’ve not reviewed Adam’s proposal thoroughly enough yet, though superficially it looks reasonable).

--Martin

From: [email-masked] [mailto:[email-masked]] On Behalf Of Ian Fette (????????)
Sent: Wednesday, 18 August 2010 9:11 AM
To: gustav trede
Cc: [email-masked]
Subject: Re: [hybi] Framing Take VI (a compromise proposal)

It would also have been possible for HTTP to say you have an address that maps to an IP address, you use TCP on port 80, and everything else is left to a sub protocol. After all, that's all a transport protocol really needs. However, that wouldn't really be useful because if there's something we expect all implementations to want to know or do (e.g. have a standard way to request a resource in HTTP) we want to standardize that in the protocol. So, there is some tradeoff between saying "just specify a length and ship it" and shipping something that is useful for inter-operable implementations.

I think what we're trying to figure out here is where that threshold is. Length only is too simplistic (e.g. IMO pushing the binary/text into a sub-protocol would not fly), there's pushback that baking in how the extensions' data is represented may be too detailed, and so I think people like John are offering reasonable compromises that we should consider.
On Tue, Aug 17, 2010 at 4:05 PM, gustav trede <[email-masked]<mailto:[email-masked]>> wrote:

On 18 August 2010 00:53, Thomson, Martin <[email-masked]<mailto:[email-masked]>> wrote:
Anne writes:
> On Fri, 13 Aug 2010 23:16:14 +0200, Ian Hickson <[email-masked]<mailto:[email-masked]>> wrote:
> > 2010/8/12 Ian Fette (イアンフェッティ) <[email-masked]<mailto:[email-masked]>>:
> >>

Just to add another +1.

That sounds great, getting the essential stuff out the door and leave the complex (and needed) but almost endless debates to later.

regards
  gustav

_______________________________________________
hybi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hybi

---

From: John Tamplin <[email-masked]>
Date: Tue, 17 Aug 2010 20:05:26 -0400

On Tue, Aug 17, 2010 at 7:50 PM, Thomson, Martin
<[email-masked]>wrote:

>  That wasn’t the suggestion at all.  Binary/text was included.  The
> proposal was length + opcode.  Two opcodes only…for now.
>
>
>
> The other stuff (fragmentation, channels, etc) are well enough understood
> now to say that they can be built in later. Hence, we reserve the bits and
> plan to build them later (or not at all).
>

I think we do know that we have a use-case for fragmentation, as it has been
accepted as a requirement for this protocol.  As previously mentioned, the
inclusion of a total message length was required to remove objections to
fragmentation.  So, other than having two options for the frame length to
preserve efficient encoding of small messages, I think the current proposal
is about as minimal as it can get.

I think if we leave out fragmentation and planned extensibility from the
base protocol, we haven't really accomplished anything and we will either
wind up having to make incompatible changes later or having suboptimal
framing once we do define some extensions.

Let's say we did strip out fragmentation, and even defined the INI/FIN bits
as proposed, but required that they be 1 in the initial version.  Then, any
intermediaries and receivers that are written now will break when
fragmentation is added later, and experience shows that those intermediaries
will be very slowly upgraded.  I think that would add years to when
fragmentation could usefully be used, so I think it is entirely appropriate
that fragmentation support be in the base protocol.

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 18 Aug 2010 08:24:44 +0800

I think that you’ve misunderstood my suggestion as well.  I’m not suggesting that we avoid fragmentation (even if there is consensus, and I’ll defer to the chairs on this point).  What I’m suggesting is that fragmentation can be accommodated as an extension.

I am absolutely NOT suggesting that we avoid the topic of extensibility in the base - to do so would be foolish.

Repeating Jamie’s suggestion:

  <length> <opcode>

Define opcodes for fragmented messages (start, middle, end), then you have five opcodes:

  0 - this is a complete message, and it contains binary data
  1 - this is a complete message, and it contains UTF-8 text data
  2 - this is the start of a fragmented message
  3 - this is the middle of a fragmented message
  4 - this is the end of a fragmented message

For 2, 3 and 4, you follow that with another opcode - 0 or 1.

<length> <fragmentation opcode> <opcode>

You might practice a little judicious optimization, but fundamentally, opcodes 2-4 are simply extensions.  An intermediary that doesn’t understand just passes these on unmodified (that’s the extensibility bit you have to include).

--Martin

From: John Tamplin [mailto:[email-masked]]
Sent: Wednesday, 18 August 2010 10:05 AM
To: Thomson, Martin
Cc: [email-masked]; gustav trede; [email-masked]
Subject: Re: [hybi] Framing Take VI (a compromise proposal)

On Tue, Aug 17, 2010 at 7:50 PM, Thomson, Martin <[email-masked]<mailto:[email-masked]>> wrote:
That wasn’t the suggestion at all.  Binary/text was included.  The proposal was length + opcode.  Two opcodes only…for now.

I think we do know that we have a use-case for fragmentation, as it has been accepted as a requirement for this protocol.  As previously mentioned, the inclusion of a total message length was required to remove objections to fragmentation.  So, other than having two options for the frame length to preserve efficient encoding of small messages, I think the current proposal is about as minimal as it can get.

I think if we leave out fragmentation and planned extensibility from the base protocol, we haven't really accomplished anything and we will either wind up having to make incompatible changes later or having suboptimal framing once we do define some extensions.

Let's say we did strip out fragmentation, and even defined the INI/FIN bits as proposed, but required that they be 1 in the initial version.  Then, any intermediaries and receivers that are written now will break when fragmentation is added later, and experience shows that those intermediaries will be very slowly upgraded.  I think that would add years to when fragmentation could usefully be used, so I think it is entirely appropriate that fragmentation support be in the base protocol.

---

From: John Tamplin <[email-masked]>
Date: Tue, 17 Aug 2010 20:53:32 -0400

On Tue, Aug 17, 2010 at 8:24 PM, Thomson, Martin
<[email-masked]>wrote:

>  I think that you’ve misunderstood my suggestion as well.  I’m not
> suggesting that we avoid fragmentation (even if there is consensus, and I’ll
> defer to the chairs on this point).  What I’m suggesting is that
> fragmentation can be accommodated as an extension.
>
>
>
> You might practice a little judicious optimization, but fundamentally,
> opcodes 2-4 are simply extensions.  An intermediary that doesn’t understand
> just passes these on unmodified (that’s the extensibility bit you have to
> include).
>

Right, and before long we wind up with a long linear chain of extensions
that all have to be evaluated sequentially, and if an intermediary doesn't
know any one of them it can no longer do anything at all with the message,
which I think is a problem.  An intermediary that is just interested in one
extension's data (say a tag that indicates an ingres point to the corporate
network) has to decode all outer extensions before it can get the data it
wants.  Imagine how well routers would perform if IP fields were encoded
that way.

---

From: Scott Ferguson <[email-masked]>
Date: Tue, 17 Aug 2010 18:27:53 -0700

John Tamplin wrote:
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     The other stuff (fragmentation, channels, etc) are well enough
>     understood now to say that they can be built in later. Hence, we
>     reserve the bits and plan to build them later (or not at all). 
>
>
> I think we do know that we have a use-case for fragmentation, as it 
> has been accepted as a requirement for this protocol.  As previously 
> mentioned, the inclusion of a total message length was required to 
> remove objections to fragmentation.  So, other than having two options 
> for the frame length to preserve efficient encoding of small messages, 
> I think the current proposal is about as minimal as it can get.

A variable-length frame-header is not as minimal as it can get.

In your design, initial fragments have a 18 byte header (!), 
non-fragmented messages > 127 have 10 byte header, non-initial fragments 
are 10 bytes, and 127 byte messages (shorter than a tweet) have a 
special 2 byte header.

That is not a minimal proposal.

-- Scott

>
> I think if we leave out fragmentation and planned extensibility from 
> the base protocol, we haven't really accomplished anything and we will 
> either wind up having to make incompatible changes later or having 
> suboptimal framing once we do define some extensions.
>
> Let's say we did strip out fragmentation, and even defined the INI/FIN 
> bits as proposed, but required that they be 1 in the initial version. 
>  Then, any intermediaries and receivers that are written now will 
> break when fragmentation is added later, and experience shows that 
> those intermediaries will be very slowly upgraded.  I think that would 
> add years to when fragmentation could usefully be used, so I think it 
> is entirely appropriate that fragmentation support be in the base 
> protocol.
>
> -- 
> John A. Tamplin

---

From: Scott Ferguson <[email-masked]>
Date: Tue, 17 Aug 2010 19:03:34 -0700

John Tamplin wrote:
> On Sat, Aug 14, 2010 at 4:03 PM, Scott Ferguson <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Well, the complexity is more of a spec issue than a coding one
>     because none of these proposals are hugely complicated to
>     implement. (Implementing them efficiently is a different matter.)
>
>     A Jamie-style proposal less complex because it leaves the basic
>     frame clean without needing the extension length or the extension
>     payload and without the extra complete-length field. The frame
>     would look like:
>
>
>     * a receiver would prefer to allocate the buffer at once for a
>       message rather than having to reallocate/copy it, which means
>       the sender should send the complete message length with the
>       first fragment if it is available
>

Yes, but it shouldn't be in the frame itself. Any of the extension 
methods would be fine for the TML. Besides, your proposal assumes that 
an implementation is fragmenting when it knows the TML, which doesn't 
make sense. If it knows the TML, it can just send a big frame.

>     * having a short length that was "mostly enough" would encourage
>       buggy implementations that don't support or test the long length
>       case
>

That's not a good reason to use 127 bytes. Picking an unusable length 
just to force people to test it is silly.

If you're concerned about people not properly testing variable lenghts, 
use a fixed length of 64 (or 56 to make a nice 8-byte frame.)

(The weird scrambling of the hash in the handshake is silly for the same 
reason.)

> Regarding d, as mentioned above those arguing against fragmentation 
> objected to not having the full message length, and after proposing 
> that addition they seemed happy with the compromise.  As fragmentation 
> support has been accepted as a requirement

Yes, the TML needs to be passed as metadata, but it should not be in the 
frame. Nor should it be tied to fragmentation. The whole point of 
fragmentation is that you don't know the total length.

I'm okay with several options for passing the TML. Just not in the frame 
definition itself.

>  
>
>     As a separate issue, your code's signature doesn't capture the
>     essence of the problem, because you really need to consider all
>     the APIs when evaluating a framing proposal, not just one that
>     happens to be easy to implement for the proposal. Your example was
>     a specialized API that really only works for utf-8 encoded 8-bit
>     C-strings.
>
> length-based framing options, though supporting fragmentation means 
> you can have a fixed-size send buffer rather than having to allocate 
> all of it at once.

The difference between text and binary is that you know the frame length 
for binary immediately. With text, you do not. It's sufficiently 
different that you need to consider both signatures.
>  
>
>     a*) sendTextMessage(byte []buffer, int offset, int length); //
>     utf-8 encoded text (like PHP)
>
Correct, which is why your code sample was misleading, because it made 
the simplifying assumptions of binary, but pretended to be a solution 
for the text signature.
> Again, I don't see these as showing any differences between the 
> various framing proposals, and fragmentation makes it easier to 
> support streams as you just write a fragment when the buffer fills.

That's only true if the frame header size is fixed. If your frame header 
size is variable, you either need to shift the bytes in the buffer, or 
are forced to use something like writev with extra pointer metadata, 
instead of just preallocating the fixed frame header size.

Variable frame header sizes are bad.
>     b) byte []receiveBinaryMessage() // if returning utf-8 8-bit
>     strings, this is the text receive
>
> I would think there would also be an event-driven API, since that is 
> what the JS API is going to look like -- where there is an onMessage 
> event called with a complete message after it has been received.  The 
> only public server-side API I know of (Jetty7) also operates this way.

Yes, but event-based doesn't make a difference to the protocol, because 
you're either moving the string read by receiveTextMessage() or passing 
a notification to a stream-based receiver.
>     receiver API styles in a straightforward/efficient manner. You
>     can't just pick one API that happens to make a frame proposal work.
>
>
> Can you explain how your proposed framing is any different for any of 
> these APIs?  The difference is about the size of the short frame 
> length, leaving out the extension information which isn't going to be 
> used in the base protocol anyway, and leaving out the overall message 
> length which means you have to guess a buffer size for fragmented 
> messages and grow it if necessary (or fail the receive if using a 
> too-small caller-supplied buffer).

The 127 byte short frame length can only be used efficiently for the 
sendBinaryMessage(), because you know the total length.

For the stream-based binary sender, a simple implementation is to use a 
fixed-length buffer if you care about efficiency, and reserve the bytes 
for the frame header, which means its length needs to be fixed, no 
matter whether the message happened to be 4 bytes, 1024 bytes, or extend 
past the buffer size. It's the variable-length frame header that's a 
killer for streams.

In the case of your proposal, the stream writer (for an 8k fixed buffer) 
needs a different frame header pre-allocation for a message that's less 
than 127 bytes, from a message that's less than 8k - 10, from a message 
that overflows and goes into fragment mode. Basically, your proposal 
takes away a simple and efficient stream implementation, forcing more 
complicated implementations (like shifting or pointers or writev, etc.).

The stream-based text sender is essentially the same as the binary 
(although there is the additional complication of splitting the final 
character across buffers.)

For sendTextMessage(), if you want a single-pass encoding, you have the 
same issues as the stream based senders. You could calculate the length 
in a second pass, but that's an added inefficiency (which would almost 
certainly be more expensive than the end-to-end overhead of the 
additional byte.) In other words, it has it's own issues and trade-offs 
distinct from the other APIs.

-- Scott

---

From: John Tamplin <[email-masked]>
Date: Tue, 17 Aug 2010 22:44:21 -0400

On Tue, Aug 17, 2010 at 9:27 PM, Scott Ferguson <[email-masked]> wrote:

> I think we do know that we have a use-case for fragmentation, as it has
>> been accepted as a requirement for this protocol.  As previously mentioned,
>> the inclusion of a total message length was required to remove objections to

Variable length refers to having the two options for frame length.  If you
don't want variable-length length fields, then it seems the only option
would be an 8-byte length field.  I believe that would impose excessive
overhead on very small messages, and as I demonstrate below, I believe such
messages will be prevalent in real-world WebSocket apps.

In your design, initial fragments have a 18 byte header (!), non-fragmented
> messages > 127 have 10 byte header, non-initial fragments are 10 bytes, and
> 127 byte messages (shorter than a tweet) have a special 2 byte header.
>

Please clarify exactly what you are objecting to, as it seems you are mixing
different objections together.

   - Are you objecting to the efficiency concerns of having 18 bytes for
   initial fragments which are larger than 126 bytes?  If so, I think the
   overhead of an extra 8 bytes on a large message is small.  At absolute
   worst, fragmenting a 254 byte message into two 127 byte fragments would mean
   have 28 overhead bytes or 9.9% overhead.  For a reasonable but small buffer
   size of 4k, the worst case would be a 4223 (4096+127) byte message taking
   the same number of overhead bytes which is now only .6% overhead.  So, I
   disagree that this is unacceptable overhead.
   - Are you saying the small frames are not frequent enough to merit
   special consideration?  The data I posted here back in May regarding
   compression showed that 99.93% of uncompressed WebSocket payloads in GWT
   Quake client->server fit in 126 bytes (compression would increase that to
   99.98%), while server->client gets 30.34% of messages under that size (and
   compression gets it to 98.88%).  Wave gets only 1.21% under 126 bytes for
   client->server, but compression brings that to 98.22% and server->client is
   78.05% and 98.31% with compression.  So, in summary, I do think special
   handling for small frames is warranted and will be even more so once
   compression is supported.
   - Are you suggesting the proposal is too difficult to implement?

---

From: John Tamplin <[email-masked]>
Date: Tue, 17 Aug 2010 23:15:35 -0400

On Tue, Aug 17, 2010 at 10:03 PM, Scott Ferguson <[email-masked]> wrote:

> Yes, but it shouldn't be in the frame itself. Any of the extension methods
> would be fine for the TML. Besides, your proposal assumes that an
> implementation is fragmenting when it knows the TML, which doesn't make
> sense. If it knows the TML, it can just send a big frame.
>

One of the reasons for supporting fragmentation in the base protocol is to
support multiplexing.  You don't want lots of small messages blocked behind
one huge message, so you want the multiplexor to be able to fragment the
large message and interleave it with messages on other channels.  Thus, you
might very well take one very large message and split it into fragments
where you do know the TML.  Also, a client might also choose to write a
fragmented message in chunks as well, though I suspect they would write it
in one large message if it is known ahead of time.

Personally, I wasn't a big fan of including TML at all, but it seemed a
necessary concession to those objecting to receiver buffer management
complication due to fragmentation.  I personally would have to problem
leaving it out entirely if that didn't prevent us from achieving consensus
(which would also allow us to go to a single "MORE" bit for fragment
support).  I would also be willing to say that the TML be included in a
multiplexing extension, as that will be the primary use-case (as I see it,
anyway).

> That's not a good reason to use 127 bytes. Picking an unusable length just
> to force people to test it is silly.
>

As mentioned in the previous message, it is not an unusable length for
real-world WebSocket apps.  It also is appropriate to take advantage of
wireless network characteristics.

> If you're concerned about people not properly testing variable lenghts, use
> a fixed length of 64 (or 56 to make a nice 8-byte frame.)
>
>> objected to not having the full message length, and after proposing that
>> addition they seemed happy with the compromise.  As fragmentation support
>> has been accepted as a requirement
>>
>

See above for an example.

> for binary immediately. With text, you do not. It's sufficiently different
> that you need to consider both signatures.

I believe the UTF16->UTF8 conversion works essentially the same as a
compression filter, and as such I think the result is the same:

while input left {
   transform input to buffer, stopping when buffer full or input exhausted
   write buffer (using short header if < 127 bytes, long header otherwise)
}

> size is variable, you either need to shift the bytes in the buffer, or are
> forced to use something like writev with extra pointer metadata, instead of
> just preallocating the fixed frame header size.
>

You could leave room at the beginning of the buffer for the largest header,
and start the write at the correct offset when you flush the buffer.  Or if
you don't want to bother with that, just make two write calls.  Admittedly,
there is an efficiency cost there but implementations more concerned with
implementation simplicity than overall efficiency aren't likely to care.

> Variable frame header sizes are bad.

There are three options:

   - have a fixed-size header sufficient for all messages - inefficient for
   small messages
   - have a fixed-size header and use fragmentation for larger messages -
   requires all senders to support fragmentation, which was objected to
   - use variable-length headers and use an appropriate length for the size
   required (you can define the lengths in whatever steps you like, such as the
   v76 framing protocol which essentially had 9 steps using the high bit to
   indicate there were more frame length bits)

Each has pros and cons.  It seemed the objectors to the first two were more
entrenched -- for example, I would strongly object to a fixed 8-byte length
on the grounds of efficiency of small messages, but I would be happy with a
16-bit fixed length and requiring fragmentation.

> sendBinaryMessage(), because you know the total length.

I don't see why you can't decide which header to use when you flush the
buffer -- if it is under 127 bytes, you use the short header, otherwise you
use the long header.

> fixed-length buffer if you care about efficiency, and reserve the bytes for
> the frame header, which means its length needs to be fixed, no matter
> whether the message happened to be 4 bytes, 1024 bytes, or extend past the
> buffer size. It's the variable-length frame header that's a killer for
> streams.
>

Such a simple implementation could choose to send the long header if it
didn't care about efficiency of small frames.  It could also make two writes
or use scatter/gather APIs for the header buffer separate from the message
buffer.

A larger point is that with the proposals for extension using a nested
series of opcodes, the header in front of the final payload data is going to
be variable anyway.

> needs a different frame header pre-allocation for a message that's less than
> 127 bytes, from a message that's less than 8k - 10, from a message that
> overflows and goes into fragment mode. Basically, your proposal takes away a
> simple and efficient stream implementation, forcing more complicated
> implementations (like shifting or pointers or writev, etc.).
>

Let's say I wanted to use an 8k buffer to write a stream.  I could start
putting payload data at byte 18 (in the current proposal, 10 if TML is
removed).  When I finish writing data or hit the end of the buffer, I
generate the header in the 2-18 bytes before the start of that message and
write it in a single write.  Then I start filling the next buffer the same
way, etc.  No scatter/gather required, no shifting bytes around, etc.

---

From: Dave Cridland <[email-masked]>
Date: Wed, 18 Aug 2010 09:05:25 +0100

On Wed Aug 18 03:03:34 2010, Scott Ferguson wrote:
> That's not a good reason to use 127 bytes. Picking an unusable  
> length just to force people to test it is silly.
> 
> 
Quite. But having a "mini header" is very useful, and the useful  
packet-size for it to be happens, in currently deployed networks, to  
be 127 octets (including, I note, the IP, TCP and TLS headers).

At 128 octets, packets cost more than double for a mobile handset to  
receive, so long-running, low-volume mobile applications *really*  
benefit from this mini-header, since it allows vastly more usable  
data.

> If you're concerned about people not properly testing variable  
> lenghts, use a fixed length of 64 (or 56 to make a nice 8-byte  
> frame.)

Overheads are, mostly, insignificant compared to the volume of data.  
So an 18 byte header for the initial fragment of a message is  
noticeable, but still not large compared to the IP, TCP, and TLS  
overheads - it's adding a further third. Depending on fragment size,  
this could easily be very small indeed.

>> Regarding d, as mentioned above those arguing against  
>> fragmentation objected to not having the full message length, and  
>> after proposing that addition they seemed happy with the  
>> compromise.  As fragmentation support has been accepted as a  
>> requirement
> 
> Yes, the TML needs to be passed as metadata, but it should not be  
> in the frame. Nor should it be tied to fragmentation. The whole  
> point of fragmentation is that you don't know the total length.
> 
> I'm okay with several options for passing the TML. Just not in the  
> frame definition itself.
> 
> 
The problem there is that it coerces us into defining the actual  
format for metadata, rather than (as we're doing now) punting the  
decision of how metadata works by simply defining it as a (currently  
opaque) block.

We don't have time to be able to adequately investigate metadata  
formatting, unfortunately.

What we could do is define a mechanism for making the TML optional,  
however that would involve having to set aside an additional bit  
somewhere to indicate its presence. That's kind of what INI && !FIN  
means, mind, so we could go back to a FRAG, and recover a bit.

> That's only true if the frame header size is fixed. If your frame  
> header size is variable, you either need to shift the bytes in the  
> buffer, or are forced to use something like writev with extra  
> pointer metadata, instead of just preallocating the fixed frame  
> header size.
> 

Yeah, I took this position, but was convincingly argued down, on the  
basis that IP and TCP both have variable headers. I still find it  
counterintuitive that variable-length headers are fine.

I think the only case this makes things less efficient for the sender  
is if you're shipping binary directly, in which case you can always  
do a double-write (or write+sendfile), which, with TCP CORK, won't  
even hamper network efficiency.

> The 127 byte short frame length can only be used efficiently for  
> the sendBinaryMessage(), because you know the total length.
> 
> For the stream-based binary sender, a simple implementation is to  
> use a fixed-length buffer if you care about efficiency, and reserve  
> the bytes for the frame header, which means its length needs to be  
> fixed, no matter whether the message happened to be 4 bytes, 1024  
> bytes, or extend past the buffer size. It's the variable-length  
> frame header that's a killer for streams.

As above, I'm satisfied that this is not the case in the vast  
majority of cases.

In those few where it might make sense (like embedded stuff), I'd  
note that the frame header *could* be fixed size by not using the  
mini-header. One hopes an intermediary would reframe, mind, to gain  
the advantage.

Dave.

---

From: John Tamplin <[email-masked]>
Date: Wed, 18 Aug 2010 11:27:21 -0400

On Tue, Aug 17, 2010 at 11:15 PM, John Tamplin <[email-masked]> wrote:

> ... I would also be willing to say that the TML be included in a

I remembered now why I thought it should go in the base protocol.  Imagine
we have endpoints that only speak v1.0 of the protocol, which does not
include multiplexing.  They write their messages in single frames, so don't
have to deal with fragmentation while sending (but do on the receiving end
of course).

Later, we define a multiplexing extension.  Some intermediate network
decides to use the multiplexing extension to coalesce all these small
WebSocket messages going through their network.  To do that without
impacting latency, they need to fragment large messages so you don't wind up
starving the small messages.  At the exit point of their network, they demux
the packets back into separate streams.  However, unless they want a serious
buffering problem at the exit node, they don't want to defragment the
messages there, so they send it to the ultimate receiver still fragmented.
 If the TML is not included in the first fragment of the base protocol, then
the receiver has to reallocate/copy the buffer because it doesn't know how
big it is going to be.  Even if the MUX extension defined some way to pass
the TML to the end node, it would have to be modified to support this
extension even though it doesn't actually care about multiplexing.  Instead,
having the TML included in the first fragment in the base protocol means
these receivers can avoid buffer management problems even without knowing
anything about the MUX extension.