Subject: [hybi] Is it important to know frame length at the start of frame? (was: Re: Discontinuation of mux ...)

From: Takeshi Yoshino <[email-masked]>
Date: Fri, 7 Feb 2014 16:30:00 +0900

I remember that in early days of the standardization, somebody was
proposing (*) that we should have a message length field in the first
fragment to allow for allocating buffer for full message at the beginning.

On Thu, Jan 16, 2014 at 3:27 PM, "Martin J. Dürst"
<[email-masked]>wrote:

> On 2014/01/16 15:05, Yutaka Hirano wrote:
>
>> Hi Joakim,
>>
>> Thanks for the comment.
>>
>> Knowing the size of a frame at the start of the frame, traditional
>>
>>> websocket behavior, is desired, as it allows us to fast fail for the 1009
>>> (message too big) use case.
>>>
>>
>> Do you know how effective it is?
>>
>> Having no prior knowledge of frame size is undesirable to us.
>>
>> On the other hand, the frame sender have more freedom because they don't
>> have to know the size of the frame in front.
>>
>
> My understanding of why messages are split into frames was that this would
> allow to start a message without knowing its length. But if you don't know
> the size of a frame before sending it, then the right solution is to make
> it smaller. Frames are just arbitrary pieces of a message.
>

We should be clear if we want to make it able to
(a) know the length of WebSocket frame on receiving the header of each
fragment
(b) know the length of WebSocket message on receiving the first fragment

As the proposal (*) has been rejected and not in the RFC6455 header, (b) is
not true even in RFC6455. What Joakim said should be to keep (a) which is
true now in RFC6455.

But if a message is fragmented into smaller frames, we cannot reject (and
send 1009) the message on receiving the first frame. "Whether we can
fast-reject a big message on receiving some non-final frame" depends on how
the peer fragments a message.

In some plans discussed in the main thread, it's planned to drop frame
length info for efficiency. But something else such as the length field of
HTTP/2.0 DATA frame tells how much data the fragment (when multiple DATAs
convey a WebSocket frame, each DATA would be called fragments of a frame)
contains. So, we won't be able to know the length of fragments
(original/encapsulated WebSocket frames) on receiving each of them, but
still be able to know the length of sub-fragments (e.g. HTTP/2.0 data
frames).

I think this change is acceptable and shouldn't be considered as breaking
of RFC6455 semantics. Even in the mux extension I was editing (
http://tools.ietf.org/html/draft-ietf-hybi-websocket-multiplexing-11), I
made the same change. The length of the encapsulated frame is unknown until
receiving all the fragments of the encapsulating message. No one was
objecting to that when we were discussing the spec, IIRC.

---

From: Micheil Smith <[email-masked]>
Date: Wed, 12 Feb 2014 13:34:15 +0000

As far as I recall, this was also to enable more efficient reading and error checking. 
Prefixing with the frame size means you don’t need to scan every byte being 
received in order to find an end of frame delimiter.

This is a particularly useful thing when doing something like upstreaming on a
websocket server, where all you care about is the very start of the message, 
and the length, such that you can then just read() N bytes which is the size of 
your frame and just write() directly.

In other words, I agree on your reply Takeshi.

– Micheil

On 7 Feb 2014, at 7:30 am, Takeshi Yoshino <[email-masked]> wrote:

> I remember that in early days of the standardization, somebody was proposing (*) that we should have a message length field in the first fragment to allow for allocating buffer for full message at the beginning.
> 
> On Thu, Jan 16, 2014 at 3:27 PM, "Martin J. Dürst" <[email-masked]> wrote:
> My understanding of why messages are split into frames was that this would allow to start a message without knowing its length. But if you don't know the size of a frame before sending it, then the right solution is to make it smaller. Frames are just arbitrary pieces of a message.
> 
> (a) know the length of WebSocket frame on receiving the header of each fragment
> As the proposal (*) has been rejected and not in the RFC6455 header, (b) is not true even in RFC6455. What Joakim said should be to keep (a) which is true now in RFC6455.
> 
> But if a message is fragmented into smaller frames, we cannot reject (and send 1009) the message on receiving the first frame. "Whether we can fast-reject a big message on receiving some non-final frame" depends on how the peer fragments a message.
> 
> In some plans discussed in the main thread, it's planned to drop frame length info for efficiency. But something else such as the length field of HTTP/2.0 DATA frame tells how much data the fragment (when multiple DATAs convey a WebSocket frame, each DATA would be called fragments of a frame) contains. So, we won't be able to know the length of fragments (original/encapsulated WebSocket frames) on receiving each of them, but still be able to know the length of sub-fragments (e.g. HTTP/2.0 data frames).
> 
> I think this change is acceptable and shouldn't be considered as breaking of RFC6455 semantics. Even in the mux extension I was editing (http://tools.ietf.org/html/draft-ietf-hybi-websocket-multiplexing-11), I made the same change. The length of the encapsulated frame is unknown until receiving all the fragments of the encapsulating message. No one was objecting to that when we were discussing the spec, IIRC.
> 
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: Joakim Erdfelt <[email-masked]>
Date: Wed, 12 Feb 2014 06:49:37 -0700

For various application level APIs that depend on delivery of complete
messages, the need to know the frame level size at the start of the frame
becomes important for memory management.  Eclipse Jetty uses this
information to implement Java's javax.websocket API for limiting memory
use, along with lower level (WS+HTTP/1.1) protocol validation (close code
1009).

Since this group has a history (now) of making protocol decisions based on
existing public APIs (re Javascript WebSocket), I would entreat the members
here to think about the other public, industry standard, non-javascript,
WebSocket APIs already widely deployed.

--
Joakim Erdfelt <[email-masked]>
webtide.com <http://www.webtide.com/> - intalio.com/jetty
Expert advice, services and support from from the Jetty & CometD experts
eclipse.org/jetty - cometd.org

On Wed, Feb 12, 2014 at 6:34 AM, Micheil Smith <[email-masked]>wrote:

> Prefixing with the frame size means you don't need to scan every byte being
> - Micheil
> > My understanding of why messages are split into frames was that this
> would allow to start a message without knowing its length. But if you don't
> know the size of a frame before sending it, then the right solution is to
> make it smaller. Frames are just arbitrary pieces of a message.
> >
> > As the proposal (*) has been rejected and not in the RFC6455 header, (b)
> is not true even in RFC6455. What Joakim said should be to keep (a) which
> is true now in RFC6455.
> >
> > But if a message is fragmented into smaller frames, we cannot reject
> (and send 1009) the message on receiving the first frame. "Whether we can
> > I think this change is acceptable and shouldn't be considered as
> breaking of RFC6455 semantics. Even in the mux extension I was editing (

---

From: Zhong Yu <[email-masked]>
Date: Wed, 12 Feb 2014 10:00:44 -0600

Why not send the message in one frame? The frame length is then the
message length.

One argument for fragmentation is to insert control frames while
transmitting a long message. However that reasoning is not very
convincing:

PONG - as long as we are writing data, any data, the peer should see
that the connection is alive. It's unreasonable for the peer to think
that our end is unresponsive while it is receiving data from us.

PING - not necessary while we are writing data. If peer is
unresponsive, we'll learn that from TCP layer.

CLOSE - closing the connection in the middle of a message is abnormal
anyway, it's probably not important to do the close handshake.

Zhong Yu

On Fri, Feb 7, 2014 at 1:30 AM, Takeshi Yoshino <[email-masked]> wrote:
> I remember that in early days of the standardization, somebody was proposing
> (*) that we should have a message length field in the first fragment to
> allow for allocating buffer for full message at the beginning.
>
>> the size of a frame before sending it, then the right solution is to make it
>> smaller. Frames are just arbitrary pieces of a message.
>
>
> I think this change is acceptable and shouldn't be considered as breaking of
> RFC6455 semantics. Even in the mux extension I was editing
> (http://tools.ietf.org/html/draft-ietf-hybi-websocket-multiplexing-11), I

---

From: Zhong Yu <[email-masked]>
Date: Wed, 12 Feb 2014 10:09:36 -0600

Why does it matter if a frame (of unknown size) is composed of
fragments (of known sizes), w.r.t. memory management?

On Wed, Feb 12, 2014 at 7:49 AM, Joakim Erdfelt <[email-masked]> wrote:
> information to implement Java's javax.websocket API for limiting memory use,
> along with lower level (WS+HTTP/1.1) protocol validation (close code 1009).
>
> webtide.com - intalio.com/jetty
>> > <[email-masked]> wrote:
>> > is not true even in RFC6455. What Joakim said should be to keep (a) which is
>> > true now in RFC6455.
>> >
>> > breaking of RFC6455 semantics. Even in the mux extension I was editing

---

From: Joakim Erdfelt <[email-masked]>
Date: Wed, 12 Feb 2014 09:13:18 -0700

Scenario:
  Sending side uses a message stream based API to write out 1 websocket
message.
  The sending side's API is configured/implemented to buffer only 16k at a
time, writing out a frame of 16k for each buffer filled.
  The sending side writes out a 600k message.
  The receiving side is using a different API, one that is based on
complete messages.
  The receiving side is configured to not accept messages over 100k.
  The receiving side's API/implementation tosses a close code 1009 to the
sending side, per API.
  The receiving side's endpoint receives an error and close notification
indicating a protocol closure due to excessive message size from sending
side.

On Wed, Feb 12, 2014 at 9:00 AM, Zhong Yu <[email-masked]> wrote:

> proposing
> > On Thu, Jan 16, 2014 at 3:27 PM, "Martin J. Dürst" <
> [email-masked]>
> > wrote:
> >>
> would
> is
> breaking of

---

From: Peter Thorson <[email-masked]>
Date: Wed, 12 Feb 2014 10:19:11 -0600

It doesn’t. Multiple layers of fixed length framing don’t conceptually affect this sort of memory management. It does make the whole operation more complicated though.

---

From: Joakim Erdfelt <[email-masked]>
Date: Wed, 12 Feb 2014 09:19:55 -0700

Eliminating the size from the frame also eliminates the ability to
fast-fail on excessive messages.

Scenario (with current RFC6455 / WS+HTTP/1.1):
  Sending side starts to write out a single message of 100MB.
  Receiving side, using a whole message delivery based API is configured
for max messages of 20MB.
  Receiving side sees the 100MB frame size and automatically closes the
connection with 1009 and doesn't bother reading past the current network
buffer.

Scenario (without frame sizes):
  Sending side starts to write out a single message of 100MB.
  Receiving side, using a whole message delivery based API is configured
for max messages of 20MB.
  Receiving side reads till the 20MB mark, then fails the connection with
1009.  Wasting time, memory, and resources for no good reason.

I really don't see how a discussion of a protocol without sizes is even
going on.
I mean, even the IP layer and TCP layer have sizes for their frames.

On Wed, Feb 12, 2014 at 9:13 AM, Joakim Erdfelt <[email-masked]> wrote:

>> (and

---

From: Peter Thorson <[email-masked]>
Date: Wed, 12 Feb 2014 10:25:28 -0600

> Why not send the message in one frame? The frame length is then the

The message length might not be known or might not be fully buffer-able by the sending endpoint.

> unresponsive, we'll learn that from TCP layer.

It does provide some separation of concerns and the ability to do things like measure RTT during a long message.

> anyway, it's probably not important to do the close handshake.

It can be used to cleanly close a connection after a message has been sent but before it is completed. Say a client initiates a transfer then decides to leave halfway through (browser tab closes, etc). The client can indicate that it wants to close and that the in flight message should be discarded. Otherwise the client would have to either truncate the message (which might result in bogus data getting processed by the server) or just close the TCP connection (which can leave the connection in a bad state on the server and causes the server to have to report an unclean close to its application, which isn’t entirely accurate).

---

From: Zhong Yu <[email-masked]>
Date: Wed, 12 Feb 2014 10:31:14 -0600

On Wed, Feb 12, 2014 at 10:25 AM, Peter Thorson <[email-masked]> wrote:
> It can be used to cleanly close a connection after a message has been sent but before it is completed. Say a client initiates a transfer then decides to leave halfway through (browser tab closes, etc). The client can indicate that it wants to close and that the in flight message should be discarded. Otherwise the client would have to either truncate the message (which might result in bogus data getting processed by the server) or just close the TCP connection (which can leave the connection in a bad state on the server and causes the server to have to report an unclean close to its application, which isn't entirely accurate).

I don't quite buy this argument. Browsers initial TCP FIN all the
time, to shutdown persistent HTTP connections for example.

---

From: Zhong Yu <[email-masked]>
Date: Wed, 12 Feb 2014 10:37:04 -0600

The sending side does not need to respond with a CLOSE in this
scenario. It is in the middle of sending a huge message in one huge
frame, while it gets a message from the peer saying that the peer is
no longer interested in the message. The sending side can simply drop
the TCP connection here.

On Wed, Feb 12, 2014 at 10:13 AM, Joakim Erdfelt <[email-masked]> wrote:
>   The receiving side is using a different API, one that is based on complete
> messages.

---

From: Tobias Oberstein <[email-masked]>
Date: Wed, 12 Feb 2014 08:41:46 -0800

> Why does it matter if a frame (of unknown size) is composed of fragments
> (of known sizes), w.r.t. memory management?

I agree with Joakim that frame size length can be important for memory management.

It is also important to be able to fragment a message, for reasons of

- control frame injection (ordered close)
- application data independent keep alive (ping/pong)
- message interleaving

/Tobias

> > messages, the need to know the frame level size at the start of the
> > frame becomes important for memory management.  Eclipse Jetty uses
> > this information to implement Java's javax.websocket API for limiting
> > memory use, along with lower level (WS+HTTP/1.1) protocol validation
> (close code 1009).
> >
> > Since this group has a history (now) of making protocol decisions
> > based on existing public APIs (re Javascript WebSocket), I would
> > entreat the members here to think about the other public, industry
> > standard, non-javascript, WebSocket APIs already widely deployed.
> >
> >
> > --
> > Expert advice, services and support from from the Jetty & CometD
> > experts eclipse.org/jetty - cometd.org
> >
> >
> > On Wed, Feb 12, 2014 at 6:34 AM, Micheil Smith
> > <[email-masked]>
> > wrote:
> >>
> >> Prefixing with the frame size means you don't need to scan every byte
> >> being received in order to find an end of frame delimiter.
> >>
> >> This is a particularly useful thing when doing something like
> >> upstreaming on a websocket server, where all you care about is the
> >> very start of the message, and the length, such that you can then
> >> just read() N bytes which is the size of your frame and just write()
> >> directly.
> >>
> >> > proposing (*) that we should have a message length field in the
> >> > first fragment to allow for allocating buffer for full message at the
> beginning.
> >> >
> >> > websocket behavior, is desired, as it allows us to fast fail for
> >> > the
> >> > 1009
> >> > On the other hand, the frame sender have more freedom because they
> >> > don't have to know the size of the frame in front.
> >> >
> >> > My understanding of why messages are split into frames was that
> >> > this would allow to start a message without knowing its length. But
> >> > if you don't know the size of a frame before sending it, then the
> >> > right solution is to make it smaller. Frames are just arbitrary pieces of a
> message.
> >> >
> >> > As the proposal (*) has been rejected and not in the RFC6455
> >> > header, (b) is not true even in RFC6455. What Joakim said should be
> >> > to keep (a) which is true now in RFC6455.
> >> >
> >> > But if a message is fragmented into smaller frames, we cannot
> >> > reject (and send 1009) the message on receiving the first frame.
> >> > "Whether we can fast-reject a big message on receiving some
> >> > non-final frame" depends on how the peer fragments a message.
> >> >
> >> > In some plans discussed in the main thread, it's planned to drop
> >> > frame length info for efficiency. But something else such as the
> >> > length field of
> >> > HTTP/2.0 DATA frame tells how much data the fragment (when multiple
> >> > DATAs convey a WebSocket frame, each DATA would be called
> fragments
> >> > of a frame) contains. So, we won't be able to know the length of
> >> > fragments (original/encapsulated WebSocket frames) on receiving
> >> > each of them, but still be able to know the length of sub-fragments
> >> > (e.g. HTTP/2.0 data frames).
> >> >
> >> > (http://tools.ietf.org/html/draft-ietf-hybi-websocket-multiplexing-
> >> > 11), I made the same change. The length of the encapsulated frame
> >> > is unknown until receiving all the fragments of the encapsulating
> message. No one was objecting to that when we were discussing the spec,
> IIRC.
> >> >

---

From: Zhong Yu <[email-masked]>
Date: Wed, 12 Feb 2014 10:52:30 -0600

> - application data independent keep alive (ping/pong)

Fragmentation is certainly necessary for mux. But if we are talking
about a simple RFC6455 connection, it seems to me that it's ok to send
a message (of known size) in one frame, regardless of its size.

---

From: Takeshi Yoshino <[email-masked]>
Date: Thu, 13 Feb 2014 13:14:18 +0900

> like measure RTT during a long message.

There can be an intermediary which silently terminates TCP and relay
WebSocket traffic. In such case, app level request for pong is useful for
liveness detection. The intermediary still can relay unresponsiveness of
the peer endpoint to the endpoint (by disconnecting the TCP connection) and
maybe it needs to do so due to limited buffer. So, I admit that this
argument is not so strong support for necessity of PING.

---

From: Takeshi Yoshino <[email-masked]>
Date: Thu, 13 Feb 2014 13:38:21 +0900

Joakim,

In my post, I was doubting necessity of frame length header IN THE CONTEXT
OF WebSocket over HTTP/2.0 discussion as the HTTP/2.0 DATA frame also has a
frame length that is immediately decodable and you can choose to send a big
one DATA frame to send a big message to tell the peer that it's a big frame
at the start of arrival of the frame. Not doubting necessity of frame
length in general.

However, I noticed that I didn't consider that the size of the frame length
field of the current HTTP/2.0 spec is very small. It's 14 bit. Old SPDYs
used 24 bits (and the latest SPDY/4 uses 16 bits).

So, without help by WebSocket level length field, even by using an
un-fragmented WebSocket frame (at WebSocket semantics level), we cannot
tell the peer that "we're sending a message which is N (N > 16k) bytes
long" on the first DATA frame. Sorry that I didn't notice this point.

On Thu, Feb 13, 2014 at 1:19 AM, Joakim Erdfelt <[email-masked]> wrote:

> On Wed, Feb 12, 2014 at 9:13 AM, Joakim Erdfelt <[email-masked]>wrote:
>
>> Scenario:
>>   The sending side's API is configured/implemented to buffer only 16k at
>> a time, writing out a frame of 16k for each buffer filled.
>>> don't
>>> DATAs

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 18 Feb 2014 14:37:49 +0900

So, to be clear, Joakim, you're not objecting to make it unable to know the
size of each frame, but objecting to make it completely unable to know the
size of big messages on receiving the head of it, right?

Your use case (fail-fast) seems to require the sender to send messages
unfragmented. So, it sounds like application (or kind of application)
specific needs. You could do some application specific hack such as
introducing "complete-message-size" header in the payload than depending on
protocol level length header. If not, clients in the wild which are out of
your control may do arbitrary fragmentation. FYI, Chromium's WebSocket
stack is being refactored and after that it starts sending big messages as
fragmented into small frames.

---

From: Joakim Erdfelt <[email-masked]>
Date: Tue, 18 Feb 2014 11:08:33 -0700

I'm not sure I was clear, as you seem to still have questions and are
hinting at alternatives.

The most important thing I said:
   The size is important at the start of any payload about to be received.

All other statements are based off of that.
The nuanced questions you are asking seems to indicate that you treat a
frame differently than a message is irrelevant.
If the frame happens to represent a complete message (eg:
opcode=(text|binary) + fin=true) vs being only 1 frame belonging to a
series of frames that make up a whole message makes no difference to the
statement.

Since websocket is message based, not stream based, the above core
statement remains true.

The fail-fast example applies for fragmented or whole messages.

some pseudo code:

   header = frame.parse_header(network_bytes)
   if (header.size > system_memory_constraints) then
      fail_protocol(1009)
      return
   end
   // is message still being collected and not yet delivered to local
endpoint?
   if (message.delivered == false) then
      if (message.size + header.size > message.max_constraints) then
        fail_protocol(1009)
        return
      else
        payload = frame.read_payload(network_bytes)
        message.append(payload)
      end
   end
   if (header.fin == true) then
      message.deliver_to(local_endpoint)
      message.reset
   end

On Mon, Feb 17, 2014 at 10:37 PM, Takeshi Yoshino <[email-masked]>wrote:

> So, to be clear, Joakim, you're not objecting to make it unable to know
> the size of each frame, but objecting to make it completely unable to know
> the size of big messages on receiving the head of it, right?
>

---

From: Joakim Erdfelt <[email-masked]>
Date: Tue, 18 Feb 2014 11:12:05 -0700

Clicked too soon on that pseudo code.
This is more clear. (i hope)

      // do not read payload
      // do not read payload
      // free up any message resources, reset message size to 0
      message.reset
   end

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 19 Feb 2014 12:59:10 +0900

Thanks, Joakim. So you're still be able to do the fail-fast without
WebSocket frame length header if we adopt HEADERS+DATA+END_SEGMENT approach
but with WebSocket frame length header eliminated. HTTP/2.0 DATA frame has
its own payload length field though it's small compared to the range
WebSocket header can express.

Takeshi

On Wed, Feb 19, 2014 at 3:12 AM, Joakim Erdfelt <[email-masked]> wrote:

> On Tue, Feb 18, 2014 at 11:08 AM, Joakim Erdfelt <[email-masked]>wrote:
>

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 25 Feb 2014 03:28:32 +0900

Thanks all for your comments.

First, please note that I wanted to discuss the question in the context of
WS/HTTP/2.0 layering. Not about redesign of RFC 6455 itself.

I asked this question since the length header is the biggest field when
encoded into HTTP/2.0 HEADERS format. Whether we can eliminate it or not in
WS/HTTP/2.0 is one of the most important points to evaluate the plans
employing HTTP/2.0 HEADERS.

Though the ranges the length headers represent differ (HTTP/2.0: 14 bit, WS
length format: 63 bit), it seems everyone can live without a header of
"original WebSocket frame size".

Please reply to this post if you have any objection, but please don't write
objection to use of HTTP/2.0 HEADERS itself in this thread.

Thanks

---

From: Joakim Erdfelt <[email-masked]>
Date: Mon, 24 Feb 2014 11:43:11 -0700

Let me see if I grok this entirely...

You are saying that HTTP/2.0 HEADERS has a 14 bit value already.
You would like to utilize this existing 14 bit value to represent websocket
payload data.
You would like to avoid having to adjust or supplement HTTP/2.0 HEADERS
just for WebSocket behaviors.
You are indicating that this 14 bit value is not a 1 to 1 representation of
"original WebSocket frame size" as determined by various APIs or extensions
before hitting the HTTP/2.0 layer.
You hint at the behavior of large (over 14 bit length) "original WebSocket
frames" being likely sent in multiple HTTP/2.0 DATA/END_SEGMENT portions.

Wouldn't this also mean that any "original WebSocket frame" length
information not being present on the receiving side. right?
The receiving side would just see HTTP/2.0 DATA/END_SEGMENT with some
indicator that the DATA is for WebSocket, right?
The receiving side would never see the "original WebSocket frame" as this
information is now lost.  Is that correct?
Instead you have HTTP/2.0 DATA length header for WebSocket payload data,
along with an END_SEGMENT indicating the separation between WebSocket
messages.

On Mon, Feb 24, 2014 at 11:28 AM, Takeshi Yoshino <[email-masked]>wrote:

> Though the ranges the length headers represent differ (HTTP/2.0: 14 bit,
> WS length format: 63 bit), it seems everyone can live without a header of
> Please reply to this post if you have any objection, but please don't
> write objection to use of HTTP/2.0 HEADERS itself in this thread.
>
> Thanks
>

---

From: Roberto Peon <[email-masked]>
Date: Mon, 24 Feb 2014 10:59:24 -0800

As I understand it:

If the previous DATA frame on the stream included END_SEGMENT, and the
current DATA frame on the stream includes END_SEGMENT, then the HTTP2 frame
length == the websocket frame length.
This is extremely likely to be the common case.

If the websocket frame needs to be broken up (generally only because it has
a large payload), then some other signaling of the frame length is
necessary, e.g. by using an HTTP/2 HEADERS frame with content-length key.
-=R

On Mon, Feb 24, 2014 at 10:43 AM, Joakim Erdfelt <[email-masked]> wrote:

> You would like to utilize this existing 14 bit value to represent
> websocket payload data.
> You are indicating that this 14 bit value is not a 1 to 1 representation
> of "original WebSocket frame size" as determined by various APIs or
> extensions before hitting the HTTP/2.0 layer.
>> First, please note that I wanted to discuss the question in the context
>> of WS/HTTP/2.0 layering. Not about redesign of RFC 6455 itself.
>>

---

From: Zhong Yu <[email-masked]>
Date: Mon, 24 Feb 2014 14:29:04 -0600

It there any problem if we mandate that 1 http frame corresponds to 1
ws frame, and 1 ws frame must reside in 1 http frame? What's the point
of supporting ws frames that need multiple http frames?

---

From: Zhong Yu <[email-masked]>
Date: Mon, 24 Feb 2014 14:32:53 -0600

On Mon, Feb 24, 2014 at 12:43 PM, Joakim Erdfelt <[email-masked]> wrote:
> You would like to avoid having to adjust or supplement HTTP/2.0 HEADERS just
> for WebSocket behaviors.

I never understand why APIs expose the concept of frames. It's like
TCP APIs exposing the concept of TCP segments.

---

From: Roberto Peon <[email-masked]>
Date: Mon, 24 Feb 2014 12:39:12 -0800

The browser API doesn't have a limit on the size of a message, so the
layers below that API should probably support it.
It would be incomplete not to, for sure.

As for why someone would do it? Who knows. Perhaps someone is exploiting
the fact that WS stuff never gets put into a cache for security reasons? Or
perhaps because using the API is easier and better expresses the
application semantics, even if they're not doing web browsing? Any of these
seem reasonable. There is no requirement for WS to include only small
messages...

-=R

On Mon, Feb 24, 2014 at 12:29 PM, Zhong Yu <[email-masked]> wrote:

> of
> WS
> write

---

From: Zhong Yu <[email-masked]>
Date: Mon, 24 Feb 2014 14:58:35 -0600

Uh? I was talking about ws frames, not ws messages. We can always
break up big frames into small frames (unless frame boundary has any
semantics to anyone).

On Mon, Feb 24, 2014 at 2:39 PM, Roberto Peon <[email-masked]> wrote:
> The browser API doesn't have a limit on the size of a message, so the layers
> below that API should probably support it.
> As for why someone would do it? Who knows. Perhaps someone is exploiting the
> fact that WS stuff never gets put into a cache for security reasons? Or
> perhaps because using the API is easier and better expresses the application
> semantics, even if they're not doing web browsing? Any of these seem
> reasonable. There is no requirement for WS to include only small messages...
>
> -=R
>
>

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 25 Feb 2014 06:23:36 +0900

> of supporting ws frames that need multiple http frames?

That's not what I'd like to discuss in this thread. I wanted to get
agreement on that HTTP/2.0 level length + END_SEGMENT is enough regardless
of whether we keep WebSocket frame boundary or not.

So, let's discuss that in another thread.

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 25 Feb 2014 06:31:46 +0900

On Tue, Feb 25, 2014 at 3:59 AM, Roberto Peon <[email-masked]> wrote:

> If the websocket frame needs to be broken up (generally only because it
> has a large payload), then some other signaling of the frame length is
> On Mon, Feb 24, 2014 at 10:43 AM, Joakim Erdfelt <[email-masked]>wrote:
>
Yes. Using the 14 bit length combined with the END_SEGMENT signal, we can
represent a block by sequence of HTTP/2.0 DATA frames.

As Roberto said, the simplest case is (for simplicity, padding in DATA
frames is assumed to be 0):

DATA(length=n, END_SEGMENT(2nd bit in flags)=true, Data=payload data of n
bytes)

Where Data field contains one byte to represent FIN, RSV1-3 and opcode. Or,
instead we can send HEADERS before the DATA which contains FIN, RSV1-3 and
opcode header since they would compress well compared to length header.

If payload is large, a sequence

DATA(length=s_0, END_SEGMENT=false, Data=payload data of s_0 bytes),
DATA(length=s_1, END_SEGMENT=false, Data=payload data of s_1 bytes),
...
DATA(length=s_n, END_SEGMENT=true, Data=payload data of s_n bytes)

represents the original frame. Where the first one byte in the
concatenation of Data fields has FIN, RSV1-3 and opcode, or instead we can
send HEADERS before the sequence of DATAs which contains FIN, RSV1-3 and
opcode.

>> just for WebSocket behaviors.
Yes. I'm trying to use what we have now as-is.

>> extensions before hitting the HTTP/2.0 layer.
Right

>  You hint at the behavior of large (over 14 bit length) "original
>> WebSocket frames" being likely sent in multiple HTTP/2.0 DATA/END_SEGMENT
>> portions.
>>
>
Right

>> information not being present on the receiving side. right?
 Not present on receiving the first DATA, but will be revealed on receiving
the length header of the last DATA frame (with END_SEGMENT on).

>> indicator that the DATA is for WebSocket, right?
I don't quite understand this question. It's already known that DATAs
contain WebSocket traffic by ":scheme" in the opening handshake.

>> information is now lost.  Is that correct?
END_SEGMENT tells where the boundary of the frames was.

>> along with an END_SEGMENT indicating the separation between WebSocket
That depends on another topic. If we separate messages with END_SEGMENT,
then yes. If we separate frames with END_SEGMENT, then no.

I'd like to get consensus on that length header in HTTP/2.0 DATA frame is
sufficient. It's a topic independent from whether we preserve frame
boundary or not.

[1] http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-00