Subject: [hybi] how do we move forward on agreeing on framing?

From: John Tamplin <[email-masked]>
Date: Thu, 19 Aug 2010 01:28:28 -0400

There have been various discussions about various proposals, and to be sure
there will be no proposal that everyone is happy with.  What can we do to
come to some consensus on something that is acceptable for the base protocol
so we can move forward?

To me, I think the following things are critical:

   - fragmentation support - this has been accepted as a requirement without
   much dissension when it was declared that we achieved consensus on it, and
   if we don't have support for it in the base protocol it will greatly
   complicate efforts for certain extensions a number of people think will be
   important in the future, such as multiplexing
   - low overhead for small frames - as previously indicated, real-world
   WebSocket apps do make use of a substantial percentage of small messages,
   and when compression is added the percentage grows to an overwhelming
   majority.  Sure, there will be some WebSocket apps which won't get any
   benefit from small frames, but that shouldn't remove the capability from
   those that do.
   - we can't afford to decide on the extension format now - given the rate
   of building consensus, we won't ever finish the base protocol if we have to
   define that now.  That means anything that is critical to support in the
   base protocol, such as fragmentation or (if we decide it is important for
   implementations of the base protocol) the total message length, cannot be
   defined in an extension.
   - we need to leave our options open for extensibility - since we don't
   yet agree on exactly how the protocol should be extended or exactly what
   facilities such extensions may need, we need to have some flexibility for
   how those extensions can be represented

Does anyone disagree with these points?

So here is another attempt at incorporating feedback to try and get
something which could gain consensus as the framing protocol:

   - MORE - 1 bit - if set, this is not the final frame of the message
   - TML - 1 bit - if set, the total message length is included (which is
   not required to be on the first frame, and if repeated overrides earlier
   values so it may be used as an estimate for dynamic content)
   - Reserved - 2 bits - must be 0 except for private use
   - Opcode - 4 bits - 0=Continuation, 1=Control, 2=Text, 3=Binary,
   4-11=Reserved for future assignment, 12-15=Reserved for private use
   - Reserved - 1 bit - must be 0
   - Length - 7 bits - payload length in octets; if 127, then there is an
   extended length
   - Extended length - 8 bytes - payload length in octets (high bit must be
   0 for ease of use with languages without an unsigned data type), present
   only if Length=127
   - Total Message Length - 8 bytes - total message length in octets (high
   bit must be 0 as above), present only if TML=1
   - Payload - (Extended) Length bytes - payload, interpreted as defined by
   the Opcode

This addresses the concern about requiring the total message length to be in
the first frame or to be present at all if it is unknown.  It allows us to
use a bit to indicate extension data is present, or to define extensions by
new opcodes (and to grow the opcode field if necessary by using adjacent
reserve bits).  Some bits are available for direct assignment, such as if we
find frame-by-frame compression a useful future extension. It also cleans up
room for confusion in the previous proposal where you could change opcodes
between fragments.

Example messages:

   - MORE=0,TML=0,Opcode=Text, Length=12, Payload="Hello world!" -
   single-frame text message

   - MORE=1,TML=1,Opcode=Text, Length=5, MsgLength = 12, Payload="Hello" -
   first frame of fragmented message
   - MORE=0,TML=0,Opcode=Continuation, Length=7, Payload=" world!" - final
   frame of fragmented message

   - MORE=1,TML=1,Opcode=Text, Length=127, ExtLength=4096, MsgLength=16000,
   Payload=... - first frame of dynamically generated message, with estimated
   message length
   - MORE=1,TML=0,Opcode=Continuation, Length=127,
   ExtLength=4096,Payload=... - several frames delivering dynamic content
   - MORE=1,TML=1,Opcode=Continuation, Length=127, ExtLength=4096,
   MsgLength=14781, Payload=... - sender now knows length of dynamic content
   and sends it
   - MORE=0,TML=0,Opcode=Continuation, Length=127, ExtLength=2493,
   Payload=... - final frame of message

Extension Possibilities:

   - if we decide creating new opcodes for extensions, which then nest the
   next layer of extension within their payload data, we have room to allocate
   additional opcodes, and we can grow the opcode space if we need to
   - if we decide to include per-frame extension data, we can allocate a
   reserved bit to indicate the presence of an extension length which describes
   the length of extension data
   - if we decide to include per-frame compression, we can allocate a
   reserved bit to indicate that this frame is compressed
   - we can allocate new control frame types

Is this something we can all get behind, even if it doesn't do everything we
want, in the interest of getting something useful done soon?  If not, what
specific changes would be necessary for you to agree?

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Dave Cridland <[email-masked]>
Date: Thu, 19 Aug 2010 08:43:03 +0100

> specific changes would be necessary for you to agree?

I'd feel more comfortable with knowing that the extension framing  
(not actual support) was done. Otherwise, any extension mechanism is  
going to smell like a real hack, even if it's possible. I also worry  
about actors seeing extensions and choking on them, whether that's  
intermediaries or whatever. Finally, it's not clear to me if  
extension data needs to be per-message or per-frame - I can't think  
of a per-frame use-case, but still, this effects possible extension  
mechanisms heavily.

It could be my fears are misplaced, of course, and I'm willing to be  
persuaded, but this is my current stance.

Aside from this one issue, this design - or the previous one, in fact  
- are fine with me. I marginally prefer this one, modulo the absence  
of an extension data block.

It'd be useful at this point, though, to produce a new draft  
containing this design, which seems to have broad support if not yet  
actual consensus - ie, this should become the group's current working  
design.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 19 Aug 2010 07:38:37 -0700

On Thu, Aug 19, 2010 at 12:43 AM, Dave Cridland <[email-masked]> wrote:

> I'd feel more comfortable with knowing that the extension framing (not
> actual support) was done. Otherwise, any extension mechanism is going to
> smell like a real hack, even if it's possible. I also worry about actors
> seeing extensions and choking on them, whether that's intermediaries or
> whatever. Finally, it's not clear to me if extension data needs to be
> per-message or per-frame - I can't think of a per-frame use-case, but still,
> this effects possible extension mechanisms heavily.
>
>
There are different definitions of "need", but I think the people interested
in multiplexing have made a good case that that's extension data that should
be present per-frame assuming that a message gets fragmented. (The "control
frame" argument that switches the state of the stream did not get people
excited.)

I also would be a bit happier if we had a better sense for how the
extensibility worked. For instance, let's say we have an intermediary that
is unaware of extensions, and so it re-fragments a frame that contained e.g.
a stream ID, and now all of a sudden we have one frame that contains an
stream ID and another frame that doesn't. Oops. The receiver is probably
going to choke. If you have to deal with this, then you have to keep state
around and now have all the problems that were brought up with trying to do
multiplexing via a control frame.

That said, I don't think anyone is going to rush to produce intermediaries
for WS, so maybe we do have some time (6m timeframe) to figure some of that
out and circle back.

-Ian

> Aside from this one issue, this design - or the previous one, in fact - are
> fine with me. I marginally prefer this one, modulo the absence of an
> extension data block.
>
> It'd be useful at this point, though, to produce a new draft containing
> this design, which seems to have broad support if not yet actual consensus -
> ie, this should become the group's current working design.
>
> Dave.
> --
> Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]<[email-masked]>
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Francis Brosnan Blazquez <[email-masked]>
Date: Thu, 19 Aug 2010 17:30:03 +0200

Hi John,

Some thoughts...

> There have been various discussions about various proposals, and to be
> sure there will be no proposal that everyone is happy with.  What can
> we do to come to some consensus on something that is acceptable for
> the base protocol so we can move forward?

My impression is that the base protocol should by simple, as close as
possible to TCP. For example, your proposal, which is length limited
looks good.

>       * fragmentation support - this has been accepted as a
>         requirement without much dissension when it was declared that
>         we achieved consensus on it, and if we don't have support for
>         it in the base protocol it will greatly complicate efforts for
>         certain extensions a number of people think will be important
>         in the future, such as multiplexing

I think it is taking to much attention to include advanced features in
the base protocol, or to define extensions that allow them, especially
when those features can be implemented by protocols already defined like
BEEP [1]. 

I'm sure many people is expecting to access to a simple protocol that
allows direct connection and bidirectional content transfer from the
browser. Nothing else. 

>       * low overhead for small frames - as previously indicated,
>         real-world WebSocket apps do make use of a substantial
>         percentage of small messages, and when compression is added
>         the percentage grows to an overwhelming majority.  Sure, there
>         will be some WebSocket apps which won't get any benefit from
>         small frames, but that shouldn't remove the capability from
>         those that do.
>       * we can't afford to decide on the extension format now - given
>         the rate of building consensus, we won't ever finish the base
>         protocol if we have to define that now.  That means anything
>         that is critical to support in the base protocol, such as
>         fragmentation or (if we decide it is important for
>         implementations of the base protocol) the total message
>         length, cannot be defined in an extension.

+1

>       * we need to leave our options open for extensibility - since we
>         don't yet agree on exactly how the protocol should be extended
>         or exactly what facilities such extensions may need, we need
>         to have some flexibility for how those extensions can be
>         represented

+1 

It would be great to define base protocol as something simple, a common
base factor, where consensus is more easy. A protocol that allow to
build on instead of including features that you may not use.

>       * MORE - 1 bit - if set, this is not the final frame of the
>         message
>       * TML - 1 bit - if set, the total message length is included
>         (which is not required to be on the first frame, and if
>         repeated overrides earlier values so it may be used as an
>         estimate for dynamic content)
>       * Reserved - 2 bits - must be 0 except for private use
>       * Opcode - 4 bits - 0=Continuation, 1=Control, 2=Text, 3=Binary,
>       * Reserved - 1 bit - must be 0
>       * Length - 7 bits - payload length in octets; if 127, then there
>         is an extended length
>       * Extended length - 8 bytes - payload length in octets (high bit
>         must be 0 for ease of use with languages without an unsigned
>         data type), present only if Length=127
>       * Total Message Length - 8 bytes - total message length in
>         octets (high bit must be 0 as above), present only if TML=1
>       * Payload - (Extended) Length bytes - payload, interpreted as
>         defined by the Opcode
> This addresses the concern about requiring the total message length to
> be in the first frame or to be present at all if it is unknown.  It
> allows us to use a bit to indicate extension data is present, or to
> define extensions by new opcodes (and to grow the opcode field if
> necessary by using adjacent reserve bits).  Some bits are available
> for direct assignment, such as if we find frame-by-frame compression a
> useful future extension. It also cleans up room for confusion in the
> previous proposal where you could change opcodes between fragments.
> 
> 
> Example messages:
>       * MORE=0,TML=0,Opcode=Text, Length=12, Payload="Hello world!" -
>       * MORE=1,TML=1,Opcode=Text, Length=5, MsgLength = 12,
>         Payload="Hello" - first frame of fragmented message
>       * MORE=0,TML=0,Opcode=Continuation, Length=7, Payload=" world!"
>         - final frame of fragmented message
> 
> 
>       * MORE=1,TML=1,Opcode=Text, Length=127, ExtLength=4096,
>         MsgLength=16000, Payload=... - first frame of dynamically
>         generated message, with estimated message length
>       * MORE=1,TML=0,Opcode=Continuation, Length=127,
>       * MORE=1,TML=1,Opcode=Continuation, Length=127, ExtLength=4096,
>         MsgLength=14781, Payload=... - sender now knows length of
>         dynamic content and sends it
>       * MORE=0,TML=0,Opcode=Continuation, Length=127, ExtLength=2493,
>       * if we decide creating new opcodes for extensions, which then
>         nest the next layer of extension within their payload data, we
>         have room to allocate additional opcodes, and we can grow the
>         opcode space if we need to
>       * if we decide to include per-frame extension data, we can
>         allocate a reserved bit to indicate the presence of an
>         extension length which describes the length of extension data
>       * if we decide to include per-frame compression, we can allocate
>         a reserved bit to indicate that this frame is compressed
>       * we can allocate new control frame types
> Is this something we can all get behind, even if it doesn't do
> everything we want, in the interest of getting something useful done
> soon?  If not, what specific changes would be necessary for you to
> agree?

Your proposal looks good to me. Cheers!

[1] http://www.beepcore.org
-- 
Francis Brosnan Blazquez <[email-masked]>
ASPL

---

From: Roberto Peon <[email-masked]>
Date: Thu, 19 Aug 2010 09:15:52 -0700

> allows direct connection and bidirectional content transfer from the

That is true for the API users. The people who have to implement this
en-masse and at scale have additional requirements.

We've torpedoed multiplexing for now (and multiplexing is the one thing that
can help scalability here). Ok, so then we need an extension mechanism in
order to implement and experiment with enabling multiplexing.

> > previous proposal where you could change opcodes between fragments.

This is pretty similar to IanF's proposal and is also workable.

---

From: John Tamplin <[email-masked]>
Date: Thu, 19 Aug 2010 12:43:49 -0400

On Thu, Aug 19, 2010 at 3:43 AM, Dave Cridland <[email-masked]> wrote:

> whatever.

The problem is we seem pretty strongly split between two different extension
methods:

   - having extension metadata separate from the payload, with some defined
   structure so that non-critical unknown extensions can be ignored.  This
   would also open the possibility of negotiating fixed offsets into the
   extension area for certain extension data for performance reasons
   - using nested layers of opcodes to define extensions -- ie the top-level
   opcode would be MUX, and the payload would have the mux data and the
   next-layer opcode and data, which might be another extension, etc, until you
   finally get to the real opcode/data.

The former gives the possibility of identifying the extension data where
intermediaries can safely ignore extensions they don't understand.  This is
not a proposal for extension data, but just a thought of how it might work
out to illustrate this point:

   - define one of the reserved bits as an EXT bit - if set, an extension
   length follows the header before the payload data, and that length is
   followed by a block of extension data
   - Each extensions data is preceded by an extension header
   - 1 bit - extension is critical - frame cannot be manipulated if this bit
      is set and the receiver does not understand the extension (and
an ultimate
      endpoint would drop the frame if it didn't understand the
extension, though
      something would be wrong since it negotiated it)
      - 7 bits - extension id - defined in a registry or negotiated during
      the handshake
      - 8 bits - extension data length (maybe support variable length if we
      think we need more than 255 bytes per extension)
      - extension data

This would allow a receiver to find the extension data it is looking for
even if it doesn't understand all the extensions.  Certainly some extensions
would render the data uninterpretable if they are not understood, but I
think many would not.  For example, an extension including a MIME-type for
the payload could safely be ignored, as could an extension that gave the
count of UTF16 characters.

The nested opcode approach essentially requires all outer extensions to be
understood, since otherwise you don't even know where in the payload the
next layer is.  It has the advantage of taking less space to define the
extension data and basically means there is no global extension format --
every extension defines its own payload data format.  Finally, the extension
data can be intermixed with lower-level payloads which may be easier to
handle different nesting of extensions.  For example, if you have a MUX
extension that aggregates multiple logical frames into a single frame, if
each of those frames have different extensions, it is less obvious how to
encode that into the aggregated frame, so that is an advantage of the
nested-opcode approach.

So, this split is why I gave a rationale that we don't have time to resolve
how extensions should be represented for the base protocol.  If you really
think it is critical that it is resolved now, then I suppose we have to do
it now, but I was hoping to basically encode what we have agreement on now
into the base protocol while leaving hooks for whatever we ultimately
decide.

> Finally, it's not clear to me if extension data needs to be per-message or
> per-frame - I can't think of a per-frame use-case, but still, this effects
> possible extension mechanisms heavily.
>

Clearly, if you have the ability to attach extension data to each frame, you
can define in the extension that it is present only on initial or final
frames as appropriate.  Regarding the per-frame use case, Ian has already
mentioned the mux stream id case.

---

From: John Tamplin <[email-masked]>
Date: Thu, 19 Aug 2010 12:51:26 -0400

> when those features can be implemented by protocols already defined like

Unless you are suggesting that BEEP be adopted as the WebSocket protocol, I
am not sure how the fact that BEEP supports these things helps us here.  I
think BEEP is a bit heavy-weight for what we are trying to accomplish here,
though certainly it can be a source of ideas.

---

From: gustav trede <[email-masked]>
Date: Thu, 19 Aug 2010 19:02:35 +0200

>> >         data type), present only if Length=127
Seems good to me.
Only one thing that i fail to understand, regarding the payload length
encoding:
What technical reason makes it so much better then the original binary frame
length encoding( if (byte[i]&0x80)==0x80) to detect the length bytes) that
its rather massive waste of bandwidth is justified ?.

regards
 gustav trede

---

From: Francis Brosnan Blazquez <[email-masked]>
Date: Thu, 19 Aug 2010 19:15:00 +0200

> en-masse and at scale have additional requirements. 

Certainly, if we assume websocket should also solve requirements that
may be implemented by other protocols on top of it. 

> We've torpedoed multiplexing for now (and multiplexing is the one
> thing that can help scalability here). 

I don't think so. By making websocket simple and leaving room to
application protocols on top achieves both goals: you still can do
multiplexing and your transport protocol keeps simple.

> Ok, so then we need an extension mechanism in order to implement and
> experiment with enabling multiplexing.

BEEP already provides channel multiplexing and profile selection and it
can be layered on top of websocket. This can provide the scalability you
are looking for without compromising the base protocol.
-- 
Francis Brosnan Blazquez <[email-masked]>
ASPL

---

From: John Tamplin <[email-masked]>
Date: Thu, 19 Aug 2010 13:28:34 -0400

On Thu, Aug 19, 2010 at 1:02 PM, gustav trede <[email-masked]>wrote:

> What technical reason makes it so much better then the original binary
> frame length encoding( if (byte[i]&0x80)==0x80) to detect the length bytes)
> that its rather massive waste of bandwidth is justified ?.
>

There are a couple of problems with it:

   - you can have an arbitrary number of leading 0x80 bytes, so there is no
   single representation of a length
   - you can have an arbitrary number of length bytes, which means to
   properly support the spec you have to implement multi-precision arithmetic.
    In practice, implementations will likely use 32 or 64-bit ints, and then
   some attacker will exploit those implementations.  With this proposal, every
   implementor knows they must support 64-bit lengths

If you fix those problems by disallowing leading 0x80 bytes and making the
maximum number of length bytes 9 (for the same 63-bit maximum length), you
now have 9 steps in the length, and I don't think they are justified.  I
think the 8 bytes of length is not significant for a larger message, so in
the worst case you have 127 payload bytes, so the overall message length is
137 bytes (2 byte fixed header, 8 byte extended length, payload) -- giving
an overhead of 7.3%.  More typical payload sizes will result in overheads of
the extra length bytes of well under half a percent.

The two length encoding mechanisms compare like this (ignoring the
opcode/flags byte, and any TCP/IP headers), with overall overhead
percentages for each:

   - 0-126 octets - both take one additional byte
   - 127 octets - v76 takes 1 byte, this proposal takes 9 (1.6%, 7.3%)
   - 128-16383 octets - v76 takes 2 bytes, this proposal takes 9 (2.3%-.02%,
   7.2%-.06%)
   - 16384-2097152 octets - v76 takes 3 bytes, this proposal takes 9
   (.03%-.0002%, .06%-.0005%)
   - ...

You can already see that there is no significant difference in overhead for
the larger packet sizes, yet we have the complexity of all these different
size length values for little benefit.  When you consider TCP/IP headers as
well, the difference in overhead is negligible.

Yes, it is more efficient to have a 2-byte length format for small packets
just above 127 bytes, but is that worth the additional complexity of extra
length steps?  Profiling of real-world WebSocket apps (admittedly, there are
few currently) suggests that there will be lots of very small packets
(especially once compression is supported), and a few larger packets.  So I
think that matches well with the proposal to provide exactly two steps of
length sizes.

---

From: Francis Brosnan Blazquez <[email-masked]>
Date: Thu, 19 Aug 2010 19:32:23 +0200

> Unless you are suggesting that BEEP be adopted as the WebSocket
> protocol, I am not sure how the fact that BEEP supports these things
> helps us here.  I think BEEP is a bit heavy-weight for what we are
> trying to accomplish here, 

Ironically you state BEEP is to heavy-weight but at the end, if the
websocket protocol pretend to implement multiplexing and protocol
extension it will also have to solve the same problems (already)
addressed by BEEP.

If that point is reached, I'm sure anyone will be able to state again
BEEP is heavy-weight compared to WS because both will be the same.

> though certainly it can be a source of ideas.

Certainly.

Anyway. I'm not suggesting to use BEEP as websocket protocol because
that will force anyone to layer anything (from simple to complex
application protocol requirements) on top of a solution that is far more
complex than TCP. 

This also apply to WS.

Websocket should keep its focus on solving transport issues and let
other specialized protocols, like BEEP, to solve complex requirements on
top of it.
-- 
Francis Brosnan Blazquez <[email-masked]>
ASPL

---

From: Roberto Peon <[email-masked]>
Date: Thu, 19 Aug 2010 10:46:46 -0700

> multiplexing and your transport protocol keeps simple.
Not everyone who is scaling things has control over both endpoints, thus
that layer of application protocol is inaccessible for these needs.

> are looking for without compromising the base protocol.

I don't think that is the case since I'm likely to have to load-balance
millions or billions of connections and not have the ability to modify or
place interesting requirements on the servers while remaining competitive.
-=R

---

From: gustav trede <[email-masked]>
Date: Thu, 19 Aug 2010 21:18:45 +0200

On 19 August 2010 19:28, John Tamplin <[email-masked]> wrote:

>    - you can have an arbitrary number of leading 0x80 bytes, so there is
>    no single representation of a length
>
> you have to parse yes, like you must do regardless while interpreting the
bits and bytes the protocol is. whats the _real_ problem here ?.

>    implementor knows they must support 64-bit lengths
What is the real world gain regarding guarantied super frame lengths ?.
Servers are supposed to do exactly what with valid but impossible to handle
frames lengths ?.

Whacking connection is what IMO will be done with the spec "valid" evil
clients, unless we add standardized max frame length negotiation like TCP
does for its various limits (yay! for complex protocol..),
That brings us back to square 1 again:
The supported length will for a few decades be, just like with unlimited
frame lengths something that is limited below the application implementation
layer.

If we are to guarantee a frame length in the spec, it must be something that
servers can handle today,
and then we can add negotiation that client can initiate if it wants more.
Or no limit at all and always do negotiation.

you now have 9 steps in the length, and I don't think they are justified.  I
>    - 128-16383 octets - v76 takes 2 bytes, this proposal takes 9
>    (2.3%-.02%, 7.2%-.06%)
> think that matches well with the proposal to provide exactly two steps of
Compression algorithms choice that is optimized for each specific usage case
is possible once binary frames are allowed.
That will still be something to do even if there is algorithm x added to the
standard protocol, because x is not the universal best choice.
With the current development of jscript performance its viable to let it do
some compression, at least for chat text and game state data.

---

From: Francis Brosnan Blázquez <[email-masked]>
Date: Thu, 19 Aug 2010 21:19:26 +0200

Hi Roberto,

> I don't think that is the case since I'm likely to have to
> load-balance millions or billions of connections and not have the
> ability to modify or place interesting requirements on the servers
> while remaining competitive.

While I see interesting the case you are exposing (zillions of
connections), it does not make it a representative case at all. As you
know, good protocol design is based on function delegation: each layer
makes a particular defined function.

I still believe WS should focus more on transport features and let
application protocols on top of it to complete the work....

...because it will be always better than any solution worked by the WG.
In other words: let the people to choose what to place on top of WS.
-- 
Francis Brosnan Blázquez <[email-masked]>
ASPL

---

From: Roberto Peon <[email-masked]>
Date: Thu, 19 Aug 2010 12:35:53 -0700

> In other words: let the people to choose what to place on top of WS.

If that is all WS supplies, why are we even bothering? Security?
We could easily decide to just forgo message framing too. That can be done
in a sub protocol!

-=R

---

From: Dave Cridland <[email-masked]>
Date: Thu, 19 Aug 2010 22:44:35 +0100

> makes a particular defined function.
Actually, good protocol design is about making stuff that works.  
Anything else is very much a secondary consideration, and while  
certain principles are useful, the one I'm most concerned with is the  
end-to-end principle, which (loosely) states that you minimize the  
amount of smarts in intermediaries, and force complexity to the ends.

> In other words: let the people to choose what to place on top of WS.

No, because it means that intermediaries need to work higher up the  
stack in order to do things like multiplexing and refragmentation,  
both of which are very low-level tasks.

Dave.

---

From: Patrick McManus <[email-masked]>
Date: Fri, 20 Aug 2010 10:27:57 -0400

On Thu, 2010-08-19 at 21:19 +0200, Francis Brosnan Blázquez wrote:
>  As you

I don't agree with such a broad brushed statement like that at all.
Layers are a convenient way to think about things, but truly layered
implementations are almost always inferior.

See varghese's _network algorithmics_ for an amazing treatment of the
topic.
(http://www.amazon.com/Network-Algorithmics-Interdisciplinary-Designing-Networking/dp/0120884771/ref=sr_1_1?ie=UTF8&s=books&qid=1282314428&sr=8-1 )

Protocol designers might use layers as a useful way to articulate the
protocol, but it is incumbent on us to consider all the various
requirements of different implementation scenarios. That often means
layering violations in practice. And that's ok.

---

From: Francis Brosnan Blazquez <[email-masked]>
Date: Fri, 20 Aug 2010 20:34:47 +0200

> layering violations in practice. And that's ok.

Obviously we can't take any rule as inviolable, and obviously perfect
layer separation is not possible nor desirable...however, what we have
here is a flagrant attempt to design a complex protocol that was
expected to only provide transport.

It is clear in our particular case, websocket, that it will provide far
more better benefits if the protocol, as long as possible, keeps its
features at the transport level, letting other specialized application
protocols on top of it to provide what the user is looking for.

Without considering the websocket final design, inevitably people will
layer its favorite protocol on top of it. Assuming that, WG should focus
on providing features that won't be provided by upper level apps, AND,
as long as possible do to include features that are already available on
them.
-- 
Francis Brosnan Blazquez <[email-masked]>
ASPL

---

From: Roberto Peon <[email-masked]>
Date: Fri, 20 Aug 2010 11:40:08 -0700

On Fri, Aug 20, 2010 at 11:34 AM, Francis Brosnan Blazquez
<[email-masked]>wrote:

> > (
> http://www.amazon.com/Network-Algorithmics-Interdisciplinary-Designing-Networking/dp/0120884771/ref=sr_1_1?ie=UTF8&s=books&qid=1282314428&sr=8-1)
> >

Yes, but what does it mean to only transport? Do you want it reliably?
Scalably?
There have been flagrant attempts to do both of these things :)
There have also been attempts to ensure that we have a modicum of security.
These all add complexity and are potentially compelling end results.

> as long as possible do to include features that are already available on

And, hopefully, to reduce overall system complexity by putting complexity in
the appropriate places.
-=R

---

From: "Shelby Moore" <[email-masked]>
Date: Fri, 20 Aug 2010 15:01:54 -0400

> protocols on top of it to provide what the user is looking for.

But the framing and fragmentation are unavoidable, receiving end has to
know the size to expect and there is no advantage of fixed length over
variable sizing:

http://www.ietf.org/mail-archive/web/hybi/current/msg03402.html

So what exactly are they putting in the base protocol that isn't transport?

BONUS FOR FREE: and WS sits on top of a base framing algorithm (TCP/IP)
and the way to optimize it is to send data in the largest possible chunks,
so variable size is a big win. Plus as a small extra benefit, it is less
headaches for developers to not have to chunk large data blocks to a
smaller fixed size on sending data.

This is one of those cases where you make things as simple and optimized
as possible but not more simple and not more optimized.  For example
analogy, when writing code I can do all sorts of hand-coded optimizations
that the compiler does any way, which only makes my code harder to read
and maintain, and may even prevent some (current or future) compiler
optimizations.  So no need for us to fragment more than necessary with
fixed size frames.  Lets always send the maximum size frame (because the
TCP/IP will optimize best), but the problem is that intermediaries might
not handle our preferred size, so we need fragmentation.  And ditto for
any size we choose, even a fixed size.

I have not yet read enough of the archives about the proposed multiplexing
impacts:

http://www.ietf.org/mail-archive/web/hybi/current/msg03403.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03408.html

---

From: "Shelby Moore" <[email-masked]>
Date: Fri, 20 Aug 2010 23:05:45 -0400

Synopsis: your correct attempts to maximize frame size (versus proposals
for fixed maximum frame size) is analogous to eliminating framing as much
possible, because we already have the framing otimization at the TCP
layer.

There is no way to entire eliminate framing from our layer, but maximized
frame size is no framing (with re-fragmentation to handle the degenerative
fallback).