Subject: [hybi] WS framing alternative

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 27 Oct 2009 09:32:43 +0800

WS is binary.  I believe that the benefits (reduced frame size, etc...) are outweighed by the disadvantages (debugging, etc...).  There would be little cost in a minor change that would also enable use of MIME.

  WS-frame = WS-length CRLF WS-headers CRLF WS-body CRLF
  WS-length = 1*DIGIT
  WS-headers = WS-header-name ":" WS-header-value CRLF
     ; add whatever MIME requires here, except the ugly stuff
  WS-body = *OCTET

Set some basic defaults for MIME headers (i.e. Content-Type = text/plain;charset=utf-8) and the impact on existing implementations is minimal.  Rather than sending:

  [0x80, 0x0d]Hello, World!

You send (indenting for readability):
  
  13

  Hello, World!

The unknown length thing is harder to replicate, but you could do as Greg suggests for BWTP, or something even simpler:

  WS-length = 1*DIGIT [WS-incomplete-frame]
  WS-incomplete-frame = "+"

That is:

  4+

  Hell
  9
 
  o, World!

MIME headers can then be used for a range of evils.  Including ones that we don't agree with.

Of course, your super simple implementation can:

  Read digits, convert to number b.
  Read incomplete marker.
  Discard until you reach a CRLF followed by another CRLF.
  Read b octets, assume UTF-8.
  Read CRLF.
  If incomplete, loop to start, else report message.
  Rinse, repeat.

--Martin

---

From: Ian Hickson <[email-masked]>
Date: Tue, 27 Oct 2009 09:11:59 +0000 (UTC)

On Tue, 27 Oct 2009, Thomson, Martin wrote:
>
> WS is binary.  I believe that the benefits (reduced frame size, etc...) 
> are outweighed by the disadvantages (debugging, etc...).

WS is really a text/binary hybrid. The handshake is text-based (UTF-8), 
and the frames come in two types, one for pure binary data which is binary 
delimited with a packed integer length prefix, and one for text (UTF-8) 
with a null prefix and a 0xFF suffix.

> There would be little cost in a minor change that would also enable use 
> of MIME.
> 

This would have some pretty major costs:

- It requires length delimiting for text frames, which is more complicated 
to implement (it's non-trivial to tell the difference between characters 
and bytes).

- It requires parsing using a presized buffer for variable-encoding text, 
which risks character/byte mismatches and thus buffer overruns.

- It allows arbitrary extensions, which introduces an implementation, QA, 
documentation, and tutorial cost without providing any new features at the 
API level.

- It makes it possible for the headers to have errors in them, which 
requires us to define how to handle errors, and requires significantly 
more code to implement the parsing.

> Set some basic defaults for MIME headers (i.e. Content-Type = 
> text/plain;charset=utf-8) and the impact on existing implementations is 
> minimal.

I disagree that it's minimal.

> Rather than sending:

Currently you would send a text frame thus:

   [0x00] Hello, World! [0xFF]

> The unknown length thing is harder to replicate, but you could do as 
> Greg suggests for BWTP, or something even simpler:
> 

That seems like orders of magnitude more complexity than necessary, 
especially given that the only tangible benefit is a modicum of 
improvement in ease of debugging.

In fact, I would argue that the increased complexity actually increases 
the need for debugging more than the text-based approach actually gives 
you in terms of debugging. Debugging binary protocols isn't especially 
difficult. You just pipe tcpdump through hexdump and less, and the issue 
is done. The protocol is so simple that you can hack together a read/write 
console in less than an hour.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 27 Oct 2009 21:08:40 +1100

Even though I said that discussion about protocol requirements
is more important, here is some more debate about protocol design:

> to implement (it's non-trivial to tell the difference between characters 

But length delimited frames are in the protocol anyway, so they have to
be implemented anyway.

If there was only 1 framing type, then we have approximately half the
framing complexity.

> which risks character/byte mismatches and thus buffer overruns.

With sentinel encoding, sending data might be marginally simpler,
but receiving data is much harder.

You will still have buffers of fixed sizes when you receive bytes.  You
don't know how much data is coming, so you don't know how big to make
your buffer or when to start turning bytes into characters.

I can see implementations reading a buffer.... scanning for 0x00, not
finding it... allocating a larger buffer... copying the bytes ...
reading more bytes ... scanning again for 0x00.... still not finding
it.... allocating yet another larger buffer... copying the bytes ....

etc.  etc.  until either you get a denial of service or you find
0x00, when you can finally scan over all the bytes again to
convert to characters.

If the message is larger than you want to read, then you don't
know this until you have read all the bytes you can and scanned for
a 0x00.  Resource limits are always better applied before the resource
is consumed than after.

To avoid wastful copying, you have to allocate wasteful large buffers,
which can still fill up and might need to grow.

Existing UTF-8 to character conversions will error if they get a partial
multipart character, so you can start conversion to characters until
you have the entire message.   Efficient implementations will be forced
to reimplement utf-8 converters to scan for 0x00 while converting.
This will allow smaller fixed byte buffers, but will still need
growing and/or large character buffers.  This is probably also
what needs to be done to be able to use direct kernel buffers.

I don't think there is a clear winner between the two types
of framing.  Both have their pros and cons.    But implementing
both will be approximately twice as complex and it will mean that
you have to deal with both sets of cons.

regards

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 27 Oct 2009 20:15:45 +0900

On 2009/10/27 19:08, Greg Wilkins wrote:

> multipart character,

No, good transcoding libraries have options that say what to do in such 
a case. Iconv is definitely an example, String#encode in Ruby is another.

> so you can start conversion to characters until
> you have the entire message.

I assume you wanted to say "cannot start conversion". False as explained 
above.

> Efficient implementations will be forced

No. These two concerns can be separated completely.

And there's the case that no conversion from utf-8 to characters is 
needed (when your implementation uses utf-8 for its internal 
representation, as e.g. in Perl or (mostly) in Ruby). (Still, you better 
check for consistent UTF-8 byte sequences.)

Regards,   Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: Thomas Broyer <[email-masked]>
Date: Tue, 27 Oct 2009 12:26:24 +0100

> console in less than an hour.

I totally agree; and for debugging, I believe a
protocol-decoder/dissector/display-filter could also be quite easily
added to Wireshark or similar network packet analyzers.

-- 
Thomas Broyer
/tɔ.ma.bʁwa.je/

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 27 Oct 2009 22:53:21 +1100

> a case. Iconv is definitely an example, String#encode in Ruby is another.

and such libraries make working out the length in bytes very
simple.   So if you have a good library, then the argument that
the byte length can't be known is not applicable.

regards

---

From: Ian Hickson <[email-masked]>
Date: Tue, 27 Oct 2009 18:34:30 +0000 (UTC)

On Tue, 27 Oct 2009, Greg Wilkins wrote:
> > - It requires length delimiting for text frames, which is more 
> > complicated to implement (it's non-trivial to tell the difference 
> > between characters and bytes).
> 

Only by generic clients (i.e. the browsers). Servers only have to 
implement the binary frames if they want to support binary frames in the 
protocol they implement, and dedicated clients (that just implement 
WebSocket as a necessary part of implementing whatever protocol it is that 
they are written for) would only need the binary frame support if the 
protocol they implement uses binary frames.

> If there was only 1 framing type, then we have approximately half the 

I've previously explained the reasoning for wanting to minimise the length 
measurements for UTF-8 data, so I won't repeat it here.

> > - It requires parsing using a presized buffer for variable-encoding 
> > text, which risks character/byte mismatches and thus buffer overruns.
> 
> With sentinel encoding, sending data might be marginally simpler, but 
> receiving data is much harder.

Not particularly. In languages with automatic dynamic strings (like Perl, 
Python, ObjectPascal, etc) you just concatenate and all the complexity is 
hidden from you by the compiler or language runtime. If you are using 
explicit buffers, then the complexity consists of just doubling the buffer 
size when you reach the size of the buffer; it's not a big deal either.

> your buffer or when to start turning bytes into characters.

You don't know that anyway, if your internal representation isn't UTF-8, 
since UTF-8 is a variable-length encoding (e.g. Win32 uses UTF-16 
internally, which is variable-length encoded in a different way from 
UTF-8, so you can't know how big the destination buffer should be without 
examining the whole byte string).

> etc.  etc.  until either you get a denial of service or you find 0x00, 
> when you can finally scan over all the bytes again to convert to 
> characters.

It's trivial to impose an arbitrary limit. It doesn't even have to be that 
arbitrary -- it can be whatever the server knows its protocol needs to 
support. Indeed, if the implementation uses fixed-size buffers like this, 
then it could just set its per-connection buffer to the maximum size it 
knows its protocol will ever handle, and just discard data if the limit is 
reached (or close the connection).

(Martin responded to the rest of your comments.)

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 28 Oct 2009 08:08:07 +1100

Ian Hickson wrote:

> hidden from you by the compiler or language runtime. 

Hidden complexity is still complexity.

On enterprise level servers, limiting the amount of data copying
and buffer allocation/resizing is key to achieving scalability.
Managing when and how data moves from kernel space to user-space
is also important and knowing a length up front can help with
that.

Having a protocol optimized for ease of perl implementation, but that
is difficult to optimize for scale is not ideal.

Besides - you say that there are quality utf-8 libraries available. So I
don't see the argument that says it is hard to know the length that is
being sent?

regards

---

From: Ian Hickson <[email-masked]>
Date: Tue, 27 Oct 2009 21:22:12 +0000 (UTC)

On Wed, 28 Oct 2009, Greg Wilkins wrote:
> 
> Having a protocol optimized for ease of perl implementation, but that is 
> difficult to optimize for scale is not ideal.

Agreed, but this isn't the case here. If you're willing to use advanced 
techniques, then there's lots of things you can do to avoid copying -- for 
example, the same thing as people do now to avoid copying HTTP headers 
around. After all, HTTP headers are sentinel based; they don't have length 
declarations. And those are longer than most WebSocket messages likely 
will be.

> don't see the argument that says it is hard to know the length that is 

The existence of quality UTF-8 libraries does not imply that everyone will 
competently use them. Many uses of WebSockets will likely not need 
anything beyond basic ASCII -- it's only when the script is sending user 
data and you want to interpret it on the server-side that you need to 
decode the data.

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 28 Oct 2009 06:12:07 +0800

Hi Ian,

> -----Original Message-----
> From: Ian Hickson [mailto:[email-masked]]
> Sent: Tuesday, 27 October 2009 8:12 PM
> To: Thomson, Martin
> Cc: [email-masked]
> Subject: Re: [hybi] WS framing alternative
> 
> > WS is binary.
> WS is really a text/binary hybrid. 

...which is even worse.  A hybrid loses the benefits of either approach.  But I tend to think of WS as a purely binary protocol that occasionally looks like text (the header) and can only have textual payloads (frame bodies).  That's consistent with its description.

> complicated

I agree with Greg on this point.  Sentinel-based delimiting has its own complexity costs.  Either position has been presented in the rhetoric as better or worse depending on what you value most.

> text,

That's untrue.  

Or you've made an assumption about my proposal that I never stated: i.e. that length is stated in characters.  Which would, of course, lead to these conclusions.  My intent (which I incorrectly assumed would be clear) was that length was octets.  It seems obvious to me - character encoding changes with different content types, thus you can't use character-based encodings.

> documentation, and tutorial cost without providing any new features at

Yes, arbitrary extensions have a cost, but I believe that we've benefitted from others having the wherewithal to pay that price before.

Besides, there are two mitigating factors:

 - When interoperating between vendors you cannot guarantee that the opposite end supports an extension, therefore you have to be prepared for them to ignore it.  Therefore, there is no reason that an implementation couldn't do as I suggested and ignore ALL extensions.

 - When not interoperating (the standard web sockets use case of JS talking to the same server), if there's no benefit gained from having extensions, then they can again be ignored.

> more code to implement the parsing.

Only if you require syntax checking of the headers.  If you allow for simple ignoring of headers, then it's straight back to the original.

> I disagree that it's minimal.

Excellent.  Then we all disagree ;)

>    [0x00] Hello, World! [0xFF]

Sure.  See below.

> improvement in ease of debugging.

That's why I'm not particularly wed to the idea.  Just like I'm not sold on the benefits of being able to start sending data without knowing its length beforehand.  That's complexity without benefit as far as I can tell.

It's hardly orders of magnitude of complexity, though.  Don't wear out your superlatives.

--Martin

---

From: Rory Byrne <[email-masked]>
Date: Thu, 29 Oct 2009 13:31:21 +0100

Hi Everyone,

I'm new to this list.

Just wanted to jump in here and throw my support in with the server
developers who want to see a byte count on all frames.

I think that Greg Wilkins is right on the money in what he says
about managing server resources. A byte count which tells a server
how much data a client intends to push to it, is a critical piece
of information. We're talking about servers that are going to have
many thousands of clients permanently connected to them for weeks,
or months, on end. To stay alive, these servers will need to be
excellent at resource management. IMO, the protocol as it stands
at the moment, is not giving these servers the information they
require to enable them to adequately manage their resources.

In a further effort to boost our chances of building robust
WebSocket servers, I would hope that we might consider adding some
sort of maximum frame length negotiation. Nothing fancy, there
could be a suitable default maximum, and a 'Max-Frame-Length' header
which enables a client to negotiate a higher maximum. Maybe something
like this:

        GET /demo HTTP/1.1
        Upgrade: WebSocket
        Connection: Upgrade
        Host: example.com
        Origin: http://example.com
        WebSocket-Protocol: sample
        Max-Frame-Length: 2097152

        HTTP/1.1 101 Web Socket Protocol Handshake
        Upgrade: WebSocket
        Connection: Upgrade
        WebSocket-Origin: http://example.com
        WebSocket-Location: ws://example.com/demo
        WebSocket-Protocol: sample
        Max-Frame-Length: 1048576

The server would only send a 'Max-Frame-Length' header if it wanted to
set the maximum to be lower than the client suggested. Any thoughts?

On the whole, I'm really excited about this effort to introduce the
Web Socket protocol. It's great to see that there's buy-in from browser
developers, and I'm really looking forward to testing out their
implementations!

Thanks,

Rory

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 30 Oct 2009 08:51:40 +1100

> set the maximum to be lower than the client suggested. Any thoughts?

I think having the ability to negotiate such parameters is a good
way to fail fast if a server and client are not compatible (eg
client needs to send larger messages than server is willing to
receive).

But I'm sure that there will be resistance to having such negotiation
as a mandatory part of the protocol.

However, the problem with making this kind of negotiation optional
(and this goes for my earlier examples of a load balancer
communicating SSL info and/or node stickyness), is that the current
WS protocol has no place for meta data to be added in an optional
manner - so that it can be ignored by implementations that don't care.

Ian has previously said that instead of adding headers to the
Upgrade request/response, messages should be injected into the
stream to communicate this meta data.      However, because
there is no way to flag these messages as meta data, simple
implementations are going to try to handle them as messages.

This limitation of no standard meta-data paths, means that it
is nigh impossible to introduce features like multiplexing,
load balancing, fragmentations, HTTP transport etc. as optional
additional specifications built on top of ws.   Because there
is no meta channel, simple implementations will treat everything
as a message and break if there is any new protocol layered
on top.

This is SO simple to fix:

 1) WS should allow arbitrary headers in the upgrade request/response

 2) WS should define a frame type for transport meta data, whose
    content is defined only as Mime (or just mime headers).
    The semantics of these can be left to other protocols built
    on top of WS.

With these 2 simple additions, then we could later develop a
standards that (for example):

 + implement  multi-plexing by negotiated support  in the
   upgrade request/response and then injecting a meta data
   packet, before each data packet, containing the information
   needed to route the subsequent data message.

 + implement standards to send content advisory messages
   that would contain info such as language, content-type etc
   for a connection.   These could be used by implementations
   that wanted to be flexible, or ignored by implementations
   that are sure they know what they are connecting to.

The current proposal is just too nailed down.  There is no where
to insert any optional handling.   The API is wired to the TCP/IP
layer.  The handshake messages are locked down to binary equivalence.
There are spare framing types available - but any suggestions to
allocate that space for meta-data have been rejected.

It appears that the whatwg want layered protocols to be controlled by
the allocation of frame byte values to them and only officially
sanctified extensions will get the nod.   The Whatwg needs to loosen
up and realize  that it should not try to completely control how a
protocol will be used in the wild (eg an not force all servers
to be 100 line perl scripts).  They have to allow it some room to
breath and grow.    They need to allocate some space for some
arbitrary meta data to be used or ignored as implementations see fit.

regards

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 30 Oct 2009 02:53:21 +0000

Greg Wilkins wrote:
> sanctified extensions will get the nod.

Let's not get confused about what will and won't work.

It doesn't matter whether WHATWG sanctions any new frame type or not:
you can't _use_ a frame type if the other end won't recognise it.

That's because the message boundaries are determined by recognising
the frame type byte, then parsing the message in a type-specific way.

If message boundaries cannot be recognised when an unknown type is
received, a receiver can't do anything more with the connection.  It
just has to drop the connection at that point.

That means any intermediary (which may be just code inside a client or
server framework, or a proxy) will have to pass all all the bytes it
receives as it receives them, with little or no delays, because
intermediaries can't know where a message ends in general, except for
the frame types defined in the first version.

> The Whatwg needs to loosen

Implementations can do that themselves.

A curious irony of WebSocket in it's current form is that there's no
reason a particular implementation should bother to use WebSocket
framing, if it doesn't wish to be accessible to Javascript browsers.

Since you must negotiate any custom protocol layered on top of
WebSocket out of band anyway (e.g. by agreeing what is present on some
URL), and because message boundaries cannot be recognised for unknown
frame types, and because there is no way to report per-frame errors,
in general all software which doesn't block WebSocket and intends to
be future compatible must pass the byte stream along verbatim, without
delays.

That means when implementing clients and servers of custom protocols,
you have two choices:

    - You want it to be accessible to Javascript in web browsers (in
      the short term future).  So you must frame everything into UTF-8
      frames, whatever kind of data it is, and define your own text
      protocol within that.

    - You don't care about Javascript in web browsers.  Because you
      know anything forwarding your WebSocket connection which is
      future compatible must pass along arbitrary bytes without
      delays, because there is no generic way for it to know frame
      lengths, you might as well send frame type byte NN (pick a
      number that is not used) followed by raw TCP with your own
      protocol after the upgrade request, if that is convenient.

      There seems to be no reason to use the spec's binary frames, if
      a TCP stream is more convenient for some application.

-- Jamie

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 30 Oct 2009 12:01:39 +0900

I'm in general very sympathetic to the WebSocket protocol, because it's 
exactly what we would have needed a year ago in some research we did 
here (using browsers for collaborative editing, coordinated through a 
server).

Also I agree with Ian that for communication between a single server and 
a single client, applications can work out their own "protocol", or some 
classes of applications (e.g. collaborative editing of some type of 
structures) can converge on a "protocol".

As far as the client and the server are concerned, negotiating things 
such as frame length seems rather pointless to me, because: a) we don't 
have any such negotiation currently for e.g. Web pages or images. b) The 
client should know the server reasonably well, and for applications 
where frame length is an issue (e.g. image upload, although I think 
there are way better ways to do image upload than with WebSockets), an 
external description of the service may contain some limits, or the 
"protocol" will say that you have to send the size first and wait for a 
"uploading is okay" from the server.

Where length issues or the like may come into play may be 
intermediaries. And (generic) intermediaries are where metadata is most 
valuable.

That said, however, I also agree with some other people on this list 

 >      The semantics of these can be left to other protocols built

seems to be a very good starting point.

Regards,   Martin.

On 2009/10/30 6:51, Greg Wilkins wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 30 Oct 2009 14:51:21 +1100

> the frame type byte, then parsing the message in a type-specific way.

Is this true?

The high order byte is currently defined as selecting between
sentinel or length encoding, and the 7 lower order bits are ignored.

So as the spec is written - regardless of the value passed, it
will be one of those 2 framing types.

In essence, the byte is 1 bit framing type and 7 bits undefined.
The 1 bit is also indicating content-type (or at least content-encoding),
which is a strange mix of purposes.

I hope I'm right because if what you say is true, a server might
eventually have to implement up to 256 different framing types!

... and as much as I'm able to endlessly debate the benefits of
one framing type over another, I really don't care that much.
Can't we just pick one and use it!   If the sentinel framing
was good enough to carry anything, then I could live with it.
But it's not, so let's just do length encoding.

Note also that i think it is very valuable to have a known
framing type, so that intermediaries can know what message
boundaries are and treat them appropriately
either with forwarding data, logging, debugging etc.

regards

---

From: Ian Hickson <[email-masked]>
Date: Fri, 30 Oct 2009 04:22:06 +0000 (UTC)

On Wed, 28 Oct 2009, Thomson, Martin wrote:
> Or you've made an assumption about my proposal that I never stated: i.e. 
> that length is stated in characters.  Which would, of course, lead to 
> these conclusions.  My intent (which I incorrectly assumed would be 
> clear) was that length was octets.  It seems obvious to me - character 
> encoding changes with different content types, thus you can't use 
> character-based encodings.

You are proposing that given a length and a series of bytes encoding text 
in a variable-sized encoding (UTF-8), the application return a series of 
characters. My point is that this means that you have two lengths (the 
length of the string in characters and the length of the string in bytes), 
so you risk inexperienced software authors making elementary yet dangerous 
mistakes in terms of how to read (or write) data to the stream. WebSocket 
tries to avoid ever mixing the two (you either deal with bytes and byte 
lengths, or you use sentinel bytes and no lengths -- you never have 
characters and byte lengths mixed together).

This isn't a problem if you assume that there will be only a few people 
implementing this and that they will all be competent. This is no an 
assumption I believe one can make if one's goal is to enable even amateurs 
to make use of this technology.

Regarding the rest of your e-mail, I don't disagree with the facts, but I 
disagree with the cost-benefit analyses, with your opinion of what is and 
what is not an acceptable risk, and with some of your goals. I don't know 
of any objective way to argue those points, and I agree that if you start 
with your positions, that you wouldn't develop WebSockets the way it is 
currently specced.

What is the process for proceeding in the IETF when people have 
fundamentally different and mutually exclusive opinions?

On Thu, 29 Oct 2009, Rory Byrne wrote:
> 
> I think that Greg Wilkins is right on the money in what he says about 
> managing server resources. A byte count which tells a server how much 
> data a client intends to push to it, is a critical piece of information. 
> We're talking about servers that are going to have many thousands of 
> clients permanently connected to them for weeks, or months, on end. To 
> stay alive, these servers will need to be excellent at resource 
> management. IMO, the protocol as it stands at the moment, is not giving 
> these servers the information they require to enable them to adequately 
> manage their resources.

The server will know what the clients will generally be sending -- e.g. a 
chat client is going to be expecting messages in the range of half a 
kilobyte or less, generally. So the server can just assume that that is 
the expected size, and only needs to do anything else if a message goes 
above that point (either bail, if the protocol the server supports will 
_never_ need such a large message, or switch to a bigger buffer).

> In a further effort to boost our chances of building robust WebSocket 
> servers, I would hope that we might consider adding some sort of maximum 
> frame length negotiation. Nothing fancy, there could be a suitable 
> default maximum, and a 'Max-Frame-Length' header which enables a client 
> to negotiate a higher maximum. Maybe something like this:
> 
>         GET /demo HTTP/1.1
>         Upgrade: WebSocket
>         Connection: Upgrade
>         Host: example.com

It's not necessary. The person in charge of the server is the one who 
invents the application-level protocol, and so the server can just define 
what its maximum is, without any negotiation.

It'd be like an HTTP client and server negotiating the maximum size of a 
<form> POST. If the client sends too much data, the server can just refuse 
it. The server is the one who decides what is acceptable.

On Fri, 30 Oct 2009, Greg Wilkins wrote:
> 
> I think having the ability to negotiate such parameters is a good way to 
> fail fast if a server and client are not compatible (eg client needs to 
> send larger messages than server is willing to receive).

How could that ever happen? A client isn't ever going to just randomly 
connect to a WebSocket server and start sending messages -- it's only 
going to send messages the server wants to support.

> However, the problem with making this kind of negotiation optional (and 
> this goes for my earlier examples of a load balancer communicating SSL 
> info and/or node stickyness), is that the current WS protocol has no 
> place for meta data to be added in an optional manner - so that it can 
> be ignored by implementations that don't care.

Sure, you just put them in the frames, as part of the higher-level 
protocol.

> Ian has previously said that instead of adding headers to the Upgrade 
> request/response, messages should be injected into the stream to 
> communicate this meta data.  However, because there is no way to flag 
> these messages as meta data, simple implementations are going to try to 
> handle them as messages.

You can flag messages however you like. WebSocket is like TCP -- it's just 
a transport on which you define your own protocol. Is there a way in TCP 
to define the URL of a stream? Or a way to add metadata?

> This limitation of no standard meta-data paths, means that it is nigh 
> impossible to introduce features like multiplexing, load balancing, 
> fragmentations, HTTP transport etc. as optional additional 
> specifications built on top of ws.  Because there is no meta channel, 
> simple implementations will treat everything as a message and break if 
> there is any new protocol layered on top.

This is either completely false, or we have dramatically different goals.

It's easy to add multiplexing, load balancing, or fragmentation as layers 
above WebSocket. For example, with multiplexing, you could just define 
that your frames have payloads like:

   C->S: open channel

   S->C: opened channel 1

   C->S: channel 1
         bla bla bla...

   S->C: channel 1
         foo bar baz...

   C->S: close channel 1

...or whatever.

Given the goal of just providing TCP for Web pages, it doesn't really make 
sense to have a more involved protocol.

It *would* make sense if your goal is something else, like tunneling 
Jabber over HTTP or something. But that's a different project than 
WebSockets. It's probably a perfectly reasonable thing for Hybi to work 
on, but it should be articulated as its own project.

> sanctified extensions will get the nod.

Layered protocols don't need frame types. In fact, frame types are 
completely transparent to layered protocols. If your protocol is just 
text-based, you'll never need anything beyond this version of WebSocket.

> The Whatwg needs to loosen up and realize that it should not try to 
> completely control how a protocol will be used in the wild (eg an not 
> force all servers to be 100 line perl scripts).  They have to allow it 
> some room to breath and grow.  They need to allocate some space for some 

I think you are misunderstanding the point of the frame types.

> the frame types defined in the first version.

Actually the protocol defines how you parse (find message ends) for every 
frame type possible. (0x00-0x7F use 0xFF delimiters, 0x80-0xFF use a 
length declaration.)

> framing, if it doesn't wish to be accessible to Javascript browsers.

There's no reason a particular implementation should bother to use 
WebSocket _at all_, if it doesn't wish to be accessible to Javascript 
browsers. Seriously, the whole point of WebSocket is to provide an API to 
scripts. That's it. If your use case is something else, then WebSockets is 
not the appropriate solution. 

> URL)

Actually the WebSocket-Protocol header lets you negotiate that at 
connection time.

> and because message boundaries cannot be recognised for unknown frame 
> types,

Actually that's well-defined already for all possible frame types.

> and because there is no way to report per-frame errors,

That's an application-level concern; there's no concept of errors at the 
protocol level.

> in general all software which doesn't block WebSocket and intends to be 
> future compatible must pass the byte stream along verbatim, without 
> delays.

Right.

>       frames, whatever kind of data it is, and define your own text

Right.

>       a TCP stream is more convenient for some application.

There seems to be no reason to use WebSocket at all in this situation. It 
doesn't provide anything more than TCP if you're not targetting JavaScript 
in Web browsers.

On Fri, 30 Oct 2009, "Martin J. Dürst" wrote:
> 
> That said, however, I also agree with some other people on this list that
> extension points should be considered carefully. Greg's proposal:
> 

Could you elaborate on what problem this solves?

---

From: SM <[email-masked]>
Date: Thu, 29 Oct 2009 22:01:19 -0700

>fundamentally different and mutually exclusive opinions?

The group seeks rough consensus.  I guess that you could think of it 
as the view that reconciles the various opinions.  That does not 
require everyone to agree; although that's better.  It's not a hard 
rule; there's no voting.  It's up to the Working Group Chair to 
evaluate the consensus.

Regards,
-sm

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 30 Oct 2009 13:05:26 +0800

> in a variable-sized encoding (UTF-8), the application return a series

Sure.  It's like the old argument over whether C or Pascal strings are better.  It's not really a fun argument.  It was not my intent to argue over the merits of either approach.

The point was to allow for arbitrary binary content as well as UTF-8 text together.  The point was to adopt a framing scheme that would allow for MIME.  If you want to avoid the problem you refer to (which I don't see as being as big as you make it out to be), then that's quite difficult if you want to allow for binary data - hence the two framing schemes in WS.

...
> fundamentally different and mutually exclusive opinions?

I believe that we use the methods of the Sophists.  We beat each other around the head with rhetoric.  Then we either come to a compromise, someone concedes, or the different parties begin to enjoy the argument too much and we end up with proprietary solutions.

Thankfully, we aren't arguing in a vacuum.  I'm happy to cede to the will of a majority on the issue, just as I hope that you are.  And deployment counts for a lot.  Until I feel that I'm arguing without support, I will continue to argue for a text-based protocol with (marginally) greater provision for extension than currently exists.

--Martin

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 30 Oct 2009 16:10:09 +0900

Hello Ian,

[order of citations somewhat rearranged.]

 >Actually the WebSocket-Protocol header lets you negotiate that at

Does that mean that WS allows arbitrary headers in the upgrade 
request/response? Or that there is one header, WebSocket-Protocol, with 
potentially a lot of data stuffed into it?

> Could you elaborate on what problem this solves?

You seem to assume that Web Sockets communication is basically just 
between a client and a server, without any intermediaries getting 
involved in any significant way. In that case, you are fine. But if 
there's anything where intermediaries get involved in a specific way 
(other than just letting everything through or just blocking), then 
there's probably a need to let these intermediaries know some info. 
Because the payload frames are for data exchange between the client and 
the server, they are essentially opaque to intermediaries, so you can't 
use them for that purpose. That's where a separate frame type comes in 
handy.

Regards,    Martin.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 30 Oct 2009 19:03:26 +1100

Ian Hickson wrote:

> kilobyte or less, generally.

Why will the server know what the client will be sending?

The application built on top of the server might well know what
it is expecting, and maybe that will be of a size within a know
range and maybe that wont be able to be influenced by users doing
silly things etc.

but the application knowing and the code that implements the protocol
knowing are two entirely separate issues.

The application could tell the server implementation what size
to expect - but insert all your arguments against buffer size
negotiation on the wire here as arguments against passing
buffers sizes from app to server.

Besides, you keep saying that the people programming this don't
know how to calculate their own messages sizes - so they don't
know the length and couldn't tell you anyway.

> going to send messages the server wants to support.

People write new versions of software
Utilities and debugging tools can be written to talk to more than one server.
Mysterious intermediaries are inserted between you and the server.
Layered protocols are written on top of websocket that have options.
Programmers make mistakes

> Sure, you just put them in the frames, as part of the higher-level 

But unless you allocate a standard flag for meta data, then
implementations that don't implement these optional features will
crash because they don't understand these injected frames.

If a ws client connects to a ws server, and neither knows if
the other supports some layered protocol - how do they discover
that? neither one can just send a message asking, because it
will break the other end if they don't support it!

> This is either completely false, or we have dramatically different goals.

Let's say that we have dramatically different goals.... sometimes that
happens and the sort of changes I'm minimally asking for are absolutely
trivial to the protocol and you can meet your goals and I can meet mine.

> sense to have a more involved protocol.

I'm not asking for a more involved protocol.
I'm asking for 1 bit to be allocated to indicate that a frame
contains mime-encoded data.    That is mostly sufficient
for layered protocols to be built on.

> on, but it should be articulated as its own project.

The sensible way to proceed would be to develop a simple but
extensible protocol that provides bidirectionality over the
web for all types of clients.

This can then be extended and/or layered upon for each type
of client.   So browsers could add on their origin security model
for example.

>From a servers/intermediaries point of view, browser are just
not that special that they deserve their entirely their own
protocol.

> I think you are misunderstanding the point of the frame types.

yes - kind of on purpose.

The WS Frame "byte" is really a frame bit, which is also
used indicate content-type (or at least content-encoding).

There are  7 unused bits in that byte. Allocating 1 of them
to indicate a meta data frame seams an entirely simple and
reasonable way to address one of the major concerns raised
by many people on this list.

Are you really saying you need to reserve space for 254 more
types of framing?

It's not like you are suggesting alternative ways to solve
the issues that we are raising.  You just keep saying "these
are not the droids that you are looking for....".  ie you
just say that the problems we raise are not problems for
you, so you don't care about them.

regards

---

From: Ian Hickson <[email-masked]>
Date: Fri, 30 Oct 2009 09:17:40 +0000 (UTC)

> potentially a lot of data stuffed into it?

You can set the desired value for -Protocol header by passing a parameter 
to the WebSocket constructor. That sets the value from the client. The 
server then responds with the protocol _it_ supports. If the two match, 
the client provides the connection to the script. To support multiple 
protocols, the server can use the client-provided value to decide which 
one to use.

> involved in any significant way.

Without any independent intermediaries, anyway. On the server side, there 
can be whatever intermediaries the server-side developer likes; they can 
all be considered to be the server-side implementation from the 
perspective of the client (and the spec).

> But if there's anything where intermediaries get involved in a specific 
> way (other than just letting everything through or just blocking), then 

What's the use case for independent intermediaries?

---

From: Ian Hickson <[email-masked]>
Date: Fri, 30 Oct 2009 09:57:56 +0000 (UTC)

On Fri, 30 Oct 2009, Greg Wilkins wrote:
> Ian Hickson wrote:
> > 
> > The server will know what the clients will generally be sending -- 
> > e.g. a chat client is going to be expecting messages in the range of 
> > half a kilobyte or less, generally.
> 

It defines it.

Consider TCP. The server knows what kind of traffic the clients will be 
sending -- it depends on the kind of server. An IMAP server knows that the 
client will be trying to talk IMAP. An HTTP server knows that the client 
will be trying to talk HTTP. An SSH server knows that the client will be 
talking SSH.

The same applies to WebSocket. A Wave server knows that the client will be 
talking Wave. A VoidWars game server knows that the client will be talking 
the VoidWars game protocol.

If you try to talk IMAP to an SSH server, the server's just going to close 
the connection on you. If you try to talk the VoidWars game protocol to a 
Wave server, the Wave server is just going to close the connection on you.

The Wave server can therefore know that it's only expecting messages of a 
certain length, and that it can just close the connection if the messages 
are too long.

> The application built on top of the server might well know what it is 
> expecting, and maybe that will be of a size within a known range and 
> maybe that wont be able to be influenced by users doing silly things 
> etc.
> 

Only if you make them separate, which seems to just be inviting trouble, 
given how simple the protocol is.

You're proposing making changes to the protocol to make it easier to make 
dedicated frameworks for the protocol.

I'm proposing making the protocol so simple that dedicated frameworks are 
superfluous.

> The application could tell the server implementation what size to expect 
> - but insert all your arguments against buffer size negotiation on the 
> wire here as arguments against passing buffers sizes from app to server.

My argument against on-the-wire buffer size negotiation ("it's 
unnecessary, both sides already know the protocol") doesn't apply in the 
case you are talking about (since by definition, the information is being 
passed from the part of the server implementation that does know the 
protocol to the part that doesn't know the protocol).

> Besides, you keep saying that the people programming this don't know how 
> to calculate their own messages sizes - so they don't know the length 
> and couldn't tell you anyway.

I'm saying that _some_ people programming this don't know how to calculate 
their own messages sizes. That doesn't mean that none do. Making the 
assumption that we have to cater for inexperienced programmers is not the 
same as making the assumption that we should _only_ cater to inexperienced 
programmers. Experts are also part of the ecosystem.

When designing a protocol for the broad range of authors from 
inexperienced to expert, we have to make sure that the inexperienced can 
get simple results easily, and that complexity is handled by the experts. 
This means making the simple things simple and the complex things more 
complex, rather than making everything a middling level of complexity. 
Making everything equally complex is optimising simplicity for the 
experts, who are the group who (by definition) need it the least.

> implementations that don't implement these optional features will crash 
> because they don't understand these injected frames.

Yes, if you define a protocol and people don't implement it right, you'll 
have bugs. The same applies to any protocol.

> If a ws client connects to a ws server, and neither knows if the other 
> supports some layered protocol - how do they discover that?

If a TCP client connects to a TCP server, and neither knows if the other 
supports some layered protocol - how do they discover that?

The answer is the same.

> >> simple implementations will treat everything as a message and break 
> >> if there is any new protocol layered on top.
> > 

I do not see how to meet my goals of making the protocol the simplest 
possible layer on top of TCP if we make it more complicated than the 
simplest possible layer on top of TCP.

That is, I do not see how our goals could ever be met by the same 
protocol, if I understand your goals correctly.

> > Given the goal of just providing TCP for Web pages, it doesn't really 
> > make sense to have a more involved protocol.
> 

Do you mean a third framing mechanism? Or would it reuse the text or 
binary framing mechanism?

If the latter, then it's unnecessary -- you just have to define that all 
frames a MIME encoded, or all frames are MIME encoded if their first 
byte/character is 0x01/U+0001, or some such. It doesn't have to be at the 
WebSocket protocol level.

The WebSocket protocol level basically is just a reflection of the API. 
Just like the API doesn't have a sendDataMimeEncoded() method, and doesn't 
have a message-mime-encoded event, the protocol doesn't have anything 
special for sending or receiving MIME-encoded data. It's purely an 
application-layer concern. Why would we add a sendDataMimeEncoded() 
method? What problem would it solve for scripts?

> > WebSockets. It's probably a perfectly reasonable thing for Hybi to 
> > work on, but it should be articulated as its own project.
> 
> The sensible way to proceed would be to develop a simple but extensible 
> protocol that provides bidirectionality over the web for all types of 
> clients.

I disagree that that would be sensible. I think overgeneralisation is one 
of the biggest mistakes one can make in protocol and language design. My 
experience has led me to be a firm believer in picking specific problems
and addressing them directly, rather than trying to solve everything at 
once (especially when one might not know all the problems one is trying to 
solve -- a common problem when discussing extensibility mechanisms).

> So browsers could add on their origin security model for example.

Security is not a feature you bolt on. It's an architecture that has to 
be designed in from the beginning.

> From a servers/intermediaries point of view, browser are just not that 
> special that they deserve their entirely their own protocol.

Browsers are the only reason for WebSocket existing. If it wasn't for the 
needs of scripts in Web pages, TCP would be a perfectly adequate solution 
for bidirectional communication for me, and I wouldn't be here. (Which 
isn't to say that there aren't perfectly valid other needs that deserve 
their own protocols, but those aren't the needs WebSocket was designed to 
address.)

> The WS Frame "byte" is really a frame bit, which is also used indicate 
> content-type (or at least content-encoding).
> 
> There are 7 unused bits in that byte.

That isn't an invitation to use them up. :-)

> Are you really saying you need to reserve space for 254 more types of 
> framing?

No, I'm saying that if it was possible, I would be just using one bit to 
carry the bit, and not a whole byte.

> It's not like you are suggesting alternative ways to solve the issues 
> that we are raising.

Could I ask you to more concretely describe your problem so that I could 
more concretely (maybe with actual code examples) describe the alternative 
solution that I am describing?

---

From: Pieter Hintjens <[email-masked]>
Date: Fri, 30 Oct 2009 12:33:48 +0100

>> fundamentally different and mutually exclusive opinions?

Speaking from experience of the AMQP design process over the last five
years, which was riven by such arguments...

Technical designs should not be opinion-driven at all.  There are
optimal (provably optimal) solutions for any well-defined problem.  If
two people do not agree on such solutions then either they are solving
different problems (this is most common), or they are not aiming for
optimal solutions but instead familiar and/or pre-existing solutions
(this is also common).  Or they are just incompetent with delusions of
competence (perhaps most common).

Even the expression of opinion is a sign that the design process is
failing or will fail to pinpoint the most accurate solution.

A forced compromise often creates a mediocre solution.

A better way to resolve the conflicts of interest that many of us
bring to such discussions is to create competition between solutions,
i.e. multiple protocols that implementors can choose.  Over time that
will drive the solutions towards an optimum.

This is all opinion, of course and does not answer the question of how
the IETF works, but may be a useful guideline for knowing how to
resolve conflicts of opinion.  Namely, if you start by saying "I
think", you are probably wrong.  :-)

-Pieter Hintjens

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 30 Oct 2009 23:08:08 +1100

> I'm proposing making the protocol so simple that dedicated frameworks are 

You are proposing a revolution in server infrastructure.

There is just no way that any of the organizations that I work
with are going to allow a server side application developer
access to a raw socket exposed on the internet!

No matter how much you think it a bad idea, there will be frameworks
that implement websocket on the server side - and they will have
little or no idea of the type of application that might be run
on top of them.

> supports some layered protocol - how do they discover that?

No it's not.

You run an *extensible* protocol like HTTP over TCP, so that
new protocols can be layered on top without breaking clients,
servers or intermediaries.

> protocol, if I understand your goals correctly.

I don't see how allocating 1 bit of 7 unused can render
the protocol unsuitable for your goals.

You are being a tad melodramatic I think.

> binary framing mechanism?

I would reuse the text and binary framing mechanism.

I would suggest that for meta-data packets if the framing
type was utf-8 then the packet would contain just headers (which
layered protocols could define to applying to the connection, the next
frame or whatever).

If the framing type was binary, then the packet can contain standard
mime encoded content.

[ I would also advocate that the high bit be just a selection
between utf-8/binary and that all frames be length encoded,
but that's a different debate ]

> WebSocket protocol level.

but how do you know if the other end supports mime encoding at all.
you end up needing to do all sorts of discovery and other complexities.

Browser vendors and server vendors might want to collaborate
on a multiplexing protocol built on top ws.  They could use the
the mime frames to do this - but they would be able to continue
to work with browsers and servers that did not implement the
extension - as they would ignore meta data they didn't understand
that was used to try to negotiate a multi-channel connection.

> method? What problem would it solve for scripts?

I'm not asking you to add a sendDataMimeEncoded method for the
application developers to use.

The extra capabilities could be used by the browser, intermediaries
and/or servers.   They need not be exposed to application developers,
you could even argue that they shouldn't be exposed to application
developers.

> I think overgeneralisation is one 

You have already generalized the protocol by allowing for other
frame types.    The only difference is who controls the use
of that generalization.

You obviously want to control how people extend WS and will only
allocate new frame byte values to what you think is worthy cause.

If there was a meta data frames, then anybody could extend
the protocol without the need to ask permission ( even if it was
to support evil monolithic servers and their unholy load balancing
companions).

What authority do you plan to manage the allocation of frame types
into the future?  IANA?

> be designed in from the beginning.

RUBBISH!

If this was the case we would have to put the same origin policy
into TCP... no wait into IP... no wait into ethernet..

You can design a good protocol for the security needs of
browser that is built on a more general transport protocol.
It also helps if that transport protocol was designed to
allows such extensions.

> for bidirectional communication for me, and I wouldn't be here. 

Well browsers don't do you much good without servers
and other network infrastructure.  So you have to consider
their needs as well.

Plus there are plenty of others here that do care about clients
other than scripts in web pages.

It is my understanding of IETF policy that if you want to
get this protocol approved you need to get a rough consensus
of people with more concerns than just your own.

> solution that I am describing?

sticky load balancer with SSL offload that works with any ws app.

multiplexing protocol implemented in browser/server (not in application).

orderly closing a ws connection implemented in an application
independent way.

regards

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 30 Oct 2009 12:34:31 +0000

> sentinel or length encoding, and the 7 lower order bits are ignored.

No it's not true; you are right.  The frame length is determined
by the high order bit.

Sorry for reading too much into mailing list postings and not enough
reading the spec properly, which I will now do.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 30 Oct 2009 12:46:44 +0000

> characters and byte lengths mixed together).

Inexperienced authors, especially those writing 100 lines of Perl,
will send ISO-8859-1 or other text which occasionally contains 0xff
bytes in the middle.

Even experience authors will make that mistake sometimes.  What do you
think will happen when someone does something like this:

    - Read list of filenames in a directory.  They're UTF-8 (assumed),
      or the author is unfamiliar with character encodings and everything
      works fine in their ASCII development environment.

    - Concatenate the list with newlines, as people do.

    - Send the result as a frame.

Or this:

    - Read lines from a text file, which is in UTF-8 encoding.

    - Send each line as a frame.

    - (Oops, one of the text files you gave me had an 0xff byte in it.)

Result: Because of assumptions, 0xff bytes will be sent occasionally
in the middle of a frame.  Everything afterwards will break, but it'll
be rare enough that the author doesn't notice.  For the same reason
you've explained authors get lengths wrong.

The sentinel approach does not solve this fragility problem, it merely
shifts it around to a different place.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 30 Oct 2009 13:11:00 +0000

> sticky load balancer with SSL offload that works with any ws app.

This is the key thing, I believe.

WebSockets forces all negotiation of extensions to be done by the
application.

This prevents automatic extensions being done below the application,
so "works with any WS app" is prevented by the current proposal.

Here's an example: Just like HTTP, there could be value in an
extension which compresses some frames.  If it worked well, that'd be
a good thing to do _transparently_ in the browser+server, without
awareness by the WS application.

But you can't provide a transparent compression extension, because the
browser has no way to signal to the server "I handle this extension".

In this respect, WS is like TCP where it's hard to modify
transparently.  This may be intentional: WS is meant to be a web
application compatible TCP after all.  But there is some value in
automatic extensions below the application, as we've seen in HTTP.

Given that the only thing blocking this is the client's inability to
signal the server that it supports an extension, and that is so easy
and natural to do by allowing clients to add headers to the initial
request, I think that should be at least seriously considered.  Not
dismissed summarily.

> multiplexing protocol implemented in browser/server (not in application).

That's similar to the compression example above.

> orderly closing a ws connection implemented in an application

Orderly close is more important than people realise.

A major problem with HTTP pipelining is you can't use it for
non-idempotent requests because the server may close the connection
any time, without implying an error condition.  That's one of the many
reasons HTTP pipelining often cannot be used, and that makes HTTP in
general slower and less network efficient than it could be.

The same applies to other request-response based protocols, such as
many of those expected to run over WebSocket.

Another problem with closing is that thing not mentioned in the HTTP
RFCs, but essential: lingering half-closed sockets.  If a server sends
a response followed by closing the socket fully, some clients sometimes
don't receive the whole response because of the way TCP and pipelined
requests clash.

Both of these are why orderly close is important - more important than
it appears.  Because it makes it safe to do other things.

I expect "pipelining" in the meaning used by HTTP, that is sending
multiple messages before getting a response to each one, to be quite
prevalant with protocols running over WebSocket.

That makes orderly close important for WebSocket, to avoid fragile
applications.  Hopefully I've illustrated the kinds of problems to
expect.  Unfortunately it's all a bit arcane.

There's nothing wrong with doing it in the application.  But I don't
have high expectation of it being done correctly in those "we don't
need no steenkin' framework" 100 line Perl scripts by inexperienced
authors.

-- Jamie

---

From: SM <[email-masked]>
Date: Fri, 30 Oct 2009 13:30:43 -0700

Hi Pieter,
>competence (perhaps most common).

It is much more complicated that that and it is not always about 
finding the optimal solution.

>will drive the solutions towards an optimum.

Having multiple protocols for the same problem space doesn't favor 
interoperability.  One or more of the protocols will take over which 
is like "let the market decide".

>think", you are probably wrong.  :-)

I don't think there is a guideline for knowing how to resolve 
conflicts of opinion. :-)  There are a lot of factors 
involved.  Strong opposing opinions are resolved in different 
ways.  There is generally a person or a group of persons who come up 
with a proposal on how to reach consensus.

Regards,
-sm

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 31 Oct 2009 10:20:08 +1100

> will drive the solutions towards an optimum.

Pieter,

After initial failures to get my concerns addressed in WS, I
did decide to propose alternative solutions. BWTP mk I and BWTP mk II
resulted.

But now it appears that a working group cannot be formed to consider
multiple solutions and that it will be free to consider any solution
so long as it is websocket.

Thus I've stopped efforts on developing BWTP as it appears it will
be wasted effort.

We are in the strange circumstance that whatwg does not want to
deal with concerns other than js in browsers and the IETF does not
want to consider solutions other than what the whatwg is proposing.

I do feel that the process has been setup for failure and unless
we can broaden the requirements or consider other alternatives
then we are doomed to endless unproductive debates between people
with different agendas.

regards

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 31 Oct 2009 10:55:19 +1100

> shifts it around to a different place.

The sentinel approach also opens an easy attack vector.   If user data
is sent, then tricking a poor implementation into sending an OxFF
will allow packet insertion.

This is similar to CRLFCRLF insertion attacks that can happen
if user data is set unfiltered into a HTTP header and/or cookie.

length framing avoids this vulnerability.

Note also that my proposal for a meta-data bit would allow
headers to be sent in one length frame and data in another, so
the CRLFCRLF sentinel would not be needed and that vulnerability
would also be avoided.

regards