Subject: [hybi] WebSocket sub protocol name.

From: Fumitoshi Ukai (鵜飼文敏) <[email-masked]>
Date: Mon, 7 Dec 2009 13:58:06 +0900

Hi,

At 1.130 of "The Web Socket API", it adds the sub-protocol name must be an
ASCII string with no U+000A LINE FEED (LF) or U+000D CARRIAGE RETURN (CR)
characters in it.
But "The Web Socket protocol" 3.1 Parsing Web Socket URLs, says
   1.   If /protocol/ is specified but is either the empty string or
        contains characters that are not in the range U+0021 to U+007E,
        then fail this algorithm.

Alghough I think it should not be a part of "Parsing Web Socket URLs", but
the /protocol/ here is talking about sub-protocol name in Web Socket
constructor, right?
I'm wondering why "The Web Socket API" has loose definition about
sub-protocol name.  If sub-protocol name contains a character less than
U+0021, it fails and throw a SYNTAX_ERR exception ?

-- 
ukai

---

From: Ian Hickson <[email-masked]>
Date: Mon, 7 Dec 2009 06:21:04 +0000 (UTC)

>         then fail this algorithm.

Woops, that step is a copy-and-paste mistake from a past version. My 
apologies. I've removed that sentence; it should not have been present. 
Control characters are allowed (though using them would be silly).

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Fumitoshi Ukai (鵜飼文敏) <[email-masked]>
Date: Mon, 7 Dec 2009 15:41:15 +0900

On Mon, Dec 7, 2009 at 3:21 PM, Ian Hickson <[email-masked]> wrote:

> On Mon, 7 Dec 2009, Fumitoshi Ukai (榈~\椋兼~V~G鎫U~O) wrote:
> >

Thanks for fixing.

> Control characters are allowed (though using them would be silly).

Why are control characters (except LF and CR) allowed?

-- 
ukai

---

From: Ian Hickson <[email-masked]>
Date: Mon, 7 Dec 2009 07:26:52 +0000 (UTC)

On Mon, 7 Dec 2009, Fumitoshi Ukai (�~\飼�~V~G�~U~O) wrote:
> > > At 1.130 of "The Web Socket API", it adds the sub-protocol name must 
> > > be an ASCII string with no U+000A LINE FEED (LF) or U+000D CARRIAGE 
> > > RETURN (CR) characters in it.

There doesn't seem to be a good reason to exclude them, and excluding them 
would lead to a more complicated processing model.

---

From: Julian Reschke <[email-masked]>
Date: Mon, 07 Dec 2009 08:31:01 +0100

Ian Hickson wrote:
> ...
> ...

There isn't a good reason to disallow control characters in a *name*????

Best regards, Julian

---

From: Fumitoshi Ukai (鵜飼文敏) <[email-masked]>
Date: Mon, 7 Dec 2009 16:40:02 +0900

On Mon, Dec 7, 2009 at 4:26 PM, Ian Hickson <[email-masked]> wrote:

> On Mon, 7 Dec 2009, Fumitoshi Ukai (�µ~\飼æ~V~Gæ~U~O) wrote:

In HTTP, field-content is TEXT or  combinations of token, separators, and
quoted-string. TEXT, or token, separators excludes CTLs.
So, we must use quoted-string in WebSocket-Protocol: if protocol contains
CTLs?

And, why is it limited to ASCII instead of UTF-8?

-- 
ukai

---

From: Ian Hickson <[email-masked]>
Date: Mon, 7 Dec 2009 08:05:42 +0000 (UTC)

On Mon, 7 Dec 2009, Fumitoshi Ukai (�~\飼�~V~G�~U~O) wrote:
> > There doesn't seem to be a good reason to exclude them, and excluding 
> > them would lead to a more complicated processing model.
> 
> quoted-string. TEXT, or token, separators excludes CTLs. So, we must use 
> quoted-string in WebSocket-Protocol: if protocol contains CTLs?

Oh, I forgot that HTTP had the no-CTL restriction. Good point. I've 
updated the spec to be consistent with this.

> And, why is it limited to ASCII instead of UTF-8?

Because the HTTP working group refuse to allow UTF-8 in HTTP headers for 
reasons that I don't really understand, and the handshake is supposed to 
be valid HTTP.

> There isn't a good reason to disallow control characters in a *name*????

Not as far as I can tell, no... what would a good reason be?

---

From: Julian Reschke <[email-masked]>
Date: Mon, 07 Dec 2009 09:45:56 +0100

> reasons that I don't really understand, and the handshake is supposed to 

A change of the default encoding would be incompatible with HTTP/1.1.

> Not as far as I can tell, no... what would a good reason be?

Consistency with every other single spec on the planet with respect to 
names?

Best regards, Julian

---

From: Fumitoshi Ukai (鵜飼文敏) <[email-masked]>
Date: Mon, 7 Dec 2009 19:07:56 +0900

> updated the spec to be consistent with this.

In "The Web Sockets API", could you fix the following statement?
  The second, protocol, if present, specifies a sub-protocol that the server
must support for the connection to be successful. The sub-protocol name must
be an ASCII string with no U+000A LINE FEED (LF) or U+000D CARRIAGE RETURN
(CR) characters in it.

IIRC, in old spec draft, protocol should not be an empty string, but current
spec draft accepts it?

> reasons that I don't really understand, and the handshake is supposed to

Hmm, then it should be mentioned in The Web Socket protocol 1.2 Protocol
overview?
It looks websocket message accept UTF-8 in handshake message.

-- 
ukai

---

From: Ian Hickson <[email-masked]>
Date: Mon, 7 Dec 2009 10:25:09 +0000 (UTC)

On Mon, 7 Dec 2009, Fumitoshi Ukai (�~\飼�~V~G�~U~O) wrote:
> > > > There doesn't seem to be a good reason to exclude them, and 
> > > > excluding them would lead to a more complicated processing model.
> > >
> > > In HTTP, field-content is TEXT or combinations of token, separators, 
> > > and quoted-string. TEXT, or token, separators excludes CTLs. So, we 
> > > must use quoted-string in WebSocket-Protocol: if protocol contains 
> > > CTLs?
> >

Fixed, thanks.

> IIRC, in old spec draft, protocol should not be an empty string, but 
> current spec draft accepts it?

Man, I'm just incompetent today. Sorry about that. Fixed.

> > Because the HTTP working group refuse to allow UTF-8 in HTTP headers 
> > for reasons that I don't really understand, and the handshake is 
> > supposed to be valid HTTP.
> 
> overview? It looks websocket message accept UTF-8 in handshake message.

There should be no way to _send_ UTF-8 in the handshake at this point, 
given the requirements in the spec, but once you _receive_ the handshake, 
it's no longer HTTP, it's just WebSocket, so you can use UTF-8 fine. (In 
practice it doesn't mean much, since there's nothing in the 
server-to-client handshake other than the client-to-server handshake, so 
if there ever was UTF-8 in the server-to-client handshake, it would either 
be ignored, or the connection would be dropped, depending on where exactly 
the UTF-8 was found.)

HTH,

---

From: Ian Hickson <[email-masked]>
Date: Mon, 7 Dec 2009 10:41:58 +0000 (UTC)

> A change of the default encoding would be incompatible with HTTP/1.1.

How so? Isn't HTTP/1.1 limited to ASCII?

> Consistency with every other single spec on the planet with respect to names?

Arbitrary consistency isn't a particularly good reason, especially given 
how wildly varying rules on what can be in an identifier are.

---

From: Julian Reschke <[email-masked]>
Date: Mon, 07 Dec 2009 11:51:37 +0100

> How so? Isn't HTTP/1.1 limited to ASCII?

It depends on how you read the spec, which is somehow ambiguous. It 
hints that certain protocol elements can use ISO-8859-1, and this is 
what many libraries actually implement (when they expose headers as Java 
or Javascript strings); I bet it's the case for XHR, for example.

> how wildly varying rules on what can be in an identifier are.

I agree that the rules vary wildly (such as non-ASCII or not, hyphen or 
not), but that doesn't mean that introducing more variation is a good thing.

Best regards, Julian

---

From: Fumitoshi Ukai (鵜飼文敏) <[email-masked]>
Date: Tue, 8 Dec 2009 14:17:18 +0900

> Man, I'm just incompetent today. Sorry about that. Fixed.

Thanks!

protocol now accepts U+0020. Is it ok to use U+0020 only in /protocol/ ?
(e.g. new WebSocket("ws://example.com/", " "); )
It seems space is optional after colon in field of handshake message, how
can we distinguish U+0020 and U+0020 U+0020 ?

> be ignored, or the connection would be dropped, depending on where exactly

Ok. I see.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 08 Dec 2009 16:37:03 +0900

> or Javascript strings); I bet it's the case for XHR, for example.

The number of protocol elements that can use ISO-8859-1 is few and far 
between. Because ISO-8859-1, on the wire, is essentially just a sequence 
of bytes, there is nothing at all inhibiting a "higher-level" protocol 
to say "interpret this string as UTF-8". So, if you really need UTF-8, 
just say it's interpreted as UTF-8. If you are fine with US-ASCII only, 
just stay within 7bit.

>> Arbitrary consistency isn't a particularly good reason, especially
>> given how wildly varying rules on what can be in an identifier are.
>

There is definitely a wide range of variation. But as you (Ian) have 
sad, using control characters would be silly. Or may we conclude from a 
later message of yours that you already eliminated control characters? 
That would be great.

Regards,   Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: Ian Hickson <[email-masked]>
Date: Tue, 8 Dec 2009 07:59:10 +0000 (UTC)

> can we distinguish U+0020 and U+0020 U+0020 ?

The space isn't optional (both the client and the server are required to 
send it before the subprotocol name), it's just that the client will 
handle a missing space gracefully. Maybe we should remove that "feature", 
and require that the space be present, closing the connection otherwise?
I don't feel strongly either way.

Anyway, the point is that "WebSocket-Protocol: " is the empty string, and 
"WebSocket-Protocol:  " is a one-space protocol name.

---

From: Fumitoshi Ukai (鵜飼文敏) <[email-masked]>
Date: Tue, 8 Dec 2009 17:04:49 +0900

On Tue, Dec 8, 2009 at 4:59 PM, Ian Hickson <[email-masked]> wrote:

> On Tue, 8 Dec 2009, Fumitoshi Ukai (榈~\椋兼~V~G鎫U~O) wrote:
> >

Thanks for clarifying this.

-- 
ukai

---

From: Yuzo Fujishima <[email-masked]>
Date: Tue, 8 Dec 2009 17:06:57 +0900

> "WebSocket-Protocol:  " is a one-space protocol name.

Hi, Ian, let me confirm one thing: Empty string is not allowed for
subprotocol, is it?

Yuzo

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: Julian Reschke <[email-masked]>
Date: Tue, 08 Dec 2009 09:25:54 +0100

> "WebSocket-Protocol:  " is a one-space protocol name.

Out of curiosity: why do you think it's wise to allow SP in protocol names?

BR, Julian

---

From: Ian Hickson <[email-masked]>
Date: Tue, 8 Dec 2009 08:26:56 +0000 (UTC)

> later message of yours that you already eliminated control characters? 

Yes, the /protocol/ and /resource name/ inputs to the Web Socket client 
handshake algorithm are explicitly required to be non-empty strings of 
ASCII characters in the range U+0020 to U+007E.

http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol#section-5.1

---

From: Ian Hickson <[email-masked]>
Date: Tue, 8 Dec 2009 08:31:28 +0000 (UTC)

On Tue, 8 Dec 2009, Yuzo Fujishima wrote:
> > The space isn't optional (both the client and the server are required 
> > to send it before the subprotocol name), it's just that the client 
> > will handle a missing space gracefully. Maybe we should remove that 
> > "feature", and require that the space be present, closing the 
> > connection otherwise? I don't feel strongly either way.
> > 
> > Anyway, the point is that "WebSocket-Protocol: " is the empty string, 
> > and "WebSocket-Protocol:  " is a one-space protocol name.
> 

The API will throw if the string is empty. However, if no protocol was 
specified, and the server returns an empty WebSocket-Protocol, it's 
ignored (as would any other WebSocket-Protocol value sent by the server in 
that case).

---

From: Ian Hickson <[email-masked]>
Date: Tue, 8 Dec 2009 08:34:23 +0000 (UTC)

On Tue, 8 Dec 2009, Julian Reschke wrote:
> > and "WebSocket-Protocol:  " is a one-space protocol name. ...
> 

It's effectively an opaque binary string; what difference does it make 
what values are allowed? The only reason to limit it to ASCII with no 
controls at all is to keep it theoretically valid HTTP. Other than that, 
only CR and LF are excluded because they are used as field delimiters. Why 
would we exclude anything else?

---

From: Julian Reschke <[email-masked]>
Date: Tue, 08 Dec 2009 09:42:58 +0100

> would we exclude anything else?

For consistency with other things called a "name". Maybe one way to 
achieve this is to rename the protocol element from "sub protocol name" 
to "sub protocol identifier".

BR, Julian

---

From: Ian Hickson <[email-masked]>
Date: Tue, 8 Dec 2009 08:51:17 +0000 (UTC)

On Tue, 8 Dec 2009, Julian Reschke wrote:
> Ian Hickson wrote:
> > It's effectively an opaque binary string; what difference does it make what
> > values are allowed? The only reason to limit it to ASCII with no controls at
> > all is to keep it theoretically valid HTTP. Other than that, only CR and LF
> > are excluded because they are used as field delimiters. Why would we exclude
> > anything else?
> 

Wouldn't that be worse? Identifiers have far more history in terms of what 
characters are allowed in them than names do.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 08 Dec 2009 10:03:13 +0100

> characters are allowed in them than names do.

Right.

So pick another name, or make the subprotocol name consistent with other 
names in the spec.

BR, Julian

---

From: Ian Hickson <[email-masked]>
Date: Tue, 8 Dec 2009 09:50:14 +0000 (UTC)

On Tue, 8 Dec 2009, Julian Reschke wrote:
> > > > It's effectively an opaque binary string; what difference does it 
> > > > make what values are allowed? The only reason to limit it to ASCII 
> > > > with no controls at all is to keep it theoretically valid HTTP. 
> > > > Other than that, only CR and LF are excluded because they are used 
> > > > as field delimiters. Why would we exclude anything else?
> > > achieve this is to rename the protocol element from "sub protocol 
> > > name" to "sub protocol identifier".
> > 
> > Wouldn't that be worse? Identifiers have far more history in terms of 
> > what characters are allowed in them than names do.
> 
> Right.
> 

I don't particularly think that consistency is a compelling argument here. 
Restrictions should be in place for good reason, e.g. to eliminate 
ambiguities, to catch likely mistakes, etc. Here, I see absolutely no 
benefit to preventing authors from using whatever string they like. It's 
still the subprotocol name -- that's what it is, regardless of what we 
call it. Calling it something else just because otherwise it might not be 
consistent with other uses of the word is just dogmatic.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 08 Dec 2009 10:55:56 +0100

> consistent with other uses of the word is just dogmatic.

Allowing a single space character as a name still sounds like a bad idea 
to me.

BR, Julian

---

From: Ian Hickson <[email-masked]>
Date: Tue, 8 Dec 2009 10:27:04 +0000 (UTC)

On Tue, 8 Dec 2009, Julian Reschke wrote:
> Ian Hickson wrote:
> >
> > I don't particularly think that consistency is a compelling argument 
> > here. Restrictions should be in place for good reason, e.g. to 
> > eliminate ambiguities, to catch likely mistakes, etc. Here, I see 
> > absolutely no benefit to preventing authors from using whatever string 
> > they like. It's still the subprotocol name -- that's what it is, 
> > regardless of what we call it. Calling it something else just because 
> > otherwise it might not be consistent with other uses of the word is 
> > just dogmatic.
> 

Can you articulate a technical reason for it, other than consistency with 
the other places that have restrictions on names?

Personally I would recommend considering WebSocket as being a binary 
protocol. A "name" that consists of a single 0x20 byte is nothing special.

---

From: Pieter Hintjens <[email-masked]>
Date: Tue, 8 Dec 2009 12:54:33 +0100

> protocol. A "name" that consists of a single 0x20 byte is nothing special.

You're messing with people's models of what things mean.  "Name" means
something, it is a subset of "word" and that is a string of letters
that explicitly, in English at least, excludes spaces.

A name containing or consisting of space characters is pathological.
It works technically but it is a mess conceptually.

Call it "identifying string" or "token" or "

---

From: Pieter Hintjens <[email-masked]>
Date: Tue, 8 Dec 2009 12:56:03 +0100

> Call it "identifying string" or "token" or "

Keyboard blip...

Call it anything but "name" unless you really enjoy confusing people.
There is a difference between "so simple anyone can implement it" and
"no rules that make any sense".

-Pieter

---

From: Thomas Broyer <[email-masked]>
Date: Tue, 8 Dec 2009 16:32:36 +0100

> that explicitly, in English at least, excludes spaces.

 - What we're talking about is "The Web Socket Protocol" (even if you
consider "The" and "Protocol" are not part of its name, it still
leaves "Web Socket"), and web browsers communicate with web servers
using a protocol whose name is "Hypertext Transport Protocol".
 - The name of the French President from 1974 to 1981 is "Giscard
d'Estaing" (this is his last name only)
 - "San Francisco" is well-known for its "Golden Gate Bridge", and
"New York" for its "Statue of Liberty" and "Empire State Building"

should I add more?

In computer science however, name is often a synonym for identifier,
which generally rules out spaces (among many other characters). See
for instance http://en.wikipedia.org/wiki/Name_(computer_science)

> It works technically but it is a mess conceptually.

I agree that a name *consisting* of space characters is pathological,
but I have much less strong opinions about names *containing* spaces.

> Call it "identifying string" or "token"

"token" has much stronger biases as "name" for any developer IMO.
http://en.wikipedia.org/wiki/Token_(parser)

But the whole thing isn't much about what is "valid" than what is
*allowed* and how error recovery works. If the sub-protocol name
cannot contain a space, then how should a client/server behave when it
receives one? should it consider the space part of the name? (which
actually implies the space is allowed) ignore everything after the
space? (that complicates things in ways that are not worth it IMO) or
refuse the whole thing and close the connection?

-- 
Thomas Broyer
/tɔ.ma.bʁwa.je/

---

From: Pieter Hintjens <[email-masked]>
Date: Tue, 8 Dec 2009 17:07:21 +0100

> but I have much less strong opinions about names *containing* spaces.

Sure.  You're agreeing that "name" has a well-defined sense, that
includes multiple words.

> http://en.wikipedia.org/wiki/Token_(parser)

True.

> refuse the whole thing and close the connection?

I'm sure there are several good options.  However, that's not my
concern here, it's about using terminology that does not mangle common
sense meaning.

If one is to allow a string containing spaces and other unusual
characters, one needs to describe it accurately, or implementors will
make avoidable mistakes.  Call it "string", if nothing else.

If one wants to use strings that are meaningful to the outside world,
they might treat spaces as separators (if allowed), and then "name"
would be a valid label.

It does shock me to read a protocol designer say, "a space is a valid
name"... because it makes me wonder how many other alien concepts I'll
stumble on in the spec.

One needs to apply the Principle of Least Surprise, Ian.

-Pieter

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 14 Dec 2009 13:05:07 +1100

> reasons that I don't really understand, and the handshake is supposed to 

From <http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-08#section-3.2>:

   Historically, HTTP has allowed field content with text in the ISO-
   8859-1 [ISO-8859-1] character encoding and supported other character
   sets only through use of [RFC2047] encoding.  In practice, most HTTP
   header field values use only a subset of the US-ASCII character
   encoding [USASCII].  Newly defined header fields SHOULD limit their
   field values to US-ASCII characters.  Recipients SHOULD treat other
   (obs-text) octets in field content as opaque data.

If you really want to define a UTF-8 HTTP header, you're allowed to do so by this text (which makes ASCII only a SHOULD); processors not expecting it SHOULD still treat it as if it were opaque data.

--
Mark Nottingham     http://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Fri, 18 Dec 2009 10:39:35 +0100

> name"... because it makes me wonder how many other alien concepts I'll

I just stumbled over 
<http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-68#section-1.1>:

      field         = 1*name-char colon [ space ] *any-char cr lf
      colon         = %x003A ; U+003A COLON (:)
      space         = %x0020 ; U+0020 SPACE
      cr            = %x000D ; U+000D CARRIAGE RETURN (CR)
      lf            = %x000A ; U+000A LINE FEED (LF)
      name-char     = %x0000-0009 / %x000B-000C / %x000E-0039 / 
%x003B-10FFFF
                      ; a Unicode character other than U+000A LINE FEED 
(LF), U+000D CARRIAGE RETURN (CR), or U+003A COLON (:)
      any-char      = %x0000-0009 / %x000B-000C / %x000E-10FFFF
                      ; a Unicode character other than U+000A LINE FEED 
(LF) or U+000D CARRIAGE RETURN (CR)

Note that name-char allows U+00.

BR, Julian

---

From: Pieter Hintjens <[email-masked]>
Date: Fri, 18 Dec 2009 13:15:39 +0100

On Fri, Dec 18, 2009 at 10:39 AM, Julian Reschke <[email-masked]> wrote:

>     name-char     = %x0000-0009 / %x000B-000C / %x000E-0039 / %x003B-10FFFF
>                     ; a Unicode character other than U+000A LINE FEED (LF),
> U+000D CARRIAGE RETURN (CR), or U+003A COLON (:)

I'm struggling to digest the logic that allows nulls (and spaces) but
not colons in a "name"...

-Pieter

---

From: Thomas Broyer <[email-masked]>
Date: Fri, 18 Dec 2009 15:12:32 +0100

On Fri, Dec 18, 2009 at 1:15 PM, Pieter Hintjens wrote:
> On Fri, Dec 18, 2009 at 10:39 AM, Julian Reschke wrote:
>

Because colons (and CRs and LFs) are used as separators?
If nulls (and spaces) have no single role and/or processing rule, why
shuold they be ruled out?
And again, if your concern is about the name chosen to denote this
part of the syntax ("name"), why don't you come with something better?
("key" could be better, but not "id" or "token", as has been proposed
earlier)

Let's move to "plain English": a field is a name/value pair, on one
single line (so no CR or LF), with the name and value being separated
by a colon (optionally followed by a space; and the name preceding the
value, of course).
Parse that with, say, a regex: [^:\r\n]+: ?[^\r\n]*\r\n
Now, express that regex as ABNF: (see what Julian quoted)
Now, either you're saying that the first part shouldn't allow nulls
and spaces, but then you have to define how to handle a line received
with either a null or space before the colon (and if you say "treat
them as being part of the 'name'", then you've not actually changed
anything to the spec, as implementations would be implemented with the
exact same regex –or equivalent code, of course– as the one I gave
above), or you're bothered that the field's name is called "name"
because it only rules out what *needs* to be ruled out (colon, CR and
LF) rather than what is generally ruled out in other contexts by
things called "name" (and then please propose another name)

Now, could we move on the discussion about the protocol itself rather
than nitpicking, please?

-- 
Thomas Broyer
/tɔ.ma.bʁwa.je/

---

From: Julian Reschke <[email-masked]>
Date: Fri, 18 Dec 2009 15:21:30 +0100

> shuold they be ruled out?

Maybe because otherwise people using NUL-terminated strings *will* 
misimplement this?

> Now, could we move on the discussion about the protocol itself rather

With all due to respect, this isn't nitpicking but a serious concern.

BR, Julian

---

From: Pieter Hintjens <[email-masked]>
Date: Fri, 18 Dec 2009 16:05:18 +0100

> Now, could we move on the discussion about the protocol itself rather

I'm not sure that it's appropriate or productive to dismiss a concern
about interoperability as "nitpicking".

My question was, "if we define a 'name' as a string that can contain
arbitrary nulls and spaces, what other strangeness can we expect in
the protocol".  Put this another way: if the designers seem to feel
that "[^:\r\n]+: ?[^\r\n]*\r\n" is a sane pattern for name: value,
then my confidence in the protocol falls by a significant degree.

Let me take three specific concerns then.

First, this breaks in any language where a null is significant in a
string.  What rules are there for making this work in C?  Or is that
not considered a valid language for implementors?

Second, it's a lazy design that puts an onus of interpretation onto
the implementor, with predictable consequences.  Some implementors
will literally allow anything (such as a null), and some will disallow
characters their particular languages disallow.  This will create
interoperability failures.  Grey areas always result in interop
problems.

Third, it's made worse here by the confused terminology.  The pattern
appears to imitate an HTTP "field-name: field-value" but in fact does
not.  Again, some people will assume HTTP rules (where the name is a
'token') and others will read the spec literally and think of clever
ways to use nulls and spaces and other not-explicitly-disallowed
characters.

It would be much better to explicitly define what is valid in a "name"
so that there is no grey area.  And to make this explicitly safe for
all languages.  And to make it work exactly like HTTP headers.

-Pieter

---

From: Justin Erenkrantz <[email-masked]>
Date: Fri, 18 Dec 2009 09:51:33 -0800

> about interoperability as "nitpicking".

Ditto.

I think having a binary-only protocol (and ones where NULL characters
must be transferred no less) is much harder to implement - especially
when there are so many things hidden inside the current "Web Sockets"
protocol drafts that are simply going to trip up basic
implementations.

In reading the archives of this list, I guess my overarching concern
with the "Web Sockets" protocol is that it apparently intentionally
tries to make it hard to implement.  Jamie's earlier comments that it
has nothing to do with HTTP/1.1 and should be treated akin to a new
protocol shows just how far separated from reality this RFC is.

As a developer and implementor, that's why BWTP was so attractive to
me - it focuses on getting the core problem solved (doing async
requests/responses) while not wasting its time dealing with
insignificant optimizations.  In my mind, incremental improvements are
better than wholesale changes when those wholesale changes are made to
just make wholesale changes.  If I wanted a brand-new binary protocol
that had no relationship to HTTP/1.1, I wouldn't bother attaching the
name "Web" to it - it just smells of false advertising.

My $.02.  -- justin

---

From: Scott Ferguson <[email-masked]>
Date: Fri, 18 Dec 2009 10:28:44 -0800

> protocol drafts that are simply going to trip up basic
Has there been any thought to splitting the protocol into two layers? 
(I've just joined the list recently, so may have missed it.)

The current draft is really defining two separate capabilities:

  1) HTTP/1.1 Upgrade handshake to a raw TCP socket.
  2) two packet/framing protocols on top of the raw socket.

If these separate concerns were split, and the sub-protocol given full 
control over the framing protocol, then a lightweight binary framing 
protocol (like I need) wouldn't be crippled by something like BWTP, and 
BWTP wouldn't need to layer on top of a predefined framing protocol that 
it doesn't need.

In other words, I think agreeing on #1 is straightforward, but designing 
#2 to fit all applications is less likely to work. 
And even if a universal framing protocol could be designed, it may be 
better to be in a separate spec anyway for layering purposes. The 
browser API, of course, does need a framing protocol (the current text 
one should be fine for that purpose), but as the spec itself shows with 
the binary addition, that framing protocol isn't universal.

-- Scott

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 18 Dec 2009 19:38:43 +0000

> not considered a valid language for implementors?

Additional language notes:

Java encodes ASCII NUL in UTF-8 with a different encoding to standard
UTF-8.  Therefore, Java and C-like environments do not interoperate
perfectly in the presence of ASCII NUL, unless extra steps are taken -
which are, of course, often ignored.

Unix shells also fail when asked to handle NUL for the most part, due
to their C heritage.

C++ can handle NUL, up until the point you use the string with c_str().
Then it will truncate at the NUL.

Many system protocols, directory services and database queries will
not like being asked to look up something starting with a name
containing NULs and, often because they use C style NUL-terminated
strings in their APIs.

The POSIX regular expression matching functions do not operate
correctly on a string containing embedded NULs.  Unless they are
encoded with Java-UTF-8, that is.

These concerns may apply to the data framing later in the protocol.
When NUL is transmitted in a data frame, either as the byte 0x00, or
the Java-UTF-8 encoding of ASCII NUL, you may expect some
implementations to get confused about the content of a frame.
However, as long as they operate on bytes at the frame boundary
finding stage, boundaries are likely to be found correctly.

-- Jamie

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 19 Dec 2009 11:31:32 +1100

Wow - what a thread about such a simple concept as a name!!!!

But to inject a lateral thought.   I think the protocol name
should be a URL (kind of in the same way that DTD's and style sheets
are specified by URI).

Otherwise we are going to have problems with uniqueness and allocation
of protocol names - and will have to have central authorities policing
them etc.

If we make it a URL, then we have uniqueness and the URL can point
to a webpage that describes the protocol.

cheers

---

From: Thomas Broyer <[email-masked]>
Date: Sat, 19 Dec 2009 03:04:08 +0100

> With all due to respect, this isn't nitpicking but a serious concern.

Point taken.

How about ruling out U+00, and treat it as if it were an "error in
UTF-8" in case you actually receive one (§4.4 and §5.4 in -68)?

-- 
Thomas Broyer
/tɔ.ma.bʁwa.je/

---

From: Julian Reschke <[email-masked]>
Date: Sat, 19 Dec 2009 15:30:27 +0100

> UTF-8" in case you actually receive one (§4.4 and §5.4 in -68)?

That would help, although I'd prefer to exclude more, or to avoid the 
confusion we currently have, making this in-between binary and text 
(come on, CRLF and ":" as delimiters, that smells like text).

BR, Julian

---

From: Pieter Hintjens <[email-masked]>
Date: Mon, 21 Dec 2009 16:05:49 +0100

> should be a URL (kind of in the same way that DTD's and style sheets

Would that be a conventional URL or a new custom URL that allows nulls
and spaces?

:-)  Sorry for being unkind but once we break convention, where does it stop?

-Pieter