Subject: [hybi] WebSocket feedback

From: Ian Hickson <[email-masked]>
Date: Thu, 4 Mar 2010 03:21:28 +0000 (UTC)

In writing this e-mail I went through all the recent threads, and tried to 
respond to every point. (I haven't quoted all the e-mails, or this one 
would be kilometers long, but I think I've responded to every substantive 
point somewhere.) In this e-mail I discuss changes that I've made to the 
opening handshake, the addition of a closing handshake, and the reasons 
for not changing some other aspects of the protocol.

On an unrelated note, it has been brought to my attention that uploading 
drafts to the IETF site requires human intervention, and that the person 
behind this process has requested that I not update the draft as 
frequently. In order to maintain the transparency of the process, I've 
switched to putting the latest drafts here:

   http://www.whatwg.org/specs/web-socket-protocol/

When I update the IETF copy (which I'll now do more rarely) I'll also 
update it to clearly point to the above draft to avoid any confusion about 
where the latest copy is.

OPENING HANDSHAKES

There are several requirements that were not met by the previous 
handshake:

1. it didn't fail quickly enough with man-in-the-middle proxies, such that 
   the server's response would be sent back in some cases even though the
   connection wasn't established

2. its cross-protocol attack protection could be made stronger

3. it was hard to implement in some existing http stacks 

To address these, I've made some changes to the protocol.

To address #1, I've moved some of the handshake to the WebSocket 
connection, after the headers.

To address #2, I've introduced some of the ideas various people have 
suggested, such as requiring the server to prove that it has read the 
handshake, and designing the protocol such that it is impossible to 
include some of the unique elements using features that the Web platform 
supports. (For example, using Sec-prefixed headers.)

With the changes made for #1 and #2, I've been able to relax the parsing 
requirements, which should hopefully address #3.

On Wed, 3 Feb 2010, Maciej Stachowiak wrote:
> 
> According to a security expert I asked, the strongest defense against 
> cross-protocol attacks is in the first few bytes. So I think removing 
> all requirements on the status-phrase would make the protocol less 
> robust against cross-protocol attacks.

This is true.

For the blind attack case (causing a non-WebSocket-protocol client service 
to cause a WebSocket server to change state), I've tried to mitigate that 
issue by splitting the "proof" part of the handshake into three separate 
parts of the client handshake, and by making use of certain features that 
can't be emulated from the HTTP features of Web browsers, but it's 
certainly possible that these changes have actually made it easier for 
other protocols to be used to attack WebSocket servers. Fundamentally I 
don't know what we can do about this while remaining HTTP-compatible; the 
ideal solution, namely making the handshake appear to be essentially 
noise, is basically a non-starter if we have to work with HTTP.

For a WebSocket-to-other attack, where the goal is to connect to a remote 
server using the WebSocket API, the new handshake really does make it 
quite hard to achieve passively. You'd need to know what the client is 
expecting back, which means seeing the handshake and causing the server to 
somehow respond with the right MD5 fingerprint. I don't know of a way to 
do that without control of the server and the network. I wouldn't know how 
to defend against an attacker who already controls the network, even with 
the first few bytes being fixed.

On Wed, 3 Feb 2010, Jamie Lokier wrote:
> 
> The real trouble is how will a WebSocket/1.1 client interoperate with a 
> WebSocket/1.0 server, without trying, failing, then starting another 
> connection to try again which is wasteful and not entirely reliable.

The WebSocket protocol is (by design) not versioned. New features can be 
added by opt-in options, in a manner very similar to how the 
Sec-WebSocket-Protocol header is sent.

FRAMING

On Wed, 3 Feb 2010, Jamie Lokier wrote:
> Greg Wilkins wrote:
> >   + Why have two framing techniques when binary is sufficient to carry
> >     everything.
> 
> I agree.  While I acknowledge the argument that
> 
>     print length($text), $text
> 
> is an invitation to do the wrong thing in some languages, I think that 
> there are more ways that 0xff can lead to the wrong thing, because so 
> many languages pass around "nominal UTF-8" which is not guaranteed to be 
> UTF-8, making 0xff delimiting unreliable too.

It's true that if the server doesn't handle UTF-8 correctly, it can end up 
outputting 0xFF bytes. In practice, this would need some out-of-band 
erroneous data (e.g. an ISO-8859-1 form submission); you couldn't trigger 
this bug easily by sending data to the server and having it return it 
later, for instance. The length bug, on the other hand, could occur with 
no external data: sending non-ASCII data could very easily result in the 
server screwing things up if we use lengths rather than delimiters.

For scalability it's probably ideal if we can use lengths, but on the long 
run for environments where that matters we'll probably just use compressed 
frames which would be binary anyway (and thus length-encoded), so this 
will probably become a non-issue in that kind of environment.

> >   + Who controls allocation of the frame type byte?  So far every
> >     suggestion of usage for that (eg a bit to indicate that the
> >     frame contains meta-data headers) has been rejected.  So are
> >     binary users simply to pick their own bytes and hope for no
> >     collisions?  Will IANA eventually allocate values?  is 7 bits
> >     enough?
> 
> There will be no collisions for frame bytes which depend on the 
> sub-protocol name, as those frame bytes are privately agreed between 
> client and server.

If a client and a server are speaking a specific sub-protocol, they don't 
actually have to even use WebSockets -- they can just use a protocol that 
happens to look like WebSockets but defines whatever frame types they 
want.

If the client is a browser, and thus they're speaking generic WebSockets, 
then the frame types would be just those supported by the API, and there 
wouldn't be any custom types. So extensions would just be "registered" by 
revving the protocol.

> > [...] users can't be trusted to always provide valid utf-8 data, so if 
> > user data is not validated then sentinel encoding allows frame 
> > injection attacks.  After all we have learnt with HTTP, it seams silly 
> > to repeat the mistake of a protocol that is exposed to such attacks
> 
> If you have several bits of code sharing a connection - even if it's 
> just by sharing a common Javascript framework on a single page, then you 
> have security issues from this.

You certainly have security issues, but you don't have this issue (user 
data containing 0xFF), because the protocol requires the browser to make 
the data be valid UTF-8, and so you'll never be able to inject a 0xFF byte 
from the client.

Of course, if the server has other sources of data, and it doesn't 
validate them, then it's possible you'll have this problem in the other 
direction.

> If nowhere else, this should be in the SECURITY CONSIDERATIONS section 
> of the draft.

The spec does actually mention this in the security section:

# The biggest security risk when sending text data using this
# protocol is sending data using the wrong encoding. If an attacker
# can trick the server into sending data encoded as ISO-8859-1
# verbatim (for instance), rather than encoded as UTF-8, then the
# attacker could inject arbitrary frames into the data stream.

I'm happy to elaborate on this if you think it should be changed.

On Mon, 15 Feb 2010, Thomson, Martin wrote:
> 
> 1. The ABNF for "the wire protocol as allowed by this specification" 
> does not permit binary frames.

That is correct. There are no binary frames in this version of the 
protocol.

>      frame         = text-frame / binary-frame
>      binary-frame  = %x80 length *%x00-FF
>      length        = %x00 / %x01-7f / ( %x81-FF *%x80-FF %x00-7F )
> 
> This is a canonical length encoding in that it doesn't allow for leading 
> zeroes.

Leading zeros are not disallowed. (There wouldn't be much point 
disallowing them as far as I can tell.)

> 2. Value range alternatives (Section 3.4 of RFC 5234) should not include 
> a second "%x".

Fixed, thanks.

> 3. The definition for text-frame in the "the wire protocol including 
> error-handling and forward-compatible parsing rules" could be readily 
> simplified to
> 
>     text-frame   = %x00-7F *%x00-FE %xFF

Fair enough.

> I assume that %x80-7E, as included, is in error and %xFE was the 
> intended end of this range.

Oops, yes, thanks. Fixed.

CLOSING HANDSHAKES

As I see it, there are several use cases here that are important:

1. Server is done, and wants to close the connection, but doesn't want to 
miss any messages from the client, and doesn't want the client to miss any 
messages that the server just sent. The server might be done because it 
needs to shut down and have the client reconnect to some other server, or 
because e.g. the server is a game server and the game is finished, but the 
client might still send some user profile updates that need saving.

2. Server is done, and doesn't care about whether the client receives any 
of its messages or whether it receives any of the client's messages. e.g. 
the server could be sending regular stock ticker updates and the only 
possible messages from the client might be to decide which stocks to 
return.

3. Client is done, and wants to close the connection, but doesn't want to 
miss any messages that the server sent, and wants to ensure the server has 
had a chance to receive all the messages the client sent. The client might 
be done because e.g. the server is an IM server, and the user asked to go 
offline but the server might still send some configuration information 
e.g. the host to use when reconnecting.

4. Client is done, and wants to close the connection, and doesn't care 
about any pending messages either way. For example, the user closed the 
tab that was using the socket.

5. Client is done, and wants to close the connection, but wants all its 
messages to make it to the other side safely. It doesn't care about 
incoming messages from the server. (For example, the script called 
.close() or unregistered its event listener or the page unloaded.)

6. Neither server not client is done, but the connection terminates 
anyway. The client wants to reconnect but wants to make sure the 
connection picks up where it left off.

Use cases #2 and #4 are trivial. The server doesn't have to do anything 
special to handle them -- for #2 it can close the connection willy nilly, 
and for #4 it doesn't need to do anything when the connection is closed 
except close it.

Use case #3 can either be handled by the browser having an API that means 
"start closing but don't actually close yet, just don't accept to send any 
more messages", or by the application layer sending a message to that 
effect. It seems like having an API to do this is redundant, since it's so 
trivial to just encode this into the application-layer protocol (basically 
by asking the server to close the connection).

Use case #6 requires some processing overhead to track which messages have 
been received, which won't always be desired. In particular, I don't think 
we can make it automatic without requiring either all servers to implement 
it (which seems like a non-starter) or making it optional. Making it 
optional suggests leaving it to a future version, so that we get the 
basics right first. It would be relatively easy to add this as an option 
-- just have UAs that support it say so in their handshake, and servers 
that want to opt-in say so in _their_ handshake, and if both parties said 
it, then modify the framing accordingly. For example, maybe when the 
option is enabled, all messages are preceeded by a frame giving a message 
number, and periodically a message is sent back reporting the last message 
seen. We could add that relatively easily.

That leaves #1 and #5.

Both use cases are relatively easy to do at the protocol level if one side 
is always responsible for starting the connection shutdown. It gets more 
exciting if both peers are allowed to start the shutdown. Case #5 in 
particular is interesting, because it would be reasonable to assert that 
the API should guarantee that when you .send() some text and then .close() 
the WebSocket object, the server should be given a chance to receive the 
text.

I think the net result is that ideally .close() should map to sending a 
FIN, aka using shutdown(SHUT_WR). Unfortunately this is not always 
possible -- people cited numerous examples of cases where we can't rely on 
TCP semantics in practice -- so we probably want to simulate this at the 
WebSocket layer or the application layer. Since the JS API exposes a 
.close() method, we should probably expose it at the WebSocket layer.

This suggests that each side should be able to tell the other "I'm done 
sending". When it receives the "I'm done sending", or when it gets a FIN 
(recv() == 0), it can close the connection as soon as it's down sending.

I've now added this to the spec, using a simplified version of some of the 
ideas people put forward that involved a new frame.

As far as I can tell, it works fine if both peers decide to terminate at 
the same time; it's not dependent on who is the close initiator. It also 
handles anyone who can't do half-closes, since that is essentially an 
optional part of the protocol (with no black-box detectable effects at the 
Web Socket semantic layer).

On Tue, 9 Feb 2010, Greg Wilkins wrote:
> 
> if we have orderly close and error close, then
> it would be good to indicate some status in the onclose
> call back, so a websocket user/framework can handle
> differently the cases of - a) can't connect via network
> b) can connect, but permission denied c) was connected
> but got disorderly disconnected d) was connected and
> got orderly shutdown.

Done.

ISSUES OF EDITORIAL STYLE

I've made a few concessions regarding the editorial style, based on some 
of the more commonly raised issues: I've moved away from listing byte 
sequences, and have instead, where appropriate, listed things using their 
UTF-8 equivalent; and I've included more rationale in the introduction, to 
explain some of the design decisions.

On Mon, 1 Feb 2010, Justin Erenkrantz wrote:
> On Mon, Feb 1, 2010 at 3:48 PM, Ian Hickson <[email-masked]> wrote:
> > On Fri, 29 Jan 2010, Justin Erenkrantz wrote:
> >>
> >> Again, this is why the current draft is so impenetrable (to me) since 
> >> it expects that the only person implementing the draft is a client 
> >> vendor using synchronous socket methods...which sort of defeats the 
> >> purpose when you are trying to write a fully async client...or any 
> >> type of server.
> >
> > Could you elaborate on this? I looked for text that assumed 
> > synchronous socket methods but couldn't find any. Could you quote the 
> > offending text?
> 
> All of them are...such as:
> 
> ---
>           Run these steps.  If at any point during these steps a read is
>           attempted but fails because the Web Socket connection is
>           closed, then abort.
> 
>           1.  Let /length/ be zero.
> 
>           2.  _Length_: Read a byte, let /b/ be that byte.
> 
>           3.  Let /b_v/ be integer corresponding to the low 7 bits of
>               /b/ (the value you would get by _and_ing /b/ with 0x7F).
> 
>           4.  Multiply /length/ by 128, add /b_v/ to that result, and
>               store the final result in /length/.
> 
>           5.  If the high-order bit of /b/ is set (i.e. if /b/ _and_ed
>               with 0x80 returns 0x80), then return to the step above
>               labeled _length_.
> 
>           6.  Read /length/ bytes.
> 
>           7.  Discard the read bytes.
> ---

I don't understand why this implies synchronous socket methods. The spec 
explicitly says "Conformance requirements phrased as algorithms or 
specific steps may be implemented in any manner, so long as the end result 
is equivalent". There's nothing that requires that you implement the above 
using synchronous socket methods, so long as the end result is the same. 

On Wed, 3 Feb 2010, Jamie Lokier wrote:
> 
> Looking at the above, it would be quite a bit clearer to talk about 
> reading the end of the input (EOF, TCP FIN) than "if the connection is 
> closed", as the latter has several ambiguous meanings, and you don't 
> actually want to abort parsing when you know the socket is closed, if 
> you still have bytes to read from your buffer

It doesn't say to abort when the connection is closed; it says to abort 
when a read fails because the connection is closed. As far as I can tell, 
tha doesn't have the same ambiguities. In any case, hopefully some of the 
other changes made due to the closing handshake chages have resolved any 
other similar ambiguities.

On Tue, 2 Feb 2010, Greg Wilkins wrote:
> >> 
> >> The length encoding currently allows for a length of 0x80 0x80 0x80 
> >> .... to be sent forever.  This is a nonsense length, but could be 
> >> used for DOS attacks on servers.  I think the 0x80 value should be 
> >> explicitly defined as an error if given as the first byte of a 
> >> length.
> > 
> > How would this be different than sending 0x81 forever?
> 
> Sending 0x81 forever should also be caught, but by the server detecting 
> an overflow of the accumulating length byte.

Ok, how is it different from just sending infinite text in a 0x00 frame? 
Or, in HTTP, sending an HTTP header with an infinite value?

> Again, the point is - if the algorithm style is meant to convey all the 
> error handling, then it needs to be thorough.

That's not an error, just an inefficient way of encoding the length.

The point isn't to catch ever error, but that the processing be defined 
for all inputs. It doesn't matter how errors are handled, so long as they 
are handled.

On Tue, 2 Feb 2010, Justin Erenkrantz wrote:
> On Mon, Feb 1, 2010 at 6:32 PM, Ian Hickson <[email-masked]> wrote:
> > 
> > It may disconnect, but it doesn't have to (there's no "must", it's a 
> > "may" -- the spec uses RFC2119 terminology). So there's no guarantee 
> > that the type is 0x00 at step 3.
> 
> If, as you dictate in your reply, code up *exactly* what it says - you 
> must ignore the "may" (as you don't provide the conditions to satisfy), 
> then the server never disconnects.

"may" means that it is allowed, there are no conditions to satisfy.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: "Thomson, Martin" <[email-masked]>
Date: Thu, 4 Mar 2010 11:38:48 +0800

The concatenation of the number obtained from processing the |Sec-
   WebSocket-Key1| field, expressed as a little-endian 32 bit number,
   the number obtained from processing the |Sec-WebSocket-Key2| field,
   again expressed as a little-endian 32 bit number, and finally the
   eight bytes at the end of the handshake, form a 128 bit string whose
   MD5 sum is then used by the server to prove that it read the
   handshake.

Little-endian?

I'm from Blefuscu (i.e. the Internet), where we break our eggs on the big end.

This all seems a little complex and overcomplicated to me.

---

From: Ian Hickson <[email-masked]>
Date: Thu, 4 Mar 2010 03:43:53 +0000 (UTC)

> I'm from Blefuscu (i.e. the Internet), where we break our eggs on the big end.

Er, my bad. That was meant to be big endian. Fixed.

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 04 Mar 2010 08:25:10 +0100

Ian,

thanks for taking on board a lot of feedback and introducing some
interesting updates to the draft.

However, I hope that this type of mass update is a once off due
to the circumstances we find in starting off the WG.  In general
I think it would be far easier handle changes in smaller
increments and individual threads.    Also it would be good
to see proposed diffs to the draft before they are actually
just put in the draft.   Sorry - don't mean to rain on your
parade when you've done so much.

A couple of minor points.

If the handshake messages are to contain content, then they
MUST have headers to indicate the content length to be legal
HTTP: in this case, a Content-Length header would be appropriate

Considering that we are already shipping products with
websocket implementations from the existing draft, can we
specify a transition to the new handshake in an appendix.

Ie while the standard is under development connections without
websocket keys MAY be accepted, but implementation SHOULD warn
that support for such connections is deprecated.

The text of 1.3, still kind of implies that the HTTP fields
must be ordered as per the spec.  Can we add a sentence to say
that header ordering may be different.

Also I'm not sure this sentence is really that clear:

 "Additional fields are used to select options in the WebSocket
   protocol.  The only option available in this version is the
   subprotocol selector, |Sec-WebSocket-Protocol|:"

I thought we were going to allow arbitrary extra headers,
but that their interpretation was not defined by the ws spec
other than the optional Sec-WebSocket-Protocol.    We still
need to allow headers for cookies, authentication etc.

regards

---

From: Vladimir Katardjiev <[email-masked]>
Date: Thu, 4 Mar 2010 09:22:50 +0100

I like this update, especially that the handshake is human-readable in the spec now. Caveat that I haven't implemented it yet so I guess proper feedback comes then.

> HTTP: in this case, a Content-Length header would be appropriate

I interpret the "content" to not relate to the HTTP handshake, but belong to the WebSocket stream. To quote:

"I've moved some of the handshake to the WebSocket 
_connection_, after the headers." (emphasis mine)

Besides, a GET request can't have a body, so it's pretty much intended to break. However, if you instead view the sequence as WS Upgrade -> client sent body -> server sent reply it is perfectly valid.

> that support for such connections is deprecated.

Actually, it's not enough to just drop the keys. The header names changed too so you'd need to read WebSocket-Location instead of Sec-WebSocket-Location, for example. It's almost as if we're at a new "version" now.
> that header ordering may be different.

YMMV, but I don't see any such implication in the text.

Later on, the client handshake is explicitly asked to send the headers in a _random_ order (although _any_ order should do fine?) so there should be no confusion on that point.
> need to allow headers for cookies, authentication etc.

Yes. Client handshake step 15 explicitly adds those cookies. This part I guess is intended for things that can be modified directly using the API. The "arbitrary" extra headers permitted thus far are only things added by the browser implicitly. 

Vladimir

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 04 Mar 2010 09:44:42 +0100

Vladimir,

GET's can have content.  It's just the browsers mostly don't send
content with GETs.

The server needs to consider the bytes of content before formulating
the accepting the upgrade request and sending the 101 response, so
it can't be considered as data in the stream after the 101.

regards

Vladimir Katardjiev wrote:
> I like this update, especially that the handshake is human-readable in
> the spec now. Caveat that I haven't implemented it yet so I guess proper
> feedback comes then.
> 
> I interpret the "content" to not relate to the HTTP handshake, but
> belong to the WebSocket stream. To quote:
> 
> Besides, a GET request can't have a body, so it's pretty much intended
> to break. However, if you instead view the sequence as WS Upgrade ->
> client sent body -> server sent reply it is perfectly valid.
> 
> Actually, it's not enough to just drop the keys. The header names
> changed too so you'd need to read WebSocket-Location instead of
> Sec-WebSocket-Location, for example. It's almost as if we're at a new
> "version" now.
>>
> Later on, the client handshake is explicitly asked to send the headers
> in a _random_ order (although _any_ order should do fine?) so there
> should be no confusion on that point.
>>
> Yes. Client handshake step 15 explicitly adds those cookies. This part I
> guess is intended for things that can be modified directly using the
> API. The "arbitrary" extra headers permitted thus far are only things
> added by the browser implicitly. 
> 
> Vladimir
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 04 Mar 2010 09:47:21 +0100

> GET's can have content.  It's just the browsers mostly don't send

I'd also not be opposed to using POST.  That would prevent any automatic
retries of the GET request by HTTP infrastructure that does not know
about websocket.

cheers

---

From: Vladimir Katardjiev <[email-masked]>
Date: Thu, 4 Mar 2010 10:00:58 +0100

Heya Greg,

> GET's can have content.  It's just the browsers mostly don't send

Damn. I've been mired in REST stuff for so long, I could've sworn GETs shouldn't contain any body. You're correct, of course. Still, I do believe in this case it's not intended as an entity body, so not including Content-Length and Content-Type implies there is none (and is a valid way to send a request).

Again, I think the intention is to break the request if an intermediary doesn't handle WebSockets so sending the first WebSocket message as an entity body is probably not the desired effect.

> it can't be considered as data in the stream after the 101.

Unless I'm reading it completely wrong, the contents of the 101 handshake (since it has no content-length) can and are constructed purely from the headers. The challenge response contains data from both headers and the 8-byte challenge that comes later, but it's fine to do it. The steps of the algorithm could be reordered with no change of the output so what we call it is a matter of interpretation. Po-tay-to, poh-ta-to.

Vladimir

---

From: Julian Reschke <[email-masked]>
Date: Thu, 04 Mar 2010 13:39:02 +0100

On 04.03.2010 04:21, Ian Hickson wrote:
> ...
 > ...

Either this is a misunderstanding, it's a very *hard* working person, or 
actually many -- otherwise I don't see how it's possible to publish 
drafts at any time of day world-wide.

That being said, it's good advice not to use the IETF I-D publication 
process as substitute for version control.

Recommendation to this WG: once there are Working Group drafts, use 
trac/svn and tools.ietf.org to version control and issue tracking.

 > ...
> ...

Keep in mind that the I-D cut-off date is Monday; in general, IETF 
participants can not be expected to have read drafts that weren't 
published at that time.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 04 Mar 2010 14:08:29 +0100

> participants can not be expected to have read drafts that weren't

...clarifying: s/IETF participants/attendees of the WG meeting at IETF 77/

Best regards, Julian

---

From: Dave Cridland <[email-masked]>
Date: Thu, 04 Mar 2010 14:46:47 +0000

> server screwing things up if we use lengths rather than delimiters.
This is nonsensical - I don't see how the (rare) case of someone  
failing to distinguish between character counts and octet lengths is  
going to be more common than broken or invalid UTF-8.

> will probably become a non-issue in that kind of environment.

Except you've already thrown away binary framing.

> wouldn't be any custom types. So extensions would just be  

If we ever need to use frame types as part of the protocol (for  
chunking, to give one discussed example), then we *will* need to  
consider standardized frame types.

I'd be happy with, at this point, mere reservation of a range for  
protocol purposes, and leaving a range clear for subprotocol usage.

> validate them, then it's possible you'll have this problem in the  
Up above, you implied this was rare.

> disallowing them as far as I can tell.)

Yes, because multiple representations of the same values has *never*  
caused a security issue.

> Or, in HTTP, sending an HTTP header with an infinite value?

There is no difference, which is yet another reason why that's a  
silly framing style.

I still have no clear reason why a simple network-order 32-bit  
integer isn't acceptable, especially given the ideas on chunking.

> for all inputs. It doesn't matter how errors are handled, so long  

That's an awesome quote.

So, your defined behaviour in the face of a DOS or overflow attack is  
to, what? Just work?

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 04 Mar 2010 16:44:01 +0100

Vladimir Katardjiev wrote:

>> The server needs to consider the bytes of content before formulating the accepting the upgrade request and sending the 101 response, so it can't be considered as data in the
>> stream after the 101.
> 
> Unless I'm reading it completely wrong, the contents of the 101 handshake (since it has no content-length) can and are constructed purely from the headers. The challenge
> response contains data from both headers and the 8-byte challenge that comes later, but it's fine to do it. The steps of the algorithm could be reordered with no change of the
> output so what we call it is a matter of interpretation. Po-tay-to, poh-ta-to.

But the problem with this is it assumes success and that a 101 is the only
possible response.

I think that once we have gone through the process, it is very likely
that we will see the need for return types other than 101.

For example, it will be hard to stop some servers sending a 401
unauthorized if there are missing credentials. It might also be
desirable to send 302 responses if a server has moved or wants
to redirect to a non HTTP port or a secure https/wss port.

So if websocket wants to have an exchange on the wire after the
upgrade and before the framing starts, then sending non content
bytes after the upgrade request has to be conditional
receiving the 101... but then we have another round trip.

I really think we should stop trying to re-invent the HTTP upgrade
mechanism.  If there are problems with normal HTTP upgrade, then
raise them with HTTP bis.

If the bytes after the request header are sent as request content,
what attack verctor is opened up?

Note that it is OK for the response bytes to be sent as the
first bytes of the stream (and not part of the 101).

regards

---

From: Vladimir Katardjiev <[email-masked]>
Date: Thu, 4 Mar 2010 18:09:45 +0100

> But the problem with this is it assumes success and that a 101 is the only

According to the current draft, page 22, step 31, there are two possible status codes accepted: 101 and 407. In the case of 407, the entire request (including the random challenge) will be resent. In the case of 101, the assumption holds. In any other case, the websocket handshake is considered to have failed.
> to redirect to a non HTTP port or a secure https/wss port.

I'm not sure what a 401 could do to the client (the API doesn't allow credentials to be added, and I've never been a fan of the browser-generated "please log in" dialogs).

A 302 could be interesting. Apparently it's not in the spec right now, so maybe a new thread for that is in order?
> what attack verctor is opened up?

I don't think it's an attack vector anymore. Originally, it was to prevent XHR/CORS from creating a WebSocket connection, but the requirements on Sec-* headers should avert that. Right now it has more to do with having a websocket connection fail early in case of an intermediary that doesn't understand websockets.

Vladimir

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 04 Mar 2010 18:20:46 +0100

Vladimir Katardjiev wrote:

> I don't think it's an attack vector anymore. Originally, it was to 
> prevent XHR/CORS from creating a WebSocket connection, but the
> requirements on Sec-* headers should avert that. Right now it has
> more to do with having a websocket connection fail early in case
> of an intermediary that doesn't understand websockets.

I think there are plenty of intermediaries that don't
understand websocket that will pass these bytes on well enough
for the handshake to pass.

Problems with transparent intermediaries that don't understand
websocket are more likely to manifest themselves on small blocks
of bytes sent independently of a valid HTTP request.

Having a header (with spaces) inserted by the web socket
client that the server needed to echo back, was a solution
to the problem that appear to have pretty good consensus.
I think this latest proposal goes a little too far.

Probably we need to spend some more time getting the
requirements well written and accepted before we can decide
if such complications in the handshake are actually
needed by any agreed requirement.

cheers

---

From: Joe Hildebrand <[email-masked]>
Date: Thu, 04 Mar 2010 11:27:48 -0700

> parade when you've done so much.

The chairs were concerned about the number of changes that had to flow
through the IETF backend systems associated with many small changes.  Larger
changes getting batched up before formal submittal is the way we want to go
for now.

Keep in mind that we'll need to have consensus over the whole document by
the time we're done, so I don't see a need to propose individual chunks of
text before they're inserted in the document, particularly at this early
stage.

At the same time, particularly as this becomes a WG draft, a quick "I'm
about to..." mail to the list before a large change will help build that
consensus more quickly.  Each working group is a little different in this
regard, and we're still trying to find out what works for this crowd.

Of course, as always, whenever proposing a change to the document, it is
much more likely to get included in the way you want if you also propose
text.

-- 
Joe Hildebrand
(as chair)

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 04 Mar 2010 23:08:20 +0100

> text before they're inserted in the document, particularly at this early

Really?

I find creating a consensus for inclusion on merit far more
positive and productive than trying to find a consensus about
exclusion due to problems.

Otherwise, it makes for a bit of an asymmetric relationship
with the editors.

If I make a proposal, I've got to convince others that its a good
idea and then it should get included.

If the editor(s) have a proposal, they can just put it in the
document and if it is not liked, we have to have a create a
consensus to have it removed.

Take this latest idea of including a random string in the
stream after a valid request/response which goes significantly
beyond  anything that was discussed on the list.   Surely a
proposal like that should have raised on the lists first and
only if some support for it was achieve should it go into
the draft.

As it is, we have a solution in the draft for a problem
that we've not yet even fully defined or discussed.
Surely we should strive to work from requirements towards
solutions rather than the otherway around?

Note only that, it makes it hard for implementations that wish
to track the draft.  ie should I be updating my imple now to
match the draft, only to have to revert later if the idea
get's knocked back?

regards

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 05 Mar 2010 20:42:25 +0100

Graham,

it is my experience that GETs with bodies transit the internet without
problem - which is as the spec says they should.

But I agree that having a GET with a body can stretch the semantics
of the method.  For this reason and for avoiding auto retries, I think
POST would be a better method.

cheers

Graham Klyne wrote:
> From the "everything you always knew is (almost) false" department, this
> comment sent me off to re-read bits of RFC2616.  While what you say is
> (almost) technically correct, I think it's misleading.
> 
> [[
> A message-body MUST NOT be included in a request if the specification of
> the request method (section 5.1.1) does not allow sending an entity-body
> in requests. A server SHOULD read and forward a message-body on any
> request; if the request method does not include defined semantics for an
> entity-body, then the message-body SHOULD be ignored when handling the
> request.
> ]]
> 
> [1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.3
> 
> Where GET is defined in section 9.3 [2], there is no mention of message
> body either to prohibit or to define semantics, so it appears that while
> the presence of a message body on GET is just about legal, it SHOULD be
> ignored by any server that handles it.
> 
> [2] http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.3
> 
> #g
> -- 
> 
> Greg Wilkins wrote:
>> Vladimir,
>>
>>> "I've moved some of the handshake to the WebSocket _connection_,
>>> after the headers." (emphasis mine)
>>>

---

From: Julian Reschke <[email-masked]>
Date: Fri, 05 Mar 2010 20:51:49 +0100

> POST would be a better method.

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/19>

---

From: Mridul Muralidharan <[email-masked]>
Date: Sat, 6 Mar 2010 03:57:03 +0530 (IST)

This is very interesting !
Thanks for clarifying.

Regards,
Mridul

----- Original Message ----
> From: Julian Reschke <[email-masked]>
> To: Greg Wilkins <[email-masked]>
> Cc: Hybi <[email-masked]>; Graham Klyne <[email-masked]>
> Sent: Sat, 6 March, 2010 1:21:49 AM
> Subject: Re: [hybi] WebSocket feedback
> 

      The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 10 Mar 2010 09:04:23 +0100

All,

Is there some plan to transition from the browsers that support
the current draft to the browsers that support the document that
is produced as a result of this process?

Is it a requirement that we consider backwards compatibility to
the solutions already being shipped?

Obviously there will have to be some breakage, but currently
without any version information that could be hard to handle.
Even if we use Sec headers to infer versions, we need to define
behaviour for miss-matched versions.

regards

---

From: "Anne van Kesteren" <[email-masked]>
Date: Wed, 10 Mar 2010 10:42:29 +0100

> behaviour for miss-matched versions.

I don't see why that should be a requirement. With only one implementation  
shipping so far it is not more than an experimental feature. If we had two  
implementations shipping for well over a year things would be different...

-- 
Anne van Kesteren
http://annevankesteren.nl/

---

From: Vladimir Katardjiev <[email-masked]>
Date: Wed, 10 Mar 2010 14:24:23 +0100

> behaviour for miss-matched versions.

Strictly speaking, behaviour of version mismatch is failed handshake, and the client/server should disconnect. For socket access to the protocol this is easy to identify, detect and log. It's a bit harder from within a browser, though. (is there any handy way to see if/when chromium/webkit include new revision of WS?)

Cheers,
Vladimir