Subject: [hybi] Websocket algorithmic specification

From: Greg Wilkins <[email-masked]>
Date: Tue, 02 Feb 2010 09:41:26 +1100

Sorry for the cross post hybi/whatwg.
I raised these issues on hybi list, but there was no
specific response (probably due to the other issues
being debated there).

I think the issues below indicate ambiguities in the
spec.  I raised them as an argument to change away from
the algorithmic style of specification, but even if that
is not accepted, they need need to be somehow clarified
in this spec.

Error Handling
==============
The client side sentinel framing algorithm includes the
text in 4.2-2.-(second 2.):

    "If the client runs out of resources for buffering the
     incoming data, or hits an artificial resource limit
     intended to avoid resource starvation, then it must
     fail the Web Socket connection and abort these steps."

Surely this text must either be copied to every data
read (for length framing and for both server side framings)
or (better) moved to a overall "If at any point during these
steps ..." type statement.

The phrase
   "If at any point during these steps a read is
    attempted but fails because the Web Socket connection is
    closed, then abort"

is repeated for both client side framing algorithms, but
is not included in the server side algorithms.

None of the framing algorithms specify how a timeout
should be handled.

Error handling for low resources, closed connections
and timeouts is probably best specified once rather
than repeated everywhere the algorithms say read.
But if it is to be specified in the algorithms, then
it needs to be thorough and consistent.

Bad lengths
===========

The length encoding currently allows for a length
of 0x80 0x80 0x80 .... to be sent forever.   This
is a nonsense length, but could be used for DOS
attacks on servers.   I think the 0x80 value should
be explicitly defined as an error if given as the
first byte of a length.

If the spec is to include error handling in it's
algorithms, then it should state that the length
loop can be terminated if the accumulated value
exceeds some maximum.

Definition of "Send"
====================
The specification algorithms use the phrase "send the
following bytes" frequently.   For example 4.1-5 says:

 send the following bytes to the remote server
 47 45 54 20

A literal reading of the spec would interpret that as
meaning that the bytes actually have to be sent, while
any sane implementation is going to append the bytes to
a buffer to be sent/flushed sometime later.

Does this mean that ws server could validly reject a
handshake as non compliant if these bytes arrived
in the same packet as the subsequent bytes?

I know this point sounds pedantic, but give that the
spec is already deliberately pendantic about HTTP header
ordering a uppercase/lowercase, it is best to be clear.
The meaning of send needs to be clarified and if algorithmic
style is to be maintained, perhaps flush points should be
included?

regards

---

From: Ian Hickson <[email-masked]>
Date: Tue, 2 Feb 2010 03:10:54 +0000 (UTC)

> steps ..." type statement.

Fixed in response to the earlier e-mail.

> is not included in the server side algorithms.

Good point. Fixed.

> None of the framing algorithms specify how a timeout should be handled.

In what sense does this differ from the connection being closed?

> Error handling for low resources, closed connections and timeouts is 
> probably best specified once rather than repeated everywhere the 
> algorithms say read. But if it is to be specified in the algorithms, 
> then it needs to be thorough and consistent.

I think the spec is thorough now, though I agree it could be more 
consistent.

> The length encoding currently allows for a length of 0x80 0x80 0x80 .... 
> to be sent forever.  This is a nonsense length, but could be used for 
> DOS attacks on servers.  I think the 0x80 value should be explicitly 
> defined as an error if given as the first byte of a length.

How would this be different than sending 0x81 forever?

> If the spec is to include error handling in it's algorithms, then it 
> should state that the length loop can be terminated if the accumulated 
> value exceeds some maximum.

The spec says "If the user agent is faced with content that is too large 
to be handled appropriately, then it must fail the Web Socket connection".

> The specification algorithms use the phrase "send the following bytes" 
> frequently.  For example 4.1-5 says:
> 
> A literal reading of the spec would interpret that as meaning that the 
> bytes actually have to be sent, while any sane implementation is going 
> to append the bytes to a buffer to be sent/flushed sometime later.

I've explicitly mentioned that "send" can be delayed, but this is 
redundant with another statement in the spec which reads "Conformance 
requirements phrased as algorithms or specific steps may be implemented in 
any manner, so long as the end result is equivalent". In particular, 
there's no practical difference between buffering as you describe, and the 
program simply not running for a short while before the "send" step.

> Does this mean that ws server could validly reject a handshake as non 
> compliant if these bytes arrived in the same packet as the subsequent 
> bytes?

Servers can reject a handshake for any reason whatsoever.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 02 Feb 2010 14:22:49 +1100

> Fixed in response to the earlier e-mail.

great

> Good point. Fixed.

thanks

> In what sense does this differ from the connection being closed?

a closed connection is one that is no longer open.

a timeout is when you try to read from an open connection, but
nothing comes for a period of time.

I know you will say: "the server can close the connection at
any time".   But either the algorithms should specify all the
error handling or none.  Having a bit in the algorithms and
a bit in imperative style is confusing.

> How would this be different than sending 0x81 forever?

Sending 0x81 forever should also be caught, but by the
server detecting an overflow of the accumulating length byte.

Again, the point is - if the algorithm style is meant to
convey all the error handling, then it needs to be thorough.

If it's not specifying the error handling, then I'm not sure
what its giving use that the ABNF doesn't?

> to be handled appropriately, then it must fail the Web Socket connection".

Which is fine.   But if it also sometimes has explicit
error handling for resources in some algorithms, but not others,
then that is confusing.

> program simply not running for a short while before the "send" step.

The two approaches are not equivalent.  The result in entirely different
packets being sent over the wire.

But they are equivalent in terms of the ABNF definition of the framing,
which gives a precise definition, and can even be formally proved.

So still don't see any clarity from the algorithms.

> Servers can reject a handshake for any reason whatsoever.

Well that certainly makes for a very simple implementation of a
minimally compliant server!

regards

---

From: Justin Erenkrantz <[email-masked]>
Date: Tue, 2 Feb 2010 09:02:48 -0800

> convey all the error handling, then it needs to be thorough.

+1!

If the intent is to force implementors to "stop reading between the
lines", then the lines had better be crystal clear about what needs to
happen.  -- justin

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Feb 2010 17:10:53 +0000

>  send the following bytes to the remote server

No, because TCP does not preserve write boundaries.

-- Jamie

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 04 Feb 2010 09:08:59 +1100

> No, because TCP does not preserve write boundaries.

I know that... you know that..... but will the script kiddie
trying his hand at writing his own server know that?

Anyway, I think the point is moot, as Ian has been
adding text to the spec (in imperative voice :) to
specify what "send" means.

cheers