Subject: [hybi] Buffering and error handling (Was: Serf support for BWTP)

From: Ian Hickson <[email-masked]>
Date: Tue, 2 Feb 2010 01:03:52 +0000 (UTC)

On Mon, 1 Feb 2010, Greg Wilkins wrote:
> 
> I believe that the algorithmic specifications that remain in the 
> document actually introduce a significant lack of clarity and ambiguity.
> 
> The algorithms if implemented as described, would have measurable 
> external artefacts.
> 
> For example, when section 4.1 - 5 says "send the following bytes to the 
> remote server 47 45 54 20", do I actually have to send them?  Or can I 
> buffer them with the following bytes.

I've clarified the spec to say that a node can delay arbitrarily long 
(e.g. for buffering) whenever a byte is to be sent.

> Would a server be justified in rejecting the handshake (on security 
> grounds?) because those bytes are not sent in their own TCP/IP packet?

A server can reject the handshake based on the phase of the moon, the time 
of day, the size of packets, anything. No justification is needed.

> The specification relies on assumed common practise and understanding 
> that "send" can actually mean "buffer and flush sometime later, perhaps 
> aggregated with previous and/or following sends".  Relying on assumed 
> interpretation is not a good thing for a spec.

Agreed. That's a good point. I have updated hte spec accordingly.

> Another issue is that only the client-side sentinel framing algorithm 
> says:
>               "If the client
>               runs out of resources for buffering the incoming data, or
>               hits an artificial resource limit intended to avoid
>               resource starvation, then it must fail the Web Socket
>               connection and abort these steps."
> 
> Does this mean that the serverside or client length framing cannot
> reject a connection based on resource limits?  Must this sentence be
> put into every step of the algorithm that says "read a byte"?
>
> Surely it would be better just to express this error handling 
> non-algorithmicaly?

Good point. In fact there is already a paragraph to that effect under that 
algorithm. I've removed the above text.

> In which case your point about "sanely define all the error handling 
> rules" does not stand up well to this example.

By "error handling" I mean logic, not arbitrary limits. e.g. how to parse 
data using an unknown frame type, or how to handle unknown headers, etc.

> There are numerous error handling cases not handled in your algorithms:
> 
>  + int or long  overflow if a very very large number is sent as a
>    frame length.
>  + what to do if a client sends for the length 0x80, 0x80, 0x80, ....
>    do we read and multiply forever or can we conclude that somebody
>    is messing with us?

These is handled by the aforementioned paragraph after the algorithm.

> In the client framing, we are allowed to abort "If no byte could be read 
> because the Web Socket connection is closed".  But the server is not 
> granted that permission.  What can we do if a read times out but the 
> websocket is not closed?

The server is granted blanket permission to close whenever it wants. 
("Servers may close the Web Socket connection whenever desired.", in the 
section titled "Closing the connection".)

> In fact I can see no sanely defined error handling in these algorithms 
> at all.  Can you point me to one example in section 4.2 or 5.3 that is 
> both complete, un-reliant on common sense interpretation and could not 
> be better and less verbosely defined in imperative style?

Step 2 of the "Data framing" algorithm for clients handles all frame 
types, including all 254 errorneous types.

I've designed the framing so as to minimise the number of possible 
errors, so it's true that there aren't many cases of error handling. The 
point is that behaviour is defined for all possible byte streams.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Ian Hickson <[email-masked]>
Date: Tue, 2 Feb 2010 02:32:01 +0000 (UTC)

On Sun, 31 Jan 2010, Justin Erenkrantz wrote:
> 
> There is simply no way that I could translate the current draft to an 
> implementation without introducing serious bugs, flaws, or 
> interoperability issues.

Seriously?

> As a concrete example, let's look at Section 5.3:
> 
> ---
> 5.3.  Data framing
> 
>    The server must run through the following steps to process the bytes
>    sent by the client:
> 
>    1.  _Frame_: Read a byte from the client.  Let /type/ be that byte.
> 
>    2.  If /type/ is not a 0x00 byte, then the server may disconnect from
>        the client.
> 
>    3.  If the most significant bit of /type/ is not set, then run the
>        following steps:
> 
>    ....
> --
> 
> If I read this, what is the rationale behind the if clause of step 3? 
> We're guaranteed that /type/ is 0x00 at this point because the server 
> has just been told it's okay to disconnect in step 2.

It may disconnect, but it doesn't have to (there's no "must", it's a "may" 
-- the spec uses RFC2119 terminology). So there's no guarantee that the 
type is 0x00 at step 3.

> But, if I step back and think, I think that it is step 2 that is bogus 
> because, in other parts of the draft, you mention other frame types.

Neither step is bogus.

> In fact, the data framing rules for the client are different than the 
> server.
>
> Oops.

Why "oops"? The requirements are different on both sides. It's an 
asymetric protocol, like most.

> The confusion that was attempted to be prevented by this radically 
> different editorial style is exactly what was created.

With all due respect, don't try to read between the lines, and it'll be 
easier. In specs that don't use this explicit style, you have to read 
between the lines precisely because the information isn't there. But 
that's exactly the problem that this style solves. You just code up what 
it says, and you'll be interoperable.

---

From: Justin Erenkrantz <[email-masked]>
Date: Tue, 2 Feb 2010 08:58:45 -0800

> Seriously?

Yes.

> -- the spec uses RFC2119 terminology). So there's no guarantee that the

If, as you dictate in your reply, code up *exactly* what it says - you
must ignore the "may" (as you don't provide the conditions to
satisfy), then the server never disconnects.

That's absurd.  So, why mention the step at all?

If you are attempting to specify algorithms as the normative
definitions to force me to stop "reading between the lines" - but,
yet, at the same time, don't specify it to the correct level of
precision, then don't specify it at all or stop claiming that it is
normative!  This is *exactly* the ambiguity inherent in this style
that is so maddening as a prospective implementor.  -- justin