Subject: [hybi] Why Hello frames?

From: Greg Wilkins <[email-masked]>
Date: Thu, 16 Dec 2010 15:15:31 +0100

I thought it useful to restate the case for Hello frames separately
from the GET+Upgrade vs CONNECT discussion or any reference to the
security concerns of Adam.

Essentially the current -76, -00, -01 and -02  handshake is based on:

  HTTP request ->
  Unframed Client Nonce ->
    <- HTTP response
    <- Hashed nonce
  <- WS messages -->

The intent of the bytes after the HTTP messages was two-fold:
 a) to carry the nonce and hash that is needed to prove the server
understands websockets.
 b) to check that a channel has been established between client and server.

With regards to a), there is not actual need for the nonce and hash to
be carried in the unframed data. They could equally be carried in
headers in the request/response, so let's ignore the nonce and hash
stuff for now and the handshake becomes:

  HTTP request ->
  unframed non-HTTP data ->
    <- HTTP response
    <- unframed non-HTTP data
  <- WS messages -->

Unfortunately  this does not work for reason b) because it is not HTTP
compliant. A valid intermediary may wait for a 101 (if the request is
GET+Upgrade) or a 200 if the request is a CONNECT before forwarding
the unframed non-HTTP data.   Thus a server trying to accept a WS
handshake make block forever waiting for the unframed bytes.

To make the handshake HTTP compliant, no non-HTTP bytes should be sent
until after the HTTP response has been seen (this may be a 101 in
response to a GET+Upgrade or a 200 in response to a CONNECT).   Thus
the handshake could become:

  HTTP request ->
    <- HTTP Response
  unframed non-HTTP data ->
    <- unframed non-HTTP data
  <- WS messages -->

But this suffers from an extra round trip before any app data can be
sent.   However, there is no reason that the client needs to check the
availability of a channel before the server does. So we can avoid the
round trip by doing:

  HTTP request ->
    <- HTTP Response
    <- unframed non-HTTP data
  unframed non-HTTP data ->
  <- WS messages -->

Note that the server can send messages before the clients unframed
non-HTTP data arrives.  The server only needs to wait if it wants to
validate that the client has hashed a server nonce, but that is a
proposed feature that would need a RTT no matter what the handshake
is.

So this handshake is now HTTP compliant and tests to see if a channel
has been established after the HTTP request and response.   It is a
good test for the channel because the data flow is in the reverse
direction for what a normal request/response cycle would be. But it
can still be improved.

Firstly, we are checking to see if the channel has been established by
sending non-HTTP data and seeing if it is forwarded.   But how do we
know it is non-HTTP?    If it is just random bytes, then there is a
chance that the data will either be HTTP or white space or just not
bad enough to trip up a HTTP parser. Also since we are sending the
non-HTTP data to check if we can send WS frames, why not actually send
WS frames to do the check?    The handshake thus becomes:

  HTTP request ->
    <- HTTP Response
    <- WS framed non-HTTP
  WS framed non-HTTP  ->
  <- WS messages -->

Now we might not have any application data to send in those WS frames,
so we need to invent some WS frames to send. These could be ping/pong
frames or a new Hello frame.   Let's go with a the Hello frame, so the
handshake becomes:

  HTTP request ->
    <- HTTP Response
    <- WS Hello frame
  WS Hello frame  ->
  <- WS messages -->

So do we now know that the channel will forward non HTTP data?
Currently there is still a small chance that a WS frame can look like
a HTTP data.    Thus we need to change WS framing so that frames
cannot be HTTP - hence the proposal to switch the MORE bit to a FIN
bit, so for a single frame it will start with a non-ascii character,
which is certainly not HTTP.

Also what content should we put in the hello frames.   Well the
current handshake sends the unrelated nonce and hash in the bytes
after the request/response, so we can do the same - although the
argument can be made to move those entirely to the headers to keep the
features separate.    I could also see an argument to include the URL,
subprotocol and Origin information in the Hello frames as that would
allow them to be used as part of a non HTTP handshake to a dedicated
port.   But essentially for the purposes of the Hello frame proposal,
the contents of the frames in mostly unimportant.

Note that this argument for the use of Hello frames and the FIN bit is
made entirely independently of using GET+Upgrade or CONNECT.  The
change to make sure the framing is not HTTP is also justified without
reference to the vulnerability discovered by Adam (the change may help
with that vulnerability, but that is for a different discussion).
Inverting the initial direction of non-HTTP bytes is also an
improvement to checking if a non HTTP channel has been established.

The reasons for the Hello frame and inverted FIN bit remain even if we
use CONNECT and/or masking of data in the requests.

regards

---

From: Joe Mason <[email-masked]>
Date: Thu, 16 Dec 2010 13:11:15 -0500

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Greg Wilkins
> Sent: Thursday, December 16, 2010 9:16 AM
> To: Hybi
> Subject: [hybi] Why Hello frames?
> 

What is the case for defining a new Hello frame rather than just using the existing Ping/Pong frames?

> which is certainly not HTTP.

I have not heard any argument that flipping the MORE bit will hurt anything, only that it is not sufficient to prevent the payload from looking like HTTP when there is fragmentation.  But even if not sufficient it will decrease the attack surface.

I propose releasing a -04 spec identical to -03 except with the MORE bit flipped.  That way the only framing change still under discussion will be the new Hello frame type, which is an addition and not an alteration.  That way it will be clear that the most recent spec represents the consensus on the frame but with the old handshake.

Joe

---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

---

From: Gabriel Montenegro <[email-masked]>
Date: Thu, 16 Dec 2010 19:33:02 +0000

> existing Ping/Pong frames?

Ping/Pong frames are currently defined to carry the same content, which is not the case for these Hello frames.

> I have not heard any argument that flipping the MORE bit will hurt anything,
> only that it is not sufficient to prevent the payload from looking like HTTP
> when there is fragmentation.  But even if not sufficient it will decrease the
> attack surface.

Yes, it is a mitigation. We could certainly look at masking, but there are issues about that, e.g:

http://www.ietf.org/mail-archive/web/hybi/current/msg04944.html 

Alternatively, one could send the connect exchange after the Upgrade exchange (as proposed by Dave Cridland), but flipping the bit is an incremental change that will help.  

> I propose releasing a -04 spec identical to -03 except with the MORE bit
> flipped.  That way the only framing change still under discussion will be the
> new Hello frame type, which is an addition and not an alteration.  That way it
> will be clear that the most recent spec represents the consensus on the frame
> but with the old handshake.

I think having the Hello frames makes sense as it allows flexibility in what the server and the client send in them without requiring them to be the same (as Ping/Ping would).
For example, after some side discussions, I'm not convinced that the client's including the hash of the server nonce in its hello serves much purpose.  If so, the client could send a very simple Hello without the hash of the server nonce, which, in turn, could simplify the protocol by getting rid of the server nonce altogether. A simple Hello frame's main purpose is to have naïve proxies trip on this non-ASCII message and the server nonce is not required for that.