Subject: [hybi] Proposal: HTTP upgrade process

From: Greg Wilkins <[email-masked]>
Date: Mon, 16 Aug 2010 11:23:41 +1000

All,

there has been a lot of posting about the -76/-00 style handshake,
it's HTTP compliance issues, it's fast fail (or otherwise)
characteristics, it security features etc.    I don't think any of the
conversations have been very productive nor is there any apparent
convergence on a solution.

I think the reason for his is that we are starting with a solutions
(the 8 random bytes etc.) and trying to reverse engineer the
requirements for it and a retrospective consensus for it's inclusion
into the draft. Thus I would like to propose  that we re-start
consideration of the handshake with the -75 style handshake and try to
move forward from there by identifying problems/requirements,
discussing solutions and then applying the consensus solution to move
forward.

Instead of contributors having to make the case of why the random
8-bytes etc. should be excluded from the protocol, those that support
random bytes should clearly make the case for their inclusion.   We
need to identify the issues with a simple upgrade request, and then
discuss various solutions as to how to address these - it may be that
the random 8 bytes solution is the best solution, but if so, then
there can be no harm in conducting a process that verifies this is the
case.

Note that I'm not saying that we should immediately publish a -01
draft with the handshake reverted, as I think it would be sufficient
for us just to restart our discussions from that simpler base (but I
would not object to a reset -01 draft either).

regards

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 16 Aug 2010 08:09:05 +0200

> discussing solutions and then applying the consensus solution to move

Greg, I agree with your point and am willing to help spot practical
issues with various proposals.

Regards,
Willy

---

From: Maciej Stachowiak <[email-masked]>
Date: Mon, 16 Aug 2010 02:05:01 -0700

> discussing solutions and then applying the consensus solution to move

It's not clear to me if the random 8 bytes have been sufficiently justified. My understanding is that it's intended as a fast-fail mechanism for problematic intermediaries, but I don't think it's been demonstrated that it works. However, I do not think reverting to an earlier version of the protocol would be a productive step:

(1) The WG adopted the -76 version. It seems reasonable to me that we should move forward from there. Jumping back to earlier versions will just force us to re-solve various problems.

(2) The -75 version had significant vulnerability to cross-protocol attacks, which is a critical security issue. While -76 is not the best effort we've seen, it is far better than -75. These security fixes are not all dependent on the magic 8 bytes. I think it would be irrational to revert critical security security fixes over a tangentially related issue.

(3) At the time of the -75 draft, it was claimed by many WG that it was not HTTP compliant. If that is indeed the case, then I do not see what purpose is served by trading the non-HTTP-compliance of -76 for the different non-HTTP-compliance of -75.

Conclusion: we should solve the "8 random bytes" problem, or at least, figure out if the benefits outweigh the costs, but not by reverting to an earlier draft.

Regards,
Maciej

---

From: Daniel Stenberg <[email-masked]>
Date: Mon, 16 Aug 2010 19:37:25 +0200 (CEST)

On Mon, 16 Aug 2010, Maciej Stachowiak wrote:

> Conclusion: we should solve the "8 random bytes" problem, or at least, 
> figure out if the benefits outweigh the costs, but not by reverting to an 
> earlier draft.

+1

-- 

  / daniel.haxx.se

---

From: Vladimir Katardjiev <[email-masked]>
Date: Tue, 17 Aug 2010 10:12:23 +0200

On 16 aug 2010, at 03.23, Greg Wilkins wrote:

> [snip for length]

That's (relatively) easy. From Ian's omnibus WebSocket Feedback mail on April 15:

>> If the handshake messages are to contain content, then they MUST have 
>> headers to indicate the content length to be legal HTTP: in this case, a 
>> Content-Length header would be appropriate
> 
> The idea here is that the 8 random bytes are part of the post-Upgrade Web 
> Socket connection, not the GET request, so that any intermediaries will 
> fail to send the data along if they do any interpretation, thus failing 
> early. If we included them in the Content-Length then unfortunately the 
> early failure mode of intermediaries would be lost.

If memory serves, it was also used for checking if the server programmer is also capable of following instructions and generating hashes, a notion that might not be required if the working assumption is no longer that every script implements its own WebSocket stack. This would leave the main reason for those eight bytes a sort of "fail-early" test, so that a WebSocket connection will terminate before the "open" event has been fired in the browser. This is considered preferable to firing the "open" event after a successful handshake, but then losing the connection because, oops, after all it was an incompatible intermediary.

The reason it was baked into the same request package was to avoid having an additional RTT. That is, after the 101 response was received, send a WebSocket-specific frame to the server and await a reply to receive a guarantee that the connection has been successfully established and can transfer WebSocket frames.

> would not object to a reset -01 draft either).

If it is just for these eight bytes I don't see why. As Maciej pointed out, -75 had its own share of problems, so I don't see why we should trade one for the other.

Cheers,
Vladimir

---

From: "Shelby Moore" <[email-masked]>
Date: Tue, 17 Aug 2010 05:07:46 -0400

[snip]

> If memory serves, it was also used for checking if the server programmer
> is also capable of following instructions and generating hashes, a notion
> that might not be required if the working assumption is no longer that
> every script implements its own WebSocket stack. This would leave the main
> reason for those eight bytes a sort of "fail-early" test, so that a
> WebSocket connection will terminate before the "open" event has been fired
> in the browser. This is considered preferable to firing the "open" event
> after a successful handshake, but then losing the connection because,
> oops, after all it was an incompatible intermediary.

But then forcing every proxy in the universe to be upgraded with a
specific code path every time someone else uses HTTP Upgrade to create a
new protocol with "early fail" hack:

http://www.ietf.org/mail-archive/web/hybi/current/msg03203.html

If not failing after WebSocket "open" event is a high priority, then
"officially" (admittedly imperfect and flexible process) declared
concensus was we need to find another transport option, rather than hack
the HTTP spec.

Maybe this is off-topic for this thread is I think about "process", not
the details of the technical debate.

[snip]

---

From: Vladimir Katardjiev <[email-masked]>
Date: Tue, 17 Aug 2010 12:18:58 +0200

Hm. It seems I accidentally sent out the wrong version of the mail. I didn't intend to sound as if I was arguing for the eight bytes themselves. What I thought I wrote in addition was that the intention was for the 8 bytes to be functionally part of the WS stream, so figure this:

1) HTTP Upgrade request
2) 101 response
3) 8 bytes random req
4) WS OK response

Since this would be 2 RTTs (1-2 + 3-4), in -76, 1&3 and 2&4 were baked into each-other. I don't necessarily agree with that; I'd prefer separation of concerns, but I was mainly intending to explain why those eight bytes found their way into the text in the first place.

Vladimir

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 17 Aug 2010 13:11:38 +0200

> Hm. It seems I accidentally sent out the wrong version of the mail. I didn't intend to sound as if I was arguing for the eight bytes themselves. What I thought I wrote in addition was that the inte…

Yes, that's it. And the 8 bytes random could even be a PING message.

> Since this would be 2 RTTs (1-2 + 3-4), in -76, 1&3 and 2&4 were baked into each-other.

no, not necessarily 2 RTTs if the sender sends the 1+3 at the same time,
then the responder can receive 1, send 2, then receive 3 and send 4.

Please check the various cases of HTTP vs round trips in my earlier e-mail here :

   http://www.ietf.org/mail-archive/web/hybi/current/msg03238.html

> I don't necessarily agree with that; I'd prefer separation of concerns, but I was mainly intending to explain why those eight bytes found their way into the text in the first place.

It was to ensure that intermediaries were able to forward them. But in my
opinion, it does not bring that many advantages over sending them as headers.

Regards,
Willy

---

From: Vladimir Katardjiev <[email-masked]>
Date: Tue, 17 Aug 2010 14:30:47 +0200

> opinion, it does not bring that many advantages over sending them as headers.

When it was under discussion, it was quite the opposite actually. The random eight bytes were intended to _break_ intermediaries that didn't follow HTTP Upgrade correctly. That's why there's no point to those bytes if they are sent in a header -- they might as well not be sent at all.

It has to be something that comes where HTTP would expect there to be a "body", otherwise it won't adequately test whether or not the established connection can support websocket messages. At least that was the intention. I still feel that it's a worthwhile goal to ensure that the established connection is usable before we throw it over to the application.

Vladimir

---

From: Salvatore Loreto <[email-masked]>
Date: Tue, 17 Aug 2010 14:31:12 +0200

> (1) The WG adopted the -76 version. It seems reasonable to me that we should move forward from there. Jumping back to earlier versions will just force us to re-solve various problems.
as explained in previous mail

    draft-ietf-hybi-thewebsocketprotocol-00 was used as an input document
    for the HyBi WG.  As it is an Internet-Draft, it is work in
    progress.  In other words, it is subject to change.

having said that, I agree that any changes should move the spec forward, 
be an improvement and solve the issues that
have been founded and largely discussed within the WG.
So restart from -75 does not make sense.

> Conclusion: we should solve the "8 random bytes" problem, or at least, figure out if the benefits outweigh the costs, but not by reverting to an earlier draft.
there has been a lot of mails and good technical discussion about
- the need of "8 random bytes" for security reasons
- the fact that as the "8 random bytes" are used  in the 00 wg version 
they do not work for reverse proxies and are non-HTTP-compliance,
- the possibility to use the "8 random bytes" in a way that both work 
with reverse proxies and are HTTP-compliance among the others:

    * inserting the 8 random bytes in an header
    * using the Willy Tarreau analysis/proposal explained in
      http://www.ietf.org/mail-archive/web/hybi/current/msg03238.html

So in order to move the work forward I would invite people to send out 
text (in an adhoc thread) proposing an handshake that address all the 
concerns raised from
different people (in the same way is happening for the frame issue).

regards
/Sal

> Regards,
> Maciej
>
>
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>    

-- 
Salvatore Loreto
www.sloreto.com

---

From: "Shelby Moore" <[email-masked]>
Date: Tue, 17 Aug 2010 08:55:41 -0400

[snip]

> different people (in the same way is happening for the frame issue).

Process question follows.

I understand you are referring to moving forward on the handshake for WS
on HTTP port 80.  I understand that can be an orthogonal process to
decisions about whether we should use HTTP port 80 for WS.

I have an unanswered question/challenge about whether we can use HTTP port
80 for WS reliably:

http://www.ietf.org/mail-archive/web/hybi/current/msg03299.html
http://www.ietf.org/mail-archive/web/hybi/current/msg03303.html

What is the standard process to follow? Is the standard or amicable
procedure to assume that if no one replies a challenge, then the challenge
does not deserve merit, or to assume it was correct on merits? Or is the
interpretation an individual decision and ambigious for the group
decision?

I intuitively understand that a WG is a concensus building process, so am
I correct to assume that if no one replies to a challenge, then it means
you've been effectively told that your ideas are not worthy?

Thanks in advance for the clarification on process.