Subject: [hybi] Final HTTP responses and WebSocket handshake

From: [email-masked]
Date: Thu, 10 Feb 2011 17:40:24 -0800

After a careful reading of -05, the following text got me thinking of 
someting important, and
I would like to point that out here, and I would like to hear your 
thoughts.

In Section 1.3:
"Any status code other than 101 must be treated as a failure and the 
websocket connection aborted."

CURRENT STATUS (from what I understand):

I realized, the WebSocket handshake is done in the first half of HTTP GET 
transaction
(GET -> 101) according to the current specification, and it never expects 
final response codes
(2xx, 3xx, 4xx, 5xx) processed as a part of handshake. The 'websocket' is 
resides in the 
second half of the GET transaction, and as WebSocket is a 
persistent/bidirectional 
connection, it will never see a final reponse to the GET request. 
WebSocket handshake has
clearly been modeled after RFC 2817 (HTTP Upgrade to TLS) and it could be 
perceived as 
'similar' to what WebSocket is trying to achieve. However, I see a 
fundamental difference
in between RFC 2817 and WebSocket as following.

ESSENTIAL DIFFERENCE FROM RFC 2817 ("Upgrade TLS"):

Difference 1:
Upgrade TLS: Upgrading underlying transport layer to TLS, but not HTTP 
layer
WebSocket: Switching HTTP to WebSocket, but not transport layer

Difference 2:
Upgrade TLS: It eventually receives final respoonse to complete 
transaction. (No compomise in HTTP functionality)
WebSocket: The transaction never ends in terms of HTTP transaction.

With Upgrade TLS, the provisional response was necessary because 
subsequent response 
needs to be conveyed over the TLS, while WebSocket shouldn't have had to 
use this 'Upgrade'
mechanism to begin with.

POSSIBLE CONSEQUENCES:

o  Cannot get full benefit out of HTTP protocol during the handshake as it 
does not expect a final response:
  For example:
   - Can not redirect WebSocket handshake to another server (for load 
balancing, fail-over, etc.)
   - Can not perform user authentication (Basic, Digest, etc.) during the 
handshake.
   - Similarly, 407 authentication cannot be forced by the server.

   ## These impact on deployment of WebSocket services.

o Relatively more complexity in adding WebSocket to the existing 
implemtation of HTTP server.
  Reasons:
   - Existing HTTP implementation is likely designed to treat 101 as a 
indefinitive 
     acceptance of the request wihere it tries to prepare for the final 
response...
   - Where, on receipt of a final response, such as 200 OK, the likelihood 
of exisiting implementaion
     would prepare for releasing HTTP protocol specific resources in 
memory to terminate
     the transaction.
 
   ## This impact on adoption of WebSocket procol by existing HTTP 
servers.
   ## It is desure a looser coupling from HTTP specific function - easy to 
add WebSocket to existing system (e.g. Apache)

MY THOUGHTS/DOUBTS

- Why couldn't WebSocket handshake be done in a complete form of HTTP 
transaction?
- Any specific reasons for ruling out the usefulness of final responses 
for the handshake?
- Is the current spec really acceptable from deployment perspectives??
- Has 'non-browser' use of WebSocket truely been considered?
  (non-browser app may not have priori knowlege of final desitnation or 
any requirement of server/proxy authentication)
- Finally, I must be completely wrong about the current spec. (my hope! 
please give me a hint!)

DESIRED SEQUENCE EXAMPLES AS I SEE:

(1) Successful handshake

Client                             server
------                             ------
  |             GET                   |
  +---------------------------------->| << Connection: websocket is most 
important
  |                                   |    'Upgrade: websocket' isn't 
necessary.
  |             200 OK                |
  +<----------------------------------| << Release HTTP sepcific resource,
  |                                   |    (it's natural as HTTP 
transaction is complete)
  |<========== WebSocket ============>|    then, mark the socket as 
'websocket'
                                           optionally read/write data on a 
new socket
                                           from a special 
process/thread...
                                           (enables loose coupling from 
HTTP server core)
(2) 307 Temporary Redirect

Client                             server1  server2
------                             -------  -------
  |             GET                   |        |
  +---------------------------------->|        | \
  |                                   |        |  : Handshake == complete 
HTTP transaction
  |       307 Temporary Redirect      |        |  :
  +<----------------------------------|        | /
  |                                   |        |
  |             GET [1]               |        |
  +------------------------------------------->| \
  |                                   |        |  : Handshake == complete 
HTTP transaction
  |             200 OK                |        |  :
  +<-------------------------------------------| /
  |                                   |        |
  |<================ WebSocket ===============>|
  |                                   |        |

# I just thought I might as well speak out my concern. I am aware it is 
pretty late for this level of dicussion at this point, however...

Awaiting any of your response/objection/denials.

Best.
- Yutaka

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 11 Feb 2011 12:48:40 +1100

>    - Similarly, 407 authentication cannot be forced by the server.

+1

In the ancient history of this list, I have long argued for exactly this.

Apparently there are some security concerns about browser redirection,
which I never really understood... and even if there are, I don't see
the reason that the protocol should deny the use of non 101 handshake
responses to all clients.  I think the protocol should allow arbitrary
response codes (because it can't actually prevent them being sent by
non WS servers anyway), and then individual clients can decide which
responses they will handle (which may be better discuss in the
appropriate HTML 5 forums regarding the specific websocket API).

cheers

---

From: Gabriel Montenegro <[email-masked]>
Date: Fri, 11 Feb 2011 01:57:51 +0000

This is not aligned with the WG consensus for HTTP compatibility (up until the websocket handshake is complete) declared some time back by the chairs.
The websocket handshake is not complete until a 101 comes back. So anything before than (including other response codes, reauthentication, etc) are not anything the websockets draft can say anything about. It's just compatible HTTP as usual.

So this is a bug in the spec, hopefully to get taken care of in 06.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Greg Wilkins
> Sent: 10 February, 2011 17:49
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [hybi] Final HTTP responses and WebSocket handshake
> 
> > o  Cannot get full benefit out of HTTP protocol during the handshake
> > as it does not expect a final response:
> >   For example:
> >    - Can not perform user authentication (Basic, Digest, etc.) during
> > the handshake.
> Apparently there are some security concerns about browser redirection, which
> I never really understood... and even if there are, I don't see the reason that the
> protocol should deny the use of non 101 handshake responses to all clients.  I
> think the protocol should allow arbitrary response codes (because it can't
> actually prevent them being sent by non WS servers anyway), and then
> individual clients can decide which responses they will handle (which may be
> better discuss in the appropriate HTML 5 forums regarding the specific
> websocket API).
> 
> cheers
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: [email-masked]
Date: Fri, 11 Feb 2011 15:26:02 -0800

> This is not aligned with the WG consensus for HTTP compatibility (up
> until the websocket handshake is complete) declared some time back 
> by the chairs.
> The websocket handshake is not complete until a 101 comes back. So 
> anything before than (including other response codes, 
> reauthentication, etc) are not anything the websockets draft can say
> anything about. It's just compatible HTTP as usual.

Ah, I see. So, HTTP transaction can be iterated until 101 comes in.

I agree, this needs to be fixed. In fact, any status code other than 101 
MUST be handled
in a way that conforms with HTTP. (which is out of scope of this 
document).

Does any have any thoughts on my other concern?

> o Relatively more complexi in adding WebSocket to the existing 
indefinitive 
response... 
>    - Where, on receipt of a final response, such as 200 OK, the 
likelihood of exisiting implementaion 
memory to terminate 
servers. 
>    ## It is desired a loose coupling with HTTP specific function - easy 
to add WebSocket to existing system (e.g. Apache) 

Say, we try to implement WebSocket in Apache, or via Apache module. Since 
actual WebSocket
connection resides in the middle of the HTTP GET transaction, the process 
or thread that is
processing the transaction has taken up its sufficiently large resources 
for handling any type of
HTTP transaction, and most of which will no longer be necessary once the 
WS handshake is 
complete. Consequently, such implementation in Apache will be very 
expensive and probably
not practical for large scale deployment.

If we could complete the WS handshake in a complete HTTP transaction 
(successful 
handshake completes on receipt of 200 OK, for example), then this will 
have a completely
different view. Coming back to the example with Apache, the resources for 
the transaction
(process/thread/mempool/context, etc.) can be released completely. What 
needs to done
is in its socket management code be modified to leave the socket (at this 
point being 
'websocket' open and transfer it to an internal or external facility that 
can efficiently handle
persistent/bidirectional communication.

More specifically, the Connection header is about what to do with the 
underlying connection
*after* the current transaction is complete. So, we only needed to define 
a new value to
the Connection header like:

    Connection: websocket

(# of course, 'Sec-Websocket-xxx' headers are still necessary, 'Upgrade: 
websocket' can be eliminated.)

This can be reported to the socket management layer to tell, "the socket 
is going to be used
as WebSocket". (don't close and don't use it for subsequent HTTP 
transactions.)
Please notice that, there's no HTTP layer involved at this point!

This is clearly MUCH less work than tackling situation where transaction 
is left open upon 
101 response, and yet MUCH cleaner solution for introducing WebSocket 
feature into
existing HTTP servers.

Any thoughs?

--
Yutaka Takeda - US R&D Sony Computer Entertainment, Inc.
Office: 650 655 7484   Fax: 650 655 8060

[email-masked] wrote on 02/10/2011 05:57:51 PM:

> This is not aligned with the WG consensus for HTTP compatibility (up
> until the websocket handshake is complete) declared some time back 
> by the chairs.
> The websocket handshake is not complete until a 101 comes back. So 
> anything before than (including other response codes, 
> reauthentication, etc) are not anything the websockets draft can say
> anything about. It's just compatible HTTP as usual.
> 
which

---

From: John Fallows <[email-masked]>
Date: Fri, 11 Feb 2011 16:59:40 -0800

Agreed.

I would much prefer to see full HTTP compatibility before the handshake
completes with a 101 response.

John.

-- 
>|< Kaazing Corporation >|<
John Fallows | CTO | +1.650.960.8148
444 Castro St, Suite 1100 | Mountain View, CA 94041, USA

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 14 Feb 2011 11:17:00 +1100

On 12 February 2011 10:26,  <[email-masked]> wrote:
> in a way that conforms with HTTP. (which is out of scope of this document).

I have a little concern with saying MUST be handled, as that could be
a little confusing. I don't think we can say that every client MUST
follow redirects, or MUST implement BASIC/DIGEST authentication.
Ignoring a redirect is still HTTP compliant behaviour.

I think we need to consider this in two parts: 1) what the handshake
can carry ; 2) what clients will commonly handle/respect.

For 1) I do not think their should be any restriction in the protocol
document about what the server can send back.  Any valid HTTP response
should be allowable.

For 2) I personally think that browsers at least should follow
redirects and support some common forms of authentication, but I
accept that is probably more an issue for the HTML5 community to
discuss rather than this WG.  But I certainly don't want to be telling
other non browser clients that they must follow redirects.

cheers

---

From: [email-masked]
Date: Mon, 14 Feb 2011 11:44:02 -0800

Hi,

> ...

Please let me take back the above comments.

What HTTP offers with Upgrade header and 101 response is not just
for switching underlying protocol but also including HTTP layaer 
itself. Therefore, what I was pointing out wan inherent in HTTP 
protocol, not in WebSocket. Under WebSocket's requirements, 
REQ.8 and REQ.9, I agree that the current approach with Upgrade
header/101 is the best possible way for WebSocket to reuse 
existing HTTP features.

- Yutaka

[email-masked] wrote on 02/11/2011 03:26:02 PM:

> > "Any status code other than 101 must be treated as a failure and 
> the websocket connection aborted." 
> 
> I agree, this needs to be fixed. In fact, any status code other than
> 101 MUST be handled 
document). 
> >    - Existing HTTP implementation is likely designed to treat 101 
> as a indefinitive 
> >      acceptance of the request wihere it tries to prepare for the 
> final response... 
> >      would prepare for releasing HTTP protocol specific resources 
> in memory to terminate 
> >      the transaction. 
> > 
servers.
> >    ## It is desired a loose coupling with HTTP specific function -
> easy to add WebSocket to existing system (e.g. Apache) 
> 
> Say, we try to implement WebSocket in Apache, or via Apache module. 
> Since actual WebSocket 
> connection resides in the middle of the HTTP GET transaction, the 
> process or thread that is 
> processing the transaction has taken up its sufficiently large 
> resources for handling any type of 
> HTTP transaction, and most of which will no longer be necessary once
> the WS handshake is 
> handshake completes on receipt of 200 OK, for example), then this 
> will have a completely 
> different view. Coming back to the example with Apache, the 
> resources for the transaction 
> (process/thread/mempool/context, etc.) can be released completely. 
> What needs to done 
> is in its socket management code be modified to leave the socket (at
> this point being 
> 'websocket' open and transfer it to an internal or external facility
> that can efficiently handle 
> More specifically, the Connection header is about what to do with 
> the underlying connection 
> *after* the current transaction is complete. So, we only needed to 
> define a new value to 
> (# of course, 'Sec-Websocket-xxx' headers are still necessary, 
> 'Upgrade: websocket' can be eliminated.) 
> 
> This can be reported to the socket management layer to tell, "the 
> socket is going to be used 
> This is clearly MUCH less work than tackling situation where 
> transaction is left open upon 
> 101 response, and yet MUCH cleaner solution for introducing 
> WebSocket feature into 
> existing HTTP servers. 
> 
> 
> Any thoughs? 
> 
> --