Subject: Re: [hybi] [whatwg] HttpOnly cookie for WebSocket?

From: Salvatore Loreto <[email-masked]>
Date: Mon, 01 Feb 2010 12:26:31 +0200

-as individual-

I agree too and I see the need to use HttpOnly.
I was just trying to highlight that at the end WebSocket does not only 
share port with HTTP/Web Server,
it also supposed to share other information (i.e. in this case cookies!!!).
The spec implicitly assume that the WebSocket server can access those 
information that the WebServer posses.

On 01/29/2010 08:22 AM, Ian Fette (????????) wrote:
> 2010/1/28 Salvatore Loreto <[email-masked] 
> <mailto:[email-masked]>>
>
>     Hi,
>
>     thanks to have moved back to the original question.
>     My original mail my was a call to discuss the technical aspects
>     openly in the mailing list, so to let the spec move forward.
>
>
>     I agree that it would be useful to use cookie on WebSocket,
>     however I have some perhaps stupid doubts and questions on its
>     usage that I'd like to be clarified
>
>     1) Is the usage of cookies optional or it is mandatory?
>     what will happen in the few cases where the WebSocket will be
>     established without the user has already logged into a page?
>
>
> I assume this would be left up to the application. If the server is 
> expecting some sort of authentication cookie and doesn't get one, it 
> can attempt to do authentication in an application specific manner 
> over the websocket connection, it can close the websocket connection, 
> it can transmit some applicaiton-specific error message to the client 
> over the websocket connection, etc.

I'd like the spec be more detailed about the correct server behaviour in 
this situation.

regards
Sal

On 01/30/2010 04:28 AM, Maciej Stachowiak wrote:
>
> On Jan 29, 2010, at 6:13 AM, Greg Wilkins wrote:
>
>> Ian Fette (????????) wrote:
>>
>>>
>>> cookies are already sent with WS, the only question is whether that
>>> includes or excludes cookies that are HttpOnly
>>
>> The upgrade request is a HTTP requests (well at least it should be
>> a HTTP request, and not just something that strongly resembles one),
>> so I believe HttpOnly cookies should be included.
>>
>> This would not expose the cookie and it's value to the
>> javascript in browser, nor can I think of any way that this reduces
>> the security provided by HttpOnly.
>
> I agree. The purpose of HttpOnly is to prevent the cookie from being 
> seen by scripting APIs, not to limit the network protocols over which 
> it is provided. Thus, sending it over WebSocket connections would be 
> in line with its purpose, and I think this is the case whether or not 
> we think the WebSocket upgrade request is or is not HTTP.
>
> Regards,
> Maciej
>

---

From: Ian Hickson <[email-masked]>
Date: Mon, 1 Feb 2010 11:25:45 +0000 (UTC)

On Fri, 29 Jan 2010, Julian Reschke wrote:
> Ian Hickson wrote:
> > 
> > The same way it works with the HTML5 specification and the various Web 
> > Apps specifications -- feedback is collected from both groups (and 
> > indeed, anywhere else that feedback is provided, e.g. on blogs or 
> > forums), and changes are made that take into account all the feedback. 
> > The most active members of both groups stay in regular contact, e.g. 
> > on IRC, or by e-mail, to ensure that everyone is on the same page. 
> > Where editorial differences arise (e.g. the IETF prefers text/plain 
> > specs, the WHATWG prefers HTML- based specs), the groups ensure that 
> > normative requirement remain identical across different versions. 
> > Basically, exactly as has been happening for the past few years 
> > already. ...
> 
> Feedback that affects the contents of a WG deliverable should be 
> submitted as "IETF Contribution", as described in 
> <http://www.ietf.org/about/note-well.html>.

I'm not going to ignore feedback that is sent outside the context of this 
working group.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Maciej Stachowiak <[email-masked]>
Date: Mon, 01 Feb 2010 03:38:28 -0800

> I'm not going to ignore feedback that is sent outside the context of this 

I don't think the rules that Julian linked require that. The RFCs that apply to IETF contributions per that page seem to be about requirements relating to intellectual property rights. Specifically, grant of non-exclusive copyright license to the IETF Trust, and patent disclosure obligations. I am not sure of their relevance to this thread. 

Regards,
Maciej

---

From: Julian Reschke <[email-masked]>
Date: Mon, 01 Feb 2010 13:18:31 +0100

Ian Hickson wrote:
> ...

I'm encouraging you to convince those contributors to give their 
feedback on the IETF mailing list.

BR, Julian

---

From: Ian Hickson <[email-masked]>
Date: Mon, 1 Feb 2010 12:58:05 +0000 (UTC)

On Fri, 29 Jan 2010, Francis Brosnan Blazquez wrote:
>
> Maybe we have missed something, so I ask again:
> 
> 1) It is possible to send binary content (that includes octets 0x00 and 
> 0xFF) from a browser javascript?

Not currently, but this will be possible in due course. Right now we're 
waiting for TC39 to add binary support to JS.

The protocol is written such that it will be able to support it, however.

> 2) Will be Websocket protocol be able to use existing HTTP proxies?

Yes, see step 2 of the client handshake:

   http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol#section-4

HTH,

---

From: Ian Hickson <[email-masked]>
Date: Mon, 1 Feb 2010 13:18:22 +0000 (UTC)

On Fri, 29 Jan 2010, SM wrote:
> > 
> > Does it not seem odd that the people who discussed where the spec 
> > should be edited did not include the person editing the spec?
> 
> Yes, that would look odd.  You took part of the discussion ( 
> http://www.ietf.org/mail-archive/web/hybi/current/msg00753.html )

Yes, I did. Yet a "consensus" was reached which I didn't agree with, and 
my feedback to that effect was ignored. (In the very e-mail you quote, 
from last October, I mentioned that the WHATWG should be involved.)

> > (I did not at any point see agreement on this list that the HyBi group 
> > should take over the WHATWG work without working with the WHATWG. I've 
> > read every e-mail sent to this list since it was created.)
> 
> The content of the message at 
> http://www.ietf.org/mail-archive/web/hybi/current/msg00765.html may be 
> relevant.

I replied later in that thread:

   http://www.ietf.org/mail-archive/web/hybi/current/msg00785.html

...and received no further reply. Note that I already indicated my concern 
over this "primary responsibility" business in that thread. However, the 
discussion in that thread, which included a suggestion from the chairs of 
text mentioning the WHATWG to be added to the charter, apparently had no 
effect on the charter.

I sent similar comments again (to the IESG) when the charter was finally 
proposed. That was ignored also.

> By the way, some browser vendors supported the creation of this Working 
> Group ( see 
> http://www.ietf.org/mail-archive/web/hybi/current/msg00604.html and 
> http://www.ietf.org/mail-archive/web/hybi/current/msg00647.html ).
>
> Could the WHATWG inform this Working Group about the name of the 
> person(s) speaking on behalf of the group?

For administrative matters, I currently do.

For technical matters, the WHATWG has over a thousand subscribers and they 
speak for themselves on the WHATWG mailing list.

---

From: Salvatore Loreto <[email-masked]>
Date: Mon, 01 Feb 2010 21:33:49 +0200

just to make order in this particular thread discussion

as I understood, two different problems have been arised

1) "safely" shutdown a websocket connection.
      The possibility to lose data while closing a TCP connection is a 
well-known problem,
      as has also been discussed and described in the thread.

      Some people think there is a value to add to the spec gracefully 
shutdown the websocket connection.

      However I haven't seen a clear consensus on it.

      So please if you have an opinion on this, speak up!!!

2) what happen if/when the connection is lost; this can happen for 
several different reasons:
    e.g. a NAT restarting, the mobile terminal go out of network 
coverage etc.

     here we have different sub problems in my opinion:

     2.1) how detect as fast as possible that the connection has been lost

     2.2.) what to do after have reconnected.

     Are those something people think is important to spend cycles on??

cheers
/Sal

On 01/31/2010 01:52 AM, Justin Erenkrantz wrote:
> On Sat, Jan 30, 2010 at 6:49 AM, Jamie Lokier<[email-masked]>  wrote:
>    
>> Orderly close does not help with network-level disconnects, so other
>> techniques like duplicate elimination are valuable too.
>>      
> True, but I think it's important that we - as a group - crawl before we run.
>
> I'd like to see consensus that orderly close is important and should
> be added and how we go about doing so in a sane way.
>
> If we can't even get buy-in for that, then other more advanced
> optimization techniques are likely to go nowhere as well.  -- justin
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 02 Feb 2010 07:18:10 +1100

>      So please if you have an opinion on this, speak up!!!

I've spoken enough in support of orderly close... so I 'll let others
speak up here.

>    e.g. a NAT restarting, the mobile terminal go out of network coverage
> etc.
> 

Yes.

I think that it is very important for an endpoint and/or application to be
able to distinguish between:

   a) orderly close in response to an application request, in which case
      the app probably should not attempt to re-establish.

   b) close in response to an error (message too large or similar),
      in which case the app/endpoint should refrain from any
      retries it might otherwise have attempted.

   c) close in response to an idle timeout.
      in which case the endpoint should not recreate the connection.
      But an application might, if presence is required.

   d) unexpected close due to some failure (or undisclosed intermediary
      timeout).     In this case an application may wish to retransmit
      messages (either all recent idempotent messages or un-acked
      non-idempotent messages).

   e) inability to open a connection due to no connectivity.  In this
      case an app might do a retry at regular intervals

   f) inability to open a connection due to permission denial from the
      server.  In this case, the app probably wants to alert the user
      of the error.

I think that the endpoint code will be able to do very little
of the handling of reconnections and retries, simply because
it does not know enough about the messages.   So the application
will have to decide about reconnects and retransmits, but it
currently does not have the information needed to make those
decisions.

I think fail fast is also desirable, but I'm not sure it is so
desirable that I'd advocate a regular keep-alive message just
failure detection.  But if keep-alive messages are required to
keep open connections through transparent proxies, then it could
be 2 birds with 1 stone.

cheers

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 01 Feb 2010 23:25:49 +0100

>       So please if you have an opinion on this, speak up!!!

We discussed this on IRC earlier today. The idea is to introduce a frame  
for closing. If A wants to close the connection it transmits 0xFF 0x00.  
Once the B receives that and has no more messages to transmit it transmits  
0xFF 0x00 as well. When A receives that it knows it can close the  
connection.

This would not be required of servers however as there are scenarios (e.g.  
news tickers) where it does not really matter whether everything has been  
received by the client.

-- 
Anne van Kesteren
http://annevankesteren.nl/

---

From: Maciej Stachowiak <[email-masked]>
Date: Mon, 01 Feb 2010 14:33:49 -0800

On Feb 1, 2010, at 2:25 PM, Anne van Kesteren wrote:

> On Mon, 01 Feb 2010 20:33:49 +0100, Salvatore Loreto <[email-masked]> wrote:
>>      The possibility to lose data while closing a TCP connection is a well-known problem,
>>      Some people think there is a value to add to the spec gracefully shutdown the websocket connection.
>> 
> We discussed this on IRC earlier today. The idea is to introduce a frame for closing. If A wants to close the connection it transmits 0xFF 0x00. Once the B receives that and has no more messages to transmit it transmits 0xFF 0x00 as well. When A receives that it knows it can close the connection.
> 
> This would not be required of servers however as there are scenarios (e.g. news tickers) where it does not really matter whether everything has been received by the client.

I think it's simpler to just always do the close handshake than to have an exception to opt in to random data loss in the rare cases where it is acceptable. Certainly general-purpose frameworks for using WebSocket should always do this.

I also wonder if messages both ways are really needed for a close.

Here is an alternative:

If A wants to close the connection it does the following:

1) transmits 0xFF 0x00 to B.
2) Calls shutdown(SHUT_WR) on its socket (instead of close)
3) Reads anything else transmitted from B until it gets EOF, at which point it calls close().

When B receives 0xFF 0x00, it sends any remaining buffered messages and calls close.

People who understand the TCP issue: would that be sufficient to avoid the TCP reset hazard? From the description I saw, I believe it would, since neither endpoint would send data after the other has closed its connection. If it does work, then it is on less round trip than the close handshake Anne described.

Regards,
Maciej

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 02 Feb 2010 09:58:46 +1100

Anne van Kesteren wrote:
> Once the B receives that and has no more messages to transmit it
> transmits 0xFF 0x00 as well. When A receives that it knows it can close
> the connection.

So I think talking implementation before we have requirements is
premature.  While I'm sure that a solution might look something like
that, there are other use-cases to be considered that might need
a bit more content in the frame.

Intermediaries should be able to initiate and orderly close,
so they would need to send another message to either/both
endpoints, which would then initiate the normal orderly close.

I think there is also a need for an error close.  Currently all errors
are handled by just closing the connection, but if that looks like
a network failure, then a client might retry.  If the error is due
to a message being too large, or too many connection or some other
server resource limitation, then the last thing we need is the
client doing a retry.

> This would not be required of servers however as there are scenarios
> (e.g. news tickers) where it does not really matter whether everything
> has been received by the client.

I think it would still be good for the newsfeed client to know if
the connection was severed normally (ie there is no more news today)
or if it just failed for some other reason (and thus it might
retry to connect).

regards

---

From: "Anne van Kesteren" <[email-masked]>
Date: Tue, 02 Feb 2010 00:06:03 +0100

On Mon, 01 Feb 2010 23:58:46 +0100, Greg Wilkins <[email-masked]> wrote:
> [...]

Why does all that need to be part of the base protocol? It seems all that  
can be handled by the sub protocols, if deemed necessary.

---

From: Ian Hickson <[email-masked]>
Date: Mon, 1 Feb 2010 23:48:37 +0000 (UTC)

On Fri, 29 Jan 2010, Justin Erenkrantz wrote:
> 
> Again, this is why the current draft is so impenetrable (to me) since it 
> expects that the only person implementing the draft is a client vendor 
> using synchronous socket methods...which sort of defeats the purpose 
> when you are trying to write a fully async client...or any type of 
> server.

Could you elaborate on this? I looked for text that assumed synchronous 
socket methods but couldn't find any. Could you quote the offending text?

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 02 Feb 2010 10:58:54 +1100

Anne van Kesteren wrote:
> Why does all that need to be part of the base protocol? It seems all
> that can be handled by the sub protocols, if deemed necessary.

Well this is something that needs to be discussed during the
requirements phase.   What are the requirements for a base
protocol.

But I would think that reasonable error handling should be part
of the base protocol.  It's like suggesting that 400 Bad Request
be taken out of HTTP.

Note that I'm not advocating that the base protocol actually
do  any reopening of connection or retrying of messages. Simply
that it be capable  of passing the information on which such
things could be implemented as part of a layered solution.

regards

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 2 Feb 2010 08:00:22 +0800

> When B receives 0xFF 0x00, it sends any remaining buffered messages and

Anne's described solution works better.

The 0xff00 frame is the indication that close() can be called safely.  Besides, it's two extra octets, hardly an imposition on either end.

---

From: Ian Hickson <[email-masked]>
Date: Tue, 2 Feb 2010 00:00:34 +0000 (UTC)

On Mon, 1 Feb 2010, Maciej Stachowiak wrote:
> 
> I think it's simpler to just always do the close handshake than to have 
> an exception to opt in to random data loss in the rare cases where it is 
> acceptable.

On the client side, I agree in principle (at least to ensure that no 
client-to-server data is lost in a case where the API is used to send 
frames then immediately close the connection). However, what if the server 
never closes its side of the connection, even after the client initiates a 
close? Should we have some sort of timeout?

On the server side, I don't think there's any point _requiring_ that the 
server close gracefully. If the server really doesn't care if data it just 
sent is received, e.g. because it knows the client side has already GC'ed 
its WebSocket object, then it should just be able to close abruptly.

> People who understand the TCP issue: would that be sufficient to avoid 
> the TCP reset hazard? From the description I saw, I believe it would, 
> since neither endpoint would send data after the other has closed its 
> connection. If it does work, then it is on less round trip than the 
> close handshake Anne described.

If it works, I can certainly add this to the spec. Does it?

What's the right way of saying "call shutdown(SHUT_WR)" vs "call close()" 
in the spec text? Is there some TCP language that those map to that I can 
use? Also, if shutdown() and close() are mentioned, I should probably also 
mention the Win32 versions of those APIs; does anyone know what they are?

---

From: "Anne van Kesteren" <[email-masked]>
Date: Tue, 02 Feb 2010 01:08:27 +0100

> of the base protocol.  It's like suggesting that 400 Bad Request

Well yes, but WebSockets is not meant to be like HTTP. The idea is to just  
have a simple layer on top of TCP that makes it suitable for deployment in  
Web browsers. I would expect something like SPDY to provide the (far) more  
complex variant.

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 02 Feb 2010 11:20:26 +1100

Anne van Kesteren wrote:
> Well yes, but WebSockets is not meant to be like HTTP. The idea is to
> just have a simple layer on top of TCP that makes it suitable for
> deployment in Web browsers. I would expect something like SPDY to
> provide the (far) more complex variant.

TCP does have  difference between an orderly shutdown (FIN),
error shutdown (RST) and timeouts.

So if you want to be like TCP, then you'll have those
concepts.

Currently websocket is more like webserialport than webtcp

regards

---

From: Maciej Stachowiak <[email-masked]>
Date: Mon, 01 Feb 2010 16:30:28 -0800

> Anne's described solution works better.

How is it better? Can my solution lead to data loss?

> The 0xff00 frame is the indication that close() can be called safely.  Besides, it's two extra octets, hardly an imposition on either end.

It's not the octets that are wasteful, it's the extra round trip. The only material difference in my protocol is that the party responding to, rather than initiating the close doesn't have to send a special message, it just counts on the close initiator getting an EOF. Since the initiator has to stop sending after the close request, there's no risk of data loss in the other direction either (assuming I understand the TCP issue correctly.)

Regards,
Maciej

---

From: Maciej Stachowiak <[email-masked]>
Date: Mon, 01 Feb 2010 16:32:18 -0800

> its WebSocket object, then it should just be able to close abruptly.

How can the server possibly know that the client has already GC'd its WebSocket object, if the server and not the client is the initiator of the close? (If the client initiates the close, then at least in my proposal there's no extra work for the server.)

Regards,
Maciej

---

From: SM <[email-masked]>
Date: Mon, 01 Feb 2010 16:39:43 -0800

>speak for themselves on the WHATWG mailing list.

People on this list can object to the technical changes made on that 
mailing list and  vice-versa.  Some of the objections on this mailing 
list might even be non-technical.  Overlooking such details is not 
recommended as it can end in a messy situation.

Regards,
-sm

---

From: Ian Hickson <[email-masked]>
Date: Tue, 2 Feb 2010 00:54:51 +0000 (UTC)

> recommended as it can end in a messy situation.

Indeed.

---

From: Ian Hickson <[email-masked]>
Date: Tue, 2 Feb 2010 00:58:14 +0000 (UTC)

On Mon, 1 Feb 2010, Maciej Stachowiak wrote:
> > frames then immediately close the connection). However, what if the 
> > server never closes its side of the connection, even after the client 
> > initiates a close? Should we have some sort of timeout?
> > 
> > On the server side, I don't think there's any point _requiring_ that 
> > the server close gracefully. If the server really doesn't care if data 
> > it just sent is received, e.g. because it knows the client side has 
> > already GC'ed its WebSocket object, then it should just be able to 
> > close abruptly.
> 
> How can the server possibly know that the client has already GC'd its 
> WebSocket object, if the server and not the client is the initiator of 
> the close? (If the client initiates the close, then at least in my 
> proposal there's no extra work for the server.)

The server and the client are typically written by the same person. If the 
client sends a message in the subprotocol saying "ok after this I'm 
throwing away all references to the object", then the server can know that 
there's no point sending further data.

I'm not saying this is good practice, I'm just saying that there's no 
point making such practice non-conforming.

---

From: Justin Erenkrantz <[email-masked]>
Date: Mon, 1 Feb 2010 16:59:42 -0800

> socket methods but couldn't find any. Could you quote the offending text?

All of them are...such as:

---
          Run these steps.  If at any point during these steps a read is
          attempted but fails because the Web Socket connection is
          closed, then abort.

          1.  Let /length/ be zero.

          2.  _Length_: Read a byte, let /b/ be that byte.

          3.  Let /b_v/ be integer corresponding to the low 7 bits of
              /b/ (the value you would get by _and_ing /b/ with 0x7F).

          4.  Multiply /length/ by 128, add /b_v/ to that result, and
              store the final result in /length/.

          5.  If the high-order bit of /b/ is set (i.e. if /b/ _and_ed
              with 0x80 returns 0x80), then return to the step above
              labeled _length_.

          6.  Read /length/ bytes.

          7.  Discard the read bytes.
---

Greg pointed out issues as to what "send" or "receive" means -
buffering, etc, etc.

The algorithmic descriptions can only be precisely followed when you
write a synchronous client (or server) - the algorithms for an network
async read or write framework would appear hugely different.  Again, I
think this challenges the usefulness of specifying the protocol in
pseudo-code when many (optimized?) implementations won't be able to
follow those same algorithms.  -- justin

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 2 Feb 2010 01:25:34 +0000

Thomson, Martin wrote:
> The 0xff00 frame is the indication that close() can be called
> safely.  Besides, it's two extra octets, hardly an imposition on
> either end.

The major cost on most networks is packet counts and round trips, not
bytes.  (Bytes of large messages, yes, but that is something else.)

Anne's solution adds an extra packet, because Martin's can combine the
first data and TCP FIN into a single packet, although it depends on
TCP implementation as well as the TCP_NODELAY (Nagle) setting at that
moment, and whether there is already other data buffered to send.

The minimal version does not need a frame at all:

  1) Calls shutdown(SHUT_WR).
  2) Reads anything else transmitted from B until it gets EOF, at which
     point it calls close().

Meanwhile, B receives an EOF and calls close() immediately.

That is fine for TCP; the 0xff 0x00 is redundant.

But we must also pay attention to some important reality: There will
be implementations which run over pipes (CGI), internal APIs and so on
which don't support the half-closed semantic.  There will be relays
from TCP WebSocket directly to those other forms.  It does nobody any
favours to define a protocol which only works over TCP and forces
those implementations to have to invent something extra and shoehorn
it in.

Note, there are some ancient TCP implementations which don't support
the shutdown() call.  That might be why it's not mentioned in HTTP.

My vote would for an endpoint to be able to send *either* a close
frame 0xff 0x00, or EOF as a result of shutdown(SHUT_WR), or both, as
it chooses.  Receivers would treat EOF and close-frame the same.

(Though it should be considered how that would affect a simple
WebSocket to CGI relay - the relay must parse every byte because of
the sentinel-delimited frames, to inject/strip 0xff 0x00 where
needed without doing so in the middle of a truncated message.)

Please note: The close-frame is *not* for acknowledgement of prior
messages.  It is a mechanism to prevent closes from turning into
"connection reset" TCP errors and the data-discarding behaviour.  It
would be *most unwise* of an endpoint to refuse to send the
close-frame (or shutdown) because it was keeping it to acknowledge an
expected message - because that would force the other end to keep a
connection open for a long timeout, when receiving close and lack of
an expected message is precisely a situation when connections should
be closed quickly because an unrecoverable error has occured.

-- Jamie

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 2 Feb 2010 10:42:28 +0800

> That is fine for TCP; the 0xff 0x00 is redundant.

What I get from this is that TCP connections haemorrhaging data can be solved without support in the higher layer - all that it requires is that the implementation of the next layer be aware of the limitations of TCP and that it uses shutdown(...) or whatever is necessary.

That's a pretty strong argument against putting specific close semantics in WS.  I don't buy the "let's make our protocol transport-layer-agnostic argument"; that only leads to the protocol picking up unnecessary baggage. 

If an explicit close notification is required for an alternative transport, then let those who are porting the protocol to that environment add a shim for that purpose.

Having text that explains the assumptions of the current solution and maybe even suggests an implementation option (preferably by reference) seems most appropriate to me.

--Martin

> be closed quickly because an unrecoverable error has occured.

This too is useful information - but this could be applied to the FIN, rather than the close frame.  That is, if you receive a FIN, that doesn't mean that the other peer received everything that you've already sent.  When you put it like that, it doesn't seem like much of a stretch ;)

---

From: Maciej Stachowiak <[email-masked]>
Date: Mon, 01 Feb 2010 19:56:06 -0800

> That's a pretty strong argument against putting specific close semantics in WS.  I don't buy the "let's make our protocol transport-layer-agnostic argument"; that only leads to the protocol picking…

I'm not overly concerned about being transport-layer-agnostic on this feature - after all, other transports may not need the special close semantics. For example, SCTP likely does not. But I am concerned about whether a close handshake using only half-closes is as widely implementable as we would like. Two concerns here: (1) it's been reported that some TCP stacks do not actually expose the ability to do a half-close - it would be a shame if you could never write a WebSocket client or server for such an OS; (2) When using SSL as the transport, does the same issue arise, and can SSL implementations be convinced to do a half-close on demand?

If either of these concerns makes half-close-only impractical, then I think Jamie's suggestion is the best: allow any of half-close, explicit close frame, or both. 

 - Maciej

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 02 Feb 2010 20:18:33 +0900

Hello Maciej,

> I don't think the rules that Julian linked require that. The RFCs that apply
> to IETF contributions per that page seem to be about requirements relating
> to intellectual property rights. Specifically, grant of non-exclusive copyright
> license to the IETF Trust, and patent disclosure obligations. I am not sure
> of their relevance to this thread.

Well, I am not a lawyer, and the simple summary here may ignore all 
kinds of details, but essentially a lot of communities (standards 
organizations as well as open source communities) and the people and 
companies involved therein have, over the years, made painful 
experiences with the following scenario:

Person A: "wouldn't it be great if you spec did X."

Editor, WG: "not a bad idea, we'll take that"

(some years later)

Person A: "Hello everybody, I have a patent on X, please start paying."

Patent disclosure obligations are designed to avoid the above scenario. 
People participating in the IETF (and the organizations supporting them) 
appreciate the fact that the IETF has patent disclosure obligations.

People participating in the IETF may have some concerns about "taking 
all input available", not because they don't think that wouldn't be a 
good idea in general, in a world where everybody was nice and there were 
no patents, but because one way or another, they know that this can be 
dangerous.

That's one of the areas where larger and more experienced standards 
organizations (such as the IETF) may have an advantage over smaller and 
more ad-hoc organizations (such as WHATWG). If somebody can convincingly 
show us that WHATWG deals with this problem in a fashion equivalent with 
the IETF, then I think there will be much less concern here on this list.

So I think at least in the sense explained here, what Julian brought up 
here is very relevant for the process thread.

[Copyright is much easier to deal with, because it essentially would go 
like this:

Person B: "Here is some proposal Y for your spec."

Editor, WG: "Great text, we'll just paste it in."

(later)

Person B: "Why the hell are you taking my text? It's me who wrote it, 
it's my copyright, and I don't like it sitting there after that other 
paragrap."

Editor: "Sorry, didn't know. Too bad, have to rewrite these few 
paragraphs so that they say the same thing with different words."
(way cheaper than changing implementations to get around a patent or 
paying royalities)]

Regards,    Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: Justin Erenkrantz <[email-masked]>
Date: Tue, 2 Feb 2010 09:05:14 -0800

> If either of these concerns makes half-close-only impractical, then I think Jamie's suggestion is the best: allow any of half-close, explicit close frame, or both.

Given how hard it is to get useful (or consistent) TCP error
information from most OS stacks, I think having an explicit close
frame in the common case is going to be well worth it in the long run.

My $.02.  -- justin

---

From: Francis Brosnan Blazquez <[email-masked]>
Date: Tue, 02 Feb 2010 19:25:55 +0100

Hi Greg,

> So if you want to be like TCP, then you'll have those

Why not?

I think people is interested on Websocket because they will layer its
favourite protocol on top of it. I not stating I'm against this but I
still don't see this point and how it would improve the protocol...

In other words, people that cares about orderly close will use a
protocol on top of Websocket that provides graceful close (like BEEP)
and people that do not cares about this will have an additional feature
not requested....so it looks to me both kind of users won't use this. 
-- 
Francis Brosnan Blazquez <[email-masked]>
ASPL

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 03 Feb 2010 09:35:19 +1100

Francis,

I'm all for protocol layering and if features can be
implemented in a subprotocol, then they probably should be.

However, we don't want to force subprotocols into having to
re-implement TCP with timeouts, keep-alives, acks etc.
if a few simple features in the base protocol can expose
more of TCP/IP's capabilities to the layered protocol.

In the case of orderly close, a layered protocol
could implement its own orderly close.  But it would
still have a problem with error closes, as the
servers response to all sorts of issues is simply
to close the socket.

If my layered protocol does:

  websocket.send(moderatelyLargeMessage)

and then receives a onclose callback before any
acks or orderly close are received, then the layered
protocol knows there has been a failure, but does not
know the cause.  So it will probably open another
connection and send the moderatelyLargeMessage again....
and again... and again....

If it new that the message was being rejected by
the other endpoint rather than a network failure,
then it would know not to retry.

The same problem goes if the user opens too many
tabs and exceeds some connection limit on the
server. The sub protocol will be unable to tell
the difference between that and network failure.

TCP is a reliable transport, but if a layered protocol
is unable to see the difference between an orderly close,
and error close and a network failure, then it cannot use
TCP reliably.

If I do a websocket.send(message) and then a short
time later I get a websocket.onclose() call back, then
I have no idea if that message was delivered.

So websockets is obscuring the reliable capabilities
of TCP.     As I've said before, it's more like
webserialport than websocket and subprotocols will
need to be more complex as a result.

regards

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 2 Feb 2010 22:49:49 +0000

> server resource limitation, then the last thing we need is the

Yesterday I decided to analyse Ian's question "does it work", and
found that in some cases, the above method doesn't quite.  It depends
on what, exactly, the orderly close is meant to ensure.

Interestingly, the cases I found are different from Greg's list above.

So I agree with Greg there.  Let's agree the requirement for orderly
close (it seems like maybe we have, just not formally), and we'll look
at those implementation cases.

I won't object if Ian wants to add something preliminary to the draft,
so long as it's possible to add corrections after.

I hope to post my analysis in a day or two.

> or if it just failed for some other reason (and thus it might

It's _very_ useful to know when it's safe to retry things.  Apart from
avoiding the TCP reset hazard causing truncated delivery, this is the
other useful reason for orderly close.

(E.g. see HTTP persistent connections, especially the reason POST needs
a new connection every time to be safe.  E.g.
see http://www2.tw.squid-cache.org/mail-archive/squid-dev/199812/0036.html
If HTTP had orderly close, POST could be used on persistent and even
pipelined connections.)

When it's not known, and a request is unsafe to repeat, generally you
have to report an error to the user, or go wrong.

Ideally, a protocol ensures that uncertainty can only occur on real
and unexpected transport errors (and then uncertainty is theoretically
unavoidable anyway).

The following is a good basis for apps to layer higher level protocols
on (like automatic safe retries):

    In the absence of errors (network, application and OS), ensure the
    receiving side knows that a close was intentional from the sending
    app (e.g. not crashes, not TCP proxy shutdowns), and ensure data
    sent before the close is transmitted reliably.

Let's make sure this part of WebSocket Just Works(tm).  It's got a bit
too much subtlety, and requires a bit too much low-level knowledge, to
push the responsibility onto each application designer.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 2 Feb 2010 23:26:18 +0000

> > error shutdown (RST) and timeouts.

Btw, TCP is missing one: Distinguish between application crash and
application orderly shutdown.  Both result in FIN.

And TCP is actively unhelpful with it's TCP RST data-discard hazard.

> still don't see this point and how it would improve the protocol...

Think about what happens when:

   - Assume an app which is all request-response, originated by the client.
   - Assume some requests are state changing ("INSERT INTO...").

   - client opens a WebSocket
   - sends a few requests to the server, server responds
   - after 30 seconds, the server decides to close the connection
     because it's not being used(*)
   - at around the same time, the client sends another request
   - client receives TCP RST from the server, as "connection reset" error

(*) Note that servers which don't send or expect keepalive probes
    *must* timeout connections like that, because it's impossible to
    tell the difference between an idle connection and one which
    has gone away due to the network.

Two problems happen:

   1. What does the client do with that last request?  It's not safe to open
      a new connection and repeat it, because it might be acted on twice.
   2. What happens to the response to the second to last request?
      It gets truncated, because it was unlucky enough that the last
      packets were delayed by >30 seconds.

Now, I know how to use WebSocket to solve both of those, and so do you.

Anyone serious about their WebSocket app will handle it.  We don't
need to worry about them.

But seriously, do you think the millions of authors who write little
web applications in a few pages of Javascript and a few pages of Python
will think of that, or even care?  It almost never happens, and it is
much rarer on the LAN where they are testing.  It'll be one of those
rare glitches that are left broken, like random double-posting to a blog.

Mostly it only causes problems for the unlucky schmuck on some obscure
network in a remote location on a 56k modem - or an over-subscribed
mobile network; inevitably the application author doesn't experience
the problem, and it settles on the bugzilla (if there is one) as
"CANNOT REPRODUCE".

And, getting it right does demand a little more mystical TCP knowledge
than most web app authors will ever care about.  Heck, half the people
I know who write TCP apps don't realise write boundaries aren't
preserved (because their app works fine on Windows over a LAN except
for those pesky rare unexplainable lockups...) - and those are people
writing socket apps in C who should know better, not web app authors.

I wouldn't mention _those_ apps and authors, except that it's been
explicitly spelled out elsewhere that they are a target audience for
using WebSocket directly.

> not requested....so it looks to me both kind of users won't use this. 

Orderly close is a bit subtle, and the failure modes when it's done
wrong don't occur for everyone because they are quite dependent on
network configuration.

It's not _complicated_, it just involves non-obvious issues - the TCP
reset hazard being totally non-obvious.

I agree that orderly close can be overlaid on top of WebSocket
(with a small overhead).

But do we really want to encourage fragile applications, because many
of them will be written by people who don't think about this sort of
thing every day like us protocol designers do?

I think that's what Greg's getting at with deciding the base requirements.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Feb 2010 00:54:23 +0000

> frame in the common case is going to be well worth it in the long run.

I agree.  I didn't think of it until a really long hard think.

And then I realised that an explicit close message(*) is essential if
you want to know that the application cleanly closed the socket, versus:

    - Application crash looks the same as shutdown() to the receiver.

    - OS killing the application due to, e.g., memory pressure looks
      the same as shutdown() to the receiver.

    - Intermediate TCP relay (there are many types; see my other post)
      killing the connection looks the same as shutdown() to the receiver.

    - Virtual machine containing the application is hard-rebooted, looks
      the same as shutdown() to the receiver with some VM types.

    - HTTP proxy forwarding over CONNECT, killing the connection looks
      the same as shutdown() to the receiver.

In a nutshell, there are *two* distinct uses for signalling clean shutdown:

    1. To avoid the TCP reset hazard.

    2. To signal that it was a clean shutdown, so there is no protocol
       uncertainty about what has been processed / what can be retried
       on another connection.(**)(***)

shutdown() and lingering close does deal with 1, but it isn't reliable
for 2 because of all the other things which can look like shutdown().

So I agree with Justin and others, that an explicit "clean close"
message is more useful and we'll be glad of it in the long run,
compared with just shutdown().

Oh yes, and there is another reason not to rely on shutdown():

    - Not all TCP relays will forward a half-close.  I know
      for a fact that some I've written (long ago ;-) didn't have a
      shutdown() call anywhere.

    - I would not be surprised to find some HTTP proxy over CONNECT
      doesn't forward a half-close reliably, or at all.

Here are some more technical issues we should consider:

    a. Because the sender must have a timeout after sending the close
       message(***), for clean shutdown from *both* sides, it's
       desirable that the receiver finishes the connection quickly,
       rather than writing "all queued messages", which can be any
       amount of data and take any amount of time.  The receiver
       should detect the close message as soon as possible, and finish
       sending its current message only, or even abort the current
       message (if we used byte 0xfe for that purpose, say).
       Detecting the close message is made quicker by following it
       with FIN (EOF), because processing code will see the EOF before
       it parses incoming queued data.  (OOB would expedite it further
       but I don't see it being popular :-)

    b. It is possible for *both* sides to decide to close around the
       same time, independently.  E.g. if they both implement an idle
       timeout or are driven by resource limits.  I'd appreciate if
       someone double checks this works out ok.

    c. This "lingering close" (of any kind) takes time - up to the
       timeout, which might be 30 seconds, 2 minutes or whatever makes
       sense.  (E.g. see Apache).  What should applications do when
       they want to terminate and are in this state?  E.g. a web
       browser sends a close message, and then the user quits the
       browser.  Does it abruptly close(), or is it important to leave
       a background process handling the lingering close?

    d. Sending a close message followed by shutdown() to half-close.
       Is that *actually* reliable over HTTP proxies when using
       CONNECT?  I wonder if some proxies will close() the next hop,
       thereby breaking orderly close if shutdown() is used.  Perhaps
       Anne's idea to send the close message _without_ using
       shutdown() is a good idea after all.  Does anyone have data?

Footnotes

(*)    Let's not call them frames, to avoid confusion with TCP
       frames/packets :-)

(**)   ACK messages alone cannot do this, because without a clean
       shutdown message, you can't tell if an ACK was lost.

(***)  It also signals "don't bother sending more requests because I
       won't be sending any more responses on this connection".
       Unless you get into the dubious area of reviving half-dead
       connections.

(****) It's related to keepalive.  You can't use a keepalive to avoid
       needing the timeout; that'd just be an equivalent timeout.

And a couple of notes about TCP stacks:

     - Good TCP stacks (e.g. Linux (MSG_MORE/TCP_CORK), FreeBSD
       (TCP_NOPUSH)) can be made to combine the close message and FIN
       in the same packet.  So the close message is essentially free
       on them anyway.

     - Many TCP stacks actually do implement "lingering close" in the
       OS.  See SO_LINGER.  But it is often not correctly implemented
       or not useful, which is why Apache et al must do it themselves.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Feb 2010 00:56:50 +0000

> there's no point sending further data.

Isn't that the same as a close message with the same semantics?

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Feb 2010 01:04:11 +0000

> keep open connections through transparent proxies, then it could

Sad keepalive story.

I have used a 3G mobile network where a keepalive is needed every 30
seconds to keep an SSH connection alive, and I've used some other NAT
where 30 seconds was too long.

Keepalives are particularly unwelcome on mobile devices because of
the battery drain, but there is often no other easy way to reliably
pick up incoming data in a timely fashion.

I guess recommended keepalive values are something so go into the HTTP
long-polling recommendations document too.  We could do with measurements.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Feb 2010 01:12:31 +0000

> follow those same algorithms.  -- justin

To be honest, I don't see a big problem with the draft in this area.

But perhaps I already have a clear idea how I would implement it and
it's not clear to someone doing it from scratch.

You just have to assume there are buffers.  "send" means queuing bytes
in your output buffer (it's up to the implementation when it actually
sends them), and "receive" means reading them from your input buffer
(they will be read in blocks, no doubt).  "is closed" means "if
you read EOF at this point in the stream", not "is closed" :-)

There is no other meaningful way to interpret it.

Maybe it just needs to say the implementation can use buffering, and
everywhere in the document which mentions reading and writing, it can
be reading and writing the buffers.

Looking at the above, it would be quite a bit clearer to talk about
reading the end of the input (EOF, TCP FIN) than "if the connection is
closed", as the latter has several ambiguous meanings, and you don't
actually want to abort parsing when you know the socket is closed, if
you still have bytes to read from your buffer

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 3 Feb 2010 01:35:00 +0000

Greg Wilkins wrote:

>   + Why have two framing techniques when binary is sufficient to carry
>     everything.

I agree.  While I acknowledge the argument that

    print length($text), $text

is an invitation to do the wrong thing in some languages, I think that
there are more ways that 0xff can lead to the wrong thing, because so
many languages pass around "nominal UTF-8" which is not guaranteed to
be UTF-8, making 0xff delimiting unreliable too.

>   + Who controls allocation of the frame type byte?  So far every
>     suggestion of usage for that (eg a bit to indicate that the
>     frame contains meta-data headers) has been rejected.  So are
>     binary users simply to pick their own bytes and hope for no
>     collisions?  Will IANA eventually allocate values?  is 7 bits
>     enough?

There will be no collisions for frame bytes which depend on the
sub-protocol name, as those frame bytes are privately agreed between
client and server.

The only ones needing global agreement are:

   - Whatever agents and proxies need to handle the basic frame
     boundaries, e.g. for choosing when to forward sooner and when to
     buffer indefinitely for TCP performance.

     This is provided by the high bit.  Once widely deployed, this
     must not be changed unless there is already a "mandatory feature"
     negotation rule already in place.

   - Any subprotocol frame bytes which are given generic meanings,
     e.g. as discussed for the close message.

     Even these can be subprotocol negotiation dependent.

>   + Sentinel framing is unsafe.   It relies on the fact that there
>     are no 0 bytes

I made that mistake, before reading the draft properly :-)
It's 0xff.

A 0 byte is *valid* UTF-8, so of course it's ok to send that.

But you're still right:

>     in the utf-8 strings that are passed to it.  Strangely enough,
>     users can't be trusted to always provide valid utf-8 data, so if
>     user data is not validated then sentinel encoding allows frame
>     injection attacks.  After all we have learnt with HTTP, it seams
>     silly to repeat the mistake of a protocol that is exposed to
>     such attacks

If you have several bits of code sharing a connection - even if
it's just by sharing a common Javascript framework on a single page,
then you have security issues from this.

Less obviously, frame injection attacks can occur *even* with a
private connection between one client and one server.  All it takes is
for user data to get in which isn't really UTF-8 (or the author didn't
even try non-ASCII text), and attacks are possible then.

If nowhere else, this should be in the SECURITY CONSIDERATIONS section
of the draft.

Another common place for invalid UTF-8 injection is filenames and text
files that haven't been checked:

   - Imagine a simple WebSocket server which reads a directory,
     on a system where filenames are UTF-8, and sends the names.  But on
     such systems, even though filenames are *nominally* UTF-8, there are
     inevitably some which aren't.

   - Imagine a simple WebSocket server which sends a file, whose
     contents are expected to be UTF-8, but for whatever reason are not.

And then there are URI query arguments, which code may assume is UTF-8
encoded because it always is when used normally on the accompanying
web pages... But maliciously crafted URIs break the assumption.

As a general rule, I've seen that many languages which handle UTF-8 do
not check that all inputs are valid UTF-8, and those which do often
emit a warning rather than preventing it from infecting outputs.  This
is not an area to expect care from authors.

>   + the utf-8 Sentinel framing is inflexible.  It sends only raw
>   utf-8.  What if I want to send gzipped utf-8, or utf-16 etc.  This
>   could simply be handled with a content encoding header in the
>   upgrade request and use of binary framing.

Not so much an issue.  You can just pick a binary frame type in the
existing framework, and use that to carry gzipped frames.

Provided both ends agree, you can just tunnel UTF-8 WebSocket
over binary WebSocket :-)

What it should not is assume it's ok to use frame types < 0x80 to
carry binary frames when accompanied by mutually agreed content
negotation.  (Assuming the current format gets deployed).  Because it
is *inevitable* that WebSocket-aware proxies will parse frame
boundaries at some point - if only to improve buffering decisions.

Aside from frame injection attacks due to "not really UTF-8", the
other reason to prefer length-delimited framing is performance.  I'm
not surprised at Google's measurement.  With length-delimited frames,
you don't have to examine every byte when forwarding it to keep track
of boundaries for buffering decisions.

-- Jamie

---

From: Maciej Stachowiak <[email-masked]>
Date: Tue, 02 Feb 2010 17:35:51 -0800

> there's no point sending further data.

Why would the client do that instead of sending a close message (thus making the client the close initiator)? 

Also, just throwing away all references to the object would not make it eligible for GC. Per the API spec, it is protected from GC if the connection is open and the WebSocket still has event listeners. I would say that explicitly closing the client side should send a close message (thus making the client the close initiator), and removing all listeners and dropping all references without closing should lead to GC which should close the connection (thus again making the client the close initiator).

In either case, the client sending an extra "I'm dropping all references" message is extra work for the client and doesn't actually save work for the server, so what's the point?

> point making such practice non-conforming.

I don't think there is a valid use case for allowing it, and it makes the protocol more complicated to have multiple options.

Regards,
Maciej

---

From: Maciej Stachowiak <[email-masked]>
Date: Tue, 02 Feb 2010 17:45:18 -0800

> for 2 because of all the other things which can look like shutdown().

I think everyone agrees that we need to address #1. I also personally agree with #2.

Also, there's a potential third use:

3. Knowing the other endpoint has definitely received all of your messages sent prior to the close.

I'm not sure, but I think to achieve #3 may require at either a 3-way close handshake, or a close reply message distinct from close initiation. Consider the case both endpoints initiate close at the same time and their messages cross. Then one endpoint may think it is the close initiator, and when it sees the close from the other side, it may think the other side got all of its messages successfully. Another possibility is to make the close response message look different from the close initiation message. Then you can distinguish a close reply from simultaneous close. I haven't yet analyzed whether this is fully sufficient. We may need to explicitly identify the last message received by each side somehow.

Clearly there are subtle issues here, and we'll have to carefully analyze what can go wrong in any close protocol.

Regards,
Maciej

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 03 Feb 2010 15:29:34 +1100

> sub-protocol name, as those frame bytes are privately agreed between

Good point.

But then we do need to ensure that there are no sub-protocol name
clashes.     My suggestion is to use real URLs as sub-protocol
names, like DTDs

Also I think a single sub protocol name is not very flexible.
You might want to combine multiple sub protocols - one for
gzip, one for orderly close etc.   or is that a subprotocol
that supports optional features?

regards

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 03 Feb 2010 15:44:06 +1100

... and I'd just like to repeat that I believe it is important to
have a mechanism where an intermediary can initiate a clean shutdown.

cheers

---

From: Maciej Stachowiak <[email-masked]>
Date: Tue, 02 Feb 2010 20:50:34 -0800

> have a mechanism where an intermediary can initiate a clean shutdown.

In the case of an SSL proxy, there's no obvious way to do that, since an intermediary can't inject anything into the SSL stream. Or at least, we'd have to break through the SSL layer for this, which I expect would be hard to implement. For other intermediaries, I think we would have to design how they can participate at all, before worrying about how they can initiate a close.

Regards,
Maciej

---

From: Mridul Muralidharan <[email-masked]>
Date: Wed, 3 Feb 2010 12:19:36 +0530 (IST)

Detection of lost messages is slightly harder problem than orderly close.
In xmpp space, where we had to deal with it, the base protocol mentions about orderly close - there are extensions which deal with detecting (on reconnect) if server/client received all messages sent & retransmit those : but this also depends on whether the server/client supports it, etc.
Actually detection for need for retransmission for httpbind is different from message level retransmission ... but I am not getting into that here.

Regards,
Mridul

----- Original Message ----
> From: Maciej Stachowiak <[email-masked]>
> To: Jamie Lokier <[email-masked]>
> Cc: Hybi <[email-masked]>; "Thomson, Martin" <[email-masked]>
> Sent: Wed, 3 February, 2010 7:15:18 AM
> Subject: Re: [hybi] Reliable message delivery (was Re: Technical feedback.)
> 
> 
> I'm not sure, but I think to achieve #3 may require at either a 3-way close 
> handshake, or a close reply message distinct from close initiation. Consider the 
> case both endpoints initiate close at the same time and their messages cross. 
> Then one endpoint may think it is the close initiator, and when it sees the 
> close from the other side, it may think the other side got all of its messages 
> successfully. Another possibility is to make the close response message look 
> different from the close initiation message. Then you can distinguish a close 
> reply from simultaneous close. I haven't yet analyzed whether this is fully 
> sufficient. We may need to explicitly identify the last message received by each 
> side somehow.
> 

      Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

---

From: Ian Hickson <[email-masked]>
Date: Wed, 3 Feb 2010 10:24:11 +0000 (UTC)

On Wed, 3 Feb 2010, Graham Klyne wrote:
> Ian Hickson wrote:
> > The server and the client are typically written by the same person.
> 
> If this is the basis on which the protocol is being specified, I don't
> understand why it's being done in a standards body.

So that the platform on top of which the client is written can be 
interoperably implemented such that it is possible for the clients to run 
on any implementation of the platform interoperably.

To clarify the terminology, from the perspective of the subprotocol 
running on Web Sockets, the platform is the browser with the Web Socket 
API, and the client is the JS code invoking the API, and implementing the 
client part of the subprotocol. The server implements both the Web Socket 
protocol and the subprotocol running over it.

The subprotocol is like HTTP, IRC, FTP, etc; Web Sockets is like TCP, and 
the platform is like the OS's network stack. Except everything is moved up 
one level.

HTH,

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 04 Feb 2010 01:10:20 +1100

I don't think you can assume that most websocket using software
and/or subprotocols will have the same developer working on
the client side and the server side.

I can see separate impls for opera, firefox, jetty, tomcat, httpd etc.
for some a standard subprotocol for multiplexing ws connections.

I can also see standards emerge for ws connection management that
supports reconnects and retransmission.  These might be implemented
in server frameworks and in js libraries like dojo and jquery.

Even if the dev team is the same client and server, it's just
as likely that different developers will work on the javascript
vs server language portions.    Even if the same person wrote
both parts, it sounds like fragile design to assume implementation
details of the other part.

Websockets is meant to be a significant change in web application
development and new app models are likely to emerge.   I think it
is very dangerous to say anything is "typical" of a protocol
not yet deployed.  Even if there are typical usage patterns, I
don't think it is good to design a protocol that only works for
them.

regards

---

From: Mridul Muralidharan <[email-masked]>
Date: Wed, 3 Feb 2010 20:52:50 +0530 (IST)

----- Original Message ----
> From: Greg Wilkins <[email-masked]>
> To: "[email-masked]" <[email-masked]>
> Sent: Wed, 3 February, 2010 7:40:20 PM

+1
Cant agree more !
We have already seen this in action with the long poll based spec in xmpp.
I am sure hybi will end up being much more aggressively developed and deployed due to the greater reach and interest.

Regards,
Mridul

> > the platform is like the OS's network stack. Except everything is moved up 

      The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

---

From: Pieter Hintjens <[email-masked]>
Date: Wed, 3 Feb 2010 16:51:09 +0100

> the client side and the server side.

I've never seen a protocol that makes this assumption and it is a
quite extraordinary one from the perspective of interoperabilty.

Ian, could you elaborate please?  I do not understand your intention
in making this statement, nor how significant it is to implementing
WebSocket.

-Pieter

---

From: Maciej Stachowiak <[email-masked]>
Date: Wed, 03 Feb 2010 07:59:04 -0800

> in making this statement, nor how significant it is to implementing

The likely deployments of WebSocket are going to be one of these:

JavaScript code running in a browser --> browser's WebSocket implementation ====> WebSocket service that directly implements the protocol

JavaScript code running in a browser --> browser's WebSocket implementation ====> generic WebSocket server --> service code

Ian is saying that he thinks the JS code running in the browser and the specific WebSocket service are likely to be created by the same entity on some level, and he apparently thinks it is likely that many WebSocket services will implement the protocol directly rather than use any kind of framework or standard server. Note though that the browser code that implements the client side of the WebSocket protocol will almost never be written by the same party. Also, if the WebSocket service accepts cross-site connections, the assumption will not be true at all.

Regards,
Maciej

---

From: Mridul Muralidharan <[email-masked]>
Date: Wed, 3 Feb 2010 22:24:06 +0530 (IST)

--- On Wed, 3/2/10, Maciej Stachowiak <[email-masked]> wrote:

> To: "Pieter Hintjens" <[email-masked]>
> Cc: "[email-masked]" <[email-masked]>
> Date: Wednesday, 3 February, 2010, 9:29 PM
> 
> JavaScript code running in a browser --> browser's
> WebSocket implementation ====> WebSocket service that
> directly implements the protocol
> 
> JavaScript code running in a browser --> browser's
> WebSocket implementation ====> generic WebSocket server
> --> service code

Is hybi expected to be limited only to javascript or even browser based clients ?
I was assuming other clients might also be interested in leveraging it ? (if nothing else, for async notifications on top of http).

Regards,
Mridul

> 
> 
> Ian is saying that he thinks the JS code running in the
> browser and the specific WebSocket service are likely to be
> created by the same entity on some level, and he apparently
> thinks it is likely that many WebSocket services will
> implement the protocol directly rather than use any kind of
> framework or standard server. Note though that the browser
> code that implements the client side of the WebSocket
> protocol will almost never be written by the same party.
> Also, if the WebSocket service accepts cross-site
> connections, the assumption will not be true at all.
> 
> Regards,
> Maciej
>

---

From: Justin Erenkrantz <[email-masked]>
Date: Wed, 3 Feb 2010 08:55:17 -0800

> I was assuming other clients might also be interested in leveraging it ? (if nothing else, for async notifications on top of http).

As I have mentioned before, I am certainly interested in non-browser
applications.  -- justin

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 03 Feb 2010 09:02:30 -0800

> As I have mentioned before, I am certainly interested in non-browser
We're using it for server to server messaging (using the binary packet 
encoding) as well as non-browser clients.

-- Scott

---

From: Graham Klyne <[email-masked]>
Date: Thu, 04 Feb 2010 09:38:43 +0000

> the platform is like the OS's network stack. Except everything is moved up 

Thanks: this clarifies for me what you meant by saying that, but I do feel it's 
potentially missing a trick.  The kind of interoperability you describe seems to 
me akin to the kind of interoperability one might get for, say, an application 
based on an RPC library, where one does often (but not always) see both ends 
implemented by the same team.

But on the Web, I'd say this is a rather small part of the overall 
interoperability picture.  I suggest that the vast number of interactions that 
take place over the web are between clients and servers implemented by different 
parties.  E.g. Firefox talks to Apache; Safari talks to IIS; and similarly for 
the other examples you give.

I suppose you might say that increasing use of AJAX applications and their ilk 
is changing the game, which to some extent it is, but I feel that would be short 
sighted: surely we want to be enabling an environment which allows interoperable 
  Ajax services and clients to be developed.  (This is close to one of my 
interests, in which I'd like to work towards using third party widgets with 
home-grown and third party services for data integration and visualization.)

The thrust of my comments here is to suggest that "The server and the client are 
typically written by the same person" is not a good criterion for guiding design 
decisions for an open protocol, which I thought WebSockets was now intended to 
become.

#g