Subject: Re: [Uri-review] ws: and wss: schemes

From: Ian Hickson <[email-masked]>
Date: Fri, 14 Aug 2009 10:47:56 +0000 (UTC)

(Resending to [email-masked] and [email-masked] with fewer recipients 
to avoid recipient limit. Sorry for screwing up the threading.)

On Fri, 7 Aug 2009, Julian Reschke wrote:
> Ian Hickson wrote:
> > The formal registrations for the ws: and wss: schemes, part of the Web
> > Socket protocol, will be available in the Web Socket protocol ID as soon as
> > the IETF upload process completes:
> > 
> >    http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol#section-7
> 
> Looking at:
> 
> > 7.1. Registrarion of ws: scheme
> > 
> >    URI scheme name.
> >       ws
> > 
> >    Status.
> >       Permanent.
> > 
> >    URI scheme syntax.
> >       "ws" ":" hier-part [ "?" query ]
> 
> I assume you are using ABNF syntax (RFC5234) and terminology from the URI
> spec, but you really need to state that.

Thanks, fixed.

(I tried referencing STD68 instead of RFC5234, as we do in HTML5, but 
apparently there's no index of STD references for xml2rfc?)

> >    URI scheme semantics.
> >       The only operation for this scheme is to open a connection using
> >       the Web Socket protocol.
> > 
> >    Encoding considerations.
> >       UTF-8 only.
> 
> What does this mean?

That the only encoding that can be used with this scheme is UTF-8. What is 
unclear?

> >    Applications/protocols that use this URI scheme name.
> >       Web Socket protocol.
> > 
> >    Interoperability considerations.
> >       None.
> > 
> >    Security considerations.
> >       See "Security considerations" section above.
> > 
> >    Contact.
> >       Ian Hickson <[email-masked]>
> > 
> >    Author/Change controller.
> >       Ian Hickson <[email-masked]>
> > ...
> 
> I believe that for a permanent registry on the standards track, change control
> will have to move to IETF or IESG.

When a registration is placed on the permanent registry, the change 
controller line is removed.

On Fri, 7 Aug 2009, Julian Reschke wrote:
> 
> Furthermore I see no mention at all of these URI schemes in the 
> remainder of the spec. So is the IANA registration template supposed to 
> *be* the definition?

Yes.

> In particular:
> 
> 1) What does a "ws" URI identify?

I have added text speaking to this.

> 2) What is the semantics of "hier-part" and "query"?

I've refactored the Web Sockets API and Web Sockets Protocol specs so that 
the parsing of Web Socket URLs happens all within the Web Sockets Protocol 
spec. Let me know if that's not enough. I can include more details if you 
would like.

(If you do want more, a reference to how another scheme registration 
defines the semantics of parts of the URL would be useful, so that I can 
use a similar style.)

On Fri, 7 Aug 2009, Greg Wilkins wrote:
> 
> Note also that ws is commonly used to denote Web Service.
> For example:
> 
>   http://www.w3.org/TR/ws-arch/
>   http://www.w3.org/TR/ws-i18n/
>   http://ws.apache.org/
>   http://www.ws-i.org/
>   https://jax-ws.dev.java.net/
> 
> While you will never avoid all name clashes, it would be desirable to 
> avoid a clash within the same field of endeavour.
> 
> Perhaps wskt would be a better scheme name for WebSocKeT, unless here 
> are any objections from Wichita Southeast Kansas Transit Inc 
> http://acronyms.thefreedictionary.com/Wichita+Southeast+Kansas+Transit+Inc. 
> (note wsock is already used for winSock drivers).

On Fri, 7 Aug 2009, Maciej Stachowiak wrote:
> 
> I think it's unlikely the "ws" family of formats and protocols will want 
> to register their own URI scheme. That being said, how about "wsp" for 
> WebSocket Protocol?

On Fri, 7 Aug 2009, Chris Anderson wrote:
> 
> As long as we are throwing out acronyms, my suggestion would be for the 
> simple "websocket" or maybe "websock" if bytes are considered expensive. 
> I'm a big fan of the mailto: link b/c it's easy to understand what it 
> means.

On Sun, 9 Aug 2009, Mark Nottingham wrote:
>
> websock seems like a good balance...

On Sun, 9 Aug 2009, Pieter Hintjens wrote:
> 
> 'ws' is confusing to many and cryptic to the rest.  'websock' has the 
> advantage of immediate clarity to all.  The 5-characters difference is 
> not worth the cost of defining a new abbreviation that adds no value.
> 
> Think of this as an opportunity for marketing.  "websock" reinforces the 
> brand, while "ws" weakens it.

I haven't changed the names of the schemes.

"wskt" looks really ugly. "wskts" sounds like a plural.

"wsp" and "wsps" don't seem to reduce the confusion with Web Services, 
should they ever add a protocol (and the closeness to "wasp" is not 
especially good, IMHO).

"websock" and "wsock" are too close to the Windows Sockets names, IMHO.

I'm not really concerned about branding; I mean, 90% of the time these 
URLs are going to be seen in contexts like:

   var socket = new WebSocket('ws://example.com/');

...so it doesn't seem like much of an issue.

On Fri, 7 Aug 2009, David Booth wrote:
> > 
> > This is only expected to be used from a WebSocket API call. What 
> > fallback behaviour did you have in mind?
> 
> I'm sure I could come up with some ideas, but I imagine those who have 
> been working on HTML5 could come up with much better ones.  I tried to 
> look in the HTML5 spec for some inspiration, but I find the word 
> "socket" appearing only once, in section 8.1: 
> http://dev.w3.org/html5/spec/Overview.html#event-definitions Where 
> should I look for the info that gives the context for the WebSocket API?  
> OTOH, protocols have a way of finding uses far beyond those for which 
> they were originally envisioned.

The Web Socket API is here:

   http://dev.w3.org/html5/websockets/

I don't really see any sane way to have the API itself do some sort of 
fallback, especially not to HTTP.

On Sat, 8 Aug 2009, Jamie Lokier wrote:
> 
> Tunnelling WebSocket two-way communications over standard HTTP messages, 
> using any of the methods used for that, would be natural and probably 
> useful behaviour.

That is a radically different system on the server-side. While authors 
should naturally be allowed to do that using XHR if they desire, forcing 
servers into having that kind of fallback behaviour radically increases 
the complexity of a server-side deployment. A Web Socket protocol 
implementation can be written in 100 lines of Perl or so; having to add to 
that some sort of HTTP server or CGI scripting would make things 
significantly more complicated.

On Sun, 9 Aug 2009, Julian Reschke wrote:
> 
> Are you saying the URI scheme is *only* needed within WebSocket's API? 
> Why do you need a URI scheme in the first place, then?

It would also be useful when redirecting connections to other servers, if 
we keep that feature, and in general for identifying Web Socket servers 
going forward. For example, we are considering using the Web Socket 
protocol for push notifications; this would need to be declared in an HTML 
page, say in a <link> header, for which having to separately specify the 
host, port, and resource name would be quite inconvenient.

On Sun, 9 Aug 2009, David Booth wrote:
> 
> I can't see that as a significant issue, as there is only a trivial 
> difference between dispatching based on the string prefix 
> "http://wss.example/" and the string prefix "wss:".  Both are simple, 
> constant strings and both are equally "magic": they cause agent to 
> attempt the WSS protocol.

I don't understand how one would reuse the http: scheme in this way in 
order to connect to 'example.com' as both an HTTP server and a Web Socket 
server.

On Tue, 11 Aug 2009, Maciej Stachowiak wrote:
> 
> The difference is that "http://wss.example/" already has a meaning, 
> which is not the intended one. Whereas "wss:" currently has no meaning. 
> Thus the former has greater risk of either colliding with an existing 
> resource, or being misinterpreted by a legacy client (instead of just 
> rejected).

Indeed.

On Tue, 11 Aug 2009, David Booth wrote:
> 
> That's not a risk, that's the *intent*.

I disagree that that is a desireable result in this instance.

> The point is that a prefix like "http://wss.example/" gives agents that 
> do not know the WSS protocol the possibility of doing something useful 
> with the URI, by falling back to the HTTP protocol, whereas if a prefix 
> like "wss:" were used those same agents would have to reject it 
> entirely.

UAs that don't support Web Socket are going to have failed long before 
they see the URI, since they don't support the API either.

On Wed, 12 Aug 2009 [email-masked] wrote:
> 
> So, here's an example.  First, let's make the assumption that there is 
> an HTTP server at port 80 at "http://wss.example/", presumably run by an 
> organization that supports the use of wss.  Assuming that the normal 
> path through the Web sockets client apis does not access this, the HTTP 
> server will be used only by legacy clients.
> 
> Where's the value?  Let's assume that a link to a WS resource winds up 
> in a page somewhere for some reason.  It could be a bug report, 
> whatever. Now a search engine crawler stumbles on the bug report page.  
> If we use the wss: scheme, then either the crawler has special knowledge 
> of WS, or nothing much useful happens.  If we use 
> "http://wss.example/..... then the crawler sends a GET to that.  Choose 
> your favorite metadata access mechanism (perhaps [1], maybe RDFa, 
> whatever), and the crawler has the opportunity to discover "ah, this is 
> a WS resource", or at least to learn some things about it.  To some 
> extent that's true with either approach (the crawler at least knows it's 
> got a link in a scheme that's not understood with wss:), but the 
> opportunities for incremental discovery seem to be significantly greater 
> with HTTP.
> 
> As Dave Orchard points out, these issues were debated in great detail 
> with XRI came up for consideration at Oasis, and I think it's fair to 
> say that the starting position of those proposing xri was initially at 
> least as firm as that of advocates of wss.  I think Dave is right that 
> at least many of those same people came to believe that an http-based 
> approach was in fact either better, or at least a reasonable compromise.  
> You might want to check with them.

Do you believe that the above applies to Web Socket protocol connections 
more than it does to telnet or SSH connections? If so, why?

I don't understand why we would even want a spider to attempt to connect 
to a Web Socket server. How would it know what application-level protocol 
to use? How would it expose this information to the user?

On Wed, 12 Aug 2009, Greg Wilkins wrote:
> 
> The ws protocol has been designed in respect to a single proposed 
> javascript API to run within a browser.
> 
> There are many many uses for bidirectional web which go beyond that 
> simple restricted example.

For those purposes, we already have TCP/IP, no?

On Wed, 12 Aug 2009, Jamie Lokier wrote:
>
> ws:// URLs *will* be entered on web forms at some point, you can count 
> on it.
> 
> Someone will write a web form that says something like "Tell me the 
> address of a FOOCHAT server to begin your FOOCHAT session", expecting a 
> ws:// URL to be entered if WebSockets is the protocol to be used.

Indeed, that's a good reason for having a single URL rather than having 
multiple fields in the API.

Once the IETF system updates to the latest submission, the changes I have 
made in response to the above feedback will be visible in version 32 or 
later of the following document:

   http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Toby Inkster <[email-masked]>
Date: Fri, 14 Aug 2009 16:12:34 +0100

On Fri, 2009-08-14 at 10:47 +0000, Ian Hickson wrote:
> order to connect to 'example.com' as both an HTTP server and a Web
> Socket server.

The idea is not necessarily to reuse the HTTP protocol - just the URI
scheme. How to do this is pretty easy and would require very few changes
to the web sockets API.

1. You, Ian Hickson, in your role as editor of the API spec. (Or a
person or organisation nominated by you) would register a domain or
subdomain such as websockets.net or sockets.w3.org. For the rest of this
message, I'll assume websockets.net.

2. In part 4 of the draft, point #1 which starts "Parse a Web Socket
URL's components..." describe an algorithm of how to parse a WebSocket
URL like this (square brackets are used to indicate variables):

	i) A web socket is parsed as a standard HTTP/HTTPS URL
	  obtaining [protocol], [host], [port] and [path].
	ii) If [host] is not websockets.net, throw SYNTAX_ERR.
	iii) If [port] is blank and [protocol] is "http" then
	  set [port] to 81.
	iv) If [port] is blank and [protocol] is "https" then
	  set [port] to 815.
	v) Match [path] against the following regular expression:
	  ^\/([^\/]+)(/.*)?$
	  Throw SYNTAX_ERR if the expression is not matched.
	  Set [host] to the first matched subexpression.
	  Set [path] to the second matched subexpression.
	vi) Set [secure] to true if [protocol] is "http".
	  Set [secure] to false if [protocol] is "https".
	  Throw SYNTAX_ERR if neither of the above.
	
The above procedure will always end up with four variables:

	* [host] representing the host the API should connect to
	* [port] representing the port the API should connect to
	* [path] representing the Web Socket resource name
	* [secure] indicating whether the Socket should be tunnelled
	  over TLS.

or will throw SYNTAX_ERR.

3. ???

4. Profit.

-- 
Toby A Inkster
<mailto:[email-masked]>
<http://tobyinkster.co.uk>

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 14 Aug 2009 12:29:27 -0700

> scheme. How to do this is pretty easy and would require very few  

It only makes sense to do that if the resources identified for
websockets are the same as those for "http".  That is why xri
was not needed -- it was a simple alias of http space.  I do not
think that is Ian's intention for websockets (although, as Julian
suggested, there is no documentation for that in the spec yet).

....Roy

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 14 Aug 2009 20:35:05 +0100

> fallback, especially not to HTTP.

Ah, that'll be why you didn't respond to discussions about that :-)

> That is a radically different system on the server-side.

Yes.

> While authors should naturally be allowed to do that using XHR if
> they desire,

All authors can just use the WebSocket API.

When running on older browsers they'll use a Javascript
backward-compatibility package which emulates WebSocket, using some
other mechanism underneath.

> forcing servers into having that kind of fallback behaviour
> radically increases the complexity of a server-side deployment.

Not much.  The art of good software development is to hide the
complexity where it has to exist.

"apt-get install ws2http".  Problem solved.

> A Web Socket protocol implementation can be written in 100 lines of
> Perl or so;

If using libraries, 5 lines of Perl.

If avoiding libraries, you can, barely, but:

   - Trust me there will be bugs, if only in mishandling UTF-8.

   - Anyone doing that values simplicity over performance.
     Which is great, see next section.

> having to add to that some sort of HTTP server or CGI scripting
> would make things significantly more complicated.

No, it's very simple to the person writing 100 lines of Perl.

They'll install a WebSocket <-> HTTP proxy which is written by
professionals in C.  "apt-get install ws2http".

On the client side, they'll use a Javascript file which emulates
WebSocket (when necessary), using HTTP which works with ws2http.

Job done.  Johnny's interactive Tic-Tac-Toe (his very first web app!)
will work over the HTTP internet with *no changes at all* except
minimal configuration.

> order to connect to 'example.com' as both an HTTP server and a Web Socket 

The implication is that it's context dependent - you know you're
making a WebSockets connection - and the prefix would serve much the
same role as URIs do for XML namespaces.

But I must agree that http://www.example/my.domain.org/path/to/service
would be profoundly ugly and pointless.

On the other hand, http://my.domain.org/path/to/service
would be very nice if that Just Worked.

That is, if WebSockets connected and did a standardised and safe
negotiation over HTTP to your own site's URL, and acquired a
WebSockets connection that way.  No need for the ws: prefix!

That would be nice because it leaves open the option to do drop down
automatically to something else if the WebSockets negotiation fails.
(Automatically in the WebSockets implementation, that is, without the
app author having to care).

> they see the URI, since they don't support the API either.

They'll acquire a Javascript emulation of it, if that's technically possible.

> to use? How would it expose this information to the user?

Spiders will follow any text that looks like a URL.  Even comments.
You might not _want_ them to, but they will try anyway, because they
and their users benefit from serendipitous discovery.

It's not about what _we_ want them to do.  It's about what they will
do given a chance.

> For those purposes, we already have TCP/IP, no?

Because WebSockets isn't _that_ simple on the server, especially with
things layered on top, there will be frameworks to make it easier on
the server side.  Before you know it, that will be more familiar for a
lot of people than TCP/IP.  In the same way that people can write
CGI/WSGI now, but would find writing a TCP/IP server difficult.

Presumably you'd like WebSockets to become familiar to large numbers
of web application developers?

With millions of programmers familiar with writing WebSockets code on
browser clients and servers, and good server-side frameworks for
handling it - of course it will be used in non-browser applications,
whether you think it's a good choice or not.

It's not about *us* the experts, it's about everyone else who only has
the time to learn one thing well.  The same people who cut and paste
Javascript and CGI/WSGI code right now - because it works.

> multiple fields in the API.

I agree.

Thanks,
-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 14 Aug 2009 20:44:08 +0100

> suggested, there is no documentation for that in the spec yet).

I think you're probably right that it's not Ian's intention for WebSockets.

However I can say for sure that I intend to deploy applications which
serve exactly the same resources over WebSockets and HTTP.

The technical methods used have to be very different, and once you get
past the initial URL there's a certain amount of fudging.  You
wouldn't start 1000 WebSockets connections at different URLs to access
different resources on the same server; you'd create an application
protocol on top of it, and request all the resources through a shared
endpoint.  (That's a consequence of WebSockets' design).

But the conceptual resources being accessed behind WebSockets and HTTP
will be identical in the applications I have in mind.  And the
conceptual "endpoint" resource at a single WebSockets address has a
corresponding endpoint resource over HTTP too.  So it seems natural to
use similar URLs for the two access methods.

-- Jamie