Subject: Re: [Uri-review] ws: and wss: schemes

From: Ian Hickson <[email-masked]>
Date: Fri, 4 Sep 2009 05:33:29 +0000 (UTC)

On Fri, 14 Aug 2009, Julian Reschke wrote:
>
> [...] it now says:
> 
> >    URI scheme syntax.
> >       In ABNF terms using the terminals from the IRI specifications:
> >       [RFC5238] [RFC3987]
> > 
> >            "ws" ":" ihier-part [ "?" iquery ]
> 
> That is even worse than before, because it now uses productions from the 
> IRI spec defining *URI* syntax.

ws: and wss: URLs are i18n-aware; why would we want to limit them to 
ASCII?

> Furthermore, it still doesn't answer what the semantics of these parts 
> are. What do "ihier-part" and "iquery" represent in a ws URI?

This is defined by the RFC 3987, no? Surely we wouldn't want IRI 
components to have different meanings in different schemes?

> What's the effect? How are they used?

This is defined earlier in the Web Socket specification.

> PS: what does RFC5238 have to do with this?

Oops, typo. Fixed. (Meant 5234.)

On Fri, 14 Aug 2009, Julian Reschke wrote:
> Ian Hickson wrote:
> > > I assume you are using ABNF syntax (RFC5234) and terminology from the URI
> > > spec, but you really need to state that.
> > 
> > Thanks, fixed.
> > 
> > (I tried referencing STD68 instead of RFC5234, as we do in HTML5, but
> > apparently there's no index of STD references for xml2rfc?)
> 
> Just day "STD68" instead of "RFC5234" in the reference/@anchor element.

I have no <reference> elements, I'm using the <?rfc include=""?> feature 
and reference.RFC.xxxx.xml files. I couldn't find STD reference files.

> > > >    URI scheme semantics.
> > > >       The only operation for this scheme is to open a connection using
> > > >       the Web Socket protocol.
> > > > 
> > > >    Encoding considerations.
> > > >       UTF-8 only.
> > >
> > > What does this mean?
> > 
> > That the only encoding that can be used with this scheme is UTF-8. What is
> > unclear?
> 
> You can only have ASCII characters in a URI. I believe you're trying to do the
> right thing, but it really requires a few more words (...when non-URL
> characters are to be used in a ws URI, they need to be encoded using UTF-8 and
> then percent-escaped...)

I've deferred to RFC3987 to sidestep this issue.

> > I've refactored the Web Sockets API and Web Sockets Protocol specs so 
> > that the parsing of Web Socket URLs happens all within the Web Sockets 
> > Protocol spec. Let me know if that's not enough. I can include more 
> > details if you would like.
> 
> Maybe than that spec should carry the URI definition and registration.

Which spec? I'm confused. The protocol spec _does_ have the registrations.

> > (If you do want more, a reference to how another scheme registration 
> > defines the semantics of parts of the URL would be useful, so that I 
> > can use a similar style.)
> 
> Looking at relatively recent RFCs, and example would be
> <http://tools.ietf.org/html/rfc4516#section-2>.
> 
> If there is no defined semantics for the two parts then the definition 
> should just state that.

Ok, added a sentence describing the relevant components.

On Fri, 14 Aug 2009, Toby Inkster wrote:
> On Fri, 2009-08-14 at 10:47 +0000, Ian Hickson wrote:
> > I don't understand how one would reuse the http: scheme in this way in 
> > order to connect to 'example.com' as both an HTTP server and a Web 
> > Socket server.
> 
> The idea is not necessarily to reuse the HTTP protocol - just the URI 
> scheme. How to do this is pretty easy and would require very few changes 
> to the web sockets API.
> 
> 1. You, Ian Hickson, in your role as editor of the API spec. (Or a 
> person or organisation nominated by you) would register a domain or 
> subdomain such as websockets.net or sockets.w3.org. For the rest of this 
> message, I'll assume websockets.net.
> 
> 2. In part 4 of the draft, point #1 which starts "Parse a Web Socket
> URL's components..." describe an algorithm of how to parse a WebSocket
> URL like this (square brackets are used to indicate variables):
> 
> 	i) A web socket is parsed as a standard HTTP/HTTPS URL
> 	  obtaining [protocol], [host], [port] and [path].
> 	ii) If [host] is not websockets.net, throw SYNTAX_ERR.
> 	iii) If [port] is blank and [protocol] is "http" then
> 	  set [port] to 81.
> 	iv) If [port] is blank and [protocol] is "https" then
> 	  set [port] to 815.
> 	v) Match [path] against the following regular expression:
> 	  ^\/([^\/]+)(/.*)?$
> 	  Throw SYNTAX_ERR if the expression is not matched.
> 	  Set [host] to the first matched subexpression.
> 	  Set [path] to the second matched subexpression.
> 	vi) Set [secure] to true if [protocol] is "http".
> 	  Set [secure] to false if [protocol] is "https".
> 	  Throw SYNTAX_ERR if neither of the above.
> 	
> The above procedure will always end up with four variables:
> 
> 	* [host] representing the host the API should connect to
> 	* [port] representing the port the API should connect to
> 	* [path] representing the Web Socket resource name
> 	* [secure] indicating whether the Socket should be tunnelled
> 	  over TLS.
> 
> or will throw SYNTAX_ERR.
> 
> 3. ???
> 
> 4. Profit.

That seems like a wild layering violation.

On Fri, 14 Aug 2009, Jamie Lokier wrote:
> 
> However I can say for sure that I intend to deploy applications which 
> serve exactly the same resources over WebSockets and HTTP.
>
> The technical methods used have to be very different, and once you get 
> past the initial URL there's a certain amount of fudging.  You wouldn't 
> start 1000 WebSockets connections at different URLs to access different 
> resources on the same server; you'd create an application protocol on 
> top of it, and request all the resources through a shared endpoint.  
> (That's a consequence of WebSockets' design).
> 
> But the conceptual resources being accessed behind WebSockets and HTTP 
> will be identical in the applications I have in mind.  And the 
> conceptual "endpoint" resource at a single WebSockets address has a 
> corresponding endpoint resource over HTTP too.  So it seems natural to 
> use similar URLs for the two access methods.

If HTTP is applicable, then I really don't think you would want to use 
WebSockets. The two have completely different, non-overlapping use cases, 
as far as I can tell.

On Fri, 14 Aug 2009, Jamie Lokier wrote:
> Ian Hickson wrote:
> > The Web Socket API is here:
> > 
> >    http://dev.w3.org/html5/websockets/
> > 
> > I don't really see any sane way to have the API itself do some sort of 
> > fallback, especially not to HTTP.
> 
> Ah, that'll be why you didn't respond to discussions about that :-)

Actually I was just slow. :-)

> > On Sat, 8 Aug 2009, Jamie Lokier wrote:
> > > Tunnelling WebSocket two-way communications over standard HTTP 
> > > messages, using any of the methods used for that, would be natural 
> > > and probably useful behaviour.
> > 
> > That is a radically different system on the server-side.
> 
> Yes.
> 
> > While authors should naturally be allowed to do that using XHR if they 
> > desire,
> 
> All authors can just use the WebSocket API.
> 
> When running on older browsers they'll use a Javascript 
> backward-compatibility package which emulates WebSocket, using some 
> other mechanism underneath.
> 
> > forcing servers into having that kind of fallback behaviour radically 
> > increases the complexity of a server-side deployment.
> 
> Not much.  The art of good software development is to hide the 
> complexity where it has to exist.
> 
> "apt-get install ws2http".  Problem solved.

You have more faith in people developing libraries and software tools than 
I do.

> > A Web Socket protocol implementation can be written in 100 lines of 
> > Perl or so;
> 
> If using libraries, 5 lines of Perl.
> 
> If avoiding libraries, you can, barely, but:
> 
>    - Trust me there will be bugs, if only in mishandling UTF-8.
> 
>    - Anyone doing that values simplicity over performance.
>      Which is great, see next section.

What are the bugs?

   http://damowmow.com/playground/demos/websocket/blank-server.pl

> > having to add to that some sort of HTTP server or CGI scripting would 
> > make things significantly more complicated.
> 
> No, it's very simple to the person writing 100 lines of Perl.

Having written the 100 lines of Perl, I disagree. I would never want to 
have to add an HTTP server to that.

> They'll install a WebSocket <-> HTTP proxy which is written by 
> professionals in C.  "apt-get install ws2http".
>
> On the client side, they'll use a Javascript file which emulates 
> WebSocket (when necessary), using HTTP which works with ws2http.
> 
> Job done.  Johnny's interactive Tic-Tac-Toe (his very first web app!) 
> will work over the HTTP internet with *no changes at all* except minimal 
> configuration.

Well it'd be cool if such tools become available in the transition period, 
but I'm not making any assumptions that they will be.

> > On Sun, 9 Aug 2009, David Booth wrote:
> > > I can't see that as a significant issue, as there is only a trivial 
> > > difference between dispatching based on the string prefix 
> > > "http://wss.example/" and the string prefix "wss:".  Both are 
> > > simple, constant strings and both are equally "magic": they cause 
> > > agent to attempt the WSS protocol.
> > 
> > I don't understand how one would reuse the http: scheme in this way in 
> > order to connect to 'example.com' as both an HTTP server and a Web 
> > Socket server.
> 
> The implication is that it's context dependent - you know you're making 
> a WebSockets connection - and the prefix would serve much the same role 
> as URIs do for XML namespaces.
>
> But I must agree that http://www.example/my.domain.org/path/to/service 
> would be profoundly ugly and pointless.
> 
> On the other hand, http://my.domain.org/path/to/service would be very 
> nice if that Just Worked.
>
> That is, if WebSockets connected and did a standardised and safe 
> negotiation over HTTP to your own site's URL, and acquired a WebSockets 
> connection that way.  No need for the ws: prefix!
> 
> That would be nice because it leaves open the option to do drop down 
> automatically to something else if the WebSockets negotiation fails. 
> (Automatically in the WebSockets implementation, that is, without the 
> app author having to care).

We can drop down even with the ws: prefix.

> > I don't understand why we would even want a spider to attempt to 
> > connect to a Web Socket server. How would it know what 
> > application-level protocol to use? How would it expose this 
> > information to the user?
> 
> Spiders will follow any text that looks like a URL.  Even comments.

Then we definitely don't want to use http:/https:.

> > On Wed, 12 Aug 2009, Greg Wilkins wrote:
> > > The ws protocol has been designed in respect to a single proposed 
> > > javascript API to run within a browser.
> > > 
> > > There are many many uses for bidirectional web which go beyond that 
> > > simple restricted example.
> > 
> > For those purposes, we already have TCP/IP, no?
> 
> Because WebSockets isn't _that_ simple on the server, especially with 
> things layered on top, there will be frameworks to make it easier on the 
> server side.  Before you know it, that will be more familiar for a lot 
> of people than TCP/IP.  In the same way that people can write CGI/WSGI 
> now, but would find writing a TCP/IP server difficult.

Your faith in WebSockets' success is heart-warming, though I think it's a 
little optimistic. :-)

> With millions of programmers familiar with writing WebSockets code on 
> browser clients and servers, and good server-side frameworks for 
> handling it - of course it will be used in non-browser applications, 
> whether you think it's a good choice or not.

Fair enough, but we certainly shouldn't optimise for those cases.

On Fri, 14 Aug 2009, Jamie Lokier wrote:
> 
> I suspect WebSockets could benefit from a look at the mechanisms used by 
> cross-domain XHR to check for permission to proceed.

The handshake in Web Sockets is based on the CORS (XHR) mechanism.

> The only difference with WebSockets is that it (so far) seems to avoid 
> any descriptive metadata, which means there will still be applications 
> which ask for a WebSockets URL, but when the URL is for a different 
> protocol on top, it'll simply break with undefined behaviour instead of 
> a clean error message or fallback behaviour.

Web Sockets has an out-of-band protocol specifier to avoid this issue.

On Sun, 16 Aug 2009, Jamie Lokier wrote:
> 
> WebSockets philosophy seems to be to check for WebSockets itself, but 
> then avoid the problem at the next level, passing it on to the 
> application to do an initial handshake, which has the same problems as 
> raw TCP in that there is no _standard_ negotiation handshake.

The Web Sockets protocol does have an application-layer protocol handshake 
as part of the main Web Sockets handshake, for exactly this reason.

On Thu, 20 Aug 2009, Jamie Lokier wrote:

> Three notable things for which there is no consensus are:
> 
>   1. Does the WebSockets really have to use a fresh, new TCP
>      connection for every individual instance of the WebSockets object
>      created on every web page in a browser to the same site?  Or
>      could it upgrade an HTTP connection which has already been used
>      for something else?

I wouldn't expect more than one WebSocket connection per page; clients 
would presumably want to multiplex multiple services over one connection 
(typically making use of a single socket from a shared worker).

I also think that the case of an upgrade from HTTP to Web Socket will be 
relatively rare, such that optimising for that case (e.g. by reusing an 
HTTP connection if there is one) wouldn't necessarily make sense.

>   2. Since it begins with an HTTP request for upgrade to WebSockets,
>      is there any reason for that request not to be to an arbitrary
>      HTTP host+port+path by simply trying the upgrade method on the
>      appropriate HTTP path at the server?

I don't understand the question. Could you elaborate?

>   3. If a WebSockets connection cannot be established, clients
>      will(**) fall back to an equivalent (but bulkier/slower) protocol
>      which uses HTTP only.  What URI will they use in that case?  Do
>      we leave it to individual application/framework designers, or do
>      we suggest a common strategy?
> 
> ** - As in, even if the spec does not provide this and browsers do not
>      provide it, there will certainly be Javascript
>      WebSockets-emulation modules and corresponding server-sides which
>      do.  We can choose to ignore this and leave the mapping from
>      WebSockets URI to HTTP-fallback URI unspecified, or recommend
>      a mapping, or they can simply by the same URI given point 2.

Since this will only be relevant for a transition period, I think it makes 
more sense to just leave that up to the authors.

On Thu, 20 Aug 2009 [email-masked] wrote:
> 
> Some application is using these ws: or wss: URIs.  The URIs wind up in 
> documents or in other places where they can be discovered, perhaps 
> because that is how the application keeps track of or advertises various 
> web socket endpoints.  Now some other agent, perhaps a search crawler, 
> comes upon the documents and finds the references.  If they are 
> http-scheme URIs, then the crawler will know how to dereference them.  
> Of course, this agent will not go through the upgrade protocol, and will 
> attempt an ordinary HTTP GET.  If the server (I.e. the same software 
> that would have processed the Web socket upgrade) chooses to, it can 
> respond with metadata explaining that this is a Websocket resource, 
> perhaps providing information about its purpose and correct use, etc.  
> The means for returning that metadata might be along the lines of [1].  
> That discoverability seems to have some value.

From Google's perspective, I can't really see what we would do with this, 
to be honest. It's not like we'd ever surface a Web Socket URL in the 
results or anything like that -- and people can already expose HTML files 
easily, no need for a new way to discover them as per here.

> Websockets seem different in two ways, one of which is more or less a 
> happy accident:
> 
> 1) Websockets are new, so there's an awful lot of software out there 
> that will just not recognize the new schemes at all.

None of this software should ever get exposed to them, though.

> 2) (happy accident) I suspect the choice of an HTTP-compatible handshake 
> was made to get through firewalls, but I think it also facilitates the 
> use of HTTP for metadata discovery for web socket resources.

It was made only to allow port sharing; I do not believe anything will 
really get us through firewalls (other than tunnelling over TLS).

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Joseph A Holsten <[email-masked]>
Date: Fri, 4 Sep 2009 02:14:48 -0500

> ws: and wss: URLs are i18n-aware; why would we want to limit them to

URIs are not i18n-aware, you're thinking of IRIs. But since there is a  
standard mapping for IRIs, it's pretty clear what you actually want.  
The *URI* syntax should be:

   "ws" ":" heir-part [ "?" query ]

Then the encoding considerations should be something like:

   Because many characters are not permitted with this syntax, the
   "heir-part" and "query" elements may contain characters from the
   Unicode Character Set [UCS] as suggested by URI [RFC3986] using the
   reg-name and percent-encoding translations of IRI to URI
   mapping [RFC3937]. Translation is performed by first encoding those
   Unicode characters as octets to the UTF-8 character
   encoding [RFC3629]. Replace the reg-name part of the heir-part by
   the part converted using the ToASCII operation specified in section
   4.1 of [RFC3490] on each dot-separated label, and by using U+002E
   (FULL STOP) as a label separator, with the flag UseSTD3ASCIIRules
   set to TRUE, and with the flag AllowUnassigned set to TRUE. Then
   only those octets that do not correspond to characters in the
   unreserved set should be percent-encoded.

   By using UTF-8 encoding, there are no known compatibility issues
   with mapping Internationlized Resource Identifiers to websocket
   URIs according to [RFC3987].

> This is defined earlier in the Web Socket specification.

Section 3.1 Parsing Web Socket URLs seems to make the semantics pretty  
clear to me. How about adding "See Section 3.1" to URI scheme  
semantics portions of the IANA Considerations sections? Would that be  
sufficient?

Joseph Holsten

---

From: Julian Reschke <[email-masked]>
Date: Fri, 04 Sep 2009 19:00:02 +0200

> ws: and wss: URLs are i18n-aware; why would we want to limit them to 

Because that's how URI and thus URLs are defined.

> components to have different meanings in different schemes?

If you can point to a section in RFC 3987 which defines more than the 
syntax, and can state that that also applies to "ws", then, great...

> This is defined earlier in the Web Socket specification.

I don't think it was when I wrote that emaol.

> and reference.RFC.xxxx.xml files. I couldn't find STD reference files.

Don't use the include feature then.

> I've deferred to RFC3987 to sidestep this issue.

A URI is not a IRI.

You can refer to the mapping, but that really needs a few more words 
than "See RFC3987.".

> ...

BR, Julian

---

From: Ian Hickson <[email-masked]>
Date: Fri, 4 Sep 2009 19:54:56 +0000 (UTC)

On Fri, 4 Sep 2009, Julian Reschke wrote:
> > ws: and wss: URLs are i18n-aware; why would we want to limit them to ASCII?
> 

The ws: and wss: URLs are IRIs; why would we limit them to URIs? I'm not 
especially interested in ASCII-only URIs at this point. These URLs are 
only ever going to be used in contexts that accept full IRIs.

> > > Furthermore, it still doesn't answer what the semantics of these 
> > > parts are. What do "ihier-part" and "iquery" represent in a ws URI?
> > 

Isn't what the Web Socket protocol spec now says suitable?

> > I have no <reference> elements, I'm using the <?rfc include=""?> feature and
> > reference.RFC.xxxx.xml files. I couldn't find STD reference files.
> 

The reference feature allows me to automatically generate the references, 
which is of more benefit to me than referencing STD numbers instead of 
RFC numbers.

> You can refer to the mapping, but that really needs a few more words 

I don't care about the URI part, only the IRI part.

On Fri, 4 Sep 2009, Phillips, Addison wrote:
> 
> I agree with Julian. If you are defining a URI syntax, you can't use IRI 
> to do so.

I've no intention of defining a URI, only an IRI.

> Section 2.5 of URI, however, does allow what you mean here, when it 
> says:
> 
>    When a new URI scheme defines a component that represents textual
>    data consisting of characters from the Universal Character Set [UCS],
>    the data should first be encoded as octets according to the UTF-8
>    character encoding [STD63]; then only those octets that do not
>    correspond to characters in the unreserved set should be percent-
>    encoded.  For example, the character A would be represented as "A",
>    the character LATIN CAPITAL LETTER A WITH GRAVE would be represented
>    as "%C3%80", and the character KATAKANA LETTER A would be represented
>    as "%E3%82%A2".
> 
> If 'ws:' were defined as an IRI scheme, you could then use RFC 3987 to 
> define its mapping to a URI. This is what is done in specs like XLink 
> 1.1. Defining 'ws:' as an IRI scheme would not necessarily be a bad 
> thing

Ok... Let's do that then. I couldn't find any documentation on how to do 
that. I've just changed the words "URI" to "IRI" in the registration. Is 
that what needs to happen?

> I've found that confusion tends to surround when an IRI is 
> happily being an IRI and when it needs to be mapped down to a URI.

I'm still confused as to why we still have URIs at all. They're such an 
anachronism.

> It may not need many more words, but certainly a few more words.

Could you elaborate? Which words should I add?

---

From: Kristof Zelechovski <[email-masked]>
Date: Fri, 4 Sep 2009 22:05:54 +0200

URI restrictions are not an anachronism.  Imagine an Englishman with no
knowledge of Japanese and no pocket translator to access a resource under a
Japanese IRI that he has printed on paper.  Good luck with that.
Best regards,
Chris

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Ian Hickson
Sent: Friday, September 04, 2009 9:55 PM
To: URI; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Uri-review] [hybi] ws: and wss: schemes

---

From: Julian Reschke <[email-masked]>
Date: Fri, 04 Sep 2009 22:05:36 +0200

> only ever going to be used in contexts that accept full IRIs.

But that's not who registering an URI scheme works. Check the relevant 
RFCs. Essentially you register the *URI* scheme, and get IRIs based on 
the mapping rules defined in RFC 3987.

> Isn't what the Web Socket protocol spec now says suitable?

I haven't checked yet.

It would be helpful if you didn't abuse the IETF ID submission system as 
local storage, and only submitted new drafts when you want community 
review. Is now a good moment to read it?

> which is of more benefit to me than referencing STD numbers instead of 

The RFC reference is immutable. Just paste the content in your source 
file, and change the anchor attribute value.

> I don't care about the URI part, only the IRI part.

You can't register an IRI scheme. (Unless I missed something).

> that what needs to happen?

Nope. (See above)

> Could you elaborate? Which words should I add?

You need to state how you want to encode non-ASCII characters. "See 
RFC3987" goes into the right direction but really isn't sufficient. 
Please see RFC 4395, Section 2.6:

"2.6. Internationalization and Character Encoding

    When describing URI schemes in which (some of) the elements of the
    URI are actually representations of human-readable text, care should
    be taken not to introduce unnecessary variety in the ways in which
    characters are encoded into octets and then into URI characters; see
    RFC 3987 [6] and Section 2.5 of RFC 3986 [5] for guidelines.  If URIs
    of a scheme contain any text fields, the scheme definition MUST
    describe the ways in which characters are encoded, and any
    compatibility issues with IRIs of the scheme."

BR, Julian

---

From: Ian Hickson <[email-masked]>
Date: Fri, 4 Sep 2009 20:19:47 +0000 (UTC)

On Fri, 4 Sep 2009, Julian Reschke wrote:
> > The ws: and wss: URLs are IRIs; why would we limit them to URIs? I'm 
> > not especially interested in ASCII-only URIs at this point. These URLs 
> > are only ever going to be used in contexts that accept full IRIs.
> 

That's what I thought, but then I got feedback saying I had to register an 
IRI scheme if I wanted to use IRIs.

I've no interest in making ws: and wss: URIs. Only IRIs.

If I define the syntax to be a subset of the full URI syntax, how does it 
ever get extended to be a subset of the full IRI syntax?

What should I put in the spec to make you happy and to make the use of ws: 
and wss: IRIs fully well-defined?

> It would be helpful if you didn't abuse the IETF ID submission system as local
> storage, and only submitted new drafts when you want community review.

Release early, release often. I always want community review.

> Is now a good moment to read it?

It's always a good moment to read it.

> > The reference feature allows me to automatically generate the 
> > references, which is of more benefit to me than referencing STD 
> > numbers instead of RFC numbers.
> 

My source file is an HTML document, so I don't think that would work well.

> You need to state how you want to encode non-ASCII characters. "See RFC3987"
> goes into the right direction but really isn't sufficient. Please see RFC
> 4395, Section 2.6:
> 

I've read this, but as far as I can tell, "Always UTF-8" and "See IRI" are 
both complete and accurate ways of addressing this.

Since apparently neither of these options satisfies you, could you state 
exactly what literal text would satisfy you?

---

From: Joseph A Holsten <[email-masked]>
Date: Fri, 4 Sep 2009 18:51:42 -0500

> and wss: IRIs fully well-defined?

The only scheme I can think of that was defined as an IRI was XMPP  
[RFC4622]. It actually makes more sense when you start with IRIs. If  
that's what you need, please just do that.

Traditionally, every other scheme defined since RFC3987 has defined  
itself as a URI and defined the exact encoding considerations to  
handle reserved characters that may occur given the semantics of a  
particular part. You have very standard semantics: userinfo, host,  
port, path segments, query. Those that might meaningfully contain  
reserved characters are userinfo, reg-name segments, and query. reg- 
name parts get ToASCII, everything else gets mapped with percent- 
encoding. You actually have to say this because it's not obvious.  
There's more than one way to do it.

>> RFC3987"

No. There's at least two ways to encode reg-names, tons of UCS  
encoding issues, and more. Pedantic, but that's the point of spec  
review, no?

> exactly what literal text would satisfy you?

If you're going to define it as URI and handle IRIs by mapping, I  
believe my text[1] should satisfy.

1: http://lists.w3.org/Archives/Public/uri/2009Sep/0001.html

Joseph Holsten

---

From: "Phillips, Addison" <[email-masked]>
Date: Fri, 4 Sep 2009 10:41:26 -0700

Hello uri@,

[personal note, not representative of i18n wg]

> Because that's how URI and thus URLs are defined.

I agree with Julian. If you are defining a URI syntax, you can't use IRI to do so. Section 2.5 of URI, however, does allow what you mean here, when it says:

If 'ws:' were defined as an IRI scheme, you could then use RFC 3987 to define its mapping to a URI. This is what is done in specs like XLink 1.1. Defining 'ws:' as an IRI scheme would not necessarily be a bad thing, but I've found that confusion tends to surround when an IRI is happily being an IRI and when it needs to be mapped down to a URI.

> You can refer to the mapping, but that really needs a few more
> words than "See RFC3987.".
> 

Best Regards,

Addison

Addison Phillips
Globalization Architect -- Lab126
Chair -- W3C Internationalization WG

Internationalization is not a feature.
It is an architecture.

---

From: Julian Reschke <[email-masked]>
Date: Sat, 05 Sep 2009 08:43:32 +0200

> and wss: IRIs fully well-defined?

The point is not to make me happy, but to do the right thing.

Just define a URI scheme; use of ws IRIs will be defined automatically 
in terms of RFC 3987 (IRI experts, please correct me if I'm wrong).

> My source file is an HTML document, so I don't think that would work well.

The source file that you feed into xml2rfc is an XML file using the 
RFC2629bis syntax. You control that file. Put into it what you need.

> exactly what literal text would satisfy you?

I already pointed you to RFC 5092 as relatively recent example, see 
<http://tools.ietf.org/html/rfc5092#section-8>.

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Sat, 05 Sep 2009 08:47:39 +0200

> that's what you need, please just do that.

Actually, that RFC *registers* a URI scheme; see 
<http://tools.ietf.org/html/rfc4622#section-3>.

BR, Julian

---

From: Joseph A Holsten <[email-masked]>
Date: Sat, 5 Sep 2009 02:56:53 -0500

Julian Reschke supposedly wrote:
>> [RFC4622]. It actually makes more sense when you start with IRIs.  
>> If that's what you need, please just do that.
>> ...
>
> Actually, that RFC *registers* a URI scheme; see <http://tools.ietf.org/html/rfc4622#section-3 
> >.

It does, and so should websocket. But every other scheme RFC defines  
the URI first and foremost, then describes how to map IRIs. If  
websocket will be an IRI scheme first and foremost, defining it in  
terms of ihier-part and iquery makes sense. Then just adapt the text  
from RFC4622 sections 2 and 3.

The first sentence from <http://tools.ietf.org/html/rfc4622#section-2.8.1 
 > seems quite similar to Ian's preference with websocket:
"If a processing application is presented with an XMPP URI and not  
with an XMPP IRI, it MUST first convert the URI into an IRI by  
following the procedure specified in Section 3.2 of [IRI]."
--
Joseph Holsten

---

From: Julian Reschke <[email-masked]>
Date: Sat, 05 Sep 2009 10:34:51 +0200

Joseph A Holsten wrote:
> It does, and so should websocket. But every other scheme RFC defines the 

Yes.

> URI first and foremost, then describes how to map IRIs. If websocket 
> will be an IRI scheme first and foremost, defining it in terms of 
> ihier-part and iquery makes sense. Then just adapt the text from RFC4622 
> sections 2 and 3.

But there's no registry for IRI scheme, as far as I can tell.

So you always define the URI scheme, and then, when needed, talk about 
how IRIs are mapped.

> The first sentence from 
> <http://tools.ietf.org/html/rfc4622#section-2.8.1> seems quite similar 
> to Ian's preference with websocket:
> "If a processing application is presented with an XMPP URI and not with 
> an XMPP IRI, it MUST first convert the URI into an IRI by following the 
> procedure specified in Section 3.2 of [IRI]."
> ...

But to decide whether something is a URI, not a IRI, you need context, 
right? (Any URI *is* a IRI, after all).

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Sat, 05 Sep 2009 13:08:57 +0200

Ian Hickson wrote:
> That's what I thought I had done; since you disagree that I had done that, 
> please provide the copy that you believe would do the job, so that we can 
> stop dancing around playing "warmer/colder" with this text.
> ...

I'm not going to write the spec for you.

A definition of a URI describes it's ASCII-based syntax, it's purpose, 
and how it's used in the protocol. I think that's pretty clear from RFC 
4395.

> There's no such file; the XML is generated by a script and posted straight 
> to the xml2rfc Web service. I have every intention of keeping this as 
> automatic as possible; I already have to go out of my way to make the 
> references to [WEBADDRESSES] and [HTML5] work, I really don't want to have 
> to start doing the same for IETF documents when I don't actually have to.

Well, you asked how to do it, and I explained how it can be done. 
Apparently you're not happy with the answer, but that's all I can do 
from here.

>>> I've read this, but as far as I can tell, "Always UTF-8" and "See IRI" 
>>> are both complete and accurate ways of addressing this.
>>>
> I tried following the lead given in that draft, but it didn't satisfy you. 
> Could you provide literal text that would satisfy you?

By putting in "See RFC3987"? See Joseph's reply to that, and in 
particular his proposal in 
<http://lists.w3.org/Archives/Public/uri/2009Sep/0001.html>.

BR, Julian

---

From: Ian Hickson <[email-masked]>
Date: Sat, 5 Sep 2009 10:46:17 +0000 (UTC)

On Sat, 5 Sep 2009, Julian Reschke wrote:
> Ian Hickson wrote:
> >
> > That's what I thought, but then I got feedback saying I had to 
> > register an IRI scheme if I wanted to use IRIs.
> > 
> > If I define the syntax to be a subset of the full URI syntax, how does 
> > it ever get extended to be a subset of the full IRI syntax?
> > 
> > What should I put in the spec to make you happy and to make the use of 
> > ws: and wss: IRIs fully well-defined?
> 

My idea of "the right thing" and your idea of "the right thing" have very 
little in common, so really, for me, it boils down to making you happy.

> > > The RFC reference is immutable. Just paste the content in your 
> > > source file, and change the anchor attribute value.
> >

---

From: Jamie Lokier <[email-masked]>
Date: Sat, 5 Sep 2009 14:14:32 +0100

> Japanese IRI that he has printed on paper.  Good luck with that.

Of course these days, an advert or business card will show the IRI and
not the URI, so he is still stuck :-)

But the *existence* of the URI means he can telephone his Japanese
friend and be told the URI to enter to reach that resource.  The URI
restrictions mean it can be spoken too.

It's not language neutral, but virtually every web user can handle the
ASCII characters used in URIs in many human contexts that don't use a
computer.  That's one reason why URIs continue to exist.

-- Jamie

---

From: Toby Inkster <[email-masked]>
Date: Sat, 5 Sep 2009 17:04:29 +0100

On 4 Sep 2009, at 06:33, Ian Hickson wrote:
>> to the web sockets API. [...]
 >

As I understand it, if you, Ian Hickson, own websockets.net, then you  
are the authority for deciding what resources are represented by URIs  
starting with:

	http://websockets.net/

In particular, you in your role as authority are free to decree that  
this:

	http://websockets.net/example.com/foo

Represents a Web Sockets path of "/foo" running on port 81 of the  
host example.com.

-- 
Toby A Inkster
<mailto:[email-masked]>
<http://tobyinkster.co.uk>

---

From: "Daniel R. Tobias" <[email-masked]>
Date: Sat, 05 Sep 2009 13:56:00 -0400

> Represents a Web Sockets path of "/foo" running on port 81 of the  

...which reminds me of Abraham Lincoln declaring that a tail is a 
leg, and asking how many legs a horse has... for which the answer is 
still four, because a tail doesn't become a leg even if somebody in 
"authority" says so.

-- 
== Dan ==
Dan's Mail Format Site: http://mailformat.dan.info/
Dan's Web Tips: http://webtips.dan.info/
Dan's Domain Site: http://domains.dan.info/

---

From: Kristof Zelechovski <[email-masked]>
Date: Sat, 5 Sep 2009 21:53:54 +0200

Ian, you are already going out of your way to make HTML back into XML.  You
are swimming against the current.  A noble exercise, I must admit, but
hardly worth the effort in this case :-)
Cheers,
Chris

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Ian Hickson
Sent: Saturday, September 05, 2009 12:46 PM
To: URI; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Uri-review] [hybi] ws: and wss: schemes

---

From: Julian Reschke <[email-masked]>
Date: Mon, 07 Sep 2009 15:47:26 +0200

> Isn't what the Web Socket protocol spec now says suitable?

It's better.

What I still miss is a reference from the URI registration template to 
the section which defines the syntax (*), and in that section, a 
statement about what the resource name exactly is good for. (It's 
definitively not obvious by just reading the parsing algorithm).

BR, Julian

(*) I think that section would be much more readable when it used ABNF 
as everybody else does. I hear that by specifying an algorithm you want 
to exclude certain standard things like fragments, and include error 
handling; but I think ABNF + prose would be much easier to understand. 
Furthermore, fragment identifiers are orthogonal to the URI scheme, see 
<http://greenbytes.de/tech/webdav/rfc3986.html#rfc.section.3.5.p.2>:

"Fragment identifier semantics are independent of the URI scheme and 
thus cannot be redefined by scheme specifications."

---

From: Kristof Zelechovski <[email-masked]>
Date: Mon, 7 Sep 2009 16:04:31 +0200

And that is why the URN scheme specification badly needs an update.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Julian Reschke
Sent: Monday, September 07, 2009 3:47 PM
To: Ian Hickson
Cc: URI; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Uri-review] [hybi] ws: and wss: schemes

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 08 Sep 2009 19:16:39 +0900

>> are both complete and accurate ways of addressing this.

Ian, you are famous for pointing out your low trust in implementors. 
Given that, I'd tend to spend a few more words on this, to help them 
along a bit better.

>> Since apparently neither of these options satisfies you, could you
>> state exactly what literal text would satisfy you?
>> ...
>

A newer example (the RFC numbers are confusing, because RFC 5092 goes 
back to RFC 2192) would be the XMPP IRI/URI spec, see 
http://tools.ietf.org/html/rfc4622.

Also, please note that there may be nothing like an "IRI only" scheme. 
There is always the chance that in some strange context, a conversion to 
ASCII (i.e. to URIs) happens.

Regards,   Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: David Booth <[email-masked]>
Date: Tue, 08 Sep 2009 17:14:15 -0400

> still four, because a tail doesn't become a leg even if somebody in 

You seem to have missed the point.  A string like
"http://websockets.net/" is not *intrinsically* tied to the Web Sockets
protocol any more than the string "wss: is.  It only becomes tied to the
Web Sockets protocol if the social entity that has the *authority* to
decree such a link does so.  In the case of "wss:", that authority
belongs to the Internet Assigned Numbers Authority (IANA).  In the case
of "http://websockets.net/" that authority belongs to the owner of
websockets.net.  (See Architecture of the World Wide Web section
2.2.2.1:
http://www.w3.org/TR/webarch/#def-uri-ownership )
The principle is *exactly* the same.  

In one case, an agent supporting Web Sockets would recognize the "wss:
prefix, and in the other case, an agent supporting Web Sockets would
recognize the "http://websockets.net/" prefix.  But in the latter case,
an agent that does *not* recognize the "http://websockets.net/" prefix
*might* still be able to do something useful with the URI, by invoking
the HTTP protocol.  Whereas in the former case, an agent that does not
recognize the "wss:" prefix would be unable to do anything further with
it.  This is the benefit of *layering* special purpose protocols on top
of http URIs: it enables good old HTTP to be used as a fallback when
that special purpose protocol is not supported.

-- 
David Booth, Ph.D.
Cleveland Clinic (contractor)

Opinions expressed herein are those of the author and do not necessarily
reflect those of Cleveland Clinic.

---

From: "Daniel R. Tobias" <[email-masked]>
Date: Tue, 08 Sep 2009 20:43:08 -0400

> The principle is *exactly* the same.  

Not really, to me at least.  That would be like saying that because 
you own that particular horse, you can declare its tail to be a leg 
even if the general community of experts in horses and biology in 
general have a definition of legs that it doesn't meet.

Since I own the dan.info domain, I get to define what any particular 
URI within it (or its subdomains), like http://dan.info/FooBarBaz/ or 
http://foo.bar.baz.dan.info/Some.Silly.Stuff, resolves to under the 
HTTP protocol (or whatever other protocol might be referenced in the 
URI scheme, like https://dan.info/Something.Else or 
ftp://dan.info/OtherJunk/).  I can make it resolve to a picture of 
me, a picture of a horse, a political rant, a redirect to a different 
URI, a 404 Not Found error, or not resolve at all because the 
hostname has no DNS record. It doesn't, however, give me the right or 
power to insist that the definition of the "http" scheme or its 
associated protocol doesn't apply to my URIs, and instead those 
particular URIs should be resolved using my own "FooBarBaz" protocol.

Even if the page that is retrieved under a HTTP fetch of the document 
at the URI says "This URI is intended to be resolved under FooBarBaz 
instead of HTTP", that doesn't make it so, any more than writing 
"This sentence is written in French" makes it be in French instead of 
English.

---

From: "Daniel R. Tobias" <[email-masked]>
Date: Tue, 08 Sep 2009 21:13:24 -0400

I wonder if the "HTTP Uber Alles" crowd, if they were active 20 years 
ago, would be insisting that everything, including that newfangled 
HTTP protocol, be expressed in the form of a Gopher address, or 
perhaps FTP or Telnet, or maybe an e-mail address with the standard, 
adopted by the owner of the address, that the subject line contain 
the actual protocol intended to be used?

I also wonder if, should their side win all its battles, 1000 years 
from now all URIs in use will be at least 1000 characters long, of 
which at least 800 of these characters will be fossilized deadwood of 
obsolete protocols that are preserved as magic incantations to begin 
a URI.  So they'll be something like:

http://ipp.solarsystem.net/earth/galacticgateway.net/andromeda/tachyon
.protocol.net/ ...[snip]... /actualsite.actualgalaxy/path-in-site

where "ipp.solarsystem.net", under "http", is the magic indicator of 
the InterPlanetaryProtocol that became dominant in 2067, and was 
followed by the actual address being reached by that protocol, 
starting with its home planet, but then "galacticgateway.net" within 
"earth" became the magic string to indicate that you are actually 
using the InterGalacticProtocol which became dominant in 2152, and 
similarly the "tachyon.protocol.net" signifies the Tachyon Protocol 
that caught on around 2272.

User agent identifiers for the browsers people use with their 31st 
century protocols are similarly long and convoluted, beginning with 
"Mozilla/5.0" and containing references to MSIE, Gecko, and various 
other browser names and codenames that were trendy at some point or 
other over the millennium.

---

From: Toby Inkster <[email-masked]>
Date: Wed, 9 Sep 2009 12:05:46 +0100

> general have a definition of legs that it doesn't meet.

To call a horse's tail a leg would be wrong, because there is an  
established definition for what a horse's leg is. Taking a word which  
already has a meaning and recycling it to mean something else causes  
ambiguity.

Conversely, creating a new word to describe an existing concept that  
already has a perfectly good name risks confusion too.

But what we're talking about is a using a brand new name which has  
never existed before (URLs beginning with "http://websockets.net/" or  
some other currently unregistered domain name) to describe a brand  
new thing which also has never existed before (Web Sockets).

> Since I own the dan.info domain, [...] It doesn't, however, give me  
> the right or

No, it doesn't.

It *does* however give you the right and power to *allow* your URIs  
to be accessed by non-HTTP protocols too, and to promote that fact.  
HTTP still applies to your URIs, but FooBarBaz does too.

HTTP 1.x is only one possible protocol that can be used to resolve  
URIs beginning with "http://". A user agent is free to choose other  
protocols instead. In practice, using HTTP 1.x is often a last- 
resort: most user agents implement a proprietary local disk based  
protocol to resolve URIs, only falling back to HTTP if that fails.  
(i.e. they implement caching.)

Software that understands your protocol will recognise the URI and  
handle it via your protocol. Software which doesn't will be none the  
wiser and simply attempt to handle the URI via HTTP.

---

From: David Booth <[email-masked]>
Date: Wed, 09 Sep 2009 09:33:58 -0400

On Tue, 2009-09-08 at 20:43 -0400, Daniel R. Tobias wrote:
> hostname has no DNS record. 

Right.

> It doesn't, however, give me the right or 

You're missing the point about *layering*.  You are correct that you
don't have the authority to *revoke* the HTTP semantics of your URI.
Rather, your new FooBarBaz semantics would be *layered*
(non-destructively) on top of the HTTP semantics.   

A FooBarBaz URI would still have its HTTP semantics -- that's the whole
point; that's what makes this layering technique useful -- but it *also*
acquires FooBarBaz semantics.  The FooBarBaz semantics will only be used
by those agents that recognize your "http://dan.info/FooBarBaz/" URI
prefix as having special significance.  Those agents could safely ignore
the additional FooBarBaz semantics if they chose, and merely fall back
to the HTTP semantics as non-FooBarBaz-aware agents must do.  But
presumably they will get additional benefit if they choose to recognize
the FooBarBaz semantics and use the FooBarBaz protocol.

In short, every FooBarBaz URI is also an http URI, just as every http
URI is also a URI.

---

From: David Booth <[email-masked]>
Date: Wed, 09 Sep 2009 10:18:11 -0400

> other over the millennium.

History is riddled with discarded protocols, the proponents of which
naively thought that *their* protocol was so important that they needed
to invent a whole new naming scheme for it.  

If the Tachyon Protocol does become dominant in 2272 it will be a simple
matter to define a new URI scheme, so that Tachyon URIs can be shortened
henceforth.  (That's the *easy* part.  The hard part was convincing the
186,000 makers of nearly 4.2 trillion devices to *implement* the Tachyon
Protocol.)  But until that point, there is greater value in layering the
Tachyon Protocol on top of the dominant protocol, whether that happens
to be HTTP or the InterPlanetaryProtocol.

There is nothing intrinsically special about HTTP.  What matters is just
that it is so widely implemented.  That's what the network effect is all
about.

---

From: Kristof Zelechovski <[email-masked]>
Date: Wed, 9 Sep 2009 16:25:49 +0200

Caching does go around HTTP, it is built into HTTP by the specification.
Chris

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Toby Inkster
Sent: Wednesday, September 09, 2009 1:06 PM
To: Daniel R. Tobias
Cc: URI; [email-masked]; [email-masked]
Subject: Re: [Uri-review] ws: and wss: schemes

-- 
Toby A Inkster

---

From: Lisa Dusseault <[email-masked]>
Date: Fri, 11 Sep 2009 12:12:12 -0700

Didn't Vernor Vinge sketch something like this out as part of the
multi-solar-system-spanning network in the Zones of Thought Sci Fi
series?  His future programmers needed mind-altering substances (well
beyond caffeine, of course) to change any small part of the system and
they still caused chaos with every change.

Your addressing example provides additional concreteness and realism
to the scenario :)

Lisa

---

From: Infinity Linden <[email-masked]>
Date: Fri, 11 Sep 2009 12:51:16 -0700

hold on.. are we talking about using IPN ( http://www.ipnsig.org/ ) or
replacing it?

On Fri, Sep 11, 2009 at 12:12 PM, Lisa Dusseault
<[email-masked]> wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Lloyd Wood <[email-masked]>
Date: Fri, 11 Sep 2009 23:00:39 +0100

(this thread seems to have wandered somewhat from the subject line,
but hey, right in my area.)

Vinge's "Zones of Thought" spanned the entire galaxy as the ultimate
physical firewall with trusted zones and a DMZ. His "A Fire upon the
Deep" riffed on usenet conversations, with civilizations communicating,
and discussed aspects of network communication of group minds
(the "Tines").

"A Deepness in the Sky" comments on the invention of Unix, latency
as a significant factor in the communication of civilizations,
and the difficulties of consistent timekeeping in the space
environment. There's nothing on drugs for programming
(Focus is biological, leads to code only maintainable by the authors,
and sets up a set of jokes on life in cubes), but programming is  
respected
as an archaeological discipline.

HTTP has been considered for interplanetary networking - or rather,
the 'delay-tolerant networking' that the 'Interplanetary Internet'
(which was originally just a catchy phrase to get NASA to move its
space probes to some form of networking, and nothing to do with the
Internet at all) morphed into via scope creep. See
http://tools.ietf.org/html/draft-wood-dtnrg-http-dtn-delivery
This is just using HTTP for transfers - it's not a server/browser client
model, and doesn't require a lot of the framework we take for
granted terrestrially. HTTP ueber alles (multiple transports) is  
discussed in
http://www.ietf.org/proceedings/75/slides/tsvarea-1.pdf

But URIs in space? URIs are predicated on an authoritative DNS.
In deep space, DNS as it stands is likely not to be useful - lookups
take too long, contacts are scheduled and planned, late binding,
rather than early binding, is needed. And TCP's assumptions
and timers aren't a good fit with the environment. But that doesn't
rule out IP or other protocols in the suite.

>>> other over the millennium.

DTN work: http://info.ee.surrey.ac.uk/Personal/L.Wood/saratoga/

<http://info.surrey.ac.uk/Personal/L.Wood/><[email-masked]>

---

From: Kristof Zelechovski <[email-masked]>
Date: Sat, 12 Sep 2009 22:55:33 +0200

1. DNS can use UDP.
  2. There is no need to do interplanetary name lookup each time; the local
resolver can have all desired hosts preconfigured.
  3. Even if this fails, you can have numeric addresses in an URI authority,
so it does not preclude using URI if only numeric addresses are feasible.
(BTW, domain names are for people; appliances do not need them at all unless
their host has an unstable IP number.)
IMHO,
Chris

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Lloyd Wood
Sent: Saturday, September 12, 2009 12:01 AM
To: Infinity Linden
Cc: [email-masked]; Lloyd Wood; [email-masked]; Daniel R. Tobias; URI;
Lisa Dusseault
Subject: Re: [Uri-review] [hybi] ws: and wss: schemes

[snip]