Subject: Re: [urn] URN "semantics clarification", aka "syntax only" draft

From: Graham Klyne <[email-masked]>
Date: Fri, 20 Feb 2015 10:31:10 +0000

Hi John, all,

(bcc: [email-masked])

I've just taken a skim of 
http://tools.ietf.org/html/draft-ietf-urnbis-semantics-clarif-01, and I think 
the intent is fine but have a couple of questions or points of clarification.

My main concern is that URNs should be considered to be a kind of URI, and 
should be consistently usable in all contexts where a URI is usable (though, 
clearly, use in a context that requires dereferencing would be undefined in the 
same way as use of an unknown URI scheme would be).  Failure to satisfy this 
constraint would, IMO, force a bifurcation between URIs and URNs, which is 
something you say you want to avoid.

This means that not only should they conform to URI syntax, but also should 
conform to RFC3986 URI relative resolution rules.  For example, my code applies 
relative resolution without concern for the URI scheme used.  ("Traditional" 
URNs avoid syntactic constructions that include the possibility of relative 
references.)  (FWIW, I also happen to find that relative references are useful 
in naming contexts as well as retrieval contexts.)

Jumping to section 4, which appears to be the substantive part of this document, 
the main change I am seeing is a change of terminology from "path", "query" and 
"fragment" to "p-component", "q-component" and "f-component".  I take this to be 
a distancing of these elements in URNs from some commonly used conventions used 
with location-based URIs (aka URLs).  To the extent that this helps to reduce 
confusion, this seems fine to me (thought I also think it's an open question as 
to whether new terminology does reduce confusion).

...

You claim:

"In particular, the Generic URI Syntax specification goes beyond
    syntax to specify the meaning and interpretation of various fields,
    especially the "query" and "fragment" ones and the various syntax
    forms and interpretations it allows for <hier-part>. "

I disagree with this characterization with respect to "query" elements ("The 
query component contains non-hierarchical data that, along with data in the path 
component (Section 3.3), serves to identify a resource within the scope of the 
URI's scheme and naming authority (if any)." -- 
http://tools.ietf.org/html/rfc3986#section-3.4), though I accept it's commonly 
used to invoke dynamic search type operations.

For fragments, I accept that for a URN, where there is no dereferencing, the 
dependence on media type is difficult.  RDF 
[http://www.w3.org/TR/rdf11-concepts/] addresses this by simply treating the 
fragment as part of the overall name string, without further interpretation.

It then becomes an application design/implementation issue to avoid use of 
fragments of retrieved entities in ways that conflict with their use in names 
(cf. http://www.w3.org/TR/webarch/#fragid, 
http://www.w3.org/TR/webarch/#media-type-fragid).  In partyicular: "If no such 
representation exists, then the semantics of the fragment are considered unknown 
and, effectively, unconstrained." [ibid]

For paths, it's not clear to me whether you consider the relative resolution 
algorithm of RFC3986 to be part of the path semantics.  Per my comments above, I 
would be very concerned if you try to say that the resolution algorithm is not 
applicable to URNs, as that would mean that an application needs to know if its 
dealing with a URN or URI when performing otherwise scheme-independent processing.

...

In summary, assuming that you do not intend to exclude relative resolution from 
URN handling, the documents appears to me to be a "null specification", in that 
it does not actually exclude anything that is required by RFC3986.  I don't mean 
that its not useful - I think there are issues here that are usefully 
articulated and explained, but in the final analysis these clarifications should 
apply to *any* URI scheme used as a name rather than a locator.

And URI schemes are always allowed to layer their own semantics on top of 
RFC3986, so again the notion "removes URN semantics from the scope of RFC 3896" 
isn't really saying anything new that I can see.

So I think section 4 would more usefully be titled "Clarifications to RFC3986 as 
applied to URNs".

#g
--

By way of (partial) background to these comments, I'm currently working on a 
tool for small-scale research data management for which these are very real 
issues [https://github.com/gklyne/annalist].  I use URIs as both names and as 
locators, depending on context (not on the form of URI used).  Goals of my work 
include facilitation of data sharing, and provision of a path for data on 
researchers' desktops to library-managed data repositories - so both location 
and naming issues come in to play.

---

From: Nico Williams <[email-masked]>
Date: Fri, 20 Feb 2015 12:59:02 -0600

> (bcc: [email-masked])

Thanks, I'd not have seen this.

> http://tools.ietf.org/html/draft-ietf-urnbis-semantics-clarif-01,
> and I think the intent is fine but have a couple of questions or
> points of clarification.
> 
> My main concern is that URNs should be considered to be a kind of
> URI, and should be consistently usable in all contexts where a URI
> is usable (though, clearly, use in a context that requires
> dereferencing would be undefined in the same way as use of an
> unknown URI scheme would be).  Failure to satisfy this constraint
> would, IMO, force a bifurcation between URIs and URNs, which is

Rephrased as a question for proponents: what is the difference between a
URN that some code doesn't know what to do with, and a URI whose scheme
the same code doesn't know?

> This means that not only should they conform to URI syntax, but also
> should conform to RFC3986 URI relative resolution rules.  For
> example, my code applies relative resolution without concern for the
> URI scheme used.  ("Traditional" URNs avoid syntactic constructions
> that include the possibility of relative references.)  (FWIW, I also
> happen to find that relative references are useful in naming
> contexts as well as retrieval contexts.)

I see no reason that a URN couldn't name a resource with dynamic
behavior (think of a document with embedded code) or fragments (think of
an HTML document).

> Jumping to section 4, which appears to be the substantive part of
> [...]

Section 1 says

  This specification excludes URNs from those definitions of meaning and
  interpretation so that RFC 3986 applies to their syntax only.

where "those definitions of meaning and interpretation" refers to
"especially the "query" and "fragment" ones and the various syntax forms
and interpretations it allows for <hier-part>".

So... keep the syntax and throw away the semantics.  But why bother?

What harm results from keeping the semantics?  What is the benefit of
dropping the semantics?

I grant that in the absence of knowledge of the semantics of a URN's
query and fragment parts one can only treat the whole thing as an opaque
string, or, at best, drop the query and fragment (semantics!) to see if
the remainder is known or not.  E.g., a URN sans query & fragment might
appear in a registry, which might help determine whether the URN can be
dereferenced (in which case the query and fragment might gain meaning),
how it might be described to a user (in which case there may be no
meaning of the query & fragment, or the user might be the only one to
determine the meaning of the query & fragment), and so on.

What is the meaning of keeping the syntax but dropping the related
semantics?  The answer to this one seems to be: URNs become strings with
no meaning that must adhere to the RFC 3986 syntax pro-forma, because
too many bits of code will want to parse them, so don't break them.

Back to section 4, the consequences of this change are unclear.  The
change itself is unclear.  "This specification removes URN semantics
from the scope of RFC 3896."  But then we keep the syntax, with
different names for some elements.  What's the point?  What's the
benefit?

> In summary, assuming that you do not intend to exclude relative
> resolution from URN handling, the documents appears to me to be a
> "null specification", in that it does not actually exclude anything
> that is required by RFC3986.  I don't mean that its not useful - I

Exactly, that's my take as well, except that I don't see the use.  You
seem to see the use.  Can you explain it to me?

Incidentally, where in the WG's charter does this I-D fall?

> think there are issues here that are usefully articulated and
> explained, but in the final analysis these clarifications should

Well, yes (assuming there is a use to this).

> And URI schemes are always allowed to layer their own semantics on
> top of RFC3986, so again the notion "removes URN semantics from the
> scope of RFC 3896" isn't really saying anything new that I can see.

+1

Nico
--

---

From: Sean Leonard <[email-masked]>
Date: Fri, 20 Feb 2015 12:05:09 -0800

(also bcc’ing apps-discuss…although not exactly sure why it’s bcc’ed)

On Feb 20, 2015, at 2:31 AM, Graham Klyne <[email-masked]> wrote:

> I've just taken a skim of http://tools.ietf.org/html/draft-ietf-urnbis-semantics-clarif-01, and I think the intent is fine but have a couple of questions or points of clarification.
> 
> My main concern is that URNs should be considered to be a kind of URI, and should be consistently usable in all contexts where a URI is usable (though, clearly, use in a context that requires dereferencing would be undefined in the same way as use of an unknown URI scheme would be).  

+1. I agree with Graham, namely, that a URN should be considered a kind of URI. (On that particular sentence I agree +1000.) It’s not just that they are syntactically the same—a URN is a species of URI, namely, a URI scheme. Compare with HTTP URI being a URI. It’s worth pointing out that since RFC 3986 says that URIs are semantics-free (mostly), that part of the URI spec may be cited in support of the premise that just because a URN is a kind of URI, it doesn’t imply much else with regard to semantics.

> Failure to satisfy this constraint would, IMO, force a bifurcation between URIs and URNs, which is something you say you want to avoid.
> 
> This means that not only should they conform to URI syntax, but also should conform to RFC3986 URI relative resolution rules.  For example, my code applies relative resolution without concern for the URI scheme used.  ("Traditional" URNs avoid syntactic constructions that include the possibility of relative references.)  (FWIW, I also happen to find that relative references are useful in naming contexts as well as retrieval contexts.)

There are philosophical and practical problems with URI’s relative resolution rules as applied to urn: URIs, which have been covered extensively on the urn mailing list around October-November 2014, so best to check that. I am happy to elaborate on my understanding if that helps things but want to keep this e-mail short now.

> 
> Jumping to section 4, which appears to be the substantive part of this document, the main change I am seeing is a change of terminology from "path", "query" and "fragment" to "p-component", "q-component" and "f-component".  I take this to be a distancing of these elements in URNs from some commonly used conventions used with location-based URIs (aka URLs).  To the extent that this helps to reduce confusion, this seems fine to me (thought I also think it's an open question as to whether new terminology does reduce confusion).

I actually disagree with the terms p-component, q-component, and f-component on editorial grounds. The problem is that “path”, “query”, and “fragment” were thought to imply some things that URNBIS people disagreed with. However, calling them x-component is annoying to me…particularly because a fragment is a fragment is a fragment.

For discussion purposes in I-Ds I was okay with it, but in any final RFCs I request that we go back to calling them “path”, “query”, and “fragment”. Specifically, how about referring to them as the “path production of [RFC3986]” and similar? By calling it “path production” you are specifically referring to the RFC 3986 URI ABNF grammar, which is about syntax, and does not imply that URNs are “path-oriented” or the like.

Cheers,

Sean

---

From: John C Klensin <[email-masked]>
Date: Sat, 21 Feb 2015 08:23:59 -0500

--On Friday, 20 February, 2015 10:31 +0000 Graham Klyne
<[email-masked]> wrote:

> http://tools.ietf.org/html/draft-ietf-urnbis-semantics-clarif-
> 01, and I think the intent is fine but have a couple of
> questions or points of clarification.
> 
> My main concern is that URNs should be considered to be a kind
> of URI, and should be consistently usable in all contexts
> where a URI is usable (though, clearly, use in a context that
> requires dereferencing would be undefined in the same way as
> use of an unknown URI scheme would be).  Failure to satisfy
> this constraint would, IMO, force a bifurcation between URIs
> and URNs, which is something you say you want to avoid.
>...

Graham,

Independent of my personal opinions, 2141bis already contains
very explicit text allowing relative resolution (Section 5,
paragraph 2 of draft-ietf-urnbis-rfc2141bis-urn-09).  Do you
think that text needs to be repeated in
"draft-ietf-urnbis-semantics-clarif and, if so,

(i) Why?
(ii) Could you please propose text and where to put it?

> (FWIW, I also happen to find that
> relative references are useful in naming contexts as well as
> retrieval contexts.)

You might also want to suggest some text for 2141bis (or for the
various "revise 3986" efforts that are far out of scope for this
WG) that explains how the base reference from which relative
references are to be expanded is reliably obtained for URNs.  If
may be that I've just gotten bleary-eyed trying to read it, but
I see nothing in 3986 that defines using a URI from one scheme
to provide a base for a relative URI from another.

> Jumping to section 4, which appears to be the substantive part
> of this document, the main change I am seeing is a change of
> terminology from "path", "query" and "fragment" to
> "p-component", "q-component" and "f-component".  I take this
> to be a distancing of these elements in URNs from some
> commonly used conventions used with location-based URIs (aka
> URLs).  To the extent that this helps to reduce confusion,
> this seems fine to me (thought I also think it's an open
> question as to whether new terminology does reduce confusion).

That text was inserted for consistency with 2141bis.  It seems
to me that is the document you should be reading (and fairly
carefully, rather than skimming).

> I disagree with this characterization with respect to "query"
> elements ("The query component contains non-hierarchical data
> that, along with data in the path component (Section 3.3),
> serves to identify a resource within the scope of the URI's
> scheme and naming authority (if any)." --
> http://tools.ietf.org/html/rfc3986#section-3.4), though I
> accept it's commonly used to invoke dynamic search type
> operations.

I believe this is another element of the discussion about what
3986 { requires | says | implies | can be interpreted to say |
is read by some as saying }.  I think that making progress in
this WG involves getting past those discussions (moving them
into a forum where changes or interpretations of 3986 are
discussed) and I believe that is consistent with Andy's
consensus call on the separation of syntax from semantics. 

FWIW, because of the way the syntax is specified, part of what
causes the interpretations with which you disagree is the term
"non-hierarchical".  If that is a constraint on syntax, I can't
find it in 3986, especially its ABNF.   If it is a semantic
constraint, I don't see any reason why it should apply _as a
requirement_ to URNs that use q-components.

If you believe the text in draft-ietf-urnbis-rfc2141bis-urn-09
would be improved by saying something like "some have
interpreted the Generic URI Syntax specification as going beyond
syntax...", I'd be happy to make the change if others agree and
it would help get us unstuck.  If you think other changes are in
order, please suggest text rather than simply expressing
disagreement.

> For fragments, I accept that for a URN, where there is no
> dereferencing, the dependence on media type is difficult.  RDF
> [http://www.w3.org/TR/rdf11-concepts/] addresses this by
> simply treating the fragment as part of the overall name
> string, without further interpretation.

I believe RDF is out of scope for this WG, but I personally
believe that is non-conformant with fairly clear language in
3986.  Again, to the extent that interpretation is correct, it
is an issue between RDF and 3986 and relates to this WG only as
evidence that other schemes are finding 3986's specifications
unreasonably restrictive.   It is perhaps worth noting that this
WG went down the "if parts of 3986 seem inapplicable or
excessively constraining, just ignore them and specify something
else" years ago.  It was forced out of that position by people
complaining that various things that people wanted to do with/in
URNs were impossible because they were inconsistent with 3986,
starting the path that led to the "syntax only" approach.

> It then becomes an application design/implementation issue to
> avoid use of fragments of retrieved entities in ways that
> conflict with their use in names (cf.
> http://www.w3.org/TR/webarch/#fragid,
> http://www.w3.org/TR/webarch/#media-type-fragid).  In
> partyicular: "If no such representation exists, then the
> semantics of the fragment are considered unknown and,
> effectively, unconstrained." [ibid]

I look forward to the I-D, and perhaps a WG, that establishes
the Webarch work as superceding, or as a formal clarification
to, 3986.

> For paths, it's not clear to me whether you consider the
> relative resolution algorithm of RFC3986 to be part of the
> path semantics.  Per my comments above, I would be very
> concerned if you try to say that the resolution algorithm is
> not applicable to URNs, as that would mean that an application
> needs to know if its dealing with a URN or URI when performing
> otherwise scheme-independent processing.

See above.
 
> ...
> 
> In summary, assuming that you do not intend to exclude
> relative resolution from URN handling, the documents appears
> to me to be a "null specification", in that it does not
> actually exclude anything that is required by RFC3986.  I
> don't mean that its not useful - I think there are issues here
> that are usefully articulated and explained, but in the final
> analysis these clarifications should apply to *any* URI scheme
> used as a name rather than a locator.

See above.

> And URI schemes are always allowed to layer their own
> semantics on top of RFC3986, so again the notion "removes URN
> semantics from the scope of RFC 3896" isn't really saying
> anything new that I can see.
> 
> So I think section 4 would more usefully be titled
> "Clarifications to RFC3986 as applied to URNs".

But some people with different interpretations of 3986 than you
(or I, see below) have claimed that we could not couch things as
clarifications because they contracted 3986.

Again, FWIW, the position you describe in the two paragraphs
quoted above is exactly where I, personally, started and, to a
considerable extent, still am.   I believe that this
clarification draft should never have been needed and that the
year or 18 months the WG spent getting to it was a waste of
time.  Were I inclined toward conspiracy theories, I might even
suggest that the causes of our having to spend time on this lie
in theories that URNs (as distinct from assorted locators and
abstractions of locators) are fundamentally unnecessary and
undesirable and that the WG has, so far, survived efforts to
have it die the death of a thousand cuts (or ten thousand picked
nits).  But we are where were are and some formal clarification
of interpretation of and separation from the semantics (whether
examples, recommendations, or requirements) of 3986 seems to be
in order.

If my frustration with all of this and the time it has taken up
already is coming through, I apologize.

> By way of (partial) background to these comments, I'm
> currently working on a tool for small-scale research data
> management for which these are very real issues
> [https://github.com/gklyne/annalist].  I use URIs as both
> names and as locators, depending on context (not on the form
> of URI used).  Goals of my work include facilitation of data
> sharing, and provision of a path for data on researchers'
> desktops to library-managed data repositories - so both
> location and naming issues come in to play.

Good luck.

Please read 2141bis. 
  
    john

---

From: [email-masked] (Henry S. Thompson)
Date: Mon, 23 Feb 2015 10:17:27 +0000

> to provide a base for a relative URI from another.

Point of clarification?  "a relative URI from another [scheme]"?  Not
sure what you mean here.  There are, IIUC, three strings involved in
"reference resolution", as defined in 3986 [1].  Only two of these are
(usually/necessarily) URIs:

 1) The base URI;
 2) A string which is to be resolved against (1);
 3) The result URI.

None of these is "a relative URI".  There is no production in the
spec. named ...relative-URI. 3986 [2] calls it a "relative reference".
Having said that, I too often use the phrase "relative URI", to mean
(2), but it's actually not sanctioned by the spec., and often gets us
in to trouble.

If (2) begins with a scheme, the result just _is_ (2) (see the
pseudo-code for reference resolution in 3986 [3]); If (2) does not
begin with a scheme, it's not "from another scheme".

So in neither case can I see what you're looking for and not
finding. . .  Please clarify (an example would probably help).

ht

[1] https://tools.ietf.org/html/rfc3986#section-5
[2] https://tools.ietf.org/html/rfc3986#section-4
[3] https://tools.ietf.org/html/rfc3986#section-5.2.2
-- 
       Henry S. Thompson, School of Informatics, University of Edinburgh
      10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440
                Fax: (44) 131 650-4587, e-mail: [email-masked]
                       URL: http://www.ltg.ed.ac.uk/~ht/
 [mail from me _always_ has a .sig like this -- mail without it is forged spam]

---

From: Graham Klyne <[email-masked]>
Date: Mon, 23 Feb 2015 11:16:55 +0000

Oops, good catch!  I missed the "from another" when making my previous response. :(

RFC3986 mentions a non-recommended extension for interpreting relative 
references with scheme names:

[[
Some parsers allow the scheme name to be present in a relative
    reference if it is the same as the base URI scheme.  This is
    considered to be a loophole in prior specifications of partial URI
    [RFC1630].  Its use should be avoided but is allowed for backward
    compatibility.
]]
-- https://tools.ietf.org/html/rfc3986#section-5.4.2

But even here, the schemes must be the same, so there's no issue of the relative 
resolution going across schemes.

<aside>
This is noted as an issue from earlier specs.  It seems to me that the URN 
effort has been going, off-and-on, for a long time.  I suspect that many of the 
misunderstandings attributed to RFC3906 may actually be hang-overs from previous 
versions of the spec.  A key role for RFC 3986 was to clean up a number of these 
complicating issues from earlier versions of the spec (in part by drawing back 
on specifying semantics of URIs and focusing on the syntax?).
</aside>

#g
--