Subject: [urn] Why bother with f-components (fragments) and/or p-components ?

From: John C Klensin <[email-masked]>
Date: Mon, 13 Apr 2015 16:10:56 -0400

Hi.

Those who have been following the apps-discuss list and the Apps
Area WG has seen a long, difficult, and sometimes unpleasant
thread about what a URN specification (or any other
specification for a particular URI scheme) is allowed to say
about fragments.  The discussion appears to extend to whether
fragments can be disallowed at all by a scheme (if they cannot,
2141 has problems quite independent of what we might do), to
whether a scheme can constrain fragment syntax in any way, and
to whether scheme-specific or namespace-specific restrictions or
semantic [1] rules are allowed.

Similar issues arise with p-components.   If a p-component
(i.e., introduced by a "/", at least before a q-component or
f-component) appears in a URN, it is claimed to be impossible
[2] to avoid relative resolution, something that might cause
havoc or other damage with URNs.

An obvious question is why we don't just disallow one or both
and move on.

For me, the answer is tied up with the question I've been asking
about the difference between instructions or qualifiers that are
part of the URN (see below) and instructions or qualifiers that
are intended to be passed to whatever the URN points to or
resolves to.   If no one else sees that as an issue, or believes
that we can reasonable handle it on a per-namespace basis (that
fragment discussion definitely would complicate handling those
on a per-namespace basis), please explain why and I'd drop this.
If others are concerned and believe we should handle that issue
using syntax, then I believe we need to consider three cases.
(I don't see any alternative to distinguishing by syntax if we
aren't going to make the distinction per-namespace, but maybe
I'm missing something):

I'm going to use the term "qualifier" below to identify things
that are not inherently part of the NSS (using the syntax in
2141bis-11) but that are part of the <namestring>.

 Case 1: The qualifier is part of the URN to the extent
	that it is included in comparisons and affects
	persistence.  As part of the URN, it is expected to
	affect or provide information to a URN processor or
	resolver.
	
 Case 2: The qualifier is part of the URN in the sense
	that if affects or provides information to a URN
	processor or resolver, but does not participate in
	equality comparisons.    Information about locale (so a
	URN resolver could find the most convenient object or a
	pointer to it) would presumably fall into this category.
	I think distinctions between object and various types of
	metadata would too, but could be talked out of that.
	
 Case 3: The qualifier is not part of the URN but
	contains information or specifications that should be
	passed on to the URN's target if that is meaningful.
	The qualifier is probably not meaningful if there is no
	resolution and no target.

The WG hasn't seemed to be interested in forcing q-components
into a name-value form or other scheme-wide syntax.  It has
seemed still less interested in reserved some specific keywords
for use in that syntax.  But, it we don't do that, we seem to
have three significant cases and three components types.  The
implications seem obvious, at least to me.  If they are, then we
need to sort p-components and f-components out, rather than
getting rid of them.

If anyone sees problems with the analysis above or considers one
or more of the cases uninteresting, it would be helpful (at
least for me) to hear about it.

    john

[1] At least by my definition of "semantics".

[2] I am not disputing that claim, I just have problems with
"impossible".

---

From: Keith Moore <[email-masked]>
Date: Tue, 14 Apr 2015 12:05:04 -0400

> 	resolution and no target.

I think there are actually four categories of "qualifier":

1. Qualifiers that potentially represent portions of resources, and/or 
relationships with other resources named with URNs, that should be 
included in comparisons and considered relevant for persistence.

2. Qualifiers that identify portions of resources that should not be 
included in comparisons or when considering persistence.

3. Qualifiers that are transmitted to resources (when applicable) for 
interpretation by the resources.   These should not be included in 
comparisons or when considering persistence.

4. Qualifiers that affect resolution of the URN, and which should be 
transmitted to the resolution service if one is used, either to request 
a specific kind of resolution service, or to narrow down between 
multiple versions and/or representations of the content.

So even without considering 3986 relative reference issues, it's a bit 
difficult to shoehorn 4 kinds of qualifiers into the three kinds of 
modifiers available in 3986 syntax.

There are at least two additional sets of considerations that further 
complicate things:

a) if 3986-style relative references in a resource are interpreted 
relative to URNs, that constrains how f-components, p-components, and 
q-components can be used - particularly when the same resources are also 
reachable via ordinary URLs that already assign meanings to these 
qualifiers.

b) there is considerable mindshare around what '#', '/', and '?' mean 
when they appear in URIs, based on widespread experience with HTTP[S] 
URLs, and it will be confusing if URNs adopt meanings for these that 
conflict with those used in HTTP[S].

Keith

---

From: John C Klensin <[email-masked]>
Date: Tue, 14 Apr 2015 14:06:16 -0400

Keith,

This is very helpful.  I was trying to keep the count to three
precisely because what was obviously available in syntax.  I
interpret that narrowly and, e.g., assume that "role" implies
"semantics" unless it is an absolute lexical property.  As a
specific example, it is a statement about syntax to say that "#"
has a role, which is to delimit and denote the beginning of a
syntax element called a <fragment>.  Statements about how
<fragment> is used or applied or about the relationship between
the syntax element specified in a production as <fragment> and
the English word "fragment" are all semantics.

More inline and below.

--On Tuesday, April 14, 2015 12:05 -0400 Keith Moore
<[email-masked]> wrote:

> 1. Qualifiers that potentially represent portions of
> resources, and/or relationships with other resources named
> with URNs, that should be included in comparisons and
> considered relevant for persistence.
> 
> 2. Qualifiers that identify portions of resources that should
> not be included in comparisons or when considering persistence.
> 
> 3. Qualifiers that are transmitted to resources (when
> applicable) for interpretation by the resources.   These
> should not be included in comparisons or when considering
> persistence.
> 
> 4. Qualifiers that affect resolution of the URN, and which
> should be transmitted to the resolution service if one is
> used, either to request a specific kind of resolution service,
> or to narrow down between multiple versions and/or
> representations of the content.
> 
> So even without considering 3986 relative reference issues,
> it's a bit difficult to shoehorn 4 kinds of qualifiers into
> the three kinds of modifiers available in 3986 syntax.

However, as far as I can tell, there have only been four types
of suggestions on the WG list.  If you (or others) know of, or
have, others, this would be a good time to speak up.  In no
particular order:

(i) Force things into three categories of qualifiers and bind
each category to a "component" type, using as close a match to
the 3986 (and HTTP) assumptions about how those components are
used as feasible.

(ii) Declare everything to be per-namespace that doesn't
actually violate the 3986 syntax constraints (again, using
"syntax" narrowly) and move on.  I think I understand your
concerns about that and share at least some of them.

(iii) Forget p-components and f-components as posing the most
issues for generic URI processing and user expectations and
impose urn-scheme-specific conventions on q-components to allow
distinguishing among my three cases or your four.  There are a
number of possible ways to subdivide the q-component that way.
The two most obvious, both of which we have discussed on the
list, involving forcing name value pairs with some reserved
keywords and using some delimiters in very specific ways.  One
obvious bit of bad news about this approach is that, if anything
in your first category is allowed outside the NSS (as defined in
2141bis-11) then it will be necessary to parse and subdivide the
q-component in order to make an equivalence (or persistence)
determination.

(iv) Give it up and stick with 2141, i.e., urn namestrings
consist only of "urn:" plus NID plus NSS; no p-, q-, or
f-components; and a prohibition on any of "/", "?", or "#"
appearing within the NSS except in %-encoded form.  In part
because such things are in use already, we can be quite
confident that, whether the result was a conflicting standard or
not, the IETF's advise on that subject would be ignored or
explicitly contradicted.

Again, do you see other options and, if not, which one do you
consider the least bad?

> There are at least two additional sets of considerations that
> further complicate things:
> 
> a) if 3986-style relative references in a resource are
> interpreted relative to URNs, that constrains how
> f-components, p-components, and q-components can be used -
> particularly when the same resources are also reachable via
> ordinary URLs that already assign meanings to these qualifiers.

Only if there is an established mapping relationship between a
URN and some URL type, such as the frequently-cited relationship
between, e.g.,
    urn:ietf:rfc:1373    and
    https://datatracker.ietf.org/doc/rfc1737/
The latter, interestingly, points to a metadata page, not the
document-resource.  If the relationship were to 
    http://www.rfc-editor.org/rfc/rfc1737.txt       or
    ftp://ftp.rfc-editor.org/in-notes/rfc1737.txt
it would identify the document.

See below.

> b) there is considerable mindshare around what '#', '/', and
> '?' mean when they appear in URIs, based on widespread
> experience with HTTP[S] URLs, and it will be confusing if URNs
> adopt meanings for these that conflict with those used in
> HTTP[S].

I think we can go one of two ways here.  One is to say "URNs are
not URLs, much less HTTP[s] URLs" and that, for a lot of the
purposes for which the distinction is useful, our long-ago
decisions to have a unified syntax and theory of URIs still
requires that, for many URN purposes (including some of those
contemplated in 1737 and the discussions surrounding it), users
are going to need to be familiar with the difference.  The other
is to say "URNs really are URLs, just a different shorthand form
from relative references.  The latter actually allows a
universal resolver in the form of something that would maintain
a per-NID that provides rules (perhaps in regex form) for
getting from any given NID-NSS combination to a corresponding
URL, presumably with any qualifying information just passed into
the URL.  I don't see a third choice.  Do you?

<micro-rant> Many of these discussions are convincing me that we
made a fundamentally incorrect design decision when we decided
that URLs and URNs were just too subspecies of a fundamental
"URI" concepts and that they should share the same syntax and
maybe most semantics.  In retrospect, we would have been better
off, especially from human interface design perspectives, had we
made the two sufficiently different in naming and/or syntax that
the immediate user inference would be "these are two different
things" rather than "these things are nearly the same and I can
extrapolate from what I know about one to the other one".  It
would also mean that an application design would need to decide
whether to accept URLs, URNs, or both and that discussions about
putting URNs into URL (or URI) slots would have a lot of
characteristics in common with pounding square pegs into round
holes.  I see no way to go back and change things at this point,
but many of these discussions, the difficulties that are being
identified, and the proposed solutions are feeling more and more
like creating ever more complex epicycles around a URL-centered
universe.
</micro-rant>

   john

---

From: Keith Moore <[email-masked]>
Date: Tue, 14 Apr 2015 22:50:07 -0400

> Again, do you see other options and, if not, which one do you

I'm looking into a variant of (iii) in which a q-component (while still 
adhering to 3986 syntax) can itself be parsed into a portion which 
affects resolution and a portion which can get transmitted to the 
resource for interpretation by the resource.   f-components can still be 
used to refer to fragments of resource content.    Whether p-components 
can be used depends in part on other factors.   (If they are used by a 
namespace then the namespace has to impose certain constraints in 
resources with NSSs that have p-components, and I think those 
constraints might be tricky to define even if they are easy for some 
namespaces to conform to).

> it would identify the document.

I think the two good options for handling of relative references are:

(a) permit relative references to base URNs, but impose constraints on 
URNs that contain p-components
(b) don't permit relative references to base URNs - if a URN is used to 
access content that contains relative references, a base URL must either 
appear in the content itself or be obtained during the resolution 
process.   Then '/'s could appear in URNs, but there's no significance 
to them.   (aside: I think it would be nice if DOIs could map cleanly 
into URN space without %-encoding.)

I'm leaning to the latter at the moment, because ultimately I think it 
is cleaner to avoid having 3986 rules for relative references be used to 
manipulate URNs.  You really don't want URNs (at least the assigned 
portions of URNs) to have any more exposed structure than absolutely 
necessary.   Except for being able to recognize them as URNs by their 
prefixes, and to identify the namespace (for purposes of finding a 
resolution service), most software should be able to treat URNs as 
opaque tokens.

Option (b) provides a "established mapping relationship":   Some of the 
metadata for a particular URN could specify which URL(s) at which the 
named resource's content can be accessed (when applicable), and the 
URL(s) chosen should be actually point to the content rather than a 
metadata page.   The "established mapping relationship" would be 
provided by the resolution services.

It's possible to consider a hybrid between (a) and (b): path-relative 
references can appear in URNs that contain p-components as long as those 
namespaces adhere to certain constraints.   But I think this would 
mostly serve to make URN behavior more confusing.

> the URL.  I don't see a third choice.  Do you?

IMO: URNs are not URLs.   By design, URNs have never been URLs. The 
reason that the term URI was defined was to have a name for a context in 
which both URLs and URNs could potentially appear. Yes, users of URNs 
(i.e. at least those users that put URNs into content or otherwise 
compose URNs with qualifiers) are expected to be familiar with the 
differences, which is precisely why a common urn: prefix is used as an 
umbrella or superclass for multiple namespaces - it's to provide a 
visible indication that this kind of URI, whatever its namespace, is 
different than a URL, and operates under somewhat different rules.   If 
every URN-like object had its own unique URI prefix, users would have to 
be able to recognize each kind of URN-like object and be aware of the 
differences not only between those objects and URLs, but between the 
different kinds of URN-like objects.   That, I suspect, would make most 
of those URN-like objects less viable as long-term identifiers.

(And yet, people who insist on the world being flat can insist that 
indeed they're all URLs, define a few terms in slightly different ways, 
use slightly different language, and end up with a mostly 
self-consistent view.   From a certain perspective, neither view is 
"right", and it's just a battle over language.   But of course language 
affects perception, and part of the reason for the lingering dispute is 
a desire from multiple parties to dictate how these names are 
perceived.  And that's unfortunate, because the lingering conflict over 
the language is generally confusing to those who aren't immersed in this 
world.)

> maybe most semantics.

I don't see those two decisions as having been inherently coupled. IMO, 
the URI concept is extremely useful.   But the only syntax features that 
URNs and URLs needed to have in common was (1) for URNs to have a scheme 
prefix that looked like a URL prefix and could be distinguished from 
other URL prefixes, and (2) for the two kinds of URIs to be composed 
from the same set of permissible characters, so that a scanner that was 
attempting to extract a URI from surrounding text would know when to 
terminate that scan - when it had found the last character of the URI.

But I recognize that there's a huge amount of momentum, mindshare, 
political investment, etc. around 3986 syntax for URNs, and that maybe 
it's not worth trying to find the political capital needed to revise 
3986 if we can somehow provide what URNs need within that framework.   
And I think the answer to whether we can do that is "maybe".

Keith

---

From: John C Klensin <[email-masked]>
Date: Wed, 15 Apr 2015 08:34:51 -0400

Keith,

Very briefly, first because I have no more cycles available
today and second because I'd like to hear from others but (and
trimming a lot because people should read your original
message).   I hope your note and these comments on it will
stimulate conversation on the list or on Friday's call.

Inline.

--On Tuesday, April 14, 2015 22:50 -0400 Keith Moore
<[email-masked]> wrote:

>...
>> impose urn-scheme-specific conventions on q-components to
>> allow distinguishing among my three cases or your four.
>...

> I'm looking into a variant of (iii) in which a q-component
> (while still adhering to 3986 syntax) can itself be parsed
> into a portion which affects resolution and a portion which
> can get transmitted to the resource for interpretation by the
> resource.   f-components can still be used to refer to
> fragments of resource content.    Whether p-components can be
> used depends in part on other factors.   (If they are used by
>...

Fwiw, that would work for me if the details can be worked out.
My recollection is that something slightly similar was proposed
a couple of years ago and the WG wasn't enthused but perhaps now
that the other options are more clear the answer will be
different and less ambiguous.

>...
>...
> (b) don't permit relative references to base URNs - if a URN
> is used to access content that contains relative references, a
> base URL must either appear in the content itself or be
> obtained during the resolution process.   Then '/'s could
> appear in URNs, but there's no significance to them.   (aside:
> I think it would be nice if DOIs could map cleanly into URN
> space without %-encoding.)
> 
> I'm leaning to the latter at the moment, because ultimately I
> think it is cleaner to avoid having 3986 rules for relative
> references be used to manipulate URNs.

That is consistent with my instinct if we can sort the details
out.

>...

>...
>> I think we can go one of two ways here.  One is to say "URNs
>> are not URLs, much less HTTP[s] URLs" and that, for a lot of
>> the purposes for which the distinction is useful, our long-ago
>> contemplated in 1737 and the discussions surrounding it),
>> users are going to need to be familiar with the difference.
>> The other is to say "URNs really are URLs, just a different
>> shorthand form from relative references.  The latter actually
>> allows a universal resolver in the form of something that
>> would maintain a per-NID that provides rules (perhaps in
>> regex form) for getting from any given NID-NSS combination to
>> a corresponding URL, presumably with any qualifying
>> information just passed into the URL.  I don't see a third
>> choice.  Do you?
> 
> IMO: URNs are not URLs.   By design, URNs have never been
> URLs. The reason that the term URI was defined was to have a
> name for a context in which both URLs and URNs could
> potentially appear.

I understood all of that.  The difficulty is that, to a very
considerable degree, while 3986 mentions URNs, it is becoming
more and more obvious (at least to me) that it was designed
around a great many URL assumptions without making adequate
allowances for the ways in which URNs ought to be different.
The "URNs are not URIs" terminology was wrong (as well as
inflammatory), but the underlying idea that URNs are distinct
from URLs and that a lot of 3986, despite its title is about
URLs and not URNs may still be correct.  I continue to hope that
the syntax vs. semantics distinction will be sufficient to let
URN work progress but, if one reads some of the comments on the
Apps-Discuss list, that distinction is either being dismissed or
attempts are made to cover it over (see my comments there about
the use of the term "role").

I have to admit that I've always had trouble with 3986 and the
way it and its predecessors are written -- my model of design,
parsing structure, and modularization is sufficiently different
that I've been inclined to give the authors of 3986 the benefit
of the doubt rather than going through the document and trying
to translate it into my frame of reference.  However, when I've
tried to do the latter in the last year, what I've encountered
is many problems and inconsistencies.  I continue to hope those
can be kept out of the URN debates, but they are, for me, fairly
intrusive.

> Yes, users of URNs (i.e. at least those
> users that put URNs into content or otherwise compose URNs
> with qualifiers) are expected to be familiar with the
> differences, which is precisely why a common urn: prefix is
> used as an umbrella or superclass for multiple namespaces -
> it's to provide a visible indication that this kind of URI,
> whatever its namespace, is different than a URL, and operates
> under somewhat different rules.

In that sense, our difficulty with the URN - URI relationship is
that a few people are quite forcefully telling us that URNs
really don't get to be different from URLs, particularly wrt
different rules.

>...
> (And yet, people who insist on the world being flat can insist
> that indeed they're all URLs, define a few terms in slightly
> different ways, use slightly different language, and end up
> with a mostly self-consistent view.

Yes, I think that is what we have been seeing.  I think the WG
is going to need to be quite forceful to get past that, rather
than having it drive us around in circles.

>...
> But I recognize that there's a huge amount of momentum,
> mindshare, political investment, etc. around 3986 syntax for
> URNs, and that maybe it's not worth trying to find the
> political capital needed to revise 3986 if we can somehow
> provide what URNs need within that framework.   And I think
> the answer to whether we can do that is "maybe".

Yes.  And I think that is the core of the issue.  From my point
of view, there are no URN issues that we cannot work out in a
reasonable way, paying as much attention to 3986 as is feasible
given the needs of URNs (but no more), as long as we don't get
paralyzed by "this has to be more like a URL because there are a
lot of URLs" assertions and requirements.

    john

---

From: Keith Moore <[email-masked]>
Date: Wed, 15 Apr 2015 09:50:20 -0400

> allowances for the ways in which URNs ought to be different.

I certainly agree.

> the use of the term "role").

I haven't had the cycles to follow apps-discuss lately, but might find 
some time after today.
> really don't get to be different from URLs, particularly wrt

I'm not sure who those "few people" are.   But we're not supposed to 
have kings in IETF, and we're supposed to make decisions based on 
technical soundness rather than wishful thinking.

Keith

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Wed, 15 Apr 2015 19:51:41 -0600

Hi Keith, I concur with John that this is very helpful input to the 
discussion. Comments inline.

> I think there are actually four categories of "qualifier":

I would love to see examples of each of these, by the way.

> included in comparisons and considered relevant for persistence.

I would especially like to see examples of #1, because I am not sure 
that they exist.

> multiple versions and/or representations of the content.

I continue to struggle with two things (or perhaps misconceptions in my 
own mind) about our discussions:

(a) Not all URNs need to be or can be resolved, and it seems to me that 
even URNs that can be resolved are primarily used to identify resources 
and not to resolve resources (which is why we have URLs, after all).

(b) We don't know enough about the running code of modern, actual URN 
resolution services to helpfully specify what kind of syntax they might 
require.

Regarding (a), we have (i) a large number of URN namespaces that are 
used by other SDOs or quasi-SDOs to assign identifiers for uses like XML 
namespaces. I strongly encourage the WG participants to spend some time 
looking through the list of formal namespaces:

http://www.iana.org/assignments/urn-namespaces/urn-namespaces.xhtml

As far as I can see, most of the URNs that are assigned in those 
namespaces aren't intended to be resolved at all. They are, in the 
perhaps less than perfect terminology of 2141bis, "abstract 
designators". In these cases, there is simply no need to worry about 
qualifiers 3 & 4.

We also have (ii) a few URN namespaces (albeit namespaces with a huge 
number of assigned URNs) where resolution is at least possible. These 
are namespaces such as those for ISBNs, ISSNs, and NBNs. However, even 
here I think (perhaps without sufficient basis) that these URNs are 
assigned primarily for the purpose of identification ("this URN 
identifies that work") and not for the purpose of resolution ("this URN 
enables you to find a copy of that work on a shelf in this building").

When I combine the (to me) secondary importance of resolution for URNs 
with the fact that we don't know much about the running code of URN 
resolution services, I find myself at a loss to define the syntax, and 
certainly the semantics, of qualifiers 3 & 4 with any degree of 
precision. That is why I think 2141bis needs to be somewhat general 
about the syntax of those qualifiers (other than making sure that syntax 
is consistent with the URI syntax).

As an example, we might need to wait until we know a lot more about 
modern URN resolution systems before we can define name-value pairs for 
q-components.

And, to be clear, for the limited purposes of 2141bis I think that is 
fine. As I see it, the alternative is that this WG would postpone its 
delivery of updates to RFC 2141 much longer than it already has, with 
very little to show for that further delay and a great risk of failure 
or fragmentation.

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: "Hakala, Juha E" <[email-masked]>
Date: Thu, 16 Apr 2015 06:57:21 +0000

Hello,

> -----Original Message-----
> From: urn [mailto:[email-masked]] On Behalf Of Keith Moore
> Sent: 15. huhtikuuta 2015 16:50
> To: [email-masked]
> Subject: Re: [urn] Why bother with f-components (fragments) and/or p-
> components ?
> 
> >> URLs. The reason that the term URI was defined was to have a name for
> >> a context in which both URLs and URNs could potentially appear.
> > considerable degree, while 3986 mentions URNs, it is becoming more and
> > more obvious (at least to me) that it was designed around a great many
> > URL assumptions without making adequate allowances for the ways in
> > which URNs ought to be different.
> 
> I certainly agree.

+ 1. 

>From identifier community point of view, URNs - in order to be useful for us - must accommodate existing standard identifier systems, but there is no such need for URLs since they are just locators, not names (identifiers).

This means - for instance - that in URNs fragment and query cannot have a role in identification, since that would change the scope of  many existing standard identifiers beyond recognition. All of a sudden ISBNs could be used to identify (physical) fragments of ebooks, or even - with query - bibliographic records describing books. Moreover, anyone would be able to assign ISBNs for book fragments at any time, which would make the administration of the system complicated.

Whenever URI syntax seems to intervene with the existing identifier assignment practices URNBIS is putting a cart in front of a horse. It is the URN syntax which should be adapted, not existing identifier assignment practices.   

Libraries, publishers and other organizations in the book trade have traditionally separated location and identification because it is practical to do so. And we have carried on with this practice also in the Internet - scientific publishers and research libraries are among major users of persistent identifiers such as URN and DOI. It is unfortunate that URI mixes location with naming since that gives some people an opportunity to claim that URLs can in theory be just as persistent than URNs. In practice, that is not the case. A thorough research article published last December indicates that URLs are simply not cool enough (see http://dx.doi.org/10.1371/journal.pone.0115253). My conclusion from this is that something else (both technically and from administrative point of view) should be used to identify resources which should be preserved indefinitely such as (scientific) publications. But I am also sure that these research results will have no impact on the actions of cool URI supporters.     
 
> > inflammatory), but the underlying idea that URNs are distinct from
> > URLs and that a lot of 3986, despite its title is about URLs and not
> > URNs may still be correct.  I continue to hope that the syntax vs.
> > semantics distinction will be sufficient to let URN work progress but,
> > if one reads some of the comments on the Apps-Discuss list, that
> > distinction is either being dismissed or attempts are made to cover it
> > over (see my comments there about the use of the term "role").

Those who dismiss the difference between URLs and URNs should check why most organizations responsible of long term preservation of digital resources are using persistent identifiers. It is of course possible that we are all wrong ;-).  

> I haven't had the cycles to follow apps-discuss lately, but might find some
> time after today.
> >> Yes, users of URNs (i.e. at least those users that put URNs into
> >> content or otherwise compose URNs with qualifiers) are expected to be
> >> familiar with the differences, which is precisely why a common urn:
> >> prefix is used as an umbrella or superclass for multiple namespaces -
> >> it's to provide a visible indication that this kind of URI, whatever
> >> its namespace, is different than a URL, and operates under somewhat
> >> different rules.
> > In that sense, our difficulty with the URN - URI relationship is that
> > a few people are quite forcefully telling us that URNs really don't
> > get to be different from URLs, particularly wrt different rules.
> 
> have kings in IETF, and we're supposed to make decisions based on technical
> soundness rather than wishful thinking.

I am not sure who these "few people" are either. But keep in mind that there are organizations - most of which not involved with IETF - which have assigned tens of millions URNs already and will assign hundreds of millions of them in the future. We expect URNBIS/IETF to deliver results which will enable us to enhance our existing URN-based services.  

And I wish this technical soundness -criteria were applied to the so called cool URIs as well. If they are not working in practice, what is IETF going to do?

Juha

> 
> Keith
> 
> _______________________________________________
> urn mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/urn

---

From: Keith Moore <[email-masked]>
Date: Thu, 16 Apr 2015 03:18:12 -0400

> I would especially like to see examples of #1, because I am not sure 

When I first wrote up that list, I wasn't sure about #1 myself.   I 
almost stated that #1 wasn't very useful, then I realized that that type 
of qualifier probably is needed after all, or at least that it wasn't 
safe to say that they're not needed.

One reason for having such a qualifier is this: you'd really like to be 
able to have persistent references to portions of a resource. Fragments, 
as currently defined and implemented, don't serve that purpose.   (I 
could explain why but it would be a long side discussion, so will leave 
it to a separate message if the explanation is needed).   So 
*urn:example:/a* might refer to the entirety of a resource and 
*urn:example:/a/b* might refer to a portion of that resource, and 
*urn:example:/a/b/c* refer to a (presumably) still smaller portion - and 
all of these URNs would be managed by the namespace and expected to 
remain persistent over time (including their relationships to one 
another, if any of them used relative references).

>> included in comparisons or when considering persistence.

I think this ends up being the fragments to which we're already 
accustomed.   IMO, the ability to specify a fragment of a URN-named 
resource is still useful, it's just that the combination of a URN and a 
fragment name (or if you prefer, f-component) no longer has any 
assurance of persistence.   So *urn:example:a* has an assurance of 
persistence but *urn:example:a#b* does not.   I think this will confuse 
people for a while, because they'll have to learn to mentally parse URNs 
differently than URLs.   But overall this might be the best path 
forward.   It would be difficult for us to exclude use of fragments with 
URNs, or to define fragment behave differently with URNs than with URLs, 
without breaking relative references.

So I am thinking that the meaning of the fragment should still be the 
same for both URNs and URLs.   Ideally if *urn:example:a* resolves to 
*http://example.com/b*, and c is a valid fragment of that resource, then 
*urn:example:a#c* and *http://example.com/b#c *should refer to the same 
portion of that resource's content.   The alternative of letting 
fragments work differently for URNs vs. URLs strikes me as /very/ 
unlikely to work well, especially if 3986-style relative reference 
processing can be applied to URNs.
>> comparisons or when considering persistence.

This is analogous to query strings in HTTP.   IMO it's vital to let URNs 
be able to bundle the name of the base resource with some input to be 
transmitted to the resource.   So similar to fragments, I think that if 
*urn:example:a* resolves to *http://example.com/b*, then 
*urn:example:a?c* should ultimately result in a GET of 
*http://example.com/b?c* .   Again, *urn:example:a* is persistent while 
*urn:example:a?c* has no assurance of persistence.
>> multiple versions and/or representations of the content.

For the sake of an example of how this might be used, let's say that 
such a qualifier must begin with "??", that it appears before 
q-component, and the value of that qualifier is not allowed to contain 
"?" without it being %-encoded.  I'll tentatively call this an 
"r-component" ("r" for resolution)

  * *urn:example:a* would refer to the resource without specifying what
    to do with the URN.  If such a URN appeared in a link in a resource
    being viewed in a browser, and a user clicked on that link, the
    browser might try to present the content to the user.   Or failing
    that, it might simply try to get a description of the resource and
    display that.   If OTOH, such a URN appeared in an IMG tag the
    browser might give up if it couldn't get the content - presumably
    the description of the resource isn't an image.
  * *urn:example:a??request=content* would tell the resolution service
    (and perhaps also the client) that the reference was specifically
    intended to be for the resource's content, and not to locations, or
    any kind of description or metadata of the resource
  * *urn:example:a??request=metadata;format=dc-rdf *might request that
    the resolution service return a description of the resource in RDF
    format and the Dublin Core schema.
  * *urn:example:a??request=locations* could request locations of the
    resource
  * *urn:example:a??request=content;version=first* could request the
    earliest version of the resource

Basically these qualifiers would serve as a kind of query to the 
resolution service (as opposed to a query to be sent to the resource).  
It's like searching for a resource in a library's catalog and specifying 
some details that narrow the results along with the query.

As far as I can tell, this kind of capability is vitally important, 
because almost every time I see any functionality of a resolution 
service described, there's at least one or two examples of how one might 
include such queries with a URN.   IMO, if URNs don't provide a standard 
way to do it, URN resolution services will overload "?" and the result 
will be that (a) you can't specify a URN that includes a query to be 
sent to the resource itself (at least not in any uniform way) and (b) 
different schemes will behave differently and it will be difficult to 
write clients that treat Uniform Resource Names in any kind of uniform 
fashion.

Of course the above are just examples of what could be done, rather than 
a well thought-out proposal.   I would also suggest that namespaces 
should be able to define their own requests and parameters independently 
of others.   So for the "ietf" namespace, IETF should be able to support 
things like *
urn:ietf:rfc:XXXX??request=ietf.version-history;option=include-I-Ds
*
>
> I continue to struggle with two things (or perhaps misconceptions in 
> my own mind) about our discussions:
>
> (a) Not all URNs need to be or can be resolved, and it seems to me 
> that even URNs that can be resolved are primarily used to identify 
> resources and not to resolve resources (which is why we have URLs, 
> after all).
>
> resolution services to helpfully specify what kind of syntax they 
> might require.
>
> used by other SDOs or quasi-SDOs to assign identifiers for uses like 
> XML namespaces. I strongly encourage the WG participants to spend some 
> time looking through the list of formal namespaces:
>

There's been a lot of misunderstanding and misuse of URNs.   URNs were 
intended to name resources, not to merely be unique strings. I don't 
mind if people sometimes use URNs as merely unique strings, as I don't 
think it does much harm - except that some people have gotten the idea 
that this is the primary purpose of URNs.  It's not.  I don't even 
recall that particular use case even being mentioned through years of 
discussions about URNs in the mid-1990s.   To the extent URNs are usable 
in this way, it's a corner-case, an unintended consequence, not the 
general or typical case.

More generally, statements about how URNs "should be" that are based on 
observations of current namespaces or currently-observed usage of URNs 
strike me as about as valid as statements about how IPv6 should be, 
based on currently-observed usage of IPv6.   URNs, like IPv6, have been 
around for ~20 years, were designed to be usable for a very long time, 
but are really just starting to be used.   What we're seeing now with 
URNs (just like what we saw with IPv6 until recently) is still mostly 
due to early adopters, and shouldn't be considered as any indication of 
what will be typical of future use.   The communities that are most 
interested in using URNs properly are conservative by their very nature.

> identifies that work") and not for the purpose of resolution ("this 
> URN enables you to find a copy of that work on a shelf in this 
> building").

See above.   Of course ISBNs and ISSNs were in wide use long before URNs 
were standardized, and all of these numbers were "grandfathered" as 
URNs, so it's not at all surprising that there are huge numbers of 
these.   Again, their use should not be taken as a prescription for how 
URNs will, or should, be "typically" used in the future.

Of course - the primary purpose of a URN *is* to identify a resource.   
But if you encounter a reference to a URN on a computer that's attached 
to the Internet (or even an isolated, private network), then it's quite 
likely you will find it useful to not merely be able to identify that 
resource, but also to query for information about that resource, and/or 
try to access the resource or obtain its content.   You won't always be 
able to do these things for any of a large number of reasons, e.g. the 
URN isn't known to any of the resolution services you consult; the URN 
refers to a resource that isn't amenable to network access or isn't 
online; you don't have permission to access it, etc.   But just because 
such services won't always be available is no reason to cripple URNs in 
such a way as to make it more difficult to provide those services.

URNs were always intended to be able to support resolution.   Back to 
the IPv6 analogy, saying that URNs supporting resolution is of secondary 
importance is about like saying that the ability IPv6 to access hosts 
not on the IPv4 Internet is of secondary importance, because hardly 
anyone is doing that at the moment.   (of course that's really not true 
but many haven't noticed yet)

> about the syntax of those qualifiers (other than making sure that 
> syntax is consistent with the URI syntax).

I don't think we need (or want) a huge amount of precision.   In 
particular I don't think we should try to define a complete vocabulary 
for requests to resolution services and their parameters, and would 
actually prefer to see that work done outside of IETF. What I'd like to 
see is:

  * a small modification to the 2414bis grammar for URNs, to permit
    "r-components" in addition to f-, p- and q-components. (I think we
    can do this without breaking 3986 parsers, and with low risk of
    conflict with existing use of query strings.)
  * a grammar for r-component that allows multiple parameter names and
    associated values within that r-component to be specified to
    resolution services
  * an IANA registry for parameter names and/or associated values, with
    expert review required to add new ones, and each namespace having
    its own reserved prefix that it can do with as it wishes
  * a very minimal initial set of parameters, probably just defining
    "request" and "format" with an IANA registry for the names
    associated each, and 3 or 4 very basic requests.

I'm thinking this should be around 2-3 pages added to 2141bis, and I'm 
willing to draft text.

> modern URN resolution systems before we can define name-value pairs 
> for q-components.

IMO, rather than wait, we should let people who are building URN 
resolution systems define that vocabulary.   IETF's role should be to 
try to get the underlying protocol right, and  to make sure that URNs (+ 
3986 rules) don't inhibit resolution or uniform handling, and don't 
cause breakage when used with resources also named by URLs.   IETF's 
role should not be to try to discover for itself how to design a query 
language for a resource cataloging system.

> very little to show for that further delay and a great risk of failure 

I think that the risk of failure or fragmentation is actually much 
greater if we fail to specify how to distinguish between each of the 4 
types of qualifiers in a uniform way.

Or to put it a different way, there are two ways to get 
failure/fragmentation:   One is for us to delay so long that each 
namespace or community goes off and does things its own way, and the 
other is for us to fail to specify things completely or correctly enough 
so that each namespace or community is forced to do things its own way.

Keith

---

From: "Svensson, Lars" <[email-masked]>
Date: Fri, 17 Apr 2015 13:55:49 +0000

> on a per-namespace basis), please explain why and I'd drop this.

I see it as an issue and i vaguely remember that I suggested that we only should discuss queries (or q-components...) in the context of urn resolution services. So yes, we're on the same page here.

<snip/>

> I would especially like to see examples of #1, because I am not sure that they
> exist.
> 
> When I first wrote up that list, I wasn't sure about #1 myself.   I almost stated
> that #1 wasn't very useful, then I realized that that type of qualifier probably is
> needed after all, or at least that it wasn't safe to say that they're not needed.
> 
> One reason for having such a qualifier is this: you'd really like to be able to
> have persistent references to portions of a resource.    Fragments, as currently
> defined and implemented, don't serve that purpose.   (I could explain why but it
> would be a long side discussion, so will leave it to a separate message if the
> explanation is needed).   So urn:example:/a might refer to the entirety of a
> resource and urn:example:/a/b might refer to a portion of that resource, and
> urn:example:/a/b/c refer to a (presumably) still smaller portion - and all of
> these URNs would be managed by the namespace and expected to remain
> persistent over time (including their relationships to one another, if any of
> them used relative references).

This is what many in the science community wish for since it would allow them to cite parts of resources persistently (e. g. parts of a zip archive).

However, I do see that this is not very likely to work with URNs.
 
> accustomed.   IMO, the ability to specify a fragment of a URN-named resource
> is still useful, it's just that the combination of a URN and a fragment name (or if
> you prefer, f-component) no longer has any assurance of persistence.   So
> urn:example:a has an assurance of persistence but urn:example:a#b does
> not.   I think this will confuse people for a while, because they'll have to learn to
> mentally parse URNs differently than URLs.   But overall this might be the best
> path forward.   It would be difficult for us to exclude use of fragments with
> So I am thinking that the meaning of the fragment should still be the same for
> both URNs and URLs.   Ideally if urn:example:a resolves to
> http://example.com/b, and c is a valid fragment of that resource, then
> urn:example:a#c and http://example.com/b#c should refer to the same portion
> of that resource's content.   The alternative of letting fragments work
> differently for URNs vs. URLs strikes me as very unlikely to work well, especially
> if 3986-style relative reference processing can be applied to URNs.

+1. There is already now a risk that a fragment identifiers points to a place in the resource that doesn't exist anymore or that will not exist in future. As long as the f-components are not required to be persistent, that is not a problem.
 
> This is analogous to query strings in HTTP.   IMO it's vital to let URNs be able to
> bundle the name of the base resource with some input to be transmitted to the
> resource.   So similar to fragments, I think that if urn:example:a resolves to
> http://example.com/b, then urn:example:a?c should ultimately result in a GET
> of http://example.com/b?c .   Again, urn:example:a is persistent while
> urn:example:a?c has no assurance of persistence.

+1
 
> For the sake of an example of how this might be used, let's say that such a
> qualifier must begin with "??", that it appears before q-component, and the
> value of that qualifier is not allowed to contain "?" without it being %-
> encoded.  I'll tentatively call this an "r-component" ("r" for resolution)
> • urn:example:a would refer to the resource without specifying what to do with
> the URN.  If such a URN appeared in a link in a resource being viewed in a
> browser, and a user clicked on that link, the browser might try to present the
> content to the user.   Or failing that, it might simply try to get a description of
> the resource and display that.   If OTOH, such a URN appeared in an IMG tag
> the browser might give up if it couldn't get the content - presumably the
> description of the resource isn't an image.
> • urn:example:a??request=content would tell the resolution service (and
> perhaps also the client) that the reference was specifically intended to be for
> the resource's content, and not to locations, or any kind of description or
> metadata of the resource
> • urn:example:a??request=metadata;format=dc-rdf might request that the
> resolution service return a description of the resource in RDF format and the
> Dublin Core schema.
> • urn:example:a??request=locations could request locations of the resource
> • urn:example:a??request=content;version=first could request the earliest
> version of the resource
> Basically these qualifiers would serve as a kind of query to the resolution
> service (as opposed to a query to be sent to the resource).  It's like searching
> for a resource in a library's catalog and specifying some details that narrow the
> results along with the query.
> As far as I can tell, this kind of capability is vitally important, because almost
> every time I see any functionality of a resolution service described, there's at
> least one or two examples of how one might include such queries with a
> URN.   IMO, if URNs don't provide a standard way to do it, URN resolution
> services will overload "?" and the result will be that (a) you can't specify a URN
> that includes a query to be sent to the resource itself (at least not in any
> uniform way) and (b) different schemes will behave differently and it will be
> difficult to write clients that treat Uniform Resource Names in any kind of
> uniform fashion.
> Of course the above are just examples of what could be done, rather than a
> well thought-out proposal.   I would also suggest that namespaces should be
> able to define their own requests and parameters independently of others.   So
> for the "ietf" namespace, IETF should be able to support things like

This is nice. I was always concerned that the proposed use of q-components in 2141bis would conflate queries to the identified resources and queries sent to the resolution service, but this seems to solve it.
> 
> I continue to struggle with two things (or perhaps misconceptions in my own
> mind) about our discussions:
> 
> (a) Not all URNs need to be or can be resolved, and it seems to me that even
> URNs that can be resolved are primarily used to identify resources and not to
> resolve resources (which is why we have URLs, after all).

Yes, I agree with others that the primary goal is identification. My take is that resolvers are mainly there to help the user find a place where to get the resource, not to provide the resource itself.

> resolution services to helpfully specify what kind of syntax they might require.

The code of the resolver we use at the DNB is available on SourceForge [1], if that is of any help.

[1] http://sourceforge.net/projects/metaresolver/?source=directory

> Regarding (a), we have (i) a large number of URN namespaces that are used by
> other SDOs or quasi-SDOs to assign identifiers for uses like XML namespaces. I
> strongly encourage the WG participants to spend some time looking through
> the list of formal namespaces:
> 
> As far as I can see, most of the URNs that are assigned in those namespaces
> aren't intended to be resolved at all. They are, in the perhaps less than perfect
> terminology of 2141bis, "abstract designators". In these cases, there is simply
> no need to worry about qualifiers 3 & 4.
> 
> intended to name resources, not to merely be unique strings.   I don't mind if
> people sometimes use URNs as merely unique strings, as I don't think it does
> much harm - except that some people have gotten the idea that this is the
> primary purpose of URNs.  It's not.  I don't even recall that particular use case
> even being mentioned through years of discussions about URNs in the mid-
> 1990s.   To the extent URNs are usable in this way, it's a corner-case, an
> unintended consequence, not the general or typical case.

Hmm, so you say that while it's OK to have a name for a resource that isn't there, this is not how it was supposed to be? Like it's OK to produce URLs like http://example.bla/foo that don't resolve to anything.

Best,

Lars

---

From: Keith Moore <[email-masked]>
Date: Fri, 17 Apr 2015 10:11:44 -0400

On 04/17/2015 09:55 AM, Svensson, Lars wrote:
> Hmm, so you say that while it's OK to have a name for a resource that isn't there, this is not how it was supposed to be?

I think Peter was referring to URNs that don't even name resources, but 
just exist to be unique identifiers that are distinguishable from other 
unique identifiers.   And my opinion is that it's OK to have them but 
that's not the purpose for which URNs were created, and that use case 
shouldn't be used as an argument against making URNs more generally useful.

Keith

---

From: "Leslie Daigle (ThinkingCat)" <[email-masked]>
Date: Fri, 17 Apr 2015 17:27:00 -0400

Hi,

Thinking through the r-component idea a bit...   I think your 
argumentation for r-components is pretty compelling, especially as we 
work to make clear why f-components don't fit that particular bill. 
However, I am more than a little concerned about getting too complex and 
bolting stuff on at this point.

I won't argue that DDDS has been used for just about everything *except* 
providing the standard for URN resolution, but in working it out we I'm 
sure you recall we did did actually talk through "resolution services" 
(for ease of reference for anyone who missed it -- 
https://tools.ietf.org/html/rfc2483).

Again, not quite what you are describing now (I read you to be thinking 
more about parameters one would send, not services one would request), 
but perhaps enough to illustrate that there is a whole farm of things 
one might wish to consider if one stops and thinks about using URNs in 
action.

I think pursuing r-components as part of this document would slow this 
one down a fair bit.

Is there a way to leave enough of a hook in the syntax to pursue 
r-component as a separate document?  (BTW...  does "??" actually fly 
with URI syntax, or wouldn't the 2nd "?" be part of the q-component?).

Leslie.

-- 

-------------------------------------------------------------------
Leslie Daigle
Principal, ThinkingCat Enterprises
[email-masked]
-------------------------------------------------------------------

---

From: Keith Moore <[email-masked]>
Date: Fri, 17 Apr 2015 20:05:41 -0400

On 04/17/2015 05:27 PM, Leslie Daigle (ThinkingCat) wrote:
> However, I am more than a little concerned about getting too complex 
> and bolting stuff on at this point.
>
> I won't argue that DDDS has been used for just about everything 
> *except* providing the standard for URN resolution, but in working it 
> out we I'm sure you recall we did did actually talk through 
> "resolution services" (for ease of reference for anyone who missed it 
> -- https://tools.ietf.org/html/rfc2483).
>
> Again, not quite what you are describing now (I read you to be 
> thinking more about parameters one would send, not services one would 
> request), but perhaps enough to illustrate that there is a whole farm 
> of things one might wish to consider if one stops and thinks about 
> using URNs in action.
>

"Leaving [just] enough of a hook" is exactly what I want to do in this 
document.  I think that means:  (a) define the syntax for URNs including 
that of r-components, (b) explain the role of r-components in URN 
resolution, and (c) set up an IANA registry of r-component names and 
values.   Maybe also (d) explain how this is syntax compatible with 
3986.  I think that's about enough for 2141bis.

(I'm thinking that I should suggest explicit text for this if for no 
other reason than that that exercise will give us a good idea of just 
how much would need to be added to 3986.)

Separate from 2141bis, I'm thinking I might like to write up an 
Experimental or Informational document defining a simple URN resolution 
service and a client API for such a service.   (perhaps cribbing a bit 
from RFC 2483).   This simple service would define a minimal set of 
simple requests and the parameters for such requests.   The main purpose 
in doing so would be to serve as a proof-of-concept and to encourage 
further development in this area by the library community.

And yes, the 2nd "?" would be part of the q-component if this were 
parsed according to 3986.   Given a URN

*urn:example:something??a=b;c=d?mumble*

a 3986-style parser would return

{
    "scheme": "urn",
    "path": "example:something",
    "query": "??a=b;c=d?mumble"
}.

However, it doesn't violate 3986 syntax, because "?" is explicitly 
permitted within <query>.  (See RFC 3986, section 3.4, top of page 24)

My take on this is that an ordinary URL parser shouldn't cause 
catastrophic failure (like causing the program to crash) with something 
that conforms to 3986 syntax.   Of course, most existing programs don't 
try to do URN resolution anyway, so their inability to parse URNs 
exactly like I'm proposing for 2141bis won't really do them much harm.   
By contrast, programs that support URN resolution in the future can 
parse them according to the 2141bis grammar, which would yield a result 
like:

{
    "scheme": "urn",
    "namespace": "example",
    "nss": "something",
    "r-component": { "a": "b", "c": "d" },
    "q-component": "mumble"
}

---

From: John C Klensin <[email-masked]>
Date: Fri, 17 Apr 2015 22:58:55 -0400

>     "query": "??a=b;c=d?mumble"

Actually, 

	{
	    "scheme": "urn",
	    "path": "example:something",
	    "query": "?a=b;c=d?mumble"
	}
	
because 3986 says

   absolute-URI  = scheme ":" hier-part [ "?" query ]

and not, e.g.,

   absolute-URI  = scheme ":" hier-part [ query ]
   query = "?" query-component

Right?

>...
> By contrast, programs that support URN resolution in the
> future can parse them according to the 2141bis grammar, which
> would yield a result like:
> 
> {
>     "scheme": "urn",
>     "namespace": "example",
>     "nss": "something",

Yes, AFAICT.

    john

---

From: Keith Moore <[email-masked]>
Date: Fri, 17 Apr 2015 23:07:00 -0400

>     query = "?" query-component

Correct.   I realized that after I sent the message.

Keith

---

From: "Leslie Daigle (ThinkingCat)" <[email-masked]>
Date: Tue, 21 Apr 2015 12:32:58 -0400

[Apologies for delayed/lost e-mail -- mail server issues. ]

In-line...

> 3986.  I think that's about enough for 2141bis.

I think that's considerably more than a hook, it's in danger of being a 
fishing expedition.

I might support (a) and (b), (d) is a requirement, but (c) is a real 
reach unless or until there's  (electronic) ink on (digital) paper for 
where to go with this idea, as a general idea, with consensus support.

Otherwise -- it's one person's great idea, and if you don't happen to 
finish writing it down, someone else will come along in a decade and 
fill it in very differently than anything you might ever have thought of 
(or that we, collectively, might have agreed to).

Leslie.

--

---

From: Keith Moore <[email-masked]>
Date: Tue, 21 Apr 2015 17:19:53 -0400

On 04/21/2015 12:32 PM, Leslie Daigle (ThinkingCat) wrote:
> I think that's considerably more than a hook, it's in danger of being 
> a fishing expedition.
>
> fill it in very differently than anything you might ever have thought 
> of (or that we, collectively, might have agreed to).

It's reasonable to want to look at actual text.   And we've certainly 
seen a lot of examples of people coming along in a decade and filling in 
things very differently than originally conceived, in this space.

Of course it's not the registry which is difficult to set up, it's the 
rules for what can go in the registry.

Keith