Subject: Re: [urn] Working Group Last Call of draft-ietf-urnbis-rfc2141bis-urn-06.txt

From: Keith Moore <[email-masked]>
Date: Tue, 03 Sep 2013 10:26:26 -0400

On 08/28/2013 07:37 AM, Juha Hakala wrote:
> Hello,
>
>
> On 20.8.2013 1:52, Keith Moore wrote:
>> On 08/19/2013 05:54 PM, Peter Saint-Andre wrote:
>>>>> As I understand it, so far the only use case envisioned for query
>>>>> components (and perhaps also fragment identifier components) is
>>>>> exactly
>>>>> that of passing additional information to a URN resolution
>>>>> service. So I
>>>>> agree with John that people who really want this feature will ignore
>>>>> whatever the spec might say.
> Yes, it is my intention to specify in rfc2483bis a query syntax for
> sending resolution service related requests to URN resolvers. If we do
> not provide such a syntax, there will be a lot of mutually
> incompatible local solutions for how to use query in this manner, even
> if rfc2141bis tries to forbid or disencourage the use of query.

It is essential that any syntax used to pass queries to resolvers be
different than the syntax used to pass queries to the resources named by
URNs.   "?" is already in wide use to pass queries to resources named by
URLs, and it would be very confusing if "?" had a different purpose for
use with URNs. (e.g. passing queries to resolution services)

>
> Alfred Hoenes and myself have started the query specification in
> rfc2141bis-urn-03.  Our idea was to use the service names from RFC
> 2483, and to complement them with service related parameters; for
> instance, having just I2C (URI to URC) is not sufficient in the
> situation where there are multiple metadata formats.
>>> Correction: they will ignore any hard prohibition on using query
>>> components and fragment identifier components in that way.
> Yes, it would be very difficult to convince the URN users why they
> should not use query, especially when other persistent identifiers are
> using them already.

It will be very difficult to convince users to use "?" differently for
URNs than they are accustomed to using it for HTTP URLs.
 
>
>> The most that the standard can do is explain to people why this is a bad
>> idea.  But that's easy to do.  If people want to build resolution
>> servers that break URNs, I don't pretend that we can stop them.
> The most that the standard can do is to help people in using query in
> a useful manner. I don't think that it is our task to tell people why
> they should not do something that is allowed elsewhere. 

"?" is used elsewhere to pass queries to named resources.   Why should
people not use "?" to pass queries to resources named URNs?

There is an apparent conflict between different uses of "?".   The HTTP
usage of "?" is very common and there is a lot of mindshare behind it.

> And we should not write standards in such a way that the users cannot
> utilize features such as query which are likely to be useful for them.
>
> As an aside, this is not the first time there is a need to use query
> to pass service requests. Information retrieval systems use SRU
> protocol (http://www.loc.gov/standards/sru/) for sending queries to
> remote systems like this:
>
> http://z3950.loc.gov:7090/voyager?version=1.1&operation=searchRetrieve&query=dinosaur&maximumRecords=1&recordSchema=dc
>
>
> In comparison, the planned use of query in the URN context should be
> relatively simple. But the URN resolvers may need to get smarter. For
> instance, if I want a resource description instead of the resource
> itself, the URN resolver may need to extract the required data
> elements from the URN query into an SRU searchRetrieve URL (if the
> target system prefers SRU to URN queries).

I don't think that makes any sense at all, as it either (a) prevents
users from passing queries to end resources, or (b) forces both end
resources and resolvers to know which queries apply to them.

There seems to be a serious disconnect here.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 03 Sep 2013 10:29:49 -0400

On 08/28/2013 10:12 AM, Svensson, Lars wrote:
> Yes, and the question (to me) still is if the query attached to the urn is to be applied to the resource identified by that urn or if it is supposed to pass information on to a (http-based) resolution service.
This is precisely the question, and it seems to me that the answer that
makes more sense is that the "?" syntax is only used to pass information
to the resource identified to the URN.

Any queries passed to the resolution service should use a different syntax.

Keith

---

From: "Svensson, Lars" <[email-masked]>
Date: Tue, 3 Sep 2013 15:49:02 +0000

On 3. September 2013 16:30 Keith Moore wrote
> > Yes, and the question (to me) still is if the query attached to the urn is to be
> > applied to the resource identified by that urn or if it is supposed to pass
> > information on to a (http-based) resolution service.

> This is precisely the question, and it seems to me that the answer that makes
> more sense is that the "?" syntax is only used to pass information to the
> resource identified to the URN.
> 

Is this what you want to use the "/" for? If so, we need to be very sure that there are no existing urns with that character in them.

Lars

---

From: Keith Moore <[email-masked]>
Date: Tue, 03 Sep 2013 12:03:23 -0400

> Is this what you want to use the "/" for? If so, we need to be very sure that there are no existing urns with that character in them.

"/" is about the only character left that we could use for that purpose.

But since "/" was reserved in 2141, no existing URNs should be using it.

Keith

---

From: "Svensson, Lars" <[email-masked]>
Date: Tue, 3 Sep 2013 16:09:52 +0000

On 3. September 2013 18:03 Keith Moore wrote:
> An: Svensson, Lars
> Cc: Juha Hakala; [email-masked]
> Betreff: Re: AW: [urn] Working Group Last Call of draft-ietf-urnbis-rfc2141bis-
> urn-06.txt
> 
> >>> Yes, and the question (to me) still is if the query attached to the
> >>> urn is to be applied to the resource identified by that urn or if it
> >>> is supposed to pass information on to a (http-based) resolution service.
> >> This is precisely the question, and it seems to me that the answer
> >> that makes more sense is that the "?" syntax is only used to pass
> >> information to the resource identified to the URN.
> >>
> > Is this what you want to use the "/" for? If so, we need to be very sure that
> there are no existing urns with that character in them.
> 

The operative word being "should". RFC 2141 § 2.3.2 explicitly says "Namespace developers SHOULD NOT use these characters in unencoded form, but rather use the appropriate %-encoding for each character." It wasn't explicitly forbidden.

Lars

---

From: Keith Moore <[email-masked]>
Date: Tue, 03 Sep 2013 12:20:18 -0400

>>> Is this [queries passed to a resolution service] what you want to use the "/" for? If so, we need to be very sure that

Ok, valid point.  But I suspect it still works better than "?"

Note that even if there are some URNs that use "/", it might be possible
to define a "resolution service parameter" syntax that would make it
possible for resolution services to reliably distinguish such parameters
from "/"s in existing URNs.   For instance, if parameters passed to
resolution services had to be of the form

/[A-Za-z][A-a-z0-9]*=[A-Za-z0-9]+

but none of the existing URNs used "=" after "/", the resolution service
would know the difference.

Keith

p.s. is it necessary to poll each of the URN namespace operators to
learn to what extent "/" is being used?   Should that poll also ask
about other reserved characters?

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 03 Sep 2013 21:16:45 -0600

On 9/3/13 8:26 AM, Keith Moore wrote:
> users from passing queries to end resources, 

Do we have any examples of passing a query to the sorts of resources
that are identified by URNs?

> or (b) forces both end

Agreed, that strikes me as a dangerous path to go down.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Keith Moore <[email-masked]>
Date: Tue, 03 Sep 2013 23:30:00 -0400

> that are identified by URNs?

What are "the sorts of resources that are identified by URNs"?  

I thought URNs were supposed to be usable with anything, including (but
certainly not limited to) resources that can be named (less
persistently) with ordinary URLs and are accessible via HTTP or other
protocols that support queries. 

Keith

---

From: "Svensson, Lars" <[email-masked]>
Date: Wed, 4 Sep 2013 14:23:48 +0000

On 3. September 2013 18:20 Keith Moore wrote
 
> about other reserved characters?

That might be a good idea, both for "/" and the other reserved characters.

As an aside, I am slightly concerned with the silence from urn implementers from outside of the library domain. I fear that we might end up with a solution that could work well for one _some_ namespaces (nbn, isbn, issn, etc.) and not work at all for others. But we cannot force them.

/Lars

---

From: Michael Mealling <[email-masked]>
Date: Wed, 4 Sep 2013 10:34:44 -0400

I helped write or consulted on several of the existed URN registrations and by and large the other communities use URNs as we all originally expected: opaque strings to the outside world and meaning only within their systems. I would say almost half had no intention of trying to use things like query strings or fragment ids. Since they were being used in non-browser, non-HTTP oriented systems those concepts were alien and probably incompatible. Especially the ones related to hierarchy.

I think its instructive for people involved in these discussions to go look at some of the existing registrations and see how and why they're being used. MOST of the registrations are not library related and even less are meant to be used in browsers by humans:

http://www.ietf.org/assignments/urn-namespaces/urn-namespaces.txt

-MM

Michael Mealling
Co-Founder
Pipefish.com
+1-678-640-6884	@mmealling
Schedule a meeting:  http://meetme.so/michaelmealling

On Sep 4, 2013, at 10:23 AM, "Svensson, Lars" <[email-masked]> wrote:

> _______________________________________________
> urn mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/urn

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 04 Sep 2013 08:39:32 -0600

On 9/4/13 8:34 AM, Michael Mealling wrote:
> I helped write or consulted on several of the existed URN
> registrations and by and large the other communities use URNs as we
> all originally expected: opaque strings to the outside world and
> meaning only within their systems. I would say almost half had no
> intention of trying to use things like query strings or fragment ids.
> Since they were being used in non-browser, non-HTTP oriented systems
> those concepts were alien and probably incompatible. Especially the
> ones related to hierarchy.
> 
> I think its instructive for people involved in these discussions to
> go look at some of the existing registrations and see how and why
> they're being used. MOST of the registrations are not library related
> and even less are meant to be used in browsers by humans:
> 

Agreed. Many or most of the namespace operators I shepherded through the
NID registration process as an AD were issuing XML namespaces. In terms
of numbers of NIDs, use by libraries, publishers, and the like are in
the minority. However, I think the bibliographic and museum communities
have issued far more URNs than all the others combined.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Keith Moore <[email-masked]>
Date: Wed, 04 Sep 2013 10:50:30 -0400

> I think its instructive for people involved in these discussions to go look at some of the existing registrations and see how and why they're being used. MOST of the registrations are not library r…
Ok, sure, but do we really think that the way that URNs are used now
should dictate or constrain how URNs can be used in the future?   I
think URNs are still very early on the adoption curve and the usage that
we're seeing now is probably not a good indicator of how they'll be used
in the future - unless of course we impose restrictions that limit the
applicability of URNs based on current experience.

Keith

---

From: [email-masked]
Date: Wed, 04 Sep 2013 19:26:09 +0300

Hello,

>> ones related to hierarchy.

Those namespaces which have no use for fragment and query for the time  
being do not need to implement them. In the future they may find  
especially the latter relevant.

IMO there is nothing library specific in either in fragment or query  
as such. Of course library, museum and archival communities may have  
some specific uses for query since our applications have features (and  
data model) which are not commonly shared. But other users and  
communities are free to specify their own resolution services and  
query syntaxes for requesting these services from the resolvers they  
maintain.

>> http://www.ietf.org/assignments/urn-namespaces/urn-namespaces.txt

Yes, it is important to make sure that the URN syntax and resolution  
service registration mechanism can be adapted to the needs of  
different communities.
> have issued far more URNs than all the others combined.

This may be true. The largest URN user I am aware of is the National  
library of Norway, which allocated about 600 million URNs to the files  
in the Norwegian web archive. There are several libraries which have  
assigned millions of URNs to digital resources (articles, reports etc)  
as a part of automated ingest processes.

Usage of persistent identifiers will increase further, when libraries  
start assigning identifiers to metadata records and individual data  
elements such as names of people and organisations. The system created  
by the Bibliotheque de France (http://data.bnf.fr/) is a good example  
of this practice which will spread fast with open linked data projects.

Juha

---

From: [email-masked]
Date: Wed, 04 Sep 2013 19:58:29 +0300

Hello,

>>> > users from passing queries to end resources,

Passing query to the URN resolver is relatively easy since the  
functionality provided by the relevant resolvers (specified in RDS or  
indicated more directly with HTTP URI) will be known. Assuming that  
the users will be provided a GUI where they indicate the services they  
want (which are then encoded in queries) people will seldom require  
something that cannot be supplied. For instance, when I see a link to  
the resource, I may indicate with the tick box that I want rights  
metadata about the resource, or technical metadata which will tell me  
which applications will be able to open the file.

It will be an interesting task to manage the situation in which the  
query is somehow passed to the end resource. Do you mean the  
browser/viewer/whatever in the client end of the process should be  
able to fulfill the service request? I would not like to be the person  
with the task to specify anything like full set of such resolution  
services. Also, given the diversity of the applications used, it will  
not be easy to guarantee that the client process does not query  
services that cannot be supplied.

As I see it, queries will primarily be used for passing service  
requests to the URN resolvers. Such functionality makes a lot of  
sense, at least from implementer point of view. But in order to future  
proof the syntax it should also be able to specify queries which  
should be processed in the client end.

>> that are identified by URNs?

One standard-based example is OpenURL:

http://resolver.example.edu/cgi?genre=book&isbn=0836218310&title=The+Far+Side+Gallery+3

See http://en.wikipedia.org/wiki/OpenURL for more information.

OpenURL and URN are regarded as complementary services which will not  
collide, for instance because they will be resolved in different  
services. Of course, if I want to find a copy of an identified  
resource which I can access, the resolver may translate may urn +  
query into OpenURL.

Juha
> protocols that support queries.

The scope of URN is the sum of scopes of all identifier systems that  
have registered namespaces. Given that UUID is included, pretty much  
anything goes.

I find it interesting that there is a trend towards providing  
persistent identifiers also to e.g. names which "live" in deep web and  
can only be accessed via HTTP if a query protocol such as SRU is  
applied in order to create a database query.

Juha

---

From: Keith Moore <[email-masked]>
Date: Thu, 05 Sep 2013 18:03:48 -0400

On 09/04/2013 12:58 PM, [email-masked] wrote:
> something that cannot be supplied.
> For instance, when I see a link to the resource, I may indicate with
> the tick box that I want rights metadata about the resource, or
> technical metadata which will tell me which applications will be able
> to open the file.
>

I am very hesitant to make assumptions about the user interface,
especially when we don't really have much of an idea of what resolution
services will exist.   (I don't even want to assume that ordinary
clients will want to expose this much functionality to the user - useful
though it might be, it seems like it would be a drastic departure from
how people are accustomed to having "the web" work.)

And anyway, I thought we were discussing how to use query syntax (or
some other syntax) to embed resolver requests along with URNs, rather
than how to present options for resolution to users.  I'm not saying
that the latter isn't important, but presumably the purpose of embedding
the request in the URN is to predetermine what the user gets back, at
least as a default.

Issues of query syntax aside, how much should clients be expected to
know about different URN namespaces and different URN resolvers?  
Sometimes I get the impression that people in this WG expect clients
using URNs to be specific to particular namespaces and for those clients
to know specific information about particular resolution services.   I
have no doubt that such clients will exist, but I think the architecture
should facilitate at least some generic handling across all URN
namespaces, a common (yet extensible) core resolution protocol that is
applicable to most or all URN namespaces, and a common means of
discovering resolution services.

It is because I envision a common architecture that is independent of
URNs that it makes no sense to me that different URN namespaces should
treat "?" differently, or that some query parameters should be delegated
to resolution services and others to the resources, or that there should
be some sort of magic flag that, when it appears in a URN, specifies
that all query parameters are passed to the resolver and that there's no
way to pass such parameters to the resource.

And sure, the way that different URL types handle query syntax is not
consistent from one URL to another.  That's mostly because a plethora of
URL types (each with different ways of treating query syntax) were
developed prior to any serious attempt to promote uniform URL syntax.  
But I don't see that as a justification for repeating the same mistake
in URN space.

> able to fulfill the service request? 
I mean that when you have a resource (like a search engine, or any
program on a server that accepts form data) that, by its very nature,
accepts queries or is able to do so, passing queries to that resource is
a very natural and desirable thing to be able to do.   And sure, in
those cases, where the interface for passing the query to the resource
is known (say, because it is accessible via HTTP or some other
well-known protocol that permits transmitting queries), I'd expect the
client to be able to properly format the query and pass it to the resource.

> I would not like to be the person with the task to specify anything
> like full set of such resolution services. Also, given the diversity
> of the applications used, it will not be easy to guarantee that the
> client process does not query services that cannot be supplied.

If the client sends an invalid query, presumably it gets back an error
indication.   Why is this a problem?

> requests to the URN resolvers. 

Given that there is today a huge number of network-accessible resources
that accept queries and process them, why is your assumption that this
won't be a common scenario for resources named by URNs?

> Such functionality makes a lot of sense, at least from implementer
> point of view. But in order to future proof the syntax it should also
> be able to specify queries which should be processed in the client end.

I think there is a need to be able to both send queries to the resource
and send queries to the resolver.  But I don't think the two should
share a common syntax.

> resource which I can access, the resolver may translate may urn +

I'm having a difficult time seeing how this example illustrates what (I
think) we're talking about.

> have registered namespaces. Given that UUID is included, pretty much

Here's an important architectural principle for URNs:

A URN namespace has nothing inherently to do with the resources that are
named by URNs within that namespace.

At least as far as the overall URN architecture is concerned, any URN in
any namespace can name any kind of resource.   (If a particular
namespace wants to constrain what kinds of resources it names, that's
its business.   But it's free to change its policy at some point in the
future.)

So to answer my own question, it's impossible to characterize the sorts
of resources that are or will be identified by URNs.   And it's probably
a mistake to try to make design or architecture choices based on
assumptions about what sorts of resources will be used.

Keith

---

From: Juha Hakala <[email-masked]>
Date: Thu, 19 Sep 2013 09:06:30 +0300

>> Quoting Keith Moore <[email-masked]>:
<snip>
> the request in the URN is to predetermine what the user gets back, at
Yes. User interfaces are important in this context only because a user 
may not know what services are available for a given document, and how 
to construct an appropriate query.
> know about different URN namespaces and different URN resolvers?
Clients such as Web browsers should not be expected to know much of what 
is going on. The main "players" are URN resolvers and the applications 
they contact to request the service the user is asking for.

For instance, Finnish national bibliography may contain a description of 
an e-book with a URN in it. The GUI of the national bibliography 
database may give a user an option to find a copy of the book he is 
allowed to use. When the user clicks that button, the URN + query are 
passed to the resolver. The resolver passes the request on the the valid 
OpenURL resolver, which will check the users' permissions and choose an 
appropriate copy of the resource.

> discovering resolution services.
URN resolver discovery service would be useful, but unfortunately does 
not exist yet although incorporating it into DNS has been foreseen ever 
since the first URN WG.

The difficulty of finding an appropriate resolver varies from one 
namespace to the next. If the identifier is "dumb", there is no hint 
where to find a resolver and only one can exist. ISSN is an example of 
this; the only place where they can be resolved is the ISSN database of 
the International ISSN Centre. The more "intelligent" the identifier, 
the more resolvers then can / must be. NBN and ISBN resolution can / 
must decentralized to the national level. IMO an identifier which a 
priori cannot be resolved ("dumb" identifier with no central database) 
does not deserve a URN namespace.

If all URNs from a given namespace are resolved in a single place we 
know exactly what services will be available. But often there will be 
multiple resolvers, and the only means of "quality control" is to 
recommend a minimum set of services in the namespace registration.

We have earlier decided that some namespaces may not provide any 
resolution services at all. This means that core resolution protocol 
will not apply to all namespaces.
> way to pass such parameters to the resource.
As I see it, different URN namespaces will provide different services, 
but in all namespaces, a query like "?s=I2L" should provide the same 
result.

And I guess there can be services where the agent can be either the 
resolver or the client.
> client to be able to properly format the query and pass it to the resource.
OK. I have misunderstood your point; to me resource is the object that 
is the URN identifies.

The scenario above is plausible. Let us assume that we want to retrieve 
bibliographic data about a resource from a library database. The URN + 
query can be passed to the resolver, which then sends it to the library 
system as SRU query. But there is no reason why the library system 
itself would not be able to process URN + query as such.

> won't be a common scenario for resources named by URNs?
See above.
> and send queries to the resolver.  But I don't think the two should
Why?

Juha

-- 

  Juha Hakala
  Senior advisor

  The National Library of Finland
  Library Network Services
  P.O.Box 26 (Teollisuuskatu 23)
  FIN-00014 Helsinki University
  Tel. +358 9 191 44293
  Mobile +358 50 3827678