Subject: [urn] URN fragments

From: Juha Hakala <[email-masked]>
Date: Thu, 21 Feb 2013 13:36:28 +0200

Hello Peter; all,

Concerning this:
> Could you also explain (perhaps in a separate thread) the intended use
> of the fragment identifier?
It took a while before the bibliographic community got a sufficient grasp of how fragment can best be used in the URN context. Initially we thought that fragment should be part of the URN namespace specific string, but we realized soon that that would complicate things a lot. Since many (most, probably) standard namespaces do not allow add-ons to identifier strings, this:

  Ex. 1http://urn.fi/URN:ISBN:978-952-10-7670-1

would have been possible, but not this (NOTE: this is only an example)

Ex. 2http://urn.fi/URN:ISBN:978-952-10-7670-1#chapter2

Moreover, adding a fragment to an existing URN would in this case have been an act of identifier assignment, which in some namespaces is a managed process, so too few people would have been allowed to use fragments with URNs.

Eventually Alfred Hoenes, myself and others concluded that the best solution is to separate identification & URN assignment from fragment assignment, that is, not to include fragment in the NSS. Which means that fragments can be added to the URN by anybody, any time after the URN has been assigned and the document made available (via URN resolution) in the Internet.

Then a typical use case is a scientist who wants to cite a structured (and identified) resource. For instance, citation to chapter 2 of the book with ISBN978-952-10-7670-1  <http://urn.fi/URN:ISBN:978-952-10-7670-1>  could now be done with the HTTP URI in Example 2 above. Of course, using fragment is only possible if the URN is actionable, supports resolution to the resource and the file format of that resource supports fragment in the spirit of RFC 3986. There are a lot of if's here, but this infrastructure is maintained by a national library / archive, service is likely to persist for quite some time.

As an aside, standard guidelines for citing & referencing are out of date. When PIDs are not assigned to start with, people have to use URLs when citing networked resources. In some cases URLs are used even if there is a resolvable PID. Most of us have probably seen 5 -10 year old documents in which most URL links in the reference list are already dead. This will make it very difficult to confirm that citing and referencing has been made correctly, or that the cited document has ever even existed. It is important to improve the situation by using PIDs such as URNs and DOIs, and by providing guidelines on how to use them in citing & referencing.

When fragment usage was last discussed on the list, the IETF community disapproved of the idea (if I remember correctly) on the basis that URNs will not be bound to a single manifestation of a resource, such as a PDF version of a book. When the file format changes, the probability that the fragment no longer works is too close for comfort to 1.

There are two responses to this. The obvious one is that if the fragment is not part of the URN itself (but just tells the browser a location where to go within the identified resource) the URN will nevertheless still be valid, and the user will get a modernized version of the resource. The other one, with which the bibliographic community did not get anywhere, was that there are namespaces where identifiers must be assigned separately for each manifestation of a resource. ISBN is an example of this, and although some publishers do misuse the system, the vast majority of ISBNs have been correctly assigned - which means that we have a good reason to believe that fragments would work fine longer than the browsers are able to cope with the ancient file formats. Guaranteeing access to original versions of resources, or digital archaeology, may well keep the memory institutions busy in the future...

There may of course exist namespaces where fragments cannot be used. There are identifier systems for immaterial works (libraries do make a difference between the original english version of "Hamlet"; its expressions in other languages, and physical manifestations of these). There are identifiers for names, collections, metadata elements, and so on. These identifiers do not have URN namespace registrations yet, but such registrations can be made in the future. Therefore namespace registrations should say whether fragment usage is applicable at least in theory. In the same way, they could provide examples of resolution services that can be supported.

Even if the IETF URN syntax specification would not say anything about the usage of fragment, the Finnish standard will. Given that RFC 3986 specifies fragment usage and all popular browsers support this functionality, it is more or less certain that people start using fragments with URNs (expressed as HTTP URIs) when e.g. citing documents. If guidelines for doing this are given in RFCs or other standards, it will be easier to co-ordinate and perhaps even foster this process, and to renew for instance ISO 690, the standard for bibliographic referencing (http://en.wikipedia.org/wiki/ISO_690).

Best regards,

Juha

-- 

  Juha Hakala
  Senior advisor

  The National Library of Finland
  P.O.Box 15 (Unioninkatu 36, room 503)
  FIN-00014 Helsinki University
  tel +358 9 191 44293

---

From: Jonathan A Rees <[email-masked]>
Date: Thu, 21 Feb 2013 08:56:18 -0500

On Thu, Feb 21, 2013 at 6:36 AM, Juha Hakala <[email-masked]>wrote:

> That is not my recollection. The problem with defining fragment ids in the
urn: registration is that fragment id semantics comes from RFC 3986 and
simply cannot be changed or overridden by any URI scheme registration.
Quoth 3986:

   Fragment identifier semantics are independent of the
   URI scheme and thus cannot be redefined by scheme specifications.

Jonathan

---

From: [email-masked]
Date: Thu, 21 Feb 2013 16:52:31 +0200

Hello,

As I said, our conclusion has been that fragment is not part of the  
URN (NSS). Therefore the fragment semantics comes from RFC 3986 and  
nowhere else. Note however that we should not talk about fragment  
identifiers; fragment is just something that is attached to the  
identifier as a guideline to the browser to take the user into a given  
location within the resource.

If it were not possible to use fragment with URN, we might have two  
URIs (one URN and one URL) both resolving to the same resource  
(location), but it would only be OK to use fragment with the URL,  
although the fragment string would be exactly the same in both cases.  
I doubt this was the intention of RFC 3986.

In practice, people will attach fragments to URNs (when they are  
expressed as HTTP URIs). For these Internet users, this will be like  
using fragments with URLs. It is important to support them by  
explaining the rules of the game for them in RFC 2141.

Juha

Quoting Jonathan A Rees <[email-masked]>:

>> When fragment usage was last discussed on the list, the IETF  
>> community disapproved of the idea (if I remember correctly) on the  
>> basis that URNs will not be bound to a single manifestation of a  
>> resource, such as a PDF version of a book. When the file format  
>> changes, the probability that the fragment no longer works is too  
>> close for comfort to 1.
>>

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 21 Feb 2013 20:36:54 -0700

On 2/21/13 4:36 AM, Juha Hakala wrote:
> Hello Peter; all,
> 
> Concerning this:
>> Could you also explain (perhaps in a separate thread) the
>> intended use of the fragment identifier?
> It took a while before the bibliographic community got a
> sufficient grasp of how fragment can best be used in the URN
> context. Initially we thought that fragment should be part of the
> URN namespace specific string, but we realized soon that that would
> complicate things a lot. Since many (most, probably) standard
> namespaces do not allow add-ons to identifier strings, this:
> 
> Moreover, adding a fragment to an existing URN would in this case
> have been an act of identifier assignment, which in some namespaces
> is a managed process, so too few people would have been allowed to
> use fragments with URNs.
> 
> Eventually Alfred Hoenes, myself and others concluded that the
> best solution is to separate identification & URN assignment from
> fragment assignment, that is, not to include fragment in the NSS.
> Which means that fragments can be added to the URN by anybody, any
> time after the URN has been assigned and the document made
> available (via URN resolution) in the Internet.

Hi Juha, this model sounds similar to what you described for the query
component: the fragment can be "added to the URN". I'm not sure what
that means. When you add a query component or a fragment identifier to
the URN, is that now a different URN (which wasn't assigned by the
minting organization or in accordance with the minting algorithm)? Or
is "a URN with some additional data appended" and therefore not really
a URN anymore? Or is actually a URI (such as an HTTP URI) whose path
is the URN itself as just a string of characters embedded in the
broader URI?

Still confused,

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 22 Feb 2013 13:10:47 +0900

On 2013/02/22 12:36, Peter Saint-Andre wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>

>> best solution is to separate identification&  URN assignment from

This is different for the query part and for the fragment part. The 
fragment part is independent of the scheme. The query part is not.

You will see that in Appendix A of RFC 3986 
(http://tools.ietf.org/html/rfc3986#appendix-A). There, URI is defined 
as including an (optional) fragment part, but absolute-URI is defined 
excluding the fragment part. Same in RFC 3987 for IRIs. So in terms of 
terminology, it might be possible to call an URN without a fragment part 
an absolute URN (in places where such a distinction is necessary).

For fragments, the conclusion that Juha explains above is how it was 
intended for all URIs from the start, and how RFC 3986 is written. It 
may have taken some time for the URN community to arrive at this point, 
but it's a very good conclusion, because it's what works best with the 
rest of the infrastructure.

Regards,   Martin.

---

From: Juha Hakala <[email-masked]>
Date: Fri, 22 Feb 2013 08:57:10 +0200

Hi,

> is the URN itself as just a string of characters embedded in the 

Despite the addition of fragment and query the URN itself remains the 
same, even though the HTTP URI changes.

"Added to the URN" means that neither query nor fragment are part of the 
URN, or more specifically, its namespace specific string. When something 
is identified, more or less formally, the NSS is and must always be 
sufficient for that, and RFC 2141 does not provide any functionality 
beyond that. Full alignment with RFC 3986 would in this interpretation 
mean that once a resource has been identified with a URN, anybody can 
add a fragment to the URN in order to pinpoint a location within the 
resource (for the browsers to act upon), or a query to demand a service 
from the URN resolver.

Syntactically and semantically this interpretation should match the one 
in RFC 3986. Note that fragments do not identify anything since they are 
not part of the NSS. IMHO, since we must use RFC 3986 as the starting 
point it is not even appropriate to say that in the URN context fragment 
actually identifies something. So Peter's term "fragment identifier" 
above is not proper language :-).

One reason why we should clarify the usage of fragment and query in the 
URN context is that there is a need to explain which fragment + query 
combinations make sense. In principle, it is only possible to use 
fragment when the resource itself is requested. In practice, there may 
be namespaces or service environments in which the situation is or will 
be more complicated than that.

Juha

-- 

  Juha Hakala
  Senior advisor

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Feb 2013 17:01:12 +0100

On 2013-02-22 07:57, Juha Hakala wrote:
> ...

Well, it's a different string, thus a different URN.

 > ...

Best regards, Julian

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 26 Feb 2013 10:10:24 -0700

On 2/26/13 9:01 AM, Julian Reschke wrote:
>>> Hi Juha, this model sounds similar to what you described for
>>> the query component: the fragment can be "added to the URN".
>>> I'm not sure what that means. When you add a query component or
>>> a fragment identifier to the URN, is that now a different URN
>>> (which wasn't assigned by the minting organization or in
>>> accordance with the minting algorithm)? Or is "a URN with some
>>> additional data appended" and therefore not really a URN
>>> anymore? Or is actually a URI (such as an HTTP URI) whose path 
>>> is the URN itself as just a string of characters embedded in
>>> the broader URI?
>> 
>> Despite the addition of fragment and query the URN itself remains
>> the same, even though the HTTP URI changes. ...
> 

Correct. As Martin suggested, one might refer to the
organizationally-assigned or algorithmically-determined URN as an
"absolute URN", but that doesn't change the fact that absolute+query
or absolute+fragment is still a different URN.

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: [email-masked]
Date: Tue, 26 Feb 2013 21:26:21 +0200

Hello,

> or absolute+fragment is still a different URN.

It is a different string, thus a different URI, as dictated by RFC 3986.

But it is the same URN, since fragment is not part of the Namespace  
specific string, as specified by the Alfred Hoenes' version of the URN  
syntax, or any other syntax specification which approves the current  
idea that fragment is not part of the NSS. And we know already that  
incorporating the fragment into NSS would cause a lot of avoidable  
trouble (for instance, many namespaces could not use fragment at all).

Of course this whole point is entirely theoretical. If IETF cannot  
live with the idea that URN and URN + fragment still identify the same  
thing and are in that sense the same URN, we can extend the syntax so  
that after NSS one can add fragment and / or query into the URN string  
itself. Then, but only then, URN and URN + fragment are different.  
Different in which way exactly should of course be discussed, and  
whatever we do with the chapter dealing with semantic equivalence in  
the URN syntax document. the text has to be drafted rather carefully  
so as to make it very clear to everyone what URNs identify. It is a  
bit alarming that even in this WG we tend to have diverse views on  
things - which may lead to dead ends that would have been avoidable.

It is important that IETF does develop URN syntax which is fully  
aligned with RFC 3986. Since there are no generally accepted  
guidelines on how to use fragment or query, people have been  
inventive. For instance in the Clarin project, "@" is used to indicate  
fragment (or part, as they call it). So a Handle

11858/1234@test=a

will be resolved to http://clarin.eu?test=a

But no other application but the Clarin Handle server will be able to  
process these Clarin Handles; all other systems out there would be  
perplexed by it. One might think of few reasons why creating this kind  
of URIs can be risky.

Juha

>
> Peter
>
> - --
> Peter Saint-Andre
> https://stpeter.im/
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBAgAGBQJRLOx/AAoJEOoGpJErxa2p9aMQAKoZUm0Uq0Sa0xQ23TsmKqhL
> i7C4PKkYJ8dpjKWJ65SVFTKn1jCr+td2oR+qNMny7MqGm58GTcmmkYmTPlCx39I8
> UhpAXw3jL9HXScLpdUCGtV3IPPuiMVT+FLHfqVdXm/sbew/ICe9KwDVmCYKEve0y
> fMzlW3PdQPPjncBjTFdHsPpmAVS/gn7NZ4sX7hD8q2bMn3Cym+bSjRRK8L4k/it8
> h+xGPOXi6waOE4EfZJcL2Sme+4u5uN98aSE6lBxa1J1J9L8ZzNU7z1hhdrv+iztH
> 1yT1QaS2jEj5uN7V8dhI6LkRf3yy883j8WKyBs4BvLYQG++C+hboQgyAuh/O7+O5
> LmHQtsLPQWjsNoawa2lceAs+MNcLVH3+PPkT6jJ3rxPDITYay/d/0Byr6SAPYs6Y
> mfiWg5w+1A69KSEu+n9nJnpPyqoUZBjJ8YfiiwIr1F/Ft1NjFD2+vpTKGFsj06Ry
> 0xwZH5WMinlpMhXLnztKvWBgx/9d0RbLlxsYNvzw7+v1NM0dMmaAWfmr90g1Xiyq
> h4t6RIaYRxkYsmEx3AEWDGw7jtV+rgUtI3Jf1qn8iS6R/BmIWYpyiKvuwVRu1Lke
> apGp7nFROuv7g2AntWPi7Ur8av6TfZijllcmKSwhxAVHeKj5JU+BHz+BeZvlN5js
> UbAwq+ORA7UFM8MtGlnR
> =EMFB
> -----END PGP SIGNATURE-----

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Feb 2013 20:53:20 +0100

> trouble (for instance, many namespaces could not use fragment at all).

I was actually talking about the query part; sorry for the confusion.

It's up to the URN or NIS syntax to describe what the query part means, 
and how it applies to different namespaces. But in *general*, adding a 
query part to a URI changes the resource being identified.

Best regards, Julian

---

From: Juha Hakala <[email-masked]>
Date: Wed, 27 Feb 2013 09:03:20 +0200

Hello,

On 26.2.2013 21:53, Julian Reschke wrote:
> It's up to the URN or NIS syntax to describe what the query part 
> means, and how it applies to different namespaces. But in *general*, 
> adding a query part to a URI changes the resource being identified.

You have to be careful with the word "identify" in the URN context.

 From the URN point of view, the resource identified does not change 
when a query is added, if queries are used to facilitate the services 
specified in RFC 2483. All these services are related to one and only 
one identified resource.  A user can request for instance the resource 
itself, metadata about it, its current URL or URLs, and so forth. In 
practice, HTTP URIs (locations) the URN resolves to and the things that 
are being retrieved change, but the resource identified by the (location 
independent) URN does not.

I suppose it will not be possible to prevent URN implementers from 
constructing systems in which query is used to enable identification of 
multiple resources using the "absolute" URN as the starting point. But 
if the URN syntax says that the query is not part of the namespace 
specific string, then such an URN implementation is violating the syntax 
specification. And IANA should not approve namespace registration 
requests which use query in such non-standard way.

If query were made part of the namespace specific string, we might end 
up discussing whether a resource can be identified both by its absolute 
URN and the absolute URN + query indicating URI to resource service. Or, 
if a metadata record describing a resource can be identified by the URN 
of the resource + query indicating URI to URC service. It might be a 
good idea to avoid such a debate.

All the best,

Juha

-- 

  Juha Hakala
  Senior advisor

---

From: "Svensson, Lars" <[email-masked]>
Date: Wed, 27 Feb 2013 08:37:46 +0000

Juha, Julian, all,

On February 27, 2013, 08:03, Juha wrote 
> >> specific string, as specified by the Alfred Hoenes' version of the
> >> URN syntax, or any other syntax specification which approves the
> >> current idea that fragment is not part of the NSS. And we know
> >> already that incorporating the fragment into NSS would cause a lot
> of
> >> avoidable trouble (for instance, many namespaces could not use
> fragment at all).
> >

I think this depends on your urn namespace. In my opinion, we need to differ between queries that "operate on the urn itself" (in lack of a better description) and queries aimed at passing information to resolution services. As to the question if the query is part of the urn or not, my reading of rfc 3986 §3.4 is that it _is_ part of the urn
[[
The query component is indicated by the first question mark ("?") character and terminated by a number sign ("#") character or by the end of the URI.]]
If the query component ends at the end of the URI, it must be part of it.

Regarding the difference between queries in the context of the urn itself and in the context of resolution services, I made up a case to illustrate my point (thereby deliberately moving away from urn:nbn and urn:isbn).

In order to ensure that we can always identify integers in a location-independent way, we have registered the urn namespace 'int'. The NSS is defined as eight hexadecimal digits. Examples would be urn:int:0000000F ; urn:int:A0001234 . By doing this, we have unique names for those numbers and nothing more.

1)	Now we want to define other representations (decimal, octal, binary) of those hex numbers, and we do that by adding query parts to the urn. Example: urn:int:0000000F?base=10 would identify 15 (decimal). In this case adding the query does not change the resource (the integer remains the same, the query merely changes its representation). In this case we have two different URIs identifying the same thing (the number 15 = 0x0F = 00001111 = .)

2)	We also want to be able to identify the next and the previous integer and we do that through the relations "next" and "previous". As an example: urn:int:0000000F?rel=next would identify the same as urn:int:00000010. In this case the query part turns urn:int:000000F into another resource identifying something else (the number 16 = 0x10 = 00010000 = . ).

In the second case, the query part modifies the resource itself. You could say that the query "belongs to the urn".
In the first case, the query part modifies the representation of the resource. This should (in my opinion) be handled by a resolution service.

In this discussion we need to differ between queries in the context of the urns themselves and queries in the context of resolution services. Those are in my opinion not the same.

I hope this helps.

All the best,

Lars

***Lesen. Hören. Wissen. Deutsche Nationalbibliothek***
***Reading. Listening. Understanding. German National Library***

-- 
Dr. Lars G. Svensson
Deutsche Nationalbibliothek / Informationstechnik
http://www.dnb.de/
[email-masked]