Subject: Re: [urn] 2141bis equivalence rules

From: John C Klensin <[email-masked]>
Date: Fri, 12 Dec 2014 15:13:54 -0500

Sorry I've been largely tuned out for the last month.  Too many
other things going on that had very short deadlines and that
necessarily got in front of URNs in the queue.  I will try to be
fully caught up --and get a new version of the "syntax only"
document posted-- within the next week (it would be sooner, but
I have to spend half of it traveling).  I'm glad to see that
others seem to be making progress and want to welcome Melinda on
board.

Where I have something to say, I'm going to try to respond to
threads and where they seemed to end up or break off, not
individual messages.  

Now, as to equivalence...

>From my perspective, there are two important things to remember.
They could lead to conflicting conclusions but I hope we can
figure out a balance (and, fwiw, think that the current version
of 2141bis may express that).

First, the situation we have gotten ourselves into with URNs is
that a lot of namespaces are going to be invented elsewhere and
may be just URN instantiations of naming or identification
systems invented and controlled elsewhere, many of them very
widely well entrenched.  If we establish anything resembling a
URN-wide set of canonicalization rules (e.g., "remove hyphens")
or strong syntax restrictions about anything that appears in the
string once one gets past the NID (i.e., the NSS and anything
that follows it, for which see elsewhere), we are going to end
up, in big trouble or worse.  Phrases like shooting ourselves in
the foot apply.   However, if carried to its logical conclusion,
everything after the scheme name has to be treated as an opaque
blob, modulo maybe case-insensitivity in the NID.  None of the
other URI syntax rules apply and, in particular, no special
character, even "%", has special meaning that can be
interpreted, or even parsed as an introducer of something
specific, by a processor that doesn't know the details of the
particular namespace.

We considered that although not quite in those terms.  It has a
certain appeal and the "URNs are not URIs" approach can be seen
as a less-extreme version of that thinking.  The WG decided not
to go that way, so fine.

The other issue is that URNs are supposed to be about _names_.
If one examines one of the parts of the RFC 3996 URI spec that
led us to reach the "syntax only" conclusion, one finds a
collection of equivalence tests with different degrees of
stringency with schemes and/or processor able to choose among
them (some readers think they constitute an ordered list with
choices made about how far down the list to go, others think
they are just a menu, but we don't need to deal with that
further for URNs).   However, the assumption is that, for most
purposes, the ultimate equivalency test is to go retrieve the
object to which the URI points: if two URIs point to the same
object, then they are equivalent.  URNs are, by definition,
names.  Equivalence tests have to be made on the names because
there is no guarantee there is any object to be retrieved.
Indeed, different applications of a given name may yield
different objects (or information about objects, etc., etc.).

In a more perfect world, we might have a typology of equivalence
operators over the set of URIs.  Then we could ban the
standalone term "equivalent" and talk about foo-equivalence,
bar-equivalence, and so on.  For those who are fond of arguments
about "persistent", we could probably do the same job for
"change" (perhaps defined as "what cool URIs don't do"):
alpha-change, beta-change,...  Probably too late for that now
and not this WG's job, at least if we want to accomplish
anything before the end of the century.

If we take the two issues above together and don't like the
"opaque after the scheme name" approach, then we can state the
current rule without using terms like "equivalent".  The
equivalence question can be restated as "for two URNs to be
equal, the names must be identical except for case-independence
in the scheme and NID identifiers.  Then the question is what
makes up the "name". I (and, at least at the moment, 2141bis)
answer that question by saying that the "name" is the scheme id,
the NID, and the NSS (either including or plus the p-component
if it is present).  f-components and q-components, if present,
are supplemental, qualifying, or directive information that
supplement the name but are not part of it.

Whether p-, f-, or q-components are allowed is a per-NID issue.
If a parser accepts one for an NID that can't make use of it,
that is not a seriously bad thing although we probably should
have some guidance about whether an NID-specific (or
NID-knowledgeable) processor encounters one it doesn't
understand should ignore it or take some other action.

Of course, particular namespaces might impose somewhat different
rules on the whole string, but that would be about
URN-string-equivalence, not URN-equivalence.

Now, in that context...
  
--On Wednesday, November 19, 2014 13:35 +0200 Juha Hakala
<[email-masked]> wrote:

>...
 > On 19.11.2014 12:27, "Martin J. Dürst" wrote:
>...
>> But if we take ISBNs just as an example for a future
>> registration,  then I think it's a very valid idea to e.g.
>> say that ISBNs can contain  hyphens, but these are removed
>> when constructing an urn:isbn: URN. It  may not always be the
>> best solution, but it would be strange to reject  such as
>> solution from the get-go.

Actually, maybe not so strange.  As a superficial example, I
note that "Dürst" and "Duerst" are equivalent.  You have a
strong and reasonable preference for the first but the
observation that most people treat them as equivalent doesn't
make them "the same".   And mechanical rules about what to
remove or transform don't work unless one wants to end up
confusing the not-so-famous poet Göthe with the more famous
Goethe.

> IMO this is something which should be considered whenever a
> URN namespace is registered. The ISBN community could afford
> to be liberal since all applications which deal with ISBNs
> know that hyphens are to be ignored when ISBNs are processed.
> But some other communities may have valid reasons for
> providing additional guidelines for how to express their
> identifiers as URNs (that is, something that goes beyond the
> simple percent encoding).

Sure.  For example, if we refuse to allow "/" in URNs (as the
introducer for nominal p-components) we would give the portion
of the DOI community that wants to embed their identifiers in
URNs three choices:

(1) Write the "/" character as %2F.
(2) Declare a pox on our house and use "/" anyway
(3) Return to denouncing URNs as an ill-conceived system
developed by people who had an incomplete model and no clue.

>From a certain amount of experience, I think the first is
unlikely, as would following any suggestions we might make about
an alternate character to be used for their namespace only.  I'm
not sure which of the other two is worse.  When 2141 was
written, it was possible to ignore DOI issues.  Anyone who
thinks we should be doing so today should be talking with the
RFC Editor, the IAB, and the Nomcom.

>>> and that can be done by creating namespace
>>> specific rules for lexical equivalence which says that
>>> hyphens must be ignored.
>> 
>> Yes, it can. But we should write our document so that the
>> other choice  (eliminating gratuitous equivalences by
>> restricting the permitted  syntax) is at least also
>> permitted, if not preferred.

Well, I think one can write a rule that is very similar to one
we have used elsewhere.  It is more or less that a given string
has a preferred/canonical form, that the form looks like
so-and-so for this particular namespace, and that using other
forms is likely to cause bad behavior including some things not
matching that probably should.  But for us to go to, e.g.,
another standards body and say "despite the fact that your
Standard allows these flexibilities in the identifiers it
specifies, were are not going to allow them when those
identifiers are embedded in URNs" strikes me as inviting one of
those "and who do you think you are?" situations as well as a
hunt for the protocol policeperson and the supporting army.

>...

> Valid and invalid / canceled ISBNs are tagged differently in
> MARC 21 records. In this example

> It is important to provide canceled ISBN since that ISBN is
> often the one printed in the book.
>...
>> *020* 	*##**$z*0835200028 :*$c*$10.00
>> *020* 	*##**$a*0835200019*$q*pbk.*$c*$2.95/
>> /
>...

>>> It is important to show that they are equivalent (identify
>>> the same resource) since both versions of the URN:ISBN may
>>> be actionable in the Web. For instance, the publisher may
>>> prefer to use the invalid identifier and libraries the
>>> correct one.

Again, let's distinguish between "equivalent using a certain
criterion" (in this case, only one resource is associated with
the two names) and "names are equal", which those strings
clearly are not.  Note also that I don't  think p-components,
much less f- or q-components, have anything to do with that.

>...
> In your example there is nothing wrong, according to the
> catalogue of Der Deutschen Nationalbibliothek. The book just
> happens to have both ISBN-10 and ISBN-13, which have different
> check sum calculation algorithms. In the record retrieved by
> 
> https://portal.dnb.de/opac.htm?method=simpleSearch&query=978-1
> -449-32188-9
> 
> neither ISBN has been tagged as invalid: they are in separate
> tag 020 $a occurrences in the metadata record. And according
> to the rules of the ISBN community, it is OK to give a book
> both ISBN-13 and ISBN-10 (or generate automatically one from
> the another). This is necessary during a transitory period
> since some organizations may still have outdated systems which
> cannot deal with ISBN-13.

>... 
> ISBN and other semantic identifiers make it easy for humans to
> spot certain kinds of errors (for instance, it is easy to see
> if there is a typo in the publisher code). Machines are rarely
> capable of this kind of analysis because they do not dig deep
> enough (for instance, they never compare the publisher name
> and the publisher code in ISBN).

Well, they could even if it is not the practice to do so.  But I
can't see any reason to try to turn that into a URN problem
rather than a per-namespace one that is relevant only to
processors for/using that namespace.

best,
   john