Subject: [urn] URN standard

From: "Leslie Daigle (ThinkingCat)" <[email-masked]>
Date: Tue, 02 Jun 2015 09:20:49 -0400

Some of John’s comments through the last week have caused me to have a 
sense of deja vu and also to wonder what is a standard, anyway.

I think John and I agree on facts but diverge in conclusion.

[John Klensin wrote, on another thread:]
> At the time 2276, and especially 3401ff, were written, many
> people assumed that _all_ namespaces using the "urn:" scheme
> would behave this way, i.e., that this approach is very similar
> to the first one but loosens the binding between namespace and
> actual resolver by introducing an intermediate level
> resolver-finding and syntax-finding layer, thereby eliminating
> the most important of the persistence problems.

My perspective is that we didn’t only assume that, we believed that 
the URN standard was the syntax plus the means of finding the resolution 
system (DDDS).

20 years ago, we were at least as stuck on the questions of “what’s 
a name” and “how do you achieve persistence” as we are today in 
this discussion.  Can you achieve it with sub-second resolution 
(Mitra-just-Mitra’s requirement); can you prove it will persist, etc.

It seems the IETF is not very good at finding answers to information 
system questions.  We only broke free of the log jam in the 
then-Applications Area when we found a technical framework to specify.  
Given the URN (syntax), what would be standard operations on it (DDDS).

And, we (the IETF) called that the URN standard.

>
> When combined with the observation that not all URNs used today
> work that way, that view of things is one of the reasons I (and
> maybe others) periodically make comments that we should have
> left "URN" as an abstract term paralleling "URL" and used
> something like "ddds:" where we now use "urn:" and expect that
> treatment.   In more practical form, it is a key reason why
> discussions about putting strong references to 2276 and/or 3401
> into 2141bis cause me (at least) some anxiety.

Indeed, we probably should have done that.  But, we didn’t.  And, here 
we are.

I believe that, if you want to deprecate DDDS in its role for URNs, that 
takes an IETF standards action.

If this group wants to solve the information systems problems of naming 
in the Internet differently, then it probably needs to call them 
something _else_ at this point.

Leslie.

-- 

-------------------------------------------------------------------
Leslie Daigle
Principal, ThinkingCat Enterprises
[email-masked]
-------------------------------------------------------------------

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Tue, 02 Jun 2015 08:45:31 -0600

On 6/2/15 7:20 AM, Leslie Daigle (ThinkingCat) wrote:
> My perspective is that we didn’t only assume that, we believed that the
> URN standard was the syntax plus the means of finding the resolution
> system (DDDS).
>
> 20 years ago, we were at least as stuck on the questions of “what’s a
> name” and “how do you achieve persistence” as we are today in this
> discussion.  Can you achieve it with sub-second resolution

Clearly I need to re-read all of the relevant documents (I suppose that 
would be at least RFCs 2168, 2915, 2276, 3401-3406 - any others?), 
because I haven't know about that original intent until just now.

> takes an IETF standards action.

Perchance. But does updating the URN syntax document require us to 
revisit the entire "system" as you have outlined it above?

> something _else_ at this point.

That's sounds like a larger effort than anyone has energy for.

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: John C Klensin <[email-masked]>
Date: Tue, 02 Jun 2015 13:42:07 -0400

--On Tuesday, June 02, 2015 09:20 -0400 "Leslie Daigle
(ThinkingCat)" <[email-masked]> wrote:

> Some of John's comments through the last week have caused me
> to have a sense of deja vu and also to wonder what is a
> standard, anyway.
> 

Or maybe not even completely about the facts and conclusions,
because some of the actual text is inconsistent with your
thinking at the time (as you report it and that I have no reason
to question).

>> would behave this way, i.e., that this approach is very
>> similar to the first one but loosens the binding between
>> namespace and actual resolver by introducing an intermediate
>> level resolver-finding and syntax-finding layer, thereby
>> eliminating the most important of the persistence problems.
> 
> My perspective is that we didn't only assume that, we
> believed that the URN standard was the syntax plus the means
> of finding the resolution system (DDDS).

Rather than quibble about "we", let's assume that for now (I
question it further below).  Then, from my perspective, "we"
have an interesting problem or set of problems.  The XML
community has created URIs using the "urn:" scheme that ignore
that standard/connection.  The XMPP community has created URIs
using the "urn:" scheme that ignore that standard/connection.
So has the library community and a bunch of others.  There are
millions of URNs out there that do not use DDDS and therefore do
not conform, almost certainly far more, and far more visible,
than the URIs with scheme "urn:" that do conform to the DDDS
linkage.

So, what should we do?

(1) One option is to freeze things as they are, reaffirm the
linkage, tell everyone with those other uses that they are
non-conforming, evil, and should, in the words of a recent
draft, get off the URN lawn.

(2) Another is to accept the breaking of that linkage as a fact
on the ground, construct a 2141bis that makes clear that DDDS is
one way to resolve URNs, that there are other ways, and that
there are some UNRs that are really indicators, flags, or
abstract designators that don't resolve.

(3) A third is to become even more aggressive than the second,
assume that, after more than a dozen years after 3401ff were
published, the IETF has some obligations to the opposite of
"[deployed and] running code".   With this option, we recognize
how little DDDS has been used for URNs in practice.   We also
note that no one to my knowledge has even discussed updates to
3401ff (including 3406) that would supply enough definitional
material to pass the pieces of a URN parse tree that contained
some or all of r-components, q-components, or f-components into
an RFC 3402 regular expression, much less analyzed how pieces of
r-components that specify resolver instructions or actions would
mesh with the DDDS discovery model.  The logical result there
would be an Applicability Statement about the relationship
between the "urn:" scheme and DDDS that is consistent with
common practice as of 2015 and reasonably expected practices
going forward.   I don't know whether development of that A/S
would require protocol work leading to DDDSbis or not; I do know
that neither is on the WG's charter at present.

FWIW, I note that the likely, almost certain, consequence of
choice one would be that those other communities would say some
rude words about us and/or the IETF and then proceed to do
whatever seems sensible to them.

> 20 years ago, we were at least as stuck on the questions of
> "what's a name" and "how do you achieve persistence"
> as we are today in this discussion.  Can you achieve it with
> sub-second resolution (Mitra-just-Mitra's requirement); can
> you prove it will persist, etc.

Yes, I have noticed that.

> It seems the IETF is not very good at finding answers to
> information system questions.  

But good at getting itself into places that require answers to
those questions, even the subset of them that what we would now
call the information science community (and, in some cases,
parts of the philosophy community) concluded centuries ago were
unsolvable and/or unanswerable.  I'm told that a WG was even
shut down over that set of problems.  I've believed for years
that decision enabled useful progress that was clearly not
occurring with the original model.  I'm beginning to have my
doubts.

> We only broke free of the log
> jam in the then-Applications Area when we found a technical
> framework to specify.  Given the URN (syntax), what would be
> standard operations on it (DDDS).
> 

And yet we failed to update 2141 to say "it isn't a URN if it
doesn't use DDDS".  The Knoxville report (reference [7] in RFC
3404, http://www.dlib.org/dlib/february96/02arms.html) contains
a forward pointer to the DDDS work and some handwaving, but
stops far short of establishing or recommending that URNs be
limited, even in the short term, to those resolved by DDDS.  The
DDDS specs tell how to get a record for a particular URN
namespace into URN.ARPA and how it is set up (at least given
2141 URN syntax and restrictions), and says that URNs registered
according to RFC 3406 are candidates for entries in URN.ARPA
[RFC 3405, Section 3.2] but do not impose that registration as
even a temporary requirement on something being a valid URNs.
RFC 3406 seems quite clear whether what it says was the intended
meaning or not: the last paragraph of Section 1 reads:

   "Note that this document restricts itself to the description
   of processes for the creation of URN namespaces.  If
   "resolution" of any so-created URN identifiers is desired, a
   separate process of registration in a global NID directory,
   such as that provided by the DDDS system [RFC3401], is
   necessary.  See [RFC3405] for information on obtaining
   registration in the DDDS global NID directory."

That not only doesn't say "it isn't a URN unless it goes through
DDDS or is otherwise bound to URN.ARPA", but I take it to mean
both that some URNs don't resolve.  While it does require that a
global NID directory exist, it does not require that there be
only one of them, DDDS is identified as an example, not a
requirement, and the registration comment is about that
particular NID directory only.  One could have nailed that
situation down further by requiring that all URN NIDs have
entries in URN.ARPA for as long as the DNS lasted, established a
form of NAPTR record (or some other record) that would have
returned a clear "don't even think about resolving this"
indication, provided for evolution to records other than NAPTR
(something that has mostly been done without explicit binding to
URNs), and established a transition procedure for representing
the who database in some way alternative to the DNS (i.e.,
establishing an abstract database and is structures and
properties, but making sure that the DNS representa6ation was
only one such representation 

 Aside: RFC 3401 and 3403 take some small steps in this
	direction -- I don't think they are sufficient, but YMMD
	and I don't think they would be very hard to fix/upgrade.

RFC 3986 could also have imposed a requirement, most logically
in the section where it says that not all URNs are part of the
"urn:" scheme.  But it didn't either.  So we have had a _lot_ of
chances to clearly establish the requirement of a binding
between URNs (or resolvable URNs) and "the" IETF-designated NID
database.  We haven't done so.  For the record, I believe that
happened because folks were being super-careful to preserve
persistence by not tying things to particular formats or
mechanisms -- a goal that I support.  But the bottom line is
that, however much the bindings may have been intended and were
considered part of the oral tradition associated with "the URN
standard", it just doesn't seem to be in the Standard itself.
In other words, while I understand your intent and
understanding, nothing in the relevant text(s) seems to me to
make DDDS or those linkages a requirement.

>> When combined with the observation that not all URNs used
>> today work that way, that view of things is one of the
>> reasons I (and maybe others) periodically make comments that
>> we should have left "URN" as an abstract term paralleling
>> "URL" and used something like "ddds:" where we now use "urn:"
>> and expect that treatment.   In more practical form, it is a
>> key reason why discussions about putting strong references to
>> 2276 and/or 3401 into 2141bis cause me (at least) some
>> anxiety.
> 
> Indeed, we probably should have done that.  But, we didn't.
> And, here we are.

Yes.  See three options above.

> I believe that, if you want to deprecate DDDS in its role for
> URNs, that takes an IETF standards action.

I agree.   However, possible applicability statements aside (and
they require IETF standards actions too), the above, and the
present language in 2141bis (and 2141bis and 3406bis before I
got involved with the writing) effectively amount to "URNs can
use DDDS if they like, but it is not required and no strong
recommendation --one way or the other-- is being made."   Unless
that changes, there may be arguments for either updating or
deprecating DDDS (and some of those arguments are at least
implicit above), there is no requirement to do so.

As a token of good will --which I actually do feel although I
fear it isn't clear from this note -- if we can get 2141bis and
friends wrapped up and, after that, you decided it was worth
revisiting DDDS, cleaning up or clarifying some language that I
don't think has stood the course of time, adding bits as needed
to accommodate 2141bis URNs rather than just 2141 ones, and
making a strong applicability statement, I'd be happy to try to
find time to collaborate.   I've got a lot of notes from going
through the documents in the last few months and can probably
find those I made myself during the 1996-2002 period.  So far, I
think everything necessary could be done from an updating and
clarifying document and that actual revision of 3401ff would not
be needed -- the task is not monstrous. 

> If this group wants to solve the information systems problems
> of naming in the Internet differently, then it probably needs
> to call them something _else_ at this point.

Well, sure.  But see comments about (1) above including the
prediction of likely consequences.  I also continue to believe,
as I believed when it caused me to take actions long ago that
were perceived as precipitous, that we need to figure out how to
pragmatically work around those information systems/ information
science problems, especially those that represent centuries of
experience about things that can be handled by conventions, but
not "solved".   That belief is based now, as it was then, on a
conviction that getting actual solutions to those problems into
the critical path will transport us into a rathole of
unimaginable depth.

best,
      john

---

From: "Leslie Daigle (ThinkingCat)" <[email-masked]>
Date: Wed, 03 Jun 2015 17:35:46 -0400

Thanks for the thoughtful reply.  As always, a lot to chew on.

If I can summarize in tweet-length, I get:

URN standard did not write down that they require DDDS
Many urn: namespaces diverge from intention; none use DDDS
URNs are a concept and a standard
What do we do now to align concept, standard and “urn:” in the wild.

As difficult as it is to align “urn:” and “standard”, when we 
start bringing in “concept”, we’re cycling, IMO.

This is where I have extreme difficulties with Keith’s desire to have 
everybody resolving everything, with no one able to preferentially hold 
metadata for an item:  it’s “concept”.  An interesting one, but 
neither where URNs were standardized, nor part of the “urn:” in the 
wild part of the problem.

Leslie.

On 2 Jun 2015, at 13:42, John C Klensin wrote:

> _______________________________________________
> urn mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/urn

--

---

From: John C Klensin <[email-masked]>
Date: Wed, 03 Jun 2015 18:50:14 -0400

--On Wednesday, June 03, 2015 17:35 -0400 "Leslie Daigle

> Many urn: namespaces diverge from intention; none use DDDS

I don't know enough to assert "none", but certainly not a lot of
widely-used ones.

> URNs are a concept and a standard

See below.

> What do we do now to align concept, standard and "urn:" in
> the wild.

Yes, it seems to me that is the challenge.  To make it a little
worse, as evidence by the comment to the effect that only some
URNs are part of the 'urn:' scheme" in 3986, we have:

 * A very broad concept about persistent,
	location-independent, named identifiers.  That concept
	comes with a bit more ambiguity about what "persistent"
	and "location-independent" mean.  A problem we don't
	have right now, but might, is whether that concept is
	necessarily also bound to URIs.  In particular, DOIs,
	which the RFC Editor and IAB have chosen to endorse and
	use, are certainly persistent, location-independent
	names and, while there is a URI (but not URN) embedding,
	they are not natively URIs.
	
 * A much narrower concept, involving URNs linked to a
	DDDS-based resolver directory.
	
 * The standard, now and future.

> As difficult as it is to align "urn:" and "standard",
> when we start bringing in "concept", we're cycling, IMO.

Certainly it makes it harder to have a conversation that
converges.

> This is where I have extreme difficulties with Keith's
> desire to have everybody resolving everything, with no one
> able to preferentially hold metadata for an item:  it's
> "concept".  An interesting one, but neither where URNs
> were standardized, nor part of the "urn:" in the wild part
> of the problem.

Agreed.  That is also why I'm very anxious to defer that set of
issues and concentrate on what we can do now, without opening
those issues.  I would love to see someone do the work implied
by Keith's broader concepts (and those of others), but don't
want the potential for that work to block us from moving forward
on "urn:" as a practical standard that reflects or enhances
existing practice.

     john

---

From: Keith Moore <[email-masked]>
Date: Thu, 04 Jun 2015 07:58:49 -0400

On 06/03/2015 05:35 PM, Leslie Daigle (ThinkingCat) wrote:
> everybody resolving everything, with no one able to preferentially 
> hold metadata for an item:  it’s “concept”.  An interesting one, but 

I haven't had time to dig through my list archives yet, but the best of 
my recollection is that certainly at the time of the original URN WG, 
and actually even by 1995 or so, it was widely understood that there 
should be no single resolution service for any URN namespace - that 
while we fully expected that many URNs would be resolvable, and also 
that some resolution services would be associated with particular 
namespaces (presumably via a DDDS-like mechanism), resolution of URNs 
should not depend on their being a designated resolution service for any 
particular namespace.   This was important both for long-term viability 
of URNs (old URNs should not be deemed useless just because a resolution 
service once-associated with that namespace had ceased operation) and 
for security reasons (the namespace should not inherently be in a 
position to control access to the resources to which it issued names, 
since the role of namespace is not one of managing access rights - to 
the extent that this is done, it should be done by the owner of the 
content).

However, just to try to clarify, I don't have a desire for "everybody" 
to resolve "everything".    Instead, I think that:

- It should be possible for parties other than those authorized by a 
namespace to operate services that accept that namespace's URNsas input 
and return locations, resources, and/or metadata associated with those URNs;
- These non-authorized services should be able to use the same 
resolution protocols that namespace-authorized services use;
- Clients/users should be able to select among namespace-authorized and 
non-authorized services so that they do not need to rely entirely on 
authorized services to obtain information about resources named with 
URNs; and
- Determination of authenticity/provenance of information associated 
with URNs should be accomplished using digital signatures/certificates 
rather than by virtue of having been obtained from a 
namespace-authorized service (so, for instance, that signed metadata can 
potentially be cached by and returned by non-authorized services, and 
that metadata once signed by the namespace cannot later be repudiated).

I'm not sure what "preferentially hold metadata" means, but I don't have 
any objection to namespaces operating resolution services for the names 
that they coin, nor to clients being configured to prefer those services 
over non-authorized services.

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 04 Jun 2015 08:24:30 -0400

> 	they are not natively URIs.

IMO we should declare URIs (whether or not persistent) that don't begin 
with "urn:", or which do begin with "urn:" but don't use registered 
namespaces, as out-of-scope.   If someone eventually wishes to register 
doi as a URN namespace, then they would be in-scope, but that is no 
different in principle than registering isbn or any other set of 
persistent identifiers that wasn't originally designed to be a URN 
namespace.   The same rules and procedures apply.   Why wouldn't they?

> 	DDDS-based resolver directory.

This is another thing that I need to check, but my recollection is that 
we never intended for DDDS to be "the" sole resolver directory - it was 
only an example of a way to find resolvers.   More broadly, URNs are 
still valid even if they can't be resolved, and whether a particular URN 
can be resolved is something that can change (in either direction) over 
time.   So I don't find the "DDDS-resolvable URN" concept to be terribly 
useful.

> Certainly it makes it harder to have a conversation that

The first step in an engineering exercise is supposed to be to define 
the problem.    It seems to me that the problem definition for URNs was 
never really stated to a sufficient degree, and that some want to 
dismiss or change the part of the definition that was stated.   Then 
when there's some disagreement about some aspect of what we're trying to 
do, we don't have any agreed-on problem definition to refer to in order 
to evaluate competing ideas.   So some people want to refer to 1737, and 
others want to ignore it. Some people want to cite the spectrum of 
existing usage of URNs as the problem definition, even though those are 
not representative of the problem that we were trying to solve in the 
original URI/URN WGs.   And some of us who were actively participating 
way back when want to rely on the shared concept that we believed many 
of us had arrived at (through years of hard work) even if some of that 
concept wasn't ever captured in a document.  etc.

> on "urn:" as a practical standard that reflects or enhances

I'm fine with deferring that set of issues except when we start setting 
bad precedents or going in a direction that breaks something 
important.     But again, it's hard to argue that a precedent is bad, or 
that something important is being broken, without a problem definition 
to refer to.

Keith

---

From: Michael Mealling <[email-masked]>
Date: Thu, 4 Jun 2015 09:13:41 -0400

DDDS author here....

DDDS is a general system that can resolve any syntactically regular string. ENUM is a proper subset of DDDS. It was never my nor anyone else's intention for DDDS to be the only resolution system for URNs. Indeed there are several URN schemes that can't be resolved via DDDS because the authority hierarchy is far to difficult to setup (UUIDs for example). In other cases the entity that registered and manages the scheme doesn't find public DDDS style resolution all that useful so they would never setup their piece of the system.

Also, remember that DDDS isn't a final resolution system so much as a way of discovery AUTHORITATIVE services that CAN tell you things about a URN (or any other identifier). There are shortcuts in the system to optimize for common queries (find me one or more instances of this Resource) but there's no reason you have to use that. I see no reason why you couldn't use DDDS to quickly find a Z39.50 server to answer all sorts of fun questions about that Resource). DDDS was designed to respect the hierarchy of authority embedded in the urn:<scheme>: system. 

Has anyone come up with a set of requirements for what they want their 'resolution' service to do? 

Is the issue that DDDS requires knowledge of DNS as opposed to URL encoding the URN and sticking it in a URL as many want to do? A service that hides DDDS authority chains behind a HTTP 1.1 gateway seems easy enough to write.

-MM

> On Jun 4, 2015, at 8:24 AM, Keith Moore <[email-masked]> wrote:
> 
> This is another thing that I need to check, but my recollection is that we never intended for DDDS to be "the" sole resolver directory - it was only an example of a way to find resolvers.   More broadly, URNs are still valid even if they can't be resolved, and whether a particular URN can be resolved is something that can change (in either direction) over time.   So I don't find the "DDDS-resolvable URN" concept to be terribly useful.

---

From: John C Klensin <[email-masked]>
Date: Fri, 05 Jun 2015 14:26:15 -0400

--On Thursday, June 04, 2015 08:24 -0400 Keith Moore
<[email-masked]> wrote:

> IMO we should declare URIs (whether or not persistent) that
> don't begin with "urn:", or which do begin with "urn:" but
> don't use registered namespaces, as out-of-scope. 

I think we have done the latter.  Indeed, I think we've don't
something more strongly by declaring them invalid.  That
assertion is part of note about experimental namespaces in
Section 6, but could be moved to a more prominent place and
strengthened if someone thought it necessary and suggested text
and a place to put it.  

With regard to other URIs, I don't think there is anything in
any of our documents that suggests they are within scope.  If
you are aware of exceptions, please point them out.  On the
other hand, as far as our discussions are concerned, the WG's
declining to adopt the "URNs are not URIs" approach makes a
certain amount of discussion of the relationship inevitable much
as I'd like to minimize it.

>...
> This is another thing that I need to check, but my
> recollection is that we never intended for DDDS to be "the"
> sole resolver directory - it was only an example of a way to
> find resolvers.   More broadly, URNs are still valid even if
> they can't be resolved, and whether a particular URN can be
> resolved is something that can change (in either direction)
> over time.   So I don't find the "DDDS-resolvable URN" concept
> to be terribly useful.

I certainly agree and Michael's note certainly reinforces that
view.  However, I continue to see a logical problem in that
either the same resolution mechanism or resolver-finding
mechanism is applicable to all URNs (which we agree is not the
case and should not be) or resolver-finding is per-namespace.
I think you have strongly objected to the latter, so I'm not
sure where you think we should go or what words should go into
2141bis.

>...
> The first step in an engineering exercise is supposed to be to
> define the problem.    It seems to me that the problem
> definition for URNs was never really stated to a sufficient
> degree, and that some want to dismiss or change the part of
> the definition that was stated.   Then when there's some
> disagreement about some aspect of what we're trying to do, we
> don't have any agreed-on problem definition to refer to in
> order to evaluate competing ideas.   So some people want to
> refer to 1737, and others want to ignore it. Some people want
> to cite the spectrum of existing usage of URNs as the problem
> definition, even though those are not representative of the
> problem that we were trying to solve in the original URI/URN
> WGs.   And some of us who were actively participating way back
> when want to rely on the shared concept that we believed many
> of us had arrived at (through years of hard work) even if some
> of that concept wasn't ever captured in a document.  etc.

Suppose I agree with that as a statement of the meta-problem if
not the problem (personally, I'm coming to a much more general
conclusion, but it isn't helpful).  How do you think we should
proceed from it?  Presumably, mourning our fate and the bad
situation we are in is not particularly helpful in moving
forward.

> I'm fine with deferring that set of issues except when we
> start setting bad precedents or going in a direction that
> breaks something important.     But again, it's hard to argue
> that a precedent is bad, or that something important is being
> broken, without a problem definition to refer to.

Agreed.  But, if you don't believe we have an adequate problem
definition on which to proceed, we either have to face the
question about the meta-problem above, or we are at dead stop
until we fire up and complete a problem definition process.    I
don't see any energy for that.  Even if I did, I think it would
be no more likely to completely converge and reach consensus
than we were nearly 20 years ago.    

I really hate being in a situation in which the best way we can
characterize a collection that is (or is not) suitable for a URN
namespace is on the basis of "I know one when I see one" or,
perhaps better, "Keith, Leslie, and/or Michael know one when
they see one".  However, if that is where history has left us
and the notion of starting over with new definitions of the
problem (and of "persistence", etc.) is as implausible and
impractical as I think it is, then shouldn't we be concentrating
on how to move forward and, in particular what text should be
added to or changed in 2141bis?

    john

---

From: John C Klensin <[email-masked]>
Date: Fri, 05 Jun 2015 14:36:56 -0400

--On Thursday, June 04, 2015 09:13 -0400 Michael Mealling
<[email-masked]> wrote:

> DDDS author here....
> 
> DDDS is a general system that can resolve any syntactically
> regular string. ENUM is a proper subset of DDDS. It was never
> my nor anyone else's intention for DDDS to be the only
> resolution system for URNs. Indeed there are several URN
> schemes that can't be resolved via DDDS because the authority
> hierarchy is far to difficult to setup (UUIDs for example). In
> other cases the entity that registered and manages the scheme
> doesn't find public DDDS style resolution all that useful so
> they would never setup their piece of the system.
>...

Michael, I found this note/discussion extremely helpful, so many
thanks.

Question: Are the resolver/ DDDS questions confusing enough
people that we (presumably the WG) should be thinking about a
document that summarizes the situation as of 2015?  Presumably
much of such a document could be built from Michael's note and a
few other recent notes on the list.  If the answer is "yes",
does anyone have the time and inclination to do the work?

    john

---

From: "Hakala, Juha E" <[email-masked]>
Date: Wed, 10 Jun 2015 12:31:51 +0000

Hello, 

Comments below. 

> -----Original Message-----
> From: urn [mailto:[email-masked]] On Behalf Of Michael Mealling
> Sent: 4. kesäkuuta 2015 16:14
> To: Keith Moore
> Cc: [email-masked]
> Subject: Re: [urn] URN standard
> 
> DDDS author here....
> 
> DDDS is a general system that can resolve any syntactically regular string.
> ENUM is a proper subset of DDDS. It was never my nor anyone else's intention
> for DDDS to be the only resolution system for URNs. Indeed there are several
> URN schemes that can't be resolved via DDDS because the authority hierarchy
> is far to difficult to setup (UUIDs for example). In other cases the entity that
> registered and manages the scheme doesn't find public DDDS style resolution
> all that useful so they would never setup their piece of the system.

Library community has regarded DDDS as one option to set up resolver discovery service, but not as the only one. 

So far, the ambition level has been fairly modest; resolvers in the same namespace (like URN:NBN and URN:ISBN) may be aware of each other and harvest each others' URN - URL linking tables. 

> Also, remember that DDDS isn't a final resolution system so much as a way of
> discovery AUTHORITATIVE services that CAN tell you things about a URN (or
> any other identifier). There are shortcuts in the system to optimize for
> common queries (find me one or more instances of this Resource) but there's
> no reason you have to use that. I see no reason why you couldn't use DDDS to
> quickly find a Z39.50 server to answer all sorts of fun questions about that
> Resource). DDDS was designed to respect the hierarchy of authority
> embedded in the urn:<scheme>: system.

First the user should find out which questions the Z39.50 server can answer. These servers are complex, and can tell a lot about themselves and databases accessible via them. And this metadata should be machine understandable, in order to allow Z39.50 clients to adapt themselves to the searching facilities and (bibliographic) databases available. 

A few years ago, ANSI/NISO (National Information Standards Organization) launched Metasearch Initiative to support sharing of information about information retrieval services. The project was not completed, but draft standard Z39.92-200X is still available at 

http://www.niso.org/apps/group_public/project/details.php?project_id=56

Usage of DDDS was never discussed in this context, but we could have done that. 

> 'resolution' service to do?

Interesting question! 

RFC 2483 never intended to provide a full set of services. From a librarian's point of view, RFC 2483 is a good start, but a lot of important things are missing, and the number of missing things has grown all the time. 

The first problem is the data model: RFC 2483 assumes that there are resources and metadata (in URC format) about them, and 0-n copies of these resources. This model may be OK for some URN users, but it is far too simple for e.g. libraries. 

The model libraries are using / implementing starts from immaterial works like James Joyce's Ulysses. Libraries have metadata (including identifiers) about these immaterial things. And (metadata about) work may be linked to (metadata about) other works; for Ulysses it is necessary to establish a connection to (for instance) Homer's Odyssey and Portrait of the Artist as a Young Man. 

There are a lot of adapted versions of Ulysses, and also translations to other languages. In library vernacular, these are called expressions of Ulysses. We have metadata (and identifiers) about them as well, and routinely link metadata about expressions to metadata about works.

Each work and expression may have 0-n physical embodiments (manifestations) such as hardback or PDF versions of Ulysses. All these have their own identifiers. And of course manifestation metadata is linked to upper level metadata records.  

In this context there are a lot of potential requirements for resolution services. Just to give some examples, user may want to find metadata about a particular work, expression or manifestation; identifiers of all works related to a work; metadata about all expressions related to a work; metadata about all expressions related to one another (for instance, Finnish translations of Ulysses; there are two), manifestations related to a work or expression, or all locations of a manifestation. All these requests can be answered if there are stable links between works, expressions, manifestations and items. 

This kind of interlinking is essential for persistence. For instance, 100 years from now a user may find a reference from a scientific article pointing to an article in PDF published in 2013. He may not have SW with which to open a PDF document, but he may find out that this work is also available in a more modern file format which is still readable with the HW/SW tools available to him. 

Another issue with RFC 2483 is that it assumes that metadata (Uniform Resource Characteristics) is a single "blob" available in a single format (alas, URC never got past an idea; Dublin Core was established instead). Since applications are now much more sophisticated than they were almost two decades ago, this is no longer the case. A user may need descriptive metadata in a particular metadata format used for the resource in question (for instance, DCAT for research data, ONIX for commercial publications or EAD for governmental documents). Or, she may need just technical metadata (what HW / SW do I need to render this file), rights metadata (what can I do with this document after I have copied it on my computer) or preservation metadata (how is this new version of the document different from the original one with respect to look & feel and intellectual content).  
 
Other user communities are likely to have requirements which are beyond what the libraries need. 

Because resolution services will change over time due to changes in data models, technology etc. it is necessary to create a system where new resolution services and service parameters can be easily added. We also need a mechanism via which resolvers can share information about resolution services they support. Then applications such as library systems will know what resolution requests any given URN resolver can deal with. In Z39.50 context robots have been built in order to find out via trial and error what queries a Z39.50 server & database it connects to support. This can be done in URN context as well, but it would be better to share this information. 

> Is the issue that DDDS requires knowledge of DNS as opposed to URL encoding
> the URN and sticking it in a URL as many want to do? A service that hides DDDS
> authority chains behind a HTTP 1.1 gateway seems easy enough to write.

It is a pity that so far nobody has done it. But let's hope the situation will change. 

Juha

> > This is another thing that I need to check, but my recollection is that we
> never intended for DDDS to be "the" sole resolver directory - it was only an
> example of a way to find resolvers.   More broadly, URNs are still valid even if
> they can't be resolved, and whether a particular URN can be resolved is
> something that can change (in either direction) over time.   So I don't find the
> "DDDS-resolvable URN" concept to be terribly useful.
>

---

From: Michael Mealling <[email-masked]>
Date: Wed, 10 Jun 2015 11:42:31 -0400

> On Jun 10, 2015, at 8:31 AM, Hakala, Juha E <[email-masked]> wrote:
> 

Good.

> So far, the ambition level has been fairly modest; resolvers in the same namespace (like URN:NBN and URN:ISBN) may be aware of each other and harvest each others' URN - URL linking tables. 

I would like to be careful about terminology with respect to the word 'resolver'. A resolver is authoritative since it is either run by or delegated its authority by the registrant of the NID. Without that authority its just another database spouting its own opinion. 

> First the user should find out which questions the Z39.50 server can answer. These servers are complex, and can tell a lot about themselves and databases accessible via them. And this metadata shou…

That sounds like a great requirement for a Z39.50 server. I can see someone writing a 'profile' of Z39.50 for answering questions about URIs. But I'm not really sure that's germane to this discussion. 

> Usage of DDDS was never discussed in this context, but we could have done that. 

I wouldn't have. Its like discussing DNS while you're designing an Oracle API: You know DNS exists but you really don't need to spend much time talking about it.

> RFC 2483 never intended to provide a full set of services. From a librarian's point of view, RFC 2483 is a good start, but a lot of important things are missing, and the number of missing things ha…

Heh... It wasn't intended to be much beyond getting a few interesting services off the ground and start the ball rolling. 

> The first problem is the data model: RFC 2483 assumes that there are resources and metadata (in URC format) about them, and 0-n copies of these resources. This model may be OK for some URN users, b…

No where in RFC 2483 does it say that the services listed are the only ones or that the data model (if you can even attach that much formality to it) is the only one. Just because it doesn't talk about other ways to ask questions doesn't mean those other methods are wrong or illegal. So lets start with what you need.

> < general summary of Cataloging >

> This kind of interlinking is essential for persistence. For instance, 100 years from now a user may find a reference from a scientific article pointing to an article in PDF published in 2013. He ma…

OK. Here is where we have a slight breakdown in terminology that goes back a LONG way and even gets to Larry Masinter's use of 'persistence'. In your use case and in Larry's view  (sometimes) persistence of the thing you want to retrieve is far more important. If you can't retrieve the thing you are talking about then having a persistant identifier has greatly reduced, if any, utility. My view and the view of many of us those 20 years ago is that persistence of the thing the identifier identifies is 'someone else's problem'. That's where the Archiving department comes in. The Catalog can still contain the record and all that metadata comfortable in the knowledge that someone isn't going to reassign the URN to something else. Whether or not the actual resource it identifies still exists is a separate problem.

> Another issue with RFC 2483 is that it assumes that metadata (Uniform Resource Characteristics) is a single "blob" available in a single format (alas, URC never got past an idea; Dublin Core was es…

Might I recommend redacting any mention of RFC 2483 beyond using it as a list of possible classes of services meant for optimizing certain well known use cases? I don't think anyone anywhere is suggesting that 2483 limit any of those use cases. Is there a candidate system (protocol and data model) that does some of those things now?

> Other user communities are likely to have requirements which are beyond what the libraries need. 

Definitely... that was always our assumption.

> Because resolution services will change over time due to changes in data models, technology etc. it is necessary to create a system where new resolution services and service parameters can be easil…

And DDDS, as a system of discovering authoritative resolvers, supports adding new services and endpoint identifiers. But its not meant to encompass ALL of those things. DDDS isn't the resolver itself, its simply meant to find the authoritative one. E.g. you use DNS to find the web server to ask for the document, it doesn't give you the document itself nor does it give you the equivalent of an 301 redirect to tell you where to get it. It simply says "that server knows how to answer complex questions about the thing you are after using a complex and robust protocol".

> It is a pity that so far nobody has done it. But let's hope the situation will change. 

I'll play around with setting one up this weekend. 

-MM

---

From: Michael Mealling <[email-masked]>
Date: Wed, 10 Jun 2015 11:48:40 -0400

I can help someone write this if necessary but I was hoping my RFC writing days were over. ;-)

I think an 'update' would be useful since I'm seeing definite confusion about the 'standards' aspects of some of these documents. One observation is that, if you remember back when these documents were being written, new protocols and markup formats were flying around thick as flies. Many of these documents were written assuming that better metadata search and lookup protocols would be coming to the rescue. Everyone had this hope that someone would write a version of Z39.50 that mere mortals could understand and that Dublin Core would save us all from attribute naming hell. Sadly no such knight on a white horse came to save us...

-MM

> On Jun 5, 2015, at 2:36 PM, John C Klensin <[email-masked]> wrote:

---

From: "Hakala, Juha E" <[email-masked]>
Date: Thu, 11 Jun 2015 07:12:29 +0000

Hello,

There is no simple & efficient solution to the attribute naming problem. Things are actually getting worse: richer and richer metadata is needed to manage digital resources and rights related to them.

There is a modernized version of Z39.50. SRU has been an OASIS standard since 2014. Replacing ASN.1/BER with XML does make the document more palatable for some readers, but there are still all those search attributes rich library metadata supports. Dublin Core does simplify semantics a lot and it has been kept stable since 2007, but most people outside libraries may think that 70+ terms to describe resources is too much (libraries have several hundreds of them). 

I am afraid that the knight on the white horse is galloping away ;-), but there are no problems we librarians wouldn't be able to solve without his help. But we do need some support from IETF to make the identifiers do the tricks required.   

Juha

> Sent: 10. kesäkuuta 2015 18:49
> To: John C Klensin
> Cc: [email-masked]; Keith Moore
> I think an 'update' would be useful since I'm seeing definite confusion about
> the 'standards' aspects of some of these documents. One observation is that,
> if you remember back when these documents were being written, new
> protocols and markup formats were flying around thick as flies. Many of these
> documents were written assuming that better metadata search and lookup
> protocols would be coming to the rescue. Everyone had this hope that
> someone would write a version of Z39.50 that mere mortals could understand
> and that Dublin Core would save us all from attribute naming hell. Sadly no
> such knight on a white horse came to save us...
> 
> -MM
> 
> 
> >> DDDS is a general system that can resolve any syntactically regular
> >> string. ENUM is a proper subset of DDDS. It was never my nor anyone
> >> else's intention for DDDS to be the only resolution system for URNs.
> >> Indeed there are several URN schemes that can't be resolved via DDDS
> >> because the authority hierarchy is far to difficult to setup (UUIDs
> >> for example). In other cases the entity that registered and manages
> >> the scheme doesn't find public DDDS style resolution all that useful
> >> so they would never setup their piece of the system.
> >> ...
> >
> > Question: Are the resolver/ DDDS questions confusing enough people
> > that we (presumably the WG) should be thinking about a document that
> > summarizes the situation as of 2015?  Presumably much of such a
> > document could be built from Michael's note and a few other recent
> > notes on the list.  If the answer is "yes", does anyone have the time
> > and inclination to do the work?
>

---

From: John C Klensin <[email-masked]>
Date: Tue, 16 Jun 2015 06:00:39 -0400

--On Thursday, June 11, 2015 07:12 +0000 "Hakala, Juha E"
<[email-masked]> wrote:

>...
> I am afraid that the knight on the white horse is galloping
> away ;-), but there are no problems we librarians wouldn't be
> able to solve without his help. But we do need some support
> from IETF to make the identifiers do the tricks required.

Hi.

(personal opinion only)

I've found this discussion very useful in terms of understanding
both the context of the issues and the issues themselves.
However, I --and I hope all of us-- have a very specific and
narrow problem, which is to establish a syntax and registration
model in 2141bis that is sufficient to support whatever problems
or requirements are out there or that might be developed in the
reasonably near future.  

To that end and noting that 2141bis-12 has been posted...

* If what it specifies does not provide the "support from IETF
to make the identifiers do the tricks required", it would be
good (actually, it is probably vital) to know, really soon now,
what is missing or wrong and, in very specific terms, how it
might reasonably be corrected.

* Part of that support question is whether the current version
of the registration templates and descriptions allow the
necessary information to be supplied, require what should be
required, provide enough hints and justification about what
information is needed that we are likely to get that
information, and, where appropriate, are clear about what needs
to be supplied in-line and what can be provided by reference.
If anything is not clear or complete enough, this is the time to
say so.

* There are some examples in 2141bis to illustrate applicability
of r-components and q-components.  I think they may be wrong and
that they really need checking (and other ones
specified/provided as appropriate) by people who have a clear
idea of what they are intended to mean.  The text about where
those queries go is also not as clear as I would like it to be
and that may be part of the problem, but it is about as good as
Peter and have been able to do so far.  Text would be
appreciated.

* The text about persistence has been removed from 2141bis.  The
recent threads of conversations have convinced me (at least),
that an updated and up-to-date "theory of URNs" document would
be very useful, but that (i) it isn't in the WG's charter, (ii)
getting it into the WG's critical path would probably kill this
effort, (iii) putting arbitrary small bits of it into 2141bis is
likely to add to, rather than reducing, confusion, and (iii) we
should almost certainly postpone work on that document (or even
figuring how to organize that work until after 2141bis is
finished).  

* I discovered in a very helpful conversation last week (with
someone who still considers herself a librarian, by the way),
that there is a great deal of research going on about
identifiers in the Internet era, work that still pays attention
to non-electronic "resources" and that probes at what terms
like, e.g., "resource" are coming to mean.  The conversation
also convinced me even more of the wisdom of confining 2141bis
as much as possible to syntax and then moving the other
discussions to forums where the level of carefully-researched
knowledge is much higher than it is in the IETF.   Details
(including references and pointers) after I get my notes
together, but probably not until after 2141bis is locked down.
If anyone needs them for ongoing work that won't get in the way
to 2141bis, write me offlist.

I want to stress that I'm not speaking for the WG, the
Co-chairs, or even for Peter, but my assumption at the moment is
the penultimate version of 2141bis before WG and IETF Last Call.
In other words, we get the input asked for above, do one more
revision, and then should be ready to wind this effort up.
That schedule is also consistent with my/our not running out of
energy to keep waiting for the comments needed to make revisions.

I look forward to hearing from everyone who is involved and
interested.

best, 
     john

---

From: "Hakala, Juha E" <[email-masked]>
Date: Tue, 16 Jun 2015 14:23:48 +0000

Hello John; all,

(also a personal opinion only)

Considering just the URN syntax, the only library related requirements I am aware of involved URI fragment and query. We wanted to use the former to indicate locations within identified documents, and the latter to pass resolution related parameters to URN resolvers. Since our needs were relatively modest, URNBIS has already met our requirements and then went on to specify additional features.  And IMO there is nothing wrong with what 2141bis says (but I need to check the latest draft, just in case). 

I am not aware of any immediate need for newly minted URN features such as r-components, but library community may find uses for it later when our resolvers become smarter. As John says below, traditional identifiers are not and cannot be carved in stone. Digital publishing has changed and is still changing the ways in which ISBN, ISSN etc are used. Several systems are currently under revision, and in some cases changes may be substantial. Many identifier systems will specify how they can be made actionable as DOIs, URNs etc. New ISBN standard is one of the first examples of this.  

There are also new challenges, such as increasing need within publishing community and libraries to develop a standard-based solution for identification of logical fragments (assuming that URN + fragment can deal with physical ones, we still have a lot of cases when namespaces must develop their own solutions to identify logical parts of documents which lack proper physical encoding). 

Once the syntax revision is complete, we (I hope) need to take a closer look on mechanism for registering resolution services and parameters, and then specify to they are encoded as queries. How resolvers then deal with requests is a different matter; often it will be sufficient to know, which server(s) will be able to process the request and which protocol(s) they support. 
   
 Juha Hakala
 Senior advisor

 The National Library of Finland
 Library Network Services
 P.O.Box 26 (Teollisuuskatu 23)
 FIN-00014 Helsinki University

 Gsm: +358 50 3827678
 email: [email-masked]

________________________________________
Lähettäjä: John C Klensin <[email-masked]>
Lähetetty: 16. kesäkuuta 2015 13:00
Vastaanottaja: Hakala, Juha E; Michael Mealling
Kopio: [email-masked]; Keith Moore
Aihe: RE: [urn] URN standard

Hi.

(personal opinion only)

best,
     john

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Tue, 16 Jun 2015 09:01:51 -0600

Hi Juha,

On 6/16/15 8:23 AM, Hakala, Juha E wrote:
> Considering just the URN syntax, the only library related
> requirements I am aware of involved URI fragment and query. We wanted
> to use the former to indicate locations within identified documents,
> and the latter to pass resolution related parameters to URN
> resolvers. Since our needs were relatively modest, URNBIS has already
> met our requirements

That's good to know.

> and then went on to specify additional features.
> And IMO there is nothing wrong with what 2141bis says (but I need to
> check the latest draft, just in case).

Your careful review at this stage would be very much appreciated! :-)

Peter

---

From: "Svensson, Lars" <[email-masked]>
Date: Thu, 25 Jun 2015 12:19:03 +0000

Juha, all,

Another personal opinion:

> Considering just the URN syntax, the only library related requirements I am
> aware of involved URI fragment and query. We wanted to use the former to
> indicate locations within identified documents, and the latter to pass resolution
> related parameters to URN resolvers. Since our needs were relatively modest,
> URNBIS has already met our requirements and then went on to specify
> additional features.  And IMO there is nothing wrong with what 2141bis says
> (but I need to check the latest draft, just in case).
> 
> I am not aware of any immediate need for newly minted URN features such as
> r-components, but library community may find uses for it later when our
> resolvers become smarter.

While I agree with what Juha that the requirements of libraries are satisfied by 2141bis, I have a slightly different view on query and r-component. Juha says that "we wanted to use [...] the latter [e.g. queries] to pass resolution related parameters to URN resolvers" and that he is "not aware of any immediate need for newly minted URN features such as r-components". Aren't r-components exactly for passing resolution-related parameters to the resolvers?

/Lars

(who is now moving on to read 2141bis-12)

---

From: Keith Moore <[email-masked]>
Date: Thu, 25 Jun 2015 08:36:23 -0400

> While I agree with what Juha that the requirements of libraries are satisfied by 2141bis, I have a slightly different view on query and r-component. Juha says that "we wanted to use [...] the latte…

Yes.   The purpose of r-components is to pass parameters to resolvers, 
so that q-components can be used exclusively to pass parameters to 
resources, and there's no ambiguity about where a parameter is to be used.

Keith