Subject: [urn] Keith's review of draft-ietf-urnbis-rfc2141bis-urn-10

From: Keith Moore <[email-masked]>
Date: Thu, 26 Mar 2015 11:58:25 -0400

General summary:

This version of the document has several serious technical omissions and 
errors that should, according to RFC 2026 criteria, prevent its 
publication as a standards-track document until those omissions and 
errors are remedied.  Some of these omissions and errors appear to 
result from failure to consider use of URNs for their primary intended 
purpose of being identifiers for network-accessible resources.   It's 
fine to use URNs to name resources that aren't network-accessible, but 
the URN specifications should not be compromised in such a way as to 
impair their use to name network-accessible resources, and this document 
imposes several such compromises.

There are other choices reflected in the document that I think are poor 
technical choices that should be remedied, though they fall short of 
being "technical omissions" in the RFC 2026 sense.

The document also reflects, in at least one case, a failure to resolve 
an identified technical issue, which RFC 2026 requires be resolved.

Details:

Broadly speaking, the document has the following problems (in no 
particular order):

1. Use of p-component in URNs combined with use of relative URIs with 
URN as the base URI.

If relative URIs are permitted to be used with URNs, this has the 
potential to harm persistence of URNs unless there are constraints on 
how NSSs are assigned.   If an NSS under a particular NID has no 
internal structure, maintaining persistence for every URN assigned under 
that NID is relatively simple - just don't ever reassign an NID to a 
different resource.   But if the NSS does have structure that reflects 
relationships between resources, and those resources can ever be updated 
in a way that causes those relationships to need to change, there's a 
good chance that some combinations of base URNs + relative URIs that 
formerly resolved to valid URNs, will no longer resolve to valid URNs, 
or will resolve to URNs that don't have the same meaning/purpose as 
before.   Thus there will be a significant temptation to change the 
meanings of existing URNs, i.e. to rebind existing URNs to different 
resources than before.

There are lots of ways that a namespace can make relative URIs safe to 
use with its URNs.   For example,  if none of the resources named by a 
namespace are machine-parseable, there's no issue.  Or if the resources 
under a namespace are constrained to never change at all, there's no 
issue.   Or if NSSs are of the form x/y such that x is never reused 
(every new resource gets a unique x, and y is just used to differentiate 
between parts or facets of that resource), there's no issue.

The problem is trying to write a general rule that permits use of 
p-components in an NSS and permits use of relative URIs with those URNs, 
in such a way that doesn't break things and encourage URNs under that 
NSS to become non-persistent.

Perhaps the easiest way to fix the problem is this:   If a namespace 
assigns NSSs containing '/', it is responsible for ensuring that NSSs 
are assigned in such a way that relative URI references appearing in 
machine-readable resources named by its URNs are also persistent - and 
give examples of how this might be accomplished. (Of course, any 
resource can claim any URN as its base URI, and namespaces have no 
control over the behavior of resources that they don't name.  We can't 
stop the authors of such resources from doing truly stupid things with 
relative URIs, but we can discourage use of relative URIs with URNs that 
weren't designed to be used that way.)

IMO, this is a technical omission.

2.  The document fails to specify which parts of a URN containing 
f-components or q-components must be persistent.

For example, since URN x is required to be persistent, is x#y also 
required to be persistent?   Similarly for x?y and x/y.   Or do the 
persistence requirements only apply to some of these and not others?

IIRC, the reason RFC 2141 doesn't permit '/', '?', or '#' in NSSs is 
that we couldn't figure out how to define or ensure persistence in these 
cases in a way that made sense across a wide variety of use cases.   The 
current rfc2141bis document still fails to address these problems.

At one level the problems with p-components, f-components, and 
q-components are the same problem - how to ensure persistence of a 
structured identifier for a resource that can be revised over time, 
including potentially in ways that change its internal structure and/or 
relationships with other resources.   But at another level I suspect 
that these are subtly different problems.   If we're going to use 
p-components, f-components, and q-components in ways that are even 
vaguely similar to their use in HTTP URLs then p-components are going to 
reflect structure between resources, f-components are going to reflect 
structure internal to the named resource, and q-components for ... 
something else (more on that later).   (And if we're not going to use 
these components in those ways, why burden URNs with a syntax that is 
certain to confuse users?)

IMO, this is a serious technical omission, one that is likely to 
undermine the fundamental URN property of persistence, and 
{f,p,q}-components shouldn't be permitted in URNs until this is settled.

3.  Permitting each namespace to define its own rules for use of 
f-components, p-components, and q-components is problematic, at least as 
applied to resolvable URNs that refer to network-accessible and/or 
machine-parseable resources.  One reason this is problematic is that it 
potentially forces client software to know about the details of each 
particular namespace, thus hindering interoperability and potentially 
limiting the useful life of a namespace even when resolution services 
continue to be available. Otherwise, the client won't know what to do 
with each particular kind of component.   Does the f-component refer to 
an identifier internal to a resource that it downloads (as in an HTML 
document) or does the client need to interact with the service in some 
way to access that component?  Does a q-component get sent as part of 
the request-URI in a GET request or what?    Clients that access 
resources named by URNs over the network shouldn't have to have 
different wired-in behavior for different namespaces.  (and no, the fact 
that URLs work this way is NOT a justification for propagating the same 
situation to URNs - URNs were, after all, partially intended to address 
such issues by decoupling resource names from the protocols used to 
access them.)

In general I think the interpretation of a p-component, f-component, or 
q-component need to be more-or-less the same for all URNs.   (To some 
degree this is already constrained by the decision to allow use of 
relative URIs with base URNs).   I think the interpretation of these 
components needs to be exactly the same for all URNs that refer to 
network-accessible / machine-parseable resources.

I think this is less critical for URNs that refer to fixed-media 
resources that aren't intended to be machine-parseable, though I still 
think it will be confusing to users if f-components, p-components, 
and/or q-components are used in ways that aren't more-or-less consistent 
with HTTP conventions.   So one compromise might be to define different 
constraints on use of f-components, p-components, and q-components when 
used to name machine-parseable resources than when used to name 
resources that aren't intended to be machine-parseable.

One potential way to resolve this issue might be to declare that the 
rules for use of these components can be different for machine-parseable 
resources than for resources not intended to be machine-parseable.  Then 
the people who care a lot about using URNs with these components with 
non-parseable resources can define rules (perhaps very simple rules) 
that make sense to them, and the people who care a lot about using URNs 
with parseable resources can define rules that make sense in that 
environment.   The rules for naming resources that can't be parsed 
anyway won't affect clients that need to interact with parseable 
resources.   And if it's simple to define the rules for non-parseable 
resources, namespaces that only assign URNs to non-parseable resources 
could begin to use f-,p-,and/or q-components.   This isn't such a huge 
departure from "namespaces get to choose the rules for interpreting 
these components with respect to URNs from their namespaces" - except 
that in doing so they should be expected to cite one of a small number 
of already-approved standards for such rules.

(Note: "machine-parseable" might not be the right dividing line, but I 
think it's closer than say "resolvable" or "network-accessible". But 
there's definitely room for discussion here.)

Another possible way to resolve this issue might be to define a 
universal URN resolution protocol that would allow a client to determine 
the meaning of f-, p-, and q-components in any particular URN.   But I 
think that solution, while more general, is also more complex and will 
take much longer to settle on.

IMO, this is a serious technical omission.

4.  Constraining URNs to RFC 3986 conventions is problematic.   One 
reason is that when using URNs to refer to network-accessible resources 
there is a need for more functions than those analogous to: references 
to internal structure (presumably f-component), references to external 
structure (presumably p-component), and access to the resource as a 
service (presumably q-component).   In particular there seems to be a 
strong desire to be able to construct URIs containing URNs to query a 
resolution service that is implicitly determined by the URN itself - 
i.e. without having to embed the location of that service in the URI.   
Or to more simply state an example of the problem: given a URN that 
refers to a network-accessible resource, how to distinguish between 
queries sent to the resolver and queries sent to the resource itself?

Another reason that RFC 3986 syntax convention is problematic is that it 
creates a conflict between (a) the use of q-components used to send a 
query or request another service of a network-accessible resource, and 
(b) the use of q-components for other purposes for non-accessible resources.

It might be possible to work around the problem while still sticking to 
RFC 3986 syntax, perhaps by reserving some part of q-component keyword 
space for use by resolvers.   But that would ideally be done without 
interfering with use of keywords used by existing HTTP-based services, 
since we'd like to be able to use URNs to provide stable names for such 
services.

I started out thinking that this was only a poor technical choice, 
though perhaps a defensible one - but on reflection I think it borders 
on being a technical omission.   And I realize we have AD direction to 
use RFC 3986, but that doesn't do anything to address the technical 
problems that this causes.

5. I suspect the way %-encoding is used is problematic for any case 
where human-supplied strings are converted to URNs, in that there are 
multiple valid ways to %-encode a name according to RFC 3986.

One way to fix this is to require each particular namespace that maps 
its own names to URNs, and uses/permits %-encoded NSSs, to specify a 
distinguished %-encoding rule that produces repeatable results and is 
compatible with RFC 3986 decoding.

6. This document fails to acknowledge that even though the binding 
between a URN and the resource named by that URN cannot change over 
time, URNs do not preclude the resource content being revised. This a 
very important, though subtle, distinction.

I trust this is noncontroversial, and easily fixed.   (If it's 
controversial we have a huge problem.)

7.  I'm somewhat sympathetic to the idea that IETF Consensus is not the 
right template for creating new namespaces, but Expert Review is even 
worse.   I think we need widespread public review, and that our very 
experience in this working group is a good indicator of why - there is 
as yet no small collection of "experts" that really understands 
everything about what it takes to make URNs function well, especially 
when applied to network-accessible resources.

I suspect that another part of the problem is that we've conflated IETF 
Consensus with RFC publication because the two are so often coupled.   
We shouldn't have to publish a new RFC every time a NSS format changes.

I trust that discussion on this will continue.

Nits:

1. Introduction, 2nd paragraph, first sentence.   Omit the word 
"typically", or otherwise reword to avoid creating the impression that 
it's ok to use URNs as other than persistent, location-independent 
resource identifiers.

2. Section 3, paragraph following ABNF section: I would substitute "MAY" 
for "can".

3. Section 6, paragraph starting "With regard to global uniqueness":    
Just simply state that a single resource can have more than one URN 
assigned to it.   It doesn't matter whether those URNs exist for 
different purposes.   Also clarify that if there is no such thing as a 
distinguished or preferred URN for a resource that can be inferred from 
looking at the URNs.   At least as far as the URN architecture is 
concerned, all URNs are equally valid.   (If the resource wants to 
specify which of several URNs is the distinguished name for itself, 
that's a different matter.)

---

From: John C Klensin <[email-masked]>
Date: Thu, 26 Mar 2015 12:45:10 -0400

--On Thursday, March 26, 2015 11:58 -0400 Keith Moore
<[email-masked]> wrote:

> General summary:
> 
> This version of the document has several serious technical
> omissions and errors that should, according to RFC 2026
> criteria, prevent its publication as a standards-track
> document until those omissions and errors are remedied.  Some
> of these omissions and errors appear to result from failure to
> consider use of URNs for their primary intended purpose of
> being identifiers for network-accessible resources.   It's
> fine to use URNs to name resources that aren't
> network-accessible, but the URN specifications should not be
> compromised in such a way as to impair their use to name
> network-accessible resources, and this document imposes
> several such compromises.
> 
> There are other choices reflected in the document that I think
> are poor technical choices that should be remedied, though
> they fall short of being "technical omissions" in the RFC 2026
> sense.
> 
> The document also reflects, in at least one case, a failure to
> resolve an identified technical issue, which RFC 2026 requires
> be resolved.
> 
> Details:
>...

(Elided because I will come back to them in subsequent notes)

Keith,

Not speaking for anyone but myself.

I really appreciate your getting down to specifics.  I have to
get to another meeting but will respond in detail later.  In the
interim, I do want to suggest that several of these issues have
been discussed in the WG before (in public and on-list, contrary
to your implication last night) and the conclusion has been that
you are in the rough (and, fwiw, in several cases I am with you
in it).   Let me take the most sweeping example as an example. 

I have believed for some time that 3986 (in at least some
readings, which is another element of the problem)
over-constrains URNs and, at least in retrospect, that 3986
should never have been approved (as anything, much less a full
standard) until those issues --generally and with 2141 in
particular-- were fully sorted out.  That is precisely why, when
it became clear that the WG was going to get so bogged down in
those discussions that even some 2141-conformant URN namespaces
might end up being considered noncomformant or invalid in one
way or another, my first reaction was to separate URNs from 3986
entirely with the plan of getting URNs right as URNs and then,
as necessary, going back and sorting out an update to 3986.
That preference showed up on the (in retrospect) badly-named and
hence inflammatory "URNs are not URIs" approach, document, and
discussion.   

However, the WG --or at least those who were speaking up
loudly-- thoroughly rejected that approach in Toronto and in
subsequent mailing list discussions.  We ended up with "syntax
only, no semantics" as a sort of compromise, partially because
arguments about breaking generic URI parsers seemed to stop with
syntax.  Even then, the "syntax only" position has dragged us
back into relative resolution, and idea that I (and I gather
you) would prefer to simply prohibit for use in the "urn:"
scheme.  However, if the WG and the community believe that
"generic URI parsers" are an important issue and that, if one
has a generic URI parser one must have relative resolution (and,
based on input to the WG, I think we believe both), then...
well, there we go down the slippery slope and it becomes more
important to discuss how to work around those constraints than
to complain about them as constraints.    I haven't read it
carefully enough, but I think your note makes a proposal about
that to which (at least) I plan to pay careful attention.

Or you, or someone else, could come up with a persuasive new
argument to reopen the question of whether we should separate
URNs from 3986 entirely.   I'd be very concerned about the time
that discussion would take and the number of people who would
walk away in disgust if we started having it again, but, in case
it isn't clear from the above, it is otherwise a discussion I
would welcome _if_ there was a basis for it in issues we haven't
already been over several times.

best,
    john

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Mar 2015 12:07:00 -0500

Two things:

1.  RFC 2026 imposes several requirements on standards track documents, among which are that known design choices have been resolved and that there are no known technical omissions with respect to the requirements placed on the protocol.  These criteria are completely independent of the requirement for rough consensus, and nearly everything that I cited in my previous message is an example of a clear failure to meet those criteria.  So it shouldn't matter if a person pointing out technical omissions or failure to resolve important issues is "in the rough".  A WG cannot be the sole judge of the technical quality of its work.

2.  The issues with 3986 were included mostly for completeness, because the problems are still glaring and I think the use of 3986 syntax still borders on a technical omission.  It's only when you ignore the use of URNs to name network accessible resources and insist on using 3986 syntax in a very different manner than it is used with http / html that it appears at all adequate for URNs.

Sent from my iPhone

> On Mar 26, 2015, at 11:45 AM, John C Klensin <[email-masked]> wrote:
>

---

From: Melinda Shore <[email-masked]>
Date: Thu, 26 Mar 2015 12:16:26 -0500

On 3/26/15 12:07 PM, Keith Moore wrote:
> Two things:
>
> 1.  RFC 2026 imposes several requirements on standards track
> documents, among which are that known design choices have been
> resolved and that there are no known technical omissions with respect
> to the requirements placed on the protocol.  These criteria are
> completely independent of the requirement for rough consensus, and
> nearly everything that I cited in my previous message is an example
> of a clear failure to meet those criteria.  So it shouldn't matter if
> a person pointing out technical omissions or failure to resolve
> important issues is "in the rough".  A WG cannot be the sole judge of
> the technical quality of its work.

Of course, and this is why we have IESG review and IETF-wide
last call.  We can, however, conclude that we've gotten as far
as we can go and that it's time to move the document along.

My concern is that we not get stuck is a loop where someone
keeps raising the same issue and others keep providing the same
answer, and there's no progress or willingness to compromise.
It's an unfortunate fact that very few IETF participants have the
grace to be able to say "I don't agree but I can live with that"
and sometimes it seems more common than not for disagreements to
become blocking.

In the absence of new information we're really unable to go back
and revisit issues that have been closed.  If there is new information,
by all means, bring it forward.

One thing that would be helpful to me would be to understand
objections in terms of what the proposed text breaks, and I'd
be grateful if objections can continue to be couched in those
terms.  In the meantime, we really can't revisit closed issues
without a compelling reason, where "compelling" means it hasn't
yet been considered or discussed.

Melinda

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Mar 2015 13:34:53 -0400

> as we can go and that it's time to move the document along.

The WG can do that.  From that perspective, I'm just making sure that 
I'm doing due diligence in informing the WG of the serious technical 
problems with the current document, so that when I raise the issues 
again at Last Call or subsequently to that, people can't say that I 
didn't raise the issues in WG discussion.

> answer, and there's no progress or willingness to compromise.

That's a valid concern.   Please note that these issues for the most 
part have been ignored in the past rather than the WG attempting to 
address them, and also that I did try to suggest compromises in my 
recent review.

Also, when the consistent answer that I'm given whenever I raise 
objections is not technical, but rather, "we have to do what this narrow 
set of people want otherwise they'll take URNs to be standardized 
elsewhere", I think that actually creates a serious process problem.  
Nothing in 2026 says that we can standardize technically flawed 
documents just because of a threat from another standards body.

Also, it's entirely possible that the reason that the WG is stuck in a 
loop is that it has failed to attract the attention of, or even 
alienated, the constituency that created URNs in the first place. I'm 
hoping that it's not too late to get some of those people to contribute 
their input.

> and sometimes it seems more common than not for disagreements to

I say that all the time.   This, however, is not one of those cases.  
Several of us spent several years of our lives trying to get URNs 
defined.  I personally don't want to see them made useless for most of 
the use cases for which they were intended, and I suspect I'm not the 
only one who feels that way.

> objections in terms of what the proposed text breaks, 

I didn't cite section by section because I don't think you can fix the 
problems I cited by simple changes to the text - or at least to the 
extent that you can do so, you first have to get agreement on the broad 
approach.   Again, I did try to suggest ways of resolving the issues I 
raised.

But if you're trying to suggest that it's out of scope to raise valid 
technical issues, I'll appeal that decision to IESG.

Keith

---

From: John C Klensin <[email-masked]>
Date: Fri, 27 Mar 2015 07:37:49 -0400

Keith,

I want to respond to what I think are a few meta-issues -- the
more specific ones are covered in separate notes which I'm
working on but may not finish before the weekend.

--On Thursday, March 26, 2015 13:34 -0400 Keith Moore
<[email-masked]> wrote:

>> keeps raising the same issue and others keep providing the
>> same answer, and there's no progress or willingness to
>> compromise.
> 
> That's a valid concern.   Please note that these issues for
> the most part have been ignored in the past rather than the WG
> attempting to address them, and also that I did try to suggest
> compromises in my recent revie

In some cases, I think you are confusing "didn't know what to do
with your comments" or "disagreed with you", with "ignored".
More specifics with the detailed response to your first note in
this sequence.

> Also, when the consistent answer that I'm given whenever I
> raise objections is not technical, but rather, "we have to do
> what this narrow set of people want otherwise they'll take
> URNs to be standardized elsewhere", I think that actually
> creates a serious process problem.  Nothing in 2026 says that
> we can standardize technically flawed documents just because
> of a threat from another standards body.

And no one has proposed standardizing anything technically
flawed, at least unless the "flaws" are the result of balancing
tradeoffs with other technical issues and concluding that the
balance is against your preferred outcome.   

On the issue of other communities and standards bodies, I've
been louder about the issue, and the issue of trying to avoid a
fork, than, I think, anyone else.  There are several separate
issues involved in that and my motivation for it:

(i) I've have raised the potential for a fork -- either to a
competing standard to to a community just doing whatever they
like and ignoring the IETF-- several times, not in the interest
of getting particular features but as a caution about continuing
to go around in circles without making progress.  It is worth
noting in this regard that the working group is now over four
years behind on what was supposed to be about a six month
schedule (see
https://tools.ietf.org/wg/urnbis/charters?item=./charter.d/charter.2011-02-02_2210,1.txt
which is the earliest charter version I can find with milestones
in it).   

(ii) What actually happened here was that the folks from the
other communities came to us, asking that extension work on URNs
be done in the IETF.  Their doing so was based on the beliefs
that one standard was better than conflicting ones, that we had
critical expertise, and that we could get the work done quickly
and efficiently (something that the "IETF Last Call request on
everything by April and July 2011" schedule reinforced).
Various people from that community have been active on the WG's
mailing list, have been authors or co-authors of documents and
contributed text to others, and so on -- much more active than
any of the original URN designers than anyone but Andy (although
I don't know if you put him in that category or not or how you
define that category).  Their bringing the work to us are
participating in our efforts should, IMO, be cited as an
outstanding example of cooperation, not criticized for trying to
explain to us what functionality they want and need.Ѫ What I
think we owe them in return (both ethically and pragmatically)
is to move forward expeditiously and to either adapt to their
needs or explain why not and propose alternatives.  We have done
neither.   Instead, we have spent most of four years on what
must look from the outside like an extended family feud about
the interpretation of, and relationships among, _our_ standards.
That family feud has also involved several replays of "we don't
need the 'urn' scheme at all (most recently PH-B's comments on
the IETF list this week), an argument those other communities
aren't interested in, partially because they believe they know
what URNs are for and why they are important (and have millions
of them deployed and in use in various namespaces) and believe
that they have several centuries more experience with
persistent, location-independent, identifiers than the IETF
does.  I think they have shown exceptional forbearance by
sticking with us, but we are holding up their work and
constraining choices by these long delays.  The part of this
that involves low participation in the WG by the original URN
designers leads to...

(iii) The IETF has been consistent about wanting change control
over its standards.  There is some pride of ownership in that,
but it has been, at least IMO, largely to avoid accidental or
deliberate forks and the confusion they cause.  At the same
time, most of us are aware, most of the time, that the Protocol
Police are hard to find when needed and/or ineffective: the IETF
has no way to require compliance and, if people want to ignore
us and go their own way, even while using that same identifiers
we have defined and used, there is little or nothing we can do
to stop them.   But, while the IETF has claimed (and, so far,
kept) change control over URNs, that is _IETF_ change control.
Contrary to an inference that could be made from your plenary
remarks, our procedures and culture have never given special
precedence to either the views of original authors or designers
nor to people who have been active elsewhere in the IETF.  For
better or worse, those who participate actively in a WG tend to
have more influence on it than those who either do not
participate at all or who show up intermittently, complain, and
then disappear again for an extended period.  Sometimes that
works out well, sometimes less so -- I'd like to believe that,
if you, myself, and a few obvious others had been more
consistently active when what became 3986 was being developed,
we'd be having different discussions now.

(iv) Partially because of those millions of deployed URN
references (something the IETF should normally consider "running
code"), the one choice the external communities have not given
us is to say "freeze URNs to the technical specification content
of RFC 2141; anyone who wants to do anything else needs to adopt
a new scheme".   Insofar as that position has merit (and you
will recall that it was more or less proposed again as recently
as the Toronto meeting), the IETF rejected it in the first
charter for this WG.

> Also, it's entirely possible that the reason that the WG is
> stuck in a loop is that it has failed to attract the attention
> of, or even alienated, the constituency that created URNs in
> the first place. I'm hoping that it's not too late to get some
> of those people to contribute their input.

I'm not sure which constituency (and designers) you are
referring to.  Karen Sollins (RFC 2276, 1737, etc.) has not be
active in the IETF for many years.  Michael Mealing (author of
the RFC 3401-3405 DDDS specs) has been gone almost as long as
has Ryan Moats (author of 2141).  Larry Masinter (co-author of
1737) has popped up periodically in WG meetings or on the list
but mostly, AFAICT, to advocate for the interpretation and
application of 3986 that you appear to detest.   I'd certainly
be happy to hear more from those people (and from you and
Leslie), but I don't know how long the WG can reasonably be
expected to wait.   Again, the WG is now five years old.

>> It's an unfortunate fact that very few IETF participants have
>> the grace to be able to say "I don't agree but I can live
>> with that" and sometimes it seems more common than not for
>> disagreements to become blocking.
> 
> I say that all the time.   This, however, is not one of those
> cases.  Several of us spent several years of our lives trying
> to get URNs defined.  I personally don't want to see them made
> useless for most of the use cases for which they were
> intended, and I suspect I'm not the only one who feels that
> way.

I appreciate your engaging in an actual discussion this time,
but I just don't see "made useless for most of the use cases for
which they were intended" in any of this.  The WG has, IMO, been
very clear about its intention that no URN or namespace
definition that is valid under RFC 2141 will become invalid with
whatever changes we are making.  My sense is that principle is
one of the things about which there is very broad consensus that
includes just about all participants other than those who
believe that URNs were a bad idea from the beginning and should
be abolished.

I'll address your relative reference and comparison issues in
the response to details, but I don't think they change the above.

Now we (or at least I) face one of those nasty tradeoffs that,
in the light of your comments, seems worth making explicit and
public.   I have limited time to spend on IETF issues and cannot
devote even all of that time to URNs.  Peter and I have
committed to get -11 of 2141bis posted before Wednesday and to
have it reflect issues that seem to have been resolved in WG
mailing list discussions.  I need to cross-check text that Peter
has developed this week before we post.   I'm going to
prioritize that over finishing the point-by-point response to
your detailed comments that I've been working on since shortly
after you posted that note.  The comments are _not_ being
ignored, I'm just trying to make decisions about the most
efficient overall way to proceed and I'm quite confident that,
even if I were to respond to you immediately, there would be no
way to reach enough of a WG conclusion about the issues to have
the result reflected in an I-D next week ... especially so given
your comments about the changes you believe are required
implying major rewrites, not just adjustment to the text of a
few sections.

best,
    john

---

From: Keith Moore <[email-masked]>
Date: Sat, 28 Mar 2015 17:59:48 -0400

> with your comments" or "disagreed with you", with "ignored".
There have definitely been cases when I didn't know what to do with 
others' comments also.  Especially when they were insulting, or veiled 
personal attacks, or they indicated that the author didn't understand 
URNs, or at the very least that the author was coming from such a 
different frame of reference that it was unclear as to how to reconcile 
the differences.

But for most of the issues I've raised, if there's actually been 
substantive discussion that came from people who actually understood the 
issues, I've missed it.    Feel free to point out those occasions so I 
can go back and reread the mailing list discussion on the topic.

But of course when I try to bring up an important issue and the feedback 
I get seems to indicate that the people who responded don't understand 
the issue, it doesn't make the issue any less important.

> to go around in circles without making progress.

Perhaps that's because certain people's definitions of "progress" 
conflict with others.  IMO, balkanazation of URN space is not progress, 
and neither is redefining URNs in such a way that it's infeasible to 
build a general-purpose resolution service (or that, because the 
resolution service has to special-case the handling of namespaces, 
certain namespaces get marginalized).

> explain to us what functionality they want and need.
Let me be clear that I don't fault people from other communities for 
trying to get URNs revised within IETF.   I'm happy, even flattered, 
that they find URNs useful, and I want to find a way to let them 
continue to use URNs without getting in the way of other uses of URNs.   
But I think IETF has done a really lousy job of connecting those people 
with the constituencies that created URNs in the first place, and trying 
to help them find a common vocabulary and vision and to reconcile their 
differences.   You can't accomplish that by just setting up a WG and and 
a mailing list and hoping that everything will be peachy.   That kind of 
approach can work when everyone in the room has more-or-less the same 
investment and the same problem definition, but it doesn't work when 
you're trying to get people with vastly different views, business 
models, etc. to compromise.   Neither consensus nor technical soundness 
is likely to result from that.

(Of course, IETF has problems apart from that, including that the vast 
majority of the IETF participants really don't understand why URNs are 
useful at all, or why they need to be different than URLs.   And of 
course there multiple constituencies who continue to be annoyed by URNs 
because URNs conflicted with their One True Design for either resource 
identifiers.  The IETF consensus process really assumes that there 
aren't too many people trying to be disruptive - a problem we've been 
aware of for a very long time but have never figured out how to address 
because it's often difficult to distinguish someone being disruptive 
from someone who really does have deeper insight.)

> Ѫ What I
> neither.
I've tried, on various occasions, to do that, and my recent list of 
issues was another attempt.

> nor to people who have been active elsewhere in the IETF.

I'm not sure exactly what I said that you're referring to.   But I don't 
claim "special precedence" for myself or anyone else.   I do claim that 
the people who were deeply involved in getting URNs standardized 
actually understand URNs, that they spent many years hashing out the 
delicate technical compromises that went in 2141 and other documents, 
and which 2141bis simply discards without addressing any of the problems 
that caused those compromises to be crafted, and that simply dismissing 
the views of any of those people (including myself) as either 
uninformed, irrelevant, or merely a difference of opinion is 
counterproductive at best, and perhaps even delusional.

>   For

My recollection is that I tried, but 3986 was railroaded through. But it 
has been a disaster.  We've already found it necessary to try to correct 
its redefinition of "URN", the syntax is problematic as I've pointed out 
elsewhere, and the point Larry raised about the way it defines fragments 
is also problematic.   And if 3986 has misled URN implementors into 
thinking that it's okay to use / and ? and # with URNs, that's another 
huge set of problems that it has caused.

> as the Toronto meeting), the IETF rejected it in the first

I want to make very clear that I don't want to break any "running code" 
that conforms to 2141.   I don't even want to break "running code" that 
doesn't conform to 2141.   I don't want to invalidate any URNs that have 
already been minted.   I just want URNs to continue to be useful as 
names for network-accessible resources, and to allow them to be resolved 
in a general way, with as few special-case exceptions for individual 
namespaces as possible.

Whenever we face a situation where there's a significant body of 
"running code" that doesn't conform to our standards, it presents us 
with a dilemma, especially when that body of "running code" is contrary 
to good practice or impedes our ability to evolve the standard in a way 
that's technically sound and meets community needs.   The only way that 
we can be useful is by building consensus, so we don't want to fragment 
the user or implementor community.   And yet we're also here to try to 
promote technical soundness and improve the utility of the Internet, so 
when people insist on standardizing poor or variant practice, that's 
counterproductive to our goals.

If there are namespaces that have been using f-components, p-components, 
and/or q-components, contrary to RFC 2141, I think we should document 
and bless them as special-case exceptions and continue to allow them to 
operate in that way.   We should describe how each of those schemes uses 
those components, which parts are persistent, etc.   But that doesn't 
compel us to standardize that syntax, or to standardize use of those 
components in such a way that code that processes URNs has to know about 
the specifics of each namespace.  An implementor of a general-purpose 
resolution service or the client code for such a service can live with a 
small number of special-case exceptions.   But permitting an arbitrary 
number of exceptions, or allowing every namespace to define its own 
behavior, really isn't a standard at all.   It's precisely the sort of 
failure to resolve known design choices that 2026 forbids.
> expected to wait.   Again, the WG is now five years old.
Granted that it's been a long time, and old WGs often have trouble 
making any progress at all.  The worst possible way for a WG to conclude 
is for everyone to be so tired that nobody is willing to expend the 
energy required to understand anyone else, and/or the participants are 
so fed up with one another that nobody wants to listen to anyone else 
who disagrees with him.   And yet, I honestly believe that 2141bis has 
serious problems, and from the folks I talked to last week in Dallas, 
I'm pretty sure I'm not the only one who shares that opinion.   So I'm 
trying to provide useful input, also trying to get others to do so.   
And I'd appreciate it if people wouldn't just dismiss it out of hand.

> which they were intended" in any of this.
That by itself is reason for concern.   I don't think it's because I've 
failed to explain why this is the case.

>   The WG has, IMO, been

I agree with that also.   I don't think any of the concerns that I 
mention are about 2141-conforming behavior; I think they're all about 
ways that 2141bis departs from 2141 without addressing the problems that 
result from such departures.   In particular, 2141 disallowed fragments, 
paths, and queries for the very good reason that we didn't know how to 
do them in ways that preserved the persistence properties that we wanted 
for URNs.   I don't think that we thought those problems were 
insurmountable, but we did think they'd need to be addressed explicitly, 
that it would take awhile to understand how to do that, and that we 
shouldn't block standardizing  bare URNs until we got those problems 
addressed.

> implying major rewrites, not just adjustment to the text of a
That's fine, and I appreciate the heads up.

Keith

---

From: John C Klensin <[email-masked]>
Date: Sun, 29 Mar 2015 14:18:52 -0400

--On Saturday, March 28, 2015 17:59 -0400 Keith Moore
<[email-masked]> wrote:

>> like and ignoring the IETF-- several times, not in the
>> interest of getting particular features but as a caution
>> about continuing to go around in circles without making
>> progress.
> 
> Perhaps that's because certain people's definitions of
> "progress" conflict with others.  IMO, balkanazation of URN
> space is not progress, and neither is redefining URNs in such
> a way that it's infeasible to build a general-purpose
> resolution service (or that, because the resolution service
> has to special-case the handling of namespaces, certain
> namespaces get marginalized).

I want to isolate this issue because it may be important.

There are, for better or worse, some topics that, however
important or interesting, are properly just not within the WG's
scope.  To start with one we probably agree about, there has
been a recent discussion on the IETF list about getting rid of
the "urn prefix".  I think Phillip is wrong, so, apparently, do
you, Peter, Dave Cridland, and others.  I don't know whether all
of us completely agree on the reasons.  But, as far is the WG is
concerned, the bottom line is that we aren't chartered to figure
out whether URNs as an URI scheme were a good idea.  At least as
important, the ship has clearly sailed on the question: as
someone pointed out on that list, even if the IETF were to
somehow magically conclude tomorrow that URNs were a bad idea,
reclassifying the scheme name as "a prefix" and then getting rid
of it would cause more disruption than it could possibly be
worth.

It may or may not be relevant, but I also note that Phillip has
an interpretation of generic URI syntax that, however much it
appeals to some of us, is diametrically opposed to and
inconsistent with any interpretation of 3986 as normative beyond
a division between scheme name and whatever follows it.

With that as preface, a good deal of the early thinking about
URNs was closely tied notions of a single, general-purpose,
resolution service and, in particular, to the one that became
DDDS.  From reading the DDDS specs, I assume that what at least
some people had in mind was that one namespace would imply one
NID and one instance of a resolver that could be found by
interpreting  "/urn:([^:]+)/\\1/i" and that there would be no
exceptions to that convention.

Whatever the merits of that idea, it was not a requirement in
RFC 3406 (section 1 says:

	'If "resolution" of any so-created URN identifiers is
	desired, a separate process of registration in a global
	NID directory, such as that provided by the DDDS system
	[RFC3401], is necessary.'
)

and note "such as" rather than a requirement and the
anticipation that "resolution" may not be necessary.

Equally important, the idea of a single required resolution
mechanism or database, or a restriction of URNs to online
objects, was essentially dead as soon as RFC 2288 -- a document
on which RFC 3406 and its predecessor RFC 2611 rely --  was
published in 1998.  If anything, it was even more obsolete as an
assumption when we started using URNs that don't resolve at all
such as the XMPP ones that have been referred to as "indicators"
in discussions in this WG.   

If those changes and uses are what you are thinking of when you
warn about balkanizing (sic) URNs, it seems to me it is much too
late to be raising that issue.

Perhaps sadly, that leaves the choice of resolution mechanisms,
or even mechanisms for finding resolution mechanisms (however
general or specific) to be sorted out on a per-namespace basis.
The WG has discussed the best way to say that and deal with some
of its consequences in 2141bis, but the actual decision dates to
2141 and earlier.

It is possible that regrettable decisions were made along the
line.  I suggested in an earlier note that, in retrospect, it
might have been wise to break URNs up into classes/ categories
and use a different scheme for each so that, e.g., we would have
had "ddds:" and "indicator:" schemes.  But that wasn't done and
reconsidering it, as well as designating an exclusive or even
primary class of URN usage, isn't in scope for the WG as far as
I can tell.  

Again, that ship has sailed and, IMO, complaining that this WG
is doing bad things or ignoring key issues because it was not
chartered to go back and revisit those very old issues (and has
not done so)-- issues that have been settled by decade-old
documents and common practices--  seems unreasonable to me.
YMMD, but, if that is where we disagree, I hope we can at least
be clear about it.

best,
    john

---

From: John C Klensin <[email-masked]>
Date: Sun, 29 Mar 2015 21:28:37 -0400

Hi.

As promised, I'm going to try to respond to your detailed review
comments.  In order to lower the odds that any of them fall
through the cracks, I'm breaking them up into separate messages
with different subject lines.  I can't get to the issue tracker
at present, but trust that we can get the new ones listed there
soon as well.

--On Thursday, March 26, 2015 11:58 -0400 Keith Moore
<[email-masked]> wrote:

>...
> 1. Use of p-component in URNs combined with use of relative
> URIs with URN as the base URI.
> 
> If relative URIs are permitted to be used with URNs, this has
> the potential to harm persistence of URNs unless there are
> constraints on how NSSs are assigned.   If an NSS under a
> particular NID has no internal structure, maintaining
> persistence for every URN assigned under that NID is
> relatively simple - just don't ever reassign an NID to a
> different resource.   But if the NSS does have structure that
> reflects relationships between resources, and those resources
> can ever be updated in a way that causes those relationships
> to need to change, there's a good chance that some
> combinations of base URNs + relative URIs that formerly
> resolved to valid URNs, will no longer resolve to valid URNs,
> or will resolve to URNs that don't have the same
> meaning/purpose as before.   Thus there will be a significant
> temptation to change the meanings of existing URNs, i.e. to
> rebind existing URNs to different resources than before.

I had been concerned about a different aspect of applying
relative references to URNs, but, at least to the extent to
which I understand the above, I share your concern as well.
My instinct at the time of the "URNs are not URIs" draft and for
a while thereafter had been to simply prohibit the use of any
sort of relative resolution with URNs.

As you presumably noticed, Graham argued strongly that any such
ban would interfere with some things he needed to do with URIs
(even though he didn't expect to use the "urn" scheme).   There
was no pushback on the list to the position he recommended that
we take; in the absence of such pushback or feedback, I (and I
assume Peter) concluded that it was appropriate to modify the
documents as he suggested.  

> There are lots of ways that a namespace can make relative URIs
> safe to use with its URNs.   For example,  if none of the
> resources named by a namespace are machine-parseable, there's
> no issue.

I'm not sure I understand why the absence of
machine-parseability helps with this, especially when, as
mentioned in my last note and discussed extensively on-list over
the last year or so, a given URN and namespace can be "resolved"
to either a target object (maybe machine-parseable or not) or
metadata about either the object or the entry in the namespace.

>   Or if the resources under a namespace are
> constrained to never change at all, there's no issue.   Or if
> NSSs are of the form x/y such that x is never reused (every
> new resource gets a unique x, and y is just used to
> differentiate between parts or facets of that resource),
> there's no issue.

I think I understand what you are getting at and it makes sense
to me if "parts or facets" can also provide information about
the aspect of the resource being considered or retrieved.

> The problem is trying to write a general rule that permits use
> of p-components in an NSS and permits use of relative URIs
> with those URNs, in such a way that doesn't break things and
> encourage URNs under that NSS to become non-persistent.
 
> Perhaps the easiest way to fix the problem is this:   If a
> namespace assigns NSSs containing '/', it is responsible for
> ensuring that NSSs are assigned in such a way that relative
> URI references appearing in machine-readable resources named
> by its URNs are also persistent - and give examples of how
> this might be accomplished. (Of course, any resource can claim
> any URN as its base URI, and namespaces have no control over
> the behavior of resources that they don't name.  We can't stop
> the authors of such resources from doing truly stupid things
> with relative URIs, but we can discourage use of relative URIs
> with URNs that weren't designed to be used that way.)

Speaking for myself only, if you, Graham, and Larry (who had
what I think is a slightly different suggestion) can agree on
something that that does not disallow p-components if those make
sense to whomever is defining a namespace, it is almost certain
that I'll be able to live with it.  It seems to me that we need
a discussion among the three of you, ideally on-list but at
least in the form of a conclusion, explanation, and suggested
text.

    john

---

From: John C Klensin <[email-masked]>
Date: Sun, 29 Mar 2015 21:46:13 -0400

Note that this issue is, I believe, somewhat redundant with one
discussed in an earlier note.  I've tried to refer to it when
appropriate; please forgive remaining redundancy.

--On Thursday, March 26, 2015 11:58 -0400 Keith Moore
<[email-masked]> wrote:

>... 
> 4.  Constraining URNs to RFC 3986 conventions is problematic.
> One reason is that when using URNs to refer to
> network-accessible resources there is a need for more
> functions than those analogous to: references to internal
> structure (presumably f-component), references to external
> structure (presumably p-component), and access to the resource
> as a service (presumably q-component).

Those presumptions are not clear to me and may not be shared.
See my earlier discussion about interactions with a presumed URN
resolver versus pass-through to URL results and then try to
generalize it to URNs with different kinds of resolution
mechanisms and targets/results (if you have any trouble at all
picturing that for the general case, we are in the same boat and
that is why I didn't expand on that example).

>   In particular there
> seems to be a strong desire to be able to construct URIs
> containing URNs to query a resolution service that is
> implicitly determined by the URN itself - i.e. without having
> to embed the location of that service in the URI.

If I understand what you are saying, isn't that exactly what
DDDS-connected NIDs and namespaces do?

>   Or to more
> simply state an example of the problem: given a URN that
> refers to a network-accessible resource, how to distinguish
> between queries sent to the resolver and queries sent to the
> resource itself?

Modulo some possible issues with what you mean by "sent", see my
earlier note with Subject starting "Fragments and URN
pass-through components".  It doesn't contain a solution, but
does identify what I think is the same problem.   On the other
hand, as it points out, I've actually tried to start that
discussion several times before and the WG has effectively
rejected all solutions other than per-namespace rules.

> Another reason that RFC 3986 syntax convention is problematic
> is that it creates a conflict between (a) the use of
> q-components used to send a query or request another service
> of a network-accessible resource, and (b) the use of
> q-components for other purposes for non-accessible resources.

I don't think that conflict is created by 3986 although I do
believe there is a problem that should be resolved more clearly
than we have done it.    As far as 3986 is concerned, I think
its involvement in the issue ended with the "syntax only; no
semantics" decision.

> It might be possible to work around the problem while still
> sticking to RFC 3986 syntax, perhaps by reserving some part of
> q-component keyword space for use by resolvers.   But that
> would ideally be done without interfering with use of keywords
> used by existing HTTP-based services, since we'd like to be
> able to use URNs to provide stable names for such services.

Yes.  Suggested months and months ago.  Got no traction.  Happy
to hear new suggestions or someone taking up the previous ideas. 

> I started out thinking that this was only a poor technical
> choice, though perhaps a defensible one - but on reflection I
> think it borders on being a technical omission.   And I
> realize we have AD direction to use RFC 3986, but that doesn't
> do anything to address the technical problems that this causes.

As I said, I think there is a problem in this area, but I don't
think 3986 is much of it unless the "syntax only" decision is
either ignored or repealed.

    john

---

From: John C Klensin <[email-masked]>
Date: Sun, 29 Mar 2015 22:17:01 -0400

--On Thursday, March 26, 2015 11:58 -0400 Keith Moore
<[email-masked]> wrote:

>...
> 5. I suspect the way %-encoding is used is problematic for any
> case where human-supplied strings are converted to URNs, in
> that there are multiple valid ways to %-encode a name
> according to RFC 3986.

I'm not sure whether I can say anything useful about this.  It
is no secret that I believe the community made a serious mistake
by not rethinking %-encoding when the Web started to make the
transition between ISO 8859-1 (with the specific characteristics
of one octet per character and no combining sequences) and
Unicode.  It is also no secret that I believe some much stronger
statements about canonical forms should have made it into the
IRI revision work before the IETF abandoned it and that I
believe the community should have given very serious
consideration to deprecating RFC 3987 when that revision effort
collapsed rather than leaving that spec in an ambiguous limbo
while other bodies were effectively encouraged to create more or
less incompatible revisions.     Given that, I sometimes also
suspect that it would be entirely reasonable for any document
that contains significant dependency on RFC 3987 (including
draft-ietf-appsawg-uri-scheme-reg) to go into reference hold and
sit there until there is a 3987bis on the standards track).   

That collection of opinions and beliefs and a few dollars will
get me the proverbial overpriced cup of coffee.

> One way to fix this is to require each particular namespace
> that maps its own names to URNs, and uses/permits %-encoded
> NSSs, to specify a distinguished %-encoding rule that produces
> repeatable results and is compatible with RFC 3986 decoding.

Another is do effectively the same thing by specifying that
%-encodings are not touched (except for upper-casing) and that
URNs that contain %-encodings in relevant components do not
compare equal unless they are identical.   2141bis does that
now.  If you think it would be helpful to remind namespace
designers or reviewers that not having more of a canonical form
is likely to lead to false negatives, explain why and suggest
text.

However, fwiw, I think this is part of a more general URI
problem that URNs should not be expected to solve.  If you
described it as a technical deficiency in 3986, I'd probably
agree but still argue that trying to solve it for URNs only
would be a mistake.

    john

---

From: John C Klensin <[email-masked]>
Date: Sun, 29 Mar 2015 22:22:24 -0400

--On Thursday, March 26, 2015 11:58 -0400 Keith Moore
<[email-masked]> wrote:

> 6. This document fails to acknowledge that even though the
> binding between a URN and the resource named by that URN
> cannot change over time, URNs do not preclude the resource
> content being revised. This a very important, though subtle,
> distinction.
> 

It isn't controversial with me, at least.  Please suggest text
and where to put it.

I suggest, however, that it may be part of exactly what we mean
by "persistent" and that, while the above seems clear, the term
has been hugely controversial with some IETF participants
arguing that the whole idea is a fiction.

    john

---

From: John C Klensin <[email-masked]>
Date: Sun, 29 Mar 2015 22:41:38 -0400

--On Thursday, March 26, 2015 11:58 -0400 Keith Moore
<[email-masked]> wrote:

> 7.  I'm somewhat sympathetic to the idea that IETF Consensus
> is not the right template for creating new namespaces, but
> Expert Review is even worse.   I think we need widespread
> public review, and that our very experience in this working
> group is a good indicator of why - there is as yet no small
> collection of "experts" that really understands everything
> about what it takes to make URNs function well, especially

This issue has been discussed fairly extensively within the WG
and consensus to make the change to Expert Review was reached,
IIR, a couple of years ago.  The counterargument to "Expert
Review is even worse" is based on something we've seen played
out many times with media types as the best-known example.  If
we create a registration procedure that is perceived of as
onerous, the main effect is to encourage squatting (or whatever
one chooses to call using unregistered names) and perhaps having
some Wikipedia page over which the IETF has no control turn into
the registry that people consult rather than having an IANA
registry serve that role.  In that situation, we also often end
up with no useful documentation at all.  So the model in 2141bis
(somewhat revised for more clarity in -11 which we still expect
to have posted Tuesday or earlier) not only defines the Expert
Review process but makes it more consultative rather than
judgmental to encourage documentation and registration.  

-11 also contains some text (also previously discussed in the WG
and reflecting experience elsewhere in the IETF) to handle
namespace specifications that are intended to become standards a
little bit differently from those that are not.    If it isn't
right, I hope you will speak up quickly.

But, as to the fundamental question of whether Expert Review is
appropriate, I hope we don't need to reopen the issue unless you
have some substantive new information or arguments and an
alternate suggestion.

> I suspect that another part of the problem is that we've
> conflated IETF Consensus with RFC publication because the two
> are so often coupled.   We shouldn't have to publish a new RFC
> every time a NSS format changes.

The trend in a number of IETF activities that involve
registration templates (including, fwiw,
draft-ietf-appsawg-uri-scheme-reg as well as 2141bis) is to
require approximately the same procedure for revising a template
that was used to register it in the first place.  I can see that
would be overkill in some cases, but have no idea how to write a
set of rules that would define those cases sufficiently well to
allow a lighter-weight procedure in appropriate cases while
requiring the heavier-weight one at other times and for other
namespaces.  If you have specific suggestions, please post them.
In particular, would you propose to allow each namespace
registration to specify the rules for updating it rather than
having a scheme-wide rule based on the registration procedure
itself.

> I trust that discussion on this will continue.

I'm sure it will if people stay engaged.

best,
    john

p.s. I'm running out of steam for the evening.   Failure to get
a note posted for some of the issues you raised is an indication
that I haven't gotten to it yet, not that it is being ignored.

---

From: John C Klensin <[email-masked]>
Date: Mon, 30 Mar 2015 16:03:15 -0400

--On Thursday, March 26, 2015 11:58 -0400 Keith Moore
<[email-masked]> wrote:

> 3.  Permitting each namespace to define its own rules for use
> of f-components, p-components, and q-components is
> problematic, at least as applied to resolvable URNs that refer
> to network-accessible and/or machine-parseable resources.  One
> reason this is problematic is that it potentially forces
> client software to know about the details of each particular
> namespace, thus hindering interoperability and potentially
> limiting the useful life of a namespace even when resolution
> services continue to be available. Otherwise, the client won't
> know what to do with each particular kind of component.   Does
> the f-component refer to an identifier internal to a resource
> that it downloads (as in an HTML document) or does the client
> need to interact with the service in some way to access that
> component?  Does a q-component get sent as part of the
> request-URI in a GET request or what?    Clients that access
> resources named by URNs over the network shouldn't have to
> have different wired-in behavior for different namespaces.
> (and no, the fact that URLs work this way is NOT a
> justification for propagating the same situation to URNs -
> URNs were, after all, partially intended to address such
> issues by decoupling resource names from the protocols used to
> access them.)

I've addressed much of this in other notes, or hope I have.  To
reprise part of it from a different perspective, unless we
establish a resolution (or non-resolution) model that all
namespaces follow, perhaps with plug-in parameters, some of what
you are concerned about is an issue whether these new (to URNs)
components are added in or not.  If a client needs to understand
a particular namespace in order to know if it is to be resolved
or not and, if so, using what mechanisms and/or databases, then
it seems to me that having scheme-wide rules for *-components,
even if feasible, would not add much.  Similar comments apply to
"at least as applied to resolvable URNs that refer to
network-accessible and/or machine-parseable resources": if one
has to figure out whether a URN is one or those or not before
applying some supposedly-generic set rules, then the value of
those rules goes down significantly, at least assuming the
components are allowed for other varieties of URNs.

Paralleling an argument I made earlier, had folks chosen to have
two (or more) separate schemes, one for that set of resolvable
URNs and targets and others for other situations, we'd be having
a different discussion today.  But it seems to me that we
started allowing the "urn:" scheme to be used for URNs that do
not resolve to network-accessible or computer-processable
(parseable or not) resources even before URNs got off the ground.

> In general I think the interpretation of a p-component,
> f-component, or q-component need to be more-or-less the same
> for all URNs.   (To some degree this is already constrained by
> the decision to allow use of relative URIs with base URNs).
> I think the interpretation of these components needs to be
> exactly the same for all URNs that refer to network-accessible
> / machine-parseable resources.

I think this would be a worthwhile goal if it were feasible.  I
suggest that it is not and, as I suggested in an earlier note,
that the ship sailed many years ago.

> I think this is less critical for URNs that refer to
> fixed-media resources that aren't intended to be
> machine-parseable, though I still think it will be confusing
> to users if f-components, p-components, and/or q-components
> are used in ways that aren't more-or-less consistent with HTTP
> conventions.   So one compromise might be to define different
> constraints on use of f-components, p-components, and
> q-components when used to name machine-parseable resources
> than when used to name resources that aren't intended to be
> machine-parseable.

Note too that one cannot distinguish "URN for fixed-media
resource" from "URN for machine-parseable resource" even at the
namespace level.   As one obvious example, at least some
publishers are assigning ISBNs to so-called eBooks.   The result
is that one could have
   urn:isbn:979-99-999-1234-x
and
   urn:isbn:979-99-999-1345-x
(where "x" is whatever the check digit turns out to be)
And have one of them identify a book printed on paper and the
other one a electronic publication.

Because I don't think the commonality you want can be achieved
for the reasons above, this is more of an aside than a
suggestion, but one way to make users really crazy is to adopt a
rule that seems to work almost all of the time but that
sometimes don't.
 
> One potential way to resolve this issue might be to declare
> that the rules for use of these components can be different
> for machine-parseable resources than for resources not
> intended to be machine-parseable.  Then the people who care a
> lot about using URNs with these components with non-parseable
> resources can define rules (perhaps very simple rules) that
> make sense to them, and the people who care a lot about using
> URNs with parseable resources can define rules that make sense
> in that environment.   The rules for naming resources that
> can't be parsed anyway won't affect clients that need to
> interact with parseable resources.   And if it's simple to
> define the rules for non-parseable resources, namespaces that
> only assign URNs to non-parseable resources could begin to use
> f-,p-,and/or q-components.   This isn't such a huge departure
> from "namespaces get to choose the rules for interpreting
> these components with respect to URNs from their namespaces" -
> except that in doing so they should be expected to cite one of
> a small number of already-approved standards for such rules.

See above.  I just don't think this works without going to two
(or more) separate schemes or an NID naming convention or
reserved part of the NSS that makes the distinction.   And I
think it is too late for any of those options.

> (Note: "machine-parseable" might not be the right dividing
> line, but I think it's closer than say "resolvable" or
> "network-accessible". But there's definitely room for
> discussion here.)

Indeed.  And probably useful regardless of the outcome of the
above.
 
> Another possible way to resolve this issue might be to define
> a universal URN resolution protocol that would allow a client
> to determine the meaning of f-, p-, and q-components in any
> particular URN.   But I think that solution, while more
> general, is also more complex and will take much longer to
> settle on.

One would want it to also be able to determine what "resolve"
means and would resources were to be used to support it.  See
earlier "General-purpose resolution service" note
(http://www.ietf.org/mail-archive/web/urn/current/msg02877.html)

> IMO, this is a serious technical omission.

Certainly it is a difficulty, especially if one's design
religion prohibits handling these things on a per-namespace
basis.  But, as I have discussed earlier (and effectively repeat
above vis-a-vis the "machine-parseable" or not distinction you
are trying to make), the fundamental design decisions that got
us to where were are were made well before this WG came into
being.  The WG is not chartered to revisit them, nor is doing so
really feasible.  So it seems to me that your argument that this
is a "technical omission" is somewhat circular: you assert that
a particular outcome is unacceptable, then claim that the WG has
left out ways to avoid that outcome.   From my point of view,
that outcome, attractive or not, is inevitable given a
collection of decisions made long ago so this is no omission and
would not be an omission even if the remedies you propose were
feasible.

best,
     john

---

From: Andrew Newton <[email-masked]>
Date: Mon, 30 Mar 2015 18:00:29 -0400

On Mon, Mar 30, 2015 at 4:03 PM, John C Klensin <[email-masked]> wrote:

> But it seems to me that we

This was the very first question that popped into my head when I started
thinking about a universal/generic URN resolver. How is it suppose to know
which ones are meant to be resolved to network resources and which are not?

Beyond any theory of URNs or previous design concepts, what is the current
state of URN resolvers? Are there generic ones? Googling around I found
plenty of NBN-specific URN resolvers and some for OGC. But I found nothing
that was generic. There are a couple of Firefox plugins but they list
support for very specific NIDs (one even uses Google as a last resort).

That's not to say that such things do not exist, but if they do are they
actively maintained or actively used?

-andy

---

From: Keith Moore <[email-masked]>
Date: Mon, 30 Mar 2015 22:33:00 -0400

Maybe I didn't express myself clearly, so I'll try an example or two:

1. Say that "frobs" are a kind of stable resource identifier that match 
the regexp

[a-zA-Z0-9?#]+[~][a-zA-Z0-9?#]+

(in other words, a string consisting only of letters, digits, ? and #, 
followed by '~', followed by another string consisting only of letters, 
digits, ?, and #)

Also say that there's a URN "frob" scheme that allows frobs to be 
expressed as URNs.   And there are clients that accept raw "frobs" and 
translate them to "frob" URNs.

I'm not exactly sure why, but I've observed that some %-encoders encode 
'~' as %7E, while others do not.

So client A could encode "abcde#~wxyz" as "urn:frob:abcde%23%7Ewxyz"
and client B could encode "abcde#~wxyz" as "urn:frob:abcde%23~wxyz"
and these two would not be seen as equivalent under comparison rules.

2.  Let's say that "gizmos" are another identifier type that have the 
syntax [0-9]+[.][0-9]+/[0-9a-z]+ .   But the usage of "gizmos" are such 
that it's never appropriate to use them with relative URIs.  So rather 
than leave the '/' unencoded, the '/' should be encoded as %2F when 
converting a gizmo to a URN.

(end of examples)

This is, IMO, a minor concern relative to the other problems I've cited, 
especially since few URN namespaces that are derived from existing 
identifiers will probably have the problem.   But I think the fix is 
easy.  Some URN namespaces that incorporate existing identifier schemes 
under the URN umbrella already need to define how to handle variant 
spellings of those names, e.g ISBN numbers that permit '-' characters.   
I'm just suggesting that a specific algorithm for %-encoding should 
probably be part of that namespace definition for any namespace that 
permits characters that are either forbidden or have special meaning in 
the URN context.

As you point out, there are lots of problems with %-encoding in URIs 
completely apart from URNs.   But I don't see that that justifies adding 
to the problems, and I think we can fairly easily avoid doing so.

Keith

---

From: John C Klensin <[email-masked]>
Date: Tue, 31 Mar 2015 12:30:42 -0400

Keith,

However unintentionally (or undesirably), I think this gets
tangled up with different models of how URIs are parsed and
analyzed.  I've heard each of the following suggested, usually
with the implication that it is the One True Approach:

	(1) Canonicalize the URI string to the extent allowed by
	3986, then do an octet-by-octet comparison, shrugging
	off false negatives.
	
	(2) Canonicalize as above, discard some components of
	the URI, and then proceed as in (1).
	
	(3) Parse out the scheme identifier, then pass
	everything, including comparison questions, off to
	action routines for that scheme.
	
	(4) For URNs, expect the action routines (I'm not going
	to say "resolver" because this really isn't about
	resolution) to pass things off to per-namespace action
	or comparison routines.
	
	(5) Some sequential combination of the above, with each
	operation in sequence returning "these match" for
	positives and passing things along to the next one for
	possible negatives.

Unless we can resolve which of the above is really intended (a
3996bis issue, not a URN one), I don't see any way to
normatively do what I think you are looking for.  However, let
me make a suggestion based on one of your comments.  See if you
think it would help or if I still don't understand what you are
after.

--On Monday, March 30, 2015 22:33 -0400 Keith Moore
<[email-masked]> wrote:

>...
> I'm just suggesting that a specific algorithm
> for %-encoding should probably be part of that namespace
> definition for any namespace that permits characters that are
> either forbidden or have special meaning in the URN context.

Much as some of my instincts want to take us in that direction,
I don't think we have discussed or accepted a goal that would
turn namespace definitions into something a generic URN action
routine (parser, resolver, or anything in between).  Perhaps, if
we did that, it would provide a mechanism for dealing with the
"is this resolvable", "where to I find whatever is needed to
resolve it", and/or "what parts of p-/q-/f-components belong to
URN processing and what parts are intended to be passed through
to the next steps and/or target resources" questions that have
come up in other notes.  

However, it seems to me that Section 7.2 Bullet 4 of
draft-ietf-urnbis-rfc2141bis-urn-10 (and Section 7.3.2 Bullet 4
of -11, which should be posted later today - I don't expect the
numbering to change, but it might) already addresses this
general issue although encoding examples are not called out.   I
also note that this is tied to the "equivalence rules, #8" entry
in the tracker.

Would it make you more comfortable if we added, e.g., "special
encoding considerations, especially for URNs that contain
embedded forms of other types of identifiers" to that list of
examples?   If we did that, would anyone who is concerned about
interactions with generic URI parsers object or have significant
concerns?

Because I think that even if you like that change, we need to
see if it causes concerns among those who are concerned about
generic URI parsers and processing so it won't make it into -11
(except as a placeholder comment in the XML, which is already
there).  If useful discussion continues and it is useful, I
think there should be time to spin a -12 before the scheduled
interim meeting.

best,
    john

---

From: John C Klensin <[email-masked]>
Date: Tue, 31 Mar 2015 12:52:21 -0400

Keith,

Interestingly, whether these are "nits" or not is itself an
interesting question because each one seems to me to raise
additional questions or issues.  We certainly agree that none of
them are (or should be) earthshaking or showstoppers.  I/we have
not incorporated changes for any of them into -11.  I suggest
that you and the rest of the WG have a look at the comments
below to see if we can reach conclusions about what should be in
-12.  The numbering of subsections and summary titles below are
intended to lower the odds of anything getting lost.

(1) "Typically" 

> 1. Introduction, 2nd paragraph, first sentence.   Omit the
> word "typically", or otherwise reword to avoid creating the
> impression that it's ok to use URNs as other than persistent,
> location-independent resource identifiers.

My recollection is that we put "typically" in there to resolve
other concerns and complaints.  They _may_ have been about
whether "persistent, location-independent resource identifier or
abstract designator" actually covered all of the reasonable
cases.   Could others comment on this?  Note that almost the
same phrasing appears in the Abstract so, if it should be
changed in the Introduction, it should presumably be changed
there too.

(2) "MAY" and "can"

> 2. Section 3, paragraph following ABNF section: I would
> substitute "MAY" for "can".

After some list discussion (and, IIR, considerable confusion
about what 3986 allowed), I chose "can" because this statement
is a reminder about 3986-reality rather than a normative
requirement (even a permissive one) imposed by this
specification.    If your reasoning is different, please explain.

(3) "Global uniqueness"

> 3. Section 6, paragraph starting "With regard to global
> uniqueness":    Just simply state that a single resource can
> have more than one URN assigned to it.   It doesn't matter
> whether those URNs exist for different purposes.   Also
> clarify that if there is no such thing as a distinguished or
> preferred URN for a resource that can be inferred from looking
> at the URNs.   At least as far as the URN architecture is
> concerned, all URNs are equally valid.   (If the resource
> wants to specify which of several URNs is the distinguished
> name for itself, that's a different matter.)

This paragraph has been slightly rewritten in -11 to reflect
Ted's comments.  I'd like the WG to have a chance to look at
this and compare it to the approach you recommend before making
further changes.  

best,
    john

---

From: John C Klensin <[email-masked]>
Date: Tue, 31 Mar 2015 14:28:19 -0400

Keith (and others),

I believe this is the last in my long series of initial
responses to the "details" raised in your review.  If anyone
thinks there are separate issues I have missed, please identify
them, ideally with subject lines that distinguish them from
those already addressed.

While I assume (and hope) that discussions will continue along
each of these threads, I note that several of the issues appear
to me to be more intertwined than the review suggested.  I hope
that we can now consolidate them as part of the resolution
process, especially because some of them overlap with issues in
the tracker or that were resolved years ago.

--On Thursday, March 26, 2015 11:58 -0400 Keith Moore
<[email-masked]> wrote:

>...
> 2.  The document fails to specify which parts of a URN
> containing f-components or q-components must be persistent.
 
> For example, since URN x is required to be persistent, is x#y
> also required to be persistent?   Similarly for x?y and x/y.
> Or do the persistence requirements only apply to some of these
> and not others?

See my comments below yours about the general issue of
"persistence requirements".  I'm quoting this entire section of
your note because the original is now several days old and
because it seems useful to focus on this issue.

> IIRC, the reason RFC 2141 doesn't permit '/', '?', or '#' in
> NSSs is that we couldn't figure out how to define or ensure
> persistence in these cases in a way that made sense across a
> wide variety of use cases.   The current rfc2141bis document
> still fails to address these problems.
> 
> q-components are the same problem - how to ensure persistence
> of a structured identifier for a resource that can be revised
> over time, including potentially in ways that change its
> internal structure and/or relationships with other resources.
> But at another level I suspect that these are subtly different
> problems.   If we're going to use p-components, f-components,
> and q-components in ways that are even vaguely similar to
> their use in HTTP URLs then p-components are going to reflect
> structure between resources, f-components are going to reflect
> structure internal to the named resource, and q-components for
> ... something else (more on that later).   (And if we're not
> going to use these components in those ways, why burden URNs
> with a syntax that is certain to confuse users?)
> 
> IMO, this is a serious technical omission, one that is likely
> to undermine the fundamental URN property of persistence, and

I've commented on pieces of this issue earlier, but, if we have
to reach consensus about precisely what "persistence" means and
what its boundaries are, I think we are in big trouble with no
way to get out of it.   You obviously have a clear vision on
what it means, clear enough to believe it is a "fundamental
property" and that there are "persistence requirements".  Others
have made rather strong arguments that there is no such thing --
that everything passes eventually and that any assertions about
persistence have to be made in a very specific content.  Some of
those arguments have included assertions that, in the long run,
a DNS-dependent URL with appropriate precautions can be as
"persistent" as any URN.

I believe it is not an accident that RFC 2141 does not attempt
to define "persistent".   RFC 1737 does and says:

	"Persistence: It is intended that the lifetime of a URN
	be permanent.  That is, the URN will be globally unique
	forever, and may well be used as a reference to a
	resource well beyond the lifetime of the resource it
	identifies or of any naming authority involved in the
	assignment of its name."

but that definition does absolutely nothing to resolve the
questions addressed in this note, nor does it provide any sort
of bright line.   I also note, partially addressing Larry's
concerns, that the "any resource that might conceivably be
available on the network" provision of the following statement
would ban indicator-type URNs where there is no underlying
"resource", much less one that is "available on the network".
URNs have evolved a great deal since 1994 (including in part due
to the collapse of the IIIA model and the disappearance of URCs,
neither of which are, AFAICT, even mentioned in 3986) and all,
or almost all, of the pivotal decisions in that evolution
occurred before this WG came into being and are not within its
scope to try to reverse even if that were desirable.

I also note that Juha and others have told us that the way in
which we use the term "identifier" is often inconsistent with
the use of that term in communities that have far more specific
definitions (and a few centuries more experience).  As with
"persistence", I think trying to draw exact lines will result in
paralysis without any obvious path forward.

The notion of persistence also gets tangled up with an old
distinction and example. Consider three URNs:

 (i) One that references the current weather wherever I
	happen to be right now.
 (ii) One that references the current weather in
	Stockholm.
 (iii) One that references the weather in Stockholm as of
	2015-03-31 17:00:00 UTC.

At least according to my memory from well over a decade ago (two
if one counts RFC 1737), each of these could be a valid URN.
Each is persistent at the URN identifier level (but so is a URL
whose domain name vanishes or is reassigned to a different
owner).  None of them are network location-dependent.  Each is
resolvable and resolvable in more than one way, although they
impose different requirements on the external knowledge that
must be available to the resolution mechanism.  Only the third
resolves to a constant result and, even then, the constancy/
persistency of that result is dependent on some notions of
official statistics versus other sorts of measurements (and
micro-geography) that might be equally or more useful or
"accurate".

None of these distinctions require moving beyond 2141.   If
2141bis has issues with persistence, so does 2141.  What is new
with 2141bis and these various components is some design choices
about whether, e.g., information or specifications belong as
part of an extended NSS or in a {p,q,f}-component.   If you
imagine, as I think we did a decade ago, that the third example
above might be written

    urn:weather:SE:stockholm:2015-03-31-17:00:00

then there is now a potential design decision that might have it
written as

    urn:weather:SE:stockholm?date=2015-03-31-17:00:00

I think it would be really hard to argue that one of these (or
other permutations of the information) is more or less
persistent than the other, but I think it is hard to think about
generic guidance especially because, while that choice might
exist for the hypothetical "weather" NID and namespace, it
doesn't exist for something like an ISBN whose NSS is
effectively specified by other standards.

It seems to me that we are getting close to a religious argument
in which people behave as if features can be included or
excluded or assigned particular properties by pronouncing the
term "persistence" in appropriately pious tones.  That seems to
me to be bad news, especially if we are then going to have
sectarian battles about the meaning of the term.  

I see two ways forward although I hope things are not quite this
much of a dichotomy:

(1) We accept persistence as a general goal and principle of
URNs, clearly excluding things that are
network-location-dependent.  We consider as valid both pure
indicators and things that represent a high level of abstraction
and that can identify any of the members of a family of objects
that are considered equivalent (at least within the definitions
of the namespace).  We also accept that, however clear those
cases may be, there are _lots_ of gray areas.

Design guidance about the choices implied by the example above
would certainly be desirable but I don't think we are ready to
provide it, much less reach consensus on it.  I hope we can keep
that work out of the critical path of the WG because I think we
will fail if we try.

I don't believe that approach will "undermine the fundamental
URN property of persistence" because, while I respect the
general principle of persistence, I think we were already well
past the point that there was a "fundamental property" to be
undermined... and well past it before this WG got started and
proposed {p,q,f}-components.

(2) We start down a path that, especially given the presence in
the community of people who believe that "persistence"
discussions are meaningless, leads to our either being stuck
with 2141 or abandoning and deprecating URNs entirely except,
perhaps, for a very narrow range of cases.  Given that there are
millions of deployed URNs and that many of them would probably
not conform to whatever restricted rules were adopted to ensure
persistent purity, I can't imagine that working out very well if
only because the user community would probably ignore us.

But I may be wrong.  That could be easily demonstrated if there
were a clear and concise definition of "persistence" that would
be generally accepted (including surviving the objections of
those who claim there is no such thing) and that would enable
the distinctions you are seeking.  If you have such a
definition, please post it to the list.

     best,
      john

---

From: Keith Moore <[email-masked]>
Date: Tue, 31 Mar 2015 17:14:00 -0400

> It isn't controversial with me, at least.  Please suggest text

will do.   thanks.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 31 Mar 2015 17:15:04 -0400

> judgmental to encourage documentation and registration.

Perhaps the simplest and clearest response that I can offer is that I 
don't care what the label is ("Expert Review" vs "IETF Consensus" or 
whatever), so much as I care that there be an opportunity for public 
review.   If the process specified for "Expert Review" requires that 
there be public review, that the opportunity for review is advertised in 
IETF and other appropriate fora, and especially if we can somehow 
arrange that the "expert(s)" considering the results of that public 
review really do understand the subtleties of URNs (admitting that this 
is really difficult), I'm okay with that and think it's about the best 
that we can do. It's potentially even better than "IETF Consensus" 
because while that does require public review, nothing about the process 
for choosing Area Directors ensures that IESG will have URN expertise.

So I'll be very interested to read -11's text on this subject.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 31 Mar 2015 17:19:16 -0400

>> namespaces get marginalized).

1.  Ok, first let me clarify what I meant by "balkanization of URN 
space", because I clearly didn't express myself adequately there. What I 
meant was allowing the meaning/interpretation of f-, p-, and 
q-components of URNs to be potentially different for every different 
namespace - especially if it requires a client or resolution service 
that can handle multiple URN namespaces to have wired-in special-case 
handling for each URN namespace.   (And to restate what I stated 
earlier, I think we can tolerate a small number of special-case 
exceptions if that's what we need for compatibility with existing practice.)

2.  Let me also clarify my expectation about a "general-purpose 
resolution service" and the role of DDDS:

Probably by 1995 or so, and certainly by the time that 2141 was 
published, most of us in the thick of the URN WG discussions understood 
that URNs could not be inherently tied to any specific resolution 
service, not even on a per-namespace basis.  This was a direct result of 
the persistence requirement.  Even if there's an authoritative 
resolution service for a namespace (or some subset of a namespace) when 
a particular URN from that namespace (or subset) is assigned, that 
resolution service might not continue to be maintained for as long as 
the URN is used.   So a URN has to continue to be useful even when the 
original resolution service no longer exists.   But just as it's 
reasonable to look up an ISBN in any library's catalog, it's also 
reasonable to expect that there might be one or more "alternative" 
resolution services or catalogs that are indexed by resources' URNs.

DDDS was never intended to be "the" general-purpose resolution service 
in any sense.   It was intended as a near-term solution of locating 
namespace-specific resolution services, as well as perhaps a 
demonstration that a distributed database of resolution services was 
possible, and that it wasn't necessary for a URN client to have 
special-case code to handle every namespace.   Using DNS to advertise 
delegation of URN resolution was and is ugly, and we wouldn't have done 
that if we'd had a better alternative.  But at the time DNS was really 
the only widely-deployed large-scale federated lookup service on the 
Internet.

RFC 3404 itself states:

    The major principle behind the Knoxville framework is
    that the resolution system must be separate from the way names are
    assigned.  This is in marked contrast to most URIs, which identify
    the host to contact and the protocol to use. [...]

    Separating the way names are resolved from the way they are
    constructed provides several benefits./*It allows multiple naming
    approaches and resolution approaches to compete, as it allows
    different protocols and resolvers to be used.*

    /[...]/

/    */For the short term, the Domain Name System (DNS) is the obvious
    candidate for the resolution framework/*, since it is widely deployed
    and understood.

(emphasis mine).   So clearly DDDS wasn't intended to be a sole provider 
of URN resolution service delegation.  It was, by its specification's 
own admission, considered only one of potentially several competing 
approaches, and perhaps only a short-term solution.

Even if DDDS is never widely used for URN resolution purposes, 
general-purpose URN resolution services and general-purpose URN 
resolution code based on other mechanisms would still be useful. If one 
or more organizations wants to each build a URN database and a 
resolution service based on that database, that's a completely 
reasonable and desirable thing to do.   If some such services have the 
capability of incorporating URN citation information from one or more 
namespaces' or countries' or publishers' catalogs, that is also 
reasonable and desirable.

Also, it was never a requirement that every URN be resolvable, and it 
was generally understood since the early days of URN discussions that 
this could not be a requirement.   (I recall that Ron Daniel argued 
quite convincingly, for instance, that there would be resources named 
with URNs whose very existence was secret, so a URN resolver couldn't 
even be expected to return a reliable indication of whether a particular 
URN were valid.)

3.   In the hopes of clarifying further discussion, I offer the 
following definitions and goals:

a) A general-purpose URN resolution service is a service which can 
potentially be queried for information about any URN, independent of 
namespace.   Multiple, independent general-purpose URN resolution 
services can potentially coexist.   Such a service might or might not be 
able to return information about that particular URN - there's no 
requirement that the service has a complete database of all URNs.   Such 
a service might or might not be able to delegate that query to one or 
more other services, based on the URN's namespace ID and/or information 
within that URN's NSS, but there's no requirement that the service know 
how to delegate to a resolution service for every namespace or other 
subset of URN-space.

b) A general-purpose URN resolution client is one that can consult one 
or more general-purpose URN resolution services in order to attempt to 
obtain desired information about a URN when applicable, such as its 
citation/catalog information, network locations, content of the 
resource, etc., and which is not inherently limited to use with URNs of 
specific namespaces.

I claim that generic resolution services and generic clients are 
desirable and anticipated by the URN work.   For example RFC 1727 
section 5 states "It may also be the case that there are generic ones" 
(i.e. resolvers or location services) "... providing service for many 
resources of differing naming authorities".   Indeed, a principal 
motivation for the URN work was to be able to resolve resource names in 
a generic fashion, even across potential changes in access protocols or 
lookup services.   (and this is precisely why we wouldn't have tried to 
divide URNs up into schemes like "ddds:" or "indicator:" - we were 
deliberately not taking that approach because doing so would harm URN 
persistence.)

RFC 2141 impedes neither of the above services [*].  But 2141bis -10 
impedes both of them, if the interpretation of f-components, 
p-components, or q-components by either the service or the client needs 
to differ from one namespace to another.   Note that it should *not* be 
expected that the resolution service is specific to a particular 
namespace, as there will be a need for resolution services that handle 
URNs from a variety of namespaces.  (if not immediately, then 
eventually, as the authoritative services for some namespaces will 
inevitably disappear, and there will be a desire to provide lookup 
services for URNs in namespaces that never had authoritative services.)

If, on the other hand, all namespaces delegate evaluation of these 
components to a resolution service, that solves the problem to some 
degree, but it then presents other problems: e.g. evaluation of relative 
URIs with respect to these URNs, the inability to address fragments in 
HTML documents named by URNs, the inability to send "queries" using the 
HTTP GET method to resources that accept them (at least using an 
ordinary HTML tag), some namespaces won't have resolution services (thus 
there's no way for a client to know how to interpret their components) 
and so on.

[*] RFC 2141 section 7 does say that "Functional equivalence is 
determined by practice within a given namespace and managed by resolvers 
for that namespace."   It may be the case that only an authoritative 
resolution service for a particular namespace can reliably determine 
functional equivalence for two URNs from that namespace, but generic URN 
resolution services are useful in other ways even if they cannot 
reliably determine functional equivalence.   (Note also that two URNs 
can be functionally equivalent even if they are from different 
namespaces; there's no requirement that all URNs for a particular 
resource be from the same namespace.)

---

From: Keith Moore <[email-masked]>
Date: Tue, 31 Mar 2015 17:42:45 -0400

On 03/30/2015 04:03 PM, John C Klensin wrote:
> I've addressed much of this in other notes, or hope I have.

And I think I've responded in my other responses to your other notes, so 
I'll try to not be (too) redundant here.

>    To
> even if feasible, would not add much.

Agreed.   But as I've argued elsewhere, having generic resolution 
services is both necessary for URNs to be viable as identifiers in the 
long term (since some namespaces or their resolution services are likely 
to be abandoned but we'd still like for those identifiers to be useful), 
and explicitly anticipated by the earlier URN work.

If we're going to give up any possibility of general-purpose URN 
handling, at either the client or the resolution service level, that 
should be a very explicit decision and not just a consequence of other 
WG decisions made by people who might not understand the implications of 
those decisions.   (And I really want to make sure that the original URN 
proponents are aware that the WG is contemplating such a decision, 
before IETF discards many years worth of effort invested in making URNs 
generally resolvable.)

> Similar comments apply to

I think this is less of an issue.   It will always be possible to 
compose URIs in ways that don't actually make sense - e.g. for an 
HTTP-based service that takes SQL commands as query parameters, some 
user can always supply an invalid SQL command, one which refers to a 
nonexistent table or column, etc.

The problem with URN overloading of 3986 syntax is not so much that one 
has to know what the parameters need to be - that's necessary for URLs 
also - it's that either (a) there's no way at all to supply parameters 
that will be transmitted to or evaluated in the context of the resource, 
or (b) in order to supply such parameters, one needs to know not only 
what parameters work for that particular resource, but also how to 
encode them in a URN according to the rules for its namespace.

In the case (b), if the only way that service is ever accessed is via a 
single URN (and never by a URL or a URN from a different namespace), 
maybe that's not such a big deal.   But it still introduces the problem 
that the developer/maintainer of that resource has to deal with 
parameters using a different syntax and/or encoding than that supplied 
with the URN, which if nothing else makes it harder to document the 
usage of that resource.   With namespace-specific rules for mapping from 
URN syntax to the input fed to the resource, the namespace is adding an 
extra layer of obfuscation.   All of my experience tells me that such 
layers decrease usability and reliability.

> a different discussion today.
Right, but the whole idea of URNs was not to tie them to any specific 
protocols, even resolution protocols.   We didn't want to repeat the 
situation that existed with http and other URLs where the identifier and 
access protocols were inherently tied together, because the successful 
access protocols would clearly need to evolve over time, and some of 
them would fall into disuse (as indeed has happened).   And for 
transition purposes if nothing else, it should be possible for multiple 
URN resolution protocols to coexist, even if there's overlap between the 
sets of URNs that they will provide answers for, without breaking any 
URN functionality.

> that the ship sailed many years ago.

I don't agree that the ship has sailed, and I think this conclusion 
derives from some misunderstandings which I hope are cleared up here or 
in one of my other recent replies.    Though if there is significant 
deployment of URNs which have used p-, q-, and/or f-components in 
violation of RFC 2141, we need to make some specific exceptions to 
accommodate them.

(in other words, what I'm proposing is that rather than say "any 
namespace can do whatever it wishes with these components" we should 
instead say "these components are to be used in this way, but for the 
sake of backward compatibility the following pre-existing namespaces 
have alternate specifications for their use of these components")

> other one a electronic publication.

I have always assumed that that would eventually happen, despite strong 
statements to the contrary from some people who insisted (e.g.) that 
ISBNs would always refer only to print media.

Of course, if it's not appropriate for a particular namespace to impose 
such a rule on the resources that it names, it shouldn't do so, and that 
way of avoiding the problem with {f,p,q}-components doesn't work for 
that namespace.  It might not work for any namespace.    It was just a 
suggestion for one possible way to resolve the issue.   But I'd actually 
rather resolve the issue in a way that doesn't further constrain what 
any namespace can do with its names beyond the minimum requirements 
already imposed for URNs.

> rule that seems to work almost all of the time but that

indeed.

> think it is too late for any of those options.

Nor, for reasons cited above and elsewhere, would I suggest that 
approach.   But the issue still needs to be resolved, even if it can't 
be resolved using my suggestion to have different rules for parsable and 
non-parsable resources.

> being.
I don't think so.   These problems, as far as I can tell, don't exist 
with 2141 URNs.   They exist only as a consequence of the decisions made 
by this WG and/or nonstandard departures from 2141 syntax.

>    The WG is not chartered to revisit them, nor is doing so

And as I've written elsewhere, URNs were always intended to work with 
general-purpose resolution.   I would argue that this WG is not 
chartered to revisit that decision.

But I also think that there's some widespread misunderstanding of 
"decisions made long ago" within this WG which is perhaps adding some 
confusion to the discussion.   The compromises reflected in RFC 2141 are 
a lot more subtle and delicate than the text in that document reveals, 
because the document only specifies the syntax and not so much the 
reasons for those choices.

(The same is true for many important standards.   The specifications 
never capture the subtleties necessary to understand the design 
choices.   But IETF would never let a WG modify the TCP protocol without 
making sure that people who really understood TCP were in the loop, nor 
would IETF let a WG modify the TLS protocol without making sure that TLS 
experts were in the loop.... and yet IETF seems to have made little or 
no effort to make sure that people who understand the reasons for URN 
design decisions were in the loop for this WG.)

Keith

p.s.  Maybe an analogy would help.   Having interpretation of 
{f,p,q}-components of URNs be on a per-namespace basis reminds me a lot 
of the situation with email addresses prior to the mid-1990s, when it 
was common to see an admixture of rfc822, bitnet, uucp, x.400, and 
decnet addresses in email messages, encoded in various ways, sometimes 
within the headers of the same email message, but certainly different 
ways of expressing addresses in correspondence.   Though in principle 
every address that appeared in a message header should have been valid 
for the particular context in which it appeared, in practice even this 
wasn't the case. Instead of handling all email addresses in a uniform 
fashion, mail administrators who were trying to get their users' mail 
delivered had to employ all kinds of special-case rules and heuristics, 
e.g. deciding which parts of an address were more significant than 
others.   (e.g. should [email-masked] be forwarded to a via 
UUCP, thence to be and c, thence to [email-masked], or should the 
message be forwarded to example.com over SMTP and allow it to forward it 
to a via UUCP and so on?)

It's not a perfect analogy, but my take is that every URN processor 
should be able to apply the same set of rules to URNs not only for 
separating them into components, but also for looking them up in 
databases, for finding resolution services, for processing relative URIs 
if those are allowed, for finding fragments, etc., and it should be 
possible to define one or more namespace-independent protocols for URN 
resolution.   The last thing we need is the kind of second-guessing 
about URNs that we used to need to do for email addresses.

---

From: Keith Moore <[email-masked]>
Date: Tue, 31 Mar 2015 17:42:58 -0400

John,

Let's take a step back here.   I'm not asking for this group or this 
document to revisit RFC 1737s definition of persistence. Nor do I think 
it is in scope for this working group (as defined by its charter) to do 
so, as that's tantamount to redefining URNs.

I'm simply asking for one or two sentences to be added to rfc2141bis to 
clarify which of f-component, p-component, and q-component are relevant 
when considering persistence.   RFC 2141 didn't need to address this 
issue because with RFC 2141 URNs, the entire URN was assigned, so the 
persistence property applied to the entire URN.

The text I have in mind is something like:

Only the [list of components] of the URN are required to meet the 
"persistence" property specified in RFC 1737.   Individual namespaces 
may, however, impose more stringent requirements on use of [other 
components].

I just don't know what should be that list of components.  I can, 
however, somewhat arbitrarily, suggest concrete text:

Only the assigned-name and p-component of the URN are required to meet 
the "persistence" property specified in RFC 1737.   Individual 
namespaces may, however, impose additional restrictions on the use of 
q-component and/or f-component.

(and if a normative reference to RFC 1737 is a problem, just leave out 
"specified in RFC 1737".   RFC 1737 would probably have been a BCP, had 
BCPs existed at the time.)

Keith

p.s. now if we really want to talk about why RFC 1737's definition of 
persistence is the correct one for URNs, and why nobody (from this 
planet, at least) has centuries of experience naming network-accessible 
resources that have the potential to change continuously, and why the 
fact that some people claim that other kinds of URIs can have similar 
persistence properties is irrelevant because we're only talking about 
URNs here, etc., etc., we can do that.   But I think it's 
counterproductive to make any of those discussions block this document.

---

From: Keith Moore <[email-masked]>
Date: Tue, 31 Mar 2015 17:51:32 -0400

On 03/30/2015 06:00 PM, Andrew Newton wrote:
>
> On Mon, Mar 30, 2015 at 4:03 PM, John C Klensin <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
> This was the very first question that popped into my head when I 
> started thinking about a universal/generic URN resolver. How is it 
> suppose to know which ones are meant to be resolved to network 
> resources and which are not?

The client doesn't know this, necessarily.   But the URN might have 
appeared in a context (say an HTML tag) for which the URN only make 
sense if the resource can be accessed, and the first step in doing that 
is to query a resolver.   (Of course, if the named resource can't be 
accessed for whatever reason, it's more-or-less equivalent to a URL that 
can't be accessed for whatever reason.)

If the resolver has information about the URN, it might be able to 
respond with information about how to access the resource over the 
network, or it might actually be able to provide such access by 
itself.   But it might only know citation information for the URN, even 
if the resource is network-accessible, and that citation information 
might or not include information of network accessibility.

Also, part of that citation information might be some sort of resource 
type indication, so presumably a book or sound recording or whatever 
that wasn't accessible over the network would have a different resource 
type than a web page or web server or whatever that can be accessed over 
the network.   But there the line is a bit blurry, since it's completely 
reasonable to think in terms of accessing, say, a physical DVD over the 
network, say from a library's collection, even though it's a physical 
data storage medium.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 31 Mar 2015 18:01:01 -0400

> documents as he suggested.

I sorry that I missed this argument.  I'll try to find his messages on 
the topic.   I do acknowledge that we probably want to be able to use 
URNs to refer to documents that contain relative URIs, and that those 
documents may have other URIs pointing to those documents for which the 
relative URIs make sense.   I don't think that RFC 2141-style URNs have 
this issue, at least to this degree, not because they can't point to 
documents that contain relative URIs, but because they don't have paths, 
so any relative URI within the document resolves to something meaningless.

But I do think that having paths in URNs, and the ability to combine 
relative URIs with base URNs, creates a number of technical issues that 
need to be sorted out before we publish a revision to rfc2141 that 
permits them.   I'm not saying that they can't be addressed, only that 
we need to do so.

> metadata about either the object or the entry in the namespace.

I'm assuming, perhaps incorrectly, that the vast majority of use of 
relative URIs are in documents/resources that are named by their base 
URIs.   So if the resource named by a base URN isn't machine-readable or 
can't contain relative URIs in the first place, having a p-component in 
that base URN doesn't seem to me to cause a significant problem.

(It might be that I'm wrong about this, and that there are other reasons 
that problems that can result from this solution.  I was just 
brainstorming to try to find a solution that didn't require p-components 
to be forbidden in URNs, and this seemed like a potential way out.)

> the aspect of the resource being considered or retrieved.

In the example above I think I should have also have clarified that the 
resource named by x/y never gets updated in such a way that some 
combinations of x and a relative URI change meanings.

It's very similar to the persistent fragment question: if urn:foo:bar is 
persistent, is urn:foo:bar#zot also persistent? Implicit in the design 
of URNs is the notion that people and software can assume that 
identifiers beginning with urn: are persistent, which is why there is 
language in RFC 2141 about whether something is "part of" a URN.    
(There were extensive discussions about whether a combination of a base 
URN and a modifier should be be considered persistent - if the modifier 
were "part of" the URN, then the entire URN would be required to be 
persistent, but if the modifier were not "part of" the URN, only the 
base URN would be required to be persistent.)

However when considering a URN with p-components, the question is 
somewhat more subtle, because relative URIs are defined in terms of 
constructing a new URI from a base URI and a relative URI.   So if you 
have a base URN urn:foo:x/y and relative URIs in the context of such a 
base URN are interpreted with the same algorithm as that used for base 
URLs, every reference to one of those relative URIs effectively creates 
a new URN that doesn't actually have to appear anywhere as a complete 
identifier.   e.g. if the base URN is urn:foo:x/y and there are links in 
that document to "a", "b", and "c", this effectively creates URNs 
urn:foo:x/a, urn:foo:x/b, and urn:foo:x/c.

> least in the form of a conclusion, explanation, and suggested

Good.  I hope we can work this out.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 31 Mar 2015 18:18:59 -0400

Regarding 3986 in general, I recognize that there's a conundrum here.   
As I understand things, the AD explicitly imposed 3986 syntax on this 
group.   So I undertand that the group has a hard time recommending a 
different approach.   Also, much of the group has long believed that 
3986 is adequate, and maybe there's already some use of that syntax in 
deployed URNs, so there's a lot of inertia behind that syntax.

So, regardless of what I personally believe, I'm not really trying to 
persuade you or the group that 3986 should be abandoned.    I do, 
however, think that, having decided to use that syntax (or having had 
that syntax imposed on it), the group has to solve the problems that 
result from that decision before it can expect its document to be 
approved as a proposed standard.

I continue to believe that the decision to use that syntax will result 
in a tremendous amount of harm and confusion, because it's using a 
syntax that people are accustomed to using with http and other URLs, but 
in very different ways - and perhaps even in ways that vary arbitrarily 
as far as users are concerned.   But at least for the time being, that's 
irrelevant unless/until either IESG rejects the document (perhaps 
realizing that the result just creates too many problems) or an AD or 
IESG action gets overturned on appeal.

So basically I think the immediate task before us is to clean up the 
messes created by RFC 3986 and the decision to allow {f,p,q}-components 
in URNs.

> appropriate; please forgive remaining redundancy.

Yes, there is overlap, because these various AD and WG decisions 
interact and have cumulative effects.  It's difficult to talk about 
these effects one at a time without introducing some redundancy, though 
splitting up the topics is probably still a good way to have the discussion.

> that is why I didn't expand on that example).

A small number of examples illustrate why it is problematic.

1. A URN from a namespace which uses f-components is associated with, 
and/or resolves to, either an HTML document or a service that produces 
an HTML document when it is accessed.   That HTML document contains 
relative URIs that refer to fragments internal to, and identified 
within, that HTML document.

  * With the f-component used for some other purpose, there is no way to
    externally refer to those fragments.
  * Also, any relative URI references within that document will cause
    the document reader to construct identifiers that look like URNs
    contining the base URN + the fragment identifier, and those
    identifiers will likely leak in various ways (in the user interface,
    browser history, etc.) that at the very least cause user confusion
    about how URNs are used.

(note that there's no assumption here that there's any intermediate URL 
that also refers to the document, though there might be.)

2. A URN from a namespace which uses p-components is associated with, 
and/or resolves to, either an HTML document or a service that produces 
an HTML document when it is accessed.   That HTML document contains one 
or more relative URIs that contain "paths" that will, in the context of 
that HTML document, be evaluated relative to the URN used to access or 
obtain that document.   Again, those relative URI references will leak 
in various ways that will likely cause confusion.

3. A URN from a namespace which uses q-components is associated with, 
and/or resolves to, either an HTML document or a service that produces 
an HTML document when it is accessed.   That HTML document contains 
either one or more relative URIs that contain "queries" and/or one or 
more HTML forms which use the GET method.   In order to submit the form, 
the client reading the HTML document will construct a URI from the base 
URN and the form parameters using RFC 3986 "query" syntax.

4. A URN is used to identify a service which is accessed using 
HTTP[S].   That service accepts query parameters using RFC3986 
syntax.    With the current 2141bis language, there is no way that a 
client, user, or document author can reliably construct a URI which is a 
composition of the URN used to identify the service and the query 
parameters.   If that client, user, or document author happens to know 
how that particular namespace's resolver works, that person might be 
able to construct such a URI - but those assumptions fail with generic 
resolvers.

5. A URI or other string is constructed by adding one or more f-, p-, or 
q- components to a base URN, following RFC 3986 syntax. It's not clear 
from 2141bis -10 whether the resulting string is persistent or not, and 
this might vary on a per-namespace basis.

6. Users who are familiar with HTTP and other URLs will see '#', '/', 
and '?' in URNs and think they know what these mean, think they know how 
to manipulate them.   (There's been a tremendous amount of 
misunderstanding about URNs, even within IETF, which has hampered the 
actual utility of URNs even by those who understand them.)

(This is not an exhaustive list.   Also, I used HTML documents in these 
examples because they're very familiar, but I doubt that HTML documents 
are the only documents that use 3986-style relative references.)

> DDDS-connected NIDs and namespaces do?

I must have left out some text here.  I think I was trying to say that 
people in this WG seem to want to be able to compose base URNs and 
modifiers beginning with '/', '#', and/or '?' but without any kind of 
consistent meaning for those modifiers across resolution servers, and 
without even specifying which resolution server to use.

(Perhaps there was an implicit assumption that there would never be a 
need for general-purpose resolution services independent of any 
particular namespace, and I emphatically don't believe that.   I think 
that URNs need to be usable in queries to general-purpose resolution 
services even after a particular namespace has ceased to operate its own 
resolvers, so that it will be possible for libraries or other concerns 
to maintain databases about those URNs and the resources that they refer 
to, and for clients to potentially query those databases using those 
URNs without any particular database having to provide complete coverage 
for any particular URN namespace.)

> hand, as it points out,/*I've actually tried to start that
> rejected all solutions other than per-namespace rules.*/

This makes me wonder whether some WG participants don't understand the 
IETF process.   A WG generally gets a lot of latitude in deciding how a 
problem is to be addressed, but identified problems generally do have to 
be addressed.   The insistence on 3986 syntax and use of f-, p-, and q- 
components introduces several problems that weren't present with 2141 
syntax.   The original URN WG elected to avoid those problems in the 
near term by reserving '#', '?' and '/', for future use.   If URNBIS 
wants to define what those things mean, it's free to do so, but it has 
to address the resulting problems - and simply saying "each namespace 
can do as it likes" is not a solution because it effectively precludes 
namespace-independent use of URNs.

It's almost as if many WG participants assume that URNs will never be 
used in any namespace-independent context - and if that's the case, why 
bother with the urn: prefix at all?   Almost the only purpose of the 
urn: prefix is to enable some generalized handling of URNs across 
different namespaces.   (Yes, it's also an indication that the following 
identifier is persistent, but that's also being diluted by use of 3986 
syntax, as in example #5 above.)

> its involvement in the issue ended with the "syntax only; no

While I think something along the lines of "syntax only" is about as 
good a compromise as the WG can reach given the imposed mandate to use 
RFC 3986, there are too many ambiguities in the current spec. Whether we 
like it or not, those characters '#', '/', and '?' have meaning, 
partially from the decision to permit relative URIs to be used with 
URNs, partially from public mindshare.    People want to use them with 
URNs because they do have meaning, even if they want to use them in ways 
that are at least somewhat inconsistent with their current meanings as 
applied to URLs.   So while we might claim that this is a "syntax only" 
decision, we all know better.  That's just a bit of political whitewashing.

> either ignored or repealed.

As I see it, use of 3986 syntax with URNs carries with it several kinds 
of baggage, as illustrated by the above examples.

I can hardly expect the WG to countermand direct AD direction to use 
3986 syntax.   But I'm not sure that the supporters of this approach 
really understand how much of a rat's nest this creates.   And while I 
think there are probably workarounds for around all of these problems, 
the net result is probably going to be really ugly and confusing.

Still, the task before us is to try to make it as non-ugly and 
non-confusing as we can make it.
(sort of like putting makeup on a pig)

Keith