Subject: Re: URI canonicalization

From: Robert Sayre <[email-masked]>
Date: Mon, 31 Jan 2005 23:53:49 -0500

Over here in the thread with technical commentary,

Graham wrote:

> On 1 Feb 2005, at 3:10 am, Martin Duerst wrote:
>
>> 1) switch sections 3.5.1 and 3.5.2 to make clear that
>>    comparison of IDs is the most important operation.
>
>
> Yes.

I eliminated 3.5.1 as a distinct entity in this proposed edit:
http://www.imc.org/atom-syntax/mail-archive/msg12807.html

That got the +1 from Paul and Tim (who seemed to be on opposing sides... 
I don't care if we are using IRIs). None of the changes proposed below 
obsolete that edit. I will put it on the wiki as a "Pace" if that's 
what's needed.

>
>>    This makes it clear that we are talking about "here is how
>>    you do it", rather than "here's one way to do it".
>
>
> We might be treading on toes making that assertion.

I can't think of anyone in this thread that would object. Esp if we are 
using IRIs.

>
>> 4) Add a sentence saying something like "Feeds or Entries
>>    are identical if their IDs compare identical.".
>>    Seems obvious, but isn't stated anywhere.
>
>
> No. Feeds/entries with the same id are different versions or instances 
> of a common ancestor. They are not "the same".

Martin is correct, IMHO. A URI (or GUID, for that matter) identifies 
*one* resource. We don't get to change that. OTOH, I don't think it will 
happen unless there is a commonly seen need for it to happen, so it 
seems like a non-issue.

>
>> 5) Add a note saying something like "Comparison functions
>>    provided by many URI classes/implementations make additional
>>    assumptions about equality that are not true for Identity
>>    Constructs. Atom processors therefore should use simple
>>    string functions for comparing Identity Constructs."
>>    I think such a note could be a good balance to the normalization
>>    advice.
>
>
> Yes, though I don't like that wording.

Agree.

Robert Sayre

---

From: Robert Sayre <[email-masked]>
Date: Tue, 01 Feb 2005 00:02:20 -0500

Robert Sayre wrote:

>> No. Feeds/entries with the same id are different versions or 
>> instances of a common ancestor. They are not "the same".
>
>
>
> *one* resource. We don't get to change that. OTOH, I don't think it 
> will happen unless there is a commonly seen need for it to happen, so 
> it seems like a non-issue.

After reading Tim's response, it's obvious I totally misread this 
exchange. Graham is right.

Robert Sayre

---

From: Martin Duerst <[email-masked]>
Date: Tue, 01 Feb 2005 13:44:38 +0900

>No. Feeds/entries with the same id are different versions or instances of 
a common ancestor. They are not "the same".

Agreed. I'd not mind better wording (e.g. as proposed by Tim),
but I think it's good to actually have something say what the
fact that two IDs are the same means.

Regards,   Martin.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 31 Jan 2005 21:27:05 -0800

>    I think such a note could be a good balance to the normalization

That would be a falsehood.  Identifiers are not subject to
"simplification" -- they are either equivalent or not.  We can
add all of the implementation requirements we like to prevent
software from detecting false negatives, but that doesn't change
the fact that equivalent identifiers always identify the same
resource.  It is the author's responsibility to use URIs
(or IRIs) that are actually different, not the responsibility
of the protocol or implementation.

I am disappointed that a MUST requirement was added to IRI in the
last draft without working group review.  This part

   Applications using IRIs as identity tokens with no relationship to a
   protocol MUST use the Simple String Comparison (see section 5.3.1).
   All other applications MUST select one of the comparison practices
   from the Comparison Ladder (see section 5.3 or, after IRI-to-URI
   conversion, select one of the comparison practices from the URI
   comparison ladder in [RFC3986], section 6.2)

is completely missing the point of the ladder.  The identifiers may
or may not be equivalent and there is absolutely no reason for
protocols to require inaccurate comparisons.  The reason for
simplification of comparison is ONLY that false negatives are
an acceptable fact of life and their elimination is an
implementation-specific decision that has no impact on
interoperable use of identifiers.  That is why there is no such
requirement for URIs.

....Roy

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 31 Jan 2005 22:16:10 -0800

On Jan 31, 2005, at 8:40 PM, Tim Bray wrote:
>
> Graham's right, the word "identical" is wrong, because in fact you 
> will commonly encounter two instances of the same entry which aren't 
> identical (e.g. the one in your cache and the one you just fetched).  
> I suggest "Software MUST treat any two entries which have the same ID 
> as instances of the same entry, regardless of how they were obtained 
> or any differences in their content."  Or if you want it less 
> prescriptive "By definition, any two entries which have the same ID 
> are instances of the same entry, regardless of how they were obtained 
> or any differences in their content."

Over-specification is just too fun.  So that would mean I am
required by Atom format to treat two different entries with the id
    "http://tbray.org/uid/1000"
as the same entry, even when I received the first one
from tbray.org and the second from mymonkeysbutt.net?

Oh yeah, that'll be fun. ;-)

It is not necessary for the format to require world peace, or
anything generally equivalent to it.  Give implementations the
freedom to do whatever they like with the format -- just tell
them what the syntax means and the implementations will sort
themselves out.

....Roy

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 01 Feb 2005 07:33:04 +0100

>them what the syntax means and the implementations will sort

YMMD.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Bill de hÓra <[email-masked]>
Date: Tue, 01 Feb 2005 10:07:52 +0000

> of a common ancestor. They are not "the same".

+1.

It is right that you shouldn't conflate two entries based on solely 
their ID (because they might represent different states).

The main value of IDs seems to be a better user experience rather than 
optimized storage - but I imagine any client tool that wants to 
rationalize the entries presented to the reader is going to use dates 
heuristics as well as ids to organize what Tim is calling the entry 
instances.

Otherwise, this thread sounds like resources and representations all 
over with the caveat that entry representations are being sourced from 
multiple origin servers.

cheers
Bill

---

From: Bill de hÓra <[email-masked]>
Date: Tue, 01 Feb 2005 10:12:29 +0000

> Oh yeah, that'll be fun. ;-)

:)

And yet, it's a feature of syndication to get Tim's feed entries from 
different origins. I already get each one from at least two sources - 
Planet XML, and Ongoing.

cheers
Bill

---

From: Danny Ayers <[email-masked]>
Date: Tue, 1 Feb 2005 11:45:37 +0100

> over with the caveat that entry representations are being sourced from

It was suggested ages ago that the use of a different URI in <id> than
the one which would usually provide representations of the entry
resource might be problematic.

There's an extra level of indirection between the identified entry
resource and it's representations (which are in fact also
representations of *other* resources). There doesn't seem to be any
violation of webarch with that per se. But if we decide Cool URIs are
to be considered an exception rather than the norm and use workarounds
to fit with the half-baked 'permalink' practice then confusion is to
be expected.

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Danny Ayers <[email-masked]>
Date: Tue, 1 Feb 2005 11:52:10 +0100

Nearly forgot -

+1 to including some kind of explanatory note on comparisons, Martin's
version looks better than the current text

-- 

http://dannyayers.com

---

From: Martin Duerst <[email-masked]>
Date: Tue, 01 Feb 2005 16:56:55 +0900

>of the protocol or implementation.

It's okay for some network-oriented usage to work that way.
The network can fail any moment, anyway. But it's not helpful
at all in some contexts that are not network-oriented. A very
good case in point would be XML Namespaces. In that case,
unless there is a single rule for comparison that can be used
by all implementations, it would lead to some XML being
declared namespace valid by one XML processor, and not
namespace valid by some other XML processor. That was
clearly not acceptable.

 >is completely missing the point of the ladder.

It was added due to a security review initiated by the IESG.
I think I wrote the actual text, but the suggestion for
the MUSTs didn't come from the authors, but from the security
expert.

 >The identifiers may

The difference to the URI spec is not a good thing. But I disagree
with your explanation. Both XML Namespaces and RDF (which is
based on XML Namespaces) *require* character-by-character
comparison, for good reasons. Otherwise, concepts such as
conformance to XML Namespaces or equivalence of RDF statements
would just hang in the air. Security protocols are of course
another area of application where consistent behavior is
extremely important. "Sometimes it may match, sometimes not"
or "if you know all the schemes and protocols involved, the
server used on the other side, and the intent of the creator/
maintainer of the resource for what to do about it in the future,
you'll get consistent results" are just sometimes not good enough.

Regards,    Martin.

---

From: Graham <[email-masked]>
Date: Tue, 1 Feb 2005 14:23:34 +0000

On 1 Feb 2005, at 5:27 am, Roy T. Fielding wrote:

> Identifiers are not subject to

Define "equivalent" and/or "actually different" in spec text please. 
Good luck.

Graham

---

From: Tim Bray <[email-masked]>
Date: Tue, 01 Feb 2005 07:46:10 -0800

> Oh yeah, that'll be fun. ;-)

Well, it would be much better than we have now.  Anyone who subscribes 
to aggregations (for example, I subscribe to the planetsun.org 
aggregated feed), is used to seeing the same entry over and over and 
over again.  This problem is only going to get worse.  With Atom's ID 
semantics and compulsory <updated> timestamp, I would hope that my 
aggregator would have a chance of not showing me the same entry unless 
it's got a later timestamp than what I've seen.  -Tim

---

From: Henry Story <[email-masked]>
Date: Tue, 1 Feb 2005 17:06:48 +0100

>> Oh yeah, that'll be fun. ;-)

Yes, the id is an inverse functional property relating an Entry to a 
URI/IRI.
It is really just as simple as that. See the Ontology at:

<http://www.intertwingly.net/wiki/pie/AtomOWL>

All the rest is over specification.

Henry

---

From: Henry Story <[email-masked]>
Date: Tue, 1 Feb 2005 17:13:53 +0100

Duh. Sorry. It is a functional property. I explained why myself at 
length here:

<http://www.imc.org/atom-syntax/mail-archive/msg12067.html>

All the rest can arguably be thought to be overspecification.

Henry Story

---

From: Sam Ruby <[email-masked]>
Date: Tue, 01 Feb 2005 14:20:33 -0500

> it's got a later timestamp than what I've seen.  -Tim

Also remember that subscriptions are "opt-in".  I may chose to subscribe 
to PlanetApache.org (with whom I have a modicum of trust), but not to 
mymonkeysbutt.net (with which I presumably don't).

- Sam Ruby

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 1 Feb 2005 16:28:18 -0800

> it's got a later timestamp than what I've seen.  -Tim

The problem is that the requirement you suggested removes their
ability to use common sense in dealing with same-ID entries and
remain compliant with the format.  You don't need to tell aggregators
how to implement their trust mechanism.  Just tell them what the
format means and they will either improve their presentation or
lose to some other aggregator who does it right.

If you do want to place requirements on aggregators, then please
do so in a separate spec or in an appendix.  That way, us folks
who don't implement aggregators don't need to obey requirements
that conflict with basic content management and security principles.

....Roy

---

From: Tim Bray <[email-masked]>
Date: Tue, 01 Feb 2005 17:15:12 -0800

On Feb 1, 2005, at 4:28 PM, Roy T. Fielding wrote:

>> Anyone who subscribes to aggregations (for example, I subscribe to 
>> the planetsun.org aggregated feed), is used to seeing the same entry 
>> over and over and over again.  This problem is only going to get 
>> worse.  With Atom's ID semantics and compulsory <updated> timestamp, 
>> I would hope that my aggregator would have a chance of not showing me 
>> the same entry unless it's got a later timestamp than what I've seen. 
>>  -Tim
>
> format means

Uh... right.  What are we supposed to be disagreeing about?  I would 
think that a hint in the spec pointing out how you can use 
atom:id/atom:updated to improve handling of dupes in aggregated feeds 
couldn't hurt, I'm convinced that saying anything normative about what 
they should do won't help.  -Tim

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 1 Feb 2005 18:25:35 -0800

On Jan 31, 2005, at 11:56 PM, Martin Duerst wrote:
> namespace valid by some other XML processor.

Where is this myth coming from?  If a single document uses two
different URIs to name the same namespace (or multiple documents
using different URIs are merged) then an XML processor will consider
those qualified names as different names. That does not change their
validity whatsoever. It doesn't even change their data model, since
to be equivalent the two URIs must refer to the same namespace and
therefore the same data model. Validity is a property of DTDs and
Schema, not XML namespace processing, and neither DTDs nor Schema
are able to redefine the meaning of URIs.  The only purpose of the
comparison algorithm is to allow those technologies to take the
shortcut of evaluating structures based solely on the strcmp URI
comparison, leaving false-negatives to be resolved by the author.

> the MUSTs didn't come from the authors, but from the security

*grumble* If I had a dime for every time an IETF "security expert"
screwed up an application protocol, I'd at least have a grande
caramel machiatto to numb this headache.  It is never a good idea
to add requirements that make additional security-checking
a non-compliant application.

> comparison, for good reasons.

No, they require strcmp comparison for certain procedures related
to matching names.  They cannot require implementations to ignore
the equivalence of some URIs because that would change the meaning
of the statements being made, particularly for RDF.

> Otherwise, concepts such as
> would just hang in the air.

No, they don't -- the processing algorithm does not change the
meaning of the identifiers.  It only defines minimal conformance
criteria for implementations.  RDF statements do not know the
meaning of the URIs used to create those statements, nor do they
know what URIs are equivalent, but not knowing that two URIs are
equivalent does not change the fact that any statements made about
the resource of one URI must also be valid for the resource of the
other equivalent URI, since URIs cannot be equivalent if they
do not both identify the same resource.  In other words, there
are no closed-world theories of equivalence that override the
universality of URIs.  The same applies to IRIs.

> Security protocols are of course

Security protocols make comparisons based on what is being secured,
not based on some abstract theory.  They are capable of defining
that for themselves, consistently, and with respect to the resources
being secured rather than one identifier used to access those
resources.  The IRI spec doesn't know enough about an application's
needs to declare one form of comparison to be better than others.

....Roy