Subject: RE: name space application, copy of submitted internet draft

From: "Larry Masinter" <[email-masked]>
Date: Fri, 18 Jul 2003 12:18:26 -0700

Peter,

Consider instead of a URN namespace something that was more
compatible with "data:" (RFC 2397),

   "data-hash:" [ mediatype ] [ ";" hash-scheme ] "," hash-value

where mediatype is as in RFC 2397, and hash-scheme and
hash-value are as you've specified.

Note that RFC 2397 allows full content-type strings as 'mediatype',
e.g.,   data-hash:text/plain;charset=utf8;md5,01a1abc41123123121

I'm assuing it's useful to have a default for 'hash-scheme', too.

If you want to default mediatype to 'application/octet-stream',
that would be OK with me.

It seems like you want to identify the body but not the header.
I don't like the idea of making 'hash-scheme' extensible.
It's not "easy" to add a new hash scheme, because old software
would have to be updated.

I'm not particularly happy with the hack you've added for
separating message headers and bodies for message/rfc822. Are
you sure you're putting the multiplex at the right place?
You've added considerable complexity for blanking particular
fields of one particular media type, a whole 'mode' parameter,
all in the name of being able to hash the message body and
some, but not all, of the filters on what actually gets hashed.

Perhaps getting rid of those hacks would make this scheme
useless for the purpose for which you imagined it, but then,
maybe you don't actually need a URI/URN scheme at all, since
this really looks like a special protocol element which is
mainly only useful embedded within some other protocol.

Larry

---

From: Peter Thiemann <[email-masked]>
Date: 20 Jul 2003 21:41:14 +0200

Dear Larry,
>>>>> "lmm" == Larry Masinter <[email-masked]> writes:

    lmm> Peter,

    lmm> Consider instead of a URN namespace something that was more
    lmm> compatible with "data:" (RFC 2397),

    lmm>    "data-hash:" [ mediatype ] [ ";" hash-scheme ] "," hash-value

    lmm> where mediatype is as in RFC 2397, and hash-scheme and
    lmm> hash-value are as you've specified.

    lmm> Note that RFC 2397 allows full content-type strings as 'mediatype',
    lmm> e.g.,   data-hash:text/plain;charset=utf8;md5,01a1abc41123123121

    lmm> I'm assuing it's useful to have a default for 'hash-scheme', too.

    lmm> If you want to default mediatype to 'application/octet-stream',
    lmm> that would be OK with me.

I received a lot of feedback from several p2p mailinglists about the
draft proposal. Here is the summary of those comments and my proposed
reactions. 

1. the quasi-standard encoding of hash-value in p2p applications is
   base32 (RFC 3548) because
1.0 base32 is 25% shorter than base16
1.1 Content identifiers may appear inside other URIs or as
    filenames. Base64 characters may need to be escaped.
1.2 Content identifiers may get catalogued by full-text indexers
    (eg. google), Base64 contains punctuation characters. 
1.3 Truncated base32 to 4-6 characters is used as a nickname

1-action: base32 encoding for hash-value

    lmm> I don't like the idea of making 'hash-scheme' extensible.
    lmm> It's not "easy" to add a new hash scheme, because old software
    lmm> would have to be updated.

2. There was a lot of debate about hash-scheme. The biggest concern
   was that it should not be possible to add new hash-schemes
   easily, but mostly due to fear of broken hash functions.

2-action: fix all usable hash-schemes in the namespace definition. The
currently discussed list includes md5 (deprecated), sha1, sha256,
tiger and the combinations sha1.tiger (in common use) and
sha256.tiger. The precise semantics of a combined hash is still under
discussion.
[re your remark: from my understanding, it would be in line with the
meaning of URNs if not every resolver supported every possible
hash-scheme]

3. The meaning of the mediatype part was subject to discussion,
   too. The idea was that it should be verifiable or endorsable with
   the hash of the resource.

3-action: if media-type is specified, then the hash is computed from
the concatenation of the content-type/subtype and the actual
resource. This way, an endorsement of a URN means that the endorsing
party "vouches" for both the content of the resource and its
mediatype. This procedure also seems to make URNs harder to forge
(since not all content bits are arbitrary).

4. Compatibility with data was also discussed. 

4-action: Since the main target audience of the identifiers are
machines, the format should be trivial to parse. That lead to the
following revised syntax proposal

    lmm>    "data-hash:" [ mediatype ] [ ";" hash-scheme ] "," hash-value

   nss         = [mediatype] ":" [hash-scheme] ":" hash-value

   hash-scheme = hash-item *("." hash-item)
   hash-item   = "md5" / "sha1" / "sha256" / "tiger"
   hash-values = hash-value *("." hash-value)
   hash-value  = token

5. The name of the scheme.

5-action: does not really matter to me. Another proposal was to use
just "hash" (which would be short and sweet).

If that sounds acceptable to you, I'll revise the draft and resubmit it.

    lmm> I'm not particularly happy with the hack you've added for
    lmm> separating message headers and bodies for message/rfc822. Are
    lmm> you sure you're putting the multiplex at the right place?
    lmm> It seems like you want to identify the body but not the header.
    lmm> You've added considerable complexity for blanking particular
    lmm> fields of one particular media type, a whole 'mode' parameter,
    lmm> all in the name of being able to hash the message body and
    lmm> some, but not all, of the filters on what actually gets hashed.

    lmm> Perhaps getting rid of those hacks would make this scheme
    lmm> useless for the purpose for which you imagined it, but then,
    lmm> maybe you don't actually need a URI/URN scheme at all, since
    lmm> this really looks like a special protocol element which is
    lmm> mainly only useful embedded within some other protocol.

The discussion so far has convinced me that your argumentation here is
right. What the hack was meant for was a replacement for IMAP's UIDs,
so I will propose it separately just as that.

-Peter

---

From: Benja Fallenstein <[email-masked]>
Date: Mon, 21 Jul 2003 14:20:50 +0200

Hi Peter,

two points on your latest thoughts--

> sha256.tiger. The precise semantics of a combined hash is still under

The one in common use is sha1.tigertree, i.e. a Merkle tree hash based 
on the Tiger function.

Will have to ask Gordon and Justin for the latest spec, there was some 
backwards-incompatible change because someone discovered a serious 
security hole. The URNs should use the new spec.

> (since not all content bits are arbitrary).

This would break compatibility with my current software, at least, and 
our project would probably not adopt the URN namespace then (we had 
something more complicated before and decided that for simplicity we 
would put the content type in the URI rather than the hashed data, and I 
don't believe we'll switch back...).

Also, it wouldn't allow for something I see as very desirable: You 
discover that an audio recording of a politician's speech is already on 
Gnutella, so you create a URN of this form:

     urn:hash:audio/ogg;sha1,<hash>

and send it to a friend; your friend can download this through Gnutella 
from anybody who has a copy.

So, I would strongly suggest that you always simply use the hash of the 
content, not factoring in the content type.

Other than that, I'm fine with your design choices and would be fine 
with using your scheme, even though I think that there is a point in 
considering to standardize current practice-- which is 'urn:sha1:'.

All the best & thanks for your efforts,
- Benja

---

From: Peter Thiemann <[email-masked]>
Date: 22 Jul 2003 17:53:07 +0200

>>>>> "bf" == Benja Fallenstein <[email-masked]> writes:

    >> (since not all content bits are arbitrary).

    bf> This would break compatibility with my current software, at least, and
    bf> our project would probably not adopt the URN namespace then (we had
    bf> something more complicated before and decided that for simplicity we
    bf> would put the content type in the URI rather than the hashed data, and
    bf> I don't believe we'll switch back...).

My feeling is that input is needed from a specialist in hash functions
if the proposed way of computing the hash really makes a difference. I
was hoping that this had been investigated before, but I seem to be
wrong. In that case, it seems premature to put it into a name space
definition. Hence

3-action-revised: just take the hash of the resource as in the
original proposal.

    bf> Other than that, I'm fine with your design choices and would be fine
    bf> with using your scheme,

There is still the syntactic difference of the separation of the
optional parts (";" and "," vs ":"):

    bf>     urn:hash:audio/ogg;sha1,<hash>

One option would be to allow both syntaxes and declare one of them
the normalized syntax. But this is probably not a good idea.

    bf> even though I think that there is a point in
    bf> considering to standardize current practice-- which is 'urn:sha1:'.

This is a political issue of the IANA and would probably require a separate
internet draft / namespace application. On the positive side, it would
sanction a lot of ad-hoc uses of this scheme. On the negative side, it
may lead to a proliferation of top-level URN namespaces (one for each
hash scheme).

-Peter