Subject: Comments on draft-thiemann-cbuid-urn-00

From: Benja Fallenstein <[email-masked]>
Date: Thu, 10 Jul 2003 21:54:00 +0200

[cc:ed to p2p-hackers and urn-nid]

Dear Peter,

I just saw your recent I-D in the archives of the urn-nid mailing list. 
(I'm not subscribed, thus the delay in reaction.)

http://lists.research.netsol.com/pipermail/urn-nid/2003-June/000348.html

I'm working on a system (Storm, <http://sv.gnu.org/projects/storm/>) 
having very similar addressing needs; in fact, I've been in the process 
of preparing a registration for an informal URN namespace using MIME 
type + cryptographic hash to identify typed octet stream resources. We 
are building a p2p-based extension to the Web, based on this.

I think there are also plans to register a formal namespace for 
'bitprints'--

     http://bitzi.com/developer/bitprint

which are combinations of a SHA-1 hash and a Merkle hash tree root using 
  Tiger hashes. This namespace would not include content types.

Bitprints are used by Bitzi <http://bitzi.com> and Onion Networks 
<http://onionnetworks.com>. Our project also uses bitprints for 
compatibility. (The only thing setting our namespace apart is that we 
include MIME types.)

I hope that we can archieve some collaboration here. It would be good if 
we could reach agreement on a single namespace, so that URNs in this 
namespace can be resolved with either of our systems.

Below are some high-level comments regarding the approach of your I-D. 
(I'll save low-level comments for a later stage of the discussion. :) )

- It seems obvious that a shared namespace would be a good thing. If 
someone made a link between two files in one of your repositories, for 
example, and someone else published the linked-to file using our system, 
then a browser understanding both systems could follow the link from 
your repository by downloading the file through our system.

- Your proposal attempts to be general by allowing different hash 
functions to be used. I am wondering about that: It seems good practice 
to keep protocol parameters extensible, but it also means that there 
will be different ids for the same resource-- impractical when you try 
to look it up under one id, but it's stored under another!

On the other hand, if you say "it's *this* hash" there will be people 
who'll want to use another, and they'd have to create their own 
namespace. Since this is not a standard-- it's an informational RFC-- 
there is no reason why they wouldn't.

I think the way to go would be to provide a general namespace for 
hash-based URNs, but to specify one 'prefered' way to use it, noting 
that there are several systems implementing this way, already. But I'm 
not sure.

- All your identifiers use only a single hash function. Especially for 
URNs, which are supposed to be long-lived, there is a good reason to use 
hashes generated by more than one function: If one of the functions is 
broken (but the other isn't simultaneously), the ids don't become 
useless; you can use timestamping to extend the lifetime of your ids 
indefinitely (as long as the two hash functions you happen to use at a 
given time are never broken simultaneously). [unforch the only reference 
I can find on this right now is the US patent on it :-(, # 5,373,561 
<http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=/netahtml/srchnum.htm&r=1&f=G&l=50&s1=5,373,561.WKU.&OS=PN/5,373,561&RS=PN/5,373,561>.]

I would suggest adding the ability to use more than one hash function in 
the same URN.

The syntax could look for example like this:

     urn:cbuid:md5.sha1:<md5-hash>.<sha1-hash>

- Our project picked bitprint to be compatible-- because there were 
already at least two other independent projects using it, and it was 
explicitly promoted in the interest of compatibility. So I would suggest 
bitprints as the 'recommended' set of hash functions for this system. 
Using the above example syntax, they'd read like this:

     urn:cbuid:sha1.tigertree:<sha1-hash>.<tigertree-hash>

- The emerging 'industry standard' at least in the p2p community seems 
to encode hashes in base32; it provides shorter ids than base16 (hex), 
yet is also case-insensitive and uses only alphanumerics. Using the same 
hash, but different ASCII representations seems really icky: You *are* 
using the same technology, yet your ids don't resemble each other at 
all. I suggest that you use base32 in your namespace; see RFC 3548 if 
you need a definition.

- Your mechanism for hashing parts of an email separately is quite 
application-specific; that's a pity, given that the namespace is quite 
generally useful otherwise. Different applications may very easily have 
different needs for breaking up an entity into parts; for example, I 
could easily imagine that somebody would like to hash each body part of 
a multipart message independently.

It seems like solving your need through some mechanism outside this 
namespace registration would make the namespace simpler and more 
generally useful. What exactly do you need it for, anyway?

- Two syntax considerations: Firstly, it would seem like a good idea to 
choose a syntax similar to that of RFC 2397 (The "data" URL scheme) 
since it also represents the MIME types of the resources it identifies 
in the URI. So you'd have something like,

     urn:cbuid:sha1:text/plain,<sha1-hash>

In the flavor of URI that doesn't include content types, you could 
simply leave that part off.

     urn:cbuid:sha1:<sha1-hash>

(Unambiguous because hashes cannot contain commas.) This would probably 
make the scheme more attractive to folks who don't want to include 
content types, since there would be no 'artefacts' (":*:") related to 
them in the content-type-less syntax.

Secondly and finally, cbuid is hard to remember and easy to misspell, 
IMHO. If it's going to be a general namespace for cryptographic 
hash-based id, I'd propose simply calling it 'hash'--

    urn:hash:sha1:<sha1-hash>

:-)

Hoping to spark some discussion,
- Benja

---

From: "Zooko" <[email-masked]>
Date: 15 Jul 2003 12:22:32 -0400

> Can you give me a reason why p2p folks stick to base32? I don't really
> see an advantage over base64 for ids that are never handled by humans.

This was discussed on the p2p-hackers mailing list:

http://zgp.org/pipermail/p2p-hackers/2001-September/date.html

Look for messages with the subject "please prefer base 64 over base 32".  
I named that thread, because I started it by arguing in favor of base 64.  

In the course of the discussion Gordon Mohr and others convinced me to change 
my mind, and I adopted base-32 afterward.

Regards,

Zooko

http://zooko.com/

---

From: Justin Chapweske <[email-masked]>
Date: Tue, 15 Jul 2003 20:12:39 -0500

I won't both the CFRG with this post.

> 
> I need the first kind of semantics, though, if I want to use hash URNs 
> as reliable, trusted, *context-less* identifiers. (I.e., without being 
> able to give more than one URN in an <a href>.) I think having a way to 
> convey these semantics would be good, even if some people will implement 
> it in an insecure way.
> 

I see how your requirements are different.  Perhaps the solution is to 
define a generic URI scheme that allows composition of multiple URIs to 
identify the content.  It could be done in a fashion similar to the DURI 
draft 
(http://www.watersprings.org/pub/id/draft-masinter-dated-uri-03.txt) and 
could be made independant of hashing if you wished.

So, it could be similar in spirit to the urn:hashes that you mentioned 
in a previous post.

I'm actually rather suprised that I've never seen a URI format defined 
that simply implies equivilence between a set of sub-URIs.

-- 
Justin Chapweske, Onion Networks
http://onionnetworks.com/

---

From: Carl Ellison <[email-masked]>
Date: Wed, 16 Jul 2003 08:06:07 -0700

Benja,

	I apparently don't know the rules for URN formation.  I thought it
was completely free after the "urn:".  Is that not true?

	I'm skipping most of this conversation, to comment on a single point
at the end..

	Is urn:sha1:dRDPBgZzTFq7Jl2Q2N/YNghcfj8= not now legal?

At 12:57 AM 7/16/2003 +0200, Benja Fallenstein wrote:
>If we allow all possible subsets, using URN namespace names becomes 
>problematic; we'd have to register
>
>     urn:sha1:...
>     urn:sha256:...
>     urn:tiger:...
>     urn:sha1+sha256:...
>     urn:sha1+tiger:...
>     urn:sha256+tiger:...
>     urn:sha1+sha256+tiger:...
>
>(urks). Or, we'd have to do something like, use urn:sha1:... for
>just  one hash but urn:hashes:sha1+sha256:... for more than one
>hash. Then,  we'd only have to register
>
>     urn:sha1:...
>     urn:sha256:...
>     urn:tiger:...
>     urn:hashes:...
>
>Opinions?

When you combine hash functions, you need to specify the combining
function also.  I assume you were assuming mere concatenation, here. 
It's often more than that.  So, the concatenation function would have
to be listed also.

My personal preference is for anyone who wants to do this to declare
a new hash function name and define it as the particular combination
function of the particular other hash functions, but use that new
hash function name in things like the URN construct.  We've seen a
couple of these, but not very many and I don't see a reason to
encourage people to do this kind of concatenation.

 - Carl

+--------------------------------------------------------+
|Carl Ellison      Intel R & D       E: [email-masked] |
|2111 NE 25th Ave                    T: +1-503-264-2900  |
|Hillsboro OR 97124                  F: +1-503-264-3375  |
|PGP Key ID: 0xFE5AF240                                  |
|  1FDB 2770 08D7 8540 E157  AAB4 CC6A 0466 FE5A F240    |
+--------------------------------------------------------+

---

From: Benja Fallenstein <[email-masked]>
Date: Wed, 16 Jul 2003 17:35:50 +0200

Hi Carl,

> was completely free after the "urn:".  Is that not true?

No. After the "urn:" comes a "namespace identifier," a colon, and a 
"namespace-specific string," i.e.

     urn:<namespace id>:<namespace-specific string>

Assignment of namespace ids is a manged process, i.e., you cannot just 
create your own, you have to register it with the IETF. Normally this 
requires an RFC. The namespace registration explains how the 
namespace-specific string is interpreted.

For more info, see RFCs 2141 and 3406. The official registry of 
namespace ids is at:

     http://www.iana.org/assignments/urn-namespaces

> 	Is urn:sha1:dRDPBgZzTFq7Jl2Q2N/YNghcfj8= not now legal?

No, since there is no registered 'sha1' namespace.

>>    urn:sha1+sha256+tiger:...
[...]
> It's often more than that.  So, the concatenation function would have

Hm, can you explain what else would be needed? Do we really need other 
combinations than concatenation in URNs-- what would be the 
applications? (I'm not familiar with other ways in which you would want 
to combine hash functions.)

> encourage people to do this kind of concatenation.

Hm. Maybe you're right. My one point against this would be that if you use

     urn:sha1.tigertree:...

it's easier for a developer who supports only SHA-1 to realize that they 
can provide at least partial verification than it is if you use

     urn:bitprint:...

and define 'bitprint' as, 'SHA-1 concatenated with a Tiger tree-hash.'

But I think I'd be fine with the latter, if it is consensus.

- Benja

---

From: Benja Fallenstein <[email-masked]>
Date: Wed, 16 Jul 2003 00:57:12 +0200

Hi all,

This is a discussion about the registration of a URN namespace based on 
cryptographic hashes, i.e., a namespace of octet streams as identified 
by their hashes.

Some security-related issues have (unsurprisingly) turned up, so I'm now 
cross-posting to three mailing lists--

- urn-nid (technical discussion of URN namespace registrations)
- p2p-hackers (because one important user base would be the p2p community)
- the IRTF crypto forum (to pick their minds about the security issues).

I think it's appropriate in all three, but please stay strictly on topic :-)

Justin Chapweske wrote:
 > [Peter Thiemann wrote: -bf]
>> Well, I'm pretty sure that the URN scheme should *not* fix one
>> particular hash function. Instead, it should be extensible so that it
>> does not become obsolete just because a hash function is broken or
>> somebody discovers a new super-safe or super-efficient hash function.

[Later in the mail, Peter even suggested a registry for hash functions.]

> I actually disagree.  In order to avoid chosen-algorithm attacks,
 > the set of standardized hashes should be kept very small and directly
 > under the control of the IETF and at the guidance of the CFRG.

I agree; if we have a urn:hash: namespace or similar, the list of 
allowable hash functions should be fixed in the namespace registration, 
and only be changeable by going through the RFC process.

Regarding chosen-algorithm attacks, we have to keep in mind what our 
adversary model is. It should be noted that the hash function is picked 
by the person publishing a URN, not by the person a file is downloaded 
from. An attack using a hash collision would still be possible, but it 
would go more like this:

- Find a hash collision, H = h(x) = h(y), x != y.
- Use x to obtain some form of a "good rating" on H; this could include 
something like having an independent, widely trusted entity publish H as 
the hash of some good data, or even obtaining a signature on H.
- Make y available for download with H as its id.

If the URN is published by someone else than the adversary, the 
adversary doesn't get to choose the algorithm.

 > By having it as a top level URN scheme such as urn:sha1, implementors
 > will tend to focus on the algorithm itself rather than the notion
 > of generic hash-based naming.

Maybe you're right. OTOH this doesn't generalize to using combinations 
of hash functions (more on this below).

> We must avoid having a generic hash name space that allows developers
 > to add new hashes willy-nilly.  I believe that many non crypto-savvy
 > developers would tend to support every algorithm specified in the
 > name space without understanding that by supporting multiple agorithms
 > you introduce a weakest-link condition.

These are separate issues. Having developers add non-standard hash 
functions seems like a really bad idea. Supporting every function in the 
namespace registration may be a bad idea, but do you trust 
non-crypto-savvy developers to make a sensible *choice* of which 
algorithms to support (assuming that there *are* namespaces for 
different functions, including maybe md5 for backward compatibility).

It should also be noted that the spec can try to educate those people 
that do read it. Of course there are always those that never read the 
spec, just use what the others use.

> Also, for reasons of interoperability, it would be useful if the number
 > of different URNs be kept small.

I agree, but what about developers who don't want to use e.g. SHA1 for 
technical reasons? (In particular hash size; possibly also backwards 
compatibility, especially if they have a running system they want to 
augment with URNs.) Well, maybe such developers aren't plentiful; 
anybody know any? (Not sure how on-topic this is for CFRG tho.)

 > A large number of the P2P developers have agreed upon SHA1 for the time
 > being, which could potentially enable very simple interoperability
> between these systems.

True, but there are also the two technical points against using SHA1 alone:

- Many systems today use hash trees for multisource downloading. (Of 
course you can use a SHA1 hash tree, but that obviously doesn't give you 
the interoperability with plain SHA1 systems.)
- Using just one function makes repair after the function is broken much 
more difficult (again, more below).

> I also think some mechanism should be introduced to deprecate hash schemes
 > over time.  So while urn:md5 would be a decent name space to add for
 > compatibility with existing MD5-based applications, such a scheme 
should be
 > immediately denoted as being deprecated to avoid having new applications
 > adopt MD5.

Agreed.

>>     BF> I would suggest adding the ability to use more than one hash function
>>     BF> in the same URN.
>>
>>     BF> The syntax could look for example like this:
>>
>>     BF>      urn:cbuid:md5.sha1:<md5-hash>.<sha1-hash>
>>
>> That sounds like a good proposal to me, it gives you increased
>> confidence and robustness virtually for free. I'll put a concrete
>> syntax proposal at the end of this message.
> 
> I don't believe this is a good idea from a security perspective.  I fear that
 > most implementors would only verify one of the hashes, and unless 
they make a
 > judicious choice about the hash to verify, they are again opening 
themselves to
 > a chosen-algorithm attack.  Otherwise, implementors w/o the proper 
context to
 > decide which hash is the strongest are forced to verify both hashes, 
which will
 > cut their hashing performance roughly in half.
 >
> Zooko, do you have any thoughts on this?  These seem like the types of attacks
 > that you've spent a lot of time thinking about.

Zooko replied:
> I *have* thought about this issue.  The short of it is: I'm not sure how to do 
> it right, so I'm not going to do it.

(I understand this as: "I'm not going to use more than a single hash 
function in the id.")

> For example, what should the required/allowed behavior be when one of the 
> hashes fails to match and the matches?  Are implementors required to check all 
> hashes that are included?

My take on this is: Generally, yes.

If you don't, you open yourself to a very similar attack to the one I 
described above regarding chosen-message attacks. Assume that two hash 
functions, g(.) and h(.) are in use.

- Create a URN containing g(x), h(y) for x != y.
- Using x, obtain an endorsement for the URN from someone who verifies 
only g(.).
- Use the endorsement to "sell" y to someone who verifies only h(.).

Yes, verifying only one of the two hashes takes longer. Well, verifying 
one hash takes longer than verifying zero hashes; let's specify a system 
without this kind of security holes.

I think that using a URN which includes two hashes should be read as a 
statement by the URN's creator that they think verifying two hashes is 
reasonable. If you don't agree with them, don't resolve their URN in the 
first place...

(I don't see why Justin thinks that verifying only one hash would 
introduce chosen-algorithm attacks, though. Assume that a verifier is 
capable of verifying both g(.) and h(.), but verifies only g(.), which 
happens to be insecure. I think it is reasonable to think that the 
verifier would also accept a URN including *only* a g(.) hash; thus, the 
double hash isn't necessary for the attack. And if the developer of the 
verifier knew that g(.) was insecure, surely they wouldn't make their 
program verify only g(.) in the first place. Maybe someone can enlighten 
me on chosen-algorithm attacks made possible by using two functions.)

The one exception I would make to having to verify both hashes is when 
the verifier doesn't have an implementation of both hash functions. In 
these cases, I would say that an implementation MAY reject the URN, but 
if it does not, it SHOULD provide an indication to the user that the 
verifier cannot guarantee that everybody else will see the same file 
behind this URN as given. The verifier could also provide an alternative 
URN where this *can* be guaranteed (i.e., one containing only one hash).

> I suspect that there might be a way to do secure, smooth, backwards-
> compatible upgrade past certain kinds of hash algorithm breakages.  However, 
> I haven't seen a complete description of how it would work.

Ok. I assume that you mean, a way to continue to use identifiers even 
after the hash functions used in these identifiers are all broken.

For the following, you have to have a secure timestamping service. 
Assume that your identifiers include two hashes, using g(.) and h(.). 
Assume that g(.) is broken, but h(.) still works. Assume that you have 
decided upon a new hash function, i(.), which you will in the future use 
together with h(.). For every file x where you have the contents, not 
just the hash, you can:

- Compute h(x) and i(x).
- Timestamp the statement: "H = h(x) and I = i(x) are hashes of the same 
file."

Now suppose that h(.) is broken, but i(.) still works. Assume that you 
have an old-style identifier containing g(x) and h(x). Assume that you 
have downloaded y, an alleged copy of x; you want to verify it against 
the identifier containing G = g(x) and H2 = h(x). You have the timestamp 
certificate of the above statement.

You now verify that g(y) = G; h(y) = H = H2; and i(y) = I.

If this holds, you can be sure that y = x. Proof: Assume that y != x. We 
know that h(y) = h(x). That's not something big; at the time of 
verification, we know ways of computing collisions for h(.).

However, the timestamped statement contains I = i(y). At the time of 
verification, it is believed that nobody can obtain hash collisions on 
i(.). Therefore, at the time of verification, the adversary must have 
used y to compute i(y), or, alternatively, created a timestamp for every 
possible value of i(.). Since timestamping involves hashing, creating a 
timestamp for every possible value of i(.) requires at least as much 
computing power as mounting a birthday attack on the function used, 
which is held to be impossible. Thus, someone has intentionally 
associated h(x) with i(y), and thus y.

To know that it makes sense to associate h(x) with y, the person must 
have known, at the time, that h(x) = h(y).

Therefore, if y != x, then someone must have known that h(x) = h(y), 
**at a time where no way to compute a hash collision on h(.) was known, 
yet**. This is held to be impossible; thus, y = x, q.e.d.

Does this satisfy you?

Note that you cannot use the above in a system using only one hash 
function-- because once your hash function breaks, your timestamps, 
using that hash function, break as well! At every time, you need at 
least one hash function that remains unbroken.

I should also note that the above idea is patented in the US and 
probably elsewhere. However, patents expire (this one in the 2010s, I 
think), and this is long-term thinking.

> (The algorithm-negotiation features of SSL/TLS might be considered an example 
> of what *not* to do, and in any case they cannot be carried over to this 
> application since they are interactive.)

Yup and yup. (Some protocols may use hash URNs as return values and may 
include algorithm negotiation, but that would be a security issue with 
those protocols, not with the hash URN spec.)

We can of course use a different URN namespace identifier for each hash 
function, i.e.,

     urn:sha1:...

A point in favor of this would be that people use it already.

I'm not sure how this interacts with using different hashes in a single 
URN, though. I think there should be a method to do so, in order to 
enable the lifetime extension as described above, and in order to 
combine 'industry standard' SHA-1 hashes with a tree hash that enables 
multisource downloading.

I think that the specification should probably say that an 
implementation must check all hashes, except at explicit user 
discretion. I think that if an an application does check all hashes, 
there are no security considerations with using more than one hash. OTOH 
there are security benefits-- if only that birthday attacks become 
harder to mount because of the additional bits.

The question then is, if we allow e.g. {sha1,sha256,tiger} each by 
itself, do we also allow every possible combination of the above? Or do 
we limit ourselves to a specified set of combinations, e.g. sha1+tiger 
but not sha1+sha256?

I believe that there is no security gain from allowing the former but 
disallowing the latter. OTOH I could easily imagine that if sha1+tiger 
is popular, some people would like to use sha1+tiger+sha256 for added 
security.

There is the issue of having less URNs per resource, of course, which is 
outside the security domain I think. I'm not sure how much a concern 
this is... opinions?

     urn:sha1:...
     urn:sha256:...
     urn:tiger:...
     urn:sha1+sha256:...
     urn:sha1+tiger:...
     urn:sha256+tiger:...

(urks). Or, we'd have to do something like, use urn:sha1:... for just 
one hash but urn:hashes:sha1+sha256:... for more than one hash. Then, 
we'd only have to register

     urn:sha1:...
     urn:sha256:...
     urn:tiger:...
     urn:hashes:...

Opinions?

Thanks,
- Benja

---

From: "Zooko" <[email-masked]>
Date: 16 Jul 2003 13:25:48 -0400

Coincidentally, there is an active (and contentious) discussion of 
cryptography-based naming at the "cryptography" mailing list.

Three different concrete proposals, including one already deployed and one 
newly announced, have been mentioned.

[email-masked]

---

From: Justin Chapweske <[email-masked]>
Date: Wed, 16 Jul 2003 12:12:39 -0500

> 	Is urn:sha1:dRDPBgZzTFq7Jl2Q2N/YNghcfj8= not now legal?

The sha1 urn scheme is not yet registered, but its de facto usage 
utilizes a Base32 encoding, not Base64.

---

From: Benja Fallenstein <[email-masked]>
Date: Wed, 16 Jul 2003 19:32:35 +0200

Hi Zooko,

> newly announced, have been mentioned.

Is there an archive?

> [email-masked]

"subscribe cryptography" gives:
**** subscribe: unknown list 'cryptography'.

- Benja

---

From: "Gordon Mohr" <[email-masked]>
Date: Tue, 15 Jul 2003 08:51:19 -0700

> see an advantage over base64 for ids that are never handled by humans.

Content identifiers are often sent via email; they are 
also (very occasionally) put on paper. 

Content identifiers also appear inside other URIs (such
as HTTP) or as filenames. Base64 characters aren't always
legal in filenames, and may need to be escaped in other
URIs.

Content identifiers also occasionally get usefully catalogued
by full-text indexers, but Base64 characters are usually
considered word-boundaries or other punctuation by such tools. 
Thus Base32 identifiers can be sought as atomic words, while 
Base64 identifiers cannot. 

For example, try:
  http://www.google.com/search?q=EPH3MTGDELUYJU7UDWSRA6B3PAYVEILO

Sometimes it may be handy to use a truncation of the full 
identifier -- say the first 4-6 characters -- as a nickname
of the full identifier to distinguish (in a non-secure way)
between variants of otherwise similar files. Such use only
heightens the other problems with Base64: your short
identifier could be "/a+A/4" instead of something like
"AZB3A4".

- Gojomo @ Bitzi
____________________   
Gordon Mohr <gojomo@ . . . At Bitzi, people cooperate to identify, rate,
bitzi.com> Bitzi CTO . . . describe and discover files of every kind. 
_ http://bitzi.com _ . . . Bitzi knows bits -- because you teach it!

---

From: "Zooko" <[email-masked]>
Date: 15 Jul 2003 12:43:57 -0400

[...]
> I don't believe this is a good idea from a security perspective.  [...]
[...]
> Zooko, do you have any thoughts on this?  These seem like the types of 
> attacks that you've spent a lot of time thinking about.

So, lacking a clear understanding of how to do secure and useful algorithm 
upgrade, I have satisfied myself with simply hardcoding SHA-1.  I accept that 
the identifiers thus generated might become unreliable in as little as ten 
years' time.  When I *do* decide to change algorithms, I intend to do so 
unambiguously by either changing the namespace or relying on the length of the 
identifier.

Mnet URI's currently look like this:

mnet:38ppp56jbb8b64zrh8reoadzgn1zpdxc76enkmqduwtf4tug

(They use SHA-1, exclusively and non-optionally.)

If I were to change to SHA-256 or something, I would probably make them look 
like one of these:

mnet:7nbcku4ijbk848kgzakqs316hdnbb6magsqag5hybrw1gmqf5b46xwenwz9um1qqhw8o

(Uses the new, longer length to indicate that it uses the new algorithm.)

or:

mneu:7nbcku4ijbk848kgzakqs316hdnbb6magsqag5hybrw1gmqf5b46xwenwz9um1qqhw8o
znet:7nbcku4ijbk848kgzakqs316hdnbb6magsqag5hybrw1gmqf5b46xwenwz9um1qqhw8o
mnet2:7nbcku4ijbk848kgzakqs316hdnbb6magsqag5hybrw1gmqf5b46xwenwz9um1qqhw8o

(Uses a different namespace.)

I'm not at all concerned about using more namespace identifiers.  There will 
be tens of thousands of different namespace identifiers used during the next 
ten years, nearly all of which will immediately die and become corpse 
namespaces.  If Mnet still has any relevance at all in 2013, then it will be 
deserving of another namespace identifier.

Regards,

Zooko

http://zooko.com/

---

From: Justin Chapweske <[email-masked]>
Date: Tue, 15 Jul 2003 11:11:35 -0500

> somebody discovers a new super-safe or super-efficient hash function.

I actually disagree.  In order to avoid chosen-algorithm attacks, the 
set of standardized hashes should be kept very small and directly under 
the control of the IETF and at the guidance of the CFRG.  By having it 
as a top level URN scheme such as urn:sha1, implementors will tend to 
focus on the algorithm itself rather than the notion of generic 
hash-based naming.

We must avoid having a generic hash name space that allows developers to 
add new hashes willy-nilly.  I believe that many non crypto-savvy 
developers would tend to support every algorithm specified in the name 
space without understanding that by supporting multiple agorithms you 
introduce a weakest-link condition.

Also, for reasons of interoperability, it would be useful if the number 
of different URNs be kept small.  A large number of the P2P developers 
have agreed upon SHA1 for the time being, which could potentially enable 
very simple interoperability between these systems.

I also think some mechanism should be introduced to deprecate hash 
schemes over time.  So while urn:md5 would be a decent name space to add 
for compatibility with existing MD5-based applications, such a scheme 
should be immediately denoted as being deprecated to avoid having new 
applications adopt MD5.

> syntax proposal at the end of this message.

I don't believe this is a good idea from a security perspective.  I fear 
that most implementors would only verify one of the hashes, and unless 
they make a judicious choice about the hash to verify, they are again 
opening themselves to a chosen-algorithm attack.  Otherwise, 
implementors w/o the proper context to decide which hash is the 
strongest are forced to verify both hashes, which will cut their hashing 
performance roughly in half.

---

From: Justin Chapweske <[email-masked]>
Date: Tue, 15 Jul 2003 19:13:16 -0500

> more difficult (again, more below).

As they are defined today 
(http://open-content.net/specs/draft-jchapweske-thex-02.html), the tree 
hashes are parametizable to allow different file segment (leaf) sizes to 
be specified.  IFAIK other traditional digest algorithms are set in 
stone, allowing no parameterization(?).

If appropriate, I would be interested in certain hash tree forms being 
validated and standardized as normal digest functions.  This would allow 
hash tree forms to be incorporated into existing standards that rely on 
a normal digest function.

In regards to using multiple functions, most current systems that output 
multiple functions simply output them as multiple seperate pieces of 
metadata.  For instance, a system could output something like the following:

X-Content-URN: urn:sha1:FAB6CX2GZSWOOWCPXXBYSFBUSN4LIGTF
X-Content-URN: urn:tree:tiger:3FOBMWPE2JED5DUN2VA6J7DGSNJNJILE4HRF6SQ

I believe that in many systems the hashes will simply be treated as 
pieces of meta-data that are used to verify the integrity of a file. 
Just because they contain 'urn' in them doesn't mean that they have to 
be used as identifiers.  So, if you look at it from the meta-data 
perspective, I think its natural to use multiple independant hashes 
rather them glomming them all together.

> me on chosen-algorithm attacks made possible by using two functions.)

My point is subtle and perhaps irrelevant, but let me try to clarify:

When I see something like:

urn:hash:sha1.md5:<foo>

This implies to me that I am obligated (not sure to whom) to verify both 
hashes.  I believe these semantics are reasonable, however you will find 
very few developers will be willing to follow these semantics and verify 
both hashes.

However, when I see something like:

X-Content-URN: urn:md5:42J46YB3Y3OLLFYL52B4LNDE34
X-Content-URN: urn:sha1:FAB6CX2GZSWOOWCPXXBYSFBUSN4LIGTF
X-Content-URN: urn:tree:tiger:3FOBMWPE2JED5DUN2VA6J7DGSNJNJILE4HRF6SQ
X-Content-URN: urn:crc32:<foo>

This implies to me that I should apply some sort of ranking between the 
algorithms and not use any that are below my standards.  If I am not 
confident about any single hash, I am free to verify multiple of them.

Obviously from a technical perspective, both approaches are the same, it 
just seems to me that the first approach invites developers to defy the 
semantics, while the second approach is likely to be a healthier approach.

I am not a developer psychologist, so I'm very open to other viewpoints 
on this.

---

From: Benja Fallenstein <[email-masked]>
Date: Wed, 16 Jul 2003 03:02:25 +0200

Hi,

> a normal digest function.

Seems appropriate to me.

> rather them glomming them all together.

Well, I use them as context-less names that resolve to something; and 
you simply cannot give two different URNs in one <a href="...">. If you 
use them as metadata about a different resource, then I agree you don't 
need to put different hashes into a single URN.

(OTOH, if you go the other way around, and want to give RDF metadata 
about a resource identified by a hash-URN-- e.g., ratings or 
locations--, there are good reasons for using a URN with more than one 
hash again, so that the resource name you store information about 
doesn't need to become invalid if one hash function is broken.)

>> verifier would also accept a URN including *only* a g(.) hash; thus, 
>> the double hash isn't necessary for the attack. And if the developer 
>> of the verifier knew that g(.) was insecure, surely they wouldn't make 
>> their program verify only g(.) in the first place. Maybe someone can 
>> enlighten me on chosen-algorithm attacks made possible by using two 
>> functions.)
> 

I dunno... if the spec explains how this is a security leak, and the 
developers still do it, I think there's little I can do...

> confident about any single hash, I am free to verify multiple of them.

I would actually agree with your assessments of the meaning of both 
statements -- the first implies you have to verify both, the second 
implies you have to verify as many as you deem necessary.

So your point is: Because many developers will-- maybe-- use the second 
kind of semantics always, having the form aiming for the first kind of 
semantics is futile.

Seems to me like this is generally useful where people use URNs for 
identifiers. I mean, when I post a URN in a forum, giving two URNs seems 
less practical and posting a number of URNs and saying "Download as many 
of these as you need to meet your security requirements, and verify that 
they are all equal" seems weird. :-) You should click on it and your 
software should handle the rest.

My current thinking is that if the spec explains in which way verifying 
only one hash is a security leak, and if developers still do it, then 
it's an issue between the developers and their users-- if the users and 
developers are willing to accept the security leak of having somebody 
recommend a URN they haven't fully checked, then that's their problem, 
I'd think.

- Benja

---

From: Peter Thiemann <[email-masked]>
Date: 15 Jul 2003 17:14:37 +0200

Dear Benja,

    BF> [cc:ed to p2p-hackers and urn-nid]

    BF> I just saw your recent I-D in the archives of the urn-nid mailing
    BF> list. (I'm not subscribed, thus the delay in reaction.)

    BF> http://lists.research.netsol.com/pipermail/urn-nid/2003-June/000348.html
[
Nice to see that somebody is listening to this list, since nobody is
really subscribed to it. I only received a message that my message
awaits moderator approval.
]

    BF> I'm working on a system (Storm,
    BF> <http://sv.gnu.org/projects/storm/>)

The project description link on that page is stale.

    BF> having very similar addressing needs; in fact, I've been in the
    BF> process of preparing a registration for an informal URN namespace
    BF> using MIME type + cryptographic hash to identify typed octet stream
    BF> resources. We are building a p2p-based extension to the Web, based on
    BF> this.

This is not quite what we are after. We are interested in using p2p
techniques for implementing a global sharable mail store. The main
application is to use the ids as a replacement for IMAP's concept of
message uids to allow for easy synchronization of client caches and
replicas of the mail server. Hence the bias towards the message/rfc822
type in the draft. However, the intention was to design a scheme which
is usable for other information storage and retrieval. Perhaps, your
storm project can provide the storage layer that we need? 

    BF> 'bitprints'-- ...

Thanks for the pointer! I was not aware of those efforts.

    BF> I hope that we can archieve some collaboration here. It would be good
    BF> if we could reach agreement on a single namespace, so that URNs in
    BF> this namespace can be resolved with either of our systems.

Same here.

    BF> Below are some high-level comments regarding the approach of your
    BF> I-D. (I'll save low-level comments for a later stage of the
    BF> discussion. :) )

    BF> - It seems obvious that a shared namespace would be a good thing. If
    BF>   someone made a link between two files in one of your repositories,
    BF>   for example, and someone else published the linked-to file using our
    BF>   system, then a browser understanding both systems could follow the
    BF>   link from your repository by downloading the file through our
    BF>   system.

I agree completely. That's why we made some effort in keeping the
scheme simple and extensible. This kind of support also makes the
namespace application stronger. 

    BF> - Your proposal attempts to be general by allowing different hash
    BF>   functions to be used. I am wondering about that: It seems good
    BF>   practice to keep protocol parameters extensible, but it also means
    BF>   that there will be different ids for the same resource-- impractical
    BF>   when you try to look it up under one id, but it's stored under
    BF>   another!

My general feeling is that this is not a huge problem as long as you
stay inside one application framework. For example, in our
application, you perform search queries against a database of meta
information and then you get the results in terms of these URNs. Next,
you try to retrieve (some of) these URNs basically from known hosts. 
So our application relies more on the uniqueness properties to achieve
distribution and replication rather than on the naming function.

    BF> On the other hand, if you say "it's *this* hash" there will be
    BF> people who'll want to use another, and they'd have to create
    BF> their own namespace. Since this is not a standard-- it's an
    BF> informational RFC--  
    BF> there is no reason why they wouldn't.

    BF> I think the way to go would be to provide a general namespace for
    BF> hash-based URNs, but to specify one 'prefered' way to use it, noting
    BF> that there are several systems implementing this way, already. But I'm
    BF> not sure.

    BF> - All your identifiers use only a single hash function. Especially for
    BF>   URNs, which are supposed to be long-lived, there is a good reason to
    BF>   use hashes generated by more than one function: If one of the
    BF>   functions is broken (but the other isn't simultaneously), the ids
    BF>   don't become useless; you can use timestamping to extend the
    BF>   lifetime of your ids indefinitely (as long as the two hash functions
    BF>   you happen to use at a given time are never broken
    BF>   simultaneously). [unforch the only reference I can find on this
    BF>   right now is the US patent on it :-(, # 5,373,561
    BF>   <http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=/netahtml/srchnum.htm&r=1&f=G&l=50&s1=5,373,561.WKU.&OS=PN/5,373,561&RS=PN/5,373,561>.]   

This is interesting but not directly relevant (I think) because we are not
dealing with certificates here. Rather you want to increase the
confidence (if a server supports more than one hash function) and
robustness (if a server supports just one of a selection of hashes) of
a data access. I really don't see why timestamping should be required
because each hash value lives indefinitely long.

    BF> - Our project picked bitprint to be compatible-- because there were
    BF>   already at least two other independent projects using it, and it was
    BF>   explicitly promoted in the interest of compatibility. So I would
    BF>   suggest bitprints as the 'recommended' set of hash functions for
    BF>   this system. Using the above example syntax, they'd read like this:

    BF>      urn:cbuid:sha1.tigertree:<sha1-hash>.<tigertree-hash>

I don't think this recommendation should be a part of a namespace
application. This choice is really application dependent, so it is
part of the application's description. I would not mind, though, to
use it in an example (like: if you transform the id like this, then
you get a valid bitprint id), I just would not want to make a
normative statement. 

    BF> - The emerging 'industry standard' at least in the p2p community seems
    BF>   to encode hashes in base32; it provides shorter ids than base16
    BF>   (hex), yet is also case-insensitive and uses only
    BF>   alphanumerics. Using the same hash, but different ASCII
    BF>   representations seems really icky: You *are* using the same
    BF>   technology, yet your ids don't resemble each other at all. I suggest
    BF>   that you use base32 in your namespace; see RFC 3548 if you need a
    BF>   definition.

This is a hairy issue. I understand your reasoning. After studying the
RFC I tend *not* to commit to any particular coding, but rather make
the encoding a parameter with some reasonable default. Then the
identifier equivalence section should state explicitly which
representation is considered the normalized one. 

I'm not sure about the reasonable default.
Can you give me a reason why p2p folks stick to base32? I don't really
see an advantage over base64 for ids that are never handled by humans.

    BF> - Your mechanism for hashing parts of an email separately is quite
    BF>   application-specific; that's a pity, given that the namespace is
    BF>   quite generally useful otherwise. Different applications may very
    BF>   easily have different needs for breaking up an entity into parts;
    BF>   for example, I could easily imagine that somebody would like to hash
    BF>   each body part of a multipart message independently.

This is a misunderstanding of the intention, so this requires
clarification in an update of the draft. The point is that many data
formats contain header or other meta information (emails, images,
mp3). The mode parameter signals that this meta information is
ignored and only the raw contents are hashed. Since the hash only
determines the raw contents, the specification needs to define how to
complete the contents to a valid instance document of the specified
type. In the case of an email message, this means to add the required
fields from the RFC2822 specification. For other formats, other things
have to be done. And for formats that only consist of raw contents, it
will not make sense to define a mode 1 id.

    BF> It seems like solving your need through some mechanism outside this
    BF> namespace registration would make the namespace simpler and more
    BF> generally useful. What exactly do you need it for, anyway?

see the top of this mesage.

    BF> - Two syntax considerations: Firstly, it would seem like a good idea
    BF>   to choose a syntax similar to that of RFC 2397 (The "data" URL
    BF>   scheme) since it also represents the MIME types of the resources it
    BF>   identifies in the URI. So you'd have something like,

    BF>      urn:cbuid:sha1:text/plain,<sha1-hash>

I don't think it is a good idea to put the mediatype at this point
because it separates the sha1 and the <sha1-hash> fields which belong
together, logically. 

    BF> In the flavor of URI that doesn't include content types, you could
    BF> simply leave that part off.

    BF>      urn:cbuid:sha1:<sha1-hash>

    BF> (Unambiguous because hashes cannot contain commas.) This would
    BF> probably make the scheme more attractive to folks who don't want to
    BF> include content types, since there would be no 'artefacts' (":*:")
    BF> related to them in the content-type-less syntax.

I don't see the point of doing so. Fact is that the ids are to be
processed by machines and only occasionally will they pop up "in the
human eye". Reading and writing them should be straightforward
to implement. Hence, my preference is to have a fixed number of fields
separated by ":" and simply leave unspecified slots empty (admittedly,
the "*" is artificial). That is

	urn:cbuid::sha1.tigertree::<sha1-hash>.<tigertree-hash>

[the empty slot after the sha1.tigertree is reserved for specifying the
encoding of the following hashes, see the grammar below]

    BF> Secondly and finally, cbuid is hard to remember and easy to misspell,
    BF> IMHO. If it's going to be a general namespace for cryptographic
    BF> hash-based id, I'd propose simply calling it 'hash'--

    BF>     urn:hash:sha1:<sha1-hash>

Well, I could be talked into that. The name 'cbuid' stems from the idea
that those URNs are going to generalize IMAP's uids as mentioned
above. However, given that these URNs are not for human consumption
(or are they), I see no convincing technical argument in favor of any
particular name. Do you have one?

Proposed Grammar:

   cbuid-nss   = type-spec ":" hash *1(":" type-specific-extension)
   type-spec   = [media-type *parameter]
   parameter   = ";" "mode" "=" 1*DIGIT / ";" token "=" token
   token       = 1*(ALPHA / DIGIT)
   hash        = hash-scheme ":" hash-enc ":" hash-values *("/" hash-values)
   hash-scheme = hash-item *("." hash-item)
   hash-item   = "md5" / "sha1" / "hash127" / hash-token
   hash-token  = token
   hash-enc    = ["base16" / "base32" / "base64"]
   hash-values = hash-value *("." hash-value)
   hash-value  = token

Changes:
- typespec can be empty
- more than one hash function
- hash-enc[oding] added

Questions:
* does there have to be a registry for names of hash functions?
  Clearly, the namespace definition cannot be updated for each new
  hash function.
* which value of <hash-enc> should be the default (and probably be the
  result of the normalization process)? I can guess your answer...

Best wishes
-Peter

---

From: Michael Mealling <[email-masked]>
Date: 20 Jul 2003 18:35:10 -0400

> was completely free after the "urn:".  Is that not true?

Essentially, yes. There are syntactic restrictions that exist for all
URIs but that's fairly generic. URNs still have to be persistent so you
can put things like domain-names in there unless you time/date stamp
them....

-Michael Mealling

---

From: Dirk-Willem van Gulik <[email-masked]>
Date: Sun, 20 Jul 2003 20:48:30 +0200 (CEST)

> 	Is urn:sha1:dRDPBgZzTFq7Jl2Q2N/YNghcfj8= not now legal?

Until there is a registered URN namespace/authority called 'sha1' - nope.

You could use x-sha1 - but that is kind of frowned upon.

> >     urn:sha1+sha256+tiger:...

Or regigster

	urn:assortedhashes:....
or
	urn:hash:....

and write an RFC which would document that it has to have the
shape:

	urn:hash:[a-z\+]+:<base32string>

And then in that RFC either define all of the above; or devise a scheme
by which additional A+B+C's can be added to that block.

Obviously in the above replace s/hash/by-whatever/ you want.

Dw.