Subject: Re: Message-ID draft, comments please

From: Graham Klyne <[email-masked]>
Date: Tue, 08 Jun 1999 20:29:36 +0100

At 14:45 08/06/99 +0200, Brad Knowles wrote:
>At 6:11 PM -0400 6/7/99, Matt Curtin wrote:
>
>> 3.3. Bringing it all together
>
>	[ ... deletia ... ]
>
>>   4. For each of the numbers generated in steps 2 and 3, convert the
>>      numbers to base 36 (using the characters 0-9 and A-Z), and write
>>      the numbers, with a "." between them.
>>
>>      (Base36 is recommended in order to reduce the overall length of
>>      the message ID; economy in message ID length is a good thing,
>>      given how prevalent they are.)
>
>	Given that base64 is a reasonably well-known algorithm (at least, 
>in the context of recent MTAs and MUAs that might need to do MIME 
>parsing), and that it is a more compact representation than base-36, 
>might I suggest that we recommend it instead?

Trouble is, it breaks in environments that cannot guarantee case sensitivity.

I'd suggest base32, since that is nearly as good as base36 and doesn't
require extended precision multiplication/division to decode/encode.

#g

------------
Graham Klyne
([email-masked])

---

From: Brad Knowles <[email-masked]>
Date: Wed, 9 Jun 1999 00:32:58 +0200

> require extended precision multiplication/division to decode/encode.

	Well, we'd have to document how to implement it properly, but 
since you'd be losing only one bit and the result would be 
insensitive to problems with case, I could support doing base-32 
instead.

-- 
   These are my opinions -- not to be taken as official Skynet policy
  ____________________________________________________________________
|o| Brad Knowles, <[email-masked]>            Belgacom Skynet NV/SA |o|
|o| Systems Architect, News & FTP Admin      Rue Col. Bourg, 124   |o|
|o| Phone/Fax: +32-2-706.11.11/12.49         B-1140 Brussels       |o|
|o| http://www.skynet.be                     Belgium               |o|
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
  Unix is like a wigwam -- no Gates, no Windows, and an Apache inside.
   Unix is very user-friendly.  It's just picky who its friends are.

---

From: Paul Overell <[email-masked]>
Date: Wed, 9 Jun 1999 10:18:33 +0100

In article <[email-masked]>, Graham
Klyne <[email-masked]> writes
>Trouble is, it breaks in environments that cannot guarantee case sensitivity.
Such an environment would, of course, be broken.  A local-part *is* case
sensitive.

Is this a real problem?  Turnpike has been using Base64 for its Message-
Ids for years, I've never had any reports of case sensitivity problems.
The only minor modification we made to Base64 was to replace the / with
$, this is because RFC1036 2.1.5. says "Slashes ("/") are strongly
discouraged", and our Message-Id generation is used for both mail and
news.

Should I be worried?

>require extended precision multiplication/division to decode/encode.

I prefer Base64, keeps the size of the message-ids down and Base64 is an
established standard.  Whether or not to replace / with $ is moot.

Regards
-- 
Paul Overell                                        T U R N P I K E  Ltd

---

From: Brad Knowles <[email-masked]>
Date: Wed, 9 Jun 1999 11:54:48 +0200

> established standard.  Whether or not to replace / with $ is moot.

	Well, let's assume you're encoding 128-bit MD5 (or SHA-1) hashes. 
If you used base-32 (five bits per character), this would require 26 
characters to encode that hash.  Base-64 could do it in 22 
characters.  I used to know what the math was that could convert 
to/from odd bases, but using bc and setting obase=36, it appears that 
you could represent 2^129-1 in 25 characters.

	If we could safely represent a 128-bit hash in 8-bit binary, we 
would only need 16 characters.

	So, in terms of how many characters are used to represent a 
128-bit md5 (or SHA-1) hash, we're certainly not going to get better 
than 16, and aren't likely to get any better than 22.

	However, if we were to use base-32 (the current worst case I 
believe that is being considered), we'd use only four more characters 
than are required for base-64.  I don't think that's much of a loss, 
although for purely aesthetic reasons, I would prefer base-64 over 
base-32.  Base-36 doesn't seem to be a sufficiently significant win 
over base-32 to make it worth the effort.

	If we did 160-bit hashes instead, the numbers would be 32, 32, 
and 27.  Again, not really all that much difference.

	So, I think the question the comes down to how much do we care 
about those environments that can't preserve case, and how does 
requiring that case be preserved in message-id hashes break things in 
ways that weren't already broken by their being required to preserve 
case elsewhere?

---

From: Matt Curtin <[email-masked]>
Date: 09 Jun 1999 09:22:01 -0400

>>>>> On Wed, 9 Jun 1999 11:54:48 +0200, Brad Knowles <[email-masked]> said:

Brad> 	So, I think the question the comes down to how much do we care
Brad> about those environments that can't preserve case, and how does
Brad> requiring that case be preserved in message-id hashes break
Brad> things in ways that weren't already broken by their being
Brad> required to preserve case elsewhere?

Agreed.  Does anyone have any examples of systems still in use that
cannot preserve case in the local-part?

-- 
Matt Curtin [email-masked] http://www.interhack.net/people/cmcurtin/

---

From: Robert Elz <[email-masked]>
Date: Thu, 10 Jun 1999 16:39:49 +1000

Date:        Wed, 9 Jun 1999 11:54:48 +0200
    From:        Brad Knowles <[email-masked]>
    Message-ID:  <v04205105b383e7d68e41@[195.238.1.121]>

  | 	So, in terms of how many characters are used to represent a 
  | 128-bit md5 (or SHA-1) hash, we're certainly not going to get better 
  | than 16, and aren't likely to get any better than 22.

It can be done in 20.   See rfc1924 ...   (unfortunately I suspect that
some generated would then need quoting, which would make those 22 again).

kre

---

From: Brad Knowles <[email-masked]>
Date: Thu, 10 Jun 1999 12:23:49 +0200

> some generated would then need quoting, which would make those 22 again).

	Whether it's a best non-binary case of 20 or 22, I don't think it 
really matters all that much.

	We're only talking about two characters, and in the overall 
scheme of things (with regards to suggested message-id formats), I 
don't think two characters is going to make a significant difference 
in the length of the generated ids.

---

From: Matt Curtin <[email-masked]>
Date: 10 Jun 1999 23:00:27 -0400

It sounds like we have consensus on recommending base64.

Has anyone been able to give any attention to the appendix?

(It was noted that the equation is suspect, since we assumed an even
distribution, which isn't true.  During the process of drafting the
appendix, we decided that the rate of blasting messages was high
enough that we could assume an even distribution that assumes every
second is greater than what a site of that configuration in practice.
Is this business of assuming that every second is the worst case
unreasonable?)

---

From: Robert Elz <[email-masked]>
Date: Fri, 11 Jun 1999 17:13:26 +1000

Date:        10 Jun 1999 23:00:27 -0400
    From:        Matt Curtin <[email-masked]>
    Message-ID:  <[email-masked]>

  | It sounds like we have consensus on recommending base64.

Please, no.  Not because there is anything wrong with base64 as such
(my reference to rfc1924 was a joke, as is that rfc), but because we
don't need to "recommend" anything with respect to the local part of
RFC generation.

It is just fine to pubish one (or more) informational RFCs pointing out
the pitfalls, and suggesting methods that generally will work, but to
go so far as to recommend one approach is probably just a bit too far.

kre

---

From: Brad Knowles <[email-masked]>
Date: Fri, 11 Jun 1999 10:34:33 +0200

> Is this business of assuming that every second is the worst case

	Whether it's actually unreasonable or not, it does not coincide 
with my experience.