Subject: RE: Message-ID (was: RE: Take 2: References/In-Reply-To replaceme nt text)

From: "Larry Osterman (Exchange)" <[email-masked]>
Date: Tue, 1 Jun 1999 12:57:59 -0700

The RFC822 grammer specifies that a message id be:

     optional-field =
                 /  "Message-ID"        ":"   msg-id

     msg-id      =  "<" addr-spec ">"            ; Unique message id

     addr-spec   =  local-part "@" domain        ; global address

     local-part  =  word *("." word)             ; uninterpreted
                                                 ; case-preserved

     domain      =  sub-domain *("." sub-domain)

     sub-domain  =  domain-ref / domain-literal

     domain-ref  =  atom                         ; symbolic reference

Note that it is totally legal for there to be only one sub-domain in the
<domain> construct.  However a FQDN requires a FULL dns name, which means
that there will be more than one sub-domain in the <domain> part of the
addr-spec.

The text in 6.2.3 indicates that a domain-ref must be THE official name of a
registry, network, or host, but it says NOTHING about the requirement that
this be fully qualified.

Your comment about authority gets to Robert Elz's comments about message-id
generation algorithm naming.  The reality is that even a FQDN is
insufficient to guarantee authenticity.  For example, if there are two
different clients with two different message-ID generation algorithms in use
on a single host (which is entirely possible), it is totally possible to
generate two different messages with the same message-ID.  Even if they use
the SAME generation algorithm (say # of seconds since 1-1-1970 + process
ID), there is still a window (albeit a very small one) where two clients
that happen to get the same process ID in the same second could generate the
same message ID.  And as someone I once knew was fond of commenting: "You
have to remember even if there's a one in a million chance of something
happening, in systems programming, one in a million happens next Tuesday".

The reality is that the requirement stated in RFC822 is sufficient - the
ENTIRE message-ID must be unique across time and space.  Not just the LHS,
and not just the RHS, but the entire message-id.  There are many mechanisms
to accomplish this, and I claim that RFC822-bis should not REQUIRE any of
them, but should instead should allow variability in algorithm.

The current text in RFC822-bis accomplishes this, it recommends an algorithm
but allows deviation from the algorithm, as long as the entire message-id is
guaranteed to be unique.

Larry Osterman
Sent from larryo-laptop.dns.microsoft.com running NT5 and Outlook 2000
and Exchange Server 5.5.  Please notify the sender of any difficulties

> -----Original Message-----
> From: Matt Curtin [mailto:[email-masked]]
> Sent: Tuesday, June 01, 1999 12:44 PM
> To: Larry Osterman (Exchange)
> Cc: 'Robert Elz'; Pete Resnick; DRUMS
> Subject: Message-ID (was: RE: Take 2: References/In-Reply-To 
> replacement
> text)
> 
> 
> >>>>> On Tue, 1 Jun 1999 11:46:18 -0700 ,
>       "Larry Osterman (Exchange)" 
> <[email-masked]> said:
> 
> Larry> Anyway, as to your comment about the RHS of the @ in a mesage
> Larry> ID being required to be a FQDN.  I don't understand this
> Larry> requirement at all.  It seems to me that it would be a
> Larry> significant modification of the RFC822 grammar, and I can't see
> Larry> any technical requirements for it.
> 
> How does the requirement for an FQDN in the RHS of a Message-ID modify
> RFC822 grammar?
> 
> The reason that is a requirement is because it is the RHS that defines
> authority.  That is, you may generate Message-IDs that are guaranteed
> to be unique only within the scope in which you have any authority or
> influence.  You have absolutely *no* way to prevent me, for example,
> from generating Message-IDs whose LHS values are identical to yours.
> The way that we solve such potential conflicts is to make the RHS of
> the Message-ID an FQDN.
> 
> Your assertion that the RHS is insignificant given your scheme is only
> valid when you assume that everyone on the planet is using your
> scheme, which I assure you, will never be the case.
> 
> -- 
> Matt Curtin [email-masked] 
http://www.interhack.net/people/cmcurtin/

---

From: Matt Curtin <[email-masked]>
Date: Tue, 1 Jun 1999 16:48:19 -0400 (EDT)

>>>>> On Tue, 1 Jun 1999 12:57:59 -0700 ,
      "Larry Osterman (Exchange)" <[email-masked]> said:

Larry> Note that it is totally legal for there to be only one
Larry> sub-domain in the <domain> construct.

Your assertion that a requirement for FQDN changes RFC822 grammar is
incorrect.  We are doing _nothing_ to change the grammar: we are
merely suggesting that codification of the current practice would be
to require that <domain> be an FQDN.

Larry> Your comment about authority gets to Robert Elz's comments
Larry> about message-id generation algorithm naming.  The reality is
Larry> that even a FQDN is insufficient to guarantee authenticity.

We're not talking about authenticity at all.

What we're talking about is the authority that a given generator has
to claim that its number is unique.  I have no authority to claim that 
a number I generate is unique within your namespace: I can only
influence that which is in my own namespace.

The way that we identify namespace is via FQDN.

I do not see why this is so difficult for you to understand.

Larry> And as someone I once knew was fond of commenting: "You
Larry> have to remember even if there's a one in a million chance of
Larry> something happening, in systems programming, one in a million
Larry> happens next Tuesday".

Blah blah blah.  We beat this to death on USEFOR.  The odds of
collision are in the same sort of probability as miscalculation.  You
are welcome to read the Internet Drafts and calculate the probability
of collision for yourself.

Larry> Not just the LHS, and not just the RHS, but the
Larry> entire message-id.  There are many mechanisms to accomplish
Larry> this, and I claim that RFC822-bis should not REQUIRE any of
Larry> them, but should instead should allow variability in algorithm.

We allow for variability in algorithm.  But the standard practice that 
has emerged is to all-but-eliminate the probability of collision by
allowing any given Message-ID generator a namespace in which to dole
Message-IDs.  And that is precisely the purpose of the RHS part of
Message-ID, as stated in RFC822.

-- 
Matt Curtin [email-masked] http://www.interhack.net/people/cmcurtin/

---

From: Keith Moore <[email-masked]>
Date: Tue, 01 Jun 1999 21:32:21 -0400

> registry, network, or host, but it says NOTHING about the requirement that

RFC 822 predates DNS, and largely predates the practice of domain naming,
though it was anticipated.  At the time 822 was written, there were 
still large numbers of "official" names in the NIC's HOSTS.TXT file 
which were only a single "atom", but there may have been some 
domain-style names also.  

However, the sense of "official" in RFC 822 was "registered with the NIC"
and today's analog would certainly be "registered in DNS".  Today,
an FQDN is the only kind of name which could be considered "official".
Part of the reason, after all, why it was important for the name to
be "official" (i.e. "registered with the NIC") was so that the name
would be visible everywhere and the meaning of that name would be 
unambiguous.

>  For example, if there are two
> same message ID.  

on reasonable systems, there's no way two different clients could get the
same process ID.  so message-id collisions would require time warps across 
reboots or across process-id wrapping.  no it's not impossible but 
the cases where message-id collisions happen "by accident" are far
fewer than the cases where they happen as a result of ignorance on
the part of the programmer.

> them, but should instead should allow variability in algorithm.

I agree that this is probably sufficient for the purpose of message-id,
and there is some precedent for it in MIXER's use of X.400 message-id
followed by "@MHS".

so should RFC822-bis also include a warning about the security risks of
basing message-ids on hardware serial numbers (as in when UUIDs are used)?

Keith

---

From: John C Klensin <[email-masked]>
Date: Tue, 01 Jun 1999 22:34:57 -0400

> would be visible everywhere and the meaning of that name would be 

Keith,

I don't have it in front of me, but my recollection is that we cleaned up
all or most of this in RFC 1123, or at least managed to express the clear
intent that "official" meant "primary/ canonical name registered in DNS".
Just in case anyone needs reminding (I know you don't) RFC 822 cannot be
read as a Standard without the qualifying/clarifying sections of 1123.

     john

---

From: Brad Knowles <[email-masked]>
Date: Wed, 2 Jun 1999 11:30:50 +0200

> but allows deviation from the algorithm, as long as the entire message-id is

	As far as this statement goes, that's fine.  Please tell me how 
the message-id generation algorithm you use (which does not include 
an FQDN on the RHS) *GUARANTEES* this uniqueness?

-- 
   These are my opinions -- not to be taken as official Skynet policy
  ____________________________________________________________________
|o| Brad Knowles, <[email-masked]>            Belgacom Skynet NV/SA |o|
|o| Systems Architect, News & FTP Admin      Rue Col. Bourg, 124   |o|
|o| Phone/Fax: +32-2-706.11.11/12.49         B-1140 Brussels       |o|
|o| http://www.skynet.be                     Belgium               |o|
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
  Unix is like a wigwam -- no Gates, no Windows, and an Apache inside.
   Unix is very user-friendly.  It's just picky who its friends are.

---

From: Robert Elz <[email-masked]>
Date: Wed, 02 Jun 1999 23:16:43 +1000

Date:        Tue, 1 Jun 1999 12:57:59 -0700 
    From:        "Larry Osterman (Exchange)" <[email-masked]>
    Message-ID:  <01D6C7224936D211BA450000F805D5380542167F@TOTO>

  | The RFC822 grammer specifies that a message id be:

I have no idea what this has to do with anything, the grammar for addresses
in From lines contains exactly the same "domain" token that the grammar for
message-ids contains.   The grammar doesn't pretend to include all the rules
that apply to language defined, it only includes the syntax.   There's also
a whole host of semantic requirements that need to be satisfied.  No-one is
planning on changing the grammar here, the discussion relates entirely to the
semantics.

That is, we're talking about what particular name should be used, not how
the name should be written.

  | However a FQDN requires a FULL dns name, which means
  | that there will be more than one sub-domain in the <domain> part of the
  | addr-spec.

Not necessarily.   "COM" is a FQDN.   It isn't very useful for anything
without subdomains, but it is a FQDN nevertheless.   (as is EDU, NET, even AU
(and should I determine it would be useful to do so, I could make AU as a FQDN
entirely reasonable to use, so that mail to postmaster@au would, or should,
work).

  | The text in 6.2.3 indicates that a domain-ref must be THE official name of a
  | registry, network, or host, but it says NOTHING about the requirement that
  | this be fully qualified.

If it isn't fully qualified, it isn't the official name of anything.
What it is then is an abbreviation.

  | The reality is that even a FQDN is
  | insufficient to guarantee authenticity.

Of course.   No-one ever claimed that the use of a FQDN itself was a panacea
and that all the rest of the problems would magically disappear.   What it
does is allow an algorithm for the LHS to be deployed such that that algorithm
does generate unique mesage-ids - it doesn't cause any algorithm to magically
acquire this property.

  | For example, if there are two
  | different clients with two different message-ID generation algorithms in use
  | on a single host (which is entirely possible), it is totally possible to
  | generate two different messages with the same message-ID.

Yes.   This can be a big problem, and I for one would like to see (somewhere)
a description of exactly this problem, and how to circumvent it.   Avoiding
it can be done many ways - one is to have the LHS contain an algorithm ID of
some kind, another is to use a different RHS for each algorithm, a sub-domain
of the host (or domain) name the software is running on.  Another is for the
algorithms to have their own domain names registered (best, as sub-domains
of the domain of their creators, or distributors) and to simply use those,
if the algorithm happens to be one where the LHS algorithm generates unique
ids regardless of the host on which it is running (possibly by including some
host identifciation in the LHS).

  | Even if they use
  | the SAME generation algorithm (say # of seconds since 1-1-1970 + process
  | ID), there is still a window (albeit a very small one) where two clients
  | that happen to get the same process ID in the same second could generate the
  | same message ID.

This is irrelevant.   All you're doing here is describing an environment and
algorithm pair where the algorithm is broken.   Big deal.   I can describe
a hundred other broken algorithms, and none of that would get us anywhere at
all.

The first assumption is that the algorithm in use actually does generate
unique LHS values - then we just need some way to identify the algorithm
so that the LHS values it generates won't clash with those generated by some
other algorithm.   That's what the RHS does.   And "@toto" as a RHS just
won't do it, unless the "@toto" is registered as a TLD, or is standardised
as a string to use for this purpose.

  | The reality is that the requirement stated in RFC822 is sufficient - the
  | ENTIRE message-ID must be unique across time and space.  Not just the LHS,
  | and not just the RHS, but the entire message-id.

Yes, that is exactly what we need.

  | There are many mechanisms
  | to accomplish this, and I claim that RFC822-bis should not REQUIRE any of
  | them, but should instead should allow variability in algorithm.

Of course.   On the other hand, we have to ensure that those variations in
algorithms don't create clashes amongst themselves.

You have yet to describe how that is supposed to be done.   All you have done
so far is to show how it is possible to screw up the algorithm in various
other ways.  No-one doubts that.   But assume for a minute that several of
us have LHS algorithms that are different to, but just as good as, yours.
If you like, you can assume that one of them is esentially the same as yours,
except that it combines the components in a different order (switches the
UUID from first to last, or vice versa, or something).

How are you proposing that these algorithms are to be prevented from generating
the same LHS at different times?   Or are you just hoping that the very
large number of bits will make it unlikely enough that the complaints won't
bother you too much?

  | The current text in RFC822-bis accomplishes this, it recommends an algorithm
  | but allows deviation from the algorithm, as long as the entire message-id is
  | guaranteed to be unique.

That is also fine.   What I initially wondered about, when I started this
thread, was exactly how that guarantee could possibly be made, without
making the RHS be a FQDN.   Now obviously there are lots of other methods
that could be defined - but are there any which make any less changes to
the basic rfc822 definitions, and current usages.

Until now this has mostly all "just worked", because "everyone" has simply
understood the mechanism involved.   It is exactly this which DRUMS is supposed
to be fixing - actually documenting all of this "extra knowledge" that is
required to actually make rfc822 messages work correctly.   The proper 
construction of the message-id is one of those things that ought to be being
documented by drums.  If we do that correctly, we won't have to repeat all of
this discussion in the future when someone else believes that their LHS
algorithm generates globally unique IDs already, and hence they don't have to
care about what is in the RHS, as clearly a globally unique ID, concatenated
with any other string, is still globally unique.   That is, I think, your
position, and it is, quite simply, wrong.  It doesn't work that way.

kre