Subject: Re: draft-resnick-2822upd-02 and Netnews
From: Russ Allbery <[email-masked]>
To: [email-masked]
Date: Tue, 31 Jul 2007 09:58:09 -0700
List-Id: <ietf-822.imc.org>

"Charles Lindsey" <[email-masked]> writes:

> I think, as Russ has explained, that just about every existing Netnews
> relaying agent would not be able to do that.

> Imagine a news server that is offered 100,000 articles a day, and that
> keeps a History file of two million or so articles that it knows about,
> and it has to discover, for each of those 100,000 articles, whether its
> <msg-id> is already recorded in its History file.

> No implementor can afford to examine each incoming <msg-id> for special
> cases such as quoted-pairs that in practice are virtually never seen. The
> most he can afford to do is to read the incoming stream until he finds a
> '>', and that is then the msg-id he is going to use.

Well, it's not quite as impossible as that.  It's conceivable that one
could write a normalizer for message IDs and use it on all incoming
message IDs before hashing and before doing the uniqueness check.  But
that's what it would take: a normalizer, not just a parser, and the
additional overhead and (more to the point) code complexity on the
handling path of every incoming article offer.

I'm fairly certain no netnews implementor has ever bothered, given that
the only failure case is that two message IDs using syntactical elements
rarely seen in practice will appear to be different if they use those
elements differently.

> Yes, the USEFOR WG discussed these cases (but not extensively, since the
> bulk of its members had a good understanding of how Netnews worked and
> were well aware of what would not work). So the necessity for that SP
> was reaffirmed (several times, because newcomers regularly raised it).

Yes, I guarantee that substantial amounts of netnews software will fail
completely to parse headers that do not have a SP after the colon.  I
can't speak to e-mail software.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.mime:5054
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-resnick-2822upd-02 and Netnews
Date: Tue, 31 Jul 2007 11:19:58 GMT
Lines: 103
Sender: [email-masked]

In <p06250108c2d3f6265dd0@[172.28.170.243]> Pete Resnick <[email-masked]> writes:

>On 7/30/07 at 11:32 AM +0000, Charles Lindsey wrote:

>>It would probably be helpful for a start to state clearly that what 
>>is actually written in a domain literal (whether in an address or a 
>>mesg-id) MUST/SHOULD agree with the specification of some widely 
>>recognized numbering/routeing scheme (which, at the moment means IP4 
>>or IP6 of course).

>Why can't that be done in the documents which define where these 
>things get used, thereby limiting dependencies.

I think you misunderstand what I was saying. Sure some document (e.g. RFC
2821, but there may be more in the future) will say that, for its
purposes, domain literals are limited to such and such (likely described
by ABNF).

What I am suggesting for RFC 2822bis is that it should state clearly that
domain literals SHOULD (maybe even MUST) NOT be used unless there exists
some identifiable document which allows that particular domain literal.

>>However, suppose some future addressing scheme _does_ admit quoted-pairs

>First off, let's note that it is not the addressing scheme that 
>admits quoted pairs; it is the representation of those addresses in a 
>message. Things that interpret those addresses parse out the quoted 
>pairs (and square brackets and the CFWS for that matter) to get the 
>address.

Sure. If we allow '\' in dcontent (as I suggested, but without any
connotation that it is a quoted-pair) then some new addressing protocol is
welcome to say that `\` has some special semantic meaning such as to quote
the following character - supposing that such a feature facilitates the
routeing/transport/whatever which uses that new kind of address.

But that should be totally opaque to pure mail protocols which create or
handle messages (i.e. they have no business "improving" things by
substituting 'q' when they see '\q'). If and when such a messages comes to
be tranported by the new protocol using the new address format, then sure
that protocol can choose to regard '\q' and 'q' as eqivalent, if that is
how that protocol works.

>That said, imagine that it makes perfect sense for a new addressing 
>scheme decides to use a special other than "." in a domain literal, 
>and it further turns out that parsers do bad jobs if those characters 
>are not quoted. We might want to encourage the use of quoted pairs in 
>those cases.

As I said, that is an internal matter to be specified by and for the new
addressing scheme.

>And again, I can't get all that excited (without some real examples) 
>about implementations that can't go through a text string and unquote 
>quoted pairs.

I think, as Russ has explained, that just about every existing Netnews
relaying agent would not be able to do that.

Imagine a news server that is offered 100,000 articles a day, and that
keeps a History file of two million or so articles that it knows about,
and it has to discover, for each of those 100,000 articles, whether its
<msg-id> is already recorded in its History file.

No implementor can afford to examine each incoming <msg-id> for special
cases such as quoted-pairs that in practice are virtually never seen. The
most he can afford to do is to read the incoming stream until he finds a
'>', and that is then the msg-id he is going to use.

>>There undoubtedly exists software within Netnews that will break if 
>>this SP is absent...
>>... since it is the invariable practice within email MUAs to include 
>>this SP, putting such a MUST in email would incur no problems...
>[and later]
>>There undoubtedly exists Netnews software which will break without 
>>that whitespace...
>>
>>Yes there are undoubtedly news implementation that would break in 
>>that situation...

>Undoubtedly? Invariable? Use of these words *increase* rather than 
>decrease my suspicion, and therefore my desire to leave things 
>exactly as they are now.

Yes, the USEFOR WG discussed these cases (but not extensively, since the
bulk of its members had a good understanding of how Netnews worked and
were well aware of what would not work). So the necessity for that SP was
reaffirmed (several times, because newcomers regularly raised it).

When our draft was finally submitted to the IESG, one of its members
queried this particular matter, and after we had explained the reasons and
the situation, our text was accepted as it stood.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 31 Jul 2007 14:00:04 +0100
From: Tony Finch <[email-masked]>
To: Pete Resnick <[email-masked]>
cc: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: draft-resnick-2822upd-02 and Netnews
Sender: [email-masked]

On Sun, 29 Jul 2007, Pete Resnick wrote:
>
> My main issue here is that we already have examples where limiting the
> syntax in "obvious" ways in 2822 screwed up 2821 (e.g., "Received:"
> syntax) and I'd prefer not to shoot myself in the foot again. Do we know
> of implementations (especially non-netnews implementations) which can't
> deal with quoted-pair in domain literals or message ids, or won't strip
> whitespace out of domain literals? Should this really come out?

821 is much stricter about the syntax of domain literals than 822, so an
SMTP agent that doesn't look at the message data can reasonably decline to
handle quoted pairs in domain literals.

Exim goes a bit further and does not support domain literals at all unless
you turn the feature on. Even then it only supports 821-style strict
syntax (plus IPv4: and IPv6 tagged address literals), without space or
quotes.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
IRISH SEA: SOUTHERLY, BACKING NORTHEASTERLY FOR A TIME, 3 OR 4. SLIGHT OR
MODERATE. SHOWERS. MODERATE OR GOOD, OCCASIONALLY POOR.

From: Russ Allbery <[email-masked]>
To: Pete Resnick <[email-masked]>
Cc: "Charles Lindsey" <[email-masked]>, [email-masked]
Subject: Re: draft-resnick-2822upd-02 and Netnews
Organization: The Eyrie
Date: Mon, 30 Jul 2007 20:32:15 -0700
Sender: [email-masked]

Pete Resnick <[email-masked]> writes:

> And again, I can't get all that excited (without some real examples)
> about implementations that can't go through a text string and unquote
> quoted pairs.

I know you may be asking for a mail implementation, for which this isn't
so useful.

NNTP absolutely bans all whitespace in message IDs, in a quoted-pair or
not, in the protocol.  Using any form of whitespace, however escaped, in a
message ID breaks the NNTP protocol at a fundamental level and breaks
every NNTP implementation I'm aware of.  Hence, messages with message IDs
containing whitespace cannot be conveyed over NNTP, and therefore are
essentially impossible in netnews.

Apart from that, all netnews server implementations I'm aware of do a
character-by-character comparison of message IDs, without regard for
escaping, quoting, or other subtleties that may render two differently
encoded message IDs identical.

Appendix A.2 of RFC 3977 is a good summary from a netnews perspective.
Note that this specification allows for converting the message ID in the
message to a canonical form for NNTP, in part because the ongoing
difference between message ID specifications in netnews and e-mail may
make this useful, but I don't know of any existing implementations that
make use of this allowance.

A.2.  Message-IDs

   Every article handled by an NNTP server MUST have a unique
   message-id.  For the purposes of this specification, a message-id is
   an arbitrary opaque string that merely needs to meet certain
   syntactic requirements and is just a way to refer to the article.

   Because there is a significant risk that old articles will be
   reinjected into the global Usenet system, RFC 1036 [RFC1036] requires
   that message-ids are globally unique for all time.

   This specification states that message-ids are the same if and only
   if they consist of the same sequence of octets.  Other specifications
   may define two different sequences as being equal because they are
   putting an interpretation on particular characters.  RFC 2822
   [RFC2822] has a concept of "quoted" and "escaped" characters.  It
   therefore considers the three message-ids:

      <[email-masked]>
      <"ab.cd"@example.com>
      <"ab.\cd"@example.com>

   as being identical.  Therefore, an NNTP implementation handing email
   articles must ensure that only one of these three appears in the
   protocol and that the other two are converted to it as and when
   necessary, such as when a client checks the results of a NEWNEWS
   command against an internal database of message-ids.  Note that
   RFC 1036 [RFC1036] never treats two different strings as being
   identical.  Its successor (as of the time of writing) restricts the
   syntax of message-ids so that, whenever RFC 2822 would treat two
   strings as equivalent, only one of them is valid (in the above
   example, only the first string is valid).

   This specification does not describe how the message-id of an article
   is determined; it may be deduced from the contents of the article or
   derived from some external source.  If the server is also conforming
   to another specification that contains a definition of message-id
   compatible with this one, the server SHOULD use those message-ids.  A
   common approach, and one that SHOULD be used for email and Netnews
   articles, is to extract the message-id from the contents of a header
   with name "Message-ID".  This may not be as simple as copying the
   entire header contents; it may be necessary to strip off comments and
   undo quoting, or to reduce "equivalent" message-ids to a canonical
   form.

   If an article is obtained through the IHAVE command, there will be a
   message-id provided with the command.  The server MAY either use it
   or determine one from the article contents.  However, whichever it
   does, it SHOULD ensure that, if the IHAVE command is repeated with
   the same argument and article, it will be recognized as a duplicate.

   If an article does not contain a message-id that the server can
   identify, it MUST synthesize one.  This could, for example, be a
   simple sequence number or be based on the date and time when the
   article arrived.  When email or Netnews articles are handled, a
   Message-ID header SHOULD be added to ensure global consistency and
   uniqueness.

   Note that, because the message-id might not have been derived from
   the Message-ID header in the article, the following example is
   legitimate (though unusual):

      [C] HEAD <[email-masked]>
      [S] 221 0 <[email-masked]>
      [S] Path: pathost!demo!whitehouse!not-for-mail
      [S] Message-ID: <[email-masked]>
      [S] From: "Demo User" <[email-masked]>
      [S] Newsgroups: misc.test
      [S] Subject: I am just a test article
      [S] Date: 6 Oct 1998 04:38:40 -0500
      [S] Organization: An Example Net, Uncertain, Texas
      [S] .

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: [email-masked]
Cc: Charles Lindsey <[email-masked]>, [email-masked]
Date: Mon, 30 Jul 2007 17:17:37 -0700 (PDT)
Subject: Re: draft-resnick-2822upd-02 and Netnews
To: Pete Resnick <[email-masked]>
Sender: [email-masked]

> I want to start out by saying that in answering this, I'm trying to
> take RFC 2119, section 6 *very* seriously:

> 6. Guidance in the use of these Imperatives

>     Imperatives of the type defined in this memo must be used with care
>     and sparingly.  In particular, they MUST only be used where it is
>     actually required for interoperation or to limit behavior which has
>     potential for causing harm (e.g., limiting retransmisssions)  For
>     example, they must not be used to try to impose a particular method
>     on implementors where the method is not required for
>     interoperability.

> Insofar as I interpret this:

> MUST means, "You must do this or seriously bad things will happen,
> either with regard to interoperation with other implementers of this
> specification or known harm will come to this or other parts of the
> Internet." SHOULD means, "You probably want to do this, and not doing
> so may cause some interoperability problems, but you can do this if
> you fully understand the ramifications and you have a good reason to
> do so."

I am in complete agreement with your assessment here.

> If below I ask why something should be a MUST or a SHOULD, I'm
> looking for an answer in light of the above.

> Also, Charles alone does not rough consensus make. If there are
> others who strongly feel that these changes should be made (and
> strongly would be good; we're trying to push this along through the
> standards track, and changes at this point would be less than
> desirable), please speak up and indicate why the current
> specification does "the wrong thing".

I won't object strongly if these changes are made, however my personal position
is that none of them have been sufficiently justified to be done at this stage
of the game.

> Preliminaries out of the way:

> On 7/26/07 at 9:44 PM +0000, Charles Lindsey wrote:

> >...a <quoted-pair> is still allowed within a <no-fold-literal>.
> >True, there is a mention that "other specifications" will limit it,
> >but no such specifications are mentioned.  Pointers to
> >draft-ietf-usefor-usefor-12 and to RFC2821-bis would be in order
> >here (cf such a mention in 3.4.1).

> I'll do something similar to 3.4.1.

> >However, I cannot conceive that any future IP-address format or
> >other domain literal would ever contain a <quoted-pair> or any
> >whitespace (that, together with '>', is the other thing Netnews
> >cannot tolerate at this point) - there are just too many existing
> >protocols that would promptly
> >fall over. So why not fix the problem once and for all now? And fix
> >it for <domain-literal> at the same time - that and
> ><no-fold-literal> and the only places where <dtext> and <dcontent>
> >are used, so it is easily done, and the obs-syntax would remain, of
> >course.

> My main issue here is that we already have examples where limiting
> the syntax in "obvious" ways in 2822 screwed up 2821 (e.g.,
> "Received:" syntax) and I'd prefer not to shoot myself in the foot
> again. Do we know of implementations (especially non-netnews
> implementations) which can't deal with quoted-pair in domain literals
> or message ids, or won't strip whitespace out of domain literals?
> Should this really come out?

Our implementation has handled quoted-pairs in domain literals since it was
first written. However, while we allow whitspace in domain literals, we don't
drop whitespace embedded inside domain literal content as we should when
interpreting the value. But I'd rather fix our implementation (in fact I've
already done so) than change the specification to deal with this - I don't
think we have any way to be sure that such a change wouldn't have some adverse
impact somewhere. Frankly, I think dropping the ability to quote stuff would
actually be a bit safer than mucking around with whitespace rules that impact
where folding can occur.

> >And finally, a couple of nits I have spotted:
> >
> >In 3.2.2, in the Note:, "no-fold-quote" no longer exists, and
> >"no-fold-literal" need not be mentioned because the mention of
> >"dcontent" covers it.

> Fixed.

> >In 3.6.4, near the bottom of page 27, you need to add quoted-strings
> >to the list of things now not allowed in msg-ids.

> Fixed.

> >2. SP after the ':' in headers
> >------------------------------
> >
> >That is REQUIRED in draft-ietf-usefor-usefor-12, and it is REQUIRED
> >in the new NNTP standard RFC 3977. Since every known MUA already
> >generates headers that way

> Please some support for this, including a review of the DRUMS
> archive. If we're going to make this change, you do the research. I
> seem to remember a discussion of this during DRUMS, and having the
> space optional was where we ended up. Can we check?

The problem is we're not just talking about full fledged MUAs. A simple message
submission agent can be written in 50 or so lines of code in most languages and
on most platforms. As a result there are a bagillion such agents out there
hardcoded into all sorts of applications and even hardware. And since a
significant fraction of MTAs fix up stuff like this, especially when dealing
with submission rather than relay, there's a lot of marginal code in use that
could be pushed over the edge by such a change. And yes, while I've yet to see
a full-fledged MUA that doesn't put in the space, I've seen several of these
bits of code that do not. (I usually fix them when I come across them, which is
why I know people write them this way.)

This issue has abated somewhat now that people code more and more in higher
level languages with built in mail sending capabilities or "mail classes" ready
to hand or whatever, but I think we're still a long way from this being a
completely safe change to make.

> >There is one slight consequence that you should then forbid header
> >lines with only whitespace after the header-name (because trailing
> >whitespace has a habit of getting lost).

> And this gets us into a "changes with potential side-effects" problem
> that worries me.

Me too.

> >3. CFWS between msg-ids in References
> >-------------------------------------
> >
> >It is currently [CFWS], but usage in Netnews has always placed
> >whitespace there (and References has always been more of a News
> >featrure than an Email feature). Would it hurt to REQUIRE it?

> See top of this message. Would it hurt to *not* have it as a MUST?

> >And also, please can we promote its use in Replies from SHOULD to
> >MUST? It was SHOULD in RFC2822, but many MUAs have not taken the
> >"hint" and, as a result, threading in mailing lists often gets
> >broken, which is a confounded nuisance. It is invariably done
> >properly within News.

> This is absolutely a violation of the 2119 definition in section 6.
> You must provide a better reason to make it a MUST.

Agreed.

> >4. Allowed positions for <comment>s
> >-----------------------------------
> >
> >Allowing <comment>s in Message-ID header fields will break Netnews,
> >and therefore the Usefor draft disallows them. Would it hurt to
> >bring Email into line? For sure nobody ever uses them there.

> Again, potentially breaking some netnews implementations is not by
> itself a reason to remove this IMO. Are there known non-netnews
> implementations that break because of this? (And why are netnews
> implementations so non-robust in the first place, unable to parse
> comments and quoted-pairs?)

I've only seen commetns show up in an id field a couple of times, and each time
it apparently was because of some very broken logic attempting to generate an
id based in part on what was in the From: field. I have no problem breaking
stuff that's already deeply broken, but OTOH are we really sure nobody does
this? We said something similar about MIME-Version:, only to find an
implementation almost immediately that put comments in every time.

				Ned

Date: Mon, 30 Jul 2007 15:31:15 -0500
To: "Charles Lindsey" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: draft-resnick-2822upd-02 and Netnews
Cc: [email-masked]
Sender: [email-masked]

On 7/30/07 at 11:32 AM +0000, Charles Lindsey wrote:

>It would probably be helpful for a start to state clearly that what 
>is actually written in a domain literal (whether in an address or a 
>mesg-id) MUST/SHOULD agree with the specification of some widely 
>recognized numbering/routeing scheme (which, at the moment means IP4 
>or IP6 of course).

Why can't that be done in the documents which define where these 
things get used, thereby limiting dependencies.

>However, suppose some future addressing scheme _does_ admit quoted-pairs

First off, let's note that it is not the addressing scheme that 
admits quoted pairs; it is the representation of those addresses in a 
message. Things that interpret those addresses parse out the quoted 
pairs (and square brackets and the CFWS for that matter) to get the 
address.

That said, imagine that it makes perfect sense for a new addressing 
scheme decides to use a special other than "." in a domain literal, 
and it further turns out that parsers do bad jobs if those characters 
are not quoted. We might want to encourage the use of quoted pairs in 
those cases.

And again, I can't get all that excited (without some real examples) 
about implementations that can't go through a text string and unquote 
quoted pairs.

>There undoubtedly exists software within Netnews that will break if 
>this SP is absent...
>... since it is the invariable practice within email MUAs to include 
>this SP, putting such a MUST in email would incur no problems...
[and later]
>There undoubtedly exists Netnews software which will break without 
>that whitespace...
>
>Yes there are undoubtedly news implementation that would break in 
>that situation...

Undoubtedly? Invariable? Use of these words *increase* rather than 
decrease my suspicion, and therefore my desire to leave things 
exactly as they are now.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Mon, 30 Jul 2007 15:08:04 -0500
To: Alan Barrett <[email-masked]>, Philip Guenther <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Cc: ietf-822 mailing list <[email-masked]>
Sender: [email-masked]

On 7/30/07 at 9:56 AM -0600, Philip Guenther wrote:

>>In message <[email-masked]>, Alan 
>>Barrett <[email-masked]> writes:
>>
>>>At 10:25 PM -0500 7/29/07, Pete Resnick wrote:
>>>>I don't know how to read 3.1.4 and 3.4.3 in RFC 822 and not come 
>>>>up with that interpretation.
>>>
>>>So I would say that, given input like "+0000", the sign and the 
>>>digits together make up an "atom" per the description in section 
>>>3.1.4 and the grammar in section 3.3.  The ( ("+" / "-") 4DIGIT ) 
>>>production in the grammar in section 5.1 is looking inside a 
>>>single token, not looking at two tokens.
>
>I agree.

I think now I agree too. Unless someone can come up with a better 
explanation, consider this fixed.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

To: [email-masked]
Xref: clerew local.mime:5043
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-resnick-2822upd-02 and Netnews
Date: Mon, 30 Jul 2007 11:32:23 GMT
Lines: 177
Sender: [email-masked]

In <p06250101c2d30b5454a1@[172.28.170.243]> Pete Resnick <[email-masked]> writes:

>I want to start out by saying that in answering this, I'm trying to 
>take RFC 2119, section 6 *very* seriously:

>6. Guidance in the use of these Imperatives

>    Imperatives of the type defined in this memo must be used with care
>    and sparingly.  In particular, they MUST only be used where it is
>    actually required for interoperation or to limit behavior which has
>    potential for causing harm (e.g., limiting retransmisssions)  For
>    example, they must not be used to try to impose a particular method
>    on implementors where the method is not required for
>    interoperability.

>Insofar as I interpret this:

>MUST means, "You must do this or seriously bad things will happen, 
>either with regard to interoperation with other implementers of this 
>specification or known harm will come to this or other parts of the 
>Internet."...

Yes, and I shall try to justify them on that basis.

>>However, I cannot conceive that any future IP-address format or 
>>other domain literal would ever contain a <quoted-pair> or any 
>>whitespace (that, together with '>', is the other thing Netnews 
>>cannot tolerate at this point) - there are just too many existing 
>>protocols that would promptly
>>fall over. So why not fix the problem once and for all now? And fix 
>>it for <domain-literal> at the same time - that and 
>><no-fold-literal> and the only places where <dtext> and <dcontent> 
>>are used, so it is easily done, and the obs-syntax would remain, of 
>>course.

>My main issue here is that we already have examples where limiting 
>the syntax in "obvious" ways in 2822 screwed up 2821 (e.g., 
>"Received:" syntax) and I'd prefer not to shoot myself in the foot 
>again. Do we know of implementations (especially non-netnews 
>implementations) which can't deal with quoted-pair in domain literals 
>or message ids, or won't strip whitespace out of domain literals? 
>Should this really come out?

It would probably be helpful for a start to state clearly that what is
actually written in a domain literal (whether in an address or a mesg-id)
MUST/SHOULD agree with the specification of some widely recognized
numbering/routeing scheme (which, at the moment means IP4 or IP6 of
course).

However, suppose some future addressing scheme _does_ admit quoted-pairs,
then accoring to the rest of RFC 2822 [xyz\qabc] would be semantically
equivalent to [xyzqabc]. In email that might cause threading algorithms to
break (e.g. if the Reference header contained a different form to the
original msg-id). But for Netnews it would be an utter disaster. So there
is an interoperability problem (mild with email, severe with news), but
the problem is simply avoided by forbidding quoted-pair in the first place
(or, alternatively, allowing '\' as a permitted character in dcontent, but
without the quoted-pair semantics). Removing SP and '>' from dcontent
would also simplify implemtations and avoid problems with naive
implementations.

>>2. SP after the ':' in headers
>>------------------------------
>>
>>That is REQUIRED in draft-ietf-usefor-usefor-12, and it is REQUIRED 
>>in the new NNTP standard RFC 3977. Since every known MUA already 
>>generates headers that way

>Please some support for this, including a review of the DRUMS 
>archive. If we're going to make this change, you do the research. I 
>seem to remember a discussion of this during DRUMS, and having the 
>space optional was where we ended up. Can we check?

There undoubtedly exists software within Netnews that will break if this
SP is absent, hence the reason we felt it necessary to REQUIRE that SP in
both USEFOR and NNTP.

So if you want email to interoperate with Netnews (and gateways from
mailing lists to News are quite common), then you indeed have an
interoperability problem sufficient to justify a "MUST". As for DRUMS, I
doubt this particular consideration was ever taken into account.

OTOH, since it is the invariable practice within email MUAs to include
this SP, putting such a MUST in email would incur no problems with the
installed base (of course, it would still be acceptable in the
obs-syntax). Whilst we may not be able to remove all discrepancies between
email and news, it would be desirable not to have any differences where
they can so easily be avoided.

>>There is one slight consequence that you should then forbid header 
>>lines with only whitespace after the header-name (because trailing 
>>whitespace has a habit of getting lost).

>And this gets us into a "changes with potential side-effects" problem 
>that worries me.

Yes, that is a bit messy, and so I would not push the point too hard.
Problems only arise with rogue agents that insist on discarding trailing
whitespace (which I regard as broken anyway).

>>3. CFWS between msg-ids in References
>>-------------------------------------
>>
>>It is currently [CFWS], but usage in Netnews has always placed 
>>whitespace there (and References has always been more of a News 
>>featrure than an Email feature). Would it hurt to REQUIRE it?

>See top of this message. Would it hurt to *not* have it as a MUST?

There undoubtedly exists Netnews software which will break without that
whitespace, hence the reason USEFOR RECOMMENDS it. So there is indeed an
interoperability problem, but to follow USEFOR a "SHOULD" would be
enough.  But the problem is nowhere as severe as the SP after ':' one.

Question. Does current typical email software routinely include that
whitespace? I had a quick look at a mailing list I subscribe to, and I did
not find any example where that whitespace was absent.

>>And also, please can we promote its use in Replies from SHOULD to 
>>MUST? It was SHOULD in RFC2822, but many MUAs have not taken the 
>>"hint" and, as a result, threading in mailing lists often gets 
>>broken, which is a confounded nuisance. It is invariably done 
>>properly within News.

There is also an interopability problem here, but is is not so clearcut as
the other cases. If this header is omitted, then threading algorithms just
do not work. Now you might argue that threading (an operation performed by
MUAs) is not a REQUIRED feature of the email protocol - just a smart
feature that some MUAs provide. However, even the smart MUAs can't do it
if the header is absent, and you will find it hard to convince the average
user that threading is not a 'normal' part of his mailing-list experience.

We agonized over this in USEFOR (and even had some wording at one stage
declaring threading to be an 'optional' feature of the protocol).
Eventually, what we wrote was:

   A "followup" is an article containing a response to the contents of
   an earlier article, its "precursor".  Every followup includes a
   "References" header field identifying that precursor (but note that
   non-followup articles may also use a References header field).

That wording is normative, so if the References is absent it ain't a
'followup'. So any user agent with a "followup" button which failed to
include a References could hardly claim compliance with the standard.

>>4. Allowed positions for <comment>s
>>-----------------------------------
>>
>>Allowing <comment>s in Message-ID header fields will break Netnews, 
>>and therefore the Usefor draft disallows them. Would it hurt to 
>>bring Email into line? For sure nobody ever uses them there.

>Again, potentially breaking some netnews implementations is not by 
>itself a reason to remove this IMO. Are there known non-netnews 
>implementations that break because of this? (And why are netnews 
>implementations so non-robust in the first place, unable to parse 
>comments and quoted-pairs?)

Yes there are undoubtedly news implementation that would break in that
situation, and the lack of that robustness is simply because of the severe
performance penalty in what is the critical inner-loop of relaying agents.

But I don't think the case for the same restriction in email is as strong
as some of the others I have raised, even though interopability problems
would arise.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 30 Jul 2007 09:56:29 -0600
From: Philip Guenther <[email-masked]>
To: [email-masked]
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Sender: [email-masked]

[Resending with my actual subscribed address so that this gets through in 
a timely fashion.  My apologies to all.  And yes, I've re-added the pine 
rule to do this automatically that I lost when my laptop died...]

On Mon, 30 Jul 2007, Paul Overell wrote:
> In message <[email-masked]>, Alan Barrett 
> <[email-masked]> writes
>> 
>> On Mon, 30 Jul 2007, Alan Barrett wrote:
>>> > It sure was. Comments and folding white space were allowed between
>>> > any two tokens in RFC 822. And since the syntax for zone was:
>>> >
>>> >                         ("+" / "-") 4DIGIT
>>> >
>>> > a comment was allowed between those two. I don't know how to read
>>> > 3.1.4 and 3.4.3 in RFC 822 and not come up with that interpretation.
>>> 
>>> Yes, you are correct.
>> 
>> Hang on...  "+" and "-" are not in the list of special characters that
>> RFC822 section 3.1.4 says are used to delimit tokens.
> 
> No, but quote marks are in the list of specials, "+" and "-" are 
> quoted-strings.

The quote marks around the plus and minus signs are part of the 822-BNF syntax 
and do not make those the quoted-strings referred to by RFC 822 section 3.1.4.

(If that isn't true, then someone needs to explain domain-literal, as its 
definition starts with "[", but it's described in section 3.1.4 as a lexical 
symbol distinct from quoted-string.)

>> So I would say
>> that, given input like "+0000", the sign and the digits together make
>> up an "atom" per the description in section 3.1.4 and the grammar in
>> section 3.3.  The ( ("+" / "-") 4DIGIT ) production in the grammar in
>> section 5.1 is looking inside a single token, not looking at two tokens.

I agree.

> Disagree, a quoted-string is a token

Please identify the quoted-strings in this example Date header field from the 
RFC:
 	Date:     26 Aug 76 1430 EDT

So, I disagree with Pete and Paul and instead agree with Alan that RFC 822 did 
not permit comments or folding whitespace between the sign and the number of a 
numeric zone.

Philip Guenther

Date: Mon, 30 Jul 2007 09:46:49 -0600
From: Philip Guenther <[email-masked]>
To: Paul Overell <[email-masked]>
cc: [email-masked]
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Sender: [email-masked]

On Mon, 30 Jul 2007, Paul Overell wrote:
> In message <[email-masked]>, Alan Barrett 
> <[email-masked]> writes
>> 
>> On Mon, 30 Jul 2007, Alan Barrett wrote:
>>> > It sure was. Comments and folding white space were allowed between
>>> > any two tokens in RFC 822. And since the syntax for zone was:
>>> >
>>> >                         ("+" / "-") 4DIGIT
>>> >
>>> > a comment was allowed between those two. I don't know how to read
>>> > 3.1.4 and 3.4.3 in RFC 822 and not come up with that interpretation.
>>> 
>>> Yes, you are correct.
>> 
>> Hang on...  "+" and "-" are not in the list of special characters that
>> RFC822 section 3.1.4 says are used to delimit tokens.
>
> No, but quote marks are in the list of specials, "+" and "-" are 
> quoted-strings.

The quote marks around the plus and minus signs are part of the 822-BNF 
syntax and do not make those the quoted-strings referred to by RFC 822 
section 3.1.4.

(If that isn't true, then someone needs to explain domain-literal, as its 
definition starts with "[", but it's described in section 3.1.4 as a 
lexical symbol distinct from quoted-string.)

>> So I would say
>> that, given input like "+0000", the sign and the digits together make
>> up an "atom" per the description in section 3.1.4 and the grammar in
>> section 3.3.  The ( ("+" / "-") 4DIGIT ) production in the grammar in
>> section 5.1 is looking inside a single token, not looking at two tokens.

I agree.

> Disagree, a quoted-string is a token

Please identify the quoted-strings in this example Date header field from 
the RFC:
 	Date:     26 Aug 76 1430 EDT

So, I disagree with Pete and Paul and instead agree with Alan that RFC 822 
did not permit comments or folding whitespace between the sign and the 
number of a numeric zone.

Philip Guenther

Date: Mon, 30 Jul 2007 02:08:37 -0700
To: Pete Resnick <[email-masked]>, Alan Barrett <[email-masked]>
From: SM <[email-masked]>
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Cc: ietf-822 mailing list <[email-masked]>
Sender: [email-masked]

At 20:25 29-07-2007, Pete Resnick wrote:
>How about this instead (which is a little closer to the text 
>throughout the rest of the document)?:
>
>                         ...it is RECOMMENDED that the right hand 
> side contain some domain
>                         identifier (either of the host itself, of 
> the implementer, or otherwise)
>                         such that the generator of the message 
> identifier can guarantee the
>                         uniqueness of the left hand side within the 
> scope of that domain.

That looks fine.  It covers the suggestion made by Alan Barrett.

Regards,
-sm

Date: Mon, 30 Jul 2007 09:46:22 +0100
To: [email-masked]
From: Paul Overell <[email-masked]>
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Sender: [email-masked]

In message <[email-masked]>, Alan Barrett 
<[email-masked]> writes
>
>On Mon, 30 Jul 2007, Alan Barrett wrote:
>> > It sure was. Comments and folding white space were allowed between
>> > any two tokens in RFC 822. And since the syntax for zone was:
>> >
>> >                         ("+" / "-") 4DIGIT
>> >
>> > a comment was allowed between those two. I don't know how to read
>> > 3.1.4 and 3.4.3 in RFC 822 and not come up with that interpretation.
>>
>> Yes, you are correct.
>
>Hang on...  "+" and "-" are not in the list of special characters that
>RFC822 section 3.1.4 says are used to delimit tokens.

No, but quote marks are in the list of specials, "+" and "-" are 
quoted-strings.

>  So I would say
>that, given input like "+0000", the sign and the digits together make
>up an "atom" per the description in section 3.1.4 and the grammar in
>section 3.3.  The ( ("+" / "-") 4DIGIT ) production in the grammar in
>section 5.1 is looking inside a single token, not looking at two tokens.

Disagree, a quoted-string is a token

RFC822 3.1.4:

>   These symbols are:
>
>                     -  individual special characters
>                     -  quoted-strings
>                     -  domain-literals
>                     -  comments
>                     -  atoms
>

So the "+", "-" and 4DIGIT are distinct tokens in RFC822, two quoted 
strings and an atom.

>
>For another example of grammar productions that look inside a single
>lexical token, consider RFC822's definitions of quoted-pair and
>quoted-string.  I hope nobody argues that the "\" and the CHAR in a
>quoted-pair are two tokens and that LWSP is allowed between them.
>

Agreed, because a quoted-string is a single token.

Regards

-- 
Paul Overell         Internet Platform Development Manager, Thus plc

Date: Mon, 30 Jul 2007 09:33:55 +0200
From: Alan Barrett <[email-masked]>
To: ietf-822 mailing list <[email-masked]>
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Sender: [email-masked]

On Mon, 30 Jul 2007, Alan Barrett wrote:
> > It sure was. Comments and folding white space were allowed between 
> > any two tokens in RFC 822. And since the syntax for zone was:
> > 
> >                         ("+" / "-") 4DIGIT
> > 
> > a comment was allowed between those two. I don't know how to read 
> > 3.1.4 and 3.4.3 in RFC 822 and not come up with that interpretation.
> 
> Yes, you are correct.

Hang on...  "+" and "-" are not in the list of special characters that
RFC822 section 3.1.4 says are used to delimit tokens.  So I would say
that, given input like "+0000", the sign and the digits together make
up an "atom" per the description in section 3.1.4 and the grammar in
section 3.3.  The ( ("+" / "-") 4DIGIT ) production in the grammar in
section 5.1 is looking inside a single token, not looking at two tokens.

For another example of grammar productions that look inside a single
lexical token, consider RFC822's definitions of quoted-pair and
quoted-string.  I hope nobody argues that the "\" and the CHAR in a
quoted-pair are two tokens and that LWSP is allowed between them.

--apb (Alan Barrett)

Date: Mon, 30 Jul 2007 08:58:07 +0200
From: Alan Barrett <[email-masked]>
To: Pete Resnick <[email-masked]>
Cc: ietf-822 mailing list <[email-masked]>
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Sender: [email-masked]

On Sun, 29 Jul 2007, Pete Resnick wrote:
> How about this instead (which is a little closer to the text
> throughout the rest of the document)?:
>
>           ...it is RECOMMENDED that the right hand side contain
>           some domain identifier (either of the host itself, of
>           the implementer, or otherwise) such that the generator
>           of the message identifier can guarantee the uniqueness
>           of the left hand side within the scope of that domain.
>
> We don't talk of "software" or "vendors" in this document.
> "Implementations" and "implementers" are about it.

Yes, that looks fine to me.

> >In section 4.3, why does obs-zone have [CFWS] between the sign and 
> >the digits of the timezone offset?  Such space was not permitted by 
> >earlier specifications.
> 
> It sure was. Comments and folding white space were allowed between 
> any two tokens in RFC 822. And since the syntax for zone was:
> 
>                         ("+" / "-") 4DIGIT
> 
> a comment was allowed between those two. I don't know how to read 
> 3.1.4 and 3.4.3 in RFC 822 and not come up with that interpretation.

Yes, you are correct.

Thank you.

--apb (Alan Barrett)

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: draft-resnick-2822upd-02 and Netnews
Organization: The Eyrie
Date: Sun, 29 Jul 2007 21:30:06 -0700
Sender: [email-masked]

Pete Resnick <[email-masked]> writes:

> Also, Charles alone does not rough consensus make. If there are others
> who strongly feel that these changes should be made (and strongly would
> be good; we're trying to push this along through the standards track,
> and changes at this point would be less than desirable), please speak up
> and indicate why the current specification does "the wrong thing".

I strongly agree with the message ID changes, and agree in general with
the rest (although not as strongly).  The space after colon may warrant a
SHOULD rather than a MUST, though, following your point about being very
serious about the RFC 2119 definitions.  (It's truly a MUST in netnews for
interoperability reasons -- headers without a space after a colon won't
work at all with most common implementations since RFC 1036 has always
required the space -- but that doesn't necessarily mean it is in e-mail as
well.)

> Again, potentially breaking some netnews implementations is not by
> itself a reason to remove this IMO. Are there known non-netnews
> implementations that break because of this? (And why are netnews
> implementations so non-robust in the first place, unable to parse
> comments and quoted-pairs?)

RFC 1036 doesn't mention comments in the References header, and while RFC
1036 references RFC 822 as well, when there's no real use of comments in
References and no netnews implementation has ever generated them, it's not
particularly suprising to me that few netnews implementations have ever
bothered to expect them and strip them.  Netnews has been using References
in a structured and protocol-significant fashion for a lot longer than
e-mail.

As for quoted-pairs in message IDs, message ID parsing is at the heart of
the critical path of a netnews server and a typical high-performance
server matches message IDs hundreds or thousands of times a second.
Simplification of this code path both has a leveraged effect on
performance since it's in the heart of the processing loop (not that this
sort of parsing is the performance-critical operation, but everything adds
up) and makes the core of a server more robust.  Properly parsing all the
bizarre bits allowed in historic mail message IDs is kind of nuts, and
since RFC 1036, netnews has always required a more restrictive syntax for
the field.  Message ID is far, far more important in netnews than it has
been historically in e-mail.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Sun, 29 Jul 2007 22:56:25 -0500
To: "Charles Lindsey" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: draft-resnick-2822upd-02 and Netnews
Cc: [email-masked]
Sender: [email-masked]

I want to start out by saying that in answering this, I'm trying to 
take RFC 2119, section 6 *very* seriously:

6. Guidance in the use of these Imperatives

    Imperatives of the type defined in this memo must be used with care
    and sparingly.  In particular, they MUST only be used where it is
    actually required for interoperation or to limit behavior which has
    potential for causing harm (e.g., limiting retransmisssions)  For
    example, they must not be used to try to impose a particular method
    on implementors where the method is not required for
    interoperability.

Insofar as I interpret this:

MUST means, "You must do this or seriously bad things will happen, 
either with regard to interoperation with other implementers of this 
specification or known harm will come to this or other parts of the 
Internet." SHOULD means, "You probably want to do this, and not doing 
so may cause some interoperability problems, but you can do this if 
you fully understand the ramifications and you have a good reason to 
do so."

If below I ask why something should be a MUST or a SHOULD, I'm 
looking for an answer in light of the above.

Also, Charles alone does not rough consensus make. If there are 
others who strongly feel that these changes should be made (and 
strongly would be good; we're trying to push this along through the 
standards track, and changes at this point would be less than 
desirable), please speak up and indicate why the current 
specification does "the wrong thing".

Preliminaries out of the way:

On 7/26/07 at 9:44 PM +0000, Charles Lindsey wrote:

>...a <quoted-pair> is still allowed within a <no-fold-literal>. 
>True, there is a mention that "other specifications" will limit it, 
>but no such specifications are mentioned.  Pointers to 
>draft-ietf-usefor-usefor-12 and to RFC2821-bis would be in order 
>here (cf such a mention in 3.4.1).

I'll do something similar to 3.4.1.

>However, I cannot conceive that any future IP-address format or 
>other domain literal would ever contain a <quoted-pair> or any 
>whitespace (that, together with '>', is the other thing Netnews 
>cannot tolerate at this point) - there are just too many existing 
>protocols that would promptly
>fall over. So why not fix the problem once and for all now? And fix 
>it for <domain-literal> at the same time - that and 
><no-fold-literal> and the only places where <dtext> and <dcontent> 
>are used, so it is easily done, and the obs-syntax would remain, of 
>course.

My main issue here is that we already have examples where limiting 
the syntax in "obvious" ways in 2822 screwed up 2821 (e.g., 
"Received:" syntax) and I'd prefer not to shoot myself in the foot 
again. Do we know of implementations (especially non-netnews 
implementations) which can't deal with quoted-pair in domain literals 
or message ids, or won't strip whitespace out of domain literals? 
Should this really come out?

>And finally, a couple of nits I have spotted:
>
>In 3.2.2, in the Note:, "no-fold-quote" no longer exists, and 
>"no-fold-literal" need not be mentioned because the mention of 
>"dcontent" covers it.

Fixed.

>In 3.6.4, near the bottom of page 27, you need to add quoted-strings 
>to the list of things now not allowed in msg-ids.

Fixed.

>2. SP after the ':' in headers
>------------------------------
>
>That is REQUIRED in draft-ietf-usefor-usefor-12, and it is REQUIRED 
>in the new NNTP standard RFC 3977. Since every known MUA already 
>generates headers that way

Please some support for this, including a review of the DRUMS 
archive. If we're going to make this change, you do the research. I 
seem to remember a discussion of this during DRUMS, and having the 
space optional was where we ended up. Can we check?

>There is one slight consequence that you should then forbid header 
>lines with only whitespace after the header-name (because trailing 
>whitespace has a habit of getting lost).

And this gets us into a "changes with potential side-effects" problem 
that worries me.

>3. CFWS between msg-ids in References
>-------------------------------------
>
>It is currently [CFWS], but usage in Netnews has always placed 
>whitespace there (and References has always been more of a News 
>featrure than an Email feature). Would it hurt to REQUIRE it?

See top of this message. Would it hurt to *not* have it as a MUST?

>And also, please can we promote its use in Replies from SHOULD to 
>MUST? It was SHOULD in RFC2822, but many MUAs have not taken the 
>"hint" and, as a result, threading in mailing lists often gets 
>broken, which is a confounded nuisance. It is invariably done 
>properly within News.

This is absolutely a violation of the 2119 definition in section 6. 
You must provide a better reason to make it a MUST.

>4. Allowed positions for <comment>s
>-----------------------------------
>
>Allowing <comment>s in Message-ID header fields will break Netnews, 
>and therefore the Usefor draft disallows them. Would it hurt to 
>bring Email into line? For sure nobody ever uses them there.

Again, potentially breaking some netnews implementations is not by 
itself a reason to remove this IMO. Are there known non-netnews 
implementations that break because of this? (And why are netnews 
implementations so non-robust in the first place, unable to parse 
comments and quoted-pairs?)

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sun, 29 Jul 2007 22:25:43 -0500
To: Alan Barrett <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Cc: ietf-822 mailing list <[email-masked]>
Sender: [email-masked]

On 7/26/07 at 11:01 AM +0200, Alan Barrett wrote:

>    It is RECOMMENDED that the right hand side contain a domain
>    identifier of an entity that guarantees the uniqueness of the left
>    hand side within the scope of that domain (for example, the domain
>    name of the host itself, or a domain name controlled by the vendor of
>    the software that generates the left hand side).

How about this instead (which is a little closer to the text 
throughout the rest of the document)?:

                         ...it is RECOMMENDED that the right hand side 
contain some domain
                         identifier (either of the host itself, of the 
implementer, or otherwise)
                         such that the generator of the message 
identifier can guarantee the
                         uniqueness of the left hand side within the 
scope of that domain.

We don't talk of "software" or "vendors" in this document. 
"Implementations" and "implementers" are about it.

>In section 4.3, why does obs-zone have [CFWS] between the sign and 
>the digits of the timezone offset?  Such space was not permitted by 
>earlier specifications.

It sure was. Comments and folding white space were allowed between 
any two tokens in RFC 822. And since the syntax for zone was:

                         ("+" / "-") 4DIGIT

a comment was allowed between those two. I don't know how to read 
3.1.4 and 3.4.3 in RFC 822 and not come up with that interpretation.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated

Date: Sat, 28 Jul 2007 15:43:19 +0200
From: Alan Barrett <[email-masked]>
To: [email-masked]
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Sender: [email-masked]

On Fri, 27 Jul 2007, Charles Lindsey wrote:
> Essentially, such vendors have to rely on generating a sufficiently
> long random number for the id-left to make it astronomically
> improbable that a clash will occur.

Right.  And, if a clash does occur, it's under a domain controlled by
the vendor, so we know who to blame, unlike <${random}@localhost> or
<${random}@${one_word_host_name}>.  The point is that the owner of the
id-right domain takes responsibility for ensuring uniqueness of the
id-left within the scope of that domain.

> We mght have some discussion as to whether we regard
> that as a good method worthy of our blessing.

I am not proposing to change the requirements, only to add an example
and change the wording.  The existing text already says that the
generator must guarantee uniqueness of the left hand side within the
scope of the right hand side domain, and already says that the right
hand side domain does not have to belong to the host itself.  Using
a vendor's domain would fit all these requirements, but I have seen
discussions in various places over the years that suggests that vendors
are not aware that they are allowed to do this.

--apb (Alan Barrett)

To: [email-masked]
Xref: clerew local.mime:5038
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Date: Fri, 27 Jul 2007 13:20:04 GMT
Lines: 28
Sender: [email-masked]

In <[email-masked]> Alan Barrett <[email-masked]> writes:

>The intent behind my suggested wording is to make it more clear to
>software vendors that the domain name on the right hand side of the
>message-id is allowed to be the vendor's domain name; that it doesn't
>have to be the domain name of the host that's running the software, and
>that inability to know the domain name of the host is not necessarily a
>problem.

There is indeed one well known vendor which does what you say, but I am not
sure whether it is a practice we want to encourage (that is not to say
that vendor is likely to be swayed by such non-encouragement).

Essentially, such vendors have to rely on generating a sufficiently long
random number for the id-left to make it astronomically improbable that a
clash will occur. We mght have some discussion as to whether we regard
that as a good method worthy of our blessing.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Thu, 26 Jul 2007 22:03:58 -0700
To: ietf-822 mailing list <[email-masked]>
From: SM <[email-masked]>
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Sender: [email-masked]

Hi Alan,
At 13:48 26-07-2007, Alan Barrett wrote:
>It has happened in the past that software vendors have created
>message-id algorithms that did not guarantee unique message-ids in the
>face of non-unique host names.  The common wisdom has been that it's
>better to generate message-id's of the form <${something derived from
>a random number, a timestamp, and the non-unique hostname}@${something
>that encodes the software version}.${vendor's domain name}> than to
>generate message-id's of the form <${anyhing}@${non-unique hostname}>.

Having the hostname on the left or on the right in terms of 
uniqueness would not be much of a difference as you face the same 
situation when doing a binary comparison of two message-ids.  The use 
of the domain name/IP address on the right is based on the assumption 
that it guarantees that the message-id is globally unique.  That 
doesn't always hold nowadays with NAT.

>The intent behind my suggested wording is to make it more clear to
>software vendors that the domain name on the right hand side of the
>message-id is allowed to be the vendor's domain name; that it doesn't
>have to be the domain name of the host that's running the software, and
>that inability to know the domain name of the host is not necessarily a
>problem.

The rationale of having the domain name on the right is that it 
guarantees uniqueness of the left-hand side within the scope of that 
domain.  If there is a recommendation to use the vendor's domain on 
the right-hand side, it lessens the scope for uniqueness within that 
domain.  If the domain name of the host cannot be determine, an IP 
address could be used.  Your message-id example is good example for 
generating a unique identifier.  You could have the software version 
and vendor's domain name on the left though.

Regards,
-sm 

To: [email-masked]
Xref: clerew local.mime:5034
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: draft-resnick-2822upd-02 and Netnews
Date: Thu, 26 Jul 2007 21:44:53 GMT
Lines: 88
Sender: [email-masked]

Sorry, I should have made these comments before, but recent threats of
Last Calls have brought it rapidly to the top of my 'todo' list :-( .

The principlal change in this latest draft is the removal of
<quoted-string> from the <id-left>. This is much to be welcomed, but it
still does not address several other Netnews incompatibility problems that
I raised in my original message entitled "Compatibility with Netnews"
which I posted on April 27th.

I also have some niggles with the latest draft not related to Netnews, but
I shall save those for a separate thread. So here are the issues that
remain vis a vis Netnews.

1. Message-ID
-------------

The latest change is a huge improvement, but some niggles still remain.

Netnews has a requirement to be able to compare two msg-ids by a simple
octer-by-octet comparison (which also implies case sensitivity). That
surely is a desirable aim for Email also, and indeed it would be useful to
mention that property in the paragraph of section 3.6.4 that starts with
"The message identifier (msg-id) MUST ...".  But we are not quite there
yet, because a <quoted-pair> is still allowed within a <no-fold-literal>.
True, there is a mention that "other specifications" will limit it, but no
such specifications are mentioned.  Pointers to
draft-ietf-usefor-usefor-12 and to RFC2821-bis would be in order here (cf
such a mention in 3.4.1).

However, I cannot conceive that any future IP-address format or other
domain literal would ever contain a <quoted-pair> or any whitespace (that,
together with '>', is the other thing Netnews cannot tolerate at this
point) - there are just too many existing protocols that would promptly
fall over. So why not fix the problem once and for all now? And fix it for
<domain-literal> at the same time - that and <no-fold-literal> and the
only places where <dtext> and <dcontent> are used, so it is easily done,
and the obs-syntax would remain, of course.

And finally, a couple of nits I have spotted:

In 3.2.2, in the Note:, "no-fold-quote" no longer exists, and
"no-fold-literal" need not be mentioned because the mention of "dcontent"
covers it.

In 3.6.4, near the bottom of page 27, you need to add quoted-strings to
the list of things now not allowed in msg-ids.

2. SP after the ':' in headers
------------------------------

That is REQUIRED in draft-ietf-usefor-usefor-12, and it is REQUIRED in the
new NNTP standard RFC 3977. Since every known MUA already generates
headers that way, it would cost nothing to require it here also, and of
course the obs-syntax would still allow its omission to be accepted.

There is one slight consequence that you should then forbid header lines
with only whitespace after the header-name (because trailing whitespace
has a habit of getting lost).

3. CFWS between msg-ids in References
-------------------------------------

It is currently [CFWS], but usage in Netnews has always placed whitespace
there (and References has always been more of a News featrure than an
Email feature). Would it hurt to REQUIRE it?

And also, please can we promote its use in Replies from SHOULD to MUST? It
was SHOULD in RFC2822, but many MUAs have not taken the "hint" and, as a
result, threading in mailing lists often gets broken, which is a
confounded nuisance. It is invariably done properly within News.

4. Allowed positions for <comment>s
-----------------------------------

Allowing <comment>s in Message-ID header fields will break Netnews, and
therefore the Usefor draft disallows them. Would it hurt to bring Email
into line? For sure nobody ever uses them there.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Thu, 26 Jul 2007 22:48:13 +0200
From: Alan Barrett <[email-masked]>
To: ietf-822 mailing list <[email-masked]>
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Sender: [email-masked]

On Thu, 26 Jul 2007, SM wrote:
> >In section 3.6.4, I suggest that we mention the idea that a software vendor
> >can use their domain name when generating message IDs.  For example, change
> >this:
> 
> I don't see a reason for this change.  Could you please elaborate on 
> why you are recommending this?

It has happened in the past that software vendors have created
message-id algorithms that did not guarantee unique message-ids in the
face of non-unique host names.  The common wisdom has been that it's
better to generate message-id's of the form <${something derived from
a random number, a timestamp, and the non-unique hostname}@${something
that encodes the software version}.${vendor's domain name}> than to
generate message-id's of the form <${anyhing}@${non-unique hostname}>.

The intent behind my suggested wording is to make it more clear to
software vendors that the domain name on the right hand side of the
message-id is allowed to be the vendor's domain name; that it doesn't
have to be the domain name of the host that's running the software, and
that inability to know the domain name of the host is not necessarily a
problem.

--apb (Alan Barrett)

Date: Thu, 26 Jul 2007 10:14:20 -0700
To: Alan Barrett <[email-masked]>, ietf-822 mailing list <[email-masked]>
From: SM <[email-masked]>
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Sender: [email-masked]

Hi Alan,
At 02:01 26-07-2007, Alan Barrett wrote:
>In section 3.6.4, I suggest that we mention the idea that a software vendor
>can use their domain name when generating message IDs.  For example, change
>this:

I don't see a reason for this change.  Could you please elaborate on 
why you are recommending this?

Regards,
-sm 

Date: Thu, 26 Jul 2007 11:01:29 +0200
From: Alan Barrett <[email-masked]>
To: ietf-822 mailing list <[email-masked]>
Subject: Re: Mailing List Last Call for 2822 update internet-draft
Sender: [email-masked]

On Thu, 26 Jul 2007, Tony Hansen wrote:
> This is a "formal" Mailing List Last Call on
> draft-resnick-2822upd-02.txt. The last call will last for two weeks
> time, ending on August 10, 2007.

In section 3.6.4, I suggest that we mention the idea that a software vendor
can use their domain name when generating message IDs.  For example, change
this:

"  Though other algorithms will work, it is RECOMMENDED that the right
"  hand side contain some domain identifier (either of the host itself
"  or otherwise) such that the generator of the message identifier can
"  guarantee the uniqueness of the left hand side within the scope of
"  that domain.

to this:

   It is RECOMMENDED that the right hand side contain a domain
   identifier of an entity that guarantees the uniqueness of the left
   hand side within the scope of that domain (for example, the domain
   name of the host itself, or a domain name controlled by the vendor of
   the software that generates the left hand side).

In section 4.3, why does obs-zone have [CFWS] between the sign and the
digits of the timezone offset?  Such space was not permitted by earlier
specifications.

--apb (Alan Barrett)

Date: Thu, 26 Jul 2007 00:22:28 -0400
From: Tony Hansen <[email-masked]>
To: ietf-822 mailing list <[email-masked]>
CC: SMTP Interest Group <[email-masked]>, LEMONADE WG <[email-masked]>, IMAP extensions mailing list <[email-masked]>, EAI WG <[email-masked]>, POP3 extensions mailing list <[email-masked]>, USEFOR WG <[email-masked]>, SIMPLE WG <[email-masked]>
Subject: Mailing List Last Call for 2822 update internet-draft
Sender: [email-masked]

There have been some comments on draft-resnick-2822upd-* but they have
dwindled down to none.

This is a "formal" Mailing List Last Call on
draft-resnick-2822upd-02.txt. The last call will last for two weeks
time, ending on August 10, 2007.

The document will be discussed on the 822 mailing list,
<[email-masked]>. Please send your comments there.

For a copy of the current draft, you can find it at

	http://www.ietf.org/internet-drafts/draft-resnick-2822upd-02.txt
or
	http://tools.ietf.org/html/draft-resnick-2822upd

Depending on the outcome of the Mailing List Last Call, the next step
will be to submit the document (or its revision) to the IESG for
IETF-wide Last Call.

Thanks!

	Tony Hansen
	[email-masked]