Subject: Fwd: I-D ACTION:draft-klyne-msghdr-registry-00.txt
From: Graham Klyne <[email-masked]>
To: [email-masked]
Date: Fri, 28 Sep 2001 16:34:46 +0100
List-Id: <ietf-822.imc.org>

My first stab at a header registry proposal is available...

>A New Internet-Draft is available from the on-line Internet-Drafts 
>directories.
>
>
>         Title           : Registration procedures for message headers
>         Author(s)       : G. Klyne
>         Filename        : draft-klyne-msghdr-registry-00.txt
>         Pages           : 11
>         Date            : 27-Sep-01
>
>This specification defines registration procedures for the message
>headers used by Internet mail, newsgroup feeds, HTTP and other
>Internet applications.
>
>A URL for this Internet-Draft is:
>http://www.ietf.org/internet-drafts/draft-klyne-msghdr-registry-00.txt

#g

------------------------------------------------------------
Graham Klyne                    Baltimore Technologies
Strategic Research              Content Security Group
<[email-masked]>    <http://www.mimesweeper.com>
                                 <http://www.baltimore.com>
------------------------------------------------------------

From: Keith Moore <[email-masked]>
To: Jacob Palme <[email-masked]>
cc: [email-masked]
Subject: Re: Locating RFC [2]822 headers 
Date: Wed, 26 Sep 2001 13:38:43 -0400
Sender: [email-masked]

> Are you really sure of this? English is a rich language.
> See for example the following header field names:
> Obsoletes/Replaces/Supersedes, which all are variants
> of the same thing.

right, but the meanings are subtly different.

and then there are fields like Priority which have conflicting 
implementations, and which get confused with Importance and Precedence.

I suppose the main thing I'd like to discourage in the future is the
casual use of simple English words for field names, because such field
names appear legitimate (when they're not) and seem to obvious meanings
(when they don't).

Keith

Date: Wed, 26 Sep 2001 16:25:52 +0200
To: [email-masked]
From: Jacob Palme <[email-masked]>
Subject: Re: Locating RFC [2]822 headers
Sender: [email-masked]

At 09.00 -0400 01-09-26, Keith Moore wrote:
>yes, but this really does cause a problem, as the set of
>"obvious" field names for some purpose is rather small, and
>there is a tendency for the "obvious" field names to get
>usurped by naive implementations.

Are you really sure of this? English is a rich language.
See for example the following header field names:
Obsoletes/Replaces/Supersedes, which all are variants
of the same thing.
-- 
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/

From: Keith Moore <[email-masked]>
To: Jacob Palme <[email-masked]>
cc: [email-masked]
Subject: Re: Locating RFC [2]822 headers 
Date: Wed, 26 Sep 2001 09:00:15 -0400
Sender: [email-masked]

> The existence of a header in such a registry need not mean
> that this header is "legal" other than when the registry
> refers to a standard for the definition of the header.
> The existence could just be interpreted that you should
> not re-use an existing header name for a new purpose.

yes, but this really does cause a problem, as the set of
"obvious" field names for some purpose is rather small, and
there is a tendency for the "obvious" field names to get
usurped by naive implementations.

Date: Wed, 26 Sep 2001 11:17:14 +0100
To: [email-masked] (Charles Lindsey)
From: Graham Klyne <[email-masked]>
Subject: Re: Locating RFC [2]822 headers
Cc: [email-masked]
Sender: [email-masked]

At 02:06 PM 9/25/01 +0000, Charles Lindsey wrote:
> >I agree with the need to publically discuss and vett new fields before they
> >are widely implemented.  I'm not sure about the idea of accepting 
> registrations
> >in advance of approval, and having them lapse if not approved.
>
>Well there are precedents for IANA to do that sort of thing. But it is a
>matter that can best be discussed when someone has placed a draft in front of
>us. I believe the man who originally set this going has some plan to do
>that.

Yup... watch this space:  I've got something nearly ready to ship.

Thanks everyone for your thoughts.

#g

------------------------------------------------------------
Graham Klyne                    Baltimore Technologies
Strategic Research              Content Security Group
<[email-masked]>    <http://www.mimesweeper.com>
                                 <http://www.baltimore.com>
------------------------------------------------------------

Date: Wed, 26 Sep 2001 11:18:15 +0100
To: Keith Moore <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: Locating RFC [2]822 headers 
Cc: [email-masked]
Sender: [email-masked]

At 11:48 PM 9/25/01 -0400, Keith Moore wrote:
> > Well there are precedents for IANA to do that sort of thing. But it is a
> > matter that can best be discussed when someone has placed a draft in 
> front of
> > us.
>
>draft beers?   one in front of each of us?  an excellent idea.

If it wasn't my round, I'd say that's a grand scheme!

#g

------------------------------------------------------------
Graham Klyne                    Baltimore Technologies
Strategic Research              Content Security Group
<[email-masked]>    <http://www.mimesweeper.com>
                                 <http://www.baltimore.com>
------------------------------------------------------------

Date: Wed, 26 Sep 2001 08:50:50 +0200
To: [email-masked]
From: Jacob Palme <[email-masked]>
Subject: Re: Locating RFC [2]822 headers
Sender: [email-masked]

At 16.52 -0700 01-09-19, Dave Crocker wrote:
>At 10:13 AM 9/19/2001, Keith Moore wrote:
>>the danger in creating such a registry is that it will be used
>>to legitimize certain bad practices that have been implemented with
>>little thought.
>
>That is an inherent danger in any open registration 
>environment.  It has not proved  problem in any other IANA 
>registration activities, as far as I am aware.  If there 
>is strong enough concern, of course, we can direct IANA to 
>permit only those registrations that are standards track.

The existence of a header in such a registry need not mean
that this header is "legal" other than when the registry
refers to a standard for the definition of the header.
The existence could just be interpreted that you should
not re-use an existing header name for a new purpose.

This could be specified in the rules for the registry.
-- 
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/

Date: Wed, 26 Sep 2001 08:51:06 +0200
To: [email-masked]
From: Jacob Palme <[email-masked]>
Subject: Re: questions on RFC2822
Sender: [email-masked]

At 13.17 -0500 01-09-10, Eric A. Hall wrote:
>Out of curiousity, has anybody ever tested the maximum length conditions?
>IE, what happens when a field name is 997 bytes, and is followed by a
>field body which is 998 bytes of continuous data? This appears to be
>technically valid based on my reading, although it would probably be
>self-defeating for the sender.

Reading your letter, one wonders why you are asking these
questions. Possible reasons:

(a) You are asking whether you should to handle incoming
     mail which have, for example, 997 or 1025 bytes long header
     fields.

(b) You are wondering whether you should modify incoming mail
     which has illegal headers.

(c) You are asking whether you should intentionally produce such
     long headers.

(d) You are wondering whether you should, in an API within the
     mailer you are working with, refuse requests from another
     module in the mailer which exceeds the limits.

The answer would depend on which of these cases apply.

(a) Answer is yes according to Internet Golden Rule 1.

(b) Answer is no according to Internet golden Rule 3.

(c) Answer is no according to Internet Golden Rule 1
     and Golden Rule 2.

(d) Answer may be yes for data generated by a UA, no for
     data coming from another mailer. A conclusion of this
     is that you should definitely not have precompiled
     maximum lengthes of such strings, they should all
     be variable length with no explicit upper length.

The Internet Golden Rules:

1.  Be liberal in what you accept, be conservative
     in what you produce.

2.  Do not produce data which is likely to harm some
     receiving agents, even if this is legal and
     serves a useful purpose for you.

3.  Do not munge incoming data from other agents,
     except in cases where your agent and the incoming
     agent can be seen as a system co-operating in
     letting your agent do certain things.
-- 
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/

From: Keith Moore <[email-masked]>
To: [email-masked] (Charles Lindsey)
cc: [email-masked]
Subject: Re: Locating RFC [2]822 headers 
Date: Tue, 25 Sep 2001 23:48:37 -0400
Sender: [email-masked]

> Well there are precedents for IANA to do that sort of thing. But it is a
> matter that can best be discussed when someone has placed a draft in front of
> us. 

draft beers?   one in front of each of us?  an excellent idea.

> I believe the man who originally set this going has some plan to do
> that.

sounds good. 

Keith

To: [email-masked]
Xref: clerew local.mime:759
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Locating RFC [2]822 headers
Date: Tue, 25 Sep 2001 14:06:56 GMT
Lines: 21
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>I agree with the need to publically discuss and vett new fields before they
>are widely implemented.  I'm not sure about the idea of accepting registrations 
>in advance of approval, and having them lapse if not approved.

Well there are precedents for IANA to do that sort of thing. But it is a
matter that can best be discussed when someone has placed a draft in front of
us. I believe the man who originally set this going has some plan to do
that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Keith Moore <[email-masked]>
To: [email-masked] (Charles Lindsey)
cc: [email-masked]
Subject: Re: Locating RFC [2]822 headers 
Date: Mon, 24 Sep 2001 13:19:40 -0400
Sender: [email-masked]

> >> Sure, you don't register a header unless there is paper somewhere that
> >> describes the proper way to use it. That could always include an internet
> >> draft in the case of a provisional registration.
> 
> >We need more than documentation to use headers widely, we need good design.
> >And experience suggests that fields designed by individual implementors
> >aren't suitable for widespread use.
> 
> Indeed so, which is why a mechanism that forces at least an internet draft
> (which automatically provides a forum for discussion) and that causes the
> registration to lapse if the draft is not pursued to RFC status would
> provide some degree of control.

I agree with the need to publically discuss and vett new fields before they
are widely implemented.  I'm not sure about the idea of accepting registrations 
in advance of approval, and having them lapse if not approved.

Keith

To: [email-masked]
Xref: clerew local.mime:757
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Locating RFC [2]822 headers
Date: Mon, 24 Sep 2001 09:21:44 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:
>> 
>> Sure, you don't register a header unless there is paper somewhere that
>> describes the proper way to use it. That could always include an internet
>> draft in the case of a provisional registration.

>We need more than documentation to use headers widely, we need good design. 
>And experience suggests that fields designed by individual implementors
>aren't suitable for widespread use.

Indeed so, which is why a mechanism that forces at least an internet draft
(which automatically provides a forum for discussion) and that causes the
registration to lapse if the draft is not pursued to RFC status would
provide some degree of control.

Contrast that with the present situation where headers just appear, get
implemented in lots of places, and it is then too late to change them. A
good example is the Netnews Mail-Copies-To header. I am not sure where it
came from, but it is now widely recognised and acted upon. So much so that
we have included it in the Usefor Draft, even though it is badly named and
comes with two sets of keywords to do the same thing (though we have now
deprecated one of those).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: 24 Sep 2001 01:03:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
Subject: Re: Locating RFC [2]822 headers
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked] (Keith Moore)  wrote on 21.09.01 in <[email-masked]>:

> > >Problem is, absent good design review and an agreed-on specification,
> > >reuse of existing fields can cause interoperability problems.
> > >Precedence, Priority, and Return-Receipt-To are all examples.
> >
> > Sure, you don't register a header unless there is paper somewhere that
> > describes the proper way to use it. That could always include an internet
> > draft in the case of a provisional registration.
>
> We need more than documentation to use headers widely, we need good design.
> And experience suggests that fields designed by individual implementors
> aren't suitable for widespread use.

How about a registry that says:

1. In general: this is just a registry, it neither endorses nor ... bla  
bla bla ...

1a. If you want advice about headers, these IETF mailing lists are  
probaply appropriate: ...

1b. If you want to register a new header, or change an existing  
registration, here is a form, fill in the blanks and send it to ...

1c. You are strongly encouraged to write at least an informational RFC  
about any header you register (much better than pointing to any other  
documentation); advice on how to do that is at ... (and if you're  
considering a *new* header, see 1a!)

2. For every entry, has (at least) the following fields:

a. Standardization status (standard, proposed standard, experimental,  
proprietary, cooperating-subnet, internal-use (or however we want to call  
various non-RFC headers), ...)

b. List of RFCs (and possibly currently active drafts) that talk about  
this header (even - or especially - those that say "bad idea")

c. List of relevant non-RFC material that talks about this header  
(preferrably only if it is not defined by some RFC) - and yes, I do mean  
explicitely a different field

d. Contact addresses, if relevant (mainly for non-standards track headers)

That is, the registry is essentially open to all, but *also* open to (RFC  
based) comments at any time. If you want to argue that a specific header  
(or set of headers) is a bad idea, you can always write a BCP or  
Informational RFC and have it referenced in the entry you want to say  
something about. (Or, of course, a standards track RFC, if that's  
warranted.)

Note that I put standardization status first, list of RFCs next, and only  
after that any non-RFC  references. That was intentional.

MfG Kai

Date: 21 Sep 2001 15:43:56 -0700
From: Tim Showalter <[email-masked]>
To: [email-masked]
Subject: decoding nested encodings
Sender: [email-masked]

We've seen cases recently of Quoted-Printible being used to encode
message/rfc822 by contemporary user agents, usually with forwarded
messages, such that the inner message ends up with a header containing a
string like boundary=3D"marker", making decoding difficult for MIME
decoders.  Does anyone know how widespread this is?

Tim

From: Keith Moore <[email-masked]>
To: [email-masked] (Charles Lindsey)
cc: [email-masked]
Subject: Re: Locating RFC [2]822 headers 
Date: Fri, 21 Sep 2001 17:57:46 -0400
Sender: [email-masked]

> >Problem is, absent good design review and an agreed-on specification,
> >reuse of existing fields can cause interoperability problems.
> >Precedence, Priority, and Return-Receipt-To are all examples.
> 
> Sure, you don't register a header unless there is paper somewhere that
> describes the proper way to use it. That could always include an internet
> draft in the case of a provisional registration.

We need more than documentation to use headers widely, we need good design. 
And experience suggests that fields designed by individual implementors
aren't suitable for widespread use.

Keith

To: [email-masked]
Xref: clerew local.mime:752
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Locating RFC [2]822 headers
Date: Fri, 21 Sep 2001 11:23:08 GMT
Lines: 24
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>Problem is, absent good design review and an agreed-on specification, 
>reuse of existing fields can cause interoperability problems.  
>Precedence, Priority, and Return-Receipt-To are all examples.

Sure, you don't register a header unless there is paper somewhere that
describes the proper way to use it. That could always include an internet
draft in the case of a provisional registration.

And there should be an agreed format for what IANA is to be asked to keep,
with references to protcols, RFCs (or drafts), and an agreed short
description.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:753
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Locating RFC [2]822 headers
Date: Fri, 21 Sep 2001 11:24:25 GMT
Lines: 20
Sender: [email-masked]

In <[email-masked]> Graham Klyne <[email-masked]> writes:

>(My original query here was a precursor to possibly undertaking such a 
>task.  The feedback here has been useful.  I'm not completely decided, but 
>I'm inclined towards composing a simple registry draft and testing the 
>waters further.)

I think your next move should be to have a word with IANA, to see how they
would like it done.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Keith Moore <[email-masked]>
To: "D. J. Bernstein" <[email-masked]>
cc: [email-masked]
Subject: Re: Locating RFC [2]822 headers 
Date: Thu, 20 Sep 2001 15:59:02 -0400
Sender: [email-masked]

> Contributors can
> focus on the needs of their users, without artificially separating IETF
> fields from non-IETF fields, SMTP from NNTP, etc.

heaven forbid that contributors should distinguish one protocol from another.

Keith

Date: 20 Sep 2001 19:21:28 -0000
Mail-Followup-To: [email-masked]
Automatic-Legal-Notices: Copyright 2001, D. J. Bernstein. My transmission of this message to you does not constitute a copyright waiver or any other limitation of my rights, even if you have told me otherwise.
From: "D. J. Bernstein" <[email-masked]>
To: [email-masked]
Subject: Re: Locating RFC [2]822 headers
Sender: [email-masked]

Lawrence Greenfield writes:
> I would much prefer an open registry, since it is the best way to
> avoid name conflicts and show what's out there.  A registry doesn't
> bless a particular entry, it just prevents accidental collision.

My header-format reference manual includes an open list:

   http://cr.yp.to/immhf/index.html

I also have a mailing list for Internet mail implementors. To subscribe,
send an empty message to [email-masked]. Contributors can
focus on the needs of their users, without artificially separating IETF
fields from non-IETF fields, SMTP from NNTP, etc.

---Dan

Date: Thu, 20 Sep 2001 18:11:45 +0100
To: [email-masked] (Charles Lindsey)
From: Graham Klyne <[email-masked]>
Subject: Re: Locating RFC [2]822 headers
Cc: [email-masked]
Sender: [email-masked]

At 10:03 AM 9/20/01 +0000, Charles Lindsey wrote:
>Please can someone enlighten us as to the mechanisms necessary to
>establish a new IANA registry? Does one write an RFC
>instructing/authorising them to do it, or is it done just by some edict of
>IESG? If an RFC is needed, then I suspect this List is a good place to
>prepare the draft, but of course we need to bring the News and HTTP people
>on board (plus some input from IANA, of course).

My understanding is this:  basically, write an I-D, and see if consensus 
can be found progress it to RFC/BCP publication.

(My original query here was a precursor to possibly undertaking such a 
task.  The feedback here has been useful.  I'm not completely decided, but 
I'm inclined towards composing a simple registry draft and testing the 
waters further.)

#g

------------------------------------------------------------
Graham Klyne                    Baltimore Technologies
Strategic Research              Content Security Group
<[email-masked]>    <http://www.mimesweeper.com>
                                 <http://www.baltimore.com>
------------------------------------------------------------

To: [email-masked]
Xref: clerew local.mime:744
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Locating RFC [2]822 headers
Date: Thu, 20 Sep 2001 10:03:00 GMT
Lines: 73
Sender: [email-masked]

In <[email-masked]> Graham Klyne <[email-masked]> writes:

>The usual IETF way to deal with these situations is to have an IANA 
>registry of values.  But none exists for message headers (I can find no 
>reference to RFC822 or email messages in 
>http://www.iana.org/numbers.htm).  The nearest to a registry we have seems 
>to be Jacob Palme's RFC 2076 and more recent Internet Draft updates 
>(currently draft-palme-mailext-headers-05.txt).

Yes, I agree that this is a task for IANA, but not tied irrevocably to RFC
2076, as others have said.

This issue has already been discussed on the USEFOR Working Group, which
could see much benefit in such a scheme. Essentially, there are three sets
of protocols which use headers in this style:

Mail (RFC 2822, plus many extensions for MIME and much else)
Netnews (RFC1036, or usefor in due course)
HTPP (RFC 2616)

There are many headers which are used in two, or even all three, of those
protocols with essentially the same meaning (or with small but carefully
documented differences).

Therefore, what we need is an IANA Registry for header names which would
record which protocol(s) used it, and the RFC(s) where its usage was
defined (that might be up to 3 RFCs if all of Mail, Netnews and HTTP were
involved, and even more if a header was defined in one RFC, but had some
extended usage added in another). Here I am speaking of both
standards-track and experimental protocol RFCs (since the prime objective
is to prevent two independent groups trying to define the same header).

But I would go further than that. I would also allow IANA to issue
provisional registrations for proposed new headers. There would have to be
stringent conditions for these, such as the existence of an internet draft
defining the proposed new header (thus provisional registrations would
lapse after 6 months in the same way as internet drafts). I understand
that there are already precedents for IANA to issue provisional
registrations (such as for Port Numbers, I believe).

The advantage of that scheme is that someone proposing a new header with
the intent that it would eventually see widespread use can get it
registered and immediately start using it on an experimental basis.
Currently, all he can do is to use an X-Header; then people start using
the X-Header and its name gets built into various pieces of software; then
someone decides to make it official in an RFC; and then they have to
change it to something different without an X-, and all the working
software has to change again :-( . We are currently going through that
process as regards the commands in the NNTP protocol.  That is the
particular aspect of the matter which has been duscussed on USEFOR, and is
why I believe USEFOR would support such a move.

But of course, whoever proposes such a header has to show evidence of his
good intentions by at least producing an internet draft (which should be
enough to deter all but the most determined of the pranksters).

Please can someone enlighten us as to the mechanisms necessary to
establish a new IANA registry? Does one write an RFC
instructing/authorising them to do it, or is it done just by some edict of
IESG? If an RFC is needed, then I suspect this List is a good place to
prepare the draft, but of course we need to bring the News and HTTP people
on board (plus some input from IANA, of course).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Thu, 20 Sep 2001 08:31:07 -0700 (PDT)
From: [email-masked]
Subject: Re: Locating RFC [2]822 headers
To: Keith Moore <[email-masked]>
Cc: Graham Klyne <[email-masked]>, [email-masked], [email-masked], Jacob Palme <[email-masked]>
Sender: [email-masked]

> > - to have a way to find if there is an existing header that serves some
> > given purpose.

> Problem is, absent good design review and an agreed-on specification,
> reuse of existing fields can cause interoperability problems.
> Precedence, Priority, and Return-Receipt-To are all examples.

And X-Sender (or, if you prefer, X-X-Sender). And X-Priority. And
X-Sun-Charset, X-Sun-Content-Encoding, X-Sun-Content-Length,
X-Sun-Content-lines, etc. Even Encoding, which has an experimental
specification but no generally agreed-upon use, is interpreted differently by
different agents.

Do we really want to get into this?

				Ned

Date: Thu, 20 Sep 2001 15:37:02 +0100
To: Keith Moore <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: Locating RFC [2]822 headers 
Cc: [email-masked]
Sender: [email-masked]

At 10:25 AM 9/20/01 -0400, Keith Moore wrote:
> > - to have a way to find if there is an existing header that serves some
> > given purpose.
>
>Problem is, absent good design review and an agreed-on specification,
>reuse of existing fields can cause interoperability problems.
>Precedence, Priority, and Return-Receipt-To are all examples.

I agree... my own interest has been for headers defined in standards-track 
or putative standards-track specifications.  (Although I acknowledge that 
there is also interest in a less strictly controlled collection of defined 
headers.)

#g

------------------------------------------------------------
Graham Klyne                    Baltimore Technologies
Strategic Research              Content Security Group
<[email-masked]>    <http://www.mimesweeper.com>
                                 <http://www.baltimore.com>
------------------------------------------------------------

From: Keith Moore <[email-masked]>
To: Graham Klyne <[email-masked]>
cc: [email-masked], [email-masked], Jacob Palme <[email-masked]>
Subject: Re: Locating RFC [2]822 headers 
Date: Thu, 20 Sep 2001 10:25:16 -0400
Sender: [email-masked]

> - to have a way to find if there is an existing header that serves some
> given purpose.  

Problem is, absent good design review and an agreed-on specification, 
reuse of existing fields can cause interoperability problems.  
Precedence, Priority, and Return-Receipt-To are all examples.

Date: Thu, 20 Sep 2001 05:23:31 +0100
To: [email-masked]
From: Graham Klyne <[email-masked]>
Subject: Re: Locating RFC [2]822 headers
Cc: [email-masked], Jacob Palme <[email-masked]>
Sender: [email-masked]

At 08:26 AM 9/19/01 -0700, [email-masked] wrote:
>>How does one reliably track down all of the defined RFC [2]822
>>headers?  That is, when proposing a new header, how is one to be sure that
>>it is not already defined somewhere?
>
>Simple: Search all the RFCs and internet-drafts. This is hardly rocket 
>science.
>
>But as a practical matter the real issue isn't standardized use, or even
>proposed use in a standard. It is the fact that anyone can use any field name
>at any time without any specification whatsoever. This is much more difficult
>to gauge. Personally, I do it by keeping track of all the fields I encounter
>that seem reasonable -- 511 so far.

OK, I oversimplified my motivation for considering this.

There are two reasons that I have wanted there to be a header registry:

- to have a way to find if there is an existing header that serves some 
given purpose.  RFC2076 and subsequent drafts serve this purpose pretty 
well, but there's no mechanism for catching new headers as they are defined.

- to have a way to transfer RFC822 labelling and semantics to different 
environments.  Currently, there is no easy reference to say "here are the 
header names and this is what they mean".  My immediate interest here is 
message metadata for message archive/retrieval type of systems, using an 
XML-based format (e.g. draft-klyne-message-rfc822-xml-02.txt).  I can 
imagine other possibilities.

>Seems to me like a lot of work that won't really address the problem.

I think Keith's suggestion is a manageable approach.

#g

------------------------------------------------------------
Graham Klyne                    Baltimore Technologies
Strategic Research              Content Security Group
<[email-masked]>    <http://www.mimesweeper.com>
                                 <http://www.baltimore.com>
------------------------------------------------------------

Date: Wed, 19 Sep 2001 20:57:21 -0400
From: Lawrence Greenfield <[email-masked]>
To: Dave Crocker <[email-masked]>
Cc: [email-masked]
Subject: Re: Locating RFC [2]822 headers 
Sender: [email-masked]

   Date: Wed, 19 Sep 2001 16:52:20 -0700
   From: Dave Crocker <[email-masked]>

   >the danger in creating such a registry is that it will be used
   >to legitimize certain bad practices that have been implemented with
   >little thought.

   That is an inherent danger in any open registration environment.  It has 
   not proved  problem in any other IANA registration activities, as far as I 
   am aware.  If there is strong enough concern, of course, we can direct IANA 
   to permit only those registrations that are standards track.

I would much prefer an open registry, since it is the best way to
avoid name conflicts and show what's out there.  A registry doesn't
bless a particular entry, it just prevents accidental collision.

We could suggest some sort of Vendor-Foo- prefixes to make such things
even clearer---but I'd much rather have a registry that shows what's
in existence than one that merely shows standards track stuff (and
thus is perpetually out of date even with internet drafts).

Larry

Date: Wed, 19 Sep 2001 16:52:20 -0700
To: Keith Moore <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Locating RFC [2]822 headers 
Cc: Graham Klyne <[email-masked]>, [email-masked], Jacob Palme <[email-masked]>
Sender: [email-masked]

At 10:13 AM 9/19/2001, Keith Moore wrote:
>If it were desirable to establish a header registry, the last thing I'd
>want to do is to make an update of 2076 be critical path for that effort.
>These are large documents that take a long time to get right, and it's
>difficult to ensure that they get adequate review.

I agree with your concern, and wondered about it when sending my earlier 
note, but decided to see what reactions surfaced.

Ned's and your responses suggest that, at most, we should do the minimal 
effort to create a registry, rather than tackle full-scale header 
cleanup.  Or, at least, they should be de-coupled.

>the danger in creating such a registry is that it will be used
>to legitimize certain bad practices that have been implemented with
>little thought.

That is an inherent danger in any open registration environment.  It has 
not proved  problem in any other IANA registration activities, as far as I 
am aware.  If there is strong enough concern, of course, we can direct IANA 
to permit only those registrations that are standards track.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.273.6464

From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: Graham Klyne <[email-masked]>, [email-masked], Jacob Palme <[email-masked]>
Subject: Re: Locating RFC [2]822 headers 
Date: Wed, 19 Sep 2001 13:13:05 -0400
Sender: [email-masked]

If it were desirable to establish a header registry, the last thing I'd 
want to do is to make an update of 2076 be critical path for that effort.
These are large documents that take a long time to get right, and it's
difficult to ensure that they get adequate review.

It seems far simpler to write a short RFC that establishes the registry,
the rules for initially populating the registry, and the rules for 
adding or updating entries.    The rules for initially populating the
registry could say that 

- headers mentioned in 2076 are automatically included, and 
- when multiple definitions are available, all are listed, and
- those that are higher on the standards track take precedence. 

Keith

p.s. IMHO, the danger in creating such a registry is that it will be used
to legitimize certain bad practices that have been implemented with
little thought.

Date: Wed, 19 Sep 2001 08:37:58 -0700
To: Graham Klyne <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: Locating RFC [2]822 headers
Cc: [email-masked], Jacob Palme <[email-masked]>
Sender: [email-masked]

Graham,

Thanks for raising this.  I had a task something like this on my work queue 
from, perhaps, a year ago.  sigh.

On the other hand, I had only thought of documenting a procedure; you have 
suggested a larger task that is, really, a header cleanup project.

However, casting the approach as an extension to DRUMS work seems like the 
right thing to do.  That would justify having a document that repeats the 
definitions found in earlier documents, since DRUMS is about coalescing, as 
well as clarifying and fixing.  It does mean that we would be required to 
go over, and modify, RFC2076 carefully and achieve rough consensus on it.

The standards status of the result should be the same the same track as 2822.

d/

At 04:29 AM 9/19/2001, Graham Klyne wrote:
>The usual IETF way to deal with these situations is to have an IANA 
>registry of values.  But none exists for message headers (I can find no 
>reference to RFC822 or email messages in 
>http://www.iana.org/numbers.htm).  The nearest to a registry we have seems 
>to be Jacob Palme's RFC 2076 and more recent Internet Draft updates 
>(currently draft-palme-mailext-headers-05.txt).
>
>I would like to suggest taking Jacob's draft, and:
>(a) adding an IANA considerations section to create a message header 
>registry (or creating a new companion document)
>(b) incorporating all of the standards-track-defined headers noted in 
>Jacob's draft into the registry
>(c) issuing the result as BCP RFC(s)
>(d) noting in the RFC index that the document updates RFC 2822.

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.273.6464

Date: Wed, 19 Sep 2001 08:26:17 -0700 (PDT)
From: [email-masked]
Subject: Re: Locating RFC [2]822 headers
To: Graham Klyne <[email-masked]>
Cc: [email-masked], Jacob Palme <[email-masked]>
Sender: [email-masked]

> How does one reliably track down all of the defined RFC [2]822
> headers?  That is, when proposing a new header, how is one to be sure that
> it is not already defined somewhere?

Simple: Search all the RFCs and internet-drafts. This is hardly rocket science.

But as a practical matter the real issue isn't standardized use, or even
proposed use in a standard. It is the fact that anyone can use any field name
at any time without any specification whatsoever. This is much more difficult
to gauge. Personally, I do it by keeping track of all the fields I encounter
that seem reasonable -- 511 so far.

> I would like to suggest taking Jacob's draft, and:
> (a) adding an IANA considerations section to create a message header
> registry (or creating a new companion document)
> (b) incorporating all of the standards-track-defined headers noted in
> Jacob's draft into the registry
> (c) issuing the result as BCP RFC(s)
> (d) noting in the RFC index that the document updates RFC 2822.

> Before taking on any drafting work, I'd like to gauge the sense of whether
> this general approach is likely to fly.

Seems to me like a lot of work that won't really address the problem.

				Ned

Date: Wed, 19 Sep 2001 12:29:16 +0100
To: [email-masked]
From: Graham Klyne <[email-masked]>
Subject: Locating RFC [2]822 headers
Cc: Jacob Palme <[email-masked]>
Sender: [email-masked]

How does one reliably track down all of the defined RFC [2]822 
headers?  That is, when proposing a new header, how is one to be sure that 
it is not already defined somewhere?

 From the RFC index:

0822 Standard for the format of ARPA Internet text messages. D.
      Crocker. Aug-13-1982. (Format: TXT=109200 bytes) (Obsoletes RFC0733)
      (Obsoleted by RFC2822) (Updated by RFC1123, RFC1138, RFC1148,
      RFC1327, RFC2156) (Also STD0011) (Status: STANDARD)

and

2822 Internet Message Format. P. Resnick, Editor. April 2001. (Format:
      TXT=110695 bytes) (Obsoletes RFC0822) (Status: PROPOSED STANDARD)

Apart from the RFC822 "updated by" references to MIXER, there is no 
indication here that MIME adds a new set of header names that occupy (part 
of) the same naming scope as RFC [2]822 headers.  Also, there are numerous 
other documents that define additional headers that may appear in an 
Internet message.

The usual IETF way to deal with these situations is to have an IANA 
registry of values.  But none exists for message headers (I can find no 
reference to RFC822 or email messages in 
http://www.iana.org/numbers.htm).  The nearest to a registry we have seems 
to be Jacob Palme's RFC 2076 and more recent Internet Draft updates 
(currently draft-palme-mailext-headers-05.txt).

I would like to suggest taking Jacob's draft, and:
(a) adding an IANA considerations section to create a message header 
registry (or creating a new companion document)
(b) incorporating all of the standards-track-defined headers noted in 
Jacob's draft into the registry
(c) issuing the result as BCP RFC(s)
(d) noting in the RFC index that the document updates RFC 2822.

Before taking on any drafting work, I'd like to gauge the sense of whether 
this general approach is likely to fly.

------------
Graham Klyne
([email-masked])

To: [email-masked]
Xref: clerew local.mime:737
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: questions on RFC2822
Date: Mon, 17 Sep 2001 09:27:31 GMT
Lines: 26
Sender: [email-masked]

In <a05100303b7c2c23c729e@[216.43.25.67]> Pete Resnick <[email-masked]> writes:

>On 9/10/01 at 1:17 PM -0500, Eric A. Hall wrote:

>>1) There does not appear to be a requirement for a space after the 
>>colon in the header field...

>That is correct. Though a space definitely makes it more readable and 
>lots of clients produce the space, it is not required at all, I've 
>seen messages without, and I've never seen anybody fail to parse a 
>header without a space.

Apparently some existing news software barfs if that space is missing,
which is why Usefor-05 disallows it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 12 Sep 2001 17:25:57 -0700 (PDT)
From: [email-masked]
Subject: Re: questions on RFC2822
To: Russ Allbery <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> ned <[email-masked]> writes:

> > What's the actual message/second number here?

> I've done sustained rates as high as 60 messages a second with INN and no
> particularly interesting hardware, and I'm pretty sure people are doing a
> lot more than that.

Interesting. FYI, the performance of a reasonable MTA on ho-hum hardware these
days tends to be in the 30-60 message/second range, and rates of up to 200
message/second aren't unheard of.

I've seen rates in the 200-800 message/second range on better hardware.

> Average daily traffic I believe is somewhat less than
> that, probably something around 10-20 messages a second for a completely
> full feed.

Any reasonable MTA could handle this rate.

> Many of those messages are >1MB in size.

MTAs tend to be sized and tuned to handle smaller messages than this, so this
could be an issue.

> One of the differences between mail and news is that with mail, once you
> get traffic beyond a certain level, you just add more MX hosts, and you
> distribute your users across multiple mail servers, so that no one machine
> has to handle all your traffic.  There really isn't any way to do that
> with news effectively right now; one server pretty much has to process all
> incoming news.

MX is indeed an advantage MTAs have. However, the ability to rack and stack
tends to disappear once you switch from talking about relay rates to delivery
rates. Building a store that can keep up with the rates a highly optimized MTA
can achieve is, well, challenging.

				Ned

To: [email-masked]
Subject: Re: questions on RFC2822
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: 12 Sep 2001 16:02:42 -0700
Lines: 19
Sender: [email-masked]

ned <[email-masked]> writes:

> What's the actual message/second number here?

I've done sustained rates as high as 60 messages a second with INN and no
particularly interesting hardware, and I'm pretty sure people are doing a
lot more than that.  Average daily traffic I believe is somewhat less than
that, probably something around 10-20 messages a second for a completely
full feed.  Many of those messages are >1MB in size.

One of the differences between mail and news is that with mail, once you
get traffic beyond a certain level, you just add more MX hosts, and you
distribute your users across multiple mail servers, so that no one machine
has to handle all your traffic.  There really isn't any way to do that
with news effectively right now; one server pretty much has to process all
incoming news.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Wed, 12 Sep 2001 08:05:02 -0700 (PDT)
From: [email-masked]
Subject: Re: questions on RFC2822
To: [email-masked]
Cc: [email-masked]
Sender: [email-masked]

> Not so much newsreaders as news*servers*. Newsservers are often highly
> optimized, because they need to be able to process a full feed of Usenet -
> which is probably significantly more articles per second than even the
> largest mail servers do mails per second, *and* people want articles to go
> out after a delay measured only in milliseconds.

What's the actual message/second number here?

				Ned

Date: 12 Sep 2001 13:50:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
Subject: Re: questions on RFC2822
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked] (Pete Resnick)  wrote on 10.09.01 in <a05100303b7c2c23c729e@[216.43.25.67]>:

> On 9/10/01 at 1:17 PM -0500, Eric A. Hall wrote:

> >...particularly since usefor-05 says that some of the field body
> >must appear on the same line as the field name.
>
> Usefor is a particular profile of 2822 since evidently newsreaders
> have problems parsing all of 2822.

Not so much newsreaders as news*servers*. Newsservers are often highly  
optimized, because they need to be able to process a full feed of Usenet -  
which is probably significantly more articles per second than even the  
largest mail servers do mails per second, *and* people want articles to go  
out after a delay measured only in milliseconds.

And remember, Usenet doesn't have an envelope like SMTP does. That makes  
for completely different performance requirements.

OTOH, Newsservers - at least in their high-performance transit parts - are  
typically only interested in a subset of all headers (those relevant to  
decide how to handle each article).

So, what they need is to be able to locate the relevant headers fast (this  
is where general header rules come in) and then to parse those few headers  
fast (which is where stricter rules for a few headers come in).

MfG Kai

Date: Tue, 11 Sep 2001 21:45:42 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: questions on RFC2822
Sender: [email-masked]

dots (e.g., lots of X.400-foo headers, as well as vnd.foo.bar headers).

	Tony

[email-masked] wrote:
> 
> > ned <[email-masked]> writes:
> > >> On 9/10/01 at 1:17 PM -0500, Eric A. Hall wrote:
> 
> > >>> 2) Although any printable character from US-ASCII except colon may
> > >>> be used in header names, I've never seen any headers use anything
> > >>> other than alphanumerics and hyphen.
> 
> > >> I've never seen one that I can recall. Anyone else?
> 
> > > Hmm. I've seen $ and # and * for sure. I think I've seen % and ^ once or
> > > twice, but it was probably an error of some sort that led to it. I've
> > > seen ( and ) and ; used, but it was _definitely_ bogus usage -- leakage
> > > from GroupWise, if memory serves.
> 
> > I've seen plenty of underscores.
> 
> Yup, you're right. Missed that one.
> 
>                                 Ned

Date: Tue, 11 Sep 2001 12:51:57 +0200 (CEST)
From: Simon Josefsson <[email-masked]>
To: <[email-masked]>
cc: Pete Resnick <[email-masked]>, "Eric A. Hall" <[email-masked]>, <[email-masked]>
Subject: Re: questions on RFC2822
Sender: [email-masked]

On Mon, 10 Sep 2001 [email-masked] wrote:

> > On 9/10/01 at 1:17 PM -0500, Eric A. Hall wrote:
>
> > >1) There does not appear to be a requirement for a space after the
> > >colon in the header field...
>
> > That is correct. Though a space definitely makes it more readable and
> > lots of clients produce the space, it is not required at all, I've
> > seen messages without, and I've never seen anybody fail to parse a
> > header without a space.
>
> I've seen these as well, and I've never seen anything that had a problem.

Programs created for UseNet but later adapted to mail has been seen
having such problems. (UseNet articles must have SP after the colon.)

Date: Mon, 10 Sep 2001 17:08:41 -0700 (PDT)
From: [email-masked]
Subject: Re: questions on RFC2822
To: Russ Allbery <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> ned <[email-masked]> writes:
> >> On 9/10/01 at 1:17 PM -0500, Eric A. Hall wrote:

> >>> 2) Although any printable character from US-ASCII except colon may
> >>> be used in header names, I've never seen any headers use anything
> >>> other than alphanumerics and hyphen.

> >> I've never seen one that I can recall. Anyone else?

> > Hmm. I've seen $ and # and * for sure. I think I've seen % and ^ once or
> > twice, but it was probably an error of some sort that led to it. I've
> > seen ( and ) and ; used, but it was _definitely_ bogus usage -- leakage
> > from GroupWise, if memory serves.

> I've seen plenty of underscores.

Yup, you're right. Missed that one.

				Ned

To: [email-masked]
Subject: Re: questions on RFC2822
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: 10 Sep 2001 15:29:35 -0700
Lines: 18
Sender: [email-masked]

ned <[email-masked]> writes:
>> On 9/10/01 at 1:17 PM -0500, Eric A. Hall wrote:

>>> 2) Although any printable character from US-ASCII except colon may
>>> be used in header names, I've never seen any headers use anything
>>> other than alphanumerics and hyphen.

>> I've never seen one that I can recall. Anyone else?

> Hmm. I've seen $ and # and * for sure. I think I've seen % and ^ once or
> twice, but it was probably an error of some sort that led to it. I've
> seen ( and ) and ; used, but it was _definitely_ bogus usage -- leakage
> from GroupWise, if memory serves.

I've seen plenty of underscores.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Mon, 10 Sep 2001 14:14:02 -0700
To: [email-masked]
From: Dave Crocker <[email-masked]>
Subject: Re: questions on RFC2822
Sender: [email-masked]

At 12:17 PM 9/10/2001, Pete Resnick wrote:
>On 9/10/01 at 1:17 PM -0500, Eric A. Hall wrote:
>>1) There does not appear to be a requirement for a space after the colon 
>>in the header field...
>
>That is correct. Though a space definitely makes it more readable and lots 
>of clients produce the space, it is not required at all, I've seen 
>messages without, and I've never seen anybody fail to parse a header 
>without a space.

To carry the explanation further:

The original specification (RFC733) was quite careful to make the presence 
of an arbitrary of spaces (1 or more) entirely optional.

As with a number of format standards, RFC733/822/2822 walks a tightrope 
between specifying a computer-processable format and a human-friendly 
format.  The latter was the reason for variable spacing.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.273.6464

From: Keith Moore <[email-masked]>
To: Marc Mutz <[email-masked]>
cc: Keith Moore <[email-masked]>, [email-masked]
Subject: Re: how to merge obs-id-{left,right} into a new msg-id on reply? 
Date: Mon, 10 Sep 2001 16:59:52 -0400
Sender: [email-masked]

> That wasn't my intention. It was more a question of whether to keep
> CFWS, which is allowed in the obs-msg-id, but not in (rfc2822-) msg-id
> anymore, or strip it. The semantic value of the message-id would not
> change. Consider e.g.
> 
> Message-ID: <ABCFD@(domain-part)
>  kde.org>
> 
> RFC2822 would disallow the creation of this. Now I could convert it to
> Message-ID: <[email-masked]>
> or keep it exactly like I received it. The only problem I see with
> converting is breaking old mailers that do not tokenize headers
> according to rfc822 before interpreting them, but use the verbatim text
> (incl. comment and folding wsp) to compare msgid's.

The safer thing to do is to leave it as-is, but my guess is that this
case occurs so rarely that it really doesn't matter.

> > Say you were replying to a message that had a newly-invalid From
> > address (valid under 822 and invalid under 2822). Would you create a
> > new address as the destination of the reply?
> 
> There are rfc822-from addresses that are invalid under 2822? Didn't know
> that yet...

I think so, though I didn't check to be absolutely sure.
IIRC, the forms that are excluded in 2822 are extremely unlikely to be 
used in practice.

the basic point I was trying to make is that the new syntax restrictions
in 2822 should probably not be interpreted as applying to those portions 
of a newly composed message which were automagically obtained from a reply.

Keith

Date: Mon, 10 Sep 2001 12:45:52 -0700 (PDT)
From: [email-masked]
Subject: Re: questions on RFC2822
To: Pete Resnick <[email-masked]>
Cc: "Eric A. Hall" <[email-masked]>, [email-masked]
Sender: [email-masked]

> On 9/10/01 at 1:17 PM -0500, Eric A. Hall wrote:

> >1) There does not appear to be a requirement for a space after the
> >colon in the header field...

> That is correct. Though a space definitely makes it more readable and
> lots of clients produce the space, it is not required at all, I've
> seen messages without, and I've never seen anybody fail to parse a
> header without a space.

I've seen these as well, and I've never seen anything that had a problem.

> >2) Although any printable character from US-ASCII except colon may
> >be used in header names, I've never seen any headers use anything
> >other than alphanumerics and hyphen.

> I've never seen one that I can recall. Anyone else?

Hmm. I've seen $ and # and * for sure. I think I've seen % and ^ once or twice,
but it was probably an error of some sort that led to it. I've seen ( and ) and
; used, but it was _definitely_ bogus usage -- leakage from GroupWise, if
memory serves.

> >...with the entire unfolded header field within 998 bytes.

> I've met many an address field that is well over 998 bytes unfolded.

Biggest real and intentional usage I've seen was on the order of 25Mb.
Lots of things didn't handle it at all well.

				Ned

Date: Mon, 10 Sep 2001 14:17:34 -0500
To: "Eric A. Hall" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: questions on RFC2822
Cc: [email-masked]
Sender: [email-masked]

On 9/10/01 at 1:17 PM -0500, Eric A. Hall wrote:

>1) There does not appear to be a requirement for a space after the 
>colon in the header field...

That is correct. Though a space definitely makes it more readable and 
lots of clients produce the space, it is not required at all, I've 
seen messages without, and I've never seen anybody fail to parse a 
header without a space.

>2) Although any printable character from US-ASCII except colon may 
>be used in header names, I've never seen any headers use anything 
>other than alphanumerics and hyphen.

I've never seen one that I can recall. Anyone else?

>3) There is no explicit length limit on field names.

Correct. You only have to stay with in the line length limit.

>...particularly since usefor-05 says that some of the field body 
>must appear on the same line as the field name.

Usefor is a particular profile of 2822 since evidently newsreaders 
have problems parsing all of 2822. It is silly to implement 
folding/unfolding and not be able to handle it on the first line, but 
it's fine if usefor wants to impose stricter limitations. They do not 
apply to mail.

On the flip side, there is nothing in 2822 that requires making 
ridiculously long header names.

>4) There is no explicit length limit on an unfolded header 
>field...My assumption is that the friendliest behavior is to 
>endeavor to keep folded parts at 78 bytes...

Absolutely. That follows the 78 recommendation.

>...with the entire unfolded header field within 998 bytes.

I've met many an address field that is well over 998 bytes unfolded. 
Parsing much larger fields is a necessity for any implementation. 
There is no need to stick to 998 for unfolded field length.

>5) String length in header bodies is similarly undefined.

As above.

pr
-- 
Pete Resnick <mailto:[email-masked]>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Mon, 10 Sep 2001 13:17:01 -0500
From: "Eric A. Hall" <[email-masked]>
Organization: EHS Company
To: [email-masked]
Subject: questions on RFC2822
Sender: [email-masked]

I've been reading through RFC2822 and have some questions particularly in
regards to how the robustness principle should be applied to some of the
formatting rules when used on multi-purpose messaging systems. More
specifically, I'm looking for confirmation on my interpretations.

1) There does not appear to be a requirement for a space after the colon
in the header field, although I don't think I've ever seen name:data
(without a space) used anywhere. I assume the friendliest behavior is for
messaging systems to use colon-space on messages they send, but to expect
colon-only in messages they receive.

2) Although any printable character from US-ASCII except colon may be used
in header names, I've never seen any headers use anything other than
alphanumerics and hyphen. Moreover, chances are probably pretty good that
the use of chars like open parenthesis and single-quote would be harmful
to some implementations. usefor-05 only allows letter-digit-hyphen chars
and that is a reasonable safe set for new headers to use, although the
full set should be tolerated. Any communal knowledge of experimental field
names which use other chars?

3) There is no explicit length limit on field names. This would seem to
indicate that the maximum allowable field name is 997 chars [(leaving room
for the colon) see related question in #4 below], with a preferred maximum
of 77 chars (again, leaving room for the colon). However, this would also
mean that the field body would start on a continuation line, which may
cause problems, particularly since usefor-05 says that some of the field
body must appear on the same line as the field name. This means that field
names must be short enough to accomodate at least some data from the field
body, but what this number is exactly is undefinable as a universal value.
"should be as long as necessary but no longer"?

4) There is no explicit length limit on an unfolded header field, but 
only on the folded parts of the field (since they are the elements which
are parsed as "lines"). However, it is somewhat likely that there exists
at least one implementation which limits operations to 1000 bytes of data.
My assumption is that the friendliest behavior is to endeavor to keep
folded parts at 78 bytes, with the entire unfolded header field within 998
bytes.

5) String length in header bodies is similarly undefined.

Out of curiousity, has anybody ever tested the maximum length conditions?
IE, what happens when a field name is 997 bytes, and is followed by a
field body which is 998 bytes of continuous data? This appears to be
technically valid based on my reading, although it would probably be
self-defeating for the sender.

Thanks for any insights, comments, corrections, etc.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

Date: Mon, 10 Sep 2001 17:35:15 +0200
From: Marc Mutz <[email-masked]>
Subject: Re: how to merge obs-id-{left,right} into a new msg-id on reply?
To: Keith Moore <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

On Monday 10 September 2001 02:44, Keith Moore wrote: 
> > Suppose I reply to a message whose Message-ID field has a left id 
> > that conforms to obs-left-id, but not left-id. How should I copy 
> > the msg-id then? Strip the non-comforming parts and risk to break 
> > older mailers' threading or copy everything from the "<" through to 
> > the ">" verbatim from the parent, risking to thus create a message 
> > in violation of rfc2822? 
> 
> I'd say the latter.  It's certainly incorrect to create a completely 
> new message-id (correct or not) and say that you've replied to it. 
 
That wasn't my intention. It was more a question of whether to keep 
CFWS, which is allowed in the obs-msg-id, but not in (rfc2822-) msg-id 
anymore, or strip it. The semantic value of the message-id would not 
change. Consider e.g. 
 
Message-ID: <ABCFD@(domain-part) 
 kde.org> 
 
RFC2822 would disallow the creation of this. Now I could convert it to 
Message-ID: <[email-masked]> 
or keep it exactly like I received it. The only problem I see with 
converting is breaking old mailers that do not tokenize headers 
according to rfc822 before interpreting them, but use the verbatim text 
(incl. comment and folding wsp) to compare msgid's. 
 
> Say you were replying to a message that had a newly-invalid From 
> address (valid under 822 and invalid under 2822). Would you create a 
> new address as the destination of the reply? 
 
There are rfc822-from addresses that are invalid under 2822? Didn't know 
that yet... 
 
Marc 
 
- --  
The Commission and Member States are urged to devise appropriate 
measures to promote, develop and manufacture European encryption 
technology and software and above all to support projects aimed at 
developing user-friendly open-source encryption software. 
- -- EuroParl. Temp. Committee on the ECHELON Interception System 
   http://www.europarl.eu.int/tempcom/echelon/pdf/prechelon_en.pdf 
 
pèŽ A~pèŽwA�pèŽsA‰pèŽpAŽpèŽ)A”

From: Keith Moore <[email-masked]>
To: Marc Mutz <[email-masked]>
cc: [email-masked]
Subject: Re: how to merge obs-id-{left,right} into a new msg-id on reply? 
Date: Sun, 09 Sep 2001 20:44:31 -0400
Sender: [email-masked]

> Suppose I reply to a message whose Message-ID field has a left id that
> conforms to obs-left-id, but not left-id. How should I copy the msg-id
> then? Strip the non-comforming parts and risk to break older mailers'
> threading or copy everything from the "<" through to the ">" verbatim
> from the parent, risking to thus create a message in violation of
> rfc2822?

I'd say the latter.  It's certainly incorrect to create a completely 
new message-id (correct or not) and say that you've replied to it.

Say you were replying to a message that had a newly-invalid From address
(valid under 822 and invalid under 2822). Would you create a new address
as the destination of the reply?  

Keith

Date: Sun, 09 Sep 2001 22:52:49 +0200
From: Marc Mutz <[email-masked]>
Subject: how to merge obs-id-{left,right} into a new msg-id on reply?
To: [email-masked]
Sender: [email-masked]

Hi! 
 
Since I'm new on this low-volume mailing list, let me introduce myself 
first: My name is Marc Mutz and I am a student of Physics at Bielefeld 
University, Germany. Till lately I hacked a bit at the Linux kernel's 
cryptoapi support (www.kerneli.org) and have written the corresponding 
HOWTO (Encryptionhowto.sf.net). I'm currently working on a new core for 
the message handling of KMail, the KDE mail reader (www.kde.org). 
 
Here's my problem with RFC 2822: 
I've run into the question whether to follow the rule: 
 
1. (e.g.) copy the contents of parent's "Message-ID", "In-Reply-To" and  
"References" to the child's "References" (and similar) 
- - -or- 
2. obs-* MUST not be generated by conformant mailers. 
 
Suppose I reply to a message whose Message-ID field has a left id that  
conforms to obs-left-id, but not left-id. How should I copy the msg-id  
then? Strip the non-comforming parts and risk to break older mailers'  
threading or copy everything from the "<" through to the ">" verbatim  
from the parent, risking to thus create a message in violation of  
rfc2822? 
 
Thanks, 
Marc 
 
- --  
"The problem with today's Internet is that it's dumb, boring and 
isolated," says Colony [CEO and chairman of Forrester Research]. 
Perhaps he and many like him are missing the real point: is it really 
the Internet, or is it users like himself that are dumb, boring and 
isolated?  -- John Horvath: "Internet X", Telepolis 2001/08/14 (#9309) 

puA 3AµäÈpFø”°uA€