Subject: Problems with in-reply-to threading
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked], [email-masked]
Date: Fri, 31 Aug 2001 14:00:57 +0200
List-Id: <ietf-822.imc.org>

Hi,
recently I have had some problems with messages being threaded 
inappropriately in my (IMAP) inbox.

Being the kind of person I am, I more or less hunted it down, finding these 
gems:

1) Header:
In-Reply-To: Message from Harald Tveit Alvestrand <[email-masked]>
   of "Fri, 06 Jul 2001 13:17:51 +0200." 
<759246899.994425471@[192.168.1.31]>

2) RFC 2822:

4.5.4. Obsolete identification fields

   The obsolete "In-Reply-To:" and "References:" fields differ from the
   current syntax in that they allow phrase (words or quoted strings) to
   appear.  The obsolete forms of the left and right sides of msg-id
   allow interspersed CFWS, making them syntactically identical to
   local-part and domain respectively.

obs-message-id  =       "Message-ID" *WSP ":" msg-id CRLF

obs-in-reply-to =       "In-Reply-To" *WSP ":" *(phrase / msg-id) CRLF

Note the absence, inherited from 822, of anything indicating the proper
content of a "phrase".

3) THREAD specification - draft-ietf-imapext-thread-07, section 6.3:

            If a message does not contain a References header line, or
            the References header line does not contain any valid
            Message IDs, then use the FIRST (if any) valid Message ID
            found in the In-Reply-To header line as the only reference
            (parent) for this message.

               Note: Although RFC 822 permits multiple Message IDs in
               the In-Reply-To header, in actual practice this
               discipline has not been followed.  For example,
               In-Reply-To headers have been observed with email
               addresses after the Message ID, and there are no good
               heuristics for software to determine the difference.
               This is not a problem with the References header however.

            If a message does not contain an In-Reply-To header line, or
            the In-Reply-To header line does not contain a valid Message
            ID, then the message does not have any references (NIL).

My capitalization.

Even more worrisome is this header:

References: <[email-masked]>
	<257869316.998951974@[192.168.1.31]>
	<[email-masked]>
	<[email-masked]>
	<4475164.999069825@localhost>

which was apparently created on the basis of a different incarnation of the 
previous one, following the algorithm of RFC 2822 section 3.6.4:

   The "References:" field will contain the contents of the parent's
   "References:" field (if any) followed by the contents of the parent's
   "Message-ID:" field (if any).  If the parent message does not contain
   a "References:" field but does have an "In-Reply-To:" field
   containing a single message identifier, then the "References:" field
   will contain the contents of the parent's "In-Reply-To:" field
   followed by the contents of the parent's "Message-ID:" field (if
   any).  If the parent has none of the "References:", "In-Reply-To:",
   or "Message-ID:" fields, then the new message will have no
   "References:" field.

(It seems to have missed the part about In-Reply-To field containing a 
single messgae identifier, though...)

The product creating the initial problem seems to be an MH variant's 
group-reply-to init file (replgroupcomps); we can fix it one install at a 
time, but this seems like a glorious time for clarifications....

Suggested fixes:

1) In RFC 2822bis, state that the msg-id form of obs-in-reply-to MUST 
contain a message-ID, and NOT an email address (a "phrase" cannot contain 
an unquoted angle bracket, so it is only the msg-id that allows it)

2) In RFC 2822bis section 3.6.4, state that the In-reply-to should only be 
used to form References if it has a single message-ID, and that the reason 
is that users of obs-in-reply-to sometimes put emails in their in-reply-to 
fields.

3) Unless someone says that a reasonably widespread implementation exists 
that puts FIRST a message-ID and THEN an email address into the in-reply-to 
field, change the THREAD specification to pick up the LAST instead of the 
first identifier.

What do people think?

                  Harald

Date: Thu, 16 Aug 2001 11:14:55 -0700
From: Chris Newman <[email-masked]>
Subject: Re: Quoted Local-Part
To: Lee Thompson <[email-masked]>, [email-masked]
Sender: [email-masked]

You _really_ need to read and understand the ABNF in these specifications. 
All of your questions can be answered simply by reading the ABNF.

To get you started in how to use the ABNF, I'll answer your questions:

--On Wednesday, August 15, 2001 19:14 -0700 Lee Thompson 
<[email-masked]> wrote:
> 1. Are quotes taken as literal delimiters in comments?   E.g.
>                 Joe(A comment ")" here)[email-masked]

No.  See the ABNF in RFC 2822.  Note that a double-quote is part of "ctext" 
which is part of "ccontent".

> 2.  Is there such a thing as a comment in a literal such that the comment
> gets stripped?
>                 "Joe(The Thug) User"@arrgh.org

No.  See the ABNF in RFC 2822.  Parens are part of "qtext".

> 3. Should comments be omitted in any 821/2821 envelope?

Yes.  See ABNF for "Mailbox" in RFC 2821.  No comments permitted in there.

>    Is it better
> just to pass through quoted stuff completely untouched?

Yes.

> Should a quote
> char '\' be stripped out when found - or just passed on?

Within a quoted string, I'd leave quoted-pairs alone (indeed if it quotes 
'\' or '"', you have to leave it alone or you're breaking the address). 
Outside the quoted string, it depends.  If you're claiming conformance with 
RFC 2821 you'd have to reject the address or strip it (since it's a 
protocol syntax violation to generate a quoted pair outside of a quoted 
string when acting in the SMTP client role).  If you're only claiming 
conformance with RFC 821, you could pass it on.

> 4. We've run into a particular MTA that does not escape commas in quoted
> local-parts which appear to be contrary to at least RFC 2822.   E.g.
> "8379,39082"@domain.com          It seems prudent to accept this as is and
> continue on.

That is a perfectly legal and correct address by RFC 2822 since "," is in 
qtext.

		- Chris

Date: Thu, 16 Aug 2001 00:10:35 -0400
From: Sam Roberts <[email-masked]>
To: [email-masked]
Subject: Re: Quoted Local-Part
Mail-Followup-To: [email-masked]
Sender: [email-masked]

Quoting Lee Thompson <[email-masked]>, who wrote:
> Have a couple quick questions regarding quoted local-part addresses.     I
> realize this is the 822 group but this is affecting our MTA on both the RFC
> 821 MAIL FROM/RCPT TO and the 822 headers.
> 
> 1. Are quotes taken as literal delimiters in comments?   E.g.
>                 Joe(A comment ")" here)[email-masked]

I was pretty sure you could put a quoted string in a comment, but I
checked, and no, you can't. You can put a quoted pair in though:

    Joe(A comment \) here)[email-masked]
    Joe(A comment with a double quote " here)[email-masked]

> 2.  Is there such a thing as a comment in a literal such that the comment
> gets
> stripped?
>                 "Joe(The Thug) User"@arrgh.org

No, it's not a comment if its in a quoted string.

> 3. Should comments be omitted in any 821/2821 envelope?    Is it better just
> to pass through quoted stuff completely untouched?  Should a quote char '\'
> be stripped out when found - or just passed on?

Sorry, I'm not really familiar with 821.

> 4. We've run into a particular MTA that does not escape commas in quoted
> local-parts which appear to be contrary to at least RFC 2822.   E.g.
> "8379,39082"@domain.com          It seems prudent to accept this as is and
> continue on.

The only thing you have to quote in a quoted string is the double
quote and backslash.

Cheers,
Sam

-- 
Sam Roberts <[email-masked]> (Vivez sans temps mort!)

From: "Lee Thompson" <[email-masked]>
To: <[email-masked]>
Subject: Quoted Local-Part
Date: Wed, 15 Aug 2001 19:14:05 -0700
Sender: [email-masked]

Have a couple quick questions regarding quoted local-part addresses.     I
realize this is the 822 group but this is affecting our MTA on both the RFC
821 MAIL FROM/RCPT TO and the 822 headers.

Our engineers have been over the RFCs 821, 822, 2821 and 2822 and had some
questions -- I thought I'd pass the questions along because -- well -- I'd
like to get this right ;)

1. Are quotes taken as literal delimiters in comments?   E.g.
                Joe(A comment ")" here)[email-masked]

2.  Is there such a thing as a comment in a literal such that the comment
gets
stripped?
                "Joe(The Thug) User"@arrgh.org

3. Should comments be omitted in any 821/2821 envelope?    Is it better just
to pass through quoted stuff completely untouched?  Should a quote char '\'
be stripped out when found - or just passed on?

4. We've run into a particular MTA that does not escape commas in quoted
local-parts which appear to be contrary to at least RFC 2822.   E.g.
"8379,39082"@domain.com          It seems prudent to accept this as is and
continue on.

From: Keith Moore <[email-masked]>
To: Vince Sabio <[email-masked]>
cc: [email-masked]
Subject: Re: bang-formatted addresses 
Date: Wed, 15 Aug 2001 14:51:44 -0400
Sender: [email-masked]

> Question about bangs (exclamation marks) in local-parts of addresses:
> My reading of the expansion of local-part to dot-atom to atext is
> that a bang is (1) permitted in the local-part of the address and (2)
> may appear _anywhere_ in the local-part.
> 
> Specifically, constructions such as <[email-masked]> and
> <[email-masked]> are permitted, even though they technically violate
> parsing rules for bang addresses.
> 
> Correct?

Internet mail standards don't specify the interpretation of local-parts;
such interpretation is done only by the domain named on the right side 
of the @ sign.  (for instance, nothing requires that a domain interpret 
a local-part containing ! as if it were a uucp address)

So yes, the ! can appear anywhere in a local-part, but that doesn't mean
that the domain on the right side of the @ sign will consider that local
part a valid address.

[email-masked] is perfectly valid if example.com thinks it is.

Keith

Date: Wed, 15 Aug 2001 14:00:32 -0400
To: [email-masked]
From: Vince Sabio <[email-masked]>
Subject: bang-formatted addresses
Sender: [email-masked]

Question about bangs (exclamation marks) in local-parts of addresses: 
My reading of the expansion of local-part to dot-atom to atext is 
that a bang is (1) permitted in the local-part of the address and (2) 
may appear _anywhere_ in the local-part.

Specifically, constructions such as <[email-masked]> and 
<[email-masked]> are permitted, even though they technically violate 
parsing rules for bang addresses.

Correct?

Much gracias.

- Vince

Date: Mon, 6 Aug 2001 17:44:19 +0200
To: [email-masked], [email-masked]
From: Jacob Palme <[email-masked]>
Subject: Re: Searching for implementations of content-language header
Cc: Torgny Tholerus  <[email-masked]>, Lars Enderin <[email-masked]>
Sender: [email-masked]

>Hi,
>in progressing the content-language draft (the part of RFC 
>1766 that wasn't just defining language tags) to Draft 
>Standard, I need to know about implementations.
>
>I know about the HTTP implementation report, but would like 
>to know about usage outside of HTTP. (Verification of usage 
>in HTTP is welcome too!)
>All protocols that use 822-like headers welcome - feel free 
>to forward this request to relevant mailing lists!
>
>In particular:
>
>- Content-language usage
>- Accept-language usage
>	- With quality factors
>	- Without quality factors
>- Usage of content-language/accept-language in negotiation
>
>Please send to me privately; I will collect, and send the 
>summary to ietf-languages.
>
>The draft is draft-alvestrand-content-language-02.txt
>
>Thank you!
>
>            Harald

We are planning an implementation of multi-linguality in our
system. It will probably not be ready until early next year.

At present, I am thinking of implementing the following:

Multipart/alternative:

     First body part: All languages in sequence in plain text format

     2nd to n-1th body part: Each language version as a 
different body part

     Last body part: All languages in sequence in HTML format with
     a table of contents in the beginning with internal links to
     the different languages.

Since my investigation showed that all existing mailers
display either the first or the last body part in
multipart/alternative, and completely disregard any
"Content-Language" in choosing which body part to display,
the solution above will ensure that everyone will get the
message in a language they can understand. They will also get
the message in other languages, but the message can be
organised to make it easy to skip the language you do not
want.

And the above format makes it possible for future mailers to
chose to display only the appropriate language version
depending on the user choices.

-- 
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/

Date: Mon, 6 Aug 2001 17:52:09 +0200
To: [email-masked]
From: Jacob Palme <[email-masked]>
Subject: Re: Searching for implementations of content-language header
Cc: IETF mailing list on MIME and e-mail <[email-masked]>
Sender: [email-masked]

You might be interested in the kind of HTML wich Microsoft
Word produces. Below is an example of what I got when I saved
a Word document in HTML format:

<p class=MsoNormal><span lang=EN-US 
style='font-size:12.0pt;font-family:Times'>
This text is in English.
<o:p></o:p></span></p>

<p class=MsoNormal><span lang=FR 
style='font-size:12.0pt;font-family:Times;
mso-ansi-language:FR'>
Ce texte est en fran&ccedil;ais.
<o:p></o:p></span></p>

Their use of the "lang" attribute is in accordance with the HTML 4.0
standard.
-- 
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/

Date: Mon, 06 Aug 2001 11:02:39 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: [email-masked], [email-masked]
Subject: Searching for implementations of content-language header
Sender: [email-masked]

Hi,
in progressing the content-language draft (the part of RFC 1766 that wasn't 
just defining language tags) to Draft Standard, I need to know about 
implementations.

I know about the HTTP implementation report, but would like to know about 
usage outside of HTTP. (Verification of usage in HTTP is welcome too!)
All protocols that use 822-like headers welcome - feel free to forward this 
request to relevant mailing lists!

In particular:

- Content-language usage
- Accept-language usage
	- With quality factors
	- Without quality factors
- Usage of content-language/accept-language in negotiation

Please send to me privately; I will collect, and send the summary to 
ietf-languages.

The draft is draft-alvestrand-content-language-02.txt

Thank you!

            Harald

Date: Fri, 3 Aug 2001 14:46:35 +0200
To: IETF mailing list on MIME and e-mail  <[email-masked]>
From: Jacob Palme <[email-masked]>
Subject: Putting tables in e-mail
Sender: [email-masked]

I have written a web page on how to put tables into e-mail.
You can find it at http://dsv.su.se/jpalme/ietf/tables-in-email.html
-- 
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/