Subject: Re: RFC 822/MIME/DSN/MDN "Message Lint"
From: Dave Crocker <[email-masked]>
To: "D. J. Bernstein" <[email-masked]>
Cc: [email-masked]
Date: Wed, 14 Apr 1999 17:10:26 +0700

At 10:14 PM 4/8/99 +0000, D. J. Bernstein wrote:
>msglint barfs on the comments in the example on page 7 of RFC 822.
>
>Is a broken validator better than no validator at all?

probably about as useful as a critic lacking in proper socialization.  we 
are all forced to tolerate imperfections in this world.

d/

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Dave Crocker                                         Tel: +1 408 246 8253
Brandenburg Consulting                               Fax: +1 408 273 6464
675 Spruce Drive                             <http://www.brandenburg.com>
Sunnyvale, CA 94086 USA                 <mailto:[email-masked]>

X400-Received: by mta SKIMA in /c=IS/admd=isgatt/; Relayed;  14 Apr 1999 08:39:18 Z
X400-Received: by /c=IS/admd=isgatt/; Relayed; 14 Apr 1999 08:39:18 Z
X400-MTS-Identifier: [/c=IS/admd=isgatt/; 02D5C3714543606F-SKIMA]
Content-Identifier: 02D5C3714543606F
Content-Return: Allowed
X400-Content-Type: P2-1988 ( 22 )
Conversion: Allowed
Original-Encoded-Information-Types: (1)(0)(10021)(7)(1)(0)(1),  (1)(0)(10021)(7)(1)(0)(6), (1)(0)(10021)(7)(1)(0)(100)
Disclose-Recipients: Prohibited
Alternate-Recipient: Allowed
X400-Originator: [email-masked]
X400-Recipients: non-disclosure;
Date: 14 Apr 1999 08:39:18 Z
From: "Skima: Gudbjorn Hreinsson" <[email-masked]>
To: [email-masked], <[email-masked]>
Subject: Re: Misuse of Content-Disposition (Was: Content-Disposition ancillary)
Sender: [email-masked]

Hi there,

I've been 'lurking on this list for quite a while now, but this topic
interests me. I've been trying to get my Mail*Hub to accept all
kinds of differently formatted Content-Disposition messages as
per the below.

I'm not an expert, but reading the rfc's they mention that a filename
parameter is optional for inline and up to the implementation of the
client what it should do with it.

Netscape seems to generate inline for all text/plain messages and
shows this both as inline and optionally as a file to it's users. Quite
handy for NS users but quite painful for other users who have mail
clients having other implementations.

Exchange is a wildcard, all kinds of differently formatted messages
originate from there all depending on versions and configuration.

Regarding HP's OpenMail it seems to quite often use the Subject
as the name and filename parameter on single message body parts.

My conclusion was that the rfc's provided no means for a transparent
method of indicating if it was a inline body part or an attachment, too 
many possibilities for different implementations. I wrote both Netscape 
and HP but only got the reply from HP that this was rfc compliant, which
I cannot object to.

Now, if the rfc's are really more specific than this I'd like some pointers
in that regards, or maybe this is being worked on in further rfc's?

Hope this is within the scope of this list,

Rgds,
-GSH

[email-masked] on 13.04.1999 22:27:00
To:	[email-masked]@lnisgatt
cc:	 (bcc: Guðbjörn Hreinsson/Skima)
Subject:	Re: Misuse of Content-Disposition (Was: Content-Disposition ancillary)

Just for fun, I did some digging for "content-disposition: inline" in
my April archives.
What I found:

Mozilla 4.51 uses it for forwarded messages - without filename.

Novell Groupwise 5.5 uses them on a SINGLE-part message
message - without filename.

Lotus SMTP MTA (Internal build v4.6.2, and others) uses them on a
single-part message without filename.
A more interesting example is what a Lotus SMTP MTA (probably the same build)
did to a multipart message. Content-disposition inline for multiparts?

Content-type: multipart/mixed;

 Boundary="0__=GT4D6kwm4SS7ntBH6vf0MmbyUG4Vjt0gsOTahntmbzlBMm7VJU8CPx8V"
Content-Disposition: inline
Sender: [email-masked]

--0__=GT4D6kwm4SS7ntBH6vf0MmbyUG4Vjt0gsOTahntmbzlBMm7VJU8CPx8V
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline

Mozilla 4.08 is able to generate the following for an attached GIF:
Content-Disposition: inline; filename="26879.gif"
(in context it made SENSE to have the attachment viewed inline; it's small)

Mulberry (Mac) 1.4.2.1 uses it (without filename) on single-part messages.

Mozilla 4.51 also generated an attachment that *I* wouldn't have had
inline with the following:
Content-Disposition: inline;
 filename="ntpserver.patch"

Some version of HP OpenMail generated the following quite bizarre
combination for a multipart:

Part 1:
Content-Type: application/octet-stream; name="1.txt"
Content-Disposition: attachment; filename="1.txt"
Part 2:
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline; filename="Re:"

I don't know the intent of the sender.

Some version of Mail*Hub (TurboSendmail?) X.400->SMTP gateway added
a "Content-disposition: inline" (no filename) to its single body part.

Finally, just for laughs, this came out of some spammer's engine:

--------------51A269FA31DF
Content-Type: text/html; charset=iso-8859-1; name="2dollars.htm"
Content -Transfer - Encoding: quoted -printable
Content-Disposition: inline; filename="2dollars.htm"
Content-Base: "file:///C|/My%20Documents/Project/2dollars.htm"

==============================================================
Conclusions?????????????????????????

- A lot of stuff will attach "Content-disposition: inline" to single-part
  messages under some, still unknown, conditions.
- Some stuff will inline all body parts, and seems to be consistent in doing
  so; it's not clear that this represents the user's intent, or is being
  honored by the recipients' MUA.
- One fairly rare mailer (sorry, HP) generates "text of message" body parts
  that contain a filename in its "content-disposition: inline" header.

Very unscientific sample, of course; I found 87 messages with
"content-disposition: inline" in my April archive (out of 2.529), and 3 of
them had a filename attached to them.
All the usual caveats apply; this is what msgs looked like when they came
to me.
And I didn't check very hard for counterexamples, either.

                         Harald A

--
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

Date: Tue, 13 Apr 1999 23:12:29 +0200
To: [email-masked]
From: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: Misuse of Content-Disposition (Was: Content-Disposition ancillary)
Sender: [email-masked]

Just for fun, I did some digging for "content-disposition: inline" in
my April archives.
What I found:

Mozilla 4.51 uses it for forwarded messages - without filename.

Novell Groupwise 5.5 uses them on a SINGLE-part message
message - without filename.

Lotus SMTP MTA (Internal build v4.6.2, and others) uses them on a
single-part message without filename.
A more interesting example is what a Lotus SMTP MTA (probably the same build)
did to a multipart message. Content-disposition inline for multiparts?

Content-type: multipart/mixed; 
        Boundary="0__=GT4D6kwm4SS7ntBH6vf0MmbyUG4Vjt0gsOTahntmbzlBMm7VJU8CPx8V"
Content-Disposition: inline
Sender: [email-masked]

--0__=GT4D6kwm4SS7ntBH6vf0MmbyUG4Vjt0gsOTahntmbzlBMm7VJU8CPx8V
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline

Mozilla 4.08 is able to generate the following for an attached GIF:
Content-Disposition: inline; filename="26879.gif"
(in context it made SENSE to have the attachment viewed inline; it's small)

Mulberry (Mac) 1.4.2.1 uses it (without filename) on single-part messages.

Mozilla 4.51 also generated an attachment that *I* wouldn't have had
inline with the following:
Content-Disposition: inline;
 filename="ntpserver.patch"

Some version of HP OpenMail generated the following quite bizarre
combination for a multipart:

Part 1:
Content-Type: application/octet-stream; name="1.txt"
Content-Disposition: attachment; filename="1.txt"
Part 2:
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline; filename="Re:"

I don't know the intent of the sender.

Some version of Mail*Hub (TurboSendmail?) X.400->SMTP gateway added
a "Content-disposition: inline" (no filename) to its single body part.

Finally, just for laughs, this came out of some spammer's engine:

--------------51A269FA31DF
Content-Type: text/html; charset=iso-8859-1; name="2dollars.htm"
Content -Transfer - Encoding: quoted -printable
Content-Disposition: inline; filename="2dollars.htm"
Content-Base: "file:///C|/My%20Documents/Project/2dollars.htm"

==============================================================
Conclusions?????????????????????????

- A lot of stuff will attach "Content-disposition: inline" to single-part
  messages under some, still unknown, conditions.
- Some stuff will inline all body parts, and seems to be consistent in doing
  so; it's not clear that this represents the user's intent, or is being
  honored by the recipients' MUA.
- One fairly rare mailer (sorry, HP) generates "text of message" body parts
  that contain a filename in its "content-disposition: inline" header.

Very unscientific sample, of course; I found 87 messages with
"content-disposition: inline" in my April archive (out of 2.529), and 3 of
them had a filename attached to them.
All the usual caveats apply; this is what msgs looked like when they came
to me.
And I didn't check very hard for counterexamples, either.

                         Harald A

-- 
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

Date: Mon, 12 Apr 1999 09:36:43 -0500
To: John C Klensin <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Misuse of Content-Disposition (Was: Content-Disposition ancillary)
Cc: Harald Tveit Alvestrand <[email-masked]>, [email-masked], Chris Newman <[email-masked]>
Sender: [email-masked]

On 4/12/99 at 9:40 AM -0400, John C Klensin wrote:

>"Ancillary" may not be the appropriate example to induce the 
>discussion that follows...

Indeed, I think "ancillary" has nothing to do with the discussion you 
start below. In fact, I'm not even convinced that it is a particulary 
interesting standards issue, but may be simply an issue of "how do we 
deal with people who can't follow the rules."

I think (and correct me if I've missed the point), your entire 
discussion comes down to this section:

>For example, a number of popular MUAs (the five that I've examined 
>probably collectively include a very large fraction of the desktops 
>in the world among their users) interpret valid but unnecessary 
>content-disposition fields as an indication that the mail is "no 
>message, just attachments".

This seems to simply say that there are certain sending clients out 
there which are generating "Content-Disposition: attachment" for text 
parts which are not intended by the author to be attachments, and 
that this leads to bad behavior on the part of recipients who honor 
the C-D. A few points:

1) Broken sending UAs are broken sending UAs. It is possible to have, 
and we have seen, UAs who send HTML in body parts that are text/plain 
and who send non-US-ASCII text in body parts with incorrect charset 
information. When these arrive at the other end which honors the 
(conformant) labelling, bad things happen. This is not the fault of 
the MIME standard which requires labelling; it's the fault of poorly 
implemented *sending* UAs.

2) There are clearly cases (I make use of them myself) where "no 
message, just attachments" is truly the intention of the sender of 
the message: I want to send a text document to someone who is 
expecting a text document. To put such information into the body of 
the message in my UI simply makes the process of getting it into an 
attachment one step more annoying, just as is the case you cite.

3) This can easily be dealt with by receiving UAs and you should talk 
to your UA writer about dealing with this bug. We've done such things 
before. In this case, we could throw in a piece of code which says, 
"If the message is multipart and the first part is text, display it 
inline regardless of the Content-Disposition." Even better, have this 
as a message by message toggle which you can set for yourself.

>This is not a situation in which those receiving MUAs are 
>non-conformant: their behavior is well within the interpretation 
>flexibilities of content-disposition.  But, from a user standpoint, 
>we have MUA-senders producing conforming content-disposition fields 
>that produce bizarre behavior in conforming MUA-receivers/readers.

You leave out the key factor: These messages are conforming, but sent 
*against the intentions of the message authors*.

>There is a case to be made that the marketplace is beginning to 
>speak clearly to us here and that some radical reconsideration is in 
>order.

I'm waiting patiently to see the long list of complaints sent to the 
bug list for my UA. I don't. I don't even remember seeing one of 
these when last it was my week to do the bug list review. I don't see 
any marketplace speaking clearly to us and no justification 
whatsoever for "radical reconsideration." That seems like utter 
hyperbole on your part, John. There is a serious lack of real 
examples in your message of a widespread number of UAs that are 
sending out this broken Content-Disposition. Until such time as there 
is such a demostration of a real problem, I must insist that this is 
not an IETF issue.

pr
-- 
Pete Resnick <mailto:[email-masked]>
Eudora Engineering - QUALCOMM Incorporated
Ph: (217)337-6377 or (619)651-4478, Fax: (619)651-1102

Date: Mon, 12 Apr 1999 09:40:40 -0400 (Eastern Daylight Time)
From: John C Klensin <[email-masked]>
Subject: Re: Content-Disposition ancillary
To: Harald Tveit Alvestrand <[email-masked]>
Cc: [email-masked], Chris Newman <[email-masked]>
Sender: [email-masked]

On Thu, 08 Apr 1999 00:04:00 +0200 Harald Tveit Alvestrand 
<[email-masked]> wrote:

> I still think that content-disposition is an inferior approach to
> those based on multipart/related; the disposition is part of a bodypart's
> context, not part of the bodypart itself.
> 
> But I recognize that this battle is lost, and support moving ahead with
> this specification; in our current situation, this is the useful thing to do.

Harald,

"Ancillary" may not be the appropriate example to induce the
discussion that follows, but it appears to be time to review the
context in which we are making these decisions and the "bodypart
context" questions themselves.

While I was mostly indifferent to this set of issues when they
first came up, some recent unpleasant experiences with systems
interpreting content-disposition in bizarre ways (see below)
have convinced me that it is time to revisit the issue.  Stated
more strongly, the current pattern of use for
content-disposition can be demonstrated to be causing
interoperability problems; such problems are usually considered
a strong argument for blocking things from progressing to or
beyond Draft and going back to the proverbial drawing board.

Explanation of the problem, for those who haven't seen it or
understood the symptoms (sorry, but this is going to be a little
long):

MIME was designed very carefully to interoperate to the extent
possible with MIME-ignorant and MIME-weak MUAs.  The main
failure in that regard was anticipated and accepted, i.e., that
encoding message components for safe transport would make them
difficult or impossible to read with MIME-ignorant (and really
stupid) readers, e.g., it was generally understood that
multimedia, multi-character-set, MIME mail would be the end of
convenient mail reading with "cat".  But, the encoding issue
aside, the intent was that relatively simple MIME messages
appear in useful and comprehensible form to a MUA-reader that
lacked extensive MIME capabilities (even if some useless
protocol chatter was inflicted on the user).

One feature of MIME that has turned out in retrospect to be
radical was that it changes the then-common metaphor of a
"message" with "attachments" to a "multiple body parts" model.
While many MUAs have been "taught" (more or less) to read MIME,
the "message and attachments" metaphor has persisted, both in
old MUAs that have been upgraded and in newly-developed ones.
In the latter cases (at least), that metaphor presumably
persists because users find it comfortable.  If their designers
are correct, we may have done a reasonable protocol job, but
slipped up on the UI implications.

Now content-disposition was, to some extent, supposed to solve
the "message and attachments" problem by permitting that model
to be specified by the sender in the context of the much more
general "body part" system.  In practice, it has often made it
worse. It appears to me that MUA designers have found it just
too painful to elicit the right information from users and that
their attempts to figure things out heuristically have either
failed outright or created a level of complexity that leads to
more subtle failures.  

For example, a number of popular MUAs (the five that I've
examined probably collectively include a very large fraction of
the desktops in the world among their users) interpret valid but
unnecessary content-disposition fields as an indication that the
mail is "no message, just attachments".  All of them are
sufficiently attachment-competent that the user can open the
text attachment (what the sender intended to be the message
body) without much difficulty.  But, for most of them, that
requires an extra step for the user, which causes irritation.
Worse, attempting to reply to the message and include some of
the original text ranges in difficulty from "time consuming and
irritating" to "nearly impossible" and facilities in those MUAs
for quoting, etc., usually are not available.

This is not a situation in which those receiving MUAs are
non-conformant: their behavior is well within the interpretation
flexibilities of content-disposition.  But, from a user
standpoint, we have MUA-senders producing conforming
content-disposition fields that produce bizarre behavior in
conforming MUA-receivers/readers.

If we did this stuff (or parts of it) with multipart/related,
the (IMO very clear) "what to do with multipart when you can't
figure out what is going on" rules would protect us.  The worst
case would be that the user would see the disposition protocol
elements, not that various hidden mechanisms would essentially
foul up the message.

Of course, an alternative might be much more detail, and
conformance rules that contain a lot of "MUST"s, about just
exactly where and how content-disposition is to be used.  We
haven't had much success with that type of approach lately, but
it is an alternative.  Of course, going that way probably would
require simplifying content-disposition considerably, which
might impact "ancillary" and other debatable add-ons.

There is a case to be made that the marketplace is beginning to
speak clearly to us here and that some radical reconsideration
is in order.

    john

Date: Fri, 09 Apr 1999 16:02:18 -0700 (PDT)
From: Chris Newman <[email-masked]>
Subject: Re: RFC 822/MIME/DSN/MDN "Message Lint"
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: [email-masked]
Sender: [email-masked]

On Thu, 8 Apr 1999, D. J. Bernstein wrote:
> msglint barfs on the comments in the example on page 7 of RFC 822.

Thank you, there was a subtle bug.  It's fixed now (V0.95).

		- Chris

Date: 8 Apr 1999 23:14:46 -0000
Mail-Followup-To: [email-masked]
From: "D. J. Bernstein" <[email-masked]>
To: [email-masked]
Subject: Re: RFC 822/MIME/DSN/MDN "Message Lint"
Sender: [email-masked]

msglint barfs on the comments in the example on page 7 of RFC 822.

Is a broken validator better than no validator at all?

---Dan

Date: Thu, 08 Apr 1999 13:27:52 -0700 (PDT)
From: Chris Newman <[email-masked]>
Subject: RFC 822/MIME/DSN/MDN "Message Lint"
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: [email-masked]
Sender: [email-masked]

I have a utility available which takes a full MIME email message and
performs a number of standards compliance and related tests to the
message.  You can use this to test if your email client or RFC 822/MIME
generator is correct.

You can either go to the web page:
 
   <http://www2.innosoft.com/~chris/msglint.html>
 
where you can paste a message into a form and get the results, or you can
send email to:
 
   <[email-masked]>
 
And you will get the results in an auto-reply message (to the envelope
sender address).

		- Chris

Date: Wed, 07 Apr 1999 17:50:12 -0700 (PDT)
From: Chris Newman <[email-masked]>
Subject: Re: Content-Disposition ancillary
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: Harald Tveit Alvestrand <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

On Thu, 8 Apr 1999, Harald Tveit Alvestrand wrote:
> I still think that content-disposition is an inferior approach to
> those based on multipart/related; the disposition is part of a bodypart's
> context, not part of the bodypart itself.

I really think they're solving two different problems.

multipart/related, multipart/appledouble, multipart/report,
multipart/signed and their ilk all create a "context" which takes
precedence over the Content-Disposition header iff the receiving client is
aware of the semantics of the multipart (which in the case of
multipart/related includes the semantics of the start body part).

Content-Disposition in general and ancillary in particular are the
fallback position if the receiving client doesn't understand the multipart
context (or if it's multipart/mixed, so there's no semantics in the
multipart context).

It might be appropriate to add some discussion along these lines to the
next revision of the Content-Disposition spec.

I want to see ancillary put on things like vcards, ms-tnef and similar
cruft so clients know to make them unobtrusive.  But it also has value
when the next multipart/foo is introduced and clients aren't familiar with
it, or when the new application/whizbang type is used as the base for 
multipart/related.

> But I recognize that this battle is lost, and support moving ahead with
> this specification; in our current situation, this is the useful thing to do.

I'm not sure I recognize a battle.  They seem like complimentary services
to me.

		- Chris

Date: Thu, 08 Apr 1999 00:04:00 +0200
To: Chris Newman <[email-masked]>, [email-masked]
From: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: Content-Disposition ancillary
Sender: [email-masked]

At 11:03 07.04.99 -0700, Chris Newman wrote:
>I'd like to move the "ancillary" disposition type (formerly
>"metadata") to proposed status.  This was discussed at the mailrev BOF. 
>Any final comments on the specification?
>
><ftp://ftp.ietf.org/internet-drafts/draft-newman-mime-cdisp-metadata-02.txt>
>
>		- Chris

I still think that content-disposition is an inferior approach to
those based on multipart/related; the disposition is part of a bodypart's
context, not part of the bodypart itself.

But I recognize that this battle is lost, and support moving ahead with
this specification; in our current situation, this is the useful thing to do.

Go ahead!

                           Harald A

-- 
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

Date: Wed, 07 Apr 1999 11:03:13 -0700 (PDT)
From: Chris Newman <[email-masked]>
Subject: Content-Disposition ancillary
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: [email-masked]
Sender: [email-masked]

I'd like to move the "ancillary" disposition type (formerly
"metadata") to proposed status.  This was discussed at the mailrev BOF. 
Any final comments on the specification?

<ftp://ftp.ietf.org/internet-drafts/draft-newman-mime-cdisp-metadata-02.txt>

		- Chris

To: [email-masked]
Xref: clerew local.mime:178
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Signed headers in email (was Re: Draft for signed headers)
Date: Thu, 1 Apr 1999 14:58:31 GMT
Lines: 23
Sender: [email-masked]

In <[email-masked]> Chris Newman <[email-masked]> writes:

>I skimmed your proposal and it was both too complex and had ambiguities. 
>The obvious way to simplify it is to stop worrying about email-safety. 
>It's not worth pointing out the ambiguities until the algorithm is
>sufficiently simple to be viable.

Fair enough! The ambiguities are fixable (that was just a first draft).

I think I would now want to take out the bit about removing '"' only
outside matching <...> and just remove them either regardless (except when
preceded by '\') or else not at all (but I worry about agents that create
unnecessary quoted-strings "just to be safe"). I don't see that the rest
is over-complicated, and I have in any case offered to write a model Perl
script for an appendix, which might indeed show up unexpected
difficulties.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5