Subject: provisional registration: Archived-At for email
From: Martin Duerst <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Date: Mon, 01 Nov 2004 13:51:59 +0900
List-Id: <ietf-822.imc.org>

This is a proposal for a provisional registration of the Archived-At
header field for email, sent here for the two-week review according
to RFC 3864.

    Header field name:
       Archived-At

    Applicable protocol:
       mail (RFC 2822)

    Status:
       provisional

    Author/Change controller:
       Martin Duerst, W3C/Keio University, [email-masked]

    Specification document(s):
       http://www.w3.org/2004/10/archived-at/draft-duerst-archived-at-02.txt

    Related information:
       It is planned to advance this registration to
       permanent by advancing the internet-draft to an RFC.
       The syntax of the header field may change.

Regards,    Martin. 

Cc: ietf-822 <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Sun, 31 Oct 2004 18:06:12 -0500
To: Bruce Lilly <[email-masked]>, =?ISO-8859-1?Q?Martin_D=FCrst?= <[email-masked]>
Sender: [email-masked]

>> I don't see that this is a showstopper for archived-at.
>
> Probably not, but as the issue has been identified (i.e. it is a
> known technical issue), it should be addressed properly rather
> than being swept under the rug, especially if the draft is intended
> to be Standards Track ("no known technical omissions").

agreed.

Keith

Cc: ietf-822 <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Sun, 31 Oct 2004 18:05:09 -0500
To: Bruce Lilly <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

> We can distinguish general "computer programs" from those
> that specifically handle RFC [2]822 messages. While it certainly
> seem to be true that it's desirable for the URI conveyed to be
> usable to *some* computer programs (viz. URI handlers, a.k.a.
> browsers), it is not necessary for it to be part of some structure
> which is handled by RFC [2]822 message-handling programs.
> I.e. the URI can be conveyed in an unstructured field as far as
> RFCs [2]822 are concerned, while still being a URI.

yes it can.  but as soon as we might want MUAs to be able to use those 
URIs for specific purposes (like accessing archives from within the 
mail client) and to associate other information about those URIs (like 
a description of the format of the archive) then it becomes useful to 
have archived-at be a structured field.

Keith

Cc: =?ISO-8859-1?Q?Martin_D=FCrst?= <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Sun, 31 Oct 2004 18:03:21 -0500
To: ietf-822 <[email-masked]>
Sender: [email-masked]

>> 2. For archived-at to be useful by email clients generally requires
>>    one or two of the following, in addition to the obvious client
>>    support for the protocol and server support for the message format:
>>
>>    - mail archives that support IMAP access (or possibly NNTP)
>>    - a specification for making collections of mail messages available
>>      via HTTP (maybe WebDav) and/or FTP
>>    - mail archives that follow the aforementioned specification
>
> I don't think that the second item (http/ftp) is going to be
> generally useful for email clients.  Perhaps for combined
> MUA/browsers, but at most there seems to be only a loose
> connection between the browsers and MUA portions of those
> (see below re. Mozilla).

having implemented HTTP and FTP client libs from scratch in a few 
hundred lines of code each, I don't see either of these as onerous.  
and there's an advantage in that for HTTP and FTP, "anonymous" access 
is widely supported by servers.

> I believe a great deal more is needed. I've mentioned lack of
> URI support in MUAs and limited support of the most useful
> URIs (imap) in browsers. Some details:

thanks for looking into this.  none of what you found surprises me.  I 
didn't expect that many existing MUAs would be able to take advantage 
of URIs pointing to archives without some upgrade.  I do, however, 
suspect that it's feasable to upgrade an MUA that already supports IMAP 
to point to an IMAP archive of rfc822 messages, or even to upgrade an 
MUA to be able to read an archive of 822 messages using FTP or 
HTTP+WebDav.

right now I'm thinking in terms of writing an Informational or 
Experimental RFC called "Recommendations for Net-Accessible Mail 
Archives".  It would describe how to set up archives, and also describe 
how to use Archived-At to point to those archives.  I'm thinking that 
archives should be based on maildir at the server end, though for the 
sake of efficiency I'm wondering about options to use mbox format 
(yeech) and compression, and also how to avoid having several years' 
worth of archives in a single directory or file while still making the 
whole archive appear seamless to the client.  Offhand it looks like FTP 
is probably the most straightforward access method to implement, but 
that it should be possible to define reasonable access via other 
protocols also.

Keith

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Sun, 31 Oct 2004 17:26:36 -0500
Cc: Keith Moore <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Sender: [email-masked]

On Thu October 28 2004 16:38, Keith Moore wrote:

> 2. For archived-at to be useful by email clients generally requires
>    one or two of the following, in addition to the obvious client
>    support for the protocol and server support for the message format:
> 
>    - mail archives that support IMAP access (or possibly NNTP) 
>    - a specification for making collections of mail messages available
>      via HTTP (maybe WebDav) and/or FTP
>    - mail archives that follow the aforementioned specification

I don't think that the second item (http/ftp) is going to be
generally useful for email clients.  Perhaps for combined
MUA/browsers, but at most there seems to be only a loose
connection between the browsers and MUA portions of those
(see below re. Mozilla).

I believe a great deal more is needed. I've mentioned lack of
URI support in MUAs and limited support of the most useful
URIs (imap) in browsers. Some details:

I sent several imap URIs to myself in the body of a text message
and opened the message in several MUAs.  Only one
(Mozilla 1.7.3) recognized the URIs as imap URIs, and it
does nothing itself with imap URIs.  The URIs could be
opened in the companion (Mozilla) browser by right-clicking
on the URI in the MUA and making a selection.  However the
Mozilla broswer just sits there dumb when handed an imap
URI -- no result, no dialog, just a "Done" status line. Two
MUAs tested (Kmail and Sylpheed) misinterpreted the URIs
as if they were mailto URIs, which is pointless and useless.
One MUA simply displayed the URIs as text (not unreasonable);
that was Ximian Evolution.

The Opera browser appears not to support the imap scheme
in URIs; it pops up an error message complaining about an
unrecognized "address".

The Konqueror browser, as mentioned in February, does seem
to have limited imap URI support.  It could handle the (pasted)
URIs, and would display folders and list folder contets. However
the messages were shown with a "type" of "Unknown". Right-
clicking allowed selecting an application to handle the message;
handling consists of copying the message to a local temporary
file, invoking the application with an attempt to open the
temporary file, then deleting the temporary file when the
selected application terminates.  None of the usual MUA
applications seemed to work with Konqueror (Pine, Kmail,
etc., however simple text editors (e.g. gvim) seemed to work);
Konqueror had already deleted the temporary file by the time
the MUAs had started up, resulting in a "file not found"
error dialog. That's not terribly useful.

The best that seems to be possible at the moment is the
following kludge: Copy the URI text, open a Konqueror
browser window, paste the URI there, edit out any
message uid (remember, Konqueror doesn't handle the
content), memorizing that uid for future use, supply any
password that may be required, select the appropriate
message from the folder listing (via the memorized uid),
right-click, select a text editor to handle the message,
save a copy of the message somewhere from the text
editor, quit the editor, open the saved copy of the
message in an MUA to reply or whatever.

There certainly seems to be a great deal that would have to
be done beyond merely making archives available via IMAP
to make access to archived massages readily available
in native message form to MUAs via imap URIs. 

From: Bruce Lilly <[email-masked]>
Reply-To: Bruce Lilly <[email-masked]>, Martin Duerst <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Sun, 31 Oct 2004 16:33:44 -0500
Cc: Keith Moore <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

On Fri October 29 2004 11:08, Keith Moore wrote:
> > Structured fields (as far as RFC 2822's definition of structured
> > vs. unstructured fields is concerned) are primarily used by
> > message handling programs (not only clients, but e.g. IMAP
> > servers, MTAs, Sieve filtering language (RFC 3028), etc.).
> > Fields intended to be used only by humans are generally
> > unstructured (again, in the RFC 2822 sense), e.g. Subject,
> > Comments, Content-Description.  
> 
> Yes, this _tends_ to be the case, because fields designed to be parsed
> by computer programs need to be structured, while fields designed
> primarily for human consumption have less need to be structured.
> However it's not a rule, it's just the way things usually work.
> 
> In practice it's often the case that we want a field to be both
> human readable and usable by computer programs.  I think that would
> have been true for Archived-At even if we had started out with
> the understanding that it was originally intended for human use.

We can distinguish general "computer programs" from those
that specifically handle RFC [2]822 messages. While it certainly
seem to be true that it's desirable for the URI conveyed to be
usable to *some* computer programs (viz. URI handlers, a.k.a.
browsers), it is not necessary for it to be part of some structure
which is handled by RFC [2]822 message-handling programs.
I.e. the URI can be conveyed in an unstructured field as far as
RFCs [2]822 are concerned, while still being a URI.

Since there is no defined role for message-handling programs,
the field body plays no role in message protocols.  If the URI
is used by humans for pasting into other programs (browsers),
then as far as messages and massage-handling programs are
concerned, it plays no role in any message-related protocol.
It is a convention, rather than a protocol.  As a convention,
it doesn't matter whether the URI is conveyed in an unstructured
field, as a comment in some specific structured field, as body
content, embedded in some existing field, etc., so long as the
particular convention is understood by interested parties.

From: Bruce Lilly <[email-masked]>
Reply-To: Bruce Lilly <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Sun, 31 Oct 2004 15:50:30 -0500
Cc: Keith Moore <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Sender: [email-masked]

On Sat October 30 2004 16:17, Keith Moore wrote:

> a) specify that Archived-At is one of those fields that should be taken 
> from the encapsulated message header,

Agreed (I think I suggested as much).  However, as noted earlier,
existing implementations of message/partial fragmentation and
reassembly will continue to use the existing RFC 2046 rules
(possibly as amended by RFC 2298 for recent implementations).
As might be expected, there are interoperability considerations;
for example a fragmented message prepared with amended
rules, but reassembled by a legacy implementation would lose
all Archived-At fields. And vice versa. [As an aside, that seems
to be an inherent problem with the message/partial rules, as
discussed on ietf-822 in the past.  I think it may be possible to
specify fragmentation/reassembly rules that do not have those
problems, but I'm not yet ready to make a detailed proposal.
Which would be orthogonal to the current discussion.]

> and perhaps 
> b) specify that Archived-At should not be used to point to archives of 
> message/partial

Sounds reasonable, at least as a recommendation (I think
RFC 2119 "MUST NOT" would be too strong).

> or
> c) specify that Archived-At fields should be removed from any message 
> that is fragmented

That's possible, but would of course remove any corresponding
functionality. It would call for explanation of who should remove
the field, etc. [As noted above, removal may indeed result as a
side-effect of fragmentation/reassembly rule interaction.]
 
> I don't see that this is a showstopper for archived-at.

Probably not, but as the issue has been identified (i.e. it is a
known technical issue), it should be addressed properly rather
than being swept under the rug, especially if the draft is intended
to be Standards Track ("no known technical omissions").

Date: Sun, 31 Oct 2004 16:54:30 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 <[email-masked]>
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 <[email-masked]>
Sender: [email-masked]

<snip nontechnical discussion>
(I'm willing to respond to those points if you insist, but they
seem to me to be matters of opinion which bring nothing of value)

On Oct 30 2004, Bruce Lilly wrote:

> > That's the whole point of the discussion we're having. If you know how
> > to successfully forge the Processed field as I originally described it
> > and its variations as already discussed with other list members, please
> > give a concrete scenario.
> 
> Given a message with a valid Received field; it's trivial; simply
> construct a Processed field accordingly and insert it where
> desired, eliding any preexisting "processed" field that corresponds
> to the Received field.  W/o any Received field; forge one and
> insert it and a corresponding "processed" field where desired.

I'm afraid none of this is yet convincing as a successful attack. 

You must be able to forge the Processed field before the message
contains the pertinent Received field, otherwise you're not actually
forging anything of value. In fact, quoting existing Received fields
is legal, and marks your location correctly. 

If you forge an appropriate Received field as well, you still have no
control over the Received fields added by the subsequent receiving
systems, which will always have precedence. Nor can you assume that
the ultimate reader will trust your forged Received field, whence your
associated Processed field.  So your forgery will be trivial to detect
at the destination, unless its trust policy is very permissive.
That's not much of an attack as it stands, if you rely on cooperation
from the victim.

If you simply rewrite an existing, presumably syntactically valid,
Processed field, then you are at the same point as in the previous
paragraph. Alternatively, it means that you have found a way to
compromise a notionally trusted mail system close to the
destination. In those cases, there are much more interesting things to
do than forge a Processed field.

I don't yet see any actual effective attack in any of your suggestions,
but see below.
 
> Bear in mind the fact that not all message transfer involved SMTP,
> but that inly SMTP requires inserting a Received field (and
> moreover, there is no enforcement of that requirement).

That's a good point. I believe the Processed field degrades gracefully
in this respect, since if there is no existing Received field to be
quoted, a Processed field cannot quote it. A later reader of this
Processed field will be immediately alerted to the fact by the absence
of an "auth-received" key value.

> > Neither does the Processed field. Its keyword/value pairs will have
> > standard meanings which can be interpreted without the need for
> > configuration information.
> 
> Not so.  Given your example
>    Processed: name="SpamAssassin"; location-ip="1.2.3.4";
>        version="2.63"; function="spamcheck";
>        auth-received="B17C07174B5E4546A2B04EB096E83FD7081936B8"; 
>        result-tag="spam";
> I cannot interpret (in any meaningful sense) the "result-tag='spam'"
> part without knowing specific configuration information and run-time
> environment information for the instance of "SpamAssassin" version
> "2.63" from which the purported "result-tag" was allegedly obtained.
> I know nothing about its spam threshold, the myriad of fudge factors
> that are used to weigh various characteristics, whether any address
> patterns are used as whitelists or blacklists and if so what those
> patterns are, etc.

This makes no sense as an objection. The Processed field is a record
of what some other process computed about the message, it has no
other meaning.

All you need for interpretion is to know what this other process
intended the fields to mean, and that's to be detailed in a common
standard.

As long as processes follow the standard, the meaning will be clear,
viz.  "the process writing this field considers the message to be
spam". There is no need to second-guess why, how etc. It's a record
of some other process' computation, for informational purposes. If that
process wants to express other characteristics, it will do so with 
appropriate key/values.

> > You're sidestepping the issue. I've given an example (Reply-To) where
> > you (and I and we all) act based on zero hard facts about the writer
> > of the Reply-To field. If we can act on unreliable fields such as
> > Reply-To, why can't we act on fields such as Processed? What's the
> > fundamental difference you perceive and are pointing out?
> 
> The fundamental difference is that the Reply-To field is specifically
> provided as a means for the message author(s) to suggest a list
> of mailboxes for responses. "Acting on" a Reply-To field presupposes
> that one has already decided to make a response (prior to consulting
> the field content) and therefore one already presumably already
> knows *why* he is responding and therefore has some idea of where
> his response should be directed. He can choose to respond directly
> to the author(s), completely ignoring Reply-To, he can choose to
> send his response to some specific place without consulting Reply-To,
> he can choose any number of places to direct his response, one of
> which is the author's suggestion.  I.e. one makes a decision (to
> respond), another decision (where to respond), and in a subset of
> cases, might subsequently consult a Reply-To field which conveyed
> the original message author's suggestion; if one does not choose
> to respond, the presence, absence, or content of any Reply-To field
> is simply irrelevant, and in any event is not involved in the decision
> *to* respond or not.  You are proposing that some decision be
> based on incomplete and inadequate information contained in a
> "processed" field.   I.e. the field must exist and must have some
> content before the relevant decision (a guess, really) can be made.

If I may borrow your language: the Processed field is specifically
provided as a means for filter processes to suggest facets of the 
message as observed during transit. 

A subsequent message reader can choose to file the message away,
completely ignoring any Processed fields, he can choose to file
the message away, taking as a suggested destination some expression
derived from the contents of a Processed field, or offering several
possible destinations if there are several Processed fields, etc.

There is nothing inherently in a Processed field which mandates 
particular decisions to be taken by any subsequent readers. It is 
strictly informational.

> You are proposing that some decision be based on incomplete and
> inadequate information contained in a "processed" field.  I.e. the
> field must exist and must have some content before the relevant
> decision (a guess, really) can be made.

I'm sorry if I gave that impression. I merely intended to point out that 
decisions are made all the time by humans and machines. If an information
bearing field such as Processed exists, somewhere sometime it will be 
used for decision making. That does not mean that Processed would have any
authority whatsoever, beyond that which readers individually place into it
of their own will.

There are a multitude of X- headers used by filters, each with its own
syntax and interpretations, which are currently unparseable by all the
other filters.  For the subset of spam filters, a commonly adopted
format will allow better filter cooperation, and reduce the need for
future filters to reinvent variations on the same format.

> > > Your proposed "processed" field differs from all of the above;
> > > unlike Return-Path and Received trace fields, its content is
> > > not well-defined.  
> > 
> > Of course not, you'll have to wait for a proper draft.
> 
> You have missed the point.  A Received line "from" component
> documents the domain name used by a client in an SMTP
> transaction. That documents a fact about what was used. It
> is not a value judgment and is in no way dependent upon
> the configuration of the MTA that documents that fact. That
> is the sense in which the existing trace field content is
> "well-defined", and is not the case for your "result-tag" etc.
> component.

I still don't see it. To paraphrase your request for a complete
record of the filter configuration: the Received line "from" component
is useless unless I have complete information about the state of the
DNS system in the world at the moment of the transaction, as well
as a complete record of which IP addresses are used in the world
at that point in time. So in this sense, the "from" component is
simply not well defined due to lack of information. 

-- 
Laird Breyer.

Cc: "ietf-822" <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Sat, 30 Oct 2004 16:17:38 -0400
To: Bruce Lilly <[email-masked]>, =?ISO-8859-1?Q?Martin_D=FCrst?= <[email-masked]>
Sender: [email-masked]

> Clearly, the proposed Archived-At field applies to a specific message
> (like Message-ID), and should be in the same category as those
> specifically mentioned.  However, unlike RFC 3798 and its
> predecessor, the archived-at draft does not amend the RFC2046
> list of fields which are reassembled from the encapsulated message
> header.  Consequently, Archived-At would be treated like Comments;
> when fragmenting a message, the field would be copied to the
> first fragment message header, and would be reassembled from there
> to the reconstituted facsimile of the original unfragmented
> message (any such fields within the encapsulated message header
> would be ignored, in accordance with the RFC 2046 rules). That
> is inconsistent with the semantics proposed for Archived-At, viz. that
> it references an archived copy of the specific message in which it
> is conveyed.  An Archived-At field that originally referred to an
> archive of an unfragmented message would therefore be
> inappropriately placed in the message header of a different message
> (viz. the first fragment message).  Moreover, it would be
> indistinguishable there from an appropriate Archived-At field which
> referred to an archive of that (first fragment) message.  And
> as fragments may be further fragmented, that issue may be
> compounded.

seems like an easy problem to fix:

a) specify that Archived-At is one of those fields that should be taken 
from the encapsulated message header, and perhaps
b) specify that Archived-At should not be used to point to archives of 
message/partial
or
c) specify that Archived-At fields should be removed from any message 
that is fragmented

I don't see that this is a showstopper for archived-at.  even if 
archived-at and message/partial were completely incompatible, 
archived-at (in some form) seems a lot more valuable than 
message/partial.

Keith

From: Bruce Lilly <[email-masked]>
Reply-To: Bruce Lilly <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Sat, 30 Oct 2004 13:05:08 -0400
Cc: Keith Moore <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Sender: [email-masked]

On Thu October 28 2004 19:54, Keith Moore wrote:

> >   A message may contain an arbitrary
> >    number of Comments fields.  However, it does not solve the
> >    problem of the incompatibility with RFC 2046 message/partial.
> 
> could you explain that supposed incompatibility again?

Fragmentation/reassembly is defined in RFC 2046 section 5.2.2 and
its subsections.  The issue was discussed in some detail on the ietf-822
mailing list in June 2003 and again very briefly regarding specific
field (References, In-Reply-To, and Resent-Message-ID) in September
2004.  Briefly, the fragmentation/reassembly process distributes
original (unfragmented) message header fields between the header
of the first fragment and the header of the encapsulated message
(which appears within the message/partial wrapper in the first
fragment; reassembly combines fields from those two headers to
produce a facsimile of the original message from fragment messages.
RFC 2046 lists specific fields which are taken from the encapsulated
header -- MIME Content- fields, Subject, Message-ID, Encrypted,
and MIME-version -- that list is amended by RFCs 2298/3798 to
include Disposition-Notification-To, Disposition-Notification-Options,
and Original-Recipient; these fields are the ones which pertain
specifically  to the unfragmented original message as opposed to
any fragment.

Clearly, the proposed Archived-At field applies to a specific message
(like Message-ID), and should be in the same category as those
specifically mentioned.  However, unlike RFC 3798 and its
predecessor, the archived-at draft does not amend the RFC2046
list of fields which are reassembled from the encapsulated message
header.  Consequently, Archived-At would be treated like Comments;
when fragmenting a message, the field would be copied to the
first fragment message header, and would be reassembled from there
to the reconstituted facsimile of the original unfragmented
message (any such fields within the encapsulated message header
would be ignored, in accordance with the RFC 2046 rules). That
is inconsistent with the semantics proposed for Archived-At, viz. that
it references an archived copy of the specific message in which it
is conveyed.  An Archived-At field that originally referred to an
archive of an unfragmented message would therefore be
inappropriately placed in the message header of a different message
(viz. the first fragment message).  Moreover, it would be
indistinguishable there from an appropriate Archived-At field which
referred to an archive of that (first fragment) message.  And
as fragments may be further fragmented, that issue may be
compounded.

> not that I'm  
> really worried about it as (a) I haven't seen a message/partial since 
> the early 1990s (even then it was only as a test)

Hey, good for you; that's one data point.  It is a valid media type
and corresponding mechanism which is in use (Ned reported having
seen just over one hundred fragmented messages in a year when
the issue was discussed on the ietf-822 list).  Unless one is operating
a human-powered gateway, it is not surprising that one would not
see fragments, as fragmented messages are supposed to be
reassembled by gateways (RFC 1344) before being forwarded (see
also
http://lists.sans.org/pipermail/list/2002-September/054073.html
and the CERT advisory which can be found by following links for
issues that arise when careless or ignorant vendors of gateway
products fail to reassemble fragmented messages).

> and (b) somehow I  
> doubt that most MUAs can reassemble message/partials anyway.

"Most MUAs" do not handle one or more aspects of messaging well.
Many MUAs, however, can reassemble fragmented messages. And
as noted above, fragmented messages are supposed to be
reassembled before reaching an MUA if there is a gateway (e.g.
spam- or virus-scanner) involved; MUAs are not the only things
that handle and/or process messages.

> I don't see why in the world an MUA would download the URI in an 
> Archived-At field, because it already has the message on hand.

Neither do I, and that's why I raised the issue; if the field is in
fact intended solely for human use, then we've been going
down the wrong track for 8 months discussing syntax issues
for a structured field.

> (no, I  
> don't consider message/partial a threat in this regard).

The security threat is not directly related to massage/partial; it
arises due to external references.
 
> > An imap scheme certainly indicates that the format is message/rfc822
> 
> I've certainly seen IMAP servers that would provide arbitrary contents. 
>    Whether that was a bug or a feature, I was never sure.

Let's be clear about the issue; the IMAP protocol as implemented by
IMAP servers is capable of returning message sections or metadata
in response to specific client queries.  The IMAP URI scheme (RFC
2192)  can also access some of those objects.  Note that it is
clear from the IMAP URI syntax what is being accessed (RFC 2192
section 2).

> > Accessing such an archive requires access credentials, which
> > would have to be communicated (probably possible within
> > the URI).
> 
> IIRC, IMAP was specifically designed to be able to provide read-only 
> access to archives.  But it's been awhile.

Read-only vs. read-write authorization is intimately connected
with access credentials in IMAP.  It is possible to permit anonymous
access, and the default user name is in fact "anonymous".
Other user names can be supplied in the URI, or interactively
from client/user dialog.  I'm not sure if all schemes have similar
capability.

> >>    - a specification for making collections of mail messages available
> >>      via HTTP (maybe WebDav) and/or FTP
> >
> > multipart/digest is an existing media type, which can be
> > transferred via HTTP and/or ftp, which *is* a collection
> > of messages.
> 
> yup, but as an archive format, it's pretty pessimal.

I think we were discussing transfer specifications/formats, not
necessarily what's stored on the archive server.

> you can't easily  
> locate a particular message within the file

An MUA implementation issue. Some MUAs *can* locate
particular messages in a message/digest collection.

> , nor can you append new  
> messages to the file.

I'm not sure I'd want random persons to be able to modify
existing archive files -- that defeats one of the purposes of
having an archive.  That doesn't prevent messages being
added to the archive; serving the collection as message/digest
would serve a snapshot of the collection as it stands at the
time of the request.

> >>    and as much as I'd like to believe that email client vendors would
> >>    enthusiastically add support for these, market conditions don't 
> >> seem
> >>    to favor supporting new functionality standards in email clients
> >>    right now.

[incidentally, I'm not sure that I buy the "market conditions" line;
there are open source clients that continually add feature support --
what you say might be true for *commercial* clients, but not all
clients fall into that category]

> > There may be a more fundamental issue: IMAP (or NNTP) access
> > requires either handling a URI or configuration of the necessary
> > parameters which would be conveyed in a URI (host name, port,
> > user credentials).  Certainly MUAs that provide IMAP client
> > functionality could be used without any additional design, but
> > the user would need to get the parameters from somewhere and
> > configure his MUA(s) accordingly.  MUAs don't typically handle
> > generic URIs directly (mailto URIs are a notable exception), but
> > instead may hand them off to a browser (which in turn might
> > invoke an MUA for an imap scheme URI).
> 
> Parsing an IMAP URI is not rocket science, and doesn't need more than a 
> couple of hundred lines of code.  Somehow I don't think this is the 
> limiting factor.

Show me an MUA that can access a collection of existing
messages by typing (or pasting) a URI (as opposed to
entering configuration information in a file or via some
GUI dialog).  Now browsers do tend to have URI support,
and back in February I pointed to an example of a browser
with IMAP URI support; but browsers aren't MUAs.  Merely
parsing a URI is an insignificant part of what needs to be
done to utilize a URI, much less configure IMAP access.

> (Actually I think that all MUAs should allow their mailboxes and 
> submission servers to be specified using URLs - it's much simpler and 
> less error-prone than giving users lots of blanks with misleading 
> labels to fill in.)

Simpler for whom?  A casual user who has no idea what a URL
is or how to construct one?  My experience with a particular MUA
which required a specific type of URI for a particular piece of
configuration (without specifying that it in fact needed a
URI) was rather annoying.

> > I think the issue of a list of URIs might be revisited (and not
> > necessarily in a structured field), so that multiple access
> > methods to a given message could be specified (that does
> > not preclude multiple fields to indicate multiple places where
> > copies may be archived, except of course if comments in the
> > Message-ID field are used).
> 
> I don't see the point in specifying multiple ways to do the same thing. 

The point is to enable an archive which has multiple access
schemes (e.g. http, ftp, IMAP, POP, NNTP, mailto) to be able
to provide pointers for that multiplicity of schemes for accessing
a particular archived message, distinct from the issue of
multiple separate archives.  If you don't care which access
method is used, pick one at random -- but it sure sounds
like you do care, judging by your comments about HTML
archives.

>   There's a good argument for allowing multiple Archived-At fields 
> because the originator, the recipient's MTA, and any number of 
> intermediaries might archive a message - it's much cleaner to just let 
> each of them add a separate field.  Given that I don't see any need to 
> have multiple URIs within a field also.

See above.
 
> as for RFC 2231, I agree that they  
> can get ugly, but I think this will be rare.

Ugliness isn't the issue; the issue is that given that specification,
there needs to be code to handle what the specification
requires.  Now of course, it is possible to come up with yet another
specification, but then that requires yet another mass of code.

> and the MUA ought to have  
> code to handle RFC 2231 anyway.

In the specific places where it is specified (Content-Type and
Content-Disposition), yes.   That is not necessarily available
for use in arbitrary fields, and cannot be applied to all fields
(there are fields with syntax that is not amenable to 2231
parameters, e.g. unstructured fields).

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From: Bruce Lilly <[email-masked]>
Reply-To: Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>, ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>, "ietf-822" <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Sat, 30 Oct 2004 14:27:34 -0400
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Fri October 29 2004 03:42, Martin Duerst wrote:

> Yes. One issue is that if there are a lot of comments fields,
> the user will have difficulties to choose the right one.

The same could be said of archived-at and x-archived-at.

>  > However, it does not solve the
>  >   problem of the incompatibility with RFC 2046 message/partial.
> 
> As far as I'm concerned, Keith's description of the (non-)deployment
> of this feature has pretty much solved this issue for me.

Keith did not say that the feature was not deployed, he said that
he could not recall having seen individual fragments recently.

>  >2. Message-ID:
>  >      <[email-masked]>
>  >      (archived at http://www.imc.org/ietf-822/mail-archive/msg05043.html)
>  >  Parenthesized comments are provided for in Message-ID fields.
>  >  This method puts the archive information in the same place as
>  >  the message identifier,
> 
> But Message-ID wouldn't usually be shown by email clients, or would it?

That's an MUA implementation issue. Not incidentally, using
a comment associated with msg-id provides for additional
functionality (consider the comments remaining associated
with msg-ids in In-Reply-To and References fields).

> Also, I'm concerned about the length of the field. Having separate
> fields with reasonably short length seems better than having long
> fields squeezing everything in.

Why?  Overall, there is less bulk w/o the additional CRLFs and
"Archived-At:".  Fields may be folded; field length is not an
issue.

>  >  and is fully compatible with message/partial
>  >  fragmentation and reassembly.
> 
> Again, as shown by Keith, a non-issue.

Again, you're reading too much into his remarks.

>  >Accessing such an archive requires access credentials, which
>  >would have to be communicated (probably possible within
>  >the URI).
> 
> No, not within the URI. That's a very bad practice, much more
> of a security issue than your 'automatic download endless loop
> denial of service' scenario. What browsers do here is that they
> ask for the password

Whoa; who said anything about a password? I was referring
to the user name which various schemes and protocols
require.

> Having the MUA invoke the browser, and the browser then invoke
> the MUA, for an imap scheme, sounds really weird.

Do you know of any MUAs that directly handle IMAP URIs?
Do you know of any IMAP URI-handling browsers that can
reply to messages (rather than invoking a separate MUA)?

> Either the 
> MUA can handle the imap scheme, then it doesn't have to invoke
> the browser, but can just handle imap: in the same way as
> mailto:.

A mailto URI specifies a new message, which is a completely
different thing from an imap URI.  So thay cannot be handled
"in the same way".

> On the other hand, if it doesn't handle imap:, there 
> is no point to configure the browser to invoke the MUA.
> [this leaves the edge case of the MUA invoking the browser,
> and the browser invoking another MUA]

Or an IRC client invoking a browser, which invokes an MUA.
It depends on how one receives the URI.
 
> What seems more interesting to me is the following: The MUA
> finds an http: URI, and invokes the browser. The browser
> resolves the http: URI, and fetches a document with media
> type message/rfc882. The browser then invokes the MUA to
> handle that media type. That way, the browser actually
> does some work, rather than just playing ping-pong.

I don't see how changing "http" to "imap" in that description
is substantively different.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>, [email-masked]
Subject: Re: a header authentication scheme
Date: Sat, 30 Oct 2004 14:08:18 -0400
Sender: [email-masked]

On Thu October 28 2004 21:29, Laird Breyer wrote:

> Now you're trolling ;-)

No, I'm refuting your claim that "ordinary, trustworthy people
send spam".

> > > How does authentication help in that model? You'll know 
> > > *who* sent the spam, but you can't stop them all (for if you did, you
> > > will have solved a good part of the virus problem).
> > 
> > Knowledge of precisely who sent it is a necessary part of a
> > remedy. [and applies regardless of the specific remedy; whether
> > e.g. that involves removal of a "virus" or disconnection of the
> > machine from the Internet or both]
> 
> That's a generality which is as likely true as false depending
> on the problem.

No.

> In fact, your mention of virus removal gives a simple 
> counterexample: there is no need to know who sent a virus to be able
> to remove it from an infected system.

One needs to identify the "infected system". In the specific
case of a worm or virus that result in sending of spam after
a user installs that worm or virus, stopping the spew of
spam being sent by that worm or virus requires removing
or disabling it from that "infected system" (or removing
network connections from that system).

> There is also no need to know  
> who sent it to be able to block it from entering other systems. 

Blocking entry to one system is not a remedy.

> > > If you want to restrict yourself to non-clueless users, then how do
> > > you verify that a person sending you mail isn't clueless?
> > 
> > For a person known to me?  That's a judgment call.
> > 
> > > And will you 
> > > blacklist the others?
> > 
> > If the offense is repeated, yes.
> 
> It rarely is. The supply of clueless users is big enough that each one
> need offend once only, and you'll be blacklisting day in, day out. 

The supply of clueless users known to me is quite limited.
 
> > > So in particular it can sign its messages with the user's private key
> > > if the user has one. Then you could trace the spam back to the user
> > > and that's it. 
> > 
> > That user *is* the spammer -- his machine(s), running software that
> > he has installed and which is under his administrative control, is the
> > source of the spam, and the machine(s) involved is/are the ones
> > that need to be disconnected from the internet to curtail the
> > problem.
> 
> Ok, serious point: disconnecting the spamming computer from the internet
> is not always an option. Who can disconnect the machine? The relevant ISP.
> Why should the ISP do so? 

Because the activity violates the terms of service agreement between
the ISP and the user.  Because it is the ISP's costly and valuable
resources  (network bandwidth, storage space) that are being
consumed, Because the ISP will gain a bad reputation if it knowingly
harbors spammers.
 
> Take a provider like AOL, and let's say 30% of its users at one time
> are infected with a spam sending trojan. Are they going to cut off 30%
> of their customers for the dubious benefit of making life more
> pleasant for people who aren't their customers?

In the first place, you are incorrectly assuming that an AOL
user who has installed a spam-sending program will not send
spam to other AOL users.  In reality, AOL users are likely at
least equally affected.  Second, don't ask me what AOL would
do in such a hypothetical situation; ask them.  While you're
at it, you might ask for some realistic numbers.

> It's perfectly ok to configure your software to not trust any 
> Received lines at all, and therefore none of the Processed lines either.
> 
> Some other user, with different configuration, may elect to trust
> some Received lines and not others, and therefore some Processed lines
> and not others.

All of this blather about hypothetical configuration of hypothetical
software to trust some hypothetical Received lines on some
hypothetical basis ignores the plain fact that Received fields in
practice are frequently unparseable and simply are not intended
for machine parsing.
 
> > there is in fact no guarantee that a "processed" field or the
> > information that it purports to convey cannot be forged in the
> > presence of a valid Received field.
> 
> That's the whole point of the discussion we're having. If you know how
> to successfully forge the Processed field as I originally described it
> and its variations as already discussed with other list members, please
> give a concrete scenario.

Given a message with a valid Received field; it's trivial; simply
construct a Processed field accordingly and insert it where
desired, eliding any preexisting "processed" field that corresponds
to the Received field.  W/o any Received field; forge one and
insert it and a corresponding "processed" field where desired.
 
> For example, take the variation we discussed where the Received line
> is unfolded, and hashed (with md5 say), and the hash is quoted
> in the Processed field. Describe a method which succeeds with high
> probability in fooling a receiver into thinking a (forged) Processed
> line was added after the topmost Received line seen by this receiver.

The "topmost Received line" isn't necessarily the only one, nor
is it necessarily the only one with a corresponding "processed"
field in your scheme. If one has access to the message, one
can add a field with forged information even more easily than
it is to add "legitimate" information.  Bear in mind the fact that
not all message transfer involved SMTP, but that inly SMTP
requires inserting a Received field (and moreover, there is no
enforcement of that requirement).

> > No, I did not say that I "don't trust that information"; I said
> > that there is insufficient information to make any determination.
> > Citing insufficient information to make a determination is *NOT*
> > that same as making a particular determination!
> 
> The MUA is pretty dumb. It only wants to know if it should act in 
> predefined ways on the extra information or not. I take it you would
> choose the "No" option.

No, I'd get a better MUA.  Would you use an MUA that constantly
asks you "have you stopped beating your wife, yes or no?"?
 
> > No currently standardized field purports to convey any information
> > which requires detailed configuration information to be able to
> > interpret the conveyed information.
> 
> Neither does the Processed field. Its keyword/value pairs will have
> standard meanings which can be interpreted without the need for
> configuration information.

Not so.  Given your example
   Processed: name="SpamAssassin"; location-ip="1.2.3.4";
       version="2.63"; function="spamcheck";
       auth-received="B17C07174B5E4546A2B04EB096E83FD7081936B8"; 
       result-tag="spam";
I cannot interpret (in any meaningful sense) the "result-tag='spam'"
part without knowing specific configuration information and run-time
environment information for the instance of "SpamAssassin" version
"2.63" from which the purported "result-tag" was allegedly obtained.
I know nothing about its spam threshold, the myriad of fudge factors
that are used to weigh various characteristics, whether any address
patterns are used as whitelists or blacklists and if so what those
patterns are, etc.

> > > For example, you 
> > > decide to read messages addressed to you, you decide to reply to the
> > > mailbox listed in a Reply-To fields etc. even though you have no idea
> > > what configuration the software which created those fields was in.
> > 
> > Ultimately a human is responsible for setting recipient addresses
> > and Reply-To fields; they are set by a human user, either directly
> > or by configuration of an MUA acting on his behalf and for which
> > he bears responsibility (and for which no recipient needs any
> > configuration information).  In cases where Reply-To has been
> > (inappropriately) set by autonomous agents (e.g. misconfigured
> > list expanders), problems have resulted.
> 
> You're sidestepping the issue. I've given an example (Reply-To) where
> you (and I and we all) act based on zero hard facts about the writer
> of the Reply-To field. If we can act on unreliable fields such as
> Reply-To, why can't we act on fields such as Processed? What's the
> fundamental difference you perceive and are pointing out?

The fundamental difference is that the Reply-To field is specifically
provided as a means for the message author(s) to suggest a list
of mailboxes for responses. "Acting on" a Reply-To field presupposes
that one has already decided to make a response (prior to consulting
the field content) and therefore one already presumably already
knows *why* he is responding and therefore has some idea of where
his response should be directed. He can choose to respond directly
to the author(s), completely ignoring Reply-To, he can choose to
send his response to some specific place without consulting Reply-To,
he can choose any number of places to direct his response, one of
which is the author's suggestion.  I.e. one makes a decision (to
respond), another decision (where to respond), and in a subset of
cases, might subsequently consult a Reply-To field which conveyed
the original message author's suggestion; if one does not choose
to respond, the presence, absence, or content of any Reply-To field
is simply irrelevant, and in any event is not involved in the decision
*to* respond or not.  You are proposing that some decision be
based on incomplete and inadequate information contained in a
"processed" field.   I.e. the field must exist and must have some
content before the relevant decision (a guess, really) can be made.

> > Your proposed "processed" field differs from all of the above;
> > unlike Return-Path and Received trace fields, its content is
> > not well-defined.  
> 
> Of course not, you'll have to wait for a proper draft.

You have missed the point.  A Received line "from" component
documents the domain name used by a client in an SMTP
transaction. That documents a fact about what was used. It
is not a value judgment and is in no way dependent upon
the configuration of the MTA that documents that fact. That
is the sense in which the existing trace field content is
"well-defined", and is not the case for your "result-tag" etc.
component.

> I'm bouncing 
> off ideas for how best to deal with a sub-issue (proving writer
> location), I thought that was clear.

For messages, the existing MIME security multiparts seen to
have the only hope of providing such proof.  Of course
there's little point in "proving writer location" if what is
written is inadequate or is mere gibberish.

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

Cc: <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: Understanding response protocols
Date: Fri, 29 Oct 2004 17:10:05 -0400
To: "Hector Santos" <[email-masked]>
Sender: [email-masked]

> I believe it is an "opinion" of whether it is a "propagating a bad 
> practice" to the user.

Yes it is an opinion.  More specifically, it is an opinion that is 
informed by the experience of many people in a variety of situations 
who each have several years' experience in email and mailing lists.  It 
is something that was not obvious when rfc822 was written but which 
came to be realized over time.  It shouldn't surprise anyone that after 
using email for 25 years or so, we learned a few things.

In a nutshell, the reason that having lists set Reply-To is bad is that 
it masks the intent of the original author.  If you receive a message 
from a list with a Reply-To field, did the author set Reply-To, or did 
the list do so?  The recipient's decision about whether to honor that 
field could quite reasonably hinge on the difference.

Lists set Reply-To in part because there was a time when many MUAs 
didn't have the notion of "reply all".  (for that matter, some lists 
used to set From or another return address because many of the MUAs in 
the environments served by those lists - BITNET/RSCS and UUCP - didn't 
really use RFC822 headers).  So in some sense setting Reply-To is an 
anachronism that was inherited from obsolete systems.

I'd be in favor of giving lists a way to indicate their preference 
separately from the author's preference (say, List-Reply-To) -- 
_provided_ that it specified that MUAs don't accept it automatically 
but instead make it easy for the replier to know about the list's 
preference and to choose to use it or not.   (and no, this isn't the 
same as List-Post because a list's policy might be that replies go to 
the author only, or to some other set of addresses).

Keith

p.s. regarding your arguments and experience:  You have given us no new 
reasons for using Reply-To in this way, you have merely repeated things 
we have already heard.  You like to cite your own experience, but many 
of us have comparable experience in terms of years working with email 
in general, and more experience with Internet email.  Most of the 
regular participants of this list have known each other for several 
years and have learned to respect each others' opinions (we've also 
learned where our differences lie).  I get the impression that most of 
us have never heard of you until recently.  Which doesn't mean that 
you're wrong, but it does mean that neither citing your experience nor 
rehashing old arguments is going to convince us that you're right.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Understanding response protocols
Date: Fri, 29 Oct 2004 16:05:17 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Subject: Re: Understanding response protocols

>
> LISTSERV and ListProc users who were used to Reply-To being
> set by those servers also tended to think that having lists set
> Reply-To was "the right thing".

Question:

I have been made aware that this is the "default" server setup behavior
("reply to list" via Repl-To) for most of the list server products in the
market place.   Is this true?  It is true for the above two products you
mentioned?

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Understanding response protocols
Date: Fri, 29 Oct 2004 15:51:17 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Keith Moore" <[email-masked]>
Subject: Re: Understanding response protocols

> It's hardly surprising that your customers who have already become
> accustomed to the way your product behaves are annoyed when you change
> it.  LISTSERV and ListProc users who were used to Reply-To being
> set by those servers also tended to think that having lists set
> Reply-To was "the right thing".
>
> None of this changes the fact that having lists set Reply-To is less
> functional and does harm to the message.  All it means is that users
> often choose the behavior they have adapted to in their isolated
> communities, rather than the behavior that works best overall.  More
> generally, people tend to keep doing things the same way that they first
> learned to do them, even if they find out later that there are better
> ways to do that. That's not an argument for propagating bad practice to
> new users.
>

Yes, I understand this point about the learning curves.   Thanks for not
saying my customers are stupid like you said once before.  You probably
believe that anyway.

Look, I don't pretend to believe there will be further progress conversing
with you, but I will make this comment anyway.

I believe it is an "opinion" of whether it is a "propagating a bad practice"
to the user.  I don't agree with you on that point from the simple nature
that I believe your viewpoint on it is different than mind, and I believe my
view is more open minded that considers what are the "possible solutions"
ideal, realistic or not, and its consideration that deal with across board
issues.

I believe it is clearly an MUA issue. But it's only an MUA issue because of
the middle ware, and the MUA not being about to "react" to the expected
interfaces.  We muddle the water between Online vs. Offline.  You think to
think it is isn't an issue. I do.

You position is that the integrity of the mail should be maintained endpoint
to endpoint.   You are not going to find too many people that would be a
5000% proponent of the importance of keeping with mail integrity.  I've be
among the first to fight for mail integrity every day.  Our products depends
on this operational assurance, technically and legally.

But when it comes to a LIST or GROUPWARE concept, the question is now who is
the responsible new "owner" of the distribution.  It is not the same idea of
a 1 to 1 email conversation where there is laws (both natural in and the
books) governing the tampering of private email and all passthru mail for
that matter.

Again, the MUA comes into play. We muddle the water again with 'what kind of
MUA?"  open RFC based MUA with maybe EMAIL, NEWS and IMAP,  or other MUAs
with "Exchange" like interfaces which exposes the backend mail conferencing.

Now again, I don't wish to argue with you on because I know you are not the
type to compromise on anything. You don't even recognize my expertise,
questioned my knowledge of the mail system, dismiss my entire 25+ career in
exclusive telecommunications mail/file hosting and distribution products.
This says more about you, not me. I see that and say "Why bother?" the guy
is clearly an moron.

Finally, please don't use the same old mantra that I "don't listen."   On
the contrary, I understand your view 100%. I give all the benefit of the
doubt looking all aspects of a suggestion, good and bad, and when bad I look
at "Well, what will make it better or good?"  That is how I work in general.
I just don't think in some cases, your input is realistically correct for
dealing with "today's" real world issues  There is more new considerations
that need to be dealt with and all I see you doing is just throwing them
away by the waste side. That's the difference here between you and I.  That
is my only criticism of you as the #1 show stopper in the IETF when it comes
to new people presenting new ideas.  My only advice to you is to be more
open minded. If you see a problem with a suggestion, well, help in maybe
making it better if all possible.

Ciao

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

Date: Fri, 29 Oct 2004 14:32:06 -0400
From: Keith Moore <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

It's hardly surprising that your customers who have already become
accustomed to the way your product behaves are annoyed when you change
it.  LISTSERV and ListProc users who were used to Reply-To being 
set by those servers also tended to think that having lists set
Reply-To was "the right thing".   

None of this changes the fact that having lists set Reply-To is less
functional and does harm to the message.  All it means is that users
often choose the behavior they have adapted to in their isolated
communities, rather than the behavior that works best overall.  More
generally, people tend to keep doing things the same way that they first
learned to do them, even if they find out later that there are better
ways to do that. That's not an argument for propagating bad practice to
new users. 

Keith

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Understanding response protocols
Date: Fri, 29 Oct 2004 13:59:25 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Nathaniel Borenstein" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>; "Keith Moore" <[email-masked]>
Sent: Thursday, October 28, 2004 2:57 PM
Subject: Re: Understanding response protocols

>
> On Oct 20, 2004, at 12:24 AM, Hector Santos wrote:
>
> > You continue to show your true colors, odd and evil nature and
> > character.
>
> ?Evil?  Keith is *evil* because he is unwilling to accept a new
> interpretation of a standard that has been interpreted another way for
> a quarter century?  I would be most grateful if everyone involved could
> try to keep this discourse a bit more civil.

Apparently, you have not endured the wrath of his private emails.

> > Its too bad the IETF needs to get a bad rap because of people like you.
>
> My impression is that the IETF's "bad rap" is predominantly of the form
> "IETF is too slow to get the job done."  The primary reason we are slow
> is that we are open to a constant influx of newcomers who don't
> understand the past,

It would be a mistake to associate the term "newcomer" with not
understanding the past.

> who seem to be doing their best to condemn us to
> repeat it, and who need to be (slowly) educated on the complexities of
> the underlying problems and the existing systems.

I have always work in ways that shows a compromise in progress with the old
and new.   We have products in the market place.  Why would I want to
"break it?"

No, the only reason I began to participate in the IETF last year was to
hopefully be part of the new 'aura of change' that is among us in the
industry, an "aura of change" that has has not happen since the last
revamping of the industry towards the internet were we had to invest
resources to migrate to.  No, we understand the past quite well.  We know
how it fits then and now, and have a pretty good feel if whats needed
today - both ideally and realistically.

In any case, I don't think I said anything wrong about the current issues
with List Servers, Mailing list and how MUA behaves and the possible ideas
that are needed to address some of them.

I can tell you this: I do listen and I take all comments very seriously.

Our Wildct! list server (wcListServe) is designed to fit into and around an
"intranet" model of our "Wildcat! Internet Net Server" and RPC based
CLIENT/SERVER central storage system where the clients are exposes the
server databases to the world.    This means we have to deal with for the
most part with online hosting and offline hosting concepts.

WcListServer used a "reply to list" concept that forced the the "reply-to"
in the distribution.  Never a complaint about this.  Never an report or
comment about "duplicate mails."  However, we knew there was an ergonomic
issue with the MUA not making it easy to reply offlist.

It was an issue that was on the table for a long time.   So when I saw this
thread, I had alot of interest in it.  The issue was mainly what was a
better way to handle the list concept so that the layman user is not
confused.

As a result, we added new options in our recent gamma update to disable the
"reply to list" option and keep whatever reply-to: was there, if any.

I think it was an option that should be there but in a off default setting.

We tested it out for our own support mailing list   Result?

- Direct private mails from members who did not realize they were not
replying to the list.

- and comments that the new behavior is going to KILL our product!  I got
comments list this:

    "See... Already wcListserve no longer operates like a list
     server anymore. =)"

We turned the new option off for our own usage and we are now back to
normal.  Now everyone is happy in our support mailing list:

    "Finally! Hit reply and didn't have to change the TO: address..... =)"

Go figure.

Thanks for your comments

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Fri, 29 Oct 2004 11:08:42 -0400
From: Keith Moore <[email-masked]>
To: ietf-822 <[email-masked]>, Martin Duerst <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

> Structured fields (as far as RFC 2822's definition of structured
> vs. unstructured fields is concerned) are primarily used by
> message handling programs (not only clients, but e.g. IMAP
> servers, MTAs, Sieve filtering language (RFC 3028), etc.).
> Fields intended to be used only by humans are generally
> unstructured (again, in the RFC 2822 sense), e.g. Subject,
> Comments, Content-Description.  

Yes, this _tends_ to be the case, because fields designed to be parsed
by computer programs need to be structured, while fields designed
primarily for human consumption have less need to be structured.
However it's not a rule, it's just the way things usually work.

In practice it's often the case that we want a field to be both
human readable and usable by computer programs.  I think that would
have been true for Archived-At even if we had started out with
the understanding that it was originally intended for human use.

Keith

Date: Fri, 29 Oct 2004 11:02:51 -0400
From: Keith Moore <[email-masked]>
To: ietf-822 <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: General considerations for new message field specifications
Sender: [email-masked]

> Can anybody think of additional considerations (other than
> the obvious ones which are required in all RFCs (security
> considerations, IP disclosure, etc.))?

10.  who is expected/allowed to set the field?  originator, 
submission agent, intermediaries, delivery agent, end user?

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, Martin Duerst <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Fri, 29 Oct 2004 10:11:49 -0400
Cc: Martin Duerst <[email-masked]>
Sender: [email-masked]

On Thu October 28 2004 03:08, Martin Duerst wrote:

> Yes, but can you forward more than one message in a single message?

Of course; that's what multipart/digest is for.

>  >Yes, context is useful, but that context is provided by the
>  >related messages and the archive infrastructure (or, if I have
>  >access to the messages in an MUA, by that MUAs features).
>  >List-Archive can point to such an archive.  Nor is Archived-At
>  >necessary to get a starting point at a particular message;
>  >a URI could be obtained from any suitable mechanism,
> 
> Could you be a bit more specific here?

A List-Archive URI could be an imap URI or a pop URI, for
example.  Also, as described in a separate message, a URI
could be conveyed in a Comments field, as a text comment
in a suitable structured field, etc.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: General considerations for new message field specifications
Date: Fri, 29 Oct 2004 10:08:30 -0400
Cc: Martin Duerst <[email-masked]>
Sender: [email-masked]

Based on recent discussion and several recent drafts, there seem
to be a number of relevant issues that are often overlooked when
a new message field is proposed.  For future reference, here is a
suggested list of items that should be addressed by any message
field proposal:

1. Utility of the field; what purpose does it serve, and to whom.
    How is it envisioned that the field will be used?

2. Semantics of the field; what does it mean?

3. Syntax of the field (using RFC 2234/2822 grammar).
3a. If the field syntax uses any keywords, what keyword values
      are defined, what mechanisms are specified for registering
      additional keywords, what provisions are made for testing
      and experimental use, etc.

4. Minimum and maximum number of instances of the field
    permitted per header, per message or per report section.

5. Permitted and/or required context for the field. Is it
   permitted or required in message headers, in MIME-part
   headers, as a per-message field in one or more
   multipart/report report types, as a per-recipient field in
   a multipart/report report type, etc.  Are there any specific
   conditions that affect whether or not the field may/must
   be included?

6. Do any existing mechanisms require amendment to
   address handling of the proposed field? Examples include
   message/partial fragmentation and reassembly,
   handling by gateways, and processing by MSAs.

7. How does the field fit into the overall message format;
    is the field a trace field, an address field, etc.?

8. Is a Resent- version of the field applicable?

9. Is the field or any portions considered experimental in
    the sense of RFC 3692 (viz. requiring specific configuration
    to enable use)?

Can anybody think of additional considerations (other than
the obvious ones which are required in all RFCs (security
considerations, IP disclosure, etc.))?

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, Martin Duerst <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Thu, 28 Oct 2004 22:46:04 -0400
Cc: Martin Duerst <[email-masked]>
Sender: [email-masked]

On Thu October 28 2004 02:39, Martin Duerst wrote:
 
> It's probably only partially useful for email clients. Indeed Bruce
> Lilly makes a rather good point that it is not useful at all for
> email clients (he however makes the mistake of thinking that
> email clients are everything that counts).

To be precise, I pointed out some limited use for MUAs.  Focus on
MUAs was based on the fact that the field is defined as a structured
field and has no defined uses for other agents and/or protocols
(MSAs, MTAs, MDAs, filters, access protocols).

> It's very useful for 
> humans. Is there a rule that email headers have to be used by
> email clients?

Structured fields (as far as RFC 2822's definition of structured
vs. unstructured fields is concerned) are primarily used by
message handling programs (not only clients, but e.g. IMAP
servers, MTAs, Sieve filtering language (RFC 3028), etc.).
Fields intended to be used only by humans are generally
unstructured (again, in the RFC 2822 sense), e.g. Subject,
Comments, Content-Description.  So if it had been clear back
in February that the intent was solely for human use, we
could have avoided a lot of syntax discussion and the field
could have been defined as unstructured; or we could have
arrived then where we are today, with the suggestion that
any existing mechanism for conveying human-readable
non-protocol content could be used, without having to
invent "Yet Another Header Field".

> As an example, how much is the X-Mailer: header 
> used my email clients?

Well, it's not a standardized field... as such it has no
specified uses, semantics, or syntax.  As a user-specified
field (RFC 822, fields beginning with "x-"), any use is
subject to and dependent upon agreement between
cooperating parties.
 
>  >and at least some
>  >email clients are already able to download files using http and/or
>  >ftp.
> 
> But that's just files in general, yes? Or is is message/rfc882 in
> particular? And if the later, what clients would that be?

MIME is used with the HTTP protocol as well as with RFC [2]822
message format.  As such, HTTP can be used to transfer content
of any MIME media type, including message/rfc822.  Although
it has been elided from the current version of the FTP specification,
the FTP protocol did have specific commands (MAIL and MLFL)
for message transfer; these have been superseded by message-
specific transfer protocols, viz. MTP, SMTP (introduction of
which predated dropping of the message-specific FTP commands
by several years) and more recent variations such as POP,
IMAP, and NNTP.

Cc: [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Fri, 29 Oct 2004 07:46:34 -0400
To: Martin Duerst <[email-masked]>
Sender: [email-masked]

> >1. There really are significant differences between
> >    a) how email clients would use archived-at,
> >    b) how humans would use it to cut-and-paste into HTML, and
> >    c) how web browsers would use it.
>
> I'm not really sure about the difference between b) and c), or
> about c) as such. If you put an URI into an href attribute in
> HTML, then ultimately, it will be used by a browser, anyway.

actually I miswrote that.  I'm not sure there's a significant 
difference between b and c either - just that there's a significant 
difference between a and either b or c.

> >2. For archived-at to be useful by email clients generally requires
> >    one or two of the following, in addition to the obvious client
> >    support for the protocol and server support for the message 
> format:
> >
> >    - mail archives that support IMAP access (or possibly NNTP)
>
> My understanding (not being an IMAP expert) is that this would
> not be too difficult. However, I can't see W3C to do that any
> time soon.

I suspect the easiest way to do it would be to have the archiver store 
messages in maildir format; then just use an IMAP server that supports 
maildir and read-only access.

> >    - a specification for making collections of mail messages 
> available
> >      via HTTP (maybe WebDav) and/or FTP
>
> I'm not sure I understand this. List-Archive: points to a full
> list archive. Archived-At: points to a single message. It seems
> to me you either have the two confused, or what you want is really
> a format (or similar) that not only contains the actual message
> (message/rfc822 or equivalent), but also some additional context
> (that WebDav may be able to give you with some properties, for 
> example).

That's it.  I want to be able to refer to a single message in such a 
way that the surrounding context can also be found.  List-Archive (and 
the other List-* fields) have some of the same problems as Archived-At 
in that the contents can be arbitrary and there's no way for the mail 
reader to use them.  It's partially because of problems I've observed 
with List-* fields that I'm expressing concerns over Archived-At.

> Please note that any kind of inference from the actual URI to
> other URIs should be avoided, i.e. it would be a bad idea to
> say that if Archived-At contained
>    http://www.imc.org/ietf-822/mail-archive/msg05043.html
> we may assume that things like
>    http://www.imc.org/ietf-822/mail-archive/msg05042.html
> and
>    http://www.imc.org/ietf-822/mail-archive/msg05044.html
> are somewhat related (by being in the same mailing list,
> or whatever).

I share that concern to some degree.  Which is why I would have a 
keyword in Archived-At that says "you can assume that other files in 
the same directory/collection/whatever are also part of the same 
archive".  I don't think the MUA should assume this without an explicit 
indication.

> >3. The obvious compromise that makes sense in the short term (let
> >    archives be in other formats besides message/rfc822, and don't
> >    require message/rfc822 support) is harmful in the long term.
>
> Why? If there is uptake on mailers with such support, it's not
> too difficult to also upgrade the servers.

if, by the time mailers get upgraded to support that, there are a lot 
of messages that have Archived-At pointing to archives that aren't 
usable by mail readers, the feature is too awkward/dysfunctional to 
use.  either it always launches a web browser, or it's only supported 
by mailers that have built-in web browsers. it may also be too late to 
upgrade archives that were stored only in HTML.

> >Best compromise I see at this point:  Define some sort of
> >keywords for archived-at.  e.g.
> >
> >Archived-at: "<" URI ">" *(";" keyword [ "=" value ] )
> >
> >where keywords might include
> >
> >"native" 	message available in native message/rfc822 format
> >		(either because that's the only format available
> >                 at this URI or via content-negotiation)
> >
> >"collection"	other messages in the same collection are also
> >	        accessible, where the collection is defined by
> >		the IMAP folder, NNTP newsgroup, FTP directory,
> >		WebDav collection, etc. indicated by the URI.
> >
> >		the value associated with this keyword would
> >		indicate the name of the collection associated
> >		with the URI (since a message might appear in
> >		more than one collection)
>
> Ah, I see, so there would essentially be two URIs, one for
> the message itself, and one for the collection.

not what I was proposing.  here's a concrete example

Archived-At: 
<ftp://ftp.cs.utk.edu/pub/moore/mail-archives/ietf-822/20041030.2214>;
  native; collection=ietf-822

which would indicate (a) the message is available in rfc822 format (and 
perhaps others), (b) other messages in the "rfc822" collection could 
also be accessed (in this case by doing a LIST or whatever of the ftp 
directory).

I'd rather avoid having two URIs, though maybe it is cleaner overall.  
(would WebDav need two URIs?  it's been too long...)

> >This would (a) let tools record locations of ordinary HTTP/HTML 
> archives
> >such as are (too) often the only format available today,
>
> What would the keyword be for that case?

the absence of a keyword could mean "you can make no assumptions about 
the content-type of this resource".  if desired an "html" keyword could 
be added.  (or we could go back to Arnt Gulbrandsen's suggestion of 
having a content-type parameter, which is looking better to me now than 
it did when he proposed it.)

> >(b) encourage
> >archives to provide messages in their original format without 
> requiring
> >them to do so, (c) give implementors a hint that better functionality
> >can be had, (d) give email readers a clue as to whether they could
> >actually make use of the URI internally or whether they needed to pass
> >it to a separate browser (this is a common problem in HTML also)
>
> Well, this is solved on the level of media types, not URIs, for 
> browsers.

and the implication is that anything that uses URIs needs to be able to 
handle whatever comes back.  this often loses if "whatever comes back" 
can't be handled by the  browser and whatever program is launched to 
deal with "whatever comes back" needs information about the context.  
for instance I've seen links in PDF documents that couldn't be followed 
(apparently) because the PDF viewer didn't know what URI the document 
was retrieved from.

> >and (e)
> >leave room for expanded functionality without needing to define a new
> >header field.  And if you don't use any keywords, they don't take up
> >space.
> >
> >It might be better to defer the definition of any keywords to a
> >separate document, because I can imagine needing to define specifics
> >of how collections look within the context of various protocols.
> >I can also imagine needing to define things like collections that
> >consist of a directory of several mbox files, each containing
> >multiple messages.
>
> If that would mean that the only change to my current draft
> would be something like "there may be keywords, but currently,
> there are none defined", then I probably could live with that.
> But I'm still not convinced that we need keywords, things
> should work without them.

I think there's pretty good experience that things don't work without 
them, at least, not without requiring the email reader to also be a web 
browser.   even then you can get into a conflict between what the email 
reader can do and users' preferences - what if the user wants HTML 
documents to be handled by a web browser?  does he then lose the 
ability to handle relative links in HTML documents initially linked to 
from email?

Keith

Cc: <[email-masked]>, "Keith Moore" <[email-masked]>
From: Nathaniel Borenstein <[email-masked]>
Subject: Re: Understanding response protocols
Date: Thu, 28 Oct 2004 14:57:39 -0400
To: "Hector Santos" <[email-masked]>
Sender: [email-masked]

On Oct 20, 2004, at 12:24 AM, Hector Santos wrote:

> You continue to show your true colors, odd and evil nature and 
> character.

?Evil?  Keith is *evil* because he is unwilling to accept a new 
interpretation of a standard that has been interpreted another way for 
a quarter century?  I would be most grateful if everyone involved could 
try to keep this discourse a bit more civil.

Those of us who've been  working on these things for a long time have 
come to understand -- sometimes very painfully -- the incredible cost 
and difficulty of tinkering with the installed base.  It's never easy, 
and I've never seen anything accomplished simply by redefining the 
meaning of an existing standard.

> Its too bad the IETF needs to get a bad rap because of people like you.

My impression is that the IETF's "bad rap" is predominantly of the form 
"IETF is too slow to get the job done."  The primary reason we are slow 
is that we are open to a constant influx of newcomers who don't 
understand the past, who seem to be doing their best to condemn us to 
repeat it, and who need to be (slowly) educated on the complexities of 
the underlying problems and the existing systems.  I think it's great 
that there is new blood constantly coming into the organization, but 
we're paying a very high price for that, in that we have to continually 
recapitulate past discussions, design decisions, and lessons learned.

I know of no other standards body that makes it so easy for people to 
come in and present their own perspective than IETF.  Unfortunately, 
this also means that no other standards body is so easily derailed by 
misinformed or wrongheaded volunteers.  Before you get too negative 
about the IETF, you should try getting yourself a seat at the table for 
discussions of W3C, ISO, or even OASIS standards.  The fact that people 
are willing to listen to almost anyone here should, in my opinion, be a 
strong motivator for people (especially those who don't represent giant 
corporations, for whom the IETF should be a particularly precious 
institution) to restrain themselves to polite and civil discourse. -- 
Nathaniel

Cc: ietf-822 <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Fri, 29 Oct 2004 07:19:55 -0400
To: Martin Duerst <[email-masked]>
Sender: [email-masked]

> What seems more interesting to me is the following: The MUA
> finds an http: URI, and invokes the browser. The browser
> resolves the http: URI, and fetches a document with media
> type message/rfc882. The browser then invokes the MUA to
> handle that media type. That way, the browser actually
> does some work, rather than just playing ping-pong.

while this kind of "display this URI" functionality is typically 
supported by web browsers, I'm not aware of any email MUAs that support 
it.

it's pretty easy to implement enough of http to do a GET (including 
handling of redirects), so I'm not worried about the burden on the MUA 
of doing that.  what does concern me is that the MUA doesn't reliably 
know whether it can handle the resource until it actually does the GET. 
  it could then pass the downloaded resource to the browser, but that 
tends to mess up things like relative URIs.  so the decision of whether 
to handle the resource itself or to pass it off to the browser has to 
happen after the GET. (granted, it could do a HEAD, but that's still 
awkward).

Date: Fri, 29 Oct 2004 17:12:07 +0900
To: Keith Moore <[email-masked]>, [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: the gap regarding Archived-At
Sender: [email-masked]

At 05:38 04/10/29, Keith Moore wrote:
 >
 >okay, I find myself coming to some somewhat uncomfortable conclusions:
 >
 >1. There really are significant differences between
 >    a) how email clients would use archived-at,
 >    b) how humans would use it to cut-and-paste into HTML, and
 >    c) how web browsers would use it.

I'm not really sure about the difference between b) and c), or
about c) as such. If you put an URI into an href attribute in
HTML, then ultimately, it will be used by a browser, anyway.

 >    These differences are significant enough that I'm wondering if we
 >    really do want separate archived-at fields for web use and email use,
 >    or tags on the archived-at field that indicates whether the message
 >    is in original format and/or if other messages in the same collection
 >    can also be accessed.
 >
 >2. For archived-at to be useful by email clients generally requires
 >    one or two of the following, in addition to the obvious client
 >    support for the protocol and server support for the message format:
 >
 >    - mail archives that support IMAP access (or possibly NNTP)

My understanding (not being an IMAP expert) is that this would
not be too difficult. However, I can't see W3C to do that any
time soon.

 >    - a specification for making collections of mail messages available
 >      via HTTP (maybe WebDav) and/or FTP

I'm not sure I understand this. List-Archive: points to a full
list archive. Archived-At: points to a single message. It seems
to me you either have the two confused, or what you want is really
a format (or similar) that not only contains the actual message
(message/rfc822 or equivalent), but also some additional context
(that WebDav may be able to give you with some properties, for example).
Please note that any kind of inference from the actual URI to
other URIs should be avoided, i.e. it would be a bad idea to
say that if Archived-At contained
    http://www.imc.org/ietf-822/mail-archive/msg05043.html
we may assume that things like
    http://www.imc.org/ietf-822/mail-archive/msg05042.html
and
    http://www.imc.org/ietf-822/mail-archive/msg05044.html
are somewhat related (by being in the same mailing list,
or whatever).

 >    - mail archives that follow the aforementioned specification
 >
 >    and as much as I'd like to believe that email client vendors would
 >    enthusiastically add support for these, market conditions don't seem
 >    to favor supporting new functionality standards in email clients
 >    right now.

Sad but true.

 >3. The obvious compromise that makes sense in the short term (let
 >    archives be in other formats besides message/rfc822, and don't
 >    require message/rfc822 support) is harmful in the long term.

Why? If there is uptake on mailers with such support, it's not
too difficult to also upgrade the servers.

 >4. There's a real tendency for the web to degrade other services.
 >    Partly this is because it's easy to prototype new services using
 >    the web, and partly this is because once those services are
 >    prototyped, the limitations of web browsers, HTTP, and HTML
 >    impose a more significant barrier to the functionality that
 >    can be supported than existed in the original environment.

I'm still not sure I understand how serving a message as HTML
would inhibit serving it later as message/rfc822 (or whatever)
also. Maybe what could be pointed out in the draft is that
rather than using a full URI with (e.g. .html) extension,
a shorter URI that allows to more easily introduce
content negotiation later is preferable.

 >Best compromise I see at this point:  Define some sort of
 >keywords for archived-at.  e.g.
 >
 >Archived-at: "<" URI ">" *(";" keyword [ "=" value ] )
 >
 >where keywords might include
 >
 >"native" 	message available in native message/rfc822 format
 >		(either because that's the only format available
 >                 at this URI or via content-negotiation)
 >
 >"collection"	other messages in the same collection are also
 >	        accessible, where the collection is defined by
 >		the IMAP folder, NNTP newsgroup, FTP directory,
 >		WebDav collection, etc. indicated by the URI.
 >
 >		the value associated with this keyword would
 >		indicate the name of the collection associated
 >		with the URI (since a message might appear in
 >		more than one collection)

Ah, I see, so there would essentially be two URIs, one for
the message itself, and one for the collection.

 >This would (a) let tools record locations of ordinary HTTP/HTML archives
 >such as are (too) often the only format available today,

What would the keyword be for that case?

 >(b) encourage
 >archives to provide messages in their original format without requiring
 >them to do so, (c) give implementors a hint that better functionality
 >can be had, (d) give email readers a clue as to whether they could
 >actually make use of the URI internally or whether they needed to pass
 >it to a separate browser (this is a common problem in HTML also)

Well, this is solved on the level of media types, not URIs, for
browsers.

 >and (e)
 >leave room for expanded functionality without needing to define a new
 >header field.  And if you don't use any keywords, they don't take up
 >space.
 >
 >It might be better to defer the definition of any keywords to a
 >separate document, because I can imagine needing to define specifics
 >of how collections look within the context of various protocols.
 >I can also imagine needing to define things like collections that
 >consist of a directory of several mbox files, each containing
 >multiple messages.

If that would mean that the only change to my current draft
would be something like "there may be keywords, but currently,
there are none defined", then I probably could live with that.
But I'm still not convinced that we need keywords, things
should work without them.

Regards,   Martin. 

Date: Fri, 29 Oct 2004 16:42:03 +0900
To: ietf-822 <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: the gap regarding Archived-At
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

At 08:27 04/10/29, Bruce Lilly wrote:
 >On Thu October 28 2004 16:38, Keith Moore wrote:
 >>
 >> okay, I find myself coming to some somewhat uncomfortable conclusions:
 >>
 >> 1. There really are significant differences between
 >>    a) how email clients would use archived-at,
 >>    b) how humans would use it to cut-and-paste into HTML, and
 >>    c) how web browsers would use it.
 >
 >Incidentally, Keith's message (which contains no indication that
 >it was copied to Martin) is archived at
 >http://www.imc.org/ietf-822/mail-archive/msg05043.html

Keith forwarded it to me separately, probably after realizing
he had dropped me. Thanks for keeping an eye out for this!

 >1.  Comments: archived at
 >       http://www.imc.org/ietf-822/mail-archive/msg05043.html
 >   The Comments field has been around since at least RFC 724
 >   and therefore is more likely to be displayed by existing MUAs
 >   than Archived-At.

I'm not sure. My email client seems to have a tendency to display
unknown headers more easily than known ones. But this is only
a single example.

 > A message may contain an arbitrary number of Comments fields.

Yes. One issue is that if there are a lot of comments fields,
the user will have difficulties to choose the right one. It
also makes e.g. selection of which headers to display (in
highly configurable email agents), or assignement of a keyboard
shortcut, and so on, more difficult.

 > However, it does not solve the
 >   problem of the incompatibility with RFC 2046 message/partial.

As far as I'm concerned, Keith's description of the (non-)deployment
of this feature has pretty much solved this issue for me.

 >2. Message-ID:
 >      <[email-masked]>
 >      (archived at http://www.imc.org/ietf-822/mail-archive/msg05043.html)
 >  Parenthesized comments are provided for in Message-ID fields.
 >  This method puts the archive information in the same place as
 >  the message identifier,

But Message-ID wouldn't usually be shown by email clients, or would it?
(it's definitely not shown in mine, unless I choose 'display all headers)
Also, I'm concerned about the length of the field. Having separate
fields with reasonably short length seems better than having long
fields squeezing everything in.

 >  and is fully compatible with message/partial
 >  fragmentation and reassembly.

Again, as shown by Keith, a non-issue.

 >Both methods use existing mechanisms explicitly designed for human
 >use. As both methods involve content which is not subject to
 >automatic retrieval of content, both avoid the security trap
 >implicit in (possibly automated use of) Archived-At.

I hope that my writeup of the corresponding text for the security
section shows quite well that this isn't really something to be
very worried about in practice.

 >> 2. For archived-at to be useful by email clients generally requires
 >>    one or two of the following, in addition to the obvious client
 >>    support for the protocol and server support for the message format:
 >>
 >>    - mail archives that support IMAP access (or possibly NNTP)
 >
 >Accessing such an archive requires access credentials, which
 >would have to be communicated (probably possible within
 >the URI).

No, not within the URI. That's a very bad practice, much more
of a security issue than your 'automatic download endless loop
denial of service' scenario. What browsers do here is that they
ask for the password on the first use, and also allow users to
store the password. That's in wide use at W3C, because we have
archives only accessible for W3C Members, or for W3C Team.

 >>    and as much as I'd like to believe that email client vendors would
 >>    enthusiastically add support for these, market conditions don't seem
 >>    to favor supporting new functionality standards in email clients
 >>    right now.
 >
 >There may be a more fundamental issue: IMAP (or NNTP) access
 >requires either handling a URI or configuration of the necessary
 >parameters which would be conveyed in a URI (host name, port,
 >user credentials).  Certainly MUAs that provide IMAP client
 >functionality could be used without any additional design, but
 >the user would need to get the parameters from somewhere and
 >configure his MUA(s) accordingly.  MUAs don't typically handle
 >generic URIs directly (mailto URIs are a notable exception), but
 >instead may hand them off to a browser (which in turn might
 >invoke an MUA for an imap scheme URI).

Having the MUA invoke the browser, and the browser then invoke
the MUA, for an imap scheme, sounds really weird. Either the
MUA can handle the imap scheme, then it doesn't have to invoke
the browser, but can just handle imap: in the same way as
mailto:. On the other hand, if it doesn't handle imap:, there
is no point to configure the browser to invoke the MUA.
[this leaves the edge case of the MUA invoking the browser,
and the browser invoking another MUA]

What seems more interesting to me is the following: The MUA
finds an http: URI, and invokes the browser. The browser
resolves the http: URI, and fetches a document with media
type message/rfc882. The browser then invokes the MUA to
handle that media type. That way, the browser actually
does some work, rather than just playing ping-pong.

The alternative to that would be to implement http: HEAD
or GET requests, and a table for which application to
invoke for which content type, for the MUA. (and there is
a chance that it would download a text/html document but
would want to invoke the browser with the URI rather
than the document (essentially a local URI), which would
result in double downloads.

Regards,    Martin. 

Date: Fri, 29 Oct 2004 11:29:16 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 <[email-masked]>
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 <[email-masked]>
Sender: [email-masked]

On Oct 28 2004, Bruce Lilly wrote:
> On Wed October 27 2004 21:19, Laird Breyer wrote:
> 
> > I doubt that. The current trend is well on its way to the following
> > model: ordinary, trustworthy people send spam to strangers without
> > knowing it.
> 
> No.  If they send spam, they are not trustworthy. If they run an
> "operating" system with multiple known security vulnerabilities
> without a properly-configured external firewall, they are not
> trustworthy.

Now you're trolling ;-)

> 
> > How does authentication help in that model? You'll know 
> > *who* sent the spam, but you can't stop them all (for if you did, you
> > will have solved a good part of the virus problem).
> 
> Knowledge of precisely who sent it is a necessary part of a
> remedy. [and applies regardless of the specific remedy; whether
> e.g. that involves removal of a "virus" or disconnection of the
> machine from the Internet or both]

That's a generality which is as likely true as false depending
on the problem. In fact, your mention of virus removal gives a simple
counterexample: there is no need to know who sent a virus to be able
to remove it from an infected system. There is also no need to know 
who sent it to be able to block it from entering other systems. 

>  
> > If you want to restrict yourself to non-clueless users, then how do
> > you verify that a person sending you mail isn't clueless?
> 
> For a person known to me?  That's a judgment call.
> 
> > And will you 
> > blacklist the others?
> 
> If the offense is repeated, yes.

It rarely is. The supply of clueless users is big enough that each one
need offend once only, and you'll be blacklisting day in, day out. 

> > So in particular it can sign its messages with the user's private key
> > if the user has one. Then you could trace the spam back to the user
> > and that's it. 
> 
> That user *is* the spammer -- his machine(s), running software that
> he has installed and which is under his administrative control, is the
> source of the spam, and the machine(s) involved is/are the ones
> that need to be disconnected from the internet to curtail the
> problem.

Ok, serious point: disconnecting the spamming computer from the internet
is not always an option. Who can disconnect the machine? The relevant ISP.
Why should the ISP do so? 

Take a provider like AOL, and let's say 30% of its users at one time
are infected with a spam sending trojan. Are they going to cut off 30%
of their customers for the dubious benefit of making life more
pleasant for people who aren't their customers? It's not exactly
increasing shareholder value.

> 
> > The method proposed is not an oracle. It doesn't give a truth value
> > to the proposition "is the Processed field trustworthy?", instead
> > it gives a truth value to the proposition "given that the Received
> > line is trustworthy, is the Processed field trustworthy?"
> 
> Bad premises rarely lead to reliable conclusions. Not all Received
> fields are "trustworthy".  Flawed logic also rarely leads to
> reliable conclusions; 

The bad premises are entirely yours to make or not. It's up to you whether 
you feel some Received lines are trustworthy (to you) or not.

It's perfectly ok to configure your software to not trust any 
Received lines at all, and therefore none of the Processed lines either.

Some other user, with different configuration, may elect to trust
some Received lines and not others, and therefore some Processed lines
and not others.

Both users can look at the exact same message and come to different
conclusions, as appropriate for each.

> there is in fact no guarantee that a "processed" field or the
> information that it purports to convey cannot be forged in the
> presence of a valid Received field.

That's the whole point of the discussion we're having. If you know how
to successfully forge the Processed field as I originally described it
and its variations as already discussed with other list members, please
give a concrete scenario.

For example, take the variation we discussed where the Received line
is unfolded, and hashed (with md5 say), and the hash is quoted
in the Processed field. Describe a method which succeeds with high
probability in fooling a receiver into thinking a (forged) Processed
line was added after the topmost Received line seen by this receiver.

> > That is an excellent answer. The MUA told you it found a 
> > piece of information you might wish to use, and you told it
> > that you don't trust that information. Mission accomplished!
> 
> No, I did not say that I "don't trust that information"; I said
> that there is insufficient information to make any determination.
> Citing insufficient information to make a determination is *NOT*
> that same as making a particular determination!

The MUA is pretty dumb. It only wants to know if it should act in 
predefined ways on the extra information or not. I take it you would
choose the "No" option.

> 
> > That's a straw man. None of the headers in any mail message, ever,
> > carry full configuration information.
> 
> No currently standardized field purports to convey any information
> which requires detailed configuration information to be able to
> interpret the conveyed information.

Neither does the Processed field. Its keyword/value pairs will have
standard meanings which can be interpreted without the need for
configuration information.

> 
> > Do you or any recipient know how 
> > exim or Postfix were configured when they added their trace fields to
> > transported messages? How about the sender's MUA configuration when the
> > message was composed?
> 
> No need to know.  Trace field definitions (RFCs 821, 2821) are
> generally clear about what is supposed to appear in those trace
> fields; Received fields document the client host name as specified
> in the SMTP session HELO or EHLO command, the host name of
> the host adding the field, a time stamp in standard format, and
> optionally some additional information, most of which is well-defined
> (and none of which is essential).  Return-Path documents the sender
> envelope address.  No MTA-specific configuration alters or in any
> way affects the client's argument to the HELO/EHLO command,
> for example.

(*)
The same will be true for a Processed field. A Processed field shall
consist of keyword="value" elements, with a number of predefined keywords
whose appropriate values are explicitly defined. Nonstandard keyword="value"
elements will be allowed, but discouraged. A reader of the Processed field
shall only act on predefined keywords, or all keywords at its discretion
and its own risk. A well defined authentication based on quoting the latest
Received line at time of writing will be optional, but strongly encouraged.
Any reader of the Processed field will be strongly encouraged to only
read those Processd fields whose authentication can be verified, and 
whose associated Received line it trusts by other means. Whether it truly 
does so in the end is at its own discretion of course.  

> 
> > But you (or anybody) still routinely make decisions based on the
> > unreliable information written by the above programs.
> 
> No, trace fields are examined in rare cases, not routinely.
> The specific information in those fields documents facts
> about a particular SMTP session (client-supplied identification,
> etc.); it does not purport to make a value judgment about
> the message content.

Trace fields are routinely used by spam filters to help decide whether
to block the message or not. So unless a user's mail is not filtered
at any intermediate MTA location, a decision has already been made
based on this information, even before the message hits the user's
inbox.

> 
> > For example, you 
> > decide to read messages addressed to you, you decide to reply to the
> > mailbox listed in a Reply-To fields etc. even though you have no idea
> > what configuration the software which created those fields was in.
> 
> Ultimately a human is responsible for setting recipient addresses
> and Reply-To fields; they are set by a human user, either directly
> or by configuration of an MUA acting on his behalf and for which
> he bears responsibility (and for which no recipient needs any
> configuration information).  In cases where Reply-To has been
> (inappropriately) set by autonomous agents (e.g. misconfigured
> list expanders), problems have resulted.

You're sidestepping the issue. I've given an example (Reply-To) where
you (and I and we all) act based on zero hard facts about the writer
of the Reply-To field. If we can act on unreliable fields such as
Reply-To, why can't we act on fields such as Processed? What's the
fundamental difference you perceive and are pointing out?

> 
> Your proposed "processed" field differs from all of the above;
> unlike Return-Path and Received trace fields, its content is
> not well-defined.  

Of course not, you'll have to wait for a proper draft. I'm bouncing
off ideas for how best to deal with a sub-issue (proving writer
location), I thought that was clear. And incidentally I thank the list
for the valuable feedback.

> Unlike Reply-To, it is not set by a human user.
> Unlike all of the above, it purports to convey information,
> including a value judgment, which is unusable without
> additional detailed information which it does not convey.

I've given an outline of the structure of the Processed field in
the paragraph marked (*) above, which should address these points
except for the human element. I don't see why being a human rather
than a machine is significant.

-- 
Laird Breyer.

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Thu, 28 Oct 2004 19:54:58 -0400
To: Bruce Lilly <[email-masked]>
Sender: [email-masked]

> The discussion to date hasn't really addressed a or c, it
> has focused on b.

I'm not sure why you think that.  I've addressed a on more than one 
occasion.

> 1.  Comments: archived at
>        http://www.imc.org/ietf-822/mail-archive/msg05043.html
>    The Comments field has been around since at least RFC 724
>    and therefore is more likely to be displayed by existing MUAs
>    than Archived-At.  A message may contain an arbitrary
>    number of Comments fields.  However, it does not solve the
>    problem of the incompatibility with RFC 2046 message/partial.

could you explain that supposed incompatibility again?  not that I'm 
really worried about it as (a) I haven't seen a message/partial since 
the early 1990s (even then it was only as a test) and (b) somehow I 
doubt that most MUAs can reassemble message/partials anyway.

> Both methods use existing mechanisms explicitly designed for human
> use. As both methods involve content which is not subject to
> automatic retrieval of content, both avoid the security trap
> implicit in (possibly automated use of) Archived-At.

I don't see why in the world an MUA would download the URI in an 
Archived-At field, because it already has the message on hand.  (no, I 
don't consider message/partial a threat in this regard).

>>    These differences are significant enough that I'm wondering if we
>>    really do want separate archived-at fields for web use and email 
>> use,
>>    or tags on the archived-at field that indicates whether the message
>>    is in original format and/or if other messages in the same 
>> collection
>>    can also be accessed.
>
> That information may be implicit in the specific URI scheme indicated.

yes, it might - but generally, it's not implicit in the URI scheme.

> An imap scheme certainly indicates that the format is message/rfc822

I've certainly seen IMAP servers that would provide arbitrary contents. 
   Whether that was a bug or a feature, I was never sure.

>> 2. For archived-at to be useful by email clients generally requires
>>    one or two of the following, in addition to the obvious client
>>    support for the protocol and server support for the message format:
>>
>>    - mail archives that support IMAP access (or possibly NNTP)
>
> Accessing such an archive requires access credentials, which
> would have to be communicated (probably possible within
> the URI).

IIRC, IMAP was specifically designed to be able to provide read-only 
access to archives.  But it's been awhile.

>>    - a specification for making collections of mail messages available
>>      via HTTP (maybe WebDav) and/or FTP
>
> multipart/digest is an existing media type, which can be
> transferred via HTTP and/or ftp, which *is* a collection
> of messages.

yup, but as an archive format, it's pretty pessimal.  you can't easily 
locate a particular message within the file, nor can you append new 
messages to the file.  that might be why nobody uses it for that 
purpose.  (okay, it's a big Internet.  _somebody_ might use it)

> [...]
>>    and as much as I'd like to believe that email client vendors would
>>    enthusiastically add support for these, market conditions don't 
>> seem
>>    to favor supporting new functionality standards in email clients
>>    right now.
>
> There may be a more fundamental issue: IMAP (or NNTP) access
> requires either handling a URI or configuration of the necessary
> parameters which would be conveyed in a URI (host name, port,
> user credentials).  Certainly MUAs that provide IMAP client
> functionality could be used without any additional design, but
> the user would need to get the parameters from somewhere and
> configure his MUA(s) accordingly.  MUAs don't typically handle
> generic URIs directly (mailto URIs are a notable exception), but
> instead may hand them off to a browser (which in turn might
> invoke an MUA for an imap scheme URI).

Parsing an IMAP URI is not rocket science, and doesn't need more than a 
couple of hundred lines of code.  Somehow I don't think this is the 
limiting factor.

(Actually I think that all MUAs should allow their mailboxes and 
submission servers to be specified using URLs - it's much simpler and 
less error-prone than giving users lots of blanks with misleading 
labels to fill in.)

>> Best compromise I see at this point:  Define some sort of
>> keywords for archived-at.  e.g.
>>
>> Archived-at: "<" URI ">" *(";" keyword [ "=" value ] )
>>
>> where keywords might include
>>
>> "native" 	message available in native message/rfc822 format
> [...]
>> "collection"	other messages in the same collection are also
> [...]
>
> I think the issue of a list of URIs might be revisited (and not
> necessarily in a structured field), so that multiple access
> methods to a given message could be specified (that does
> not preclude multiple fields to indicate multiple places where
> copies may be archived, except of course if comments in the
> Message-ID field are used).

I don't see the point in specifying multiple ways to do the same thing. 
  There's a good argument for allowing multiple Archived-At fields 
because the originator, the recipient's MTA, and any number of 
intermediaries might archive a message - it's much cleaner to just let 
each of them add a separate field.  Given that I don't see any need to 
have multiple URIs within a field also.

>>  And if you don't use any keywords, they don't take up
>> space.
>
> The code needed to parse them takes space in the MUA.

only for MUAs that care about them.  as for RFC 2231, I agree that they 
can get ugly, but I think this will be rare.  and the MUA ought to have 
code to handle RFC 2231 anyway.

Keith

From: Bruce Lilly <[email-masked]>
Reply-To: Bruce Lilly <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Thu, 28 Oct 2004 19:41:26 -0400
Cc: Graham Klyne <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Sender: [email-masked]

On Thu October 28 2004 09:09, Graham Klyne wrote:

> To paraphrase (British) parliament-speak:  I refer the gentleman to the 
> answer given previously:
>    http://lists.w3.org/Archives/Public/www-archive/2004Oct/0050.html

That does not explain the utility of the particular structured
header field proposed (vs. e.g. an unstructured Comments field
or a parenthesized comment in e.g. the Message-ID field, or
text in a body part, etc., all of which would be at least as
usable for clickety-clickety or copy and paste).

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Thu, 28 Oct 2004 19:27:54 -0400
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Thu October 28 2004 16:38, Keith Moore wrote:
> 
> okay, I find myself coming to some somewhat uncomfortable conclusions:
> 
> 1. There really are significant differences between 
>    a) how email clients would use archived-at,
>    b) how humans would use it to cut-and-paste into HTML, and
>    c) how web browsers would use it.

Incidentally, Keith's message (which contains no indication that
it was copied to Martin) is archived at
http://www.imc.org/ietf-822/mail-archive/msg05043.html

The discussion to date hasn't really addressed a or c, it
has focused on b.  As Arnt has noted, use by humans does
not require a structured field at all. I can see no disadvantage
for human use as has been discussed to the following
alternative means of communicating information about
archived copies (using the URI for the copy of Keith's
message as an example):

1.  Comments: archived at
       http://www.imc.org/ietf-822/mail-archive/msg05043.html
   The Comments field has been around since at least RFC 724
   and therefore is more likely to be displayed by existing MUAs
   than Archived-At.  A message may contain an arbitrary
   number of Comments fields.  However, it does not solve the
   problem of the incompatibility with RFC 2046 message/partial.

2. Message-ID:
      <[email-masked]>
      (archived at http://www.imc.org/ietf-822/mail-archive/msg05043.html)
  Parenthesized comments are provided for in Message-ID fields.
  This method puts the archive information in the same place as
  the message identifier, and is fully compatible with message/partial
  fragmentation and reassembly.  However, it may be slightly more
  difficult to add (e.g. at a list expander) to a message which already
  has a Message-ID field (compared to adding a new Comments or
  Archived-At field).

Both methods use existing mechanisms explicitly designed for human
use. As both methods involve content which is not subject to
automatic retrieval of content, both avoid the security trap
implicit in (possibly automated use of) Archived-At.

>    These differences are significant enough that I'm wondering if we
>    really do want separate archived-at fields for web use and email use,
>    or tags on the archived-at field that indicates whether the message
>    is in original format and/or if other messages in the same collection
>    can also be accessed.

That information may be implicit in the specific URI scheme indicated.
An imap scheme certainly indicates that the format is message/rfc822
and implies that other messages may be accessible.

> 2. For archived-at to be useful by email clients generally requires
>    one or two of the following, in addition to the obvious client
>    support for the protocol and server support for the message format:
> 
>    - mail archives that support IMAP access (or possibly NNTP) 

Accessing such an archive requires access credentials, which
would have to be communicated (probably possible within
the URI).

>    - a specification for making collections of mail messages available
>      via HTTP (maybe WebDav) and/or FTP

multipart/digest is an existing media type, which can be
transferred via HTTP and/or ftp, which *is* a collection
of messages.

[...]
>    and as much as I'd like to believe that email client vendors would
>    enthusiastically add support for these, market conditions don't seem
>    to favor supporting new functionality standards in email clients
>    right now.

There may be a more fundamental issue: IMAP (or NNTP) access
requires either handling a URI or configuration of the necessary
parameters which would be conveyed in a URI (host name, port,
user credentials).  Certainly MUAs that provide IMAP client
functionality could be used without any additional design, but
the user would need to get the parameters from somewhere and
configure his MUA(s) accordingly.  MUAs don't typically handle
generic URIs directly (mailto URIs are a notable exception), but
instead may hand them off to a browser (which in turn might
invoke an MUA for an imap scheme URI).
 
> Best compromise I see at this point:  Define some sort of 
> keywords for archived-at.  e.g.
> 
> Archived-at: "<" URI ">" *(";" keyword [ "=" value ] )
> 
> where keywords might include
> 
> "native" 	message available in native message/rfc822 format
[...] 
> "collection"	other messages in the same collection are also
[...]

I think the issue of a list of URIs might be revisited (and not
necessarily in a structured field), so that multiple access
methods to a given message could be specified (that does
not preclude multiple fields to indicate multiple places where
copies may be archived, except of course if comments in the
Message-ID field are used). So (a la RFC 2369 URI lists) a given
field could list URIs to access a particular message via http
(format indeterminate), imap, ftp (format indeterminate),
pop, nntp, etc.

I'd rather avoid anything resembling RFC 2231 parameters if
at all possible; you probably don't really want to have to deal
with
   Archived-At: <http://www.imc.org/ietf-822/mail-archive/msg05043.html>
     ; native*2*=%F2%F0%F0%F4%40 ;
     native*0*=EBCDIC-INT''%E3%A4%85k%40 
     ;native*1*= %F0%F4%40%C6%85%82%40 ;
     native*3*=%F2%F0%7A%F5%F7%7A%F5%F8%40
    ;native*4*=N%F0%F0%F0%F0

[...]
>  And if you don't use any keywords, they don't take up
> space. 

The code needed to parse them takes space in the MUA.

Date: Thu, 28 Oct 2004 16:38:50 -0400
From: Keith Moore <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: the gap regarding Archived-At
Sender: [email-masked]

okay, I find myself coming to some somewhat uncomfortable conclusions:

1. There really are significant differences between 
   a) how email clients would use archived-at,
   b) how humans would use it to cut-and-paste into HTML, and
   c) how web browsers would use it.

   These differences are significant enough that I'm wondering if we
   really do want separate archived-at fields for web use and email use,
   or tags on the archived-at field that indicates whether the message
   is in original format and/or if other messages in the same collection
   can also be accessed.

2. For archived-at to be useful by email clients generally requires
   one or two of the following, in addition to the obvious client
   support for the protocol and server support for the message format:

   - mail archives that support IMAP access (or possibly NNTP) 
   - a specification for making collections of mail messages available
     via HTTP (maybe WebDav) and/or FTP
   - mail archives that follow the aforementioned specification

   and as much as I'd like to believe that email client vendors would
   enthusiastically add support for these, market conditions don't seem
   to favor supporting new functionality standards in email clients
   right now.

3. The obvious compromise that makes sense in the short term (let 
   archives be in other formats besides message/rfc822, and don't
   require message/rfc822 support) is harmful in the long term.

4. There's a real tendency for the web to degrade other services.
   Partly this is because it's easy to prototype new services using
   the web, and partly this is because once those services are 
   prototyped, the limitations of web browsers, HTTP, and HTML
   impose a more significant barrier to the functionality that 
   can be supported than existed in the original environment.

Best compromise I see at this point:  Define some sort of 
keywords for archived-at.  e.g.

Archived-at: "<" URI ">" *(";" keyword [ "=" value ] )

where keywords might include

"native" 	message available in native message/rfc822 format
		(either because that's the only format available
                at this URI or via content-negotiation)

"collection"	other messages in the same collection are also
	        accessible, where the collection is defined by
		the IMAP folder, NNTP newsgroup, FTP directory,
		WebDav collection, etc. indicated by the URI.

		the value associated with this keyword would
		indicate the name of the collection associated
		with the URI (since a message might appear in 
		more than one collection)

This would (a) let tools record locations of ordinary HTTP/HTML archives
such as are (too) often the only format available today, (b) encourage
archives to provide messages in their original format without requiring
them to do so, (c) give implementors a hint that better functionality
can be had, (d) give email readers a clue as to whether they could
actually make use of the URI internally or whether they needed to pass
it to a separate browser (this is a common problem in HTML also) and (e)
leave room for expanded functionality without needing to define a new
header field.  And if you don't use any keywords, they don't take up
space. 

It might be better to defer the definition of any keywords to a
separate document, because I can imagine needing to define specifics
of how collections look within the context of various protocols.
I can also imagine needing to define things like collections that
consist of a directory of several mbox files, each containing 
multiple messages.

Keith

Date: Thu, 28 Oct 2004 16:05:26 -0400
From: Keith Moore <[email-masked]>
To: Bill McQuillan <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

> One issue I see with a text/rfc822 message is that it provides *no*
> way to determine the "collection" by itself.

nor should it.  you do want a way to find other messages in the 
collection, but they shouldn't be embedded in the message.
the problem is that vanilla HTTP doesn't have the notion of
collections.

Keith

Date: Thu, 28 Oct 2004 11:09:16 -0700
From: Claus Assmann <[email-masked]>
To: [email-masked]
Subject: Re: a header authentication scheme
Mail-Followup-To: Claus Assmann <[email-masked]>, [email-masked]
Sender: [email-masked]

On Fri, Oct 22, 2004, Arnt Gulbrandsen wrote:

> Step 1. I'd send five messages to [email-masked] and wait for the 
> bounces. For each of them, I'd compute the delay from my sending time 
> (using my clock) to the receiving MTA's receive time. Next, I'd average 
> the five values.

Where do you get the time of the server? Many MTAs don't accept
mail for unknown recipients, so it's your MTA that generates the
bounce.

Date: Thu, 28 Oct 2004 10:43:04 -0700
From: Bill McQuillan <[email-masked]>
To: RFC822 <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

On Thu, 2004-10-28, Keith Moore wrote:

> Again, I think the real utility with this field and email readers is 
> being able to use that message as a starting point to browse a 
> collection.  They are as likely to perform operations (like replies, 
> refiles, forwards) on other messages in that collection as they are 
> with the original one.

One issue I see with a text/rfc822 message is that it provides *no* way to determine the "collection" by itself. It is just a message, sitting alone. It might be possible to parse an imap: URL to determine the mailbox that the message resides in, but IMAP is not as widely supported as we might like for Archived-At purposes. On the other hand, with all of their drawbacks, HTML archives usually provide navigation links on the same page as the message to the containing collection.
  
-- 
Bill McQuillan <[email-masked]>

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>, [email-masked]
Subject: Re: a header authentication scheme
Date: Thu, 28 Oct 2004 10:58:03 -0400
Sender: [email-masked]

On Wed October 27 2004 21:19, Laird Breyer wrote:

> I doubt that. The current trend is well on its way to the following
> model: ordinary, trustworthy people send spam to strangers without
> knowing it.

No.  If they send spam, they are not trustworthy. If they run an
"operating" system with multiple known security vulnerabilities
without a properly-configured external firewall, they are not
trustworthy.

> How does authentication help in that model? You'll know 
> *who* sent the spam, but you can't stop them all (for if you did, you
> will have solved a good part of the virus problem).

Knowledge of precisely who sent it is a necessary part of a
remedy. [and applies regardless of the specific remedy; whether
e.g. that involves removal of a "virus" or disconnection of the
machine from the Internet or both]
 
> If you want to restrict yourself to non-clueless users, then how do
> you verify that a person sending you mail isn't clueless?

For a person known to me?  That's a judgment call.

> And will you 
> blacklist the others?

If the offense is repeated, yes.
 
> > If I receive a signed message with a valid signature that turns out
> > to be spam, the credentials associated with the signature may
> > provide a means of identifying the spammer.
> 
> I don't see why it should. A trojan on a Windows box has access to
> both the system API, if it wants to send mail programmatically, and
> also access to the GUI by simulating mouse and keypresses.  Together,
> these two methods can bypass most protections, ie anything a user can
> do a trojan can do, too.
> 
> So in particular it can sign its messages with the user's private key
> if the user has one. Then you could trace the spam back to the user
> and that's it. 

That user *is* the spammer -- his machine(s), running software that
he has installed and which is under his administrative control, is the
source of the spam, and the machine(s) involved is/are the ones
that need to be disconnected from the internet to curtail the
problem.

> The method proposed is not an oracle. It doesn't give a truth value
> to the proposition "is the Processed field trustworthy?", instead
> it gives a truth value to the proposition "given that the Received
> line is trustworthy, is the Processed field trustworthy?"

Bad premises rarely lead to reliable conclusions. Not all Received
fields are "trustworthy".  Flawed logic also rarely leads to
reliable conclusions; there is in fact no guarantee that a
"processed" field or the information that it purports to convey
cannot be forged in the presence of a valid Received field.

> > My answer would have to be that I cannot make that
> > determination without knowing details of the configuration
> > and run-time options, and without knowing exactly what
> > site is meant by "the third hop from the end" and without a
> > considerable amount of effort to determine if that site is in
> > fact what is claimed, and unless I can verify that that site
> > in fact added the fields that have been claimed to have been
> > added there, and unless I have absolute trust in that site.
> 
> That is an excellent answer. The MUA told you it found a 
> piece of information you might wish to use, and you told it
> that you don't trust that information. Mission accomplished!

No, I did not say that I "don't trust that information"; I said
that there is insufficient information to make any determination.
Citing insufficient information to make a determination is *NOT*
that same as making a particular determination!

> That's a straw man. None of the headers in any mail message, ever,
> carry full configuration information.

No currently standardized field purports to convey any information
which requires detailed configuration information to be able to
interpret the conveyed information.

> Do you or any recipient know how 
> exim or Postfix were configured when they added their trace fields to
> transported messages? How about the sender's MUA configuration when the
> message was composed?

No need to know.  Trace field definitions (RFCs 821, 2821) are
generally clear about what is supposed to appear in those trace
fields; Received fields document the client host name as specified
in the SMTP session HELO or EHLO command, the host name of
the host adding the field, a time stamp in standard format, and
optionally some additional information, most of which is well-defined
(and none of which is essential).  Return-Path documents the sender
envelope address.  No MTA-specific configuration alters or in any
way affects the client's argument to the HELO/EHLO command,
for example.

> But you (or anybody) still routinely make decisions based on the
> unreliable information written by the above programs.

No, trace fields are examined in rare cases, not routinely.
The specific information in those fields documents facts
about a particular SMTP session (client-supplied identification,
etc.); it does not purport to make a value judgment about
the message content.

> For example, you 
> decide to read messages addressed to you, you decide to reply to the
> mailbox listed in a Reply-To fields etc. even though you have no idea
> what configuration the software which created those fields was in.

Ultimately a human is responsible for setting recipient addresses
and Reply-To fields; they are set by a human user, either directly
or by configuration of an MUA acting on his behalf and for which
he bears responsibility (and for which no recipient needs any
configuration information).  In cases where Reply-To has been
(inappropriately) set by autonomous agents (e.g. misconfigured
list expanders), problems have resulted.

Your proposed "processed" field differs from all of the above;
unlike Return-Path and Received trace fields, its content is
not well-defined.  Unlike Reply-To, it is not set by a human user.
Unlike all of the above, it purports to convey information,
including a value judgment, which is unusable without
additional detailed information which it does not convey.

Date: Thu, 28 Oct 2004 14:09:01 +0100
To: Bruce Lilly <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: ietf-822 <[email-masked]>
Sender: [email-masked]

At 13:44 27/10/04 -0400, Bruce Lilly wrote:

>On Wed October 27 2004 12:58, Graham Klyne wrote:
> > At 10:07 27/10/04 -0400, Bruce Lilly wrote:
> > > > Or a least, if a message is going to point to an
> > > > archive of itself, it should be a faithful archive.
> > >
> > >That raises several new issues with Martin's draft.   They can be
> > >summarized as "what is the utility of the proposed field; who
> > >will use it, under what circumstances, and for what purpose(s)?".
> > >Note that that is not a question about syntax, or about semantics,
> > >or about validity, but about utility.
> >
> > Utility?  That's easy!  The system described by Martin's draft
>
>The question is not about the w3 list archive, it is about
>the specific proposal in Martin's draft (which mentions the
>w3c list only by way of example).

As a demonstration of utility, I find that example to be quite compelling.

>Please reread the question and the draft.  The draft title
>and topic pertain to a message header field; one that is
>supposed to provide information pointing to an archived
>copy of that very message.  Now please explain the
>utility *of such a field*, given that if I already have the
>field, I already have the message that it supposedly
>references.

To paraphrase (British) parliament-speak:  I refer the gentleman to the 
answer given previously:
   http://lists.w3.org/Archives/Public/www-archive/2004Oct/0050.html

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Thu, 28 Oct 2004 14:06:04 +0100
To: Keith Moore <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

At 13:35 27/10/04 -0400, Keith Moore wrote:
> > Utility?  That's easy!  The system described by Martin's draft is 
> extremely
> > useful.  It is implemented by the w3.org mailing list systems, and in my
> > work using those mailing lists I've made frequent and much-appreciated use
> > of the feature.
>
>Maybe you could provide some concrete examples?

I explained it here:
   http://lists.w3.org/Archives/Public/www-archive/2004Oct/0050.html

> > I find it faintly ironic that one of the best (as in most usable and most
> > useful) mailing list services I've used anywhere on the Internet is run by
> > the World Wide *Web* Consortium.  The archive system is a major factor in
> > that usability.  Mail archives really take on additional value when they
> > become a fully linked and cross-linked part of the Web.  For this to 
> really
> > work well --as it does-- the messages really need to be served using a
> > hypertext format, such as HTML.  I've no objection if others would like to
> > design and build systems that serve messages using whatever format they
> > like, but to claim that the HTML-served messages are in some sense not
> > useful or non-functional is to ignore the daily experience of the many
> > people who use them.
>
>I guess the question is this - do you want the archived email messages to
>be usable as email messages, or do you want them to be a part of the web?

I don't really care if it's called web or email.  What I do like about this 
proposal is that it can help me to achieve my goals, by making it easy to 
point other people to previous messages, and making it easy for other 
people to do the same for me.

>It's very difficult to do both, because web browsers tend not to support
>message/rfc822 very well.

I suggest that is not because it's fundamentally difficult to do, but 
because little need has been felt for such a facility.  Where's the itch 
here?  I think a message/rfc822 plugin for firefox wouldn't be a major 
task, if someone felt strongly enough to do it.  Given such support, there 
might be more demand for archives in RFC822 format.  (Maybe it exists, but 
a quick search didn't reveal it to me.)

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Thu, 28 Oct 2004 14:36:08 +0100
To: Martin Duerst <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: http://www.ietf.org/internet-drafts/draft-duerst-archived-at-02.txt
Cc: [email-masked]
Sender: [email-masked]

At 18:01 28/10/04 +0900, Martin Duerst wrote:
>At 17:38 04/10/27, Graham Klyne wrote:
> >Martin,
> >
> >Two nits concerning the header field registration (copied below).
> >
> >1. Specify that the entry is for the permanent header field registry, 
> assuming the relevant conditions are satisfied; e.g.
> >[[
> >    IANA is herewith requested to register the Archived-At header field
> >    in the Permanent Message Header Fields Registry, per RFC 3864, as 
> follows:
> >]]
>
>What I wanted was to make sure that we get a provisional registration
>as soon as possible, and then can move to the permanent registration
>as the draft moves forward. That was the way I understood RFC 3864.
>Are you saying that I shouldn't care about the provisional registration?
>Or should I do the provisional totally separately? Rereading
>RFC 3864, I now seem to understand that registering separately
>is how it should be done.

Provisional registration is, by design, a lightweight process.

If you want a provisional registration quickly, I suggest you put a copy of 
this draft on the web at a persistent location (e.g. as an attachment to 
www-archive ;-) and then submit a message to 
[email-masked] (also to here would be polite) 
requesting provisional registration, then barring a substantiated claim 
that the proposal is in some way harmful send a request to IANA for 
inclusion in the provisional registry.  At this stage, the template should 
indicate a status of "provisional", and focus on the header field details 
as implemented by w3.org.  (I'd also include a template for X-Archived-At, 
since that's what is currently used.)

Doing this in no way precludes the proper review and discussion taking 
place here, with a view to inclusion in the permanent header field registry.

BTW, the way the registry is evolving in practice seems to be that IANA 
posts just three values:
1. the header field name
2. the applicable protocol
3. a reference to a document containing the registration template.

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Cc: [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Thu, 28 Oct 2004 09:41:33 -0400
To: Martin Duerst <[email-masked]>
Sender: [email-masked]

> >> You are essentially just confirming what I said: It's not 
> implemented.
> >> The fact that I'm using a browser isn't some kind of bias, it's just
> >> currently the only way it works.
> >
> >That's an odd way of looking at it.
>
> Well, somewhat, but in terms of "rough consensus and running code"
> not really.

This seems to be a frequently-misunderstood phrase.  The "running code" 
part is supposed to happen _after_ the specification is written (as in, 
before we advance to draft standard).  It's often taken to mean that 
IETF should standardize things that were developed on an ad hoc basis,  
which I don't believe to be the intent.

> >If you're trying to define a
> >new extention to email, presumably to be used by email clients,
> >_of course_ it's not already implemented by email clients.
>
> It's probably only partially useful for email clients. Indeed Bruce
> Lilly makes a rather good point that it is not useful at all for
> email clients

And I've described a couple of ways that it could be useful for email 
clients.

> Is there a rule that email headers have to be used by email clients?

no.  But it seems suboptimal to define a header field that _could_ be 
useful for email clients, in a way that is not very useful for email 
clients.

> >And of course native format archives _do_ exist
>
> Good. But the fact that they exist doesn't mean that we have
> to try and force everybody to provide them.

"Force" is a strong word, since we have no police force.  "Encourage" 
or SHOULD would be more appropriate.  To me the purpose of an archive 
is preservation, and it makes little sense to preserve messages in a 
format that takes up more space and is less functional than the 
original when the original format is widely supported.

To look at it another way, part of the conflict exists because web 
browsers support message/rfc822 so poorly, and part of it exists 
because vanilla HTTP (as implemented in web browsers) doesn't have much 
support for dealing with separate resources that are part of a 
hierarchal, sequenced collection.  So this stuff gets pushed to HTML, 
which has several undesirable effects.  One is that the original form 
of the document is no longer preserved, and some of the semantic 
information from that format is lost or mapped to less functional forms 
such as "next in thread" buttons. or if it is preserved, it's buried 
(e.g. the message/rfc822 is conveyed in a frame), and the URI of the 
resource no longer refers to the actual resource.  it's really a huge 
limitation of the web as it is currently implemented.  What I don't 
want to do is to burden email with the web's limitations.  (email has 
enough limitations of its own to overcome)

Also the experience with use of arbitrary URIs in the List-* fields 
does not seem favorable.  Even though they're supported by list 
software, they're not in my experience widely supported by email 
readers, even when those email readers recognize URIs in other 
contexts.  Lots of people seem to think that "reply to list" would be a 
useful function, but it's awkward to use List-Post for this purpose 
because it doesn't necessarily point to an email address (though it 
does seem feasible for the email reader to check: "is there a mailto: 
URI that doesn't fill in the header fields or message body?" and if so, 
use that for reply to list).  Perhaps this doesn't bear directly on 
Archived-At but it does indicate to me that just putting URIs in header 
fields may not be as functional as we'd like, and part of the reason 
might be that email user agents can't use them.  Or maybe the problem 
is just that market conditions have killed innovation in email readers 
so even trivial fixes like "display List-* fields and make their URIs 
clickable" don't happen.

> >and at least some
> >email clients are already able to download files using http and/or
> >ftp.
>
> But that's just files in general, yes? Or is is message/rfc882 in
> particular? And if the later, what clients would that be?

I was thinking in terms of message/external-body support, which was 
implemented in mh and is also supported by metamail.  It applies to any 
content-type.  I'm not claiming that it's widely supported, only that 
it has been implemented, so it's clearly feasible.

> Mentioning various media types and http content negotiation, I think
> it should be quite clear from my draft that there is no particular
> expectation one way or another. That's very similar to the HTML
> spec not saying what type of document you should find at the end
> of an anchor. There is quite some expectation in that case that
> that might be another HTML document, but there are lots of other
> cases.

In a way I think this is correct.  But I also think that realistically 
implementors will "just provide HTML" unless given guidance to do 
otherwise, which will make this field far less functional in practice.

> Although W3C, as an example, currently does not serve the archived
> mails in message/rfc822 format, if we did it, we would certainly
> do the right thing to make sure that we would not need additional
> headers (e.g. two Archived-At, one for text/html and another for
> message/rfc822 format), let alone a new field name.

My mention of a new field name comes from the following scenario: 
Archived-At is implemented so widely to be "just HTML" that this 
becomes the de facto expectation.  Mail readers tend to not implement 
it, or to just use it to launch a web browser, because it's rarely in a 
format that can be used effectively by the mail reader.  Eventually 
someone realizes that it would be very useful to have a way for email 
messages to refer to archived messages within a collection.  But they 
can't use Archived-At for this purpose because it's so likely to point 
to an HTML document rather than an email message that supporting it 
will just annoy users.

> There is another aspect here: Assume that one of the reasons we
> want a message/rfc822 format is for replies to archived messages.
> In that case, having a pointer from the actual message to the
> archive in order to fetch the message from the archive and then
> answer to it seems heavy overkill, because the message is already
> available locally and can be replied to directly. So it is possible
> to immagine usage patterns where the Archived-At header points to
> a HTML version, and that HTML version includes a link to the
> message/rfc822 version, for use by people who don't have the
> original message. Of course, this is only one possible usage
> pattern, the draft doesn't require that at all.

Yes but that's awkward in several ways.  One is that you need a 
different name for the rfc822 version of the resource than for the html 
version of the resource.  Another is that the email reader now needs an 
HTML viewer just in order to access the native version of the message.

Again, I think the real utility with this field and email readers is 
being able to use that message as a starting point to browse a 
collection.  They are as likely to perform operations (like replies, 
refiles, forwards) on other messages in that collection as they are 
with the original one.

Keith

Date: Thu, 28 Oct 2004 12:55:44 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Keith Moore <[email-masked]>, Graham Klyne <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

Martin Duerst writes:
> As said in other mails, the way we use these headers, it's mostly 
> humans rather than email programs using them. Is there something in 
> the email specs (e.g. RFC 2822 or so) that would forbid such headers?

No, although...

I believe that as a rule of thumb, it's bad to go against the grain of a 
design. In theory it can be sensible and right, but generally it turns 
out to be a bad idea eventually. And in mail, the header fields that 
are meant for direct human consumption are generally unstructured (ie. 
they use the 'unstructured' production in RFC2822). The ones that 
aren't unstructured are generally meant to be parsed.

> Thinking the other way round, your comments above seem to indicate 
> that you think the Web and Email should both work as two separate 
> worlds. The reason for the proposal at hand, however, is to create 
> possibilies for better connections between these worlds.

I realized that later. Sounds sensible to my ears.

Arnt

Date: Thu, 28 Oct 2004 12:26:15 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Keith Moore <[email-masked]>, Graham Klyne <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

Martin Duerst writes:
> At 21:54 04/10/27, Keith Moore wrote:
> >I think this is needlessly complex.
>
> I agree. It's also not how URIs and the Web work.

I certainly agree that it's... flexible. I suggested a 100% flexibility 
solution. If the desired-for benefit can be defined, it should be easy 
to find something that gives 80% of that benefit for 20% of the 
complexity.

Btw, doesn't the web work pretty much like this? <link href="blah.css" 
type="text/css">?

> >Just provide the archives in 822 format.
>
> This is one possible solution, but not the only one.

It would be interesting to hear what benefits w3c.org mailing list used 
obtain from the Archived-At field.

Arnt

Date: Thu, 28 Oct 2004 18:01:23 +0900
To: Graham Klyne <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: http://www.ietf.org/internet-drafts/draft-duerst-archived-at-02.txt
Cc: [email-masked]
Sender: [email-masked]

Hello Graham,

Just some cross-checks, because I don't seem to have understood
RFC 3864 correctly.

At 17:38 04/10/27, Graham Klyne wrote:
 >Martin,
 >
 >Two nits concerning the header field registration (copied below).
 >
 >1. Specify that the entry is for the permanent header field registry, 
assuming the relevant conditions are satisfied; e.g.
 >[[
 >    IANA is herewith requested to register the Archived-At header field
 >    in the Permanent Message Header Fields Registry, per RFC 3864, as follows:
 >]]

What I wanted was to make sure that we get a provisional registration
as soon as possible, and then can move to the permanent registration
as the draft moves forward. That was the way I understood RFC 3864.
Are you saying that I shouldn't care about the provisional registration?
Or should I do the provisional totally separately? Rereading
RFC 3864, I now seem to understand that registering separately
is how it should be done.

 >2. You need to decide if this is or is not to be usable with netnews.  If 
it is to be usable in netnews, then (for clarity) there should be two 
registration templates, one for mail and one for netnews, otherwise the same.

I think it could be potentially useful for netnews, although netnews
has URI schemes that make it possible to refer to a news item, and
the way netnews is distributed makes it somewhat easier to find them
than email messages. Also, adding an 'Archived-At' header to netnews
can be trickier than for email unless it would be done at the source.

 >If you are really not sure about netnews, then I suggest you leave it out 
and separately request an entry for netnews in the provisional header field 
registry:  it can be moved later if netnews community think that's reasonable.

That seems like a reasonable solution.

Regards,     Martin. 

Date: Thu, 28 Oct 2004 17:36:57 +0900
To: Arnt Gulbrandsen <[email-masked]>, [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Keith Moore <[email-masked]>, Graham Klyne <[email-masked]>
Sender: [email-masked]

At 18:38 04/10/27, Arnt Gulbrandsen wrote:
 >Graham Klyne writes:
 >> Archives that exist and work (as does the scheme described by Martin) 
are infinitely more functional than ones that don't.
 >>
 >> Isn't that the point of "running code..."?
 >
 >Work is the key word, and the meaning of work depends on context.
 >
 >Archives on the web must work in the context of the web, IMNSHO. And 
similarly, archives pointed to by message/rfc822 header fields must/should 
work in the context of message/rfc822 handling programs, because those are 
the programs that look at such fields.

As said in other mails, the way we use these headers, it's mostly
humans rather than email programs using them. Is there something
in the email specs (e.g. RFC 2822 or so) that would forbid such
headers?

Thinking the other way round, your comments above seem to indicate
that you think the Web and Email should both work as two separate
worlds. The reason for the proposal at hand, however, is to
create possibilies for better connections between these worlds.

Another point, mentioning it in this mail just because it came
to my mind just now: as far as I remember, RFC 2369 does not
say anything about how email messages should or should not be
archived.

Regards,    Martin.

 >(That doesn't necessarily mean that the retrieved object must have a 
certain content-type. I can imagine other ways to achieve the same end.)
 >
 >Arnt
 > 

Date: Thu, 28 Oct 2004 17:31:29 +0900
To: Keith Moore <[email-masked]>, Arnt Gulbrandsen <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked], Graham Klyne <[email-masked]>
Sender: [email-masked]

At 21:54 04/10/27, Keith Moore wrote:
 >> My preferred approach would be to give archived-at the same parameter 
list that MIME fields have, and define a "content-type=x/y" parameter 
naming a type expected to be available. For a list that provides archives 
in two forms and uses fancy HTTP to provide the right one, the field could 
be like this:
 >>
 >> Archived-At: ...; content-type=message/rfc822, content-type=text/html
 >>
 >> For a less fancy archive providing both forms under separate URLs, two 
Archived-At fields could be used:
 >>
 >> Archived-At: ...; content-type=message/rfc822
 >> Archived-At: ...; content-type=text/html
 >
 >I think this is needlessly complex.

I agree. It's also not how URIs and the Web work.

 >Just provide the archives in 822 format.

This is one possible solution, but not the only one.

Regards,    Martin. 

Date: Thu, 28 Oct 2004 17:28:07 +0900
To: ietf-822 <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

Hello Bruce,

At 23:07 04/10/27, Bruce Lilly wrote:
 >On Tue October 26 2004 18:05, Keith Moore wrote:
 >
 >> Or a least, if a message is going to point to an
 >> archive of itself, it should be a faithful archive.
 >
 >That raises several new issues with Martin's draft.   They can be
 >summarized as "what is the utility of the proposed field; who
 >will use it, under what circumstances, and for what purpose(s)?".
 >Note that that is not a question about syntax, or about semantics,
 >or about validity, but about utility.
 >
 >I also discuss a security issue towards the end of this message.
 >
 >Some details about the utility issues:

They have been discussed at length in other messages, so I
won't answer here again. However, given the fact that the
question of usage has come up over and over again, I have
added a new section, "Usage Considerations", to my internal
draft.

It currently reads:

 >>>>>>>>
It may at first seem strange to have a pointer to an archived form of a 
message in a header field of that same message. After all, if one has the 
message, why would one need a pointer to it? It turns out that such 
pointers can be extremely useful. This section describes some of the 
scenarios for their use.

One may want to refer to messages in a non-message context, such as on a 
Web page or in an instant messaging context. In such a case, being able to 
take the URI out of the Archived-At header avoids having to search the 
correct message in the archive.

One may want to refer to other messages in a message context. While 
refering to a single message is often done by replying to that message, 
when referring to more than one message, providing a pointer to these 
messages is a widespread practice. The Archived-At header makes it easier 
to provide these pointers.

One may want to find related messages, for which one may need to go to the 
archive if one has not received potentially related messages. Also, it may 
often be easier to find related messages in an archive than in an email 
client. The Archived-At header makes it easier to find the starting point 
in the archive to find related messages.

Please note that in the above usage scenarios, it is mostly the human 
reader, rather than the email client software, that makes use of the URI in 
the Archived-At header. However, this does not rule out the use of the URI 
in the Archived-At header by the email client or other software if such use 
is found helpful.
 >>>>>>>>

Comments appreciated (actual suggested text preferred).

 >2. There is only one circumstance that I can think of where one
 >    might have the proposed field but not the entire message. And
 >    that depends for proper operation on specification of issues
 >    which are not currently specified in the draft under
 >   discussion.  The situation would be a message/partial fragment
 >   numbered 1 containing the encapsulated message header in the
 >   absence of one or more fragments (RFC 2046).

Thanks for providing these details. I'm personally not really
very much concerned with split-up messages, and so I would
be fine with some short warning saying that the Archived-At
header field may not work as expected in the case of message/
partial. Alternatively, I can say that in the case of split/
reassembled messages, one might get two headers, one pointing
to the original whole message, and another to the first part,
and one should be careful to take the right one.

 >If a
 >message is so large that fragmentation would occur, putting the
 >message on a server and sending retrieval instructions would be
 >preferable in most instances, and there are existing mechanisms
 >to handle that scenario (simply sending a URI as text, or using
 >message/external-body).

Agreed! If you think that's helpful, I can say that the intent
of the Archived-At header isn't to replace these mechanism.

 >Providing a pointer to a message from
 >some media other than that message is not unreasonable, and
 >there are a number of standardized mechanisms for that. However,
 >either something is fundamentally wrong with the concept of
 >putting a pointer to a message within the referenced message or
 >the author needs to provide some explanation in the draft as to
 >how and why he deems that to be a useful capability.

I don't think I had to add these explanations, but I have
added something anyway, see above.

 >Self-reference can lead to security issues.

Thanks for mentioning that, I hadn't thought about it.
I have added a paragraph reading:

 >>>>>>>>
Because the Archived-At header field points to some archived form of the 
message itself, which in turn may contain the Archived-At field, there is a 
potential for a denial-of-service attack on the server pointed to by the 
URI in the Archived-At header field if the archived form of the message is 
downloaded automatically, and further URIs in that message are followed and 
downloaded recursively without checking for already downloaded resources. 
However, this kind of scenario can easily be avoided by implementations. 
First, the URI in the Archived-At header field should not be dereferenced 
automatically, and second, appropriate measures for loop detection should 
be used.
 >>>>>>>>

to my internal draft. Any suggestions for improvement are welcome
(actual text highly preferred).

Given that the Archived-At header should usually not be downloaded
automatically, and that even if that would be found useful in some
case, the denial-of-service attack can still easily be avoided
by a reasonable implementation, and that on top of that, it's not
in the interest of the downloader to waste resources either,
I think having this warning in the security section should be
good enough to cover this point.

If you know about other security issues, again, actual text
I can put into the draft directly is preferred.

Regards,    Martin. 

Date: Thu, 28 Oct 2004 16:21:52 +0900
To: Keith Moore <[email-masked]>, Graham Klyne <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked], [email-masked]
Sender: [email-masked]

At 02:35 04/10/28, Keith Moore wrote:

 >I guess the question is this - do you want the archived email messages to
 >be usable as email messages, or do you want them to be a part of the web?
 >It's very difficult to do both, because web browsers tend not to support
 >message/rfc822 very well.

They wouldn't have to. Web browsers can easily be configured to hand
off documents of a specific media type to another, external, application.
Assuming that your email client can handle message/rfc822, you would just
set that as that external application.

 >I'm not surprised if W3C is good at making email messages a part of the
 >web.

Thanks! [I'm actually not sure we are that good; things could work
even better.]

 >But it also shouldn't be surprising if people who care about email
 >want archived email messages to have the functionality we've come to
 >expect from email messages.

I'm not surprised. I think it's a good thing. I just don't think
you should impose your expectations on others.

Regards,    Martin. 

Date: Thu, 28 Oct 2004 15:54:21 +0900
To: ietf-822 <[email-masked]>, ietf-822 <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

At 04:41 04/10/28, Bruce Lilly wrote:
 >On Wed October 27 2004 13:48, Arnt Gulbrandsen wrote:

 >> When I do, that field is useful. So... is it highly unusual to want to
 >> link to a message in a mail archive?
 >
 >Perhaps not, but if one is already looking at the archive, one
 >can simply copy and paste the URI of the message in that archive
 >rather than grubbing through the message content itself for the
 >field, stripping brackets, etc.

Yes, of course.

 >And if one has the entire message
 >available (as is necessarily the case with Archived-At), it is just
 >as easy to forward the message or quote relevant parts of it.

Quoting sometimes helps, but is really a separate issue; the record
of where something was said is often as important as what was said,
and pointing to something can reduce the problem of quoting out of
context. As for forwarding, that may work with a single message,
but doesn't work e.g. for minutes of a meeting with dozens of
pointers to other messages. It also does not work if one wants
to link from a Web page to a message.

 >Or it can be referenced by message-id.

Referencing by message-id requires that either your correspondent(s)
have received that message, and have an easy way to access it in
their MUA (some MUAs may allow that, and more should allow it,
but I don't know that many actually do), or have a way to find
an arbitrary message archived somewhere on the Internet by messageid.
To implement the former is relatively easy, for the later, Google
may help, but it's not being updated fast enough.

Regards,    Martin. 

Date: Thu, 28 Oct 2004 16:08:12 +0900
To: ietf-822 <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

At 04:30 04/10/28, Bruce Lilly wrote:
 >On Wed October 27 2004 13:38, Keith Moore wrote:
 >
 >> 1. I can certainly see using this field as providing a way to pass
 >> along a message to others (who might not have received a copy of the
 >> message) without actually having to send them a copy.  Instead of
 >> forwarding the message to them, paste the URL into the message text.
 >
 >Well I suppose it is possible.  It sure sounds like a heck of a lot of
 >effort compared to simply invoking a "forward message" function
 >in an MUA --

Yes, but can you forward more than one message in a single message?

 >first convince the MUA to display the Archived-At
 >field, then select the text (w/o the angle brackets) and copy it
 >somewhere, start composition of a new message, paste the copied
 >URI text into the message body (if possible; one may have to resort
 >to re-keyboarding it), edit the URI if it was line-folded in the field,
 >fill out the message header fields, add text explaining that the copied
 >text is a URI, then send the message.  At the receiving end, the
 >recipient will have to go through similar contortions to make use of
 >that URI -- figure out what the URI means, undo any line folding,
 >quoted-printable encoding, etc., guess what application might
 >support the particular scheme, fire up an instance of that
 >application, copy and paste (or re-keyboard) the URI into that
 >application. And to make use of the retrieved message -- assuming
 >that everything has worked flawlessly up to that point, that
 >application may have to save the message to a file and the user
 >may have to open or import that file in an MUA to make use of it.

The description above assumes worst-case scenarios. Many mail
clients these days make it very easy to click on an URI in header
or body. Operating systems are configured to invoke the right
application, depending on the URI scheme. And so on. The copy/paste
is still tedious, but I can assure you that it's a lot less
tedious with the Archived-At header than before, where one had
to hunt-and-peck through an archive to find the right mail.

 >Yes, context is useful, but that context is provided by the
 >related messages and the archive infrastructure (or, if I have
 >access to the messages in an MUA, by that MUAs features).
 >List-Archive can point to such an archive.  Nor is Archived-At
 >necessary to get a starting point at a particular message;
 >a URI could be obtained from any suitable mechanism,

Could you be a bit more specific here?

 >and since Archived-At is defined only for a specific message, it
 >seems rather wasteful to send an entire message if the only
 >thing desired is a URI.

Sorry, but I don't get that. If the only thing desired is an URI,
you send the URI. The message is sent with the Archived-At because
the recipient wants to read the message, AND be able to put a link
to it in another mail or web page, or tell somebody else to look
at it.

 >There are also other ways to
 >identify a particular message; given that message's
 >msg-id, one could find it in the archive via IMAP's (or
 >some archive interface's) search function, possibly
 >directly via an MID URI.

One could, of course. But clicking on a link is so much
faster than having to find the right search page and copy/paste
the MID.

 >I think you've made a start at identifying some uses.

I'm not concerned about identifying uses. I'm very sure
there are some good uses, because I'm using it daily,
as do other people such as Graham.
What we probably have made a start at is at getting you
to understand these uses.

 >However,
 >I'm still concerned about the imbalance between the rather
 >limited utility within the framework

Within what 'framework'? The framework for IETF specs is
the Internet, and in that framework, it's definitely very
usable.

 >(viz. Archived-At is only
 >available when the entire message is available (excluding the
 >unusual combination of circumstances mentioned earlier today))
 >vs. the incompatibility with the existing message/partial
 >fragmentation/reassembly algorithm

Could you be more specific about this?

 >and implementations and the security issues.

I'll reply to these separately.

Regards,    Martin. 

Date: Thu, 28 Oct 2004 15:42:49 +0900
To: Keith Moore <[email-masked]>, ietf-822 <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 05:50 04/10/28, Keith Moore wrote:

 >I could imagine automatically adding Archived-At fields to about 90% of
 >my incoming list mail, and having the URIs in those fields point to my
 >own personal archives of those messages.  That way, I could easily
 >reference messages in those archives when corresponding with others.

Interesting idea. It would of course mean that (part of) your
email archives would be (publicly) available over the network.

Regards,    Martin. 

Date: Thu, 28 Oct 2004 15:39:08 +0900
To: Keith Moore <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked], [email-masked]
Sender: [email-masked]

At 09:34 04/10/28, Keith Moore wrote:
 >
 >>  >The reason you haven't seen it implemented is that you are looking
 >>  >at
 >> archives with web browsers.  Internet email readers don't currently
 >> have a way of accessing archives (except perhaps for read-only IMAP),
 >> so there aren't many archives that are set up for use by email
 >> readers.  But all internet email readers support message/rfc822 to
 >> some degree, whereas not all of them support HTML or other formats.
 >>
 >> You are essentially just confirming what I said: It's not implemented.
 >> The fact that I'm using a browser isn't some kind of bias, it's just
 >> currently the only way it works.
 >
 >That's an odd way of looking at it.

Well, somewhat, but in terms of "rough consensus and running code"
not really.

 >If you're trying to define a
 >new extention to email, presumably to be used by email clients,
 >_of course_ it's not already implemented by email clients.

It's probably only partially useful for email clients. Indeed Bruce
Lilly makes a rather good point that it is not useful at all for
email clients (he however makes the mistake of thinking that
email clients are everything that counts). It's very useful for
humans. Is there a rule that email headers have to be used by
email clients? As an example, how much is the X-Mailer: header
used my email clients?

 >This is
 >the normal situation when we're defining a protocol extension.

As far as I understand the way the IETF works, there is often
already some implementation experience. In the case at hand,
this is certainly true.

 >And of course native format archives _do_ exist

Good. But the fact that they exist doesn't mean that we have
to try and force everybody to provide them.

 >and at least some
 >email clients are already able to download files using http and/or
 >ftp.

But that's just files in general, yes? Or is is message/rfc882 in
particular? And if the later, what clients would that be?

 >so the unimplemented part of this is rather small.

Agreed.

 >>  >If you are designing a way to allow email messages to reference
 >>  >archives,
 >> it makes good sense to make those messages usable by email readers.
 >>
 >> There is absolutely nothing in the design or description of
 >> Archived-At that would prevent doing that. It's all a question of
 >> implementation.
 >
 >I see it as a question of interoperability.  If people get the idea that
 >it's appropriate to set Archived-At fields to point to HTML documents,
 >then (a) a number of email readers won't be able to read them, and (b)
 >even those email readers that can read them won't be able to treat them
 >as email messages.  This is pretty dysfunctional - it results in yet
 >another new feature that is not very flexible.  Sooner or later we'll
 >be trying to define Yet Another Header Field that does the same
 >thing as Archived-At except that the archives are actually intended
 >to be accessible by email readers.  (And we'll be bothered by the
 >fact that the Archived-At field name is already taken.)

Mentioning various media types and http content negotiation, I think
it should be quite clear from my draft that there is no particular
expectation one way or another. That's very similar to the HTML
spec not saying what type of document you should find at the end
of an anchor. There is quite some expectation in that case that
that might be another HTML document, but there are lots of other
cases.

Although W3C, as an example, currently does not serve the archived
mails in message/rfc822 format, if we did it, we would certainly
do the right thing to make sure that we would not need additional
headers (e.g. two Archived-At, one for text/html and another for
message/rfc822 format), let alone a new field name.

There is another aspect here: Assume that one of the reasons we
want a message/rfc822 format is for replies to archived messages.
In that case, having a pointer from the actual message to the
archive in order to fetch the message from the archive and then
answer to it seems heavy overkill, because the message is already
available locally and can be replied to directly. So it is possible
to immagine usage patterns where the Archived-At header points to
a HTML version, and that HTML version includes a link to the
message/rfc822 version, for use by people who don't have the
original message. Of course, this is only one possible usage
pattern, the draft doesn't require that at all.

 >> If you think this is that important, why don't you go and add such
 >> a feature to an open-source MUA, or convince somebody to do that?
 >> Or add such a feature to some mailing-list expander/archival software?
 >
 >I'm quite willing to do so.

Great!

 >I just don't want to wait until that code
 >is written and shipped before making comments about your proposal.

You don't have to. My proposal clearly allows message/rfc822 format,
and based on your comments to my -00 draft, it explicitly mentions
that.

What I don't want is to prescribe to archive maintainers how they
should serve their archived messages. If there is a good reason to
use message/rfc822, let them do it. If there is a good reason to
serve as HTML, let them do it. If there is a good reason to use
something else, let them do it. And let them use more than one
of these if they have good reasons for that, too.

 >(Actually if we were still using my list software here, I would have
 >already done the latter.)

Great!

 >Being able to compare one's own copy of a message with the
 >archived version might be marginally useful, but not very often.

Agreed.

 >There are other cases also.  For instance, a user receives a
 >message that has been highly modified in transit - say some of the
 >attachments have been stripped off by a media filter because the
 >recipient's cell phone wasn't expected to be able to display
 >PDF documents.  I expect this kind of filtering to become more
 >common in the future as more and more "nontraditional" devices
 >are used to access email and there is more and more use of email
 >technology to send faxes and voice mail.  Anyway, archived-at
 >would allow the recipient to access an archived/unmodified
 >copy of the message.

Yes, that's another good example.

Regards,    Martin. 

Date: Thu, 28 Oct 2004 09:39:41 +0900
To: ietf-822 <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-02.txt
Sender: [email-masked]

At 21:58 04/10/27, Bruce Lilly wrote:
 >On Mon October 25 2004 01:58, Martin Duerst wrote:

 >>        By submitting this Internet-Draft, each
 >>     author represents that any applicable patent or other IPR claims of
 >>     which he or she is aware have been or will be disclosed, and any of
 >>     which he or she
 >[...]
 >
 >As only one author is listed, and under the assumption that the
 >gender of that author is not indeterminate, the repeated use of
 >"he or she" is quite tedious.

Good point. But that's boilerplate, produced automatically and
required in this exact form. If you feel strongly about this,
you need to take it up with the IPR list ([email-masked]).

Regards,    Martin. 

Date: Thu, 28 Oct 2004 11:19:13 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 <[email-masked]>
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 <[email-masked]>
Sender: [email-masked]

On Oct 27 2004, Bruce Lilly wrote:
> On Mon October 25 2004 23:37, Laird Breyer wrote:
> 
> > The problem with full MIME authentication is that we don't have the
> > trust networks and ubiquitous public key infrastructures this would need,
> > so in practice authentication would simply not be used.
> 
> PGP/MIME requires very little.  And authentication can be

I think you underestimate the problem. Building and maintaining a
public key infrastructure, offering verification and revocation
facilities, making sure that all programs that need it have access to
it, setting up central internet wide root authorities acceptable to
all, keeping complex authority hierarchies up to date, reloving
conflicts...

PGP between two people who exchange public keys on a diskette is one
thing, PGP for ubiquitous use by mail transport systems and associated
filtering and viewing progams, even if it is only within a large
organization, is a heavyweight administration problem.

> a useful tool to fight spam. If i receive an authenticated
> message from a known party who isn't clueless, I can be
> sure that it isn't spam. 

I doubt that. The current trend is well on its way to the following
model: ordinary, trustworthy people send spam to strangers without
knowing it. How does authentication help in that model? You'll know
*who* sent the spam, but you can't stop them all (for if you did, you
will have solved a good part of the virus problem).

If you want to restrict yourself to non-clueless users, then how do
you verify that a person sending you mail isn't clueless? And will you
blacklist the others?

> If I receive a signed message with a valid signature that turns out
> to be spam, the credentials associated with the signature may
> provide a means of identifying the spammer.

I don't see why it should. A trojan on a Windows box has access to
both the system API, if it wants to send mail programmatically, and
also access to the GUI by simulating mouse and keypresses.  Together,
these two methods can bypass most protections, ie anything a user can
do a trojan can do, too.

So in particular it can sign its messages with the user's private key
if the user has one. Then you could trace the spam back to the user
and that's it. 

> > If you can verify that the header was added after the message went
> > through a certain machine, and if you trust your system admin when
> > he says he's got SpamAssassin in vanilla configuration along that 
> > segment of the message path, then you can trust the header.
> 
> He's not necessarily *my* "system admin"!  Or *any* "system
> admin". He may in fact be a spammer who has forged Received
> fields and your "Processed" field.  I would have to know a great
> deal about that "certain machine", and I would require much
> stronger authentication than what you're indicating to be certain
> that the message in fact "went through" that "certain machine"
> and is not simply a message with a few easily-forged fields.

The method proposed is not an oracle. It doesn't give a truth value
to the proposition "is the Processed field trustworthy?", instead
it gives a truth value to the proposition "given that the Received
line is trustworthy, is the Processed field trustworthy?"

With the latter proposition, you have full control, on a case by case
basis, over which Received lines you want to trust, and consequently
which Processed lines you can trust. How could this be improved?

> 
> > In a futuristic MUA, you would get a message stating: "I detected
> > a filter called SpamAssassin which operates inside the third hop
> > from the end, would you like me to use its recommendation?"
> 
> My answer would have to be that I cannot make that
> determination without knowing details of the configuration
> and run-time options, and without knowing exactly what
> site is meant by "the third hop from the end" and without a
> considerable amount of effort to determine if that site is in
> fact what is claimed, and unless I can verify that that site
> in fact added the fields that have been claimed to have been
> added there, and unless I have absolute trust in that site.

That is an excellent answer. The MUA told you it found a 
piece of information you might wish to use, and you told it
that you don't trust that information. Mission accomplished!

> 
> No, because it doesn't provide configuration information.

That's a straw man. None of the headers in any mail message, ever,
carry full configuration information. Do you or any recipient know how
exim or Postfix were configured when they added their trace fields to
transported messages? How about the sender's MUA configuration when the
message was composed?

But you (or anybody) still routinely make decisions based on the
unreliable information written by the above programs. For example, you
decide to read messages addressed to you, you decide to reply to the
mailbox listed in a Reply-To fields etc. even though you have no idea
what configuration the software which created those fields was in.

-- 
Laird Breyer.

Date: Wed, 27 Oct 2004 20:34:30 -0400
From: Keith Moore <[email-masked]>
To: Martin Duerst <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

>  >The reason you haven't seen it implemented is that you are looking
>  >at 
> archives with web browsers.  Internet email readers don't currently
> have a way of accessing archives (except perhaps for read-only IMAP),
> so there aren't many archives that are set up for use by email
> readers.  But all internet email readers support message/rfc822 to
> some degree, whereas not all of them support HTML or other formats.
> 
> You are essentially just confirming what I said: It's not implemented.
> The fact that I'm using a browser isn't some kind of bias, it's just
> currently the only way it works.

That's an odd way of looking at it.  If you're trying to define a 
new extention to email, presumably to be used by email clients, 
_of course_ it's not already implemented by email clients.  This is
the normal situation when we're defining a protocol extension.

And of course native format archives _do_ exist and at least some
email clients are already able to download files using http and/or
ftp.  so the unimplemented part of this is rather small.

>  >If you are designing a way to allow email messages to reference
>  >archives, 
> it makes good sense to make those messages usable by email readers.
> 
> There is absolutely nothing in the design or description of
> Archived-At that would prevent doing that. It's all a question of
> implementation.

I see it as a question of interoperability.  If people get the idea that
it's appropriate to set Archived-At fields to point to HTML documents,
then (a) a number of email readers won't be able to read them, and (b) 
even those email readers that can read them won't be able to treat them
as email messages.  This is pretty dysfunctional - it results in yet
another new feature that is not very flexible.  Sooner or later we'll
be trying to define Yet Another Header Field that does the same
thing as Archived-At except that the archives are actually intended
to be accessible by email readers.  (And we'll be bothered by the 
fact that the Archived-At field name is already taken.)

> If you think this is that important, why don't you go and add such
> a feature to an open-source MUA, or convince somebody to do that?
> Or add such a feature to some mailing-list expander/archival software?

I'm quite willing to do so.  I just don't want to wait until that code
is written and shipped before making comments about your proposal.

(Actually if we were still using my list software here, I would have
already done the latter.)

> I think one of the major misunderstandings here is how (X-)Archived-At
> is currently used at W3C (and as Graham has confirmed again, it is
> indeed used a lot, and appreciated a lot). It is not used by MUAs, it
> is used by humans. A typical case is that during a teleconference,
> somebody mentions an email message. That person has that message in
> front of them in their MUA, but want others to look at it too, with as
> little a delay as possible. The best way to do that is to take the URI
> in the Archived-At header and paste it into the IRC channel used for
> the teleconference.
> 
> Another frequent use case is that we maintain a lot of Web pages (e.g.
> about issue resolution, or for minutes of meetings,...) with links to
> archived email messages. To update these, it's often very helpful to
> start with the URI in the Archived-At header.
> 
> In both cases, this is done by humans, not by the MUA. Apart from it
> not being implemented, there wouldn't actually be a need for an MUA
> to follow the Archived-At link, because the MUA already has the
> message itself, in raw (i.e. message/rfc822) form.

As far as I can tell, the major utility in having email readers be
able to use Archived-At would that it would enable email users to
look at other messages in the surrounding context - predecessors to that
message and replies to that message, etc.  You really want a pointer
into a folder.  This would have been immensely useful to me when
I was on IESG, because I was constantly having to dig through list
archives and try to  understand conversations that had taken place and
the circumstances that produced some message or another.

Being able to compare one's own copy of a message with the 
archived version might be marginally useful, but not very often.

There are other cases also.  For instance, a user receives a 
message that has been highly modified in transit - say some of the
attachments have been stripped off by a media filter because the
recipient's cell phone wasn't expected to be able to display 
PDF documents.  I expect this kind of filtering to become more
common in the future as more and more "nontraditional" devices
are used to access email and there is more and more use of email
technology to send faxes and voice mail.  Anyway, archived-at 
would allow the recipient to access an archived/unmodified 
copy of the message.

Thanks for providing concrete explantions of how W3C folks
use x-archived-at.

Keith

Date: Thu, 28 Oct 2004 09:02:48 +0900
To: Keith Moore <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: ietf-822 list <[email-masked]>
Sender: [email-masked]

Hello Keith,

[sorry it took me more than a day to complete this message]

At 07:27 04/10/27, Keith Moore wrote:

 >> As explained in my answer to Charles, I don't want to recommend stuff
 >> that I haven't seen implemented.
 >
 >The reason you haven't seen it implemented is that you are looking at 
archives with web browsers.  Internet email readers don't currently have a 
way of accessing archives (except perhaps for read-only IMAP), so there 
aren't many archives that are set up for use by email readers.  But all 
internet email readers support message/rfc822 to some degree, whereas not 
all of them support HTML or other formats.

You are essentially just confirming what I said: It's not implemented.
The fact that I'm using a browser isn't some kind of bias, it's just
currently the only way it works.

 >>  I think making messages available as
 >> message/rfc822 is a great idea at least on paper, but in W3Cs case,
 >> things are currently working quite well with serving the messages
 >> just as HTML,
 >
 >Apparently you don't expect much from email archives.  IMHO you should be 
able to do anything with an archived message that you can do with a message 
you received in your inbox - read it with any mail reader, reply to it, 
forward it (as a message) to somewhere else, search it as if it were a 
message, display or save any of its body parts, follow threads to other 
messages (including those in your inbox), etc.  In addition, HTML has 
proven to be a poor format for use in email because of numerous security 
risks associated with scripts, objects, and referenced images along with 
the expectation that these are invoked or displayed without user consent.

[We are not using HTML for email.]

 >If you are designing a way to allow email messages to reference archives, 
it makes good sense to make those messages usable by email readers.

There is absolutely nothing in the design or description of Archived-At
that would prevent doing that. It's all a question of implementation.
If you think this is that important, why don't you go and add such
a feature to an open-source MUA, or convince somebody to do that?
Or add such a feature to some mailing-list expander/archival software?

Mind you, if I had enough free time, I'd maybe try to do it myself.
I actually agree very much with most of what you are saying, and
I have also been trying to convince our system people to help.
The responses I got so far are things like "on the someday pile,
below a lot of other stuff" and "maybe sounds cool, but we don't
have the resources for stuff that just sounds cool, but where
we don't see any actual use cases". If there were a couple
MUAs that actually could use message/rfc822, that could easily
change.

 >> and I don't want to give potential users the impression
 >> that they have to support message/rfc822 or they better won't use
 >> Archived-At at all.
 >
 >That's very close to the the impression that they should get.  More precisely:
 >
 >a) those who use archived-at should take pains to archive their messages 
faithfully (meaning in the original format) and in such a way as to be 
readable by the vast majority of mail readers (i.e. in message/rfc822), and
 >
 >b) If you want interoperability the last thing you want to do is to 
burden email readers with having to support arbitrary formats for archived 
messages.

I think one of the major misunderstandings here is how (X-)Archived-At
is currently used at W3C (and as Graham has confirmed again, it is indeed
used a lot, and appreciated a lot). It is not used by MUAs, it is used by
humans. A typical case is that during a teleconference, somebody mentions
an email message. That person has that message in front of them in their
MUA, but want others to look at it too, with as little a delay as possible.
The best way to do that is to take the URI in the Archived-At header and
paste it into the IRC channel used for the teleconference.

Another frequent use case is that we maintain a lot of Web pages (e.g.
about issue resolution, or for minutes of meetings,...) with links to
archived email messages. To update these, it's often very helpful to
start with the URI in the Archived-At header.

In both cases, this is done by humans, not by the MUA. Apart from it
not being implemented, there wouldn't actually be a need for an MUA
to follow the Archived-At link, because the MUA already has the
message itself, in raw (i.e. message/rfc822) form. The only case
where I can immagine the MUA going to check an archive of a message
it already has is some kind of verification scenario, but I'm not
exactly clear when that would really be needed, and whether there
are not better ways to do that (e.g. digital signatures,...). But
maybe you have some ideas.

This doesn't mean that it wouldn't be a good idea to have the message
available in message/rfc822 form. The main use case I can immagine
at the moment is to respond to a message that one sees in the archive.
Our archives currently have a [Respond] button
(see e.g. http://lists.w3.org/Archives/Public/uri/2004Oct/0017.html),
but that uses the mailto: URI, where it's practically impossible to
squeeze the whole message in. We try to do a reasonable job with
the mailto: URI; for the above message, e.g. it's
mailto:uri%40w3.org?Subject=Re%3A%20editorial%20suggestions%20for%20RFC%202396%20bis&amp;In-Reply-To=&lt;0I5U00G08DFGCR%40mailsj-v1.corp.adobe.com&gt;&amp;References=&lt;0I5U00G08DFGCR%40mailsj-v1.corp.adobe.com&gt;
i.e. it contains quite a bit of information that should allow to
do reasonable threading. But with better MUAs, it could certainly
work better.

I hope that these explanations helps you to understand the actual,
very much in-use and prooven use cases for Archived-At.

Regards,    Martin. 

Date: Wed, 27 Oct 2004 16:50:16 -0400
From: Keith Moore <[email-masked]>
To: ietf-822 <[email-masked]>, Martin Duerst <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

> On Wed October 27 2004 13:38, Keith Moore wrote:
> 
> > 1. I can certainly see using this field as providing a way to pass 
> > along a message to others (who might not have received a copy of the
> > 
> > message) without actually having to send them a copy.  Instead of 
> > forwarding the message to them, paste the URL into the message text.
> 
> Well I suppose it is possible.  It sure sounds like a heck of a lot of
> effort compared to simply invoking a "forward message" function
> in an MUA 

URLs to message archives are useful for references.  I could send a
message to you referring to another message.   You could choose to view
that message or not depending on your needs. Perhaps you might have seen
that message already and remember it, or perhaps you might believe what
I have to say without consulting the references. 

Here's how I see using this:  I want to send a message to you that
includes some references.  So I go and find those references - 
in my own personal email folders, or in some public archives, or
whereever.  For each of those references that are in my personal
email folders, if they have archived-at fields, I can use the URIs
in those fields as references in the message I send to you.

> > 2. Another thing that would be extremely useful is to be able to see
> > a message in context - in relation to other messages in the same 
> > discussion.  So if an archive points to a message in a read-only
> > IMAP mailbox, the context could be supplied by other messages in
> > that mailbox.  Or if an archive points to a HTML document on an HTTP
> > server, the context could be supplied by "next in thread" style
> > links.  In this case the Archived-at field serves as much to point
> > to a collection of messages as to a particular message within that
> > collection.
> 
> Yes, context is useful, but that context is provided by the
> related messages and the archive infrastructure (or, if I have
> access to the messages in an MUA, by that MUAs features).
> List-Archive can point to such an archive. 

Yes, but list-archive doesn't indicate a particular message.  It's 
like being able to reference a multi-volume set of books without 
being able to indicate the relevant volume and page.

I could imagine automatically adding Archived-At fields to about 90% of
my incoming list mail, and having the URIs in those fields point to my
own personal archives of those messages.  That way, I could easily 
reference messages in those archives when corresponding with others.

Keith

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, Martin Duerst <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Wed, 27 Oct 2004 15:41:37 -0400
Cc: Arnt Gulbrandsen <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

On Wed October 27 2004 13:48, Arnt Gulbrandsen wrote:

> >>  Don't you ever type '<a href="' and then pause, realizing you want 
> >>  to link to a message that's somewhere in your mailbox?
> >
> > No, I don't; what's your point?
> 
> When I do, that field is useful. So... is it highly unusual to want to 
> link to a message in a mail archive?

Perhaps not, but if one is already looking at the archive, one
can simply copy and paste the URI of the message in that archive
rather than grubbing through the message content itself for the
field, stripping brackets, etc.   And if one has the entire message
available (as is necessarily the case with Archived-At), it is just
as easy to forward the message or quote relevant parts of it.
Or it can be referenced by message-id.

As mentioned in another message, I'm concerned about the
balance between utility, (in)compatibility, and security
(vulnerabilities).

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, Martin Duerst <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Wed, 27 Oct 2004 15:30:50 -0400
Cc: Keith Moore <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

On Wed October 27 2004 13:38, Keith Moore wrote:

> 1. I can certainly see using this field as providing a way to pass 
> along a message to others (who might not have received a copy of the 
> message) without actually having to send them a copy.  Instead of 
> forwarding the message to them, paste the URL into the message text.

Well I suppose it is possible.  It sure sounds like a heck of a lot of
effort compared to simply invoking a "forward message" function
in an MUA -- first convince the MUA to display the Archived-At
field, then select the text (w/o the angle brackets) and copy it
somewhere, start composition of a new message, paste the copied
URI text into the message body (if possible; one may have to resort
to re-keyboarding it), edit the URI if it was line-folded in the field,
fill out the message header fields, add text explaining that the copied
text is a URI, then send the message.  At the receiving end, the
recipient will have to go through similar contortions to make use of
that URI -- figure out what the URI means, undo any line folding,
quoted-printable encoding, etc., guess what application might
support the particular scheme, fire up an instance of that 
application, copy and paste (or re-keyboard) the URI into that
application. And to make use of the retrieved message -- assuming
that everything has worked flawlessly up to that point, that
application may have to save the message to a file and the user
may have to open or import that file in an MUA to make use of it.

> 2. Another thing that would be extremely useful is to be able to see a 
> message in context - in relation to other messages in the same 
> discussion.  So if an archive points to a message in a read-only IMAP 
> mailbox, the context could be supplied by other messages in that 
> mailbox.  Or if an archive points to a HTML document on an HTTP server, 
> the context could be supplied by "next in thread" style links.  In this 
> case the Archived-at field serves as much to point to a collection
> of messages as to a particular message within that collection.

Yes, context is useful, but that context is provided by the
related messages and the archive infrastructure (or, if I have
access to the messages in an MUA, by that MUAs features).
List-Archive can point to such an archive.  Nor is Archived-At
necessary to get a starting point at a particular message;
a URI could be obtained from any suitable mechanism, and
since Archived-At is defined only for a specific message, it
seems rather wasteful to send an entire message if the only
thing desired is a URI.  There are also other ways to
identify a particular message; given that message's
msg-id, one could find it in the archive via IMAP's (or
some archive interface's) search function, possibly
directly via an MID URI.

I think you've made a start at identifying some uses. However,
I'm still concerned about the imbalance between the rather
limited utility within the framework (viz. Archived-At is only
available when the entire message is available (excluding the
unusual combination of circumstances mentioned earlier today))
vs. the incompatibility with the existing message/partial
fragmentation/reassembly algorithm and implementations
and the security issues.

Date: Wed, 27 Oct 2004 19:48:09 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Bruce Lilly <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

Bruce Lilly writes:
> On Wed October 27 2004 11:05, Arnt Gulbrandsen wrote:
>
>>  Bruce Lilly writes, at magnificent and impressive length:
>>  > 1. Since the proposed field is contained within a message header
>>  >    and purportedly points to an archive of that message, it seems
>>  >    of little utility except in highly unusual circumstances (see below).
>>
>>  Don't you ever type '<a href="' and then pause, realizing you want 
>>  to link to a message that's somewhere in your mailbox?
>
> No, I don't; what's your point?

When I do, that field is useful. So... is it highly unusual to want to 
link to a message in a mail archive?

Arnt

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, Martin Duerst <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Wed, 27 Oct 2004 13:46:36 -0400
Cc: Arnt Gulbrandsen <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

On Wed October 27 2004 11:05, Arnt Gulbrandsen wrote:
> 
> Bruce Lilly writes, at magnificent and impressive length:
> > 1. Since the proposed field is contained within a message header
> >    and purportedly points to an archive of that message, it seems
> >    of little utility except in highly unusual circumstances (see below).
> 
> Don't you ever type '<a href="' and then pause, realizing you want to 
> link to a message that's somewhere in your mailbox?

No, I don't; what's your point?

From: Bruce Lilly <[email-masked]>
Reply-To: Bruce Lilly <[email-masked]>, Martin Duerst <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Wed, 27 Oct 2004 13:44:29 -0400
Cc: Graham Klyne <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

On Wed October 27 2004 12:58, Graham Klyne wrote:
> At 10:07 27/10/04 -0400, Bruce Lilly wrote:
> > > Or a least, if a message is going to point to an
> > > archive of itself, it should be a faithful archive.
> >
> >That raises several new issues with Martin's draft.   They can be
> >summarized as "what is the utility of the proposed field; who
> >will use it, under what circumstances, and for what purpose(s)?".
> >Note that that is not a question about syntax, or about semantics,
> >or about validity, but about utility.
> 
> Utility?  That's easy!  The system described by Martin's draft

The question is not about the w3 list archive, it is about
the specific proposal in Martin's draft (which mentions the
w3c list only by way of example).

Please reread the question and the draft.  The draft title
and topic pertain to a message header field; one that is
supposed to provide information pointing to an archived
copy of that very message.  Now please explain the
utility *of such a field*, given that if I already have the
field, I already have the message that it supposedly
references.

Date: Wed, 27 Oct 2004 13:38:15 -0400
From: Keith Moore <[email-masked]>
To: ietf-822 <[email-masked]>, Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

> That raises several new issues with Martin's draft.   They can be
> summarized as "what is the utility of the proposed field; who
> will use it, under what circumstances, and for what purpose(s)?".
> Note that that is not a question about syntax, or about semantics,
> or about validity, but about utility.

okay, let's talk about utility.

1. I can certainly see using this field as providing a way to pass 
along a message to others (who might not have received a copy of the 
message) without actually having to send them a copy.  Instead of 
forwarding the message to them, paste the URL into the message text.

2. Another thing that would be extremely useful is to be able to see a 
message in context - in relation to other messages in the same 
discussion.  So if an archive points to a message in a read-only IMAP 
mailbox, the context could be supplied by other messages in that 
mailbox.  Or if an archive points to a HTML document on an HTTP server, 
the context could be supplied by "next in thread" style links.  In this 
case the Archived-at field serves as much to point to a collection
of messages as to a particular message within that collection.

Date: Wed, 27 Oct 2004 13:35:33 -0400
From: Keith Moore <[email-masked]>
To: Graham Klyne <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

> Utility?  That's easy!  The system described by Martin's draft is extremely 
> useful.  It is implemented by the w3.org mailing list systems, and in my 
> work using those mailing lists I've made frequent and much-appreciated use 
> of the feature.

Maybe you could provide some concrete examples?

> I find it faintly ironic that one of the best (as in most usable and most 
> useful) mailing list services I've used anywhere on the Internet is run by 
> the World Wide *Web* Consortium.  The archive system is a major factor in 
> that usability.  Mail archives really take on additional value when they 
> become a fully linked and cross-linked part of the Web.  For this to really 
> work well --as it does-- the messages really need to be served using a 
> hypertext format, such as HTML.  I've no objection if others would like to 
> design and build systems that serve messages using whatever format they 
> like, but to claim that the HTML-served messages are in some sense not 
> useful or non-functional is to ignore the daily experience of the many 
> people who use them.

I guess the question is this - do you want the archived email messages to
be usable as email messages, or do you want them to be a part of the web?
It's very difficult to do both, because web browsers tend not to support
message/rfc822 very well.

I'm not surprised if W3C is good at making email messages a part of the 
web.  But it also shouldn't be surprising if people who care about email 
want archived email messages to have the functionality we've come to 
expect from email messages.

Keith

Date: Wed, 27 Oct 2004 17:58:14 +0100
To: Bruce Lilly <[email-masked]>, ietf-822 <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Martin Duerst <[email-masked]>
Sender: [email-masked]

At 10:07 27/10/04 -0400, Bruce Lilly wrote:
> > Or a least, if a message is going to point to an
> > archive of itself, it should be a faithful archive.
>
>That raises several new issues with Martin's draft.   They can be
>summarized as "what is the utility of the proposed field; who
>will use it, under what circumstances, and for what purpose(s)?".
>Note that that is not a question about syntax, or about semantics,
>or about validity, but about utility.

Utility?  That's easy!  The system described by Martin's draft is extremely 
useful.  It is implemented by the w3.org mailing list systems, and in my 
work using those mailing lists I've made frequent and much-appreciated use 
of the feature.

I find it faintly ironic that one of the best (as in most usable and most 
useful) mailing list services I've used anywhere on the Internet is run by 
the World Wide *Web* Consortium.  The archive system is a major factor in 
that usability.  Mail archives really take on additional value when they 
become a fully linked and cross-linked part of the Web.  For this to really 
work well --as it does-- the messages really need to be served using a 
hypertext format, such as HTML.  I've no objection if others would like to 
design and build systems that serve messages using whatever format they 
like, but to claim that the HTML-served messages are in some sense not 
useful or non-functional is to ignore the daily experience of the many 
people who use them.

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Wed, 27 Oct 2004 17:05:32 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Bruce Lilly <[email-masked]>, Keith Moore <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

Bruce Lilly writes, at magnificent and impressive length:
> 1. Since the proposed field is contained within a message header
>    and purportedly points to an archive of that message, it seems
>    of little utility except in highly unusual circumstances (see below).

Don't you ever type '<a href="' and then pause, realizing you want to 
link to a message that's somewhere in your mailbox?

Arnt

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Organization: Bruce Lilly
To: Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-02.txt
Date: Wed, 27 Oct 2004 08:58:21 -0400
Cc: "ietf-822" <[email-masked]>
Sender: [email-masked]

On Mon October 25 2004 01:58, Martin Duerst wrote:

> I would highly appreciate any additional feedback and comments on
> the draft. Please copy me directly, as I'm not subscribed to this list.
[...] 
> =========================================================================
[...]
> Network Working Group                                          M. Duerst
> Internet-Draft                                       W3C/Keio University
> Expires: April 25, 2005                                 October 25, 2004
[...]
>        By submitting this Internet-Draft, each
>     author represents that any applicable patent or other IPR claims of
>     which he or she is aware have been or will be disclosed, and any of
>     which he or she
[...]

As only one author is listed, and under the assumption that the
gender of that author is not indeterminate, the repeated use of
"he or she" is quite tedious.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, Martin Duerst <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Wed, 27 Oct 2004 10:07:56 -0400
Cc: Keith Moore <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

On Tue October 26 2004 18:05, Keith Moore wrote:

> Or a least, if a message is going to point to an 
> archive of itself, it should be a faithful archive.

That raises several new issues with Martin's draft.   They can be
summarized as "what is the utility of the proposed field; who
will use it, under what circumstances, and for what purpose(s)?".
Note that that is not a question about syntax, or about semantics,
or about validity, but about utility. 

I also discuss a security issue towards the end of this message.

Some details about the utility issues:

1. Since the proposed field is contained within a message header
   and purportedly points to an archive of that message, it seems
   of little utility except in highly unusual circumstances (see below).
   If I already have the entire message at my disposal, why should
   I want a field that tells me where I can find a copy of that same
   message?   I already have the message, so there's little point in
   parsing it to find that field, parsing the field to get a URI, parsing
   that URI to get a scheme/authority/path/query/fragment, looking
   up the scheme to find a suitable protocol, locating the server,
   making a connection --  if not offline -- etc., only to eventually
   retrieve a copy of what I already have.  And if I don't have the
   message, I probably don't have that field.  In short, if I have the
   message, the field is superfluous, and if I don't have the message,
   I cannot make use of the field to get the message.  That is in
   contrast to something like message/external-body, where some
   content (which could well be an entire message or even a digest
   of messages) can be accessed via some specified mechanism(s)
   without having to already have the desired content at hand;
   i.e. one message contains information for obtaining a
   different message.

2. There is only one circumstance that I can think of where one
    might have the proposed field but not the entire message. And
    that depends for proper operation on specification of issues
    which are not currently specified in the draft under
   discussion.  The situation would be a message/partial fragment
   numbered 1 containing the encapsulated message header in the
   absence of one or more fragments (RFC 2046). Unlike RFCs
   2298 and 3798, the subject draft does not amend RFC 2046's
   rules regarding which header fields are reconstituted from the
   encapsulated message header vs. from the header of the first
   fragment message which encapsulates it.  Lacking such an
   amendment, RFC 2046's rules require that the field be taken
   from the enclosing message, which I believe is undesirable.
   Consider that the entire original message and that first
   fragment --  which are different messages -- might both be
   archived (probably at different places).  In that case, the
   message header of the first fragment will contain (at least)
   two of the proposed fields, one referring to an archive of the
   entire message and the other referring to an archive of just
   the first fragment message -- with no way to tell which is
   which.  And the resulting "reassembled" message will also
   contain those fields, one of which points to an archive of the
   complete message, while the other points to a completely
   different message (the first fragment).  And I would add that
   fragmentation/reassembly is not perfectly transparent, so
   neither archive will be a faithful copy of the "reassembled"
   message.  The alternative, which would have to be provided
   for by an amendment to RFC 2046 rules similar to that in
   RFCs 2298/3798, is that the proposed field would be obtained
   from the encapsulated message's header.  In either case (the
   unamended RFC 2046 rules which are problematic for the
   proposed field, or with an amendment), one would have the
   proposed field if and only if one had the first fragment message;
   it would be unavailable if that fragment was missing.  The
   combination of circumstances in which the proposed field
   might have some use is:
   a) the message would have to have had the field added
   b) the message would have to have been fragmented after
       the field was added
   c) one or more of the fragments, but not the first fragment,
       would have to be missing
   d) the purported archive of the complete message would
       have to in fact exist and be accessible
   e) for full utility, the archive would have to be in RFC [2]822
       format (see Keith's remarks)
   f) none of the differences which might be expected to exist
       between the archive and the reconstituted message (had
       all fragments been available) could be significant (Received
       or other trace fields, incidental details like time of arrival,
       etc.).
   g,h) fragmentation and reassembly should both have been
      performed in accordance with an amended set of rules;
      otherwise one is likely to retrieve a copy of an already-
      available fragment.
   That is rather a large number of necessary conditions, several
   of which are likely to be rare occurrences.

Issues of syntax, semantics, and validity aside, I'm not convinced
that the proposed field is useful except in a tiny fraction of cases
that result from a combination of unusual circumstances.  If a
message is so large that fragmentation would occur, putting the
message on a server and sending retrieval instructions would be
preferable in most instances, and there are existing mechanisms
to handle that scenario (simply sending a URI as text, or using
message/external-body).  Providing a pointer to a message from
some media other than that message is not unreasonable, and
there are a number of standardized mechanisms for that. However,
either something is fundamentally wrong with the concept of
putting a pointer to a message within the referenced message or
the author needs to provide some explanation in the draft as to
how and why he deems that to be a useful capability.

Self-reference can lead to security issues. Consider:

   From: [email-masked]
   To: [email-masked]
   Date: 27 Oct 2004 08:09:10 -0500
   Subject: don't try this at home
   Archived-At:<http://bogus.server.example.org/123xyz>
   Content-Type: message/external-body ; access-type=url ;
      name="http://bogus.server.example.org/123xyz"

  Content-Type: message/rfc822
  Content-ID: <[email-masked]>

  This is not really the body.

Note that this is a message/external-body message, and that
the URI in the proposed Archived-At field points to the same
location as the external body reference.  Such a message
constitutes a two-pronged denial of service attack by the
sender against the recipient and against the specified server.
Such an attack is of course possible w/o the Archived-At field;
Archived-At provides an additional attack vector.  The draft
does not specify whether automated use of the Archived-At
field to retrieve a message is prohibited or encouraged. If it
is not prohibited, network damage (viz. the denial of service
scenario specified here and the resulting network congestion)
can occur.  It is not difficult to see how such an attack can
be extended to a multi-pronged attack on multiple servers
(construct a loop of messages with external references);
the Archived-At fields need not refer to the same URI as the
external-body reference in the same message, and the number
of attack vectors is increased with each additional message
(and Archived-At field) involved.  Note that MIME has provision
for caching and requires a Content-ID field for use with
caching; The draft under discussion does not mention caching
at all and makes no provision for an identifier that might be
used to facilitate caching mechanisms.  Note that
message/external-body is not necessary for the security issue
mentioned to arise; any automatic external reference
mechanism with no provision for caching (or where
caching can be disabled or is ineffective) is vulnerable.
Text/html, message/http, application/pdf are examples of
other media types that have provision for external references.
Archived-At is inherently an external reference and
self-referential, which opens the door to this security issue
in the absence of any specific media type vulnerabilities.
Those media types are mentioned here to illustrate the
mechanisms, to show how the proposed field exacerbates
existing problems, and to facilitate discussion of mechanisms
provided to afford some degree of protection against the
vulnerabilities by properly-implemented MIME mechanisms --
mechanisms for which there is no corresponding provision
in the draft under discussion.

The bottom line is that if there is no utility for the proposed
field, there's not much point in pursuing the draft further.
Conversely if there is some utility, that needs to be clearly
described, and the next draft version should address
interaction with message/partial and should also more fully
address security issues.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>, [email-masked]
Subject: Re: a header authentication scheme
Date: Wed, 27 Oct 2004 08:39:58 -0400
Sender: [email-masked]

On Mon October 25 2004 22:13, Laird Breyer wrote:

> I hadn't realized that (I thought it was just for reading obsolete
> messages). Slightly off topic, I wonder why the trend would be towards 
> less precision in time stamps? Doesn't that go against the purpose of 
> time stamps in the first place?

Clocks often vary by many seconds between different hosts.
Processing of a single message may take many seconds (DNS
lookups, timeouts, congestion, etc.).  Forged Received lines
usually have timestamps that differ from genuine lines in the
same messages by many hours (or more).  Excessive precision
is neither necessary nor useful.  And the purpose of the time
stamp lines is (RFC 1123) "primarily intended for humans tracing
mail routes, primarily of diagnosis of faults".

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>, [email-masked]
Subject: Re: a header authentication scheme
Date: Wed, 27 Oct 2004 08:38:38 -0400
Sender: [email-masked]

--nextPart2906714.Wi0pTeurNi
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

On Mon October 25 2004 23:37, Laird Breyer wrote:

> The problem with full MIME authentication is that we don't have the
> trust networks and ubiquitous public key infrastructures this would need,
> so in practice authentication would simply not be used.

PGP/MIME requires very little.  And authentication can be
a useful tool to fight spam. If i receive an authenticated
message from a known party who isn't clueless, I can be
sure that it isn't spam. If I receive an unauthenticated
message claiming to be from a known party that always
sends signed messages, it's almost certainly spam. If I
receive a signed message that fails signature validation,
I can be fairly confident that it's a forgery. If I receive a
signed message with a valid signature that turns out to
be spam, the credentials associated with the signature
may provide a means of identifying the spammer.

> If you can verify that the header was added after the message went
> through a certain machine, and if you trust your system admin when
> he says he's got SpamAssassin in vanilla configuration along that=20
> segment of the message path, then you can trust the header.

He's not necessarily *my* "system admin"!  Or *any* "system
admin". He may in fact be a spammer who has forged Received
fields and your "Processed" field.  I would have to know a great
deal about that "certain machine", and I would require much
stronger authentication than what you're indicating to be certain
that the message in fact "went through" that "certain machine"
and is not simply a message with a few easily-forged fields.

> In a futuristic MUA, you would get a message stating: "I detected
> a filter called SpamAssassin which operates inside the third hop
> from the end, would you like me to use its recommendation?"

My answer would have to be that I cannot make that
determination without knowing details of the configuration
and run-time options, and without knowing exactly what
site is meant by "the third hop from the end" and without a
considerable amount of effort to determine if that site is in
fact what is claimed, and unless I can verify that that site
in fact added the fields that have been claimed to have been
added there, and unless I have absolute trust in that site.
=20
> > I have no idea what configuration files were used with it or
> > what run-time options were specified.  In short, purported results
> > of an indeterminate process with indeterminate configuration
> > and indeterminate run-time parameters allegedly run at a
> > remote site are of little value.
>=20
> That's the trouble we have today and why a standard filtering header
> (such as the Processed example I ued) would help.

No, because it doesn't provide configuration information.
And I don't really want 400 kB of configuration files added
to a 500 byte message.  Also "no" because it doesn't provide
information about run-time environment and options. Again
"no" because there is no guarantee that any claimed test was
actually performed. Yet again "no" because there is no
guarantee that the claimed results were in fact obtained. And
"no" once more because there is no guarantee that the claims
were inserted by the site where it has been asserted that those
claims have been inserted.  On top of which is a hesitant "maybe"
which is contingent on a great deal of knowledge about remote
sites and their administrative staff. =20

--nextPart2906714.Wi0pTeurNi
Content-Type: application/pgp-signature

--nextPart2906714.Wi0pTeurNi--

Date: Wed, 27 Oct 2004 15:15:18 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Keith Moore <[email-masked]>, Graham Klyne <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

Keith Moore writes:
> You can even do content-negotiation so that a client gets HTML or 822 
> depending on its preferences.

That's in draft-01 already.

Arnt

Cc: [email-masked], Graham Klyne <[email-masked]>, Martin Duerst <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Wed, 27 Oct 2004 08:54:10 -0400
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

> My preferred approach would be to give archived-at the same parameter 
> list that MIME fields have, and define a "content-type=x/y" parameter 
> naming a type expected to be available. For a list that provides 
> archives in two forms and uses fancy HTTP to provide the right one, 
> the field could be like this:
>
> Archived-At: ...; content-type=message/rfc822, content-type=text/html
>
> For a less fancy archive providing both forms under separate URLs, two 
> Archived-At fields could be used:
>
> Archived-At: ...; content-type=message/rfc822
> Archived-At: ...; content-type=text/html

I think this is needlessly complex.  Just provide the archives in 822 
format.  It's not as if existing archivers are going to be used without 
changes anyway - some modifications are going to be necessary in any 
event to add the new header field to outgoing messages.  If those 
archives don't already keep a copy of the message in original format, 
it's trivial to add code to do that.  (sometimes they do keep a copy of 
the message in original format because it's easier to rebuild "next in 
thread" style links from original messages than from HTML messages).

Keith

Cc: Martin Duerst <[email-masked]>, ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Wed, 27 Oct 2004 08:47:41 -0400
To: Graham Klyne <[email-masked]>
Sender: [email-masked]

>> I strongly disagree.  When a message is archived, it should be 
>> faithfully archived.  Or a least, if a message is going to point to 
>> an archive of itself, it should be a faithful archive.  Archives in 
>> other formats are less functional and often lose information as 
>> compared to the original.
>
> Archives that exist and work (as does the scheme described by Martin) 
> are infinitely more functional than ones that don't.

It's much easier to make the _entire system_ "work" if the archives use 
a format that is already supported by the email readers that will use 
this header field.  All of those email readers support rfc 822 and know 
how to do reasonable things with 822 messages.  Even those that support 
HTML don't know how to do things like reply to messages formatted as 
HTML because when you reformat a message into HTML all of the semantic 
content from the original message is lost.

Archives that preserve the original message format do exist and work, 
are trivial to implement, and used to be the norm.  When I ran several 
IETF mailing lists, every single one of them archived messages in 
original format.  It took around five lines of script to generate a 
unique file name, and write the message to that file.

The only reason we don't see mail archives in original format so much 
today is that many recent archive packages are designed to allow 
messages to be accessed from web browsers.  But there's no reason why 
an archive package can't make a message available in its original 
format (instead of, or in addition to, HTML).  And it's not as if it's 
difficult to tell an http server that an extension of (say) .MIME or 
.822 equates to message/rfc822.  You can even do content-negotiation so 
that a client gets HTML or 822 depending on its preferences.

Date: Wed, 27 Oct 2004 12:25:47 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Keith Moore <[email-masked]>, Graham Klyne <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

Arnt Gulbrandsen writes:
> (That doesn't necessarily mean that the retrieved object must have a 
> certain content-type. I can imagine other ways to achieve the same 
> end.)

That was stupid of me. Concrete suggestions beat handwaving. Sorry.

My preferred approach would be to give archived-at the same parameter 
list that MIME fields have, and define a "content-type=x/y" parameter 
naming a type expected to be available. For a list that provides 
archives in two forms and uses fancy HTTP to provide the right one, the 
field could be like this:

Archived-At: ...; content-type=message/rfc822, content-type=text/html

For a less fancy archive providing both forms under separate URLs, two 
Archived-At fields could be used:

Archived-At: ...; content-type=message/rfc822
Archived-At: ...; content-type=text/html

For today's usual sort of archive, the expected content-type would be 
text/html by default:

Archived-At: ...

Arnt

Date: Wed, 27 Oct 2004 11:38:31 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Keith Moore <[email-masked]>, Graham Klyne <[email-masked]>, Martin Duerst <[email-masked]>
Sender: [email-masked]

Graham Klyne writes:
> Archives that exist and work (as does the scheme described by Martin) 
> are infinitely more functional than ones that don't.
>
> Isn't that the point of "running code..."?

Work is the key word, and the meaning of work depends on context.

Archives on the web must work in the context of the web, IMNSHO. And 
similarly, archives pointed to by message/rfc822 header fields 
must/should work in the context of message/rfc822 handling programs, 
because those are the programs that look at such fields.

(That doesn't necessarily mean that the retrieved object must have a 
certain content-type. I can imagine other ways to achieve the same 
end.)

Arnt

Date: Wed, 27 Oct 2004 09:38:53 +0100
To: "Martin J. Duerst" <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: http://www.ietf.org/internet-drafts/draft-duerst-archived-at-02 .txt
Cc: [email-masked]
Sender: [email-masked]

Martin,

Two nits concerning the header field registration (copied below).

1. Specify that the entry is for the permanent header field registry, 
assuming the relevant conditions are satisfied; e.g.
[[
    IANA is herewith requested to register the Archived-At header field
    in the Permanent Message Header Fields Registry, per RFC 3864, as follows:
]]

2. You need to decide if this is or is not to be usable with netnews.  If 
it is to be usable in netnews, then (for clarity) there should be two 
registration templates, one for mail and one for netnews, otherwise the same.

If you are really not sure about netnews, then I suggest you leave it out 
and separately request an entry for netnews in the provisional header field 
registry:  it can be moved later if netnews community think that's reasonable.

#g
--

[[
7.  IANA considerations

    IANA is herewith requested to register the Archived-At header field
    in the Message Header Fields Registry as follows:

       Header field name:
          Archived-At

       Applicable protocol:
          mail (RFC 2822), optionally also netnews (RFC 1036)

       Status:
          provisional
          (Note to RFC Editor: Replace this with "standard")

       Author/Change controller:
          Martin Duerst, [email-masked], W3C/Keio University
          (Note to RFC Editor: Replace this with "IETF")

       Specification document(s):
          Internet-Draft draft-duerst-archived-at-02.txt
          (Note to RFC Editor: Replace this with
           RFC YYYY (RFC number of this spec))

       Related information:
          none
]]

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Wed, 27 Oct 2004 10:14:55 +0100
To: Keith Moore <[email-masked]>, Martin Duerst <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: ietf-822 list <[email-masked]>
Sender: [email-masked]

At 18:05 26/10/04 -0400, Keith Moore wrote:

>> >So what Martins draft needs to say is something like "If the scheme of the
>> >URI is 'http', then the entity that is returned SHOULD have the
>> >Content-Type message/rfc822". Then whatever system asked for the
>> >Archived-At object to be retrieved has a decent chance of being able to
>> >display and process it like an email.
>>
>>I don't think my draft should say anything normatively about what
>>should come back. Saying that it should be message/rfc822 would be
>>against something like 100% of current practice and running code.
>
>I strongly disagree.  When a message is archived, it should be faithfully 
>archived.  Or a least, if a message is going to point to an archive of 
>itself, it should be a faithful archive.  Archives in other formats are 
>less functional and often lose information as compared to the original.

Archives that exist and work (as does the scheme described by Martin) are 
infinitely more functional than ones that don't.

Isn't that the point of "running code..."?

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Tue, 26 Oct 2004 18:27:18 -0400
To: Martin Duerst <[email-masked]>
Sender: [email-masked]

> >> So what Martins draft needs to say is something like "If the scheme 
> of the
> >> URI is 'http', then the entity that is returned SHOULD have the
> >> Content-Type message/rfc822".
> >
> >Maybe.  I'd like to encourage more archives to support native format 
> access.  I do think it' s reasonable for a standard for accessing mail 
> archives to behave predictably (as in, provide some minimum format for 
> the sake of interoperability), and that it's reasonable to give mail 
> readers the message in a format that they can use.  I'm concerned that 
> simply saying "SHOULD use 822" will alienate those that don't.  
> "SHOULD provide 822; MAY provide HTML or other formats if the access 
> protocol supports content-negotiation seems about right to me.
>
> As explained in my answer to Charles, I don't want to recommend stuff
> that I haven't seen implemented.

The reason you haven't seen it implemented is that you are looking at 
archives with web browsers.  Internet email readers don't currently 
have a way of accessing archives (except perhaps for read-only IMAP), 
so there aren't many archives that are set up for use by email readers. 
  But all internet email readers support message/rfc822 to some degree, 
whereas not all of them support HTML or other formats.

>  I think making messages available as
> message/rfc822 is a great idea at least on paper, but in W3Cs case,
> things are currently working quite well with serving the messages
> just as HTML,

Apparently you don't expect much from email archives.  IMHO you should 
be able to do anything with an archived message that you can do with a 
message you received in your inbox - read it with any mail reader, 
reply to it, forward it (as a message) to somewhere else, search it as 
if it were a message, display or save any of its body parts, follow 
threads to other messages (including those in your inbox), etc.  In 
addition, HTML has proven to be a poor format for use in email because 
of numerous security risks associated with scripts, objects, and 
referenced images along with the expectation that these are invoked or 
displayed without user consent.

If you are designing a way to allow email messages to reference 
archives, it makes good sense to make those messages usable by email 
readers.

> and I don't want to give potential users the impression
> that they have to support message/rfc822 or they better won't use
> Archived-At at all.

That's very close to the the impression that they should get.  More 
precisely:

a) those who use archived-at should take pains to archive their 
messages faithfully (meaning in the original format) and in such a way 
as to be readable by the vast majority of mail readers (i.e. in 
message/rfc822), and

b) If you want interoperability the last thing you want to do is to 
burden email readers with having to support arbitrary formats for 
archived messages.

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Tue, 26 Oct 2004 18:05:49 -0400
To: Martin Duerst <[email-masked]>
Sender: [email-masked]

> >So what Martins draft needs to say is something like "If the scheme 
> of the
> >URI is 'http', then the entity that is returned SHOULD have the
> >Content-Type message/rfc822". Then whatever system asked for the
> >Archived-At object to be retrieved has a decent chance of being able 
> to
> >display and process it like an email.
>
> I don't think my draft should say anything normatively about what
> should come back. Saying that it should be message/rfc822 would be
> against something like 100% of current practice and running code.

I strongly disagree.  When a message is archived, it should be 
faithfully archived.  Or a least, if a message is going to point to an 
archive of itself, it should be a faithful archive.  Archives in other 
formats are less functional and often lose information as compared to 
the original.

Keith

From: "Hector Santos" <[email-masked]>
To: "IETF-822" <[email-masked]>
Subject: Fw: [spf-discuss] New Sender-ID draft
Date: Tue, 26 Oct 2004 16:29:07 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

To all it may concern,

This coming month the FTC will be holding the "Email Authentication Submit."

The powers to be are intent on making a technology I don't believe is
technically sound the new standard.  Microsoft will make sure it happens.
The MARID rejected Sender-ID draft has been revised and resubmitted this
week.  I did not find a link to the draft at the IETF web site, but I found
one here:

http://www.spfhelp.com/downloads/draft-lyon-senderid-core-00.txt

There are many issues here that need reviewed by RFC 2822/2821 experts.  I
am posting this here because I have never seen the SPF or SENDER-ID
proposals discussed in IETF-SMTP or IETF-822 where there have great
relevance.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: "IETF-SMTP" <[email-masked]>, "IETF-822" <[email-masked]>
Subject: Fw: [spf-discuss] New Sender-ID draft
Date: Tue, 26 Oct 2004 15:58:57 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

To all it may concern,

This coming month the FTC will be holding the "Email Authentication Submit."

The powers to be are intent on making a technology I don't believe is
technically sound the new standard.  Microsoft will make sure it happens.
The MARID rejected Sender-ID draft has been revised and resubmitted this
week.  I did not find a link to the draft at the IETF web site, but I found
one here:

http://www.spfhelp.com/downloads/draft-lyon-senderid-core-00.txt

There are many issues here that need reviewed by RFC 2822/2821 experts.  I
am posting this here because I have never seen the SPF or SENDER-ID
proposals discussed in IETF-SMTP or IETF-822 where there have great
relevance.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Organization: Bruce Lilly
To: Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Tue, 26 Oct 2004 10:22:31 -0400
Cc: "ietf-822" <[email-masked]>
Sender: [email-masked]

On Sun October 24 2004 22:46, Martin Duerst wrote:

> I have added a very clear sentence saying that the field MUST only be
> created if the message is actually being made available at the URI given
> in the header field. Do you think that this should say anything more?

Since (unlike Reply-To and Sender) the proposed field plays no
role in message protocols, but is simply a way of conveying
additional information, "MUST" may be too strong.  RFC 2119
specifically states that it must not be used except to ensure
interoperation or to avoid network damage (retransmissions,
etc.).  Incidentally, it is customary to include some boilerplate
text referring to the RFC 2119-defined terms and to include
RFC 2119 as a normative reference when making use of those
terms.  Of course if you intentionally did not reference RFC
2119 because you intend your use of "MUST", "MUST NOT",
"SHOULD", "SHOULD NOT" and "MAY" to have different
meanings than the ones ascribed by RFC 2119, that raises
the question of precisely what meaning *is* intended.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Organization: Bruce Lilly
To: Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Tue, 26 Oct 2004 10:05:06 -0400
Cc: "ietf-822" <[email-masked]>
Sender: [email-masked]

On Mon October 25 2004 00:42, Martin Duerst wrote:
> 
> I tried to adapt the syntax below to the changes I made.
> The problem with URIs/URI references is solved by RFC2396bis.
> I don't think [FWS] and *WSP are needed, the syntax given for
> similar headers in RFC 2822 doesn't explicitly mention these,
> either.
> 
> Regards,    Martin.

Actually, RFC 2822 does include [FWS] etc., but they are part of
the lower-level construct specifications, so aren't readily apparent
when looking only at the field definitions; they do become visible
if one expands "domain", "dot-atom", "date-time", etc.  That
does not apply to the constructs used in the proposed field, since
the ABNF for those constructs does not explicitly include [FWS]
etc.  Incidentally, the successor to RFC 2822 may change in that
respect; a proposed revised grammar has been discussed on the
ietf-822 list.

Given the draft ABNF

       archived-at = "Archived-At:" '<' URI '>' CRLF ; URI not empty

and the example implicit in the text

    As an example, the URI
    "http://www.w3.org/mid/[email-masked]",
    containing the message id "[email-masked]",
    is redirected to the URI of this message in the W3C mailing-list
    archive at http://lists.w3.org/Archives/Public/uri/2004Oct/0017.html.

implies that

 Archived-At:<http://lists.w3.org/Archives/Public/uri/2004Oct/0017.html>

is legal but

 Archived-At: <http://lists.w3.org/Archives/Public/uri/2004Oct/0017.html>
                    ^ N.B. SP here
 Archived-At: (w3 list archive)
   <http://lists.w3.org/Archives/Public/uri/2004Oct/0017.html>

etc. are not legal.

Concerning that example, please note that a message identifier (RFC 2822)
includes angle brackets; i.e. the message id is presumably
"<[email-masked]>".  When using mid
schemes or conversions which implicitly involve msg-id to mid or
mid-like constructs, it might be advisable to note the difference in
angle bracket inclusion and/or to refer to the mid RFC (2387) as a
normative or informative reference.  The angle bracket issue has also
been recently discussed on the ietf-822 list.

Note also that RFC 2822 has provisions in the parse (obs-) syntax
for accommodation of whitespace between the field name and the colon
delimiter (which was allowed in earlier versions of the text message format).
You might wish to consider whether to provide for that for parsing,
particularly in the obs- rule for X-Archived-At.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Organization: Bruce Lilly
To: Martin =?iso-8859-1?q?D=FCrst?= <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Tue, 26 Oct 2004 09:48:00 -0400
Cc: ietf-822 <[email-masked]>
Sender: [email-masked]

On Sun October 24 2004 23:20, Martin Duerst wrote:

> This is now solved, because RFC 2396bis defines an URI as
> URI           = scheme ":" hier-part [ "?" query ] [ "#" fragment ].

Probably OK, assuming:
1. that doesn't change relative to the issue before publication as an RFC
2. that publication of the 2396bis draft as an RFC precedes publication
    of the archived-at draft as an RFC (RFCs aren't supposed to make
    normative references to drafts).

Date: Tue, 26 Oct 2004 15:31:55 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: a header authentication scheme
Sender: [email-masked]

Laird Breyer writes:
> Yet another way of inserting the requisite unguessability in a 
> Received line might be a specially formatted comment. This would not 
> require the ID, and if the comment was present, it would be a 
> guarantee of sorts from the MTA that the token is unguessable (so the 
> MTA authors would not have to change their ID scheme).

It would be the same as the ID: Optional, sometimes present. The major 
differences are that it'd be present in 0% instead of 99% of the cases 
to start with, and that it would require evil parser hacks to 
distinguish that comment from other comments.

Arnt

Date: Tue, 26 Oct 2004 13:37:40 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 <[email-masked]>
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 <[email-masked]>
Sender: [email-masked]

On Oct 22 2004, Bruce Lilly wrote:
> 
> Once a message leaves the SMTP transport environment, of
> course there can be additional processing.  However, if the
> message content (header or body) is changed, then one
> is no longer dealing with exactly the *same* message.

That's the unfortunate problem with filtering services at present. In
your sense, the messages delivered to end users are practically never
the same messages that were sent. That shouldn't really matter as long
as users receive some RFC 2822 message to read.

What causes headaches in the present context is the decentralized
nature of the modifications. Once the message hits the receiving MUA,
there really is no way to know who did what, but I believe a simple
standard could be useful.

> There is an existing mechanism that can be used to allow
> addition of arbitrary content, with authentication, and
> while permitting the original message to be recovered
> intact: simply wrap the original as MIME type message/rfc822,
> if there is additional content, it and the wrapped message
> can be packaged as multipart/mixed, and if authentication
> is desired that can then be signed using the multipart/signed
> MIME type (S/MIME and PGP/MIME use that mechanism and
> are widely deployed).

That is a nice way of wrapping up the various issues, but I fear it
will never fly in a world where the already implemented alternatives
are flawed, but much simpler (ie who would switch from single-line
X-headers, and why?).

If you ask each filter processor to wrap the original message as it
receives it into a message/rfc822 attachment, then by the time the
fourth filter say looks at the message, the original content will be 3
levels deep.  So you're forcing each filter to know how to parse MIME
properly, even if all the filter does is to verify the IP addresses
within the trace headers of the message, say [ie an example filter
which would never look at the message body otherwise].

The problem with full MIME authentication is that we don't have the
trust networks and ubiquitous public key infrastructures this would need,
so in practice authentication would simply not be used.

The header authentication I'm talking about is not very sophisticated,
but it doesn't require any cryptography infrastructure, or managing
shared secrets etc.  It's a lightweight refinement intended to help
software decide what to trust, based on message structure.

For example, I would see a future MUA parsing an incoming message, and
displaying to the user a limited list of headers. The user could elect
to ignore headers which weren't added by his organization, by
specifying a subnet mask or similar. Alternatively, he might have his
own custom configuration of some popular filter software, which is
also deployed organization wide. He would not want to use the common
version except in some cases, so would want to limit himself to
headers added by his personal machine. These are personal choices
which need not involve the whole organization.

> 
> > identifying date-time: unfold the Received line, normalize spaces,
> > look for the first ';' from the end and take everything after that.
> 
> No.

(snip) I knew I was going to regret writing those paragraphs. I accept
your points. I think in a final document, an appropriate parsing
algorithm will have to be given explicitly.

> 
> There's a more fundamental issue with the whole scheme; you're
> simply trying to authenticate some added content (and as noted
> above, there are existing mechanisms that can do so without
> running roughshod over the existing transport mechanism). Let's

Again, I would like to say that I am not trying to fully authenticate,
rather I am trying to allow accurate verification of the 
content writer's location in the message transport. I'm using
authentication for lack of a better vocabulary.

Whether the added content is trusted by the recipient is a different
issue, although that is blurred because the rule of thumb is 
"if it was added inside my computer, I trust it" and variations thereof. 

> assume for the moment that a 100% foolproof authentication
> mechanism is in place and that the field above is guaranteed to
> have been inserted by the party that claimed to do so.  I submit
> that it tells me nothing particularly useful; I have no guarantee
> that the claimed processing was performed or that the claimed
> results are accurate.

That's true, but is not a show stopping issue. 

For example, you might trust that each message in your mail spool was
delivered via the internet if the top Received line checks out,
say. In actual fact, your machine could have been rootkitted, and an
appropriately formatted message was simply delivered directly to your
mailspool, with a fake but authentic looking topmost Received line.

So you have no guarantees about your mailspool, but it doesn't stop you
from acting as if the messages were sent to you in the normal way.
Similarly, how the recipient acts on an added header is up to them, 
as is whether they want to trust those headers in the first place.

> Even if I have blind trust in the party that claimed to have
> inserted that field, I have no way of knowing if the claimed name
> "SpamAssassin" is what I think it is, and if it is, I have no idea
> whether it was modified at that site or if it is "vanilla"
> SpamAssassin.  Even if I assume that it is unmodified,

If you can verify that the header was added after the message went
through a certain machine, and if you trust your system admin when
he says he's got SpamAssassin in vanilla configuration along that 
segment of the message path, then you can trust the header.

In a futuristic MUA, you would get a message stating: "I detected
a filter called SpamAssassin which operates inside the third hop
from the end, would you like me to use its recommendation?"

> I have no idea what configuration files were used with it or
> what run-time options were specified.  In short, purported results
> of an indeterminate process with indeterminate configuration
> and indeterminate run-time parameters allegedly run at a
> remote site are of little value.

That's the trouble we have today and why a standard filtering header
(such as the Processed example I ued) would help. With a standard, 
some common outputs can be formalized, offering a standard interpretation
which anyone can use. It's up to the filter authors to honour the contract
and actually output quantities whose interpretation matches the standard's 
mandated interpretation. It works well enough for MIME.

-- 
Laird Breyer.

Date: Tue, 26 Oct 2004 12:13:24 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 <[email-masked]>
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 <[email-masked]>
Sender: [email-masked]

On Oct 22 2004, Bruce Lilly wrote:
> 
> Two problems:
> 1. date-time is pervasive; it is used in many places in addition to
>    time stamp lines

Good point. This change would impact other header lines.

> 2. while providing seconds was once mandatory (RFC 821), it is
>    now optional (RFCs 2821/2822). So you might not even have
>    seconds.  You seem to want to buck the trend.

I hadn't realized that (I thought it was just for reading obsolete
messages). Slightly off topic, I wonder why the trend would be towards 
less precision in time stamps? Doesn't that go against the purpose of 
time stamps in the first place?

> >  Yes. From the RFC, this is optional. Do you know any good reasons
> > why the ID might not be added to a Received line sometimes?
> 
> Umm, because it is optional.  Past implementations have associated
> the identifier with a set of files used in store-and-forward
> processing.  In some cases, messages might be handled w/o
> any associated file storage, therefore no need for an id.

This is interesting. Presumably, when used for such a purpose, IDs would
be fairly random and unique in practice. On the other hand, 
if the IDs are no longer used much, then repurposing them as random
tokens in future would not be met with too much opposition from MTA 
implementors.

> 
> There also appears to be some confusion between "with"
> (which specifies a protocol (SMTP, ESMTP, LMTP, etc.) and
> "id" which provides an identifier of some sort (RFCs
> 821/822/2821/2822 all differ on what is allowed).

There is another way to include an "unguessable" token in Received lines
which doesn't do damage to the existing structure. MTA implementors
could include a specially formatted comment, which would contain
a random string. 

There is precedent for this in RFC 2821, for example:

Extended-Domain = Domain /
           ( Domain FWS "(" TCP-info ")" ) /
           ( Address-literal FWS "(" TCP-info ")" )

The comment could be formatted in such a way that it's easy to scan
the Received line without needing to parse it. The presence of the
comment would be a guarantee from the MTA that the associated random
token is "unguessable", and there would be no need for others to rely on
assumptions about ID or date-time.

For example, we could have

   Received: from 185129182.virtua.com.br (185129182.virtua.com.br
       [200.185.129.182]) by smtpin-3211.bay.webtv.net (WebTV_Postfix+sws)
       with SMTP id 7280B11DCC; Fri,  5 Mar 2004 19:00:48 -0800 (PST)    
       (random-seed ABCDEFGHIJKLMNOP)

where the unguessable value is ABCDEFGHIJKLMNOP, and a scanner looks
for the string "(random-seed ".

The downside is that existing MTAs don't include such a token, so there's
a natural delay in adoption. Of course, quoting the ID if present or the 
date-time could be considered low grade protection intended for a transition
period, while the world upgrades its MTAs over time. 

-- 
Laird Breyer.

Date: Tue, 26 Oct 2004 11:43:16 +1000
From: Laird Breyer <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked]
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: Arnt Gulbrandsen <[email-masked]>, [email-masked]
Sender: [email-masked]

On Oct 22 2004, Arnt Gulbrandsen wrote:
> 
> Step 1. I'd send five messages to [email-masked] and wait for the 
> bounces. For each of them, I'd compute the delay from my sending time 
> (using my clock) to the receiving MTA's receive time. Next, I'd average 
> the five values.
> 
> Step 2. I'd send the target a hundred messages or so, with a hundred 
> guesses centered around the average computed in step 1.
> 
> Assuming that 80% of mail from me to the target are delivered within the 
> range [t-50,t+50> seconds, I'd have an 80% chance of success.
> 
> A more sophisticated attack would use more initial probes and vary the 
> number of guesses based on the spread of the probed delays.
> 
> Should I bother to do it, or is the description persuasive? ;)
> 

I think that works as a method. Looking at some mail archives I have
around, differences in received times are a few seconds at most in the
majority of cases (single digit), so you'd be realistically aiming for
[t-5, t+5> which gives a chance of success of 1/10 if you spread out
uniformly (slightly better if you know more about the distribution,
worse if you attempt to forge over two hops or more).

So the forgery detection based on time would be about 90% successful
at present. If microseconds were allowed, forgery detection would
improve to about 99.99% successful, ie only 1 in 10,000 forgeries
would fail to be detected.

So I agree with you, the received time on its own isn't such a good
idea at present, and the ID alternative is a better choice (except for
its optional nature). Similarly, combining the two (as in hashing the whole
Received: line) would give about x+1 digit protection, where x is the number
of digits provided by the randomness inherent in the ID. 

Yet another way of inserting the requisite unguessability in a
Received line might be a specially formatted comment. This would not
require the ID, and if the comment was present, it would be a
guarantee of sorts from the MTA that the token is unguessable (so the
MTA authors would not have to change their ID scheme).

-- 
Laird Breyer.

Date: Mon, 25 Oct 2004 14:58:39 +0900
To: [email-masked]
From: Martin Duerst <[email-masked]>
Subject: New Internet Draft: draft-duerst-archived-at-02.txt
Sender: [email-masked]

Dear Email Experts,

I have just submitted an update of the Archived-At proposal.
It should appear at
http://www.ietf.org/internet-drafts/draft-duerst-archived-at-02.txt
within a few days, and is also appended below.

I hope that this new draft, together with the various emails sent
out a moment ago, sufficiently addresses the comments that have
been made on previous versions of this proposal. Of course, I was
not able to fully take into account all of the comments made, because
there were some contradicting opinions as well as some interesting
ideas that however didn't match deployed usage.

I would highly appreciate any additional feedback and comments on
the draft. Please copy me directly, as I'm not subscribed to this list.

Regards,    Martin.

=========================================================================

Network Working Group                                          M. Duerst
Internet-Draft                                       W3C/Keio University
Expires: April 25, 2005                                 October 25, 2004

                   The Archived-At Message Header Field
                       draft-duerst-archived-at-02

Status of this Memo

    This document is an Internet-Draft and is subject to all provisions
    of section 3 of RFC 3667.  By submitting this Internet-Draft, each
    author represents that any applicable patent or other IPR claims of
    which he or she is aware have been or will be disclosed, and any of
    which he or she become aware will be disclosed, in accordance with
    RFC 3668.

    Internet-Drafts are working documents of the Internet Engineering
    Task Force (IETF), its areas, and its working groups.  Note that
    other groups may also distribute working documents as
    Internet-Drafts.

    Internet-Drafts are draft documents valid for a maximum of six months
    and may be updated, replaced, or obsoleted by other documents at any
    time.  It is inappropriate to use Internet-Drafts as reference
    material or to cite them other than as "work in progress."

    The list of current Internet-Drafts can be accessed at
    http://www.ietf.org/ietf/1id-abstracts.txt.

    The list of Internet-Draft Shadow Directories can be accessed at
    http://www.ietf.org/shadow.html.

    This Internet-Draft will expire on April 25, 2005.

Copyright Notice

    Copyright (C) The Internet Society (2004).

Abstract

    This memo defines a new email header field, Archived-At:, to provide
    a direct link to the archived form of an individual email message.

Duerst                   Expires April 25, 2005                 [Page 1]

Internet-Draft    The Archived-At Message Header Field      October 2004

Table of Contents

    1.   Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
    2.   Syntax Definition  . . . . . . . . . . . . . . . . . . . . . . 3
    3.   Multiple Archived-At Header Fields . . . . . . . . . . . . . . 4
    4.   Formats of Archived Message  . . . . . . . . . . . . . . . . . 4
    5.   Implementation Considerations  . . . . . . . . . . . . . . . . 4
    6.   Security Considerations  . . . . . . . . . . . . . . . . . . . 5
    7.   IANA considerations  . . . . . . . . . . . . . . . . . . . . . 6
    8.   Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 6
    9.   Normative References . . . . . . . . . . . . . . . . . . . . . 6
         Author's Address . . . . . . . . . . . . . . . . . . . . . . . 7
         Intellectual Property and Copyright Statements . . . . . . . . 8

Duerst                   Expires April 25, 2005                 [Page 2]

Internet-Draft    The Archived-At Message Header Field      October 2004

1.  Introduction

    [RFC2369] defines a number of header fields that can be added to
    email messages sent by email distribution lists.  One of them is the
    'List-Archive' header field that describes how to access archives for
    the list.  This allows to access the archives as a whole, but not the
    individual message.

    There is often a need or desire to refer to the archived form of a
    single message.  This memo defines a new header, 'Archived-At', to
    refer to a single message at an archived location.  This can help to
    find a message to a mailing list in the archive of that mailing list
    much more quickly and directly.  It can also be used independently of
    mailing lists, for example in connection with legal requirements to
    archive certain messages.

2.  Syntax Definition

    The name of the header field is 'Archived-At'.  The content of the
    header field mostly consist of an URI [RFCXXXX] enclosed in angle
    brackets ('<', '>'), with internal whitespace being ignored.  MTAs
    MUST NOT insert whitespace within the angle brackets, but client
    applications should treat any whitespace, which might have been
    inserted by poorly behaved MTAs, as characters to ignore.  The URI
    points to an archived version of the message.  See Section 4 for some
    more details.

    This header field is subject to the encoding and character
    restrictions for mail headers as described in [RFC2822].
    Additionally, the URI content is further restricted to the set of URI
    safe characters [RFCXXXX].

    More formally, the header field is defined as follows in ABNF
    according to [RFC2234]:

       archived-at = "Archived-At:" '<' URI '>' CRLF ; URI not empty

    where URI is defined in [RFCXXXX] and CRLF is defined in [RFC2234].

    For backward-compatibility, the following MAY also be parsed (but
    MUST not be generated):

       obs-archived-at = "X-Archived-At:" URI CRLF ; URI not empty

    This syntax is kept simple in that only one URI per header field is
    allowed.  In this point, the syntax is different from [RFC2369].

Duerst                   Expires April 25, 2005                 [Page 3]

Internet-Draft    The Archived-At Message Header Field      October 2004

3.  Multiple Archived-At Header Fields

    If a message is archived at several independent locations, several
    header fields MUST be used.  Multiple Archived-At header fields with
    the same URI SHOULD be avoided.  An Archived-At header field MUST
    only be created if the message is actually being made available at
    the URI given in the header field.

    If a message is forwarded from a list to a sublist, and both lists
    support adding the Archived-At header field, then the sublist SHOULD
    add a new Archived-At header field without removing the already
    existing one(s), unless the header field is exactly the same as an
    already existing one, in which case the new header field SHOULD NOT
    be added.

4.  Formats of Archived Message

    There is no restriction on the format used to serve the archived
    message from the URI in the 'Archived-At' header field.  It is
    expected that in many cases, the archived message will be served as
    (X)HTML, as plain text, or as message/rfc822 [RFC2046].  Some forms
    of URIs may imply the format in which the archived message is served,
    although this should not be relied upon.  If the protocol used to
    retrieve the message allows for content negotiation, then it is also
    possible to serve the archived message in several different formats.

    As an example, a HTTP URI in an 'Archived-At' header may allow to
    serve the archived message both as text/html for human consumption in
    a browser and as message/rfc822 for use by a mail user agent without
    loss of information.

5.  Implementation Considerations

    This section shortly discusses some implementation considerations.
    Mailing list expanders and email archives are often separate pieces
    of software.  It may therefore be difficult to create an
    'Archived-At' header field in the mailing list expander software.

    One way to address this difficulty is to have the mailing list
    expander software generate an unambiguous URI, e.g.  an URI based on
    the message id of the incoming email, and to set up the archiving
    system so that it redirects requests for such URIs to the actual
    messages.

    Such a system has been implemented and is in productive use at W3C.
    As an example, the URI
    "http://www.w3.org/mid/[email-masked]",
    containing the message id "[email-masked]",

Duerst                   Expires April 25, 2005                 [Page 4]

Internet-Draft    The Archived-At Message Header Field      October 2004

    is redirected to the URI of this message in the W3C mailing-list
    archive at http://lists.w3.org/Archives/Public/uri/2004Oct/0017.html.

    Source code for this implementation is available at
    http://dev.w3.org/cvsweb/search/, in particular
    http://dev.w3.org/cvsweb/search/cgi/mid.pl and
    http://dev.w3.org/cvsweb/search/bin/msgid-db.pl.

    When using the message id to create an address for the archived mail,
    care has to be taken to escape characters in the message id that are
    not allowed in the URI.

6.  Security Considerations

    There are many potential security issues when activating and
    dereferencing an URI.  For more details, please see [RFCXXXX].  In
    the context of this proposal, the following are particularly
    relevant: An intruder may get access to the message transmission and
    be able to insert an URI pointing to some malicious content.  Also,
    somebody may be able to construct a message that when received
    directly is harmless, but that when accessed via the URI produces
    some problems because in the format used in the archive, some content
    has not been adequately escaped.

Duerst                   Expires April 25, 2005                 [Page 5]

Internet-Draft    The Archived-At Message Header Field      October 2004

7.  IANA considerations

    IANA is herewith requested to register the Archived-At header field
    in the Message Header Fields Registry as follows:

       Header field name:
          Archived-At

       Applicable protocol:
          mail (RFC 2822), optionally also netnews (RFC 1036)

       Status:
          provisional
          (Note to RFC Editor: Replace this with "standard")

       Author/Change controller:
          Martin Duerst, [email-masked], W3C/Keio University
          (Note to RFC Editor: Replace this with "IETF")

       Specification document(s):
          Internet-Draft draft-duerst-archived-at-02.txt
          (Note to RFC Editor: Replace this with
           RFC YYYY (RFC number of this spec))

       Related information:
          none

8.  Acknowledgements

    The members of the W3C system team, in particular Gerald Oskoboiny,
    Olivier Thereaux, and Jose Kahan, created the mid-based email archive
    lookup system and the experimental form of the Archived-At header.
    Pete Resnik provided the motivation for writing up this memo.
    Discussion on the [email-masked] mailing list, in particular
    contributions by Arnt Gulbrandsen, Graham Klyne, Bruce Lilly, Charles
    Lindsey, and Keith Moore, has led to further improvements of the
    proposal.

9  Normative References

    [RFC2046]  Freed, N. and N. Borenstein, "Multipurpose Internet Mail
               Extensions (MIME) Part Two: Media Types", RFC 2046,
               November 1996.

    [RFC2234]  Crocker, D. and P. Overell, "Augmented BNF for Syntax
               Specifications: ABNF", RFC 2234, November 1997.

    [RFC2369]  Baer, J. and G. Neufeld, "The Use of URLs as Meta-Syntax

Duerst                   Expires April 25, 2005                 [Page 6]

Internet-Draft    The Archived-At Message Header Field      October 2004

               for Core Mail List Commands and their Transport through
               Message Header Fields", RFC 2369, July 1998.

    [RFC2822]  Resnick, P., "Internet Message Format", RFC 2822, April
               2001.

    [RFC3864]  Klyne, G., Nottingham, M. and J. Mogul, "Registration
               Procedures for Message Header Fields", RFC 3864, BCP 90,
               September 2004.

    [RFCXXXX]  Berners-Lee, T., Fielding, R. and L. Masinter, "Uniform
               Resource Identifier (URI): Generic Syntax",
               draft-fielding-uri-rfc2396bis-07.txt (work in progress),
               April 2004.

Author's Address

    Martin Duerst (Note: Please write "Duerst" with u-umlaut wherever
                  possible, for example as "D&#252;rst" in XML and HTML.)
    World Wide Web Consortium/Keio University
    5322 Endo
    Fujisawa, Kanagawa  252-8520
    Japan

    Phone: +81 466 49 1170
    Fax:   +81 466 49 1171
    EMail: mailto:[email-masked]
    URI:   http://www.w3.org/People/D%C3%BCrst/

Duerst                   Expires April 25, 2005                 [Page 7]

Internet-Draft    The Archived-At Message Header Field      October 2004

Intellectual Property Statement

    The IETF takes no position regarding the validity or scope of any
    Intellectual Property Rights or other rights that might be claimed to
    pertain to the implementation or use of the technology described in
    this document or the extent to which any license under such rights
    might or might not be available; nor does it represent that it has
    made any independent effort to identify any such rights.  Information
    on the procedures with respect to rights in RFC documents can be
    found in BCP 78 and BCP 79.

    Copies of IPR disclosures made to the IETF Secretariat and any
    assurances of licenses to be made available, or the result of an
    attempt made to obtain a general license or permission for the use of
    such proprietary rights by implementers or users of this
    specification can be obtained from the IETF on-line IPR repository at
    http://www.ietf.org/ipr.

    The IETF invites any interested party to bring to its attention any
    copyrights, patents or patent applications, or other proprietary
    rights that may cover technology that may be required to implement
    this standard.  Please address the information to the IETF at
    [email-masked].

Disclaimer of Validity

    This document and the information contained herein are provided on an
    "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
    OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
    ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
    INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
    INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
    WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Copyright Statement

    Copyright (C) The Internet Society (2004).  This document is subject
    to the rights, licenses and restrictions contained in BCP 78, and
    except as set forth therein, the authors retain all their rights.

Acknowledgment

    Funding for the RFC Editor function is currently provided by the
    Internet Society.

Duerst                   Expires April 25, 2005                 [Page 8]
 

Date: Mon, 25 Oct 2004 12:29:53 +0900
To: Keith Moore <[email-masked]>, Charles Lindsey <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Keith Moore <[email-masked]>, [email-masked], [email-masked]
Sender: [email-masked]

At 19:05 04/02/26, Keith Moore wrote:

 >> So what Martins draft needs to say is something like "If the scheme of the
 >> URI is 'http', then the entity that is returned SHOULD have the
 >> Content-Type message/rfc822".
 >
 >Maybe.  I'd like to encourage more archives to support native format 
access.  I do think it' s reasonable for a standard for accessing mail 
archives to behave predictably (as in, provide some minimum format for the 
sake of interoperability), and that it's reasonable to give mail readers 
the message in a format that they can use.  I'm concerned that simply 
saying "SHOULD use 822" will alienate those that don't.  "SHOULD provide 
822; MAY provide HTML or other formats if the access protocol supports 
content-negotiation seems about right to me.

As explained in my answer to Charles, I don't want to recommend stuff
that I haven't seen implemented. I think making messages available as
message/rfc822 is a great idea at least on paper, but in W3Cs case,
things are currently working quite well with serving the messages
just as HTML, and I don't want to give potential users the impression
that they have to support message/rfc822 or they better won't use
Archived-At at all.

 >>> - how a mail reader (or for that matter a web browser) should decide
 >>>  whether to handle the message itself or hand it off to a different
 >>>  tool.
 >>
 >> You can't really give detailed instructions in a standard about how
 >> individual readers or browsers are supposed to behave.
 >
 >no, but I don't think a paragraph or two is too much.

There is now some text about formats in the draft, although that's
mostly about that (and how) different formats can be returned.
I really don't want to add something like "hey, if it's HTML,
you better show that in a browser" or so, because it really sounds
too trivial, and implementations usually get it right anyway.

Regards,    Martin. 

Date: Mon, 25 Oct 2004 11:46:21 +0900
To: Keith Moore <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked], [email-masked]
Sender: [email-masked]

Hello Keith (and others),

Sorry for taking so long to answer this message. I put out
draft-duerst-archived-at-01.txt before the last IETF, but
was unable to answer to your mails because they temporarily
got lost when I moved to a new PC. In the meantime, I have
been able to recover these.

At 18:01 04/02/24, Keith Moore wrote:
 >> >>Good point. But it is not really restricted to lists. There are
 >> >>other potential applications, which are not related to mailing lists.
 >> >>So starting with List-* would be confusing.
 >> >
 >> >Maybe it should only start with List- when added by a list.
 >>
 >> Although I expect this header to mostly be used in connection
 >> with mailing lists, the archiving functionality is really not
 >> at all tied to mailing lists. Also, what would be the benefit
 >> of having Archived-At and List-Archived-At for the user who
 >> wants to find an archived copy of the message?
 >
 >Having different fields for different parties adding an archived-at
 >field would allow the recipient to determine which party added that
 >information.   That way, if the information is misleading or inaccurate
 >or the archive is not properly maintained, the recipient has some idea
 >of who is responsible.

There are really several scenarios here for things going wrong:
1) The header is added 'correctly', but the archiving system is
    amiss. In that case, you'll get something like an HTTP 404,
    and hopefully, that's good enough to know how to follow on
    further. You can find an example of such a message at
    http://www.w3.org/mid/[email-masked]
    (just created a phony messageid to get you there).
2) The header is added wrongly, most possibly pointing to something
    completely different than the archive or the archive resolution
    system. In that case, you just don't have a clue. Let's assume
    that e.g. our mailing list software added
    http://wrong.example.org/mid/[email-masked]
    instead of
    http://www.w3.org/mid/[email-masked].
    Knowing this is a mailing list wouldn't help you at all;
    complaining to wrong.example.org wouldn't help you either.

Also, please note that 1) is quite a bit harder to get right
than 2), at least in the scenarios that I know about.

 >> In our implementation, the URI you get with Archived-At is
 >> completely independent of any particular mailing list, and
 >> is the same for all mailing lists in case of cross-posting.
 >
 >The question is not so much whether the archive is associated with
 >a list, as what party supplied the information.

For that, you would need to have a two-part header field, one part
being the URI pointing to the message, and the other being a
pointer to the supplier. But in general, the pointer to the
message should identify the supplier of the info, because there
has to be some coordination between where the message is archived
and who put a pointer to that location in the message.
On the other hand, if somebody wants to cheat/confuse recipients,
they will be able to fake a two-part field as easily as a one-part
field, so adding a separate part doesn't really help, I think.

 >(though I'd also assert
 >that it's useful to see how a message fit into the conversation of
 >a particular list, even if the message was posted to more than one
 >list.)

Yes. But you can easily do that by going to the actual archived message
(in particular if that's in HTML with various links to other messages).
Also, please note that in the system we have running, there is only
one Archived-At per message, even if the message gets sent to various
W3C lists. To find out that the message went to other lists, you either
get a 'multiple choice' page if there are several lists with lowest
confidentiality (e.g. it went to several public lists), or you get
directly to the message in the archive of the list with the lowest
confidentiality (assuming of course that you have access rights
to that level of confidentiality). If you need to check out other
mailing lists, that can always be done looking at the headers of
the actual message. This is definitely not perfect, but it works
quite well in most cases.

 >That and sometimes you want to filter out certain kinds
 >of header fields (e.g. you want to remove List- fields from messages
 >that were re-submitted to other lists)

In the cases of the Archived-At header, that would usually not be the
case. You would just add another header, and end up with two headers.

 >> >But it seems like we're finding more and more cases where it's a bad idea
 >> >to have a header field without very clear rules on who is allowed to set
 >> >that field.  We've seen this with Reply-To and Sender and also with Carl
 >> >Malamud's Solicitation field proposal.

I have added a very clear sentence saying that the field MUST only be
created if the message is actually being made available at the URI given
in the header field. Do you think that this should say anything more?

 >> These are certainly good examples. But I think there is a difference
 >> between a header that can be added several times (as the Archived-At
 >> header) and a header that can only be added once (as I think is the
 >> case with the above examples).
 >
 >> Or in other words: The rules are very clear, anybody who archives
 >> the message can add an Archived-At header, and nobody should
 >> overwrite existing ones. I clarified this in the draft.
 >
 >Being able to have multiple instances of the field does help, but it still
 >begs the question of which intermediary added the field.

See above. I don't think this is an issue for legitimately added
headers, and I don't think there is anything we can do about
forgeries,...

 >> >Being able to access the archived message from IMAP is potentially a lot
 >> >better, but (as with many things) even IMAP needs some tweaking to make it
 >> >work well for this.
 >>
 >> Very good idea. I have mentioned this in the draft. I thought
 >> that the IMAP URI scheme only allows to address folders,
 >> but that's not at all the case.
 >
 >I'll look forward to seeing the next draft.

I'll send out the new draft (draft-duerst-archived-at-02.txt) in a
separate mail.

Regards,     Martin. 

Date: Mon, 25 Oct 2004 11:59:55 +0900
To: Charles Lindsey <[email-masked]>, [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

At 12:09 04/02/26, Charles Lindsey wrote:

 >RFC 2369 appears to set the best precedents, including delimiting within
 ><...>, allowing FWS within those <...> (to be eliminiated before use as a
 >URI),

I have adopted these suggestions.

 >and provision for comma-separated lists of such URIs.

I haven't adopted this suggestion, because multiple headers can do
the same job. While I have received comments asking me to go to
full RFC 2396 syntax, I have also received comments to keep things
simple. I hope what I've done is an acceptable compromize.

 >Since the List-Archive header and the proposed [List-]Archived-At header
 >are quite likley to occur in the same message, it would be a great pity if
 >they did not follow the same syntactic conventions.

For a consumer, they do. Producers easily can get it right.

 >And I would much prefer a comma-separated list of URIs than multiple
 >Archived-At headers. For a start, it gives you a way to express some
 >priority between the various alternatives (as in the List-Archive header).

I'm not sure that's needed. Of course thery may be some odd cases where
it might be helpful, but I don't want to overdesign the proposal.

 >And then, multiple headers with the same name are Bad Thing for other
 >reasons (for example if ever you come to have digital signatures of
 >headers, such as have been mentioned recently in other threads on this
 >list).

If there are digital signatures of headers, those digital signatures
have to be able to deal with multiple headers, because there are some
already. Doing that is not impossible.

Regards,    Martin. 

Date: Mon, 25 Oct 2004 13:42:24 +0900
To: [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

I tried to adapt the syntax below to the changes I made.
The problem with URIs/URI references is solved by RFC2396bis.
I don't think [FWS] and *WSP are needed, the syntax given for
similar headers in RFC 2822 doesn't explicitly mention these,
either.

Regards,    Martin.

At 15:31 04/02/26, Bruce Lilly wrote:
 >Martin Duerst wrote:
 >
 >> Any definitive answers appreciated here. For the moment,
 >> I'm just keeping it with "one URI, no comments, no special
 >> mechanisms, follow the length limitations in RFC 2822".
 >
 >In terms of ABNF, similar to RFC 2822, that would be something like:
 >
 >archived-at = "Archived-At" ":" [FWS] URI-reference *WSP CRLF ;
 >URI-reference not empty
 >
 >where "URI-Reference" is defined in RFC 2396.  You may also want to
 >support parsing (but not generation in new messages) of older forms,
 >e.g.:
 >
 >obs-archived-at = ("Archived-At" / "X-Archived-At") *WSP ":" [FWS]
 >URI-reference [FWS] CRLF
 >
 >You might want to replace URI-reference with absoluteURI in both
 >places, though that would preclude use of a fragment identifier.
 >Yet another alternative would be to use "absoluteURI ["#" fragment]"
 >which precludes relative URIs while permitting a fragment identifier.
 >It really depends on what restrictions you want to place on the types
 >of URIs to be permitted. 

Date: Mon, 25 Oct 2004 12:12:44 +0900
To: Charles Lindsey <[email-masked]>, [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

At 12:29 04/02/26, Charles Lindsey wrote:
 >Keith Moore writes:
 >
 >>So I'm currently thinking that Archived-At needs to supply more
 >>information: namely the content types that are available.  An
 >>alternative might be to use a HEAD request to find out what
 >>content-types are available.
 >
 >I think the problem you are raising is a generic URI problem, and not just
 >a problem with this particular header.
 >
 >AIUI, a URI gives you access to a "resource"; i.e. it tells you where to
 >find it. It tells you nothing about what to do with it when you have got
 >it, except what is implicit in the "scheme".

Yes. In many contexts, that's considered a feature. For example, it allows
to serve the same content in a new format from the same location.

 >Now one of the possible benefits of the 'http' scheme is that the object
 >that comes back should have a Content-Type header. And what we want is for
 >this Content-Type to be message/rfc822.
 >
 >So what Martins draft needs to say is something like "If the scheme of the
 >URI is 'http', then the entity that is returned SHOULD have the
 >Content-Type message/rfc822". Then whatever system asked for the
 >Archived-At object to be retrieved has a decent chance of being able to
 >display and process it like an email.

I don't think my draft should say anything normatively about what
should come back. Saying that it should be message/rfc822 would be
against something like 100% of current practice and running code.
I don't think that it's a bad idea, quite to the contrary, but with
(as far as I know) currently 0 email messages served that way, and
(again as far as I know) 0 email agents able to get such a message
and then use it e.g. to reply to it, I think it would be strongly
against the IETF principles to require or recommend this.

 >>Either way, this document probably needs to say some things about

It now does; I have added a section talking about formats of an
archived message, and mentioned message/rfc822 as well as HTML,
plain text, and content negotiation.

 >>- how a mail reader (or for that matter a web browser) should decide
 >>  whether to handle the message itself or hand it off to a different
 >>  tool.
 >
 >You can't really give detailed instructions in a standard about how
 >individual readers or browsers are supposed to behave. The general
 >intention is that all readers/browsers that recognize Content-Type headers
 >are supposed to use the best available tool for handling that Content-Type
 >(possibly even using a special plugin). Naturally, some current systems
 >make a better job of this than others :-( .

I agree with Charles here.

Regards,    Martin. 

Date: Mon, 25 Oct 2004 12:20:24 +0900
To: [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

At 15:31 04/02/26, Bruce Lilly wrote:

 >You might want to replace URI-reference with absoluteURI in both
 >places, though that would preclude use of a fragment identifier.
 >Yet another alternative would be to use "absoluteURI ["#" fragment]"
 >which precludes relative URIs while permitting a fragment identifier.
 >It really depends on what restrictions you want to place on the types
 >of URIs to be permitted.

This is now solved, because RFC 2396bis defines an URI as
URI           = scheme ":" hier-part [ "?" query ] [ "#" fragment ].

Regards,    Martin. 

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Sun, 24 Oct 2004 15:21:45 -0400
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

On Sat October 23 2004 16:35, Charles Lindsey wrote:

> But if there IS some way to distinguish LIST addresses from others, then
> many benefits accrue, as many people have noted in these threads. For
> example, it would show if a message had been crossposted to several lists,
> enabling a Reply-to-List button to reply to them all.

Which is not necessarily what the author suggested, and is not
necessarily what the responder wants, and is not necessarily
appropriate (consider announce-only lists, which are lists
nevertheless).  Your claimed "benefit" isn't a benefit, it's a bug.

> But I would be happy to hear of other suggestions as to how that
> information might be conveyed along with the message.

If you want to go around in circles, there are the optional
RFC 2369 and RFC 2919 fields which have already been
mentioned.

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

To: LIST: [email-masked];
Xref: clerew local.mime:3996
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Sat, 23 Oct 2004 20:35:39 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>> The use of the group syntax <LIST:[email-masked];> meets the syntax of
>> RFC 2822, and messages containing it should therefore pass freely through
>> all normal agents. It does not violate RFC 2822 in any way.
>> 
>> RFC 2277 is not mandatory. However, it is indeed somewhat dubious
>> practice to use keywords in fields not specifically designed for them.
>> OTOH, what CAN you use in a group-name that is not intended as an
>> identifier?

>you're not using "LIST" to name the group, you are using it to
>make an assertion about the group.   and even then, it's not at all 
>clear why a list should be treated differently than any other recipient.

Yes, it stretches the intent of group syntax slightly.

But if there IS some way to distinguish LIST addresses from others, then
many benefits accrue, as many people have noted in these threads. For
example, it would show if a message had been crossposted to several lists,
enabling a Reply-to-List button to reply to them all.

But I would be happy to hear of other suggestions as to how that
information might be conveyed along with the message.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3995
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: a header authentication scheme
Date: Sat, 23 Oct 2004 20:32:22 GMT
Lines: 69
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Wed October 20 2004 23:05, Laird Breyer wrote:
>> 
>> On Oct 20 2004, Justin Mason wrote:
>> 
>> > As you said previously, the Received header is always prepended.  This can
>> > be exploited more simply for the Processed header, as long as the
>> > Processed header is *also* prepended *only*, and never appended, by each
>> > processor.
>> 
>> Just a quick explanation on why quoting text is perhaps a good idea.
>> 
>> If each piece of software which writes a Processed header always puts
>> it at the top, then you're right it's enough to look for the next
>> Received header down to discover where the Processed header was added,
>> so there's no need to quote ID or datestamp.
>>
>> The ID or datestamp quoting described is intended to cope with
>> Processed headers which are either added somewhere else (e.g. many
>> filters add headers at the end of the list - sometimes it's simply
>> easier to add at the top, sometimes it's simply easier to add at the
>> bottom), and also to cope with possible third party rearranging of
>> headers (because header order is only guaranteed for Received headers
>> in general).

>A fundamental problem is that the Internet mail model does
>not permit adding arbitrary fields "at the top", "at the bottom",
>or anywhere else. It provides only for addition or modification
>of specific fields by Submission agents, and for time stamp
>fields (a.k.a. Received fields) by SMTP receivers, and for a
>Return-Path field when final delivery is performed.  There is no
>provision for any other addition, and provision only for removal
>of Return-Path fields by an SMTP receiver.  Otherwise,
>message transport via SMTP is intentionally transparent.

Nevertheless, it is often done, and it is not going to stop being done. It
is even useful in tracking problems, provided it is understood that the
extra headers are for tracing of some form, and are always added at the
front so as to give a full history of how the message propagated.

For example, a mail sent to a list may acquire a few Received fields on
the way to the list-expander. There it may acquire an assortment of
List.*: fields, and then some more Received fields as it finds it way to
some final recipient.

Then again, if it passes through a site that does forwarding to some other
address, then you may see assorted X-fields describing its progression
through the forwarding machinery.

And it some site provides a spam or virus filtering service, then one
would expect to see some account of that in the headers, e.g. an account
of the Spamassassin score and the things that contributed to it.

And so if Laird wants to propose his Processed: field and have it
accepted as a proposed standard, then the standard has only to say that it
updates RFC 2822, and that the extra field is to be a "trace" field for
the purposes of RFC 2822.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Cc: IETF-822 IETF-822 <[email-masked]>
From: Martin Trautmann <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Sat, 23 Oct 2004 22:19:51 +0200
To: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

On 2004-Oct-22, at 13:45, Charles Lindsey wrote:

> A file in mbox format cannot contain a news article. If the file is in
> some "mbox-like" format, then it is up to the implementor of the 
> format to
> make provision.

Why can't it be mbox files? There's lots of postings that I archive 
locally. It looks to me like a news article in mbox format. What's the 
difference?

> Indeed so. The message you are now reading has passed through a
> news-to-mail gateway and is therefore, by definition, an email message.

It may be considered as an email form your point of view. But you may 
get the same message from a news server as its news version - and both 
should share the same message-id, marking them as the identical 
message.

On the other hand: what's a posting that I Cc: the same moment to 
someone else? For one it's news, for the other it's mail, but it 
started as both and is the same message. Almost every news reader is 
able to manage both email and news since you can write both a reply or 
a follow-up.

> No. Email and Netnews are distinct media. If a dual-use agent cannot 
> keep
> track of which medium it is handling, then it is broken, as is any 
> system
> which uses a non-specific protocol without suitable tagging.

You might be a hardliner (which is good on an RFC list, but should not 
ignore the real world outside) that something had to be transported via 
its mail or news protocol in order to qualify as mail or news. I wonder 
how according to your definition something from your mbox still does 
qualify as mail, since it is no longer on transport but arrived 
somewhere.

For me even a printout of a e-mail still qualifies as an e-mail, since
- it was sent to a certain receiver
- has a message-id as electronic 'signature'

The difference of news and mail is more my understanding of the 
processing where it differs (e.g. an open posting to everyone on many 
servers I don't know vs. a 'closed' list or unique person). For one or 
the other more or less rules do apply - while lots of these rules 
should be extended and shared for both (such as References, Re:, 
charsets etc.)

>>
>>> And if it absolutely MUST guess which medium it is from examination 
>>> of the
>>> message, then a better discriminator would be the presence or 
>>> absence of a
>>> Return-Path header, of a "^From " line.
>
>> Return-Path fields are only inserted by SMTP when making final 
>> delivery;
>
> Hence the presence of a Return-Path (or of an equivalent "^From " line
> arising from an mbox file) is a sure sign that it is an Email. But 
> absence
> of either of those does not mean it is an email, so this mechanism is 
> no
> more than a means of guessing (fairly reliably) to be used if other
> methods are not available.

On the other hand Newsgroups would qualify as a good news indicator - 
while the message may have been Cc-ed and thus is in fact an email.

But I lost track what for we do need to know the difference: If it's 
here for me, I may add one or the other processing. But I can build a 
posting from an E-Mail or do the opposit as well. It would not be good 
behavior because of many reasons anyway (e.g. someone might be confused 
because of a reply while he can't get the originated message).

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Fri, 22 Oct 2004 20:36:21 -0400
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

On Fri October 22 2004 08:28, Charles Lindsey wrote:

> RFC 2277 is not mandatory.

It is BCP, and as such it constrains IETF protocols, and in fact
uses RFC 2119 terminology in several places to do so, e.g.:

   protocols MUST specify which parts are protocol and
   which are text

RFCs 822 and 2822 clearly specify that display names are
text, not protocol.

> I see nothing of relevance in RFC 3028, unless you are telling me that
> implementations of RFC 3028 cannot accept messages using group syntax at
> all, which would be a pretty poor situation.

It would help if you would actually read the documents that
you are referred to.  RFC 3028 (a Standards Track RFC) is
quite clear:

   That is, routes and group syntax are not permitted.  If multiple
   addresses are required, use a string list.  Named groups are not used
   here.
[...]
   The address primitive never acts on the phrase part of an email
   address, nor on comments within that address.  It also never acts on
   group names, although it does act on the addresses within the group
   construct.

It would help even more if you would familiarize yourself with
relevant Standards Track RFCs and Standards so as to avoid
proposing cockamamie schemes.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Fri, 22 Oct 2004 20:43:22 -0400
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

On Fri October 22 2004 07:54, Charles Lindsey wrote:

> But once the domain name has been copied into the id-right

There is no "copying" involved; RFC 2822 recommends that
id-right *be* a domain name, full stop.

> (whether in 
> upper or lower case) it is no longer a domain name for any protocol
> purpose.

Says who? Certainly not RFC 2822.

>  its only purpose is to be compared for identity
> with other <id-right>s for the removal of duplicates, for threading, etc.

No, Charles; the original message in this discussion mentioned
content identifiers, which explicitly use RFC 822 syntax. There
was no mention of "removal of duplicates" or "threading" or
anything vaguely similar.

> No, there are no such circumstances, and you have explained none, in spite
> of being asked to do so

Several have been provided; the messages were sent to this
mailing list and to you. The message headers in the copies
in this mailing lists archives bear that out.

> Nobody gives a damn whether an <id-right> is a domain or not.

Yet another unsubstantiated Lindsey blanket claim. As is
typical of those, it is false.

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Fri, 22 Oct 2004 20:24:52 -0400
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

On Fri October 22 2004 07:45, Charles Lindsey wrote:

> A file in mbox format cannot contain a news article.

The newsreader rn and its derivatives and a number of other
newsreaders save news articles in that format, and can read
messages (which might be "news" messages or "mail" messages)
in that format.

> There are plenty of hooks within IMAP to enable an implementor to
> distinguish Email from Netnews.

No.

> For example, it is customary to store news 
> articles in folders within a "#news" hierarchy.

No; they may be accessible from another subsystem via that
convention -- and it is merely a convention -- but as messages
may be freely copied from any folder that is readable to any
folder that is writable and/or moved among any writable folders,
folders may contain a mix of messages from various provenances --
with no way to tell what those provenances are.  This has
been explained to you multiple times by multiple people.

> Messages also carry flags 
> with them which can be used to distinguish various classes of message.

There is no "news" flag or anything remotely resembling
one.

> >c) it "got it from" an NNTP server after having passed through a
> >    mail-to-news gateway
> 
> If it came from a mail-to-news gateway, then it was, by definition, a news
> article.
> 
> >I believe that case might apply for some people who read *this*
> >mailing list; N.B. this is a mailing list, not a newsgroup, and if one
> >insists on making a distinction, the messages sent by this list's
> >expander are unquestionably mail.
> 
> Indeed so. The message you are now reading has passed through a
> news-to-mail gateway and is therefore, by definition, an email message.

Yes, and the message that it is a response to was also an email
message, whether or not you viewed it in a newsreader.
 
> I see nothing in RFC 3458 of relevance to this matter.

Try actually reading the RFC -- it defines a field which can identify
one of several applications of the RFC 2822 message format; fax,
voice messaging, etc.  If the applicability isn't clear to you, then
you can of course ask Ned for clarification, since he suggested it.

> If it is in an
> IMAP box, then it depends how it was put there, and that is a matter for
> the IMAP implementor.

No. IMAP is a Standards Track protocol that handles messages;
"mail", "news", "hand-crafted" is immaterial. Human users using
a variety of standards-compliant UAs can copy/move messages
between folders at will, as many times as they wish, and there is
no way (or need) to track such movement.  This too has been
explained to you many times, and you have been informed in
no uncertain terms that any attempt by you to try to impose
any such requirements on IMAP will not pass IESG muster.

> >It's handling "messages".  When a message is being transported, it
> >might be transported via a news-specific protocol, or via a mail-specific
> >protocol, or via a protocol which is non-specific.  Once it has been
> >transported, it is simply a message.
> 
> No. Email and Netnews are distinct media.

Some (but not all) transport methods are specific to one or
the other. Some (but not all) message header fields have
defined semantics for one or the other.  But the basic format
is intentionally the same, for reasons of code reuse etc. that
are spelled out in RFCs 850 and 1036 (section 2 of each).

> If a dual-use agent cannot keep 
> track of which medium it is handling, then it is broken, as is any system
> which uses a non-specific protocol without suitable tagging.

What are "broken" are your ability to comprehend what has been
explained to you several times by several people, and your ability
to realize that your ability to ram your odd ideas down the IESG's
throats is nil.

> >Return-Path fields are only inserted by SMTP when making final delivery;
> 
> Hence the presence of a Return-Path (or of an equivalent "^From " line
> arising from an mbox file) is a sure sign that it is an Email.

No; as usual you haven't been paying attention. A message having
been delivered by SMTP to a mail-to-news gateway may very well
still have a Return-Path field when it is transported by a "news"
protocol.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>, [email-masked]
Subject: Re: a header authentication scheme
Date: Fri, 22 Oct 2004 19:18:24 -0400
Sender: [email-masked]

On Thu October 21 2004 22:34, Laird Breyer wrote:

> In another post, I mentioned possibly extending the RFC 2822 date-time stamp
> to include microseconds.

Two problems:
1. date-time is pervasive; it is used in many places in addition to
   time stamp lines
2. while providing seconds was once mandatory (RFC 821), it is
   now optional (RFCs 2821/2822). So you might not even have
   seconds.  You seem to want to buck the trend.

> > 2. Quoting the "with" ID may provide protection, depending on whether 
> > the ID is guessable.
> > 
> 
> Yes. From the RFC, this is optional. Do you know any good reasons why the ID 
> might not be added to a Received line sometimes? 

Umm, because it is optional.  Past implementations have associated
the identifier with a set of files used in store-and-forward
processing.  In some cases, messages might be handled w/o
any associated file storage, therefore no need for an id.

There also appears to be some confusion between "with"
(which specifies a protocol (SMTP, ESMTP, LMTP, etc.) and
"id" which provides an identifier of some sort (RFCs
821/822/2821/2822 all differ on what is allowed).

Date: Fri, 22 Oct 2004 12:39:59 -0400
From: Keith Moore <[email-masked]>
To: "Charles Lindsey" <[email-masked]>
Cc: [email-masked], LIST: [email-masked];
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Sender: [email-masked]

> The use of the group syntax <LIST:[email-masked];> meets the syntax of
> RFC 2822, and messages containing it should therefore pass freely through
> all normal agents. It does not violate RFC 2822 in any way.
> 
> RFC 2277 is not mandatory. However, it is indeed somewhat dubious
> practice to use keywords in fields not specifically designed for them.
> OTOH, what CAN you use in a group-name that is not intended as an
> identifier?

you're not using "LIST" to name the group, you are using it to
make an assertion about the group.   and even then, it's not at all 
clear why a list should be treated differently than any other recipient.

To: LIST: [email-masked];
Xref: clerew local.mime:3988
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Fri, 22 Oct 2004 11:54:44 GMT
Lines: 47
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon October 18 2004 09:56, Charles Lindsey wrote:

>> Ergo there is no necessity for <id-right>s to be compared
>> case-insensitively

>No, in the first place it is recommended to be a domain name,
>which is always case-insensitive.

But once the domain name has been copied into the id-right (whether in
upper or lower case) it is no longer a domain name for any protocol
purpose. It is merely a string of characters consistent with the
<id-right> syntax. No subsequent transport or copying process has any
business changing it, and its only purpose is to be compared for identity
with other <id-right>s for the removal of duplicates, for threading, etc.
For those purposes, a case-sensitive comparison is perfectly adequate.

>Moreover, there are various circumstances (explained at length)
>where case of the RHS may be changed, and only a case-
>insensitive comparison will work correctly under those (not
>unusual) circumstances.

No, there are no such circumstances, and you have explained none, in spite
of being asked to do so (I do not regard your friend reading a message
identifier over the telephone as a significant problem).

>> and the change introduced by RFC 2822 is a welcome 
>> simplification of the protocol.

>It is not a simplification; it complicates matters since it is
>no longer possible to determine whether or not the RHS is
>a domain name, and if it is not, there is no indication of
>whether or not it is case-sensitive.

Nobody gives a damn whether an <id-right> is a domain or not.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3989
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Fri, 22 Oct 2004 12:28:39 GMT
Lines: 41
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon October 18 2004 10:16, Charles Lindsey wrote:

>> So I have suggested a method using group-syntax that would be entirely be
>> compatible with the present protocols,

>No, attempting to change display text into a protocol keyword is
>incompatible with the text message format (both the full Standard
>and the Proposed Standard), is incompatible with the Internet
>Architecture and with BCP (RFC 2277), and use of named groups
>is incompatible with the Standards Track Sieve protocol (RFC 3028).
>It is also anglo-centric and ambiguous.  All of which has been
>explained, with cited references.  Your claim of compatibility is
>a falsehood.

The use of the group syntax <LIST:[email-masked];> meets the syntax of
RFC 2822, and messages containing it should therefore pass freely through
all normal agents. It does not violate RFC 2822 in any way.

RFC 2277 is not mandatory. However, it is indeed somewhat dubious
practice to use keywords in fields not specifically designed for them.
OTOH, what CAN you use in a group-name that is not intended as an
identifier? To whom is it _meant_ to be meaningful. But the point is
taken, which is why a have admitted several times that my proposal is not
a "pretty solution"; but nobody has come up with a better one.

I see nothing of relevance in RFC 3028, unless you are telling me that
implementations of RFC 3028 cannot accept messages using group syntax at
all, which would be a pretty poor situation.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3987
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Fri, 22 Oct 2004 11:45:01 GMT
Lines: 94
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon October 18 2004 10:07, Charles Lindsey wrote:

>> So a Dual-Use User Agent needs some better way to determine whether the
>> message in hand is an email message or a news article. The proper way to
>> do this is to keep track of where it got it from.

>A couple of typical cases are:
>a) it "got it from" a flat mbox-like file or a single text message file

A file in mbox format cannot contain a news article. If the file is in
some "mbox-like" format, then it is up to the implementor of the format to
make provision.

>b) it "got it from" an IMAP "folder"

There are plenty of hooks within IMAP to enable an implementor to
distinguish Email from Netnews. For example, it is customary to store news
articles in folders within a "#news" hierarchy. Messages also carry flags
with them which can be used to distinguish various classes of message.

>c) it "got it from" an NNTP server after having passed through a
>    mail-to-news gateway

If it came from a mail-to-news gateway, then it was, by definition, a news
article.

>I believe that case might apply for some people who read *this*
>mailing list; N.B. this is a mailing list, not a newsgroup, and if one
>insists on making a distinction, the messages sent by this list's
>expander are unquestionably mail.

Indeed so. The message you are now reading has passed through a
news-to-mail gateway and is therefore, by definition, an email message.

>As Ned Freed has explained to you, if there is a legitimate need to
>be able to make a distinction, there is a standardized mechanism
>available via RFC 3458.  However I doubt that there is such a
>need in the first place, and I doubt that any mechanism (including
>RFC 3458) will work reliably when messages pass through gateways,
>are copied to files, moved and/or copied between folders, etc.

I see nothing in RFC 3458 of relevance to this matter. It seemes to be
directed more to determining whether you are to read/listen-to/obey the
message when it arrives.

>And if it was delivered via SMTP to a mail-to-news gateway, then
>transported via NNTP? Or transported via NNTP to a news-to-mail
>gateway, then delivered via SMTP?  What if it was delivered via
>UUCP? Or accessed via IMAP?

In the first case it is a news article, in the second it was a mail
message. If it arrived via UUCP, then the transport will have delivered it
either to the 'rmail' program or to the 'rnews' program. If it is in an
IMAP box, then it depends how it was put there, and that is a matter for
the IMAP implementor.

>> If an Agent is setting out to be for "dual-use" between two 
>> media, then it behoves it to know which medium it is handling at any given
>> moment.

>It's handling "messages".  When a message is being transported, it
>might be transported via a news-specific protocol, or via a mail-specific
>protocol, or via a protocol which is non-specific.  Once it has been
>transported, it is simply a message.

No. Email and Netnews are distinct media. If a dual-use agent cannot keep
track of which medium it is handling, then it is broken, as is any system
which uses a non-specific protocol without suitable tagging.
> 
>> And if it absolutely MUST guess which medium it is from examination of the
>> message, then a better discriminator would be the presence or absence of a
>> Return-Path header, of a "^From " line.

>Return-Path fields are only inserted by SMTP when making final delivery;

Hence the presence of a Return-Path (or of an equivalent "^From " line
arising from an mbox file) is a sure sign that it is an Email. But absence
of either of those does not mean it is an email, so this mechanism is no
more than a means of guessing (fairly reliably) to be used if other
methods are not available.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>, [email-masked]
Subject: Re: a header authentication scheme
Date: Fri, 22 Oct 2004 11:06:52 -0400
Sender: [email-masked]

On Wed October 20 2004 23:05, Laird Breyer wrote:
> 
> On Oct 20 2004, Justin Mason wrote:
> 
> > As you said previously, the Received header is always prepended.  This can
> > be exploited more simply for the Processed header, as long as the
> > Processed header is *also* prepended *only*, and never appended, by each
> > processor.
> 
> Just a quick explanation on why quoting text is perhaps a good idea.
> 
> If each piece of software which writes a Processed header always puts
> it at the top, then you're right it's enough to look for the next
> Received header down to discover where the Processed header was added,
> so there's no need to quote ID or datestamp.
>
> The ID or datestamp quoting described is intended to cope with
> Processed headers which are either added somewhere else (e.g. many
> filters add headers at the end of the list - sometimes it's simply
> easier to add at the top, sometimes it's simply easier to add at the
> bottom), and also to cope with possible third party rearranging of
> headers (because header order is only guaranteed for Received headers
> in general).

A fundamental problem is that the Internet mail model does
not permit adding arbitrary fields "at the top", "at the bottom",
or anywhere else. It provides only for addition or modification
of specific fields by Submission agents, and for time stamp
fields (a.k.a. Received fields) by SMTP receivers, and for a
Return-Path field when final delivery is performed.  There is no
provision for any other addition, and provision only for removal
of Return-Path fields by an SMTP receiver.  Otherwise,
message transport via SMTP is intentionally transparent.

Incidentally, RFC 2822 requires that order of trace (i.e.
Received and Return-Path) and Resent- fields be preserved;
"only guaranteed for Received" is incorrect.

Once a message leaves the SMTP transport environment, of
course there can be additional processing.  However, if the
message content (header or body) is changed, then one
is no longer dealing with exactly the *same* message.
There is an existing mechanism that can be used to allow
addition of arbitrary content, with authentication, and
while permitting the original message to be recovered
intact: simply wrap the original as MIME type message/rfc822,
if there is additional content, it and the wrapped message
can be packaged as multipart/mixed, and if authentication
is desired that can then be signed using the multipart/signed
MIME type (S/MIME and PGP/MIME use that mechanism and
are widely deployed).

Such additional content need not be formatted as message
header fields; it could be any media type, including plain
text, some XML variant, etc.   In particular, it need not
inflict on parsers the need to be able to handle the
complexities associated with RFC 2231 parameters.

> identifying date-time: unfold the Received line, normalize spaces,
> look for the first ';' from the end and take everything after that.

No.
  Received: (a comment; blah blah) ....
will not be handled properly by that simplistic algorithm. Nor
will a field with a semicolon anywhere else (e.g. in the identifier).

> identifying ID: unfold the Received line, take the first string after
> the keyword "id" if it exists (if you want to be more correct, remove
> comments (in parentheses),

Again, no.
  Received: from from . by . id by example . com ...
will mistake the (valid!) top-level domain name component "id"
for the "id" attribute name.

> then look for the keywords from, by and 
> pick the first time you see " id " after that.

No, still NG for the simple example above.

> Or you can properly 
> parse the line according to RFC 2822 and try to figure out the id.

There are several problems with that:
1. Received fields may have been generated in accordance with
    RFC 2821 (indeed, that is likely the case, since it is SMTP
    receivers which are supposed to generate the field), and there
   are conflicts between RFCs 2821 and 2822.
2. Received fields may have been generated in accordance with
    RFC 821, and there are conflicts between RFCs 821 and 822.
3. Some broken software generates fields which cannot be parsed
    even with the very liberal RFC 2822 parse syntax, e.g.
  Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
         Sun, 15 Aug 2004 23:38:16 -0700
4. The id component is optional; it might not be present at all.

For that matter, RFC 2822 has relaxed the syntax to the oxymoronic
point that the time stamp need not be present in the time stamp
line:
  Received:
is a perfectly legal complete Received field per RFC 2822 parse syntax.

> By the way, there's an even easier way to quote a piece of the
> Received line for use within a Processed header field:
> unfold the Received line, normalize spaces, then compute a hash value
> of the whole line.

Provided that the algorithm for unfolding and normalization are clearly
and carefully specified and implemented, that could avoid the problems
noted above.

> Then if you want to verify whether a Processed 
> header is associated with a given Received line, do the same thing and
> check the hash value.
> 
>    Received: from 185129182.virtua.com.br (185129182.virtua.com.br
>        [200.185.129.182]) by smtpin-3211.bay.webtv.net (WebTV_Postfix+sws)
>        with SMTP id 7280B11DCC; Fri,  5 Mar 2004 19:00:48 -0800 (PST)    
> 
> This string could hash to the value B17C07174B5E4546A2B04EB096E83FD7081936B8 say,
> and then you would have 
> 
>    Processed: name="SpamAssassin"; location-ip="1.2.3.4";
>        version="2.63"; function="spamcheck";
>        auth-received="B17C07174B5E4546A2B04EB096E83FD7081936B8"; 
>        result-tag="spam";

There's a more fundamental issue with the whole scheme; you're
simply trying to authenticate some added content (and as noted
above, there are existing mechanisms that can do so without
running roughshod over the existing transport mechanism). Let's
assume for the moment that a 100% foolproof authentication
mechanism is in place and that the field above is guaranteed to
have been inserted by the party that claimed to do so.  I submit
that it tells me nothing particularly useful; I have no guarantee
that the claimed processing was performed or that the claimed
results are accurate.  Even if I have blind trust in the party that
claimed to have inserted that field, I have no way of knowing if
the claimed name "SpamAssassin" is what I think it is, and if it is,
I have no idea whether it was modified at that site or if it is
"vanilla" SpamAssassin.  Even if I assume that it is unmodified,
I have no idea what configuration files were used with it or
what run-time options were specified.  In short, purported results
of an indeterminate process with indeterminate configuration
and indeterminate run-time parameters allegedly run at a
remote site are of little value.

From: "Hector Santos" <[email-masked]>
To: "IETF-822" <[email-masked]>
Subject: Re: Survey: Mail User Agents (MUA) Email Software 
Date: Fri, 22 Oct 2004 09:11:52 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

More ISP/Customer feedback:

----- Original Message -----
From: "Chris Taylor" <[email-masked]>
To: <[email-masked]>
Sent: Friday, October 22, 2004 7:27 AM
Subject: Re: Survey: Mail User Agents (MUA) Email Software

> I would like to see an option for each list. 2 radio buttons. One being
"Set
> reply address to the list". The other "Set reply address to the sender"
>
> The default should be "Set reply address to the list" as that has been the
> behaviour with wcListserver in the past and one would not want surprises.
>
> This topic is the source of many a flame war on various lists. As you
said,
> there is no right or wrong answer, so let administrators choose.
>
> Chris
>

----- Original Message -----
From: "Rick Spilman" <[email-masked]>
To: <[email-masked]>
Sent: Friday, October 22, 2004 8:49 AM
Subject: Re: Survey: Mail User Agents (MUA) Email Software

> > Questions:
> >
> > 1) What MUA are you using?
>
> Outlook Express
>
> > 2) What are some of the MUAs used by your users?
>
> Outlook Express, Outlook, Incredimail
>
> > 3) What Reply Options does it have for email?
>
> The usual, Reply to Sender, Reply to All, Forward
>
> > 4) What is better?
>
> I would concur with some of the other respondents that what is presently
> there should remain the 'Default'.  I say this as most SYSOPS and Users
are
> already familiar with how the systems work and it works well for most all
> situations.  However, for those situations as some have mentioned, the
idea
> of 2 buttons in the setup would be ideal if that is possible.  Since the
> present usage would be the default, most folks would never have to use
> them.  But, when setting up a new list, the option to choose would give
the
> best of both worlds and the best flexibility for a List Server on the Net.
>
> Rick Spilman
> [email-masked]
> http://www.srupc.com/
> http://rbs.srupc.com/            Remote Backup A Must Have!
> http://www.petatorium.com/
>

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Fri, 22 Oct 2004 13:30:35 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: a header authentication scheme
Cc: [email-masked]
Sender: [email-masked]

Laird Breyer writes:
> On Oct 21 2004, Arnt Gulbrandsen wrote:
>>  I guess I wasn't clear. Sorry. Let me try again.
>>
>>  1. Quoting the time isn't good. If I'm allowed 150 tries and can 
>>  bounce some mail off your server to read the bounce messages, I can 
>>  guess when you'll process a message.
>
> Ah, I think I understand your point. You could use 150 bounces as 
> tracers, to compute an estimate of the processing time: e.g. send a 
> test message designed to bounce, containing the time that the SMTP 
> transaction was initiated, and later read back the timestamp which 
> was added. Then do a regression line estimate. Something like that 
> could probably work, although a proof of concept would be nice.

Give me an address and I can try.

Step 1. I'd send five messages to [email-masked] and wait for the 
bounces. For each of them, I'd compute the delay from my sending time 
(using my clock) to the receiving MTA's receive time. Next, I'd average 
the five values.

Step 2. I'd send the target a hundred messages or so, with a hundred 
guesses centered around the average computed in step 1.

Assuming that 80% of mail from me to the target are delivered within the 
range [t-50,t+50> seconds, I'd have an 80% chance of success.

A more sophisticated attack would use more initial probes and vary the 
number of guesses based on the spread of the probed delays.

Should I bother to do it, or is the description persuasive? ;)

> In another post, I mentioned possibly extending the RFC 2822 date-time 
> stamp to include microseconds. Would that be sufficient protection 
> against such a timing attack, or would Moore's law render this 
> useless over time?

Microseconds would help, since it's harder for the attacker to cover the 
likely correct answers. But it's only provides three digits. If the ID 
is used, the recipient can add unguessable entropy to taste, without 
requiring RFC2822 changes.

>>  2. Quoting the "with" ID may provide protection, depending on 
>>  whether the ID is guessable.
>
> Yes. From the RFC, this is optional. Do you know any good reasons why the ID
> might not be added to a Received line sometimes?

I don't know any good reason. If it matters, MTA authors quickly add one.

Arnt

From: "Hector Santos" <[email-masked]>
To: "IETF-822" <[email-masked]>
Subject: Fw: Survey: Mail User Agents (MUA) Email Software 
Date: Fri, 22 Oct 2004 02:55:47 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

Since we never got a complaint or even a comment about this issue from our
ISP customers in regards to how our WINS (Wildcat! Interactive Net Server)
List Server product forces the Reply-To: for the list distribution,   I
asked in our support mailing list for some feedback.  I posted the survey
questions less than 1 hour go and received a few feedbacks already I would
like to share.

The basic questions were:

1) What MUAs do they and their users use?
2) What should be the natural reply default behavior for a list?
3) Making the List Server "Reply-To:" distribution feature optional?

----- Original Message -----
From: "Xavier de la Piedra" <[email-masked]>
To: <[email-masked]>
Sent: Friday, October 22, 2004 2:22 AM
Subject: RE: Survey: Mail User Agents (MUA) Email Software

> I use Outlook 2003.  It's added email filtering really helps.  Most of my
> users use some version of Outlook.  I would say a small percentage (1-5%)
> use Netscape as their mail client.
>
> Personally on the list server I like the Reply going staight to the list.
> Less tech support since most of my users understand that method to send
info
> back to the list.  Just like I did now in fact.
>
> Xavier

----- Original Message -----
From: "Mike Davis" <[email-masked]>
To: <[email-masked]>
Sent: Friday, October 22, 2004 2:29 AM
Subject: Re: Survey: Mail User Agents (MUA) Email Software

> I'm glad you're looking at this... the MUAs I'm aware of being used on the
> two WINS systems I administer are Outlook, Outlook Express, and Eudora,
they
> all function pretty much the same.
>
> I would really like to see the reply-to option that you discussed.  In
most
> cases with a standard list I would keep things configured just like they
> are... but I also use listserver for mail distribution lists.  This
differs
> in the fact that anybody can post to the list, and the message then gets
> distributed to those addresses configured.  I set this up in two steps...
> first configure a standard WINS list but with no posting allowed, for
> example operations-dist, stored in a conference called operations-dist.
> Then I go into the mail server setup and configure a mail list to put all
> mail addressed to operations into the operations-dist conference.  This
way
> I can have copies of all e-mail sent to operations sent to everybody who
> needs to see it without everybody sending the mail needing to join a list.
> The only problem is that if these recipients then hit reply, the address
is
> the list address... which is a non-posting list, and not who they wanted
to
> send it to anyway.  Being able to turn off this "reply-to the list"
function
> would make this a perfect way to setup distribution lists.
>
> Mike

Mike's reply to Xavier:

----- Original Message -----
From: "Mike Davis" <[email-masked]>
To: <[email-masked]>
Sent: Friday, October 22, 2004 2:31 AM
Subject: Re: Survey: Mail User Agents (MUA) Email Software

> > Personally on the list server I like the Reply going staight to the
list.
> > Less tech support since most of my users understand that method to send
> > info
> > back to the list.  Just like I did now in fact.
>
> I agree, this would be the most common, and default configuration, but as
I
> described before, I could and would certainly use the ability to turn this
> off.
>
> Mike

Date: Fri, 22 Oct 2004 12:34:43 +1000
From: Laird Breyer <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: Arnt Gulbrandsen <[email-masked]>, [email-masked], [email-masked]
Sender: [email-masked]

On Oct 21 2004, Arnt Gulbrandsen wrote:
> I guess I wasn't clear. Sorry. Let me try again.
> 
> 1. Quoting the time isn't good. If I'm allowed 150 tries and can bounce 
> some mail off your server to read the bounce messages, I can guess when 
> you'll process a message.
> 

Ah, I think I understand your point. You could use 150 bounces as tracers,
to compute an estimate of the processing time: e.g. send a test 
message designed to bounce, containing the time that the SMTP transaction
was initiated, and later read back the timestamp which was added. Then 
do a regression line estimate. Something like that could probably work,
although a proof of concept would be nice.

In another post, I mentioned possibly extending the RFC 2822 date-time stamp
to include microseconds. Would that be sufficient protection against such
a timing attack, or would Moore's law render this useless over time?

> 2. Quoting the "with" ID may provide protection, depending on whether 
> the ID is guessable.
> 

Yes. From the RFC, this is optional. Do you know any good reasons why the ID 
might not be added to a Received line sometimes? 

> 3. A receiver can make its IDs less guessable. A few bytes of randomness 
> are easily obtained. (A receiver cannot easily make the time of day 
> more difficult to predict.)

One easy way is to add a small random delay in the middle of the
processing step.  If for example each transaction is on a different
thread, this delay would be used to sleep while another thread
executes, so time isn't wasted.
 
> 
> 4. Most MTA authors seem eager to implement anything that helps against spam.
> 

I expect so. I think it's important to try to work with the existing
standards as much as possible, which is why I'm looking at ID and date
rather than say something like: MTA authors must insert a new random
string token in each Received line which they write.

> Conclusion. Quoting the "with" ID is good, and the MTAs that currently 
> have guessable IDs will change that quickly if spammers ever exploit 
> it.

Thanks, those comments are helpful.

-- 
Laird Breyer.

Date: Fri, 22 Oct 2004 12:11:53 +1000
From: Laird Breyer <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], Bart Schaefer <[email-masked]>
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: Arnt Gulbrandsen <[email-masked]>, [email-masked], Bart Schaefer <[email-masked]>
Sender: [email-masked]

On Oct 21 2004, Arnt Gulbrandsen wrote:
> 
> Bart Schaefer writes:
> > I didn't say there's no reason to allow the processing.  I said 
> > there's no reason for the end recipient to believe the Processed: 
> > header, whether or not said header appears to be tied to a Received: 
> > line, except on the last hop.
> 
> Oh, sorry. I misunderstood then. I more or less agree with what you're 
> saying, in that case.  (Except that IMO, hops internal to the 
> destination organization can often be trusted.)

I agree with both of you on this. There's no intrinsic reason to 
trust anything added before the last hop. 

I argue here that there is sometimes a real need to trust more than the
last hop (maybe two or three hops only), but the ultimate decision requires
another mechanism (edict from admin, centrally configured CIDR masks
as posted by Justin Mason, etc).

What the Processed scheme is intended for is to reliably hang the processing
result onto a hop if possible, not authenticate the hop or the
Processed field itself. This way, if the hop is trusted for some
reason, then those associated Processed fields can be trusted, too.

I'm using words such as authentication and trust for this, but really
I don't know of a better terminology that would be more appropriate.
Suggestions welcome.

-- 
Laird Breyer.

Date: Thu, 21 Oct 2004 17:59:37 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Cc: Bruce Lilly <[email-masked]>
Sender: [email-masked]

Bruce Lilly writes:
> The point is that the fields provided for are optional; therefore in 
> their absence one cannot determine whether a message has been 
> expanded by a list expander (for that matter, although the fields are 
> intended for use by list expanders, one cannot be certain that the 
> presence of any of the fields guarantees that that message has passed 
> through a list expander).

Isn't this intrinsic to all in-message flags? That is, isn't this 
something that no in-message proposal can solve, and that therefore, 
2369/2919 are as good as it gets in-message?

Even if a new header field (or a modification of a current header field) 
made it to Full Standard before Christmas, very little software would 
be aware of the new RFC.

> Moreover and more importantly, as you pointed out a few weeks ago, the 
> provision for such fields does absolutely nothing to identify whether 
> or not a given mailbox (e.g. [email-masked]) might expand to a list 
> when one is composing a message with that mailbox as a recipient; 
> that is precisely the time when an author might want to know that 
> list expansion may be involved so that he can set the Reply-To field 
> to point to that list if he wishes responses to go to that list.

Right.

Arnt

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Thu, 21 Oct 2004 11:25:47 -0400
Cc: "Hector Santos" <[email-masked]>
Sender: [email-masked]

On Tue October 19 2004 13:47, Hector Santos wrote:
>  You treat input as it is all nonsense opinions.

No -- not "all".  But there is certainly *some* nonsense.

> I clearly stated my view of the problem and I stated a solution.

Your "solution" (list expander overriding author-set Reply-To)
is no solution; is the source of a great many problems which
have been explained to you by several people.

>  quite frankly, my credentials as a
> software developer and product producer in this market is very much evident
> over yourself. 

You -- or somebody at your site -- seem to have inordinate
difficulty in handling receipt of mail:

  [email-masked]
    SMTP error from remote mailer after RCPT TO:<[email-masked]>:
    host mail.winserver.com [208.247.131.9]: 550 Return Path not verifiable.

> Now, if you can
> tell me I am wrong, well, you better be ready to show me how with sound
> reasons.

Sound reasoning has been given to patiently explain to you not
only that you are wrong, but why that is the case.

> I am not stubborn to understand

So you say, yet you persist in making the same erroneous
statements over and over and scream "SHUT UP" whenever
anybody points out your errors.

> The RFC based OFFLINE MUA does not understand mailing list concepts for two
> reasons:
> 
> - They are NOT trained to understand the concept,

If *your* MUA can't handle mailing list messages, then by
all means feel free to "train" it.

> - The BACKEND does not have a standard to help them better work in this
> area.

You still haven't defined "backend"; SCREAMING won't help.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: Understanding response protocols
Date: Thu, 21 Oct 2004 11:27:37 -0400
Cc: "Hector Santos" <[email-masked]>
Sender: [email-masked]

On Tue October 19 2004 14:17, Hector Santos wrote:
> 
> ----- Original Message -----
> From: "Bruce Lilly" <[email-masked]>

> > Again, the concept of a "default" is an rMUA issue, not a
> > characteristic of a Reply-To field.
> 
> I disagree.  From a technical standpoint, this is a sound definition of
> a default reply address.

You can disagree all you like, but you need a valid basis for
such disagreement. The semantics of the Reply-To field are
well-defined, and have been for a quarter century; the definition
of the field certainly does not include anything about "default
reply address".  Claiming that something is a definition when it
clearly is not is simply wrong.

> Again, we need to take into again the the 1:1 vs 1:many concept we
> are dealing with here.  The MUA is not "designed" to handle this concept
> of how the original intention of the message list "holder" is distributing
> the mail.

You keep repeating the same mantra; repeating it does not
make it true.  Many MUAs work fine with authors who are
composing messages to be sent to mailing lists; if a particular
MUA does not, that is an issue with that particular MUA
implementation.  Mailing lists didn't just pop into existence
the middle of last week; they have been around longer than
some of the core email protocols, and MUA authors have had
ample opportunity to consider the long-standing provisions
in the message protocols for dealing with list expansion -- some
have done well at it; just because some few MUA authors
have done a miserable job is no reason to dismantle the long-
standing message format.  What makes you think that MUA
authors who have done a miserable job up to the present
will suddenly get things right if the message format is
changed?  

>  like a Newsgroup Reader
> where the default is a "reply to group" and the "reply to sender" is a
> secondary
> option.

Newsgroups and mailboxes are different entities; *of course* there
is a difference at some protocol levels between posting to a
newsgroup and mailing to a mailbox (however, as the basic
message format has been the same since RFC 850 (two decades),
and as there are gateways between news and mail, there are
also a great many situations where it is not possible to make
a distinction).  A UA performs a specific set of functions; among
them is acting as an intermediary between an author and separate
transport agents. A message can be posted to a newsgroup
(e.g. via NNTP) or mailed to a mailbox (e.g. via SMTP), but it cannot
be posted to a newsgroup (directly) with SMTP or mailed to a
specific set of mailboxes via NNTP; *of course* the author needs to
indicate to a UA which method of transport is desired!  Perhaps
somewhere there is some newsreader which operates as you describe
"the default is...", but the ones that I have used generally have
separate but equal functionality for posting to newsgroups, mailing
to originators, and -- additionally since you neglected to mention
it -- doing both.

> > One rather obvious way would be to present multiple choices
> > equally (and correctly labeled) rather than a default (with or
> > without other alternatives which are less accessible).
> 
> Ok, but why?  Most MUA designers would prefer to a have a "concise"
> offering to the user

There is no reason why multiple choices cannot be a concise set
of choices; I specifically mentioned a particular UA which provides
three choices in its default configuration.

> for that the intentions of the backend is.

Be wary of assumptions of a particular architecture.

> If the user  
> wants to go offline, those are secondary options.

Offline vs. online is irrelevant to the discussion. It doesn't
matter whether messages are sent (or received) immediately
or queued.

> Options are always to have, but when it comes to a default, the user should
> be presented with a clearly understanding of how to response to a list.

You are assuming that users can always identify whether a list is
involved; that is clearly incorrect, and any conclusions drawn from
premises including that incorrect assumption are suspect.

> You can't saying the backend has nothing to do with it. I disagree.

You haven't clearly defined what you mean by "backend".

> A list server can fix the reply-to address to the
> list address once it takes responsibility for the new distribution as a
> "List User" entity.
>
> The issue with this?

It doesn't need to be fixed; it's not broken -- Reply-To is set by the
author.  Indeed, any list expander that overwrites the author's
explicit setting of the Reply-To field is broken; it prevents the author's
specific recommendation from being communicated to his recipients
and misrepresents something as coming from the author when it has
been produced elsewhere -- it is censorship combined with forgery.

> For the legacy MUA, the secondary option to "reply direct" is no longer an
> easy option.

That may be so for a *particular* MUA, but it is not true in
general. There are *many* MUAs which have separate capability
of addressing a response to the author(s) of an original message
as an alternative to using the Reply-To field recommendation.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Thu, 21 Oct 2004 09:21:22 -0400
Cc: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

On Tue October 19 2004 10:24, Arnt Gulbrandsen wrote:
> >>  >There is no general mechanism to identify a list,
> >>
> >>  Then invent one!
> >
> > Perhaps you should consider the possibility that that is an impossible task.
> 
> We have two RFCs that do the task

The point is that the fields provided for are optional; therefore
in their absence one cannot determine whether a message has
been expanded by a list expander (for that matter, although
the fields are intended for use by list expanders, one cannot
be certain that the presence of any of the fields guarantees
that that message has passed through a list expander). 
Moreover and more importantly, as you pointed out a few
weeks ago, the provision for such fields does absolutely
nothing to identify whether or not a given mailbox (e.g.
[email-masked]) might expand to a list when one is
composing a message with that mailbox as a recipient; that
is precisely the time when an author might want to know
that list expansion may be involved so that he can set the
Reply-To field to point to that list if he wishes responses to
go to that list.

Date: Thu, 21 Oct 2004 17:16:24 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: a header authentication scheme
Cc: Bart Schaefer <[email-masked]>
Sender: [email-masked]

Bart Schaefer writes:
> I didn't say there's no reason to allow the processing.  I said 
> there's no reason for the end recipient to believe the Processed: 
> header, whether or not said header appears to be tied to a Received: 
> line, except on the last hop.

Oh, sorry. I misunderstood then. I more or less agree with what you're 
saying, in that case.  (Except that IMO, hops internal to the 
destination organization can often be trusted.)

Arnt

Date: Thu, 21 Oct 2004 08:00:30 -0700
From: Bart Schaefer <[email-masked]>
Reply-To: Bart Schaefer <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Subject: Re: a header authentication scheme
Cc: [email-masked]
Sender: [email-masked]

On Thu, 21 Oct 2004 13:23:38 +0200, Arnt Gulbrandsen
<[email-masked]> wrote:
> 
> (In a different message, someone mentioned that there's no reason to
> allow such processing before the last hop. [...])

I didn't say there's no reason to allow the processing.  I said
there's no reason for the end recipient to believe the Processed:
header, whether or not said header appears to be tied to a Received:
line, except on the last hop.

Date: Thu, 21 Oct 2004 12:57:52 +0100
From: Tony Finch <[email-masked]>
To: Justin Mason <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: a header authentication scheme 
Sender: [email-masked]

On Wed, 20 Oct 2004, Justin Mason wrote:
>
> As you said previously, the Received header is always prepended.  This can
> be exploited more simply for the Processed header, as long as the
> Processed header is *also* prepended *only*, and never appended, by each
> processor.

There's a bug in 2822 that it doesn't allow Received: fields to be
interleaved with other trace fields, except for Resent- fields.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SOUTH BISCAY: SOUTHWEST 6 OR 7, DECREASING 4 OR 5. SHOWERS. GOOD.

Date: Thu, 21 Oct 2004 13:23:38 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: a header authentication scheme
Cc: [email-masked], [email-masked]
Sender: [email-masked]

I guess I wasn't clear. Sorry. Let me try again.

1. Quoting the time isn't good. If I'm allowed 150 tries and can bounce 
some mail off your server to read the bounce messages, I can guess when 
you'll process a message.

2. Quoting the "with" ID may provide protection, depending on whether 
the ID is guessable.

3. A receiver can make its IDs less guessable. A few bytes of randomness 
are easily obtained. (A receiver cannot easily make the time of day 
more difficult to predict.)

4. Most MTA authors seem eager to implement anything that helps against spam.

Conclusion. Quoting the "with" ID is good, and the MTAs that currently 
have guessable IDs will change that quickly if spammers ever exploit 
it.

(In a different message, someone mentioned that there's no reason to 
allow such processing before the last hop. There is, IMO: It's too 
difficult to do otherwise. For example, suppose [email-masked] is an 
alias for local user Alice and remote user [email-masked]. For mail to 
[email-masked], the example.com MTA is last hop for one user, not 
for the other.)

Arnt

Date: Thu, 21 Oct 2004 11:26:30 +0200
From: Alan Barrett <[email-masked]>
To: [email-masked]
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Reply-To: [email-masked]
Sender: [email-masked]

On Wed, 20 Oct 2004, Charles Lindsey wrote:
> Now if you want a really insoluble problem, then how about a system that
> lets some people choose to see a reply with top posting, and others see
> the same reply with bottom posting. Now there's a _real_ challenge -;) .

Assuming "top posting" and "bottom posting" refer to the habit of
including an un-trimmed copy of the previous message, I don't
want to see either of them.  I want to see replies that contain
carefully trimmed quotes.

--apb (Alan Barrett)

Date: Thu, 21 Oct 2004 09:20:21 +0100 (BST)
From: Philip Hazel <[email-masked]>
To: Laird Breyer <[email-masked]>
cc: Arnt Gulbrandsen <[email-masked]>, [email-masked], [email-masked]
Subject: Re: a header authentication scheme
Sender: [email-masked]

On Thu, 21 Oct 2004, Laird Breyer wrote:

> But I don't know how SMTP implementors generate ID numbers. If somebody
> can explain this, it would be useful.

An Exim ID is computed from the time of arrival combined with the
process number of the receiving process. The process number will be
different for different SMTP connections. (The detail is messy because
the algorithm was changed, without changing the form of the ID, when
operating systems stopped generating process numbers sequentially.)

Philip

-- 
Philip Hazel            University of Cambridge Computing Service,
[email-masked]      Cambridge, England. Phone: +44 1223 334714.

Date: Thu, 21 Oct 2004 14:33:57 +1000
From: Laird Breyer <[email-masked]>
To: Bart Schaefer <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: Bart Schaefer <[email-masked]>, [email-masked], [email-masked]
Sender: [email-masked]

On Oct 20 2004, Bart Schaefer wrote:
> 
> What's the point of inserting such a Processed: header anywhere before
> final delivery?

Good question. The receiving user/organization may wish to or be forced
to take advantage of external filtering services. 

This can be either commercial, such as http://death2spam.com/, where
the point is that people who use that service do not want to handle
filtering themselves. 

Or it can be a generic bundled service, for example some mailing lists
(e.g. sourceforge) insert filter tags for the convenience of their
users automatically, and there's really no choice or control a
receiving user has about exactly what gets added and exactly how
trustworthy the data is.

> 
> In the event of a message that appears to have traversed A --> M1 -->
> M2 --> B, there's no way for B to validate the Processed: headers that
> seem to have been added by M1 -- because it may be that M1 is really
> the bad guy, and he's just making it look like he's transmitting a
> message from A -- so of what use are they?

Correct. The ultimate decision to trust a segment of the mail path is
bigger than looking at Processed: headers only. 

What the Processed: header can do is assist this ultimate decision in
a way which doesn't yet exist today, namely: it can point back to
where it lies on the path segment (maybe it tells the truth, maybe it lies), 
and there's a simple verification procedure which can be applied. 
Existing headers today don't even offer this assurance, let alone a simple
way to verify it.

The user at B still needs to decide independently whether to trust
M1 or not. For example with the sourceforge mailing list, B could
choose to trust every header which can prove it was added after 
the list received the message, etc.

> 
> Why not just specify that M2 should discard all Processed: headers
> that it receives, just as it would do for Return-Path:, and then you
> can always trust whatever such headers are present, and there's no
> reason to tie them to a token in the Received: line?

That's one way to solve the issue in one case. You're saying M2 should
act like the final arbiter for the user B, removing any information which
M2's configuration considers useless or misleading. 

If M2 wants to take advantage of a service like death2spam, then
the headers added by death2spam cannot be removed, while others should
be. This also assumes that M2's sysadmin trusts the death2spam 
company to remove or label any forgeries which M2 doesn't want to see.

If B wants to take advantage of the sourceforge mailing list 
installation of SpamAssassin, then that may be impossible because
M2 decided to remove those headers when the message arrived there.

Finally, it's actually a good idea to leave forgeries in a message,
because their presence (and recognition) can act as strong clues for
spam. So there might be a spam defense at M2, and whatever gets
through is looked at by some other spam defense at B, and what seems
useless for one may be very valueable for the other, or the software
at M2 is an older generation compared with the software at B, etc.
  
> 
> Or write the spec in terms of trust relationships, such that if M2
> trusts M1 then it can accept all Processed: headers, but otherwise it
> must discard them.  Transitively this means that if the path was A -->
> M0 --> M1 --> M2 --> B then M1 must have thown away all Processed:
> headers if M0 is untrusted, so it's safe for M2 to accept whatever M1
> presents.

Solving the trust problem in general is difficult. What you're proposing
seems nice from the point of view of a single architect who controls the
mail service in an organization, but there are certain practical issues:

How do you communicate to all the software filtering packages in use 
within the organization what paths to trust? What if you have two users
(or software packages) with conflicting trust models? One user wants
to trust M0 as well, but the other user only wants to trust from M1 onwards? 
What happens when there's a reorganization, or somebody installs new software
which plays by different rules (e.g. convert a newsfeed into a mail feed)?

I'm just making these up, but seems easier to leave each software to 
create its trust models however it wants, but give them the tools to 
verify trust whenever needed.

> 
> (Any untrusted MTA anywhere in the chain could mung the existing
> Processed: headers, so there's no point in preserving them across any
> hop that isn't known to implement the spec, is there?)

As I said, the munged headers may in fact be valuable spam indicators,
why remove them?

-- 
Laird Breyer.

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Wed, 20 Oct 2004 23:57:56 -0400
To: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

> We don't want to standardize notions of etiquette (though some mailing
> list admins might try to standardize it within their own lists).
>
> What we want to do is to keep everyone happy by letting the one who 
> thinks
> some behaviour is rude say so in a way that others can respond in the 
> (to
> him) polite manner.

What I want is to get rid of the conflict between two valid concerns. 
One is that senders should be able to specify (both to their MUAs and 
to recipients) who was intended to receive a message without having to 
worry about which recipient addresses include other recipient 
addresses.  The other is that recipients shouldn't have to see 
duplicate messages.

The "rudeness" arguments are a bit specious.  I assume Charles does not 
intend to annoy me by refusing to copy me on replies to my messages.  I 
assume people do not (usually) intend to annoy recipients who get 
duplicate copies by explicitly including them as recipients of replies. 
  What we have now is a system that makes it easy to be annoying, and 
also makes it difficult to reliably avoid being annoying.  I think we 
would do well to reduce the annoyance. I just don't want us to have to 
sacrifice message context or flexibility when we do so.

Keith

p.s. fwiw, the Mac mail client doesn't seem to handle group syntax 
correctly - it doesn't let you delete an address within a group, move 
it to a different header field, etc.  nor does it let you perform 
operations on the entire group.  just a data point.

Date: Thu, 21 Oct 2004 13:05:19 +1000
From: Laird Breyer <[email-masked]>
To: Justin Mason <[email-masked]>
Cc: William Leibzon <[email-masked]>, [email-masked], [email-masked]
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: Justin Mason <[email-masked]>, William Leibzon <[email-masked]>, [email-masked], [email-masked]
Sender: [email-masked]

On Oct 20 2004, Justin Mason wrote:

> As you said previously, the Received header is always prepended.  This can
> be exploited more simply for the Processed header, as long as the
> Processed header is *also* prepended *only*, and never appended, by each
> processor.

Just a quick explanation on why quoting text is perhaps a good idea.

If each piece of software which writes a Processed header always puts
it at the top, then you're right it's enough to look for the next
Received header down to discover where the Processed header was added,
so there's no need to quote ID or datestamp.

The ID or datestamp quoting described is intended to cope with
Processed headers which are either added somewhere else (e.g. many
filters add headers at the end of the list - sometimes it's simply
easier to add at the top, sometimes it's simply easier to add at the
bottom), and also to cope with possible third party rearranging of
headers (because header order is only guaranteed for Received headers
in general).

Finally, if you have a MUA like Outlook or Lotus Notes, the header is
split up into individual database elements, so you simply lose the
ordering. If you write an addon for such a MUA to do spam filtering
say, then you can't rely on physical ordering of the Processed header
directly for trusting. However, you can still reconstruct the ordering
of the Received headers by comparing the BY and FROM fields in such a
case. Then you could reconstruct which Processed headers go with which
Received headers by looking at the quoted ID or timestamps. Not
trivial, but then again Outlook is not a very mail friendly
mailreader.

> 
> That gets around the problem of identifying ID or datestamp strings from
> arbitrary formats of Received headers, which is certainly a hard problem,
> I can tell you.  (we do it for some bizarre reason ;)

identifying date-time: unfold the Received line, normalize spaces,
look for the first ';' from the end and take everything after that.

identifying ID: unfold the Received line, take the first string after
the keyword "id" if it exists (if you want to be more correct, remove
comments (in parentheses), then look for the keywords from, by and
pick the first time you see " id " after that.  Or you can properly
parse the line according to RFC 2822 and try to figure out the id.

;-) In practice, a standard which used this would give a preferred algorithm.

By the way, there's an even easier way to quote a piece of the
Received line for use within a Processed header field:
unfold the Received line, normalize spaces, then compute a hash value
of the whole line. Then if you want to verify whether a Processed
header is associated with a given Received line, do the same thing and
check the hash value.

   Received: from 185129182.virtua.com.br (185129182.virtua.com.br
       [200.185.129.182]) by smtpin-3211.bay.webtv.net (WebTV_Postfix+sws)
       with SMTP id 7280B11DCC; Fri,  5 Mar 2004 19:00:48 -0800 (PST)    

This string could hash to the value B17C07174B5E4546A2B04EB096E83FD7081936B8 say,
and then you would have 

   Processed: name="SpamAssassin"; location-ip="1.2.3.4";
       version="2.63"; function="spamcheck";
       auth-received="B17C07174B5E4546A2B04EB096E83FD7081936B8"; 
       result-tag="spam";
 
The only reason for quoting id or date-time in the proposal is that
it's easier for a human to verify the quote by looking at the header
contents, because if it's a hash value then how do you compute the
hash in your head?

-- 
Laird Breyer.

Date: Wed, 20 Oct 2004 19:15:25 -0700
From: Bart Schaefer <[email-masked]>
Reply-To: Bart Schaefer <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: a header authentication scheme
Sender: [email-masked]

What's the point of inserting such a Processed: header anywhere before
final delivery?

In the event of a message that appears to have traversed A --> M1 -->
M2 --> B, there's no way for B to validate the Processed: headers that
seem to have been added by M1 -- because it may be that M1 is really
the bad guy, and he's just making it look like he's transmitting a
message from A -- so of what use are they?

Why not just specify that M2 should discard all Processed: headers
that it receives, just as it would do for Return-Path:, and then you
can always trust whatever such headers are present, and there's no
reason to tie them to a token in the Received: line?

Or write the spec in terms of trust relationships, such that if M2
trusts M1 then it can accept all Processed: headers, but otherwise it
must discard them.  Transitively this means that if the path was A -->
M0 --> M1 --> M2 --> B then M1 must have thown away all Processed:
headers if M0 is untrusted, so it's safe for M2 to accept whatever M1
presents.

(Any untrusted MTA anywhere in the chain could mung the existing
Processed: headers, so there's no point in preserving them across any
hop that isn't known to implement the spec, is there?)

To: LIST: [email-masked];
Xref: clerew local.mime:3963
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Wed, 20 Oct 2004 22:02:44 GMT
Lines: 48
Sender: [email-masked]

In <[email-masked]> Nathaniel Borenstein <[email-masked]> writes:

>This is one of the silliest threads I've ever read on this mailing list.

>If some people think including a CC is rude, while others think that 
>*not* including a CC is rude, there is simply no right answer.  Burping 
>after a meal is very bad manners in some cultures and very good in 
>others.  We're not going to be able to standardize disputed notions of 
>etiquette.  Thus I would contend that the current reply choices are 
>"good enough" for current needs.

We don't want to standardize notions of etiquette (though some mailing
list admins might try to standardize it within their own lists).

What we want to do is to keep everyone happy by letting the one who thinks
some behaviour is rude say so in a way that others can respond in the (to
him) polite manner. We can see mechanisms that might work, and indeed
those mechanisms seem able to help with a few other situations too, which
is a bonus.

>If you can't find any more important problems to worry about, please 
>let me know and I'll do my best to help you find some.

No, it is clear that some people regard this as an important matter to
fix. It used to be a bone of contention on Usenet, so they invented the
Mail-Copies-To header. Not much used actually, but since it was invented,
people seem to have accepted which way round is rude, and hardly anyone
does the rude thing anymore (so maybe it was the threat that solved the
problem :-) ).

But the disagreement is rampant in Email, as has been amply demonstrated
here; but since solutions seem to be possible, it is worthwhile to pursue
them.

Now if you want a really insoluble problem, then how about a system that
lets some people choose to see a reply with top posting, and others see
the same reply with bottom posting. Now there's a _real_ challenge -;) .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: William Leibzon <[email-masked]>, [email-masked], [email-masked]
Subject: Re: a header authentication scheme 
From: [email-masked] (Justin Mason)
Date: Wed, 20 Oct 2004 18:02:18 -0700
Sender: [email-masked]

Laird Breyer writes:
> On Oct 20 2004, William Leibzon wrote:
> > 
> > I have commented on authentication of received headers before some time 
> > ago on asrg and I think on ietf-smtp (??) or ietf-822, would have to 
> > search archives to find those posts.
> 
> Thanks, I'll try to look them up. Do you know roughly when?
> 
> > 
> > But basicly the only way to truly authenticate this data is to use 
> > cryptography, anything else is herecy and easily defeated and forged.
> > 
> 
> You've obviously thought about this, so it should be fairly easy to provide
> a concrete example. This would be very valuable for the filtering group.
> Can you explain a successful forgery attack on the problem I posted? 
> To fix ideas, here is perhaps the simplest concrete problem:
> 
> A -> M3 -> B. Here A is a spammer which sends mail to B, by talking
> directly to the SMTP server M3, whose domain name is
> mail.bigbucks.com. B's email address is [email-masked], and you can
> assume that B reads mail directly from the mail spool.
> 
> Your task is to write down for me a simple RFC2822 email message which
> contains a forged Processed: header as outlined in my proposal, and
> which can be submitted by SMTP to M3, in such a way that when B
> receives the mail, it is guaranteed that B thinks the Processed:
> header was added by an internal filter on the machine M3.
> 
> You may assume that it is known to A that the filters on machine M3
> are named SpamAssassin, Bogofilter and GreatGooglyMoogly, and you may
> also assume that those filters are stupid enough to allow the message
> through to B in this instance.

BTW, I've been very remiss in commenting, sorry guys!   But just worth
noting at this point that we've implemented a header trust-path algorithm
in SpamAssassin to deal with the similar problem of what Received headers
can be trusted:

- - 1. user or admin can specify a set of CIDR masks in a config parameter
  called "trusted_networks".
- - 2. SpamAssassin will not trust Received headers added by hosts that
  aren't in those netmasks.

As you said previously, the Received header is always prepended.  This can
be exploited more simply for the Processed header, as long as the
Processed header is *also* prepended *only*, and never appended, by each
processor.

That way we can use the Received-header trust path to figure out which
Processed headers are trustworthy; if the nearest Received line *after* a
Processed header was added by an untrusted host, we can't trust the
Processed header either.  OTOH, if the Received header after the Processed
header is trusted, we can trust it.

That gets around the problem of identifying ID or datestamp strings from
arbitrary formats of Received headers, which is certainly a hard problem,
I can tell you.  (we do it for some bizarre reason ;)

- --j.

Date: Thu, 21 Oct 2004 10:49:21 +1000
From: Laird Breyer <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: Arnt Gulbrandsen <[email-masked]>, [email-masked], [email-masked]
Sender: [email-masked]

On Oct 20 2004, Arnt Gulbrandsen wrote:
> The idea sounds good to me.
> 
> Laird Breyer writes:
> > For example, if the topmost Received: line (at M3) is
> >
> >    Received: from 185129182.virtua.com.br (185129182.virtua.com.br
> >        [200.185.129.182]) by smtpin-3211.bay.webtv.net (WebTV_Postfix+sws)
> >        with SMTP id 7280B11DCC; Fri,  5 Mar 2004 19:00:48 -0800 (PST)
> >
> > then a Processed: header of the form
> >
> >    Processed: name="SpamAssassin"; location-ip="1.2.3.4";
> >        version="2.63"; function="spamcheck";
> >        auth-received="7280B11DCC"; result-tag="spam";
> >
> > is guaranteed to have been added *after* the Received: line was 
> > inserted in the message, ie at or after M3. Such a Processed: line is 
> > unforgeable, unless the auth-received value can somehow be predicted 
> > with high probability.
> 
> Actually, the spammer only needs to try harder. If he can predict it 
> with 1% accuracy, he only needs to deliver 150 spams to have 80% chance 
> of passing the test. I've seen spammers try much more than 150 times.

What you say is true in general, but the whole point is to see whether
the guessing game is difficult enough to be impractical, so I'll give some
arguments against your point so we can understand the issue better.

The general tactic for a spammer is to guess the ID which will be 
added to that particular instance of submitted mail. For the protection
to be effective, this guessing game has to be very hard, ie as difficult
as a cryptography guessing game.

Now each mail passing through the smtpin-3211.bay.webtv.net server
will get its own unique ID number when the Received: line is written,
and it won't do for the spammer to guess some other message's ID. So
if the spammer sends 150 mail messages, each one that gets through
will receive a different ID by the receiving SMTP server. 

A typical example where spammers submit 150 or more messages is when
they are fishing for email addresses. They'll start at the top of the 
alphabet and try variations of email addresses. Since these email
addresses don't change over short time frames, once they hit a working mail
address, the spam gets through. 

With the authentication by ID (and similarly by date stamp), a spammer
needs to be able to predict the sequence of IDs used by the SMTP
server at the precise moment when the message reception is being
negotiated, which I believe is a lot more difficult. I would expect
that guessing an ID such as 7280B11DCC correctly at a busy server
would require much more than 150 tries between successes, certainly an
order of magnitude more work than guessing an email address.

And once an ID has been guessed successfully, its value is useless for the 
next message, since that will have a different ID altogether.

But I don't know how SMTP implementors generate ID numbers. If somebody
can explain this, it would be useful.

> 
> But since the receiver can make predicting arbitrarily difficult, I 
> think it's okay. SMTP receivers seem fairly eager make life difficult 
> for spammers. Quelle surprise.

Yes, it's possible to implement variations of the idea, but what's nice
with what I'm presenting is that it slots directly into the existing
RFC 2822 framework, so could be implemented today and everywhere.

-- 
Laird Breyer.

Date: Thu, 21 Oct 2004 10:23:36 +1000
From: Laird Breyer <[email-masked]>
To: William Leibzon <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: William Leibzon <[email-masked]>, [email-masked], [email-masked]
Sender: [email-masked]

On Oct 20 2004, William Leibzon wrote:
> 
> I have commented on authentication of received headers before some time 
> ago on asrg and I think on ietf-smtp (??) or ietf-822, would have to 
> search archives to find those posts.

Thanks, I'll try to look them up. Do you know roughly when?

> 
> But basicly the only way to truly authenticate this data is to use 
> cryptography, anything else is herecy and easily defeated and forged.
> 

You've obviously thought about this, so it should be fairly easy to provide
a concrete example. This would be very valuable for the filtering group.
Can you explain a successful forgery attack on the problem I posted? 
To fix ideas, here is perhaps the simplest concrete problem:

A -> M3 -> B. Here A is a spammer which sends mail to B, by talking
directly to the SMTP server M3, whose domain name is
mail.bigbucks.com. B's email address is [email-masked], and you can
assume that B reads mail directly from the mail spool.

Your task is to write down for me a simple RFC2822 email message which
contains a forged Processed: header as outlined in my proposal, and
which can be submitted by SMTP to M3, in such a way that when B
receives the mail, it is guaranteed that B thinks the Processed:
header was added by an internal filter on the machine M3.

You may assume that it is known to A that the filters on machine M3
are named SpamAssassin, Bogofilter and GreatGooglyMoogly, and you may
also assume that those filters are stupid enough to allow the message
through to B in this instance.

-- 
Laird Breyer.

Date: Wed, 20 Oct 2004 16:59:02 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Understanding response protocols
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

Arnt Gulbrandsen writes:
> http://humane.sourceforge.net/humane_interface/summary_of_thi.html is 
> a rather short summary.

(which leaves out the most relevant bits, actually, such as those about 
I/O value of dialogs. sorry.)

Date: Wed, 20 Oct 2004 16:23:15 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Understanding response protocols
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

Charles Lindsey writes:
> The user just hits the one and only reply button,
...
>  Now the Compose window says:
>    "would you like to reply just to the original author,
>     or to the list,
>     or to what the original author/list-maintainer suggested,
>     or to everybodt mentioned,
>     or fill it all in by hand".
> Well that looks like 5 buttons, but we can probably manage with less 
> (or you can configure it with extra buttons to your choice).

That's not dissimilar to what pine does (if you use the reply command, 
it asks you a question), and what a lot of non-mail programs do. For 
example, if you invoke the quit command of some stupid programs, they 
ask "are you sure? y/n".

It doesn't work.

Users of these programs are adept at hitting the right follow-up key. 
They learn to do unconsciously, as a single multi-key or multi-click 
command. The command to reply becomes "r y", the command to quit 
becomes "control-q enter", etc.

There are ways around this. For example, the five buttons suggested by 
Charles can be located at random, such that the window looks different 
every time and no user can learn what the program will do. A bullet in 
the head is a sure cure for a headache, too.

Those of you who want to make good suggestions that involve user 
behaviour might do well to read a little first. One short and good book 
that's relevant to this thread is Jef Raskin's "The Humane Interface". 
http://humane.sourceforge.net/humane_interface/summary_of_thi.html is a 
rather short summary.

Arnt

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: Understanding response protocols
Date: Wed, 20 Oct 2004 09:27:58 -0400
To: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

> The user just hits the one and only reply button,
> which brings up a compose window and hides all the hurly burly of the
> remaining 99 messages in his Inbox. Now the Compose window says:
>    "would you like to reply just to the original author,
>     or to the list,
>     or to what the original author/list-maintainer suggested,
>     or to everybodt mentioned,
>     or fill it all in by hand".

something like that.  What I currently imagine is that the Compose 
window fills in the recipient list by default with To: {From} and CC: 
{To}, {cc}, but the CC field is "grayed out" meaning they are inactive. 
  The user can click in some obvious place (maybe a check box to the 
left of the field name) to make them active, thus changing "reply to 
author" to "reply to all".  The user could also type in addresses, or 
click any of the addresses to toggle whether they are active.

If there's an MFT or Reply-To field in the subject message, the same 
default set of recipients is used as if the field were not present, but 
a note is displayed (say black on a yellow background) at the top of 
the compose window to the effect that the author requested redirection 
of replies, and a new To:  field is added (grayed out) containing the 
recipients from MFT or Reply-To.  clicking on that To field activates 
the addresses in that field and deactivates the addresses in the 
"default" To or CC field.

To accommodate authors who routinely set MFT to point to "everything 
but From", there would be an option to silently accept MFT done that 
way from particular authors, or from any author.  This would get rid of 
the warning note and pre-activate the alternate To field, and 
de-activate the normal To and CC fields.  But I don't think MFT should 
be silently accepted by default.

>> Note that a similar problem exists for reply body text.  We find it
>> quite useful for an MUA to be able to include the  subject message 
>> text
>> in a reply, but quite tedious to edit out the irrelevant portions.
>
> Sure, so then the Compose window might say:
>    "Would you like the original message included as a quotation,
>     and with what delimiter (e.g. '>'),
>     and would you like it reformatted".
> and so on.

well, that seems pretty clunky and inflexible.  what I currently want 
is for the compose window to include the text of the original message 
in gray, meaning that while I can see it, it is inactive and will not 
appear in the reply when I send it.  I'd have the option of activating 
all of it, or just parts of it, to include that text in the message.  
I'd also have the option of hiding all of it, or parts of it.  but I 
could still insert "new" active text anywhere in the compose window.  I 
think this would make it easy to see the message that I'm replying to 
while I'm replying to it without having to manage a separate window 
(one for the old message, one for the new) and without having to 
explicitly delete the old text.  and I think it would let me respond to 
messages point by point without explicitly quoting so much text.

Date: Wed, 20 Oct 2004 12:45:16 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: a header authentication scheme
Cc: [email-masked], [email-masked]
Sender: [email-masked]

The idea sounds good to me.

Laird Breyer writes:
> For example, if the topmost Received: line (at M3) is
>
>    Received: from 185129182.virtua.com.br (185129182.virtua.com.br
>        [200.185.129.182]) by smtpin-3211.bay.webtv.net (WebTV_Postfix+sws)
>        with SMTP id 7280B11DCC; Fri,  5 Mar 2004 19:00:48 -0800 (PST)
>
> then a Processed: header of the form
>
>    Processed: name="SpamAssassin"; location-ip="1.2.3.4";
>        version="2.63"; function="spamcheck";
>        auth-received="7280B11DCC"; result-tag="spam";
>
> is guaranteed to have been added *after* the Received: line was 
> inserted in the message, ie at or after M3. Such a Processed: line is 
> unforgeable, unless the auth-received value can somehow be predicted 
> with high probability.

Actually, the spammer only needs to try harder. If he can predict it 
with 1% accuracy, he only needs to deliver 150 spams to have 80% chance 
of passing the test. I've seen spammers try much more than 150 times.

But since the receiver can make predicting arbitrarily difficult, I 
think it's okay. SMTP receivers seem fairly eager make life difficult 
for spammers. Quelle surprise.

Arnt

Date: Wed, 20 Oct 2004 20:05:47 +1000
From: Laird Breyer <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: a header authentication scheme
Reply-To: [email-masked]
Mail-Followup-To: [email-masked], [email-masked]
Sender: [email-masked]

I'd like to solicit some comments or opinions on part of an idea
which is being developed on a different mailing list, namely
the ASRG spam filtering list

http://asrg.sp.am/subgroups/filtering.shtml

I'm CC'ing the filtering list, but you may need to be subscribed to
post there, so you might elect only to reply here.

I'll give a few paragraphs of introduction, and a synopsis of the 
idea. Your input will hopefully be helpful to refine and improve the
work there. 

Introduction:

The ASRG spam filtering subgroup is trying to come up with ideas for
eventual standards related to the filtering performed by spam filter
software on RFC 2822 messages. With the plethora of spam filters and
virus filters and so on, there's an explosion in the number of
proprietary headers being added to a message during transport to the
MUA (ie not necessarily just SMTP transport, but for example during
POP3 download or procmail processing).

It seems like a good idea to look for ways to standardize such
headers, so that other software and humans can inspect the headers
added by filtering software, not just the originating software.

There's currently a filtering draft being edited (it's not an official
IETF draft, more a working document), in which the proposal
is to define an extensible header which in one particular instance
might look like this:

   Processed: name="SpamAssassin"; version="2.63"; function="spamcheck";
       auth-received="7280B11DCC"; result-tag="spam";

The idea is to define key value pairs, some of which would have
universal meanings while others would be added as required by the
software writing the header. The reason for choosing those MIME-style
key value pairs is that it should make it a fairly uniform task for
software to parse the header, regardless of the software writing
it. Unrecognized keys would be ignored, while recognised keys may or
may not be acted upon by downstream software which would receive or
pass along the message.

The problem:

Now that you have a little background, I want to describe the idea I'm
particularly interested in. When you see a header such as the one I've
sketched above in a message, it's not clear where in the message
transport this header was added. Suppose A sends a message to B, and
the message passes through several machines M1, M2, M3 say:

A -> M1 -> M2 -> M3 -> B

For argument's sake, we'll assume A is the sender's MUA, B is the
receiver's MUA, and little is known about the other machines. The
software which adds the Processed: header shown above may reside in A,
M1, M2 or M3.  The task is to facilitate B's discovery of where the
header was added, through a standard.

For example, if A is a spammer, then he might add the Processed: header
at A, expecting to fool B into accepting the verdict at face value.
That would be a forgery which B wants to recognize.

This is a fairly realistic situation, since spammers do forge headers 
routinely, and if the message's path before arrival at B is sufficiently
complex, then B may not have a full picture of which filters are
installed at which locations, and how each such filter behaves. So the
location hint must be self-explanatory.

Of course, simply inserting the filter's location (e.g. IP address) 
among the key-value pairs in the Processed: header is not reliable, e.g.

   Processed: name="SpamAssassin"; location-ip="1.2.3.4"; 
       version="2.63"; function="spamcheck";
       auth-received="7280B11DCC"; result-tag="spam";

does not help solve the problem at all, since a forgery originating
at A could simply forge the correct IP address. 

A particularly important subproblem is the following: how can B recognize
if the Processed: header was added at M3 versus added earlier?

The (partial) solution:

I shall now propose a partial solution for discussion, which was
worked out on the ASRG filtering list: recall that a fundamental fact
of SMTP transport is that Received: headers are added at the top and
must not be rearranged.

A consequence of this is that, once the message is at B, the top most
Received: line is guaranteed to have been added by M3 (assuming M3 is
an SMTP server). This means that a subsequent filter (at or after M3)
will be handed a message containing the topmost (for B) Received:
line, and such a filter can prove its location beyond M3, by quoting a
substring from that topmost Received: line.

The question is which piece of the Received: line should be quoted?
It would have to be a piece which cannot be easily guessed by forgers
located before M3 (ie at A, M1 or M2). For example, the server M3's 
domain name would be  apoor choice, since that name could be discovered
through various means by the spammer, and once known, quoted many times.

There are two pieces of information which change constantly in a 
Received: line, these are the ID (which is optional) and the date-time stamp.
So one of these two must be quoted in each Processed: header line.

For example, if the topmost Received: line (at M3) is

   Received: from 185129182.virtua.com.br (185129182.virtua.com.br
       [200.185.129.182]) by smtpin-3211.bay.webtv.net (WebTV_Postfix+sws)
       with SMTP id 7280B11DCC; Fri,  5 Mar 2004 19:00:48 -0800 (PST)
 
then a Processed: header of the form
 
   Processed: name="SpamAssassin"; location-ip="1.2.3.4"; 
       version="2.63"; function="spamcheck";
       auth-received="7280B11DCC"; result-tag="spam";

is guaranteed to have been added *after* the Received: line was inserted
in the message, ie at or after M3. Such a Processed: line is unforgeable,
unless the auth-received value can somehow be predicted with high probability.

Similarly, a Processed: header of the form

   Processed: name="SpamAssassin"; location-ip="1.2.3.4"; 
       version="2.63"; function="spamcheck";
       auth-received="Fri,  5 Mar 2004 19:00:48 -0800 (PST)"; 
       result-tag="spam";

is unforgeable unless the timestamp inserted at M3 can be predicted
at M2 or earlier. 

I am a little worried that the time stamp is a little easier to
predict from M2, as it only lists seconds, and a second can be a long
time in an SMTP transaction. If the Received: timestamp included also
microseconds (which would require an extension of RFC 2821), this would be
less risky. I also don't know how easy the optional ID would be to predict
in practice, but I expect quite hard.

I'd like comments and opinions, particularly (but not exclusively) from
people who have implemented SMTP servers, to see whether this idea
is robust, or if not whether it is salvageable. I can't see much that's
wrong with it.
 
-- 
Laird Breyer.

From: "Hector Santos" <[email-masked]>
To: "Keith Moore" <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: Understanding response protocols
Date: Wed, 20 Oct 2004 00:24:50 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Keith Moore" <[email-masked]>

> > But what is your solution?

> I'm still trying to understand what the problem is.

Too bad. But I figured as much.

> I don't respect "commercial" implementors more than "noncommercial"
implementors.
>
> Perhaps we are closed-minded.  Perhaps you are.  Perhaps we all are.

You continue to show your true colors, odd and evil nature and character.
Its too bad the IETF needs to get a bad rap because of people like you.

Rest assured, you not going to stop me from hanging around, providing input
when I feel necessary regardless what you don't understand nor recognize.

Ciao

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: LIST: [email-masked];
Xref: clerew local.mime:3954
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Tue, 19 Oct 2004 20:18:28 GMT
Lines: 93
Sender: [email-masked]

In <[email-masked]> Nathaniel Borenstein <[email-masked]> writes:

>However, I question whether MFT might not cause as many problems as it 
>solves.  Consider:

>	To: [email-masked]
>	CC:  [email-masked], [email-masked], [email-masked]

>This is something I have done in the past in order to bring into a list 
>conversation a few people who are not on the mailing list.  Now, what 
>should the list processor at example.com do with this?  As I understand 
>it (please correct me if I'm wrong) current implementations would 
>redistribute the message like this:

Well, the strict answer is anything it likes :-) .

But for a sensible thing for it to do (and let us assume it was a
discussion list like this one, and not an announce-list or a for-sale-list
which might be configured entirely differently), how about:

It observes that no MFT is present. So it adds
    Mail-Followup-To: [email-masked]
(that being standard list policy).

If it was a bit smarter than that, it might notice that Bill, Bob and Jim
were not known subscribers, so it might add them to the MFT. But some
people might regard that as being "too smart".

>	Mail-Followup-To: [email-masked]
>	To: [email-masked]
>	CC:  [email-masked], [email-masked], [email-masked]

That is how I would expect you to have your MUA configured for normal
postings to the list (unless you were happy to accept whatever the list
maintainer did - see above). Your MUA, your choice.

>Thus people whose UA's respected MFT would end up omitting Bill, Bob, 
>and Jim from their replies, which was not my expectation as the sender.

Indeed, but this is not your normal way of posting to the list. You want
Bill, Bob and Jim to see this message, so you went to extra trouble to
include them in Cc, and you want them to see your replies, so you have to
take a bit more trouble by hand-crafting your MFT:

	Mail-Followup-To: [email-masked],[email-masked], [email-masked], [email-masked]
	To: [email-masked]
	CC:  [email-masked], [email-masked], [email-masked]

and here the list-maintainer would note your MFT, and not override it with
his own.
 
>  Is this really the desired behavior?  Or should all the non-list 
>addresses except the sender in fact be copied into the followup header? 

It's your choice. Sometimes you might want to Cc your message to the list
and NOT have replies sent to the Cc address (e.g. the Cc was just to send
your message to your local message archive, or to your boss, to show him
that you had been busy).

Again, if your MUA was smart enough, and you often wanted to bring
non-list-members in on the discussions, you might configure it to always
include the Cc people in the MFT (but then, if one day you DID include
your boss in the Cc, you would have to manually remove him from the MFT).
It's your choice.

>  One could argue that the MFT should include everyone that would 
>normally be in "Reply to All" *except* those who are on the list -- 
>after all, if the list processor is doing the rewriting, it's in a 
>position to do that, right?

Yes, that is one of the possibilities, and has already been suggested as
one of the things a smart list-expander could do.

Essentially, MFT opens up many possibilities, both for the original author
and for the mail list expander.

And then, there is the separate issue of what the replying MUA is supposed
to do when it sees an MFT, and with that is associated the question of how
many buttons it needs to give the replier sufficient control. But
hopefully, it will somehow make it easy for the replier to just reply to
the MFT, since that is what it is expected most repliers will do most of the
time.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3955
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Tue, 19 Oct 2004 20:29:29 GMT
Lines: 56
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>Especially these days, in the browse environment the user is often in
>"whack-a-mole" mode - he's got various kinds of stuff in his inbox,
>including maybe important personal messages, trivial personal messages,
>important "work" messages, trivial "work" messages, list traffic, spam,
>and viruses -all jumbled together.  His main goal is to keep the list of
>things that haven't been looked at from getting too long.  Some things
>get deleted unseen, some get looked at and then deleted, some get
>forwarded, some may get refliled if the user is sophisticated enough. 
>Basically the user is performing triage.

>Even though a lot of messages do get terse replies, the "compose" screen
>at least gives the user a chance to shift gears and focus on the message
>he's composing.    That's why I think that the  selection of reply
>recipients needs to be part of the compose  environment.  To be
>effective the compose environment needs to make it easy to reference,
>quote, and or incorporate portions of the subject message - and this
>includes making it easy to reference, quote, and/or incorporate portions
>of the recipient list from the subject message into the recipient list
>of the reply.

OK, that's a fair point. The user just hits the one and only reply button,
which brings up a compose window and hides all the hurly burly of the
remaining 99 messages in his Inbox. Now the Compose window says:
   "would you like to reply just to the original author,
    or to the list,
    or to what the original author/list-maintainer suggested,
    or to everybodt mentioned,
    or fill it all in by hand".
Well that looks like 5 buttons, but we can probably manage with less (or
you can configure it with extra buttons to your choice). But, in any case,
we have removed the clutter of the Inbox, so there is less problem with
having several options in view (and they don't have to be buttons - other
interesting ways have been mentioned in these threads).

>Note that a similar problem exists for reply body text.  We find it
>quite useful for an MUA to be able to include the  subject message text
>in a reply, but quite tedious to edit out the irrelevant portions.

Sure, so then the Compose window might say:
   "Would you like the original message included as a quotation,
    and with what delimiter (e.g. '>'),
    and would you like it reformatted".
and so on.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Cc: "ietf-822" <[email-masked]>
From: Nathaniel Borenstein <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Tue, 19 Oct 2004 16:55:42 -0400
To: "Hector Santos" <[email-masked]>
Sender: [email-masked]

This is one of the silliest threads I've ever read on this mailing list.

If some people think including a CC is rude, while others think that 
*not* including a CC is rude, there is simply no right answer.  Burping 
after a meal is very bad manners in some cultures and very good in 
others.  We're not going to be able to standardize disputed notions of 
etiquette.  Thus I would contend that the current reply choices are 
"good enough" for current needs.

If you can't find any more important problems to worry about, please 
let me know and I'll do my best to help you find some.  -- Nathaniel

Date: Tue, 19 Oct 2004 16:23:43 -0400
From: Keith Moore <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

Hector,

I do not recognize your expertise.  In part this is because I've been
working on email systems since 1981 or so and with Internet email since
1985 or so, and with IETF since 1990 or so, and I had never heard of you 
until recently.  I generally do respect implementors, but I have never 
heard of the implementations you profess to have worked on, so I have no
way to gauge their quality or significance.  I don't respect "commercial"
implementors more than "noncommercial" implementors.  A lot of commercial
software is garbage, a lot of noncommercial software is high quality.

We do not share the same goals for email.  That which you find 
inherently desirable, I find dubious at best.

I believe your proposals will do harm and work poorly, and have been 
demonstrated to do harm and work poorly on numerous occasions.

You have repeatedly failed to give consideration to points that others
have raised, and you have repeatedly failed to make points to support
your position other than "it works for me and my customers".  That's 
fine but it might mean that your case is weak.

It appears that we have little (if any) common ground on which to base
a discussion.

Perhaps we are closed-minded.  Perhaps you are.  Perhaps we all are.
But continuing to repeat ourselves serves no useful purpose.

From: "Hector Santos" <[email-masked]>
To: "ietf-822" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Tue, 19 Oct 2004 14:17:11 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Bruce Lilly" <[email-masked]>

> > or give any guidance about what to do if the
> > replier wants to change the default.
>
> Again, the concept of a "default" is an rMUA issue, not a
> characteristic of a Reply-To field.

I disagree.  From a technical standpoint, this is a sound definition of
a default reply address.

Again, we need to take into again the the 1:1 vs 1:many concept we
are dealing with here.  The MUA is not "designed" to handle this concept
of how the original intention of the message list "holder" is distributing
the mail.

This is quite evident when you shift the thinking to a MUA who understands
groupware conferences and/or an ONLINE based MUA, like a Newsgroup Reader
where the default is a "reply to group" and the "reply to sender" is a
secondary
option.

This is clearly becoming a "close mindset" think - open it up bruce.

> > Because I'm looking at how to make email more effective, one of the
> > things I'm concerned about is the rarity of conscious choice when
> > replying to emails.  So I'm wondering if there are ways that MUAs can
> > be improved to encourage conscious choices without being too annoying.
>
> One rather obvious way would be to present multiple choices
> equally (and correctly labeled) rather than a default (with or
> without other alternatives which are less accessible).

Ok, but why?  Most MUA designers would prefer to a have a "concise"
offering to the user for that the intentions of the backend is.  If the user
wants to go offline, those are secondary options.

Options are always to have, but when it comes to a default, the user should
be presented with a clearly understanding of how to response to a list.

So the issue seems to be coming down to two things:

- For legacy systems, how do you best offer a default for a mailing list?

- For new systems, what is needed to offer all the possible response options
that includes considerations for a mailing list?

You can't saying the backend has nothing to do with it. I disagree.

It is very much part of the picture.

The proof?

I already showed how.  A list server can fix the reply-to address to the
list address once it takes responsibility for the new distribution as a
"List User" entity.

The issue with this?

For the legacy MUA, the secondary option to "reply direct" is no longer an
easy option.

---
Hector Santos, CTO
WINSERVER "Wildcat! Interactive Net Server"
support: http://www.winserver.com
sales: http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: "ietf-822" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Tue, 19 Oct 2004 13:47:05 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

From: "Bruce Lilly" <[email-masked]>

> You seem to be unable to accept the fact that several people disagree
> with your opinions for valid reasons (which have been patiently
> explained to you).  That is your problem, and you'll have to learn
> to deal with it.  Telling others to shut up is inflammatory,
> counter-productive, and unprofessional; that creates problems
> for mailing lists like this one.  I suggest that you take a more
> professional approach.  You might also want to read RFC 3683.
>

Bruce,  from an outsider standpoint, it is people like yourself, the Keith
Moores, etc, it is very evident to see how your antigonistic input ruins the
process of communications.  You treat input as it is all nonsense opinions.
People need to go to extremes to get anything said.

I clearly stated my view of the problem and I stated a solution.  You have
the right to disagree, which is fine.  But to tell me I am wrong, well, that
is  where it "create" problems because quite frankly, my credentials as a
software developer and product producer in this market is very much evident
over yourself.   Do you think I like to say this? No. But you give no one
the benefit of the doubt.   I still don't understand what make you the sage
of all that is being debate. Why does it seem it has to pass the "Bruce or
Keith" test?

The facts:

In regards to mail only, only 1 part:

o 20+ year of Mail Hosting/Communication Systems.
o One of the original developers for Offline MUA systems.
o One of the original Groupware Systems.
o Multi-Access/device Support (GUI, POP3, Telnet, Dialup, WebMail)
o Some specifics features:

- External and Local Message Conferencing
- SMTP, UUCP mail support
- Fidonet Support
- UTI, QWK, OPX and BlueWave support
- Windows Exchange Messaging Support
- List Server Support
- News Server Support
- Advance Scripting Systems for RBM (Rule Based Messaging)
- Event Signaling for all Mail Actions
- IM (Instant Messaging) Completely tied into system

etc, etc, etc.

Please note,  I am not talking about those MUA designed to understand
groupware systems, such as a Newsgroup reader, IMAP or have specific
proprietary groupware concepts such as MS Exchange or Lotus Notes.   Small
note: Outlook Express and Outlook are two different MUAs. OE is your common
MUA. Outlook supports conferencing with exchange interfacing.  Our Wildcat!
Exchange product add-on offers our backend support for Outlook for example.

So when I provide input, I like to think I can help to provide a perspective
and insight that is based on a multi-million company long term design and
product engineering. It is not just an whimsy opinion.   Now, if you can
tell me I am wrong, well, you better be ready to show me how with sound
reasons. I am not stubborn to understand where thing can be improved and
trust me, I respect good technical people.

But I am not here to fight with you. I purposely introduced myself into the
thread with due care because there is a history here (IETF wg in general)
that things get out of hand.  Who was the one to begin attacking?  You.  Not
me  Follow the thread.   Just like you were argueing with everyone else, I
was sort of waiting to see when you will begin to jump on me.  Keith held
his ground until he could no longer!  What was his single input?  "You are
wrong. Fix your program!"   This is all crazy!  You both failed to read
anything and picked off specific items with moronic responses that had
nothing to do with anything.

Case in point? The hilarious "MUAs are not trained!"   Now I am made to feel
like I am immigrant, a terrorist of the english language!   It was so
hilarious I gave up talking to you!  But you had to continue attacking?  Its
in your blood.

Well, lets talk about "Training."

Frankly, when you have the software engineering background I do,  we do have
such "out of the box" understanding of things.  Software is trained in the
sense they are moded and learn to behave in a common way understood.   While
they are some good specs to work with,  you still need to understand how the
ergonomics of the system is suppose to work.  There is no specs on the
ergonomics.  This usually means, they learn to see how others behave as
well.

In any case, that is all stupid.  The point was clear:

The RFC based OFFLINE MUA does not understand mailing list concepts for two
reasons:

- They are NOT trained to understand the concept,
- The BACKEND does not have a standard to help them better work in this
area.

This is a professional design opinion which at a snap of a finger, we can
help address for our own system - the backend and our MUAs.  In most cases,
it isn't issue.  Here is why:

When a conference is created, the admin has the option to

- Create it as a List Server storage conference
- Expose it as a Local Online Reading conference
- Expose it as a Local Newsgroup Forum

This means that all devices may have access to the conference.

- Telnet/Dialup Console (Dump Terminals) user access
- Our free GUI user access
- Web Mail user access
- Newsgroup Reader user acess
- POP3 user access.

POP3 users will get their mail personalized from the expanded copies that
remain within the domain. They don't access the list storage conferences but
their own personal inbox.

Never mind the complexities of supporting multi-devices when it somes to the
advent of MIME mail, asyou can see, there is a  bigger picture that is
considered here.  The natural reply for each device must be done in a way
that is more obvious by default.

But it should go without saying the the Offline POP3 access point has done
more in the area of helping to create conflicts in system mail policies.
This debate is just one of them.  I can provide you with more insight where
there needs to be more standardization for the offline EMAIL reader.

I will give you two of them:

- Receive Date:
- Honoring Server Mail Policy - the "Snoop/Preview Problem"

The above are problem the two top issues we had to deal with, more so than
this "natural reply" issue for mailing list messages.

- Receive Date:

This was done a long time ago so I can't recall off hand which MUAs exhibit
the same behavior.  Outlook was one of them, but it also shows up in others
as well.

You see, an MUA like Outlook will have two date column by which they can
sort by:

    Message Date:  The 822.Date Header
    Received Date:  The 822.Received Header

The question is, what is the "actual receive date" for a message picked up
by an offline reader?

Well, if you use the Received: Date, this will be the timestamp arrived at
the MDA.  Ergonomically, this date could be older than the user expected.

It could depend on the user's POP3 frequency.  If you pop every minute, then
you are more likely to get a time of arrival that is closer to the user's PC
time.

But regardless,  if you are looking at the new mail listing on your MUA and
you see 1-2 new messages that were arrived "2-3 hours" ago, well, it makes
some user's wonder as it did for us when it was reported as to why they are
having a hard time sorting the columns.

You see, you really can't sort by the 821.Date which they are is mixed of
messages coming from all over the place and there is the small change that
there some errors in some 821.dates.  So the ideal sort column is the
receive date.

Our solution was to proivde an sysop option for the POP3 server to add a new
"Received:" header that represents the actual PICK UP time.   It is off by
default.

- Honoring Server Mail Policy - the "Snoop/Preview Problem"

This one is probably the #1 conflict that one can see when dealing with
offline mail systems.

You really need to have an understanding of long established mail server
frameworks to understand the issues this introduced.   Historistically, only
admins had "god powers" to snoop on mail, to preview mail without marking
the message as received/read.  User's simply didn't have such power and for
good reasons. Namely the following:

    - Subscription based controlled systems,
    - System Notifications,
    - Request Receipt Notificatons,

and other related ideas that dealt with tracking and control.

For online systems, it was all under the control of the server side.
However, with the advent of the offline mail reader, namely POP3, it
provided a new requirement to "preview mail" without marking the messages as
already received.

A good example why this because a requirement is for roaming users, who may
be on vacation or off-site to read private mail.  But when they get back
home or work, they want to once again pick up the same mail.

To introduce a new violation of mail security and integrity for the backend.
Now it was possible for users to preview mail without any tracking or
recording. Now it was a possible for a system to send an "expiration notice"
and the user claiming that he never saw it.  "Hey, why did you lock me out?"

To make a long story short, here, the final analysis required a revamping of
new server side features that allowed for both offline and online mail
previewing while still keeping a track of the system. So when you go to the
web mail client, you have two buttons, one with "binoculars" to preview the
message, the other to do a normal read.

Thats it for now. I will say one final thing.  I have no admonisity towards
anyone. I don't know you, but I admire Keith Moore and I wish we can be more
friends than exhibit cyber agreesiveness.  Maybe I am naive, but I would
like to see the IETF become a more productive in this work, especially in
the new era where change is in the "air."  We are faced with a new
opportunity for change once again.  Yet, we see a vast conflictive
convoluted, disjointed efforts . Much of it is clearly based on stubborn -
the old dogs don't want to change.  You know what? I don't either, but I
understand and recognized when you need to progress with some changes.  This
is why I began to participate in the IETF last year.  In anycase,  I suggest
you should learn listen to what people are saying and proposing. Give them
the benefit of the doubt because quite frankly they might be seeing an angle
of it you simply don't see. You might be set in your ways that it creates
unreasonable lack of consensus.  This is what you need to learn.

See ya

--
Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com

Date: Tue, 19 Oct 2004 16:24:04 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Cc: Bruce Lilly <[email-masked]>, Charles Lindsey <[email-masked]>
Sender: [email-masked]

>>  >There is no general mechanism to identify a list,
>>
>>  Then invent one!
>
> Perhaps you should consider the possibility that that is an impossible task.

We have two RFCs that do the task. Support for them is not wide, but I 
can't see any reason why Charles' idea would be wider supported.

Arnt

From: Bruce Lilly <[email-masked]>
Reply-To: The ietf-822 mailing list <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Tue, 19 Oct 2004 09:53:38 -0400
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Mon October 18 2004 17:09, Keith Moore wrote:

> I think there are some problems with the very notion of a
> sender-settable _default_ recipient list - one that encourages the
> reader to use that list for replies without thinking about it.   This is
> equally a problem for MFT or Reply-To, and probably for other ways of
> setting that default.  And while it's certainly possible to build MUAs
> that don't  treat MFT or Reply-To or whatever as a "default" but instead
> do something else with it - let the user choose it explicitly, insist
> that the user make an explicit choice, whatever

Reply-To is an author-settable suggestion for responses. Whether
or not an rMUA has a default type of response, and if so whether
or not that uses that field are separate issues. There is nothing in
the Reply-To specification that requires or encourages a default
type of response in rMUAs; in fact RFC 2822 is quite explicit in
stating that behavior of reply commands is implementation-
specific and outside of the scope of the message format
specification.

> I wonder if it will 
> violate the sender's expectation if recipients consciously choose to
> reply to somewhere else.

Given the defined semantics, the author's expectation for his
use of Reply-To should be that he has provided a suggestion;
no more.
 
> Both MFT and Reply-To seem to expect repliers to understand, via
> out-of-band information, the  relationships between the specified 
> recipients and the other recipients of a message
[...]
> They don't say _why_ the replies are being
> redirected that way, 

The (Reply-To) field simply provides a mechanism for authors
to provide suggestions for responses. There are in-band
mechanisms (parenthesized comments, display names, "Comments"
message header field, body text) that are also provided for
authors to indicate such additional information to (human)
recipients.

> or give any guidance about what to do if the 
> replier wants to change the default.

Again, the concept of a "default" is an rMUA issue, not a
characteristic of a Reply-To field.  Neither an author nor
his oMUA can tell what defaults might be in effect at any
single recipient's rMUA; when multiple recipients are
involved -- explicitly, via list expansion, or via forwarding
or resending -- those multiple rMUA defaults, where such
rMUAs have defaults, may well differ.

> Because I'm looking at how to make email more effective, one of the
> things I'm concerned about is the rarity of conscious choice when
> replying to emails.  So I'm wondering if there are ways that MUAs can 
> be improved to encourage conscious choices without being too annoying.

One rather obvious way would be to present multiple choices
equally (and correctly labeled) rather than a default (with or
without other alternatives which are less accessible). I notice
that you use Sylpheed, and it provides a way for you to
configure the toolbar (Configuration -> Other Preferences ->
Customize Toolbars -> Main Window). The default configuration,
at least in the version "0.9.1claws" that I have here, seems to
be to provide icons for three types of responses: "Reply" (which
uses Reply-To if present), "All" for a reply-to-all function, and
"Sender" (which is actually a reply-to-author, ignoring Sender
and Return-Path).  Many other GUI-based rMUAs have similar
capabilities to customize toolbars (and menus).

> And because I want to encourage conscious choices if it is possible to
> do so (and I'm not sure that it is), I'm wary of new header fields which
> seem to assume that users will not make conscious choices.  I think such
>  fields could even discourage conscious choice, by creating a wider
> expectation that the recipient will do what the sender said (thus 
> negative feedback if the recipient does something different even when he
> had a good reason); and because, lacking information as to _why_ the
> sender set MFT, the recipient might decide (again, without much
> justification) that the sender's judgement is better than his own.

All good points. 

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Tue, 19 Oct 2004 09:45:54 -0400
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

On Mon October 18 2004 09:56, Charles Lindsey wrote:

> Ergo there is no necessity for <id-right>s to be compared
> case-insensitively

No, in the first place it is recommended to be a domain name,
which is always case-insensitive. In addition, it has always been
a domain name (and remains so under the current full Standard).
Moreover, there are various circumstances (explained at length)
where case of the RHS may be changed, and only a case-
insensitive comparison will work correctly under those (not
unusual) circumstances.

> and the change introduced by RFC 2822 is a welcome 
> simplification of the protocol.

It is not a simplification; it complicates matters since it is
no longer possible to determine whether or not the RHS is
a domain name, and if it is not, there is no indication of
whether or not it is case-sensitive.

RFC 2822 certainly does not state, imply, or suggest that
the RHS should be treated as case-sensitive. So there is
nothing in that RFC to support your unconventional opinion
that case-sensitive comparisons should be used for identifier
domains. Indeed, it defines id-right as dot-atom-text
(dot-atoms are used in case-insensitive domain names),
domain literals (IPv4 literals are numeric, so case-sensitivity
is not an issue; however IPv6 literals may have hexadecimal
digits, which are case-insensitive), or as traditional RHS 
domain names (case-insensitive as always) or domain
literals with CFWS).  The sole difference between the
dot-atom construct used in a domain and the dot-atom-text
specified for id-right is the optional presence of CFWS in the
former, and in fact RFC 2822 explicitly states that that
difference in CFWS is the difference between its definitions
of msg-id and angle-addr.  Absence of CFWS does not imply
case-sensitivity.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Originator field interaction
Date: Tue, 19 Oct 2004 02:02:51 -0400
Sender: [email-masked]

Some statements have been made about use of Reply-To; that
its use to direct responses to a mailing lists precludes indicating
a preferred mailbox for direct responses to the author(s), that
it provides no mechanism for indicating why a suggestion is
being made, that it provides no mechanism for an author to
offer guidance to a potential respondent who wishes to direct
his responses elsewhere. I will address those issues in this
message.  There are several reasons why an author might
suggest a place for responses; RFC 822 lists three, and I'll
use those three:

       Three typical
        uses for this feature can  be  distinguished.   In  the  first
        case,  the  author(s) may not have regular machine-based mail-
        boxes and therefore wish(es) to indicate an alternate  machine
        address.   In  the  second case, an author may wish additional
        persons to be made aware of, or responsible for,  replies.   A
        somewhat  different  use  may be of some help to "text message
        teleconferencing" groups equipped with automatic  distribution
        services:   include the address of that service in the "Reply-
        To" field of all messages  submitted  to  the  teleconference;
        then  participants  can  "reply"  to conference submissions to
        guarantee the correct distribution of any submission of  their
        own.

For brevity, I'll refer to those as "alternative", "delegation", and
"list", and when discussing combinations, the individual cases
will be numbered based on bitwise combinations of those three
assigning binary power values 4, 2, and 1 respectively.

The claim that use for lists precludes use for alternatives presumes
that combinations of uses are not possible. That is not necessarily
true for all combinations; moreover it ignores other mechanisms
available.  Given the three uses mentioned above, there are a
total of 2 raised to power 3, or 8 possible combinations:

0: some use other than the three listed; i.e. none of those three

1, 2, 4: these are single use cases. 1 and 2 are straightforward,
   but case 4 warrants some additional discussion.  The case
   where "the author(s) may not have regular [...] mailboxes"
   begs the question of what appears in the mandatory From
   field, which is supposed to list the author's (or authors')
   mailbox(es).  The example in RFC 822 A.2.6 uses the Sender's
   mailbox with a different display name.  That provides an obvious
   way for an author to provide a recommended mailbox for
   individual responses as well as providing a suggestion for
   other responses; viz. the mailbox for individual responses can
   simply be used in the From field (with a suitable display name
   and/or parenthesized comments).

5, 6: these are handled as just described; the issue of an
   individual mailbox is handled via the From field, the remaining
   use is addressed via Reply-To.

3, 7: combine delegation and list. One implication may be that
   the handling of responses is being delegated to a list (a list
   expander, a named group, or simply a list of mailboxes).
   Are there any circumstances where one might wish to delegate
   individual responsibility yet suggest responses to a list?
   Perhaps (case 3), but probably not in combination also with
   an alternative individual mailbox for the originator. Even these
   cases can be handled using the available originator fields:

Case 3:
   From: (on behalf of, and delegating authority to) [email-masked]
   Sender: (authored by) [email-masked]
   Reply-To: The Committee: [email-masked], Dick at example.com,
     [email-masked];

Case 7:
   From: (on behalf of, and delegating authority to) [email-masked]
   Sender: (authored by [email-masked] while on vacation) [email-masked]
   Reply-To: The Committee: [email-masked], Dick at example.com,
     [email-masked];
  Comments: If you value your job, don't bug me; general discussion
     among the committee, call on J. Doe if necessary

The use of comments, display names (in the named group), and
Comments message header fields demonstrate how an author can
provide indication of why a suggestion has been made for
responses, and to give some guidance to prospective respondents
who might consider deviating from that suggestion. Body text
provides yet another mechanism.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Mon, 18 Oct 2004 23:42:45 -0400
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

On Mon October 18 2004 10:07, Charles Lindsey wrote:

> So a Dual-Use User Agent needs some better way to determine whether the
> message in hand is an email message or a news article. The proper way to
> do this is to keep track of where it got it from.

A couple of typical cases are:
a) it "got it from" a flat mbox-like file or a single text message file
b) it "got it from" an IMAP "folder"
In neither case is there any information to indicate the distinction
between an "email" message vs. a "news" message.  In either case,
the message may have passed through one or more gateways
between news transport and mail transport.  Another somewhat
more unusual case is:
c) it "got it from" an NNTP server after having passed through a
    mail-to-news gateway
I believe that case might apply for some people who read *this*
mailing list; N.B. this is a mailing list, not a newsgroup, and if one
insists on making a distinction, the messages sent by this list's
expander are unquestionably mail.

As Ned Freed has explained to you, if there is a legitimate need to
be able to make a distinction, there is a standardized mechanism
available via RFC 3458.  However I doubt that there is such a
need in the first place, and I doubt that any mechanism (including
RFC 3458) will work reliably when messages pass through gateways,
are copied to files, moved and/or copied between folders, etc.

> If it arrived by NNTP 
> from a news-server, then it was a news article; if it arrived via SMTP or 
> > from a message store which obtained it from SMTP, then it is an email 
> message.

And if it was delivered via SMTP to a mail-to-news gateway, then
transported via NNTP?  Or transported via NNTP to a news-to-mail
gateway, then delivered via SMTP?  What if it was delivered via
UUCP? Or accessed via IMAP?

> If an Agent is setting out to be for "dual-use" between two 
> media, then it behoves it to know which medium it is handling at any given
> moment.

It's handling "messages".  When a message is being transported, it
might be transported via a news-specific protocol, or via a mail-specific
protocol, or via a protocol which is non-specific.  Once it has been
transported, it is simply a message.
 
> And if it absolutely MUST guess which medium it is from examination of the
> message, then a better discriminator would be the presence or absence of a
> Return-Path header, of a "^From " line.

Return-Path fields are only inserted by SMTP when making final delivery;
they do not exist for mail delivered via UUCP (for example). "^From " lines
are an artifact of a particular storage format, not of the means of
transportation that a message may have been subjected to.  Newsreaders
(e.g. trn) save "news" messages in that format.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: Understanding response protocols
Date: Mon, 18 Oct 2004 23:03:45 -0400
Cc: "Charles Lindsey" <[email-masked]>, LIST: [email-masked];
Sender: [email-masked]

On Mon October 18 2004 09:49, Charles Lindsey wrote:
> Newsreaders do not, in general, put up
> 'alerts' and display headers in RED as has been suggested

You are the only one to have suggested display in red; if
you feel the need to erect strawmen for the sole purpose
of subsequently showing how easily they can be knocked
down, perhaps you should consider that there is a
fundamental weakness in your position.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Mon, 18 Oct 2004 22:58:30 -0400
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

On Mon October 18 2004 10:16, Charles Lindsey wrote:

> So I have suggested a method using group-syntax that would be entirely be
> compatible with the present protocols,

No, attempting to change display text into a protocol keyword is
incompatible with the text message format (both the full Standard
and the Proposed Standard), is incompatible with the Internet
Architecture and with BCP (RFC 2277), and use of named groups
is incompatible with the Standards Track Sieve protocol (RFC 3028).
It is also anglo-centric and ambiguous.  All of which has been
explained, with cited references.  Your claim of compatibility is
a falsehood.

> but I have been shouted down.  

No, you have been told that you are wrong; you have been told
*why* you are wrong, and you have been reminded that you have
been told that you are wrong and why -- in all cases calmly and
as courteously as possible under the circumstances of your
repetition of debunked false claims and displays of seriously flawed
arguments.  That is not being "shouted down"; it is part of
a technical discussion.  If you do not understand that you are
wrong, you can consult the references which have been
provided.  If you do not understand those references, you can
ask questions.

> I 
> note that none of the shouters has come up with a better method.
[...]
> >There is no general mechanism to identify a list,
> 
> Then invent one!

Perhaps you should consider the possibility that that is an
impossible task.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Mon, 18 Oct 2004 22:13:59 -0400
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

On Mon October 18 2004 10:46, Charles Lindsey wrote:

> there is a sustained campaign 
> by one poster to shout down the whole of the rest of the community.
[...]
> symptomatic of the huge gulf between yourself and Real
> (TM) software engineers.
[...]
> Bullshit!
[...]
> No, you are the only person here who believes
[...]

Now what was that about "a sustained campaign by one
poster to shout down"...?

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: Hector Santos <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Mon, 18 Oct 2004 22:08:19 -0400
Cc: Keith Moore <[email-masked]>, "ietf-822" <[email-masked]>
Sender: [email-masked]

On Mon October 18 2004 16:58, Hector Santos wrote:

> Now, if you want to work on a SOLUTION to address the "Groupware" emulation
> of an email mailing list concept for MUAs, then do so, otherwise SHUT UP.

You seem to be unable to accept the fact that several people disagree
with your opinions for valid reasons (which have been patiently
explained to you).  That is your problem, and you'll have to learn
to deal with it.  Telling others to shut up is inflammatory,
counter-productive, and unprofessional; that creates problems
for mailing lists like this one.  I suggest that you take a more
professional approach.  You might also want to read RFC 3683. 

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: Understanding response protocols
Date: Mon, 18 Oct 2004 21:31:11 -0400
Cc: "Hector Santos" <[email-masked]>
Sender: [email-masked]

On Mon October 18 2004 17:43, Hector Santos wrote:

> The issue is how the MUA is not designed to expect an "email message" to be
> viewed as a "group conferencing" concept.

Many MUAs work fine for lists; if the one you use does not, that's an
implementation issue with that particular MUA.

> The natural Reply for an email is to go back to the one person who wrote it.

There are several things wrong with that statement: first, there
may well be multiple authors; that is not uncommon. Second,
there is no such thing as a "natural" reply; there are several
types of automatic responses (RFC 3834) and manual responses
may go to different places depending on the expressed
recommendation of the author(s) and the desires of the person
responding.

> However, in a list distribution, the "list address" is the new responsible
> "person" or "entity" for where a natural reply should go to.

Absolutely not; automatic responses (delivery failures, vacation
messages, etc.) should never go to a mailing list.  Depending
on the original message's authors' intent, direct responses to
the author (to be summarized, for example) may be desired. In
other cases, the author(s) may have reason to recommend
some other destination for responses.

In either case, there is only the author's recommendation (if
provided), and a series of protocols describing where responses
should be sent under various circumstances.

Cc: <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Mon, 18 Oct 2004 23:53:42 -0400
To: "Hector Santos" <[email-masked]>
Sender: [email-masked]

> But what is your solution?

I'm still trying to understand what the problem is.
(that is, other than the fact that some people get annoyed when they 
get multiple copies of replies)

I feel fairly certain that there's more to it than that, but offhand I 
don't know what it is.

If anything, it seems like a really effective medium for group 
conversation would need to be a lot more functional than anything we 
can expect to implement via a user interface for interpersonal email.   
My impression is that modern IM systems are a lot further along this 
curve than either email or usenet is - not just because they are 
immediate, but because they offer a very rich set of tools for group 
interaction.

Anyway, in my mind, the primary purpose for email is to support 
interpersonal communication, not group discussion.  Tweaking email to 
better support group discussion in a way that harms interpersonal 
communication seems like an inappropriate diversion.

Keith

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Mon, 18 Oct 2004 21:35:42 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, October 18, 2004 6:58 PM
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member
messages to lists]

> > Please understand what is being said here first before injected more
> > stubborn responses that lead no where.
>
> What I understand is that you're making this stuff up, trying to
> interpret the mail protocol in your own way without regard for
> interoperability or other people's experience.
>

Keith,  it seems very natural of you to call the kettle black. That is
exactly most people see you.

In any case, I am not making anything up.  I pointed out the conflicted
expected behaviors for MUAs in regards to a EMAIL vs. MAILING List concept.
I believe most agree with this.

The issue now seems to be HOW you believe it is not a requirement (or
disagree) to defined what is the "default" group address for a natural reply
response.

I believe it is when it comes to a LIST concept.  This isn't making things
up.  I can atleast point to real commercial product operations with this
stuff and its not exclusive to just offline mail reading, but also online.
Can you?    It is a long time product design, that at a minimum,
distinguishes an operation that works better for the most common MUA, and
that includes both OFFLINE and ONLINE operations.  If there is anyone
swaging it, it is you.

Yet, I understand very clearly that when it comes to the MUA offering better
support for a natural list reply, makes it more difficult for the user to
reply off-list directly to the author.

On the other hand, the list system (such as the one used here) that doesn't
make it natural, makes it more difficult for the user as well to reply just
to the list, which also includes a propensity for the creation of extra
unnecessary bandwidth.

I prefer the first design.   Nonetheless, once again, I think the ultimate
solution is as follows:

1) List Expanders include a standard List-Address: header or
    something that reflects the address to response back to the list(s)
   that may be different from the author's desirable Reply-To:

2) This will help future MUA to better handle all types:
    Pure Email, List Email and News as well.

If you don't agree. Fine. I can expect that from you.

But what is your solution?

--
Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com

Cc: <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Mon, 18 Oct 2004 18:58:55 -0400
To: "Hector Santos" <[email-masked]>
Sender: [email-masked]

>>> First, please don't send duplicate messages to me.
>>
>> I'm not.  I'm sending a reply to you, and cc'ing the list.
>
> And you honestly do not see the significance of this? and how it 
> relates to
> the convoluted conflictive philosophical "opinion" of expected 
> behavior?

I do not pretend to be able to read your mind.

>> This is appropriate because both are interested parties.
>
> I'm already part of the list.

I'm not going to keep track of who is on which list and who is not.  
For all I know, you've already quit the ietf-822 list in disgust.  (I 
can always hope...)

> I'm replying to the list. I don't need or want to send directly to you.

You are being rude by not explicitly including me.  Perhaps this is 
deliberate, and it is your choice to make.  Still, it shows a certain 
lack of consideration.  Because you don't know whether I'm on the list 
either.  In fact "[email-masked]" is NOT on the list, but I get the 
list via other means.

> Yet, you were able to be a part of the communication.  There is no 
> need to send me a 2nd copy - its a waste.

If I had a reliable way to (a) accurately know in real time who was on 
which lists, and (b) still be able to indicate that you were an 
intended recipient, I'd be happy to save the network the burden of 
transporting those few bits.  But (a) is actually a hard problem if you 
try to deal with race conditions.  It's technically much easier to 
eliminate duplicates on your end.  And for most messages, it's probably 
more efficient to send multiple copies than to try to eliminate 
duplicates.

>> If it happens that this results in multiple copies arriving at your 
>> inbox that's your problem,  not mine -
>
> Which explains why there isn't going to be solution with this ever
> persistent (no matter what it is) attitude of yours.

You are the one expecting other people to solve your problems...  And 
you want to blame my unwillingness to do so (and my realization that 
others may not be willing to solve your problem either) on _my_ 
attitude?  get a clue.

> Yet, the irony is you say this but in the same breath you say.......
>
>> at least until we have some universally
>> accepted way of (a) indicating to other recipients that you don't
>> want to be cc'ed on replies and (b) indicating to other recipients
>> that you're implicitly getting replies even though you are not
>> explicitly being sent a reply.
>
> So because this isn't available, its "my problem?"   No, the problem 
> begins
> because you needed to "REPLY to ALL" to get the list address.

False.  I INTENDED the reply to go to you and the list because both you 
and the list are appropriate recipients of the message.   In this case 
it's not a failure of my user interface to provide me with more or 
easier-to-use options - it's the fact that we don't have the necessary 
protocol support.

> It relates to the concept that the email output generation of a LIST
> Distribution is not the same as would be the 1 to 1 email direct 
> transport
> where the MAIL FROM is persistent.   It is NOT persistent in a List
> Distribution.

The envelope is different.  The payload is the same, modulo List-* 
fields.  We are talking about the payload.

> Please understand what is being said here first before injected more
> stubborn responses that lead no where.

What I understand is that you're making this stuff up, trying to 
interpret the mail protocol in your own way without regard for 
interoperability or other people's experience.

Keith

Cc: <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: Understanding response protocols
Date: Mon, 18 Oct 2004 18:42:04 -0400
To: "Hector Santos" <[email-masked]>
Sender: [email-masked]

> The issue is how the MUA is not designed to expect an "email message" 
> to be
> viewed as a "group conferencing" concept.

or maybe, the issue is that email freely mixes groups and individuals, 
without distinguishing between the two, but there are some people who 
want "groups" to work differently.

(question: other than not generating duplicates - which can be solved 
by other means - what are other ways that group addresses should work 
differently than individual addresses? )

> The natural Reply for an email is to go back to the one person who 
> wrote it.

not clear.  in paper correspondence a reply is often sent to other 
recipients and cc's of the original message.  basically, it is sent to 
people who should get the reply.

> However, in a list distribution, the "list address" is the new 
> responsible
> "person" or "entity" for where a natural reply should go to.

also not clear, especially when some recipients of the original message 
are lists and some are not, some of the recipients of the original 
message are not included in any of the lists and others are, some 
recipients of the original message deal with their list mail 
differently than their personal mail, and there's no way for the 
replier to know which of these is true for which recipients.

> So the question is, if the MUA is currently designed to see the EMAIL 
> style
> response logic to include the user of Reply-To:/From:  for the natural 
> reply
> address, should the backend list manager do more in this area to 
> assist in
> the process?
>
> I say yes, and this works 100% in this area.

you are using dubious premises, so it's hardly surprising if your 
conclusion is wrong.

most of us have used lists that munged reply-to.  most of us have seen 
the problems associated with doing so.  if you haven't seen those 
problems yet, fine, feel free to read various missives on the subject 
that are on the net.  but don't expect us to ignore what our 
experiences have taught us.

Keith

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Mon, 18 Oct 2004 18:21:17 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Monday, October 18, 2004 5:49 PM
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member
messages to lists]

> > First, please don't send duplicate messages to me.
>
> I'm not.  I'm sending a reply to you, and cc'ing the list.

And you honestly do not see the significance of this? and how it relates to
the
convoluted conflictive philosophical "opinion" of expected behavior?  No
need to
answer this.

> This is appropriate because both are interested parties.

I'm already part of the list. I am already going to get a copy of the
message. I don't need a direct private offlist email.

I'm replying to the list. I don't need or want to send directly to you.
Yet, you were able to be a part of the communication.  There is no need to
send me a 2nd copy - its a waste.

> If it happens
> that this results in multiple copies arriving at your inbox that's
> your problem,  not mine -

Which explains why there isn't going to be solution with this ever
persistent (no matter what it is) attitude of yours. Yet, the irony is you
say this but in the same breath you say.......

> at least until we have some universally
> accepted way of (a) indicating to other recipients that you don't
> want to be cc'ed on replies and (b) indicating to other recipients
> that you're implicitly getting replies even though you are not
> explicitly being sent a reply.

So because this isn't available, its "my problem?"   No, the problem begins
because you needed to "REPLY to ALL" to get the list address. You didn't
manually add all these addresses to your Reply Dialog.  In this effort, it
includes all the other addresses which for some odd reason, you decided not
to clean it up.

> > I don't need to get two
> > copies.  That's the problem.  In order to send to the LIST, you have to
> > naturally SEND to ALL.
>
> False.  I can specify the list explicitly, and sometimes do.

Yet, you did not here. Why?  List participants don't need second direct
offlist copies

> > You even SENT two copies to yourself!
>
> nope.  I sent one copy to my inbox, and another to my personal
> archive of the mailing list.  They are not the same.

Your two addresses shown above they are the same. Anyway. It is a mute
point.

> > Second, there is nothing wrong with our list server.   Once you post to
the
> > LIST, the LIST MANAGER takes responsibility for any distribution at the
> > point.  The proof can be viewed as simply this:
> >
> >     At the distribution, the 821.MailFrom is NO Longer the original
> >     poster
>
> who said anything about MAIL FROM?

I am!

> we're talking message headers here.
> the [2]822/MIME protocol is between users and their agents.  transport
> has nothing to do with it.

It relates to the concept that the email output generation of a LIST
Distribution is not the same as would be the 1 to 1 email direct transport
where the MAIL FROM is persistent.   It is NOT persistent in a List
Distribution. It is changed once the responsible domain takes over.   Hence
this means the list manager has a new responsibility to provide the best
"expectations" for the end-user agent reading the email list message, not as
a pure direct/private 1 to 1 email message.

Please understand what is being said here first before injected more
stubborn responses that lead no where.

---
Hector Santos, CTO
WINSERVER "Wildcat! Interactive Net Server"
support: http://www.winserver.com
sales: http://www.santronics.com

Date: Mon, 18 Oct 2004 17:49:48 -0400
From: Keith Moore <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Sender: [email-masked]

> > > Again,  once that message hits the LIST manager, it is going to be
> > > "handled" how it sees fit whether you like it or not!  The list 
> > > expander has a vital role in it.
> >
> > You are wrong.  Your list manager is broken.  Please fix it.
> 
> First, please don't send duplicate messages to me. 

I'm not.  I'm sending a reply to you, and cc'ing the list.  This is 
appropriate because both are interested parties.  If it happens
that this results in multiple copies arriving at your inbox that's 
your problem, not mine - at least until we have some universally 
accepted way of (a) indicating to other recipients that you don't 
want to be cc'ed on replies and (b) indicating to other recipients
that you're implicitly getting replies even though you are not
explicitly being sent a reply.

> I don't need to get two
> copies.  That's the problem.  In order to send to the LIST, you have to
> naturally SEND to ALL. 

False.  I can specify the list explicitly, and sometimes do.

> You even SENT two copies to yourself!

nope.  I sent one copy to my inbox, and another to my personal
archive of the mailing list.  They are not the same.
 
> Second, there is nothing wrong with our list server.   Once you post to the
> LIST, the LIST MANAGER takes responsibility for any distribution at the
> point.  The proof can be viewed as simply this:
> 
>         At the distribution, the 821.MailFrom is NO Longer the original
> poster

who said anything about MAIL FROM?  we're talking message headers here.
the [2]822/MIME protocol is between users and their agents.  transport
has nothing to do with it.

Keith

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Understanding response protocols
Date: Mon, 18 Oct 2004 17:43:52 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Nathaniel Borenstein" <[email-masked]>
To: "Keith Moore" <[email-masked]>
Cc: <[email-masked]>; "Charles Lindsey" <[email-masked]>
Sent: Monday, October 18, 2004 10:33 AM
Subject: Re: Understanding response protocols

> We need to map system behavior onto abstractions that normal users will
> be able to comprehend.  The present situation is actually be pretty
> good in this regard, except for the little matter of duplicate
> elimination.  But most of us realize that isn't possible in all case --
> if I send mail to "[email-masked], [email-masked]" I have no way of telling
> that [email-masked] actually forwards to [email-masked].  In the most general
> case, the only duplicate detection you can completely depend on will be
> on the receiver's side.  This suggests two possible courses of action:
>
> -- Change nothing, and leave most duplicate-eliminateion to the
> receiver-side software.
>
> -- Try to optimize some specific common cases, which is the goal of MFT.

The issue is how the MUA is not designed to expect an "email message" to be
viewed as a "group conferencing" concept.

The natural Reply for an email is to go back to the one person who wrote it.

However, in a list distribution, the "list address" is the new responsible
"person" or "entity" for where a natural reply should go to.

So the question is, if the MUA is currently designed to see the EMAIL style
response logic to include the user of Reply-To:/From:  for the natural reply
address, should the backend list manager do more in this area to assist in
the process?

I say yes, and this works 100% in this area.

However, when the user wants to send directly to a user offlist, then it is
not so natural anymore with the current MUAs.

Not adjusting the Reply-To: forces the natural reply for a email-list
message to go to the author, not the list address.

So far for our customer base, we have never had any report, wish item or
comment about the "lack of ability" send directly to a list member.  That
could be because they understand that they are part of a list and the
natural reply is to go back to the list.   Do anything else, needs extra
steps.

In any case, from a design standpoint, I see the solution is to have a
"indicator" that could help prepare future MUAs the difference between:

    - 1 to 1 Email message
    - 1 to Many Email Message

I think we need a standard set of new headers that might begin with having a
"List-Address:" which would be the natural reply address option.  The future
MUA would have a 3rd option:

    Reply to List        --> List-Address:
    Reply to Author    --> Reply-To: | From:
    Reply to All --> whatever

Thanks

---
Hector Santos, CTO
WINSERVER "Wildcat! Interactive Net Server"
support: http://www.winserver.com
sales: http://www.santronics.com

Date: Mon, 18 Oct 2004 17:09:59 -0400
From: Keith Moore <[email-masked]>
To: "D. J. Bernstein" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

> 
> Keith Moore writes:
> > Actually some people are proposing blind acceptance.  They want MFT
> > to be honored automagically, regardless of the content of the reply.
> 
> No. Mail-Followup-To does _not_ override the wishes of the user
> sending the followup. It is simply a _default_ recipient list.
...

> You say that some MUAs have bad list-editing interfaces: in
> particular, they don't make the recipient list clear to the user.
> That's certainly something to fix---no matter what the default
> recipient list is! You keep bringing this up in a Mail-Followup-To
> context as if it were a new problem; but the same problem existed long
> before Mail-Followup-To.

agreed. 

I think there are some problems with the very notion of a
sender-settable _default_ recipient list - one that encourages the
reader to use that list for replies without thinking about it.   This is
equally a problem for MFT or Reply-To, and probably for other ways of
setting that default.  And while it's certainly possible to build MUAs
that don't  treat MFT or Reply-To or whatever as a "default" but instead
do something else with it - let the user choose it explicitly, insist
that the user make an explicit choice, whatever - I wonder if it will
violate the sender's expectation if recipients consciously choose to
reply to somewhere else.

Both MFT and Reply-To seem to expect repliers to understand, via
out-of-band information, the  relationships between the specified 
recipients and the other recipients of a message when the repliers they
make a conscious choice. They don't say _why_ the replies are being
redirected that way, or give any guidance about what to do if the
replier wants to change the default.  Or perhaps they assume (not
entirely without justification) that repliers will rarely bother to make
a conscious choice, so the number of misdirected replies that result
from such choices will not be significant.

Because I'm looking at how to make email more effective, one of the
things I'm concerned about is the rarity of conscious choice when
replying to emails.  So I'm wondering if there are ways that MUAs can 
be improved to encourage conscious choices without being too annoying.
And because I want to encourage conscious choices if it is possible to
do so (and I'm not sure that it is), I'm wary of new header fields which
seem to assume that users will not make conscious choices.  I think such
 fields could even discourage conscious choice, by creating a wider
expectation that the recipient will do what the sender said (thus 
negative feedback if the recipient does something different even when he
had a good reason); and because, lacking information as to _why_ the
sender set MFT, the recipient might decide (again, without much
justification) that the sender's judgement is better than his own.

As I think I said earlier, if I do manage to find time to construct an
MUA to try out some of these ideas, I'll arrange for it to recognize
MFT, alert the replier to its presence, and make it easy for the replier
to use that recipient list.  It won't be quite like treating MFT as a
default but maybe it will strike an appropriate balance.  

Keith

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Mon, 18 Oct 2004 16:58:14 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Monday, October 18, 2004 3:54 PM
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member
messages to lists]

>
> > Again,  once that message hits the LIST manager, it is going to be
"handled"
> > how it sees fit whether you like it or not!  The list expander has a
vital
> > role in it.
>
> You are wrong.  Your list manager is broken.  Please fix it.

First, please don't send duplicate messages to me.  I don't need to get two
copies.  That's the problem.  In order to send to the LIST, you have to
naturally SEND to ALL.  This is evident by
the header block indicated above.  You even SENT two copies to yourself!

Second, there is nothing wrong with our list server.   Once you post to the
LIST, the LIST MANAGER takes responsibility for any distribution at the
point.  The proof can be viewed as simply this:

        At the distribution, the 821.MailFrom is NO Longer the original
poster

Our List Server makes it very convenient for the expectations of the RFC
based reader MUA behavior so the user clicks the REPLY button, it goes BACK
to the list.  Not back to the ORIGINAL author which is NOT the natural
expectation of the user.

Now, if you want to work on a SOLUTION to address the "Groupware" emulation
of an email mailing list concept for MUAs, then do so, otherwise SHUT UP.

--
Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com

Date: Mon, 18 Oct 2004 16:15:19 -0400
From: Keith Moore <[email-masked]>
To: Nathaniel Borenstein <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

> 
> I'll probably regret getting back into this thread, but I wanted to 
> amplify something Keith said a while back about typical users:
> 
> > I suspect that when people hit "reply to all" to a message from a
> > list they generally want the reply to go to everyone who received
> > the original message, not "just" to the list.  They don't want to
> > worry about which of those recipients are on the list and which ones
> > are not.
> 
> I'd go further and claim that the typical user lacks the 
> interest/attention to deal effectively with more than 2 reply choices,
> which the user will naively think of as "the sender" and "everyone."  
>  
> Trying to get most users to handle a third choice -- e.g. "reply to 
> all" vs. "followup" -- is asking at least as much of the user as 
> expecting them to simply edit out the duplicates by hand.  

I agree.  But I think a lot of the problem is that most MUAs expect
users to make this choice from the "read" or "browse" environment
rather than the "compose" environment.  Adding more buttons in the
"browse" environment only makes things worse.  

Another part of the problem is that most MUAs don't provide a better way
to alter the recipient list of a reply than to edit the list "by hand" -
and even then, without an effective means of viewing the recipient list
of the subject message.

Especially these days, in the browse environment the user is often in
"whack-a-mole" mode - he's got various kinds of stuff in his inbox,
including maybe important personal messages, trivial personal messages,
important "work" messages, trivial "work" messages, list traffic, spam,
and viruses -all jumbled together.  His main goal is to keep the list of
things that haven't been looked at from getting too long.  Some things
get deleted unseen, some get looked at and then deleted, some get
forwarded, some may get refliled if the user is sophisticated enough. 
Basically the user is performing triage.

Even though a lot of messages do get terse replies, the "compose" screen
at least gives the user a chance to shift gears and focus on the message
he's composing.    That's why I think that the  selection of reply
recipients needs to be part of the compose  environment.  To be
effective the compose environment needs to make it easy to reference,
quote, and or incorporate portions of the subject message - and this
includes making it easy to reference, quote, and/or incorporate portions
of the recipient list from the subject message into the recipient list
of the reply.

Note that a similar problem exists for reply body text.  We find it
quite useful for an MUA to be able to include the  subject message text
in a reply, but quite tedious to edit out the irrelevant portions.  So
just like we get a lot of replies that cc too many recipients, we also
get a lot of bloated replies that copy too much text from earlier
messages in the thread.  Cut-and-paste doesn't work as well as we'd like
to believe - it's sort of effective for small portions of text, less so 
for large chunks.  We need to be able to see text in a subject message
while we're replying to it (and even have the right part of that text on
the screen while we're replying to that part) without actually having
to have that text copied into the reply, and we need to be able to 
include that text in our replies without having to drag it from another
window.

To: LIST: [email-masked];
Xref: clerew local.mime:3929
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Mon, 18 Oct 2004 14:46:02 GMT
Lines: 72
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Fri October 15 2004 21:12, Hector Santos wrote:

FWIW, I agree with almost everything Hector Santos has said in this
exchange. Although I accept that "me too" postings are not generally a
good idea, I think they do have a place when there is a sustained campaign
by one poster to shout down the whole of the rest of the community.

>> Bruce,  does this suggest not a software engineer or developer or even close
>> to understanding the concept?  If so, I won't hold that against you thought.

>Writing of MUAs as being "trained" demonstrates at best a
>very odd way of thinking about MUAs; certainly not to be
>expected from a software engineer or developer or anybody
>"even close to understanding the concept" of MUAs.

I found that metaphor completely appropriate, and your failure to
understand it as symptomatic of the huge gulf between yourself and Real
(TM) software engineers.

>It can't be done. While there are specific (RFC 2369, 2919) message
>header fields that a mailing list expander MAY insert, and while a
>mailing list expander SHOULD appropriately set the envelope sender
>address to the mailing list maintainer's mailbox, there is no guaranteed
>way that any MUA can always identify all list messages.  That is because
>the mailing list-specific fields are optional and because a mailing list
>mailbox is indistinguishable from an individual mailbox.  Mailing list-
>specific fields cannot be made mandatory under IETF rules, because
>imposing requirements is reserved for cases where it is necessary
>to ensure interoperation and to avoid network damage (such as
>severe congestion caused by retransmissions); since the list-specific
>fields play no role in protocol but are merely informative, there is no
>way that omitting any or all of those fields can lead to such problems.

Bullshit! The present mess that we are discussing is indeed an
interoperability problem, since there is no clear and consistent way of
conducting discussions between users on arbitrary lists using arbitrary
MUAs without being aware of the particular quirks of each user, list and
MUA and making manual corrections to allow for them. Therefore there is no
reason why an appropriate standard should not use at least SHOUL wording
as regards inserting those headers.

If the IETF mechanisms will not allow that to be done, then there is
something seriously wrong with the IETF mechanisms. But even if that is
the case, then it can (and should) be done in a BCP document.

>Moreover, there is no need for an MUA to determine that a list is
>involved; an author is supposed to set the Reply-To field to indicate
>where responses should go if that is somewhere other than directly
>to him (the author), and a respondent only needs to be able to see
>the author's recommendation and to determine whether he (the
>respondent) wishes to comply with that recommendation or to send
>his response elsewhere.  That applies for *all* messages, mailing list
>or otherwise.

No, you are the only person here who believes that that solution is
adequate. Everybody else realises a need to distinguish between the
address that the author wants personal replies to go to (which may not be
his From address) and the address that the author wants list replies to go
to. You cannot convey that information with just one header.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3927
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Mon, 18 Oct 2004 14:16:16 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Tue October 12 2004 23:04, Hector Santos wrote:

>> a) Does a MUA need to detect a Mailing List Message as oppose to a just a
>> pure email message?

>No, and in general it cannot do so.

Things would certainly be much simplified if it were possible to tell.

The presence of one of the List-* headers would give a clear indication,
but one cannot rely on them being present, and even if they are they only
tell you about one list, and the message may have been crossposted to
several lists.

So I have suggested a method using group-syntax that would be entirely be
compatible with the present protocols, but I have been shouted down. I
note that none of the shouters has come up with a better method.
> 
>> c) What are the most desirable reply/new mail options for a mailing list?
>> 
>>     Reply to List (very similar to a news Reply To Group idea)?

>There is no general mechanism to identify a list,

Then invent one!

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3928
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Mon, 18 Oct 2004 14:22:57 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> "D. J. Bernstein" <[email-masked]> writes:

>Yes, those are the basic two buttons, but you're probably confusing some
>people here by using the name ``reply to list'' for the second button;
>there's a widespread MUA with a third button labelled ``list reply,''
>quite different from the second button.

>(The third button screws up crossposts, and it fails to send Keith the
>cc's that he wants. So why was it added? Because the second button,
>_without_ Mail-Followup-To, was even worse. On the other hand, in a
>Mail-Followup-To world, the second button works fine, and the third
>button should be eliminated.)

>USENET has consistently used the name ``reply'' for the first button and
>the name ``follow up'' for the second button. Why invent new names?

I think you still need the Reply-to-All button for when that IS really
want you want to do, and even newsreaders could make good use of it in cases
where Cc: etc headers were present in additional to the usual From.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3925
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Mon, 18 Oct 2004 13:56:22 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Wed October 13 2004 07:51, Charles Lindsey wrote:
>> 
>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
>> >I would add that you have failed to provide any concrete example
>> >of specific software which generates identifiers referring to
>> >multiple distinct entities which are identical on the LHS and differ
>> >only in case in the RHS.
>> 
>> I am aware of no such software, and doubt that any such software exists.

>Ergo you know of no reason to deviate from the
>long-standing (*decades*) rule that an identifier RHS is a
>case-insensitive domain name.

Wrong!

Ergo there is no necessity for <id-right>s to be compared
case-insensitively, and the change introduced by RFC 2822 is a welcome
simplification of the protocol.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3924
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Mon, 18 Oct 2004 13:49:59 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> "D. J. Bernstein" <[email-masked]> writes:

>Keith Moore writes:
>> Actually some people are proposing blind acceptance.  They want MFT to 
>> be honored automagically, regardless of the content of the reply.

>No. Mail-Followup-To does _not_ override the wishes of the user sending
>the followup. It is simply a _default_ recipient list. The old de-facto
>standard was

Exactly so.

In Usenet, posters have the option of letting followups go to the original
newsgroup(s) (that is the default), or of asking them to go to a subset of
those groups (or even to a totally disparate set of newsgroup(s)), or of
asking them to be mailed back to the poster.

And yet I have heard no demands to abolish this system on the grounds that
the replier's rights to expect followups to go to the original newsgroups
are being usurped. Usenet users are used to the idea that they should take
note of where the followup is actually going to go, and to override
manually if they do not like that. Newsreaders do not, in general, put up
'alerts' and display headers in RED as has been suggested (though I grant
that it is considered polite to say in the body that followups have been
directed other than naive users might expect).

So if that has been working fine for 20 or more years in Unsenet, why the
great issue if a means to achieve essentially the same effect is proposed
for mailing lists?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3926
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Mon, 18 Oct 2004 14:07:37 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> Martin Trautmann <[email-masked]> writes:

>> PS: There should not be a Newsgroups: header when sending your email.
>> Atleast this MUA is thinking it is a newsgroup article when replying.

>On the other hand you HAVE to set Newsgroups when you do e.g. a 
>followup while you send a Cc per email.

And in that case the Newsgroups header needs to be left in the mailed
version so that the recipient of the mail is aware that it was also
posted, and to which group(s).

So a Dual-Use User Agent needs some better way to determine whether the
message in hand is an email message or a news article. The proper way to
do this is to keep track of where it got it from. If it arrived by NNTP
from a news-server, then it was a news article; if it arrived via SMTP or
from a message store which obtained it from SMTP, then it is an email
message. If an Agent is setting out to be for "dual-use" between two
media, then it behoves it to know which medium it is handling at any given
moment.

And if it absolutely MUST guess which medium it is from examination of the
message, then a better discriminator would be the presence or absence of a
Return-Path header, of a "^From " line.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 18 Oct 2004 15:54:00 -0400
From: Keith Moore <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Sender: [email-masked]

> 
> Again,  once that message hits the LIST manager, it is going to be "handled"
> how it sees fit whether you like it or not!  The list expander has a vital
> role in it.

You are wrong.  Your list manager is broken.  Please fix it.

To: Nathaniel Borenstein <[email-masked]>
Cc: [email-masked]
Subject: Re: Understanding response protocols
From: Nick Shelness <[email-masked]>
Date: Mon, 18 Oct 2004 16:35:17 +0100
Sender: [email-masked]

Nathaniel,

> I'd go further and claim that the typical user lacks the 
> interest/attention to deal effectively with more than 2 reply choices, 
> which the user will naively think of as "the sender" and "everyone." 
> Trying to get most users to handle a third choice -- e.g. "reply to 
> all" vs. "followup" -- is asking at least as much of the user as 
> expecting them to simply edit out the duplicates by hand. 

IMHO, the "typical user" or perhaps the "non-technical business user" 
lacks the interest/attention to deal effectively with more than a single 
reply choice -- "everyone". I continue to be amazed at the stuff I'm 
completely needlessly copied on, and have come to the conclusion that for 
these users there is only a single, and usually inappropriate, reply 
action.

Nick

Date: Mon, 18 Oct 2004 16:56:21 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Understanding response protocols
Cc: Nathaniel Borenstein <[email-masked]>, Keith Moore <[email-masked]>, Charles Lindsey <[email-masked]>
Sender: [email-masked]

Nathaniel Borenstein writes:
> I'd go further and claim that the typical user lacks the 
> interest/attention to deal effectively with more than 2 reply 
> choices, which the user will naively think of as "the sender" and 
> "everyone."    Trying to get most users to handle a third choice -- 
> e.g. "reply to all" vs. "followup" -- is asking at least as much of 
> the user as expecting them to simply edit out the duplicates by hand.

Hear! Hear!

> However, I question whether MFT might not cause as many problems as it 
> solves.  Consider:
>
>        To: [email-masked]
>        CC:  [email-masked], [email-masked], [email-masked]
>
> This is something I have done in the past in order to bring into a 
> list conversation a few people who are not on the mailing list.  Now, 
> what should the list processor at example.com do with this?

Nothing?

> As I understand it (please correct me if I'm wrong) current 
> implementations would redistribute the message like this:
>
>        Mail-Followup-To: [email-masked]
>        To: [email-masked]
>        CC:  [email-masked], [email-masked], [email-masked]

This matches my understanding, and I have seen messages like this, but I 
had the impression that the MFT field was added by the sender's MUA or 
MTA?

Arnt

Cc: [email-masked], "Charles Lindsey" <[email-masked]>
From: Nathaniel Borenstein <[email-masked]>
Subject: Re: Understanding response protocols
Date: Mon, 18 Oct 2004 10:33:40 -0400
To: Keith Moore <[email-masked]>
Sender: [email-masked]

I'll probably regret getting back into this thread, but I wanted to 
amplify something Keith said a while back about typical users:

> I suspect that when people hit "reply to all" to a message from a list 
> they generally want the reply to go to everyone who received the 
> original message, not "just" to the list.  They don't want to worry 
> about which of those recipients are on the list and which ones are 
> not.

I'd go further and claim that the typical user lacks the 
interest/attention to deal effectively with more than 2 reply choices, 
which the user will naively think of as "the sender" and "everyone."    
Trying to get most users to handle a third choice -- e.g. "reply to 
all" vs. "followup" -- is asking at least as much of the user as 
expecting them to simply edit out the duplicates by hand.  (This is why 
I ultimately gave up on the three-choices model in Andrew.)

We need to map system behavior onto abstractions that normal users will 
be able to comprehend.  The present situation is actually be pretty 
good in this regard, except for the little matter of duplicate 
elimination.  But most of us realize that isn't possible in all case -- 
if I send mail to "[email-masked], [email-masked]" I have no way of telling 
that [email-masked] actually forwards to [email-masked].  In the most general 
case, the only duplicate detection you can completely depend on will be 
on the receiver's side.  This suggests two possible courses of action:

-- Change nothing, and leave most duplicate-eliminateion to the 
receiver-side software.  

-- Try to optimize some specific common cases, which is the goal of MFT.

However, I question whether MFT might not cause as many problems as it 
solves.  Consider:

	To: [email-masked]
	CC:  [email-masked], [email-masked], [email-masked]

This is something I have done in the past in order to bring into a list 
conversation a few people who are not on the mailing list.  Now, what 
should the list processor at example.com do with this?  As I understand 
it (please correct me if I'm wrong) current implementations would 
redistribute the message like this:

	Mail-Followup-To: [email-masked]
	To: [email-masked]
	CC:  [email-masked], [email-masked], [email-masked]

Thus people whose UA's respected MFT would end up omitting Bill, Bob, 
and Jim from their replies, which was not my expectation as the sender. 
  Is this really the desired behavior?  Or should all the non-list 
addresses except the sender in fact be copied into the followup header? 
  One could argue that the MFT should include everyone that would 
normally be in "Reply to All" *except* those who are on the list -- 
after all, if the list processor is doing the rewriting, it's in a 
position to do that, right?  -- Nathaniel

Date: Mon, 18 Oct 2004 09:23:44 +0100 (BST)
From: Philip Hazel <[email-masked]>
To: ietf-822 <[email-masked]>
cc: Hector Santos <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Sender: [email-masked]

On Sun, 17 Oct 2004, Bruce Lilly wrote:

> Writing of MUAs as being "trained" demonstrates at best a
> very odd way of thinking about MUAs; 

Somebody whose first language is not English might well say "trained" 
when a native speaker would say "configured". 

-- 
Philip Hazel            University of Cambridge Computing Service,
[email-masked]      Cambridge, England. Phone: +44 1223 334714.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Sun, 17 Oct 2004 09:56:26 -0400
Cc: "Hector Santos" <[email-masked]>
Sender: [email-masked]

On Fri October 15 2004 21:12, Hector Santos wrote:

> > > No, I think I am pretty verse of the MUA "standard practice" [...]
> >
> > A response to the person or people identified in a message's
> > From field is a response to the author(s), not necessarily the
> > "sender".  See RFC 2822 section 3.6.2.
> 
> Why are you indicating the obvious?

Because you keep writing "reply to sender" when you are
discussing "reply to author", indicating that you're confused
about the difference between "sender" and "author", and
therefore ignorant of how MUAs function.

> > > I mean, we are trying to get the MUA to do something it was simply
> > > not trained to do.
> >
> > MUAs aren't "trained".
> 
> Bruce,  does this suggest not a software engineer or developer or even close
> to understanding the concept?  If so, I won't hold that against you thought.

Writing of MUAs as being "trained" demonstrates at best a
very odd way of thinking about MUAs; certainly not to be
expected from a software engineer or developer or anybody
"even close to understanding the concept" of MUAs.
 
> > > If sent to the list, it list server has a big role here.
> >
> > From the moment a respondent decides to prepare a response and
> > until he sends it, there is no role for any list expander; it is simply
> > not involved.
> 
> Again,  once that message hits the LIST manager, it is going to be "handled"
> how it sees fit whether you like it or not!  The list expander has a vital
> role in it.

But that has nothing to do with a respondent preparing and
sending a response; there is no involvement of a list expander
in that process.
 
> > > I explained how
> > > the list server used here seems to not do this, but our list server will
> > > make sure that the proper  Reply-To: is used for the distribution.
> There is
> > > nothing technically wrong with that
> >
> > It *is* wrong for anybody other than a message author to set
> > a Reply-To field. See RFC 2822 section 3.6.2.
> 
> Not quite. Thats an interpretation for 2822 from an END-USER such as
> yourself with no experience in mail gateway, distribution product design.

You are wrong on all counts.

> In fact RFC 822 has no special "author request" phrase.

First, I referred you to a specific section of RFC 2822 (not 822),
which says in part:

   When the "Reply-To:" field is present, it
   indicates the mailbox(es) to which the author of the message suggests
   that replies be sent.

Second, RFC 822 says:

        Note:  The "Reply-To" field is added  by  the  originator  and
               serves  to  direct  replies,  whereas the "Return-Path"
               field is used to identify a path back to  the  origina-
               tor.

and

      The  "Reply-To"  field  is added by the message
               originator and is intended to direct replies.

and that is the same concept, in slightly different wording.

> The Reply-To: is a 
> NETWORK control header

First, it is a message header field, not a "NETWORK control header".

> that allows transport system (821, Fidonet)  to work 
> and helps From: to work as a display name

Second, it has nothing to do with transport under any circumstances.
Third, it does nothing to affect the optional display name part of
a From field.

> The list expander can indeed 
> set the Reply-To: as that is where ALL list responses should go to.

No. Reply-To is defined as an originator field (RFC 822:

    4.4.  ORIGINATOR FIELDS

          The standard allows only a subset of the combinations possi-
     ble  with the From, Sender, Reply-To, Resent-From, Resent-Sender,
     and Resent-Reply-To fields.
[...]
     4.4.3.  REPLY-TO / RESENT-REPLY-TO

        This field provides a general  mechanism  for  indicating  any
        mailbox(es)  to which responses are to be sent.  Three typical
        uses for this feature can  be  distinguished.   In  the  first
        case,  the  author(s) may not have regular machine-based mail-
        boxes and therefore wish(es) to indicate an alternate  machine
        address.   In  the  second case, an author may wish additional
        persons to be made aware of, or responsible for,  replies.   A
        somewhat  different  use  may be of some help to "text message
        teleconferencing" groups equipped with automatic  distribution
        services:   include the address of that service in the "Reply-
        To" field of all messages  submitted  to  the  teleconference;
        then  participants  can  "reply"  to conference submissions to
        guarantee the correct distribution of any submission of  their
        own.

and RFC 2822:

   3.6.2. Originator fields

      The originator fields of a message consist of the from field, the
      sender field (when applicable), and optionally the reply-to field.
[...]
      The originator fields also provide the information required when
      replying to a message.  When the "Reply-To:" field is present, it
      indicates the mailbox(es) to which the author of the message suggests
      that replies be sent.

); it is supposed to be set by the author/originator, not by a list
expander.  Specific fields for use by list expanders are defined in RFCs
2369 and 2919.  Note that in the quoted text above, RFC 822
specifically addressed use of Reply-To by author/originators when
submitting messages *to* mailing list expanders ("'text message
teleconferencing' groups"), as did its predecessors.

> But to keep the Reply-TO: as the
> original author reply address as it was still a pure EMAIL 1 to 1 transport
> concept, then maybe this it explains why this debate exist.

The Reply-To field indicates where the author suggests that
responses should be sent, for a variety of purposes including
mailing lists.  The Reply-To field does *not* indicate "the original
author reply address"; the author's mailbox is indicated by the
message header "From" field, which may be used by a respondent
for direct responses to the author if that's where a respondent
wishes to send his response, whether or not the original message
was sent via a list expander.

> Yet, there is a mailing list/email/MUA conflict, you wish to continue to
> suggest there isn't.

The only "conflict" is with specific UA implementations, viz. those
that fail to display the Reply-To field and/or fail to provide a way
for an author to set the Reply-To field on messages in accordance
with the "three typical uses" specified in RFC 822 (and other uses).

> Bruce, you reread what I was saying. 

It would help if you could put together a few coherent sentences...

> > > But the MUA does not.
> >
> > *Which* MUA? That's an implementation issue.
> 
> It is an implementation issues across the board.

No.  There are in fact UAs -- *many* UAs -- that display all relevant
originator fields and which provide mechanisms by which an author
can set the Reply-To field on outgoing messages. Several, including
the one I'm using right now, preset Reply-To in accordance with
some configuration, so that it is automatically set appropriately for
most messages sent to a mailing list (and can be easily edited when
necessary).

> If you follow this LIST format,  your natural REPLY will not be to the LIST

No, as RFC 822 has indicated, if an author sets the Reply-To field to
point to the mailbox associated with the list expander service on
messages sent to a list for which he expects responses to be likewise
sent to that list, a typical reply (as opposed to specific action taken
by a respondent to respond to some other place) will in fact be sent
to the list expander; that is the entire point of that intended use of
the Reply-To field.

> You need to "Reply to All"

Only if the author -- ignoring RFC 822 and its predecessors -- has
failed to set the Reply-To field appropriately, or if the respondent
wishes to do something very much out of the ordinary.

> and for  
> the layman, he has to make a guess if he doing this right.

If he intends to send a response to where he believes the author
would like responses to be sent, and that author has not explicitly
indicated such a preference by setting a Reply-To field -- or if he
knows or believes that some misconfigured list expander has
overwritten the author's Reply-To field -- then yes, he is reduced
to having to make a guess.  Common courtesy indicates that
authors should therefore explicitly indicate preferences using
the standard Reply-To field which is provided specifically for that
purpose, and that under no circumstances should a list expander
overwrite that field; so as to avoid forcing respondents to hazard
guesses.

> How many times 
> have you received a private response from someone on a list where in
> reality, he thought it was going to the list?

I have no way of knowing what a respondent thought.  I have rarely
received private responses to list messages (because I *do* set Reply-To
in accordance with RFC 822), and on the occasions when I do receive
private responses it is generally because the respondent intentionally
wished to send an off-list message (and they usually say so explicitly
in those private responses).  And I would add that with Reply-To set
by the author to point to the list mailbox, it would require a great
deal of confusion for  a respondent to send a private response while
thinking it was going to the list; said respondent would have to use a
specific "reply to author" function or manually edit the recipient list in
order to do so, as the protocol for a generic "reply" is to use the Reply-To
field.

> I see it quite often

Then that's probably because *you* failed to set the Reply-To field
to point to the list (as you have repeatedly failed to do when sending
to this list).

> > Then you still don't understand the difference between "sender"
> > and "author".
> 
> if you honestly believe that, and I know you don't

You know no such thing.  If you mean author, then write "author";
if you write "sender" when you evidently mean author, it is hardly
surprising to find that others believe that you fail to understand the
difference.

> > > Hhmmm, sure, for creating the reply address in the new/response message.
> >
> > In *all* messages.
> 
> Oh, we can be here all day Bruce.  It for EMAIL 1 to 1 DIRECT only!

No, see RFC 822 text quoted above regarding mailing lists ("'text message
teleconference' groups").

> > Briefly, there is no certain way for an MUA to determine that a mailing
> > list is involved.
> 
> And this is what we are trying to address. Yet you don't even want to go
> there.

It can't be done. While there are specific (RFC 2369, 2919) message
header fields that a mailing list expander MAY insert, and while a
mailing list expander SHOULD appropriately set the envelope sender
address to the mailing list maintainer's mailbox, there is no guaranteed
way that any MUA can always identify all list messages.  That is because
the mailing list-specific fields are optional and because a mailing list
mailbox is indistinguishable from an individual mailbox.  Mailing list-
specific fields cannot be made mandatory under IETF rules, because
imposing requirements is reserved for cases where it is necessary
to ensure interoperation and to avoid network damage (such as
severe congestion caused by retransmissions); since the list-specific
fields play no role in protocol but are merely informative, there is no
way that omitting any or all of those fields can lead to such problems.

Moreover, there is no need for an MUA to determine that a list is
involved; an author is supposed to set the Reply-To field to indicate
where responses should go if that is somewhere other than directly
to him (the author), and a respondent only needs to be able to see
the author's recommendation and to determine whether he (the
respondent) wishes to comply with that recommendation or to send
his response elsewhere.  That applies for *all* messages, mailing list
or otherwise.

> > > I think most List Servers do add some information to pick up on this.  I
> > > think List-Address: is a good candidate.
> >
> > There is no such field.
> 
> Then INVENT it god-damn it!

No need. It would serve no useful purpose; it could not be mandatory.
There are already informative optional fields for mailing list use. There
is already a standard field that an author can use to indicate a
suggested set of mailboxes for responses to a message, whether or
not that message may have passed through a list expander en route
to its recipient(s).
 
> > > > Probably not; there should be a function to respond to where
> > > > the author has requested responses to go.  Generally that is
> > > > labeled simply "Reply" in UAs.
> > >
> > > Yes, but for a direct 1 to 1 EMAIL concept only.
> >
> > No, for general use.
> 
> No, for 1 to 1 private email only!  sorry you can't see that.

You are wrong. Again I direct your attention to the specific text in
RFC 822 that addresses use of Reply-To with mailing lists.

> Yet, we have 
> this debate going on and you act like the MUA can solve this problem on its
> own.

No, I've clearly stated that the author of each message is responsible
for setting the Reply-To field appropriately.  I have not said anything
remotely like "the MUA can solve this problem on its own"; in fact I
have pointed out that there is very little that an MUA *can* do -- at
best it can assist the author in setting Reply-To, and it should display
relevant information to message recipients.

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From: Martin Trautmann <[email-masked]>
Subject: Re: Locating forwarded messages
Date: Sat, 16 Oct 2004 16:32:55 +0200
To: [email-masked]
Sender: [email-masked]

On 2004-Oct-05, at 19:13, Grant Baillie wrote:

> What header(s) should the new message contain to identify it as being 
> a forwarded version of the original? This would be useful, say, so 
> that clients can provide a way for users to locate any forwarded 
> message(s) (or go from the forwarded message to the original).

I'm very happy about a simple solution:

- References are required: especially if a certain range of people may 
have missed one of the levels, a threading by one level as In-Reply-To 
is not as good as a full chain within references

- Apart from threading, only a simple 'Fwd: ' subject prefix is 
required to mark a message as forwarded

>

Cc: IETF-822 IETF-822 <[email-masked]>
From: Martin Trautmann <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Sat, 16 Oct 2004 16:31:42 +0200
Sender: [email-masked]

On 2004-Oct-13, at 13:51, Charles Lindsey wrote:

> You have mentioned transcriptions which MIGHT occur. You mentioned no
> examples which HAD occurred, nor any current software which could cause
> them to occur.

> I am aware of no such software, and doubt that any such software 
> exists.

I feel it's a flawed understandig of what a standard SHOULD be: It's 
not a summary of what the old and known software actually does. It 
should describe the best way what should be done how. A standard never 
is up-to-date as long as improvements are possible. A standard is never 
complete since you can't (and should not) describe every possible risk.

But it's a bad idea just to ignore something that may become a real 
problem: Defining what a right-hand-part actuallly is and how it has to 
be formatted seems to be a reasonable idea.

I'd wish someone would have done the simple thing to define e.g. "Re: " 
for replies. Up to then most software should have been well behaved, 
using "Re:" only. But since that time every software engineer seems to 
be eager to create his own local language option here...

Cc: IETF-822 IETF-822 <[email-masked]>
From: Martin Trautmann <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Sat, 16 Oct 2004 16:31:09 +0200
Sender: [email-masked]

On 2004-Oct-08, at 15:07, Hector Santos wrote:
>> Have you heard of any case where this esoteric change of semantics
>> actually caused a problem?
>>
>> IOW, is this a theoretical curiosity, or a cause of real problems in
>> meatspace?

I don't see any need to weaken an established standard jut because you 
don't expect any malfunction up to now.

> Well,  traditionally, for the few networks I've been involved with, the
> <id-right> has always been the domain and besides for tracking and
> threading,  it might be used for dupe processing at the original domain
> source.

Taking my usenet access, this is composed from
   [email-masked]

I don't feel to be a domain by myself - but who knows.

So there's at least a part of the RHS which is not a domain.

> PS: There should not be a Newsgroups: header when sending your email.
> Atleast this MUA is thinking it is a newsgroup article when replying.

On the other hand you HAVE to set Newsgroups when you do e.g. a 
followup while you send a Cc per email.

Taking just the Newsgroup as prior identifier seems to be not 
reasonable. If you have both email and usenet headers, you should use 
both. What's wrong about the backend to handle postings and email? They 
are more identical than different. On the other hand - what should a 
MUA do about usenet at all?

Date: 16 Oct 2004 03:29:04 -0000
Mail-Followup-To: [email-masked]
Automatic-Legal-Notices: See http://cr.yp.to/mailcopyright.html.
From: "D. J. Bernstein" <[email-masked]>
To: [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

Hector Santos writes:
> I am seriously thinking of adding support for Mail-Followup-To:  if I
> can understand it better.

See http://cr.yp.to/proto/replyto.html.

The easy part is using Mail-Reply-To and Mail-Followup-To for incoming
messages. You don't need any changes to the normal two-button interface,
one button for quiet responses and one button for loud responses. You
simply have the buttons pay attention to some extra fields:

   Quiet responses, old behavior:
     Reply-To, or From if Reply-To isn't set
   Quiet responses, new behavior:
     Mail-Reply-To, or the old behavior if Mail-Followup-To isn't set
   Loud responses, old behavior:
     To, plus Cc, plus the quiet-response list
   Loud responses, new behavior:
     Mail-Followup-To, or the old behavior if Mail-Followup-To isn't set

The harder part is generating Mail-Followup-To for outgoing messages---
the expert user will want Mail-Followup-To automatically generated for
any message that include certain magical addresses in To+Cc. (Typically
the magical addresses are the mailing lists that the user has subscribed
to.) You need to let the user configure those addresses.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

From: "Hector Santos" <[email-masked]>
To: "ietf-822" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Fri, 15 Oct 2004 21:12:35 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Bruce Lilly" <[email-masked]>

> > No, I think I am pretty verse of the MUA "standard practice" [...]
>
> A response to the person or people identified in a message's
> From field is a response to the author(s), not necessarily the
> "sender".  See RFC 2822 section 3.6.2.

Why are you indicating the obvious?

> > I mean, we are trying to get the MUA to do something it was simply
> > not trained to do.
>
> MUAs aren't "trained".

Bruce,  does this suggest not a software engineer or developer or even close
to understanding the concept?  If so, I won't hold that against you thought.

Do you have more significant comments other than these no-win responses?

> > If sent to the list, it list server has a big role here.
>
> From the moment a respondent decides to prepare a response and
> until he sends it, there is no role for any list expander; it is simply
> not involved.

Again,  once that message hits the LIST manager, it is going to be "handled"
how it sees fit whether you like it or not!  The list expander has a vital
role in it.

> > I explained how
> > the list server used here seems to not do this, but our list server will
> > make sure that the proper  Reply-To: is used for the distribution.
There is
> > nothing technically wrong with that
>
> It *is* wrong for anybody other than a message author to set
> a Reply-To field. See RFC 2822 section 3.6.2.

Not quite. Thats an interpretation for 2822 from an END-USER such as
yourself with no experience in mail gateway, distribution product design.
In fact RFC 822 has no special "author request" phrase.  The Reply-To: is a
NETWORK control header that allows transport system (821, Fidonet)  to work
and helps From: to work as a display name   In addition, once again, we are
talking about a reply address is the LIST address. You are muddying the
concept of EMAIL and GROUPWARE distribution.  The list expander can indeed
set the Reply-To: as that is where ALL list responses should go to. Not
Direct. List Responses.  If you want to go outside that automated list
response, then you use something else.  But to keep the Reply-TO: as the
original author reply address as it was still a pure EMAIL 1 to 1 transport
concept, then maybe this it explains why this debate exist.

> Not. Reread what I wrote in its entirety. Current and past RFCs have
> always taken mailing lists into account.

Yet, there is a mailing list/email/MUA conflict, you wish to continue to
suggest there isn't. Bruce, you reread what I was saying.

> > But the MUA does not.
>
> *Which* MUA? That's an implementation issue.

It is an implementation issues across the board. Again, why does this debate
exist?

If you follow this LIST format,  your natural REPLY will not be to the LIST
which I will venture to believe the majority of people will believe should
not be the _default_ expected behavior.   You need to "Reply to All" and for
the layman, he has to make a guess if he doing this right.    How many times
have you received a private response from someone on a list where in
reality, he thought it was going to the list?  I see it quite often until he
finally understands whats going on and how to use his MUA for one or more
different list systems.  Worst, you see much redundant because he takes the
fields that are set and got mail going all over the place.  So which is it?

> > I beg to differ.
>
> Then you still don't understand the difference between "sender"
> and "author".

if you honestly believe that, and I know you don't, then I should be
stopping right here.  But I'm having fun with you.  <g>  You are no match
for me bud!  But hey, I can play dumb and try to work with people to solve a
problem.  Can you?

> > Hhmmm, sure, for creating the reply address in the new/response message.
>
> In *all* messages.

Oh, we can be here all day Bruce.  It for EMAIL 1 to 1 DIRECT only!  Sorry,
your MUA is seeing this message as a EMAIL and nothing else.  If you want to
make it for a LIST, you have to do a few extra steps.  Say it ain't so Joe?

> > Why?  Please give a reason.  I mean, I would love to put my trust in
your
> > expertise. :-)  But I need a reason why this will not "help" in
addressing
> > the issue?
>
> Please read the past discussion, where this has been addressed in detail.

All I see you and one other that I won't mention is constantly knocking
everything and anyone who have anything to say.

> Briefly, there is no certain way for an MUA to determine that a mailing
> list is involved.

And this is what we are trying to address. Yet you don't even want to go
there.  Hmmm, incidentally, why do we need your permission again?

> > I think most List Servers do add some information to pick up on this.  I
> > think List-Address: is a good candidate.
>
> There is no such field.

Then INVENT it god-damn it!

> > > Probably not; there should be a function to respond to where
> > > the author has requested responses to go.  Generally that is
> > > labeled simply "Reply" in UAs.
> >
> > Yes, but for a direct 1 to 1 EMAIL concept only.
>
> No, for general use.

No, for 1 to 1 private email only!  sorry you can't see that.  Yet, we have
this debate going on and you act like the MUA can solve this problem on its
own.  Oye Vey!  It can't.  If it was so, it would of been done by know.
Sure, you can make it smarter, but not by default and there will need to be
some some of standardization in the list server area to make it work better.

This was a mistake to get involved here.  I must be bored!

ciao

--
Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>, [email-masked]
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Date: Fri, 15 Oct 2004 19:45:37 -0400
Sender: [email-masked]

On Tue October 12 2004 23:25, Laird Breyer wrote:

> What are you talking about? I'm very confused by this statement.
> 
> I'm talking about a mailing list server which has an internal list of
> subscriber addresses, and any message sent to the list address is
> distributed to each of the subscribers. If some of the subscribers
> have indicated special preferences for how they would like to receive
> *their* copies of the messages sent by others to the list, then the
> server can perform whatever action is preferred on *their* copies of
> the messages, based on both the message content and the archive of
> past messages if needed.

Standard operating procedure for list expanders.

> The fact that such action can depend on  
> message content and past archive is what makes the server "smart".

No effect on the above.
 
> The other members of the list don't need to know anything at all, except
> whether they want to send a particular message to the list address. Once
> it hits the list address, they have no more say in where it goes or what
> happens to that copy, except for their own personal copy delivered to them
> as list members. 

Agreed.  But your earlier "render inadequate the default behaviour of 
sending list responses to the address that was subscribed" makes
little or no sense; responses to list messages are sent by some respondent
to wherever he wishes to send them, not "to the address that was
subscribed".

> If all the MUAs in the world followed
> all applicable standards consistently, we probably wouldn't be having this
> conversation in the first place.

We're having the discussion primarily because some authors fail to
indicate where they wish responses to be sent, yet expect everybody
else to be able to read their minds regarding those preferences. That's
not an MUA issue, it's a human issue (and in fact many, if not most,
MUAs have all of the necessary functionality, and the protocols
certainly do).

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Fri, 15 Oct 2004 19:35:57 -0400
Cc: "Hector Santos" <[email-masked]>
Sender: [email-masked]

On Fri October 15 2004 15:36, Hector Santos wrote:
> 
> ----- Original Message -----
> From: "Bruce Lilly" <[email-masked]>
> 
> 
> > On Tue October 12 2004 23:04, Hector Santos wrote:
> >
> > >         "Reply to Sender"
> >
> > You are in danger of confusing yourself (or of being
> > confused by non-standard terminology used by your
> > MUA); you probably mean "Author(s)", though it is
> > conceivable that some respondents might wish to
> > communicate with a message sender (as indicated
> > by the Sender field if present, or via the envelope
> > sender).
> 
> No, I think I am pretty verse of the MUA "standard practice" [...]

A response to the person or people identified in a message's
>From field is a response to the author(s), not necessarily the
"sender".  See RFC 2822 section 3.6.2.

> I mean, we are trying to get the MUA to do something it was simply
> not trained to do. 

MUAs aren't "trained".

> > >         "Reply to All"
> > >
> > > and how these two MUA "standard" reply actions are handled or "prepared"
> by
> > > list handlers.
> >
> > They aren't; responses are generated by a respondent,
> > usually with an MUA -- the mailing list expander is not
> > involved in that process.
> 
> If sent to the list, it list server has a big role here.

>From the moment a respondent decides to prepare a response and
until he sends it, there is no role for any list expander; it is simply
not involved.

> I explained how 
> the list server used here seems to not do this, but our list server will
> make sure that the proper  Reply-To: is used for the distribution.  There is
> nothing technically wrong with that

It *is* wrong for anybody other than a message author to set
a Reply-To field. See RFC 2822 section 3.6.2.

> > Incorrect. In fact mailing lists predate email, initial.....
> 
> and I am sure there was a time where engraved flintstones were used to
> communicate. :-) This doesn't change the fact the current RFC-based mail
> readers are what they are today based on an EMAIL and NEWS design. Not
> MailIng List concepts.  Is this correct or not?

Not. Reread what I wrote in its entirety. Current and past RFCs have
always taken mailing lists into account.

> 
> > ....SMTP itself has explicit provisions for
> > mailing lists.
> 
> But the MUA does not.

*Which* MUA? That's an implementation issue.

> > > In one case (the list server used here), the "Reply to Sender" will go
> to
> > > the author.  In our list server, the "Reply to Sender" goes to the list
> > > address.
> >
> > There's that problem with use of non-standard terminology.
> 
> I beg to differ.

Then you still don't understand the difference between "sender"
and "author".

> > > I guess the difference is that we make sure the "Reply-To:" address is
> > > properly set in the distribution.
> > >
> > > Is that more correct?
> >
> > No. Its use is reserved for the author to indicate where he
> > wishes responses to be sent.
> 
> Hhmmm, sure, for creating the reply address in the new/response message.

In *all* messages.

> > > a) Does a MUA need to detect a Mailing List Message as oppose to a just
> a
> > > pure email message?
> >
> > No, and in general it cannot do so.
> 
> Why?  Please give a reason.  I mean, I would love to put my trust in your
> expertise. :-)  But I need a reason why this will not "help" in addressing
> the issue?

Please read the past discussion, where this has been addressed in detail.
Briefly, there is no certain way for an MUA to determine that a mailing
list is involved.

> I think most List Servers do add some information to pick up on this.  I
> think List-Address: is a good candidate.

There is no such field.

> > Probably not; there should be a function to respond to where
> > the author has requested responses to go.  Generally that is
> > labeled simply "Reply" in UAs.
> 
> Yes, but for a direct 1 to 1 EMAIL concept only.

No, for general use.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Understanding response protocols
Date: Fri, 15 Oct 2004 16:43:52 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "D. J. Bernstein" <[email-masked]>
To: <[email-masked]>

> Keith Moore writes:
> > Actually some people are proposing blind acceptance.  They want MFT to
> > be honored automagically, regardless of the content of the reply.
>
> No. Mail-Followup-To does _not_ override the wishes of the user sending
> the followup. It is simply a _default_ recipient list. The old de-facto
> standard was
>
>    To+Cc+(Reply-To||From)
>
> as the _default_ recipient list; the replacement is
>
>    Mail-Followup-To||(To+Cc+(Reply-To||From))
>
> as the _default_ recipient list. The user is free to edit the list if he
> wants something different---but we choose _defaults_ to save time for
> the user in the common case that he's doing something predictable.
>
> You say that some MUAs have bad list-editing interfaces: in particular,
> they don't make the recipient list clear to the user. That's certainly
> something to fix---no matter what the default recipient list is! You
> keep bringing this up in a Mail-Followup-To context as if it were a new
> problem; but the same problem existed long before Mail-Followup-To.

I may add there is also a consideration for a MUA where the reply fields are
not "editable," such as for online interactions.

D.J,

I am seriously thinking of adding support for Mail-Followup-To:  if I can
understand it better.

Above you seem to agree that performing the default "common case behavior is
desirable. If so, I agree.

Currently a "Reply to Sender" goes to you.  To respond to this list, I have
to to a "Reply to All."  I guess may make sense.  But here I have to remove
your address and I only "consciously" did so because I seem to recall an
message here from you where you indicated you don't want the direct mail
from a list.  Maybe it was someone else, but in any case, my questions are:

Does MFT apply only to a mailing list?

If any, how will a MUA support MFT?  How it is ergonomically offered to the
user?

    Reply To List? -->  Mail-Followup-To: | Reply-To: | From:
    Reply To Sender? --> Reply-To: | From:
    Reply To All? --> To+Cc+(Reply-To||From)

    Follow-up? -->  Mail-Followup-To: | Reply-To: | From:

If you added the Mail-Followup-To:   then why not add the Reply-To:?

Thanks

---
Hector Santos, CTO
WINSERVER "Wildcat! Interactive Net Server"
http://www.santronics.com

Date: 15 Oct 2004 20:32:18 -0000
Mail-Followup-To: [email-masked]
Automatic-Legal-Notices: See http://cr.yp.to/mailcopyright.html.
From: "D. J. Bernstein" <[email-masked]>
To: [email-masked]
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Sender: [email-masked]

Charles Lindsey writes:
> So the "Reply(-to-Sender)" button sends it to the Reply-To address,
> and the "Reply-To-List" button sends it to the MFT address.

Yes, those are the basic two buttons, but you're probably confusing some
people here by using the name ``reply to list'' for the second button;
there's a widespread MUA with a third button labelled ``list reply,''
quite different from the second button.

(The third button screws up crossposts, and it fails to send Keith the
cc's that he wants. So why was it added? Because the second button,
_without_ Mail-Followup-To, was even worse. On the other hand, in a
Mail-Followup-To world, the second button works fine, and the third
button should be eliminated.)

USENET has consistently used the name ``reply'' for the first button and
the name ``follow up'' for the second button. Why invent new names?

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

Date: 15 Oct 2004 20:07:43 -0000
Mail-Followup-To: [email-masked]
Automatic-Legal-Notices: See http://cr.yp.to/mailcopyright.html.
From: "D. J. Bernstein" <[email-masked]>
To: [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

Keith Moore writes:
> Actually some people are proposing blind acceptance.  They want MFT to 
> be honored automagically, regardless of the content of the reply.

No. Mail-Followup-To does _not_ override the wishes of the user sending
the followup. It is simply a _default_ recipient list. The old de-facto
standard was

   To+Cc+(Reply-To||From)

as the _default_ recipient list; the replacement is

   Mail-Followup-To||(To+Cc+(Reply-To||From))

as the _default_ recipient list. The user is free to edit the list if he
wants something different---but we choose _defaults_ to save time for
the user in the common case that he's doing something predictable.

You say that some MUAs have bad list-editing interfaces: in particular,
they don't make the recipient list clear to the user. That's certainly
something to fix---no matter what the default recipient list is! You
keep bringing this up in a Mail-Followup-To context as if it were a new
problem; but the same problem existed long before Mail-Followup-To.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

From: "Hector Santos" <[email-masked]>
To: "ietf-822" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Fri, 15 Oct 2004 15:36:15 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Bruce Lilly" <[email-masked]>

> On Tue October 12 2004 23:04, Hector Santos wrote:
>
> >         "Reply to Sender"
>
> You are in danger of confusing yourself (or of being
> confused by non-standard terminology used by your
> MUA); you probably mean "Author(s)", though it is
> conceivable that some respondents might wish to
> communicate with a message sender (as indicated
> by the Sender field if present, or via the envelope
> sender).

No, I think I am pretty verse of the MUA "standard practice"  having written
few myself, following consistency in competitive products, and also from the
host side making sure, the support for all of them is consistent following
the standard and BCP. Sure, on a personal level, its so old, that one gets
too confortable to the point I am not even sure that the debate is all
about.  I mean, we are trying to get the MUA to do something it was simply
not trained to do.  You have Online Readers and Offline readers.  Which one
are we concern about?  How about those with "exchange-like" modules?

Independent of the actually "UI" terminology used, I am pretty confident the
"standard" RFC based MUA has the following two basic supports:

EMAIL
    Reply to Sender -> Offline Direct to Sender
    Reply to All -> Offline Direct to "List" of users.

NEWS
    Reply to Group   -> Online Post to Group
    Reply to Sender  -> Offline Direct to Sender

Anything beyond this common expectation?  That is how we have to make sure
our backends, gateways, expanders must work to support the common
expectation of the RFC-based reader MUA.   We are not talking about IMAP or
Exchange or Newsgroup Emulators or are we?

> >         "Reply to All"
> >
> > and how these two MUA "standard" reply actions are handled or "prepared"
by
> > list handlers.
>
> They aren't; responses are generated by a respondent,
> usually with an MUA -- the mailing list expander is not
> involved in that process.

If sent to the list, it list server has a big role here.  I explained how
the list server used here seems to not do this, but our list server will
make sure that the proper  Reply-To: is used for the distribution.  There is
nothing technically wrong with that but it does have an effect on the MUA
for off-list communications.  The MUA can be smarter there as you pointed
out.

> > So to help facilate the MUA, the list handler
> > should prepare the headers to best provide the most meaningful or
correct
> > behavior as expected by the user.
>
> Message headers' fields set by authors or whose
> semantics are reserved for authors' use should not
> be modified by list expanders.

I don't agree when it comes to the network control headers.  I think the
backend needs to make that the input is correct and output is correct.  Of
course, I am not talking about mail tampering. I am talking about a
groupware concept - mailing list.

> > The problem as I see it is that we have a standard for 1-1 (direct)
email
> > messages, but no standard for a list distribution to handle the above
two
> > reply options.
>
> Incorrect. In fact mailing lists predate email, initial.....

and I am sure there was a time where engraved flintstones were used to
communicate. :-) This doesn't change the fact the current RFC-based mail
readers are what they are today based on an EMAIL and NEWS design. Not
MailIng List concepts.  Is this correct or not?  In general, groupware is
currently done using something like IMAP, Exchange Interfaces or NewsGroups
Emulation and in a growing large part - a return to online hosting
communications.  No?  I think so.

> ....SMTP itself has explicit provisions for
> mailing lists.

But the MUA does not. Hence this debate?

> > In one case (the list server used here), the "Reply to Sender" will go
to
> > the author.  In our list server, the "Reply to Sender" goes to the list
> > address.
>
> There's that problem with use of non-standard terminology.

I beg to differ. Again, independent of the exact usage of the CUI (Common
User Interface) language used to expose the various reply actions,  it is
generally what I outlined above.  I don't see how you can see it is not
simply because there is no other technology that the common/standard
RFC-Based reader MUA was trained to work with.  That is basically the point
I am making.

In essence, we are trying to deal with a mailing list as it was really a
"newgroup" per se.  In fact, I find the handling of a mailing list is more
suitable using a Newsgroup emulator for the reasons of providing a more
natural "reply to group" and "reply to sender" actions.  The only think
missing would be a "reply to all."

> > I guess the difference is that we make sure the "Reply-To:" address is
> > properly set in the distribution.
> >
> > Is that more correct?
>
> No. Its use is reserved for the author to indicate where he
> wishes responses to be sent.

Hhmmm, sure, for creating the reply address in the new/response message. But
the Target (TO:) is taking from the Reply-To: in the original message.

Again,  that is the standard behavior of the MUA.  Maybe this where a
Reply-To is changed to MFT before adding the Reply-To: list address?

> [more problems based on non-standard terminology elided]
> > Unfortunately, atleast with Outlook the "Reply to All" does not grab
> > the "From:" address.
>
> Sure it does -- the problem is that in your case something
> has overridden it by setting the Reply-To field.

Ok, good point.

> > In this regard, I would say our list server makes it more difficult to
send
> > direct.  The user has to understand whats going on and copy and paste
the
> > From: address from the message properties.
>
> That's a problem with that specific MUA.

I still say there is a conflict either way you look at it.  A list expander
adding the Reply-To: list address is more technically correct in my view.
The point is the MUA is not trained to understand a "mailing list."

> > a) Does a MUA need to detect a Mailing List Message as oppose to a just
a
> > pure email message?
>
> No, and in general it cannot do so.

Why?  Please give a reason.  I mean, I would love to put my trust in your
expertise. :-)  But I need a reason why this will not "help" in addressing
the issue?

> > c) What are the most desirable reply/new mail options for a mailing
list?
> >
> >     Reply to List (very similar to a news Reply To Group idea)?
>
> There is no general mechanism to identify a list, messages
> may be sent to multiple lists, etc.

I think most List Servers do add some information to pick up on this.  I
think List-Address: is a good candidate.   No?  For multiple list, you would
add the list address(es) in the single header  just like its done for
Newsgroups:.

        List-Address:  address1, address2,  etc

Will that work?

> >     Reply to Directly to Author?
>
> Probably yes -- many UAs do provide that functionality.
>
> >     Reply to All?
>
> Probably yes as a catch-all.
>
> > Is that enough?
>
> Probably not; there should be a function to respond to where
> the author has requested responses to go.  Generally that is
> labeled simply "Reply" in UAs.

Yes, but for a direct 1 to 1 EMAIL concept only.  That is the issue as I
see.   The MUA is going to see the output of a list as "email" and thus
behave as it was a 1 to 1 EMAIL communication. Hence the "reply" as you say,
should go to the requested address.   The issue I think is depending on the
list expander, it can go to the Author or it can go to the List.  What is
better based on current MUA practice?

> > How about follow ups?
> [...]
> > Does that make sense?
>
> No. How did you intend "follow up" to differ from "reply"
> (which in any case would be more accurately be called
> "respond")?

It depends. Follow up has been designed different ways.  I mean consider the
design that in some cases only the original author can follow up on his own
mail.  :-)

Thanks for your comments Bruce. I hope it can lead to something, rather than
nothing.

---
Hector Santos, CTO
WINSERVER "Wildcat! Interactive Net Server"
http://www.santronics.com

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Fri, 15 Oct 2004 10:52:30 -0400
Cc: "Hector Santos" <[email-masked]>
Sender: [email-masked]

On Tue October 12 2004 23:04, Hector Santos wrote:

>         "Reply to Sender"

You are in danger of confusing yourself (or of being
confused by non-standard terminology used by your
MUA); you probably mean "Author(s)", though it is
conceivable that some respondents might wish to
communicate with a message sender (as indicated
by the Sender field if present, or via the envelope
sender).

>         "Reply to All"
> 
> and how these two MUA "standard" reply actions are handled or "prepared" by
> list handlers.

They aren't; responses are generated by a respondent,
usually with an MUA -- the mailing list expander is not
involved in that process.
 
> I mean, lets take a step back and understand the the list concept is still
> based on a email concept.

OK so far, if we don't get literal.

> So to help facilate the MUA, the list handler 
> should prepare the headers to best provide the most meaningful or correct
> behavior as expected by the user.

Message headers' fields set by authors or whose
semantics are reserved for authors' use should not
be modified by list expanders.

> The problem as I see it is that we have a standard for 1-1 (direct) email
> messages, but no standard for a list distribution to handle the above two
> reply options.

Incorrect. In fact mailing lists predate email, initial
electronic equivalents of mailing lists (using FTP MAIL)
predate MTP and SMTP, and the text message format
that was developed in those days and which has
evolved slightly to its current format explicitly provides
for mailing lists; it has done so since RFC 724 (possibly
earlier).  The format began with RFC 561 more than
three decades ago , specifically to address issues with
mailing lists.  SMTP itself has explicit provisions for
mailing lists.

> In one case (the list server used here), the "Reply to Sender" will go to
> the author.  In our list server, the "Reply to Sender" goes to the list
> address.

There's that problem with use of non-standard terminology.
 
> I guess the difference is that we make sure the "Reply-To:" address is
> properly set in the distribution.
> 
> Is that more correct?

No. Its use is reserved for the author to indicate where he
wishes responses to be sent.
 
[more problems based on non-standard terminology elided]
> Unfortunately, atleast with Outlook the "Reply to All" does not grab 
> the "From:" address.

Sure it does -- the problem is that in your case something
has overridden it by setting the Reply-To field.

> In this regard, I would say our list server makes it more difficult to send
> direct.  The user has to understand whats going on and copy and paste the
> From: address from the message properties.

That's a problem with that specific MUA.
 
> So I think maybe it may help us to get to a point where we might want to
> outline design requirements for a modern MUA  to better understands a
> mailing list?

There isn't much specific to lists; certainly some UAs
are poorly implemented (incorrect terminology, failure to
provide convenient access to message content, failure to
provide some functionality, etc.). Those implementation
issues affect usability for all messages.

> a) Does a MUA need to detect a Mailing List Message as oppose to a just a
> pure email message?

No, and in general it cannot do so.
 
> c) What are the most desirable reply/new mail options for a mailing list?
> 
>     Reply to List (very similar to a news Reply To Group idea)?

There is no general mechanism to identify a list, messages
may be sent to multiple lists, etc.

>     Reply to Directly to Author?

Probably yes -- many UAs do provide that functionality.

>     Reply to All?

Probably yes as a catch-all.
 
> Is that enough?

Probably not; there should be a function to respond to where
the author has requested responses to go.  Generally that is
labeled simply "Reply" in UAs.

> How about follow ups? 
[...]
> Does that make sense?

No. How did you intend "follow up" to differ from "reply"
(which in any case would be more accurately be called
"respond")?

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>, Nick Ing-Simmons <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Fri, 15 Oct 2004 09:13:10 -0400
Sender: [email-masked]

On Wed October 13 2004 04:40, Nick Ing-Simmons wrote:

> Or transcoding into morse perhaps? 

If you like.
 
> But these, as you say elsewhere, are "non text fields".
> I don't think it would ever make sense for mail _transport_ to 
> work by converting headers to voice and back. 

Since these are *message* and *content* (i.e. media)
identifiers, "mail transport" isn't the only situation that
applies.  There are literally hundreds of media types, a
dozen message subtypes, many non-mail methods of
transporting content (message or other types), as well
as non-transport applications for comparison (e.g.
"search this folder for all messages with message
identifiers containing the top-level domain NET
(case-insensitive) or any subdomain under it").

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>, Nick Ing-Simmons <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Fri, 15 Oct 2004 09:32:24 -0400
Sender: [email-masked]

On Wed October 13 2004 04:29, Nick Ing-Simmons wrote:
> 
> Bruce Lilly <[email-masked]> writes:
> >
> >Evidently you fail to grasp fundamental logic, the nature of the
> >problem with RFC 2822, and the nature of software development.
> 
> If we are going to be patronising

No "patronising": simply pointing out (repeated) flaws
in Mr. Lindsey's "reasoning" (using the term loosely).

> you have failed to grasp the  
> nature of e-mail communication in the real world.

The issue isn't limited to email.
 
> In the real world we are lucky if mail tools even attempt 
> to conform to the spirit of an RFC let alone the letter.

Nevertheless we have specifications for how things
are supposed to work, to ensure that conforming
implementations interoperate (and as a side-effect
to be able to identify specifically what is wrong with
a non-conforming, non-interoperable implementation).
The fact that there are lousy implementations does
not itself indicate that the rules should be discarded;
the lousy implementations should be discarded if they
cannot be repaired.

> On the other hand the consequences if using the wrong sort of compare
> for these fields seem to be rather unimportant.

Beware the rule of unintended consequences; such
a poor choice is likely to cause a problem at the
most inopportune moment.
 
> Doing case sensitive compare "seems to work well enough" even when it is a 
> domain name. 

You are certainly entitled to your opinion, just as
a certain well-known headstrong, swaggering dumbass
was entitled to his opinion that dropping bombs "seems
to work well enough" when what was called for (literally)
was merely a little more diligence in determining
that certain alleged objects which were claimed to
have existed in a certain region in fact did not exist.
Others do not necessarily share that "seems to work
well enough" attitude.  Still others may suffer the
(possibly) unintended consequences of such a choice. 

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Fri, 15 Oct 2004 08:17:07 -0400
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

On Wed October 13 2004 10:26, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

> >A trivial case is transcription from spoken or written material,
> >and I can think of at least one individual who habitually
> >transcribes domain names to all upper-case.
> 
> Please can you explain why your friend should ever find it necessary to
> transcribe a message identifier by hand?

Do try to pay attention; an identifier (N.B. the discussion is
about content identifiers as well as message identifiers) may
have been handed to him on a piece of paper, it may have
been spoken (e.g. one two three at eks wye zee dot net (
s/zee/zed if it makes you happy)), it may be entered via
keyboard from one screen display to another device, etc., as
mentioned above.  And I did not say that the individual
was a friend.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "Charles Lindsey" <[email-masked]>
Subject: Re: Fw: Angle brackets surrounding Content-ID
Date: Fri, 15 Oct 2004 06:32:27 -0400
Cc: ietf-822 <[email-masked]>
Sender: [email-masked]

On Wed October 13 2004 10:24, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >Charles Lindsey wrote:
> 
> >> An MUA that chooses whether to followup to Usenet or to reply using email
> >> simply on the basis of whether a Newsgroups-header is present is severely
> >> broken, since it is normal behaviour for a message which is both posted to
> >> Usenet and emailed to individual recipients to include a Newsgroup-header
> >> (how else are recipients to know that both media were involved).
> 
> >Charles, as you did not include a Reply-To field pointing to this list,
> >you forced respondents into using a "reply-to-all" function, and those
> >UAs which support both news and mail (including Netscape/Mozilla,
> >Outlook Express, and Pine, several of the most widely used UAs for either
> >mail or news) include specified newsgroups in such a "reply-to-all".
> 
> It does not surprise me in the least that Outlook Express is broken in
> that respect, but I would have expected Netscape/Mozilla to have known
> better. The only meaning that can be ascribed to the presence of a
> Newsgroups header in an email message is that it signifies a message that
> was both posted and mailed.

Charles, there is simply no way to distinguish "an email message" from
any other message once it's in a folder that a UA is accessing.  You
really do need to at least make an attempt to understand how messaging
works -- it has been explained to you often enough, in words *and* in
pictures.

> A User Agent that uses a "Reply" button to initiate a "Followup" is most
> certainly broken (see www.xs4all.nl/~js/gnksa). It is a sad fact that most
> MUAs which have been adapted to dual use as newsreaders make a pretty poor
> job of their newsreading role.

In the first place, as has been explained to you multiple times, a "reply-to-all"
function was used -- because you failed to provide any recommendation for
responses to the mailing list; there was no "'Reply' button" involved. In the
second place, all instances under discussion have involved mailing lists;
"Followup" has no meaning and there was no "newsreading role" involved.
In the third place, a personal rant page is not a standard.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Fri, 15 Oct 2004 08:58:49 -0400
Cc: "Charles Lindsey" <[email-masked]>, LIST: [email-masked];
Sender: [email-masked]

On Wed October 13 2004 07:51, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >Charles Lindsey wrote:
> >> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >> If it never actually happens in the Real World (TM), then it is not a real
> >> situation. It is merely an interesting quirk of the way RFC 2822 was
> >> written.
> 
> >Such comparisons do in fact occur. Jacob's message which started
> >this discussion was in fact based on exactly that type of
> >comparison scenario.
> 
> No, Jacob's message made no mention of any comparison which HAD occurred
> (and failed). It was purely concerned with a discrepancy between two RFCs,
> and an implementor who wanted to do the "right thing".

... with respect to comparisons; "comparing" and "comparison"
are specifically mentioned in Jacob's original message. Incidentally,
a "scenario" does not necessarily refer to a specific actual
event; consult a dictionary.

> >I have in fact mentioned transcription of various types.
> 
> You have mentioned transcriptions which MIGHT occur. You mentioned no
> examples which HAD occurred, nor any current software which could cause
> them to occur.

Software is not the only source of transcriptions, as I have
mentioned at length.
 
> >I would add that you have failed to provide any concrete example
> >of specific software which generates identifiers referring to
> >multiple distinct entities which are identical on the LHS and differ
> >only in case in the RHS.
> 
> I am aware of no such software, and doubt that any such software exists.

Ergo you know of no reason to deviate from the
long-standing (*decades*) rule that an identifier RHS is a
case-insensitive domain name.
 
> Presumably it makes no specification because it was not considered
> relevant.

Pure conjecture.

> In fact RFC 2822 make no mention of comparison of message 
> identifiers because such comparison does not arise in the course of any
> process that it envisages.

RFC 2822 is not the sole RFC; its scope is the Internet Message
Format. The RFCs specifically mentioned in my response to
Jacob's query do mention comparisons. RFC 2822 does
specifically mention domain names, and the relevant Standards
for domain names (referenced by RFC 2822) do specify
case-independence.

> AFAICS, the only situation where it considers 
> processes involving message identifiers is when it gives instructions for
> constructing a References field from the Message-ID field of some
> presursor, and the instruction is to take the "contents" of that field
> which means, in the absence of wording permitting otherwise, without
> changing the case of anything.

Which might or might not have already been changed by that time.
 
> Likewise I would suppose (I have not checked in detail) that RFC 2821
> gives no authority for changing the Message-ID field in any way during
> transit.

You should have checked; as usual your supposition is dead
wrong. RFC 2821 specifically mentions gateways. Gateways
may transform from Internet Text Message format to/from
some other environment; a cycle through such an environment
(e.g. X.400) may well result in case changes in the domain.

> Thus there seems no circumstance in which a message identifier, once
> generated, can legitimately be changed subsequently,

Wrong on several counts -- incorrect premise.

> in which case it 
> makes not a blind bit of difference whether any comparison is
> case-sensitive or otherwise.

One cannot draw conclusions of any value from an incorrect
premise.

> >...  And the defect in the RFC 2822 specification
> >needs to be corrected consistent with long-standing use, viz.
> >by reaffirming that an identifier RHS is to be interpreted as
> >a case-insensitive domain name.
> 
> If there is nothing broke,

Another incorrect premise.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Fri, 15 Oct 2004 08:09:02 -0400
Cc: Charles Lindsey <[email-masked]>, Keith Moore <[email-masked]>
Sender: [email-masked]

On Wed October 13 2004 09:03, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >Charles Lindsey wrote:
> >> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> >>>>
> >>>>>>Which is why I said "The only place where it *can* be known ...". For
> >>>>>>sure, if the author does not know, then there is even less chance that
> >>>>>>subsequent mailing list expanders would know
> 
> >> No, it is NOT repeated above, since you have snipped it.
> 
> >It's *still* there.
> 
> You can read what's written inside the "..."?

Anybody who has been paying attention to the discussion can remember
what was claimed; others can look at the original.  Moreover the
text with the ellipsis is yours, verbatim -- now kindly apologize for
for your accusatory lie in which you claimed that I "snipped it".

> >That was not your claim; your claim was "The only place where it is
> >known that a crosspost is occurring is at the original sender.
> >Therefore, any information to inform subsequent agents that
> >crossposting is afoot HAS to be placed in the message by the original
> >sender."  Ignoring the fallacious first statement,
> 
> Why is that falacious?

As has already been explained to you, it is fallacious because list
expansion may take place (possibly multiple times) without the
knowledge of the author; you have admitted that fact, yet you
still repeat your fallacious claim.

> You failed to answer my question about how the 
> expander for the mailing list 'foo' could also know that the recipient
> 'bar' was another mailing list.

Incorrect; I answered that on October 11 -- read it again.

> > the second
> >statement is clearly not about an indication of desire or
> >recommendation regarding responses -- Reply-To (and the non-
> >standard MFT, at least according to some claims) indicates a
> >recommendation for responses; it does not "inform subsequent
> >agents that crossposting is afoot".
> 
> Yes it does, because they (in particular the recipients' agents) can now
> see that replies are requested to be sent to both lists.

No, the semantics of Reply-To says nothing about lists or
"crossposting", and a list mailbox looks like any other
mailbox; there is no indication to any recipient or UA that any
mailbox is a list. All that can be known is that the author has
specifically recommended that responses go to some
non-empty set of mailboxes.

> >Authors do not always want responses to go to all lists in all
> >cases where multiple lists are involved (in some cases one list
> >is notified of discussion taking place on a different list, with
> >responses directed to the discussion list), sometimes an author
> >may want personal responses rather than list responses, and as
> >you have been informed quite clearly, not all persons wish to
> >be excluded from the list of recipients receiving responses
> >whether or not some list is involved.
> 
> Yes, of course there are all sorts of exceptions, but it is apparent that
> many, if not most, authors want replies to go to the list in normal
> circumstances.

In "normal circumstances" there may very well be no "list"
involved.  Protocols are not based on wild guesses about
what some authors might or might not want or whether or not
some mailbox is expanded to a list; they provide a mechanism
for authors to explicitly specify recommendations and provide
specified behavior in the absence of such a recommendation.

> If, for particular reasons, they don't want that then they 
> can set Reply-To/MFT somewhere else.

The default for a generic response (as opposed to reply-to-author,
reply-to-all, etc.) in the absence of a specific recommendation (q.v.
Reply-To) is to go to the author(s).  If an author wants responses
to go elsewhere or to additional places, it is the author's'
responsibility to set the Reply-To field accordingly.  When a respondent
wishes to send a response that includes some original message
recipient, that takes place via a reply-to-all function which also
(again, in the absence of a specific recommendation via a Reply-To
field) includes the author. And again, if the original message author
doesn't want that to happen (i.e. copies sent to him as well as to
original message recipients) -- whether an original message recipient
mailbox was expanded to a mailing list or not -- it is the author's
responsibility to provide such a specific recommendation for
responses. Remember that mailing list mailboxes are indistinguishable
from other mailboxes; the default described above (as specified in
the applicable standards) applies whether or not any mailing lists
are involved.

> It is your fixation with the exceptional and unusual cases that is
> blinding you to the need to get the commonly occurring cases working
> smoothly with minimal intervention by users at either end.

No, Charles.  General mechanisms are being discussed w.r.t. a number
of scenarios.  You are the one with a fixation on special cases. And
you seem to fail to understand that the general mechanism also
applies to those special cases; several of which -- including mailing
lists, a.k.a. "text message teleconferencing groups" -- are specifically
mentioned in RFC 822 section 4.4.3 (Reply-To/Resent-Reply-To) [and
prior to that in RFC 733 section IV.A.2.c, and prior to *that* in RFC
724 II.C.1.b.3].  That general mechanism has been in place, with
provision for use with mailing lists as well as other uses, for a quarter
of a century; since before there was a Mail Transfer Protocol (RFC
772) or SMTP (RFC 788).  And as has been analyzed for a variety of
scenarios, that mechanism works very well when it is used. What
*doesn't* work is the assumption made by some authors that
recipients should guess what the author intends for responses --
that has been shown to be a disaster.

> >If an author makes a recommendation for responses to go to "foo"
> >and to "bar", and expansion of "foo" includes "bar" or vice
> >versa, or "foo" and "bar" when expanded both include "baz",
> >then duplicates may still be received somewhere.
> 
> Again, it is your fixation with the exceptional cases that is showing
> through. You seem to prefer that is is better for a system to never get it
> right is there is a 0.001% chance that it will sometimes get it wrong.

Please
a) define precisely what "it" means in "get it right" and "get it wrong"
b) provide hard evidence for that "0.001%" figure.

In any event, you are simply wrong; I have detailed several scenarios
including common and unusual situations.  I have shown that use
of the Reply-To field (as standardized for a quarter of a century)
works quite well; as well as can be expected by any other suggestion
made to date -- indeed I have pointed out the specific limitations
(which apply as well to other suggestions).

> >2. An appropriate question is how response recommendations
> >   can be communicated under various scenarios.
> 
> >Regarding the last point, I have detailed how standard fields
> >expressing standard semantics with standard syntax can be used
> >under various scenarios identified during this discussion
> 
> You have signally failed to convince anyone, even Keith, that the Reply-To
> field provides an acceptable solution in this matter.

I see that you have now stooped (again) to putting words in
others mouths and again have failed to copy the person whom
you purport to speak for, in an unconscionable breach of
etiquette.  Keith is well able to speak for himself, as are other
list participants.
 
> I have made no proposal because I see an acceptable proposal already oj
> the table.

Your vision is faulty or you are having delusions; there is no
table and no proposal.

Date: Thu, 14 Oct 2004 12:41:41 -0400
From: Keith Moore <[email-masked]>
To: "Charles Lindsey" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Sender: [email-masked]

> >The problem as I see it is that we have a standard for 1-1 (direct)
> >email messages, but no standard for a list distribution to handle the
> >above two reply options.  I can see this in comparison with the list
> >server handling this IETF-2822 list and our list server.  Both are
> >doing slighty things in preparing headers so that the two actions are
> >slightly different.
> 
> >In one case (the list server used here), the "Reply to Sender" will
> >go to the author.  In our list server, the "Reply to Sender" goes to
> >the list address.
> 
> >I guess the difference is that we make sure the "Reply-To:" address
> >is properly set in the distribution.
> 
> >Is that more correct?
> 
> Many lists set Reply-To in that way, but the general wisdom, as
> promulgated in various FAQs, is that it is a Bad Thing. The chief
> reason for that is that it mucks up the case where you really DO want
> to reply individually to the author (a few MUAs have a
> "Reply-to-From-address" button which sort-of gets around the problem).
> 
> So the solution that has been suggested (and implemented in some
> places) is to have a Mail-Followup-To header (MFT) in addition to
> Reply-To.

This is not a solution.  It's another version of the same problem. 

> Several people have expressed broad satisfaction with the MFT idea
> (modulo working out the precise details), but a couple of regulars on
> this list seem violently opposed to it, so there we are.

I think all of us, myself included, see _some_ merit in MFT in that we
recognize that Reply-To is broken for "reply all" and we need something 
to replace or augment it.  We may disagree on whether it's a good idea 
for lists to set MFT, or whether it's a good idea for recipient MUAs to
blindly honor MFT when composing replies, or whether MFT is a good way
to suppress duplicate copies of replies, or whether the semantics of MFT
and Mail-Reply-To as Dan has defined them (allowing the author of a
message to separately redefine the meaning of "followup" and "reply" for
that message) are appropriate and desirable. 

Some of us may believe that those issues can be solved by "working out
the precise details".  My personal belief is that we're not ready to
think of this in terms of tweaking of details because there are still
too many differences of opinion as to the appropriate overall goals.

Keith

To: LIST: [email-masked];
Xref: clerew local.mime:3900
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Thu, 14 Oct 2004 15:25:13 GMT
Lines: 76
Sender: [email-masked]

In <001701c4b0d1$562e60f0$6401a8c0@hdev1> "Hector Santos" <[email-masked]> writes:

>The problem as I see it is that we have a standard for 1-1 (direct) email
>messages, but no standard for a list distribution to handle the above two
>reply options.  I can see this in comparison with the list server handling
>this IETF-2822 list and our list server.  Both are doing slighty things in
>preparing headers so that the two actions are slightly different.

>In one case (the list server used here), the "Reply to Sender" will go to
>the author.  In our list server, the "Reply to Sender" goes to the list
>address.

>I guess the difference is that we make sure the "Reply-To:" address is
>properly set in the distribution.

>Is that more correct?

Many lists set Reply-To in that way, but the general wisdom, as
promulgated in various FAQs, is that it is a Bad Thing. The chief reason
for that is that it mucks up the case where you really DO want to reply
individually to the author (a few MUAs have a "Reply-to-From-address"
button which sort-of gets around the problem).

So the solution that has been suggested (and implemented in some places)
is to have a Mail-Followup-To header (MFT) in addition to Reply-To. So the
"Reply(-to-Sender)" button sends it to the Reply-To address, and the
"Reply-To-List" button sends it to the MFT address. (What the Reply-to-All
button does is not clear and would need discussion).

Several people have expressed broad satisfaction with the MFT idea (modulo
working out the precise details), but a couple of regulars on this list
seem violently opposed to it, so there we are.

>If this makes sense with all, I can begin with the following:

>a) Does a MUA need to detect a Mailing List Message as oppose to a just a
>pure email message?

Yes. The presence of some of the List-* headers should do that, but
different mailing list expanders add different subsets of them (including
none at all).

>b) If so, what headers can assist the MUA to detect a mailing list message?

See above.

>c) What are the most desirable reply/new mail options for a mailing list?

>    Reply to List (very similar to a news Reply To Group idea)?
>    Reply to Directly to Author?
>    Reply to All?

I think that is the minimum collection that is needed. But too many would
just cause more confusion.

>Is that enough?  How about follow ups?

>    Follow up to List?
>    Follow up directly to author?
>    Follow up to all?

>Does that make sense?

It is not clear what you intend those ones to do.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Cc: [email-masked], Charles Lindsey <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: Understanding response protocols
Date: Thu, 14 Oct 2004 09:59:07 -0400
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

>>> I wasn't going to step into this discussion again. Bad mistake. I 
>>> think I'll try very hard to shut up from now on.
>>
>> Please don't. The silent majority needs to be heard from time to time.
>
> Judging by the silence of the majority, one might surmise that while 
> the current situation is far from ideal, no suggested alternative 
> seems worth the bother.

I don't think this group is a representative sample of email users 
anyway.  One reason that IETF makes decisions by consensus rather than 
by voting is that its participants are self-selecting.

Date: Thu, 14 Oct 2004 12:15:45 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Understanding response protocols
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

Charles Lindsey writes:
> In <[email-masked]> Philip 
> Hazel <[email-masked]> writes:
>
>> I wasn't going to step into this discussion again. Bad mistake. I 
>> think I'll try very hard to shut up from now on.
>
> Please don't. The silent majority needs to be heard from time to time.

Judging by the silence of the majority, one might surmise that while the 
current situation is far from ideal, no suggested alternative seems 
worth the bother.

Arnt

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: Understanding response protocols
Date: Wed, 13 Oct 2004 23:59:13 -0400
To: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

>> Actually some people are proposing blind acceptance.  They want MFT to
>> be honored automagically, regardless of the content of the reply.
>
> They want MFT to be honored when the replier uses the Reply-to-List
> button.

Well, that's just nuts.  MFT doesn't specify the address of the list.  
For that matter, it doesn't specify the address(es) of "all" or 
"author" either.  If the author of a reply tells the user agent "reply 
to list" or "reply to author" or "reply to all" that's what should 
happen.  MFT and Reply-To need to alert a recipient to the subject 
message author's request, and make it easy for the recipient to honor 
that request, without changing the behavior of the user agent when the 
recipient explicitly specifies what kind of reply to use.

People seem to be so accustomed to user agents where "reply" and "reply 
all" are broken in this way that they have a hard time understanding 
that there might be a better way.

>> The duplicated message problem is broader than that.  And no, MFT
>> doesn't solve even that subset of the problem - at least, not without
>> creating other problems.
>
>>>> a) have both list addresses appear in To or CC of the original 
>>>> message
>>>> b) recipients use "reply all" to reply to both lists.
>
>> It's also the only way for the replier to respond if he wants to 
>> ensure
>> that all recipients of the subject message receive the reply. The
>> replier should not assume that honoring MFT will produce the same
>> result.
>
> Honoring MFT is not intended to produce the same result. If a MUA has 
> been
> upgraded to understand MFT, then it needs to have a separate 
> Reply-to-List
> button.

Nope, MFT and the list address(es) are completely separate.  Nothing 
assures that the author who sets MFT intends that a "followup" go to 
the list when MFT is honored.  If MFT is honored the followup goes to 
wherever the author who set MFT thinks followups should go, which might 
or might not include the list(s).

>> Yup, it's a problem.  But MFT has several problems also.  If an author
>> has multiple MUAs he needs to configure each of them to be aware of
>> MFT.
>
> Yes. But if he is not fussy, then he can leave it to the mail-expander 
> to
> insert the MFT according to the list policy.

This violates the purpose of MFT.  Having lists rewrite MFT is even 
worse than having them rewrite Reply-To.

>>> .... Much
>>> simpler to
>>> avoid sending the duplicates in the first place.
>
>> It's simpler in some sense.  But it makes repliers responsible for
>> fixing a problem that is on the sender's end.
>
> What problem on the Sender's end do you have in mind? That he might set
> MFT to [email-masked]?

The sender of the message is the one who is expecting the recipient's 
MUA to not duplicate messages for him.  That's something to avoid if 
you can.  What you want - as much as you can - is for each party's 
software to be responsible for acting in that party's interests.  That 
way, when the software doesn't work to your satisfaction, you are in a 
better position to fix it.  Or to use a concrete example - if you want 
your message store or MUA to process duplicates in a particular way you 
can upgrade or configure your message store or MUA to do that, and 
you'll be happy.  But if you want everyone else's MUA to eliminate 
duplicates for you then you might wait a long time for something to 
happen that is completely outside of your control.

>>  It hides the fact that
>> the replier intended for certain recipients to receive a copy of the
>> message.  It makes it difficult for a replier to know whether the
>> recipients omitted by MFT will actually receive that message.
>
> If the replier is fussy, then he will use "Reply-to-All" and edit it 
> the
> hard way.

Again, if we want lists to be more effective tools for discussion then 
we need to make it easy for people to do the right thing when replying, 
not to arrange things so that only those who are "fussy" will consider 
it worth the bother.

It is my belief that an MUA that makes it easy to specify who should 
get replies will make its user into a more effective communicator, and 
those users will start to see a benefit.  Once other users realize 
this, they will also want MUAs that work that way.

Keith

To: LIST: [email-masked];
Xref: clerew local.mime:3896
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Wed, 13 Oct 2004 13:03:05 GMT
Lines: 128
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:
>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
>>>>
>>>>>>Which is why I said "The only place where it *can* be known ...". For
>>>>>>sure, if the author does not know, then there is even less chance that
>>>>>>subsequent mailing list expanders would know

>> No, it is NOT repeated above, since you have snipped it.

>It's *still* there.

You can read what's written inside the "..."?

>>>>Now, supposing some author wants to raise a matter which is likely to be
>>>>relevant to both [email-masked] and [email-masked]. ...
>> 
>> 
>>>He can indicate that desire trivially:
>>>Reply-To: [email-masked], [email-masked]
>> 
>> 
>> Exactly so (or use MFT). So why have you been denying my claim that the
>> mailing list expanders cannot do it without assistance from the author?

>That was not your claim; your claim was "The only place where it is
>known that a crosspost is occurring is at the original sender.
>Therefore, any information to inform subsequent agents that
>crossposting is afoot HAS to be placed in the message by the original
>sender."  Ignoring the fallacious first statement,

Why is that falacious? You failed to answer my question about how the
expander for the mailing list 'foo' could also know that the recipient
'bar' was another mailing list. OK, you can s/is known/can be known/, as I
have already explained.

> the second
>statement is clearly not about an indication of desire or
>recommendation regarding responses -- Reply-To (and the non-
>standard MFT, at least according to some claims) indicates a
>recommendation for responses; it does not "inform subsequent
>agents that crossposting is afoot".

Yes it does, because they (in particular the recipients' agents) can now
see that replies are requested to be sent to both lists.

>>>The solution to the "problem" of an author indicating his recommendation
>>>for responses is trivially solved.  You have yet to explain what "problem"
>>>requires determining if/when list expansion takes place.
>> 
>> 
>> Because you want replies to go to all the lists crossposted to, without
>> duplicate personal copies going to people who are already subscribed to
>> one of those lists.

>Authors do not always want responses to go to all lists in all
>cases where multiple lists are involved (in some cases one list
>is notified of discussion taking place on a different list, with
>responses directed to the discussion list), sometimes an author
>may want personal responses rather than list responses, and as
>you have been informed quite clearly, not all persons wish to
>be excluded from the list of recipients receiving responses
>whether or not some list is involved.

Yes, of course there are all sorts of exceptions, but it is apparent that
many, if not most, authors want replies to go to the list in normal
circumstances. If, for particular reasons, they don't want that then they
can set Reply-To/MFT somewhere else.

It is your fixation with the exceptional and unusual cases that is
blinding you to the need to get the commonly occurring cases working
smoothly with minimal intervention by users at either end.

>If an author makes a recommendation for responses to go to "foo"
>and to "bar", and expansion of "foo" includes "bar" or vice
>versa, or "foo" and "bar" when expanded both include "baz",
>then duplicates may still be received somewhere.

Again, it is your fixation with the exceptional cases that is showing
through. You seem to prefer that is is better for a system to never get it
right is there is a 0.001% chance that it will sometimes get it wrong.

>2. An appropriate question is how response recommendations
>   can be communicated under various scenarios.

>Regarding the last point, I have detailed how standard fields
>expressing standard semantics with standard syntax can be used
>under various scenarios identified during this discussion

You have signally failed to convince anyone, even Keith, that the Reply-To
field provides an acceptable solution in this matter.

 to
>communicate such response recommendations, how such usage works
>both with common "plain" reply functions as well as with
>frequently-used "reply-to-all" functions, and how failure of an
>author to communicate such recommendations when lists are
>involved leads to a number of problems for authors, recipients,
>and for third parties.  It works for various list scenarios
>(with some limitations for a specific identified unusual
>situation), for interpersonal communications not involving
>lists, for communications involving both lists and individuals,
>for delegation, for authors who want individual responses to
>their messages, for authors who want individual copies in addition
>to list responses, and for authors who want responses to go only
>to a list.  It requires no changes to MUAs, MTAs, or established
>protocols.

>You have made no substantive comment on that.  You have made
>no proposal which would address the specific identified issue
>of communicating response preferences.

I have made no proposal because I see an acceptable proposal already oj
the table.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3898
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Wed, 13 Oct 2004 14:26:22 GMT
Lines: 25
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Nick Ing-Simmons wrote:
>> Bruce Lilly <[email-masked]> writes:

>> None the less can you please give an actual case when case would 
>> change during transport? 

>A trivial case is transcription from spoken or written material,
>and I can think of at least one individual who habitually
>transcribes domain names to all upper-case.

Please can you explain why your friend should ever find it necessary to
transcribe a message identifier by hand?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3893
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Wed, 13 Oct 2004 11:51:35 GMT
Lines: 84
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:
>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>> If it never actually happens in the Real World (TM), then it is not a real
>> situation. It is merely an interesting quirk of the way RFC 2822 was
>> written.

>Such comparisons do in fact occur. Jacob's message which started
>this discussion was in fact based on exactly that type of
>comparison scenario.

No, Jacob's message made no mention of any comparison which HAD occurred
(and failed). It was purely concerned with a discrepancy between two RFCs,
and an implementor who wanted to do the "right thing".

>I have in fact mentioned transcription of various types.

You have mentioned transcriptions which MIGHT occur. You mentioned no
examples which HAD occurred, nor any current software which could cause
them to occur.

>I would add that you have failed to provide any concrete example
>of specific software which generates identifiers referring to
>multiple distinct entities which are identical on the LHS and differ
>only in case in the RHS.

I am aware of no such software, and doubt that any such software exists.

>> writers of new software should simply follow RFC 2822, and that is
>> the end of the matter.

>Evidently you fail to grasp fundamental logic, the nature of the
>problem with RFC 2822, and the nature of software development.
>Case-sensitive and case-insensitive string comparisons are
>mutually exclusive (while there are other types of comparisons
>for text, especially for certain character sets representing
>specific languages, they do not apply to the non-text protocol
>elements being compared here).  One must choose one or the other.
>One *cannot* "simply follow RFC 2822", because RFC 2822 fails to
>specify which type of comparison is to be used for an id-right
>which is not a domain name, nor does it provide any mechanism to
>identify when a particular id-right is a (recommended) domain
>name vs. when it is something else.

Presumably it makes no specification because it was not considered
relevant. In fact RFC 2822 make no mention of comparison of message
identifiers because such comparison does not arise in the course of any
process that it envisages. AFAICS, the only situation where it considers
processes involving message identifiers is when it gives instructions for
constructing a References field from the Message-ID field of some
presursor, and the instruction is to take the "contents" of that field
which means, in the absence of wording permitting otherwise, without
changing the case of anything.

Likewise I would suppose (I have not checked in detail) that RFC 2821
gives no authority for changing the Message-ID field in any way during
transit.

Thus there seems no circumstance in which a message identifier, once
generated, can legitimately be changed subsequently, in which case it
makes not a blind bit of difference whether any comparison is
case-sensitive or otherwise.

>...  And the defect in the RFC 2822 specification
>needs to be corrected consistent with long-standing use, viz.
>by reaffirming that an identifier RHS is to be interpreted as
>a case-insensitive domain name.

If there is nothing broke, then there is no need to fix it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3894
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Wed, 13 Oct 2004 12:42:42 GMT
Lines: 66
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>Actually some people are proposing blind acceptance.  They want MFT to 
>be honored automagically, regardless of the content of the reply.

They want MFT to be honored when the replier uses the Reply-to-List
button. If the replier wants to reply to the author "off-list", as is
sometimes useful, then he hits "Reply" (which is why the Reply-To field
is no substitute for a distinct MFT field). And if he wants to compose
something else he hits "Reply-to-All" and picks and chooses from the
selection of addresses on offer.

>The duplicated message problem is broader than that.  And no, MFT 
>doesn't solve even that subset of the problem - at least, not without 
>creating other problems.

>>> a) have both list addresses appear in To or CC of the original message
>>> b) recipients use "reply all" to reply to both lists.

>It's also the only way for the replier to respond if he wants to ensure 
>that all recipients of the subject message receive the reply. The 
>replier should not assume that honoring MFT will produce the same 
>result.

Honoring MFT is not intended to produce the same result. If a MUA has been
upgraded to understand MFT, then it needs to have a separate Reply-to-List
button.

>Yup, it's a problem.  But MFT has several problems also.  If an author 
>has multiple MUAs he needs to configure each of them to be aware of 
>MFT.

Yes. But if he is not fussy, then he can leave it to the mail-expander to
insert the MFT according to the list policy.

>> .... Much 
>> simpler to
>> avoid sending the duplicates in the first place.

>It's simpler in some sense.  But it makes repliers responsible for 
>fixing a problem that is on the sender's end.

What problem on the Sender's end do you have in mind? That he might set
MFT to [email-masked]?

>  It hides the fact that 
>the replier intended for certain recipients to receive a copy of the 
>message.  It makes it difficult for a replier to know whether the 
>recipients omitted by MFT will actually receive that message.

If the replier is fussy, then he will use "Reply-to-All" and edit it the
hard way.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3897
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fw: Angle brackets surrounding Content-ID
Date: Wed, 13 Oct 2004 14:24:46 GMT
Lines: 37
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:

>> An MUA that chooses whether to followup to Usenet or to reply using email
>> simply on the basis of whether a Newsgroups-header is present is severely
>> broken, since it is normal behaviour for a message which is both posted to
>> Usenet and emailed to individual recipients to include a Newsgroup-header
>> (how else are recipients to know that both media were involved).

>Charles, as you did not include a Reply-To field pointing to this list,
>you forced respondents into using a "reply-to-all" function, and those
>UAs which support both news and mail (including Netscape/Mozilla,
>Outlook Express, and Pine, several of the most widely used UAs for either
>mail or news) include specified newsgroups in such a "reply-to-all".

It does not surprise me in the least that Outlook Express is broken in
that respect, but I would have expected Netscape/Mozilla to have known
better. The only meaning that can be ascribed to the presence of a
Newsgroups header in an email message is that it signifies a message that
was both posted and mailed.

A User Agent that uses a "Reply" button to initiate a "Followup" is most
certainly broken (see www.xs4all.nl/~js/gnksa). It is a sad fact that most
MUAs which have been adapted to dual use as newsreaders make a pretty poor
job of their newsreading role.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3895
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Wed, 13 Oct 2004 12:44:28 GMT
Lines: 17
Sender: [email-masked]

In <[email-masked]> Philip Hazel <[email-masked]> writes:

>I wasn't going to step into this discussion again. Bad mistake. I think 
>I'll try very hard to shut up from now on.

Please don't. The silent majority needs to be heard from time to time.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Subject: Re: Angle brackets surrounding Content-ID
To: [email-masked]
Date: Wed, 13 Oct 2004 09:40:41 +0100
From: Nick Ing-Simmons <[email-masked]>
Reply-To: Nick Ing-Simmons <[email-masked]>
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:
>Nick Ing-Simmons wrote:
>> Bruce Lilly <[email-masked]> writes:
>
>> None the less can you please give an actual case when case would 
>> change during transport? 
>
>A trivial case is transcription from spoken or written material,

Or transcoding into morse perhaps? 

But these, as you say elsewhere, are "non text fields".
I don't think it would ever make sense for mail _transport_ to 
work by converting headers to voice and back. 

>and I can think of at least one individual who habitually
>transcribes domain names to all upper-case.

Then they deserve to have their references mangled ;-)

Cc: Charles Lindsey <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
To: [email-masked]
Date: Wed, 13 Oct 2004 09:29:38 +0100
From: Nick Ing-Simmons <[email-masked]>
Reply-To: Nick Ing-Simmons <[email-masked]>
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:
>
>Evidently you fail to grasp fundamental logic, the nature of the
>problem with RFC 2822, and the nature of software development.

If we are going to be patronising : you have failed to grasp the 
nature of e-mail communication in the real world.

In the real world we are lucky if mail tools even attempt 
to conform to the spirit of an RFC let alone the letter.

On the other hand the consequences if using the wrong sort of compare
for these fields seem to be rather unimportant.

>One must choose one or the other.
>One *cannot* "simply follow RFC 2822", because RFC 2822 fails to
>specify which type of comparison is to be used for an id-right
>which is not a domain name, 

Doing case sensitive compare "seems to work well enough" even when it is a 
domain name. 

From: "Hector Santos" <[email-masked]>
To: "ietf-822 list" <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Date: Wed, 13 Oct 2004 00:10:56 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

From: "Laird Breyer" <[email-masked]>

> I'm talking about a mailing list server which has an internal list of
> subscriber addresses, and any message sent to the list address is
> distributed to each of the subscribers. If some of the subscribers
> have indicated special preferences for how they would like to receive
> *their* copies of the messages sent by others to the list, then the
> server can perform whatever action is preferred on *their* copies of
> the messages, based on both the message content and the archive of
> past messages if needed. The fact that such action can depend on
> message content and past archive is what makes the server "smart".

Good point.   However, it is easily overcome by creating two separate
replies. One to the list, and then resending the sent message in the "Sent
folder" to the direct address.

But in a default behavior, very good point.  The SMTP server would be
getting the original message as a multi RCPT TO: transaction can maybe
handle this directly?   I guess it would need to have access to the separate
list server subscriber database, more integrated.

Our SMTP receiver thread will get the single transaction with multi RCPT TO:
which then signals the SMTP router thread which will expand it for the
separate RCPT TO:   The directs emails are now on their own and queued for
the SMTP sender thread.. The specific list address message is locally stored
which is then picked up by the list server.

Off hand, this sounds like its better handled by a smarter MUA with dupe
processing logic.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: "ietf-822 list" <[email-masked]>
Subject: Re: MUA Mail Options for a Mailing List
Date: Tue, 12 Oct 2004 23:33:57 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

I'm following up on my message:

----- Original Message -----
From: "Hector Santos" <[email-masked]>
To: "ietf-822 list" <[email-masked]>

> a) Does a MUA need to detect a Mailing List Message as oppose to a just a
> pure email message?
>
> b) If so, what headers can assist the MUA to detect a mailing list
message?

I think maybe standardizing of a "List-Address:" header would help address
both questions.

> c) What are the most desirable reply/new mail options for a mailing list?
>
>     Reply to List (very similar to a news Reply To Group idea)?
>     Reply to Directly to Author?
>     Reply to All?

If the above is used, what would be the addresses used?

For maximum backward compatibility:

    Reply to List: <-- List-Address: |  Reply-To: |  From:
    Reply to Sender: <--  Reply-To: |  From:
    Reply to All <--  List-Address:?,  Reply-To:, To:, Cc:

Should a reply to All? Go to the list as well?

> Is that enough?  How about follow ups?
>
>     Follow up to List:
>     Follow up directly to author?
>     Follow up to all?
>
> Does that make sense?

I think Follow ups are similar to replies with the difference is how the
mail is quoted or not quoted, including possible a body header and footer
being address:

        --- This is a follow up ....
        [original body]

Comments?

Do we need a  Message-Followup-To? (MFT) header?  how does it fit here?

So I see so far two possible new headers:

    List-Address: ??
    Message-Followup-To: ??

More considerations:

While helping the MUA with the network headers, how about the Display
Headers?

The issue here I think will be based on how much an expander should or even
if it should prepare each message so that the following is correct:

        From:        Subscriber address
        To:            List Address for new or  From: address for replies
        List-Header:   List address
        Reply-To:   (keep original, if any, otherwise defined by MUA)

Comments?

 Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Wed, 13 Oct 2004 13:25:56 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 list <[email-masked]>
Sender: [email-masked]

On Oct 12 2004, Bruce Lilly wrote:
> > 
> > I responded that's a possibility, but by far in most cases an author
> > wants responses sent to him via the mailing list, not directly, so that
> > ought to be default behaviour for this kind of advanced mailing list server. 
> 
> That makes little sense; a person subscribes to a mailing list
> specifying his email address as "the address that was subscribed".
> If he posts a message to that list and some reader responds "to
> the address that was subscribed" (however he is supposed to be able

(snipped).

What are you talking about? I'm very confused by this statement.

I'm talking about a mailing list server which has an internal list of
subscriber addresses, and any message sent to the list address is
distributed to each of the subscribers. If some of the subscribers
have indicated special preferences for how they would like to receive
*their* copies of the messages sent by others to the list, then the
server can perform whatever action is preferred on *their* copies of
the messages, based on both the message content and the archive of
past messages if needed. The fact that such action can depend on 
message content and past archive is what makes the server "smart".

The other members of the list don't need to know anything at all, except
whether they want to send a particular message to the list address. Once
it hits the list address, they have no more say in where it goes or what
happens to that copy, except for their own personal copy delivered to them
as list members. 

> 
> > I was thinking of Mutt's list-reply command. From the manual:
> [...]
> 
> That involves configuration, and any "list" involved is either some
> preconfigured static mailbox or is derived (with some additional
> configuration) from the content of some non-standard field set by
> somebody else (or by his UA following some extensive configuration
> on his end).  There is no guarantee that that mailbox is a list at
> all; a preconfigured mailbox won't be able to handle a message sent
> to multiple lists, author recommendations, etc.

That's the current situation with MUAs. People configure them according
to their whims and computer skills. If all the MUAs in the world followed
all applicable standards consistently, we probably wouldn't be having this
conversation in the first place.

> 
> Sounds like what you're looking for is an ESP-based server...

As long as nobody thinks of the Giant Marshmallow Man, we'll be safe...

-- 
Laird Breyer.

From: "Hector Santos" <[email-masked]>
To: "ietf-822 list" <[email-masked]>
Subject: MUA Mail Options for a Mailing List [was Re: non-member messages to lists]
Date: Tue, 12 Oct 2004 23:04:16 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Bruce Lilly" <[email-masked]>

> > I responded that's a possibility, but by far in most cases an author
> > wants responses sent to him via the mailing list, not directly, so that
> > ought to be default behaviour for this kind of advanced mailing list
server.
>
> That makes little sense; a person subscribes to a mailing list
> specifying his email address as "the address that was subscribed".
> If he posts a message to that list and some reader responds "to
> the address that was subscribed" (however he is supposed to be able
> to determine that -- it might not be the mailbox in the author's
> From field), that response goes solely to that address and the
> list is not involved in any capacity -- either in the transmission
> of the message from respondent to the author or in expanding that
> response to the list (since the response isn't sent to the list
> mailbox).  If a respondent sends his response to the list mailbox,
> it is expanded to the list, including "the address that was subscribed";
> that's what a list expander does, and "default behaviour" doesn't
> enter into the matter.  That happens regardless of and independently
> of whether or not the author in question provided any recommendation
> for where responses should be sent.

Bruce I think the issue does relate to the default behavior" for two type of
standard (or BCP) email reply concepts:

        "Reply to Sender"
        "Reply to All"

and how these two MUA "standard" reply actions are handled or "prepared" by
list handlers.

I mean, lets take a step back and understand the the list concept is still
based on a email concept. So to help facilate the MUA, the list handler
should prepare the headers to best provide the most meaningful or correct
behavior as expected by the user.

The problem as I see it is that we have a standard for 1-1 (direct) email
messages, but no standard for a list distribution to handle the above two
reply options.  I can see this in comparison with the list server handling
this IETF-2822 list and our list server.  Both are doing slighty things in
preparing headers so that the two actions are slightly different.

Atleast I see the different with the Outlook MUA in how it does the above
two reply actions depending on the mailing list server.

In one case (the list server used here), the "Reply to Sender" will go to
the author.  In our list server, the "Reply to Sender" goes to the list
address.

I guess the difference is that we make sure the "Reply-To:" address is
properly set in the distribution.

Is that more correct?

The way I see it,  Yes and No.

Yes, I think it is more technically correct when you think about what is the
real reply address the message should to when sending to the list.

No, when you think about the ergonomics of the current standard MUAs which
not trained to understand a mailing list message.

The user sees "Reply to Sender" well, one can say that the list server used
here is more correct as far as "meaning" of the action.  The problem is to
write to the list, the user must use "reply to all" and then either ignore
all the addresses prepared or clean it up to just use the list address.  I
would venture the majority of people over time will get tired of the editing
and just use whats there, hence all the direct and list mail going out.

With our list server, the "Reply to Sender" goes to the list address.  But
what if the user wanted to so send a private message directly to the author?
He needs to do the "Reply to All" and hope that the personal address is in
list.   Unfortunately, atleast with Outlook the "Reply to All" does not grab
the "From:" address.

In this regard, I would say our list server makes it more difficult to send
direct.  The user has to understand whats going on and copy and paste the
From: address from the message properties.

Of course, it is all due to getting things to work right "by default" for
the common MUA.

So I think maybe it may help us to get to a point where we might want to
outline design requirements for a modern MUA  to better understands a
mailing list?  What would be new MUA desireable reply or new mail options
for an mailing list?

By understanding what are the MUA needs, then we can better work with the
backend, and list servers to help make this happen and also possible come up
with new proposals.

If this makes sense with all, I can begin with the following:

a) Does a MUA need to detect a Mailing List Message as oppose to a just a
pure email message?

b) If so, what headers can assist the MUA to detect a mailing list message?

c) What are the most desirable reply/new mail options for a mailing list?

    Reply to List (very similar to a news Reply To Group idea)?
    Reply to Directly to Author?
    Reply to All?

Is that enough?  How about follow ups?

    Follow up to List?
    Follow up directly to author?
    Follow up to all?

Does that make sense?

Anyway,  I would love to someone agree with this approach.  Maybe this can
be a start in helping set a new set of MUA standards for groupware concepts.

Thanks

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Tue, 12 Oct 2004 21:28:57 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: [email-masked]
CC: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

Laird Breyer wrote:
> On Oct 11 2004, Bruce Lilly wrote:
> 
>>>I do accept that the possibility is there, but am yet to be convinced that
>>>it happens so frequently to render inadequate the default behaviour of 
>>>sending list responses to the address that was subscribed.
>>
>>What exactly is that supposed to mean? Responses go wherever the
>>respondent sends them.
> 
> 
> Whatever you meant it to mean. I responded to the following quote from you:
> 
> 
>>An author might very well change his recommendation for responses
>>on a per-message basis.  
> 
> 
> I responded that's a possibility, but by far in most cases an author
> wants responses sent to him via the mailing list, not directly, so that
> ought to be default behaviour for this kind of advanced mailing list server. 

That makes little sense; a person subscribes to a mailing list
specifying his email address as "the address that was subscribed".
If he posts a message to that list and some reader responds "to
the address that was subscribed" (however he is supposed to be able
to determine that -- it might not be the mailbox in the author's
>From field), that response goes solely to that address and the
list is not involved in any capacity -- either in the transmission
of the message from respondent to the author or in expanding that
response to the list (since the response isn't sent to the list
mailbox).  If a respondent sends his response to the list mailbox,
it is expanded to the list, including "the address that was subscribed";
that's what a list expander does, and "default behaviour" doesn't
enter into the matter.  That happens regardless of and independently
of whether or not the author in question provided any recommendation
for where responses should be sent.

> I was thinking of Mutt's list-reply command. From the manual:
[...]

That involves configuration, and any "list" involved is either some
preconfigured static mailbox or is derived (with some additional
configuration) from the content of some non-standard field set by
somebody else (or by his UA following some extensive configuration
on his end).  There is no guarantee that that mailbox is a list at
all; a preconfigured mailbox won't be able to handle a message sent
to multiple lists, author recommendations, etc.

>>Fine, but that affects what he receives from the list, not what he
>>sends to wherever he sends it to.  In particular, it does nothing
>>to address communicating his recommendations for responses to messages
>>that he authors.
> 
> 
> That was my intention with the 'intelligent server" all along, but it 
> appears I failed to communicate that. 

Sounds like what you're looking for is an ESP-based server...

Date: Tue, 12 Oct 2004 21:04:19 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Understanding response protocols
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> 
>>Charles Lindsey wrote:
>>
>>>In <[email-masked]> Bruce Lilly <[email-masked]> writes:
>>>
>>>
>>>
>>>>Charles Lindsey wrote:
>>>
>>>
>>>>>Which is why I said "The only place where it *can* be known ...". For
>>>>>sure, if the author does not know, then there is even less chance that
>>>>>subsequent mailing list expanders would know

Nota bene.

>> This
>>started with your claim -- repeated above -- that only the message
>>author can know when list expansion takes place.
> 
> 
> No, it is NOT repeated above, since you have snipped it.

It's *still* there.

>>>Now, supposing some author wants to raise a matter which is likely to be
>>>relevant to both [email-masked] and [email-masked]. ...
> 
> 
>>He can indicate that desire trivially:
>>Reply-To: [email-masked], [email-masked]
> 
> 
> Exactly so (or use MFT). So why have you been denying my claim that the
> mailing list expanders cannot do it without assistance from the author?

That was not your claim; your claim was "The only place where it is
known that a crosspost is occurring is at the original sender.
Therefore, any information to inform subsequent agents that
crossposting is afoot HAS to be placed in the message by the original
sender."  Ignoring the fallacious first statement, the second
statement is clearly not about an indication of desire or
recommendation regarding responses -- Reply-To (and the non-
standard MFT, at least according to some claims) indicates a
recommendation for responses; it does not "inform subsequent
agents that crossposting is afoot".

>>Yes, it is true that the author is in the best position to provide
>>*his* recommendation for responses, if he is willing to do so, but
>>that is unrelated to your claim regarding identification of if/when
>>list expansion takes place.
> 
> 
> So finally you admit that my point was correct.

No, your original statement about list expansion was and remains
incorrect.  Your newly stated repetition of earlier statements
made on this list by others that author(s) may provide response
recommendations via the Reply-To field is of course correct, but
it is neither novel nor related to your initial incorrect claim.

>>The solution to the "problem" of an author indicating his recommendation
>>for responses is trivially solved.  You have yet to explain what "problem"
>>requires determining if/when list expansion takes place.
> 
> 
> Because you want replies to go to all the lists crossposted to, without
> duplicate personal copies going to people who are already subscribed to
> one of those lists.

Authors do not always want responses to go to all lists in all
cases where multiple lists are involved (in some cases one list
is notified of discussion taking place on a different list, with
responses directed to the discussion list), sometimes an author
may want personal responses rather than list responses, and as
you have been informed quite clearly, not all persons wish to
be excluded from the list of recipients receiving responses
whether or not some list is involved.  And in any event, there
is no standard and practical mechanism for an author to determine
who is and who is not subscribed to any particular list at any
particular time.  In short, the twelve word first phrase of your
statement above is incorrect much of the time, the remainder is
incorrect at least part of the time, and the underlying unstated
assumption that an author can determine list subscription status
for others is incorrect almost all of the time.

An author can make recommendations for responses regardless of
whether or not he is aware of list expansion, regardless of
whether or not any list expansion takes place, and regardless of
whether or not any (or many) lists are involved. It is a
generic mechanism.

If an author makes a recommendation for responses to go to "foo"
and to "bar", and expansion of "foo" includes "bar" or vice
versa, or "foo" and "bar" when expanded both include "baz",
then duplicates may still be received somewhere. No proposal
(or suggestion) to date can prevent that. In the latter case
even if the author knows that "baz" results from expansion of
both "foo" and "bar" there is nothing that he can do to prevent
bar from receiving duplicates if he in fact wants his message
to go to lists "foo" and "bar" (unless he is in the peculiar
position of knowing the full expansions of both "foo" and "bar"
and is willing and able to send copies individually to at least
a subset of one list -- in which case he's not sending *to*
that list).

> No amount of setting policies for individual lists, or
> super-smart lists as Laird has been proposing, will bring that about.

Agreed.

> The
> *only* place where it can be solved is at the author's MUA.

With existing core protocols the Holy Grail of preventing all
duplicates under all circumstances cannot be achieved, via any
reasonable action that an author may take, or elsewhere, either
with existing fields or with any extensions proposed to date.

> And the sort
> of smartness that Dan wants to put into such MUAs, together with MFT, will
> in fact do it rather nicely.

No. Even with the modifications which would be required for
MUAs and MTAs it would do no more (in the very distant long
term) than can be achieved now with the standard Reply-To
field (with its standard syntax and semantics). And that
includes the limitation noted above -- some duplicates are
inevitable under some (not particularly unusual) circumstances.
MFT also suffers from the limitation of being a one-trick
pony -- it is a special-purpose mechanism that does not
lend itself to general use (delegation, etc.).

Keith is right on several points:
1. Since we cannot achieve all things for all users under all
   conditions, we ought to focus on achieving the greatest
   benefit attainable with the least disruption to existing
   protocols and infrastructure.
2. Duplicate suppression is best achieved near the receiving
   end (MDA/MS/rMUA). [some is of course possible via some
   list expanders which can be configured to recognize that
   a mailbox in To or Cc fields of a message matches a mailbox
   in the list expansion -- but that capability is also subject
   to abuse (i.e. there are security implications), and is not
   to all users' liking]
3. Some advice to UA implementors is likely to be necessary
   regardless of what else is or isn't done
Keld is correct in several observations:
1. Guessing doesn't work.
2. An appropriate question is how response recommendations
   can be communicated under various scenarios.

Regarding the last point, I have detailed how standard fields
expressing standard semantics with standard syntax can be used
under various scenarios identified during this discussion to
communicate such response recommendations, how such usage works
both with common "plain" reply functions as well as with
frequently-used "reply-to-all" functions, and how failure of an
author to communicate such recommendations when lists are
involved leads to a number of problems for authors, recipients,
and for third parties.  It works for various list scenarios
(with some limitations for a specific identified unusual
situation), for interpersonal communications not involving
lists, for communications involving both lists and individuals,
for delegation, for authors who want individual responses to
their messages, for authors who want individual copies in addition
to list responses, and for authors who want responses to go only
to a list.  It requires no changes to MUAs, MTAs, or established
protocols.

You have made no substantive comment on that.  You have made
no proposal which would address the specific identified issue
of communicating response preferences.  What you have done is
to make several fallacious statements with no supporting
evidence or analysis, most of which involve only a tiny subset
of the issue under discussion, and you have proposed a major
change to established syntax/semantics (changing a display
name to a keyword) which simply won't with existing standard
protocols (e.g. the Sieve protocol explicitly does not handle
named groups) and which has been widely criticized here.

Date: Tue, 12 Oct 2004 19:26:24 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>>No Charles, this is not merely "possible scenarios". It is about
>>the very real situation of comparing identifiers.
> 
> 
> If it never actually happens in the Real World (TM), then it is not a real
> situation. It is merely an interesting quirk of the way RFC 2822 was
> written.

Such comparisons do in fact occur. Jacob's message which started
this discussion was in fact based on exactly that type of
comparison scenario.

> Since I have repeatedly asked you for instances of where this has actually
> happened in the Real World, or for examples of existing agents which
> explicitly change the case of the id-right of a msg-id when copying it
> (during transit or otherwise), and you have signally failed to provide
> such examples, then I conclude that you are unaware of any such examples.
> Moreover, nobody else on this list has produced such examples.

I have in fact mentioned transcription of various types.

I would add that you have failed to provide any concrete example
of specific software which generates identifiers referring to
multiple distinct entities which are identical on the LHS and differ
only in case in the RHS.  Not that that would make any substantive
difference to the discussion, since any such software would fail
to conform to the (full Standard) RFC 822 requirement for uniqueness
of such identifiers -- but since you seem to be enamored of pointless
ultimata...  "Moreover, nobody else on this list has produced such examples."

> Therefore, I conclude that this minor change from RFC 822 to RFC 2822 has
> no practical significance in the Real World and has not lead to any
> problems of interoperability. In which case, since RFC 2822 supersedes RFC
> 822,

RFC 2822 does not in fact supersede RFC 822. RFC 822 is a full
Standard, whereas RFC 2822 is merely a Proposed Standard. If and
when some successor to RFC 2822 achieves full Standard status,
that document may well supersede RFC 822.  Hopefully by then the
problems identified with RFC 2822 will have been repaired.

> writers of new software should simply follow RFC 2822, and that is
> the end of the matter.

Evidently you fail to grasp fundamental logic, the nature of the
problem with RFC 2822, and the nature of software development.
Case-sensitive and case-insensitive string comparisons are
mutually exclusive (while there are other types of comparisons
for text, especially for certain character sets representing
specific languages, they do not apply to the non-text protocol
elements being compared here).  One must choose one or the other.
One *cannot* "simply follow RFC 2822", because RFC 2822 fails to
specify which type of comparison is to be used for an id-right
which is not a domain name, nor does it provide any mechanism to
identify when a particular id-right is a (recommended) domain
name vs. when it is something else. One cannot rely on querying
DNS for several reasons: one may be offline and therefore DNS is
unavailable, a name which may once (i.e. at time of message
generation) have been a valid domain name may have since
expired, a string which was not at time of generation a valid
domain name may have become registered as a domain name,
temporary DNS failures may result in inability to determine
status of a prospective name, etc.  RFC 2822 of course provides
for parsing "old" messages, including those generated under
RFC 822 and its predecessors; in fact UAs and other software
does need to process archived messages and that should be
performed according to the long-established rules (viz.
identifier RHS is a domain name and domain names are
case-insensitive).  Existing software couldn't change
worldwide overnight even if RFC 2822 provided some concrete
mandate w.r.t. case-sensitivity (which it doesn't).  Even
if "King Charles" could wave a magic wand and produce such a
change, there is still a need to be able to properly handle
archived messages in accordance with RFC 724/733/822/1034/1035
rules.  The only sensible thing for software developers at
this time is to continue to use the clear and precise RFC 822
rules which continue to be compatible with the RFC 2822
recommendation.  And the defect in the RFC 2822 specification
needs to be corrected consistent with long-standing use, viz.
by reaffirming that an identifier RHS is to be interpreted as
a case-insensitive domain name.

Date: Tue, 12 Oct 2004 14:59:19 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Nick Ing-Simmons <[email-masked]>
CC: [email-masked]
Subject: Re: Angle brackets surrounding Content-ID
Sender: [email-masked]

Nick Ing-Simmons wrote:
> Bruce Lilly <[email-masked]> writes:

> None the less can you please give an actual case when case would 
> change during transport? 

A trivial case is transcription from spoken or written material,
and I can think of at least one individual who habitually
transcribes domain names to all upper-case.

> If nothing ever changes the case then you make as well do 
> the faster case-sensitive compare.

The difference in speed is quite small (if implemented well).

> If anything could change the case then you must do the insensitive 
> compare.

Several things could, therefore case insensitive comparisons
are called for.

Date: Tue, 12 Oct 2004 17:05:47 +0100 (BST)
From: Philip Hazel <[email-masked]>
To: Keith Moore <[email-masked]>
cc: [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

On Tue, 12 Oct 2004, Keith Moore wrote:

> Is that really the "point"?  i.e. Is that really why you want only the LIST
> copy, so you'll benefit from the extra information in the message header (or -
> as this happens also - from the removal of information from the message header
> that was done by the list)?

Yes. The benefit is that my MUA can then know it is replying to a 
particular list, and behave in an appropriate manner, as I have 
configured it. For instance (a trivial example), I might want to use a 
different .sig file for lists. Also, it can file the list message 
correctly.

> If someone replies to a message you sent to a list and (for whatever reason)
> the list blocks that reply, do you still want your personal copy of the
> message to be blocked also?

I do not want you even to SEND a personal copy. I'll take a chance on 
the list. You may think I'm misguided, but that's what I want. And I 
know I'm not alone.

I wasn't going to step into this discussion again. Bad mistake. I think 
I'll try very hard to shut up from now on.

Philip

-- 
Philip Hazel            University of Cambridge Computing Service,
[email-masked]      Cambridge, England. Phone: +44 1223 334714.

Cc: Charles Lindsey <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
To: [email-masked]
Date: Tue, 12 Oct 2004 16:51:00 +0100
From: Nick Ing-Simmons <[email-masked]>
Reply-To: Nick Ing-Simmons <[email-masked]>
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:
>> 
>> And yet again you repeat a whole string of possible scenarios, about
>> which there is no dispute, whilst steadfastly refusing to answer the
>> question that was asked, which specifically related to "actual" behaviour.
>
>No Charles, this is not merely "possible scenarios". It is about
>the very real situation of comparing identifiers. Either a case-
>sensitive or a case-insensitive comparison could be made; one
>must decide which. 

>Under RFC 822 and its predecessors, where the
>RHS of an identifier was always a domain name, the correct
>decision is to compare those domain names case-insensitively. The
>change in RFC 2822 -- where we are told that the RHS might or
>might not be a domain name, and if it is not, we are not told
>whether or not it is case-sensitive -- makes that decision
>more difficult.  It is an issue of the design of such a
>comparator, irrespective of the source of the identifiers.

None the less can you please give an actual case when case would 
change during transport? 

If nothing ever changes the case then you make as well do 
the faster case-sensitive compare.
If anything could change the case then you must do the insensitive 
compare.

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: Understanding response protocols
Date: Tue, 12 Oct 2004 10:43:49 -0400
To: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

>>> So please answer me. HOW is the mail expander for 'foo' supposed to 
>>> know
>>> about the totally separate mailing list 'bar' to which the author
>>> crossposted his message. And if it does not know about 'bar', how 
>>> can it
>>> possibly arrange for replies to be crossposted to both lists (by
>>> appropriate setting of Reply-To/MFT/whatever)?
>
>> It can't.  But that's also mostly irrelevant, for several reasons.
>
>> 1.  The vast majority of senders are not going to want to configure 
>> their
>> MUAs to know about mailing lists.  (and if you say that they should be
>> automatically configured, be sure you understand the security 
>> implications)
>
> The vast majority of senders do not regularly use mailing lists. But 
> those
> who do regularly use them (of which the subscribers to this list are
> probably typical) would surely be happy to configure their MUAs,
> especially if this would get rid of difficulties they were 
> experiencing.

Not clear.  I know a lot of list subscribers who are not sophisticated 
mail users, and who are intimidated by suggestions that they make even 
slight configuration changes to their MUAs.

I don't agree that the subscribers to this list are typical.  This list 
consists of people who have a specific interest in email.  Most of us 
are probably far more technically adept than the typical list 
subscriber.  To the extent that we are intimidated by user agent 
configuration options, it's probably because they don't use the same 
terms as the RFCs.

>> 2.  ... it's not clear what should happen when replying to multiple 
>> lists,
>> and there is a conflict between the lists' policies.
>
> In that case, only the author is in a position to define a policy for 
> for
> that particular communication (and hopefully for the ensuing thread).

I'd argue that this is the case for all replies.  Original authors can 
have preferences, lists can have policies, but the person composing the 
reply is the ultimate decision maker.

>> 3.  Blind acceptance of MFT when composing replies is a bad idea,
>> for the same reasons that blind acceptance of Reply-To is a bad idea.
>
> Nobody is proposing "blind" acceptance. By all means let there be 
> alerts
> and warnings in RED; but most recipients will be happy to follow the
> author's (or the list maintainer's) recommendation.

Actually some people are proposing blind acceptance.  They want MFT to 
be honored automagically, regardless of the content of the reply.

>> 4.  MFT doesn't fix the problem of duplicated messages anyway.
>
> If by "duplicated messages" you mean unwanted personal replies in 
> addition
> to replies sent to the list, then MFT does indeed solve it.

The duplicated message problem is broader than that.  And no, MFT 
doesn't solve even that subset of the problem - at least, not without 
creating other problems.

>>> Now, supposing some author wants to raise a matter which is likely 
>>> to be
>>> relevant to both [email-masked] and [email-masked]. So he
>>> cross-posts to both lists, and ideally he would like all respondents 
>>> on
>>> both lists to reply to both lists. How can this be brought about?
>
>> a) have both list addresses appear in To or CC of the original message
>> b) recipients use "reply all" to reply to both lists.
>
> No, that is the current situation, and is the cause of the mess we are 
> in.

It's also the only way for the replier to respond if he wants to ensure 
that all recipients of the subject message receive the reply. The 
replier should not assume that honoring MFT will produce the same 
result.

>> What I suspect you really want is to do this without having reply
>> authors see duplicate messages.  The only way to solve that problem
>> is to have recipients' MUAs or message stores do duplicate detection.
>
> No, MFT applied by the author can solve it. The problem with expecting
> MUAs or message stores to solve it is that some users may have several
> MUAs and several message stores and several layers of filtering to 
> handle
> their incoming mail.

Yup, it's a problem.  But MFT has several problems also.  If an author 
has multiple MUAs he needs to configure each of them to be aware of 
MFT.

(note that _most_ users probably do not have several MUAs and several 
message stores and especially several layers of filtering on their 
incoming mail.  the more sophisticated users who do have such setups 
are probably capable of adapting to message stores or MUAs that can 
handle duplicates.)

> Moreover, direct replies and replies via the list
> tend to arrive by different routes and at different times. Much 
> simpler to
> avoid sending the duplicates in the first place.

It's simpler in some sense.  But it makes repliers responsible for 
fixing a problem that is on the sender's end.  It hides the fact that 
the replier intended for certain recipients to receive a copy of the 
message.  It makes it difficult for a replier to know whether the 
recipients omitted by MFT will actually receive that message.  If MFT 
is blindly trusted, it (and reply-to) are potentially useful to a 
message forger in seeing that the owner of the forged address do not 
receive replies to the forgery.  If MFT is not blindly trusted, it 
won't result the ease of use that its proponents desire.  (there may be 
some middle ground).

Keith

Date: Tue, 12 Oct 2004 00:12:57 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Fw: Angle brackets surrounding Content-ID
Sender: [email-masked]

Charles Lindsey wrote:
> In <005d01c4ad37$c1f8efa0$6401a8c0@hdev1> "Hector Santos" <[email-masked]> writes:
> 
> 
>>PS: There should not be a Newsgroups: header when sending your email.
>>Atleast this MUA is thinking it is a newsgroup article when replying.   We
>>have a similar newsgroup emulator for email/mailing list.  Your news reader
>>is doing the right thing, but whatever is on the backend, needs to clean it
>>up before it exports it back into the email system as a "email message" not
>>as a news article.    I am just saying is that the stupid MUA will use the
>>Newsgroups: header an email vs. newsgroup indicator.
> 
> 
> An MUA that chooses whether to followup to Usenet or to reply using email
> simply on the basis of whether a Newsgroups-header is present is severely
> broken, since it is normal behaviour for a message which is both posted to
> Usenet and emailed to individual recipients to include a Newsgroup-header
> (how else are recipients to know that both media were involved).

Charles, as you did not include a Reply-To field pointing to this list,
you forced respondents into using a "reply-to-all" function, and those
UAs which support both news and mail (including Netscape/Mozilla,
Outlook Express, and Pine, several of the most widely used UAs for either
mail or news) include specified newsgroups in such a "reply-to-all".  That
problem caused by your failure to indicate a response recommendation which
includes the mailing list is exactly the subject of an ongoing discussion
on this mailing list, and the specific case of the bogus Newsgroups field
in your messages has been raised here within the past few weeks.

> I agree that, in this case where the newsgroup in question is purely local
> to my own server, my backend could well have edited out the superfluous
> header, and I really ought to fix that (and maybe I shall make it put a
> Reply-To in at the same time).

And this is neither the first nor second time that the problems you have
caused have been brought to your attention in painstaking detail; e.g.
(edited for brevity):
----------------------
Date: Wed, 19 Feb 2003 20:07:32 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: Charles Lindsey <[email-masked]>
Cc: [email-masked],
	IMAP Interest List <[email-masked]>
Subject: Re: IMAP and Netnews
In-Reply-To: <[email-masked]>
Message-ID: <[email-masked]>
References: <[email-masked]>
 <[email-masked]>
 <[email-masked]> <[email-masked]>
 <[email-masked]>
Organization: Networks & Distributed Computing
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: [email-masked]
Precedence: bulk

On Wed, 19 Feb 2003, Charles Lindsey wrote:

> >Your message had a header of "Newsgroups: local.usefor" which is
> >unreplyable.  There is no such newsgroup as "local.usefor" here at the
> >University of Washington.
> Now I find that remark odd. Like many others on this list, I gateway my
> mailing lists into local newsgroups for convenience of handling. Not
> surprisingly, the evidence of that is visible when gatewayed back into the
> list. But that should not affect your ability to reply.

Your message arrived to me in email with these headers:
	Date: Wed, 19 Feb 2003 15:54:36 GMT
	Message-Id: <[email-masked]>
	Newsgroups: local.usefor
	Path: chl
	From: "Charles Lindsey" <[email-masked]>
	To: [email-masked]
	Subject: Re: IMAP and Netnews

Since there is no mention of the [email-masked] email
address, any reply that I send would not go to the Usefor mailing list.

Furthermore, the reply would try to send to a newsgroup called
"local.usefor" which does not exist here.

I was forced to edit out the bogus newsgroup, and edit in the mailing list
email address, manually in order to compose this reply.

Please be courteous and use properly replyable headers.  It is unseemly
for a document editor to send unreplyable messages.

> The Newsgroups
> header currently has no defined meaning within email, though it is
> commonly used as an informational header to indicate that a message has
> been posted and mailed

Not only is that the only sane definition of that header, but mail/news
software used by millions of people worldwide use that definition.

> I emailed two copies of that message, one using the "cc to poster" feature
> on my news agent, which would have included "To: [email-masked]"
> and one to the Usefor list using "To: [email-masked]".
> Both would have had my usual "From: ...". Both fully compliant AFAICS.

Are you saying that you deliberately composed a header to prevent my reply
to go to the Usefor mailing list?  Or is your software broken?
----------------------------------------

That was nearly twenty months ago, and was crystal clear about the
problems you are causing others.  What's the delay in fixing or
replacing your broken software (and/or as Mark has intimated,
your attitude)?

Cc: [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: Understanding response protocols
Date: Tue, 12 Oct 2004 09:55:28 -0400
To: Philip Hazel <[email-masked]>
Sender: [email-masked]

>> MFT doesn't solve the duplicates problem,
>
> It would for me.  (In the case of discussion lists like this.)
>
> I know you have stated that duplicates should be eliminated at the
> receiving end. However, this does not work for people like myself, who,
> when somebody like yourself replies both to ietf-882 AND to me, want to
> get and keep only the LIST copy of the message, not the personal copy.
>
> The point is that these two messages are *not* duplicates. One goes
> direct and the other goes via the list explosion process and thereby
> gets extra headers added, etc, etc.

Is that really the "point"?  i.e. Is that really why you want only the 
LIST copy, so you'll benefit from the extra information in the message 
header (or - as this happens also - from the removal of information 
from the message header that was done by the list)?

If someone replies to a message you sent to a list and (for whatever 
reason) the list blocks that reply, do you still want your personal 
copy of the message to be blocked also?

> I don't want the personal copy. I only want the list copy. Writing 
> logic
> to implement this at my end is not straightforward, since the messages
> may arrive in either order and possibly one will be delayed.

Yes, but hopefully you would not have to write the logic yourself.

> I want to be able to encode somewhere in the message the equivalent of
> "Please don't send me a personal copy when you reply to the list. It
> wastes net bandwidth and my time."
>
>> IMHO the thing to do is to try to build better interfaces and 
>> encourage
>> people to try them out.
>
> I understand that people are already "trying out" MFT...

Indeed.  And if I build a better interface for people to "try out" it 
will have the option of supporting MFT.  though it won't support it 
blindly, at least not by default.

I suppose I should point out that I view all of this (both the new MUA 
behavior and the duplicate processing) as potentially useful  
_research_ rather than things that should be widely implemented at this 
time.   I view MFT in the same category.

Keith

Cc: [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: Understanding response protocols
Date: Tue, 12 Oct 2004 09:33:44 -0400
To: Philip Hazel <[email-masked]>
Sender: [email-masked]

>> 3.  Blind acceptance of MFT when composing replies is a bad idea,
>> for the same reasons that blind acceptance of Reply-To is a bad idea.
>
> Well, I'm sorry, but 99% of the time when I reply to emails I blindly
> accept Reply-To. I don't have the time/patience/concern/whatever to
> scrutinize the recipients of every reply that I send.

What I'm wanting is user interfaces that make it easy for repliers to 
put the same degree of time/patience/concern/whatever into the 
recipient list of replies as they put into the content of replies.  
Right now our user interfaces encourage repliers to _not_ pay attention 
to the recipient lists.

I am reasonably certain that such software would need to have the 
option to do "blind replies" for people who cannot be bothered to read 
the recipient lists.  But it would be nice if MUAs didn't presume that 
everyone wanted to be so careless.

>  I assume that the software will "do the right thing". And I suspect 
> that I am pretty typical in this.

Software cannot "do the right thing" because the software has no idea 
_who_ you are replying to and whether that is the appropriate set of 
recipients for the content in your reply.

> What if there's no Reply-To? Is "blind acceptance" of From a bad idea?

If MUAs hid From from you when you read a message, it would be.  But 
usually From is prominently displayed both in the message itself and in 
the summary view, so it's rarely a surprise when replies go to the From 
address.  (It can still be a surprise if the MUA shows only the _name_ 
portion of the From field, and the address isn't the one that the human 
reader normally associates with the name).

Keith

To: LIST: [email-masked];
Xref: clerew local.mime:3872
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Tue, 12 Oct 2004 11:04:17 GMT
Lines: 80
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:
>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
>> 
>> 
>>>Charles Lindsey wrote:
>> 
>> 
>>>>Which is why I said "The only place where it *can* be known ...". For
>>>>sure, if the author does not know, then there is even less chance that
>>>>subsequent mailing list expanders would know
>> 
>> So please answer me. HOW is the mail expander for 'foo' supposed to know
>> about the totally separate mailing list 'bar' to which the author
>> crossposted his message.

>I never claimed that it could;

The whole purpose of this particular sub-thread is to establish what can
or cannot be done when messages are crossposted to multiple lists. I have
stated, repeatedly, that only the author is in a position to know what is
happening in that case, and to take action to deal with it (such as by
setting MFT or Reply-To to both lists).

>*that* is what is irrelevant.

It is totally relevant, since it demonstrates that if the author fails to
do anything, then neither of the mailing lists can fix it. You now seem to
be agreeing with this.

>  This
>started with your claim -- repeated above -- that only the message
>author can know when list expansion takes place.

No, it is NOT repeated above, since you have snipped it.

>> Now, supposing some author wants to raise a matter which is likely to be
>> relevant to both [email-masked] and [email-masked]. ...

>He can indicate that desire trivially:
>Reply-To: [email-masked], [email-masked]

Exactly so (or use MFT). So why have you been denying my claim that the
mailing list expanders cannot do it without assistance from the author?

>Yes, it is true that the author is in the best position to provide
>*his* recommendation for responses, if he is willing to do so, but
>that is unrelated to your claim regarding identification of if/when
>list expansion takes place.

So finally you admit that my point was correct. Mailing list expanders are
aware of all expansions etc relating to their own lists, but are totally
lost when it comes to identifying expansions made by other lists.

>The solution to the "problem" of an author indicating his recommendation
>for responses is trivially solved.  You have yet to explain what "problem"
>requires determining if/when list expansion takes place.

Because you want replies to go to all the lists crossposted to, without
duplicate personal copies going to people who are already subscribed to
one of those lists. No amount of setting policies for individual lists, or
super-smart lists as Laird has been proposing, will bring that about. The
*only* place where it can be solved is at the author's MUA. And the sort
of smartness that Dan wants to put into such MUAs, together with MFT, will
in fact do it rather nicely.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3871
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Tue, 12 Oct 2004 10:37:56 GMT
Lines: 71
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>> So please answer me. HOW is the mail expander for 'foo' supposed to know
>> about the totally separate mailing list 'bar' to which the author
>> crossposted his message. And if it does not know about 'bar', how can it
>> possibly arrange for replies to be crossposted to both lists (by
>> appropriate setting of Reply-To/MFT/whatever)?

>It can't.  But that's also mostly irrelevant, for several reasons.

>1.  The vast majority of senders are not going to want to configure their
>MUAs to know about mailing lists.  (and if you say that they should be
>automatically configured, be sure you understand the security implications)

The vast majority of senders do not regularly use mailing lists. But those
who do regularly use them (of which the subscribers to this list are
probably typical) would surely be happy to configure their MUAs,
especially if this would get rid of difficulties they were experiencing.

>2.  ... it's not clear what should happen when replying to multiple lists,
> and there is a conflict between the lists' policies.

In that case, only the author is in a position to define a policy for for
that particular communication (and hopefully for the ensuing thread).

>3.  Blind acceptance of MFT when composing replies is a bad idea, 
>for the same reasons that blind acceptance of Reply-To is a bad idea.

Nobody is proposing "blind" acceptance. By all means let there be alerts
and warnings in RED; but most recipients will be happy to follow the
author's (or the list maintainer's) recommendation.

>4.  MFT doesn't fix the problem of duplicated messages anyway.

If by "duplicated messages" you mean unwanted personal replies in addition
to replies sent to the list, then MFT does indeed solve it. The author
merely puts the list(s) and nothing else in his MFT (and a smart MUA will
automate this). Or else, if the author is silent, the list-maintainer sets
such an MFT if that is list policy.

>> Now, supposing some author wants to raise a matter which is likely to be
>> relevant to both [email-masked] and [email-masked]. So he
>> cross-posts to both lists, and ideally he would like all respondents on
>> both lists to reply to both lists. How can this be brought about?

>a) have both list addresses appear in To or CC of the original message
>b) recipients use "reply all" to reply to both lists.

No, that is the current situation, and is the cause of the mess we are in.

>What I suspect you really want is to do this without having reply 
>authors see duplicate messages.  The only way to solve that problem 
>is to have recipients' MUAs or message stores do duplicate detection.

No, MFT applied by the author can solve it. The problem with expecting
MUAs or message stores to solve it is that some users may have several
MUAs and several message stores and several layers of filtering to handle
their incoming mail. Moreover, direct replies and replies via the list
tend to arrive by different routes and at different times. Much simpler to
avoid sending the duplicates in the first place.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3870
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Tue, 12 Oct 2004 10:15:10 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:
>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>> And yet again you repeat a whole string of possible scenarios, about
>> which there is no dispute, whilst steadfastly refusing to answer the
>> question that was asked, which specifically related to "actual" behaviour.

>No Charles, this is not merely "possible scenarios". It is about
>the very real situation of comparing identifiers.

If it never actually happens in the Real World (TM), then it is not a real
situation. It is merely an interesting quirk of the way RFC 2822 was
written.

Since I have repeatedly asked you for instances of where this has actually
happened in the Real World, or for examples of existing agents which
explicitly change the case of the id-right of a msg-id when copying it
(during transit or otherwise), and you have signally failed to provide
such examples, then I conclude that you are unaware of any such examples.
Moreover, nobody else on this list has produced such examples.

Therefore, I conclude that this minor change from RFC 822 to RFC 2822 has
no practical significance in the Real World and has not lead to any
problems of interoperability. In which case, since RFC 2822 supersedes RFC
822, writers of new software should simply follow RFC 2822, and that is
the end of the matter.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 12 Oct 2004 18:54:23 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 list <[email-masked]>
Sender: [email-masked]

On Oct 11 2004, Bruce Lilly wrote:
> 
> > I do accept that the possibility is there, but am yet to be convinced that
> > it happens so frequently to render inadequate the default behaviour of 
> > sending list responses to the address that was subscribed.
> 
> What exactly is that supposed to mean? Responses go wherever the
> respondent sends them.

Whatever you meant it to mean. I responded to the following quote from you:

> An author might very well change his recommendation for responses
> on a per-message basis.  

I responded that's a possibility, but by far in most cases an author
wants responses sent to him via the mailing list, not directly, so that
ought to be default behaviour for this kind of advanced mailing list server. 

> 
> I have no idea what version of mutt you're using; the version I've
> seen (1.5.6i) has a group reply function and a reply function. I
> believe that you may be mistaken in your beliefs.
> 

I was thinking of Mutt's list-reply command. From the manual:

  list-reply (default: L)

  Reply to the current or tagged message(s) by extracting any addresses
  which match the addresses given by the ``lists or subscribe''
  commands, but also honor any Mail-Followup-To header(s) if the
  ``$honor_followup_to'' configuration variable is set.  Using this when
  replying to messages posted to mailing lists helps avoid duplicate
  copies being sent to the author of the message you are replying to.

> You're missing something fundamental.  List expanders don't work
> that way. Any proposal for a solution to the issue of communicating
> authors' response preferences that is dependent upon replacing all
> mailing list software is a non-starter.

We're going in circles. I accept that point fully, and have already 
offered to end this discussion here before. Somehow, we're still 
stuck on this thread ;-) 

Thanks for the list of relevant RFCs, I'll be reading them shortly.

> 
> Fine, but that affects what he receives from the list, not what he
> sends to wherever he sends it to.  In particular, it does nothing
> to address communicating his recommendations for responses to messages
> that he authors.

That was my intention with the 'intelligent server" all along, but it 
appears I failed to communicate that. 

-- 
Laird Breyer.

Date: Tue, 12 Oct 2004 09:26:58 +0100 (BST)
From: Philip Hazel <[email-masked]>
To: Keith Moore <[email-masked]>
cc: [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

On Mon, 11 Oct 2004, Keith Moore wrote:

> 3.  Blind acceptance of MFT when composing replies is a bad idea, 
> for the same reasons that blind acceptance of Reply-To is a bad idea.

Well, I'm sorry, but 99% of the time when I reply to emails I blindly
accept Reply-To. I don't have the time/patience/concern/whatever to
scrutinize the recipients of every reply that I send. I assume that the
software will "do the right thing". And I suspect that I am pretty
typical in this.  

What if there's no Reply-To? Is "blind acceptance" of From a bad idea?

Philip

-- 
Philip Hazel            University of Cambridge Computing Service,
[email-masked]      Cambridge, England. Phone: +44 1223 334714.

Date: Tue, 12 Oct 2004 09:20:04 +0100 (BST)
From: Philip Hazel <[email-masked]>
To: Keith Moore <[email-masked]>
cc: [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

On Mon, 11 Oct 2004, Keith Moore wrote:

> MFT doesn't solve the duplicates problem, 

It would for me.  (In the case of discussion lists like this.)

I know you have stated that duplicates should be eliminated at the 
receiving end. However, this does not work for people like myself, who, 
when somebody like yourself replies both to ietf-882 AND to me, want to 
get and keep only the LIST copy of the message, not the personal copy.

The point is that these two messages are *not* duplicates. One goes
direct and the other goes via the list explosion process and thereby 
gets extra headers added, etc, etc.

I don't want the personal copy. I only want the list copy. Writing logic 
to implement this at my end is not straightforward, since the messages 
may arrive in either order and possibly one will be delayed. 

I want to be able to encode somewhere in the message the equivalent of 
"Please don't send me a personal copy when you reply to the list. It
wastes net bandwidth and my time."

> IMHO the thing to do is to try to build better interfaces and encourage
> people to try them out.

I understand that people are already "trying out" MFT...

Philip

-- 
Philip Hazel            University of Cambridge Computing Service,
[email-masked]      Cambridge, England. Phone: +44 1223 334714.

Cc: [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: Understanding response protocols
Date: Tue, 12 Oct 2004 00:03:27 -0400
To: "D. J. Bernstein" <[email-masked]>
Sender: [email-masked]

>> The vast majority of senders are not going to want to configure their
>> MUAs to know about mailing lists.
>
> Actually, subscribers would _love_ to have an MUA that understands
> mailing lists. The current de-facto subscription process---
>
>    * send a message to the subscription address, getting all the 
> request
>      details into the right format;
>    * receive a message checking that the request really did come from
>      someone who can read this mailbox;
>    * send a response to that message, again in the right format
>
> ---becomes much easier for the user with some help from the MUA.

I will admit that if MUAs were to automate all of those details (and 
lists were to adopt a uniform interface for these functions) it might 
be very popular.  Though there has in the past been some reluctance 
from lists to do this, out of concern that this would facilitate bots 
that would subscribe to lists for various purposes - e.g. to allow them 
to spam the list, mine subscriber addresses, etc.

Keith

Date: 12 Oct 2004 02:04:57 -0000
Mail-Followup-To: [email-masked]
Automatic-Legal-Notices: See http://cr.yp.to/mailcopyright.html.
From: "D. J. Bernstein" <[email-masked]>
To: [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

Keith Moore writes:
> The vast majority of senders are not going to want to configure their
> MUAs to know about mailing lists.

Actually, subscribers would _love_ to have an MUA that understands
mailing lists. The current de-facto subscription process---

   * send a message to the subscription address, getting all the request
     details into the right format;
   * receive a message checking that the request really did come from
     someone who can read this mailbox;
   * send a response to that message, again in the right format

---becomes much easier for the user with some help from the MUA. Of
course, an MUA that manages subscriptions can also set up
Mail-Followup-To by default.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

Date: Mon, 11 Oct 2004 21:02:28 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Understanding response protocols
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> 
>>Charles Lindsey wrote:
> 
> 
>>>Which is why I said "The only place where it *can* be known ...". For
>>>sure, if the author does not know, then there is even less chance that
>>>subsequent mailing list expanders would know
> 
> 
>>Amazing -- this has been explained to you in detail several times
>>and you still don't get it!  One last time:
> 
> 
> What is so amazing is that you still continue to repeat the same
> explanation "one last time" without actually attacking the real problem.
> 
> 
>>Author A sends a message to mailbox foo. That mailbox, unbeknownst
>>to A, is a mailing list mailbox, which is expanded to multiple
>>addresses (any one of which might be yet another mailing list
>>mailbox).  Although A knows nothing about any of those lists,
>>clearly the administrator of the list associated with mailbox foo
>>knows that it expands to a mailing list, as do any administrators
>>of any such downstream lists.
> 
> 
> Yes! Yes! Yes! That is all perfectly true. It is also totally irrelevant.
> 
> So please answer me. HOW is the mail expander for 'foo' supposed to know
> about the totally separate mailing list 'bar' to which the author
> crossposted his message.

I never claimed that it could; *that* is what is irrelevant.  This
started with your claim -- repeated above -- that only the message
author can know when list expansion takes place.  It has been
explained to you that you were wrong, *why* you were wrong, and
you have had ample opportunity to admit your error or simply stop
repeating that erroneous statement, but instead you have repeated
that false claim, over and over.

> This whole argument relates to *discussion* lists (of which ietf-822 is a
> typical example). Such lists are characterized by long threads of
> discussion, with many participants, all posted to the list. It is most
> improbable that any participant is unaware that his messages are being
> sent to a list. Yes, I know there are other kinds of lists around, but
> this is the kind we are particularly concerned with.

The discussion is about responses in general; lists are a small part
of that issue, and specific lists are a tiny fraction of that small
part.

> Now, supposing some author wants to raise a matter which is likely to be
> relevant to both [email-masked] and [email-masked]. So he
> cross-posts to both lists, and ideally he would like all respondents on
> both lists to reply to both lists. How can this be brought about?

He can indicate that desire trivially:
Reply-To: [email-masked], [email-masked]

> This is a hard problem.

No, that (which not incidentally is unrelated to your repeated
false claim) is trivially solved; if the author wishes to indicate
a recommendation for responses, he simply uses the standard field
which is designed for precisely that purpose.

> But what is clear is that IF the *author* does not solve it, then
> there is NO WAY that the list expanders at either [email-masked] and
> [email-masked] can solve it (because they just do not have the
> information to do so).

The indication of response recommendations by an author is unrelated
to your original claim of where it is and is not possible to
determine that list expansion takes place.

> But it is at least possible that the author can solve it. It is
> inconceivable that he is not aware in this case that he is posting to two
> lists. He is in a position to set Reply-To/MFT/whatever to point to both
> lists. Nobody else is in a position to do that.

Yes, it is true that the author is in the best position to provide
*his* recommendation for responses, if he is willing to do so, but
that is unrelated to your claim regarding identification of if/when
list expansion takes place.

> I have been trying to point out this simple fact for nearly two weeks, but
> you persist in going on about how you cannot be sure that the author knows
> that 'foo' is a list, and that 'foo' might be expanded to lots of other
> lists. So what? Have you any better solution to propose?

The solution to the "problem" of an author indicating his recommendation
for responses is trivially solved.  You have yet to explain what "problem"
requires determining if/when list expansion takes place.

Date: Mon, 11 Oct 2004 20:38:13 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>>>Have you heard of any case where this esoteric change of semantics
>>>actually caused a problem?
> 
> 
>>That problem does occur for anybody attempting to write any
>>software which involves comparisons of such identifiers, which
>>is exactly the situation initially posed by Jacob. A choice
>>must be made between comparing identifier right-hand sides
>>as domain names (viz. case-insensitive) or as RFC 2822 id-right
>>(2822 doesn't say whether that's case-sensitive or not). Under
>>the circumstances, comparing as case-insensitive domain names
>>seems the most reasonable course of action; it is consistent
>>with the 20+ year history of identifier RHS as domain names and
>>it is consistent with RFC 2822's recommendation to use domain
>>names.
> 
> 
> And yet again you repeat a whole string of possible scenarios, about
> which there is no dispute, whilst steadfastly refusing to answer the
> question that was asked, which specifically related to "actual" behaviour.

No Charles, this is not merely "possible scenarios". It is about
the very real situation of comparing identifiers. Either a case-
sensitive or a case-insensitive comparison could be made; one
must decide which. Under RFC 822 and its predecessors, where the
RHS of an identifier was always a domain name, the correct
decision is to compare those domain names case-insensitively. The
change in RFC 2822 -- where we are told that the RHS might or
might not be a domain name, and if it is not, we are not told
whether or not it is case-sensitive -- makes that decision
more difficult.  It is an issue of the design of such a
comparator, irrespective of the source of the identifiers.

Date: Mon, 11 Oct 2004 20:28:20 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: [email-masked]
CC: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

Laird Breyer wrote:

> I do accept that the possibility is there, but am yet to be convinced that
> it happens so frequently to render inadequate the default behaviour of 
> sending list responses to the address that was subscribed.

What exactly is that supposed to mean? Responses go wherever the
respondent sends them.

> Is it not true that in this scenario, the author still depends on the
> cooperation of any respondents to honour his specification, rather
> than reply to the list regardless?

You mean recommendation, not specification. A respondent always
has the prerogative to send responses wherever he pleases.

> For example, I use Mutt, and have a
> key for list replies and a key for ordinary replies. When I press the former,
> my reply always goes to the list, I believe.

I have no idea what version of mutt you're using; the version I've
seen (1.5.6i) has a group reply function and a reply function. I
believe that you may be mistaken in your beliefs.

> If such cooperation is required now for your example, then the same
> would be true with an "intelligent" list. Namely: if I want
> respondents to not reply through the list, I must communicate the
> requirement to their MUA and I still depend on them to follow my
> wishes. Or am I missing something?

You're missing something fundamental.  List expanders don't work
that way. Any proposal for a solution to the issue of communicating
authors' response preferences that is dependent upon replacing all
mailing list software is a non-starter.

>>Or he might copy one list on a message of potential interest to that
>>list based on the topic of a discussion taking place on a different
>>list, with responses directed to the list on which the discussion is
>>taking place.
> 
> 
> Why would the members of the copied list follow this recommendation
> and respond on the primary discussion list?

Because that's where the discussion is.

> I understand that is
> probably desirable, but what mechanism will actually enforce this
> behaviour now?

There is no enforcement of recommendations.  That's why they're
recommendations and not absolute requirements.

> Is it Cc: ? 

No, the standard place for an author to indicate response
recommendations is the standard Reply-To field.  The Cc field has
nothing to do with response recommendations.

> Also, this may be my inexperience with mailing list issues talking, but
> I don't see anything wrong with each list treating non-member addresses
> uniformly according to its nonmember rules (chosen by each list administrator).

If such hypothetical rules for such a hypothetical piece of list
software included sending multiple unsolicited messages to a
non-member, that non-member would probably (and justifiably)
regard that list software's operator as a spammer.

> But I think this discussion is becoming very technical, and I'm getting lost.
> Can you perhaps recommend some review document which describes mailing list
> problems and various historical solutions, if there is one?  

Start with RFC 1958 for the Internet Architecture, then RFCs 821
and 2821 which describe list expansion. See RFCs 2369 and 2919
for list-specific message header fields.  RFC 3834 regarding
automatic responses.  See RFC 3117 for some advice regarding
application protocols.  Some historical information exists in
RFC 1429.  RFC 1211 deals with administrative issues.

> This makes no sense to me. On the one hand, you seem to oppose mail
> propagation decisions by mailing lists because they may differ from
> what the author wants, but on the other you fully accept the
> prerogative of respondents to do as they please.

mailing list software != human respondents

> If the prospective respondent makes a configuration choice directly
> with the mailing list, and empowers it to alter the propagation of
> received messages destined for himself, surely that's his prerogative?

Fine, but that affects what he receives from the list, not what he
sends to wherever he sends it to.  In particular, it does nothing
to address communicating his recommendations for responses to messages
that he authors.

Date: Mon, 11 Oct 2004 13:27:51 -0400
From: Keith Moore <[email-masked]>
To: "Charles Lindsey" <[email-masked]>
Cc: [email-masked], [email-masked];
Subject: Re: Understanding response protocols
Sender: [email-masked]

> >Author A sends a message to mailbox foo. That mailbox, unbeknownst
> >to A, is a mailing list mailbox, which is expanded to multiple
> >addresses (any one of which might be yet another mailing list
> >mailbox).  Although A knows nothing about any of those lists,
> >clearly the administrator of the list associated with mailbox foo
> >knows that it expands to a mailing list, as do any administrators
> >of any such downstream lists.
> 
> Yes! Yes! Yes! That is all perfectly true. It is also totally irrelevant.
> 
> So please answer me. HOW is the mail expander for 'foo' supposed to know
> about the totally separate mailing list 'bar' to which the author
> crossposted his message. And if it does not know about 'bar', how can it
> possibly arrange for replies to be crossposted to both lists (by
> appropriate setting of Reply-To/MFT/whatever)?

It can't.  But that's also mostly irrelevant, for several reasons.

1.  The vast majority of senders are not going to want to configure their
MUAs to know about mailing lists.  (and if you say that they should be
automatically configured, be sure you understand the security implications)

2.  Different lists exist for different purposes, and have different 
conventions for whether it is appropriate to reply to the list, or to
the author, or to copy other lists on replies.  Even if this information
is available to the user agent (and yes, that is a solvable problem), 
it's not clear what should happen when replying to multiple lists, and 
there is a conflict between the lists' policies.

3.  Blind acceptance of MFT when composing replies is a bad idea, 
for the same reasons that blind acceptance of Reply-To is a bad idea.

4.  MFT doesn't fix the problem of duplicated messages anyway.

> Now, supposing some author wants to raise a matter which is likely to be
> relevant to both [email-masked] and [email-masked]. So he
> cross-posts to both lists, and ideally he would like all respondents on
> both lists to reply to both lists. How can this be brought about?

a) have both list addresses appear in To or CC of the original message
b) recipients use "reply all" to reply to both lists.

What I suspect you really want is to do this without having reply 
authors see duplicate messages.  The only way to solve that problem 
is to have recipients' MUAs or message stores do duplicate detection.

(and yes there are problems even with that if you try to use To or CC
fields to classify your mail.  but - while you are free to do so -
this is generally not a good idea anyway.  if you want to classify
incoming mail based on whether it went to a list you are much better
off using list-id, return-path, or subaddresses)

> This is a hard problem. It is entirely possible it cannot always be
> solved. But what is clear is that IF the *author* does not solve it, then
> there is NO WAY that the list expanders at either [email-masked] and
> [email-masked] can solve it (because they just do not have the
> information to do so).

If you want MUAs to be able to learn information about recipient policies,
use the domains to look up NAPTR records that map the addresses onto
http URLs, and then use HTTP (or preferably, something more efficient)
to retrieve metadata about the addresses.  This is very useful, very
flexible, and not limited to lists.  And no, it doesn't work when you're
offline but user agents can cache the address information, and/or
pre-load the information when they download mail.

-Keith

Date: Mon, 11 Oct 2004 13:04:43 -0400
From: Keith Moore <[email-masked]>
To: "Charles Lindsey" <[email-masked]>
Cc: [email-masked], LIST: [email-masked];
Subject: Re: Understanding response protocols
Sender: [email-masked]

> >more to the point, careful thinking is necessary.  we don't understand
> >what we need yet. 
> 
> But the problem we face is that Bruce refuses to consider any proposal
> unless there us an official document describing it 

It's too early to consider fully-formed proposals anyway.  If you want to
write up an internet-draft to circulate ideas, feel free.

> Enough people have indicated that they understand MFT sufficiently well to
> be able to say that it looks like a workable solution. 

MFT doesn't solve the duplicates problem, and it makes some other problems
worse.  The fundamental problem is one of poor user interfaces.  You can't
fix this by defining more protocol elements.

IMHO the thing to do is to try to build better interfaces and encourage
people to try them out.  Internet-Drafts are not the best medium for this.

Keith

To: LIST: [email-masked];
Xref: clerew local.mime:3860
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Mon, 11 Oct 2004 11:13:28 GMT
Lines: 50
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:

>> Yes, I know all that. You have demonstrated that a problem could,
>> _in_principle_ arise. But my question, which I repeat again and which you
>> still have not answered, was:
>> 
>> Have you heard of any case where this esoteric change of semantics
>> actually caused a problem?

>That problem does occur for anybody attempting to write any
>software which involves comparisons of such identifiers, which
>is exactly the situation initially posed by Jacob. A choice
>must be made between comparing identifier right-hand sides
>as domain names (viz. case-insensitive) or as RFC 2822 id-right
>(2822 doesn't say whether that's case-sensitive or not). Under
>the circumstances, comparing as case-insensitive domain names
>seems the most reasonable course of action; it is consistent
>with the 20+ year history of identifier RHS as domain names and
>it is consistent with RFC 2822's recommendation to use domain
>names.

And yet again you repeat a whole string of possible scenarios, about
which there is no dispute, whilst steadfastly refusing to answer the
question that was asked, which specifically related to "actual" behaviour.

Message identifiers are generated by the agent that creates the message.
The id-right will usually be a domain name, and it may be in UC or LC. They
may get copied during transport, gatewaying, resending, or when creating
References fields for a reply. Various agents may compare them (e.g. when
threading or when eliminating duplicates).

For anything to go wrong, some agent somewhere has to deliberately change
the case of an id-right during a copying operation. If no agent ever does
that, then no problem can ever arise.

Now, please answer me. Do you know of any ACTUAL agent that changes the
case of an id-right when making such a copy?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3863
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fw: Angle brackets surrounding Content-ID
Date: Mon, 11 Oct 2004 11:57:28 GMT
Lines: 31
Sender: [email-masked]

In <005d01c4ad37$c1f8efa0$6401a8c0@hdev1> "Hector Santos" <[email-masked]> writes:

>PS: There should not be a Newsgroups: header when sending your email.
>Atleast this MUA is thinking it is a newsgroup article when replying.   We
>have a similar newsgroup emulator for email/mailing list.  Your news reader
>is doing the right thing, but whatever is on the backend, needs to clean it
>up before it exports it back into the email system as a "email message" not
>as a news article.    I am just saying is that the stupid MUA will use the
>Newsgroups: header an email vs. newsgroup indicator.

An MUA that chooses whether to followup to Usenet or to reply using email
simply on the basis of whether a Newsgroups-header is present is severely
broken, since it is normal behaviour for a message which is both posted to
Usenet and emailed to individual recipients to include a Newsgroup-header
(how else are recipients to know that both media were involved).

I agree that, in this case where the newsgroup in question is purely local
to my own server, my backend could well have edited out the superfluous
header, and I really ought to fix that (and maybe I shall make it put a
Reply-To in at the same time).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3861
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Mon, 11 Oct 2004 11:22:54 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>[response to another rude reply that didn't CC the author of the subject 
>message]

>> >perhaps.  But the semantics and intended method are a lot harder to
>> >define and get right than the syntax of the header field.
>> 
>> Indeed, careful writing is necessary.

>more to the point, careful thinking is necessary.  we don't understand
>what we need yet. 

But the problem we face is that Bruce refuses to consider any proposal
unless there us an official document describing it (e.g. most of us except
Bruce are happy to discuss MFT on the basis of Jacob's expired draft or
Dan's informal descriptions). Whereas you are not prepared for any draft
to be written until we have a full understanding of everything.

Enough people have indicated that they understand MFT sufficiently well to
be able to say that it looks like a workable solution. There is no other
solution on the table, except for relying on Reply-To (and Bruce seems to
be the only one who believes that would work). So it seems to me that the
only way forward is for somebody to write a draft. Then we can all tear it
to shreds.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3862
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Mon, 11 Oct 2004 11:49:51 GMT
Lines: 68
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:

>> Which is why I said "The only place where it *can* be known ...". For
>> sure, if the author does not know, then there is even less chance that
>> subsequent mailing list expanders would know

>Amazing -- this has been explained to you in detail several times
>and you still don't get it!  One last time:

What is so amazing is that you still continue to repeat the same
explanation "one last time" without actually attacking the real problem.

>Author A sends a message to mailbox foo. That mailbox, unbeknownst
>to A, is a mailing list mailbox, which is expanded to multiple
>addresses (any one of which might be yet another mailing list
>mailbox).  Although A knows nothing about any of those lists,
>clearly the administrator of the list associated with mailbox foo
>knows that it expands to a mailing list, as do any administrators
>of any such downstream lists.

Yes! Yes! Yes! That is all perfectly true. It is also totally irrelevant.

So please answer me. HOW is the mail expander for 'foo' supposed to know
about the totally separate mailing list 'bar' to which the author
crossposted his message. And if it does not know about 'bar', how can it
possibly arrange for replies to be crossposted to both lists (by
appropriate setting of Reply-To/MFT/whatever)?

This whole argument relates to *discussion* lists (of which ietf-822 is a
typical example). Such lists are characterized by long threads of
discussion, with many participants, all posted to the list. It is most
improbable that any participant is unaware that his messages are being
sent to a list. Yes, I know there are other kinds of lists around, but
this is the kind we are particularly concerned with.

Now, supposing some author wants to raise a matter which is likely to be
relevant to both [email-masked] and [email-masked]. So he
cross-posts to both lists, and ideally he would like all respondents on
both lists to reply to both lists. How can this be brought about?

This is a hard problem. It is entirely possible it cannot always be
solved. But what is clear is that IF the *author* does not solve it, then
there is NO WAY that the list expanders at either [email-masked] and
[email-masked] can solve it (because they just do not have the
information to do so).

But it is at least possible that the author can solve it. It is
inconceivable that he is not aware in this case that he is posting to two
lists. He is in a position to set Reply-To/MFT/whatever to point to both
lists. Nobody else is in a position to do that.

I have been trying to point out this simple fact for nearly two weeks, but
you persist in going on about how you cannot be sure that the author knows
that 'foo' is a list, and that 'foo' might be expanded to lots of other
lists. So what? Have you any better solution to propose?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Sat, 9 Oct 2004 12:07:30 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 list <[email-masked]>
Sender: [email-masked]

On Oct 08 2004, Bruce Lilly wrote:

> > True, but how often does an author change his response addresses
> > when posting messages to the same list over and over? Your return
> > path [email-masked] hasn't changed on this list for months... 
> 
> An author might very well change his recommendation for responses
> on a per-message basis.  Most of my messages to this list recommend

I do accept that the possibility is there, but am yet to be convinced that
it happens so frequently to render inadequate the default behaviour of 
sending list responses to the address that was subscribed.

> it is certainly conceivable that an author
> might specify responses sent directly to him (i.e. the author), with
> the intention of subsequently posting a summary of those responses
> to the list. 

Is it not true that in this scenario, the author still depends on the
cooperation of any respondents to honour his specification, rather
than reply to the list regardless? For example, I use Mutt, and have a
key for list replies and a key for ordinary replies. When I press the former,
my reply always goes to the list, I believe.

If such cooperation is required now for your example, then the same
would be true with an "intelligent" list. Namely: if I want
respondents to not reply through the list, I must communicate the
requirement to their MUA and I still depend on them to follow my
wishes. Or am I missing something?

> Or he might copy one list on a message of potential interest to that
> list based on the topic of a discussion taking place on a different
> list, with responses directed to the list on which the discussion is
> taking place.

Why would the members of the copied list follow this recommendation
and respond on the primary discussion list? I understand that is
probably desirable, but what mechanism will actually enforce this
behaviour now? Is it Cc: ? 

Also, this may be my inexperience with mailing list issues talking, but
I don't see anything wrong with each list treating non-member addresses
uniformly according to its nonmember rules (chosen by each list administrator).
Whether such an address refers to a person or another mailing list should
not matter.

But I think this discussion is becoming very technical, and I'm getting lost.
Can you perhaps recommend some review document which describes mailing list
problems and various historical solutions, if there is one?  

> > 
> > E.g. if my MUA sets Reply-To: and expects responses to that address,
> > but the responder's MUA ignores Reply-To: for some reason, and only
> > uses From: say, then the responder may not reply correctly.
> 
> "Correctly" doesn't enter into the discussion. The issue is whether
> or not an author's recommendation for responses can be expressed,
> and that is precisely the semantics of the Reply-To field. If a
> respondent chooses to send a response elsewhere, that is his
> prerogative.

This makes no sense to me. On the one hand, you seem to oppose mail
propagation decisions by mailing lists because they may differ from
what the author wants, but on the other you fully accept the
prerogative of respondents to do as they please.

If the prospective respondent makes a configuration choice directly
with the mailing list, and empowers it to alter the propagation of
received messages destined for himself, surely that's his prerogative?

In other words, for mailing list traffic passing through the list
server, I'm saying that the rights of an author stop when the list
server receives the message, and the rights of a respondent begin when
the list server is about to distribute this message. I'm not saying anything
for off-list traffic.

-- 
Laird Breyer.

Date: Fri, 08 Oct 2004 18:04:50 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: [email-masked]
CC: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

Laird Breyer wrote:

> True, but how often does an author change his response addresses
> when posting messages to the same list over and over? Your return
> path [email-masked] hasn't changed on this list for months... 

An author might very well change his recommendation for responses
on a per-message basis.  Most of my messages to this list recommend
responses to the list, but it is certainly conceivable that an author
might specify responses sent directly to him (i.e. the author), with
the intention of subsequently posting a summary of those responses
to the list. Or he might copy one list on a message of potential
interest to that list based on the topic of a discussion taking place
on a different list, with responses directed to the list on which the
discussion is taking place.

> True. But a human responder cannot know a priori the preferences of 
> the original message author, unless this author's MUA was configured
> to send messages in a way which is interpreted identically by the 
> responder's MUA, ie both comply to exactly the same standard. 
> 
> E.g. if my MUA sets Reply-To: and expects responses to that address,
> but the responder's MUA ignores Reply-To: for some reason, and only
> uses From: say, then the responder may not reply correctly.

"Correctly" doesn't enter into the discussion. The issue is whether
or not an author's recommendation for responses can be expressed,
and that is precisely the semantics of the Reply-To field. If a
respondent chooses to send a response elsewhere, that is his
prerogative.

Date: Fri, 08 Oct 2004 17:51:46 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Nick Ing-Simmons <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Sender: [email-masked]

Nick Ing-Simmons wrote:

> Standards wise that is probably true.
> But pragmatically whatever generates the message-id isn't 
> likely to change how it cases the strings it builds.

Bear in mind that an identifier may have passed through some
gateway, which may have made such a change, identifiers may be
obtained from diverse sources (cid URIs, mid URIs, mailto
URIs, from Content-Type or Content-Disposition parameters,
from Message-ID, References, In-Reply-To, Original-Message-ID,
Resent-Message-ID, Supersedes, or Delivery-Report-Content-Original
header fields, etc.), several of which may involve encoding
of various types (RFC 2231, RFC 2369).

> So if case differs message was presumably generated by 
> at last a different instance of the thing - so it is a different 
> message.

Not necessarily. One identifier may have been obtained from a
URI via HTTP, having had RFC 2369 encoding reversed; another
may have been obtained from a parameter (having had RFC 2231
encoding reversed) in a field of a message that may have
passed through one or more gateways.  Or transcribed from
a spoken conversation (over the telephone for example), or
from written material.

Date: Fri, 08 Oct 2004 17:42:39 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Sender: [email-masked]

Charles Lindsey wrote:

> Yes, I know all that. You have demonstrated that a problem could,
> _in_principle_ arise. But my question, which I repeat again and which you
> still have not answered, was:
> 
> Have you heard of any case where this esoteric change of semantics
> actually caused a problem?

That problem does occur for anybody attempting to write any
software which involves comparisons of such identifiers, which
is exactly the situation initially posed by Jacob. A choice
must be made between comparing identifier right-hand sides
as domain names (viz. case-insensitive) or as RFC 2822 id-right
(2822 doesn't say whether that's case-sensitive or not). Under
the circumstances, comparing as case-insensitive domain names
seems the most reasonable course of action; it is consistent
with the 20+ year history of identifier RHS as domain names and
it is consistent with RFC 2822's recommendation to use domain
names.

Date: Fri, 08 Oct 2004 17:33:30 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Understanding response protocols
Sender: [email-masked]

Charles Lindsey wrote:

> Which is why I said "The only place where it *can* be known ...". For
> sure, if the author does not know, then there is even less chance that
> subsequent mailing list expanders would know

Amazing -- this has been explained to you in detail several times
and you still don't get it!  One last time:
Author A sends a message to mailbox foo. That mailbox, unbeknownst
to A, is a mailing list mailbox, which is expanded to multiple
addresses (any one of which might be yet another mailing list
mailbox).  Although A knows nothing about any of those lists,
clearly the administrator of the list associated with mailbox foo
knows that it expands to a mailing list, as do any administrators
of any such downstream lists.

To: LIST: [email-masked];
Xref: clerew local.mime:3850
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Fri, 8 Oct 2004 11:21:24 GMT
Lines: 72
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:
>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
>> 
>> 
>>>Charles Lindsey wrote:
>>>
>>>>In <[email-masked]> Bruce Lilly <[email-masked]> writes:
>>>>
>>>>
>>>>
>>>>>Charles Lindsey wrote:
>>>>
>>>>
>>>>>>The only place where it is known that a crosspost is occurring is at the
>>>>>>original sender.
>>>>
>>>>
>>>>>Not true, on at least two counts:
>>>>
>>>>
>>>>>1. an author might not know that a particular mailbox expands to a list.
>>>>
>>>>
>>>>Sure, the author might not know [...]
>> 
>> 
>>>So you agree that your original statement was untrue.
>> 
>> 
>> OK, if you want to be unnecessarily pedantic:
>> 
>>    The only place where it can be known that a crosspost is occurring is at the
>>    original sender.

>No, Charles. You still have it wrong -- the author doesn't
>necessarily know about all instances of list expansion.

Yes, I KNOW that.

Which is why I said "The only place where it *can* be known ...". For
sure, if the author does not know, then there is even less chance that
subsequent mailing list expanders would know (they would recognize their
own list(s), of course, but we are talking about crossposts between
otherwise unrelated lists).

>>>That's obviously wrong; clearly a downstream list expander (or
>>>its maintainer) is aware that list expansion is taking place.

For his own lists, yes, but not for crossposted lists unrelated to his
own.

>> And I am still awaiting your solution.

>You still haven't said exactly what the problem is that requires
>a solution.

The problem is arranging that, where a message is crossposted to several
unrelated lists, replies get sent to them all, even when the recipients
are unaware of the situation.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Fri, 8 Oct 2004 11:56:53 -0400
From: Keith Moore <[email-masked]>
To: "Charles Lindsey" <[email-masked]>
Cc: [email-masked], LIST: [email-masked];
Subject: Re: Understanding response protocols
Sender: [email-masked]

[response to another rude reply that didn't CC the author of the subject 
message]

> >perhaps.  But the semantics and intended method are a lot harder to
> >define and get right than the syntax of the header field.
> 
> Indeed, careful writing is necessary.

more to the point, careful thinking is necessary.  we don't understand
what we need yet. 

From: "Hector Santos" <[email-masked]>
To: "IETF-822" <[email-masked]>
Cc: <[email-masked]>
Subject: Fw: Angle brackets surrounding Content-ID
Date: Fri, 8 Oct 2004 09:07:22 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Charles Lindsey" <[email-masked]>
Newsgroups: local.mime
To: <LIST: [email-masked]>
Sent: Friday, October 08, 2004 6:49 AM
Subject: Re: Angle brackets surrounding Content-ID

> Have you heard of any case where this esoteric change of semantics
> actually caused a problem?
>
> IOW, is this a theoretical curiosity, or a cause of real problems in
> meatspace?

Well,  traditionally, for the few networks I've been involved with, the
<id-right> has always been the domain and besides for tracking and
threading,  it might be used for dupe processing at the original domain
source.

In other words,  the normal approach is to check for your own domain, not
others.  The thinking is that each domain will take care of its own.

For netmail (email), it is not as important because most networks don't
bother to dupe check person-to-person mail, but this is not the case for
conference mail;  news, echos, etc.  So when my news gateway program is
importing new articles, it expects the id-right to be the domain. If a
incoming id-right is your own domain, then that is considered a dupe,
skipped and/or not imported.

So what if someone else does something different with id-right?

I guess, as long as you are consistent for your own mail, processing and
protecting you own mail from coming back in and more importantly, not
accidentally reexporting, well,  I guess it should not really matter for
other's people mail.    But I think it should remain a standard expectation
across the board.  I mean, the odds are low, but if its not inconsistent you
risk conflicts with similar id-rights.

PS: There should not be a Newsgroups: header when sending your email.
Atleast this MUA is thinking it is a newsgroup article when replying.   We
have a similar newsgroup emulator for email/mailing list.  Your news reader
is doing the right thing, but whatever is on the backend, needs to clean it
up before it exports it back into the email system as a "email message" not
as a news article.    I am just saying is that the stupid MUA will use the
Newsgroups: header an email vs. newsgroup indicator.

Thanks

---
Hector Santos
WINSERVER "Wildcat! Interactive Net Server"
http://www.santronics.com

To: LIST: [email-masked];
Xref: clerew local.mime:3849
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Fri, 8 Oct 2004 11:09:40 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>> >I don't think that's true at all.  The way they tend to be implemented,
>> >these extra header fields just make things more confusing. 
>> 
>> Well the way(s) in which Reply-To is implemented certainly have that
>> effect, as you and others have pointed out.
>> 
>> Which merely argues for a new header with a carefully written semantics
>> and intended method of use.

>perhaps.  But the semantics and intended method are a lot harder to
>define and get right than the syntax of the header field.

Indeed, careful writing is necessary.

But I think this discussion has now reached the point where we need a
concrete draft to look at and criticize. I would suggest that Jacob Palme
is the right person to write it (if he is listening and willing) since he
was the author of the last draft written on this subject. Possibly with
assistance from DJB.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3847
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Fri, 8 Oct 2004 10:49:17 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:
>> 
>> But you have not answered the question. We were discussing the <id-right>
>> in <msg-id>s. Have you heard of any case where this esoteric change of
>> semantics actually caused a problem?

>I have indeed answered exactly that question, in detail.
>For example:
>Interpreting a msg-id RHS as RFC 2822 id-right means that one
>cannot determine whether <[email-masked]> matches <[email-masked]>
>because RFC 2822 doesn't specify whether or not id-right is
>a domain name. It might be or it might not be.  In the event
>that it is not, RFC 2822 is silent on whether or not case
>matters.  Under RFC 822, the RHS of a msg-id was always a
>domain, and as domain names are always case-insensitive, the
>two msg-ids above would match under RFC 822. Under RFC 2822,
>one cannot tell whether or not they match.

Yes, I know all that. You have demonstrated that a problem could,
_in_principle_ arise. But my question, which I repeat again and which you
still have not answered, was:

Have you heard of any case where this esoteric change of semantics
actually caused a problem?

IOW, is this a theoretical curiosity, or a cause of real problems in
meatspace?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3848
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Fri, 8 Oct 2004 11:03:21 GMT
Lines: 38
Sender: [email-masked]

In <KEAa/[email-masked]> Arnt Gulbrandsen <[email-masked]> writes:

>Do you think it is better once all those fun-loving translators have 
>been at it? We've seen "Re" mutate into dozens of translated prefixes. 
>I see no reason to think that your proposed keyword would escape the 
>fate of Re.

Yes, we have been trying hard in the USEFOR WG to limit it to a single
"Re", with a strong hint that even that is a Bad Thing (though the
practice is too entrenched to forbid it entirely).

OTOH, the IMAP people are still writing drafts that document all the known
variants of "Re" and "Fwd" and expecting software to deal with them :-( .

However the particular proposal I have made here has a slightly better
chance of working, since the published submission address of a mailing
list could include that group syntax LIST (and even if some rogue mailing
list used LISTE, it would still be recognized by users of that list).

But I agree it is not a pretty solution, and so it could never be more
than a _convention_ established in some BCP document.

However, the problem remains (several posters here have indicated that
lots of problems would go away if there was some way to distinguish the
list addresses out of a bunch of addresses culled from the various From:,
To:, Cc:, Reply-To: and MFT headers found in the original message). So if
anyone has a better solution, then please let us hear it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Cc: Charles Lindsey <[email-masked]>, 822 <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
To: [email-masked]
Date: Fri, 08 Oct 2004 11:08:34 +0100
From: Nick Ing-Simmons <[email-masked]>
Reply-To: Nick Ing-Simmons <[email-masked]>
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:
>> 
>> But you have not answered the question. We were discussing the <id-right>
>> in <msg-id>s. Have you heard of any case where this esoteric change of
>> semantics actually caused a problem?
>
>I have indeed answered exactly that question, in detail.
>For example:
>Interpreting a msg-id RHS as RFC 2822 id-right means that one
>cannot determine whether <[email-masked]> matches <[email-masked]>
>because RFC 2822 doesn't specify whether or not id-right is
>a domain name. It might be or it might not be.  In the event
>that it is not, RFC 2822 is silent on whether or not case
>matters.  Under RFC 822, the RHS of a msg-id was always a
>domain, and as domain names are always case-insensitive, the
>two msg-ids above would match under RFC 822. Under RFC 2822,
>one cannot tell whether or not they match.

Standards wise that is probably true.
But pragmatically whatever generates the message-id isn't 
likely to change how it cases the strings it builds.
So if case differs message was presumably generated by 
at last a different instance of the thing - so it is a different 
message.

Only the bit after the '@' is a domain in any case, so 
transport shouldn't mess with case.

>
>There are other, related issues as well.

Date: Fri, 8 Oct 2004 13:20:45 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 list <[email-masked]>
Cc: [email-masked]
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 list <[email-masked]>, [email-masked]
Sender: [email-masked]

I'm not sure if we want to continue this on the list, but
I'll respond to open questions publically.

> 
> >>For somebody not on the list? Obviously he shouldn't receive list
> >>messages unrelated to any that he might have submitted. If he wants
> >>copies of any extended discussion (such as the example below) he'll
> >>have to indicate so and rely on list members to cooperate with that
> >>request.
> >>
> > 
> > 
> > That is a problem which an intelligent server as proposed can address
> > automatically.
> 
> I fail to see how that will work (i.e. automatically) for list
> non-members.

Remember my assumptions: 1) the server has an internal forest of threads,
2) a new message submitted to the list is attached to the appropriate thread
when received by the list server.

With these two assumptions, and if the list administrator configures the
list server to do so, then 

3) upon receipt of 2), the list server walks back through the parent
nodes of the thread, obtaining node ids. Each node id points to a
stored rfc2822 message, where the author for that comment is to be
found in the usual way. After walking the thread parent nodes, the
list server now has a set of parent message authors who might be
interested in this new message. Some of these authors may be
non-members, it doesn't matter.

4) the list server propagates the message obtained in 2) to each list
member, and also sends the message to the authors listed in 3).

That is the basic idea, contingent on solving 1) and 2).

> 
> >>basic function of a mailing list is to facilitate human group
> >>communication by distributing messages to that group. The humans
> >>involved need to indicate (to the other humans) their individual
> >>preferences for responses [do they want responses to go only to
> >>the list, do they want personal responses only, do they want both,
> >>etc.]. 
> > 
> > 
> > But the list server can act as a proxy for these mundane tasks.  If
> > the human can indicate his response preferences inside the message he
> > is sending, then surely the human can tell the server those response
> > preferences once, and the server can then insert these inside all the
> > messages being transmitted to the list.
> 
> You seem to be assuming that each author will want to indicate
> the same response addresses for each message, which is not
> necessarily the case.

True, but how often does an author change his response addresses
when posting messages to the same list over and over? Your return
path [email-masked] hasn't changed on this list for months... 

A more practical issue, imho, would be whether and how such an author
can, on the rare occasions where a different response preference is
wanted, indicate to the list server that his normal list preferences
should be overridden.

> 
> The point is that expecting mailing list software to do more
> than expand distribution has authentication implications
> above and beyond the issue of how recipients deal with (lack
> of) authentication.

For list members, the extra services offered by the list server only
affect each member according to their configured preferences. The
list server is individually authorized to perform certain actions or
not.

In the system envisaged, a list member cannot send a message to the
list in such a way as to override other members' configured
preferences, unless say he harvests all email addresses and sends
messages directly, bypassing the list server. So this list member
doesn't gain escalated privileges compared with the current situation
with ordinary list expanders, and I therefore don't see an extra
authentication burden.

If as a list member, all I want is a single copy of each
message say, the server knows that and will only send a single copy
of any messages submitted to the list.

If as a list member, I want extra courtesy notification of all the
replies and subreplies to one of my messages say, then the server will
send me an extra copy of any messages which are attached to some
reply-thread descending from one of my messages, by the algorithm I
outlined above.

As a non-list-member, my preferences are decided by the list
administrator generically. For example, as a non-list-member, I don't
receive each message sent to the list by definition. But if the
administrator gave non-list-members the courtesy notification option,
then I get sent copies of all messages attached to some reply-thread
descending from one of my messages to the list, by the algorithm I
outlined above.

> A recipient who wishes to prepare some sort of response needs to
> decide whether to respond in accordance with any (possibly
> message-specific) recommendation made by the original message author
> and act accordingly, and that would not change with the hypothetical
> "intelligent" remailer as I understand your description to date.

True. But a human responder cannot know a priori the preferences of 
the original message author, unless this author's MUA was configured
to send messages in a way which is interpreted identically by the 
responder's MUA, ie both comply to exactly the same standard. 

E.g. if my MUA sets Reply-To: and expects responses to that address,
but the responder's MUA ignores Reply-To: for some reason, and only
uses From: say, then the responder may not reply correctly.

By contrast, with the list server as a middle man, my MUA sends mail
to the list address, and expects responses according to configured 
list preferences. The message is propagated, responded to by responders
sending mail to the list server only, and the list server handles the 
details of my preferences as required, without involving the responder's MUA.

Nothing stops me or my responder to bypass the list server and communicate
out-of-list in an ad-hoc manner.

-- 
Laird Breyer.

Date: Thu, 07 Oct 2004 19:22:25 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> 
>>Charles Lindsey wrote:

>>>Have you heard of any cases where this change has caused a problem?
> 
> 
>>I have mentioned specific issues where RFC 2822 causes problems.
> 
> 
> But you have not answered the question. We were discussing the <id-right>
> in <msg-id>s. Have you heard of any case where this esoteric change of
> semantics actually caused a problem?

I have indeed answered exactly that question, in detail.
For example:
Interpreting a msg-id RHS as RFC 2822 id-right means that one
cannot determine whether <[email-masked]> matches <[email-masked]>
because RFC 2822 doesn't specify whether or not id-right is
a domain name. It might be or it might not be.  In the event
that it is not, RFC 2822 is silent on whether or not case
matters.  Under RFC 822, the RHS of a msg-id was always a
domain, and as domain names are always case-insensitive, the
two msg-ids above would match under RFC 822. Under RFC 2822,
one cannot tell whether or not they match.

There are other, related issues as well.

Date: Thu, 07 Oct 2004 19:14:50 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Understanding response protocols
Sender: [email-masked]

Charles Lindsey wrote:

> No, what I have proposed still falls withint the existing syntax of
> <address>.

You have proposed changing the semantics of display names within
some addresses drastically; from text to a protocol keyword. That's
an incompatible change.

Date: Thu, 07 Oct 2004 19:09:31 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Understanding response protocols
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> 
>>Charles Lindsey wrote:
>>
>>>In <[email-masked]> Bruce Lilly <[email-masked]> writes:
>>>
>>>
>>>
>>>>Charles Lindsey wrote:
>>>
>>>
>>>>>The only place where it is known that a crosspost is occurring is at the
>>>>>original sender.
>>>
>>>
>>>>Not true, on at least two counts:
>>>
>>>
>>>>1. an author might not know that a particular mailbox expands to a list.
>>>
>>>
>>>Sure, the author might not know [...]
> 
> 
>>So you agree that your original statement was untrue.
> 
> 
> OK, if you want to be unnecessarily pedantic:
> 
>    The only place where it can be known that a crosspost is occurring is at the
>    original sender.

No, Charles. You still have it wrong -- the author doesn't
necessarily know about all instances of list expansion.

>>>But for sure if the author is not aware of it, then agents further
>>>down the chain will be even less aware of it.
> 
> 
>>That's obviously wrong; clearly a downstream list expander (or
>>its maintainer) is aware that list expansion is taking place.
> 
> 
> If a message is crossposted to List-A and List-B, then in general the
> author will be aware of that fact. The message now goes to two
> list-expanders, but the List-B expander is totally unaware that List-A is
> another list, and therefore cannot modify its behaviour (e.g. in its
> Reply-To or [...] So what is "obviusly wrong" about that?

1. The author may not know that address "foo" that he sent a
   message to is expanded into a list. The list expander that
   does the expansion does "know".
2. The mailbox associated with "List-B" is expanded into possibly
   many mailboxes, some of which may be additional list mailboxes;
   even if an author knows that "List-B" is a list, he may not know
   that that mailboxes expanded from that list are further expanded
   into additional lists.

>>>Hence the original sender is that latest point
> 
> 
>>Still drawing bad conclusion from false premises, I see.
> 
> 
> No, you don't see. Please explain how the expander of List-A or List-B is
> is a better situation to deal with this scenario that the original sender.

See above.

>>>Unless, of course, you have a better solution to propose.
> 
> 
> And I am still awaiting your solution.

You still haven't said exactly what the problem is that requires
a solution.

Date: Thu, 7 Oct 2004 16:05:18 -0400
From: Keith Moore <[email-masked]>
To: "Charles Lindsey" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

> >I don't think that's true at all.  The way they tend to be implemented,
> >these extra header fields just make things more confusing. 
> 
> Well the way(s) in which Reply-To is implemented certainly have that
> effect, as you and others have pointed out.
> 
> Which merely argues for a new header with a carefully written semantics
> and intended method of use.

perhaps.  But the semantics and intended method are a lot harder to
define and get right than the syntax of the header field.

> >That's not sufficient.  At a minimum, the replier needs to be able to
> >edit the recipient fields AND see the fields of the subject message
> >while he's  composing the reply.  More realistically, the replier needs
> >to 
> 
> >a) Know (probably via an alert of some kind) when the sender of the
> >subject message requested that replies go to some non-default set of
> >recipients
> 
> I have no objection to alerts provided I don't have to give extra clicks
> to turn them off, or if I can configure them off.

They should be designed so that it is rarely necessary to turn them off.
For instance, pop-up dialog boxes suck for this.  Just put some text
with a yellow background at the top of the compose pane.  e.g.

"Note: the author requested that replies go to [email-masked]"

or perhaps

"Note: the author requested that he not be copied on replies"

Then let the user click on a button to accept that request, if he so
chooses.  I don't think the requests should be accepted by default.

(now if you want the alert to have a check box like:
[ ] in the future, honor similar requests from this sender without asking

that might be very useful)

> >b) Be able to easily select from "reply to author" "reply to all" and 
> >"reply to where the author of the subject message recommended" 
> >(in addition to being able to add or delete recipients from this list)
> >_at any time before sending the reply_
> 
> Yes, that last one is what the "Reply-to-List" button would do 

maybe, maybe not.  "reply to where the list recommended" might be an
additional option.

> >And if you design an MUA only for the "average user", you
> >degrade everyone's capabilities.  Because even the average user
> >sometimes needs capabilities beyond his "average" use.
> 
> No, you provide some extra hoops for the expert user to jump through if he
> wants to do something different than the "average" user. 

the thing to do is to make it easy for any user to do the right thing,
not to penalize people who want to do the right thing by making them
jump through extra hoops.

Keith

Date: Thu, 7 Oct 2004 15:56:22 -0400
From: Keith Moore <[email-masked]>
To: "Charles Lindsey" <[email-masked]>
Cc: [email-masked], [email-masked];
Subject: Re: Understanding response protocols
Sender: [email-masked]

> >> Specifically, in various situations, the recipient needs to know that
> >> followups go to T, and replies go to R, where R isn't contained in T.
> 
> >No, the recipient needs to _decide_ where his particular reply goes,
> >based on the content of his reply -- not take the sender's blind
> >recommendation on faith.
> 
> This recipient would be perfectly happy to take the sender's
> recommendation on faith (with the aid of a single button that was known to
> produce that effect). 

I have no problem with your being able to make that choice.  I have a big
problem with user agents or an architecture that _expects_ users to take
a sender's recommendation on faith.

Keith

To: LIST: [email-masked];
Xref: clerew local.mime:3838
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Thu, 7 Oct 2004 14:01:21 GMT
Lines: 54
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>I don't think that's true at all.  The way they tend to be implemented,
>these extra header fields just make things more confusing. 

Well the way(s) in which Reply-To is implemented certainly have that
effect, as you and others have pointed out.

Which merely argues for a new header with a carefully written semantics
and intended method of use.

>That's not sufficient.  At a minimum, the replier needs to be able to
>edit the recipient fields AND see the fields of the subject message
>while he's  composing the reply.  More realistically, the replier needs
>to 

>a) Know (probably via an alert of some kind) when the sender of the
>subject message requested that replies go to some non-default set of
>recipients

I have no objection to alerts provided I don't have to give extra clicks
to turn them off, or if I can configure them off.

>b) Be able to easily select from "reply to author" "reply to all" and 
>"reply to where the author of the subject message recommended" 
>(in addition to being able to add or delete recipients from this list)
>_at any time before sending the reply_

Yes, that last one is what the "Reply-to-List" button would do (though it
might reflect the list maintainer's recommendation if the author had made
no special request).

>And if you design an MUA only for the "average user", you
>degrade everyone's capabilities.  Because even the average user
>sometimes needs capabilities beyond his "average" use.

No, you provide some extra hoops for the expert user to jump through if he
wants to do something different than the "average" user. Just so long as
the defaults provided for the average user are not such as to upset
everybidy else (that is where Microsoft came unstuck- and also in that
their hoops were sometimes almost impossible to find :-( ).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3837
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Thu, 7 Oct 2004 13:49:16 GMT
Lines: 25
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>> Specifically, in various situations, the recipient needs to know that
>> followups go to T, and replies go to R, where R isn't contained in T.

>No, the recipient needs to _decide_ where his particular reply goes,
>based on the content of his reply -- not take the sender's blind
>recommendation on faith.

This recipient would be perfectly happy to take the sender's
recommendation on faith (with the aid of a single button that was known to
produce that effect). Especially so on a list such of this where the sender
was a regular contributor and was known to set his MFT/whatever in an
acceptable manner.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 06 Oct 2004 21:24:03 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 list <[email-masked]>
Organization: Bruce Lilly
To: [email-masked]
CC: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

Laird Breyer wrote:

>>For somebody not on the list? Obviously he shouldn't receive list
>>messages unrelated to any that he might have submitted. If he wants
>>copies of any extended discussion (such as the example below) he'll
>>have to indicate so and rely on list members to cooperate with that
>>request.
>>
> 
> 
> That is a problem which an intelligent server as proposed can address
> automatically.

I fail to see how that will work (i.e. automatically) for list
non-members.

>>basic function of a mailing list is to facilitate human group
>>communication by distributing messages to that group. The humans
>>involved need to indicate (to the other humans) their individual
>>preferences for responses [do they want responses to go only to
>>the list, do they want personal responses only, do they want both,
>>etc.]. 
> 
> 
> But the list server can act as a proxy for these mundane tasks.  If
> the human can indicate his response preferences inside the message he
> is sending, then surely the human can tell the server those response
> preferences once, and the server can then insert these inside all the
> messages being transmitted to the list.

You seem to be assuming that each author will want to indicate
the same response addresses for each message, which is not
necessarily the case.

>>* lacking strong authentication in the core mail protocol, it
>>  is not possible for mailing list software (or humans for that
>>  matter) to accurately determine the identity of a message
>>  author or sender [without depending on the sender to include
>>  some optional authentication information such as a digital
>>  signature].  Inability to identify the author/sender means
>>  an inability to perform some feat of magic based on author/
>>  sender identity.
> 
> 
> The identity/authentication of the author is not of concern (more
> precisely, all these problems exist now in exactly this same form -
> you expect other humans on the mailing list to accurately determine
> the identity of some message's author, and send him a courtesy copy
> etc. - does that make authors magicians?)

The point is that expecting mailing list software to do more
than expand distribution has authentication implications
above and beyond the issue of how recipients deal with (lack
of) authentication.  A recipient who wishes to prepare some
sort of response needs to decide whether to respond in
accordance with any (possibly message-specific) recommendation
made by the original message author and act accordingly,
and that would not change with the hypothetical "intelligent"
remailer as I understand your description to date.

Date: Thu, 7 Oct 2004 10:23:24 -0400
From: Keith Moore <[email-masked]>
To: "Charles Lindsey" <[email-masked]>
Cc: [email-masked], LIST: [email-masked];
Subject: Re: Understanding response protocols
Sender: [email-masked]

> No, what I have proposed still falls withint the existing syntax of
> <address>. Granted it is not the prettiest of solutions, it is still
> better tnat no solution at all.

no it's not.  overloading is almost never a good idea, especially when
it contradicts 20+ years of pre-existing practice.

Date: Thu, 7 Oct 2004 12:50:37 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Understanding response protocols
Cc: Charles Lindsey <[email-masked]>
Sender: [email-masked]

Charles Lindsey writes:
> No, what I have proposed still falls withint the existing syntax of 
> <address>. Granted it is not the prettiest of solutions, it is still 
> better tnat no solution at all. You have a better one?

Do you think it is better once all those fun-loving translators have 
been at it? We've seen "Re" mutate into dozens of translated prefixes. 
I see no reason to think that your proposed keyword would escape the 
fate of Re.

Arnt

To: LIST: [email-masked];
Xref: clerew local.mime:3827
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Wed, 6 Oct 2004 18:44:48 GMT
Lines: 64
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:
>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
>> 
>> 
>>>Charles Lindsey wrote:
>> 
>> 
>>>>The only place where it is known that a crosspost is occurring is at the
>>>>original sender.
>> 
>> 
>>>Not true, on at least two counts:
>> 
>> 
>>>1. an author might not know that a particular mailbox expands to a list.
>> 
>> 
>> Sure, the author might not know [...]

>So you agree that your original statement was untrue.

OK, if you want to be unnecessarily pedantic:

   The only place where it can be known that a crosspost is occurring is at the
   original sender.

>> But for sure if the author is not aware of it, then agents further
>> down the chain will be even less aware of it.

>That's obviously wrong; clearly a downstream list expander (or
>its maintainer) is aware that list expansion is taking place.

If a message is crossposted to List-A and List-B, then in general the
author will be aware of that fact. The message now goes to two
list-expanders, but the List-B expander is totally unaware that List-A is
another list, and therefore cannot modify its behaviour (e.g. in its
Reply-To or MFT) so that replies from its members go to both lists. And
vice versa for the other list. So what is "obviusly wrong" about that?

>> Hence the original sender is that latest point

>Still drawing bad conclusion from false premises, I see.

No, you don't see. Please explain how the expander of List-A or List-B is
is a better situation to deal with this scenario that the original sender.

>> Unless, of course, you have a better solution to propose.

And I am still awaiting your solution.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3829
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Wed, 6 Oct 2004 18:53:55 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> "D. J. Bernstein" <[email-masked]> writes:

>Bruce Lilly writes:
>> It appears that MFT proponents cannot agree what it is supposed to do

>In fact, all of us are in perfect agreement as to what Mail-Followup-To
>does: it replaces To etc. as the default followup target, in exactly the
>same way that Reply-To replaces From as the default reply target.

>Comments such as ``In this case, Mail-Followup-To includes both of the
>crossposted lists'' are simply _examples_ of how senders commonly use
>and benefit from Mail-Followup-To. They don't change the _meaning_ of
>the field. The Mail-Followup-To field always means the same thing: it
>replaces To etc. as the default followup target. Nothing more, nothing
>less.`

Yes, but Mail-Followup-To is no use unless it is clear which of the "reply
buttons" causes it to be invoked in preference to From/Reply-To. Jacob
Palme's original draft proposal always presumed that it would somehow be
apparent when a "group reply" was wanted, but the only really satisfactory
way to ensure that is a separate "Reply-to-List" button.  Without that,
the MUA is going to have to resort to some rather messy heuristics.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3830
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Angle brackets surrounding Content-ID
Date: Wed, 6 Oct 2004 18:58:34 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:
>> 
>> It is the clear intention that RFC 2822 is to supersede RFC 822, and that
>> all standards that referred to RFC 822 are to be interpreted, so far as is
>> possible, in the light of RFC 2822.

>No, on all counts. RFC 2822 itself does not supersede RFC 822; at the
>earliest, the successor of its successor might be a full Standard which
>would supersede RFC 822.  Standards that specifically refer to RFC 822
>must be interpreted in terms of RFC 822; EBNF differs from ABNF, etc.

Is that the general understanding of others on this list? That we follow
RFC 2822 in some places/headers and follow RFC 822 in others?

>>>RFC 2822 id-right is a problem because it changes the semantics.
>> 
>> Good! It is an improvement.

>> Have you heard of any cases where this change has caused a problem?

>I have mentioned specific issues where RFC 2822 causes problems.

But you have not answered the question. We were discussing the <id-right>
in <msg-id>s. Have you heard of any case where this esoteric change of
semantics actually caused a problem?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked];
Xref: clerew local.mime:3828
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Understanding response protocols
Date: Wed, 6 Oct 2004 18:48:24 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:
>> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>>>Try RFCs 1958 and 2277, paying attention to the difference between
>>>protocol elements and text.
>> 
>> Yes, using "LIST" a keyword in group syntax is not the most perfect
>> solution, but then we are not living in a perfect world.
>[...]
>> I don't think we are in a position seriously to consider new syntax for
>> To:, Cc: and Reply-To.

>Then why have you proposed such a change -- that's exactly what changing
>non-protocol text to "a keyword" would do?

No, what I have proposed still falls withint the existing syntax of
<address>. Granted it is not the prettiest of solutions, it is still
better tnat no solution at all. You have a better one?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 6 Oct 2004 18:29:58 +0200 (Romance Daylight Time)
From: Xavier Nodet <[email-masked]>
Subject: Re: Understanding response protocols
To: [email-masked]
cc: [email-masked]
Sender: [email-masked]

--=_Es2P22kzIerJRlZ6gpuwFzJVXcfnfW
Mime-Version: 1.0
Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
Content-Disposition: INLINE
content-transfer-encoding: quoted-printable

On Wed, 6 Oct 2004 11:10:22 -0400 Keith Moore =
<[email-masked]> wrote:

> > <http://xavier.nodet.free.fr/Mahogany-composing.png>
>
> I wonder - does the "one address per line" get cumbersome for=
 messages
> with lots of addresses?

Generally, anything (deleting, moving from 'To' to 'Cc', etc.) =
that is
done for one address at a time is at least as easy, if not much=
 easier.
But at this time, there is no way to apply the same action to a=
 set of
addresses, and in those cases it becomes cumbersome.

Actually, I suppose that MUAs that use several addresses per =
field could
also have something like an 'Unused' field: this field would be=
 populated
by the MUA with addresses that by default will not get the =
answer, but
the user could e.g. drag and drop addresses between this field =
and the
'To' and 'Cc' fields. Does this already exist?

--=20
Xavier Nodet
"They that can give up essential liberty to obtain a little =
temporary
safety deserve neither liberty nor safety." - Benjamin =
Franklin, 1759.

--=_Es2P22kzIerJRlZ6gpuwFzJVXcfnfW
Content-Type: application/pgp-signature
Content-Disposition: inline

--=_Es2P22kzIerJRlZ6gpuwFzJVXcfnfW--

Date: Wed, 6 Oct 2004 12:00:16 -0400
From: Keith Moore <[email-masked]>
To: Philip Hazel <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: reply etiquette
Sender: [email-masked]

> Er, you wrote this: "it is presumptuous to assume ... that the recipient
> reads the list traffic in the same way that he reads other traffic."
> 
> My point is that it is equally presumptious to assume the opposite.

It is.  But that's not what I was presuming.

> It seemed to me that you were saying "I can't assume the recipient will 
> read list traffic like other traffic, therefore I will send him some 
> other traffic, _presuming_ that he will read other traffic more
> quickly". 

Nope.  I was saying "I can't assume that the recipient will see the
message if I only send it to the list, and it's important that this 
recipient see the message (if only because it's impolite to not copy
him).  Therefore I will explicitly specify this recipient (a) to make
sure he gets a copy and (b) to notify other recipients that I 
sent him a copy.

> > What I presume is that if I reply to a message that someone sent, it's simple
> > courtesy to ensure that he reliably gets a copy of the reply.  It's also
> > simple courtesy to explicitly notify other recipients of the message that the
> > author of the subject message was sent a copy of the reply - so that they know
> > he was not snubbed.
> 
> And unfortunately it seems that your courtesy is different from several 
> others'. That's the problem with culture. 

Well, there's a conflict between what is dictated by courtesy and what the
technology provides.  In particular, we don't tend to have message stores
or user agents that manage duplicate messages in a useful way.

> What's the solution? I can see 
> none other than providing sufficient switches/knobs in the software so 
> that people can configure it to suit themselves.

I can imagine solutions that seem much better than that.  But I'll probably
have to build them in order to make a convincing demonstration. 

Keith

Date: Wed, 6 Oct 2004 11:40:16 -0400
From: Keith Moore <[email-masked]>
To: Paul Smith <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: reply etiquette
Sender: [email-masked]

> I can't think of any logical, legitimate, reason why anyone would want
> to post to [email-masked]' unless they were a member of the list -
> that'd be like shouting across a room and interrupting someone else's 
> conversation, then running away.

on the contrary, it's quite a reasonable place to ask questions about
internet mail and MIME.  I wouldn't be surprised if this list only
accepted posts from subscribers - these days it's not unusual for lists
to get hundreds of spams in a day and it's simply too much trouble to
filter them - but it's not _inherently_ unreasonable for a discussion
list to accept outside traffic.  

Keith

Date: Wed, 6 Oct 2004 11:35:10 -0400
From: Keith Moore <[email-masked]>
To: "Charles Lindsey" <[email-masked]>
Cc: [email-masked], LIST: [email-masked];
Subject: Re: Understanding response protocols
Sender: [email-masked]

> 
> In <[email-masked]> Keith Moore
> <[email-masked]> writes:
> 
> >Yes.  The more I look at the problem of replies, the more it looks
> >like the problem is 90% UA construction and 10% protocol.   Which
> >probably means that adding more protocol isn't the most important
> >part of a solution.  And since adding more protocol tends to mean
> >more complexity, doing so could easily make the situation worse.  
> >The thing to do is to get UA functionality right, _then_ to see what
> >new protocol frobs (if any) are appropriate.
> 
> But the situation seems to be that you can't get the UA functionality
> right without some extra protocol, because the information for the UA
> to do a decent job just isn't available at present (I think few of us
> are convinced by Bruce's claim that Reply-To can be made to do it
> all).

I don't think that's true at all.  The way they tend to be implemented,
these extra header fields just make things more confusing.  

> Actually, is is not display of Reply-To and MFT by agents that is
> essential (though it is highly desirable). What is essential is for
> compose windows to exhibit clearly the To: and Cc: for the reply (with
> alerts and bright red colours if they are not what the user might have
> expected, if you like). And the user MUST have the ability to edit
> those fields before committing the reply.

That's not sufficient.  At a minimum, the replier needs to be able to
edit the recipient fields AND see the fields of the subject message
while he's  composing the reply.  More realistically, the replier needs
to 

a) Know (probably via an alert of some kind) when the sender of the
subject message requested that replies go to some non-default set of
recipients

b) Be able to easily select from "reply to author" "reply to all" and 
"reply to where the author of the subject message recommended" 
(in addition to being able to add or delete recipients from this list)
_at any time before sending the reply_

> >I think it would be better if no "logic" were needed.  The decision
> >of who to reply to belongs to the person composing the reply.  For
> >the UA to make it easy for that person to "fill in the blanks" is
> >reasonable.  For the UA to try to emply "logic" to _decide_ (or
> >equivalently, _guess_) where the message should go is not reasonable.
> 
> The ultimate decision certainly belongs there, but the average user
> will surely be happy to go along with the what the original author of
> the list expander asked for or whatever is the "culture" of the list.

The user will be happy with it until he embarasses himself by sending a
reply to the wrong address(es).  Even if the user remains happy, the
discussions he participates in may be less effective because too many
people have blindly replied to everyone who was a recipient of the
subject message, because their MUAs encouraged them to do this.  He
may be annoyed that "the other people on the list send too many 
messages" without realizing that the real problem is that everyone's 
MUAs discourage users from thinking about where they send replies.

> Moreover, the
> average user just wants a quiet life. He just wants to hit one button,
> compose his reply, and hit 'send'. 

Yup.  But he wants _other_ users to not send him replies that (for
whatever reason) he doesn't want to see.

> He does not want to be "filling in any
> blanks" in the case of what he regards as routine replies.

My argument is that it's possible to make an MUA user interface that is 
more effective than current ones, but still "user friendly".

> Yes, just occasionaly he will want to do something different, and some
> users may want to do something different all the time, but MUAs should
> be designed to make life convenient for the average user (though by
> all means let them be configurable to do special things for users who
> want to be different).

The issues associated with sending replies to the wrong place
affect average users as much as anyone else.  "Average users"
get embarassed when they send messages to the wrong place. 
"Average users" get annoyed when too much traffic is generated
because everyone replies to too many recipients.  "Average 
users" suffer when the signal-to-noise ratio of their discussions
degrades due to too many replies to all.

And if you design an MUA only for the "average user", you
degrade everyone's capabilities.  Because even the average user
sometimes needs capabilities beyond his "average" use.  

What is really needed is not to dumb down the user interface, but to
help users be aware of what they're doing, make the most frequent cases
work with a minimum of effort, while still making the less frequent
cases easy to use when that's what the user wants to do. 

Keith

Date: Wed, 6 Oct 2004 11:10:22 -0400
From: Keith Moore <[email-masked]>
To: Xavier Nodet <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

> The MUA I use has a different way of doing things: instead of having one
> 'To:' and one 'Cc:' field with possibly many addresses in each, it uses
> one line per address, and this line has a drop-down box in front of it
> with (among others) 'To', 'Cc', 'Bcc' and 'None'. When 'None' is
> selected for a particular address, this address is grayed, and of course
> not used to send the reply.
> 
> When I chose 'reply-To-List' for your message, and because my MUA knows
> that <[email-masked]> is the address of a list, only this address had
> 'To' in front of it. But the other addresses where still available, and
> I just had to change the selection from 'None' to 'Cc' to include you in
> the recipients of my reply.
> 
> You can see that on
>   <http://xavier.nodet.free.fr/Mahogany-composing.png>

thanks for the illustration.  I wonder - does the "one address per line" get
cumbersome for messages with lots of addresses?

Keith

Date: Wed, 6 Oct 2004 11:02:09 -0400
From: Keith Moore <[email-masked]>
To: "D. J. Bernstein" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

> 
> Keith Moore writes:
> > The thing to do is to get UA functionality right, _then_ to see what
> > new protocol frobs (if any) are appropriate.
> 
> One of the crucial MUA features is to automatically communicate
> certain information to recipients. Communication _requires_ protocol
> support.
> 
> Specifically, in various situations, the recipient needs to know that
> followups go to T, and replies go to R, where R isn't contained in T.

No, the recipient needs to _decide_ where his particular reply goes,
based on the content of his reply -- not take the sender's blind
recommendation on faith.

--
Nero fiddled.  Bush read _My Pet Goat_.

Date: Wed, 6 Oct 2004 11:35:41 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 list <[email-masked]>
Sender: [email-masked]

On Oct 04 2004, Martin Trautmann wrote:

> Here's an example from [email-masked]:
> September 2004:
> Msgs:	7642
> Mbox:	23 MB
> Thrds:	668
> noref:  1093
> 
> That is: 1093 of 7642 messages lack either References or In-Reply-To and
> are grouped together by Subject only.
> 
> Having a closer look at the threads, I see that 80 of 668 are not new
> threads but could not be grouped to the matching thread since neither
> matching subject nor References were given.
> 
> That's 15 % which lack proper references!

Very nice example. You've convinced me. Either we have to wait
for the world to upgrade their MUAs, or we have to find some other
computationally cheap way of discovering thread structures. 

> 
> Concernint the numbers above, I'd recommend this highly for digest modes.
> It would not be the thread only, but both a thread ID and a subthread or
> message id within this thread.
> 
> In fact it would result in a very long thread id - so the easiest would be
> just to compose a subject from maybe 'Re: ', Original Subject and
> [<message-ID>] which would be filtered by comparing the message-ID and
> references - and putting it in there if omitted.

The example I gave was just to keep the idea easy to see. If implemented
by a list server, I would expect some internal hash number for the parent
node within the thread tree. The id number inserted in a subject would
not need to be descriptive for humans just an opaque label.

However, if you also expect the recipient's
MUA to use those ids as hints for the display of the thread, then 
the embedded id would need a simple structure, such as you propose below.

> How about the answer to Antw: Re: this is an example subject (thread-id
> 7.1)? Should be [tid: 7.1.1] - and not only [tid: 7.3]!?

> Worst might be (or is) a virus that takes the sender's subscribed name,
> sends to the list and blocks the list itself by e.b. mass postings,
> virusses, huge attachments or even list server commands which are
> attempted to process.

That's already a problem. Nothing to do with threads, but a serious 
problem nevertheless. 

> > For the case of a courtesy copy, if every list member wanted one, the
> > effect would be insignificant, since only the parent poster should
> > receive a courtesy copy.
> 
> That's not necessarily that helpful - maybe the answer to the answer was
> the more interesting part. 

I think the question has to be: is there a mechanical benefit from 
letting the list server perform some actions which are already performed
by list members individually? As a result, the list members' MUA software's
shortcomings could be overridden by the list server acting as a proxy. 

If you like, another way of viewing this is: 

The combination of (MUA + list server) would act as a MUA with both
local (on the user's PC) and remote (on the list server) components.
The remote component can be upgraded (for bug fixes, standards
compliance etc.) without requiring each list member to upgrade their 
preferred MUA. A message is composed on the local MUA, transmitted to
the list server, which rewrites it according to user preferences and
the list standards, and finally transmits it to the other list members.

An intelligent list server could perform many more, new things as well
(e.g. your examples), but all of this is getting very off topic
for the 822 list.

-- 
Laird Breyer.

Date: Wed, 6 Oct 2004 11:05:38 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 list <[email-masked]>
Sender: [email-masked]

Bruce,

You make some good points. I'll answer them below for completeness,
but you've convinced me this approach is somewhat off topic for the
822 standards list, so it may be best to leave the topic here.

On Oct 04 2004, Bruce Lilly wrote:

> 
> What are you going to index? Message-ID, In-Reply-To, References are
> not always present, Subject is a poor choice (it changes when the
> topic doesn't change, it can be similar or the same for unrelated
> messages, etc.).  The fact of the matter is that you cannot achieve
> absolute accuracy in identifying relationships between messages,
> and any scheme that relies on such accuracy (whether explicitly stated
> or not) is doomed.

No, the messages in this scheme should be indexed by node id within
the forest of threads comprising the mailing list. Remember that
the server's job in this case requires building an internal 
forest of thread trees, which grows over time as new messages are received.

When a new message is received by the server, its parent node must be
computed from its contents. I fully accept that the current Message-ID,
In-Reply-To, References and Subject are insufficient for reliably
computing the parent node (see nice example given by Martin Trautmann),
but don't (yet?) accept that a scheme for reliably computing the parent
node is impossible with the current crop of MUAs. 

I gave such a scheme, based on the idea of the server embedding a node
id in the Subject field, and called it a hack. It has no relation to
2822 standards, which is why we're off topic, but I still think it is
quite workable within the system envisaged, namely a server system
which already keeps an internal forest of threads.

> 
> > I suspect I got confused. What is your position on threading in
> > mailing lists? Are you suggesting that:
> 
> First, it's a UA display issue, and a UA author can use whatever
> heuristics he likes. It doesn't affect any protocol. Second, not
> every reader wants a "threaded" display -- some might prefer
> most-recent first or other methods of arranging message display.

Ok, that makes sense. The threads I'm discussing are kept internally
by the server for its own purposes, and do not represent guidelines
for display in a UA.

> 
> There are several problems with that:
> 1. The Subject field is defined as an unstructured field. Attempting
>    to impose structure on it is a very bad idea.
> 2. The Subject field is intended as an only human-readable
>    description of the message topic. Attempting to turn that into
>    a machine-parsed repository for protocol information is a
>    very bad idea.
> 3. As a general principle, mailing list software should not
>    alter any message header fields supplied by the author,
>    including the Subject field.
> 

I agree that my proposal breaks these principles. 

For completeness: the important aspect here is only to find a part of
the message headers which is highly likely to be returned intact to
the mailing list server when replies are sent, even by stupid MUAs. If
we could rely on MUAs always following standards, this whole
discussion would be moot. Other than the Subject field (whose prefix
may change in a thread reply, but whose suffix is typically intact), I
don't know of another candidate.

> 
> For somebody not on the list? Obviously he shouldn't receive list
> messages unrelated to any that he might have submitted. If he wants
> copies of any extended discussion (such as the example below) he'll
> have to indicate so and rely on list members to cooperate with that
> request.
> 

That is a problem which an intelligent server as proposed can address
automatically.  If the server keeps its own internal forest of
threads, then there is no need to rely on the cooperation of list
members. The list server can examine the thread topology, and decide
what messages are to be sent to whom with high accuracy. Of course,
it doesn't preclude extra cooperation from list members if they choose.

> 
> The problem was intended to be obvious; A only sees an incorrect
> answer. That problem stems from the presumption that a mailing
> list expander can magically determine message relationships and
> send out copies of "related" messages to a non-list author. The

You're seeing more into this than I intended. I presume that a (new
generation) list server can keep a reasonable record of who replies to
whom, and allows humans to construct preference rules which affect
only themselves, based on the pattern of responses. Clustering
messages by topic is beyond the scope.

> basic function of a mailing list is to facilitate human group
> communication by distributing messages to that group. The humans
> involved need to indicate (to the other humans) their individual
> preferences for responses [do they want responses to go only to
> the list, do they want personal responses only, do they want both,
> etc.]. 

But the list server can act as a proxy for these mundane tasks.  If
the human can indicate his response preferences inside the message he
is sending, then surely the human can tell the server those response
preferences once, and the server can then insert these inside all the
messages being transmitted to the list.

> * expecting software to be able to deduce message relationships
>   is unrealistic because there is no mandatory feature of the
>   mail protocols which can be used (Message-ID, In-Reply-To, and
>   References are all optional). That leaves examining the entire
>   archive content and making some sort of guess as an alternative;
>   albeit an expensive alternative that still amounts to mere
>   guesswork.

Yes. I accept this is unrealistic with current software.

> * lacking strong authentication in the core mail protocol, it
>   is not possible for mailing list software (or humans for that
>   matter) to accurately determine the identity of a message
>   author or sender [without depending on the sender to include
>   some optional authentication information such as a digital
>   signature].  Inability to identify the author/sender means
>   an inability to perform some feat of magic based on author/
>   sender identity.

The identity/authentication of the author is not of concern (more
precisely, all these problems exist now in exactly this same form -
you expect other humans on the mailing list to accurately determine
the identity of some message's author, and send him a courtesy copy
etc. - does that make authors magicians?)

> * because different humans have different preferences, a one-
>   size-fits-all approach enforced by fascist software won't
>   work for a group with diverse preferences.

That is already the problem we face with mailing lists now. 
In fact, we are discussing this very problem on this list.

> But that's not a problem, because human authors *can* indicate
> their preferences to others, within existing protocol framework,
> with no meddling from list expanders. Indeed, any such meddling
> is more likely to cause problems than to solve them, as has
> been the case with some list expansion software which has been
> implemented in such a manner as to overwrite the authors'
> expressed preferences.

That is a very good point. All systems have bugs and new systems are
poorly understood quasi by definition. Intelligent list servers are
no exception.

(Just to repeat: I'm willing to end this thread here, given its off
topic nature for this list, but I didn't want to leave your good
objections hanging.)

-- 
Laird Breyer.

Date: Tue, 5 Oct 2004 18:27:43 +0100
From: Tony Finch <[email-masked]>
To: [email-masked]
Subject: Re: Locating forwarded messages
Sender: [email-masked]

On Tue, 5 Oct 2004, Grant Baillie wrote:
>
> I have a question (and some rambling possible answers) about forwarding
> messages, in the following sense (from rfc 2822): [snip]
>
> One possibility might be to set References and not In-Reply-To. It's not clear
> to me whether or not users would find this surprising, since it would lead to
> their forwarded message being part of the originals' thread trees.

I would find this useful, and I wich MUAs did it.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
MULL OF KINTYRE TO ARDNAMURCHAN POINT: WEST VEERING NORTHWEST 6 TO GALE 8,
OCCASIONALLY SEVERE GALE 9 IN THE WEST. RAIN OR SQUALLY SHOWERS. GOOD
OCCASIONALLY MODERATE. ROUGH OR VERY ROUGH.

Reply-To: [email-masked]
From: Grant Baillie <[email-masked]>
Subject: Locating forwarded messages
Date: Tue, 5 Oct 2004 10:13:43 -0700
To: [email-masked]
Sender: [email-masked]

I have a question (and some rambling possible answers) about forwarding 
messages, in the following sense (from rfc 2822):

> Reintroducing a message into the transport system and using
> resent fields is a different operation from "forwarding".
> "Forwarding" has two meanings: One sense of forwarding is that a mail
> reading program can be told by a user to forward a copy of a message
> to another person, making the forwarded message the body of the new
> message. A forwarded message in this sense does not appear to have
> come from the original sender, but is an entirely new message from
> the forwarder of the message.
> ...

What header(s) should the new message contain to identify it as being a 
forwarded version of the original? This would be useful, say, so that 
clients can provide a way for users to locate any forwarded message(s) 
(or go from the forwarded message to the original).

The case of replying to a message, where you set References and 
In-Reply-To, is well covered by RFC 2822. However, in my reading, the 
above text ("... is an entirely new message...") seems to imply 
forwards aren't really the same as replies. And, indeed, none of the 
clients I tried (with one exception) set these headers when you forward 
messages.

Also, the first sentence of the above paragraph seems to say that 
Resent-Message-ID would not be the correct thing to do.

One possibility might be to set References and not In-Reply-To. It's 
not clear to me whether or not users would find this surprising, since 
it would lead to their forwarded message being part of the originals' 
thread trees.

Another possible answer is not to set any headers, but just store the 
information internally in the client's message store. I'd rather not do 
this if possible, since then it wouldn't work well with IMAP users who 
access their messages from different clients.

Lastly, I could make up a custom header, I guess.

Anyway, suggestions/comments will be appreciated.

--Grant

From: "Hector Santos" <[email-masked]>
To: "ietf-822 list" <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Date: Mon, 4 Oct 2004 10:36:55 -0400
Organization: Santronics Software, Inc
Sender: [email-masked]

From: "Bruce Lilly" <[email-masked]>

I agree with most of your statements. I have some comments about the
following.

> Expecting a mailing list expander to become such a magic
> oracle is an unrealistic expectation, at least with current
> mail protocols [and this is not the place to discuss possible
> future protocols, that would be the mail-ng list]. The implicit
> assumptions that you made all bear upon that:

> * expecting software to be able to deduce message relationships
>   is unrealistic because there is no mandatory feature of the
>   mail protocols which can be used (Message-ID, In-Reply-To, and
>   References are all optional). That leaves examining the entire
>   archive content and making some sort of guess as an alternative;
>   albeit an expensive alternative that still amounts to mere
>   guesswork.

In my design philoshophy (which is incorporated into our mail products), I
believe the host is very much in the middle and has technical
responsibilities in making sure that the technical expectations for the
targeted end-point mail reader or device is statisfied.

Just consider the high possibility for hetergenous systems  - non-RFC/RFC
gateways and transformations.  When that user finally pops into a server, if
that stored message is not RFC 822 ready, the pop3 server needs to make sure
it is RFC 822 prepared accordingly when transferring it.

Same is true for mail receivers and mail list server and "expanders." when
sending out.

Again, that has been my design approach in helping to create the "best"
natural and/or technical expectations for the end-point.

> * lacking strong authentication in the core mail protocol, it
>   is not possible for mailing list software (or humans for that
>   matter) to accurately determine the identity of a message
>   author or sender [without depending on the sender to include
>   some optional authentication information such as a digital
>   signature].  Inability to identify the author/sender means
>   an inability to perform some feat of magic based on author/
>   sender identity.

[Minor side note: Generally True, but trust always begins at the submission
point.  I trust all mail coming into this list is not based on abuse, that
atleast there is a 'subscription' concept to the service.   That in itself
is a "authentication" as small it may be.   Coupled with recent Return Path
Authethication ideas, more strength is added.]

Ergonomically, the mailing list "kludge" simulate the idea groupware
conferencing where frankly, the name of the author is usually oblivious to
the readers.  Content is more important.  You say your are "bruce lilly"
from so so address.com, well, that means nothing to me and most readers.
Sure, many people have no problem exposing what is actually "real contact
information."  but it is only when we try to do more personal direct contact
and communications, face to face or otherwise, where it becomes important.
Sure, the name can become "branded" in some way, but If you understand my
point, I had no control of the receipt of your message.  It came from a
common "Dummy User" (the list address) that is authorized on our end. But
generally, it (your address) is really meaningless until it becomes
important, if ever, for pure direct communications.

> * because different humans have different preferences, a one-
>   size-fits-all approach enforced by fascist software won't
>   work for a group with diverse preferences.

But there should be some level of standard conformity at both ends, and as I
starting above, I believe the host is an important factor as well,
providing the necessary expectations for the natural behavior of the end
point.

This includes two parts:

    Network Control Lines
    Information/Display Control Lines.

A simple idea is this.

The standard expectation for a MUA are:

Display (ignoring CC/BC):

    From:        822.From
    To:            822.To

For a Reply, the Network Control Lines are:

  821.MailFrom   MUA account Reply Address or Display Address
  821.RcptTo     Taken from message Reply-To: or From:
  822.From       MUA account Display Address
  822.To         Taken from message Reply-To: or From:

These are the general across the aboard framework.  So regardless if the
EMAIL message is a mailing list-based  "kludge,"   I believe it is still
necessary that the host makes sure the network controls are proper for
natural MUA reaction as it is most common, expected and "defined" today.

So when hitting the REPLY button the natural technical expection should be
preserved.

Is there such a concept of standard "Mailing List Technology" and ideas that
a host and MUA can be trained to provide this natural functionality?

I think so.  I would like to see what that is.  That is my interest in this
thread.  This is where the MUAs differ when it comes to the variant Reply
ideas, "Reply to All" is easy, but "Forwarding, Followup" need some work
especially for a group ware concept.

> But that's not a problem, because human authors *can* indicate
> their preferences to others, within existing protocol framework,
> with no meddling from list expanders. Indeed, any such meddling
> is more likely to cause problems than to solve them, as has
> been the case with some list expansion software which has been
> implemented in such a manner as to overwrite the authors'
> expressed preferences.

Well, I think it does so in such a way, as I indicated above, to prepare the
minimum requirements, that I don't see it as a problem.  I see it as network
technical transport requirement.  As long as it does not change the final
user intent of the message, its all "legal" and good in my book.

Anyway, those are my comments.

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

Date: Mon, 04 Oct 2004 08:34:34 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 list <[email-masked]>
Organization: Bruce Lilly
To: [email-masked]
CC: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

Laird Breyer wrote:

> I'm not suggesting rescanning the entire archive each time a new message
> is submitted (accepted), an up to date index is all that's needed.

What are you going to index? Message-ID, In-Reply-To, References are
not always present, Subject is a poor choice (it changes when the
topic doesn't change, it can be similar or the same for unrelated
messages, etc.).  The fact of the matter is that you cannot achieve
absolute accuracy in identifying relationships between messages,
and any scheme that relies on such accuracy (whether explicitly stated
or not) is doomed.

> I suspect I got confused. What is your position on threading in
> mailing lists? Are you suggesting that:
> 
> 1) In-Reply-To is the only reliable existing means of discovering 
>    relations between messages?
> 2) In-Reply-To is not honoured by MUAs with sufficient frequency in practice?
> 3) Therefore, threading is not worth considering for mailing lists?

First, it's a UA display issue, and a UA author can use whatever
heuristics he likes. It doesn't affect any protocol. Second, not
every reader wants a "threaded" display -- some might prefer
most-recent first or other methods of arranging message display.

> and separately, for example:
> 
> 4) embedding a thread ID token into subject lines of list propagated messages
>   cannot mitigate the shortcomings of 1-3?

There are several problems with that:
1. The Subject field is defined as an unstructured field. Attempting
   to impose structure on it is a very bad idea.
2. The Subject field is intended as an only human-readable
   description of the message topic. Attempting to turn that into
   a machine-parsed repository for protocol information is a
   very bad idea.
3. As a general principle, mailing list software should not
   alter any message header fields supplied by the author,
   including the Subject field.

> How do lists handle posting privileges now?

Not all lists are closed.  Different lists have different policies
and use different mechanisms.

>>>You're right. If such a person is not a member, then his/her preferences
>>>can't be known.
>>
>>They can be known if and only if he as a message author indicates
>>where he wants responses to be sent, in his message (via Reply-To).
> 
> 
> What about his receipt preferences for messages sent by anybody else
> on the list?

For somebody not on the list? Obviously he shouldn't receive list
messages unrelated to any that he might have submitted. If he wants
copies of any extended discussion (such as the example below) he'll
have to indicate so and rely on list members to cooperate with that
request.

>>>Ok, it's a good question, but I think that's bordering on recursively
>>>reimplementing the mailing list. My personal inclination is that a
>>>response to a response is not a response to the original poster, so
>>>needn't concern him or her.
>>
>>Hypothetical situation:
>>A asks a question
>>B gives a wrong answer, which is copied to A
>>C, D, E, and F respond that B's answer was incorrect, and giving
>>  the correct answer -- none of which is seen by A.
> 
> 
> I don't see the problem with that. When did the matter of correctness
> of an answer enter the debate? I see the point of what you're saying,
> but I don't see that it is a list server's job to inspect the message
> content for accuracy.

The problem was intended to be obvious; A only sees an incorrect
answer. That problem stems from the presumption that a mailing
list expander can magically determine message relationships and
send out copies of "related" messages to a non-list author. The
basic function of a mailing list is to facilitate human group
communication by distributing messages to that group. The humans
involved need to indicate (to the other humans) their individual
preferences for responses [do they want responses to go only to
the list, do they want personal responses only, do they want both,
etc.]. They need to so indicate to the other humans involved
because the responses come from those humans, not from some magic
oracle. Expecting a mailing list expander to become such a magic
oracle is an unrealistic expectation, at least with current
mail protocols [and this is not the place to discuss possible
future protocols, that would be the mail-ng list]. The implicit
assumptions that you made all bear upon that:
* expecting software to be able to deduce message relationships
  is unrealistic because there is no mandatory feature of the
  mail protocols which can be used (Message-ID, In-Reply-To, and
  References are all optional). That leaves examining the entire
  archive content and making some sort of guess as an alternative;
  albeit an expensive alternative that still amounts to mere
  guesswork.
* lacking strong authentication in the core mail protocol, it
  is not possible for mailing list software (or humans for that
  matter) to accurately determine the identity of a message
  author or sender [without depending on the sender to include
  some optional authentication information such as a digital
  signature].  Inability to identify the author/sender means
  an inability to perform some feat of magic based on author/
  sender identity.
* because different humans have different preferences, a one-
  size-fits-all approach enforced by fascist software won't
  work for a group with diverse preferences.

But that's not a problem, because human authors *can* indicate
their preferences to others, within existing protocol framework,
with no meddling from list expanders. Indeed, any such meddling
is more likely to cause problems than to solve them, as has
been the case with some list expansion software which has been
implemented in such a manner as to overwrite the authors'
expressed preferences.

Date: Mon, 4 Oct 2004 11:47:42 +0200
From: Martin Trautmann <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

On 2004-10-03 11:21, Laird Breyer wrote:
> 
> On Oct 02 2004, Bruce Lilly wrote:
> I could answer that a reply message without any form of useful
> identification tokens is relatively rare, but that's a copout,
> and I don't have any numbers to back this up.

Here's an example from [email-masked]:
September 2004:
Msgs:	7642
Mbox:	23 MB
Thrds:	668
noref:  1093

That is: 1093 of 7642 messages lack either References or In-Reply-To and
are grouped together by Subject only.

Having a closer look at the threads, I see that 80 of 668 are not new
threads but could not be grouped to the matching thread since neither
matching subject nor References were given.

That's 15 % which lack proper references!

> Perhaps a more constructive suggestion would be this: if we 
> want threading to work properly on a mailing list, why not
> enlist the help of the list server, e.g. in the following way:
> 
> Since in most MUAs, the original subject is quoted when a reply is
> sent, the list server could imbed a thread id into the subject line of
> the messages it sends out. 

Concernint the numbers above, I'd recommend this highly for digest modes.
It would not be the thread only, but both a thread ID and a subthread or
message id within this thread.

In fact it would result in a very long thread id - so the easiest would be
just to compose a subject from maybe 'Re: ', Original Subject and
[<message-ID>] which would be filtered by comparing the message-ID and
references - and putting it in there if omitted.

> The list server propagates this message to all list members as follows
> Subject: this is an example subject (thread-id 7.0)
> 
> This would indicate e.g. thread number 7, root message.

Don't use (), which is a typical text part. Take e.g. [] or {} - or just
<>.

7".0" is redundancy. I'd suggest e.g. "[tid: 7]"

> And the list server propagates this message to the list members as follows
> Subject Re: this is an example subject (thread-id 7.1)
> 
> Somebody else replies to the list address with
> Subject: Antw: Re: this is an example subject (thread-id 7.1)
> 
> Which is propagated as, you guessed it, 
> Subject: Antw: Re: this is an example subject (thread-id 7.2)

How about the answer to Antw: Re: this is an example subject (thread-id
7.1)? Should be [tid: 7.1.1] - and not only [tid: 7.3]!?

> That's a security issue, which exists independently of the 
> issue of accomodating list member preferences. What's the worst
> that can happen now on mailing lists, and would it get significantly
> worse with an "intelligent" list server? 

Worst might be (or is) a virus that takes the sender's subscribed name,
sends to the list and blocks the list itself by e.b. mass postings,
virusses, huge attachments or even list server commands which are
attempted to process.

> For the case of a courtesy copy, if every list member wanted one, the
> effect would be insignificant, since only the parent poster should
> receive a courtesy copy.

That's not necessarily that helpful - maybe the answer to the answer was
the more interesting part. 

Another rule could be e.g. to set a level of answers you want - or the
level is matching to the level of quoted text: as long as an article
contained quoted text from yourself, you want a courtesy copy, and maybe
the next one as well.

I don't see a very reasonable behavior here - apart from that the user
should take care on his own and on the mails he does receive himself how
to process them. Since most mailing lists don't show significant delays,
there's no major speed advantage on its own.

However, mailing lists should do their best to encourage proper threading
and efficient quoting mechanisms to keep the net load small. One good
example is e.g. not to forward attachments but to save them in a web
archive and insert a link instead.

Regards
Martin

Date: Mon, 4 Oct 2004 11:47:43 +0200
From: Martin Trautmann <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

On 2004-10-03 11:21, Laird Breyer wrote:
> 
> On Oct 02 2004, Bruce Lilly wrote:
> I could answer that a reply message without any form of useful
> identification tokens is relatively rare, but that's a copout,
> and I don't have any numbers to back this up.

Here's an example from [email-masked]:
September 2004:
Msgs:	7642
Mbox:	23 MB
Thrds:	668
noref:  1093

That is: 1093 of 7642 messages lack either References or In-Reply-To and
are grouped together by Subject only.

Having a closer look at the threads, I see that 80 of 668 are not new
threads but could not be grouped to the matching thread since neither
matching subject nor References were given.

That's 15 % which lack proper references!

> Perhaps a more constructive suggestion would be this: if we 
> want threading to work properly on a mailing list, why not
> enlist the help of the list server, e.g. in the following way:
> 
> Since in most MUAs, the original subject is quoted when a reply is
> sent, the list server could imbed a thread id into the subject line of
> the messages it sends out. 

Concernint the numbers above, I'd recommend this highly for digest modes.
It would not be the thread only, but both a thread ID and a subthread or
message id within this thread.

In fact it would result in a very long thread id - so the easiest would be
just to compose a subject from maybe 'Re: ', Original Subject and
[<message-ID>] which would be filtered by comparing the message-ID and
references - and putting it in there if omitted.

> The list server propagates this message to all list members as follows
> Subject: this is an example subject (thread-id 7.0)
> 
> This would indicate e.g. thread number 7, root message.

Don't use (), which is a typical text part. Take e.g. [] or {} - or just
<>.

7".0" is redundancy. I'd suggest e.g. "[tid: 7]"

> And the list server propagates this message to the list members as follows
> Subject Re: this is an example subject (thread-id 7.1)
> 
> Somebody else replies to the list address with
> Subject: Antw: Re: this is an example subject (thread-id 7.1)
> 
> Which is propagated as, you guessed it, 
> Subject: Antw: Re: this is an example subject (thread-id 7.2)

How about the answer to Antw: Re: this is an example subject (thread-id
7.1)? Should be [tid: 7.1.1] - and not only [tid: 7.3]!?

> That's a security issue, which exists independently of the 
> issue of accomodating list member preferences. What's the worst
> that can happen now on mailing lists, and would it get significantly
> worse with an "intelligent" list server? 

Worst might be (or is) a virus that takes the sender's subscribed name,
sends to the list and blocks the list itself by e.b. mass postings,
virusses, huge attachments or even list server commands which are
attempted to process.

> For the case of a courtesy copy, if every list member wanted one, the
> effect would be insignificant, since only the parent poster should
> receive a courtesy copy.

That's not necessarily that helpful - maybe the answer to the answer was
the more interesting part. 

Another rule could be e.g. to set a level of answers you want - or the
level is matching to the level of quoted text: as long as an article
contained quoted text from yourself, you want a courtesy copy, and maybe
the next one as well.

I don't see a very reasonable behavior here - apart from that the user
should take care on his own and on the mails he does receive himself how
to process them. Since most mailing lists don't show significant delays,
there's no major speed advantage on its own.

However, mailing lists should do their best to encourage proper threading
and efficient quoting mechanisms to keep the net load small. One good
example is e.g. not to forward attachments but to save them in a web
archive and insert a link instead.

Regards
Martin

Date: Mon, 4 Oct 2004 14:57:27 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 list <[email-masked]>
Sender: [email-masked]

On Oct 04 2004, Bruce Lilly wrote:
> 
> > If the mailing list doesn't keep an archive of itself, then there's
> > little hope of making it intelligent at all.
> 
> There's a big difference between keeping an archive (write once)
> and grubbing through an entire archive when each new message is
> submitted to the list.

I'm not suggesting rescanning the entire archive each time a new message
is submitted (accepted), an up to date index is all that's needed.
But it does require careful programming to ensure scalability, I 
completely agree, and if someone tries this it won't be done with
ten lines of perl. 

> 
> > Also, it doesn't suffer from the problem of recognizing variations in
> > Re:, Re[2] etc. to identify replies (which is what is problematic now).
> 
> Presence of "Re:" does not identify a reply.
> Subject: Re: the note between Do and Mi
> Subject: Re: Element number 75
> etc. do not indicate replies.  The only reliable indication
> of a response is the presence of an (optional) In-Reply-To
> field (if the original had an optional Message-ID field).
> 

I suspect I got confused. What is your position on threading in
mailing lists? Are you suggesting that:

1) In-Reply-To is the only reliable existing means of discovering 
   relations between messages?
2) In-Reply-To is not honoured by MUAs with sufficient frequency in practice?
3) Therefore, threading is not worth considering for mailing lists?

and separately, for example:

4) embedding a thread ID token into subject lines of list propagated messages
  cannot mitigate the shortcomings of 1-3?

> > That's a security issue, which exists independently of the 
> > issue of accomodating list member preferences. What's the worst
> > that can happen now on mailing lists, and would it get significantly
> > worse with an "intelligent" list server? 
> 
> Most lists do not depend on accurately identifying an author
> at submission time.  You seem to be proposing that some automatic
> action is taken by a server based on unreliable information.
> That's generally a bad idea.  While digitally signing messages
> could help, use of signed messages on mailing lists is quite
> rare.

How do lists handle posting privileges now? For example, what does
[email-masked] say do about accurately identifying me when I post to
the list? I don't sign my messages digitally. 

> 
> > You're right. If such a person is not a member, then his/her preferences
> > can't be known.
> 
> They can be known if and only if he as a message author indicates
> where he wants responses to be sent, in his message (via Reply-To).

What about his receipt preferences for messages sent by anybody else
on the list?

> 
> > Ok, it's a good question, but I think that's bordering on recursively
> > reimplementing the mailing list. My personal inclination is that a
> > response to a response is not a response to the original poster, so
> > needn't concern him or her.
> 
> Hypothetical situation:
> A asks a question
> B gives a wrong answer, which is copied to A
> C, D, E, and F respond that B's answer was incorrect, and giving
>   the correct answer -- none of which is seen by A.

I don't see the problem with that. When did the matter of correctness
of an answer enter the debate? I see the point of what you're saying,
but I don't see that it is a list server's job to inspect the message
content for accuracy.

-- 
Laird Breyer.

Date: Mon, 04 Oct 2004 00:15:33 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 list <[email-masked]>
Organization: Bruce Lilly
To: [email-masked]
CC: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

Laird Breyer wrote:

> If the mailing list doesn't keep an archive of itself, then there's
> little hope of making it intelligent at all.

There's a big difference between keeping an archive (write once)
and grubbing through an entire archive when each new message is
submitted to the list.

> Also, it doesn't suffer from the problem of recognizing variations in
> Re:, Re[2] etc. to identify replies (which is what is problematic now).

Presence of "Re:" does not identify a reply.
Subject: Re: the note between Do and Mi
Subject: Re: Element number 75
etc. do not indicate replies.  The only reliable indication
of a response is the presence of an (optional) In-Reply-To
field (if the original had an optional Message-ID field).

> That's a security issue, which exists independently of the 
> issue of accomodating list member preferences. What's the worst
> that can happen now on mailing lists, and would it get significantly
> worse with an "intelligent" list server? 

Most lists do not depend on accurately identifying an author
at submission time.  You seem to be proposing that some automatic
action is taken by a server based on unreliable information.
That's generally a bad idea.  While digitally signing messages
could help, use of signed messages on mailing lists is quite
rare.

> You're right. If such a person is not a member, then his/her preferences
> can't be known.

They can be known if and only if he as a message author indicates
where he wants responses to be sent, in his message (via Reply-To).

> Ok, it's a good question, but I think that's bordering on recursively
> reimplementing the mailing list. My personal inclination is that a
> response to a response is not a response to the original poster, so
> needn't concern him or her.

Hypothetical situation:
A asks a question
B gives a wrong answer, which is copied to A
C, D, E, and F respond that B's answer was incorrect, and giving
  the correct answer -- none of which is seen by A.

Date: Sun, 3 Oct 2004 11:21:21 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 list <[email-masked]>
Sender: [email-masked]

On Oct 02 2004, Bruce Lilly wrote:

> Whether you realize it or not you're making quite a few assumptions
> which you haven't mentioned:

I'm just playing with a toy model which I thought would be interesting
to discuss. Exploring the implications/assumptions is exactly what's
needed. (unless perhaps I'm too uninformed, I'd rather not waste
everyone's time).

> 1. That mailing list software is designed to keep track of every
>    message sent to the list, and to examine every such message for
>    possible relationships to other messages. Even were that always
>    feasible (see next assumption), it is likely to be impractical
>    especially in the situations where you apparently believe it
>    would be most useful ("For example, consider a high volume
>    mailing list"). As the number of messages increases, the number
>    of comparisons increases quadratically, and the complexity of
>    each comparison may also increase rapidly (see below).

If the mailing list doesn't keep an archive of itself, then there's
little hope of making it intelligent at all.

I think that keeping track of high volume mailing lists is quite practical
(but I'm not arguing it's trivial to code). For example, the lkml is a
good example of high volume I think, and it receives 200-300 messages
per day.

That's only about 100,000 messages per year, which is not impossible
to index. Assuming your point 2 below can be solved, then looking up a
unique computed ID of some sort for the parent of each incoming
message, parsing this parent once and taking all actions related to it
(such as sending a courtesy copy) can be done without putting a big
load on the system.

> 2. That it is always possible to establish relationships between
>    messages (Message-ID, In-Reply-To, and References fields are
>    optional).

That's a good objection. I would extend this by noting that people
sometimes also change subject lines, so comparing subjects in 
the usual way also is not entirely reliable.

I could answer that a reply message without any form of useful
identification tokens is relatively rare, but that's a copout,
and I don't have any numbers to back this up.

Perhaps a more constructive suggestion would be this: if we 
want threading to work properly on a mailing list, why not
enlist the help of the list server, e.g. in the following way:

Since in most MUAs, the original subject is quoted when a reply is
sent, the list server could imbed a thread id into the subject line of
the messages it sends out. 

It's an ugly hack, but it does make use of what I think is a reliable
lowest common denominator assumption, that MUAs quote the original
subject when replying (with extra modifications such as Re: etc.). 

Also, it doesn't suffer from the problem of recognizing variations in
Re:, Re[2] etc. to identify replies (which is what is problematic now).
Instead, the list server simply scans the subject line of a received
message to see if it contains a quoted thread-id (example below).

Of course, it's ugly to not rely on the specially designed headers
such as References, but your point is that MUAs don't reliably set
them in the first place, so Im working with that. I believe that
quoting/mangling original subjects is much more reliable, enough so
that it could be relied on for threading.

So for example, I might send a message to the list with
Subject: this is an example subject

The list server propagates this message to all list members as follows
Subject: this is an example subject (thread-id 7.0)

This would indicate e.g. thread number 7, root message.

When you reply to the list, your MUA sends a message with the subject
Subject: Re: this is an example subject (thread-id 7.0)

And the list server propagates this message to the list members as follows
Subject Re: this is an example subject (thread-id 7.1)

Somebody else replies to the list address with
Subject: Antw: Re: this is an example subject (thread-id 7.1)

Which is propagated as, you guessed it, 
Subject: Antw: Re: this is an example subject (thread-id 7.2)

> 3. That it is possible to accurately determine whether a sender is
>    a list member (avoiding forgeries where e.g. a spammer forges
>    either a message header address field or the envelope sender
>    address, while accurately recognizing messages sent by a list
>    member from an alternate account, via a public wireless access
>    point, via a web mail interface, etc.).

That's a security issue, which exists independently of the 
issue of accomodating list member preferences. What's the worst
that can happen now on mailing lists, and would it get significantly
worse with an "intelligent" list server? 

For the case of a courtesy copy, if every list member wanted one, the
effect would be insignificant, since only the parent poster should
receive a courtesy copy. In other words, on a normal mailing list, a
gate crashing spammer can reach all members, whereas if the list
implements courtesy copies say, the spammer reaches all members, and
the parent poster gets one extra copy.

> 4. That every person who submits a message to a mailing list
>    wishes to receive one or more email responses (that is untrue
>    for some people who monitor list activity via a web archive
>    of the list, whether or not they are registered as being "on"
>    the list).

You're right. If such a person is not a member, then his/her preferences
can't be known. If the person is "on" the list, I'm assuming that if
they receive messages without wanting replies by email, they can and have
selected this preference when they signed up for the list.

> 
> You also haven't indicated whether you think that a response to
> a response to a message sent by the hypothetical X should also
> be mailed to X.

Ok, it's a good question, but I think that's bordering on recursively
reimplementing the mailing list. My personal inclination is that a
response to a response is not a response to the original poster, so
needn't concern him or her. Of course, this issue only matters if the
original poster is not "on" the list, as he will see the full response
thread if he monitors the mailing list.

FWIW, if the list server does keep an indexed archive of messages
organized by threads, then accomodating yet another preference 
(which I would loosely describe as receiving the full response thread) 
is not difficult:

For a newly received message, instead of finding only the parent
poster, the list server walks the chain of parent messages and extracts
all the parents. The total number of recipients is not greater than
the "height" of the thread, which is orders of magnitude smaller than 
the full mailing list volume. So in terms of complexity, even this
preference is scalable to high volume lists.

-- 
Laird Breyer.

Date: Sat, 02 Oct 2004 11:36:23 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 list <[email-masked]>
Organization: Bruce Lilly
To: [email-masked]
CC: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

Laird Breyer wrote:

> For example, consider a high volume mailing list (Arnt's bugtracking
> mailing list example). A non-list member X makes a post, but has no
> intention of monitoring the mailing list, or even to subscribe to it
> and keep it in a separate folder to see if somebody responds. He
> doesn't know how quickly a response might come, a few hours, a few
> weeks, or whether a response will come at all. Having better threading
> or filtering is useless, because the mailing list traffic is not going
> to be monitored by his MUA.
>  
> But the mailing list server knows immediately that X is not a list
> member, so if his post is accepted (this is a policy issue), then if
> somebody else responds to this post on the mailing list, the list
> server can send X a copy of the message. The people who are list members
> don't need to know if X is a list member or not, everything is handled
> by the list server.

Whether you realize it or not you're making quite a few assumptions
which you haven't mentioned:
1. That mailing list software is designed to keep track of every
   message sent to the list, and to examine every such message for
   possible relationships to other messages. Even were that always
   feasible (see next assumption), it is likely to be impractical
   especially in the situations where you apparently believe it
   would be most useful ("For example, consider a high volume
   mailing list"). As the number of messages increases, the number
   of comparisons increases quadratically, and the complexity of
   each comparison may also increase rapidly (see below).
2. That it is always possible to establish relationships between
   messages (Message-ID, In-Reply-To, and References fields are
   optional).
3. That it is possible to accurately determine whether a sender is
   a list member (avoiding forgeries where e.g. a spammer forges
   either a message header address field or the envelope sender
   address, while accurately recognizing messages sent by a list
   member from an alternate account, via a public wireless access
   point, via a web mail interface, etc.).
4. That every person who submits a message to a mailing list
   wishes to receive one or more email responses (that is untrue
   for some people who monitor list activity via a web archive
   of the list, whether or not they are registered as being "on"
   the list).

You also haven't indicated whether you think that a response to
a response to a message sent by the hypothetical X should also
be mailed to X.

Date: Sat, 2 Oct 2004 12:29:45 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 list <[email-masked]>
Sender: [email-masked]

On Oct 01 2004, Martin Trautmann wrote:

> Germans tend to use those, and comparable translations in English writing
> as well - and maybe the receiver wonders about. I was told that any kind
> of greeting in the states may be accepted as some kind of very formal
> attitude. Thus the reader will feel more distracted/distanced here?

Since you asked, I'll give my view, but I'm not American, so it's not
authoritative ;)

Email is much more informal than paper letters. Greetings in email are
nice on personal email, but for near realtime discussion lists it's
just extra junk, like big signatures. In an email discussion, it's
important to know who wrote the quote you're responding to, but
otherwise formalities just waste the reader's time and effort.

I'm on at least one other list which uses full quoting and replies on
top. The damn emails grow exponentially in size, but it makes sense in
a world where threading is nonexistent and replies can occur several
weeks after the initial post.

> 
> On the other hand someone in Germany may feel weird if any stranger would
> send him an letter and sign this with 'Love, ..."
> 

I think you're getting too much spam ;-) 

> 
> Behavior has changed significantly via email - where everyone uses the
> direct 'you' ('du' instead of 'Sie', 'third' person 'thou'?). But I do
> note those differences e.g.  on salutation or on perfectly different
> quoting behavior, where fullquotes in German newsgroups / usenet are an
> almost obvious sign of a newbie, while most of the people want short and
> helpful quoting only.

This may be the influence of English, where email originated, on the
German behaviour (also occurs in French). The polite forms
(e.g. 'thou') have disappeared in English, and formality is expressed
differently, but there is still a big difference between 
informality and familiarity. Public emails don't tend to be familiar.

-- 
Laird Breyer.

Date: Sat, 2 Oct 2004 12:03:33 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 list <[email-masked]>
Sender: [email-masked]

On Oct 01 2004, Martin Trautmann wrote:

> But apart from the user's clash of culture, which is based on a very
> personal flavour, it should depend on the idea of a list as well.
> 
> 
> What's the idea of a mailing list?
> A) one way announcements - e.g. some kind of newsletter or market group
> B) dicussion lists

I've always thought of mailing lists as a proxy for one to many
communication, ie an artificial construct which allows an individual
(me!) to interact with a group of people as if it was a single person.

In your case A, I would be "interacting" by never responding to messages,
while in case B I would respond to messages if I like. I can also 
mine the mailing list messages for information, such as a person's
email address, and do something such as send them email directly, but
that's "beside" the mailing list interaction, I don't consider it part
of the interaction.

> 
> Thus I'm very comfortable about my most interesting dicussion lists, which
> set a 'Reply-To' header of the list and a mail reader that does ask me on
> reply whether I actually want to answer to the reply-to (prefered default)
> or to the original sender. Problem: If the sender defined a Reply-To on
> his own, this is replaced by the list. Thus if you want to answer to the
> author directly, you can't.

Yes, which is why people have come up with many different types of 
extra headers to handle such cases. There are also the complementary problems
where an author may not want their address added, so that people can't 
reply directly to him or her.

To me, these are list policy issues. Some lists may insist that posters
display a correct reply address, other lists might anonymise posters, 
other lists might restrict who can post and when in complex ways. 

When personal addresses are displayed in headers such as Reply-To, it
gives the MUA the ability to interact directly with list posters, in a
way which doesn't involve the list. So I don't see it as a mailing
list problem. Only if a message is sent to the list address (possibly
with other destinations as well) will it become a mailing list problem. 

> 
> The other option that you mentionned concerned that some people prefer to
> view messages as soon as possible which are replies to their own message.
> That's a reasonable question. But I feel the best idea here would be some
> kind of better threading or filtering in order to detect those messages
> which are replies to my own message - e.g. control of my own message-id
> and filtering on the In-Reply-To or References.
> 

If the mental picture I described in the previous post is reflective
of the truth, then the source of difficulty in that case is an
information overload problem.  Some people subscribe to many lists or
high volume lists, and even with better threading or filtering keeping
an overview may be difficult or require too much time investment.

For such people, the more practical method is not to actively monitor
a large number of sources for interesting things, but instead to 
arrange for interesting things to find them on their own whenever
they occur.

For example, consider a high volume mailing list (Arnt's bugtracking
mailing list example). A non-list member X makes a post, but has no
intention of monitoring the mailing list, or even to subscribe to it
and keep it in a separate folder to see if somebody responds. He
doesn't know how quickly a response might come, a few hours, a few
weeks, or whether a response will come at all. Having better threading
or filtering is useless, because the mailing list traffic is not going
to be monitored by his MUA.
 
But the mailing list server knows immediately that X is not a list
member, so if his post is accepted (this is a policy issue), then if
somebody else responds to this post on the mailing list, the list
server can send X a copy of the message. The people who are list members
don't need to know if X is a list member or not, everything is handled
by the list server.

> Unfortunately, a good threading is broken on many lists: 

> 
> That's one of the reason why newsgroups are much more efficient here: Both
> subject manipulation (one Re: only) and References are within a much
> tighter standard.
> 

Yes, it's a big problem, and requires a better standard.  

> > What I'm suggesting is that the mailing list server should act as a
> > more intelligent multiplexer, which could interface with people who prefer
> > the 1) custom according to the 1) custom, and could interface with
> > the people who like 2) according to their own preferences, etc.
> 
> No, I feel
> - the list server should work reasonable
> - people should adopt reasonable tooling which does what they actually
>   want.
> 
> ok, I know, wishful thinking...

The fundamental issue is whether you want the logic to reside at the
extremities of the network (ie in the MUA software) or at the center
(ie the mailing list software). Currently, I consider a mailing list
as a star topology:

            list member       list member
                    \        / 
 list member    -  list server  - list member
                      /     \
            list member       list member

(where perhaps some members have more privileges than others) When you
want people to have smart MUAs, then conceptually you are imposing
extra direct links between all the list members. That's fine for non
list communication, but for list communication I think it's simpler to
pass everything to and from the list server only. 

This line of thinking also reduces the need for extra message headers,
since in this view, the function of extra mailing list headers is
mainly to transfer enough addressing information to a receiving MUA,
so that the logic residing within this MUA can perform the replying
and other interaction it wants. 

> 
> > Simulating custom 1) for list members who like that custom:
> > 
> > * To send mail, a list member sends a single mail to the list address only.
> > * For each new mail posted to the list, the list server sends a single
> >   copy to the said list member.
> 
> custom 1) can be handled more flexibly. 
> If the list does not set a Reply-To, you have to use e.g. the less known
> List-Reply option or the easiest, best known way as a 'group reply'.

Yes, but this is logic which exists inside the MUA. Before the MUA can
perform this type of reply, it needs to learn the destination addresses
by reading both the Reply-To and List-Reply fields. 

In the scenario I'm proposing above, there is no need to read the
List-Reply field or other specialized fields. In my scenario, the
Reply-To field contains the mailing list address, and the list server,
when it receives the reply, decides what to do with it, how many
copies to send to each list member, etc.

If the MUA wants to reply directly to the poster without passing through 
the mailing list, then for example the From: field can be used. 
There are only two operations: list reply and personal reply off list.
If it's a list reply, the logic inside the list server should take care
of everything, if it's reply off list, then the MUA should take care of
everything.

> Thus the list server COULD handle that a member won't get a message twice.
> Each member could tell the server that he either prefers duplicate
> messages (directly + list) or one only (so that the list server won't send
> a copy to him if he's in the To/Cc).
> 
> The sender still can inforce both e.g. by putting a recipient as Bcc.

Yes, but doing so would become unnecessary as list servers become smarter.

> 
> > Simulating custom 2) for list members who prefer that custom:
> > 
> > * To send mail, the list member sends a single mail to the list address only.
> > * For each new mail posted to the list, the list server sends a copy
> >   to the said list member. Furthermore, the list server checks in the 
> >   list archive to see who posted the parent message, and if the parent is
> >   identical to the said list member, then a second courtesy copy is sent
> >   to this list member.
> 
> Hm - strange thing that I actually never wanted. But probably this
> courtesy copy could act in most different styles, e.g. by sending a
> digested overview of answered messages. Or by sending a SMS or any other
> type of message on any other message channel.

I should explain this better, as I see it. Nowadays, people's MUAs can
look in several fields (To, Cc, ...) for the presence of the list
address (e.g. [email-masked]). If this address is found, then the
message is a list message, so it is put in another folder say, away
from the inbox.

But with custom 2), the personal courtesy copy doesn't contain the
list address in those places, so when it is received it probably stays
in the inbox.  When the human being looks at his inbox, he sees the
courtesy copy and realizes that there has been a reply on the mailing
list. He then takes action as he wants to. Without the courtesy copy,
he would only see the reply inside the list folder, and he only looks
at this folder once a week maybe...

So in my simulation scenario above, the list server sends the normal
copy containing the list address, which will probably be filed away 
automatically in the folder. But the list server also sends a courtesy
copy where the list address is absent, which will end up in the 
inbox and be seen. Of course, the exact details are a question of 
preferences chosen by the list member.

> 
> However, this kind of service could be done not only by the list server,
> but more powerful
> - on the user's mail agent which does proper sorting/filtering or
> - by some kind of dedicated service provider which will filter
>   incoming messages from the list and will provide further actions,
>   such as your courtesy copy.

Yes, that is possible as well. However, in that case, we already have
the current system, where people get unwanted duplicates and other people
don't receive personal replies. This extra traffic can only be 
"fixed" by making it unnecessary, ie if the list server handles it.
Behind the scenes, the list server implementation could involve
a dedicated service provider etc. of course.

> 
> > This sort of system accomodates everybody at a high level, and the
> > cost is small (modifying list servers) compared with some alternatives
> > proposed (modifying everybody's MUA, or modifying everybody's
> > behaviour). 
> 
> It depends on the kind of service that you want. At the moment you feel a
> courtesy copy is good enough. But maybe someone else prefers a short note
> on his pager or mobile phone. Maybe someone wants a short automated phone
> call - or even a telegram or nice piece of paper on a silver platter ;-)

Luckily, people don't yet phone me up for every message
they send to some mailing list, but it's possible ;-) 

-- 
Laird Breyer.

Date: 1 Oct 2004 20:04:49 -0000
Mail-Followup-To: [email-masked]
Automatic-Legal-Notices: See http://cr.yp.to/mailcopyright.html.
From: "D. J. Bernstein" <[email-masked]>
To: [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

Bruce Lilly writes:
> It appears that MFT proponents cannot agree what it is supposed to do

In fact, all of us are in perfect agreement as to what Mail-Followup-To
does: it replaces To etc. as the default followup target, in exactly the
same way that Reply-To replaces From as the default reply target.

Comments such as ``In this case, Mail-Followup-To includes both of the
crossposted lists'' are simply _examples_ of how senders commonly use
and benefit from Mail-Followup-To. They don't change the _meaning_ of
the field. The Mail-Followup-To field always means the same thing: it
replaces To etc. as the default followup target. Nothing more, nothing
less.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

Date: Thu, 30 Sep 2004 21:57:51 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Keith Moore <[email-masked]>
CC: [email-masked]
Subject: Re: Understanding response protocols
Sender: [email-masked]

Keith Moore wrote:
> 
> Traditionally, with a meatspace meeting room, you were
> either inside the room (and participating in the meeting) or you were
> not. [ ...]
> Including someone outside the room in a discussion that was going on
> inside the room was generally infeasible without special arrangements. 

Some minor nits: it is not unheard of for individuals to participate
in meetings via written correspondence (I'm thinking of paper, but it
works now with electronic communications also) or via telephone.

Date: Thu, 30 Sep 2004 21:53:20 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Understanding response protocols
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>>Try RFCs 1958 and 2277, paying attention to the difference between
>>protocol elements and text.
> 
> Yes, using "LIST" a keyword in group syntax is not the most perfect
> solution, but then we are not living in a perfect world.
[...]
> I don't think we are in a position seriously to consider new syntax for
> To:, Cc: and Reply-To.

Then why have you proposed such a change -- that's exactly what changing
non-protocol text to "a keyword" would do?

> I have proposed a solution that could work.

No, you've proposed fundamentally changing fields in a way that is extremely
unlikely to work; a way that depends on guesswork, a way that invites
confusion and ambiguity. And it does noting at all to address the
fundamental issues, viz. that an author needs to provide a clear indication
of where he wishes responses to be sent.

Date: Thu, 30 Sep 2004 21:01:47 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: 822 <[email-masked]>
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: 822 <[email-masked]>
Subject: Re: Understanding response protocols
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> 
>>Charles Lindsey wrote:
> 
> 
>>>The only place where it is known that a crosspost is occurring is at the
>>>original sender.
> 
> 
>>Not true, on at least two counts:
> 
> 
>>1. an author might not know that a particular mailbox expands to a list.
> 
> 
> Sure, the author might not know [...]

So you agree that your original statement was untrue.

> But for sure if the author is not aware of it, then agents further
> down the chain will be even less aware of it.

That's obviously wrong; clearly a downstream list expander (or
its maintainer) is aware that list expansion is taking place.

> Hence the original sender is that latest point

Still drawing bad conclusion from false premises, I see.

> Unless, of course, you have a better solution to propose.

Prior to a "solution", one needs to identify a real "problem".
You haven't done so; you've simply made some false assertions
and have drawn wrong conclusions -- that itself is o "solution".

>>2. In the case of a mailing list expander where one of the expansion
>>  mailboxes is a list submission mailbox, the maintainer of the
>>  first list expander might indeed know that another list is being fed.
> 
> 
> In that case, the first list expander is in the position of the author

Emphatically no. An author and a list expander are two distinct
entities which perform very different functions.

>>Bad conclusion based on a false premise.
> 
> 
> And you have a better solution?

"Solution" to what? To your predilection for making false
statements, then drawing bogus conclusions from them? You'll
have to solve that for yourself. I suggest thinking before
writing.

>>Which proposal? I haven't seen one that states that MFT "indicates that
>>a crosspost is occurring".
> 
> 
> It can be arranged that MFT includes all the crossposted lists, which will
> at least get replies sent to all the lists. If the MFT can also indicate
> which of its addresses were lists, that would be even better.

It appears that MFT proponents cannot agree what it is supposed
to do; you claim that it "includes all the crossposted lists",
Bernstein claims that it requests that an "address be excluded from"
some response, others claim that it indicates either addresses or
mailboxes (they can't agree on which) for some sort of response.
That latter seems closest to the way it is used, in those rare
cases when some minor UA that supports it it used, i.e. as a
non-standard equivalent of Reply-To.

Date: Fri, 1 Oct 2004 16:21:36 +0200
From: Martin Trautmann <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

On 2004-10-01 22:26, Laird Breyer wrote:
> What is the culture clash? 

Since we moved slightly (?) offtopic here:

yet a perfectly different question for cultural differences is the kind of
'salutation'. I suppose there are major differences here from country to
country, which may result in even more confusion as long as you don't know
about the cultural differences elsewhere.

Here in Germany it would be considered impolite NOT to start a letter with
a greeting and NOT to end it with some kind of complimentary close.

Formal letters most of the times start with
"Sehr geehrte Damen und Herren" (Dear ladies and sirs) and end with
"Mit freundlichen Grüßen" (With kind regards; Sincerely yours).

Private letters start e.g. with
"Lieber <name>" (Dear <name>) and end e.g. with
"Viele Grüße" (greetings).

Germans tend to use those, and comparable translations in English writing
as well - and maybe the receiver wonders about. I was told that any kind
of greeting in the states may be accepted as some kind of very formal
attitude. Thus the reader will feel more distracted/distanced here?

On the other hand someone in Germany may feel weird if any stranger would
send him an letter and sign this with 'Love, ..."

Behavior has changed significantly via email - where everyone uses the
direct 'you' ('du' instead of 'Sie', 'third' person 'thou'?). But I do
note those differences e.g.  on salutation or on perfectly different
quoting behavior, where fullquotes in German newsgroups / usenet are an
almost obvious sign of a newbie, while most of the people want short and
helpful quoting only.

Regards
Martin

Date: Fri, 1 Oct 2004 16:07:35 +0200
From: Martin Trautmann <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

On 2004-10-01 22:26, Laird Breyer wrote:
> 
> Martin,
> 
> Thanks for the list of cases. I realize that when I wrote the comment
> earlier, I formulated things clumsily, so I'd like to restate things
> more precisely (hopefully) instead of replying to your points directly.
> I can reply more directly in another comment if necessary. 

Hi Laird,

thanks for the clarification. Unfortunately, I changed the subject since I
felt the topic was related to non-members.

Since I don't want to change the topic now again, let's live with this
topic above for some while. Maybe the discussion will be on non-members,
too :-/  

Ok, general topic 'list replies', here we go...

> The main issue I see is that some people like the following custom
> 
> 1) always post messages only to the list address.

> What is the culture clash? It appears that people who favour 1) do so
> because they don't want to see duplicate messages in their MUA
> systems.  [I myself favour 1)] From what I understand, it appears to
> me that people who favour 2) want to receive personal copies so that
> they don't inadvertently miss a reply to one of their own posts.
> 
> So as a mental picture, we could say that people who favour 1) are
> happy to browse mailing lists and monitor all threads equally at their
> leisure. People who favour 2) aren't prepared to monitor all threads
> in a mailing list, but want to know asap anything that affects them
> directly, such as a direct reply to one of their posts. This is just
> a mental model to fix ideas.

I don't see that much of a problem here. I agree that people want to use -
or abuse - lists the way you explain.

But apart from the user's clash of culture, which is based on a very
personal flavour, it should depend on the idea of a list as well.

What's the idea of a mailing list?
A) one way announcements - e.g. some kind of newsletter or market group
B) dicussion lists

Some lists are configured in order to keep the traffic los: List admins
felt it's a good idea to keep feedback off the list. Only if the user
actually wants to send somethin to the list, he will have to change this
for his reply manually. I feel that's good for certain newsletters and
ANNounce boards. It's even good for a question - answer list.

But I feel that a 'discussion' list lives not only from the first question
but from the answer to this question which may be good (in order so that
others can learn from it) or bad (so that others should react on it) or
may give new questions.

Thus I'm very comfortable about my most interesting dicussion lists, which
set a 'Reply-To' header of the list and a mail reader that does ask me on
reply whether I actually want to answer to the reply-to (prefered default)
or to the original sender. Problem: If the sender defined a Reply-To on
his own, this is replaced by the list. Thus if you want to answer to the
author directly, you can't.

The other option that you mentionned concerned that some people prefer to
view messages as soon as possible which are replies to their own message.
That's a reasonable question. But I feel the best idea here would be some
kind of better threading or filtering in order to detect those messages
which are replies to my own message - e.g. control of my own message-id
and filtering on the In-Reply-To or References.

Unfortunately, a good threading is broken on many lists: 
I)   most, but not all mail readers set References / In-Reply-To properly
II)  digest members have almost no choice to reuse references
III) many members don't mind, since there own mail reader does not
     know about threading

I) is a petty - and even worse since there's no hard standard how to mark
   a reply. Thus I use some kind of awkward 'set reply_regexp'
   which groups together at least which conforms to a set of
   subject prefixes out of
     Re: Re1: Re:2: Re^3: Re-4: RE[5]: AW: Antw: Antort: FW: FWD: WG: VS:
     SV: Accepted: Zugesagt:   (tbc.)
   in order to group at least by subject

II) view digests show the original message number at all. There's a small
set of lists around that does try to guess the original message by
comparing quoted text and former messages. Some of the do delete those
undesired full quotes as well. 

I guess some digested lists do work as well by adding a thread or
subthread number to the subject which is stripped and replaced by proper
references.

III) is the problem that only few list members know about the profit of
good threading.

That's one of the reason why newsgroups are much more efficient here: Both
subject manipulation (one Re: only) and References are within a much
tighter standard.

> I believe the clash cannot be solved at the MUA/client ends of the
> network, because it requires that each list member knows something
> about each other list member.

Impossible - most members do not even know about passive lurkers on the
list.

> What I'm suggesting is that the mailing list server should act as a
> more intelligent multiplexer, which could interface with people who prefer
> the 1) custom according to the 1) custom, and could interface with
> the people who like 2) according to their own preferences, etc.

No, I feel
- the list server should work reasonable
- people should adopt reasonable tooling which does what they actually
  want.

ok, I know, wishful thinking...

> Simulating custom 1) for list members who like that custom:
> 
> * To send mail, a list member sends a single mail to the list address only.
> * For each new mail posted to the list, the list server sends a single
>   copy to the said list member.

custom 1) can be handled more flexibly. 
If the list does not set a Reply-To, you have to use e.g. the less known
List-Reply option or the easiest, best known way as a 'group reply'.

Thus the list server COULD handle that a member won't get a message twice.
Each member could tell the server that he either prefers duplicate
messages (directly + list) or one only (so that the list server won't send
a copy to him if he's in the To/Cc).

The sender still can inforce both e.g. by putting a recipient as Bcc.

> Simulating custom 2) for list members who prefer that custom:
> 
> * To send mail, the list member sends a single mail to the list address only.
> * For each new mail posted to the list, the list server sends a copy
>   to the said list member. Furthermore, the list server checks in the 
>   list archive to see who posted the parent message, and if the parent is
>   identical to the said list member, then a second courtesy copy is sent
>   to this list member.

Hm - strange thing that I actually never wanted. But probably this
courtesy copy could act in most different styles, e.g. by sending a
digested overview of answered messages. Or by sending a SMS or any other
type of message on any other message channel.

However, this kind of service could be done not only by the list server,
but more powerful
- on the user's mail agent which does proper sorting/filtering or
- by some kind of dedicated service provider which will filter
  incoming messages from the list and will provide further actions,
  such as your courtesy copy.

> This sort of system accomodates everybody at a high level, and the
> cost is small (modifying list servers) compared with some alternatives
> proposed (modifying everybody's MUA, or modifying everybody's
> behaviour). 

It depends on the kind of service that you want. At the moment you feel a
courtesy copy is good enough. But maybe someone else prefers a short note
on his pager or mobile phone. Maybe someone wants a short automated phone
call - or even a telegram or nice piece of paper on a silver platter ;-)

> The question is, could such an approach solve most of the issues brought
> up by various people in this discussion?

Simple answer: no :-)
Martin

Date: Fri, 1 Oct 2004 22:26:04 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: non-member messages to lists (was Re: reply etiquette)
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 list <[email-masked]>
Sender: [email-masked]

Martin,

Thanks for the list of cases. I realize that when I wrote the comment
earlier, I formulated things clumsily, so I'd like to restate things
more precisely (hopefully) instead of replying to your points directly.
I can reply more directly in another comment if necessary. 

I am not directly trying to propose policies for membership posting
rights in a mailing list. I'll explain the issue I see, and why I
think the list server should be responsible for solving it.

The main issue I see is that some people like the following custom

1) always post messages only to the list address.

While others prefer the custom

2) always post messages to the list address and the parent poster.

Variations are possible, but the "culture clash" is simplest with 1) and 2).

What is the culture clash? It appears that people who favour 1) do so
because they don't want to see duplicate messages in their MUA
systems.  [I myself favour 1)] From what I understand, it appears to
me that people who favour 2) want to receive personal copies so that
they don't inadvertently miss a reply to one of their own posts.

So as a mental picture, we could say that people who favour 1) are
happy to browse mailing lists and monitor all threads equally at their
leisure. People who favour 2) aren't prepared to monitor all threads
in a mailing list, but want to know asap anything that affects them
directly, such as a direct reply to one of their posts. This is just
a mental model to fix ideas.

The clash exists because people who favour 1) cannot predict which
people favour 2) and vice versa. Or perhaps, accomodating the 2) people
would be too much work for the 1) people, etc.

I believe the clash cannot be solved at the MUA/client ends of the
network, because it requires that each list member knows something
about each other list member.

What I'm suggesting is that the mailing list server should act as a
more intelligent multiplexer, which could interface with people who prefer
the 1) custom according to the 1) custom, and could interface with
the people who like 2) according to their own preferences, etc.

To see if that is possible, we must look at the streams of messages
to and from the mailing list address, for each individual which might
interact with the mailing list in one way or another. 

Now, I don't want to widen the problem unnecessarily, and my earlier
post brought up authentication and membership policies, etc. 
Those things should be left for the individual list administrators.

I only want to look at how a list can technically simulate custom 1)
only for those who like that custom, and at the same time simulate
custom 2) only for those who like that other custom. People who aren't
list members could be a third category, and so on, but I'll leave that
aside for now, and I should have left it aside in my earlier post.

Simulating custom 1) for list members who like that custom:

* To send mail, a list member sends a single mail to the list address only.
* For each new mail posted to the list, the list server sends a single
  copy to the said list member.

Simulating custom 2) for list members who prefer that custom:

* To send mail, the list member sends a single mail to the list address only.
* For each new mail posted to the list, the list server sends a copy
  to the said list member. Furthermore, the list server checks in the 
  list archive to see who posted the parent message, and if the parent is
  identical to the said list member, then a second courtesy copy is sent
  to this list member.

This sort of system accomodates everybody at a high level, and the
cost is small (modifying list servers) compared with some alternatives
proposed (modifying everybody's MUA, or modifying everybody's
behaviour). 

I'll stop this comment here rather than go into implementation
details, so as to keep the approach as clear as possible and prevent
the confusion of my previous comment. But it is clear to me that variations
of the above ideas can be thought up for non-members etc.

The question is, could such an approach solve most of the issues brought
up by various people in this discussion?

-- 
Laird Breyer.

Date: Fri, 1 Oct 2004 13:37:52 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: reply etiquette
Cc: Bruce Lilly <[email-masked]>
Sender: [email-masked]

Bruce Lilly writes:
> Arnt Gulbrandsen wrote:
>
>>  Don't google. Tell me: Is [email-masked] a list or not?
>
> One cannot tell from the name alone.

Agreed.

> If you insist on a guess, I'd guess that it probably expands to at 
> least one list, since
> both KDE and FreeBSD are open source projects with multiple participants.

I don't insist on a guess. But I have the impression that some 
participants of this discussion insisti that the sender guess and 
insert mail-followup-to based on the guess.

Arnt

Date: Fri, 1 Oct 2004 13:07:11 +0200
From: Martin Trautmann <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: non-member messages to lists (was Re: reply etiquette)
Sender: [email-masked]

On 2004-10-01 12:13, Laird Breyer wrote:
> From the point of view of a non-member of the list, who just wants to
> send a single message without subscribing, the list server can have the
> following default behaviour:
> 
> 1) if a new list message is a direct reply to the non-member's email, it
> should be forwarded to his address, possibly with instructions on
> how to disable this feature if he wishes.

I did not understand this option.

First of all, is the non-member's email to the list accepted or refused?
You assume that it is always accepted, since otherwise nobody could reply
to him!?

The first default behavior on acceptin a non-member's email could be
1) ignore
2) refuse
3) return some handling instructions
4) let the message be administrated
5) accept by reply handling
6) accept by spam filtering
7) accept everything

1), just ignore should be discouraged

2) refuse is very reasonable and a default behavior on most lists

3) handling instructions is part of a refuse message, most of the time.
The typical answer is 'you must be a subscribed member'.

However, a better method might be to return a 'one time license':
"If you really want to send your message to the list, please do it once
again, but add the first line  'non-member-posting: 123456' to your
message"

Or e.g. send the message e.g. to 'list-approved-<number>@listserver' once
again which will accept the message for 48 hours to send it to the list

4) admin - another typical behavior is that non-members' messages are
forwarded to the list admin who can approve and forward them to the list. 

5) external messages could be accepted as long as they are a reply on a
message that passed the list within e.g. the last four weeks

Here Laird's option may jump in so that a list reply to a member's message
could be sent to this list member only, instead of the full list. Thus
this member could administrate whether he would like to forward the
non-member message to the list. However, this may cause plenty of other
problems, e.g. that 

* the list member guessed that everyone else had received this message as well
* the non-member did not expect a forward to the list
* the non-member actually is a member. He sees that his message did not
  make it to the list and resends it, this time as a member

6) external messages could be verified by rigid spam filters that let pass
well-behaved messages only

7) ... accept just every spam who might pass through. Still a very common
behavior, as long as it's an unknown, unarchived list where spam adress
lists don't know the name.

Regards
Martin