Subject: X- fields (Re: Getting 2822 to Draft)
From: Bruce Lilly <[email-masked]>
To: Keith Moore <[email-masked]>
Cc: Pete Resnick <[email-masked]>, [email-masked]
Date: Thu, 26 Feb 2004 11:41:54 -0500
List-Id: <ietf-822.imc.org>

Keith Moore wrote:
> 
>> Start saying things like that and you'll only encourage me to write
>> the "X- considered harmful" document. ;-)
> 
> 
> and if you do, I'll write the "X- considered wonderful" document :)
> 
> actually I think we could probably find a compromise position that we
> could agree on, as long as we don't make it part of the critical path
> for more important work.

I just skimmed RFC 3692, which appears to be such a document (though
not specifically addressing X- fields).

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

Date: Thu, 26 Feb 2004 12:21:44 +0000
To: Keith Moore <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

Keith,

I think our debate has pretty much run its course.  I happen to think 
HTTP-based archives have greater utility than you appear to do, but that's 
fine.

(Answering another question on this list recalled to me that one of the 
benefits of this proposal used in conjunction with HTTP-based archives is 
that it makes it easier to construct threads of discussion and reasoning 
that may be completely separate from specific message/response sequences 
seen in any single email thread.)

While I don't regard it as essential, I've no objection that the spec might 
say some things about the topics you suggest, assuming that it doesn't 
become overbearing.  And, of course, someone has to provide some suitable text.

#g
--

At 10:39 24/02/04 -0500, Keith Moore wrote:
>So I'm currently thinking that Archived-At needs to supply more
>information: namely the content types that are available.  An
>alternative might be to use a HEAD request to find out what
>content-types are available.
>
>Either way, this document probably needs to say some things about
>
>- use of different representations of email messages, with native
>   (message/rfc822) format being strongly encouraged as the primary
>   format and HTML as a useful alternative.
>
>- use of HTTP vs. IMAP (maybe also mentioning POP or even NNTP)
>   as an access protocol
>
>- how a mail reader (or for that matter a web browser) should decide
>   whether to handle the message itself or hand it off to a different
>   tool.
>
>- (maybe) organization of mail archives.

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Thu, 26 Feb 2004 12:10:29 +0000
To: Bill McQuillan <[email-masked]>, [email-masked]
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

At 15:51 25/02/04 -0800, Bill McQuillan wrote:
>What is the usefulness of a header field containing a URI which only
>fetches another copy of the same message?

It is extraordinarily useful in group discussions by email, in which a 
participant creates messages with direct, easy-to-retrieve references to 
previous messages.  While looking at one message, one can cut-and-paste the 
URI into another message under construction.  It is especially useful when 
reading messages without a live Internet connection (*).

The alternative is to spend time online rummaging around in the mail 
archives to find the same message in archived form, which can be a pain, 
especially when the archives are not well organized.

Partly because of this feature, but also because of the way the archives 
are presented, I have found the W3C mailing lists service to be the most 
usable of any that I have experienced.

#g
--

(*) I sometimes think that there are protocol designers who take an 
always-on Internet connection for granted.  One doesn't have to live in 
some desert region of the world to be reliant on dial-up for Internet access.

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Cc: Keith Moore <[email-masked]>, IETF RFC-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Wed, 25 Feb 2004 22:13:27 -0500
To: Bill McQuillan <[email-masked]>
Sender: [email-masked]

> This was actually my point! Some sort of access other than an imap: or 
> ftp:
> access to the raw message/rfc822 is what most of us would like to get.

Well, if the IMAP archive were organized so that all of the messages 
sent to one list were in the same folder, the URI would actually serve 
as a pointer to not only the message, but the location of the message 
within the list archive.  And from there (especially with a threads 
extension) the MUA could presumably look for other messages in the same 
thread.

Cc: Keith Moore <[email-masked]>, [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Wed, 25 Feb 2004 22:10:29 -0500
To: Bill McQuillan <[email-masked]>
Sender: [email-masked]

> What is the usefulness of a header field containing a URI which only
> fetches another copy of the same message?

a) so you can copy the URI to anywhere else you would like to put a 
reference to that message.
b) so you can delete the message but keep the URI.
c) so you can use that URI as a starting point to get to other messages 
that appeared in the same context.  the trick is how to identify what 
that context is (is it the mailing list in which the message appeared? 
which mailing list?) and how to navigate around that context.

Date: Wed, 25 Feb 2004 18:24:42 -0800
To: IETF RFC-822 list <[email-masked]>
From: Bill McQuillan <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

At 01:50 2004/02/26 +0000, Adam M. Costello wrote:
>
>Bill McQuillan <[email-masked]> wrote:
>
>> What is the usefulness of a header field containing a URI which only
>> fetches another copy of the same message?
>
>Even if that is all it did, it would be useful in this scenario:  I have
>a copy of a message in my personal mailbox, and I'm creating a web page
>in which I want to refer to that message.  Rather than copy the message
>into my own web page, I can copy the URI from the Archived-At: field
>of the message into a href attribute in the web page, and let readers
>fetch the message from the public archive.

In this case shouldn't the referred to item be a web page rather than a
message/rfc822?

>                                            A similar argument applies
>when I am sending a message to a large number of recipients, a few of
>whom might be interested in following my reference to another message.
>Rather than including a copy of the other message (causing it to be
>pushed out to everyone), I can copy the URI and let the readers fetch it
>if they care to.

This is one legitimate use I can see, but it seems to be of low utility to me.

>There is also the possibility that the proposed header field will allow
>more than just fetching the message; in particular, it might allow an
>easy way to access other messages in the same thread.

This was actually my point! Some sort of access other than an imap: or ftp:
access to the raw message/rfc822 is what most of us would like to get.
-- 
Bill McQuillan <[email-masked]>

Date: Thu, 26 Feb 2004 01:50:02 +0000
From: "Adam M. Costello" <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Reply-To: IETF RFC-822 list <[email-masked]>
Sender: [email-masked]

Bill McQuillan <[email-masked]> wrote:

> What is the usefulness of a header field containing a URI which only
> fetches another copy of the same message?

Even if that is all it did, it would be useful in this scenario:  I have
a copy of a message in my personal mailbox, and I'm creating a web page
in which I want to refer to that message.  Rather than copy the message
into my own web page, I can copy the URI from the Archived-At: field
of the message into a href attribute in the web page, and let readers
fetch the message from the public archive.  A similar argument applies
when I am sending a message to a large number of recipients, a few of
whom might be interested in following my reference to another message.
Rather than including a copy of the other message (causing it to be
pushed out to everyone), I can copy the URI and let the readers fetch it
if they care to.

There is also the possibility that the proposed header field will allow
more than just fetching the message; in particular, it might allow an
easy way to access other messages in the same thread.

AMC

Date: Wed, 25 Feb 2004 15:51:12 -0800
To: [email-masked]
From: Bill McQuillan <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

Asking the question straightforwardly:

What is the usefulness of a header field containing a URI which only
fetches another copy of the same message?

-- 
Bill McQuillan <[email-masked]>

Cc: Keith Moore <[email-masked]>, [email-masked], [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Wed, 25 Feb 2004 15:05:38 -0500
To: Charles Lindsey <[email-masked]>
Sender: [email-masked]

>> So I'm currently thinking that Archived-At needs to supply more
>> information: namely the content types that are available.  An
>> alternative might be to use a HEAD request to find out what
>> content-types are available.
>
> I think the problem you are raising is a generic URI problem, and not 
> just
> a problem with this particular header.

yes it's a generic problem with URIs (or to look at it a different way, 
a design choice made for URIs)

> AIUI, a URI gives you access to a "resource"; i.e. it tells you where 
> to
> find it. It tells you nothing about what to do with it when you have 
> got
> it, except what is implicit in the "scheme".

some would say that making assumptions about the nature of the resource 
based on the "scheme" is a bad idea.

> So what Martins draft needs to say is something like "If the scheme of 
> the
> URI is 'http', then the entity that is returned SHOULD have the
> Content-Type message/rfc822".

Maybe.  I'd like to encourage more archives to support native format 
access.  I do think it' s reasonable for a standard for accessing mail 
archives to behave predictably (as in, provide some minimum format for 
the sake of interoperability), and that it's reasonable to give mail 
readers the message in a format that they can use.  I'm concerned that 
simply saying "SHOULD use 822" will alienate those that don't.  "SHOULD 
provide 822; MAY provide HTML or other formats if the access protocol 
supports content-negotiation seems about right to me.

>> - how a mail reader (or for that matter a web browser) should decide
>>  whether to handle the message itself or hand it off to a different
>>  tool.
>
> You can't really give detailed instructions in a standard about how
> individual readers or browsers are supposed to behave.

no, but I don't think a paragraph or two is too much.

Keith

Date: Wed, 25 Feb 2004 13:29:00 GMT
From: Charles Lindsey <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Newsgroups: local.mime
Sender: [email-masked]

Keith Moore writes:

>So I'm currently thinking that Archived-At needs to supply more
>information: namely the content types that are available.  An
>alternative might be to use a HEAD request to find out what
>content-types are available. 

I think the problem you are raising is a generic URI problem, and not just
a problem with this particular header.

AIUI, a URI gives you access to a "resource"; i.e. it tells you where to
find it. It tells you nothing about what to do with it when you have got
it, except what is implicit in the "scheme".

So if the scheme is 'ftp', then all you know is that you are going to get
a file of some sort.

If the scheme is 'imap', then you know that the object to be retrieved
will be something like an email message, and the rules for imap URIs will
doubtless tell you much more.

Now one of the possible benefits of the 'http' scheme is that the object
that comes back should have a Content-Type header. And what we want is for
this Content-Type to be message/rfc822.

So what Martins draft needs to say is something like "If the scheme of the
URI is 'http', then the entity that is returned SHOULD have the
Content-Type message/rfc822". Then whatever system asked for the
Archived-At object to be retrieved has a decent chance of being able to
display and process it like an email.

>Either way, this document probably needs to say some things about 

>- use of different representations of email messages, with native 
>  (message/rfc822) format being strongly encouraged as the primary
>  format and HTML as a useful alternative.

Yes, see above.

>- use of HTTP vs. IMAP (maybe also mentioning POP or even NNTP) 
>  as an access protocol

Yes.

>- how a mail reader (or for that matter a web browser) should decide 
>  whether to handle the message itself or hand it off to a different 
>  tool.

You can't really give detailed instructions in a standard about how
individual readers or browsers are supposed to behave. The general
intention is that all readers/browsers that recognize Content-Type headers
are supposed to use the best available tool for handling that Content-Type
(possibly even using a special plugin). Naturally, some current systems
make a better job of this than others :-( .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 25 Feb 2004 13:09:03 GMT
From: Charles Lindsey <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Newsgroups: local.mime
Sender: [email-masked]

Bruce Lilly writes:

>RFC 2017 provides for URIs in parameters. RFC 2557 uses URIs in a structured
>field. 2557 syntax is broken because it allows comments (this has been
>discussed on the MHTML list). RFC 2369 uses URIs in various List- (structured)
>fields.  2369 uses a quoting mechanism which permits comments outside of the
>quoted URIs.

RFC 2017 deals with URIs as parameters, insists that they be in the form
of quoted-strings, and then allows you to insert FWS at selected places,
e.g.:

                   URL="ftp://ftp.deepdirs.org/1/2/3/4/5/6/7/
                        8/9/10/11/12/13/14/15/16/17/18/20/21/
                        file.html"

The broken-ness of RFC 2557 seems to arise from the fact that it does not
insist that any parentheses within the URI get %-quoted. Also from the
fact that it tells you to use the RFC 2017 mechanism for long URIs,
whereas that mechanism only applies to parameters. Also, it makes life
hard for itself by not providing delimiters (whether "..." or <...>)
around the URI. But, worryingly, Martin's proposed syntax also suffers
from some of these problems.

RFC 2369 appears to set the best precedents, including delimiting within
<...>, allowing FWS within those <...> (to be eliminiated before use as a
URI), and provision for comma-separated lists of such URIs.

Since the List-Archive header and the proposed [List-]Archived-At header
are quite likley to occur in the same message, it would be a great pity if
they did not follow the same syntactic conventions.

And I would much prefer a comma-separated list of URIs than multiple
Archived-At headers. For a start, it gives you a way to express some
priority between the various alternatives (as in the List-Archive header).
And then, multiple headers with the same name are Bad Thing for other
reasons (for example if ever you come to have digital signatures of
headers, such as have been mentioned recently in other threads on this
list).

>Whitespace and CRLF are not permitted in URIs.  So folding isn't a problem.
>See RFC 2396 Appendix E.

Not quite so.

<http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html>

seems to recommend, in its Appendix C, that you should "just put spaces"
into your URI (all genuine SP having already been replaced by %20) and
then fold, and it especially recommends delimiting within <...> at the
same time. So if that is the way the URI people are thinking, then that is
the way we should be going.

One wishes, however, that the provision had been made in the body of their
draft, rather than as an afterthought in an Appendix.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 25 Feb 2004 11:31:44 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Martin Duerst <[email-masked]>
CC: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

Martin Duerst wrote:

> Any definitive answers appreciated here. For the moment,
> I'm just keeping it with "one URI, no comments, no special
> mechanisms, follow the length limitations in RFC 2822".

In terms of ABNF, similar to RFC 2822, that would be something like:

archived-at = "Archived-At" ":" [FWS] URI-reference *WSP CRLF ; URI-reference not empty

where "URI-Reference" is defined in RFC 2396.  You may also want to
support parsing (but not generation in new messages) of older forms,
e.g.:

obs-archived-at = ("Archived-At" / "X-Archived-At") *WSP ":" [FWS] URI-reference [FWS] CRLF

You might want to replace URI-reference with absoluteURI in both
places, though that would preclude use of a fragment identifier.
Yet another alternative would be to use "absoluteURI ["#" fragment]"
which precludes relative URIs while permitting a fragment identifier.
It really depends on what restrictions you want to place on the types
of URIs to be permitted.

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

Date: Tue, 24 Feb 2004 10:39:57 -0500
From: Keith Moore <[email-masked]>
To: Graham Klyne <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

Graham,

I think the answer is that something else has been bugging me about this
proposal that I'm only starting to get a handle on.  I got some more
idea about it this morning when I woke up.

As I was saying earlier, there's something perverse in asking an email
reader to fire up a web browser to read an email message.  Part of the
reason that this is perverse, of course, is that the email reader is
tailor-made to do things with email messages and the web browser is not.
 Unless the MUA and web browser are very tightly coupled (which seems to
be the case less often these days) this seems like a suboptimal
arrangement.

And yes, HTTP can be used to download email messages, and HTTP content
negotiation can be used to allow a mail user agent to pick between
getting the message in message/rfc822 format and text/html format (or
some other format).  If the message is in rfc822 format it should stay
in the mail reader; if it's in html format it should probably go to the
web browser (Just like web browsers make poor mail readers; mail readers
make poor web browsers. Even if they support HTML they probably don't
support scripting, tabs, cookie management, security, history - things
that we have come to expect in order to use the web effectively, even
to read email archives).  But at the time the MUA decides what to do
with the URI, it doesn't know whether it should download the message
itself or hand it off to a web browser. 

So I'm currently thinking that Archived-At needs to supply more
information: namely the content types that are available.  An
alternative might be to use a HEAD request to find out what
content-types are available.  

Either way, this document probably needs to say some things about 

- use of different representations of email messages, with native 
  (message/rfc822) format being strongly encouraged as the primary
  format and HTML as a useful alternative.

- use of HTTP vs. IMAP (maybe also mentioning POP or even NNTP) 
  as an access protocol

- how a mail reader (or for that matter a web browser) should decide 
  whether to handle the message itself or hand it off to a different 
  tool.

- (maybe) organization of mail archives.

Some of this I understand Martin has already written, but I haven't
seen the revised document yet.

You asked me to comment on specific text that you suggested.  The reason
I didn't do so was because I sensed that the issues needed more thought
and that it was premature to try to write specific text.  

I do accept that there's a limit to how far this document should try to
go to address these issues -  this is after all a document about a new
header field, not about how to set up email archives.  My guess is that
about one paragraph for each of the above points is about right.

We could of course write a separate document on the subject of mail
archives, and perhaps we should.  But we don't want to hold up this
document waiting for a recommendation for mail archives that we can
agree on.  

> I think there's a real danger that your quest for something else will 
> impede the standardization of a Really Useful Feature, for which there
> is running code.  I think I have heard you subscribe to the idea that
> the best is the enemy of the good in standards development ... I think
> this could be a shining case in point.

I don't think it's too much to ask to discuss a Really Useful Feature
for a few days to try to understand its broader implications before we
try to promote some particular way of implementing it.  Email header
fields are a data model, and there's a large body of experience that
data models are really difficult to get right.  We want standards to be
stable, and we want them to be flexible enough to use for a long time. 
We're not going to get that by standardizing the first idea that comes
to mind -- even if it's easy to implement and there's running code and
that running code is found to have some utility.

It may be that we place too much emphasis on running code.  Any
programmer who has been around for a few years knows that a lot of code
that is "running" - maybe most of it - even if it is found to be useful,
often has a slapdash quality that won't necessarily hold up to the
strain of widespread use.  Also there's an important difference between
running code in the form of a prototype and running code that implements
a  well-designed specification.  The prototype is useful as a
proof-of-concept and to identify flaws in the initial design, but it's
rarely suitable for widespread deployment.  The running code that IETF
demands as a condition of advancement to Draft Standard serves a
different purpose - it is a check on the clarity and precision of the
specification, and that implementation is not too onerous.

--
He not busy being born, is busy dying.		- Bob Dylan

Date: Tue, 24 Feb 2004 09:58:27 +0000
To: Keith Moore <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

Keith... I made a suggestion that I think addresses your concern as 
previously expressed, but you don't respond to that.  Viz:
[[[[
If you feel there's an implied dependency, maybe something like this could 
be added to the text:
[[
One way to use this header field is to include an HTTP URI, which can be 
passed to a web browser to retrieve and display a copy of the archived 
message. But other URI schemes may be used, such as imap:, and mail 
applications may use the information thus provided in any way that assists 
access to archived mail messages.
]]
]]]] -- http://www.imc.org/ietf-822/mail-archive/msg04050.html

If developers don't choose to deploy imap: in the face of such a comment, 
then I submit that they are never likely to do so at the exhortation of any 
standard.

I think there's a real danger that your quest for something else will 
impede the standardization of a Really Useful Feature, for which there is 
running code.  I think I have heard you subscribe to the idea that the best 
is the enemy of the good in standards development ... I think this could be 
a shining case in point.

What is it that you want to happen to this proposal?

#g
--

PS:  FWIW, I use the feature described on an almost-daily basis, and I find 
that using a browser to search email archives is not at all 
perverse.  Quite often, I find myself switching between a mail archive and 
Google to track down information, and the browser is a natural user 
interface that spans both of these.  The feature described here is really 
very useful and is deployed today (modulo an X- at the front of the header 
field name).

At 19:38 23/02/04 -0500, Keith Moore wrote:
>>>Is this extension for email readers or for web browsers?
>>
>>Neither, or both.  It's an extension to the message *format*, not the 
>>protocol.  Applications may use it (or not) as they see fit.
>
>While that might be true in principle, it's not true in practice.  A mail 
>reader is much more likely to be able to make effective use of an IMAP URL 
>than an HTTP URL.  And it's more likely to do a good job handling a 
>message/rfc822 resource than with a text message encoded as 
>text/html.  There's something perverse in asking a mail reader to fire up 
>a web browser to read an email message.  And for some strange reason I 
>think that extensions to the email format that exist for the purpose of 
>reading mail should be usable by mail readers.

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Tue, 24 Feb 2004 08:41:02 +0000
From: "Adam M. Costello" <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Reply-To: IETF RFC-822 list <[email-masked]>
Sender: [email-masked]

Bill McQuillan <[email-masked]> wrote:

> I guess that the data object that we are missing here is
> the Archive itself with some sort of standardized access
> protocol.  Then the Archived-At field becomes some sort of
> <msgarch://host.name/archive.name?msg-id>.

You mean like <imap://host/mailbox/;UID=uid> ?

(See RFC-2192.)

AMC

Date: Mon, 23 Feb 2004 21:33:30 -0800
To: [email-masked]
From: Bill McQuillan <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

At 19:38 2004/02/23 -0500, Keith Moore wrote:
>
>mail reader is much more likely to be able to make effective use of an 
>IMAP URL than an HTTP URL.  And it's more likely to do a good job 
>handling a message/rfc822 resource than with a text message encoded as 
>text/html.  There's something perverse in asking a mail reader to fire 
>up a web browser to read an email message.  And for some strange reason 
>I think that extensions to the email format that exist for the purpose 
>of reading mail should be usable by mail readers.

It seems to me that the problem we're dancing around is that, as Keith said
earlier, there is little reason to access the same message in an archive
that one already has since the Archived-At field was gotten from it. On the
other hand, message/rfc822 doesn't have a good mechanism for linking the
the "context" of the message in an archive. References coule be used in
concert with List-Archive to access its precursor messages, but finding its
successors and reconstructing the thread tree would require a huge amount
of effort.

I guess that the data object that we are missing here is the Archive itself
with some sort of standardized access protocol. Then the Archived-At field
becomes some sort of <msgarch://host.name/archive.name?msg-id>.
-- 
Bill McQuillan <[email-masked]>

Cc: Keith Moore <[email-masked]>, [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Mon, 23 Feb 2004 19:38:32 -0500
To: Graham Klyne <[email-masked]>
Sender: [email-masked]

>> Is this extension for email readers or for web browsers?
>
> Neither, or both.  It's an extension to the message *format*, not the 
> protocol.  Applications may use it (or not) as they see fit.

While that might be true in principle, it's not true in practice.  A 
mail reader is much more likely to be able to make effective use of an 
IMAP URL than an HTTP URL.  And it's more likely to do a good job 
handling a message/rfc822 resource than with a text message encoded as 
text/html.  There's something perverse in asking a mail reader to fire 
up a web browser to read an email message.  And for some strange reason 
I think that extensions to the email format that exist for the purpose 
of reading mail should be usable by mail readers.

Date: Mon, 23 Feb 2004 21:44:40 +0000
To: Keith Moore <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

At 13:33 23/02/04 -0500, Keith Moore wrote:
> > >1. I really don't think we should standardize a header field pointer to
> > >mail archives that depends on HTML and HTTP.
> >   :
> > >3. However, I don't think that we should limit such a standard to be
> > >used exclusively with IMAP or any other protocol.  Clearly it's much
> > >easier to deploy HTTP archvies than IMAP archives.
> >
> > On this, we agree.
> >
> > And there is nothing in Martin's draft that has any such dependency.
>
>I think there is an implied dependency of sorts.  Without specifying how
>to access archives in IMAP people are going to assume that HTTP is the
>only thing that can be used.  The danger is that we will paint ourselves
>into a corner where we are stuck with using HTTP.
>
>Is this extension for email readers or for web browsers?

Neither, or both.  It's an extension to the message *format*, not the 
protocol.  Applications may use it (or not) as they see fit.

My mail reader recognizes the http: protocol in am x-archived-at header and 
invokes a browser to retrieve and display the message.  But I'd expect it 
to respond differently to other URI schemes.

If you feel there's an implied dependency, maybe something like this could 
be added to the text:
[[
One way to use this header field is to include an HTTP URI, which can be 
passed to a web browser to retrieve and display a copy of the archived 
message.  But other URI schemes may be used, such as imap:, and mail 
applications may use the information thus provided in any way that assists 
access to archived mail messages.
]]

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Mon, 23 Feb 2004 14:09:45 -0500
To: Charles Lindsey <[email-masked]>, [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

At 12:49 04/02/23 +0000, Charles Lindsey wrote:
>Bruce Lilly writes:
>
> >Comments are incompatible with URIs unless some quoting mechanism is used.
> >That is because URIs may contain parentheses, as in
> >http://users.erols.com/blilly/(foo)(bar)
> >See RFC2396 for URI syntax details.  RFC 2369 quotes URIs in angle brackets,
> >which are not themselves allowed in URIs.
>
>Well this is not the first time someone has proposed a structured header
>with a URI in it, surely? So what has been done about this on previous
>occasions?

Any definitive answers appreciated here. For the moment,
I'm just keeping it with "one URI, no comments, no special
mechanisms, follow the length limitations in RFC 2822".

>Parentheses in URIs are not all that common, so they could be
>escaped. Or else the whole thing could be put inside a quoted-string.
>
>Using angle-brackets would doubtless also be a good idea, though I do not
>think that on its own would allow comments to appear outside of the angle
>brackets with parentheses inside being treated differently.

The less complications, the better in my opinion.

> >One issue to consider is long URIs.  If comments are disallowed, no quoting
> >is required.  URIs cannot contain whitespace characters, so a simple way to
> >handle a long URI is to allow a URI to be line-folded; it can be 
> reconstructed
> >by unfolding and eliding any whitespace.
>
>Long URIs are a pain in message/article bodies too. Is there any
>possibility that those responsible to fixing the format of URIs could
>include a folding mechansim? I do not like the idea of fixing what is a
>more widespread URI problem within just a few particular email headers.

There is something explicit in the new URI draft, at
http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html,
but as far as I know, it's not followed by MUAs.

Regards,    Martin.

Date: Mon, 23 Feb 2004 14:01:03 -0500
From: Keith Moore <[email-masked]>
To: Martin Duerst <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

> >>Good point. But it is not really restricted to lists. There are
> >>other potential applications, which are not related to mailing lists.
> >>So starting with List-* would be confusing.
> >
> >Maybe it should only start with List- when added by a list.
> 
> Although I expect this header to mostly be used in connection
> with mailing lists, the archiving functionality is really not
> at all tied to mailing lists. Also, what would be the benefit
> of having Archived-At and List-Archived-At for the user who
> wants to find an archived copy of the message?

Having different fields for different parties adding an archived-at
field would allow the recipient to determine which party added that
information.   That way, if the information is misleading or inaccurate
or the archive is not properly maintained, the recipient has some idea
of who is responsible.  

> In our implementation, the URI you get with Archived-At is
> completely independent of any particular mailing list, and
> is the same for all mailing lists in case of cross-posting.

The question is not so much whether the archive is associated with 
a list, as what party supplied the information.  (though I'd also assert
that it's useful to see how a message fit into the conversation of
a particular list, even if the message was posted to more than one
list.)  That and sometimes you want to filter out certain kinds
of header fields (e.g. you want to remove List- fields from messages
that were re-submitted to other lists)

> >But it seems like we're finding more and more cases where it's a bad idea 
> >to have a header field without very clear rules on who is allowed to set 
> >that field.  We've seen this with Reply-To and Sender and also with Carl 
> >Malamud's Solicitation field proposal.
> 
> These are certainly good examples. But I think there is a difference
> between a header that can be added several times (as the Archived-At
> header) and a header that can only be added once (as I think is the
> case with the above examples).

> Or in other words: The rules are very clear, anybody who archives
> the message can add an Archived-At header, and nobody should
> overwrite existing ones. I clarified this in the draft.

Being able to have multiple instances of the field does help, but it still
begs the question of which intermediary added the field.
 
> >Being able to access the archived message from IMAP is potentially a lot 
> >better, but (as with many things) even IMAP needs some tweaking to make it 
> >work well for this.
> 
> Very good idea. I have mentioned this in the draft. I thought
> that the IMAP URI scheme only allows to address folders,
> but that's not at all the case.

I'll look forward to seeing the next draft.

Keith

Date: Mon, 23 Feb 2004 13:42:58 -0500
To: Keith Moore <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Keith Moore <[email-masked]>, [email-masked]
Sender: [email-masked]

At 17:23 04/02/22 -0500, Keith Moore wrote:

>>Good point. But it is not really restricted to lists. There are
>>other potential applications, which are not related to mailing lists.
>>So starting with List-* would be confusing.
>
>Maybe it should only start with List- when added by a list.

Although I expect this header to mostly be used in connection
with mailing lists, the archiving functionality is really not
at all tied to mailing lists. Also, what would be the benefit
of having Archived-At and List-Archived-At for the user who
wants to find an archived copy of the message?

In our implementation, the URI you get with Archived-At is
completely independent of any particular mailing list, and
is the same for all mailing lists in case of cross-posting.
I don't even know whether the header is added once when the
email comes into our mailing list system, or it is added
independently for each mailing list. Indeed, my guess is
that it's something in the middle. If we had been able to
tightly integrate the email sending and the archiving
process, we wouldn't have needed the MID redirection.

>But it seems like we're finding more and more cases where it's a bad idea 
>to have a header field without very clear rules on who is allowed to set 
>that field.  We've seen this with Reply-To and Sender and also with Carl 
>Malamud's Solicitation field proposal.

These are certainly good examples. But I think there is a difference
between a header that can be added several times (as the Archived-At
header) and a header that can only be added once (as I think is the
case with the above examples).

Or in other words: The rules are very clear, anybody who archives
the message can add an Archived-At header, and nobody should
overwrite existing ones. I clarified this in the draft.

>Being able to access the archived message from IMAP is potentially a lot 
>better, but (as with many things) even IMAP needs some tweaking to make it 
>work well for this.

Very good idea. I have mentioned this in the draft. I thought
that the IMAP URI scheme only allows to address folders,
but that's not at all the case.

Regards,    Martin.

Date: Mon, 23 Feb 2004 13:33:36 -0500
From: Keith Moore <[email-masked]>
To: Graham Klyne <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

> >1. I really don't think we should standardize a header field pointer to
> >mail archives that depends on HTML and HTTP.
>   :
> >3. However, I don't think that we should limit such a standard to be
> >used exclusively with IMAP or any other protocol.  Clearly it's much
> >easier to deploy HTTP archvies than IMAP archives.
> 
> On this, we agree.
> 
> And there is nothing in Martin's draft that has any such dependency.

I think there is an implied dependency of sorts.  Without specifying how
to access archives in IMAP people are going to assume that HTTP is the
only thing that can be used.  The danger is that we will paint ourselves
into a corner where we are stuck with using HTTP.

Is this extension for email readers or for web browsers?

Date: Mon, 23 Feb 2004 18:16:03 +0000
To: Keith Moore <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

At 12:16 23/02/04 -0500, Keith Moore wrote:
>Let me put it another way.
>
>1. I really don't think we should standardize a header field pointer to
>mail archives that depends on HTML and HTTP.
  :
>3. However, I don't think that we should limit such a standard to be
>used exclusively with IMAP or any other protocol.  Clearly it's much
>easier to deploy HTTP archvies than IMAP archives.

On this, we agree.

And there is nothing in Martin's draft that has any such dependency.

So I don't think there is a problem with [1] on these grounds.

(I've skipped the rest of your message:  I did read it, and I agree with 
some points you make, and not with others, but I don't think they're really 
germane to the specification on the table.)

#g
--

[1] http://www.ietf.org/internet-drafts/draft-duerst-archived-at-00.txt

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Mon, 23 Feb 2004 13:17:38 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

Charles Lindsey wrote:

> Well this is not the first time someone has proposed a structured header
> with a URI in it, surely? So what has been done about this on previous
> occasions? Parentheses in URIs are not all that common, so they could be
> escaped. Or else the whole thing could be put inside a quoted-string.

RFC 2017 provides for URIs in parameters. RFC 2557 uses URIs in a structured
field. 2557 syntax is broken because it allows comments (this has been
discussed on the MHTML list). RFC 2369 uses URIs in various List- (structured)
fields.  2369 uses a quoting mechanism which permits comments outside of the
quoted URIs.

> Using angle-brackets would doubtless also be a good idea, though I do not
> think that on its own would allow comments to appear outside of the angle
> brackets with parentheses inside being treated differently.

See RFC 2369.  The quoting mechanism differentiates what is quoted (the
URI) from what is not quoted (commas in the case of lists, whitespace,
line folding, comments).

> Long URIs are a pain in message/article bodies too. Is there any
> possibility that those responsible to fixing the format of URIs could
> include a folding mechansim? I do not like the idea of fixing what is a
> more widespread URI problem within just a few particular email headers.

Whitespace and CRLF are not permitted in URIs.  So folding isn't a problem.
See RFC 2396 Appendix E.

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

Date: Mon, 23 Feb 2004 12:16:07 -0500
From: Keith Moore <[email-masked]>
To: Graham Klyne <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

> >mail archives won't get better without specifications for how to make
> >them better.
> 
> I feel as if we're each missing the other's point.  Mine was two-fold:
> 
> (a) that a new facility that depended on IMAP might not be easy to get
> 
> deployed.  Maybe.  I was asking a question rather than making an
> assertion.
>
> (b) that I didn't see the need for any fundamental change to 
> draft-duerst-archived-at-00.txt to allow IMAP to used in conjunction
> with the new header.
> 
> So it seems I'm missing something in your position that makes mention
> of IMAP important to this specification?

Let me put it another way.  

1. I really don't think we should standardize a header field pointer to
mail archives that depends on HTML and HTTP.

2. IMAP seems like the best protocol available for the purpose of
accessing mail archives, so I think that IMAP should be cited in 
any standard for such archives.

3. However, I don't think that we should limit such a standard to be
used exclusively with IMAP or any other protocol.  Clearly it's much
easier to deploy HTTP archvies than IMAP archives.  

There's a conflict between what is easy to deploy and what would be 
a desirable state to encourage.  I don't want to insist that we only
approve a header that works in that more desirable state - I just 
want to make sure there's a clear upgrade path to that more desirable
state.  

> (Also, RFC3205 [1] notwithstanding, I do tend to wonder what it is
> that IMAP offers in this context that isn't performed equally well
> using an HTTP-based implementation.  With respect to section 2.5 of
> RFC 3205, HTTP seems pretty appropriate on all counts, given that most
> email archives are accessed today using HTTP. Accessing email
> archives seems to be a function that fits right into HTTP's core
> competence.)

HTML archives of mail suck, for reasons that RFC 3205 didn't even begin
to touch on.  You don't want to have to use a different interface to read 
mail achives than to read ordinary mail.  You want the same kinds of
sorting, searching, (re)filing, address book handling, annotation,
replying, etc. available to mail archived elsewhere as for your personal
mail archives.  HTTP could be used as a mailbox access protocol, but
it lacks most of the features needed - it is less functional than POP
for that purpose.

Here are some problems with using HTTP to access mail archives:
- Most MUAs don't know how to access messages using HTTP, which pretty
  much limits access to web browsers.   See below.
- There are no conventions for the mapping of HTTP URLs onto commonly
  understood abstractions for groups of email messages such as
  folders.  There is no way to list all messages in a folder, or to
  search through a folder for messages matching a particular pattern,
  etc.  WEBDAV might provide some of the missing functionality, 
  but it still needs to be defined how to use it for email.

Here are some problems with representing mail messages in HTML:
- it discards the fact that the resource was a mail message
- it translates valuable semantic content into mere text
  (more specific examples below)
- mail readers won't know what to do with it other than to display it.
  they can't reply to it, refile it, etc.
- server-side mail interfaces are slow, and cannot take advantage
  of client-side context such as address books, preferences for
  how outgoing mail is sent, how replies are composed, etc.
- can't do duplicate message suppression when the same message
  appears from multiple sources

Date: Mon, 23 Feb 2004 12:49:46 GMT
From: Charles Lindsey <[email-masked]>
To: [email-masked]
Cc: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Newsgroups: local.mime
Sender: [email-masked]

Bruce Lilly writes:

>Comments are incompatible with URIs unless some quoting mechanism is used.
>That is because URIs may contain parentheses, as in
>http://users.erols.com/blilly/(foo)(bar)
>See RFC2396 for URI syntax details.  RFC 2369 quotes URIs in angle brackets,
>which are not themselves allowed in URIs.

Well this is not the first time someone has proposed a structured header
with a URI in it, surely? So what has been done about this on previous
occasions? Parentheses in URIs are not all that common, so they could be
escaped. Or else the whole thing could be put inside a quoted-string.

Using angle-brackets would doubtless also be a good idea, though I do not
think that on its own would allow comments to appear outside of the angle
brackets with parentheses inside being treated differently.

>One issue to consider is long URIs.  If comments are disallowed, no quoting
>is required.  URIs cannot contain whitespace characters, so a simple way to
>handle a long URI is to allow a URI to be line-folded; it can be reconstructed
>by unfolding and eliding any whitespace.

Long URIs are a pain in message/article bodies too. Is there any
possibility that those responsible to fixing the format of URIs could
include a folding mechansim? I do not like the idea of fixing what is a
more widespread URI problem within just a few particular email headers.

Another possibility is some adaptation of the mechanism used for long
parameters in RFC 2231. Not that I am enamoured of the particular solution
provided there, but I am even less enamoured of inventing YAFM.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:3079
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Mon, 23 Feb 2004 12:34:18 GMT
Lines: 61
Sender: [email-masked]

In <4.2.0.58.J.20040219180004.0535c678@localhost> Martin Duerst <[email-masked]> writes:

>It defines a new email header, Archived-At:, to provide a direct
>link to the archived form of an individual mail message. We use
>this extremely successfully (currently in the form X-Archived-At)
>at W3C.

This could also have applications within Usenet. Currently, FAQs etc
posted to news.answers have a "pseudo-header" in the body of the article,
for example:

Archive-name: news-answers/guidelines
Version: $Id: guidelines,v 2.49 98/09/28 20:02:32 jjnichol Exp $
Posting-Frequency: monthly
Copyright: see Section 5

The meaning of that header is "look in the usual archive site, or one of
its mirrors, and look for the file news-answers/guidelines". I don't
overmuch like that "pseudo-header" convention, and I would like to see a
better mechanism, but for now there it is.

Now if an Archived-Header were to be used for the same purpose, it would
look like:

Archived-At: <ftp://rtfm.mit.edu/pub/usenet/news.answers/news-answers/guidelines>

Observe that the URI doesn't need to be 'http', and even if it is, it
doesn't mean that the document retrieved would be in HTML.

Anyway, this raises two further issues:

1. The syntax should allow a comma-separated list of URIs. This might
allow for the article to be retrieved in various different formats
(presumably the file extension at the end would tell you which).

2. The situation where there are numerous mirrors of the archive site
needs to be borne in mind. Is there already a mechanism within the URI
system for retrieving a resource from one of a number of sites?

If not, would something like the following work:

List-Archive: <ftp://rtfm.mit.edu/pub/usenet/news.answers/>,
              <ftp://rtfm.mirror.someplace.net/news.answers/>
Archived-At: <news-answers/guidelines>

where the Archived-At is presumed to be relative to one of the
List-Archive alternatives.

This also suggests that the name of the new header should be
List-Archived-At, as others have suggested.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 23 Feb 2004 16:05:45 +0000
To: Keith Moore <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

At 08:15 23/02/04 -0500, Keith Moore wrote:

>>>There are limits on what you can do with HTML and HTTP.  Being able to 
>>>access the archived message from IMAP is potentially a lot better, but 
>>>(as with many things) even IMAP needs some tweaking to make it work well 
>>>for this.
>>
>>How many people actually use IMAP?  (I don't.)
>
>why does it matter what they use to read their mail, as long as the right 
>thing happens when they click on an IMAP URI?
>
>lots of MUAs do support IMAP.  whether they support the ability to 
>reference messages via IMAP URLs is a different question.
>
>mail archives won't get better without specifications for how to make them 
>better.

I feel as if we're each missing the other's point.  Mine was two-fold:

(a) that a new facility that depended on IMAP might not be easy to get 
deployed.  Maybe.  I was asking a question rather than making an assertion.

(b) that I didn't see the need for any fundamental change to 
draft-duerst-archived-at-00.txt to allow IMAP to used in conjunction with 
the new header.

So it seems I'm missing something in your position that makes mention of 
IMAP important to this specification?

(Also, RFC3205 [1] notwithstanding, I do tend to wonder what it is that 
IMAP offers in this context that isn't performed equally well using an 
HTTP-based implementation.  With respect to section 2.5 of RFC 3205, HTTP 
seems pretty appropriate on all counts, given that most email archives are 
accessed today using HTTP.  Accessing email archives seems to be a function 
that fits right into HTTP's core competence.)

#g
--

[1] http://www.ietf.org/rfc/rfc3205.txt

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Mon, 23 Feb 2004 13:49:58 +0000
From: "Adam M. Costello" <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Reply-To: IETF RFC-822 list <[email-masked]>
Sender: [email-masked]

Keith Moore <[email-masked]> wrote:

> lots of MUAs do support IMAP. whether they support the ability to
> reference messages via IMAP URLs is a different question.

Here's one data point:  The way you tell Mutt to open an IMAP (or POP)
mailbox rather than a local mailbox is by giving it an imap: (or pop:)
URI instead of a pathname.  I don't think it understands the more
fine-grained imap: URIs though (the ones that refer to resources smaller
than a mailbox, like individual messages).

AMC

Date: Mon, 23 Feb 2004 08:37:51 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

Arnt Gulbrandsen wrote:

> IMAP certainly is more complex than POP, but I'm not sure that's
> relevant to this issue.
> 
> If all you want to do is fetch a single message based on a URL, using
> IMAP should be about as much work as POP. Things like IMAP's complex
> support for server-side search don't matter to this simple task.

It depends on what protocols and implementation languages one is
familiar with.  If one has been using something like C and is familiar
with other simple protocols (e.g. SMTP, NNTP, HTTP), POP is easy to
implement.  Unless one is familiar with and actually likes programming
languages like LISP, one is unlikely to find implementing an IMAP
client to be pleasant, let alone easy. Aside from that, the simplicity
of POP's state-based protocol makes for a straightforward
implementation, as opposed to IMAP's asynchronous protocol (where
commands are tagged and response tags have to be matched to the
corresponding command).

The big issue though is that there doesn't appear to be a good mechanism
for responding to a mailing list archived message (or a message in a
list digest, in most cases) while preserving References etc.  Lack of
support for IMAP and POP URI schemes may be a factor, but even widespread
support for those schemes would still require a means of getting the
message content into an application that can generate a response with
appropriate References, In-Reply-To, etc. fields.  I suppose that could
in theory be done by having a browser launch a POP or IMAP email
application to retrieve the message, however in practice there would be
problems with that approach (e.g. launching Mozilla when there is already
an instance running doesn't work).  And yes, the choice of POP or IMAP
doesn't really matter as far as addressing that big issue is concerned.

Cc: Keith Moore <[email-masked]>, [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Mon, 23 Feb 2004 08:15:38 -0500
To: Graham Klyne <[email-masked]>
Sender: [email-masked]

>> There are limits on what you can do with HTML and HTTP.  Being able 
>> to access the archived message from IMAP is potentially a lot better, 
>> but (as with many things) even IMAP needs some tweaking to make it 
>> work well for this.
>
> How many people actually use IMAP?  (I don't.)

why does it matter what they use to read their mail, as long as the 
right thing happens when they click on an IMAP URI?

lots of MUAs do support IMAP.  whether they support the ability to 
reference messages via IMAP URLs is a different question.

mail archives won't get better without specifications for how to make 
them better.

From: Arnt Gulbrandsen <[email-masked]>
To: Bruce Lilly <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked], Graham Klyne <[email-masked]>
Date: Mon, 23 Feb 2004 14:06:47 +0100
Sender: [email-masked]

Bruce Lilly writes:
> Graham Klyne wrote:
>> That said, I see no reason why the URI specified should not be an IMAP URI.
>
> Or POP. Browser support for the POP URI scheme is also an issue for 
> that, though it would likely be much easier to add support for POP 
> than for IMAP since the POP protocol is simpler.

IMAP certainly is more complex than POP, but I'm not sure that's 
relevant to this issue.

If all you want to do is fetch a single message based on a URL, using 
IMAP should be about as much work as POP. Things like IMAP's complex 
support for server-side search don't matter to this simple task.

Arnt

Date: Mon, 23 Feb 2004 07:47:58 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Graham Klyne <[email-masked]>
CC: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

Graham Klyne wrote:

> How many people actually use IMAP?  (I don't.)

That depends on the question:
IMAP the protocol? Many.
IMAP the URI scheme? Probably very few.

There are many software packages that support message storage and access
via IMAP, several of which are in use at quite large private installations
(ca.100000 users), as well as at ISPs. http://www.imap.org/about/whatisIMAP.html
is a good place to start to learn more about what is available.

As far as IMAP the URI scheme, I have tried a few browsers, and only one
(Konqueror) appears to support it.

> That said, I see no reason why the URI specified should not be an IMAP URI.

Or POP. Browser support for the POP URI scheme is also an issue for that,
though it would likely be much easier to add support for POP than for IMAP
since the POP protocol is simpler.

Date: Mon, 23 Feb 2004 10:38:26 +0000
To: Keith Moore <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

At 17:23 22/02/04 -0500, Keith Moore wrote:
>There are limits on what you can do with HTML and HTTP.  Being able to 
>access the archived message from IMAP is potentially a lot better, but (as 
>with many things) even IMAP needs some tweaking to make it work well for this.

How many people actually use IMAP?  (I don't.)

That said, I see no reason why the URI specified should not be an IMAP URI.

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Sun, 22 Feb 2004 23:41:58 +0000
From: "Adam M. Costello" <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Reply-To: IETF RFC-822 list <[email-masked]>
Sender: [email-masked]

Martin Duerst <[email-masked]> wrote:

> look for "Mail actions: [ respond to this message ]". The
> functionality is limited to what can be done with the 'mailto' URI,
> and only preserves some headers, and not the body.  If you have ideas
> on how to implement improvements, that would be great.

My first idea is to add a "raw message" link to a message/rfc822
resource.  The browser could invoke an internal or affiliated MUA, or
provide the ability to save it to a local mailbox (so that you could
then use your favorite MUA), or provide the ability to resend the
message to yourself (again, so that you could then use your favorite
MUA).  Those would all involve some special browser support for
message/rfc822.  Without such support, users of Unix-like systems could
save the message to a file and then run sendmail to send it to their own
address.

This suggests a second idea: the archive server accepts requests to mail
a copy of a message to the user.  There are various ways such requests
could be issued (typing your address into a form field, logging in and
having a saved preference, sending the request via mail [via a mailto:
link] and having the server reply), but they all raise the possibility
that the system could be abused and made to send arbitrary messages to
arbitrary recipients.

A third idea is for the HTML version to provide a link to
imap://server/msg-id, which would refer to a public (anonymous)
read-only virtual mailbox that contains one message.  Your browser might
be able to invoke an MUA and point it at that mailbox, or you might be
able to paste the URI into your favorite MUA.

AMC

Cc: Keith Moore <[email-masked]>, [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Sun, 22 Feb 2004 17:23:26 -0500
To: Martin Duerst <[email-masked]>
Sender: [email-masked]

>> It might be appropriate for the field name to start with List-*, at
>> least for those cases where the archive is associated with a list.  
>> That
>> way, it's easier to separate fields added by a list from fields 
>> supplied
>> by the sender.
>
> Good point. But it is not really restricted to lists. There are
> other potential applications, which are not related to mailing lists.
> So starting with List-* would be confusing.

Maybe it should only start with List- when added by a list.  But it 
seems like we're finding more and more cases where it's a bad idea to 
have a header field without very clear rules on who is allowed to set 
that field.  We've seen this with Reply-To and Sender and also with 
Carl Malamud's Solicitation field proposal.

>> I'd also recommend that the link point to an archived copy of the
>> message in original form, rather than, say, one that is translated to
>> HTML.  Translating to HTML causes a loss of information and 
>> potentially
>> a loss of functionality. You should be able to reply to an archived
>> message, refile it into a folder,  follow threads, etc., but those
>> things are harder to do if the message is no longer in its original
>> format.
>
> I have added some text about different formats, along the lines
> suggested by Graham. I think functions such as being able to reply,...
> would be great. Actually, our archives have such a function;
> look for "Mail actions: [ respond to this message ]". The functionality
> is limited to what can be done with the 'mailto' URI, and only
> preserves some headers, and not the body. If you have ideas on
> how to implement improvements, that would be great.

There are limits on what you can do with HTML and HTTP.  Being able to 
access the archived message from IMAP is potentially a lot better, but 
(as with many things) even IMAP needs some tweaking to make it work 
well for this.

Date: Sun, 22 Feb 2004 11:53:17 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: Revisiting RFC 2822 grammar
Sender: [email-masked]

Bruce Lilly wrote:

> I've updated the revised grammar file, which is available from
> http://users.erols.com/blilly/mparse/rfc2822grammar_simplified.txt
> and
> http://mysite.verizon.net/vze74u4p/mparse/rfc2822grammar_simplified.txt
> Changes include:
> 1. aforementioned issues hopefully fixed
> 2. corrections to date-time syntax as discussed here
> 3. separated obs- and non-obs rules for clarity
> 4. rearranged so that most productions are defined before use, the only
>    exceptions being "comment" and "ccontent" (and variations), which are
>    mutually dependent. This should make the revised grammar easier to
>    read.
> 5. incorporated alternatives for handling encoded-words and for ignoring
>    RFC 2047
> 6. yanked out the Subject field hacks
> 
> A side effect of dealing with the FWS-at-field-end issue is that much of
> the CFWS use has percolated up to higher-level constructs (further down in
> the file).  I think that makes it easier to understand, but maybe that's
> just because I've been working on the file for much of the past week.
> 
> As Pete has said that he doesn't want to incorporate the encoded-word
> syntax, if there is any discussion about that, please start a separate
> thread.  The encoded-word syntax is a bit complex; it is certainly possible
> that there are errors or that it can be simplified.  [F]or the purpose of
> discussing the RFC 2822 successor, please concentrate on the alternatives
> that do not address the encoded-word issues.
> 
> I would like to request help in verifying the following:
> A. that the changes that we've discussed have been correctly incorporated,
>    including the unstructured field grammar and the FWS-at-field-end issues
> B. that the revised grammar (ignoring encoded-words and related issues) is
>    compatible with RFC 2822 except in areas where 2822 has errors
> C. that no new errors have been introduced
> 
> There are notes in the file regarding some differences from RFCs 821, 822,
> and 2821, but let's defer discussion of those issues until after we've
> verified that we have a grammar that is correct and consistent with things
> as they stand. Then we can discuss what should be changed (that will cleanly
> separate the issue of rearranging the grammar from changing the grammar per
> se).

I've found a few typos and encountered another set of conflicts in the past
week, so I've once again updated the file, available at the URIs above. One
set of conflicts was related to "phrase" and resolving that moved many of the
[CFWS], [FWS] instances back into the lower-level productions.  I've added
more notes on issues that have been brought up on this mailing list.  Once
again I request that others look over the file to check for completeness
and lack of errors.

Date: Sun, 22 Feb 2004 10:08:14 -0500
To: Keith Moore <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked], [email-masked]
Sender: [email-masked]

Hello Keith,

At 18:21 04/02/20 -0500, Keith Moore wrote:
>I like the idea.

Thanks. Not exactly mine, but I'm glad to spread the word.

>It might be appropriate for the field name to start with List-*, at
>least for those cases where the archive is associated with a list.  That
>way, it's easier to separate fields added by a list from fields supplied
>by the sender.

Good point. But it is not really restricted to lists. There are
other potential applications, which are not related to mailing lists.
So starting with List-* would be confusing. I added a sentence
to that effect.

>I'd also recommend that the link point to an archived copy of the
>message in original form, rather than, say, one that is translated to
>HTML.  Translating to HTML causes a loss of information and potentially
>a loss of functionality. You should be able to reply to an archived
>message, refile it into a folder,  follow threads, etc., but those
>things are harder to do if the message is no longer in its original
>format.

I have added some text about different formats, along the lines
suggested by Graham. I think functions such as being able to reply,...
would be great. Actually, our archives have such a function;
look for "Mail actions: [ respond to this message ]". The functionality
is limited to what can be done with the 'mailto' URI, and only
preserves some headers, and not the body. If you have ideas on
how to implement improvements, that would be great.

>Thinking about this I realized that one reason I'd like such a field is
>so that I could, given a message, more easily find other messages in the
>thread.  After all, if I already have a copy of the message with the
>archived-at field, why would I want to download it?  I'm much more likely
>to want to look at either the messages that preceeded or followed that
>message.

Yes. The other way we use it at W3C is to point others to emails
without having to look them up in the archive. The typical example
goes as follows: As a chair of a WG, I have to prepare an agenda for
a teleconference. I go through the mails (in my email archive) that
I think are relevant. With the system we have, I just copy/paste
the URI to the agenda. Without that system, I'd either have to
go to the archives and search for the message, or try to describe
the message in terms of sender, date/time, subject,...

>As long as the field remains completely unstructured, I see no need to
>support comments.  I'd probably change my mind if the field were changed
>to be, say, a list of URIs.

I have added a sentence saying that multiple headers should be
used for multiple URIs. I also mentionned that for HTTP URIs,
there may be a "300 Multiple Choices" response. We actually
produce such a response if a mail goes to more than one mailing
list, and therefore appears more than once in the archive.

Regards,   Martin.

Date: Sun, 22 Feb 2004 10:06:52 -0500
To: [email-masked], Keith Moore <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

Hello Bruce,

Many thanks for your comments.

At 20:37 04/02/20 -0500, Bruce Lilly wrote:
>Keith Moore wrote:
>
> > As long as the field remains completely unstructured, I see no need to
> > support comments.  I'd probably change my mind if the field were changed
> > to be, say, a list of URIs.
>
>Comments are incompatible with URIs unless some quoting mechanism is used.
>That is because URIs may contain parentheses, as in
>http://users.erols.com/blilly/(foo)(bar)
>See RFC2396 for URI syntax details.  RFC 2369 quotes URIs in angle brackets,
>which are not themselves allowed in URIs.

I have disallowed comments.

>One issue to consider is long URIs.  If comments are disallowed, no quoting
>is required.  URIs cannot contain whitespace characters, so a simple way to
>handle a long URI is to allow a URI to be line-folded; it can be reconstructed
>by unfolding and eliding any whitespace.  To handle multiple URIs, a quoting
>mechanism can be used where line folding and whitespace (but not comments)
>are permitted within the quotes; the URI can be reconstructed as described
>above. Comments could be permitted outside the quotes.

I have mentioned the length limitations of RFC 2822. I think the 78
limitation may be a bit tough in some cases, but the 998 limitation
should not cause any problems in practice.

Regards,   Martin.

Date: Sat, 21 Feb 2004 17:42:39 +0000
To: Keith Moore <[email-masked]>, Martin Duerst <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: [email-masked]
Sender: [email-masked]

At 18:21 20/02/04 -0500, Keith Moore wrote:

>I like the idea.

As someone who'se made extensive use of the W3C list archive facility, I 
heartily concur.  It is a really valuable idea, and works very well in 
practice.

>It might be appropriate for the field name to start with List-*, at
>least for those cases where the archive is associated with a list.  That
>way, it's easier to separate fields added by a list from fields supplied
>by the sender.

No argument.

>I'd also recommend that the link point to an archived copy of the
>message in original form, rather than, say, one that is translated to
>HTML.  Translating to HTML causes a loss of information and potentially
>a loss of functionality. You should be able to reply to an archived
>message, refile it into a folder,  follow threads, etc., but those
>things are harder to do if the message is no longer in its original
>format.

Hmmm... the way the W3C system works, the URL works by redirection (or 
something) to the actual archived message.  I guess that multiple formats 
could be served at the same URI using existing Web protocols.  (e.g. use 
HTTP GET with Accept: message/rfc822 -- then it's up to the archive 
implementation to determine if it honours or rejects that.)

The existing W3C system provides links for following threads, etc., using a 
regular browser.

(Did anyone mention "running code"? ;-)

#g
--

>Thinking about this I realized that one reason I'd like such a field is
>so that I could, given a message, more easily find other messages in the
>thread.  After all, if I already have a copy of the message with the
>archived-at field, why would I want to download it?  I'm much more likely
>to want to look at either the messages that preceeded or followed that
>message.
>
>As long as the field remains completely unstructured, I see no need to
>support comments.  I'd probably change my mind if the field were changed
>to be, say, a list of URIs.

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

To: [email-masked]
From: Simon Josefsson <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Sat, 21 Feb 2004 09:16:26 +0100
Lines: 30
Cancel-Lock: sha1:0ZuGfMwaZLJGkWDNK4sQ0vhqe/4=
Sender: [email-masked]

"Adam M. Costello" <[email-masked]> writes:

> Keith Moore <[email-masked]> wrote:
>
>> I'd also recommend that the link point to an archived copy of the
>> message in original form, rather than, say, one that is translated
>> to HTML.  Translating to HTML causes a loss of information and
>> potentially a loss of functionality.  You should be able to reply to
>> an archived message, refile it into a folder, follow threads, etc.,
>> but those things are harder to do if the message is no longer in its
>> original format.
>>
>> Thinking about this I realized that one reason I'd like such a field
>> is so that I could, given a message, more easily find other messages
>> in the thread.
>
> Actually, if the link points to a message/rfc822 resource, won't it be
> harder to find the other messages in the thread than if the link points
> to a text/html page with hyperlinks?  On the other hand, it's easier to
> reply to a message if it's given in message/rfc822 form (or at least, it
> could be, with a little browser support).  Maybe it would be best for
> the Archived-At: field to point to a text/html page which in turn links
> to a message/rfc822 version of the same message.

Another idea is to serve both message/rfc822 and text/html at the same
URI.  Browsers generally indicate the format they want, and servers
can pick the right content accordingly.  As far as conformance is
concerned, message/rfc822 could be a MUST (to make it useful from
progrms), and text/html be a MAY (to improve rendering for humans).
On the other hand, perhaps this solution is too fragile.

Date: Fri, 20 Feb 2004 20:37:21 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Keith Moore <[email-masked]>
CC: Martin Duerst <[email-masked]>, [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

Keith Moore wrote:

> As long as the field remains completely unstructured, I see no need to 
> support comments.  I'd probably change my mind if the field were changed
> to be, say, a list of URIs.

Comments are incompatible with URIs unless some quoting mechanism is used.
That is because URIs may contain parentheses, as in
http://users.erols.com/blilly/(foo)(bar)
See RFC2396 for URI syntax details.  RFC 2369 quotes URIs in angle brackets,
which are not themselves allowed in URIs.

One issue to consider is long URIs.  If comments are disallowed, no quoting
is required.  URIs cannot contain whitespace characters, so a simple way to
handle a long URI is to allow a URI to be line-folded; it can be reconstructed
by unfolding and eliding any whitespace.  To handle multiple URIs, a quoting
mechanism can be used where line folding and whitespace (but not comments)
are permitted within the quotes; the URI can be reconstructed as described
above. Comments could be permitted outside the quotes.

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

Cc: Keith Moore <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Date: Fri, 20 Feb 2004 19:59:05 -0500
To: IETF RFC-822 list <[email-masked]>
Sender: [email-masked]

> Actually, if the link points to a message/rfc822 resource, won't it be
> harder to find the other messages in the thread than if the link points
> to a text/html page with hyperlinks?

obviously it depends on the html page with hyperlinks.  there are a lot 
of poorly designed archives out there.

the point is, if the fact that the resource is a message is lost, it 
really cannot be handled correctly.  you no longer have a message, you 
have a translation of a message into text.

it's too bad that we don't have a mailbox access protocol that handles 
threads well.  maybe we need yet another IMAP extension.

>  On the other hand, it's easier to
> reply to a message if it's given in message/rfc822 form (or at least, 
> it
> could be, with a little browser support).  Maybe it would be best for
> the Archived-At: field to point to a text/html page which in turn links
> to a message/rfc822 version of the same message.

no, I doubt it.  once something is html, it's really not reasonable to 
associate special semantics with it.

the client and server could do http content negotiation, but that only 
works for http.

>> After all, if I already have a copy of the message with the
>> archived-at field, why would I want to download it?
>
> You probably wouldn't, but regardless of what data type the 
> Archived-At:
> field points at, it will make citing that message easier.

agreed.  I just don't think I'd want to cite _a message_ as often as 
I'd want to cite _the context of a message_.

Date: Sat, 21 Feb 2004 00:05:10 +0000
From: "Adam M. Costello" <[email-masked]>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Reply-To: IETF RFC-822 list <[email-masked]>
Sender: [email-masked]

Keith Moore <[email-masked]> wrote:

> I'd also recommend that the link point to an archived copy of the
> message in original form, rather than, say, one that is translated
> to HTML.  Translating to HTML causes a loss of information and
> potentially a loss of functionality.  You should be able to reply to
> an archived message, refile it into a folder, follow threads, etc.,
> but those things are harder to do if the message is no longer in its
> original format.
>
> Thinking about this I realized that one reason I'd like such a field
> is so that I could, given a message, more easily find other messages
> in the thread.

Actually, if the link points to a message/rfc822 resource, won't it be
harder to find the other messages in the thread than if the link points
to a text/html page with hyperlinks?  On the other hand, it's easier to
reply to a message if it's given in message/rfc822 form (or at least, it
could be, with a little browser support).  Maybe it would be best for
the Archived-At: field to point to a text/html page which in turn links
to a message/rfc822 version of the same message.

> After all, if I already have a copy of the message with the
> archived-at field, why would I want to download it?

You probably wouldn't, but regardless of what data type the Archived-At:
field points at, it will make citing that message easier.  Currently,
you can either include a copy of the message (which is inefficient),
or figure out the URI yourself by searching the archive (which is
inconvenient for you) or provide the message-id and let the reader
search the archive (which is inconvenient for them).

AMC

Date: Fri, 20 Feb 2004 18:21:13 -0500
From: Keith Moore <[email-masked]>
To: Martin Duerst <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

I like the idea.

It might be appropriate for the field name to start with List-*, at
least for those cases where the archive is associated with a list.  That
way, it's easier to separate fields added by a list from fields supplied
by the sender.

I'd also recommend that the link point to an archived copy of the
message in original form, rather than, say, one that is translated to
HTML.  Translating to HTML causes a loss of information and potentially
a loss of functionality. You should be able to reply to an archived
message, refile it into a folder,  follow threads, etc., but those
things are harder to do if the message is no longer in its original
format.

Thinking about this I realized that one reason I'd like such a field is
so that I could, given a message, more easily find other messages in the
thread.  After all, if I already have a copy of the message with the 
archived-at field, why would I want to download it?  I'm much more likely
to want to look at either the messages that preceeded or followed that
message.

As long as the field remains completely unstructured, I see no need to 
support comments.  I'd probably change my mind if the field were changed
to be, say, a list of URIs.

Date: Fri, 20 Feb 2004 16:13:24 -0500
To: [email-masked]
From: Martin Duerst <[email-masked]>
Subject: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

Dear email specialists,

I have recently submitted a proposal for a new email header,
the Archived-At header. You can find it at
http://www.ietf.org/internet-drafts/draft-duerst-archived-at-00.txt
	
It defines a new email header, Archived-At:, to provide a direct
link to the archived form of an individual mail message. We use
this extremely successfully (currently in the form X-Archived-At)
at W3C.

I would highly appreciate any feedback and comments on this proposal.
Please make sure you copy me, as I'm not subscribed to this list.

Regards,    Martin.

Cc: <[email-masked]>, "Seth Honeycutt" <[email-masked]>
From: Nathaniel Borenstein <[email-masked]>
Subject: Re: Suggestion for "mailboxes" in ddress
Date: Sat, 14 Feb 2004 18:49:26 -0500
To: "Ken Hirsch" <[email-masked]>
Sender: [email-masked]

A brief note on the choice of separation character:  The + was first 
used, I believe, back in the mid-1980's as part of the Andrew Mail 
system.  We chose "+" after an exhaustive analysis of every ASCII 
character for potential conflicts.  It was one of a very small set of 
characters that were suitable, i.e. invariant across a huge range of 
email systems, Internet and non-Internet.

Since then, a few *new* systems have been built that have trouble with 
the "+", so I sympathize with those who now prefer "-".  I'm not sure I 
remember properly, but I think the main objection to using "-" was that 
quite a few mailing list names already used it, although there may have 
been others.  I remember that our second choice was "#".

The problem with ":" is that it can cause parsing ambiguities with 
valid RFC 822 addresses. (In particular it was one of the "specials" in 
the 822 grammar.)

I'm still partial to "+" but "-" is certainly fine for this too.  -- 
Nathaniel

On Feb 13, 2004, at 4:24 PM, Ken Hirsch wrote:

>
> From: "Seth Honeycutt" <[email-masked]>
>> This is a suggested possible idea for Internet Mail with RFC 822 /
> RFC2822,etc.  It is similar to "disposable e-mail addresses",
>
> This is already widely used.  See http://tmda.net/config-pre.html.  It 
> was
> discussed back in 1997 on this list whether it should be standardized. 
>  See
> http://www.imc.org/ietf-822/mail-archive/mail8.html for discussion of
> "subaddressing".
>
>
>

From: "Ken Hirsch" <[email-masked]>
To: "Seth Honeycutt" <[email-masked]>, <[email-masked]>
Subject: Re: Suggestion for "mailboxes" in ddress
Date: Fri, 13 Feb 2004 16:24:03 -0500
Sender: [email-masked]

From: "Seth Honeycutt" <[email-masked]>
> This is a suggested possible idea for Internet Mail with RFC 822 /
RFC2822,etc.  It is similar to "disposable e-mail addresses",

This is already widely used.  See http://tmda.net/config-pre.html.  It was
discussed back in 1997 on this list whether it should be standardized.  See
http://www.imc.org/ietf-822/mail-archive/mail8.html for discussion of
"subaddressing".

Date: Fri, 13 Feb 2004 15:31:08 -0600
To: Seth Honeycutt <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Suggestion for "mailboxes" in address
Cc: [email-masked]
Sender: [email-masked]

On 2/13/04 at 12:39 PM -0800, Seth Honeycutt wrote:

>The suggestion is to add an optional "mailbox" specification to the 
>username part of the address text string of an e-mail address.

This sort of thing is already in pretty widespread use, using the "+" 
instead of the ":" that you suggest.
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated

Date: Fri, 13 Feb 2004 12:45:27 -0800 (PST)
From: Seth Honeycutt <[email-masked]>
Subject: "Mailbox" Specification in Addresses
To: [email-masked]
Sender: [email-masked]

This is a suggested possible idea for Internet Mail
with RFC 822 / RFC
2822,
etc.&nbsp; It is similar to "disposable e-mail
addresses", but
implemented in a different way.&nbsp; It is suggested
as a standard so
that it can be used by anyone with a currently
existing e-mail account
and an e-mail client.&nbsp; (The phrase "mailbox" in
quotes used below
does not refer to the word [mailbox] that is used in
RFC 822 that
refers to a username / account mailbox on a mail
server.)&nbsp; The
suggestion is to add an optional “mailbox”
specification to the
username part of the address text
string of an e-mail address.&nbsp; This “mailbox”
specification could
be used as an effective way to virtually eliminate
SPAM (which I will
explain about in more detail) and also, as a side
benefit, give a nice
tool to organize a user’s incoming mail.&nbsp; An
example address could
look something like this:<br>
<br>
<a
href="mailto:user101:[email-masked]">user101:[email-masked]</a><br>
<br>
The [:] colon character could be used to separate the
username from the
“mailbox” name and then the rest of the address after
the [@] symbol
would be the same.&nbsp; Adding a “mailbox”
specification should be
compatible with SMTP and any e-mail sender’s client
software since it
will consider everything in front the [@] symbol to be
a
username.&nbsp; There would only need to be slight
changes with the
server software of a receiving mail server <span
style="font-style: italic;">that
wants to offer this service to their
own users</span>.&nbsp; A server that allows this
service would need to
analyze
the username in an incoming e-mail address differently
by only looking
at everything before the [:] colon when looking for
the username.&nbsp;
The server (especially one the uses POP3 when users
check their mail)
doesn’t need to concern itself with the “mailbox”
specification, but
can just let the user’s client software take care of
that.&nbsp; One
way you could use “mailbox” names for organizational
purposes is that,
on a business card, you could say [<a

href="mailto:user.name:[email-masked]">user.name:[email-masked]</a>]
so all mail from potential business clients who use
that address will
end up in a clients “mailbox” view in your e-mail
program.<br>
<br>
In regards to how this could be used to virtually
eliminate unsolicited
mail, you could use the “mailbox” specification in the
following
ways.&nbsp; When you give your e-mail address to
someone, like all your
family members, you could give them something like
this (which they
could still put in their e-mail program’s address book
just like any
e-mail address): [<a
href="mailto:user101:[email-masked]">user101:[email-masked]</a>].&nbsp;
Your e-mail client software (and possibly the server)
could be set to
not accept any e-mail that doesn’t include a “mailbox”
specification or
includes a “mailbox” name that you haven’t
created.&nbsp; You could
then have a special folder or “mailbox” view that
would show all of
your e-mail from family members.&nbsp; The “mailbox”
name could act
almost like a password (something that SPAMers
wouldn’t be able to
guess without sending out thousands of attempts) but
that you could
give out as part of your e-mail address.&nbsp; It
would be very
unlikely that a family member would give that
“mailbox” name away on a
SPAMers list; but even if they did you would only need
to change the
“mailbox” name with your family members to stop
it.&nbsp; With
“mailbox” names your whole e-mail account should never
be completely
corrupted if it ends up on a SPAMers list, since you
can just delete
the “mailbox” name the SPAMer’s are using to send you
e-mail and not
having to go looking for a new e-mail account.<br>
<br>
Probably the number-one way your e-mail address
ends-up on a SPAMer’s
list is by entering your e-mail address on a website
to sign-up for a
newsletter or if it is required to use their website’s
services.&nbsp;
You could create a special mailbox just for
newsletters that you
solicit and want to receive.&nbsp; Whenever you
sign-up for a
newsletter you could type in something like [<a

href="mailto:user101:[email-masked]">user101:[email-masked]</a>].&nbsp;
You could increase security with a “mailbox” by
creating virtual
“mailbox” names that includes both fixed and variable
parts.&nbsp; In
your e-mail client you could specify a “mailbox” name
for your
newsletters by including a wildcard character(s) [*]
or [?] in the
name.&nbsp; For instance you could name the mailbox
something like
these: [news*], [???news91], [news*letters], or
[newsletters-*].&nbsp;
The e-mail client will then accept any email to that
virtual “mailbox”
that has those fixed parts, but can have any number of
alpha-numeric
characters wherever the wildcard [*] character
appears, or just a
single varying alpha-numeric character where a [?]
wildcard character
appears.&nbsp; This would allow you to give a random
set of numbers or
letters where the wildcard is in the newsletter
“mailbox” name each
time you give your e-mail address to sign-up for a
newsletter on a
website.&nbsp; For instance, on one website you could
enter
[user101:news2549] and then for another newsletter
something like
[user101:news8f3c], etc.&nbsp; If just one of the
newsletters you
signed-up for leaked your e-mail address to a SPAM
list, you can just
block that one virtual “mailbox” name that you used
when signing-up for
that newsletter and still receive the other
newsletters just the
same.&nbsp; Another benefit is that you would be able
to tell who
leaked your e-mail address by comparing the e-mail
address “mailbox”
name that was used by the SPAMers that also matches
the “mailbox” name
used by one of the newsletter senders.&nbsp; Wildcards
shouldn't be
used too liberally because wildcards increase the
number of accepted
"mailbox" names significantly (depending on how you
use them), but that
decision would be up to each user on how secure they
want their
"mailbox" names to be.<br>
<br>
Thank you for your time in considering something like
this for
improving e-mail.&nbsp; I have had to abandon a few
e-mail accounts
already from getting too much SPAM there, and by
having “mailbox” names
you could keep your e-mail address forever and just
change certain
“mailbox” names that get corrupted (which by having
“mailbox” names in
the first place would make that unlikely to happen
anyway).&nbsp; I
currently live in fear of my new address getting into
the wrong hands
and having to get a new address all over again (even
though I am
careful and have a free address just for bulk mail and
entering on
websites for mailing lists).&nbsp; Something similar
to this should
work with the current
Internet e-mail protocols (especially on the sender’s
side and with
SMTP in general) except for adding a patch to the
server software for
receiving servers that want to offer this service to
their users, that
will
interpret everything before the [:] colon as the
username.&nbsp; The
POP3 protocol wouldn’t need to be changed, except for
possibly adding
new features like a user choosing to have the server
block all e-mail
that doesn’t designate a “mailbox” name or possibly
storing “mailbox”
names on the server to block unwanted e-mail at the
server level (which
would help reduce SPAM Internet traffic).&nbsp; IMAP
would probably
want to integrate the “mailbox”
names into its features since they already have things
like user-side
folders
and mailboxes that could work with the address-side
“mailbox”
names in the e-mail address itself.&nbsp; Thank you
again for you time
in considering some way to designate address-side
“mailboxes” in an
Internet mail address.<br>
<br>
(P.S. Another standard separator character (or just an
extra
alternative character) besides the [:] colon [which
is currently not allowed in usernames because it is
designated for
specifying groups which shouldn't cause a conflict
from what I figured]
could be used if there would be
problems in using it in some applications or
protocols.&nbsp; Since the
[:] is currently not allowed it should not appear in
any existing user
/ account names on a mail server.&nbsp; I tested
the &lt;a
href="mailto:user101:[email-masked]"&gt;&lt;/a&gt;
HTML
tags in both Internet Explorer 6.0 and Netscape 7.0
and they both
worked properly creating a new message in an e-mail
client, as I
expected they would.&nbsp; I also attempted to send an
e-mail to [<a

href="mailto:user101:[email-masked]">user101:[email-masked]</a>]
and it seemed to go through SMTP okay with just a
return e-mail from
the domain.com mail server saying the username was
unknown as I
expected.)<br>

__________________________________
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online.
http://taxes.yahoo.com/filing.html

Date: Fri, 13 Feb 2004 12:39:14 -0800 (PST)
From: Seth Honeycutt <[email-masked]>
Subject: Suggestion for "mailboxes" in ddress
To: [email-masked]
Sender: [email-masked]

--0-1202569472-1076704754=:95158
Content-Type: text/plain; charset=us-ascii

This is a suggested possible idea for Internet Mail with RFC 822 / RFC2822,etc.  It is similar to "disposable e-mail addresses", butimplemented in a different way.  It is suggested as a standard sothat it can be used by anyone with a currently existing e-mail accountand an e-mail client.  (The phrase "mailbox" in quotes used belowdoes not refer to the word [mailbox] that is used in RFC 822 thatrefers to a username / account mailbox on a mail server.)  Thesuggestion is to add an optional “mailbox” specification to theusername part of the address textstring of an e-mail address.  This “mailbox” specification couldbe used as an effective way to virtually eliminate SPAM (which I willexplain about in more detail) and also, as a side benefit, give a nicetool to organize a user’s incoming mail.  An example address couldlook something like this:

user101:[email-masked]

The [:] colon character could be used to separate the username from the“mailbox” name and then the rest of the address after the [@] symbolwould be the same.  Adding a “mailbox” specification should becompatible with SMTP and any e-mail sender’s client software since itwill consider everything in front the [@] symbol to be ausername.  There would only need to be slight changes with theserver software of a mail server thatwants to offer this service to theirown users.  A server that allows this service would need toanalyzethe username in an incoming e-mail address differently by only lookingat everything before the [:] colon when looking for the username. The server (especially one the uses POP3 when users check their mail)doesn’t need to concern itself with the “mailbox” specification, butcan just let the user’s client software take care of that.  Oneway you could use “mailbox” names for organizational purposes is that,on a business card, you could say [user.name:[email-masked]]so
 all mail from potential business clients who use that address willend up in a clients “mailbox” view in your e-mail program.

In regards to how this could be used to virtually eliminate unsolicitedmail, you could use the “mailbox” specification in the followingways.  When you give your e-mail address to someone, like all yourfamily members, you could give them something like this (which theycould still put in their e-mail program’s address book just like anye-mail address): [user101:[email-masked]]. Your e-mail client software (and possibly the server) could be set tonot accept any e-mail that doesn’t include a “mailbox” specification orincludes a “mailbox” name that you haven’t created.  You couldthen have a special folder or “mailbox” view that would show all ofyour e-mail from family members.  The “mailbox” name could actalmost like a password (something that SPAMers wouldn’t be able toguess without sending out thousands of attempts) but that you couldgive out as part of your e-mail address.  It would be veryunlikely that a family member would give that “mailbox” name away on aSPAMers list; but even
 if they did you would only need to change the“mailbox” name with your family members to stop it.  With“mailbox” names your whole e-mail account should never be completelycorrupted if it ends up on a SPAMers list, since you can just deletethe “mailbox” name the SPAMer’s are using to send you e-mail and nothaving to go looking for a new e-mail account.

Probably the number-one way your e-mail address ends-up on a SPAMer’slist is by entering your e-mail address on a website to sign-up for anewsletter or if it is required to use their website’s services. You could create a special mailbox just for newsletters that yousolicit and want to receive.  Whenever you sign-up for anewsletter you could type in something like [user101:[email-masked]]. You could increase security with a “mailbox” by creating virtual“mailbox” names that includes both fixed and variable parts.  Inyour e-mail client you could specify a “mailbox” name for yournewsletters by including a wildcard character(s) [*] or [?] in thename.  For instance you could name the mailbox something likethese: [news*], [???news91], [news*letters], or [newsletters-*]. The e-mail client will then accept any email to that virtual “mailbox”that has those fixed parts, but can have any number of alpha-numericcharacters wherever the wildcard [*] character appears, or just asingle
 varying alpha-numeric character where a [?] wildcard characterappears.  This would allow you to give a random set of numbers orletters where the wildcard is in the newsletter “mailbox” name eachtime you give your e-mail address to sign-up for a newsletter on awebsite.  For instance, on one website you could enter[user101:news2549] and then for another newsletter something like[user101:news8f3c], etc.  If just one of the newsletters yousigned-up for leaked your e-mail address to a SPAM list, you can justblock that one virtual “mailbox” name that you used when signing-up forthat newsletter and still receive the other newsletters just thesame.  Another benefit is that you would be able to tell wholeaked your e-mail address by comparing the e-mail address “mailbox”name that was used by the SPAMers that also matches the “mailbox” nameused by one of the newsletter senders.  Wildcards shouldn't beused too liberally because wildcards increase the number of accepted"mailbox" names
 significantly (depending on how you use them), but thatdecision would be up to each user on how secure they want their"mailbox" names to be.

Thank you for your time in considering something like this forimproving e-mail.  I have had to abandon a few e-mail accountsalready from getting too much SPAM there, and by having “mailbox” namesyou could keep your e-mail address forever and just change certain“mailbox” names that get corrupted (which by having “mailbox” names inthe first place would make that unlikely to happen anyway).  Icurrently live in fear of my new address getting into the wrong handsand having to get a new address all over again (even though I amcareful and have a free address just for bulk mail and entering onwebsites for mailing lists).  Something similar to this shouldwork with the currentInternet e-mail protocols (especially on the sender’s side and withSMTP in general) except for adding a patch to the server software forreceiving servers that want to offer this service to their users, thatwillinterpret everything before the [:] colon as the username.  ThePOP3 protocol wouldn’t need to be changed, except
 for possibly addingnew features like a user choosing to have the server block all e-mailthat doesn’t designate a “mailbox” name or possibly storing “mailbox”names on the server to block unwanted e-mail at the server level (whichwould help reduce SPAM Internet traffic).  IMAP would probablywant to integrate the “mailbox”names into its features since they already have things like user-sidefoldersand mailboxes that could work with the address-side “mailbox”names in the e-mail address itself.  Thank you again for you timein considering some way to designate address-side “mailboxes” in anInternet mail address.

(P.S. Another standard separator character (or just an extraalternative character) besides the [:] colon [whichis currently not allowed in usernames because it is designated forspecifying groups which shouldn't cause a conflict from what I figured]could be used if there would beproblems in using it in some applications or protocols.  Since the[:] is currently not allowed it should not appear in any existing user/ account names on a mail server.  I testedthe <a href=”mailto:user101:[email-masked]></a> HTMLtags in both Internet Explorer 6.0 and Netscape 7.0 and they bothworked properly creating a new message in an e-mail client, as Iexpected they would.  I also attempted to send an e-mail to [user101:[email-masked]]and it seemed to go through SMTP okay with just a return e-mail fromthe domain.com mail server saying the username was unknown as Iexpected.)

---------------------------------
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online
--0-1202569472-1076704754=:95158
Content-Type: text/html; charset=us-ascii

This is a suggested possible idea for Internet Mail with RFC 822 / RFC
2822,
etc.&nbsp; It is similar to "disposable e-mail addresses", but
implemented in a different way.&nbsp; It is suggested as a standard so
that it can be used by anyone with a currently existing e-mail account
and an e-mail client.&nbsp; (The phrase "mailbox" in quotes used below
does not refer to the word [mailbox] that is used in RFC 822 that
refers to a username / account mailbox on a mail server.)&nbsp; The
suggestion is to add an optional “mailbox” specification to the
username part of the address text
string of an e-mail address.&nbsp; This “mailbox” specification could
be used as an effective way to virtually eliminate SPAM (which I will
explain about in more detail) and also, as a side benefit, give a nice
tool to organize a user’s incoming mail.&nbsp; An example address could
look something like this:<br>
<br>
<a href="mailto:user101:[email-masked]">user101:[email-masked]</a><br>
<br>
The [:] colon character could be used to separate the username from the
“mailbox” name and then the rest of the address after the [@] symbol
would be the same.&nbsp; Adding a “mailbox” specification should be
compatible with SMTP and any e-mail sender’s client software since it
will consider everything in front the [@] symbol to be a
username.&nbsp; There would only need to be slight changes with the
server software of a mail server <span style="font-style: italic;">that
wants to offer this service to their
own users</span>.&nbsp; A server that allows this service would need to
analyze
the username in an incoming e-mail address differently by only looking
at everything before the [:] colon when looking for the username.&nbsp;
The server (especially one the uses POP3 when users check their mail)
doesn’t need to concern itself with the “mailbox” specification, but
can just let the user’s client software take care of that.&nbsp; One
way you could use “mailbox” names for organizational purposes is that,
on a business card, you could say [<a
 href="mailto:user.name:[email-masked]">user.name:[email-masked]</a>]
so all mail from potential business clients who use that address will
end up in a clients “mailbox” view in your e-mail program.<br>
<br>
In regards to how this could be used to virtually eliminate unsolicited
mail, you could use the “mailbox” specification in the following
ways.&nbsp; When you give your e-mail address to someone, like all your
family members, you could give them something like this (which they
could still put in their e-mail program’s address book just like any
e-mail address): [<a href="mailto:user101:[email-masked]">user101:[email-masked]</a>].&nbsp;
Your e-mail client software (and possibly the server) could be set to
not accept any e-mail that doesn’t include a “mailbox” specification or
includes a “mailbox” name that you haven’t created.&nbsp; You could
then have a special folder or “mailbox” view that would show all of
your e-mail from family members.&nbsp; The “mailbox” name could act
almost like a password (something that SPAMers wouldn’t be able to
guess without sending out thousands of attempts) but that you could
give out as part of your e-mail address.&nbsp; It would be very
unlikely that a family member would give that “mailbox” name away on a
SPAMers list; but even if they did you would only need to change the
“mailbox” name with your family members to stop it.&nbsp; With
“mailbox” names your whole e-mail account should never be completely
corrupted if it ends up on a SPAMers list, since you can just delete
the “mailbox” name the SPAMer’s are using to send you e-mail and not
having to go looking for a new e-mail account.<br>
<br>
Probably the number-one way your e-mail address ends-up on a SPAMer’s
list is by entering your e-mail address on a website to sign-up for a
newsletter or if it is required to use their website’s services.&nbsp;
You could create a special mailbox just for newsletters that you
solicit and want to receive.&nbsp; Whenever you sign-up for a
newsletter you could type in something like [<a
 href="mailto:user101:[email-masked]">user101:[email-masked]</a>].&nbsp;
You could increase security with a “mailbox” by creating virtual
“mailbox” names that includes both fixed and variable parts.&nbsp; In
your e-mail client you could specify a “mailbox” name for your
newsletters by including a wildcard character(s) [*] or [?] in the
name.&nbsp; For instance you could name the mailbox something like
these: [news*], [???news91], [news*letters], or [newsletters-*].&nbsp;
The e-mail client will then accept any email to that virtual “mailbox”
that has those fixed parts, but can have any number of alpha-numeric
characters wherever the wildcard [*] character appears, or just a
single varying alpha-numeric character where a [?] wildcard character
appears.&nbsp; This would allow you to give a random set of numbers or
letters where the wildcard is in the newsletter “mailbox” name each
time you give your e-mail address to sign-up for a newsletter on a
website.&nbsp; For instance, on one website you could enter
[user101:news2549] and then for another newsletter something like
[user101:news8f3c], etc.&nbsp; If just one of the newsletters you
signed-up for leaked your e-mail address to a SPAM list, you can just
block that one virtual “mailbox” name that you used when signing-up for
that newsletter and still receive the other newsletters just the
same.&nbsp; Another benefit is that you would be able to tell who
leaked your e-mail address by comparing the e-mail address “mailbox”
name that was used by the SPAMers that also matches the “mailbox” name
used by one of the newsletter senders.&nbsp; Wildcards shouldn't be
used too liberally because wildcards increase the number of accepted
"mailbox" names significantly (depending on how you use them), but that
decision would be up to each user on how secure they want their
"mailbox" names to be.<br>
<br>
Thank you for your time in considering something like this for
improving e-mail.&nbsp; I have had to abandon a few e-mail accounts
already from getting too much SPAM there, and by having “mailbox” names
you could keep your e-mail address forever and just change certain
“mailbox” names that get corrupted (which by having “mailbox” names in
the first place would make that unlikely to happen anyway).&nbsp; I
currently live in fear of my new address getting into the wrong hands
and having to get a new address all over again (even though I am
careful and have a free address just for bulk mail and entering on
websites for mailing lists).&nbsp; Something similar to this should
work with the current
Internet e-mail protocols (especially on the sender’s side and with
SMTP in general) except for adding a patch to the server software for
receiving servers that want to offer this service to their users, that
will
interpret everything before the [:] colon as the username.&nbsp; The
POP3 protocol wouldn’t need to be changed, except for possibly adding
new features like a user choosing to have the server block all e-mail
that doesn’t designate a “mailbox” name or possibly storing “mailbox”
names on the server to block unwanted e-mail at the server level (which
would help reduce SPAM Internet traffic).&nbsp; IMAP would probably
want to integrate the “mailbox”
names into its features since they already have things like user-side
folders
and mailboxes that could work with the address-side “mailbox”
names in the e-mail address itself.&nbsp; Thank you again for you time
in considering some way to designate address-side “mailboxes” in an
Internet mail address.<br>
<br>
(P.S. Another standard separator character (or just an extra
alternative character) besides the [:] colon [which
is currently not allowed in usernames because it is designated for
specifying groups which shouldn't cause a conflict from what I figured]
could be used if there would be
problems in using it in some applications or protocols.&nbsp; Since the
[:] is currently not allowed it should not appear in any existing user
/ account names on a mail server.&nbsp; I tested
the &lt;a href=”mailto:user101:[email-masked]&gt;&lt;/a&gt; HTML
tags in both Internet Explorer 6.0 and Netscape 7.0 and they both
worked properly creating a new message in an e-mail client, as I
expected they would.&nbsp; I also attempted to send an e-mail to [<a
 href="mailto:user101:[email-masked]">user101:[email-masked]</a>]
and it seemed to go through SMTP okay with just a return e-mail from
the domain.com mail server saying the username was unknown as I
expected.)<br><p><hr SIZE=1>
Do you Yahoo!?<br>
Yahoo! Finance: <a href="http://us.rd.yahoo.com/evt=22055/*http://taxes.yahoo.com/filing.html">Get your refund fast by filing online</a>
--0-1202569472-1076704754=:95158--

Date: Thu, 12 Feb 2004 20:02:24 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
Subject: Re: VMN: Virtual Mail Networks (ESMTP)
Sender: [email-masked]

On Thu, Feb 12, 2004 at 02:23:49PM +0000, Charles Lindsey wrote:
> 
> In <[email-masked]> Keld Jørn Simonsen <[email-masked]> writes:
> 
> >1. Always check for virus/spam before checking for valid receipient, or
> >whether the mailbox is full or some such.
> 
> >2. Generate a specific error message, maybe we should introduce a
> >standard error code for this, like 551 - mail rejected as virus or spam
> 
> >3. If the mail is virus or spam, then do not send it back to the
> >sender - as this is most likely a forged address anyway. Discard it
> >instead.  But if you must, then use the standard error mesaage as
> >described above. 
> 
> I think the first thing you need is some standardized header that says
> "this message is a bounce", preferebly giving some brief reason including
> an error code.
> 
> It is much easier to filter out messages containing such a header that to
> search the body for clues that might indicate its filterability.

Yes, I agree that some reliable message in the subject would be a great
step forward.

Best regards
Keld

To: [email-masked]
Xref: clerew local.mime:3058
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: VMN: Virtual Mail Networks (ESMTP)
Date: Thu, 12 Feb 2004 14:23:49 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> Keld Jørn Simonsen <[email-masked]> writes:

>1. Always check for virus/spam before checking for valid receipient, or
>whether the mailbox is full or some such.

>2. Generate a specific error message, maybe we should introduce a
>standard error code for this, like 551 - mail rejected as virus or spam

>3. If the mail is virus or spam, then do not send it back to the
>sender - as this is most likely a forged address anyway. Discard it
>instead.  But if you must, then use the standard error mesaage as
>described above. 

I think the first thing you need is some standardized header that says
"this message is a bounce", preferebly giving some brief reason including
an error code.

It is much easier to filter out messages containing such a header that to
search the body for clues that might indicate its filterability.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 10 Feb 2004 13:20:24 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: Andrzej Filip <[email-masked]>
Cc: [email-masked]
Subject: Re: VMN: Virtual Mail Networks (ESMTP)
Sender: [email-masked]

On Sat, Feb 07, 2004 at 11:15:50PM +0100, Andrzej Filip wrote:
> [reposted from [email-masked]]
> 
> The above mentioned VMN could reduce "spam as we know it" by at least order of
> magnitude by excluding most of open relays/proxies and forcing easy
> identification of real sender.

Well, I think the current situation is quite manageable, with the RBS,
the virus checking and spam-checking software. The only thing I would
like to get rid of, is all the error reporting mails, saying that I have
sent out virus infected mail, or that I have sent mail to somebody whose
address is unknown or their mailbix is full.

I have by the way collected strings for postfix filters to discard
messages reporting bogus virus, available at
http://www.dkuug.dk/keld/virus/

Actually there is some good chances of getting rid of the bogus error
reports. We are dealing with the good guys here, so they may be able to
take proper advice.

I would advice that we recommend some best practice procedures,
hopefully to be implemented in the MTAs spftware productss of the world. 

1. Always check for virus/spam before checking for valid receipient, or
whether the mailbox is full or some such.

2. Generate a specific error message, maybe we should introduce a
standard error code for this, like 551 - mail rejected as virus or spam

3. If the mail is virus or spam, then do not send it back to the
sender - as this is most likely a forged address anyway. Discard it
instead.  But if you must, then use the standard error mesaage as
described above. 

I think with this scheme, we would have avoided alomost all of the
virus/spam and also annoying error traffic .

No need for new protocols, closed networks etc. Maybe a need for some
RBL listing virus/spam infected machines, I don't knos.

Best regards
Keld

To: [email-masked]
Xref: clerew local.mime:3056
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Date: Tue, 10 Feb 2004 11:54:35 GMT
Lines: 28
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:
>> 
>>     "Comments" ":" SP CRLF

>Note that unstructured uses utext, and I have defined utext as 1*text.
>text includes SP (%d32).  So your example is matched by the proposed
>grammar. [I'm not 100% sure that 1*text is the right thing to do, but
>it definitely works for this example]

In that case you have reintroduced the two-FWS-in-succession ambiguity,
because

    "foo" SP SP "bar"

can now be parsed in two ways (well, three ways actually).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 9 Feb 2004 12:00:28 -0500
From: Keith Moore <[email-masked]>
To: Nick Ing-Simmons <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Sender authentication?
Sender: [email-masked]

> I hope this isn't re-hashing old ground but an idea has occured to me
> that would, I think, help with the current deluge of mail from worms that 
> forge From: addresses.
> 
> The idea is to "sign" the From: and Message-ID: pair using 
> a public key scheme, and add the signature as a new header.

You need to sign the message content.  Otherwise a worm could take a From and
Message-id from a completely different message and prepend it to its own
message.

You don't want to use From as the sender identity, because there are perfectly
legitimate cases where From is different from the actual sender.  You don't
want to use Return-Path either, because that's where bounces go and there are
sometimes good reasons to send bounces to a different place than the address
associated with the sender's identity.  The Sender field was originally
intended for this purpose but that's been corrupted due to wide misuse by
mailing lists. So you need a separate identity for "the person who signed the
message".

Of course if you just want to sign messages then S/MIME will do the job. What
you seem to want is to make it difficult for worms to forge messages. That's
very tricky because the sender's machine has already been compromised. So for
instance the worm could set itself up to record the sender's password (or
whatever is used to encrypt the signing key) whenever it was typed in, and the
worm could sign messages on behalf of that person.  It could even transmit
that person's public key to other machines so that the other machines could
sign messages from that person.

To really fix the spam problem, and the virus/worm problem, Windows needs to
be eradicated and replaced with something that actually has some degree of
security.  That's a bit over-simplistic, but it's close.

Keith

Date: Mon, 09 Feb 2004 11:58:19 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>>comments         =   "Comments" ":" unstructured CRLF
>>obs-comments     =   "Comments" *WSP ":" [FWS] obs-unstructured CRLF
>>etc.

> No, that does not work, because you have just forbidden a header made up
> exactly as follows:
> 
>     "Comments" ":" SP CRLF

Note that unstructured uses utext, and I have defined utext as 1*text.
text includes SP (%d32).  So your example is matched by the proposed
grammar. [I'm not 100% sure that 1*text is the right thing to do, but
it definitely works for this example]

Date: Mon, 09 Feb 2004 11:52:20 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Andrzej Filip <[email-masked]>
CC: [email-masked]
Subject: Re: Sender authentication?
Sender: [email-masked]

Andrzej Filip wrote:

> Only POP offers an option to download headers without body.

I believe IMAP has the same capability.

Date: Mon, 09 Feb 2004 06:01:55 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked], [email-masked]
Organization: Bruce Lilly
To: Nick Ing-Simmons <[email-masked]>
CC: [email-masked]
Subject: Re: Sender authentication?
Sender: [email-masked]

Nick Ing-Simmons wrote:
[...]
> The idea is to "sign" the From: and Message-ID: pair using 
> a public key scheme, and add the signature as a new header.
[...]
> The goal is to allow recipient to have some confidence that mail is 
> really from the sender without collecting the whole mail, but just 
> by examining the headers. (Signed mail is all very well 
> but sucking 5M of spam body to allow me to check it isn't helping 
> with my bandwidth problem.)
[...]
> The idea is that the unique Message-Id is a "challenge" for the sender's
> key pair. 
> 
> As many tools also discard mails with duplicate ids then forger 
> cannot just re-use signature headers from a previous mail.
> 
> Issues I can see are where public key comes from (key server or embedded in the 
> mail) and possible size of key and signature data.
> 
> Is this worth discussing further?

Probably. There has been some limited discussion about something like
that here; some issues are mentioned in the message archived
at http://www.imc.org/ietf-822/mail-archive/msg03987.html
and a brief outline of how keys might be handled via an SMTP extension is
in http://www.imc.org/ietf-822/mail-archive/msg03941.html

The hashing method needs to be robust in the face of header field rewriting.

Although message-ids are supposed to be unique, not all systems discard messages
with duplicates, permitting a replay attack.  Moreover, if an attacker is able
to replay the fields and signature quickly, such that both a genuine message and
a forgery arrive in the same batch, which one gets discarded by a system that
does discard duplicates may depend on order of retrieval.

The From field is probably not the right source for sender information. It may
contain multiple addresses (which key would you use in that case?).  A better
choice would be the sender envelope address, which appears in the Return-Path
field on final delivery.  However, the sender envelope address may be an empty
path (e.g. for a delivery status notification), so that's not universally
applicable either.

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

Date: Mon, 09 Feb 2004 15:09:02 +0100
From: Andrzej Filip <[email-masked]>
Organization: It is for me to know and for you to find out
To: [email-masked]
Subject: Re: Sender authentication?
Sender: [email-masked]

Nick Ing-Simmons wrote:
> Andrzej Filip <[email-masked]> writes:
>>[...]
>>* Recipient signing is tricky vide signing BCC: recipients
>  
> So BCC recipients don't get the protection - I tend to be suspicious 
> of mail I receive via BCC anyway.

What would you say about mailing lists ?

> [...]

-- 
Andrzej [en:Andrew] Adam Filip http://anfi.freeshell.org backup: [email-masked]

To: [email-masked]
Xref: clerew local.mime:3049
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Date: Mon, 9 Feb 2004 11:37:35 GMT
Lines: 54
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Bruce Lilly wrote:

>> obs-text only appears in text (where obs-char makes more sense) and
>> obs-utext (which is defined as obs-text). obs-utext appears only in utext,
>> and utext appears only in unstructured.
>> 
>> I'll take another look at the unstructured and related productions.

>OK, here's another attempt.  Working backwards from unstructured fields:

>comments         =   "Comments" ":" unstructured CRLF
>obs-comments     =   "Comments" *WSP ":" [FWS] obs-unstructured CRLF
>etc.

>N.B. no [FWS] after the colon in the non-obs fields (an unstructured
>field cannot be allowed to end with FWS (which it would do if the
>"unstructured" production is empty) using non-obs rules, since that
>would leave CRLF 1*WS CRLF, i.e. a whitespace only "continuation"
>line at the end of the field, which is prohibited by sect. 3.2.3.).
>Per contra, that must be permitted in the obs- unstructured fields.
>That is why separate unstructured and obs-unstructured productions
>are required.

No, that does not work, because you have just forbidden a header made up
exactly as follows:

    "Comments" ":" SP CRLF

That is currently permitted by RFC 2822 (even for generating), and is
indeed what I would expect most MUAs to generate (assuming they did not
omit the header altogether).

Preventing folded headers with only WSP in a line through the syntax would
be so horrendously complex that it should not be attempted. Which is why
it is currently done by verbiage in 3.2.3. So I think the best you can do
is to allow [FWS] in that header, and leave the wording in 3.2.3 to cover
the situation.

Having said that, the wording in 3.2.3 could be tightened up. Currently,
it applies to CFWS only (and not to FWS, so you could seemingly have a
Date header folded over many empty lines).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:3048
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Sender authentication?
Date: Mon, 9 Feb 2004 11:04:56 GMT
Lines: 48
Sender: [email-masked]

In <[email-masked]> Nick Ing-Simmons <[email-masked]> writes:

>I hope this isn't re-hashing old ground but an idea has occured to me
>that would, I think, help with the current deluge of mail from worms that 
>forge From: addresses.

>The idea is to "sign" the From: and Message-ID: pair using 
>a public key scheme, and add the signature as a new header.

Yes, it can be done, and indeed it has been done and is in regular use for
signing Control Messages on Usenet, and less regularly so for signing
moderated articles.

The bad news is that the protocols for the two purposes are different and
incompatible and suffer from various limitations which make them
unsuitable for general use.

1: PGPVerify is used for signing control messages. The signature header
includes a list of the other headers included within the signature. Its
canonicalization is minimal (e.g. doesn't understand folding).

2: PGPMoose is used for signing moderated articles. The list of headers
included within the signature is fixed and non-configurable. Its
canonicalization is a little better.

Both of the above suffer from the disadvantage that they also include the
whole of the article body within the signature. That should have been a
configurable option IMHO (so, for example, changes of CTE en route would
break it, though that is also true of PGP-Mime).

3. www.landfield.com/usefor/drafts/draft-lindsey-usefor-signed-01.txt.
This is unfinished work intended to overcome the problems identified
above. Its most notable feature is an exceedingly comprehensive
canonicalization scheme. Probably too comprehensive in some ways and
insufficiently comprehensive in others. I would probably do it differently
if doing it again. Not implemented, except for on a demonstration basis in
Perl.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 09 Feb 2004 08:23:47 +0100
From: Andrzej Filip <[email-masked]>
Organization: It is for me to know and for you to find out
CC: [email-masked]
Subject: Re: Sender authentication?
Sender: [email-masked]

Nick Ing-Simmons wrote:
> 
> I hope this isn't re-hashing old ground but an idea has occured to me
> that would, I think, help with the current deluge of mail from worms that 
> forge From: addresses.
> 
> The idea is to "sign" the From: and Message-ID: pair using 
> a public key scheme, and add the signature as a new header.

How do you want to check signature validity for John Doe ?
Do you want to use it only for "well known friends" ?

> The goal is to allow recipient to have some confidence that mail is 
> really from the sender without collecting the whole mail, but just 
> by examining the headers. (Signed mail is all very well 
> but sucking 5M of spam body to allow me to check it isn't helping 
> with my bandwidth problem.)

Only POP offers an option to download headers without body. Well behaving SMTP 
servers can not accept headers but refuse to accept body.
Virus on A person host can "reuse" signature of B->A message to fake B->C message.

> (I currently discard any mail which has its Message-ID inserted 
>  by my ISP - this is wonderfully successful for an ad. hoc. hack
>  e.g. it managed to discard 80% of the 10000 emails I received 
>  in one day of recent MyDoom peak. Snag is that there are 
>  presumably false discards - including mails from my ISP!)

Viruses get smarter every outbreak.

> The idea is that the unique Message-Id is a "challenge" for the sender's
> key pair. 
> 
> As many tools also discard mails with duplicate ids then forger 
> cannot just re-use signature headers from a previous mail.
> 
> Issues I can see are where public key comes from (key server or embedded in the 
> mail) and possible size of key and signature data.
> 
> Is this worth discussing further?

The idea is good BUT IMHO signing Sender, Recipient, Message-ID, Date makes 
more sense.
* "Date:" signing would prohibit "reuse" of signatures.
* Recipient signing is tricky vide signing BCC: recipients

-- 
Andrzej [en:Andrew] Adam Filip http://anfi.freeshell.org backup: [email-masked]

Cc: [email-masked]
Subject: Sender authentication?
To: [email-masked]
Date: Sun, 08 Feb 2004 23:10:24 +0000
From: Nick Ing-Simmons <[email-masked]>
Reply-To: Nick Ing-Simmons <[email-masked]>
Sender: [email-masked]

I hope this isn't re-hashing old ground but an idea has occured to me
that would, I think, help with the current deluge of mail from worms that 
forge From: addresses.

The idea is to "sign" the From: and Message-ID: pair using 
a public key scheme, and add the signature as a new header.

The goal is to allow recipient to have some confidence that mail is 
really from the sender without collecting the whole mail, but just 
by examining the headers. (Signed mail is all very well 
but sucking 5M of spam body to allow me to check it isn't helping 
with my bandwidth problem.)

(I currently discard any mail which has its Message-ID inserted 
 by my ISP - this is wonderfully successful for an ad. hoc. hack
 e.g. it managed to discard 80% of the 10000 emails I received 
 in one day of recent MyDoom peak. Snag is that there are 
 presumably false discards - including mails from my ISP!)

The idea is that the unique Message-Id is a "challenge" for the sender's
key pair. 

As many tools also discard mails with duplicate ids then forger 
cannot just re-use signature headers from a previous mail.

Issues I can see are where public key comes from (key server or embedded in the 
mail) and possible size of key and signature data.

Is this worth discussing further?

--
Nick Ing-Simmons

Date: Sun, 08 Feb 2004 09:54:05 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Pete Resnick <[email-masked]>
CC: [email-masked]
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Sender: [email-masked]

Pete Resnick wrote:
> On 2/7/04 at 9:11 AM -0500, Bruce Lilly wrote:

>> I think it's clearer to keep the obs- constructs out of the non-obs
>> productions. Indeed, it becomes clearer still if that is pursued to
>> its logical extreme:
>> [...]
>> i.e. the obs- constructs appear only in the obs-productions, rather
>> than as an alternative in non-obs productions.
> 
> 
> Though it may be clearer for someone actually coding a parsing engine,
> I'm not at all sure that it's clearer for other folks. I like being able
> to read through the "generation" grammar and find forward references to
> a difference in the "parser" grammar. I also kind of like the idea that
> you can't read the "generation" grammar and miss that there's some work
> to do in the parsing end. Finally, in terms of the prose, I think it
> will make it more confusing: In the current layout, any time you see
> "obs-" in section 4, you're going to see an explanation of that "obs-"
> construct. If we were to switch it, the explanations would be scattered
> about. I for one would rather leave this as-is.

One item that's clarified is the fact that the parse (obs-) grammar is
a superset of the generate grammar.  As it stands now, it looks like it's
the other way around because the generate grammar also subsumes the obs-
productions.

Forward references in the grammar are confusing; on encountering one of
them, the reader is sidetracked into hunting down that forward reference,
then backtracking through the prose to find a description of the obs-
construct before returning to the point of the diversion.  One could of
course have forward references in the prose without having them appear
in the grammar, to preserve any value in such forward references.  The
layout and order of the explanations need not be changed.

Anyway, I'd be interested to hear what others think.

> I don't see "misunderstandings" of the "obs-" really
> being a problem.

I believe at least two people have pointed out the confusion about the
unquoted '.' in obs-phrase, which isn't really "obsolete".  Changing "obs-"
to "parse-" would resolve that issue.

Date: Sun, 08 Feb 2004 12:21:03 +0100
From: Andrzej Filip <[email-masked]>
Organization: It is for me to know and for you to find out
To: [email-masked]
Subject: Re: VMN: Virtual Mail Networks (ESMTP)
Sender: [email-masked]

Bruce Lilly wrote:
> Andrzej Filip wrote:
>>Sample VMN specification:
>>* mandatory strong SPF specification for any domain participating
>>(e.g. up to 10 hosts which can send messages with sender in specific
>>domain)
>>* mandatory requirement of domain reachability
>>(removal from VMN domains which do not accept mail for more than 4 days)
>>* mandatory check for open relay/proxy of any host participating in the VMN

What I posted was not "the only reasonable VMN". It was merely an example of 
one of many possible VMNs.

> SPF and open relay checks can be troublesome; it is occasionally necessary
> for some senders to specify an alternate sender envelope address for
> legitimate mail (e.g. when traveling and a different ISP is to be used
> for bounces and replies).   SPF etc. might be useful for ISP-ISP transfers,
> but they would cause trouble at the sender's initial SMTP session.

So "the sample VMN" may specify separate policies for initial submission e.g. 
SMTP AUTH to MSA port (587).

> Believe it or not, some SMTP servers do go down; that happened recently
> for a server handling an IETF mailing list -- it took a few days for
> anybody to notice, then took a few more days to leave a message for the
> machine's administrator (obviously email couldn't be used), and it was a
> few more days before the server was back on-line.
> 
> So far, I see problems for legitimate mail traffic, and with not-unusual
> outages.

So you say that "remove from VMN" timeout proposed by me was too short.
But it would be nice too keep list of active domains and make initial MTA 
avoid kludging its queue with messages to domain with long period problems.

>>The above mentioned VMN could reduce "spam as we know it" by at least
>>order of
>>magnitude by excluding most of open relays/proxies and forcing easy
>>identification of real sender.
> 
> So far it's unclear how you expect the scheme to exclude open relays
> without also preventing legitimate mail delivery.  

Now we have DNS based "black list", DNS based "white list" have been already 
proposed (with periodic automatic retests).

> And nothing you've mentioned provides any mechanism for
> true sender identification.

SPF is not "true sender identification" but an "SPF like" method can 
significantly reduce the problem.

> ESMTP AUTH could be used for sender authentication w/o adversely affecting
> legitimate mail traffic in the cases mentioned above, but of course ESMTP
> AUTH is possible (and used by some ISPs) today -- no need for "VMN".

You are right for MUA->MTA transmissions but how SMTP AUTH is supposed to 
authenticate sender for MTA->MTA transfers ?
SPF idea is a step in right direction for MTA->MTA transmission.
[ I do not say that SPF itself is optimal]

>>Requirements to implement VMN
>>* VMN indicator in email address e.g. [email-masked]
> 
> Gack. You want to turn the DNS-based hierarchical address scheme into a
> flat namespace?  Fooey and double-fooey.

?!
MTA would treat the above address as:
[email-masked]' via 'SPF' virtual mail network

I can see no flat name space.

>>* enforcement of specific VMN rules by MTA
> 
> I'm not sure how you envision the "mandatory check for open relay" for
> participating hosts. Let's suppose hosts A and B are participating, and
> A wishes to send a message to B.  B checks to see if A is an open relay,
> presumably by attempting to send a message to A. A in turn checks to see
> if B is an open relay.  Doesn't that lead to a stalemate?

I have mentioned use of DNS based "white lists" already.

>>What is you opinion about this "evolutionary path" ?
> 
> You need a better "VMN indicator".  Perhaps a negotiated ESMTP extension.
> Definitely not a kludge in the address.

My design was driven by desire to make VMN available to unmodified email 
clients and restricting modification only to MTAs.

I treat my addressing scheme proposal as option zero to be replaced ASAP if a 
better scheme is proposed.

> The scope of VMN needs to be specified more clearly, and its role in
> various types of SMTP traffic explained. E.g. is it intended for use
> between the sender's MUA and the first-hop SMTP MTA? 
> Or by the recipient's SMTP MTA and connecting clients? 
> Between SMTP MTAs and not MUA SMTP
> clients (how do you propose differentiating an MUA client from an MTA
> client?)?  

My idea was to limit requirement for MUAs and make it available to unmodified 
MUAs:
* MUA->MTA : modified recipient address used by client, optional requirement 
to use SMTP AUTH to MSA port
* MTA->MUA : "X-VMN-Name:" header generated my MTA

VMN requiring "SMTP AUTH to MSA port" may treat check VMN policies per 
"recipient" assuming that the other side is MTA.

> What happens with traditional SMTP (sender's MUA SMTP client
> connects to recipient's SMTP MTA)? How does it work for a sender who
> has multiple ISPs and wishes to use one sender address with another
> ISPs first-hop SMTP server?

It is up to the specific VMN to decide.

OT: It is possible to use "mail in signed mail" relay.

-- 
Andrzej [en:Andrew] Adam Filip http://anfi.freeshell.org backup: [email-masked]

Date: Sun, 08 Feb 2004 05:12:17 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Andrzej Filip <[email-masked]>
CC: [email-masked]
Subject: Re: VMN: Virtual Mail Networks (ESMTP)
Sender: [email-masked]

Andrzej Filip wrote:

> Sample VMN specification:
> * mandatory strong SPF specification for any domain participating
> (e.g. up to 10 hosts which can send messages with sender in specific
> domain)
> * mandatory requirement of domain reachability
> (removal from VMN domains which do not accept mail for more than 4 days)
> * mandatory check for open relay/proxy of any host participating in the VMN

SPF and open relay checks can be troublesome; it is occasionally necessary
for some senders to specify an alternate sender envelope address for
legitimate mail (e.g. when traveling and a different ISP is to be used
for bounces and replies).   SPF etc. might be useful for ISP-ISP transfers,
but they would cause trouble at the sender's initial SMTP session.

Believe it or not, some SMTP servers do go down; that happened recently
for a server handling an IETF mailing list -- it took a few days for
anybody to notice, then took a few more days to leave a message for the
machine's administrator (obviously email couldn't be used), and it was a
few more days before the server was back on-line.

So far, I see problems for legitimate mail traffic, and with not-unusual
outages.

> The above mentioned VMN could reduce "spam as we know it" by at least
> order of
> magnitude by excluding most of open relays/proxies and forcing easy
> identification of real sender.

So far it's unclear how you expect the scheme to exclude open relays
without also preventing legitimate mail delivery.  And nothing you've
mentioned provides any mechanism for true sender identification.

ESMTP AUTH could be used for sender authentication w/o adversely affecting
legitimate mail traffic in the cases mentioned above, but of course ESMTP
AUTH is possible (and used by some ISPs) today -- no need for "VMN".

> Requirements to implement VMN
> * VMN indicator in email address e.g. [email-masked]

Gack. You want to turn the DNS-based hierarchical address scheme into a
flat namespace?  Fooey and double-fooey.

> * enforcement of specific VMN rules by MTA

I'm not sure how you envision the "mandatory check for open relay" for
participating hosts. Let's suppose hosts A and B are participating, and
A wishes to send a message to B.  B checks to see if A is an open relay,
presumably by attempting to send a message to A. A in turn checks to see
if B is an open relay.  Doesn't that lead to a stalemate?

> What is you opinion about this "evolutionary path" ?

You need a better "VMN indicator".  Perhaps a negotiated ESMTP extension.
Definitely not a kludge in the address.

The scope of VMN needs to be specified more clearly, and its role in
various types of SMTP traffic explained. E.g. is it intended for use
between the sender's MUA and the first-hop SMTP MTA? Or by the recipient's
SMTP MTA and connecting clients? Between SMTP MTAs and not MUA SMTP
clients (how do you propose differentiating an MUA client from an MTA
client?)?  What happens with traditional SMTP (sender's MUA SMTP client
connects to recipient's SMTP MTA)? How does it work for a sender who
has multiple ISPs and wishes to use one sender address with another
ISPs first-hop SMTP server?

Date: Sat, 7 Feb 2004 18:10:55 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Cc: [email-masked], Bruce Lilly <[email-masked]>
Sender: [email-masked]

On 2/7/04 at 9:11 AM -0500, Bruce Lilly wrote:

>Two more points related to the 2822 grammar:
>1. mixing vs. separating obs- and non-obs constructs
>2. clarification of "obs"

I'm still going over the unstructured stuff from the last message 
(which look good from my first read), but on these two points 
specifically:

>I think it's clearer to keep the obs- constructs out of the non-obs 
>productions. Indeed, it becomes clearer still if that is pursued to 
>its logical extreme:
>[...]
>i.e. the obs- constructs appear only in the obs-productions, rather 
>than as an alternative in non-obs productions.

Though it may be clearer for someone actually coding a parsing 
engine, I'm not at all sure that it's clearer for other folks. I like 
being able to read through the "generation" grammar and find forward 
references to a difference in the "parser" grammar. I also kind of 
like the idea that you can't read the "generation" grammar and miss 
that there's some work to do in the parsing end. Finally, in terms of 
the prose, I think it will make it more confusing: In the current 
layout, any time you see "obs-" in section 4, you're going to see an 
explanation of that "obs-" construct. If we were to switch it, the 
explanations would be scattered about. I for one would rather leave 
this as-is.

>The intent becomes further clarified with some name changes, 
>especially for non-native English readers and/or casual readers:
>
>gen-comments       =  "Comments" ":" gen-unstructured CRLF
>parse-comments     =  "Comments" *WSP ":" parse-unstructured CRLF
>etc.
>
>to avoid a misunderstanding ("it's obsolete, therefore I don't have 
>to deal with it at all").

I'm un-moved on this. I think having two labels makes things worse 
(clarity-wise), and I don't see "misunderstandings" of the "obs-" 
really being a problem.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sat, 07 Feb 2004 23:15:50 +0100
From: Andrzej Filip <[email-masked]>
Organization: It is for me to know and for you to find out
To: [email-masked]
Subject: VMN: Virtual Mail Networks (ESMTP)
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------040707070706010101040805
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

[reposted from [email-masked]]

We should know if we want to create a new set of email standards or enhance
the existing ones. ESMTP can be easily extended but most features we would
like require support at both sides to be useful.

One possible way of forcing such two-sides features in ESMTP can be VMN
(Virtual Mail Networks). VMN can enforce support for some set of ESMTP
extensions and some additional requirements.

Sample VMN specification:
* mandatory strong SPF specification for any domain participating
(e.g. up to 10 hosts which can send messages with sender in specific domain)
* mandatory requirement of domain reachability
(removal from VMN domains which do not accept mail for more than 4 days)
* mandatory check for open relay/proxy of any host participating in the VMN

The above mentioned VMN could reduce "spam as we know it" by at least order of
magnitude by excluding most of open relays/proxies and forcing easy
identification of real sender.

It is not the only sensible VMN spec but (IMHO) it shows strength of the idea.

Requirements to implement VMN
* VMN indicator in email address e.g. [email-masked]
* enforcement of specific VMN rules by MTA

What is you opinion about this "evolutionary path" ?
IMHO it can be used to create non trivial email systems to test some ideas
presented here and to see which solution will get users' support.

-- 
Andrzej [en:Andrew] Adam Filip http://anfi.freeshell.org backup: [email-masked]

--------------040707070706010101040805
Content-Type: application/pgp-signature;
 name="signature.asc"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
 filename="signature.asc"

LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0KVmVyc2lvbjogR251UEcgdjEuMC43IChH
TlUvTGludXgpCkNvbW1lbnQ6IE15IHB1YmxpYyBrZXlzOiBodHRwOi8vYW5maS5mcmVlc2hl
bGwub3JnL3B1YmtleXMvCgppRDhEQlFGQUpKeUZaODB2V2drUlVwWVJBa1doQUtDM2V6SnU0
c1lJRVc5WHdqK21vUnZ5azVPaUVnQ2ZZMFcrCnBVUHJMdWhndlZNMlpGVFpkSk5jd2t3PQo9
QWNQLwotLS0tLUVORCBQR1AgU0lHTkFUVVJFLS0tLS0KCg==
--------------040707070706010101040805--

Date: Sat, 07 Feb 2004 09:11:38 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Sender: [email-masked]

Two more points related to the 2822 grammar:
1. mixing vs. separating obs- and non-obs constructs
2. clarification of "obs"

Bruce Lilly wrote:
> comments         =   "Comments" ":" unstructured CRLF
> obs-comments     =   "Comments" *WSP ":" [FWS] obs-unstructured CRLF
> etc.
[...]
> That is why separate unstructured and obs-unstructured productions
> are required.
[...]
> unstructured     =   [[FWS] *(utext FWS) utext]
> 
> obs-unstructured =   *(utext FWS) [utext]

These could be partially unified as in 2822 as follows:

comments           =  "Comments" ":" unstructured CRLF
obs-comments       =  "Comments" *WSP ';' unstructured CRLF
unstructured       =  [[FWS] *(utext FWS) utext] / obs-unstructured
obs-unstructured   =  [FWS] *(utext FWS) [utext]

However, I think it's clearer to keep the obs- constructs out of
the non-obs productions. Indeed, it becomes clearer still if that
is pursued to its logical extreme:

comments           =  "Comments" ":" unstructured CRLF
obs-comments       =  "Comments" *WSP ":" obs-unstructured CRLF
unstructured       =  [[FWS] *(utext FWS) utext]
obs-unstructured   =  [obs-FWS] *(utext obs-FWS) [utext]
FWS                =  *WSP CRLF 1*WSP
obs-FWS            =  1*WSP *(CRLF 1*WSP)

i.e. the obs- constructs appear only in the obs-productions, rather than
as an alternative in non-obs productions.  The general structure would
change from:

foo = bar / obs-foo
obs-foo = baz

to:

foo = bar
obs-foo = baz / foo

(recognizing that in some cases, as with obs-FWS, "/ foo" is redundant)

The intent becomes further clarified with some name changes, especially
for non-native English readers and/or casual readers:

gen-comments       =  "Comments" ":" gen-unstructured CRLF
parse-comments     =  "Comments" *WSP ":" parse-unstructured CRLF
etc.

to avoid a misunderstanding ("it's obsolete, therefore I don't have
to deal with it at all").

Date: Thu, 05 Feb 2004 23:21:02 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Sender: [email-masked]

Bruce Lilly wrote:

> obs-text only appears in text (where obs-char makes more sense) and
> obs-utext (which is defined as obs-text). obs-utext appears only in utext,
> and utext appears only in unstructured.
> 
> I'll take another look at the unstructured and related productions.

OK, here's another attempt.  Working backwards from unstructured fields:

comments         =   "Comments" ":" unstructured CRLF
obs-comments     =   "Comments" *WSP ":" [FWS] obs-unstructured CRLF
etc.

N.B. no [FWS] after the colon in the non-obs fields (an unstructured
field cannot be allowed to end with FWS (which it would do if the
"unstructured" production is empty) using non-obs rules, since that
would leave CRLF 1*WS CRLF, i.e. a whitespace only "continuation"
line at the end of the field, which is prohibited by sect. 3.2.3.).
Per contra, that must be permitted in the obs- unstructured fields.
That is why separate unstructured and obs-unstructured productions
are required.

"unstructured" must therefore permit beginning with FWS, but only
if there is content after the FWS. It must permit an empty instance
so that the unstructured field body may be empty.  Since there is
[FWS] in the obs- unstructured fields, and we don't want an explicit
case of two adjacent instances of FWS (CRLF 1*WSP CRLF being provided
for by obs-FWS), a separate obs-unstructured is required, and it must
NOT begin with FWS, but may end with FWS.

unstructured     =   [utext] *(FWS utext)

That doesn't group FWS with a token to its left. It could be rewritten
to do so for the repetition, but would still need provision for leading
FWS (as noted above):

unstructured     =   [[FWS] *(utext FWS) utext]

obs-unstructured =   *(utext FWS) [utext]

To avoid multiple adjacent FWS, and to avoid ending with FWS as mentioned
above, utext must be non-empty. Because any two instances of utext must
be separated by FWS, utext may begin with LF and/or end with CR with no
danger of an unintended CRLF followed by non-WSP appearing. utext must
permit any non-zero number of characters, with any sequence except CRLF
(Charles, you may suggest a different name if you like, but I'll use utext
for consistency with 2822). The obs- form of utext must permit any
US-ASCII octet.  obs-char includes all octets except LF and CR.  So, it's
not too difficult to come up with a definition of utext given those
characteristics and constraints:

utext            =  1*text / obs-utext
obs-utext        =  (CR / 1*(LF / (*CR 1*obs-char))) *CR

text will be as in 2822 except obs-char instead of obs-text, and
with the comment noting exclusion of NUL from non-obs text.  There
is no need for an obs-text production (obs-char could be renamed).

Going back over the required characteristics:
utext must be non-empty. Check.
utext must permit any nonzero number of characters in any sequence
  except CRLF. The first octet can be CR, LF, or obs-char (that's
  every possibility).  The last octet can be CR, LF, or obs-char. In
  between, one can have any number of obs-char, any number of LF, and
  any number of CR provided the last CR is followed by something other
  than LF (i.e. it must be followed by at least one obs-char).  Check.
the non-obs- variety of utext consists of one or more text characters
  (excluding obs-char), which allows any octet except CR, LF, and NUL.
  Seems OK to me.
the "unstructured" production can be empty or can begin with utext or FWS.
  If it is not empty (e.g. if it begins with FWS), it must have some utext.
  Any utext may be followed by FWS and more utext. And so on, ending with
  utext (unless of course unstructured is completely empty).
obs-unstructured can be empty or it can begin with utext. If there is
  additional utext, there must be FWS between the utext instances. It
  may end with FWS or with utext. It cannot begin with FWS.

Can anybody see any problems with that?

Date: Thu, 05 Feb 2004 09:18:22 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Sender: [email-masked]

Charles Lindsey wrote:

> Oh Dear! You misread what I wrote. When I said "*text", I didn't meant
> "*text", I meant the set of rules named 'utext', 'ctext', 'dtext', etc"
> which all have the property that they produce just a single character.
> Your latest offering seemed to be breaking with that convention.

ctext, dtext, qtext, and text are single-octet productions (or should be).
In 2822, obs-text and obs-utext are not.

obs-text only appears in text (where obs-char makes more sense) and
obs-utext (which is defined as obs-text). obs-utext appears only in utext,
and utext appears only in unstructured.

I'll take another look at the unstructured and related productions.

> What I would like him to do would be
>     Subject: Re: foo
> but the proper way to address that issue would be to establish some rule
> or convention about whether subjects could be refolded or have WSP
> collapsed in the course of generating a followup/reply.
> 
> But following a strict reading of 3.6.5 in RFC 2822, I would argue that
> the only compliant way would be
>     Subject:Re:           foo
> from which it is evident that every MUA implementation known to me is
> non-compliant with RFC 2822 :-( . Another bug for Pete to worry over...

The main point is that any such "rule or convention" imposes structure
and that is clearly not compatible with the description and history of
the Subject field being unstructured.

To: [email-masked]
Xref: clerew local.mime:3038
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Date: Thu, 5 Feb 2004 10:59:43 GMT
Lines: 79
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>Charles Lindsey wrote:

>> Yes, but that is getting a long way from what seems to be the established
>> convention that *text things consist of just a single character (or
>> perhaps a single character with some naked CF or LF attached).

>?!?  "text" should instantiate a single character, and it does both
>in 2822 and in the revised grammar.  "*text" by definition (see RFC
>2234) means any number (0 to infinity) of occurrences of "text" and
>therefore could not possibly be restricted to a single character.

Oh Dear! You misread what I wrote. When I said "*text", I didn't meant
"*text", I meant the set of rules named 'utext', 'ctext', 'dtext', etc"
which all have the property that they produce just a single character.
Your latest offering seemed to be breaking with that convention.

>> And I don't think we want two adjacent FWS. Your revised grammar went to
>> much trouble to avoid adjacent CFWS (or FWS in some cases), and that was
>> seen as a great improvement. Now they seem to have come back in.

>RFC 2822 section 4:

>   Another key difference between the obsolete and the current syntax is
>   that the rule in section 3.2.3 regarding lines composed entirely of
>   white space in comments and folding white space does not apply.  See
>   the discussion of folding white space in section 4.2 below.

Again, you miss my point. That rule (or lack of it in the obs case) is not
enforced by syntax but by verbiage. Allowing the grammar to produce
'FWS FWS' does not of itself cause 3.2.3 to cease to apply.

My point was that 'FWS FWS' means exactly the same as 'FWS' (well, to be
pedantic, in insists on a minimum of 2 WSP). All it does it to provide
further opportunity to confuse a parser, which was why you went to the
trouble of not allowing it to occur in the rest of the syntax.

>I still think it's best to simply leave Subject defined as unstructured,
>with no comment about "Re:" (since, as you pointed out, that is already
>allowed by "unstructured"), just as RFC 822 did.  Anything else inevitably
>imposes structure on Subject.  For example if one starts with
>   Subject:          foo
>what should an implementor do to add "Re:":
>   Subject:Re:           foo
>   Subject:          Re: foo
>   Subject: Re:         foo
>etc.? 

What I would like him to do would be
    Subject: Re: foo
but the proper way to address that issue would be to establish some rule
or convention about whether subjects could be refolded or have WSP
collapsed in the course of generating a followup/reply.

But following a strict reading of 3.6.5 in RFC 2822, I would argue that
the only compliant way would be
    Subject:Re:           foo
from which it is evident that every MUA implementation known to me is
non-compliant with RFC 2822 :-( . Another bug for Pete to worry over...

Actually, your revised grammar arranges that any FWS or CFWS that is
encountered is automatically associated with the non-WS-object to its
right. You could equally well have written it so that any FWS or CFWS was
automatically associated with the object to its left. Then the problem
would go away (or, more accurately, would be transferred to the end of an
unstructured where it would no harm).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 04 Feb 2004 22:41:38 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Sender: [email-masked]

Charles Lindsey wrote:

> Yes, but that is getting a long way from what seems to be the established
> convention that *text things consist of just a single character (or
> perhaps a single character with some naked CF or LF attached).

?!?  "text" should instantiate a single character, and it does both
in 2822 and in the revised grammar.  "*text" by definition (see RFC
2234) means any number (0 to infinity) of occurrences of "text" and
therefore could not possibly be restricted to a single character.

> And I don't think we want two adjacent FWS. Your revised grammar went to
> much trouble to avoid adjacent CFWS (or FWS in some cases), and that was
> seen as a great improvement. Now they seem to have come back in.

RFC 2822 section 4:

   Another key difference between the obsolete and the current syntax is
   that the rule in section 3.2.3 regarding lines composed entirely of
   white space in comments and folding white space does not apply.  See
   the discussion of folding white space in section 4.2 below.

Permitting multiple adjacent FWS instances in the obs- constructs was
intended to comply with those parsing requirements, but I suppose that's
handled by obs-FWS (which unlike the other productions in 2822, is not
left-justified in the RFC text).

> May I suggest you take another look at the grammar I originally suggested.

Blech.  Having a non-optional obs- construct in the definition of a
non-obs- production is at best confusing.

>>Note that an unstructured field body begins with [FWS], explicitly at
>>least in the non-obs cases.  Therefore, in
>>  Subject: foo
>>the field body is " foo", not "foo", and
>>  Subject: Re: foo
>>begins with " Re:", not with "Re:", so the wording of section 3.6.5
>>should be revised (or the field name/field body delimiter formally
>>redefined to include any [FWS] or [CFWS} (as the case may be) following
>>the colon). [And A.2 needs to warn about line length limits, including
>>those in effect when encoded-words are present; "prepending" is
>>inadequate as a means of implementation.]
> 
> 
> Yes, I regard that as a problem. What people usually mean by a "subject"
> starts at the first non-blank character, and a subject consisting of just
> FWS would normally be described as "empty". An instruction in some
> protocol that "the subject of the reply SHOULD be the same as the subject
> of the original" should not extend to having exactly the same number or
> initial SPs. So that needs thinking about.

The issue only arises when one discusses modifying the start of a field
body, which only happens in the "Re:" discussion.  Now,
   Subject:Re:  foo
is fine per RFC 2822 rules, but it's not going to make you happy when you're
wearing your Usefor hat.

I still think it's best to simply leave Subject defined as unstructured,
with no comment about "Re:" (since, as you pointed out, that is already
allowed by "unstructured"), just as RFC 822 did.  Anything else inevitably
imposes structure on Subject.  For example if one starts with
   Subject:          foo
what should an implementor do to add "Re:":
   Subject:Re:           foo
   Subject:          Re: foo
   Subject: Re:         foo
etc.?  Clearly an implementor who chooses to support adding "Re:" has to
choose _some_ method for doing so in such a circumstance, and that means
assuming some specific type of structure.  If the document specifies some
structure, then it's silly to maintain that Subject is unstructured.  If
the document is vague about the matter, that opens the possibility that
implementor B will accuse implementor A of being inadequately conservative
in what he generates, or that implementor A will accuse implementor B of
being insufficiently liberal in his interpretation of what he receives, or
(most likely) both. And that's not limited to where "Re:" goes w.r.t.
whitespace, but also "Re" vs. "re" vs. "RE" vs. "rE" vs.
"=?us-ascii*la?q?Re?=" ...

To: [email-masked]
Xref: clerew local.mime:3036
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Date: Wed, 4 Feb 2004 12:44:19 GMT
Lines: 60
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>unstructured = *(text [FWS])
>   assuming unstructured fields are defined as in my revised grammar, e.g.
>   comments = "Comments" ":" [FWS] unstructured CRLF
>   (see discussion below)
>   optionally one could define
>   utext = *(text [FWS])
>   and then define unstructured as utext, but what would be the point...

>obs-utext either as defined in 2822 or as above, i.e. empty, can start
>   or end with obs-char, CR, or LF, but can't have CRLF pair

Yes, but that is getting a long way from what seems to be the established
convention that *text things consist of just a single character (or
perhaps a single character with some naked CF or LF attached).

>obs-unstructured = *(obs-utext FWS) [obs-utext]
>   i.e. cannot have two adjacent instances of obs-utext strings (must
>   have FWS separator), may have multiple adjacent FWS instances (since
>   obs-utext may be empty, and in order to comply with the section 4
>   normative text regarding parsing of WS-only continuation lines), may
>   be empty, may begin or end with any obs-utext string or with FWS,
>   any CRLF pair is followed by WS (as part of FWS)

And I don't think we want two adjacent FWS. Your revised grammar went to
much trouble to avoid adjacent CFWS (or FWS in some cases), and that was
seen as a great improvement. Now they seem to have come back in.

May I suggest you take another look at the grammar I originally suggested.

>Note that an unstructured field body begins with [FWS], explicitly at
>least in the non-obs cases.  Therefore, in
>   Subject: foo
>the field body is " foo", not "foo", and
>   Subject: Re: foo
>begins with " Re:", not with "Re:", so the wording of section 3.6.5
>should be revised (or the field name/field body delimiter formally
>redefined to include any [FWS] or [CFWS} (as the case may be) following
>the colon). [And A.2 needs to warn about line length limits, including
>those in effect when encoded-words are present; "prepending" is
>inadequate as a means of implementation.]

Yes, I regard that as a problem. What people usually mean by a "subject"
starts at the first non-blank character, and a subject consisting of just
FWS would normally be described as "empty". An instruction in some
protocol that "the subject of the reply SHOULD be the same as the subject
of the original" should not extend to having exactly the same number or
initial SPs. So that needs thinking about.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 03 Feb 2004 19:54:20 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Charles Lindsey <[email-masked]>, Pete Resnick <[email-masked]>
CC: [email-masked]
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> 
>>I think that the following does the right thing (obs-utext can be empty
>>or it can start or end with any ASCII octet and can have any sequence
>>except CRLF, which is handled in unstructured by FWS, and unstructured
>>can be completely empty or it can begin or end with utext or FWS, but
>>any instance of utext is separated from any other instance by FWS):
> 
> 
>>obs-utext       =       *(*obs-char (*LF / (*CR 1*obs-char))) *CR
> 
> 
> So, if X is some randon obs-char, "XCR" is an obs-utext, and "LF" in an
> obs-utext. Therefore "XCRLF" is an unstructured. Q.N.E.D.
> 
> 
>>unstructured    =       *(utext FWS) *utext
> 
> 
> No, that is not right with either your subsequent fix or with Pete's
> subsequent fix, because you allow two FWS adjacent, and Pete does not
> allow an unstructured consisting of FWS and nothing else.

OK, there's a problem with 2822; it needs an obs-unstructured as well
as unstructured (see section 4 introductory paragraphs, specifically
the reference to 3.2.3).  Here's one way to tie everything together:

text as currently defined in 2822, with the comment extended to note that
     NUL is also excluded

obs-char as currently defined in 2822, i.e. any ASCII character except
    CR and LF

unstructured = *(text [FWS])
   assuming unstructured fields are defined as in my revised grammar, e.g.
   comments = "Comments" ":" [FWS] unstructured CRLF
   (see discussion below)
   optionally one could define
   utext = *(text [FWS])
   and then define unstructured as utext, but what would be the point...

obs-utext either as defined in 2822 or as above, i.e. empty, can start
   or end with obs-char, CR, or LF, but can't have CRLF pair

obs-unstructured = *(obs-utext FWS) [obs-utext]
   i.e. cannot have two adjacent instances of obs-utext strings (must
   have FWS separator), may have multiple adjacent FWS instances (since
   obs-utext may be empty, and in order to comply with the section 4
   normative text regarding parsing of WS-only continuation lines), may
   be empty, may begin or end with any obs-utext string or with FWS,
   any CRLF pair is followed by WS (as part of FWS)

with obs- forms of unstructured fields as in

obs-comments = "Comments" *WSP ":" obs-unstructured CRLF

Discussion: my revised grammar starts unstructured field bodies with
[FWS} after the colon, partly as a result of handling Subject hacks
and partly to be consistent in grouping (possibly optional) FWS/CFWS
to the right of another token to enable unambiguous LR(1) parsing.
The obs- forms of unstructured fields could be similarly defined,
however [FWS] after the colon would be redundant since obs-unstructured
may begin with FWS.  A formal field definition consistent with the
non-obs fields w.r.t. explicit [FWS] after the colon still might be
desirable, but the redundancy should be noted in a comment in the ABNF
or in nearby text.

Other considerations: "unstructured" might be referred to by extension
RFCs.  Are there any problems with the definition above (e.g. w.r.t.
a possible RFC 2047 successor, which currently refers to unstructured
fields as being defined as (RFC 822) *text)?

Note that an unstructured field body begins with [FWS], explicitly at
least in the non-obs cases.  Therefore, in
   Subject: foo
the field body is " foo", not "foo", and
   Subject: Re: foo
begins with " Re:", not with "Re:", so the wording of section 3.6.5
should be revised (or the field name/field body delimiter formally
redefined to include any [FWS] or [CFWS} (as the case may be) following
the colon). [And A.2 needs to warn about line length limits, including
those in effect when encoded-words are present; "prepending" is
inadequate as a means of implementation.]

To: [email-masked]
Xref: clerew local.mime:3034
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Date: Tue, 3 Feb 2004 12:10:40 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>I think that the following does the right thing (obs-utext can be empty
>or it can start or end with any ASCII octet and can have any sequence
>except CRLF, which is handled in unstructured by FWS, and unstructured
>can be completely empty or it can begin or end with utext or FWS, but
>any instance of utext is separated from any other instance by FWS):

>obs-utext       =       *(*obs-char (*LF / (*CR 1*obs-char))) *CR

So, if X is some randon obs-char, "XCR" is an obs-utext, and "LF" in an
obs-utext. Therefore "XCRLF" is an unstructured. Q.N.E.D.

>unstructured    =       *(utext FWS) *utext

No, that is not right with either your subsequent fix or with Pete's
subsequent fix, because you allow two FWS adjacent, and Pete does not
allow an unstructured consisting of FWS and nothing else.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 2 Feb 2004 11:12:26 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Cc: Bruce Lilly <[email-masked]>
Sender: [email-masked]

On 2/1/04 at 8:00 AM -0500, Bruce Lilly wrote:

>>I think that the following does the right thing (obs-utext can be 
>>empty or it can start or end with any ASCII octet and can have any 
>>sequence except CRLF, which is handled in unstructured by FWS, and 
>>unstructured can be completely empty or it can begin or end with 
>>utext or FWS, but any instance of utext is separated from any other 
>>instance by FWS):
>>
>>obs-utext = *(*obs-char (*LF / (*CR 1*obs-char))) *CR
>>
>>unstructured = *(utext FWS) *utext
>
>Oops, the last line should be:
>
>unstructured    =       *(*utext FWS) *utext

No, that doesn't look right. It allows multiple adjacent occurrences 
of FWS. How about:

unstructured = *(1*utext FWS) *utext

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sun, 01 Feb 2004 08:00:02 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
CC: Pete Resnick <[email-masked]>
Subject: Re: Revisiting RFC 2822 grammar (obs-utext and unstructured)
Sender: [email-masked]

Bruce Lilly wrote:

> I think that the following does the right thing (obs-utext can be empty
> or it can start or end with any ASCII octet and can have any sequence
> except CRLF, which is handled in unstructured by FWS, and unstructured
> can be completely empty or it can begin or end with utext or FWS, but
> any instance of utext is separated from any other instance by FWS):
> 
> obs-utext       =       *(*obs-char (*LF / (*CR 1*obs-char))) *CR
> 
> unstructured    =       *(utext FWS) *utext

Oops, the last line should be:

unstructured    =       *(*utext FWS) *utext