Subject: Re: MTS transparency and anonymity
From: Keith Moore <[email-masked]>
To: Frank Ellermann <[email-masked]>
Cc: [email-masked]
Date: Tue, 1 Mar 2005 00:48:23 -0500
List-Id: <ietf-822.imc.org>

>> my point is that if the message is rejected a bounce will be
>> created, which is likely to be collateral spam.
>
> Yes.  And RfC 3834 replies, all "legacy" auto-responders, and
> all broken-by-design C/R-systems cause more "collateral spam".

until someone figures out a reliable way to distinguish spam from 
legitimate mail, any auto-responder is going to create some collateral 
spam.  there's no way around that.  RFC 3834 does expect responders to 
decide whether or not it's appropriate to respond to a message.

> The essence of this problem is "bounces-to" versus "source" as
> in <http://article.gmane.org/gmane.mail.spam.spf.discuss:14678>

no that's not the essence of the problem.  and fwiw, you're wrong about 
something you said in that article, in particular this:

> It's extremely important to understand, that the MAIL FROM
> address is not only some potentially unrelated Errors-To
> address, it is THE source, THE sender, THE originator.

MAIL FROM is fundamentally the address where bounces go.  It is NOT 
necessarily any of the source, the sender, or the originator.  There 
are and always have been valid reasons for MAIL FROM to be set to some 
other address than the source/sender/originator.

...and it's extremely important that you understand this.

Keith

From: "Hector Santos" <[email-masked]>
To: "Tony Finch" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 23:31:01 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Tony Finch" <[email-masked]>
To: "Keith Moore" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Monday, February 28, 2005 1:50 PM
Subject: Re: MTS transparency and anonymity

>
> > > We've found that SMTP call-back return path verification is quite
> > > effective, though it has very nasty behaviour for the victims of joe
jobs.
> >
> > I have somewhat different experience - I've found that SMTP call-back
> > verification often produces unnecessary failures, as it's not unusual
for
> > the sender's SMTP server to fail to respond promptly even though the
> > message is legitimate.
>
> Oh yes, I didn't say it was easy to use and without interop problems :-)
> But they can be mitigated with judicious use of exemption lists etc.

In practice, it is very effective simply because the fact is  60-80% of the
transactions are forged or problematic.  It will undoubted catch a very
signficant portion these exact real "failure" points.

The implementation should use a non-permanent 45x response code to allow a
remote system to retry.

However, as it been proven in practice, as also illustrated with the growth
of Grey-Listing usage,  it will work 100% for 100% failure points (the other
system will not retry), and if its a unexpected downage, the legitimate
sender will try again.

Now, the issue is what here?

SMTP compliant.  If you use a MAIL FROM Return Path, it is deemed to be
reliable and researchable. If you feel difference, then we can stop here.

On the other hand, since it is deemed to be a researchable address, if it is
not, then the sender is NOT SMTP compliant - period.

This is why it works for thousands of our installations with many HAPPY
faces and little to know false positives. And as noted above, when there is
a false positive,  the legitimate transaction will be flagged one we or
another and the issue resolved as asap.  Normally, it all points to a poor
setup of SMTP compliant systems.  Not in one instance has a customer has
reported -  "This is a serious problem we can not resolve or unrelated to
SMTP compliancy."

It is a matter of HOW you view the SMTP compliancy issue of the Return Path.

Note: This has nothing to with zombie sites. These are picked up other ways.
The point of the Return Path Verification Call Back it so make sure the
transaction is SMTP compliant - nothing more, nothing less and again, since
the majority of all transactions are spoofed or NXDOMAINS,  it is proven to
be a very reliable method against malicious transactions.

Two words -  "SMTP COMPLIANCY" is the first step to securing a system.

This is my only comment about this and I will steer clear of this thread.  I
just wanted to point out that the real experts are in the field actually
putting this stuff to work in practice - it is not theory, a raised back
hair or a devil on your shoulder whispering into your ear.

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Tue, 01 Mar 2005 02:48:44 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 13
Sender: [email-masked]

Tony Finch wrote:
 
> my point is that if the message is rejected a bounce will be
> created, which is likely to be collateral spam.

Yes.  And RfC 3834 replies, all "legacy" auto-responders, and
all broken-by-design C/R-systems cause more "collateral spam".

The essence of this problem is "bounces-to" versus "source" as
in <http://article.gmane.org/gmane.mail.spam.spf.discuss:14678>

                    Sigh, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 23:10:12 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

Bruce Lilly wrote:
 
> Ambiguity to non-English and English speakers alike:
> http://dictionary.reference.com/search?q=invalid

One of the offered entries reflects the 2119 meaning:

| Not legally or factually valid; null

Adding a reserved IDNA example to the existing example.cno
domains could be a nice idea, I tend to abuse xn--ko-eka.de
for this purpose.
                   Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 22:49:41 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 31
Sender: [email-masked]

Bruce Lilly wrote:

> It's also noteworthy that use of domain literals is strongly
> discouraged, accompanied by SCREAMING ALL CAPS in 822

Yes, it's funny.  But the idea is clear, today we would use a
SHOULD NOT + a statement about the terminology + a normative
reference to 2119.

OTOH you and Keith think that TLD .invalid is an obscure idea,
if that's true it justifies to ignore "SHOULD NOT" for any@[]

> But there's the 2821 ABNF that doesn't permit that.

The RfC 2821 ABNF also doesn't permit localhost, tv, and other
FQDN host names without a dot.  No problem for any@[], because
that's the idea of this construct.  Or at least not worse than
[email-masked] until users and UAs know these conventions.

>> Your "updates RfC" section could be a new record.
[...]
> I'm not sure what you mean; making the field optional will
> update 822 and 2822.

Sorry, I confused this with your chapter 3 (scope) and its
nine (?) affected RfCs not counting "etc.".  It's also a pain
for mail2news gateways, for message/news, for the processing
of cancel messages in net news, "etc.".

                          Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 22:04:10 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

Tony Finch wrote:
 
>> ideally, return-paths would be authenticated (in the sense
>> that the sender needs to be able to demonstrate he has the
>> right to use that return-path), and MTAs wouldn't bounce
>> messages that failed authentication.
 
> This is the fallacy of universal deployment: you can't assume
> that everyone will upgrade their systems to conform to best 
> current practice, so you can't rely on BCP to eliminate
> backscatter.

There's no fallacy and no FUSSP if you can get the spammers 
to think that it's not in their best interest to forge any
authenticated return-paths.  It's a self-fulfilling prophecy
for the "collateral spam" part of the problem.  Without it
the best 3834 application would be "shoot [Auto] on sight."

                          Bye, Frank

Date: Mon, 28 Feb 2005 21:02:21 +0000
From: Tony Finch <[email-masked]>
To: Frank Ellermann <[email-masked]>
cc: [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

On Mon, 28 Feb 2005, Frank Ellermann wrote:
>
> The systems based on RMX assume that most emails go through
> a single imaginary point for the one hop from any mailout of
> the sender to any MX of the receiver.  That's a singularity,
> but not single-hop.

Yes, but my point is that if the message is rejected a bounce will be
created, which is likely to be collateral spam.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SOUTH FITZROY: NORTHEAST 7 TO SEVERE GALE 9 DECREASING 4 OR 5. OCCASIONAL
RAIN. MODERATE OR GOOD.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 21:48:40 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 13
Sender: [email-masked]

Tony Finch wrote:

> other email authentication systems don't solve the problem -
> or if they claim to do so they assume that email delivery is
> single-hop
[...]

The systems based on RMX assume that most emails go through
a single imaginary point for the one hop from any mailout of
the sender to any MX of the receiver.  That's a singularity,
but not single-hop.
                        Bye, Frank

Date: Mon, 28 Feb 2005 18:50:37 +0000
From: Tony Finch <[email-masked]>
To: Keith Moore <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

> > We've found that SMTP call-back return path verification is quite
> > effective, though it has very nasty behaviour for the victims of joe jobs.
>
> I have somewhat different experience - I've found that SMTP call-back
> verification often produces unnecessary failures, as it's not unusual for
> the sender's SMTP server to fail to respond promptly even though the
> message is legitimate.

Oh yes, I didn't say it was easy to use and without interop problems :-)
But they can be mitigated with judicious use of exemption lists etc.

> [bounce address verification]  Though IMHO all of these systems are
> "not ready for prime time" and it's way too early to try to avoid
> duplicating effort.

Indeed. Which is why I'm working on my own flavour of the idea so we get
as many data points as possible.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BISCAY: NORTHEAST 5 OR 6 BECOMING VARIABLE 3 OR 4, THEN SOUTHWEST 4 OR 5 LATER
IN NORTH. OCCASIONAL RAIN. MAINLY GOOD.

Date: Mon, 28 Feb 2005 13:31:54 -0500
From: Keith Moore <[email-masked]>
To: Tony Finch <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

> > the usual trick is to try to arrange things so that those who do upgrade
> > their systems will see an immediate benefit - i.e. to put the incentives
> > in the right place.  I think this would be true for RP authentication -
> > MTAs today see a substantial load from boucing forged mail.
> 
> We've found that SMTP call-back return path verification is quite
> effective, though it has very nasty behaviour for the victims of joe jobs.

I have somewhat different experience - I've found that SMTP call-back 
verification often produces unnecessary failures, as it's not unusual for
the sender's SMTP server to fail to respond promptly even though the 
message is legitimate.
 
> Detecting backscatter by inserting a cookie in the return path does have
> the right incentives: it can be implemented unilaterally by a site and
> should be effective without co-operation from the rest of the Internet
> (though we need to study its interoperability in more detail).

Yes, there are lots of dragons here.  

> If you couple this with a means for recipients to find out whether to
> expect a cookie, and to verify one if it is present, then you have a
> general return path authentication system. However this idea has a number
> of security holes in the area of replay attacks and message modification
> attacks, so it is not a panacea. It's also skating very close to the
> territory of DomainKeys and IIM so working on it might be a pointless
> duplication of effort.

It's very close to IIM.   Though IMHO all of these systems are
"not ready for prime time" and it's way too early to try to avoid 
duplicating effort. 

Keith

Date: Mon, 28 Feb 2005 13:21:53 -0500
From: Keith Moore <[email-masked]>
To: Vince Sabio <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Site policy vs. HELO
Sender: [email-masked]

>     [QUOTE RFC2821 SECT 7.7]
> 
>     In recent years, use of the relay function through arbitrary sites
>     has been used as part of hostile efforts to hide the actual origins
>     of mail.  Some sites have decided to limit the use of the relay
>     function to known or identifiable sources, and implementations SHOULD
>     provide the capability to perform this type of filtering.  When mail
>     is rejected for these or other policy reasons, a 550 code SHOULD be
>     used in response to EHLO, MAIL, or RCPT as appropriate.
> 
>     [END QUOTE]
> 
> So, the question: Is this a valid interpretation of Sect. 7.7 -- 
> i.e., may an MTA provide a 550 response to HELO where it would 
> otherwise have given a 250 response to an EHLO if the site policy for 
> that MTA forbids HELO?

no.  7.7 is about third-party relaying, not about differentiating between
EHLO and HELO.  

Differentiating between EHLO and HELO for filtering purposes is a Really 
Bad Idea.  We have longstanding practice that HELO is a valid way to send
mail that doesn't need some ESMTP feature, and a lot of legitimate mail
is still sent using HELO.  Even if it were found that at the present time,
HELO vs. EHLO had some correlation to spam or viruses, the perps would
quickly adapt to any filters that tried to differentiate between the two.

Keith

Date: Mon, 28 Feb 2005 18:17:42 +0000
From: Tony Finch <[email-masked]>
To: Keith Moore <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

> the usual trick is to try to arrange things so that those who do upgrade
> their systems will see an immediate benefit - i.e. to put the incentives
> in the right place.  I think this would be true for RP authentication -
> MTAs today see a substantial load from boucing forged mail.

We've found that SMTP call-back return path verification is quite
effective, though it has very nasty behaviour for the victims of joe jobs.
A more lightweight and scalable alternative is needed for Internet-wide
deployment.

Detecting backscatter by inserting a cookie in the return path does have
the right incentives: it can be implemented unilaterally by a site and
should be effective without co-operation from the rest of the Internet
(though we need to study its interoperability in more detail).

If you couple this with a means for recipients to find out whether to
expect a cookie, and to verify one if it is present, then you have a
general return path authentication system. However this idea has a number
of security holes in the area of replay attacks and message modification
attacks, so it is not a panacea. It's also skating very close to the
territory of DomainKeys and IIM so working on it might be a pointless
duplication of effort.

I've written more on this topic elsewhere:
http://www.cus.cam.ac.uk/~fanf2/hermes/doc/talks/2005-02-ukuug/
http://www.cus.cam.ac.uk/~fanf2/hermes/doc/antiforgery/cam.txt

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
HUMBER: SOUTHWEST 5 OR 6 BECOMING VARIABLE 3 OR 4. RAIN OR SLEET. MODERATE OR
GOOD.

Date: Mon, 28 Feb 2005 13:00:49 -0500
From: Keith Moore <[email-masked]>
To: Tony Finch <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

> > the best way I know to deal with that problem is to recognize bounces
> > (not just by return-path but by message format) and to correlate them with
> > a Sent IMAP mailbox.
> 
> This is just another variant of including a cookie in outgoing email which
> you check for in the bounces that come back.
> 
> Putting the cookie in the message data doesn't work in general because
> many bounces are unparseable, or they don't include enough of the original
> message. The extreme example of this is vacation messages which usually
> include nothing of the original; also some bounces have extremely
> abbreviated copies of the original header.

putting the cookie in return-path might work better.   putting it elsewhere
in the envelope would be cleaner, but would require all MTAs in the path 
to be upgraded before it became useful.
 
> > more ideally, return-paths would be authenticated (in the sense that the
> > sender needs to be able to demonstrate he has the right to use that
> > return-path), and MTAs wouldn't bounce messages that failed authentication.
> 
> This is the fallacy of universal deployment: you can't assume that
> everyone will upgrade their systems to conform to best current practice,
> so you can't rely on BCP to eliminate backscatter.

the usual trick is to try to arrange things so that those who do upgrade
their systems will see an immediate benefit - i.e. to put the incentives
in the right place.  I think this would be true for RP authentication -
MTAs today see a substantial load from boucing forged mail.

Keith

Date: Mon, 28 Feb 2005 12:54:43 -0500
From: Keith Moore <[email-masked]>
To: Tony Finch <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

> 
> On Mon, 28 Feb 2005, Charles Lindsey wrote:
> >
> > There are four solutions that have been proposed:
> >
> > 1. Omit the From field entirely
> > 2. Use some invalid domain, e.g. @blah.invalid.
> > 3. Use some invalid IP, e.g. @[] or @[0,0,0,0]
> > 4. Introduce some new syntax, e.g. From: "Mickey Mouse" <>
> 
> 5. Don't make any change because pseudonymity is enough.

You're talking about a different problem.  It's one thing to say
"this message doesn't have a known sender, or the sender doesn't 
correspond to any email address", and something else to say
"the sender of this message doesn't want to reveal his identity".
We've been discussing how to solve the first of these problems.

For the first problem, you don't want to reply to the message because
the reply will go nowhere.  You want either no address or some kind of
address that the mail system will quickly realize is invalid. And
you would like to have some convention for these addresses so that
they can be identified as early as possible, ideally by the recipient's
MUA so that it can say "sorry, you can't reply to this address".

For pseydonymity, replies are often quite useful - you just need to
arrange for the messages to get sent by a path that makes them difficult
to trace.  You want a valid address that the mail system will treat
normally, but which will get routed via a mechanism that makes it
difficult to associate the pseudonym with the actual correspondent.

> Some systems reject messages that have no valid originator address in
> the message header, so if there is no From: there would have to be a
> Sender:

That, and because the specifications require it.  But the Sender address
doesn't have to be that of a person, and it doesn't have to be useful
for replies.  It's just a tag that can be associated with whoever or
whatever sent the message.

> Some systems reject messages that have no valid originator address in the
> message header, so if there is no From: there would have to be a Sender:
> In addition to that, this kind of checking would also cause them to reject
> [email-masked] From: addresses, or those using bogon IP addresses in
> domain literals.

Filters employ lots of bogus criteria.  Even messages that claim to not have
a usable From address can still be valid.  They can even be authenticated.

Keith

Date: Mon, 28 Feb 2005 12:44:28 -0500
To: ietf-822 <[email-masked]>
From: Vince Sabio <[email-masked]>
Subject: Site policy vs. HELO
Sender: [email-masked]

I'm bringing this question over here from the SPAM-L list.

The following graf in RFC2821 appears to require an MTA (not an 
RFC2476-type MSA) to fall back to HELO for compatibility with older 
SMTP clients and servers):

    [QUOTE RFC2821 SECT 2.2.1]

    Contemporary SMTP implementations MUST support the basic extension
    mechanisms.  For instance, servers MUST support the EHLO command even
    if they do not implement any specific extensions and clients SHOULD
    preferentially utilize EHLO rather than HELO.  (However, for
    compatibility with older conforming implementations, SMTP clients and
    servers MUST support the original HELO mechanisms as a fallback.)

    [END QUOTE]

However, it has recently been pointed out that the following graf in 
RFC2821 may permit sites to _not_ be required to fall back to HELO -- 
and issue a 550 response to HELO instead -- if local policy dictates:

    [QUOTE RFC2821 SECT 7.7]

    In recent years, use of the relay function through arbitrary sites
    has been used as part of hostile efforts to hide the actual origins
    of mail.  Some sites have decided to limit the use of the relay
    function to known or identifiable sources, and implementations SHOULD
    provide the capability to perform this type of filtering.  When mail
    is rejected for these or other policy reasons, a 550 code SHOULD be
    used in response to EHLO, MAIL, or RCPT as appropriate.

    [END QUOTE]

So, the question: Is this a valid interpretation of Sect. 7.7 -- 
i.e., may an MTA provide a 550 response to HELO where it would 
otherwise have given a 250 response to an EHLO if the site policy for 
that MTA forbids HELO?

__________________________________________________________________________
Vince Sabio                                                  [email-masked]

Date: Mon, 28 Feb 2005 17:43:25 +0000
From: Tony Finch <[email-masked]>
To: Keith Moore <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

On Mon, 28 Feb 2005, Keith Moore wrote:
>
> the best way I know to deal with that problem is to recognize bounces
> (not just by return-path but by message format) and to correlate them with
> a Sent IMAP mailbox.

This is just another variant of including a cookie in outgoing email which
you check for in the bounces that come back.

Putting the cookie in the message data doesn't work in general because
many bounces are unparseable, or they don't include enough of the original
message. The extreme example of this is vacation messages which usually
include nothing of the original; also some bounces have extremely
abbreviated copies of the original header.

> more ideally, return-paths would be authenticated (in the sense that the
> sender needs to be able to demonstrate he has the right to use that
> return-path), and MTAs wouldn't bounce messages that failed authentication.

This is the fallacy of universal deployment: you can't assume that
everyone will upgrade their systems to conform to best current practice,
so you can't rely on BCP to eliminate backscatter.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
ROCKALL MALIN HEBRIDES: WEST VEERING NORTH 6 TO GALE 8, PERHAPS SEVERE GALE 9
LATER. RAIN THEN WINTRY SHOWERS. MODERATE OR GOOD.

Date: Mon, 28 Feb 2005 12:30:05 -0500
From: Keith Moore <[email-masked]>
To: Tony Finch <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

> On Mon, 28 Feb 2005, Keith Moore wrote:
> >
> > could you explain the backscatter problem?  I'm not sure what you're referring
> > to.
> 
> If a spammer or a virus sends out forged email (a "joe job") a certain
> proportion of it is likely to be rejected, e.g. because of incorrect
> recipient addresses or anti-virus or anti-spam or anti-forgery checks. If
> the rejection results in a bounce message the bounce will be sent to the
> apparent sender of the message, who is an innocent third party and the
> victim of the joe job. These bounces are backscatter from forged spam, aka
> collateral spam.

the best way I know to deal with that problem is to recognize bounces
(not just by return-path but by message format) and to correlate them with
a Sent IMAP mailbox.  bounces that don't match outgoing mail are silently
discarded.  of course, not everybody use IMAP, and not all IMAP clients 
support storing sent mail in a Sent mailbox.

more ideally, return-paths would be authenticated (in the sense that the 
sender needs to be able to demonstrate he has the right to use that 
return-path), and MTAs wouldn't bounce messages that failed authentication.

Keith

Date: Mon, 28 Feb 2005 12:26:29 -0500
From: Keith Moore <[email-masked]>
To: ietf-822 <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-03.txt
Sender: [email-masked]

> > This feature is about having mboxes containing all the mails
> > from a given period available, in application/mbox format.
> 
> I'm not sure that downloading an entire huge collection of
> messages in one gigantic gulp is the best approach.

it depends on what you need.  often when researching a particular
working group's decisions I have found it convenient to do exactly that.

Keith

Date: Mon, 28 Feb 2005 17:25:14 +0000
From: Tony Finch <[email-masked]>
To: Charles Lindsey <[email-masked]>
cc: [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

On Mon, 28 Feb 2005, Charles Lindsey wrote:
>
> There are four solutions that have been proposed:
>
> 1. Omit the From field entirely
> 2. Use some invalid domain, e.g. @blah.invalid.
> 3. Use some invalid IP, e.g. @[] or @[0,0,0,0]
> 4. Introduce some new syntax, e.g. From: "Mickey Mouse" <>

5. Don't make any change because pseudonymity is enough.

Some systems reject messages that have no valid originator address in the
message header, so if there is no From: there would have to be a Sender:
In addition to that, this kind of checking would also cause them to reject
[email-masked] From: addresses, or those using bogon IP addresses in
domain literals.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SELSEY BILL TO LYME REGIS: WEST OR SOUTHWEST 3 OR 4 INCREASING 5 OR 6 FOR A
TIME. FAIR THEN RAIN OR SHOWERS. GOOD BECOMING OCCASIONALLY MODERATE. SLIGHT
BECOMING MODERATE.

Date: Mon, 28 Feb 2005 12:21:57 -0500
From: Keith Moore <[email-masked]>
To: ietf-822 <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

> On Mon February 28 2005 09:29, Keith Moore wrote:
> 
> > nobody cares about 733 anymore.  most things in use today were written 
> > with 822 in mind.
> 
> Messages tend to stick around for a long time in archives, and
> may occasionally need to be read.  RFCs are not carved in
> granite; they're amendable and have been amended (e.g. 1123
> amends 822's date-time syntax, among other things).  For that
> matter, when a 2822 successor reaches full Standard status, 822
> will be of as much interest as 733 now is -- as an historical
> document.

Nope.  There are hardly any messages around now that were written to 
conform to rfc 733.  OTOH even after 2822's successor reaches
full standard status, there will still be (US) billions of 
messages written to conform to rfc 822.  And there will still be
a few other specifications that reference rfc 822 rather than
[245]822.

Standardization status should not be confused with deployment.

>  Note that Return-Path theoretically can be an empty
> (bracketed) path, as provided for by RFC 1123. But it was not
> always so (i.e. that has changed), and a careful read of RFC 2821
> reveals that it lacks provision for an empty return path in the
> Return-Path field ABNF.  The point is that if there is sufficient
> cause for a modification, there's no absolute barrier to
> such modification. 

That's true.  But part of what determines "sufficient cause" is the
anticipated impact on the installed base.  It was much easier to
change things in the days of 1123, as the net wasn't nearly as large
or diverse then.  There wasn't so much inertia to overcome, and it
was easier to assess the impact of a proposal.

> A secondary point re. 733 is that given
> a need to make some modification to syntax, a modification
> that has some established precedent may be preferable to one
> which lacks any operational history.

It was so long ago, and the conditions are so different, that any 
precedent is of dubious value.

> > > HTTP and
> > > SIP are mentioned because they refer back to the (RFC 822)
> > > defined syntax of the From field; any change to that syntax
> > > will affect those protocols as well as the Internet Message
> > > Format.
> > 
> > no.  they refer to 822, not to some newer specification.  822 hasn't 
> > changed, and won't change.
> 
> The published text document is static, but it can be amended by
> several established mechanisms: the RFC Editor errata page, and
> through amendment by subsequent Standards Track RFCs (e.g. 1123). 

Yes, but I trust that when they amend HTTP they will do whatever
is necessary to preserve compatibity with HTTP implementations and
previous versions of the HTTP standard, not blindly update the
reference to point to the current email specification.

> > > When you propose such syntax changes, you must consider
> > > the effects on all protocols affected by the change.
> > 
> > if they are truly affected by the change, yes.  but it's not as if HTTP 
> > and SIP use email as a substrate.
> 
> They use the RFC 822 From field as defined by RFC 822 as amended
> by the mechanisms discussed above.  It would be irresponsible to
> introduce a syntax change without considering the consequences
> for HTTP and SIP.

It would be irresponsible to saddle any of HTTP, SIP, or email with
the baggage from the other protocols.

> The intent for the draft under discussion is to become a Standards
> Track RFC amending RFCs 822 and 2822.  The approach of making the
> from field optional in the Internet Message Format message header
> does not change the syntax of the From field, and does not affect
> either HTTP or SIP (because those protocols define their respective
> formats and determine which fields are permitted and/or required
> for those formats, independently of the Internet Message Format).

Yes, those things can be done.  But we do care about compatibility.
It's my opinion that the gain resulting from making the From field
optional is not worth the impact on the installed base.  
 
> > I still think that it makes more sense  
> > to use a domain literal than a domain.
> 
> We're agreed that a domain name has problems.  I'm not thrilled
> about a hypothetical domain literal which has magic properties
> unfathomable to casual users -- academic since there doesn't
> appear to be a suitable literal anyway.
> 
> > I still think it will break  
> > fewer things to have a From field with an actual address than to have 
> > no From field or From:<>.
> 
> I don't think a definitive statement is possible. Syntax changes
> would require hearing from HTTP and SIP folks. 

Not unless you're explicitly changing HTTP and SIP in addition to
email.  Of course if you are proposing to do that then you do need
to get buyin from those folks.  I don't see this as either necessary
or particularly useful.

Keith

Date: Mon, 28 Feb 2005 17:19:16 +0000
From: Tony Finch <[email-masked]>
To: Keith Moore <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

On Mon, 28 Feb 2005, Keith Moore wrote:
>
> could you explain the backscatter problem?  I'm not sure what you're referring
> to.

If a spammer or a virus sends out forged email (a "joe job") a certain
proportion of it is likely to be rejected, e.g. because of incorrect
recipient addresses or anti-virus or anti-spam or anti-forgery checks. If
the rejection results in a bounce message the bounce will be sent to the
apparent sender of the message, who is an innocent third party and the
victim of the joe job. These bounces are backscatter from forged spam, aka
collateral spam.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
THAMES DOVER WIGHT PORTLAND PLYMOUTH: VARIABLE BECOMING SOUTHWEST 3 OR 4,
INCREASING 5 TO 7. RAIN OR SLEET LATER. GOOD BECOMING MODERATE.

To: [email-masked]
Xref: clerew local.mime:4476
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 13:07:04 GMT
Lines: 96
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>So far as I can see, there is no perfectly clean solution to the
>issues covered in the draft.

Agreed.

>  However, simply making the From
>field optional -- while that may present some issues -- seems to
>have the least negative impact of the options considered (including
>syntax changes and extension, which present rather more serious
>backwards compatibility issues than making the field optional).

But I don't think you have established that at all.

>A syntax change to permit "From: <>" or named group syntax "From:
>phrase: (possibly empty) list;" would have the desired semantics,
>but appears to have more serious backwards compatibility issues
>than making the field optional;

Nor that.

The concern is that whatever solution we adopt may have some negative
impact on the installed base (clearly, future implementations could be
made to cope with any of them).

There are four solutions that have been proposed:

1. Omit the From field entirely
2. Use some invalid domain, e.g. @blah.invalid.
3. Use some invalid IP, e.g. @[] or @[0,0,0,0]
4. Introduce some new syntax, e.g. From: "Mickey Mouse" <>

The requirements for acceptable impact on the installed base are:

A. The message (once past the initiating MUA and submission agent) must be
   delivered reliably to its intended destination, and, when so delivered,
   should be unaltered.

B. The message must be unreplyable, and the failure to reply should be
   done gracefully.

So we can consider how those requirements are met in the 4 cases:

                1. No From      2. blah.invalid 3. @[]          4. From <>

A. delivery     98% [1]         100%            98%             98%

   unaltered    maybe not [2]   100%            98%             90% [3]

B. replyable    possibly [2]    2% [1]          2%              0%	

   graceful     Yes             maybe [4]       maybe [5]       probably [6]

NOTES:

[1] 98% means I find it inconceivable that any current system would drop
    the whole message on the floor or change it in any way.

    2% means I find it inconceivable that any current system would
    construct a deliverable reply.

[2] It is known that some systems DO insert a (supposedly plausible) From
    field, and these are quite likely to be replyable (though not
    necesarily to the original sender).

[3] It is possible that a syntactically invalid from field would be
    dropped, or changed into some X-field. That is probably an acceptable
    behaviour, particularly if it only happens at the final MUA.

[4] If DNS failures are not cached (as they should be), then there will be
    some load on the root servers.

[5] The treatment of supposedly 'safe' IPs, such as [0,0,0,0],
    [127,255,255,255] might be somewhat bizarre. [127,0,0,1] might even be
    delivered, but hopefully to the person replying.

[6] I think a core dump on attempting to reply is most unlikely, but the
    error message produced might be less than helpful.

Clearly, this analysis shows that no solution is going to be perfect (but
we knew that already). OTOH, it does seem to shown that #1 has worse
problems than the others.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 28 Feb 2005 12:06:57 -0500
From: Keith Moore <[email-masked]>
To: Tony Finch <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

> > > I agree that it is wrong to reject null return paths altogether. However
> > > your suggestion is incompatible with backscatter detection systems (such
> > > as BATV or Signed Envelope Sender) which assume that null return paths are
> > > only used for bounce messages.
> >
> > those systems are operating on false assumptions.  it's never been the case
> > that <> was limited to bounce messages.
> 
> Yes. It'll be interesting to discover the extent of the interop problems :-)
> 
> I think we're going to have to tighten up the rules about null return
> paths and bounces, because of the backscatter problem. 

could you explain the backscatter problem?  I'm not sure what you're referring
to.

Keith

Date: Mon, 28 Feb 2005 16:48:12 +0000
From: Tony Finch <[email-masked]>
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

On Mon, 28 Feb 2005, Bruce Lilly wrote:
>
> > - if you can trace the message's sender, you can probably
> > identify its author
>
> I disagree with the premise.  There are ways for an anonymous
> author to arrange for a sender to send a message w/o knowing
> the author's identity (some sort of drop-box, for example).

That's essentially what anonymous remailers do.

> > - why can't people just use pseudonymity instead?
>
> It's unclear exactly what you mean

In the context of email a pseudonymous email address would be one that
is valid but not traceable to any particular real-world person. The old
[email-masked] addresses are a good example.

> > Mixmaster
>
> Tracing back (given a sufficient quantity of guns and/or lawyers
> or wiretaps) seems feasible (with the proviso that it traces senders,
> not necessarily authorship).

Yes, though mixmaster makes this considerably harder than the erzats
anonymity you are suggesting.

> It's unclear what you mean by "a new message header"; if you mean
> eliminating transport trace fields, that's one thing, if you mean
> Subject, Comments, From, To, Cc, Date, and other originator fields that
> are an integral part of the message, then the message integrity is
> essentially destroyed (N.B. body content is unnecessary in a message;
> the entire message may consist exclusively of header fields).

The next-hop message header is derived from the decrypted contents of the
original message - i.e. each mixmaster hop is an unencapsulation of a
message wrapped in an onion-like manner. Intermediate mixmaster hops do
not know about the originator or final recipients of the message.

> > All of the crypto keys belong to recipients, not senders.
>
> So a message to multiple recipients isn't possible (as opposed
> to multiple messages, each to a single recipient)?

Perhaps you should read up on the basics of public key crypto. The usual
way that a message is encrypted for multiple recipients is that the
message is encrypted with a symmetric block cipher (asymmetric crypto is
too slow for bulk applications), and the key (which works for both
encryption and decryption) is encrypted once for each recipient with that
recipient's public key.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
LUNDY FASTNET IRISH SEA SHANNON: WEST OR NORTHWEST 3 OR 4 INCREASING 5 TO 7.
RAIN OR SHOWERS. MODERATE OR GOOD.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 11:00:40 -0500
Cc: Tony Finch <[email-masked]>
Sender: [email-masked]

On Mon February 28 2005 10:07, Tony Finch wrote:

> So given that obfuscating the From: field does not provide proper
> anonymity

Let's be clear about terms:
lack of a From field is not obfuscation; it is anonymity
a field with a null addr-spec ("<>") is not obfuscated; it is
 another sort of anonymity
changing "[email-masked]" to "[email-masked]" or "[email-masked]"
  or equivalent is obfuscation
using a valid address belonging to somebody else is not obfuscation,
  it is impersonation
using a made-up addr-spec guaranteed to belong to nobody, e.g.
  [email-masked], might qualify as obfuscation
using a loopback indications such as "localhost" or equivalents is
  not obfuscation.

> - if you can trace the message's sender, you can probably 
> identify its author

I disagree with the premise.  There are ways for an anonymous
author to arrange for a sender to send a message w/o knowing
the author's identity (some sort of drop-box, for example).

> - why can't people just use pseudonymity instead?

It's unclear exactly what you mean -- "George Orwell" is
  a pseudonym (for the late Eric Arthur Blair) -- but
  while something like that might be usable in a display name
  (phrase associated with an angle-addr or group), it doesn't
  seem applicable to the addr-spec.

> Mixmaster messages are encrypted with the remailer's public key. The
> remailer decrypts the message using its private key and sends it on,
> with a new message header (thus with no trace back to the message's
> origin). This can be repeated via a number of hops in order to make
> tracing the message more difficult, since tracing requires a compromise of
> the anonymity provided by all the remailers.

Tracing back (given a sufficient quantity of guns and/or lawyers
or wiretaps) seems feasible (with the proviso that it traces senders,
not necessarily authorship).  It's unclear what you mean by "a new
message header"; if you mean eliminating transport trace fields,
that's one thing, if you mean Subject, Comments, From, To, Cc, Date,
and other originator fields that are an integral part of the message,
then the message integrity is essentially destroyed (N.B. body
content is unnecessary in a message; the entire message may consist
exclusively of header fields).

> All of the crypto keys belong to recipients, not senders.

So a message to multiple recipients isn't possible (as opposed
to multiple messages, each to a single recipient)?

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 10:35:38 -0500
Sender: [email-masked]

On Sun February 27 2005 15:52, Frank Ellermann wrote:

> Possible objections:  
> 
> 1 - TLD .invalid
[...]
> 5 - The DNS root servers would be flooded by stupid queries.

6. Ambiguity to non-English and English speakers alike:
http://dictionary.reference.com/search?q=invalid
Note that the majority of definitions have nothing to do
with validity.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 10:30:08 -0500
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Mon February 28 2005 09:29, Keith Moore wrote:

> nobody cares about 733 anymore.  most things in use today were written 
> with 822 in mind.

Messages tend to stick around for a long time in archives, and
may occasionally need to be read.  RFCs are not carved in
granite; they're amendable and have been amended (e.g. 1123
amends 822's date-time syntax, among other things).  For that
matter, when a 2822 successor reaches full Standard status, 822
will be of as much interest as 733 now is -- as an historical
document. Note that Return-Path theoretically can be an empty
(bracketed) path, as provided for by RFC 1123. But it was not
always so (i.e. that has changed), and a careful read of RFC 2821
reveals that it lacks provision for an empty return path in the
Return-Path field ABNF.  The point is that if there is sufficient
cause for a modification, there's no absolute barrier to
such modification.  A secondary point re. 733 is that given
a need to make some modification to syntax, a modification
that has some established precedent may be preferable to one
which lacks any operational history.

> > HTTP and
> > SIP are mentioned because they refer back to the (RFC 822)
> > defined syntax of the From field; any change to that syntax
> > will affect those protocols as well as the Internet Message
> > Format.
> 
> no.  they refer to 822, not to some newer specification.  822 hasn't 
> changed, and won't change.

The published text document is static, but it can be amended by
several established mechanisms: the RFC Editor errata page, and
through amendment by subsequent Standards Track RFCs (e.g. 1123). 

> > When you propose such syntax changes, you must consider
> > the effects on all protocols affected by the change.
> 
> if they are truly affected by the change, yes.  but it's not as if HTTP 
> and SIP use email as a substrate.

They use the RFC 822 From field as defined by RFC 822 as amended
by the mechanisms discussed above.  It would be irresponsible to
introduce a syntax change without considering the consequences
for HTTP and SIP.

The intent for the draft under discussion is to become a Standards
Track RFC amending RFCs 822 and 2822.  The approach of making the
from field optional in the Internet Message Format message header
does not change the syntax of the From field, and does not affect
either HTTP or SIP (because those protocols define their respective
formats and determine which fields are permitted and/or required
for those formats, independently of the Internet Message Format).

> Everything is 
> broken in some way or another.

That's been my premise from the start.

> I still think that it makes more sense  
> to use a domain literal than a domain.

We're agreed that a domain name has problems.  I'm not thrilled
about a hypothetical domain literal which has magic properties
unfathomable to casual users -- academic since there doesn't
appear to be a suitable literal anyway.

> I still think it will break  
> fewer things to have a From field with an actual address than to have 
> no From field or From:<>.

I don't think a definitive statement is possible. Syntax changes
would require hearing from HTTP and SIP folks.  I've yet to hear
of a single specific thing that breaks badly in the absence of
a From field in a message -- aside from systems that lack any
provision for anonymity in the first place.

Date: Mon, 28 Feb 2005 15:07:28 +0000
From: Tony Finch <[email-masked]>
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

On Mon, 28 Feb 2005, Bruce Lilly wrote:
>
> Trace fields are used with SMTP, not necessarily with other
> protocols that use the Internet Message Format.  IP address
> alone is not sufficient to identify an individual (esp. w/
> DHCP, public access points, etc.) and might identify the sender
> (i.e. the person who caused a message to be entered into the
> transport stream) who may be different from the message author
> (i.e. the person who composed the message).

So given that obfuscating the From: field does not provide proper
anonymity - if you can trace the message's sender, you can probably
identify its author - why can't people just use pseudonymity instead?

The following points demonstrate a lack of understanding of the mixmaster
anonymizing remailers and of public key cryptography:

> > Why not propose a specification for properly strong anonymity based on
> > mixmaster or some other multi-stage cryptographic system?
>
> While the message body content (and originator-specified header
> fields if encapsulated in a MIME message/rfc822 wrapper) can be
> encrypted by existing S/MIME and PGP/MIME methods (adding more
> MIME wrappers), there are still some issues:
> 1. If SMTP is used, SMTP trace fields in the (unencryptable)
>    outer message header will still leave a trail to the sender
>    (but not necessarily the author, as noted above).
> 2. Decrypting requires knowledge of an encryption key, which
>    implies knowing who sent the message.
> 3. While a hypothetical well-known public key unassociated with
>    an individual would allow decrypting the message, once done
>    (by anybody), the unencrypted content is available; if it
>    contains some indication of authorship (viz. a From header
>    field), there is no anonymity.

Mixmaster messages are encrypted with the remailer's public key. The
remailer decrypts the message using its private key and sends it on,
with a new message header (thus with no trace back to the message's
origin). This can be repeated via a number of hops in order to make
tracing the message more difficult, since tracing requires a compromise of
the anonymity provided by all the remailers.

Nothing in the mixmaster system refers to the message's originator, except
for the first hop trace headers which are discarded by the first remailer.
All of the crypto keys belong to recipients, not senders.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
FAEROES SOUTHEAST ICELAND: NORTH 6 TO GALE 8 INCREASING SEVERE GALE 9 FOR A
TIME. WINTRY SHOWERS. MODERATE OR GOOD.

Date: Mon, 28 Feb 2005 14:57:14 +0000
From: Tony Finch <[email-masked]>
To: Keith Moore <[email-masked]>
cc: Bruce Lilly <[email-masked]>, ietf-822 list <[email-masked]>
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

On Mon, 28 Feb 2005, Keith Moore wrote:

> > I agree that it is wrong to reject null return paths altogether. However
> > your suggestion is incompatible with backscatter detection systems (such
> > as BATV or Signed Envelope Sender) which assume that null return paths are
> > only used for bounce messages.
>
> those systems are operating on false assumptions.  it's never been the case
> that <> was limited to bounce messages.

Yes. It'll be interesting to discover the extent of the interop problems :-)

I think we're going to have to tighten up the rules about null return
paths and bounces, because of the backscatter problem. If anyone has any
ideas for dealing with it that are better that BATV-alikes then I'm all
ears. Note that the other email authentication systems don't solve the
problem - or if they claim to do so they assume that email delivery is
single-hop and that malware does not retry and that everyone will deploy
the new technology, all of which are wrong.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SHETLAND ISLES: NORTH 4 OR 5 BACKING WEST 3 OR 4 IN AFTERNOON AND THEN
VARIABLE 3 OR 4 IN EVENING BEFORE BECOMING NORTHEAST 5 OR 6 OVERNIGHT.
OCCASIONAL RAIN OR SLEET TODAY, WINTRY SHOWERS TONIGHT. MODERATE OR GOOD.
MAINLY MODERATE.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 09:45:40 -0500
Cc: Tony Finch <[email-masked]>
Sender: [email-masked]

On Mon February 28 2005 09:07, Tony Finch wrote:

> This idea of obfuscating the message header does not provide proper
> anonymity, since the trace fields will still contain the IP address of the
> sender which is enough to identify them.

Trace fields are used with SMTP, not necessarily with other
protocols that use the Internet Message Format.  IP address
alone is not sufficient to identify an individual (esp. w/
DHCP, public access points, etc.) and might identify the sender
(i.e. the person who caused a message to be entered into the
transport stream) who may be different from the message author
(i.e. the person who composed the message).

> Why isn't pseudonimity enough? 
> Why not propose a specification for properly strong anonymity based on
> mixmaster or some other multi-stage cryptographic system?

While the message body content (and originator-specified header
fields if encapsulated in a MIME message/rfc822 wrapper) can be
encrypted by existing S/MIME and PGP/MIME methods (adding more
MIME wrappers), there are still some issues:
1. If SMTP is used, SMTP trace fields in the (unencryptable)
   outer message header will still leave a trail to the sender
   (but not necessarily the author, as noted above).
2. Decrypting requires knowledge of an encryption key, which
   implies knowing who sent the message.
3. While a hypothetical well-known public key unassociated with
   an individual would allow decrypting the message, once done
   (by anybody), the unencrypted content is available; if it
   contains some indication of authorship (viz. a From header
   field), there is no anonymity.

As I see it, a necessary first step is provision for anonymity
in the message format.

Cc: Bruce Lilly <[email-masked]>, ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 09:32:57 -0500
To: Tony Finch <[email-masked]>
Sender: [email-masked]

>> in the envelope I recommend that MAIL FROM:<> be used.  a lot of
>> MTAs reject it these days, but they are clearly broken.
>
> I agree that it is wrong to reject null return paths altogether. 
> However
> your suggestion is incompatible with backscatter detection systems 
> (such
> as BATV or Signed Envelope Sender) which assume that null return paths 
> are
> only used for bounce messages.

those systems are operating on false assumptions.  it's never been the 
case that <> was limited to bounce messages.

> This idea of obfuscating the message header does not provide proper
> anonymity, since the trace fields will still contain the IP address of 
> the
> sender which is enough to identify them. Why isn't pseudonimity enough?
> Why not propose a specification for properly strong anonymity based on
> mixmaster or some other multi-stage cryptographic system?

I don't view this as trying to provide true anonymity, in the sense 
that the message cannot be traced. I view it as trying to provide a way 
to say "we don't know who actually sent this message, or whether he 
even has an email address".

Keith

From: Keith Moore <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 09:29:15 -0500
To: ietf-822 <[email-masked]>
Sender: [email-masked]

>>>> in practice, it has the semantics of "nowhere", or "you can't get
>>>> there
>>>> from here, no matter where 'here' is."
>>>
>>> If I repeat Claus Assmann's experiment using 127.0.0.1 (the 
>>> loopback),
>>> what I get has semantics of "here".
>>
>> and if you do it with 0.0.0.0 or 127.0.0.2, you get the semantics of
>> "you can't get there from here"
>
> No, I get what Claus and Russ got, semantics of "here":

so do I, now.  and when I tried earlier I made the mistake of not 
carefully recording what I did and what I got, so now I can't be sure 
what happened.

>> <> is illegal syntax, and some things do check.  even those things 
>> that
>> do parse <> might end up extracting a null string, which could break
>> scripts.  and we're talking about email here, not HTTP or SIP.
>
> It's illegal now, but was legal under RFC 733.

nobody cares about 733 anymore.  most things in use today were written 
with 822 in mind.

> HTTP and
> SIP are mentioned because they refer back to the (RFC 822)
> defined syntax of the From field; any change to that syntax
> will affect those protocols as well as the Internet Message
> Format.

no.  they refer to 822, not to some newer specification.  822 hasn't 
changed, and won't change.  HTTP and SIP are separate protocols (with 
their own baggage) and are evolving separately from the email protocols 
(and its baggage).  which is as it should be.

> When you propose such syntax changes, you must consider
> the effects on all protocols affected by the change.

if they are truly affected by the change, yes.  but it's not as if HTTP 
and SIP use email as a substrate.

>> overall, my preference is for something@[0.0.0.0], maybe something
>> short like anon@[0.0.0.0] or something like 0@[0.0.0.0]
>
> But Keith, by definition (RFC 3330) 0.0.0.0 doesn't mean anything
> like "anonymous", it means ""this' host on 'this' network", and
> that has been confirmed in practice by experiment on three separate
> systems.  Your assertion otherwise seems to be rather unconvincing.

Frankly at this point I'm not sure what to recommend.   Everything is 
broken in some way or another.  I still think that it makes more sense 
to use a domain literal than a domain.  I still think it will break 
fewer things to have a From field with an actual address than to have 
no From field or From:<>.  But I share at least some of your dislike of 
connecting to the loopback address (or something that acts like the 
loopback address).  And my gut sense is that a lot of mail parsers will 
fail if an IPv6 domain literal like [ipv6:::0] is used.

Keith

Date: Mon, 28 Feb 2005 14:07:18 +0000
From: Tony Finch <[email-masked]>
To: Keith Moore <[email-masked]>
cc: Bruce Lilly <[email-masked]>, ietf-822 list <[email-masked]>
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

On Mon, 28 Feb 2005, Keith Moore wrote:
>
> in the envelope I recommend that MAIL FROM:<> be used.  a lot of
> MTAs reject it these days, but they are clearly broken.

I agree that it is wrong to reject null return paths altogether. However
your suggestion is incompatible with backscatter detection systems (such
as BATV or Signed Envelope Sender) which assume that null return paths are
only used for bounce messages. These systems insert a cookie in the return
path of outgoing messages and require that bounce messages are sent to an
address containing a valid cookie.

This idea of obfuscating the message header does not provide proper
anonymity, since the trace fields will still contain the IP address of the
sender which is enough to identify them. Why isn't pseudonimity enough?
Why not propose a specification for properly strong anonymity based on
mixmaster or some other multi-stage cryptographic system?

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
NORTH UTSIRE SOUTH UTSIRE: CYCLONIC BECOMING NORTH OR NORTHEAST 5 OR 6. RAIN
OR SLEET THEN WINTRY SHOWERS. MODERATE OR GOOD.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 08:56:33 -0500
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Mon February 28 2005 08:18, Keith Moore wrote:

> >> in practice, it has the semantics of "nowhere", or "you can't get 
> >> there
> >> from here, no matter where 'here' is."
> >
> > If I repeat Claus Assmann's experiment using 127.0.0.1 (the loopback),
> > what I get has semantics of "here".
> 
> and if you do it with 0.0.0.0 or 127.0.0.2, you get the semantics of 
> "you can't get there from here"

No, I get what Claus and Russ got, semantics of "here":

marty:/src/abnff # telnet 0.0.0.0 smtp
Trying 0.0.0.0...
Connected to 0.0.0.0.
Escape character is '^]'.
220 mail.blilly.com ESMTP Sendmail /8.13.1/mail.blilly.com sendmail.mc.mail 1.18 2004/05/15 07:23:45: Mon, 28 Feb 2005 08:34:20 -0500
quit
221 2.0.0 mail.blilly.com closing connection
Connection closed by foreign host.
marty:/src/abnff # telnet 127.0.0.2 smtp
Trying 127.0.0.2...
Connected to 127.0.0.2.
Escape character is '^]'.
220 mail.blilly.com ESMTP Sendmail /8.13.1/mail.blilly.com sendmail.mc.mail 1.18 2004/05/15 07:23:45: Mon, 28 Feb 2005 08:34:42 -0500
quit
221 2.0.0 mail.blilly.com closing connection
Connection closed by foreign host.
marty:/src/abnff # uname -a
Linux marty 2.6.8-24.11-default #1 Fri Jan 14 13:01:26 UTC 2005 i686 i686 i386 GNU/Linux

So that's SunOS and two versions of Linux kernels... On what
sort of system are you getting different semantics for 0.0.0.0
and 127.0.0.2?
 
> >> it's a backward compatibility issue.  MUAs and MTAs that know about 
> >> the
> >> convention are not the problem, since it's easy for them to recognize
> >> such addresses and treat them accordingly. the problem is making this
> >> work with MUAs and MTAs that don't know about the convention.
> >
> >> hey, I'd be fine with anonymous@[ipv6:::0]
> >>
> >> or perhaps even anonymous@[no-such-network:]
> >
> > And the backwards compatibility of either is ... ?
> 
> hard to say.  my gut sense is that an IPv4 domain literal is less 
> likely to break things than an IPv6 domain literal.

One is an RFC 2821 IPv6 literal, the other is an as-yet undefined
and unregistered literal that fits 2821 "General" literal syntax.

> <> is illegal syntax, and some things do check.  even those things that 
> do parse <> might end up extracting a null string, which could break 
> scripts.  and we're talking about email here, not HTTP or SIP.

It's illegal now, but was legal under RFC 733. HTTP and
SIP are mentioned because they refer back to the (RFC 822)
defined syntax of the From field; any change to that syntax
will affect those protocols as well as the Internet Message
Format.  When you propose such syntax changes, you must consider
the effects on all protocols affected by the change.
 
> overall, my preference is for something@[0.0.0.0], maybe something 
> short like anon@[0.0.0.0] or something like 0@[0.0.0.0]

But Keith, by definition (RFC 3330) 0.0.0.0 doesn't mean anything
like "anonymous", it means ""this' host on 'this' network", and
that has been confirmed in practice by experiment on three separate
systems.  Your assertion otherwise seems to be rather unconvincing.

> It is easily identified by 
> humans as a "special" address that probably doesn't correspond to a 
> real user.

I suggest that you try an experiment: ask a random sample of
people outside of the field of computer science what "0@[0.0.0.0]"
means.  I doubt that any will say anything like "anonymous"; I
suspect most will say that they don't know. [moot point, since
the real semantics point to the local host]
 
> something@[127.0.0.2] would also work

As a way of addressing the mailbox "something" known to the
local host, yes, but that's not what's desired.

> either of those would be better than [127.0.0.1]

All three literals have the same meaning.

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: [0.0.0.0]
Date: Mon, 28 Feb 2005 08:21:57 -0500
To: Russ Allbery <[email-masked]>
Sender: [email-masked]

>>> nice try.  but in practice if you try to connect to 0.0.0.0 you'll 
>>> get
>>> an error, which is what we want to happen.
>
>> That depends on your OS:
>
>> mine$ uname -a
>> SunOS mine 5.9 Generic_117171-12 sun4u sparc SUNW,Ultra-2
>> mine$ telnet 0.0.0.0 25
>> Trying 0.0.0.0...
>> Connected to 0.0.0.0.
>> Escape character is '^]'.
>> 220 mine ESMTP Sendmail 8.13.1/8.13.1; Mon, 28 Feb 2005 05:48:09 
>> +0100 (MET)
>
> windlord:~> uname -a
> Linux windlord 2.4.26 #1 Tue Apr 20 16:17:15 PDT 2004 i686 GNU/Linux
> windlord:~> telnet 0.0.0.0 smtp
> Trying 0.0.0.0...
> Connected to 0.0.0.0 (0.0.0.0).
> Escape character is '^]'.
> 220 windlord.stanford.edu ESMTP
> quit
> 221 windlord.stanford.edu
> Connection closed by foreign host.

sigh.  I stand corrected.  (I thought I had tried these cases, but must 
have tried something else.)

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 08:18:51 -0500
To: Bruce Lilly <[email-masked]>
Sender: [email-masked]

>> you'll get
>> an error, which is what we want to happen.
>
> No, what we want to have is either a clear indication of lack
> of an author mailbox (preferred, if practical), or a lack of
> any indication of an author mailbox.

they're not mutually exclusive.  one is the goal, the other is a 
necessary condition.

>>> Yes, the loopback connection is supposed to be internal to the IP
>>> implementation, and not be translated to any physical network.  That
>>> still doesn't have semantics of "anonymous" or of "nonexistent"; it
>>> has semantics of "here".
>>
>> in practice, it has the semantics of "nowhere", or "you can't get 
>> there
>> from here, no matter where 'here' is."
>
> If I repeat Claus Assmann's experiment using 127.0.0.1 (the loopback),
> what I get has semantics of "here".

and if you do it with 0.0.0.0 or 127.0.0.2, you get the semantics of 
"you can't get there from here"

>> it's a backward compatibility issue.  MUAs and MTAs that know about 
>> the
>> convention are not the problem, since it's easy for them to recognize
>> such addresses and treat them accordingly. the problem is making this
>> work with MUAs and MTAs that don't know about the convention.
>
>> hey, I'd be fine with anonymous@[ipv6:::0]
>>
>> or perhaps even anonymous@[no-such-network:]
>
> And the backwards compatibility of either is ... ?

hard to say.  my gut sense is that an IPv4 domain literal is less 
likely to break things than an IPv6 domain literal.

>> no, but it's the most important technical constraint on the convention
>> for an anonymous address.  see above.
>
> "anonymous address" is an oxymoron.

no more so than "anonymous" is an oxymoron when used to fill in a 
"name" blank.   the meaning is clear, even if technically its a 
contradiction.

> "<>" (i.e. no mailbox) seems
> less problematical (even with HTTP, SIP compatibility issues) than
> some of the recent proposals.  At least "<>" has some sort of
> precedent (in Return-Path).

<> is illegal syntax, and some things do check.  even those things that 
do parse <> might end up extracting a null string, which could break 
scripts.  and we're talking about email here, not HTTP or SIP.

>> Too many things extract data from the From field.
>
> If the field isn't there, they can't extract anything from it.

and thus may exhibit anomalous behavior.

---

overall, my preference is for something@[0.0.0.0], maybe something 
short like anon@[0.0.0.0] or something like 0@[0.0.0.0] that isn't 
dependent on English and doesn't expect people to know how to spell 
anonymous (ah, for the days when everyone used ftp clients).  I think 
it will be correctly parsed by most M*As.  It is easily identified by 
humans as a "special" address that probably doesn't correspond to a 
real user.

something@[127.0.0.2] would also work, but isn't as easily recognized.  
either of those would be better than [127.0.0.1] or an ipv6 literal.

  <> and omitting From entirely are my least favorite alternatives.

most of this is informed by my sense of what M*As are likely to be able 
to tolerate, which may be different than others' sense.

Keith

Cc: Bruce Lilly <[email-masked]>, ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 08:02:06 -0500
To: Tony Finch <[email-masked]>
Sender: [email-masked]

>> if anonymous@[] doesn't quite work, there are similar alternatives 
>> worth
>> considering.  among them are: anonymous@[0.0.0.0], 
>> anonymous@[127.0.0.1],
>> anonymous@[127.255.255.255], anonymous@[::0], and anonymous@[::1].
>
> These might be OK in the message header. In the envelope they are 
> likely
> to be rejected as invalid, because many sites put restrictions on the 
> use
> of domain literals, and checks on envelope addresses tend to be more
> thorough. In addition to that, many sites reject domains that do not 
> refer
> to globally routable addresses.

exactly so.  in the envelope I recommend that MAIL FROM:<> be used.  a 
lot of MTAs reject it these days, but they are clearly broken.

Date: Mon, 28 Feb 2005 11:11:35 +0000
From: Tony Finch <[email-masked]>
To: Keith Moore <[email-masked]>
cc: Bruce Lilly <[email-masked]>, ietf-822 list <[email-masked]>
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

On Sun, 27 Feb 2005, Keith Moore wrote:
>
> if anonymous@[] doesn't quite work, there are similar alternatives worth
> considering.  among them are: anonymous@[0.0.0.0], anonymous@[127.0.0.1],
> anonymous@[127.255.255.255], anonymous@[::0], and anonymous@[::1].

These might be OK in the message header. In the envelope they are likely
to be rejected as invalid, because many sites put restrictions on the use
of domain literals, and checks on envelope addresses tend to be more
thorough. In addition to that, many sites reject domains that do not refer
to globally routable addresses.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
LOUGH FOYLE TO CARLINGFORD LOUGH: SOUTHWEST VEERING NORTHWEST 4 OR 5 TODAY,
THEN BACKING WEST 5 OR 6 IN EVENING BEFORE VEERING NORTH OR NORTHWEST 4 OR 5
TONIGHT. RAIN TODAY, WINTRY SHOWERS TONIGHT. MODERATE OR GOOD. SLIGHT BECOMING
MODERATE LATER TODAY AND THEN MAINLY ROUGH TONIGHT.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 01:30:47 -0500
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Sun February 27 2005 23:18, Keith Moore wrote:
> you'll get 
> an error, which is what we want to happen.

No, what we want to have is either a clear indication of lack
of an author mailbox (preferred, if practical), or a lack of
any indication of an author mailbox.
 
> > Yes, the loopback connection is supposed to be internal to the IP
> > implementation, and not be translated to any physical network.  That
> > still doesn't have semantics of "anonymous" or of "nonexistent"; it
> > has semantics of "here".
>
> in practice, it has the semantics of "nowhere", or "you can't get there 
> from here, no matter where 'here' is."

If I repeat Claus Assmann's experiment using 127.0.0.1 (the loopback),
what I get has semantics of "here".
 
> it's a backward compatibility issue.  MUAs and MTAs that know about the 
> convention are not the problem, since it's easy for them to recognize 
> such addresses and treat them accordingly. the problem is making this 
> work with MUAs and MTAs that don't know about the convention.

> hey, I'd be fine with anonymous@[ipv6:::0]
> 
> or perhaps even anonymous@[no-such-network:]

And the backwards compatibility of either is ... ?
 
> no, but it's the most important technical constraint on the convention 
> for an anonymous address.  see above.

"anonymous address" is an oxymoron.  "<>" (i.e. no mailbox) seems
less problematical (even with HTTP, SIP compatibility issues) than
some of the recent proposals.  At least "<>" has some sort of
precedent (in Return-Path).

> Too many things extract data from the From field.

If the field isn't there, they can't extract anything from it.
Less of a problem than extracting something and misinterpreting
it (or bombarding DNS with cruft, or misdirecting responses, or
crashing...).

To: ietf-822 list <[email-masked]>
Subject: Re: [0.0.0.0]
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sun, 27 Feb 2005 22:22:22 -0800
Sender: [email-masked]

Claus Assmann <[email-masked]> writes:
> On Sun, Feb 27, 2005, Keith Moore wrote:

>> nice try.  but in practice if you try to connect to 0.0.0.0 you'll get 
>> an error, which is what we want to happen.

> That depends on your OS:

> mine$ uname -a
> SunOS mine 5.9 Generic_117171-12 sun4u sparc SUNW,Ultra-2
> mine$ telnet 0.0.0.0 25
> Trying 0.0.0.0...
> Connected to 0.0.0.0.
> Escape character is '^]'.
> 220 mine ESMTP Sendmail 8.13.1/8.13.1; Mon, 28 Feb 2005 05:48:09 +0100 (MET)

windlord:~> uname -a
Linux windlord 2.4.26 #1 Tue Apr 20 16:17:15 PDT 2004 i686 GNU/Linux
windlord:~> telnet 0.0.0.0 smtp
Trying 0.0.0.0...
Connected to 0.0.0.0 (0.0.0.0).
Escape character is '^]'.
220 windlord.stanford.edu ESMTP
quit
221 windlord.stanford.edu
Connection closed by foreign host.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Mon, 28 Feb 2005 15:45:49 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 <[email-masked]>
Sender: [email-masked]

On Feb 27 2005, Bruce Lilly wrote:
> On Thu February 24 2005 21:09, Laird Breyer wrote:
> 
> > [...] users will
> > never have the kind of control over receipt of anonymous messages that
> > Bruce is positing.
> 
> I'm not positing any sort of control by individual users; rather I
> am stating that organizational entities that have a need to accept
> anonymous information -- and which have provisions for doing so now
> by means other than Internet Messages -- will need to take appropriate
> steps to ensure that they can do so.  That may include careful
> (administrative, rather than individual recipient) selection and
> configuration of filters.

That's the kind of thing I'm saying can't be assumed in the presence
of a filter. I claim that the trend in spam filters is to be opaque to
users and administrators (super-users) alike, for both commercial reasons and
because of the increasing complexity under the hood (expanded in the
other message I sent recently). 

Certainly, an organization which does its own mail handling can always
set up completely unfiltered addresses, and will have to painfully
sift through the junk. But if the addresses are filtered (possibly
upstream), then there can be no guarantees, since even the people who
build the filters don't understand fully how they operate.

For the purpose of discussing your draft, I doubt all this matters much.
Spam filters already operate outside of the "mail transport law" ;-),
and will continue to pick up clues they think are useful, regardless of
RFC semantics. Wondering whether they wil honour blah@[] is like
wondering whether they will honour a loophole. If spammers notice that
blah@[] gets through, then they'll all use those kinds of addresses and 
filters will block them. 

-- 
Laird Breyer.

Date: Sun, 27 Feb 2005 20:59:03 -0800
From: Claus Assmann <[email-masked]>
To: ietf-822 list <[email-masked]>
Subject: Re: [0.0.0.0] (was: MTS transparency and anonymity)
Mail-Followup-To: Claus Assmann <[email-masked]>, ietf-822 list <[email-masked]>
Sender: [email-masked]

On Sun, Feb 27, 2005, Keith Moore wrote:

> nice try.  but in practice if you try to connect to 0.0.0.0 you'll get 
> an error, which is what we want to happen.

That depends on your OS:

mine$ uname -a
SunOS mine 5.9 Generic_117171-12 sun4u sparc SUNW,Ultra-2
mine$ telnet 0.0.0.0 25
Trying 0.0.0.0...
Connected to 0.0.0.0.
Escape character is '^]'.
220 mine ESMTP Sendmail 8.13.1/8.13.1; Mon, 28 Feb 2005 05:48:09 +0100 (MET)

mine# date | /usr/lib/sendmail -Am -v 'user@[0.0.0.0]'
user@[0.0.0.0]... Connecting to [0.0.0.0] via smtp8...
220 mine ESMTP Sendmail 8.13.1/8.13.1; Mon, 28 Feb 2005 05:48:19 +0100 (MET)

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 23:19:02 -0500
Sender: [email-masked]

On Sun February 27 2005 22:38, Frank Ellermann wrote:
> 
> Bruce Lilly wrote:
> 
> > We discussed syntax options months ago, and your earlier
> > message prompted a detailed analysis of issues (it was
> > clear that an empty literal wasn't permitted by 822
> 
> If that's clear then it's not in the domain-literal syntax:

It seemed clear from the normative text reference to RFC 820.
[It's also noteworthy that use of domain literals is strongly
discouraged, accompanied by SCREAMING ALL CAPS in 822].
 
> The semantics later says "its contents must conform with the
> needs of the sub-domain in which it is interpreted".  If @[]
> is interpreted as "nowhere", then that's fine.

Possibly. But there's the 2821 ABNF that doesn't permit that.
And any change to syntax that leaves an address field with
semantics of nowhere conflicts with 2821 sect. 3.8.4.
 
> > simply making the From field optional
> 
> Your "updates RfC" section could be a new record.  Bye, Frank

I'm not sure what you mean; making the field optional will
update 822 and 2822.  Changing syntax would at minimum add
2821, 2616, and 3261 to the list (probably others as well).

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 23:18:17 -0500
To: Bruce Lilly <[email-masked]>
Sender: [email-masked]

>>>> if anonymous@[] doesn't quite work, there are similar alternatives
>>>> worth considering.  among them are: anonymous@[0.0.0.0],
>>>
>>> No, see RFC 3330; that's a local net.
>>
>> No, see RFC 3330; it's only intended for use as a _source_ address.
>> Trying to connect to the address fails on every platform I know of.
>
> Well, we're talking about a mailbox in the From field, which has
> semantics of a source if anything.

nice try.  but in practice if you try to connect to 0.0.0.0 you'll get 
an error, which is what we want to happen.

>> note that there are a lot fewer TCP/IP stacks in wide use than there
>> are mail-handling programs.  it's much easier to be confident about 
>> the
>> behavior of an IP stack.
>
> Maybe. I recall early implementations where 0.0.0.0 is a
> broadcast address; not all of those implementations have vanished.

I've never seen one.  I've seen lots of implementations where the lower 
bits of a subnet were the broadcast address, but that's a different 
issue.

> But that's really a digression from the issue, because no IPv4
> address has semantics of either "anonymous" or "not on the
> Internet"

nor, for that matter, does any domain.  but it's easier to find an IPv4 
address that has desirable properties than it is to find a domain that 
has them.

> I mentioned RFC 3330 because I looked through it in search of
> just such an invalid (IP) address, and could find none guaranteed
> to remain invalid.  And I looked rather closely at 0.0.0.0 and
> 127.0.0.1.

either 0.0.0.0 or 127.0.0.2 seems to work well as far as I can tell.

>> also, if it happens that a dns query for example.com yields ip address
>> X, it should not be assumed that [email-masked] is equivalent to
>> user@[X].
>
> That's not the problem, nor even related to the problem.  The problem
> is that there may be an SMTP (or other mail protocol) server listening
> on the loopback address, and any local-part might be a valid one on
> some such system.

yes, it might happen.  but you can pick a local-part such that it's 
extremely unlikely to happen - less likely to happen than numerous 
other kinds of failures.  even so, 127.0.0.1 would not be my first 
choice.

>>>> anonymous@[127.255.255.255],
>>>
>>> No, also covered by RFC 3330.
>>
>> and if the host correctly implements RFC 3330 ("no addresses within
>> this block should ever appear on any network anywhere")
>
> Yes, the loopback connection is supposed to be internal to the IP
> implementation, and not be translated to any physical network.  That
> still doesn't have semantics of "anonymous" or of "nonexistent"; it
> has semantics of "here".

in practice, it has the semantics of "nowhere", or "you can't get there 
from here, no matter where 'here' is."

>> then this will
>> work just fine, as the host IP stack will return an error when the MTA
>> tries to connect to that address.
>
> You again seem to be focusing on non-replyability, which isn't the
> topic of draft-lilly-from-optional.

it's a backward compatibility issue.  MUAs and MTAs that know about the 
convention are not the problem, since it's easy for them to recognize 
such addresses and treat them accordingly. the problem is making this 
work with MUAs and MTAs that don't know about the convention.

>>>> anonymous@[::0],
>>>> and anonymous@[::1].
>>>
>>> Not valid RFC 2821 syntax.
>>
>> sounds like a bug in RFC 2821.
>
> You're certainly free to make a case for that. The legal 2821 versions
> of IPv6 domain literals would be something like [ipv6:::0]. Yes, it's
> ugly.

hey, I'd be fine with anonymous@[ipv6:::0]

or perhaps even anonymous@[no-such-network:]

>> but if the MTA strictly implements RFC
>> 2821 and rejects the message at RCPT time, that's perfectly okay.
>
> OK for what? For preventing a response (assuming that the message
> gets to a recipient)?  Again, that's not the subject of the draft.

no, but it's the most important technical constraint on the convention 
for an anonymous address.  see above.

>> it  doesn't really matter whether the MTA rejects the address because 
>> the
>> local-part doesn't exist or because the domain has an invalid syntax,
>> as long as it fails.
>
> Are you discussing transport, or the message format?

I'm discussing what it takes to make something like this work in 
practice.  anything that isn't reasonably compatible with existing 
MTAs/MSAs is a non-starter.

> So far as I can see, there is no perfectly clean solution to the
> issues covered in the draft.  However, simply making the From
> field optional -- while that may present some issues -- seems to
> have the least negative impact of the options considered (including
> syntax changes and extension, which present rather more serious
> backwards compatibility issues than making the field optional).

I disagree.  Too many things extract data from the From field.

>>> 2. peeking at the local-part is a layering violation unless
>>>    the domain is yours.
>>
>> for MTAs and UAs, yes.  not for humans.
>
> If there's something there indicating either a domain or a local-part
> within a domain, it fails to provide the desired semantics, viz.
> nothing (i.e. no mailbox).  That itself is not a showstopper, but in
> conjunction with the requirements for gateways, it presents some
> obstacles.

so use the loopback address domain literal.  then it's yours, and you 
can define anonymous addresses however you want :)
(not a serious proposal - but IMHO you're making way too big a deal of 
semantic issues based on strained reading of the specifications and 
giving far too little concern to interoperability with the installed 
base)

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 28 Feb 2005 04:38:57 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 24
Sender: [email-masked]

Bruce Lilly wrote:

> We discussed syntax options months ago, and your earlier
> message prompted a detailed analysis of issues (it was
> clear that an empty literal wasn't permitted by 822

If that's clear then it's not in the domain-literal syntax:

   domain-literal =  "[" *(dtext / quoted-pair) "]"

Of course (2)821 doesn't need to waste time with explanations
how mail is "sent" to or from empty domain-literals, but STD
11 has e.g. route = 1#("@" domain) ":", so it's not that
there was no alternative to using *(dtext /  quoted-pair) in
the domain-literal syntax.

The semantics later says "its contents must conform with the
needs of the sub-domain in which it is interpreted".  If @[]
is interpreted as "nowhere", then that's fine.

> simply making the From field optional

Your "updates RfC" section could be a new record.  Bye, Frank

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: the gap regarding Archived-At
Date: Sun, 27 Feb 2005 22:13:06 -0500
Cc: Martin Duerst <[email-masked]>
Sender: [email-masked]

On Fri February 18 2005 02:48, Martin Duerst wrote:

>  >Clearly, the proposed Archived-At field applies to a specific message
>  >(like Message-ID), and should be in the same category as those
>  >specifically mentioned.  However, unlike RFC 3798 and its
>  >predecessor, the archived-at draft does not amend the RFC2046
>  >list of fields which are reassembled from the encapsulated message
>  >header.
> 
> Well, I could just say that it updates RFC 2046. But I'm not
> sure this is necessary.
> 
>  >Consequently, Archived-At would be treated like Comments;
>  >when fragmenting a message, the field would be copied to the
>  >first fragment message header, and would be reassembled from there
>  >to the reconstituted facsimile of the original unfragmented
>  >message (any such fields within the encapsulated message header
>  >would be ignored, in accordance with the RFC 2046 rules). That
>  >is inconsistent with the semantics proposed for Archived-At, viz. that
>  >it references an archived copy of the specific message in which it
>  >is conveyed. An Archived-At field that originally referred to an
>  >archive of an unfragmented message would therefore be
>  >inappropriately placed in the message header of a different message
>  >(viz. the first fragment message).
> 
> Well, I could just say that for partial messages, the semantics
> may be either-or. Not the best solution, but that would make
> spec an reality match, at least, without any updates to
> infrastructure.
> 
>  >Moreover, it would be
>  >indistinguishable there from an appropriate Archived-At field which
>  >referred to an archive of that (first fragment) message.  And
>  >as fragments may be further fragmented, that issue may be
>  >compounded.
> 
> Ok, so that would mean that the semantics would have to be
> extended even a bit more. What about the following sentence:
> 
> "Note that due to the way header fields are handled with message
> fragmentation and reassembly [RFC2046], there
> may be cases where the URI in an Archived-At message header field
> points to the archived version of an original (unfragmented) message
> from a fragmented part of a message or the archivedversion of a
> fragmented part of a message from the reassembled message."

It's a tough call, and the cause of the difficulty is the way
message/partial fragmentation and reassembly operates.  Long-term
the best solution would be a replacement for message/partial that
doesn't have the same difficulties -- that's on my list of things
to do.

In the meantime, the architecturally-consistent (but backwards
incompatible) resolution would be to amend RFC 2046 in the same
manner as RFCs 2998/3798.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-03.txt
Date: Sun, 27 Feb 2005 22:05:05 -0500
Sender: [email-masked]

On Wed February 23 2005 00:20, Martin Duerst wrote:

> This feature is about having mboxes containing all the mails
> from a given period available, in application/mbox format.

I'm not sure that downloading an entire huge collection of
messages in one gigantic gulp is the best approach.

Some time ago, I gave a rough outline of how a collection
could be made accessible via IMAP, POP, and NNTP using
mostly off-the-shelf software -- but IIRC there were some
issues regarding getting the Archived-At field(s) into the
messages, which seems to be a chicken-vs.-egg problem.

>From a consumer point of view, IMAP provides the richest
set of features, since it supports server-side searches,
etc.  That in turn may reduce bandwidth requirements for
access, since messages of interest can be selected for
transfer (as opposed to copying thousands of messages in
order to gain access to one message of interest).

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>, [email-masked]
Subject: Re: draft-lilly-from-optional-01.txt
Date: Sun, 27 Feb 2005 21:56:40 -0500
Sender: [email-masked]

On Thu February 24 2005 21:09, Laird Breyer wrote:

> [...] users will
> never have the kind of control over receipt of anonymous messages that
> Bruce is positing.

I'm not positing any sort of control by individual users; rather I
am stating that organizational entities that have a need to accept
anonymous information -- and which have provisions for doing so now
by means other than Internet Messages -- will need to take appropriate
steps to ensure that they can do so.  That may include careful
(administrative, rather than individual recipient) selection and
configuration of filters.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Date: Sun, 27 Feb 2005 21:44:03 -0500
Cc: SM <[email-masked]>
Sender: [email-masked]

On Fri February 25 2005 14:56, SM wrote:
> 
> At 07:18 23-02-2005, Bruce Lilly wrote:

> >The draft addresses two separate cases where the From field is
> >inappropriate: 1) where the author has no Internet mailbox (this
> >is not a case of anonymity)

> We rarely if ever see a genuine case one in practice.

Many web-based systems provide for sending mail (other than invoking
a client-side mailer via a mailto URI); many neither "know" nor "care"
whether the author has an Internet mailbox -- most will expect an
author to provide one, but cannot authenticate.

> My suggestion is to have the mailbox of the 
> Sender in the Sender field.

The draft explicitly permits a Sender field, but notes that it is not
required.

> My point is not about server-side authentication.  Message originator 
> fields can and may be used for authentication.  Quoting RFC 822 section 4.4.2:
> 
>    "This field contains the authenticated identity of the AGENT (person, 
> system or
>     process) that sends the message."

That means that the originating MUA (or possibly MSA) is expected to
authenticate the content of the field if it is supplied or generated.
It does not mean that a recipient's MUA should make any assumptions
about the content (which may have been forged, or altered in transit).

End-to-end use of the originator and recipient header fields is meant
for user-to-user communication, like the header of written communications.
While I expect (by analogy) a magnifying glass to show the header of
a written letter, I do not expect it to attempt to analyze the content
of the message.  Likewise, while an MUA should display message header
fields, more elaborate processing usually leads to problems, such as
"helpful" UAs that insist on filling electronic address books with
bogus and incorrect address fields.
 
> On a different note, you wrote "As such, it improves security for message 
> authors who require anonymity." under security considerations.  People 
> requiring anonymity are better served by using anonymous remailers instead 
> of excluding the From field from their message.

At one time there was such a remailer (somewhere in Finland I think),
but as far as I know it has been shut down quite some time ago, and
I'm not aware of any current ones.  In light of various "Big Brother
is watching you" provisions in various countries, I'm not sure that
any such wide-use system is truly feasible at present.

----
War is Peace
Freedom is Slavery
Ignorance is Strength
  -- Eric Arthur Blair, a.k.a. George Orwell, "1984"

1984 arrived a decade and a half late. We apologize for the convenience
while it lasted.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 21:03:59 -0500
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Sun February 27 2005 12:53, Keith Moore wrote:

> >> if anonymous@[] doesn't quite work, there are similar alternatives
> >> worth considering.  among them are: anonymous@[0.0.0.0],
> >
> > No, see RFC 3330; that's a local net.
> 
> No, see RFC 3330; it's only intended for use as a _source_ address.  
> Trying to connect to the address fails on every platform I know of.

Well, we're talking about a mailbox in the From field, which has
semantics of a source if anything.  0.0.0.0/32 means "this" host on
"this" network, and has the same issues as 127.0.0.1.  "Trying to
connect to" has semantics of a destination, which is the opposite
of those of the From field.

> note that there are a lot fewer TCP/IP stacks in wide use than there 
> are mail-handling programs.  it's much easier to be confident about the 
> behavior of an IP stack.

Maybe. I recall early implementations where 0.0.0.0 is a
broadcast address; not all of those implementations have vanished.
But that's really a digression from the issue, because no IPv4
address has semantics of either "anonymous" or "not on the
Internet" -- to those familiar with IP addresses, much less to
casual users to whom such hypothetical IP address literals would
be presented.  Trying to encode such semantics into a numeric
domain literal seem positively user-hostile.
 
> >> anonymous@[127.0.0.1],
> >
> > Absolutely not; that's (the recipient's) local host, and there
> > might well be a mailbox named "anonymous" (or any other legal
> > local-part) there.
> 
> so use an invalid address, rather than a loopback address.

I mentioned RFC 3330 because I looked through it in search of
just such an invalid (IP) address, and could find none guaranteed
to remain invalid.  And I looked rather closely at 0.0.0.0 and
127.0.0.1.

> also, if it happens that a dns query for example.com yields ip address 
> X, it should not be assumed that [email-masked] is equivalent to 
> user@[X].

That's not the problem, nor even related to the problem.  The problem
is that there may be an SMTP (or other mail protocol) server listening
on the loopback address, and any local-part might be a valid one on
some such system.  That is, given user@[X], X may be interpretable as
a local source and "user" may be a valid mailbox within that domain
(using "domain" broadly), and has nothing to do with any domain names,
"example.com" or otherwise.  Rather the issue is that some values of
X, including but not limited to 0.0.0.0 and 127.0.0.1, have the
semantics of "here" regardless what domain names or other IP address
literals also map to either a general or specific instance of "here".

> >> anonymous@[127.255.255.255],
> >
> > No, also covered by RFC 3330.
> 
> and if the host correctly implements RFC 3330 ("no addresses within 
> this block should ever appear on any network anywhere")

Yes, the loopback connection is supposed to be internal to the IP
implementation, and not be translated to any physical network.  That
still doesn't have semantics of "anonymous" or of "nonexistent"; it
has semantics of "here".

> then this will  
> work just fine, as the host IP stack will return an error when the MTA 
> tries to connect to that address.

You again seem to be focusing on non-replyability, which isn't the
topic of draft-lilly-from-optional.  Now it's true that one cannot
reply to an anonymous source (otherwise the source isn't really
anonymous), nor directly to a source unreachable via a reverse path,
but there are other circumstances that might lead to non-replyability
(e.g. transcription errors during transport), so non-replyability
does not conclusively indicate either anonymity or lack of an Internet
mailbox.  I.e. non-replyability is a symptom, not an indicator.

> >> anonymous@[::0],
> >> and anonymous@[::1].
> >
> > Not valid RFC 2821 syntax.
> 
> sounds like a bug in RFC 2821.

You're certainly free to make a case for that. The legal 2821 versions
of IPv6 domain literals would be something like [ipv6:::0]. Yes, it's
ugly. 

Incidentally, there is an inconsistency between 2822 and 2821; 2822
ABNF allows empty square brackets for the domain literal, and refers in
the normative text to 2821 for details, which does not permit an empty
literal -- it permits untagged IPv4 dotted decimal quads and tagged
literals (IPv6 and "General").  It appears that the ABNF in 2822 is
overly broad and does not correspond to the normative text referring to
2821s treatment of domain literals; it should probably incorporate
the 2821 tagging mechanism (assuming that that is deemed appropriate;
I haven't seen it used extensively).

> but if the MTA strictly implements RFC  
> 2821 and rejects the message at RCPT time, that's perfectly okay.

OK for what? For preventing a response (assuming that the message
gets to a recipient)?  Again, that's not the subject of the draft.

> it  
> doesn't really matter whether the MTA rejects the address because the 
> local-part doesn't exist or because the domain has an invalid syntax, 
> as long as it fails.

Are you discussing transport, or the message format? The draft deals
with the latter.

> > Two problems:
> > 1. the only things suitable for "whatever" have problems as
> >    detailed separately;
> 
> see above.  I think you're trying too hard to find problems with 
> alternatives to your proposal while failing to recognize the problems 
> with yours.

Not at all. We discussed syntax options months ago, and your
earlier message prompted a detailed analysis of issues (it was
clear that an empty literal wasn't permitted by 822, but I needed
to double-check 2822 and the references cited in 822 and 2822).

So far as I can see, there is no perfectly clean solution to the
issues covered in the draft.  However, simply making the From
field optional -- while that may present some issues -- seems to
have the least negative impact of the options considered (including
syntax changes and extension, which present rather more serious
backwards compatibility issues than making the field optional).

A syntax change to permit "From: <>" or named group syntax "From:
phrase: (possibly empty) list;" would have the desired semantics,
but appears to have more serious backwards compatibility issues
than making the field optional; we have both identified those forms
as syntax errors (under rules in effect for most of the history
of electronic mail; they were briefly quasi-legal in the presence
of a Sender field under RFC 733 (1977-1982) [*], but was illegal
before RFC 733 and has been again illegal since RFC 822).  If there
seems to be a strong consensus that flip-flopping yet again on the
legality of such syntax is desirable, I don't mind revising the
draft to indicate that as the proposed amendment to 822/2822, but
I think it warrants rather close examination regarding the impact
on existing UAs, MSAs, and gateways.  "<>" is legal in a Return-Path
field, and named groups are legal in recipient address fields (but
not From or Sender), so presumably there would not be insuperable
difficulties for new designs; those constructs need to be parseable
in their respective fields, so could be made parseable in the From
field by developers.  Likewise, any still-used RFC 733-compliant
parsers can still handle the syntax.  However, the concern is
about designs from the past 22 years, where the From field has not
permitted those constructs.  The question is whether the nature
and quantity of problems that would be caused by (re-)introducing
such syntax is of greater or lesser magnitude that those resulting
from making the From field optional.  I don't have a definitive
answer, and as you noted, "[t]here's no way we can expect to
survey all of that software"; making the field optional for the
Internet Message Format seems to have relatively limited impact.
Changing the field syntax has rather far-reaching implications;
HTTP (RFC 2616 section 14.22) also has a From field based on RFC
822, as does SIP (SIP, RFC 3261, has some provision for anonymity,
but it is the sort of hack that the draft under discussion seeks
to provide a reasonable alternative to for the Internet Message
Format).  Any amendment to RFC 822 syntax would affect those
protocols as well, and to date neither the HTTP nor SIP communities
have been involved in this discussion (precisely because the
discussion has been focused on the Internet Message Format with
a proposed amendment to that format that does not impinge upon
those separate protocols).  I am unsure of the implications for
such syntax changes for those protocols; however both HTTP and
SIP are relative newcomers to the Internet protocols and deal
primarily with ephemeral connections -- it is highly unlikely that
there are HTTP or SIP implementations that were designed to handle
the richer RFC 733 syntax.

In addition to the effect on HTTP and SIP, a syntax change would
also necessitate an amendment to the RFC 2821 gateway requirement
discussed earlier today.

To date I haven't heard any comments specifically regarding any
implications for HTTP and/or SIP (probably for the reasons indicated
above), and I would be uncomfortable proposing a syntax change
without input from the HTTP and SIP communities.  So if you are
or know of an HTTP or SIP expert, I'd appreciate comments about
the implications of syntax changes to the From field, either on
the ietf-822 list or off-list.

> > 2. peeking at the local-part is a layering violation unless
> >    the domain is yours.
> 
> for MTAs and UAs, yes.  not for humans.

If there's something there indicating either a domain or a local-part
within a domain, it fails to provide the desired semantics, viz.
nothing (i.e. no mailbox).  That itself is not a showstopper, but in
conjunction with the requirements for gateways, it presents some
obstacles.

------
* RFC 733 syntax excerpts and analysis:

originator-fields =
               (  "From"     ":" mailbox    ; Single author
                 ["Reply-To" ":" #address] )
            /  (  "From"     ":" 1#address  ; Multiple authors &
                  "Sender"   ":" mailbox    ;  may have non-mach-
                 ["Reply-To" ":" #address] );  ine addresses

address     =  host-phrase                  ; Machine mailbox
            / ( [phrase] "<" #address ">")  ; Individual / List
            / ( [phrase] ":" #address ";")  ; Group
            /  quoted-string                ; Arbitrary text
            / (":" ( "Include"              ; File, w/ addr list
                   / "Postal"               ; (U.S.) Postal addr
                   /  atom )                ; Extended data type
               ":" address)

mailbox     =  host-phrase /  (phrase mach-id)

mach-id     =  "<" host-phrase ">"          ; Contents must never
                                            ;  be modified!

host-phrase =  phrase  host-indicator       ; Basic address

host-indicator =  1*( ("at" / "@") node )   ; Right-most node is
                                            ;  at top of network
                                            ;  hierarchy; left-
                                            ;  most must be host

node        =  word / 1*DIGIT               ; Official host or
                                            ;  network name or
                                            ;  decimal address
----------
mailbox is NG because it requires at least a phrase, "@" (or "at"), and
a node; mailbox is required for the single-author version From field.
#address means zero or more "address" constructs in a comma-separated
list.
address itself has as one option [phrase] "<" #address ">"
therefore with a zero count, a legal address is [phrase] "<" ">"
That is allowable as the "Multiple authors" From field case if a
Sender field with a mailbox is provided.
Another address option is [phrase] ":" #address ";" which permitted
an empty (possibly un-)named group (also nested lists and groups).

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-03.txt
Date: Sun, 27 Feb 2005 03:25:56 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 58
Sender: [email-masked]

Martin Duerst wrote:

> I have bumped up the number in the subject to the one of the
> draft I submitted yesterday.

There might be still two problems in your draft archived-at-03:

| archived-at = "Archived-At:" *WSP '<' URI '>' CRLF ; URI not empty

You probably want "<" and ">" as delimiters here.  Your draft
also says:

| This header field is subject to the encoding and character
| restrictions for mail headers as described in [RFC2822].

That would limit line lenghts to 78 charcaters (a SHOULD in
2822), minus "Archived-At:<" and ">" the Archive-URI SHOULD
NOT be be longer than 64 characters.  And it MUST NOT be longer
than 998-14 charcters.  Any WSP after the colon limits the URI
further.

You could avoid these limits by header folding within the URI:

  archived-at = "Archived-At:" [FWS] "<" uri ">" CRLF
  uri         = +uri-char *( FWS +uri-char )
  uri-char    = TBD

"uri-char" is another issue in your draft:

| Additionally, the URI content is further restricted to the
| set of URI safe characters [STD66].

STD 66 nowhere specifies "URI safe characters" directly, it's
a pain to find this set.  Apparently it's the union of the sets
"reserved", "unreserved", and "%, that is "gen-delim" (7) +
"sub-delim" (11) + ALPHA (52) + DIGIT (10) + 4 other unreserved
char.s, + "%" (1) for a total of 85 characters.

128 - CTL = 94, apparently 9 printable US-ASCII characters are
excluded from your "URI safe character" set.  Only "<" and ">"
are obvious.  Therefore please specify the complete eumeration:

  uri-char    = %d33 / %d35-59 / ; printable ASCII
                %d61 / %d63-91 / ; excl. DQUOTE, "<", ">"
                %d93 / %d95 /    ; excl.    "\", "^", "`"
                %d97-122 / %d126 ; excl.    "{", "|", "}"

Please correct me if I still haven't got the STD 66 idea... :-(

If you use this solution for the length limit problem, you'd
need another explanation:

  where uri without any inserted FWS is an URI as defined in
  [STD66], and CRLF and FWS are defined in [RFC2822].

                        Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 21:52:03 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 49
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

Keith Moore wrote:
 
> I intended <anonymous@[]> as an example, not as a
> well-thought, concrete proposal.

The general form is any@[] with any valid local part.  It's
less flexible than [email-masked], but good enough where
it's supported.  Probably some news servers already support
TLD .invalid, but cannot yet handle an empty domain-literal.

Possible objections:  

1 - TLD .invalid can't be hardwired as an invalid TLD, only
    DNS can decide this.  That's wrong, the specification
    explicitly says that TLD .invalid is always (guess).

2 - If TLD .invalid is identified by some UAs to be (guess),
    then it can't be used for DNS tests.  That's true, but
    the specification explicitly offers TLD (guess) for tests.

3 - TLD .invalid might be a bad idea in examples.  That's why
    there is another TLD (guess) especially for examples.

4 - What if somebody needs a "real" TLD for local purposes ?
    BCP 62 offers TLD .localhost for this case, there's no
    reason to use TLD .invalid for anything but (guess).

5 - The DNS root servers would be flooded by stupid queries.
    That's utter dubious, in fact TLD .invalid is the only
    TLD guaranteed to be invalid without a DNS query, so if
    bogus TLDs in munged addresses are a problem, then using
    TLD .invalid could minimize this problem.

And bogus addresses are a problem, if UAs offer to compose
replies without displaying the address (maybe it's a text
mode UA with a blind user).  Therefore it's good to have 
some clear conventions like TLD .invalid or any@[] instead
of numerous me@privacy, nobody@spamcop, anon[127.0.0.1], and
similar ad-hoc "inventions".  Let alone [email-masked] etc. :-(

> I do think it makes more sense to use a domain literal of
> some kind rather than a domain name that could cause DNS
> queries.

Yes, but if it "must" be a domain name then it should be TLD
.invalid for an invalid domain.  Otherwise any@[] is better.

                        Bye, Frank

Date: Mon, 28 Feb 2005 09:52:48 +1000
From: Laird Breyer <[email-masked]>
To: [email-masked]
Subject: Re: MTS transparency and anonymity
Reply-To: [email-masked]
Mail-Followup-To: [email-masked]
Sender: [email-masked]

On Feb 27 2005, Arnt Gulbrandsen wrote:
> 
> I wonder how many spam filters reject blah@[]. Does anyone on the list know?
> 
> Arnt

This is not an easy question to answer, because modern spam filtering
technology is very dynamic. There was a presentation by Ken Schneider
of Brightmail at the 2003 MIT spam conference (unfortunately his
presentation seems to be a dead link now, here's the abstract

http://spamconference.org/abstracts2003.txt

IIRC, his overview of Brightmail described a complex feedback system
built on rule specifications which are created around the clock 
automatically and have to compete with thousands of other rules. The
best rules are sent to their filtering systems around the world
every few minutes.

So you could say a rule such as From:.*Arnt@[] might not trigger
anything while From:.*Fred@[] might do so at one time or another. I
doubt that spam filter rules look very closely at RFC 2822 semantics
in general. Some rules may well pick up illegal RFC constructs, but
other rules may just pick up particular substrings of the message
viewed as an ASCII string, and all rules have variable lifetimes.

In summary: I'm not sure that your question makes a lot of sense for modern
spam filters.

-- 
Laird Breyer.

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 12:53:13 -0500
To: Bruce Lilly <[email-masked]>
Sender: [email-masked]

>> if anonymous@[] doesn't quite work, there are similar alternatives
>> worth considering.  among them are: anonymous@[0.0.0.0],
>
> No, see RFC 3330; that's a local net.

No, see RFC 3330; it's only intended for use as a _source_ address.  
Trying to connect to the address fails on every platform I know of.

note that there are a lot fewer TCP/IP stacks in wide use than there 
are mail-handling programs.  it's much easier to be confident about the 
behavior of an IP stack.

>> anonymous@[127.0.0.1],
>
> Absolutely not; that's (the recipient's) local host, and there
> might well be a mailbox named "anonymous" (or any other legal
> local-part) there.

so use an invalid address, rather than a loopback address.  or just 
pick a very obscure local-part.  anyone who really wants to accept mail 
at
-anonymous-@[127.0.0.1]
deserves to get replies from anonymous mail.  and if he doesn't want to 
get such replies, it's a solvable problem.

also, if it happens that a dns query for example.com yields ip address 
X, it should not be assumed that [email-masked] is equivalent to 
user@[X].  the only requirement for domain literals is that the 
postmaster address work.

>> anonymous@[127.255.255.255],
>
> No, also covered by RFC 3330.

and if the host correctly implements RFC 3330 ("no addresses within 
this block should ever appear on any network anywhere") then this will 
work just fine, as the host IP stack will return an error when the MTA 
tries to connect to that address.

>> anonymous@[::0],
>> and anonymous@[::1].
>
> Not valid RFC 2821 syntax.

sounds like a bug in RFC 2821.  but if the MTA strictly implements RFC 
2821 and rejects the message at RCPT time, that's perfectly okay.   it 
doesn't really matter whether the MTA rejects the address because the 
local-part doesn't exist or because the domain has an invalid syntax, 
as long as it fails.

>> also there might be some merit in being able to differentiate between
>> "anonymous" mail (where the sender chose to be anonymous) and mail 
>> from
>> an unknown sender (such as sent from a web form).  so we could have
>> anonymous@[whatever] vs. unknown@[whatever].
>
> Two problems:
> 1. the only things suitable for "whatever" have problems as
>    detailed separately;

see above.  I think you're trying too hard to find problems with 
alternatives to your proposal while failing to recognize the problems 
with yours.

> 2. peeking at the local-part is a layering violation unless
>    the domain is yours.

for MTAs and UAs, yes.  not for humans.  of course one could also 
differentiate such cases by using

Anonymous <anoymous@[0.0.0.0]>
vs
Unknown Sender <anonymous@[0.0.0.0]>

Keith

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 12:30:53 -0500
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Sun February 27 2005 11:42, Keith Moore wrote:

> if anonymous@[] doesn't quite work, there are similar alternatives 
> worth considering.  among them are: anonymous@[0.0.0.0], 

No, see RFC 3330; that's a local net.

> anonymous@[127.0.0.1],

Absolutely not; that's (the recipient's) local host, and there
might well be a mailbox named "anonymous" (or any other legal
local-part) there.

> anonymous@[127.255.255.255],

No, also covered by RFC 3330.

> anonymous@[::0],   
> and anonymous@[::1].

Not valid RFC 2821 syntax.

> the null local-part is 
> not needed; it is sufficient if the domain is valid syntax but not a 
> DNS name.

I thought we were discussing literals rather than names. Either
way, a necessary additional constraint is that every site must
interpret that name or literal as something other than a reference
to its domain (and therefore must treat the local-part as opaque);
otherwise there is a potential for name clashes as with
<anonymous@[127.0.0.1]>.

> also there might be some merit in being able to differentiate between 
> "anonymous" mail (where the sender chose to be anonymous) and mail from 
> an unknown sender (such as sent from a web form).  so we could have 
> anonymous@[whatever] vs. unknown@[whatever].

Two problems:
1. the only things suitable for "whatever" have problems as
   detailed separately; at best one would need to invent and
   register a new RFC 2821 literal tag and wait decades for
   UAs, TAs, and gateways to be upgraded to support it
2. peeking at the local-part is a layering violation unless
   the domain is yours.  If there is a need to differentiate
   different classes of literal-based mailboxes unassociated
   with any particular site, then multiple registered 2821
   literal tags could be registered (with issues as noted above). 

Cc: [email-masked], Bruce Lilly <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 12:13:56 -0500
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

>> among them are: anonymous@[0.0.0.0], anonymous@[127.0.0.1], 
>> anonymous@[127.255.255.255], anonymous@[::0], and anonymous@[::1].
>
> (Don't these have mostly the same DNS problems as [email-masked]?)

they shouldn't.  any MTA that tries to do a DNS lookup of a domain 
literal is seriously broken.

(and at least at one time there was a popular MTA that would look up 
the PTR record for the IP address, get a domain name, and try to 
substitute that domain name - which is completely ridiculous.  the 
whole point of using a domain literal is to connect directly to the 
SMTP server at that address.)

Date: Sun, 27 Feb 2005 17:55:35 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: MTS transparency and anonymity
Cc: Bruce Lilly <[email-masked]>, Keith Moore <[email-masked]>
Sender: [email-masked]

Keith Moore writes:
> among them are: anonymous@[0.0.0.0], anonymous@[127.0.0.1], 
> anonymous@[127.255.255.255], anonymous@[::0], and anonymous@[::1].

(Don't these have mostly the same DNS problems as [email-masked]?)

Arnt

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 11:42:09 -0500
To: Bruce Lilly <[email-masked]>
Sender: [email-masked]

Bruce,

I think you're making this too difficult, by drilling too far down into 
details at this stage.

if anonymous@[] doesn't quite work, there are similar alternatives 
worth considering.  among them are: anonymous@[0.0.0.0], 
anonymous@[127.0.0.1], anonymous@[127.255.255.255], anonymous@[::0], 
and anonymous@[::1].

the problem with ""@[] is that, while valid syntax, the null local-part 
provides another excuse for something to break.  the null local-part is 
not needed; it is sufficient if the domain is valid syntax but not a 
DNS name.

also there might be some merit in being able to differentiate between 
"anonymous" mail (where the sender chose to be anonymous) and mail from 
an unknown sender (such as sent from a web form).  so we could have 
anonymous@[whatever] vs. unknown@[whatever].

Keith

Cc: [email-masked], Frank Ellermann <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 11:18:07 -0500
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

> Sorry, I phrased that badly. I meant something more like:
>
> I wonder how many mail handling programs reject blah@[]. Does anyone 
> on the list know?
>
> (I wrote "spam filters" because that's where I instinctively expected 
> the most failures to be. A pretty bad reason.)
>
> Intuitively, I agree that a domain literal makes more sense... but in 
> the end I think my preference would be for whatever causes the least 
> breakage with current software.

mine also.  for me the choice is a narrow one between From: 
<something@[something]> and From: <>.   but the latter is invalid 
syntax.

my assumption is that it's better to pick something that is valid 
syntax for two reasons:

a) it's impossible to survey the implementations out there to find out 
what will break, but there's a good chance that something out there 
will check strictly enough to break on an invalid address.

b) given a choice between breaking a few things that conform to the 
spec (by being strict about syntax), and breaking a few things that 
don't conform to the spec (by failing to accept valid syntax), it makes 
more sense to break the latter set.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 11:07:02 -0500
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Fri February 25 2005 17:34, Keith Moore wrote:

> At the same time, the From field has been required for at least
> 25 years now, and there's a lot of software out there that was
> written to that specification.  There's no way we can expect to
> survey all of that software to determine whether omission of From
> will adversely impact it.  Email is already fragile enough (for
> instance, due to the filters that Ned refers to) that we should
> be very reluctant to create additional reasons for things to break -
> particularly when we don't gain any additional functionality to
> compensate the public for the breakage.

There is existing functionality (purported to exist) but which is
not achievable (correctly) w/o some change; see the discussion of
RFC 822 section A.2.6 and RFC 2822 section 3.6.2 (which are in
conflict).  That is the case of a message author with no Internet
mailbox [not necessarily unusual, given availability of we-based
interfaces for sending mail by anyone with HTTP access, even if
that person has no Internet mailbox].

There is additional functionality in provision for anonymous
authorship, which is important for reasons detailed in the draft.

And, as you point out, it is unclear whether there would be any
breakage.

> So I'd much rather define some 
> convention for a non-replyable address (e.g. From: <anonymous@[]> ) 
> which is at least syntactically compatible with RFC [2]822,
> than to say it's okay to omit the From field.

First, the intent is to provide for messages where the author
either has no Internet mailbox or is unable to disclose an
Internet mailbox, not specifically for "non-replyable"
conventions (for some unspecified purpose(s) and/or
condition(s)).  That is a matter of scope.

Second, the local-part can only be interpreted within the domain
in the mailbox specification.  It is therefore largely irrelevant
to either the scope of the draft or to your "non-replyable" case.
Aside from the local-part, your present example is the same as
the <""@[]> that was discussed quite some time ago.

Third, RFC 822 does not directly give ABNF for a domain literal,
instead deferring to RFC 820, which specifies four dot-separated
decimal integers in the 0-255 range.  RFC 2822 does give ABNF
syntax, but it is so broad (IPv6 issues not having been settled
at the time) as to permit almost anything, including non-whitespace
control characters; 2822 provides no guidance regarding the
semantics of domain literals, deferring to RFC 2821, which
has  provision for IPv4 dotted decimal quads and for tagged
IPv6 and tagged "General" literals.  The tags referred to in
2821 "MUST be specified in a standards-track RFC and registered
with IANA" -- I know of no such RFCs or registrations as of this
time.

So while I am not outright rejecting any such convention, I
believe that there are several issues that would have to be
carefully considered:
1. Conformance to RFC 2821 syntax as referenced by 2822 for syntax
   and semantics of domain literals.  This is also necessary for
   the case of SMTP gateways, which are expected to ensure
   that address fields (incl. From) are "effective and useful
   for sending replies".  Given the realities of the situations
   covered in the draft and your "non-replyable" case, I
   suspect that's a non-starter (unless you want to amend RFC
   2821 section 3.6.4 as well; and as noted, the draft under
   discussion carefully avoids transport issues -- there may
   well be other transport mechanisms that impose other,
   possibly conflicting, constraints on syntax and semantics;
   that would need to be researched).
2. Local-parts would have to remain irrelevant. To avoid
   possible conflicts with local-parts in use at a recipient
   site, any domain literal would have to be one guaranteed
   not to be interpreted as local anywhere.  That pretty
   much rules out all IPv4 literals (RFC 3330). I do not
   know if there are any IPv6 literals which would carry such
   a guarantee; that would have to be investigated as a
   possible approach.  Failing that, a Standards Track RFC
   defining an LDH tag per RFC 2821 would have to be published
   and the tag registered by IANA.  Then MUAs, MSAs, MTAs,
   etc. would have to be modified to recognize the tag. I
   suspect that we have another show-stopper, because ...
3. The impact on the existing infrastructure would have to be
   considered.  While it is certainly possible that there
   exists some software which does not gracefully handle
   absence of a From field, it is certainly the case that
   all transport and UA software currently fails to recognize
   an as-yet unspecified and unregistered tag in an RFC 2821
   domain literal.  I suspect that most don't handle the
   2821 "IPv6" tag.  I strongly suspect that older
   implementations in particular would attempt to hand off
   literal content, tag and all, to a resolver; that is
   unlikely to produce desirable results, and may cause
   confusion (e.g. the first 32 bits being interpreted as
   a binary IPv4 address) or other problems.  In short, I
   believe such an approach would lack backwards compatibility
   (indeed, the RFC 2821 "General" literals seem to have
   that problem, and as noted, I suspect that the "IPv6"
   tag itself poses compatibility problems).

In light of the above considerations, I cannot see that a
convention such as your example would (if properly designed
to conform to RFC 2821 requirements) result in fewer
problems than simply making the From field optional. Because
of the nearly universal loathing of <""@[]> when it was
last discussed, I did not include syntactically valid
(and that case really isn't valid if one follows 2822s
redirection to 2821) constructs in the list of alternatives
considered.  I'll rectify that omission in the next draft
revision.

Date: Sun, 27 Feb 2005 16:45:52 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: MTS transparency and anonymity
Cc: Frank Ellermann <[email-masked]>, Keith Moore <[email-masked]>
Sender: [email-masked]

Sorry, I phrased that badly. I meant something more like:

I wonder how many mail handling programs reject blah@[]. Does anyone on 
the list know?

(I wrote "spam filters" because that's where I instinctively expected 
the most failures to be. A pretty bad reason.)

Intuitively, I agree that a domain literal makes more sense... but in 
the end I think my preference would be for whatever causes the least 
breakage with current software.

Arnt

Date: Sun, 27 Feb 2005 10:37:56 -0500 (EST)
From: <anonymous@[]>
Bcc:
Reply-To: [email-masked]
Subject: anonymous@[] test
Sender: [email-masked]

just testing to see if this works.  reply privately if you have any 
interesting observations - I'll summarize to the list.

Keith

Cc: [email-masked], Frank Ellermann <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 10:35:12 -0500
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

> I wonder how many spam filters reject blah@[]. Does anyone on the list 
> know?

And if they do reject it, is that a bug or a feature?

Lots of bogus criteria are employed by spam filters.  So I don't think 
we should use whether a spam filter rejects a particular kind of 
address as a criterion for whether or not to use that address as a 
token of anonymity.  Spam filters have to adapt to changes in spam, 
they may as well adapt to changes in standards also.

More to the point, how many MTAs and UAs barf on blah@[] ?

Also I intended <anonymous@[]> as an example, not as a well-thought, 
concrete proposal.  I literally spent about two seconds thinking about 
it before suggesting it.  I do think it makes more sense to use a 
domain literal of some kind rather than a domain name that could cause 
DNS queries.

Keith

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 10:10:00 -0500
Cc: Frank Ellermann <[email-masked]>, Keith Moore <[email-masked]>
Sender: [email-masked]

On Sun February 27 2005 02:51, Frank Ellermann wrote:
> 
> Keith Moore wrote:
> 
>  [From: <anonymous@[]>]
> > since this doesn't define a new domain name (it uses a
> > domain literal), there's no need to update BCP 32.

It's not clear that it's a domain literal per RFC 822; more
detail separately.
 
> Sure, but you want this idea to replace the "best common 
> practice" From: <[email-masked]> based on BCP 32, and
> therefore the best place to document it is an updated BCP.

The scope of BCP 32 is for testing and examples, and ".invalid"
is specifically reserved "for use in online construction of domain
names".  There's nothing there about use in mailboxes, email
responses, etc.  BCP 32 nowhere recommends widespread of those
reserved names in core Internet protocols, and certainly says
nothing about the Internet Message Format From field; indeed, the
intent is stated as being to reduce "confusion and conflict" which
may occur when names used for testing and in examples escape
onto the Internet -- it certainly doesn't encourage deliberately
bombarding the root nameservers with those reserved names.

> It's important that user agents support these conventions,
> and don't offer to send mail replies to invalid addresses.

The only reliable way to determine whether the domain part of
a mailbox is valid is to query DNS (see RFC 3696).  An off-line
UA (or one on the non-Internet side of a gateway) clearly cannot
do so as DNS is unavailable.  So we need a mechanism which does
not depend on domain names.

Moreover, actually querying DNS for large number of messages
containing bogus domain names would place an unwarranted burden
on DNS (specifically, the root nameservers), and might be
construed as a denial-of-service attack.
 
> That was the point of Bruce's draft, and that's also the
> reason why Usefor drafts proposed TLD .invalid for about
> five years.

The point of the draft is to standardize a mechanism which
avoids inappropriate hacking of UAs and inappropriate
bombardment of DNS root nameservers by inappropriate use of
a TLD reserved for testing and examples.  See section 1,
second paragraph labeled "b.".  The idea is to have something
more suitable for use in places like the USEFOR WG work than
such inappropriate use of reserved names.

Date: Sun, 27 Feb 2005 14:37:09 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: MTS transparency and anonymity
Cc: Frank Ellermann <[email-masked]>, Keith Moore <[email-masked]>
Sender: [email-masked]

I wonder how many spam filters reject blah@[]. Does anyone on the list know?

Arnt

Date: Sun, 27 Feb 2005 03:25:56 +0100
From: Frank Ellermann <[email-masked]>
Organization: <URL:http://purl.net/xyzzy>
To: [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-03.txt
Sender: [email-masked]

Martin Duerst wrote:

> I have bumped up the number in the subject to the one of the
> draft I submitted yesterday.

There might be still two problems in your draft archived-at-03:

| archived-at = "Archived-At:" *WSP '<' URI '>' CRLF ; URI not empty

You probably want "<" and ">" as delimiters here.  Your draft
also says:

| This header field is subject to the encoding and character
| restrictions for mail headers as described in [RFC2822].

That would limit line lenghts to 78 charcaters (a SHOULD in
2822), minus "Archived-At:<" and ">" the Archive-URI SHOULD
NOT be be longer than 64 characters.  And it MUST NOT be longer
than 998-14 charcters.  Any WSP after the colon limits the URI
further.

You could avoid these limits by header folding within the URI:

  archived-at = "Archived-At:" [FWS] "<" uri ">" CRLF
  uri         = +uri-char *( FWS +uri-char )
  uri-char    = TBD

"uri-char" is another issue in your draft:

| Additionally, the URI content is further restricted to the
| set of URI safe characters [STD66].

STD 66 nowhere specifies "URI safe characters" directly, it's
a pain to find this set.  Apparently it's the union of the sets
"reserved", "unreserved", and "%, that is "gen-delim" (7) +
"sub-delim" (11) + ALPHA (52) + DIGIT (10) + 4 other unreserved
char.s, + "%" (1) for a total of 85 characters.

128 - CTL = 94, apparently 9 printable US-ASCII characters are
excluded from your "URI safe character" set.  Only "<" and ">"
are obvious.  Therefore please specify the complete eumeration:

  uri-char    = %d33 / %d35-59 / ; printable ASCII
                %d61 / %d63-91 / ; excl. DQUOTE, "<", ">"
                %d93 / %d95 /    ; excl.    "\", "^", "`"
                %d97-122 / %d126 ; excl.    "{", "|", "}"

Please correct me if I still haven't got the STD 66 idea... :-(

If you use this solution for the length limit problem, you'd
need another explanation:

  where uri without any inserted FWS is an URI as defined in
  [STD66], and CRLF and FWS are defined in [RFC2822].

                        Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 08:51:23 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 18
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

Keith Moore wrote:

 [From: <anonymous@[]>]
> since this doesn't define a new domain name (it uses a
> domain literal), there's no need to update BCP 32.

Sure, but you want this idea to replace the "best common 
practice" From: <[email-masked]> based on BCP 32, and
therefore the best place to document it is an updated BCP.

It's important that user agents support these conventions,
and don't offer to send mail replies to invalid addresses.

That was the point of Bruce's draft, and that's also the
reason why Usefor drafts proposed TLD .invalid for about
five years.
                  Bye, Frank

Date: Sat, 26 Feb 2005 21:49:31 -0500
From: Keith Moore <[email-masked]>
To: Frank Ellermann <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

> 
> Keith Moore wrote:
> 
> > I'd much rather define some convention for a non-replyable
> > address (e.g. From: <anonymous@[]> ) which is at least
> > syntactically compatible with RFC [2]822,
> 
> Nice idea, please don't forget to update BCP 32.  Bye, Frank

since this doesn't define a new domain name (it uses a domain
literal), there's no need to update BCP 32.

Keith

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 27 Feb 2005 00:49:01 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 8
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

Keith Moore wrote:

> I'd much rather define some convention for a non-replyable
> address (e.g. From: <anonymous@[]> ) which is at least
> syntactically compatible with RFC [2]822,

Nice idea, please don't forget to update BCP 32.  Bye, Frank

Cc: [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: Use of Sender in authentication considered unacceptable (was Re: draft-lilly-from-optional-01.txt)
Date: Sat, 26 Feb 2005 09:57:37 -0500
To: SM <[email-masked]>
Sender: [email-masked]

>> If you want a viable authentication proposal, you need to define a new
>> field which will have a well-established meaning, not try to borrow 
>> some
>> existing field which has either a different meaning (From) or some
>> existing field which has no consistent use in practice (Sender).
>
> I agree that we don't see a consistent use of the Sender field in 
> practice.  If the agent responsible for the transmission of the 
> message makes proper use of the Sender field, I don't see why we 
> should redefine a new field when there is already a field which was 
> set out for that purpose.

because in practice Sender is almost never used properly, and use of 
Sender is so varied that the recipient cannot tell what is meant by it.

for example: mailing lists should almost never set Sender, because it 
obscures who actually sent the message, and this is of interest to list 
recipients.  (there are exceptions: it's reasonable for digests to set 
Sender to point to the mailing list.  similarly for lists that 
deliberately try to hide the actual addresses of the correspondents)

Keith

Date: Sat, 26 Feb 2005 06:22:03 -0800
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Use of Sender in authentication considered unacceptable (was Re: draft-lilly-from-optional-01.txt)
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

At 14:44 25-02-2005, Keith Moore wrote:
>If you want a viable authentication proposal, you need to define a new
>field which will have a well-established meaning, not try to borrow some
>existing field which has either a different meaning (From) or some
>existing field which has no consistent use in practice (Sender).

I agree that we don't see a consistent use of the Sender field in 
practice.  If the agent responsible for the transmission of the message 
makes proper use of the Sender field, I don't see why we should redefine a 
new field when there is already a field which was set out for that purpose.

Regards,
-sm 

Date: Fri, 25 Feb 2005 17:44:30 -0500
From: Keith Moore <[email-masked]>
To: SM <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Use of Sender in authentication considered unacceptable (was Re: draft-lilly-from-optional-01.txt)
Sender: [email-masked]

> 
> At 07:18 23-02-2005, Bruce Lilly wrote:
> >Responding to your points in reverse order:
> >
> >The message header field "Sender" is an originator field which is
> >not necessarily related to the SMTP envelope sender return path (or
> >similar mechanisms in protocols other than SMTP).  Consequently, it
> >plays no role in authentication.
> 
> I agree that the "Sender" field may not necessarily be related to the
> SMTP envelope sender.  The field may be used for end-to-end
> authentication technologies.

we need to nip this idea in the bud.  

It is completely inappropriate to use Sender for e2e authentication.

When applied to a diverse group of users, such use inevitably conflicts
with both the email specifications and with the installed base of
software.  

> The From and/or Sender headers are the only fields available 
> to the MUA for making any determination as to the reputation of the
> sending domain.

And neither one is appropriate for authentication in general.    Sender
is just too haphazardly used, and there are too many legitimate reasons
for the From field to contain some address or addresses other than that
of the message originator.

If you want a viable authentication proposal, you need to define a new 
field which will have a well-established meaning, not try to borrow some
existing field which has either a different meaning (From) or some
existing field which has no consistent use in practice (Sender).

IMHO, there's no way that any proposal reusing Sender can meet the 
requirements for Proposed Standard.

There's another idea that we need to nip in the bud, and it's that
any single authentication system will make a dent in the spam or 
phishing or virus problems.  We're going to need layered defenses
in order to have a useful effect.  And for layered defenses to work
implies that they can't use the same protocol fields in ways that
conflict with the specifications, established practice, or each other.

Keith

Date: Fri, 25 Feb 2005 17:34:27 -0500
From: Keith Moore <[email-masked]>
To: [email-masked]
Cc: [email-masked], [email-masked], [email-masked]
Subject: MTS transparency and anonymity (was Re: draft-lilly-from-optional-01.txt)
Sender: [email-masked]

> > If a particular user or set of users wishes to ignore anonymous
> > messages, they are of course free to do so; it may indeed be the
> > case that any such messages received by some such users(s) would
> > be considered spam.  On the other hand, recipients who might
> > reasonably expect to receive anonymous messages would almost
> > certainly not use anti-spam software that discards such messages.
> 
> Unfortunately this is just not the way things are headed. Users are
> increasingly not able to control these sorts of actions being done on
> their behalf. For example, Sun filters all incoming mail and summarily
> discards everything it thinks is spam. (In fact a recent
> misconfiguration led to a bunch of mail being discarded incorrectly.)
> I have absolutely no control over this - no way to change filter
> threshholds, no personal whitelist, no Bayesian training, no option to
> file spam in a spam folder, nothing. And while it is perhaps true that
> I wouldn't want and have no business receiving anonymous mail sent to
> my Sun account, I deal with large ISP setups  all the time and I can
> tell you that many if not most of them are similarly set up and are
> similarly inflexible.

We need to be careful that we don't let our ideas of "the way things are
headed" (especially in the short term) color our notion of "what it
takes for things to work well".  IETF's job is to discover, define, and
state the latter, not to legitimize the former.

MTA transparency is important.  It's what allows email to behave
predictably and reliably.  We've always known this to be a necessary
condition.  It's true that a lot of MTAs don't behave transparently
these days, and it's also true that this invariably has an adverse
effect on the reliability, predictability, and usability of email.
Either we'll find a way to fix the problems that cause people to want
to make email less transparent, or email will cease to be useful.
Either way, the lack of transparency in the MTS is a short-term problem.

(note that transparency isn't the same thing as saying that all MTAs
must blindly relay all messages intact.)

> Of course it is hard to tell when the ubiquity of such setups reaches
> a point where it truly compromises the general ability to send
> anonymous email. But my guess is that if we haven't passed that point
> yet we're about to.

Truly anonymous email doesn't exist.  Even if there's no From address
there's still a source IP address and port number and timestamp which
can often be traced to the originator.  This is in some sense a "name".
The existence or absence of a From field doesn't change that.  There
are lots of ways to achieve an effect similar to anonymity, such as
using someone else's "name", using a pseudonym, communicating through
an intermediary, etc., but all of these leave some trace to the 
originator.  And it's already been aptly demonstrated that miscreants 
are at least as capable of establishing apparently-valid "names" for 
themselves as the general public.  So it seems unlikely that either
anonymity or pseudonymity will be very useful as a discriminator for
mail filters.

As to the proposal:

I'm somewhat sympathetic to the view that one shoudn't bother 
including a From field when the reply won't get there anyway.
At the same time, the From field has been required for at least
25 years now, and there's a lot of software out there that was
written to that specification.  There's no way we can expect to
survey all of that software to determine whether omission of From
will adversely impact it.  Email is already fragile enough (for
instance, due to the filters that Ned refers to) that we should
be very reluctant to create additional reasons for things to break -
particularly when we don't gain any additional functionality to
compensate the public for the breakage. So I'd much rather define some
convention for a non-replyable address (e.g. From: <anonymous@[]> ) 
which is at least syntactically compatible with RFC [2]822,
than to say it's okay to omit the From field.

Keith

To: [email-masked]
Xref: clerew local.mime:4446
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Date: Fri, 25 Feb 2005 17:31:31 GMT
Lines: 80
Sender: [email-masked]

In <[email-masked]> [email-masked] writes:

>We already do know: Alteration of message content between submission and
>delivery is done routinely for all sorts of reasons, some good and some not so
>good. Pretending otherwise is simply not realistic. I will also add that
>blaming sendmail for this or getting wrappped up in terminology (e.g., a
>gateway can do blah blah but an MTA cannot) is pointless and silly.

>However, it isn't clear to me that the reality that messages are routinely
>modified by the transport infrastructure in and of itself poses a unsuperable
>obstacle for the draft being discussed here. To the extent that message
>modification impinges on missing From fields, I believe they do so by filling
>in the field with "something". Common choices for "something" include:

>(0) The envelope from address (if present).
>(1) An address derived from available authentication information. (The
>    information can come from SASL AUTH, TLS, or even some entirely
>    external mechanism like RADIUS.)
>(2) A known-bogus address (e.g., missing-address@missing-domain)
>(3) A mail administrator address of some sort.

I think you need to add

(4) A known not-obviously-bogus address. Could be something you are
regularly used to seeing in messages, such as MAILER-DAEMON@, or a mail
administrator using his own address rather than one pertaining to his
function, or the -request or -owner address of a mailing list.

Note that sendmail currently does (0).

>The only one of these that poses a significant problem is (1) since it can have
>the effect of revealing exactly the information the sender needed to have
>hidden. The others mostly have nuisance value and nothing else. (1) is a real
>risk and needs to be called out in the draft, but I don't think the practice is
>so common that by itself it will preclude deploying this change.

I think it is just up to someone who wants to be truly anonymous to be
careful to use a service that does not reveal his identity, such as one of
the anonymising services such as privacy.net.

>These days one of the things spam filters commonly do is to check various
>aspects of message syntax, including but not limited to the message having all
>the mandatory header fields.

Or that it contains addresses such as <>, etc. Some people would think
that these things are features rather than bugs :-) .

>This issue is exacerbated by the fact that the spam landscape changes
>constantly and so do the countermeasures undertaken to stop it.

But that actually works in our favour here. If we introduce some new
feature into the standards, such as non-mandatory From, or "<>" as a valid
addr-spec, or whatever, then it will be a long time before the bulk of
agents worldwide catches up. So we have to look at the bahaviour of
existing agents and decide which solution to the alleged problem will have
the least adverse effect (and I don't think that will turn out to be the
non-mandatory From).

But spam traps are continually being updated. They have to be, or they
will no longer work. Anybody using a spam filter untouched from two years
ago will find that current spam is passing straight through it. Therefore,
the timescale on which spam filters are upgraded is almost an order of
magnitude shorter than the timescale on which people upgrade their MTAs
and MUAs.

So if a fashion delelops for "legitimate" anonymity, which seems to be the
intent of this proposal, then spam filters will be upgraded to recognize
it (whether to pass it or to bar it) in pretty short order.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Fri, 25 Feb 2005 11:56:14 -0800
To: ietf-822 <[email-masked]>
From: SM <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Sender: [email-masked]

At 07:18 23-02-2005, Bruce Lilly wrote:
>Responding to your points in reverse order:
>
>The message header field "Sender" is an originator field which is
>not necessarily related to the SMTP envelope sender return path (or
>similar mechanisms in protocols other than SMTP).  Consequently, it
>plays no role in authentication.

I agree that the "Sender" field may not necessarily be related to the SMTP 
envelope sender.  The field may be used for end-to-end authentication 
technologies.  The From and/or Sender headers are the only fields available 
to the MUA for making any determination as to the reputation of the sending 
domain.

>The draft addresses two separate cases where the From field is
>inappropriate: 1) where the author has no Internet mailbox (this
>is not a case of anonymity) and 2) where the author requires
>anonymity.  In the first case, specifying a Sender field is certainly
>possible, though not particularly useful (details below).  In the
>second case, a Sender field may be harmful; it might directly
>or indirectly (when the guys with firearms show up demanding to
>know who a message's author is) defeat anonymity.  In the case of
>an individual author who requires some degree of anonymity, the
>claim that supplying that author's mailbox in a Sender field does not
>affect anonymity doesn't bear much scrutiny.

We rarely if ever see a genuine case one in practice.  Case two is 
interesting.  The guys will firearms will show up demanding who is the 
message's author even if you don't include a Sender as they will have the 
IP address of the relay.  My suggestion is to have the mailbox of the 
Sender in the Sender field.  That does not affect the anonymity of the 
message's author.

>The Sender field is not particularly useful; it is NEVER used
>automatically in responses (RFC 822 sect, 4.4.2, also RFC 3834) --
>the Reply-To field exists for support of the first case above where

I understand that the Sender field is never used in response.

>So while I would carefully consider such a requirement if there's
>a sound case for it, the "authentication" argument does not
>constitute such a case, since message originator fields are not
>intended for authentication protocol use.  Such consideration also
>needs to take into account both reasons for making the From field
>optional; putting information in a differently-named field does
>not take anonymity into account.

My point is not about server-side authentication.  Message originator 
fields can and may be used for authentication.  Quoting RFC 822 section 4.4.2:

   "This field contains the authenticated identity of the AGENT (person, 
system or
    process) that sends the message."

On a different note, you wrote "As such, it improves security for message 
authors who require anonymity." under security considerations.  People 
requiring anonymity are better served by using anonymous remailers instead 
of excluding the From field from their message.

Regards,
-sm 

To: [email-masked]
Xref: clerew local.mime:4445
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Date: Fri, 25 Feb 2005 17:09:13 GMT
Lines: 100
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:

>>> What do you mean by "altogether breaks"?

Yes, "altogether breaks" was possibly an exaggeration.

>>>  It looks to me like it causes
>>> sendmail to corrupt the message by introducing a spurious From header
>>> with no actual content ...

>Yeah, that was ambiguous.  I meant that it contained no useful semantic
>content that wasn't already available, not that the header was completely
>empty.

The problem is that the content it creates may _look_ like useful semantic
content (i.e. it leads the recipient to believe that it _did_ come from
that address - it might even be an address the recipient already
recognises).

>I get all of that.  It doesn't bother me, frankly; whenever any
>modification is made to a standard in the area of previously undefined
>behavior, some existing software isn't going to be immediately up-to-date.
>sendmail has for years violated many existing best practices for MTAs, and
>it shouldn't be at all surprising that it doesn't support new drafts that
>depend on those best practices.

Indeed, if you change a standard, you may expect slightly odd things to
happen when seen by existing software. The question to address, therefore,
is which of the likely "odd things" will be most acceptable to typical
users, and to choose a standard modification which leads to such an
acceptable behaviour.

In the present case, what we want is that the recipient is made
immediately aware that the message was from an anonymous source. Ideally,
he would see "ANONYMOUS MESSAGE" in big red letters, but existing
software is unlikely to provide that :-( .

Now it will be clear to him if he sees:
    No From line at all
    A From line with obviously unusable contents, such as
        From: <>, or
	From: "Mickey Mouse" <>, or
	From: "Mickey Mouse"

It will be less clear to him (but probably acceptable) if he sees
something recognisable as an improper or unusual address, such as:
        From: <[email-masked]>, or
	From: <[email-masked]>, or
        From: <[email-masked]>, or even
	From: [email-masked]

But it would be a Bad Thing if he were to see an address which looked like
a genuine sender, such as
        From: Russ Allbery <[email-masked]>, or even
	From: [email-masked]

On looking into what sendmail actually does, I see now that it takes the
envelope From and constructs a From: header from it (and if the envelope
is <>, it creates no From header at all). That is not as evil as I had
first thought. But we may be sure that, taking sendmail, all the other
MTAs out there, and all the various existing gatewaye, listservs, etc out
there, you are going to get a variety of invented From headers, some of
which will certainly mislead your average non-technical reader.

Whereas, if you modify the From header syntax to allow some explicit
indication of anonymity (such as "<>"), then the worst that will happen is
that some agent will drop the whole message on the floor (I think that
most unlikely, given the propensity of existing agents to "mend" things
rather than drop them), or else it will omit the header (not too bad), or
change it to X-From: <> (not too bad either).

So the question to address is which is the least of the various inevitable
evils? The (quite likely) possibility that the reader will be misled, or
the (most unlikely) possibility that the message will fail to arrive?

>> He claims that it will do no harm to the existing network. We already
>> have one counter example to that,

>We do?  Please, by all means, fill us in!  Or do you consider addition of
>a spurious header by sendmail to be harm, in which case I'm completely
>mystified by the definition of harm that you're applying?

I think anything that misleads the reader can justifiably be regarded as
"harm".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Date: Fri, 25 Feb 2005 09:01:03 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 19
Sender: [email-masked]

Bruce Lilly wrote:

> In the meantime, though, we shouldn't avoid solving existing
> problems where solutions are possible.

The main problem of your draft is that you can't publish it
1975 or maybe 1985.  In the real world today users would be
forced to use a non-emtpy From even if your draft is an RfC
updating numerous other RfCs.  But that won't update the
<joke> 4 billion </joke> pieces of software insisting on a
non-empty From.

What you really get is a new feature for new implementations
which nobody can use.  From: [email-masked] hacks are ugly,
but users and UAs will learn to interpret it correctly without
killing some poor DNS root-servers.

                      Bye, Frank

Date: Fri, 25 Feb 2005 12:09:10 +1000
From: Laird Breyer <[email-masked]>
To: ietf-822 <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Reply-To: [email-masked]
Mail-Followup-To: ietf-822 <[email-masked]>
Sender: [email-masked]

> 
> > [...] On the other hand, recipients who might
> > reasonably expect to receive anonymous messages would almost
> > certainly not use anti-spam software that discards such messages.
> 

>
> Unfortunately this is just not the way things are headed. Users are
> increasingly not able to control these sorts of actions being done on their
> behalf. For example, Sun filters all incoming mail and summarily discards
> everything it thinks is spam. (In fact a recent misconfiguration led to a bunch
> of mail being discarded incorrectly.) I have absolutely no control over this -
> no way to change filter threshholds, no personal whitelist, no Bayesian
> training, no option to file spam in a spam folder, nothing. And while it is
> perhaps true that I wouldn't want and have no business receiving anonymous mail
> sent to my Sun account, I deal with large ISP setups  all the time and I can
> tell you that many if not most of them are similarly set up and are similarly
> inflexible.
> 

I'd like to add to this that commercial filter providers consider
their rules a competitive trade secret. Thus, until there is an outcry
against closed source filters (which may never happen), users will
never have the kind of control over receipt of anonymous messages that
Bruce is positing. Moreover, most users have no idea how the email system
works in the first place, and wouldn't know what they are missing.

-- 
Laird Breyer.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Date: Thu, 24 Feb 2005 20:49:17 -0500
Sender: [email-masked]

On Thu February 24 2005 17:23, [email-masked] wrote:

> Unfortunately this is just not the way things are headed. Users are
> increasingly not able to control these sorts of actions being done on their
> behalf. For example, Sun filters all incoming mail and summarily discards
> everything it thinks is spam. (In fact a recent misconfiguration led to a bunch
> of mail being discarded incorrectly.) I have absolutely no control over this -
> no way to change filter threshholds, no personal whitelist, no Bayesian
> training, no option to file spam in a spam folder, nothing. And while it is
> perhaps true that I wouldn't want and have no business receiving anonymous mail
> sent to my Sun account, I deal with large ISP setups  all the time and I can
> tell you that many if not most of them are similarly set up and are similarly
> inflexible.

I understand, and have a similar situation.  What I had in mind
for recipients who might reasonably expect anonymous mail would
be agencies getting reports from "whistleblowers", e.g. US OSHA.
 
> Of course it is hard to tell when the ubiquity of such setups reaches a point
> where it truly compromises the general ability to send anonymous email.

Or indeed any email.

> But my 
> guess is that if we haven't passed that point yet we're about to.

Some observations:

o mail used to be reasonably reliable; now an over-aggressive spam
  filter is likely to cause non-spam to disappear
o other spam-related "solutions", are so obnoxious that they
  discourage correspondence via email
o spam filters frequently mangle messages, especially MIME
  messages, to the point that content is unusable

So, sure, some anonymous email will probably fall into a black
hole.  That happens to some non-anonymous email.

My conclusion is that overzealousness in the fight against spam
is as much a detriment to utility of email communication as spam
itself is.  And I suspect things will get worse before they get
better.  In the meantime, though, we shouldn't avoid solving
existing problems where solutions are possible.

From: [email-masked]
Cc: ietf-822 <[email-masked]>
Date: Thu, 24 Feb 2005 14:23:14 -0800 (PST)
Subject: Re: draft-lilly-from-optional-01.txt
To: Bruce Lilly <[email-masked]>
Sender: [email-masked]

> On Thu February 24 2005 13:14, [email-masked] wrote:

> > RFC 2821 does distinguish between SMTP systems that don't modify messages
> > (relays) and ones that do (gateways).

> You are correct that gateways are not only permitted to modify content,
> they are required to ensure that address fields contain content which
> is "effective and useful for sending replies" (RFC 2821 sect. 3.8.4).
> Indeed, that is one of the reasons that use of an invalid or
> unresolvable domain name is not a viable option.

> In the case of either anonymity or lack of an Internet mailbox,
> "sending replies" isn't a possibility (except in the latter case
> given a Reply-To field).  I can address this issue more fully in
> the next revision.

> > To the extent that message
> > modification impinges on missing From fields, I believe they do so by filling
> > in the field with "something". Common choices for "something" include:
> >
> > (0) The envelope from address (if present).
> > (1) An address derived from available authentication information. (The
> >     information can come from SASL AUTH, TLS, or even some entirely
> >     external mechanism like RADIUS.)
> > (2) A known-bogus address (e.g., missing-address@missing-domain)
> > (3) A mail administrator address of some sort.
> >
> > The only one of these that poses a significant problem is (1) since it can have
> > the effect of revealing exactly the information the sender needed to have
> > hidden. The others mostly have nuisance value and nothing else. (1) is a real
> > risk and needs to be called out in the draft, but I don't think the practice is
> > so common that by itself it will preclude deploying this change.

> Either (0) or (1) could defeat anonymity,

(0) only comes into play if the envelope from contains information about the
sender, in which case there is no anonymity to be had.

> but they are predicated
> on transport mechanisms.  The draft is intended to address the
> message format issues rather than transport of the content (some
> transport mechanisms have an envelope or use authentication; others
> do not).

> > Another major problem is that the draft as written deals with the technical
> > issue of making From: optional but doesn't discuss the myriad security issues
> > that surround the idea of anonymous email. IMO it is borderline irresponsible
> > (*) to publish a document that discusses this one small aspect of email
> > anonymity while not providing any guidance about the more general set of issues
> > here. If this is to move forward this material needs to be provided - possibly
> > by a companion document.
> [...]
> > (*) It is precisely because the potential consequences of breach of anonymity
> > are so grave that this omission is of such great concern.

> Yes, as noted above it deals with the message format rather than
> transport (including non-email transport of Internet Message Format
> messages such as via Usenet).

> Point taken; the next revision will mention the possibility of
> transport mechanisms leaking information under security
> considerations.

OK.

> > Even if you
> > assume SpamAssassin is the norm and that using the absence of a From: field as
> > a check for possible spam doesn't happen at all currently, all it would take
> > would be a major influx of spam without From: fields to change this. And  I
> > somehow doubt that arguments that this breaks the ability to send mail
> > anonymously will impress the folks that build anti-spam software much if at
> > all.

> If a particular user or set of users wishes to ignore anonymous
> messages, they are of course free to do so; it may indeed be the
> case that any such messages received by some such users(s) would
> be considered spam.  On the other hand, recipients who might
> reasonably expect to receive anonymous messages would almost
> certainly not use anti-spam software that discards such messages.

Unfortunately this is just not the way things are headed. Users are
increasingly not able to control these sorts of actions being done on their
behalf. For example, Sun filters all incoming mail and summarily discards
everything it thinks is spam. (In fact a recent misconfiguration led to a bunch
of mail being discarded incorrectly.) I have absolutely no control over this -
no way to change filter threshholds, no personal whitelist, no Bayesian
training, no option to file spam in a spam folder, nothing. And while it is
perhaps true that I wouldn't want and have no business receiving anonymous mail
sent to my Sun account, I deal with large ISP setups  all the time and I can
tell you that many if not most of them are similarly set up and are similarly
inflexible.

Of course it is hard to tell when the ubiquity of such setups reaches a point
where it truly compromises the general ability to send anonymous email. But my
guess is that if we haven't passed that point yet we're about to.

				Ned

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Date: Thu, 24 Feb 2005 16:59:22 -0500
Sender: [email-masked]

On Thu February 24 2005 13:14, [email-masked] wrote:

> RFC 2821 does distinguish between SMTP systems that don't modify messages
> (relays) and ones that do (gateways).

You are correct that gateways are not only permitted to modify content,
they are required to ensure that address fields contain content which
is "effective and useful for sending replies" (RFC 2821 sect. 3.8.4).
Indeed, that is one of the reasons that use of an invalid or
unresolvable domain name is not a viable option.

In the case of either anonymity or lack of an Internet mailbox,
"sending replies" isn't a possibility (except in the latter case
given a Reply-To field).  I can address this issue more fully in
the next revision.

> To the extent that message
> modification impinges on missing From fields, I believe they do so by filling
> in the field with "something". Common choices for "something" include:
> 
> (0) The envelope from address (if present).
> (1) An address derived from available authentication information. (The
>     information can come from SASL AUTH, TLS, or even some entirely
>     external mechanism like RADIUS.)
> (2) A known-bogus address (e.g., missing-address@missing-domain)
> (3) A mail administrator address of some sort.
> 
> The only one of these that poses a significant problem is (1) since it can have
> the effect of revealing exactly the information the sender needed to have
> hidden. The others mostly have nuisance value and nothing else. (1) is a real
> risk and needs to be called out in the draft, but I don't think the practice is
> so common that by itself it will preclude deploying this change.

Either (0) or (1) could defeat anonymity, but they are predicated
on transport mechanisms.  The draft is intended to address the
message format issues rather than transport of the content (some
transport mechanisms have an envelope or use authentication; others
do not).
[rearranging comments somewhat...]

> Another major problem is that the draft as written deals with the technical
> issue of making From: optional but doesn't discuss the myriad security issues
> that surround the idea of anonymous email. IMO it is borderline irresponsible
> (*) to publish a document that discusses this one small aspect of email
> anonymity while not providing any guidance about the more general set of issues
> here. If this is to move forward this material needs to be provided - possibly
> by a companion document.
[...]
> (*) It is precisely because the potential consequences of breach of anonymity
> are so grave that this omission is of such great concern.

Yes, as noted above it deals with the message format rather than
transport (including non-email transport of Internet Message Format
messages such as via Usenet).

Point taken; the next revision will mention the possibility of
transport mechanisms leaking information under security
considerations. 

> At present a message without a From: field is syntactically illegal.[...]
> These days one of the things spam filters commonly do is to check various
> aspects of message syntax, including but not limited to the message having all
> the mandatory header fields.

True; also checks for valid syntax, sometimes additional checks on
address field contents.  The draft enumerates 4 possible approaches
to the problems it identifies.  Of those, "a" (use of an alternate
mailbox) is itself often used by spammers and results in collateral
damage (inappropriate "bounce" messages, etc.).  Method "b"
(theoretically unresolvable domain name) has, in conjunction with
the Verisign SiteFinder "service" resulted in anti-spam problems,
and as noted in the draft is unusable where DNS is unavailable or
otherwise unreliable.  That leaves changing the From field syntax
or making the field optional as proposed (unless there is another
possible solution that I missed).  Syntax changes would require a
different set of changes to 822 and 2822, and would still be an
issue w.r.t. anti-spam software; additionally, there would be UA
issues related to a very large and diverse installed base having
to be updated to accept revised syntax.

> Even if you 
> assume SpamAssassin is the norm and that using the absence of a From: field as
> a check for possible spam doesn't happen at all currently, all it would take
> would be a major influx of spam without From: fields to change this. And  I
> somehow doubt that arguments that this breaks the ability to send mail
> anonymously will impress the folks that build anti-spam software much if at
> all.

If a particular user or set of users wishes to ignore anonymous
messages, they are of course free to do so; it may indeed be the
case that any such messages received by some such users(s) would
be considered spam.  On the other hand, recipients who might
reasonably expect to receive anonymous messages would almost
certainly not use anti-spam software that discards such messages.

From: [email-masked]
Cc: [email-masked]
Date: Thu, 24 Feb 2005 10:14:04 -0800 (PST)
Subject: Re: draft-lilly-from-optional-01.txt
To: Charles Lindsey <[email-masked]>
Sender: [email-masked]

> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

> > On Wed February 23 2005 07:37, Charles Lindsey wrote:

> > > This latest draft fails to address the problem I reaised earlier, namely
> > > that omitting the From field altogether breaks a widely deployed MTA,
> > > namely sendmail (and maybe others too) which is regularly configured to
> > > insert a From field of its own choosing if it finds none present already.

> > MTAs do not alter content; they are forbidden to do so.

RFC 2821 does distinguish between SMTP systems that don't modify messages
(relays) and ones that do (gateways). But not only are gateways not prohibited,
there are plenty of cases where content is supposed to be modified during
transit. An obvious example of this is 8bit to 7bit downgrading of MIME
messages. Another example is:

  http://www.ietf.org/internet-drafts/draft-ietf-fax-esmtp-conneg-13.txt

which has been approved as a proposed standard. And then there's

  http://www.ietf.org/internet-drafts/draft-ietf-opes-smtp-use-cases-00.txt

which I believe allows for modification of both envelope and content.

> > MSAs can be
> > configured to do so, but are recommended to tread lightly.

> You fail to distinguish between what MTAs "do" and what they are
> "supposed" to do (or not do). Sadly, in the real world, the perfect and
> ideal MTA does not exist. It appears that one widely deployed MTA does the
> "wrong thing". There may be others. We need to know.

We already do know: Alteration of message content between submission and
delivery is done routinely for all sorts of reasons, some good and some not so
good. Pretending otherwise is simply not realistic. I will also add that
blaming sendmail for this or getting wrappped up in terminology (e.g., a
gateway can do blah blah but an MTA cannot) is pointless and silly.

However, it isn't clear to me that the reality that messages are routinely
modified by the transport infrastructure in and of itself poses a unsuperable
obstacle for the draft being discussed here. To the extent that message
modification impinges on missing From fields, I believe they do so by filling
in the field with "something". Common choices for "something" include:

(0) The envelope from address (if present).
(1) An address derived from available authentication information. (The
    information can come from SASL AUTH, TLS, or even some entirely
    external mechanism like RADIUS.)
(2) A known-bogus address (e.g., missing-address@missing-domain)
(3) A mail administrator address of some sort.

The only one of these that poses a significant problem is (1) since it can have
the effect of revealing exactly the information the sender needed to have
hidden. The others mostly have nuisance value and nothing else. (1) is a real
risk and needs to be called out in the draft, but I don't think the practice is
so common that by itself it will preclude deploying this change.

However, I worry that this draft isn't viable for a different reason: At
present a message without a From: field is syntactically illegal. Ten years ago
the governing principle in handling syntactically illegal email was to "be
liberal in what you accept". But that was before spam became such a problem.
These days one of the things spam filters commonly do is to check various
aspects of message syntax, including but not limited to the message having all
the mandatory header fields. Failure to comply with the syntax rules
increasingly tends to get your message labelled as spam. (As a side note, an
out-of-the-box SpamAssassin setup doesn't appear to care about missing From:
fields, but SpamAssassin is hardly the only spam filter out there.)

This issue is exacerbated by the fact that the spam landscape changes
constantly and so do the countermeasures undertaken to stop it. Even if you
assume SpamAssassin is the norm and that using the absence of a From: field as
a check for possible spam doesn't happen at all currently, all it would take
would be a major influx of spam without From: fields to change this. And  I
somehow doubt that arguments that this breaks the ability to send mail
anonymously will impress the folks that build anti-spam software much if at
all.

Another major problem is that the draft as written deals with the technical
issue of making From: optional but doesn't discuss the myriad security issues
that surround the idea of anonymous email. IMO it is borderline irresponsible
(*) to publish a document that discusses this one small aspect of email
anonymity while not providing any guidance about the more general set of issues
here. If this is to move forward this material needs to be provided - possibly
by a companion document.

				Ned

(*) It is precisely because the potential consequences of breach of anonymity
are so grave that this omission is of such great concern.

To: [email-masked]
Subject: Re: draft-lilly-from-optional-01.txt
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 24 Feb 2005 10:07:00 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> What do you mean by "altogether breaks"?  It looks to me like it causes
>> sendmail to corrupt the message by introducing a spurious From header
>> with no actual content (a bug in sendmail, IMO, as an MTA should not be
>> modifying even messages that it deems syntactically invalid), but that
>> seems a far cry short of "altogether breaks" and just would put
>> sendmail into the class of programs that don't correctly comply with
>> this draft.

> Not sure what you mean by "a spurious From header with no actual
> content".  It most certainly did have a content (my own mailbox, as it
> happens).

Yeah, that was ambiguous.  I meant that it contained no useful semantic
content that wasn't already available, not that the header was completely
empty.

> Yes, it may well be regarded as a sendmail bug, but it is so widely
> deployed that it cannot be ignored. And if sendmail contains that bug,
> who is to say what other systems (e.g. exim) have fallen into the same
> error.

I get all of that.  It doesn't bother me, frankly; whenever any
modification is made to a standard in the area of previously undefined
behavior, some existing software isn't going to be immediately up-to-date.
sendmail has for years violated many existing best practices for MTAs, and
it shouldn't be at all surprising that it doesn't support new drafts that
depend on those best practices.

What I'm trying to understand is where you came up with "altogether
breaks," which would indicate some sort of interoperability issue or some
sort of break with backward compatibility that would mean that Bruce's
draft would cause systems not to interoperate in practice.  I'm still not
seeing where you saw that problem.  Could you please fill us in?

> He claims that it will do no harm to the existing network. We already
> have one counter example to that,

We do?  Please, by all means, fill us in!  Or do you consider addition of
a spurious header by sendmail to be harm, in which case I'm completely
mystified by the definition of harm that you're applying?

If you consider modification of a message in ways not covered by RFC 2822
to be harm, sendmail is already routinely causing harm via its behavior of
rewriting From, To, and Cc headers to canonicalize CNAMEs and corrupting
messages by adding spurious "Undisclosed recipients" headers.  This is
certainly no worse.

> Who knows? It might not even get through the mailing list expander.

I expect it probably wouldn't if the mailing list expander refuses mail
from unknown senders, which would most certainly include anonymous
senders, but that's not an argument against Bruce's proposal.  It is,
rather, a quite reasonable effect of local filtering configurations.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.mime:4437
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Date: Thu, 24 Feb 2005 15:06:51 GMT
Lines: 41
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Wed February 23 2005 07:37, Charles Lindsey wrote:
>> 
>> This latest draft fails to address the problem I reaised earlier, namely
>> that omitting the From field altogether breaks a widely deployed MTA,
>> namely sendmail (and maybe others too) which is regularly configured to
>> insert a From field of its own choosing if it finds none present already.

>MTAs do not alter content; they are forbidden to do so.  MSAs can be
>configured to do so, but are recommended to tread lightly.

You fail to distinguish between what MTAs "do" and what they are
"supposed" to do (or not do). Sadly, in the real world, the perfect and
ideal MTA does not exist. It appears that one widely deployed MTA does the
"wrong thing". There may be others. We need to know.

>> I asked you (Bruce) to post a test message to this list containing no From
>> field (and no Message-ID field for that matter) so that the effect of your
>> proposal on existing MTAs could be evaluated. Why have you not done this?

>Because of the general rule "don't feed the trolls".  But since you
>are so persistent,.. "What do you mean -- is that an African swallow
>or a European one?".

Sorry, your subtlety excapes me. Are you going to post such a test
message, or are you going to bury your head in the sand and assert that it
cannot possibly go wrong because you say it cannot possibly go wrong?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:4436
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Date: Thu, 24 Feb 2005 14:58:36 GMT
Lines: 42
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> This latest draft fails to address the problem I reaised earlier, namely
>> that omitting the From field altogether breaks a widely deployed MTA,
>> namely sendmail (and maybe others too) which is regularly configured to
>> insert a From field of its own choosing if it finds none present
>> already.

>What do you mean by "altogether breaks"?  It looks to me like it causes
>sendmail to corrupt the message by introducing a spurious From header with
>no actual content (a bug in sendmail, IMO, as an MTA should not be
>modifying even messages that it deems syntactically invalid), but that
>seems a far cry short of "altogether breaks" and just would put sendmail
>into the class of programs that don't correctly comply with this draft.

Not sure what you mean by "a spurious From header with no actual content".
It most certainly did have a content (my own mailbox, as it happens).

Yes, it may well be regarded as a sendmail bug, but it is so widely
deployed that it cannot be ignored. And if sendmail contains that bug, who
is to say what other systems (e.g. exim) have fallen into the same error.

That is why I asked Bruce to post a test message to this list with No From
header. He claims that it will do no harm to the existing network. We
already have one counter example to that, but there are so many ways that
mail will be delivered to members of this list, and so many MTAs that it
will, have passed through, that it would soon show up if the problem were
widespread. Who knows? It might not even get through the mailing list
expander.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Subject: Re: draft-lilly-from-optional-01.txt
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 23 Feb 2005 11:46:12 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> This latest draft fails to address the problem I reaised earlier, namely
> that omitting the From field altogether breaks a widely deployed MTA,
> namely sendmail (and maybe others too) which is regularly configured to
> insert a From field of its own choosing if it finds none present
> already.

What do you mean by "altogether breaks"?  It looks to me like it causes
sendmail to corrupt the message by introducing a spurious From header with
no actual content (a bug in sendmail, IMO, as an MTA should not be
modifying even messages that it deems syntactically invalid), but that
seems a far cry short of "altogether breaks" and just would put sendmail
into the class of programs that don't correctly comply with this draft.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Date: Wed, 23 Feb 2005 13:43:09 -0500
Cc: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

On Wed February 23 2005 07:37, Charles Lindsey wrote:
> 
> This latest draft fails to address the problem I reaised earlier, namely
> that omitting the From field altogether breaks a widely deployed MTA,
> namely sendmail (and maybe others too) which is regularly configured to
> insert a From field of its own choosing if it finds none present already.

MTAs do not alter content; they are forbidden to do so.  MSAs can be
configured to do so, but are recommended to tread lightly.

> I asked you (Bruce) to post a test message to this list containing no From
> field (and no Message-ID field for that matter) so that the effect of your
> proposal on existing MTAs could be evaluated. Why have you not done this?

Because of the general rule "don't feed the trolls".  But since you
are so persistent,.. "What do you mean -- is that an African swallow
or a European one?".

To: [email-masked]
Xref: clerew local.mime:4432
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Date: Wed, 23 Feb 2005 12:37:34 GMT
Lines: 19
Sender: [email-masked]

This latest draft fails to address the problem I reaised earlier, namely
that omitting the From field altogether breaks a widely deployed MTA,
namely sendmail (and maybe others too) which is regularly configured to
insert a From field of its own choosing if it finds none present already.

I asked you (Bruce) to post a test message to this list containing no From
field (and no Message-ID field for that matter) so that the effect of your
proposal on existing MTAs could be evaluated. Why have you not done this?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: draft-lilly-from-optional-01.txt
Date: Wed, 23 Feb 2005 10:18:10 -0500
Cc: SM <[email-masked]>
Sender: [email-masked]

On Tue February 22 2005 14:46, SM wrote:
> 
> draft-lilly-from-optional-01.txt makes the From: field optional.  According to
> 4.2. RFC 822 section 4.4.2, the Sender field is optional as well.
> 
> I suggest that your draft amends Section 3.6.2 of RFC 2822 and 4.2. RFC 822 
> section 4.4.2 as follows:
> 
> If the from field is omitted then the sender field, containing the field 
> name "Sender" and a single mailbox specification, MUST appear in the message.
> 
> If the From field is omitted, a Sender field MUST be supplied.
> 
> The above changes do not affect the anonymity of the author or authors 
> while allowing domain-based email authentication if the sender implements it.

Responding to your points in reverse order:

The message header field "Sender" is an originator field which is
not necessarily related to the SMTP envelope sender return path (or
similar mechanisms in protocols other than SMTP).  Consequently, it
plays no role in authentication.

The draft addresses two separate cases where the From field is
inappropriate: 1) where the author has no Internet mailbox (this
is not a case of anonymity) and 2) where the author requires
anonymity.  In the first case, specifying a Sender field is certainly
possible, though not particularly useful (details below).  In the
second case, a Sender field may be harmful; it might directly
or indirectly (when the guys with firearms show up demanding to
know who a message's author is) defeat anonymity.  In the case of
an individual author who requires some degree of anonymity, the
claim that supplying that author's mailbox in a Sender field does not
affect anonymity doesn't bear much scrutiny.

The Sender field is not particularly useful; it is NEVER used
automatically in responses (RFC 822 sect, 4.4.2, also RFC 3834) --
the Reply-To field exists for support of the first case above where
some proxy is able to receive responses on behalf of an author with
no Internet mailbox, and SMTP envelope (or equivalent) is used for
delivery failure notifications.  In the case of multiple authors,
it may help to identify which of multiple authors sent a message;
provided that the sending author has an Internet mailbox and that
anonymity is not required.

So while I would carefully consider such a requirement if there's
a sound case for it, the "authentication" argument does not
constitute such a case, since message originator fields are not
intended for authentication protocol use.  Such consideration also
needs to take into account both reasons for making the From field
optional; putting information in a differently-named field does
not take anonymity into account.

Best regards,
  Bruce Lilly

Date: Wed, 23 Feb 2005 14:20:46 +0900
To: Graham Klyne <[email-masked]>, Keith Moore <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-03.txt
Cc: [email-masked]
Sender: [email-masked]

At 19:20 05/02/22, Graham Klyne wrote:
 >At 10:02 22/02/05 +0900, Martin Duerst wrote:

 >>I'm trying to get the W3C list archives available in message/rfc822.
 >>I hope you'll contribute to making our common vision a reality,
 >>e.g. by working on a mailer so that it accepts message/rfc822
 >>messages (via the 'open this file' 'protocol' of the OS) or can
 >>download them directly.
 >
 >FWIW, I just tested the mailbox facility for one of the W3C mailing 
lists, and it worked just fine with my browser/mailer combination:  Firefox 
passes the downloaded file to Eudora, which opens it just fine.

Please note that this is a feature that was deployed very recently
and is simpilar, but not the same, as the one Keith and I are discussing.

This feature is about having mboxes containing all the mails
from a given period available, in application/mbox format.
That means that the original format of the message is available
(with some rather severe restrictions to protect the information
from spammers), but it is not single messages, and it is not
negotiated at the same location as the HTML format. We are
still working on the later.

Regards,    Martin. 

Date: Tue, 22 Feb 2005 11:46:05 -0800
To: [email-masked]
From: SM <[email-masked]>
Subject: draft-lilly-from-optional-01.txt
Sender: [email-masked]

draft-lilly-from-optional-01.txt makes the From: field optional.  According to
4.2. RFC 822 section 4.4.2, the Sender field is optional as well.

I suggest that your draft amends Section 3.6.2 of RFC 2822 and 4.2. RFC 822 
section 4.4.2 as follows:

If the from field is omitted then the sender field, containing the field 
name "Sender" and a single mailbox specification, MUST appear in the message.

If the From field is omitted, a Sender field MUST be supplied.

The above changes do not affect the anonymity of the author or authors 
while allowing domain-based email authentication if the sender implements it.

Regards,
-sm

Cc: [email-masked], [email-masked], [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-03.txt
Date: Tue, 22 Feb 2005 12:40:56 -0500
To: Martin Duerst <[email-masked]>
Sender: [email-masked]

> I'm trying to get the W3C list archives available in message/rfc822.
> I hope you'll contribute to making our common vision a reality,
> e.g. by working on a mailer so that it accepts message/rfc822
> messages (via the 'open this file' 'protocol' of the OS) or can
> download them directly.

A couple of things I neglected to include in the last message I sent:

First, and most importantly: I'm very appreciative of your efforts to 
make this proposal something that is broadly applicable, and also of 
your efforts to make W3C archives available in message/rfc822 format.

Second: My impression is that there's a deeper issue under the surface 
here - one which I'm still trying to understand.  I do think that "the 
web" has mostly avoided this problem by having the implicit assumption 
that nearly everything linked to will be more-or-less HTML (by 
more-or-less, I mean that it's either HTML or easily converted to HTML 
without significant loss of information content or semantics).  And now 
that I think about it, that assumption seems to be less true even for 
"the web" than it used to be.  There's more PDF, Flash, WM*, Real*, 
QuickTime, and PowerPoint than there used to be.  And even when the 
content is HTML, it's less and less often "pure" HTML and more and more 
often combined with Java, JavaScript, ActiveX, Flash, or some other 
language that is used in a way that makes it essential to the content.  
I often find myself getting annoyed because I waited for some link to 
download only to find that it isn't usable in my browser.  So I don't 
think this is just an email problem, and maybe it shouldn't be treated 
as one.  It just happens that the issue comes up in the context of 
archives of email that are converted to HTML, because that's one of 
those cases where such conversion results in significant loss of 
information and semantic content.  If I'm right, then we really should 
have the same kinds of "metadata caching" in <A ...>, <IMG...>, 
<SCRIPT..> and other HTML tags in addition to Archived-At email header 
fields.  But I know this isn't the first time this has come up in 
discussion - so maybe there's some collected wisdom about it somewhere?

Keith

Date: Tue, 22 Feb 2005 10:20:34 +0000
To: Martin Duerst <[email-masked]>, Keith Moore <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-03.txt
Cc: [email-masked]
Sender: [email-masked]

At 10:02 22/02/05 +0900, Martin Duerst wrote:
>I can very clearly tell you that this is not the case.
>I'm trying to get the W3C list archives available in message/rfc822.
>I hope you'll contribute to making our common vision a reality,
>e.g. by working on a mailer so that it accepts message/rfc822
>messages (via the 'open this file' 'protocol' of the OS) or can
>download them directly.

FWIW, I just tested the mailbox facility for one of the W3C mailing lists, 
and it worked just fine with my browser/mailer combination:  Firefox passes 
the downloaded file to Eudora, which opens it just fine.

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Cc: [email-masked], [email-masked], [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-03.txt
Date: Tue, 22 Feb 2005 00:54:23 -0500
To: Martin Duerst <[email-masked]>
Sender: [email-masked]

> Maybe not in this mail, but in another one, I mentioned a very
> concrete example of why adding type information inhibits
> future developments, or inhibits tools taking advantage of
> future developments. W3C currently serves archived emails
> as HTML only. So with your proposal, we would send out
> (X-)Archived-At headers with whatever additional parameter
> (maybe no parameter) that means "HTML only". Now there are
> actually some developments on making message/rfc822 available,
> not only for new messages, but also for old ones.
>
> But the headers have already gone out, and all of them say
> "HTML only". This would mean that either we are no longer
> very much motivated to make message rfc822 available, because
> most people will 'know' that we only serve HTML, or that
> we make them available, but they don't get used because
> the headers tell the software not to even try (or both).
>
> I'd very much appreciate if you could explain how your
> parameter proposal addresses this case.

I would consider those parameters "hints".  That is, they're assumed to 
be accurate at the time the Archived-At field is generated, but may not 
stay accurate.  Implementations would be expected to gracefully handle 
the case where the hints aren't accurate, and they would also be 
allowed to "ask" the server for features (other content-types, other 
messages in the same archive) even when those features were 
inconsistent with the hints.  Also, the hints would be optional.

If you want, think of the hints as a cache of certain resource 
metadata.  They could even include expiry or TTL information.

Now consider two cases: one in which the hints are provided, another in 
which they're not provided.

In the second case, a client always has to "ask" before it can tell 
whether or not it can make use of the archived message.   That question 
will typically involve a DNS lookup, a TCP open, and some sort of layer 
7 interaction - a HEAD request being the minimum.  This can easily take 
a few seconds on average, and minutes in corner cases where multiple 
servers have to be consulted.  User agent implementors then have to 
decide whether it's worth the performance hit to offer a feature that 
has a dubious chance of being useful to the user.

In the first case, the client can still "ask" - but the client might 
have more information available to it.  For instance, the client might 
compare the current time to the date in the message, and if there's a 
small delta, assume that the hint is accurate.  The client might decide 
whether to "ask" based on some estimate of available bandwidth.  That 
way, a client operating over a slow modem might trust the hints, 
whereas a client operating over a broadband connection would ask.

> >and failing to provide type information along with the URI is a bad
> >idea because it forces the client (and user) to set up a network
> >connection before it even knows whether the resources is usable.
>
> Well, in principle yes. But these days, a large number of network
> connections are just established (in the sense that the user is
> connected to the network), and making e.g. a HEAD request to a
> server isn't really a big deal.

When my home DSL connection is working, I don't consider those things a 
terribly big deal. (though I do often find HTTP referral chains 
annoyingly slow even then).  When my DSL connection fails (as it does 
fairly frequently lately) and I have to read my mail over a modem, 
extra HTTP connections do turn out to be a big deal.

Keith

Date: Tue, 22 Feb 2005 10:02:23 +0900
To: Keith Moore <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-03.txt
Cc: [email-masked], [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

Hello Keith,

[I have bumped up the number in the subject to the one of the
draft I submitted yesterday.]

Maybe not in this mail, but in another one, I mentioned a very
concrete example of why adding type information inhibits
future developments, or inhibits tools taking advantage of
future developments. W3C currently serves archived emails
as HTML only. So with your proposal, we would send out
(X-)Archived-At headers with whatever additional parameter
(maybe no parameter) that means "HTML only". Now there are
actually some developments on making message/rfc822 available,
not only for new messages, but also for old ones.

But the headers have already gone out, and all of them say
"HTML only". This would mean that either we are no longer
very much motivated to make message rfc822 available, because
most people will 'know' that we only serve HTML, or that
we make them available, but they don't get used because
the headers tell the software not to even try (or both).

I'd very much appreciate if you could explain how your
parameter proposal addresses this case.

At 05:46 05/02/22, Keith Moore wrote:
 >
 >> As I pointed out in a separate message, providing type information
 >> with an URI is a bad idea because it inhibits future developments.
 >
 >and failing to provide type information along with the URI is a bad
 >idea because it forces the client (and user) to set up a network
 >connection before it even knows whether the resources is usable.

Well, in principle yes. But these days, a large number of network
connections are just established (in the sense that the user is
connected to the network), and making e.g. a HEAD request to a
server isn't really a big deal.

 >more to the point, behind the desire to avoid providing type informtion
 >is an implicit (and false) assumption that HTML content is usually
 >sufficient.

I can very clearly tell you that this is not the case.
I'm trying to get the W3C list archives available in message/rfc822.
I hope you'll contribute to making our common vision a reality,
e.g. by working on a mailer so that it accepts message/rfc822
messages (via the 'open this file' 'protocol' of the OS) or can
download them directly.

Regards,   Martin. 

Date: Mon, 21 Feb 2005 15:46:05 -0500
From: Keith Moore <[email-masked]>
To: Martin Duerst <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

> As I pointed out in a separate message, providing type information
> with an URI is a bad idea because it inhibits future developments.

and failing to provide type information along with the URI is a bad
idea because it forces the client (and user) to set up a network 
connection before it even knows whether the resources is usable.

more to the point, behind the desire to avoid providing type informtion
is an implicit (and false) assumption that HTML content is usually 
sufficient.

Keith

Date: Fri, 18 Feb 2005 17:00:30 +0900
To: Arnt Gulbrandsen <[email-masked]>, [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Keith Moore <[email-masked]>, Graham Klyne <[email-masked]>
Sender: [email-masked]

Hello Arnt,

As I pointed out in a separate message, providing type information
with an URI is a bad idea because it inhibits future developments.

Regards,    Martin.

At 19:25 04/10/27, Arnt Gulbrandsen wrote:
 >Arnt Gulbrandsen writes:
 >> (That doesn't necessarily mean that the retrieved object must have a 
certain content-type. I can imagine other ways to achieve the same end.)
 >
 >That was stupid of me. Concrete suggestions beat handwaving. Sorry.
 >
 >My preferred approach would be to give archived-at the same parameter 
list that MIME fields have, and define a "content-type=x/y" parameter 
naming a type expected to be available. For a list that provides archives 
in two forms and uses fancy HTTP to provide the right one, the field could 
be like this:
 >
 >Archived-At: ...; content-type=message/rfc822, content-type=text/html
 >
 >For a less fancy archive providing both forms under separate URLs, two 
Archived-At fields could be used:
 >
 >Archived-At: ...; content-type=message/rfc822
 >Archived-At: ...; content-type=text/html
 >
 >For today's usual sort of archive, the expected content-type would be 
text/html by default:
 >
 >Archived-At: ...
 >
 >Arnt
 > 

Date: Fri, 18 Feb 2005 16:48:36 +0900
To: Bruce Lilly <[email-masked]>, "ietf-822" <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: the gap regarding Archived-At
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

Hello Bruce,

Please see below.

At 02:05 04/10/31, Bruce Lilly wrote:
 >On Thu October 28 2004 19:54, Keith Moore wrote:
 >
 >> >   A message may contain an arbitrary
 >> >    number of Comments fields.  However, it does not solve the
 >> >    problem of the incompatibility with RFC 2046 message/partial.
 >>
 >> could you explain that supposed incompatibility again?
 >
 >Fragmentation/reassembly is defined in RFC 2046 section 5.2.2 and
 >its subsections.  The issue was discussed in some detail on the ietf-822
 >mailing list in June 2003 and again very briefly regarding specific
 >field (References, In-Reply-To, and Resent-Message-ID) in September
 >2004.  Briefly, the fragmentation/reassembly process distributes
 >original (unfragmented) message header fields between the header
 >of the first fragment and the header of the encapsulated message
 >(which appears within the message/partial wrapper in the first
 >fragment; reassembly combines fields from those two headers to
 >produce a facsimile of the original message from fragment messages.
 >RFC 2046 lists specific fields which are taken from the encapsulated
 >header -- MIME Content- fields, Subject, Message-ID, Encrypted,
 >and MIME-version -- that list is amended by RFCs 2298/3798 to
 >include Disposition-Notification-To, Disposition-Notification-Options,
 >and Original-Recipient; these fields are the ones which pertain
 >specifically  to the unfragmented original message as opposed to
 >any fragment.
 >
 >Clearly, the proposed Archived-At field applies to a specific message
 >(like Message-ID), and should be in the same category as those
 >specifically mentioned.  However, unlike RFC 3798 and its
 >predecessor, the archived-at draft does not amend the RFC2046
 >list of fields which are reassembled from the encapsulated message
 >header.

Well, I could just say that it updates RFC 2046. But I'm not
sure this is necessary.

 >Consequently, Archived-At would be treated like Comments;
 >when fragmenting a message, the field would be copied to the
 >first fragment message header, and would be reassembled from there
 >to the reconstituted facsimile of the original unfragmented
 >message (any such fields within the encapsulated message header
 >would be ignored, in accordance with the RFC 2046 rules). That
 >is inconsistent with the semantics proposed for Archived-At, viz. that
 >it references an archived copy of the specific message in which it
 >is conveyed. An Archived-At field that originally referred to an
 >archive of an unfragmented message would therefore be
 >inappropriately placed in the message header of a different message
 >(viz. the first fragment message).

Well, I could just say that for partial messages, the semantics
may be either-or. Not the best solution, but that would make
spec an reality match, at least, without any updates to
infrastructure.

 >Moreover, it would be
 >indistinguishable there from an appropriate Archived-At field which
 >referred to an archive of that (first fragment) message.  And
 >as fragments may be further fragmented, that issue may be
 >compounded.

Ok, so that would mean that the semantics would have to be
extended even a bit more. What about the following sentence:

"Note that due to the way header fields are handled with message
fragmentation and reassembly [RFC2046], there
may be cases where the URI in an Archived-At message header field
points to the archived version of an original (unfragmented) message
from a fragmented part of a message or the archivedversion of a
fragmented part of a message from the reassembled message."

If that's not good enough, please feel free (or better, encouraged)
to provide better text.

Regards,    Martin. 

Date: Fri, 18 Feb 2005 16:24:31 +0900
To: ietf-822 <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Sender: [email-masked]

At 23:22 04/10/26, Bruce Lilly wrote:
 >On Sun October 24 2004 22:46, Martin Duerst wrote:
 >
 >> I have added a very clear sentence saying that the field MUST only be
 >> created if the message is actually being made available at the URI given
 >> in the header field. Do you think that this should say anything more?
 >
 >Since (unlike Reply-To and Sender) the proposed field plays no
 >role in message protocols, but is simply a way of conveying
 >additional information, "MUST" may be too strong.

Good point, added a reference to RFC 2119 and changed "MUST"
to "SHOULD".

Regards,    Martin.

 >RFC 2119
 >specifically states that it must not be used except to ensure
 >interoperation or to avoid network damage (retransmissions,
 >etc.).  Incidentally, it is customary to include some boilerplate
 >text referring to the RFC 2119-defined terms and to include
 >RFC 2119 as a normative reference when making use of those
 >terms.  Of course if you intentionally did not reference RFC
 >2119 because you intend your use of "MUST", "MUST NOT",
 >"SHOULD", "SHOULD NOT" and "MAY" to have different
 >meanings than the ones ascribed by RFC 2119, that raises
 >the question of precisely what meaning *is* intended. 

Date: Fri, 18 Feb 2005 15:28:52 +0900
To: Bruce Lilly <[email-masked]>, "ietf-822" <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: Graham Klyne <[email-masked]>
Sender: [email-masked]

At 08:41 04/10/29, Bruce Lilly wrote:
 >On Thu October 28 2004 09:09, Graham Klyne wrote:
 >
 >> To paraphrase (British) parliament-speak:  I refer the gentleman to the
 >> answer given previously:
 >>    http://lists.w3.org/Archives/Public/www-archive/2004Oct/0050.html
 >
 >That does not explain the utility of the particular structured
 >header field proposed (vs. e.g. an unstructured Comments field
 >or a parenthesized comment in e.g. the Message-ID field, or
 >text in a body part, etc., all of which would be at least as
 >usable for clickety-clickety or copy and paste).

It is clear (e.g. from Keith's comments) that there are usages
for both computers and humans. I think it's better to define
this as a structured field and end up with no machine use
than to define it as a Comments field and end up with
all kinds of incompatible formats and code trying to interpret
it if an when usage by machines catches up.

Regards,    Martin. 

Date: Fri, 18 Feb 2005 16:09:08 +0900
To: ietf-822 <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: New Internet Draft: draft-duerst-archived-at-00.txt
Cc: "ietf-822" <[email-masked]>
Sender: [email-masked]

At 23:05 04/10/26, Bruce Lilly wrote:
 >On Mon October 25 2004 00:42, Martin Duerst wrote:
 >>
 >> I tried to adapt the syntax below to the changes I made.
 >> The problem with URIs/URI references is solved by RFC2396bis.
 >> I don't think [FWS] and *WSP are needed, the syntax given for
 >> similar headers in RFC 2822 doesn't explicitly mention these,
 >> either.
 >>
 >> Regards,    Martin.
 >
 >Actually, RFC 2822 does include [FWS] etc., but they are part of
 >the lower-level construct specifications, so aren't readily apparent
 >when looking only at the field definitions; they do become visible
 >if one expands "domain", "dot-atom", "date-time", etc.  That
 >does not apply to the constructs used in the proposed field, since
 >the ABNF for those constructs does not explicitly include [FWS]
 >etc.  Incidentally, the successor to RFC 2822 may change in that
 >respect; a proposed revised grammar has been discussed on the
 >ietf-822 list.
 >
 >Given the draft ABNF
 >
 >       archived-at = "Archived-At:" '<' URI '>' CRLF ; URI not empty
 >
 >and the example implicit in the text
 >
 >    As an example, the URI
 >    "http://www.w3.org/mid/[email-masked]",
 >    containing the message id "[email-masked]",
 >    is redirected to the URI of this message in the W3C mailing-list
 >    archive at http://lists.w3.org/Archives/Public/uri/2004Oct/0017.html.
 >
 >implies that
 >
 > Archived-At:<http://lists.w3.org/Archives/Public/uri/2004Oct/0017.html>
 >
 >is legal but
 >
 > Archived-At: <http://lists.w3.org/Archives/Public/uri/2004Oct/0017.html>
 >                    ^ N.B. SP here
 > Archived-At: (w3 list archive)
 >   <http://lists.w3.org/Archives/Public/uri/2004Oct/0017.html>
 >
 >etc. are not legal.

Thanks for pointing this out. I changed the definition to:

archived-at = "Archived-At:" *WSP '<' URI '>' CRLF ; URI not empty

(and similar for obs-archived-at), because that's what was intended.
No whitespace after the colon looks really weird, but I don't care
about comments.

 >Concerning that example, please note that a message identifier (RFC 2822)
 >includes angle brackets; i.e. the message id is presumably
 >"<[email-masked]>".  When using mid
 >schemes or conversions which implicitly involve msg-id to mid or
 >mid-like constructs, it might be advisable to note the difference in
 >angle bracket inclusion and/or to refer to the mid RFC (2387) as a
 >normative or informative reference.  The angle bracket issue has also
 >been recently discussed on the ietf-822 list.

I have tweaked the text in a way that I hope will avoid
misunderstandings.

 >Note also that RFC 2822 has provisions in the parse (obs-) syntax
 >for accommodation of whitespace between the field name and the colon
 >delimiter (which was allowed in earlier versions of the text message format).
 >You might wish to consider whether to provide for that for parsing,
 >particularly in the obs- rule for X-Archived-At.

Considered but left out. There are no X-Archived-At headers
out that have such a space, so we don't need to introduce this.

Regards,   Martin.

Regards,    Martin. 

Date: Fri, 18 Feb 2005 15:23:58 +0900
To: Keith Moore <[email-masked]>, [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: the gap regarding Archived-At
Sender: [email-masked]

Hello Keith,

I'm just working now on another version of my draft.
I'll address your points below.

At 05:38 04/10/29, Keith Moore wrote:
 >
 >okay, I find myself coming to some somewhat uncomfortable conclusions:
 >
 >1. There really are significant differences between
 >    a) how email clients would use archived-at,
 >    b) how humans would use it to cut-and-paste into HTML, and
 >    c) how web browsers would use it.
 >
 >    These differences are significant enough that I'm wondering if we
 >    really do want separate archived-at fields for web use and email use,
 >    or tags on the archived-at field that indicates whether the message
 >    is in original format and/or if other messages in the same collection
 >    can also be accessed.

I think a single header can go a long way. If it turns out
that it's not enough, we (you?) can always create another one.

 >2. For archived-at to be useful by email clients generally requires
 >    one or two of the following, in addition to the obvious client
 >    support for the protocol and server support for the message format:
 >
 >    - mail archives that support IMAP access (or possibly NNTP)

The archived-at header could point to it easily, or because
IMAP is more about collections, it would actually be the
List-Archive header, which we already have.

 >    - a specification for making collections of mail messages available
 >      via HTTP (maybe WebDav) and/or FTP

Things could be as simple as defining an HTML 'rel' value to
point from an archived message in HTML format to the whole archive,
or to use List-Archive and do a robot-like search on URIs that
have the same path as the URI in List-Archive. You are of course
free to write such a specification.

 >    - mail archives that follow the aforementioned specification

I don't expect W3C to provide IMAP access to archives soon, but
there is quite a chance that we will provide negotiated access
to message/rfc822 in the nearer future (can't promise anything
at the moment, but things are looking better than they have in
the past).

 >    and as much as I'd like to believe that email client vendors would
 >    enthusiastically add support for these, market conditions don't seem
 >    to favor supporting new functionality standards in email clients
 >    right now.

I agree it's a tough climate now, but Web browsers were in a similar
situation a couple years ago and the situation is improving now, so
all hope is not lost.

 >3. The obvious compromise that makes sense in the short term (let
 >    archives be in other formats besides message/rfc822, and don't
 >    require message/rfc822 support) is harmful in the long term.

I don't see that. At W3C, we currently don't have message/rfc822
support, but we may be able to add it rather soon, after having
served our archives as HTML only for years.

 >Best compromise I see at this point:  Define some sort of
 >keywords for archived-at.  e.g.
 >
 >Archived-at: "<" URI ">" *(";" keyword [ "=" value ] )
 >
 >where keywords might include
 >
 >"native" 	message available in native message/rfc822 format
 >		(either because that's the only format available
 >                 at this URI or via content-negotiation)
 >
 >"collection"	other messages in the same collection are also
 >	        accessible, where the collection is defined by
 >		the IMAP folder, NNTP newsgroup, FTP directory,
 >		WebDav collection, etc. indicated by the URI.
 >
 >		the value associated with this keyword would
 >		indicate the name of the collection associated
 >		with the URI (since a message might appear in
 >		more than one collection)

This is just totally against Web architecture. Ever seen a link
that tells you beforehand where exactly it's going? You might
be able to guess some things e.g. from file extentions, but
you really only know when you get it (or when you do an HTTP
HEAD if you want to be cautious). Still the Web works amazingly
well.

 >This would (a) let tools record locations of ordinary HTTP/HTML archives
 >such as are (too) often the only format available today,

Yes, that's the primary purpose of Archived-At as used today.

 >(b) encourage
 >archives to provide messages in their original format without requiring
 >them to do so,

The current draft already does so, the original format and
the fact that it doesn't loose information are explicitly
mentioned.

 >(c) give implementors a hint that better functionality
 >can be had,

See above.

 >(d) give email readers a clue as to whether they could
 >actually make use of the URI internally or whether they needed to pass
 >it to a separate browser (this is a common problem in HTML also) and

HTTP HEAD can easily take care of that. Adding hints about what
may be on the other end to URIs hasn't been done up to now, and
for a very good reason: It would make it impossible to update the
archived representation. As an example, immagine W3C would add a
"HTMLonly" value to each of the messages going through one of our
mailing lists. This would reduce the incentive for us to also offer
message/rfc822, because even if we did (and we keep the original
data for all of our archives, that's not the problem), the email
agents out in the field would see "HTMLonly" and not try to get
the available information.

 >(e)
 >leave room for expanded functionality without needing to define a new
 >header field.  And if you don't use any keywords, they don't take up
 >space.

I don't mind another header if it really brings new functionality.

 >It might be better to defer the definition of any keywords to a
 >separate document, because I can imagine needing to define specifics
 >of how collections look within the context of various protocols.
 >I can also imagine needing to define things like collections that
 >consist of a directory of several mbox files, each containing
 >multiple messages.

Yes. Why don't you go ahead with a draft on collections? But
I think the pointer to a collection should not come from
Archived-At, but either from List-Archive or from some information
in the retrieved message or it's (e.g. HTTP) header.

Regards,    Martin. 

Date: Mon, 14 Feb 2005 21:55:22 +0900
To: [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Updating the mailto URI scheme for better I18N
Cc: "Larry Masinter" <[email-masked]>
Sender: [email-masked]

Dear Email Experts,

I have just submitted draft-duerst-mailto-bis-00.txt to the Internet
Drafts Editor. Here's the abstract:

    This document defines the format of Uniform Resource Identifiers
    (URI) for designating electronic mail addresses.  The syntax of
    'mailto' URIs from [RFC2368] is extended to be compatible with IRIs
    ([RFC3987]) for better internationalization.

You can find a full copy at
http://lists.w3.org/Archives/Public/uri/2005Feb/0013.html.

Comments are welcome. The list for discussing URI schemes, including
this one, is [email-masked] (subscribing by sending a mail with subject
'subscribe' to [email-masked]).

Regards,    Martin.

P.S.: Just in case, this already works in at least one browser (Opera) 

To: [email-masked]
Xref: clerew local.mime:4416
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Mandatory From field, anonymity, and hacks
Date: Wed, 2 Feb 2005 11:03:57 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon January 31 2005 16:35, Charles Lindsey wrote:
>> 
>> Have you ever tried sending a message with no Message-ID through sendmail?

>Sendmail acts as an MSA; what's your point (if you have one)?

Sendmail is widely used as an MTA, and has been since long before separate
MSAs were thought of.

May I suggest that you submit a message to this list containing no From
field and no Message-ID field, and let us see exactly what ends up in
various people's mailboxes.

>> False premise, leading to false conclusion.

>Excuse me?

You are excused.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:4417
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Mandatory From field, anonymity, and hacks
Date: Wed, 2 Feb 2005 11:12:25 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon January 31 2005 16:27, Charles Lindsey wrote:

>RFC 2821 is part of the core Internet protocol suite, and
>its section 3.8.4 requirements are indeed serious.
> 
>> But can you
>> point to some realistic scenario (or, better, some actual MUA) in which
>> something might actually break if such a modified header were to arrive at
>> an agent?

>See RFC 2821 section 3.8.4.

That section of RFC 2821 is concerned only with gatewaying. What we are
discussing is ordinary mail produced by ordinary MUAs and remaining in the
SMTP environment until final delivery. Please answer the question with
respect to that environment.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: "Hector Santos" <[email-masked]>
To: "ietf-822" <[email-masked]>
Subject: Re: Mandatory From field, anonymity, and hacks
Date: Tue, 1 Feb 2005 12:36:34 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "Bruce Lilly" <[email-masked]>
To: "ietf-822" <[email-masked]>
Cc: "Charles Lindsey" <[email-masked]>
Sent: Tuesday, February 01, 2005 12:00 PM
Subject: Re: Mandatory From field, anonymity, and hacks

> He did not "suggest[...] the possibility"; he flatly (and incorrectly)
> claimed that none were needed.  But, I suppose this is just another
> instance of the (in)famous Lindsey reading comprehension problem
> rearing its head.

Bruce,  you incorrectly read the statements I made as well so lightning up.
You made a proposal. Be ready for comments

There many different types of software in the loop and I don't think you
have taken into account how they will behave when one or more of the RFC
headers (depending on which one you are based on) is not present.

The fact is most if not all major mail servers do not require a RFC header
to be present in the DATA block.  This is NOT to suggest they are not
required.  The MSA will add an RFC.  The question I raised is what is what
is the expected  behavior or what is deemed a valid RFC header.

I don't wish to go thru the different and conflictive requirements as they
evolved in the last two decades. But you now are proposing a new (relaxed)
requirement, that wasn't there in the first place.

In that vain, your proposals is not harmful.  But again, the difference is
how the downlink software is going to behave.

Do I need to outline the different requirements based on each MAIL Related
RFC as it evolved?

Sit down and write then the functional model for a valid RFC and do it
chronologically.  Let me help you with this starting point:

   RFC 821  SMTP, August 1982
   RFC 822  Mail Format, August 1982
   RFC 1123 Hosting Requirements, October, 1989
   RFC 2046 MIME extensions, November 1996
   RFC 2476 SUBMIT December 1998
   RFC 2821 SMTP, April 2001
   RFC 2822 Mail Format, April 2001

Think of terms on how software that existed since the 80's evolved and keep
in mind the all important 1123 played a very strong role in cleaning up the
requirements for nearly 12 years from 89 to 2001.

I've do it for you if you like.

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: Mandatory From field, anonymity, and hacks
Date: Tue, 1 Feb 2005 12:09:47 -0500
Cc: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

On Mon January 31 2005 16:27, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >On Fri January 28 2005 11:07, Charles Lindsey wrote:

> >> 1. Is it really necessary to modify RFC 822 and well as RFC 2822?
> 
> >Yes.
> 
> Why?

RFC 2026.
 
> As clearly stated in the "Scope" section on RFC 2822, the "Internet
> Message Format" applies first and foremost to "electronic mail".

That is not what RFC 2822 says; it speaks of a "framework", using
language that has remained unchanged from RFC 822, RFC 733, etc.,
several of which predate adoption of the message format by other
RFCs (e.g. 850).

> >That is certainly possible via a number of existing mechanisms
> >(From field with an "effective and useful for replies" [RFC 2821
> >section 3.8.4] mailbox, a standard Comments or Organization (for
> >Usenet specifically) field w/o a  mailbox, or indication in the
> >body of the message), or an extension field which is not an
> >address field could be defined.
> 
> Very droll. I presume those are not intended to be serious suggestions.

RFC 2821 is part of the core Internet protocol suite, and
its section 3.8.4 requirements are indeed serious.
 
> But can you
> point to some realistic scenario (or, better, some actual MUA) in which
> something might actually break if such a modified header were to arrive at
> an agent?

See RFC 2821 section 3.8.4.

> Of course, one
> successful experiment does not prove very much.

To be precise, it proves nothing of consequence.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: Mandatory From field, anonymity, and hacks
Date: Tue, 1 Feb 2005 12:00:32 -0500
Cc: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

On Mon January 31 2005 16:35, Charles Lindsey wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:
> 
> >On Sat January 29 2005 04:23, Hector Santos wrote:

> >No, MTAs are not supposed to alter the message content except
> >to add trace fields (Received, Return-Path).  MSAs are permitted
> >to make specific modifications.
> 
> Have you ever tried sending a message with no Message-ID through sendmail?

Sendmail acts as an MSA; what's your point (if you have one)?
  
> >> Technically, a message needs no headers at all.
> 
> >No, a message must have a message header, and that message
> >header must have at least one header field.  See RFC 822
> >section 4.1, RFC 2822 sections 3.5 & 3.6, and RFC 2046
> >section 5.2.1.
> 
> I see nothing in those places that requires "at least one header field"
> except for the provision that certain header fields are not optional, and
> hence must be present.

Duh.

> But since you seem prepared to reduce the number of 
> required header fields, then you are in no position to object when Hector
> suggests the possibility of reducing them to zero.

He did not "suggest[...] the possibility"; he flatly (and incorrectly)
claimed that none were needed.  But, I suppose this is just another
instance of the (in)famous Lindsey reading comprehension problem
rearing its head.

> And RFC 2046 section 5.2.1 says nothing about headers of messages.

It addresses MIME-part headers.
 
> >If that's an SMTP question, yes, because SMTP transfers
> >messages, and the message format (822/2822) requires a
> >header with certain fields.
> 
> False premise, leading to false conclusion.

Excuse me?