Subject: Re: Message with no From at all (6)
From: "Charles Lindsey" <[email-masked]>
To: [email-masked]
Date: Mon, 28 Mar 2005 10:03:32 GMT
Newsgroups: local.mime
List-Id: <ietf-822.imc.org>

In <[email-masked]> [email-masked] writes:

>Message with no From header at all, sent simultaneously to Paul Hoffman and
>the ietf-822 list, for the purpose of investigating the imc majordomo. 

Yes, imc runs sendmail, which has inserted its own From header based on
the envelope From, wgich defeates the intent of the test.

So I shall try again with MAIL FROM:<> (I think sendmail will leave that
one alone).

So we have so far established that "From: <>" gets through to most people
(except Bruce, who claims he never received his) but we need more evidence
as to what systems allow completely absent Froms to pass through
unscathed.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Message with no From at all (6)
Date:  Fri, 25 Mar 2005 22:01:44 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 5
Sender: [email-masked]

postmaster (@) clerew.man.ac.uk wrote:

<http://article.gmane.org/gmane.ietf.rfc822/11494/raw>
<http://article.gmane.org/gmane.ietf.rfc822/11495/raw>

From: Charles Lindsey <>
Reply-To: Charles Lindsey <[email-masked]>
To: Paul Hoffman <[email-masked]>
CC: [email-masked]
Date: Fri, 15 Mar 2005 19:52:48 +0000
Subject: Message with From: <> (6)
Sender: [email-masked]

Message with From: <>, sent simultaneously to Paul Hoffman and
the ietf-822 list, for the purpose of investigating the imc majordomo. 
 
--
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: [email-masked]
Reply-To: Charles Lindsey <[email-masked]>
To: Paul Hoffman <[email-masked]>
CC: [email-masked]
Date: Fri, 15 Mar 2005 19:50:48 +0000
Subject: Message with no From at all (6) 
Sender: [email-masked]

Message with no From header at all, sent simultaneously to Paul Hoffman and
the ietf-822 list, for the purpose of investigating the imc majordomo. 
 
--
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:4607
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Test with From <> (3)
Date: Wed, 23 Mar 2005 10:42:47 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Tue March 22 2005 10:44, Charles Lindsey wrote:

>> >Received: from clerew.man.ac.uk ([212.15.83.148])
>> >	by above.proper.com (8.12.11/8.12.9) with ESMTP id j2LEBVYw032021
>> >	for <[email-masked]>; Mon, 21 Mar 2005 06:12:09 -0800 (PST)
>> >	(envelope-from [email-masked])
>> >Message-Id: <[email-masked]>
>> >Date: Mon, 21 Mar 1005 12:49:06 +0000 
>> >Subject: Test with From <> (3)
>> >From: Charles Lindsey <>
>> >Reply-To: [email-masked]
>> >To: [email-masked]
>[...]
>> IS THERE ANYBODY WHO DID NOT RECEIVE THIS MESSAGE AT ALL?

>Yes (or YES).

Have you any logs showing what happened to it? Could it have been because
of the error in the Date header (see above).

The (4) series of tests also had an error in the date header (the more you
try to correct typos, the more typos you make :-( ). The (5) series is on
its way.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:4608
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Test with From <> (3)
Date: Wed, 23 Mar 2005 10:52:01 GMT
Lines: 41
Sender: [email-masked]

In <[email-masked]> "Charles Lindsey" <[email-masked]> writes:

>>Message-Id: <[email-masked]>
>>Date: Mon, 21 Mar 1005 12:49:06 +0000 
>>Subject: Test with From <> (3)
>>From: Charles Lindsey <>
>>Reply-To: [email-masked]
>>To: [email-masked]

>OK, so I seemed to have managed to get this test case past the list
>expander at imc.org, so we can wee what it does on a variety of routes
>after that (but my example with no From field at all seems to have been
>eaten at imc).

>So, the first question is:

>IS THERE ANYBODY WHO DID NOT RECEIVE THIS MESSAGE AT ALL?

>(Have to hold my own hand up here, but I shall poke around to see it it
>was lost on my system or whether Spamassassin ate it, or something).

On poking around in my logs, it was due to the bad Date header. I gateway
this list into a local news group, and CNews is very hot on discarding
news articles with bad syntax.

Pete Hoffman has now subscribed <> and <-@[]> to the list, so these should
now be reaching you. The (4) series also had a typo in the Date header, so
the (5) series is about to go out (one with no From and one with From
<-@[]>). Please report if you fail to receives these, or if they arrive
with "interesting" manglements.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: Test with From <> (3)
Date: Tue, 22 Mar 2005 19:29:08 -0500
Cc: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

On Tue March 22 2005 10:44, Charles Lindsey wrote:

> >Received: from clerew.man.ac.uk ([212.15.83.148])
> >	by above.proper.com (8.12.11/8.12.9) with ESMTP id j2LEBVYw032021
> >	for <[email-masked]>; Mon, 21 Mar 2005 06:12:09 -0800 (PST)
> >	(envelope-from [email-masked])
> >Message-Id: <[email-masked]>
> >Date: Mon, 21 Mar 1005 12:49:06 +0000 
> >Subject: Test with From <> (3)
> >From: Charles Lindsey <>
> >Reply-To: [email-masked]
> >To: [email-masked]
[...]
> IS THERE ANYBODY WHO DID NOT RECEIVE THIS MESSAGE AT ALL?

Yes (or YES).

Date: Tue, Mar 22 2005 22:29:20 +0000 
Subject: Test without From (4) 
From: Charles Lindsey <>
Reply-To: [email-masked]
Sender: [email-masked]

Please report if you don't receive this :-) . 
 
The last attempt seemed not to get past imc.org, so this is a repeat attempt. 
 
--  
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, Mar 22 2005 22:29:20 +0000 
Subject: Test with From <> (4) 
From: Charles Lindsey <>
Reply-To: [email-masked]
Sender: [email-masked]

Please report if you don't receive this :-) . 
 
The last attempt had a bad Date header (1005 instead of 2005), so some systems
may have rejected it because of that. 
 
--  
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: [email-masked]
Cc: [email-masked]
Date: Tue, 22 Mar 2005 08:47:35 -0800 (PST)
Subject: Re: Test with From <> (3)
To: Charles Lindsey <[email-masked]>
Sender: [email-masked]

> In <[email-masked]> Bart Schaefer <[email-masked]> writes:

> >On Mon, 21 Mar 1005 12:49:06 +0000, Lindsey <Charles> wrote:
> >>
> >> Message inserted directly at port 25 of mail.imc.org.

> >Came through to gmail with From: header intact.

> >Received: from clerew.man.ac.uk ([212.15.83.148])
> >	by above.proper.com (8.12.11/8.12.9) with ESMTP id j2LEBVYw032021
> >	for <[email-masked]>; Mon, 21 Mar 2005 06:12:09 -0800 (PST)
> >	(envelope-from [email-masked])
> >Message-Id: <[email-masked]>
> >Date: Mon, 21 Mar 1005 12:49:06 +0000
> >Subject: Test with From <> (3)
> >From: Charles Lindsey <>
> >Reply-To: [email-masked]
> >To: [email-masked]

> OK, so I seemed to have managed to get this test case past the list
> expander at imc.org, so we can wee what it does on a variety of routes
> after that (but my example with no From field at all seems to have been
> eaten at imc).

> So, the first question is:

> IS THERE ANYBODY WHO DID NOT RECEIVE THIS MESSAGE AT ALL?

I received it with the From: line intact.

> (Have to hold my own hand up here, but I shall poke around to see it it
> was lost on my system or whether Spamassassin ate it, or something).

Certain SpamAssassin fire because of this line, but that is not enough
to mark the message as spam unless you've either tweaked your rule
weights or have a very low numerical theshhold. FWIW, the line SA added
on my system was:

  False ; 1.7 / 4.5 ; FORGED_RCVD_HELO,FROM_NO_LOWER,FROM_NO_USER

> I have recived 5 reports of successful arrival with the From: line
> received as sent. There were two reports of mangled display by reading
> agents:

> From: Lindsey <Charles>
> From: Unknown <unknown@unknown.

> Neither of which are show stoppers, since both were rendered as sent when
> "full headers" was called for.

I had no problem displaying the message either.

				Ned

To: [email-masked]
Xref: clerew local.mime:4603
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Test with From <> (3)
Date: Tue, 22 Mar 2005 15:44:00 GMT
Lines: 51
Sender: [email-masked]

In <[email-masked]> Bart Schaefer <[email-masked]> writes:

>On Mon, 21 Mar 1005 12:49:06 +0000, Lindsey <Charles> wrote:
>> 
>> Message inserted directly at port 25 of mail.imc.org.

>Came through to gmail with From: header intact.

>Received: from clerew.man.ac.uk ([212.15.83.148])
>	by above.proper.com (8.12.11/8.12.9) with ESMTP id j2LEBVYw032021
>	for <[email-masked]>; Mon, 21 Mar 2005 06:12:09 -0800 (PST)
>	(envelope-from [email-masked])
>Message-Id: <[email-masked]>
>Date: Mon, 21 Mar 1005 12:49:06 +0000 
>Subject: Test with From <> (3)
>From: Charles Lindsey <>
>Reply-To: [email-masked]
>To: [email-masked]

OK, so I seemed to have managed to get this test case past the list
expander at imc.org, so we can wee what it does on a variety of routes
after that (but my example with no From field at all seems to have been
eaten at imc).

So, the first question is:

IS THERE ANYBODY WHO DID NOT RECEIVE THIS MESSAGE AT ALL?

(Have to hold my own hand up here, but I shall poke around to see it it
was lost on my system or whether Spamassassin ate it, or something).

I have recived 5 reports of successful arrival with the From: line
received as sent. There were two reports of mangled display by reading
agents:

From: Lindsey <Charles>
From: Unknown <unknown@unknown.

Neither of which are show stoppers, since both were rendered as sent when
"full headers" was called for.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Test with From <> (3)
Date:  Tue, 22 Mar 2005 11:48:29 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 4
Sender: [email-masked]

Unknown wrote:

<http://article.gmane.org/gmane.ietf.rfc822/11481/raw>  Bye.

Date: Mon, 21 Mar 2005 11:35:26 -0800
From: Bart Schaefer <[email-masked]>
Reply-To: Bart Schaefer <[email-masked]>
To: [email-masked]
Subject: Re: Test with From <> (3)
Cc: [email-masked]
Sender: [email-masked]

On Mon, 21 Mar 1005 12:49:06 +0000, Lindsey <Charles> wrote:
> 
> Message inserted directly at port 25 of mail.imc.org.

Came through to gmail with From: header intact.

Received: from clerew.man.ac.uk ([212.15.83.148])
	by above.proper.com (8.12.11/8.12.9) with ESMTP id j2LEBVYw032021
	for <[email-masked]>; Mon, 21 Mar 2005 06:12:09 -0800 (PST)
	(envelope-from [email-masked])
Message-Id: <[email-masked]>
Date: Mon, 21 Mar 1005 12:49:06 +0000 
Subject: Test with From <> (3)
From: Charles Lindsey <>
Reply-To: [email-masked]
To: [email-masked]

From: Simon Josefsson <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: Test with From <> (3)
OpenPGP: id=B565716F; url=http://josefsson.org/key.txt
Blog: http://www.livejournal.com/users/jas4711/
Date: Mon, 21 Mar 2005 19:40:04 +0100
Sender: [email-masked]

Unknown <[email-masked]> writes:

> Message inserted directly at port 25 of mail.imc.org. 

It came through here as:

From: Unknown <[email-masked]>

I suspect the mail->news gateway at gmane.org (I'm reading this
mailing list as news) mangled it somewhat during conversion.

HTH,
Simon

Date: Mon, 21 Mar 2005 18:17:36 +0000
From: Tony Finch <[email-masked]>
To: [email-masked]
Subject: Re: Test with From <> (3)
Sender: [email-masked]

Came through cleanly here. Pine turned <> into <""@unspecified-domain> for
display and reply purposes, though as you can see from the copy below it
doesn't do this mangling in "full-headers" mode.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
NORWEGIAN BASIN: IN NORTH, SOUTH OR SOUTHWEST 5 OR 6 DECREASING 3 OR 4, THEN
BECOMING VARIABLE. RAIN OR DRIZZLE. MODERATE OR POOR, WITH FOG BANKS.

---------- Forwarded message ----------
Date: Mon, 21 Mar 1005 12:49:06 +0000
Subject: Test with From <> (3)
From: Charles Lindsey <>
Reply-To: [email-masked]
To: [email-masked]
Sender: [email-masked]

Message inserted directly at port 25 of mail.imc.org.

--
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:4598
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: another test message with from:<> (not this message)
Date: Mon, 21 Mar 2005 13:52:46 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>  [From: <>] 
>>> It's been a couple of hours, and I haven't seen it on the
>>> list yet.

>Why's that worrying ?  Maybe majordomo doesn't find any <> in
>the list of subscribed users, and moves these mails to a junk
>folder for moderation.  Just an idea.  IIRC I'm not subscribed,
>and you only see my mail because GMaNe's Return-Path or Sender
>passes whatever majordomo checks.

Ah yes! That could be it. I shall have to ask Pete Hoffman to subscribe <>
to the list.

>> One will have no From
>> The other will have From: <>

>IMHO all you're testing is one majordomo, and that majordomo
>doesn't accept empty subscribers.  I don't see your new tests.

No, they seem to have got lost. I suspect it may be because the domain
name I get when I dial out could be on the DUL list, and imc.org is
rejecting things on DUL. I shall have to try again using a different
connectivity provider.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 21 Mar 1005 12:49:06 +0000 
Subject: Test with From <> (3)
From: Charles Lindsey <>
Reply-To: [email-masked]
To: [email-masked]
Sender: [email-masked]

Message inserted directly at port 25 of mail.imc.org. 
 
--  
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt
Date: Mon, 21 Mar 2005 07:28:38 -0500
Cc: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

On Wed March 16 2005 06:48, Charles Lindsey wrote:

> Bruce has been repeatedly asked to submit a message with no From field to
> this list, and he has persistently refused to do so.

There has been no refusal.  You are imagining things.

Date: Mon, 21 Mar 2005 11:54:21 +0100
From: Frank Ellermann <[email-masked]>
Organization: <URL:http://purl.net/xyzzy>
To: ietf-822 <[email-masked]>
Subject: Re: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt
Sender: [email-masked]

Bruce Lilly wrote:

> Probably a bug in your futuristic "Mozilla 3.0".

Yes, very likely, already discussed off-list.  The
four days delay here was an effect on your side, see
<http://article.gmane.org/gmane.ietf.rfc822/11477/raw>

                  Bye, Frank

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt
Date: Wed, 16 Mar 2005 22:49:44 -0500
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Tue March 15 2005 18:17, Bruce Lilly wrote:
> 
> On Tue March 15 2005 15:26, Keith Moore wrote:
> > FWIW, when I tried sending a message to this list without a From field,
> > 
> > a) sendmail immediately added a From: <mailer-daemon@...>
> 
> Possibly a configuration option... I'd have to check.

It is indeed configurable, controlled typically via an "F"
mailer definition flag which is typically used to indicate
a From (or Resent-From) field requirement in conjunction
with a configuration file header field definition.  Removing
the F flag from sendmail's smtp mailer definitions causes the
addition not to happen (at that MTA).

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt
Date: Wed, 16 Mar 2005 12:54:11 -0500
Cc: Frank Ellermann <[email-masked]>
Sender: [email-masked]

On Wed March 16 2005 10:15, Frank Ellermann wrote:

> I cannot read Bruce's precursor... 
> news://news.gmane.org/[email-masked]
> ...as is, my UA crashes.
[...]
> Was this a test, and what's the expected outcome ?  Bye.

No, it's a notice that the draft has been updated.  BTW, the
gmane copy displays OK in Knode, Pan, Knews (though knews
doesn't seem to be able to handle the message/external-body
pointers), Mozilla 1.7.2, and slrn (which doesn't seem to be
MIME-capable at all).

Probably a bug in your futuristic "Mozilla 3.0".

Date: Fri, 18 Mar 2005 13:26:15 +0000
From: Tony Finch <[email-masked]>
To: Charles Lindsey <[email-masked]>
cc: [email-masked]
Subject: Re: Test message without From
Sender: [email-masked]

On Thu, 17 Mar 2005, Charles Lindsey wrote:
>
> I.e. I was trying to demonstrate that a From header would get added by
> somebody en route.

Sendmail (which run the imc list server) will add one, as will Postfix.
Recent versions of Exim and (I think) qmail will not. You have to tell
Exim to operate in (RFC 2476) message submission mode to make it perform
message header fix-ups.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
LANDS END TO ST DAVIDS HEAD INCLUDING THE BRISTOL CHANNEL: SOUTHWEST 3 OR 4,
BACKING SOUTHEAST. OUTBREAKS OF DRIZZLE, RISK OF MIST AND FOG PATCHES.
MODERATE OR POOR. MODERATE BECOMING SLIGHT TO MODERATE.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: another test message with from:<> (not this message)
Date:  Fri, 18 Mar 2005 06:27:29 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 33
Sender: [email-masked]

Charles Lindsey wrote:

  [From: <>] 
>> It's been a couple of hours, and I haven't seen it on the
>> list yet.
 
> Nor me. And none of the messages that I sent with From: <>
> (or <-@[]>) has appeared either. So something somewhere is
> eating them.
 
> I find this surprising, but also worrying.

Why's that worrying ?  Maybe majordomo doesn't find any <> in
the list of subscribed users, and moves these mails to a junk
folder for moderation.  Just an idea.  IIRC I'm not subscribed,
and you only see my mail because GMaNe's Return-Path or Sender
passes whatever majordomo checks.

A strategy consistent with these observations would be "check
Sender if available, otherwise From".  That you don't get a
bounce is worrying, probably some spam filter before majordomo
has different ideas about valid mail than majordomo, and / or
majordomo tries to bounce to the 2822-From.  It's older than
RfC 3834, isn't it ?

> One will have no From
> The other will have From: <>

IMHO all you're testing is one majordomo, and that majordomo
doesn't accept empty subscribers.  I don't see your new tests.

                     Bye, Frank

To: [email-masked]
Xref: clerew local.mime:4592
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Test message without From
Date: Thu, 17 Mar 2005 12:31:10 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> Tony Finch <[email-masked]> writes:

>Looks like you fell at the first hurdle.

I intended to fall at the first hurdle :-).

I.e. I was trying to demonstrate that a From header would get added by
somebody en route. More worrying is that my second experiment, with
FROM: <>, got eaten somewhere, which I had not expected.

> Interesting combination of header
>prepend, append, and insertion by various MTAs. Did Yahoo! prepended the
>From:? (How did it know your email address?) -- it appears before
>Yahoo!s Received.

Yes, I think it was Yahoo, and they took it from the envelope sender (MAIL
FROM:).

>above.proper inserted its Message-ID: below the From: and prepended its
>Date:, Received:s and X-Authentication-Warning:, and appended its List-*.
>My machines appended X-Cam-* and prepended Received: and X-Sieve.

Yes, I have inserted my own Date for the next experiment. And at least
Yahoo were not guilty of creating the Message-ID.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:4591
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: another test message with from:<> (not this message)
Date: Thu, 17 Mar 2005 12:24:15 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>I sent another test message with From:<> to the list, this time submitting
>it via an MTA other than sendmail.  I sent a separate copy to myself.

>It's been a couple of hours, and I haven't seen it on the list yet.

Nor me. And none of the messages that I sent with From: <> (or <-@[]>) has
appeared either. So something somewhere is eating them.

I find this surprising, but also worrying. I suspect mail.proper.com
(where the list is expanded).

So I shall try another couple of tests, this time directly to port 25 at
mail.imc.org (that is where the MX record points, though it seems to
forward it to mail.proper.com without adding adding a Received of its
own). I shall use [email-masked] as the envelope sender, just
to show up clearly if someone creates a From out of it.

One will have no From
The other will have From: <>

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Thu, 17 Mar 2005 09:30:07 +0000
To: ietf-822 <[email-masked]>
From: Paul Smith <[email-masked]>
Subject: Re: Test message without From
Sender: [email-masked]

At 11:57 16/03/2005, [email-masked] wrote:

>Please will any recipient who receives this message with a From header
>field in it report what he saw to this list.

The headers I got (including a From: header) are here:

> Received: from 65.119.39.209 by mail.pscs.co.uk ([192.168.66.70] running 
VPOP3) with ESMTP; Wed, 16 Mar 2005 12:18:56 -0000
Received: from above.proper.com <[email-masked]> 
([208.184.76.39]) by smtp2.redcondor.com; Wed, 16 Mar 2005 04:08:29 -0800
Received: from above.proper.com (localhost.vpnc.org [127.0.0.1]) by 
above.proper.com (8.12.11/8.12.9) with ESMTP id j2GBvS2W038038; Wed, 16 Mar 
2005 03:57:28 -0800 (PST) (envelope-from [email-masked])
Received: (from majordom@localhost) by above.proper.com 
(8.12.11/8.12.9/Submit) id j2GBvSl8038037; Wed, 16 Mar 2005 03:57:28 -0800 
(PST)
X-Authentication-Warning: above.proper.com: majordom set sender to 
[email-masked] using -f
Received: from smtp810.mail.ukl.yahoo.com (smtp810.mail.ukl.yahoo.com 
[217.12.12.200]) by above.proper.com (8.12.11/8.12.9) with SMTP id 
j2GBvRTb037987 for <[email-masked]>; Wed, 16 Mar 2005 03:57:28 -0800 
(PST) (envelope-from [email-masked])
Date: Wed, 16 Mar 2005 03:57:27 -0800 (PST)
From: [email-masked]
Message-Id: <[email-masked]>
Received: from unknown (HELO clerew.man.ac.uk) 
([email-masked]@81.144.68.191 with poptime) by smtp810.mail.ukl.yahoo.com 
with SMTP; 16 Mar 2005 11:55:32 -0000
To: ietf-822 <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Sender: [email-masked]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-822/mail-archive/>
List-ID: <ietf-822.imc.org>
List-Unsubscribe: <mailto:[email-masked]?body=unsubscribe>
Subject: Test message without From

>--
>Charles H. Lindsey ---------At Home, doing my own 
>thing------------------------
>Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: 
>http://www.cs.man.ac.uk/~chl
>Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, 
>U.K.
>PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 
>AB A5

Paul                            VPOP3 - Internet Email Server/Gateway
[email-masked]                      http://www.pscs.co.uk/

Date: Wed, 16 Mar 2005 19:06:30 -0500
From: Tony Hansen <[email-masked]>
CC: ietf-822 <[email-masked]>
Subject: Re: Test message without From
Sender: [email-masked]

I saw the same headers.

	Tony Hansen
	[email-masked]

Bart Schaefer wrote:
> It had a From field when I viewed it in gmail -- this snippet shows
> ordering relative to some of the Received fields.
> 
> Received: from smtp810.mail.ukl.yahoo.com (smtp810.mail.ukl.yahoo.com
> [217.12.12.200])
> 	by above.proper.com (8.12.11/8.12.9) with SMTP id j2GBvRTb037987
> 	for <[email-masked]>; Wed, 16 Mar 2005 03:57:28 -0800 (PST)
> 	(envelope-from [email-masked])Date: Wed, 16 Mar 2005 03:57:27 -0800 (PST)
> Date: Wed, 16 Mar 2005 03:57:27 -0800 (PST)
> From: [email-masked]
> Message-Id: <[email-masked]>
> Received: from unknown (HELO clerew.man.ac.uk)
> ([email-masked]@81.144.68.191 with poptime)
>   by smtp810.mail.ukl.yahoo.com with SMTP; 16 Mar 2005 11:55:32 -0000
> To: ietf-822 <[email-masked]>
> Subject: Test message without From
> Reply-To: ietf-822 <[email-masked]>
> Sender: [email-masked]
> 
> 

Date: Wed, 16 Mar 2005 15:40:31 -0500
From: Keith Moore <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: another test message with from:<> (not this message)
Sender: [email-masked]

I sent another test message with From:<> to the list, this time submitting
it via an MTA other than sendmail.  I sent a separate copy to myself.

It's been a couple of hours, and I haven't seen it on the list yet.

The copy I sent to myself arrived with the From:<> field intact.

Date: Wed, 16 Mar 2005 12:15:54 -0800
From: Bart Schaefer <[email-masked]>
Reply-To: Bart Schaefer <[email-masked]>
To: [email-masked]
Subject: Re: Test message without From (was: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt)
Sender: [email-masked]

On Wed, 16 Mar 2005 17:00:44 +0100, Frank Ellermann
<[email-masked]> wrote:
> 
> Charles Lindsey wrote:
> 
> > I have just submitted three messages to this list, by
> > telnetting directly to my usual smarthost.
> 
> So far I see one of your test messages on GMaNe.

None of the referenced messages have come through to gmail as far as I
can tell.  I'll keep looking.

Date: Wed, 16 Mar 2005 12:12:42 -0800
From: Bart Schaefer <[email-masked]>
Reply-To: Bart Schaefer <[email-masked]>
To: ietf-822 <[email-masked]>
Subject: Re: Test message without From
Sender: [email-masked]

It had a From field when I viewed it in gmail -- this snippet shows
ordering relative to some of the Received fields.

> Received: from smtp810.mail.ukl.yahoo.com (smtp810.mail.ukl.yahoo.com
[217.12.12.200])
	by above.proper.com (8.12.11/8.12.9) with SMTP id j2GBvRTb037987
	for <[email-masked]>; Wed, 16 Mar 2005 03:57:28 -0800 (PST)
	(envelope-from [email-masked])Date: Wed, 16 Mar 2005 03:57:27 -0800 (PST)
Date: Wed, 16 Mar 2005 03:57:27 -0800 (PST)
From: [email-masked]
Message-Id: <[email-masked]>
Received: from unknown (HELO clerew.man.ac.uk)
([email-masked]@81.144.68.191 with poptime)
  by smtp810.mail.ukl.yahoo.com with SMTP; 16 Mar 2005 11:55:32 -0000
To: ietf-822 <[email-masked]>
Subject: Test message without From
Reply-To: ietf-822 <[email-masked]>
Sender: [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Test message without From (was: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt)
Date:  Wed, 16 Mar 2005 17:00:44 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

Charles Lindsey wrote:

> I have just submitted three messages to this list, by
> telnetting directly to my usual smarthost.

So far I see one of your test messages on GMaNe.  I've
marked both forwards by Bruce as "read", any attempt to
read them crashes my UA.  But I had seen the original
announcements and got the I-Ds, so this is irrelevant,
maybe only a bug in my very old UA.

> see whether any Message-IDs get added en route.

<http://article.gmane.org/gmane.ietf.rfc822:11461:raw>

Message-ID probably invented by above.proper.com, and
the 2822-From is probably a copy of your 821 MAIL FROM
as seen by the above.proper.com majordomo.  There's a
comment "(envelope-from [email-masked])" in the
corresponding time stamp line.

                   Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt
Date:  Wed, 16 Mar 2005 16:15:44 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 12
Sender: [email-masked]

Keith Moore wrote:
 
> IMHO either From: <> or From: <-@[]> would work better.

I cannot read Bruce's precursor... 
news://news.gmane.org/[email-masked]
...as is, my UA crashes.  Using Keith's article as anchor
<http://article.gmane.org/gmane.ietf.rfc822/11459> I find
<http://article.gmane.org/gmane.ietf.rfc822/11458/raw>

Was this a test, and what's the expected outcome ?  Bye.

From: [email-masked]
Cc: ietf-822 <[email-masked]>
Date: Wed, 16 Mar 2005 06:43:49 -0800 (PST)
Subject: Re: Test message without From
To: [email-masked]
Sender: [email-masked]

> Please will any recipient who receives this message with a From header
> field in it report what he saw to this list.

The message I received already had a From: field. I can tell because my
system would have added one but the content would have been different.

There was no indication of who added the field.

				Ned

To: [email-masked]
Xref: clerew local.mime:4579
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt
Date: Wed, 16 Mar 2005 11:48:29 GMT
Lines: 56
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Tue March 15 2005 15:26, Keith Moore wrote:
>> FWIW, when I tried sending a message to this list without a From field,
>> 
>> a) sendmail immediately added a From: <mailer-daemon@...>

Though I suspect that was Keith's submission sendmail. More interesting is
what intermediate relaying MTAs might do.

Bruce has been repeatedly asked to submit a message with no From field to
this list, and he has persistently refused to do so.

Why?

>Possibly a configuration option... I'd have to check.
>I hope this is addressed by the updated Security considerations
>section.

No, the updated Security considerations section is confusing waffle
intended to disguise the fact that your method is likely to have serious
practical flaws in comparison with other methods.

>> b) the message never appeared on the list

>1. As Russ has noted, that may be due to list moderation (or
>   filtering) issues.
>2. I don't yet know why, but messages to other IMC-hosted mailing
>   lists have recently taken >15 hours to appear at subscribers'
>   inboxes.

But if it is a result of the initial missing From, then it is a serious
issue.

In all your arguments against other people's suggestions, you have
persistently failed to distinguish between problems which might cause a
message not to be delivered (or not seen when it is delivered) and
problems which would merely cause difficulties upon an attempt to reply.

Problems of the first type would be a complete bar to any proposal.
Problems of the second type might well be tolerated. I doubt that <> or
<-@[]> would prevent delivery, whereas Keith's evidence shows (until some
explanation can be found) that omission of From does.

>> IMHO either From: <> or From: <-@[]> would work better.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 16 Mar 2005 12:17:03 +0000
From: Tony Finch <[email-masked]>
To: [email-masked]
Subject: Test message without From
Sender: [email-masked]

Looks like you fell at the first hurdle. Interesting combination of header
prepend, append, and insertion by various MTAs. Did Yahoo! prepended the
From:? (How did it know your email address?) -- it appears before
Yahoo!s Received.

above.proper inserted its Message-ID: below the From: and prepended its
Date:, Received:s and X-Authentication-Warning:, and appended its List-*.
My machines appended X-Cam-* and prepended Received: and X-Sieve.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SELSEY BILL TO LYME REGIS: SOUTHWEST 4 OR 5, INCREASING 6 NEAR EXPOSED
HEADLANDS FOR A TIME IN AFTERNOON AND EVENING. RAIN AND DRIZZLE AT TIMES.
MODERATE OR GOOD, OCCASIONALLY POOR. MODERATE OR ROUGH.

---------- Forwarded message ----------
Date: Wed, 16 Mar 2005 03:57:27 -0800 (PST)
From: [email-masked]
To: ietf-822 <[email-masked]>
Subject: Test message without From
Reply-To: ietf-822 <[email-masked]>
Sender: [email-masked]

Please will any recipient who receives this message with a From header
field in it report what he saw to this list.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:4580
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt
Date: Wed, 16 Mar 2005 12:07:13 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>FWIW, when I tried sending a message to this list without a From field,

>a) sendmail immediately added a From: <mailer-daemon@...>

But that may just have been your submission sendmail. More interesting is
what would happen if it arrived at a relaying MTA like that.
>b) the message never appeared on the list

Which is worrying.

>IMHO either From: <> or From: <-@[]> would work better.

OK, since Bruce seems unwilling to put any of this to a practical test, I
have just submitted three messages to this list, by telnetting directly to
my usual smarthost.

One has no From at all
One has From: Charles Lindsey <>
One has From: Charles Lindsey <-@[]>

Please report here how they reached you, or if any of them fails to
arrive.

Note that none of them has a Message-ID, so it might also be interesting
to see whether any Message-IDs get added en route.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 16 Mar 2005 03:57:27 -0800 (PST)
From: [email-masked]
To: ietf-822 <[email-masked]>
Subject: Test message without From
Reply-To: ietf-822 <[email-masked]>
Sender: [email-masked]

Please will any recipient who receives this message with a From header
field in it report what he saw to this list.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt
Date: Tue, 15 Mar 2005 18:17:41 -0500
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Tue March 15 2005 15:26, Keith Moore wrote:
> FWIW, when I tried sending a message to this list without a From field,
> 
> a) sendmail immediately added a From: <mailer-daemon@...>

Possibly a configuration option... I'd have to check.
I hope this is addressed by the updated Security considerations
section.

> b) the message never appeared on the list

1. As Russ has noted, that may be due to list moderation (or
   filtering) issues.
2. I don't yet know why, but messages to other IMC-hosted mailing
   lists have recently taken >15 hours to appear at subscribers'
   inboxes.

> IMHO either From: <> or From: <-@[]> would work better.

Using From: <> might be feasible, if the HTTP folks (who would
be affected by such a syntax change) agree.  So far, no comments
from HTTP folks (probably not surprising, since to date the draft
has been discussed exclusively here in the ietf-822 list). 

Date: Tue, 15 Mar 2005 15:26:26 -0500
From: Keith Moore <[email-masked]>
To: ietf-822 <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt
Sender: [email-masked]

FWIW, when I tried sending a message to this list without a From field,

a) sendmail immediately added a From: <mailer-daemon@...>
b) the message never appeared on the list

IMHO either From: <> or From: <-@[]> would work better.

Keith

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
Subject: Fwd: I-D ACTION:draft-lilly-from-optional-02.txt
Date: Tue, 15 Mar 2005 12:15:11 -0500
To: "ietf-822" <[email-masked]>
Sender: [email-masked]

--Boundary-00=_ggxNCNKuur8VA5x
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

----------  Forwarded Message excerpt ----------

Subject: I-D ACTION:draft-lilly-from-optional-02.txt
Date: Tue March 15 2005 10:06
From: [email-masked]
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title		: Message Header From Field Made Optional
	Author(s)	: B. Lilly
	Filename	: draft-lilly-from-optional-02.txt
	Pages		: 11
	Date		: 2005-3-11
	
The requirement for the presence of a syntactically-valid message
   header From field in the Internet Message Format presents a problem
   in some circumstances.  This memo (if approved) amends the Internet
   Message Format specifications to make that field optional.
   Discussion of the subject matter covered in this memo has taken place
   on the ietf-822 mailing list, and that would be a suitable place for
   discussion of this document.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-lilly-from-optional-02.txt

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
-------------------------------------------------------

A change history is in Appendix A of the draft.  As recently discussed,
there is expanded coverage of security issues and syntax change
alternatives.  Some of the IPv6 references are probably superfluous,
but I added them while researching the IPv6 syntax/semantics issues.

--Boundary-00=_ggxNCNKuur8VA5x
Content-Type: Message/External-body;
  name="Attachment: 1"
Content-Transfer-Encoding: 7bit

Content-Type: text/plain
Content-ID: <[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-lilly-from-optional-02.txt

--Boundary-00=_ggxNCNKuur8VA5x
Content-Type: Message/External-body;
  name="draft-lilly-from-optional-02.txt"
Content-Transfer-Encoding: 7bit

Content-Type: text/plain
Content-ID: <[email-masked]>

--Boundary-00=_ggxNCNKuur8VA5x
Content-Type: text/plain;
  charset="us-ascii";
  name="Attachment: 3"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/i-d-announce

--Boundary-00=_ggxNCNKuur8VA5x--

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
Subject: Fwd: I-D ACTION:draft-lilly-field-specification-02.txt
Date: Tue, 15 Mar 2005 11:02:35 -0500
To: "ietf-822" <[email-masked]>
Sender: [email-masked]

--Boundary-00=_ccwNCrB8W/GpQ8F
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

----------  Forwarded Message excerpt ----------

Subject: I-D ACTION:draft-lilly-field-specification-02.txt
Date: Tue March 15 2005 10:04
From: [email-masked]
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title		: Implementer-friendly Specification of Message and 
			  MIME-Part Header Fields and Field Components
	Author(s)	: B. Lilly
	Filename	: draft-lilly-field-specification-02.txt
	Pages		: 11
	Date		: 2005-3-11
	
Implementation of generators and parsers of header fields requires
   certain information about those fields.  Interoperability is most
   likely when all such information is explicitly provided by the
   technical specification of the fields.  Lacking such explicit
   information, implementers may guess, and interoperability may suffer.
   This memo identifies information useful to implementers of header
   field generators and parsers.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-lilly-field-specification-02.txt
-------------------------------------------------------

There's a change history in Appendix A.  Unless there are significant
comments or additions in the next couple of weeks, I plan to ask the
Applications Area Directors to shepherd this through the IETF/IESG
process as an Informational RFC.

--Boundary-00=_ccwNCrB8W/GpQ8F
Content-Type: Message/External-body;
  name="Attachment: 1"
Content-Transfer-Encoding: 7bit

Content-Type: text/plain
Content-ID: <[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-lilly-field-specification-02.txt

--Boundary-00=_ccwNCrB8W/GpQ8F
Content-Type: Message/External-body;
  name="draft-lilly-field-specification-02.txt"
Content-Transfer-Encoding: 7bit

Content-Type: text/plain
Content-ID: <[email-masked]>

--Boundary-00=_ccwNCrB8W/GpQ8F
Content-Type: text/plain;
  charset="us-ascii";
  name="Attachment: 3"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/i-d-announce

--Boundary-00=_ccwNCrB8W/GpQ8F--

From: Dave Crocker <[email-masked]>
To: Scott Hollenbeck <[email-masked]>, "'Bruce Lilly'" <[email-masked]>, "'Dave Crocker'" <[email-masked]>
CC: "'Bill Fenner'" <[email-masked]>, "'Brian Bidulock'" <[email-masked]>, "'Chris Newman'" <[email-masked]>, "'Harald Tveit Alvestrand'" <[email-masked]>, "'Julian Reschke'" <[email-masked]>, "'Ned Freed / 822'" <[email-masked]>, "'Patrik Fdltstrvm'" <[email-masked]>, "'Paul Overell'" <[email-masked]>, "'Pete Resnick'" <[email-masked]>, "'Rob Cameron'" <[email-masked]>, "'Sandro Mazzucato'" <[email-masked]>, <[email-masked]>, "'ietf-822'" <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Tue, 8 Mar 2005 11:08:50 -0600
Subject: RE: ABNF to Draft -- pending issues
Sender: [email-masked]

 On Tue, 8 Mar 2005 11:21:39 -0500, Scott Hollenbeck wrote:
>  However, because this is a new I-D I do have
>
>  to create a new IESG review ballot for it.  That will happen after the I-D
>  is posted.

i am planning to post it on Sunday, on the assumption that the i-d posting queue is opened back up by then.

 d/
 --
 Dave Crocker
 Brandenburg InternetWorking
 +1.408.246.8253
 dcrocker  a t ...
 WE'VE MOVED to:  www.bbiw.net

From: "Scott Hollenbeck" <[email-masked]>
To: "'Bruce Lilly'" <[email-masked]>, "'Dave Crocker'" <[email-masked]>
Cc: "'Bill Fenner'" <[email-masked]>, "'Brian Bidulock'" <[email-masked]>, "'Chris Newman'" <[email-masked]>, "'Harald Tveit Alvestrand'" <[email-masked]>, "'Julian Reschke'" <[email-masked]>, "'Ned Freed / 822'" <[email-masked]>, "'Patrik Fdltstrvm'" <[email-masked]>, "'Paul Overell'" <[email-masked]>, "'Pete Resnick'" <[email-masked]>, "'Rob Cameron'" <[email-masked]>, "'Sandro Mazzucato'" <[email-masked]>, [email-masked], "'ietf-822'" <[email-masked]>
Subject: RE: ABNF to Draft -- pending issues
Date: Tue, 8 Mar 2005 11:21:39 -0500
Importance: Normal
Sender: [email-masked]

> Whether the bugs and editorial issue are addressed before submission
> as an I-D, after I-D announcement but prior to first Last Call, or
> after Last Call (and probably requiring a New Last Call) is, of
> course, your choice.

There will be no new last call on this I-D because it is a nothing more than
a modification of RFC 2234 required as a result of IESG review to advance
2234 to draft status.  In other words, the last call to advance 2234 has
already happened and this situation is similar to modifying an I-D to
address IESG review comments.  However, because this is a new I-D I do have
to create a new IESG review ballot for it.  That will happen after the I-D
is posted.

-Scott-

From: Bruce Lilly <[email-masked]>
Organization: Bruce Lilly
To: Dave Crocker <[email-masked]>
Subject: RE: ABNF to Draft -- pending issues
Date: Tue, 8 Mar 2005 10:30:27 -0500
Cc: "'Bill Fenner'" <[email-masked]>, "'Brian Bidulock'" <[email-masked]>, "'Chris Newman'" <[email-masked]>, "'Harald Tveit Alvestrand'" <[email-masked]>, "'Julian Reschke'" <[email-masked]>, "'Ned Freed / 822'" <[email-masked]>, "'Patrik Fdltstrvm'" <[email-masked]>, "'Paul Overell'" <[email-masked]>, "'Pete Resnick'" <[email-masked]>, "'Rob Cameron'" <[email-masked]>, "'Sandro Mazzucato'" <[email-masked]>, Scott Hollenbeck <[email-masked]>, [email-masked], "ietf-822" <[email-masked]>
Sender: [email-masked]

Dave,

> >  On Tue, 15 Feb 2005 10:22:41 -0800, Dave Crocker wrote:
> > >  Folks,
> > >
> > >  It's too late to submit the ABNF revision as an I-D, until after
> > >  Minneapolis, but here is a pointer to the revision:
[...]
> >  ; <http://bbiw.net/specifications/draft-crocker-abnf-rfc2234bis-02dc.txt>
[...]
> I believe ABNF is done, which is to say that it is ready for Draft.
> 
> 1. The revised spec is at the above location, obviously needing formal i-d
> submission after this week.

Don't forget to update the filename in the title area (it refers to
-00<mumble> rather than 02<mumble>) and the dates (more than a year
ago).

> 2. All of the outstanding items, concerning the document, pertain to
> enhancements, rather than to 'fixing bugs in the spec'.  Although most or
> all of the enhancements are reasonable, they would require re-cycling at
> Proposed.  In response to my queries about whether to recycle vs. go to
> draft, the overwhelming preference, is for moving to Draft.  Hence, no
> enhancements to abnf have been done.

There are at least two genuine bugs:
1. ABNF lines need not use CRLF line endings (in fact the canonical
   versions of I-Ds and RFCs at ISI.EDU do not).  I.e., *none* of
   the approved ABNF in use meets the ABNF ABNF CR requirement.
2. There is an unresolved ambiguity regarding comments beginning
   with whitespace, but unassociated with any rule; consider:

   a = b ; comment
     ; comment continues
   c = d

   vs.

   a = b ; comment
     ; supposedly unrelated comment matching rulelist production
   c = d

   vs.

   a = b ; comment
   ; comment unrelated to any specific rule
   c = d

   The ambiguity can be resolved by requiring comments unassociated
   with any rule to have no whitespace preceding the semicolon. 

I strongly suspect that the reason that at least two implementations
have failed to recognize %B, %D, and %X is that the prose in section
2.3 implies only lower-case letters after the percent sign, whereas
the ABNF ABNF (tucked away in a late section) denotes otherwise (but
only upon referring back to a separate paragraph of 2.3 discussing
case-insensitivity of quoted strings).  While that's not a bug per se,
it's an editorial issue that - in light of implementation experience -
probably ought to be addressed.

Whether the bugs and editorial issue are addressed before submission
as an I-D, after I-D announcement but prior to first Last Call, or
after Last Call (and probably requiring a New Last Call) is, of
course, your choice.

Best regards,
  Bruce Lilly

Date: Tue, 08 Mar 2005 08:12:24 -0600
From: Harald Tveit Alvestrand <[email-masked]>
To: Scott Hollenbeck <[email-masked]>, "'Dave Crocker'" <[email-masked]>
Cc: "'Patrik Fdltstrvm'" <[email-masked]>, "'ietf-822'" <[email-masked]>, "'Paul Overell'" <[email-masked]>, "'Rob Cameron'" <[email-masked]>, "'Chris Newman'" <[email-masked]>, "'Julian Reschke'" <[email-masked]>, "'Bill Fenner'" <[email-masked]>, "'Ned Freed / 822'" <[email-masked]>, "'Sandro Mazzucato'" <[email-masked]>, "'Pete Resnick'" <[email-masked]>, "'Brian Bidulock'" <[email-masked]>, [email-masked]
Subject: RE: ABNF to Draft -- pending issues
Sender: [email-masked]

DISCUSS is cleared now. Scott will deal with the formalities.

--On 8. mars 2005 08:34 -0500 Scott Hollenbeck <[email-masked]> wrote:

> Thanks for the update.  I need to work out a procedural issue with Harald
> (he's still holding a discuss on 2234), but I can do that.  I'll keep you
> posted.
>
> -Scott-
>
>> -----Original Message-----
>> From: Dave Crocker [mailto:[email-masked]]
>> Sent: Monday, March 07, 2005 3:11 PM
>> To: Scott Hollenbeck
>> Cc: Patrik Fdltstrvm; ietf-822; Paul Overell; Rob Cameron;
>> Chris Newman; Julian Reschke; Bill Fenner; Ned Freed / 822;
>> Harald Tveit Alvestrand; Dave Crocker; Sandro Mazzucato; Pete
>> Resnick; Brian Bidulock; [email-masked]
>> Subject: Re: ABNF to Draft -- pending issues
>> Importance: High
>>
>>
>>  On Tue, 15 Feb 2005 10:22:41 -0800, Dave Crocker wrote:
>> >   Folks,
>> >
>> >   It's too late to submit the ABNF revision as an I-D, until after
>> >   Minneapolis, but here is a pointer to the revision:
>> >
>> >   ;
>> <http://bbiw.net/specifications/draft-crocker-abnf-rfc2234bis-
> 02dc.html>
>>
>>   ;
>> <http://bbiw.net/specifications/draft-crocker-abnf-rfc2234bis-02dc.txt>
>>
>>
>>   As noted in the Acknowledgements section, Julian Reschke gets a very
>> big   THANKS! for converting the source to xml.  I've tweaked it only a
>> bit.
>>
>>   d/
>
>
>
> Scott,
>
> I believe ABNF is done, which is to say that it is ready for Draft.
>
> 1. The revised spec is at the above location, obviously needing formal i-d
> submission after this week.
>
> 2. All of the outstanding items, concerning the document, pertain to
> enhancements, rather than to 'fixing bugs in the spec'.  Although most or
> all of the enhancements are reasonable, they would require re-cycling at
> Proposed.  In response to my queries about whether to recycle vs. go to
> draft, the overwhelming preference, is for moving to Draft.  Hence, no
> enhancements to abnf have been done.
>
> So I believe you may continue processing the specification for Draft
> status.
>
>  d/
>  --
>  Dave Crocker
>  Brandenburg InternetWorking
>  +1.408.246.8253
>  dcrocker  a t ...
>  WE'VE MOVED to:  www.bbiw.net
>
>
>
>
>

From: "Scott Hollenbeck" <[email-masked]>
To: "'Dave Crocker'" <[email-masked]>
Cc: "'Patrik Fdltstrvm'" <[email-masked]>, "'ietf-822'" <[email-masked]>, "'Paul Overell'" <[email-masked]>, "'Rob Cameron'" <[email-masked]>, "'Chris Newman'" <[email-masked]>, "'Julian Reschke'" <[email-masked]>, "'Bill Fenner'" <[email-masked]>, "'Ned Freed / 822'" <[email-masked]>, "'Harald Tveit Alvestrand'" <[email-masked]>, "'Sandro Mazzucato'" <[email-masked]>, "'Pete Resnick'" <[email-masked]>, "'Brian Bidulock'" <[email-masked]>, [email-masked]
Subject: RE: ABNF to Draft -- pending issues
Date: Tue, 8 Mar 2005 08:34:24 -0500
Importance: Normal
Sender: [email-masked]

Thanks for the update.  I need to work out a procedural issue with Harald
(he's still holding a discuss on 2234), but I can do that.  I'll keep you
posted.

-Scott-

> -----Original Message-----
> From: Dave Crocker [mailto:[email-masked]] 
> Sent: Monday, March 07, 2005 3:11 PM
> To: Scott Hollenbeck
> Cc: Patrik Fdltstrvm; ietf-822; Paul Overell; Rob Cameron; 
> Chris Newman; Julian Reschke; Bill Fenner; Ned Freed / 822; 
> Harald Tveit Alvestrand; Dave Crocker; Sandro Mazzucato; Pete 
> Resnick; Brian Bidulock; [email-masked]
> Subject: Re: ABNF to Draft -- pending issues
> Importance: High
> 
> 
>  On Tue, 15 Feb 2005 10:22:41 -0800, Dave Crocker wrote:
> >  Folks,
> >
> >  It's too late to submit the ABNF revision as an I-D, until after
> >  Minneapolis, but here is a pointer to the revision:
> >
> >  ; 
> <http://bbiw.net/specifications/draft-crocker-abnf-rfc2234bis-
02dc.html>
>
>  ; <http://bbiw.net/specifications/draft-crocker-abnf-rfc2234bis-02dc.txt>
>
>
>  As noted in the Acknowledgements section, Julian Reschke gets a very big
>  THANKS! for converting the source to xml.  I've tweaked it only a bit.
>
>  d/

Scott,

I believe ABNF is done, which is to say that it is ready for Draft.

1. The revised spec is at the above location, obviously needing formal i-d
submission after this week.

2. All of the outstanding items, concerning the document, pertain to
enhancements, rather than to 'fixing bugs in the spec'.  Although most or
all of the enhancements are reasonable, they would require re-cycling at
Proposed.  In response to my queries about whether to recycle vs. go to
draft, the overwhelming preference, is for moving to Draft.  Hence, no
enhancements to abnf have been done.

So I believe you may continue processing the specification for Draft status.

 d/
 --
 Dave Crocker
 Brandenburg InternetWorking
 +1.408.246.8253
 dcrocker  a t ...
 WE'VE MOVED to:  www.bbiw.net

Date: Tue, 08 Mar 2005 02:14:58 -0500
From: Tony Hansen <[email-masked]>
To: wayne <[email-masked]>
CC: [email-masked]
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Sender: [email-masked]

You say that there are implementations out there that implement the 
older syntax, and that other implementations use a 2822 compatible 
syntax. So for this particular issue, I think you should:

	Explain (and describe) the two implemented syntaxes; one that is 
incompatible with 2822 and a subset that IS compatible. Then explain 
that new implementations MUST use the compatible format.

The same goes for any other issue: start your description with what's 
currently out there. Then, if necessary, describe what new 
implementations SHOULD or MUST do.

Remember, people aren't just writing new code to generate the stuff. 
People also have to implement code that accepts the older stuff.

	Tony Hansen
	[email-masked]

wayne wrote:
> Maybe we could use the example I gave in a previous message about the
> Received-SPF header.  Should I document what was allowed under the
> older SPF specs, which doesn't even match RFC2822's "obsolete" syntax,
> or should I document something that is largely compatible with all
> implementations that I know of and compatible with RFC2822?  I think
> it reasonable to say that an already large percent of Received-SPF
> headers conform to what is in the I-D (e.g. RFC2822 compatible), and
> that in the future, this percentage will increase.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>, Dave Crocker <[email-masked]>
Subject: Re: Update to "simplified" RFC 2822 grammar
Date: Mon, 7 Mar 2005 23:06:25 -0500
Cc: Pete Resnick <[email-masked]>
Sender: [email-masked]

On Mon March 7 2005 15:14, Dave Crocker wrote:

> your work falls into the category of "sure wish we had had access to it 10 years ago."

So do I :-).
 
> in order to incorporate abnf changes to rfc2822 at this point, we would have to re-cycle the specification at Proposed, because the abnf would be new and -- inherently - unstable.

The prevailing opinion when this was last discussed (over a year ago)
was that a reset would be unwarranted. See
http://www.imc.org/ietf-822/mail-archive/msg03790.html
and related messages.
 
> The reading of the community is that it would rather advance the existing work than start over learning to work with the new ABNF, no matter how well done the new abnf might be.

Which community? By whose poll?

Date: Mon, 07 Mar 2005 20:10:03 -0600
From: Harald Tveit Alvestrand <[email-masked]>
To: Bill Fenner <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: ABNF to Draft -- pending issues
Sender: [email-masked]

Bill,

thanks for generating the real data!

I withdraw my objections at this time.

(part of the confusion is because I've done a lousy job of keeping versions 
of my parser straight; I believe that the parser I'm currently using for 
myself implements the blank-line fix; I don't know what versions are "out 
there", and don't even know how to tell you to figure out which version 
they are.)

--On 7. mars 2005 19:00 -0600 Bill Fenner <[email-masked]> wrote:

>
> Harald,
>
>   I used my ABNF extractor on RFCs 3000-3999, resulting
> in 128 ABNF documents.  I parsed them with both your parser (from
> http://www.ops.ietf.org/abnf/parser.tar.gz) and mine.  Your parser
> presumably implements the rules you think should exist; mine implements
> exactly RFC2234 (modulo bugs; none currently known).
>
>   Of these 128 documents, mine failed none that yours didn't also fail;
> yours failed 14 that mine passed; mostly because yours appears to only
> accept at most two characters in %x00.00.00.00 style string and because
> you only accept uppercase hex digits in %x constructs.
>
>   I'll also note that your parser doesn't permit either blank lines
> or multi-line prose-vals;
>
> forbin% cat blankline
> a = b
>
>  c
> forbin% ./parser blankline
> Rule: a = b
> Not OK
> Syntax error: c<*>
>
> forbin% cat proseval
> proseval = <This is a multi-line prose-val,
>             which rfc2234 does not permit.>
>
> forbin% ./parser proseval
> Not OK
> Syntax error: proseval = <This is a multi-line prose-val,
> <*>
>
> so I'm not sure what implementation it is you're pointing to that
> implements the thing that's not 2234.
>
>   Bill
>
>

From: Bruce Lilly <[email-masked]>
Organization: Bruce Lilly
To: Bill Fenner <[email-masked]>
Subject: Re: ABNF to Draft -- pending issues
Date: Mon, 7 Mar 2005 22:33:47 -0500
Cc: [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], "ietf-822" <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

On Mon March 7 2005 20:00, Bill Fenner wrote:

> http://www.ops.ietf.org/abnf/parser.tar.gz
[...] 
> appears to only
> accept at most two characters in %x00.00.00.00 style string and because
> you only accept uppercase hex digits in %x constructs.

I believe that implementation also fails to recognize %D, %X, and %B.
[i.e. upper-case letters introducing decimal, hexadecimal, and binary
character literals]

>   I'll also note that your parser doesn't permit either blank lines
> or multi-line prose-vals;
> 
> forbin% cat blankline
> a = b
> 
>  c

If the "blank" line contains whitespace, then it is an RFC 2234
continuation line:

        c-wsp          =  WSP / (c-nl WSP)

N.B. the WSP after c-nl.  If the "blank" line does not contain WSP,
then it terminates the rule, and indeed the line beginning " c" is
a syntax error:

        rule           =  rulename defined-as elements c-nl
                               ; continues if next line starts
                               ;  with white space

        c-nl           =  comment / CRLF
                               ; comment or newline

        comment        =  ";" *(WSP / VCHAR) CRLF

[ignoring the bogus CR requirement].

> forbin% ./parser blankline
> Rule: a = b
> Not OK
> Syntax error: c<*>
> 
> forbin% cat proseval 
> proseval = <This is a multi-line prose-val,
>             which rfc2234 does not permit.>
> 
> forbin% ./parser proseval 
> Not OK
> Syntax error: proseval = <This is a multi-line prose-val,
> <*>          

Note that there is (or was) a published erratum which would make
that legal.  As discussed on the rfc-interest mailing list [
consensus appeared to be that that erratum is itself an error].

To: [email-masked]
Subject: Re: ABNF to Draft -- pending issues
Cc: [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked]
Date: Mon, 7 Mar 2005 19:00:24 -0600
From: Bill Fenner <[email-masked]>
Versions: dmail (linux) 2.6d/makemail 2.10
Sender: [email-masked]

Harald,

  I used my ABNF extractor on RFCs 3000-3999, resulting
in 128 ABNF documents.  I parsed them with both your parser (from
http://www.ops.ietf.org/abnf/parser.tar.gz) and mine.  Your parser
presumably implements the rules you think should exist; mine implements
exactly RFC2234 (modulo bugs; none currently known).

  Of these 128 documents, mine failed none that yours didn't also fail;
yours failed 14 that mine passed; mostly because yours appears to only
accept at most two characters in %x00.00.00.00 style string and because
you only accept uppercase hex digits in %x constructs.

  I'll also note that your parser doesn't permit either blank lines
or multi-line prose-vals;

forbin% cat blankline
a = b

 c
forbin% ./parser blankline
Rule: a = b
Not OK
Syntax error: c<*>

forbin% cat proseval 
proseval = <This is a multi-line prose-val,
            which rfc2234 does not permit.>

forbin% ./parser proseval 
Not OK
Syntax error: proseval = <This is a multi-line prose-val,
<*>          

so I'm not sure what implementation it is you're pointing to that
implements the thing that's not 2234.

  Bill

Date: Mon, 07 Mar 2005 16:45:41 -0600
From: Harald Tveit Alvestrand <[email-masked]>
To: Dave Crocker <[email-masked]>, Scott Hollenbeck <[email-masked]>
Cc: Julian Reschke <[email-masked]>, Paul Overell <[email-masked]>, Bill Fenner <[email-masked]>, Chris Newman <[email-masked]>, Sandro Mazzucato <[email-masked]>, Brian Bidulock <[email-masked]>, Rob Cameron <[email-masked]>, Pete Resnick <[email-masked]>, "Ned Freed / 822" <[email-masked]>, Patrik FXltstrXm <[email-masked]>, [email-masked], ietf-822 <[email-masked]>
Subject: Re: ABNF to Draft -- pending issues
Sender: [email-masked]

dave,

just to reiterate my concern:

A grammar that does not parse the majority of the ABNF that has been 
approved after "grammar check" is not, IMHO, a grammar that passes the 
"found usable" check for Draft.
Neither does it pass the "interoperable implementations" test, since the 
implementations have in fact implemented the "fixed" grammar, not the 
grammar in the spec.
(verifying whether or not my concern making sense requires pulling out a 
reasonable collection of ABNF from the approved standards and seeing 
whether they match the 2234 grammar or not. I haven't done that, and won't 
do so before I go on holiday.)

I think we need blank lines in specifications.

But notwithstanding, I think submitting the I-D is a good idea.

                 Harald

--On 7. mars 2005 14:11 -0600 Dave Crocker <[email-masked]> wrote:

>  On Tue, 15 Feb 2005 10:22:41 -0800, Dave Crocker wrote:
>>   Folks,
>>
>>   It's too late to submit the ABNF revision as an I-D, until after
>>   Minneapolis, but here is a pointer to the revision:
>>
>>   ;
>> <http://bbiw.net/specifications/draft-crocker-abnf-rfc2234bis-02dc.html>
>>
>>   ;
>> <http://bbiw.net/specifications/draft-crocker-abnf-rfc2234bis-02dc.txt>
>>
>>
>>   As noted in the Acknowledgements section, Julian Reschke gets a very
>> big   THANKS! for converting the source to xml.  I've tweaked it only a
>> bit.
>>
>>   d/
>
>
>
> Scott,
>
> I believe ABNF is done, which is to say that it is ready for Draft.
>
> 1. The revised spec is at the above location, obviously needing formal
> i-d submission after this week.
>
> 2. All of the outstanding items, concerning the document, pertain to
> enhancements, rather than to 'fixing bugs in the spec'.  Although most or
> all of the enhancements are reasonable, they would require re-cycling at
> Proposed.  In response to my queries about whether to recycle vs. go to
> draft, the overwhelming preference, is for moving to Draft.  Hence, no
> enhancements to abnf have been done.
>
> So I believe you may continue processing the specification for Draft
> status.
>
>  d/
>  --
>  Dave Crocker
>  Brandenburg InternetWorking
>  +1.408.246.8253
>  dcrocker  a t ...
>  WE'VE MOVED to:  www.bbiw.net
>
>
>

From: Dave Crocker <[email-masked]>
To: ietf-822 <[email-masked]>, ietf-822 <[email-masked]>
CC: Pete Resnick <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Mon, 7 Mar 2005 14:14:13 -0600
Delivery-Date: Mon, 7 Mar 2005 14:14:13 -0000
Subject: Re: Update to "simplified" RFC 2822 grammar
Sender: [email-masked]

 On Wed, 2 Mar 2005 11:49:43 -0500, Bruce Lilly wrote:
>  have made some fairly minor changes to the "simplified"
>  RFC 2822 grammar at

Bruce,

your work falls into the category of "sure wish we had had access to it 10 years ago."

in order to incorporate abnf changes to rfc2822 at this point, we would have to re-cycle the specification at Proposed, because the abnf would be new and -- inherently - unstable.

The reading of the community is that it would rather advance the existing work than start over learning to work with the new ABNF, no matter how well done the new abnf might be.

 d/
 --
 Dave Crocker
 Brandenburg InternetWorking
 +1.408.246.8253
 dcrocker  a t ...
 WE'VE MOVED to:  www.bbiw.net

Date: Mon, 7 Mar 2005 20:35:23 +0000
From: Tony Finch <[email-masked]>
To: ietf-822 <[email-masked]>
cc: Keith Moore <[email-masked]>, Bruce Lilly <[email-masked]>
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

On Mon, 7 Mar 2005, Bruce Lilly wrote:
>
> I assume that by "collateral spam" you are still referring to bounce
> messages sent to joe-job victims.  If a null return path is used for
> the original message, there is no "collateral spam", as there is no
> bounce message.

No reliability either. Let's all join the folks over at dsn.rfc-ignorant.org.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
NORTH ICELAND: WEST OR NORTHWEST 5 TO 7. MAINLY FAIR. GOOD. LIGHT ICING,
TEMPERATURES MS01 TO MS03.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 7 Mar 2005 14:56:08 -0500
Cc: Tony Finch <[email-masked]>, Keith Moore <[email-masked]>
Sender: [email-masked]

On Sun March 6 2005 19:04, Tony Finch wrote:
> If you encourage people to use null return paths for normal messages and
> spam, then you eliminate the possibility of using something like BATV to
> detect collateral spam.

I assume that by "collateral spam" you are still referring to bounce
messages sent to joe-job victims.  If a null return path is used for
the original message, there is no "collateral spam", as there is no
bounce message.

Cc: ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Mon, 7 Mar 2005 14:00:43 -0500
To: Tony Finch <[email-masked]>
Sender: [email-masked]

> I would like the rule to be that a null return path MUST only be used 
> for
> an auto-response (which includes DSNs and other kinds of bounce) and 
> MUST
> only be used with a recipient address which is the return path of the
> message that triggered the auto-response.

it's not going to happen.  there are too many other kinds of messages 
for which a null return-path is valid, and this is an incompatible 
change to a protocol that has ~25 years of deployment.

Keith

To: [email-masked]
Xref: clerew local.mime:4559
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Mon, 7 Mar 2005 11:21:35 GMT
Lines: 27
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>> X-Info: This message was accepted for relay by
>> 	smtp02.mrf.mail.rcn.net as the sender used SMTP authentication
>> Whoopee -- anybody with a working printf can forge such a line,
>> so that doesn't seem to do much.

>Well, duh.  Obviously you need something stronger than a header field
>which doesn't contain any way of verifying what message it was attached
>to.  Take the input message, canonicalize it, hash that, sign the hash,
>put the signature in the header field.  Not rocket science (though the
>canonicalization step is a bit tricky to get right)

Such as:

http://www.imc.org/ietf-usefor/drafts/draft-lindsey-usefor-signed-01.txt

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 7 Mar 2005 00:04:42 +0000
From: Tony Finch <[email-masked]>
To: Bruce Lilly <[email-masked]>
cc: Keith Moore <[email-masked]>, ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

On Sun, 6 Mar 2005, Bruce Lilly wrote:
>
> So bounce messages get sent to victims. No specific relationship to
> null paths there.

Except that, as you say:

> Bounce messages themselves are supposed to be sent with a null return
> path.  That's intentional, to prevent looping bounce messages.

> If a spam message were to be sent with a null return path, there
> would be no bounce.  I know of no rule that prohibits setting a null
> return path for a message other than a bounce message. If there were
> such a rule, loosening it would seem to be an improvement (would
> result in less collateral damage).
>
> Could you explain what you had in mind w.r.t. "tighten up the rules
> about null return paths".

I would like the rule to be that a null return path MUST only be used for
an auto-response (which includes DSNs and other kinds of bounce) and MUST
only be used with a recipient address which is the return path of the
message that triggered the auto-response.

With a system like BATV, messages with a null return path that are not
sent to a valid return path are rejected as forgeries, because BATV
enables a site to distinguish valid and invalid return paths.

If you encourage people to use null return paths for normal messages and
spam, then you eliminate the possibility of using something like BATV to
detect collateral spam. You directly encourage people to treat email as an
unreliable transport, and by removing any means of dealing with collateral
spam you indirectly encourage people to just ignore and disable email's
error reporting mechanisms, which turns the perception of unreliability
into reality.

I intend to find out (by practical experiment) how common it is for
non-bounce messages to use null return paths, in order to assess the gap
between reality and the proposed tighter null return path rule.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
DENMARK STRAIT: SOUTHWEST 4 OR 5, OCCASIONALLY 6 IN THE EAST AT FIRST,
BECOMING VARIABLE 3 OR 4 IN THE WEST. MAINLY FAIR, BUT RAIN IN THE SOUTHWEST.
MODERATE OR GOOD, BUT POOR IN THE NORTH.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: Tony Finch <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Sun, 6 Mar 2005 18:25:36 -0500
Cc: Keith Moore <[email-masked]>, ietf-822 <[email-masked]>
Sender: [email-masked]

On Mon February 28 2005 09:57, Tony Finch wrote:

> On Mon, 28 Feb 2005, Keith Moore wrote:
> 
> > > I agree that it is wrong to reject null return paths altogether. However
> > > your suggestion is incompatible with backscatter detection systems (such
> > > as BATV or Signed Envelope Sender) which assume that null return paths are
> > > only used for bounce messages.
> >
> > those systems are operating on false assumptions.  it's never been the case
> > that <> was limited to bounce messages.
> 
> Yes. It'll be interesting to discover the extent of the interop problems :-)
> 
> I think we're going to have to tighten up the rules about null return
> paths and bounces, because of the backscatter problem.

> > could you explain the backscatter problem?  I'm not sure what you're referring
> > to.
> 
> If a spammer or a virus sends out forged email (a "joe job") a certain
> proportion of it is likely to be rejected, e.g. because of incorrect
> recipient addresses or anti-virus or anti-spam or anti-forgery checks. If
> the rejection results in a bounce message the bounce will be sent to the
> apparent sender of the message, who is an innocent third party and the
> victim of the joe job. These bounces are backscatter from forged spam, aka
> collateral spam.

So bounce messages get sent to victims. No specific relationship to
null paths there.

Bounce messages themselves are supposed to be sent with a null return
path.  That's intentional, to prevent looping bounce messages.

If a spam message were to be sent with a null return path, there
would be no bounce.  I know of no rule that prohibits setting a null
return path for a message other than a bounce message. If there were
such a rule, loosening it would seem to be an improvement (would
result in less collateral damage).

Could you explain what you had in mind w.r.t. "tighten up the rules
about null return paths".

Date: Sun, 6 Mar 2005 20:19:04 +0000
From: Tony Finch <[email-masked]>
To: Bruce Lilly <[email-masked]>
cc: ietf-822 <[email-masked]>, Keith Moore <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Sender: [email-masked]

On Sun, 6 Mar 2005, Bruce Lilly wrote:
> On Fri March 4 2005 16:11, Keith Moore wrote:
>
> > I should clarify that by "unauthorized mail" I meant sending a message
> > using MAIL FROM:<address> and/or From:<address> without having the
> > authorization of the party associated with <address>.
>
> There's simply no way to detect that situation in current
> protocols.

That's true in general. However a site can implement a policy such as
"return paths must contain a security tag" (e.g. as specified in BATV)
which allows that site to distinguish between authorized and unauthorized
use of an address in a return path. At the moment the extent of the
interoperability issues with this idea has not yet been examined in
detail.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
LANDS END TO ST DAVIDS HEAD INCLUDING THE BRISTOL CHANNEL: NORTHEAST BACKING
NORTH OR NORTHWEST 3 OR 4, OCCASIONALLY 5 LATER IN WEST. FAIR. GOOD. SLIGHT.

From: Bruce Lilly <[email-masked]>
Reply-To: Bruce Lilly <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Sun, 6 Mar 2005 14:35:13 -0500
Cc: Keith Moore <[email-masked]>, [email-masked]
Sender: [email-masked]

On Fri March 4 2005 16:11, Keith Moore wrote:

> > The problem is that if the ISP that handles my *incoming* mail
> > decides to use SPF and list only their *outbound* MTA IP addresses,
> > I am screwed.
> 
> For that matter, if your ISP that handles your incoming mail lists the
> wrong MX records, you are also screwed.  You can think of it as a flaw
> in SPF, but most of our protocols require correct configuration of DNS
> to work properly in practice.

But there is a crucial difference in the two cases; MX records are
by definition associated with inbound mail.  SPF conflates inbound
and outbound. 

> It's not immediately clear whether a domain holder has any right to
> insist that mail traffic "from" that domain

There isn't really any such thing as "mail 'from' [a] domain";
there is only the return path (for delivery error notifications,
which is not necessarily related to anything other than the desired
path for such notifications), and the forward (recipient delivery)
path(s).

> only originate via  
> particular IP addresses - my guess is that if nothing else, it's error-
> prone, as too many network admins will assume (without adequate
> justification) that all mail "from" their domains is sourced via some
> set of well-known IP addresses.

That, in a nutshell, is the (flawed) set of assumptions behind SPF.
 
> > Yeah, that's the situation I have at the moment (local ISP blocks
> > 25, my ISP doesn't use SSL (port 465) or run a submission server
> > on port 587).  The local ISP isn't so fascist that they block
> > 465 and 587, but I suppose some might do that also.
> 
> Especially given that RFC 2476 doesn't require authentication.

If you've looked at the draft successor which was intended to be
elevated to Draft Standard status (and for which Last Call has
already ended), you may have noticed that AUTH is now required.

> I should clarify that by "unauthorized mail" I meant sending a message
> using MAIL FROM:<address> and/or From:<address> without having the
> authorization of the party associated with <address>.

There's simply no way to detect that situation in current
protocols (and SPF does nothing to change that, due to the
flawed assumptions noted).

> There are a lot of stupid policies out there.  People are so desperate
> to block SPAM that they block a lot of legitimate mail in the process.
> Our job is to improve the reliability of mail - which includes both
> telling people how to do a better job of blocking unauthorized mail
> and telling people which countermeasures do more harm than good.

Which is another reason why producing a purely descriptive
document (as Informational) that accurately documents SPF 
isn't a big deal; the logical next step would be to move
that document to Historic status on the basis that what is
described is harmful (and that is one of the criteria for
reassignment to Historic given in RFC 2026).

> Well, duh.  Obviously you need something stronger than a header field
> which doesn't contain any way of verifying what message it was attached
> to.  Take the input message, canonicalize it, hash that, sign the hash,
> put the signature in the header field.  Not rocket science (though the
> canonicalization step is a bit tricky to get right)

Which is why I suggested MIME security-part wrapping; it avoids
the tricky canonicalization and prevents downstream munging.
Now that might still have similarly low chances of being
(currently, at least) widely deployed, but at this stage that's
an irrelevant detail (as neither of us seems to be making any
sort of specific proposal). 

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Sun, 6 Mar 2005 14:18:17 -0500
Cc: wayne <[email-masked]>
Sender: [email-masked]

On Fri March 4 2005 10:50, wayne wrote:
> 
> In <[email-masked]> Bruce Lilly <[email-masked]> writes:

> > From my perspective (as a mobile
> > user (i.e. connecting via an unpredictable IP address) with need
> > to use only a few domain names) I'd phrase the issue as "being
> > able to use any IP address"; and that's where SPF utterly fails.
> 
> I don't believe that SPF utterly fails for roaming users.  There are
> many options for you to use.
> 
> For example, you can use SMTP AUTH over the SMTP submission port to
> always submit your email to an acceptable MSA instead of using your
> ISP's MTA.

Which MSA? operated by whom? Using AUTH based on what prearranged
information?

> Another option is to give neutral results on other IP addresses.  You
> can even do something like:
> 
>         v=spf1 mx -exists:%{ir}.xbl.spamhaus.org ?all

No, I can't, because I don't own my ISP's domain name (which
appears as the domain of my mailbox).
 
> you can use pretty aggressive DNSBLs with out fear of
> the domain's MTA being listed on the DNSBLs.

No, same reason as above.

> With more work, you can create specialized DNS servers and use the
> exists: mechanism to do some fancy things.  About a year ago, someone
> created a "rate limiting" DNS server, so you could do something like:

Again no and no; same reason.

> Similar techniques can be used check for a special token in the return
> path and allow email using it.

Most UAs set the SMTP return path using the sender's mailbox;
there's no "special token" there.

> One person was working on creating 
> something similar to SMTP-after-POP, which would authorize an IP
> address via a specialized DNS server after a POP.

Sending and retrieving are separate operations; many ISPs
provide separate servers.  Moreover, it is sometimes necessary
to send mail w/o using the ISP's SMTP relay, for example to
work around temporary DNS botches [which is why blocking port
25 is unreasonable].
 
> All of these techniques have been known/discussed for well over a
> year.  

And (as has also been discussed/is well known) none of them
are applicable in the vast majority of cases (i.e. in general).

Cc: [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Fri, 4 Mar 2005 18:54:10 -0500
To: wayne <[email-masked]>
Sender: [email-masked]

> I don't believe that SPF utterly fails for roaming users.  There are
> many options for you to use.

it doesn't utterly fail.  it does, however, tempt DNS zone admins to 
impose unreasonable constraints on how mail is sent.

> For example, you can use SMTP AUTH over the SMTP submission port to
> always submit your email to an acceptable MSA instead of using your
> ISP's MTA.  While there are many ISPs that block port 25, there is no
> reason to block port 587 and almost no one does.

even if your domain has MSAs that accept port 587 and your ISP does not 
block it, this may not be a good option.  if requires, among other 
things, that your MUA know which MSA to use for which return address.  
some MUAs do this fine, others poorly, others not at all.  (Apple Mail 
seems to choose an MSA at random, and if that fails to accept the 
message, complain about it.)

and it's certainly true that the zone admin can set up SPF records so 
that they don't interfere with legitimate activity.  though almost any 
attempt to do fine-grained control over which domains can send mail 
from which IP addresses is of limited applicability.  for any large 
group of users, zone admins aren't likely to know enough about their 
users' needs to decide what is legitimate for those users and what is 
not.  and there are scaling and reliability and performance concerns 
with doing lots of DNS queries.

of course, there are exceptions, domains for which this works well.  
some of them serve millions of users.

Keith

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Fri, 04 Mar 2005 23:57:56 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 9
Sender: [email-masked]

wayne wrote:

 [Received-SPF: result key = value ; key = value]
> My understanding is that this is allowed.

I'm a UseFor invalid and confused dot-atom with dot-atom-text.

                     Sorry, Frank

To: [email-masked]
From: wayne <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Fri, 04 Mar 2005 09:50:12 -0600
Lines: 58
Cancel-Lock: sha1:3MYmoYQAMOpXnRTjc9EIbs4xm/g=
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

> On Fri March 4 2005 00:19, Keith Moore wrote:
>> >> Well, current SMTP specifications allow for anyone to use any domain
>> >> in either the rfc2821 identities, or any place in rfc2822.  All
>> >> authentication schemes intend to change that.
>> >
>> > AFAIK, none of SSL, TLS, or SMTP AUTH make any such change.
>> 
>> well, it depends on what you mean by "being able to use any domain"
>
> Well, that was Wayne's comment.  From my perspective (as a mobile
> user (i.e. connecting via an unpredictable IP address) with need
> to use only a few domain names) I'd phrase the issue as "being
> able to use any IP address"; and that's where SPF utterly fails.

I don't believe that SPF utterly fails for roaming users.  There are
many options for you to use.

For example, you can use SMTP AUTH over the SMTP submission port to
always submit your email to an acceptable MSA instead of using your
ISP's MTA.  While there are many ISPs that block port 25, there is no
reason to block port 587 and almost no one does.

Another option is to give neutral results on other IP addresses.  You
can even do something like:

        v=spf1 mx -exists:%{ir}.xbl.spamhaus.org ?all

This give positive results for certain IP addresses, negative results
for those most likely to be abusive, and neutral results for everyone
else.  Note that since the domain's MTA addresses are checked before
the DNSBL check, you can use pretty aggressive DNSBLs with out fear of
the domain's MTA being listed on the DNSBLs.

With more work, you can create specialized DNS servers and use the
exists: mechanism to do some fancy things.  About a year ago, someone
created a "rate limiting" DNS server, so you could do something like:

        v=spf1 mx exists:${ir}._spf_rate.%{d} -all

The DNS server was put on the subdomain _spf_rate, and it would allow
only a few emails per hour from a given IP address.  This allows
roaming users to send a few emails, but prevent spammers from sending
lots of email using their domain.

Similar techniques can be used check for a special token in the return
path and allow email using it.  One person was working on creating
something similar to SMTP-after-POP, which would authorize an IP
address via a specialized DNS server after a POP.

All of these techniques have been known/discussed for well over a
year.  

-wayne

Date: Fri, 4 Mar 2005 16:11:51 -0500
From: Keith Moore <[email-masked]>
To: Bruce Lilly <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Sender: [email-masked]

> On Fri March 4 2005 09:24, Keith Moore wrote:
> 
> > > Well, that was Wayne's comment.  From my perspective (as a mobile
> > > user (i.e. connecting via an unpredictable IP address) with need
> > > to use only a few domain names) I'd phrase the issue as "being
> > > able to use any IP address"; and that's where SPF utterly fails.
> > 
> > or to put it another way, SPF is not applicable to that case.
> 
> The problem is that if the ISP that handles my *incoming* mail
> decides to use SPF and list only their *outbound* MTA IP addresses,
> I am screwed.

For that matter, if your ISP that handles your incoming mail lists the
wrong MX records, you are also screwed.  You can think of it as a flaw
in SPF, but most of our protocols require correct configuration of DNS
to work properly in practice.

It's not immediately clear whether a domain holder has any right to
insist that mail traffic "from" that domain only originate via 
particular IP addresses - my guess is that if nothing else, it's error-
prone, as too many network admins will assume (without adequate
justification) that all mail "from" their domains is sourced via some
set of well-known IP addresses.

> > and a number of ISPs and networks are blocking 
> > outbound access to port 25, thus forcing their users to send
> > outgoing mail via their SMTP servers (they may permit use of another
> > network's submission server on port 587).
> 
> Yeah, that's the situation I have at the moment (local ISP blocks
> 25, my ISP doesn't use SSL (port 465) or run a submission server
> on port 587).  The local ISP isn't so fascist that they block
> 465 and 587, but I suppose some might do that also.

Especially given that RFC 2476 doesn't require authentication.

> > it doesn't do much to address the 
> > problem of an MTA or recipient that wants to avoid handling 
> > unauthorized mail.
> 
> In the context of mail, i.e. requiring no prior arrangement
> as noted in Dave Crocker's architecture draft, "unauthorized
> mail" makes little sense. 

I should clarify that by "unauthorized mail" I meant sending a message
using MAIL FROM:<address> and/or From:<address> without having the
authorization of the party associated with <address>.

> An SMTP receiver for some domain,
> say example.net, might avoid relaying by refusing to accept
> message DATA which is neither MAIL FROM its domain nor destined
> for an RCPT TO its domain (and it had better not be too picky
> about the MAIL FROM part, as legitimate users within the
> example.net domain might desire or need to specify a different
> mailbox for bounces).  As far as acceptance of messages for
> RCPT TO in example.net, nothing else is relevant: the return
> path (MAIL FROM) indicates where to send bounces, and plays
> no role in any sort of authentication, EHLO/HELO domain is
> often forced by client software to be the same as the return
> path domain, so is uninformative, and IP address is irrelevant.

There are a lot of stupid policies out there.  People are so desperate
to block SPAM that they block a lot of legitimate mail in the process.
Our job is to improve the reliability of mail - which includes both
telling people how to do a better job of blocking unauthorized mail
and telling people which countermeasures do more harm than good.

> > what might be useful is for an MSA to use SMTP AUTH to establish 
> > credentials of an originator, and then use that to add headers to a 
> > message authenticating that message as being from that originator.  
> > (though you need some way for the originator to defeat that in case
> > he wants to be anonymous)
> 
> Maybe. My ISP (when I can reach it) does that, and if you check
> past list messages (from more than a few days ago) you'll
> probably see (assuming that you get list messages in  undigested
> form) lines like:
> X-Info: This message was accepted for relay by
> 	smtp02.mrf.mail.rcn.net as the sender used SMTP authentication
> Whoopee -- anybody with a working printf can forge such a line,
> so that doesn't seem to do much.

Well, duh.  Obviously you need something stronger than a header field
which doesn't contain any way of verifying what message it was attached
to.  Take the input message, canonicalize it, hash that, sign the hash,
put the signature in the header field.  Not rocket science (though the
canonicalization step is a bit tricky to get right)

Keith

To: [email-masked]
From: wayne <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Fri, 04 Mar 2005 09:21:02 -0600
Lines: 84
Cancel-Lock: sha1:ryJDSk/DQOr0DpL+3sHGva1t4Sg=
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>>> all of the authentication schemes I've seen suffer from one or both of
>>> these problems:
>>>
>>> -  trying to do more than is reasonable for that particular approach
>>
>> This seems to be a pretty subjective criteria.  I suspect that
>> reasonable people can disagree on what might be considered
>> "reasonable" for a particular approach.
>
> yes, there will be some disagreement, but the opinions will often
> cluster in one general area, and you can get rough consensus on them.
> this does require engineering judgment.  it's not as if, for instance,
> there are objective criteria for how many DNS lookups are too many to
> use to validate an email address, but you'll get general agreement
> among most knowledgeable folks that there's a limit to how many
> lookups is reasonable, and those opinions will cluster around some
> compromise number.

Two comments:

First off, the SPF spec was developed in just such an open and mostly
by consensus manor.  However, you ignore the possiblity that instead
of one cluster, you may have several different clusters.

Secondly, there is an objective criteria for the number of DNS
lookups: "The number of DNS lookups must be limited to the point that
SPF can not be effectively used for DoS purposes".  (Most seriously,
on third parties.  That is, not the owner of the domain with the SPF
record, nor the receiving MTA, but someone that the SPF record
targets.) 

One of the changes I made to the current draft is to impose limits
first added to my libspf2 implementation, about a year ago.  This *is*
a significant change to the draft since most SPF implementations
didn't not have these limits, but I consider such a change important
enough to justify the breakage that happen.  I think it is simply
unacceptable to have a standard that allows for DoS attacks on third
parties. 

>>> -  trying to retroactively change how the mail protocol is used, or to
>>> restrict future use of the mail protocol such that valid use cases
>>> will no longer work
>>
>> Well, current SMTP specifications allow for anyone to use any domain
>> in either the rfc2821 identities, or any place in rfc2822.  All
>> authentication schemes intend to change that.
>
> it's tempting to respond that all of these authentication schemes are
> therefore "broken".  but that's so simple a statement that it's likely
> to be taken the wrong way.
>
> part of the problem is that none of the existing fields in either SMTP
> or [2]822 are intended to hold an authenticated ID of the originator,

Agreed.

> with the possible exception of Sender, even that one isn't well
> defined enough (or consistently implemented enough) to use.  and for
> every one of those fields there are numerous valid use cases for that
> field not being an authenticated ID for the originator, even though
> you might want to authenticate the message.  we simply don't have a
> field for that purpose defined at present.  various authentication
> schemes try to work around that by constraining or overloading
> existing fields.  the goal in doing so is to ease deployment, but what
> it actually does is limit the scheme's applicability.

The reason for wanting to authenticate the fields isn't just "ease of
deployment".  I don't even think that was the primary reason.

Many domain owners want to have some control over how their domain is
used, and many email receivers wish to listen to the domain owners.
Both snail mail and email allow anyone to put any name as the return
address and on the letter head of the letter.  Sometimes this isn't
abusive, but sometimes it is.  Making a letter/email look like it
comes from "Big Bank" when not authorized by "Big Bank" can get you in
legal trouble.  With email, and the various email authentication
systems, "Big Bank" can take technical steps, instead of just legal
steps, to prevent this.

-wayne

To: [email-masked]
From: wayne <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Fri, 04 Mar 2005 08:58:37 -0600
Lines: 43
Cancel-Lock: sha1:L1cXg3BWukZBLFMNz/Mk9CkUS+A=
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

> All we can do here is help Wayne to get the syntax for the
> Received-SPF header field right.  Or let's help Martin to get
> his Archived-At right.

Yeah, that's what I came here for.

> At the moment (spf-classic-00) he allows apparently no [CFWS],
> {FWS], or *WSP between "=" and ";" in a key-value-pair.  If
> that's for some reason not okay I'd be interested to hear it.
>
> Received-SPF: Pass ip = 127.0.0.1 ; (not allowed ?)
> .......................^.........^                   Bye, Frank

My understanding is that this is allowed.  The current draft defines
key-value-pair as:

        key-value-pair   = key [CFWS] "=" ( dot-atom / quoted-string )

With both dot-atom or a quoted-string as defined in RFC2822.  Both of
these ABNF terminals have [CFWS] at the beginning and the end.

This was also allowed under spf-draft-200406, although it used

        key-value-pair = 1*VCHAR "=" *(WSP / VCHAR) ";"

This is an example of something I have changed.  The old ABNF allowed
for something like:

        Received-SPF: Pass (comment) a==; "=;

And this could be parsed many different ways, such as the key being
"a" or "a=", and the value being "=; \"=", or ";\"=", or "", etc.

-wayne

From: Bruce Lilly <[email-masked]>
Reply-To: Bruce Lilly <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Fri, 4 Mar 2005 13:19:11 -0500
Cc: Keith Moore <[email-masked]>, wayne <[email-masked]>
Sender: [email-masked]

On Fri March 4 2005 09:24, Keith Moore wrote:

> > Well, that was Wayne's comment.  From my perspective (as a mobile user 
> > (i.e. connecting via an unpredictable IP address) with need to use 
> > only a few domain names) I'd phrase the issue as "being able to use 
> > any IP address"; and that's where SPF utterly fails.
> 
> or to put it another way, SPF is not applicable to that case.

The problem is that if the ISP that handles my *incoming* mail
decides to use SPF and list only their *outbound* MTA IP addresses,
I am screwed.  That is a fundamental flaw in SPF; it not only
makes the mistake of trying to tie mailbox domains to IP addresses,
it ties *inbound* mailbox domains to *outbound* IP addresses (at
least with the usual MUAs that use that domain in EHLO/HELO).
 
> and a number of ISPs and networks are blocking 
> outbound access to port 25, thus forcing their users to send outgoing 
> mail via their SMTP servers (they may permit use of another network's 
> submission server on port 587).

Yeah, that's the situation I have at the moment (local ISP blocks
25, my ISP doesn't use SSL (port 465) or run a submission server
on port 587).  The local ISP isn't so fascist that they block
465 and 587, but I suppose some might do that also.
 
> it doesn't do much to address the 
> problem of an MTA or recipient that wants to avoid handling 
> unauthorized mail.

In the context of mail, i.e. requiring no prior arrangement
as noted in Dave Crocker's architecture draft, "unauthorized
mail" makes little sense.  An SMTP receiver for some domain,
say example.net, might avoid relaying by refusing to accept
message DATA which is neither MAIL FROM its domain nor destined
for an RCPT TO its domain (and it had better not be too picky
about the MAIL FROM part, as legitimate users within the
example.net domain might desire or need to specify a different
mailbox for bounces).  As far as acceptance of messages for
RCPT TO in example.net, nothing else is relevant: the return
path (MAIL FROM) indicates where to send bounces, and plays
no role in any sort of authentication, EHLO/HELO domain is
often forced by client software to be the same as the return
path domain, so is uninformative, and IP address is irrelevant.

> what might be useful is for an MSA to use SMTP AUTH to establish 
> credentials of an originator, and then use that to add headers to a 
> message authenticating that message as being from that originator.  
> (though you need some way for the originator to defeat that in case he 
> wants to be anonymous)

Maybe. My ISP (when I can reach it) does that, and if you check
past list messages (from more than a few days ago) you'll
probably see (assuming that you get list messages in  undigested
form) lines like:
X-Info: This message was accepted for relay by
	smtp02.mrf.mail.rcn.net as the sender used SMTP authentication
Whoopee -- anybody with a working printf can forge such a line,
so that doesn't seem to do much.

About the only workable authentication scenario that I can
imagine that is consistent with email architecture would
involve:
1. some sort of PKI
2. use of secure MIME (S/MIME or PGP/MIME)
At each hop, the DATA would be MIME wrapped and signed, then
that wrapped MIME body would have trace header fields prepended
and would be sent to the next hop.  The receiver would consult
the PKI to verify the previous hop, then unwrap the DATA and
repeat for the next hop (or return a failure response).

Or something like that.

The weak links are of course the PKI, and how the sender indicates
a key to the next hop.  The key could be passed in a new trace field,
or an extension to the Received field.  The PKI could be implemented
via DNS or a similar mechanism, or an established PKI could be used.
Of course, both senders and receivers would have to be modified to
use such a mechanism, it use would have to be negotiated via some
sort of ESMTP keyword and there would have to be broad agreement on
PKI.  Negotiation and fallback means that a sender would either
need to be able to sign and wrap messages on the fly, or pre-sign
and wrap and choose pre-signed or plain DATA depending on
negotiation.  The PKI would need to reasonably accessible to
individuals (so that their MUAs can be configured to sign and wrap
messages with a key associated with the sender).

The MIME wrapping would be so-called triple-wrapped MIME, and
I've some time ago expressed my doubts about significantly
deployed and functional implementations of that.

Obviously, such a mechanism isn't going to spring up and become
widely deployed overnight; it might simply be best to wait for a
next-generation mail system that can be designed with an
authentication mechanism up-front.

Cc: ietf-822 <[email-masked]>, wayne <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Fri, 4 Mar 2005 09:24:39 -0500
To: Bruce Lilly <[email-masked]>
Sender: [email-masked]

>>>> Well, current SMTP specifications allow for anyone to use any 
>>>> domain in either the rfc2821 identities, or any place in rfc2822.  
>>>> All authentication schemes intend to change that.
>>>
>>> AFAIK, none of SSL, TLS, or SMTP AUTH make any such change.
>>
>> well, it depends on what you mean by "being able to use any domain"
>
> Well, that was Wayne's comment.  From my perspective (as a mobile user 
> (i.e. connecting via an unpredictable IP address) with need to use 
> only a few domain names) I'd phrase the issue as "being able to use 
> any IP address"; and that's where SPF utterly fails.

or to put it another way, SPF is not applicable to that case.

> To be precise, I mean that when sending mail, I need to specify a 
> mailbox in the return path which uses my ISP's domain name (because 
> that (my mailbox) is where legitimate bounces should go) and likewise 
> in the From and/or Reply-To and/or Sender fields of the message header 
> -- regardless of my IP address, and regardless of the ISP that I 
> happen to be sending the message(s) via, if I use an ISP's SMTP relay, 
> and regardless of whether I choose to use such a relay (e.g. to store 
> a single copy of a message for distribution to multiple recipients) or 
> to directly connect to the designated MX host(s) for the message 
> recipient(s).

entirely agree.  and a number of ISPs and networks are blocking 
outbound access to port 25, thus forcing their users to send outgoing 
mail via their SMTP servers (they may permit use of another network's 
submission server on port 587).

> And SMTP AUTH provides for SMTP session authentication independent of 
> transport layer security, and without imposing unreasonable 
> restrictions on IP address or domain name.

then again, in most cases, SMTP AUTH does nothing to authenticate the 
message originator to an intermediate MTA or recipient.  it can 
authenticate one MTA to another, or it can authenticate an originator 
to his message submission agent, but it doesn't do much to address the 
problem of an MTA or recipient that wants to avoid handling 
unauthorized mail.  so SMTP AUTH is really not comparable to the 
authentication schemes Wayne was referring to.  SMTP AUTH was mostly 
intended to allow servers to accept outgoing mail without being open 
relays.

what might be useful is for an MSA to use SMTP AUTH to establish 
credentials of an originator, and then use that to add headers to a 
message authenticating that message as being from that originator.  
(though you need some way for the originator to defeat that in case he 
wants to be anonymous)

Keith

From: Bruce Lilly <[email-masked]>
Reply-To: Bruce Lilly <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Fri, 4 Mar 2005 08:20:21 -0500
Cc: Keith Moore <[email-masked]>, wayne <[email-masked]>
Sender: [email-masked]

On Fri March 4 2005 00:19, Keith Moore wrote:
> >> Well, current SMTP specifications allow for anyone to use any domain
> >> in either the rfc2821 identities, or any place in rfc2822.  All
> >> authentication schemes intend to change that.
> >
> > AFAIK, none of SSL, TLS, or SMTP AUTH make any such change.
> 
> well, it depends on what you mean by "being able to use any domain"

Well, that was Wayne's comment.  From my perspective (as a mobile
user (i.e. connecting via an unpredictable IP address) with need
to use only a few domain names) I'd phrase the issue as "being
able to use any IP address"; and that's where SPF utterly fails.
To be precise, I mean that when sending mail, I need to specify
a mailbox in the return path which uses my ISP's domain name
(because that (my mailbox) is where legitimate bounces should go)
and likewise in the From and/or Reply-To and/or Sender fields of
the message header -- regardless of my IP address, and regardless
of the ISP that I happen to be sending the message(s) via, if I
use an ISP's SMTP relay, and regardless of whether I choose to
use such a relay (e.g. to store a single copy of a message for
distribution to multiple recipients) or to directly connect to
the designated MX host(s) for the message recipient(s).
 
> for instance, SSL/TLS server certificates do not allow the server to 
> [convincingly] claim to be any domain it wants to be within the SSL/TLS 
> protocol.   they don't try to restrict the server's domain name 
> according to the IP address,

or the client's!

> and they don't inherently impose  
> constraints on what domains the protocols layered over SSL or TLS use.  

> in the case of email, it's generally not 
> reasonable for an SMTP client or server to expect the domains in EHLO, 
> HELO, MAIL, or RCPT to match those used at the SSL or TLS layer, 
> because third-party relaying is an extremely useful feature of the SMTP 
> architecture.

And SMTP AUTH provides for SMTP session authentication independent
of transport layer security, and without imposing unreasonable
restrictions on IP address or domain name.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Fri, 04 Mar 2005 09:58:19 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 27
Sender: [email-masked]

Dave Crocker wrote:

> i really love their marketing...

Are you talking about the elected SPF Council, and Wayne as one
of its five members, or the "SPF community" in general, which
consists of active posters on its mailing list like you and me ?

In both cases the marketing is not much better than say MIPAS'.
It doesn't pretend to be IETF when it's Clear-ly not.  Maybe
Jabber could be a proper model for both MIPAS and SPF.

We're not exactly on topic here, it's Sender-ID that tries to
do wild and wonderful things with (2)822, abusing SPF policies,
and in violation of all SPF and the last MARID spf2.0 drafts.

All we can do here is help Wayne to get the syntax for the
Received-SPF header field right.  Or let's help Martin to get
his Archived-At right.

At the moment (spf-classic-00) he allows apparently no [CFWS],
{FWS], or *WSP between "=" and ";" in a key-value-pair.  If
that's for some reason not okay I'd be interested to hear it.

Received-SPF: Pass ip = 127.0.0.1 ; (not allowed ?)
.......................^.........^                   Bye, Frank

Cc: wayne <[email-masked]>, ietf-822 list <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Fri, 4 Mar 2005 00:19:34 -0500
To: Bruce Lilly <[email-masked]>
Sender: [email-masked]

>> Well, current SMTP specifications allow for anyone to use any domain
>> in either the rfc2821 identities, or any place in rfc2822.  All
>> authentication schemes intend to change that.
>
> AFAIK, none of SSL, TLS, or SMTP AUTH make any such change.

well, it depends on what you mean by "being able to use any domain"

for instance, SSL/TLS server certificates do not allow the server to 
[convincingly] claim to be any domain it wants to be within the SSL/TLS 
protocol.   they don't try to restrict the server's domain name 
according to the IP address, and they don't inherently impose 
constraints on what domains the protocols layered over SSL or TLS use.  
what they do is allow a server to convincingly claim to be a domain - 
if the client trusts the CA in the server certificate.
similar limitations apply to client certs.

whether it's appropriate for the higher-layer protocol to expect its 
domains to match those in the SSL/TLS certificates varies from one 
protocol to another.  in the case of email, it's generally not 
reasonable for an SMTP client or server to expect the domains in EHLO, 
HELO, MAIL, or RCPT to match those used at the SSL or TLS layer, 
because third-party relaying is an extremely useful feature of the SMTP 
architecture.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Thu, 3 Mar 2005 23:39:02 -0500
Cc: wayne <[email-masked]>
Sender: [email-masked]

On Thu March 3 2005 20:41, wayne wrote:

> Well, current SMTP specifications allow for anyone to use any domain
> in either the rfc2821 identities, or any place in rfc2822.  All
> authentication schemes intend to change that.

AFAIK, none of SSL, TLS, or SMTP AUTH make any such change. [One
would have been sufficient to refute that "All"; I'm a heavy tipper]

Cc: [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Thu, 3 Mar 2005 22:38:14 -0500
To: wayne <[email-masked]>
Sender: [email-masked]

>> all of the authentication schemes I've seen suffer from one or both of
>> these problems:
>>
>> -  trying to do more than is reasonable for that particular approach
>
> This seems to be a pretty subjective criteria.  I suspect that  
> reasonable people can disagree on what might be considered 
> "reasonable" for a particular approach.

yes, there will be some disagreement, but the opinions will often 
cluster in one general area, and you can get rough consensus on them.  
this does require engineering judgment.  it's not as if, for instance, 
there are objective criteria for how many DNS lookups are too many to 
use to validate an email address, but you'll get general agreement 
among most knowledgeable folks that there's a limit to how many lookups 
is reasonable, and those opinions will cluster around some compromise 
number.

>> -  trying to retroactively change how the mail protocol is used, or to
>> restrict future use of the mail protocol such that valid use cases
>> will no longer work
>
> Well, current SMTP specifications allow for anyone to use any domain 
> in either the rfc2821 identities, or any place in rfc2822.  All 
> authentication schemes intend to change that.

it's tempting to respond that all of these authentication schemes are 
therefore "broken".  but that's so simple a statement that it's likely 
to be taken the wrong way.

part of the problem is that none of the existing fields in either SMTP 
or [2]822 are intended to hold an authenticated ID of the originator, 
with the possible exception of Sender, even that one isn't well defined 
enough (or consistently implemented enough) to use.  and for every one 
of those fields there are numerous valid use cases for that field not 
being an authenticated ID for the originator, even though you might 
want to authenticate the message.  we simply don't have a field for 
that purpose defined at present.  various authentication schemes try to 
work around that by constraining or overloading existing fields.  the 
goal in doing so is to ease deployment, but what it actually does is 
limit the scheme's applicability.

there are also subtle but important differences between authenticating 
the return-path value as the originator and assuring an MTA or 
recipient that the originator had permission to use that return-path 
value.  we don't want to do the former, we do want to do the latter.

similarly there are subtle but important differences between 
authenticating the From address as the originator of a message and 
assuring a recipient that the originator was acting on behalf of the 
address(es) named in the From field. we need to support both.

it's also important that the architecture _not_ constrain originators 
to send their mail via a particular submission service.  it's okay if 
this is one way that mail can get authenticated, not okay if this is 
the only way that mail can get authenticated.

it's essential to get these details right.

>> what we really need is a framework that allows multiple schemes to
>> coexist and work constructively together.  but that can't happen as
>> long as the schemes try to change the mail protocol in incompatible
>> ways.
>
> Many of the email authentication schemes that I know of can co-exist
> and even work very well together.  I'm not sure that this is the
> appropriate forum to discuss which ones can and can't, and why.

I think that with slight modifications most of the schemes that have 
been proposed can co-exist and work well together, and that the schemes 
together will be much more useful than any of the schemes alone.

And if this isn't the appropriate forum to discuss the individual 
schemes, it probably is the appropriate forum to discuss a framework 
that would let the schemes coexist with each other and with Internet 
mail as it's actually used.

Keith

To: [email-masked]
From: wayne <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Thu, 03 Mar 2005 19:41:17 -0600
Lines: 47
Cancel-Lock: sha1:znCaXf7+ZxCJRVQPEQ7IrnMPVW8=
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>> The IETF, in the form of the IESG, has made it clear that they do not
>> want SPF going through a working group.  That is part of why MARID was
>> shut down.  They are pushing through both this SPF I-D, and the
>> Microsoft Sender-ID I-Ds, apparently without even an IETF last call.
>
> just as Informational or Experimental.  they can't standardize them
> without a Last Call.  even so a Last Call would still be advisable.

I agree that a last call would be be advisable.  I'll take your word on
information and/or experimental RFCs not needing them.  My reading of
RFC2026 section 6.1.2 seems to imply that both standard track (section
4.1) and non-standard track (section 4.2) I-Ds are handled the same
way.   But then, I'm not an RFC/IETF guru.

> all of the authentication schemes I've seen suffer from one or both of
> these problems:
>
> -  trying to do more than is reasonable for that particular approach

This seems to be a pretty subjective criteria.  I suspect that
reasonable people can disagree on what might be considered
"reasonable" for a particular approach.

> -  trying to retroactively change how the mail protocol is used, or to
> restrict future use of the mail protocol such that valid use cases
> will no longer work

Well, current SMTP specifications allow for anyone to use any domain
in either the rfc2821 identities, or any place in rfc2822.  All
authentication schemes intend to change that.  This looks pretty much
like a tautology to me.

> what we really need is a framework that allows multiple schemes to
> coexist and work constructively together.  but that can't happen as
> long as the schemes try to change the mail protocol in incompatible
> ways.

Many of the email authentication schemes that I know of can co-exist
and even work very well together.  I'm not sure that this is the
appropriate forum to discuss which ones can and can't, and why.

-wayne

To: [email-masked]
From: wayne <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Thu, 03 Mar 2005 19:22:32 -0600
Lines: 53
Cancel-Lock: sha1:4lNhukvCYhhvxndyZXSkr4h5K6U=
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

> On Thu March 3 2005 13:13, wayne wrote:
>
>> (Actually, off hand, I can't think of anything about SPF
>> that Dave does like, but...)
>
> He's not alone...

I understand, and I don't think it would be productive to rehash all
issues with SPF here.

>> They are pushing through both this SPF I-D, and the
>> Microsoft Sender-ID I-Ds, apparently without even an IETF last call.
>
> That would be a procedure violation, and could be appealed to the IAB.

Maybe, but it is pretty well shown here:
http://datatracker.ietf.org/state_diagram.gif

>> The IETF seems to be a pretty funny organization.  They clearly value
>> politics over technical considerations.
>
> That's quite a broad brush you're trying to paint with.  Perhaps
> you fail to understand some of the technical arguments against SPF...

Until this I-D, the IETF never evaluated SPF.  They created a working
group (MARID) and created a new proposal (SenderID) that was based on
many ideas, including both SPF and Caller-ID.  However, neither of
those proposals were adopted by the IETF.  

>> I'm not sure that I want to 
>> go through the effort of trying to create a working group, knowing
>> that I don't have the political clout to keep things focused on
>> technical issues.
>
> Is that a perverse way of stating that a) those who understand the
> technical issues aren't interested in SPF and/or b) those who are
> interested in SPF are unable or unwilling to confront its technical
> defects?

There have been lively debates, open to all, about SPF on the
SPF-discuss mailing list since 2003.  Most SPF proponents, and the I-D
in question also, acknowledge that SPF has problems in certain cases.
But, as I said in a previous email, if there was a clean, simple
solution to the problem of email authnetication, I think it would have
been found years ago.

-wayne

To: [email-masked]
From: wayne <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Thu, 03 Mar 2005 19:16:41 -0600
Lines: 21
Cancel-Lock: sha1:v7vfzJjMeVtZcc6kgbuXhIO+6Hk=
Sender: [email-masked]

In <200533113926.110456@bbfujip7> Dave Crocker <[email-masked]> writes:

>  On Thu, 03 Mar 2005 12:13:37 -0600, wayne wrote:
>>  Actually, off hand, I can't think of anything about SPF
>>  that Dave does like, but...)
>
> i really love their marketing...

Thanks for the kinds words Dave.

I'm a little confused though.

Did you have questions or comments about the draft?  If so, what are
they?  If not, why did you ask about this?

Also, what about the changes to the Received-SPF header?  Was that an
appropriate thing to change from previous drafts?

-wayne

Cc: [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Thu, 3 Mar 2005 18:49:52 -0500
To: wayne <[email-masked]>
Sender: [email-masked]

> The IETF, in the form of the IESG, has made it clear that they do not
> want SPF going through a working group.  That is part of why MARID was
> shut down.  They are pushing through both this SPF I-D, and the
> Microsoft Sender-ID I-Ds, apparently without even an IETF last call.

just as Informational or Experimental.  they can't standardize them 
without a Last Call.  even so a Last Call would still be advisable.

all of the authentication schemes I've seen suffer from one or both of 
these problems:

-  trying to do more than is reasonable for that particular approach
-  trying to retroactively change how the mail protocol is used, or to 
restrict future use of the mail protocol such that valid use cases will 
no longer work

what we really need is a framework that allows multiple schemes to 
coexist and work constructively together.  but that can't happen as 
long as the schemes try to change the mail protocol in incompatible 
ways.

Keith

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Thu, 3 Mar 2005 16:35:25 -0500
Sender: [email-masked]

On Thu March 3 2005 13:13, wayne wrote:

> (Actually, off hand, I can't think of anything about SPF
> that Dave does like, but...)

He's not alone...

> They are pushing through both this SPF I-D, and the
> Microsoft Sender-ID I-Ds, apparently without even an IETF last call.

That would be a procedure violation, and could be appealed to the IAB.

> The IETF seems to be a pretty funny organization.  They clearly value
> politics over technical considerations.

That's quite a broad brush you're trying to paint with.  Perhaps
you fail to understand some of the technical arguments against SPF...

> I'm not sure that I want to 
> go through the effort of trying to create a working group, knowing
> that I don't have the political clout to keep things focused on
> technical issues.

Is that a perverse way of stating that a) those who understand the
technical issues aren't interested in SPF and/or b) those who are
interested in SPF are unable or unwilling to confront its technical
defects?

From: Dave Crocker <[email-masked]>
To: wayne <[email-masked]>, <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Thu, 3 Mar 2005 11:39:26 -0800
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Sender: [email-masked]

 On Thu, 03 Mar 2005 12:13:37 -0600, wayne wrote:
>  Actually, off hand, I can't think of anything about SPF
>  that Dave does like, but...)

i really love their marketing...

 d/
 --
 Dave Crocker
 Brandenburg InternetWorking
 +1.408.246.8253
 dcrocker  a t ...
 WE'VE MOVED to:  www.bbiw.net

To: [email-masked]
From: wayne <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Thu, 03 Mar 2005 12:13:37 -0600
Lines: 85
Cancel-Lock: sha1:unDdIRTWJxTv01jdl7ByJcP/FyQ=
Sender: [email-masked]

note:

For those of you who have avoided the MARID WG and email
authentication in general, both Dave and I have been very active in
this area.  Dave has a somewhat competing proposal called "CSV" and
has made it clear that there are various things that SPF does that he
doesn't like.  (Actually, off hand, I can't think of anything about SPF
that Dave does like, but...)

In <200532223445.613242@bbfujip7> Dave Crocker <[email-masked]> writes:

>  On Wed, 02 Mar 2005 23:32:29 -0600, wayne wrote:
>>  I am really trying to aim toward your first goal.
>>
>>  SPF, as a protocol, isn't so entrenched that nothing can really be
>>  changed, but it is far too deployed to make any significant changes.
>
> I suspect you did not really understand Keith's point.
>
> You really do have to choose one of the two goals he listed.  "Trying" is not enough.

I'm sorry Dave, but I don't know everything.  Since I don't know how
every SPF implementation out there works, all I can do is try.  I do
know that various SPF implementations have had bugs that have been
fixed, and the percentage of deployed SPF implementations that still
have those bugs is decreasing.   I know of many different SPF
implementations and I know that several of the most popular ones
implement some things differently.

I do not think it would be appropriate to list every bug/difference in
every version of every SPF implementation.  Neither RFC822 nor RFC2822
do that either.

> As it is, I still have no idea what is in scope and what is out of
> scope for the effort.

Well, I'm not sure how to make it clearer.  Try asking your questions
and/or making your comments.  That will likely clear up the
miscommunication. 

> If you are trying to document what exists, then there is no 'still
> changing'.  None.

Sorry, but as I learn more about various things, my understanding of
what exists changes.  "Existing practices" are also changing, and will
continue to change.  Some of these changes are predictable, many
aren't.  I don't see this as a black and white area.  It is not
possible, and I don't think worthwhile, to document exactly what
things were like on a precise point of time.

Maybe we could use the example I gave in a previous message about the
Received-SPF header.  Should I document what was allowed under the
older SPF specs, which doesn't even match RFC2822's "obsolete" syntax,
or should I document something that is largely compatible with all
implementations that I know of and compatible with RFC2822?  I think
it reasonable to say that an already large percent of Received-SPF
headers conform to what is in the I-D (e.g. RFC2822 compatible), and
that in the future, this percentage will increase.

> If you are trying to document what you would like, I encourage you
> to either join the IETF process, by getting a working group
> chartered, or come back when you are done. It does not make sense to
> use part of the IETF process, except not really using it.

The IETF, in the form of the IESG, has made it clear that they do not
want SPF going through a working group.  That is part of why MARID was
shut down.  They are pushing through both this SPF I-D, and the
Microsoft Sender-ID I-Ds, apparently without even an IETF last call.
Maybe the MS folks made the right choice by not letting the other
relevant WG know about their I-Ds and I should have just kept quiet.

The IETF seems to be a pretty funny organization.  They clearly value
politics over technical considerations.   I'm not sure that I want to
go through the effort of trying to create a working group, knowing
that I don't have the political clout to keep things focused on
technical issues.

-wayne

From: Dave Crocker <[email-masked]>
To: wayne <[email-masked]>, <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Wed, 2 Mar 2005 22:34:45 -0800
Delivery-Date: Wed, 2 Mar 2005 22:34:44 -0000
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Sender: [email-masked]

 On Wed, 02 Mar 2005 23:32:29 -0600, wayne wrote:
>  I am really trying to aim toward your first goal.
>
>  SPF, as a protocol, isn't so entrenched that nothing can really be
>  changed, but it is far too deployed to make any significant changes.

I suspect you did not really understand Keith's point.

You really do have to choose one of the two goals he listed.  "Trying" is not enough.

As it is, I still have no idea what is in scope and what is out of scope for the effort.

If you are trying to document what exists, then there is no 'still changing'.  None.  

If you are trying to document what you would like, I encourage you to either join the IETF process, by getting a working group chartered, or come back when you are done. It does not make sense to use part of the IETF process, except not really using it.

 d/
 --
 Dave Crocker
 Brandenburg InternetWorking
 +1.408.246.8253
 dcrocker  a t ...
 WE'VE MOVED to:  www.bbiw.net

Date: Thu, 03 Mar 2005 22:52:13 +0800
From: "Z. Rahman" <[email-masked]>
To: [email-masked]
Subject: (no subject)
Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAFVBMVEWeqbXeup5FhLaybDbN fzTNdhZUPTZrEQqFAAABsklEQVQ4jW3SwY7bIBAG4Em66jmjoD0jKvW8zhDOtRj3BQx73sYN7/8I HWywjdORNt7w5Z/BNsBjh4j32FQYArDvWwiRY+AM3EBg5rCAZ/b7BA810QIL8AxWZINBepVECzHP kD+wxC8gNQDZFSh/DrIoCxE8FxhTSs81ITMqpFzuvkG+QYHPGSSzAOcEYn+PboGpBXVfOkmt0Kvl WZX1KYQyAxfII1C5XStU/QqopNWWkHUcBP7I1W0zECvI/1dpFY4wya69ex4To8Cv7pKeIQwNRMyJ q4t5u0MDVi7qdx2xQZ9vB9836Ap0GdT7GHjYg4+U74ccrfBREp9/5XJ1xMddjZM8GuXI2gPEAvMZ CCvIq52sw+sDjPlWQc0w/iA3OZPrrcDyokYtjzYZcwbQbFegUQDpw2itAd7kFEI3v0GKN3NCvJwl IGXJQkfcoyI6GwlfAFbIZxeZbsbAKcMSIQc0H2qSzYA+1QSkVOCmwWjY6nsF6SCRXSXwRHn/0ryB n+AnOVEP6XPer8s3P0ngoeGlfO70aH5bIc0jNihpIpnQzNX17ufRtS6vcNyVtPr6D5h/dL/cLyoS djMAAAAASUVORK5CYII=
Sender: [email-masked]

subscribe

To: [email-masked]
From: wayne <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Wed, 02 Mar 2005 23:32:29 -0600
Lines: 97
Cancel-Lock: sha1:qaBJPK5lPuVjsps2TUwuaHFZxRs=
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>> While I'm very interested in learning about things that SPF
>> implementations usually do that aren't (adequately) document, or
>> things that are documented that most SPF implementations don't do,
>> there are still a few things that are being changed around the edges.
>> Basically, the more incompatible the suggested change, the more
>> evidence will be required that it is a *really* important and
>> necessary change.
>
> If I may make a suggestion -
>
> I have generally found that when documenting an existing protocol it's
> important to pick one of two goais:
>
> 1) document the protocol as it is used in practice, or as it was used
> as of a particular date
> 2) document how the protocol SHOULD work

I am really trying to aim toward your first goal.

SPF, as a protocol, isn't so entrenched that nothing can really be
changed, but it is far too deployed to make any significant changes.

As an example of things that have changed in this round of SPF
spec-writing is the definition of the Received-SPF: header.  It had
numerous things that did not conform to what RFC2822 said that headers
should look like.  With a few, somewhat rare exceptions, the actual
SPF implementations produce headers that can be defined with
RFC2822-compatible ABNF, so I have updated the ABNF in the SPF spec to
match both existing practices, and RFC2822.

The Received-SPF header isn't a critical part of the SPF specification
anyway, so I think that compatiblity with RFC2822 is a bigger gain
than the slight breakage that changing the ABNF causes.

There are other changes like going from "MAY" to "SHOULD", or vice
versa.

> so if you're going to document how the protocol is/was used in
> practice, resist the temptation to define new features.

Actually, new features are nowhere on the list of things I'm trying to
do.  Just the opposite.  I have identified quite a few features in
older versions of the specs that have never been implemented by
anyone, and I have removed them.  The -01 version, which should be
release Real Soon Now, will remove another of these features.

As a side benefit of removing the unused features, I have been able to
keep the number of pages used to describe the spec constant for quite
a while now.  There are always things you could add to try and make
things clearer, or more fully explain some detail, *BUT* ever time I
do, I try very hard to find something else in the spec that isn't as
important and delete it.

> another nice thing about documenting how the protocol is/was used is
> that you can avoid a lot of controversy on whether or not the protocol
> is a good idea.

Yes, very much agreed.  During 2003, the SPF spec was hotly debated
and a pretty good spec was written up, things were pretty much
finalized, quite afew implementations were written and a ton of SPF
records were published.  Then came the IETF MARID WG, and everyone
thought it would be A-OK to change the heck out of the semantics and
there was a repeat of all the debates from 2003 about how things work
and which were the right choices.

Basically everything from that MARID working group is being thrown
out, except for the much needed wordsmithing on the spec.

>                                               the only controversies
> should be about whether you're documenting the protocol accurately,
> and perhaps about whether you're doing due diligence in pointing out
> any hazards (e.g. security or operational considerations) associated
> with the legacy protocol. 

This is my goal, and I would greatly appreciate comments about what
things still need to be done better.  

>                                                          (frankly I
> think that there's a limit to how effective any proposal in this space
> can be, and it's probably not worth the trouble to try to make SPF be
> the ultimate email authentication scheme.  and I'd say the same thing
> about the other authentication proposals I've seen)

SPF was never designed to be the Final and Ultimate Solution to the
Spam Problem.  Many people in the SPF community also support other
authentication proposals as complimentary.  I think that if a clean,
easy solution to the email authentication problem existed, it would
have been thought of long ago.

-wayne

Cc: [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Wed, 2 Mar 2005 23:54:57 -0500
To: wayne <[email-masked]>
Sender: [email-masked]

> While I'm very interested in learning about things that SPF
> implementations usually do that aren't (adequately) document, or
> things that are documented that most SPF implementations don't do,
> there are still a few things that are being changed around the edges.
> Basically, the more incompatible the suggested change, the more
> evidence will be required that it is a *really* important and
> necessary change.

If I may make a suggestion -

I have generally found that when documenting an existing protocol it's 
important to pick one of two goais:

1) document the protocol as it is used in practice, or as it was used 
as of a particular date
2) document how the protocol SHOULD work

trying to do both tends to produce a document which is effective at 
neither.  often the legacy protocol has enough shortcomings (whether 
these are actual bugs or new features that people demand) that trying 
to solve all of those shortcomings delays completion of the 
specification until it is no longer relevant.  or if you're trying to 
document how the protocol SHOULD work then you often need to deprecate 
or discourage features of the legacy protocol.  if nothing else, trying 
to intermix new features (not supported in the legacy protocol) and 
features of the legacy protocol which should be deprecated tends to 
produce a confusing document.  and when people try to use that document 
as a basis for new implementations, you get more interoperability 
problems as a result.

(a good example is RFC 1179, which did a poor job of documenting the 
LPD protocol, and also tried to define some new features of dubious 
value.  the result is that RFC 1179 actually degraded interoperability 
of the LPD protocol rather than improving it)

so if you're going to document how the protocol is/was used in 
practice, resist the temptation to define new features.  if you must 
define new features, get the text about the old features settled first, 
and put the new features in a separate section (or appendix) in order 
to clearly distinguish the new features from the old ones.  and while 
it's probably useful and appropriate to point out things about the 
legacy protocol that didn't work well, it may be a good idea to put 
those in a separate section also.

another nice thing about documenting how the protocol is/was used is 
that you can avoid a lot of controversy on whether or not the protocol 
is a good idea.  you're just documenting something for the record, not 
trying to get consensus on a recommendation.  the only controversies 
should be about whether you're documenting the protocol accurately, and 
perhaps about whether you're doing due diligence in pointing out any 
hazards (e.g. security or operational considerations) associated with 
the legacy protocol.  for instance it would seem appropriate to point 
out the conflicts between the assumptions that SPF makes regarding use 
of SMTP protocol data elements and/or 822 header fields, the way these 
fields are specified, and they way they're actually used in practice.

otoh, if you're going to document how the protocol SHOULD work, then 
put the deprecated features (if any) in a separate section.  and be 
prepared to take a couple of years to get consensus on the document, as 
people try to get it to be all things to all people.  (frankly I think 
that there's a limit to how effective any proposal in this space can 
be, and it's probably not worth the trouble to try to make SPF be the 
ultimate email authentication scheme.  and I'd say the same thing about 
the other authentication proposals I've seen)

but it sounds like you're trying to do the former.

Keith

To: [email-masked]
From: wayne <[email-masked]>
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Date: Wed, 02 Mar 2005 21:14:49 -0600
Lines: 37
Cancel-Lock: sha1:ps7WP/yWvSGzzqg6jCelQ8H/f3o=
Sender: [email-masked]

In <20053293512.007816@bbfujip7> Dave Crocker <[email-masked]> writes:

>>  There is a new I-D for the SPF email anti-forgery system available for
>>  review.  ...
>>  Discussions about this, and previous, drafts have been taking place on
>>  the [email-masked] mailing list.  ...
>>  for the purposes of this
>>  draft, I am very reluctant to try debate these issues.  The goal is to
>>  document a de-facto standard.  Debates about better techniques, why
>>  SPF is evil, etc. are probably best discussed on things like the IRTF
>
> so the constraint to review feedback is either "yes it works this
> way" or "no, it doesn't, it work's this other way"?

Not entirely.

While I'm very interested in learning about things that SPF
implementations usually do that aren't (adequately) document, or
things that are documented that most SPF implementations don't do,
there are still a few things that are being changed around the edges.
Basically, the more incompatible the suggested change, the more
evidence will be required that it is a *really* important and
necessary change.

What I really wanted to head off with what I said above are comments
like "we should use SMTP callbacks instead!", or "instead of an ad-hoc
language, we should use XML", or "we shouldn't use DNS to communicate
the sender policy", or "SPF's HELO checking could be done with one DNS
lookup instead of the current one or two if we used an A RR type.".
All of these positions have been strongly argued both pro and con.
The point is that any of these changes would result in something that
*isn't* the existing SPF and therefore *isn't* the goal of this I-D.

Most of all, I wanted to avoid re-hashing these issues here.

-wayne

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Wed, 2 Mar 2005 12:54:54 -0500
Cc: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

On Wed March 2 2005 10:04, Charles Lindsey wrote:

> There speaks a man whose mind is made up, and who does not wish to be
> confused by the facts.

You've presented no facts, just a figment of your imagination.

From: Dave Crocker <[email-masked]>
To: wayne <[email-masked]>, <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Wed, 2 Mar 2005 09:35:12 -0800
Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt
Sender: [email-masked]

>  There is a new I-D for the SPF email anti-forgery system available for
>  review.  ...
>  Discussions about this, and previous, drafts have been taking place on
>  the [email-masked] mailing list.  ...
>  for the purposes of this
>  draft, I am very reluctant to try debate these issues.  The goal is to
>  document a de-facto standard.  Debates about better techniques, why
>  SPF is evil, etc. are probably best discussed on things like the IRTF

so the constraint to review feedback is either "yes it works this way" or "no, it doesn't, it work's this other way"?

 d/
 --
 Dave Crocker
 Brandenburg InternetWorking
 +1.408.246.8253
 dcrocker  a t ...
 WE'VE MOVED to:  www.bbiw.net

To: [email-masked]
Xref: clerew local.mime:4525
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Wed, 2 Mar 2005 15:04:09 GMT
Lines: 34
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>On Mon February 28 2005 08:07, Charles Lindsey wrote:

>> So we can consider how those requirements are met in the 4 cases:
>> 
>> 
>>                 1. No From      2. blah.invalid 3. @[]          4. From <>
>> 
>> A. delivery     98% [1]         100%            98%             98%
>[...]
>> Clearly, this analysis shows that no solution is going to be perfect (but
>> we knew that already). OTOH, it does seem to shown that #1 has worse
>> problems than the others.

>What it shows is that you can start from preconceived notions,
>make up a bunch of random numbers unrelated to any sort of
>testing or scientific basis, then say "See, it shows I was right".

>Bollocks.

There speaks a man whose mind is made up, and who does not wish to be
confused by the facts.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Update to "simplified" RFC 2822 grammar
Date: Wed, 2 Mar 2005 11:49:43 -0500
Cc: Pete Resnick <[email-masked]>
Sender: [email-masked]

I have made some fairly minor changes to the "simplified"
RFC 2822 grammar at
http://users.erols.com/blilly/mparse/rfc2822grammar_simplified.txt

1. added some lines/characters to support parsing/formatting
   by an ABNF formatter (http://users.erols.com/formatting/abnff)

2. the formatter found a few errors (3 or 4 extraneous right
   parentheses, and an obs-comments alternative incorrectly
   labelled comments; these have been fixed

Date: Tue, 1 Mar 2005 14:16:48 -0500
From: Keith Moore <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: MTS transparency and anonymity
Sender: [email-masked]

> Now, the issue is what here?

timing hazards.  client A talks to server B, server B talks to server C,
which is the MX for the originator's email address.

if server C doesn't respond until near the end of server B's timeout
(due to server load or network congestion or whatever), server B
will give up.  when client A retries, often (in my experience) the same 
thing happens again.  that is, a lot of the times the delays aren't
randomly scattered (which would be recoverable) but are clustered for
certian sender addresses.

SMTP as currently specified wasn't designed to handle real-time 
callbacks.  for instance, there is a single set of timeout limits
for all servers.

> This is why it works for thousands of our installations with many HAPPY
> faces and little to know false positives.

most of my thousands of users had no complaints. but a few were extremely
unhappy, to say the least, when their mail stopped working.

my situation is probably different than yours.  I run a server which
provides two services to its users: mail forwarding and delivery of a
moderated weekly digest.  the callbacks were used to filter out spam 
and viruses from mail sent to forwarding addresses, from digest 
messages, and from service requests.  the users are scattered all over
the world, mostly in academic institutions, but some have better 
connectivity than others.

when our users have a complaint, they're not usually able to resolve
it at their site.  it sounds like your customers are more likely to
be sites than individual users, and that your customers' problems are
more likely to be site-wide than my customers' problems.

> This is my only comment about this and I will steer clear of this thread.  I
> just wanted to point out that the real experts are in the field actually
> putting this stuff to work in practice - it is not theory, a raised back
> hair or a devil on your shoulder whispering into your ear.

and some of the real experts are getting unfavorable results.

Keith

From: "Hector Santos" <[email-masked]>
To: "ietf-822" <[email-masked]>, "Vince Sabio" <[email-masked]>
Subject: Re: Site policy vs. HELO
Date: Tue, 1 Mar 2005 09:16:00 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

Typo Fix:

----- Original Message -----
From: "Hector Santos" <[email-masked]>
To: "ietf-822" <[email-masked]>; "Vince Sabio" <[email-masked]>
Sent: Tuesday, March 01, 2005 9:08 AM
Subject: Re: Site policy vs. HELO

> Vince,
>
> I would think section 7.7 inherently implies HELO to be included when it
> mentioned "EHLO." It might be a slip in not including HELO along with
EHLO.
> I think the principle idea was about site policy rejection code
> recommendation and using a 55x response code.
>
> The fallback response code is 50x which is the class of "unknown command"
> response codes. This is what you would normally expect for a non-ESMTP
host
> site who does not support or recognize HELO. See section 4.2.4 about 502
vs.
> 500.
>

I meant "....who does not support or recognize EHLO."

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

From: "Hector Santos" <[email-masked]>
To: "ietf-822" <[email-masked]>, "Vince Sabio" <[email-masked]>
Subject: Re: Site policy vs. HELO
Date: Tue, 1 Mar 2005 09:08:44 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

Vince,

I would think section 7.7 inherently implies HELO to be included when it
mentioned "EHLO." It might be a slip in not including HELO along with EHLO.
I think the principle idea was about site policy rejection code
recommendation and using a 55x response code.

The fallback response code is 50x which is the class of "unknown command"
response codes. This is what you would normally expect for a non-ESMTP host
site who does not support or recognize HELO. See section 4.2.4 about 502 vs.
500.

For example:

The ESMTP compliant sender will issue an EHLO and generally, you can get a
clue by looking for "ESMTP" as part of the standard welcome line (BCP), but
this is not reliable.  So the ESMTP client should always try EHLO first.

S: 250 %hostdomain%, %ProductName% ESMTP server %version% ready.
C: EHLO xxxxx
S: 500 Unrecognized command: EHLO

This is when you should fall back when a 50x is received.  You should not
fall back for a 55x response which is what 7.7 is basically saying, and I
believe that includes a 55x response to HELO as well per the site policy.

PS: A 55x response to HELO/EHLO is a growing practice.  It kills a
significant amount of bulk spammers specially thus who pipe line the
transaction. Simply add a strict SMTP compliancy and you will catch much of
it.  The only issue you will run across - legacy applications using
incorrect domain literals and/or NAT translated addresses.

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

----- Original Message -----
From: "Vince Sabio" <[email-masked]>
To: "ietf-822" <[email-masked]>
Sent: Monday, February 28, 2005 12:44 PM
Subject: Site policy vs. HELO

>
> I'm bringing this question over here from the SPAM-L list.
>
> The following graf in RFC2821 appears to require an MTA (not an
> RFC2476-type MSA) to fall back to HELO for compatibility with older
> SMTP clients and servers):
>
>     [QUOTE RFC2821 SECT 2.2.1]
>
>     Contemporary SMTP implementations MUST support the basic extension
>     mechanisms.  For instance, servers MUST support the EHLO command even
>     if they do not implement any specific extensions and clients SHOULD
>     preferentially utilize EHLO rather than HELO.  (However, for
>     compatibility with older conforming implementations, SMTP clients and
>     servers MUST support the original HELO mechanisms as a fallback.)
>
>     [END QUOTE]
>
> However, it has recently been pointed out that the following graf in
> RFC2821 may permit sites to _not_ be required to fall back to HELO --
> and issue a 550 response to HELO instead -- if local policy dictates:
>
>     [QUOTE RFC2821 SECT 7.7]
>
>     In recent years, use of the relay function through arbitrary sites
>     has been used as part of hostile efforts to hide the actual origins
>     of mail.  Some sites have decided to limit the use of the relay
>     function to known or identifiable sources, and implementations SHOULD
>     provide the capability to perform this type of filtering.  When mail
>     is rejected for these or other policy reasons, a 550 code SHOULD be
>     used in response to EHLO, MAIL, or RCPT as appropriate.
>
>     [END QUOTE]
>
> So, the question: Is this a valid interpretation of Sect. 7.7 --
> i.e., may an MTA provide a 550 response to HELO where it would
> otherwise have given a 250 response to an EHLO if the site policy for
> that MTA forbids HELO?
>
> __________________________________________________________________________
> Vince Sabio                                                  [email-masked]
>
>

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Tue, 1 Mar 2005 08:24:22 -0500
Cc: "Charles Lindsey" <[email-masked]>
Sender: [email-masked]

On Mon February 28 2005 08:07, Charles Lindsey wrote:

> So we can consider how those requirements are met in the 4 cases:
> 
> 
>                 1. No From      2. blah.invalid 3. @[]          4. From <>
> 
> A. delivery     98% [1]         100%            98%             98%
[...]
> Clearly, this analysis shows that no solution is going to be perfect (but
> we knew that already). OTOH, it does seem to shown that #1 has worse
> problems than the others.

What it shows is that you can start from preconceived notions,
make up a bunch of random numbers unrelated to any sort of
testing or scientific basis, then say "See, it shows I was right".

Bollocks.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: MTS transparency and anonymity
Date: Tue, 01 Mar 2005 09:22:35 +0100
Organization: <URL:http://purl.net/xyzzy>
Lines: 44
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

Keith Moore wrote:

> any auto-responder is going to create some collateral spam.
> there's no way around that.

If enough forgeries are rejected before they can hit their
victims incl. auto-responders, then the spammers will use
other addresses in their reverse paths.  And then it would
be again "some damage" instead of the "huge damage" today.

If that's impossible the huge damage would force users to
delete anything which might be "collateral spam" without
manually checking it, and that would be the end of SMTP as
we know it.

 [bounces-to vs. sender] 
> that's not the essence of the problem.

IBTD, and after Bruce mentioned RfC 733 here I looked into
it.  It's the same concept of sender as in STD 10:  "a MAIL
command indicating the sender of the mail", "a reverse-path,
which specifies who the mail is from".

> MAIL FROM is fundamentally the address where bounces go.  It
> is NOT necessarily any of the source, the sender, or the
> originator.

You need another net or another way to transport mail before
the 1st Internet MTA for differences.  MAIL FROM is what the
name says, fundamentally the same idea as the RfC 733 sender.

> it's extremely important that you understand this.

Maybe we could join Bruce and Dave in their time travel, when
they fix RfCs starting with 733.  But actually I'm happy with
the old MAIL FROM sender concept, if it's not abused as an
arbitrary "bounces-not-to-me" address by spammers.

I had about 180,000 of these "collateral spams" too many for
my V.90 connection to a POP3 mailbox with a catch-all vanity
host, before an IP-based MAIL FROM protection scheme stopped
this.
                         Bye, Frank