Subject: Re: rfc2821bis-04, Issue 38: MX queuing behavior with options
From: SM <[email-masked]>
To: [email-masked]
Date: Thu, 31 May 2007 20:57:50 -0700
List-Id: <ietf-smtp.imc.org>

At 16:13 31-05-2007, Randall Gellens wrote:
>recipients don't see the others, and hence a 'reply all' misses them 
>entirely.  In addition, with UTF8SMTP, there is some assumption that 
>a UTF8 address implies that the recipient's server supports the 
>extension, otherwise where did the address come from?

That assumption can cause problems if the address is generated by a 
rogue process.

Regards,
-sm 

Date: Thu, 31 May 2007 17:01:34 -0700
To: John C Klensin <[email-masked]>, [email-masked]
From: SM <[email-masked]>
Subject: Re: rfc2821bis-04, Issue 38: MX queuing behavior with options
Sender: [email-masked]

Hi John,
At 13:39 31-05-2007, John C Klensin wrote:
>While I have no recollection that we ever thought of or
>discussed it, the situation changed when we adopted the
>extension mechanism.  Suddenly, we could have two servers at the
>same preference level, one of which supported a given extension
>and one which did not.   There is a case to be made that this
>would be a dumb way to configure things, but I don't think we
>say that anywhere, much less make equivalent configurations a
>requirement.   The situation is similar, but arguably worse, for
>systems with less-desirable preference levels: again, there is
>no requirement that they support all of the extension
>capabilities of the destination server.

It may happen that two servers at the same preference level may not 
support the same extensions.  I'll assume that it's not the general 
case and that if it happens, remedial action can be taken as the two 
servers would most likely be under the same administrative control.

>Suppose we have a client that would really prefer to send mail
>to [email-masked] using the FOOBAR extension.  An MX query on
>foo.example yields
>     foo.example. IN MX 10 smtp1.foo.example.
>                  IN MX 10 smtp2.foo.example.
>                  IN MX 20 backup.foo.example.
>
>Suppose it tries smtp2.foo.example, gets the SMTP connection
>open, but the EHLO response does not contain FOOBAR.  Is it
>required to proceed using the non-FOOBAR behavior, or may it
>(should it?) close the connection to smtp2, open one to
>smtp1.foo.example, and see if *it* supports FOOBAR?  If a
>connection to smtp1 cannot be opened immediately, how long
>should it wait (for a timeout) before giving up and going back
>to smtp2?

If the client were to proceed to smtp1.foo.example because the EHLO 
response for smtp2.foo.example does not contain FOOBAR, then we have 
a strategy where the client may go looking at all hosts until it 
finds one which supports FOOBAR.  Section 2.2.3 mentions an overall 
timeout only.  I assume that the same sending strategy would apply 
for going back to smtp2.

>Before answering, see if the answer changes if the nature of the
>FOOBAR extension is such that, if that service is not available,
>the message will be rejected or returned, not forwarded with
>some restricted capabilities.

The nature of the FOOBAR extension may determine the behavior.

>Note that, if they client believes that it needs both the FOOBAR
>and BARFOO extensions, and smtp2 offers only BARFOO while smtp1
>offers only FOOBAR, things get even more interesting.
>
>I believe that, with the way the text in 2821 is written, a
>client that is successful in opening an SMTP session (getting
>the 220 greeting, sending an EHLO command, and getting a
>response) to at least one of smtp1 or smtp2 is not permitted to
>fall back to backup.foo.example under any circumstances.  But,
>even there, odd cases exist if the message is queued.

I discussed about a SMTP extension which allows the client to go to 
backup.foo.example.  I don't think that it should fall back to 
backup.foo.example as the problem (SMTP extension not support) would 
persist downstream.

There are two cases, one where the message is not affected and the 
other where it is.  In the first case, if FOOBAR is not present, the 
message is either forwarded or rejected.  In the second case, the 
message can either be forwarded, downgraded or rejected.

Using the following example:

     foo.example. IN MX 10 smtp1.foo.example.
                  IN MX 10 smtp2.foo.example.
                  IN MX 20 backup.foo.example.

Mail gets delivered to mail.foo.example.

In case one, the client uses its local policy to decide whether to 
fall back to non-FOOBAR behavior or reject.

In case two, it's the final hop (mail.foo.example) which really 
dictates whether the message will be downgraded or not.  Going from 
smtp1.foo.example to smtp2.foo.example won't change that.

Regards,
-sm 

Date: Thu, 31 May 2007 16:13:16 -0700
To: John C Klensin <[email-masked]>, [email-masked]
From: Randall Gellens <[email-masked]>
Subject: Re: rfc2821bis-04, Issue 38: MX queuing behavior with  options
Sender: [email-masked]

I think it would be good to have some discussion.  The harder 
question is what to say.  I'd like to say that there are some 
extensions where lack of support is an especially bad thing, and 
where there are some hints to the client-side that the recipient 
server normally supports them.  The example I'm thinking of is 
UTF8SMTP, since lack of support when the message requires it means 
the message must be bounced or possibly worse, downgraded.  The 
downgrade can split the recipients into separate trees, where some 
recipients don't see the others, and hence a 'reply all' misses them 
entirely.  In addition, with UTF8SMTP, there is some assumption that 
a UTF8 address implies that the recipient's server supports the 
extension, otherwise where did the address come from?  For these 
extensions, it might be a better choice to try another server rather 
than do violence to the message.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Pohl's law:
    Nothing is so good that somebody, somewhere, will not hate it.

Date: Thu, 31 May 2007 16:39:58 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: rfc2821bis-04, Issue 38: MX queuing behavior with options
Sender: [email-masked]

Hi.

A discussion elsewhere has brought this up and I want to note it
here.  I don't think, given the rules, that there is much that
can be said in 2821bis other than give a little advice and there
may be about as much of that as possible in there already).

The problem goes like this:

In RFC 974 and 1123, and the text that was picked up in 2821,
the assumption was that all SMTP servers were pretty much the
same: if one found a server at a given preference level, its
capabilities were the same as any other server at that
preference level and, even if one went to a less-preferred
server, the capabilities were equivalent.  In all of these cases
the "capability" was simply ability to move mail to its
destination.

While I have no recollection that we ever thought of or
discussed it, the situation changed when we adopted the
extension mechanism.  Suddenly, we could have two servers at the
same preference level, one of which supported a given extension
and one which did not.   There is a case to be made that this
would be a dumb way to configure things, but I don't think we
say that anywhere, much less make equivalent configurations a
requirement.   The situation is similar, but arguably worse, for
systems with less-desirable preference levels: again, there is
no requirement that they support all of the extension
capabilities of the destination server.

Suppose we have a client that would really prefer to send mail
to [email-masked] using the FOOBAR extension.  An MX query on
foo.example yields
    foo.example. IN MX 10 smtp1.foo.example.
                 IN MX 10 smtp2.foo.example.
                 IN MX 20 backup.foo.example.

Suppose it tries smtp2.foo.example, gets the SMTP connection
open, but the EHLO response does not contain FOOBAR.  Is it
required to proceed using the non-FOOBAR behavior, or may it
(should it?) close the connection to smtp2, open one to
smtp1.foo.example, and see if *it* supports FOOBAR?  If a
connection to smtp1 cannot be opened immediately, how long
should it wait (for a timeout) before giving up and going back
to smtp2?  

Before answering, see if the answer changes if the nature of the
FOOBAR extension is such that, if that service is not available,
the message will be rejected or returned, not forwarded with
some restricted capabilities.

Note that, if they client believes that it needs both the FOOBAR
and BARFOO extensions, and smtp2 offers only BARFOO while smtp1
offers only FOOBAR, things get even more interesting.

I believe that, with the way the text in 2821 is written, a
client that is successful in opening an SMTP session (getting
the 220 greeting, sending an EHLO command, and getting a
response) to at least one of smtp1 or smtp2 is not permitted to
fall back to backup.foo.example under any circumstances.  But,
even there, odd cases exist if the message is queued.

Anyone want to argue for a discussion of this in 2821bis and, if
so, what should be said?

     john

Date: Fri, 25 May 2007 14:14:48 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: draft-hutzler-spamops-07
Sender: [email-masked]

Hi,

In draft-hutzler-spamops-07, the introduction mentions that the near 
ubiquity, rapid delivery and low cost of email makes it a fertile 
ground for the distribution of unwanted or malicious content.  That 
misses the main characteristic.

The very characteristic that make email such a convenient 
communications medium is that it does not require any prior 
arrangement between the sender and the receiver for delivery.  The 
lack of authentication combined with the near ubiquity, rapid 
delivery and low cost of email makes it a fertile ground for the 
distribution of unwanted or malicious content.

Regards,
-sm

From: [email-masked]
Cc: [email-masked], John Leslie <[email-masked]>, Dave Crocker <[email-masked]>, [email-masked]
Date: Tue, 22 May 2007 09:04:42 -0700 (PDT)
Subject: Re: Criteria for "interoperability" for email-arch
To: Lisa Dusseault <[email-masked]>
Sender: [email-masked]

> On May 15, 2007, at 12:22 PM, [email-masked] wrote:

> >
> > I went so far to recommend more or less this approach for the ABNF
> > specification and even went so far as to generate a list of
> > references and uses
> > of each of ABNF's "features". This was in effect rejected by the
> > IESG, however,
> > in favor of documenting the existance of at least two ABNF validators.

> That wasn't so much rejected as it just happened that I felt like
> fooling around with ABNF validators.  I learned a lot while doing it
> and once done, writing the interop report went quickly.  I admit this
> was probably not my highest priority thing to do at the time but it
> was fun.

> >
> > I thought this was a poor decision at the time and still do. IMO
> > what matters
> > about ABNF - and by extension any other document that provides
> > definitional
> > tools for other specifications - is whether or not it is actually
> > used by
> > mulitplle specifications to achieve bettter specification clarity
> > and therefore
> > increase the chance of creating interoperable implementations.
> > Whether or not
> > there happens to be a way to backstop the use of such a document
> > with a
> > validation tool strikes me as largely irrelevant.

> Not really sure why you think it was a poor decision.  I also told
> other ADs my strong opinion that the real reason to move ABNF to
> Internet Standard was that we use it all the time and it works for us
> so "duh".

I completely agree that this really is a big "duh" in the ABNF case. It is
obvious that ABNF "works" pretty darned well and should be at Standard. My
problem with the process that was used is more general in nature: The existance
or nonexistance of validation tools really tells us nothing about the
underlying utility or lack thereof of the specification format. What needs to
be assessed is the actual use of the format in other documents and whether or
not it accomplished what it was supposed to.

Another case you might consider when thinking about this is XML Schema versus
Relax NG. It isn't an exact parallel since both XML Schema and Relax are used
operationally in addition to being specification aids. (ABNF is not used
operationally AFAIK.) But excluding the operational considerations and ignoring
personal preferences, it would not make sense to argue that XML Schema is more
suitable for a given task simply because there's more support out there for XML
Schema validation. What's supported and where is more an accident of timing and
labelling than anything else.

				Ned

Date: Mon, 21 May 2007 16:22:12 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Lisa Dusseault <[email-masked]>
CC: [email-masked]
Subject: Re: Criteria for "interoperability" for email-arch
Sender: [email-masked]

Lisa Dusseault wrote:
> No, I can't agree with that at all.  How about one writes 3 
> Standards-track documents, each of which normatively refers to the other 

Clever scenario.  Are you seriously worried about that sort of "attack"?  The 
implication that there would be such a deeply meretricious conspiracy within 
the IETF is pretty stunning.

IN any event, I suppose that making the requirement be citation in "protocol 
specifications"  would remedy that concern.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Cc: [email-masked], [email-masked]
From: Lisa Dusseault <[email-masked]>
Subject: Re: Criteria for "interoperability" for email-arch
Date: Mon, 21 May 2007 15:27:08 -0700
To: [email-masked]
Sender: [email-masked]

No, I can't agree with that at all.  How about one writes 3 Standards- 
track documents, each of which normatively refers to the other 2.  No  
implementations are written.  How does the use of references  
demonstrate interoperability?  What am I missing?

Lisa

On May 21, 2007, at 11:11 AM, Dave Crocker wrote:

>
>
> Lisa Dusseault wrote:
>>>        IMO what matters about ABNF - and by extension any other  
>>> document that provides definitional tools for other  
>>> specifications - is whether or not it is
>>> actually used by mulitplle specifications to achieve bettter
>>> specification clarity
> ...
>>       I also told other ADs my strong opinion that the real reason  
>> to move ABNF to Internet Standard was that we use it all the time  
>> and it works for us so "duh".
>
>
>
> Independent of discussing history, this sounds as if there is  
> agreement that use as two normative references in standards track  
> documents should satisfy the interoperability requirement, yes?
>
> d/
>
> -- 
>
>   Dave Crocker
>   Brandenburg InternetWorking
>   bbiw.net

Date: Mon, 21 May 2007 11:11:12 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Lisa Dusseault <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: Criteria for "interoperability" for email-arch
Sender: [email-masked]

Lisa Dusseault wrote:
>>        IMO what 
>> matters about ABNF - and by extension any other document that provides 
>> definitional tools for other specifications - is whether or not it is
>> actually used by mulitplle specifications to achieve bettter
>> specification clarity 
...
> 
>       I also told other 
> ADs my strong opinion that the real reason to move ABNF to Internet 
> Standard was that we use it all the time and it works for us so "duh".

Independent of discussing history, this sounds as if there is agreement that 
use as two normative references in standards track documents should satisfy 
the interoperability requirement, yes?

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Cc: John Leslie <[email-masked]>, Dave Crocker <[email-masked]>, [email-masked]
From: Lisa Dusseault <[email-masked]>
Subject: Re: Criteria for "interoperability" for email-arch
Date: Mon, 21 May 2007 10:58:55 -0700
To: [email-masked]
Sender: [email-masked]

On May 15, 2007, at 12:22 PM, [email-masked] wrote:

>
> I went so far to recommend more or less this approach for the ABNF
> specification and even went so far as to generate a list of  
> references and uses
> of each of ABNF's "features". This was in effect rejected by the  
> IESG, however,
> in favor of documenting the existance of at least two ABNF validators.

That wasn't so much rejected as it just happened that I felt like  
fooling around with ABNF validators.  I learned a lot while doing it  
and once done, writing the interop report went quickly.  I admit this  
was probably not my highest priority thing to do at the time but it  
was fun.

>
> I thought this was a poor decision at the time and still do. IMO  
> what matters
> about ABNF - and by extension any other document that provides  
> definitional
> tools for other specifications - is whether or not it is actually  
> used by
> mulitplle specifications to achieve bettter specification clarity  
> and therefore
> increase the chance of creating interoperable implementations.  
> Whether or not
> there happens to be a way to backstop the use of such a document  
> with a
> validation tool strikes me as largely irrelevant.

Not really sure why you think it was a poor decision.  I also told  
other ADs my strong opinion that the real reason to move ABNF to  
Internet Standard was that we use it all the time and it works for us  
so "duh".

Lisa

Date: Mon, 21 May 2007 13:43:18 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: Consensus call:  draft-crocker-email-arch --  envelope vs. trace vs. ...?
Sender: [email-masked]

works for me

	Tony Hansen
	[email-masked]

Dave Crocker wrote:
> ... After a little bit of further though, I find myself still wishing
> for a single term to refer to all MHS-related information. In a
> blinding moment of finally seeing what now feels obvious, I want to
> suggest the blanket term:
> 
> 
>    Handling Information, comprising:
> 
>       Envelope:  Information consumed by the MHS.
>       Trace:     Information produced by the MHS.
> 
> 
> Does this work for folks?  (And by "work for" I am really looking to get
> two kinds of responses:
> 
>      1) Does anyone object?
> 
>      2) Does anyone find this helpful?

Date: Mon, 21 May 2007 09:09:15 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: Consensus call:  draft-crocker-email-arch --  envelope vs. trace vs. ...?
Sender: [email-masked]

Folks,

John C Klensin wrote:
>> "Trace Information" has two advantages:  1) it is an entirely
>> different term, and b) it accurately describes the core set of
>> information...
...
> It has a third advantage.  

The latest version of email-arch distinguishes envelope from trace information.

The former is information consumed by the  MHS, so it is the same as "SMTP 
envelope".  The latter is the information produced by the MHS, so it comprises 
such things as Received and Return-Path header fields.

After a little bit of further though, I find myself still wishing for a single 
term to refer to all MHS-related information.  In a blinding moment of finally 
seeing what now feels obvious, I want to suggest the blanket term:

    Handling Information, comprising:

       Envelope:  Information consumed by the MHS.
       Trace:     Information produced by the MHS.

Does this work for folks?  (And by "work for" I am really looking to get two 
kinds of responses:

      1) Does anyone object?

      2) Does anyone find this helpful?

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Sun, 20 May 2007 18:09:12 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: new email-arch version
Sender: [email-masked]

Folks,

In response to the recent round of comments, I've submitted a new version of 
email-arch as an I-D.

Various formats of -08 are available at <http://bbiw.net/recent.html#emailarch>.

Please note the changes section, at the end of the Introduction.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Sun, 20 May 2007 20:51:25 +0100
From: Alexey Melnikov <[email-masked]>
To: Tony Hansen <[email-masked]>
CC: SMTP Interest Group <[email-masked]>
Subject: Re: 2821bis open issues
Sender: [email-masked]

Tony Hansen wrote:

>The only issues still open in 2821bis-04 from before are 24 and 34.
>
>	34	Replace non-2606 ("example.com", etc) domains
>  
>
I agree with John that this is an editorial suggestion at this point.
I would recommend *against* doing this change and will support any appeal.

Date: Wed, 16 May 2007 11:29:42 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Criteria for "interoperability" for email-arch
Cc: [email-masked], Dave Crocker <[email-masked]>
Sender: [email-masked]

Dave Crocker writes:
> Let's count your note as a +1 for the suggested criterion, unless you 
> say otherwise.

+1 from me too, somewhat doubtfully. At least it's better than 
everything else I could think of.

Arnt

Date: Tue, 15 May 2007 16:26:50 -0700
From: Dave Crocker <[email-masked]>
Organization: Brandenburg InternetWorking
To: [email-masked]
CC: [email-masked]
Subject: Re: Criteria for "interoperability" for email-arch
Sender: [email-masked]

[email-masked] wrote:
> In the present case perhaps the inability to write a validation tool will
> make this approach viable. However, given the past history I have my doubts.

Time has passed...  Hope springs eternal...  Cliches abound...

For the moment, let's focus on what the group thinks is a meaningful and 
practical test.

Let's count your note as a +1 for the suggested criterion, unless you say 
otherwise.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

From: [email-masked]
Cc: Dave Crocker <[email-masked]>, [email-masked]
Date: Tue, 15 May 2007 12:22:35 -0700 (PDT)
Subject: Re: Criteria for "interoperability" for email-arch
To: John Leslie <[email-masked]>
Sender: [email-masked]

> Dave Crocker <[email-masked]> wrote:
> >
> > It thas  been suggested that a request to place email-arch on the standards
> > track should carry with it a specification of what constitutes
> > "interoperability" for the document.
> > ...
> >
> > Here's a simple candidate:
> >
> >      If a document has achieved a significant level of use as a normative
> > reference, it probably qualifies as being "interoperabille" by virtue of
> > being integrated into the technical community. Perhaps the best
> > non-protocol example of this is [RFC2119] Bradner, S., ?Key words for use
> > in RFCs to Indicate Requirement Levels?.

>    The obvious analogue would be "cited as a normative reference by at
> least two standards-track RFCs." This has the advantage of being easily
> measured.

I went so far to recommend more or less this approach for the ABNF
specification and even went so far as to generate a list of references and uses
of each of ABNF's "features". This was in effect rejected by the IESG, however,
in favor of documenting the existance of at least two ABNF validators.

I thought this was a poor decision at the time and still do. IMO what matters
about ABNF - and by extension any other document that provides definitional
tools for other specifications - is whether or not it is actually used by
mulitplle specifications to achieve bettter specification clarity and therefore
increase the chance of creating interoperable implementations. Whether or not
there happens to be a way to backstop the use of such a document with a
validation tool strikes me as largely irrelevant.

In the present case perhaps the inability to write a validation tool will
make this approach viable. However, given the past history I have my doubts.

				Ned

Date: Tue, 15 May 2007 11:01:37 -0400
From: John Leslie <[email-masked]>
To: Dave Crocker <[email-masked]>
Cc: [email-masked]
Subject: Re: Criteria for "interoperability" for email-arch
Sender: [email-masked]

Dave Crocker <[email-masked]> wrote:
> 
> It thas  been suggested that a request to place email-arch on the standards 
> track should carry with it a specification of what constitutes 
> "interoperability" for the document.
> ...
> 
> Here's a simple candidate:
> 
>      If a document has achieved a significant level of use as a normative 
> reference, it probably qualifies as being "interoperabille" by virtue of 
> being integrated into the technical community. Perhaps the best 
> non-protocol example of this is [RFC2119] Bradner, S., ?Key words for use 
> in RFCs to Indicate Requirement Levels?.

   The obvious analogue would be "cited as a normative reference by at
least two standards-track RFCs." This has the advantage of being easily
measured.

--
John Leslie <[email-masked]>

Date: Tue, 15 May 2007 07:05:07 -0700
From: Dave Crocker <[email-masked]>
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Criteria for "interoperability" for email-arch
Sender: [email-masked]

Folks,

It thas  been suggested that a request to place email-arch on the standards 
track should carry with it a specification of what constitutes 
"interoperability" for the document.

This anticipates eventually seeking movement from Proposed to Draft, which 
requires interoperability.  The challenge of deciding on criteria for 
interoperability, for standards track documents that do not specify a 
protocol, has come up before.  So I think it is quite a good idea to deal with 
it sooner rather than wait until seeking Draft status.

I suspect that another advantage to doing this exercise now is that it also 
provides a rather concrete basis for deciding whether the document is of a 
type that is appropriate for standards track.

I'd like to request discussion and suggestions from you, with the goal of 
reaching consensus.

Here's a simple candidate:

      If a document has achieved a significant level of use as a normative 
reference, it probably qualifies as being "interoperabille" by virtue of being 
integrated into the technical community. Perhaps the best non-protocol example 
of this is [RFC2119] Bradner, S., �Key words for use in RFCs to Indicate 
Requirement Levels�.

Comments?  Suggestions for other criteria?

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Tue, 15 May 2007 06:52:12 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: Consensus call:  draft-crocker-email-arch --  envelope vs. trace vs. ...?
Sender: [email-masked]

Folks,

It's turned into a cliche, to note the significance of John's and my agreeing. 
  However, what has turned that into a cliche in the last few years has been 
IETF process issues.

Our agreeing on some aspect of a technical *solution* remains rare enough to 
be non-existent.

In light of that, I'll note that anyone suggesting a term other than "trace 
information" is going to have to put forward a pretty strong argument...

d/

John C Klensin wrote:
>> I think that "Extended Envelope" and "Trace Information" are
>> both entirely reasonable.  On reflection, I'm inclined towards
>> the latter, but would like folks to comment.
>>
>> Here's my logic:  A term that differs from another core term
>> only by having an added qualify probably invites
>> misunderstanding, even if we say the "SMTP Envelope" is a
>> subset of "Extended Envelope".
>>
>> "Trace Information" has two advantages:  1) it is an entirely
>> different term, and b) it accurately describes the core set of
>> information... Or does it?  Is there anything that one would
>> include in this set, other than Received and Return-Path?
>> Does this additional information make the term "trace"
>> inappropriate?
> 
> It has a third advantage.  One of the things that I've found a
> little scary about this architectural description effort is that
> is introduces terminology and concepts that do not appear in the
> core email specifications.  For the reader, untangling those
> differences may lead to more, rather than less, confusion.
> While some terminology or conceptual differences may be
> unavoidable, I think they should be avoided where possible.  And
> "trace information" is exactly the term used to describe that
> material in 2821, whose prose even allows it to be extended to
> include additional information provided as part of the transport
> process.

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Tue, 15 May 2007 01:43:08 -0400
From: John C Klensin <[email-masked]>
To: Dave Crocker <[email-masked]>, "Vaudreuil, Greg M \\(Greg\\)" <[email-masked]>
cc: [email-masked]
Subject: Re: Consensus call:  draft-crocker-email-arch --  envelope vs. trace vs. ...?
Sender: [email-masked]

--On Monday, 14 May, 2007 22:22 -0700 Dave Crocker
<[email-masked]> wrote:

> 
> 
> Folks,
> 
> I'm coming to believe that any attempt to have the word
> "envelope" refer to more than the SMTP Envelope is doomed.
> Anyone feeling otherwise is strongly encouraged to speak up.

Concur.

> That leaves the choice of terms for this non-SMTP handling
> information.
> 
> I think that "Extended Envelope" and "Trace Information" are
> both entirely reasonable.  On reflection, I'm inclined towards
> the latter, but would like folks to comment.
> 
> Here's my logic:  A term that differs from another core term
> only by having an added qualify probably invites
> misunderstanding, even if we say the "SMTP Envelope" is a
> subset of "Extended Envelope".
> 
> "Trace Information" has two advantages:  1) it is an entirely
> different term, and b) it accurately describes the core set of
> information... Or does it?  Is there anything that one would
> include in this set, other than Received and Return-Path?
> Does this additional information make the term "trace"
> inappropriate?

It has a third advantage.  One of the things that I've found a
little scary about this architectural description effort is that
is introduces terminology and concepts that do not appear in the
core email specifications.  For the reader, untangling those
differences may lead to more, rather than less, confusion.
While some terminology or conceptual differences may be
unavoidable, I think they should be avoided where possible.  And
"trace information" is exactly the term used to describe that
material in 2821, whose prose even allows it to be extended to
include additional information provided as part of the transport
process.

> If anyone wants a term other than trace information, please
> speak up.

     john

Date: Mon, 14 May 2007 22:22:35 -0700
From: Dave Crocker <[email-masked]>
Organization: Brandenburg InternetWorking
To: "Vaudreuil, Greg M \(Greg\)" <[email-masked]>
CC: [email-masked]
Subject: Consensus call:  draft-crocker-email-arch --  envelope vs. trace vs. ...?
Sender: [email-masked]

Folks,

I'm coming to believe that any attempt to have the word "envelope" refer to 
more than the SMTP Envelope is doomed.  Anyone feeling otherwise is strongly 
encouraged to speak up.

That leaves the choice of terms for this non-SMTP handling information.

I think that "Extended Envelope" and "Trace Information" are both entirely 
reasonable.  On reflection, I'm inclined towards the latter, but would like 
folks to comment.

Here's my logic:  A term that differs from another core term only by having an 
added qualify probably invites misunderstanding, even if we say the "SMTP 
Envelope" is a subset of "Extended Envelope".

"Trace Information" has two advantages:  1) it is an entirely different term, 
and b) it accurately describes the core set of information... Or does it?  Is 
there anything that one would include in this set, other than Received and 
Return-Path?  Does this additional information make the term "trace" 
inappropriate?

If anyone wants a term other than trace information, please speak up.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Tue, 15 May 2007 00:08:15 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>, Frank Ellermann <[email-masked]>, Arnt Gulbrandsen <[email-masked]>
Subject: 2821bis open issues
Sender: [email-masked]

The only issues still open in 2821bis-04 from before are 24 and 34.

	34	Replace non-2606 ("example.com", etc) domains

We already have suggested textual changes for this issue, but John
didn't get a chance to work on 34 before -04 went out.

	24	Revise all ABNF to make a closed system starting with a
		definition for "command" or "command-verb"

Frank and Arnt, you've both indicated a desire to work on the ABNF.
Would you please work together off line and come back with a concrete
proposal? If possible, I'd like you to follow these guidelines:

   1)	Places where the text and the ABNF do not match are fair game.
   2)	Changing instances of "pseudo-ABNF" into true ABNF is fair game.
   3)	Adding terminals to make the ABNF parsable by the ABNF checkers
	is allowable.
   4)	Not all semantics need to be expressed in the syntax.
	For example, if the ABNF allows a string but the text constrains
	that string to certain sets of values, a change is not needed
	nor should be made to the ABNF.
   5)	It is not necessary to have a separate "collected grammar"
	section.

Thanks.

	Tony Hansen
	[email-masked]

Date: Mon, 14 May 2007 23:50:40 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-04
Sender: [email-masked]

rfc2821bis-04 is out. To see diffs from -03, I suggest going to
tools.ietf.org/html/draft-klensin-rfc2821bis and clicking on either
Diff1 or Diff2 at the top.

As John's issues list indicates, text has been added to deal with issues
2, 10, 23, 25, 32b and 35. Please review those areas. If there are no
problems, those issues will be considered closed.

John's issues list shows issue 5 (Sect 4.4. Syntax of ID in trace) as
still open. However, a change was put in for that one back in -02. There
was sufficient consensus to change it to "atom", but no one took up the
suggestion to change to "dot-string". Since there's been no further
discussion, I'll declare that one as closed as well.

I'll address the still open issues in another email message.

	Tony Hansen
	[email-masked]

John C Klensin wrote:
> Hi.
> 
> rfc2821bis-04 is in the posting queue.  I assume that it will
> show up in the directories sometime today or tomorrow and that
> Tony will post an issues status list and further instructions
> shortly thereafter. 
> 
> This is beginning to feel to me as if we are getting into the
> home stretch, aka the point of diminishing returns, but
> decisions about that are up to Tony.
> 
> I have just added Issue 36 to reflect the general topic of
> revising the IANA Considerations Section to include pointers to
> sundry registry-creation documents.  Note that, if we do this,
> we are likely to need downrefs to some existing documents (per
> draft-klensin-norm-ref-04, to be RFC 4897 sometime soon) and/or
> to move them to Draft along with this one and, if we need a
> reference to draft-hansen-4468upd-mailesc-registry, to move that
> along and into Last Call and the RFC Editor queue really soon
> now so it can _even_ be a downref.
> 
> Current issues list attached.
> 
>      john

Date: Mon, 14 May 2007 13:45:54 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: rfc2821bis-04
Sender: [email-masked]

--==========F42CBF6063BBC9423AC8==========
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi.

rfc2821bis-04 is in the posting queue.  I assume that it will
show up in the directories sometime today or tomorrow and that
Tony will post an issues status list and further instructions
shortly thereafter. 

This is beginning to feel to me as if we are getting into the
home stretch, aka the point of diminishing returns, but
decisions about that are up to Tony.

I have just added Issue 36 to reflect the general topic of
revising the IANA Considerations Section to include pointers to
sundry registry-creation documents.  Note that, if we do this,
we are likely to need downrefs to some existing documents (per
draft-klensin-norm-ref-04, to be RFC 4897 sometime soon) and/or
to move them to Draft along with this one and, if we need a
reference to draft-hansen-4468upd-mailesc-registry, to move that
along and into Last Call and the RFC Editor queue really soon
now so it can _even_ be a downref.

Current issues list attached.

     john

--==========F42CBF6063BBC9423AC8==========
Content-Type: application/pdf; name="RFC2821bis-status.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="RFC2821bis-status.pdf";
 size=13691

--==========F42CBF6063BBC9423AC8==========--

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: 2821bis + CONVERT
Date:  Mon, 14 May 2007 19:03:25 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 69
Sender: [email-masked]

John C Klensin wrote:
 
> Your diligence in finding these things and tracking them
> down continues to amaze me.

It was really completely unrelated to 2821bis, a discussion
with Jukka about UNKNOWN-8BIT and the W3C validator:
<http://permalink.gmane.org/gmane.org.w3c.validator/9194>

After I stumbled over it I thought that CONVERT might be 
relevant for 2821bis, it's also (remotely) related to some 
recent EAI discussions.
 
> (1) 1428 is, indeed, ancient history.

Good.  

> since 821 defined <opt-info> as precisely
>     [<via>] [<with>] [<id>] [<for>]
> terminated in a semicolon, it is not clear where additional
> clauses go.  Conversely, if additional clauses really are
> common practice and multiple addresses in <for> are not,
> then we should perhaps fix 2821bis so that Opt-info is
>     [Via] [With] [ID] [For] [Additional-registered-clauses]
> "For" is restricted to a single Mailbox (or Path), and
> Additional-registered-clauses is given a strict name-value
> syntax.

"IFF multiple addresses in <for> are not".  I can't judge it.
In theory I could try to check the mails I have, but that's
not much.  I'd bet that I've never got any CONVERT, but I'd
also bet that I've seen some rather odd timestamp lines.

> (3) If we are going to push through as document that defines
> and creates one or more additional IANA registries in this
> space (see  draft-hansen-4468upd-mailesc-registry), it would
> probably be wise to define as many of the SMTP-related
> registries as possible.

If that means that you'll add a normative reference to Tony's
I-D in -04, because you expect that it will be finished and
approved before 2821bis, it's fine.  For these obscure trace
clauses it would depend on how fast you expect an answer for
the "IFF"-question.  

You could also get away with a paragraph in the IANA section:

| The timestamp clauses FROM, BY, VIA, WITH, ID, and FOR as
| defined in this document form an initial registry of such
| clauses, adding CONVERT defined in RFC 1428 as deprecated.
|
| Additional clauses have to be specified in standards track
| RFCs, and IANA is asked to maintain a registry of such
| clauses with pointers to the RFCs specifying the details
| on the order of their publication.  The initial registry is:
|
| FROM     RFC xxxx
| BY       RFC xxxx  
| VIA      RFC xxxx
| WITH     RFC xxxx 
| ID       RFC xxxx
| FOR      RFC xxxx
| CONVERT  RFC 1428 (deprecated) 

If there are more "grandfathered" clauses this could deserve
its own I-D.  

Frank

Date: Mon, 14 May 2007 11:19:55 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: 2821bis + CONVERT
Sender: [email-masked]

--On Monday, 14 May, 2007 16:14 +0200 Frank Ellermann
<[email-masked]> wrote:

> 
> Hi,
> 
> at the moment there are 3+1 registries for 2821bis:
> 
>  1: ESMTP extensions
>  2: Domain literal tags, i.e. "IPv6:"
>  3: VIA links like "TCP", WITH protocols like "ESMTPA"
> +1: trace header fields in the header field registry
> 
> There's no registry for trace clauses.  FROM and BY
> are required.  VIA, WITH, ID, and FOR are optional.
> 
> Looking for the definition of UNNOWN-8BIT I stumbled
> over RFC 1428, it defines a trace clause CONVERT.
> 
> Can we ignore this as ancient history, or ignore it
> as out of scope for 2821bis, or does it need a IANA
> registry ?

Frank,

Your diligence in finding these things and tracking them down
continues to amaze me.   Three observations:

(1) 1428 is, indeed, ancient history.

(2) regardless of what 1428 says, the status of additional
clauses and keywords before the ";" in a Received header line is
a little dubious.  Even if we now permit only a single "For"
clause, it is not clear how these things are to be parsed:
effectively, the "for" clause terminates with that semicolon,
with any tokens appearing between "for" and the semicolon being
candidates for addresses.   One can use various heuristics to
get around that difficulty, or one can interpret <path> from 821
more narrowly than has traditionally been the case but, since
821 defined <opt-info> as precisely 
    [<via>] [<with>] [<id>] [<for>]
terminated in a semicolon, it is not clear where additional
clauses go.

Conversely, if additional clauses really are common practice and
multiple addresses in <for> are not, then we should perhaps fix
2821bis so that Opt-info is
    [Via] [With] [ID] [For] [Additional-registered-clauses]
"For" is restricted to a single Mailbox (or Path), and
Additional-registered-clauses is given a strict name-value
syntax.

Since whether that change is feasible or not depends on the
implementation status of multiple mailboxes in "For", I'll leave
it to Tony as to whether an issue number should be assigned

(3) If we are going to push through as document that defines and
creates one or more additional IANA registries in this space
(see  draft-hansen-4468upd-mailesc-registry), it would probably
be wise to define as many of the SMTP-related registries as
possible.

   best,
   john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  2821bis + CONVERT
Date:  Mon, 14 May 2007 16:14:07 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Hi,

at the moment there are 3+1 registries for 2821bis:

 1: ESMTP extensions
 2: Domain literal tags, i.e. "IPv6:"
 3: VIA links like "TCP", WITH protocols like "ESMTPA"
+1: trace header fields in the header field registry

There's no registry for trace clauses.  FROM and BY
are required.  VIA, WITH, ID, and FOR are optional.

Looking for the definition of UNNOWN-8BIT I stumbled
over RFC 1428, it defines a trace clause CONVERT.

Can we ignore this as ancient history, or ignore it
as out of scope for 2821bis, or does it need a IANA
registry ?

Frank

Date: Sun, 13 May 2007 22:27:17 +0100
From: Alexey Melnikov <[email-masked]>
To: Dave Crocker <[email-masked]>
CC: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Dave Crocker wrote:

>   CONSENSUS CALL --  Folks need to speak up about this:
>
>      1. Retain the current language about "envelope"
>
>      2. Modify the language, to restrict the term "envelope" to refer 
> only to the information contained in an SMTP exchange -- "consumed 
> by", to use Tony's language -- starting with a MailFrom and excluding 
> its corresponding DATA.

I would prefer 2. John Leslie's suggestion to use "extended envelope" 
would work for me.

Date: Sun, 13 May 2007 11:26:21 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Hi Dave,
At 14:59 06-05-2007, Dave Crocker wrote:
>I've sent in draft-crocker-email-arch-07 to the I-D folks.
>
>If you have any final comments, please get them to me.

In Section 3.3.1, it is mentioned that the message-id is associated 
with the RFC2822.From field, although any actor within the 
originating ADMD might assign it.

The message-id is an identifier which refers to a particular message 
(RFC 2822 Section 3.6.4).  It identifies the message as a whole and 
not the instantiation of the RFC2822.From field.

In Section 4.1.4:

  RFC2822.Sender
   Set by: Source
   This specifies the address responsible for submitting the message 
into the transfer
   service. For efficiency this field can be omitted if it contains 
the same address as
   RFC2822.From. However this does not mean there is no Sender

The RFC2822.Sender should not be used if it is the same as the 
RFC2822.From (RFC 2822 Section 3.6.2).

You could say:

For efficiency, this field is omitted if it contains the same address 
as RFC2822.From.

In Section 5.1:

Aliasing mixes local aliasing and forwarding.  I'll use the term 
"Address rewriting" and define it as a re-addressing facility that is 
performed before the message is delivered.  If the rewrite is a 
"Forward", the message does not have to be submitted back to the 
transfer service unless the forwarding is done by the MDA.  The 
message's RFC2821.MailFrom is not changed during the forwarding process.

The RFC2821.RcptTo value can be outside the scope of the ADMD doing 
the address rewriting.  There is no need to change the envelope 
information if the rewrite is local aliasing as it is more of final 
delivery decision as to which mail store should be used.

In Section 5.2:

Re-Sending does not create a new message.  According to RFC 2822, 
Section 3.6.6, a Resent is when a message is reintroduced in the 
transport system.

Re-Sending should not pass the RFC2822.BCC field set by the original 
Originator (RFC 2822, Section 5).

In Section 5.3:

   Mailing lists have explicit email addresses and they re-post 
messages to a list of
   subscribed members. The Mailing List Actor performs a task that 
can be viewed as an
   elaboration of the Re-Director role. In addition to sending the 
new message to a
   potentially large number of new Recipients, the Mediator can 
modify content, such as
   deleting attachments, formatting conversion, and adding 
list-specific comments. In
   addition, archiving list messages is common. Still the message 
retains characteristics
   of being "from" the original Originator.

could be changed to:

   Mailing lists have an explicit list of email addresses of 
subscribers to which
   they repost messages. The Mailing List Actor performs a task that 
can be viewed as an
   elaboration of the Re-Director role. In addition to sending the 
new message to a
   potentially large number of new Recipients, the Mediator can 
modify content, such as
   deleting attachments, format conversion, and adding list-specific 
comments. In
   addition, archiving list messages is common. Still the message 
retains most of the
   characteristics of being from the original Originator.

I added "most of the characteristics" as the RFC2822.Sender is 
obviously changed.

"Bounces" do not go to the RFC2821.MailFrom when a message passes 
through a mailing list.  There should be a RFC2822.Sender to which 
the bounce goes to.

In Section 5.5:

  Organizations often enforce security boundaries by subjecting 
messages to analysis, for
  conformance with the organization's safety policies.

As the next sentence talks about spam, it may be better to drop "safety".

   A Filter might alter the content, to render it safe, such as by 
removing content deemed
   unacceptable. Typically these actions will result in the addition 
of content that
   records the actions.

A Filter may reject a message or alter the content if the message is 
deemed unacceptable.

Regards,
-sm 

Date: Sat, 12 May 2007 10:30:09 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Eric Allman <[email-masked]>
CC: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Eric,

Thanks for the detailed comments.  After two years of iterating on detailed 
community review, it's interesting that new readers continue to raise new 
issues.  If anything, this underscores the need for a common architectural 
reference, to try to getter better community alignment.  It also suggests that 
the document could iterate forever...

Eric Allman wrote:
> I've get a few comments, with one big meta-comment.  I really don't 
> think this is a Standards Track document.  The language is too loose to 
> be ST, and it doesn't really prescribe very much (there are only six 
> MUSTs and one MAY in the whole thing).  It's primary benefit seems to be 
> to define language and give an intro to how the pieces fit together.  If
> the document is to be Informational then I feel much much better about it.

1.  Most/all IETF architecture documents are standards track (or BCP).

2.  Since the entire purpose of this exercise has been to provide a common
referential base, it doesn't do much good to have it issued without the 
explicit stamp of formal consensus approval.  For Informational, the document 
could merely have been sent in to the RFC Editor directly, and neither I nor 
anyone else would have had to deal with 2 years of iterative review to make it 
work towards a consensus view.  The community purpose of the document is 
served better if the document is assessed as representing a community view. 
That's what standards track (and BCP) accomplish.

3.  An artifact of documenting what already is -- especially when what is is 
rather messy -- is that it says "is" and "does" and the like, rather than 
claiming to be directive.  The reality of Internet Mail is that it is filled 
with variable behavior; so it's not clear that musts and must nots are all 
that appropriate, if the document is to be aligned with reality.

    That said, if folks believe that the benefits of having an architecture 
document for email require that it be laced with the vocabulary of musts and 
must nots, they should speak up.  Massaging the text into that form would be a 
hassle, and more importantly, it well could be counter-productive, but as 
always, consensus rules.

     Perhaps you can clarify why you see this linguistic point as being important?

4.  As for 'loose', I have previously heard concerns for the document's having
too much detail, not too little.  So I'm not sure how to interpret your
assessment.

5.  I think you and I see the potential utility of the document rather 
differently. Given the document's history, as well as its exploration of 
concepts such as actors, trust, responsibility and interactions, I'd class it 
as more than simply an intro to how things fit together. My own view is that 
it can be quite helpful for future work, not merely for explaining past work. 
  Perhaps you could clarify what functional aspects of the document are 
needed, to satisfy your own criteria?

> In section 3.3.1 you say that Message-Id "is associated with the 
> RFC2822.From field."  In fact that is not a requirement.  The Message-Id 
> has to be unique but there is no requirement that the RHS be the same as 
> the RHS of From, nor even any requirement that it be an actual host or 
> domain name.  In hosted mail in particular it may quite commonly be 
> different.

It appears that you believe that "associated with" carries a requirement for
having the same domain name.  That certainly was not intended (or it would 
have said that.)  At the least, I think folks are pretty clear that there is 
no such requirement for the RFC2821.MailFrom address, even though it is
"associated with" the RFC2821.Sender actor.  So I don't see whya domain name 
relationship should apply for other "associated" relationships.

RFC2822 says:
> The "Message-ID:" field provides a unique message identifier that
>    refers to a particular version of a particular message.  The
>    uniqueness of the message identifier is guaranteed by the host that
>    generates it (see below)

My reading of this takes "version" to refer to a particular version of the 
content.  The only identities in a message that can be characterized as 
plausibly responsible for the content are rfc2822.From and rfc2822.Sender.

As a rule, we hold the From field identity as responsible for content, with 
the Sender field as responsible for MHS interaction.  That other agents can 
take action *of behalf of* the From does not make the content less
"associated with" that origination identity.

Confusion about this distinction between having primary responsibility, versus 
being a delegated actor, continues to plague email discussions.  It was, in 
fact, one of the motivations for the document, and one of the reasons the 
document tries to note the links between fields.

All of this, by extension, makes the Message-ID value "associated with" the 
 From value.

As with many Internet functions that facilitate human interactions, things
cannot be nearly as clear and precise as any of us would like.  That is why
section 3.3.1  on Message-ID rambles on at some length about the fuzziness.

> You also say that more than one Message-Id is sometimes assigned.  It 
> may be that some broken messages have more than one, but 2822 allows 
> only zero or one Message-Id field.

So it's probably good that the email-arch document that comments on the
"sometimes" begins with "Internet Mail standards provide for a single Message-ID"?

This is another example of the document's trying to be clear about the 
difference between formal and de facto behaviors.  Would the document be 
better if it ignored the latter?

> Some cases in your list of guidelines and examples that aren't covered 
> are forwarding (new message-id), resending (same, but with a 
> Resent-Message-Id), and digests (new).

I suspect there are quite a few cases not covered.  That's why the text says
"some".

If you feel it important to have these added, please explain the need.  Again
note that the pressure has been to find ways to make the document shorter, not
longer.

> Figure 5 includes a whole bunch of acronyms that aren't defined until 
> later, such as oMS, hMSA, rMDA, etc.  I found I really couldn't 
> understand the figure until several pages later (explanations start in 
> section 4.2.1).

You seem to be saying that a picture of the pieces for a system should not be
presented to the reader until all of the pieces have first been described?

This could be an example of the basic tension between bottom-up versus
top-down pedagogy.

I prefer to introduce the big picture, so that people can see where each piece
fits, as it is discussed.  The danger with bottom-up pedagogy is that nothing
hangs together until the end.  The reader has to juggle all those bits,
without a framework, until the denouement, at that end.

> Table 1, SMTP RcptTo, you say it's sent by the Originator.  It can also 
> be set by a Mediator.

Well, since a Mediator is a Recipient and an Originator, it can be argued that 
reference to it should appear anywhere either one of those terms appears. 
Somehow I do not think that would add clarity to the document.

So I chose to have the Table cite Mediator only for the identity fields that
are inherent to the Mediator role.  By contast, Section 5 is focused on the 
Mediator role, so its discussion of fields cites "Mediator Originator" 
explicitly, for RcptTo.

> In section 5.1 you say that aliasing SHOULD replace RFC2821.MailFrom. I 
> think a lot of people would disagree with this, at least for some 
> cases.  For example, forwarding services such as pobox.com or acm.com or 
> alumni.*.edu simply don't have mailboxes there to be responsible. You're 
> also assuming that aliasing happens in the MDA "just before placing a 
> message into the specified Recipient's mailbox."  That's obviously not 
> going to be true for forwarding services.

The two paragraphs that discuss Aliasing attempted to deal with exactly the
kinds of concerns you raise.  It well might be that the language simply does
not do the job intended for it.  Since 'mailbox' is already cited as not
necessarily being storage, the language "placed into" might be ill-advised.

But frankly, I was trying to avoid using the word "deliver" because that is a
formally defined construct, for a transfer of responsibility. Technically the 
message is already delivered when aliasing is performed.  The text notes that 
it might not seem that way, given how the function is usually implemented, and 
the text then attempts to defend its view:  The emssage has reached its 
designated RcptTO address. The new address is provided under the control of 
that original recipient. So it seemed simpler to avoid using the word 
"deliver" entirely, for this particular discussion.

Another possibility is that this represents a confusion between implementation 
and architecture.  Your citing forwarding services suggests this rather
strongly, since they are a particular operational implementation, rather than 
a distinctive architecture. That is, forwarding services are set up for the 
purpose of re-directing a message, to a different mailbox than the one 
specified by the Originator. This is an implementation choice, but does not -- 
or at least should not -- alter the architecture-based nature of what tasks 
are being done and where the responsibilities lie.

A forwarding service sends a message with a brand-new RCPT-TO.  This is
strictly under the control of the Recipient, rather than the Originator, or 
any of the MHS components. That's a rather basic distinction, in terms of 
where to assign responsibility.  And the forwarding service is provided merely 
as an agent of the Recipient, since it is the Recipient who specifies what 
final address to use.

I see it is important -- in fact, inherent in the nature of a legitimate 
architecture -- that a single Internet Mail architecture cover a variety of 
significantly different operational implementations, including forwarding.  As 
much as the document tries to keep the distinction between implementation and 
architecture clear, confusion about it continues to plague discussions.  I'm 
not sure how to deal with that any better; suggestions are more than welcome.

The document's discussion of the RFC2821.MailFrom address, when there is
aliasing, notes the choices between retaining the original one, versus having 
the Mediator assign a new one, and explains why one is better than the other,
albeit acknowledging that both occur in the wild.

It occurs to me that this is another good example of the potentially
counter-productive effect that would occur for an Internet Mail architecture
document to have too many MUSTs and SHOULDs.  It would be pretending that we
can get reasonably strict conformance, when we have several decades of
experience clearly demonstrating that we can not.

> If you're interested I did a graphic version of Figure 5 (I pretty much 
> had to in order to understand it).  I'll forward that to you if you're 
> interested.

I had decided to wait on making the pretty graphics until the document was
published, for two reasons.  One is that things kept changing and I'd rather
work only on a single copy.  The more important reason is that the text copy 
of an RFC is definitive, so that the ascii art has to be viable.  Having a 
pretty graphics version will tend to bias people towards consulting it, rather 
than the ascii art version, and thereby make it likely that we will miss 
problems with the ascii art.

Hmmm.  Before I began that paragraph, I was going to say "sure, send it over"
but I think I just talked myself into wanting to wait until the document has
passed IETF Last Call...

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Final draft:  draft-crocker-email-arch
Date:  Sat, 12 May 2007 05:03:27 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 10
Sender: [email-masked]

Hector Santos wrote:
 
> The case in point would be a mailing list; Bounces will typically
> go to the list admin.

IMO the I-D is already clear on this and related "mediator" issues
for some rounds, IIRC it was always clear for the mailing list case.  

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Final draft:  draft-crocker-email-arch
Date:  Sat, 12 May 2007 04:52:34 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 130
Sender: [email-masked]

Dave Crocker wrote:
 
> If you have any final comments, please get them to me.

| The body may be unstructured simple lines of text, or it may
| be a MIME tree of multi-media subordinate objects, called 
| body-parts, or attachments.  [RFC2045], [RFC2046], [RFC2047],
| [RFC4288], [RFC4289], [RFC2049].

s/or attachments/including attachments/

An "attachment" is a Content-Disposition for MIME parts, MIME
parts can be also inline.  With my MUA it's a major difference.

| It can be generated by an rMUA and is sent to the 
| Disposition-Notification-To address(es).  The mailbox for 
| this is shown as Disp in Figure 5.  It

? It
+ It is typically the same address as the Envelope-Sender address.

| Upon delivery, some SMTP-level envelope information is typically
| encoded within additional message object header fields, such as
| Return-Path.  [RFC2821],[RFC2822]

The Return-Path is not optional, it's required for 3834 and SIEVE.

s/such as Return-Path./especially the Return-Path./

| Procedures for registering header fields are defined in [RFC4021].
| An extensive set of existing header field registrations is
| provided in [RFC3864].

Swap 3864 (procedure) and 4021 (initial registry).

       | Message header fields | From        | Originator          |
       |                       | Sender      | Source              |
[...]
       |                       | Message-ID  | Source              |

+1  That's probably Peter's observation about Message-IDs in 3.3.1,
    it should say "associated with the 2822.Sender", not 2822.From,
    corresponding to the figure in 4.1.4.

| Specification of the error return addresses -- the "Bounce"
| address, contained in RFC2821.MailFrom -- is made by the
| RFC2822.Sender.  Typically the Bounce address is the same as the
| Sender address.  However some usage scenarios require it to be
| different.

The term "envelope sender address" (after a definition) would be
much better than all those "bounce" and "error return addresses".

It's not only used for NDNs, it's for (almost) all auto-replies.

| The MSA can specify its hosting domain identity for the SMTP HELO
| or EHLO command operation.

s/MSA/MTA/ or something in this direction, not only the MSA.  The
I-D says "set by source", but I think it should use a similar "set
by" as RFC2821.Received (source, relay, mediator, dest).  Probably
minus "dest".

| RFC2821.RcptTo
|
|     Set by: Originator

Is that "set" as in "initially set" ?  And why isn't it set by the
"source" as for the 2821.MailFrom ?  It's also set by a "mediator".

| This indicates trace information, including originating host,
| relays, Mediators, and MSA host domain names and/or IP Addresses.

s/MSA/MTA/

| RFC2821.Return-Path
|
|     Set by: Source

s/Source/Dest/  (reflecting the last Mail From of course, maybe
                 it depends on the precise definition of "set".)

| A recent alternative is SUBMISSION [RFC4409].  Although SUBMISSION
| derives from SMTP, it uses a separate TCP port and imposes distinct
| requirements, such as access authorization.

Please strike "recent" and "Although".

|      RFC2822.Received
|
|         Set by: Relay Server
|
| 4.3.3.  Mail Delivery Agent (MDA)

Is the "Set by:" blurb here a dupe of 4.1.4 (?)

In 4.3.3 there are two similar "set by:" blurbs, I think you want 
this only in 4.1.4 (?)  

Ditto in 5, but there it makes sense to show the differences from
4.1.4.    

In the "resent" section you have:

|        RFC2822.Received
|
|        Set by: Mediator Dest
|
|        When resending a message the submission agent can record a
|        Received header field, to indicate the transition from
|        original posting to resubmission.

What's "Mediator Dest", the MDA before the resending (last Received
below the Resent-*) or the MSA used for resending (first Received
above the Resent-*) ?  The text sounds like the latter, but that's
a "Mediator Source".  Something's messy with those "set by" blurbs.

In 4.1.4 it's "RFC2821.Received", all later "RFC2822.Received" are
dubious, ITYM "RFC2821.Received", not 2822.

You could mention Resent-Message-ID in 5.2.  For the 2821.RcptTo in
5.* I'd say "mediator source" instead of "mediator originator" (as
mentioned above for 4.1.4), same issue in 5.1 + 5.2 + 5.3

Giving up at 5.4 for today and -07, this "set by" stuff is not yet
consistent everywhere.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Message-ID (was: Final draft:  draft-crocker-email-arch)
Date:  Sat, 12 May 2007 02:51:25 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 30
Sender: [email-masked]

Dave Crocker wrote:
 
>> * 3.3.1 Message-ID:

>>   "it is associated with the RFC2822.From field, ..." How so? I don't
>>   see any conceptual association between the From and the Message-ID
>>   field: While some MUAs may use (parts of) the From field in generating
>>   the Message-ID, others ignore it.
 
> The discussion about Message-ID attempts to characterize it as distinguishing
> among different messages.  The difference among messages is primarily due to
> choices made by the authors.  That's the From field.
 
> Or, at least, that's the theory behind the statement...

...but maybe incorrect.  If I'm secy@ and get a message from boss@
to be sent out it's IMO my job as the sender to guarantee that its
Message-ID is "worldwide unique forever", not the job of the author:

|      Date: 26 August 1976 1430-EDT
|      From:George Jones<Group at Host>
|      Sender:Secy at SHOST
|      To:Al Neuman at Mad-Host,
|               Sam Irving at Other-Host
|      Message-ID:  <some string at SHOST>

In this RFC 733 example both Sender and Message-ID use "at SHOST".

Frank

Date: Thu, 10 May 2007 11:02:45 +0100
From: Alexey Melnikov <[email-masked]>
To: Tony Finch <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Tony Finch wrote:

>The envelope is for consumption by transport-level agents, and is
>discarded on final delivery. (Therefore IMAP and POP don't provide access
>to it because it isn't there.
>
Interestingly enough, there are extensions for accessing envelope FROM 
from IMAP, as well as checking if the received message was authenticated 
using SMTP AUTH.

>) This means that user agents can't see the
>envelope. Any feature that depends on envelope information to work (e.g.
>DSNs) must therefore be implemented by a transport agent. Any feature that
>must be implemented by a user agent (e.g. MDNs) must be signalled in the
>header not the envelope.
>  
>

Date: Thu, 10 May 2007 11:12:28 +0100
From: Alexey Melnikov <[email-masked]>
To: Tony Finch <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 19: Explanatory text after literal in EHLO
Sender: [email-masked]

Tony Finch wrote:

>On Thu, 3 May 2007, Tony Hansen wrote:
>  
>
>>I'm going to close the issue, but note, as Frank suggests, that it
>>should be checked as part of the implementation and interoperability report.
>>    
>>
>
>Based on my testing, common implmentations (Sendmail, Exim, Postfix,
>Exchange) do not accept this syntax.
>
I've just checked Isode M-Switch and it doesn't accept this syntax either.

> (Some of them can be configured to
>accept any kind of junk after EHLO but if their syntax checker is turned
>on they reject explained-literals.) My servers have seen no (zero)
>instances of this syntax by clients in the last month (out of two to three
>million messages per day). (We don't keep logs longer than that.)
>
>As far as I can tell from the DRUMS archive this wording was added to
>draft-ietf-drums-smtpupd-04 without any pertinent discussion - it seemed
>to be a reaction to an argument about canonical names.
>
>I think this all indicates that the feature should be removed.
>  
>
Agreed.

Date: Thu, 10 May 2007 07:59:52 +0100
From: Tony Finch <[email-masked]>
To: Matti Aarnio <[email-masked]>
cc: [email-masked], Tony Hansen <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 19: Explanatory text after literal in EHLO
Sender: [email-masked]

On Thu, 10 May 2007, Matti Aarnio wrote:
>
> A lot where the parameter is 1.2.3.4 -type badly formatted address
> literal (should have been [1.2.3.4]) but that wasn't the interesting
> question..

These are all spam bots.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
LUNDY FASTNET IRISH SEA: SOUTH VEERING SOUTHWEST 3 OR 4, INCREASING 6 TO GALE
8, PERHAPS SEVERE GALE 9 LATER. MODERATE OR ROUGH, OCCASIONALLY VERY ROUGH.
RAIN OR SHOWERS. MODERATE OR GOOD, OCCASIONALLY POOR AT FIRST.

Date: 	Thu, 10 May 2007 03:01:52 +0300
From: Matti Aarnio <[email-masked]>
To: [email-masked]
Cc: Tony Hansen <[email-masked]>
Subject: Re: rfc2821bis-01 Issue 19: Explanatory text after literal in EHLO
Sender: [email-masked]

On Wed, May 09, 2007 at 02:50:06PM -0400, Tony Hansen wrote:
> <developer hat on>
> Our implementation will accept this syntax but does not generate it.
> 
> Does *anyone* generate it?
> </developer hat off>

I recall having seen it generated by some network cameras that send
images out in email.  They have embedded software so most likely it
won't be updated during gadget's lifetime...

Checking my current inbound SMTP logs covering several months I do see
couple dozen cases where the HELO/EHLO parameter has more than one
one token in it.  A lot where the parameter is  1.2.3.4 -type badly
formatted address literal (should have been [1.2.3.4]) but that
wasn't the interesting question..

Most of those "multi-token" cases are of style:

    HELO john smiths computer.isp.domain.here

So if anything, I would emphasize that in real world the HELO/EHLO parameter
existence or value has no real meaning, it should be sent as specified but
failure to comply with strict syntax is not to be considered an error.

I did notice this week that sendmail 8.13.8 wants to have properly formatted
HELO-parameter...  which broke email sending from one application when it had
broken configuration.  At least that config setup is practically useless in
generic ISP job, but works internally among UNIX servers..

> 	Tony Hansen
> 	[email-masked]

-- 
/Matti Aarnio	<[email-masked]>

Date: Wed, 09 May 2007 18:21:21 -0400
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 19: Explanatory text after literal in EHLO
Sender: [email-masked]

--On Wednesday, 09 May, 2007 14:50 -0400 Tony Hansen
<[email-masked]> wrote:

> <developer hat on>
> Our implementation will accept this syntax but does not
> generate it.
> 
> Does *anyone* generate it?
> </developer hat off>

<editor hat off>
<sometime-developer hat on>
I had a gateway-to-the-Internet implementation that could, and
did, generate it some years back, even before 2821 officially
made it valid.  A few receiver-SMTPs balked; most, apparently
(we asked a few) interpreting the "don't reject on this basis"
text in 1123 even more liberally than 2821 did, just accepted it
and ignored it.  

As far as I know, that implementation is no longer in use.
Perhaps more important, unless the added information is copied
into trace fields, it is probably fairly useless.  And I'm not
aware of any implementation, ever, that copied the added
information into trace fields.   

So while I'm a little sad to see this go, and perhaps more sad
to find it not in use, I can't make an argument for retaining it.
</developer>
</editor>

     john

Date: Wed, 09 May 2007 14:50:06 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 19: Explanatory text after literal in EHLO
Sender: [email-masked]

<developer hat on>
Our implementation will accept this syntax but does not generate it.

Does *anyone* generate it?
</developer hat off>

	Tony Hansen
	[email-masked]

John Leslie wrote:
> [email-masked] <[email-masked]> wrote:
>>> On Thu, 3 May 2007, Tony Hansen wrote:
>>>> I'm going to close the issue, but note, as Frank suggests, that it
>>>> should be checked as part of the implementation and interoperability
>>>> report.
> 
>    We've got to drop this if we can't find interoperable implementations.
> The question is: should we even try to?
> 
>>> Based on my testing, common implmentations (Sendmail, Exim, Postfix,
>>> Exchange) do not accept this syntax. (Some of them can be configured to
>>> accept any kind of junk after EHLO but if their syntax checker is turned
>>> on they reject explained-literals.) My servers have seen no (zero)
>>> instances of this syntax by clients in the last month (out of two to
>>> three million messages per day). (We don't keep logs longer than that.)
> 
>    I don't have that much traffic, but I can't find a use of it in our
> logs either.
> 
>>> As far as I can tell from the DRUMS archive this wording was added to
>>> draft-ietf-drums-smtpupd-04 without any pertinent discussion - it
>>> seemed to be a reaction to an argument about canonical names.
>>> I think this all indicates that the feature should be removed.
>> I have to agree with Tony [Finch] here. I like the idea behind this
>> and as it happens our implementation can accept this syntax.
> 
>    We might include text allowing a receiving SMTP server the option to
> parse the EHLO string and treat only the part before SP as the domain
> or address-literal. (I recall discussing this exact issue with Dave
> Crocker during the design of CSV -- we decided to follow the BNF...)
> 
>> However, the checks many people have in place will prevent it from
>> working on a lot of actual systems out there. I can virtually guarantee
>> a ton of interop problems if this is adopted.
> 
>    The evidence is clear that its use is not mainstream. Our analysis
> shows it to be unusable except in special cases which amount to prior
> arrangement between client and server. It looks to me to be pretty
> close to consensus to drop it from the text (and BNF).
> 
>    I recommend dropping it: adding it to the BNF would clearly violate
> the principle of least amazement.
> 
> --
> John Leslie <[email-masked]>
> 

Date: Wed, 9 May 2007 13:23:11 -0400
From: John Leslie <[email-masked]>
To: Dave Crocker <[email-masked]>
Cc: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

   I _almost_ promised Dave I'd stop beating this horse, but he's asking
for consensus. Bottom line: I can live with his current text, but it
worries me. (If that's all you want to know, you can skip the rest of
this message.)

Dave Crocker <[email-masked]> wrote:
> [Tony Finch <[email-masked]> wrote:]
> 
>> The STANDARD (RFC 822, RFC 1123) definition is just the MAIL and RCPT
>> commands and excludes all message data. The only other meaning of
>> envelope in Internet Mail specifications is the IMAP envelope which is
>> just the basic header fields that MUAs usually display, and has nothing
>> to do with transport or tracing. i.e. the definition in
>> draft-crocker-email-arch is an entirely new invention.
> 
> The distinction is something I've asserted for a long time. So, if I 
> invented it, it was a long time ago:
> 
> RFC822:
]> 4.7.3.  ENCRYPTED
]...
]>
]>     Note:  Unfortunately, headers must contain envelope,  as  well
]>            as  contents,  information.  Consequently, it is neces-
]>            sary that they remain unencrypted, so that  mail  tran-
]>            sport   services   may   access   them.   Since  names,
]>            addresses, and "Subject"  field  contents  may  contain

   Note, this says "headers" "contain" "envelope information" (which does
not contradict the RFC2821 usage of "envelope").

> My recollection is that the distinction was more commonly held, back
> then. (And I'm not citing the above 822 text as resolving what the
> current draft should say, merely trying to highlight that the issue is
> long-standing.)
> 
> I think that restricting 'envelope' to refer only to SMTP-level
> information perpetuates a failure in Internet Mail, to characterize
> MHS-related information that is in the RFC2822 header as not being
> part of the envelope -- it loses the distinction between MUA-MUA
> information and MUA-MHS (and intra-MHS) information.

   RFC2821 is clear that it uses "envelope" to refer to information in
SMTP commands. I entirely agree that that is not a sufficient definition
for an architectural overview. I merely would prefer using "extended
envelope" or some other term (different from "envelope) to refer to the
"envelope information" concept in an architectural overview.

>> The envelope is for consumption by transport-level agents, and is
> 
> You used the language "for consumption by transport-level agents".
> The language the draft uses is "used by the MHS, or generated by it."
> 
> Obviously what the current draft is trying to do is to categorize 
> information that is generated by the MHS as being part of the envelope, 
> rather than part of the message content.

   IMHO, our problem stems from trying to separate "envelope" from
"content" in the architectural overview. This would make sense if we
were talking about stuff which is _always_ removed before delivery;
but in practice, are talking about stuff which is only _sometimes_
removed before delivery. And, to make matters worse, it's information
_very_ useful for spam identification (so the trend would seem to be
to remove less of it).

   The problem I see is that vagueness in a specification almost always
comes back to haunt us. (Vagueness in an architectural overview, OTOH,
very seldom comes back to haunt us.)

   I'd much prefer to see clarity about what's included in "envelope"
and vagueness about what's included in "content" in a document which
is likely to sometime be cited as a _normative_ reference. "Content"
is delivered between parties with an ongoing relationship; "envelope"
information is exchanged between parties with no prior arrangement.

> That is, the draft is trying to make "message content" refer only to 
> MUA-MUA information.

   As I said, this is where I'd rather see the vagueness.

>> discarded on final delivery. (Therefore IMAP and POP don't provide
>> access to it because it isn't there.) 
> 
> Return-path?

   As I said, this is where I'd rather see vagueness.

>> This means that user agents can't see the envelope. Any feature that
>> depends on envelope information to work (e.g. DSNs) must therefore
>> be implemented by a transport agent.

   Surprise! I see MUAs (trying to) generate NDNs all the time. :^(

>> Any feature that must be implemented by a user agent (e.g. MDNs)
>> must be signalled in the header not the envelope.

   (Tony is sticking with the RFC2821 usage; Dave isn't...)

> Failure to provide an MUA with envelope information -- notably the
> RCPT-TO that produced delivery to it -- sometimes causes problems.

   (Note how much it helps, even just using "envelope information" to
clarify the difference...)

> Interestingly, some MUAs do take note of that (approximate)
> information, by implication:  When doing a POP or IMAP pickup, the[y]
> note the account through which the message was retrieved, and they
> default to having a reply use the email address associated with that
> account.

   Vagueness about "content" is your friend!

>> There used to be a fairly firm architectural principle that MTAs
>> should not have to examine the message data in order to perform
>> their function in the usual case. 

   (Note, they _do_ have to examine trace headers to detect forwarding
loops. Of course, such loops are not "the usual case"...)

> The current draft does not change that.  (Hmmm.  Probably worth
> having the draft include some text to that effect, since it really
> is an architectural point.)

   ++

> But my perspective on all this could merely be an academic point.

   Academic points are our friend!

> The purpose of the current document is to record the current model,
> and that requires current consensus.
> 
> So:
> 
>   CONSENSUS CALL --  Folks need to speak up about this:
> 
>   1. Retain the current language about "envelope"

   There's room for improvement.

>   2. Modify the language, to restrict the term "envelope" to refer
>      only to the information contained in an SMTP exchange --
>      "consumed by", to use Tony's language -- starting with a MailFrom
>      and excluding its corresponding DATA.

   My preference, provided that we retain the "envelope information"
concept -- under a name different from "envelope" to refer to at least
RFC2821 "envelope" plus trace headers.

-- 
John Leslie <[email-masked]>

Date: Wed, 09 May 2007 10:20:56 -0700
From: Dave Crocker <[email-masked]>
Organization: Brandenburg InternetWorking
To: "Vaudreuil, Greg M \(Greg\)" <[email-masked]>
CC: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Vaudreuil, Greg M (Greg) wrote:
> The biggest "wart" remaining is the lack of a loop-detection algorithm that
> does not require peeking into the headers to review previous trace info.
> That's a wart that seems too expensive to fix.  Clearly an SMTP extension
> to add the accumulated received lines to the MAIL FROM would make things
> easier if only it could be incrementally deployed.  (I don't think it can
> since it requires widespread adoption before it can be useful)

Hmmm.  Loop detect might serve as a good example of why information that is 
created by the MHS, and placed into the rfc2822 header, qualifies as "envelope"...

Had we done this part of email correctly, there would be two headers.  One for 
MUA-MUA and the other involving the MHS and MUA.  But we didn't.  Hassles 
resulting from this crop up regularly.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Wed, 09 May 2007 09:04:59 -0700
From: Dave Crocker <[email-masked]>
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Folks,

> The STANDARD (RFC 822, RFC 1123) definition is just the MAIL and RCPT
> commands and excludes all message data. The only other meaning of
> envelope in Internet Mail specifications is the IMAP envelope which is
> just the basic header fields that MUAs usually display, and has nothing to
> do with transport or tracing. i.e. the definition in
> draft-crocker-email-arch is an entirely new invention.

The distinction is something I've asserted for a long time. So, if I invented 
it, it was a long time ago:

RFC822:
> 4.7.3.  ENCRYPTED
...
> 
>         Note:  Unfortunately, headers must contain envelope,  as  well
>                as  contents,  information.  Consequently, it is neces-
>                sary that they remain unencrypted, so that  mail  tran-
>                sport   services   may   access   them.   Since  names,
>                addresses, and "Subject"  field  contents  may  contain

My recollection is that the distinction was more commonly held, back then. 
(And I'm not citing the above 822 text as resolving what the current draft 
should say, merely trying to highlight that the issue is long-standing.)

I think that restricting 'envelope' to refer only to SMTP-level information 
perpetuates a failure in Internet Mail, to characterize MHS-related 
information that is in the RFC2822 header as not being part of the envelope -- 
it loses the distinction between MUA-MUA information and MUA-MHS (and 
intra-MHS) information.

> The envelope is for consumption by transport-level agents, and is

You used the language "for consumption by transport-level agents".

The language the draft uses is "used by the MHS, or generated by it."

Obviously what the current draft is trying to do is to categorize information 
that is generated by the MHS as being part of the envelope, rather than part 
of the message content.

That is, the draft is trying to make "message content" refer only to MUA-MUA 
information.

> discarded on final delivery. (Therefore IMAP and POP don't provide access
> to it because it isn't there.) 

Return-path?

>     This means that user agents can't see the
> envelope. Any feature that depends on envelope information to work (e.g.
> DSNs) must therefore be implemented by a transport agent. Any feature that
> must be implemented by a user agent (e.g. MDNs) must be signalled in the
> header not the envelope.

Failure to provide an MUA with envelope information -- notably the RCPT-TO 
that produced delivery to it -- sometimes causes problems.

Interestingly, some MUAs do take note of that (approximate) information, by 
implication:  When doing a POP or IMAP pickup, the note the account through 
which the message was retrieved, and they default to having a reply use the 
email address associated with that account.

> There used to be a fairly firm architectural principle that MTAs should
> not have to examine the message data in order to perform their function in
> the usual case. 

The current draft does not change that.  (Hmmm.  Probably worth having the 
draft include some text to that effect, since it really is an architectural 
point.)

But my perspective on all this could merely be an academic point.

The purpose of the current document is to record the current model, and that 
requires current consensus.

So:

   CONSENSUS CALL --  Folks need to speak up about this:

      1. Retain the current language about "envelope"

      2. Modify the language, to restrict the term "envelope" to refer only to 
the information contained in an SMTP exchange -- "consumed by", to use Tony's 
language -- starting with a MailFrom and excluding its corresponding DATA.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Wed, 9 May 2007 11:53:21 -0400
From: John Leslie <[email-masked]>
To: [email-masked]
Cc: Tony Finch <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-01 Issue 19: Explanatory text after literal in EHLO
Sender: [email-masked]

[email-masked] <[email-masked]> wrote:
>> On Thu, 3 May 2007, Tony Hansen wrote:
>>>
>>> I'm going to close the issue, but note, as Frank suggests, that it
>>> should be checked as part of the implementation and interoperability
>>> report.

   We've got to drop this if we can't find interoperable implementations.
The question is: should we even try to?

>> Based on my testing, common implmentations (Sendmail, Exim, Postfix,
>> Exchange) do not accept this syntax. (Some of them can be configured to
>> accept any kind of junk after EHLO but if their syntax checker is turned
>> on they reject explained-literals.) My servers have seen no (zero)
>> instances of this syntax by clients in the last month (out of two to
>> three million messages per day). (We don't keep logs longer than that.)

   I don't have that much traffic, but I can't find a use of it in our
logs either.

>> As far as I can tell from the DRUMS archive this wording was added to
>> draft-ietf-drums-smtpupd-04 without any pertinent discussion - it
>> seemed to be a reaction to an argument about canonical names.
> 
>> I think this all indicates that the feature should be removed.
> 
> I have to agree with Tony [Finch] here. I like the idea behind this
> and as it happens our implementation can accept this syntax.

   We might include text allowing a receiving SMTP server the option to
parse the EHLO string and treat only the part before SP as the domain
or address-literal. (I recall discussing this exact issue with Dave
Crocker during the design of CSV -- we decided to follow the BNF...)

> However, the checks many people have in place will prevent it from
> working on a lot of actual systems out there. I can virtually guarantee
> a ton of interop problems if this is adopted.

   The evidence is clear that its use is not mainstream. Our analysis
shows it to be unusable except in special cases which amount to prior
arrangement between client and server. It looks to me to be pretty
close to consensus to drop it from the text (and BNF).

   I recommend dropping it: adding it to the BNF would clearly violate
the principle of least amazement.

--
John Leslie <[email-masked]>

Date: Wed, 9 May 2007 08:36:25 +0100
From: Tony Finch <[email-masked]>
To: [email-masked]
cc: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

On Tue, 8 May 2007, Dave Crocker wrote:
>
> The header contains two very different types of information.  One is between
> the originator and the recipient, such as TO and Subject and Date.  The other
> type is part of the handling information associated with message transfer.

All of this information is for consumption by user-level agents, some
generated by the originating user agent, some by intermediate user agents
(e.g. List- and Resent- fields) and some by intermediate MTAs.

> If the latter is not part of the envelope, then what is?

The STANDARD (RFC 822, RFC 1123) definition is just the MAIL and RCPT
commands and excludes all message data. The only other meaning of
envelope in Internet Mail specifications is the IMAP envelope which is
just the basic header fields that MUAs usually display, and has nothing to
do with transport or tracing. i.e. the definition in
draft-crocker-email-arch is an entirely new invention.

The envelope is for consumption by transport-level agents, and is
discarded on final delivery. (Therefore IMAP and POP don't provide access
to it because it isn't there.) This means that user agents can't see the
envelope. Any feature that depends on envelope information to work (e.g.
DSNs) must therefore be implemented by a transport agent. Any feature that
must be implemented by a user agent (e.g. MDNs) must be signalled in the
header not the envelope.

There used to be a fairly firm architectural principle that MTAs should
not have to examine the message data in order to perform their function in
the usual case. No examination is needed to add trace information.
Downgrading for 8BITMIME is an instructive example: the whole point of the
BODY= parameter on the MAIL command is so that the MTA can avoid looking
at the data in the normal cases.

The envelope/data distinction is pretty fundamental, and corresponds
fairly exactly to the user/transport layering.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
CROMARTY FORTH TYNE DOGGER: NORTHWEST BACKING SOUTHEAST 4 OR 5, OCCASIONALLY
6. MODERATE OR ROUGH. SHOWERS. MAINLY GOOD.

Date: Tue, 08 May 2007 22:06:26 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Peter,

Peter J. Holzer wrote:
> * Envelope:
>   I find the usage of the term envelope for the combination of the SMTP
>   envelope and part of the header very unusual and somewhat confusing,
>   despite the explanation in 4.1.1. (especially since it is first used
>   in 2.2.3). I also don't really see a reason for it: Why not keep
>   envelope and header separate and merely note that MTAs may add some
>   information to the header?

The header contains two very different types of information.  One is between 
the originator and the recipient, such as TO and Subject and Date.  The other 
type is part of the handling information associated with message transfer.

If the latter is not part of the envelope, then what is? Entries such as 
Return-path underscore this class of information within the header.  As do 
Received header fields.

In other words, the "header" is a syntactic construct within the message, but 
it contains information that divides into two broad semantic categories.

> * 3.3.1 Message-ID: 
> 
>   "it is associated with the RFC2822.From field, ..." How so? I don't
>   see any conceptual association between the From and the Message-ID
>   field: While some MUAs may use (parts of) the From field in generating
>   the Message-ID, others ignore it.

The discussion about Message-ID attempts to characterize it as distinguishing 
among different messages.  The difference among messages is primarily due to 
choices made by the authors.  That's the From field.

Or, at least, that's the theory behind the statement...

> * 4.1 Message Data:
> 
>   "Message Filtering (SIEVE)". It might be worth pointing out that while
>   SIEVE is the only filtering language specified in an RFC, other such
>   languages are in use.

A similar qualifier would apply to virtually every mechanism described in the 
document.  While true, I'm not sure how helpful it would be.  For simplicity 
and focus, the paper limits itself to discussing standards-based mechanisms 
that have gained substantial use in the open Internet.

> * 4.1.4 Identity references in a message:
>   
>   RFC2822.BCC: "An MUA will typically make separate postings for TO and
>   CC recpients, versus BCC recipients". Is this really typical? In my
>   experience, the message is normally only posted once, with the BCC
>   header removed, and all recipients in the envelope. 

I have not tracked the latest dominant behavior.  I know that MUAs have a long 
history of using a couple of different schemes.  If folks tell me to remove 
the "typically", I'm glad to do it.  (I might do it anyhow, but at this point, 
I'd rather not make any change that isn't due to consensus pressure or 
identifying a plain error.

> * 5.1 Aliasing:
> 
>   "It resubmits the message, replacing the envelope address, on behalf
>   of the mailbox address that was listed in the envelope". It would be
>   clearer if the sentence read "... replacing the envelope recipient
>   address ...". 

ack.

> * 5.3 Mailing Lists
> 
>   "... can be viewed as an elaboration of the Re-Director role"
>   Since the previous section is called "Re-Sending" and not
>   "Re-Directing", that probably should be "... of the Re-Sender role".

ack.  I'll review the text.

Thanks!

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

From: [email-masked]
Cc: Tony Hansen <[email-masked]>, [email-masked]
Date: Tue, 08 May 2007 18:47:18 -0700 (PDT)
Subject: Re: rfc2821bis-01 Issue 19: Explanatory text after literal in EHLO
To: Tony Finch <[email-masked]>
Sender: [email-masked]

> On Thu, 3 May 2007, Tony Hansen wrote:
> >
> > I'm going to close the issue, but note, as Frank suggests, that it
> > should be checked as part of the implementation and interoperability report.

> Based on my testing, common implmentations (Sendmail, Exim, Postfix,
> Exchange) do not accept this syntax. (Some of them can be configured to
> accept any kind of junk after EHLO but if their syntax checker is turned
> on they reject explained-literals.) My servers have seen no (zero)
> instances of this syntax by clients in the last month (out of two to three
> million messages per day). (We don't keep logs longer than that.)

> As far as I can tell from the DRUMS archive this wording was added to
> draft-ietf-drums-smtpupd-04 without any pertinent discussion - it seemed
> to be a reaction to an argument about canonical names.

> I think this all indicates that the feature should be removed.

I have to agree with Tony here. I like the idea behind this and as it happens
our implementation can accept this syntax. However, the checks many people have
in place will prevent it from working on a lot of actual systems out there. I
can virtually guarantee a ton of interop problems if this is adopted.

				Ned

Date: Wed, 9 May 2007 00:37:50 +0100
From: Tony Finch <[email-masked]>
To: Tony Hansen <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-01 Issue 19: Explanatory text after literal in EHLO
Sender: [email-masked]

On Thu, 3 May 2007, Tony Hansen wrote:
>
> I'm going to close the issue, but note, as Frank suggests, that it
> should be checked as part of the implementation and interoperability report.

Based on my testing, common implmentations (Sendmail, Exim, Postfix,
Exchange) do not accept this syntax. (Some of them can be configured to
accept any kind of junk after EHLO but if their syntax checker is turned
on they reject explained-literals.) My servers have seen no (zero)
instances of this syntax by clients in the last month (out of two to three
million messages per day). (We don't keep logs longer than that.)

As far as I can tell from the DRUMS archive this wording was added to
draft-ietf-drums-smtpupd-04 without any pertinent discussion - it seemed
to be a reaction to an argument about canonical names.

I think this all indicates that the feature should be removed.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BAILEY: EAST OR SOUTHEAST 4 INCREASING 5 TO 7, PERHAPS GALE 8 LATER. MODERATE
OR ROUGH. SHOWERS. GOOD.

Date: Wed, 9 May 2007 00:13:32 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--SUOF0GtieIMvvwua
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-05-06 14:59:06 -0700, Dave Crocker wrote:
> I've sent in draft-crocker-email-arch-07 to the I-D folks.
>=20
> The html version is at:
>=20
>  <http://bbiw.net/specifications/draft-crocker-email-arch-07.html>
>=20
> After two years and six major revisions, this is the version that will be=
=20
> submitted to the IESG for standardization.
>=20
> If you have any final comments, please get them to me.

Very nice, overall.

I haven't read the previous drafts (although I think I was already
subscribed to this list when -06 came out - I certainly remember seeing
some discussion), so I apologize if the matters I noticed have already
been discussed to death. This is almost certainly the case for the
first one:

* Envelope:
  I find the usage of the term envelope for the combination of the SMTP
  envelope and part of the header very unusual and somewhat confusing,
  despite the explanation in 4.1.1. (especially since it is first used
  in 2.2.3). I also don't really see a reason for it: Why not keep
  envelope and header separate and merely note that MTAs may add some
  information to the header?

* 3.3.1 Message-ID:=20

  "it is associated with the RFC2822.From field, ..." How so? I don't
  see any conceptual association between the From and the Message-ID
  field: While some MUAs may use (parts of) the From field in generating
  the Message-ID, others ignore it.

* 4.1 Message Data:

  "Message Filtering (SIEVE)". It might be worth pointing out that while
  SIEVE is the only filtering language specified in an RFC, other such
  languages are in use.

* 4.1.4 Identity references in a message:
 =20
  RFC2822.BCC: "An MUA will typically make separate postings for TO and
  CC recpients, versus BCC recipients". Is this really typical? In my
  experience, the message is normally only posted once, with the BCC
  header removed, and all recipients in the envelope.=20

* 5.1 Aliasing:

  "It resubmits the message, replacing the envelope address, on behalf
  of the mailbox address that was listed in the envelope". It would be
  clearer if the sentence read "... replacing the envelope recipient
  address ...".=20

* 5.3 Mailing Lists

  "... can be viewed as an elaboration of the Re-Director role"
  Since the previous section is called "Re-Sending" and not
  "Re-Directing", that probably should be "... of the Re-Sender role".

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--SUOF0GtieIMvvwua
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--SUOF0GtieIMvvwua--

Date: Tue, 08 May 2007 13:08:52 -0700
From: Dave Crocker <[email-masked]>
Organization: Brandenburg InternetWorking
To: Tony Finch <[email-masked]>
CC: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Tony Finch wrote:
> On Tue, 8 May 2007, Dave Crocker wrote:
>> The discussion text is about the Bounce Handler, not a Bounce Creator, or the
>> like.
> 
> Ahright, I misundersttod exactly what was being discussed. It makes sense
> now.

Absent push-back from anyone else, I'm going to take your response as meaning 
that my proposal is acceptable to other folks.

(I'm working on the theory that satisfying someone who initially objected is a 
good indication that other potential objectors will also be satisfied. Of 
course, this only works if current and potential objectors are reasonable 
people and the topic isn't too difficult.  I think the current situation 
qualifies...)

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Tue, 8 May 2007 20:50:48 +0100
From: Tony Finch <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: "Mark E. Mallett" <[email-masked]>, [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

On Tue, 8 May 2007, Dave Crocker wrote:
>
> The discussion text is about the Bounce Handler, not a Bounce Creator, or the
> like.

Ahright, I misundersttod exactly what was being discussed. It makes sense
now.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
HUMBER: WEST OR NORTHWEST 5 TO 7 DECREASING 4 OR 5, BACKING SOUTHEAST LATER.
MODERATE OR ROUGH. RAIN OR SHOWERS. MODERATE OR GOOD.

Date: Tue, 08 May 2007 15:38:32 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Dave Crocker <[email-masked]>
CC: "Mark E. Mallett" <[email-masked]>, [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Dave Crocker wrote:

> I propose to move all Bounce description to be in the MUA, unless folks 
> think that is a problem.  For me, the deciding factor, is that the use 
> of the address does deliver to an MUA.

Interesting view point.

I would suggest to also add a clause or sentence that it MAY NOT 
necessarily be the same MUA from which it originated (A.K.A the MUA 
author, the MUA writer of the message).

The case in point would be a mailing list; Bounces will typically go to 
the list admin.

--
HLS

Date: Tue, 08 May 2007 12:07:52 -0700
From: Dave Crocker <[email-masked]>
Organization: Brandenburg InternetWorking
To: Tony Finch <[email-masked]>
CC: "Mark E. Mallett" <[email-masked]>, [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Tony Finch wrote:
> On Tue, 8 May 2007, Dave Crocker wrote:
>> I propose to move all Bounce description to be in the MUA, unless folks think
>> that is a problem.  For me, the deciding factor, is that the use of the
>> address does deliver to an MUA.
> 
> Um, everything delivers to an MUA. Bounce generation is an MTA feature -
> the MUA can't do it properly because it doesn't have access to the DSN
> options in the envelope.

Well, certainly the message is *generated* by an MHAS module, but it is 
delivered to a mailbox, which is an MUA/MS construct.

The diagram is depicting the destination, so the box needs to be outside of 
the MHS.

That leaves where to list it on the Table and where to place the discussion 
text about it.

The Table lists who sets the value, so that has to be Source.

The discussion text is about the Bounce Handler, not a Bounce Creator, or the 
like.  The Handler is simply a mailbox, not a special MHS function.  No?

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Tue, 8 May 2007 19:52:15 +0100
From: Tony Finch <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: "Mark E. Mallett" <[email-masked]>, [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

On Tue, 8 May 2007, Dave Crocker wrote:
>
> I propose to move all Bounce description to be in the MUA, unless folks think
> that is a problem.  For me, the deciding factor, is that the use of the
> address does deliver to an MUA.

Um, everything delivers to an MUA. Bounce generation is an MTA feature -
the MUA can't do it properly because it doesn't have access to the DSN
options in the envelope.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
HEBRIDES: WESTERLY BECOMING VARIABLE 3 OR 4, OCCASIONALLY 5. SLIGHT OR
MODERATE. SHOWERS. GOOD.

Date: Tue, 08 May 2007 11:25:15 -0700
From: Dave Crocker <[email-masked]>
Organization: Brandenburg InternetWorking
To: "Mark E. Mallett" <[email-masked]>
CC: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Mark,

thanks for the comments.

responses on the items needing one:

Mark E. Mallett wrote:
> Looks pretty nice.  Some very small stuff (against the -07 version on
> ietf.org, which I would assume is the same):

yes.

> ====================
> 
> Change notes say that "Bounce" has been moved out of the MHS (diagram),
> and so it is; but there is an MHS Actors subsection (2.2.2) for Bounce
> Handler, implying that Bounce Handler is inside the MHS.  This is
> confusing.

Oh.  Right.  Hmmm.

I've been so hung up on emphasizing that the address is set by the 
rfc2822.sender (Source) addressee that I forgot to follow through on the 
implication that it really is an MUA address.

I propose to move all Bounce description to be in the MUA, unless folks think 
that is a problem.  For me, the deciding factor, is that the use of the 
address does deliver to an MUA.

>> 4.1.4.  Identity References in a Message
> 
> This section ends with subsections for "List-ID" and "List-*" which are
> not included in the table at the beginning of the section.  Is this
> intentional?  Relatedly, "IP Address" appears in the table but has
> no subsection.

No, so thanks for catching it.  I moved stuff around quite a bit, so it's not 
surprising that there are some inconsistencies.  John Leslie also noted that 
ENVID got inconsistent treatment.  The next version fixes that.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 8 May 2007 14:08:20 -0400
To: [email-masked]
Cc: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

On Sun, May 06, 2007 at 02:59:06PM -0700, Dave Crocker wrote:
> 
> Folks,
> 
> I've sent in draft-crocker-email-arch-07 to the I-D folks.
> 
> The html version is at:
> 
>  <http://bbiw.net/specifications/draft-crocker-email-arch-07.html>
> 
> After two years and six major revisions, this is the version that will be 
> submitted to the IESG for standardization.
> 
> If you have any final comments, please get them to me.

Looks pretty nice.  Some very small stuff (against the -07 version on
ietf.org, which I would assume is the same):

====================

Change notes say that "Bounce" has been moved out of the MHS (diagram),
and so it is; but there is an MHS Actors subsection (2.2.2) for Bounce
Handler, implying that Bounce Handler is inside the MHS.  This is
confusing.

> 2.3.  Administrative Actors
> 
>    Actors often will are associated with different organizations, each

omit "will"

> 4.1.  Message Data
> 
>       Delivery Status Notification (DSN):
> 
>          A Delivery Status Notification (DSN) is a message that can be
>          generated by the MHS (MSA, MTA or MDA) and sent to the
>          RFC2821.MailFrom address.  The mailbox for this is shown as
>          Bounces in Figure 5.  It provides information about message
>          transit, such as transmission errors or successful delivery.

Quirk:  the "It" that begins the last sentence appears to refer to the
bounce mailbox, which can be confusing.  Better to say "The DSN".

Also in 4.1:

>       Message Disposition Notification (MDN):
> 
>          A Message Disposition Notification (MDN) is a message that
>          provides information about user-level, Recipient-side message
>          processing, such as indicating that the message has been
>          displayed [RFC3798] or the form of content that can be
>          supported.  [RFC3297] It can be generated by an rMUA and is
>          sent to the Disposition-Notification-To address(es).  The
>          mailbox for this is shown as Disp in Figure 5.  It

Dangling "It", dang it.

> 4.1.4.  Identity References in a Message

This section ends with subsections for "List-ID" and "List-*" which are
not included in the table at the beginning of the section.  Is this
intentional?  Relatedly, "IP Address" appears in the table but has
no subsection.

> 5.3.  Mailing Lists

>       RFC2822.TO, RFC2822.CC
> 
>          Set by: original Originator
> 
>          These usually contains the original list of Recipient
>          addresses.

"usually contain" (not contains)

> 5.4.  Gateways

>       RFC2822.Sender
> 
>          Set by: Originator Source or Mediator Source
> 
>          This can retain the original value or can be set to a new
>          address

Punctuation:  most subsections end with a period, this one doesn't.

mm

Date: Mon, 07 May 2007 08:50:05 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: John Leslie <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

John Leslie wrote:
> ] 3.3.2 ENVID
> 
>    This looks very interesting; but seems to fail to explain how it
> might be set and/or read. Am I missing something?

Sorry.  Hadn't had my coffee yet, but now I think I understand.

Let's see:

 From RFC3461:
> 4.4 The ENVID parameter to the ESMTP MAIL command
> 
>    The ENVID esmtp-keyword of the SMTP MAIL command is used to specify
>    an "envelope identifier" to be transmitted along with the message and
>    included in any DSNs issued for any of the recipients named in this
>    SMTP transaction.  The purpose of the envelope identifier is to allow
>    the sender of a message to identify the transaction for which the DSN
>    was issued.

So I think that means that email-arch should show ENVID as being set by 
Source, yes?

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Mon, 07 May 2007 07:48:14 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: John Leslie <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

John,

Please clarify.  I think that the document does not explain "how" any values 
are set, so I'm not sure what you are looking for.

d/

John Leslie wrote:
> Dave Crocker <[email-masked]> wrote:
>>  <http://bbiw.net/specifications/draft-crocker-email-arch-07.html>
> 
>    I notice
> 
> ] 3.3.2 ENVID
> 
>    This looks very interesting; but seems to fail to explain how it
> might be set and/or read. Am I missing something?

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Mon, 7 May 2007 08:38:55 -0400
From: John Leslie <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Dave Crocker <[email-masked]> wrote:
> 
>  <http://bbiw.net/specifications/draft-crocker-email-arch-07.html>

   I notice

] 3.3.2 ENVID

   This looks very interesting; but seems to fail to explain how it
might be set and/or read. Am I missing something?

] The ENVID (envelope identifier) is a tag that is primarily for use
] within Delivery Status Notifications (DSN), so that the Bounce
] Address (RFC2821.MailFrom) recipient can correlate the DSN with a
] particular message. [RFC3461] The ENVID is therefore used from one
] message posting, until the directly-resulting message deliveries.
] It does not survive re-postings.
] 
] The format of an ENVID is free-form. Although its creator might
] choose to impose structure on the string, none is imposed by
] Internet standards. By implication, the scope of the string is
] defined by the domain name of the Bounce Address.

--
John Leslie <[email-masked]>

Date: Sun, 06 May 2007 14:59:06 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Final draft:  draft-crocker-email-arch
Sender: [email-masked]

Folks,

I've sent in draft-crocker-email-arch-07 to the I-D folks.

The html version is at:

  <http://bbiw.net/specifications/draft-crocker-email-arch-07.html>

After two years and six major revisions, this is the version that will be 
submitted to the IESG for standardization.

If you have any final comments, please get them to me.

Thanks.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

From: [email-masked]
Cc: [email-masked]
Date: Thu, 03 May 2007 14:06:03 -0700 (PDT)
Subject: Re: DNS VRFY (was: Somewhat OT: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3)
To: Frank Ellermann <[email-masked]>
Sender: [email-masked]

> [email-masked] wrote:

> > Hash function agility. Recent experience has shown that hard coding a
> > specific hash function in an application is a really bad idea. (MD5 in
> > particular is sufficiently broken at this point that using it in any
> > new application is essentially a nonstarter, but SHA-1 or even SHA-256
> > should not be hard coded either.)

> Somewhat OT, but maybe you confused "hash" and "cryptographic hash" (?)

No confusion on my part -  I think the security considerastions here are just
sufficient to call for a reasonably strong hash. Especially since it is so
simple to start with a strong hash and also provide a means to upgrade should
the need arise.

> For the purpose discussed here you have a fixed right hand side, and we
> can (arbitrarily) say that this can be restricted to max. 50 characters.

> That gives us about 200 octets to "encode" a hash of a complete address,
> let's say four labels of length 50.  To avoid trouble we can limit these
> labels to a variant of B32 char.s (we're not forced to use LDH labels),
> so that gives us about 5 * 200 = 1000 bits.  MD5 isn't long enough for
> 1000 bits, it would result in more collisions than necessary.

I do agree that MD5 is a bit short, but we don't need anything like 1000 bits
of hash, birthday paradox notwithstanding. SHA-256 is more than sufficient to
start with for this application and will likely prove sufficient for a long
time.

> But I've another objection to this idea:  For the example "New Orleans
> is down", where are the name servers answering these queries from the
> backup MX ?  If the name servers are also down it won't work when it's
> needed.

That's purely an operational issue that existing DNS facilities are more than
capable of handling. Here's one easy way to do it: Have the secondary MX also
set up a local DNS secondary. Now the entire zone will be transferred and kept
up to date. If the primary goes down the secondary will continue to operate,
albeit with stale data, which is why the secondary almost certainly needs to
return 4yz errors for unknown addresses.

				Ned

Date: Thu, 3 May 2007 22:41:38 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: Restrictive ~ priority
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--u3/rZRmxL6MmkK24
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-05-03 16:05:31 -0400, Hector Santos wrote:
> Frank Ellermann wrote:
> >>You especially don't want an "accept everything" policy on the
> >>lowest-priority MX because spammers do expect and exploit that.
> >=20
> >>(Hmm, is that another recommendation that should go into the RFC if
> >>it isn't already there? "If there are multiple MX for a domain, they
> >>SHOULD implement the same policy for accepting or rejecting mail. In
> >>particular, a lower priority MX should not accept mail that a higher
> >>priority MX rejects")
> >
> >Yes, but I'd say "for MXs with a different priority the MX with the
> >numerically higher priority SHOULD be at least as restrictive with
> >respect to accepting mails for the domain in question as the MX with
> >the numerically lower priority."=20
>=20
>=20
> Here is a MX version of "GreyListing," that was getting some interesting=
=20
> positive results from some of our sysops who tried it:
>=20
>     http://www.joreybump.com/code/howto/nolisting.html

I was aware of that, but I realize that the sentence I suggested covers
that case poorly. The lowest priority MX in this scheme doesn't usually
even listen on port 25, and if it does, it answers every request with a
temporary error. So it never really accepts or rejects mail.

What I wanted to prevent is the case where a conforming client that
tries the MXs in the correct order cannot deliver a mail because it gets
a 5xx error, but a non-conforming client which tries the MXs in some
other order can deliver the mail. If that is wide-spread it encourages
writing non-conforming clients, both for spam and legitimate purposes.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--u3/rZRmxL6MmkK24
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--u3/rZRmxL6MmkK24--

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: DNS VRFY (was: Somewhat OT: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3)
Date:  Thu, 03 May 2007 21:51:47 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

[email-masked] wrote:

> Hash function agility. Recent experience has shown that hard coding a
> specific hash function in an application is a really bad idea. (MD5 in
> particular is sufficiently broken at this point that using it in any
> new application is essentially a nonstarter, but SHA-1 or even SHA-256
> should not be hard coded either.)

Somewhat OT, but maybe you confused "hash" and "cryptographic hash" (?)
For the purpose discussed here you have a fixed right hand side, and we
can (arbitrarily) say that this can be restricted to max. 50 characters.

That gives us about 200 octets to "encode" a hash of a complete address,
let's say four labels of length 50.  To avoid trouble we can limit these
labels to a variant of B32 char.s (we're not forced to use LDH labels),
so that gives us about 5 * 200 = 1000 bits.  MD5 isn't long enough for
1000 bits, it would result in more collisions than necessary.

But I've another objection to this idea:  For the example "New Orleans
is down", where are the name servers answering these queries from the
backup MX ?  If the name servers are also down it won't work when it's
needed.

Frank

Date: Thu, 03 May 2007 16:05:31 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: Restrictive ~ priority
Sender: [email-masked]

Frank Ellermann wrote:

>> You especially don't want an "accept everything" policy on the
>> lowest-priority MX because spammers do expect and exploit that.
>  
>> (Hmm, is that another recommendation that should go into the RFC if
>> it isn't already there? "If there are multiple MX for a domain, they
>> SHOULD implement the same policy for accepting or rejecting mail. In
>> particular, a lower priority MX should not accept mail that a higher
>> priority MX rejects")
> 
> Yes, but I'd say "for MXs with a different priority the MX with the
> numerically higher priority SHOULD be at least as restrictive with
> respect to accepting mails for the domain in question as the MX with
> the numerically lower priority." 

Here is a MX version of "GreyListing," that was getting some interesting 
positive results from some of our sysops who tried it:

     http://www.joreybump.com/code/howto/nolisting.html

Note: I'm not endorsing it, nor do I know how spread the usage is. Its 
just caught my attention when its as discussed in our anti-spam support 
forum.

--
HLS

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Restrictive ~ priority (was: Somewhat OT: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3)
Date:  Thu, 03 May 2007 21:25:40 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 38
Sender: [email-masked]

Peter J. Holzer wrote:

> If you are doing any serious anti-spam filtering you really want the
> same filter rules on all your MXs.
..^^^^
For the backup MX example earlier in this thread I'd say that an MX
should be stricter if its priority is numerically higher.  Something
as in <[email-masked]> can't work:

| The main problem is that the backup MX may have a less restrictive
| filtering policy than the lower numbered MX.

And as Hector mentioned elsewhere the spammers will make sure that
this setup is fixed before it's seriously needed (= primary MX down).

> You especially don't want an "accept everything" policy on the
> lowest-priority MX because spammers do expect and exploit that.
 
> (Hmm, is that another recommendation that should go into the RFC if
> it isn't already there? "If there are multiple MX for a domain, they
> SHOULD implement the same policy for accepting or rejecting mail. In
> particular, a lower priority MX should not accept mail that a higher
> priority MX rejects")

Yes, but I'd say "for MXs with a different priority the MX with the
numerically higher priority SHOULD be at least as restrictive with
respect to accepting mails for the domain in question as the MX with
the numerically lower priority." 

The "SHOULD" is clearly justified, because it would cause unnecessary
backscatter otherwise.  Admittedly it's not possible to define what
"restrictive" precisely means.  A backup MX at third party will use
its own definition of "restrictive", and to determine what's "more 
restrictive" would need a statistical evaluation.  But I think the
meaning should be clear for implementors and admins.
   
Frank

Date: Thu, 03 May 2007 15:01:09 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2181bis-01 Issue 20: General security considerations overhaul
Sender: [email-masked]

This topic has also stalled. Particular issues from
draft-klensin-rfc2821-security-00 MAY be raised later, but the catchall
issue 20 will be closed.

	Tony

John C Klensin wrote:
> Hi.
> 
> Another issue from digging through the document.... This is
> prior to trying to trying to post -02, but I don't expect
> responses to make it into -02 unless they are very quick, very
> clear, and represent a clear consensus.
> 
> RFC 3552 contained a fairly extensive analysis of what it
> thought the Security Considerations section of RFC 2821 ought to
> look like.   Some of those changes have already been
> incorporated in 2821bis, but a number of the suggestions are
> controversial or possibly just plain wrong.  An extended
> discussion of some of the issues appears in the long-expired
> I-D, draft-klensin-rfc2821-security-00.txt, which is now
> referenced in the 2821bis-01 text.
> 
> I can repost that I-D if needed, but that seems worthwhile only
> if we are going to turn every point it raises into an issue and
> discuss it.  Otherwise, I need to delete the discussion that now
> appears at the beginning of the Security Considerations section
> and the reference.
> 
> Instructions?
> 
>     john
> 

Date: Thu, 03 May 2007 14:55:14 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-01 Issue 19: Explanatory text after literal in EHLO
Sender: [email-masked]

This issue has stalled.

I'm going to close the issue, but note, as Frank suggests, that it
should be checked as part of the implementation and interoperability report.

	Tony Hansen
	[email-masked]

John C Klensin wrote:
> 
> 
> --On Thursday, 12 April, 2007 14:28 -0400 "Mark E. Mallett"
> <[email-masked]> wrote:
> 
>> On Thu, Apr 12, 2007 at 07:08:43PM +0100, Tony Finch wrote:
>>>    ehlo              = "EHLO" SP ( Domain / Explained-literal
>>>    ) CRLF
>>>
>>>    Explained-literal = address-literal [string]
>>>
>>> Um, boggle. Why was this new syntax introduced?
>> I'm guessing the choice was either remove it from the
>> narrative or add it to the BNF.
> 
> yes, probably that too, since the relevant narrative hasn't
> changed significantly from 2821.
> 
>>  Flipping the coin the other way might be better.
> 
> No opinion, but you are the second person to speak up, so I'm
> assigning Issue 19 to this.
> 
>    john
> 

Date: Thu, 03 May 2007 14:48:13 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: Updated status on 2821bis 4/24
Sender: [email-masked]

There has been no additional discussion on issues 3, 16, 17, 26, 27 and
28 since 2821bis-03 came out. These issues are now considered resolved
and closed.

Thank you everyone for your help.

	Tony Hansen
	[email-masked]

Tony Hansen wrote:
> This is a recap of where we stand on issues that I consider to be closed
> with respect to 2821bis-02. 
> ...
> Text has been changed in -03 for the following issues:
> 
> 3	EHLO validation in sect 4.1.4, para 6
> 16	General use of "is permitted", etc
> 17	Should text be clarified to single code in continuations
> 	and dealing with 1yz
> 26	Revise description of reverse-paths to prevent repeated
> 	citations of "server adds its own…"
> 27	Straighten out and define terminology for talking about headers
> 28	Client behavior with unrecognized code
> 
> Unless there's a problem with the updated text in -03, these issues will
> be considered closed.

Date: Thu, 03 May 2007 14:45:53 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: Recap: 2821bis-01 Issue 3: EHLO validation in sect 4.1.4, para 6
Sender: [email-masked]

I'm sorry; this should have said "since 2821bis-03 came out".

	Tony Hansen
	[email-masked]

Tony Hansen wrote:
> There has been no additional discussion on this issue since 2821bis-02
> came out. This issue is now considered closed.
> 
> 	Tony Hansen
> 	[email-masked]
> 
> Tony Hansen wrote:
>> The following issue with 2821bis-01 is still open, but we have a
>> tentative conclusion that needs to be affirmed after 2821bis-02 comes out.
>>
>> 3)	the use and validation of the domain name value in EHLO or HELO
>>
>> The discussion on this issue was all over the place with no totally
>> clear consensus. Some of the text in section 4.1.4 has been rewritten to
>> eliminate redundancies and to add more cross references. When 2821bis-02
>> comes out, please review the text in section 4.1.4.
>>
>> 	Tony Hansen
>> 	[email-masked]
>>
> 

Date: Thu, 03 May 2007 14:37:34 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: Recap: 2821bis-01 Issue 3: EHLO validation in sect 4.1.4, para 6
Sender: [email-masked]

There has been no additional discussion on this issue since 2821bis-02
came out. This issue is now considered closed.

	Tony Hansen
	[email-masked]

Tony Hansen wrote:
> The following issue with 2821bis-01 is still open, but we have a
> tentative conclusion that needs to be affirmed after 2821bis-02 comes out.
> 
> 3)	the use and validation of the domain name value in EHLO or HELO
> 
> The discussion on this issue was all over the place with no totally
> clear consensus. Some of the text in section 4.1.4 has been rewritten to
> eliminate redundancies and to add more cross references. When 2821bis-02
> comes out, please review the text in section 4.1.4.
> 
> 	Tony Hansen
> 	[email-masked]
> 

From: [email-masked]
Cc: [email-masked]
Date: Thu, 03 May 2007 10:43:15 -0700 (PDT)
Subject: Re: DNS VRFY (was: Somewhat OT: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3)
To: SM <[email-masked]>
Sender: [email-masked]

> Hi David,

> This reply is off-topic and is not related to the current rfc2821-bis
> discussion.

> At 19:58 01-05-2007, David F. Skoll wrote:

> >Perhaps an RFC for querying whether or not an e-mail address is
> >valid using DNS? :-)
> >
> >   host -t txt some.email.AT.example.com.email-verification-zone.example.com

> This is an interesting idea.  How about:

> host -t txt f098474401a124aa5a2738b6353e22da.example.com._vrfy.example.com

> where f098474401a124aa5a2738b6353e22da is a MD5 hash of the email
> address.  That's to avoid disclosing the actual email address.

Just off the top of my head:

(1) Case sensitivity. Domains aren't case sensitive but local parts can be
    at the whim of the domain administration. Case insensitive local parts
    would have to be upper or lower cases prior to hashing.

(2) Subaddresses. Lots of sites support various forms of subaddressing, and
    when subaddresses are allowed they may provide users with the ability to
    place essentially arbitrary strings in the local part somewhere.

(3) Hash function agility. Recent experience has shown that hard coding a
    specific hash function in an application is a really bad idea. (MD5 in
    particular is sufficiently broken at this point that using it in any
    new application is essentially a nonstarter, but SHA-1 or even SHA-256
    should not be hard coded either.)

These issues can be addreessed by adding in some additional domain-specific
information, possibly composed as a list of regexps plus an option setting or
two. Interestingly, the DNS actually provides a record for conveying exactly
this sort of information: The NAPTR record. I don't think the entire DDDS
mechanism described in RFCs 3402-3403 is needed here, but a careful profile of
NAPTR records which are then attached to _vrfy.example.com would be sufficient.

I think we can ignore domain literals here since there's no way to look
up an MX for a domain literal.

The hard part of this for some sites is going to be actually enumerating all
their valid addresses. Email address handlers are as a rule constructed around
the ability to determine if a given address is valid, which often is done
through a fairly complex series of checks and normalization operations. Systems
typically aren't as amenable to creating an exhaustive enumeration of all
possib le valid addresses.

> Queries for the zone can be restricted to the DNS server used by the
> secondary MX.  It may not scale well for a site with a lot of email
> addresses.  It should suite the needs of small and medium sites as
> they can use their existing DNS infrastructure to implement it.

Really large sites already have their own backup mail servers, replicated
directories and multihoming solutions in place and hence are unlikely to be
interested in this. So yes, it does make sense to target this for small to
medium sized sites.

					Ned

Date: Thu, 03 May 2007 01:59:21 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: Recap: Re: RFC2821bis-02 Issue 30: Deprecate use of source routes completely and remove text
Sender: [email-masked]

While there has been lots of discussion about source routes in other
threads, there has still been no discussion on this specific suggestion,
which is to deprecate source routes completely. So this specific
suggestion, issue 30, is dead. The other issues with source routes will
be dealt with as per those other threads.

	Tony Hansen
	[email-masked]

Tony Hansen wrote:
> Since John formulated this summary and suggestion almost a week ago,
> there has been no comment. Nix, nada, zilch.
> 
> It does reflect suggestions made in passing in several other messages,
> but I feel that it is important enough that needs to be an explicit
> decision made on the suggestion.
> 
> If there are no further messages on this issue, the *default* decision
> will be "no change".
> 
> 	Tony Hansen
> 	[email-masked]
> 
> John C Klensin wrote:
>> Hi.
>>
>> This has been discussed, but just to give it a number and insure
>> that it doesn't fall through the cracks...
>>
>> 2821 sort of danced around the issue of source routes, going a
>> little further than 1123 did in discouraging their use but not
>> getting rid of them entirely.  The method used to do this in
>> 2821 is a little fuzzy, since it left the text in place
>> instructing relays to copy their own domains into the
>> backward-pointing address.  That was discussed as issue 26 and
>> the working draft for -03 now has that issue straightened out
>> (assuming I've gotten it right).
>>
>> However, that discussion raised another possibility, which is
>> that now is the time to deprecate source routes entirely.  More
>> precisely that would:
>>
>> 	(i) Leave the syntax in place and continue to require
>> 	that systems be able to accept and parse it.
>> 	
>> 	(ii) Forbid systems from using source routes, either
>> 	changing the "they should be ignored but may be used"
>> 	language that now appears in Section 3.3 (and more or
>> 	less repeated in 3.6) to forbidding their use for
>> 	routing or leaving those two sections alone (I consider
>> 	eliminating the redundancy to be an editorial matter if
>> 	I can figure out how to do it).
>> 	
>> 	(iii) Continue with (ii) by changing the "SHOULD not
>> 	generate" in the second paragraph of 4.1.1.3 to MUST NOT.
>> 	
>> 	(iv) Get rid of Appendix C, replacing it with some
>> 	handwaving to the effect that, if people really need to
>> 	know how this obsolete feature worked, they should read
>> 	821.
>> 	
>> 	(v) Strengthen the prohibitions in Appendix F.2.
>>
>> So far, we've heard from a couple of people on-list who feel
>> that this is a much better idea than the patching contemplated
>> by issue 26 and no one who has argued for keeping them.  Whether
>> it is realistic to further deprecate them or not probably
>> depends on interoperability testing although we could arguably
>> prohibit the things as a bad idea that is not widely supported.
>>
>>      john
>>
>>     
>>
> 

Date: Thu, 03 May 2007 01:28:59 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: Recap: Re: rfc2821bis-03: Issue 31: Retain new FTP comparison text in 4.2.1
Sender: [email-masked]

Ok, the additional discussion on this text has indicated that John
should leave the text in, as per his preference, with the possible
addition of Mark's "FTP's" text.

This issue is closed.

	Tony Hansen
	[email-masked]

John C Klensin wrote:
> 
> 
> --On Sunday, 29 April, 2007 23:04 -0400 Tony Hansen
> <[email-masked]> wrote:
> 
>> The consensus on this text
>>
>>   It is worth noting that the file transfer protocol (FTP
>> [35]) uses a   very similar code architecture and that the
>> SMTP codes are based on   the FTP model.  However, SMTP uses a
>> one-command, one-response model,   while FTP is asynchronous
>> and the 1yz codes are not part of the SMTP   model.
>>
>> is almost dead even. I'll leave this one open for two more
>> days, then give a consensus call. If it remains even, it will
>> become "editor's choice".
> 
> Speaking personally (not as editor), I'm slightly inclined to
> leave it, simply because dropping of the 1yz text would seem odd
> without _some_ comment.  Very slightly, however.
> 
>      john

Date: Thu, 03 May 2007 01:22:56 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue  33:  SMTP-Sender, etc.
Sender: [email-masked]

I agree with Frank here: we should *not* spend a lot of time on this.

If there are any *specific* cases where the use of "sender" is
ambiguous, please raise them, and they can be fixed.

Otherwise, this one is not a priority.

	Tony Hansen
	[email-masked]

Frank Ellermann wrote:
> John C Klensin wrote:
> 
>>> Would it be possible to use SMTP-Sender instead of just sender?
> 
>> Possible?  yes.   I'll have to leave to others to figure out if
>> this is a priority.
> 
> Maybe a case by case approach, not a global modification effort.
> 
> Anything related to terms like "sender", "originator", "return
> path", "reverse path", etc. is obviously critical, their meaning
> in RFC 821 depends on the original design, and the context when
> it was designed.
> 
> Nobody would try "mail from B" for mail actually sent from A with
> the known exceptions like "different local part is okay", or "as
> long as A and B are behind the same MX it's fine".  It would be a
> bad plan if something on the route from A to the receiver doesn't
> work, to introduce an unrelated route to B for the error reports.
> 
> An additional route is a potential source for more trouble, and
> if error reports don't make it they're lost, empty reverse path.
> 
> All considerations in this direction were likely "obvious" when
> RFC 821 was written, it could afford to be sloppy with some terms.
> 
> What should a "mail from" be, certainly no arbitrary "bounces to"
> a 3rd party.  From the ten or more aliases of this beast I think
> "envelope sender" as used in RFC 3461 is the best.  It avoids any
> confusion of (2)821 and (2)822, it doesn't mention the less clear
> "from", it doesn't talk about reverse paths and return routes, and
> it says "sender", the entity found guilty to have sent a mail.  On
> whose behalf it sent the mail is beside the point.  The "envelope
> sender" is related to the header field (Resent-) Sender, i.e. the
> responsible agent for the message.  For the "envelope sender" it's
> the responsible agent wrt SMTP.
> 
> IOW, be careful with those "senders", please.
> 
>> Everyone should remember that this was supposed to be a quick
>> effort.
> 
> If anybody promised that this would be a "quick effort" then their
> definition of "quick effort" is "not as bad as 2026bis, more like
> 2616bis or 3490bis, nothing to worry about, no grandson-of-1036" :-)
> 
>> I'd rather just declare the effort to switch to "client-server"
>> terminology a failure.  That would mean going back to the 821
>> terminology of "sender-SMTP" and "receiver-SMTP"
> 
> Possible, but not desirable, "client" and "server" is clear in all
> contexts with two MTAs talking.  It's somewhat more confusing in
> RFC 4408, where "the client" is actually an SMTP-server in its role
> as DNS-client of a name server related to the alleged sender.  If
> you use the client-server terminology only for SMTP, not for DNS,
> it should be clear.
> 
>> not introducing more and entirely new terminology like
>> "transmitter" or "transmitting-client"
> 
> Yes, RFC 2822 uses "transmitter", but it's unnecessary in 2821bis,
> and "transmitting-client" would be a mouthful, also unnecessary.
> 
> Frank
> 
> 

Date: Thu, 03 May 2007 01:15:54 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Recap 2: Re: Recap Issues 0b/21/25
Sender: [email-masked]

Ok, the consensus seems to be for the text below, as long as it is
accompanied by additional changes to section 2.1 on the transfer of
responsibility and proper reporting. See my recap on Issue 32 and
subsequent discussion there for additional information on section 2.1.

For those advocating lots of changes dealing with "proper reporting",
please review sections 6.1, 6.2 and 7.8. As mentioned by Frank, these
sections already deal with the issues of "proper reporting" and when
it's appropriate and not appropriate. However, adding pointers to these
sections to the "proper reporting" text seems called for.

John has some text in hand, based on all of the fine suggestions here,
that will be included in 2821bis-04. Once that comes out, please review
sections 2.1 and 3.6.

	Tony Hansen
	[email-masked]

Tony Hansen wrote:
> I've been rereading all of the discussions around issues 0b/21/25. Some
> of the discussion wants to require verification of the sender before
> ever sending an non-delivery notification.
> 
> While this does reflect what some implementations are doing and is the
> effort of some recent standards and non-standards efforts, it is not
> common deployment. So I'm going to make a ruling that requiring such
> validation goes too far in the direction of a "new behavior" requirement.
> 
> However, I'd like to follow the lead of the 3463 reference found in
> section 2.3.11, and suggest that some wording be added, probably to
> section 3.6, along these lines:
> 
>     This specification does not deal with the verification of return
>     paths for use in delivery notifications. Recent work, such as
>     [add REFERENCES here], has been done in providing ways to
>     ascertain that an address is valid or belongs to the person who
>     actually sent the message. A server MAY attempt to verify the return
>     path before using its address for delivery notifications, but
>     methods of doing so are not defined here nor is any particular
>     method recommended at this time.
> 
> Thoughts on an addition like this? I know it doesn't go far enough for
> some people, and probably goes too far for some other people. So I'm
> offering it as a possible way forward.
> 
> 	Tony Hansen
> 	[email-masked]
> 

Date: Wed, 02 May 2007 15:20:50 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: OT Brainstorm: Email  Validation among different systems
Sender: [email-masked]

> PKI Sign and Encrypt the message body/list of addresses. Since the list 
> is going to end up in an automated process, the need to do the 
> decryption and signature verification is not that big an issue to build 
> in. The Signature validates the list while the encryption protects the 
> addresses themselves from interception of the message.

dkim.org.

or, perhaps, goodmailsystems.com -- as far as I know, they are the only vendor 
with a solution that includes rfc2821.RCPT-TO in the hashed data for the 
signature.

(Disclosure: they are a client.)

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Wed, 02 May 2007 13:15:28 -0700
From: "Carl S. Gutekunst" <[email-masked]>
To: Jeff Macdonald <[email-masked]>
CC: [email-masked]
Subject: Re: OT: 5xy - Do we need a "I MEAN IT!" do not retry response code?
Sender: [email-masked]

Jeff Macdonald wrote:
> I work for an ESP. So while I can only speak for my company, I'm pretty
> certain that most ESPs honor 4xx and 5xx at the MTA level.

I had an ESP as a customer in a past life. I know for a fact they 
retried at least least 3 times on a 5xx error before deciding to give 
up, and needed at least three different messages (with different 
content) in a row to fail before automatic delisting. The reason, I was 
told, was their engineers had observed a Tier 1 ISP occasionally return 
"550 Unknown User" on a valid mailbox. I couldn't argue that point 
because I'd seen that same Tier 1 ISP do that myself.

<csg>

Date: Wed, 2 May 2007 16:05:11 -0400
From: Jeff Macdonald <[email-masked]>
To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: OT: 5xy - Do we need a "I MEAN IT!" do not retry response code?
Sender: [email-masked]

On Wed, May 02, 2007 at 08:54:55AM -0400, Hector Santos wrote:
> 
> Peter J. Holzer wrote:
> >On 2007-05-02 05:28:16 -0400, Hector Santos wrote:
> 
> >I'm not sure either. "hard and soft bounces" could be either 5xx and 4xx
> >respectively, or a "soft bounce" could be a "couldn't deliver for $n
> >hours, will keep trying" message.
> 
> It kind of sounded to me that hard was "for sure rejects" at the SMTP 
> level, and soft being "Bounceback Messages" or NDNs.  But in either 
> case, it tries 3 times before it moves the email address to the 
> do-not-send list. :-(

I work for an ESP. So while I can only speak for my company, I'm pretty
certain that most ESPs honor 4xx and 5xx at the MTA level. Eventually a
message with a 4xx will either be delivered or expire. At that point
the resulting bounce, whether it is a bounce that happended during SMTP
or an out of band bounce that happens after issuing a 250 and then
later sending a NDN[1] (like AOL use to do), an ESP can not just use a
4xx/5xx[2] code to determine whether that email should be removed from
a subscriber list. They have to go to the text portion and make a
determination from that. Consider the following examples:

This would cause an addition to do-not-send list:
550 Unknown User

while these would not:
550 Over Quota
550 5.7.1 Spammer!
550 Spam phrases found

If a mail server is issuing a policy reason, the assumption being made
is that the next message that is sent will not violate the policy. That
is because the next message will contain different content. The
previous message could of been blocked in error. Think of a large drug
company - Merck(?) - and one of their biggest selling products in which
an investor gets a monthly newsletter talking about sales.

I do agree the ESP industry needs to move away from hard and soft when
referring to list maintenance. I typically use permanent and temporary
undel.

I don't think we need a 'I mean it'. I think we need a way to say 'no
more mail will be accepted from this mailstream'. This is to help when
a receiver has decided to block all email from a certain IP or domain.
No sense in having a sending MTA keep sending email. Obviously that is
currently out of scope.

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

Date: Wed, 2 May 2007 15:20:52 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: OT Brainstorm: Email  Validation among different systems
Sender: [email-masked]

At 08:29 -0400 on 05/02/2007, David F. Skoll wrote about Re: OT 
Brainstorm: Email  Validation among different system:

>Hector Santos wrote:
>
>>  My point here is simply that a professional AVS email security service
>>  bureau will most likely have and offer (if they want the business) a
>>  "interface" specification such as http://www.virtualconnect.net which
>>  allows the operator to update their list of valid users via an email
>>  automated fashion.
>
>Ponder the security risks of such an implementation.

PKI Sign and Encrypt the message body/list of addresses. Since the 
list is going to end up in an automated process, the need to do the 
decryption and signature verification is not that big an issue to 
build in. The Signature validates the list while the encryption 
protects the addresses themselves from interception of the message.

Date: Wed, 02 May 2007 10:20:12 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: OT: Re: DNS VRFY
Sender: [email-masked]

Hi David,
At 07:03 02-05-2007, David F. Skoll wrote:
>Actually, this would be generally useful, not just for a secondary MX.
>The lookup could return more than one piece of information.  For
>example, it could say:
>
>o This address will be accepted in a RCPT command.
>o This address should be accepted in a MAIL command.
>
>For example, we have an e-mail address "[email-masked]"
>that's only used for inbound mail; we never send mail from that
>address.  Being able to specify via DNS that MAIL
>FROM:<[email-masked]> is a forgery would be useful.

The idea here is to provide a mechanism to decrease NDNs generated 
because the MX doesn't hold the recipient list (LDAP etc. methods not 
feasible).  This means rejecting during the SMTP phase.  If a SMTP 
client sends an email with "[email-masked]" in the return 
path and it fails recipient verification, the sender bears the brunt 
of the problem.  We are also not introducing any restrictions to the 
current mail architecture.

I suggesting using this mechanism on the MX only to prevent it from 
being used to whitewash a list of addresses.  We also don't have to 
deal with DNS attacks and DNS caching issues.  Access control can be 
addressed through DNS views.

As I see it, the zone holding the hash can be either at a provider's 
end or near the final delivery point.  One can use dynamic DNS or DLZ 
to maintain the zone.  You can even push updates through XFER.  DNS 
caching should not be much of an issue in such a setup.  The zone can 
be hosted on a DNS server different from the one for example.com.

>In addition, allowing SMTP servers to more quickly reject recipient
>addresses is useful; if an ISP can quickly reject a recipient based on
>a DNS lookup rather than attempting an SMTP session, it can save
>resources.

For Direct-to-MX, no third-party gets "hurt" as we are not sending 
any bounces.  In the case of the ISP, they'll have more incentive to 
solve the problem at their end.  After all, they sent the message and 
they can easily track down the offending client and take whatever 
action they see fit.

I'm not attempting to save resources as this may turn into 
over-engineering and we'll have to consider a wider range of issues 
(rate-limit DNS queries, hash reversal, etc.).

>Why not?  People operate RBLs with many thousands of entries and the
>DNS handles it fine.  A site with a lot of e-mail addresses *must* do
>a directory lookup at some point, so it must have the infrastructure
>to do it efficiently.

I expected that argument. :-) The people operating RBLs may be better 
equipped to deal with thousand of entries.  A large ESP may already 
have a LDAP backend to do recipient verification.  What we can do 
here is to propose a simple solution for sites who do not have such means.

Regards,
-sm 

Date: Wed, 02 May 2007 12:45:57 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: OT Brainstorm: Email  Validation among different systems
Sender: [email-masked]

David F. Skoll wrote:

>>> Ponder the security risks of such an implementation.
> 
>> Like what?
> 
> You see no security risks in allowing some random person to set your
> valid recipient list using unauthenticated e-mail?  (X-Note: Password
> is pretty weak authentication and easily sniffed.  At the very least,
> the e-mail should be PGP-signed and possibly encrypted before being
> accepted.)

hmmmm, David, that was a template to create the require email. How it 
was sign, sent or deliver is really not the point. Maybe it was signed, 
maybe TLS/AUTH was used. Even then I would personally, and still have 
more trust in a standard 1 to 1 email transaction, then what you are 
proposing.  Once received, who knows that the other end did. It is 
private mail. So it has weak X-Note? Maybe it is used to confirm the 
TLS/AUTH.  Who knows? That's there internal issue. Hands down, no 
question. The method is already in place is more secure than a public 
database proposal.

>> Lets see, why not use existing user validation LDAP? RADIUS?
> 
> Because DNS is widely-implemented, lightweight, scalable, cacheable
> and has built-in facilities for replication.

True.  RADIUS is pretty fast. LDAP, RADIUS already exist for user 
authentication.  Even then, you still have to write the protocol, and 
get everyone to get on board which will take time. And who is the 
customer?  The Secondary MX Service Bureau who more than likely has a 
system in place?

(NOTE: Devil Advocate here <g>)

 >> Correct. That was the goal of the example. The IP input
 >> was my address using your address - SPF rejected it
 >> as expected.

 > No, check the URL in my original e-mail again.  I used my IP address.

What you used was:

    ip   = 209.191.13.82
    cdn  = [209.191.13.82]
    from = [email-masked]

and SPF reported FAIL which is correct.

The SPF DNS TXT query for roaringpenguin.com is:

       v=spf1 a -all

which says:

Query the A record of the domain roaringpenguin.com, which is:

        206.191.13.82

and test the IP against the connecting IP 209.191.13.82 (simulated 
input).  Since it did not match, the hard fail, -all, resulted in a SPF 
FAIL.

Change the IP to 206.191.13.82 at the test form or the URL and you will 
get a correct pass:

20070502 12:28:01 00006e74 cip : 206.191.13.82
20070502 12:28:01 00006e74 cdn : [206.191.13.82]
20070502 12:28:01 00006e74 from : [email-masked]
....
20070502 12:28:02 00006e74 sapspf : v=spf1 a -all
20070502 12:28:02 00006e74 sapspf : pass (time:0)
20070502 12:28:02 00006e74 finaltest : SPF GlobalResult=-1
                                        CodeResponse=250
20070502 12:28:02 00006e74 result : accept (-1)
20070502 12:28:02 00006e74 wcsap finish (453 msecs)

>> But I will still love to see your DNS-VRFY I-D proposal! <g>
> 
> I've never done an I-D before.  If anyone on the list wants to help
> out, please e-mail me off-list.

Here is what most everyone uses with the spiffy XML2RFC outline/editor.

     http://xml.resource.org/

With this tool, you would be be create the TXT, HTML and XML.

--
HLS

Date: Wed, 2 May 2007 17:40:24 +0200
From: Alex van den Bogaerdt <[email-masked]>
To: [email-masked]
Subject: Re: OT: Re: DNS VRFY
Reply-To: [email-masked]
Mail-Followup-To: [email-masked]
Sender: [email-masked]

On Wed, May 02, 2007 at 10:03:48AM -0400, David F. Skoll wrote:

> For example, we have an e-mail address "[email-masked]"
> that's only used for inbound mail; we never send mail from that
> address.  Being able to specify via DNS that MAIL
> FROM:<[email-masked]> is a forgery would be useful.

You already can, using SPF and its macros.

HTH
Alex

Date: Wed, 02 May 2007 11:10:42 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: OT Brainstorm: Email  Validation among different systems
Sender: [email-masked]

Hector Santos wrote:

>>> My point here is simply that a professional AVS email security service
>>> bureau will most likely have and offer (if they want the business) a
>>> "interface" specification such as http://www.virtualconnect.net which
>>> allows the operator to update their list of valid users via an email
>>> automated fashion.

>> Ponder the security risks of such an implementation.

> Like what?

You see no security risks in allowing some random person to set your
valid recipient list using unauthenticated e-mail?  (X-Note: Password
is pretty weak authentication and easily sniffed.  At the very least,
the e-mail should be PGP-signed and possibly encrypted before being
accepted.)

> Let me put it this way, speaking for myself, lets just say, I have a
> very strong ethical philosophical engineering delima with the idea of
> publicly exposing user information, especially in large database
> amounts.

The fact that:   [email-masked]   exists or doesn't exist
is already exposed.  Exposing it via DNS makes no difference.

[...]

> Lets see, why not use existing user validation LDAP? RADIUS?

Because DNS is widely-implemented, lightweight, scalable, cacheable
and has built-in facilities for replication.

>> Your server incorrectly claims an SPF failure for:

> Correct. That was the goal of the example. The IP input was my address
> using your address - SPF rejected it as expected.

No, check the URL in my original e-mail again.  I used my IP address.

> Personally, I have a problem with exposing my user database or
> suggesting to customers to expose their user database via DNS - I
> prefer to do that in a very exclusive trust exchange environment -
> not publicly, opening it up to MDx/SHaX security threat hacking
> exploits.

You're not exposing your user database.  You're simply saying
whether or not an e-mail address will be accepted in a RCPT command;
such information is already publicly available!

> But I will still love to see your DNS-VRFY I-D proposal! <g>

I've never done an I-D before.  If anyone on the list wants to help
out, please e-mail me off-list.

Regards,

David.

Date: Wed, 02 May 2007 10:34:12 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: Re: OT Brainstorm: Email  Validation among different systems
Sender: [email-masked]

David F. Skoll wrote:
> Hector Santos wrote:
> 
>> My point here is simply that a professional AVS email security service
>> bureau will most likely have and offer (if they want the business) a
>> "interface" specification such as http://www.virtualconnect.net which
>> allows the operator to update their list of valid users via an email
>> automated fashion.
> 
> Ponder the security risks of such an implementation.

Like what?

I could imagine there is always security risk everywhere. But this is a 
100% trusted private exchange and thus lean more on an internal security 
threat, not external, if there is going to be a problem.  Also, I would 
imagine a broken or cancel contract will prohibit the service bureau 
from selling or using the email list any further.  I doubt a legitimate 
service would risk a bad reputation by abusing customer trust.  Not 
worth it.  But I can't tell ya, because we never experienced any 
customer security related issue or problem with our export tool. :-)

If it wasn't obvious, the operator always had the ability to export 
their user accounts.  But this tool was a simulation output of a real 
dynamic  SMTP level validation.  That means all the translations, 
aliases, mailing list names, administrative names, any other special 
names that would be normally be valid at SMTP is outputted.

Example, user account is "David Scroll", there could be 1 or 20 valid 
SMTP level addresses that represent "David Scroll" and this would be 
outputted by the tool.

> DNS has proven pretty scalable, I reckon. :-)  

Well, it depends how widely popular it would be and also how might the 
databases. But I would think the larger it is, the more likely the 
customer would not need other people services. :-)

> As another poster mentioned,
> you could look up the hash of an address to reduce information exposure.
 >
>> Who would create/maintain the zone?
> 
> If example.com publishes an MX record, then example.com could equally
> publish e-mail validation records.

Let me put it this way, speaking for myself, lets just say, I have a 
very strong ethical philosophical engineering delima with the idea of 
publicly exposing user information, especially in large database 
amounts.  The concept is well understood (at least to me and its not the 
first time the DNS based "userid" idea has been explored.  Unless it is 
coupled with a client access concept, negotiated authorization, 
permission, public/private or secret key, what have you, I would be 
interested in seeing your proposal. Otherwise, I won't touch it.  But 
thats just me. :-)

Lets see, why not use existing user validation LDAP? RADIUS?  Existing 
protocols. DKIM also has some user-level authentication, which I believe 
has been put on the back burner.

> Your server incorrectly claims an SPF failure for:

Correct. That was the goal of the example. The IP input was my address 
using your address - SPF rejected it as expected.

> But that's ad-hoc and isn't what I'm talking about.  I'm talking about
> a way to validate RECIPIENT addresses.

Of course. Simply pointing out more than one way to skin a cat.  Trying 
to keep your mind open here. :-)  This Secondary MX services were borne 
yesterday.  These needs were already addressed.  Personally, I have a 
problem with exposing my user database or suggesting to customers to 
expose their user database via DNS - I prefer to do that in a very 
exclusive trust exchange environment - not publicly, opening it up to 
MDx/SHaX security threat hacking exploits.

Consider, that for many old school systems, such as ours when users 
accounts are created, and email service is provided, we use a choice of 
old school Novell's MHS naming convention or an old school straight 
forward user account to email translation:

     first last -->  [email-masked]

So right there that is HALF the battle for guessing user login names. 
Not everyone uses "email addresses" to login.

Anywho, I think the only value from this exercise, at least for 2821bis 
is a possible consideration to talk about persistent SMTP behavior for 
multiple MX environments.  That that is best advice it can give, IMO.

But I will still love to see your DNS-VRFY I-D proposal! <g>

--
HLS

Date: Wed, 02 May 2007 10:03:48 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: OT: Re: DNS VRFY
Sender: [email-masked]

SM wrote:

> This reply is off-topic and is not related to the current rfc2821-bis
> discussion.

Yes.  We should start another list.

> host -t txt f098474401a124aa5a2738b6353e22da.example.com._vrfy.example.com

> where f098474401a124aa5a2738b6353e22da is a MD5 hash of the email
> address.  That's to avoid disclosing the actual email address.

I like it.

> Queries for the zone can be restricted to the DNS server used by the
> secondary MX.

Actually, this would be generally useful, not just for a secondary MX.
The lookup could return more than one piece of information.  For
example, it could say:

o This address will be accepted in a RCPT command.
o This address should be accepted in a MAIL command.

For example, we have an e-mail address "[email-masked]"
that's only used for inbound mail; we never send mail from that
address.  Being able to specify via DNS that MAIL
FROM:<[email-masked]> is a forgery would be useful.

In addition, allowing SMTP servers to more quickly reject recipient
addresses is useful; if an ISP can quickly reject a recipient based on
a DNS lookup rather than attempting an SMTP session, it can save
resources.

> It may not scale well for a site with a lot of email addresses.

Why not?  People operate RBLs with many thousands of entries and the
DNS handles it fine.  A site with a lot of e-mail addresses *must* do
a directory lookup at some point, so it must have the infrastructure
to do it efficiently.

Regards,

David.

Date: Wed, 02 May 2007 08:54:55 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: OT: 5xy - Do we need a "I MEAN IT!" do not retry response code?
Sender: [email-masked]

Peter J. Holzer wrote:
> On 2007-05-02 05:28:16 -0400, Hector Santos wrote:

> My guess is that this is about maintaining the database of mail
> addresses, not about sending individual messages.
> 
> For the mailing-lists which I maintain manually, I do something very
> similar: If I get a bounce, I look at the reason. If its something like
> "no such user" I immediately unsubscribe the address from the list; but
> it's "mailbox full" or "procmail exited with unknown code 42" or
> something like that I assume that the user is going to fix the problem
> and unsubscribe them only if the problem persists for a few days.
> 
> So that looks like that software is actually trying to clean up the
> database with reasonable heuristics.

Right, this emailer made some comments about do manual inspections as 
you say. Feedback is important to them - that includes irregular bounces.

>> I didn't catch well what it meant by "hard and soft bounces" and 
>> "BounceBack Messages?"  Does that main a real accept/bounce transaction 
>> or a 5xy response? or both?
> 
> I'm not sure either. "hard and soft bounces" could be either 5xx and 4xx
> respectively, or a "soft bounce" could be a "couldn't deliver for $n
> hours, will keep trying" message.

It kind of sounded to me that hard was "for sure rejects" at the SMTP 
level, and soft being "Bounceback Messages" or NDNs.  But in either 
case, it tries 3 times before it moves the email address to the 
do-not-send list. :-(

> "BounceBack Messages" are probably NDNs. I find this interesting,
> because it means that to use this software (or at least this feature)
> the spammer has to use a genuine reverse-path.

It went into great length about CAN-SPAM compliance, using valid return 
paths for obtain important feedback into its email campaign tracking.

It even showed somewhat of an aggressive position a few times about not 
tolerating blocks citing CAN-SPAM, that it will carefully scanning their 
logs for blocks and undetected bounces, contacting operators when they 
find the blocks.

It had a few other interesting points regarding "Optimizing E-mail 
Deliverability" delivery, mostly RFC 822 related, this one related to 2821.

Oh hell, maybe others will find the "Spammer Insights" interesting, 
here's the JangoMail web site.

    http://www.jangomail.com/documents.asp

PS: What attracted me to them was their press release regarding its new 
DKIM support.  All emails will be signed with DKIM now, which should 
strike a nerve with Dave Crocker (at least it should) because of their 
erroneous claims such as:

     Email messages signed with DomainKeys and DKIM are guaranteed to
     be unaltered in transit and from the claimed Sender.  They are
     treated differently by receivers like Yahoo! Mail and GMail and
     will help with delivery to the Inbox.

Yeah right, guaranteed?  help with delivery?

Anyway, different WG!

--
HLS

Date: Wed, 02 May 2007 08:29:00 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: OT Brainstorm: Email  Validation among different systems
Sender: [email-masked]

Hector Santos wrote:

> My point here is simply that a professional AVS email security service
> bureau will most likely have and offer (if they want the business) a
> "interface" specification such as http://www.virtualconnect.net which
> allows the operator to update their list of valid users via an email
> automated fashion.

Ponder the security risks of such an implementation.

[...]

> Ideally a good idea, I think there might be DNS overhead/scalability and
> security exposure concerns but if coupled with some client access
> concept it might get consideration.  So why do expect to see the I-D? <g>

DNS has proven pretty scalable, I reckon. :-)  As another poster mentioned,
you could look up the hash of an address to reduce information exposure.

> Who would create/maintain the zone?

If example.com publishes an MX record, then example.com could equally
publish e-mail validation records.

> I would think the service would
> want to minimize call outs and prefer to just get a list from the
> operator, its less overhead and also allows them to import the common
> list from everyone into their own high speed database system.

DNS already has a mechanism for this (zone transfers).

> Also given the direction of the world (good or bad), maybe a SOAP or
> REST API will probably get some a lot interest.

SOAP and REST APIs lose the advantages of DNS: (1) scalability, (2)
cacheability, (3) lightweight request/response, (4) well-established
mechanisms for operating backup servers.

> Similar to our WCSAP service that we use to validate the envelope
> information. It is called directly from our SMTP but it can also be
> called as a REST-like syntax:
> 
> http://www.winserver.com/public/code/html-wcsap?ip=64.26.171.99&cdn=%5b192.168.5.2%5d&from=dfs%40roaringpenguin.com&xid=public&debug=10

Your server incorrectly claims an SPF failure for:
http://www.winserver.com/public/code/html-wcsap?ip=209.191.13.82&cdn=%5b209.191.13.82%5d&from=dfs%40roaringpenguin.com&xid=public&debug=10

> The above will attempt to validate your from address.

But that's ad-hoc and isn't what I'm talking about.  I'm talking about
a way to validate RECIPIENT addresses.

Regards,

David.

Date: Wed, 2 May 2007 13:08:29 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: OT: 5xy - Do we need a "I MEAN IT!" do not retry response code?
Cc: "Peter J. Holzer" <[email-masked]>, John C Klensin <[email-masked]>
Sender: [email-masked]

John C Klensin writes:
> --On Wednesday, May 02, 2007 12:14 PM +0200 "Peter J. Holzer" 
> <[email-masked]> wrote:
>> "BounceBack Messages" are probably NDNs. I find this interesting, 
>> because it means that to use this software (or at least this 
>> feature) the spammer has to use a genuine reverse-path.
>
> Well, the return-path needs to be genuine only for NDNs.

And it only needs to work for a few minutes following MAIL FROM in order 
to get >99% of NDNs.

Arnt

Date: Wed, 02 May 2007 06:40:34 -0400
From: John C Klensin <[email-masked]>
To: "Peter J. Holzer" <[email-masked]>, [email-masked]
Subject: Re: OT: 5xy - Do we need a "I MEAN IT!" do not retry response code?
Sender: [email-masked]

--On Wednesday, May 02, 2007 12:14 PM +0200 "Peter J. Holzer" 
<[email-masked]> wrote:

>> I didn't catch well what it meant by "hard and soft bounces"
>> and  "BounceBack Messages?"  Does that main a real
>> accept/bounce transaction  or a 5xy response? or both?
>
> I'm not sure either. "hard and soft bounces" could be either
> 5xx and 4xx respectively, or a "soft bounce" could be a
> "couldn't deliver for $n hours, will keep trying" message.

I suspect 5yz and 4yz, although, if  they are looking at NDNs, 
they might even be looking at extended codes.  Or a hard bounce 
might be an SMTP reject and a soft bounce an NDN

> "BounceBack Messages" are probably NDNs. I find this
> interesting, because it means that to use this software (or at
> least this feature) the spammer has to use a genuine
> reverse-path.

Well, the return-path needs to be genuine only for NDNs.  It is 
irrelevant for SMTP-time rejections as long as those are 
considered "hard".  It does suggest that greylisting and 
silently dropping messages are, with regard to this technique, 
pathological: doing either gets you more spam; three clear 
rejections in a row and the noise stops.

     john

Date: Wed, 2 May 2007 12:14:58 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: OT: 5xy - Do we need a "I MEAN IT!" do not retry response code?
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--10jrOL3x2xqLmOsH
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-05-02 05:28:16 -0400, Hector Santos wrote:
>=20
> Here is food for thought:
>=20
> I found this interesting documentation note is a "Professional Spammer"=
=20
> Mass Emailer web site.  Name removed to protect the ignorant.
>=20
>    SPAMMERX provides bounce handling capabilities such that after
>    an e-mail address has bounced a certain number of times,
>    SPAMMERX will cease sending to that address in future e-mail
>    campaigns. SPAMMERX does not distinguish between hard and soft
>    bounces but does categorize between more granular categories
>    of bouncebacks, including out-of-office autoreplies, mailbox
>    full, and user-not-found bounces. Only bounceback messages
>    indicating that the e-mail address is truly invalid are
>    actually counted as true bounces. Out-of-office autoreplies
>    and mailbox-full bounces are never counted as true bounces. By
>    default, your SPAMMERX account is set so that after an e-mail
>    address has bounced three times, SPAMMERX will stop sending
>    future e-mail campaigns to it, even if you try to send to it.
>=20
> Interesting, so this Mass Mailer "Spam" service ignores 4xy/5yx and will=
=20
> try at least 3 times to the same address before it does something with=20
> the address.

My guess is that this is about maintaining the database of mail
addresses, not about sending individual messages.

For the mailing-lists which I maintain manually, I do something very
similar: If I get a bounce, I look at the reason. If its something like
"no such user" I immediately unsubscribe the address from the list; but
it's "mailbox full" or "procmail exited with unknown code 42" or
something like that I assume that the user is going to fix the problem
and unsubscribe them only if the problem persists for a few days.

So that looks like that software is actually trying to clean up the
database with reasonable heuristics.

> I didn't catch well what it meant by "hard and soft bounces" and=20
> "BounceBack Messages?"  Does that main a real accept/bounce transaction=
=20
> or a 5xy response? or both?

I'm not sure either. "hard and soft bounces" could be either 5xx and 4xx
respectively, or a "soft bounce" could be a "couldn't deliver for $n
hours, will keep trying" message.

"BounceBack Messages" are probably NDNs. I find this interesting,
because it means that to use this software (or at least this feature)
the spammer has to use a genuine reverse-path.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--10jrOL3x2xqLmOsH
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--10jrOL3x2xqLmOsH--

Date: Wed, 02 May 2007 05:52:21 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: Somewhat OT: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Sender: [email-masked]

Peter J. Holzer wrote:

> On 2007-05-01 22:58:12 -0400, David F. Skoll wrote:

> Yep. We can't "outlaw" this practice in the RFC, of course, but I think
> that this service isn't very useful any more. If you are doing any
> serious anti-spam filtering you really want the same filter rules on all
> your MXs. You especially don't want an "accept everything" policy on the
> lowest-priority MX because spammers do expect and exploit that.
> 
> (Hmm, is that another recommendation that should go into the RFC if it
> isn't already there? "If there are multiple MX for a domain, they
> SHOULD implement the same policy for accepting or rejecting mail. In
> particular, a lower priority MX should not accept mail that a higher
> priority MX rejects")

Good point, that is what I meant when I wrote in my msg to David S.,

     "The central idea is authorized integration of heterogeneous
      systems so that they all look like one network."

We ran into this issue a number of times with customers when we first 
introduced our strong Email AVS system. They had to decide one way or 
another how to best combine them (or not).  Many were using these 
secondary MX AVS services. Many were using appliance boxes such as 
Barracuda and others, etc.

The initial issue was mainly "What Comes First?"

For some if they used their existing service or box as the first MX, 
then routed/forwarded to their new SMTP server, they couldn't use some 
of its features, such as SPF.  For others, an "Valid Email Address" 
export tool help them for their service or box.  Over time, some 
actually dumped their external service/box.

So yes, I do see a possible statement regarding:

    "persistent SMTP behavior in multiple MX environments."

However, I think you will find some who may advocate the preference=0 
idea or "First MX" as a "greylist concept."   Isn't there an RFC or I-D 
that proposes this Preference=0 method?

--
HLS

Date: Wed, 02 May 2007 05:28:16 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: OT: 5xy - Do we need a "I MEAN IT!" do not retry response code?
Sender: [email-masked]

Here is food for thought:

I found this interesting documentation note is a "Professional Spammer" 
Mass Emailer web site.  Name removed to protect the ignorant.

    SPAMMERX provides bounce handling capabilities such that after
    an e-mail address has bounced a certain number of times,
    SPAMMERX will cease sending to that address in future e-mail
    campaigns. SPAMMERX does not distinguish between hard and soft
    bounces but does categorize between more granular categories
    of bouncebacks, including out-of-office autoreplies, mailbox
    full, and user-not-found bounces. Only bounceback messages
    indicating that the e-mail address is truly invalid are
    actually counted as true bounces. Out-of-office autoreplies
    and mailbox-full bounces are never counted as true bounces. By
    default, your SPAMMERX account is set so that after an e-mail
    address has bounced three times, SPAMMERX will stop sending
    future e-mail campaigns to it, even if you try to send to it.

Interesting, so this Mass Mailer "Spam" service ignores 4xy/5yx and will 
try at least 3 times to the same address before it does something with 
the address.

I didn't catch well what it meant by "hard and soft bounces" and 
"BounceBack Messages?"  Does that main a real accept/bounce transaction 
or a 5xy response? or both?

Gees, no wonder these guys get such a bad rap!  They can actually harm 
their own customers by not RESPECTING rejects!

Anyway, just thought this was interesting.

--
HLS

Date: Wed, 2 May 2007 10:49:35 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--nmemrqcdn5VTmUEE
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-05-01 18:17:06 -0700, SM wrote:
> At 14:13 01-05-2007, Peter J. Holzer wrote:
> >    Many of the problems described above can be avoided by not accepting
> >    a mail in the first place. An SMTP server (especially one acting as
> >    MX) SHOULD make every reasonable efford to determine whether the
> >    message will be delivered to the recipients mailbox and reject the
> >    the message during the SMTP transaction if that is not the case.
>=20
> Yes.  But that may be construed as forbidding bounces=20
> altogether.  This makes existing mail servers non-compliant overnight.

Please note that's "SHOULD" not "MUST". Violating a SHOULD doesn't make
a server non-compliant. However, the implementor should carefully
consider the pros and cons.

Also this paragraph only gives a recommendation to prefer "check during
SMTP transaction" over "accept, and check later". It doesn't say
anything about sending or not sending NDNs for messages which were
already accepted. This is adequately (in the scope of RFC 2821bis - we
could of course write a whole best practice RFC about that topic)
discussed in the paragraphs above.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--nmemrqcdn5VTmUEE
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--nmemrqcdn5VTmUEE--

Date: Wed, 2 May 2007 10:32:24 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: Somewhat OT: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--PmA2V3Z32TCmWXqI
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-05-01 22:58:12 -0400, David F. Skoll wrote:
> Hector Santos wrote:
> >> If both the primary and secondary MX servers are administered by the
> >> same organization or by closely-cooperating organizations, this works
> >> well.  However, there are ISPs that offer "secondary MX service" to
> >> their customers, and they are unlikely (or sometimes unable) to make
> >> special arrangments to synchronize user lists.

Yep. We can't "outlaw" this practice in the RFC, of course, but I think
that this service isn't very useful any more. If you are doing any
serious anti-spam filtering you really want the same filter rules on all
your MXs. You especially don't want an "accept everything" policy on the
lowest-priority MX because spammers do expect and exploit that.

(Hmm, is that another recommendation that should go into the RFC if it
isn't already there? "If there are multiple MX for a domain, they
SHOULD implement the same policy for accepting or rejecting mail. In
particular, a lower priority MX should not accept mail that a higher
priority MX rejects")

> Perhaps an RFC for querying whether or not an e-mail address is
> valid using DNS? :-)
>=20
>   host -t txt some.email.AT.example.com.email-verification-zone.example.c=
om
>=20
> After all, there are already well-established and widely-used mechanisms
> for synchronizing DNS data.  (I skip tedious details like restrictions
> on local-part of e-mail addresses, etc.)

Funny you should mention that. I thought about mentioning that
possibility but left it out because it isn't standardized, so it's on
the same level as rsyncing config files, etc.: Easy to implement within
one organisation, but each organization is likely to do it slightly
differently. An RFC would help, of course, but I'm not sure you can get
all the needed information in there. For example, I would cram
address-specific configurations (e.g., is greylisting enabled) into the
TXT record, but of course that is MTA-specific.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--PmA2V3Z32TCmWXqI
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--PmA2V3Z32TCmWXqI--

Date: Wed, 2 May 2007 10:32:07 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Cc: Frank Ellermann <[email-masked]>
Sender: [email-masked]

Frank Ellermann writes:
> Arnt Gulbrandsen wrote:
>> "foo.com, having received the message, verifies delivery to
>>   [email-masked] is permissible and possible. foo.com now does a DNS 
>>   lookup on xyz.com."
>
> Actually it should verify jones@xyz at the time when it decides to 
> accept the mail, but at this time xyz is down, and so that's not 
> possible.

I've done that by verifying against old data, ie. the secondary accepts 
mail to addresses that were valid a few days earlier and answers 4xx 
for other addresses. Very easy and solves almost all of the problem, 
with negligible side effects.

> The verification business after it has accepted the mail is pointless, 
> it's too late, foo.com can only pray that xyz.com eats its spam.

Good point.

Arnt

Date: Tue, 01 May 2007 21:40:10 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: DNS VRFY (was: Somewhat OT: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3)
Sender: [email-masked]

Hi David,

This reply is off-topic and is not related to the current rfc2821-bis 
discussion.

At 19:58 01-05-2007, David F. Skoll wrote:

>Perhaps an RFC for querying whether or not an e-mail address is
>valid using DNS? :-)
>
>   host -t txt some.email.AT.example.com.email-verification-zone.example.com

This is an interesting idea.  How about:

host -t txt f098474401a124aa5a2738b6353e22da.example.com._vrfy.example.com

where f098474401a124aa5a2738b6353e22da is a MD5 hash of the email 
address.  That's to avoid disclosing the actual email address.

Queries for the zone can be restricted to the DNS server used by the 
secondary MX.  It may not scale well for a site with a lot of email 
addresses.  It should suite the needs of small and medium sites as 
they can use their existing DNS infrastructure to implement it.

Regards,
-sm 

Date: Wed, 2 May 2007 00:09:08 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: "for" clause on Received: header field
Sender: [email-masked]

At 17:34 -0600 on 05/01/2007, Philip Guenther wrote about Re: "for" 
clause on Received: header field:

>On Tue, 1 May 2007, Robert A. Rosenberg wrote:
>...
>>The above quote was mine. I fail to see where there is any massive 
>>disk space or bandwidth usage if implemented correctly. There HAS 
>>to be some ultimate SMTP Server that is delivering the message to 
>>the user's POP/IMAP Mailbox. How hard is it for THAT SMTP Server 
>>(if not the ones that is handing the message off it it) to put the 
>>for clause into the Received Header as it places the cloned copy of 
>>the message into the user's mailbox? The delivery SMTP Server MUST 
>>clone the message at that point anyway (and it takes no extra disk 
>>space or bandwidth for the INCOMING message).
>
>"Black box" mail store that place each message in a separate file 
>(such as Cyrus IMAP) usually use filesystem hardlinks such that 
>there's only one copy of the message on disk for any number of 
>recipients up to the max link-count.  That space optimization 
>obviously can't be used if the messages aren't exact duplicates.

I admit that for THAT implementation my method will not work. I was 
thinking more of the cases where each user had their own dedicated 
mailbox which was carved out of their private space allocation.

Date: Wed, 02 May 2007 00:31:37 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "David F. Skoll" <[email-masked]>
CC: [email-masked]
Subject: OT Brainstorm: Email  Validation among different systems
Sender: [email-masked]

David F. Skoll wrote:
> Hector Santos wrote:
> 
>>> If both the primary and secondary MX servers are administered by the
>>> same organization or by closely-cooperating organizations, this works
>>> well.  However, there are ISPs that offer "secondary MX service" to
>>> their customers, and they are unlikely (or sometimes unable) to make
>>> special arrangments to synchronize user lists.
> 
>> We offer a special utility to our customers who subscribe to these
>> secondary MX services which allows them to export their user email
>> addresses, translations, aliases, etc, which is then sent to the
>> service.
> 
> That's a neat feature, but an ISP that needs to support dozens of
> different SMTP servers and many different ways of maintaining
> user lists is still not going to be happy.  A standard way to do
> this sort of thing would be welcome.

I see your valid point.

My point here is simply that a professional AVS email security service 
bureau will most likely have and offer (if they want the business) a 
"interface" specification such as http://www.virtualconnect.net which 
allows the operator to update their list of valid users via an email 
automated fashion. Here is the VirtualConnect.tpl template for this service:

From: @from@
Date: @rfcdate@
Subject: @domain@ @date1@ @time1@
Message-Id: <@mid@>
To: @to@
X-Note: <insert password>

@body@

where the body is just a list of the valid addresses.  Very simple idea.

> Perhaps an RFC for querying whether or not an e-mail address is
> valid using DNS? :-)
> 
>   host -t txt some.email.AT.example.com.email-verification-zone.example.com

Ideally a good idea, I think there might be DNS overhead/scalability and 
security exposure concerns but if coupled with some client access 
concept it might get consideration.  So why do expect to see the I-D? <g>

> After all, there are already well-established and widely-used mechanisms
> for synchronizing DNS data.  (I skip tedious details like restrictions
> on local-part of e-mail addresses, etc.)

Who would create/maintain the zone?  I would think the service would 
want to minimize call outs and prefer to just get a list from the 
operator, its less overhead and also allows them to import the common 
list from everyone into their own high speed database system.

But they might be an standard in obtaining the "common list."

Of course, there are many ideas to float around related to this idea. 
The central idea is authorized integration of heterogeneous systems so 
that they all look like one network.

For example, an SMTP extension can be written where an AUTH service is 
defined to obtain server-side validation information, i.e., like a list 
of valid users.

It is similar idea to VRFY. One future consideration to justify VRFY 
usage is to make it an AUTHORIZED concept only.  Won't work in the 
general sense, but maybe good for something like this. That is how our 
SMTP server offers VRFY - authorized clients only.

Also given the direction of the world (good or bad), maybe a SOAP or 
REST API will probably get some a lot interest.

Similar to our WCSAP service that we use to validate the envelope 
information. It is called directly from our SMTP but it can also be 
called as a REST-like syntax:

http://www.winserver.com/public/code/html-wcsap?ip=64.26.171.99&cdn=%5b192.168.5.2%5d&from=dfs%40roaringpenguin.com&xid=public&debug=10

The above will attempt to validate your from address.

You can retest this at http://www.winserver.com/testwcsap with the 
proper parameters relative to your workstation.

I can see a standard/common REST API "VRFY" function prototype that 
backup or secondary MXs can use.

--
HLS

Date: Tue, 01 May 2007 22:58:12 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Somewhat OT: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Sender: [email-masked]

Hector Santos wrote:

>> If both the primary and secondary MX servers are administered by the
>> same organization or by closely-cooperating organizations, this works
>> well.  However, there are ISPs that offer "secondary MX service" to
>> their customers, and they are unlikely (or sometimes unable) to make
>> special arrangments to synchronize user lists.

> We offer a special utility to our customers who subscribe to these
> secondary MX services which allows them to export their user email
> addresses, translations, aliases, etc, which is then sent to the
> service.

That's a neat feature, but an ISP that needs to support dozens of
different SMTP servers and many different ways of maintaining
user lists is still not going to be happy.  A standard way to do
this sort of thing would be welcome.

Perhaps an RFC for querying whether or not an e-mail address is
valid using DNS? :-)

  host -t txt some.email.AT.example.com.email-verification-zone.example.com

After all, there are already well-established and widely-used mechanisms
for synchronizing DNS data.  (I skip tedious details like restrictions
on local-part of e-mail addresses, etc.)

Regards,

David.

Date: Tue, 01 May 2007 22:22:52 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Sender: [email-masked]

David F. Skoll wrote:
> Arnt Gulbrandsen wrote:
> 
>> An example, easily done with postfix: A set of good addresses are
>> (ir/)regularly copied from the primary mx, and the secondary replies 4xx
>> for other addresses. Done.
> 
> If both the primary and secondary MX servers are administered by the
> same organization or by closely-cooperating organizations, this works
> well.  However, there are ISPs that offer "secondary MX service" to
> their customers, and they are unlikely (or sometimes unable) to make
> special arrangments to synchronize user lists.

We offer a special utility to our customers who subscribe to these 
secondary MX services which allows them to export their user email 
addresses, translations, aliases, etc, which is then sent to the 
service.  Some of the services even offer automated methods (i.e., like 
sending a CVS file to a special email address). If I recall, some even 
worked by making sure you added their own special "X-Headers" with 
account information. As you can see below, this was not just a simple 
dump utility but had many design considerations:

WcMakeEmail 1.1 - Create list of valid email addresses.
(c) copyright 2002 Santronics Software, Inc.
usage: wcMakeEmailList </new | /update> [options]

See wcMakeEmailList.htm for complete documentation

/new              - generate complete list. new history file
/update           - update mode, compare with history, create
                     difference update list in cvs format.

[options]
/nocvs            - use with /update, create new entire list if changed
/server:computer  - define remote Wildcat! server
/out:filename     - output file (def: ValidEmailAddresses.ulist)
/in:filename      - input file (def: ValidEmailAddresses.hlist)
/quiet            - quiet mode, no screen output
/comment          - add comment lines to output

[Email options]

/email:address    - email output To: address
/to:address       - same as /email
/from:address     - set email From: address
/subj:subject     - set email Subject: line
/tpl:filename     - email template file
/rfc              - template file rfc headers
/test             - test mode: creates email.txt

Before we added our own AVS related features, these Anti-Spam/Virus 
Service Bureaus were common with many of our customers.

Anyway, I think the point is that any real operation that needs this 
type of service quite often do see the secondary MX "interfacing" 
problems pretty quickly and thus take whatever action is necessary or 
that is required.

--
HLS

Date: Tue, 01 May 2007 21:38:12 -0400
From: "David F. Skoll" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Sender: [email-masked]

Arnt Gulbrandsen wrote:

> An example, easily done with postfix: A set of good addresses are
> (ir/)regularly copied from the primary mx, and the secondary replies 4xx
> for other addresses. Done.

If both the primary and secondary MX servers are administered by the
same organization or by closely-cooperating organizations, this works
well.  However, there are ISPs that offer "secondary MX service" to
their customers, and they are unlikely (or sometimes unable) to make
special arrangments to synchronize user lists.

The fact is, we will never entirely eliminate the situation in which
an SMTP server has accepted a message only to find out that it can't
be delivered, and we have to have a reasonable set of guidelines
for what to do.

I believe that SMTP servers SHOULD generate NDNs unless there is a
compelling local policy reason not to (for example, message contains a
known virus or message is strongly believed to be spam.)

Regards,

David.

Date: Tue, 01 May 2007 18:17:06 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Sender: [email-masked]

At 14:13 01-05-2007, Peter J. Holzer wrote:
>that verification can be done even if xyz is down if foo has the
>necessary information.

That's not always possible.  For example, New Orleans is 
flooded.  You set up a MX for the domain until further notice.  foo 
cannot get the necessary information for recipient 
verification.  This was an actual case.

>So if the backup mx syncs frequently with the primary mx it should a
>good idea which addresses are valid and which aren't. Frequently the

That's a best practice.

>Also true, but a backup MX which has a different configuration than the
>primary is worse than none in many (most?) situations. The days where
>"accept everything for domain X and relay it to a lower numbered MX" was
>a useful backup MX configuration are unfortunately over.

Yes.  The main problem is that the backup MX may have a less 
restrictive filtering policy than the lower numbered MX.

>     Many of the problems described above can be avoided by not accepting
>     a mail in the first place. An SMTP server (especially one acting as
>     MX) SHOULD make every reasonable efford to determine whether the
>     message will be delivered to the recipients mailbox and reject the
>     the message during the SMTP transaction if that is not the case.

Yes.  But that may be construed as forbidding bounces 
altogether.  This makes existing mail servers non-compliant overnight.

Regards,
-sm 

From: [email-masked]
Cc: [email-masked]
Date: Tue, 01 May 2007 17:04:11 -0700 (PDT)
Subject: Re: "for" clause on Received: header field
To: Robert A Rosenberg <[email-masked]>
Sender: [email-masked]

> At 18:59 -0700 on 04/30/2007, Ned Freed wrote about Re: "for" clause
> on Received: header field:

> >  > The solution to the matching issue is simple. If you are the ultimate
> >>  receiver domain (ie: Example.com for mail addressed to
> >>  [email-masked]), just clone the message so there is a separate copy
> >>  for each Rcpt-To address and place the appropriate address into the
> >>  for clause of each cloned message - IOW: Act as a SMTP Server that is
> >>  designed to not group multiple Rcpt-To commands into an outgoing
> >>  email message even though the Mail-To's point at the same MX Server.
> >
> >This simple solution only works if you're willing to eat the disk space and/or
> >bandwidth it consumes. When dealing with large local distributions which for
> >some companies can run into tens of thousands of recipients for a single
> >message, this is an absolute and complete nonstarter. Of course you can put a
> >threshold on the maximum recipients before separate copies are disabled, but
> >the cases where for clauses are most useful are those involving large, complex
> >lists.

> The above quote was mine. I fail to see where there is any massive
> disk space or bandwidth usage if implemented correctly. There HAS to
> be some ultimate SMTP Server that is delivering the message to the
> user's POP/IMAP Mailbox. How hard is it for THAT SMTP Server (if not
> the ones that is handing the message off it it) to put the for clause
> into the Received Header as it places the cloned copy of the message
> into the user's mailbox?

This is the second option I mentioned, which is for the IMAP server to
store the common part of the message for all users but to also have
a user-dependent part.

But to answer your question, whether or not it is hard depends on the
store design. In some stores it is simple, in some it is next to impossible,
and for some it adds so much complexity and fragility that it really isn't
an option.

 The delivery SMTP Server MUST clone the
> message at that point anyway (and it takes no extra disk space or
> bandwidth for the INCOMING message).

Not if it is properly designed it doesn't. We routinely end up storing
a single copy of a message with tens of thousands of recipients.

> How far back along the chain of
> SMTP Servers doing hand-off/relay between the MX Server that accepts
> the message and the SMTP Delivery server that places it into the
> mailbox (or redirects the message due to a .forward) you place the
> SMTP Server that does the actual cloning is an implementation
> decision.

You're right in saying it is an implementation decision, but you're making
assumptions about how things have to be implemented that just aren't true
for high-end server architectures, including but not limited to ours.

However, since this has almost nothing to do with the matter at hand I'm
going to stop here rather than get into how we implement all this.

				Ned

Date: Tue, 1 May 2007 17:34:48 -0600
From: Philip Guenther <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>
cc: [email-masked]
Subject: Re: "for" clause on Received: header field
Sender: [email-masked]

On Tue, 1 May 2007, Robert A. Rosenberg wrote:
...
> The above quote was mine. I fail to see where there is any massive disk space 
> or bandwidth usage if implemented correctly. There HAS to be some ultimate 
> SMTP Server that is delivering the message to the user's POP/IMAP Mailbox. 
> How hard is it for THAT SMTP Server (if not the ones that is handing the 
> message off it it) to put the for clause into the Received Header as it 
> places the cloned copy of the message into the user's mailbox? The delivery 
> SMTP Server MUST clone the message at that point anyway (and it takes no 
> extra disk space or bandwidth for the INCOMING message).

"Black box" mail store that place each message in a separate file (such as 
Cyrus IMAP) usually use filesystem hardlinks such that there's only one 
copy of the message on disk for any number of recipients up to the max 
link-count.  That space optimization obviously can't be used if the 
messages aren't exact duplicates.

Philip Guenther

Date: Tue, 1 May 2007 18:48:07 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: "for" clause on Received: header field
Sender: [email-masked]

At 18:59 -0700 on 04/30/2007, Ned Freed wrote about Re: "for" clause 
on Received: header field:

>  > The solution to the matching issue is simple. If you are the ultimate
>>  receiver domain (ie: Example.com for mail addressed to
>>  [email-masked]), just clone the message so there is a separate copy
>>  for each Rcpt-To address and place the appropriate address into the
>>  for clause of each cloned message - IOW: Act as a SMTP Server that is
>>  designed to not group multiple Rcpt-To commands into an outgoing
>>  email message even though the Mail-To's point at the same MX Server.
>
>This simple solution only works if you're willing to eat the disk space and/or
>bandwidth it consumes. When dealing with large local distributions which for
>some companies can run into tens of thousands of recipients for a single
>message, this is an absolute and complete nonstarter. Of course you can put a
>threshold on the maximum recipients before separate copies are disabled, but
>the cases where for clauses are most useful are those involving large, complex
>lists.

The above quote was mine. I fail to see where there is any massive 
disk space or bandwidth usage if implemented correctly. There HAS to 
be some ultimate SMTP Server that is delivering the message to the 
user's POP/IMAP Mailbox. How hard is it for THAT SMTP Server (if not 
the ones that is handing the message off it it) to put the for clause 
into the Received Header as it places the cloned copy of the message 
into the user's mailbox? The delivery SMTP Server MUST clone the 
message at that point anyway (and it takes no extra disk space or 
bandwidth for the INCOMING message). How far back along the chain of 
SMTP Servers doing hand-off/relay between the MX Server that accepts 
the message and the SMTP Delivery server that places it into the 
mailbox (or redirects the message due to a .forward) you place the 
SMTP Server that does the actual cloning is an implementation 
decision.

Date: Wed, 2 May 2007 00:42:28 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Cc: "Peter J. Holzer" <[email-masked]>
Sender: [email-masked]

Peter J. Holzer writes:
> On 2007-05-01 14:03:58 -0400, John C Klensin wrote:
>>  --On Tuesday, May 01, 2007 6:52 PM +0200 Frank Ellermann
>>  <[email-masked]> wrote:
>>  >Actually it should verify jones@xyz at the time when it decides
>>  >to accept the mail, but at this time xyz is down, and so that's
>>  >not possible.
>
> that verification can be done even if xyz is down if foo has the 
> necessary information.

An example, easily done with postfix: A set of good addresses are 
(ir/)regularly copied from the primary mx, and the secondary replies 
4xx for other addresses. Done.

Arnt

Date: Tue, 1 May 2007 23:13:53 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--mP3DRpeJDSE+ciuQ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-05-01 14:03:58 -0400, John C Klensin wrote:
> --On Tuesday, May 01, 2007 6:52 PM +0200 Frank Ellermann=20
> <[email-masked]> wrote:
> >Actually it should verify jones@xyz at the time when it decides
> >to accept the mail, but at this time xyz is down, and so that's
> >not possible.

that verification can be done even if xyz is down if foo has the
necessary information.

> >The verification business after it has accepted
> >the mail is pointless, it's too late, foo.com can only pray
> >that xyz.com eats its spam.
>=20
> It seems to me that this is the key problem.   Even if we=20
> (temporarily) ignore the constraints on what can go into=20
> 2821bis, the "backup MX" function -- a server that is not=20
> expected to receive mail for a particular domain unless the=20
> servers that normally support that domain are down or otherwise=20
> offline from the public network -- is an important, even if (one=20
> hopes) rarely-used, function.  Unlike some other uses of MX=20
> records, that situation poses both a technical and an=20
> administrative/ economic problem.  The first is essentially a=20
> race condition: no matter what measures are taken, if the relay=20
> (standby MX server) cannot establish connectivity to the server=20
> for the target domain in real-time, it has no possible way to=20
> know what addresses are valid there.  The very best it can do is=20
> to guess on the basis of knowledge obtained while there was=20
> connectivity.

True, but the set of valid addresses probably doesn't change much on a
server without connectivity.

So if the backup mx syncs frequently with the primary mx it should a
good idea which addresses are valid and which aren't. Frequently the
list of valid addresses isn't maintained on the primary MX anyway but on
a third system (customer database, central LDAP server, whatever), so
you can use the same mechanism to get updates to both MXs.

Some scenarios make that more difficult (e.g. some keyed-address
schemes), but these are probably exactly the scenarios where you really
want the secondary MX to be in sync with the primary.

> The administrative/economic one is that there are disincentives=20
> for the administrators of the relay machine and those of the=20
> target machine to try to keep up-to-date user lists on the relay=20
> machine.  It requires more coordination and more resources than=20
> simply providing a backup MX function, and requires them on a=20
> regular and continuing basis, even when the target machine is=20
> almost always available.

Also true, but a backup MX which has a different configuration than the
primary is worse than none in many (most?) situations. The days where
"accept everything for domain X and relay it to a lower numbered MX" was
a useful backup MX configuration are unfortunately over.

> It also raises some privacy issues.=20
> In addition, we don't have any established / standardized way to=20
> transfer and store the needed information (one could imagine a=20
> DNS-like "poll for updates" protocol, but, at least to my=20
> knowledge we don't have one of those.

Replicated LDAP servers is probably as close to "standardized" as you're
going to get. That stuff is quite implementation- and site-specific (for
example, we store the config in a subversion repository and both MXs do
a "svn update" every few minutes)

> While there is less of it today than there was a decade ago, we=20
> also have a situation, especially with some rural areas and=20
> developing countries, where an MX record points to an=20
> always-online server that acts as a temporary repository or=20
> gateway to the distribution server for network that is only=20
> intermittently connected to the rest of the Internet (i.e., the=20
> link between the accessible server and the delivery server is=20
> only occasionally available and may not be accessible to the=20
> public Internet at all).

Yup. That is a case which should not be ignored.

> For example, to say that a relay should make reasonable best efforts
> (however those might or might not be defined) to verify the address
> before accepting the mail but, if it cannot, may go ahead and try to
> deliver it, dealing with the NDN case if necessary would seem much
> more plausible to me than trying to ban NDNs ... a ban that, if taken
> seriously, would take a large number of areas out of even email
> connectivity with the Internet.

I fully agree with this. I think that the RFC should encourage
implementors to check as much as possible at the MX and try to reject
rather than bounce, but acknowledge that this isn't always practical.

Section 6.2 already suggests that rejects are preferrable to bounces (If
they cannot be delivered, and cannot be rejected by the SMTP server
during the SMTP transaction, they should be "bounced") but that is a bit
weak. So maybe an extra sentence or two here would help. I just can't
find a good place. Maybe just a paragraph at the end:

    Many of the problems described above can be avoided by not accepting
    a mail in the first place. An SMTP server (especially one acting as
    MX) SHOULD make every reasonable efford to determine whether the
    message will be delivered to the recipients mailbox and reject the
    the message during the SMTP transaction if that is not the case.

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--mP3DRpeJDSE+ciuQ
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--mP3DRpeJDSE+ciuQ--

From: [email-masked]
Cc: SMTP Interest Group <[email-masked]>
Date: Tue, 01 May 2007 13:28:41 -0700 (PDT)
Subject: Re: Recap Issues 0b/21/25
To: Tony Hansen <[email-masked]>
Sender: [email-masked]

> I've been rereading all of the discussions around issues 0b/21/25. Some
> of the discussion wants to require verification of the sender before
> ever sending an non-delivery notification.

> While this does reflect what some implementations are doing and is the
> effort of some recent standards and non-standards efforts, it is not
> common deployment. So I'm going to make a ruling that requiring such
> validation goes too far in the direction of a "new behavior" requirement.

> However, I'd like to follow the lead of the 3463 reference found in
> section 2.3.11, and suggest that some wording be added, probably to
> section 3.6, along these lines:

>     This specification does not deal with the verification of return
>     paths for use in delivery notifications. Recent work, such as
>     [add REFERENCES here], has been done in providing ways to
>     ascertain that an address is valid or belongs to the person who
>     actually sent the message. A server MAY attempt to verify the return
>     path before using its address for delivery notifications, but
>     methods of doing so are not defined here nor is any particular
>     method recommended at this time.

> Thoughts on an addition like this? I know it doesn't go far enough for
> some people, and probably goes too far for some other people. So I'm
> offering it as a possible way forward.

This seems quite reasonable to me.

				Ned

Date: Tue, 01 May 2007 14:03:58 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Sender: [email-masked]

--On Tuesday, May 01, 2007 6:52 PM +0200 Frank Ellermann 
<[email-masked]> wrote:

>> "foo.com, having received the message, verifies delivery to
>>  [email-masked] is permissible and possible. foo.com now does
>>  a DNS lookup on xyz.com."
>
> Actually it should verify jones@xyz at the time when it decides
> to accept the mail, but at this time xyz is down, and so that's
> not possible.  The verification business after it has accepted
> the mail is pointless, it's too late, foo.com can only pray
> that xyz.com eats its spam.

It seems to me that this is the key problem.   Even if we 
(temporarily) ignore the constraints on what can go into 
2821bis, the "backup MX" function -- a server that is not 
expected to receive mail for a particular domain unless the 
servers that normally support that domain are down or otherwise 
offline from the public network -- is an important, even if (one 
hopes) rarely-used, function.  Unlike some other uses of MX 
records, that situation poses both a technical and an 
administrative/ economic problem.  The first is essentially a 
race condition: no matter what measures are taken, if the relay 
(standby MX server) cannot establish connectivity to the server 
for the target domain in real-time, it has no possible way to 
know what addresses are valid there.  The very best it can do is 
to guess on the basis of knowledge obtained while there was 
connectivity.

The administrative/economic one is that there are disincentives 
for the administrators of the relay machine and those of the 
target machine to try to keep up-to-date user lists on the relay 
machine.  It requires more coordination and more resources than 
simply providing a backup MX function, and requires them on a 
regular and continuing basis, even when the target machine is 
almost always available.  It also raises some privacy issues. 
In addition, we don't have any established / standardized way to 
transfer and store the needed information (one could imagine a 
DNS-like "poll for updates" protocol, but, at least to my 
knowledge we don't have one of those.

While there is less of it today than there was a decade ago, we 
also have a situation, especially with some rural areas and 
developing countries, where an MX record points to an 
always-online server that acts as a temporary repository or 
gateway to the distribution server for network that is only 
intermittently connected to the rest of the Internet (i.e., the 
link between the accessible server and the delivery server is 
only occasionally available and may not be accessible to the 
public Internet at all).  The same issues arise as with the 
cases above: having accurate information on the relay about 
deliverable mailboxes is very problematic.  Of course, similar 
or identical issues arise with various long-delay networking 
scenarios.

Speaking personally, and again temporarily ignoring the 
constraints on the 2821bis effort, I'd be much more sympathetic 
to this class of changes that are intended to restrict use of 
NDNs if there were some clear analysis, and possibly tools (the 
privacy issues aside), for dealing with these important cases. 
I'd also be more sympathetic if the proposals were a bit more 
nuanced.  For example, to say that a relay should make 
reasonable best efforts (however those might or might not be 
defined) to verify the address before accepting the mail but, if 
it cannot, may go ahead and try to deliver it, dealing with the 
NDN case if necessary would seem much more plausible to me than 
trying to ban NDNs ... a ban that, if taken seriously, would 
take a large number of areas out of even email connectivity with 
the Internet.

        john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Date:  Tue, 01 May 2007 18:52:02 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

Arnt Gulbrandsen wrote:

> I'd try to skirt the issue

Dunno, noting that a backup MX is in a situation exactly at the
time when it's really needed makes sense.  But maybe an example
in an appendix isn't the best place for this point (?)

> "foo.com, having received the message, verifies delivery to
>  [email-masked] is permissible and possible. foo.com now does
>  a DNS lookup on xyz.com."

Actually it should verify jones@xyz at the time when it decides
to accept the mail, but at this time xyz is down, and so that's
not possible.  The verification business after it has accepted
the mail is pointless, it's too late, foo.com can only pray 
that xyz.com eats its spam.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: rfc2821bis-03 Issue 35: remove source routes from example D.3
Date:  Tue, 01 May 2007 18:41:52 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

John C Klensin wrote:
 
> (2) Note that a side-effect of the famous issue 25 (about NDN
> "bounces") is that, if the relay (foo.com) cannot verify either
> the forward-pointing address ([email-masked]) or the
> backward-pointing one ([email-masked]) prior to sending back the
> 250 response to the RCPT command, it is in big trouble.   If we
> apply this fix (or even if we don't), does it immediately go
> into the Issue 25 list?

Yes, bar.com has to be very reluctant to accept mail for xyz.com
while xyz.com is down.  It can be more confident if [email-masked]
is "plausible" (known good sender bar.com, SPF PASS, or similar),
or if [email-masked] used to exist.  

If you don't apply the fix it's a source routing example for the
source routing appendix, where the "THE issue" doesn't exist, or
what do you have in mind ?

Frank

From: [email-masked]
Cc: [email-masked]
Date: Tue, 01 May 2007 09:10:52 -0700 (PDT)
Subject: Re: Recap Issues 0b/21/25
To: David F Skoll <[email-masked]>
Sender: [email-masked]

> John Leslie wrote:

> >    I claim this data supports my contention that folks don't _use_
> > the NDNs that they get.

> I don't agree.  I do agree that almost nobody uses or understands
> the content of the NDNs they get.

Users apparently don't, despite considerable efforts to make the content more
comprehensible. (One of the product areas we are often to enhance is  the
ability to return both DSNs and MDNs in an "appropriate" language. We routinely
see requests for additional language support - if memory serves Gallegan was a
recent one - as well as for improved ways to select the right language to use
out of the many we have available.)

> However, I would guess that anywhere
> from 2% to 10% of people use the *fact* that they received an NDN to take
> further action.  We deal with many unsophisticated users, and although they
> have no clue what an NDN means, the receipt of one prompts them to contact
> their helpdesk person or the intended recipient (if they can figure out
> that much...)

And the helpdesk can ask to see the NDN, especially if it is one generated by
the user's own system. Given the vagarities of NDN formats in use it is too
much to expect even trained helpdesk personnel to decode arbitrary ones, but
teaching them what to look for in their local ones seems reasonable.

I don't deal directly with frontline support groups, but when frontline can't
figure it out and backline folks look and think it may indicate a problem with
the local mail system and contact the vendor, we frequently ask for, and often
are able to get, the entire NDN at issue in short order.

> >    Getting back to the question, "properly report" involves, at the
> > minimum, logging the incident so that it's _possible_ to reconstruct
> > what happened. I'd like to suggest forwarding those log entries to
> > the client you received the message from, but I suspect that exceeds
> > our charter.

> I think it should read something like this:

> 	In either case, a formal handoff of responsibility for
> 	the message occurs: the protocol requires that a server
> 	MUST accept responsibility for either delivering a
> 	message or properly reporting the failure to do so.

> 	A proper failure report SHOULD consist of a notification sent
> 	to the Return-Path of the original message.  Where this is not
> 	possible (because of a blank Return-Path) or not desirable
> 	(because the Return-Path is untrusted for local policy
> 	reasons), the server MUST record enough information for a
> 	human operator to reconstruct the nature of the delivery
> 	failure, the original Return-Path and the addresses of the
> 	original recipients.

> I fear, though, that may exceed the charter as you say.

I like this text as well but I'll defer to the powers-that-be to first
determine if it is in scope.

				Ned

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Recap: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Date:  Tue, 01 May 2007 18:07:36 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 51
Sender: [email-masked]

Hector Santos wrote:
 
>     When a successful transaction occurs at a SMTP-server (a message is
>     accepted by the server), the protocol requires that a server MUST
>     begin the process of delivering the message or begin the process
>     of sending a failure notification as described in section 6.1
>     "Reliable Delivery and Replies by Email."

That's likely not what you mean.  An SMTP-server accepting a mail
for the purpose of sending a failure report is broken, it should
have rejected it.  With that I get:  "MUST begin the process of
delivering the message and, if that fails, send a non-delivery
report as described" etc.  This "process of delivering" sounds a
bit awkward, but I've no better idea at the moment.

>     Please keep in mind that local site policies is the exception
>     to the SMTP protocol technical server requirement for mail
>     delivery/failure notification.  Please refer to section 7.9
>     regarding the scope of operation of SMTP Servers.

This "local site policy" business is very unclear.  If your local
site consists of an MX doing its duty and a final delivery mailer
happily rejecting anything it doesn't like the setup is dubious,
even if "accept + bounce" happens to be your "local site policy".

> Think in terms of the "new developer" when reads this new standard
> and begins to code his SMTP program, you want him to design the
> model based on this technical Delivery/Notification principle that
> is essentially burned into our society.

Especially for this new developer it's important to understand that
his code might be not used in isolation for a "one server does it
all, talk to stangers up to final delivery" scenario.  The duties
are different at the border or behind it.  Once a mail has been
accepted the top priority is "try to deliver", reject or bounce is
no issue anymore, it's too late, both result in a bounce.

At the border the top priority is to decide "should I accept this",
ideally it's the point of no return.  Some parts of the code for a 
border MTA aren't used behind it, you're not doing "CBV" etc. if
you get mail from your own MX.  The opposite also holds, the code
for final delivery isn't needed for an MX not doing final delivery.

>     7.9.  Scope of Operation of SMTP Servers

7.9 as is is much clearer, in essence it says that there SHOULD be
code to get the crucial decision at the border right.  That's no
foggy "site policy", it's a consequence of the SMTP architecture.

Frank

Date: Tue, 01 May 2007 01:27:26 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

At 07:39 30-04-2007, John Leslie wrote:
>    First, I'd like to violate one of the rules here, by including some
>actual data ;^(
>
>    We run a number of mail-servers here, several of which apply the
>same anti-spam rule and return a 5xy error including a reference to
>a jlc.net webpage. I collected one week's data for that one rule:
>
>     5xy responses sent: 368,074
>      webpage accesses:        2
>
>    (Actually, those two were from the same IP address, less than
>five minutes apart.)
>
>    I claim this data supports my contention that folks don't _use_
>the NDNs that they get. (Yes, I know, the clients we gave that 5xy
>to may not have delivered the actual text in the NDN they generated.)

That is if we assume that these 5xy responses generated NDNs at the 
sender's end.  Compromised/virused-infected hosts don't do 
that.  Most of the people may not be bothered with the NDNs as they 
don't understand it.  One of these senders found it worth it to go to 
the webpage to find out more.

>    I believe any data we could collect involving "typical" users would
>show the same trend: 99.9% of NDNs are not read. (What folks _do_,
>in my experience, is call the intended recipient and complain, "your
>spam filter is broken" -- if they do anything at all, that is...)

Yes, that would be the trend for the typical user.  I don't have any 
conclusive statistics to show that.

It's not always possible to call the intended recipient.  That's 
where the NDN may come in handy as the sender can forward that to 
their support desk for help.  If we want to achieve that with log 
files, we would have to process the information in near-real time.

>    Sending an NDN which has only one chance in a million of conveying
>the right information shouldn't be touted as "properly reporting".

Yes.

Regards,
-sm 

From: [email-masked]
Cc: [email-masked]
Date: Mon, 30 Apr 2007 18:59:00 -0700 (PDT)
Subject: Re: "for" clause on Received: header field
To: Robert A Rosenberg <[email-masked]>
Sender: [email-masked]

> At 09:45 -0700 on 04/30/2007, [email-masked] wrote about
> Re: "for" clause on Received: header field:

> >The problem I see with the matching approach, aside from the obious overhead,
> >is while you can assume that a successful match to, say, a To: field means it
> >is OK to expose that recipient address in a for clause, not finding
> >any matches
> >doesn't mean the address has bcc semantics. So you end up presenting
> >the subset
> >of the active recipient list that happen not to have been transformed by
> >forwarding/routing activites in ways that prevent matching. The result makes
> >for clause disppear in prcecisely the cases where it is most useful for
> >debugging purposes: When transformation of the recipient addresses by
> >forwarding/routing has led to a mismatch between what's in the header and
> >what's in the envelope.

> The solution to the matching issue is simple. If you are the ultimate
> receiver domain (ie: Example.com for mail addressed to
> [email-masked]), just clone the message so there is a separate copy
> for each Rcpt-To address and place the appropriate address into the
> for clause of each cloned message - IOW: Act as a SMTP Server that is
> designed to not group multiple Rcpt-To commands into an outgoing
> email message even though the Mail-To's point at the same MX Server.

This simple solution only works if you're willing to eat the disk space and/or
bandwidth it consumes. When dealing with large local distributions which for
some companies can run into tens of thousands of recipients for a single
message, this is an absolute and complete nonstarter. Of course you can put a
threshold on the maximum recipients before separate copies are disabled, but
the cases where for clauses are most useful are those involving large, complex
lists.

A more complex solution is to store separate headers (or even parts of headers)
for each copy but share the message body.

You can get even trickier and look for common message bodies or parts between
different messages and if you find them eliminate the duplicate.

					Ned

Date: Tue, 01 May 2007 14:27:12 +0100
From: Alexey Melnikov <[email-masked]>
To: Tony Hansen <[email-masked]>
CC: SMTP Interest Group <[email-masked]>
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

Tony Hansen wrote:

>I've been rereading all of the discussions around issues 0b/21/25. Some
>of the discussion wants to require verification of the sender before
>ever sending an non-delivery notification.
>
>While this does reflect what some implementations are doing and is the
>effort of some recent standards and non-standards efforts, it is not
>common deployment. So I'm going to make a ruling that requiring such
>validation goes too far in the direction of a "new behavior" requirement.
>
>However, I'd like to follow the lead of the 3463 reference found in
>section 2.3.11, and suggest that some wording be added, probably to
>section 3.6, along these lines:
>
>    This specification does not deal with the verification of return
>    paths for use in delivery notifications. Recent work, such as
>    [add REFERENCES here], has been done in providing ways to
>    ascertain that an address is valid or belongs to the person who
>    actually sent the message. A server MAY attempt to verify the return
>    path before using its address for delivery notifications, but
>    methods of doing so are not defined here nor is any particular
>    method recommended at this time.
>  
>
The text looks good to me.

Date: Tue, 01 May 2007 08:30:08 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: Retries - may be related to closed issue 12  4yz behavior
Sender: [email-masked]

Peter J. Holzer wrote:
> On 2007-05-01 04:39:38 -0400, Hector Santos wrote:

>> I accept the idea that the client may not reissue the same ENV 
>> information, but that is by far *not* the normal practice in retry 
>> logics.   Google would be the first that I know of,
> 
> Yahoo Groups did this 3 years ago. As far as I can tell they have
> changed their practice (I can only speculate that the growing popularity
> of greylisting caused too many failures).

Yes, I pretty sure they did. But its also got some aggressive:

     "are you still subscribe? click here to begin receiving
      the mail again."

confirmation thing, that seems to be based on the number of errors it 
sees over time, but the mail still does come in inbetween these random 
"restart" messages.  Doesn't appear to be your typical subscription 
reminder.  Its odd, and I didn't bother to analyze it.

--
HLS

Date: Tue, 1 May 2007 12:30:05 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: Retries - may be related to closed issue 12  4yz behavior
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--2Z2K0IlrPCVsbNpk
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2007-05-01 04:39:38 -0400, Hector Santos wrote:
> SM wrote:
> >>Greylisted or not, if a server is temporarily rejecting an email=20
> >>transaction for typical normal (4yz) reasons as indicated in the=20
> >>example reply codes, I believer server implementations is basing this=
=20
> >>idea that the retry attempts will be made again with the same=20
> >>information.
> >
> >No, it's greylisting which is based on that idea and not the SMTP=20
> >implementation.
>=20
> Well, I will argue that the successful growth of greylisting systems is=
=20
> based on the fundamental SMTP 4yz retry principles. It that wasn't the=20
> case, greylisting would not be a consideration.
>=20
> I accept the idea that the client may not reissue the same ENV=20
> information, but that is by far *not* the normal practice in retry=20
> logics.   Google would be the first that I know of,

Yahoo Groups did this 3 years ago. As far as I can tell they have
changed their practice (I can only speculate that the growing popularity
of greylisting caused too many failures).

	hp

--=20
   _  | Peter J. Holzer    | I know I'd be respectful of a pirate=20
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | [email-masked]         |
__/   | http://www.hjp.at/ |	-- Sam in "Freefall"

--2Z2K0IlrPCVsbNpk
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--2Z2K0IlrPCVsbNpk--

Date: Tue, 01 May 2007 06:06:37 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

Frank Ellermann wrote:
> SM wrote:
> 
>> We should also consider the legacy systems out there. A complete
>> reversal to reject, no bounces, makes the model unreliable.
> 
> +1  We're not going to copy the SpamCop FAQ wholesale into 2821bis.
> ( Nevertheless I've invited the SpamCop newsgroup readers to chime
>   in, some have rather radical ideas compared with what we say. :-)

450 I think we should avoid specifics of how mail is rejected.

IMO, at a minimum, SMTP compliance should be the only SMTP reason for 
rejection.  This is already implied as possible with strict SMTP site 
considerations.

Outside of that, it is policy driven and I think, at least for me, why I 
endorse the MUST for the delivery/failure notification behavior of SMTP 
is a separate issue than the how a POLICY can alter that.

The same idea is found in POP3 operations and the specs specifically 
says that SITE POLICY rules over user POP3 software mail retention 
features and POP3 commands to keep/delete mail.

In other words, even though a MUA says

      [X] Keep mail at server for _7__ days
      [X] Delete mail after _30_ days
          [X] Or until I delete it offline.

It has NO control over that the server site policy says about mail 
storage, retention or deletion.  Absolutely none, whatsoever, and the 
POP3 specs make that very clear, and at least the Outlook MUA F1 Help 
makes very clear.

The point?

The POP3 still defines the commands and defines what is the expected 
technical behavior with the commands.  Same with SMTP with what is 
expected with 250/450/550.

But site policy can change all that so this is where the responsibility 
lies - with the site.

> I think CBV isn't specified in any RFC, and so we can't add a
> reference to it.  Similarly there's no RFC for grey-listing,
> SRS, SES, CSV, or BATV.  Referencing RFC 4409 chapter 6.1 should
> be okay.  For 4408 I'm not "neutral" (4409-1=4408 is nice :-)
> Maybe one example 4409 6.1 is enough.

550- I don't think we should have any references to any validation
550  concept in 2821bis. Maybe for 3821, perhaps? :-)

I think, IMO, the best we can do is at this point is to convince John, 
Tony, that "responsible" VALIDATION is important and since it is a 
reality in practice, to help clarify the specifics for expected SMTP 
behavior to help keep a high reliability of the framework.

I personally think most of that is done, maybe too spread out and not 
consolidated enough as I would personally prefer, but I think we are 
almost there - for this draft at least.

--
HLS

Date: Tue, 01 May 2007 05:33:27 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: Recap: Re: rfc2821bis-03 Issue 32: "MUST take responsibility"
Sender: [email-masked]

John C Klensin wrote:
> 
> 
> --On Sunday, 29 April, 2007 23:24 -0400 Tony Hansen
> <[email-masked]> wrote:
> 
>> The consensus on the basic issue of changing the wording of
>> this paragraph to include the word MUST:
>>
>> 	The responsibility of an SMTP client is to transfer mail
>> 	messages to one or more SMTP servers, or report its
>> 	failure to do so.
>>
>> 	In either case, a formal handoff of responsibility for
>> 	the message occurs: the protocol requires that a server
>> 	MUST accept responsibility for either delivering a
>> 	message or properly reporting the failure to do so.
>>
>> is positive.
>>
>> However, several people raised issues with some ambiguities in
>> the text. I found Dave Crocker's description of the
>> ambiguities to be the most succinct. These ambiguities do
>> exist whether the MUST is there or not, so they pass my "chair
>> filter", so should probably be dealt with.
> 
> Text would be welcome.
>     john

I'll take a stab at it.

First, I think the first paragraph is fine.

The second needs to be reworded to separate the protocol and policy
consideration. The 2nd paragraph is within the description of the Basic 
SMTP design structure which implies the automation, the expected flow 
and programmability of this model. So you have a technical description 
of the expected flow and that is how I believe the paragraph should be 
restructured:

This is just a stab at it, changes, mods are more than welcome:

    In other words, message transfer can occur in a single connection
    between the original SMTP-sender and the final SMTP-recipient, or can
    occur in a series of hops through intermediary systems.

    At any point in the transport topology, the SMTP server may
    issue 4yx/5yz response based on site policy decisions. In such
    a case, the SMTP-client MUST follow the requirements as
    stated in section 4.2.1. "Reply Code Severities and Theory."

    When a successful transaction occurs at a SMTP-server (a message is
    accepted by the server), the protocol requires that a server MUST
    begin the process of delivering the message or begin the process
    of sending a failure notification as described in section 6.1
    "Reliable Delivery and Replies by Email."

    Please keep in mind that local site policies is the exception
    to the SMTP protocol technical server requirement for mail
    delivery/failure notification.  Please refer to section 7.9
    regarding the scope of operation of SMTP Servers.

Think in terms of the "new developer" when reads this new standard and 
begins to code his SMTP program, you want him to design the model based 
on this technical Delivery/Notification principle that is essentially 
burned into our society.

But he also needs to be aware that he SHOULD offer operations the policy 
driven options to define his local site policies - a concept that is 
also burned into our society.

So its a separation of the technical protocol vs local policy 
considerations.  Both acceptable practices.

Finally, I would personally also consider the possible rewording of 7.9 
where we inject the concept of "responsibility" into the 1st or 2nd 
sentence of "Scope of Operations."  I personally would not begin this 
section with a theme that it is a "normal consideration" to reject mail.

I would flip it around like so, again, just winging it, 250, 450 and 550 
responses are welcome :-)

    7.9.  Scope of Operation of SMTP Servers

    Cooperation among sites and installations makes the
    Internet electronic mail framework possible.

    When accepting a message, one of the important site operational
    responsibilities is to properly deliver messages to its
    intended target recipients or to properly report the
    non-delivery failures to the reverse-path sender.

    However, it should be noted, it is a well-established principle
    that an SMTP server may refuse to accept mail for any operational
    or technical reason that makes sense to the site providing
    the server.   The responsibility lies with the site for the decision
    to refuse mail. If sites take excessive advantage of the right
    to reject traffic, the ubiquity of email availability (one of
    the strengths of the Internet) will be threatened; considerable
    care should be taken and balance maintained if a site decides
    to be selective about the traffic it will accept  and process.

--
HLS

Date: Tue, 01 May 2007 04:39:38 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SM <[email-masked]>
CC: [email-masked]
Subject: Re: Retries - may be related to closed issue 12  4yz behavior
Sender: [email-masked]

SM wrote:

> There was a time when email was a collaborative effort where we do our 
> best to deliver the mail.  

IMO, I don't think it really has not changed.  Most system do follow 
this principle.  We have not changed our product model to alter that 
fundamental principle and I will go on to further say laws are written 
on that principle as well.  Our product is been ethically engineered and 
molded around that fundamental "user expectation" principle.

> Nowadays it's more about whether the message 
> is more important to the sender or the receiver.  

Thats two parts:

- Mail content interpretation which I would prefer all SMTP software 
writers still out of and let admin deal with that.

- And SMTP level compliance, which is something we concentrated on while 
still adhering to the above fundamental "User Expectation" principle.

By far, but all industry measures, the latter is where the majority of 
the exploitation has taken place to provide the threat entry points to 
given sysops and users the mail content fits.

>> Greylisted or not, if a server is temporarily rejecting an email 
>> transaction for typical normal (4yz) reasons as indicated in the 
>> example reply codes, I believer server implementations is basing this 
>> idea that the retry attempts will be made again with the same 
>> information.
> 
> No, it's greylisting which is based on that idea and not the SMTP 
> implementation.

Well, I will argue that the successful growth of greylisting systems is 
based on the fundamental SMTP 4yz retry principles. It that wasn't the 
case, greylisting would not be a consideration.

I accept the idea that the client may not reissue the same ENV 
information, but that is by far *not* the normal practice in retry 
logics.   Google would be the first that I know of, and I feel pretty 
sure that if Google is made aware of this greylist issue with their 
email validations logic, they would more than likely change it, then 
not, given the growth of greylisting systems.

In any case, if you feel that SMTP systems *should not* expect the same 
information when issuing a 4yz, then it SHOULD be mentioned in the 
specs.  Currently that is not what is implied.  Of course, it all 
depends on the where the 4yx is used.

Thanks for your comments.

--
HLS

Date: Tue, 01 May 2007 00:50:08 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Retries - may be related to closed issue 12  4yz behavior
Sender: [email-masked]

Hi Hector,
At 01:34 30-04-2007, Hector Santos wrote:
>I would like to hear comments on the idea of retries using the same 
>ENV information, in particular the same reverse-path?

The reverse-path on a retry may be different.

>I think the above has an implied consideration that suggest when the 
>server issues the 4yz response, the error is only temporary and thus 
>the client may issue the request again with the idea the parameters 
>are the same.

The client did retry again to deliver the message.

>Why do I bring this up?
>
>Well, I seen a SMTP client from Google that is getting greylisted 
>with a 451 error but it is retrying (in new sessions) with a 
>different MAIL FROM (reverse-path).

You may see the same behavior with SMTP clients using VERP.

>In the case of Google, today, I was helping my wife set up a blog at 
>Google's blogger.com and it was doing an email verification.  It 
>wasn't coming in so I checked the logs.

There was a time when email was a collaborative effort where we do 
our best to deliver the mail.  Nowadays it's more about whether the 
message is more important to the sender or the receiver.  In the 
above case, the receiver may place more value upon the message and 
the receiving end may have to customize the site policy accordingly.

>I believe many Greylist implementations simply reject at the RCPT 
>TO, and do not collect the body.

Yes.

>Greylisted or not, if a server is temporarily rejecting an email 
>transaction for typical normal (4yz) reasons as indicated in the 
>example reply codes, I believer server implementations is basing 
>this idea that the retry attempts will be made again with the same information.

No, it's greylisting which is based on that idea and not the SMTP 
implementation.

Regards,
-sm 

Date: Tue, 01 May 2007 00:06:31 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Recap Issues 0b/21/25
Sender: [email-masked]

At 20:12 30-04-2007, Frank Ellermann wrote:
>Yes, there's a similar situation in NetNews, if you go too far in
>attempts to avoid "dupes" (articles arriving more than once) you
>get "nopes" (articles arriving less than once), and for obvious
>reasons I always considered "nopes" as worse than "dupes".

Then you understand the problem.

>The border is defined by query=mx, and among the mailers acting
>as MX the first getting mail from another mailer is "the border".

The MX is not always the "border".

>That's what I think how "border" could be defined, ignoring some
>corner cases like "no MX" or all definitions of "local mail" if
>it touches an MX.

I'm not thinking of "no MX" here.  It may not always be possible to 
do recipient validation at "the border" you mentioned.  But we can 
encourage people to do the policy filtering there and reject if they 
see fit.  This in itself may decrease the number of bounces significantly.

>I think CBV isn't specified in any RFC, and so we can't add a
>reference to it.  Similarly there's no RFC for grey-listing,
>SRS, SES, CSV, or BATV.  Referencing RFC 4409 chapter 6.1 should
>be okay.  For 4408 I'm not "neutral" (4409-1=4408 is nice :-)
>Maybe one example 4409 6.1 is enough.

CBV is easily turned on in one well-known MTA and people will flip 
that switch if the RFC tells them to go and verify the sender's 
address.  4408 may be the lesser evil then. :-)  RFC 4409 6.1 solves 
part of the problem.  The major part of the problem lies at the receiver's end.

>That wording doesn't consider that this determination can be also
>as simple as one DNS lookup (or rather between 1 and 11 lookups).

It may be difficult to find a consensus for these DNS lookups.

>I'm not yet happy with the statement before your quote:
>
>    address, although the history of the network is that users are
>    typically better served by delivering any message that can be
>    delivered.  Reliably determining that a return address is invalid can
>
>This IS history.  It should say "the history of the network was
>that users were typically better served by delivering any message
>that could be delivered."  Past tense.

That may be history but we can always learn from it.  Recently, a 
large email provider was sending all mail from a particular network 
into a blackhole.  I doubt that we can call that "users being better served".

I read the last sentence you quoted as "better served by delivering 
any message that could be delivered subject to site 
policy".  Sometimes it's better to err on the cautious side or else 
we end up with the site policy described above.

>While I wrote that I've started my REXX excuse for procmail, and
>it will now delete the next about 950 of 1200 pending mails that
>arrived in the last hour.  Today users are typically better served
>by rejecting suspicious messages at the border, otherwise they end
>up in obscure blackholes.

Yes, the users would have been better served if the messages were rejected.

Regards,
-sm