Subject: Re: the place(s) for smtp discussion?
From: Frank Ellermann <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Date: Tue, 26 Jul 2005 08:15:02 +0200
List-Id: <ietf-smtp.imc.org>

Pekka Savola wrote:
 
> I guess [email-masked] should be listed on
> https://datatracker.ietf.org/public/nwg_list.cgi

I've submitted it, let's see what Scott and Paul think, bye

Date: Mon, 25 Jul 2005 11:07:36 +0300 (EEST)
From: Pekka Savola <[email-masked]>
To: [email-masked], [email-masked]
cc: [email-masked]
Subject: the place(s) for smtp discussion?
Sender: [email-masked]

Hi,

Following up on a very old thread, because a friend asked me where to 
look for the latest work on SMTP and I had to scratch my head a bit as 
an appropriate mailing lists couldn't be found anywhere on the IETF 
web pages..

I guess [email-masked] should be listed on 
https://datatracker.ietf.org/public/nwg_list.cgi , if it's used as a 
major venue for discussion (and IETF IPR rules probably apply) ?

The non-wg list currently only includes [email-masked]..

On Wed, 15 Jun 2005, Keith Moore wrote:
[... about the spamops document ...]
> I recommend that followups be sent to the [email-masked] list, as this 
> seems to be the most appropriate place for the discussion.
>
> Keith
>

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: draft-klensin-rfc2821bis-00.txt structural and textual comments
Date: Tue, 19 Jul 2005 18:45:34 -0400
Cc: "Robert A. Rosenberg" <[email-masked]>
Sender: [email-masked]

On Thu July 14 2005 09:00, Robert A. Rosenberg wrote:
> 
> At 02:33 -0400 on 07/14/2005, [email-masked] wrote about Re: 
> draft-klensin-rfc2821bis-00.txt structural and textual :
> 
> >On Thu, 14 Jul 2005 00:58:32 EDT, "Robert A. Rosenberg" said:
> >
> >>  being everything to the right of the @-sign). The Domain part has a
> >>  host if there are more levels than the number defined for the WhoIs
> >>  registration.
> >
> >Are you sure that basing it off the whois is the proper referent for this?
> 
> Yes. I am defining "Domain" based on ownership and control of that 
> level (and those to the left of it) in a FQDN. For TLDs ORG/NET/COM 
> and other non-National TLDs this is at the 2nd level. IOW, Panix.Com 
> is a Domain Name and all *.Panix.Com FQDNs are owned/controlled by 
> the owner of Panix.Com (who can be determined from a WhoIs lookup). 
[...]
> What I was  
> pointing out (or trying to say) is that a .COM domain name is 2 
> levels. If there was a 3-level .COM FQDN then the first (Left Most) 
> level is a Hostname in the 2-level domain. Any 4 or longer .COM FQDN 
> still has the first level as Hostname while all other levels to the 
> left of the 2nd level are subdomains in that 2-level domain.

No.  Some commercial entities organize their domain name space along
host.department.division.company.COM or other schemes.  From the outside,
it simply is not possible to definitively state whether or not
foo.bar.baz.qxx.COM is or is not a host name (as opposed to a domain
name that does not identify a particular host) unless the name resolves
(following CNAME records) to an A record, in which case it is a host.
Bit the converse is not conclusive; some hosts might not be identifiable
via externally-visible DNS queries.

All of which is rather academic regarding the specification text.  What
needs to be said is that the local-part is not to be interpreted outside
of the administrative domain of the "address" (really mailbox) domain
name.  Whether such interpretation occurs at an MX host or at some other
host on the recipient side of the MX host(s), or even via some mechanism
distributed over multiple hosts within the administrative domain is
an irrelevant detail.

Date: Fri, 15 Jul 2005 11:12:59 -0400
From: John C Klensin <[email-masked]>
To: SM <[email-masked]>, [email-masked]
Subject: Re: Definition of server connection failure?
Sender: [email-masked]

--On Friday, 15 July, 2005 06:36 -0700 SM <[email-masked]>
wrote:

> 
> Hi Michael,
> At 10:04 14-07-2005, Michael Kenny wrote:
>> We use the Tumbleweed EMF as our outward facing mail relay.
>> It does anti-virus scanning and spam filtering for us and
>> then relays to one of three Domino R5 servers for further
>> inbound delivery. The Tumbleweed server maintains an internal
>> list of the inbound mail relay servers to try, in order, as
>> listed in their Relays options setup.
>>...
>> The answer from Tumbleweed was this: "The EMF server detected
>> the server was up, so it never retried the others."
> 
> The SMTP service was not up.  The server should have retried
> delivery to the other relay servers according to your
> description in paragraph one.  This relay strategy is a local
> implementation by the vendor and is not covered by RFC 2821.

To put that a little more strongly, it is a member of the family
of local strategies that 2821 intends to permit.  That
permission comes, essentially by assuming that final delivery
_from the SMTP transport environment_ is made to your server and
that anything after that is an internal matter.  But the
behavior is not conformant to the standard, it is just, from the
standard's point of view, local processing.

This is one of the reasons why there are actual advantages to
the slightly fuzzy mail model of 2821 as compared to more
rigorous models of the Internet mail system that identify
particular roles and than assume that, having identified them,
many specific attributes can be attached to the identification.
In the real world, a receiving mail server for an organization
can behave like a relay to "internal" SMTP hosts, as a gateway
to an entirely different mail format and transport environment,
as a host that delivers, directly or indirectly, to a final mail
store, etc.  All of these cases have uses in the real world, and
all of them can be made to "work" reasonably well.  The mail
server sending traffic into one of these environments can't tell
what happens in the hop or function at occurs after it delivers
to the server it delivers to.  That is actually an advantage
from both security and operations standpoints.

>> Two questions: Is that assumption correct? That is, should an
>> SMTP client rely solely on correct SMTP responses and not
>> some other machine state to determine if a server is up and
>> whether to try the next server in line?
> 
> The client should rely on the SMTP response.  It should retry
> the next server in line if that implementation is like MX
> records.
> 
>> And: Is delivery failure definition and the resulting
>> behavior clearly outlined in an RFC somewhere? I've seen the
>> rules about how to handle MX record requests and how to order
>> them and the resulting deliveries. But I guess I missed
>> something that clearly defines when an SMTP server is
>> supposed to be considered "Down"
> 
> This is discussed in RFC 2821.  See the "Sending Strategy"
> section.  If you connect to a SMTP service and you don't get
> an initial reply (Section 4.5.3.2), you consider that service
> as down.

I thought that text was about as specific as it needed to be,
maybe a bit more so.  If it is not, please cite specific
ambiguities or, better, suggest text for 2821bis.

     john

Date: Fri, 15 Jul 2005 06:36:31 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Definition of server connection failure?
Sender: [email-masked]

Hi Michael,
At 10:04 14-07-2005, Michael Kenny wrote:
>We use the Tumbleweed EMF as our outward facing mail relay. It does
>anti-virus scanning and spam filtering for us and then relays to one
>of three Domino R5 servers for further inbound delivery. The
>Tumbleweed server maintains an internal list of the inbound mail relay
>servers to try, in order, as listed in their Relays options setup.
>
>Due to a problem on one Domino server (the first in the Tumbleweed
>relay list,) we shut down the Domino service on that box. The OS
>(Win2K Advanced Server) was still running. No other service is
>listening on the SMTP port.
>
>The problem surfaced when one Tumbleweed server tried to deliver mail
>inbound to the downed Domino server. When it could not, the Tumbleweed
>EMF server generated a delivery failure and sent it back. It never
>tried the other two servers in it's relay list.
>
>The answer from Tumbleweed was this: "The EMF server detected the
>server was up, so it never retried the others."

The SMTP service was not up.  The server should have retried delivery 
to the other relay servers according to your description in paragraph 
one.  This relay strategy is a local implementation by the vendor and 
is not covered by RFC 2821.

>Two questions: Is that assumption correct? That is, should an SMTP
>client rely solely on correct SMTP responses and not some other
>machine state to determine if a server is up and whether to try the
>next server in line?

The client should rely on the SMTP response.  It should retry the 
next server in line if that implementation is like MX records.

>And: Is delivery failure definition and the resulting behavior clearly
>outlined in an RFC somewhere? I've seen the rules about how to handle
>MX record requests and how to order them and the resulting deliveries.
>But I guess I missed something that clearly defines when an SMTP
>server is supposed to be considered "Down"

This is discussed in RFC 2821.  See the "Sending Strategy" 
section.  If you connect to a SMTP service and you don't get an 
initial reply (Section 4.5.3.2), you consider that service as down.

Regards,
-sm 

Date: Thu, 14 Jul 2005 13:04:34 -0400
From: Michael Kenny <[email-masked]>
Reply-To: Michael Kenny <[email-masked]>
To: [email-masked]
Subject: Definition of server connection failure?
Sender: [email-masked]

Greetings,
I'm a network/email admin who just joined the list. I've reviewed the
archives as best I could, and read what I think is the relevant RFC
(2821) and I'm still a little unclear on something. I'm hoping you
guys can help me clear this up, and at the same time, that I'm not
asking something obvious and thereby making myself look really stupid
with my first post on the list.

We use the Tumbleweed EMF as our outward facing mail relay. It does
anti-virus scanning and spam filtering for us and then relays to one
of three Domino R5 servers for further inbound delivery. The
Tumbleweed server maintains an internal list of the inbound mail relay
servers to try, in order, as listed in their Relays options setup.

Due to a problem on one Domino server (the first in the Tumbleweed
relay list,) we shut down the Domino service on that box. The OS
(Win2K Advanced Server) was still running. No other service is
listening on the SMTP port.

The problem surfaced when one Tumbleweed server tried to deliver mail
inbound to the downed Domino server. When it could not, the Tumbleweed
EMF server generated a delivery failure and sent it back. It never
tried the other two servers in it's relay list.

The answer from Tumbleweed was this: "The EMF server detected the
server was up, so it never retried the others."

That strikes me as extremely lame. There was not , nor could there be,
any sort of SMTP response to the delivery attempts. I don't know what
detection method they are using to determine the Domino box was up
(ping? IP connection to the port? I don't know.)

RFC 2821 says, first sentence in section 3.1: "An SMTP session is
initiated when a client opens a connection to a server and the server
responds with an opening message."

I could find no other information about what constitues a connection
failure. Perhaps it's so obvious as to not even be asked.

But is it so that if there is no correct reponse to an SMTP session
initation, then there is no session and that SMTP server cannot be
considered "Up" and the next server in line should be tried? Normally
next is determine via MX records and routing cost, I know, but in this
case the next in line is determined in their internal list.

Two questions: Is that assumption correct? That is, should an SMTP
client rely solely on correct SMTP responses and not some other
machine state to determine if a server is up and whether to try the
next server in line?

And: Is delivery failure definition and the resulting behavior clearly
outlined in an RFC somewhere? I've seen the rules about how to handle
MX record requests and how to order them and the resulting deliveries.
But I guess I missed something that clearly defines when an SMTP
server is supposed to be considered "Down"

Thanks for taking the time to read this and I hope I've been clear and
not asked something moronic.

regards,

Mike

Date: Thu, 14 Jul 2005 10:21:49 -0400
From: John Leslie <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>
Cc: [email-masked]
Subject: 2821bis- meaning of "host"
Sender: [email-masked]

Robert A. Rosenberg <[email-masked]> wrote:
> At 02:33 -0400 on 07/14/2005, [email-masked] wrote...
>> On Thu, 14 Jul 2005 00:58:32 EDT, "Robert A. Rosenberg" said:
>>> At 14:51 +0100 on 07/13/2005, [email-masked] wrote...
>>>
>>>> section 2.3.10 has a "fuzzy" reference
>>>> "the host specified in the domain part of the address."
>>>> in general the host is NOT the domain part so what is meant by the 
>>>> host specified in the domain part of the address??
>>>    
>>> The address is of the form UserID@Domain (with the "Domain Part"
>>> being everything to the right of the @-sign). The Domain part has a
>>> host if there are more levels than the number defined for the WhoIs
>>> registration. If the domain is example.com but the Domain in the
>>> address is X.SMTP.EXAMPLE.COM then X is the host in the SMTP
>>> subdomain of domain EXAMPLE.COM.
>>
>> Are you sure that basing it off the whois is the proper referent for this?
> 
> Yes. I am defining "Domain" based on ownership and control of that 
> level (and those to the left of it) in a FQDN. For TLDs ORG/NET/COM 
> and other non-National TLDs this is at the 2nd level. IOW, Panix.Com 
> is a Domain Name and all *.Panix.Com FQDNs are owned/controlled by 
> the owner of Panix.Com (who can be determined from a WhoIs lookup). 
> For national TLDs such as UK or AU, there are addition levels before 
> you get to the ownership level (IOW: If Panix were registered as a UK 
> Domain it would be Panix.CO.UK not just Panix.UK). What I was 
> pointing out (or trying to say) is that a .COM domain name is 2 
> levels. If there was a 3-level .COM FQDN then the first (Left Most) 
> level is a Hostname in the 2-level domain. Any 4 or longer .COM FQDN 
> still has the first level as Hostname while all other levels to the 
> left of the 2nd level are subdomains in that 2-level domain.

   I must admit to a little difficulty following this discussion.

   First, I must admit to having not yet completed my first reading
of 2821bis. So far I have two strong reactions:

1. It's _much_ too long; and
2. It _really_ needs a section on definitions.

   (Yes, I'm aware of the contradictory nature of these reactions.)

   In any case, I agree with Robert that his interpretation of "host"
in the context of 2.3.10 is as good as any I can think of.

   My problem is, when might "host" mean that in the rest of the
document? Is there really any hope of dozens of implementers coming
to the same conclusion about each meaning of "host"?

   I seriously recommend that we try to avoid any terms which mean
one thing in one part of the document, and something else in another.

   And I really do believe a section on definitions would help. (I'm
even willing to help put it together.)

--
John Leslie <[email-masked]>

Date: Thu, 14 Jul 2005 09:00:20 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: draft-klensin-rfc2821bis-00.txt structural and textual comments
Sender: [email-masked]

At 02:33 -0400 on 07/14/2005, [email-masked] wrote about Re: 
draft-klensin-rfc2821bis-00.txt structural and textual :

>On Thu, 14 Jul 2005 00:58:32 EDT, "Robert A. Rosenberg" said:
>
>>  being everything to the right of the @-sign). The Domain part has a
>>  host if there are more levels than the number defined for the WhoIs
>>  registration.
>
>Are you sure that basing it off the whois is the proper referent for this?

Yes. I am defining "Domain" based on ownership and control of that 
level (and those to the left of it) in a FQDN. For TLDs ORG/NET/COM 
and other non-National TLDs this is at the 2nd level. IOW, Panix.Com 
is a Domain Name and all *.Panix.Com FQDNs are owned/controlled by 
the owner of Panix.Com (who can be determined from a WhoIs lookup). 
For national TLDs such as UK or AU, there are addition levels before 
you get to the ownership level (IOW: If Panix were registered as a UK 
Domain it would be Panix.CO.UK not just Panix.UK). What I was 
pointing out (or trying to say) is that a .COM domain name is 2 
levels. If there was a 3-level .COM FQDN then the first (Left Most) 
level is a Hostname in the 2-level domain. Any 4 or longer .COM FQDN 
still has the first level as Hostname while all other levels to the 
left of the 2nd level are subdomains in that 2-level domain.

Date: Thu, 14 Jul 2005 01:50:49 -0700 (PDT)
From: "william(at)elan.net" <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>
cc: [email-masked]
Subject: Re: draft-klensin-rfc2821bis-00.txt structural and textual comments
Sender: [email-masked]

On Thu, 14 Jul 2005, Robert A. Rosenberg wrote:

>
> At 14:51 +0100 on 07/13/2005, [email-masked] wrote about Re: 
> draft-klensin-rfc2821bis-00.txt structural and textual :
>
>> section 2.3.10 has a "fuzzy" reference
>> "the host specified in the domain part of the address."
>> in general the host is NOT the domain part so what is meant by the host 
>> specified in the domain part of the address??
>
> The address is of the form UserID@Domain (with the "Domain Part" being 
> everything to the right of the @-sign). The Domain part has a host if there 
> are more levels than the number defined for the WhoIs registration. If the 
> domain is example.com but the Domain in the address is X.SMTP.EXAMPLE.COM 
> then X is the host in the SMTP subdomain of domain EXAMPLE.COM.

While "X is a host of SMTP subdomain of domain example.com" is correct,
so would it be that "X is a host of subdomain SMTP.EXAMPLE of domain .com"
or that "X host of domain SMTP.example.com" or for that matter that
X.smtp.example.com is fully qualified host name.

Even though certain levels of delegations exist in ICANN tree and can
sometimes be found based on whois - these are not technical levels and
you can not make an assumption that some part is a domain, something
is subdomain and something is a host just by looking at the dns name.

Based on dns terms in reality every hostname is a domain on its own
(subject to that there is some difference in characters allowed in
  hostnames and domains, but that rarely comes into play).

-- 
William Leibzon
Elan Networks
[email-masked]

To: "Robert A. Rosenberg" <[email-masked]>
Cc: [email-masked]
Subject: Re: draft-klensin-rfc2821bis-00.txt structural and textual comments 
From: [email-masked]
Date: Thu, 14 Jul 2005 02:33:55 -0400
Sender: [email-masked]

--==_Exmh_1121322835_25165P
Content-Type: text/plain; charset=us-ascii

On Thu, 14 Jul 2005 00:58:32 EDT, "Robert A. Rosenberg" said:

> being everything to the right of the @-sign). The Domain part has a 
> host if there are more levels than the number defined for the WhoIs 
> registration.

Are you sure that basing it off the whois is the proper referent for this?

--==_Exmh_1121322835_25165P
Content-Type: application/pgp-signature

--==_Exmh_1121322835_25165P--

Date: Thu, 14 Jul 2005 00:58:32 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: draft-klensin-rfc2821bis-00.txt structural and textual comments
Sender: [email-masked]

At 14:51 +0100 on 07/13/2005, [email-masked] wrote about 
Re: draft-klensin-rfc2821bis-00.txt structural and textual :

>section 2.3.10 has a "fuzzy" reference
>"the host specified in the domain part of the address."
>in general the host is NOT the domain part so what is meant by the 
>host specified in the domain part of the address??

The address is of the form UserID@Domain (with the "Domain Part" 
being everything to the right of the @-sign). The Domain part has a 
host if there are more levels than the number defined for the WhoIs 
registration. If the domain is example.com but the Domain in the 
address is X.SMTP.EXAMPLE.COM then X is the host in the SMTP 
subdomain of domain EXAMPLE.COM.

To: John C Klensin <[email-masked]>, [email-masked]
From: [email-masked]
Subject: Re: draft-klensin-rfc2821bis-00.txt structural and textual comments
Date: Wed, 13 Jul 2005 14:51:31 +0100
Sender: [email-masked]

Thanks John for all your work.

Here a first hand reply to version -00

While I was reading the draft I started  thinking about the structure of the document, the current structure is complicated, has duplicates, and is not easy to follow  and maybe it is a good idea to rearrange the sequence of many sections.

At the end of this email i will put my suggestion of the sequence of sections.
Off course this is all open to discussion.
And i know it will give a lot of work.

Other suggestion:
2.3 Terminology: Splitting up of sections:

section 2.3  Terminology and subssections need to be split up so that every term has its own sub.sub.section.
This is mainly with the purpose that every term gets its own entry in the contents

Requirement-levels:
better a reference to RFC 2119

section 2.3.2 senders and receivers
Is it not better to replace senders with SMTP-clients and reiceivers with SMTP-servers??

For clarity maybe it is even better to rewrite everywhere in the draft  client to SMTP-client and server to SMTP-server.

section 2.3.4
Hostnames are domainnames ???

section 2.3.9  Message Content and Mail Data, combine this with the (new -sub) section 2.3.1 Mail Objects

section 2.3.10 has a "fuzzy" reference
"the host specified in the domain part of the address."
in general the host is NOT the domain part so what is meant by the host specified in the domain part of the address??

Missing terminology:
return path a clear definition under terminology would be welcome.

section 3.6

section 3.6 also needs splitting up, the part about "undeliverable mail" notification message  (bounces) is better under section 6.1

The sentence:
SMTP servers are also permitted to ignore the route information and simply send to the final destination specified as the last element in the route and SHOULD do so.
should be rewritten as:
SMTP servers SHOULD ignore the route information and simply send to the final destination specified as the last element in the route

Textual typo's

page header line: should it not be ESMTP?

In the abstract there is a sentence
It consolidates, updates, and clarifies several previous documents, making all or parts of most of the obsolete.

contents
subsections 4.1.1.1 -> 4.1.1.10 are missing

Structure changes:

reasons:
my primary reson was that i wanted to get rid of section 4.3.1 and especially section 4,3,2 wich i found an eyesore
If you want to do this you need to integrate section 4.1 and 4.2
If you do this i realised hat section 2.2 fits under section 4.1.1

and so on and so on. It worked like a snowball...

my current suggestion is

Ps the numbers refer to the old section numbers:

1.   Introduction
1.1  Transport of electronic mail
1.2  History and context for this document
2.   The SMTP Model
2.1  Basic Structure
2.3  Terminology
2.3.1  Mail Objects
To be split up
2.3.2  Senders and Receivers
To be split up
2.3.3  Mail Agents and Message Stores
2.3.4  Host
2.3.5  Domain Names
2.3.6  Buffer and State Table
2.3.7  Lines
2.3.8  Originator, Delivery, Relay, and Gateway
To be split up
2.3.9  Message Content and Mail Data
can be put under 2.3.1
2.3.10   Mailbox and Address
2.3.11   Reply
New setion
return path

2.4  General Syntax Principles and Transaction Model
NEW section 2.4.1 8bit per byte transaction model

3.   The SMTP transactions: An overview
3.1  Session Initiation
3.2  Client Initiation
3.3  Mail Transactions

New section (3+ at the moment)
3+. SMTP Procedures: An Overview

3.4  Forwarding for Address Correction or Updating
(3.5 goes into section 4)
3.6  Relaying
3.7  Mail Gatewaying
3.7.1  Header Fields in Gatewaying
3.7.2  Received Lines in Gatewaying
3.7.3  Addresses in Gatewaying
3.7.4  Other Header Fields in Gatewaying
3.7.5  Envelopes in Gatewaying
3.8  Terminating Sessions and Connections
3.9  Mailing Lists and Aliases
3.9.1  Alias
3.9.2  List

4.   The SMTP Specifications
4.1  SMTP Commands  nd reply's general syntaxes
4.1.1  Command Semantics and Syntax
4.1.2  Command Argument Syntax
4.1.3  Address Literals
4.1.4  Order of Commands
2.2  The Extension Model
2.2.1  Background
2.2.2  Definition and Registration of Extensions
4.1.5  Private-use Commands
4.2.1  Reply Code Severities and Theory
4.2.2  Reply Codes by Function Groups
4.2.3  Reply Codes in Numeric Order
4.2.4  Reply Code 502
4.1.1.1  Extended HELLO (EHLO) or HELLO (HELO)
4.1.1.2  MAIL (MAIL)
4.1.1.3  RECIPIENT (RCPT)
4.1.1.4  DATA (DATA)
4.2.5  Reply Codes After DATA and the Subsequent <CRLF>.<CRLF>

4.1.1.5  RESET (RSET)

3.5  Commands for Debugging Addresses
3.5.1  Overview
3.5.2  VRFY Normal Response
3.5.3  Meaning of VRFY or EXPN Success Response
3.5.4  Semantics and Applications of EXPN
4.1.1.6  VERIFY (VRFY)
4.1.1.7  EXPAND (EXPN)

4.1.1.8  HELP (HELP)
4.1.1.9  NOOP (NOOP)
4.1.1.10  QUIT (QUIT

4.2  SMTP Replies
not needed anymore

4.3  Sequencing of Commands and Replies
4.3.1  Sequencing Overview
not nescesary is already under 3.1-3.3

4.3.2  Command-Reply Sequences
directly at command in section 4.1.1.1 ed

4.4  Trace Information
to be split up
4.4.1 received: header
4.4.2 return path

4.5  Additional Implementation Issues
4.5.1  Minimum Implementation
4.5.2  Transparency
4.5.3  Sizes and Timeouts
4.5.4  Retry Strategies
4.5.5  Messages with a null reverse-path
5.   Address Resolution and Mail Handling
6.   Problem Detection and Handling
6.1  Reliable Delivery and Replies by Email
split this section with a seperate section for
"undeliverable mail" notification message  (bounces)
6.2  Unwanted, unsolicited, and "attack" messages
6.3  Loop Detection
6.4  Compensating for Irregularities
7.   Security Considerations
7.1  Mail Security and Spoofing
7.2  "Blind" Copies
7.3  VRFY, EXPN, and Security
7.4  Information Disclosure in Announcements
7.5  Information Disclosure in Trace Fields
7.6  Information Disclosure in Message Forwarding
7.7  Resistance to Attacks
7.8  Scope of Operation of SMTP Servers
8.   IANA Considerations
9.   Acknowledgments
10.  References
10.1   Normative References
10.2   Informative References
Author's Address

Appendices
collected ABNF is missing

An Index would be really welcome

Date: Wed, 13 Jul 2005 05:18:56 -0700 (PDT)
From: "william(at)elan.net" <[email-masked]>
To: [email-masked]
cc: John C Klensin <[email-masked]>
Subject: Re: draft-klensin-rfc2821bis-00.txt
Sender: [email-masked]

On Wed, 13 Jul 2005, Bruce Lilly wrote:

>>> 5. "TCP" really ought to be registered with IANA for use in the
>>>    "via" Received field component (or dropped, since it wasn't
>>> in    821).
>>
>> Yes, someone should do that.  Do you think it can be just done,
>> or that we need to create a new "IANA considerations"
>> subsection? :-(
>
> One issue is whether it should have been introduced at all, since
> it had never been registered and was not in 821 or any of the
> documents which amended 821.
>
> It was introduced in 2821, but not registered.
>
> Your draft states:
>
>                     "Via" is primarily of value
>                     ; with non-Internet transports.  SMTP servers
>                     ; SHOULD NOT use unregistered names.
>
> So as it stands, "TCP" SHOULD NOT be used, since it is
> unregistered.

Actually in my view its supposed to be more like you MUST NOT use 
unregistered values and you SHOULD NOT use "via" for SMTP, but if
you do choose to use it for SMTP the convention is to use "TCP"
and it should be considered considered an optional value for via
for SMTP transport and so it should be registered.

I also see no problem with having via "TCP" value registration added to 
IANA section of this draft (with a note that what is registered is not
a new feature but value in common use for many years).

-- 
William Leibzon
Elan Networks
[email-masked]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: draft-klensin-rfc2821bis-00.txt
Date: Wed, 13 Jul 2005 08:06:55 -0400
Cc: John C Klensin <[email-masked]>
Sender: [email-masked]

On Wed July 13 2005 05:10, John C Klensin wrote:
> 
> --On Tuesday, July 12, 2005 21:22 -0400 Bruce Lilly 
> <[email-masked]> wrote:
> 
> > Some quick comments on the draft and its security counterpart.
> >
> > 1. numeric reference tags are deprecated, as discussed on the
> >    rfc-interest list several months ago
> 
> One-line change in the XML, which has been tested.  I decided to 
> leave -00 as is to permit more easy comparisons to 2821.

Given that the reference tag numbers differ and the citation text
also differs, I fail to see how that facilitates comparison.

> And discussion on rfc-interest is not 
> exactly the way normative changes are made around here.

Changing the format of a reference tag is hardly normative; it
is at most editorial.  The comment was simply a reminder (you
were aware of the discussion, as you participated mid-January).

> > 2. the security draft needs a good proofreading:
> >    o "mesage"
> >    O "among" should probably be "amount"
> >    o "approrpriate"
> >    o "appropropriately"
> >    O " if IDENT" should probably be " of IDENT"
> 
> Which part of it did you have trouble understanding?

On first reading there was something that made no sense; I could
not find it again during a quick scan when composing my comments.
I am leaving on business travel in a few hours, so I don't have
time now to look thoroughly.

> Put  
> differently, it was a hasty job to aid discussion and the choice 
> was between getting it out as is or doing the careful 
> proofreading and posting in around 8 August.

A spelling check would probably have caught many of the typos,
should have taken at most a few minutes, and would save many
reviewers much time trying to decipher the text.

> > 3. There remains a conflict with RFC 2822 w.r.t. the Received
> >    field; 2821[bis] permits a quoted-string for the "id"
> > component,    which is not permissible in the field as defined
> > in RFC 2822.
> 
> Clearly one of them should be adjusted.  Which one is, IMO, a 
> list or WG discussion item.

If the fundamental concept of 282[12], i.e. consolidation of
documents rather than introduction of new features, has not
changed, then the course of action is clear; quoted strings
were not permitted in ID by RFCs 821 or 822.  Indeed, a
number of constructs that were legal in 822 in various places
have been deprecated to simplify parsing; I know of only one
case where a new feature was added, and that was a concession
to widespread (but illegal) use -- I know of no widespread
use of quoted-strings in the ID part of Received fields.

And I would remind you that this particular discrepancy was
noted shortly after publication of 2821 and 2822 four years
ago.
 
> > 4. The same peculiar and confusing wording of RFC 1123 is
> > repeated;    RFC 822 did not "suggest" an '@' in a msg-id, it
> > clearly specified    it (along with '<', '>', '.', and
> > possibly '[' and ']') in msg-id.
> 
> And this is not 822 or 822bis.

The comment isn't about 822, it's about text in the draft
which mischaracterizes 822.

> > 5. "TCP" really ought to be registered with IANA for use in the
> >    "via" Received field component (or dropped, since it wasn't
> > in    821).
> 
> Yes, someone should do that.  Do you think it can be just done, 
> or that we need to create a new "IANA considerations" 
> subsection? :-(

One issue is whether it should have been introduced at all, since
it had never been registered and was not in 821 or any of the
documents which amended 821.

It was introduced in 2821, but not registered.

Your draft states:

                     "Via" is primarily of value
                     ; with non-Internet transports.  SMTP servers
                     ; SHOULD NOT use unregistered names.

So as it stands, "TCP" SHOULD NOT be used, since it is
unregistered.

*If* it is to be introduced as a new feature, then it ought to
be registered, and that takes one sentence (not subsection) in
the IANA considerations section.

Date: Wed, 13 Jul 2005 05:10:15 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: Re: draft-klensin-rfc2821bis-00.txt
Sender: [email-masked]

--On Tuesday, July 12, 2005 21:22 -0400 Bruce Lilly 
<[email-masked]> wrote:

> Some quick comments on the draft and its security counterpart.
>
> 1. numeric reference tags are deprecated, as discussed on the
>    rfc-interest list several months ago

One-line change in the XML, which has been tested.  I decided to 
leave -00 as is to permit more easy comparisons to 2821.  I will 
probably switch to the other form for -01, depending on what 
feedback comes in.  And discussion on rfc-interest is not 
exactly the way normative changes are made around here.

> 2. the security draft needs a good proofreading:
>    o "mesage"
>    O "among" should probably be "amount"
>    o "approrpriate"
>    o "appropropriately"
>    O " if IDENT" should probably be " of IDENT"

Which part of it did you have trouble understanding?  Put 
differently, it was a hasty job to aid discussion and the choice 
was between getting it out as is or doing the careful 
proofreading and posting in around 8 August.  This seemed 
better.  I assume we will make some decisions, somehow, which 
will be folded into the 2821bis base document -- I'd be 
surprised if there is a -01 of this that will need polishing or 
revising.

> 3. There remains a conflict with RFC 2822 w.r.t. the Received
>    field; 2821[bis] permits a quoted-string for the "id"
> component,    which is not permissible in the field as defined
> in RFC 2822.

Clearly one of them should be adjusted.  Which one is, IMO, a 
list or WG discussion item.

> 4. The same peculiar and confusing wording of RFC 1123 is
> repeated;    RFC 822 did not "suggest" an '@' in a msg-id, it
> clearly specified    it (along with '<', '>', '.', and
> possibly '[' and ']') in msg-id.

And this is not 822 or 822bis.  As I indicated in the temporary 
introductory material, very few purely editorial changes have 
been made (including your long list of suggestions of a year 
ago).  Those haven't been rejected, just deferred for/unti 
further discussion.

> 5. "TCP" really ought to be registered with IANA for use in the
>    "via" Received field component (or dropped, since it wasn't
> in    821).

Yes, someone should do that.  Do you think it can be just done, 
or that we need to create a new "IANA considerations" 
subsection? :-(

> 6. Consistent terminology would be helpful.  "Header" is
> sometimes    used in the sense given in FYI 18 and as defined
> in RFC 2822,    and other times where "header field" would be
> more appropriate.

See "editorial" above.

      john

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: draft-klensin-rfc2821bis-00.txt
Date: Tue, 12 Jul 2005 21:22:15 -0400
Cc: John C Klensin <[email-masked]>
Sender: [email-masked]

Some quick comments on the draft and its security counterpart.

1. numeric reference tags are deprecated, as discussed on the
   rfc-interest list several months ago

2. the security draft needs a good proofreading:
   o "mesage"
   O "among" should probably be "amount"
   o "approrpriate"
   o "appropropriately"
   O " if IDENT" should probably be " of IDENT"
  
3. There remains a conflict with RFC 2822 w.r.t. the Received
   field; 2821[bis] permits a quoted-string for the "id" component,
   which is not permissible in the field as defined in RFC 2822.

4. The same peculiar and confusing wording of RFC 1123 is repeated;
   RFC 822 did not "suggest" an '@' in a msg-id, it clearly specified
   it (along with '<', '>', '.', and possibly '[' and ']') in msg-id.

5. "TCP" really ought to be registered with IANA for use in the
   "via" Received field component (or dropped, since it wasn't in
   821).

6. Consistent terminology would be helpful.  "Header" is sometimes
   used in the sense given in FYI 18 and as defined in RFC 2822,
   and other times where "header field" would be more appropriate.

Date: Sat, 09 Jul 2005 17:31:38 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: draft-klensin-rfc2821bis-00.txt
Sender: [email-masked]

Hi.

This document was submitted for posting yesterday and should
show up early next week with an announcement to this list as
well as the I-D one.

As Murphy's law and general carelessness would predict, I didn't
recheck the RFC Editor's typographical errata list against the
document before submitting it for posting.  Several minor errors
listed there had not been caught.  The working copy has been
corrected and the fixes will appear with -01.   But, this close
to the posting deadline, trying to patch -00 would be unfair to
the Secretariat and would probably lead to more confusion, so
I'm going to let it go.

Sorry.

    john

To: "Robert A. Rosenberg" <[email-masked]>
Cc: IETF-SMTP <[email-masked]>
Subject: Re: IP::DOMAIN Associations [Re: Bounce/System Notification Address Verification] 
From: [email-masked]
Date: Sat, 09 Jul 2005 10:10:22 -0400
Sender: [email-masked]

--==_Exmh_1120918222_8886P
Content-Type: text/plain; charset=us-ascii

On Fri, 08 Jul 2005 05:15:34 EDT, "Robert A. Rosenberg" said:

> Unless your (ie: The VT.ETU) MSA Mail Servers do not listen on 
> Port587 but only Port25 or do support SMTP-over-SSL on Port465, there 
> is IMO no reason for their MUA to be pointing (accidentally or not) 
> at their current CONNECTIVITY ISP's MSA server.

Good. Want to come down some weekend and reconfigure 70K systems, some of which
are off-site, and some of which are running MUAs that don't understand using
port 587, and the vast majority of which are (a) not under our administrative
control and (b) run by users who don't understand why they need to make changes
and upgrades in perfectly functional systems just so *we* can deploy something
that doesn't seem to provide them any benefit?

> that do Outgoing-Port25-blocking (or the user using an old MUA that 
> did not support designating what port to use for MSA upload 
> connections).

Welcome to the real world - one where users run old software, and there are
actual costs involved in changing configurations, and you need to split effort
between "upgrade to all-singing, all-dancing, perfect configuration" and "fix
this week's problem right now because at 2M msgs/day, you can fall behind in
12 hours enough to take 3 days to get caught up".

Deploying the politically correct configuration would be all of 2 hour's work for
us.  Doing it in a manner that the resulting backlash wouldn't get us all fired
is a whole 'nother ball game.  Moving from POP on port 110 to POP/SSL on 995
was a *year long* project, and we *still* got flooded with complaints the day
it went live....

--==_Exmh_1120918222_8886P
Content-Type: application/pgp-signature

--==_Exmh_1120918222_8886P--

From: "Hector Santos" <[email-masked]>
To: "John C Klensin" <[email-masked]>, "John Leslie" <[email-masked]>, "Tony Hansen" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: OPES and Email
Date: Fri, 8 Jul 2005 15:34:33 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "John C Klensin" <[email-masked]>

>> ] In SMTP the OPES processor may be any agent participating in
>> SMTP ] exhanges, including MSA, MTA, MDA, and MUA.  This
>> document focues on ] use cases in which the OPES processor is
>> a mail transfer agent (MTA).
>
> Note that, while most of us know, more or less, what all of
> those nice abbreviations mean, there is no consensus definition
> for an MDA (the 2821 definition of a "delivery MTA" may or may
> not be the same thing) nor, IMO, does was the submission
> terminology of RFC 2476 or 2476bis (aka "originating server" in
> 2821) intended to  adequately delimit the "MSA" term.

I always felt there were conflictive usage of MSA, "SUBMIT" "SUBMISSION"
with or without 2476/bis.

In my view,

  MDA  -  Unauthorized Final Delivery Agent
  MSA  -  Authorized Submission Agent (Final or Route)

The MSA implies there is an association with the sender, like the MUA or
authorized MTA.   But if its not unauthorized, then it can only act as a MDA
(based on BCP).

> You know all of this.  The question for me is that, given that
> the OPES doc pushes really close to, or past, the boundary in
> which its recommendations and structure require violating the
> provisions of 2821, how would you like people to respond in a
> way that would be most constructive?  The answer obviously
> interacts with several of the discussion threads on the
> ietf-smtp list during the last few weeks, including such
> questions as the legitimacy or appropriateness of mail relays
> doing virus removal.

And intentionally so, because whether its OPES or not, this is no doubt a
growing reality in the SMTP process for the idea of extending the
functionality either by "shimming" it, hooking it, calling sinks, callouts,
"Web Services",  "RPC"  etc, etc. From the point of view of the "SMTP
receiver"  it is a "blackbox" concept:

       result =  CALLOUT (State, parameters)

What would be nice is to get 2821/bis to being to realize this reality and
begin to define (revisit) the integration standard which from a SMTP
standpoint is:

At a minimum for backward capability:

    - Timeout Considerations
    - Returning Proper Response Codes

New considerations (really revisit, because its already there):

    - Process Variables passing
    - Keep Alive Concepts
    - Call Out Extended Response Concept

We already have systems that have dynamic logic for sinks, hooks, or
callouts.  Exim has ACL,  SendMail has Mfilters,  WildcatSMTP calls them WCX
hooks.  We also have the standard Sieve usages as well, although I believe
most do this on a post SMTP basis, don't know of Sieve is used dynamically
within SMTP.

These ideas are being used more and more, and I will venture it will main
stream once the new email authentication/authorization "extended protocols"
are adopted on a wider basis.

Whatever the SMTP CALLOUT method,  it  would be nice is the standard or
refocus on the SMTP interfacing concept.

Anyway for this draft, OPES SMTP Use Cases,  it is functional outline that
touches base with some technical conflicts.  I am not sure if the details
should be in the document level, but it should be consistent in terminology
and usage of examples.

When I looked this document, I imagine "What if" there existed a Proxy
Service out there that my SMTP server can call out too.  What will it take,
and what will be the issues.

Well,  in the same way a CBV is a Callout to some "remote host/service" to
validation the return path (RPATH):

   result = CBV_RPATH_VALIDATOR(mail from)

the same basic issues that we confronted here, will be the same if it was
plugged and played with something else:

   result = OPES_RPATH_VALIDATOR_SERVICE(parameters)

I already added my comments here about this:

    - How parameters are passed
    - How results are turned
    - Timeout issues

The latter was discussed in this link showing how SMTP 1xy continuation
results codes can be used to Keep Alive a SMTP server during a "Call Out"
session at the receiver:

http://www.imc.org/ietf-openproxy/mail-archive/msg03282.html

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Fri, 08 Jul 2005 14:24:11 -0400
From: John C Klensin <[email-masked]>
To: John Leslie <[email-masked]>, Tony Hansen <[email-masked]>
cc: ietf-822 <[email-masked]>, [email-masked]
Subject: Re: OPES and Email
Sender: [email-masked]

--On Friday, 08 July, 2005 13:37 -0400 John Leslie
<[email-masked]> wrote:

> Tony Hansen <[email-masked]> wrote:
>> ... 
>> OPES SMTP Use Cases (draft-ietf-opes-smtp-use-cases-02.txt).
>> ...
>>    The Open Pluggable Edge Services (OPES) framework is
>>    application agnostic.  Application specific adaptations
>>    extend that framework. This document describes OPES SMTP
>>    use cases and deployment scenarios in preparation for SMTP
>>    adaptation with OPES.
> 
>    We should be very careful here. "Open Pluggable Edge
> Services" suggests we're talking only about "edge services";
> and IMHO we should be sure to explain any situations where the
> OPES WG might have a different opinion than others as to what
> an "edge service" is.
> 
> ] In SMTP the OPES processor may be any agent participating in
> SMTP ] exhanges, including MSA, MTA, MDA, and MUA.  This
> document focues on ] use cases in which the OPES processor is
> a mail transfer agent (MTA).

Note that, while most of us know, more or less, what all of
those nice abbreviations mean, there is no consensus definition
for an MDA (the 2821 definition of a "delivery MTA" may or may
not be the same thing) nor, IMO, does was the submission
terminology of RFC 2476 or 2476bis (aka "originating server" in
2821) intended to  adequately delimit the "MSA" term.

>    I'm afraid we start out with different opinions. :^(
> 
>    MTAs have a number of functions; some of which we consider
> to be "edge" functions, and some which we don't. (YMMV.)
>...

Tony,

Let me add one impression to John's useful explanation...

In terms of the structure and language of 2821 (and 2821bis,
which I just sent off for posting (!)), the only MTAs (SMTP
clients or servers) that are even vaguely "edge" services or
systems are:

	* The originator (aka the submission server or MSA)
	
	* The destination server
	
	* A gateway

The definitions of the first and second in 2821/ 2821bis are
deliberately a little fuzzy, but most of us think we know one
when we see one and would probably be in 95% agreement on any
given case.  The discriminating definitions for a gateways is
also a bit fuzzy, but there was an effort during the DRUMS
period to keep that definition firm enough to make it hard for
someone to say "I want to do what would otherwise be considered
Bad Things, hence I will claim to be a gateway".

All of the other cases are relays (with some funny edge cases
around what a non-primary MX can do or not do), and actions
committed by them, especially some of the modifications
suggested in the OPES draft, would, from a 2821 perspective,
constitute mucking around in the middle of the mail environment.
Such actions are prohibited to the extent to which 2821 can
prohibit anything.  

You know all of this.  The question for me is that, given that
the OPES doc pushes really close to, or past, the boundary in
which its recommendations and structure require violating the
provisions of 2821, how would you like people to respond in a
way that would be most constructive?  The answer obviously
interacts with several of the discussion threads on the
ietf-smtp list during the last few weeks, including such
questions as the legitimacy or appropriateness of mail relays
doing virus removal.

best,
     john

Date: Fri, 8 Jul 2005 13:37:35 -0400
From: John Leslie <[email-masked]>
To: Tony Hansen <[email-masked]>
Cc: ietf-822 <[email-masked]>, [email-masked]
Subject: Re: OPES and Email
Sender: [email-masked]

Tony Hansen <[email-masked]> wrote:
>... 
> OPES SMTP Use Cases (draft-ietf-opes-smtp-use-cases-02.txt).
>...
>    The Open Pluggable Edge Services (OPES) framework is application
>    agnostic.  Application specific adaptations extend that framework.
>    This document describes OPES SMTP use cases and deployment scenarios
>    in preparation for SMTP adaptation with OPES.

   We should be very careful here. "Open Pluggable Edge Services" suggests
we're talking only about "edge services"; and IMHO we should be sure to
explain any situations where the OPES WG might have a different opinion
than others as to what an "edge service" is.

] In SMTP the OPES processor may be any agent participating in SMTP
] exhanges, including MSA, MTA, MDA, and MUA.  This document focues on
] use cases in which the OPES processor is a mail transfer agent (MTA).

   I'm afraid we start out with different opinions. :^(

   MTAs have a number of functions; some of which we consider to be
"edge" functions, and some which we don't. (YMMV.)

] This work focuses on SMTP based services that want to modify command
] values and those that want to block commands by defining an error
] response that the MTA should send in response to the response it
] received.  OPES MTA will be involved in SMTP command modification and
] command satisfaction, analog to request modification and request
] satisfaction from HTTP [9].

   I'm afraid I don't understand why this describes an "edge service".

]  +-------+  +---------+      +---------+      +--------+  +-------+
]  |mail  M|  |M mail  M| SMTP |M mail  M| SMTP |M mail M|  |M mail |
]  |clnt  U|--|S srvr  T|------|T gway  T|------|T srvr D|--|U clnt |
]  |      A|  |A       A|      |A       A|      |A      A|  |A      |
]  +-------+  +---------+      +---------+      +--------+  +-------+
]                  |                |                |
]                  | OCP            | OCP            | OCP
]                  |                |                |
]             +----------+     +----------+     +----------+
]             |  callout |     |  callout |     |  callout |
]             |  server  |     |  server  |     |  server  |
]             +----------+     +----------+     +----------+
] 
]                        Figure 2: OPES SMTP Flow

   This describes three situations. I think most of us would agree
that the middle (gateway) function is an edge function. I'm less sure
about the other two. (In fact, I'm fresh out of any "the" answers,
here.)

   There's also the very common situation:
" 
"                -------+      +---------+      +------
"                mail  M| SMTP |M mail  M| SMTP |M mail
"                srvr  T|------|T xfr   T|------|T srvr
"                      A|      |A       A|      |A     
"                -------+      +---------+      +------

wherein the middle box is considered "an intermediate MTA", and does
no gateway functions. It's not sufficiently clear, IMHO, whether OPES
intends for it to be appropriate for it to link to an OCP. (I think
most of us would say it shouldn't.)

   There are, of course, cases where outgoing email is filtered on a
separate machine from the MSA which initially receives it. I honestly
don't know whether most of us would consider that an "edge" process.

   So, bottom line, I don't have good answers, but I'm pretty sure we
should clarify whether there's agreement as to what SMTP-related
functions deserve to interact with "edge services".

--
John Leslie <[email-masked]>

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: Tony Hansen <[email-masked]>
Subject: Re: OPES and Email
Date: Fri, 8 Jul 2005 10:12:38 -0400
Cc: ietf-822 <[email-masked]>, [email-masked]
Sender: [email-masked]

On Thu July 7 2005 14:58, Tony Hansen wrote:
> 
> Heads up, email experts!
> 
> I don't know if you saw the recent I-D documents that came out, entitled
> OPES SMTP Use Cases (draft-ietf-opes-smtp-use-cases-02.txt).
[...]
> I'm co-chair of the OPES group, and am aware of its potential
> ramifications to the email world. So, I'd appreciate it if you could
> review the use-cases document. Discussion about the document is
> occurring on the [email-masked] mailing list. (If you'd rather
> not join that list, I can channel your comments.)

I saw and downloaded it, but hadn't had time to review it.  I have now
skimmed over it; here are some initial comments not covering issues
raised by others (e.g. unchecked spelling errors), based on a cursory
scan of the draft:

1. Check against I-D checklist and I-D Guidelines (e.g. via idnits):
   there is no IANA Considerations section -- one is required per
   I-D Guidelines.  Has any thought been given to IANA considerations?
   [A debate has been raging on the IETF discussion list for close to
   a month, however the current IESG guidelines require an IANA
   Considerations section].

2. Numeric references are deprecated because they lead to RFC Editor
   busy-work (renumbering).  See the rfc-interest mailing list archive.
   They also make review and use unnecessarily difficult; I know what
   "[RFC2821]" references, but "[8]" isn't helpful.

3. Examples (some already noted by others) should be checked; a valid
   and carefully formulated example can illustrate a specification, but
   a poorly formulated or invalid example simply causes confusion.  A
   message or message fragment should be checked for validity -- there
   are validating parsers available for authors' use (see
   http://tools.ietf.org (tools inventory and wiki)).  The draft contains:
      From: [email-masked]
      To: [email-masked]
      Subject: Test
      Hi, this is a test!
   An empty line is required between the message header and the body text
   (see RFC 2822, which would be suitable as a reference...).  A valid
   message (RFC 2822 again) MUST have a Date field in the message header,
   and that is missing from the example.

4. Use of standard terminology aids clarity (RFC 1958).  FYI 18 provides
   some general terminology; FYI 36 for security.  RFC 2822 is also
   apropos for terminology.   In particular, note the standard meaning of
   "header" as defined in FYI 18 and RFC 2822.

5. Figure 2 implies that there is a direct connection between an MDA and
   MUA.  An MDA typically stores messages in a message store of some sort,
   which is accessed by an MUA using some protocol (typically one of
   direct file access, a POP version, or an IMAP version).  It is unclear
   from the draft whether or not there is any intention to address those
   other core mail protocols, whereas the draft does mention completely
   unrelated protocols such as HTTP -- that is at best rather odd.

6. Speaking of MUAs, the draft mentions filtering at MTAs, which is a
   dubious practice (as briefly touched upon by some other reviewers;
   basically it violates the end-to-end architectural principle (RFC 1958
   and others))).  Filtering may also take place from an MUA, which is
   well situated to consider filtering characteristics appropriate for a
   particular mailbox user (unlike some random MSA or MTA).  However, this
   use is not mentioned by the draft, which *seems* reasonable as a
   recipient MUA doesn't use SMTP, however as it is a primary means of
   filtering, it probably warrants a mention in the Introduction (oh dear,
   now I have to choose the order of segues).

7. Speaking of SMTP, some mention should probably be made regarding
   SMTP variants, viz. whether or not the draft is to be considered
   applicable to submission and/or LMTP.

8. Speaking of the Introduction, it would be nice to use general
   terminology until OPES-specific jargon is first defined.
   unfortunately, the Introduction starts with obscure jargon (e.g.
   "callout"), which coupled with the obtuse references (e.g. "[3]"),
   non-standard use of terminology, and lack of definitions of specific
   jargon in section 2, makes understanding the draft quite difficult.

9. Careful review of the text is warranted; how exactly can an error
   message cause removal of an attachment ("The attachment is removed
   by an error message")?

10. I know what OPES is, but what is "OEPS"?

11. Some consistency, particularly in the generality of approach, with
   existing practices documents would be nice.  In particular, draft
   sect. 4.9 is specific to one peculiar access mechanism, whereas e.g.
   RFC 1344 suggests a more general MIME mechanism.  Speaking of RFC
   1344, please pay particular attention to the implications of
   message/partial and message/external-body for filters (which act as
   a type of gateway) [much of the rest of 1344 has been superseded].
   In light of RFC 1344 and several years after 
   http://www.kb.cert.org/vuls/id/836088 a draft regarding the topic of
   content/virus scanners that doesn't address the security issues is
   bordering on unconscionable.

12. Speaking of security, the draft Security Considerations section
   needs some careful thought.  It is woefully inadequate (see above,
   BCP 72 and FYI 18).

13. The recommended organization (instructions2authors) is to place
   Appendices before References, and that implies that the references
   sections should be unnumbered.

14. Categorization of references needs to be carefully reviewed.
   Considering that the topic is SMTP and therefore requires some
   understanding of the details of that protocol (seems to be missing
   on the part of the authors, based on numerous errors in the
   examples, as noted by others), RFC 2821 should be normative rather
   than merely informative, for example.

15. Please don't encourage "Add a mark to the subject of the email";
   see RFC 4096.  That is a layering violation; Subject, To, From,
   Cc, Reply-To, Date, etc. are originator-specified fields (RFC 2822)
   and modification by others w/o the express permission of the
   author(s) is nothing less than forgery.  Modification of message
   content also invalidates security mechanisms such as digital
   signatures (RFC 1847).  Because of those considerations, such
   practices should be thoroughly and firmly prohibited rather than
   encouraged.

16. Examples of client-server dialogs use "S:" and "R:" labels, which
   would be fine if the draft were the only document using such dialog
   examples.  It is not, and the typical use would be "S:" for server
   and "C:" for client.  Unfortunately, the draft choice associates "S:"
   with the opposite part of the dialog from the typical cases, and that
   can lead to confusion.

17. Formatting could be improved dramatically; e.g. page 4 is mostly
   wasted empty space.

18. (general comment on subject matter)  Frankly, filtering can be
   accomplished in any number of ways; there are methods which have
   been in use in practice for quite some time (e.g. the simple and
   old method of passing information to a program and examining the
   exit code) and there is at least one Standards Track method
   specifically for email filtering (Sieve).  It is unclear precisely
   what benefit would justify the overhead of a heavy-weight (layers of
   new protocols) method such as seems to be the case for OPES (I haven't
   yet retrieved all of the numbered references for review, but that's
   the impression I get).   A method for filtering email that doesn't
   integrate with Sieve, and doesn't even mention it is simply mind-
   boggling.  See also item 6 above, and consider the possibility that
   this work might be abandoned.  I realize that authors don't like to
   hear that, but I think this work is going to be difficult to salvage
   considering the architectural and layering issues as well as lack
   of integration with existing mail-related standards track
   specifications.

19. What about internationalization considerations?  How can some
   random MTA in Omaha reasonably filter content being sent from a
   Croatian emigre to relatives in the old country, for example?

I think most of the other items that I noticed have already been
mentioned.  The document needs quite a bit of work, some requiring
careful thought (layering issues, protocol issues, security issues,
integration with Sieve and other mail-related Standards Track
specifications, relationship to the Internet Architecture and other
approved RFCs), some mostly mechanical (proofreading, spelling and
grammar checks, validation of examples, chacks against I-D/RFC 
guidelines, instructions, and checklists).  It's going to need at least
another version before it's ready for thorough review, unless of course
it is simply dropped in light of the architectural/layering/integration
issues noted above.

Date: Fri, 08 Jul 2005 06:00:06 -0400
From: John C Klensin <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>, [email-masked]
Subject: Re: OPES and Email
Sender: [email-masked]

--On Friday, 08 July, 2005 04:27 -0400 "Robert A. Rosenberg"
<[email-masked]> wrote:

> 
> At 06:04 +0200 on 07/08/2005, Frank Ellermann wrote about Re:
> OPES and Email:
> 
>> What's wrong
>> with RCPT TO;<paul> and To: <sandra> ?  Maybe Paul was in a
>> Bcc:, that's nobody's business but his.
> 
> There is nothing wrong with the RCPT TO:<paul> [triggered by a
> BCC:<paul>] so long as there is also a RCPT TO:<sandra>. There
> is also the case where the message is a Mailing List (or
> .forward or alias triggered forward) fan-out where all of the
> RCPT TO command entries will have no corresponding To or Cc
> headers.

There is also the following case, which is _extremely_ common
and basically required...

  To: tom@domain1, dick@domain1, harry@domain2, sandra@domain3

In general, when this arrives at "domain2", there will only be
   RCPT TO:<harry@domain2>

similarly, when it arrives at domain1, there will only be
   RCPT TO:<tom@domain1>
   RCPT TO:<dick@domain2>

In general, only the originating MUA and the MSA sees the full
list of RCPT commands.  If either domain1 or domain2 see those
not bound for them in the cases above, we get (depending on how
those servers are configured) duplicate messages, mail loops, or
claims about invalid relaying.

        john

Date: Fri, 8 Jul 2005 05:15:34 -0400
To: IETF-SMTP <[email-masked]>
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: IP::DOMAIN Associations [Re: Bounce/System Notification Address Verification]
Sender: [email-masked]

At 00:39 -0400 on 07/05/2005, [email-masked] wrote about Re: 
IP::DOMAIN Associations [Re: Bounce/System Notification:

>Mostly because *SOME* of us are concerned about *actually* getting mail
>through, and we really have better things to do than to break our *own* user's
>mail just because their MUA is accidentally configured to send with a From:
>[email-masked] address, but to send to their ISP's mail server.

Unless your (ie: The VT.ETU) MSA Mail Servers do not listen on 
Port587 but only Port25 or do support SMTP-over-SSL on Port465, there 
is IMO no reason for their MUA to be pointing (accidentally or not) 
at their current CONNECTIVITY ISP's MSA server. The only VALID (again 
IMO) hypothetical reason to be using the CONNECTIVITY ISP's MSA 
server instead of VT.EDU's MSA would be, as I stated, due to VT.EDU 
not providing a Non-PORT25 MSA to accept email from CONNECTIVITY ISPs 
that do Outgoing-Port25-blocking (or the user using an old MUA that 
did not support designating what port to use for MSA upload 
connections).

Date: Fri, 8 Jul 2005 04:27:29 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: OPES and Email
Sender: [email-masked]

At 06:04 +0200 on 07/08/2005, Frank Ellermann wrote about Re: OPES and Email:

>What's wrong
>with RCPT TO;<paul> and To: <sandra> ?  Maybe Paul was in a
>Bcc:, that's nobody's business but his.

There is nothing wrong with the RCPT TO:<paul> [triggered by a 
BCC:<paul>] so long as there is also a RCPT TO:<sandra>. There is 
also the case where the message is a Mailing List (or .forward or 
alias triggered forward) fan-out where all of the RCPT TO command 
entries will have no corresponding To or Cc headers.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: OPES and Email
Date:  Fri, 08 Jul 2005 06:04:11 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 31
Sender: [email-masked]

Tony Hansen wrote:

> I hope this helps.

Somewhat, thanks also to Hector, in fact I must have missed it
when he talked about OPES somewhere before yesterday, or he
used a different term.

 From his post I got the vague idea that it's something between
SIQ and remote procedure calls, now after reading your article
I guess that it's not neessarily related to "remote".

It's like a plugin, only server instead of browser, isn't it ?
The 3238 abstract sounds like "junkbuster", what Hector said
combined with your info could be a "milter",

Okay, looking into it, Hector already said that the commands
in the 3.2.1 example need some angle brackets, and somewhere
in the intro you have "puggable".

I'm not immediately happy with "figure 2", what's that box
in the middle (mail gway) doing with my mail ?  What's wrong
with RCPT TO;<paul> and To: <sandra> ?  Maybe Paul was in a
Bcc:, that's nobody's business but his.

For the security considerations I miss "above all get some
legal advice".  Messing with in-transit mails goes straight
to the constitution in my country, and apparently harmless
ideas like RfC 2476bis 8.1 "MAY add Sender" are dangerous
without the explicit consent of the sender.  Bye, Frank

Date: Thu, 07 Jul 2005 22:51:59 -0400
From: Tony Hansen <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: OPES and Email
Sender: [email-masked]

An edge service is a service plugged into the side of an existing
service. For example, real time call outs from within HTTP servers or
SMTP servers to virus scanners or spam blockers are examples of edge
services.

An open pluggable edge service is an edge service that meets certain
criteria that allows them to be swapped in and out. For example, you
could swap out company X's virus scanner and replace it with company
Y's. Or cascade a series of virus scanners from multiple sources.

The OPES framework has defined such a set of criteria for edge
services. In addition, the OPES framework includes a profile for how
these services could be plugged into an HTTP server.

The RFCs 3238, 3752, 3835-3838 and 4037 all discuss OPES. In particular,
RFC 3752 covers a variety of use cases that are fairly generic. The
internet draft OPES SMTP Use Cases expands on the core set with use
cases that could be considered more specific to email.

I hope this helps.

	Tony Hansen
	[email-masked]

Frank Ellermann wrote:
> Tony Hansen wrote:
> 
>> I don't know if you saw the recent I-D documents that came out, 
>> entitled OPES SMTP Use Cases
>> draft-ietf-opes-smtp-use-cases-02.txt).
> 
> Yes, I saw the announcement, and then decided that "somebody" will 
> probably explain what OPES is, I've not the faintest idea.
> 
>> The Open Pluggable Edge Services (OPES) framework is application 
>> agnostic.
> 
> Yes, and I'm OPES agnostic... ;-)  So what is it ?  Bye, Frank

From: "Hector Santos" <[email-masked]>
To: "Frank Ellermann" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: OPES and Email
Date: Thu, 7 Jul 2005 22:39:26 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "Frank Ellermann" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, July 07, 2005 9:54 PM
Subject: Re: OPES and Email

> Tony Hansen wrote:
>
> > I don't know if you saw the recent I-D documents that came
> > out, entitled OPES SMTP Use Cases
> > (draft-ietf-opes-smtp-use-cases-02.txt).
>
> Yes, I saw the announcement, and then decided that "somebody"
> will probably explain what OPES is, I've not the faintest idea.

Simplistic terms from an SMTP standpoint - is a  CALL OUT - like a CBV or
SINK or "HOOK" or "SHIM" or a "BLACKBOX" whatever you might want to call it
that calls a sub-process and comes back with a result.

If you read my messages, I've talking about it for a while :-)

It will have all the same issues like TIMEOUT considerations as well, which
I pointed out in a few message may be addressed using the 1xz response codes
to keep a client alive with a OPES callout is done.

In layman terms is is a 'service' or proxy that can be called by "any
protocol" for whatever purpose.  The OPES SMTP Use Cases shows some of the
example "proxies" you can probably invent for SMTP.

CBV could be modeled as a OPES service for the MAIL FROM state,  in fact the
entire SMTP state machine can be modeled for OPES as so:

SMTP level OPES hooks:

     result = OPES_CONNECT(ip)
     result = OPES_HELO(ip,helo)
     result = OPES_MAIL(ip,helo,mfrom)
     result = OPES_RCPT(ip,helo,mfrom,rcpt)
     result = OPES_DATA(ip,helo,mfrom,rcpt,data)

accept post smtp OPES hooks:

     result = OPES_POST_SMTP(ip,helo,mfrom,rcpt,data)

OPES for SMTP will have technical issues such as:

     - Call out Timeouts
     - Extended Response Codes/Text integration

Pretty much the same as a CBV which is basically a OPES like concept.  From
a SMTP view point, a CBV is a OPES like concept.

You can also basically say SPF is a OPES hook too for the MAIL FROM and HELO
states:

     spf-result = OPES_SPF(ip,helo)
     spf-result = OPES_SPF(ip,mailfrom)

Same with SenderID/PRA for the DATA stata:

     senderid-result = OPES_SENDERID(ip,data)

All the extended ideas for SMTP would be for all intent and purposes OPES
SMTP Use Cases.

Hope this provides some clue. :-)

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: OPES and Email
Date:  Fri, 08 Jul 2005 03:54:06 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Cc: [email-masked]
Sender: [email-masked]

Tony Hansen wrote:
 
> I don't know if you saw the recent I-D documents that came
> out, entitled OPES SMTP Use Cases
> (draft-ietf-opes-smtp-use-cases-02.txt).

Yes, I saw the announcement, and then decided that "somebody"
will probably explain what OPES is, I've not the faintest idea.

> The Open Pluggable Edge Services (OPES) framework is
> application agnostic.

Yes, and I'm OPES agnostic... ;-)  So what is it ?  Bye, Frank

Date: Thu, 07 Jul 2005 14:58:31 -0400
From: Tony Hansen <[email-masked]>
To: ietf-822 <[email-masked]>, [email-masked]
Subject: OPES and Email
Sender: [email-masked]

Heads up, email experts!

I don't know if you saw the recent I-D documents that came out, entitled
OPES SMTP Use Cases (draft-ietf-opes-smtp-use-cases-02.txt).

As the abstract says:

   The Open Pluggable Edge Services (OPES) framework is application
   agnostic.  Application specific adaptations extend that framework.
   This document describes OPES SMTP use cases and deployment scenarios
   in preparation for SMTP adaptation with OPES.

The intent is to provide access from within email to the OPES filters
being used with HTTP. (Virus scanners are an example.)

I'm co-chair of the OPES group, and am aware of its potential
ramifications to the email world. So, I'd appreciate it if you could
review the use-cases document. Discussion about the document is
occurring on the [email-masked] mailing list. (If you'd rather
not join that list, I can channel your comments.)

Thanks!

	Tony Hansen
	[email-masked]

From: "Hector Santos" <[email-masked]>
To: "John C Klensin" <[email-masked]>, "Matti Aarnio" <[email-masked]>
Cc: "IETF-SMTP" <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Date: Thu, 7 Jul 2005 00:38:09 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "John C Klensin" <[email-masked]>

> Matti,
>
> I've got two reactions to your suggestion.  The first is that it
> should be a separate BCP:  2821 is already too long.  Adding
> this type of text to it would just further discourage reading,
> and "not reading" is the problem in the first place.
>
> The second is, while your list is excellent, virtually every
> point could be summarized by saying "before doing an SMTP
> implementation, it is really important to RTFM (RTFStandard?),
> because some things aren't as obvious as you think they are and
> are mostly there for good reasons".   That is, for me at least,
> profoundly depressing.

John,  would it make sense to have a consolidated guide, similar (but not
the same) to RFC 1113 for "host requirements?"

I often find the following IMC web page extremely useful as a quick summary
site as a quick reference to the various mail related guides:

http://www.imc.org/rfcs.html

General Comment regarding the RFC style and RTFM:

Although I can appreciate and understand why Postel molded the RFC as he did
(by reading the history),  one of the difficulties I see common with the RFC
style is many times it gets the target audience mixed up.  So it can often
be difficult (or not obvious) to extract information for one or it may be
too much for the other to grasp.

The RFC are often written as Functional or Technical specifications or
blends of each.   Functional Specs basically outline a model, idea or
protocol.  Technical Specs spell out the protocol and implementation
details.

Not a real big deal, but this could explain why you have ambitiguity at
times.

Here is a recent example of functional vs. technical spec conflicts which
ironically touches base with the syntax issue discussion we had here
regarding "Mail From:SP".  This was just submitted today:

Title  : OPES SMTP Use Cases
http://www.ietf.org/internet-drafts/draft-ietf-opes-smtp-use-cases-02.txt

See the C/S SMTP session example in this draft and you will see what I am
talking about.

Great for the general modeling example and description, but you can see the
how a simple technical mistake can be generated when someone sees this in
the documents.

Also, a potential timeout discussion for OPES call out sinks is left out.
Probably not required at this level but it probably should pointed out in a
comment since it may be important in OPES SMTP Use Cases.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Wed, 06 Jul 2005 23:27:25 -0400
From: John C Klensin <[email-masked]>
To: Matti Aarnio <[email-masked]>
cc: Hector Santos <[email-masked]>, IETF-SMTP <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Sender: [email-masked]

--On Wednesday, 06 July, 2005 14:58 +0300 Matti Aarnio
<[email-masked]> wrote:

>...
 
> I have a feeling that we need a new part on the specification,
> which in itself is informal, but..
> Well, or a separate BCP RFC ?
> 
> 
>   NNN. Pitfalls
> 
>   Following details have been seen to be escaping client
>   implementers attention  (this includes both user agents,
>   and servers as SMTP client):
>  
>    ...
> 
>    - Adding spaces into what specification does not specify
>      them, most commonly as:  "MAIL FROM: <...>"
>    - Sending MAIL and RCPT without angle brackets
>    - Sending utter junk and/or wrong syntax in HELO/EHLO line
>    - Not sending HELO/EHLO greeting at all
>    - Failing to do reply data collection properly, and just
>      presuming that single network read will always yield
>      all of the reply.
>...

Matti,

I've got two reactions to your suggestion.  The first is that it
should be a separate BCP:  2821 is already too long.  Adding
this type of text to it would just further discourage reading,
and "not reading" is the problem in the first place.

The second is, while your list is excellent, virtually every
point could be summarized by saying "before doing an SMTP
implementation, it is really important to RTFM (RTFStandard?),
because some things aren't as obvious as you think they are and
are mostly there for good reasons".   That is, for me at least,
profoundly depressing.

    john

Date: Wed, 06 Jul 2005 23:20:59 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, Matti Aarnio <[email-masked]>
cc: IETF-SMTP <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Sender: [email-masked]

--On Wednesday, 06 July, 2005 09:12 -0400 Hector Santos
<[email-masked]> wrote:

>> Matti,
>> 
>> None of  this is surprising, at least to me.  The question is
>> whether you think the SMTP spec should be modified to
>> _require_ servers to accept the sorts of "variations" you
>> outline and to explicitly authorize (and thereby encourage)
>> clients to send them?  And, if so, which ones and why?
> 
> May I ask, what are your plans or considerations for 2821bis? 
> 
> What new considerations are in it?

To all intents and purposes, 2821bis-00 is functionally
identical to 2821.  A great many errors and blocks of confusing
language have been fixed, and some particularly confusing and
disorganized text has been rewritten, reorganized, or replaced.
There are also some issues, largely unresolved in -00, about
what the Security Considerations section should actually cover
(see RFC 3552).  

But the revision/update has been done on the assumption that the
goal is to fix errors and clarify whatever needs to be clarified
and then get the thing to Draft, not to introduce new
specifications, considerations, or requirements that would
require recycling at Proposed.  

If you, or anyone else, are convinced that a revision is needed
that would bring 2821bis into conformance with ways in which you
believe mail should work differently from that which is required
by the generally-accepted readings of 2821 today, you had best
start organizing a WG Charter, people willing to do the work,
etc.  And those are discussions you will need to have with the
applicable ADs, not with me.

     john

From: "Hector Santos" <[email-masked]>
To: "John C Klensin" <[email-masked]>, "IETF-SMTP" <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Date: Wed, 6 Jul 2005 13:15:13 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "John C Klensin" <[email-masked]>

>> If 2821bis is about "cleaning", removing ambiguity,
>> reaffirming consistency, etc, this would be one of them.

> I think we disagree.  See above.

If there is one thing I am strongly opposed against is anything that is not
backward compatible.  So no, I am not suggesting a "requirement" for
anything.  Nonetheless, I think it is safe to say new considerations and
insights are needed.

Imagine the perspective of tomorrows developers writing a new SMTP server
simply because he doesn't like what he sees out there or just that he wants
to write one.

What does he/she have to know to best design the software? How to best
operate it?   Is the new 2821bis going to cover technical operational ideas
that are more current today?

What will you say about the space issue?

What will you say about the domain literal issue?

What will you say about the new authentication or authentication concepts?
or more so how they may alter SMTP considerations?

What will you say about the MAIL FROM spoofing issue?

Will you update section 7.1, in particular the last statement?

7.1 Mail Security and Spoofing
   ....
   This specification does not further address the authentication
    issues associated with SMTP other than to advocate that
    useful functionality not be disabled in the hope of providing
    some small margin of protection against an ignorant user who
    is trying to fake mail.

so on an so on.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: John C Klensin <[email-masked]>, [email-masked], IETF-SMTP <[email-masked]>
From: [email-masked]
Subject: Re: Re: RFC2821bis
Date: Wed, 6 Jul 2005 17:03:52 +0100
Sender: [email-masked]

Same start as my last email.
>>
>> Thanks John
>>
>> Your comments are all very enlightening.
>> And i think I have to agree with all of them.
>> Your knowledge shines trough them all.
>>
>> I can not wait to study your new 2821bis-draft
>>
>> Is there a delay in this pipeline?
>
> Not a significant one.  The draft has been through a series of
> nasty transformations of format, with attempts to track every
> change in the text so it could be explained if anyone asked.
> I'm now -- between interruptions, e.g., to try to explain why it
> would not be a good idea to start putting a lot of largely
> untested anti-spam theories into SMTP requirements -- checking
> the post-able version, paragraph by paragraph, to try to make
> sure nothing got lost.  As soon as I've completed that process,
> or conclude that the rest of it should be left to all of you,
> the draft will be posted.

> Note that any suggestions made now will be considered for -01
> (presumably after IETF, given schedules and posting deadlines).
> If I try putting more things in now, it just won't get up before
> next Monday.   So, if you, or others, want to send change
> suggestions along, that is fine, but (i) don't expect to see
> them reflected in this draft and (ii) assume that I will defer
> anything that attempts to change the basic email model to
> reflect a "killing spam is more important than interoperability
> for normal messages" mentality for mailing list discussion.  See
> below for an explanation of part of this.

>> While I recently was studying the old RFC2821 I found a
>> strange MUST about relay-hosts.

...(snip)...

> First, it isn't "strange".  2821 maintains a fairly strong
> distinction between "relays" and other sorts of creatures.
> Relays do not mess with message bodies and especially do not do
> so without explicit permission from either the sender or
> receiver.  There are many reasons for this, but primary among
> them are exactly that chain of responsibility and associated
> questions of intent.  To give a couple of handy examples from
> the virus space...

... snip ...

Your reasoning is correct .

>> Another (textual) suggestion:
>>
>> Split the old big  RFC 2821 sections up in smaller ones.
>> (it makes discussion about them easier)

> A bit of that has been done, and some artificially split
> materials have been combined and condensed.  Otherwise, see
> comment above about changes for draft -00 and note that specific
> suggestions (of the character of "split section 99.3 after the
> second paragraph into 99.3.1 and 99.3.2") are far more helpful
> than the general observation above.

I was just testing the water :)
Split up section 4.4
in a general, return-path header and a received header part.

Split up section 5
diferend sections for the different SMTP creatures

>> Also to benefit the discussion:
>> Make seperate sections for
>> Originator systems (is this not the same as an RFC 2476- MSA?)
>> Delivery systems
>> Relay systems
>> and
>> Gateway Systems
>>   (as mentioned in section 2.3.8)

> Specific suggestions welcome, but 2.3.8 is only two paragraphs
> long.  If you are suggesting a more extensive treatment, suggest
> text and see if you can get agreement.  Note in doing so that
> this puppy is already over 80 pages long; few of us who need to
> read these things would welcome a 100 page spec and I'd really
> welcome identification of things than can come out instead of,
> or in addition to, suggestions to put more things in.

It is just becaurse
Originator systems, Delivery systems,Relay systems and Gateway Systems
are different kids of creatures so it is usefull to know which rules (don't) apply to which creature, in RFc 2821 it is all mashed together.

To scrap:
General
everything about originating systems could be removed. just a referal to RFC 2476 would do.
(Ps this is not a lot)

section 4.2.1
Is this section special to SMTP or is this section copied from another RFC?
If it is copied just a reference to there will do.

section 4.3.2
everything except the introduction.

> regards,
>   john

Thanks again

Willemien

Date: Wed, 06 Jul 2005 11:22:01 -0400
From: John C Klensin <[email-masked]>
To: [email-masked], Matti Aarnio <[email-masked]>
cc: Hector Santos <[email-masked]>, IETF-SMTP <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Sender: [email-masked]

--On Wednesday, 06 July, 2005 10:55 -0400
[email-masked] wrote:

> On Wed, 06 Jul 2005 12:42:33 +0300, Matti Aarnio said:
> 
>> commonly, without brackets, etc.    Some machines are clearly
>> behind some NAT box:  "EHLO [192.168.0.200]"  (yup, it had
>> correct brackets)
> 
> It's things like that which caused the RFC prohibition on
> bouncing mail due to the EHLO value - that machine was being
> as strictly compliant as it could.
> 
> On the other hand, I could *easily* make the case that you
> should reject on site any 'MAIL FROM: <..' crud from certain
> vendors as a security measure. That sort of blatant inability
> to code per spec indicates a site that should be blacklisted
> because the next time you hear from them, it's likely to be Yet
> Another Worm/Virus (if the one you're looking at isn't one
> itself).

For whatever it is worth, it seems to me that this is _exactly_
the right way to draw the distinctions in the standard.
Behavior that is the best possible out of an SMTP client or
server should not be banned.  That is consistent with
robustness, maximum conformance, and general good sense.  And,
if a host has no global DNS name and only a NAT-supplied private
space address, I'd rather see the domain literal above than
either a lie or 
   EHLO Jones-Family-801.11b

Conversely, if the spec is clear and imposes a requirement that
has been there for 20 years, and that requirement is easy to
conform to if only one manages to read the spec, punishing
non-spec-reading by, at least, making the behavior
non-conformant, seems entirely rational.  As you imply, such
clients, and the sites that run them, will probably exhibit
worse behavior, and behavior that is more indifferent to others,
on the next message.

     john

To: Matti Aarnio <[email-masked]>
Cc: John C Klensin <[email-masked]>, Hector Santos <[email-masked]>, IETF-SMTP <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM: 
From: [email-masked]
Date: Wed, 06 Jul 2005 10:55:25 -0400
Sender: [email-masked]

--==_Exmh_1120661725_5186P
Content-Type: text/plain; charset=us-ascii

On Wed, 06 Jul 2005 12:42:33 +0300, Matti Aarnio said:

> commonly, without brackets, etc.    Some machines are clearly behind
> some NAT box:  "EHLO [192.168.0.200]"  (yup, it had correct brackets)

It's things like that which caused the RFC prohibition on bouncing mail
due to the EHLO value - that machine was being as strictly compliant as
it could.

On the other hand, I could *easily* make the case that you should reject on
site any 'MAIL FROM: <..' crud from certain vendors as a security measure. That
sort of blatant inability to code per spec indicates a site that should be
blacklisted because the next time you hear from them, it's likely to be Yet
Another Worm/Virus (if the one you're looking at isn't one itself).

OK.  Maybe I'm just tired of MyDoom :)

--==_Exmh_1120661725_5186P
Content-Type: application/pgp-signature

--==_Exmh_1120661725_5186P--

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>, "Frank Ellermann" <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Wed, 6 Jul 2005 10:43:50 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "Frank Ellermann" <[email-masked]>

> If you know that say TLD .invalid is invalid then you don't
> need a DNS lookup.  And if you're sure that the <toplabel>
> cannot be 1*DIGIT it's almost the same situation.

Right.  So once this issue of the TLD for a valid domain can not
be in the range of 0 to 255, then we have a good solid test. The
question is can this ambiguity be settled for 2821bis.  I don't know.
I like to believe ICANN is not going to be that stupid.

> The latter case includes all recursive NXDOMAIN problems.
> Of course you're not forced to reject any SPF errors.

Right, what I am saying that if SMTP system does begin to support SPF or
anything else for that matter,   the SMTP "2821bis" document should have a
discussion item that might say something like:

   "Although out of scope for this document, any new
    HELO/EHLO level validaiton technology
    implemented should consider the DNS overhead
    potential for open-ended lookups. Applying syntax field
    validation checking may apply here."

   "For implementators with new DNS-based proposals
    based on HELO checking, they should heed the
    warning of placing a network DNS burden with
    open-ended lookups."

Or something of that nature.

> Thinking about it, this domain literal stuff is really
> tricky.  If you want to check domain literal == IP you
> must decode it.  At least I now see why 2821 allowed the
> leading zeros, we can't remove them in 2821bis.  <sigh />

>From what I can see from implementation experience, is  the ambiguiity at
the moment is where #.#.#.# (no brackets) can be viewed as possible domain.

> Plan B for known traps and pitfalls:  simply document it.
> Documented "bugs" are by definition features.  It's not
> bad enough for some new "security considerations".  Bye

Right.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Wed, 06 Jul 2005 10:38:36 -0400
From: John C Klensin <[email-masked]>
To: [email-masked], IETF-SMTP <[email-masked]>
Subject: Re: RFC2821bis
Sender: [email-masked]

--On Wednesday, 06 July, 2005 13:56 +0100
[email-masked] wrote:

> 
> Thanks John
> 
> Your comments are all very enlightening.
> And i think I have to agree with all of them.
> Your knowledge shines trough them all.
> 
> I can not wait to study your new 2821bis-draft
> 
> Is there a delay in this pipeline?

Not a significant one.  The draft has been through a series of
nasty transformations of format, with attempts to track every
change in the text so it could be explained if anyone asked.
I'm now -- between interruptions, e.g., to try to explain why it
would not be a good idea to start putting a lot of largely
untested anti-spam theories into SMTP requirements -- checking
the post-able version, paragraph by paragraph, to try to make
sure nothing got lost.  As soon as I've completed that process,
or conclude that the rest of it should be left to all of you,
the draft will be posted.

Note that any suggestions made now will be considered for -01
(presumably after IETF, given schedules and posting deadlines).
If I try putting more things in now, it just won't get up before
next Monday.   So, if you, or others, want to send change
suggestions along, that is fine, but (i) don't expect to see
them reflected in this draft and (ii) assume that I will defer
anything that attempts to change the basic email model to
reflect a "killing spam is more important than interoperability
for normal messages" mentality for mailing list discussion.  See
below for an explanation of part of this.

> While I recently was studying the old RFC2821 I found a
> strange MUST about relay-hosts.
> 
> RFC2821 section 3.7 last bit:
> 
> As discussed in section 2.4.1, a relay SMTP has no need to
> inspect or act upon the headers or body of the message data
> and MUST NOT do so except to add its own "Received:" header
> (section 4.4) and, optionally, to attempt to detect looping in
> the mail system (see section 6.2).
 
> This MUST also implies that relay hosts that do inspect the
> message data MUST NOT remove virusses and the likes from
> emails it relays.
> 
> Please comment on this.

First, it isn't "strange".  2821 maintains a fairly strong
distinction between "relays" and other sorts of creatures.
Relays do not mess with message bodies and especially do not do
so without explicit permission from either the sender or
receiver.  There are many reasons for this, but primary among
them are exactly that chain of responsibility and associated
questions of intent.  To give a couple of handy examples from
the virus space...

	(i) There are people to whom one might reasonably want
	to send viruses, and who might want to receive them.
	Examples include anti-virus evaluators and researchers
	when a possibly-new virus is discovered.  Some
	semi-random relay intervening and destroying the virus
	would not be A Good Thing.  One could claim that such
	sites would choose their MX list and relays carefully so
	as to avoid virus-deleters, but there are some subtle
	edge cases, and 2821 tries to maintain a bright line.
	
	(ii) Other than actually running one in a clean room
	environment and seeing what it does (and maybe not
	then), no virus detection mechanism is 100%.  Like
	almost anything else, they involve the risk of letting
	some viruses through and of getting false positives.
	One really doesn't want a _relay_ intercepting one's
	mail, deciding there is a virus present, trashing the
	message and either sending a "well, you got a note, but
	it contained a virus, so we trashed it" message to the
	recipient or, given other discussions on this list,
	silently dropping the message as presumably hostile. 

> Another (textual) suggestion:
> 
> Split the old big  RFC 2821 sections up in smaller ones.
> (it makes discussion about them easier)

A bit of that has been done, and some artificially split
materials have been combined and condensed.  Otherwise, see
comment above about changes for draft -00 and note that specific
suggestions (of the character of "split section 99.3 after the
second paragraph into 99.3.1 and 99.3.2") are far more helpful
than the general observation above.

> Also to benefit the discussion:
> Make seperate sections for
> Originator systems (is this not the same as an RFC 2476- MSA?)
> Delivery systems
> Relay systems
> and
> Gateway Systems
>   (as mentioned in section 2.3.8)

Specific suggestions welcome, but 2.3.8 is only two paragraphs
long.  If you are suggesting a more extensive treatment, suggest
text and see if you can get agreement.  Note in doing so that
this puppy is already over 80 pages long; few of us who need to
read these things would welcome a 100 page spec and I'd really
welcome identification of things than can come out instead of,
or in addition to, suggestions to put more things in.

regards,
   john

From: "Hector Santos" <[email-masked]>
To: "Matti Aarnio" <[email-masked]>, "John C Klensin" <[email-masked]>
Cc: "IETF-SMTP" <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Date: Wed, 6 Jul 2005 10:04:12 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "Matti Aarnio" <[email-masked]>

> Yes John, we have been at this for way too long, and have seen
> everything...  I just wish  SPF et.al. specifiers have equally
> old hands.

So you haven't seen everything? :-)

You have seen everything, but like me for a long time, we were stuck in a
time warp of not doing anything about it for a long time.

For me and for many, the last straw  was when SORBIG started to forge
addresses and exploiting every loophole in the SMTP level.

SPAM? I could care less, thats an administrative/user policy.

I am more interesting in securing the SMTP TMS (Transaction Management and
Security) process to the highest degree possible
with maximum backward compatibility.

> I have a feeling that we need a new part on the specification,
> which in itself is informal, but..
> Well, or a separate BCP RFC ?

Maybe the ultimate way, especially if the 2821bis document isn't going to
address any of the TMS issues.   If so, I don't mind saying I would be
greatly disappointed because to me, there is some form of stubborness and
even incompetence in not understand the reality of today.   Any SMTP "old
hand" that chose to ignore the MARID, SPF and SENDERID/PRA, well, to me, is
disturbing.  I have full respect for people, but we can't ignore these
issues any more.  Something got to give.

So I personally prefer if there is going to be a new 2821bis document, I
would like it to include the new considerations.

>   Following details have been seen to be escaping client
>   implementers attention  (this includes both user agents,
>   and servers as SMTP client):

My comments are geared towards the significant level as
far as TMS in a vendor that have to operate with other SMTP servers as well.

>    - Adding spaces into what specification does not specify
>      them, most commonly as:  "MAIL FROM: <...>"

Due to the wide spread. Tolerance is preferrable. This is
the current status in ~100% of all mail servers.

>    - Sending MAIL and RCPT without angle brackets

Instant reject in the majorit of servers.

>    - Sending utter junk and/or wrong syntax in HELO/EHLO line

Due the existence relaxed provision in HELO, most servers ignore everything
about the client domain name.

We need to address this at the most simplistic and purist level. I cited two
possible ways that I believe should not have any effect on reliable
operations:

        - domain literals syntax and IP verifcation
        - Local Domain Spoofing.

>    - Not sending HELO/EHLO greeting at all

Inject STATE machine violation.

>    - Failing to do reply data collection properly, and just
>      presuming that single network read will always yield
>      all of the reply.

Each system handles this differently.  But technically this is a state
machine violation, so many systems have a count of the number of sequential
50x errors and do a drop.

The problem with drops is that the sender might call back again.  So you
have to be careful.

A good dicussion item (which if I remember, is already there), but it might
be worth a revisit.

>    - Missing the requirement to support multiline replies
>      from servers, and that in multiline replies the last
>      line value is what determines the reply.

This is a BIG item for stopping BULK senders who try to blast/stream SMTP
sessions.

What I have seen here is that the sender drops on their own when they don't
see the final non-dash response code.

So I don't think you have any control here because it is the sender that is
broken and will drop.

Nonetheless, a good discussion item about developer and operations who offer
multiple/extended response lines.

>    - Missing entirely reply-code principles, e.g. that their
>      treatment MUST go by the first digit, deeper semantics
>      may be applied, but basic codes are so overloaded, that
>      only first digit is usually meaningfull.

I have seen issues here, but I suspect you are trying to use
extended codes to define text presentations or meanings?

If anything, is that that senders might not honor temporary vs permanent
response codes. I believe this is already discussed
and well understood.

This might be a good discussion item about "GreyListing" considerations.

>    - Not showing protocol exchange and remote end replies to user,
>      when sending fails for whatever reason  (so that user can
>      report them to helpdesk - or see right away, why the message
>      or one of its recipients didn't go thru.)
>      For 99% of things, showing last sent protocol line, and received
>      error reply for it will do.  For the remaining 1% a full
>      protocol exchange from the connection down to close is needed.

Ok,  however, keep in mind that there is good reasons to not "feed the
hackers/spammers" more information then they need.

But IMO, I don't see this as a SMTP thing.  It is a administrative concept.

>    - Single MAIL-transaction can not have more than 100 recipients,
>      when more are to be sent, multiple transactions are needed.
>      (Most servers allow more, some allow a lot less..)

This is already convered in implementators as options.

However, I think they might be a discussion in how RSET is used to restart a
session.

>    - If server advertises EHLO capability to do user authentication,
>      it does not mean the client must do it.
>      Especially strange client behaviour is that when the server
>      does not advertise said capability, clients won't try to
>      authenticate and just send message without trouble.

ESMTP is an option.  Not a requirement.

Is this more of a RFC 2476 issues?

>    - When server does not advertise some capability in EHLO reply,
>      client MUST NOT go ahead blindly and try to use it anyway.
>      Blind AUTH attempts are common.

Isn't this a STATE machine issue?   This is covered I think.

>    - There is only one (deprecated) case (port number), where
>      in SMTP protocol extensions the TLS stream encryption is
>      activated up front, in all other cases the processing
>      follows STARTTLS specification.

RFC 2476?

>   Following details have been seen to be escaping
>   server implementers attention:
>
>    - There is NO limit in the number of MAIL-transactions per
>      single connection.  High-efficiency MTAs will send entire
>      accumulated queue in as few connections as possible.

Ok, RSET considerations? Limits discussion for operators.

>    - Requiring sender authentication at inbound MX servers for
>      inbound message traffic is serious configuration mistake.

Expand please?   if I understand, we have a basic backward
compatibility issue:

     router (authorization required)
     vs
     final destination (no authorization required)

>    - Server may need radically different service subsets at
>      different ports, e.g. SMTP does not support STARTTLS, nor
>      any AUTH, whereas SUBMIT supports STARTTLS, and different
>      sets of AUTH depending on TLS being active or not.
>      Non-standard plain-text authentications should not be
>      available except under TLS encryption.

Ok, but these are pre-arrangements issues.

>    - Servers don't separate DNS timeouts and SERVFAILs from
>      genuine NXDOMAIN replies, and in all cases report permanent
>      five-hundred series rejects, where proper is temporary
>      four-hundred series retry.

Our server does separate the two.  I think this should be an excellent
discussion item.

>    - Servers pay too much attention on EHLO greeting data,
>      which from behind a NAT box may well be other than what
>      the server sees.

Example?

If I understand you,  right now,  servers don't do anything with the client
domain name.   That's the key issue here IMO.

For our implementation, this is one of these reported reasons why some
operations turn off the domain literal checker, because they see one or two
false positives.  But at the same time, once they turn off it off,  they see
a higher increase of spam activity.

So in our experience, the problem is often reduced to white list issue since
the sender who might have this NAT/Firewall issue, is often known..

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: Hector Santos <[email-masked]>, "John C Klensin"  <[email-masked]>, <[email-masked]>
From: [email-masked]
Subject: Re: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Wed, 6 Jul 2005 15:00:19 +0100
Sender: [email-masked]

Hector

Are you not talking about MSA (RFC2476)/ originating systems that SHOULD be more willing to accept "curious not really conforming SMTP" becaurse they interact  with user clients/ MUA / and the like?

while the SMTP in 821and  2821 is in priciple about mailtransfer in a host-to-host environment were you can be much strickter.
How much in your non-conforming list were relay systems and how much were originating systems? (to use the RFC2821 terminology of section 2.3.8)

<<-- original message -->>

John,

I don't think I am suggesting anything that is not A) backward compatible,
B) "nefarious" in providing a new level of fuzziness.

No, quite the opposite.  I have a product in the market so it serves me no
interest to "break" operations.   I am as pure as you can get in
understanding the purity of transport/interchange model. I've done it for
atleast 3-4 networks.

What I am merely highlighting are new possible considerations that may be
"generic" and "independent" and strictly SMTP based.

For one specific issue:

HELO #.#.#.#

I believe you are saying that we don't have a reliable way independent of
DNS to determine if this is valid or not.

If so, then we solve that issue.  However, does this say you have don't
include a discussion item in 2821bis about this?   I believe you can't
exclude such a item.

And so on with other similar considerations.  Again, please hear what I am
saying,  I am NOT suggesting anything that is "fuzzy" or random in its
principle idea.  I am talking raw SMTP level considerations.

In my view, this simple consideration alone will GO a LONG way in assisting
all the new proposals, including Dave Crocker's CSV.   The one big reason it
suffers (as well as SPF HELO provisions) is because we still have a relaxed
provision for HELO, and what I am saying, is even though that might continue
to be the consider, how much of it can be classified as invalid at the SMTP
level.

So we are not talking about requirements, but new considerations.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message -----
From: "John C Klensin" <[email-masked]>
To: "Hector Santos" <[email-masked]>; <[email-masked]>
Sent: Tuesday, July 05, 2005 4:37 PM
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO

>
>
>
> --On Tuesday, 05 July, 2005 13:00 -0400 Hector Santos
> <[email-masked]> wrote:
>
> >         No Brackets  ==>  presumed DOMAIN
> >
> > is not a practical idea to implement because systems will
> > endure a very high volume of HELO hits such as:
> >
> >         HELO #.#.#.#
> >
> > So what I have been trying to get a focus on is the syntax
> > field validation considerations not only for SMTP servers
> > adopting these new DNS proposals, but also view them as new
> > general SMTP related issues that may apply regardless if a
> > system implements one or more of the DNS-based proposals.
>
> Sigh.
>
> > Here is a small set of invalid HELO/EHLO captured in the first
> > two hours of July 3:
> >...
> > If you use the No Bracket == DOMAIN rule, this would produce a
> > high DNS lookup overhead when you implement a DNS-based
> > proposal.
>
> Uh... So?  2821 doesn't require you to do a DNS lookup on the
> HELO/EHLO domain at all.  It also doesn't require you to
> syntax-check it.  If you do decide to syntax-check it, it
> doesn't require that you reject [1.2.3.X] or even [1.2.3.4.5]
> nor that you accept 1.2.3.Y.4.  Regardless of which of these you
> do (or don't) do, it significantly restricts the conditions
> under which you can reject the command entirely: only the two
> domain literal forms represent invalid syntax.
>
> Of course, you can invoke the "protect myself from attack" rule,
> at which mpoint any of the above can presumably be rejected, and
> rejected without calling on the DNS.
>
> 2821 is a standard for mail transport and interchange.  It is
> very clear (I hope) that you are not obligated to accept mail
> that you consider nefarious.  Criteria for determining
> nefariousness just don't belong in the transport standard: if
> nothing else, they are too controversial and too transitory.
>
> > I don't believe this has anything to do with semantics.  A
> > SMTP system designed to address today's needs can benefit by
> > having rather logical and simple syntax field validation
> > considerations.
>
> Sure.  But beyond the above, IMO, not in the standard.  The
> problem with the sort of rules you are trying to develop is that
> "looks suspicious" can easily evolve to exclude all sorts of
> valid things -- TLDs that weren't valid a few years ago, but are
> now, domain names with odd constructions (e.g., not many years
> ago, if you saw a DNS label with two consecutive hyphens, it was
> probably bogus; today it may be part of an important standard),
> local parts with "odd" characters in them like "+" or "=" or
> "/", and so on.   So we get back to the bottom line of how much
> risk you want to run of rejecting legitimate mail in order to
> stop spammers.  2821 isn't about that: it is about how a client
> and server deal with each other when the primary purpose of both
> is to get presumably-legitimate mail through to a selected
> destination mailbox.  All of other cases, remedies for them,
> are, IMO, appropriately outside the standard.
>
> > Hope this explains it better.
>
> It was actually quite clear.
>
>     john
>
>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  toplabel (was: Strict RFC x821 Compliant: HELO/EHLO)
Date:  Wed, 06 Jul 2005 15:34:30 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 53
Sender: [email-masked]

John C Klensin wrote:
 
> While I agree, please be _very_ careful here.

Yes, as soon as it's hardwired in millions of applications
the chances to modify it are lousy.  The USEFOR WG has a lot
of fun with the IPv6 colons, because decades ago RfC 1036
said that the colon among other punctuation is a separator.

OTOH dancing around such issues as far as possible is not
always possible.  At the moment the RfC 2821 syntax allows 
digits and dots only.  RfC 821 still had an <a> (alpha) at
the begin of all labels.  RfC 2821 wants at least one dot,
RfC 821 didn't insist on a dot.  Some fine tuning between
these positions should be okay for 2821bis.  E.g. use the
<toplabel> as found in 3696 or 2396.

It's not absolutely necessary for the purposes of 2821bis,
but it makes sense elsewhere.  Like the <CRLF> in 2234bis,
where in theory a CR / ( [CR] LF ) would be good enough -
in practice it's better to stick to one simple convention.

> In some cases, that involved a lexical rule (e.g., "we
> know there are no TLDs more than four characters long

That's a possible interpretation of "extremely unlikely"
in RfC 1591.  It turned out to be wrong.  Hardwired lists
were worse.

> in actuality, because there is a firm prohibition on TLD
> names being all-digits, it is _lots_ safer to handle one
> of those as an error without looking it up than to assume
> that .invalid, which is invalid today, will continue to
> be invalid a year from now.

For invalid / test / example / localhost it's in BCP 32, if
somebody "unreserves" invalid or example in his LAN it's a
very bad idea.  

For the <toplabel> I can't tell at the moment which BCP or
standards track RfC guarantees it - 3696 is informational,
1123 and 2181 apparently (?) don't say it, 2396 is obsolete.

Oops, 1738 is still okay, waiting for new file / ftp / news
and nntp URI RfCs, then it will be moved to historic.  All
other 1738-URL-schemes are already extracted into new RfCs.

If 2821bis doesn't define a <toplabel> it's either lost, or
it's hidden somewhere deep within 1035, where I won't find
it - in that case a fresh definition in 2821bis can't hurt.

                    Bye, Frank

From: "Hector Santos" <[email-masked]>
To: "John C Klensin" <[email-masked]>, "Matti Aarnio" <[email-masked]>
Cc: "IETF-SMTP" <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Date: Wed, 6 Jul 2005 09:12:50 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message ----- 
From: "John C Klensin" <[email-masked]>

> Matti,
> 
> None of  this is surprising, at least to me.  The question is
> whether you think the SMTP spec should be modified to _require_
> servers to accept the sorts of "variations" you outline and to
> explicitly authorize (and thereby encourage) clients to send
> them?  And, if so, which ones and why?

May I ask, what are your plans or considerations for 2821bis? 

What new considerations are in it?

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: "John C Klensin" <[email-masked]>, <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Wed, 6 Jul 2005 09:05:59 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

John,

I don't think I am suggesting anything that is not A) backward compatible,
B) "nefarious" in providing a new level of fuzziness.

No, quite the opposite.  I have a product in the market so it serves me no
interest to "break" operations.   I am as pure as you can get in
understanding the purity of transport/interchange model. I've done it for
atleast 3-4 networks.

What I am merely highlighting are new possible considerations that may be
"generic" and "independent" and strictly SMTP based.

For one specific issue:

        HELO #.#.#.#

I believe you are saying that we don't have a reliable way independent of
DNS to determine if this is valid or not.

If so, then we solve that issue.  However, does this say you have don't
include a discussion item in 2821bis about this?   I believe you can't
exclude such a item.

And so on with other similar considerations.  Again, please hear what I am
saying,  I am NOT suggesting anything that is "fuzzy" or random in its
principle idea.  I am talking raw SMTP level considerations.

In my view, this simple consideration alone will GO a LONG way in assisting
all the new proposals, including Dave Crocker's CSV.   The one big reason it
suffers (as well as SPF HELO provisions) is because we still have a relaxed
provision for HELO, and what I am saying, is even though that might continue
to be the consider, how much of it can be classified as invalid at the SMTP
level.

So we are not talking about requirements, but new considerations.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message -----
From: "John C Klensin" <[email-masked]>
To: "Hector Santos" <[email-masked]>; <[email-masked]>
Sent: Tuesday, July 05, 2005 4:37 PM
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO

>
>
>
> --On Tuesday, 05 July, 2005 13:00 -0400 Hector Santos
> <[email-masked]> wrote:
>
> >         No Brackets  ==>  presumed DOMAIN
> >
> > is not a practical idea to implement because systems will
> > endure a very high volume of HELO hits such as:
> >
> >         HELO #.#.#.#
> >
> > So what I have been trying to get a focus on is the syntax
> > field validation considerations not only for SMTP servers
> > adopting these new DNS proposals, but also view them as new
> > general SMTP related issues that may apply regardless if a
> > system implements one or more of the DNS-based proposals.
>
> Sigh.
>
> > Here is a small set of invalid HELO/EHLO captured in the first
> > two hours of July 3:
> >...
> > If you use the No Bracket == DOMAIN rule, this would produce a
> > high DNS lookup overhead when you implement a DNS-based
> > proposal.
>
> Uh... So?  2821 doesn't require you to do a DNS lookup on the
> HELO/EHLO domain at all.  It also doesn't require you to
> syntax-check it.  If you do decide to syntax-check it, it
> doesn't require that you reject [1.2.3.X] or even [1.2.3.4.5]
> nor that you accept 1.2.3.Y.4.  Regardless of which of these you
> do (or don't) do, it significantly restricts the conditions
> under which you can reject the command entirely: only the two
> domain literal forms represent invalid syntax.
>
> Of course, you can invoke the "protect myself from attack" rule,
> at which mpoint any of the above can presumably be rejected, and
> rejected without calling on the DNS.
>
> 2821 is a standard for mail transport and interchange.  It is
> very clear (I hope) that you are not obligated to accept mail
> that you consider nefarious.  Criteria for determining
> nefariousness just don't belong in the transport standard: if
> nothing else, they are too controversial and too transitory.
>
> > I don't believe this has anything to do with semantics.  A
> > SMTP system designed to address today's needs can benefit by
> > having rather logical and simple syntax field validation
> > considerations.
>
> Sure.  But beyond the above, IMO, not in the standard.  The
> problem with the sort of rules you are trying to develop is that
> "looks suspicious" can easily evolve to exclude all sorts of
> valid things -- TLDs that weren't valid a few years ago, but are
> now, domain names with odd constructions (e.g., not many years
> ago, if you saw a DNS label with two consecutive hyphens, it was
> probably bogus; today it may be part of an important standard),
> local parts with "odd" characters in them like "+" or "=" or
> "/", and so on.   So we get back to the bottom line of how much
> risk you want to run of rejecting legitimate mail in order to
> stop spammers.  2821 isn't about that: it is about how a client
> and server deal with each other when the primary purpose of both
> is to get presumably-legitimate mail through to a selected
> destination mailbox.  All of other cases, remedies for them,
> are, IMO, appropriately outside the standard.
>
> > Hope this explains it better.
>
> It was actually quite clear.
>
>     john
>
>

To: John C Klensin <[email-masked]>, IETF-SMTP <[email-masked]>
From: [email-masked]
Subject: RFC2821bis
Date: Wed, 6 Jul 2005 13:56:55 +0100
Sender: [email-masked]

Thanks John

Your comments are all very enlightening.
And i think I have to agree with all of them.
Your knowledge shines trough them all.

I can not wait to study your new 2821bis-draft

Is there a delay in this pipeline?

While I recently was studying the old RFC2821 I found a strange MUST about relay-hosts.

RFC2821 section 3.7 last bit:

As discussed in section 2.4.1, a relay SMTP has no need to inspect or
act upon the headers or body of the message data and MUST NOT do so
except to add its own "Received:" header (section 4.4) and,
optionally, to attempt to detect looping in the mail system (see
section 6.2).

This MUST also implies that relay hosts that do inspect the message data MUST NOT remove virusses and the likes from emails it relays.

Please comment on this.

Another (textual) suggestion:

Split the old big  RFC 2821 sections up in smaller ones.
(it makes discussion about them easier)

Also to benefit the discussion:
Make seperate sections for
Originator systems (is this not the same as an RFC 2476- MSA?)
Delivery systems
Relay systems
and
Gateway Systems

(as mentioned in section 2.3.8)

From: "Hector Santos" <[email-masked]>
To: "Matti Aarnio" <[email-masked]>, "John C Klensin" <[email-masked]>
Cc: "IETF-SMTP" <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Date: Wed, 6 Jul 2005 08:49:48 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

You provided a good outline and I wish to review them more today.

My point is that there "might" be some of these that may be part of the new
2821bis and I am not suggesting it be "requirements" but discussion items
and insights for security and interoperability.

Why?

Because while it might suffice as a new BCP document, as Paul say, "We just
need to make everyone read it... "

I believe the following is the best compromise for backward compatibility
and for new TMS (Transaction Management and Security) considerations at the
2821bis level:

1)  SMTP syntax considerations.

2)  Client/Server handshaking Considerations (i.e. obvious malicious spoof
detection considerations for the client domain and the return path domain).

3) Return path validation considerations

4) Lowering Bandwidth considerations imposed by
new augmented methods.

5) Delay Validation considerations

6) Dynamic DATA vs. POST SMTP analysis considerations.

Please note, I consider myself an "old hand" and one of the early adopters
of the some of the new email authentications ideas for product distribution
and commercial environments.

SPF, SENDERID/PRA and SUBMITTER is here and while they are defined as
independent protocols and even extended drafts, they suffer in adoption and
implementation due to  many legacy issues in x821/x822.

They as well as future ideas need the help of a better define level of
operations at 2821 and 2822.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message -----
From: "Matti Aarnio" <[email-masked]>

>
> On Wed, Jul 06, 2005 at 05:55:25AM -0400, John C Klensin wrote:
> > Matti,
> >
> > None of  this is surprising, at least to me.  The question is
> > whether you think the SMTP spec should be modified to _require_
> > servers to accept the sorts of "variations" you outline and to
> > explicitly authorize (and thereby encourage) clients to send
> > them?  And, if so, which ones and why?
>
> Yes John, we have been at this for way too long, and have seen
> everything...  I just wish  SPF et.al. specifiers have equally
> old hands.
>
>
> I have a feeling that we need a new part on the specification,
> which in itself is informal, but..
> Well, or a separate BCP RFC ?
>
>
>   NNN. Pitfalls
>
>   Following details have been seen to be escaping client
>   implementers attention  (this includes both user agents,
>   and servers as SMTP client):
>
>    ...
>
>    - Adding spaces into what specification does not specify
>      them, most commonly as:  "MAIL FROM: <...>"
>    - Sending MAIL and RCPT without angle brackets
>    - Sending utter junk and/or wrong syntax in HELO/EHLO line
>    - Not sending HELO/EHLO greeting at all
>    - Failing to do reply data collection properly, and just
>      presuming that single network read will always yield
>      all of the reply.
>    - Missing the requirement to support multiline replies
>      from servers, and that in multiline replies the last
>      line value is what determines the reply.
>    - Missing entirely reply-code principles, e.g. that their
>      treatment MUST go by the first digit, deeper semantics
>      may be applied, but basic codes are so overloaded, that
>      only first digit is usually meaningfull.
>    - Not showing protocol exchange and remote end replies to user,
>      when sending fails for whatever reason  (so that user can
>      report them to helpdesk - or see right away, why the message
>      or one of its recipients didn't go thru.)
>      For 99% of things, showing last sent protocol line, and received
>      error reply for it will do.  For the remaining 1% a full
>      protocol exchange from the connection down to close is needed.
>    - Single MAIL-transaction can not have more than 100 recipients,
>      when more are to be sent, multiple transactions are needed.
>      (Most servers allow more, some allow a lot less..)
>    - If server advertises EHLO capability to do user authentication,
>      it does not mean the client must do it.
>      Especially strange client behaviour is that when the server
>      does not advertise said capability, clients won't try to
>      authenticate and just send message without trouble.
>    - When server does not advertise some capability in EHLO reply,
>      client MUST NOT go ahead blindly and try to use it anyway.
>      Blind AUTH attempts are common.
>    - There is only one (deprecated) case (port number), where
>      in SMTP protocol extensions the TLS stream encryption is
>      activated up front, in all other cases the processing
>      follows STARTTLS specification.
>
>
>   Following details have been seen to be escaping server implementers
>   attention:
>
>    - There is NO limit in the number of MAIL-transactions per
>      single connection.  High-efficiency MTAs will send entire
>      accumulated queue in as few connections as possible.
>    - If the server implementation is fundamentally compliant,
>      PIPELINING support reporting is highly recommended at the
>      inbound side.
>    - Requiring sender authentication at inbound MX servers for
>      inbound message traffic is serious configuration mistake.
>    - Server may need radically different service subsets at
>      different ports, e.g. SMTP does not support STARTTLS, nor
>      any AUTH, whereas SUBMIT supports STARTTLS, and different
>      sets of AUTH depending on TLS being active or not.
>      Non-standard plain-text authentications should not be
>      available except under TLS encryption.
>    - Servers don't separate DNS timeouts and SERVFAILs from
>      genuine NXDOMAIN replies, and in all cases report permanent
>      five-hundred series rejects, where proper is temporary
>      four-hundred series retry.
>    - Servers pay too much attention on EHLO greeting data,
>      which from behind a NAT box may well be other than what
>      the server sees.
>
> just off the top of my head...
>
> At work we just spotted this client authentication behaviour goofup,
> when we offered AUTH services at the SMTP port of our customer outbound
> traffic SMTP servers.
>
> After that debacle, we offer AUTH only at SUBMIT port, but there we
> also support Outlook misbehaviour with TLS:  when connection starts,
server
> waits 2 seconds to read possible start of whatever the client may
> send.  If that whatever begins with byte 0x80, that byte is pushed
> back into input stream, and SSL is initiated, and then normal start
> proceeds within the SSL stream.   If nothing comes in during those
> two seconds, normal non-encrypted start proceeds.
>
>
>
> >     john
> ....
>
> --
> /Matti Aarnio <[email-masked]>
>
>

Date: Wed, 06 Jul 2005 13:35:38 +0100
To: Matti Aarnio <[email-masked]>, John C Klensin <[email-masked]>
From: Paul Smith <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Cc: Hector Santos <[email-masked]>, IETF-SMTP <[email-masked]>
Sender: [email-masked]

At 12:58 06/07/2005, Matti Aarnio wrote:
>I have a feeling that we need a new part on the specification,
>which in itself is informal, but..
>Well, or a separate BCP RFC ?
>
>
>   NNN. Pitfalls
>
>   Following details have been seen to be escaping client
>   implementers attention  (this includes both user agents,
>   and servers as SMTP client):
>
>    ...

I agree - I think the base spec is fine, but a 'common pitfalls' document 
like this would be a really good idea!

(We just need to make everyone read it... )

Paul                            VPOP3 - Internet Email Server/Gateway
[email-masked]                      http://www.pscs.co.uk/

Date: 	Wed, 6 Jul 2005 14:58:13 +0300
From: Matti Aarnio <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: Hector Santos <[email-masked]>, IETF-SMTP <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Sender: [email-masked]

On Wed, Jul 06, 2005 at 05:55:25AM -0400, John C Klensin wrote:
> Matti,
> 
> None of  this is surprising, at least to me.  The question is
> whether you think the SMTP spec should be modified to _require_
> servers to accept the sorts of "variations" you outline and to
> explicitly authorize (and thereby encourage) clients to send
> them?  And, if so, which ones and why?

Yes John, we have been at this for way too long, and have seen 
everything...  I just wish  SPF et.al. specifiers have equally
old hands.

I have a feeling that we need a new part on the specification,
which in itself is informal, but..
Well, or a separate BCP RFC ?

  NNN. Pitfalls

  Following details have been seen to be escaping client
  implementers attention  (this includes both user agents,
  and servers as SMTP client):
 
   ...

   - Adding spaces into what specification does not specify
     them, most commonly as:  "MAIL FROM: <...>"
   - Sending MAIL and RCPT without angle brackets
   - Sending utter junk and/or wrong syntax in HELO/EHLO line
   - Not sending HELO/EHLO greeting at all
   - Failing to do reply data collection properly, and just
     presuming that single network read will always yield
     all of the reply.
   - Missing the requirement to support multiline replies
     from servers, and that in multiline replies the last
     line value is what determines the reply.
   - Missing entirely reply-code principles, e.g. that their
     treatment MUST go by the first digit, deeper semantics
     may be applied, but basic codes are so overloaded, that
     only first digit is usually meaningfull.
   - Not showing protocol exchange and remote end replies to user,
     when sending fails for whatever reason  (so that user can
     report them to helpdesk - or see right away, why the message
     or one of its recipients didn't go thru.)
     For 99% of things, showing last sent protocol line, and received
     error reply for it will do.  For the remaining 1% a full
     protocol exchange from the connection down to close is needed.
   - Single MAIL-transaction can not have more than 100 recipients,
     when more are to be sent, multiple transactions are needed.
     (Most servers allow more, some allow a lot less..)
   - If server advertises EHLO capability to do user authentication,
     it does not mean the client must do it.
     Especially strange client behaviour is that when the server
     does not advertise said capability, clients won't try to
     authenticate and just send message without trouble.
   - When server does not advertise some capability in EHLO reply,
     client MUST NOT go ahead blindly and try to use it anyway.
     Blind AUTH attempts are common.
   - There is only one (deprecated) case (port number), where
     in SMTP protocol extensions the TLS stream encryption is
     activated up front, in all other cases the processing
     follows STARTTLS specification.

  Following details have been seen to be escaping server implementers
  attention:

   - There is NO limit in the number of MAIL-transactions per
     single connection.  High-efficiency MTAs will send entire
     accumulated queue in as few connections as possible.
   - If the server implementation is fundamentally compliant,
     PIPELINING support reporting is highly recommended at the
     inbound side.
   - Requiring sender authentication at inbound MX servers for
     inbound message traffic is serious configuration mistake.
   - Server may need radically different service subsets at
     different ports, e.g. SMTP does not support STARTTLS, nor
     any AUTH, whereas SUBMIT supports STARTTLS, and different
     sets of AUTH depending on TLS being active or not.
     Non-standard plain-text authentications should not be
     available except under TLS encryption.
   - Servers don't separate DNS timeouts and SERVFAILs from
     genuine NXDOMAIN replies, and in all cases report permanent
     five-hundred series rejects, where proper is temporary
     four-hundred series retry.
   - Servers pay too much attention on EHLO greeting data,
     which from behind a NAT box may well be other than what
     the server sees.

just off the top of my head...

At work we just spotted this client authentication behaviour goofup,
when we offered AUTH services at the SMTP port of our customer outbound
traffic SMTP servers.

After that debacle, we offer AUTH only at SUBMIT port, but there we
also support Outlook misbehaviour with TLS:  when connection starts, server
waits 2 seconds to read possible start of whatever the client may
send.  If that whatever begins with byte 0x80, that byte is pushed
back into input stream, and SSL is initiated, and then normal start
proceeds within the SSL stream.   If nothing comes in during those
two seconds, normal non-encrypted start proceeds.

>     john
....

-- 
/Matti Aarnio	<[email-masked]>

Date: Wed, 6 Jul 2005 11:26:41 +0100
From: Tony Finch <[email-masked]>
To: Matti Aarnio <[email-masked]>
cc: IETF-SMTP <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Sender: [email-masked]

On Wed, 6 Jul 2005, Matti Aarnio wrote:
>
> Requiring angle brackets in MAIL and RCPT addresses works usually,
> but then some odd junk (Windows CE 1.0/2.0 at least) coders hadn't
> cared to read the syntax thru properly, and "as long as it works
> with sendmail 5.x, it is good" ..  (or some such)
>
> Rejecting messages because "MAIL FROM:" is followed by a space
> would reject apparently lots of Outlooks, which form some 60%
> of our customers email clients (when counting their x-mailer: headers)
> Counting on protocol trace logs...  the rejection would be about
> 60-70% of all MAIL FROMs in today morning traffic.

This is useful operational information for smarthost and submission server
implementors, but it is irrelevant for MXs because none of this software
routes email according to RFC 974.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BISCAY: WEST 5 OR 6 BECOMING VARIABLE 3 OR 4. SHOWERS AT FIRST. MODERATE OR
GOOD.

Date: Wed, 06 Jul 2005 06:04:52 -0400
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Sender: [email-masked]

--On Wednesday, 06 July, 2005 11:07 +0200 Frank Ellermann
<[email-masked]> wrote:

> 
> Hector Santos wrote:
> 
>> Part of the problem with integrating new DNS/domain based
>> validation methods is the overhead in the DNS lookup process.
> 
> If you know that say TLD .invalid is invalid then you don't
> need a DNS lookup.  And if you're sure that the <toplabel>
> cannot be 1*DIGIT it's almost the same situation.

Frank,

While I agree, please be _very_ careful here.  Implementations
--more of web applications than of actual email clients, but
some of the latter too-- have caused a world of trouble in the
last few years by assuming they knew which TLDs were invalid.
In some cases, that involved a lexical rule (e.g., "we know
there are no TLDs more than four characters long, so, if one
appears, we don't need to look it up").  In others, it involved
a list of known, and hence valid, TLD names with the assumption
that anything not on it was invalid.  In both cases, ICANN's
adding new TLDs a few years ago, some with rather long names,
fouled up the works and users of those domains are still
encountering "that isn't a valid email address" messages from
programs that are relying on local rules and putative knowledge
rather than DNS lookups.

So, in actuality, because there is a firm prohibition on TLD
names being all-digits, it is _lots_ safer to handle one of
those as an error without looking it up than to assume that
.invalid, which is invalid today, will continue to be invalid a
year from now.

     john

Date: Wed, 06 Jul 2005 05:55:25 -0400
From: John C Klensin <[email-masked]>
To: Matti Aarnio <[email-masked]>
cc: Hector Santos <[email-masked]>, IETF-SMTP <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Sender: [email-masked]

Matti,

None of  this is surprising, at least to me.  The question is
whether you think the SMTP spec should be modified to _require_
servers to accept the sorts of "variations" you outline and to
explicitly authorize (and thereby encourage) clients to send
them?  And, if so, which ones and why?

    john

--On Wednesday, 06 July, 2005 12:42 +0300 Matti Aarnio
<[email-masked]> wrote:

> On Tue, Jul 05, 2005 at 06:56:36PM -0400, John C Klensin wrote:
>> --On Tuesday, 05 July, 2005 13:22 -0400 Hector Santos
>> <[email-masked]> wrote:
> ...
>> > I understand the legacy issues regarding the trailing
>> > spaces, been there to during the non-interactive to
>...
>> I think we disagree.  See above.  2821/ 2821bis are not about
>> giving in to the lowest competency level commonly encountered
>> among clients and servers.  To state that again, there is no
>> reason to permit or require this.  The server implementers who
>> decide, for market reasons, that they want to support MAIL and
>> RCPT commands with the space after the colon (or with no
>>...

> Being both server coder, and occasional system operator at
> places where actual customers try to use their whatever
> programs to send email thru, I can tell, that there are levels
> of 'strictness' that work, and others that definitely don't
> work.
> 
> Requiring angle brackets in MAIL and RCPT addresses works
> usually, but then some odd junk (Windows CE 1.0/2.0 at least)
> coders hadn't cared to read the syntax thru properly, and "as
> long as it works with sendmail 5.x, it is good" ..  (or some
> such)
>...

Date: 	Wed, 6 Jul 2005 12:42:33 +0300
From: Matti Aarnio <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: Hector Santos <[email-masked]>, IETF-SMTP <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Sender: [email-masked]

On Tue, Jul 05, 2005 at 06:56:36PM -0400, John C Klensin wrote:
> --On Tuesday, 05 July, 2005 13:22 -0400 Hector Santos
> <[email-masked]> wrote:
...
> > I understand the legacy issues regarding the trailing spaces,
> > been there to during the non-interactive to interactive
> > interfacing design world. But just as there is exception in
> > 2821 (but not in the STD 10),  I think it is practical to
> > include a reason for why there exist a space tolerance in the
> > majority of servers on the network.
> > 
> > If 2821bis is about "cleaning", removing ambiguity,
> > reaffirming consistency, etc, this would be one of them.
> 
> I think we disagree.  See above.  2821/ 2821bis are not about
> giving in to the lowest competency level commonly encountered
> among clients and servers.  To state that again, there is no
> reason to permit or require this.  The server implementers who
> decide, for market reasons, that they want to support MAIL and
> RCPT commands with the space after the colon (or with no pointed
> brackets at all, which we have all seen) can do so as a
> robustness extension outside the standard.  But let's not
> require anyone to do so -- a few strictly-conformant
> implementations are probably good for the network.

Being both server coder, and occasional system operator at places
where actual customers try to use their whatever programs to send
email thru, I can tell, that there are levels of 'strictness' that
work, and others that definitely don't work.

Requiring angle brackets in MAIL and RCPT addresses works usually,
but then some odd junk (Windows CE 1.0/2.0 at least) coders hadn't
cared to read the syntax thru properly, and "as long as it works
with sendmail 5.x, it is good" ..  (or some such)

Rejecting messages because "MAIL FROM:" is followed by a space
would reject apparently lots of Outlooks, which form some 60%
of our customers email clients (when counting their x-mailer: headers)
Counting on protocol trace logs...  the rejection would be about
60-70% of all MAIL FROMs in today morning traffic.

Trailing spaces don't happen at all (except in some QUITs, which
is rather strange in itself..)

Even more sloppy processing is needed, because some customer programs
just don't bother with giving any HELO/EHLO greeting at all.
Of those that give it...  some 99% give completely rubbish text
(whatever user has decided to call the machine - or just some random
that the machine calls itself with: "FamilySmith" -style is common)
Some greet with their or server's IP address in brackets, or more
commonly, without brackets, etc.    Some machines are clearly behind
some NAT box:  "EHLO [192.168.0.200]"  (yup, it had correct brackets)

If somebody greets with correct name matching remote IP reversal,
that one is a rarity indeed among these consumer customers.

If something under 1% would be affected by strict rules, such
rules could be enforced.  Now it is impossible.

At an inbound server the statistics are different, but I won't
analyze them now.

...
>      john

-- 
/Matti Aarnio	<[email-masked]>
FUNET:  Finnish Academic and Research Network
	Network Information/Software Archival Service

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Strict RFC x821 Compliant: HELO/EHLO
Date:  Wed, 06 Jul 2005 11:07:13 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 56
Sender: [email-masked]

Hector Santos wrote:

> Part of the problem with integrating new DNS/domain based
> validation methods is the overhead in the DNS lookup process.

If you know that say TLD .invalid is invalid then you don't
need a DNS lookup.  And if you're sure that the <toplabel>
cannot be 1*DIGIT it's almost the same situation.

>      nxdomain
>      none
>      success (i.e., a SPF,, CSV record exist)
>      DNS server downtime/timeout issues

For SPF the philosophy of result codes is very simple:

NONE        == whatever it is, it has no sender policy
PASS (etc.) == valid policy with result PASS (etc.)
TempError   == temporary trouble (e.g. a DNS timeout),
               maybe try again later
PermError   == broken sender policy, dare not come back
               before you fixed your sender policy

The latter case includes all recursive NXDOMAIN problems.
Of course you're not forced to reject any SPF errors.

> Basically, the idea of:
>         No Brackets  ==>  presumed DOMAIN
> is not a practical idea to implement because systems will
> endure a very high volume of HELO hits such as:

>         HELO #.#.#.#

Fine, reject it if you don't like it.  As far as SPF is
concerned #.#.#.# doesn't have a sender policy => NONE.

You'd get the same result NONE for HELO [#.#.#.#] with
SPF, working as designed, it's about sender policies.

For a "receiver policy" you could accept HELO [#.#.#.#]
coming from IP #.#.#.#

You could also accept HELO #.#.#.# from IP #.#.#.#.  If
the IP is different it starts to get ugly, maybe reject
it, or treat it like a FAIL, your decision.  But don't
say that this was a "sender policy", because it was not.

Thinking about it, this domain literal stuff is really
tricky.  If you want to check domain literal == IP you
must decode it.  At least I now see why 2821 allowed the
leading zeros, we can't remove them in 2821bis.  <sigh />

Plan B for known traps and pitfalls:  simply document it.
Documented "bugs" are by definition features.  It's not
bad enough for some new "security considerations".  Bye

Date: Tue, 05 Jul 2005 18:56:36 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, IETF-SMTP <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Sender: [email-masked]

--On Tuesday, 05 July, 2005 13:22 -0400 Hector Santos
<[email-masked]> wrote:

> 
> John,  Thanks for lurking and I'm sorry it hurts to do so. I
> just hope it was not all in vain.
> 
> Just my opinion:
> 
> The majority of all servers tolerate the space issue for all
> commands.  I'm sure you don't believe this makes them
> non-conformant to allow for this behavior.

Of course not.  2821 is clear about not requiring rejection of
anything it does not specify, only that such things are outside
the standard.  And a reasonable person might argue that the
robustness principle requires tolerance of the space if that is
reasonable for the server.  The difference between that and
putting it into 2821bis as a requirement is that, as soon as
2821 requires that servers support it, we've all got one less
way to flag incompetently-constructed clients.   When I've had
to negotiate with the authors of such clients, a list titled "50
ways your client is broken" has been an important tool.  I don't
see any reason to give in to sloppy and/or incompetent behavior
by formally declaring what those implementations to be the norm,
no matter who supports it.

> I understand the legacy issues regarding the trailing spaces,
> been there to during the non-interactive to interactive
> interfacing design world. But just as there is exception in
> 2821 (but not in the STD 10),  I think it is practical to
> include a reason for why there exist a space tolerance in the
> majority of servers on the network.
> 
> If 2821bis is about "cleaning", removing ambiguity,
> reaffirming consistency, etc, this would be one of them.

I think we disagree.  See above.  2821/ 2821bis are not about
giving in to the lowest competency level commonly encountered
among clients and servers.  To state that again, there is no
reason to permit or require this.  The server implementers who
decide, for market reasons, that they want to support MAIL and
RCPT commands with the space after the colon (or with no pointed
brackets at all, which we have all seen) can do so as a
robustness extension outside the standard.  But let's not
require anyone to do so -- a few strictly-conformant
implementations are probably good for the network.

> It is my hope there is insight to be again, not just at the
> purity of SMTP syntax, but in how it continues to evolve and
> interface in current and future considerations.    For one
> example, what can SMTP do at the simplistic level to assist
> (in general) the new DNS-based or email security proposed
> protocols?  If you believe, nothing, then I can accept that. I
> personally would not agree, but I accept it.

See my previous note.   To the extent to which these things are
useful, they may make reasonable extensions to SMTP, using the
established extension model.  Such extensions will either
survive and thrive in practice or they won't.  But they should
impact the base standard only if either:

(i) they demonstrate a need for changes that are _required_ to
support them, i.e., that the current extension model is
inadequate in some important way.  So far, the attempts to make
that case have been rather weak, at least IMO.   Or,

(ii) they are widely deployed and proven at substantially the
level SMTP itself is.   No theories about things that might
work, especially theories that require flag days or wide
deployment to be useful at all, are welcome in the SMTP spec, at
least as far as I'm concerned.

     john

Date: Tue, 05 Jul 2005 16:37:17 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, [email-masked]
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Sender: [email-masked]

--On Tuesday, 05 July, 2005 13:00 -0400 Hector Santos
<[email-masked]> wrote:

>         No Brackets  ==>  presumed DOMAIN
> 
> is not a practical idea to implement because systems will
> endure a very high volume of HELO hits such as:
> 
>         HELO #.#.#.#
> 
> So what I have been trying to get a focus on is the syntax
> field validation considerations not only for SMTP servers
> adopting these new DNS proposals, but also view them as new
> general SMTP related issues that may apply regardless if a
> system implements one or more of the DNS-based proposals.

Sigh.  

> Here is a small set of invalid HELO/EHLO captured in the first
> two hours of July 3:
>...
> If you use the No Bracket == DOMAIN rule, this would produce a
> high DNS lookup overhead when you implement a DNS-based
> proposal.

Uh... So?  2821 doesn't require you to do a DNS lookup on the
HELO/EHLO domain at all.  It also doesn't require you to
syntax-check it.  If you do decide to syntax-check it, it
doesn't require that you reject [1.2.3.X] or even [1.2.3.4.5]
nor that you accept 1.2.3.Y.4.  Regardless of which of these you
do (or don't) do, it significantly restricts the conditions
under which you can reject the command entirely: only the two
domain literal forms represent invalid syntax.

Of course, you can invoke the "protect myself from attack" rule,
at which mpoint any of the above can presumably be rejected, and
rejected without calling on the DNS.

2821 is a standard for mail transport and interchange.  It is
very clear (I hope) that you are not obligated to accept mail
that you consider nefarious.  Criteria for determining
nefariousness just don't belong in the transport standard: if
nothing else, they are too controversial and too transitory.

> I don't believe this has anything to do with semantics.  A
> SMTP system designed to address today's needs can benefit by
> having rather logical and simple syntax field validation
> considerations.

Sure.  But beyond the above, IMO, not in the standard.  The
problem with the sort of rules you are trying to develop is that
"looks suspicious" can easily evolve to exclude all sorts of
valid things -- TLDs that weren't valid a few years ago, but are
now, domain names with odd constructions (e.g., not many years
ago, if you saw a DNS label with two consecutive hyphens, it was
probably bogus; today it may be part of an important standard),
local parts with "odd" characters in them like "+" or "=" or
"/", and so on.   So we get back to the bottom line of how much
risk you want to run of rejecting legitimate mail in order to
stop spammers.  2821 isn't about that: it is about how a client
and server deal with each other when the primary purpose of both
is to get presumably-legitimate mail through to a selected
destination mailbox.  All of other cases, remedies for them,
are, IMO, appropriately outside the standard.

> Hope this explains it better.

It was actually quite clear.

    john

From: "Hector Santos" <[email-masked]>
To: "IETF-SMTP" <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Date: Tue, 5 Jul 2005 13:22:52 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

John,  Thanks for lurking and I'm sorry it hurts to do so. I just hope it
was not all in vain.

Just my opinion:

The majority of all servers tolerate the space issue for all commands.  I'm
sure you don't believe this makes them non-conformant to allow for this
behavior.

I understand the legacy issues regarding the trailing spaces, been there to
during the non-interactive to interactive interfacing design world. But just
as there is exception in 2821 (but not in the STD 10),  I think it is
practical to include a reason for why there exist a space tolerance in the
majority of servers on the network.

If 2821bis is about "cleaning", removing ambiguity, reaffirming consistency,
etc, this would be one of them.

It is my hope there is insight to be again, not just at the purity of SMTP
syntax, but in how it continues to evolve and interface in current and
future considerations.    For one example, what can SMTP do at the
simplistic level to assist (in general) the new DNS-based or email security
proposed protocols?  If you believe, nothing, then I can accept that. I
personally would not agree, but I accept it.

Thanks for your comments

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message -----
From: "John C Klensin" <[email-masked]>
To: "Hector Santos" <[email-masked]>; <[email-masked]>
Sent: Tuesday, July 05, 2005 7:55 AM
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:

>
>
>
> --On Sunday, 03 July, 2005 03:00 -0400 Hector Santos
> <[email-masked]> wrote:
>
> >> Well, that's your opinion. My opinion is that every
> >> relaxation leads to overhead and more code (and in the worst
> >> case to security problems). Where do you draw the line?
> >
> > I feel ya.  100%. I agree with your thinking. I'm a stickler
> > when it comes to comformance.   But I thin,.  agan, just my
> > opinion,  I would make an exception simply because its already
> > burned in.
> >
> > In fact, I think 2821bis should probably consider relaxing
> > this with a similar statement it has for the trailing space
> > consideration:
> >
> > Technically, the spec for the MAIL command is::
> >
> >       MAIL FROM:<reverse-path> [SP <mail-parameters> ] <CRLF>
> >
> > No space before <CRLF> but section 4.1.1 says:
> >
> > 4.1.1 Command Semantics and Syntax
> >
> >    .....(In the interest of improved interoperability, SMTP
> >    receivers are encouraged to tolerate trailing white space
> > before the    terminating <CRLF>.)
> >
> > I think this should also include the space before the
> > reverse-path.
>
> Hector, FWIW, these are rather different creatures.  The
> encouragement to tolerate trailing white space is due to a
> history of machines and operating systems whose notion of "line"
> is quite different from the "data stream with inline line
> delimiters" approach that is assumed by the Internet.  Some of
> those have included variable length records with word counts and
> fixed length record approaches, both with padding.  On either
> one, producing output without trailing white space, especially
> when SMTP is simulated via telnet for testing purposes, can be
> quite troublesome.  We don't see a lot of those systems of late,
> but they are still around.   By contrast, there is no excuse
> (other than plain sloppiness) on any system for putting a space
> between the colon and the left pointed bracket that starts the
> reverse path in the RCPT command and the similar structure in
> the MAIL command.
>
> To permit that space (just one space?  an arbitrary amount of
> space?) violates the principle that the syntax of SMTP commands
> should be as simple, with as few variations, as possible.  It
> would make a certain number of server implementations that a
> conforming today non-conforming.   And it would accomplish what??
>
>        john
>
>

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Tue, 5 Jul 2005 13:00:03 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "Frank Ellermann" <[email-masked]>
>
> As far as RfC 2821 is concerned:  no square brackets => no IP,
> and no dot => no FQDN,  For 2821bis John proposed to allow a
> trailing dot for cases like FQDN ai. (= TLD is also a host).
>
> For SPF and CSV I don't see any problems with the first rule:
> (square brackets) <=> (IPv4 or IPv6 or IPvFuture).

Right.

Part of the problem with integrating new DNS/domain based validation methods
is the overhead in the DNS lookup process.

In short, you will the spread of DNS results:

     nxdomain
     none
     success (i.e., a SPF,, CSV record exist)
     DNS server downtime/timeout issues

and it will be high on the NXDOMAIN, NONE side especially during the early
adoption periods.  You need to consider that spammers will also remain
non-compliant until they need to adapt as well.

Basically, the idea of:

        No Brackets  ==>  presumed DOMAIN

is not a practical idea to implement because systems will endure a very high
volume of HELO hits such as:

        HELO #.#.#.#

So what I have been trying to get a focus on is the syntax field validation
considerations not only for SMTP servers adopting these new DNS proposals,
but also view them as new general SMTP related issues that may apply
regardless if a system implements one or more of the DNS-based proposals.

Here is a small set of invalid HELO/EHLO captured in the first two hours of
July 3:

00:01:43 Invalid HELO 221.209.94.61 client address [221.209.94.61]
00:01:56 Invalid HELO 213.77.14.102 client address [213.77.14.102]
00:08:02 Invalid HELO 208.247.131.9 client address [220.124.189.131]
00:14:54 Invalid HELO 208.247.131.9 client address [220.119.191.94]
00:14:55 Invalid HELO 208.247.131.9 client address [222.235.89.108]
00:14:58 Invalid HELO 208.247.131.9 client address [202.158.176.251]
00:15:07 Invalid HELO 208.247.131.9 client address [220.176.218.27]
00:38:58 Invalid HELO 218.19.207.228 client address [218.19.207.228]
00:39:49 Invalid HELO 208.247.131.9 client address [70.150.14.52]
00:46:08 Invalid HELO 61.149.132.171 client address [61.149.132.171]
00:46:15 Invalid HELO 61.149.132.171 client address [61.149.132.171]
00:46:27 Invalid HELO 61.149.132.171 client address [61.149.132.171]
00:46:35 Invalid HELO 61.149.132.171 client address [61.149.132.171]
00:46:36 Invalid HELO 61.149.132.171 client address [61.149.132.171]
00:55:01 Invalid HELO 208.247.131.9 client address [218.14.149.71]
00:55:15 Invalid HELO 208.247.131.9 client address [61.254.73.93]
00:55:20 Invalid HELO 208.247.131.9 client address [218.14.149.71]
00:55:33 Invalid HELO 208.247.131.9 client address [200.83.141.113]
01:06:03 Invalid HELO 208.247.131.9 client address [222.94.49.237]
01:06:15 Invalid HELO 208.247.131.9 client address [60.8.20.37]
01:19:02 Invalid HELO 208.247.131.9 client address [218.54.88.18]
01:19:03 Invalid HELO 208.247.131.9 client address [218.64.87.217]
01:26:16 Invalid HELO 200.63.237.123 client address [200.63.237.123]
01:27:01 Invalid HELO 211.113.209.69 client address [211.113.209.69]
01:30:00 Invalid HELO 221.220.202.183 client address [221.220.202.183]
01:30:18 Invalid HELO 221.220.203.206 client address [221.220.203.206]
01:30:41 Invalid HELO 221.220.203.206 client address [221.220.203.206]
01:30:52 Invalid HELO 221.220.203.206 client address [221.220.203.206]
01:30:56 Invalid HELO 221.220.204.74 client address [221.220.204.74]
01:31:10 Invalid HELO 221.220.203.206 client address [221.220.203.206]
01:31:25 Invalid HELO 221.220.203.206 client address [221.220.203.206]
01:31:36 Invalid HELO 221.220.204.74 client address [221.220.204.74]
01:32:05 Invalid HELO 208.247.131.9 client address [222.119.35.40]
01:32:05 Invalid HELO 221.220.204.74 client address [221.220.204.74]
01:32:21 Invalid HELO 208.247.131.9 client address [211.162.133.147]
01:33:51 Invalid HELO 208.247.131.9 client address [221.159.200.48]
01:33:59 Invalid HELO 208.247.131.9 client address [219.156.250.114]
01:37:02 Invalid HELO 61.50.165.190 client address [61.50.165.190]
01:37:03 Invalid HELO 61.50.165.190 client address [61.50.165.190]
01:37:05 Invalid HELO 61.50.165.190 client address [61.50.165.190]
01:37:06 Invalid HELO 61.50.165.190 client address [61.50.165.190]
01:37:08 Invalid HELO 61.50.165.190 client address [61.50.165.190]
01:58:52 Invalid HELO 61.80.233.222 client address [61.80.233.222]
01:58:55 Invalid HELO 61.81.128.100 client address [61.81.128.100]
01:58:58 Invalid HELO 61.81.147.99 client address [61.81.147.99]
01:59:01 Invalid HELO 61.81.149.244 client address [61.81.149.244]
01:59:04 Invalid HELO 61.81.20.103 client address [61.81.20.103]
01:59:07 Invalid HELO 61.81.204.80 client address [61.81.204.80]
01:59:09 Invalid HELO 61.81.210.125 client address [61.81.210.125]
01:59:32 Invalid HELO 61.81.25.137 client address [61.81.25.137]
01:59:35 Invalid HELO 61.81.36.176 client address [61.81.36.176]
01:59:38 Invalid HELO 61.82.113.120 client address [61.82.113.120]
01:59:44 Invalid HELO 61.82.250.148 client address [61.82.250.148]
01:59:46 Invalid HELO 61.82.250.86 client address [61.82.250.86]
01:59:49 Invalid HELO 61.82.6.209 client address [61.82.6.209]
01:59:52 Invalid HELO 61.83.224.123 client address [61.83.224.123]
01:59:55 Invalid HELO 61.83.29.215 client address [61.83.29.215]
01:59:57 Invalid HELO 61.83.79.205 client address [61.83.79.205]

If you use the No Bracket == DOMAIN rule, this would produce a high DNS
lookup overhead when you implement a DNS-based proposal.

I don't believe this has anything to do with semantics.  A SMTP system
designed to address today's needs can benefit by having rather logical and
simple syntax field validation considerations.

Hope this explains it better.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: [email-masked]
Cc: [email-masked]
To: John C Klensin <[email-masked]>
Date: Tue, 05 Jul 2005 09:28:25 -0700 (PDT)
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Sender: [email-masked]

> --On Sunday, 03 July, 2005 10:12 -0700 Claus Assmann
> <[email-masked]> wrote:

> >> See the discussion in RFC 1123 section 2.1: "a valid host
> >> name can never have the dotted-decimal form #.#.#.#, since at
> >> least the highest-level component label will be alphabetic."
> >
> > Ok, let's try this again: The statement was:
> >
> > "This would be an illegal syntax per the ABNF:"
> >
> > This statement is wrong. The parameter is _syntactically_
> > valid (as I pointed out before by quoting the grammar).  Maybe
> > it would be useful to change the syntax to reflect that "the
> > highest-level component label will be alphabetic.". As long as
> > that isn't the case, the command above is correct according to
> > the grammar given in RFC 2821 (that's why my software doesn't
> > reject it in contrast to the _syntactically_ invalid MAIL
> > command that triggered this HELO/EHLO discussion).

> Short answer: "yes".   And changing the syntax would, IMO, be a
> terrible idea.  The mail syntax is unambiguous and doesn't
> require any trick parsing or whole-string inspection: as Frank
> (I think) pointed out in a later note, "no brackets, no domain
> literal" and anything else is construed as a domain name.  If
> there are additional constraints on the latter, they are imposed
> from somewhere else (RFC 1123 in this case).  Similar comments
> apply to that prefix string for non-IPv4 literals: essentially,
> simple parsers with little or no look-ahead are, for SMTP at
> least, considered good while tricky heuristics to deduce what is
> coming or happening are considered unfortunate and/or risky.
> Unless there is a good reason for the risk, why incur it?

> --On Sunday, 03 July, 2005 14:08 -0400 Hector Santos
> <[email-masked]> wrote:

> > Section 4.1.3 Address Literals specifically defines the format
> > for each of the above.  For the sake of simplicity, using IPv4:
> >
> >       IPv4-address-literal = Snum 3("." Snum)
> >       Snum = 1*3DIGIT  ; representing a decimal integer
> >             ; value in the range 0 through 255
> >
> > This defines an alphanumeric dotted string format.

> Actually, a dotted-string format whose fields (between the dots)
> are strictly digits.  I'd describe this as a dotted-numeric
> string, not alphanumeric at all.  This is important below...
 
> > But 2821 does not specifically define or say the
> > address-literal could not be a A record in DNS.

> Huh?  I'm not sure what you are getting at here, but you
> probably need to look at 4.1.2, where it defines address-literal
> (the only context in which the above can appear) as

>    address-literal = "[" IPv4-address-literal /
>                      IPv6-address-literal /
>                      General-address-literal "]"

> Please note those nasty [...].  So, as above, address literals
> cannot be domain names.   And domain names cannot be address
> literals.  Full stop.

> >> As long as that isn't the
> >> case, the command above is correct according to the grammar
> >> given in RFC 2821 (that's why my software doesn't reject it
> >> in contrast to the _syntactically_ invalid MAIL command that
> >> triggered this HELO/EHLO discussion).
> >
> > You are technically correct per 2821, but 2821 *UPDATES* 1123,
> > hence 2821 overrides various parts in 1123, but other items in
> > 1123 not updated by 2821 still take hold.

> Sigh.  It is part of a completely different thread, but this is
> yet another example of why category-concepts like "updates" are
> insufficient and something like ISDs are needed.

> > So 1123 covers the definition of what is considered a valid
> > TLD based domain vs. a dotted-address format.

> In a section on the DNS, not SMTP.  Essentially, that section
> governs what TLDs can be created without causing problems.  SMTP
> is (and has always been) more conservative, because it doesn't
> rely on the form/syntax of a domain name to tell it from a
> literal.

> > Remember, 1123 came first before 2821 so you will find systems
> > following its guidelines as well.

> It is perfectly rational to prohibit all-digit domain names in
> an SMTP system by virtue of knowing about the 1123 rule.  But,
> as far as 2821 is concerned, that is a semantic restriction, not
> a restriction imposed by the SMTP-based syntax.   If one does
> apply that prohibition, rather than just looking the thing up,
> the cautions of RFC 3696 apply -- while I'd predict that ICANN
> wouldn't get stupid, TLD names could, in principle, have only
> one alphabetic character and still not violate the intent of the
> 1123 rule.

> --On Sunday, 03 July, 2005 18:48 -0400 Hector Santos
> <[email-masked]> wrote:

> > So we have the domain-literal,  for 2821bis, we need to make
> > sure it clearly defines what are all the possibilities.

> It does.   The key is the square brackets, which define what is
> or is not a address-literal.  Anything else that appears in one
> of those positions is a domain name.  Period.  Then there are
> restrictions on address-literals and restrictions on domain
> names, but they are, as nearly as possible, high-level syntax
> restrictions only.

> > Claus raised a valid point, how do you know it is not a
> > domain?   I think 1123 clears that up.   Something for John to
> > include in 2826bis.

> I assume you mean 2821bis.  But, if I have my way, no.   See
> above.

I agree with John. RFC 2821 is right to specify a general syntax for names and
leave semantic restrictions out.

> And yes, I am tracking these discussions although reluctantly.

Ditto.

				Ned

Date: Tue, 05 Jul 2005 07:55:24 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, [email-masked]
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Sender: [email-masked]

--On Sunday, 03 July, 2005 03:00 -0400 Hector Santos
<[email-masked]> wrote:

>> Well, that's your opinion. My opinion is that every
>> relaxation leads to overhead and more code (and in the worst
>> case to security problems). Where do you draw the line?
> 
> I feel ya.  100%. I agree with your thinking. I'm a stickler
> when it comes to comformance.   But I thin,.  agan, just my
> opinion,  I would make an exception simply because its already
> burned in.
> 
> In fact, I think 2821bis should probably consider relaxing
> this with a similar statement it has for the trailing space
> consideration:
> 
> Technically, the spec for the MAIL command is::
> 
>       MAIL FROM:<reverse-path> [SP <mail-parameters> ] <CRLF>
> 
> No space before <CRLF> but section 4.1.1 says:
> 
> 4.1.1 Command Semantics and Syntax
> 
>    .....(In the interest of improved interoperability, SMTP
>    receivers are encouraged to tolerate trailing white space
> before the    terminating <CRLF>.)
> 
> I think this should also include the space before the
> reverse-path.

Hector, FWIW, these are rather different creatures.  The
encouragement to tolerate trailing white space is due to a
history of machines and operating systems whose notion of "line"
is quite different from the "data stream with inline line
delimiters" approach that is assumed by the Internet.  Some of
those have included variable length records with word counts and
fixed length record approaches, both with padding.  On either
one, producing output without trailing white space, especially
when SMTP is simulated via telnet for testing purposes, can be
quite troublesome.  We don't see a lot of those systems of late,
but they are still around.   By contrast, there is no excuse
(other than plain sloppiness) on any system for putting a space
between the colon and the left pointed bracket that starts the
reverse path in the RCPT command and the similar structure in
the MAIL command.  

To permit that space (just one space?  an arbitrary amount of
space?) violates the principle that the syntax of SMTP commands
should be as simple, with as few variations, as possible.  It
would make a certain number of server implementations that a
conforming today non-conforming.   And it would accomplish what??

       john

Date: Tue, 5 Jul 2005 10:38:46 +0100
From: Tony Finch <[email-masked]>
To: [email-masked]
cc: [email-masked]
Subject: Re: Bounce/System Notification Address Verification 
Sender: [email-masked]

On Mon, 4 Jul 2005 [email-masked] wrote:
>
> 1) Figure out its IP address that it just got handed by DHCP.
> 2) Try to look up the PTR to get its hostname, and fail to get a valid answer.
> 3) Send the preconfigured 'SMTP Smarthost' name on the HELO, just because we know
> it and it's handy, and the programmer didn't know about 'HELO [i.p.add.ress]'

The MARID protocols are designed for email routed across the public
Internet according to RFC 974, so they do not apply to email being sent
via an SMTP smart host. There are *many* ways in which MUA SMTP
implementations are crap, and the above is a good example. However it is
not an example of a problem with strict HELO checking by an MX.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BISCAY: WEST 5 OR 6 BECOMING VARIABLE 3 OR 4. SHOWERS AT FIRST. MODERATE OR
GOOD.

To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: Bounce/System Notification Address Verification 
From: [email-masked]
Date: Tue, 05 Jul 2005 00:48:56 -0400
Sender: [email-masked]

--==_Exmh_1120538936_13480P
Content-Type: text/plain; charset=us-ascii

On Mon, 04 Jul 2005 19:52:12 EDT, Hector Santos said:

> Yes Really Bruce. Domain spoofing, forging is a real.
> 
> > -- read 2505.  If you send to an external site which does
> > aliasing and sends back to some other local-part at the same domain,
> > guess what MAIL FROM will be; exactly as you set it.
> 
> I still don't think you understood.

I wasn't sure *YOU* understood either, given that you gave Claus grief about
how he was failing to "protect his domain" by not rejecting odd values on
a HELO/EHLO.

--==_Exmh_1120538936_13480P
Content-Type: application/pgp-signature

--==_Exmh_1120538936_13480P--

To: Hector Santos <[email-masked]>
Cc: IETF-SMTP <[email-masked]>
Subject: Re: IP::DOMAIN Associations [Re: Bounce/System Notification Address Verification] 
From: [email-masked]
Date: Tue, 05 Jul 2005 00:39:54 -0400
Sender: [email-masked]

--==_Exmh_1120538394_13480P
Content-Type: text/plain; charset=us-ascii

On Mon, 04 Jul 2005 18:43:13 EDT, Hector Santos said:

> c:> nslookup - query=txt  vt.edu

> I intentionally separated each SPF directive by line to make it easier to
> read for a reason discussed below.

Right.  But what the <expletive> does that have to do with what name
is claimed on the HELO/EHLO, which is what you *started* complaining about
on Claus's box?

> Now,  I suppose you also want hear my PROS and CONS of your SPF policy. 

Actually, no, but we probably can't get you to shut up about stuff you have
no idea of why it's set up the way it is, so I guess we need to sit through
your discussion of how it would be set up in *your* corner of the world, with
total disregard of any local conditions that don't match your preconcieved
notions of How It Should Be Done.

> First, look at what you just done with your SPF relaxed provision (~all).
> You just invited all SPF spoofers to use your vt.edu domain to spoof and hid
> behind your domain across the network.

OK. Tell you what.  Give me your cell phone number, and I'll give it to our
help desk, and *YOU* can field all the calls if we change that ~all.  That was
set there for a *REASON*, namely that there still exist infrastructure issues
that mean that mail *can* pop out from other places.

You see... we have this chalet.  In Switzerland.  (seriously).  Among other
things, which emit *.vt.edu mail.  Oh, and some 70K users - the vast majority
of which aren't on campus at the moment, it being summer and all (more about
that later...)

> Second, with such a long SPF policy you have added a higher DNS overhead on
> the SPF server/receiver, and for what?  Even if a mismatch is found, you
> told the receiver "don't trust the result." So why bother?

If you prefer, I'll tell the guy in the next cubicle to pull it out, and that
will be one less domain supporting SPF.

> Look at the number of DNS function calls needed here for your policy:
> 
>    "v=spf1
>          ip4:128.173.0.0/16
>          ip4:198.82.0.0/16
> 1         mx
> 2         ptr a:lennier.cc.vt.edu
> 3         a:lyta.cc.vt.edu
> 4         a:dagger.cc.vt.edu
> 5         a:vivi.cc.vt.edu
> 6         a:steiner.cc.vt.edu
> 7         a:zidan e.cc.vt.edu
>          ~all"
> 
> Although, you will argue DNS caching will save the day, the potential is
> still there for long timeouts per attempt on a poorly setup system.  You
> just imposed 7 additional DNS lookups on the receiver.  If you are sending a
> significant amount of mail out, lets say, X number of systems,  you have
> just include the DNS overhead by a factor of 7 on each X system.

Tell you what - we'll modify this to more accurately reflect our *actual* mail
architecture - but then you're going to be looking at about 30 or 40 a:
entries, and quite likely a backoff to a TCP connection for the DNS lookup if
the DNS stub at the far end isn't EDNS0-capable (the reply is at 410 bytes now
- it *was* 513 bytes until we trimmed out some useful-to-us TXT records just
for the sake of SPF lovers).

And don't you *DARE* give me any "just fix your architecture" noise - you're
talking here about a mission-critical mail server for (what would be if we were
a private corporation) a $400M/year company with 15K employees worldwide (yes,
worldwide.  Researchers all over the place, and that site in Switzerland..)

You're welcome to contact us during our next fund drive and contribute enough
money to underwrite changing it.  Our last major upgrade ran close to $1M,
and did *not* involve dealing with all the departmental mailservers.

> It big waste because you have RELAXED the results of the lookup.  Why
> Bother?

Mostly because *SOME* of us are concerned about *actually* getting mail
through, and we really have better things to do than to break our *own* user's
mail just because their MUA is accidentally configured to send with a From:
[email-masked] address, but to send to their ISP's mail server.

But of course, you know more about what conditions are present at our site than
we do, because we're *totally* clueless about how to run a mail system that
handles more than 2 million pieces of mail every day.

(I gave up at this point - it's pretty obvious by here that Hector Knows All
about how to configure this stuff, and that anything he disagrees with must be
a stupid misconfiguration on our part rather than a carefully considered decision
made by people who know what's going on....)

--==_Exmh_1120538394_13480P
Content-Type: application/pgp-signature

--==_Exmh_1120538394_13480P--

Date: Mon, 04 Jul 2005 23:49:53 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Sender: [email-masked]

--On Sunday, 03 July, 2005 10:12 -0700 Claus Assmann
<[email-masked]> wrote:

>> See the discussion in RFC 1123 section 2.1: "a valid host
>> name can never have the dotted-decimal form #.#.#.#, since at
>> least the highest-level component label will be alphabetic."
> 
> Ok, let's try this again: The statement was:
> 
> "This would be an illegal syntax per the ABNF:"
> 
> This statement is wrong. The parameter is _syntactically_
> valid (as I pointed out before by quoting the grammar).  Maybe
> it would be useful to change the syntax to reflect that "the
> highest-level component label will be alphabetic.". As long as
> that isn't the case, the command above is correct according to
> the grammar given in RFC 2821 (that's why my software doesn't
> reject it in contrast to the _syntactically_ invalid MAIL
> command that triggered this HELO/EHLO discussion).

Short answer: "yes".   And changing the syntax would, IMO, be a
terrible idea.  The mail syntax is unambiguous and doesn't
require any trick parsing or whole-string inspection: as Frank
(I think) pointed out in a later note, "no brackets, no domain
literal" and anything else is construed as a domain name.  If
there are additional constraints on the latter, they are imposed
from somewhere else (RFC 1123 in this case).  Similar comments
apply to that prefix string for non-IPv4 literals: essentially,
simple parsers with little or no look-ahead are, for SMTP at
least, considered good while tricky heuristics to deduce what is
coming or happening are considered unfortunate and/or risky.
Unless there is a good reason for the risk, why incur it?

--On Sunday, 03 July, 2005 14:08 -0400 Hector Santos
<[email-masked]> wrote:

> Section 4.1.3 Address Literals specifically defines the format
> for each of the above.  For the sake of simplicity, using IPv4:
> 
>       IPv4-address-literal = Snum 3("." Snum)
>       Snum = 1*3DIGIT  ; representing a decimal integer
>             ; value in the range 0 through 255
> 
> This defines an alphanumeric dotted string format.

Actually, a dotted-string format whose fields (between the dots)
are strictly digits.  I'd describe this as a dotted-numeric
string, not alphanumeric at all.  This is important below...
 
> But 2821 does not specifically define or say the
> address-literal could not be a A record in DNS.

Huh?  I'm not sure what you are getting at here, but you
probably need to look at 4.1.2, where it defines address-literal
(the only context in which the above can appear) as 

   address-literal = "[" IPv4-address-literal /
                     IPv6-address-literal /
                     General-address-literal "]"

Please note those nasty [...].  So, as above, address literals
cannot be domain names.   And domain names cannot be address
literals.  Full stop.

>> As long as that isn't the
>> case, the command above is correct according to the grammar
>> given in RFC 2821 (that's why my software doesn't reject it
>> in contrast to the _syntactically_ invalid MAIL command that
>> triggered this HELO/EHLO discussion).
> 
> You are technically correct per 2821, but 2821 *UPDATES* 1123,
> hence 2821 overrides various parts in 1123, but other items in
> 1123 not updated by 2821 still take hold.

Sigh.  It is part of a completely different thread, but this is
yet another example of why category-concepts like "updates" are
insufficient and something like ISDs are needed.

> So 1123 covers the definition of what is considered a valid
> TLD based domain vs. a dotted-address format.

In a section on the DNS, not SMTP.  Essentially, that section
governs what TLDs can be created without causing problems.  SMTP
is (and has always been) more conservative, because it doesn't
rely on the form/syntax of a domain name to tell it from a
literal.  

> Remember, 1123 came first before 2821 so you will find systems
> following its guidelines as well.

It is perfectly rational to prohibit all-digit domain names in
an SMTP system by virtue of knowing about the 1123 rule.  But,
as far as 2821 is concerned, that is a semantic restriction, not
a restriction imposed by the SMTP-based syntax.   If one does
apply that prohibition, rather than just looking the thing up,
the cautions of RFC 3696 apply -- while I'd predict that ICANN
wouldn't get stupid, TLD names could, in principle, have only
one alphabetic character and still not violate the intent of the
1123 rule.

--On Sunday, 03 July, 2005 18:48 -0400 Hector Santos
<[email-masked]> wrote:

> So we have the domain-literal,  for 2821bis, we need to make
> sure it clearly defines what are all the possibilities.

It does.   The key is the square brackets, which define what is
or is not a address-literal.  Anything else that appears in one
of those positions is a domain name.  Period.  Then there are
restrictions on address-literals and restrictions on domain
names, but they are, as nearly as possible, high-level syntax
restrictions only.

> Claus raised a valid point, how do you know it is not a
> domain?   I think 1123 clears that up.   Something for John to
> include in 2826bis.

I assume you mean 2821bis.  But, if I have my way, no.   See
above.

And yes, I am tracking these discussions although reluctantly.

     john

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Strict RFC x821 Compliant: HELO/EHLO
Date:  Tue, 05 Jul 2005 03:48:47 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 60
Sender: [email-masked]

Hector Santos wrote:

> I'm hope Klensin is following the thread for consideration

For 2821bis it's ot strictly necessary, its domain-literals
are _always_ enclosed in square brackets.  IIRC dito STD 66,
but the latter is better:  No funny tag for IPv6, no leading
zeros (and therefore no questions about "octal" ;-), and no
CTL-chararacters in <IPvFuture>, the <General-Address-Literal>
in 2821 inherited the <NO-WS-CTL> nonsense from 2822.

> security aspects, etc, to basically remove ambiguity
[...]
> If ICANN do allow for numeric TLDs, in order to comply with
> RFC 2821 Today, it could be allowed for number values over
> 255.  So for example, .911  will the following be possible:
 
>         miami.911
>         newyork.911
>         houston.911

That's a horrible idea, do you know how many...

   if verify( arg( 1 ), '0123456789.' ) = 0
      then if SockGetHostByAddr( arg( 1 ), 'P.' )
              then return P.NAME ;  else  return ''
      else if SockGetHostByName( arg( 1 ), 'P.' )
              then return P.ADDR ;  else  return ''

...or similar constructs I have on my box ?  Many published
scripts, so it's not guaranteed to be only my box.  For an IP
one simple strategy is "if no GetHostByAddr() try it anyway".

That it won't work for IPv6 is documented elsewhere, but that
it cannot work for a numerical FQDN could be a security issue.

> in my view, for proposals, like SPF and CSV to have any hope
> of success in the future, the client domain name will have to
> be clearly worked out, with some level of enforcement.

As far as RfC 2821 is concerned:  no square brackets => no IP,
and no dot => no FQDN,  For 2821bis John proposed to allow a
trailing dot for cases like FQDN ai. (= TLD is also a host).

For SPF and CSV I don't see any problems with the first rule:
(square brackets) <=> (IPv4 or IPv6 or IPvFuture).

> before I can make the presumption the client domain is CSV
> or SPF ready, I need to make sure it is syntax/value ready
> first.

If you're not sure you're free to `nslookup -q=txt 127.0.0.1`
for an SPF result NONE.  That part of the SPF spec. made it to
two SPF Council reviews - roughly the same as an IESG [Discuss]
- until it was "correct" from my POV.  The old draft Lentczner
proposed a FAIL "malformed domain" for domain literals.

                           Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Bounce/System Notification Address Verification
Date:  Tue, 05 Jul 2005 02:54:04 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 35
Sender: [email-masked]

Bruce Lilly wrote:

> LMAP is IRTF. darft form, not IETF.  R = research; not ready
> for prime time.

The IRTF is essentially dead for more than a year.  The last
new and nice proposals were SIQ and a memo about blacklists:

2005-03 draft-irtf-asrg-iar-howe-siq-01.txt
2004-11 draft-irtf-asrg-dnsbl-01.txt

 [SPF vs. Sender-ID RfCs]
> None are standards or pending standards, or Standards Track,
> or anything similar.  At best, Experimental.

At best contradicting themselves and incompatible.  SPF wanted
a proper last call, review, and status as a proposed standard,
after all it's deployed widely, working as designed, and the
draft was especially written with your famous reviews in mind.

Unlike the three Sender-ID I-Ds, which never tried to survive
a proper "last call" and community review.

But the 'Industry Enforced Standards Group' decided to publish
it as "experimental" together with an incompatible commercial
"pat. pend." proposal trying to abuse several hundred thousands
of v=spf1 policies for a dubious Purported Responsible Address.

Of course an RfC 1123 5.3.6 (a) alias is bad for the 551 case,
and an SPF-FAIL could emulate the "551 user not local" effect.

If a receiver wants something better it's his business to get
it right, testing an original MAIL FROM behind his border MTA
against a set of sender-permitted IPs cannot work.  Bye, Frank

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Bounce/System Notification Address Verification
Date: Mon, 4 Jul 2005 19:52:12 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: "Bruce Lilly" <[email-masked]>

> > You believe you might have misunderstood the RFC 2505 recommendation as
it
> > is consistent with the LMAP proposals,
>
> LMAP is IRTF. darft form, not IETF.  R = research; not ready for
> prime time.  None are standards or pending standards, or
> Standards Track, or anything similar.  At best, Experimental.

I understand that Bruce and think you understand me too.  LMAP is not a
protocol. LMAP is a IRTF methodology based on research for which the
following IETF experimental drafts are based on:

http://www.ietf.org/internet-drafts/draft-schlitt-spf-classic-02.txt
http://www.ietf.org/internet-drafts/draft-katz-submitter-01.txt
http://www.ietf.org/internet-drafts/draft-lyon-senderid-core-01.txt
http://www.ietf.org/internet-drafts/draft-lyon-senderid-pra-01.txt

IETF or no IETF, these are being implemented by vendors and operators. I
would not bet my boots they will be going away.  I don't pretend to be a
IETF expert, but given the security nature of the task and the heavy
principals involved,  I will probably bet these will become standard track
items in short order.

Do you plan to have any input on these experiments?

>> In other words, if the client IP is your address, then you have
>> NO business having my domain being used on your MAIL FROM,
>> because I did not authorize your IP address to use the return
>> path with my email domain.

> Really

Yes Really Bruce. Domain spoofing, forging is a real.

> -- read 2505.  If you send to an external site which does
> aliasing and sends back to some other local-part at the same domain,
> guess what MAIL FROM will be; exactly as you set it.

I still don't think you understood.

First,  the LMAP based proposals is consistent with RFC 2505 SMTP client IP
address based rejection concepts.

Second,  RFC 2505 section 2.6.2 is outdated or not explicit enough in
regards to the types of MAIL FROM spoofs. For example, it doesn't discuss if
the spoof is based on a predefined SMTP client local IP association .  Why
would one want to use their email address to be used a return path by an
unknown remote site?  Why it is not doubt possible, it is technically
unsound, invites overhead on bounce activity and its most certainly,
invites security issues.  The experimental drafts do need to update this
section of RFC 2505 by clarifying the social network implication described
by the section 2.6.2 vs. an unknown IP social.

In short,  if one sends an email to some site with this specific planned
aliasing behavior,  this implies a predefined association or social network
of some kind.  This would be a clear white list case.

In my view, these extended checks should only be applied on 100% anonymous
sender unauthorized final destination mail.

If a legitimate sender is known to use your email for the return path in a
distribution, then you white list this predefined relationship.

If you don't expect this to be happening, then an LOCAL_IP::LOCAL_DOMAIN
association is perfectly sound with 100% trust and 0% false positives.

LOCAL DOMAIN:  TRUST = 100%
REMOTE DOMAIN: TRUST < 100%

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: "IETF-SMTP" <[email-masked]>
Subject: IP::DOMAIN Associations [Re: Bounce/System Notification Address Verification]
Date: Mon, 4 Jul 2005 18:43:13 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: <[email-masked]>

On Mon, 04 Jul 2005 14:07:37 EDT, Hector Santos said:

> Sure, it's *LIKELY* to be a spoof, but:
>
> 1) It isn't a *positive* proof.
> 2) As a result, bouncing mail because of a failure
>     is quite the anti-social thing to do.

I agree with you with this legacy issues.  But that is not what we are
talking about or dealing with here.

In short,  we are talking about the IP::DOMAIN association defined by the
LMAP methodology which by the way, you support via your SPF record for your
email domain: vt.edu:

c:> nslookup - query=txt  vt.edu

   "v=spf1
         ip4:128.173.0.0/16
         ip4:198.82.0.0/16
         mx
         ptr a:lennier.cc.vt.edu
         a:lyta.cc.vt.edu
         a:dagger.cc.vt.edu
         a:vivi.cc.vt.edu
         a:steiner.cc.vt.edu
         a:zidan e.cc.vt.edu
         ~all"

I intentionally separated each SPF directive by line to make it easier to
read for a reason discussed below.

For those not up to par with SPF,  SPF allows one to expose a DNS policy
that defines this IP::DOMAIN association. It ask the question, "Can this
machine send mail on behalf of the email domain or possible client machine
domain?"

So the simple algorithm is to take the email domain from the 2821.MainFrom,
perform a DNS TXT lookup for this domain to get the above record.

Then you compare the connection IP address against the SPF directives and
mechanisms above to see if an IP is listed or
expanded from the directives.  For your policy, if no match is found, you
have a  "SoftFail" policy.

Before I discuss some opinions on your SPF record, I want to get this out of
the way:

In regards to the client domain name (HELO/EHLO),  there is a 100% trusted
and reliable detection when the DOMAIN is your domain simply because you own
the domain.  It is lightweight no-DNS SPF like consideration for local
domains checks on.

That is what I was referring to is 100% no false positive HELO spoof
detector when doing a LOCAL_IP::LOCAL_DOMAIN association check.

Now,  I suppose you also want hear my PROS and CONS of your SPF policy.  You
talk about me breaking the system?  What you have done is in my view,
represents the bad side of SPF.

      - relaxed provisions,
      - wasted DNS overhead

First, look at what you just done with your SPF relaxed provision (~all).
You just invited all SPF spoofers to use your vt.edu domain to spoof and hid
behind your domain across the network.

Second, with such a long SPF policy you have added a higher DNS overhead on
the SPF server/receiver, and for what?  Even if a mismatch is found, you
told the receiver "don't trust the result." So why bother?

Look at the number of DNS function calls needed here for your policy:

   "v=spf1
         ip4:128.173.0.0/16
         ip4:198.82.0.0/16
1         mx
2         ptr a:lennier.cc.vt.edu
3         a:lyta.cc.vt.edu
4         a:dagger.cc.vt.edu
5         a:vivi.cc.vt.edu
6         a:steiner.cc.vt.edu
7         a:zidan e.cc.vt.edu
         ~all"

Although, you will argue DNS caching will save the day, the potential is
still there for long timeouts per attempt on a poorly setup system.  You
just imposed 7 additional DNS lookups on the receiver.  If you are sending a
significant amount of mail out, lets say, X number of systems,  you have
just include the DNS overhead by a factor of 7 on each X system.

It big waste because you have RELAXED the results of the lookup.  Why
Bother?

This is a prime example of where SPF attempts to close a loophole in SMTP
but opens up new ones.

So if you haven't guess by now, I am not a big fan of logic that has the
potential of creating more problems with any kind of relaxation,
specifically when its a new protocol.  Here you are insinuating I am doing
things when in fact, I am not, but at the same time you have a much higher
potential in creating harm with your own setup.

I have been saying this since day one of SPF and now that more people are
implementing SPF, they are beginning to recognize the reason to have either
a HARD pass/reject or NOTHING at all.

But I understand the reasons for the relaxed results. The stated reason is
for migration which I agree. However, migration should come with an
expiration concept. So the first time a SPF receiver encounters a relaxed
SPF policy, a limit is set on its usage for X amount of time. It should not
be a forever thing.

In any case, with your SPF record, you have authorized  SPF systems to add
more scrutiny to any email that has your domains associated with it.   With
your relaxed policy, you added DNS overhead and you invited spoofers to
piggy back off your domain.

I should note that there is a recent discussion in the SPF-DISCUSS forum
regarding relaxed provisions.   One fellow has discussed how he uses it with
an expiration concept augmented with a reporting/notification concept to get
a confirmation on its usage.  Really cool.  This has some merit and worth
exploring.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: Bounce/System Notification Address Verification 
From: [email-masked]
Date: Mon, 04 Jul 2005 16:55:18 -0400
Sender: [email-masked]

--==_Exmh_1120510517_13480P
Content-Type: text/plain; charset=us-ascii

On Mon, 04 Jul 2005 14:07:37 EDT, Hector Santos said:

> The same is true with the client domain name (HELO/EHLO).  If a server's
> local domain name is used and the IP address is unrecognized, then this is
> clear 100% detectable, indisputable SPOOF.  There is no LMAP forwarding
> issue here.

There's no forwarding issue because the parameter of HELO/EHLO isn't used
for forwarding decisions.

I'm curious how you arrived at "100% SPOOF" - are you *positive* that out of all
the mail software out there (including all the products that get simple stuff
like a space between : and < on a MAIL FROM wrong), *NONE* ever does the following:

1) Figure out its IP address that it just got handed by DHCP.
2) Try to look up the PTR to get its hostname, and fail to get a valid answer.
3) Send the preconfigured 'SMTP Smarthost' name on the HELO, just because we know
it and it's handy, and the programmer didn't know about 'HELO [i.p.add.ress]'

Sure, it's *LIKELY* to be a spoof, but:

1) It isn't a *positive* proof.
2) As a result, bouncing mail because of a failure is quite the anti-social thing
to do.

--==_Exmh_1120510517_13480P
Content-Type: application/pgp-signature

--==_Exmh_1120510517_13480P--

To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO 
From: [email-masked]
Date: Mon, 04 Jul 2005 04:23:13 -0400
Sender: [email-masked]

--==_Exmh_1120465392_13223P
Content-Type: text/plain; charset=us-ascii

On Sun, 03 Jul 2005 18:48:42 EDT, Hector Santos said:

> So for example, .911  will the following be possible:
> 
>         miami.911
>         newyork.911
>         houston.911
> 
> without violation the specifications?

I'd recommend that we wait for the rubble to quit bouncing on the FCC's
regulations regarding E-911 service for VoIP before we worry about this one....

--==_Exmh_1120465392_13223P
Content-Type: application/pgp-signature

--==_Exmh_1120465392_13223P--

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: Bounce/System Notification Address Verification
Date: Mon, 4 Jul 2005 14:41:37 -0400
Sender: [email-masked]

On Mon July 4 2005 14:07, Hector Santos wrote:

> You believe you might have misunderstood the RFC 2505 recommendation as it
> is consistent with the LMAP proposals,

LMAP is IRTF. darft form, not IETF.  R = research; not ready for
prime time.

> such as the pending standards in the 
> works
> :
> http://www.ietf.org/internet-drafts/draft-schlitt-spf-classic-02.txt
> http://www.ietf.org/internet-drafts/draft-katz-submitter-01.txt
> http://www.ietf.org/internet-drafts/draft-lyon-senderid-core-01.txt
> http://www.ietf.org/internet-drafts/draft-lyon-senderid-pra-01.txt

None are standards or pending standards, or Standards Track, or
anything similar.  At best, Experimental.

> In other words, if the client IP is your address, then you have NO business
> having my domain being used on your MAIL FROM, because I did not authorize
> your IP address to use the return path with my email domain.

Really -- read 2505.  If you send to an external site which does
aliasing and sends back to some other local-part at the same domain,
guess what MAIL FROM will be; exactly as you set it.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Bounce/System Notification Address Verification
Date: Mon, 4 Jul 2005 14:07:37 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "Bruce Lilly" <[email-masked]>
To: <[email-masked]>
Sent: Monday, July 04, 2005 1:01 PM
Subject: Re: Bounce/System Notification Address Verification

> > Again, another RFC violation because its not support to be your domain
there
> > with my machine as the client.
>
> No. See RFC 2505 section 2.6 (and subsections).  There are reasons
> why BCP 30 saya that an MTA MUST NOT reject based on MAIL FROM being
> within the server's local domain.

You believe you might have misunderstood the RFC 2505 recommendation as it
is consistent with the LMAP proposals, such as the pending standards in the
works
:
http://www.ietf.org/internet-drafts/draft-schlitt-spf-classic-02.txt
http://www.ietf.org/internet-drafts/draft-katz-submitter-01.txt
http://www.ietf.org/internet-drafts/draft-lyon-senderid-core-01.txt
http://www.ietf.org/internet-drafts/draft-lyon-senderid-pra-01.txt

RFC 2505 section 2.6, is consistent with LMAP IP::DOMAIN association
considerations.

A local domain identify thief is 100% detectable when he client IP address
does not comply with the policy of the server's local domains.

In other words, if the client IP is your address, then you have NO business
having my domain being used on your MAIL FROM, because I did not authorize
your IP address to use the return path with my email domain.  However,
before you say "Gotcha!" see below on the LMAP forwarding issues.

The same is true with the client domain name (HELO/EHLO).  If a server's
local domain name is used and the IP address is unrecognized, then this is
clear 100% detectable, indisputable SPOOF.  There is no LMAP forwarding
issue here.

Bruce, if you believe you wish to debate this, then I suggest to read all
the LMAP proposals and pending standards listed above before continue. Then
I will more than happy to hear your "reasonable" issues with them.  You
might find you will be spitting against the wind on this one regardless (not
only with me, but also with the drafts).

And again, I say "local domains" and in this example, the client domain name
(HELO/EHLO) where there is 100% trust in the result.

For MAIL FROM, you can run into forwarding issues, and if you like to argue
this, it is probably where you should get your comments in, in regards to
the above drafts.  The PROs says its a remote configuration and
implementation issue,  the CONs says otherwise.  The SUBMITTER draft address
the forwarding issue.  A non-IETF specification called SRS is another.

I'm on both sides of the fence on this one, but  one thing is clear - its
time to move on and get out of the past. I don't need to reread RFC 2505 to
know what the issues are.

Thanks

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: Bounce/System Notification Address Verification
Date: Mon, 4 Jul 2005 13:01:59 -0400
Sender: [email-masked]

On Thu June 30 2005 08:51, Hector Santos wrote:
> 
> ----- Original Message -----
> From: "william(at)elan.net" <[email-masked]>

> nor does check for a client domain local domain spoof as well:
> 
> 220 sokol.elan.net ESMTP Sendmail 8.13.1/8.13.1;
>         Thu, 30 Jun 2005 05:47:43 -0700
> helo mail.elan.net
> 250 sokol.elan.net Hello adsl-10-44-25.mia.bellsouth.net
>         [65.10.44.25], pleased to meet you
> 
> Again, another RFC violation because its not support to be your domain there
> with my machine as the client.

No. See RFC 2505 section 2.6 (and subsections).  There are reasons
why BCP 30 saya that an MTA MUST NOT reject based on MAIL FROM being
within the server's local domain.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Mon, 4 Jul 2005 09:26:09 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: <[email-masked]>

>> So we have the domain-literal,  for 2821bis, we need to make sure it
>> clearly defines what are all the possibilities.

> I think that will be an impossibility.

Why?

> You cannot fix in stone what people in 2525 have to do.

Like what?  Fix their TCP problems?  I fail to see what the price of rice
in china has anthing to do with dealing with the issue of HELO?

> But an list of all the current posibilities is possible and advisable.
>
> (How about the misformed IP6 literal/name of
> [BAD3:ABBA:BAD1:ABBA:BAD3:ABBA:BAD4:ABBA] for abba dislikers)

What about it?

> >Remember, in my view, for proposals, like SPF and CSV to have any hope
>of
> >success in the future, the client domain name will have to be clearly
>worked
> >out, with some level of enforcement.   For example, before I can make
>the
> >presumption the client domain is CSV or SPF ready, I need to make sure
>it is
> >syntax/value ready first.
>
> Maybe you just have to do all tests.
> precumption 1 it is a domain
> presumption 2 it is a IP4 literal.
> presumption 3 it is a IP6 litersl
> presumption 4 it is an other General-address-literal

Ok? what's the problem?

> And hope that only one exsist or all that exsist refer to the same
>  "place on the internet"

(Hand flying over head! Zoommmm!)

and the price of rice in china is what again?

I fail to see your point.  Are you saying basically "Status Quo?"

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: Hector Santos <[email-masked]>, <[email-masked]>
From: [email-masked]
Subject: Re: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Mon, 4 Jul 2005 12:19:45 +0100
Sender: [email-masked]

>> > True, at least to the present...  See RFC 3696
>>
>> Degenerated into <toplabel>, starting with a letter, ending
>> with letter or digit, otherwise also hyphen.  Something we
>> could improve in 2821bis.  The SPF-spec. needed <toplabel>.
>>
>> Only ALPHA isn't good enough for any future xn-- etc. TLD.
>>
>> > if ICANN goes crazy
>>
>> After their ".net" suicide prepare for the worst :-(  Bye
>
>I'm hope Klensin is following the thread for consideration, insight,
>perspective on the SMTP entities, the syntax, compliance, issues, the
>possible security aspects, etc, to basically remove ambiguity and also
>strengthen the entity, in particular HELO, where so many extended
>applications will be dependent on as well as making sure that it works >in
>backward compatibility mode.
>
>So we have the domain-literal,  for 2821bis, we need to make sure it >clearly
>defines what are all the possibilities.

I think that will be an impossibility.
You cannot fix in stone what people in 2525 have to do.
But an list of all the current posibilities is possible and advisable.

(How about the misformed IP6 literal/name of [BAD3:ABBA:BAD1:ABBA:BAD3:ABBA:BAD4:ABBA] for abba dislikers)

>
>Claus raised a valid point, how do you know it is not a domain?   I >think
>1123 clears that up.   Something for John to include in 2826bis.
>
>However, if that could change, then I would like to see how and is this
>"possibility" enough to ignore any syntax/value issues?

>
>If ICANN do allow for numeric TLDs, in order to comply with RFC 2821 >Today,
>it could be allowed for number values over 255.
>
>So for example, .911  will the following be possible:
>
>        miami.911
>        newyork.911
>        houston.911
>
>without violation the specifications?
>
>Remember, in my view, for proposals, like SPF and CSV to have any hope >of
>success in the future, the client domain name will have to be clearly >worked
>out, with some level of enforcement.   For example, before I can make >the
>presumption the client domain is CSV or SPF ready, I need to make sure >it is
>syntax/value ready first.

Maybe you just have to do all tests.
precumption 1 it is a domain
presumption 2 it is a IP4 literal.
presumption 3 it is a IP6 litersl
presumption 4 it is an other General-address-literal
And hope that only one exsist or all that exsist refer to the same "place on the internet"

Date: Sun, 03 Jul 2005 19:01:43 -0400
From: Keith Moore <[email-masked]>
To: Bruce Lilly <[email-masked]>
Cc: [email-masked]
Subject: Re: BCP 14 a.k.a. RFC 2119 keyword meanings (Was spamops-04)
Sender: [email-masked]

>>>it states "that there may exist valid reasons in particular
>>>circumstances to ignore a particular item ...", e.g. while the
>>>recommendation is made for good cause, there may be particular
>>>protocol states, network conditions, backward compatibility issues,
>>>etc. which may apply -- and a careful specification writer will make
>>>clear why a recommendation is made and under what sorts of 
>>>circumstances the recommendation might be ignored.
>>
>>Uh, no.  SHOULD and SHOULD NOT exist precisely because specification
>>writers too often get into the business of trying to specify exactly 
>>when it is permissible for an implementor to make an exception to a 
>>specification.
> 
> 
> I'm not sure I understand your statement...  Should/SHOULD NOT
> specifically provide for exceptions in particular circumstances;

yes, but it's not up to the author of the specification to "make 
clear...under what sorts of circumstances the recommendation might 
[reasonably ] be ignored".

> So, convince me that MAY is an imperative as you earlier implied, then
> we can move on to SHOULD.

it's hard to interpret MAY as an imperative.  then again, nobody ever 
cited 2119 just so that he could reference its definition for MAY.

>>I can see why you reached that conclusion.  But there are subtle and 
>>important differences between the definition given in 2119 and the 
>>normal meaning of "recommended" (just as there are subtle and important 
>>differences between 2119's SHOULD and "should").  If we want to 
>>encourage a practice without imposing the requirement that "the full 
>>implications _must_ be understood and carefully weighed before choosing 
>>a different course" (emphasis mine) then we need another word besides 
>>RECOMMENDED,
> 
> 
> Such as "encouraged" perhaps?

or perhaps ENCOURAGED to make it clear that the word is being used as an 
alternative to MUST, SHOULD, RECOMMENDED, etc.

Keith

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Sun, 3 Jul 2005 18:48:42 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "Frank Ellermann" <[email-masked]>

> > True, at least to the present...  See RFC 3696
>
> Degenerated into <toplabel>, starting with a letter, ending
> with letter or digit, otherwise also hyphen.  Something we
> could improve in 2821bis.  The SPF-spec. needed <toplabel>.
>
> Only ALPHA isn't good enough for any future xn-- etc. TLD.
>
> > if ICANN goes crazy
>
> After their ".net" suicide prepare for the worst :-(  Bye

I'm hope Klensin is following the thread for consideration, insight,
perspective on the SMTP entities, the syntax, compliance, issues, the
possible security aspects, etc, to basically remove ambiguity and also
strengthen the entity, in particular HELO, where so many extended
applications will be dependent on as well as making sure that it works in
backward compatibility mode.

So we have the domain-literal,  for 2821bis, we need to make sure it clearly
defines what are all the possibilities.

Claus raised a valid point, how do you know it is not a domain?   I think
1123 clears that up.   Something for John to include in 2826bis.

However, if that could change, then I would like to see how and is this
"possibility" enough to ignore any syntax/value issues?

If ICANN do allow for numeric TLDs, in order to comply with RFC 2821 Today,
it could be allowed for number values over 255.

So for example, .911  will the following be possible:

        miami.911
        newyork.911
        houston.911

without violation the specifications?

Remember, in my view, for proposals, like SPF and CSV to have any hope of
success in the future, the client domain name will have to be clearly worked
out, with some level of enforcement.   For example, before I can make the
presumption the client domain is CSV or SPF ready, I need to make sure it is
syntax/value ready first.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: Bruce Lilly <[email-masked]>
Reply-To: Bruce Lilly <[email-masked]>
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: BCP 14 a.k.a. RFC 2119 keyword meanings (Was spamops-04)
Date: Sun, 3 Jul 2005 18:38:25 -0400
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Sun July 3 2005 17:13, Keith Moore wrote:

> who said anything about making an implementor (or whomever) feel good?

An implementer may think he has a "good" reason to ignore the
recommended minimum timeouts in SMTP (for example), and it may make
him feel "good" to see that lots of messages are being rejected.  That
is not necessarily a *valid* reason...

[re SHOULD]
> > it states "that there may exist valid reasons in particular
> > circumstances to ignore a particular item ...", e.g. while the
> > recommendation is made for good cause, there may be particular
> > protocol states, network conditions, backward compatibility issues,
> > etc. which may apply -- and a careful specification writer will make
> > clear why a recommendation is made and under what sorts of 
> > circumstances the recommendation might be ignored.
> 
> Uh, no.  SHOULD and SHOULD NOT exist precisely because specification
> writers too often get into the business of trying to specify exactly 
> when it is permissible for an implementor to make an exception to a 
> specification.

I'm not sure I understand your statement...  Should/SHOULD NOT
specifically provide for exceptions in particular circumstances;
MUST/MUST NOT do not (they are "absolute").  Since SHOULD/SHOULD NOT
apply in *particular* circumstances, are you suggesting that lack of
clarity is a good thing in a specification?

> >> That language was intended to apply to all of the keywords in the 
> >> document.  e.g. SHOULD is also an imperative.

> It is unwise to assume that the author of 2119 consulted dictionary.com 
> before using that word.

No such assumption is being made.  The only assumption is that as RFCs
are written in English, words in them have their English meaning unless
otherwise stated.

So, convince me that MAY is an imperative as you earlier implied, then
we can move on to SHOULD.

> I can see why you reached that conclusion.  But there are subtle and 
> important differences between the definition given in 2119 and the 
> normal meaning of "recommended" (just as there are subtle and important 
> differences between 2119's SHOULD and "should").  If we want to 
> encourage a practice without imposing the requirement that "the full 
> implications _must_ be understood and carefully weighed before choosing 
> a different course" (emphasis mine) then we need another word besides 
> RECOMMENDED,

Such as "encouraged" perhaps?

Date: Sun, 03 Jul 2005 17:13:34 -0400
From: Keith Moore <[email-masked]>
To: Bruce Lilly <[email-masked]>
Cc: [email-masked]
Subject: Re: BCP 14 a.k.a. RFC 2119 keyword meanings (Was spamops-04)
Sender: [email-masked]

>> No, I didn't say "SHOULD means MUST", I said "MUST do this unless
>> you have a good reason to make an exception."
> 
> Yes, hence the ellipsis.
> 
> 
>> Which is a paraphrase of the more carefully worded language in
>> 2119.
> 
> Actually not.  Note that the 2119 text for SHOULD does not give 
> implementers, end users, etc. leave to "make an exception" because 
> doing so makes them feel good,

who said anything about making an implementor (or whomever) feel good?

> it states "that there may exist valid reasons in particular
> circumstances to ignore a particular item ...", e.g. while the
> recommendation is made for good cause, there may be particular
> protocol states, network conditions, backward compatibility issues,
> etc. which may apply -- and a careful specification writer will make
> clear why a recommendation is made and under what sorts of 
> circumstances the recommendation might be ignored.

Uh, no.  SHOULD and SHOULD NOT exist precisely because specification
writers too often get into the business of trying to specify exactly 
when it is permissible for an implementor to make an exception to a 
specification.

>>> In particular, MUST is an imperative and MUST only be used where 
>>> there is potential for harm or where required for interoperation.
>>> 
>> 
>> That language was intended to apply to all of the keywords in the 
>> document.  e.g. SHOULD is also an imperative.
> 
> 
> http://www.dictionary.com/search?q=imperative gives definitions of 
> imperative which include such terms as "command", "obligation", and 
> "rule [...] that compels a certain behavior".

It is unwise to assume that the author of 2119 consulted dictionary.com 
before using that word.  I can recall cases where that same author 
objected to the use of SHOULD in a document because he didn't think it 
was necessary to prevent harm or to ensure interoperability.

>> Now I think you are the one interpreting 2119 liberally.  It
>> doesn't say that SHOULD is a recommendation.
> 
> 
> Considering that SHOULD has the same BCP 14 meaning as the adjective 
> RECOMMENDED, and that a recommendation is something which is 
> recommended, it's pretty hard not to reach that conclusion.  You're 
> welcome to try to rationalize not doing so...

I can see why you reached that conclusion.  But there are subtle and 
important differences between the definition given in 2119 and the 
normal meaning of "recommended" (just as there are subtle and important 
differences between 2119's SHOULD and "should").  If we want to 
encourage a practice without imposing the requirement that "the full 
implications _must_ be understood and carefully weighed before choosing 
a different course" (emphasis mine) then we need another word besides 
RECOMMENDED, or we need to use definitions other than those in 2119.

Keith

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Strict RFC x821 Compliant: HELO/EHLO
Date:  Sun, 03 Jul 2005 22:47:26 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 18
Sender: [email-masked]

Bruce Lilly wrote:

>> "a valid host name can never have the dotted-decimal form
>> #.#.#.#, since at least the highest-level component label
>> will be alphabetic."

> True, at least to the present...  See RFC 3696

Degenerated into <toplabel>, starting with a letter, ending
with letter or digit, otherwise also hyphen.  Something we
could improve in 2821bis.  The SPF-spec. needed <toplabel>.

Only ALPHA isn't good enough for any future xn-- etc. TLD.

> if ICANN goes crazy

After their ".net" suicide prepare for the worst :-(  Bye

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Sun, 3 Jul 2005 15:16:23 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: "Bruce Lilly" <[email-masked]>

> > See the discussion in RFC 1123 section 2.1: "a valid host name can never
> > have the dotted-decimal form #.#.#.#, since at least the highest-level
> > component label will be alphabetic."
>
> True, at least to the present...  See RFC 3696 (where the top-level
> is not numeric 1123 reference is also mentioned); if ICANN goes
> crazy, that rule might not apply in the future.

You mean, like a possible ".911" TLD?  <g>

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Sun, 3 Jul 2005 14:16:40 -0400
Cc: Tony Finch <[email-masked]>
Sender: [email-masked]

On Sun July 3 2005 11:43, Tony Finch wrote:
> 
> On Sat, 2 Jul 2005, Claus Assmann wrote:

> > How do you know it's not a domain? Do you perform a DNS lookup?
> 
> See the discussion in RFC 1123 section 2.1: "a valid host name can never
> have the dotted-decimal form #.#.#.#, since at least the highest-level
> component label will be alphabetic."

True, at least to the present...  See RFC 3696 (where the top-level
is not numeric 1123 reference is also mentioned); if ICANN goes
crazy, that rule might not apply in the future.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Sun, 3 Jul 2005 14:08:50 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: "Claus Assmann" <[email-masked]>

> Ok, let's try this again: The statement was:
>
> "This would be an illegal syntax per the ABNF:"
>
> This statement is wrong. The parameter is _syntactically_ valid (as
> I pointed out before by quoting the grammar).  Maybe it would be
> useful to change the syntax to reflect that "the highest-level
> component label will be alphabetic.".

Oh, I see your point.

The ABNF is:

      address-literal = "[" IPv4-address-literal /
                            IPv6-address-literal /
                            General-address-literal "]"

Section 4.1.3 Address Literals specifically defines the format for each of
the above.  For the sake of simplicity, using IPv4:

      IPv4-address-literal = Snum 3("." Snum)
      Snum = 1*3DIGIT  ; representing a decimal integer
            ; value in the range 0 through 255

This defines an alphanumeric dotted string format.

But 2821 does not specifically define or say the address-literal could not
be a A record in DNS.

I see your point... however.

> As long as that isn't the
> case, the command above is correct according to the grammar given
> in RFC 2821 (that's why my software doesn't reject it in contrast
> to the _syntactically_ invalid MAIL command that triggered this
> HELO/EHLO discussion).

You are technically correct per 2821, but 2821 *UPDATES* 1123, hence 2821
overrides various parts in 1123, but other items in 1123 not updated by 2821
still take hold.

So 1123 covers the definition of what is considered a valid TLD based domain
vs. a dotted-address format.

Remember, 1123 came first before 2821 so you will find systems following its
guidelines as well.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: Bruce Lilly <[email-masked]>
Reply-To: Bruce Lilly <[email-masked]>
Organization: Bruce Lilly
To: [email-masked]
Subject: BCP 14 a.k.a. RFC 2119 keyword meanings (Was spamops-04)
Date: Sun, 3 Jul 2005 14:10:39 -0400
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Sun July 3 2005 11:13, Keith Moore wrote:
> > Keith said SHOULD means MUST ..., which is clearly not the case, 
> > otherwise the separate terms wouldn't be defined in separate sections
> > with separate meanings and given separate conditions for use (in
> > section 6).
> 
> No, I didn't say "SHOULD means MUST", I said "MUST do this unless you
> have a good reason to make an exception."

Yes, hence the ellipsis.

> Which is a paraphrase of the 
> more carefully worded language in 2119.

Actually not.  Note that the 2119 text for SHOULD does not give
implementers, end users, etc. leave to "make an exception" because
doing so makes them feel good, it states "that there may exist valid
reasons in particular circumstances to ignore a particular item ...",
e.g. while the recommendation is made for good cause, there may be
particular protocol states, network conditions, backward compatibility
issues, etc. which may apply -- and a careful specification writer will
make clear why a recommendation is made and under what sorts of
circumstances the recommendation might be ignored.

> > In particular, MUST is an imperative and MUST only be used where
> > there is potential for harm or where required for interoperation. 
> 
> That language was intended to apply to all of the keywords in the 
> document.  e.g. SHOULD is also an imperative.

http://www.dictionary.com/search?q=imperative gives definitions of
imperative which include such terms as "command", "obligation", and
"rule [...] that compels a certain behavior".

[re. MUST]
> > It expresses an absolute requirement.  SHOULD is a recommendation.  A
> > recommendation which should not be made lightly or ignored without a
> > valid reason, but certainly not an absolute requirement.
> 
> Now I think you are the one interpreting 2119 liberally.  It doesn't say 
> that SHOULD is a recommendation.

Considering that SHOULD has the same BCP 14 meaning as the adjective
RECOMMENDED, and that a recommendation is something which is
recommended, it's pretty hard not to reach that conclusion.  You're
welcome to try to rationalize not doing so...

http://www.dictionary.com/search?q=recommendation gives definitions
of recommendation which include none of the terms mentioned above
for imperative, instead using terms such as "advisable", "desirable",
and "favorable".   Clearly "imperative" and "recommendation"
(SHOULD/SHOULD NOT) are very different, and a recommendation, i.e.
something which is RECOMMENDED, is clearly not an imperative.

2119 says MUST means an absolute requirement, and MUST NOT means an
absolute prohibition.  http://www.dictionary.com/search?q=requirement
defines requirement in terms of "necessity" and "obligatory".  N.B.
"obligation" and "obligatory"; MUST is an imperative. 
http://www.dictionary.com/search?q=prohibition defines prohibition
using the terms "forbids" and "commanding"; N.B. "command" and
"commanding"; MUST NOT is an imperative.  Ditto for SHALL/SHALL NOT.

RFC 2119 gives the meaning of the keyword MAY as truly optional.
http://www.dictionary.com/search/q=optional gives definitions of
optional in terms such as "not compulsory" and "not necessary".
N.B. "compels" vs. "not compulsory; clearly MAY is not an imperative;
it is clearly the opposite of an imperative.  MAY is a BCP 14
keyword; you are free to try to convince me that something which is
explicity "not compulsory" somehow "compels a certain behavior"... 

> It also turns out that there are occasionally other good reasons to use 
> MUST, SHOULD, etc. which has resulted in a couple of RFCs either 
> defining these keywords internally or referencing another set of 
> definitions of these keywords.

True, but irrelevant; that doesn't bear on the BCP 14 meanings.

Date: Sun, 3 Jul 2005 10:12:48 -0700
From: Claus Assmann <[email-masked]>
To: [email-masked]
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Reply-To: [email-masked]
Mail-Followup-To: [email-masked]
Sender: [email-masked]

On Sun, Jul 03, 2005, Tony Finch wrote:

> > > helo 65.10.44.25

> > > This would be an illegal syntax per the ABNF:

> > How do you know it's not a domain? Do you perform a DNS lookup?

> See the discussion in RFC 1123 section 2.1: "a valid host name can never
> have the dotted-decimal form #.#.#.#, since at least the highest-level
> component label will be alphabetic."

Ok, let's try this again: The statement was:

"This would be an illegal syntax per the ABNF:"

This statement is wrong. The parameter is _syntactically_ valid (as
I pointed out before by quoting the grammar).  Maybe it would be
useful to change the syntax to reflect that "the highest-level
component label will be alphabetic.". As long as that isn't the
case, the command above is correct according to the grammar given
in RFC 2821 (that's why my software doesn't reject it in contrast
to the _syntactically_ invalid MAIL command that triggered this
HELO/EHLO discussion).

Date: Sun, 3 Jul 2005 10:02:25 -0700
From: Bill McQuillan <[email-masked]>
To: SMTP Discussion <[email-masked]>
Subject: Re: spamops-04
Sender: [email-masked]

On Sun, 2005-07-03, Keith Moore wrote:

>> Keith said SHOULD means MUST ..., which is clearly not the case, 
>> otherwise the separate terms wouldn't be defined in separate sections
>> with separate meanings and given separate conditions for use (in
>> section 6).

> No, I didn't say "SHOULD means MUST", I said "MUST do this unless you
> have a good reason to make an exception."  Which is a paraphrase of the
> more carefully worded language in 2119.  Of course, the 2119 language is
> more accurate than the paraphrase.

Perhaps changing the "MUST do this unless..." to "must do this unless..."
would avoid confusion about the strict meaning of the imperative MUST?

-- 
Bill McQuillan <[email-masked]>

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Sun, 3 Jul 2005 12:53:47 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: "Tony Finch" <[email-masked]>

>>> helo 65.10.44.25
>>> This would be an illegal syntax per the ABNF:

>> How do you know it's not a domain? Do you perform a DNS lookup?

> See the discussion in RFC 1123 section 2.1: "a valid host name can never
> have the dotted-decimal form #.#.#.#, since at least the highest-level
> component label will be alphabetic."

Right, the TLD (the far right .#) can not be alphanumeric. (Probably it can
also be said it is known to not exist for registered top-level domains in
ICANN.)

I do a inet_addr() conversion and if this comes back as INADDR_NONE (-1),
then you know its not a valid dotted-decimal format.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Sun, 3 Jul 2005 16:43:32 +0100
From: Tony Finch <[email-masked]>
To: [email-masked]
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Sender: [email-masked]

On Sat, 2 Jul 2005, Claus Assmann wrote:
> On Sat, Jul 02, 2005, Hector Santos wrote:
>
> [Please do NOT Cc' me.]

I suggest setting a Reply-To header if you don't want people to CC: you.

> > 220 zardoc.esmtp.org ESMTP sendmail X.0.0.Alpha4.0
> > helo 65.10.44.25
> > 250 zardoc.esmtp.org Hi there
> >
> > This would be an illegal syntax per the ABNF:
>
> How do you know it's not a domain? Do you perform a DNS lookup?

See the discussion in RFC 1123 section 2.1: "a valid host name can never
have the dotted-decimal form #.#.#.#, since at least the highest-level
component label will be alphabetic."

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BISCAY: WEST 5 OR 6 BECOMING VARIABLE 3 OR 4. SHOWERS AT FIRST. MODERATE OR
GOOD.

Date: Sun, 03 Jul 2005 11:13:30 -0400
From: Keith Moore <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: spamops-04
Sender: [email-masked]

> Keith said SHOULD means MUST ..., which is clearly not the case, 
> otherwise the separate terms wouldn't be defined in separate sections
> with separate meanings and given separate conditions for use (in
> section 6).

No, I didn't say "SHOULD means MUST", I said "MUST do this unless you
have a good reason to make an exception."  Which is a paraphrase of the
more carefully worded language in 2119.  Of course, the 2119 language is
more accurate than the paraphrase.

> In particular, MUST is an imperative and MUST only be used where
> there is potential for harm or where required for interoperation. 

That language was intended to apply to all of the keywords in the 
document.  e.g. SHOULD is also an imperative.

It also turns out that there are occasionally other good reasons to use 
MUST, SHOULD, etc. which has resulted in a couple of RFCs either 
defining these keywords internally or referencing another set of 
definitions of these keywords.

> It expresses an absolute requirement.  SHOULD is a recommendation.  A
> recommendation which should not be made lightly or ignored without a
> valid reason, but certainly not an absolute requirement.

Now I think you are the one interpreting 2119 liberally.  It doesn't say 
that SHOULD is a recommendation.

Keith

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: spamops-04
Date: Sun, 3 Jul 2005 07:31:10 -0400
Cc: Bill McQuillan <[email-masked]>
Sender: [email-masked]

On Sat July 2 2005 15:02, Bill McQuillan wrote:
> 
> On Sat, 2005-07-02, Bruce Lilly wrote:
> 
> > On Sat July 2 2005 09:23, Keith Moore wrote:
> 
> ...
> 
> >> which essentially means "you MUST do this unless you have a  
> >> good reason to make an exception".
> 
> > No. (sigh)  I wish people would stop trying to reinterpret BCP 14 (a.k.a.
> > RFC 2119).  The BCP 14 text is concise and clear.  Here in their entirety
> > are sections 1-6 of BCP 14:
> 
> ...
> 
> > 3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
> >    may exist valid reasons in particular circumstances to ignore a
> >    particular item, but the full implications must be understood and
> >    carefully weighed before choosing a different course.
> 
> I'm sorry, but with careful reading, it seems to me that Keith's statement is an accurate paraphrase. Could you expand on what you see as the difference?

Keith said SHOULD means MUST ..., which is clearly not the case,
otherwise the separate terms wouldn't be defined in separate
sections with separate meanings and given separate conditions for
use (in section 6).  In particular, MUST is an imperative and MUST
only be used where there is potential for harm or where required
for interoperation.  It expresses an absolute requirement.  SHOULD
is a recommendation.  A recommendation which should not be made
lightly or ignored without a valid reason, but certainly not an
absolute requirement.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: MAIL FROM:
Date: Sun, 3 Jul 2005 03:00:55 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "Claus Assmann" <[email-masked]>

> > And this just my personal opinion, this level of MAIL FROM:sp syntax
check
> > that has absolute no SMTP level correlation to a malicious sender, yet
also
> > inconsistent with the majority of operations on the internet.
>
> Well, that's your opinion. My opinion is that every relaxation leads
> to overhead and more code (and in the worst case to security
> problems). Where do you draw the line?

I feel ya.  100%. I agree with your thinking. I'm a stickler when it comes
to comformance.   But I thin,.  agan, just my opinion,  I would make an
exception simply because its already burned in.

In fact, I think 2821bis should probably consider relaxing this with a
similar statement it has for the trailing space consideration:

Technically, the spec for the MAIL command is::

      MAIL FROM:<reverse-path> [SP <mail-parameters> ] <CRLF>

No space before <CRLF> but section 4.1.1 says:

4.1.1 Command Semantics and Syntax

   .....(In the interest of improved interoperability, SMTP
   receivers are encouraged to tolerate trailing white space before the
   terminating <CRLF>.)

I think this should also include the space before the reverse-path.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Sat, 02 Jul 2005 21:06:43 -0400
From: Keith Moore <[email-masked]>
To: Frank Ellermann <[email-masked]>
Cc: [email-masked]
Subject: Re: PRA question
Sender: [email-masked]

>>Frank, if you were willing to write up an I-D of the form
>>"SPF experiment considered harmful", explaining just why this
>>is a Bad Idea, that might be a valuable community service.
> 
> 
> It's more like "using PRA with v=spf1 considered harmful", and
> from the v=spf1 side it's already a NOT RECOMMENDED (plus the
> "without explicit consent" disclaimer).  From the PRA side it
> is a SHOULD, do you think a new I-D addressing only why it is
> really NOT RECOMMENDED helps ?

every little bit helps.  and it's better to get this kind of feedback 
published sooner, than to hold it up waiting for a more complete document.

> The part about the option is ready, up to some of the security
> considerations.  And assuming that this I-D somehow makes it,
> should it move the PRA RFC to historic, or update it ?

your I-D wouldn't necessarily have to be published as an RFC to be 
useful.  I wouldn't worry about the reclassification of the PRA RFC just 
yet.

> With the discussions between Brian, Dave, Sam, and John on the
> general list another idea I had today was to update RFC 2026
> so that authors could insist on a proper "IETF last call" for
> an experimental RFC even if it was not created by an IETF WG.

I'd be in favor of something like that.  Experimental documents should 
describe experiments, test conditions, what is to be measured and how it 
is to be measured, and also describe whatever preventative measures are 
needed to limit harm done by the experiment.

Date: Sat, 2 Jul 2005 16:40:51 -0700
From: Claus Assmann <[email-masked]>
To: [email-masked]
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Mail-Followup-To: Claus Assmann <[email-masked]>, [email-masked]
Sender: [email-masked]

On Sat, Jul 02, 2005, Hector Santos wrote:

> From: "Claus Assmann" <[email-masked]>

> > [Please do NOT Cc' me.]

> You got it!

For this mail, but not for the other(s) :-(

> > How do you know it's not a domain? Do you
> > perform a DNS lookup?
> 
> No. Not for this specific test.

So why do you say it's an invalid syntax? As I quoted from RFC 2821
it's valid (please correct me if I'm wrong).

> And this just my personal opinion, this level of MAIL FROM:sp syntax check
> that has absolute no SMTP level correlation to a malicious sender, yet also
> inconsistent with the majority of operations on the internet.

Well, that's your opinion. My opinion is that every relaxation leads
to overhead and more code (and in the worst case to security
problems). Where do you draw the line?  If everyone makes up his
own "line" ("this is ok, but that isn't") then we have some nice
chaos (e.g., like you pointing out "But Exim and Sendmail 8 accept
this"). And last, but not least, some MTAs are so "nice" that they
accept almost all garbage, which is nice "leftover" from the good
old days, but not appropriate anymore. It leads to sloppy programming
and pressure on other MTA authors ("but MTA Z accepts

Mail From: (My Name) [email-masked]

why doesn't yours?"). I'm really tired of adding all these workarounds
to sendmail 8 just that it can "communicate" with some so-called "MTA"
from M$.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Sat, 2 Jul 2005 17:37:50 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "Claus Assmann" <[email-masked]>

> 220-winserver.com Wildcat! ESMTP Server v6.1.451.5 ready
> ehlo 1.2.3.4
> 250-winserver.com, Pleased to meet you.
>

As I noted, this test was moved to RCPT, but apparently I turned off this
testing to gather some statistics on the completed transactions.

It is enabled now at the SMTP Helo level:

220-winserver.com Wildcat! ESMTP Server v6.1.451.5 ready
220 ....
ehlo 1.2.3.4
501 Invalid EHLO client address.
ehlo [1.2.3.4]
501 Invalid EHLO client address.

I think it is a low footprint test do to.  Read my message to Valdis
regarding some implementation and support experiences behind it.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Date: Sat, 2 Jul 2005 17:05:18 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: "Claus Assmann" <[email-masked]>

> [Please do NOT Cc' me.]

You got it!

> How do you know it's not a domain? Do you
> perform a DNS lookup?

No. Not for this specific test.

> Hmm, why again are you complaining about my software?

I am not.  Just pointing out the inconsistencies.  You are enforcing one
item but not the other.

And this just my personal opinion, this level of MAIL FROM:sp syntax check
that has absolute no SMTP level correlation to a malicious sender, yet also
inconsistent with the majority of operations on the internet.

I ask this question:  Is there a correlation of bad MAIL FROM:sp syntax with
malicous senders?  Or you just hate windows users? :-)

> 220-winserver.com Wildcat! ESMTP Server v6.1.451.5 ready
> ehlo 1.2.3.4
> 250-winserver.com, Pleased to meet you.

Touche!  :-)

However, per RFC 2821 recommendation in section 3.3, we moved all extended
SMTP level checks until the RCPT forward path is known.

This is done to reduce all overhead to absolute minimum and need to do any
extended SMTP level checking.

A good bit of the transactions are bad RCPT addresses. So there is no need
to do anything unless an Anonymous Final Destination (AFD) transaction is
determined.

For my personal setup, I believe I relaxed the bracket test, but not the IP
spoof test.

If you want to see the local domain spoof trap, go further to RCPT state.

You can also try it from: http://www.winserver.com/testwcsap.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: "Bill McQuillan" <[email-masked]>, "SMTP Discussion" <[email-masked]>
Subject: Re: spamops-04
Date: Sat, 2 Jul 2005 16:42:44 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: "Bill McQuillan" <[email-masked]>

> I'm sorry, but with careful reading, it seems to me that Keith's
> statement is an accurate paraphrase. Could you expand on what
> you see as the difference?

I agree with both.

My input is the difference is the level of importance, i.e., "Security vs
Non Security" related feature.

If a "SHOULD" is used in Security related items, well, one can view that is
an highly encouraged item to implement.

For a non-security item, a SHOULD has less importance.

It can also have a "urgency" when or probably more "migration" factor as
well.

Overall I think the current concise language is necessary, but at times, the
specs have the missing the "importance and when" concepts one seeks to know
sometimes.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: "Claus Assmann" <[email-masked]>, <[email-masked]>, "Arnt Gulbrandsen" <[email-masked]>
Subject: Re: Strict RFC x821 Compliance: HELO/EHLO
Date: Sat, 2 Jul 2005 16:21:21 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

Follow up:

----- Original Message -----
From: "Hector Santos" <[email-masked]>

> > In what sense is the handling of the HELO parameter
> > failing to "protect" the domain?
>
> Because protection of your OWN domains has 100% trust behind it.
>
> If you are going to have a strict syntax for the minor space issue on the
> MAIL command which has no SMTP level correlation as a malicious
> sender, then the HELO domain literal syntax RFC requirement which
> does have a strong SMTP level correlation as a malicious sender (when
> checking for local domain/ip spoofing) should be applied.
>

Valdis,  I just want to add that there are two issues here if one attempts
to add some level of enforcement:

    - Incorrect syntax (no brackets) but correct IP
    - Correct Syntax,  but incorrect IP

For the incorrect IP, the only issue I have come across here is with NATs,
Firewall, sending the wrong IP address. But it is strong Anti-Spoof detector
when the sender is using your IP address which is an attempt to simulate an
internal network router.

For the no brackets,  I found a few legacy software that did not provide the
brackets, but the IP was correct.

When (rarely) reported by customers, the latter (no brackets) is the more
common question on how to turn off the checking.    The NAT/FireWall issue
reports were corrected on their end as the more prefer way to go rather than
turning off the low level, low overhead local domain spoof detection.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Sat, 2 Jul 2005 13:17:45 -0700
From: Claus Assmann <[email-masked]>
To: [email-masked]
Subject: Re: Strict RFC x821 Compliant: HELO/EHLO
Mail-Followup-To: Claus Assmann <[email-masked]>, [email-masked]
Sender: [email-masked]

On Sat, Jul 02, 2005, Hector Santos wrote:

[Please do NOT Cc' me.]

> 220 zardoc.esmtp.org ESMTP sendmail X.0.0.Alpha4.0
> helo 65.10.44.25
> 250 zardoc.esmtp.org Hi there
> 
> This would be an illegal syntax per the ABNF:
> 
>       address-literal = "[" IPv4-address-literal /
>                             IPv6-address-literal /
>                             General-address-literal "]"

      helo            = "HELO" SP Domain CRLF

      Domain = (sub-domain 1*("." sub-domain)) / address-literal
      sub-domain = Let-dig [Ldh-str]

How do you know it's not a domain? Do you perform a DNS lookup?

PS:

220-winserver.com Wildcat! ESMTP Server v6.1.451.5 ready
220-************** WARNING:  FOR AUTHORIZED USE ONLY! **********************
220-* THIS SYSTEM DO NOT AUTHORIZE THE USE OF ITS PROPRIETARY COMPUTERS    *
220-* AND COMPUTER NETWORKS TO ACCEPT, TRANSMIT, OR DISTRIBUTE UNSOLICITED *
220-* BULK E-MAIL SENT FROM THE INTERNET. THIS SYSTEM WILL RESTRICT ACCESS *
220-* TO CAN-SPAM (US S. 877) COMPLIANT CLIENTS ONLY.                      *
220 ************************************************************************
ehlo 1.2.3.4
250-winserver.com, Pleased to meet you.

Hmm, why again are you complaining about my software?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: spamops-04
Date:  Sat, 02 Jul 2005 22:07:56 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 6
Sender: [email-masked]

Keith Moore wrote:
 
> The "C" in "BCP" means "current", not "common".

Okay, "current" is more flexible, it covers your "ASAP" idea.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: "Claus Assmann" <[email-masked]>, <[email-masked]>, "Arnt Gulbrandsen" <[email-masked]>
Subject: Strict RFC x821 Compliance: HELO/EHLO
Date: Sat, 2 Jul 2005 15:59:43 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: <[email-masked]>

>> Wonderful!  Its worry about a space, but it doesn't
>> protect its own local domains! <g>

> What can I say? Claus is a standards-conforming
> kind of guy :)

Well, it also doesn't check for brackets syntax :-)

220 zardoc.esmtp.org ESMTP sendmail X.0.0.Alpha4.0
helo 65.10.44.25
250 zardoc.esmtp.org Hi there

No big deal. :-)

> In what sense is the handling of the HELO parameter
> failing to "protect" the domain?

Because protection of your OWN domains has 100% trust behind it.

If you are going to have a strict syntax for the minor space issue on the
MAIL command which has no SMTP level correlation as a malicious sender, then
the HELO domain literal syntax RFC requirement which does have a strong SMTP
level correlation as a malicious sender (when checking for local domain/ip
spoofing) should be applied.

Like I said, no big deal.  :-)

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: spamops-04
Date:  Sat, 02 Jul 2005 21:54:45 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 26
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

Keith Moore wrote:

> MAY
> DISCOURAGED
> STRONGLY DISCOURAGED
> SHOULD NOT
> MUST NOT

May favourite "STRONGLY DISCOURAGED" is in STD 11:

| Note:  THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.

Like SHOULD NOT today with a strong tendency towards MUST NOT.  
ENCOURAGED / OUGHT could be clear, less than SHOULD.

USEFOR once tried an OUGHTificaction, the effect was hilarious.

> I also think we'll need to find language that conveys a sense
> of timeframe.  maybe IMMEDIATELY/ASAP/WHEN PRACTICABLE/BY
> {date} ?

So far I thought that "by [date]" isn't allowed.  The others
are clear, but why as an upper-case keyword ?  The 2119-kewords
have sometimes strange effects, in my last "SHOULD NOT"-debate
others interpreted "SHOULD NOT" as "allowed" <shudder />  Bye

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Subject: Strict RFC x821 Compliant: HELO/EHLO
Date: Sat, 2 Jul 2005 15:49:18 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: <[email-masked]>

> Umm.. The *syntax* doesn't become illegal just because
> it's spoofed.  The *semantics* may be a bit dicey, but
> [ ip.addr.notation ] is perfectly legal syntax on the helo.

True, dicey, you are correct. How about this?

220 zardoc.esmtp.org ESMTP sendmail X.0.0.Alpha4.0
helo 65.10.44.25
250 zardoc.esmtp.org Hi there

This would be an illegal syntax per the ABNF:

      address-literal = "[" IPv4-address-literal /
                            IPv6-address-literal /
                            General-address-literal "]"

Also Claus's machine also disconnected after about 1 minute:

220 zardoc.esmtp.org ESMTP sendmail X.0.0.Alpha4.0
helo 65.10.44.25
250 zardoc.esmtp.org Hi there
Connection to host lost.

>From RFC 2821 Section 4.5.3.2 Timeouts

   An SMTP server SHOULD have a timeout of at least
   5 minutes while it is awaiting the next command
   from the sender.

So how far we want to go?  <g>

But you lean with you, better to have said:

    "illegal literal domain"
    "illegal value"
    "illegal parameter"

For  the MAIL command, the parameter was illegal due to an incorrect syntax.

For the HELO command, the parameter was illegal due to an incorrect domain
literal.

etc, etc.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Sat, 2 Jul 2005 12:02:24 -0700
From: Bill McQuillan <[email-masked]>
To: SMTP Discussion <[email-masked]>
Subject: Re: spamops-04
Sender: [email-masked]

On Sat, 2005-07-02, Bruce Lilly wrote:

> On Sat July 2 2005 09:23, Keith Moore wrote:

...

>> which essentially means "you MUST do this unless you have a  
>> good reason to make an exception".

> No. (sigh)  I wish people would stop trying to reinterpret BCP 14 (a.k.a.
> RFC 2119).  The BCP 14 text is concise and clear.  Here in their entirety
> are sections 1-6 of BCP 14:

...

> 3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
>    may exist valid reasons in particular circumstances to ignore a
>    particular item, but the full implications must be understood and
>    carefully weighed before choosing a different course.

I'm sorry, but with careful reading, it seems to me that Keith's statement is an accurate paraphrase. Could you expand on what you see as the difference?

-- 
Bill McQuillan <[email-masked]>

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: spamops-04
Date: Sat, 2 Jul 2005 13:35:47 -0400
Cc: Keith Moore <[email-masked]>, [email-masked], Frank Ellermann <[email-masked]>
Sender: [email-masked]

On Sat July 2 2005 09:23, Keith Moore wrote:

> Part of the problem may be that RFC 2119 equates "RECOMMENDED" to 
> "SHOULD",

Yes.

> which essentially means "you MUST do this unless you have a  
> good reason to make an exception".

No. (sigh)  I wish people would stop trying to reinterpret BCP 14 (a.k.a.
RFC 2119).  The BCP 14 text is concise and clear.  Here in their entirety
are sections 1-6 of BCP 14:

1. MUST   This word, or the terms "REQUIRED" or "SHALL", mean that the
   definition is an absolute requirement of the specification.

2. MUST NOT   This phrase, or the phrase "SHALL NOT", mean that the
   definition is an absolute prohibition of the specification.

3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
   may exist valid reasons in particular circumstances to ignore a
   particular item, but the full implications must be understood and
   carefully weighed before choosing a different course.

4. SHOULD NOT   This phrase, or the phrase "NOT RECOMMENDED" mean that
   there may exist valid reasons in particular circumstances when the
   particular behavior is acceptable or even useful, but the full
   implications should be understood and the case carefully weighed
   before implementing any behavior described with this label.

5. MAY   This word, or the adjective "OPTIONAL", mean that an item is
   truly optional.  One vendor may choose to include the item because a
   particular marketplace requires it or because the vendor feels that
   it enhances the product while another vendor may omit the same item.
   An implementation which does not include a particular option MUST be
   prepared to interoperate with another implementation which does
   include the option, though perhaps with reduced functionality. In the
   same vein an implementation which does include a particular option
   MUST be prepared to interoperate with another implementation which
   does not include the option (except, of course, for the feature the
   option provides.)

6. Guidance in the use of these Imperatives

   Imperatives of the type defined in this memo must be used with care
   and sparingly.  In particular, they MUST only be used where it is
   actually required for interoperation or to limit behavior which has
   potential for causing harm (e.g., limiting retransmisssions)  For
   example, they must not be used to try to impose a particular method
   on implementors where the method is not required for
   interoperability.

To: Hector Santos <[email-masked]>
Cc: Arnt Gulbrandsen <[email-masked]>, [email-masked]
Subject: Re: Bounce/System Notification Address Verification 
From: [email-masked]
Date: Sat, 02 Jul 2005 12:46:48 -0400
Sender: [email-masked]

--==_Exmh_1120322808_13223P
Content-Type: text/plain; charset=us-ascii

On Thu, 30 Jun 2005 07:30:24 EDT, Hector Santos said:

> Here's another one.  This is the SendMail.com server:
> 
> 220 foon.sendmail.com ESMTP Sendmail Switch-3.1.7/Switch-3.1.7;
>         Thu, 30 Jun 2005  03:55:50 -0700
> helo smtp.sendmail.com
> 250 foon.sendmail.com Hello adsl-10-44-25.mia.bellsouth.net [65.10.44.25],
> pleas
> ed to meet you
> 
> See the problem?  [Hint: Spoofing]
> 
> Lets check Claus's esmtp.org server:
> 
> 220 zardoc.esmtp.org ESMTP sendmail X.0.0.Alpha4.0
> helo zardoc.esmtp.org
> 250 zardoc.esmtp.org Hi there
> mail from: <>
> 501 5.1.7 Bad sender's mailbox address syntax.
> 
> Wonderful!  Its worry about a space, but it doesn't protect its own local
> domains! <g>

What can I say? Claus is a standards-conforming kind of guy :)

In what sense is the handling of the HELO parameter failing to "protect"
the domain?

Note that (a) the HELO isn't usually *used* for anything, (b) will get you a
nice little notation in the Received: header, and (c) the relevant RFCs
specifically say that bouncing mail solely because the machine is confused
about its name is prohibited (2821, section 4.1.4)

   An SMTP server MAY verify that the domain name parameter in the EHLO
   command actually corresponds to the IP address of the client.
   However, the server MUST NOT refuse to accept a message for this
   reason if the verification fails: the information about verification
   failure is for logging and tracing only.

And sure enough, if you were to proceed with valid MAIL FROM/RCPT TO, you'd
find that the Received: header would contain *3* fields - your claimed HELO
name, the IP you *actually* came from, and the PTR that IP has.  In addition,
if the hostname claimed by the PTR doesn't have an A record matching the IP,
there will be a 'might be forged' notation).

And rest assured, if you had tried to claim the same domain address on the
MAIL FROM, Claus's servers would have told you exactly what you could do with it.

--==_Exmh_1120322808_13223P
Content-Type: application/pgp-signature

--==_Exmh_1120322808_13223P--

To: Hector Santos <[email-masked]>
Cc: "william(at)elan.net" <[email-masked]>, Matti Aarnio <[email-masked]>, Arnt Gulbrandsen <[email-masked]>, [email-masked]
Subject: Re: Bounce/System Notification Address Verification 
From: [email-masked]
Date: Sat, 02 Jul 2005 12:30:01 -0400
Sender: [email-masked]

--==_Exmh_1120321801_13223P
Content-Type: text/plain; charset=us-ascii

On Thu, 30 Jun 2005 08:51:48 EDT, Hector Santos said:

> 220 sokol.elan.net ESMTP Sendmail 8.13.1/8.13.1;
>         Thu, 30 Jun 2005 05:45:42 -0700
> helo [216.151.192.200]
> 250 sokol.elan.net Hello adsl-10-44-25.mia.bellsouth.net
>         [65.10.44.25], pleased to meet you
> 
> which is typically an illegal syntax due to being a spoof.

Umm.. The *syntax* doesn't become illegal just because it's spoofed. The
*semantics* may be a bit dicey, but [ ip.addr.notation ] is perfectly legal
syntax on the helo.

--==_Exmh_1120321801_13223P
Content-Type: application/pgp-signature

--==_Exmh_1120321801_13223P--

Date: Sat, 02 Jul 2005 09:23:33 -0400
From: Keith Moore <[email-masked]>
To: [email-masked]
Cc: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: spamops-04
Sender: [email-masked]

> It's more "best practice, but uncommon", in my experience at least. And that's
> the problem - it most certainly is not required and I see little if any chance
> of getting consensus on recommending it, given that we cannot seem to get
> consensus on lots of even simpler, more obvious stuff.

Part of the problem may be that RFC 2119 equates "RECOMMENDED" to 
"SHOULD", which essentially means "you MUST do this unless you have a 
good reason to make an exception".  So there is an unfortunate tendency 
for anything that is merely recommended to be stated in such a way as to 
carry the weight of a requirement.

For operational recommendations to gain consensus I think we'll need to 
find some language that conveys a greater range of {en,dis}couragement 
than just MUST/SHOULD/MAY/SHOULD NOT/MUST NOT.  maybe

MUST
SHOULD
STRONGLY ENCOURAGED
ENCOURAGED
MAY
DISCOURAGED
STRONGLY DISCOURAGED
SHOULD NOT
MUST NOT

?

I also think we'll need to find language that conveys a sense of 
timeframe.  maybe IMMEDIATELY/ASAP/WHEN PRACTICABLE/BY {date} ?

Date: Sat, 02 Jul 2005 09:12:22 -0400
From: Keith Moore <[email-masked]>
To: Frank Ellermann <[email-masked]>
Cc: [email-masked]
Subject: Re: spamops-04
Sender: [email-masked]

>>>It's IMO really "best common practice" to export this
>>>knowledge to the front end.
> 
> 
>>It's more "best practice, but uncommon", in my experience
>>at least.
> 
> 
> Okay, I can't judge how "common" it is, but in some cases
> it's clear why it's at least difficult.  OTOH look at LTRU,
> with two exceptions, one more on the troll side of things,
> all are happy with saying 3066bis => BCP.  The definition
> of "common" is apparently flexible enough to cover it.

The "C" in "BCP" means "current", not "common".

Keith

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: spamops-04
Date:  Sat, 02 Jul 2005 10:11:27 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 72
Sender: [email-masked]

[email-masked] wrote:

>> It's IMO really "best common practice" to export this
>> knowledge to the front end.

> It's more "best practice, but uncommon", in my experience
> at least.

Okay, I can't judge how "common" it is, but in some cases
it's clear why it's at least difficult.  OTOH look at LTRU,
with two exceptions, one more on the troll side of things,
all are happy with saying 3066bis => BCP.  The definition
of "common" is apparently flexible enough to cover it.

> that's the problem - it most certainly is not required
> and I see little if any chance of getting consensus on
> recommending it

As always I have some difficulties with SHOULD, with valid
and good reasons where's the problem ?  If you simply want
some backup MX service run by a third party, then it can't
verify addresses or temporary conditions like "over quota",
it's the idea of this backup MX to work especially if the
main site is down.

The backup MX probably uses some very aggressive blocking,
and of course it's not affected by a "SHOULD verify".  The
SHOULD would be for those who can reasonably do it.  And if
that's a problem in the next spamops I-D maybe listing some
of the cases where it's impossible could fix it.

>> What's a "lock step relay", like CBV only forward ?
> Bruce already described it accurately, I think.

Yes, it's what I thought.  The timing issues are interesting,
a timeout would be another valid reason to violate a SHOULD,
and to check the problem with the next hop(s) manually... ;-)

 [FAIL]
> Mine has been in place for something like two years. Hasn't
> done a lick of good that I can tell.

With rule #3 (spammers are stupid) some delay is clear, but
after SA 3.0 (2004-09 IIRC) that must be a very hard case...
or it's "only" a mail worm, worms authors have more pressing
needs than to avoid FAIL-protected addresses.

> But hey, at least I'll be able to keep sending mail to
> hotmail.com...

Depends on your PRA, not if you're on Sympa mailing lists :-(

 [elaborated "lock step relay" etc. I-D]
> There's lots of stuff to cover. One thing would be how to
> define best practices for address verification done by web
> sites - both simple syntax checks and active online checks.
[...]
> There's a fair degree of overlap between this sort of
> verification and what CBV does for MAIL FROM addresses, so
> might as well do that too.

> Such document would be a bit tricky in the IETF given that
> some of it gets close to GUI guidelines. But I think that
> aspect of it could be managed.

RfC 3696 is nice... but apparently not correct, I don't find
a way to get a <quoted-pair> outside of a <quoted-string>, and
an unquoted space in a <quoted-string> is also dubious.  Maybe
the planned I-D should update 3696 - is it possible to update
an informational RfC ?  Or to submit errata ?  Bye, Frank

From: [email-masked]
Cc: [email-masked]
To: Frank Ellermann <[email-masked]>
Date: Fri, 01 Jul 2005 16:39:32 -0700 (PDT)
Subject: Re: spamops-04 (was: Bounce/System Notification Address Verification)
Sender: [email-masked]

> [email-masked] wrote:

> > while it is arguably better to do address checks on the front
> > end system, it certainly is not required, and may not even be
> > possible in some setups. And when it is possible the security
> > implications are interesting, to say the least.
 
> > This is why I pushed back on Pekka Savola's suggestion that
> > draft-hutzler-spamops-04.txt recommend/require this practice.

> Somehow I must have missed or forgot this part, was it your
> idea to say "MDAs" in the corresponding "SHALL NOT accept" ?

> It's IMO really "best common practice" to export this knowledge
> to the front end.

It's more "best practice, but uncommon", in my experience at least. And that's
the problem - it most certainly is not required and I see little if any chance
of getting consensus on recommending it, given that we cannot seem to get
consensus on lots of even simpler, more obvious stuff.

> If you'd start to suppress bounces based on
> some spam filtering it could be more messy than the _security_
> implications of an exported local user list.

That depends very much on how you do the export. A simple list in a file is one
thing, opening a channel through to your directory server is a very different
kettle of fish. Full security audits of directory servers are really fun...

> >> the MX could start a lock-step relay during RCPT TO rather
> >> than store-and-forward, but I don't know of any such split
> >> setups that work that way.
 
> > We use this sort of thing as part of large scale deployments
> > that support POP-before-SMTP

> What's a "lock step relay", like CBV only forward ?

Bruce already described it accurately, I think.

> That would
> be one way to export the knowledge about local users.  It could
> also catch some "over quota" cases depending on the MDA.  With
> more than one hop from MX to MDA it starts to get interesting.

Indeed, which is why we tend to limit use of the technique to fairly specific
cases. Using it for a backup MX situation can get messy, given that one of the
main purposes for having backup MXes is so mail keeps flowing when the primary
site (where the definitive data lives) is unreachable.

Of course you can always use the standard directory replication protocol the
LDUP group defined to create a remote copy of the data you need and keep it in
sync. Oops, the LDUP group died without ever specifying a protocol, so the ways
these is done are totally proprietary.

> > regardless of how desireable it may be to reject invalid
> > recipient addresses early, it most certainly is not required

> You'd need two MTAs or rather two IPs as mailouts, one would be
> permanently blaklisted as the soure of bogus bounces to forged
> addresses.  My attempts to discuss the issue in the spamcop NG
> had no effect, and I'd stop when you had a fair chance:  If you
> accept FAIL-protected forged mails to non-existing local users
> it would be really your problem to discuss this with BLs.
  
> > I average around a hundred of these every day.

> How long do you have your -all ?  "My" spammer needed about 14
> weeks until he got the idea.
 
Mine has been in place for something like two years. Hasn't done a lick of good
that I can tell. But hey, at least I'll be able to keep sending mail to
hotmail.com...

> > Keith mentioned earlier in this thread that an address
> > verification specification would be a good idea.  I agree,
> > and would be happy to assist in writing such a document.

> What could it say, something about catch-all and "teergrube" ?
> Or is this about some "lock step relay" / forward verification
> within an MRN (MX to MDA) ?

There's lots of stuff to cover. One thing would be how to define best practices
for address verification done by web sites - both simple syntax checks and
active online checks. (I'm personally sick and tired of being told that
[email-masked] is an invalid email address.) There's a fair degree of
overlap between this sort of verification and what CBV does for MAIL FROM
addresses, so might as well do that too. 

Such document would be a bit tricky in the IETF given that some of it
gets close to GUI guidelines. But I think that aspect of it could be
managed.

				Ned

To: Hector Santos <[email-masked]>, <[email-masked]>, <[email-masked]>
From: [email-masked]
Subject: Re: Re: Problems with CBV an discussion document
Date: Fri, 1 Jul 2005 20:39:35 +0100
Sender: [email-masked]

Hector thanks for your comments.
I hope you appreciate mine as well.

>> Problems with CBV
>>
>> Two days ago I wrote that I liked the idea of CBV,
>> but re-thinking it i am not so fond of it anymore.
>> This is a public open discussion document and if
>> you want to disagree with me please do so.

>Ok. :-)

>> Definitions used in this document:

>> SenderA  -> Mail from:<[email-masked]>
>>
>> MTA1 -> <MTA that receives emails send to  senderA.
>> the MX records of domain1.tld points to this MTA.

> In other words, the Return Path Host?
Yes,  in a later version  I will rename it to RP-MTA

>> MTA2  MTA that sends the email to be checked to MTA3
>> MTA3  MTA that does the Call back testing

>You lost me with this.

There is no reason that MTA1 and MTA2 are the same host
See Keith Moore's MON (Mail Originator Network) and MRN (Mail Reception Network).
the MX points to the MRN. the MON can be a MSA
MTA1 is part of the MRN,  MTA2 is part of the MON
I myself sends email by SMTP relay to my ISP and receive bounces back via my account at my webhoster. This is all good old proper SMTP.

And SMTP still allows multi hop transactions... so MTA2 can be between MTA-RP and MTA3. The options with SMTP are almost endless.

Remark
If you do not want to allow multi hop SMTP anymore you are really ruining SMTP.

Maybe I should have cut MTA3 in two
the MTA that does the call back doesn't have to be
the same MTA that receives the email,
But I thought that was complicating the document unnecessary.

>>
>> First remark
>>
>> MTA1 doesn't have to be the same MTA as MTA2.
>> As matter of In fact it mostly isn't.

> Think about what you are saying. Forget CBV.
> Wipe it out of your mind.

> We have typical incoming transaction, per RFC x821 definition:

>    MAIL FROM:<reserve-path>
>    RCPT TO:<forward-path>

> If what you say is true, then the reserve-path is completely unreliable for
> returning bounce notifications per RFC x821 specifications.

No it can be reliable, I receive bounce notifications all right, I receive them, they  come to me but do not go to MTA2
they go to MTA1 the RP-MTA

> But if you fundamentally believe what the RFC says for the return-path, its
> purpose and it must be a reliable address for bounce purposes, then it
> doesn't matter what backend arrangment exist, a MTA performing a BOUNCE
> process is going to go the some mail reception host defined by the email
> domain, and this MTA is going to expect that this address is valid.

> If its not valid, then some orignal sender was a PROBLEM.  It was not
> compliant with RFC x2821 expected behavior.

As long as i receive the bounce notification so the address is valid and mine.
What is wrong with it?

>> Second remark
>
>> SFP  classic tries to answer if MTA2 can be used to send emails from domain1.tld
>

>First, SPF is a domain authorization concept, not an complete email
>authorization concept.  So even if you have these two addresses:
>
>         bad.user @ microsoft.com
>         bill.gates @ microsoft.com
>
>both will be accepted by SPF, but only address is truly valid.

Only one??? ;-) (joke)

>Second, we use SPF before CBV, so even then, we put more trust in the SPF
>result at the time being.  We don't want to call the CBV if we don't have
>too.  However,
>
>Third, if the SPF domain has a relaxed policy (Neutral or Soffware) then
>this is consider a no-decision and the CBV is called.    The SPF relaxed
>provision is one of the loop holes of SPF and I said so since day one.    We
>catch a bunch of spammers piggy backing on relaxed SPF domain policies, like
>AOL.COM, MICROSOFT.COM, HOTMAIL.COM, JUNO.COM, MSN.COM etc, who the spammers
>are still using for spoofing.  The CBV works excellent when its augmented
>with the SPF relaxed provisions.

>Have you implemented proposed technologies? Like SPF?  Let me check
>amidatrust.com.... No. No SPF record.   You should add one :-)

I do not think that SPF works ...
And my situation is a bit to complex for SPF (I guess, i have not looked in it deeply and i do not test on it either)

>> The first problem is: what questions does CBV really answer?

>       Is the Return Path Valid.

Valid only in the sense that you can send email to this address, it does not imply that this address has any relation to the sender of the email.

>> CBV only answers The questions:
> Does MTA1 accepts email for SenderA?

>It answers the same question whether you were sending mail to any users.

>If you don't believe in a CBV concept, then you are basically suggesting
>the RETURN PATH is not a valid and reliable entity in the current email
>infrastructure based on RFC x821 technology.

I think we can agree to disagree on that.
You do not want me to believe that return path's can not be spoofed ???

>>That is all the CBV answers.

> Now, what are the potential implementation issues?

> Well, the same implementation answer that a regular MTA will have to send
> mail to a final destination host, plus 2:

>         - Time Out issues
>         - Call back Overhead.

> You can reduce the CBV overrhead to when its absolutely necessary usin other
>methods.

The overhead is the BIG problem, especially if CBV solves almost nothing

> But for timeouts, the question is this enough to throw the baby out with the
> bath water.

I think the time-out issue is solvable.
Like the reverse CBV issue, these are no real problems at all.

> I say no.  It has not proven to be an issue and for many good reasons in
> todays market place especially one - scalability.

>> And with the latest add-on  (try to send an email to a fake emailaddress)
>> Is MTA1 some kind of open relay?

> Our CBV implementation test for an open relay.  I think it is good idea.

In principle all right, but it tests the wrong MTA you should test MTA2 not the RP-MTA, (remember the email is coming from MTA2, not from the RP-MTA)
Wonder some time why Rbl testing uses the IP address of MTA2 and not the return path domain that links to the RP-MTA?

>> What CBV doesn't awnser
>> - Did this email come from SenderA?
>> - Is SenderA a spammer?
>> - Is MTA2 an open relay? (except if MTA2 ==MTA1)
>> - And other questions (please add)

>No need.  The only question is:
>
>       Do you believe that he Return Path is a valid address to test?
>
>If you don't think so, then the game is over :-)  Nothing I can say any
>futher but just point out, the bounce system is completely broken thing if
>you don't believe in the return path.
>
>If you don't believe it is valid because it is often SPOOFED or BAD, then
>what is what the CBV is out to test.

The problem is it can be valid (in the sense that you can send emails to it) and Spoofed (in the sense that it has nothing to do with the sender of the email) at the same time

>> This has the consequence that CBV is easely fooled.

> Nothing.  The CBV is faked out.  You don't throw your hands, accept the
> message and move on.  No harm. You can cached the results if you like.  But
> if you need to do that, you might as well white list the sender.

That's not the question, the question is
Is the cure worse than the decease, (if done on a really big scale)

>> It is as  easy as to forge the "mail from" to a real
>> emailaddress: any "[email-masked]" or
>> "[email-masked]" will do.

> Again, if the Return Path is a presume to be valid, then the remote host
> need to accept it or reject it.

> However, in our CBV implementation, for backward compatibility reasons, we
> skip the CBV for the postmaster address.  Why? Because per RFC x821,  the
> MDA must accept a POSTMASTER fowarding path, so it would be a waste of time
> to perform a CBV on the postmaster address.

What do you mean by backward compatibility?
Are you not in a real  nice way saying: sometimes CBV won't work , especially with clever spammers that know that Postmaster addresses always should be accepted by MRN.s
So please spammer do NOT use [email-masked] as return address.

>> This will also results in many false negatives.
>> (emails that are spam but not recognised as such)

> How so?

You will not recognise it as spam, if you cannot do the test, that is a false negative if it is spam.
Simple as that. Or do you have an other definition of a false negative?

> As I said,  RFC x821 clearly starts you must accept:
>
>            RCPT TO: <postmaster>
>
> so it will be a waste of time to call CBV.

> It could be SPAM, but thats not the point of the CBV.
>
> hmmmm, I guess, we can still call it to do the open relay test!  I am going
> to try that!

But please test it at the right MTA (MTA2)

>> Crashing SMTP
>>
>> Anti spam rules are build on the implicit assumption that the whole
>> email can be forged  "do not rely on anything that somebody else
>> has tested".  This implies that you can not trust headers like

> The CBV is a SMTP process and 2822 analysis is out of scope.  It has nothign
>to do with the headers. Just the return path.

This was just an introduction to this point, my point is that other MTA's cannot rely on information that other MTA's put in header lines or other parts of the email.

>> In IP load
>>
>> a typical CBV test will need about 30 IP datagrams. (Ps this is
>> only an informed guess, i will accept better guesses)

> Well, I have to confirm your number, but it doesn't matter. TCP is an
> overhead and doing an CBV in an open-ended basis is a bad idea.

Sorry I meant IP packets instead of IP Datagrams and really

.......>>>> THIS IS WHAT MATTERS <<<<<.......

This is what causes the BIG PROBLEM
This is why the cure that is worse than the disease
And more of this kind of sayings.

> You can not do a CBV without implementing other ideas to reduce the need to
>the absolute minimum.

Yes about all other possible spam tests DNS - RDNS almost "everything" even including bayes filtering  should be done before this... and even then

>> So CBV will work on a small scale  but as soon as its is
>> used in a grand scale it will resullt in a denail of service.

> I don't agree with the DOS, but I agree that an open end wide adoption
> across the board  will have new bandwidth issues.  How much.  One can make a
> good guess.  But as I said, you need to do intelligently to validate the
> return path.

> Keep in mind that does far, SPF and SENDERID/PRA are moving along and even
> though  believe there are new bandwidth requirements,  the concerns seem to
> be put to the side.

And the internet load of those UDP - DNS based methods are very small compared with CBV...

>> Where does this leads us:
>> CVB is not able to answer the importand question:
>> Is this email SPAM?

> It was never MEANT to answer that question.

>> If CBV is used by many it will result in a DoS attack on MTA1

> I don't agree with this concern.

>> CBV is easely fooled.

> With a trackable zombie cite which is alittle better than having nothing at
> all.

Not if it means that somebody else cannot send email anymore

> The fact is over 60-80% of all transactions are bad.  And it usually begins
> with a bad or spoofed. RETURN-PATH.

That maybe is only now (2005) with CBV it will become proper Spam , with a real valid return-path. Is that a real gain???
Not for the people that are the unlucky users from that return path.

>> So for a while it will stop some spam, but in the long run it will
>> not work and it will ruin the email structure.

> It stops quite of bit of the bad addresss. Where it was good or bad mail.
> Who cares?

> No one have ever said that it should be as an ultimate solution.

> I see it as a "Return Path Validator" and for now, a black box concept.  The
> CBV is the current black box, and I am waiting for you to invent the better
> black box so I can plug and play the Return Path Validator. :-)

CSV, BATV???

>
> Thanks for you comment, I have you understand the basic question. Either you
> believe in the Return Path as a verifable entity or not.
>
The first thing then is to believe it can be spoofed.

>
> Whatever implementation you use will be a overhead issue, regardless what
> you might have.   So you engineer the solution to minimize the overhead.
>

But shooting with canons to kill ants isn't the way...

> Is this optimization enough for wide adoption?  I don't know. I won't say
> yes. But I won't take a blatant "no" from people who have not truly explored
> it completely by augmenting other ideas as well.   I will say, they ought
> the be a better way.   Go find it. :-)

> By the way,  you can use this test web page to try the CBV and SPF.

>         http://www.winserver.com/testwcsap

> Go ahead and try way with different combinations of parameters.

> I will be interested to see if you find some false positives.

I played with it for a some time and did not get any positives out of the CBV test.
Only when I used fake domains but that is not really an CBV test. that just pointed out that an MX record test didn't exsist
So let alone false positives

Interesting were the timings, the CBV takes about 5 times as long as all other tests together.
Can you make it count IP packets??

Do you see what I mean with that CBV has a larger load on the internet then all the other tests together?

And I missed DNS A record, RDNS ptr and CSV testing.
and some other RBL tests.
They also help against spam and they really take less bandwidth than CBV.

> TIP: Select High Verbosity Logging to see everything.
I did that why i missed it...

Even example.net as domain and example.com as EHLO worked
Even from private IP's it worked
It needs some more testing.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: spamops-04 (was: Bounce/System Notification Address Verification)
Date: Fri, 1 Jul 2005 12:28:36 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: "Bruce Lilly" <[email-masked]>

Subject: Re: spamops-04 (was: Bounce/System Notification Address
Verification)

> > What's a "lock step relay"[...]?

> We hope the sender doesn't time out while waiting for the lock-step
> relay initiation.  SMTP minimum timeout values (2821):

First, I think it should be made clear these are recommended values and the
system has the ultimate decision base on best performance, efficiency and
scalability decisions to hand load requirements.    A server can wait 5
minutes. Well go to Yahoo.Com and see how long it will wait for you to type
in a command.   So these are recommended values only.

>   greeting: 300 seconds (5 minutes)
>   HELO:  unspecified!!!  (JCK please note)

I would think the last sentence in section 4.5.3.2 Timeouts suggest what the
default values area in both directions. But it should be clarified in
2821bis.

>   MAIL TO:  300 seconds (5 minutes)
>   RCPT TO:  300 seconds (5 minutes)
>
> While the sender is counting down its 300 seconds, the relay might
> experience 299+?+299+299 seconds of delay.  Guess what -- that could
> be more than 300 seconds.

This is why you have 35 second per command.  Exim uses 30 seconds.

        CONNECT 30-35
        HELO  30-35
        MAIL FROM: 30-35
        RCPT TO: 30-35

120-140 wait time for the remote.

You can not use the RFC x821 defined values for a sink.  And if you insist,
you need to use a 1xy continuation response codes to keep the waiting sender
alive.  This may be required for OPES.  The CBV is a example of OPES smtp
implementation.

4.2.1 Reply Code Severities and Theory
    ....

   1yz   Positive Preliminary reply
      The command has been accepted, but the requested action is being
      held in abeyance, pending confirmation of the information in this
      reply.  The SMTP client should send another command specifying
      whether to continue or abort the action.  Note: unextended SMTP
      does not have any commands that allow this type of reply, and so
      does not have continue or abort commands.

Read this link in the OPES working group where I show proof of concept.

http://www.imc.org/ietf-openproxy/mail-archive/msg03282.html

> ...
> In theory, there could be multiple levels of lock-step, but the
> timeout issue becomes progressively more critical.

Bruce, I think you theory is 100% correct, but it isn't the real world, not
exhibited in practice simply because it defies logic for proper operations.
See RFC 2821 4.5.4.1 Sending Strategy

Lets take the worst case situation, a sender issues

Sender:
      MAIL FROM:<return-path>
      RCPT TO:<forward-path>

The server now begins the CBV back to the return-path host.  The sender will
wait 5 minutes, so you assume. But this is the recommendation.  Do you
really want to wait 5 minutes?

Divide 5 minutes by 4 (for each state) and we have 75 seconds wait time per
CBV state.

This is the worst case and also the "theoretical" maximum value for each CBV
state:

        CONNECT 75
        HELO  75
        MAIL FROM: 75
        RCPT TO: 75

Total 300 seconds (5 minutes)

But that is cutting too close, the better value is between:

      Default Socket Timeout  <= Timeout <  75 seconds

I chose 35 seconds.  Exim chose 30 seconds, which for me, is low for Windows
implementations.  You can't  go higher than 75 seconds because this will
break your expectation of the compliant sender timeout value.

Now whether the timeout was 20, 35, 75 or 300 seconds (5 minutes) to comply
with SMTP values, you have:

     20 x 4 = 80 second
     35 x 4 = 140
    75 x 4 = 300
    300 x 4  = 1200 seconds

The question you have to ask regardless of the amount is how this play in
the real world implementation where performance is major consideration.

The specs says the sender should wait atleast 5 minutes. You are assuming
that all senders will wait 5 minutes.   This a false assumption.   Some will
not.  See RFC 2821 4.5.4.1 Sending Strategy.  So if one is going to explore
the CBV, they need to keep the TIMEOUT per command as low as possible.

If one says anything less 50% like Keith suggest, exhibit slows downs, then
CBV is no good. Guess what?  If it anything close to 5%, then to me, CBV is
no good. Get the hammer. Squash it. Don't even think about it any more.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Fri, 1 Jul 2005 11:19:53 -0400
From: Keith Moore <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: request discussion of two documents on SMTP relaying
Sender: [email-masked]

> > not sure what you mean by that.  with TLS the server is authenticated 
> > via the server certificate.  a hijacker doesn't have access to the 
> > session key and can't set its own session key without causing the server 
> > signature verification to fail.
> 
> TLS seems to work fine through proxies; what prevents the MitM
> from setting up a proxy?  (If OPES bears fruit, hijackers may
> feast)

it depends on what you mean by proxy.  if the client has an explicit
proxy and is talking to that proxy in cleartext, it can ask the proxy
to start a TLS session, but it won't be encrypted end-to-end.  however
in that case the client has explicitly specified a proxy to use, and an
attacker would have to intercept traffic to that proxy (which is
doable).

otoh, if someone has set up an interception proxy by whatever means,
the traffic between client and server is encrypted.  the proxy can
relay the bits without changing them but it can't tamper with the
traffic without causing an error. 

> Also bear in mind that there are different levels of TLS
> authentication: server, client, and both.  HTTP typically
> authenticates the server to the client; SMTP AUTH typically
> authenticates the client to the server.  In the former (HTTP)
> case, if the server wishes to authenticate the user, some
> additional method (e.g. login/password) is used.  For SMTP
> AUTH, well the client just can't tell for sure if it's talking
> to the real McCoy.

TLS always authenticates the server to the client; client-to-server
authentication using TLS (and client certs) is optional.  TLS client-to-
server authentication is sufficient to thwart man in middle attacks.

Whether SMTP AUTH is vulnerable to a MitM attack depends on
the particular auth method used - Kerberos and some of the
GSSAPI methods should be okay, LOGIN, PLAIN, and challenge
response methods are not.  OTOH if you do SMTP AUTH over
TLS you're safer.

> > of course, there are clients that don't check server certificates,
> 
> There's a problem with certificates (also other schemes); they rely
> on some sort of trust relationship.  For certificates in particular,
> there's a self-signed certificate at the end of the chain (or the
> moral equivalent).

right, and that party has to defend its certificates by repuation
and by publishing them widely.   

> Why should I trust a certificate from some company in South Africa
> that I've never dealt with?  Worse, what about this other certificate --
> isn't that the company that screwed up DNS for everybody with bogus
> wildcards -- "I trust [them] as far as I can comfortably spit out
> a rat"?

yup.

> In practice -- because mere mortals don't know enough about every
> individual and organization that might present a certificate or
> other token -- it tends to become just another "click on yes"
> annoyance.  Maybe once in awhile (say, isn't that the company
> that screwed up DNS...) it's "click on no", but then things don't
> work so it's quickly back to "click on yes".
> 
> This is a particularly problematic issue for mail as distinct
> from http.  The fundamental premise of email is that anybody can
> send to anybody else -- how is a user supposed to keep track of
> chains of certificates for billions of potential correspondents
> and figure out whether or not to accept one from a stranger (who
> might be a potential customer/employer/benefactor, or might be a
> spammer)?
> 
> How would this work for the typical Internet user -- whose mail
> is received by an ISP into a POP mailbox, which is then downloaded
> by the user -- when mail is received by the ISP, the user may be
> off-line (and unable to respond to certificates); POP has no
> per-message certificate exchange mechanism?

all valid issues, but I don't have time to discuss them right now.
(any chance you'll be at IETF Paris?)

> > there are multiple ways to solve the problem.  one is that if a user 
> > provides a certificate to the MSA that says "I have permission to use 
> > this set of addresses in SMTP MAIL FROM" then the MSA permits such use.
> 
> Do self-signed certificates count?  What about ones signed by
> entities know to have caused damage to the email infrastructure?
> Who decides?  On what basis?  How would this work with your
> scenario of having to use a strange MSA?  What if no MSA is
> available (some ISPs support only an MTA for sending; in other
> cases it may be necessary to connect directly to a recipient's
> MX MTA to work around DNS or other network problems)?  Who
> decides when an SMTP receiver is an MSA or MTA?  How can the
> client tell the difference?

good questions, all things that would need to be addressed in a 
formal proposal before it could be approved.  I bet you can come 
up with reasonable answers for most of them yourself.
  
> > another way (which only works for individual addresses) is that you get 
> > the MON to send a single message to a MAIL FROM address, the message 
> > contains a cookie, you reply to the message, now the MON knows you can 
> > read mail sent to that address and so are presumably able to use it.
> 
> A widely-respected (if anything, not respected widely enough) rule
> is "never respond to spam".  How is a typical user (ESR's Aunt Tillie,
> for example) supposed to be able to distinguish such an unsolicited
> message from that sent by a spammer who is simply trying to confirm
> mailboxes in preparation for a new "Two Hundred Million Verified Email
> Addresses" CD-ROM?

because in the case of establishing yourself to an MSA that you have a 
right to use a MAIL FROM address, you initiated the action that caused
the verification message to be sent, and you're expecting that message
to arrive at your inbox.

of course, the MSA has to guard itself against this technique being used
as a way to send yet more annoying mail.  e.g. by rate limiting,
by insisting that the initial request come from a web site that is
resistant to manipulation by robots.
  
> > another is that the MSA makes the mail traceable in some way other than 
> > MAIL FROM to a user who can be held accountable for his actions - then 
> > if the user sends out mail that abuses MAIL FROM then he gets billed for it.
> 
> Billed by whom? 

e.g. the MSA requires a credit card # (with all of the usual verification for
credit card purchases), and if the MSA gets abused then it bills that credit 
card for services related to cleaning up the damage (perhaps the fees
are shared with other parties who were harmed).

the devil is in the details, of course, but we do things like this routinely
in other areas.  credit card fraud exists, but that doesn't stop us from
using credit cards. 

Keith

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Subject: Re: Problems with CBV an discussion document
Date: Fri, 1 Jul 2005 10:46:43 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: <[email-masked]>

> What CBV doesn't awnser
> - Did this email come from SenderA?
> - Is SenderA a spammer?
> - Is MTA2 an open relay? (except if MTA2 ==MTA1)
> - And other questions (please add)

By the way,  you can use this test web page to try the CBV and SPF.

         http://www.winserver.com/testwcsap

Go ahead and try way with different combinations of parameters.

I will be interested to see if you find some false positives.

TIP: Select High Verbosity Logging to see everything.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Subject: Re: Problems with CBV an discussion document
Date: Fri, 1 Jul 2005 10:36:28 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

----- Original Message -----
From: <[email-masked]>
To: <[email-masked]>
Sent: Friday, July 01, 2005 8:39 AM
Subject: Problems with CBV an discussion document

> Problems with CBV
>
> Two days ago I wrote that I liked the idea of CBV, but re-thinking it i am
not
> so fond of it anymore. This is a public open discussion document and if
> you want to disagree with me please do so.

Ok. :-)

> Definitions used in this document:

> SenderA  -> Mail from:<[email-masked]>
>
> MTA1 -> <MTA that receives emails send to  senderA.
> the MX records of domain1.tld points to this MTA.

In other words, the Return Path Host?

> MTA2  MTA that sends the email to be checked to MTA3
> MTA3  MTA that does the Call back testing

You lost me with this.

>
> First remark
>
> MTA1 doesn't have to be the same MTA as MTA2.
> As matter of In fact it mostly isn't.

Think about what you are saying. Forget CBV.  Wipe it out of your mind.

We have typical incoming transaction, per RFC x821 definition:

    MAIL FROM:<reserve-path>
    RCPT TO:<forward-path>

If what you say is true, then the reserve-path is completely unreliable for
returning bounce notifications per RFC x821 specifications.

But if you fundamentally believe what the RFC says for the return-path, its
purpose and it must be a reliable address for bounce purposes, then it
doesn't matter what backend arrangment exist, a MTA performing a BOUNCE
process is going to go the some mail reception host defined by the email
domain, and this MTA is going to expect that this address is valid.

If its not valid, then some orignal sender was a PROBLEM.  It was not
compliant with RFC x2821 expected behavior.

> Second remark
>
> SFP  classic tries to answer if MTA2 can be used to send emails from
domain1.tld
>

First, SPF is a domain authorization concept, not an complete email
authorization concept.  So even if you have these two addresses:

         bad.user @ microsoft.com
         bill.gates @ microsoft.com

both will be accepted by SPF, but only address is truly valid.

Second, we use SPF before CBV, so even then, we put more trust in the SPF
result at the time being.  We don't want to call the CBV if we don't have
too.  However,

Third, if the SPF domain has a relaxed policy (Neutral or Soffware) then
this is consider a no-decision and the CBV is called.    The SPF relaxed
provision is one of the loop holes of SPF and I said so since day one.    We
catch a bunch of spammers piggy backing on relaxed SPF domain policies, like
AOL.COM, MICROSOFT.COM, HOTMAIL.COM, JUNO.COM, MSN.COM etc, who the spammers
are still using for spoofing.  The CBV works excellent when its augmented
with the SPF relaxed provisions.

Have you implemented proposed technologies? Like SPF?  Let me check
amidatrust.com.... No. No SPF record.   You should add one :-)

> The first problem is: what questions does CBV really answer?

       Is the Return Path Valid.

> CBV only answers The questions:
> Does MTA1 accepts email for SenderA?

It answers the same question whether you were sending mail to any users.

If you don't believe in a CBV concept, then you are basically suggesting
the RETURN PATH is not a valid and reliable entity in the current email
infrastructure based on RFC x821 technology.

That is all the CBV answers.

Now, what are the potential implementation issues?

Well, the same implementation answer that a regular MTA will have to send
mail to a final destination host, plus 2:

         - Time Out issues
         - Call back Overhead.

You can reduce the CBV overrhead to when its absolutely necessary usin other
methods.

But for timeouts, the question is this enough to throw the baby out with the
bath water.

I say no.  It has not proven to be an issue and for many good reasons in
todays market place especially one - scalability.

> And with the latest add-on  (try to send an email to a fake emailaddress)
> Is MTA1 some kind of open relay?

Our CBV implementation test for an open relay.  I think it is good idea.

> What CBV doesn't awnser
> - Did this email come from SenderA?
> - Is SenderA a spammer?
> - Is MTA2 an open relay? (except if MTA2 ==MTA1)
> - And other questions (please add)

No need.  The only question is:

       Do you believe that he Return Path is a valid address to test?

If you don't think so, then the game is over :-)  Nothing I can say any
futher but just point out, the bounce system is completely broken thing if
you don't believe in the return path.

If you don't believe it is valid because it is often SPOOFED or BAD, then
what is what the CBV is out to test.

> This has the consequence that CBV is easely fooled.

Nothing.  The CBV is faked out.  You don't throw your hands, accept the
message and move on.  No harm. You can cached the results if you like.  But
if you need to do that, you might as well white list the sender.

> It is as  easy as to forge the "mail from" to a real
> emailaddress: any "[email-masked]" or
> "[email-masked]" will do.

Again, if the Return Path is a presume to be valid, then the remote host
need to accept it or reject it.

However, in our CBV implementation, for backward compatibility reasons, we
skip the CBV for the postmaster address.  Why? Because per RFC x821,  the
MDA must accept a POSTMASTER fowarding path, so it would be a waste of time
to perform a CBV on the postmaster address.

> This will also results in many false negatives.
> (emails that are spam but not recognised as such)

How so?

As I said,  RFC x821 clearly starts you must accept:

            RCPT TO: <postmaster>

so it will be a waste of time to call CBV.

It could be SPAM, but thats not the point of the CBV.

hmmmm, I guess, we can still call it to do the open relay test!  I am going
to try that!

> Crashing SMTP
>
> Anti spam rules are build on the implicit assumption that the whole
> email can be forged  "do not rely on anything that somebody else
> has tested".  This implies that you can not trust headers like

The CBV is a SMTP process and 2822 analysis is out of scope.  It has nothign
to do with the headers. Just the return path.

> In IP load
>
> a typical CBV test will need about 30 IP datagrams. (Ps this is
> only an informed guess, i will accept better guesses)

Well, I have to confirm your number, but it doesn't matter. TCP is an
overhead and doing an CBV in an open-ended basis is a bad idea.

You can not do a CBV without implementing other ideas to reduce the need to
the absolute minimum.

> So CBV will work on a small scale  but as soon as its is
> used in a grand scale it will resullt in a denail of service.

I don't agree with the DOS, but I agree that an open end wide adoption
across the board  will have new bandwidth issues.  How much.  One can make a
good guess.  But as I said, you need to do intelligently to validate the
return path.

Keep in mind that does far, SPF and SENDERID/PRA are moving along and even
though  believe there are new bandwidth requirements,  the concerns seem to
be put to the side.

> Where does this leads us:
> CVB is not able to answer the importand question:
> Is this email SPAM?

It was never MEANT to answer that question.

> If CBV is used by many it will result in a DoS attack on MTA1

I don't agree with this concern.

> CBV is easely fooled.

With a trackable zombie cite which is alittle better than having nothing at
all.

The fact is over 60-80% of all transactions are bad.  And it usually begins
with a bad or spoofed. RETURN-PATH.

> So for a while it will stop some spam, but in the long run it will
> not work and it will ruin the email structure.

It stops quite of bit of the bad addresss. Where it was good or bad mail.
Who cares?

No one have ever said that it should be as an ultimate solution.

I see it as a "Return Path Validator" and for now, a black box concept.  The
CBV is the current black box, and I am waiting for you to invent the better
black box so I can plug and play the Return Path Validator. :-)

Thanks for you comment, I have you understand the basic question. Either you
believe in the Return Path as a verifable entity or not.

Whatever implementation you use will be a overhead issue, regardless what
you might have.   So you engineer the solution to minimize the overhead.

Is this optimization enough for wide adoption?  I don't know. I won't say
yes. But I won't take a blatant "no" from people who have not truly explored
it completely by augmenting other ideas as well.   I will say, they ought
the be a better way.   Go find it. :-)

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: spamops-04 (was: Bounce/System Notification Address Verification)
Date: Fri, 1 Jul 2005 10:36:12 -0400
Cc: Frank Ellermann <[email-masked]>, John C Klensin <[email-masked]>
Sender: [email-masked]

On Fri July 1 2005 03:13, Frank Ellermann wrote:

> >> the MX could start a lock-step relay during RCPT TO rather
> >> than store-and-forward, [...]
 
> What's a "lock step relay"[...]?

Normal relay:
Sender        Relay               Destination
          <-  220
HELO      ->
          <-  250
MAIL FROM ->
          <-  250
RCPT TO   ->
          <-  250
DATA      ->
          <-  354
.CRLF     ->
          <-  250
QUIT      ->
          <-  221
                              <-  220
                    HELO      ->
                              <-  250
                    MAIL FROM ->
                              <-  250
                    RCPT TO   ->
                              <-  5xx
                    QUIT      ->
                              <-  221
  Bounce  <-

Lock-step relay:
Sender        Relay               Destination
          <-  220
HELO      ->
          <-  250
MAIL FROM ->
          <-  250
RCPT TO   ->        Connect ->
                              <-  220
                    HELO      ->
                              <-  250
                    MAIL FROM ->
                              <-  250
                    RCPT TO   ->
                              <-  5xx
          <-  5xx
QUIT      ->        QUIT      ->
                              <-  221
          <-  221

We hope the sender doesn't time out while waiting for the lock-step
relay initiation.  SMTP minimum timeout values (2821):
  greeting: 300 seconds (5 minutes)
  HELO:  unspecified!!!  (JCK please note)
  MAIL TO:  300 seconds (5 minutes)
  RCPT TO:  300 seconds (5 minutes)
While the sender is counting down its 300 seconds, the relay might
experience 299+?+299+299 seconds of delay.  Guess what -- that could
be more than 300 seconds.  The relay and destination servers had better
be responsive (even under load) in such a setup, because the sender
timeout isn't under the control of the administrative entity running
the relay and destination.  "Strategies" involving intentionally
delayed server responses compound the problem.  If the sender times
out and goes away, the relay->destination session had better be
closed gracefully.

One hopes that all RCPT TO (in case of multiple recipients) resolve
to exactly the same destination host; otherwise this gets messy (data
copying as well as timeout issues) and it will still require a
bounce if one (or more, but not all) deliver{y,ies} fail after DATA
(can't return an error reply code; that would cause duplicate
delivery to successful recipients on retry).

In theory, there could be multiple levels of lock-step, but the
timeout issue becomes progressively more critical.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: request discussion of two documents on SMTP relaying
Date: Fri, 1 Jul 2005 09:54:01 -0400
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

On Fri July 1 2005 08:21, Keith Moore wrote:

> not sure what you mean by that.  with TLS the server is authenticated 
> via the server certificate.  a hijacker doesn't have access to the 
> session key and can't set its own session key without causing the server 
> signature verification to fail.

TLS seems to work fine through proxies; what prevents the MitM
from setting up a proxy?  (If OPES bears fruit, hijackers may
feast)

Also bear in mind that there are different levels of TLS
authentication: server, client, and both.  HTTP typically
authenticates the server to the client; SMTP AUTH typically
authenticates the client to the server.  In the former (HTTP)
case, if the server wishes to authenticate the user, some
additional method (e.g. login/password) is used.  For SMTP
AUTH, well the client just can't tell for sure if it's talking
to the real McCoy.
 
> of course, there are clients that don't check server certificates,

There's a problem with certificates (also other schemes); they rely
on some sort of trust relationship.  For certificates in particular,
there's a self-signed certificate at the end of the chain (or the
moral equivalent).

Why should I trust a certificate from some company in South Africa
that I've never dealt with?  Worse, what about this other certificate --
isn't that the company that screwed up DNS for everybody with bogus
wildcards -- "I trust [them] as far as I can comfortably spit out
a rat"?

In practice -- because mere mortals don't know enough about every
individual and organization that might present a certificate or
other token -- it tends to become just another "click on yes"
annoyance.  Maybe once in awhile (say, isn't that the company
that screwed up DNS...) it's "click on no", but then things don't
work so it's quickly back to "click on yes".

This is a particularly problematic issue for mail as distinct
from http.  The fundamental premise of email is that anybody can
send to anybody else -- how is a user supposed to keep track of
chains of certificates for billions of potential correspondents
and figure out whether or not to accept one from a stranger (who
might be a potential customer/employer/benefactor, or might be a
spammer)?

How would this work for the typical Internet user -- whose mail
is received by an ISP into a POP mailbox, which is then downloaded
by the user -- when mail is received by the ISP, the user may be
off-line (and unable to respond to certificates); POP has no
per-message certificate exchange mechanism?

> there are multiple ways to solve the problem.  one is that if a user 
> provides a certificate to the MSA that says "I have permission to use 
> this set of addresses in SMTP MAIL FROM" then the MSA permits such use.

Do self-signed certificates count?  What about ones signed by
entities know to have caused damage to the email infrastructure?
Who decides?  On what basis?  How would this work with your
scenario of having to use a strange MSA?  What if no MSA is
available (some ISPs support only an MTA for sending; in other
cases it may be necessary to connect directly to a recipient's
MX MTA to work around DNS or other network problems)?  Who
decides when an SMTP receiver is an MSA or MTA?  How can the
client tell the difference?
 
> another way (which only works for individual addresses) is that you get 
> the MON to send a single message to a MAIL FROM address, the message 
> contains a cookie, you reply to the message, now the MON knows you can 
> read mail sent to that address and so are presumably able to use it.

A widely-respected (if anything, not respected widely enough) rule
is "never respond to spam".  How is a typical user (ESR's Aunt Tillie,
for example) supposed to be able to distinguish such an unsolicited
message from that sent by a spammer who is simply trying to confirm
mailboxes in preparation for a new "Two Hundred Million Verified Email
Addresses" CD-ROM?
 
> another is that the MSA makes the mail traceable in some way other than 
> MAIL FROM to a user who can be held accountable for his actions - then 
> if the user sends out mail that abuses MAIL FROM then he gets billed for it.

Billed by whom?  Who collects the money?  Under which legal jurisdiction?
Who gets the money?  The joe-job victim?  Some random ISP?  Some rich
lawyer?  Some government bureaucrat?  What prevents forgery of that
mechanism (i.e. how do you prevent a joe-job victim from being erroneously
billed)?

To: [email-masked]
From: [email-masked]
Subject: Problems with CBV an discussion document
Date: Fri, 1 Jul 2005 13:39:22 +0100
Sender: [email-masked]

Problems with CBV

Two days ago I wrote that I liked the idea of CBV, but re-thinking it i am not so fond of it anymore. This is a public open discussion document and if you want to disagree with me please do so.

Definitions used in this document:

SenderA  -> Mail from:<[email-masked]>

MTA1 -> <MTA that receives emails send to  senderA.
the MX records of domain1.tld points to this MTA.

MTA2  MTA that sends the email to be checked to MTA3

MTA3  MTA that does the Call back testing

First remark
MTA1 doesn't have to be the same MTA as MTA2.
As matter of In fact it mostly isn't.

Second remark
SFP  classic tries to answer if MTA2 can be used to send emails from domain1.tld

The first problem is: what questions does CBV really answer?

CBV only answers The questions:
Does MTA1 accepts email for SenderA?
And with the latest add-on  (try to send an email to a fake emailaddress)
Is MTA1 some kind of open relay?

What CBV doesn't awnser
- Did this email come from SenderA?
- Is SenderA a spammer?
- Is MTA2 an open relay? (except if MTA2 ==MTA1)
- And other questions (please add)

This has the consequence that CBV is easely fooled.
It is as  easy as to forge the "mail from" to a real emailaddress: any "[email-masked]" or "[email-masked]" will do.

This will also results in many false negatives.
(emails that are spam but not recognised as such)

Crashing SMTP

Anti spam rules are build on the implicit assumption that the whole email can be forged  "do not rely on anything that somebody else has tested".
This implies that you can not trust headers like

"X-virustest: this email is absolutly free from virusses, i know i am one"
"X-Spamtest: This email is not spam i know i wrote it all by hand,(the mass-mailer i mean) "
This also means that a test has to be redone every time a e-mail crosses over a system boundary.
I do not define the concept of system-boundary but it does imply that a test has to be done a more than once.
Counting for virusscanning showed me that an virusfree-email was checked seven times before it unscratched reached it destination.
For the simplicity of the discussion lets assume that a CBV test is done three times during the transport from mail-client to final-MDA.

This means that for every proper email there will be three call backs to MTA1.

These calls come from different MTA's and this can resolve in a DOS attack on MTA1, because non of this can be cached.
Resolving this duplicate call scenario by adding a special header to the email will not work, the only way to be sure of an CBV check is to do the CBV check again.

In IP load
a typical CBV test will need about 30 IP datagrams. (Ps this is only an informed guess, i will accept better guesses)

So an for one email three tests will have an extra load of 90 IPdatagrams.
That doesn't look much but comparing it with other spamcounter measures it is an enormous ammount.
Even SFP is argued to cause Dos attacks and SFP only uses between 2 and 20 datagrams.
And for SFP the DNS records can be cached for CBV they cannot.

This is mainly becaurse other systems use the UDP protocol and DNS records, while CBV uses TCP protocol and SMTP sessions that both have a big overhead.

So CBV will work on a small scale  but as soon as its is used in a grand scale it will resullt in a denail of service.

Where does this leads us:
CVB is not able to answer the importand question:
Is this email SPAM?
If CBV is used by many it will result in a DoS attack on MTA1
CBV is easely fooled,

So for a while it will stop some spam, but in the long run it will not work and it will ruin the email structure.

Hoping on a fruitful discussion.

Date: Fri, 01 Jul 2005 08:21:58 -0400
From: Keith Moore <[email-masked]>
To: Frank Ellermann <[email-masked]>
Cc: IETF-SMTP <[email-masked]>
Subject: Re: request discussion of two documents on SMTP relaying
Sender: [email-masked]

>>yes, but there are legitimate reasons for not using the MSA
>>offered by the mail network where you receive your mail, and
>>there are also legitimate reasons for setting MAIL FROM to
>>something other than the address at which you normally
>>receive mail.
> 
> 
> If you use another MSA just use the MAIL FROM related to this
> MON, and if you want the error messages elsewhere forward them.

no, that's not acceptable, for lots of reasons.  one is that forwarding 
isn't always available, and when it is available it's not always 
reliable.  another is that you want to be able to use a MON that doesn't 
have any address associated with it, maybe because it works better from 
your particular location on the network (say you have a fast link to the 
MSA you are using but a slow connection to your normal MON's MSA). 
another, you sometimes need to use a different MAIL FROM address for 
each recipient, or to embed information into a MAIL FROM address so that 
bounces carry that information with them.

bottom line: insisting that everyone submit mail through the same mail 
network that corresponds to the MAIL FROM is too inflexible.

>>SMTP-after-POP is easy for an man-in-the-middle attack to
>>break.  the attacker intercepts the traffic between the MUA
>>and the POP server, using its own IP address to talk to the
>>POP server, then it can submit mail from the same IP address
>>at will.
> 
> 
> Yes, but with enforced submission rights it has to know how to
> derive a permitted MAIL FROM from the POP-user id, or it has
> to hijack both connections (POP and SMTP) from the client.

if you can hijack one, you can hijack both.  it's pretty easy to hijack 
a connection by setting up your own wireless access point in a public place.

>>SMTP-after-POP should be a SHOULD NOT or NOT RECOMMENDED just
>>like AUTH PLAIN and AUTH LOGIN and CRAM-MD5 and APOP, because
>>it's defeated just as easily.
> 
> 
> Actually I'm not that paranoid when I use one of my ISPs and
> the MSA of an unrelated mail provider.  A roaming user in an
> internet cafe using the Webmail of his ISP with https is not
> necessarily in a better situation than the user in your man-
> in-the-middle scenario.

not sure what you mean by that.  with TLS the server is authenticated 
via the server certificate.  a hijacker doesn't have access to the 
session key and can't set its own session key without causing the server 
signature verification to fail.

of course, there are clients that don't check server certificates, and 
there are clients that implement "automatic" fail over from TLS or SSL 
to unencrypted - and hijackers can easily defeat those.  (we need to 
update recommendations for MUAs also)

>>users have a wide range of needs.  it's not acceptable to
>>limit all users to what you think "normal" users need.  and
>>this won't solve the spam problem anyway, not by a long shot.
> 
> 
> If your needs are "unnormal" you won't participate in a scheme
> offering to limit your routes, you also won't sign up for any
> limited acoounts.  You then have to solve problems if somebody
> forges your MAIL FROM by other means.

this marginalizes a class of users, which is inconsistent with the goal 
trying to make email service work well for everyone.  to be really 
useful email needs to be ubiquitous, and that means it has to be 
adaptable to a wide variety of users' needs.

>>I'm all for having a way for MTAs to insist that the
>>originator had permission to use that MAIL FROM address to
>>send the message. What's not acceptable is for SMTP or MSA
>>to decide this solely on the basis of SMTP AUTH credentials
>>or source IP address.
> 
> 
> How else should it work ?  User defines permitted MAIL FROM
> addresses, MSA authenticates user (somehow, see above about the
> less desirable ways), user sends mail with permitted MAIL FROM.
> 
> If the user tries another MAIL FROM the MSA says "no" (5xx).
> RFC 2476(bis) didn't forget MAIL FROM:<>

there are multiple ways to solve the problem.  one is that if a user 
provides a certificate to the MSA that says "I have permission to use 
this set of addresses in SMTP MAIL FROM" then the MSA permits such use.

another way (which only works for individual addresses) is that you get 
the MON to send a single message to a MAIL FROM address, the message 
contains a cookie, you reply to the message, now the MON knows you can 
read mail sent to that address and so are presumably able to use it.

another is that the MSA makes the mail traceable in some way other than 
MAIL FROM to a user who can be held accountable for his actions - then 
if the user sends out mail that abuses MAIL FROM then he gets billed for it.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Strict RFC x821/x822 Checking
Date:  Fri, 01 Jul 2005 13:23:55 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 11
Sender: [email-masked]

[email-masked] wrote:
 
> Isn't something within the "< >" an parameter???

That's a path, in simple cases a mailbox, "postmaster", or
empty.  Parameters come to the right of ">" after a space,
only in ESMTP (after EHLO), and if the server indicated to
support it.
                     Bye, Frank

To: Hector Santos <[email-masked]>
Cc: <[email-masked]>
From: [email-masked]
Subject: Re: Strict RFC x821/x822 Checking
Date: Fri, 1 Jul 2005 11:19:53 +0100
Sender: [email-masked]

I am following the discussion about  the :sp< discussion
And i am wondering is it not better to just to allow it in 2821bis

RFC x822 requires an space behind the ":" and
RFC x821 doesn't allow it.

Is there an easier recipe to get confused?

You can even find in RFC 2821 reasons why there should be a space behind the ":"

4.1.1 Command Semantics and Syntax

The SMTP commands define the mail transfer or the mail system
function requested by the user.  SMTP commands are character strings
terminated by <CRLF>.  The commands themselves are alphabetic
characters terminated by <SP> if parameters follow and <CRLF>
otherwise.  (In the interest of improved interoperability, SMTP
receivers are encouraged to tolerate trailing white space before the
terminating <CRLF>.)  The syntax of the local part of a mailbox must
conform to receiver site conventions and the syntax specified in
section 4.1.2.  The SMTP commands are discussed below.  The SMTP
replies are discussed in section 4.2.

Isn't something within the "< >" an parameter???
or are ":" "<" and ">" alphabetic characters
I find it all a bit against the Internet grain, to refuse spaces here.

Other RFc2821bis question.

I find RFC 2821 is very strict in error codes, (This refers especially to Section 4.3.2, wich names all possible errorcodes per command)

Should the new RFC allow more errorcodes or not?

In favor of more errorcodes:
Intelligent clients can work with it more precise

In favor to keep it as it is:
Simple clients doesn't need upgrading

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: request discussion of two documents on SMTP relaying
Date:  Fri, 01 Jul 2005 11:11:16 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 109
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

Keith Moore wrote:

>>> just a random MTA. so authentication means different things
>>> in the two cases.

>> Yes.  For the latter MTAMARK, CSV, or the HELO-part of SPF
>> can do the trick (add SIQ and it shouldn't be too
>> expensive).

> I was referring to client-to-server authentication, not
> originator-to-server authentication.

Now all I mentioned _is_ client to server, and unrelated to any
originator:  SPF-HELO and CSV are about HELO.  MTAMARK is about
"yes, this IP is used for an MTA" vs. "no, this IP is no MTA".

SIQ simplifies the lookup of FQDN and IP simultaneously, it
could cache (FQDN, IP, TTL, result) removing the different
methods to determine such results from the equation, from the
POV of the MTA (server) interested in the result for a client.

>> MSAa offered by ISPs or mail providers normally know which
>> mailbox(es) an authorized user has, for SMTP-after-POP it's
>> more or less obvious.

> yes, but there are legitimate reasons for not using the MSA
> offered by the mail network where you receive your mail, and
> there are also legitimate reasons for setting MAIL FROM to
> something other than the address at which you normally
> receive mail.

If you use another MSA just use the MAIL FROM related to this
MON, and if you want the error messages elsewhere forward them.
The results are very similar:

1 - you submit at A with MAIL FROM:<you@B> RCPT TO:<trouble@C>
  - trouble@C, somehow C sends an error message RCPT TO:<you@B>

2 - you setup .forward your.errors@A => you@B
  - you submit MAIL FROM:<your.error@A> RCPT TO:<trouble@C> at A
  - trouble@C, C sends a error message RCPT TO:<your.errors@A>
  - A forwards your errors to you@B

The second scenario is 100% pure STD 10, you could only say
that it introduces a new source of trouble between A and B.

OTOH it removes any potential trouble between C and B in the
first scenario (which doesn't work well for trouble near A):

      1:  A --> C    2:  A --> C
              /          A <-- C
             /           |
            /            v
          B              B

>> Based on that knowledge they can "enforce submission rights"

> yes they can, but it's not something we should encourage.

IBTD, it does what STD 10 used to do, take the responsibility
for what the MTA sends, including any trouble it might cause.

> SMTP-after-POP is easy for an man-in-the-middle attack to
> break.  the attacker intercepts the traffic between the MUA
> and the POP server, using its own IP address to talk to the
> POP server, then it can submit mail from the same IP address
> at will.

Yes, but with enforced submission rights it has to know how to
derive a permitted MAIL FROM from the POP-user id, or it has
to hijack both connections (POP and SMTP) from the client.

> SMTP-after-POP should be a SHOULD NOT or NOT RECOMMENDED just
> like AUTH PLAIN and AUTH LOGIN and CRAM-MD5 and APOP, because
> it's defeated just as easily.

Actually I'm not that paranoid when I use one of my ISPs and
the MSA of an unrelated mail provider.  A roaming user in an
internet cafe using the Webmail of his ISP with https is not
necessarily in a better situation than the user in your man-
in-the-middle scenario.

I don't see SHOULD NOT, but seeral security considerations and
a SHOULD for more desirable ways.

> users have a wide range of needs.  it's not acceptable to
> limit all users to what you think "normal" users need.  and
> this won't solve the spam problem anyway, not by a long shot.

If your needs are "unnormal" you won't participate in a scheme
offering to limit your routes, you also won't sign up for any
limited acoounts.  You then have to solve problems if somebody
forges your MAIL FROM by other means.

> I'm all for having a way for MTAs to insist that the
> originator had permission to use that MAIL FROM address to
> send the message. What's not acceptable is for SMTP or MSA
> to decide this solely on the basis of SMTP AUTH credentials
> or source IP address.

How else should it work ?  User defines permitted MAIL FROM
addresses, MSA authenticates user (somehow, see above about the
less desirable ways), user sends mail with permitted MAIL FROM.

If the user tries another MAIL FROM the MSA says "no" (5xx).
RFC 2476(bis) didn't forget MAIL FROM:<>

                            Bye, Frank

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>, "Frank Ellermann" <[email-masked]>
Subject: Re:  Open relay test (was: Bounce/System Notification Address Verification)
Date: Fri, 1 Jul 2005 04:54:33 -0400
Organization: Santronics Software, Inc.
Sender: [email-masked]

From: "Frank Ellermann" <[email-masked]>

> Hector Santos wrote:
>
> > I did learn one new trick that I think I will implement and
> > explore - flip it around, test the random address first.
> > If accept, it is operating in an open relay behavior.
>
> It could be also a "teergrube" accepting one character per
> second after DATA (with a 5xx after the data).  You're not
> testing a dummy (empty) DATA, or are you ?  I report this
> crap as spam.

Interesting.

No. The CBV does not reach the DATA. You don't want to send a message.
Beside,  if you were able to get to this point, you already got the result -
a PASS for the return path,

But from a host standpoint, even YAHOO.COM abandoned the DATA level user
validation for a RCPT TO user validation.

Here's a 12/2003 field testing log:

220 YSmtp mta224.mail.scd.yahoo.com ESMTP service ready
HELO mail.winserver.com
250 mta224.mail.scd.yahoo.com
mail from: <[email-masked]>
250 sender <[email-masked]> ok
RCPT TO: <[email-masked]>
250 recipient <[email-masked]> ok
RCPT TO: <[email-masked]>
250 recipient <[email-masked]> ok
data
354 go ahead
test
.
554 delivery error: dd This user doesn't have a yahoo.ca account
     ([email-masked]) [0] - mta224.mail.scd.yahoo.com
quit
221 mta224.mail.scd.yahoo.com
Connection to host lost.

Here is a WCSAP captured log back in Jan/2005:

# connecting to 67.28.114.36
S: 220 YSmtp mta189.mail.dcn.yahoo.com ESMTP service ready
C: NOOP WCSAP v2.01 Wildcat! Sender Authentication
                Protocol http://www.santronics.com
S: 250 OK
C: HELO mail.winserver.com
S: 250 mta189.mail.dcn.yahoo.com
C: MAIL FROM: <>
S: 250 null sender <> ok
C: RCPT TO: <good user [email-masked]>
S: 250 recipient <good user [email-masked]> ok
C: RCPT TO: <[email-masked]>
S: 452 Too many recipients
C: QUIT

I have all the logs to find out exactly when they switch if I have to go
there.   Definitely shortly after 12/2003. :-)

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  PRA question (was: Bounce/System Notification Address Verification)
Date:  Fri, 01 Jul 2005 09:45:21 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 37
Cc: Keith Moore <[email-masked]>
Sender: [email-masked]

Keith Moore wrote:

  [quoting Bruce]
>> that would be the principle that an experiment only affects
>> those who choose to participate in the experiment, and if
>> the experiment is a failure, no great harm is done to the
>> deployed infrastructure at large.

> which of course is not true in this case.

> Frank, if you were willing to write up an I-D of the form
> "SPF experiment considered harmful", explaining just why this
> is a Bad Idea, that might be a valuable community service.

It's more like "using PRA with v=spf1 considered harmful", and
from the v=spf1 side it's already a NOT RECOMMENDED (plus the
"without explicit consent" disclaimer).  From the PRA side it
is a SHOULD, do you think a new I-D addressing only why it is
really NOT RECOMMENDED helps ?

Offering an option for the eplicit consent, and some "ranting"
why PRA is a very dubious idea even with consent or on its own.

The part about the option is ready, up to some of the security
considerations.  And assuming that this I-D somehow makes it,
should it move the PRA RFC to historic, or update it ?

I'm very much for creative uses of the process, but so far all
I created is an unpublished "3710 obsolete" about this problem.

With the discussions between Brian, Dave, Sam, and John on the
general list another idea I had today was to update RFC 2026
so that authors could insist on a proper "IETF last call" for
an experimental RFC even if it was not created by an IETF WG.

                            Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  spamops-04 (was: Bounce/System Notification Address Verification)
Date:  Fri, 01 Jul 2005 09:13:01 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 54
Sender: [email-masked]

[email-masked] wrote:

> while it is arguably better to do address checks on the front
> end system, it certainly is not required, and may not even be
> possible in some setups. And when it is possible the security
> implications are interesting, to say the least.
 
> This is why I pushed back on Pekka Savola's suggestion that
> draft-hutzler-spamops-04.txt recommend/require this practice.

Somehow I must have missed or forgot this part, was it your
idea to say "MDAs" in the corresponding "SHALL NOT accept" ?

It's IMO really "best common practice" to export this knowledge
to the front end.  If you'd start to suppress bounces based on
some spam filtering it could be more messy than the _security_
implications of an exported local user list.

>> the MX could start a lock-step relay during RCPT TO rather
>> than store-and-forward, but I don't know of any such split
>> setups that work that way.
 
> We use this sort of thing as part of large scale deployments
> that support POP-before-SMTP

What's a "lock step relay", like CBV only forward ?  That would
be one way to export the knowledge about local users.  It could
also catch some "over quota" cases depending on the MDA.  With
more than one hop from MX to MDA it starts to get interesting.

> regardless of how desireable it may be to reject invalid
> recipient addresses early, it most certainly is not required

You'd need two MTAs or rather two IPs as mailouts, one would be
permanently blaklisted as the soure of bogus bounces to forged
addresses.  My attempts to discuss the issue in the spamcop NG
had no effect, and I'd stop when you had a fair chance:  If you
accept FAIL-protected forged mails to non-existing local users
it would be really your problem to discuss this with BLs.
  
> I average around a hundred of these every day.

How long do you have your -all ?  "My" spammer needed about 14
weeks until he got the idea.
 
> Keith mentioned earlier in this thread that an address
> verification specification would be a good idea.  I agree,
> and would be happy to assist in writing such a document.

What could it say, something about catch-all and "teergrube" ?
Or is this about some "lock step relay" / forward verification
within an MRN (MX to MDA) ?
                            Bye, Frank