Subject: Re: STARTTLS & EHLO: Errata text?
From: Hector Santos <[email-masked]>
To: Tony Finch <[email-masked]>
Cc: [email-masked]
Date: Sat, 31 Jan 2009 16:37:32 -0500
List-Id: <ietf-smtp.imc.org>

Tony Finch wrote:

>> To me, that is enough to give the client the incentive and understanding
>> that it needs to re-issue EHLO.
> 
> Remember this thread was started by someone who wrote code that did not.

+1.

This is why I think that it should be stated that the client MUST be 
prepared to issue an EHLO/HELO in a SHOULD currently stated 
specification.  So even if it did not send the EHLO/HELO and got a 
negative response from the server, it should then proceed with an 
EHLO/HELO.

So the one question I did have was the response code from the server. 
  As shown, the server issued 550. It was something:

    [TLS established]
    C: MAIL FROM <xxxx>
    S: 550 EHLO/HELO required.

Shouldn't the server response be 503 (Bad Sequence of commands)?

If so, should this be stated in the revised text?

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Sat, 31 Jan 2009 20:38:27 +0000
From: Tony Finch <[email-masked]>
To: Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

On Fri, 30 Jan 2009, Hector Santos wrote:
>
> But I still to get the  "Client MUST NOT trust" statement.  It has to trust,
> blind or otherwise, what the server presented up to this point before it can
> go to the next step.

The client can verify when it receives the server certificate that the
connection up to that point was with the correct server. Before TLS is up
and running, anything the client gets from the server must be treated as
provisional, to be verified using data from trustworthy sources.

Many TLS clients are too trusting - they rely on the server capability
list to decide whether to use TLS or not, which opens them to downgrade
attacks. If a client has reason to expect that the server supports TLS
then it should treat the absence of TLS support as an attack. (i.e. the
common MUA configuration option of "TLS when available" does not comply
with this requirement.) This is obviously problematic for inter-domain TLS
to an MX, but TLS to MX is fundamentally broken and needs rethinking.

> In my view, I think it should say:
>
>    The client MUST NOT presume the server extensions apply
>    in the secure state as it may have changed.

It already says that in the paragraph that follows my proposed text.

> To me, that is enough to give the client the incentive and understanding
> that it needs to re-issue EHLO.

Remember this thread was started by someone who wrote code that did not.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

Date: Sat, 31 Jan 2009 19:08:17 +0100
From: Alessandro Vesely <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

John C Klensin wrote:
> --On Friday, January 30, 2009 17:19 +0100 Alessandro Vesely
> <[email-masked]> wrote:
>> Not being an anonymous operator involves choosing an ISP that 
>> does reverse DNS delegations, and registering a domain 
>> directly rather than through a whois-privacy-enhanced 
>> registrar.
>
> Thank you for finally clarifying what you (and probably others) 
> meant by "non-anonymous operator" and "verifiable domain name". 
> My personal opinion and answer to your question is that, from a 
> protocol standpoint, it is undesirable to try to require a 
> higher standard for the EHLO argument that resolvability of the 
> domain name -- resolvability to _something_.  The status of the 
> domain registration, whether it is "privacy enhanced" or not, 
> etc., are reasonable matters for local policy, but not for the 
> SMTP protocol.

"Resolvability to anything" obviously has an obscure meaning.

Voluntarily disclosing one's identification, through proper [r]DNS 
registrations, is useful for conveying good intentions. For example, 
it makes it easier to register with Hotmail/Live. (By giving up its 
own anonymity, an ESP can shield its users', to some extent.) I agree 
this pertains to local policies.

The principle that only subscribers are enabled to send, which works 
for mailing lists, has no counterpart in SMTP. On an ethical 
configurator, I'd opt for accepting mail only from domains that 
usually listen on port 25, rather than (or in addition to) 
non-anonymity. Would this principle be reasonable for VHLO? Would it 
lock out zombies?

> If someone can notice that you are supporting 
> anonymous senders (typically an operational or political 
> determination, not a protocol one) and make you disclose their 
> identities with penalties including being shut down if you do 
> not, then there is no operational anonymity at least vis-a-vis 
> whomever can compel you in that way.

I looked for an anonymous remailer, but couldn't find one. (OK, I 
didn't search very hard.) On some sites, I found declarations that 
they discontinued operations because of abuse, not governmental shut 
down. Does abuse hurt small and medium ESPs more than large ones? I 
tend to think that giant providers have more bargaining power, also 
w.r.t. one another. Current developments of FBL and ARF apparently 
confirm that small ESPs may be going to experience some disadvantages 
when competing with such established giants.

The most flagrant case of email tracing I know of is that of Shi Tao 
(http://en.wikipedia.org/wiki/Shi_Tao) who is serving a 10 years 
sentence for sending one message, after Yahoo "practically led the 
police to his door" -in the words of Tom Lantos. Browsing through 
various associations' opinions, it is easy to find concerns about non 
democratic governments and transnational corporations getting into 
agreements with them. I don't want to imply that small postmasters are 
more valorous, although some may. However, it is certainly more 
difficult for an institution to control each small operator than to 
get in touch with a few big ones.

IMHO, consigning email to transnational corporations would ultimately 
attain even less support for those situations "in which strong 
anonymity and privacy are really important" -in your words.

Date: Fri, 30 Jan 2009 15:06:58 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

Tony Finch wrote:
> I suggest that this paragraph should read:
> 
>    Upon completion of the TLS handshake, the SMTP protocol is reset to
>    the initial state (the state in SMTP after a server issues a 220
>    service ready greeting).  The requirement in [RFC5321] that "a client
>    MUST issue HELO or EHLO before starting a mail transaction" also
>    applies to this fresh state.
> 
>    The server MUST NOT trust any knowledge obtained from the client before
>    the TLS negotiation itself.  The client MUST NOT trust any knowledge
>    obtained from the server before the TLS negotiation itself.  This
>    includes all commands, arguments, replies, and extended exchanges
>    (though in typical use there is little more than the client's EHLO
>    command and the server's reply).

This is much better form of the consolidated input.

But I still to get the  "Client MUST NOT trust" statement.  It has to 
trust, blind or otherwise, what the server presented up to this point 
before it can go to the next step.

In my view, I think it should say:

    The client MUST NOT presume the server extensions apply
    in the secure state as it may have changed.

To me, that is enough to give the client the incentive and 
understanding that it needs to re-issue EHLO.

Alternatively, stop confusing people any further with these beating 
around the bush subjective reasons, nix all attempts to explain the 
"whys," and just say:

   The client MUST re-issue the EHLO/HELO before attempting to begin
   a MAIL FROM transaction after secured SMTP is established.

which would keep it consistent with 2821/5321.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Fri, 30 Jan 2009 11:27:24 -0800 (PST)
From: [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
To: Tony Finch <[email-masked]>
Cc: Tony Hansen <[email-masked]>, [email-masked]
Sender: [email-masked]

> On Thu, 29 Jan 2009, Tony Hansen wrote:
> >
> > Section:
> >    4.2 Result of the STARTTLS Command
> >
> > Old text:
> >    The server MUST discard any knowledge obtained from the client, such
> >    as the argument to the EHLO command, which was not obtained from the
> >    TLS negotiation itself.
> >
> > New text:
> >    The server MUST discard any knowledge obtained from the client that
> >    was not obtained from the TLS negotiation itself. The server state
> >    is otherwise as if the connection had just been opened.

> I prefer Alexey's suggested "MUST NOT trust" text.

> Note that my comment about AUTH information applies to the client as well
> as the server.

> Also note that AUTH means this trust issue applies to more than just SMTP
> commands, arguments, and replies - it applies to SASL exchanges as well.

> > Section:
> >    4.2 Result of the STARTTLS Command
> >
> > Old text:
> >    The client SHOULD send an EHLO command as the
> >    first command after a successful TLS negotiation.
> >
> > New text:
> >    The client MUST send either an EHLO command or a HELO command as the
> >    first command after a successful TLS negotiation.

> That's incorrect. It's OK to QUIT at this point, for example. Rather than
> paraphrasing 2821/5321, perhaps it would be better to quote verbatim.

> I suggest that this paragraph should read:

>    Upon completion of the TLS handshake, the SMTP protocol is reset to
>    the initial state (the state in SMTP after a server issues a 220
>    service ready greeting).  The requirement in [RFC5321] that "a client
>    MUST issue HELO or EHLO before starting a mail transaction" also
>    applies to this fresh state.

>    The server MUST NOT trust any knowledge obtained from the client before
>    the TLS negotiation itself.  The client MUST NOT trust any knowledge
>    obtained from the server before the TLS negotiation itself.  This
>    includes all commands, arguments, replies, and extended exchanges
>    (though in typical use there is little more than the client's EHLO
>    command and the server's reply).

This definitely works for me.

> >    Because the server state machine is reset to an initial connection
> >    state after negotiating TLS, and any modifications to the server
> >    state will be lost, the client SHOULD NOT issue any MAIL
> >    FROM or RCPT TO commands prior to using the STARTTLS command.

> I think this is unnecessary. I doubt anyone has been confused enough to
> implement software that would violate this requirement. I only mentioned
> it for sake of argument.

Agreed.

				Ned

Date: Fri, 30 Jan 2009 19:05:07 +0000
From: Tony Finch <[email-masked]>
To: Tony Hansen <[email-masked]>
cc: [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

On Thu, 29 Jan 2009, Tony Hansen wrote:
>
> Section:
>    4.2 Result of the STARTTLS Command
>
> Old text:
>    The server MUST discard any knowledge obtained from the client, such
>    as the argument to the EHLO command, which was not obtained from the
>    TLS negotiation itself.
>
> New text:
>    The server MUST discard any knowledge obtained from the client that
>    was not obtained from the TLS negotiation itself. The server state
>    is otherwise as if the connection had just been opened.

I prefer Alexey's suggested "MUST NOT trust" text.

Note that my comment about AUTH information applies to the client as well
as the server.

Also note that AUTH means this trust issue applies to more than just SMTP
commands, arguments, and replies - it applies to SASL exchanges as well.

> Section:
>    4.2 Result of the STARTTLS Command
>
> Old text:
>    The client SHOULD send an EHLO command as the
>    first command after a successful TLS negotiation.
>
> New text:
>    The client MUST send either an EHLO command or a HELO command as the
>    first command after a successful TLS negotiation.

That's incorrect. It's OK to QUIT at this point, for example. Rather than
paraphrasing 2821/5321, perhaps it would be better to quote verbatim.

I suggest that this paragraph should read:

   Upon completion of the TLS handshake, the SMTP protocol is reset to
   the initial state (the state in SMTP after a server issues a 220
   service ready greeting).  The requirement in [RFC5321] that "a client
   MUST issue HELO or EHLO before starting a mail transaction" also
   applies to this fresh state.

   The server MUST NOT trust any knowledge obtained from the client before
   the TLS negotiation itself.  The client MUST NOT trust any knowledge
   obtained from the server before the TLS negotiation itself.  This
   includes all commands, arguments, replies, and extended exchanges
   (though in typical use there is little more than the client's EHLO
   command and the server's reply).

>    Because the server state machine is reset to an initial connection
>    state after negotiating TLS, and any modifications to the server
>    state will be lost, the client SHOULD NOT issue any MAIL
>    FROM or RCPT TO commands prior to using the STARTTLS command.

I think this is unnecessary. I doubt anyone has been confused enough to
implement software that would violate this requirement. I only mentioned
it for sake of argument.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

Date: Fri, 30 Jan 2009 18:31:33 +0000
From: Tony Finch <[email-masked]>
To: Paul Smith <[email-masked]>
cc: John C Klensin <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

On Fri, 30 Jan 2009, Paul Smith wrote:
>
> Just going back a step, shouldn't it be worded something like
>
> "The client MUST send a EHLO command after a successful TLS negotiation
> if it wishes to start a mail transaction or use any SMTP extensions."
>
> Because that's actually what we mean.

Yes.

> Also, saying 'or a command that does not require that a mail transaction
> be open' means that you could use an SMTP extension which was advertised
> before the STARTTLS, as long as it doesn't involve a mail transaction,
> without resending EHLO.

No, because the client must disregard the pre-TLS extension list because
the protocol is reset to its initial state.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

Date: Fri, 30 Jan 2009 18:29:46 +0000
From: Tony Finch <[email-masked]>
To: [email-masked]
cc: Alexey Melnikov <[email-masked]>, SM <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

On Thu, 29 Jan 2009, [email-masked] wrote:
>
> > I would like suggest an alternative: how about saying
> >
> >     The server MUST NOT trust any information obtained
> >     from the client, such as command verbs and their arguments, prior to
> >     the TLS negotiation.
> >     The client MUST NOT trust any information obtained from the server,
> >     such as the list of SMTP service extensions,
> >     prior to the TLS negotiation.
> >
> > This avoid the whole issue of what the client/server must and must not
> > remember.
>
> Very clever - it focuses on the real issue and avoids the slippery slope. . I
> like it a lot. This is definitely the way to go.

+1

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

Date: Fri, 30 Jan 2009 12:15:25 -0500
From: John C Klensin <[email-masked]>
To: Alessandro Vesely <[email-masked]>
cc: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

--On Friday, January 30, 2009 17:19 +0100 Alessandro Vesely
<[email-masked]> wrote:

>...
>> It depends on _exactly_ how you define "SMTP relay".  If it 
>> includes submission servers, the same privacy arguments that 
>> have been applied to senders would apply to it too.
> 
> I beg to differ. As a postmaster, I may grant anonymity to my
> users. For example, I may suppress identification in the
> Received header. Thus, I can be readily identifiable, e.g. via
> whois information, while my users can send anonymously by
> altering their From header. Note that their anonymity is only
> granted until a judge will want to investigate my logs. (Since
> 2005, the Italian law forces "operators" to keep logs of mail
> transactions for some months; in the UK, they've been
> discussing this recently --it's an EU anti-terrorism
> determination.)

And that is _exactly_ the point.  Perhaps you have never had
judges in Italy who would violate legal requirements to obtain
information about an anonymous mail message that criticized one
of their friends or relatives, or other types of officials who
would attempt to obtain that information from you even though
they are not legally entitled to it, but there have been ample
experiences with both in other parts of the world.   If you are
concerned about privacy as a primary objective, you probably
don't want to actually know who your users are, even if you
might want to establish credentialing systems to be able to
identify whether one is the same from one message to the next.

Obviously, that strong a position about privacy protection is
caught in a tradeoff against legitimate state interests in
tracing criminals and terrorists.   In particular circumstances,
that tradeoff may work out exactly as you describe -- the relay
(or submission server) knows the identities of its users but
hides those identities from the outside world unless confronted
with a legitimate order from a legitimate authority.   But the
differences, and the potential situations in which strong
anonymity and privacy are really important, demonstrates to me
that those decisions should be local and operational matters,
not something wired into the protocol.

So, if you say "this is how it works in my jurisdiction and the
protocol must not get in my way of providing service while
tracking my users", I think that is fine.  If you suggest that
the protocol and its supporting arrangements must be constructed
so that anonymity of a submission server or relay becomes
impossible, then I believe that is too much of a political
decision influencing a protocol one.

> Not being an anonymous operator involves choosing an ISP that
> does reverse DNS delegations, and registering a domain
> directly rather than   through a whois-privacy-enhanced
> registrar. The first step is not always possible, which is why
> we had that discussion on submission identifiers. The second
> step is the rule, AFAIK. Although I know that whois queries on
> each incoming message would not be tolerated, what I meant to
> ask is whether the recognizability implied by the EHLO command
> is meant, from an ethical POV, to reject anonymous relays.

Thank you for finally clarifying what you (and probably others)
meant by "non-anonymous operator" and "verifiable domain name".
My personal opinion and answer to your question is that, from a
protocol standpoint, it is undesirable to try to require a
higher standard for the EHLO argument that resolvability of the
domain name -- resolvability to _something_.  The status of the
domain registration, whether it is "privacy enhanced" or not,
etc., are reasonable matters for local policy, but not for the
SMTP protocol.

>>  The ability 
>> to close down or restrict traffic from servers that support 
>> anonymous senders is equivalent to the ability to shut the 
>> anonymous senders down.
> 
> I don't think I know what you mean. Formally, I cannot know if
> an SMTP server supports anonymous senders.

"Formally" has little to do with this, as the discussion above
should show.  If someone can notice that you are supporting
anonymous senders (typically an operational or political
determination, not a protocol one) and make you disclose their
identities with penalties including being shut down if you do
not, then there is no operational anonymity at least vis-a-vis
whomever can compel you in that way.

     john

To: [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 30 Jan 2009 08:48:50 -0800
Sender: [email-masked]

Hector Santos <[email-masked]> writes:
> Russ Allbery wrote:

>> It's not supposed to trust what the server said before STARTTLS, since
>> everything sent before STARTTLS may have been provided by a
>> man-in-the-middle attacker.  It's stronger than just not assuming that
>> the same extensions apply.  Even if extensions happen to still be
>> available, trusting the extension return before STARTTLS can permit an
>> attacker to launch a down-negotiation attack, for example.

> Maybe I don't see it.  If the client is being fooled, one would think
> that it would be to relax the client, not push it into a more secured
> mode.

Right, that's what a down-negotiation attack is.  The attacker would, for
instance, advertise that the server supported authentication but only list
weak authentication protocols.  If the client then proceeded on the basis
of the extension list from the attacker, it would use a weaker (and
possibly vulnerable) authentication protocol instead of a stronger one
that the server actually supports.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Fri, 30 Jan 2009 08:27:37 -0800 (PST)
From: [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
To: Paul Smith <[email-masked]>
Cc: [email-masked], SM <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Sender: [email-masked]

> [email-masked] wrote:
> >
> > While I have no objection to making this change, I note in passing
> > that quite a
> > few servers, ours included, violate the "the server MUST discard any
> > knowledge
> > obtained from the client" part of this and will continue to do so no
> > matter
> > what is written in any standard.

> I think you could argue that the number of messages you have accepted
> from the client, the session time, the number of recipients etc, is NOT
> information received from the client, but information derived by the
> server itself.

Maybe, but it's exactly the sort of handwaving argument we always lambast when
somebody uses it in support of some oddball thing they want to do that's
causing interoperability problems.

> If there was an extension for the client to say 'I'm going to send 6
> messages this session', then that information would have to be
> discarded, but the server remembering that 6 messages have already been
> sent is something the server could work out for itself.

> Also, AIUI, you could always refuse to accept STARTTLS after you have
> accepted a message. (I can't think of any good reason you'd want to send
> some messages with TLS and others without, but maybe others can)

Not only have there been people arguing for this in the past, there have also
been proponents who wanted to be able to do AUTH and STARTTLS in the
middle of a transaction. Fortunately we managed to stomp out the latter.

				Ned

Date: Fri, 30 Jan 2009 17:19:48 +0100
From: Alessandro Vesely <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

John C Klensin wrote:
> The following are all 
> perfectly valid decisions under 5321 as written:
>
> 	* Rejecting the EHLO command and message because the
> 	argument does not follow the syntax rules.
>
> 	* Rejecting the EHLO command and message because the
> 	apparent FQDN in the argument does not resolve at all in
> 	the public DNS.
>
> 	* Noticing that the EHLO argument does not resolve to
> 	the address obtained from the connection, writing a
> 	private-use header into the message that records that
> 	fact, and then forwarding/delivering the message anyway.
>
> 	* Noticing that the EHLO argument does not resolve to
> 	the address obtained from the connection, delivering the
> 	message anyway, but delivering it to a folder different
> 	from the one that would normally be used for incoming
> 	messages associated with the RCPT command address.

I agree with Paul that the second point is not obvious. The text above 
is much clearer than the spec!

>> DNS/whois data has experienced a series of adjustments for the 
>> sake of privacy and users' right to anonymity. I accept that it should 
>> be possible for a user to send mail anonymously. However, I'd 
>> refuse that the operators of an SMTP relay may remain anonymous. Is that a 
>> more or less universally agreed stance?
>
> It depends on _exactly_ how you define "SMTP relay".  If it 
> includes submission servers, the same privacy arguments that 
> have been applied to senders would apply to it too.

I beg to differ. As a postmaster, I may grant anonymity to my users. 
For example, I may suppress identification in the Received header. 
Thus, I can be readily identifiable, e.g. via whois information, while 
my users can send anonymously by altering their From header. Note that 
their anonymity is only granted until a judge will want to investigate 
my logs. (Since 2005, the Italian law forces "operators" to keep logs 
of mail transactions for some months; in the UK, they've been 
discussing this recently --it's an EU anti-terrorism determination.)

Not being an anonymous operator involves choosing an ISP that does 
reverse DNS delegations, and registering a domain directly rather than 
  through a whois-privacy-enhanced registrar. The first step is not 
always possible, which is why we had that discussion on submission 
identifiers. The second step is the rule, AFAIK. Although I know that 
whois queries on each incoming message would not be tolerated, what I 
meant to ask is whether the recognizability implied by the EHLO 
command is meant, from an ethical POV, to reject anonymous relays.

>  The ability 
> to close down or restrict traffic from servers that support 
> anonymous senders is equivalent to the ability to shut the 
> anonymous senders down.

I don't think I know what you mean. Formally, I cannot know if an SMTP 
server supports anonymous senders.

Date: Fri, 30 Jan 2009 10:08:53 -0500
From: John C Klensin <[email-masked]>
To: David MacQuigg <[email-masked]>, Alessandro Vesely <[email-masked]>, Paul Smith <[email-masked]>
cc: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Suggestion noted.   My notes indicate that the existing text was
worked out after considerable discussion, so I don't know how
the group will react.

FYI, "suggestion noted" means that, when
draft-klensin-rfc5321bis-00 appears, this suggestion will be
marked in the text for consideration.

  john

--On Friday, January 30, 2009 6:37 -0700 David MacQuigg
<[email-masked]> wrote:

> At 11:18 AM 1/29/2009 -0500, John C Klensin wrote:
> 
>>  5321 rather carefully explains
>> what is valid and what is not.  To be valid under 5321, it
>> must meet the syntax rules for either a Domain or for an
>> address literal.  If it looks like a Domain, it must be a
>> resolvable FQDN.  The server is also encouraged to check the
>> relationship between the EHLO argument and whatever else it
>> knows about the sending machine (particularly its IP address
>> from the connection) but told that it must not reject the
>> mail simply because whatever tests it applies at that level
>> fails.  That doesn't somehow make the argument "valid" or
>> "invalid" as far as 5321 is concerned, it just means that an
>> optional test failed, one whose results the server is
>> prohibited from using to reject the message.
> 
> This is too broad.  The "whatever tests" could include a CSV
> test on the heloname.
> 
>> If the intention of the text was to say "the argument to EHLO
>> is noise and you can ignore it", that certainly could have
>> been said, and probably should have been.   The following are
>> all perfectly valid decisions under 5321 as written:
>> 
>>        * Rejecting the EHLO command and message because the
>>        argument does not follow the syntax rules.
>>        
>>        * Rejecting the EHLO command and message because the
>>        apparent FQDN in the argument does not resolve at all
>>        in the public DNS.
>>        
>>        * Noticing that the EHLO argument does not resolve to
>>        the address obtained from the connection, writing a
>>        private-use header into the message that records that
>>        fact, and then forwarding/delivering the message
>>        anyway.
>>        
>>        * Noticing that the EHLO argument does not resolve to
>>        the address obtained from the connection, delivering
>>        the message anyway, but delivering it to a folder
>>        different from the one that would normally be used for
>>        incoming messages associated with the RCPT command
>>        address.
>> 
>> If others disagree with that interpretation, we should discuss
>> it.  If people believe that the text needs further
>> clarification, I'd welcome suggested text modifications.
> 
> This sounds a lot better than previous discussions.  The RFC
> says "corresponds to".  I see you are using "resolves to" in
> the examples above, which to me at least, implies a matching
> address record.  The RFC should be more clear.  How about this:
> 
> Existing text:
>    An SMTP server MAY verify that the domain name argument in
> the EHLO    command actually corresponds to the IP address of
> the client.    However, if the verification fails, the server
> MUST NOT refuse to    accept a message on that basis. 
> 
> Suggested revision:
>    An SMTP server MAY verify that the domain name argument in
> the EHLO    command has an address record matching the IP
> address of the client.    However, if the verification fails,
> the server MUST NOT refuse to    accept a message on that
> basis.
> 
> This avoids what appears to be a blanket prohibition against
> IP-based authentication of the heloname.
> 
> -- Dave
> 
> 

Date: Thu, 29 Jan 2009 22:44:38 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Russ Allbery <[email-masked]>
CC:  [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

Russ Allbery wrote:
> Hector Santos <[email-masked]> writes:
>> Alexey Melnikov wrote:
> 
>>> I would like suggest an alternative: how about saying
>>>
>>> The server MUST NOT trust any information obtained from the client,
>>> such as command verbs and their arguments, prior to the TLS
>>> negotiation.  The client MUST NOT trust any information obtained from
>>> the server, such as the list of SMTP service extensions, prior to the
>>> TLS negotiation.
>>>
>>> This avoid the whole issue of what the client/server must and must not
>>> remember.
>> I don't follow the client MUST NOT trust statement.  Is it not suppose to
>> believe what the server presents for extensions?
>>
>>    S:  We supports STARTTLS, AUTH CRAM-MD5
>>    C:  Liar!! No you don't, I don't believe you.
>>
>> ??
> 
> It's not supposed to trust what the server said before STARTTLS, since
> everything sent before STARTTLS may have been provided by a
> man-in-the-middle attacker.  It's stronger than just not assuming that the
> same extensions apply.  Even if extensions happen to still be available,
> trusting the extension return before STARTTLS can permit an attacker to
> launch a down-negotiation attack, for example.

Maybe I don't see it.  If the client is being fooled, one would think 
that it would be to relax the client, not push it into a more secured 
mode.

In either case, the client has to use what is presented. It can't 
assume anything else (that wasn't presented). That was my main point.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Fri, 30 Jan 2009 06:37:14 -0700
To: John C Klensin <[email-masked]>, Alessandro Vesely <[email-masked]>, Paul Smith <[email-masked]>
From: David MacQuigg <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Cc: [email-masked]
Sender: [email-masked]

At 11:18 AM 1/29/2009 -0500, John C Klensin wrote:

>  5321 rather carefully explains
>what is valid and what is not.  To be valid under 5321, it must
>meet the syntax rules for either a Domain or for an address
>literal.  If it looks like a Domain, it must be a resolvable
>FQDN.  The server is also encouraged to check the relationship
>between the EHLO argument and whatever else it knows about the
>sending machine (particularly its IP address from the
>connection) but told that it must not reject the mail simply
>because whatever tests it applies at that level fails.  That
>doesn't somehow make the argument "valid" or "invalid" as far as
>5321 is concerned, it just means that an optional test failed,
>one whose results the server is prohibited from using to reject
>the message.

This is too broad.  The "whatever tests" could include a CSV test on the heloname.

>If the intention of the text was to say "the argument to EHLO is
>noise and you can ignore it", that certainly could have been
>said, and probably should have been.   The following are all
>perfectly valid decisions under 5321 as written:
>
>        * Rejecting the EHLO command and message because the
>        argument does not follow the syntax rules.
>        
>        * Rejecting the EHLO command and message because the
>        apparent FQDN in the argument does not resolve at all in
>        the public DNS.
>        
>        * Noticing that the EHLO argument does not resolve to
>        the address obtained from the connection, writing a
>        private-use header into the message that records that
>        fact, and then forwarding/delivering the message anyway.
>        
>        * Noticing that the EHLO argument does not resolve to
>        the address obtained from the connection, delivering the
>        message anyway, but delivering it to a folder different
>        from the one that would normally be used for incoming
>        messages associated with the RCPT command address.
>
>If others disagree with that interpretation, we should discuss
>it.  If people believe that the text needs further
>clarification, I'd welcome suggested text modifications.

This sounds a lot better than previous discussions.  The RFC says "corresponds to".  I see you are using "resolves to" in the examples above, which to me at least, implies a matching address record.  The RFC should be more clear.  How about this:

Existing text:
   An SMTP server MAY verify that the domain name argument in the EHLO
   command actually corresponds to the IP address of the client.
   However, if the verification fails, the server MUST NOT refuse to
   accept a message on that basis. 

Suggested revision:
   An SMTP server MAY verify that the domain name argument in the EHLO
   command has an address record matching the IP address of the client.
   However, if the verification fails, the server MUST NOT refuse to
   accept a message on that basis.

This avoids what appears to be a blanket prohibition against IP-based authentication of the heloname.

-- Dave

Date: Fri, 30 Jan 2009 04:45:53 -0500
From: John C Klensin <[email-masked]>
To: Paul Smith <[email-masked]>
cc: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

--On Friday, January 30, 2009 9:39 +0000 Paul Smith
<[email-masked]> wrote:

> John C Klensin wrote:
>>> Old text:
>>>    The client SHOULD send an EHLO command as the
>>>    first command after a successful TLS negotiation.
>>> 
>>> New text:
>>>    The client MUST send either an EHLO command or a HELO
>>> command as the    first command after a successful TLS
>>> negotiation.
>>>     
>> 
>> s/HELO command as/HELO command, or a command that does not
>> require that a mail transaction be open, as/
>> 
>> That can be done in several other ways, but I don't think you
>> can or should prohibit VRFY, EXPN, HELP, etc., there.
>>   
> Just going back a step, shouldn't it be worded something like
> 
> "The client MUST send a EHLO command after a successful TLS
> negotiation if it wishes to start a mail transaction or use
> any SMTP extensions."
> 
> Because that's actually what we mean.
> 
> Also, saying 'or a command that does not require that a mail
> transaction be open' means that you could use an SMTP
> extension which was advertised before the STARTTLS, as long as
> it doesn't involve a mail transaction, without resending EHLO.

That would be a better fix, as would something along the lines
of the "must not trust" suggestion, as long as it was absolutely
clear whether or not EHLO needed to be resent (I think is does
because the client can't know whether or not the server intends
to make use of the argument).

    john

Date: Fri, 30 Jan 2009 09:34:16 +0000
From: Paul Smith <[email-masked]>
To: [email-masked]
CC: SM <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

[email-masked] wrote:
>
> While I have no objection to making this change, I note in passing
> that quite a
> few servers, ours included, violate the "the server MUST discard any
> knowledge
> obtained from the client" part of this and will continue to do so no
> matter
> what is written in any standard.
I think you could argue that the number of messages you have accepted
from the client, the session time, the number of recipients etc, is NOT
information received from the client, but information derived by the
server itself.

If there was an extension for the client to say 'I'm going to send 6
messages this session', then that information would have to be
discarded, but the server remembering that 6 messages have already been
sent is something the server could work out for itself.

Also, AIUI, you could always refuse to accept STARTTLS after you have
accepted a message. (I can't think of any good reason you'd want to send
some messages with TLS and others without, but maybe others can)

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Fri, 30 Jan 2009 09:39:03 +0000
From: Paul Smith <[email-masked]>
To: John C Klensin <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------090605050603090307040702
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

John C Klensin wrote:
>> Old text:
>>    The client SHOULD send an EHLO command as the
>>    first command after a successful TLS negotiation.
>>
>> New text:
>>    The client MUST send either an EHLO command or a HELO
>> command as the    first command after a successful TLS
>> negotiation.
>>     
>
> s/HELO command as/HELO command, or a command that does not
> require that a mail transaction be open, as/
>
> That can be done in several other ways, but I don't think you
> can or should prohibit VRFY, EXPN, HELP, etc., there.
>   
Just going back a step, shouldn't it be worded something like

"The client MUST send a EHLO command after a successful TLS negotiation
if it wishes to start a mail transaction or use any SMTP extensions."

Because that's actually what we mean.

Also, saying 'or a command that does not require that a mail transaction
be open' means that you could use an SMTP extension which was advertised
before the STARTTLS, as long as it doesn't involve a mail transaction,
without resending EHLO.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

--------------090605050603090307040702
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
John C Klensin wrote:
<blockquote cite="mid:[email-masked]" type="cite">
  <blockquote type="cite">
    <pre wrap="">Old text:
   The client SHOULD send an EHLO command as the
   first command after a successful TLS negotiation.

New text:
   The client MUST send either an EHLO command or a HELO
command as the    first command after a successful TLS
negotiation.
    </pre>
  </blockquote>
  <pre wrap=""><!---->
s/HELO command as/HELO command, or a command that does not
require that a mail transaction be open, as/

That can be done in several other ways, but I don't think you
can or should prohibit VRFY, EXPN, HELP, etc., there.
  </pre>
</blockquote>
Just going back a step, shouldn't it be worded something like<br>
<br>
"The client MUST send a EHLO command after a successful TLS negotiation
if it wishes to start a mail transaction or use any SMTP extensions."<br>
<br>
Because that's actually what we mean. <br>
<br>
Also, saying 'or a command that does not require that a mail
transaction be open' means that you could use an SMTP extension which
was advertised before the STARTTLS, as long as it doesn't involve a
mail transaction, without resending EHLO. <br>
<br>
<pre class="moz-signature" cols="72">-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows</pre>
</body>
</html>

--------------090605050603090307040702--

Date: Thu, 29 Jan 2009 19:51:57 -0800
To: Ned Freed <[email-masked]>
From: SM <[email-masked]>
Subject: Re: STARTTLS & EHLO: Errata text?
Cc: [email-masked]
Sender: [email-masked]

At 15:59 29-01-2009, Ned Freed wrote:
>Yes, but the text currently says MUST, not SHOULD:
>
>   The server MUST discard any knowledge
>   obtained from the client, such as the argument to the EHLO command,
>   which was not obtained from the TLS negotiation itself.
>I'm arguing that this MUST should be a SHOULD.

Alexey proposed a change that avoids the issue as it does not change the MUST.

>Actually, the specification is quite clear that the session state is
>reset to that of where the banner line has just been returned:
>
>   Upon completion of the TLS handshake, the SMTP protocol is reset to
>   the initial state (the state in SMTP after a server issues a 220
>   service ready greeting).

I quoted that part previously.  It's quite clear to me what "reset to 
initial state" means.

>So if the client sends a MAIL FROM without first sending a EHLO/HELO it is
>doing so to a server that's supposed to be in the "initial banner sent, no
>EHLO/HELO seen" state. IMO a server is perfectly entitled to refuse the MAIL
>FROM in this case.
>
>But I fail to see how this has any bearing on the point I've raised, which has
>to do with the server retaining session history after STARTTLS - something the
>specification currently forbids.

I was not addressing the question of session history; I was 
commenting on the EHLO requirement.

>Frankly, while I have no problem with clarifying the specification, 
>I don't buy
>the argument that the nonissuance of a EHLO/HELO after STARTTLS and 
>before MAIL
>FROM is in compliance with the SHOULD. Again, the specification is very clear
>about the required session state and a predictable consequence of that is that
>a HELO/EHLO is now required. IMO this is failing to understand the likely
>consequences of omitting the EHLO/HELO.

Agreed.

Regards,
-sm 

To: [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 29 Jan 2009 18:30:00 -0800
Sender: [email-masked]

Hector Santos <[email-masked]> writes:
> Alexey Melnikov wrote:

>> I would like suggest an alternative: how about saying
>>
>> The server MUST NOT trust any information obtained from the client,
>> such as command verbs and their arguments, prior to the TLS
>> negotiation.  The client MUST NOT trust any information obtained from
>> the server, such as the list of SMTP service extensions, prior to the
>> TLS negotiation.
>>
>> This avoid the whole issue of what the client/server must and must not
>> remember.
>
> I don't follow the client MUST NOT trust statement.  Is it not suppose to
> believe what the server presents for extensions?
>
>    S:  We supports STARTTLS, AUTH CRAM-MD5
>    C:  Liar!! No you don't, I don't believe you.
>
> ??

It's not supposed to trust what the server said before STARTTLS, since
everything sent before STARTTLS may have been provided by a
man-in-the-middle attacker.  It's stronger than just not assuming that the
same extensions apply.  Even if extensions happen to still be available,
trusting the extension return before STARTTLS can permit an attacker to
launch a down-negotiation attack, for example.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Thu, 29 Jan 2009 20:57:12 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Alexey Melnikov <[email-masked]>
CC:  [email-masked], SM <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

Alexey Melnikov wrote:

> I would like suggest an alternative: how about saying
> 
>  The server MUST NOT trust any information obtained
>  from the client, such as command verbs and their arguments, prior 
 >  to the TLS negotiation.
>  The client MUST NOT trust any information obtained from the server,
>  such as the list of SMTP service extensions,
>  prior to the TLS negotiation.
> 
> This avoid the whole issue of what the client/server must and must not 
> remember.

I don't follow the client MUST NOT trust statement.  Is it not suppose 
to believe what the server presents for extensions?

    S:  We supports STARTTLS, AUTH CRAM-MD5
    C:  Liar!! No you don't, I don't believe you.

??

I think what you implying is:

    The client MUST NOT presume the same server extensions apply
    after secured SMTP is established.

This is already discussed (implied) in 3207.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Thu, 29 Jan 2009 16:27:10 -0800 (PST)
From: [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
To: Alexey Melnikov <[email-masked]>
Cc: [email-masked], SM <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Sender: [email-masked]

> > The simplest fix to bring this text in line with reality is to change
> > the MUST
> > into a SHOULD. Beyond that lies a slippery slope where we attempt to
> > categorize
> > what sorts of information a server can or cannot retain. I really
> > don't think
> > we want to go there.

> I would like suggest an alternative: how about saying

>     The server MUST NOT trust any information obtained
>     from the client, such as command verbs and their arguments, prior to
>     the TLS negotiation.
>     The client MUST NOT trust any information obtained from the server,
>     such as the list of SMTP service extensions,
>     prior to the TLS negotiation.

> This avoid the whole issue of what the client/server must and must not
> remember.

Very clever - it focuses on the real issue and avoids the slippery slope. . I
like it a lot. This is definitely the way to go.

> > While this clarification exercise is all well and good, if we're
> > actually going
> > to issue a revision to RFC 3207 we should consider fixing its most
> > serious flaw
> > (IMO) - the lack of a domain parameter on the STARTTLS command, in
> > order to
> > allow a single SMTP server to provide "virtual hosting" support for
> > multiple
> > domains.

> This is an interesting issue, because it affects pretty much any
> application protocol that have STARTTLS command or similar.

> While I agree that this functionality is needed, I came to conclusion
> that this change at the application layer is not needed, as there a TLS
> extension for doing exactly that (RFC 4366, section 3.1). It might even
> be supported by OpenSSL.

> But lack of APIs for controlling this in TLS libraries might be a problem.

My understanding is there's an API for it in NSS, but for various reasons it's
effectively unusable. (Something about use of it not playing nice with how this
stuff interfaces with the cert db.) This is not something I code myself so I
don't know all the ins and outs of it, but what I do know is that the people
who do deal with this say it's a nonstarter given the present state of play of
existing TLS libraries.

All things being equal I would much rather see this dealt with at the TLS
layer. But that's not the current situation and I am unconvinced that it's
going to change faster than we could get an extension deployed.

				Ned

Date: Thu, 29 Jan 2009 19:31:55 -0500
From: John C Klensin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

--On Thursday, January 29, 2009 23:31 +0000 Alexey Melnikov
<[email-masked]> wrote:

> Yes, I've missed that. +1 (Modulo dropping HELO)

And I agree that we should lose HELO.  As was pointed out
earlier, both client and server have to be able to handle EHLO
in order to get near setting up a TLS session with STARTTLS.
There is no excuse for either to go back to HELO once the TLS
session has started.

   john

Date: Thu, 29 Jan 2009 19:01:45 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SM <[email-masked]>
CC: Ned Freed <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

SM wrote:

> One of the questions was about the "The client SHOULD send an EHLO 
> command as the first command after a successful TLS negotiation."  As 
> with everything SMTP, there are two sides, the sender and the receiver.  
> Instead of thinking in terms of whether the sender should send the 
> command, we could look at this in terms of whether the receiver must 
> accept a mail transaction without being sent an EHLO command.  I don't 
> see anything in the specifications that say that.

The more I think about this, I think its just all fine.  We had one 
client presented here who failed to reissue a EHLO/HELO as expected in 
practice.  The server issues a negative response and the client failed 
to continue properly.  Even if the implementer read 3207 as a SHOULD, 
that alone should be enough to tell the client that it MAY have to 
reissue the EHLO/HELO if the MAIL FROM: command failed.

In other words,

   SHOULD says "Be Prepared To Reissue EHLO/HELO"

There is nothing that says that it does not need to send it and abort 
the transaction if the server requires it.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Thu, 29 Jan 2009 15:59:45 -0800 (PST)
From: [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
To: SM <[email-masked]>
Cc: Ned Freed <[email-masked]>, [email-masked]
Sender: [email-masked]

> Hi Ned,
> At 12:46 29-01-2009, Ned Freed wrote:
> >While I have no objection to making this change, I note in passing
> >that quite a
> >few servers, ours included, violate the "the server MUST discard any knowledge
> >obtained from the client" part of this and will continue to do so no matter
> >what is written in any standard.

> I read that as the proposed text does not affect your implementation.

The proposed change does not; both the original and revised text do.

> >The reason for this is simple: Limits used in controlling spam and
> >DOS attacks.
> >Servers impose limits on all sorts of things, including but not limited to the
> >number of transactions in a session, the total number of recipients, the total
> >time a session has taken, and so on. If a server follows this MUST it turns
> >STARTTLS into a one time "reset all my rate limits" pass. That's simply not
> >acceptable in today's email climate.

> The reason we are having this discussion and the proposed errata is
> because of SHOULD and MUST.

Yes, I'm aware of that.

> The above lists some of the
> considerations when deciding about the requirement to be
> specified.   My understand of a SHOULD is "unless I have a good
> reason not to do it and I fully understand the implication".  That
> leaves room for local policy decisions as you explained above.

Yes, but the text currently says MUST, not SHOULD:

   The server MUST discard any knowledge
   obtained from the client, such as the argument to the EHLO command,
   which was not obtained from the TLS negotiation itself. 

I'm arguing that this MUST should be a SHOULD.

> One of the questions was about the "The client SHOULD send an EHLO
> command as the first command after a successful TLS negotiation."  As
> with everything SMTP, there are two sides, the sender and the
> receiver.  Instead of thinking in terms of whether the sender should
> send the command, we could look at this in terms of whether the
> receiver must accept a mail transaction without being sent an EHLO
> command.  I don't see anything in the specifications that say that.

Actually, the specification is quite clear that the session state is
reset to that of where the banner line has just been returned:

   Upon completion of the TLS handshake, the SMTP protocol is reset to
   the initial state (the state in SMTP after a server issues a 220
   service ready greeting).

So if the client sends a MAIL FROM without first sending a EHLO/HELO it is
doing so to a server that's supposed to be in the "initial banner sent, no
EHLO/HELO seen" state. IMO a server is perfectly entitled to refuse the MAIL
FROM in this case.

But I fail to see how this has any bearing on the point I've raised, which has
to do with the server retaining session history after STARTTLS - something the
specification currently forbids.

> >The simplest fix to bring this text in line with reality is to change the MUST
> >into a SHOULD. Beyond that lies a slippery slope where we attempt to
> >categorize
> >what sorts of information a server can or cannot retain. I really don't think
> >we want to go there.

> Agreed.  Such a change cannot be done in an errata.  I would like a
> SHOULD instead of a MUST or else we end up with a situation where we
> have to go against an absolute requirement.  Unfortunately, it causes
> the type of confusion we have seen in this thread.  If we attempt to
> categorize what knowledge is discard or can be retained, we'll end up
> with a lengthy specification with the problem it entails.

Frankly, while I have no problem with clarifying the specification, I don't buy
the argument that the nonissuance of a EHLO/HELO after STARTTLS and before MAIL
FROM is in compliance with the SHOULD. Again, the specification is very clear
about the required session state and a predictable consequence of that is that
a HELO/EHLO is now required. IMO this is failing to understand the likely
consequences of omitting the EHLO/HELO.

> >While this clarification exercise is all well and good, if we're
> >actually going
> >to issue a revision to RFC 3207 we should consider fixing its most
> >serious flaw
> >(IMO) - the lack of a domain parameter on the STARTTLS command, in order to
> >allow a single SMTP server to provide "virtual hosting" support for multiple
> >domains.

> This has been discussed previously.  It could be done by advertising
> a separate extension as you suggested.

Right, but since we're now bumping up against a possible revision, now's
tthe time to revisit the idea.

				Ned

Date: Thu, 29 Jan 2009 23:46:36 +0000
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
CC: SM <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

Hi Ned,

[email-masked] wrote:

>> Hi Tony,
>
>> Some of the text in this message is from from RFC 3207.
>
>> At 09:04 29-01-2009, Tony Hansen wrote:
>
>> >If we were to write an Errata against RFC 3207, I'd suggest text 
>> such as
>> >the following (in Errata format):
>> >
>> >Section:
>> >    4.2 Result of the STARTTLS Command
>> >
>> >Old text:
>> >    The server MUST discard any knowledge obtained from the client, 
>> such
>> >    as the argument to the EHLO command, which was not obtained from 
>> the
>> >    TLS negotiation itself.
>> >
>> >New text:
>> >    The server MUST discard any knowledge obtained from the client that
>> >    was not obtained from the TLS negotiation itself. The server state
>> >    is otherwise as if the connection had just been opened.
>> >
>> >Reason:
>> >    The example is misleading and has lead some people to think that
>> >    knowledge of an EHLO having been sent previously should be
>> >    remembered.
>
>> Quoting the entire paragraph from the RFC:
>
>>    "Upon completion of the TLS handshake, the SMTP protocol is reset to
>>     the initial state (the state in SMTP after a server issues a 220
>>     service ready greeting).  The server MUST discard any knowledge
>>     obtained from the client, such as the argument to the EHLO command,
>>     which was not obtained from the TLS negotiation itself.  The client
>>     MUST discard any knowledge obtained from the server, such as the 
>> list
>>     of SMTP service extensions, which was not obtained from the TLS
>>     negotiation itself.  The client SHOULD send an EHLO command as the
>>     first command after a successful TLS negotiation."
>
>> Updated text:
>
>>     Upon completion of the TLS handshake, the SMTP protocol is reset to
>>     the initial state (the state in SMTP after a server issues a 220
>>     service ready greeting).  The server MUST discard any knowledge 
>> obtained
>>     from the client, such as command verbs and their arguments, that was
>>     not obtained from the TLS negotiation itself.  The client MUST 
>> discard any
>>     knowledge obtained from the server, such as the list of SMTP 
>> service extensions,
>>     which was not obtained from the TLS  negotiation itself.  As the 
>> server state is
>>     as when a SMTP session is initiated, the client SHOULD send an 
>> EHLO command as the
>>     first command after a successful TLS negotiation.
>
> While I have no objection to making this change, I note in passing 
> that quite a
> few servers, ours included, violate the "the server MUST discard any 
> knowledge
> obtained from the client" part of this and will continue to do so no 
> matter
> what is written in any standard.
>
> The reason for this is simple: Limits used in controlling spam and DOS 
> attacks.
> Servers impose limits on all sorts of things, including but not 
> limited to the
> number of transactions in a session, the total number of recipients, 
> the total
> time a session has taken, and so on. If a server follows this MUST it 
> turns
> STARTTLS into a one time "reset all my rate limits" pass. That's 
> simply not
> acceptable in today's email climate.

Good point.

> Please note that I am not arguing that such limits are effective in 
> controlling
> spam or deflecting DOS attacks. (Nor am I arguing that they aren't.) 
> My point
> is rather that mail system administrators insist on having these sort 
> of limits
> available as part of their security toolbox. If they aren't available 
> or are
> there in a form that can easily be bypassed they will run a different 
> mail
> server product instead.
>
> The simplest fix to bring this text in line with reality is to change 
> the MUST
> into a SHOULD. Beyond that lies a slippery slope where we attempt to 
> categorize
> what sorts of information a server can or cannot retain. I really 
> don't think
> we want to go there.

I would like suggest an alternative: how about saying

    The server MUST NOT trust any information obtained
    from the client, such as command verbs and their arguments, prior to 
the TLS negotiation.
    The client MUST NOT trust any information obtained from the server,
    such as the list of SMTP service extensions,
    prior to the TLS negotiation.

This avoid the whole issue of what the client/server must and must not 
remember.
 [...]

>> >Section:
>> >    4. The STARTTLS Command
>> >
>> >Old text:
>> >    The format for the STARTTLS command is:
>> >
>> >    STARTTLS
>> >
>> >    with no parameters.
>> >
>> >New text:
>> >    The format for the STARTTLS command is:
>> >
>> >    STARTTLS
>> >
>> >    with no parameters.
>> >
>> >    Because the server state machine is reset to an initial connection
>> >    state after negotiating TLS, and any modifications to the server
>> >    state will be lost, the client SHOULD NOT issue any MAIL
>> >    FROM or RCPT TO commands prior to using the STARTTLS command.
>
>> Agreed.
>
> While this clarification exercise is all well and good, if we're 
> actually going
> to issue a revision to RFC 3207 we should consider fixing its most 
> serious flaw
> (IMO) - the lack of a domain parameter on the STARTTLS command, in 
> order to
> allow a single SMTP server to provide "virtual hosting" support for 
> multiple
> domains.

This is an interesting issue, because it affects pretty much any 
application protocol that have STARTTLS command or similar.

While I agree that this functionality is needed, I came to conclusion 
that this change at the application layer is not needed, as there a TLS 
extension for doing exactly that (RFC 4366, section 3.1). It might even 
be supported by OpenSSL.
But lack of APIs for controlling this in TLS libraries might be a problem.

> This would have to be done by advertising a separate extension, say
> STARTTLSDOMAIN, which if present says the STARTTLS command accept a 
> single,
> optional parameter specifying the domain associated with the 
> certificate the
> client would like to see the server assert. Text for this can be 
> adapted from
> section 6 of RFC 3887.
>
> As things stand now these sorts of virtual hosting setups require the 
> use of
> multiple separate ports or IP addresses (usually the latter given how 
> many
> clients don't support alternate ports). That's fine and dandy when 
> you  have
> two or three domains, but unacceptable when you're hosting thousands 
> of them or
> more.

Date: Thu, 29 Jan 2009 23:31:27 +0000
From: Alexey Melnikov <[email-masked]>
To: John C Klensin <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

John C Klensin wrote:

>Tony,
>
>Given the confusion about readings, I'd recommend modifying your
>changes to make them brutally clear.  Suggestions below.
>
>--On Thursday, January 29, 2009 12:04 -0500 Tony Hansen
><[email-masked]> wrote:
>  
>
>>If we were to write an Errata against RFC 3207, I'd suggest
>>text such as the following (in Errata format):
>>
>>Section:
>>   4.2 Result of the STARTTLS Command
>>
>>Old text:
>>   The server MUST discard any knowledge obtained from the
>>client, such    as the argument to the EHLO command, which was
>>not obtained from the    TLS negotiation itself.
>>
>>New text:
>>   The server MUST discard any knowledge obtained from the
>>client that    was not obtained from the TLS negotiation
>>itself. The server state    is otherwise as if the connection
>>had just been opened.
>>    
>>
>s/opened/opened, i.e., before a session has been established by
>the client sending EHLO/
>  
>
Sure.

>>Reason:
>>   The example is misleading and has lead some people to think
>>that    knowledge of an EHLO having been sent previously
>>should be    remembered.
>>    
>>
>>Section:
>>   4.2 Result of the STARTTLS Command
>>
>>Old text:
>>   The client SHOULD send an EHLO command as the
>>   first command after a successful TLS negotiation.
>>
>>New text:
>>   The client MUST send either an EHLO command or a HELO
>>command as the    first command after a successful TLS
>>negotiation.
>>    
>>
>s/HELO command as/HELO command, or a command that does not
>require that a mail transaction be open, as/
>
>That can be done in several other ways, but I don't think you
>can or should prohibit VRFY, EXPN, HELP, etc., there.
>  
>
Yes, I've missed that. +1 (Modulo dropping HELO)

Date: Thu, 29 Jan 2009 15:08:18 -0800
To: Ned Freed <[email-masked]>
From: SM <[email-masked]>
Subject: Re: STARTTLS & EHLO: Errata text?
Cc: [email-masked]
Sender: [email-masked]

Hi Ned,
At 12:46 29-01-2009, Ned Freed wrote:
>While I have no objection to making this change, I note in passing 
>that quite a
>few servers, ours included, violate the "the server MUST discard any knowledge
>obtained from the client" part of this and will continue to do so no matter
>what is written in any standard.

I read that as the proposed text does not affect your implementation.

>The reason for this is simple: Limits used in controlling spam and 
>DOS attacks.
>Servers impose limits on all sorts of things, including but not limited to the
>number of transactions in a session, the total number of recipients, the total
>time a session has taken, and so on. If a server follows this MUST it turns
>STARTTLS into a one time "reset all my rate limits" pass. That's simply not
>acceptable in today's email climate.

The reason we are having this discussion and the proposed errata is 
because of SHOULD and MUST.  The above lists some of the 
considerations when deciding about the requirement to be 
specified.   My understand of a SHOULD is "unless I have a good 
reason not to do it and I fully understand the implication".  That 
leaves room for local policy decisions as you explained above.

One of the questions was about the "The client SHOULD send an EHLO 
command as the first command after a successful TLS negotiation."  As 
with everything SMTP, there are two sides, the sender and the 
receiver.  Instead of thinking in terms of whether the sender should 
send the command, we could look at this in terms of whether the 
receiver must accept a mail transaction without being sent an EHLO 
command.  I don't see anything in the specifications that say that.

>The simplest fix to bring this text in line with reality is to change the MUST
>into a SHOULD. Beyond that lies a slippery slope where we attempt to 
>categorize
>what sorts of information a server can or cannot retain. I really don't think
>we want to go there.

Agreed.  Such a change cannot be done in an errata.  I would like a 
SHOULD instead of a MUST or else we end up with a situation where we 
have to go against an absolute requirement.  Unfortunately, it causes 
the type of confusion we have seen in this thread.  If we attempt to 
categorize what knowledge is discard or can be retained, we'll end up 
with a lengthy specification with the problem it entails.

>While this clarification exercise is all well and good, if we're 
>actually going
>to issue a revision to RFC 3207 we should consider fixing its most 
>serious flaw
>(IMO) - the lack of a domain parameter on the STARTTLS command, in order to
>allow a single SMTP server to provide "virtual hosting" support for multiple
>domains.

This has been discussed previously.  It could be done by advertising 
a separate extension as you suggested.

Regards,
-sm 

Date: Thu, 29 Jan 2009 12:46:08 -0800 (PST)
From: [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
To: SM <[email-masked]>
Cc: Tony Hansen <[email-masked]>, [email-masked]
Sender: [email-masked]

> Hi Tony,

> Some of the text in this message is from from RFC 3207.

> At 09:04 29-01-2009, Tony Hansen wrote:

> >If we were to write an Errata against RFC 3207, I'd suggest text such as
> >the following (in Errata format):
> >
> >Section:
> >    4.2 Result of the STARTTLS Command
> >
> >Old text:
> >    The server MUST discard any knowledge obtained from the client, such
> >    as the argument to the EHLO command, which was not obtained from the
> >    TLS negotiation itself.
> >
> >New text:
> >    The server MUST discard any knowledge obtained from the client that
> >    was not obtained from the TLS negotiation itself. The server state
> >    is otherwise as if the connection had just been opened.
> >
> >Reason:
> >    The example is misleading and has lead some people to think that
> >    knowledge of an EHLO having been sent previously should be
> >    remembered.

> Quoting the entire paragraph from the RFC:

>    "Upon completion of the TLS handshake, the SMTP protocol is reset to
>     the initial state (the state in SMTP after a server issues a 220
>     service ready greeting).  The server MUST discard any knowledge
>     obtained from the client, such as the argument to the EHLO command,
>     which was not obtained from the TLS negotiation itself.  The client
>     MUST discard any knowledge obtained from the server, such as the list
>     of SMTP service extensions, which was not obtained from the TLS
>     negotiation itself.  The client SHOULD send an EHLO command as the
>     first command after a successful TLS negotiation."

> Updated text:

>     Upon completion of the TLS handshake, the SMTP protocol is reset to
>     the initial state (the state in SMTP after a server issues a 220
>     service ready greeting).  The server MUST discard any knowledge obtained
>     from the client, such as command verbs and their arguments, that was
>     not obtained from the TLS negotiation itself.  The client MUST discard any
>     knowledge obtained from the server, such as the list of SMTP service extensions,
>     which was not obtained from the TLS  negotiation itself.  As the server state is
>     as when a SMTP session is initiated, the client SHOULD send an EHLO command as the
>     first command after a successful TLS negotiation.

While I have no objection to making this change, I note in passing that quite a
few servers, ours included, violate the "the server MUST discard any knowledge
obtained from the client" part of this and will continue to do so no matter
what is written in any standard.

The reason for this is simple: Limits used in controlling spam and DOS attacks.
Servers impose limits on all sorts of things, including but not limited to the
number of transactions in a session, the total number of recipients, the total
time a session has taken, and so on. If a server follows this MUST it turns
STARTTLS into a one time "reset all my rate limits" pass. That's simply not
acceptable in today's email climate.

Please note that I am not arguing that such limits are effective in controlling
spam or deflecting DOS attacks. (Nor am I arguing that they aren't.) My point
is rather that mail system administrators insist on having these sort of limits
available as part of their security toolbox. If they aren't available or are
there in a form that can easily be bypassed they will run a different mail
server product instead.

The simplest fix to bring this text in line with reality is to change the MUST
into a SHOULD. Beyond that lies a slippery slope where we attempt to categorize
what sorts of information a server can or cannot retain. I really don't think
we want to go there.

> >New text:
> >    The client MUST send either an EHLO command or a HELO command as the
> >    first command after a successful TLS negotiation.

> As the two ends are support EHLO, there is no need to have HELO
> there.  I suggest leaving the MUST to the next revision.

Works for me.

> >Reason:
> >    Since the state is reset to that of a connection having just been
> >    opened, the requirement from RFC 5321 applies:
> >
> >         In any event, a client MUST issue HELO or EHLO before starting a
> >         mail transaction.

> I didn't use MUST for the EHLO because there isn't any restriction on
> whether the client can only perform mail transactions in RFC 3207 or
> RFC 5321.  In the updated text, it is implied that the client must
> not forget that the server supports the STARTTLS extension.  Do we
> expressly need to say that? :-)

It seems pretty obvious so I see no need to say it.

> >Section:
> >    4. The STARTTLS Command
> >
> >Old text:
> >    The format for the STARTTLS command is:
> >
> >    STARTTLS
> >
> >    with no parameters.
> >
> >New text:
> >    The format for the STARTTLS command is:
> >
> >    STARTTLS
> >
> >    with no parameters.
> >
> >    Because the server state machine is reset to an initial connection
> >    state after negotiating TLS, and any modifications to the server
> >    state will be lost, the client SHOULD NOT issue any MAIL
> >    FROM or RCPT TO commands prior to using the STARTTLS command.

> Agreed.

While this clarification exercise is all well and good, if we're actually going
to issue a revision to RFC 3207 we should consider fixing its most serious flaw
(IMO) - the lack of a domain parameter on the STARTTLS command, in order to
allow a single SMTP server to provide "virtual hosting" support for multiple
domains.

This would have to be done by advertising a separate extension, say
STARTTLSDOMAIN, which if present says the STARTTLS command accept a single,
optional parameter specifying the domain associated with the certificate the
client would like to see the server assert. Text for this can be adapted from
section 6 of RFC 3887.

As things stand now these sorts of virtual hosting setups require the use of
multiple separate ports or IP addresses (usually the latter given how many
clients don't support alternate ports). That's fine and dandy when you  have
two or three domains, but unacceptable when you're hosting thousands of them or
more.

> >Now for the $64k questions:
> >
> >1) Is there consensus behind this viewpoint?

> See updated text.

> >3) If so, who wants to file the Errata?

I guess I have no problem with an errata clarifing this one point, but a
revision is going to be needed to get at the real issues in RFC 3207.

> You have already volunteered. :-)

I'd be happy to help with revision.

				Ned

Date: Thu, 29 Jan 2009 15:03:02 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC:  [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

Tony Hansen wrote:
> If we were to write an Errata against RFC 3207, I'd suggest text such as
> the following (in Errata format):
> 
> Section:
>    4.2 Result of the STARTTLS Command
> 
> Old text:
>    The server MUST discard any knowledge obtained from the client, such
>    as the argument to the EHLO command, which was not obtained from the
>    TLS negotiation itself.
> 
> New text:
>    The server MUST discard any knowledge obtained from the client that
>    was not obtained from the TLS negotiation itself. The server state
>    is otherwise as if the connection had just been opened.

But it isn't because its in an higher secured state.  I took a look at 
our server and we have this:

BOOL TSMTPServer::STARTTLS(char *args)
{
    // possible replies from RFC 3207
    //   220 Ready to start TLS
    //   501 Syntax error (no parameters allowed)
    //   454 TLS not available due to temporary reason

     if (TLSStarted) {
         Send("501 TLS already negotiated!\r\n");
         return TRUE;
     }
     ...
     ...
}

I can only suggest that our engineers added it for a reason.  Not 
having an explicit "already in TLS state" reply code, 501 was used.

My suggestion is to add a new reply code tells the client if its 
already in a secured state or not.

> Reason:
>    The example is misleading and has lead some people to think that
>    knowledge of an EHLO having been sent previously should be
>    remembered.
> 

I always thought the key reason is that server features may change 
depending on the secured or non-secured SMTP state.

Our own server does not enforce the EHLO/HELO reissuing.  If you think 
about it, it doesn't make sense to resend it given the fact that the 
CLIENT DOMAIN information can not change.  It might want to use a 
different client domain but it MUST still be reflected of the sender 
machine regardless.

Our client does send it thought, mainly because that is what most 
servers expect.

> Section:
>    4.2 Result of the STARTTLS Command
> 
> Old text:
>    The client SHOULD send an EHLO command as the
>    first command after a successful TLS negotiation.
> 
> New text:
>    The client MUST send either an EHLO command or a HELO command as the
>    first command after a successful TLS negotiation.

I would not encourage a downgrade from EHLO to HELO potential. The 
client started in EHLO, it should probably remain in an extended state.

  > Section:
>    4. The STARTTLS Command
> 
> Old text:
>    The format for the STARTTLS command is:
> 
>    STARTTLS
> 
>    with no parameters.
> 
> New text:
>    The format for the STARTTLS command is:
> 
>    STARTTLS
> 
>    with no parameters.
> 
>    Because the server state machine is reset to an initial connection
>    state after negotiating TLS, and any modifications to the server
>    state will be lost, the client SHOULD NOT issue any MAIL
>    FROM or RCPT TO commands prior to using the STARTTLS command.

This text is confusing (to me).  I think the common usage of "server 
state will be lost" is not quite right simply because the secured 
state of the SMTP session is not lost.

> Now for the $64k questions:
> 
> 1) Is there consensus behind this viewpoint?

I don't agree with the proposes errata text as is.

> 2) If so, does the text above cover the ground?

I don't think so.

> 3) If so, who wants to file the Errata?

Paul should <g>  He's the one using a client that doesn't send the 
EHLO. :-)

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Thu, 29 Jan 2009 12:01:40 -0800
From: Bill McQuillan <[email-masked]>
To: SMTP Discussion <[email-masked]>
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

On Thu, 2009-01-29, Tony Hansen wrote:

> If we were to write an Errata against RFC 3207, I'd suggest text such as
> the following (in Errata format):

> Section:
>    4.2 Result of the STARTTLS Command

> Old text:
>    The server MUST discard any knowledge obtained from the client, such
>    as the argument to the EHLO command, which was not obtained from the
>    TLS negotiation itself.

> New text:
>    The server MUST discard any knowledge obtained from the client that
>    was not obtained from the TLS negotiation itself. The server state
>    is otherwise as if the connection had just been opened.

This could be read to mean that the server should send the 220 greeting
again. I don't think that's what you want. I think the phrase should be
"initial state" instead of "connection had just been opened".

> Reason:
>    The example is misleading and has lead some people to think that
>    knowledge of an EHLO having been sent previously should be
>    remembered.

[...]

> Section:
>    4. The STARTTLS Command

> Old text:
>    The format for the STARTTLS command is:

>    STARTTLS

>    with no parameters.

> New text:
>    The format for the STARTTLS command is:

>    STARTTLS

>    with no parameters.

>    Because the server state machine is reset to an initial connection
>    state after negotiating TLS, and any modifications to the server
>    state will be lost, the client SHOULD NOT issue any MAIL
>    FROM or RCPT TO commands prior to using the STARTTLS command.

This does not address the issue of one or more complete messages having
been sent prior to the STARTTLS command. Does this mean that such messages
should be discarded even though the "250 OK" response has been sent for
them?

All right, that's a bit of a stretch, but if we're talking about clarity
of the language, let's be clear.

-- 
Bill McQuillan <[email-masked]>

Date: Thu, 29 Jan 2009 14:59:04 -0500
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

Tony,

Given the confusion about readings, I'd recommend modifying your
changes to make them brutally clear.  Suggestions below.

--On Thursday, January 29, 2009 12:04 -0500 Tony Hansen
<[email-masked]> wrote:

> 
> If we were to write an Errata against RFC 3207, I'd suggest
> text such as the following (in Errata format):
> 
> Section:
>    4.2 Result of the STARTTLS Command
> 
> Old text:
>    The server MUST discard any knowledge obtained from the
> client, such    as the argument to the EHLO command, which was
> not obtained from the    TLS negotiation itself.
> 
> New text:
>    The server MUST discard any knowledge obtained from the
> client that    was not obtained from the TLS negotiation
> itself. The server state    is otherwise as if the connection
> had just been opened.

s/opened/opened, i.e., before a session has been established by
the client sending EHLO/
 
> Reason:
>    The example is misleading and has lead some people to think
> that    knowledge of an EHLO having been sent previously
> should be    remembered.
 
> Section:
>    4.2 Result of the STARTTLS Command
> 
> Old text:
>    The client SHOULD send an EHLO command as the
>    first command after a successful TLS negotiation.
> 
> New text:
>    The client MUST send either an EHLO command or a HELO
> command as the    first command after a successful TLS
> negotiation.

s/HELO command as/HELO command, or a command that does not
require that a mail transaction be open, as/

That can be done in several other ways, but I don't think you
can or should prohibit VRFY, EXPN, HELP, etc., there.

> Reason:
>    Since the state is reset to that of a connection having
> just been    opened, the requirement from RFC 5321 applies:
> 
> 	In any event, a client MUST issue HELO or EHLO before
> starting a 	mail transaction.
> 
>    The previous text implied that a client can get by without
> sending    one or the either.

> Now for the $64k questions:
> 
> 1) Is there consensus behind this viewpoint?

Wfm, with the changes above.

> 2) If so, does the text above cover the ground?

See suggestions above.

> 3) If so, who wants to file the Errata?

Having written it, is there any reason why you should not just
go ahead and do it?

    john

Date: Thu, 29 Jan 2009 10:46:43 -0800
To: Tony Hansen <[email-masked]>
From: SM <[email-masked]>
Subject: Re: STARTTLS & EHLO: Errata text?
Cc: [email-masked]
Sender: [email-masked]

Hi Tony,

Some of the text in this message is from from RFC 3207.

At 09:04 29-01-2009, Tony Hansen wrote:

>If we were to write an Errata against RFC 3207, I'd suggest text such as
>the following (in Errata format):
>
>Section:
>    4.2 Result of the STARTTLS Command
>
>Old text:
>    The server MUST discard any knowledge obtained from the client, such
>    as the argument to the EHLO command, which was not obtained from the
>    TLS negotiation itself.
>
>New text:
>    The server MUST discard any knowledge obtained from the client that
>    was not obtained from the TLS negotiation itself. The server state
>    is otherwise as if the connection had just been opened.
>
>Reason:
>    The example is misleading and has lead some people to think that
>    knowledge of an EHLO having been sent previously should be
>    remembered.

Quoting the entire paragraph from the RFC:

   "Upon completion of the TLS handshake, the SMTP protocol is reset to
    the initial state (the state in SMTP after a server issues a 220
    service ready greeting).  The server MUST discard any knowledge
    obtained from the client, such as the argument to the EHLO command,
    which was not obtained from the TLS negotiation itself.  The client
    MUST discard any knowledge obtained from the server, such as the list
    of SMTP service extensions, which was not obtained from the TLS
    negotiation itself.  The client SHOULD send an EHLO command as the
    first command after a successful TLS negotiation."

Updated text:

    Upon completion of the TLS handshake, the SMTP protocol is reset to
    the initial state (the state in SMTP after a server issues a 220
    service ready greeting).  The server MUST discard any knowledge obtained
    from the client, such as command verbs and their arguments, that was
    not obtained from the TLS negotiation itself.  The client MUST discard any
    knowledge obtained from the server, such as the list of SMTP 
service extensions,
    which was not obtained from the TLS  negotiation itself.  As the 
server state is
    as when a SMTP session is initiated, the client SHOULD send an 
EHLO command as the
    first command after a successful TLS negotiation.

>New text:
>    The client MUST send either an EHLO command or a HELO command as the
>    first command after a successful TLS negotiation.

As the two ends are support EHLO, there is no need to have HELO 
there.  I suggest leaving the MUST to the next revision.

>Reason:
>    Since the state is reset to that of a connection having just been
>    opened, the requirement from RFC 5321 applies:
>
>         In any event, a client MUST issue HELO or EHLO before starting a
>         mail transaction.

I didn't use MUST for the EHLO because there isn't any restriction on 
whether the client can only perform mail transactions in RFC 3207 or 
RFC 5321.  In the updated text, it is implied that the client must 
not forget that the server supports the STARTTLS extension.  Do we 
expressly need to say that? :-)

>Section:
>    4. The STARTTLS Command
>
>Old text:
>    The format for the STARTTLS command is:
>
>    STARTTLS
>
>    with no parameters.
>
>New text:
>    The format for the STARTTLS command is:
>
>    STARTTLS
>
>    with no parameters.
>
>    Because the server state machine is reset to an initial connection
>    state after negotiating TLS, and any modifications to the server
>    state will be lost, the client SHOULD NOT issue any MAIL
>    FROM or RCPT TO commands prior to using the STARTTLS command.

Agreed.

>Now for the $64k questions:
>
>1) Is there consensus behind this viewpoint?

See updated text.

>3) If so, who wants to file the Errata?

You have already volunteered. :-)

Regards,
-sm 

Date: Thu, 29 Jan 2009 17:36:22 +0000
From: Paul Smith <[email-masked]>
To: John C Klensin <[email-masked]>
CC: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

John C Klensin wrote:
> The following are all
> perfectly valid decisions under 5321 as written:
>
> 	* Rejecting the EHLO command and message because the
> 	argument does not follow the syntax rules.
>   
Yes

> 	* Rejecting the EHLO command and message because the
> 	apparent FQDN in the argument does not resolve at all in
> 	the public DNS.
>   
Not sure about this one. The wording says:

However, if the verification fails, the server MUST NOT refuse to
   accept a message on that basis.

It doesn't say 'if the domain argument resolves to a different IP
address than that of the client it MUST NOT refuse to accept the message
on that basis'. It says 'if the verification fails', that could
potentially be 'host not found' as well as different IP address or whatever.

The 'verifying the argument corresponds to the IP address' is also a bit
vague. One person could say that this means that the A record for the
argument should match the IP address, but what about CNAMEs, or, could
the argument be a name which has an MX record which has an A record
matching the IP address. In a sense, that still means the argument
corresponds to the IP address (especially since we're talking about mail
here).

Given that you shouldn't refuse the message on that basis, it doesn't
matter what you mean by 'corresponds', but if people are starting to
refuse the message (or even 'redirect' it), it might need clarification.
This could affect your point above, where 'domain.com' doesn't have an A
record (so "doesn't resolve in the public DNS") but does have an MX
record referring to the host.
> 	* Noticing that the EHLO argument does not resolve to
> 	the address obtained from the connection, writing a
> 	private-use header into the message that records that
> 	fact, and then forwarding/delivering the message anyway.
> 	
> 	* Noticing that the EHLO argument does not resolve to
> 	the address obtained from the connection, delivering the
> 	message anyway, but delivering it to a folder different
> 	from the one that would normally be used for incoming
> 	messages associated with the RCPT command address.
>   
Yes.

There's also the 'not bothering to check the EHLO argument at all',
which is valid under 5321 AFAICS.

Also AFAICS, it could also use the failure of the EHLO argument as a
'hint' for further spam filtering. Eg, if the EHLO argument didn't
resolve to the IP address, it could treat it as 'more suspicious', so do
more rigorous spam checking, as long as it doesn't refuse to accept the
message solely on the basis of a failed IP address verification.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Thu, 29 Jan 2009 17:38:21 +0000
From: Alexey Melnikov <[email-masked]>
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

Tony Hansen wrote:

>If we were to write an Errata against RFC 3207, I'd suggest text such as
>the following (in Errata format):
>
>Section:
>   4.2 Result of the STARTTLS Command
>
>Old text:
>   The server MUST discard any knowledge obtained from the client, such
>   as the argument to the EHLO command, which was not obtained from the
>   TLS negotiation itself.
>
>New text:
>   The server MUST discard any knowledge obtained from the client that
>   was not obtained from the TLS negotiation itself. The server state
>   is otherwise as if the connection had just been opened.
>
I like that.

>Reason:
>   The example is misleading and has lead some people to think that
>   knowledge of an EHLO having been sent previously should be
>   remembered.
>
>Section:
>   4.2 Result of the STARTTLS Command
>
>Old text:
>   The client SHOULD send an EHLO command as the
>   first command after a successful TLS negotiation.
>
>New text:
>   The client MUST send either an EHLO command or a HELO command as the
>   first command after a successful TLS negotiation.
>  
>
I don't think we should recommend using HELO here: the client has issued 
STARTTLS already, so clearly it knows how to use EHLO.
Besides, I agree that we should move toward deprecating HELO.

>Reason:
>   Since the state is reset to that of a connection having just been
>   opened, the requirement from RFC 5321 applies:
>
>	In any event, a client MUST issue HELO or EHLO before starting a
>	mail transaction.
>
>   The previous text implied that a client can get by without sending
>   one or the either.
>
>Section:
>   4. The STARTTLS Command
>
>Old text:
>   The format for the STARTTLS command is:
>
>   STARTTLS
>
>   with no parameters.
>
>New text:
>   The format for the STARTTLS command is:
>
>   STARTTLS
>
>   with no parameters.
>
>   Because the server state machine is reset to an initial connection
>   state after negotiating TLS, and any modifications to the server
>   state will be lost, the client SHOULD NOT issue any MAIL
>   FROM or RCPT TO commands prior to using the STARTTLS command.
>  
>
I would be Ok with making this a MUST NOT.
Also, I think it would be better to say "the client SHOULD/MUST NOT 
start a mail transaction prior to ...".

>Now for the $64k questions:
>
>1) Is there consensus behind this viewpoint?
>
We shall see :-).

>2) If so, does the text above cover the ground?
>
I frankly can't tell without rereading the RFC, but the text seems to 
address the immediate problems.

>3) If so, who wants to file the Errata?
>  
>
Go ahead and do that once there is the mailing list consensus.

Date: Thu, 29 Jan 2009 12:04:43 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: STARTTLS & EHLO: Errata text?
Sender: [email-masked]

If we were to write an Errata against RFC 3207, I'd suggest text such as
the following (in Errata format):

Section:
   4.2 Result of the STARTTLS Command

Old text:
   The server MUST discard any knowledge obtained from the client, such
   as the argument to the EHLO command, which was not obtained from the
   TLS negotiation itself.

New text:
   The server MUST discard any knowledge obtained from the client that
   was not obtained from the TLS negotiation itself. The server state
   is otherwise as if the connection had just been opened.

Reason:
   The example is misleading and has lead some people to think that
   knowledge of an EHLO having been sent previously should be
   remembered.

Section:
   4.2 Result of the STARTTLS Command

Old text:
   The client SHOULD send an EHLO command as the
   first command after a successful TLS negotiation.

New text:
   The client MUST send either an EHLO command or a HELO command as the
   first command after a successful TLS negotiation.

Reason:
   Since the state is reset to that of a connection having just been
   opened, the requirement from RFC 5321 applies:

	In any event, a client MUST issue HELO or EHLO before starting a
	mail transaction.

   The previous text implied that a client can get by without sending
   one or the either.

Section:
   4. The STARTTLS Command

Old text:
   The format for the STARTTLS command is:

   STARTTLS

   with no parameters.

New text:
   The format for the STARTTLS command is:

   STARTTLS

   with no parameters.

   Because the server state machine is reset to an initial connection
   state after negotiating TLS, and any modifications to the server
   state will be lost, the client SHOULD NOT issue any MAIL
   FROM or RCPT TO commands prior to using the STARTTLS command.

Now for the $64k questions:

1) Is there consensus behind this viewpoint?

2) If so, does the text above cover the ground?

3) If so, who wants to file the Errata?

	Tony Hansen
	[email-masked]

John C Klensin wrote:
> 
> 
> --On Thursday, January 29, 2009 10:09 -0500 Tony Hansen
> <[email-masked]> wrote:
> 
>> If this is the interpretation that we gain consensus on, that
>> it means "start over from scratch", it might as well also say
>> that it
>>
>> 	SHOULD only be executed immediately after the initial EHLO.
>>
>> The only possible exceptions to this rule would be for verbs
>> that don't affect the state machine, such as VRFY, EXPN, HELP,
>> NOOP.
> 
> RSET is also harmless immediately after EHLO.  Changing the
> state from "session open (EHLO issued), no mail transaction
> state" to "session open (EHLO issued), no mail transaction
> state" is a no-op.  And the first paragraph of 4.1.1.5 of 5321
> says that.
> 
> Out of context, I'm not sure exactly what you are suggesting
> above, but I believe that it would mean:
> 
>     S: 220 ...
>     C: EHLO ...
>     S: 250-...
>     S: 250-STARTTLS
>     S: 250-...
>     S: 250 OK
>     C: STARTTLS  ...
>     (TLS session starts)
>     and the next command must be either EHLO
>     or as many instances of any of VRFY, 
>       EXPN, HELP, NOOP, RSET as desired, followed by EHLO
> 
> Also, the "what the server MUST (or SHOULD) discard and the
> client MUST (or SHOULD) not depend on" sentences and example
> might reasonably be modified to explicitly include any
> information gained from VRFY or EXPN issued between the 220
> greeting and the initial EHLO.  While one might not trust VRFY
> or EXPN queries or results issued under TLS either, it would be
> pointless and silly to send them before the initial EHLO if one
> knew that one was going to issue STARTTLS if the server
> permitted it.   Indeed the only reason for doing so would be if
> one intended to make a decision about whether to continue with a
> mail transaction at all based on the results of VRFY or EXPN...
> and that would be very rare today except in special
> circumstances.
> 
>       john
> 

Date: Thu, 29 Jan 2009 11:45:54 -0500
From: Tony Hansen <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

I was taking Tony Finch's comments

	But there's plenty of other information that the server has to
	discard - for example any AUTH results, any partial MAIL
	transactions - which isn't explicitly listed in RFC 3207.

to its next step. He was saying that

	S: <start>
	C: 220
	S: MAIL FROM:<...>
	C: 250 OK
	S: RCPT TO:<...>
	C: 250 OK
	S: STARTTLS

that the MAIL_FROM and RCPT_TO information is also discarded. So my
observation was that

	STARTTLS SHOULD only be executed immediately after the initial
	EHLO.

Doing otherwise may cause the server to do extra processing that will
all just get thrown away.

The verbs that I gave examples of were ones that don't affect the state
machine in any way, so would be reasonable to send between <start> and
STARTTLS. That's why I said "SHOULD" instead of "MUST".

I'm going to suggest an Errata in a separate email message.

	Tony

John C Klensin wrote:
> 
> 
> --On Thursday, January 29, 2009 10:09 -0500 Tony Hansen
> <[email-masked]> wrote:
> 
>> If this is the interpretation that we gain consensus on, that
>> it means "start over from scratch", it might as well also say
>> that it
>>
>> 	SHOULD only be executed immediately after the initial EHLO.
>>
>> The only possible exceptions to this rule would be for verbs
>> that don't affect the state machine, such as VRFY, EXPN, HELP,
>> NOOP.
> 
> RSET is also harmless immediately after EHLO.  Changing the
> state from "session open (EHLO issued), no mail transaction
> state" to "session open (EHLO issued), no mail transaction
> state" is a no-op.  And the first paragraph of 4.1.1.5 of 5321
> says that.
> 
> Out of context, I'm not sure exactly what you are suggesting
> above, but I believe that it would mean:
> 
>     S: 220 ...
>     C: EHLO ...
>     S: 250-...
>     S: 250-STARTTLS
>     S: 250-...
>     S: 250 OK
>     C: STARTTLS  ...
>     (TLS session starts)
>     and the next command must be either EHLO
>     or as many instances of any of VRFY, 
>       EXPN, HELP, NOOP, RSET as desired, followed by EHLO
> 
> Also, the "what the server MUST (or SHOULD) discard and the
> client MUST (or SHOULD) not depend on" sentences and example
> might reasonably be modified to explicitly include any
> information gained from VRFY or EXPN issued between the 220
> greeting and the initial EHLO.  While one might not trust VRFY
> or EXPN queries or results issued under TLS either, it would be
> pointless and silly to send them before the initial EHLO if one
> knew that one was going to issue STARTTLS if the server
> permitted it.   Indeed the only reason for doing so would be if
> one intended to make a decision about whether to continue with a
> mail transaction at all based on the results of VRFY or EXPN...
> and that would be very rare today except in special
> circumstances.
> 
>       john
> 

Date: Thu, 29 Jan 2009 11:18:24 -0500
From: John C Klensin <[email-masked]>
To: Alessandro Vesely <[email-masked]>, Paul Smith <[email-masked]>
cc: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

--On Thursday, January 29, 2009 16:28 +0100 Alessandro Vesely
<[email-masked]> wrote:

>...
>> We 
>> have yet to come across a recipient where if they change it
>> so that it sends 'EHLO [<local ip address>]' or 'EHLO
>> domain.com' it won't work, even though the first is useless
>> and the second is strictly incorrect.

> I'd say the second is valid, since the spec says it must not
> be rejected. From an ethical POV, having a DNS record to
> confirm that the domain endorses the address being used should
> be preferred. (From an operations POV, users cannot read their
> mail from outside the office without that.)

Alessandro, I think this is what you are missing that Paul and
others are trying to explain.  5321 rather carefully explains
what is valid and what is not.  To be valid under 5321, it must
meet the syntax rules for either a Domain or for an address
literal.  If it looks like a Domain, it must be a resolvable
FQDN.  The server is also encouraged to check the relationship
between the EHLO argument and whatever else it knows about the
sending machine (particularly its IP address from the
connection) but told that it must not reject the mail simply
because whatever tests it applies at that level fails.  That
doesn't somehow make the argument "valid" or "invalid" as far as
5321 is concerned, it just means that an optional test failed,
one whose results the server is prohibited from using to reject
the message.

If the intention of the text was to say "the argument to EHLO is
noise and you can ignore it", that certainly could have been
said, and probably should have been.   The following are all
perfectly valid decisions under 5321 as written:

	* Rejecting the EHLO command and message because the
	argument does not follow the syntax rules.
	
	* Rejecting the EHLO command and message because the
	apparent FQDN in the argument does not resolve at all in
	the public DNS.
	
	* Noticing that the EHLO argument does not resolve to
	the address obtained from the connection, writing a
	private-use header into the message that records that
	fact, and then forwarding/delivering the message anyway.
	
	* Noticing that the EHLO argument does not resolve to
	the address obtained from the connection, delivering the
	message anyway, but delivering it to a folder different
	from the one that would normally be used for incoming
	messages associated with the RCPT command address.

If others disagree with that interpretation, we should discuss
it.  If people believe that the text needs further
clarification, I'd welcome suggested text modifications.  But
I'm confident that the description above is what 5321 is
intended to say and how it should be interpreted.   Other
interpretations are, IMO, a change to the spec.

> OK, DNS checks can be worked around. However, I'd reckon that
> it is
> still easier for legitimate senders than for spammers to do
> that.

In many cases, actually not.  

> DNS/whois data has experienced a series of adjustments for the
> sake of
> privacy and users' right to anonymity. I accept that it should
> be
> possible for a user to send mail anonymously. However, I'd
> refuse that
> the operators of an SMTP relay may remain anonymous. Is that a
> more or less universally agreed stance?

It depends on _exactly_ how you define "SMTP relay".  If it
includes submission servers, the same privacy arguments that
have been applied to senders would apply to it too.  The ability
to close down or restrict traffic from servers that support
anonymous senders is equivalent to the ability to shut the
anonymous senders down.

    john

Date: Thu, 29 Jan 2009 10:53:49 -0500
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

--On Thursday, January 29, 2009 10:09 -0500 Tony Hansen
<[email-masked]> wrote:

> If this is the interpretation that we gain consensus on, that
> it means "start over from scratch", it might as well also say
> that it
> 
> 	SHOULD only be executed immediately after the initial EHLO.
> 
> The only possible exceptions to this rule would be for verbs
> that don't affect the state machine, such as VRFY, EXPN, HELP,
> NOOP.

RSET is also harmless immediately after EHLO.  Changing the
state from "session open (EHLO issued), no mail transaction
state" to "session open (EHLO issued), no mail transaction
state" is a no-op.  And the first paragraph of 4.1.1.5 of 5321
says that.

Out of context, I'm not sure exactly what you are suggesting
above, but I believe that it would mean:

    S: 220 ...
    C: EHLO ...
    S: 250-...
    S: 250-STARTTLS
    S: 250-...
    S: 250 OK
    C: STARTTLS  ...
    (TLS session starts)
    and the next command must be either EHLO
    or as many instances of any of VRFY, 
      EXPN, HELP, NOOP, RSET as desired, followed by EHLO

Also, the "what the server MUST (or SHOULD) discard and the
client MUST (or SHOULD) not depend on" sentences and example
might reasonably be modified to explicitly include any
information gained from VRFY or EXPN issued between the 220
greeting and the initial EHLO.  While one might not trust VRFY
or EXPN queries or results issued under TLS either, it would be
pointless and silly to send them before the initial EHLO if one
knew that one was going to issue STARTTLS if the server
permitted it.   Indeed the only reason for doing so would be if
one intended to make a decision about whether to continue with a
mail transaction at all based on the results of VRFY or EXPN...
and that would be very rare today except in special
circumstances.

      john

Date: Thu, 29 Jan 2009 16:28:33 +0100
From: Alessandro Vesely <[email-masked]>
To: Paul Smith <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Paul Smith wrote:
> 
> In our experience of supporting small businesses' mail servers it 
> is actually very rare to check the EHLO parameter at all. [...] We 
> have yet to come across a recipient where if they change it so that
> it sends 'EHLO [<local ip address>]' or 'EHLO domain.com' it won't
> work, even though the first is useless and the second is strictly
> incorrect.

I'd say the second is valid, since the spec says it must not be
rejected. From an ethical POV, having a DNS record to confirm that the
domain endorses the address being used should be preferred. (From an
operations POV, users cannot read their mail from outside the office
without that.)

> AIUI, this is what is expected from RFC 5321, and it means that 
> spammers haven't put any effort into what EHLO parameter to send, 
> because it doesn't matter what you use if the recipient is 
> standards compliant.
> 
> If this changed, (as was suggested) so that the EHLO checking was 
> almost universal, then it would break lots of legitimate senders as
>  well as spammers, but the spammers would be able to fix it a lot 
> easier than legitimate senders.

OK, DNS checks can be worked around. However, I'd reckon that it is
still easier for legitimate senders than for spammers to do that.

DNS/whois data has experienced a series of adjustments for the sake of
privacy and users' right to anonymity. I accept that it should be
possible for a user to send mail anonymously. However, I'd refuse that
the operators of an SMTP relay may remain anonymous. Is that a more or
less universally agreed stance?

Date: Thu, 29 Jan 2009 10:09:49 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

If this is the interpretation that we gain consensus on, that it means
"start over from scratch", it might as well also say that it

	SHOULD only be executed immediately after the initial EHLO.

The only possible exceptions to this rule would be for verbs that don't
affect the state machine, such as VRFY, EXPN, HELP, NOOP.

	Tony Hansen
	[email-masked]

Paul Smith wrote:
> Tony Finch wrote:
>> On Wed, 28 Jan 2009, Paul Smith wrote:
>>   
>>> To me, it was (initially) 'clear' that the example saying 'such as the
>>> argument to the EHLO command', was precise enough to imply that the fact
>>> that the EHLO command was sent should not be discarded. It could have
>>> said 'such as the EHLO command', but it went out of its way to say '*the
>>> argument to* the EHLO command'.
>>>     
>>
>> But there's plenty of other information that the server has to discard -
>> for example any AUTH results, any partial MAIL transactions - which isn't
>> explicitly listed in RFC 3207.
>>   
> Yes, but it has an example which was more specific than it needed to be
> if it just meant 'forget that anything ever happened, and start the
> session again from scratch' (other than obviously the fact that STARTTLS
> was sent).
> 
> It's like saying 'cars, such as blue Honda Civics, aren't allowed over
> this bridge'. Does this mean that just blue Honda Civics aren't allowed
> over the bridge, or cars that look like that aren't allowed, or blue
> cars, or Honda Civics, or all Hondas, or all cars? The 'such as blue
> Honda Civics' confuses the statement, rather than clarifying it.
> 
> I'm not arguing what RFC 3207 means, just that I don't think it's as
> clear as it could be.
> 
> -- 
> Paul Smith
> 
> VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows
> 

Date: Thu, 29 Jan 2009 14:44:37 +0000
From: Paul Smith <[email-masked]>
To: Tony Finch <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------090804020803020701090303
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Tony Finch wrote:
> On Wed, 28 Jan 2009, Paul Smith wrote:
>   
>> To me, it was (initially) 'clear' that the example saying 'such as the
>> argument to the EHLO command', was precise enough to imply that the fact
>> that the EHLO command was sent should not be discarded. It could have
>> said 'such as the EHLO command', but it went out of its way to say '*the
>> argument to* the EHLO command'.
>>     
>
> But there's plenty of other information that the server has to discard -
> for example any AUTH results, any partial MAIL transactions - which isn't
> explicitly listed in RFC 3207.
>   
Yes, but it has an example which was more specific than it needed to be
if it just meant 'forget that anything ever happened, and start the
session again from scratch' (other than obviously the fact that STARTTLS
was sent).

It's like saying 'cars, such as blue Honda Civics, aren't allowed over
this bridge'. Does this mean that just blue Honda Civics aren't allowed
over the bridge, or cars that look like that aren't allowed, or blue
cars, or Honda Civics, or all Hondas, or all cars? The 'such as blue
Honda Civics' confuses the statement, rather than clarifying it.

I'm not arguing what RFC 3207 means, just that I don't think it's as
clear as it could be.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

--------------090804020803020701090303
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Tony Finch wrote:
<blockquote
 cite="mid:[email-masked]"
 type="cite">
  <pre wrap="">On Wed, 28 Jan 2009, Paul Smith wrote:
  </pre>
  <blockquote type="cite">
    <pre wrap="">To me, it was (initially) 'clear' that the example saying 'such as the
argument to the EHLO command', was precise enough to imply that the fact
that the EHLO command was sent should not be discarded. It could have
said 'such as the EHLO command', but it went out of its way to say '*the
argument to* the EHLO command'.
    </pre>
  </blockquote>
  <pre wrap=""><!---->
But there's plenty of other information that the server has to discard -
for example any AUTH results, any partial MAIL transactions - which isn't
explicitly listed in RFC 3207.
  </pre>
</blockquote>
Yes, but it has an example which was more specific than it needed to be
if it just meant 'forget that anything ever happened, and start the
session again from scratch' (other than obviously the fact that
STARTTLS was sent).<br>
<br>
It's like saying 'cars, such as blue Honda Civics, aren't allowed over
this bridge'. Does this mean that just blue Honda Civics aren't allowed
over the bridge, or cars that look like that aren't allowed, or blue
cars, or Honda Civics, or all Hondas, or all cars? The 'such as blue
Honda Civics' confuses the statement, rather than clarifying it.<br>
<br>
I'm not arguing what RFC 3207 means, just that I don't think it's as
clear as it could be.<br>
<br>
<pre class="moz-signature" cols="72">-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows</pre>
</body>
</html>

--------------090804020803020701090303--

Date: Thu, 29 Jan 2009 13:37:28 +0000
From: Tony Finch <[email-masked]>
To: Paul Smith <[email-masked]>
cc: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

On Wed, 28 Jan 2009, Paul Smith wrote:
>
> To me, it was (initially) 'clear' that the example saying 'such as the
> argument to the EHLO command', was precise enough to imply that the fact
> that the EHLO command was sent should not be discarded. It could have
> said 'such as the EHLO command', but it went out of its way to say '*the
> argument to* the EHLO command'.

But there's plenty of other information that the server has to discard -
for example any AUTH results, any partial MAIL transactions - which isn't
explicitly listed in RFC 3207.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

Date: Thu, 29 Jan 2009 08:59:03 +0000
From: Paul Smith <[email-masked]>
To: Hector Santos <[email-masked]>
CC: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Hector Santos wrote:
> Paul,
>
> I'm surprise you are suggesting these spoof attempts doesn't exist in
> the real world because of the simplicity or dubious nature. The fact
> is, the frequency of HELO/EHLO spoofing of all sorts is very high.
I'm not convinced that, at the moment, you can call it 'spoofing'. It is
currently extremely rare to block a message based on a bad EHLO
parameter (because RFC 2821, 5321 prohibit that), so spammers really
don't care. If lots of recipients started blocking messages based on
that, it'd take all of a couple of hours for the spammers to work around it.

In our experience of supporting small businesses' mail servers it is
actually very rare to check the EHLO parameter at all. We have customers
who have their server set to send 'EHLO server' for many years, and then
suddenly come across a recipient which requires a syntactically correct
host name (ie a FQDN). We have yet to come across a recipient where if
they change it so that it sends 'EHLO [<local ip address>]' or 'EHLO
domain.com' it won't work, even though the first is useless and the
second is strictly incorrect.

AIUI, this is what is expected from RFC 5321, and it means that spammers
haven't put any effort into what EHLO parameter to send, because it
doesn't matter what you use if the recipient is standards compliant.

If this changed, (as was suggested) so that the EHLO checking was almost
universal, then it would break lots of legitimate senders as well as
spammers, but the spammers would be able to fix it a lot easier than
legitimate senders.
>> In which case, how did the EHLO test *really* help? 
>
> To stop the obvious spoofing attempts, which do occur at a very high
> frequency.  I am scratching my head as to why you would be questioning
> it.
If you are doing this at the moment, you are breaking RFC 5321 which
explicitly says you MUST NOT block messages if the EHLO parameter
doesn't match the sender IP address.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Wed, 28 Jan 2009 20:14:18 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC:  [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

HI Tony,

Yes, right, I acknowledge this question. My point is that the 
simplistic statement in 4.2

    The client SHOULD send an EHLO command as the
    first command after a successful TLS negotiation.

was not "considerate" of the other key points the documents makes 
about private vs public vs authentication levels it clearly made.

IOW, programmatically, the client should of interpret these points in 
the document and see this a MUST is the only logical option simply 
because of the other key points made about how the server status can 
change, especially when accessing a private smtp host (ie. SUBMIT server),

If the private submit server presented only STARTTLS then the client 
has no choice but to start with STARTTLS and when completely, only 
then the SERVER will present the AUTH schemes, then the client has no 
choice but to reissue the EHLO to find what methods are available.

The point is that there are some states the force the issue, 
programmatically.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Tony Hansen wrote:
> Hector, what started this was a client that doesn't use any other
> extended smtp features beyond STARTTLS. So once it had started up TLS,
> it had no use for the output of EHLO, and consequently it didn't bother
> sending either HELO or EHLO after the STARTTLS. This was fine until it
> ran into a server that demanded that EHLO be sent again. Leading to the
> questions on the table: 1) which interpretation of the spec is correct,
> and 2) what should be done once this is decided?
> 
> 	Tony Hansen
> 	[email-masked]
> 
> Hector Santos wrote:
>> My take on this:
>>
>> I think the client should be aware that most, if not all, servers expect
>> a 2nd pass of EHLO, but I also believe the client should not presume
>> server features would be the same under TLS and NON-TLS sessions.
>>
>> For example, the use case where the private or even public server allows
>> a stronger AUTH for non TLS sessions and allows weaker AUTH under TLS:
>>
>>    S: 220 blah blah blah
>>    C: EHLO client-domain.com
>>    S: 250-server-domain.com
>>    S: 250 AUTH CRAM-MD5 OTHER
>>    S: 250 STARTTLS
>>    C: MAIL FROM: <xxxxxxx>
>>    S: 530 Must issue a STARTTLS or AUTH command first
>>    ...
>>
>> Note: both RFC 2487 (TLS) and RFC 2554 (AUTH) use 530 for an enforcement
>> requirement. So here, if it client support CRAM or DIGEST it can
>> continue with AUTH CRAM-MD5 or AUTH OTHER. But if it does not support
>> these stronger AUTH, and does support TLS, it can continue with a STARTLS:
>>
>>    ...
>>    C: STARTTLS
>>    S: 220 Go ahead
>>    C & S: <TLS negotiation>
>>    C: MAIL FROM: <xxxxxxx>
>>    S: 530 AUTH command first
>>    C: EHLO client-domain.com
>>    S: 250-server-domain.com
>>    S: 250 AUTH LOGIN
>>    S: 250 AUTH-LOGIN
>>    S: AUTH LOGIN
>>    S: 235 Authentication successful.
>>
>> The point is, the client MUST be aware of this possibility and should
>> always (MUST) issue an EHLO after TLS is established.
>>
>> The idea of public vs private and level of authentication is discussed
>> in 2487 and this is where the 4.2 statement:
>>
>>    The client SHOULD send an EHLO command as the
>>    first command after a successful TLS negotiation.
>>
>> is "forgetful" of its other key points.  So it should of been something
>> like:
>>
>>    For a SMTP server that is privately referenced, the client
>>    MUST issue the EHLO command after a successful TLS negotiation
>>    to determine if there are any changes in AUTH methods, such
>>    as the strength of the authentication.
>>
>>    If the server is the publicly reference, the client SHOULD
>>    send an EHLO command as the first command after a successful
>>    TLS negotiation, however, it is highly recommended it reissues
>>    the EHLO in all cases.
>>
>>    The client SHOULD not degrade from EHLO to HELO.
>>
>> My take.
>>
> 
> 
> 

Date: Wed, 28 Jan 2009 14:47:16 -0800
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

At 13:33 28-01-2009, Tony Hansen wrote:
>Hector, what started this was a client that doesn't use any other
>extended smtp features beyond STARTTLS. So once it had started up TLS,
>it had no use for the output of EHLO, and consequently it didn't bother
>sending either HELO or EHLO after the STARTTLS. This was fine until it
>ran into a server that demanded that EHLO be sent again. Leading to the
>questions on the table: 1) which interpretation of the spec is correct,
>and 2) what should be done once this is decided?

 From RFC 3207:

  "Upon completion of the TLS handshake, the SMTP protocol is reset to
    the initial state (the state in SMTP after a server issues a 220
    service ready greeting)."

The SMTP Client can either send a HELO or EHLO after that.  As the 
SMTP Client supports the service extension (it would not have been 
able to do a STARTTLS without that), the likely command would be EHLO 
to determine which extensions the SMTP server supports.  If we go to 
RFC 2821, we find that "a client MUST issue HELO or EHLO before 
starting a mail transaction."

Paul Hoffman agreed to make some clarifications a few months ago.

Regards,
-sm 

Date: Wed, 28 Jan 2009 17:42:33 -0500
From: John C Klensin <[email-masked]>
To: Peter Bowyer <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

--On Wednesday, January 28, 2009 21:15 +0000 Peter Bowyer
<[email-masked]> wrote:

> 2009/1/28 Paul Smith <[email-masked]>:
> 
>> To me, it was (initially) 'clear' that the example saying
>> 'such as the argument to the EHLO command', was precise
>> enough to imply that the fact that the EHLO command was sent
>> should not be discarded. It could have said 'such as the EHLO
>> command', but it went out of its way to say '*the argument
>> to* the EHLO command'.
> 
> But the 'domain' argument to the EHLO command is mandatory
> (RFC1869 S4.2). So a server state of having received a valid
> EHLO but not knowing what the domain argument is, is not
> attainable under 1869. I don't believe 3207's intent is to
> introduce that state as valid after STARTTLS.

Peter, I don't believe that either 1859 or 5321 _require_ that
the server the EHLO domain or maintain it as part of its state.
Even if the server tries to inspect or check the field, as 5321
recommends, it would be a reasonable interpretation by the
server implementer to just maintain binary pass/fail
information.  

That is, IMO, exactly what creates the situation here.  If the
server intends to use the domain for anything that it considers
significant, it must get that information again after STARTTLS.
But there appears to be nothing in either the SMTP spec or the
STARTTS one that requires that it consider that information
significant.  Conversely, if the client requires a list of
options (other than STARTTLS itself), it has to resend the EHLO,
but it is not clear that there is a requirement that it do so
otherwise.

>From my point of view, the first of these two problems is
actually more important than the other in terms of requiring a
clarification in one of the two documents.  If the server thinks
it needs the EHLO domain information after the TLS session has
started, it has no way to tell the client that other that by
issuing a non-fatal "command out of sequence" reply if some
other mail transaction command is issued before the second EHLO
is sent.   In an alternate universe, I could imagine 

   303 Please send EHLO first, then resend command

but I don't imagine that most clients would respond well to that
in our universe.
   
     john

Date: Wed, 28 Jan 2009 16:33:17 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

Hector, what started this was a client that doesn't use any other
extended smtp features beyond STARTTLS. So once it had started up TLS,
it had no use for the output of EHLO, and consequently it didn't bother
sending either HELO or EHLO after the STARTTLS. This was fine until it
ran into a server that demanded that EHLO be sent again. Leading to the
questions on the table: 1) which interpretation of the spec is correct,
and 2) what should be done once this is decided?

	Tony Hansen
	[email-masked]

Hector Santos wrote:
> 
> My take on this:
> 
> I think the client should be aware that most, if not all, servers expect
> a 2nd pass of EHLO, but I also believe the client should not presume
> server features would be the same under TLS and NON-TLS sessions.
> 
> For example, the use case where the private or even public server allows
> a stronger AUTH for non TLS sessions and allows weaker AUTH under TLS:
> 
>    S: 220 blah blah blah
>    C: EHLO client-domain.com
>    S: 250-server-domain.com
>    S: 250 AUTH CRAM-MD5 OTHER
>    S: 250 STARTTLS
>    C: MAIL FROM: <xxxxxxx>
>    S: 530 Must issue a STARTTLS or AUTH command first
>    ...
> 
> Note: both RFC 2487 (TLS) and RFC 2554 (AUTH) use 530 for an enforcement
> requirement. So here, if it client support CRAM or DIGEST it can
> continue with AUTH CRAM-MD5 or AUTH OTHER. But if it does not support
> these stronger AUTH, and does support TLS, it can continue with a STARTLS:
> 
>    ...
>    C: STARTTLS
>    S: 220 Go ahead
>    C & S: <TLS negotiation>
>    C: MAIL FROM: <xxxxxxx>
>    S: 530 AUTH command first
>    C: EHLO client-domain.com
>    S: 250-server-domain.com
>    S: 250 AUTH LOGIN
>    S: 250 AUTH-LOGIN
>    S: AUTH LOGIN
>    S: 235 Authentication successful.
> 
> The point is, the client MUST be aware of this possibility and should
> always (MUST) issue an EHLO after TLS is established.
> 
> The idea of public vs private and level of authentication is discussed
> in 2487 and this is where the 4.2 statement:
> 
>    The client SHOULD send an EHLO command as the
>    first command after a successful TLS negotiation.
> 
> is "forgetful" of its other key points.  So it should of been something
> like:
> 
>    For a SMTP server that is privately referenced, the client
>    MUST issue the EHLO command after a successful TLS negotiation
>    to determine if there are any changes in AUTH methods, such
>    as the strength of the authentication.
> 
>    If the server is the publicly reference, the client SHOULD
>    send an EHLO command as the first command after a successful
>    TLS negotiation, however, it is highly recommended it reissues
>    the EHLO in all cases.
> 
>    The client SHOULD not degrade from EHLO to HELO.
> 
> My take.
> 

Date: Wed, 28 Jan 2009 21:15:35 +0000
Subject: Re: STARTTLS & EHLO
From: Peter Bowyer <[email-masked]>
To: [email-masked]
Sender: [email-masked]

2009/1/28 Paul Smith <[email-masked]>:

> To me, it was (initially) 'clear' that the example saying 'such as the
> argument to the EHLO command', was precise enough to imply that the fact
> that the EHLO command was sent should not be discarded. It could have
> said 'such as the EHLO command', but it went out of its way to say '*the
> argument to* the EHLO command'.

But the 'domain' argument to the EHLO command is mandatory (RFC1869
S4.2). So a server state of having received a valid EHLO but not
knowing what the domain argument is, is not attainable under 1869. I
don't believe 3207's intent is to introduce that state as valid after
STARTTLS.

Peter

-- 
Peter Bowyer
Email: [email-masked]
Follow me on Twitter: twitter.com/peeebeee

Date: Wed, 28 Jan 2009 15:56:36 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Hansen <[email-masked]>
CC:  [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

My take on this:

I think the client should be aware that most, if not all, servers 
expect a 2nd pass of EHLO, but I also believe the client should not 
presume server features would be the same under TLS and NON-TLS sessions.

For example, the use case where the private or even public server 
allows a stronger AUTH for non TLS sessions and allows weaker AUTH 
under TLS:

    S: 220 blah blah blah
    C: EHLO client-domain.com
    S: 250-server-domain.com
    S: 250 AUTH CRAM-MD5 OTHER
    S: 250 STARTTLS
    C: MAIL FROM: <xxxxxxx>
    S: 530 Must issue a STARTTLS or AUTH command first
    ...

Note: both RFC 2487 (TLS) and RFC 2554 (AUTH) use 530 for an 
enforcement requirement. So here, if it client support CRAM or DIGEST 
it can continue with AUTH CRAM-MD5 or AUTH OTHER. But if it does not 
support these stronger AUTH, and does support TLS, it can continue 
with a STARTLS:

    ...
    C: STARTTLS
    S: 220 Go ahead
    C & S: <TLS negotiation>
    C: MAIL FROM: <xxxxxxx>
    S: 530 AUTH command first
    C: EHLO client-domain.com
    S: 250-server-domain.com
    S: 250 AUTH LOGIN
    S: 250 AUTH-LOGIN
    S: AUTH LOGIN
    S: 235 Authentication successful.

The point is, the client MUST be aware of this possibility and should 
always (MUST) issue an EHLO after TLS is established.

The idea of public vs private and level of authentication is discussed 
in 2487 and this is where the 4.2 statement:

    The client SHOULD send an EHLO command as the
    first command after a successful TLS negotiation.

is "forgetful" of its other key points.  So it should of been 
something like:

    For a SMTP server that is privately referenced, the client
    MUST issue the EHLO command after a successful TLS negotiation
    to determine if there are any changes in AUTH methods, such
    as the strength of the authentication.

    If the server is the publicly reference, the client SHOULD
    send an EHLO command as the first command after a successful
    TLS negotiation, however, it is highly recommended it reissues
    the EHLO in all cases.

    The client SHOULD not degrade from EHLO to HELO.

My take.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Tony Hansen wrote:
> Ahhh, there's where the difference in interpretation lays. One
> interpretation is that the remote side is required to forget the value
> that was passed with the original EHLO command. Another interpretation
> is that it further must forget that an EHLO command was issued at all.
> 
> I guess I can see either interpretation of the STARTTLS spec.
> 
> 	Tony Hansen
> 	[email-masked]
> 
> Tony Finch wrote:
>> On Tue, 27 Jan 2009, Paul Smith wrote:
>>> S: 220-main.remotedns.co.uk ESMTP Exim 4.63 #1 Mon, 26 Jan 2009 18:25:48 +0000
>>> S: 220-We do not authorize the use of this system to transport unsolicited,
>>> S: 220 and/or bulk e-mail.
>>> C: EHLO vpop3.company.co.uk
>>> S: 250-main.remotedns.co.uk Hello vpop3.company.co.uk [IP address]
>>> S: 250-SIZE 52428800
>>> S: 250-PIPELINING
>>> S: 250-AUTH PLAIN LOGIN
>>> S: 250-STARTTLS
>>> S: 250 HELP
>>> C: STARTTLS
>>> S: 220 TLS go ahead
>>> <TLS negotiation>
>>> C: MAIL FROM:<[email-masked]>
>>> S: 550 HELO required before MAIL
>>>
>>> (It happens with a few domains, all of which seem to be using Exim (4.63
>>> or 4.69))
>> This is a common but (obviously) non-standard anti-spam check. Practically
>> the only software that doesn't issue HELO or EHLO is malware so the check
>> has a negligible false positive rate. (Malware doesn't use TLS either, so
>> your bug is triggering a slightly over-broad check.)
>>
>>> It certainly looks as if it has forgotten the fact of the EHLO command
>>> once the STARTTLS has happened.
>> As it is required to do.
>>
>> Tony.
> 
> 
> 

Date: Wed, 28 Jan 2009 12:05:10 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Paul Smith <[email-masked]>
CC: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Paul Smith wrote:

>> hmmmmm,  if an incoming client issues
>>
>>    EHLO mail.winserver.com
 >
> So, why would a spammer do that? All they need to do is use a EHLO
> parameter that is valid for that IP address, and all your fancy tests
> are defeated with 10 seconds work by the spammer. 

Paul,

I'm surprise you are suggesting these spoof attempts doesn't exist in 
the real world because of the simplicity or dubious nature. The fact 
is, the frequency of HELO/EHLO spoofing of all sorts is very high.

> If all you are doing is checking that the EHLO name matches an IP
> address (which is all you CAN do), it tells you nothing except that the
> EHLO name matches an IP address. 

Or doesn't match.  Failure Analysis is a major part of our spam system.

> Once they have done that, they can
> trivially send mail from <[email-masked]>, and there's nothing
> stopping them, unless you have SPF or something set up. 

Well, if its a local or hosted domain, we validate the user first 
before allowing the session to continue.  If its a remote domain, then 
its  moot point because the client session must be authenticated at 
which point, most of the spam checking is bypassed (at the SMTP level).

> In which case, how did the EHLO test *really* help? 

To stop the obvious spoofing attempts, which do occur at a very high 
frequency.  I am scratching my head as to why you would be questioning it.

>> which is our domain and its not part of our IP network, its a clear
>> LMAP DOMAIN::IP violation, thus rejectable with 100% no false
>> positions (or true negatives depending on your POV).

> No, what if someone else has the 'winservers.com' domain, and mistyped
> it into their server, or what if you gave your server to someone else,
> and they didn't change it (YOU probably wouldn't do this, but other
> companies might give a server to a sister company for instance), 

Doesn't matter.  Good guys and bad guys are treated the same (from a 
SMTP compliancy and technical protocol expectation standpoint).

What is true about this world is that that Good Guys complain, Bad 
Guys do not.  So if there is a delivery problem because of a 
misconfiguration, eventually, it will get fixed with a simple phone 
call.  We don't expect phone calls from bad guys. :-)

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Wed, 28 Jan 2009 16:32:56 +0000
From: Paul Smith <[email-masked]>
To: Tony Finch <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

Tony Finch wrote:
> As far as I can see it is clear. "The server MUST discard any knowledge
> obtained from the client [...] which was not obtained from the TLS
> negotiation itself. The client MUST discard any knowledge obtained from
> the server [...] which was not obtained from the TLS negotiation itself."
>
> The parts which I have elided are just examples, because they start "such
> as".
>   
It can't be that clear, because myself, Tony Hansen and John Klensin all
saw the other interpretation...

To me, it was (initially) 'clear' that the example saying 'such as the
argument to the EHLO command', was precise enough to imply that the fact
that the EHLO command was sent should not be discarded. It could have
said 'such as the EHLO command', but it went out of its way to say '*the
argument to* the EHLO command'.

To me, the rest of the text seems to be clarifying that the client
should send another EHLO command if it wants to use any extensions,
because the list of extensions may have changed, but if it doesn't want
any extensions, it doesn't matter if it doesn't send another EHLO,
although it's recommended that it does. (ie SHOULD not MUST).

So, I don't think it was clear. I'm not really that fussed which way it
means, I just thought it might need clearing up (one way or the other)
in the next iteration of the document, or an info note on it.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Wed, 28 Jan 2009 14:40:57 +0000
From: Tony Finch <[email-masked]>
To: Tony Hansen <[email-masked]>
cc: [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

On Tue, 27 Jan 2009, Tony Hansen wrote:
>
> Ahhh, there's where the difference in interpretation lays. One
> interpretation is that the remote side is required to forget the value
> that was passed with the original EHLO command. Another interpretation
> is that it further must forget that an EHLO command was issued at all.
>
> I guess I can see either interpretation of the STARTTLS spec.

As far as I can see it is clear. "The server MUST discard any knowledge
obtained from the client [...] which was not obtained from the TLS
negotiation itself. The client MUST discard any knowledge obtained from
the server [...] which was not obtained from the TLS negotiation itself."

The parts which I have elided are just examples, because they start "such
as".

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

From: Alfred =?hp-roman8?B?SM5uZXM=?= <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter -- DNS based checking
To: [email-masked]
Date: Wed, 28 Jan 2009 15:07:07 +0100 (MEZ)
Sender: [email-masked]

Browsing through this thread, it looks like another important
scenario has not been considered yet sufficiently: asymmetric
SMTP routing.

Am analysis of the sub-cases detailed below show that all flavors
of EHLO / DNS based authentication discussed recently will not work
in practise.

Two possible cases:

(1)  company1.big.example  operates distinct inbound and outbound MTAs

   - all outbound mail from within company1 is sent to the public
     Internet by  outbound-mail-gw-{1..n}.company1.big.example ;
     such servers do not accept inbound SMTP/SUBMISSION connections
     from the public Internet;

   - the MX records for company1.big.example point to
     inbound-mail-gw-{1..m}.company1.big.example ;
     such servers never originate mail to the public Internet.

     Typically, all these servers are operated in a DMZ and interface
     to the site-local mail backbone system through a firewall; the
     latter system manages the <local-part> namespace of company1's
     mailbox addresses.

(2)  company2.small.example  operates an on-site MTA behind a NAT,
     but makes use of external (ISP provided) mail services, e.g.
     for spam filtering/deletion, virus scanning, resilience
     against CPE link downtime, etc.

   - all outbound mail from within company2 is sent to the public
     Internet directly from  mail-gw.company2.small.example ;

   - the MX records for company2.small.example point to inbound
     MTAs at the ISP, say  {pri|sec}-mail.isp2.example ;
     after processing and buffering, the remaining messages are
     forwarded from the final processing stage (via forwarding rules)
     to mail-gw.company2.small.example , which (via preconfigured NAT
     mapping and filtering) only accepts inbound SMTP connections
     from these systems.

     Experience has proven that this configuration is particularly
     useful if company2's access link is not a permanent connection,
     e.g. a dialup link, where the customer facing ISP systems have
     to trigger an ISDN callback from the CPE access router for mail
     delivery to company2's site; this configuration may save a lot
     of (dial-up line and traffic volume based) costs by avoiding
     both polling for new mail and delivery of high-rate spam.

     Note: Using non-standard means at the ISP, this may work even
     with dynamic IP addresses for the customers.

In both cases, outbound mail is never originated from the systems
listed in MX records for the customer in the public DNS; inbound
EXPN and VRFY to the outbound MTAs are 'prohibited by architecture'
(no policy needs to be configured); the MTA that "oversees" the
<local-part> of site-local mailboxes is not reachable from the
public Internet.

Kind regards,
  Alfred.

-- 

+------------------------+--------------------------------------------+
| TR-Sys Alfred Hoenes   |  Alfred Hoenes   Dipl.-Math., Dipl.-Phys.  |
| Gerlinger Strasse 12   |  Phone: (+49)7156/9635-0, Fax: -18         |
| D-71254  Ditzingen     |  E-Mail:  [email-masked]                     |
+------------------------+--------------------------------------------+

Date: Wed, 28 Jan 2009 11:45:57 +0000
From: Paul Smith <[email-masked]>
To: Alessandro Vesely <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Alessandro Vesely wrote:
>
> Paul Smith wrote:
>> A bot could use:
>> EHLO fhbdfhbeng.spammer.com
>>
>> where fhbdfhbeng.spammer.com resolves to the IP address of the bot. The
>> spammer can trivially set up a virtual DNS zone with all valid IP
>> addresses in it, and the bot just chooses the appropriate one.
>
> Uh, I may be dumb but I finally got it...
>
> I guess that by "virtual DNS zone" you mean something where "fhbdfhbe"
> is the hex IP address of the bot (possibly obtained via traceroute
> from behind a NAT) and "ng" the bot version or whatever additional
> info is necessary for virtualizing the zone.
Well, "fhbdfhbeng" was actually an impression of my cat walking on the
keyboard (rather than with any deeper significance), but apart from
that, yes. The actual IP -> host name encoding could be anything (making
it harder to spot this trick automatically).
>> Exactly, so how does having a 'correct' EHLO parameter help?
>>
>> I can see that having an incorrect one can be used to block mail, IF
>> (and this is a big 'if') you can be sure that legitimate senders set up
>> things correctly. However, if this becomes a standard check, then it is
>> trivial for a spammer to get around it. And, all that has achieved is
>> another useless check, which makes life harder for the good guys.
>
> Hm... it is useless to install an armored door in a shutterless house,
> and it is also useless to install security shutters since the door
> cannot be locked. Does that analogy fit the status quo?
Sort of, but any EHLO validation is really more like a door with 32
locks, all of whose keys are hanging from a string by the door. It's a
pain for the legitimate person to get in, and a determined burglar could
get in without that much difficulty as well.

Having a normal door, and having basic locks on the windows would be a
better start than having a complex armoured door and leaving the windows
open.
>
> IMHO, if we start designing an armored door, perhaps by the time it
> will be installed those shutters will be underway. I still like VHLO.
>
Not sure what VHLO is, I tried googling for it, and came up with 'Video
Heat Online', which might well be very likeable, but I'm not sure it's
what you meant...

Assuming it's some form of a (decently) 'verifiable helo' then, yes,
that would be better, but you have the backwards compatibility issue,
where bad people just won't use it and will pretend they're using an
SMTP sender from the 1980's. I actually don't have an issue with
breaking backwards compatibility, as long as we get something worthwhile
from it. I just don't think checking EHLO parameters is a good enough
reason.

(I actually think IPv6 mail would be a natural point to break backwards
compatibility and solve a lot of the problems with SMTP, but we've had
that argument on this list before and got nowhere..)

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Wed, 28 Jan 2009 11:42:21 +0100
From: Alessandro Vesely <[email-masked]>
To: Paul Smith <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Paul Smith wrote:
> A bot could use:
> EHLO fhbdfhbeng.spammer.com
> 
> where fhbdfhbeng.spammer.com resolves to the IP address of the bot. The
> spammer can trivially set up a virtual DNS zone with all valid IP
> addresses in it, and the bot just chooses the appropriate one.

Uh, I may be dumb but I finally got it...

I guess that by "virtual DNS zone" you mean something where "fhbdfhbe" 
is the hex IP address of the bot (possibly obtained via traceroute 
from behind a NAT) and "ng" the bot version or whatever additional 
info is necessary for virtualizing the zone.

Apparently, that argument rules out any DNS-based validation.

> Exactly, so how does having a 'correct' EHLO parameter help?
> 
> I can see that having an incorrect one can be used to block mail, IF
> (and this is a big 'if') you can be sure that legitimate senders set up
> things correctly. However, if this becomes a standard check, then it is
> trivial for a spammer to get around it. And, all that has achieved is
> another useless check, which makes life harder for the good guys.

Hm... it is useless to install an armored door in a shutterless house, 
and it is also useless to install security shutters since the door 
cannot be locked. Does that analogy fit the status quo?

IMHO, if we start designing an armored door, perhaps by the time it 
will be installed those shutters will be underway. I still like VHLO.

Date: Wed, 28 Jan 2009 09:31:12 +0000
From: Paul Smith <[email-masked]>
To: Hector Santos <[email-masked]>
CC: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Hector Santos wrote:
>
> Paul Smith wrote:
>
>> Not sure I understand that.
>>
>> It is totally valid to do:
>>
>> EHLO mail.spammer.com
>> MAIL FROM:<[email-masked]>
>>
>> The EHLO name bears no resemblance to the sender's email address. Doing
>> an SPF on the EHLO name is pointless, as all that tells you is that the
>> sending host is 'mail.spammer.com'. 
>
> hmmmmm,  if an incoming client issues
>
>    EHLO mail.winserver.com
So, why would a spammer do that? All they need to do is use a EHLO
parameter that is valid for that IP address, and all your fancy tests
are defeated with 10 seconds work by the spammer. This might be
EHLO bibblebobble.spammer.com
or
EHLO adsl-153-34-125-62.isp.com
or whatever

If all you are doing is checking that the EHLO name matches an IP
address (which is all you CAN do), it tells you nothing except that the
EHLO name matches an IP address. Once they have done that, they can
trivially send mail from <[email-masked]>, and there's nothing
stopping them, unless you have SPF or something set up. In which case,
how did the EHLO test *really* help? The SPF test on its own would have
achieved the same.
>
> which is our domain and its not part of our IP network, its a clear
> LMAP DOMAIN::IP violation, thus rejectable with 100% no false
> positions (or true negatives depending on your POV).
No, what if someone else has the 'winservers.com' domain, and mistyped
it into their server, or what if you gave your server to someone else,
and they didn't change it (YOU probably wouldn't do this, but other
companies might give a server to a sister company for instance), or what
if that was an example in documentation somewhere, and so on. You get
false positives. You only get 100% no false positives if you can be 100%
sure that (a) everyone else knows what they are doing, and (b) no one
else makes a mistake. Good luck with that.

If (as is more likely) a mail server is set up to send EHLO
server.company.local, and you reject it (against RFC 5321) then you will
get lots of false positives, just because a 5 user company with SBS
doesn't understand SMTP.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Wed, 28 Jan 2009 09:33:02 +0000
From: Paul Smith <[email-masked]>
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

Tony Hansen wrote:
> Ahhh, there's where the difference in interpretation lays. One
> interpretation is that the remote side is required to forget the value
> that was passed with the original EHLO command. Another interpretation
> is that it further must forget that an EHLO command was issued at all.
>
> I guess I can see either interpretation of the STARTTLS spec.
>   
Hence my question. I interpreted it the same as you, but it was apparent
that the authors of Exim interpreted it the same as Tony Finch. Maybe
clarification is needed.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Wed, 28 Jan 2009 09:21:08 +0000
From: Paul Smith <[email-masked]>
To: Alessandro Vesely <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Alessandro Vesely wrote:
>
> Paul Smith wrote:
>>> Spammers are welcome to use their own domains: that puts the spam
>>>  problem at the relevant ISPs.
>> Not sure I understand that.
>>
>> It is totally valid to do:
>>
>> EHLO mail.spammer.com
>
> I assume that the EHLO parameter corresponds to the IP address of
> the sending host.
Yes

>> MAIL FROM:<[email-masked]>
>>
>> The EHLO name bears no resemblance to the sender's email address.
>> Doing an SPF on the EHLO name is pointless, as all that tells you is
>> that the sending host is 'mail.spammer.com'.
>
> Yes, and spammer.com is where recipients should complain or claim any
> damage that the transmission might have caused. More likely, the IP of
> that transmitter will be blacklisted soon. I guess that's why spammers
> use zombies or bots.
A bot could use:
EHLO fgbdfhbeng.spammer.com

where fgbdfhbeng.spammer.com resolves to the IP address of the bot. The
spammer can trivially set up a virtual DNS zone with all valid IP
addresses in it, and the bot just chooses the appropriate one.

Complaining to spammer.com won't do any good, and they'll create new
'spammer.com' domains faster than you can block them.

How does it help?

>
>> You have to do the SPF check on the MAIL FROM address, and test it
>>  against the IP address of the sending host.
>
> If the MAIL FROM is given and mycompany took care of setting SPF
> properly, the receiver can reject the message. More often, the MAIL
> FROM address consists of an invalid user at a valid domain without SPF
> records.
Exactly, so how does having a 'correct' EHLO parameter help?

I can see that having an incorrect one can be used to block mail, IF
(and this is a big 'if') you can be sure that legitimate senders set up
things correctly. However, if this becomes a standard check, then it is
trivial for a spammer to get around it. And, all that has achieved is
another useless check, which makes life harder for the good guys.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

To: John C Klensin <[email-masked]>
Cc: Keith Moore <[email-masked]>, John Levine <[email-masked]>, [email-masked]
From: Mark Andrews <[email-masked]>
Subject: Re: Dotless domains 
Date: Wed, 28 Jan 2009 16:39:25 +1100
Sender: [email-masked]

In message <534FD5EDF4F26F5DE55EE826@[192.168.1.118]>, John C Klensin writes:
> --On Monday, January 26, 2009 9:56 AM +1100 Mark Andrews 
> <[email-masked]> wrote:
> 
> > In message <[email-masked]>, John C 
> Klensin writes:
> 
> >> --On Saturday, January 24, 2009 21:02 -0500 Keith Moore
> >> <[email-masked]> wrote:
> 
> >> > IMHO, there should be an RFC explaining that use of a TLD 
> as a
> >> > terminal domain name was never intended to work, and should
> >> > not be expected to be supported by the Internet 
> architecture
> >> > or software.
> >>
> >> You know where to find the virtual pencil.  I would certainly
> >> support such a document, but the real support would need to 
> come
> >> from DNS folks.
> 
> > 	How so?  The DNS is a container to hold information.  The
> > 	rules about that information are external to the DNS and
> > 	always have been.
> 
> Mark,
> 
> I don't want to single you out for abuse because the approach 
> you comment shows is pretty common, but comments like the 
> above, while certainly true, are one of the reasons we see so 
> many uses, and proposed uses, of the DNS that just don't work 
> very well operationally.   Sure, the DNS can be viewed as a 
> container that can be used to store almost any data.  People 
> have discovered that the data of an MX record is more or less 
> just a string and that lots of nonsense can be stored there, 
> including IP addresses and names of aliases.

> In principle, we 
> don't need special provisions for IDNs because, after all, 
> nothing prevents eight-bit data in labels.  And, yes, the root 
> zone is really no different from any other zone and, if folks 
> want to put a million delegations there or, for that matter, 
> install address records so that "http://./" or "ftp ." could, 
> in principle, work, then the DNS, as a data container, doesn't 
> care.

> I think there is agreement that at least some of those things 
> are dumb and would cause problems in practice.  Perhaps you 
> disagree.   But when someone says "this is dumb and there 
> should be a document that explains why it should be prohibited/ 
> avoided" and you (or some other DNS expert) responds by saying 
> "no, the DNS is just a container and doesn't care", those who 
> want to do dumb things take that as approval and start 
> insisting that they should be able to do whatever they want to 
> do and that you (and DNS experts generally) are endorsing it.

	I'm not saying that there shouldn't be a document.  The DNS
	is the wrong place to focus the document.

	I've added lots of checks to named (BIND) to catch all
	sorts of rubbish.  I've also received lots of flac from
	various quarters.

	e.g.
	     check-names (labels are ldh) but even with check-names
	you need to have exceptions like "." which passes as it is
	used as a place holder.
 
	And there is only so much a DNS implementation can do.  LDH
	applies to both alias and hostnames but as CNAMES are a
	general aliasing mechanism in the DNS you can't enforce LDH
	on CNAME owners as you do on A record owners.

	Did 1.2.3.4.example.net in the MX data start out as a address
	or a hostname?

	We do what we can to catch badness.  We can't catch all of
	it however as only the consumer of the data can tell if it
	is bad or not.

        check-integrity <boolean>;
        check-mx ( fail | warn | ignore );
        check-mx-cname ( fail | warn | ignore );
        check-names ( master | slave | response ) ( fail | warn | ignore );
        check-sibling <boolean>;
        check-srv-cname ( fail | warn | ignore );
        check-wildcard <boolean>;

> > 	One of the problems is lots of RFC's say "domain name" when
> > 	they are actually refering to a "hostname".  This has
> > 	introduced lots of confusion.
> 
> One of the problems is that lots of RFCs are ambiguous or 
> contradictory as to whether that distinction exists and all and 
> what it means.  If memory serves, two of them are RFC 1034 and 
> RFC 1123.
> 
> > 	RFC921 made it pretty clear that we were to stop using non-
> > 	heirachical names and that single labels were only ever to
> > 	be used as aliases for heirachical names.
> 
> Sure, but the context of 921 was very different -- not local 
> versus hierarchical but host tables versus domain names. 

	Host table had or was in the process of having hierarchical
	names added as well as unqualified aliases.

> Whether 921 has any particular meaning today is an interesting 
> question.  If you want to cite it as authoritative, you had 
> best convince the IESG to change its status from "UNKNOWN" to 
> "BCP" at least.
> 
> > 	Saying that non-heirachical names only have local scope and
> > 	are to not to be used for global scope addressing would be
> > 	a good thing.  This leaves "localhost" in local scope.
> 
> But, unless I misunderstood Keith's comment, that sort of 
> statement/ restriction is exactly what he was asking for and 
> with which I was trying to agree.  But a narrow reading of 
> either your comment above or some of the statements in RFC 2181 
> would say "that is not a DNS issue"... which some people would 
> promptly interpret as "'localhost.' is a perfectly reasonable 
> FQDN and, in any context in which a string is known to be an 
> FQDN, 'localhost' and 'localhost.' are exactly equivalent".

	RFC 952 says hostnames don't have trailing periods.  They
	are only seperators of labels.

	Mark
 
>     john
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

Date: Tue, 27 Jan 2009 17:12:27 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

Ahhh, there's where the difference in interpretation lays. One
interpretation is that the remote side is required to forget the value
that was passed with the original EHLO command. Another interpretation
is that it further must forget that an EHLO command was issued at all.

I guess I can see either interpretation of the STARTTLS spec.

	Tony Hansen
	[email-masked]

Tony Finch wrote:
> On Tue, 27 Jan 2009, Paul Smith wrote:
>> S: 220-main.remotedns.co.uk ESMTP Exim 4.63 #1 Mon, 26 Jan 2009 18:25:48 +0000
>> S: 220-We do not authorize the use of this system to transport unsolicited,
>> S: 220 and/or bulk e-mail.
>> C: EHLO vpop3.company.co.uk
>> S: 250-main.remotedns.co.uk Hello vpop3.company.co.uk [IP address]
>> S: 250-SIZE 52428800
>> S: 250-PIPELINING
>> S: 250-AUTH PLAIN LOGIN
>> S: 250-STARTTLS
>> S: 250 HELP
>> C: STARTTLS
>> S: 220 TLS go ahead
>> <TLS negotiation>
>> C: MAIL FROM:<[email-masked]>
>> S: 550 HELO required before MAIL
>>
>> (It happens with a few domains, all of which seem to be using Exim (4.63
>> or 4.69))
> 
> This is a common but (obviously) non-standard anti-spam check. Practically
> the only software that doesn't issue HELO or EHLO is malware so the check
> has a negligible false positive rate. (Malware doesn't use TLS either, so
> your bug is triggering a slightly over-broad check.)
> 
>> It certainly looks as if it has forgotten the fact of the EHLO command
>> once the STARTTLS has happened.
> 
> As it is required to do.
> 
> Tony.

Date: Tue, 27 Jan 2009 15:36:40 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Paul Smith <[email-masked]>
CC: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Paul Smith wrote:

> Not sure I understand that.
> 
> It is totally valid to do:
> 
> EHLO mail.spammer.com
> MAIL FROM:<[email-masked]>
> 
> The EHLO name bears no resemblance to the sender's email address. Doing
> an SPF on the EHLO name is pointless, as all that tells you is that the
> sending host is 'mail.spammer.com'. 

hmmmmm,  if an incoming client issues

    EHLO mail.winserver.com

which is our domain and its not part of our IP network, its a clear 
LMAP DOMAIN::IP violation, thus rejectable with 100% no false 
positions (or true negatives depending on your POV).

I would hope other remote systems would help themselves by using the 
expose information we provide for client machine::IP associations.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Tue, 27 Jan 2009 19:37:28 +0100
From: Alessandro Vesely <[email-masked]>
To: Paul Smith <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Paul Smith wrote:
>> Spammers are welcome to use their own domains: that puts the spam
>>  problem at the relevant ISPs.
> Not sure I understand that.
> 
> It is totally valid to do:
> 
> EHLO mail.spammer.com

I assume that the EHLO parameter corresponds to the IP address of
the sending host.

> MAIL FROM:<[email-masked]>
> 
> The EHLO name bears no resemblance to the sender's email address. 
> Doing an SPF on the EHLO name is pointless, as all that tells you 
> is that the sending host is 'mail.spammer.com'.

Yes, and spammer.com is where recipients should complain or claim any
damage that the transmission might have caused. More likely, the IP of
that transmitter will be blacklisted soon. I guess that's why spammers
use zombies or bots.

> You have to do the SPF check on the MAIL FROM address, and test it
>  against the IP address of the sending host.

If the MAIL FROM is given and mycompany took care of setting SPF
properly, the receiver can reject the message. More often, the MAIL
FROM address consists of an invalid user at a valid domain without SPF
records.

Date: Tue, 27 Jan 2009 17:41:57 +0000
From: Paul Smith <[email-masked]>
To: Alessandro Vesely <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Alessandro Vesely wrote:
>
>> All you can do is check to see if the EHLO parameter resolves to the
>> IP address of the sending host, and that tells you nothing except the
>> sender has set it up correctly... Spammers can set up domains &
>> mailing software correctly more easily than the majority of
>> legitimate users can.
>
> Hm... some spamware can obviously do a reverse lookup and use that as
> a helo name. However, it cannot easily fake MX or SPF records to make
> a zombie address valid. Spammers are welcome to use their own domains:
> that puts the spam problem at the relevant ISPs.
Not sure I understand that.

It is totally valid to do:

EHLO mail.spammer.com
MAIL FROM:<[email-masked]>

The EHLO name bears no resemblance to the sender's email address. Doing
an SPF on the EHLO name is pointless, as all that tells you is that the
sending host is 'mail.spammer.com'. You have to do the SPF check on the
MAIL FROM address, and test it against the IP address of the sending host.

ISPs do this all the time (legitimately).

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Tue, 27 Jan 2009 18:18:51 +0100
From: Alessandro Vesely <[email-masked]>
To: Paul Smith <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Paul Smith wrote:
> The HELO/EHLO parameter means nothing.

Almost agreed.

> You can't possibly do an SPF check on the EHLO parameter, and 
> expect it to mean anything.

The HELO check was introduced for the case that the MAIL FROM is null. 
However, some people thinks the HELO check is better, as it allows 
naive forwarding.

> All you can do is check to see if the EHLO parameter resolves to 
> the IP address of the sending host, and that tells you nothing 
> except the sender has set it up correctly... Spammers can set up 
> domains & mailing software correctly more easily than the majority 
> of legitimate users can.

Hm... some spamware can obviously do a reverse lookup and use that as 
a helo name. However, it cannot easily fake MX or SPF records to make 
a zombie address valid. Spammers are welcome to use their own domains: 
that puts the spam problem at the relevant ISPs.

Date: Tue, 27 Jan 2009 10:08:45 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Paul Smith <[email-masked]>
CC: Alex van den Bogaerdt <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Paul Smith wrote:
> Alex van den Bogaerdt wrote:
>>> I would argue that 'EHLO
>>> mailserver.domain.com' is more useful than 'EHLO [192.168.1.1]' even
>>> though the former is incorrect, and the latter correct (according to RFC
>>> 5321) with dynamic IP/NAT.
>> IMHO 192.168.1.1 is not an internet IP address. And that is a
>> requirement.
>>
> I'm sorry, but that can't be the case.
> 
> Otherwise, how does Thunderbird send mail to a local mail server on a
> private network? It can't, because it hasn't got an Internet host name,
> or an Internet IP address, so can't issue a valid EHLO command.

TBIRD fixed this problem in 2.01 I believe. I submitted the Bug 
request to add a user defined method to hard code the machine name or 
address  where the NAT is located the a private LAN.

Click:

   TOOLS | OPTIONS | GENERAL TAB | CONFIG EDITOR

I forget, either its there or you have to add it for each outgoing account

  mail.smtpserver.smtp1.hello_argument     [fixed NAT ip address]
  mail.smtpserver.smtp2.hello_argument     [fixed NAT ip address]
  mail.smtpserver.smtp3.hello_argument     [fixed NAT ip address]
  ....
  mail.smtpserver.smtpX.hello_argument     [fixed NAT ip address]

If set, the TBIRD MTA will use this over the local machine IP.

> Similarly, how does ANY SMTP client behind a NAT router with dynamic IP
> (ie pretty much any home user) send a message anywhere? If what you say
> is the case, then that is cutting off half the world from the SMTP network!

Another way to solve this is where EHLO checking should be delay until 
other authentication methods may have been established, e.g.; ESMTP 
AUTH, POP B4 SMTP, etc.

This was something I proposed to John and Randy for 4409 (Submit) 
where since AUTH is requirement, the strong EHLO checking can be relaxed.

> As far as I am aware, RFC 4409 doesn't relax the requirements of RFC
> 2821/5321 for the EHLO parameter, so 'use message submission' is not the
> answer.

Right, I brought this up to John, I think it was early last year or 2 
when I was helping the Mozilla team address the "Private address 
across a NAT" TBIRD issue.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

To: John C Klensin <[email-masked]>
Cc: Paul Smith <[email-masked]>, Tony Finch <[email-masked]>, David MacQuigg <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
From: Mark Andrews <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter 
Date: Wed, 28 Jan 2009 01:58:40 +1100
Sender: [email-masked]

In message <EBB53940F75E648CFD9173FE@[192.168.1.118]>, John C Klensin writes:
> 
> Using this as a back door to get rid of SMTP clients that send
> mail from private address space directly into public Internet
> address space is a different problem.  It just isn't the way to
> do that, independent of how desirable one might believe it to
> be.  Even its desirability is in doubt give IPv4 address space
> shortages and the belief that we will be seeing ever more of
> those addresses in the coming years.  If you want to pursue that
> option, please get the RIRs and the various network operations
> groups to sign off on it first.

	It will be in more doubt when you throw in NAT64 and NAT46
	as part of a IPv6 transition plan.

> I think I've now said about as much as I usefully can on this
> topic.
> 
>      john
> 
> 
> 
> 
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

Date: Tue, 27 Jan 2009 14:00:16 +0000
From: Paul Smith <[email-masked]>
To: Alex van den Bogaerdt <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Alex van den Bogaerdt wrote:
>
>> I would argue that 'EHLO
>> mailserver.domain.com' is more useful than 'EHLO [192.168.1.1]' even
>> though the former is incorrect, and the latter correct (according to RFC
>> 5321) with dynamic IP/NAT.
>
> IMHO 192.168.1.1 is not an internet IP address. And that is a
> requirement.
>
I'm sorry, but that can't be the case.

Otherwise, how does Thunderbird send mail to a local mail server on a
private network? It can't, because it hasn't got an Internet host name,
or an Internet IP address, so can't issue a valid EHLO command.
Similarly, how does ANY SMTP client behind a NAT router with dynamic IP
(ie pretty much any home user) send a message anywhere? If what you say
is the case, then that is cutting off half the world from the SMTP network!

As far as I can see RFC 5321 doesn't say it has to be a routable IP
address to be used in an address literal.

As far as I am aware, RFC 4409 doesn't relax the requirements of RFC
2821/5321 for the EHLO parameter, so 'use message submission' is not the
answer.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Tue, 27 Jan 2009 14:08:30 +0000
From: Tony Finch <[email-masked]>
To: Paul Smith <[email-masked]>
cc: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

On Tue, 27 Jan 2009, Paul Smith wrote:
>
> S: 220-main.remotedns.co.uk ESMTP Exim 4.63 #1 Mon, 26 Jan 2009 18:25:48 +0000
> S: 220-We do not authorize the use of this system to transport unsolicited,
> S: 220 and/or bulk e-mail.
> C: EHLO vpop3.company.co.uk
> S: 250-main.remotedns.co.uk Hello vpop3.company.co.uk [IP address]
> S: 250-SIZE 52428800
> S: 250-PIPELINING
> S: 250-AUTH PLAIN LOGIN
> S: 250-STARTTLS
> S: 250 HELP
> C: STARTTLS
> S: 220 TLS go ahead
> <TLS negotiation>
> C: MAIL FROM:<[email-masked]>
> S: 550 HELO required before MAIL
>
> (It happens with a few domains, all of which seem to be using Exim (4.63
> or 4.69))

This is a common but (obviously) non-standard anti-spam check. Practically
the only software that doesn't issue HELO or EHLO is malware so the check
has a negligible false positive rate. (Malware doesn't use TLS either, so
your bug is triggering a slightly over-broad check.)

> It certainly looks as if it has forgotten the fact of the EHLO command
> once the STARTTLS has happened.

As it is required to do.

Tony.
-- 
<[email-masked]>   <[email-masked]>   http://dotat.at/   ${sg{\N${sg{\
N\}{([^N]*)(.)(.)(.*)}{\$1\$3\$2\$1\$3\n\$2\$3\$4\$3\n\$3\$2\$4}}\
\N}{([^N]*)(.)(.)(.*)}{\$1\$3\$2\$1\$3\n\$2\$3\$4\$3\n\$3\$2\$4}}

Date: Tue, 27 Jan 2009 09:00:20 -0500
From: John C Klensin <[email-masked]>
To: Paul Smith <[email-masked]>, Tony Finch <[email-masked]>
cc: David MacQuigg <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

--On Tuesday, January 27, 2009 11:37 AM +0000 Paul Smith
<[email-masked]> wrote:

> The only 'correct' ways I can come up with are to use the
> internal IP address literal, or (if the SBS owner is a techie)
> to have something like 'company.dyndns.org'. The first of
> these is totally useless to anything else, and the second is
> arguably less useful than the technically incorrect
> 'company.com' or 'mail.company.com' (because there is no DNS
> record pointing those names back at the dynamic IP address)

One could also put in a record that looks like
   mail.company.com. IN CNAME company.dyndns.org.

But, while it gives more information than the Dyndns address, an
EHLO argument using the "mail.company.com" address isn't
5321-valid because that is not the primary host name.

I don't remember the details of the discussion that led to the
prohibition on rejection in RFC 1123, but the issues more
generally are very much like the ones the Paul raises (and that
I tried, less successfully, to identify in an earlier note).
The historical goal of EHLO is to provide the maximum amount of
information possible to help the server identifying the client
host and its administration.    Simply delivering the public IP
address of that host is actually fairly useless from an added
information point of view -- the server already knows that --
and, from that point of view, the permission in RFC 2821 to use
an IP address literal doesn't add much.   That "as much
information as possible" principle was also, I think, what
motivated DRUMS to permit, in 2821, a host-identifying comment
after the EHLO domain.  We took that back out of  5321 because
it was not widely implemented and causes problems, but the
principle was probably still right.

In that context, this discussion is ultimately about a tradeoff
between "get the Good Guys to provide maximum information" and
"see if we can use the EHLO argument to trap the Bad Guys".  The
decision to favor the first was made with RFC 1123 twenty years
ago and even that was arguably just a clarification to the
intent of RFC 821.  Tempting as it might be from some
perspectives, trying to change the semantics of that argument to
help in spam-rejection just isn't going to work, at least IMO.
There is too large an installed base of clients designed around
"supply maximum useful information" and one doesn't want to
discourage the Good Guys from supplying useful information (even
if it is a valid IP address in private space).   In addition,
even if one could insist on a domain name that "matches" the
public IP address and deploy servers widely that enforced that
provision, one would only temporarily inconvenience the spammer
and then, in the words of Paul Vixie and others, make them
smarter.  It really isn't all that hard to figure out the public
IP address associated with the private one that one is using (at
least if there is only one level of NAT involved).   It is
certainly not hard to take that public address and reverse-map
it to obtain a domain name and, if there is no reverse-mapping,
not much harder to use a temporary domain and dynamic dns to
make a name that will pass any test made at SMTP connect time.
It might take the spammers a while to deploy software,
especially bot software, that would do those things, but, once
they did, one would have another nearly-useless anti-spam tool
and a dramatically weakened debugging environment for more
legitimate and desired messages.

Using this as a back door to get rid of SMTP clients that send
mail from private address space directly into public Internet
address space is a different problem.  It just isn't the way to
do that, independent of how desirable one might believe it to
be.  Even its desirability is in doubt give IPv4 address space
shortages and the belief that we will be seeing ever more of
those addresses in the coming years.  If you want to pursue that
option, please get the RIRs and the various network operations
groups to sign off on it first.

I think I've now said about as much as I usefully can on this
topic.

     john

Date: Tue, 27 Jan 2009 08:53:57 -0500
From: John Leslie <[email-masked]>
To: David MacQuigg <[email-masked]>
Cc: [email-masked]
Subject: Re: Submission identifiers
Sender: [email-masked]

David MacQuigg <[email-masked]> wrote:
> 
> We've narrowed the question to just whether SMTP should drop the
> requirement that receivers MUST NOT reject apparently forged helonames.

   I disagree that any such requirement exists: the prohibition is much
narrower than that.

   For nearly five years I have been rejecting _clearly_ forged EHLOs
according to:

http://www.bbiw.net/CSV/draft-ietf-marid-csv-intro-01.html

which states:

- Extract the domain name from the EHLO command sent by the SMTP client

- Query DNS for a SRV record under the EHLO domain name (see
  [ID-CSVCSA])

- Check the SRV reply for flags returned, and check for a match in the
  list of returned IP addresses

   Granted, almost the only thing I actually reject is EHLO jlc.net...

   I claimed that was allowed under 2821; and I believe there was
general agreement that rejecting clearly forged EHLO is allowed under
5321. If there isn't such general agreement, perhaps we should add a
sentence saying it's OK to reject a session if the EHLO is clearly
forged.

   But I can't agree with David's proposed language...

> The suggested language for a change was:
> 
>   An SMTP server MAY reject a mail session if a domain name argument in
>   the EHLO command does not actually correspond to the IP address of the
>   client.  Clients that are unable to establish this correspondence MUST
>   NOT provide an invalid domain name argument.

   There's a big difference between applying a crisp statement of which
IP addresses are (currently) used by authorized MTAs, versus guessing
that A)ddress records returned on a DNS query give the list of addresses
of authorized MTAs.

> The claimed benefit was that a currently useless and frequently abused
> parameter, the heloname, could then become useful in rejecting forgeries,

   I'm using it for that right now.

> and that would motivate legitimate senders to take it seriously.

   I can't report much movement in that direction...

> This in turn could lead to large numbers of receivers making good use
> of the heloname, whitelisting legitimate domains, etc.

   I'd love to see that. Perhaps David would like to resurrect CSV to
some kind of standards status and see if that makes a difference?

> The claimed reason for not making this change was that it would be too
> hard on some small fraction of legitimate senders. Three cases were
> presented.
> 
> 1) The small office with a dynamically-assigned IP address.
> 2) The roaming laptop.
> 3) The digital camera.

   (Don't forget that any change to the _base_ spec requires compatibility
with existing usage.)

> The solutions considered too difficult were:
> 
> 1) Get a static IP address for the transmitter in the small office.

   Not really necessary: there are dynamic-DNS services to cause a
dynamically-assigned IP address to be returned by the DNS server.

> 2) Relay through a transmitter with an established identity and
>    reputation.

   Aggregating reputation inevitably makes reputation less crisp.

> 3) Authorize the entire block of addresses that might be dynamically
>    assigned to the transmitter.

   ... exposing you to the risk of bad players anywhere in the block...

> 4) Use an address literal, meaning - please accept this session without
>    a HELO ID.

   I suppose you could always do "EHLO [127.0.0.1]", but why bother?

> Have I missed anything?

   Dave has missed the poor correlation between A)ddress records and
intent to authorize particular MTAs.

   He's also missed cases where the IP address might actually change
_during_ a SMTP session (but we've all missed that, hoping it will be
hidden from us).

   And, most important, he's missed the need for compatibility with
existing usage.

   But I quite agree with him that an EHLO string _should_ be useful
for something. We just need to define ways to do so. CSV could be one.
Or we could define an extension to allow the SMTP client to send an
encrypted authorization. Or something else I haven't thought of.

   It's just that we shouldn't try to squeeze this into the base spec.

--
John Leslie <[email-masked]>

Date: Tue, 27 Jan 2009 08:00:04 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC:  [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Hi John,

John C Klensin wrote:
> Hi.
> 
> Question: Is it time to formally deprecate 821 and, in
> particular, the main feature that distinguishes it: the use of
> HELO by SMTP clients?  

I don't see a problem with adding new semantics that effectively 
deprecates HELO.

> We would still need to require that SMTP
> servers accept it, but we would tell full-capability clients
> (including the client side of relays and gateways) that HELO is
> obsolete.  

Both established servers and client will continue to support HELO for 
their customer public server operations. I don't see us removing HELO 
support. Even future developers will come across a server where 
delivery is a problem and arguing about what is right or wrong is a 
waste of time for everyone.

But overall, I see the goal is to begin instilling stronger SMTP 
compliancy protocol specification for future developers (servers and 
clients) by minimizing, removing, rewording all currently remaining 
ambiguous and relaxed SMTP previsions.

> One corollary of this is that we'd be telling
> low-capability clients, particularly those that are part of MUA
> systems, that they should be talking to Submit ports, not SMTP
> ones.

You mean private vs public? and thus, private operations authorizes 
receivers to enforce expected strong behaviors.

As long as SUBMIT requires authentication, does it not? I don't see 
how we will ever get rid of the public port.

I think we should discuss the technical problem at hand like beginning 
with questions like:

1) Why is helo still used?

    - No need for extensions
    - smaller session handshaking
    - legacy software, good vs bad.

2) Where is it a problem when used?

    - bypasses strong machine authentication extended methods

3) Why can't we allow servers to optionally enforce the same
    syntax requirements as EHLO? Over a migration period?

4) Are there any reason technical migration policy strategies?

I ask this because if we think about it, there is really subtle 
differences between EHLO and HELO and its mostly in its syntax and 
strong vs weak enforcement EHLO overs HELO.

Anyway, I don't think the angle of SPAM fighting is the solution. The 
solution has to be applied to all to close all loopholes.  Any new 
enforcement has to apply to the good guy as well.

> I don't have a strong opinion (or even a weak one) about this
> one way or the other, nor have I looked at how much I'd have to
> change the document to implement it if it were wanted.  But,
> because this is the point at which we are proposing to really
> obsolete Full Standard 821 with a Full Standard, it seems like
> the right time to ask the question.   Since RFC 1425 was
> published sixteen years ago next month, it should not be
> unreasonable to tell people they've had enough time to get used
> to the idea of something besides HELO.

+1, but you do realize that business aren't going to give a hoot about 
the new rules when there are delivery problems, even if rare and 
remote.  If some legacy server says "500 Command not understood", it 
would be silly for the worlds most advanced and modern SMTP client to 
not send the mail because the receiver is not supporting EHLO.

 From an Spam/Security standpoint, the difficulty in maximizing new 
technology effectiveness has been the requirement to continue to 
support legacy clients.  Its the Archille's Heel of all the current 
work being done.  The client does not have to comply by simply staying 
put in legacy mode.

I would suggest a I-D would where we might introduce a "Migration 
Protocol Extension" where servers have the option to reject HELO after 
a certain period and warning was issued to the legacy client.

We might even wish to borrow old mail networking policies that worked 
in the past to address continued legacy interoperability issues. Such 
as only allowing legacy SMTP clients to move mail (operate) during 
certain times of the day.  In a way, it would be like a greylist 
concept for HELO legacy clients.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Reply-To: "Alex van den Bogaerdt" <[email-masked]>
From: "Alex van den Bogaerdt" <[email-masked]>
To: <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Date: Tue, 27 Jan 2009 14:16:38 +0100
Sender: [email-masked]

> I would argue that 'EHLO
> mailserver.domain.com' is more useful than 'EHLO [192.168.1.1]' even
> though the former is incorrect, and the latter correct (according to RFC
> 5321) with dynamic IP/NAT.

IMHO 192.168.1.1 is not an internet IP address. And that is a requirement.

Date: Tue, 27 Jan 2009 12:33:18 +0000
From: Paul Smith <[email-masked]>
To: Tony Finch <[email-masked]>
CC: John C Klensin <[email-masked]>, David MacQuigg <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------060505070703090400080407
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Tony Finch wrote:
> On Tue, 27 Jan 2009, Paul Smith wrote:
>
>   
>> So, how would YOU set it up, given the (common) situation I described
>> (SBS Exchange connecting through a NAT router on a dynamic IP address)?
>>     
>
> Sorry, I missed the dynamic part. They'll probably be blacklisted for
> reasons entirely unrelated to their EHLO domain.
>   
Only on a relative few recipient domains (surprisingly). For all the
rest, you have to get by with using what would be an incorrect EHLO
parameter.

Anyway, if that is the 'answer', shouldn't SMTP be defined like that? If
you are saying 'you need a static IP address to send mail without going
through a submission server', shouldn't that be in the SMTP standard?
Currently SMTP doesn't require that (some recipients do, but that's
their policy decision), so you can't change other parts of SMTP to get
that requirement in by the back door.

One explicit reason for the 'address literal' option for EHLO is to
allow for dynamic IP addresses (see RFC5321 4.1.1.1), but its
information is useless with a NAT setup, which most people will have.
Note that RFC 5321 does not mention NAT.

If you want to enforce strict EHLO parameter checking in the standard,
that is a big change, which would break a large number of existing mail
setups, (and may have big consequences on the availability of IPv4
addresses...). If RFC5321 was strictly applied at the moment, a large
number of EHLO statements would have to be 'EHLO [192.168.x.y]', and
good luck with using that for any checking... I would argue that 'EHLO
mailserver.domain.com' is more useful than 'EHLO [192.168.1.1]' even
though the former is incorrect, and the latter correct (according to RFC
5321) with dynamic IP/NAT.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

--------------060505070703090400080407
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Tony Finch wrote:
<blockquote
 cite="mid:[email-masked]"
 type="cite">
  <pre wrap="">On Tue, 27 Jan 2009, Paul Smith wrote:

  </pre>
  <blockquote type="cite">
    <pre wrap="">So, how would YOU set it up, given the (common) situation I described
(SBS Exchange connecting through a NAT router on a dynamic IP address)?
    </pre>
  </blockquote>
  <pre wrap=""><!---->
Sorry, I missed the dynamic part. They'll probably be blacklisted for
reasons entirely unrelated to their EHLO domain.
  </pre>
</blockquote>
Only on a relative few recipient domains (surprisingly). For all the
rest, you have to get by with using what would be an incorrect EHLO
parameter.<br>
<br>
Anyway, if that is the 'answer', shouldn't SMTP be defined like that?
If you are saying 'you need a static IP address to send mail without
going through a submission server', shouldn't that be in the SMTP
standard? Currently SMTP doesn't require that (some recipients do, but
that's their policy decision), so you can't change other parts of SMTP
to get that requirement in by the back door.<br>
<br>
One explicit reason for the 'address literal' option for EHLO is to
allow for dynamic IP addresses (see RFC5321 4.1.1.1), but its
information is useless with a NAT setup, which most people will have.
Note that RFC 5321 does not mention NAT.<br>
<br>
If you want to enforce strict EHLO parameter checking in the standard,
that is a big change, which would break a large number of existing mail
setups, (and may have big consequences on the availability of IPv4
addresses...). If RFC5321 was strictly applied at the moment, a large
number of EHLO statements would have to be 'EHLO [192.168.x.y]', and
good luck with using that for any checking... I would argue that 'EHLO
mailserver.domain.com' is more useful than 'EHLO [192.168.1.1]' even
though the former is incorrect, and the latter correct (according to
RFC 5321) with dynamic IP/NAT.<br>
<br>
<br>
<pre class="moz-signature" cols="72">-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows</pre>
</body>
</html>

--------------060505070703090400080407--

Date: Tue, 27 Jan 2009 11:44:23 +0000
From: Tony Finch <[email-masked]>
To: Paul Smith <[email-masked]>
cc: John C Klensin <[email-masked]>, David MacQuigg <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

On Tue, 27 Jan 2009, Paul Smith wrote:

> So, how would YOU set it up, given the (common) situation I described
> (SBS Exchange connecting through a NAT router on a dynamic IP address)?

Sorry, I missed the dynamic part. They'll probably be blacklisted for
reasons entirely unrelated to their EHLO domain.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

Date: Tue, 27 Jan 2009 11:37:40 +0000
From: Paul Smith <[email-masked]>
To: Tony Finch <[email-masked]>
CC: John C Klensin <[email-masked]>, David MacQuigg <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------080809080303010300040006
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Tony Finch wrote:
>> Under the 'current' way yes, but if it is changed so that the EHLO
>> parameter must correspond to the IP address of the sending host, then it
>> won't work, and I can't think of anything which would make it work.
>>     
>
> If the sender's EHLO host name isn't correctly set up in the DNS then the
> postmaster has set up SBS wrong.
>   
So, how would YOU set it up, given the (common) situation I described
(SBS Exchange connecting through a NAT router on a dynamic IP address)?

The only 'correct' ways I can come up with are to use the internal IP
address literal, or (if the SBS owner is a techie) to have something
like 'company.dyndns.org'. The first of these is totally useless to
anything else, and the second is arguably less useful than the
technically incorrect 'company.com' or 'mail.company.com' (because there
is no DNS record pointing those names back at the dynamic IP address)

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

--------------080809080303010300040006
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Tony Finch wrote:
<blockquote
 cite="mid:[email-masked]"
 type="cite">
  <blockquote type="cite">
    <pre wrap="">Under the 'current' way yes, but if it is changed so that the EHLO
parameter must correspond to the IP address of the sending host, then it
won't work, and I can't think of anything which would make it work.
    </pre>
  </blockquote>
  <pre wrap=""><!---->
If the sender's EHLO host name isn't correctly set up in the DNS then the
postmaster has set up SBS wrong.
  </pre>
</blockquote>
So, how would YOU set it up, given the (common) situation I described
(SBS Exchange connecting through a NAT router on a dynamic IP address)?<br>
<br>
The only 'correct' ways I can come up with are to use the internal IP
address literal, or (if the SBS owner is a techie) to have something
like 'company.dyndns.org'. The first of these is totally useless to
anything else, and the second is arguably less useful than the
technically incorrect 'company.com' or 'mail.company.com' (because
there is no DNS record pointing those names back at the dynamic IP
address)<br>
<br>
<br>
<pre class="moz-signature" cols="72">-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows</pre>
</body>
</html>

--------------080809080303010300040006--

Date: Tue, 27 Jan 2009 11:31:42 +0000
From: Tony Finch <[email-masked]>
To: Paul Smith <[email-masked]>
cc: John C Klensin <[email-masked]>, David MacQuigg <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

On Tue, 27 Jan 2009, Paul Smith wrote:
> Tony Finch wrote:
> > On Mon, 26 Jan 2009, Paul Smith wrote:
> >
> >> There is a small business who has bought Microsoft SBS. What are they
> >> supposed to use for the EHLO parameter if it is important enough to mean
> >> the difference between delivery or not?
> >
> > If they have SBS set up correctly then it should just work.
>
> Under the 'current' way yes, but if it is changed so that the EHLO
> parameter must correspond to the IP address of the sending host, then it
> won't work, and I can't think of anything which would make it work.

If the sender's EHLO host name isn't correctly set up in the DNS then the
postmaster has set up SBS wrong.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

Date: Tue, 27 Jan 2009 11:07:45 +0000
From: Paul Smith <[email-masked]>
To: Alessandro Vesely <[email-masked]>
CC: Tony Finch <[email-masked]>, John C Klensin <[email-masked]>, David MacQuigg <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Alessandro Vesely wrote:
> Paul Smith wrote:
>>> If they have SBS set up correctly then it should just work. 
>>
>> Under the 'current' way yes, but if it is changed so that the EHLO
>> parameter must correspond to the IP address of the sending host, then
>> it won't work, and I can't think of anything which would make it work.
>
> May I ask how can that server receive any responses to the mail it sends?
You must have missed the bit where I said it uses the Exchange POP3
connector to collect mail from the ISP's server.

>
>> It would HAVE to go via a third party smarthost, many of these are
>> unreliable and/or slow, so people prefer not to use them if they can.
> It is not possible to define a possible set up, until the 'next' HELO
> parameter has a definitive wording. The general tendency is to use the
> DNS for storing that information, i.e. operative records (A, MX) if at
> all possible, declarative records (SPF, TXT) otherwise. Assuming that,
> dynamic DNS is the only way, unless the dynamically assigned address
> varies within a reasonably small pool that is established beforehand.
What I am cautious about is making changes to SMTP which will make
sending for legitimate senders more difficult without major improvements
to stopping bad senders. Changing the HELO parameter to add new
requirements to it just seems to make it more difficult for legitimate
senders without doing anything to stop bad senders.

The HELO/EHLO parameter means nothing. It has no relationship to the
MAIL FROM domain, or to anything the recipient of the message sees. You
can't possibly do an SPF check on the EHLO parameter, and expect it to
mean anything. All you can do is check to see if the EHLO parameter
resolves to the IP address of the sending host, and that tells you
nothing except the sender has set it up correctly... Spammers can set up
domains & mailing software correctly more easily than the majority of
legitimate users can.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Tue, 27 Jan 2009 11:35:10 +0100
From: Alessandro Vesely <[email-masked]>
To: Paul Smith <[email-masked]>
CC: [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

Paul Smith wrote:
> "  The client SHOULD send an EHLO command as the
>    first command after a successful TLS negotiation."
> 
> 
> (SHOULD, not MUST).

EHLO is not mandatory. One may want to use, say, secure VRFY. No?

Date: Tue, 27 Jan 2009 11:24:42 +0100
From: Alessandro Vesely <[email-masked]>
To: Paul Smith <[email-masked]>
CC: Tony Finch <[email-masked]>, John C Klensin <[email-masked]>, David MacQuigg <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Paul Smith wrote:
>> If they have SBS set up correctly then it should just work. 
> 
> Under the 'current' way yes, but if it is changed so that the EHLO 
> parameter must correspond to the IP address of the sending host, then it 
> won't work, and I can't think of anything which would make it work.

May I ask how can that server receive any responses to the mail it 
sends? I don't want to mean that ingoing and outgoing paths should be 
symmetric, as in many cases asymmetric techniques can provide more 
performance. However, I don't think that we want to encourage the 
habit of flooding recipients with spam in such a way that no answer or 
complaint can ever get back. That is to say, IMHO, boolean in and out 
capabilities should be symmetric.

> It would HAVE to go via a third party smarthost, many of these are 
> unreliable and/or slow, so people prefer not to use them if they can.

It is not possible to define a possible set up, until the 'next' HELO 
parameter has a definitive wording. The general tendency is to use the 
DNS for storing that information, i.e. operative records (A, MX) if at 
all possible, declarative records (SPF, TXT) otherwise. Assuming that, 
dynamic DNS is the only way, unless the dynamically assigned address 
varies within a reasonably small pool that is established beforehand.

Date: Tue, 27 Jan 2009 09:46:14 +0000
From: Paul Smith <[email-masked]>
To: John C Klensin <[email-masked]>
CC: David MacQuigg <[email-masked]>, [email-masked]
Subject: Re: Submission identifiers
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------050607040804050705040207
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

John C Klensin wrote:
>> The claimed reason for not making this change was that it
>> would be too hard on some small fraction of legitimate
>> senders.  Three cases were presented. 1) The small office with
>> a dynamically-assigned IP address. 2) The roaming laptop.
>> 3) The digital camera.
>>
>> The solutions considered too difficult were:
>> 1) Get a static IP address for the transmitter in the small
>> office. 2) Relay through a transmitter with an established
>> identity and reputation. 3) Authorize the entire block of
>> addresses that might be dynamically assigned to the
>> transmitter. 4) Use an address literal, meaning - please
>> accept this session without a HELO ID.
>>
>> Have I missed anything?
>>     
>
> I don't think so, as long as it is clearly understood that this
> does two things:
>
> (i) Defines a domain name argument that does not match the
> public Internet address of the sender as "invalid" --  thereby
> de facto preventing most SMTP clients that are using private
> address space behind at NAT from sending mail -- even if that
> domain name conveys accurate information about the sending
> system.
>
> (ii) Forces people toward use of IP literals, even IP literals
> in private address space (i.e., that convey little information),
> when domain names might make the sender more identifiable and
> more easily contacted.
>
>   
I agree with John here.

Either, everyone will useless IP literals, or this change will *force*
many small businesses to use either their ISP's poor quality smarthosts,
or pay extra for a decent quality smarthost (and probably push them
further towards giving up email).

Requiring people to have a static IP address is doomed to failure, as
(a) IPv4 address space is running out, so ISPs will be pushing towards
dynamic addresses, and (b) many ISPs already don't allow static IP
addresses.

I don't see it giving any benefit, since 'bad senders' already know
SMTP/domains well enough to be able to make their mail look legitimate
enough to get through a crude filter like this. (eg - A spammer could
trivially create their own domain 'spammer.com' and "create" a virtual
zone of 4 billion entries (one for each IP address) and the spambots
will use the relevant entry from that domain as their EHLO data. Instant
'pass' of any EHLO checking based on correspondence of EHLO to IP address)

IMV, arguments against SPF or DKIM pale into insignificance before the
problems of this proposed change.

Something like SPF can give a big benefit, while only requiring people
to use an authorised submission server. This requires far more people to
use a submission server, and only gives a very dubious benefit.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

--------------050607040804050705040207
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
John C Klensin wrote:
<blockquote cite="mid:96A2116FE0FB138CA8DFD66B@%5B192.168.1.118%5D"
 type="cite">
  <blockquote type="cite">
    <pre wrap="">The claimed reason for not making this change was that it
would be too hard on some small fraction of legitimate
senders.  Three cases were presented. 1) The small office with
a dynamically-assigned IP address. 2) The roaming laptop.
3) The digital camera.

The solutions considered too difficult were:
1) Get a static IP address for the transmitter in the small
office. 2) Relay through a transmitter with an established
identity and reputation. 3) Authorize the entire block of
addresses that might be dynamically assigned to the
transmitter. 4) Use an address literal, meaning - please
accept this session without a HELO ID.

Have I missed anything?
    </pre>
  </blockquote>
  <pre wrap=""><!---->
I don't think so, as long as it is clearly understood that this
does two things:

(i) Defines a domain name argument that does not match the
public Internet address of the sender as "invalid" --  thereby
de facto preventing most SMTP clients that are using private
address space behind at NAT from sending mail -- even if that
domain name conveys accurate information about the sending
system.

(ii) Forces people toward use of IP literals, even IP literals
in private address space (i.e., that convey little information),
when domain names might make the sender more identifiable and
more easily contacted.

  </pre>
</blockquote>
I agree with John here. <br>
<br>
Either, everyone will useless IP literals, or this change will *force*
many small businesses to use either their ISP's poor quality
smarthosts, or pay extra for a decent quality smarthost (and probably
push them further towards giving up email).<br>
<br>
Requiring people to have a static IP address is doomed to failure, as
(a) IPv4 address space is running out, so ISPs will be pushing towards
dynamic addresses, and (b) many ISPs already don't allow static IP
addresses.<br>
<br>
I don't see it giving any benefit, since 'bad senders' already know
SMTP/domains well enough to be able to make their mail look legitimate
enough to get through a crude filter like this. (eg - A spammer could
trivially create their own domain 'spammer.com' and "create" a virtual
zone of 4 billion entries (one for each IP address) and the spambots
will use the relevant entry from that domain as their EHLO data.
Instant 'pass' of any EHLO checking based on correspondence of EHLO to
IP address)<br>
<br>
IMV, arguments against SPF or DKIM pale into insignificance before the
problems of this proposed change.<br>
<br>
Something like SPF can give a big benefit, while only requiring people
to use an authorised submission server. This requires far more people
to use a submission server, and only gives a very dubious benefit.<br>
<br>
<pre class="moz-signature" cols="72">-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows</pre>
</body>
</html>

--------------050607040804050705040207--

Date: Tue, 27 Jan 2009 09:34:21 +0000
From: Paul Smith <[email-masked]>
To: Tony Finch <[email-masked]>
CC: John C Klensin <[email-masked]>, David MacQuigg <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------080103090302060908090503
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Tony Finch wrote:
> On Mon, 26 Jan 2009, Paul Smith wrote:
>
>   
>> There is a small business who has bought Microsoft SBS. What are they
>> supposed to use for the EHLO parameter if it is important enough to mean
>> the difference between delivery or not?
>>     
>
> If they have SBS set up correctly then it should just work. 

Under the 'current' way yes, but if it is changed so that the EHLO
parameter must correspond to the IP address of the sending host, then it
won't work, and I can't think of anything which would make it work. It
would HAVE to go via a third party smarthost, many of these are
unreliable and/or slow, so people prefer not to use them if they can.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

--------------080103090302060908090503
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Tony Finch wrote:
<blockquote
 cite="mid:[email-masked]"
 type="cite">
  <pre wrap="">On Mon, 26 Jan 2009, Paul Smith wrote:

  </pre>
  <blockquote type="cite">
    <pre wrap="">There is a small business who has bought Microsoft SBS. What are they
supposed to use for the EHLO parameter if it is important enough to mean
the difference between delivery or not?
    </pre>
  </blockquote>
  <pre wrap=""><!---->
If they have SBS set up correctly then it should just work. </pre>
</blockquote>
<br>
Under the 'current' way yes, but if it is changed so that the EHLO
parameter must correspond to the IP address of the sending host, then
it won't work, and I can't think of anything which would make it work.
It would HAVE to go via a third party smarthost, many of these are
unreliable and/or slow, so people prefer not to use them if they can.<br>
<pre class="moz-signature" cols="72">-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows</pre>
</body>
</html>

--------------080103090302060908090503--

Date: Tue, 27 Jan 2009 09:30:08 +0000
From: Paul Smith <[email-masked]>
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

Tony Hansen wrote:
> By extension, if you expect to use any further SMTP extensions after
> negotiating TLS, I think you MUST resend an EHLO.
>   
Yes, I agree.
> However, if you're *not* using any further extensions after STARTTLS was
> sent, I don't see a requirement. So consequently, since you say you're
> not using any other extensions, I don't see the case for them refusing
> the message at that point without the EHLO.
>   
Just for clarity the session goes like:

S: 220-main.remotedns.co.uk ESMTP Exim 4.63 #1 Mon, 26 Jan 2009 18:25:48
+0000
S: 220-We do not authorize the use of this system to transport unsolicited,
S: 220 and/or bulk e-mail.
C: EHLO vpop3.company.co.uk
S: 250-main.remotedns.co.uk Hello vpop3.company.co.uk [IP address]
S: 250-SIZE 52428800
S: 250-PIPELINING
S: 250-AUTH PLAIN LOGIN
S: 250-STARTTLS
S: 250 HELP
C: STARTTLS
S: 220 TLS go ahead
<TLS negotiation>
C: MAIL FROM:<[email-masked]>
S: 550 HELO required before MAIL

(It happens with a few domains, all of which seem to be using Exim (4.63
or 4.69))

It certainly looks as if it has forgotten the fact of the EHLO command
once the STARTTLS has happened.

Date: Tue, 27 Jan 2009 02:56:53 -0500
From: John C Klensin <[email-masked]>
To: David MacQuigg <[email-masked]>, Paul Smith <[email-masked]>
cc: [email-masked]
Subject: Re: Submission identifiers
Sender: [email-masked]

--On Monday, January 26, 2009 7:25 PM -0700 David MacQuigg
<[email-masked]> wrote:

>...
>> What are they supposed to use for the EHLO parameter if it is
>> important enough to mean the difference between delivery or
>> not?
> 
> This is a good case.  I think we can include Steve's example
> as a sub-case, since he has a static IP address from his ISP.
> 
> The solutions in this case include the three suggested for the
> roaming laptop, plus another option.  Get a static IP address
> from your ISP.  In my office I have a small network behind a
> NAT with one external address.  My ISP charges $10 per month
> for the static IP.  Last I checked, the phone company price
> was the same.
>...

David, situations about static IP addresses differ around the
world.  In many places, static IP addresses are not available at
all, at any price, unless one subscribes to a "business"
account... with "business" accounts costing five to ten times as
much as "residential" ones, sometimes with fewer services. 

I expect static addresses, and non-NAT addresses more
specifically, will get progressively harder to get as we move
into the IPv4 endgame.  I also expect that, unless prevented by
regulators or morals, some ISPs will take advantage of scarcity
by raising the price.  Of course, if the advent of IPv6 really
eliminates the use of private-space addresses on servers
communicating with the public Internet, the problem Paul is
concerned about will largely disappear (I'm not holding my
breath).
 
> By the way, I don't use my static IP for sending mail.  For
> reliable delivery, I relay my outgoing mail through a Giant
> Provider, not because of authentication problems, but simply
> that my domain is too small to have any clout with receivers.

The question, IMO, isn't whether the path of least resistance is
to give up on individual servers in favor of Giant Providers.
Nor is it about whether some people's lives would be easier if
servers using private address space behind NAT always used
submission servers that have public addresses.  It is about
whether we try to change the protocol to effectively _require_
one or the other or both.  My personal opinion is that such a
requirement would be a bad idea for several reasons.  However,
what is more important in this particular discussion is that it
would represent a sufficient change in requirements for a server
to conform to the standard to require a reset to Proposed.
Perhaps that would be worthwhile, but it isn't something that
you will see in any 5321bis for which I'm editor (not
necessarily because I'm being resistant, but because I will lose
interest).

>...
> I guess this discussion has about run its course, so I'll
> summarize.  We've narrowed the question to just whether SMTP
> should drop the requirement that receivers MUST NOT reject
> apparently forged helonames.  The suggested language for a
> change was:
> 
>   An SMTP server MAY reject a mail session if a domain name
> argument in   the EHLO command does not actually correspond to
> the IP address of the   client.  Clients that are unable to
> establish this correspondence MUST   NOT provide an invalid
> domain name argument.

Thanks for stating this clearly and providing text.  Note,
however, that "correspond to the IP address of the client" is
ambiguous.  For example, do you intend forward mapping or
reverse mapping?  If the client is multihomed, _exactly_ what
does "correspond" mean?

> The claimed benefit was that a currently useless and
> frequently abused parameter, the heloname, could then become
> useful in rejecting forgeries, and that would motivate
> legitimate senders to take it seriously.  This in turn could
> lead to large numbers of receivers making good use of the
> heloname, whitelisting legitimate domains, etc.  Eventually
> almost all legitimate mail would go this route, and only a
> small fraction would have to go through a spam filter,
> reducing losses, and improving the reliability of email
> communications.
> 
> The claimed reason for not making this change was that it
> would be too hard on some small fraction of legitimate
> senders.  Three cases were presented. 1) The small office with
> a dynamically-assigned IP address. 2) The roaming laptop.
> 3) The digital camera.
> 
> The solutions considered too difficult were:
> 1) Get a static IP address for the transmitter in the small
> office. 2) Relay through a transmitter with an established
> identity and reputation. 3) Authorize the entire block of
> addresses that might be dynamically assigned to the
> transmitter. 4) Use an address literal, meaning - please
> accept this session without a HELO ID.
> 
> Have I missed anything?

I don't think so, as long as it is clearly understood that this
does two things:

(i) Defines a domain name argument that does not match the
public Internet address of the sender as "invalid" --  thereby
de facto preventing most SMTP clients that are using private
address space behind at NAT from sending mail -- even if that
domain name conveys accurate information about the sending
system.

(ii) Forces people toward use of IP literals, even IP literals
in private address space (i.e., that convey little information),
when domain names might make the sender more identifiable and
more easily contacted.

    john

From: Steve Atkins <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: Submission identifiers
Date: Mon, 26 Jan 2009 18:41:13 -0800
Sender: [email-masked]

On Jan 26, 2009, at 6:25 PM, David MacQuigg wrote:

>
> At 09:50 AM 1/26/2009 +0000, Paul Smith wrote:
>
>> I have a 'common' situation in mind. Whether you like it or not, it  
>> is
>> common enough.
>> There is a small business who has bought Microsoft SBS. They are  
>> running
>> Microsoft Exchange with the POP3 connector over a dynamic IP ADSL.  
>> They
>> have Exchange set up for direct sending, with fallback to their ISP's
>> smarthost. They don't want to change to always use their ISP's  
>> smarthost
>> because that has long delays in sending.
>>
>> What are they supposed to use for the EHLO parameter if it is  
>> important
>> enough to mean the difference between delivery or not?
>
> This is a good case.  I think we can include Steve's example as a  
> sub-case, since he has a static IP address from his ISP.
>
> The solutions in this case include the three suggested for the  
> roaming laptop, plus another option.  Get a static IP address from  
> your ISP.  In my office I have a small network behind a NAT with one  
> external address.  My ISP charges $10 per month for the static IP.   
> Last I checked, the phone company price was the same.
>
> By the way, I don't use my static IP for sending mail.  For reliable  
> delivery, I relay my outgoing mail through a Giant Provider, not  
> because of authentication problems, but simply that my domain is too  
> small to have any clout with receivers.  I spent several hours  
> negotiating with just one Giant Provider to get on their whitelist.   
> The change I am suggesting will make it easier for small companies  
> to establish their identity, build a good reputation, and operate  
> their own transmitters.  That may be one reason we haven't seen a  
> solution to the identity fraud problem.  The Giant Providers like  
> the status quo.
>
> I guess this discussion has about run its course, so I'll  
> summarize.  We've narrowed the question to just whether SMTP should  
> drop the requirement that receivers MUST NOT reject apparently  
> forged helonames.  The suggested language for a change was:
>
>  An SMTP server MAY reject a mail session if a domain name argument in
>  the EHLO command does not actually correspond to the IP address of  
> the
>  client.  Clients that are unable to establish this correspondence  
> MUST
>  NOT provide an invalid domain name argument.
>
> The claimed benefit was that a currently useless and frequently  
> abused parameter, the heloname, could then become useful in  
> rejecting forgeries, and that would motivate legitimate senders to  
> take it seriously.  This in turn could lead to large numbers of  
> receivers making good use of the heloname, whitelisting legitimate  
> domains, etc.  Eventually almost all legitimate mail would go this  
> route, and only a small fraction would have to go through a spam  
> filter, reducing losses, and improving the reliability of email  
> communications.

I'm coming in a little late, so feel free to ignore the next three  
paragraphs if they've already been covered.

If the requirement for mail to bypass restrictions is that the HELO  
matches (by some metric) the reverse DNS of the IP address, and the  
only value of enforcing this rule is so that you can gather reputation  
based on this string, why not just use reputation based on the reverse  
DNS of the SMTP peer?

If the hope is that bad email will be sent from source addresses where  
the HELO string and IP don't match and good email will be sent from  
sources where the HELO string and IP do match that shows an awful lot  
of optimism about legitimate senders and a big underestimate of non  
legitimate senders.

Even SPF seems to have a more compelling value proposition.

> The claimed reason for not making this change was that it would be  
> too hard on some small fraction of legitimate senders.  Three cases  
> were presented.
> 1) The small office with a dynamically-assigned IP address.
> 2) The roaming laptop.
> 3) The digital camera.

4) MTAs that are smarthosts for a number of domains, and which use  
(for reasons that their vendors and users consider quite important)  
the domain name of the authenticated sender or the SMTP FROM as the  
HELO string. (That I think this behavior is... silly... doesn't mean  
it's not desired behavior by the users.)

>
>
> The solutions considered too difficult were:
> 1) Get a static IP address for the transmitter in the small office.
> 2) Relay through a transmitter with an established identity and  
> reputation.
> 3) Authorize the entire block of addresses that might be dynamically  
> assigned to the transmitter.
> 4) Use an address literal, meaning - please accept this session  
> without a HELO ID.
>

Cheers,
   Steve

Date: Mon, 26 Jan 2009 19:25:10 -0700
To: Paul Smith <[email-masked]>
From: David MacQuigg <[email-masked]>
Subject: Re: Submission identifiers
Cc: [email-masked]
Sender: [email-masked]

At 09:50 AM 1/26/2009 +0000, Paul Smith wrote:

>I have a 'common' situation in mind. Whether you like it or not, it is
>common enough.
>There is a small business who has bought Microsoft SBS. They are running
>Microsoft Exchange with the POP3 connector over a dynamic IP ADSL. They
>have Exchange set up for direct sending, with fallback to their ISP's
>smarthost. They don't want to change to always use their ISP's smarthost
>because that has long delays in sending.
>
>What are they supposed to use for the EHLO parameter if it is important
>enough to mean the difference between delivery or not?

This is a good case.  I think we can include Steve's example as a sub-case, since he has a static IP address from his ISP.

The solutions in this case include the three suggested for the roaming laptop, plus another option.  Get a static IP address from your ISP.  In my office I have a small network behind a NAT with one external address.  My ISP charges $10 per month for the static IP.  Last I checked, the phone company price was the same.

By the way, I don't use my static IP for sending mail.  For reliable delivery, I relay my outgoing mail through a Giant Provider, not because of authentication problems, but simply that my domain is too small to have any clout with receivers.  I spent several hours negotiating with just one Giant Provider to get on their whitelist.  The change I am suggesting will make it easier for small companies to establish their identity, build a good reputation, and operate their own transmitters.  That may be one reason we haven't seen a solution to the identity fraud problem.  The Giant Providers like the status quo.

I guess this discussion has about run its course, so I'll summarize.  We've narrowed the question to just whether SMTP should drop the requirement that receivers MUST NOT reject apparently forged helonames.  The suggested language for a change was:

  An SMTP server MAY reject a mail session if a domain name argument in
  the EHLO command does not actually correspond to the IP address of the
  client.  Clients that are unable to establish this correspondence MUST
  NOT provide an invalid domain name argument.

The claimed benefit was that a currently useless and frequently abused parameter, the heloname, could then become useful in rejecting forgeries, and that would motivate legitimate senders to take it seriously.  This in turn could lead to large numbers of receivers making good use of the heloname, whitelisting legitimate domains, etc.  Eventually almost all legitimate mail would go this route, and only a small fraction would have to go through a spam filter, reducing losses, and improving the reliability of email communications.

The claimed reason for not making this change was that it would be too hard on some small fraction of legitimate senders.  Three cases were presented.
1) The small office with a dynamically-assigned IP address.
2) The roaming laptop.
3) The digital camera.

The solutions considered too difficult were:
1) Get a static IP address for the transmitter in the small office.
2) Relay through a transmitter with an established identity and reputation.
3) Authorize the entire block of addresses that might be dynamically assigned to the transmitter.
4) Use an address literal, meaning - please accept this session without a HELO ID.

Have I missed anything?

-- Dave

Date: Mon, 26 Jan 2009 16:24:56 -0500
From: John C Klensin <[email-masked]>
To: Tony Finch <[email-masked]>
cc: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

Good.  As I tried to warn, I don't have 3207 in front of me and
don't have easy access to it today.  The text seems just right
to me.

   john

--On Monday, January 26, 2009 7:31 PM +0000 Tony Finch
<[email-masked]> wrote:

> On Mon, 26 Jan 2009, John C Klensin wrote:
>> 
>> Right. The quoted 3207 text says to me that the server is
>> required discard the data sent earlier by the client as part
>> of EHLO.  I don't see any expectation that it be required to
>> discard the fact that EHLO was sent.
> 
> The quote is:
>                              The server MUST discard any
> knowledge    obtained from the client, such as the argument to
> the EHLO command,    which was not obtained from the TLS
> negotiation itself.  The client    MUST discard any knowledge
> obtained from the server, such as the list    of SMTP service
> extensions, which was not obtained from the TLS    negotiation
> itself.
> 
> The argument to EHLO is just an example and is clearly not the
> entirety of what the server is supposed to discard.
> 
>> Indeed, unless there is something else in 3207, the client
>> isn't even required to discard the response from EHLO with the
>> server-supported feature list,
> 
> It is required to do so.
> 
> Tony.

Date: Mon, 26 Jan 2009 12:01:45 -0800
To: Jeff Macdonald <[email-masked]>
From: SM <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Cc: [email-masked]
Sender: [email-masked]

At 10:17 26-01-2009, Jeff Macdonald wrote:
>Doesn't the above also allow the NAT IP as well?

Answering questions involving NAT or SLUT66 [1] is tricky as there 
may not be any right answer.

There isn't any mention in RFC 5321 about whether IPv4 addresses 
excludes the ranges of IP addresses defined in RFC 1918.  I cannot 
find a reason to add that to the specifications as SMTP is not 
restricted to the Internet or public routable IP addresses only.

Some people may argue that the SMTP client cannot be identified if it 
uses a RFC 1918 IP address in an address literal to communicate with 
a SMTP server on the Internet.  For operational reasons, it's better 
for such hosts to relay through a smarthost using SUBMIT.

Regards,
-sm

1. http://www.ietf.org/mail-archive/web/behave/current/msg05046.html 

Date: Mon, 26 Jan 2009 20:12:02 +0000
From: Alexey Melnikov <[email-masked]>
To: Tony Finch <[email-masked]>
CC: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

Tony Finch wrote:

>On Mon, 26 Jan 2009, John C Klensin wrote:
>  
>
>>Right. The quoted 3207 text says to me that the server is
>>required discard the data sent earlier by the client as part of
>>EHLO.  I don't see any expectation that it be required to
>>discard the fact that EHLO was sent.
>>    
>>
>The quote is:
>                             The server MUST discard any knowledge
>   obtained from the client, such as the argument to the EHLO command,
>   which was not obtained from the TLS negotiation itself.  The client
>   MUST discard any knowledge obtained from the server, such as the list
>   of SMTP service extensions, which was not obtained from the TLS
>   negotiation itself.
>
>The argument to EHLO is just an example and is clearly not the entirety of
>what the server is supposed to discard.
>  
>
>>Indeed, unless there is something else in 3207, the client isn't
>>even required to discard the response from EHLO with the
>>server-supported feature list,
>>    
>>
>It is required to do so.
>  
>
Indeed. Before STARTTLS was negotiated there is no guaranty that a 
man-in-the-middle attacker hasn't modified the list of capabilities 
advertised by the server.

Date: Mon, 26 Jan 2009 19:31:12 +0000
From: Tony Finch <[email-masked]>
To: John C Klensin <[email-masked]>
cc: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

On Mon, 26 Jan 2009, John C Klensin wrote:
>
> Right. The quoted 3207 text says to me that the server is
> required discard the data sent earlier by the client as part of
> EHLO.  I don't see any expectation that it be required to
> discard the fact that EHLO was sent.

The quote is:
                             The server MUST discard any knowledge
   obtained from the client, such as the argument to the EHLO command,
   which was not obtained from the TLS negotiation itself.  The client
   MUST discard any knowledge obtained from the server, such as the list
   of SMTP service extensions, which was not obtained from the TLS
   negotiation itself.

The argument to EHLO is just an example and is clearly not the entirety of
what the server is supposed to discard.

> Indeed, unless there is something else in 3207, the client isn't
> even required to discard the response from EHLO with the
> server-supported feature list,

It is required to do so.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

Date: Mon, 26 Jan 2009 19:20:49 +0000
From: Tony Finch <[email-masked]>
To: Paul Smith <[email-masked]>
cc: John C Klensin <[email-masked]>, David MacQuigg <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

On Mon, 26 Jan 2009, Paul Smith wrote:

> There is a small business who has bought Microsoft SBS. What are they
> supposed to use for the EHLO parameter if it is important enough to mean
> the difference between delivery or not?

If they have SBS set up correctly then it should just work. Unfortunately
Microsoft encourages SBS users to make up domain names under .local for
internal use, and if you follow this bad advice you can run into trouble.
However most of the trouble will only occur if you try to get Apple
Bonjour (which uses .local for multicast DNS service discovery) with a
private SBS domain created according to Microsoft's recommendations.

http://support.microsoft.com/kb/296250

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.

Date: Mon, 26 Jan 2009 13:22:51 -0500
From: Jeff Macdonald <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

On Fri, Jan 23, 2009 at 01:46:06AM -0500, John C Klensin wrote:
>Question: Is it time to formally deprecate 821 and, in
>particular, the main feature that distinguishes it: the use of
>HELO by SMTP clients?  We would still need to require that SMTP
>servers accept it, but we would tell full-capability clients
>(including the client side of relays and gateways) that HELO is
>obsolete.  One corollary of this is that we'd be telling
>low-capability clients, particularly those that are part of MUA
>systems, that they should be talking to Submit ports, not SMTP
>ones.

I'm ok with this. Simply because EHLO's requirements for a FQDN is
clearer than HELO.

-- 
Jeff Macdonald
[email-masked]

Date: Mon, 26 Jan 2009 13:17:22 -0500
From: Jeff Macdonald <[email-masked]>
To: SM <[email-masked]>
Cc: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

On Mon, Jan 26, 2009 at 10:01:41AM -0800, SM wrote:
>
> The parameter should be the SMTP client's identity.  From RFC 5321:
>
>      "The domain name given in the EHLO command MUST be either a primary
>       host name (a domain name that resolves to an address RR) or, if
>       the host has no name, an address literal, as described in
>       Section 4.1.3 and discussed further in the EHLO discussion of
>       Section 4.1.4."
>

Doesn't the above also allow the NAT IP as well?

-- 
Jeff Macdonald
[email-masked]

Date: Mon, 26 Jan 2009 10:01:41 -0800
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

At 01:50 26-01-2009, Paul Smith wrote:
>I have a 'common' situation in mind. Whether you like it or not, it is
>common enough.
>There is a small business who has bought Microsoft SBS. They are running
>Microsoft Exchange with the POP3 connector over a dynamic IP ADSL. They
>have Exchange set up for direct sending, with fallback to their ISP's
>smarthost. They don't want to change to always use their ISP's smarthost
>because that has long delays in sending.
>
>What are they supposed to use for the EHLO parameter if it is important
>enough to mean the difference between delivery or not?

The parameter should be the SMTP client's identity.  From RFC 5321:

      "The domain name given in the EHLO command MUST be either a primary
       host name (a domain name that resolves to an address RR) or, if
       the host has no name, an address literal, as described in
       Section 4.1.3 and discussed further in the EHLO discussion of
       Section 4.1.4."

You may experience delivery issues sending mail from a connection 
using a dynamic IP address.

Regards,
-sm 

Date: Mon, 26 Jan 2009 12:57:19 -0500
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

--On Monday, January 26, 2009 12:04 PM -0500 Tony Hansen
<[email-masked]> wrote:

> 
> At the heart of this question is whether a client is free to
> use an extension supported by the server if they haven't
> issued an EHLO? This question holds true for any extension,
> not just for STARTTLS. Is issuing an EHLO an enabler for the
> extension as well as a query to determine which extensions are
> supported by the server?
> 
> To answer these questions, I note this text in RFC 5321
> highlighted with
>>>> <<<:
> 
>    3.2.  Client Initiation
> 
>    Once the server has sent the greeting (welcoming) message
> and the    client has received it, the client normally sends
> the EHLO command to    the server, indicating the client's
> identity.  In addition to    opening the session, use of EHLO
> >>>indicates that the client is able    to process service
> extensions<<< and requests that the server provide    a list
> of the extensions it supports.
> 
> So this indicates to me that EHLO does indeed act as a gateway
> and enabler to the use of extensions. So I'd argue that a
> server is free to reject the use of any extensions unless a
> client has used EHLO prior to that.

Indeed.  Also, while I don't have time to find it right now, I'm
quite sure that there is a statement in 5321 that says that the
client MUST NOT try to exercise any extension not explicitly
offered by the server.

> By extension, if you expect to use any further SMTP extensions
> after negotiating TLS, I think you MUST resend an EHLO.

Yes, that is certainly how I would read the quoted 3207 text.

> However, if you're *not* using any further extensions after
> STARTTLS was sent, I don't see a requirement. So consequently,
> since you say you're not using any other extensions, I don't
> see the case for them refusing the message at that point
> without the EHLO.

Right. The quoted 3207 text says to me that the server is
required discard the data sent earlier by the client as part of
EHLO.  I don't see any expectation that it be required to
discard the fact that EHLO was sent.    

Indeed, unless there is something else in 3207, the client isn't
even required to discard the response from EHLO with the
server-supported feature list, so one can make a case that it is
not necessary to reissue EHLO at all, although a client that
does not do so should expect "no, bozo, I will accept that
extension from some domains and not others, and you are not one
of the permitted ones" response from the server.    If that is a
plausible reading of 3207, then it could probably do with a bit
of clarification.

    john

Date: Mon, 26 Jan 2009 12:04:41 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: STARTTLS & EHLO
Sender: [email-masked]

At the heart of this question is whether a client is free to use an
extension supported by the server if they haven't issued an EHLO? This
question holds true for any extension, not just for STARTTLS. Is issuing
an EHLO an enabler for the extension as well as a query to determine
which extensions are supported by the server?

To answer these questions, I note this text in RFC 5321 highlighted with
>>><<<:

   3.2.  Client Initiation

   Once the server has sent the greeting (welcoming) message and the
   client has received it, the client normally sends the EHLO command to
   the server, indicating the client's identity.  In addition to
   opening the session, use of EHLO >>>indicates that the client is able
   to process service extensions<<< and requests that the server provide
   a list of the extensions it supports.

So this indicates to me that EHLO does indeed act as a gateway and
enabler to the use of extensions. So I'd argue that a server is free to
reject the use of any extensions unless a client has used EHLO prior to
that.

By extension, if you expect to use any further SMTP extensions after
negotiating TLS, I think you MUST resend an EHLO.

However, if you're *not* using any further extensions after STARTTLS was
sent, I don't see a requirement. So consequently, since you say you're
not using any other extensions, I don't see the case for them refusing
the message at that point without the EHLO.

	Tony Hansen
	[email-masked]

Paul Smith wrote:
> We've just had a small issue come up wrt STARTTLS and I was wondering if
> it was a mistake in RFC 3207, or just me :)
> 
> We did a quick patch for someone and added STARTTLS into a new part of
> our software in a special build. For speed of releasing the patch, we
> didn't make it issue a new EHLO afterwards, based on section 4.2 of RFC 3207
> 
> "  The client SHOULD send an EHLO command as the
>    first command after a successful TLS negotiation."
> 
> 
> (SHOULD, not MUST). We were planning to add the extra EHLO in before
> this went to a general release (we don't need to know about any
> extensions other than that STARTTLS, so it's not necessary from our PoV)
> . Anyway, 99.9% of the time, it's OK, but they have encountered one
> recipient which refuses to accept the message in this case. So, they
> seem to be treating the 'SHOULD' as 'MUST'.
> 
> Now, earlier in section 4.2, it says "
> 
> "  The server MUST discard any knowledge
>    obtained from the client, such as the argument to the EHLO command,
>    which was not obtained from the TLS negotiation itself"
> 
> 
> which I guess is why they are rejecting the message, because (since they
> have discarded the fact that we have previously sent EHLO) they are
> complaining that we haven't sent EHLO yet.
> 
> So, should the 'SHOULD' be a 'MUST' in this RFC, or is the other server
> incorrect?
> 

Date: Mon, 26 Jan 2009 16:28:02 +0000
From: Paul Smith <[email-masked]>
To: [email-masked]
Subject: STARTTLS & EHLO
Sender: [email-masked]

We've just had a small issue come up wrt STARTTLS and I was wondering if
it was a mistake in RFC 3207, or just me :)

We did a quick patch for someone and added STARTTLS into a new part of
our software in a special build. For speed of releasing the patch, we
didn't make it issue a new EHLO afterwards, based on section 4.2 of RFC 3207

"  The client SHOULD send an EHLO command as the
   first command after a successful TLS negotiation."

(SHOULD, not MUST). We were planning to add the extra EHLO in before
this went to a general release (we don't need to know about any
extensions other than that STARTTLS, so it's not necessary from our PoV)
. Anyway, 99.9% of the time, it's OK, but they have encountered one
recipient which refuses to accept the message in this case. So, they
seem to be treating the 'SHOULD' as 'MUST'.

Now, earlier in section 4.2, it says "

"  The server MUST discard any knowledge
   obtained from the client, such as the argument to the EHLO command,
   which was not obtained from the TLS negotiation itself"

which I guess is why they are rejecting the message, because (since they
have discarded the fact that we have previously sent EHLO) they are
complaining that we haven't sent EHLO yet.

So, should the 'SHOULD' be a 'MUST' in this RFC, or is the other server
incorrect?

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Mon, 26 Jan 2009 16:01:52 +0100
From: Alessandro Vesely <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: Submission identifiers
Sender: [email-masked]

John C Klensin wrote:
> And it is not at all clear to me that an SMTP server that supports
> a handful of closely-related people/accounts or a small enterprise
> should be equated with an ESP for all purposes.

I meant ESP generically. Anyone who runs one or more SMTP servers, 
Hotmail, Intel, ACM, JCK, or the Wandering Laptop. Big or small, 
public or private, they all run globally visible SMTP server(s), each 
one for its own purposes, with its own style and policy.

> As soon as one goes down the path of wanting to change,
> restrict, or reinterpret what the specs say about particular
> fields, you are restricting the applicability of your method to
> a select group of friends at best or falling into one of the
> more obvious "Anti-spam Kook" traps of requiring universal
> deployment of your "improvement".  You have the perfect right to
> reject any mail that doesn't conform to your ideas of what an
> envelope should look like (with or without support from the
> specs), just as you have the perfect right to reject any message
> with Chinese content on the grounds that you don't read Chinese
> so it couldn't possibly be for you and is therefore probably
> either malware or just unwanted.  But either path is going to
> reject legitimate messages from folks who, through no fault of
> their own, don't conform to your expectations.   Personally, I
> don't think that is a good idea.  On that subject, YMMD.  But
> making significant changes in the standards, changes that would
> invalidate many or most existing conforming implementations, to
> reflect your ideas of how things would work in a better world...
> well, probably isn't going to happen.

Those are exactly the reasons why I insist that a transport level 
mechanism for rejecting abuses should not live in an well designed 
extension, nor in a clever but optional retrofit. Message level 
mechanisms, such as DKIP and PGP, are apparently well promising. 
Transport level mechanisms, by their nature, are implemented in the 
envelope rather than in the content. Because they are not necessarily 
visible to end users, their optionality makes transport fuzzy. 
Authentication has to be at the core of the specs, for relays as well 
as for submitters.

We all have been applying various filtering expedients in these years; 
someone may have been more successful than someone else, but many have 
to continuously adjust their filters, because expedients are not 
global rules. Whatever rule I can think of today, I'll have to 
controvert it tomorrow, because I need to come to terms with the world 
out there. If you allow me to paraphrase your words, I don't want the 
standard to reflect my ideas of how things would work in a better 
world: I want it to reflect _yours_. However, I don't want you to miss 
any possible chance to heal the mail transport system. It's dying.

> As I said, big change in semantics (we differ about "slightly"),
> especially since, while it is convenient to talk about ADMDs for
> some purposes, SMTP has no such concept (and, if you don't
> understand the policy implications of the X.400 definition of
> ADMD, you probably should, because it is very much tied up with
> my comments about types of ESPs above).

Why? I'm not much into X.400 specs. That concept of domain looks very 
similar, in spirit, with the notion of Internet domain that I'm more 
familiar with. I guess the (partial?) failure of X.400 mail model is 
rooted in those details about levels and ADMD vs. PRMD distinctions 
that are difficult to understand. For the Internet, a great 
simplification has been brought about by MX records. That notion of 
domain, the part to the right of the strudel, characterizes SMTP.

> But the big problem is, again, transition, since a server would
> have no way to tell whether a client was legitimately and
> appropriately providing a name under the old rules or under your
> new ones.   And that takes me back to my earlier suggestion: if
> you want this information, define a new extension or keyword for
> doing it, one whose argument semantics you really can control
> and one whose use would explicitly invoke your rules and
> interpretations.  Although I'd predict that you'd have trouble
> getting it off the ground, another possibility would be to
> propose to replace EHLO with a validated-hello command, VHLO,
> which would be just like EHLO except for support for a different
> argument architecture.  It would take me about ten minutes to
> write an I-D specifying that.  Of course, it would take me even
> less time to shoot the idea down on the basis of deployment
> costs, etc., but, again, your mileage, and assumptions, may
> differ.

Hey, thinking about it, VHLO looks a very clever idea! Two pros come 
straightaway:

1) It would follow the steps of the successful transition from HELO to 
EHLO, reinforcing the concept that there are two possible salutations 
until all hosts have upgraded.

2) I'd imagine that matching an A record won't suffice: At least an MX 
with the same address would be required for VHLO. (To MX or not to MX 
has been discussed at length.) Would it be safe to conjecture that a 
server could rule out zombies that way?

Date: Mon, 26 Jan 2009 09:50:14 +0000
From: Paul Smith <[email-masked]>
To: John C Klensin <[email-masked]>
CC: David MacQuigg <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

John C Klensin wrote:
> So what are you asking for that isn't there already?
> Elimination of the IP address literal option?  Some criteria for
> "verifiable" that would presumably lie well outside the scope of
> SMTP?
>   
I have a 'common' situation in mind. Whether you like it or not, it is
common enough.
There is a small business who has bought Microsoft SBS. They are running
Microsoft Exchange with the POP3 connector over a dynamic IP ADSL. They
have Exchange set up for direct sending, with fallback to their ISP's
smarthost. They don't want to change to always use their ISP's smarthost
because that has long delays in sending.

What are they supposed to use for the EHLO parameter if it is important
enough to mean the difference between delivery or not?

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Sun, 25 Jan 2009 20:43:10 -0500
From: John C Klensin <[email-masked]>
To: Mark Andrews <[email-masked]>
cc: Keith Moore <[email-masked]>, John Levine <[email-masked]>, [email-masked]
Subject: Re: Dotless domains
Sender: [email-masked]

--On Monday, January 26, 2009 9:56 AM +1100 Mark Andrews 
<[email-masked]> wrote:

> In message <[email-masked]>, John C 
Klensin writes:

>> --On Saturday, January 24, 2009 21:02 -0500 Keith Moore
>> <[email-masked]> wrote:

>> > IMHO, there should be an RFC explaining that use of a TLD 
as a
>> > terminal domain name was never intended to work, and should
>> > not be expected to be supported by the Internet 
architecture
>> > or software.
>>
>> You know where to find the virtual pencil.  I would certainly
>> support such a document, but the real support would need to 
come
>> from DNS folks.

> 	How so?  The DNS is a container to hold information.  The
> 	rules about that information are external to the DNS and
> 	always have been.

Mark,

I don't want to single you out for abuse because the approach 
you comment shows is pretty common, but comments like the 
above, while certainly true, are one of the reasons we see so 
many uses, and proposed uses, of the DNS that just don't work 
very well operationally.   Sure, the DNS can be viewed as a 
container that can be used to store almost any data.  People 
have discovered that the data of an MX record is more or less 
just a string and that lots of nonsense can be stored there, 
including IP addresses and names of aliases.  In principle, we 
don't need special provisions for IDNs because, after all, 
nothing prevents eight-bit data in labels.  And, yes, the root 
zone is really no different from any other zone and, if folks 
want to put a million delegations there or, for that matter, 
install address records so that "http://./" or "ftp ." could, 
in principle, work, then the DNS, as a data container, doesn't 
care.

I think there is agreement that at least some of those things 
are dumb and would cause problems in practice.  Perhaps you 
disagree.   But when someone says "this is dumb and there 
should be a document that explains why it should be prohibited/ 
avoided" and you (or some other DNS expert) responds by saying 
"no, the DNS is just a container and doesn't care", those who 
want to do dumb things take that as approval and start 
insisting that they should be able to do whatever they want to 
do and that you (and DNS experts generally) are endorsing it.

> 	One of the problems is lots of RFC's say "domain name" when
> 	they are actually refering to a "hostname".  This has
> 	introduced lots of confusion.

One of the problems is that lots of RFCs are ambiguous or 
contradictory as to whether that distinction exists and all and 
what it means.  If memory serves, two of them are RFC 1034 and 
RFC 1123.

> 	RFC921 made it pretty clear that we were to stop using non-
> 	heirachical names and that single labels were only ever to
> 	be used as aliases for heirachical names.

Sure, but the context of 921 was very different -- not local 
versus hierarchical but host tables versus domain names. 
Whether 921 has any particular meaning today is an interesting 
question.  If you want to cite it as authoritative, you had 
best convince the IESG to change its status from "UNKNOWN" to 
"BCP" at least.

> 	Saying that non-heirachical names only have local scope and
> 	are to not to be used for global scope addressing would be
> 	a good thing.  This leaves "localhost" in local scope.

But, unless I misunderstood Keith's comment, that sort of 
statement/ restriction is exactly what he was asking for and 
with which I was trying to agree.  But a narrow reading of 
either your comment above or some of the statements in RFC 2181 
would say "that is not a DNS issue"... which some people would 
promptly interpret as "'localhost.' is a perfectly reasonable 
FQDN and, in any context in which a string is known to be an 
FQDN, 'localhost' and 'localhost.' are exactly equivalent".

    john

To: John C Klensin <[email-masked]>
Cc: Keith Moore <[email-masked]>, John Levine <[email-masked]>, [email-masked]
From: Mark Andrews <[email-masked]>
Subject: Re: Dotless domains 
Date: Mon, 26 Jan 2009 09:56:10 +1100
Sender: [email-masked]

In message <[email-masked]>, John C Klensin writes:
> 
> 
> 
> --On Saturday, January 24, 2009 21:02 -0500 Keith Moore
> <[email-masked]> wrote:
> 
> > 
> > IMHO, there should be an RFC explaining that use of a TLD as a
> > terminal domain name was never intended to work, and should
> > not be expected to be supported by the Internet architecture
> > or software.
> 
> You know where to find the virtual pencil.  I would certainly
> support such a document, but the real support would need to come
> from DNS folks.
> 
>     john

	How so?  The DNS is a container to hold information.  The
	rules about that information are external to the DNS and
	always have been.

	One of the problems is lots of RFC's say "domain name" when
	they are actually refering to a "hostname".  This has
	introduced lots of confusion.

	RFC921 made it pretty clear that we were to stop using non-
	heirachical names and that single labels were only ever to
	be used as aliases for heirachical names.

	Saying that non-heirachical names only have local scope and
	are to not to be used for global scope addressing would be
	a good thing.  This leaves "localhost" in local scope.

	Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

Date: Sun, 25 Jan 2009 14:44:57 -0500
From: John Leslie <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: Submission identifiers
Sender: [email-masked]

John C Klensin <[email-masked]> wrote:
> 
> Two very general observations:
> 
> (1) You have just explained --whether you intended to or not--
> why it is much better to tie policy information to the domain in
> the MAIL command than to the one in the EHLO/HELO command.

   Certain kinds of policies need to be tied to MAIL-FROM; others
should be tied to EHLO, IMHO.

   When domain managers want to state that mail claiming to be
from theic domain should (ordinarily?) be sent from particular MTAs,
policy tied to MAIL-FROM is much better. (Of course, it's not quite
right: tieing such policy to an end-user-visible header would be
best.)

   When domain managers want to state a policy about DSNs addressed
to their domain, MAIL-FROM is exactly the right field to tie it to.   

   OTOH, when MTA managers want to say, "We virus-check all
outgoing email," policy tied to EHLO is a much better tool.

   (It has always bothered me when SPF proponents claim it's ever
reasonable to tie the same SPF record to different fields.)

> (2) Even more generally, the various efforts to design and
> specify verification and validation procedures for email really
> need to start, IMO, with an understanding that the current specs
> and installed base are a given and that a combination of the
> robustness principle and operation experience prohibits action
> based on a possible protocol-lawyer's reading of those specs.

   I don't think John K means quite what that says: I think he
means that anything allowed by the current spec will need to
continue to work for _some_ sending and receiving MTAs.

> making significant changes in the standards, changes that would
> invalidate many or most existing conforming implementations, to
> reflect your ideas of how things would work in a better world...
> well, probably isn't going to happen.

   Such things certainly can happen by route of SMTP extensions
agreed to by both sending and receiving MTAs, and that is always a
better way to approach things than trying to change the base spec.

   (Yes, I know I have lobbied to relax some things in the standard
that I have reason to know are somewhat commonly ignored; but I
have also stopped beating the horse when it is pronounced dead...)

   As we go forward with 5321bis, I suggest we make sure that there's
ample room for extensions to fix known problems like whether to
trust a return-path, what's the best way to report emails caught
in spam filters, differing policies for different recipients in
the same transaction, and establishing/reporting reputation info.

   We might also do well to decide whether chartering a WG to work
on such extensions would be wise. (I firmly believe it is possible,
but that it needs to be quite separate from advancing 5321 to
full Standard.)

--
John Leslie <[email-masked]>

From: Steve Atkins <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Date: Sun, 25 Jan 2009 11:04:17 -0800
Sender: [email-masked]

On Jan 25, 2009, at 10:52 AM, David MacQuigg wrote:

>>
>
> Sorry for being sloppy in implying that section 4.1.4 requires that  
> receivers accept "HELO this is Jupiter".  The connection between   
> cause and effect is less direct.  I should have said that this  
> requirement (no rejection when verification fails) makes the HELO  
> name unreliable for authentication.  It means that receivers MUST  
> accept mail from transmitters that provide a fraudulent ID, i.e. one  
> that does not correspond to the IP address of the TCP connection.   
> The fact that receivers can't rely on the HELO name leads to little  
> use of that name for authentication, and attempts to find some other  
> name (MAIL FROM) that will more reliably ID the transmitter.  This  
> may be the reason so many transmitters provide useless HELO names  
> like Jupiter.
>
> The Jupiter example came from a long ago discussion in the SPF  
> group, in which the conclusion was we have to accept this, even if  
> it is obviously invalid.  The reason was that some legitimate  
> senders have HELO names like this, and we would be rejecting  
> legitimate mail.
>
> Why not say:
>
>  An SMTP server MAY reject a mail session if a domain name argument in
>  the EHLO command does not actually correspond to the IP address of  
> the
>  client.  Clients that are unable to establish this correspondence  
> MUST
>  NOT provide an invalid domain name argument.
>
> How the correspondence is established should be left to another spec  
> (CSV, SPF, whatever).  There is no change in the syntax of the HELO  
> command, no change that would affect receivers following the old  
> MUST NOT rule, just a change in emphasis that should have a  
> beneficial effect on the reliability of HELO names as IDs.
>
> I see very little downside.  The examples of the digital camera and  
> the roaming laptop don't justify degrading the reliability of HELO  
> IDs for everyone else.  Maybe I'm not understanding these examples.   
> See below for discussion of the roaming laptop.

If I'm an SMTP client behind a NAT I probably have no idea I'm behind  
a NAT, and my local IP address (and, hence, hostname) will likely have  
little connection to the visible IP address.

More importantly, perhaps, if I'm an SMTP client behind a NAT then the  
hostname used in EHLO is the only recorded trace of where the email  
originated. The only time that identifier is useful for diagnostics is  
in the case where there isn't a 1:1 correspondence between the EHLO  
identifier and the external IP address.

Mailservers behind NAT aren't at all unusual. Multiple mailservers  
NATing to a single IP address aren't unheard of.

Cheers,
   Steve

Date: Sun, 25 Jan 2009 11:52:39 -0700
To: John C Klensin <[email-masked]>, [email-masked]
From: David MacQuigg <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Hi John,

Good discussion.  I appreciate your time and effort on this.  I'll try to avoid sloppy statements that generate unnecessary discussion.

At 09:47 AM 1/24/2009 -0500, John C Klensin wrote:
>--On Saturday, January 24, 2009 6:23 -0700 David MacQuigg wrote:
>
>>> 
>>>> ...
>>>> I have yet to see any sensible use-case that would be
>>>> complicated by a requirement that the HELO/EHLO name end in a
>>>> registered, verifiable domain name.
>>>> ...
>>> 
>>> David,
>>> 
>>> Today, under 5321, EHLO clearly requires a fully-qualified
>>> domain name that can be resolved unless the host doesn't know
>>> its name and has to use an IP address, a situation that the
>>> spec doesn't encourage.  My reading of 5321 (but not
>>> necessarily 821) says that HELO does too.
>> 
>> Perhaps it is just a matter of emphasis.  As I read the spec
>> it *does* seem to encourage using HELO/EHLO names that are
>> useless for authentication.  For example, the language in
>> 4.1.4 "if the verification fails, the server MUST NOT refuse
>> to accept a message" puts the burden on the receiver to deal
>> with mis-configured transmitters.  Receivers are expected to
>> accept mail from transmitters that say "HELO this is Jupiter".
>
>Sigh.  Read more carefully...
>
>(1) It says, quite deliberately, "MUST NOT refuse to accept a
>message on that basis".  That is _very_ different from the
>excerpt you quote.

Sorry for being sloppy in implying that section 4.1.4 requires that receivers accept "HELO this is Jupiter".  The connection between  cause and effect is less direct.  I should have said that this requirement (no rejection when verification fails) makes the HELO name unreliable for authentication.  It means that receivers MUST accept mail from transmitters that provide a fraudulent ID, i.e. one that does not correspond to the IP address of the TCP connection.  The fact that receivers can't rely on the HELO name leads to little use of that name for authentication, and attempts to find some other name (MAIL FROM) that will more reliably ID the transmitter.  This may be the reason so many transmitters provide useless HELO names like Jupiter.

The Jupiter example came from a long ago discussion in the SPF group, in which the conclusion was we have to accept this, even if it is obviously invalid.  The reason was that some legitimate senders have HELO names like this, and we would be rejecting legitimate mail.

Why not say:

  An SMTP server MAY reject a mail session if a domain name argument in
  the EHLO command does not actually correspond to the IP address of the
  client.  Clients that are unable to establish this correspondence MUST
  NOT provide an invalid domain name argument.

How the correspondence is established should be left to another spec (CSV, SPF, whatever).  There is no change in the syntax of the HELO command, no change that would affect receivers following the old MUST NOT rule, just a change in emphasis that should have a beneficial effect on the reliability of HELO names as IDs.

I see very little downside.  The examples of the digital camera and the roaming laptop don't justify degrading the reliability of HELO IDs for everyone else.  Maybe I'm not understanding these examples.  See below for discussion of the roaming laptop.

>(2) The place where the names get "used", i.e., the SMTP client,
>is covered by the strongest type of statement that we can make
>in a standard:
>
>        "The SMTP client MUST, if possible, ensure that the
>        domain parameter to the EHLO command is a primary host
>        name as specified for this command in Section 2.3.5."

Then the strength of this statement is nullified with a MUST NOT in the next paragraph.

>The key case that justifies "if possible" is mentioned in the
>next sentence, but there are others.  2.3.5 is very explicit,
>arguably to the point of tedium -- the string must be
>fully-qualified (no local names or incomplete strings permitted)
>and it must be resolvable (by any plausible reading "in the
>public DNS" -- and I wrote myself a note in December to ask
>where things had gotten weird enough that we had to add that
>phrase).
>
>(3) "HELO this is Jupiter" fails the syntax rule
>
>   helo = "HELO" SP Domain CRLF
>
>in several ways.  If you receive such a string, you are
>perfectly justified, conformant, and in line with the spec to
>reject it with a 501 code.   Indeed, you would be outside the
>spec if you did not do so.  You wouldn't even have to figure out
>whether "this" was a resolvable FQDN or not.

The prohibition against an unauthorized domain name should be just as strong as the prohibition against bad syntax.  The fix is equally simple in both cases, even for digital cameras with built in transmitters.

>My imagination doesn't stretch nearly far enough to get from
>those three points to "seems to encourage using HELO/EHLO names
>that are useless for authentication".
>
>That prohibition in 4.1.4 goes back to RFC 1123, which predates
>2821 by nearly a dozen years.  There are a number of reasons for
>it.  Some of them are subtle, some of them involve cases that we
>would now prefer to see handled via submission servers and prior
>agreements.  But let me give you a contemporary example that
>might make the situation clear.   Suppose there is a laptop with
>a full service, conforming, SMTP client on it.  It sends mail,
>manages queues, etc. -- no submission server involved.  It has a
>permanent name, say roaming.example.com.  However, it is not
>connected at a fixed point to the public Internet.  It roams,
>and may have different addresses on different networks on
>different days.  If its owner is being really careful she even
>uses dynamic DNS to be sure that, at any time the host is
>online, its domain names points to its actual address.   Or, if
>she knows that the host is used at only a half-dozen locations
>and has a usual address at each, she might set up static DNS
>entries that list each of them as its address.     Now suppose a
>message is sent to you from that host, after which it is shut
>down.  When it reappears on the network a few hours later, it is
>at a different address.  If you resolve the domain name at that
>time, you get a valid address for the host, but it isn't the
>same address the host had when it initiated sending the message.
>Do you really want to reject the message?  If you do, please
>don't justify the decision on the basis that that you have done
>a careful and adequate job of authentication -- _nothing_ you
>can do with HELO/EHLO as they are defined in 5321 is going to
>get you to a point at which that is possible.

Let us call this the "roaming laptop" problem.  There are several solutions.
1) The laptop user can leave her setup as is, relaying through the same server as when she is in her home town.
2) She can publish a DNS record authorizing a collection of IP addresses, or even entire blocks that she might be using.
3) She can use an address literal, saying in effect - please accept this session request without a HELO check.

For the typical "road warrior" running Windows and Outlook, {1) is by far the most likely solution.  Even if she changes the default settings for SMTP server, she will still need to keep the same POP or IMAP server, and the same password to access her home mailbox.  Same for the other popular email programs.  They are just not expecting to use a local SMTP server.

Solution 2 is not yet available, but if senders were to start taking HELO names seriously, a solution would quickly develop.  It would be something like SPF, without the risky or ambiguous terms that have led to lack of universal acceptance.

Solution 3 is no worse than the status quo.  There is no degradation in service for those using this method, if others chose a more reliable method.

>> The purpose of changes should be to avoid lost mail, as
>> opposed to rejected mail, which doesn't have any damaging
>> effects on reliability.  SMTP REJECT should be encouraged as
>> the proper response to doubtful requests.  (Sorry, we don't
>> accept mail from unidentified transmitters.)
>
>2821/5321 were carefully written to give you the option of
>making that decision... or the decision to not accept mail from
>even-numbered IP addresses for that matter.  I personally cannot
>recommend it unless you have a much better notion of
>"identified" than anything you can get out of HELO/EHLO.  I'd
>also suggest to you that the typical user who is waiting for a
>message to arrive is not likely to distinguish between
>"rejected" and "lost".  Unless the former causes the sender to
>get on the phone, the only issue is "didn't arrive".  Your users
>and experience may differ.

The difference between rejected and lost is critical to the reliability of our email system.

>>> So what are you asking for that isn't there already?
>>> Elimination of the IP address literal option?  Some criteria
>>> for "verifiable" that would presumably lie well outside the
>>> scope of SMTP?
>> 
>> If you can deprecate HELO, surely you can do the same for
>> address literals.
>
>Independent of some practical issues and the fact that address
>literals can sometimes actually provide useful information, that
>would accomplish what?

Eliminate what seems to be useless information, and motivate senders to provide useful information instead.  What is in an address literal that we can't get more reliably from the TCP address?

>    You can already make a policy decision
>to reject any message that starts with EHLO and an address
>literal; doing so would be consistent with your position that
>false rejects are ok as long as one doesn't drop the message.
>
>I can't recommend that we try to dig into the question in the
>standard, but consider whether you would prefer:
>
>   EHLO [#.#.#.#]
>
>where "#.#.#.#" is a valid, public, IP address that might even
>be static and stable and have a legitimate reverse mapping to a
>host name,     and
>
>   EHLO comment.example.org
>
>where "comment.example.org" is a valid, registered, resolvable
>FQDN but the only information at its DNS node is
>
>     IN TXT This is a mobile host and I can't really tell you
>what or where it is without compromising my privacy.
>
>You don't have much "authentication" information either way; you
>might be able to extract as much or more contact information
>from the first.

I would split the mail into two streams.  99% of legitimate messages are sent from transmitters with verifiable HELO IDs.  1% have an address literal, or some more explicit way of saying "I can't ID just yet, please accept this mail session anyway."

Idea: The HELO name could actually specify what authentication method the sender offers in lieu of a robust HELO check.
    EHLO  _dkim.hostname.example.com

>>> Do keep in mind that we know from experience that the bad guys
>>> have absolutely no trouble obtaining perfectly valid domain
>>> names.
>> 
>> I agree that reputation requirements should be outside the
>> scope of 5321.  Same for the details of any particular
>> authentication method.  What I am suggesting is not that level
>> of detail.
>> 
>> I know you can't turn back the clock and make radical changes
>> to SMTP, but in revising the spec, we should do everything we
>> can to make up for the original deficiency in the HELO command
>> and the lack of attention to security issues.
>
>I think that, had you made this point at the time (others did,
>more or less), the DRUMS WG would have told you that the changes
>from the language of 821 about that field to the requirement for
>a resolvable FQDN _and_ the addition of the IP address literal
>option for SMTP clients that didn't reliably know their own
>names, was "everything we can to make up for...".  It seems to
>me that you are looking for something else entirely, either to
>push the Domain name that is expected to appear in that field
>beyond its plausible limits (unless you are willing to start
>prohibiting some legitimate cases) or to replace the Domain name
>with an authentication token or identifier.  And that is why I
>suggested in an earlier note that, if you or others really think
>that would be useful, you start writing an extension spec for an
>authenticator that would supply the information you want, rather
>than trying to overload it onto EHLO.  Of course, you'd probably
>then have to adopt a policy of rejecting (or assigning a very
>low score to) any message that came from a source that didn't
>adopt and support your extension, but that is how it goes with
>twenty-odd years of installed base.

Writing a new draft now would be a waste of time.  A discussion like this would be the first step, and I am not seeing in this group any desire to change the status quo, or even any recognition that there is a problem.  That is the major hurdle with all attempts to fix the identity fraud problem on the Internet.  The status quo is profitable for many, and at least acceptable to most others.  It's not a technical problem, but a problem of motivation and organization.  The technology is there if we really want it.

I appreciate your time also, and I will try to keep this discussion focused on the HELO ID.

-- Dave

Date: Sun, 25 Jan 2009 11:29:09 -0500
From: John C Klensin <[email-masked]>
To: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: Submission identifiers
Sender: [email-masked]

--On Sunday, January 25, 2009 16:27 +0100 Alessandro Vesely
<[email-masked]> wrote:

> John C Klensin wrote:
>> (2) We should not forget that there are full-service SMTP
>> clients running on portable machines, small networks, without
>> unlimited resources, and/or serving small numbers of users.
>> Forcing all of those servers to be configured to use Giant
>> Provider gateways to send mail would be... well, a very
>> significant Internet policy change.
> 
> Away from me pushing for that! Although it may seem easier to
> manage reputation in a world with only a few providers, that
> would make it hard for each of them to manage their users'
> reputations. 

Previous experience in other areas is that some of them would
equate "good reputation" with "pays our bills on time" and that
a free service, if forced to take any significant responsibility
for their users (like knowing who they are), would cease being
free.

> Roughly, I'd say there should be an optimum when
> the number of global ESPs more or less equals the number of
> users per single ESP.

Hmm.  I don't know what the current hotmail/Live, Yahoo, AOL, or
Gmail numbers are, but it one took an average of them to get a
target for total number of email providers...  :-).  And it is
not at all clear to me that one should equate an SMTP server
that supports a handful of closely-related people/accounts or a
small enterprise should be equated with an ESP for all purposes.

As with the DNS, there is a huge administrative and
policy-responsibility difference between a zone/registry
operator or administrator who maintains a domain in which
substantially all of the records are delegated to entities over
which they have little control or responsibility other than
through registration contracts and a zone/registry operator that
serves a single enterprise (even if there are delegations to
departments or other entities within that enterprise).   In both
cases, the same DNS protocols are used, just as the same SMTP
protocol is used in ours.  Another similarity is that the big,
highly-delegated registries tend to need extensive and
specialized registration and registry-maintenance tools (while
small and homogeneous zones are still often managed with text
editors) and a large ESP tends to need to worry about
sophisticated internal mail-management strategies, machine
clusters, separate sending and receiving machines, mailstores
that are not on the immediate receiving machine, etc.   Writing
a comprehensive set of rules that would apply to both groups
--for either the DNS or mail case-- is probably not possible,
even without a lot of special cases.

I think there is actually another lesson about reputations tied
to identifiers in the DNS situation.  In theory, one could have
a DNS regime in which obtaining a registration from an entity
outside one's household or enterprise zone would require
convincing proof of identity, guarantees (through credentials,
established external reputation, bonding, evidence of insurance,
or equivalent) of good behavior, and even showing of a natural
relationship between one's personal or enterprise identity and
the proposed registered names.  
Indeed, to differing degrees, some TLDs used to do just that.
But, in modern times, the privacy interests didn't like those
models very much and the commercial ones --especially those who
were interested in selling as many names as possible while
minimizing costs-- liked the idea even less.  

Unless one involves governments in a big way (and, for some
governments, maybe not then), I'm not sure why an identity and
reputation system would not go down exactly the same path.

>... 
> A real example: I receive "EHLO fk-out-0910.google.com" rather
> than "EHLO gmail.com". The latter would match the DKIM domain
> in the body. The former correctly resolves to a block of 8
> addresses, one of which is the one being used. That host does
> not pass SPF HELO checks. Google's staff certainly have more
> than moderate competence. Since they don't backscatter, they
> probably assume that passing SPF MAILFROM is enough,
> notwithstanding those who think SPF is better at HELO (doesn't
> break forwarding.)
> 
> Another commonly used SMTP-level check is greylisting, which
> also has troubles trying to match those fine grained FQDNs.

Two very general observations:

(1) You have just explained --whether you intended to or not--
why it is much better to tie policy information to the domain in
the MAIL command than to the one in the EHLO/HELO command.  In
your example above, Google is behaving in an exactly correct
way, consistent with the spec: EHLO identifies the transmitting
host, not the address of the mail sender.  If that fouls up some
verification protocol, that is a deficiency in the latter.

(2) Even more generally, the various efforts to design and
specify verification and validation procedures for email really
need to start, IMO, with an understanding that the current specs
and installed base are a given and that a combination of the
robustness principle and operation experience prohibits action
based on a possible protocol-lawyer's reading of those specs.
As soon as one goes down the path of wanting to change,
restrict, or reinterpret what the specs say about particular
fields, you are restricting the applicability of your method to
a select group of friends at best or falling into one of the
more obvious "Anti-spam Kook" traps of requiring universal
deployment of your "improvement".  You have the perfect right to
reject any mail that doesn't conform to your ideas of what an
envelope should look like (with or without support from the
specs), just as you have the perfect right to reject any message
with Chinese content on the grounds that you don't read Chinese
so it couldn't possibly be for you and is therefore probably
either malware or just unwanted.  But either path is going to
reject legitimate messages from folks who, through no fault of
their own, don't conform to your expectations.   Personally, I
don't think that is a good idea.  On that subject, YMMD.  But
making significant changes in the standards, changes that would
invalidate many or most existing conforming implementations, to
reflect your ideas of how things would work in a better world...
well, probably isn't going to happen.

>...
>>    If one assumes a sender of moderate (or greater)
>> competence who is a Bad Guy trying to trick a recipient system
>> that doesn't want his traffic into processing and accepting
>> it, then domain names, IP addresses, etc., just aren't going
>> to be good enough.
> 
> That's after that prohibition in 4.1.4. I don't know _when_ it
> will be savvy to reverse it.

Nope.  The prohibition is designed to protect a lot of
legitimate cases and is consistent with the semantics of that
field in 2821, 5321, and 1123's interpretation of 1123.    It
appears to me that what you are asking for is that we change the
semantics of the field and, having made that change, remove the
prohibition in order to make the changed semantics more useful.
See above.

> Since we're talking about banning HELO for relays, I suggest
> it may be a good occasion to also modify its argument
> slightly. As someone pointed out, we would need a way to
> verify it. For global Internet, that implies looking up the
> identifier in the DNS, in order to match the connecting IP
> address. SPF is an example of an alternative to an A record.
> I'm not trying to push SPF here, really. I'm suggesting that
> the requirement that "the argument clause contains the
> fully-qualified domain name of the SMTP client" could be
> relaxed by only requiring that the name matches the address of
> the client in some standard way. It should consistently
> identify all hosts of an ADMD when they connect to external
> hosts. Semantically, that is equivalent to overloading the
> Standardized-tag, that I already proposed.

As I said, big change in semantics (we differ about "slightly"),
especially since, while it is convenient to talk about ADMDs for
some purposes, SMTP has no such concept (and, if you don't
understand the policy implications of the X.400 definition of
ADMD, you probably should, because it is very much tied up with
my comments about types of ESPs above).

But the big problem is, again, transition, since a server would
have no way to tell whether a client was legitimately and
appropriately providing a name under the old rules or under your
new ones.   And that takes me back to my earlier suggestion: if
you want this information, define a new extension or keyword for
doing it, one whose argument semantics you really can control
and one whose use would explicitly invoke your rules and
interpretations.  Although I'd predict that you'd have trouble
getting it off the ground, another possibility would be to
propose to replace EHLO with a validated-hello command, VHLO,
which would be just like EHLO except for support for a different
argument architecture.  It would take me about ten minutes to
write an I-D specifying that.  Of course, it would take me even
less time to shoot the idea down on the basis of deployment
costs, etc., but, again, your mileage, and assumptions, may
differ.

    john

Date: Sun, 25 Jan 2009 16:27:12 +0100
From: Alessandro Vesely <[email-masked]>
To: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: Submission identifiers
Sender: [email-masked]

John C Klensin wrote:
> (2) We should not forget that there are full-service SMTP
> clients running on portable machines, small networks, without
> unlimited resources, and/or serving small numbers of users.
> Forcing all of those servers to be configured to use Giant
> Provider gateways to send mail would be... well, a very
> significant Internet policy change.

Away from me pushing for that! Although it may seem easier to manage 
reputation in a world with only a few providers, that would make it 
hard for each of them to manage their users' reputations. Roughly, I'd 
say there should be an optimum when the number of global ESPs more or 
less equals the number of users per single ESP.

>> The EHLO name has better be unique for tracing and debugging.
>> OTOH, it has better be "meaningful" for filtering and
>> reputation management. Do these two kind of activities somehow
>> match the scenarios implied by that split?
> 
> Not sure.  But let me suggest an orthogonal distinction.  If one
> assumes a sender of moderate competence acting in good
> conscience and trying to make things work as well as possible,
> the EHLO name, provides useful information for tracking down and
> debugging, e.g., mail system failures, especially if it is used
> in conjunction with other information that requires the same
> assumptions.

A real example: I receive "EHLO fk-out-0910.google.com" rather than 
"EHLO gmail.com". The latter would match the DKIM domain in the body. 
The former correctly resolves to a block of 8 addresses, one of which 
is the one being used. That host does not pass SPF HELO checks. 
Google's staff certainly have more than moderate competence. Since 
they don't backscatter, they probably assume that passing SPF MAILFROM 
is enough, notwithstanding those who think SPF is better at HELO 
(doesn't break forwarding.)

Another commonly used SMTP-level check is greylisting, which also has 
troubles trying to match those fine grained FQDNs.

Curiously, it is much easier for small providers to put A records
on the domain name that is also used smoothly for other checks. Thus, 
it seems that fine grained identifiers introduce a discontinuity 
w.r.t. the provider's network size. The opposite of fact (2) above.

>    If one assumes a sender of moderate (or greater)
> competence who is a Bad Guy trying to trick a recipient system
> that doesn't want his traffic into processing and accepting it,
> then domain names, IP addresses, etc., just aren't going to be
> good enough.

That's after that prohibition in 4.1.4. I don't know _when_ it will be 
savvy to reverse it.

Since we're talking about banning HELO for relays, I suggest it may be 
a good occasion to also modify its argument slightly. As someone 
pointed out, we would need a way to verify it. For global Internet, 
that implies looking up the identifier in the DNS, in order to match 
the connecting IP address. SPF is an example of an alternative to an A 
record. I'm not trying to push SPF here, really. I'm suggesting that 
the requirement that "the argument clause contains the fully-qualified 
domain name of the SMTP client" could be relaxed by only requiring 
that the name matches the address of the client in some standard way. 
It should consistently identify all hosts of an ADMD when they connect 
to external hosts. Semantically, that is equivalent to overloading the 
Standardized-tag, that I already proposed.

Date: 25 Jan 2009 14:31:59 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: Dotless domains
Organization: 
Cc: [email-masked]
Sender: [email-masked]

> IMHO, there should be an RFC explaining that use of a TLD as a
> terminal domain name was never intended to work, and should not be
> expected to be supported by the Internet architecture or software.

Sounds good to me, although there is the pesky detail that 2821
ambiguously and 5321 unambigously says that they do.

Since 4409 that says that an unqualified name is often treated
relative to the local domain, perhaps an update to both 5321 and 4409
would be in order, specifically saying that in view of longstanding
experience, allowing undotted names was an error.

I'd suggest it should return to the 2821 ABNF, and say that in both
SMTP and SUBMIT, multi-component domain names are absolute.  In
SUBMIT, servers may either treat unqualified names as relative or
reject them. SMTP rejects them.

>ICANN has gone off the deep end.

News flash! Sun to rise in east!

R's,
John

Date: Sun, 25 Jan 2009 08:51:33 -0500
From: John C Klensin <[email-masked]>
To: Steve Atkins <[email-masked]>, SMTP Interest Group <[email-masked]>
Subject: Re: Dotless domains
Sender: [email-masked]

--On Saturday, January 24, 2009 18:13 -0800 Steve Atkins
<[email-masked]> wrote:

> On Jan 24, 2009, at 6:01 PM, John C Klensin wrote:
> 
>> --On Sunday, January 25, 2009 12:30 +1100 Mark Andrews
>> <[email-masked]> wrote:
>> 
>>> 	Dotless hostnames are in the local namespace and can *never*
>>> 	be made to work *reliably* in a global context.
>>> 
>>> 	Note the use of non-heirachical names is undoing the changes
>>> 	introduced by RFC 921 and will introduce problems RFC 921
>>> 	was trying to remove/prevent.
>> 
>> Yeah.  Since I have not been able to find a single hint in
>> ICANN's new TLD plans that those TLDs would be restricted to
>> delegation-only uses, tell it to ICANN.  Or tell it to
>> whomever is supposed to be supplying adult supervision to
>> ICANN  :-(
>> 
>> Not an SMTP problem.  SMTP requires FQDNs, without exception,
>> and does not permit single-component ones.
> 
> 5321 explicitly says otherwise  ("A domain name (or often just
> a "domain") consists of one or more components, separated by
> dots if more than one appears. In the case of a top-level
> domain used by itself in an email address, a single string is
> used without any dots.").
> 
> The whole point of this thread is to dig a little deeper into
> why that decision was made and what the implications are, I
> think.

Arggh.  At a minimum, we missed a place where the document
should have been made completely clear and consistent.

Aside: While I strongly agree with the decision --with both 2821
and 5321-- to not make more changes than were necessary, one
result is that we have ended up with nearly the same
information, stated differently, in multiple places.  The
problem actually goes back to the 821 decision to describe the
commands and the details of their operations separately, but
accretion has made it much worse.  If I saw any hope of getting
it right and consistent with the current spec, a rewrite from
scratch to produce a more compact document with no repeated
information would be really attractive, especially since the
result would probably be only half or 2/3 the length of 5321.

    best,
      john

Date: Sun, 25 Jan 2009 08:45:20 -0500
From: John C Klensin <[email-masked]>
To: Keith Moore <[email-masked]>, John Levine <[email-masked]>
cc: [email-masked]
Subject: Re: Dotless domains
Sender: [email-masked]

--On Saturday, January 24, 2009 21:02 -0500 Keith Moore
<[email-masked]> wrote:

> 
> IMHO, there should be an RFC explaining that use of a TLD as a
> terminal domain name was never intended to work, and should
> not be expected to be supported by the Internet architecture
> or software.

You know where to find the virtual pencil.  I would certainly
support such a document, but the real support would need to come
from DNS folks.

    john

Date: Sun, 25 Jan 2009 14:03:58 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Submission identifiers
Sender: [email-masked]

[email-masked] writes:
> At 09:45 24-01-2009, David MacQuigg wrote:
>> Nobody I know makes that assumption.  A more reasonable assumption is 
>> that the Bad Guys won't be able to use the identity of a Good Guy 
>> who is honest and competent.
>
> If we were to have such an identity, we would need a way to verify it.

IMO, we'd need only a way to check whether it is in a set of such 
identities (namely the set of guys we consider good).

But I don't care very much even about that. To my mind, requiring EHLO 
from clients is really just formalising a requirement which (_AFAICT_) 
most implementers think has been there since 2821 was published, or 
even a while before that, perhaps since MIME came into use.

Why it's good or why it's bad is IMO insignificant. 2821 has said "use 
ehlo and use an FQDN" for a long time and it doesn't seem to have been 
much of a problem.

If it turns out that we get some sort of advantage from requiring FQDNs, fine.

Arnt

To: John C Klensin <[email-masked]>
Cc: John Levine <[email-masked]>, [email-masked]
From: Mark Andrews <[email-masked]>
Subject: Re: Dotless domains 
Date: Sun, 25 Jan 2009 23:44:27 +1100
Sender: [email-masked]

In message <[email-masked]>, John C Klensin writes:
> 
> 
> 
> --On Sunday, January 25, 2009 12:30 +1100 Mark Andrews
> <[email-masked]> wrote:
> 
> > 	Dotless hostnames are in the local namespace and can *never*
> > 	be made to work *reliably* in a global context.
> > 
> > 	Note the use of non-heirachical names is undoing the changes
> > 	introduced by RFC 921 and will introduce problems RFC 921
> > 	was trying to remove/prevent.
> 
> Yeah.  Since I have not been able to find a single hint in
> ICANN's new TLD plans that those TLDs would be restricted to
> delegation-only uses, tell it to ICANN.  Or tell it to whomever
> is supposed to be supplying adult supervision to ICANN  :-(
> 
> Not an SMTP problem.  SMTP requires FQDNs, without exception,
> and does not permit single-component ones.
> 
>     john

	But it is a problem for SUBMIT.

4.2.  Ensure All Domains are Fully-Qualified

   The MSA MUST ensure that all domains in the envelope are fully-
   qualified.

   If the MSA examines or alters the message text in way, except to add
   trace header fields [SMTP-MTA], it MUST ensure that all domains in
   address header fields are fully-qualified.

   Reply code 554 is to be used to reject a MAIL FROM, RCPT TO, or DATA
   command which contains improper domain references.

   NOTE:  A frequent local convention is to accept single-level domains
   (for example, 'sales') and then to expand the reference by adding the
   remaining portion of the domain name (for example, to

Gellens & Klensin           Standards Track                     [Page 6]

RFC 2476                   Message Submission              December 1998

   'sales.example.net').  Local conventions that permit single-level
   domains SHOULD reject, rather than expand, incomplete multi-level
   domains, since such expansion is particularly risky.

> 
> 
> 
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

Date: Sat, 24 Jan 2009 20:37:40 -0800
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Submission identifiers
Sender: [email-masked]

At 09:45 24-01-2009, David MacQuigg wrote:
>I disagree.  If a sender was expected to declare its identity (not a 
>machine name, but the identity by which the world will judge the 
>sender's reputation), then SMTP could say "good enough", the rest is 
>up to others.

Even if we wanted to have the HELO provide an identity along the 
lines you described, we would not be able to make the change as the 
protocol is widely deployed.

>Nobody I know makes that assumption.  A more reasonable assumption 
>is that the Bad Guys won't be able to use the identity of a Good Guy 
>who is honest and competent.

If we were to have such an identity, we would need a way to verify 
it.  As we step back and analyze the "flaws" of SMTP, we see that 
they are there for a reason.  What makes SMTP robust is that it works 
for a wide range of use-cases.  Using an identify in ways discussed 
in this thread is somewhat at odds with decisions taken for RFC 5321 
(see long discussion about MX records in the archive).  Section 4.1.3 
of RFC 5321 takes into account the problems for SMTP client to 
determine how to identify itself.  If you want a strong identity, 
you'll have to deal with all that.

Regards,
-sm 

From: Steve Atkins <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: Dotless domains
Date: Sat, 24 Jan 2009 18:13:51 -0800
Sender: [email-masked]

On Jan 24, 2009, at 6:01 PM, John C Klensin wrote:

>
>
>
> --On Sunday, January 25, 2009 12:30 +1100 Mark Andrews
> <[email-masked]> wrote:
>
>> 	Dotless hostnames are in the local namespace and can *never*
>> 	be made to work *reliably* in a global context.
>>
>> 	Note the use of non-heirachical names is undoing the changes
>> 	introduced by RFC 921 and will introduce problems RFC 921
>> 	was trying to remove/prevent.
>
> Yeah.  Since I have not been able to find a single hint in
> ICANN's new TLD plans that those TLDs would be restricted to
> delegation-only uses, tell it to ICANN.  Or tell it to whomever
> is supposed to be supplying adult supervision to ICANN  :-(
>
> Not an SMTP problem.  SMTP requires FQDNs, without exception,
> and does not permit single-component ones.

5321 explicitly says otherwise  ("A domain name (or often just a  
"domain") consists of one or more components, separated by dots if  
more than one appears. In the case of a top-level domain used by  
itself in an email address, a single string is used without any dots.").

The whole point of this thread is to dig a little deeper into why that  
decision was made and what the implications are, I think.

Cheers,
   Steve

Date: Sat, 24 Jan 2009 21:02:38 -0500
From: Keith Moore <[email-masked]>
To: John Levine <[email-masked]>
CC: [email-masked]
Subject: Re: Dotless domains
Sender: [email-masked]

IMHO, there should be an RFC explaining that use of a TLD as a terminal
domain name was never intended to work, and should not be expected to be
supported by the Internet architecture or software.

ICANN has gone off the deep end.

Keith

Date: Sat, 24 Jan 2009 21:01:39 -0500
From: John C Klensin <[email-masked]>
To: Mark Andrews <[email-masked]>, John Levine <[email-masked]>
cc: [email-masked]
Subject: Re: Dotless domains
Sender: [email-masked]

--On Sunday, January 25, 2009 12:30 +1100 Mark Andrews
<[email-masked]> wrote:

> 	Dotless hostnames are in the local namespace and can *never*
> 	be made to work *reliably* in a global context.
> 
> 	Note the use of non-heirachical names is undoing the changes
> 	introduced by RFC 921 and will introduce problems RFC 921
> 	was trying to remove/prevent.

Yeah.  Since I have not been able to find a single hint in
ICANN's new TLD plans that those TLDs would be restricted to
delegation-only uses, tell it to ICANN.  Or tell it to whomever
is supposed to be supplying adult supervision to ICANN  :-(

Not an SMTP problem.  SMTP requires FQDNs, without exception,
and does not permit single-component ones.

    john

To: John Levine <[email-masked]>
Cc: [email-masked]
From: Mark Andrews <[email-masked]>
Subject: Re: Dotless domains 
Date: Sun, 25 Jan 2009 12:30:46 +1100
Sender: [email-masked]

In message <[email-masked]>, John Levine writes:
> 
> A friend* wonders about the SMTP treatment of addresses in TLDs.
> Historically there have been a smattering of such addresses, most
> famously n@ai, and there are 21 TLDs with MX records, but spot
> checking shows that most appear to be config mistakes, with the MTAs
> not actually accepting mail to postmaster@tld.
> 
> If ICANN does what they claim they're going to do and starts selling
> thousands of new TLDs, we're likely to see a lot more dotless
> addresses.  bob@aol, anyone?
> 
> As best I understand it, the prose description of addresses in 2821
> allowed TLDs in addresses, but the ABNF didn't.  In 5321 the ABNF
> changed to match the prose.  Although the DNS allows you to put a dot
> at the end of a domain name to make it clear that it's an absolute
> address, SMTP has never allowed that.
> 
> My impression is that in the real world, mail addressed to
> postmaster@va is far more likely to be rewritten to
> [email-masked] than delivered to the Vatican (which, as it
> happens, will accept it.)  Mail addressed to postmater@va. with a dot
> isn't valid under 821, 2821, or 5321, so some MTAs accept it, some
> reject it.
> 
> Is the current situation with TLDs deliberate, or was the change in
> 5321 just aesthetic tidying up?  Is there any reason not to permit a
> trailing dot in 5321-bis to bring it in line with DNS rules?
> 
> Regards,
> John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies
> ",
> Information Superhighwayman wanna-be, http://www.johnlevine.com, ex-Mayor
> "More Wiener schnitzel, please", said Tom, revealingly.
> 
> * - no, really

	Dotless hostnames are in the local namespace and can *never*
	be made to work *reliably* in a global context.

	Note the use of non-heirachical names is undoing the changes
	introduced by RFC 921 and will introduce problems RFC 921
	was trying to remove/prevent.

	Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

Date: Sat, 24 Jan 2009 20:27:15 -0500
From: John C Klensin <[email-masked]>
To: John Levine <[email-masked]>, [email-masked]
Subject: Re: Dotless domains
Sender: [email-masked]

--On Sunday, January 25, 2009 0:14 +0000 John Levine
<[email-masked]> wrote:

> 
> A friend* wonders about the SMTP treatment of addresses in
> TLDs. Historically there have been a smattering of such
> addresses, most famously n@ai, and there are 21 TLDs with MX
> records, but spot checking shows that most appear to be config
> mistakes, with the MTAs not actually accepting mail to
> postmaster@tld.
> 
> If ICANN does what they claim they're going to do and starts
> selling thousands of new TLDs, we're likely to see a lot more
> dotless addresses.  bob@aol, anyone?
> 
> As best I understand it, the prose description of addresses in
> 2821 allowed TLDs in addresses, but the ABNF didn't.  In 5321
> the ABNF changed to match the prose.  Although the DNS allows
> you to put a dot at the end of a domain name to make it clear
> that it's an absolute address, SMTP has never allowed that.

Correct on all counts.  In the 2821bis work, the possibility of
changing the syntax to match the prose was considered too.  That
change was actually made to the document at one point.  The
group then decided it was a bad idea and the change was un-done.

> My impression is that in the real world, mail addressed to
> postmaster@va is far more likely to be rewritten to
> [email-masked] than delivered to the Vatican
> (which, as it happens, will accept it.)  Mail addressed to
> postmater@va. with a dot isn't valid under 821, 2821, or 5321,
> so some MTAs accept it, some reject it.

Yes.  And, again, the decision to leave it that way was quite
extensively reviewed and discussed during the run-up to 5321.

FWIW, we did explain the situation to ICANN at the time, and
took a little poll to find out which TLDs thought they might be
hurt if there was no way to send mail from some conforming
clients to mailboxname@TLD.  We found out that a few TLDs tried
to permit it (as you obviously know already).  But no one,
including those TLDs,  seemed very concerned if it didn't work.
I have no way to know whether ICANN remembers that discussion as
they launch into the world of more TLDs than they can easily
keep track of.

> Is the current situation with TLDs deliberate, or was the
> change in 5321 just aesthetic tidying up?  Is there any reason
> not to permit a trailing dot in 5321-bis to bring it in line
> with DNS rules?

Both pieces of this -- requiring a domain name at the second
level and lower and not permitting the trailing dot -- were
explicit decisions after consideration of relevant
implementation, installed base, and operational issues.

It is not a matter of DNS rules, either.  The DNS specs (and
1123) say, essentially, "there has to be a way to distinguish an
FQDN from a short name, and a trailing period is such a way".
The SMTP spec says (again, not in these words) "if it is a
domain name, and you put it on the wire, it MUST be an FQDN".
So there is no need for the trailing period in SMTP and it is
conformant with the DNS rules.  That is, of course, not a reason
to prohibit it.  Others can probably explain that one better
than my memory permits right now, but I think history and
existing implementations had a lot to do with it.

regards,
   john

Date: 25 Jan 2009 00:14:18 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Dotless domains
Organization: 
Sender: [email-masked]

A friend* wonders about the SMTP treatment of addresses in TLDs.
Historically there have been a smattering of such addresses, most
famously n@ai, and there are 21 TLDs with MX records, but spot
checking shows that most appear to be config mistakes, with the MTAs
not actually accepting mail to postmaster@tld.

If ICANN does what they claim they're going to do and starts selling
thousands of new TLDs, we're likely to see a lot more dotless
addresses.  bob@aol, anyone?

As best I understand it, the prose description of addresses in 2821
allowed TLDs in addresses, but the ABNF didn't.  In 5321 the ABNF
changed to match the prose.  Although the DNS allows you to put a dot
at the end of a domain name to make it clear that it's an absolute
address, SMTP has never allowed that.

My impression is that in the real world, mail addressed to
postmaster@va is far more likely to be rewritten to
[email-masked] than delivered to the Vatican (which, as it
happens, will accept it.)  Mail addressed to postmater@va. with a dot
isn't valid under 821, 2821, or 5321, so some MTAs accept it, some
reject it.

Is the current situation with TLDs deliberate, or was the change in
5321 just aesthetic tidying up?  Is there any reason not to permit a
trailing dot in 5321-bis to bring it in line with DNS rules?

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, ex-Mayor
"More Wiener schnitzel, please", said Tom, revealingly.

* - no, really

Date: Sat, 24 Jan 2009 10:45:07 -0700
To: John C Klensin <[email-masked]>, [email-masked]
From: David MacQuigg <[email-masked]>
Subject: Re: Submission identifiers
Sender: [email-masked]

At 10:15 AM 1/24/2009 -0500, John C Klensin wrote:
>--On Saturday, January 24, 2009 15:37 +0100 Alessandro Vesely wrote:
>>...
>>> Keep in mind that there are things that send SMTP that don't
>>> know their own names, may know there own addresses only in
>>> private address space behind a NAT, and have no way to anchor
>>> SPF information even if they wanted to.
>> 
>> I think the question is whether we want those things to be
>> able to send to any host but their configured gateways. Since
>> the split between submitters and relayers, time should be
>> mature for requiring the latter ones to behave somewhat
>> decently. Most of them actually do.
>
>And that was exactly why my initial note tried to say "if there
>is a sender that can't follow the rules because it doesn't have
>enough information, it should be using Submit".  I think our
>preferences there are the same, and Randy and I didn't do RFCs
>2476 and 4409 simply because we wanted to raise our RFC-count.  
>
>For better or worse, two operational realities intrude on going
>very far in that direction:
>
>(1) Many of those "configured gateways" aren't configured and
>implemented to support Submit -- in the sense of either not
>supporting 587 or not providing the "if you get trash, either
>reject the submission or fix it so that anything you push onto
>the public Internet meets the spec and intent of 5321/5322"
>functionality that Submit expects.

A "gateway" (MSA) can be much simpler than this.  All it takes is some way to avoid the MSA being used as an open relay.  587 is nice, but for operators who don't have that level of expertise, there are simpler ways.

>(2) We should not forget that there are full-service SMTP
>clients running on portable machines, small networks, without
>unlimited resources, and/or serving small numbers of users.
>Forcing all of those servers to be configured to use Giant
>Provider gateways to send mail would be... well, a very
>significant Internet policy change.

I would say "expecting" not "forcing", and I don't see any need for a Giant Provider.  People who operate radio transmitters are forced by law to abide by regulations and pass tests ensuring their competence.  People who operate Internet transmitters should be expected to be honest and competent.  That expectation can and should be established by a standards group like IETF.

I understand the principle that the Internet should be free and open, but as you say, realities intrude.  I would gladly trade the freedom to operate a transmitter from my digital camera for the benefit of getting rid of incompetent and dishonest operators.

>> The EHLO name has better be unique for tracing and debugging.
>> OTOH, it has better be "meaningful" for filtering and
>> reputation management. Do these two kind of activities somehow
>> match the scenarios implied by that split?
>
>Not sure.  But let me suggest an orthogonal distinction.  If one
>assumes a sender of moderate competence acting in good
>conscience and trying to make things work as well as possible,
>the EHLO name, provides useful information for tracking down and
>debugging, e.g., mail system failures, especially if it is used
>in conjunction with other information that requires the same
>assumptions.    If one assumes a sender of moderate (or greater)
>competence who is a Bad Guy trying to trick a recipient system
>that doesn't want his traffic into processing and accepting it,
>then domain names, IP addresses, etc., just aren't going to be
>good enough.

I disagree.  If a sender was expected to declare its identity (not a machine name, but the identity by which the world will judge the sender's reputation), then SMTP could say "good enough", the rest is up to others.

The reason we have such a mess right now is that everyone who could do their part to fix things says the problem is hopeless, we can't do anything because those others won't do their part.

>Of course, if one can assume that all Bad Guys are going to be stupid and lazy, the equation changes.

Nobody I know makes that assumption.  A more reasonable assumption is that the Bad Guys won't be able to use the identity of a Good Guy who is honest and competent.

-- Dave

Date: Sat, 24 Jan 2009 10:15:49 -0500
From: John C Klensin <[email-masked]>
To: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: Submission identifiers
Sender: [email-masked]

--On Saturday, January 24, 2009 15:37 +0100 Alessandro Vesely
<[email-masked]> wrote:

>...
>> Keep in mind that there are things that send SMTP that don't
>> know their own names, may know there own addresses only in
>> private address space behind a NAT, and have no way to anchor
>> SPF information even if they wanted to.
> 
> I think the question is whether we want those things to be
> able to send to any host but their configured gateways. Since
> the split between submitters and relayers, time should be
> mature for requiring the latter ones to behave somewhat
> decently. Most of them actually do.

And that was exactly why my initial note tried to say "if there
is a sender that can't follow the rules because it doesn't have
enough information, it should be using Submit".  I think our
preferences there are the same, and Randy and I didn't do RFCs
2476 and 4409 simply because we wanted to raise our RFC-count.  

For better or worse, two operational realities intrude on going
very far in that direction:

(1) Many of those "configured gateways" aren't configured and
implemented to support Submit -- in the sense of either not
supporting 587 or not providing the "if you get trash, either
reject the submission or fix it so that anything you push onto
the public Internet meets the spec and intent of 5321/5322"
functionality that Submit expects.

(2) We should not forget that there are full-service SMTP
clients running on portable machines, small networks, without
unlimited resources, and/or serving small numbers of users.
Forcing all of those servers to be configured to use Giant
Provider gateways to send mail would be... well, a very
significant Internet policy change.

> The EHLO name has better be unique for tracing and debugging.
> OTOH, it has better be "meaningful" for filtering and
> reputation management. Do these two kind of activities somehow
> match the scenarios implied by that split?

Not sure.  But let me suggest an orthogonal distinction.  If one
assumes a sender of moderate competence acting in good
conscience and trying to make things work as well as possible,
the EHLO name, provides useful information for tracking down and
debugging, e.g., mail system failures, especially if it is used
in conjunction with other information that requires the same
assumptions.    If one assumes a sender of moderate (or greater)
competence who is a Bad Guy trying to trick a recipient system
that doesn't want his traffic into processing and accepting it,
then domain names, IP addresses, etc., just aren't going to be
good enough.  Of course, if one can assume that all Bad Guys are
going to be stupid and lazy, the equation changes.

    john

Date: Sat, 24 Jan 2009 09:47:46 -0500
From: John C Klensin <[email-masked]>
To: David MacQuigg <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

--On Saturday, January 24, 2009 6:23 -0700 David MacQuigg
<[email-masked]> wrote:

> At 05:53 PM 1/23/2009 -0500, John C Klensin wrote:
>> --On Friday, January 23, 2009 14:32 -0700 David MacQuigg
>> wrote:
>> 
>>> ...
>>> I have yet to see any sensible use-case that would be
>>> complicated by a requirement that the HELO/EHLO name end in a
>>> registered, verifiable domain name.
>>> ...
>> 
>> David,
>> 
>> Today, under 5321, EHLO clearly requires a fully-qualified
>> domain name that can be resolved unless the host doesn't know
>> its name and has to use an IP address, a situation that the
>> spec doesn't encourage.  My reading of 5321 (but not
>> necessarily 821) says that HELO does too.
> 
> Perhaps it is just a matter of emphasis.  As I read the spec
> it *does* seem to encourage using HELO/EHLO names that are
> useless for authentication.  For example, the language in
> 4.1.4 "if the verification fails, the server MUST NOT refuse
> to accept a message" puts the burden on the receiver to deal
> with mis-configured transmitters.  Receivers are expected to
> accept mail from transmitters that say "HELO this is Jupiter".

Sigh.  Read more carefully...

(1) It says, quite deliberately, "MUST NOT refuse to accept a
message on that basis".  That is _very_ different from the
excerpt you quote.

(2) The place where the names get "used", i.e., the SMTP client,
is covered by the strongest type of statement that we can make
in a standard:

	"The SMTP client MUST, if possible, ensure that the
	domain parameter to the EHLO command is a primary host
	name as specified for this command in Section 2.3.5."

The key case that justifies "if possible" is mentioned in the
next sentence, but there are others.  2.3.5 is very explicit,
arguably to the point of tedium -- the string must be
fully-qualified (no local names or incomplete strings permitted)
and it must be resolvable (by any plausible reading "in the
public DNS" -- and I wrote myself a note in December to ask
where things had gotten weird enough that we had to add that
phrase).

(3) "HELO this is Jupiter" fails the syntax rule

   helo = "HELO" SP Domain CRLF

in several ways.  If you receive such a string, you are
perfectly justified, conformant, and in line with the spec to
reject it with a 501 code.   Indeed, you would be outside the
spec if you did not do so.  You wouldn't even have to figure out
whether "this" was a resolvable FQDN or not.

My imagination doesn't stretch nearly far enough to get from
those three points to "seems to encourage using HELO/EHLO names
that are useless for authentication".

That prohibition in 4.1.4 goes back to RFC 1123, which predates
2821 by nearly a dozen years.  There are a number of reasons for
it.  Some of them are subtle, some of them involve cases that we
would now prefer to see handled via submission servers and prior
agreements.  But let me give you a contemporary example that
might make the situation clear.   Suppose there is a laptop with
a full service, conforming, SMTP client on it.  It sends mail,
manages queues, etc. -- no submission server involved.  It has a
permanent name, say roaming.example.com.  However, it is not
connected at a fixed point to the public Internet.  It roams,
and may have different addresses on different networks on
different days.  If its owner is being really careful she even
uses dynamic DNS to be sure that, at any time the host is
online, its domain names points to its actual address.   Or, if
she knows that the host is used at only a half-dozen locations
and has a usual address at each, she might set up static DNS
entries that list each of them as its address.     Now suppose a
message is sent to you from that host, after which it is shut
down.  When it reappears on the network a few hours later, it is
at a different address.  If you resolve the domain name at that
time, you get a valid address for the host, but it isn't the
same address the host had when it initiated sending the message.
Do you really want to reject the message?  If you do, please
don't justify the decision on the basis that that you have done
a careful and adequate job of authentication -- _nothing_ you
can do with HELO/EHLO as they are defined in 5321 is going to
get you to a point at which that is possible.

> The purpose of changes should be to avoid lost mail, as
> opposed to rejected mail, which doesn't have any damaging
> effects on reliability.  SMTP REJECT should be encouraged as
> the proper response to doubtful requests.  (Sorry, we don't
> accept mail from unidentified transmitters.)

2821/5321 were carefully written to give you the option of
making that decision... or the decision to not accept mail from
even-numbered IP addresses for that matter.  I personally cannot
recommend it unless you have a much better notion of
"identified" than anything you can get out of HELO/EHLO.  I'd
also suggest to you that the typical user who is waiting for a
message to arrive is not likely to distinguish between
"rejected" and "lost".  Unless the former causes the sender to
get on the phone, the only issue is "didn't arrive".  Your users
and experience may differ.

>> So what are you asking for that isn't there already?
>> Elimination of the IP address literal option?  Some criteria
>> for "verifiable" that would presumably lie well outside the
>> scope of SMTP?
> 
> If you can deprecate HELO, surely you can do the same for
> address literals.

Independent of some practical issues and the fact that address
literals can sometimes actually provide useful information, that
would accomplish what?    You can already make a policy decision
to reject any message that starts with EHLO and an address
literal; doing so would be consistent with your position that
false rejects are ok as long as one doesn't drop the message.

I can't recommend that we try to dig into the question in the
standard, but consider whether you would prefer:

   EHLO [#.#.#.#]

where "#.#.#.#" is a valid, public, IP address that might even
be static and stable and have a legitimate reverse mapping to a
host name,     and

   EHLO comment.example.org

where "comment.example.org" is a valid, registered, resolvable
FQDN but the only information at its DNS node is

     IN TXT This is a mobile host and I can't really tell you
what or where it is without compromising my privacy.

You don't have much "authentication" information either way; you
might be able to extract as much or more contact information
from the first.

>> Do keep in mind that we know from experience that the bad guys
>> have absolutely no trouble obtaining perfectly valid domain
>> names.
> 
> I agree that reputation requirements should be outside the
> scope of 5321.  Same for the details of any particular
> authentication method.  What I am suggesting is not that level
> of detail.
> 
> I know you can't turn back the clock and make radical changes
> to SMTP, but in revising the spec, we should do everything we
> can to make up for the original deficiency in the HELO command
> and the lack of attention to security issues.

I think that, had you made this point at the time (others did,
more or less), the DRUMS WG would have told you that the changes
from the language of 821 about that field to the requirement for
a resolvable FQDN _and_ the addition of the IP address literal
option for SMTP clients that didn't reliably know their own
names, was "everything we can to make up for...".  It seems to
me that you are looking for something else entirely, either to
push the Domain name that is expected to appear in that field
beyond its plausible limits (unless you are willing to start
prohibiting some legitimate cases) or to replace the Domain name
with an authentication token or identifier.  And that is why I
suggested in an earlier note that, if you or others really think
that would be useful, you start writing an extension spec for an
authenticator that would supply the information you want, rather
than trying to overload it onto EHLO.  Of course, you'd probably
then have to adopt a policy of rejecting (or assigning a very
low score to) any message that came from a source that didn't
adopt and support your extension, but that is how it goes with
twenty-odd years of installed base.

     john

Date: Sat, 24 Jan 2009 15:37:06 +0100
From: Alessandro Vesely <[email-masked]>
To: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: Submission identifiers
Sender: [email-masked]

John C Klensin wrote:
> --On Friday, January 23, 2009 18:37 +0100 Alessandro Vesely
> <[email-masked]> wrote:
>> Paul Smith wrote:
>>> IMV, if it is expected by the base standard that the EHLO 
>>> data is to be used for anything other than logging/tracing, 
>>> then these considerations and more need to be looked at 
>>> carefully. 
>> 
>> How "meaningful" can a name be?
>>
>> [...] an MTA could say "EHLO SPF:example.com", if SPF were included 
>> in an extended registry of (meaningful) Address Literal Tags.
> 
> Free advice: Changing the argument to EHLO from FQDN-or-literal
> to trick-identifier is likely to be a non-starter.  A very large
> fraction of the deployed base would treat SPF:example.com (or
> anything vaguely like that) as a syntax error and reject the
> message.

I agree that it probably won't work. Of course, the sender might check 
if the server's greeting included "SPF tags accepted", or similar. 
However, that's reinventing SMTP extensions.

The receiving server may answer 504 or 550 to the EHLO command. As 
this is where clients use to try HELO instead, there is little margin 
for negotiation either.

> Consider instead an ESMTP extension command, say IAGG (I'm A
> Good Guy), that was required to be issued immediately after EHLO
> if one intended to use it.

Yeah, a proper SMTP extension would make for sounder specs. Yet, note 
that in both cases the actual specification of the identifier would 
live in its own RFC.

> It would avoid getting the new ideas about what the field might
> be used for tangled up with an installed based that believes
> that it knows what is there and what syntax it obeys.

Having the IP address of the sender seems useless, if it must be the 
same that the server already knows from the IP layer.

Could it be a MAC address? That might be useful in some DHCP-driven 
environment. Unwitting MTAs may just accept any token. However, MACs 
would break DNS checks too.

>> What real cases are there that cannot be handled in similar
>> ways? A careful analysis could help to eventually eliminate
>> the phrase (in 4.1.4) "if the verification fails, the server
>> MUST NOT refuse to accept a message on that basis."
> 
> Keep in mind that there are things that send SMTP that don't
> know their own names, may know there own addresses only in
> private address space behind a NAT, and have no way to anchor
> SPF information even if they wanted to.

I think the question is whether we want those things to be able to 
send to any host but their configured gateways. Since the split 
between submitters and relayers, time should be mature for requiring 
the latter ones to behave somewhat decently. Most of them actually do.

The EHLO name has better be unique for tracing and debugging. OTOH, it 
has better be "meaningful" for filtering and reputation management. Do 
these two kind of activities somehow match the scenarios implied by 
that split?

Reply-To: "Alex van den Bogaerdt" <[email-masked]>
From: "Alex van den Bogaerdt" <[email-masked]>
To: <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Date: Sat, 24 Jan 2009 15:17:09 +0100
Sender: [email-masked]

----- Original Message ----- 
From: "David MacQuigg" <[email-masked]>
To: "John C Klensin" <[email-masked]>; <[email-masked]>
Sent: Saturday, January 24, 2009 2:23 PM
Subject: Re: RFC 5321bis / 2821ter

>
> At 05:53 PM 1/23/2009 -0500, John C Klensin wrote:
>>--On Friday, January 23, 2009 14:32 -0700 David MacQuigg wrote:
>>
>>>...
>>> I have yet to see any sensible use-case that would be
>>> complicated by a requirement that the HELO/EHLO name end in a
>>> registered, verifiable domain name.
>>>...
>>
>>David,
>>
>>Today, under 5321, EHLO clearly requires a fully-qualified
>>domain name that can be resolved unless the host doesn't know
>>its name and has to use an IP address, a situation that the spec
>>doesn't encourage.  My reading of 5321 (but not necessarily 821)
>>says that HELO does too.
>
> Perhaps it is just a matter of emphasis.  As I read the spec it *does* 
> seem to encourage using HELO/EHLO names that are useless for 
> authentication.  For example, the language in 4.1.4 "if the verification 
> fails, the server MUST NOT refuse to accept a message" puts the burden on 
> the receiver to deal with mis-configured transmitters.  Receivers are 
> expected to accept mail from transmitters that say "HELO this is Jupiter".

It does not say that. Why did you forgot to copy "on that basis" ?

"An SMTP server MAY verify that the domain name argument in the EHLO
   command actually corresponds to the IP address of the client.
   However, if the verification fails, the server MUST NOT refuse to
   accept a message on that basis.  Information captured in the
   verification attempt is for logging and tracing purposes.  Note that
   this prohibition applies to the matching of the parameter to its IP
   address only
"

Verifying that "Jupiter" is not a FQDN is not covered by that statement.

It says: Note that this prohibition applies to the matching of the parameter 
to its IP address only.

In other words: one mailhost, two addresses and two names, names 
"primary.example.com" and "secondary.example.com".
This box connects to you, it says "EHLO primary.example.com." but after 
verification you know it is really secondary.example.com connecting to you. 
This is not an acceptable ground for rejecting a message.

But that does not mean a mailhost can just use any name it wants to. It 
still MUST (if possible) use its own primary name. My example box has no 
choice but to use primary.example.com in EHLO, eventhough it is connecting 
via the other interface.

But when a spammer uses the same EHLO parameter, and if I happen to have 
knowledge such as provided by SPF, it's perfectly okay to reject.

Date: Sat, 24 Jan 2009 06:23:43 -0700
To: John C Klensin <[email-masked]>, [email-masked]
From: David MacQuigg <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

At 05:53 PM 1/23/2009 -0500, John C Klensin wrote:
>--On Friday, January 23, 2009 14:32 -0700 David MacQuigg wrote:
>
>>...
>> I have yet to see any sensible use-case that would be
>> complicated by a requirement that the HELO/EHLO name end in a
>> registered, verifiable domain name.
>>...
>
>David,
>
>Today, under 5321, EHLO clearly requires a fully-qualified
>domain name that can be resolved unless the host doesn't know
>its name and has to use an IP address, a situation that the spec
>doesn't encourage.  My reading of 5321 (but not necessarily 821)
>says that HELO does too.

Perhaps it is just a matter of emphasis.  As I read the spec it *does* seem to encourage using HELO/EHLO names that are useless for authentication.  For example, the language in 4.1.4 "if the verification fails, the server MUST NOT refuse to accept a message" puts the burden on the receiver to deal with mis-configured transmitters.  Receivers are expected to accept mail from transmitters that say "HELO this is Jupiter".

The purpose of changes should be to avoid lost mail, as opposed to rejected mail, which doesn't have any damaging effects on reliability.  SMTP REJECT should be encouraged as the proper response to doubtful requests.  (Sorry, we don't accept mail from unidentified transmitters.)

>So what are you asking for that isn't there already?
>Elimination of the IP address literal option?  Some criteria for
>"verifiable" that would presumably lie well outside the scope of
>SMTP?

If you can deprecate HELO, surely you can do the same for address literals.

>Do keep in mind that we know from experience that the bad guys
>have absolutely no trouble obtaining perfectly valid domain
>names.

I agree that reputation requirements should be outside the scope of 5321.  Same for the details of any particular authentication method.  What I am suggesting is not that level of detail.

I know you can't turn back the clock and make radical changes to SMTP, but in revising the spec, we should do everything we can to make up for the original deficiency in the HELO command and the lack of attention to security issues.

-- Dave

Date: Fri, 23 Jan 2009 17:53:31 -0500
From: John C Klensin <[email-masked]>
To: David MacQuigg <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

--On Friday, January 23, 2009 14:32 -0700 David MacQuigg
<[email-masked]> wrote:

>...
> I have yet to see any sensible use-case that would be
> complicated by a requirement that the HELO/EHLO name end in a
> registered, verifiable domain name.
>...

David,

Today, under 5321, EHLO clearly requires a fully-qualified
domain name that can be resolved unless the host doesn't know
its name and has to use an IP address, a situation that the spec
doesn't encourage.  My reading of 5321 (but not necessarily 821)
says that HELO does too.

So what are you asking for that isn't there already?
Elimination of the IP address literal option?  Some criteria for
"verifiable" that would presumably lie well outside the scope of
SMTP?

Do keep in mind that we know from experience that the bad guys
have absolutely no trouble obtaining perfectly valid domain
names.

    john

Date: Fri, 23 Jan 2009 17:40:59 -0500
From: John C Klensin <[email-masked]>
To: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Submission identifiers (was: Re: RFC 5321bis / 2821ter)
Sender: [email-masked]

Subject line changed because this is really about a new feature,
not an appropriate clarification for a 5321-> 5321bis transition.

--On Friday, January 23, 2009 18:37 +0100 Alessandro Vesely
<[email-masked]> wrote:

> 
> Paul Smith wrote:
>> To get around [meaningless address literals], you use SMTP
>> submission with SASL (OK, that requires EHLO anyway, but the
>> EHLO name is irrelevant in that case).
>> 
>> IMV, if it is expected by the base standard that the EHLO
>> data is to be used for anything other than logging/tracing,
>> then these considerations and more need to be looked at
>> carefully.
> 
> That's apparently the split that Message Submission envisions.
> Either the client validates a name using a password, or it
> uses a globally registered, hence meaningful, name.
> 
> How "meaningful" can a name be? It shouldn't be overwhelmingly
> difficult for an MTA to put after EHLO a FQDN that can be
> resolved to its IP address. Probably the standard cannot be so
> rigid, but including careful considerations of what
> circumstances may or may not allow what names, seems a good
> idea.
> 
> David went so far as to ask for a reputation-ID after EHLO.
> I'm not sure whether that would ease or complicate setting up
> an MTA. For example, rather than a FQDN that resolves to an A
> record matching its address, an MTA could say "EHLO
> SPF:example.com", if SPF were included in an extended registry
> of (meaningful) Address Literal Tags. Presumably, example.com
> would then resolve to an SPF record that permits the same
> address. That could cope with not being able to know, say,
> which address from a NAT pool will be used for an outgoing
> connection.

Free advice: Changing the argument to EHLO from FQDN-or-literal
to trick-identifier is likely to be a non-starter.  A very large
fraction of the deployed base would treat SPF:example.com (or
anything vaguely like that) as a syntax error and reject the
message.   If you are trying to reject every message that
doesn't contain SPF (or whatever) information, there are easier
ways.

Consider instead an ESMTP extension command, say IAGG (I'm A
Good Guy), that was required to be issued immediately after EHLO
if one intended to use it.  It would give you complete freedom
to specify its arguments (without being limited by EHLO's
syntax) and to decide how they would be interpreted, what should
occur if the command was not present, and what should occur if
you didn't like the arguments.  If you didn't like the extra
turnaround, do it as an extension parameter to MAIL (and
possibly to HELP, VRFY, and EXPN if you meaningfully support
those at all).

It would avoid getting the new ideas about what the field might
be used for tangled up with an installed based that believes
that it knows what is there and what syntax it obeys.

> What real cases are there that cannot be handled in similar
> ways? A careful analysis could help to eventually eliminate
> the phrase (in 4.1.4) "if the verification fails, the server
> MUST NOT refuse to accept a message on that basis."

Keep in mind that there are things that send SMTP that don't
know their own names, may know there own addresses only in
private address space behind a NAT, and have no way to anchor
SPF information even if they wanted to.

     john

Date: Fri, 23 Jan 2009 14:32:15 -0700
To: Alessandro Vesely <[email-masked]>, [email-masked]
From: David MacQuigg <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

At 06:37 PM 1/23/2009 +0100, Alessandro Vesely wrote:

>Paul Smith wrote:
>>To get around [meaningless address literals], you use SMTP
>>submission with SASL (OK, that requires EHLO anyway, but the EHLO
>>name is irrelevant in that case).
>>IMV, if it is expected by the base standard that the EHLO data is
>>to be used for anything other than logging/tracing, then these
>>considerations and more need to be looked at carefully.
>
>That's apparently the split that Message Submission envisions. Either the client validates a name using a password, or it uses a globally registered, hence meaningful, name.
>
>How "meaningful" can a name be? It shouldn't be overwhelmingly difficult for an MTA to put after EHLO a FQDN that can be resolved to its IP address. Probably the standard cannot be so rigid, but including careful considerations of what circumstances may or may not allow what names, seems a good idea.
>
>David went so far as to ask for a reputation-ID after EHLO. I'm not sure whether that would ease or complicate setting up an MTA.

Just to be clear, I'm not suggesting anything that would complicate setting up an MTA, require new mechanisms, etc. (unless you mean complicate things for crooks who might have a hard time acquiring a reputable ID, or for naive users who want to send email directly from their cameras, without any reputable intermediary service.)  I have yet to see any sensible use-case that would be complicated by a requirement that the HELO/EHLO name end in a registered, verifiable domain name.  If I want to receive email directly from my coffee maker, I'll add it to my whitelist.

I am aware that many otherwise legitimate transmitters currently use invalid HELO/EHLO names, and I don't reject their mail.  I do, however, send their messages immediately to the spam filter.  If you want reliable delivery, you need a valid identity, one which we can verify in DNS, and to which we can assign reputation at any level we please.

Date: Fri, 23 Jan 2009 18:37:15 +0100
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Paul Smith wrote:
> To get around [meaningless address literals], you use SMTP
> submission with SASL (OK, that requires EHLO anyway, but the EHLO
> name is irrelevant in that case).
> 
> IMV, if it is expected by the base standard that the EHLO data is
> to be used for anything other than logging/tracing, then these
> considerations and more need to be looked at carefully.

That's apparently the split that Message Submission envisions. Either 
the client validates a name using a password, or it uses a globally 
registered, hence meaningful, name.

How "meaningful" can a name be? It shouldn't be overwhelmingly 
difficult for an MTA to put after EHLO a FQDN that can be resolved to 
its IP address. Probably the standard cannot be so rigid, but 
including careful considerations of what circumstances may or may not 
allow what names, seems a good idea.

David went so far as to ask for a reputation-ID after EHLO. I'm not 
sure whether that would ease or complicate setting up an MTA. For 
example, rather than a FQDN that resolves to an A record matching its 
address, an MTA could say "EHLO SPF:example.com", if SPF were included 
in an extended registry of (meaningful) Address Literal Tags. 
Presumably, example.com would then resolve to an SPF record that 
permits the same address. That could cope with not being able to know, 
say, which address from a NAT pool will be used for an outgoing 
connection.

What real cases are there that cannot be handled in similar ways? A 
careful analysis could help to eventually eliminate the phrase (in 
4.1.4) "if the verification fails, the server MUST NOT refuse to 
accept a message on that basis."

Date: Fri, 23 Jan 2009 11:58:38 -0500
From: John C Klensin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>, [email-masked]
cc: Willie Gillespie <[email-masked]>, Paul Smith <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Again, without arguing for one position or the other and using
Arnt's note as a framework for responding to several others...

--On Friday, January 23, 2009 13:36 +0100 Arnt Gulbrandsen
<[email-masked]> wrote:

> Paul Smith writes:
>> Doesn't HELO name the client as well?
> 
> Standards documents have to be pedantic, and there is a
> pedantic difference: 821 says HELO names the client's host
> name, 2821 says EHLO names the fully-qualified domain name.

The tighter definition of the argument in 2821/5321 is, indeed,
the key difference between the two for clients that don't want
_any_ advanced/ extended capabilities.  There are other
differences, but they have to do not with the difference in the
HELO/EHLO dialogue.  When a client sends EHLO and a server
responds positively to it, they have signed up to the much
tighter and more specific conformance language of 2821/5321
rather than the looser language (and strong expectations about
good intentions and the robustness principle) of 821.  That
signup permits both parties to make stronger assumptions about
the behavior of the other, again if one can assume good
intentions (or is willing to treat misbehavior as a sign of bad
intentions).  I probably should have been more clear, but, while
the conversation I mentioned included some discussion about
spam/ anti-spam implications and inspired my question, I didn't
mean to imply that I personally thought the change would have
any useful effect on spam-fighting or that the conversation was
about perceived bad spammer behavior rather than bad
anti-spammer behavior.

There is another, associated, possible ambiguity in the text
that we should decide whether to fix (it is less important if we
explicitly deprecate 821).  The material in 2.2.1 about the
relationship between HELO and EHLO can be read as saying that,
unless the document says something explicitly different, the
clarifications apply to HELO as well.  I think that reading was
the DRUMS intention.  Certainly the syntax given for HELO in
4.1.1.1 is consistent with that reading: it says "Domain" (which
is narrowly defined in the document) and does not invoke the
more relaxed language of 821 that has been widely interpreted as
permitted "HELO localhost".

The 1123-imposed requirement (carried forward into Section 4.1.4
Paragraph 6 of 5321) that messages not be rejected on the basis
of a validation failure with the EHLO argument would presumably
remain even if we deprecated 821 and client use of HELO.  We
could discuss changing that, but it would be a sufficient change
in requirements to require evaluation of whether a reset to
Proposed would be needed.

So, if this is worth doing at all, the major motivation would
presumably be more clarity and better interoperability among
parties with good intentions and good information.  It is not
clear to me that it would provide much pushback on those whose
intent is to sneak around the rules.  Of course, someone who
cared less about getting every possible legitimate message
through than about eliminating every possible unwanted one would
probably take advantage of the "you are not required to accept
messages" principle to return a "5yz HELO has been obsolete for
years and I see no need to talk with old servers" message... but
that would be an operational decision outside the standard and
one that could be made today.

Of course, Alessandro's suggestion of an address-range
restriction for use of HELO could be generalized and written to
take the protocol one step further in that "operational
decision" direction (without encouraging treating IP addresses
as authenticators as a side effect) by saying that HELO SHOULD
be used only by prior arrangement between consenting adults.
That approach would also weaken the "go use SUBMIT" position for
situations in which it was inappropriate.

best,
     john

p.s. to clarify something that may be a misunderstanding in some
of the messages on this thread, SUBMIT (RFC 4409) does not
require SMTP-AUTH (or Start-TLS).  What it does is to specify
that SMTP-AUTH is required "unless it has already independently
established authentication or authorization" (Section 4.3).
Presumably Start-TLS falls under the "unless" part, but so would
any other acceptable-quality mechanism.

And, while it would clearly be non-conforming and violate MUST
provisions of RFC 4409, nothing we've done or specified would
really prevent an MUA from make prior private arrangements to
communicate with a submission server (on port 587, 25, or a
non-standard/user one) using a HELO handshake (or a FOOBAR
handshake, for that matter).

Date: Fri, 23 Jan 2009 14:49:39 +0000
From: Paul Smith <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], Willie Gillespie <[email-masked]>, John C Klensin <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Arnt Gulbrandsen wrote:
> Standards documents have to be pedantic, and there is a pedantic
> difference: 821 says HELO names the client's host name, 2821 says EHLO
> names the fully-qualified domain name.
>
OK

But, I would worry about using the EHLO data for anything significant...

If you can buy a camera (for instance, using someone else's example)
which can send mail using SMTP, do you:

a) automatically use an address literal. This can easily be detected
automatically by the camera, but is pretty useless, given that most
systems will use NAT, meaning that the address detected by the camera
may not be the same as the address detected by the SMTP server, so what
can you do with something that says "EHLO [192.168.0.1]"?

b) manually use an address literal. This would fix the above problem,
but not for dynamic IP addresses

c) manually use a host name. Perfect, except it needs a host name, and
manually needs to be set up. That stops 99.99% of the world's population
being able to use that camera.

To get around that, you use SMTP submission with SASL (OK, that requires
EHLO anyway, but the EHLO name is irrelevant in that case).

IMV, if it is expected by the base standard that the EHLO data is to be
used for anything other than logging/tracing, then these considerations
and more need to be looked at carefully.

If (a) above is allowed, then that effectively defeats any extra
security provided by the EHLO data, as you can just use a private IP
address literal. If it's not, then that causes big problems. If it's
allowed in some circumstances and not others, then this needs to be
documented.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Fri, 23 Jan 2009 09:01:22 -0500
From: John C Klensin <[email-masked]>
To: David MacQuigg <[email-masked]>
cc: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

--On Friday, January 23, 2009 4:17 -0700 David MacQuigg
<[email-masked]> wrote:

> At 01:46 AM 1/23/2009 -0500, John C Klensin wrote:
> 
>> However, I had a discussion early this week with someone
>> interested in the slightly-fuzzy text about arguments to HELO
>> and/or EHLO and their relationship to spam-fighting.
> 
> The fundamental problem has always been the lack if an
> easily-verified identity associated with the mail session.  If
> you are going to make a change, it should be in the direction
> of correcting that problem.  You should make the parameter to
> the HELO/EHLO command that identity, rather than the name of a
> machine or an address literal.  We already have a
> machine-level IP address from the underlying TCP/IP protocol.
> What is needed is a higher-level identity that is 1) unique,
> 2) easily verified, and 3) useful for accumulating reputation.
> A registered domain name fulfills all three requirements.
> Machine names and IP addresses fail requirement 3. 

David, whether this is a good idea or a bad one, it would impose
entirely new requirements --and even a new mechanism-- and
therefore would be seriously out of scope for 5321bis.  As I
have suggested about other new mechanism/requirement
suggestions, please construct an I-D that is detailed enough for
people to evaluate the mechanisms and their implications and
then discuss with the ADs how it can be processed for Proposed
Standard.   It is about eight years too late for 5321bis.

    john

Date: Fri, 23 Jan 2009 04:59:26 -0800
To: John C Klensin <[email-masked]>
From: SM <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Cc: [email-masked]
Sender: [email-masked]

At 22:46 22-01-2009, John C Klensin wrote:
>However, I had a discussion early this week with someone
>interested in the slightly-fuzzy text about arguments to HELO
>and/or EHLO and their relationship to spam-fighting.  The
>discussion leads to a question, especially since 5321bis is our
>first opportunity to really do something significant.
>
>Question: Is it time to formally deprecate 821 and, in
>particular, the main feature that distinguishes it: the use of
>HELO by SMTP clients?  We would still need to require that SMTP
>servers accept it, but we would tell full-capability clients
>(including the client side of relays and gateways) that HELO is
>obsolete.  One corollary of this is that we'd be telling
>low-capability clients, particularly those that are part of MUA
>systems, that they should be talking to Submit ports, not SMTP
>ones.

RFC 2821 obsoletes RFC 821.  There is a reference to RFC 821 in RFC 
2821 and RFC 5321 for some features not in significant use 
today.  The service extension model has been around for over seven 
years and the major MTAs have implemented RFC 2821.  RFC 2821 
specifies that servers MUST support the EHLO command even if they do 
not implement any specific extensions and clients SHOULD 
preferentially utilize EHLO rather than HELO with a fallback to HELO.

I don't know how much work it would be to formally depreciate RFC 
821.  It seems the right time to look into it.  I still see traffic 
from SMTP clients which use HELO.  I don't have a strong view about 
whether HELO should be obsoleted or not.

Regards,
-sm 

Date: Fri, 23 Jan 2009 14:01:30 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Cc: John C Klensin <[email-masked]>, Matti Aarnio <[email-masked]>
Sender: [email-masked]

Matti Aarnio writes:
> Yes, this is fundamental problem in "never do plaintext 
> authentication" approach of current generation of specifications. 
> Only in some far away hidden footnotes are things like plain-text 
> login...
>
> ... which is still practical and safe, when it happens under STARTTLS.
>
> (Surprising how much "low capability" systems are able to do TLS, but 
> not SASL..)

Those that do STARTTLS also do EHLO, not HELO, so they're ok anyway.

Arnt

Date:	Fri, 23 Jan 2009 14:53:03 +0200
From: Matti Aarnio <[email-masked]>
To: [email-masked]
Cc: Arnt Gulbrandsen <[email-masked]>, John C Klensin <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

On Fri, Jan 23, 2009 at 10:36:05AM +0100, Arnt Gulbrandsen wrote:
> John C Klensin writes:
> >Question: Is it time to formally deprecate 821 and, in particular, the 
> >main feature that distinguishes it: the use of HELO by SMTP clients? 
> >We would still need to require that SMTP servers accept it, but we 
> >would tell full-capability clients (including the client side of 
> >relays and gateways) that HELO is obsolete.
> 
> I think a lot of people would answer "you mean it isn't already?"
> 
> >One corollary of this is that we'd be telling low-capability clients, 
> >particularly those that are part of MUA systems, that they should be 
> >talking to Submit ports, not SMTP ones.
> 
> I beg to differ. Low-capability clients cannot effectively talk to 
> submit, since submit tends to demand SASL support in the client.

Yes, this is fundamental problem in "never do plaintext authentication"
approach of current generation of specifications.  Only in some far
away hidden footnotes are things like plain-text login...

... which is still practical and safe, when it happens under STARTTLS.

(Surprising how much "low capability" systems are able to do TLS, but
not SASL..)

> Arnt

-- 
/Matti Aarnio	<[email-masked]>
FUNET:  Finnish Academic and Research Network
	Network Information/Software Archival Service

Date: Fri, 23 Jan 2009 13:36:55 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Cc: Willie Gillespie <[email-masked]>, John C Klensin <[email-masked]>, Paul Smith <[email-masked]>
Sender: [email-masked]

Paul Smith writes:
> Doesn't HELO name the client as well?

Standards documents have to be pedantic, and there is a pedantic 
difference: 821 says HELO names the client's host name, 2821 says EHLO 
names the fully-qualified domain name.

Arnt

Date:	Fri, 23 Jan 2009 14:32:34 +0200
From: Matti Aarnio <[email-masked]>
To: [email-masked]
Cc: John C Klensin <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

On Fri, Jan 23, 2009 at 01:46:06AM -0500, John C Klensin wrote:
> Hi.
> 
> Just as an update, I've got a first draft of this about ready to
> go, modulo the IETF Trust figuring out a way for me to post it.
> So far, it differs from 5321 only in that some changes suggested
> after Last Call (by individuals and the IESG) have been
> incorporated and there is an experiment in creating an index to
> syntax productions.
> 
> However, I had a discussion early this week with someone
> interested in the slightly-fuzzy text about arguments to HELO
> and/or EHLO and their relationship to spam-fighting.  The
> discussion leads to a question, especially since 5321bis is our
> first opportunity to really do something significant.

Validating data present in HELO or EHLO parameter would be
neat, but it is invalid in so large a share of _valid_ message
transmits, that I have had to ignore it since forever.

On the integrated usage frequency of different modes, I do have
some statistical data collected over several years in persistent
accounters within the MTA.  SNMP RRD sets are 5 years long, MTA
"SNMP" counters were last reset in July 2005.

These are publicly viewable  (http://vger.kernel.org/z/), but
I shall hilight some things:

Incoming smtp:

  SS.SMTPconnects                   55507276
  SS.SMTP_HELO                      28982564
  SS.SMTP_EHLO                      25593458

  SS.IncomingClientPipelines        19295332

Outgoing smtp:

  TA-SMTP.SmtpEHLO                 799439122
  TA-SMTP.SmtpHELO                   6839640

Some outbound systems were being sent with forced HELO mode, as
they were not receiving emails entirely successfully when sending
with EHLO greeting.  Currently no target is being explicitely
routed like that.

A 5 year SNMP trend plot on outgoing HELOs show slight decrease and
simultaneous outgoing EHLO-plot shows similar increase.

  http://vger.kernel.org/z/graphs/vger_SNMP_TA-SMTP_SmtpHELO-5year.png
  http://vger.kernel.org/z/graphs/vger_SNMP_TA-SMTP_SmtpEHLO-5year.png

That is, more and more of outgoing traffic is sent without
need to decrade the mode.

On the incoming flows..  Incoming HELOs are increasing all the time
in 5 year plot.

Also incoming client pipelining flows are increasing.
That is, while my server does support PIPELINING -mode both
incoming and outgoing, it does also note which greeting it gets
on incoming flow, and reacts accordingly.  Spammers are also
unable to handle basic multi-recipient processing rules, if
they see single non-ok response, then at least one widely deployed
spamware drops the connection.

My statistics gathering is not able to tell how much HELO-traffic
is valid non-pipelined flows.  Neither, precisely, it is able to
tell of how much are pipelined HELO spam-flows.  There is just
an obvious (to eyeball mark 1) correlation in between those two
in day period statistics.

> Question: Is it time to formally deprecate 821 and, in
> particular, the main feature that distinguishes it: the use of
> HELO by SMTP clients?  We would still need to require that SMTP
> servers accept it, but we would tell full-capability clients
> (including the client side of relays and gateways) that HELO is
> obsolete.  One corollary of this is that we'd be telling
> low-capability clients, particularly those that are part of MUA
> systems, that they should be talking to Submit ports, not SMTP
> ones.

Yes please. 

In particular the message submission via SMTP should be deprecated
in favour of authenticated SUBMIT port usage.

The issue there is that for travelling user having to submit thru
port 25 is increasingly being blocked outside ISP's own servers.
With wide-spread SUBMIT availability, travellers would be able to
send their emails thru their service provider's servers no matter
where they happen to be.

ISP:s and their users are still very much unaware of SUBMIT service,
and rarely serving it, or at least rarely documenting on how to use it.
Some may have SMTPS service, but no SUBMIT.

On the other hand, combination of SUBMIT and STARTTLS has had problems
in clients, in particular with the Outlook brand of things.
Those clients did start with SSL handshake on SUBMIT port,
if TLS was enabled and port number was anything but 25...

This could also enable stricter control on what constitutes of valid
format message, and what is not.  Things like Message-ID: -header
are practically mandatory, but at least one widely deployed MTA
product does not write it on error messages it sends...

Some MUA programs create the Message-ID, others do not, and let
the MTA to add it.  Most MTAs add it, some do not.
Some add it even if it is already present.  (Serious troubles
resulted when I tried to enforce a rule of "there is exactly one
'Message-ID:' header in incoming email.")

> I don't have a strong opinion (or even a weak one) about this
> one way or the other, nor have I looked at how much I'd have to
> change the document to implement it if it were wanted.  But,
> because this is the point at which we are proposing to really
> obsolete Full Standard 821 with a Full Standard, it seems like
> the right time to ask the question.   Since RFC 1425 was
> published sixteen years ago next month, it should not be
> unreasonable to tell people they've had enough time to get used
> to the idea of something besides HELO.
> 
>     john

-- 
/Matti Aarnio	<[email-masked]>
FUNET:  Finnish Academic and Research Network
	Network Information/Software Archival Service

Date: Fri, 23 Jan 2009 13:21:34 +0100
From: Alessandro Vesely <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

John C Klensin wrote:
> Question: Is it time to formally deprecate 821 and, in
> particular, the main feature that distinguishes it: the use of
> HELO by SMTP clients?  We would still need to require that SMTP
> servers accept it, but we would tell full-capability clients
> (including the client side of relays and gateways) that HELO is
> obsolete.  One corollary of this is that we'd be telling
> low-capability clients, particularly those that are part of MUA
> systems, that they should be talking to Submit ports, not SMTP
> ones.

Yes, it is time! What is obsolete is obsolete, and should be tagged as 
such. For example, I wouldn't want to buy, say, an "SMTP-enabled 
camera" that is not actually capable to send photos in a reliable way.

As for servers, I'd recommend that they must limit acceptance of the 
obsolete protocol to a configurable set of addresses which, in their 
default configurations, should either be empty or just include 127.0.0.1.

Thanks for continuing your effort, I'm looking forward for the I-D
Ale

Date: Fri, 23 Jan 2009 12:20:02 +0000
From: Paul Smith <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], Willie Gillespie <[email-masked]>, John C Klensin <[email-masked]>
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

Arnt Gulbrandsen wrote:
>
> Willie Gillespie writes:
>> In the interest of learning, how does EHLO help fight spam over
>> HELO?  I thought it just allowed for extensions...?
>
> The EHLO command names the client; the response names extensions. Have
> a look at CSV and BATF to see how the name can be used. 
Doesn't HELO name the client as well? If you don't care about the
extensions, then why not use HELO?

Like Willie, I'm a bit puzzled about how EHLO is better for fighting
spam than HELO. Without knowing that I can't see any reason to deprecate
HELO.

I agree that there's really no good reason for anyone to only use HELO
in new SMTP clients, since (AFAICS) you can just use EHLO with the same
data, and just ignore the response (as you do with HELO), but
deprecating it might lead to unnecessary support issues for people who
are still using HELO (eg because "if it ain't broke, don't fix it"), so
I can't see any reason to change it.

Also, there are also still many servers out there which don't support
EHLO, so clients MUST be able to fall back to HELO in that case.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Fri, 23 Jan 2009 10:36:08 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Cc: Willie Gillespie <[email-masked]>, John C Klensin <[email-masked]>
Sender: [email-masked]

Willie Gillespie writes:
> In the interest of learning, how does EHLO help fight spam over HELO?  
> I thought it just allowed for extensions...?

The EHLO command names the client; the response names extensions. Have a 
look at CSV and BATF to see how the name can be used.

Arnt

Date: Fri, 23 Jan 2009 10:36:05 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Cc: John C Klensin <[email-masked]>
Sender: [email-masked]

John C Klensin writes:
> Question: Is it time to formally deprecate 821 and, in particular, the 
> main feature that distinguishes it: the use of HELO by SMTP clients? 
> We would still need to require that SMTP servers accept it, but we 
> would tell full-capability clients (including the client side of 
> relays and gateways) that HELO is obsolete.

I think a lot of people would answer "you mean it isn't already?"

> One corollary of this is that we'd be telling low-capability clients, 
> particularly those that are part of MUA systems, that they should be 
> talking to Submit ports, not SMTP ones.

I beg to differ. Low-capability clients cannot effectively talk to 
submit, since submit tends to demand SASL support in the client.

Arnt

Date: Fri, 23 Jan 2009 01:32:42 -0700
From: Willie Gillespie <[email-masked]>
Organization: Engineering System Solutions
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: RFC 5321bis / 2821ter
Sender: [email-masked]

I don't have an opinion on this either, mostly because I don't know enough.

In the interest of learning, how does EHLO help fight spam over HELO?  I 
thought it just allowed for extensions...?

Willie

John C Klensin wrote:
> Hi.
> 
> Just as an update, I've got a first draft of this about ready to
> go, modulo the IETF Trust figuring out a way for me to post it.
> So far, it differs from 5321 only in that some changes suggested
> after Last Call (by individuals and the IESG) have been
> incorporated and there is an experiment in creating an index to
> syntax productions.
> 
> However, I had a discussion early this week with someone
> interested in the slightly-fuzzy text about arguments to HELO
> and/or EHLO and their relationship to spam-fighting.  The
> discussion leads to a question, especially since 5321bis is our
> first opportunity to really do something significant.
> 
> Question: Is it time to formally deprecate 821 and, in
> particular, the main feature that distinguishes it: the use of
> HELO by SMTP clients?  We would still need to require that SMTP
> servers accept it, but we would tell full-capability clients
> (including the client side of relays and gateways) that HELO is
> obsolete.  One corollary of this is that we'd be telling
> low-capability clients, particularly those that are part of MUA
> systems, that they should be talking to Submit ports, not SMTP
> ones.
> 
> I don't have a strong opinion (or even a weak one) about this
> one way or the other, nor have I looked at how much I'd have to
> change the document to implement it if it were wanted.  But,
> because this is the point at which we are proposing to really
> obsolete Full Standard 821 with a Full Standard, it seems like
> the right time to ask the question.   Since RFC 1425 was
> published sixteen years ago next month, it should not be
> unreasonable to tell people they've had enough time to get used
> to the idea of something besides HELO.
> 
>     john
> 
> 

Date: Fri, 23 Jan 2009 01:46:06 -0500
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: RFC 5321bis / 2821ter
Sender: [email-masked]

Hi.

Just as an update, I've got a first draft of this about ready to
go, modulo the IETF Trust figuring out a way for me to post it.
So far, it differs from 5321 only in that some changes suggested
after Last Call (by individuals and the IESG) have been
incorporated and there is an experiment in creating an index to
syntax productions.

However, I had a discussion early this week with someone
interested in the slightly-fuzzy text about arguments to HELO
and/or EHLO and their relationship to spam-fighting.  The
discussion leads to a question, especially since 5321bis is our
first opportunity to really do something significant.

Question: Is it time to formally deprecate 821 and, in
particular, the main feature that distinguishes it: the use of
HELO by SMTP clients?  We would still need to require that SMTP
servers accept it, but we would tell full-capability clients
(including the client side of relays and gateways) that HELO is
obsolete.  One corollary of this is that we'd be telling
low-capability clients, particularly those that are part of MUA
systems, that they should be talking to Submit ports, not SMTP
ones.

I don't have a strong opinion (or even a weak one) about this
one way or the other, nor have I looked at how much I'd have to
change the document to implement it if it were wanted.  But,
because this is the point at which we are proposing to really
obsolete Full Standard 821 with a Full Standard, it seems like
the right time to ask the question.   Since RFC 1425 was
published sixteen years ago next month, it should not be
unreasonable to tell people they've had enough time to get used
to the idea of something besides HELO.

    john