Subject: RE: The "such responses" extension sentence (was: Re: MX to CNAME and (mis)interpretation of 2821)
From: "Trevor Paquette" <[email-masked]>
To: <[email-masked]>
Date: Fri, 29 Feb 2008 18:38:31 -0700
List-Id: <ietf-smtp.imc.org>

>-----Original Message-----
>From: [email-masked]
[mailto:[email-masked]] On Behalf Of Mark >Martinec
>Sent: Monday, February 25, 2008 10:05 AM
>To: [email-masked]
>Subject: Re: The "such responses" extension sentence (was: Re: MX to
CNAME and >(mis)interpretation of 2821)
>
>
>SM wrote:
>>    When a domain name associated with an MX RR is looked up and
>>    the associated data field obtained, the data field of that
>>    response MUST contain a fully-qualified domain name.  That
>>    fully-qualified domain name, when queried, MUST return at
>>    least one address record (e.g., A or AAAA RR) that gives the
>>    IP address of the SMTP server to which the message should
>>    be directed.  Any other response,  specifically including a
>>    value that will return a CNAME record when queried, lies
>>    outside the scope of this standard.  The prohibition on CNAME
>>    RRs is discussed in more detail in RFC 2181, Section 10.3 [29].
>
>That looks like a good compromise to me.
>I'd remove the word 'specifically', it makes me read the statement
>more than once to be sure of its meaning, and is probably redundant.
>
>  Mark

As the culprit who actually started all of this... (sorry!). I like this
one the best.

The creation of standards and (unfortunately) the implementation thereof
can be two very different things based on someone's interpretation. This
paragraph (or how I interpret it anyway) eliminates this possibility.

I've enjoyed reading everyone's view on this; good opinions all around.

From: "Mark E. Mallett" <[email-masked]>
Date: Thu, 28 Feb 2008 15:26:59 -0500
To: [email-masked]
Subject: Re: random rfc2821bis-08 notes
Sender: [email-masked]

On Thu, Feb 28, 2008 at 01:24:43AM +0100, Frank Ellermann wrote:
> Mark E. Mallett wrote:

>  [3.1 3rd paragraph about initial 554] 
> > I believe this is actually a way to reject a "session" and not
> > a "transaction".  Since "transaction" (or "mail transaction")
> > is a specific term used in this document to refer to something
> > within a session, I think that it's not correct here.
> 
> OTOH 3.1 is "Session initiation", it's not exactly easy to be 
> confused about the topic... ;-)  A 554 in the session initiation
> in a certain *IS* a session, and it prohibits to reach a state
> where a transaction could begin.

Yeah I see that too.  But I think it's less accurate to say that the 554
greeting rejects a "transaction" than it is to say that it rejects a
"session" -- at least a session that will do anything.  And:  a session
that refuses to process commands is not really a session.  But...
reading your remarks, I can see that just substituting "session" for
"transaction" is still not exact (tho I think it's better); what I was
trying to get at was more like you did say.  E.g. more like:

   The SMTP protocol allows a server to formally enter into a session
   where no mail transactions will be accepted, as follows:

or even

   The SMTP protocol allows a server to formally enter into a [fruitless]
   session where no commands other than "QUIT" will be processed, as
   follows:

Basically I think that the term (and the concept of) "transaction" is
fairly important and some of my comments in the original posting were
related to the document being accurate and complete about it.  More
exactness may have no practical use at the moment; because indeed the
answer to the question you ask later, "Do you know practical
interoperability issues wrt the end of a transaction ?" is "no."  But I
would say that that's not all that evaluating a document is about.
I certainly don't think a big long discussion about any of those points
(related to the definition and specification of "transaction") is an
exciting idea at this late time; I made them because I was already
making some comments, and I could not help but at least think about them
while I was reading the document.

Re some of your other responses: the previous paragraph is the gist of
what I would say, rather than go point-by-point.

That said...

> > one could believe that another DATA command could be issued
> > at this point, or that further RCPTs could be given, etc.
> 
> The first paragraph of section 3.3 ends with:
> 
> | Then a DATA command initiates transfer of the mail data and
> | is terminated by the "end of mail" data indicator, which also
> | confirms the transaction.
> 
> Do you want "unrevocably initiates the end of the transaction",
> or similar ?

That is essentially what I was getting at, yes.

> Trying to figure out what you mean, IMHO it could
> make sense to split section 3.3 into an intro (1st paragraph),
> 3.3.1 (MAIL FROM, 1st step), 3.3.2 (RCPT TO, 2nd step), and 
> 3.3.3 (DATA, 3rd step).  

My point was merely that nothing says the transaction is closed after
DATA, and just numbering the steps wouldn't change that.  One might
think that a client could send:

   MAIL FROM
   RCPT TO
   DATA
   RCPT TO
   DATA

Now, somebody would run into a wall trying to talk to any existing
servers that support this conversation, but again, one should be able
to test correctness against a document as well as against the real world.

(Hmm, and I said I wouldn't go on further about it..)

> Yes, that is better.  John will anyway hate us for discussing
> original 2821 text after the 2821bis Last Call, I would write:
> 
> | When normal (2yz or 551) responses are returned from a VRFY
> | or EXPN request, the reply MUST include the <Mailbox> name
> | with a "<local-part@domain>" construct, where <Domain> is a
> | fully qualified domain name.

that is better, yes.

> 
> LOL.  Another future erratum bites the dust.  Thanks for a review
> from scratch, I fear nobody else did this for some months / years.

Well, it's been a while for me, at least.  The thing is, one can always
find things (and yes, there were things I *didn't* mention, I do have a
threshold of some sort :) ), so going over and over a document can be a
never-ending process.  Doesn't mean people shouldn't do it now and then,
though.

mm

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: rfc2821bis-08
Date:  Thu, 28 Feb 2008 09:43:18 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 55
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Alexey Melnikov wrote:

> +1.

What +1, focus on discussing the merits of FWS in
<quoted-string> as <local-part> of an <addr-spec> ?

Or +1 to kill the FWS in <local-part> in 2822upd ?
Killing it in syntax could be something like this:

- local-part    =3D dot-atom / quoted-string / obs-local-part
+ local-part    =3D dot-atom / quoted-local / obs-local-part

=3D quoted-string =3D [CFWS]=20
                  DQUOTE *([FWS] qcontent) [FWS] DQUOTE
                  [CFWS]

+ quoted-local  =3D [CFWS] DQUOTE=20
                  [SP] 1*qcontent *(SP 1*qcontent) [SP]
                  DQUOTE [CFWS]

That permits a single trailing or leading space, and a
single space between non-empty strings of <qcontent>.

It doesn't touch <quoted-pair>, if folks insist on it
they can still have " \  \  "@example with 5 spaces :-(

It doesn't permit ""@example or " "@example, so that
needs to be handled elsewhere, e.g. <obs-local-part>.

And it doesn't touch any backslash-HTAB, should that be
moved from <quoted-pair> to <obs-qp> ?  It's quite easy
to do that in syntax:

- quoted-pair =3D ("\" (VCHAR / WSP)) / obs-qp
+ quoted-pair =3D ("\" (SP / VCHAR)) / obs-qp

Bonus, modulo <obs-qp> that's the same as in 2821bis.

- obs-qp      =3D "\" (%d0 / obs-NO-WS-CTL / LF / CR)
+ obs-qp      =3D "\" CTL

Bonus, CTL is simpler.  Disadvantage, what breaks with
this relatively radical simplification ? =20

Of course 1*HTAB and 2*WSP would also need to go to an
augmented <obs-local-part> for this solution. =20

Anything with <quoted-string> that is no <quoted-local>
is automagically matched by <obs-local-part>, IMO that=20
covers 1*HTAB, 2*WSP, " "@example, backslash-HTAB, and
""@example.  If I didn't miss something <obs-local-part>
could stay as is, is that what you want ?

 Frank

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: random rfc2821bis-08 notes
Date:  Thu, 28 Feb 2008 01:24:43 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 178
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Mark E. Mallett wrote:

> Sorry that this is such a long comment on such a trivial thing.

It's correct, and better to fix this a.s.a.p. than to report it
as erratum later.  Where "a.s.a.p." can be either AUTH48 or -09.

=3D=3D=3D
 [3.1 3rd paragraph about initial 554]=20
> I believe this is actually a way to reject a "session" and not
> a "transaction".  Since "transaction" (or "mail transaction")
> is a specific term used in this document to refer to something
> within a session, I think that it's not correct here.

OTOH 3.1 is "Session initiation", it's not exactly easy to be=20
confused about the topic... ;-)  A 554 in the session initiation
in a certain *IS* a session, and it prohibits to reach a state
where a transaction could begin.

If you want to reject a session you can do that with TCP, just
don't accept the connection - depending on the IP, the phase of
the moon, or whatever.

=3D=3D=3D
 [3.3=20
> it's really more accurate to say "when a mail transaction is
> not in progress" than "without a previous MAIL command" since
> that's what's intended.

Okay.  IMO it's again not exactly easy to be confused about the
topic of 3.3 "Mail Transactions".  It's a rather long section,=20
folks could quote it out of context, and your wording is more
accurate - I'm not sure if it is clearer, quoted out of context
readers might not know what a "mail transaction" is.

> there's similar text later in the section re the DATA command

That should be fine, it says "missing accepted RCPT TO =3D> 554",
and "missing MAIL FROM =3D> 503".  Are you worried that this does
not mention SOML FROM and SAML FROM ? =20

> "if there is no transaction open, or no RCPT commands have been
> accepted," ...

Maybe, if you are optimistic that readers know what an "open=20
transaction" is, initiated by the most recent MAIL, SOML, SAML.

For hardcore nitpicking "open mail transaction" could eliminate
SEND, but I think 2821 + 2821bis intentionally ignore this cruft.

> I also notice that there is nothing that explicitly says when
> the processing of the DATA command ends a transaction, from the
> server state point of view.  There is the statement that the
> end of mail indicator "confirms the mail transaction" but this
> is not the same thing.  My understanding has been that once a
> DATA command has been OKed (with a 354 reply) and the subsequent
> response to the data transfer has been sent (even from data
> timeout), the server should then go into a state where no
> transaction is in progress.  But nothing in the text says that.

Sorry, I don't see the problem here.  Or rather, I think that
rejecting individual RCPT TO post-DATA with bounces constitutes
"net abuse" for an unverified reverse path, but that is not what
you are talking about and addressed "elsewhere", among others in
an I-D about "selective reject".

After the final dot, or whatever does the trick for BDAT + BURL,
the server sends a reply, and after doing that the transaction
is finished from its POV.  The client is supposed to wait for
this reply with a timeout.  After that timeout the session is
supposed to be broken (=3D not only the transaction failed), and
that is discussed elsewhere. =20

> one could believe that another DATA command could be issued
> at this point, or that further RCPTs could be given, etc.

The first paragraph of section 3.3 ends with:

| Then a DATA command initiates transfer of the mail data and
| is terminated by the "end of mail" data indicator, which also
| confirms the transaction.

Do you want "unrevocably initiates the end of the transaction",
or similar ?  Trying to figure out what you mean, IMHO it could
make sense to split section 3.3 into an intro (1st paragraph),
3.3.1 (MAIL FROM, 1st step), 3.3.2 (RCPT TO, 2nd step), and=20
3.3.3 (DATA, 3rd step). =20

=3D=3D=3D
=20
>| When normal (2yz or 551) responses are returned from a VRFY
>| or EXPN request, the reply normally includes the mailbox
>| name, i.e., "<local-part@domain>", where "domain" is a fully
>| qualified domain name, MUST appear in the syntax.
[...]
> I don't think that's even a sentence.  By the time you get to
> "MUST appear in the syntax", it's hard to know what exactly=20
> must appear in the "syntax".

>From my DEnglish POV it's a perfect sentence, but arguably it
could be split into two sentences.  Somehow there MUST be a=20
<local-part@domain> construct in the reply - okay, maybe I get
it now, you are talking about "syntax" vs. "reply", and the
"normally" is unclear:  There are no "normal responses" with
an "unnormal" (=3D without <Mailbox>) reply. =20

>| When normal (2yz or 551) responses are returned from a VRFY
>| or EXPN request, the reply MUST include the mailbox name,
>| i.e., "<local-part@domain>", where "domain" is a fully=20
>| qualified domain name.

Yes, that is better.  John will anyway hate us for discussing
original 2821 text after the 2821bis Last Call, I would write:

| When normal (2yz or 551) responses are returned from a VRFY
| or EXPN request, the reply MUST include the <Mailbox> name
| with a "<local-part@domain>" construct, where <Domain> is a
| fully qualified domain name.

And anybody mumbling <address-literal> now deserves to be shot
without warning... ;-)

> Not that anybody is really worrying about VRFY

My proposal that including VRFY and EXPN in a DS requires some
evidence that it is implemented and interoperable was widely
ignored.  According to the IESG page nobody reported that they
bothered to implement 2821, obviously that can't be true.  The
part I implemented (for a stupid client script) has no EXPN or
VRFY, but it insists on 8BITMIME for non-ASCII after EHLO.

=3D=3D=3D [About 4.1.4]
Maybe, but it is a bit late in the procedure to start with any
"make a shorter draft" efforts.  Inherited from RFC 2821 the
text is sometimes still convoluted, but IMO better, and in most
cases (not limited to 4.1.4) I think its "spirit" is clear. =20

Do you know practical interoperability issues wrt the end of
a transaction ?   =20

=3D=3D=3D
>| Greeting =3D ( "220 " (Domain / address-literal) [...]
[...]
> should mention be made here of the '554' style greeting?

After deleting three attempted responses trying to justify
why it's okay as is, all running into contradictions:  Yes,
it should mention 554, it's bad enough for an erratum.

=3D=3D=3D=20
>|         I: 354 -> data -> S: 250
>|                           E: 552, 554, 451, 452
>|                           E: 450, 550 (rejections for policy reasons)
>|          E: 503, 554

That is better.  Are you sure that 504 and 554 are the only=20
alternatives for 354 ?

=3D=3D=3D
>| If none do, then a bcc: header field with no
>| additional information SHOULD be inserted as specified in [32].
=20
> I imagine this is under control, but I think [32] means [10].

I imagine this is NOT under control IFF [10] changes the rules ;-)
But I'm intentionally paranoid wrt IETF and RFC-editor procedures.

=3D=3D=3D
> Appendix D.4  Verifying and Sending Scenario
[...]
> the SEND FROM command in this example will be rejected=20
> (particularly since it's not advertised in the EHLO response as
> per F.6)

LOL.  Another future erratum bites the dust.  Thanks for a review
from scratch, I fear nobody else did this for some months / years.

 Frank

From: "Mark E. Mallett" <[email-masked]>
Date: Wed, 27 Feb 2008 15:33:51 -0500
To: [email-masked]
Subject: random rfc2821bis-08 notes
Sender: [email-masked]

Hi-

I took another pass over rfc2821-08 and came up with a few things I
thought worthy of note, although given how late this is they may
not be worthy enough to do anything about, so ignore if you wish.
Nevertheless, here's the bunch.

====================

2.1, paragraph 7(?), third paragraph on pg 8, ends with:

   > the protocol requires that a server MUST either accept responsibility
   > for either delivering the message or properly reporting the failure
   > to do so (see Section 6.1, Section 6.2, Section 7.8, below).

There's too many "either"s.  This looks like it happened when adding the
"MUST".  Perhaps there was some intent to say that the server must either
reject the message or: accept responsibility for either delivering the
message or properly reporting the failure to do so.  That would explain
the extra "either," because those are the cases.  But since the text is
already focused on what happens after a "success response at the
end of the mail data" I suspect it's just an extra "either".  It might
be clearer to specifically say that this is still talking about the
situation where the message has been accepted, e.g.:

   the protocol requires that after this handoff a server MUST accept
   responsibility for either delivering the message or properly
   reporting the failure to do so (see Section 6.1, Section 6.2, Section
   7.8, below).

Sorry that this is such a long comment on such a trivial thing.

====================

3.1, paragraph 3, begins:

   > The SMTP protocol allows a server to formally reject a transaction
   > while still allowing the initial connection as follows: a 554
   > response MAY be given in the initial connection opening message
   > instead of the 220.

Realizing that this text comes from (at least) 2821 and is not new, I
believe this is actually a way to reject a "session" and not a
"transaction".  Since "transaction" (or "mail transaction") is a
specific term used in this document to refer to something within a
session, I think that it's not correct here.

====================

3.3, Mail Transactions, paragraph 8 (or so) says:

   > If a
   > RCPT command appears without a previous MAIL command, the server MUST
   > return a 503 "Bad sequence of commands" response. 

Again this text is not new..  but it's really more accurate to say "when
a mail transaction is not in progress" than "without a previous MAIL
command" since that's what's intended.  e.g. there may have been a
previous MAIL command in a previously completed transaction; there
may have been a previous MAIL command that was rejected, etc.

Just to be consistent in commenting, although I don't think it is
possible to misunderstand, there's similar text later in the section re
the DATA command:

   > If there was no MAIL, or no RCPT, command, or all such commands were
   > rejected, the server MAY return a "command out of sequence" (503) or
   > "no valid recipients" (554) reply in response to the DATA command.

which really means "if there is no transaction open, or no RCPT commands
have been accepted," ...

I also notice that there is nothing that explicitly says when the
processing of the DATA command ends a transaction, from the server state
point of view.  There is the statement that the end of mail indicator
"confirms the mail transaction" but this is not the same thing.  My
understanding has been that once a DATA command has been OKed (with a
354 reply) and the subsequent response to the data transfer has been
sent (even from data timeout), the server should then go into a state
where no transaction is in progress.  But nothing in the text says that.
With this ambiguity, one could believe that another DATA command could
be issued at this point, or that further RCPTs could be given, etc.
While section 4.1.4 (Order of Commands) has a remark about DATA and
transactions (and see later comment), I think that this does belong in a
section called "Mail Transactions" but of course understand that making
these comment at this late time is problematic.

====================

3.5.2.  VRFY Normal Response, paragraph 1 begins:

   > When normal (2yz or 551) responses are returned from a VRFY or EXPN
   > request, the reply normally includes the mailbox name, i.e.,
   > "<local-part@domain>", where "domain" is a fully qualified domain
   > name, MUST appear in the syntax.

Again this is not new text, it was in 2821.  But... I don't think that's
even a sentence.  By the time you get to "MUST appear in the syntax" ,
it's hard to know what exactly must appear in the "syntax".  Is this
referring to the mailbox name?  If so, then maybe it's clearer to say
something like:

   When normal (2yz or 551) responses are returned from a VRFY or EXPN
   request, the reply MUST include the mailbox name, i.e.,
   "<local-part@domain>", where "domain" is a fully qualified domain
   name.

Not that anybody is really worrying about VRFY; I just tripped over the
wording.

====================

4.1.4 Order of Commands, paragraph 9, contains:

   > Once started, a mail transaction consists
   > of a transaction beginning command, one or more RCPT commands, and a
   > DATA command, in that order.  A mail transaction may be aborted by
   > the RSET, a new EHLO, or the QUIT command.  There may be zero or more
   > transactions in a session.  MAIL (or SEND, SOML, or SAML) MUST NOT be
   > sent if a mail transaction is already open, i.e., it should be sent
   > only if no mail transaction had been started in the session, or if
   > the previous one successfully concluded with a successful DATA
   > command, or if the previous one was aborted with a RSET.

Again, the text is not new.  But to be complete, the last sentence
should end with "or a new EHLO" or even "if the previous one was
aborted, e.g. with a RSET or a new EHLO".

And re ambiguity about DATA and transactions: "successfully concluded
with a successful DATA command" leaves one to wonder, what about
"unsuccessfully concluded with a successful DATA command" or other
variations.  I would think more like "the previous one concluded by
acceptance and processing of mail data with the DATA command."

In fact I would rather that the document fully define a mail transaction
in only one place (like section 3.3) and refer to that definition, here
saying something like "must not be sent if a mail transaction is already
open (see section 3.3)."

====================

4.2 SMTP Replies, after paragraph 5:

   > Greeting       = ( "220 " (Domain / address-literal) [ SP text ] CRLF
   >                ) /
   >                ( "220-" (Domain / address-literal) [ SP text ] CRLF
   >                *( "220-" [ text ] CRLF )
   >                "220" [ SP text ] CRLF )

should mention be made here of the '554' style greeting?  The document
does caution that the narrative is authoritative over the abnf, I dunno
if that means it's better to omit it here though.  One thought is to
adjust the paragraph below:

   > where "Greeting" appears only in the 220 response that announces that
   > the server is opening its part of the connection.

to be something like:

   where "Greeting" describes only the response that announces that
   the server is opening its part of the connection, and where its
   numeric code can be either 220 or 554 as discussed in Section 3.1 .

====================

4.3.2  Command-Reply-Sequences, in the DATA description:

      > DATA
      > 
      >    I: 354 -> data -> S: 250
      > 
      >                      E: 552, 554, 451, 452
      > 
      >                      E: 450, 550 (rejections for policy reasons)

If I interpret the indentation correctly, the "E:" cases are for when
the '354' has been given and the mail data sent by the client.  This
does not cover the case where a negative response is sent instead of
354.  Maybe:

         I: 354 -> data -> S: 250
                           E: 552, 554, 451, 452
                           E: 450, 550 (rejections for policy reasons)
         E: 503, 554

====================

Appendix B, paragraph 2 (numbered "1.") ends:

    > If none do, then a bcc: header field with no
    > additional information SHOULD be inserted as specified in [32].

I imagine this is under control, but I think [32] means [10].

====================

Appendix D.4  Verifying and Sending Scenario

Do we really want an example using the "SEND FROM" command?  Maybe the
point of this is whooshing over my head, but if somebody implements a
server according to this document, the SEND FROM command in this example
will be rejected (particularly since it's not advertised in the EHLO
response as per F.6)

====================

Yours,
-mm-

To: Hector Santos <[email-masked]>
Cc: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: Using RSET with the EHLO/HELO fallback logic
From: [email-masked]
Date: Tue, 26 Feb 2008 18:57:23 -0500
Sender: [email-masked]

--==_Exmh_1204070243_3303P
Content-Type: text/plain; charset=us-ascii

On Tue, 26 Feb 2008 04:38:01 EST, Hector Santos said:

> - To help scale a large receiver system with less bandwidth (No
>    extended EHLO responses is needed to complete a transaction)?

Actually, at that point you're probably better off *allowing* EHLO, so that
you get niceties like the SIZE= parameter (so if the body is 25 meg in size,
you can stop it at the MAIL FROM rather than having to live through DATA),
and possibly PIPELINE as well (to reduce the number of round trips, especially
with multiple RCPT TO:).

--==_Exmh_1204070243_3303P
Content-Type: application/pgp-signature

--==_Exmh_1204070243_3303P--

Date: Mon, 25 Feb 2008 19:13:01 +0000
From: Alexey Melnikov <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821bis-08
Sender: [email-masked]

John C Klensin wrote:
 [...]

>As far as HT is concerned, I recommend avoiding the discussion of
>whether it is a nice white space character (since there are no
>white-space matching rules in 2821bis) and instead focusing on
>the question of whether it is actually helpful in email
>addresses.
>  
>
+1.

Date: Tue, 26 Feb 2008 08:22:27 -0500
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: Using RSET with the EHLO/HELO fallback logic
Sender: [email-masked]

Hector,

--On Tuesday, 26 February, 2008 03:21 -0500 Hector Santos
<[email-masked]> wrote:

>> It continues to be reasonable to have some instructions in
>> 2821bis about interoperation between 2821/2821bis
>> implementations and 821 ones, but we shouldn't expect the
>> latter to change their behavior without also becoming
>> 2821-conforming.
> 
> John, I know you and tony made a decision, which is fine, so
> none of this is to change your mind, but just as extra
> comments on the subject.

Thanks.

> I agree, but I think the matter is with active, contemporary
> SMTP clients who will read the specs and might come across
> those legacy systems who will not, can not change or can take
> MONTHS for an update.
> 
> Lets consider what might be a BCP for a client initiation and
> fall back logic.  To do it right based on what we all know
> today, to have automation, support cost reduction logic, the
> code would be close to the following:
> 
>   //--------------------------------------
>   // Client Initiation and Fall back Logic
>   //--------------------------------------
> 
>...
> But this logic an only be extracted from 2821bis if the
> insights existed in the specs.
> 
> So you have 3 scenarios for a BCP:
> 
>    1) EHLO/HELO sequence (run into potential fallback issues
> per 1869),
> 
>    2) EHLO/RSET/HELO sequence honor all response codes (like
>       we have in our code currently),
> 
>    3) EHLO/RSET/HELO sequence honoring all response codes
> except
>       any RSET 503 possible response per 1869.
> 
> All are correct (or allowed) per specs, but which one offers
> the better success across the board?
> 
> I think that would be #3.

I don't think anyone is disagreeing with you about the general
substance of this, only about the odds of the folks who are
responsible for the problems reading 2821bis carefully to
extract the information and guidance you want them to have.  One
way of looking at it is that anyone who wanted to conform to
2821, has read all of it carefully, and wants to get things
right already has this in reasonable shape -- the information is
there, although not organized optimally for this purpose.  The
fact that 2821bis is pushing toward 100 pages (so much for
"simple" :-(  ) doesn't help much with that.  The problem is
that many of the folks you are dealing with, _especially_ the
providers of firewalls with SMTP proxies and mail-filtering
appliances, have a long history of being sloppy about SMTP
implementations and not caring very much.  I think that
situation has gotten gradually better over time, but...

This leads me to the conclusion that, if one wants to have high
leverage on the situation with which you are concerned, the best
solution would be assemble a draft that specifically addresses
interoperability between "old" (821 conforming) and
"contemporary" (2821/2821bis conforming) email systems and stays
short and to the point.   One might even address interoperation
between those systems and various partial implementations (that
is a charitable way to say "broken") of 2821.  

Such a document would need to be consistent with 2821bis, of
course.  But it seems to me that it could be quite compact and
very focused and hence much more likely to be read and
understood by someone who is willing to bring their applications
into conformance if they can figure out what they should be
doing without too much effort.   If properly written and
community consensus established, it could presumably be
processed as a BCP and would be a useful contribution.

It is almost certainly too late for 2821bis but, if such a
document were produced and published, 2821ter (if we ever get
around to that) could easily contain an informative reference to
it, thereby tying everything together.

The only problem I see with going down that path is that the
document would have to be assembled by someone current access to
logs and knowledge of client and server behavior.  I don't have
that any more (even if I had the extra cycles available, which I
don't), so this is something for which you, or someone similarly
positioned, would have to take the lead.

     john

Date: Tue, 26 Feb 2008 04:38:01 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: Using RSET with the EHLO/HELO fallback logic
Sender: [email-masked]

Frank Ellermann wrote:

> Anything that can go wrong will go wrong, but I have
> no example where not accepting EHLO is a good idea.

I agree, but just a few reasons off the top why it may persist:

- As part of an AVS system/proxy w/o MX low preference host?

   I may be wrong, but this appears to be the current reason
   I see it a lot.

- To help scale a large receiver system with less bandwidth (No
   extended EHLO responses is needed to complete a transaction)?

- Its not required for a majority of the public only receiver
   systems. No incentive to adapt.

- TLS/AUTH is not supported but instead offers POP B4 SMTP or an
   IP "Allow Relay" concept. I think it safe to say that many
   ISPs predominately support end-user or smart host IP
   client authentication to allow relay operations.

   From a support standpoint, this is the preferred, less costly
   method. No users instructions are required for using ESMTP AUTH.

   Note: As with long time systems, we stated as a 821 system and
   added EHLO began when we added TLS/AUTH support.  So for other
   systems who felt it wasn't required, they might not have adapted.

- Finally, overall, simpler SMTP coding without EHLO support,
   especially on the client side.  You will find many simple
   "tools" and scripts that use HELO only.

On a personal basis (with my old bellsouth.net account), I recall a few 
years back when we got an email notice indicated a 4-5 month warning and 
cut off date (i.e, end of JULY) where ALL users will be expected to 
begin using ESMTP AUTH.  There reasons?

     - Better security against spam,
     - Better Support for Roaming Users

No problem for me, but I remember sending an small note to their support 
saying "I hope you guys will be ready for the user support issues."

When July came around, I recall it was no more than 1-2 weeks where they 
send out another email and dropped the ESMTP AUTH requirement for end 
users.  I have to go back and double check, but I remember at first, 
they didn't support it and I was among those who requested it so I can 
roam around with my lap top at the office.  When the 2nd drop notice
came, I remember checking again and it was still offered but optional
like most systems that offer ESMTP AUTH.

Thats not to say they don't accept EHLO clients, but IMO, I think 
TLS/AUTH support is probably one of main reasons why systems added EHLO 
support or turn it on.  I'm guessing on that. I'm sure many systems have 
other EHLO reasons, but if anyone wanted to add TLS/AUTH support, then 
they need first adapt to EHLO support.  TLS/AUTH was the reason for us 
to add EHLO support.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Tue, 26 Feb 2008 03:21:36 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: Using RSET with the EHLO/HELO fallback logic
Sender: [email-masked]

John C Klensin wrote:

> [email-masked] wrote:
> 
>> On Sun, 24 Feb 2008 16:16:02 EST, Hector Santos said:
>>
>>>    S: 220 **************************************************
>>>    C: EHLO yyyyyyy.com
>>>    S: 500 Syntax error, command unrecognized
>> Am I the only one who thinks that if the remote end doesn't
>> understand EHLO, it's highly unlikely that any revision of
>> 2821 is going to improve matters?  RFC1425 was a *long* time
>> ago....
> 
> Yep.  And, of course, that is also why the second paragraph of
> 2.2.1 ("Contemporary SMTP implementations MUST support...") says
> what it does.  The theory, going back to early discussions in
> DRUMS, is that support for EHLO is now mandatory and so the
> fallback cases should not arise.  One's alternative is to
> conform to 821 only, in which cas, as you suggest, anything
> 2821bis says is likely to be irrelevant.
> 
> It continues to be reasonable to have some instructions in
> 2821bis about interoperation between 2821/2821bis
> implementations and 821 ones, but we shouldn't expect the latter
> to change their behavior without also becoming 2821-conforming.

John, I know you and tony made a decision, which is fine, so none of 
this is to change your mind, but just as extra comments on the subject.

I agree, but I think the matter is with active, contemporary SMTP 
clients who will read the specs and might come across those legacy 
systems who will not, can not change or can take MONTHS for an update.

Lets consider what might be a BCP for a client initiation and fall back 
logic.  To do it right based on what we all know today, to have 
automation, support cost reduction logic, the code would be close to the 
following:

  //--------------------------------------
  // Client Initiation and Fall back Logic
  //--------------------------------------

  nReplyCode = SendReceive("EHLO "+sClientHostName, sResponse);

  if (nReplyCode >= 500 && nReplyCode < 503) {

     //--------------------------------------------------------
     // Fallback to HELO
     //
     // Some servers might need a RSET in the fallback logic
     //--------------------------------------------------------

     nReplyCode = SendReceive("RSET", sResponse);

     //--------------------------------------------------------
     // According to RFC 1869, some servers might issue a
     // "503 out of sequence" response. RFC 1869 recommends to
     // ignore the 503 response and continue with the HELO
     // fallback.
     //--------------------------------------------------------

     if (nReplyCode != 250 && nReplyCode !=503) {
        nReplyCode = SendReceive("QUIT", sResponse);
        return sFalse;
     }

     nReplyCode = SendReceive("HELO "+sHostName, sResponse);

     if (nReplyCode != 250) {
         nReplyCode = SendReceive("QUIT", sResponse);
         return sFalse;
     }
  }
  return sTrue; // continue with next command

But this logic an only be extracted from 2821bis if the insights existed 
in the specs.

So you have 3 scenarios for a BCP:

   1) EHLO/HELO sequence (run into potential fallback issues per 1869),

   2) EHLO/RSET/HELO sequence honor all response codes (like
      we have in our code currently),

   3) EHLO/RSET/HELO sequence honoring all response codes except
      any RSET 503 possible response per 1869.

All are correct (or allowed) per specs, but which one offers the better 
success across the board?

I think that would be #3.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Tue, 26 Feb 2008 02:56:47 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
CC: [email-masked], John C Klensin <[email-masked]>
Subject: Re: Using RSET with the EHLO/HELO fallback logic
Sender: [email-masked]

[email-masked] wrote:
> On Sun, 24 Feb 2008 16:16:02 EST, Hector Santos said:
> 
>>    S: 220 **************************************************
>>    C: EHLO yyyyyyy.com
>>    S: 500 Syntax error, command unrecognized
> 
> Am I the only one who thinks that if the remote end doesn't understand
> EHLO, it's highly unlikely that any revision of 2821 is going to improve
> matters?  RFC1425 was a *long* time ago....

Agree.

However, all this is based on the presumption everyone is operating with 
Contemporary (actively supported) SMTP systems.

So any issue or problem is not with active systems.  They can adjust as 
was the case here with this particular vendor.

It is also not an issue with old, no longer supported, legacy SMTP 
SERVERS because they may not have a mean of changing. e.g., the authors 
may be dead and bury, gone or whatever. They are out of the loop or some 
  low budget business found some cheap old SMTP software, who knows.

But rather the burden is with current, actively supported SMTP clients 
because it is they who will have to deal with such situations.

PS: EHLO/HELO fall back happens all the time with even contemporary 
systems especially among many who are using AVS proxies.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Tue, 26 Feb 2008 00:24:04 -0500
From: John C Klensin <[email-masked]>
To: [email-masked], Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: Using RSET with the EHLO/HELO fallback logic
Sender: [email-masked]

--On Monday, 25 February, 2008 18:08 -0500
[email-masked] wrote:

> On Sun, 24 Feb 2008 16:16:02 EST, Hector Santos said:
> 
>>    S: 220 **************************************************
>>    C: EHLO yyyyyyy.com
>>    S: 500 Syntax error, command unrecognized
> 
> Am I the only one who thinks that if the remote end doesn't
> understand EHLO, it's highly unlikely that any revision of
> 2821 is going to improve matters?  RFC1425 was a *long* time
> ago....

Yep.  And, of course, that is also why the second paragraph of
2.2.1 ("Contemporary SMTP implementations MUST support...") says
what it does.  The theory, going back to early discussions in
DRUMS, is that support for EHLO is now mandatory and so the
fallback cases should not arise.  One's alternative is to
conform to 821 only, in which cas, as you suggest, anything
2821bis says is likely to be irrelevant.

It continues to be reasonable to have some instructions in
2821bis about interoperation between 2821/2821bis
implementations and 821 ones, but we shouldn't expect the latter
to change their behavior without also becoming 2821-conforming.

      john

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Using RSET with the EHLO/HELO fallback logic
Date:  Tue, 26 Feb 2008 04:34:31 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 23
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

<[email-masked]> wrote:

> Am I the only one who thinks that if the remote end
> doesn't understand EHLO, it's highly unlikely that
> any revision of 2821 is going to improve matters?

LOL, you certainly have a point.  I always understood
the RSET blurb as a recipe how to trick "old" servers
into doing their job without new connection.

Some "old" servers might accept HELO after rejecting
EHLO.  Another set might be confused after rejecting
EHLO, and RSET fixes that before trying HELO.  And a
third set could require a new connection without EHLO.

Where "old" could mean really 821, or only pretending
to be old for reasons of their own, e.g., if they know
a client where accepting EHLO results in some trouble.
 =20
Anything that can go wrong will go wrong, but I have
no example where not accepting EHLO is a good idea.

 Frank

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:   FWS *within* <quoted-string> (was: rfc2821bis-08)
Date:  Tue, 26 Feb 2008 04:06:51 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 45
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

John C Klensin wrote:

> The "Last" in "Last Call" needs to mean something

"Your *last* chance wrt this document", rare cases
of appeals and repeated Last Calls not withstanding.

> It is not an accident that HT is still prohibited
> in addresses.

After RFC.klensin-net-utf8 was approved (14:04:33)
I'll consider this as fait accompli in 60 days. =20

Now 2822upd has to get rid of it, it makes no sense
to forbid HT in 2821bis and to allow HT in 2822upd.

Ideally it is a semantical instead of a syntactical
problem, we could say that any FWS actually stands
for a single space.

But everybody expects <quoted-string> to mean what
is quoted *as is*.  The 2822upd theory is that it
means what it says modulo <quoted-pair>s, minus the
leading backslashes of <quoted-pair>.

In practice RFC 2617, RFC 2831, RFC.usefor-usefor,
and anything building on it use a <qdstr-val>, i.e.
keep also <quoted-pair> *as is*.

Alexey's 2831bis I-Ds proposed to "fix" <qdstr-val>,
but after nobody produced the time-machine for also
fixing RFC 2069 gave up on this idea.

And on the URI list somebody proposed years ago to
get rid of these weird backslashes at least in tag
URLs, but the tag + mailto + news-nntp URI authors
declined, IIRC.

IMO that reduces the 2822upd <quoted-pair> theory to
"maybe user agents displaying a <quoted-string> get
it right".  This issue is critical, it could affect
DKIM canonicalizations or mailto-bis or RFCs sitting
in the RFC editor queue.

 Frank

To: Hector Santos <[email-masked]>
Cc: [email-masked], John C Klensin <[email-masked]>
Subject: Re: Using RSET with the EHLO/HELO fallback logic
From: [email-masked]
Date: Mon, 25 Feb 2008 18:08:16 -0500
Sender: [email-masked]

--==_Exmh_1203980896_3315P
Content-Type: text/plain; charset=us-ascii

On Sun, 24 Feb 2008 16:16:02 EST, Hector Santos said:

>    S: 220 **************************************************
>    C: EHLO yyyyyyy.com
>    S: 500 Syntax error, command unrecognized

Am I the only one who thinks that if the remote end doesn't understand
EHLO, it's highly unlikely that any revision of 2821 is going to improve
matters?  RFC1425 was a *long* time ago....

--==_Exmh_1203980896_3315P
Content-Type: application/pgp-signature

--==_Exmh_1203980896_3315P--

Date: Mon, 25 Feb 2008 13:36:16 -0500
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: rfc2821bis-08
Sender: [email-masked]

Hi.

In order to make the posting deadline and, with luck, move this
forward, draft-klensin-rfc2821bis-08 is now in the posting queue
(I'm awaiting the confirmation message).

Please see the change log for a summary of what has been done.
Note that the problematic "such response" sentence has been
removed per comments yesterday and today.

After a discussion with Tony, we've tentatively decided to take
a hard line toward post-Last-Call suggestions for small
improvements.  The "Last" in "Last Call" needs to mean
something; if we keep iterating every time someone has a new
idea about an issue that might reasonably be addressed we will
never get this long-overdue project finished.  My own position
at this point is agnostic about the merits of more text for RSET
-- had the issue come up several months ago, I would have wanted
to participate in the discussion and possibly include some text.
But, at this point, absent evidence that the spec is actually
broken, I think it is more important to get finished.

It is not an accident that HT is still prohibited in addresses.
The change is easily made in an RFC Editor note should we decide
to do so and Tony will address that issue.  In the meantime,
please don't re-post the same comments on the assumption that
you are being ignored.  My one observation on the subject is
that both 821 and 2821 permitted _any_ ASCII character if they
were escaped.  Neither HT nor SP were ever specifically
permitted; they were just part of "any".  We needed to permit SP
because there is a known (although old) use case for it.    As
far as HT is concerned, I recommend avoiding the discussion of
whether it is a nice white space character (since there are no
white-space matching rules in 2821bis) and instead focusing on
the question of whether it is actually helpful in email
addresses.

Tony will generate a new Last Call to cover the changes.

     john

From: Mark Martinec <[email-masked]>
Organization: J. Stefan Institute
To: [email-masked]
Subject: Re: The "such responses" extension sentence (was: Re: MX to CNAME and (mis)interpretation of 2821)
Date: Mon, 25 Feb 2008 18:05:02 +0100
Sender: [email-masked]

SM wrote:
>    When a domain name associated with an MX RR is looked up and
>    the associated data field obtained, the data field of that
>    response MUST contain a fully-qualified domain name.  That
>    fully-qualified domain name, when queried, MUST return at
>    least one address record (e.g., A or AAAA RR) that gives the
>    IP address of the SMTP server to which the message should
>    be directed.  Any other response,  specifically including a
>    value that will return a CNAME record when queried, lies
>    outside the scope of this standard.  The prohibition on CNAME
>    RRs is discussed in more detail in RFC 2181, Section 10.3 [29].

That looks like a good compromise to me.
I'd remove the word 'specifically', it makes me read the statement
more than once to be sure of its meaning, and is probably redundant.

  Mark

Date: Mon, 25 Feb 2008 10:26:09 -0500
From: John Leslie <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: The "such responses" extension sentence (was: Re: MX to CNAME and (mis)interpretation of 2821)
Sender: [email-masked]

John C Klensin <[email-masked]> wrote:
> 
>    "Such responses could, in principle, be covered by DNS
>    changes or extensions and an SMTP extension in the future."
> 
> I'd like to hear from John Leslie, who originally suggested 
> putting something like that in.

   Quick answer: as I said before, I think this battle is long lost:
we look to be beating a dead horse.

   (If someone wants a longer explanation, feel free to ask...)

> If he is now persuaded that it should be dropped,

   Yes.

> I'd find that convincing and perhaps others would as well.

--
John Leslie <[email-masked]>

Date: Mon, 25 Feb 2008 10:23:33 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SM <[email-masked]>
CC: [email-masked]
Subject: Re: Using RSET with the EHLO/HELO fallback logic
Sender: [email-masked]

SM wrote:
> 
> Hi Hector,
> At 02:11 25-02-2008, Hector Santos wrote:

>> Or did the system, blame game work?  I was able to show the big brand 
>> name $$$$ AVS appliance vendor the "errors of their ways" using 2821 
>> only and they made the change/fix.
> 
> The system blame game never works. :-)  You took the decision to make a 
> change that fixes the problem.  That's always the best alternative.

Ok, I think I failed to express the point.  Maybe others who understood 
can chime in.

The problem was resolved without us needing to make a change.  The 
vendor resolved the problem.  It was never revealed how they did, only 
to acknowledge it was a problem on their end.  Whether it was an option 
or a software change in this high-end proxy appliance software, it 
wasn't reveal how they were going to address it. All I know it was fixed 
within 24 hours.

What I was trying to point out was above and beyond the resolution that 
was already found,  we decided to add a change request to address any 
future issues related to this where we might come across legacy 
operation or current operation with an "run in legacy 821 mode" switch. 
Who knows.

It goes to show, as with the other MX/CNAMES, that when these ambiguous 
issues present themselves, regardless of what the specs may say, 
solutions will be found to reduce future issues.

 >> Don't you think this is important information to "keep" or bring back
 >> in some form into 2821bis?  It explains everything.
 >
 > It's good that we know about it to have a better grasp of the mail
 > environment.  The section I quoted talks about responses from
 > improperly implemented servers.  That was 13 years ago.
 > It's high time that was fixed.

I agree, which is why we ask to talk to the vendor because the talk of 
being a "big brand name", hence no way they had it wrong, was strange. 
We told our customer that if we were dealing with a vendor that was a 
legacy system and have no way to resolve it, we will provide the 
immediate patch.  But as it turned out, after pulling some teeth, the 
vendor acknowledge the situation was on their end.

 > I don't think it should be brought back into RFC 2821bis because:
 >
 >  1.  There is already text to tell us what to do (refer to
 >      my previous message).
 >
 >  2.  We cannot support improperly implemented servers forever.
 >
 >  3.  We don't want the specifications to be longer than necessary.

SM,  most professional and commercial systems does not have the luxury 
to take such a hard stance and debate it with customers.  It doesn't pay.

If its was a legacy system your customer was attempting to send mail to, 
I believe it is for the new "Practice Codifying" specs to keep this 
historical information intact so that developers who might come across 
it will be able to understand and deal with it.

In short, if a new developer uses 2821bis to frame his new SMTP system, 
it might use the RSET as part of the fall back without considering 
interops issues for one simple reason only:

         The historical and interops insights was removed.

I will venture very strongly that if a future developer had the simple 
note available as I proposed for section 3.3, or the 13 year old RFC 
1869 note carried over, he would seriously consider not using a RSET for 
EHLO/HELO fallback logic requirements or making it operation on a per 
host basis or something like that.

As it would be without any insights, he would find out the hard way.

Anyway, I am just expressing my opinion and this is my last input on the 
subject.  I must admit that there is a sense of inconsistent 
considerations which is unfortunately based primarily on who is talking. 
  I could careless how it turns out, no, thats not true, I do care, but 
it would be great to see some consistency here.

Thanks for your comments.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Mon, 25 Feb 2008 07:23:24 -0800
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: The "such responses" extension sentence (was: Re: MX to CNAME and (mis)interpretation of 2821)
Cc: John Leslie <[email-masked]>
Sender: [email-masked]

At 12:42 24-02-2008, John C Klensin wrote:
>The current sentence reads
>
>    "Such responses could, in principle, be covered by an SMTP
>    extension in the future."
>
>Since there is a normative DNS prohibition on CNAME targets (in 
>RFC2181), that sentence is, strictly speaking, untrue.  It could be 
>corrected by saying:
>
>    "Such responses could, in principle, be covered by DNS
>    changes or extensions and an SMTP extension in the future."
>
>Or we could drop it.   My instinct at the moment (i.e., I reserve 
>the right to change my

I suggest dropping that line as taking about DNS changes and SMTP 
extensions makes the it convoluted.  Suggested text is as follows:

   When a domain name associated with an MX RR is looked up and
   the associated data field obtained, the data field of that
   response MUST contain a fully-qualified domain name.  That
   fully-qualified domain name, when queried, MUST return at
   least one address record (e.g., A or AAAA RR) that gives the
   IP address of the SMTP server to which the message should
   be directed.  Any other response,  specifically including a
   value that will return a CNAME record when queried, lies
   outside the scope of this standard.  The prohibition on CNAME
   RRs is discussed in more detail in RFC 2181, Section 10.3 [29].

Regards,
-sm 

Date: Mon, 25 Feb 2008 05:57:21 -0800
To: Hector Santos <[email-masked]>
From: SM <[email-masked]>
Subject: Re: Using RSET with the EHLO/HELO fallback logic
Cc: [email-masked]
Sender: [email-masked]

Hi Hector,
At 02:11 25-02-2008, Hector Santos wrote:
>Don't you think this is important information to "keep" or bring 
>back in some form into 2821bis?  It explains everything.

It's good that we know about it to have a better grasp of the mail 
environment.  The section I quoted talks about responses from 
improperly implemented servers.  That was 13 years ago.  It's high 
time that was fixed.  I don't think it should be brought back into 
RFC 2821bis because:

  1.  There is already text to tell us what to do (refer to my 
previous message).

  2.  We cannot support improperly implemented servers forever.

  3.  We don't want the specifications to be longer than necessary.

>Or did the system, blame game work?  I was able to show the big 
>brand name $$$$ AVS appliance vendor the "errors of their ways" 
>using 2821 only and they made the change/fix.

The system blame game never works. :-)  You took the decision to make 
a change that fixes the problem.  That's always the best alternative.

Regards,
-sm 

Date: Mon, 25 Feb 2008 05:11:27 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SM <[email-masked]>
CC: [email-masked]
Subject: Re: Using RSET with the EHLO/HELO fallback logic
Sender: [email-masked]

SM wrote:
> 
> Quoting Section 4.7 of RFC 1869 (obsoleted by RFC 2821):
> 
>    Other improperly-implemented servers will not accept a HELO command
>    after EHLO has been sent and rejected.  In some cases, this problem
>    can be worked around by sending a RSET after the failure response to
>    EHLO, then sending the HELO.  Clients that do this should be aware
>    that many implementations will return a failure code (e.g., 503 Bad
>    sequence of commands) in response to the RSET.  This code can be
>    safely ignored.
> 
> That's the historical context.

SM,

Don't you think this is important information to "keep" or bring back in 
some form into 2821bis?  It explains everything.

Or did the system, blame game work?  I was able to show the big brand 
name $$$$ AVS appliance vendor the "errors of their ways" using 2821 
only and they made the change/fix.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Mon, 25 Feb 2008 10:54:21 +0100
From: Alessandro Vesely <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: MX to CNAME and (mis)interpretation of 2821
Sender: [email-masked]

John C Klensin wrote:
> The tentative text in 2821bis-08 now reads as follows
> 
> # When a domain name associated with an MX RR is looked up and
> # the associated data field obtained, the data field of that
> # response MUST contain a domain-name.  That domain-name, when

How about changing the latter line with the following?

#   response MUST contain a host name. That host name, when

Section 2.3.4, "Host", defines exactly that. I think "host name" is a 
better term because the hyphenated "domain-name" may induce causal 
readers to search for a corresponding ABNF or other definition in vain.

> # queried, MUST return at least one address record (e.g., A or
> # AAAA RR) that gives the IP address of the SMTP server to
> # which the message should be directed.  Any other response,
> # specifically including a value that will return a CNAME
> # record when queried, lies outside the scope of this
> # standard.  Such responses could, in principle, be covered by
> # an SMTP extension in the future.

+1 for removing the latter sentence.

> #                                   The prohibition on labels
> # in the data that resolve to CNAMEs is discussed in more
> # detail in RFC 2181, Section 10.3 [29].

BTW, is it correct to cite "RFC974" (three times in 2821bis-07) even 
if it has been obsoleted? I don't know if the tag "obsoletes: 974" 
that is present in rfc2821 would also be obsoleted by rfc2821bis so 
that rfc974 would eventually return to a non-obsolete status.

Date: Sun, 24 Feb 2008 22:45:15 -0800
To: Hector Santos <[email-masked]>, [email-masked]
From: SM <[email-masked]>
Subject: Re: Using RSET with the EHLO/HELO fallback logic 
Sender: [email-masked]

Hi Hector,
At 13:16 24-02-2008, Hector Santos wrote:
>Here is the overall transaction that was in question.  This is our 
>customer using our WCSMTP software to send mail to their new 
>customer who is using large brand name server appliance as an SMTP host.
>
>   S: 220 **************************************************
>   C: EHLO yyyyyyy.com
>   S: 500 Syntax error, command unrecognized
>   C: RSET
>   S: 503 Bad sequence of commands
>   C: QUIT
>   S: 221 xxxxxxxx.com Service closing connection
>
>Overall, the issue is that the remote server did not honor the RSET 
>command and issued a BAD sequence of commands.  So our software 
>abandoned the transaction and QUIT.

Quoting Section 4.7 of RFC 1869 (obsoleted by RFC 2821):

    Other improperly-implemented servers will not accept a HELO command
    after EHLO has been sent and rejected.  In some cases, this problem
    can be worked around by sending a RSET after the failure response to
    EHLO, then sending the HELO.  Clients that do this should be aware
    that many implementations will return a failure code (e.g., 503 Bad
    sequence of commands) in response to the RSET.  This code can be
    safely ignored.

That's the historical context.
>But after the situation was resolved, we made a decision here to put 
>in a change request in our SMTP client software to make the RSET 
>command optional in the EHLO/HELO fall back to reduce future support 
>cost issues.
>
>I looked at RFC 2821 section 3.2
>
>    3.2 Client Initiation
>
>    Once the server has sent the welcoming message and the client has
>    received it, the client normally sends the EHLO command to the
>    server, indicating the client's identity.  In addition to opening the
>    session, use of EHLO indicates that the client is able to process
>    service extensions and requests that the server provide a list of the
>    extensions it supports.  Older SMTP systems which are unable to
>    support service extensions and contemporary clients which do not
>    require service extensions in the mail session being initiated, MAY
>    use HELO instead of EHLO.  Servers MUST NOT return the extended
>    EHLO-style response to a HELO command.  For a particular connection
>    attempt, if the server returns a "command not recognized" response to
>    EHLO, the client SHOULD be able to fall back and send HELO.
>
>If someone reads RFC 2821, I guess, in reading the last sentence, 
>one can reasonably argue how one may get the idea that a RSET is not 
>expected after an EHLO command, hence issuing a "503 Bad sequence of 
>commands" response as this large appliance SMTP server did.  I don't 
>think that argument jives 4.1.1.5, but I can see where that mistake 
>can be made.

It's not an EHLO command there; it's an unrecognized command.  The 
client SMTP did not fall back and send HELO.

>Who is at truly at fault?  Who knows?  I really don't care.  Our 
>software has the RSET in its EHLO/HELO fallback for a engineering 
>reason, I'm sure.  But because of the above possibilities, we are 
>going to make the change too.

Please don't read what I am saying as "who is truly at fault".

Section 4.1.4 of the RFC 2821bis mentions:

   "The NOOP, HELP, EXPN, VRFY, and RSET commands can be used at any time
    during a session, or without previously initializing a session.  SMTP
    servers SHOULD process these normally (that is, not return a 503
    code) even if no EHLO command has yet been received; clients SHOULD
    open a session with EHLO before sending these commands."

>Summary and recommendation:
>
>I am not sure which way you may to do this.
>
>  a) Add Text to 3.2 to make servers aware a RSET after EHLO is
>     possible?

The text is already there (see above).

>  b) Do not allow RSET after EHLO for possible 821 historical reasons?

That means turning this into an issue.

>The latter may be a problem with existing software designed around 
>2821 which may already have logic to issue RSET after EHLO.

That's what the client SMTP did in this case.

>Personally, I prefer the former (a).
>
>I think section 4.1.1.5 is fine.  I think maybe the last sentence in 
>3.2 first paragraph can be slightly changed, augmented with 
>additional clauses, sentence or some note.

Before we get into that, you mentioned that the vendor misinterpreted 
the RFC 2821 specifications.  Section 2.2.1 of RFC 2821 says:

   "Contemporary SMTP implementations MUST support the basic extension
    mechanisms.  For instance, servers MUST support the EHLO command even
    if they do not implement any specific extensions and clients SHOULD
    preferentially utilize EHLO rather than HELO."

It's not an issue about EHLO/HELO fallback logic or RSET then as the 
server does not support the EHLO command.

Regards,
-sm 

Date: Sun, 24 Feb 2008 14:39:09 -0800
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: The "such responses" extension sentence
Sender: [email-masked]

John C Klensin wrote:
> Or we could drop it.   My instinct at the moment (i.e., I reserve the 
> right to change my mind) is that dropping it would be better.  

+1.

Having a specification contain language about hypothetical future capabilities 
is almost never useful and often turns out to be just plain wrong.  The only 
benefit of reference to the future is when the specification contains pedagogy 
about extensibility by way of showing how to add capabilities.

A spec should specify.  And it should explain the specification.  Predicting the 
future does neither.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Sun, 24 Feb 2008 16:16:02 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked], John C Klensin <[email-masked]>
Subject: Using RSET with the EHLO/HELO fallback logic 
Sender: [email-masked]

John et al,

I honestly do not wish this to be a major thread/debate, and certainly 
not delay any process for 2821bis.   I just want to point it out and 
leave the rest to you.  I don't wish to leave out any details, but I 
don't also want to get too deep.  So I will be depending on your 
experience and foresight to fill in any gaps.

I was going to post this note last month, but knowing how you and Tony 
have worked hard and wish to complete this 2821bis process, I decide to 
forgo it.

But I bring it up now primarily for two reasons:

1) I was reminded of it with the MX/CNAME discussion and I am one that 
prefers to reduce the "blame game" in customer support and production 
market, and

2) A major brand name SMTP/AVS Appliance (VENDOR) misinterpreted the RFC 
2821 specs when it came to the RSET command.  If a major brand name had 
this issue, certainly it can happen with others.

The RSET situation created a sensitive customer support issue with one 
of their new large accounts. It escalated to "who is more RFC 2821 
correct" blame game between four parties with teleconference calls 
between our customer, their new customer, their customer mail system 
appliance vendor and us.

So if you think there is amble room/time and deemed reasonable to 
include some extra text regarding the usage of the RSET command, I leave 
you to decide that.

Here is the overall transaction that was in question.  This is our 
customer using our WCSMTP software to send mail to their new customer 
who is using large brand name server appliance as an SMTP host.

   S: 220 **************************************************
   C: EHLO yyyyyyy.com
   S: 500 Syntax error, command unrecognized
   C: RSET
   S: 503 Bad sequence of commands
   C: QUIT
   S: 221 xxxxxxxx.com Service closing connection

Overall, the issue is that the remote server did not honor the RSET 
command and issued a BAD sequence of commands.  So our software 
abandoned the transaction and QUIT.

In the final analysis, the VENDOR realized the "error in their ways" and 
fixed the situation once the specific RSET text in RFC 2821 was shown to 
them:

    4.1.1.5 RESET (RSET)

    A reset command may be issued by the client at any time.  It is
    effectively equivalent to a NOOP (i.e., if has no effect) if issued
    immediately after EHLO, before EHLO is issued in the session, after
    an end-of-data indicator has been sent and acknowledged, or
    immediately before a QUIT.

So here, we can pop up a bottle of champagne and proclaim victory:

     "The system work! The RFC docs showed the errors of
      their ways!"

But after the situation was resolved, we made a decision here to put in 
a change request in our SMTP client software to make the RSET command 
optional in the EHLO/HELO fall back to reduce future support cost issues.

I looked at RFC 2821 section 3.2

    3.2 Client Initiation

    Once the server has sent the welcoming message and the client has
    received it, the client normally sends the EHLO command to the
    server, indicating the client's identity.  In addition to opening the
    session, use of EHLO indicates that the client is able to process
    service extensions and requests that the server provide a list of the
    extensions it supports.  Older SMTP systems which are unable to
    support service extensions and contemporary clients which do not
    require service extensions in the mail session being initiated, MAY
    use HELO instead of EHLO.  Servers MUST NOT return the extended
    EHLO-style response to a HELO command.  For a particular connection
    attempt, if the server returns a "command not recognized" response to
    EHLO, the client SHOULD be able to fall back and send HELO.

If someone reads RFC 2821, I guess, in reading the last sentence, one 
can reasonably argue how one may get the idea that a RSET is not 
expected after an EHLO command, hence issuing a "503 Bad sequence of 
commands" response as this large appliance SMTP server did.  I don't 
think that argument jives 4.1.1.5, but I can see where that mistake can 
be made.

One the other hand, if someone reads RFC 821, there is little semantics 
on when RSET can be issued.  So in some defense for the vendor, I can 
also see how their software offers an option to run in RFC 821 legacy 
mode and may see the RSET before a HELO as out of sequence.

Who is at truly at fault?  Who knows?  I really don't care.  Our 
software has the RSET in its EHLO/HELO fallback for a engineering 
reason, I'm sure.  But because of the above possibilities, we are going 
to make the change too.

Summary and recommendation:

I am not sure which way you may to do this.

  a) Add Text to 3.2 to make servers aware a RSET after EHLO is
     possible?

  b) Do not allow RSET after EHLO for possible 821 historical reasons?

The latter may be a problem with existing software designed around 2821 
which may already have logic to issue RSET after EHLO.

Personally, I prefer the former (a).

I think section 4.1.1.5 is fine.  I think maybe the last sentence in 3.2 
first paragraph can be slightly changed, augmented with additional 
clauses, sentence or some note.

I am just giving an example, maybe others can write it better.

    For a particular connection attempt, if the server returns
    a "command not recognized" response to EHLO, the client
    SHOULD be able to fall back and send HELO.

    Note: Clients MAY issue the RSET command after the EHLO command
    before issuing the HELO fall back command.  Servers MUST not
    view this as an out of sequence state.

By having a simple text like above, future servers will not fall into 
this RSET after EHLO error.

Hope this make sense and I hope I provide enough reason, information and 
  how it can be misinterpreted as reflected in a large mail vendor behavior.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: The "such responses" extension sentence
Date:  Sun, 24 Feb 2008 22:12:05 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 9
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

John C Klensin wrote:

> My instinct at the moment (i.e., I reserve the right to
> change my mind) is that dropping it would be better.

+1 following Derek's argument.=20

Date: Sun, 24 Feb 2008 15:42:57 -0500
From: John C Klensin <[email-masked]>
To: [email-masked]
cc: John Leslie <[email-masked]>
Subject: The "such responses" extension sentence (was: Re: MX to CNAME and (mis)interpretation of 2821)
Sender: [email-masked]

Hi.

After further thought and several comments (some off-list) from 
people who don't feel that it is a showstopper one way or the 
other but would prefer to see it go, I propose the following.

This will make no sense unless you have read notes from the 
earlier thread today.

The current sentence reads

    "Such responses could, in principle, be covered by an SMTP
    extension in the future."

Since there is a normative DNS prohibition on CNAME targets (in 
RFC2181), that sentence is, strictly speaking, untrue.  It could 
be corrected by saying:

    "Such responses could, in principle, be covered by DNS
    changes or extensions and an SMTP extension in the future."

Or we could drop it.   My instinct at the moment (i.e., I 
reserve the right to change my mind) is that dropping it would 
be better.  Given the language that already occurs elsewhere in 
2821/ 2821bis about the things that extensions can change, it 
doesn't really add any information.  And, unlike the original 
form, this is started to get convoluted.  2821bis already has 
enough of a surplus of convoluted text and another instance, 
while not a horrible precedent, is not needed.

I'd like to hear from John Leslie, who originally suggested 
putting something like that in.  If he is now persuaded that it 
should be dropped, I'd find that convincing and perhaps others 
would as well.  On the other hand, if he has a strong argument 
for retaining it (presumably in the revised form above), we 
should all hear and consider it.

      john

Cc: Hector Santos <[email-masked]>, Alessandro Vesely <[email-masked]>, SM <[email-masked]>, [email-masked]
From: "Derek J. Balling" <[email-masked]>
To: John C Klensin <[email-masked]>
Subject: Re: MX to CNAME and (mis)interpretation of 2821
Date: Sun, 24 Feb 2008 13:35:14 -0500
Sender: [email-masked]

--Apple-Mail-36-85158822
Content-Type: text/plain;
	charset=US-ASCII;
	format=flowed;
	delsp=yes
Content-Transfer-Encoding: 7bit

On Feb 24, 2008, at 1:06 PM, John C Klensin wrote:
>    I am a little dubious about the practical likelihood of the  
> sentence starting with "Such responses...", but don't see it as  
> harmful. Any such extension would, of course, have to update 2181 or  
> equivalent as well as extending SMTP.

My $0.02 worth:

I think you're right that's dubious that what this sentence implies  
would ever come into play. My concern would be that some idiot will  
say "Well, see, right there is says that the times are a'changin', and  
you're gonna have to accept it sooner or later", even though that's  
not at all the intent of that sentence.

I think that what you're saying in that sentence is true -- that a  
future RFC/STD could in theory make it permissible to do -- but I also  
feel that doesn't need to be said. After all, RFCs have been  
superseding other RFCs for years. This document doesn't need to grant  
permission to its descendants to do so.

That said, I don't think that verbiage is the end of the world (and  
its encapsulating para is a world of improvement). IMHO, I would drop  
that sentence. But I could live with it, grudgingly, if that was the  
price of moving this forward quickly.

Cheers,
D

--Apple-Mail-36-85158822
Content-Disposition: attachment;
	filename=smime.p7s
Content-Type: application/pkcs7-signature;
	name=smime.p7s
Content-Transfer-Encoding: base64

--Apple-Mail-36-85158822--

From: [email-masked]
Cc: Hector Santos <[email-masked]>, Alessandro Vesely <[email-masked]>, SM <[email-masked]>, [email-masked]
Date: Sun, 24 Feb 2008 10:27:29 -0800 (PST)
Subject: Re: MX to CNAME and (mis)interpretation of 2821
To: John C Klensin <[email-masked]>
Sender: [email-masked]

> Folks,

> This has been an interesting discussion.  However, the problem
> remains a DNS one and it is covered quite clearly in RFC 2181
> (something I didn't fully realize when I wrote my previous note
> on the subject).

> The tentative text in 2821bis-08 now reads as follows, in the
> hope of being tediously clear (thanks go to John Leslie for the
> core of this and to several others for convincing me that even
> more clarification was needed):

> # When a domain name associated with an MX RR is looked up and
> # the associated data field obtained, the data field of that
> # response MUST contain a domain-name.  That domain-name, when
> # queried, MUST return at least one address record (e.g., A or
> # AAAA RR) that gives the IP address of the SMTP server to
> # which the message should be directed.  Any other response,
> # specifically including a value that will return a CNAME
> # record when queried, lies outside the scope of this
> # standard.  Such responses could, in principle, be covered by
> # an SMTP extension in the future.  The prohibition on labels
> # in the data that resolve to CNAMEs is discussed in more
> # detail in RFC 2181, Section 10.3 [29].

This seems fine to me.

> Please note that this does not forbid CNAMEs except by example.
> It forbids literal addresses and any label that points to a
> non-address record type. That is consistent with 2181.   I am a
> little dubious about the practical likelihood of the sentence
> starting with "Such responses...", but don't see it as harmful.

Agreed.

> Any such extension would, of course, have to update 2181 or
> equivalent as well as extending SMTP.

Seems fairly unlikely, but as you say, it's harmless.

> Now, while I would welcome suggestions for additional
> improvements, I don't believe that it is worth spending a lot
> more time on the subject, especially since these are
> post-last-call changes.   So, if you make comments, please
> explicitly indicate whether or not you can live with the current
> text.  If you can, I'll apply editorial judgment.  If you
> cannot, it becomes Tony's problem to create a formal issue and
> manage a discussion of it.

This doesn't constrain client handling of invalid cases so I'm good with
it.

				Ned

From: [email-masked]
Cc: Hector Santos <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Date: Sun, 24 Feb 2008 07:03:47 -0800 (PST)
Subject: Re: rfc2821bis-07
To: John C Klensin <[email-masked]>
Sender: [email-masked]

> --On Thursday, 21 February, 2008 16:20 -0500 Hector Santos
> <[email-masked]> wrote:

> > Unless I am missing something, it isn't just better to say
> > something to the effect?
> >
> >     "SMTP DNS Administration MUST|SHOULD NOT include CNAME resource
> >      records when creating email domain MX records for the SMTP
> >      server setup."
> >
> > I mean,  I don't think it is reasonable to discourage DNS
> > CLIENT software to ignore the very good possibility that a
> > query might historically return a CNAME which needs to resolve
> > as well for the final expansion.

I'm mostly in agreement with Hector here.

> To the extent to which we consider such CNAMEs to be a problem,
> the _only_ way to get rid of them is precisely to encourage
> client software to complain.

Exactly what form is this "encouragement" supposed to take? From a client
operator's point of view you have three options when you encounter an MX that
resolves to a CNAME:

(1) Resolve the CNAMEs and use the results if they make sense, bounce if they
    do not.
(2) Same as (1), but send a warning somewhere.
(3) Reject the mail unconditionally.

The main concern of client operators is always to insure mail is successfully
relayed off their system. In the present climate of filtering, blacklisting,
feedback loops, and other assorted fallout of the fight against spam, this has
become increasingly difficult and time-consuming, especially if you're dealing
with a large volume of messages.

So where's the incentive to be picky about MXes resolving to CNAMEs? If a
client operator rejects mail on this basis it preumably gets reported to the
originator, who is (a) Not in a position to fix the problem and (b) Is then
likely to contact the client operator and complain. Support calls are a
significant expense for many client operators and ones which the client
operator is not going to be able to fix the problem anyway are a resounding
lose all around.

I therefore assert that option (3) makes absolutely no sense operationally.

As for option (2), the question then becomes who to alert about the problem.
Sending mail about it to, say, postmaster@problemdomain is pretty much
guaranteed to be a waste of time. And even if it it manages to find someone
who cares, engaging in a dialogue with someone who doesn't know how
to set up MX records is not helping the client operator any.

I therefore claim that option (2) is, at the very best, "mostly harmless" and
makes little if any operational sense.

Now, from an implementor's perspective, depending on the APIs you use it can
actually be difficult to turn off CNAME resolution, and even more difficult to
detect that an address lookup went through CNAME redirection. So here we
have something that is hard to implement and makes no operational sense.

So I repeat: What encouragement are we able to provide to get people to
implement and operate things this way? We are now light years past the point
where people do stuff simply because they get warm fuzzies from conforming to
sometihng a specification recommends.

(Of course none of this applies if the CNAME causes a loop - that has to be
dealt with. And if enforcing this restriction prevented all such loops that
would be ample justification for it. But it doesn't do that: There are many
other ways MX records and MTAs can be misconfigured to cause a loop without
using CNAMEs.)

> If we don't care, this should be a
> SHOULD at most (and a SHOULD here will be taken as license to
> use CNAMEs there).  The audience for this document is not, for
> better or worse, domain administrators.

Frankly, I think we're being awfully kind to ourselves if we think whatever we
say is going to give license, or prevent for that matter, this behavior. As you
say, domain administrators aren't going to read this no matter what it says. 
The reality is that having MXes point at CNAMEs mostly works and this is
unlikely to change. And that means people will on occasion set it up, find it
works, and go with it.

> > In other words, am I suppose to remove the logic from my DNS
> > client based on the idea that they should never be a CNAME in
> > a MX query? Sure, its the other guys fault, but I don't think
> > any implication to abort, abend or stop a transaction because
> > of it is reasonable.

> At some level, we don't care what you do that goes beyond the
> bounds of the standard to deal with a case that the standard
> prohibits.  If you come to the conclusion that you should handle
> these things, then you should do it -- you are still conforming
> wrt the specified behavior.

This ignores the fact that in many cases it is actually more work to prohibit
CNAME resolution than to allow it. You might get some traction if it was mostly
the other way around and client vendors had to go to some effort to handle
CNAME, but that's not how it is.

> The only thing that would make you
> non-conforming would be an explicit statement that says that you
> MUST reject such a thing.  And it rather carefully doesn't say
> that.

Which IMO is the only reasonable course. Although I think the continued
prohibition of MXes pointing to CNAMEs is a little silly I have no problem
retaining it as long as it doesn't recommend, let alone require, clients check
for and reject or warn in this case.
 
> > To paraphrase Postel's Law:
> >
> >     Be conservative with what you do (define proper MX record setup),
> >     be liberal in what you receiver. (CNAME might appear in MX
> >     expansions)

> Sigh.  Without going into a long rant, if this type of
> interpretation of the robustness principle were widely applied,
> we wouldn't have an interoperable email fabric today because
> various extensions and other changes would have been impossible
> and everything would have long ago descended to some least
> common denominator (or below it).

Taken to an extreme, perhaps - but anything taken to an extreme tends to be
bad. However, email actually affords us some opportunity to observe the merits
of this approach to the robustness principle in an operational context. We have
essentially four protocols here: RFC2822/MIME, SMTP/SUBMIT, POP3, and IMAP4.
SMTP and RFC2822/MIME are in many ways the embodiment of the robustness
principle: The specifications are fairly loose syntactically and allow for lots
of variability plus Implementations tend to be very forgiving and will in many
cases attempt to make sense out of complete crap. (And wonder of wonders, they
even succeed sometimes.)

IMAP4 and to a lesser extent POP3 is the exact opposite. The specifications are
syntactically pretty tight and implementations tend to follow suit. Slight
deviations from the specifications tend to give rise to errors.

So what has this accomplished? The SMTP relay fabric, despite haveing taken
numerous hits in recent times from various well-meaning but misguided attempts
to combat spam, is operationally very robust and achieves an extremely high
degree of interoperability. Indeed, we're now at a point where reports of SMTP
interop problem reports are now pretty rare (for us at least).

The same cannot be said for IMAP4 and to a lesser extent POP3. Problems abound.
We have have vendors who insist on reading those tightly written specifications
in very wierd ways, we have vendors who cut corners left and right  on
compliance to those specifications, we have clients that insist on certain
extensions being present when they could easily do without, and so on. It's
gotten to the point where longtime implementors like Mark Crispin are publicly
pessimistic about what the future holds. (Mark, feel free to correct me if you
feel I've misstated your position here.)

Of course this is a far from perfect comparison, for several reasons. One is
that IMAP4 is a lot more complex than SMTP and therefore easier to screw up.
OTOH, the interplay between state changes and error handling in SMTP is
surprisingly complex in practice, especially in the pipelining case. And both
protocols have tons of extensions defined, which adds sigifnicant complexity to
both.

There are also differences in usage patterns that arguably make it easier for
problems to creep into IMAP4 setups and stay.

But even taking all the equities into account, I think there's still a case to
be made that laxity taken in moderation has helped SMTP more than it has hurt.
(Note, however, that there's a very big difference between tightening down a
loosely specified protocol versus loosening up a tightly specified protocol.
I'm arguing that the former is bad, not that the latter is good.)

Indeed, the one problem with SMTP interoperability we still encounter from time
to time is the direct result of a tightening of the rules in RFC 2821: The
requirement that bare CR and LF not be interpreted as a line break. When RFC
2821 came out with this change we changed our default behavior and it has
caused nothing but problems. (We no longer get reports of this very often
because we have now advised many customrs to revert to the old way of doing
things, standards compliance be damned. I should also add that I can count the
number of times tightening things down actually helped us solve an interop
problem in this area using just my index fingers.)

> The current text precisely
> permits you to apply the intent of that principle because it
> doesn't prohibit you from accepting and interpreting something
> that is non-conforming to the standard.  It just doesn't require
> you to do that, which would effectively allow CNAME in all cases.

Which is why I have no objection to the current language. After the experience
with bare CR and LF I would strongly object to any similar change.

> > My apology if I missed something here.  I certainly don't wish
> > to waste anyone's time.

> No, I think your question is quite reasonable.  But this is very
> much one of the reasons why neither 2821bis nor its predecessors
> make unqualified references to 2119.

> And, if you think that concept needs to be explained better in
> the text, please suggest words.

I don't especially care what the words say as long as clients are
free to handle this case as they see fit.

				Ned

Date: Sun, 24 Feb 2008 13:06:30 -0500
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, Alessandro Vesely <[email-masked]>, SM <[email-masked]>
cc: [email-masked]
Subject: Re: MX to CNAME and (mis)interpretation of 2821
Sender: [email-masked]

Folks,

This has been an interesting discussion.  However, the problem 
remains a DNS one and it is covered quite clearly in RFC 2181 
(something I didn't fully realize when I wrote my previous note 
on the subject).

The tentative text in 2821bis-08 now reads as follows, in the 
hope of being tediously clear (thanks go to John Leslie for the 
core of this and to several others for convincing me that even 
more clarification was needed):

# When a domain name associated with an MX RR is looked up and
# the associated data field obtained, the data field of that
# response MUST contain a domain-name.  That domain-name, when
# queried, MUST return at least one address record (e.g., A or
# AAAA RR) that gives the IP address of the SMTP server to
# which the message should be directed.  Any other response,
# specifically including a value that will return a CNAME
# record when queried, lies outside the scope of this
# standard.  Such responses could, in principle, be covered by
# an SMTP extension in the future.  The prohibition on labels
# in the data that resolve to CNAMEs is discussed in more
# detail in RFC 2181, Section 10.3 [29].

Please note that this does not forbid CNAMEs except by example. 
It forbids literal addresses and any label that points to a 
non-address record type. That is consistent with 2181.   I am a 
little dubious about the practical likelihood of the sentence 
starting with "Such responses...", but don't see it as harmful. 
Any such extension would, of course, have to update 2181 or 
equivalent as well as extending SMTP.

Now, while I would welcome suggestions for additional 
improvements, I don't believe that it is worth spending a lot 
more time on the subject, especially since these are 
post-last-call changes.   So, if you make comments, please 
explicitly indicate whether or not you can live with the current 
text.  If you can, I'll apply editorial judgment.  If you 
cannot, it becomes Tony's problem to create a formal issue and 
manage a discussion of it.

      john

Date: Sun, 24 Feb 2008 09:51:56 -0800
To: Alessandro Vesely <[email-masked]>, [email-masked]
From: SM <[email-masked]>
Subject: Re: MX to CNAME and (mis)interpretation of 2821
Sender: [email-masked]

At 05:44 24-02-2008, Alessandro Vesely wrote:
>The situation is cumbersome, since rfc1035 says nothing on MX 
>semantics, referring to rfc974. Indeed, the algorithm is explained 
>in rfc974, but it is "Obsoleted by: 2821", the SMTP being updated. 
>However, rfc2181 updates rfc1035 and has a section 10.3 "MX and NS 
>records" where it says
>
>  The domain name used as the value of a NS resource record, or part of
>  the value of a MX resource record must not be an alias.  Not only is
>  the specification clear on this point, but using an alias in either
>  of these positions neither works as well as might be hoped, nor well
>  fulfills the ambition that may have led to this approach.  This
>  domain name must have as its value one or more address records.
>
>Thus, it seems it is up to rfc2821bis to be "clear on this point."

See http://www.imc.org/ietf-smtp/mail-archive/msg04337.html for the 
clarification.

Regards,
-sm 

Date: Sun, 24 Feb 2008 11:23:31 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Alessandro Vesely <[email-masked]>
CC: [email-masked]
Subject: Re: MX to CNAME and (mis)interpretation of 2821
Sender: [email-masked]

Alessandro Vesely wrote:
> 
> The situation is cumbersome, since rfc1035 says nothing on MX semantics, 
> referring to rfc974. Indeed, the algorithm is explained in rfc974, but 
> it is "Obsoleted by: 2821", the SMTP being updated. However, rfc2181 
> updates rfc1035 and has a section 10.3 "MX and NS records" where it says
> 
>  The domain name used as the value of a NS resource record, or part of
>  the value of a MX resource record must not be an alias.  Not only is
>  the specification clear on this point, but using an alias in either
>  of these positions neither works as well as might be hoped, nor well
>  fulfills the ambition that may have led to this approach.  This
>  domain name must have as its value one or more address records.
> 
> Thus, it seems it is up to rfc2821bis to be "clear on this point."

+1.

I prefer the less "Mumbo Jumbo" version:

    SMTP DNS Administration SHOULD|MUST NOT use CNAME resource records
    in preparing MX domain hosting records.

    SMTP and DNS client resolvers SHOULD|MUST be aware for the
    possibility of CNAME resource records and handle it accordingly
    during the MX query expansion process.

After all, is what it is.  Most, if not all, professional DNS client 
resolvers are already dealing with this long time possible issue.   The 
new document isn't going to change that, so at best it can only help the 
administration side of operations not to use them, and in my view, a 
insight on the growth on Web Based DNS Managers that is SO easy today to 
use even the layman SMTP operator can use.

In the past, personally, I think the old Microsoft DNS manager, had 
logic that didn't help the non-experience DNS people in "accidentally" 
creating a CNAME for the MX record.  For example, I recall a customer 
first use a WWW.XYZ.COM CNAME record for their entry into the Web market 
and then use this for the MX when adding the mail system.

My input on the matter.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Sun, 24 Feb 2008 14:44:23 +0100
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: MX to CNAME and (mis)interpretation of 2821
Sender: [email-masked]

SM wrote:
> At 02:15 23-02-2008, Alessandro Vesely wrote:
>> An MTA learns if it is part of the MX-list retrieved for delivering a 
>> given message by either comparing MX names to its own canonical name 
>> or checking if an MX IP number belongs to one of its local interfaces. 
>> However, the latter method doesn't work across NATs or split DNSes. 
>> Thus, setting the canonical name in the MX records is required for 
>> reliable operations of backup MXes.
> 
> Setting the canonical name in the MX records is not required as you can 
> always add the domain name to the MTA configuration.

IMHO, that name in the MTA configuration is what one should call 
"canonical". Besides using CNAME, one can also fool a backup MX by 
setting several vanity names all having equal A (or AAAA) records. An 
MTA may fail when it discovers that the IP it is about to deliver is 
local. Thus, a backup MX won't work even through the DNS is apparently 
compliant, as no CNAMEs are being used and each host on the MX list 
has a corresponding address record.

The situation is cumbersome, since rfc1035 says nothing on MX 
semantics, referring to rfc974. Indeed, the algorithm is explained in 
rfc974, but it is "Obsoleted by: 2821", the SMTP being updated. 
However, rfc2181 updates rfc1035 and has a section 10.3 "MX and NS 
records" where it says

  The domain name used as the value of a NS resource record, or part of
  the value of a MX resource record must not be an alias.  Not only is
  the specification clear on this point, but using an alias in either
  of these positions neither works as well as might be hoped, nor well
  fulfills the ambition that may have led to this approach.  This
  domain name must have as its value one or more address records.

Thus, it seems it is up to rfc2821bis to be "clear on this point."

>> Even if backup MXes are not widely used today [...]
> 
> The MX to CNAME restriction is not about a ban on backup MXes.

On the opposite, backup MXes are the only reason for that restriction, 
AFAIK.

Date: Sat, 23 Feb 2008 10:23:07 -0800
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: MX to CNAME and (mis)interpretation of 2821
Sender: [email-masked]

At 02:15 23-02-2008, Alessandro Vesely wrote:
>An MTA learns if it is part of the MX-list retrieved for delivering 
>a given message by either comparing MX names to its own canonical 
>name or checking if an MX IP number belongs to one of its local 
>interfaces. However, the latter method doesn't work across NATs or 
>split DNSes. Thus, setting the canonical name in the MX records is 
>required for reliable operations of backup MXes.

Setting the canonical name in the MX records is not required as you 
can always add the domain name to the MTA configuration.

>Even if backup MXes are not widely used today, I don't think it 
>would be a good idea to ban them for good, since there are fault 
>tolerant functionalities that cannot be obtained otherwise. It is 
>important to standardize this point clearly because fault tolerance 
>is obtained by enabling backup MXes on different networks, hence 
>likely involving different organizations. We'll need that stuff in 
>case WAN connections suddenly were to crash quite often.

The MX to CNAME restriction is not about a ban on backup MXes.

Regards,
-sm 

Date: Sat, 23 Feb 2008 11:15:20 +0100
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: MX to CNAME and (mis)interpretation of 2821
Sender: [email-masked]

Frank Ellermann wrote:
> John C Klensin wrote:
> 
>> the historical reason for the prohibition on names that point
>> to CNAME RRs in the data field of MX RR was, I believe, to
>> reduce the risk of looping within the DNS.
> 
> The explanation in RFC 974 is more convoluted, something about
> figuring out what's LOCAL and what's REMOTE.

An MTA learns if it is part of the MX-list retrieved for delivering a 
given message by either comparing MX names to its own canonical name 
or checking if an MX IP number belongs to one of its local interfaces. 
However, the latter method doesn't work across NATs or split DNSes. 
Thus, setting the canonical name in the MX records is required for 
reliable operations of backup MXes.

Even if backup MXes are not widely used today, I don't think it would 
be a good idea to ban them for good, since there are fault tolerant 
functionalities that cannot be obtained otherwise. It is important to 
standardize this point clearly because fault tolerance is obtained by 
enabling backup MXes on different networks, hence likely involving 
different organizations. We'll need that stuff in case WAN connections 
suddenly were to crash quite often.

Date: Fri, 22 Feb 2008 19:51:45 -0500
From: John C Klensin <[email-masked]>
To: Frank Ellermann <[email-masked]>, [email-masked]
Subject: Re: MX to CNAME and (mis)interpretation of 2821 (was: rfc2821bis-07)
Sender: [email-masked]

--On Friday, 22 February, 2008 20:25 +0100 Frank Ellermann
<[email-masked]> wrote:

> 
> John C Klensin wrote:
> 
>> if we stick with Option 1 and the language isn't clear enough,
>> the relevant sentence can be supplemented with an explicit 
>> statement to the effect that any values that point to CNAMEs
>> that appear are nonconformant and out of the scope of the
>> standard.
> 
> That would be good and hopefully address Trevor's concerns in
> <http://thread.gmane.org/gmane.ietf.smtp/6517/focus=6525>  
> 
> Paul Smith proposed 2007-12-14 (getting two ACKs):
> | 2821bis should state that "putting CNAMES as the target of
> | MX records is not allowed, as stated in RFC 2181"

After an exchange of notes with John Leslie, we have arrived at
a way to handle this that is consistent with the above, but that
really spells out the restriction and points to RFC 2181 for
additional explanation.

Incidentally, 2181 says "address records" and then does a little
handwaving about "other record types giving addressing
information".  The proposed 2821bis text now says "...one
address record (e.g., A or AAAA RR)..." which should eliminate
any possible doubt.

   john

Date: Fri, 22 Feb 2008 16:55:21 -0500
From: John C Klensin <[email-masked]>
To: Jeff Macdonald <[email-masked]>
cc: [email-masked]
Subject: Re: White space in mail addresses - new issue 2821bis
Sender: [email-masked]

--On Friday, 22 February, 2008 14:15 -0500 Jeff Macdonald
<[email-masked]> wrote:

> On Fri, Feb 22, 2008 at 01:37:20PM -0500, John C Klensin wrote:
>> 
>> Hi.
>> 
>> The discussion about whether HT should be included as a
>> permitted character in email message local-parts has
>> identified another issue that might benefit from explanation.
>> These things were discussed as "any escaped ASCII character
>> from %0 to %127" or "control characters" in the 821/2821
>> context.  Since we got rid of the control characters, we've
>> started talking about white space and whether or not HT is
>> valid white space.
>> 
>> But, one way or the other, it isn't white space (in the 2822
>> and ABNF sense) either.  Any of these characters that are
>> permitted in address local-parts are just characters.  In
>> particular, 821/2821/2821bis make no guarantees that any pair
>> of  "joe jones"@example.com  (one SP character)
>>    "joe  jones"@example.com (two SP characters)
>>    "joe	jones"@example.com (one HT character)
>>    "joe 	jones"@example.com (one SP character, then one HT
>> character)
>> 
>> Will reach the same mailbox.  
>> 
>> Should we insert a comment in paragraph 2 of section 2.4
>> (which discusses case matching or the lack thereof) that
>> points this out?  If so, should we encourage implementations
>> to treat these sequences as "white space", just as we
>> encourage them to treat addresses as case-insensitive?
>> 
>> It seems to me that this would probably be useful if we leave
>> HT out of the valid character list and even more so if we
>> include it.
> 
> Isn't the local part interpretation up to the receiving
> system? Those
> may be the same mailboxes for example.com but not foo.com?

Absolutely.   The cited paragraph is an explicit warning against
senders assuming that the receiving system will treat strings
that different in case as equivalent and advice to receiving
systems that making that distinction may confuse a lot of
people.  What I was suggesting was augmenting that text to point
out that, while we generally treat all "white space" strings as
equivalent, it is equally inappropriate (vis-a-vis
case-sensitivity) to make that assumption in sending and that
receivers should probably exercise caution.

Sorry if I wasn't clear.

    john

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  MX to CNAME and (mis)interpretation of 2821 (was: rfc2821bis-07)
Date:  Fri, 22 Feb 2008 20:25:53 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 23
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

John C Klensin wrote:

> if we stick with Option 1 and the language isn't clear enough,
> the relevant sentence can be supplemented with an explicit=20
> statement to the effect that any values that point to CNAMEs
> that appear are nonconformant and out of the scope of the
> standard.

That would be good and hopefully address Trevor's concerns in
<http://thread.gmane.org/gmane.ietf.smtp/6517/focus=3D6525> =20

Paul Smith proposed 2007-12-14 (getting two ACKs):
| 2821bis should state that "putting CNAMES as the target of
| MX records is not allowed, as stated in RFC 2181"

> the historical reason for the prohibition on names that point
> to CNAME RRs in the data field of MX RR was, I believe, to
> reduce the risk of looping within the DNS.

The explanation in RFC 974 is more convoluted, something about
figuring out what's LOCAL and what's REMOTE.

 Frank

Date: Fri, 22 Feb 2008 14:15:01 -0500
From: Jeff Macdonald <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked]
Subject: Re: While space in mail addresses - new issue 2821bis
Sender: [email-masked]

On Fri, Feb 22, 2008 at 01:37:20PM -0500, John C Klensin wrote:
>
>Hi.
>
>The discussion about whether HT should be included as a
>permitted character in email message local-parts has identified
>another issue that might benefit from explanation.  These things
>were discussed as "any escaped ASCII character from %0 to %127"
>or "control characters" in the 821/2821 context.  Since we got
>rid of the control characters, we've started talking about white
>space and whether or not HT is valid white space.
>
>But, one way or the other, it isn't white space (in the 2822 and
>ABNF sense) either.  Any of these characters that are permitted
>in address local-parts are just characters.  In particular,
>821/2821/2821bis make no guarantees that any pair of 
>    "joe jones"@example.com  (one SP character)
>    "joe  jones"@example.com (two SP characters)
>    "joe	jones"@example.com (one HT character)
>    "joe 	jones"@example.com (one SP character, then one HT
>character)
>
>Will reach the same mailbox.  
>
>Should we insert a comment in paragraph 2 of section 2.4 (which
>discusses case matching or the lack thereof) that points this
>out?  If so, should we encourage implementations to treat these
>sequences as "white space", just as we encourage them to treat
>addresses as case-insensitive?
>
>It seems to me that this would probably be useful if we leave HT
>out of the valid character list and even more so if we include
>it.

Isn't the local part interpretation up to the receiving system? Those
may be the same mailboxes for example.com but not foo.com?

-- 
:: Jeff Macdonald | Director of Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421
:: v: 781-372-1922 | f: 781-863-8118
:: www.e-dialog.com

Date: Fri, 22 Feb 2008 13:37:20 -0500
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: While space in mail addresses - new issue 2821bis
Sender: [email-masked]

Hi.

The discussion about whether HT should be included as a
permitted character in email message local-parts has identified
another issue that might benefit from explanation.  These things
were discussed as "any escaped ASCII character from %0 to %127"
or "control characters" in the 821/2821 context.  Since we got
rid of the control characters, we've started talking about white
space and whether or not HT is valid white space.

But, one way or the other, it isn't white space (in the 2822 and
ABNF sense) either.  Any of these characters that are permitted
in address local-parts are just characters.  In particular,
821/2821/2821bis make no guarantees that any pair of 
    "joe jones"@example.com  (one SP character)
    "joe  jones"@example.com (two SP characters)
    "joe	jones"@example.com (one HT character)
    "joe 	jones"@example.com (one SP character, then one HT
character)

Will reach the same mailbox.  

Should we insert a comment in paragraph 2 of section 2.4 (which
discusses case matching or the lack thereof) that points this
out?  If so, should we encourage implementations to treat these
sequences as "white space", just as we encourage them to treat
addresses as case-insensitive?

It seems to me that this would probably be useful if we leave HT
out of the valid character list and even more so if we include
it.

    john

Date: Fri, 22 Feb 2008 12:52:09 -0500
From: John C Klensin <[email-masked]>
To: John Leslie <[email-masked]>
cc: Hector Santos <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-07
Sender: [email-masked]

--On Friday, 22 February, 2008 10:25 -0500 John Leslie
<[email-masked]> wrote:

> 
> John C Klensin <[email-masked]> wrote:
>> --On Thursday, 21 February, 2008 16:20 -0500 Hector Santos
>> <[email-masked]> wrote:
>> 
>>> Unless I am missing something, it isn't just better to say
>>> something to the effect?
>>> 
>>>     "SMTP DNS Administration MUST|SHOULD NOT include CNAME
>>>      resource records when creating email domain MX records
>>>      for the SMTP server setup."
> 
>    John K doesn't seem to have responded to this particular
> text.
> 
>>> I mean,  I don't think it is reasonable to discourage DNS
>>> CLIENT software to ignore the very good possibility that a
>>> query might historically return a CNAME which needs to
>>> resolve as well for the final expansion.
>> 
>> To the extent to which we consider such CNAMEs to be a
>> problem, the _only_ way to get rid of them is precisely to
>> encourage client software to complain. If we don't care, this
>> should be a SHOULD at most (and a SHOULD here will be taken
>> as license to use CNAMEs there).

Huh?  The comment quoted above _was_ my response.  What would
you like to convince you I have responded?

>    Personally, I think this battle is long since lost;
> however...
> 
>> The audience for this document is not, for better or worse,
>> domain administrators.
> 
>    ... which is why, IMHO, we need to be especially clear if we
> wish to restrict what they do. They _may_ read a paragraph or
> two when somebody points out "the error of their ways" -- but
> they will _not_ read for context.
> 
>    The text in question, IMHO, is not sufficiently clear:
> " 
> " The result of an MX lookup MUST NOT be a CNAME.
> 
> could mean a number of things -- one of the most logical being
> that the SMTP client software is _obligated_ to resolve any
> CNAME it encounters. (The DNS administrators reading this text
> would know there are any number of middlemen between what they
> configure and what the system code called by client software
> may return.)
> 
>> At some level, we don't care what you do that goes beyond the
>> bounds of the standard to deal with a case that the standard
>> prohibits.  If you come to the conclusion that you should
>> handle these things, then you should do it -- you are still
>> conforming wrt the specified behavior.  The only thing that
>> would make you non-conforming would be an explicit statement
>> that says that you MUST reject such a thing.  And it rather
>> carefully doesn't say that.
> 
>    ... but it could be read to mean you MUST NOT reject such a
> thing...

I just do not see how it could be read that way, but bow to your
obviously more creative imagination.

>> And, if you think that concept needs to be explained better in
>> the text, please suggest words.
> 
>    If we're trying in any way to control what DNS
> administrators do, we should have text like:
> " 
> " DNS administrators MUST NOT configure DNS servers for zones
> " they control to return a CNAME for an MX lookup.
> 
>    If we're trying to control what (some) SMTP client software
> does, we should have text like:
> " 
> " SMTP client software SHOULD return an error if it receives a
> CNAME RR in response to an MX lookup.
> 
> ... or, of course, "MAY return an error..." -- I have no horse
> in this race. We could include both, or neither, for all I
> care. I just think the current text is not helpful.

Let me repeat what I tried to say to Hector.   The question
isn't what we tell the client, it is what we tell server
operators and, by extension --but _only_ by extension-- those
who set up DNS configurations.

	Option 1: If you put a string associated with a CNAME
	record in the DATA field of an MX record, you are
	violating the SMTP standard and all bets are off.  The
	client system can use its own imagination about what to
	do with your mail traffic but certainly isn't obligated
	to process it.
	
	Option 2: If you put a string associated with a CNAME
	record in the DATA field of an MX record, the client
	system is expected to process it.   Note that "is
	expected to" can be expressed as either a MUST or SHOULD
	requirement on the client -- the sender has the right to
	assume that it will probably work.

	Option 3: If you put a string associated with a CNAME in
	the DATA field of an MX record, the client is required
	to reject it.  I.e., that form will not work with any
	conforming client.
	
	Option 4: If you put a string associated with a CNAME
	record in the DATA field of an MX records, the client
	system may process it or may not; you are engaging in a
	guessing game.

I suggest that option 4 makes no sense at all in a standard.  We
shouldn't do guessing games.   Option 2, however worded, is the
option we have rejected, i.e., that data strings that identify
CNAMEs are fully valid and one can expect (by invoking the
robustness principle if necessary) that they will be looked up
and resolved.

And that leaves us with Option 1.  Note that, while we decided
in DRUMS to leave the problem to the DNS specs and so 2821
doesn't say anything about it (at least that I could find
quickly), this is exactly the same way we handle the broken
practice of putting IP addresses, rather than domain names, in
the data fields of MX records -- it is outside the standard, we
don't comment on it, but we don't _require_ that clients reject
such things.   That gives Hector (and others) the flexibility
they want without either (i) making them non-conformant or (ii)
requiring that everyone, in practice, support CNAME resolution
out of MX records.

Certainly, if we stick with Option 1 and the language isn't
clear enough, the relevant sentence can be supplemented with an
explicit statement to the effect that any values that point to
CNAMEs that appear are nonconformant and out of the scope of the
standard.

Now, all of that said, the historical reason for the prohibition
on names that point to CNAME RRs in the data field of MX RR was,
I believe, to reduce the risk of looping within the DNS.  That
decision was made before there were NAPTR records (or the
newly-proposed URI record), which provide such ample
opportunities for loops that any sensible DNS resolver software
must, of necessity, take precautions against the general case.
If we believe that knowledge is sufficient protection, then the
instructions to clients can sensibly to changed to "MAY" (or, if
it were not for the "no new features" rule and
interoperability/implementation testing, "SHOULD")-- it leaves
things tad unpredictable but doesn't cause significant
conformance issues.

   john

Date: Fri, 22 Feb 2008 10:25:05 -0500
From: John Leslie <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: Hector Santos <[email-masked]>, Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-07
Sender: [email-masked]

John C Klensin <[email-masked]> wrote:
> --On Thursday, 21 February, 2008 16:20 -0500 Hector Santos
> <[email-masked]> wrote:
> 
>> Unless I am missing something, it isn't just better to say
>> something to the effect?
>> 
>>     "SMTP DNS Administration MUST|SHOULD NOT include CNAME
>>      resource records when creating email domain MX records
>>      for the SMTP server setup."

   John K doesn't seem to have responded to this particular text.

>> I mean,  I don't think it is reasonable to discourage DNS
>> CLIENT software to ignore the very good possibility that a
>> query might historically return a CNAME which needs to resolve
>> as well for the final expansion.
> 
> To the extent to which we consider such CNAMEs to be a problem,
> the _only_ way to get rid of them is precisely to encourage
> client software to complain. If we don't care, this should be a
> SHOULD at most (and a SHOULD here will be taken as license to
> use CNAMEs there).

   Personally, I think this battle is long since lost; however...

> The audience for this document is not, for better or worse,
> domain administrators.

   ... which is why, IMHO, we need to be especially clear if we
wish to restrict what they do. They _may_ read a paragraph or
two when somebody points out "the error of their ways" -- but
they will _not_ read for context.

   The text in question, IMHO, is not sufficiently clear:
" 
" The result of an MX lookup MUST NOT be a CNAME.

could mean a number of things -- one of the most logical being
that the SMTP client software is _obligated_ to resolve any
CNAME it encounters. (The DNS administrators reading this text
would know there are any number of middlemen between what they
configure and what the system code called by client software
may return.)

> At some level, we don't care what you do that goes beyond the
> bounds of the standard to deal with a case that the standard
> prohibits.  If you come to the conclusion that you should handle
> these things, then you should do it -- you are still conforming
> wrt the specified behavior.  The only thing that would make you
> non-conforming would be an explicit statement that says that you
> MUST reject such a thing.  And it rather carefully doesn't say
> that.

   ... but it could be read to mean you MUST NOT reject such a
thing...

> And, if you think that concept needs to be explained better in
> the text, please suggest words.

   If we're trying in any way to control what DNS administrators
do, we should have text like:
" 
" DNS administrators MUST NOT configure DNS servers for zones
" they control to return a CNAME for an MX lookup.

   If we're trying to control what (some) SMTP client software
does, we should have text like:
" 
" SMTP client software SHOULD return an error if it receives a
CNAME RR in response to an MX lookup.

... or, of course, "MAY return an error..." -- I have no horse
in this race. We could include both, or neither, for all I care.
I just think the current text is not helpful.

--
John Leslie <[email-masked]>

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: draft-klensin-rfc2821bis-07
Date:  Fri, 22 Feb 2008 09:08:23 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 50
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

John C Klensin wrote:

> " and supplemental...client" has been deleted

Good catch.

> If we want to revisit this subject

No, EHLO syntax allowing clients to explain why=20
they use a domain literal likely can't help. =20

A note that the domain literal should match the
sending IP could make sense, but I guess that is
obvious, this scenario doesn't need any MUSTard.

> The text has been tentatively changed to

>| If the FOR clause appears, it MUST contain exactly one
>| <path> entry, even when multiple RCPT commands have been
>| given.  Multiple <path>s raise some security issues and
>| have been deprecated, see Section 7.2.

+1

> That conclusion was, again, that (i) FOR clauses
> with multiple paths had not been widely implemented
> and supported since provision was made for them in
> 2821 and (ii) the security issues associated with
> that information outweighed the debugging advantages.

(iii) Parsing <Additional-Registered-Clauses> after
<For> could be messy if multiple paths are permitted.

This new feature is important.  Changing <For> again
could also confuse <uFor> in UTF8SMTP for EAI, please
keep the -07 syntax.

>> This also is a significant change from RFC 2821 not
>> expected in this update.

Significant, and noted in appendix G.7 for -05.  After
discussions on the SMTP list.  The old solution with
huge amounts of prose boiling down to DON'T was worse.

It is also related to BCP 72 6.1.1.5 and 6.1.1.2.  In
-07 the RFC 3552 reference went AWOL, please add it
again as justification for this *significant* change,
and more important *intentional* change.

 Frank

Date: Thu, 21 Feb 2008 17:59:44 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-07
Sender: [email-masked]

Thanks for your insights.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

John C Klensin wrote:

>> I mean,  I don't think it is reasonable to discourage DNS
>> CLIENT software to ignore the very good possibility that a
>> query might historically return a CNAME which needs to resolve
>> as well for the final expansion.
> 
> To the extent to which we consider such CNAMEs to be a problem,
> the _only_ way to get rid of them is precisely to encourage
> client software to complain.  If we don't care, this should be a
> SHOULD at most (and a SHOULD here will be taken as license to
> use CNAMEs there).  The audience for this document is not, for
> better or worse, domain administrators.

>> In other words, am I suppose to remove the logic from my DNS
>> client based on the idea that they should never be a CNAME in
>> a MX query? Sure, its the other guys fault, but I don't think
>> any implication to abort, abend or stop a transaction because
>> of it is reasonable.
> 
> At some level, we don't care what you do that goes beyond the
> bounds of the standard to deal with a case that the standard
> prohibits.  If you come to the conclusion that you should handle
> these things, then you should do it -- you are still conforming
> wrt the specified behavior.  The only thing that would make you
> non-conforming would be an explicit statement that says that you
> MUST reject such a thing.  And it rather carefully doesn't say
> that.
>> To paraphrase Postel's Law:
>>
>>     Be conservative with what you do (define proper MX record
>> setup),
>>     be liberal in what you receiver. (CNAME might appear in MX
>>     expansions)
> 
> Sigh.  Without going into a long rant, if this type of
> interpretation of the robustness principle were widely applied,
> we wouldn't have an interoperable email fabric today because
> various extensions and other changes would have been impossible
> and everything would have long ago descended to some least
> common denominator (or below it).  The current text precisely
> permits you to apply the intent of that principle because it
> doesn't prohibit you from accepting and interpreting something
> that is non-conforming to the standard.  It just doesn't require
> you to do that, which would effectively allow CNAME in all cases.
> 
>> My apology if I missed something here.  I certainly don't wish
>> to waste anyone's time.
> 
> No, I think your question is quite reasonable.  But this is very
> much one of the reasons why neither 2821bis nor its predecessors
> make unqualified references to 2119.
> 
> And, if you think that concept needs to be explained better in
> the text, please suggest words.
> 
>     john
> 
> 
> 
> 
> 

Date: Thu, 21 Feb 2008 16:59:28 -0500
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-07
Sender: [email-masked]

--On Thursday, 21 February, 2008 16:20 -0500 Hector Santos
<[email-masked]> wrote:

> Unless I am missing something, it isn't just better to say
> something to the effect?
> 
>     "SMTP DNS Administration MUST|SHOULD NOT include CNAME
> resource
>      records when creating email domain MX records for the SMTP
>      server setup."
> 
> I mean,  I don't think it is reasonable to discourage DNS
> CLIENT software to ignore the very good possibility that a
> query might historically return a CNAME which needs to resolve
> as well for the final expansion.

To the extent to which we consider such CNAMEs to be a problem,
the _only_ way to get rid of them is precisely to encourage
client software to complain.  If we don't care, this should be a
SHOULD at most (and a SHOULD here will be taken as license to
use CNAMEs there).  The audience for this document is not, for
better or worse, domain administrators.

> In other words, am I suppose to remove the logic from my DNS
> client based on the idea that they should never be a CNAME in
> a MX query? Sure, its the other guys fault, but I don't think
> any implication to abort, abend or stop a transaction because
> of it is reasonable.

At some level, we don't care what you do that goes beyond the
bounds of the standard to deal with a case that the standard
prohibits.  If you come to the conclusion that you should handle
these things, then you should do it -- you are still conforming
wrt the specified behavior.  The only thing that would make you
non-conforming would be an explicit statement that says that you
MUST reject such a thing.  And it rather carefully doesn't say
that.

> To paraphrase Postel's Law:
> 
>     Be conservative with what you do (define proper MX record
> setup),
>     be liberal in what you receiver. (CNAME might appear in MX
>     expansions)

Sigh.  Without going into a long rant, if this type of
interpretation of the robustness principle were widely applied,
we wouldn't have an interoperable email fabric today because
various extensions and other changes would have been impossible
and everything would have long ago descended to some least
common denominator (or below it).  The current text precisely
permits you to apply the intent of that principle because it
doesn't prohibit you from accepting and interpreting something
that is non-conforming to the standard.  It just doesn't require
you to do that, which would effectively allow CNAME in all cases.

> My apology if I missed something here.  I certainly don't wish
> to waste anyone's time.

No, I think your question is quite reasonable.  But this is very
much one of the reasons why neither 2821bis nor its predecessors
make unqualified references to 2119.

And, if you think that concept needs to be explained better in
the text, please suggest words.

    john

Date: Thu, 21 Feb 2008 16:20:50 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-07
Sender: [email-masked]

Unless I am missing something, it isn't just better to say something to 
the effect?

    "SMTP DNS Administration MUST|SHOULD NOT include CNAME resource
     records when creating email domain MX records for the SMTP
     server setup."

I mean,  I don't think it is reasonable to discourage DNS CLIENT 
software to ignore the very good possibility that a query might 
historically return a CNAME which needs to resolve as well for the final 
expansion.

In other words, am I suppose to remove the logic from my DNS client 
based on the idea that they should never be a CNAME in a MX query? 
Sure, its the other guys fault, but I don't think any implication to 
abort, abend or stop a transaction because of it is reasonable.

To paraphrase Postel's Law:

    Be conservative with what you do (define proper MX record setup),
    be liberal in what you receiver. (CNAME might appear in MX
    expansions)

My apology if I missed something here.  I certainly don't wish to waste 
anyone's time.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

John C Klensin wrote:
> This (reparagraphing) change has been tentatively made to -08.
> I can also change the final sentence to read "The DNS DATA field
> associated with the lookup of an MX record must not contain a
> domain that, in turn, is associated with a CNAME record" or
> something to that general effect if people are convinced it
> would be more clear rather than more confusing.
> 
>      john
> 
> 
> --On Thursday, 21 February, 2008 11:16 -0500 Tony Hansen
> <[email-masked]> wrote:
> 
>> Alessandro Vesely wrote:
>>  >
>>  > Derek J. Balling wrote:
>>  >> I know this may be a dead horse, but...
>>  >> "The result of an MX lookup MUST NOT be a CNAME."
>>  >> Can this *please* be slightly reworded? "The RR value of
>> an MX
>>  >> lookup..." perhaps?
>>  >
>>> It is the compound "MX lookup" that generates ambiguity, as
>>> it is used to indicate the initial lookup of the domain name.
>>> Actually, that's what the whole first paragraph in 5.1 is
>>> talking about. If it were considered a minor change, I'd
>>> propose moving that sentence to the end of the next
>>> paragraph, where its rationale can be grasped more easily.
>> <techie hat on>
>>
>> I'm thinking that the problem really lies in the paragraph
>> being so long and covering multiple steps. If it were split
>> apart like this, I think it would be more obvious where each
>> statement applies in the flow:
>>
>>     Once an SMTP client lexically identifies a domain to which
>> mail will
>>     be delivered for processing (as described in sections
>> Section 2.3.5
>>     and Section 3.6), a DNS lookup MUST be performed to
>> resolve the
>>     domain name (RFC1035 [6]).  The names are expected to be
>> fully-
>>     qualified domain names (FQDNs): mechanisms for inferring
>> FQDNs from
>>     partial names or local aliases are outside of this
>> specification.
>>     Due to a history of problems, SMTP servers used for initial
>>     submission of messages SHOULD NOT make such inferences
>> (Message
>>     Submission Servers [41] have somewhat more flexibility) and
>>     intermediate (relay) SMTP servers MUST NOT make them.
>>
>>     The lookup
>>     first attempts to locate an MX record associated with the
>> name.  If a
>>     CNAME record is found instead, the resulting name is
>> processed as if
>>     it were the initial name.  If no MX records are found, but
>> an address
>>     RR (i.e., either an IPv4 A RR or an IPv6 AAAA RR, or their
>>     successors) is found, the address RR is treated as if it
>> was
>>     associated with an implicit MX RR, with a preference of 0,
>> pointing
>>     to that host.
>>
>>     If one or more MX RRs are found for a given name, SMTP
>>     systems MUST NOT utilize any address RRs associated with
>> that name
>>     unless they are located using the MX RRs; the "implicit
>> MX" rule
>>     above applies only if there are no MX records present.  If
>> MX records
>>     are present, but none of them are usable, this situation
>> MUST be
>>     reported as an error.  The result of an MX lookup MUST NOT
>> be a
>>     CNAME.
>>
>> </techie hat off>
>>
>> 	Tony Hansen
>> 	[email-masked]
>>
> 
> 
> 
> 
> 
> 

Date: Thu, 21 Feb 2008 15:58:33 -0500
From: John C Klensin <[email-masked]>
To: "Alfred H?nes" <[email-masked]>
cc: [email-masked]
Subject: Re: draft-klensin-rfc2821bis-07
Sender: [email-masked]

--On Thursday, 21 February, 2008 01:37 +0100 "Alfred H?nes"
<[email-masked]> wrote:

> Hello,
> just found the updated version, draft-klensin-rfc2821bis-07
> and roughly checked what happened with my comments.
> 
> I clearly accept that editorials are going to be deferred to
> the RFC Editor, although this tends to increase their work
> load. I hope that the messages pointing out the editorial
> details are getting forwarded along with the draft to ensure
> they must not be dectected again.
> 
> Yet, there remain two serious inconsistencies I had pointed
> out.
> 
> 
> (A)  ----  'supplemental information' in EHLO  ----
> 
> Section 4.1.1.1, 2nd para, says:
> 
>    RFC2821, and some earlier informal practices, encouraged
> following    the literal by information that would help to
> identify the client    system.  That convention was not widely
> supported and many SMTP |  servers considered it an error.  In
> the interest of interoperability, |  it is probably wise for
> servers to be prepared for this string to |  occur, but SMTP
> clients SHOULD NOT send it.
>                            ^^^^^^^^^^^^^^^^^^
> 
> Contrary to that recommendation, Section 4.1.4, 5th para says:
> 
>    The SMTP client MUST, if possible, ensure that the domain
> parameter    to the EHLO command is a primary host name as
> specified for this |  command in Section 2.3.5.  If this is
> not possible (e.g., when the |  client's address is
> dynamically assigned and the client does not have |  an
> obvious name), an address literal SHOULD be substituted for the
> |  domain name and supplemental information provided that will
> assist in |  identifying the client.

Sorry.  The first pointer I got to this was sufficiently
imprecise that I couldn't find the problematic text and hence
assumed it had been removed already.   " and
supplemental...client" has been deleted from the statement above
in -08.

>...
> To resolve this issue, one of the two paragraphs needs to be
> changed. Perhaps the least invasive change might be to modify
> the 4.1.4 para as follows:
> 
>    The SMTP client MUST, if possible, ensure that the domain
> parameter    to the EHLO command is a primary host name as
> specified for this    command in Section 2.3.5.  If this is
> not possible (e.g., when the    client's address is
> dynamically assigned and the client does not have    an
> obvious name), an address literal SHOULD be substituted for the
> |  domain name, and supplemental information MAY be provided
> that will               ^                              ^^^^^^^
>    assist in identifying the client.
> 
> This results in the general SHOULD NOT accompanied by the
> specific MAY under well described exceptional conditions.

The conclusion from list discussion was that this should be a
MUST NOT, partially because, apparently, many implementations
would reject such a string.  So the change above has been made.
If we want to revisit this subject, I think we are headed back
into another Last Call (but that is just my opinion).

 
> (B)  ----  multiple recipients and tracing  ----
> 
> Section 2.3.6 introduces the buffer model.
> 
>   [ BTW: the "and" in the last line there *is*
> confusing/distorting     and should better be replaced by
> "or"! ]

"and" is actually important since, e.g., it would be common to
discard the buffer associated with a message state (e.g., MAIL
... DATA) and revert to the session state (which might contain
client or server authentication, etc.).  That paragraph could be
rewritten completely but I'm reluctant to do anything that would
force another Last Call at this point and that change, IMO,
would.  Tony may feel differently, of course.

> Section 3.3 says:

The decision to restrict FOR to a single argument was made very
explicitly on the list after a fairly long discussion, including
a discussion about the information loss if multiple addresses
were not listed in response to multiple RCPT commands.  That
decision, and the associated syntax restriction, are consistent
with RFC 821 which has

  <stamp> ::= <from-domain> <by-domain> <opt-info> ";"
              <daytime> 
  <opt-info> ::= [<via>] [<with>] [<id>] [<for>]
  <for> ::= "FOR" <SP> <path> <SP>

that syntax permitted neither multiple FOR clauses nor multiple
paths in a given FOR clause.

Please see if the proposed corrections below adequately realign
other text with that decision.  I don't see any way to change
the decision itself unless you can convince Tony to ask Lisa to
reopen the Last Call.

>    The second step in the procedure is the RCPT command.  This
> step of    the procedure can be repeated any number of times.
>                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>    ...
>                                  [...].  If accepted, the SMTP
> server    returns a 250 OK reply and stores the forward-path.
> [..]                               ^^^^^^^^^^^^^^^^^^^^^^^
> 
>    ...
> 
>    The third step in the procedure is the DATA command (or some
>    alternative specified in a service extension).
> 
>    ...
> 
>    The end of mail data indicator also confirms the mail
> transaction and    tells the SMTP server to now process the
> stored recipients and mail    data.  [...]
>                                         ^^^^^^^^^^^^^^^^^^^^^
> 
> This means that the mail server has taken responsibility for
> delivery of the message to *all* recipients from the accepted
> RCPT TO commands.

Yes, that is correct.   This, so far, has nothing to do with
what information appears in trace header fields.

> That's what the trace header field added for this mail
> transport 'hop' is presumed to describe precisely.

In making the decision to restrict "For" to one address (and to
revert the specification of this field from 2821 to 821), the
list concluded that this presumption did not exist.

> Therefore, in Section 4.4, the 3rd bullet correctly states,
> regarding the "Received" trace header field:
> 
>    o  The FOR clause MAY contain a list of <path> entries when
> multiple       RCPT commands have been given.  This may raise
> some security       issues and is usually not desirable; see
> Section 7.2.

This should have been corrected when the decision was
implemented to restrict FOR to one <path>.   The text has been
tentatively changed to

		If the FOR clause appears, it MUST contain exactly one
		<path> entry, even when multiple RCPT commands have been
		given.  Multiple <path>s raise some security issues and
		have been deprecated, see Section 7.2.

Alternate suggestions for text would be welcome as long as they
are consistent with the decision to have only one FOR <path>
clause.

> As can be seen, consistent with due diligence, all recipients
> should be listed in the FOR clause (unless it is suppresssed
> for policy reasons); otherwise, debugging of complicated
> forwarding problems would become impossible.

That is not the conclusion reached on the list.  That conclusion
was, again, that (i) FOR clauses with multiple paths had not
been widely implemented and supported since provision was made
for them in 2821 and (ii) the security issues associated with
that information outweighed the debugging advantages.

> The MAY above cannot be followed unless the syntax allows to
> do so. Nonetheless, the ABNF at the very end of Section 4.4
> says:
> 
>    For            = CFWS "FOR" FWS ( Path / Mailbox )
> 
> This syntax only allows for a *single* RCPT entry.
> 
> This is a serious inconsistency with all other related
> portions of the memo, as exemplified above.  This also is a
> significant change from RFC 2821 not expected in this update.
> Also, the current draft does not contain any provisions for
> the receiving MTA to select from multiple RCPTs for entry of
> only a single recipient address into the "Received" trace
> header field it has to add.
> 
> Therefore, this ABNF rule must be in error.
> 
> So please change the ABNF rule back to what RFC 2821 says:
> 
>    For            = CFWS "FOR" 1*( FWS ( Path / Mailbox ))

Again, see above.

best regards,
   john

Date: Thu, 21 Feb 2008 15:19:41 -0500
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-07
Sender: [email-masked]

This (reparagraphing) change has been tentatively made to -08.
I can also change the final sentence to read "The DNS DATA field
associated with the lookup of an MX record must not contain a
domain that, in turn, is associated with a CNAME record" or
something to that general effect if people are convinced it
would be more clear rather than more confusing.

     john

--On Thursday, 21 February, 2008 11:16 -0500 Tony Hansen
<[email-masked]> wrote:

> 
> Alessandro Vesely wrote:
>  >
>  > Derek J. Balling wrote:
>  >> I know this may be a dead horse, but...
>  >> "The result of an MX lookup MUST NOT be a CNAME."
>  >> Can this *please* be slightly reworded? "The RR value of
> an MX
>  >> lookup..." perhaps?
>  >
>> It is the compound "MX lookup" that generates ambiguity, as
>> it is used to indicate the initial lookup of the domain name.
>> Actually, that's what the whole first paragraph in 5.1 is
>> talking about. If it were considered a minor change, I'd
>> propose moving that sentence to the end of the next
>> paragraph, where its rationale can be grasped more easily.
> 
> <techie hat on>
> 
> I'm thinking that the problem really lies in the paragraph
> being so long and covering multiple steps. If it were split
> apart like this, I think it would be more obvious where each
> statement applies in the flow:
> 
>     Once an SMTP client lexically identifies a domain to which
> mail will
>     be delivered for processing (as described in sections
> Section 2.3.5
>     and Section 3.6), a DNS lookup MUST be performed to
> resolve the
>     domain name (RFC1035 [6]).  The names are expected to be
> fully-
>     qualified domain names (FQDNs): mechanisms for inferring
> FQDNs from
>     partial names or local aliases are outside of this
> specification.
>     Due to a history of problems, SMTP servers used for initial
>     submission of messages SHOULD NOT make such inferences
> (Message
>     Submission Servers [41] have somewhat more flexibility) and
>     intermediate (relay) SMTP servers MUST NOT make them.
> 
>     The lookup
>     first attempts to locate an MX record associated with the
> name.  If a
>     CNAME record is found instead, the resulting name is
> processed as if
>     it were the initial name.  If no MX records are found, but
> an address
>     RR (i.e., either an IPv4 A RR or an IPv6 AAAA RR, or their
>     successors) is found, the address RR is treated as if it
> was
>     associated with an implicit MX RR, with a preference of 0,
> pointing
>     to that host.
> 
>     If one or more MX RRs are found for a given name, SMTP
>     systems MUST NOT utilize any address RRs associated with
> that name
>     unless they are located using the MX RRs; the "implicit
> MX" rule
>     above applies only if there are no MX records present.  If
> MX records
>     are present, but none of them are usable, this situation
> MUST be
>     reported as an error.  The result of an MX lookup MUST NOT
> be a
>     CNAME.
> 
> </techie hat off>
> 
> 	Tony Hansen
> 	[email-masked]
> 

Date: Thu, 21 Feb 2008 12:55:53 -0500
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "Derek J. Balling" <[email-masked]>
CC: Tony Hansen <[email-masked]>, [email-masked]
Subject: Re: rfc2821bis-07
Sender: [email-masked]

Derek J. Balling wrote:
> 
> On Feb 21, 2008, at 11:16 AM, Tony Hansen wrote:
>>  The result of an MX lookup MUST NOT be a
>>   CNAME.
> 
> I still say that verbiage, wherever it is, will be unnecessarily 
> confusing. If a lookup for "the MX of a domain" yields a CNAME RR, then 
> it could be said that the MX lookup was a CNAME, and that's *not* what 
> is being proscribed against.
> 
> That above sentence needs to be reworded to actually be syntactically 
> correct, IMHO.
> 
> Cheers,
> D
> 

+1, especially since the DNS client performing a MX lookup will resolve 
the CNAME and that that overall request would not result in a CNAME for 
SMTP MX expansions.  It essence, the semantics seem to be more 
appropriately applied as a reminder to DNS clients doing a MX request in 
general.

I actually recall seeing this before but decided it wasn't worth the 
aggravation.  After all, it isn't the current SMTP vendors problem. They 
are doing it right and future SMTP vendors/clients who decide to roll 
their own DNS Client will find out the hard way that a CNAME is part of 
the MX expansion process.  But if they use a DNS client API, these 
library already figured it out. It will be transparent to the SMTP 
client using a DNS client API.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Cc: [email-masked]
From: "Derek J. Balling" <[email-masked]>
To: Tony Hansen <[email-masked]>
Subject: Re: rfc2821bis-07
Date: Thu, 21 Feb 2008 12:10:20 -0500
Sender: [email-masked]

--Apple-Mail-9--179161997
Content-Type: text/plain;
	charset=US-ASCII;
	format=flowed;
	delsp=yes
Content-Transfer-Encoding: 7bit

On Feb 21, 2008, at 11:16 AM, Tony Hansen wrote:
>  The result of an MX lookup MUST NOT be a
>   CNAME.

I still say that verbiage, wherever it is, will be unnecessarily  
confusing. If a lookup for "the MX of a domain" yields a CNAME RR,  
then it could be said that the MX lookup was a CNAME, and that's *not*  
what is being proscribed against.

That above sentence needs to be reworded to actually be syntactically  
correct, IMHO.

Cheers,
D

--Apple-Mail-9--179161997
Content-Disposition: attachment;
	filename=smime.p7s
Content-Type: application/pkcs7-signature;
	name=smime.p7s
Content-Transfer-Encoding: base64

--Apple-Mail-9--179161997--

Date: Thu, 21 Feb 2008 11:16:53 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-07
Sender: [email-masked]

Alessandro Vesely wrote:
 >
 > Derek J. Balling wrote:
 >> I know this may be a dead horse, but...
 >> "The result of an MX lookup MUST NOT be a CNAME."
 >> Can this *please* be slightly reworded? "The RR value of an MX
 >> lookup..." perhaps?
 >
> It is the compound "MX lookup" that generates ambiguity, as it is
> used to indicate the initial lookup of the domain name. Actually,
> that's what the whole first paragraph in 5.1 is talking about. If it
> were considered a minor change, I'd propose moving that sentence to
> the end of the next paragraph, where its rationale can be grasped
> more easily.

<techie hat on>

I'm thinking that the problem really lies in the paragraph being so long 
and covering multiple steps. If it were split apart like this, I think 
it would be more obvious where each statement applies in the flow:

    Once an SMTP client lexically identifies a domain to which mail will
    be delivered for processing (as described in sections Section 2.3.5
    and Section 3.6), a DNS lookup MUST be performed to resolve the
    domain name (RFC1035 [6]).  The names are expected to be fully-
    qualified domain names (FQDNs): mechanisms for inferring FQDNs from
    partial names or local aliases are outside of this specification.
    Due to a history of problems, SMTP servers used for initial
    submission of messages SHOULD NOT make such inferences (Message
    Submission Servers [41] have somewhat more flexibility) and
    intermediate (relay) SMTP servers MUST NOT make them.

    The lookup
    first attempts to locate an MX record associated with the name.  If a
    CNAME record is found instead, the resulting name is processed as if
    it were the initial name.  If no MX records are found, but an address
    RR (i.e., either an IPv4 A RR or an IPv6 AAAA RR, or their
    successors) is found, the address RR is treated as if it was
    associated with an implicit MX RR, with a preference of 0, pointing
    to that host.

    If one or more MX RRs are found for a given name, SMTP
    systems MUST NOT utilize any address RRs associated with that name
    unless they are located using the MX RRs; the "implicit MX" rule
    above applies only if there are no MX records present.  If MX records
    are present, but none of them are usable, this situation MUST be
    reported as an error.  The result of an MX lookup MUST NOT be a
    CNAME.

</techie hat off>

	Tony Hansen
	[email-masked]

Date: Thu, 21 Feb 2008 12:30:36 +0100
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-07
Sender: [email-masked]

Derek J. Balling wrote:
> I know this may be a dead horse, but...
> "The result of an MX lookup MUST NOT be a CNAME."
> Can this *please* be slightly reworded? "The RR value of an MX 
> lookup..." perhaps?

It is the compound "MX lookup" that generates ambiguity, as it is used to 
indicate the initial lookup of the domain name. Actually, that's what the 
whole first paragraph in 5.1 is talking about. If it were considered a 
minor change, I'd propose moving that sentence to the end of the next 
paragraph, where its rationale can be grasped more easily.

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: rfc2821bis-07
Date:  Thu, 21 Feb 2008 07:57:01 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 96
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Tony Hansen wrote:
=20
> Diffs between -06 and -07 can be viewed here.

Thanks.

| ehlo-greet =3D 1*(%d0-9 / %d11-12 / %d14-127)

RFC.klensin-net-utf8 (almost RFC, 8:0:0 of 15)
proposes to avoid control codes.  Allowing NUL
could be problematic (=3D without tests showing
that it's okay).

| ehlo-param  =3D 1*(%d33-127)
|               ; any CHAR excluding <SP> and all
|               ; control characters (US-ASCII 0-31 inclusive)

Just using 1*VCHAR would be clearer.  Defining
that DEL is no control character in a comment
is IMO an unnecessary exercise.

| esmtp-value =3D 1*(%d33-60 / %d62-127)

Similar, s/127/126/ would be clearer if there is
no plausible reason why DEL has to be permitted.

If <esmtp-value> could somehow end up in 2822upd
header fields (example), where DEL isn't allowed,
it could mean trouble.

The prose definition of "control character" in
section 4.1.2 includes DEL, two syntax comments=20
exclude DEL.  Please fix the "127"-syntax for a
consistent story - unless it's some magic where
I just don't see what the DEL is supposed to do.

4.1.1.3:  The postmaster note got an extra-wide
indent, if it's actually a part of the syntax the
semicolons are missing.  [[The RFC editor has no
good chance to catch this, it is in a piece of
RCPT-TO syntax needing manual interventions to
check the ABNF.]]

Domain, sub-domain, Let-dig, and Ldh-str are all
okay, justifying an "updates 3696" (in theory -
in practice I guess all interested parties can
already sing it ;-)

| quoted-pairSMTP  =3D %d92 %d32-126
|                  ; i.e., backslash followed by any ASCII
|                  ; graphic (including itself) or SPace

Note that 2822upd-06 allows HTAB in <quoted-pair>:

| quoted-pair      =3D ("\" (VCHAR / WSP)) / obs-qp

My gut feeling is that this should be in sync, i.e.

| quoted-pairSMTP  =3D "\" ( WSP / VCHAR )

Note that <dcontent> is NOT defined in 2822upd-06:

| General-address-literal  =3D Standardized-tag ":" 1*dcontent

You certainly don't want <obs-dtext>, maybe copy
<dtext> minus <obs-dtext> and says it's <dcontent>:

| dcontent         =3D %d33-90 /        ; Printable US-ASCII
|                    %d94-126         ; excl. "[", "\", "]"

The following clause is no valid ABNF:

| Standardized-tag  ; MUST be specified in a standards-track RFC
|                  ; and registered with IANA

Maybe say  Standardized-Tag =3D 1*ALPHA  or use a prose
definition within angle brackets.  I'm not going to
explain again why I think that the STD 66 definition
for <IPv6address> is better than 2821bis <IPv6-addr>.

| A mail transaction may be aborted by the RSET, a new
| EHLO, or the QUIT command. there may be zero or more
| transactions in a session. =20

s /there/There/

In the section 4.2 ABNF <text> is apparently undefined,
it is not the 2822upd <text>.  Sorry, I missed that in
the Last Call... :-(

In section 4.5.2 please join 251 and 551 in one group.

This was no review or ABNF check, I looked only at the
diff and some known Last Call rat-holes.

 Frank

Date: Wed, 20 Feb 2008 15:41:15 -0500
From: John C Klensin <[email-masked]>
To: SM <[email-masked]>
cc: [email-masked]
Subject: Re: rfc2821bis-07
Sender: [email-masked]

--On Wednesday, 20 February, 2008 12:08 -0800 SM
<[email-masked]> wrote:

> 
> At 07:50 20-02-2008, Tony Hansen wrote:
> 
>> Diffs between -06 and -07 can be viewed here.
>> 
>> <http://tools.ietf.org/rfcdiff?difftype=--hwdiff&url2=http://
>> tools.ietf.org/id/draft-klensin-rfc2821bis-07.txt>
> 
> In Section 3.9.2:
> 
>    "There exist mailing lists that perform additional,
> sometimes
>     extensive, modifications to a message and its envelope.
> Such mailing
>     lists need to be viewed as full MUAs, which accept a
> delivery and
>     post a new message.b"
> 
> The word "body" was cut off.
> 
>     There are mailing lists that perform additional, sometimes
>     extensive, modifications to a message and its envelope.
> Such mailing
>     lists need to be viewed as full MUAs, which accept a
> delivery and
>     post a new message body.

Actually, no.  The "b" was an editing error and it should have
been "...post a new message.".

Fixed in, unfortunately, -08 (this would have otherwise been a
short warning note to the RFC Editor).

thanks,
   john

Date: Wed, 20 Feb 2008 12:08:58 -0800
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: rfc2821bis-07
Sender: [email-masked]

At 07:50 20-02-2008, Tony Hansen wrote:

>Diffs between -06 and -07 can be viewed here.
>
><http://tools.ietf.org/rfcdiff?difftype=--hwdiff&url2=http://tools.ietf.org/id/draft-klensin-rfc2821bis-07.txt>

In Section 3.9.2:

   "There exist mailing lists that perform additional, sometimes
    extensive, modifications to a message and its envelope.  Such mailing
    lists need to be viewed as full MUAs, which accept a delivery and
    post a new message.b"

The word "body" was cut off.

    There are mailing lists that perform additional, sometimes
    extensive, modifications to a message and its envelope.  Such mailing
    lists need to be viewed as full MUAs, which accept a delivery and
    post a new message body.

Regards,
-sm

Date: Wed, 20 Feb 2008 13:28:59 -0500
From: John C Klensin <[email-masked]>
To: [email-masked]
cc: Tony Hansen <[email-masked]>
Subject: Re: rfc2821bis-07 (and -08)
Sender: [email-masked]

Hi.

I've gotten several "whoops, didn't get this to you on time"
proposed changes/corrections to 2821bis-07.

Tony has only had a chance to review one of them, and I'm not
making further changes without his go-ahead.   I also want to
point out that this can go on forever, with people raising new
issues, or almost-new issues, on a continual basis until we all
run out of patience and abandon the 2821bis effort.

However, it is clear that there will be a -08.  I carved out
time to get -07 posted in the hope that we could be finished
with this before Philadelphia.  Given other commitments, we have
probably lost that window.  It is not clear whether -08 will
appear prior to the posting deadline but, even if Tony gets to
me quickly enough and I can get it out, I can pretty much
guarantee that, if there is a need for a -09, the odds of its
appearing much before -08 expires will be extremely low.

    john

Cc: [email-masked]
From: "Derek J. Balling" <[email-masked]>
To: Tony Hansen <[email-masked]>
Subject: Re: rfc2821bis-07
Date: Wed, 20 Feb 2008 12:08:06 -0500
Sender: [email-masked]

--Apple-Mail-18--265696312
Content-Type: text/plain;
	charset=US-ASCII;
	format=flowed;
	delsp=yes
Content-Transfer-Encoding: 7bit

I know this may be a dead horse, but...

"The result of an MX lookup MUST NOT be a CNAME."

Can this *please* be slightly reworded? "The RR value of an MX  
lookup..." perhaps?

One could argue if I do an MX lookup for "host.example.com" and get

host.example.com. IN CNAME otherhost.example.com.

That the result of an lookup was a CNAME, which is *not* what we're  
talking about being verboten.

--Apple-Mail-18--265696312
Content-Disposition: attachment;
	filename=smime.p7s
Content-Type: application/pkcs7-signature;
	name=smime.p7s
Content-Transfer-Encoding: base64

--Apple-Mail-18--265696312--

Date: Wed, 20 Feb 2008 10:50:42 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821bis-07
Sender: [email-masked]

Diffs between -06 and -07 can be viewed here.

<http://tools.ietf.org/rfcdiff?difftype=--hwdiff&url2=http://tools.ietf.org/id/draft-klensin-rfc2821bis-07.txt>

<http://tools.ietf.org/rfcdiff?url2=http://tools.ietf.org/id/draft-klensin-rfc2821bis-07.txt>

	Tony Hansen
	[email-masked]

John C Klensin wrote:
> --On Wednesday, 13 February, 2008 00:26 -0600 Tony Hansen
> <[email-masked]> wrote:
> 
>> <pseudo-chair hat on>
>>
>> I'm going to be posting a bunch of notes shortly, closing out
>> all of the extant issues.
>>
>> John will be posting an updated draft shortly, reflecting
>> these consensus calls.
>>
>> </pseudo-chair hat off>
> 
> Ok.  2821bis-07 is on its way into the posting queue.  This
> responds to and fixes all of the items on Tony's list except as
> noted below.  It also contains a few minor cleanups that are
> noted in a drastically trimmed Section 1.1 (trimming it was one
> of the cleanups) and in the change log (Appendix G.9).
> 
> The changes that have not been made involve some small issues of
> formatting (notably blank lines in odd places such as syntax and
> reference annotations).  They are artifacts of the way that
> xml2rfc is producing the document.  After an extremely brief
> discussion with Tony, I'm leaving them for the RFC Editor.
> 
>     john
> 

Date: Tue, 19 Feb 2008 20:26:15 -0500
From: John C Klensin <[email-masked]>
To: Tony Hansen <[email-masked]>, [email-masked]
cc: Lisa Dusseault <[email-masked]>
Subject: rfc2821bis-07 (was: Re: Closing out the comments)
Sender: [email-masked]

--On Wednesday, 13 February, 2008 00:26 -0600 Tony Hansen
<[email-masked]> wrote:

> <pseudo-chair hat on>
> 
> I'm going to be posting a bunch of notes shortly, closing out
> all of the extant issues.
> 
> John will be posting an updated draft shortly, reflecting
> these consensus calls.
> 
> </pseudo-chair hat off>

Ok.  2821bis-07 is on its way into the posting queue.  This
responds to and fixes all of the items on Tony's list except as
noted below.  It also contains a few minor cleanups that are
noted in a drastically trimmed Section 1.1 (trimming it was one
of the cleanups) and in the change log (Appendix G.9).

The changes that have not been made involve some small issues of
formatting (notably blank lines in odd places such as syntax and
reference annotations).  They are artifacts of the way that
xml2rfc is producing the document.  After an extremely brief
discussion with Tony, I'm leaving them for the RFC Editor.

    john

Date: Sat, 16 Feb 2008 19:34:47 +0100
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: Closing out all other issues in draft-klensin-2821bis
Sender: [email-masked]

John,

>> Section 3.9.2 "List" is different in rfc2821bis-0N (0N>= 03)
>> w.r.t the current rfc2821. The added final note,
>> distinguishing alias expansion and forwarding, would be
>> correct if that subsection were named "Forwarding".
> [...]
>    The forwarding model is tricky, and various
> theories about when one should, and should not, change envelope
> addresses --and how to respond when others do it-- to counter
> spam and other bad behavior make it worse.

As I see that carefully set up mailing lists work quite well, while it is 
rather impossible to configure and maintain alias expansions, I think the 
crux can be easily identified in that _key difference_. I just would have 
liked that point to be clearly visible by every reader, without making any 
technical change, nor altering any formal specification. (As Frank said, 
nothing's wrong if a mailing list has only one member.)

> I believe that a major piece of work is required in this area,
> probably of the variety that requires a working group rather
> than a discussion of a draft that is supposed to reflect tuning
> improvements to work done many years ago that reflected current
> and preferred practice at the time.

I share your belief, that's why I was hoping the new SMTP standard would 
have somehow taken into account existing and future work on that subject.

>   Trying to "patch" 2821bis,
> especially under the constraints of advancing to Draft Standard
> and post-last-call, is just not going to do it, either
> procedurally or technically.

I see. I apologize again for my ignorance about bureaucratic intricacies.

> I would suggest --again, speaking only as an individual
> contributor-- what has been suggested to others [...]

Thank you very much for explaining those points once more
Alessandro

Date: Sat, 16 Feb 2008 10:00:21 -0500
From: John C Klensin <[email-masked]>
To: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: Closing out all other issues in draft-klensin-2821bis
Sender: [email-masked]

--On Saturday, 16 February, 2008 12:36 +0100 Alessandro Vesely
<[email-masked]> wrote:

> 
> Hi,
> 
> Tony Hansen wrote:
>> This message finishes the list of other issues with 
>> draft-klensin-2821bis or bugs that were found during the last
>> call, and  their disposition if consensus was gained.
> 
> I've posted a few messages about section 3.9. I understand I
> should have posted that 4 or 5 years ago, but I only joined
> this list last Jan 16th. Thus, if at all possible, I'd ask if
> some cleanup can be considered before the text is standardized.
> 
> Section 3.9.2 "List" is different in rfc2821bis-0N (0N>= 03)
> w.r.t the current rfc2821. The added final note,
> distinguishing alias expansion and forwarding, would be
> correct if that subsection were named "Forwarding".
> Because the surrounding text specifically describes mailing
> lists, it may instead convey the wrong impression that the
> envelope sender address should be changed in that case _only_.
> (Of course, the envelope sender MAY be changed when privacy or
> policy concerns require it, according to section 4.4.)
> 
> I apologize for my scarce knowledge of standardization
> processes; that notwithstanding, I hope the IETF will release
> a coherent standard.

Alessandro,

Speaking strictly as an individual contributor, we have a bit of
a problem here.   The forwarding model is tricky, and various
theories about when one should, and should not, change envelope
addresses --and how to respond when others do it-- to counter
spam and other bad behavior make it worse.  At least one or two
major email providers offer forwarding services (mail comes in
on one address and goes out to another) from their systems but
reject mail forwarded from others.  

I believe that a major piece of work is required in this area,
probably of the variety that requires a working group rather
than a discussion of a draft that is supposed to reflect tuning
improvements to work done many years ago that reflected current
and preferred practice at the time.   Trying to "patch" 2821bis,
especially under the constraints of advancing to Draft Standard
and post-last-call, is just not going to do it, either
procedurally or technically.

I would suggest --again, speaking only as an individual
contributor-- what has been suggested to others who expressed a
desire for substantive changes in 2821bis.  That is to generate
a separate draft, discuss how it should be progressed with the
Area Directors, and move that work forward with the intent of
moving it into the _next_ generation of descendents of 821.

     john

Date: Sat, 16 Feb 2008 12:36:29 +0100
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: Closing out all other issues in draft-klensin-2821bis
Sender: [email-masked]

Hi,

Tony Hansen wrote:
> This message finishes the list of other issues with 
> draft-klensin-2821bis or bugs that were found during the last call, and 
> their disposition if consensus was gained.

I've posted a few messages about section 3.9. I understand I should have 
posted that 4 or 5 years ago, but I only joined this list last Jan 16th. 
Thus, if at all possible, I'd ask if some cleanup can be considered before 
the text is standardized.

Section 3.9.2 "List" is different in rfc2821bis-0N (0N>= 03) w.r.t the 
current rfc2821. The added final note, distinguishing alias expansion and 
forwarding, would be correct if that subsection were named "Forwarding".
Because the surrounding text specifically describes mailing lists, it may 
instead convey the wrong impression that the envelope sender address 
should be changed in that case _only_. (Of course, the envelope sender MAY 
be changed when privacy or policy concerns require it, according to 
section 4.4.)

I apologize for my scarce knowledge of standardization processes; that 
notwithstanding, I hope the IETF will release a coherent standard.

Ale

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Just excude "obs" ? [was: Closing out all other issues in draft-klensin-2821bis}
Date:  Sat, 16 Feb 2008 12:12:40 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 11
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

> Various 2822upd details were modified in the last six weeks,
> e.g., 2821bis now needs its own <dcontent> excluding "obs".

Maybe it's more straight forward if all 2822upd imports just
exclude "obs" constructs (same idea as in RFC.usefor-usefor).

With that strategy some of the proposed new <xyzSMTP> terms
might be unnecessary.  However this needs a MUST somewhere,
a good place could be where CTL is already noted as MUST NOT.

 Frank

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Closing out all other issues in draft-klensin-2821bis
Date:  Sat, 16 Feb 2008 11:47:54 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 16
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Tony Hansen wrote:

> Everyone, please double check his edits.

Please produce a complete ABNF with all STD 68 and 2822upd
imports for checking as I've done it for the Last Call in

http://hmdmhdfmhdjmzdtjmzdtzktdkztdjz.googlepages.com/2821bis.ABNF.2

Various 2822upd details were modified in the last six weeks,
e.g., 2821bis now needs its own <dcontent> excluding "obs".

I also had a .../2821bis.ABNF.1 and meanwhile forgot what
its purpose was, likely ABNF.2 fixed minor typos in ABNF.1

 Frank

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Closing out all other issues in draft-klensin-2821bis
Date:  Sat, 16 Feb 2008 11:00:53 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 19
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Tony Hansen wrote:

[...]
>      quoted-pairSMTP      =3D %d92 %d32-126
[...]
=20
> None of the other ABNF issues raised gained any consensus.

There were a bunch of 127 in the ABNF which should be 126
in the style shown above.  A clearer notation could be:

|      quoted-pairSMTP      =3D "\" ( SP / VCHAR )

Note that this excludes HTAB, are you sure about this ?

Was there an interoperability report indicating that DEL
(127) actually "works" for some definition of "works" ?

 Frank

Date: Sat, 16 Feb 2008 00:51:16 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Closing out all other issues in draft-klensin-2821bis
Sender: [email-masked]

<pseudo-chair hat on>

This message finishes the list of other issues with 
draft-klensin-2821bis or bugs that were found during the last call, and 
their disposition if consensus was gained.

John will be posting a diff shortly with these changes and any other 
editorial tweaks he made as a result of the last call comments. 
Everyone, please double check his edits.

The document will then go back to the IESG for final determination of if 
it is ready for publication.

</pseudo-chair hat off>

	Tony Hansen
	[email-masked]

7) ABNF discussions. The following items were deemed bugs or gained 
consensus:

a) Change the definition of Quoted-string in 2821bis as follows:

     Quoted-string  = DQUOTE *qcontentSMTP DQUOTE

     QcontentSMTP   = qtextSMTP / quoted-pairSMTP

     quoted-pairSMTP      = %d92 %d32-126
                      ; i.e., backslash followed by any
                      ; ASCII graphic (including itself) or
                      ; SPace

     qtextSMTP            = %d32-33 / %d35-91 / %d93-126
                      ; i.e., within a quoted string, any
                      ; ASCII graphic or space is permitted
                      ; without blackslash-quoting except the
                      ; backslash itself.

b) Add the following sentences to the last paragraph of Section 2 (the 
paragraph that starts "The metalinguistic notation used..."):

     The reader is cautioned that the grammar expressed in
     the metalanguage is not comprehensive.  There are many
     instances in which provisions in the text constrain or
     otherwise modify the syntax or semantics implied by the
     grammar.

c) The token address-literal is spelled Address-literal in two places.

d) The token Domain is spelled domain in two places

e) A-d-l and Domain simplified:

    <   A-d-l          = At-domain *( "," A-d-l )
    >   A-d-l          = At-domain *( "," At-domain )

    <   Domain         = ( sub-domain 1*("." sub-domain) ) / sub-domain
    >   Domain         = sub-domain *("." sub-domain)

f) section 4.1.1.1: blank lines inside grammar. Remove extra blank lines 
after first line of each grammar productions for ehlo-ok-resp, etc. that 
were introduced in the XML conversion.

None of the other ABNF issues raised gained any consensus.

8) IPv4-only and IPv6-only walled gardens discussion.

Insufficient consensus to change anything here.

9) Removing trace headers text.

No change on this issue.

10) Strengthen text on keeping trace headers intact:

    change
     An Internet mail program MUST NOT change a Received: line ...
    to
     An Internet mail program MUST NOT change or delete a Received:
     line ...

11) list processing (section 3.9.*)

An extensive rewrite to section 3.9.* was suggested but garnered 
insufficient support. However, there was sufficient support for text 
similar to the following.

Add to the end of section 3.9.2:

    When a list constrains its processing to the very limited set of
    modifications and actions described here, it is attempting to emulate
    an MTA; such lists can be treated as a continuation in email transit.

Add a new paragraph to the end of section 3.9.2:

    There exist mailing lists that perform additional, sometimes
    extensive, modifications to a message and its envelope. Such mailing
    lists need to be viewed as full MUAs, which accept a delivery and
    post a new message.

12) change references from 2822 to 2822upd

Update reference [9] to point to 2822upd. This will then be fixed by the 
RFC editor to be the right thing upon publication.

13) add a glossary

No support expressed for this.

14) section 1.4 listing of obsoleted/updated documents lists 821 twice 
and omits 1869

15) section 4.1.1.3: missing information on buffer manipulation. (All 
other commands describe buffer manipulations.)

add somewhere to section 4.1.1.3
    This command appends its forward-path argument to the forward-path
    buffer; it does not change the reverse-path buffer nor the mail data
    buffer.

16) section 4.1.4: There is a mismatch between 3rd & 4th paragraphs with 
respect to EHLO clearing the buffers or not:

17) sections 4.1.1.10 and 4.1.4: There is a mismatch in these two 
sections regarding 1) the use of QUIT at any time, and 2) the fact that 
QUIT can also abort a transaction.

18) section 4.3.2 lists of command responses

	MAIL FROM is missing 455, 555
	RCPT TO is missing 455, 555
	RCPT TO has an extra 550 listed

19) section 4.5.3.2.6 discusses both client side and server side 
timeouts and should be split.

20) appendix D.3 Step 2: space was lost in XML formatting

       C: Received: from bar.com by foo.com ; Thu, 21 May 1998
       C: 05:33:29 -0700

       C: Received: from bar.com by foo.com ; Thu, 21 May 1998
       C:    05:33:29 -0700

Date: Thu, 14 Feb 2008 13:14:24 -0600
From: Tony Hansen <[email-masked]>
To: Mark Martinec <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2821 New Issue: Section 4.3.2, the "anywhere" codes, and related topics
Sender: [email-masked]

Thanks for the correction.

Mark Martinec wrote:
> Tony Hansen wrote:
> 
>> QUIT
>>     change
>>      This command specifies that the receiver MUST send an OK reply,
>>     to
>>      This command specifies that the receiver MUST send a "250 OK"
>>      reply,
> 
> The QUIT was supposed to be answered with a 221 reply
> according to RFC 2821.
> 
>   Mark
> 

From: Mark Martinec <[email-masked]>
Organization: J. Stefan Institute
To: [email-masked]
Subject: Re: rfc2821 New Issue: Section 4.3.2, the "anywhere" codes, and related topics
Date: Thu, 14 Feb 2008 16:57:09 +0100
Sender: [email-masked]

Tony Hansen wrote:

> QUIT
>     change
>      This command specifies that the receiver MUST send an OK reply,
>     to
>      This command specifies that the receiver MUST send a "250 OK"
>      reply,

The QUIT was supposed to be answered with a 221 reply
according to RFC 2821.

  Mark

Date: Wed, 13 Feb 2008 00:49:06 -0600
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: Last Call: draft-klensin-rfc2821bis         (Simple Mail Transfer Protocol) to Draft Standard
Sender: [email-masked]

<pseudo-chair hat on>

This bug needs to be fixed.

1) 455, 555 missing

Add this section

    4.1.1.11 Mail-parameter and Rcpt-parameter Error Responses

    If the server SMTP does not recognize or cannot implement one or more
    of the parameters associated with a particular MAIL FROM or RCPT TO
    command, it will return code 555.

    If for some reason the server is temporarily unable to accommodate
    one or more of the parameters associated with a MAIL FROM or RCPT TO
    command, and if the definition of the specific parameter does not
    mandate the use of another code, it should return code 455.

    Errors specific to particular parameters and their values will be
    specified in the parameter's defining RFC.

In sections 4.2.2 and 4.2.3

    455 Server unable to accommodate parameters
    555 MAIL FROM/RCPT TO parameters not recognized or not implemented

</pseudo-chair hat off>

     Tony Hansen
     [email-masked]

[email-masked] wrote:
> 
> 
>> John,
>> I have one more question:
> 
>> FRC 1869 defines 2 new SMTP reply codes in section 6.1:
> 
>>    If the server SMTP does not recognize or cannot implement one or more
>>    of the parameters associated with a particular MAIL FROM or RCPT TO
>>    command, it will return code 555.
> 
>>    If for some reason the server is temporarily unable to accomodate one
>>    or more of the parameters associated with a MAIL FROM or RCPT TO
>>    command, and if the definition of the specific parameter does not
>>    mandate the use of another code, it should return code 455.
> 
>> They are not in RFC 2821 which obsoleted RFC 1869. Bug or feature?
> 
> Missed this, sorry. I'd have to call it a bug. IMO these need to be added
> to 2821bis.
> 
>                 Ned
> 

Date: Wed, 13 Feb 2008 00:46:56 -0600
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: MX to CNAME and (mis)interptretation of 2821
Sender: [email-masked]

<pseudo-chair hat on>

Consensus was reached on this issue:

1) MX => CNAME

In section 5.1, add to the end of the 1st paragraph:
     The result of an MX lookup MUST NOT be a CNAME.

</pseudo-chair hat off>

     Tony Hansen
     [email-masked]

Trevor Paquette wrote:
> Sounds to me like this is a great opportunity to get this resolved once
> and for all. If the intent of 2821 was never to override the provisions
> of 2181, then Mr. Klensin could state this specifically within the RFC.
> 
> I believe that the confusion lies around the CNAME provision dealing the
> left hand side of the MX record, not the right hand side. This is where
> people get confused. The following paragraph can be interpreted to allow
> CNAMES as the result of an MX lookup; this paragraph may need to be
> clarified:
> 
> 5.1.  Locating the Target Host
>  ..... The lookup first attempts to locate an MX record associated with
> the name.  If a CNAME record is found instead, the resulting name is
> processed as if it were the initial name.  If no MX records are found,
> but an address RR (i.e., either an IPv4 A RR or an IPv6 AAAA RR, or
> their successors) is found, the address RR is treated as if it was
> associated with an implicit MX RR, with a preference of 0, pointing to
> that host....  
> 
> That is the section that is confusing people. You can read it to say
> "When you do an MX record lookup, and the result is a CNAME, then ...
> blah blah blah.." To the folks that I've talked to, they say that this
> one statement allows an MX record to point to a CNAME. Again, if is this
> not the case nor the intention, I'd love to see this latest revision
> updated to categorically state that.
> 
> 
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> On Behalf Of Frank Ellermann
> Sent: Wednesday, December 12, 2007 4:13 PM
> To: [email-masked]
> Subject: Re: MX to CNAME and (mis)interptretation of 2821
> 
> 
> Trevor Paquette wrote:
> 
>> Would it also be possible to state directly within the next
>> revision of the RFC where or not this RFC overrides the 
>> provision 10.3 of RFC 2181.
> 
> The Last Call for 2821bis ends 2007-12-24:
> 
> <http://tools.ietf.org/html/draft-klensin-2821bis> is 2821bis,
> and <http://permalink.gmane.org/gmane.ietf.announce/22584> is
> a copy of the running Last Call.
> 
>> I don't want to leave things left to misinterpretation.
> 
> I've seen at least one flamewar about this on another list. 
> 
>  Frank
> 
> 

Date: Wed, 13 Feb 2008 00:43:23 -0600
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: Last Call: draft-klensin-rfc2821bis (Simple Mail Transfer Protocol) to Draft Standard (2)
Sender: [email-masked]

<pseudo-chair hat on>

Consensus has been reached on this issue.

1) RFC 2119-compliance

Replace section 2.3 text with the following:

       The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
       NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and
       "OPTIONAL" in this document are to be interpreted as described in
       RFC 2119.

and add a sentence such as the following:

	As each of these terms were intentionally and carefully chosen
	to improve the interoperability of email, the use of these terms
	is to be treated as NORMATIVE in all cases.

</pseudo-chair hat off>

     Tony Hansen
     [email-masked]

Frank Ellermann wrote:
> The IESG wrote:
> 
>> <draft-klensin-rfc2821bis-06.txt> as a Draft Standard
>  
>> The IESG plans to make a decision in the next few weeks,
>> and solicits final comments on this action.
> 
> I-D.klensin-2821bis 2.3 contains a list of defined terms
> which are *apparently* copied verbatim from RFC 2119 
> without stating the source.  This is hostile to readers
> trying to figure out what the difference might be, and
> if bypassing RFC 2119 chapter 6 really is the goal a
> less irritating solution should be possible, compare
> <http://article.gmane.org/gmane.ietf.smtp/6260/>
> 
>  Frank
> 
> 
> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ietf

Date: Wed, 13 Feb 2008 00:30:10 -0600
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: rfc2821 New Issue: Section 4.3.2, the "anywhere" codes, and related topics
Sender: [email-masked]

<pseudo-chair hat on>

Sufficient support was shown for these suggestions.

1) the 500 code

modify the description of 500 by adding to the end...

     Similarly, producing a "command line too long" message
     for a command line shorter than 512 characters would
     violate the provisions of Section 4.5.3.1.4.

2) the anywhere codes

insert text in the command descriptions for the commands that have only 
a single positive reply (EHLO/HELO, DATA, RSET, NOOP, and QUIT) that 
explicitly identifies the code that is expected for a normal/successful 
reply

EHLO/HELO already describes 250

DATA
    change
     In sending a positive completion reply to the end of data
     indication,
    to
     In sending a positive "250 OK" completion reply to the end of
     data indication,

RSET already describes 250

NOOP
    change
     It specifies no action other than that the receiver send an OK
     reply.
    to
     It specifies no action other than that the receiver send a
     "250 OK" reply.

QUIT
    change
     This command specifies that the receiver MUST send an OK reply,
    to
     This command specifies that the receiver MUST send a "250 OK"
     reply,

</pseudo-chair hat off>

	Tony Hansen
	[email-masked]

John C Klensin wrote:
> Hi.
> 
> In the process of trying to sort through the correspondence
> about NOOP, I've gone back and carefully read 4.3.2.   I believe
> that Hector correctly spotted a real issue that could use some
> clarification even though, in looking at one particular piece of
> misbehavior (the use of 421), we all missed what I now think is
> the key point.  
> 
> Section 4.3.2 specifies three codes that can be used in response
> to any command.  They are 421 (which we have been discussing),
> 500, and 501.  The 500 text reads:
> 
> # 500  For the "command line too long" case or if the command 
> #     name was not recognized.  Note that producing a "command
> #     not recognized" error in response to the required subset
> #     of these commands is a violation of this specification.
> 
> Now, suppose that, instead of the 421 that Hector observed and
> reported on, the server had responded to 
>>  NOOP aa
> with
>>  500 5.5.2 Syntax error (command line too long)
> 
> Now, that would have avoided violating the several constraints
> that we have been discussing with 421 as well as the disconnect
> between the first digits of the primary and extended codes
> (while I agree with Tony that a bogus string is valid, if the
> server advertises ENHANCEDSTATUSCODES, I believe that a bogus
> enhanced code is not, even though it is not a 2821bis problem).  
> 
> However, I contend it would still be a bug relative to the
> 2821[bis] spec and that the comments in the Microsoft Knowledge
> Base article that Hector kindly identified reinforce that view.
> That article talks about a "default filter value" for a "NOOP
> SMTP command" being set at six characters (presumably
> NOOP<crlf>), acknowledges that it is a problem and provides a
> fix (the vendor already knows that they have a bug, although
> they apparently haven't figured out the 421 problem).   
> 
> 	(Incidentally, I can see nothing in that article or
> 	Hectors client-server exchange that leads me to believe
> 	that they are trying to conform to 821 and not 2821.
> 	They cite both specs, but respond to EHLO with a long
> 	list of extensions, quote extensively from 2821 and not,
> 	where they are different, from 821, etc.  The only
> 	justification for this behavior is the "defend
> 	themselves" provision, which they do not cite, and
> 	even that would be a stretch.)
> 
> Now, my reading of Section 4.5.3.1.4 (the "command line"
> material in the "Size limits and minimums" section) is that one
> doesn't get to use "command line too long" in an error reply
> without violating the spec, i.e., that the notion that one can
> figure out a maximum length, in the absence of extensions, on a
> per-command basis and then apply it as a "filter" that results
> in length-based rejection messages is incorrect and broken.
> But, unlike the applicability of 421, I don't think that is
> completely clear.
> 
> So, if others agree with my interpretation, I propose to modify
> the description of 500 quoted above by adding to the end...
> 
> 		Similarly, producing a "command line too long" message
> 		for a command line shorter than 1000 characters would
> 		violate the provisions of Section 4.5.3.1.4.
> 
> 
> Hector's concerns about NOOP also raise another, more general,
> issue, which is whether we are being clear enough about the
> expected normal replies to commands.  I think that, if it isn't
> obvious, one reads the command description, understands the
> theory of reply codes (there is a cross-reference from the
> introduction to Section 4.1, which lists the commands and syntax
> to Section 4.2, which describes replies), and then looks in
> Section 4.3.2 for a 2yz code associated with the particular
> command.  That is the way it has been since 821 and, if one
> believes it, then the answer to the question "what is the normal
> reply to NOOP" is clearly already answered as "250".  The
> commands that can return more than one 2yz codes should have the
> differences between those codes described in the command
> descriptions; I believe all of them do, but I'd welcome someone
> else checking on that.  
> 
> But one of the implied goals for 2821bis has been making the
> spec easier to read and understand without a comprehensive
> understanding of how it is organized.  So, with the
> understanding that this is a question and not a recommendation
> (since I do not have an opinion), would it be useful to insert
> text in the command descriptions for the commands that have only
> a single positive reply (EHLO/HELO, DATA, RSET, NOOP, and QUIT)
> that explicitly identifies the code that is expected for a
> normal/successful reply?  Or would that be just clutter that
> makes the document longer to no good end?
> 
>      john
> 

Date: Wed, 13 Feb 2008 00:27:04 -0600
From: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: New Issue: NOOP clarification
Sender: [email-masked]

<pseudo-chair hat on>

There were a number of suggestions both pro and con, but this suggestion 
did not garner enough support to make a change to the spec.

</pseudo-chair hat off>

	Tony Hansen
	[email-masked]

Hector Santos wrote:
> 
> Glenn Anderson wrote:
>>
>> At 6:12 pm -0500 29/11/2007, Hector Santos wrote:
>>> So 421 is implied for real server critical shut down reasons in 2821 
>>> compliant servers across the board.
>>
>> However shutting down because it doesn't like a parameter on a NOOP 
>> command is to me a very clear violation of section 3.9 of RFC 2821 
>> (section 3.8 in 2821bis):
>>    In particular, a server that closes connections in response to
>>    commands that are not understood is in violation of this
>>    specification.
> 
> 
> +1, I agree.
> 
> The question is, why did it happen? was the specs ambiguous?  Was it a 
> legacy 821 system which does not have an string for NOOP who had to 
> decide between 500 and 421, both wrong, but nonetheless used in a major 
> vendor product line, and possibly others?  Was it a proxy and this major 
> vendor had nothing to do with it?
> 
> Whatever the reason, maybe 2821bis can help minimize NOOP issues in 
> future SMTP servers and also help legacy developers or proxy writers 
> fine tune their own products.
> 
> Thanks for your comments.
> 

Date: Wed, 13 Feb 2008 00:26:13 -0600
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Closing out the comments
Sender: [email-masked]

<pseudo-chair hat on>

I'm going to be posting a bunch of notes shortly, closing out all of the 
extant issues.

John will be posting an updated draft shortly, reflecting these 
consensus calls.

</pseudo-chair hat off>

	Tony Hansen
	[email-masked]

Date: Mon, 04 Feb 2008 12:28:19 -0800
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
To: [email-masked]
Subject: [Fwd: STD 68, RFC 5234 on Augmented BNF for Syntax Specifications: ABNF]
Sender: [email-masked]

FYI.

(Sorry for the cross-posting, but I can't think of a single-best list to post to.)

-------- Original Message --------
Subject: STD 68, RFC 5234 on Augmented BNF for Syntax Specifications: ABNF
Date: Thu, 31 Jan 2008 17:24:14 -0800 (PST)
From: [email-masked]
To: [email-masked], [email-masked]
CC: [email-masked]

A new Request for Comments is now available in online RFC libraries.

         STD 68
         RFC 5234

         Title:      Augmented BNF for Syntax Specifications:
                     ABNF
         Author:     D. Crocker, Ed.,
                     P. Overell
         Status:     Standards Track
         Date:       January 2008
         Mailbox:    [email-masked],
                     [email-masked]
         Pages:      16
         Characters: 26359
         Obsoletes:  RFC4234
         See Also:   STD0068

         I-D Tag:    draft-crocker-rfc4234bis-01.txt

         URL:        http://www.rfc-editor.org/rfc/rfc5234.txt

Internet technical specifications often need to define a formal
syntax.  Over the years, a modified version of Backus-Naur Form
(BNF), called Augmented BNF (ABNF), has been popular among many
Internet specifications.  The current specification documents ABNF.
It balances compactness and simplicity with reasonable
representational power.  The differences between standard BNF and
ABNF involve naming rules, repetition, alternatives,
order-independence, and value ranges.  This specification also
supplies additional rule definitions and encoding for a core lexical
analyzer of the type common to several Internet specifications.
[STANDARDS TRACK]

This is now a Standard Protocol.

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Mon, 04 Feb 2008 10:18:01 +0100
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: fix forwarding: sect. 3.9 of draft-klensin-rfc2821bis
Sender: [email-masked]

Frank Ellermann wrote:
> Alessandro Vesely wrote:
>>| Because of its administrative relevance, this address MAY
>>| be used to devise local acceptance policies based on the
> [...]
>>| While recognizing the existence of such policies, this
>>| paragraph only describes general issues intended for
>>| efficient and reliable transport and delivery of mail messages.
> 
> I'd remove that part, it is rather vague.  Of course the
> MAIL FROM like anything else in the envelope can be used
> for the decision to reject or accept a mail.  And as the
> reverse path MUST be used to report non-delivery to the
> originator it's arguably not only an option (MAY) to make
> sure that it's no nonsense when accepting it.

I agree that having a reliable envelope sender is necessary for mail 
delivery reliability, even if that conflicts with current specifications 
and practices. However, what should an MTA do when it has a piece of spam 
with a faked return path?

In the context and notes for the draft, the spam problem is dismissed 
because its solutions change too rapidly. That situation somehow resembles 
the status of SMTP extensions. However, spam defense methods are _not_ 
quite similar to SMTP extensions. One key difference is that they alter 
the semantics of SMTP rather than altering the protocol. For example, 
viruses require an accept-and-discard approach to avoid bounces to yet 
another victim in the return path; large ESPs, e.g. Hotmail, discard 
legitimate mail in the same manner. Not mentioning that, IMHO, may put 
2821bis beyond belief.