Subject: Re: Logging port numbers
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Date: 17 Jun 2011 16:47:00 -0000
List-Id: <ietf-smtp.imc.org>

>The above example already belongs to "lot of funky Received: in the world",
>it is missing an "IPv6:" prefix in front of the IPv6 address.

Oops, you're right.  That happens to be my own mail server, defunked.

R's,
John

VBR-Info: md=ijs.si; mc=all; mv=dwl.spamhaus.org;
From: Mark Martinec <[email-masked]>
Organization: J. Stefan Institute
To: [email-masked]
Subject: Re: Logging port numbers
Date: Fri, 17 Jun 2011 17:16:30 +0200
Sender: [email-masked]

John Levine wrote:

> [...] That's technically easy enough to do, it took about
> 5 minutes to patch my SMTP server to log the port number [...]
> 
> Received: from cl-t108-372cl.privatedns.com
>   ([2607:f748:1200:fa:230:48ff:febd:a370]) by mail6.iecc.com
>   ([2001:470:1f07:1126::5370:616d:6d79])
>   with ESMTP via TCP id 2467293036; 28 May 2011 16:18:48 -0000

> That has the disadvantage that RFC 5321 has a nice clear syntax
> definition for a Time-stamp-line, and it doesn't include a :port after
> the IP address.  I realize that there are a lot of funky Received:
> lines in the world, but I would prefer not to add yet more gratuitous
> funkitude.

The above example already belongs to "lot of funky Received: in the world",
it is missing an "IPv6:" prefix in front of the IPv6 address.

   TCP-info = address-literal / ( Domain FWS address-literal )

   address-literal  = "[" ( IPv4-address-literal /
                    IPv6-address-literal /
                    General-address-literal ) "]"

   IPv6-address-literal  = "IPv6:" IPv6-addr

   IPv6-addr = IPv6-full / IPv6-comp / IPv6v4-full / IPv6v4-comp

   General-address-literal  = Standardized-tag ":" 1*dcontent

Keith Moore wrote:
> I don't have a strong opinion between
> 
> Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]:46703)
>        by mail.highwayman.com ([10.1.2.3]:25) with esmtp (Exim 4.76)
>        (envelope-from <[email-masked]>)
>        id 1QTwHl-000JhQ-W1
>        for [email-masked]; Tue, 07 Jun 2011 14:25:34 +0100
> 
> and 
> Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141] port 46703)
>        by mail.highwayman.com ([10.1.2.3] port 25) with esmtp (Exim 4.76)
>        (envelope-from <[email-masked]>)
>        id 1QTwHl-000JhQ-W1
>        for [email-masked]; Tue, 07 Jun 2011 14:25:34 +0100
> 
> but I do think the former is an extension of a very common convention
> and is likely to be readily understood.

I agree, the [131.111.8.141]:46703 notation is widely used elsewhere.
Either of these breaks the TCP-info syntax.

A possible (but ugly) standard-compliant way would be to make use of
the syntax:

  General-address-literal  = Standardized-tag ":" 1*dcontent

and define Standardized-tag such as IPv4-with-port and IPv6-with-port.

  Mark

Subject: Re: Logging port numbers
From: "J.D. Falk" <[email-masked]>
Date: Thu, 9 Jun 2011 19:06:09 -0700
To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

On Jun 7, 2011, at 1:42 PM, John Levine wrote:

>> p.s. Though this does beg the question: how useful is it to log port
>> numbers when there's a very good chance that one or both ends are
>> behind one or more layers of NATs that don't log state changes?  I
>> understand and agree with the notion that LSN will dilute the value
>> of IP addresses as source identifiers.  But as far as I can tell,
>> they are already of pretty marginal value even without LSN.
> 
> I'm sitting in a meeting talking about this very topic (LSN) with
> people from organizations that manage a lot of users and a lot of
> address space, and the consensus is that there will be enough logging
> at the NAT end that it's worth logging the ports.

Whether it's the "right" thing to do or not, it sounds very likely that multiple operators will start trying to cram port numbers into Received: headers fairly soon.  It'd be helpful if the assembled experts here could give them some guidance.  Otherwise, they'll each make their own decisions -- and the trace headers will become even more confused.

--
J.D. Falk
the leading purveyor of industry counter-rhetoric solutions

Date: Wed, 8 Jun 2011 13:57:46 +0100
From: Tony Finch <[email-masked]>
To: Richard Clayton <[email-masked]>
cc: [email-masked]
Subject: Re: Logging port numbers
Sender: [email-masked]

Richard Clayton <[email-masked]> wrote:
>
> this dates back to at least 2003 (I suspect it was new in Exim 4)

Exim 3 had a log_incoming_port option but it did not have the side-effect
of including port numbers in Received: headers.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Northeast Viking, North Utsire: Variable becoming southwesterly, 3 or 4.
Slight or moderate. Rain. Moderate or poor.

Date: 7 Jun 2011 20:42:40 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: Logging port numbers
Organization: 
Sender: [email-masked]

>p.s. Though this does beg the question: how useful is it to log port
>numbers when there's a very good chance that one or both ends are
>behind one or more layers of NATs that don't log state changes?  I
>understand and agree with the notion that LSN will dilute the value
>of IP addresses as source identifiers.  But as far as I can tell,
>they are already of pretty marginal value even without LSN.

I'm sitting in a meeting talking about this very topic (LSN) with
people from organizations that manage a lot of users and a lot of
address space, and the consensus is that there will be enough logging
at the NAT end that it's worth logging the ports.

R's,
John

Subject: Re: Logging port numbers
From: Keith Moore <[email-masked]>
Date: Tue, 7 Jun 2011 13:40:48 -0400
Cc: [email-masked], [email-masked]
To: John Levine <[email-masked]>
Sender: [email-masked]

--Apple-Mail-14-596250075
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On Jun 7, 2011, at 1:18 PM, John Levine wrote:

>> BTW: Received header fields have always been discussed (not always in =
a
>> consistent way) in both the x821 and x822 documents (and probably
>> earlier than that as well). So you do need to look in both documents =
to
>> be sure what you propose is OK :)
>=20
> Yes, I looked. The syntax in RFC 5322 is considerably looser than in
> 5321, so 5321 is the spec that matters in this case.
>=20
> I'm not unalterably opposed to [ip]:port but I worry that it would =
break
> more header parsers than adding a port or lport/rport clause.

parsers that rely on the syntax of comments in received fields should =
expect to be broken occasionally.

I don't have a strong opinion between

> Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]:46703)
       by mail.highwayman.com ([10.1.2.3]:25) with esmtp (Exim 4.76)
       (envelope-from <[email-masked]>)
       id 1QTwHl-000JhQ-W1
       for [email-masked]; Tue, 07 Jun 2011 14:25:34 +0100

and=20

> Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141] port 46703)
       by mail.highwayman.com ([10.1.2.3] port 25) with esmtp (Exim =
4.76)
       (envelope-from <[email-masked]>)
       id 1QTwHl-000JhQ-W1
       for [email-masked]; Tue, 07 Jun 2011 14:25:34 +0100

but I do think the former is an extension of a very common convention =
and is likely to be readily understood.

Keith

p.s. Though this does beg the question: how useful is it to log port =
numbers when there's a very good chance that one or both ends are behind =
one or more layers of NATs that don't log state changes?  I understand =
and agree with the notion that LSN will dilute the value of IP addresses =
as source identifiers.  But as far as I can tell, they are already of =
pretty marginal value even without LSN.

--Apple-Mail-14-596250075
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div><div>On Jun 7, 2011, at 1:18 PM, John Levine =
wrote:</div><br><blockquote type=3D"cite"><div><blockquote =
type=3D"cite">BTW: Received header fields have always been discussed =
(not always in a<br></blockquote><blockquote type=3D"cite">consistent =
way) in both the x821 and x822 documents (and =
probably<br></blockquote><blockquote type=3D"cite">earlier than that as =
well). So you do need to look in both documents =
to<br></blockquote><blockquote type=3D"cite">be sure what you propose is =
OK :)<br></blockquote><br>Yes, I looked. The syntax in RFC 5322 is =
considerably looser than in<br>5321, so 5321 is the spec that matters in =
this case.<br><br>I'm not unalterably opposed to [ip]:port but I worry =
that it would break<br>more header parsers than adding a port or =
lport/rport clause.<br></div></blockquote></div><br><div>parsers that =
rely on the syntax of comments in received fields should expect to be =
broken occasionally.</div><div><br></div><div>I don't have a strong =
opinion between</div><div><br></div><div>Received: from =
ppsw-41.csi.cam.ac.uk =
([131.111.8.141]:46703)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;by&nb=
sp;<a =
href=3D"http://mail.highwayman.com/">mail.highwayman.com</a>&nbsp;([10.1.2=
.3]:25)&nbsp;with esmtp (Exim =
4.76)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(envelope-from &lt;<a =
href=3D"mailto:[email-masked]">[email-masked]</a>&gt;)<br>&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;id =
1QTwHl-000JhQ-W1<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;for&nbsp;<a =
href=3D"mailto:[email-masked]">[email-masked]</a>; Tue, =
07 Jun 2011 14:25:34 =
+0100<br></div><div><br></div><div>and&nbsp;</div><div><br></div><div>Rece=
ived: from ppsw-41.csi.cam.ac.uk ([131.111.8.141] port =
46703)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;by&nbsp;<a =
href=3D"http://mail.highwayman.com/">mail.highwayman.com</a>&nbsp;([10.1.2=
.3] port 25)&nbsp;with esmtp (Exim =
4.76)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(envelope-from &lt;<a =
href=3D"mailto:[email-masked]">[email-masked]</a>&gt;)<br>&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;id =
1QTwHl-000JhQ-W1<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;for&nbsp;<a =
href=3D"mailto:[email-masked]">[email-masked]</a>; Tue, =
07 Jun 2011 14:25:34 +0100<br></div><div><br></div><div>but I do think =
the former is an extension of a very common convention and is likely to =
be readily =
understood.</div><div><br></div><div>Keith</div><div><br></div><div>p.s. =
Though this does beg the question: how useful is it to log port numbers =
when there's a very good chance that one or both ends are behind one or =
more layers of NATs that don't log state changes? &nbsp;I understand and =
agree with the notion that LSN will dilute the value of IP addresses as =
source identifiers. &nbsp;But as far as I can tell, they are already of =
pretty marginal value even without =
LSN.</div><div><br></div><div><br></div></body></html>=

--Apple-Mail-14-596250075--

Date: 7 Jun 2011 17:18:13 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: Logging port numbers
Organization: 
Sender: [email-masked]

>BTW: Received header fields have always been discussed (not always in a
>consistent way) in both the x821 and x822 documents (and probably
>earlier than that as well). So you do need to look in both documents to
>be sure what you propose is OK :)

Yes, I looked. The syntax in RFC 5322 is considerably looser than in
5321, so 5321 is the spec that matters in this case.

I'm not unalterably opposed to [ip]:port but I worry that it would break
more header parsers than adding a port or lport/rport clause.

R's,
John

Date: Tue, 7 Jun 2011 14:41:26 +0100
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: Logging port numbers
Sender: [email-masked]

In article <[email-masked]>, John Levine
<[email-masked]> writes

>I see that this draft will shortly be an RFC:
>
>http://datatracker.ietf.org/doc/draft-ietf-intarea-server-logging-
>recommendations/
>
>It tells us that in the coming era of giant NATs, to do useful logging,
>along with the IP address and an accurate timestamp, you also need to
>log the port number.  That's technically easy enough to do, it took about
>5 minutes to patch my SMTP server to log the port number

in Exim you use:
        
        log_selector = +incoming_port

and a typical received line is then:

> Received: from ppsw-41.csi.cam.ac.uk ([131.111.8.141]:46703)
        by mail.highwayman.com with esmtp (Exim 4.76)
        (envelope-from <[email-masked]>)
        id 1QTwHl-000JhQ-W1
        for [email-masked]; Tue, 07 Jun 2011 14:25:34 +0100

this dates back to at least 2003 (I suspect it was new in Exim 4)

>That has the disadvantage that RFC 5321 has a nice clear syntax
>definition for a Time-stamp-line, and it doesn't include a :port after
>the IP address.  I realize that there are a lot of funky Received:
>lines in the world, but I would prefer not to add yet more gratuitous
>funkitude.

following the lead taken by such a widely used program as Exim won't add
much to bio-diversity!

BTW: Received header fields have always been discussed (not always in a
consistent way) in both the x821 and x822 documents (and probably
earlier than that as well). So you do need to look in both documents to
be sure what you propose is OK :)

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Subject: Re: Logging port numbers
From: Keith Moore <[email-masked]>
Date: Mon, 6 Jun 2011 02:29:26 -0400
Cc: "[email-masked]" <[email-masked]>
To: "Murray S. Kucherawy" <[email-masked]>
Sender: [email-masked]

On Jun 6, 2011, at 1:01 AM, Murray S. Kucherawy wrote:

>> -----Original Message-----
>> From: Keith Moore [mailto:[email-masked]]
>> Sent: Sunday, June 05, 2011 9:39 AM
>> To: Murray S. Kucherawy
>> Cc: [email-masked]
>> Subject: Re: Logging port numbers
>> 
>> I'll second the notion that source port and destination port should be
>> clearly distinguished.  Also, putting the source port # next to the
>> destination IP address strikes me as confusing.
> 
> Why?  It's a fairly common notation.

putting the source port # next to the source address, and putting the destination port # next to the destination address, makes sense.   putting the source port number next to the destination address doesn't make as much sense to me.

Keith

From: "Murray S. Kucherawy" <[email-masked]>
To: Keith Moore <[email-masked]>
CC: "[email-masked]" <[email-masked]>
Date: Sun, 5 Jun 2011 22:01:03 -0700
Subject: RE: Logging port numbers
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: Keith Moore [mailto:[email-masked]]
> Sent: Sunday, June 05, 2011 9:39 AM
> To: Murray S. Kucherawy
> Cc: [email-masked]
> Subject: Re: Logging port numbers
> 
> I'll second the notion that source port and destination port should be
> clearly distinguished.  Also, putting the source port # next to the
> destination IP address strikes me as confusing.

Why?  It's a fairly common notation.

Date: 6 Jun 2011 04:44:39 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: Logging port numbers
Organization: 
Sender: [email-masked]

>Also, IP addresses of either end of the connection are not recorded in
>any formally defined manner in trace headers- they're just stuffed into a
>comment string in a not entirely consistent or predictable way (try HELOing
>with a dotted quad and see what gets recorded where).

RFC 5321 is pretty clear about what goes in a Received header.  I
agree that the implementations are not always ideal.

R's,
John

Date: Sun, 05 Jun 2011 22:29:22 -0400
From: John C Klensin <[email-masked]>
To: John Levine <[email-masked]>, [email-masked]
Subject: Re: Logging port numbers
Sender: [email-masked]

--On Sunday, June 05, 2011 05:56 +0000 John Levine
<[email-masked]> wrote:

>...
> So scrutinizing page 60 of RFC 5321, I see a list of opt-info
> clauses which offers Additional-Registered-Clauses.  So I
> propose a port clause, like this:
> 
> Received: from vance3.state.gov (vance.state.gov
> [169.253.4.31])   by mail1.iecc.com ([64.57.183.56])
>   with ESMTP via TCP port 1234 id 2467626873; 03 Jun 2011
> 14:34:46 -0000
> 
> Received: from cl-t108-372cl.privatedns.com
> ([2607:f748:1200:fa:230:48ff:febd:a370])   by mail6.iecc.com
> ([2001:470:1f07:1126::5370:616d:6d79])   with ESMTP via TCP
> port 1234 id 2467293036; 28 May 2011 16:18:48 -0000
> 
> There is a registry for Additional-registered-clauses, which is
> currently empty, but we could fix that.  Seems reasonable?

Yes.  I don't have a strong opinion about whether there should
be two port keywords or one with a syntax that could be used to
record one, the other, or both.

best,
    john

Date: Sun, 05 Jun 2011 22:25:18 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: RFC5321 SMTP Cleanup - Retries Strategy
Sender: [email-masked]

This is no big deal, I noticed in section 2.2.3 it has a 
recommendation of 3 days and in section 4.5.4.1, it has a 
recommendation of 4-5 days.

2.2.3 talks about "extensions" and it notes:

    (e.g., if normal timeout is three days, the requeue timeout before
    attempting to transmit the mail without the extension might be one 
day).

I am talking about the usage of the word "normal" as this seems to 
imply the recommended normal retry timeout which would be different 
than what it implied in 4.5.4.1:

    Retries continue until the message is transmitted or the sender gives
    up; the give-up time generally needs to be at least 4-5 days.

So I guess maybe in 2.2.3 (when crossing this bridge (BIS) again), it 
would be changed to say:

    (e.g., if normal timeout is 4-5 days, the requeue timeout before
    attempting to transmit the mail without the extension might be one 
day).

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sun, 05 Jun 2011 19:26:41 +0200
From: Alessandro Vesely <[email-masked]>
To: SMTP Interest Group <[email-masked]>
CC: [email-masked]
Subject: Re: Logging port numbers
Sender: [email-masked]

On 05/Jun/11 18:51, Steve Atkins wrote:
> If the point of this is NAT tracking then, AIUI, the meaningful bit of data
> is always the source and destination port pair, so maybe it would simplify
> things to always record that pair, rather than having two separate records.
> 
> Also, IP addresses of either end of the connection are not recorded in
> any formally defined manner in trace headers- they're just stuffed into a
> comment string in a not entirely consistent or predictable way (try HELOing
> with a dotted quad and see what gets recorded where).
> 
> Adding formal structure for port numbers where there isn't any for source
> and destination addresses seems a bit strange. Either the port numbers
> can go in a comment, or the additional structure should record the full
> source-ip / source-port / destination-ip / destination-port, maybe?

+1, but this is probably the wrong list, if we expect the I-D authors
to populate their IANA considerations section with the relevant clause
for "Sub-registry: Additional-registered-clauses" of
http://www.iana.org/assignments/mail-parameters

Subject: Re: Logging port numbers
From: Steve Atkins <[email-masked]>
Date: Sun, 5 Jun 2011 09:51:04 -0700
To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

On Jun 5, 2011, at 7:42 AM, John Levine wrote:

> 
>> Works for me.  The only thing I can see that might come up is that not
>> all SMTP servers listen on port 25, so maybe we need a way of
>> (optionally) logging the port on which the SMTP server was listening as
>> well?
> 
> Sure, could be PORT and IPORT.  Purists might argue that if it's port
> 587 it should be WITH SUBMIT rather than WITH SMTP.

If the point of this is NAT tracking then, AIUI, the meaningful bit of data
is always the source and destination port pair, so maybe it would simplify
things to always record that pair, rather than having two separate records.

Also, IP addresses of either end of the connection are not recorded in
any formally defined manner in trace headers- they're just stuffed into a
comment string in a not entirely consistent or predictable way (try HELOing
with a dotted quad and see what gets recorded where).

Adding formal structure for port numbers where there isn't any for source
and destination addresses seems a bit strange. Either the port numbers
can go in a comment, or the additional structure should record the full
source-ip / source-port / destination-ip / destination-port, maybe?

Cheers,
  Steve

Subject: Re: Logging port numbers
From: Keith Moore <[email-masked]>
Date: Sun, 5 Jun 2011 12:39:19 -0400
Cc: "[email-masked]" <[email-masked]>
To: "Murray S. Kucherawy" <[email-masked]>
Sender: [email-masked]

I'll second the notion that source port and destination port should be clearly distinguished.  Also, putting the source port # next to the destination IP address strikes me as confusing.

Keith

On Jun 5, 2011, at 2:04 AM, Murray S. Kucherawy wrote:

> 
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On Behalf Of John Levine
>> Sent: Saturday, June 04, 2011 10:57 PM
>> To: [email-masked]
>> Subject: Logging port numbers
>> 
>> There is a registry for Additional-registered-clauses, which is
>> currently empty, but we could fix that.  Seems reasonable?
> 
> Works for me.  The only thing I can see that might come up is that not all SMTP servers listen on port 25, so maybe we need a way of (optionally) logging the port on which the SMTP server was listening as well?

Date: 5 Jun 2011 14:42:56 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: Logging port numbers
Organization: 
Sender: [email-masked]

>Works for me.  The only thing I can see that might come up is that not
>all SMTP servers listen on port 25, so maybe we need a way of
>(optionally) logging the port on which the SMTP server was listening as
>well?

Sure, could be PORT and IPORT.  Purists might argue that if it's port
587 it should be WITH SUBMIT rather than WITH SMTP.

R's,
John

Date: Sun, 05 Jun 2011 02:37:37 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: [email-masked]
Subject: Re: Logging port numbers
To: [email-masked]
Sender: [email-masked]

There is a different in LOGS vs RFC5322 Trace (Received) lines.

I will not want to add a non-public private port on a RFC5322 trace 
line that is publicly viewable. It can present undesirable security 
openings.

-1

-- 
Sincerely

Hector Santos
http://www.santronics.com

John Levine wrote:
> I see that this draft will shortly be an RFC:
> 
> http://datatracker.ietf.org/doc/draft-ietf-intarea-server-logging-recommendations/
> 
> It tells us that in the coming era of giant NATs, to do useful logging,
> along with the IP address and an accurate timestamp, you also need to
> log the port number.  That's technically easy enough to do, it took about
> 5 minutes to patch my SMTP server to log the port number, or more accurately
> to get to the point where I couldn't figure out where in the Received:
> line to put the port number. Here's what my Received: lines look like now:
> 
> Received: from vance3.state.gov (vance.state.gov [169.253.4.31])
>   by mail1.iecc.com ([64.57.183.56])
>   with ESMTP via TCP id 2467626873; 03 Jun 2011 14:34:46 -0000
> 
> Received: from cl-t108-372cl.privatedns.com ([2607:f748:1200:fa:230:48ff:febd:a370])
>   by mail6.iecc.com ([2001:470:1f07:1126::5370:616d:6d79])
>   with ESMTP via TCP id 2467293036; 28 May 2011 16:18:48 -0000
> 
> One possibility would be to put them after the IP separated by a colon, e.g.
> 
> Received: from vance3.state.gov (vance.state.gov [169.253.4.31]:1234)
>   by mail1.iecc.com ([64.57.183.56])
>   with ESMTP via TCP id 2467626873; 03 Jun 2011 14:34:46 -0000
> 
> Received: from cl-t108-372cl.privatedns.com ([2607:f748:1200:fa:230:48ff:febd:a370]:1234)
>   by mail6.iecc.com ([2001:470:1f07:1126::5370:616d:6d79])
>   with ESMTP via TCP id 2467293036; 28 May 2011 16:18:48 -0000
> 
> That has the disadvantage that RFC 5321 has a nice clear syntax
> definition for a Time-stamp-line, and it doesn't include a :port after
> the IP address.  I realize that there are a lot of funky Received:
> lines in the world, but I would prefer not to add yet more gratuitous
> funkitude.  Also, one of the more typical mistakes is to leave out the
> [ ] in which case a port number on an IPv6 address could be ambiguous.
> 
> So scrutinizing page 60 of RFC 5321, I see a list of opt-info clauses
> which offers Additional-Registered-Clauses.  So I propose a port
> clause, like this:
> 
> Received: from vance3.state.gov (vance.state.gov [169.253.4.31])
>   by mail1.iecc.com ([64.57.183.56])
>   with ESMTP via TCP port 1234 id 2467626873; 03 Jun 2011 14:34:46 -0000
> 
> Received: from cl-t108-372cl.privatedns.com ([2607:f748:1200:fa:230:48ff:febd:a370])
>   by mail6.iecc.com ([2001:470:1f07:1126::5370:616d:6d79])
>   with ESMTP via TCP port 1234 id 2467293036; 28 May 2011 16:18:48 -0000
> 
> There is a registry for Additional-registered-clauses, which is
> currently empty, but we could fix that.  Seems reasonable?
> 
> R's,
> John
> 
> 
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Sat, 4 Jun 2011 23:04:29 -0700
Subject: RE: Logging port numbers
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of John Levine
> Sent: Saturday, June 04, 2011 10:57 PM
> To: [email-masked]
> Subject: Logging port numbers
> 
> There is a registry for Additional-registered-clauses, which is
> currently empty, but we could fix that.  Seems reasonable?

Works for me.  The only thing I can see that might come up is that not all SMTP servers listen on port 25, so maybe we need a way of (optionally) logging the port on which the SMTP server was listening as well?

Date: 5 Jun 2011 05:56:39 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Subject: Logging port numbers
Organization: 
Sender: [email-masked]

I see that this draft will shortly be an RFC:

http://datatracker.ietf.org/doc/draft-ietf-intarea-server-logging-recommendations/

It tells us that in the coming era of giant NATs, to do useful logging,
along with the IP address and an accurate timestamp, you also need to
log the port number.  That's technically easy enough to do, it took about
5 minutes to patch my SMTP server to log the port number, or more accurately
to get to the point where I couldn't figure out where in the Received:
line to put the port number. Here's what my Received: lines look like now:

Received: from vance3.state.gov (vance.state.gov [169.253.4.31])
  by mail1.iecc.com ([64.57.183.56])
  with ESMTP via TCP id 2467626873; 03 Jun 2011 14:34:46 -0000

Received: from cl-t108-372cl.privatedns.com ([2607:f748:1200:fa:230:48ff:febd:a370])
  by mail6.iecc.com ([2001:470:1f07:1126::5370:616d:6d79])
  with ESMTP via TCP id 2467293036; 28 May 2011 16:18:48 -0000

One possibility would be to put them after the IP separated by a colon, e.g.

Received: from vance3.state.gov (vance.state.gov [169.253.4.31]:1234)
  by mail1.iecc.com ([64.57.183.56])
  with ESMTP via TCP id 2467626873; 03 Jun 2011 14:34:46 -0000

Received: from cl-t108-372cl.privatedns.com ([2607:f748:1200:fa:230:48ff:febd:a370]:1234)
  by mail6.iecc.com ([2001:470:1f07:1126::5370:616d:6d79])
  with ESMTP via TCP id 2467293036; 28 May 2011 16:18:48 -0000

That has the disadvantage that RFC 5321 has a nice clear syntax
definition for a Time-stamp-line, and it doesn't include a :port after
the IP address.  I realize that there are a lot of funky Received:
lines in the world, but I would prefer not to add yet more gratuitous
funkitude.  Also, one of the more typical mistakes is to leave out the
[ ] in which case a port number on an IPv6 address could be ambiguous.

So scrutinizing page 60 of RFC 5321, I see a list of opt-info clauses
which offers Additional-Registered-Clauses.  So I propose a port
clause, like this:

Received: from vance3.state.gov (vance.state.gov [169.253.4.31])
  by mail1.iecc.com ([64.57.183.56])
  with ESMTP via TCP port 1234 id 2467626873; 03 Jun 2011 14:34:46 -0000

Received: from cl-t108-372cl.privatedns.com ([2607:f748:1200:fa:230:48ff:febd:a370])
  by mail6.iecc.com ([2001:470:1f07:1126::5370:616d:6d79])
  with ESMTP via TCP port 1234 id 2467293036; 28 May 2011 16:18:48 -0000

There is a registry for Additional-registered-clauses, which is
currently empty, but we could fix that.  Seems reasonable?

R's,
John

Date: Thu, 02 Jun 2011 16:12:18 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC: Peter Bowen <[email-masked]>, [email-masked]
Subject: Re: LOGIN mechanism
Sender: [email-masked]

Follow up:

A quick GOOGLE,

QMAIL shows example using it:

      http://qmail.jms1.net/test-auth.shtml

Here is a tutorial, and a mentioning of some client parsing problem 
and what it calls a workaround using AUTH=LOGIN:

      http://www.fehcom.de/qmail/smtpauth.html

I'm sure its an old compatibility issue and today, not a problem with 
updated software. But most smtp vendors only need one complaint from 
an operator one of their users has connection problem and generally it 
all depends on how big a hat the user wears before people react. So if 
you already have it - leave it in, and if you don't, eh, its up to 
you.  Odds are good if you never saw it then, you will never see it 
now. :)

Hector Santos wrote:
> Tony Finch wrote:
> 
>> I have never seen a client that depends on seeing AUTH=LOGIN in the EHLO
>> service extensions list, nor have I heard of any interop problems caused
>> by Exim not advertising AUTH=LOGIN.
> 
> Note this is an old client problem, and the inline code comments I have is:
> 
>     // Support for NON-RFC AUTH=LOGIN
>     // - Designed by NETSCAPE, SUPPORTED by MICROSOFT
>     // - supported by at least NS4, NS4.5, OL4, OL5
> 
> This is in the last recorded source control version dating back to 2001.
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 02 Jun 2011 15:51:00 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC: Peter Bowen <[email-masked]>, [email-masked]
Subject: Re: LOGIN mechanism
Sender: [email-masked]

Tony Finch wrote:

> I have never seen a client that depends on seeing AUTH=LOGIN in the EHLO
> service extensions list, nor have I heard of any interop problems caused
> by Exim not advertising AUTH=LOGIN.

Note this is an old client problem, and the inline code comments I 
have is:

     // Support for NON-RFC AUTH=LOGIN
     // - Designed by NETSCAPE, SUPPORTED by MICROSOFT
     // - supported by at least NS4, NS4.5, OL4, OL5

This is in the last recorded source control version dating back to 2001.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 2 Jun 2011 21:50:08 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: RSET command - possible security loophole
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--KsGdsel6WgEHnImy
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2011-06-01 17:34:58 -0400, John C Klensin wrote:
> --On Wednesday, June 01, 2011 16:57 -0400 Hector Santos
> <[email-masked]> wrote:

> > The following is invalid because it clears the MAIL FROM:
> >=20
> >   C: RCPT TO:<[email-masked]>
> >   S: 530 Authentication required for relaying
> >   C: RSET
> >   S: 250 OK
> >   C: AUTH LOGIN
>=20
> Assuming there was a MAIL command prior to the RCPT one,
> otherwise it is invalid for other reasons (the response to RCPT
> should be "503 Bad sequence of commands").

Yup.

> AUTH after RSET but before MAIL is invalid only because 4894 says so.
> 5321 doesn't have a position on the subject.

AUTH after RSET but before MAIL is valid (not "during a mail
transaction"). AUTH after MAIL would be invalid (it is "during a mail
transaction").

	hp

--=20
   _  | Peter J. Holzer    | Web 2.0 k=F6nnte man also auch =FCbersetzen als
|_|_) | Sysadmin WSR       | "Netz der kleinen Geister".
| |   | [email-masked]         |=20
__/   | http://www.hjp.at/ |  -- Oliver Cromm in desd

--KsGdsel6WgEHnImy
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--KsGdsel6WgEHnImy--

Date: Thu, 2 Jun 2011 13:06:37 +0100
From: Tony Finch <[email-masked]>
To: Peter Bowen <[email-masked]>
cc: [email-masked]
Subject: Re: LOGIN mechanism
Sender: [email-masked]

Peter Bowen <[email-masked]> wrote:
>
> I've been working on implementing a framework for building SMTP
> servers in ruby (https://github.com/pzb/groat-smtpd).  One of the
> issues I've run into several times is interoperability of the LOGIN
> mechanism with various clients.  The latest problem is certain
> Microsoft SMTP implementations that send an initial client response,
> and have major issues when it is not accepted.  Is there any reliable
> specification for the LOGIN mechanism and/or how it is used with SMTP?

My understanding of LOGIN comes from
http://exim.org/exim-html-current/doc/html/spec_html/ch34.html#SECID173

Exim's authentication configuration is rather runic but it should be
fairly clear.

Exim's plaintext authenticator allows clients to use LOGIN as if it were
PLAIN with an empty authorization identity, or to send the username as the
initial response and password as a second response, or to send no initial
response and separate username and password responses.

I have never seen a client that depends on seeing AUTH=LOGIN in the EHLO
service extensions list, nor have I heard of any interop problems caused
by Exim not advertising AUTH=LOGIN.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Rockall, Malin, Hebrides: South 5 to 7, occasionally gale 8 at first in
Rockall and Malin, veering west or northwest 4 or 5, then backing southwest 5
or 6 later. Rough or very rough. Occasional rain. Moderate or good,
occasionally poor.

Date: Wed, 01 Jun 2011 18:44:54 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Peter Bowen <[email-masked]>, [email-masked], "Murray S. Kucherawy" <[email-masked]>, [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

Its says once authorized, you can't issue it again:

Old 2554:

   Restrictions:
     After an AUTH command has successfully completed, no more AUTH
     commands may be issued in the same session.  After a successful
     AUTH command completes, a server MUST reject any further AUTH
     commands with a 503 reply.

     The AUTH command is not permitted during a mail transaction.

Updated to:

   Restrictions:
     After an AUTH command has been successfully completed, no more
     AUTH commands may be issued in the same session.  After a
     successful AUTH command completes, a server MUST reject any
     further AUTH commands with a 503 reply.

     The AUTH command is not permitted during a mail transaction.
     An AUTH command issued during a mail transaction MUST be
     rejected with a 503 reply.

I think we are debating two things: the relationship between the 
authenticated identity and that of the SENDER (MAIL FROM) of the 
message, and how this applications to subsequent same session 
transactions that have completed an initial transaction.

Personally, we need to look at it as what multiple transactions help 
resolve as oppose to doing it separately.  In the latter, each (AUTH) 
can be different, but if thats the case, than these individual 
messages should not be viewed as under the same group.

Practically, targets have properties of smart hosting, port, tls and 
authentication requirements.

So to me, this is just am implementation queuing consolidation 
heuristic design issue.

John C Klensin wrote:
> Yes, exactly.
> 
> And the "only one AUTH per session" condition (which you
> describe as "nly allowed before the first MAIL in the
> session"... well, while 4954 could have imposed that condition,
> I don't think it does (and, IMO, it would have been be really
> bad design if it had)
> 
>     john
> 
> 
> --On Wednesday, June 01, 2011 17:24 -0400 Peter Bowen
> <[email-masked]> wrote:
> 
>> On Wed, Jun 1, 2011 at 15:39, John C Klensin
>> <[email-masked]> wrote:
>>> On Wednesday, June 01, 2011 12:09 -0400 Peter Bowen
>>> <[email-masked]> wrote:
>>>
>>>> But this would be allowed, correct?
>>> No, it is not. Â See below.
>>>
>>>> (Assuming no previous AUTH was sent)
>>>>
>>>> C: MAIL FROM:<whoever@wherever>
>>>> S: 250 Go ahead, you unauthenticated thing you
>>>> C: RCPT TO:<highmucketymuck@wherever>
>>>> S: 559 Authorization failure, need to authenticate to send to
>>>> Â  Â  this address
>>>> C: RSET
>>>> S: 250 State reset
>>>> C: AUTH blah blah blah
>>>> S: 250 OK, you're authorized now
>>>> C: RCPT TO:<highmucketymuck@wherever>
>>>> S: 250 OK, you're allowed to do that now
>>>>
>>>> Once the RSET is issued, I would think the state it is not
>>>> "during a mail transaction".
>>> That is correct: unless some extension changes it, mail
>>> transactions start with a MAIL command. Â Since RSET resets
>>> any mail transaction that happens to be in progress, the
>>> sequence above ought to produce a "command out of order"
>>> response to the second RCPT command.
>> You are of course correct.  What I intended did not make it
>> into the example.  Corrected:
>>
>> C: MAIL FROM:<whoever@wherever>
>> S: 250 Go ahead, you unauthenticated thing you
>> C: RCPT TO:<highmucketymuck@wherever>
>> S: 559 Authorization failure, need to authenticate to send to
>> this address C: RSET
>> S: 250 State reset
>> C: AUTH blah blah blah
>> S: 250 OK, you're authorized now
>> C: MAIL FROM:<whoever@wherever>
>> S: 250 Go ahead, I know who you are
>> C: RCPT TO:<highmucketymuck@wherever>
>> S: 250 OK, you're allowed to do that now
>>
>> My point is that sending a RSET sets the state back to "not
>> during mail transaction", hence you can AUTH after RSET.  The
>> other alternative is that AUTH is only allowed before the
>> first MAIL in the session.
>>
>> Thanks,
>> Peter
> 
> 
> 
> 
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 01 Jun 2011 17:41:29 -0400
From: John C Klensin <[email-masked]>
To: Peter Bowen <[email-masked]>
cc: [email-masked], "Murray S. Kucherawy" <[email-masked]>, [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

Yes, exactly.

And the "only one AUTH per session" condition (which you
describe as "nly allowed before the first MAIL in the
session"... well, while 4954 could have imposed that condition,
I don't think it does (and, IMO, it would have been be really
bad design if it had)

    john

--On Wednesday, June 01, 2011 17:24 -0400 Peter Bowen
<[email-masked]> wrote:

> On Wed, Jun 1, 2011 at 15:39, John C Klensin
> <[email-masked]> wrote:
>> On Wednesday, June 01, 2011 12:09 -0400 Peter Bowen
>> <[email-masked]> wrote:
>> 
>>> But this would be allowed, correct?
>> 
>> No, it is not.  See below.
>> 
>>> (Assuming no previous AUTH was sent)
>>> 
>>> C: MAIL FROM:<whoever@wherever>
>>> S: 250 Go ahead, you unauthenticated thing you
>>> C: RCPT TO:<highmucketymuck@wherever>
>>> S: 559 Authorization failure, need to authenticate to send to
>>>     this address
>>> C: RSET
>>> S: 250 State reset
>>> C: AUTH blah blah blah
>>> S: 250 OK, you're authorized now
>>> C: RCPT TO:<highmucketymuck@wherever>
>>> S: 250 OK, you're allowed to do that now
>>> 
>>> Once the RSET is issued, I would think the state it is not
>>> "during a mail transaction".
>> 
>> That is correct: unless some extension changes it, mail
>> transactions start with a MAIL command.  Since RSET resets
>> any mail transaction that happens to be in progress, the
>> sequence above ought to produce a "command out of order"
>> response to the second RCPT command.
> 
> You are of course correct.  What I intended did not make it
> into the example.  Corrected:
> 
> C: MAIL FROM:<whoever@wherever>
> S: 250 Go ahead, you unauthenticated thing you
> C: RCPT TO:<highmucketymuck@wherever>
> S: 559 Authorization failure, need to authenticate to send to
> this address C: RSET
> S: 250 State reset
> C: AUTH blah blah blah
> S: 250 OK, you're authorized now
> C: MAIL FROM:<whoever@wherever>
> S: 250 Go ahead, I know who you are
> C: RCPT TO:<highmucketymuck@wherever>
> S: 250 OK, you're allowed to do that now
> 
> My point is that sending a RSET sets the state back to "not
> during mail transaction", hence you can AUTH after RSET.  The
> other alternative is that AUTH is only allowed before the
> first MAIL in the session.
> 
> Thanks,
> Peter

Date: Wed, 01 Jun 2011 17:34:58 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: Peter Bowen <[email-masked]>, [email-masked], "Murray S. Kucherawy" <[email-masked]>, [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

--On Wednesday, June 01, 2011 16:57 -0400 Hector Santos
<[email-masked]> wrote:

> John C Klensin wrote:
> 
>> Again, I believe that 5321 is quite clear on this.  If someone
>> believes otherwise, please identify the text that leads you to
>> believe that RCPT following after RSET with no intervening
>> MAIL command is valid so we can try to get it fixed.
> 
> Reading RSET it says "Any stored sender, recipients, and mail
> data MUST be discarded, and all buffers and state tables
> cleared."

Exactly.

Either I'm confused or this is not nearly as complicated as
people are making it.

As far as when AUTH can appear, let's be clear that is a matter
of the specification for AUTH in RFC 4954 (or anywhere else one
can find an active specification for AUTH).  As far as 5321
_alone_ is concerned, it is simply an unrecognized command.
Unrecognized commands can be issued in any state at all and,
with a possible exception between connection opening and EHLO,
always get the same "unrecognized command" response (code 500,
see the last paragraph of Section 4.2.4 in RFC 5321).

>...
> I think we need to clarify "Mail Transaction" where it begins
> and where it ends.  The spec says it starts with MAIL and I
> believe it says it ends with DATA EOD.

Yeah, it does.  It also says that the effect of RSET or of a new
EHLO command is to end any "Mail Transactions" that happen to be
lying around.
 
> So under that basis:
> 
> The following is valid because it hasn't started a mail
> transaction.
> 
>   C: MAIL FROM:<FOO>
>   S: 530 User Not Authorized
>   C: AUTH LOGIN

Yes, but only because of what 4984 says, not 5321.

> The following is invalid because it clears the MAIL FROM:
> 
>   C: RCPT TO:<[email-masked]>
>   S: 530 Authentication required for relaying
>   C: RSET
>   S: 250 OK
>   C: AUTH LOGIN

Assuming there was a MAIL command prior to the RCPT one,
otherwise it is invalid for other reasons (the response to RCPT
should be "503 Bad sequence of commands").  AUTH after RSET but
before MAIL is invalid only because 4894 says so.  5321 doesn't
have a position on the subject.

> The following MAY BE invalid because it has started a mail
> transaction:
> 
>   C: RCPT TO:<[email-masked]>
>   S: 530 Authentication required for relaying
>   C: AUTH LOGIN

Unless your example is incomplete, it is invalid.  RCPT cannot
start a mail transaction.  The mail transaction must be started
by a MAIL command before RCPT appears.  The first paragraph of
Section 3.3 of RFC 5321 is, I believe, exceptionally clear.  If
there was a mail transaction open (i.e., the MAIL command was
successfully issued but not shown in your example), whether the
above is valid or not is a matter for 4954 and...

> and according to 2554/4954, it is not permitted to issue the
> AUTH once a mail transaction has started.

If that is the case (I haven't reviewed 4954 today), then it is
invalid because either there is a mail transaction open, making
AUTH invalid, or there is no mail transaction open, making RCTP
invalid for more fundamental reasons than the authentication
requirement.   I wouldn't claim returning 530 instead of 503 is
definitely a bug, but it is at least bad taste.

>...

    john

Date: Wed, 1 Jun 2011 17:24:05 -0400
Subject: Re: RSET command - possible security loophole
From: Peter Bowen <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked], "Murray S. Kucherawy" <[email-masked]>, [email-masked]
Sender: [email-masked]

On Wed, Jun 1, 2011 at 15:39, John C Klensin <[email-masked]> wrote:
> On Wednesday, June 01, 2011 12:09 -0400 Peter Bowen <[email-masked]> wrote:
>
>> But this would be allowed, correct?
>
> No, it is not.  See below.
>
>> (Assuming no previous AUTH was sent)
>>
>> C: MAIL FROM:<whoever@wherever>
>> S: 250 Go ahead, you unauthenticated thing you
>> C: RCPT TO:<highmucketymuck@wherever>
>> S: 559 Authorization failure, need to authenticate to send to
>>     this address
>> C: RSET
>> S: 250 State reset
>> C: AUTH blah blah blah
>> S: 250 OK, you're authorized now
>> C: RCPT TO:<highmucketymuck@wherever>
>> S: 250 OK, you're allowed to do that now
>>
>> Once the RSET is issued, I would think the state it is not
>> "during a mail transaction".
>
> That is correct: unless some extension changes it, mail
> transactions start with a MAIL command.  Since RSET resets any
> mail transaction that happens to be in progress, the sequence
> above ought to produce a "command out of order" response to the
> second RCPT command.

You are of course correct.  What I intended did not make it into the
example.  Corrected:

C: MAIL FROM:<whoever@wherever>
S: 250 Go ahead, you unauthenticated thing you
C: RCPT TO:<highmucketymuck@wherever>
S: 559 Authorization failure, need to authenticate to send to this address
C: RSET
S: 250 State reset
C: AUTH blah blah blah
S: 250 OK, you're authorized now
C: MAIL FROM:<whoever@wherever>
S: 250 Go ahead, I know who you are
C: RCPT TO:<highmucketymuck@wherever>
S: 250 OK, you're allowed to do that now

My point is that sending a RSET sets the state back to "not during
mail transaction", hence you can AUTH after RSET.  The other
alternative is that AUTH is only allowed before the first MAIL in the
session.

Thanks,
Peter

Date: Wed, 01 Jun 2011 16:57:46 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Peter Bowen <[email-masked]>, [email-masked], "Murray S. Kucherawy" <[email-masked]>, [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

John C Klensin wrote:

> Again, I believe that 5321 is quite clear on this.  If someone
> believes otherwise, please identify the text that leads you to
> believe that RCPT following after RSET with no intervening MAIL
> command is valid so we can try to get it fixed.

Reading RSET it says "Any stored sender, recipients, and mail data 
MUST be discarded, and all buffers and state tables cleared."

Our code RSET() code will clear the sender address and looking at 
Exim's smtp_reset(), it also clears the sender. Looking at Sendmail's 
CLEAR_STATE(cmd) macro which has a more complex readability but I see 
a dropenvelope() call and that seems to also clear envelope information.

This is what I believe what has always been implemented.  I don't 
believe it is a "Mulligan" or "Redo" last state concept.  If this is 
wrong, we need to correct it.

I think we need to clarify "Mail Transaction" where it begins and 
where it ends.  The spec says it starts with MAIL and I believe it 
says it ends with DATA EOD.

So under that basis:

The following is valid because it hasn't started a mail transaction.

  C: MAIL FROM:<FOO>
  S: 530 User Not Authorized
  C: AUTH LOGIN

The following is invalid because it clears the MAIL FROM:

  C: RCPT TO:<[email-masked]>
  S: 530 Authentication required for relaying
  C: RSET
  S: 250 OK
  C: AUTH LOGIN

The following MAY BE invalid because it has started a mail transaction:

  C: RCPT TO:<[email-masked]>
  S: 530 Authentication required for relaying
  C: AUTH LOGIN

and according to 2554/4954, it is not permitted to issue the AUTH once 
a mail transaction has started.

 From a practical standpoint, it make sense because authentication is 
generally needed to be established before RCPT is issued simply to 
determine Local vs Relay authorization at the most fundamental level 
of SMTP operations.

However, this could be semantics based on private port 587 (SUBMIT) vs 
public port 25 operations.

In other words, for us, under SUBMIT, the MAIL FROM: must be an local 
account, under a public port 25, it is not enforced.

However, this was found to be a problem under public port 25 AUTH 
operations where the AUTH identity did not have to be associated with 
the MAIL FROM and the domain part was a locally hosted domain and the 
user-id was illegal.  In other words, if there was bounces, the mail 
when back to LA-LA land.

    S: 220-example-host.com ESMTP Server
    C: EHLO client.example.com
    S: 250-example-host.com Hello client.example.com
    S: 250-AUTH LOGIN
    S: 250 AUTH=LOGIN
    C: AUTH LOGIN
    .....
    S: 235 2.7.0 Authentication successful
    C: MAIL FROM:<[email-masked]>
    S: 250 OK

For bounces, if not the return path is verified, the mail is goes to 
unknown return path.  So what we do is check the user if the return 
path domain is local under all auth, non-auth sessions, like if it a 
4409 logic.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 01 Jun 2011 15:39:11 -0400
From: John C Klensin <[email-masked]>
To: Peter Bowen <[email-masked]>, [email-masked]
cc: "Murray S. Kucherawy" <[email-masked]>, [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

--On Wednesday, June 01, 2011 12:09 -0400 Peter Bowen
<[email-masked]> wrote:

> But this would be allowed, correct?

No, it is not.  See below.

> (Assuming no previous AUTH was sent)
> 
> C: MAIL FROM:<whoever@wherever>
> S: 250 Go ahead, you unauthenticated thing you
> C: RCPT TO:<highmucketymuck@wherever>
> S: 559 Authorization failure, need to authenticate to send to
>     this address 
> C: RSET
> S: 250 State reset
> C: AUTH blah blah blah
> S: 250 OK, you're authorized now
> C: RCPT TO:<highmucketymuck@wherever>
> S: 250 OK, you're allowed to do that now
> 
> Once the RSET is issued, I would think the state it is not
> "during a mail transaction".

That is correct: unless some extension changes it, mail
transactions start with a MAIL command.  Since RSET resets any
mail transaction that happens to be in progress, the sequence
above ought to produce a "command out of order" response to the
second RCPT command.   As far as the server is concerned, no
MAIL command has been issued (because RSET cleared the previous
one), so the second RCPT command, which requires a mail session,
has been issued without one.

Deciding what 5321 commands and state RSET resets is not
discretionary with either the client or the server.  Either one
would cause a nightmare.

Again, I believe that 5321 is quite clear on this.  If someone
believes otherwise, please identify the text that leads you to
believe that RCPT following after RSET with no intervening MAIL
command is valid so we can try to get it fixed.

   best,
    john

From: [email-masked]
Cc: [email-masked], "Murray S. Kucherawy" <[email-masked]>, "[email-masked]" <[email-masked]>
Date: Wed, 01 Jun 2011 11:16:11 -0700 (PDT)
Subject: Re: RSET command - possible security loophole
To: Peter Bowen <[email-masked]>
Sender: [email-masked]

> On Wed, Jun 1, 2011 at 00:05,  <[email-masked]> wrote:
> >> > -----Original Message-----
> >> > From: Ned Freed [mailto:[email-masked]]
> >> > Sent: Tuesday, May 31, 2011 3:29 PM
> >> > To: Murray S. Kucherawy
> >> > Cc: [email-masked]
> >> > Subject: RE: RSET command - possible security loophole
> >> >
> >> > Authentication (AUTH) and session security (STARTTLS) are the obvious examples,
> >> > but it is also very common for implementations to support proprietary
> >> > mechanisms to allow setting of various session attributes (including but not
> >> > limited to the real IP address of the client). Various sorts of proxies are the
> >> > usual use-case for this stuff.
> >
> >> Sorry, right, AUTH and STARTTLS of course.  My point is that once you hit
> >> MAIL and thus begin message-specific operations rather than
> >> connection-specific ones, I can't think of any SMTP commands that go back
> >> and potentially alter session state in any obvious way that I can think of.
> >
> > Goodness, I certainly hope not. We discussed the possibility of allowing
> > someone to authenticate back when SMTP AUTH was first being specified. There
> > were a couple of people who wanted to allow this sort of thing:
> >
> >    C: MAIL FROM:<whoever@wherever>
> >    S: 250 Go ahead, you unauthenticated thing you
> >    C: RCPT TO:<highmucketymuck@wherever>
> >    S: 559 Authorization failure, need to authenatice to send to this address
> >    C: AUTH blah blah blah
> >    S: 250 OK, you're authorized now
> >    C: RCPT TO:<highmucketymuck@wherever>
> >    S: 250 OK, you're allowed to do that now
> >
> > I protested vehemently that the resulting security semantics would be a HUGE
> > mess. Just as one example, supppose the AUTH would have impacted the semantics
> > of the previous part of the transaction in some way, e.g., a recipient address
> > that looks successful but got silently blackholed because of the
> > lack of authorization.

> But this would be allowed, correct? (Assuming no previous AUTH was sent)

> C: MAIL FROM:<whoever@wherever>
> S: 250 Go ahead, you unauthenticated thing you
> C: RCPT TO:<highmucketymuck@wherever>
> S: 559 Authorization failure, need to authenticate to send to this address
> C: RSET
> S: 250 State reset
> C: AUTH blah blah blah
> S: 250 OK, you're authorized now
> C: RCPT TO:<highmucketymuck@wherever>
> S: 250 OK, you're allowed to do that now

Sure, this is fine. I doubt it will happen very often if at all, but it's
fine.

> Once the RSET is issued, I would think the state it is not "during a
> mail transaction".

Yep.

				Ned

Date: Wed, 01 Jun 2011 14:09:33 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Peter Bowen <[email-masked]>
CC: [email-masked], "Murray S. Kucherawy" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

Peter Bowen wrote:
>>>> From: Ned Freed [mailto:[email-masked]]
>> I protested vehemently that the resulting security semantics would be a HUGE
>> mess. Just as one example, supppose the AUTH would have impacted the semantics
>> of the previous part of the transaction in some way, e.g., a recipient address
>> that looks successful but got silently blackholed because of the
>> lack of authorization.
> 
> But this would be allowed, correct? (Assuming no previous AUTH was sent)
> 
> C: MAIL FROM:<whoever@wherever>
> S: 250 Go ahead, you unauthenticated thing you
> C: RCPT TO:<highmucketymuck@wherever>
> S: 559 Authorization failure, need to authenticate to send to this address
> C: RSET
> S: 250 State reset
> C: AUTH blah blah blah
> S: 250 OK, you're authorized now
> C: RCPT TO:<highmucketymuck@wherever>
> S: 250 OK, you're allowed to do that now
> 
> Once the RSET is issued, I would think the state it is not "during a
> mail transaction".
> 
> Thanks,
> Peter

Well, the RSET will clear the MAIL FROM.  You need to basically 
restart the MAIL transaction.  Your point is understood though.

Most of the time, this is more about authenticated relay that the 
sender or user is authorized to do.  So if authorized relay is not 
established before MAIL is issued or at the connection level, there 
needs to be a reset or start at the top again.

RFC2554/RFC4954 both says its not permitted during the mail transaction.

RFC2554:

     The AUTH command is not permitted during a mail transaction.

Fined tuned in 4954:

     The AUTH command is not permitted during a mail transaction.
     An AUTH command issued during a mail transaction MUST be
     rejected with a 503 reply.

I wish I was reviewing RFC4954 because I would of asked for 
clarification, i.e. after RCPT issues a 530.

    530 5.7.0  Authentication required

    This response SHOULD be returned by any command other than AUTH,
    EHLO, HELO, NOOP, RSET, or QUIT when server policy requires
    authentication in order to perform the requested action and
    authentication is not currently in force.

It doesn't not "prohibit" issuing AUTH at MAIL and RCPT.  MAIL make 
sense because an AUTH can be issued after MAIL has a negative response 
(the transaction has not started).  But after MAIL, a 530 negative 
response for RCPT should prohibit a AUTH if the sender/user has not be 
authorized.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 1 Jun 2011 12:09:56 -0400
Subject: Re: RSET command - possible security loophole
From: Peter Bowen <[email-masked]>
To: [email-masked]
Cc: "Murray S. Kucherawy" <[email-masked]>, "[email-masked]" <[email-masked]>
Sender: [email-masked]

On Wed, Jun 1, 2011 at 00:05,  <[email-masked]> wrote:
>> > -----Original Message-----
>> > From: Ned Freed [mailto:[email-masked]]
>> > Sent: Tuesday, May 31, 2011 3:29 PM
>> > To: Murray S. Kucherawy
>> > Cc: [email-masked]
>> > Subject: RE: RSET command - possible security loophole
>> >
>> > Authentication (AUTH) and session security (STARTTLS) are the obvious examples,
>> > but it is also very common for implementations to support proprietary
>> > mechanisms to allow setting of various session attributes (including but not
>> > limited to the real IP address of the client). Various sorts of proxies are the
>> > usual use-case for this stuff.
>
>> Sorry, right, AUTH and STARTTLS of course.  My point is that once you hit
>> MAIL and thus begin message-specific operations rather than
>> connection-specific ones, I can't think of any SMTP commands that go back
>> and potentially alter session state in any obvious way that I can think of.
>
> Goodness, I certainly hope not. We discussed the possibility of allowing
> someone to authenticate back when SMTP AUTH was first being specified. There
> were a couple of people who wanted to allow this sort of thing:
>
>    C: MAIL FROM:<whoever@wherever>
>    S: 250 Go ahead, you unauthenticated thing you
>    C: RCPT TO:<highmucketymuck@wherever>
>    S: 559 Authorization failure, need to authenatice to send to this address
>    C: AUTH blah blah blah
>    S: 250 OK, you're authorized now
>    C: RCPT TO:<highmucketymuck@wherever>
>    S: 250 OK, you're allowed to do that now
>
> I protested vehemently that the resulting security semantics would be a HUGE
> mess. Just as one example, supppose the AUTH would have impacted the semantics
> of the previous part of the transaction in some way, e.g., a recipient address
> that looks successful but got silently blackholed because of the
> lack of authorization.

But this would be allowed, correct? (Assuming no previous AUTH was sent)

C: MAIL FROM:<whoever@wherever>
S: 250 Go ahead, you unauthenticated thing you
C: RCPT TO:<highmucketymuck@wherever>
S: 559 Authorization failure, need to authenticate to send to this address
C: RSET
S: 250 State reset
C: AUTH blah blah blah
S: 250 OK, you're authorized now
C: RCPT TO:<highmucketymuck@wherever>
S: 250 OK, you're allowed to do that now

Once the RSET is issued, I would think the state it is not "during a
mail transaction".

Thanks,
Peter