Subject: Re: Proposal for Adjusted DATA Timeout
From: [email-masked]
To: Paul Smith <[email-masked]>
Cc: Robert A Rosenberg <[email-masked]>, ietf-smtpimcorg SMTP Interest Group <[email-masked]>
Date: Wed, 28 May 2008 14:31:43 -0700 (PDT)
List-Id: <ietf-smtp.imc.org>

> Robert A. Rosenberg wrote:
> > One way to eliminate the duplicates if the SMTP server has accepted
> > and sent the message but the MUA did not hang around to get the
> > acceptance ACK (or did not send the QUIT) is to save the Message-ID of
> > messages that the MUA did this for (I assume the retransmission will
> > have the same MID. If/when it attempts reinjection, compare the
> > supplied MID against the saved list and reject the attempt with a 5xx
> > status code (and text saying "You submitted this already and I
> > accepted and forwarded it"). You can prune the entry after a
> > designated wait time or when the message is resubmitted (the latter
> > possibly before the wait time expiration).

> Our server does something like this - it actually stores a set of data
> of message-id, date, sender & recipient(s) to (a) handle the situation
> where the same message comes in twice, once for one recipient and once
> for another, both with the same message-id, and (b) to try to handle the
> rare (but not impossible) possibility of a duplicate message-id. It
> doesn't reject the message if it's seen it before, but it silently
> accepts it then discards it. (Rejecting the message would cause
> confusion for the sender who might then think the message hadn't got
> through).

Our server has the ability to do this as well, but we're quite a bit more
cautious. We hash the entire transaction, not just some subset of it - there
are just too many corner cases where ids get duplicated, where messages are
very close but nevertheless distinct, and so on.

There are various ways to speed up the hashes by using two level hashes,
or reusing hashes done for other purposes, that I won't bother to get into
here.

The other problem is hash storage. In an environment that handles hundreds of
messages a second or more (the  kind at most risk of timeouts and
resubmissions) keeping all those hashes around for all received messages is
problematic in and of itself. We therefore only bother when there's an
indication that the client actually timed out and gave up - the status write
failed, or the subsequent read failed. A client that then proceeeds to send a
QUIT, waits for a response, then closes the connection, is unlikely to be one
that resends unnecessarily. And this is another case where distinguishing
between initial submission and relay may be important.

This approach has basically eliminated the duplicate message problem for us -
when we're the server. But none of this works when you  don't control the
server. Hector's proposal is mainly for when you have admin control over the
client but not server. After all, if you control both ends, why not just coax
the server into presenting as little latency as possible and then adjust the
client timeouts to deal with whatever is left? (This IMO is why none of the
checkpointing schemss have caught on - if you control both ends you can usually
adjust things os they aren't necessary, if you don't then it doesn't help since
hte guy on the other end is unlikely to implement checkpointing for you.)

> I must say that, personally, I've never come across a situation where a
> 10 minute timeout has caused a problem, but far too many will timeout at
> the end of DATA even at 1 minute, and 5 minutes is not unusual.

We have run into quite a few servers where 10 minutes wasn't enough - usually
wierd gateways of one sort or another. But yes, the issue of clients with
overly short timeouts is much more common.

> Personally, I'd prefer either a 'keep-alive' or an analogue to the NNTP
> 'IHAVE', so the sending MTA can say to the receiving MTA 'I have message
> ID xxxxx' and the receiving MTA can say 'I've already seen that, don't
> bother sending it again'. The latter would be more complex, but would
> reduce resource consumption on the sending MTA.

I think this is worth considering, but I'm not sure it will help as much as you
might think. Leaving aside the DOS attack  potential (you can usually
accomplish much the same thing now with a server that talks R E A L L Y  S L O
W L Y), I worry that the way people will implement this on the server side now
that threads are readily available is by having a background thread that sits
there throwing out these responses. Done badly this could easily lead to
situations where the main thread is stuck in an infinite loop but the keepalive
thread keeps on saying "I'm here".

				Ned

P.S. In the intersts of full disclosure, the main reason why I have never
botherred to write up the approach we use for server-side duplicate suppression
is that Sun has a patent on it: 7,080,123.

Date: Tue, 27 May 2008 17:41:21 -0700 (PDT)
From: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
To: John Levine <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

> >More specifically, there can be legal requirements for all message
> >recipients to be listed in the header. Just putting in the address of
> >a mailing list is unaccepatable because the content of that list can
> >vary over time. Depending on the specific requirements in a given
> >jurisdiction, it may be necessary to expand the list into the header
> >- alternative schemes such as keeping a precide revision history for
> >the list, or accurate logs of message traffic, while technically OK,
> >may not be allosed. (These systems typically also impose draconian
> >restrictions on forwarding, but that's a separate matter.)

> I can understand why each of the recipients might need to know who
> else got the message, but why would the list have to be in the header
> rather than, say, at the end of the message body?

Many reasons: Because if you do that the message content is cluttered up with
stuff that clients cannot handle and most people don't want to see, because
putting stuff in the body turns it into unstructured data that cannot be easily
indexed or otherwise processed, because when there's already a well-defined
place to put something why not use it, and almost certainly because the
applicable rules don't allow it.

More generally, the approach of stuffing structured material into nonstructured
message body areas is something that's been tried several times, including
several attempts to standardize it: Part manifests for systems that don't
support MIME structure information as well as header "overflow" schemes along
the lines of what you're proporsing, and so on. I can think of exactly one case
where there was a good outcome: MIME. And even though MIME went out of its way
not to muck up regular ASCII text messages (unlike header overflow schemes) I
still occasionally get flack for how it was designed.

				Ned

Date: Tue, 27 May 2008 17:23:31 +0100
From: Tony Finch <[email-masked]>
To: Paul Smith <[email-masked]>
cc: "iet >> SMTP Interest Group" <[email-masked]>
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

On Tue, 27 May 2008, Paul Smith wrote:
> Tony Finch wrote:
> >
> > http://www-uxsup.csx.cam.ac.uk/~fanf2/hermes/doc/qsmtp/draft-fanf-smtp-rfc1845bis.html
>
> OK. Does anyone support that though?

Not as far as I know :-)

> It does seem overly complicated. I'd be reluctant to put the effort in to
> support that (unless it became very widely supported) For instance, I'd have
> thought that having to resume partial DATA sessions is pretty much unnecessary
> now.

Servers don't have to implement suport for partial data retransmission.

Part of the motivation behind the draft was mobile users who suffer from
mobility-related dropouts, in which situation partial data retransmission
would be helpful.

I suppose it might be reasonable to allow clients to use TRANSOFF=0
instead of echoing the server's offset, which would make partial
retransmission optional for clients too.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
SOUTH UTSIRE: EASTERLY 4 OR 5, OCCASIONALLY 6. SLIGHT OR MODERATE. FAIR. GOOD.

Date: Tue, 27 May 2008 16:59:43 +0100
From: Paul Smith <[email-masked]>
To: Tony Finch <[email-masked]>, "iet >> SMTP Interest Group" <[email-masked]>
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------070805020202000003060701
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Tony Finch wrote:
> On Tue, 27 May 2008, Paul Smith wrote:
>   
>> Personally, I'd prefer either a 'keep-alive' or an analogue to the NNTP
>> 'IHAVE', so the sending MTA can say to the receiving MTA 'I have message ID
>> xxxxx' and the receiving MTA can say 'I've already seen that, don't bother
>> sending it again'.
>>     
>
> http://www-uxsup.csx.cam.ac.uk/~fanf2/hermes/doc/qsmtp/draft-fanf-smtp-rfc1845bis.html
>   
OK. Does anyone support that though?

It does seem overly complicated. I'd be reluctant to put the effort in 
to support that (unless it became very widely supported) For instance, 
I'd have thought that having to resume partial DATA sessions is pretty 
much unnecessary now. It could have been useful in 1995 when RFC 1845 
was first written, but things have changed.

OTOH, a 'have you seen this message before' extension could be much 
simpler to implement. (It's essentially what we do already with our dupe 
detector, but with explicit command support, so the sender doesn't need 
to resend if it's a dupe).

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

--------------070805020202000003060701
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Tony Finch wrote:
<blockquote
 cite="mid:[email-masked]"
 type="cite">
  <pre wrap="">On Tue, 27 May 2008, Paul Smith wrote:
  </pre>
  <blockquote type="cite">
    <pre wrap="">Personally, I'd prefer either a 'keep-alive' or an analogue to the NNTP
'IHAVE', so the sending MTA can say to the receiving MTA 'I have message ID
xxxxx' and the receiving MTA can say 'I've already seen that, don't bother
sending it again'.
    </pre>
  </blockquote>
  <pre wrap=""><!---->
<a class="moz-txt-link-freetext" href="http://www-uxsup.csx.cam.ac.uk/~fanf2/hermes/doc/qsmtp/draft-fanf-smtp-rfc1845bis.html">http://www-uxsup.csx.cam.ac.uk/~fanf2/hermes/doc/qsmtp/draft-fanf-smtp-rfc1845bis.html</a>
  </pre>
</blockquote>
OK. Does anyone support that though?<br>
<br>
It does seem overly complicated. I'd be reluctant to put the effort in
to support that (unless it became very widely supported) For instance,
I'd have thought that having to resume partial DATA sessions is pretty
much unnecessary now. It could have been useful in 1995 when RFC 1845
was first written, but things have changed. <br>
<br>
OTOH, a 'have you seen this message before' extension could be much
simpler to implement. (It's essentially what we do already with our
dupe detector, but with explicit command support, so the sender doesn't
need to resend if it's a dupe).<br>
<br>
<pre class="moz-signature" cols="72">-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows</pre>
</body>
</html>

--------------070805020202000003060701--

Date: Tue, 27 May 2008 11:15:24 +0100
From: Tony Finch <[email-masked]>
To: Paul Smith <[email-masked]>
cc: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

On Tue, 27 May 2008, Paul Smith wrote:
>
> Personally, I'd prefer either a 'keep-alive' or an analogue to the NNTP
> 'IHAVE', so the sending MTA can say to the receiving MTA 'I have message ID
> xxxxx' and the receiving MTA can say 'I've already seen that, don't bother
> sending it again'.

http://www-uxsup.csx.cam.ac.uk/~fanf2/hermes/doc/qsmtp/draft-fanf-smtp-rfc1845bis.html

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
FITZROY: CYCLONIC OR NORTHERLY 5 TO 7, OCCASIONALLY GALE 8 IN NORTHWEST AT
FIRST, DECREASING 4 OR 5. MODERATE OR ROUGH, OCCASIONALLY VERY ROUGH AT FIRST.
THUNDERY RAIN. MODERATE OR GOOD, OCCASIONALLY POOR.

Date: Tue, 27 May 2008 09:57:42 +0100
From: Paul Smith <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>, "[email-masked] >> SMTP Interest Group"  <[email-masked]>
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

Robert A. Rosenberg wrote:
> One way to eliminate the duplicates if the SMTP server has accepted 
> and sent the message but the MUA did not hang around to get the 
> acceptance ACK (or did not send the QUIT) is to save the Message-ID of 
> messages that the MUA did this for (I assume the retransmission will 
> have the same MID. If/when it attempts reinjection, compare the 
> supplied MID against the saved list and reject the attempt with a 5xx 
> status code (and text saying "You submitted this already and I 
> accepted and forwarded it"). You can prune the entry after a 
> designated wait time or when the message is resubmitted (the latter 
> possibly before the wait time expiration).
>
Our server does something like this - it actually stores a set of data 
of message-id, date, sender & recipient(s) to (a) handle the situation 
where the same message comes in twice, once for one recipient and once 
for another, both with the same message-id, and (b) to try to handle the 
rare (but not impossible) possibility of a duplicate message-id. It 
doesn't reject the message if it's seen it before, but it silently 
accepts it then discards it. (Rejecting the message would cause 
confusion for the sender who might then think the message hadn't got 
through).

I must say that, personally, I've never come across a situation where a 
10 minute timeout has caused a problem, but far too many will timeout at 
the end of DATA even at 1 minute, and 5 minutes is not unusual.

Personally, I'd prefer either a 'keep-alive' or an analogue to the NNTP 
'IHAVE', so the sending MTA can say to the receiving MTA 'I have message 
ID xxxxx' and the receiving MTA can say 'I've already seen that, don't 
bother sending it again'. The latter would be more complex, but would 
reduce resource consumption on the sending MTA.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Mon, 26 May 2008 23:59:04 -0700
From: Bill McQuillan <[email-masked]>
To: SMTP Discussion <[email-masked]>
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

On Mon, 2008-05-26, Robert A. Rosenberg wrote:

> One way to eliminate the duplicates if the SMTP server has accepted 
> and sent the message but the MUA did not hang around to get the 
> acceptance ACK (or did not send the QUIT) is to save the Message-ID 
> of messages that the MUA did this for (I assume the retransmission 
> will have the same MID. If/when it attempts reinjection, compare the 
> supplied MID against the saved list and reject the attempt with a 5xx 
> status code (and text saying "You submitted this already and I 
> accepted and forwarded it"). You can prune the entry after a 
> designated wait time or when the message is resubmitted (the latter 
> possibly before the wait time expiration).

Wouldn't it be better to accept the duplicate message, reply with 250, and
route the message to the bit bucket without any further processing?

This tells the sending MTA that the message "got through" (which it did) so
that it will not retry here or elsewhere and allows the receiving MTA to
short circuit the processing that presumably delayed the original reply and
thus respond quickly enough to the sending MTA so as to be within its
timeout.

-- 
Bill McQuillan <[email-masked]>

Date: Mon, 26 May 2008 22:33:21 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

At 07:58 -0400 on 05/23/2008, Hector Santos wrote about Proposal for 
Adjusted DATA Timeout:

>Hi,
>
>I am wondering if writing a I-D or BCP is worth the effort here and 
>your comments are welcome.
>
>Basically, with the advent of larger emails and the direction of 
>mail sophisticated mail receivers performing DATA pre-response 
>callouts to process the message before determining what the response 
>code will be, there is a greater potential for client timeout 
>issues, duplicate resends and messages and of course, wasteful 
>bandwidths and overheads.
>[SNIP]

One way to eliminate the duplicates if the SMTP server has accepted 
and sent the message but the MUA did not hang around to get the 
acceptance ACK (or did not send the QUIT) is to save the Message-ID 
of messages that the MUA did this for (I assume the retransmission 
will have the same MID. If/when it attempts reinjection, compare the 
supplied MID against the saved list and reject the attempt with a 5xx 
status code (and text saying "You submitted this already and I 
accepted and forwarded it"). You can prune the entry after a 
designated wait time or when the message is resubmitted (the latter 
possibly before the wait time expiration).

Date: Mon, 26 May 2008 22:23:06 -0400
To:  [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

At 08:19 -0700 on 05/26/2008, [email-masked] wrote about 
Re: Proposal for Adjusted DATA Timeout:

>More specifically, there can be legal requirements for all message recipients
>to be listed in the header. Just putting in the address of a mailing list is
>unaccepatable because the content of that list can vary over time. 
>Depending on
>the specific requirements in a given jurisdiction, it may be necessary to
>expand the list into the header - alternative schemes such as 
>keeping a precide
>revision history for the list, or accurate logs of message traffic, while
>technically OK, may not be allosed. (These systems typically also impose
>draconian restrictions on forwarding, but that's a separate matter.)

Depending on if these SMTP Servers are accepting Email from anyone 
who wants to use them and has this "List All Recipients" requirement 
or a restricted set of senders (who can thus use a special MUA), a 
modification of the old "Apparently-To" Header injection bug (which 
happened if the To and Cc were empty and only Bcc/Envelope-Rcpt-To 
supplied the recipients) might work. The SMTP sends a EHLO 
220-X-List-All response and the MUA suppresses the To and Cc list 
(having only a  "To: (List Follows)" header) along with a X-List-All 
command. The SMTP server then injects X-List-All (with Recipient 
addresses) headers into the RECEIVED message's headers. The headers 
are comments anyway and can be created from the Rcpt-To command data 
instead of duplicating the Rcpt-To list in the To/Cc headers sent by 
the MUA.

Date: Mon, 26 May 2008 20:17:17 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC:  [email-masked], [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
To: [email-masked]
Sender: [email-masked]

John Levine wrote:

> But I agree it would be nice to do that in less than 10 minutes.
> 
> As I hardly need tell people here, if you make stuff more idiot proof,
> you'll meet more ingenious idiots.  I'm wondering how many of the over
> 10 minute messages were a) of interest to the people they were sent to
> and b) couldn't have been worked around reasonably easily.

We wouldn't want to make processes take long. Of course not.

But when a vendor offers generic features to "operators" so that they 
can do multiple things, some of which are independent of each other:

     - RBM rule based messaging (filters)
     - SpamAssassin
     - Sieve
     - And we can add hash-heavyweight DKIM to this list

and I'm sure I'm excluded a lot of unknowns, then we are no longer in 
"control" here.

I think 10 minutes is a good enough SWAG, but the issue is not the 10 
minutes.

It is MTAs out there, like verizon.net, gmail.com, who are fixed on 
using 5 minutes across the board for all the SMTP states.

Unless I'm reading the Mozilla Thunderbird MTA source code incorrectly, 
I see a 1 minute socket timeout across the board.  I see settings for 
IMAP but not for the SMTP client.

If we want to address the blow back seriously, then we need to get SMTP 
clients on board to better understand that it isn't the same old days 
where quick mail drops off and post mail processing was the name of the 
game.

In conversation with the customer who highlighted all this, he clearly 
stated the case when he wrote to us:

     "Well, one of my selling points has been that I DON'T HAVE
      size limits. Some people (using AOL, and other free email)
      have been frustrated by this, so I don't have any size limits.
      In the past, I have sent/received 30-40MB files with no trouble.
      However, since I upgraded to the anti-spam version of the
      mailserver, the duplicate email (because the sending server
      doesn't think it was sent OK) has been coming up as a
      serious problem."

(The AVS version added hooks into the DATA state).

But I pointed out that because there is a 10 minute standard, but in 
practice, many use lower like 5, this correlates to must having a size 
limit in order to keep with the current SMTP model.

Finally, yes, in this case, we can speed up the callouts.  He was using 
an simple RBM example we provided and just added lots of rules to it and 
it also wasn't too smart on skipping thousands of base64 lines.  So we 
can handle that and most likely it will be the end of it.

But I still think 5 minutes is too small to be use by am MTA as a 
constant value regardless of the file size it is sending.

At the very least, these MTAs need to step it up to the next limit - the 
  2821 recommendation of 10 minutes when sending multi-megabytes emails.

Short of that, we need a new ESMTP proposal with "Keep Alive" concepts.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: 26 May 2008 20:51:38 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>More specifically, there can be legal requirements for all message
>recipients to be listed in the header. Just putting in the address of
>a mailing list is unaccepatable because the content of that list can
>vary over time. Depending on the specific requirements in a given
>jurisdiction, it may be necessary to expand the list into the header
>- alternative schemes such as keeping a precide revision history for
>the list, or accurate logs of message traffic, while technically OK,
>may not be allosed. (These systems typically also impose draconian
>restrictions on forwarding, but that's a separate matter.)

I can understand why each of the recipients might need to know who
else got the message, but why would the list have to be in the header
rather than, say, at the end of the message body?

R's,
John

Date: 26 May 2008 20:49:31 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>> 1. Obscenities such as multimegabyte-long To fields. If each To address is
>> processed somehow and the processing takes a 0.01 second, that's a limit of
>> only 60,000 addresses in To.
>
>Ouch :-)
>
>To some extent this is a quality of implementation matter. The server has
>a clear time budget for whatever processing it wants to do, so it ought to
>have a sensible way of coping with running out of time.

I'd think that the chances of a message with 60,000 addresses in the To:
line being of interest to the people to whom it was sent would be vanishingly
small, so if the MTA screws up and loses it, that would be a sensible
result.

But I agree it would be nice to do that in less than 10 minutes.

As I hardly need tell people here, if you make stuff more idiot proof,
you'll meet more ingenious idiots.  I'm wondering how many of the over
10 minute messages were a) of interest to the people they were sent to
and b) couldn't have been worked around reasonably easily.

R's,
John

Date: Mon, 26 May 2008 22:28:47 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Cc: Ned Freed <[email-masked]>
Sender: [email-masked]

Ned Freed writes:
> Ahem. While it is true that the common case these days of having huge 
> numbers of addresses in the header is the result of incompetent spam, 
> there are  also situations where legitimate email does this.

Of course. And I'm sure there are cases (very few, but not none) where 
legitimate messages have many thousands of body parts, which might trip 
up servers which do bodypart processing. And so on and so forth.

> Once again, there's a lot more variety out there than you might think.

A misunderstanding perhaps. I'm not saying there are no legitimate 
strange messages. My opinion is that senders of really, really unusual 
messages need to deal with their unusual nature themselves: It's 
unreasonable to ask the rest of the world to test all the odd corners.

Arnt

Date: Mon, 26 May 2008 08:19:07 -0700 (PDT)
From: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> I've had problems even at ten minutes. Two types of cases, both rare and
> easily ignorable.

> 1. Obscenities such as multimegabyte-long To fields. If each To address
> is processed somehow and the processing takes a 0.01 second, that's a
> limit of only 60,000 addresses in To.

> 2. Slow upstream connections from submitting MUAs that start the clock
> when they hand the last byte over to their TCP or SSL implementation.

> I find it difficult to be very excited about either of these. Just
> another inept spammer, just another MUA bug. Shrug.

Ahem. While it is true that the common case these days of having huge numbers
of addresses in the header is the result of incompetent spam, there are  also
situations where legitimate email does this.

More specifically, there can be legal requirements for all message recipients
to be listed in the header. Just putting in the address of a mailing list is
unaccepatable because the content of that list can vary over time. Depending on
the specific requirements in a given jurisdiction, it may be necessary to
expand the list into the header - alternative schemes such as keeping a precide
revision history for the list, or accurate logs of message traffic, while
technically OK, may not be allosed. (These systems typically also impose
draconian restrictions on forwarding, but that's a separate matter.)

Some years back we dealt with several setups where this was being done. In one
of them it was routine to see 40,000 addresses or more in a single header
field. And since this was a gateway situation there was no alternative but to
process them all. It took some doing to get the performance up to acceptable
levels, too.

Once again, there's a lot more variety out there than you might think.

				Ned

Date: Mon, 26 May 2008 10:20:03 +0100
From: Tony Finch <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

On Mon, 26 May 2008, Arnt Gulbrandsen wrote:
>
> 1. Obscenities such as multimegabyte-long To fields. If each To address is
> processed somehow and the processing takes a 0.01 second, that's a limit of
> only 60,000 addresses in To.

Ouch :-)

To some extent this is a quality of implementation matter. The server has
a clear time budget for whatever processing it wants to do, so it ought to
have a sensible way of coping with running out of time.

If the client plays tricks with its timeout then the MTA doesn't have a
clear budget to work to. So I think the idea of adaptive timeouts is
misguided.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
FITZROY: CYCLONIC OR NORTHERLY 5 TO 7, OCCASIONALLY GALE 8 AT FIRST. ROUGH OR
VERY ROUGH. SHOWERS. MODERATE OR GOOD.

Date: Mon, 26 May 2008 10:20:31 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

I've had problems even at ten minutes. Two types of cases, both rare and 
easily ignorable.

1. Obscenities such as multimegabyte-long To fields. If each To address 
is processed somehow and the processing takes a 0.01 second, that's a 
limit of only 60,000 addresses in To.

2. Slow upstream connections from submitting MUAs that start the clock 
when they hand the last byte over to their TCP or SSL implementation.

I find it difficult to be very excited about either of these. Just 
another inept spammer, just another MUA bug. Shrug.

Arnt

Date: Mon, 26 May 2008 08:46:16 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

Hector Santos wrote:
> John C Klensin wrote:
>> --On Saturday, May 24, 2008 5:38 PM -0700 [email-masked] 
>> wrote:
>>> ...
>>> Well, not only do we have an option to increase the client
>>> wait time based on message size
>>> (STATUS_DATA_RECV_PER_BLOCK_TIME), we also have an option to
>>> increase it based on number of recipients
>>> (STATUS_DATA_RECV_PER_ADDR_TIME) and on size*recipient
>>> (STATUS_DATA_RECV_PER_ADDR_PER_BLK_TIME).
>>>
>>> As for their utility, our experience has been that this is one
>>> of those things that's rarely needed, but when it is needed
>>> you REALLY need it, because without it you have to choose
>>> between crap client performance or large messages being
>>> duplicated. Accordingly, I think writing it down somewhere
>>> makes quite a bit of sense.
>>
>> I agree.   I also think an option to increase based on perceived link 
>> performance (e.g., how long TCP ACKs take to arrive if that 
>> information were available from the local stack, which, of course, it 
>> almost never is) would be worthwhile.
> 
> IMO, I don't think its a TCP or COM I/0 bandwidth issue.

However, it could be. SMTP is not interactive, it thus deserves a TOS 
based on minimal cost, where possible.

> With more systems doing dynamic DATA processing (i.e, AVS processing, 
> and soon more DKIM hash calculations of large files), the timeout is now 
> directly proportional factor in the maximum email size allowed per 
> receiver (worker thread/spawned process).
> 
> So we can equate this to:
> 
>   EMS = SPR x (DTT *60)
> 
> where
> 
>    DTT -> minimum expected data termination timeout (per min)
>    SPR -> system data processing rate (bytes per sec)
>    EMS -> email max size (bytes)

Yes, an MTA should take all the time it needs to validate DATA.

IMHO, an MTA can devise better strategies than maximizing throughput. 
In that respect, the last paragraph in section 6.1 is fine for 
authenticated/whitelisted clients only. For spammers, tarpitting makes 
sense; on my system, for one, that adds up to 128 seconds to each 
response.

Date: Mon, 26 May 2008 00:16:46 +0100
From: Tony Finch <[email-masked]>
To: [email-masked]
cc: John Levine <[email-masked]>, [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

On Sun, 25 May 2008, [email-masked] wrote:
>
> > Are you increasing timeouts past 10 minutes, or past some lower default
> > you use for normal mail?
>
> Usually, but not always, the latter.

The only times I have had timeout-related problems is when someone has
told their MTA to use a timeout less than the minimum permitted.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
FISHER: NORTHEAST 5 OR 6. SLIGHT OR MODERATE. FAIR. GOOD.

Date: Sun, 25 May 2008 21:49:23 +0200
From: mouss <[email-masked]>
CC: [email-masked]
Subject: Re: private key BATV is useful
Sender: [email-masked]

mouss wrote:
>
> [snip]
>
> While I am in, the recommendation in
>    http://babel.de/batv.html
> (cited from the implementations link) to use regular expressions based 
> canonical breaks recipient validation and will cause backscatter. 
> which breaks the whole purpose of BATV INMHO.

I was wrong here. I missed the trick to "stack" proxy_filter's.

My appologies.

>
> Also, whatever the scheme is, there is another class of applications 
> to think of. these are applications that use the sender address as an 
> element to determine the "reputation" of a transaction (spamassassin 
> AWL for instance). such apps should be made aware of the various BATV 
> styles.
>
>

Date: Sun, 25 May 2008 10:08:01 -0700 (PDT)
From: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
To: John C Klensin <[email-masked]>
Cc: [email-masked], Hector Santos <[email-masked]>, [email-masked]
Sender: [email-masked]

> --On Saturday, May 24, 2008 5:38 PM -0700
> [email-masked] wrote:

> >...
> > Well, not only do we have an option to increase the client
> > wait time based on message size
> > (STATUS_DATA_RECV_PER_BLOCK_TIME), we also have an option to
> > increase it based on number of recipients
> > (STATUS_DATA_RECV_PER_ADDR_TIME) and on size*recipient
> > (STATUS_DATA_RECV_PER_ADDR_PER_BLK_TIME).
> >
> > As for their utility, our experience has been that this is one
> > of those things that's rarely needed, but when it is needed
> > you REALLY need it, because without it you have to choose
> > between crap client performance or large messages being
> > duplicated. Accordingly, I think writing it down somewhere
> > makes quite a bit of sense.

> I agree.   I also think an option to increase based on perceived
> link performance (e.g., how long TCP ACKs take to arrive if that
> information were available from the local stack, which, of
> course, it almost never is) would be worthwhile.

RIght. As you're undoubtedly aware, the channel concept in our MTA gives us the
ability to adapt to link-specific conditions - all timeout settings can be
made per-channel.

However, since channels are widely used and well understood by our users, it
creates a situation where people are able to make link accomodations without
asking for help. As a result we really have no idea to what extent people are
doing this. These size and recipient accomodation settings, OTOH, are pretty
obscure so people using them tend to ask about them first. In fact if memory
serves we actually had a recent case where having the
STATUS_DATA_RECV_PER_ADDR_PER_BLK_TIME value set too high was causing a problem
for someone.

				Ned

P.S. Of course the irony here is that in some ways the worse we do at defining
and/or documenting such capabilities the more we end up knowing about what
people are actually doing out there. It can really suck when you remove a
feature you thought nobody was using only to find that it was just that you
didn't know people were using it.

Date: Sun, 25 May 2008 10:05:43 -0700 (PDT)
From: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
To: John Levine <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

> >As for their utility, our experience has been that this is one of
> >those things that's rarely needed, but when it is needed you REALLY
> >need it, because without it you have to choose between crap client
> >performance or large messages being duplicated. Accordingly, I think
> >writing it down somewhere makes quite a bit of sense.

> Are you increasing timeouts past 10 minutes, or past some lower default
> you use for normal mail?

Usually, but not always, the latter.

				Ned

Date: Sun, 25 May 2008 12:01:45 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC:  [email-masked], [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

John C Klensin wrote:

> --On Saturday, May 24, 2008 5:38 PM -0700 [email-masked] wrote:
>> ...
>> Well, not only do we have an option to increase the client
>> wait time based on message size
>> (STATUS_DATA_RECV_PER_BLOCK_TIME), we also have an option to
>> increase it based on number of recipients
>> (STATUS_DATA_RECV_PER_ADDR_TIME) and on size*recipient
>> (STATUS_DATA_RECV_PER_ADDR_PER_BLK_TIME).
>>
>> As for their utility, our experience has been that this is one
>> of those things that's rarely needed, but when it is needed
>> you REALLY need it, because without it you have to choose
>> between crap client performance or large messages being
>> duplicated. Accordingly, I think writing it down somewhere
>> makes quite a bit of sense.
> 
> I agree.   I also think an option to increase based on perceived link 
> performance (e.g., how long TCP ACKs take to arrive if that information 
> were available from the local stack, which, of course, it almost never 
> is) would be worthwhile.
> 
> I was just trying to be sure I understood the question.
> 
>     john

IMO, I don't think its a TCP or COM I/0 bandwidth issue.

But rather the local receiver system performance issue has become more 
of a factor today.

Put another way, this DATA Termination Timeout (DTT), whether it is 2, 5 
or even the 10 minutes was never really a factor in the past because of 
the dominant mode of operations - post SMTP mail processing, and the 
presumption by clients of post SMTP operations where a fast mail drop 
off, with a quick 250 acknowledgment and QUIT expectation from the 
client was presumed.  DTT was never a big issue here.

No mas.

With more systems doing dynamic DATA processing (i.e, AVS processing, 
and soon more DKIM hash calculations of large files), the timeout is now 
directly proportional factor in the maximum email size allowed per 
receiver (worker thread/spawned process).

So we can equate this to:

   EMS = SPR x (DTT *60)

where

    DTT -> minimum expected data termination timeout (per min)
    SPR -> system data processing rate (bytes per sec)
    EMS -> email max size (bytes)

SPR will be a system performance factor. COM I/O is not a factor, but 
might include:

    - CPU speed,
    - Hard drive speeds,
    - LAN/WAN networking speeds,
    - # of worker threads

etc, etc.

All I am saying is that these factors or timing considerations was a 
lesser big deal before because of the POST SMTP mentality, or rather the 
client side "presumption" that quick mail drops off is the modus 
operandi regardless of how big the burden with regard to any file size 
they might be sending.

With the increasing direction of dynamic SMTP operations, the final DATA 
termination timeout needs to be re-thought as well as the NO-QUIT-ABORT 
concept, as this affects the successful mail delivery and scalability of 
the dynamic SMTP systems.

Of course, we all have allergies towards big changes in SMTP, so I 
figured the practical thing for now is to highlight the issue and 
"advise" these clients using just a static 5 minutes is too low for all 
payload sizes, including large payloads.  10 minutes may not cover all, 
but I think this might be a practical limit for receivers to use since 
that has been the long time recommendation - not 5 minutes.

Of course, an ideal approach (with an eye towards some future ESMTP 
I-D), would be a "Keep Alive" concept that we discussed in the past.

Thanks

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Sun, 25 May 2008 10:04:51 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
cc: Hector Santos <[email-masked]>, [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

--On Saturday, May 24, 2008 5:38 PM -0700 
[email-masked] wrote:

>...
> Well, not only do we have an option to increase the client
> wait time based on message size
> (STATUS_DATA_RECV_PER_BLOCK_TIME), we also have an option to
> increase it based on number of recipients
> (STATUS_DATA_RECV_PER_ADDR_TIME) and on size*recipient
> (STATUS_DATA_RECV_PER_ADDR_PER_BLK_TIME).
>
> As for their utility, our experience has been that this is one
> of those things that's rarely needed, but when it is needed
> you REALLY need it, because without it you have to choose
> between crap client performance or large messages being
> duplicated. Accordingly, I think writing it down somewhere
> makes quite a bit of sense.

I agree.   I also think an option to increase based on perceived 
link performance (e.g., how long TCP ACKs take to arrive if that 
information were available from the local stack, which, of 
course, it almost never is) would be worthwhile.

I was just trying to be sure I understood the question.

     john

Date: 25 May 2008 11:26:22 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>As for their utility, our experience has been that this is one of
>those things that's rarely needed, but when it is needed you REALLY
>need it, because without it you have to choose between crap client
>performance or large messages being duplicated. Accordingly, I think
>writing it down somewhere makes quite a bit of sense.

Are you increasing timeouts past 10 minutes, or past some lower default
you use for normal mail?

R's,
John

Date: Sat, 24 May 2008 23:37:10 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Proposal for Adjusted DATA Timeout
Cc: [email-masked]
Sender: [email-masked]

Hi Ned,
At 17:38 24-05-2008, [email-masked] wrote:
>As for their utility, our experience has been that this is one of those things
>that's rarely needed, but when it is needed you REALLY need it, 
>because without
>it you have to choose between crap client performance or large messages being

Some sites use a lower timeout than the MTA defaults for the data 
block (i.e. the overall timeout for the data phase).  When a client 
connecting over slow links or having issues on the connection between 
the client and the server tries to send a large message, it hit this 
limit.  As you pointed out, it's a rare case but it can really be needed.

This seems more like something for RFC 1870 as both sides have to 
support the SIZE extension.  Adjusting the data timeout like that can 
also open the way for a denial of service (you already know that :-) ).

Regards,
-sm 

Date: Sat, 24 May 2008 17:38:55 -0700 (PDT)
From: [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
To: John C Klensin <[email-masked]>
Cc: Hector Santos <[email-masked]>, [email-masked]
Sender: [email-masked]

> --On Friday, 23 May, 2008 07:58 -0400 Hector Santos
> <[email-masked]> wrote:

> >
> > Hi,
> >
> > I am wondering if writing a I-D or BCP is worth the effort
> > here and your comments are welcome.
> >
> > Basically, with the advent of larger emails and the direction
> > of mail sophisticated mail receivers performing DATA
> > pre-response callouts to process the message before
> > determining what the response code will be, there is a greater
> > potential for client timeout issues, duplicate resends and
> > messages and of course, wasteful bandwidths and overheads.
> >...

> Hector,

> I'm not quite sure what you are proposing here.  Examples:

> 	* If the client sends SIZE with a big number, it should
> 	wait longer?
	
> 	* If the client knows that it is sending a big payload,
> 	it should wait longer.
	
I'm pretty sure this is what Hector is talking about.

> 	* The server should advertise, presumably as part of a
> 	an extension announcement, "I'm a little slow, so, if
> 	you send a lot of stuff, give me extra time".

> Something else?

> Clearly nothing in SMTP today prevents a client from using a
> longer timeout if it thinks that is justified.  On the other
> hand, for a pair of SMTP implementation connected to the
> Internet with decent broadband connections (or better) 10
> minutes is a very long period of time unless one of them is
> bogged down for other reasons.  So I wonder if longer timeouts
> alone would solve the problem or whether some sort of clunking
> or restart facility would be a better solution.

Well, not only do we have an option to increase the client wait time based on
message size (STATUS_DATA_RECV_PER_BLOCK_TIME), we also have an option to
increase it based on number of recipients (STATUS_DATA_RECV_PER_ADDR_TIME) and
on size*recipient (STATUS_DATA_RECV_PER_ADDR_PER_BLK_TIME).

As for their utility, our experience has been that this is one of those things
that's rarely needed, but when it is needed you REALLY need it, because without
it you have to choose between crap client performance or large messages being
duplicated. Accordingly, I think writing it down somewhere makes quite a bit of
sense.

				Ned

Date: Fri, 23 May 2008 10:24:12 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, [email-masked]
Subject: Re: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

--On Friday, 23 May, 2008 07:58 -0400 Hector Santos
<[email-masked]> wrote:

> 
> Hi,
> 
> I am wondering if writing a I-D or BCP is worth the effort
> here and your comments are welcome.
> 
> Basically, with the advent of larger emails and the direction
> of mail sophisticated mail receivers performing DATA
> pre-response callouts to process the message before
> determining what the response code will be, there is a greater
> potential for client timeout issues, duplicate resends and
> messages and of course, wasteful bandwidths and overheads.
>...

Hector,

I'm not quite sure what you are proposing here.  Examples:

	* If the client sends SIZE with a big number, it should
	wait longer?
	
	* If the client knows that it is sending a big payload,
	it should wait longer.
	
	* The server should advertise, presumably as part of a
	an extension announcement, "I'm a little slow, so, if
	you send a lot of stuff, give me extra time".

Something else?

Clearly nothing in SMTP today prevents a client from using a
longer timeout if it thinks that is justified.  On the other
hand, for a pair of SMTP implementation connected to the
Internet with decent broadband connections (or better) 10
minutes is a very long period of time unless one of them is
bogged down for other reasons.  So I wonder if longer timeouts
alone would solve the problem or whether some sort of clunking
or restart facility would be a better solution.

    john

Date: Fri, 23 May 2008 07:58:18 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To:  [email-masked]
Subject: Proposal for Adjusted DATA Timeout
Sender: [email-masked]

Hi,

I am wondering if writing a I-D or BCP is worth the effort here and your 
comments are welcome.

Basically, with the advent of larger emails and the direction of mail 
sophisticated mail receivers performing DATA pre-response callouts to 
process the message before determining what the response code will be, 
there is a greater potential for client timeout issues, duplicate 
resends and messages and of course, wasteful bandwidths and overheads.

Summary of my proposal:

     Clients should consider adjusting their DATA termination
     state timeout based on the size of the message they are
     sizing and just use a low 5 minutes across the board for
     all payload mail sizes. If the client is not using the
     recommended 10 minute timeout [RFC 2821], it should
     consider possible receiver lengthy processing resend and
     duplicate message issues are increasingly possible, thus
     the client SHOULD adjust the DATA termination timeout
     as follows:

       Use 5 minutes for 5 megabytes or less.
       Use 10 minutes for over 5 megabytes.

     Or use some block transfer rate calculation as it proceeds
     to determine what the timeout will be when complete.

     Overall, Using a constant 5 minutes is TOO low for
     large file transfers. It needs to be adjusted.

Background:

The issue is 100% highlighted in 1998 two page RFC 1047 "DUPLICATE 
MESSAGES AND SMTP".

   INTRODUCTION

   ....

   It may be hard to believe that this problem is the cause of many
   duplicate messages.  Intuitively, one might expect that the time
   spent in the state between the final dot and its accepting 250 reply
   is quite small.  In practice, however, this period is often quite
   long; long enough that timeouts by the sending mailer (or possibly
   network failures) are quite common.  Observations by the author
   suggest that this synchronization problem may be the second leading
   cause of duplicate messages on the Internet (second to mail loops).

   ....

   Many mailers delay responding to the final dot because they are doing
   sophisticated processing of the message, in an attempt to confirm
   that they can deliver the message.

RFC 2821(bis) has a 10 minute recommendation:

    DATA Termination: 10 minutes.

    This is while awaiting the "250 OK" reply.  When the receiver gets
    the final period terminating the message data, it typically
    performs processing to deliver the message to a user mailbox.  A
    spurious timeout at this point would be very wasteful and would
    typically result in delivery of multiple copies of the message,
    since it has been successfully sent and the server has accepted
    responsibility for delivery.  See section 6.1 for additional
    discussion.

The reference to 6.1 states:

    To avoid receiving duplicate messages as the result of timeouts, a
    receiver-SMTP MUST seek to minimize the time required to respond to
    the final <CRLF>.<CRLF> end of data indicator.  See RFC 1047 [28] for
    a discussion of this problem.

which points to RFC 1047.

We ran into the exact issue highlighted in RFC 1047 where a customer has 
a local set of AVS rule based processing policies with lengthy callouts 
after the DATA termination is received and before the response code is 
sent.  In this case, the callouts maxed out 5 minutes because it detects 
the client dropped the connection.

However, our SMTP server continues to send the 250 and because it still 
some RFC 821 behavior, as RFC 1047 described:

    RFC-821 (on page 22) states that unless the receiving mailer is
    completely unable to process a message it should accept the message
    and acknowledge any errors in processing in a separate message or
    messages sent back to the originator of the message.  As a result,
    receiving mailers should be able to acknowledge the final dot as soon
    as the message has been safely put in a non-volatile (e.g., disk)
    queue for further processing.  Fast acceptance of a message does not
    violate RFC-821.

In short, our server issues the 250 and signals the router to process 
the mail, and logs the event for the operator to see.

The acceptance of this message appears to violates 2821 (showing 2921bis 
which is 99% the same with a few changes):

    4.1.1.10.  QUIT (QUIT)

    ...

    The receiver MUST NOT intentionally close the transmission channel
    until it receives and replies to a QUIT command (even if there was an
    error).  The sender MUST NOT intentionally close the transmission
    channel until it sends a QUIT command and SHOULD wait until it
    receives the reply (even if there was an error response to a previous
    command).  If the connection is closed prematurely due to violations
    of the above or system or network failure, the server MUST cancel any
    pending transaction, but not undo any previously completed
    transaction, and generally MUST act as if the command or transaction
    in progress had received a temporary error (i.e., a 4yz response).

    The QUIT command may be issued at any time.  Any current uncompleted
    mail transaction will be aborted.

So we are now debating if this is good or bad.

The customer received the message.  His complaint is that the mail 
client sending large emails are trying again with the same thing 
happening. However, our dupe processor is catching it so it isn't a 
problem of getting new duplicate mail, just a processing overhead problem.

If we followed QUIT to the letter in 2821 even after a successful DATA 
termination was received, and delay the router processing until a QUIT 
is finally issued, otherwise CANCEL the transaction, then the customer 
will never receive the message in the first place. And this was exactly 
what he expressed when we found out it was not canceling the message and 
indicated we might have to fix that:

     Customer comment:

     Well, unfortunately, your proposed fix just might prevent
     ANY large emails rom being received.

     I'd rather get "23" of the same email than "0" of them.
     I really just want 1, of course.

Anyway, the callout issue can be adjusted per customer but I think there 
is a conflict of the standard recommends 10 minutes with "buts" in it.

I can understand older days and many systems used POST SMTP processing 
but we all know they are suffering from major blow back problems. So the 
direction is to apply DATA level callouts to provide dynamic SMTP level 
rejection capabilities.  This is a godsend for our customers and no way 
will be removed.  Better notes will be provided regarding lengthy 
callouts taking more than 5 minutes, but I think we should also teach 
our SMTP clients to adjust to todays changing times.

Even RFC 1047 concludes that mailers should be aware of this and not
just use a low 5 minutes across the board, especially when sending 
larger email payloads which will obviously add processing time at the 
today's aware AVS receivers:

     Finally, some mailers allow remote mailers only a minute or two to
     acknowledge the final dot before timing out and trying again.  Given
     the increasing round-trip times on the Internet, and that some
     processing after the final dot is required, the timeout for reply to
     the final dot should probably be at least 5 minutes and a timeout of
     10 minutes would not be unreasonable.

I can understand how some may thing "10 minutes" is too long, so the 
proposal is to adjust it based on the size of the file being transfered.

Comments?

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
To: John Levine <[email-masked]>
Subject: Re: Existing email address meta-syntax standards, especially GSTN
Date: Thu, 22 May 2008 18:31:09 -0700
Sender: [email-masked]

On May 22, 2008, at 2:57 PM, John Levine wrote:

>> BATV could then rely upon the use of the "-" in Verp to ensure  
>> differentiation.
>
> No such luck.  Existing VERP implementations use all sorts of  
> delimiters, and you can't count on finding a hyphen.

It still seems placing more restrictions on the BATV tag character set  
would be a safer choice.  In addition, perhaps the tag could be  
bracketed by "=" to better ensure uniqueness as well.

=batv1=kiiihhhh=local-part@dom.   (iii could be an index maintained  
per local-part or the entire domain)

The danger seem to be when services within a domain uses something  
like VERP which might become confused with an invalid BATV tag.  The  
concept of having MUAs apply their own BATV tag requires outbound MTAs  
to be robust in recognizing these tags.

-Doug

  

Date: Thu, 22 May 2008 17:49:30 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
To: Tony Hansen <[email-masked]>
Cc: Robert A Rosenberg <[email-masked]>, [email-masked]
Sender: [email-masked]

> A Bcc header *can* be included in the version of the message going to
> the bcc'd user. *Some* MUAs split the message into multiple versions:
> the ones going to the To's/Cc's do not have the Bcc: header, and the
> ones going to each of the Bcc's have just that user listed. Other MUAs
> choose a simpler method of just stripping Bcc: entirely and sending the
> same version to all users.

And one of the wins of the bcc header appraoch is that it works with vacation
list checks. The one-copy-for-all approach does not.

				Ned

Date: 22 May 2008 21:57:24 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: Existing email address meta-syntax standards, especially GSTN
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>BATV could then rely upon the use of the "-" in Verp to ensure  
>differentiation.

No such luck.  Existing VERP implementations use all sorts of
delimiters, and you can't count on finding a hyphen.

R's,
John

Date: Thu, 22 May 2008 22:57:49 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--jITzwD3HDGXid3BE
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2008-05-20 07:49:59 -0700, [email-masked] wrote:
> P.S. One way to finesse this would be to have the submission server echo =
the
> changed address in  response to a MAIL FROM. That way a client could get =
its
> address "signed" by starting a transaction, getting the MAIL FROM respons=
e, and
> issuing a RSET. It's a way to do it with essentially no protocol addition=
s or
> changes.

Incidentally, that's what my batv plugin for qpsmtpd does (not by design
- it's just a side effect of how the address rewriting works):

hrunkner:~ 22:13 165% telnet hrunkner.hjp.at 25
Trying 81.223.91.232...
Connected to hrunkner.hjp.at.
Escape character is '^]'.
220 hrunkner ESMTP qpsmtpd 0.32 ready; send us your mail, but not your spam.
ehlo hrunkner.hjp.at
250-hrunkner Hi hrunkner.hjp.at [81.223.91.232]
250-PIPELINING
250 8BITMIME
mail from:<[email-masked]>
250 <prvs=3D0028e2eca9=[email-masked]>, sender OK - how exciting to g=
et mail from you!
rset
250 OK

	hp

--=20
   _  | Peter J. Holzer    | It took a genius to create [TeX],
|_|_) | Sysadmin WSR       | and it takes a genius to maintain it.
| |   | [email-masked]         | That's not engineering, that's art.
__/   | http://www.hjp.at/ |	-- David Kastrup in comp.text.tex

--jITzwD3HDGXid3BE
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--jITzwD3HDGXid3BE--

Date: Thu, 22 May 2008 16:40:08 -0400
From: Tony Hansen <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>
CC: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
Sender: [email-masked]

A Bcc header *can* be included in the version of the message going to 
the bcc'd user. *Some* MUAs split the message into multiple versions: 
the ones going to the To's/Cc's do not have the Bcc: header, and the 
ones going to each of the Bcc's have just that user listed. Other MUAs 
choose a simpler method of just stripping Bcc: entirely and sending the 
same version to all users.

	Tony Hansen
	[email-masked]

Robert A. Rosenberg wrote:
> Thank you for your reply. I read RFC5230 after I had sent my query so I 
> had seen that info. OTOH, I think the RFC in allowing bcc and/or 
> resent-bcc headers to be considered is in error since these headers 
> should never occur in the transmitted text of the message (the bcc is 
> stripped when the message is originally injected so it is not there and 
> thus I do not know if a resent-bcc can be created) unless something is 
> broken (such as the Apparently-To header injection bug that occurred in 
> the absence of any To/Cc addresses).
> 

Date: Thu, 22 May 2008 22:12:03 +0200
From: "Peter J. Holzer" <[email-masked]>
To: IETF SMTP list <[email-masked]>
Subject: Re: Mailing Lists v. BATV
Mail-Followup-To: IETF SMTP list <[email-masked]>
Sender: [email-masked]

--zOcTNEe3AzgCmdo9
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2008-05-22 15:34:10 -0400, John Leslie wrote:
> Peter J. Holzer <[email-masked]> wrote:
> > I think most people these days use web interface to subscribe to
> > mailing-lists. People probably don't know their current BATV address, so
> > a user will enter [email-masked]' into the web form. He will get
> > the confirmation mail to this address, click on the confirmation url,
> > and get all the mails delivered to this address.
>=20
>    Note that the opt-in confirmation presumably _will_ contain a BATV-
> coded MailFrom.

The user may not ever send a confirmation mail. For example, con
confirmation requests sent by mailman look like this:

| We have received a request from 192.0.2.7 for subscription of your
| email address, "[email-masked]", to the [email-masked]
| mailing list.  To confirm that you want to be added to this mailing
| list, simply reply to this message, keeping the Subject: header
| intact.  Or visit this web page:
|=20
|     http://example.net/mailman/confirm/community/59cf758b185b8c0dc5487b58=
321fc83fbe042ede
|=20
[...]

I am sure many users will confirm by clicking on the URL and not by
replying to the message. So the mailing list software will not see the
BATV-coded MailFrom.

> > So it appears to work fine. Until he actually tries to send mail to the
> > list - the mail comes from prv=3D53638f9=[email-masked], which
> > doesn't match the address he's subscribed with, so it will be rejected.
>=20
>    To tell truth, that's broken.
>=20
>    Requiring a MailFrom you've never seen isn't nearly as reasonable as
> requiring a 2822-From you have seen.

Actually, you haven't seen either a 2821-MailFrom or a 2822-From yet.
What you have seen is a 2821-RcptTo (You know that this works because
the user was able to click on the link in the message).

>    Nonetheless, if we observe such behavior in the wild,

ezmlm is the canonical example. Ned tells us that his mailinglists use
the envelope, too. I don't know if either uses a webbased subscription
mechanism like mailman, but I suspect they do.

> we should at the very least warn about it; and IMHO we should design
> in a workaround.

That's why I mentioned it.

	hp

--=20
   _  | Peter J. Holzer    | It took a genius to create [TeX],
|_|_) | Sysadmin WSR       | and it takes a genius to maintain it.
| |   | [email-masked]         | That's not engineering, that's art.
__/   | http://www.hjp.at/ |	-- David Kastrup in comp.text.tex

--zOcTNEe3AzgCmdo9
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--zOcTNEe3AzgCmdo9--

Date: Thu, 22 May 2008 15:56:34 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: BATV breaks rfc2821bis?
Sender: [email-masked]

At 13:49 -0600 on 05/21/2008, Philip Guenther wrote about Re: BATV 
breaks rfc2821bis?:

>On Wed, 21 May 2008, Robert A. Rosenberg wrote:
>>At 19:11 -0700 on 05/20/2008, [email-masked] wrote about 
>>Re: BATV breaks rfc2821bis?:
>>
>>>FWIW, the ongoing problem we have with vacation messages is the "don't send
>>>unless the recipient's address actually appears in the header" rule.
>>
>>What Header? A mailing list (or BCC'ed) message sent to a user as 
>>the only recipient at that domain, will usually have the user's 
>>address in a "for" clause of a Received Header even though the 
>>address does not occur in a To or Cc Header so it will still occur 
>>(and meet the rule criteria) if Received Headers count.
>
>Nope, the 'for' clause of Received headers is not part of the check. 
>RFC 5230, section 4.5 has the full details, starting with:
>    "Vacation" MUST NOT respond to a message unless the recipient user's
>    email address is in a "To", "Cc", "Bcc", "Resent-To", "Resent-Cc", or
>    "Resent-Bcc" line of the original message.  An email address is
>    considered to belong to the recipient if it is one of:

Thank you for your reply. I read RFC5230 after I had sent my query so 
I had seen that info. OTOH, I think the RFC in allowing bcc and/or 
resent-bcc headers to be considered is in error since these headers 
should never occur in the transmitted text of the message (the bcc is 
stripped when the message is originally injected so it is not there 
and thus I do not know if a resent-bcc can be created) unless 
something is broken (such as the Apparently-To header injection bug 
that occurred in the absence of any To/Cc addresses).

Date: Thu, 22 May 2008 15:34:10 -0400
From: John Leslie <[email-masked]>
To: IETF SMTP list <[email-masked]>
Subject: Mailing Lists v. BATV
Sender: [email-masked]

Peter J. Holzer <[email-masked]> wrote:
>=20
> I think most people these days use web interface to subscribe to
> mailing-lists. People probably don't know their current BATV address, so
> a user will enter [email-masked]' into the web form. He will get
> the confirmation mail to this address, click on the confirmation url,
> and get all the mails delivered to this address.

   Note that the opt-in confirmation presumably _will_ contain a BATV-
coded MailFrom.

> So it appears to work fine. Until he actually tries to send mail to the
> list - the mail comes from prv=3D53638f9=[email-masked], which
> doesn't match the address he's subscribed with, so it will be rejected.

   To tell truth, that's broken.

   Requiring a MailFrom you've never seen isn't nearly as reasonable as
requiring a 2822-From you have seen.

   Nonetheless, if we observe such behavior in the wild, we should at
the very least warn about it; and IMHO we should design in a workaround.

   All of which goes to show why BATV is worth standardizing: it will
work more comfortably with some cooperation between senders and
receivers.

--
John Leslie <[email-masked]>

Date: Thu, 22 May 2008 19:35:06 +0200
From: "Peter J. Holzer" <[email-masked]>
To: IETF SMTP list <[email-masked]>
Subject: Re: Lose the time stamp in BATV
Mail-Followup-To: IETF SMTP list <[email-masked]>
Sender: [email-masked]

--uAgJxtfIS94j9H4T
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2008-05-20 00:15:25 -0400, John R Levine wrote:
> It occurs to me that the majority of BATV's interoperability problems are=
=20
> due to the time stamp.  Let's assume for the moment that we simplify a=20
> BATV signature to something like this:
>=20
>   prv=3DDHHHH=3Dlocal@host
>=20
> The D is still a version number, and the HHH is a hash, but there's no=20
> timestamp.  Now, unless you change your key, which seems pretty rare in=
=20
> practice, a given plain address always turns into the same BATV address.

If you want BATV addresses to be stable over time you may want to use a
different secret for each address. My (admittedly very limited)
experience with blowbacks is that only a few addresses so are affected.
I assume these were harvested from mailing-list archives, address-books,
mailfolders, etc. So it is likely that when you need to switch to the
next value of D because a BATV address is used by a spammer, the next
version of the address will soon be used, too. OTOH, another user of the
same domain won't be affected at all and will want to continue using his
old BATV address.=20

> Now if you have a mailing list manager that recognizes senders by bounce=
=20
> address, it works fine, because each sender always has the same bounce=20
> address.  (The bounce address won't match the From: address, but it is my=
=20
> impression that software that keys on bounce address is unlikely to be=20
> picky about From: addresses.)  There will be a one-time issue when a=20
> sending site starts using BATV, but that's a whole lot less pain than whe=
n=20
> your bounce address changes every day.

I think most people these days use web interface to subscribe to
mailing-lists. People probably don't know their current BATV address, so
a user will enter [email-masked]' into the web form. He will get
the confirmation mail to this address, click on the confirmation url,
and get all the mails delivered to this address. So it appears to work
fine. Until he actually tries to send mail to the list - the mail comes
=66rom prv=3D53638f9=[email-masked], which doesn't match the address
he's subscribed with, so it will be rejected. (Most mailinglists allow
additional poster addresses. But users often have trouble doing this
right and mailing-list admins vary in their willingness to accommodate
users).

> What you lose is the ability to reject bounces to mail sent a long time=
=20
> ago.

Rather: Reject bounces to mail sent with a sender address which was
harvested a long time ago.=20

	hp

--=20
   _  | Peter J. Holzer    | It took a genius to create [TeX],
|_|_) | Sysadmin WSR       | and it takes a genius to maintain it.
| |   | [email-masked]         | That's not engineering, that's art.
__/   | http://www.hjp.at/ |	-- David Kastrup in comp.text.tex

--uAgJxtfIS94j9H4T
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--uAgJxtfIS94j9H4T--

Cc: [email-masked], [email-masked]
From: Douglas Otis <[email-masked]>
To: Douglas Otis <[email-masked]>
Subject: Re: Existing email address meta-syntax standards, especially GSTN
Date: Thu, 22 May 2008 10:07:15 -0700
Sender: [email-masked]

On May 21, 2008, at 6:46 PM, Douglas Otis wrote:
> RFC 3340,1,3 apex=
>
> RFC 3191 [/] SMS= (example)
> RFC 3192 FAX=
> RFC 3804 VPIM=, VOICE=, AMIS=
>
> http://www.openspf.org/SRS
>
> (original paper used "#" instead of "=" and example uses "+")
>
> http://www.libsrs2.org/srs/srs.pdf
> SRS0=
> SRS1=
>
> http://cr.yp.to/proto/verp.txt
>
> Where "=" replaces "@" for local-part cascaded addresses.
>
> list-lp"-"subscrib-lp"="subscrib-dom@list-dom
>
> This means verp overlaps with "tag=" convention.

By changing the BATV tag ABNF from:

tag-type         = 1*( DIGIT / ALPHA / "-" )

to:

tag-type         = ALPHA *( ALPHA / DIGIT )

BATV could then rely upon the use of the "-" in Verp to ensure  
differentiation.

By omitting the timer, a need to standardize algorithms is driven  
solely by the need to ensure compatible tag generation.  When only a  
secret change permits the generation of tags, some of the timer digits  
should be allocated as selectors.

One implementation might be:

"pih="[pass-phrase:1][index:3][hash:4]"="

Where index is incremented when abuse is detected.

Another might be:

"pwh="[pass-phrase:1][week:2][hash:4]"=".

week = (((epoch-sec + hash(pass-phrase) % 604800) % 31449600) / 604800;

hash = hash(pass-phrase + email-address + week + tag) % 65535;

The week interval would include an induced error to distribute the  
impact the change may make.

This would tend to automate dealing with abuse, rather than requiring  
a response.

-Doug

Date: Thu, 22 May 2008 13:40:33 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
Sender: [email-masked]

John Leslie wrote:
> [email-masked] <[email-masked]> wrote:
>> To: Alessandro Vesely <[email-masked]>
>> 
>>> Section 4.1.1.2 additionally states that "The reverse-path consists of
>>> the sender mailbox", not a variation thereof. That wording apparently
>>> bans using time-varying tags, unless we reinterpret BATV as a
>>> redistribution service for ephemeral ad-hoc lists, in the sense of
>>> section 3.9.2 (but beware poor subscription policies.) A rather
>>> cumbersome way to standardize things.
>> 
>> OK, I have to admit I hadn't thought of or noticed this conflict. As a
>> practical matter there are already an abundance of schemes that violated
>> the letter, if not the intent, of this language: SRS and VERP immediately
>> come to mind. I therefore wonder if this isn't something we ought to
>> consider "relaxing" in 2821bis.

Or 2821ter, as Frank suggested. However, please don't mistake me for a 
foxy Faust if I say I'd like to be still alive the day it comes :-)

>    Though it's still possible to do substantive changes to 2821bis, since
> the IESG hasn't acted on it, I'm not sure there's any need. After all,
> "mailbox" means whatever 2821bis chooses to mean by it, and that would
> seem to be defined in 2.3.11:
> " 
> " As used in this specification, an "address" is a character string
> " that identifies a user to whom mail will be sent or a location into
> " which mail will be deposited.  The term "mailbox" refers to that
> " depository.  The two terms are typically used interchangeably unless
> " the distinction between the location in which mail is placed (the
> " mailbox) and a reference to it (the address) is important.  An
> " address normally consists of user and domain specifications.  The
> " standard mailbox naming convention is defined to be
> " "local-part@domain";

The above quote is general enough, and clarifies some important concepts.

> " contemporary usage permits a much broader set of
> " applications than simple "user names".  Consequently, and due to a
> " long history of problems when intermediate hosts have attempted to
> " optimize transport by modifying them, the local-part MUST be
> " interpreted and assigned semantics only by the host specified in the
> " domain part of the address.

The latter part provides room for local mangling, but does not help 
devising ways to "drop messages with invalid return addresses", as 
mentioned in section 6.2. If it is possible to explicitly encode 
validity information into a local part, it should be legal for a 
remote host to take advantage of it. Thus, it should be up to the BATV 
(or analogous) specifications to state who may do what with any meta 
information encoded in the local part. The scope of the last quoted 
sentence could be restricted by appending, say ", unless otherwise 
stated by the relevant SMTP extension." However, for that to make 
sense, it is necessary to mention that extensions can also be encoded 
in mail addresses, besides additional SMTP commands or parameters.

It might be something similar to, e.g.,

    Local-part     = Dot-string / Quoted-string / Ext-string
    Ext-string     = Ext-prefix 1*("=" Atom) "=" Local-part
    Ext-prefix     = 1*Let-dig *Atom
    Atom           = 1*(atext except "=")

That would allow a whole new tide of extensions, in addition to 
traditional encodings such as VERP, which is alluded to, but not 
mentioned, in section 3.9.2. In this case, interpreting the local part 
may be useful to avoid sending multiple copies of the same message to 
the same MTA, as described e.g. in the expired 
http://tools.ietf.org/draft/draft-varshavchik-verp-smtpext/

As for section 4.1.1.2, in the phrase "the reverse-path consists of 
the sender mailbox" I'd use a different verb, e.g. "involves", 
"depends on", "belongs to", that retains the intended semantics 
without mandating formal sameness. It may help legal concerns to state 
that the reverse-path is owned by the sender.

The historical note at the end of section 4.4 says that "The reverse 
path address (as copied into the Return-path) MUST be used as the 
target of any mail containing delivery error messages." Perhaps that 
could be extended saying that the return-path should be used (only?) 
for messages concerning the delivery of the message, in order to 
account for vacation messages and similar stuff.

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
To: [email-masked]
Subject: Re: Existing email address meta-syntax standards, especially GSTN
Date: Wed, 21 May 2008 18:46:02 -0700
Sender: [email-masked]

On May 21, 2008, at 11:04 AM, Dave Crocker wrote:

> Folks,
>
> It is worth mentioning that there already are mutliple meta-syntax  
> standards for email local-part strings.
>
> While I have a vague memory of there being more than two, the ones I  
> know about in detail are:
>
>  RFC 3191, Minimal GSTN address format in Internet Mail

RFC 3340,1,3 apex=

RFC 3191 [/] SMS= (example)
RFC 3192 FAX=
RFC 3804 VPIM=, VOICE=, AMIS=

>  RFC 3801, Voice Profile for Internet Mail - version 2 (VPIMv2)

???

http://cr.yp.to/proto/verp.txt

Where "=" replaces "@" for local-part cascaded addresses.

list-lp"-"subscrib-lp"="subscrib-dom@list-dom

This means verp overlaps with "tag=" convention.

FWIW:

http://www.openspf.org/SRS

(original paper used "#" instead of "=" and example uses "+")

http://www.libsrs2.org/srs/srs.pdf
SRS0=
SRS1=

> Please note that the BATV proposal seems to be compatible with  
> RFC3191, in terms of having a registered string followed by an equal  
> sign.
>
> Hmmm.
>
> Perhaps this means we do not need to define a new registry, but  
> merely add values to the IANA GSTN Service-Selector table?
>
> Why create a new table if we don't have to?
>
> For that matter, why debate about a new syntax if we don't have to?...

Agreed.  There is at least some experience in adding meta information  
using the "=".

-Doug

Date: Wed, 21 May 2008 22:56:57 +0200
From: mouss <[email-masked]>
To: John R Levine <[email-masked]>
CC: Philip Guenther <[email-masked]>, IETF SMTP list <[email-masked]>
Subject: Re: vacation programs, was BATV breaks rfc2821bis?
Sender: [email-masked]

John R Levine wrote:
>
>> If you're going to cast doubt on the choice of the envelope sender 
>> address, perhaps you could describe what you think the right choice is.
>
> For a vacation program, Reply-To: or lacking that From:.  You know, 
> like replying to a message in a MUA.

not really. A human can't click 200 times on the reply button in a 
second. an auto-responder can.

> This is not an innovative design.

It seems to me that it is. AFAIK, the reference implementation is the 
"old" bsd vacation program.

>
>> I should mention that the behavior you describe, where the vacation 
>> notice goes to the list management software which then treats it as a 
>> bounce sounds like the Right Thing to me
>
> Aw, come on.  Any vacation program that sends any response at all to 
> list mail is terminally broken.

If it's broken, I still prefer that they send their scatter to the list 
owner (when the user can be unsubscribed or the scatter discarded).

Date: Wed, 21 May 2008 13:15:16 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
Sender: [email-masked]

Philip Guenther wrote:
> Nope, the 'for' clause of Received headers is not part of the check.  

Folk,

How does this sub-thread relate to BATV?

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Wed, 21 May 2008 13:49:11 -0600
From: Philip Guenther <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>
cc: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
Sender: [email-masked]

On Wed, 21 May 2008, Robert A. Rosenberg wrote:
> At 19:11 -0700 on 05/20/2008, [email-masked] wrote about Re: BATV 
> breaks rfc2821bis?:
>
>> FWIW, the ongoing problem we have with vacation messages is the "don't send
>> unless the recipient's address actually appears in the header" rule.
>
> What Header? A mailing list (or BCC'ed) message sent to a user as the only 
> recipient at that domain, will usually have the user's address in a "for" 
> clause of a Received Header even though the address does not occur in a To or 
> Cc Header so it will still occur (and meet the rule criteria) if Received 
> Headers count.

Nope, the 'for' clause of Received headers is not part of the check.  RFC 
5230, section 4.5 has the full details, starting with:
    "Vacation" MUST NOT respond to a message unless the recipient user's
    email address is in a "To", "Cc", "Bcc", "Resent-To", "Resent-Cc", or
    "Resent-Bcc" line of the original message.  An email address is
    considered to belong to the recipient if it is one of:
...

Ned's "ongoing problem" involves the "belong to recipient" part: people 
want to send vacation responses to messages that reached them via 
automatic forwarding from other sites, but the system doing the vacation 
processing needs the user to tell it "this other address is 'me', so go 
ahead and send vacation replies when that address is in the header". 
Sieve's vacation extension gives users a way of doing that; actually
convincing users to fill it in is difficult.

Philip Guenther

Date: Wed, 21 May 2008 14:49:06 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: BATV breaks rfc2821bis?
Sender: [email-masked]

At 19:11 -0700 on 05/20/2008, [email-masked] wrote about 
Re: BATV breaks rfc2821bis?:

>FWIW, the ongoing problem we have with vacation messages is the "don't send
>unless the recipient's address actually appears in the header" rule.

What Header? A mailing list (or BCC'ed) message sent to a user as the 
only recipient at that domain, will usually have the user's address 
in a "for" clause of a Received Header even though the address does 
not occur in a To or Cc Header so it will still occur (and meet the 
rule criteria) if Received Headers count.

Date: Wed, 21 May 2008 11:04:26 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Existing email address meta-syntax standards, especially GSTN
Sender: [email-masked]

Folks,

It is worth mentioning that there already are mutliple meta-syntax standards 
for email local-part strings.

While I have a vague memory of there being more than two, the ones I know 
about in detail are:

   RFC 3191, Minimal GSTN address format in Internet Mail

   RFC 3801, Voice Profile for Internet Mail - version 2 (VPIMv2)

Please note that the BATV proposal seems to be compatible with RFC3191, in 
terms of having a registered string followed by an equal sign.

Hmmm.

Perhaps this means we do not need to define a new registry, but merely add 
values to the IANA GSTN Service-Selector table?

Why create a new table if we don't have to?

For that matter, why debate about a new syntax if we don't have to?...

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Wed, 21 May 2008 09:07:37 -0700 (PDT)
From: [email-masked]
Subject: Re: vacation programs, was BATV breaks rfc2821bis?
To: Philip Guenther <[email-masked]>
Cc: John R Levine <[email-masked]>, IETF SMTP list <[email-masked]>
Sender: [email-masked]

> On Tue, 20 May 2008, John R Levine wrote:
> >> If you're going to cast doubt on the choice of the envelope sender
> >> address, perhaps you could describe what you think the right choice is.
> >
> > For a vacation program, Reply-To: or lacking that From:.  You know, like
> > replying to a message in a MUA.  This is not an innovative design.

> Sure sounds innovative (in the "newly introduced" sense) to me: the
> 'vacation' program shipped with BSD has been using the envelope sender
> originally from the "From " separator line) for the entire 27 years of its
> existence.  The 'formail' program has been generating autoreplies for
> vacation message that go to the envelope sender since at least 1999 (and
> before then it only misrouted messages with Resent-* fields).

Interesting. Back in the early 80s the sample "trip" script for VMS MAIL used
the From: field, but since VMS MAIL only has one originator address field
there wasn't much choice. And no, it didn't work very well.

More to the point... Of course there can be problems no matter what address you
choose. Reply-to: should work pretty well as long as your list detection is
really good - there are still plenty of lists that put the list address in the
Reply-to: field.

From: is another matter. One case where it fails rather badly is when messages
are resent. Resending leaves From: intact but changes envelope from and add
Resent- fields. And since the new recipient is listed in the header the message
will pass those sorts of checks. But if the vacation reply goes to the From:
field it ends up going to the original sender, which is almost always wrong.

Another case where this can get tetchy is some "send on behalf of" scenarios.
In such cases the envelope from is best thing to use - the originator is free
to set the envelope from to match the Sender: if they want to see automatic
responses, or it can be set to agree with the From: if they want the person the
message is being sent on behalf of to get them. There is simply no way for the
recipient to determine which of the From: or Sender: field is right to use.
(This one, incidentially, was the main problem report generator for us back
when we used to use From:.)

				Ned

Date: Tue, 20 May 2008 21:08:40 -0600
From: Philip Guenther <[email-masked]>
To: John R Levine <[email-masked]>
cc: IETF SMTP list <[email-masked]>
Subject: Re: vacation programs, was BATV breaks rfc2821bis?
Sender: [email-masked]

On Tue, 20 May 2008, John R Levine wrote:
>> If you're going to cast doubt on the choice of the envelope sender 
>> address, perhaps you could describe what you think the right choice is.
>
> For a vacation program, Reply-To: or lacking that From:.  You know, like 
> replying to a message in a MUA.  This is not an innovative design.

Sure sounds innovative (in the "newly introduced" sense) to me: the 
'vacation' program shipped with BSD has been using the envelope sender 
originally from the "From " separator line) for the entire 27 years of its 
existence.  The 'formail' program has been generating autoreplies for 
vacation message that go to the envelope sender since at least 1999 (and 
before then it only misrouted messages with Resent-* fields).

>> I should mention that the behavior you describe, where the vacation 
>> notice goes to the list management software which then treats it as a 
>> bounce sounds like the Right Thing to me
>
> Aw, come on.  Any vacation program that sends any response at all to 
> list mail is terminally broken.

Point of order!  *You're* the one that brought up vacation responses going 
to VERP addresses from mailing lists!  If "terminally broken" vacation 
programs are not worthy of consideration then then your *entire message* 
that I replied to was off-topic/out-of-scope!

As is, I no longer understand why you brought this topic up, as you 
apparently don't believe it matters.

Philip Guenther

Date: Tue, 20 May 2008 19:11:55 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
To: John Levine <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

> >As Ned said, this is a major deviation from what an envelope
> >sender address used to be for some decades, and it's IMO not
> >*obvious* how it works in conjunction with RFC 3834 + SIEVE.

> How far back do you want to go?  People have been using subaddresses
> like user+blah@domain since 1991.

Try the mid 1980s instead. Our implementation of subaddresses dates back 1990
or so, but we stole the idea from the Andrew Messaging System, which started
out in 1983. I don't know at what point along the way subaddress support was
added, but it was early enough for us to copy it.

> Mailing lists have been using VERP
> schemes to encode per-message and per-recipient information into the
> bounce address at least since 1998.

But these are semantically very different usages from what's being proposed
here. In  parficular, as Carl pointed out, VERP is used on outgoing list
messages, and bouncing them when they hit another list is something of a
feature.

Subaddresses are a little closer, but even here the primary use is in list
subsccription addresses. Perhaps some people use them to implement a sort of
BATV-like scheme, but I have a hard time believing it is a common practice.

> The advice in 3834 to send vacation responses to the bounce address
> was already dubious in 2004, since by then there was lots of list
> management software that interpreted anything sent back to a VERP
> address as a bounce.  (Yes, I know that a sensible vacation program
> that followed all the advice in 3834 wouldn't respond to list mail at
> all, but as we all know, life is not that tidy.)

Well, all I can say is that back in 2000 or so we inherited some code that sent
these to the header From: address. That was a customer complaint generator.
Once we switched it to use the envelope from, these complaints basically just
stopped. So I guess my question is: If not the envelope from address, then
what?

FWIW, the ongoing problem we have with vacation messages is the "don't send
unless the recipient's address actually appears in the header" rule. This
doesn't  interact well with autoforwarders. We provide a means to amend the
list of addresses a user "owns", but it seems it's too much to ask people to
set this up.

				Ned

Date: Tue, 20 May 2008 22:31:39 -0400 (EDT)
From: John R Levine <[email-masked]>
To: Ned Freed <[email-masked]>
cc: IETF SMTP list <[email-masked]>
Subject: Re: remote signing, was BATV pseudo-Last Call
Cleverness: None detected
Sender: [email-masked]

> The problem with that is in order to solve your problem you're asking a 
> lot of people to make some fairly pervasive changes in how they perform 
> address comparisons.

I agree with you that it's unlikely that people who key stuff on bounce 
addresses* will ever do anything other than a straight string comparison.

That's why I'm taking the timestamp out, so the BATV version of an address 
doesn't change and the string comparisons continue to work.  Yeah, you 
lose replay protection, but I think the merits of the tradeoff between a 
largely hypothetical threat and known collateral damage are obvious.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

* - disregarding, for the moment, the questions about whether that's a 
good idea in the long run independent of BATV

Date: Tue, 20 May 2008 22:21:47 -0400 (EDT)
From: John R Levine <[email-masked]>
To: Philip Guenther <[email-masked]>
cc: IETF SMTP list <[email-masked]>
Subject: Re: vacation programs, was BATV breaks rfc2821bis?
Cleverness: None detected
Sender: [email-masked]

> If you're going to cast doubt on the choice of the envelope sender address, 
> perhaps you could describe what you think the right choice is.

For a vacation program, Reply-To: or lacking that From:.  You know, like 
replying to a message in a MUA.  This is not an innovative design.

> I should mention that the behavior you describe, where the vacation notice 
> goes to the list management software which then treats it as a bounce sounds 
> like the Right Thing to me

Aw, come on.  Any vacation program that sends any response at all to list 
mail is terminally broken.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

Date: Tue, 20 May 2008 19:00:49 -0700 (PDT)
From: [email-masked]
Subject: Re: remote signing, was BATV pseudo-Last Call
To: John R Levine <[email-masked]>
Cc: Ned Freed <[email-masked]>, IETF SMTP list <[email-masked]>, [email-masked]
Sender: [email-masked]

> > As I said previously, you can restrict BATV in this way but I really have to
> > wonder if the resulting very narrowly applicable mechanism is worth the bother.

> Depends how much blowback you get.  When you get as much as I do, it's
> extremely worth the bother.

The problem with that is in order to solve your problem you're asking a lot of
people to make some fairly pervasive changes in how they perform address
comparisons. It's going to take a lot more to sell that upgrade than saying,
"my systems have an unusually big problem with blowback spam". The more general
the mechanism is the more people will use it and more incentive there will be
to support it.

				Ned

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: remote signing
Date:  Wed, 21 May 2008 02:16:07 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 21
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

John R Levine wrote:

>> As I said previously, you can restrict BATV in this way but I really =
have
>> to wonder if the resulting very narrowly applicable mechanism is =
worth the
>> bother.
=20
> Depends how much blowback you get.  When you get as much as I do, it's =

> extremely worth the bother.

Maybe you are talking about different goals, you could publish BATV
as infomational "how to" RFC (for folks who don't find the Wikipedia
article with a link to the draft).  For standards track it needs to=20
be more general.  Our chances to do odd things with local parts are
limited, look at RFC 4646 with its five dimensions language + script
+ region + extension + variant folded into a single language tag,
arguably that is already one step beyond the limit.

 Frank

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: BATV breaks rfc2821bis?
Date:  Wed, 21 May 2008 01:45:33 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 33
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

John Levine wrote:
=20
> How far back do you want to go?  People have been using subaddresses
> like user+blah@domain since 1991.  Mailing lists have been using VERP
> schemes to encode per-message and per-recipient information into the
> bounce address at least since 1998.

Let's limit it to "percent-hack" in RFC 1123, anything older is
really too old. ;-)  I can't tell if VERP qualifies as example,
that is for mailing lists, folks sending out of office vacation
mails to mailing lists are a hopeless case (=3D no argument pro or
con BATV).

> The advice in 3834 to send vacation responses to the bounce=20
> address was already dubious in 2004

Strongly disagree, in fact I'm waiting for a compelling reason
to propose its advancement to DS.  Better than the normative
references in RFC 5230 and sieve-notify-mailto.

> Yes, I know that a sensible vacation program that followed
> all the advice in 3834 wouldn't respond to list mail at all,
> but as we all know, life is not that tidy.

If they ignore RFC 3834 they get what they asked for, I have
"spamcopped" dozens of RFC 3834 violations.  Not counting, but
I always add the 3834 reference manually in these spam reports,
after all it is a relatively new RFC (in relation to say 1123).

RFCs 3834 + 5230 are no showstopppers for BATV, but it needs
more than what you have in the I-D (chapter 5). =20

 Frank

Date: Tue, 20 May 2008 17:21:43 -0600
From: Philip Guenther <[email-masked]>
To: John Levine <[email-masked]>
cc: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
Sender: [email-masked]

On Tue, 20 May 2008, John Levine wrote:
...
> The advice in 3834 to send vacation responses to the bounce address
> was already dubious in 2004, since by then there was lots of list
> management software that interpreted anything sent back to a VERP
> address as a bounce.

If you're going to cast doubt on the choice of the envelope sender 
address, perhaps you could describe what you think the right choice is.

I should mention that the behavior you describe, where the vacation notice 
goes to the list management software which then treats it as a bounce 
sounds like the Right Thing to me, both because the vacation notice is 
*NOT* going to the person who sent the message to the list and because it 
may result in the user being dropped from the list...which may drive 
someone to correct the vacation configuration so that it doesn't respond 
to list messages!

Philip Guenther

Date: Tue, 20 May 2008 18:44:47 -0400 (EDT)
From: John R Levine <[email-masked]>
To: Ned Freed <[email-masked]>
cc: IETF SMTP list <[email-masked]>, [email-masked]
Subject: Re: remote signing, was BATV pseudo-Last Call
Cleverness: None detected
Sender: [email-masked]

> As I said previously, you can restrict BATV in this way but I really have to
> wonder if the resulting very narrowly applicable mechanism is worth the bother.

Depends how much blowback you get.  When you get as much as I do, it's 
extremely worth the bother.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

Date: 20 May 2008 22:37:23 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>As Ned said, this is a major deviation from what an envelope
>sender address used to be for some decades, and it's IMO not
>*obvious* how it works in conjunction with RFC 3834 + SIEVE.

How far back do you want to go?  People have been using subaddresses
like user+blah@domain since 1991.  Mailing lists have been using VERP
schemes to encode per-message and per-recipient information into the
bounce address at least since 1998.

The advice in 3834 to send vacation responses to the bounce address
was already dubious in 2004, since by then there was lots of list
management software that interpreted anything sent back to a VERP
address as a bounce.  (Yes, I know that a sensible vacation program
that followed all the advice in 3834 wouldn't respond to list mail at
all, but as we all know, life is not that tidy.)

R's,
John

Date: Tue, 20 May 2008 15:24:43 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
To: Frank Ellermann <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> Ned wrote:

> > I therefore wonder if this isn't something we ought to consider
> > "relaxing" in 2821bis.

> If BATV really works everywhere let's do this in 2821ter, please.
> I consider 2821bis as "frozen" modulo some DISCUSSes.  A general
> local part tag syntax with a tag registry should not be limited
> to the BATV purposes, it should be open for other applications,
> EAI, SRS, SES, whatever.

And who knows, maybe even standardized subaddressing. OK, I can dream, can't I?

In any case, Frank and I are in total agreement here. We really owe it to
ourselves to get general local part structuring right the first time we put
it in place.

> The SASL RFCs also offer one or two
> very simple mechanisms, and some rules to register more complex
> stuff in separate RFCs.  Obviously following the MIME model...
 
One of MIME's best characteristics IMNSHO.

				Ned

Date: Tue, 20 May 2008 15:16:06 -0700 (PDT)
From: [email-masked]
Subject: Re: remote signing, was BATV pseudo-Last Call
To: John Levine <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

> >> There are likely to be cases within a domain where a client wants to
> >> get his address "signed" with BATV but where you don't want to hand
> >> out the shared secret (or private key for that matter) to the client.

> Sure, but this is hardly a concern unique to BATV.  DKIM has the same
> issue, as I expect does any other content signing scheme.  We didn't
> think that we had to define remote signing methods for DKIM, so it's
> not clear to me why we have to define them here.

Because DKIM is specifically designed to cover message transfer between
administrative domains. It is not a MUA-level facility, occasional
assertions to the contrary notwithstanding.

BATV could also be restricted in a similar way, but (a) The present draft
doesn't say that and I don't know if there is consensus for such a restriction
being imposed and (b) Unlike DKIM, which has to covers the entire message and
therefore gets more problematic the more hops it tries to encompass, there's no
similar issue for BATV because local parts of addresses are pretty well
preserved by the present infrastructure.

As I said previously, you can restrict BATV in this way but I really have to
wonder if the resulting very narrowly applicable mechanism is worth the bother.

				Ned

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: BATV breaks rfc2821bis?
Date:  Tue, 20 May 2008 23:44:51 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 18
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

John Levine wrote

> Again, I don't see how that contstrains the address beyond
> requiring that it accept bounces.

Standard quote:  "the originator as indicated in the reverse
path".  It *identifies* the originator, it's not designed to
be something like a Message-ID.

As Ned said, this is a major deviation from what an envelope
sender address used to be for some decades, and it's IMO not
*obvious* how it works in conjunction with RFC 3834 + SIEVE.

Likely it does, otherwise you'd know these problems already,
but you could have missed obscure issues with the completely
new semantics.
=20
 Frank

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: BATV breaks rfc2821bis?
Date:  Tue, 20 May 2008 23:34:48 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 14
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Ned wrote:

> I therefore wonder if this isn't something we ought to consider
> "relaxing" in 2821bis.

If BATV really works everywhere let's do this in 2821ter, please.
I consider 2821bis as "frozen" modulo some DISCUSSes.  A general
local part tag syntax with a tag registry should not be limited
to the BATV purposes, it should be open for other applications,
EAI, SRS, SES, whatever.  The SASL RFCs also offer one or two
very simple mechanisms, and some rules to register more complex
stuff in separate RFCs.  Obviously following the MIME model...
=20
 Frank

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: BATV pseudo-Last Call
Date:  Tue, 20 May 2008 23:08:06 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 36
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

John Levine wrote:

> First, let's see whether we expect other tagging schemes.
> The more I think about it, the less likely I think it is.

All mail RFCs are adamant that the local part are the local
business of the MON (mail originating network), and various
schemes down to "percent hack" or gmail conventions exist.

Limited to BATV, if I could do it I likely would NOT pick=20
HMAC-SHA1, and for a private scheme I wouldn't waste bytes
for "prvs=3D".

As noted elsewhere in this thread, with a registry of tags
it is not more completely private and tags make sense. =20

E.g. "u=3D" with the unique part (LHS) of the Message-ID can
make sense where a database is no showstopper, same idea as
for NNTP, with a given expiration.

For your scheme you have "modulo 1000 days", so after 1000
days you can definitely forget an expired key - of course
you can forget it earlier, bounces after three years are
ridiculous, but less than say one month might be too short.

For a completely private scheme you could use the tag to
indicate the used key, e.g. "k1=3D" for the first key, "k2=3D"
for your second key if the first was compromised, etc.

For a public scheme you could use the tag to indicate the
algorithm, "prvs=3D" for HMAC-SHA1, "hmac=3D" for HMAC-MD5,
whatever.  Just an idea, copy the registry stuff from the
SASL RFC, no expert review, but "specification required"
to protect IANA, for more examples see the new RFC 5226.

 Frank

Date: Tue, 20 May 2008 12:31:46 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: Douglas Otis <[email-masked]>
Cc: Tony Finch <[email-masked]>, [email-masked]
Sender: [email-masked]

> On May 20, 2008, at 10:42 AM, Tony Finch wrote:

> >
> > On Tue, 20 May 2008, Douglas Otis wrote:
> >>
> >> However, BATV problems and the need for outbound server co-
> >> ordination disappear by implementing the same BATV hash
> >> authentication mechanism within the Message-ID, rather than within
> >> the Return-Path as BATV does.
> >
> > That doesn't work, because a significant proportion of blowback does
> > not contain any of the original message - for example vacation
> > messages. Also, you lose the advantage of cheap envelope-time
> > rejection.

> Although only a SHOULD in RFC3834 section 3.1.6, at least such
> messages would be non-compliant with RFC5230 section 5.8 is a MUST.

Alas, this is what happens when you get to the standards party so late: Vast
amounts of infrastructure get built that don't conform to what you find to be
best practices or requirements. Had vacation responses been specified, oh, say,
20 years ago we might have had a chance to get some measure of compliance. But
it wasn't. In fact the text you cite was only added to this draft pretty much
at the last minute, so implementations done to earlier drafts do not meet this
requirement. (In our own case the majority of out cutomers  are running
versions of our software that don't do this.)

> A
> message not having a return-path also failing to identify the message
> causing the response seems ample justification for its loss.

Not if you expect to be able to receive the majority of the vacation replies
that are generated.

				Ned

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: BATV pseudo-Last Call
Date:  Tue, 20 May 2008 21:41:27 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 35
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Dave Crocker wrote:

> It has been in development for some time, with a useful
> amount of deployment experience.

For some years actually, but I'm not aware of the deployment
experience, does BATV work as good as expected ?  On the
SPF list folks used to talk about "SES" (back in 2004/2005),
a related idea, but of course "talk about" is not the same as
"deployment experience".

> We think it is ready for Proposed Standard

Dunno, ready for publication maybe, but for a PS it would help
if it is compatible with other standards, especially RFC 3834.

There should be far more references, normative and informative,
starting with 2821bis, DSN, MDN, mailing list, and Sieve RFCs.

"Known problems" AFAIK include "does not work very well with
conforming 3834 + Sieve-vacation scripts", the prose should state
this explicitly.

As always I consider HMAC-SHA1 as indication that folks want
some "magic" but are not exactly sure which "magic", HMAC-MD5
should be more than only good enough to get three bytes.

There should be a complete example. As noted by Ned, the length
issue could be a real obstacle, because it is hard to show that it is in
practise *no* issue. The draft could state why ignoring this issue is
justified (= because databases of used local parts are a nightmare).

 Frank

Date: Tue, 20 May 2008 10:59:51 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: Douglas Otis <[email-masked]>
Cc: [email-masked], Tony Finch <[email-masked]>, Ned Freed <[email-masked]>, [email-masked]
Sender: [email-masked]

> On May 20, 2008, at 8:41 AM, [email-masked] wrote:
> >> On Tue, 20 May 2008, Ned Freed wrote:
> >>>> If the client can't use its normal submission server then I don't
> >>>> see what use a message submission protocol extension would be :-)
> >>>
> >>> Firsst of all, I said nothing about not being able to use. There
> >>> are plenty of reasons (speed, policy, separate environment) why I
> >>> might be able to reach one server but prefer or be required to
> >>> actually use another for submission.
> >
> >> I think this points out something important that could perhaps be
> >> made more explicit in the specification. BATV is designed for the
> >> usage model where you must use the domain's submission servers if
> >> you want to send email claiming to be from that domain,
> >
> > I see no reason why BATV should be limied in this way.
> >
> >> and all the submission servers must implement BATV. (It has a lot
> >> in common with DKIM in this respect.)
> >
> > Well, I for one agree that DKIM is limited in this way, but I note
> > in passing that others in the DKIM group don't appear to be
> > convinced. The notion of clients using DKIM directly has come up
> > more than once.

> Agreed.  MUAs _can_ implement BATV through exchanges of outbound
> server _secrets_.  However, BATV problems and the need for outbound
> server co-ordination disappear by implementing the same BATV hash
> authentication mechanism within the Message-ID, rather than within the
> Return-Path as BATV does.

You still have to have agreement on the format plus a way to share the secret.
Not everyone uses the same MUA all the time (in fact with ISPs providing
webmail, IMAP4 and POP3 it has become quite common not to) and sharing MUA
configuration is for all intents and purposes an unsolved problem (akthough not
for lack of trying, i.e., ACAP).

An alternative way to handle this is to not bother with a secret but instead
keep track of the message-ids of every message you've sent in the past, say,
month. This is actually marginally easier in most cases to do - think shared
"Sent" folder, although it still requires upgrading all yourr MUAs.

> If MUAs universally implemented a
> standardized hash authentication mechanism within the Message-ID,
> there would less motivation for attempting to distribute spam via
> backscatter.

OK, I can see how an MUA might be able, in the case of messages with its own
address as the originator, perform this sort of screening. But we're talking
about DSN backscatter here. How is this supposed to work? While the standard
DSN formats are structured in a way that allows extraction of the original
message's message-id, countless other notification formats are used in
practice, each with their own wierd and wonderful characteristics.  Digging
through the message in hopes of finding a matching message-id seems very
dubious to me - if for no other reason that it may not even be present.

Again, if I were going to implement this at the MUA level, I would instead
correlate the message-id with sent mail. Done right that would also make
it possible to link the report with the original message in the UI.

> Moving this authentication mechanism into the MUA would
> also remove the authentication burden from SMTP.

Which IMO is the wrong direction to push it. For most ISPs this is tantamount
to giving up on the problem. Users can and will use all sorts of different
MUAs, including ancient ones. And the history of adoption of these sorts of
features in MUAs has been nothing short of dismal - look at the extent to which
proper localation of DSNs is done, or the correlations I mentioned previously,
or any number of other things.

Moreover, as email service has gotten simultaneously more centralized and more
ubiquitous, the number of MTAs and stores has dropped while the number of MUAs
in use has exploded. (Webmail has acted to reverse this trend in some cases,
but by no means all.) An MUA upgrade may be easier than an MTA upgrade, but
that fails to consider how many of each are out there.

> Reliance upon the
> MUA recognizing its own messages should also substantially increase
> the integrity of SMTP delivery as this should dramatically reduce the
> noise to signal ratio. : )

Sorry, I just don't see that happening. The numbers are against it and user
behavior is against it. But even if it were true, it isn't the problem at hand.

				Ned

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
To: Tony Finch <[email-masked]>
Subject: Re: BATV pseudo-Last Call
Date: Tue, 20 May 2008 11:52:20 -0700
Sender: [email-masked]

On May 20, 2008, at 10:42 AM, Tony Finch wrote:

>
> On Tue, 20 May 2008, Douglas Otis wrote:
>>
>> However, BATV problems and the need for outbound server co- 
>> ordination disappear by implementing the same BATV hash  
>> authentication mechanism within the Message-ID, rather than within  
>> the Return-Path as BATV does.
>
> That doesn't work, because a significant proportion of blowback does  
> not contain any of the original message - for example vacation  
> messages. Also, you lose the advantage of cheap envelope-time  
> rejection.

Although only a SHOULD in RFC3834 section 3.1.6, at least such  
messages would be non-compliant with RFC5230 section 5.8 is a MUST.  A  
message not having a return-path also failing to identify the message  
causing the response seems ample justification for its loss.

>> Moving this authentication mechanism into the MUA would also remove  
>> the authentication burden from SMTP.
>
> That would unnecessarily increase the junk mail load on the message  
> store and MUA<->MS connection.

This abuse is because the path normally lacks protection.  Widely  
adopting a simple authentication scheme should have the effect of  
reducing this burden as well as the authentication process.

>> Reliance upon the MUA recognizing its own messages
>
> Lots of people use more than one MUA - for us it's 15% - 20%.

A BATV like standard for Message-IDs that depends upon a user provided  
pass-phrase would be fairly easy to manage.  When the user desires  
protection, they would simply use a MID-TV enabled MUA and enter their  
pass-phrase.

-Doug

Date: 20 May 2008 18:08:02 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>> Section 4.1.1.2 additionally states that "The reverse-path consists of
>> the sender mailbox", not a variation thereof. That wording apparently
>> bans using time-varying tags ...

>OK, I have to admit I hadn't thought of or noticed this conflict. As
>a practical matter there are already an abundance of schemes that
>violate the letter, if not the intent, of this language: SRS and VERP
>immediately come to mind. I therefore wonder if this isn't something
>we ought to consider "relaxing" in 2821bis.

I don't get how it's a conflict.  It doesn't say that mailboxes have
to be unique or have to be valid forever.  Section 2.3.10 roughly says
that a mailbox is a thing that can receive mail, but it doesn't say
that it has to be able to receive mail a year from now, and it also
doesn't preclude having a thousand different addresses that happen to
deliver to the same place.  I don't see how that contstrains the
address beyond requiring that it can accept a bounce.

In 2821bis the argument to MAIL FROM is a reverse-path which is "the
source mailbox ... which can be used to report errors".  The
definition of mailbox is now section 2.3.11 but the wording is the
same.  Again, I don't see how that contstrains the address beyond
requiring that it accept bounces.

If the bounce address weren't valid at all, or it deliberately stopped
working in 20 minutes, that could reasonably be said not to comply,
but the intent with the BATV timestamps is that the addresses will
remain valid at least for the week during which the message might be
retried and bounces sent.

R's,
John

Date: 20 May 2008 17:45:47 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: remote signing, was BATV pseudo-Last Call
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>> There are likely to be cases within a domain where a client wants to
>> get his address "signed" with BATV but where you don't want to hand
>> out the shared secret (or private key for that matter) to the client.

Sure, but this is hardly a concern unique to BATV.  DKIM has the same
issue, as I expect does any other content signing scheme.  We didn't
think that we had to define remote signing methods for DKIM, so it's
not clear to me why we have to define them here.

R's,
John

>P.S. One way to finesse this would be to have the submission server
>echo the changed address in response to a MAIL FROM. That way a
>client could get its address "signed" by starting a transaction,
>getting the MAIL FROM response, and issuing a RSET. It's a way to do
>it with essentially no protocol additions or changes.

That's very clever, but it also seems contrived.  The client can
connect to the submit server, can log into it (I presume, since doing
this for strangers would be a huge security hole), and can even start
an SMTP session, yet it doesn't send the message that way.  While I'm
not denying that such scenarios exist, they're pretty exotic, and I
wouldn't have any confidence that I understood their issues well
enough to invent a remote signing scheme that would work for enough of
them to matter.

Heck, if you just want a hack, all you need is an e-mail address that
is known to return a DSN.  Send it a message through the submit server
you don't want to use for your real mail, and extract your signed
bounce address from the DSN you get back.

Date: Tue, 20 May 2008 18:42:28 +0100
From: Tony Finch <[email-masked]>
To: Douglas Otis <[email-masked]>
cc: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

On Tue, 20 May 2008, Douglas Otis wrote:
>
> However, BATV problems and the need for outbound server co-ordination
> disappear by implementing the same BATV hash authentication mechanism
> within the Message-ID, rather than within the Return-Path as BATV does.

That doesn't work, because a significant proportion of blowback does not
contain any of the original message - for example vacation messages. Also,
you lose the advantage of cheap envelope-time rejection.

> Moving this authentication mechanism into the MUA would also remove the
> authentication burden from SMTP.

That would unnecessarily increase the junk mail load on the message store
and MUA<->MS connection.

> Reliance upon the MUA recognizing its own messages

Lots of people use more than one MUA - for us it's 15% - 20%.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
BAILEY: SOUTHEAST 5 TO 7, PERHAPS GALE 8 LATER. ROUGH. OCCASIONAL RAIN.
MODERATE OR GOOD.

Date: Tue, 20 May 2008 13:29:19 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
Sender: [email-masked]

There has always been a many to 1 relationship between representations 
of the "sender mailbox" and what might be considered to be the 
"canonical" representation. Consider aliases. Consider subaddresses. 
Consider case differences. Consider limited-use aliases.

Which one goes into the 821.MAILFROM? Has it ever mattered? Does it 
matter if the MTA generates one "on the fly"?

Which is "the sender mailbox"? Just the canonical representation, or all 
of them?

	Tony Hansen
	[email-masked]

Alessandro Vesely wrote:
> 
> Section 4.1.1.2 additionally states that "The reverse-path consists of 
> the sender mailbox", not a variation thereof. That wording apparently 
> bans using time-varying tags, unless we reinterpret BATV as a 
> redistribution service for ephemeral ad-hoc lists, in the sense of 
> section 3.9.2 (but beware poor subscription policies.) A rather 
> cumbersome way to standardize things.

Cc: Tony Finch <[email-masked]>, Ned Freed <[email-masked]>, [email-masked]
From: Douglas Otis <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Date: Tue, 20 May 2008 10:17:14 -0700
Sender: [email-masked]

On May 20, 2008, at 8:41 AM, [email-masked] wrote:
>> On Tue, 20 May 2008, Ned Freed wrote:
>>>> If the client can't use its normal submission server then I don't  
>>>> see what use a message submission protocol extension would be :-)
>>>
>>> Firsst of all, I said nothing about not being able to use. There  
>>> are plenty of reasons (speed, policy, separate environment) why I  
>>> might be able to reach one server but prefer or be required to  
>>> actually use another for submission.
>
>> I think this points out something important that could perhaps be  
>> made more explicit in the specification. BATV is designed for the  
>> usage model where you must use the domain's submission servers if  
>> you want to send email claiming to be from that domain,
>
> I see no reason why BATV should be limied in this way.
>
>> and all the submission servers must implement BATV. (It has a lot  
>> in common with DKIM in this respect.)
>
> Well, I for one agree that DKIM is limited in this way, but I note  
> in passing that others in the DKIM group don't appear to be  
> convinced. The notion of clients using DKIM directly has come up  
> more than once.

Agreed.  MUAs _can_ implement BATV through exchanges of outbound  
server _secrets_.  However, BATV problems and the need for outbound  
server co-ordination disappear by implementing the same BATV hash  
authentication mechanism within the Message-ID, rather than within the  
Return-Path as BATV does.  If MUAs universally implemented a  
standardized hash authentication mechanism within the Message-ID,  
there would less motivation for attempting to distribute spam via  
backscatter.  Moving this authentication mechanism into the MUA would  
also remove the authentication burden from SMTP.  Reliance upon the  
MUA recognizing its own messages should also substantially increase  
the integrity of SMTP delivery as this should dramatically reduce the  
noise to signal ratio. : )

-Doug

Date: Tue, 20 May 2008 13:03:46 -0400
From: John Leslie <[email-masked]>
To: [email-masked]
Cc: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: BATV breaks rfc2821bis?
Sender: [email-masked]

[email-masked] <[email-masked]> wrote:
> To: Alessandro Vesely <[email-masked]>
> 
>> Section 4.1.1.2 additionally states that "The reverse-path consists of
>> the sender mailbox", not a variation thereof. That wording apparently
>> bans using time-varying tags, unless we reinterpret BATV as a
>> redistribution service for ephemeral ad-hoc lists, in the sense of
>> section 3.9.2 (but beware poor subscription policies.) A rather
>> cumbersome way to standardize things.
> 
> OK, I have to admit I hadn't thought of or noticed this conflict. As a
> practical matter there are already an abundance of schemes that violated
> the letter, if not the intent, of this language: SRS and VERP immediately
> come to mind. I therefore wonder if this isn't something we ought to
> consider "relaxing" in 2821bis.

   Though it's still possible to do substantive changes to 2821bis, since
the IESG hasn't acted on it, I'm not sure there's any need. After all,
"mailbox" means whatever 2821bis chooses to mean by it, and that would
seem to be defined in 2.3.11:
" 
" As used in this specification, an "address" is a character string
" that identifies a user to whom mail will be sent or a location into
" which mail will be deposited.  The term "mailbox" refers to that
" depository.  The two terms are typically used interchangeably unless
" the distinction between the location in which mail is placed (the
" mailbox) and a reference to it (the address) is important.  An
" address normally consists of user and domain specifications.  The
" standard mailbox naming convention is defined to be
" "local-part@domain"; contemporary usage permits a much broader set of
" applications than simple "user names".  Consequently, and due to a
" long history of problems when intermediate hosts have attempted to
" optimize transport by modifying them, the local-part MUST be
" interpreted and assigned semantics only by the host specified in the
" domain part of the address.

--
John Leslie <[email-masked]>

Date: Tue, 20 May 2008 09:01:29 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
To: Alessandro Vesely <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> Paul Smith wrote:
> > mouss wrote:
> >>> That's fine, as it doesn't seem that the remote server is supposed to
> >>> gain any benefit. It's the spoofed domain's MTAs which can gain the
> >>> benefit.
> >> but then why standardize the format? anybody can use "internal
> >> aliases" of any form (aka disposable addresses).
> > That's my view as well. A standard return path syntax (eg
> > 'batv=<key>=<orig-local-part>@<domain>') is a good idea as it allows the
> > original local part to be extracted if necessary, but beyond that,
> > there's no point to a standard format for private keys.

> "Extracting the local part" is obviously an interpretation of the
> local-part, which would break section 2.3.11 of 2821bis, as SM
> noticed. I don't recall anything in the standard that deprecates equal
> signs in the local-part.

As I already stated in a previou reply, this document is "breaking" this
section of 2821bis. (I prefer the term "relaxing", is so much more, um,
relaxed.) I have no problem with it doing this and even welcome having this
capability, but it is definitely a big deal and therefore reqires extra careful
consideration.

> Section 4.1.1.2 additionally states that "The reverse-path consists of
> the sender mailbox", not a variation thereof. That wording apparently
> bans using time-varying tags, unless we reinterpret BATV as a
> redistribution service for ephemeral ad-hoc lists, in the sense of
> section 3.9.2 (but beware poor subscription policies.) A rather
> cumbersome way to standardize things.

OK, I have to admit I hadn't thought of or noticed this conflict. As a
practical matter there are already an abundance of schemes that violate the
letter, if not the intent, of this language: SRS and VERP immediately come to
mind. I therefore wonder if this isn't something we ought to consider
"relaxing" in 2821bis.

				Ned

Date: Tue, 20 May 2008 08:41:54 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: Tony Finch <[email-masked]>
Cc: Ned Freed <[email-masked]>, [email-masked]
Sender: [email-masked]

> On Tue, 20 May 2008, Ned Freed wrote:
> >
> > > If the client can't use its normal submission server then I don't see
> > > what use a message submission protocol extension would be :-)
> >
> > Firsst of all, I said nothing about not being able to use. There are
> > plenty of reasons (speed, policy, separate environment) why I might be
> > able to reach one server but prefer or be required to actually use
> > another for submission.

> I think this points out something important that could perhaps be made
> more explicit in the specification. BATV is designed for the usage model
> where you must use the domain's submission servers if you want to send
> email claiming to be from that domain,

I see no reason why BATV should be limied in this way.

> and all the submission servers must
> implement BATV. (It has a lot in common with DKIM in this respect.)

Well, I for one agree that DKIM is limited in this way, but I note in passing
that others in the DKIM group don't appear to be convinced. The notion of 
clients using DKIM directly has come up more than once.

> So if
> you deploy BATV and you have users with configurations that don't conform
> with this model, they'll have to change even if that makes submission
> slower or less convenient. If you deploy BATV and your policies say
> clients on such-and-such a network must use such-and-such a submission
> server, then that submission server had better be configured to correctly
> tag messages for all the relevant domains, or you must adjust your
> policies.

> So I think all the reasons quoted above are, by design, not supported by
> BATV. I also don't think there's any point in adapting BATV to remove this
> limitation unless the same adaptations work for DKIM or origin-domain
> security protocols in general.

I see. So we're intentionally going to cripple BATV so it cannot be used in a
broader set of scenarios it could, with minimal additional work, support?

I'm sorry, but I'm emphatically not on board with that. The focus of this whole
thing is starting to look so narrow and specialized to me, while simultaneouly
requiring such widespread changes to the email infrastructure, that I'm
beginning to think it isn't worth bothering to standardize.

				Ned

Date: Tue, 20 May 2008 08:39:54 -0700
From: "Carl S. Gutekunst" <[email-masked]>
To: Tony Finch <[email-masked]>
CC: Ned Freed <[email-masked]>, [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

Tony Finch wrote:
> I think this points out something important that could perhaps be made
> more explicit in the specification. BATV is designed for the usage model
> where you must use the domain's submission servers if you want to send
> email claiming to be from that domain, and all the submission servers must
> implement BATV.

Seems obvious in the way the spec is written now? It certainly is front 
and center in our office discussions, since we (Postini) charge more for 
using our submission servers than for using our inbound servers alone. 
(Engineering considers that a liability; marketing considers that a 
feature. :-) It gets into the same issues as SPF did -- mobile users, 
sites setting up their own private send servers, etc. Unlike SPF (and 
like DKIM), you do still have the option of deploying BATV to your 
mobile users, private servers, and whatnot -- *if* they are all doing it 
the same way.

<csg>

Date: Tue, 20 May 2008 16:28:18 +0100
From: Tony Finch <[email-masked]>
To: Ned Freed <[email-masked]>
cc: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

On Tue, 20 May 2008, Ned Freed wrote:
>
> > If the client can't use its normal submission server then I don't see
> > what use a message submission protocol extension would be :-)
>
> Firsst of all, I said nothing about not being able to use. There are
> plenty of reasons (speed, policy, separate environment) why I might be
> able to reach one server but prefer or be required to actually use
> another for submission.

I think this points out something important that could perhaps be made
more explicit in the specification. BATV is designed for the usage model
where you must use the domain's submission servers if you want to send
email claiming to be from that domain, and all the submission servers must
implement BATV. (It has a lot in common with DKIM in this respect.) So if
you deploy BATV and you have users with configurations that don't conform
with this model, they'll have to change even if that makes submission
slower or less convenient. If you deploy BATV and your policies say
clients on such-and-such a network must use such-and-such a submission
server, then that submission server had better be configured to correctly
tag messages for all the relevant domains, or you must adjust your
policies.

So I think all the reasons quoted above are, by design, not supported by
BATV. I also don't think there's any point in adapting BATV to remove this
limitation unless the same adaptations work for DKIM or origin-domain
security protocols in general.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
FORTIES: NORTHWESTERLY BECOMING VARIABLE 3 OR 4. MODERATE. FAIR. GOOD.

Date: Tue, 20 May 2008 17:01:04 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: BATV breaks rfc2821bis?
Sender: [email-masked]

Paul Smith wrote:
> mouss wrote:
>>> That's fine, as it doesn't seem that the remote server is supposed to 
>>> gain any benefit. It's the spoofed domain's MTAs which can gain the 
>>> benefit.
>> but then why standardize the format? anybody can use "internal 
>> aliases" of any form (aka disposable addresses).
> That's my view as well. A standard return path syntax (eg 
> 'batv=<key>=<orig-local-part>@<domain>') is a good idea as it allows the 
> original local part to be extracted if necessary, but beyond that, 
> there's no point to a standard format for private keys.

"Extracting the local part" is obviously an interpretation of the 
local-part, which would break section 2.3.11 of 2821bis, as SM 
noticed. I don't recall anything in the standard that deprecates equal 
signs in the local-part.

Section 4.1.1.2 additionally states that "The reverse-path consists of 
the sender mailbox", not a variation thereof. That wording apparently 
bans using time-varying tags, unless we reinterpret BATV as a 
redistribution service for ephemeral ad-hoc lists, in the sense of 
section 3.9.2 (but beware poor subscription policies.) A rather 
cumbersome way to standardize things.

Cc: SMTP Interest Group <[email-masked]>
From: Douglas Otis <[email-masked]>
To: John Levine <[email-masked]>
Subject: Re: public key BATV isn't useful
Date: Tue, 20 May 2008 07:58:50 -0700
Sender: [email-masked]

On May 18, 2008, at 9:40 PM, John Levine wrote:

>
>>> Oh, and one final note. The document talks a bit about defining a  
>>> public key BATV scheme but doesn't actually define anything.
>
> Humor me for a moment here.
>
> The idea of a public key BATV is so the system generating the bounce  
> can check the signature and not even send bogus bounces.  But any  
> system that is going to DKIM sign its bounce addresses would also be
> able to DKIM sign its message bodies, so ADSP discardable already  
> allows you to declare that everything is signed so don't bounce the  
> unsigned stuff.  We're talking about DSNs here, not SMTP rejects, so
> the system generating the bounce is going to have received the  
> message already.  This isn't a situation where you might skip the  
> DATA in a SMTP session.

When ADSP declares messages should be signed, this will be abused with  
messages only appearing to be signed.  After all, "Who would do that?"  
may have been the premise.  In addition, there is not a safe means to  
discover ADSP policy assertions, and when verifiable DKIM signatures  
provide acceptance independent of domain reputation, that too will  
become heavily abused.  Bad actors already induce billions of SMTP  
related transactions against by-stander domains every day.  DKIM and  
ADSP might easily multiply these undesired transactions substantially,  
analogous to machine instead of hand gun drive-bys. : {

Even with elliptic, signature bits available as "spare" in the return- 
path may be problematic.  DKIM's use of a body hash within the  
signature allows cryptographic checks of a DSN, even when the message  
body is altered.  Such checking would require a modified version of  
DKIM verification and careful selection of headers to tease out valid  
signatures with a goal of reducing false positive discards.  It also  
seems a BATV strategy would also be useful at the MUA applied to the  
Message-ID.  Message-ID checking could be done by retaining and  
privately distributing hashes of Message-ID to all receivers.  The  
BATV approach would be simpler to implement since only a pass-phrase  
would be required one-time from a user.

-Doug

Date: Tue, 20 May 2008 07:49:59 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: Tony Finch <[email-masked]>
Cc: Ned Freed <[email-masked]>, [email-masked]
Sender: [email-masked]

> On Tue, 20 May 2008, Ned Freed wrote:

> > > > There are likely to be cases within a domain where a client wants to
> > > > get his address "signed" with BATV but where you don't want to hand
> > > > out the shared secret (or private key for that matter) to the
> > > > client. As such, a missing component here is a on-wire way to ask
> > > > some agent to form this signing operation. The obvious place to have
> > > > this is as an SMTP eubmit extension, although of course other
> > > > approaches are possible.
> >
> > > The usual approach would be to make this implicit by configuring the
> > > client to use a submission server that adds the tag. I'm not sure why
> > > you'd need or want any explicit signalling.
> >
> > That assumes that the only time such an address is needed the goal is
> > always to immediately submit the message for transport using the
> > submission server operated by client's administrative domain. That may
> > not be what I want to do with the message. Just as one example, the
> > client could be remote and could be constained to submit mail to some
> > other server.

> If the client can't use its normal submission server then I don't see what
> use a message submission protocol extension would be :-)

Firsst of all, I said nothing about not being able to use. There are plenty of
reasons (speed, policy, separate environment) why I might be able to reach one
server but prefer or be required to actually use another for submission.

But suppose we are dealing with a case where the server isn't always
accessible. In such a situation a client could go to the server when it is
available, get its address signed, and then cache the result for use when the
server isn't available.

> Since a BATV address is only useful during message transport I'm not sure
> what situations other then message submission you'd want to create one.

The fact that it is useful during transport doesn't mean I always want messages
containinng such address transported immediately and by a given submission
server.

				Ned

P.S. One way to finesse this would be to have the submission server echo the
changed address in  response to a MAIL FROM. That way a client could get its
address "signed" by starting a transaction, getting the MAIL FROM response, and
issuing a RSET. It's a way to do it with essentially no protocol additions or
changes.

Date: Tue, 20 May 2008 15:29:18 +0100
From: Tony Finch <[email-masked]>
To: Ned Freed <[email-masked]>
cc: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

On Tue, 20 May 2008, Ned Freed wrote:

> > > There are likely to be cases within a domain where a client wants to
> > > get his address "signed" with BATV but where you don't want to hand
> > > out the shared secret (or private key for that matter) to the
> > > client. As such, a missing component here is a on-wire way to ask
> > > some agent to form this signing operation. The obvious place to have
> > > this is as an SMTP eubmit extension, although of course other
> > > approaches are possible.
>
> > The usual approach would be to make this implicit by configuring the
> > client to use a submission server that adds the tag. I'm not sure why
> > you'd need or want any explicit signalling.
>
> That assumes that the only time such an address is needed the goal is
> always to immediately submit the message for transport using the
> submission server operated by client's administrative domain. That may
> not be what I want to do with the message. Just as one example, the
> client could be remote and could be constained to submit mail to some
> other server.

If the client can't use its normal submission server then I don't see what
use a message submission protocol extension would be :-)

Since a BATV address is only useful during message transport I'm not sure
what situations other then message submission you'd want to create one.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
BAILEY: SOUTHEAST 5 TO 7. ROUGH. OCCASIONAL RAIN. MODERATE OR GOOD.

Date: Tue, 20 May 2008 07:20:02 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: Tony Finch <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

> On Tue, 20 May 2008, [email-masked] wrote:
> >
> > There are likely to be cases within a domain where a client wants to get
> > his address "signed" with BATV but where you don't want to hand out the
> > shared secret (or private key for that matter) to the client. As such, a
> > missing component here is a on-wire way to ask some agent to form this
> > signing operation. The obvious place to have this is as an SMTP eubmit
> > extension, although of course other approaches are possible.

> The usual approach would be to make this implicit by configuring the
> client to use a submission server that adds the tag. I'm not sure why
> you'd need or want any explicit signalling.

That assumes that the only time such an address is needed the goal is always to
immediately submit the message for transport using the submission server
operated by client's administrative domain. That may not be what I want to  do
with the message. Just as one example, the client could be remote and could be
constained to submit mail to some other server.

				Ned

Date: Tue, 20 May 2008 07:15:14 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
CC: SMTP Interest Group <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: BATV smtp extension
Sender: [email-masked]

[email-masked] wrote:
>      There are likely to be cases within a domain where a client
> wants to get his address "signed" with BATV but where you don't want to
> hand out the shared secret (or private key for that matter) to the client.
> As such, a missing component here is a on-wire way to ask some agent to
> form this signing operation. The obvious place to have this is as an SMTP
> eubmit extension, although of course other approaches are possible.

Sounds like a useful mechanism to pursue, but it is out of scope for this 
specification.

At the least, it is typical to have interactions between components in an ADMD 
specified separately from the core, end2end mechanism.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Tue, 20 May 2008 15:10:22 +0100
From: Tony Finch <[email-masked]>
To: [email-masked]
cc: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

On Tue, 20 May 2008, [email-masked] wrote:
>
> There are likely to be cases within a domain where a client wants to get
> his address "signed" with BATV but where you don't want to hand out the
> shared secret (or private key for that matter) to the client. As such, a
> missing component here is a on-wire way to ask some agent to form this
> signing operation. The obvious place to have this is as an SMTP eubmit
> extension, although of course other approaches are possible.

The usual approach would be to make this implicit by configuring the
client to use a submission server that adds the tag. I'm not sure why
you'd need or want any explicit signalling.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
SOLE: SOUTHEASTERLY 5 TO 7, VEERING WESTERLY 4 OR 5 LATER IN WEST. MODERATE OR
ROUGH. OCCASIONAL RAIN. MODERATE OR GOOD.

Date: Tue, 20 May 2008 06:52:05 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: SM <[email-masked]>
Cc: Alessandro Vesely <[email-masked]>, [email-masked]
Sender: [email-masked]

> At 00:57 20-05-2008, Alessandro Vesely wrote:
> >1) Given that today is 019, the tag exemplified above exhibits an
> >expired day 234, thus a remote server can reject the message if it
> >adopts the policy of not accepting unbounceable messages, even if it
> >cannot verify the signature. (Possibly valid for batv1 only.)

> Quoting Section 2.3.10 of RFC 2821:

>    "Consequently, and due to a long history of problems when intermediate
>     hosts have attempted to optimize transport by modifying them, the
>     local-part MUST be interpreted and assigned semantics only by the host
>     specified in the domain part of the address."

Yep, and this proposal is effectively updating this with the ability for remote
clients to use certain syntax clues to interpret local parts for comparison
purposes. This is why this is a bigger deal than it initially appears and why 
we need to get this right.

				Ned

Date: Tue, 20 May 2008 06:36:58 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: mouss <[email-masked]>
Cc: Paul Smith <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Sender: [email-masked]

> > That's fine, as it doesn't seem that the remote server is supposed to
> > gain any benefit. It's the spoofed domain's MTAs which can gain the
> > benefit.

> but then why standardize the format? anybody can use "internal aliases"
> of any form (aka disposable addresses).

Several reasons. First of all, while the primary purpose of IETF standards is
so that stuff can interoperate properly over the global Internet, a secondary
goal is so that different components built by different implementors can
interoperate within a particular administative domain.

It is quite common for large scale messaging setups to consist of components
built by multiple manufacturers. In fact it isn't uncommon to have setups
incorporating, say, different MTAs from different vendors. Deploying something
like BATV into such a setup requires that everyone be on the same page when
it comes to creating and processing these addresses.

Second, another reason standards exist is to tell people what sorts of
capabilities they may want to consider implementiing, and how to go about it.
Consider the discussion we've just had about including a timestamp in the
format: The potential issue with replay attacks clearly wasn't something some
people had considered. If everyone goes out and does this differently
there are bound to be cases where people get some of the details wrong.

This actually points out another omission from the current specification -
something Chris Newman caught immediately when I mentioned this draft to him
yesterday. There are likely to be cases within a domain where a client wants to
get his address "signed" with BATV but where you don't want to hand out the
shared secret (or private key for that matter) to the client. As such, a
missing component here is a on-wire way to ask some agent to form this signing
operation. The obvious place to have this is as an SMTP eubmit extension,
although of course other approaches are possible.

				Ned

Date: Tue, 20 May 2008 14:30:09 +0100
From: Paul Smith <[email-masked]>
To: mouss <[email-masked]>
CC: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

mouss wrote:
> and if a signature is too short (user-part length limitations), then 
> it won't take long to break.
But there's no NEED to break it.

Since the recipient has no way of knowing if a sender domain supports 
BATV, then a spammer has no need to break the BATV key, unless their AIM 
is to cause back-scatter. Since I don't think that's a spammer's aim 
(except in the case where they really want to launch a DDoS attack), 
then I don't think there would be any incentive to try to break it if 
it's non-trivial.

>> That's fine, as it doesn't seem that the remote server is supposed to 
>> gain any benefit. It's the spoofed domain's MTAs which can gain the 
>> benefit.
> but then why standardize the format? anybody can use "internal 
> aliases" of any form (aka disposable addresses).
That's my view as well. A standard return path syntax (eg 
'batv=<key>=<orig-local-part>@<domain>') is a good idea as it allows the 
original local part to be extracted if necessary, but beyond that, 
there's no point to a standard format for private keys.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Tue, 20 May 2008 13:10:24 +0200
From: mouss <[email-masked]>
To: Paul Smith <[email-masked]>
CC: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

Paul Smith wrote:
>
> [snip]
> Can it? Day 234 is after 019 isn't it? Given that the remote server 
> has no way of knowing how long the sending MTA has specified for 
> allowing bounces, this is an assumption that can't automatically be made.
>
> Also, it would be easy for a spammer to just send a message from
> batv1=1020bibble=[email-masked]
>
> The receiving mail server can't check that 'bibble' isn't a valid 
> signature, so it would accept the message. In any case, a spammer 
> would just send it from [email-masked], as the recipient has no way 
> of knowing that the sender address should be signed.

and if a signature is too short (user-part length limitations), then it 
won't take long to break.
>
> So, the remote server gains absolutely no benefit from BATV. 

Agreed.

> That's fine, as it doesn't seem that the remote server is supposed to 
> gain any benefit. It's the spoofed domain's MTAs which can gain the 
> benefit.

but then why standardize the format? anybody can use "internal aliases" 
of any form (aka disposable addresses).

Date: Tue, 20 May 2008 10:57:25 +0100
From: Paul Smith <[email-masked]>
To: Alexey Melnikov <[email-masked]>
CC: Tony Finch <[email-masked]>, "ietf >> SMTP Interest Group" <[email-masked]>
Subject: Re: SMTP metadata transfer
Sender: [email-masked]

Alexey Melnikov wrote:
>
> Tony Finch wrote:
>> On Mon, 19 May 2008, Alexey Melnikov wrote:
>>  
>>> Regarding metadata transfer: it might be handy. I was thinking about 
>>> an SMTP
>>> extension for passing IMAP flags from sender to recipient, for 
>>> example I can
>>> annotate a message with flags using Thunderbird [something I can do 
>>> already],
>>> then forward the annotated message [something I can't].
>>>     
>> The message envelope is lost at final delivery time, so MUA-to-MUA data
>> transfer has to be done via the message header.
> I was thinking that my LMTP server can also support this extension and 
> will set IMAP flags on delivery.
>> (This is why there are
>> different mechanisms for DSNs and MDNs. DSN information is for 
>> consumption
>> by MTAs so it's in the envelope, and MDN information is for 
>> consumption by
>> MUAs so it's in the header.) The IMAP server is a very borderline case
>> between MTA and MUA so it might not be immediately clear that your 
>> idea is
>> wrong. However you might want to forward a message as an attachment with
>> IMAP flags, in which case there's no envelope in which to stash them.
>>   
> Yes, a way to encode metadata in headers would be needed as well.
>
That's not an issue is it? Just add a new/custom header field. MTAs pass 
through header data unchanged if it doesn't understand it.

The problem is with MTA-MTA data to be passed separately from the 
message data. There's no way to do that at the moment without every MTA 
on a message's route understanding the specific extension the data is 
for. So, essentially what I'm suggesting is a way to do the same as for 
header data, but for MTA relevant data, so MTAs which understand the 
data can use it, but other MTAs will just pass it on to the next MTA 
unchanged.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Tue, 20 May 2008 10:42:52 +0100
From: Paul Smith <[email-masked]>
To: Alessandro Vesely <[email-masked]>
CC: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

Alessandro Vesely wrote:
>
> A C written program can easily scan the tag even without resorting to 
> regular expressions. Why would it want to do that? Two reasons:
>
> 1) Given that today is 019, the tag exemplified above exhibits an 
> expired day 234, thus a remote server can reject the message if it 
> adopts the policy of not accepting unbounceable messages, even if it 
> cannot verify the signature. (Possibly valid for batv1 only.)
Can it? Day 234 is after 019 isn't it? Given that the remote server has 
no way of knowing how long the sending MTA has specified for allowing 
bounces, this is an assumption that can't automatically be made.

Also, it would be easy for a spammer to just send a message from
batv1=1020bibble=[email-masked]

The receiving mail server can't check that 'bibble' isn't a valid 
signature, so it would accept the message. In any case, a spammer would 
just send it from [email-masked], as the recipient has no way of 
knowing that the sender address should be signed.

So, the remote server gains absolutely no benefit from BATV. That's 
fine, as it doesn't seem that the remote server is supposed to gain any 
benefit. It's the spoofed domain's MTAs which can gain the benefit.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Tue, 20 May 2008 02:22:10 -0700
To: Alessandro Vesely <[email-masked]>, [email-masked]
From: SM <[email-masked]>
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

At 00:57 20-05-2008, Alessandro Vesely wrote:
>1) Given that today is 019, the tag exemplified above exhibits an 
>expired day 234, thus a remote server can reject the message if it 
>adopts the policy of not accepting unbounceable messages, even if it 
>cannot verify the signature. (Possibly valid for batv1 only.)

Quoting Section 2.3.10 of RFC 2821:

   "Consequently, and due to a long history of problems when intermediate
    hosts have attempted to optimize transport by modifying them, the
    local-part MUST be interpreted and assigned semantics only by the host
    specified in the domain part of the address."

Regards,
-sm 

Date: Tue, 20 May 2008 09:51:04 +0100
From: Paul Smith <[email-masked]>
To: John R Levine <[email-masked]>
CC: IETF SMTP list <[email-masked]>
Subject: Re: Lose the time stamp in BATV
Sender: [email-masked]

John R Levine wrote:
>
> It occurs to me that the majority of BATV's interoperability problems 
> are due to the time stamp.  Let's assume for the moment that we 
> simplify a BATV signature to something like this:
>
>   prv=DHHHH=local@host
>
> The D is still a version number, and the HHH is a hash, but there's no 
> timestamp.  Now, unless you change your key, which seems pretty rare 
> in practice, a given plain address always turns into the same BATV 
> address.
The problem is that that's wide open for a replay attack

Personally I think that the prvs signature specification in the BATV 
standard is possibly superfluous. The local part meta-syntax section is 
useful as it allows the original local part to be extracted, and maybe 
the current prvs specification could be added as an example of how a 
private key could be implemented, but I think a private key signature 
specification would be just that, private. The data would be opaque to 
anything except the originating MTA, so it's irrelevant. There are lots 
of different ways of doing it, including some that would be better 
protected against replay attacks and could alleviate some problems with 
mailing lists etc, but wouldn't meet the 'prvs' specification.

Maybe only a 'private signature' type name should be specified, where 
the signature is opaque and can be up to, say, 16 characters long, and 
that's all that should be specified in the original BATV spec. Possible 
ways of implementing a private signature may be made available in other 
documents. The way the BATV spec is currently written, it looks like the 
prvs way is the only way that's allowed, but since the data in it is tag 
value generated is useless to anything except something which knows how 
it was made (i.e. the key) then the actual value of the tag value 
doesn't matter.

For instance, a server could have rules to use a certain key for all 
messages which are going to a certain domain, eg all messages to imc.org 
use the key 'bibble'. The server could check that any bounce messages 
which have the key 'bibble' are coming from imc.org, and reject them if 
they aren't, this would give that list server a constant 
(non-time-dependent) return-path to work with. Or, a server could use a 
different random key for each message which it stores in a database for 
a while, thus making replay attacks much harder since it could check how 
many bounce messages have come back for each key, and reject them if 
there are too many coming in. These things are possible within the BATV 
spec, but not within the prvs spec.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Tue, 20 May 2008 09:36:03 +0100
From: Paul Smith <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: public key BATV isn't useful
Sender: [email-masked]

Dave Crocker wrote:
>
>> The BATV introduction is misleading. It doesn't mention anything 
>> about the reasoning behind BATV that you state, 
>
> #2 and #3, above, are two types of bounce messages that are invalid, 
> because they result from unauthorized creation of the bounce address 
> (MailFrom).  The Introduction says:
>
>      "existing Internet mail permits unauthorized use of addresses in 
> the MailFrom command, which results in having notices sent to 
> unwitting and unwilling recipients."
>
> which seems to state exactly that condition, and:
>
>      "Bounce Address Tag Validation (BATV) defines a framework for 
> mechanisms that validate the value in this command."
>
> says that BATV seeks to remedy that problem.
>
> How should the Introduction be different?
>
Also, this bit really confused me

Section 4.2.2

"The checking of private signatures is only performed within the domain 
specified in the MailFrom command. The first component that processes 
the MailFrom's local-part must be able to interpret the meta-syntax. It 
MAY also perform validation."

This seems to be saying that only mail servers within the domain 
specified in the MailFrom command should check signatures. So, that 
seems to mean that ONLY the MTAs *sending* the message originally should 
check the domain, as those are the MTAs in the domain specified in the 
MailFrom. This really confused me. That's why I thought private key 
taggings was pointless - what's the point of one of the sender's MTAs 
checking the signature as the message was being sent?

Now (I think) I understand what BATV's supposed to do, I think I know 
what you were trying to say, but from a 'newbie' POV it just didn't seem 
to make sense.

For a bounce message, the MailFrom command specifies a null return path, 
so there is NO domain specified in the MailFrom command when the 
checking of private signatures is performed.

I think what you meant to say was that the checking of private 
signatures is only performed within the domain specified in the *RcptTo* 
command of the incoming bounce message - i.e. the domain where the 
original message should have come from.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Tue, 20 May 2008 09:12:41 +0100
From: Paul Smith <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: public key BATV isn't useful
Sender: [email-masked]

Dave Crocker wrote:
> Paul Smith wrote:
>>
>> Tony Hansen wrote:
>>> Paul, The original idea behind BATV was so that 1) the original 
>>> sending MTA can protect itself such that 2) non-delivery reports for 
>>> messages originally sent from there can be differentiated from 3) 
>>> non-delivery reports that are being sent in response to messages 
>>> *not* originating from that sending MTA. That is, NDRs from your 
>>> users (#2) will come back using your BATV tagging, whereas NDRs from 
>>> spammers (#3) will come back without using your BATV tagging, and 
>>> your system (#1) can happily ignore the #3 NDRs.
>> Ah. I think I seem my problem... I looked at the BATV spec as a 
>> newcomer to BATV rather than someone who had known about it for ages...
>>
>> The BATV introduction is misleading. It doesn't mention anything 
>> about the reasoning behind BATV that you state, 
>
>
> #2 and #3, above, are two types of bounce messages that are invalid, 
> because they result from unauthorized creation of the bounce address 
> (MailFrom).  The Introduction says:
>
>      "existing Internet mail permits unauthorized use of addresses in 
> the MailFrom command, which results in having notices sent to 
> unwitting and unwilling recipients."
>
> which seems to state exactly that condition, and:
No. To me, that is still talking about people SENDING notices to 
unwitting and unwilling recipients. Note that it says that the 
unauthorised use results in notices being *SENT* to unwitting and 
unwilling recipients. So, I read BATV as being a solution to prevent 
these notices being *SENT*.

Nowhere does it say anything about BATV being checked by the *recipient* 
MTA of the bounce messages.

>
>      "Bounce Address Tag Validation (BATV) defines a framework for 
> mechanisms that validate the value in this command."
>
> says that BATV seeks to remedy that problem.
Again, to me, that looks like it's a framework for use by the SENDER of 
the notices.

As I said, I think my problem is that I'm looking at this as someone who 
knows nothing about BATV, rather than someone who has known about it all 
along.
> How should the Introduction be different?

Just words along the lines of those that you used would help.

Eg, something like "BATV is intended primarily to be used by the MTA 
receiving a bounce message to determine whether the bounce message is in 
response to a message sent by an authorised sender for that domain. It 
will allow the MTA receiving the bounce message to discard bounce 
messages due to blowback/back-scatter from messages using forged return 
path addresses"

Some examples of use would also be useful.

eg

"An MTA receiving a bounce message like:

MAIL FROM:<>
RCPT TO:<private-key-tagged-localpart@domain>
DATA
...bounce message...

could accept the message, where as an MTA receiving a bounce message like:

MAIL FROM:<>
RCPT TO:<untagged-localpart@domain>
DATA
...bounce message...

could decide to reject the bounce message as it doesn't appear to have 
been in response to an authorised message sent from this domain"

>
> and the introduction says
>> things like "This assessment could aid in deciding whether to *send* 
>> a bounce message, thereby reducing the Internet mail infrastructure 
>> cost for transmitting notification messages in response to addresses 
>> used without permission." (emphasis mine) - thus implying that the 
>> thing sending the bounce message (ie NOT your own server - that 
>> should already know that the address was used legitimately) is the 
>> thing checking the BATV tags. To do that, you MUST have public key 
>> BATV tagging, private key tagging doesn't make sense.
>
> Right.  And since the sentence before the one you cite says:
>
>      "Enhancements would permit processing agents that are along the 
> original message's transfer path to determine whether the MailFrom 
> adress is likely to be valid.",
>
> it is meant to be clear that this portion of the discussion is 
> referring to something beyond the current specification.  So I'm not 
> sure I understand your concern.
Because that's the ONLY thing in the introduction (AFAICS) which states 
how BATV could be used... I did initially read it as 'this is something 
for the future', but when I looked for 'something for now', I couldn't 
find anything, so I assumed I'd misunderstood, and the above was 
actually for now as well.

>> So, I think the introduction needs to have something in it about the 
>> rationale, reasoning and purpose behind BATV, as I obviously 
>> misunderstood it, and even though I've now had my mistake explained 
>> to me, I still can't see anything in the spec which explains it...
>
> I do not understand what it should say that it does not already say.  
> Please offer suggested text.
>
It doesn't say ANYTHING about how BATV could/should be used in its 
current state (AFAICS). See above for the sort of thing I think should 
be there.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Tue, 20 May 2008 09:57:23 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

Dave Crocker wrote:
> [email-masked] wrote:
>> And this is what I was getting at when I said that I'm concerned that the
>> labelling is insufficiently unique in format. Mind you, I'm not 
>> proposing a change here, merely expressing concern.
> 
> Really.  If someone wants the current syntax changed, please do propose 
> it, explaining why it is needed.

As an example, one might propose replacing "prvs" with "batv1", e.g. 
batv1=1234abcdef=[email-masked]. Not that I like it very much, as 
people will tend to read it as "BAT version 1". However, it can be 
identified with a reasonably tight regular expression. Further 
mechanisms would be numbered by their position in that IANA registry.

A C written program can easily scan the tag even without resorting to 
regular expressions. Why would it want to do that? Two reasons:

1) Given that today is 019, the tag exemplified above exhibits an 
expired day 234, thus a remote server can reject the message if it 
adopts the policy of not accepting unbounceable messages, even if it 
cannot verify the signature. (Possibly valid for batv1 only.)

2) A receiving MTA can extract [email-masked] if it needs to check 
it against a list of subscribed users at the MAIL FROM level. (Also 
valid for future BATV mechanisms.)

Date: Mon, 19 May 2008 22:06:47 -0700
From: "Carl S. Gutekunst" <[email-masked]>
To: John R Levine <[email-masked]>
CC: IETF SMTP list <[email-masked]>
Subject: Re: Lose the time stamp in BATV
Sender: [email-masked]

John R Levine wrote:
> It occurs to me that the majority of BATV's interoperability problems 
> are due to the time stamp.  

Well, changing bounce address anyway.

> What you lose is the ability to reject bounces to mail sent a long 
> time ago.  But how important is that?  Having been the recipient of a 
> rather large amount of blowback, I've never seen any evidence that it 
> was targeted at me, or at least not targeted at me in order to get me 
> to read it.

You're working two different issues here.

First, spam or malware tucked inside a DSN certainly has been used by 
the bad guys; I recall seeing a large surge of it about a year ago, and 
I know Storm used it as a vector. It's simple social engineering: people 
usually read their DSNs. (Ironic that a blowback storm makes a user less 
inclined to read them.)

Second, any mechanism that makes your DSN and regular addresses 
different is going to reduce backscatter. Bounce addresses are much 
harder to harvest. If I append "-plugh" to all my outgoing bounce 
addresses right now and checked for that incoming, my backscatter would 
drop to zero, and stay there, maybe forever. Of course it can be 
trivially replayed, but who's going to care?

But -- if a couple million users started doing the same thing, then the 
bad guys would start to get interested.

The BATV signing mechanism has to assume that, at some point, the number 
of sites using it is high enough to make attempts to circumvent it 
worthwhile. We know BATV is vulnerable to replay, so we'd be remiss if 
we didn't define a way to mitigate that. We also know that bounce 
addresses do leak; poorly written mailing lists are probably the worst 
culprit.

My suggestion: Keep the time stamp, but make the rate/frequency of 
incrementing it a matter of local policy. I'd already been assuming I 
would only increment it weekly.

<csg>

Date: Tue, 20 May 2008 00:15:25 -0400 (EDT)
From: John R Levine <[email-masked]>
To: IETF SMTP list <[email-masked]>
Subject: Lose the time stamp in BATV
Cleverness: None detected
Sender: [email-masked]

It occurs to me that the majority of BATV's interoperability problems are 
due to the time stamp.  Let's assume for the moment that we simplify a 
BATV signature to something like this:

   prv=DHHHH=local@host

The D is still a version number, and the HHH is a hash, but there's no 
timestamp.  Now, unless you change your key, which seems pretty rare in 
practice, a given plain address always turns into the same BATV address.

Now if you have a mailing list manager that recognizes senders by bounce 
address, it works fine, because each sender always has the same bounce 
address.  (The bounce address won't match the From: address, but it is my 
impression that software that keys on bounce address is unlikely to be 
picky about From: addresses.)  There will be a one-time issue when a 
sending site starts using BATV, but that's a whole lot less pain than when 
your bounce address changes every day.

Greylisters that remember each envelope to decide what mail not to delay, 
check.  Annoying C/R schemes that challenge each new bounce address, 
check.  Vacation programs that limit traffic by responding only once to 
each bounce adddress, check.

What you lose is the ability to reject bounces to mail sent a long time 
ago.  But how important is that?  Having been the recipient of a rather 
large amount of blowback, I've never seen any evidence that it was 
targeted at me, or at least not targeted at me in order to get me to read 
it.  Trading off replay protection to get lists to work seems like a big 
win to me.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

Date: Mon, 19 May 2008 23:34:38 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

John Levine wrote:
>> Again, I'm not proposing a change. I'm merely a little worried that
>> the syntax is insufficiently unique and I'm wondering if others are
>> also concerned.
> 
> It would make sense to create a registry of local part prefixes,
> probably limited to prefixes of the form xxx= where the xxx is a
> string of letters and digits.

Here's suggested text for an IANA Considerations section.

7. IANA Considerations

IANA is directed to create the registry "Bounce Address Tag
validation (BATV) Tag Schemes". The BATV Tag Scheme table has
the following entries in it:

     Tag Scheme Code:	A short tag for the tag scheme, following the
			ABNF given in RFC XXXX.

     Tag Scheme Name:	The full name for the tag scheme.

     Description:	A short description of the code.

     Reference:		A reference to the document in which the tag
			scheme is defined. This reference should note
			whether the relevant specification is
			standards-rack or not, using "(Standards track)"
			or "(Not standards track)".

     Submitter:		The identity of the submitter, usually the
			document author.

     Change Controller:	The identity of the change controller
			for the specification.  This will be "IESG" in
			the case of IETF-produced documents.

7.1. The initial entry for the prvs tag scheme defined in this RFC is 
shown here:

     Tag Scheme Code:	prvs
     Tag Scheme Name:	Simple Private Signature
     Description:	The Simple Private Signature (PRVS) scheme signs
			the original MailFrom by using a simple
			shared-key to add a hash of the address and some
			time-based randomizing information.
     Reference:		RFC XXXX
     Submitter:		J. Levine, D. Crocker, S. Silberman, T. Finch
     Reference:		RFC XXXX. (Standards track)
     Change controller:	IESG.

7.2.  Review Process for New Values

    Entries in this registry are expected to follow the "Specification
    Required" model ([RFC2434]) although, in practice, most entries are
    expected to derive from standards-track documents.  Non-standards-
    track documents that specify codes to be registered should be readily
    available.  The principal purpose of this registry is to avoid
    confusion and conflicts among different definitions or uses for the
    same code.

Date: 20 May 2008 03:02:36 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>Again, I'm not proposing a change. I'm merely a little worried that
>the syntax is insufficiently unique and I'm wondering if others are
>also concerned.

It would make sense to create a registry of local part prefixes,
probably limited to prefixes of the form xxx= where the xxx is a
string of letters and digits.  Having looked again at SRS, there
doesn't seem to be much hope for a generic de-prefixing system even if
one thought it might be useful.

R's,
John

Date: Mon, 19 May 2008 15:40:44 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
CC: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

[email-masked] wrote:
> Really, now that I go and look the list of examples of RFCs doing this sort
> of thing is pretty darned long.

OK.  Those certainly are concrete exemplars to work from.  Thanks.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Mon, 19 May 2008 18:24:57 -0400 (EDT)
From: John R Levine <[email-masked]>
To: mouss <[email-masked]>
cc: IETF SMTP list <[email-masked]>
Subject: Re: private key BATV is useful
Cleverness: None detected
Sender: [email-masked]

> but then, why not a scheme such as
>   user+tagstuff@domain
> (replace '+' with any char(s) you prefer). this would make it almost trivial 
> to implement in MTAs that support address extensions.

Because it would collide with the way that address extensions are used 
now.

BATV is handled in the MTA at SMTP time, but address extensions are 
handled at delivery time or later.  If the BATV stuff were put in an 
extension, you couldn't tell it from VERP and other delivery time 
extension usages.

> Also, whatever the scheme is, there is another class of applications to think 
> of. these are applications that use the sender address as an element to 
> determine the "reputation" of a transaction (spamassassin AWL for instance). 
> such apps should be made aware of the various BATV styles.

See subsequent message.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

Date: Tue, 20 May 2008 00:20:01 +0200
From: mouss <[email-masked]>
To: John R Levine <[email-masked]>
CC: [email-masked]
Subject: Re: private key BATV is useful
Sender: [email-masked]

John R Levine wrote:
>
>> For BATV to be useful, a domain would have to be able to say that all 
>> messages from that domain must have BATV tags. I can't see anything 
>> in the spec to cover that requirement. Otherwise a spammer will just 
>> send messages without BATV tags, and BATV will achieve nothing other 
>> than upsetting some legitimate mailing lists...
>
> The main goal of BATV is to deal with bounce blowback, that is, DSNs 
> due to spam sent with forged return addresses.  If your domain is 
> popular among spammers, blowback can be a serious issue.  On a bad 
> day, my abuse.net domain gets 400,000 bounces for mail it didn't send.
>
> The only thing prvs accomplishes is to let you tell whether an 
> incoming bounce was sent in response to a message you actually sent.  
> If you know that you sign all your own mail, you can be reasonably 
> sure that bounces to signed addresses are real, and bounces to 
> unsigned addresses are fake, give or take the edge cases we've been 
> discussing.

but then, why not a scheme such as
    user+tagstuff@domain
(replace '+' with any char(s) you prefer). this would make it almost 
trivial to implement in MTAs that support address extensions.

and this would go along with Ned comments, since you can then use 
user+tag1+tag2+...@domain (again, the '+' signs are just an example).

While I am in, the recommendation in
    http://babel.de/batv.html
(cited from the implementations link) to use regular expressions based 
canonical breaks recipient validation and will cause backscatter. which 
breaks the whole purpose of BATV INMHO.

Also, whatever the scheme is, there is another class of applications to 
think of. these are applications that use the sender address as an 
element to determine the "reputation" of a transaction (spamassassin AWL 
for instance). such apps should be made aware of the various BATV styles.

Date: Mon, 19 May 2008 13:58:44 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: Dave Crocker <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

> [email-masked] wrote:
> >>   Make it crystal clear, right up front, how you go about to
> >> implement this, and leave the technical details and dry, formal
> >> specifications, to some other part of the document."
>  >
> > Very nicely put. This is the main thing I think is wrong with the document.

> Nicely put, but I'm not sure I know what it translates into.  No doubt I'm
> just having some sort of allergy-induced fuzzi-headedness, but I am not
> immediately retrieving examples from other specifications that help me
> understand what this suggestion actually should look like in practice.

Just off the top of my head, the various Sieve documents are laden with Sieve
code examples that I find to be extremely helpful when implementing things.

The MIME specifications include step-by-step instructions for performing
encoding and decoding. Again, speaking as an implementors, these are
incredibly helpful.

RFC 2821 contains numerous example status responses. RFC 2822 contains a
sample message.

Really, now that I go and look the list of examples of RFCs doing this sort  of
thing is pretty darned long. IMAP used to be something of a holdout, I believe,
but this has changed in recent versions. (And I disagree with Mark Crispin's
assessment that this was a mistake.)

> In case that sounds like unfriendly resistance (or even friendly resistance)
> please understand it isn't:  I'm always in favor of adding text that improves
> clarity and/or improves pedagogy.  In this case, I think I have a fuzzy
> understanding of the basic desire, but I'm not seeing how to turn that into
> words.  I don't see how the Intro on its own, or the Model introduction
> paragraph on its own, or the Operation section on its own fail to satisfy the
> goal.

> And no, I'm not seeking to debate or defend them.  I'm looking for something
> concrete to help me understand what needs adding.  (In a feeble attempt to try
> to be more helpful in asking for help I'll say that I'm not worried about
> careful wording; anything roughly along the lines desired would be fine.  we
> can word-smith it later.)

> > And this is what I was getting at when I said that I'm concerned that the
> > labelling is insufficiently unique in format. Mind you, I'm not proposing a
> > change here, merely expressing concern.

> Here I am certain I *do* understand the concern.  And it's always a valid
> concern to raise about a syntax.

Um, not really. Most of the times when we define a new syntax it fits into an
entirely new niche somewhere and we know when to use it from the context.
Accordingly, the criteria for syntax selection are usually simplicity, ease of
evaluation, ease of production, extensibility, and so on. The ability to "
reliably pick the syntax out of a lineup" is rarely on the list.

Unflagged overlay syntaxes like this are the case where distinguishability from
the background is a concern, and such cases are comparatively rare. In fact
aside from the obvious example of RFC 2047 I'm having trouble thinking of other
examples in the protocols we normally deal with. (OK, there was some use of
overlay syntax in the original MIME RFCs as well as in RFC 822, but this was a
structural artefact of the ways in which those documents were written, not a
requirement imposed by trying to extend an existing protocol structure. And it
is telling to note that this usage has been eliminated from more recent
versions of those specifications without changing protocol syntax or
semantics.)

> The problem is that we can't do much with it unless someone documents
> deficiencies in the current syntax, with respect to requirements that we agree
> need to be satisfied.  While the current syntax has some evolution -- ie,
> learning -- that produced it, I don't think any of us is religious about this
> sort of detail, other than wanting changes to have a clear motivation, clear
> benefit, and the usual consensus.

> Really.  If someone wants the current syntax changed, please do propose it,
> explaining why it is needed.

Again, I'm not proposing a change. I'm merely a little worried that the syntax
is insufficiently unique and I'm wondering if others are also concerned. I
already gabe an example of another existing, albeit nonstandard, overlay syntax
for local parts it conflicts with: SRS. But this one is easy to address if we
think it is worth it: Prohibit use of SRSn as a registered tag. All it would
take is a single sentence in the registration criteria.

Now, looking at RFC 2047, I think we actually went a little too far to try and
avoid conflicts where stuff not intended to be seen as an encoded-word got
taken  as one. The result is the syntax is a little too complex and there have
been any number of implementation problems. (I note in passing that RFC 2047
has been criticized from not providing explicit step by step procedures for
dealing with encoded words.) So I'm definitely not proposing that we adopt =?
and ?= or something similar as part of this.

In any case, if nobody else sees a problem here then I think it is fine to move
forward with the current syntax.

				Ned

Date: Mon, 19 May 2008 11:17:49 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
CC: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

[email-masked] wrote:
>>   Make it crystal clear, right up front, how you go about to
>> implement this, and leave the technical details and dry, formal
>> specifications, to some other part of the document."
 >
> Very nicely put. This is the main thing I think is wrong with the document.

Nicely put, but I'm not sure I know what it translates into.  No doubt I'm 
just having some sort of allergy-induced fuzzi-headedness, but I am not 
immediately retrieving examples from other specifications that help me 
understand what this suggestion actually should look like in practice.

In case that sounds like unfriendly resistance (or even friendly resistance) 
please understand it isn't:  I'm always in favor of adding text that improves 
clarity and/or improves pedagogy.  In this case, I think I have a fuzzy 
understanding of the basic desire, but I'm not seeing how to turn that into 
words.  I don't see how the Intro on its own, or the Model introduction 
paragraph on its own, or the Operation section on its own fail to satisfy the 
goal.

And no, I'm not seeking to debate or defend them.  I'm looking for something 
concrete to help me understand what needs adding.  (In a feeble attempt to try 
to be more helpful in asking for help I'll say that I'm not worried about 
careful wording; anything roughly along the lines desired would be fine.  we 
can word-smith it later.)

> And this is what I was getting at when I said that I'm concerned that the
> labelling is insufficiently unique in format. Mind you, I'm not proposing a
> change here, merely expressing concern.

Here I am certain I *do* understand the concern.  And it's always a valid 
concern to raise about a syntax.

The problem is that we can't do much with it unless someone documents 
deficiencies in the current syntax, with respect to requirements that we agree 
need to be satisfied.  While the current syntax has some evolution -- ie, 
learning -- that produced it, I don't think any of us is religious about this 
sort of detail, other than wanting changes to have a clear motivation, clear 
benefit, and the usual consensus.

Really.  If someone wants the current syntax changed, please do propose it, 
explaining why it is needed.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Mon, 19 May 2008 19:15:52 +0100
From: Tony Finch <[email-masked]>
To: Alessandro Vesely <[email-masked]>
cc: [email-masked]
Subject: smarthosts
Sender: [email-masked]

On Mon, 19 May 2008, Alessandro Vesely wrote:
>
> > This kind of problem makes me think there's not much benefit to
> > rejecting messages at SMTP time from trusted clients, whether they are
> > MUAs or MTAs.
>
> Very much agreed! However, while SMTP AUTH clearly identifies trusted
> MUAs, a "trusted MTA" is a somewhat fuzzy concept...

I mean a client MTA on the local network that's using my MTA as an
outgoing relay. The client MTA is often running on a web server or
is a departmental email server.

[email-masked] wrote:
>
> The one exception that does arise is the occasional deeply broken MUA,
> e.g., ones that fail to present any sort of useful error information to
> the user. When such MUAs are prevalent (and there are placess where they
> are - and again, just because you haven't encountered them doesn't mean
> they don't exist) having the option to accept basically anything and
> send a DSN later is useful.

IME MUAs keep finding new ways of being deeply broken.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
FITZROY: SOUTHERLY OR SOUTHEASTERLY 3 OR 4, BUT 5 OR 6 IN NORTHWEST. SLIGHT OR
MODERATE, OCCASIONALLY ROUGH IN NORTHWEST. OCCASIONAL RAIN. MODERATE OR GOOD,
OCCASIONALLY POOR LATER.

Date: Mon, 19 May 2008 10:42:21 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: Alessandro Vesely <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> Dave Crocker wrote:

> > So let's hear what constraints and requirements folks deem
> > essential for this meta-syntax, and why.  And given that the spec
> > already has a proposal, I'll suggest that anyone claiming it is not
> > sufficient would help things a lot by explaining why.

> In general, the draft misses clear indications and guidelines for the
> implementation. There is not even a single example. Let me quote an
> implementor's perspective: "People should be able to take 5 minutes of
> their time to briefly look through the document, see exactly what's
> going on. Make it crystal clear, right up front, how you go about to
> implement this, and leave the technical details and dry, formal
> specifications, to some other part of the document."

Very nicely put. This is the main thing I think is wrong with the document.

> Another point: Is it useful to retrieve the untagged mailbox value?
> (E.g. could that be a way to patch, say, ezmlm?) If it is, it would be
> helpful to have a syntax that delivers a good level of confidence that
> a given token is a BATV-tagged representation of some tagging scheme
> even if that was not known at implementation time.

And this is what I was getting at when I said that I'm concerned that the
labelling is insufficiently unique in format. Mind you, I'm not proposing a
change here, merely expressing concern.

> Should the
> tag-types be loadable at runtime from some configuration file? Are
> they case sensitive?

No, that's the wrong way to do it because nce those configurations are deployed
it becomes very difficult to add new schemes. This is why the syntax needs to
be distinguisable without having to look for a known tag at the beginning.

				Ned

Date: Mon, 19 May 2008 10:03:59 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: Tony Finch <[email-masked]>
Cc: [email-masked], Peter J Holzer <[email-masked]>, [email-masked]
Sender: [email-masked]

> On Sun, 18 May 2008, [email-masked] wrote:
> >
> > Another issue is that if you do this sort thing you absolutely must
> > distinguish between submission and relay somehow, and lots of places are
> > unable to make such distinctions. It's one thing to force a automatic
> > relay to retry the second and subsequent recipients of a message, quite
> > another to force some random user to resubmit the message over and over
> > in order to get it through.

> This kind of problem makes me think there's not much benefit to rejecting
> messages at SMTP time from trusted clients, whether they are MUAs or MTAs.

The situation with SMTP submit is a lot more subtle than that. There are at
least two axes involved here: Temporary versus permanent failures and what I'll
call MUA quality.

Let's start with permanent failures. In this case things are strongly biased
towards immediate rejection, because:

(1) It gives the user immediate feedback.
(2) It's much more easily localizable.
(3) It gives the MUA the leeway to allow the error to be corrected,
    avoiding the whole "resending again, this time with the correct
    address" business.
(4) In many cases it avoids having to transfer the message data - especially
    an issue for mobile MUAs where bandwidth is dear.

The one exception that does arise is the occasional deeply broken MUA, e.g.,
ones that fail to present any sort of useful error information to the user.
When such MUAs are prevalent (and there are placess where they are - and again,
just because you haven't encountered them doesn't mean they don't exist) having
the option to accept basically anything and send a DSN later is useful. (FWIW,
the keyword for this in our product is acceptalladdrs.)

Moving on to temporary failures... A very high quality client could take care
of this itself by putting such messages in a queue and retrying them, just like
an MTA does. But in practice such MUAs are rare. In fact not only do a lot of
MUAs treat 4yz pretty much the same as 5yz, they may not even make the
difference clear in the message they present to the user. And that makes this
case a support call generator - and for people operating large scale setups
even serious DDOS attacks are easier for them to deal with than a big influx of
support calls.

So in this case things are strongly biased towards having the MTA accept the
address/message even though it is unsure it will be able to deliver it. (FWIW,
the bias in this case is so strong that we revert to this behavior
unconditionally when operating in SMTP submit mode. Of course we'd consider
changing that if we become aware of a population of high quality clients that
handle temporary failures better.)

An additional wrinkle in all of this is client handling of DSNs. AFAIK there
are very few that attempt to correlate DSNs with the messages that generated
them. This means it is up to the user to figure out what message generated a
given DSN - which can be a problem if the DSN isn't well constructed or the
user is without clue. This issues tends to push things towards immediate error
responses, but IMO not enough to warrant immediate responses in the temporary
error case.

Finally, lurking under or around all of this is the nature of the 4yx response
itself. Using 4yz responses to limit the number of recipients in a given
transaction to 1 is not what the mechanism was intended for. This matters
because right now 4yz errors are usually the result of a server issue of some
sort, and since most messages from legitimate senders don't end up bouncing
most 4yz resolve without a DSN being sent. This changes the minute you start
using 4yz as a sort of protocol modification mechanism and once more DSNs are
in play the issue of poor client handling of DSNs becomes much more important.

				Ned

P.S. At least some of these issues could be resolved with an SMTP extension
that let a server announce certain sorts of limits, i.e., on the number of
allowed recipients per transaction. Alternately, a mechanism for a client to
identify itself to the server would allow clever servers to accomodate various
sorts of bustage in well known clients. However, I believe it is  better to fix
problems than work around them. (Of course I'm also a server implementor, so
I'm a bit biased against schemes that create open-ended work for me to do...)

Date: Mon, 19 May 2008 09:49:45 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Paul Smith <[email-masked]>
CC: [email-masked]
Subject: Re: public key BATV isn't useful
Sender: [email-masked]

Paul Smith wrote:
> 
> Tony Hansen wrote:
>> Paul, The original idea behind BATV was so that 1) the original 
>> sending MTA can protect itself such that 2) non-delivery reports for 
>> messages originally sent from there can be differentiated from 3) 
>> non-delivery reports that are being sent in response to messages *not* 
>> originating from that sending MTA. That is, NDRs from your users (#2) 
>> will come back using your BATV tagging, whereas NDRs from spammers 
>> (#3) will come back without using your BATV tagging, and your system 
>> (#1) can happily ignore the #3 NDRs.
> Ah. I think I seem my problem... I looked at the BATV spec as a newcomer 
> to BATV rather than someone who had known about it for ages...
> 
> The BATV introduction is misleading. It doesn't mention anything about 
> the reasoning behind BATV that you state, 

#2 and #3, above, are two types of bounce messages that are invalid, because 
they result from unauthorized creation of the bounce address (MailFrom).  The 
Introduction says:

      "existing Internet mail permits unauthorized use of addresses in the 
MailFrom command, which results in having notices sent to unwitting and 
unwilling recipients."

which seems to state exactly that condition, and:

      "Bounce Address Tag Validation (BATV) defines a framework for mechanisms 
that validate the value in this command."

says that BATV seeks to remedy that problem.

How should the Introduction be different?

and the introduction says
> things like "This assessment could aid in deciding whether to *send* a 
> bounce message, thereby reducing the Internet mail infrastructure cost 
> for transmitting notification messages in response to addresses used 
> without permission." (emphasis mine) - thus implying that the thing 
> sending the bounce message (ie NOT your own server - that should already 
> know that the address was used legitimately) is the thing checking the 
> BATV tags. To do that, you MUST have public key BATV tagging, private 
> key tagging doesn't make sense.

Right.  And since the sentence before the one you cite says:

      "Enhancements would permit processing agents that are along the original 
message's transfer path to determine whether the MailFrom adress is likely to 
be valid.",

it is meant to be clear that this portion of the discussion is referring to 
something beyond the current specification.  So I'm not sure I understand your 
concern.

This reaction to the reference to a public-key version of BATV once again 
demonstrates the dangers of having a specification discuss anything about a 
hypothetical future.  Best we remove all such references.  If and when there 
is an effort to specify and deploy and enhancement such as a public key 
version, we can document it separately.

> So, I think the introduction needs to have something in it about the 
> rationale, reasoning and purpose behind BATV, as I obviously 
> misunderstood it, and even though I've now had my mistake explained to 
> me, I still can't see anything in the spec which explains it...

I do not understand what it should say that it does not already say.  Please 
offer suggested text.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Mon, 19 May 2008 18:47:18 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: BATV doesn't fix forwarding
Sender: [email-masked]

Tony Finch wrote:
> On Sun, 18 May 2008, [email-masked] wrote:
>>
>> Another issue is that if you do this sort thing you absolutely must
>> distinguish between submission and relay somehow, and lots of places are
>> unable to make such distinctions. It's one thing to force a automatic
>> relay to retry the second and subsequent recipients of a message, quite
>> another to force some random user to resubmit the message over and over
>> in order to get it through.
> 
> This kind of problem makes me think there's not much benefit to rejecting
> messages at SMTP time from trusted clients, whether they are MUAs or MTAs.

Very much agreed! However, while SMTP AUTH clearly identifies trusted 
MUAs, a "trusted MTA" is a somewhat fuzzy concept...

Date: Mon, 19 May 2008 12:24:02 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC:  [email-masked], Peter J Holzer <[email-masked]>, [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

Tony Finch wrote:
> On Sun, 18 May 2008, [email-masked] wrote:
>> Another issue is that if you do this sort thing you absolutely must
>> distinguish between submission and relay somehow, and lots of places are
>> unable to make such distinctions. It's one thing to force a automatic
>> relay to retry the second and subsequent recipients of a message, quite
>> another to force some random user to resubmit the message over and over
>> in order to get it through.
> 
> This kind of problem makes me think there's not much benefit to rejecting
> messages at SMTP time from trusted clients, whether they are MUAs or MTAs.

Thats how we do it.

As long as the the client is authenticated on one of the following was:

      - IP Relay Table
      - POP BEFORE SMTP
      - ESMTP AUTH

it allows relay and skips all SMTP level checks which currently includes

      ----   Suite of WCSAP Methods -----
        - Local Policy Filter/Rule File,
        - DNS RBL
        - SPF
        - CBV

and WCSAP is done after RCPT TO: is checked and is not local.

This keeps with the traditional BCP of accepting all local recipient or 
hosted mail.  All the extra is only required and done if its a relay.

At the DATA level, we offer a rule based filtering system but these 
rules are 100% defined by the administrator.  The SMTP software makes no 
inherent rule at the content level.  The only thing we provide is an 
example simple rule - "check for bad words", etc.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Mon, 19 May 2008 20:00:01 +0400
From: Alexey Melnikov <[email-masked]>
To: Tony Finch <[email-masked]>
CC: Paul Smith <[email-masked]>, "ietf >> SMTP Interest Group" <[email-masked]>
Subject: Re: SMTP metadata transfer
Sender: [email-masked]

Tony Finch wrote:
> On Mon, 19 May 2008, Alexey Melnikov wrote:
>   
>> Regarding metadata transfer: it might be handy. I was thinking about an SMTP
>> extension for passing IMAP flags from sender to recipient, for example I can
>> annotate a message with flags using Thunderbird [something I can do already],
>> then forward the annotated message [something I can't].
>>     
> The message envelope is lost at final delivery time, so MUA-to-MUA data
> transfer has to be done via the message header.
I was thinking that my LMTP server can also support this extension and 
will set IMAP flags on delivery.
> (This is why there are
> different mechanisms for DSNs and MDNs. DSN information is for consumption
> by MTAs so it's in the envelope, and MDN information is for consumption by
> MUAs so it's in the header.) The IMAP server is a very borderline case
> between MTA and MUA so it might not be immediately clear that your idea is
> wrong. However you might want to forward a message as an attachment with
> IMAP flags, in which case there's no envelope in which to stash them.
>   
Yes, a way to encode metadata in headers would be needed as well.

Date: Mon, 19 May 2008 16:56:03 +0100
From: Tony Finch <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Paul Smith <[email-masked]>, "ietf >> SMTP Interest Group" <[email-masked]>
Subject: Re: SMTP metadata transfer
Sender: [email-masked]

On Mon, 19 May 2008, Alexey Melnikov wrote:
>
> Regarding metadata transfer: it might be handy. I was thinking about an SMTP
> extension for passing IMAP flags from sender to recipient, for example I can
> annotate a message with flags using Thunderbird [something I can do already],
> then forward the annotated message [something I can't].

The message envelope is lost at final delivery time, so MUA-to-MUA data
transfer has to be done via the message header. (This is why there are
different mechanisms for DSNs and MDNs. DSN information is for consumption
by MTAs so it's in the envelope, and MDN information is for consumption by
MUAs so it's in the header.) The IMAP server is a very borderline case
between MTA and MUA so it might not be immediately clear that your idea is
wrong. However you might want to forward a message as an attachment with
IMAP flags, in which case there's no envelope in which to stash them.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
CROMARTY FORTH TYNE DOGGER: NORTHERLY BECOMING VARIABLE 3 OR 4. SLIGHT OR
MODERATE. SHOWERS. GOOD.

Date: Mon, 19 May 2008 16:44:21 +0100
From: Paul Smith <[email-masked]>
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: public key BATV isn't useful
Sender: [email-masked]

Tony Hansen wrote:
> Paul, The original idea behind BATV was so that 1) the original 
> sending MTA can protect itself such that 2) non-delivery reports for 
> messages originally sent from there can be differentiated from 3) 
> non-delivery reports that are being sent in response to messages *not* 
> originating from that sending MTA. That is, NDRs from your users (#2) 
> will come back using your BATV tagging, whereas NDRs from spammers 
> (#3) will come back without using your BATV tagging, and your system 
> (#1) can happily ignore the #3 NDRs.
Ah. I think I seem my problem... I looked at the BATV spec as a newcomer 
to BATV rather than someone who had known about it for ages...

The BATV introduction is misleading. It doesn't mention anything about 
the reasoning behind BATV that you state, and the introduction says 
things like "This assessment could aid in deciding whether to *send* a 
bounce message, thereby reducing the Internet mail infrastructure cost 
for transmitting notification messages in response to addresses used 
without permission." (emphasis mine) - thus implying that the thing 
sending the bounce message (ie NOT your own server - that should already 
know that the address was used legitimately) is the thing checking the 
BATV tags. To do that, you MUST have public key BATV tagging, private 
key tagging doesn't make sense.

So, I think the introduction needs to have something in it about the 
rationale, reasoning and purpose behind BATV, as I obviously 
misunderstood it, and even though I've now had my mistake explained to 
me, I still can't see anything in the spec which explains it...

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Mon, 19 May 2008 19:48:10 +0400
From: Alexey Melnikov <[email-masked]>
To: Tony Finch <[email-masked]>
CC: Paul Smith <[email-masked]>, "ietf >> SMTP Interest Group" <[email-masked]>
Subject: Re: SMTP metadata transfer
Sender: [email-masked]

Tony Finch wrote:
> On Mon, 19 May 2008, Paul Smith wrote:
>   
>> So, I wondered, could an extension not be come up with to allow 'metadata' to
>> be passed with the message. At the moment we have the envelope (sender +
>> recipient) and data. What if we could add arbitrary data to go along with the
>> 'envelope'.
>>     
> Doesn't the existing DSN ENVID parameter do this for your?
>   
No, because it has a limited size.

Regarding metadata transfer: it might be handy. I was thinking about an 
SMTP extension for passing IMAP flags from sender to recipient, for 
example I can annotate a message with flags using Thunderbird [something 
I can do already], then forward the annotated message [something I can't].

Date: Mon, 19 May 2008 16:41:48 +0100
From: Tony Finch <[email-masked]>
To: [email-masked]
cc: Peter J Holzer <[email-masked]>, [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

On Sun, 18 May 2008, [email-masked] wrote:
>
> Another issue is that if you do this sort thing you absolutely must
> distinguish between submission and relay somehow, and lots of places are
> unable to make such distinctions. It's one thing to force a automatic
> relay to retry the second and subsequent recipients of a message, quite
> another to force some random user to resubmit the message over and over
> in order to get it through.

This kind of problem makes me think there's not much benefit to rejecting
messages at SMTP time from trusted clients, whether they are MUAs or MTAs.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
SOUTHEAST ICELAND: SOUTHEASTERLY 4 OR 5, OCCASIONALLY 6 LATER IN WEST. SLIGHT
OR MODERATE, OCCASIONALLY ROUGH LATER IN SOUTHWEST. OCCASIONAL RAIN. MODERATE
OR GOOD.

Date: Mon, 19 May 2008 16:38:46 +0100
From: Paul Smith <[email-masked]>
To: John R Levine <[email-masked]>
CC: [email-masked]
Subject: Re: private key BATV is useful
Sender: [email-masked]

John R Levine wrote:
>> For BATV to be useful, a domain would have to be able to say that all 
>> messages from that domain must have BATV tags. I can't see anything 
>> in the spec to cover that requirement. Otherwise a spammer will just 
>> send messages without BATV tags, and BATV will achieve nothing other 
>> than upsetting some legitimate mailing lists...
>
> The main goal of BATV is to deal with bounce blowback, that is, DSNs 
> due to spam sent with forged return addresses.  If your domain is 
> popular among spammers, blowback can be a serious issue.  On a bad 
> day, my abuse.net domain gets 400,000 bounces for mail it didn't send.
>
> The only thing prvs accomplishes is to let you tell whether an 
> incoming bounce was sent in response to a message you actually sent.  
> If you know that you sign all your own mail, you can be reasonably 
> sure that bounces to signed addresses are real, and bounces to 
> unsigned addresses are fake, give or take the edge cases we've been 
> discussing. 
Ah, OK, I was misunderstanding the point of BATV. Maybe it should be 
made clearer - looking at the introduction to the BATV spec, it seems to 
be saying that BATV is useful for deciding whether to SEND a bounce 
message, not whether to ACCEPT one. Obviously (to me, at least) private 
key BATV is useless for deciding whether to send a bounce message, since 
you have no way of checking the key.

I'm still not really sure about this. So, what you are saying is that if 
I used BATV, and I got a message with a null return path, and an 
unsigned recipient address, I could decide to reject/discard it?

But, I can't reject anything unless it has a null return path - so, what 
if the bounce message hasn't got a null return path? (I know they SHOULD 
have, but some ISPs won't relay mail with a null return path, so bounce 
messages going through those have to be massaged to have a non-null 
return path, eg 'no-reply@' or similar)

Similarly, what if a message comes in which has a null return path, but 
used a 'reply-to' address or similar as the recipient instead of the 
return path (eg some autoresponders may do this).

I can't tell if a message is ACTUALLY a bounce message unless I get the 
content of the message (by which time a lot of the benefit is lost) and 
even then it's hard to do since most bounce messages don't conform to 
the MDN standards.

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Mon, 19 May 2008 16:17:49 +0100
From: Tony Finch <[email-masked]>
To: Paul Smith <[email-masked]>
cc: "ietf >> SMTP Interest Group" <[email-masked]>
Subject: Re: SMTP metadata transfer
Sender: [email-masked]

On Mon, 19 May 2008, Paul Smith wrote:
>
> So, I wondered, could an extension not be come up with to allow 'metadata' to
> be passed with the message. At the moment we have the envelope (sender +
> recipient) and data. What if we could add arbitrary data to go along with the
> 'envelope'.

Doesn't the existing DSN ENVID parameter do this for your?

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
IRISH SEA: SOUTHEASTERLY 4 OR 5, BECOMING VARIABLE 3 FOR A TIME. SLIGHT OR
MODERATE. FAIR. GOOD.

Subject: Re: BATV pseudo-Last Call
From: Jeff Macdonald <[email-masked]>
To: [email-masked]
Organization: e-Dialog
Date: Mon, 19 May 2008 15:14:28 +0000
Sender: [email-masked]

On Sat, 2008-05-17 at 13:57 -0400, John R Levine wrote:

> > Another one: String lengths.
> 
> You're right, it's a problem.  The hand-wave is to assert that few MTAs 
> enforce the 64 character limit, but I realize that's not very persuasive 
> for a standard.  On the other hand, since there's no nesting, the damage 
> is at least predictable.  The current spec adds 16 characters to each 
> address but I've been considering a modified version with 10 characters 
> that addresses some greylisting problems as well.

As a data point, I have seen many European receivers choke on > 64 char
local parts. I don't have a list of those domains (and it was may 4
years ago), but when I tested a BATV like scheme here at work which
allowed > 64 characters, messages were not delivered.

-- 
:: Jeff Macdonald | Director of Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421
:: v: 781-372-1922 | f: 781-863-8118
:: www.e-dialog.com

Date: 19 May 2008 15:07:14 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>Another point: Is it useful to retrieve the untagged mailbox value? 
>(E.g. could that be a way to patch, say, ezmlm?)

Yes, that's the only benefit I can see to standardizing the syntax.

> If it is, it would be helpful to have a syntax that delivers a good
>level of confidence that a given token is a BATV-tagged
>representation of some tagging scheme even if that was not known at
>implementation time.

First, let's see whether we expect other tagging schemes.  The more I
think about it, the less likely I think it is.

R's,
John

Date: Mon, 19 May 2008 10:18:29 -0400
From: Tony Hansen <[email-masked]>
To: Paul Smith <[email-masked]>
CC: [email-masked]
Subject: Re: public key BATV isn't useful
Sender: [email-masked]

Paul, The original idea behind BATV was so that 1) the original sending 
MTA can protect itself such that 2) non-delivery reports for messages 
originally sent from there can be differentiated from 3) non-delivery 
reports that are being sent in response to messages *not* originating 
from that sending MTA. That is, NDRs from your users (#2) will come back 
using your BATV tagging, whereas NDRs from spammers (#3) will come back 
without using your BATV tagging, and your system (#1) can happily ignore 
the #3 NDRs.

The notion of a public-key component would extend the BATV notion one 
step further and allow receiving MTAs to validate the MAIL FROM address. 
To achieve that would require the same level of infrastructure as DKIM 
uses. Whether it would be worth taking BATV to that level is another 
discussion.

	Tony Hansen
	[email-masked]

Paul Smith wrote:
> John Levine wrote:
>>>> Oh, and one final note. The document talks a bit about defining a
>>>> public key BATV scheme but doesn't actually define anything.
>>>>       
>>
>> Humor me for a moment here.
>>
>> The idea of a public key BATV is so the system generating the bounce
>> can check the signature and not even send bogus bounces.  But any
>> system that is going to DKIM sign its bounce addresses would also be
>> able to DKIM sign its message bodies, so ADSP discardable already
>> allows you to declare that everything is signed so don't bounce the
>> unsigned stuff.  We're talking about DSNs here, not SMTP rejects, so
>> the system generating the bounce is going to have received the message
>> already.  This isn't a situation where you might skip the DATA in a
>> SMTP session.
>>
>> Can you think of any likely scenarios where you'd use a DKIM signed
>> bounce address but not a DKIM signed message?  Again, maybe I'm dense,
>> but I can only think of contrived mailing list examples where the
>> bounce address domain and the From: address domain are different and
>> don't have enough DKIM signatures to use ADSP.  But I'd think the
>> solution wouldn't be signed bounces, it'd be a body signature from the
>> bounce address domain, and an added ADSP keyword to say that you
>> always do that.
>>   
> Well, you could say that if the bounce address isn't valid, then 
> probably the whole message isn't, so just reject it before you start 
> receiving the data.
> 
> However, I'm struggling a bit with the usefulness of BATV as well. I may 
> be missing something, so I'm quite happy to be told that :)
> 
> For BATV to be useful, a domain would have to be able to say that all 
> messages from that domain must have BATV tags. I can't see anything in 
> the spec to cover that requirement. Otherwise a spammer will just send 
> messages without BATV tags, and BATV will achieve nothing other than 
> upsetting some legitimate mailing lists...
> 
> I'm not entirely sure I understand the point of 'private tagging', and 
> since 'public tagging' seems to be documented in an airy-fairy way, I 
> can't see the benefit of that either.
> 
> If public tagging was more tightly specified, and there was some way of 
> saying that a domain must use BATV (possibly piggybacking on DKIM and 
> DKIM policies), then I could see a small point to this, but even then 
> the lack of replay protection seems like it could be a significant 
> weakness for spam protection.
> 
> 
> 
> 

Date: Mon, 19 May 2008 10:16:31 -0400 (EDT)
From: John R Levine <[email-masked]>
To: Paul Smith <[email-masked]>
cc: [email-masked]
Subject: Re: private key BATV is useful
Cleverness: None detected
Sender: [email-masked]

> For BATV to be useful, a domain would have to be able to say that all 
> messages from that domain must have BATV tags. I can't see anything in the 
> spec to cover that requirement. Otherwise a spammer will just send messages 
> without BATV tags, and BATV will achieve nothing other than upsetting some 
> legitimate mailing lists...

The main goal of BATV is to deal with bounce blowback, that is, DSNs due 
to spam sent with forged return addresses.  If your domain is popular 
among spammers, blowback can be a serious issue.  On a bad day, my 
abuse.net domain gets 400,000 bounces for mail it didn't send.

The only thing prvs accomplishes is to let you tell whether an incoming 
bounce was sent in response to a message you actually sent.  If you 
know that you sign all your own mail, you can be reasonably sure that 
bounces to signed addresses are real, and bounces to unsigned addresses 
are fake, give or take the edge cases we've been discussing.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

Date: Mon, 19 May 2008 12:20:16 +0100
From: Paul Smith <[email-masked]>
To: "ietf >> SMTP Interest Group" <[email-masked]>
Subject: SMTP metadata transfer
Sender: [email-masked]

The BATV message here got me thinking.

I can think of some potentially better solutions to the problem that 
BATV is trying to solve, but they would require SMTP extensions with 
extra data after RCPT TO, and the problem with SMTP extensions like that 
is that all servers on the path of the message need to support the 
extension for it to work properly (eg the DSN extension). This is, I 
guess, why BATV is implemented as hacking the return-path address, 
rather than the 'more correct' method of adding a parameter to the MAIL 
FROM command.

So, I wondered, could an extension not be come up with to allow 
'metadata' to be passed with the message. At the moment we have the 
envelope (sender + recipient) and data. What if we could add arbitrary 
data to go along with the 'envelope'. So, all mail servers that support 
the 'metadata' extension just pass on the metadata to the next server in 
the path, even if they don't understand the metadata.

Then, to take the example of DSN, if the initial and final MTAs 
understood DSN, and everything in between understood the generic 
'metadata' extension, DSN would work, even if MTAs in between didn't 
understand DSN.

Yes, for this to work optimally, it would require every MTA to support 
the generic 'metadata' extension - but it would allow new end-to-end 
extensions without everything in between having to support the new 
extensions as well, once they support the generic one.

Two extra ideas:
- potentially, it could be specified that MTAs could see that the 
downstream MTA doesn't support the 'metadata' extension so it could copy 
the metadata into the header somehow, and a later MTA could take it out 
and put it back in the metadata (this would need care to protect against 
spoofed metadata - if it mattered, which I'm not sure it would). This 
would lose some of the benefits - eg if the metadata is a signature of 
the sender/recipient, you wouldn't be able to check it at envelope time, 
it would have to wait until the data was received, but you wouldn't lose 
the functionality

- also, potentially, you could work out some way to copy MAIL FROM, RCPT 
TO, DATA parameters into the metadata, associated with the EHLO keyword, 
and automatically add the parameters into the MAIL FROM if the 
downstream server supports the appropriate extension.

Anyway, this is just a 'throw the idea into the ring' message - I wonder 
what anyone else thinks :)

-- 
Paul Smith

VPOP3 - POP3/SMTP/IMAP4/Webmail Email server for Windows

Date: Mon, 19 May 2008 11:55:39 +0100
From: Paul Smith <[email-masked]>
To: John Levine <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: public key BATV isn't useful
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------050907050907050804020703
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

John Levine wrote:
>>> Oh, and one final note. The document talks a bit about defining a
>>> public key BATV scheme but doesn't actually define anything.
>>>       
>
> Humor me for a moment here.
>
> The idea of a public key BATV is so the system generating the bounce
> can check the signature and not even send bogus bounces.  But any
> system that is going to DKIM sign its bounce addresses would also be
> able to DKIM sign its message bodies, so ADSP discardable already
> allows you to declare that everything is signed so don't bounce the
> unsigned stuff.  We're talking about DSNs here, not SMTP rejects, so
> the system generating the bounce is going to have received the message
> already.  This isn't a situation where you might skip the DATA in a
> SMTP session.
>
> Can you think of any likely scenarios where you'd use a DKIM signed
> bounce address but not a DKIM signed message?  Again, maybe I'm dense,
> but I can only think of contrived mailing list examples where the
> bounce address domain and the From: address domain are different and
> don't have enough DKIM signatures to use ADSP.  But I'd think the
> solution wouldn't be signed bounces, it'd be a body signature from the
> bounce address domain, and an added ADSP keyword to say that you
> always do that.
>   
Well, you could say that if the bounce address isn't valid, then 
probably the whole message isn't, so just reject it before you start 
receiving the data.

However, I'm struggling a bit with the usefulness of BATV as well. I may 
be missing something, so I'm quite happy to be told that :)

For BATV to be useful, a domain would have to be able to say that all 
messages from that domain must have BATV tags. I can't see anything in 
the spec to cover that requirement. Otherwise a spammer will just send 
messages without BATV tags, and BATV will achieve nothing other than 
upsetting some legitimate mailing lists...

I'm not entirely sure I understand the point of 'private tagging', and 
since 'public tagging' seems to be documented in an airy-fairy way, I 
can't see the benefit of that either.

If public tagging was more tightly specified, and there was some way of 
saying that a domain must use BATV (possibly piggybacking on DKIM and 
DKIM policies), then I could see a small point to this, but even then 
the lack of replay protection seems like it could be a significant 
weakness for spam protection.

--------------050907050907050804020703
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
John Levine wrote:
<blockquote cite="mid:[email-masked]"
 type="cite">
  <blockquote type="cite">
    <blockquote type="cite">
      <pre wrap="">Oh, and one final note. The document talks a bit about defining a
public key BATV scheme but doesn't actually define anything.
      </pre>
    </blockquote>
  </blockquote>
  <pre wrap=""><!---->
Humor me for a moment here.

The idea of a public key BATV is so the system generating the bounce
can check the signature and not even send bogus bounces.  But any
system that is going to DKIM sign its bounce addresses would also be
able to DKIM sign its message bodies, so ADSP discardable already
allows you to declare that everything is signed so don't bounce the
unsigned stuff.  We're talking about DSNs here, not SMTP rejects, so
the system generating the bounce is going to have received the message
already.  This isn't a situation where you might skip the DATA in a
SMTP session.

Can you think of any likely scenarios where you'd use a DKIM signed
bounce address but not a DKIM signed message?  Again, maybe I'm dense,
but I can only think of contrived mailing list examples where the
bounce address domain and the From: address domain are different and
don't have enough DKIM signatures to use ADSP.  But I'd think the
solution wouldn't be signed bounces, it'd be a body signature from the
bounce address domain, and an added ADSP keyword to say that you
always do that.
  </pre>
</blockquote>
Well, you could say that if the bounce address isn't valid, then
probably the whole message isn't, so just reject it before you start
receiving the data.<br>
<br>
However, I'm struggling a bit with the usefulness of BATV as well. I
may be missing something, so I'm quite happy to be told that :)<br>
<br>
For BATV to be useful, a domain would have to be able to say that all
messages from that domain must have BATV tags. I can't see anything in
the spec to cover that requirement. Otherwise a spammer will just send
messages without BATV tags, and BATV will achieve nothing other than
upsetting some legitimate mailing lists...<br>
<br>
I'm not entirely sure I understand the point of 'private tagging', and
since 'public tagging' seems to be documented in an airy-fairy way, I
can't see the benefit of that either.<br>
<br>
If public tagging was more tightly specified, and there was some way of
saying that a domain must use BATV (possibly piggybacking on DKIM and
DKIM policies), then I could see a small point to this, but even then
the lack of replay protection seems like it could be a significant
weakness for spam protection.<br>
<br>
<br>
<br>
<br>
</body>
</html>

--------------050907050907050804020703--

Date: Mon, 19 May 2008 10:51:13 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

Dave Crocker wrote:
> 
> So let's hear what constraints and requirements folks deem
> essential for this meta-syntax, and why.  And given that the spec
> already has a proposal, I'll suggest that anyone claiming it is not
> sufficient would help things a lot by explaining why.

In general, the draft misses clear indications and guidelines for the 
implementation. There is not even a single example. Let me quote an 
implementor's perspective: "People should be able to take 5 minutes of 
their time to briefly look through the document, see exactly what's 
going on. Make it crystal clear, right up front, how you go about to 
implement this, and leave the technical details and dry, formal 
specifications, to some other part of the document."

Another point: Is it useful to retrieve the untagged mailbox value? 
(E.g. could that be a way to patch, say, ezmlm?) If it is, it would be 
helpful to have a syntax that delivers a good level of confidence that 
a given token is a BATV-tagged representation of some tagging scheme 
even if that was not known at implementation time. Should the 
tag-types be loadable at runtime from some configuration file? Are 
they case sensitive?

Date: Mon, 19 May 2008 10:24:32 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: public key BATV isn't useful
Sender: [email-masked]

John Levine wrote:
>>> Oh, and one final note. The document talks a bit about defining a
>>> public key BATV scheme but doesn't actually define anything.
> 
> Humor me for a moment here.
> 
> The idea of a public key BATV is so the system generating the bounce
> can check the signature and not even send bogus bounces.  But any
> system that is going to DKIM sign its bounce addresses would also be
> able to DKIM sign its message bodies, so ADSP discardable already
> allows you to declare that everything is signed so don't bounce the
> unsigned stuff.  We're talking about DSNs here, not SMTP rejects, so
> the system generating the bounce is going to have received the message
> already.  This isn't a situation where you might skip the DATA in a
> SMTP session.

OTOH mere prvs doesn't even require a standardization, since it can be 
unilaterally enforced by any admin.

The generic BATV scheme allows different signing styles, e.g. pgp.

Furthermore, Mouss mentioned that the ability to reject a message at 
the RCPT level has its advantages.

Date: 19 May 2008 04:40:17 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: public key BATV isn't useful
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>> Oh, and one final note. The document talks a bit about defining a
>> public key BATV scheme but doesn't actually define anything.

Humor me for a moment here.

The idea of a public key BATV is so the system generating the bounce
can check the signature and not even send bogus bounces.  But any
system that is going to DKIM sign its bounce addresses would also be
able to DKIM sign its message bodies, so ADSP discardable already
allows you to declare that everything is signed so don't bounce the
unsigned stuff.  We're talking about DSNs here, not SMTP rejects, so
the system generating the bounce is going to have received the message
already.  This isn't a situation where you might skip the DATA in a
SMTP session.

Can you think of any likely scenarios where you'd use a DKIM signed
bounce address but not a DKIM signed message?  Again, maybe I'm dense,
but I can only think of contrived mailing list examples where the
bounce address domain and the From: address domain are different and
don't have enough DKIM signatures to use ADSP.  But I'd think the
solution wouldn't be signed bounces, it'd be a body signature from the
bounce address domain, and an added ADSP keyword to say that you
always do that.

R's,
John

Date: Sun, 18 May 2008 18:41:15 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
CC: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

Nothing like asking for comments and then getting them.  Such a pain.

Meta-comment, mostly on the meta-syntax:

    The draft notes that the meta-syntax is expected to have broader utility.
The syntax has been changed before, and there is no religion against
changing it again.  However the challenge is to formulate and agree on the
requirements that mandate change.

    Some concerns impose absolute limits.  Say, for example, we ain't gonna
choose @ as a delimiter.

    Some concerns would be ill-advised, given popular usage.  I suspect "+" is
an example of that.

    Some concerns go too far, because they relate to non-standards that have
minimal deployment.  My current impression is that SRS falls in this category,
and yes, I'm citing it as an example knowing it might be a tad delicate.

So let's hear what constraints and requirements folks deem essential for this
meta-syntax, and why.  And given that the spec already has a proposal, I'll
suggest that anyone claiming it is not sufficient would help things a lot by
explaining why.

My model for the current spec uses

     <registered string> "="

as the template.  As Ned noted it could also be modeled as

    <string> "=" <string> "="

I had thought that the current template would be a sufficiently distinct 
pattern but can well imagine that it needs tweaking.

Again before we can change it, we need to know what the requirements and 
constraints are, for the generic usage.  But we need to be careful not to get 
bogged down.  Not that there is any reason we have to.

[email-masked] wrote:
> Additionally, in order for this to work BATV encapsulation must be uniquely
> identifiable as such, must be removable, and must allow nesting. The scheme the
> draft defines come fairly close to meeting these requirements: Look for two
> equals signs with alphanumeric stuff between them, if found remove up through
> the second equals sign, repeat. However, I do worry that the "signature' of two
> equals signs is not sufficiently unique. At least one obvious conflict is with
> SRS encoding.

What characteristics of the meta-syntax are you seeing as necessary, that the
current spec lacks?

As for nesting, please explain why?  What scenarios are you wanting to cover?

I think there is a strong argument against nesting, for BATV itself, but 
perhaps there are other scenarios that make sense with nesting.  What might 
they be?

> Another, bigger, concern is that the draft seems seriously deficient in
> substantive discussion of the actual handling of BATV addresses. For one thing,
> nowhere do I see an explication of the stripping process I just (casually and
> somewhat incorrectly) described. Including this sort of information is
> essential if you expect implementors to get this stuff right.

I keep switching about this.  On the one hand, yeah, it's easy to imagine 
writing explicit encode/decode steps and plugging them in to some sort of 
larger, abstract service model.  On the other hand, this is a pretty minimal 
mechanism, we have a number of implementation, and no one has felt the spec 
was deficient.  Yes, these were early adopters, with attendant motivations and 
skills.  On the other, this really *is* a simple mechanism.

I guess I'll ask:  What is the added handling discussion intended to enable?

> Another one: A key step in the stripping procedure is that last "repeat",
> because a single address may undergo multiple BATV operations. Maybe I missed

In the case of BATV, I'm pretty sure that that MUST NOT happen.  I think it 
makes no sense to have layers of BATV encoding.  And yes, that probably needs 
to be stated in the spec explicitly.

> Another one: Registration requirements. This part of the draft is obviously
> incomplete so this may not seem like fair game.

Yeah.  That's one's my bad.  Totally forgot to worry about adding the formalities.

> down I'm actually more concerned about something a little different: Without
> specific guidance that pretty much says "if it passes muster syntactically as a
> BATV address it needs to be treated as one for validation purposes", you are
> going to have people attempting to validate the leading tag with some sort of
> registered scheme list. 

Yup.

> Another one: String lengths. One of the issues with BATV and similar schemes is
> that they necessarily increase the length of local parts. You may not have run
> into them, but there are some exceptionally long local parts out there. Some
> discussion of how to handle them is definitely needed (even if all it ends up
> saying is essentially "punt"), and if it is there I couldn't find it.

As John noted:  yup.

> Another one: Quoting. Again, you may run into them, but quoted local parts are
> used in some corners of the Internet. The draft needs to discuss how BATV
> handles quoted local parts. (I suggest dequoting before and requoting after.)

Ack.  Again, I totally missed that point and I agree with you and John on the 
handling.

> Oh, and one final note. The document talks a bit about defining a public key
> BATV scheme but doesn't actually define anything. I personally have no problem
> with this and even applaud the rather brutal comment that "none of the multiple
> existing public key services has yet gained wide adoption", but it strikes me
> as exactly the sort of thing that tends to cause bad cases of panty-wadding
> among the security mavens. I don't know how to finesse it, but some finesse is
> probably in order here.

We decided to leave a generic comment in, both because some of us really like 
the idea of that enhancement and to show that we thought of it. But we can't 
do more than than and we didn't want to do less.  So the current text is 
merely meant to acknowledge one direction of possible enhancement.  I don't 
see that that causes any problems.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Sun, 18 May 2008 19:35:34 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
cc: [email-masked]
Subject: Re: Specification "NOTE" paragraphs and normative language
Sender: [email-masked]

--On Sunday, 18 May, 2008 16:17 -0700 Dave Crocker
<[email-masked]> wrote:

> 
> 
> 
> John C Klensin wrote:
>> My concern is that, if we try to create differentiation,
>> using, e.g., "Note" and "Note well", "Normative note" and
>> "Informational note", or "Note-type-1" and "Note-type-2", the
>> careful readers won't be helped and the less-careful ones
>> won't notice and appreciate the difference. 
> 
> Bottom line:  Folks prefer to have "NOTE" be orthogonal to the
> question of normativeness.  It's role is simply for emphasis,
> no matter whether for mere commentary or for formal
> specification.
> 
> I'm fine with that, but thought it worth asking.

I agree that the question was worth raising.

   john

Date: Sun, 18 May 2008 16:17:20 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: John C Klensin <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: Specification "NOTE" paragraphs and normative language
Sender: [email-masked]

John C Klensin wrote:
> My concern is that, if we try to create differentiation, using,
> e.g., "Note" and "Note well", "Normative note" and
> "Informational note", or "Note-type-1" and "Note-type-2", the
> careful readers won't be helped and the less-careful ones won't
> notice and appreciate the difference. 

Bottom line:  Folks prefer to have "NOTE" be orthogonal to the question of 
normativeness.  It's role is simply for emphasis, no matter whether for mere 
commentary or for formal specification.

I'm fine with that, but thought it worth asking.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Sun, 18 May 2008 19:05:03 -0400
From: John C Klensin <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Specification "NOTE" paragraphs and normative language
Sender: [email-masked]

--On Sunday, 18 May, 2008 07:11 -0700 Dave Crocker
<[email-masked]> wrote:

> Go'day.
> 
> A private comment on the BATV specification raises a basic
> point about specification writing that I thought worth
> pursuing independently:
> 
> Some normative statements are placed in a paragraph labeled
> "NOTE:" so as to emphasize the requirement of the text.  It
> has been suggested that that label actually implies less
> importance, not more.  (In regular prose, a footnote, for
> example, is indeed secondary to the main text.)
> 
> I think it is worth distinguishing text that is secondary from
> normative text that is being emphasized.  So I'm inclined to
> agree that "NOTE:" should be used for non-normative stuff.
> 
> That leaves the question of what convention is appropriate for
> marking emphasized normative text.
> 
> I don't have a ready suggestion, so I thought I'd ask you
> folk...?

Dave,

I think SM is basically right about this.  Let me elaborate...

Having used the "Note" convention (following Jon Postel and RFC
1123 in the case of mail specs), I'd offer the hypothesis that
this is either a non-problem or one that cannot be fixed by
changing terminology.  "Note" is, and should be, used to call
attention to something that might not be obvious from the
preceding or surrounding text.  If the material being noted is
normative, it will ordinarily contain text that makes that very
obvious from context, e.g., one or more of the 2119 words will
be present.  Otherwise, using 2821bis as an example, it looks
like these are purely explanatory, introduce implicit cross
references, or are qualified by terms like "Historical".  

There are exceptions and marginal cases.  One of them appears in
the I-D boilerplate, where "Note" is used to introduce a comment
about "other" I-Ds that may, by now, be largely irrelevant but
which is still mostly explanatory.

My concern is that, if we try to create differentiation, using,
e.g., "Note" and "Note well", "Normative note" and
"Informational note", or "Note-type-1" and "Note-type-2", the
careful readers won't be helped and the less-careful ones won't
notice and appreciate the difference.  Worse, authors and
editors will get it wrong sometimes and that will create far
more confusion than we have today.  We will also open ourselves
to yet another opportunity for battles with IESG members who
would rather find editorial rules to enforce uncritically than
to think carefully about readability and individual situations.
I believe that would be a net loss; YMMD.

   john

Date: 18 May 2008 22:40:52 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>But regardless of how you choose to resolve these issues, the bottom
>line is that it is a mistake to assume that your own experiences with
>things like list operation will necessarily be shared by others. The
>amount of variation out there is really quite astonishing.

Indeed.  I'm collecting info on the way various list management
packages handle addresses in incoming mail on the ASRG wiki at

http://wiki.asrg.sp.am/wiki/List_managers

Contributions encouraged.  (Feel free to send me notes if you don't
want to edit it yourself.)

>You're completely missing my point. This needs to be cast as a
>general local part structuring mechanism, not as something useful for
>this one use case.

Maybe I'm dense, but I don't see what problem a general local part
structuring mechanism solves.  For the MIME example, it's clearly
useful for the MUA to be able to extract opaque chunks of typed stuff
after the message is delivered and hand them off to applications to
interpret them, but I don't see the equivalent here.  The various
prefixing applications likely happen at different points in the mail
handling process.

I also note that VERP, which I believe is the most common use of
embedded addresses, doesn't work with a fixed prefix.

R's,
John

Date: Sun, 18 May 2008 13:26:02 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Specification "NOTE" paragraphs and normative language
Sender: [email-masked]

At 07:11 18-05-2008, Dave Crocker wrote:
>Some normative statements are placed in a paragraph labeled "NOTE:" 
>so as to emphasize the requirement of the text.  It has been 
>suggested that that label actually implies less importance, not 
>more.  (In regular prose, a footnote, for example, is indeed 
>secondary to the main text.)
>
>I think it is worth distinguishing text that is secondary from 
>normative text that is being emphasized.  So I'm inclined to agree 
>that "NOTE:" should be used for non-normative stuff.

There's "Nota Bene" which in IETF parlance is "Note Well".  It's the 
equivalent of "take notice".

In some RFCs, we have "Note that this .." which clarifies or enforces 
a point made in the preceding sentence or paragraph.  There's also:

   "HISTORICAL NOTE:  Several of the mechanisms described in this set of
    documents may seem somewhat strange or even baroque at first reading."

   "FORMATTING NOTE:  Notes, such at this one, provide additional
    nonessential information which may be skipped by the reader without
    missing anything essential."

   "IMPORTANT NOTE:  These mechanisms end up being somewhat gibbous when
    they actually are used."

   "NOTE:  The previous four definitions are clearly circular."

Although the above examples do not contain normative text, there are 
cases where such notes do have them.

>That leaves the question of what convention is appropriate for 
>marking emphasized normative text.

We already have the requirement levels in RFC 2119.

>ps. Or let NOTE be used for emphasizing normative text, and have 
>some other convention for secondary.  Whatever works and is 
>appealing to the community.

A note sometimes provides the context or is a comment based on prior 
observations of implementation behavior.  Basically, it tells us how 
that particular text should be read.

Regards,
-sm 

Date: Sun, 18 May 2008 16:30:22 -0400
From: Tony Hansen <[email-masked]>
To: SMTP Discussion <[email-masked]>
Subject: Re: Specification "NOTE" paragraphs and normative language
Sender: [email-masked]

Dare I say it?

"Note Well:"

	Tony

Bill McQuillan wrote:
> 
> On Sun, 2008-05-18, Dave Crocker wrote:
> 
>> I think it is worth distinguishing text that is secondary from normative text
>> that is being emphasized.  So I'm inclined to agree that "NOTE:" should be
>> used for non-normative stuff.
> 
>> That leaves the question of what convention is appropriate for marking 
>> emphasized normative text.
> 
>> I don't have a ready suggestion, so I thought I'd ask you folk...?
> 
> How about "IMPORTANT:"?
> 

Date: Sun, 18 May 2008 10:31:05 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: Peter J Holzer <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> On 2008-05-17 07:37:47 -0700, [email-masked] wrote:
> >
> > (I haven't looked at BATV in any detail before.)
> >
> > > >Do you have more detail beyond this text in draft-levine-smtp-batv-01.txt
> > > >
> > > >   Mailing Lists:   BATV will cause problems with some mailing lists
> > > >      that identify posters by their bounce address.  The list will not
> >
> > > The only lists I know of that still key on the bounce address are ones
> > > run by ezmlm which isn't very popular any more.
> >
> > > I subscribe to lots of mailman lists, including this one, and listserv
> > > lists without any undue strangeness.
> >
> > Hate to be the bearer of bad news, but the vast majority of the lists operated
> > with our software verify based on the MAIL FROM (bounce) address.

> I did that too for some time. But while only a few users complained,
> they did so very vehemently, so I changed it again.

Many lists. 20+ years of operation. I have yet to receive a single complaint
about MAIL FROM validation. (There have been other complaints, of course.)

> > The reason it is done this way is so rejection can be done during the
> > SMTP dialogue.  Validation based on header From: requires first
> > accepting the message, with all that implies. (We've put a fair bit of
> > effort into making this sort of up-front validation extremely quick.)

> That's not quite true. You can verify the From header and then return a
> 550 response after the "." terminating DATA.

Only if there are no other recipients. There frequently are on list messages
when some list participants are on the same machine as the list.

Now, of course you can always do the trick of 4yz'ing the second and subsequent
recipients. I've been down this road already trying to solve a different
problems  - and it ended up generating quite a few complaints. (And note that
there are other situations where this is unavoidable - for us they tend to
arise when the directory server falls over.) It seems a lot of systems out
there don't handle this all that well. Amazing to me that people will put up
with such brokenness, but there you go.

Another issue is that if you do this sort thing you absolutely must distinguish
between submission and relay somehow,  and lots of places are unable to make
such distinctions. It's one thing to force a automatic relay to retry the
second and subsequent recipients of a message, quite another to force some
random user to resubmit the message over and over in order to get it through.

Of course many of these effects can be mitigated for legitimate posters by
allowing messages with a "legal" address in either the MAIL FROM or header
From:. Since in most cases these will be the same you'll grant access to most
legitimate messages at RCPT TO time. Only when fallback to checking a From:
address is needed does the single recipient requirement need to be enforced.
But now you're talking about fairly complex and, to those not familiar with all
of the ins and outs of SMTP, nonobvious bit of behavior. If you think it's hard
to code, try being on the other end of a conversation with a clueless sysadmin
who's trying to figure out what the hell their MTA is doing when all this is
going on.

But regardless of how you choose to resolve these issues, the bottom line is
that it is a mistake to assume that your own experiences with things like list
operation will necessarily be shared by others. The amount of variation out
there is really quite astonishing.

> So you don't have to accept
> the message, but you have to receive it completely before you can reject
> it. I've written plugins for qpsmtpd which implement this for majordomo
> and mailman mailing-lists.

For my part, I cannot justify spending time on a solution none of our users
have asked for and which has side effects we already know people really don't
like.

> > No, like it or not, there's already quite a lot of validation of MAIL
> > FROM addresses going on out there (indeed, somewhat ironically,
> > increasing such validation is actually the goal of BATV), and BATV
> > imposes additional requirements on such software, which need to be
> > made very very clear in the document if you want this to successfully
> > deploy.

> Speaking of validation, there is one interoperability issue the draft
> doesn't mention: Mail address verification by SMTP callout. For
> verification of envelope senders this works fine, but there are other
> applications where a mail address needs to be verified:

> For example, on some of our websites we have forms where the user needs
> to enter their email address. To reduce the number of bounces and also
> to catch typos when the user can still correct them, we immediately
> connect to the MX and send a MAIL FROM:<>/RCPT TO:$email. I realize that
> using the empty address in the envelope from isn't quite correct (I
> should use the same address that is later used in the confirmation
> mail), but I doubt we are the only ones who do it this way.

Probably not. And this is definitely an issue for BATV.

> > Another, bigger, concern is that the draft seems seriously deficient in
> > substantive discussion of the actual handling of BATV addresses. For one thing,
> > nowhere do I see an explication of the stripping process I just (casually and
> > somewhat incorrectly) described. Including this sort of information is
> > essential if you expect implementors to get this stuff right.

> I agree partially. I recently implemented a BATV plugin for qpsmtpd and
> I noticed that a few details are missing or at least rather vague. OTOH,
> maybe they don't need to be specified. The only scheme currently defined
> is prvs, which can only be validated by the original sender. Since that
> one is presumably using the same software to generate and verify the
> signature, incompatibilities between implementations don't matter. For
> schemes which are intended to be verified on another site, more specific
> rules can be included in the specification of that scheme.

But like it or not, we've observed pretty strong correlations between
specifications that lack detailed instructions and examples and an increase in
implementation botches.

Of course every specification the IETF has ever published could likely be
improved in some way or other. But that's no excuse for failing to pick really
low hanging fruit.

> > Another one: A key step in the stripping procedure is that last "repeat",
> > because a single address may undergo multiple BATV operations. Maybe I missed
> > it, but I could not find any discussion of nesting or repeated application of
> > BATV. This absolutely must be included if there's to be any chance of people
> > getting it right.

> I don't think that nesting BATV makes much sense. Can you provide an
> example?

You're completely missing my point. This needs to be cast as a general local
part structuring mechanism, not as something useful for this one use case.
Imposing structure on local parts that's visible outside of an administative
domain - a place this proposal obviously intends to go - is a very big deal and
needs to be done with sufficient generality that other stuff with different
goals can come along and reuse it. As such, the correct question to ask is,
"Can you justify preventing nesting for all possible future schemes people
might think up?" Nesting should be out of scope if and only if that question
can be answered in the affirmative, and I am extremely skeptical you can do
that.

				Ned

Cc: [email-masked], SMTP Interest Group <[email-masked]>
From: Steve Atkins <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call 
Date: Sun, 18 May 2008 11:53:46 -0700
Sender: [email-masked]

On May 16, 2008, at 9:59 PM, Dave Crocker wrote:

>
> Folks,
>
> Bounce Address Tag Validation (BATV) lets a message origination site  
> add a signature to the rfc2821.mailfrom address, in order to detect  
> invalid return (bounce) messages at the destination.  It has been in  
> development for some time, with a useful amount of deployment  
> experience.
>
> We think it is ready for Proposed Standard and want to ask you, the  
> ietf-smtp mailing list, to do a final review, before we send it in  
> as an independent submission.
>
> Pseudo-Last Call will end on Friday, 31 May.
>
> HTML, PDF and TXT versions are available at the BATV web page, as is  
> a listing of deployment experience:
>
>   <http://mipassoc.org/batv/>
>
> We also would like to hear about any implementations and deployment
> experience, beyond the listings at:
>
>   <http://mipassoc.org/batv/deploy/>.
>
> The more implementation experience we can cite, the strong the case  
> that it is stable and useful.

There's code that sends autoresponses to mail received using the  
original mails 2822 sender (i.e. Reply-to: or From:) as the responses  
2821 recipient and an empty 2821 sender.

The reasoning is that the sender of the autoresponse can't really do  
anything useful with the information that the autoresponse bounced, so  
it's better for the recipient MTA not to send a bounce message, and  
also that anything that'll encourage the receiver of an autoresponse  
not to send any mail at all in response to it is a good thing, and RFC  
2821 section 4.5.5[1] encourages that behaviour.

My understanding is that this use of a null envelope sender follows  
the letter of both 2821 and 2821bis (there's a SHOULD in section  
4.5.5, but that's all)..

This behaviour, in combination with BATV, will cause those  
autoresponses to be reliably discarded. I doubt either end of the  
connection will care, but I thought I'd mention it.

Cheers,
   Steve

[1] "Implementors of automated email processors should be careful to  
make sure that the various kinds of messages with null reverse-path  
are handled correctly, in particular such systems SHOULD NOT reply to  
messages with null reverse-path."

Date: 18 May 2008 18:31:23 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>Speaking of validation, there is one interoperability issue the draft
>doesn't mention: Mail address verification by SMTP callout. For
>verification of envelope senders this works fine, but there are other
>applications where a mail address needs to be verified:

It's certainly an issue.  I thought I mentioned it.

As you've observed, some systems attempt to verify a 2822 address by
doing a fake bounce with a null sender, which BATV doesn't like.
There's a few MTAs which do this as a rather lame anti-spam technique,
notably a plugin to EXIM.

>I noticed that a few details are missing or at least rather vague. OTOH,
>maybe they don't need to be specified. The only scheme currently defined
>is prvs, which can only be validated by the original sender.

Right.

R's,
John

Date: Sun, 18 May 2008 20:16:42 +0200
From: mouss <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

Peter J. Holzer wrote:
> [snip]
>> The reason it is done this way is so rejection can be done during the
>> SMTP dialogue.  Validation based on header From: requires first
>> accepting the message, with all that implies. (We've put a fair bit of
>> effort into making this sort of up-front validation extremely quick.)
>>     
>
> That's not quite true. You can verify the From header and then return a
> 550 response after the "." terminating DATA. So you don't have to accept
> the message, but you have to receive it completely before you can reject
> it. I've written plugins for qpsmtpd which implement this for majordomo
> and mailman mailing-lists.
>   

well, being able to reject before reading data has its advantages. It is 
also more easily implemented on say postfix (postfix header checks are 
global and apply to all mail, so can't be used here).

I guess the implementation difficulty is what makes most MLs accept the 
message during the smtp transaction, then "bounce" later. This could 
someday lead to the same levels of backscatter we are seing these days.

<very tiny corner case>
also, while probably not a real case, what if the user sends to two 
lists, while he is only subscribed to one. rejecting at RCPT TO stage 
still allow mail to the subscribed to list. but I agree that this is 
very tiny corner case...
</very ...>

Date: Sun, 18 May 2008 19:35:43 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: Specification "NOTE" paragraphs and normative language
Sender: [email-masked]

Dave Crocker wrote:
> Some normative statements are placed in a paragraph labeled "NOTE:" so 
> as to emphasize the requirement of the text.  It has been suggested that 
> that label actually implies less importance, not more.  (In regular 
> prose, a footnote, for example, is indeed secondary to the main text.)

The actual degree of conveyed importance depends on the kind of note.
 From http://www.thefreedictionary.com/nota+bene
no-ta be-ne
Abbr. NB
Used to direct attention to something particularly important.
[Latin nota bene, note well : nota, sing. imperative of notare, to 
note + bene, well.]

Date: Sun, 18 May 2008 09:50:46 -0700
From: Bill McQuillan <[email-masked]>
To: SMTP Discussion <[email-masked]>
Subject: Re: Specification "NOTE" paragraphs and normative language
Sender: [email-masked]

On Sun, 2008-05-18, Dave Crocker wrote:

> I think it is worth distinguishing text that is secondary from normative text
> that is being emphasized.  So I'm inclined to agree that "NOTE:" should be
> used for non-normative stuff.

> That leaves the question of what convention is appropriate for marking 
> emphasized normative text.

> I don't have a ready suggestion, so I thought I'd ask you folk...?

How about "IMPORTANT:"?

-- 
Bill McQuillan <[email-masked]>

Date: Sun, 18 May 2008 08:41:25 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

At 02:19 18-05-2008, Peter J. Holzer wrote:
>Speaking of validation, there is one interoperability issue the draft
>doesn't mention: Mail address verification by SMTP callout. For
>verification of envelope senders this works fine, but there are other
>applications where a mail address needs to be verified:

This is a general comment on address verification and not about 
whether it should be done or not.

There was a time when it was recommended to disable VRFY to prevent 
address verification.  Nowadays we get around that with SMTP 
callouts.  Such a practice goes against the explicit policy of the 
receiving MTA.

Regards,
-sm 

Date: Sun, 18 May 2008 07:11:27 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Specification "NOTE" paragraphs and normative language
Sender: [email-masked]

Go'day.

A private comment on the BATV specification raises a basic point about 
specification writing that I thought worth pursuing independently:

Some normative statements are placed in a paragraph labeled "NOTE:" so as to 
emphasize the requirement of the text.  It has been suggested that that label 
actually implies less importance, not more.  (In regular prose, a footnote, 
for example, is indeed secondary to the main text.)

I think it is worth distinguishing text that is secondary from normative text 
that is being emphasized.  So I'm inclined to agree that "NOTE:" should be 
used for non-normative stuff.

That leaves the question of what convention is appropriate for marking 
emphasized normative text.

I don't have a ready suggestion, so I thought I'd ask you folk...?

d/

ps. Or let NOTE be used for emphasizing normative text, and have some other 
convention for secondary.  Whatever works and is appealing to the community.
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Sun, 18 May 2008 11:19:40 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--JP+T4n/bALQSJXh8
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2008-05-17 07:37:47 -0700, [email-masked] wrote:
>=20
> (I haven't looked at BATV in any detail before.)
>=20
> > >Do you have more detail beyond this text in draft-levine-smtp-batv-01.=
txt
> > >
> > >   Mailing Lists:   BATV will cause problems with some mailing lists
> > >      that identify posters by their bounce address.  The list will not
>=20
> > The only lists I know of that still key on the bounce address are ones
> > run by ezmlm which isn't very popular any more.
>=20
> > I subscribe to lots of mailman lists, including this one, and listserv
> > lists without any undue strangeness.
>=20
> Hate to be the bearer of bad news, but the vast majority of the lists ope=
rated
> with our software verify based on the MAIL FROM (bounce) address.

I did that too for some time. But while only a few users complained,
they did so very vehemently, so I changed it again.

> The reason it is done this way is so rejection can be done during the
> SMTP dialogue.  Validation based on header From: requires first
> accepting the message, with all that implies. (We've put a fair bit of
> effort into making this sort of up-front validation extremely quick.)

That's not quite true. You can verify the From header and then return a
550 response after the "." terminating DATA. So you don't have to accept
the message, but you have to receive it completely before you can reject
it. I've written plugins for qpsmtpd which implement this for majordomo
and mailman mailing-lists.

> No, like it or not, there's already quite a lot of validation of MAIL
> FROM addresses going on out there (indeed, somewhat ironically,
> increasing such validation is actually the goal of BATV), and BATV
> imposes additional requirements on such software, which need to be
> made very very clear in the document if you want this to successfully
> deploy.

Speaking of validation, there is one interoperability issue the draft
doesn't mention: Mail address verification by SMTP callout. For
verification of envelope senders this works fine, but there are other
applications where a mail address needs to be verified:

For example, on some of our websites we have forms where the user needs
to enter their email address. To reduce the number of bounces and also
to catch typos when the user can still correct them, we immediately
connect to the MX and send a MAIL FROM:<>/RCPT TO:$email. I realize that
using the empty address in the envelope from isn't quite correct (I
should use the same address that is later used in the confirmation
mail), but I doubt we are the only ones who do it this way.

> Another, bigger, concern is that the draft seems seriously deficient in
> substantive discussion of the actual handling of BATV addresses. For one =
thing,
> nowhere do I see an explication of the stripping process I just (casually=
 and
> somewhat incorrectly) described. Including this sort of information is
> essential if you expect implementors to get this stuff right.

I agree partially. I recently implemented a BATV plugin for qpsmtpd and
I noticed that a few details are missing or at least rather vague. OTOH,
maybe they don't need to be specified. The only scheme currently defined
is prvs, which can only be validated by the original sender. Since that
one is presumably using the same software to generate and verify the
signature, incompatibilities between implementations don't matter. For
schemes which are intended to be verified on another site, more specific
rules can be included in the specification of that scheme.

> Another one: A key step in the stripping procedure is that last "repeat",
> because a single address may undergo multiple BATV operations. Maybe I mi=
ssed
> it, but I could not find any discussion of nesting or repeated applicatio=
n of
> BATV. This absolutely must be included if there's to be any chance of peo=
ple
> getting it right.

I don't think that nesting BATV makes much sense. Can you provide an
example?

	hp

--=20
   _  | Peter J. Holzer    | It took a genius to create [TeX],
|_|_) | Sysadmin WSR       | and it takes a genius to maintain it.
| |   | [email-masked]         | That's not engineering, that's art.
__/   | http://www.hjp.at/ |	-- David Kastrup in comp.text.tex

--JP+T4n/bALQSJXh8
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--JP+T4n/bALQSJXh8--

Date: Sat, 17 May 2008 22:10:25 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
CC: [email-masked]
Subject: Re: BATV pseudo-Last Call
Sender: [email-masked]

[email-masked] wrote:
> We don't use any sort of external package. Our MTA handles lists natively;
> it has done so for longer than most of the list processing packages have 
> been in existance.

Maybe 25 years?

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Sat, 17 May 2008 16:00:07 -0400 (EDT)
From: John R Levine <[email-masked]>
To: Ned Freed <[email-masked]>
cc: [email-masked]
Subject: Re: BATV pseudo-Last Call
Cleverness: None detected
Sender: [email-masked]

> You're assuming that the only purpose for this scheme is this specific 
> sort of signature. Sorry, if we're going to define a generic 
> encapsulation format for local parts -

Holy Mission Creep, Batman.  I certainly don't want do do that and doubt 
that anyone else involved with BATV does.  That would be a quicksand 
filled swamp of continental proportions.  I'm not very interested in SRS, 
which merely compounds the wrongness that is SPF, but VERP in various 
forms is used very heavily by all sorts of mailing list and other bulk 
mail packages, and there's something called SES which is sort of DK with 
the signature coded into the bounce address.

They address different problems.  BATV is to tell who was the sender of a 
message that caused a bounce.  SRS is to tell who was the sender of a 
message that caused a forward.  VERP is to tell who was the recipient of a 
message that caused a bounce.  SES is to tell whether a bounce address 
matches the message's contents.

BATV has the particular interaction with mailing lists that makes it 
useful to tell whether a particular BATV address is the same sender as a 
prior BATV or non-BATV sender, but I don't see that as an issue with any 
of the others, so I don't see any benefit in creating a common syntax.

Incidentally, BATV and VERP work just fine together as is, viz. this 
bounce address on a message I just sent myself:

<prvs=001659e973=testlist-owner+m1=jrlevine2=[email-masked]>

>> I'm astonished.  What package do you use?
>
> We don't use any sort of external package. Our MTA handles lists natively; it
> has done so for longer than most of the list processing packages have been in
> existance.

Ah, it's old.  I think you will discover that you are in dwindling 
minority if you're keying on the 2821 address.  I haven't done a formal 
survey but it is my strong impression that all of the widely used list 
management packages these days other than ezmlm use the body address.  I 
guess it's worth doing some research on packages that people actually use. 
(Preferably with significant grant funding.)

>> On the theory that standards shouldn't speculate about the future, we
>> really should take this part out other than say that the syntax is
>> designed to permit other signing schemes to be added.
>
> Sigh. You're almost certainly correct, but i have to say I like what's there
> and wish we could keep it.

Either it should go or we should implement it.  The main blocking item was 
the lack of a source of public keys, but DKIM may have solved that 
problem.  This may be worth turning into a WG, although I'd like to get 
BATV out in finite time so I'd rather do BATV now, WG later.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

Date: Sat, 17 May 2008 11:04:41 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: John R Levine <[email-masked]>
Cc: Ned Freed <[email-masked]>, [email-masked], [email-masked]
Sender: [email-masked]

> > Hate to be the bearer of bad news, but the vast majority of the lists
> > operated with our software verify based on the MAIL FROM (bounce)
> > address.

> I'm astonished.  What package do you use?

We don't use any sort of external package. Our MTA handles lists natively; it
has done so for longer than most of the list processing packages have been in
existance.

> I'm on a lot of lists and it's
> really the case that other than a few ezmlm lists, they all use the body
> From: address.

Just goes to show how risky it is to place too much reliance on your own
personal experience. (I'm a serial offender in this regard as well, but honest,
your honor, I'm trying to break the habit.) The Internet is a very big place
and there are lots of behavioral enclaves out there.

> > Additionally, in order for this to work BATV encapsulation must be uniquely
> > identifiable as such, must be removable,

> Right, it does that.

> > and must allow nesting.

> Nope.  Remember that BATV depends on a shared secret betweeen signer and
> the MX for the domain.  It only makes sense to sign if you know the key
> for the domain.  So if a signed message isn't in one a domain for which
> you have the signing key, you have no business signing it, while if it is
> in such a domain, you (or one of your friends) have already signed it.

You're assuming that the only purpose for this scheme is this specific sort of
signature. Sorry, if we're going to define a generic encapsulation format for
local parts - a HUGE implementation burden when you consider how much stuff
needs to change -  it needs to be able to accomodate more than that.

Mind you, a generic encapsulation without a specific purpose won't deploy
either - there has to be a "hook", something specific that it brings to the
table that will get people to buy in to it. And one defined scheme is is a fine
hook IMO. But when the opportunity presents itself to do something like this it
is nothing short of folly to not make it as flexibile as possible as
long as it remains simple enough.

> If you're thinking of something like VERP that embeds an arbitrary address
> in a bounce address, I suppose you're right, but BATV doesn't introduce
> any new problems that VERP doesn't already have.

I'm not thinking of anything specific, and that's sort of the point. This is
like the original MIME debates, where there was a vocal minority that thought
MIME was overstepping its bounds in providing more than the ability to put
accented characters into email content, while we felt that as long as we were
going to pry open the message body format we might as well provide a framework
for solving a much more general set of problems.

Think where we'd be if we had caved to that approach - most likely we'd be
operating some sort of insane X.400-with-Internet-addresses hybrid. Yuck.

> > Another one: A key step in the stripping procedure is that last "repeat",
> > because a single address may undergo multiple BATV operations.

> Again, no.  We certainly need to clarify why double BATV signing is a bug.

Double use of a particular BATV scheme may well be a bug. Double use of the
format in general may not be.

> > Another one: Registration requirements.

> Right.  I'd think it would be easy enough to add a registry for tags and
> populate it with prvs=.  How upset would people be to see semantics
> declared for local parts?  (Although I'd think that battle was lost a
> decade ago with X.400 addresses.)

The situations are not really comparable. First, it has always been possible to
impose your own semantics on the local parts associated with any domains you
have administrative authority over. This is all the various X.400 mapping
schemes ended up doing - you defined a domain (or overlaid an existing domain)
in such a way that it gets stuff to an X.400 gateway and then marvel at the
wierd and wonderful addresses that come in and out. (We actually defined
similar schemes for FAX and a bunch of the LAN email systems. Used to be there
was lots of demand for gateways to all this stuff.)

Now, I'm well aware that there's some verbiage in a couple of places in the
various X.400-related RFCs that kinda sorta implies that if you saw something
like:

    "/admd= /prmd=doof/s=doe/g=john/"@someone-elses-domain

it was OK to drop the domain and treat the local part as an X.400 address. But
while this may have been the veiled intent of some subset of the X.400
supporters (I was never quite sure), there was never any consensus behind it. I
know this because I was there - I specifically raised the issue and was told
quite specifically that this was not the intent and therefore it was nothing to
worry about. Since it was already clear at that point that X.400 was, in the
immortal words of Marshall Rose, "road kill on the information superhighway", I
was mainly interested in gateway issues within an enclave and never bothered to
press the point and get the text clarified to say that you can only interpet
X.400-style local parts when its your domain on the right.

But even if this had been the intent and there had been a consensus behind it,
the reality is X.400 addresses never achieved sufficient regularity for this to
actually work, so anyone who tried it quickly got their fingers burned. (The
magic space in the /admd= clause has a lot to do this but now is not the time
to review this bit of ancient history.) One of the reasons we have a multistep
standards process is so that when consensus arrives at a bogus result it can be
corrected. If there was any internet in moving the X.400 documents along this
would be one of many things that would have to be addressed.

Nor is this comparable to some other proposals that basically tried to impose
structure on local parts in such a way that basically all address handling
would have to be updafed. No, this is somewhere in the middle - the vast
majority of address processing need not change, but some types of address
validity processing need to know enough to remove the tag(s). And I do think
there is sufficient apetite to have some additioanl tools for address
verification to carry this forward.

> > Another one: String lengths.

> You're right, it's a problem.  The hand-wave is to assert that few MTAs
> enforce the 64 character limit, but I realize that's not very persuasive
> for a standard.  On the other hand, since there's no nesting, the damage
> is at least predictable.  The current spec adds 16 characters to each
> address but I've been considering a modified version with 10 characters
> that addresses some greylisting problems as well.

Yes, and we're indeed one of those that doesn't enforce the 64 character limit,
although we do limit the entire address to 256 characters.

> > Another one: Quoting. Again, you may run into them, but quoted local
> > parts are used in some corners of the Internet.

> Good point.  I agree that unquote, batv or unbatv, requote is the only
> approach that makes sense.

Yep. Having already been down this path with SRS support, and IMO it's the only
workable approach.

> > Oh, and one final note. The document talks a bit about defining a public key
> > BATV scheme but doesn't actually define anything.

> On the theory that standards shouldn't speculate about the future, we
> really should take this part out other than say that the syntax is
> designed to permit other signing schemes to be added.

Sigh. You're almost certainly correct, but i have to say I like what's there
and wish we could keep it.

				Ned

Date: Sat, 17 May 2008 13:57:39 -0400 (EDT)
From: John R Levine <[email-masked]>
To: Ned Freed <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: BATV pseudo-Last Call
Cleverness: None detected
Sender: [email-masked]

> Hate to be the bearer of bad news, but the vast majority of the lists 
> operated with our software verify based on the MAIL FROM (bounce) 
> address.

I'm astonished.  What package do you use?  I'm on a lot of lists and it's 
really the case that other than a few ezmlm lists, they all use the body 
From: address.

> Additionally, in order for this to work BATV encapsulation must be uniquely
> identifiable as such, must be removable,

Right, it does that.

> and must allow nesting.

Nope.  Remember that BATV depends on a shared secret betweeen signer and 
the MX for the domain.  It only makes sense to sign if you know the key 
for the domain.  So if a signed message isn't in one a domain for which 
you have the signing key, you have no business signing it, while if it is 
in such a domain, you (or one of your friends) have already signed it.

If you're thinking of something like VERP that embeds an arbitrary address 
in a bounce address, I suppose you're right, but BATV doesn't introduce 
any new problems that VERP doesn't already have.

> Another one: A key step in the stripping procedure is that last "repeat",
> because a single address may undergo multiple BATV operations.

Again, no.  We certainly need to clarify why double BATV signing is a bug.

> Another one: Registration requirements.

Right.  I'd think it would be easy enough to add a registry for tags and 
populate it with prvs=.  How upset would people be to see semantics 
declared for local parts?  (Although I'd think that battle was lost a 
decade ago with X.400 addresses.)

> Another one: String lengths.

You're right, it's a problem.  The hand-wave is to assert that few MTAs 
enforce the 64 character limit, but I realize that's not very persuasive 
for a standard.  On the other hand, since there's no nesting, the damage 
is at least predictable.  The current spec adds 16 characters to each 
address but I've been considering a modified version with 10 characters 
that addresses some greylisting problems as well.

> Another one: Quoting. Again, you may run into them, but quoted local 
> parts are used in some corners of the Internet.

Good point.  I agree that unquote, batv or unbatv, requote is the only 
approach that makes sense.

> Oh, and one final note. The document talks a bit about defining a public key
> BATV scheme but doesn't actually define anything.

On the theory that standards shouldn't speculate about the future, we 
really should take this part out other than say that the syntax is 
designed to permit other signing schemes to be added.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

Date: Sat, 17 May 2008 07:37:47 -0700 (PDT)
From: [email-masked]
Subject: Re: BATV pseudo-Last Call
To: John Levine <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

(I haven't looked at BATV in any detail before.)

> >Do you have more detail beyond this text in draft-levine-smtp-batv-01.txt
> >
> >   Mailing Lists:   BATV will cause problems with some mailing lists
> >      that identify posters by their bounce address.  The list will not

> The only lists I know of that still key on the bounce address are ones
> run by ezmlm which isn't very popular any more.

> I subscribe to lots of mailman lists, including this one, and listserv
> lists without any undue strangeness.

Hate to be the bearer of bad news, but the vast majority of the lists operated
with our software verify based on the MAIL FROM (bounce) address. The reason it
is done this way is so rejection can be done during the SMTP dialogue.
Validation based on header From: requires first accepting the message, with all
that implies. (We've put a fair bit of effort into making this sort of up-front
validation extremely quick.)

I for one would be EXTREMELY reluctant to reconfigure the lists I run to use
some other address, because doing so would mean I have little choice but to
review at least some of the messages that are rejected by list policy.

Now, the obvious way to address this so that everyone wins is to BATV-enable
list processing software. I have no objection to this approach, but simply
saying "this may cause problems with list software" in the draft and then
justifying this here by saying in effect "the limited subset of list sofware I
know about doesn't have this problem, oh, except for this one over there that I
don't think is very popular". No, like it or not, there's already quite a lot
of validation of MAIL FROM addresses going on out there (indeed, somewhat
ironically, increasing such validation is actually the goal of BATV), and BATV
imposes additional requirements on such software, which need to be made very
very clear in the document if you want this to successfully deploy.

Additionally, in order for this to work BATV encapsulation must be uniquely
identifiable as such, must be removable, and must allow nesting. The scheme the
draft defines come fairly close to meeting these requirements: Look for two
equals signs with alphanumeric stuff between them, if found remove up through
the second equals sign, repeat. However, I do worry that the "signature' of two
equals signs is not sufficiently unique. At least one obvious conflict is with
SRS encoding.

Another, bigger, concern is that the draft seems seriously deficient in
substantive discussion of the actual handling of BATV addresses. For one thing,
nowhere do I see an explication of the stripping process I just (casually and
somewhat incorrectly) described. Including this sort of information is
essential if you expect implementors to get this stuff right.

Another one: A key step in the stripping procedure is that last "repeat",
because a single address may undergo multiple BATV operations. Maybe I missed
it, but I could not find any discussion of nesting or repeated application of
BATV. This absolutely must be included if there's to be any chance of people
getting it right.

Another one: Registration requirements. This part of the draft is obviously
incomplete so this may not seem like fair game. (And this alone, incidentally,
means this isn't even clsoe to ready for last call for proposed. Come on, do
you really think an IANA Considerations Section that says in its entirety "It
may be desirable to establish a registry of BATV tagging schemes and tag
types." is good enough?) But while the registration details have to be nailed
down I'm actually more concerned about something a little different: Without
specific guidance that pretty much says "if it passes muster syntactically as a
BATV address it needs to be treated as one for validation purposes", you are
going to have people attempting to validate the leading tag with some sort of
registered scheme list. 

Another one: String lengths. One of the issues with BATV and similar schemes is
that they necessarily increase the length of local parts. You may not have run
into them, but there are some exceptionally long local parts out there. Some
discussion of how to handle them is definitely needed (even if all it ends up
saying is essentially "punt"), and if it is there I couldn't find it.

Another one: Quoting. Again, you may run into them, but quoted local parts are
used in some corners of the Internet. The draft needs to discuss how BATV
handles quoted local parts. (I suggest dequoting before and requoting after.)

In summary, the idea behind BATV is sound, but this draft needs a lot of work
before it will be ready for last call for proposed. At a minimum it needs to
impose specific requirements on agents that perform address comparison
operations, the registrations requirements need to be nailed down and spelled
out, and a lot more detail as to what constitutes correct handling of BATV
addresses needs to be provided.

Oh, and one final note. The document talks a bit about defining a public key
BATV scheme but doesn't actually define anything. I personally have no problem
with this and even applaud the rather brutal comment that "none of the multiple
existing public key services has yet gained wide adoption", but it strikes me
as exactly the sort of thing that tends to cause bad cases of panty-wadding
among the security mavens. I don't know how to finesse it, but some finesse is
probably in order here.

				Ned

Date: 17 May 2008 13:20:11 -0000
From: John Levine <[email-masked]>
To: [email-masked]
Subject: Re: BATV pseudo-Last Call
Organization: 
Cc: [email-masked]
Sender: [email-masked]

>Do you have more detail beyond this text in draft-levine-smtp-batv-01.txt
>
>   Mailing Lists:   BATV will cause problems with some mailing lists
>      that identify posters by their bounce address.  The list will not

The only lists I know of that still key on the bounce address are ones
run by ezmlm which isn't very popular any more.

I subscribe to lots of mailman lists, including this one, and listserv
lists without any undue strangeness.

R's,
John

From: "Dan Wing" <[email-masked]>
To: <[email-masked]>, <[email-masked]>, "'SMTP Interest Group'" <[email-masked]>
Subject: RE: BATV pseudo-Last Call 
Date: Sat, 17 May 2008 00:47:02 -0700
Sender: [email-masked]

> We also would like to hear about any implementations and deployment
> experience, beyond the listings at:
> 
>     <http://mipassoc.org/batv/deploy/>.
> 
> The more implementation experience we can cite, the strong 
> the case that it is stable and useful.

Do you have more detail beyond this text in draft-levine-smtp-batv-01.txt

   Mailing Lists:   BATV will cause problems with some mailing lists
      that identify posters by their bounce address.  The list will not
      recognize the identical MailFrom addresses, because it will
      interpret the differing BATV attributes as part of the address.
      These services will either reject postings or pass them all to the
      moderator.

As this very concern is why I have not considered enabling BATV for a domain
that I help manage.

Specifically, I am interested in knowing if this is a problem in
significantly-deployed mailing list software such as, for example, mailman,
listserv, or Yahoo Groups.

Thanks,
-d

Date: Fri, 16 May 2008 21:59:33 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked], SMTP Interest Group <[email-masked]>
Subject: BATV pseudo-Last Call 
Sender: [email-masked]

Folks,

Bounce Address Tag Validation (BATV) lets a message origination site add a 
signature to the rfc2821.mailfrom address, in order to detect invalid return 
(bounce) messages at the destination.  It has been in development for some 
time, with a useful amount of deployment experience.

We think it is ready for Proposed Standard and want to ask you, the ietf-smtp 
mailing list, to do a final review, before we send it in as an independent 
submission.

Pseudo-Last Call will end on Friday, 31 May.

HTML, PDF and TXT versions are available at the BATV web page, as is a listing 
of deployment experience:

    <http://mipassoc.org/batv/>

We also would like to hear about any implementations and deployment
experience, beyond the listings at:

    <http://mipassoc.org/batv/deploy/>.

The more implementation experience we can cite, the strong the case that it is 
stable and useful.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Content-class: urn:content-classes:message
Subject: Preferred Originator
Date: Fri, 16 May 2008 11:26:49 +0530
From: "Santoshkumar K S" <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

This is a multi-part message in MIME format.

------_=_NextPart_001_01C8B719.A10FDF34
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi,

            Can any one explain preferredRFC822Originator in the
forwarding mails by MTA ?      =20

=20

Thanks and Regards,

Santhosh Kumar. K. S | Barcelona | Techmahindra Ltd.

KRM Centre, 2 Harrington Road, 5th Floor, Chetpet, Chennai 600 031=20

Office +91 44 6621 2000 | Ext 2147 | Cell:9994488067

Email: [email-masked]

www.techmahindra.com <http://www.techmahindra.com/> =20

=20

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=20
Disclaimer:

This message and the information contained herein is proprietary and=
 confidential and subject to the Tech Mahindra policy statement, you may=
 review the policy at <a href=
=3D"http://www.techmahindra.com/Disclaimer.html">http://www.techmahindra.co=
m/Disclaimer.html</a> externally and <a href=
=3D"http://tim.techmahindra.com/Disclaimer.html">http://tim.techmahindra.co=
m/Disclaimer.html</a> internally within Tech Mahindra.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
------_=_NextPart_001_01C8B719.A10FDF34
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=
=3D"urn:schemas-microsoft-com:office:word" xmlns:st1=
=3D"urn:schemas-microsoft-com:office:smarttags" xmlns=
=3D"http://www.w3.org/TR/REC-html40">

<head>
<meta http-equiv=3DContent-Type content=3D"text/html; charset=3Dus-ascii">
<meta name=3DGenerator content=3D"Microsoft Word 11 (filtered medium)">
<o:SmartTagType namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"
 name=3D"Street"/>
<o:SmartTagType namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"
 name=3D"address"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
 /* Font Definitions */
 @font-face
	{font-family:Verdana;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
	{font-family:Chiller;
	panose-1:4 2 4 4 3 16 7 2 6 2;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman";}
a:link, span.MsoHyperlink
	{color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:Arial;
	color:windowtext;}
@page Section1
	{size:8.5in 11.0in;
	margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
	{page:Section1;}
-->
</style>

</head>

<body lang=3DEN-US link=3Dblue vlink=3Dpurple>

<div class=3DSection1>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span style=
=3D'font-size:10.0pt;
font-family:Arial'>Hi,<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span style=
=3D'font-size:10.0pt;
font-family:Arial'>&nbsp;&nbsp;=
 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Can
any one explain preferredRFC822Originator in the forwarding mails by MTA=
 ?&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span style=
=3D'font-size:10.0pt;
font-family:Arial'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DVerdana><span
style=3D'font-size:10.0pt;font-family:Verdana;color:black'>Thanks and=
 Regards,</span></font><o:p></o:p></p>

<p class=3DMsoNormal><font size=3D2 color=3Dred face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:red'>Santhosh Kumar. K. S | Barcelona |
Techmahindra Ltd.</span></font><o:p></o:p></p>

<p class=3DMsoNormal><font size=3D2 color=3Dgray face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:gray'>KRM Centre, <st1:Street w:st=
=3D"on"><st1:address
 tabIndex=3D"0"
 style=3D"BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE:=
 url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x"
 w:st=3D"on"><U3:STREET ns1:tabIndex=3D"0" ns1:style=
=3D"BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE:=
 url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x" u2:endInsDate=
=3D"2007-10-31T15:23:00Z" u2:endInsAuthor=3D"sj0019711" u2:insDate=
=3D"2007-10-31T15:23:00Z" u2:insAuthor=3D"sj0019711"><U3:ADDRESS=
 ns1:tabIndex=3D"0" ns1:style=3D"BACKGROUND-POSITION: left bottom;=
 BACKGROUND-IMAGE: url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT:=
 repeat-x" u2:endInsDate=3D"2007-10-31T15:23:00Z" u2:endInsAuthor=
=3D"sj0019711" u2:insDate=3D"2007-10-31T15:23:00Z" u2:insAuthor=
=3D"sj0019711"><U1:STREET u2:st=3D"on"><U1:ADDRESS u2:st=3D"on">2
  Harrington=
 Road</st1:address></st1:Street></U1:ADDRESS></U1:STREET></U3:ADDRESS></U3:=
STREET>,
5th Floor, Chetpet, Chennai 600 031 <o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dgray face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:gray'>Office +91 44 6621 2000 | Ext 2147 |
Cell:9994488067<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dgray face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:gray'>Email:=
 [email-masked]<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D4 face=3DChiller><span style=
=3D'font-size:14.0pt;
font-family:Chiller'><a href=3D"http://www.techmahindra.com/"
title=3D"http://www.techmahindra.com/"><font size=3D2 color=3Dgray face=
=3DVerdana><span
style=3D'font-size:10.0pt;font-family:Verdana;color:gray'><span
title=3D"http://www.techmahindra.com/"><span title=
=3D"http://www.techmahindra.com/"><span
title=
=3D"http://www.techmahindra.com/">www.techmahindra.com</span></span></span>=
</font></span></a>
<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span style=
=3D'font-size:
12.0pt'><o:p>&nbsp;</o:p></span></font></p>

</div>

</body>

</html>

<table><tr><td bgcolor=3D#ffffff><font color=3D#000000>=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br>
 <br>
Disclaimer:<br>
<br>
This message and the information contained herein is proprietary and=
 confidential and subject to the Tech Mahindra policy statement, you may=
 review the policy at <a href=
=3D"http://www.techmahindra.com/Disclaimer.html">http://www.techmahindra.co=
m/Disclaimer.html</a> externally and <a href=
=3D"http://tim.techmahindra.com/Disclaimer.html">http://tim.techmahindra.co=
m/Disclaimer.html</a> internally within Tech Mahindra.<br>
<br>
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D<br>
</font></td></tr></table>
------_=_NextPart_001_01C8B719.A10FDF34--

From: "Sabahattin Gucukoglu" <[email-masked]>
To: "Santoshkumar K S" <[email-masked]>
Date: Wed, 14 May 2008 09:36:45 +0100
Subject: Re: Additional parameters to SMTP commands
CC: [email-masked]
Content-description: Mail message body
Sender: [email-masked]

Hi Santoshkumar,

On 14 May 2008 at 12:30, Santoshkumar K S <[email-masked]> said:

>                Any one give me the link for Additional parameters for
> SMTP MAIL and RCPT commands

The MAIL and RCPT commands themselves are described in RFC 2821:
http://www.ietf.org/rfc/rfc2821.txt

A list of IANA-registered service extensions for the SMTP protocol, which 
could define new parameters for MAIL and RCPT commands, is here:
http://www.iana.org/assignments/mail-parameters

Cheers,
Sabahattin

- -- 
Sabahattin Gucukoglu <mail<at>sabahattin<dash>gucukoglu<dot>com>
Address harvesters, snag this: [email-masked]
Phone: +44 20 88008915
Mobile: +44 7986 053399
http://sabahattin-gucukoglu.com/

Date: Wed, 14 May 2008 04:14:48 -0400
From: John C Klensin <[email-masked]>
To: Santoshkumar K S <[email-masked]>, [email-masked]
Subject: Re: Additional parameters to SMTP commands
Sender: [email-masked]

--On Wednesday, 14 May, 2008 12:30 +0530 Santoshkumar K S
<[email-masked]> wrote:

>                Any one give me the link for Additional
> parameters for SMTP MAIL and RCPT commands

Read RFC 2821 or, probably better at this point,
http://www.ietf.org/internet-drafts/draft-klensin-rfc2821bis-10.txt

Each of these parameters, or sets of parameters, are associated
with a particular SMTP extension.  You will need to track the
extensions down to determine the syntax and semantics of the
ones you are interested in.

There is an IANA registry (see "SMTP Service Extensions" at
http://www.iana.org/assignments/mail-parameters) that gives a
list of the current extensions and references to the documents
where they, and their parameters, are defined, but it may not be
complete.

    john

Content-class: urn:content-classes:message
Subject: Additional parameters to SMTP commands
Date: Wed, 14 May 2008 12:30:15 +0530
From: "Santoshkumar K S" <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

This is a multi-part message in MIME format.

------_=_NextPart_001_01C8B590.1A760F3F
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi,

               Any one give me the link for Additional parameters for
SMTP MAIL and RCPT commands

=20

Thanks and Regards,

Santhosh Kumar. K. S | Barcelona | Techmahindra Ltd.

KRM Centre, 2 Harrington Road, 5th Floor, Chetpet, Chennai 600 031=20

Office +91 44 6621 2000 | Ext 2147 | Cell:9994488067

Email: [email-masked]

www.techmahindra.com <http://www.techmahindra.com/> =20

=20

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=20
Disclaimer:

This message and the information contained herein is proprietary and=
 confidential and subject to the Tech Mahindra policy statement, you may=
 review the policy at <a href=
=3D"http://www.techmahindra.com/Disclaimer.html">http://www.techmahindra.co=
m/Disclaimer.html</a> externally and <a href=
=3D"http://tim.techmahindra.com/Disclaimer.html">http://tim.techmahindra.co=
m/Disclaimer.html</a> internally within Tech Mahindra.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
------_=_NextPart_001_01C8B590.1A760F3F
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=
=3D"urn:schemas-microsoft-com:office:word" xmlns:st1=
=3D"urn:schemas-microsoft-com:office:smarttags" xmlns=
=3D"http://www.w3.org/TR/REC-html40">

<head>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; charset=
=3Dus-ascii">
<meta name=3DGenerator content=3D"Microsoft Word 11 (filtered medium)">
<o:SmartTagType namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"
 name=3D"Street"/>
<o:SmartTagType namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"
 name=3D"address"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
 /* Font Definitions */
 @font-face
	{font-family:Chiller;
	panose-1:4 2 4 4 3 16 7 2 6 2;}
@font-face
	{font-family:Verdana;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman";}
a:link, span.MsoHyperlink
	{color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:Arial;
	color:windowtext;}
@page Section1
	{size:8.5in 11.0in;
	margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
	{page:Section1;}
-->
</style>

</head>

<body lang=3DEN-US link=3Dblue vlink=3Dpurple>

<div class=3DSection1>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span style=
=3D'font-size:10.0pt;
font-family:Arial'>Hi,<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span style=
=3D'font-size:10.0pt;
font-family:Arial'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp; Any one give me the link for Additional
parameters for SMTP MAIL and RCPT commands<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span style=
=3D'font-size:10.0pt;
font-family:Arial'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DVerdana><span
style=3D'font-size:10.0pt;font-family:Verdana;color:black'>Thanks and=
 Regards,</span></font><o:p></o:p></p>

<p class=3DMsoNormal><font size=3D2 color=3Dred face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:red'>Santhosh Kumar. K. S | Barcelona |
Techmahindra Ltd.</span></font><o:p></o:p></p>

<p class=3DMsoNormal><font size=3D2 color=3Dgray face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:gray'>KRM Centre, <st1:Street w:st=
=3D"on"><st1:address
 tabIndex=3D"0"
 style=3D"BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE:=
 url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x"
 w:st=3D"on"><U3:STREET ns1:tabIndex=3D"0" ns1:style=
=3D"BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE:=
 url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x" u2:endInsDate=
=3D"2007-10-31T15:23:00Z" u2:endInsAuthor=3D"sj0019711" u2:insDate=
=3D"2007-10-31T15:23:00Z" u2:insAuthor=3D"sj0019711"><U3:ADDRESS=
 ns1:tabIndex=3D"0" ns1:style=3D"BACKGROUND-POSITION: left bottom;=
 BACKGROUND-IMAGE: url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT:=
 repeat-x" u2:endInsDate=3D"2007-10-31T15:23:00Z" u2:endInsAuthor=
=3D"sj0019711" u2:insDate=3D"2007-10-31T15:23:00Z" u2:insAuthor=
=3D"sj0019711"><U1:STREET u2:st=3D"on"><U1:ADDRESS u2:st=3D"on">2
  Harrington=
 Road</st1:address></st1:Street></U1:ADDRESS></U1:STREET></U3:ADDRESS></U3:=
STREET>,
5th Floor, Chetpet, Chennai 600 031 <o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dgray face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:gray'>Office +91 44 6621 2000 | Ext 2147 |
Cell:9994488067<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dgray face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:gray'>Email:=
 [email-masked]<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D4 face=3DChiller><span style=
=3D'font-size:14.0pt;
font-family:Chiller'><a href=3D"http://www.techmahindra.com/"
title=3D"http://www.techmahindra.com/"><font size=3D2 color=3Dgray face=
=3DVerdana><span
style=3D'font-size:10.0pt;font-family:Verdana;color:gray'><span
title=3D"http://www.techmahindra.com/"><span title=
=3D"http://www.techmahindra.com/"><span
title=
=3D"http://www.techmahindra.com/">www.techmahindra.com</span></span></span>=
</font></span></a>
<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span style=
=3D'font-size:
12.0pt'><o:p>&nbsp;</o:p></span></font></p>

</div>

</body>

</html>

<table><tr><td bgcolor=3D#ffffff><font color=3D#000000>=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br>
 <br>
Disclaimer:<br>
<br>
This message and the information contained herein is proprietary and=
 confidential and subject to the Tech Mahindra policy statement, you may=
 review the policy at <a href=
=3D"http://www.techmahindra.com/Disclaimer.html">http://www.techmahindra.co=
m/Disclaimer.html</a> externally and <a href=
=3D"http://tim.techmahindra.com/Disclaimer.html">http://tim.techmahindra.co=
m/Disclaimer.html</a> internally within Tech Mahindra.<br>
<br>
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D<br>
</font></td></tr></table>
------_=_NextPart_001_01C8B590.1A760F3F--

From: "Sabahattin Gucukoglu" <[email-masked]>
To: [email-masked]
Date: Thu, 08 May 2008 20:31:56 +0100
Subject: Re: MX lookup error handling change
Content-description: Mail message body
Sender: [email-masked]

On 8 May 2008 at 18:50, Tony Finch <[email-masked]> said:

> On Thu, 8 May 2008, Robert A. Rosenberg wrote:

> > OTOH, if/when the highest priority is ONLY CNAMEs (even if lower/back-up
> > MXs are A/AAAA which you can support) do an immediate reject since the
> > back-up MXs will never be able to deliver to the primary servers (due to
> > their being invalidly defined as CNAMEs).
> 
> That's not necessarily true: (1) Backup MXs don't have to use the standard
> routing algorithm to deliver email. (2) If the verifying MTA is IPv4-only,
> the primary MX is IPv6-only, and the backup MX is dual-protocol, your
> proposed algorithm will falsely bounce the message.

The backup MXs you didn't plan on sending mail to may, furthermore, not 
mind CNAMEs, even if you do.  From the implementation point of view, 
CNAMEs can be highly transparent.

To make this algorithm work, you have to intentionally look for an error 
condition at the primary.  You would no longer be seeking a next hop 
within your IPvX capabilities only using the standardised search.  I think 
that's outside my scope for a clean implementation, but others may feel 
differently.

Cheers,
Sabahattin

- -- 
Sabahattin Gucukoglu <mail<at>sabahattin<dash>gucukoglu<dot>com>
Address harvesters, snag this: [email-masked]
Phone: +44 20 88008915
Mobile: +44 7986 053399
http://sabahattin-gucukoglu.com/

Date: Thu, 8 May 2008 19:14:14 +0100
From: Tony Finch <[email-masked]>
To: SM <[email-masked]>
cc: [email-masked]
Subject: Re: MX lookup error handling change
Sender: [email-masked]

On Thu, 8 May 2008, SM wrote:
>
> >    When a domain name associated with an MX RR is looked up and the
> >    associated data field obtained, the data field of that response MUST
> >    contain a domain-name.  That domain-name, when queried, MUST return
> >    at least one address record (e.g., A or AAAA RR) that gives the IP
> >    address of the SMTP server to which the message should be directed.
> >    Any other response, specifically including a value that will return a
> >    CNAME record when queried, lies outside the scope of this standard.
>
> I assume that you are referring to the last sentence.

All of it, really. If the MX records MUST point to domains that refer to
IP address records, then someone might read that as permission to treat
the whole lot as broken - especially since this requirement applies when a
sender looks up an MX record, not when the recipient configures it.

Though, as you point out, since the sentence from RFC 2821 is still
present, perhaps it's not as bad as I thought at first.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
LUNDY FASTNET: SOUTHEAST VEERING NORTH OR NORTHWEST 3 OR 4. SLIGHT OR
MODERATE. OCCASIONAL RAIN, FOG PATCHES. MODERATE OR GOOD, OCCASIONALLY VERY
POOR.

Date: Thu, 08 May 2008 11:01:58 -0700
To: Tony Finch <[email-masked]>, [email-masked]
From: SM <[email-masked]>
Subject: Re: MX lookup error handling change
Sender: [email-masked]

At 04:55 08-05-2008, Tony Finch wrote:
>RFC 2821 says:
>
>    If MX records are present, but none of them are usable, this situation
>    MUST be reported as an error.
>
>This implies that a partially broken RRset is not grounds for rejecting a
>message.

In Section 5.1 of draft-10, there is:

  "If MX records are present, but none of them are usable, this situation
   MUST be reported as an error."

which is exactly as the above.

>2821bis says:
>
>    When a domain name associated with an MX RR is looked up and the
>    associated data field obtained, the data field of that response MUST
>    contain a domain-name.  That domain-name, when queried, MUST return
>    at least one address record (e.g., A or AAAA RR) that gives the IP
>    address of the SMTP server to which the message should be directed.
>    Any other response, specifically including a value that will return a
>    CNAME record when queried, lies outside the scope of this standard.
>
>This implies that it's OK to reject partially broken MX RRsets. Some
>deployed software already does this.

I assume that you are referring to the last sentence.  Quoting the 
sentence following it from the draft:

   "Any other response, specifically including a value that will return a
    CNAME record when queried, lies outside the scope of this standard.
    The prohibition on labels in the data that resolve to CNAMEs is
    discussed in more detail in RFC 2181, Section 10.3 [31]."

This is more of a clarification about the CNAME issue that was 
discussed previously.  It also addresses the case where the MX lookup 
returns an IP address instead of a domain name.  I don't read that as 
a change in MX lookup error handling as the grounds for rejection are 
still when none of the RRs are usable.

Regards,
-sm 

Date: Thu, 8 May 2008 18:50:54 +0100
From: Tony Finch <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>
cc: [email-masked]
Subject: Re: MX lookup error handling change
Sender: [email-masked]

On Thu, 8 May 2008, Robert A. Rosenberg wrote:
>
> If there are two or more MX records at the highest priority (ie: preferred)
> and some point at CNAMEs and some point at A/AAAA then while the MXs are
> partly broken, the CNAMEs can be ignored (ie: Treated as Unreachable) and the
> A/AAAA records used for those addresses you have IPvX connectivity for. If
> none are usable then reject.

Yes, that's what RFC 2821 says.

> OTOH, if/when the highest priority is ONLY CNAMEs (even if lower/back-up
> MXs are A/AAAA which you can support) do an immediate reject since the
> back-up MXs will never be able to deliver to the primary servers (due to
> their being invalidly defined as CNAMEs).

That's not necessarily true: (1) Backup MXs don't have to use the standard
routing algorithm to deliver email. (2) If the verifying MTA is IPv4-only,
the primary MX is IPv6-only, and the backup MX is dual-protocol, your
proposed algorithm will falsely bounce the message.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
SHANNON: SOUTHWEST 3 OR 4 BECOMING VARIABLE 3 OR LESS. MODERATE. SHOWERS, FOG
PATCHES. MODERATE OR GOOD, OCCASIONALLY VERY POOR.

Date: Thu, 8 May 2008 12:54:21 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: MX lookup error handling change
Sender: [email-masked]

At 12:55 +0100 on 05/08/2008, Tony Finch wrote about MX lookup error 
handling change:

>RFC 2821 says:
>
>    If MX records are present, but none of them are usable, this situation
>    MUST be reported as an error.
>
>This implies that a partially broken RRset is not grounds for rejecting a
>message.
>
>2821bis says:
>
>    When a domain name associated with an MX RR is looked up and the
>    associated data field obtained, the data field of that response MUST
>    contain a domain-name.  That domain-name, when queried, MUST return
>    at least one address record (e.g., A or AAAA RR) that gives the IP
>    address of the SMTP server to which the message should be directed.
>    Any other response, specifically including a value that will return a
>    CNAME record when queried, lies outside the scope of this standard.
>
>This implies that it's OK to reject partially broken MX RRsets. Some
>deployed software already does this.

If there are two or more MX records at the highest priority (ie: 
preferred) and some point at CNAMEs and some point at A/AAAA then 
while the MXs are partly broken, the CNAMEs can be ignored (ie: 
Treated as Unreachable) and the A/AAAA records used for those 
addresses you have IPvX connectivity for. If none are usable then 
reject. OTOH, if/when the highest priority is ONLY CNAMEs (even if 
lower/back-up MXs are A/AAAA which you can support) do an immediate 
reject since the back-up MXs will never be able to deliver to the 
primary servers (due to their being invalidly defined as CNAMEs).

Date: Thu, 08 May 2008 09:11:51 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Tony Finch <[email-masked]>
CC:  [email-masked]
Subject: Re: MX lookup error handling change
Sender: [email-masked]

Tony Finch wrote:
> RFC 2821 says:
> 
>    If MX records are present, but none of them are usable, this situation
>    MUST be reported as an error.
> 
> This implies that a partially broken RRset is not grounds for rejecting a
> message.

Doesn't this depend on what the MX lookup intention was?

> 2821bis says:
> 
>    When a domain name associated with an MX RR is looked up and the
>    associated data field obtained, the data field of that response MUST
>    contain a domain-name.  That domain-name, when queried, MUST return
>    at least one address record (e.g., A or AAAA RR) that gives the IP
>    address of the SMTP server to which the message should be directed.
>    Any other response, specifically including a value that will return a
>    CNAME record when queried, lies outside the scope of this standard.
> 
> This implies that it's OK to reject partially broken MX RRsets. Some
> deployed software already does this.
> 
> Tony.

I don't see the change.

Obviously, if you are attempting to send mail out, you can't pull a 
rabbit out a hat if the mail host service is not available.

Its a delayed reject - aka bounce.

But if we thinking in terms of verification? i.e., CBV or Web Service 
Signup Email Address Verification Tool?

Would there be any semantics difference in MX error handling?

The only difference I see is the number of attempts the system is 
willing to invest to determine *WHEN* an email domain is invalid via bad 
MX/RRSet.

Maybe we should consider some old Fidonet Ideas? like a ZONE MAIL HOUR:

     NEW IDEA FOR 2821BIS:

     Section X.1 - SMTP ZONE MAIL HOUR

     All SMTP systems MUST be open for mail services between the ZULU
     (GMT) hours of 00:00 and 01:00 called the ZONE MAIL HOUR (ZMH).
     During ZMH all compliant SMTP services *MUST* make sure their
     MX and RRset are resolvable and an SMTP SERVICE PORT is available.

     During ZMH, senders *MUST* reject mail intended for SMTP
     servers whose MX/RRSet are not resolved and/or do not connect.

     During non-ZMH hours, Senders *MAY* reject but they are
     encouraged to re-queue the mail attempt.

     Senders MAY schedule all daily outgoing mail to ZMH if they wish
     to implement a strong rejection policy.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Thu, 8 May 2008 12:55:16 +0100
From: Tony Finch <[email-masked]>
To: [email-masked]
Subject: MX lookup error handling change
Sender: [email-masked]

RFC 2821 says:

   If MX records are present, but none of them are usable, this situation
   MUST be reported as an error.

This implies that a partially broken RRset is not grounds for rejecting a
message.

2821bis says:

   When a domain name associated with an MX RR is looked up and the
   associated data field obtained, the data field of that response MUST
   contain a domain-name.  That domain-name, when queried, MUST return
   at least one address record (e.g., A or AAAA RR) that gives the IP
   address of the SMTP server to which the message should be directed.
   Any other response, specifically including a value that will return a
   CNAME record when queried, lies outside the scope of this standard.

This implies that it's OK to reject partially broken MX RRsets. Some
deployed software already does this.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
FAIR ISLE: SOUTHEASTERLY 3 OR 4, INCREASING 5 OR 6 LATER. SLIGHT OR MODERATE.
FAIR. GOOD.

Content-class: urn:content-classes:message
Subject: Forwarding in SMTP
Date: Thu, 8 May 2008 12:46:41 +0530
From: "Santoshkumar K S" <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

This is a multi-part message in MIME format.

------_=_NextPart_001_01C8B0DB.767F19EC
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hi,

         Can anyone explain the forwarding in SMTP ?

             =20

=20

Thanks and Regards,

Santhosh Kumar. K. S | Barcelona | Techmahindra Ltd.

KRM Centre, 2 Harrington Road, 5th Floor, Chetpet, Chennai 600 031=20

Office +91 44 6621 2000 | Ext 2147 | Cell:9994488067

Email: [email-masked]

www.techmahindra.com <http://www.techmahindra.com/> =20

=20

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=20
Disclaimer:

This message and the information contained herein is proprietary and=
 confidential and subject to the Tech Mahindra policy statement, you may=
 review the policy at <a href=
=3D"http://www.techmahindra.com/Disclaimer.html">http://www.techmahindra.co=
m/Disclaimer.html</a> externally and <a href=
=3D"http://tim.techmahindra.com/Disclaimer.html">http://tim.techmahindra.co=
m/Disclaimer.html</a> internally within Tech Mahindra.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
------_=_NextPart_001_01C8B0DB.767F19EC
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=
=3D"urn:schemas-microsoft-com:office:word" xmlns:st1=
=3D"urn:schemas-microsoft-com:office:smarttags" xmlns=
=3D"http://www.w3.org/TR/REC-html40">

<head>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; charset=
=3Dus-ascii">
<meta name=3DGenerator content=3D"Microsoft Word 11 (filtered medium)">
<o:SmartTagType namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"
 name=3D"Street"/>
<o:SmartTagType namespaceuri=3D"urn:schemas-microsoft-com:office:smarttags"
 name=3D"address"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
 /* Font Definitions */
 @font-face
	{font-family:Chiller;
	panose-1:4 2 4 4 3 16 7 2 6 2;}
@font-face
	{font-family:Verdana;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman";}
a:link, span.MsoHyperlink
	{color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:Arial;
	color:windowtext;}
@page Section1
	{size:8.5in 11.0in;
	margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
	{page:Section1;}
-->
</style>

</head>

<body lang=3DEN-US link=3Dblue vlink=3Dpurple>

<div class=3DSection1>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span style=
=3D'font-size:10.0pt;
font-family:Arial'>Hi,<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span style=
=3D'font-size:10.0pt;
font-family:Arial'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Can=
 anyone explain the forwarding in SMTP ?<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span style=
=3D'font-size:10.0pt;
font-family:Arial'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp; <o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span style=
=3D'font-size:10.0pt;
font-family:Arial'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dblack face=3DVerdana><span
style=3D'font-size:10.0pt;font-family:Verdana;color:black'>Thanks and=
 Regards,</span></font><o:p></o:p></p>

<p class=3DMsoNormal><font size=3D2 color=3Dred face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:red'>Santhosh Kumar. K. S | Barcelona |
Techmahindra Ltd.</span></font><o:p></o:p></p>

<p class=3DMsoNormal><font size=3D2 color=3Dgray face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:gray'>KRM Centre, <st1:Street w:st=
=3D"on"><st1:address
 tabIndex=3D"0"
 style=3D"BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE:=
 url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x"
 w:st=3D"on"><U3:STREET ns1:tabIndex=3D"0" ns1:style=
=3D"BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE:=
 url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x" u2:endInsDate=
=3D"2007-10-31T15:23:00Z" u2:endInsAuthor=3D"sj0019711" u2:insDate=
=3D"2007-10-31T15:23:00Z" u2:insAuthor=3D"sj0019711"><U3:ADDRESS=
 ns1:tabIndex=3D"0" ns1:style=3D"BACKGROUND-POSITION: left bottom;=
 BACKGROUND-IMAGE: url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT:=
 repeat-x" u2:endInsDate=3D"2007-10-31T15:23:00Z" u2:endInsAuthor=
=3D"sj0019711" u2:insDate=3D"2007-10-31T15:23:00Z" u2:insAuthor=
=3D"sj0019711"><U1:STREET u2:st=3D"on"><U1:ADDRESS u2:st=3D"on">2
  Harrington=
 Road</st1:address></st1:Street></U1:ADDRESS></U1:STREET></U3:ADDRESS></U3:=
STREET>,
5th Floor, Chetpet, Chennai 600 031 <o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dgray face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:gray'>Office +91 44 6621 2000 | Ext 2147 |
Cell:9994488067<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dgray face=3DVerdana><span style=
=3D'font-size:
10.0pt;font-family:Verdana;color:gray'>Email:=
 [email-masked]<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D4 face=3DChiller><span style=
=3D'font-size:14.0pt;
font-family:Chiller'><a href=3D"http://www.techmahindra.com/"
title=3D"http://www.techmahindra.com/"><font size=3D2 color=3Dgray face=
=3DVerdana><span
style=3D'font-size:10.0pt;font-family:Verdana;color:gray'><span
title=3D"http://www.techmahindra.com/"><span title=
=3D"http://www.techmahindra.com/"><span
title=
=3D"http://www.techmahindra.com/">www.techmahindra.com</span></span></span>=
</font></span></a>
<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span style=
=3D'font-size:
12.0pt'><o:p>&nbsp;</o:p></span></font></p>

</div>

</body>

</html>

<table><tr><td bgcolor=3D#ffffff><font color=3D#000000>=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br>
 <br>
Disclaimer:<br>
<br>
This message and the information contained herein is proprietary and=
 confidential and subject to the Tech Mahindra policy statement, you may=
 review the policy at <a href=
=3D"http://www.techmahindra.com/Disclaimer.html">http://www.techmahindra.co=
m/Disclaimer.html</a> externally and <a href=
=3D"http://tim.techmahindra.com/Disclaimer.html">http://tim.techmahindra.co=
m/Disclaimer.html</a> internally within Tech Mahindra.<br>
<br>
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D<br>
</font></td></tr></table>
------_=_NextPart_001_01C8B0DB.767F19EC--

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
To: "Frank Ellermann" <[email-masked]>
Subject: Re: SPF instead of MX?  was:Queued Mail or Unreturnable Mail?
Date: Tue, 6 May 2008 10:36:47 -0700
Sender: [email-masked]

On May 5, 2008, at 1:18 PM, Frank Ellermann wrote:
> Douglas Otis wrote:
>
>> Local-part expansion and the "exists" macros seem of minimal  
>> utility for legitimate purposes.
>
> The possible purposes are obvious, "per user" policies treating the  
> local part as labels, and "exists" be a a way to get around the  
> "check at the border" limitation for MAIL FROM, by definition HELO  
> checks work everywhere.

To inhibit simple spoofing, separately labeled resource records would  
be required for each user and for each outbound server.  A Per User  
strategy also exposes valid recipients by way of DNS.  Per User  
controls should occur opaquely and before a message is sent to prevent  
harvesting abuse.  Per User unique records also place inordinate  
resource burdens upon already heavily taxed receiving MTAs.

> I'm still curious how minimal such uses are, and if they could be  
> simply deprecated as nice try.

A few domains use "exists" to selectively issue negative responses.   
The complexity, burden, and security hazard imposed by the exists and  
local-part macro mechanisms hardly justify their limited utility.   
These mechanisms should be dropped due to security concerns.  The fact  
that few domains use these mechanisms should make this decision easier.

> On the border of legitimate is using "exists" for logging, receivers  
> are likely not interested in this job.  OTOH "exists" is not the  
> only way to arrange logging.

A way to log "This receiving MTA executes dangerous scripts."

>> What handling occurs for messages resulting in NEUTRAL?
>
> Specified with not less than a MUST is "like NONE".  But what  
> receivers in practise do is "receiver policy", they could decide  
> that a Gmail NEUTRAL or a Paypal SOFTFAIL is bad.  When they decide  
> this it is their decision, no part of the protocol as specified.   
> The same goes for a "best guess" approach for domains without  
> published policy.

A receiver's questionable behaviour justifies a domain publishing an  
address list followed by '+all' since proper handling of NEUTRAL is in  
doubt.  Otherwise, white-listing using this mechanism would place  
forwarded messages at risk.

> Your MX idea, roughly "v=spf1 mx -all", is near to a best guess  
> "v=spf1 a/24 mx/24 -all", but nobody would dare put this into a RFC  
> for various good reasons.

The purpose of publishing MX records is not to support typical SPF  
default "best guess" strategies.  The MX record would clearly indicate  
whether the domain supports SMTP.  This mandate protects domains not  
publishing an MX record, and greatly simplifies the means to determine  
whether a domain is being spoofed within an email address.  Such  
determination can be used to bypass other resource intensive checks,  
such as wildcarded SPF record sets, ADSP, etc.

>> Impediments inhibiting new resource record types weigh heavily  
>> against promoting ubiquitous publication for yet another optional  
>> SMTP feature.
>
> The spammers recruit new SPF publishers, anybody finding his inbox  
> flooded with misdirected bounces is motivated to learn this within  
> hours, not days.  Fortunately the esoteric macro and "exists"  
> features are unnecessary for ordinary my.domain.example users.

The esoteric SPF macros nevertheless represent a potential for harm.   
Make this process safer.  SPF plug-ins should exclude processing  
exists mechanisms and local-part macros.

>> A mandate to publish MX records will not impact where messages  
>> might originate as this does.  However, the mandate will impact  
>> where policy records such as SPF or ADSP are to be found.
>
> Then I misunderstood your goal, are you "only" talking about "v=spf1  
> -all" for domains never sending any mail ? Some folks here  
> volunteered to address this issue in an I-D after the debate to  
> limit the A-fallback to IPv4 in 2821bis ended inconclusive.

It is not clear where this might lead.  It is absurd to suggest all  
other domains should publish bogus MX records targeting the root  
domain or SPF records declaring no mail is sent, to avoid having a few  
remaining percent of the domains who _are_ using SMTP bother to  
publish MX records.  To say the least, bogus MX records or negative  
SPF records will not scale, and would be most unfair to the Internet  
in general.

> Solving it generally is anyway better, and hopefully a solution does  
> not need "v=spf1 -all" or null-MX opt-out hacks, opt-in is so much  
> better.

Agreed.  MX should clearly indicate the domain's support of SMTP.   
SMTP receivers are best able to make exceptions in special and perhaps  
critical cases.

>> DKIM provides a means to identify sources
>
> Simplified DKIM is a "signed timestamp line" indicating some kind of  
> responsibility.  It's not about backscatter and not about delivery  
> reports "to the originator of the undeliverable mail (as indicated  
> by the reverse-path)" -that's what I thought this thread is about.

Actually, DKIM is a bit more robust than some realize.  The body hash  
is included within the signature, which allows signature verification  
in many cases for NDNs as well.  A DKIM NDN rescue strategy when  
dealing with back-scatter.

Placing and checking a simple authentication hash within the Message- 
Id by MUAs would be another method to thwart backscatter techniques.   
Perhaps someone should write a draft similar to the PRVS strategy  
found in BATV for use by MUAs in the message id.  Each user could add  
their own pass-phrase.  With this type of checking, users would never  
be fooled by spoofed NDNs.  Broad deployment and standardization of  
MUA checking by the industry should greatly discourage spoofing of the  
return-path without further burdening SMTP.

-Doug

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: Queued Mail or Unreturnable Mail?
Date:  Mon, 5 May 2008 22:18:51 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 82
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Douglas Otis wrote:
=20
> Local-part expansion and the "exists" macros seem of =20
> minimal utility for legitimate purposes.

The possible purposes are obvious, "per user" policies
treating the local part as labels, and "exists" be a
a way to get around the "check at the border" limitation
for MAIL FROM, by definition HELO checks work everywhere.

I'm still curious how minimal such uses are, and if they
could be simply deprecated as nice try.  On the border
of legitimate is using "exists" for logging, receivers
are likely not interested in this job.  OTOH "exists" is
not the only way to arrange logging.

> What handling occurs for messages resulting in NEUTRAL?

Specified with not less than a MUST is "like NONE".  But
what receivers in practise do is "receiver policy", they
could decide that a Gmail NEUTRAL or a Paypal SOFTFAIL is
bad.  When they decide this it is their decision, no part
of the protocol as specified.  The same goes for a "best
guess" approach for domains without published policy.

Your MX idea, roughly "v=3Dspf1 mx -all", is near to a best
guess "v=3Dspf1 a/24 mx/24 -all", but nobody would dare put
this into a RFC for various good reasons.

> NEUTRAL represents the same status "as if" no SPF=20
> records exist.

RFC 4408 says, yes.  And receivers do what they like with
it, as they could accept a FAIL instead of rejecting it,
no matter how bad that can be, or discard SOFTFAIL, or
treat a PASS from unknown strangers as spam on Tuesdays.

They have some additional info to "get it right" in some
cases like PASS or FAIL, but they are free to use this
info or its NEUTRAL absence in other ways.  If they are
confused they could evaluate v=3Dspf1 against Message-IDs
or a PRA, and claim that the spam recognition rate is
better than a random generator.

They can't claim that they found this recipe in RFC 4408,
like folks adopting a "discard them all" strategy cannot
say that this is what 2821bis wants them to do.

> Impediments inhibiting new resource record types weigh
> heavily against promoting ubiquitous publication for yet
> another optional SMTP feature.

The spammers recruit new SPF publishers, anybody finding
his inbox flooded with misdirected bounces is motivated
to learn this within hours, not days.  Fortunately the
esoteric macro and "exists" features are unnecessary for
ordinary my.domain.example users.

> A mandate to publish MX records will not impact where
> messages might originate as this does.  However, the
> mandate will impact where policy records such as SPF
> or ADSP are to be found.

Then I misunderstood your goal, are you "only" talking
about "v=3Dspf1 -all" for domains never sending any mail ?
Some folks here volunteered to address this issue in an
I-D after the debate to limit the A-fallback to IPv4 in
2821bis ended inconclusive.

Solving it generally is anyway better, and hopefully a
solution does not need "v=3Dspf1 -all" or null-MX opt-out
hacks, opt-in is so much better.

> DKIM provides a means to identify sources

Simplified DKIM is a "signed timestamp line" indicating
some kind of responsibility.  It's not about backscatter
and not about delivery reports "to the originator of the=20
undeliverable mail (as indicated by the reverse-path)" -
that's what I thought this thread is about.

 Frank

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
To: "Frank Ellermann" <[email-masked]>
Subject: Re: Queued Mail or Unreturnable Mail?
Date: Mon, 5 May 2008 10:56:00 -0700
Sender: [email-masked]

On May 5, 2008, at 4:38 AM, Frank Ellermann wrote:
> Douglas Otis wrote:
>
>> Mention of SPF should be accompanied with security admonishments  
>> not to expand evaluation macros.
>
> The fine print is a bit more convoluted, most macros are simply  
> shorthands for "this domain", "space", or similar harmless cases.
>
> What you are talking about is the "this local part" macro, and some  
> esoteric contructs to extract labels from "this local part".  I'd  
> love to know if anybody actually uses these baroque features.  In  
> one case I'm aware of the policy hit the RFC 4408 processing limits,  
> "PermError", tilt, game over.

Macro expansions of email-address components may induce recipients  
into generating DNS transactions from cached SPF records.  This  
feature dramatically reduces resources expended for staging DDoS  
attacks.  Indeed, expansion of local-parts permits the greatest  
amplifications.  Local-part expansion and the "exists" macros seem of  
minimal utility for legitimate purposes.

>> these lists are often incomplete and allow NEUTRAL or SOFT-FAIL  
>> results.
>
> For the "accept and bounce later" question a PASS is good enough.   
> It's also a start for white listing and reputation schemes.  I also  
> like either PASS or FAIL better, but sometimes that's no option...

Some recognize no SPF authorization can be complete.  What handling  
occurs for messages resulting in NEUTRAL?

>> Such results are easily exploited.
>
> ...exploiting NEUTRAL makes no sense, it is just what you have  
> without XYZ (not limited to SPF), you won't get a bonus for trying  
> in vain.  Quite the contrary, after receivers wasted their time and  
> bandwidth for nothing (NEUTRAL) they could treat it slightly worse  
> than only NEUTRAL.

SPF reduces delivery integrity?  NEUTRAL represents the same status  
"as if" no SPF records exist.

> Abusing SOFTFAIL cannot work, it is already tricky to use it in any  
> meaningful way.

SPF records must be evaluated?

>  It's in the direction of DKIM-ADSP "all", it lets receivers do  
> "something" with it, anything from "ignore" to "discard them all".
>
>> SPF's use of generic TXT records at base domains is unlikely to  
>> completely transition to the service specific resource record, and  
>> will conflict with future protocols and revisions.
>
> For wildcard uses that is already the case, sooner or later folks  
> hopefully decide that using SPF records is better.  If there is ever  
> a v=spf3 it will build on the new SPF RR alone, and not abuse TXT.

Impediments inhibiting new resource record types weigh heavily against  
promoting ubiquitous publication for yet another optional SMTP  
feature.  At least mandating publication of MX records permits  
discovery consolidations.

>> expecting MX for acceptance eliminates publishing or retrieving  
>> other SMTP related records within sub-domains lacking MX records.
>
> A decree that everybody should have "v=spf1 mx -all" implicitly  
> (without saying so) for all addresses, and similarly "v=spf1 a -all"  
> for HELOs, is a nice idea in the spirit of RFC 821, but I fear it  
> comes decades too late.

A mandate to publish MX records will not impact where messages might  
originate as this does.  However, the mandate will impact where policy  
records such as SPF or ADSP are to be found.

> Assume for a moment that a "v=spf1 mx -all" could be mandatory.   
> Then publishing SPF policies stating this fact would be unnecessary,  
> and we could get away with a variant of BATV.  But obviously that's  
> not the case, folks use other IPs to send, not only the MX IPs, and  
> some policies aggregate more than one sending ISP.

DKIM provides a means to identify sources without reliance upon IP  
address path registration that is unlikely to scale.  Especially while  
the IP address space increases in complexity.

-Doug

Date: Mon, 05 May 2008 13:34:12 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Sabahattin Gucukoglu <[email-masked]>
CC:  [email-masked]
Subject: Re: Queued Mail or Unreturnable Mail?
Sender: [email-masked]

Sabahattin Gucukoglu wrote:
> On Mon, 05 May 2008 11:51:30 +0200, Alessandro Vesely <[email-masked]> 
> wrote:

>> That question seems ill-conditioned to me. It is semantically correct 
 >> to reject mail because the *recipient* is undeliverable.
> 
> The question is written as intended, but we appear to be in agreement.
> 
> I still want to know why sender verification is a good idea, though.  If 
> the recipient validation is sufficient to reduce backscatter and queue 
> cloggage, why *are* we insisting upon a semantic disapproval of invalid 
> sender addresses?  

Huh?  To address the junk that does address the user's inbox?

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Date: Mon, 05 May 2008 13:28:01 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Sabahattin Gucukoglu <[email-masked]>
CC:  [email-masked]
Subject: Re: Queued Mail or Unreturnable Mail?
Sender: [email-masked]

Sabahattin Gucukoglu wrote:
> Hi all,
> 
> Before this discussion goes further, I want to just get this bit sorted 
> out, just in case people don't see a relationship between the two 
> contenders in this dilema (as indicated in the Subject line) ...
> 
> On Fri, 02 May 2008 05:11:33 -0400, Hector Santos <[email-masked]> 
> wrote:
>> Sabahattin Gucukoglu wrote:
> [...]
>>> 1.  I am naive enough to receive mail from non-verifiable senders. 
>> Yes. <g>
> 
> To be sure, it sounds like the right thing to do.  As yet, though, there's 
> no justification in doing it other than that queue clogging will often 
> result if you don't.  

Well, I don't see that as the key reason.  It just a matter of checking 
the SMTP client.  By our last 2006 site specific stats, rejection 
breakdown was:

As the transaction proceeds at each state (We check RCPT before MAIL first):

     31% EHLO/HELO, Bad Invalid Syntax, Format
     43% MAIL FROM, Delayed Sender Verification

         17% Internal Local Rules/blocks
         52% DNS RBL
          2% SPF
         29% CBV (CallBack Verification)

     34% RCPT TO, Bad Recipients
     37% DATA, Sysop defined Mail Content Local Rules

For DATA, in our specific site case, we have a simple "Bad Words" filter 
list.  Nothing fancy.

For local recipients tranactions, one of the benefits is no bounces.

If you are hosting and relaying mail, you can reduce bounces as well, 
but you can still get them because you don't know if the forwarding path 
is acceptable until its actually processed.

> Is it *semantically* correct to reject mail just 
> because bounces for the sender are undeliverable?  

IMO, since it is a SMTP requirement for the return path to be valid, it 
would be more than semantically correct and IMO, a reasonable 
justification for verification.

The question is when do you think it to be valid?  Before acceptance or 
after acceptance? or only when its needed?   The latter seems to be 
modus operandi of most systems and they are the ones that typically 
suffer the most (or bare the blunt of the issue), and because of that 
they are usually the ones with the most undelivered mail scenarios. i.e, 
they will discard mail. IMO, the systems that accept all mail before 
they check it are usually the ones with debatable issues.

> If it isn't, fine - 
> let's get to work on standardising a way for domains to indicate that they 
> don't want mail, and they'll live happily ever afterward not knowing that 
> mail they originated (or not, but that's irrelevant) didn't get through.  
> We can try and fail to deliver our DSNs straight away, keep backup MXs 
> working without fuss, etc, etc, and still have nice, clean queues without 
> going to extraordinary, expensive lengths to verify sender addresses.

I don't follow this and the rest of your statements.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
To: Sabahattin Gucukoglu <[email-masked]>
Subject: Re: Queued Mail or Unreturnable Mail?
Date: Mon, 5 May 2008 09:58:05 -0700
Sender: [email-masked]

On May 5, 2008, at 6:20 AM, Sabahattin Gucukoglu wrote:

>
> On Mon, 05 May 2008 11:51:30 +0200, Alessandro Vesely <[email-masked]>
> wrote:
>> Sabahattin Gucukoglu wrote:
>>> To be sure, it sounds like the right thing to do.  As yet, though,  
>>> there's no justification in doing it other than that queue  
>>> clogging will often result if you don't.  Is it *semantically*  
>>> correct to reject mail just because bounces for the sender are  
>>> undeliverable?
>>
>> That question seems ill-conditioned to me. It is semantically  
>> correct to reject mail because the *recipient* is undeliverable.
>
> The question is written as intended, but we appear to be in agreement.
>
> I still want to know why sender verification is a good idea,  
> though.  If the recipient validation is sufficient to reduce  
> backscatter and queue cloggage, why *are* we insisting upon a  
> semantic disapproval of invalid
> sender addresses?  What kind of high horse are we on, to insist that  
> even under normal mail delivery circumstances, sender addresses of  
> less-than-perfect form may not be used?

Messages are often carried over multiple MTAs where the majority of  
messages exchanged are undesired.   Checks performed at each differ,  
where for example, a full mailbox will still likely result in non- 
delivery notifications.  To avoid delays and high losses occurring  
with NDNs, validation of the return path _prior_ to message acceptance  
best ensures delivery integrity.  Return-path validation should become  
just a check for an associated MX record.

> Certainly, to me, this looks like an anti-spam tool which, while  
> tempting to the eye of the postmaster wishing to reduce tempfailing  
> blowback, is already seeing diminished usefullness and which is  
> known to break legitimate mail.

Legitimate mail must be made easy to recognize.  In effect, this means  
any domain transmitting SMTP messages should publish MX records.   
Otherwise this will expose any domain being spoofed as originating a  
spam campaign to what may be significant amounts of highly distributed  
undesired traffic.  This traffic is associated with attempts to  
deliver NDNs and to confirm of the return-path by establishing a  
connection.  After all, an address record says nothing about whether  
the SMTP service is supported.

-Doug

Date: Mon, 5 May 2008 14:20:52 +0100
From: Sabahattin Gucukoglu <[email-masked]>
To: [email-masked]
Subject: Re: Queued Mail or Unreturnable Mail?
Sender: [email-masked]

On Mon, 05 May 2008 11:51:30 +0200, Alessandro Vesely <[email-masked]> 
wrote:
> Sabahattin Gucukoglu wrote:
> > To be sure, it sounds like the right thing to do.  As yet, though, 
> > there's no justification in doing it other than that queue clogging 
> > will often result if you don't.  Is it *semantically* correct to reject 
> > mail just because bounces for the sender are undeliverable?
> 
> That question seems ill-conditioned to me. It is semantically correct to reject 
> mail because the *recipient* is undeliverable.

The question is written as intended, but we appear to be in agreement.

I still want to know why sender verification is a good idea, though.  If 
the recipient validation is sufficient to reduce backscatter and queue 
cloggage, why *are* we insisting upon a semantic disapproval of invalid 
sender addresses?  What kind of high horse are we on, to insist that even 
under normal mail delivery circumstances, sender addresses of 
less-than-perfect form may not be used?  Certainly, to me, this looks like 
an anti-spam tool which, while tempting to the eye of the postmaster 
wishing to reduce tempfailing blowback, is already seeing diminished 
usefullness and which is known to break legitimate mail.

Cheers,
Sabahattin

-- 
Sabahattin Gucukoglu <mail<at>sabahattin<dash>gucukoglu<dot>com>
Address harvesters, snag this: [email-masked]
Phone: +44 20 88008915
Mobile: +44 7986 053399
http://sabahattin-gucukoglu.com/

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: Queued Mail or Unreturnable Mail?
Date:  Mon, 5 May 2008 13:38:02 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 68
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Douglas Otis wrote:
=20
> Mention of SPF should be accompanied with security
> admonishments not to expand evaluation macros.

The fine print is a bit more convoluted, most macros
are simply shorthands for "this domain", "space", or
similar harmless cases.

What you are talking about is the "this local part"
macro, and some esoteric contructs to extract labels
from "this local part".  I'd love to know if anybody
actually uses these baroque features.  In one case
I'm aware of the policy hit the RFC 4408 processing
limits, "PermError", tilt, game over.

> these lists are often incomplete and allow NEUTRAL
> or SOFT-FAIL results.

For the "accept and bounce later" question a PASS is
good enough.  It's also a start for white listing and
reputation schemes.  I also like either PASS or FAIL
better, but sometimes that's no option...

> Such results are easily exploited.

...exploiting NEUTRAL makes no sense, it is just what
you have without XYZ (not limited to SPF), you won't
get a bonus for trying in vain.  Quite the contrary,
after receivers wasted their time and bandwidth for
nothing (NEUTRAL) they could treat it slightly worse
than only NEUTRAL. =20

Abusing SOFTFAIL cannot work, it is already tricky to
use it in any meaningful way.  It's in the drirection
of DKIM-ADSP "all", it lets receivers do "something"
with it, anything from "ignore" to "discard them all".

> SPF's use of generic TXT records at base domains is
> unlikely to completely transition to the service
> specific resource record, and will conflict with
> future protocols and revisions.

For wildcard uses that is already the case, sooner or
later folks hopefully decide that using SPF records
is better.  If there is ever a v=3Dspf3 it will build
on the new SPF RR alone, and not abuse TXT.

> expecting MX for acceptance eliminates publishing =20
> or retrieving other SMTP related records within
> sub-domains lacking MX records.

A decree that everybody should have "v=3Dspf1 mx -all"
implicitly (without saying so) for all addresses, and
similarly "v=3Dspf1 a -all" for HELOs, is a nice idea
in the spirit of RFC 821, but I fear it comes decades
too late.

Assume for a moment that a "v=3Dspf1 mx -all" could be
mandatory.  Then publishing SPF policies stating this
fact would be unnecessary, and we could get away with
a variant of BATV.  But obviously that's not the case,
folks use other IPs to send, not only the MX IPs, and
some policies aggregate more than one sending ISP.

 Frank
--=20
<http://www.openspf.org/Frank_Ellermann/OAQ>

Date: Mon, 05 May 2008 11:51:30 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: Queued Mail or Unreturnable Mail?
Sender: [email-masked]

Sabahattin Gucukoglu wrote:
> To be sure, it sounds like the right thing to do.  As yet, though, there's 
> no justification in doing it other than that queue clogging will often 
> result if you don't.  Is it *semantically* correct to reject mail just 
> because bounces for the sender are undeliverable?  If it isn't, fine -

That question seems ill-conditioned to me. It is semantically correct 
to reject mail because the *recipient* is undeliverable. In addition, 
the semantics of the MX record, or lack thereof, should hint that 
verifying a recipient address is much easier for a receiving server 
than investigating about the sender address. Established such basic 
point, there are a number of standard situations, ranging from 
secondary MXes to remote MDAs, where recipient verification can be 
difficult. Those are instances of forwarding within the mail receiving 
network.

Let me spend a paragraph to clarify that the global semantics of MX 
records allow to univocally determine a boundary between mail 
originating network and mail receiving network. The boundary point 
lies after the server that looks up the global MX record of the 
destination address for the first time. (That obviously depends on the 
particular forward path that a given message gets through.) 
Successively, the recipient address itself can even be changed, e.g. 
according to alias or list expansions. This considerations suggest 
that the extent to which a server accepts responsibility for 
delivering a given message, and therefore the semantics of a possible 
rejection, are entangled with the whys and wherefores of the relevant 
SMTP transaction.

Forwarding has been broken since rfc 1123 deprecated building the 
return path. In particular, the semantically widely different possible 
reasons for forwarding have never been extensively worked out: List 
expansion is currently well described, but mailing lists are not 
differentiated from newsletters. Alias expansion, as standardized, 
only suits mail delivery within the same server/domain. Privacy 
mandated final address non-disclosure is mentioned, but not thoroughly 
analyzed. Perhaps those concerns don't belong to the SMTP 
specification, but the semantic of accepting a message varies 
accordingly. For example, a company might decide to forward all 
incoming mail to an internal "big brother" mailbox for statistical 
purposes; however, a sender does not deserve a DSN in case that 
mailbox gets clogged.

I would argue that if a server is unable to verify a given recipient 
address before accepting a message, there must have been something in 
its forwarding reasons that altered the transaction's semantic 
dramatically. Except for backup MXes. The current trend to strengthen 
mail acceptance in order to avoid backscatter, leaves something to be 
desired for classically conceived backup MXes, which is probably why 
they are not much in vogue nowadays.

HTH
Ale

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
To: "Frank Ellermann" <[email-masked]>
Subject: Re: Queued Mail or Unreturnable Mail?
Date: Mon, 5 May 2008 00:19:08 -0700
Sender: [email-masked]

On May 3, 2008, at 4:01 PM, Frank Ellermann wrote:

>
> John Leslie wrote:
>
>> My personal inclination is to declare "guessing" out of scope for  
>> returning DSNs. We already know that not every domain wants them.  
>> Declaring that one MUST send them even when the receiving domain  
>> has not expressed an interest in receiving them leads to known  
>> problems _today_. It would be good, IMHO, to have a clear way to  
>> declare an interest in receiving them, or _not_ receiving them
>
> That is a solved problem for senders and receivers participating in  
> SPF, a PASS means "yes, please inform me about delivery issues as  
> specified in 2821bis".
>
> A FAIL means "please reject at your border, a (wannabe) originator  
> as indicated in the reverse-path is likely not the real originator,  
> and where that is not the case it is a problem to be solved by the  
> hop before you (forwarder or simply an erroneous policy), not your  
> problem".
>
> No guessing involved for PASS and FAIL.

Mention of SPF should be accompanied with security admonishments not  
to expand evaluation macros.  The sequence of transactions that might  
be needed to retrieve SPF authorization lists may not end guessing,  
since these lists are often incomplete and allow NEUTRAL or SOFT-FAIL  
results.  Such results are easily exploited.  SPF's use of generic TXT  
records at base domains is unlikely to completely transition to the  
service specific resource record, and will conflict with future  
protocols and revisions.

Since a large percentage of domains accepting SMTP connections already  
publish MX records, expecting MX for acceptance eliminates publishing  
or retrieving other SMTP related records within sub-domains lacking MX  
records.  Such an expectation offers domains not publishing MX records  
substantial protection from undesired connections and subsequent DNS  
transactions otherwise necessary to support SMTP and various SMTP  
extensions.  In addition, invalid return-paths can be immediately  
deduced within a single transaction.  The transmitter of the message  
must be expected to offer succinct evidence of a valid return-path.

Transactions pertaining to acceptance of anonymous initiations of  
personal messages should be limited to domains publishing resource  
records explicitly supporting the exchange protocol.  This practice  
become increasingly important to limit the level of undesired traffic  
expended by a distribution of receivers.  For SMTP, the resource  
record would be MX.

-Doug

Date: Mon, 5 May 2008 02:21:34 +0100
From: Sabahattin Gucukoglu <[email-masked]>
To: [email-masked]
Subject: Re: Queued Mail or Unreturnable Mail?
Sender: [email-masked]

Hi all,

Before this discussion goes further, I want to just get this bit sorted 
out, just in case people don't see a relationship between the two 
contenders in this dilema (as indicated in the Subject line) ...

On Fri, 02 May 2008 05:11:33 -0400, Hector Santos <[email-masked]> 
wrote:
> Sabahattin Gucukoglu wrote:
[...]
> > 1.  I am naive enough to receive mail from non-verifiable senders. 
> 
> Yes. <g>

To be sure, it sounds like the right thing to do.  As yet, though, there's 
no justification in doing it other than that queue clogging will often 
result if you don't.  Is it *semantically* correct to reject mail just 
because bounces for the sender are undeliverable?  If it isn't, fine - 
let's get to work on standardising a way for domains to indicate that they 
don't want mail, and they'll live happily ever afterward not knowing that 
mail they originated (or not, but that's irrelevant) didn't get through.  
We can try and fail to deliver our DSNs straight away, keep backup MXs 
working without fuss, etc, etc, and still have nice, clean queues without 
going to extraordinary, expensive lengths to verify sender addresses.

To relate to the original question, then: if we all agreed to keep from 
generating DSNs for mailboxes or hosts we take care of to the best of our 
abilities, and we can do that already using RCPT verification to whatever 
extent which is easy to set up and inexpensive, we'll have the DSN problem 
sorted.  If I accept mail for invalid senders, we can only hope that the 
senders help us out by telling us they don't want mail; we'll deliver mail 
from invalid senders in the usual way, but if we do have to generate a DSN 
it's for some legitimate reason and, should the sender not want them, we'll 
happily throw them away under direction from the sender's DNS.  That's the 
dilema: is our priority mail queues or is it the deliverability of errors?

We'd want deliverable errors, of course, but given a choice of having the 
mail and failing to return a DSN on those rare occasions when it's 
necessary and just not having the mail at all, I know which I'd go for: the 
first.  And since the DSNs are the problem here, destined for nowhere, we 
should, by cooperative increments of DNS zones by the people, gradually get 
rid of the long delays usually associated with queued DSNs.  The incentive 
for publishing whether or not you want mail can easily be that it could be 
used by those who want to reject senders who also happen not to receive, if 
desired.  Besides, you can't argue lost DSNs when the sender is invalid, 
any more than if it were <>; and that's often what these noreply-style 
sender addresses are for.  But somehow, you can argue that losing mail 
isn't cool.

Cheers,
Sabahattin

-- 
Sabahattin Gucukoglu <mail<at>sabahattin<dash>gucukoglu<dot>com>
Address harvesters, snag this: [email-masked]
Phone: +44 20 88008915
Mobile: +44 7986 053399
http://sabahattin-gucukoglu.com/

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: Queued Mail or Unreturnable Mail?
Date:  Sun, 4 May 2008 01:01:37 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 22
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

John Leslie wrote:

> My personal inclination is to declare "guessing" out of scope for
> returning DSNs. We already know that not every domain wants them.
> Declaring that one MUST send them even when the receiving domain
> has not expressed an interest in receiving them leads to known
> problems _today_. It would be good, IMHO, to have a clear way to
> declare an interest in receiving them, or _not_ receiving them

That is a solved problem for senders and receivers participating
in SPF, a PASS means "yes, please inform me about delivery issues
as specified in 2821bis".

A FAIL means "please reject at your border, a (wannabe) originator
as indicated in the reverse-path is likely not the real originator,
and where that is not the case it is a problem to be solved by the
hop before you (forwarder or simply an erroneous policy), not your
problem".

No guessing involved for PASS and FAIL.

 Frank

Date: Sat, 3 May 2008 08:19:33 -0400
From: John Leslie <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>
Cc: [email-masked]
Subject: Re: Queued Mail or Unreturnable Mail?
Sender: [email-masked]

Robert A. Rosenberg <[email-masked]> wrote:
> At 18:00 -0400 on 04/30/2008, John Leslie wrote about Re: Queued Mail 
> or Unreturnable Mail?:
> 
>> We certainly could design a mechanism _better_ than MX records to
>> document intent relative to receiving email, especially DSNs. But unless
>> and until we do, folks are likely to use MX records as part of the
>> balancing act of guessing the probability of NDNs reaching a responsible
>> reader.
> 
> IMO: The publication of a MX is a statement to "Send *@FQDN Addressed 
> Email HERE". Once an SMTP Server does so and delivers a NDN message 
> addressed to *@FQDN to the MX, it has done its job and turned over 
> Delivery Responsibility to a MX designated Server. The fact that the 
> MX Server (or one that it forwards the NDN message to) then turns 
> around and sends is to /dev/null is not my SMTP Server's Problem 
> since, as noted, it did its job by delivering the message to the 
> Domain Designated injection point and turning over further delivery 
> processing to the Domain's Mail System.
> 
> Thus the existence of the MX places the responsibility of "reaching a 
> responsible reader" on the MX not the Sending SMTP Server with NO 
> GUESSING involved.

   An excellent point. The MTA generating a DSN has no obligation to
guess the probability of that DSN reaching a responsible reader if
there is an MX record for the domain in question. If that MX directs
it to /dev/null, there's nothing the sending MTA could have done to
increase its chances.

   Conversely, if there _is_ no MX, it's a pure guessing game. Will
a particular IP address gleaned from a A)ddress record answer on
port 25 at all? If it doesn't answer now, might it answer an hour
from now? Might a different IP address gleaned from the same A)ddress
query give better odds? If we finally get a SMTP session, but get
"no such account", would it help to try a different IP address?

   My personal inclination is to declare "guessing" out of scope for
returning DSNs. We already know that not every domain wants them.
Declaring that one MUST send them even when the receiving domain has
not expressed an interest in receiving them leads to known problems
_today_. It would be good, IMHO, to have a clear way to declare an
interest in receiving them, or _not_ receiving them, and recognize
that in the absence of such a declaration, some MTA administrators
will choose to send them, while others won't.

--
John Leslie <[email-masked]>

Date: Sat, 3 May 2008 00:25:33 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: Queued Mail or Unreturnable Mail?
Sender: [email-masked]

At 18:00 -0400 on 04/30/2008, John Leslie wrote about Re: Queued Mail 
or Unreturnable Mail?:

>We certainly could design a mechanism _better_ than MX records to
>document intent relative to receiving email, especially DSNs. But unless
>and until we do, folks are likely to use MX records as part of the
>balancing act of guessing the probability of NDNs reaching a responsible
>reader.

IMO: The publication of a MX is a statement to "Send *@FQDN Addressed 
Email HERE". Once an SMTP Server does so and delivers a NDN message 
addressed to *@FQDN to the MX, it has done its job and turned over 
Delivery Responsibility to a MX designated Server. The fact that the 
MX Server (or one that it forwards the NDN message to) then turns 
around and sends is to /dev/null is not my SMTP Server's Problem 
since, as noted, it did its job by delivering the message to the 
Domain Designated injection point and turning over further delivery 
processing to the Domain's Mail System.

Thus the existence of the MX places the responsibility of "reaching a 
responsible reader" on the MX not the Sending SMTP Server with NO 
GUESSING involved.

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
To: "Frank Ellermann" <[email-masked]>
Subject: Re: Queued Mail or Unreturnable Mail?
Date: Fri, 2 May 2008 15:24:12 -0700
Sender: [email-masked]

On Apr 30, 2008, at 7:02 PM, Frank Ellermann wrote:

> 2821 and 2821bis have it clear that the mere presence of a listening  
> process does not say anything about the intent to receive mail for  
> the corresponding domains.  The MX can say something else, without  
> MX a listening process at A or AAAA can reject clients or individual  
> mails as it sees fit.
>
> The problematic situation is the absence of MX and listening  
> process, senders are forced to wait 100 hours, somebody might find  
> the reboot button of the affected host.  Another problematic  
> situation is mail claiming to be "from" this domain without MX,  
> receivers cannot wait until somebody finds the reboot button for a  
> quick plausibility test, they are forced to reject such suspicious  
> mails temporarily.

Since DARPA's development of a "network of networks" became known as  
the Internet, and intercommunication processes known as Internet  
protocols (IP), a means to relate hostnames and services with IP  
addresses was needed.  Jon Postel publish a concept of Sockets  
(combing network address and port) in use with assignments for  
standard programs, laying a foundation on efforts made by Steve Carr  
from Utah, Jeff Rulifson from SRI, and Steve Crocker of UCLA, who  
introduced documents describing host networking almost four decades ago.

The next decade introduced RFC761 TCP, RFC768 UDP, and RFC790 Assigned  
numbers.  Within a few years, RFC1034 DNS supplanted host files.  With  
DNS, in addition to port assignments, service support could be  
declared using specialized DNS resource records.  RFC974 recommended  
WKS (well known service) resource records to confirm support of SMTP.   
RFC1123 renounced WKS, as few domains published these records.  The  
resource record concept did not mesh with host files, or NetBIOS  
adaptation of WINS's offering referrals from specific service names.

To help transitioning from WINS to DNS, SRV records were created.   
Even with SRV records, Window's adoption of DNS remained limited.   
Within a Window's enterprise environment, DNS might be converted into  
RPC.  Avoiding proprietary development for this environment, protocols  
often employ only the limited list of resource records having data  
structures defined for Windows OS APIs.  While API provisions now  
accommodate undefined resources records, a lack of OS data structure  
definitions remains an impediment.

Resolving services and hostname addresses, generic resource records  
holding signatures and policies continues to impact SMTP.  Although  
SMTP adopted MX resource records within DNS, publishing MX records was  
unfortunately never mandated.  While ensuring the viability of local  
hostname resolution remains important, undesired traffic associated  
with spoofed email addresses represents an ever increasing problem.   
However, there does not appear to be an easy method to differentiate  
name services when resolving an address.  Facing massive amounts of  
undesired traffic, SMTP reception typically employes extensive  
examination of SMTP client IP addresses and return-path email-addresses.

Therefore, it should be reasonable to publish an RFC that warns of  
possible future changes where message acceptance may require return- 
path email-address domains to publish MX records.  Mandating MX  
records be published by domains supporting SMTP better protects  
domains not supporting SMTP, and better ensures integration of IPv6  
addresses.  The MX mandate also replaces unofficial recommendations  
that domains not supporting SMTP publish bogus MX records targeting  
roots.  This mandate would help establish conventions for receiving  
SMTP servers to predicate additional transactions upon first finding  
MX records for associated email-address domains.  The receiving SMTP  
server is also best equipped at making necessary exceptions where  
needed.

In essence, SMTP discovery resource record mandate essentially impacts  
message reception.  Justification for this mandate could be to support  
the obligation accepted by receiving SMTP servers of ensuring delivery  
or offering notification upon a delivery failure.

-Doug

Date: Fri, 02 May 2008 05:11:33 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Sabahattin Gucukoglu <[email-masked]>
CC:  [email-masked]
Subject: Re: Queued Mail or Unreturnable Mail?
Sender: [email-masked]

Sabahattin Gucukoglu wrote:
> Hi all,
> 
> This question is bothering me.  It's stalled my reading of 2821bis, and the 
> discussion around the issue (having diverged from IPv6) seems worthwhile 
> fodder for continued debate.
> 
> I've got a message in my queue here destined for natwest.co.uk.  
> Natwest.co.uk exists, but it isn't running an SMTP service; it's intended 
> as a web server.  There's no MX, and I don't check SPF.  (Both natural 
> conditions for typical hosts.)  The message is a DSN generated by this 
> host.  The original message was intended for a primary host for which I am 
> an acting secondary.  I will only accept mail for primaries when I can't 
> connect to them myself, with a few exceptions to help routing difficulties 
> of some hosts reaching the primary directly.
> 
> Which is the problem?

If I followed you message right, this is a normal undeliverable message.

Your receiver accepted the payload first, then determined it was not 
deliverable, your system properly created a bounce.

This is normal other than you now fall into the ACCEPT/BOUNCE attack 
dilemma many in the world fall into.

If your receiver was able to dynamically to determine the acceptability 
of the RCPT TO forwarding path and issue a rejection at the SMTP level, 
then you would not have this BOUNCE issue.

> 1.  I am naive enough to receive mail from non-verifiable senders. 

Yes. <g>

If this is a issue for you, then you need to take that extra step.  Some 
systems don't for many reasons, including scalability reasons and for 
many, there only option is to do a RCPT TO verification after the 
payload was accepted and SMTP session has ended.

Many old school systems run like this, including ours. But we long 
copied the same post gateway hosting verification logic to the SMTP 
process and this was the beginning of addressing the abusive 
ACCEPT/BOUNCE problem.  I remember the many customers calling "I am 
getting killed with this bounces" and they didn't even know they had the 
ability to do the same POST checking at the SMTP level dynamically. 
That was because they were so use to running like the old days, 
UUCP/SLIP, with a separate Mail Gateway that did all the hosting 
verification it was never an issue until it became abusive.

> I have 
> not gone even to the extent of checking that the domain exists; a surefire 
> but, as you can see here, pointless check.  

Right, that is why we have a CBV implementation. At the very least, you 
need to try to connect to the system to see if its even has a SMTP 
listening server port.

Also, your bounce retries can have special rules.  Like for example, in 
our implicit MX logic, we have a "1 try" concept and this serves us well 
to eliminate the junk.

> I will never be happy until 
> every conceivable check is done to verify that, should an error occur, I 
> wil *always* be able to send mail back to the sending site at the sending 
> mailbox.  

I have the same philosophy. IMV, if a client is able to send you mail, 
then it must have a companion host system available to receive, 
especially at same precise moment of time.  Low chance it will claim 
"Receiver Host is temporarily down" without the whole system not being down.

But overall,  SMTP is about two communications.  SMTP is not about send 
only outbound broadcasting system without having some contact address. 
The system is designed for feedback.

> 2.  I am not doing enough to ensure that I will not have to generate a DSN.  

Right, this goes back to considering using a DYNAMIC SMTP level RCPT 
verification.

And if possible, do no checking until RCPT TO is acceptable.  This is 
the recommendation in 2821.

We have statistics showing this will not only drastically reduce your 
bounce issues, but it will drastically reduce any necessary verification 
overhead by nearly 65-70%. Of course, that is site dependent.  We get 
hit a lot with tons of old Compuserve addresses and also I think an old 
employee sold our support system user list to spammers when it was an 
open email system back in the 90s.   But that was turned off and we 
still get hit by all the bulk spam. IOW, at least 65% of rcpt to state 
fail making it prudent that we don't do any MAIL FROM checking (like SPF 
or CBV) until RCPT TO is known.  A massive improvement across the board, 
especially in DNS lookups.

> My vote is for 2 on general grounds of practicality, ease 
 > of implementation and least harm done.

Both.

Once upon a time, such filtering was 100% left to the operator. From an 
ethical standpoint, I felt the duty of SMTP (any mail protocol) was to 
follow the rules as much as it can and transport mail with no questions ask.

But as the SPAM problem became too much, and began to directly to effect 
us, then it was time to begin "doing more" to help our customers.

The first thing we added was CBV, then SPF and a p-code filtering 
language to add rules at each state of the SMTP state machine.

Our general AVS logic policy is strong SMTP compliancy at all states, 
except DATA where we make no subjective rules based on mail content. 
Operators make those spam filtering rules. Not us.  We just provide the 
hook into the DATA state and push strongly that they all rejection be 
done at SMTP level.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com