Subject: [ietf-smtp] Per-Recipient Data Responses

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 5 Mar 2014 18:20:33 +0000

How come this never got adoption?  It comes up from time to time in "We
really should do this" sorts of discussions, but it doesn't seem like
anyone ever took the plunge and it just expired.  Is it just that nobody
does it because nobody else does it?

http://tools.ietf.org/html/draft-hall-prdr-00

-MSK

---

From: Claus Assmann <[email-masked]>
Date: Wed, 5 Mar 2014 10:56:05 -0800

> How come this never got adoption?  It comes up from time to time in "We

It has been "adopted": MeTA1 and exim implement it. Maybe someone
needs to "push" it through the IETF process?

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 5 Mar 2014 19:25:42 +0000

> On 5 Mar 2014, at 18:56, Claus Assmann <[email-masked]> wrote:
> 

+1

---

From: Brandon Long <[email-masked]>
Date: Wed, 5 Mar 2014 11:18:03 -0800

> needs to "push" it through the IETF process?

It must not be enabled by default, since I only saw about 5 MXs
yesterday that are advertising PRDR.

Brandon

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 5 Mar 2014 19:21:28 +0000

> yesterday that are advertising PRDR.

Does that mean Google's implemented it, or you just have stats about what
gets advertised?

-MSK

---

From: Дилян Палаузов <[email-masked]>
Date: Wed, 05 Mar 2014 20:26:16 +0100

Hello,

thanks for bringing this issue to our attention.

For the receiving MTA side, I have written a tool, that builds on the 
Milter interface in sendmail/postfix and provides API for other plugins, 
that can decide per recipient, if s/he wants to receive the message.

   http://mail.aegee.org/cgit/aegee-milter/

Basically, the module that decides per recipient evaluates Sieve scripts 
(src/modules/mod_sieve) after RCPT and DATA-DOT and rejects the message. 
  To build the Sieve module, you need either to install the 
libcyrus_sieve from the development version of cyrus-imap (once you 
compile the project, you need to install the libcyrus, libcyrus_min and 
libcyrus_sieve libraries), or libsieve (http://sodabrew.com/libsieve/).

The only reason I have not released the software so far, is that it 
lacks documentation.

Some years ago I managed to tweak sendmail to accept the PRDR parameter 
to the MAIL command and then it acted correctly (to the extend I 
understood PRDR).

Recently I switched my programme from C to C++ and have not tested yet, 
with the sendmail patches from that time, if the PRDR stuff is running.

However, I would be glad if somebody contributes to Sendmail to extend 
the milter-side/API, so that one can send over it arbitrary responses 
after data-end and my milter just uses that API instead of me trying to 
hack sendmail.

Next to that, I intend to write documentation for the above programme, 
so that more people can use it.  I would be very happy, if somebody is 
interested to help me.

Per recipient response is very cool, as each recipient in the same mail 
envelope can have different spam rules and PRDR provides the possibility 
to only partially reject the message.  Also, if the recipient agree, in 
the rejection message one can write the telephone/fax number of the 
recipient, so that the sender can still contact the recipient, if was 
wrongly rejected.

Greetings
   Dilian

---

From: Brandon Long <[email-masked]>
Date: Wed, 5 Mar 2014 11:26:44 -0800

> Does that mean Google's implemented it, or you just have stats about what

We just record everything that gets advertised to help us judge what
to implement and also for interoperability testing prior to releasing
new ones.

This one sounds useful, but without wider adoption, would be a hard
sell at this point.  I realize there's a chicken & egg thing there, of
course, but we also have a very limited number of cases where we'd do
mixed responses and we have to fall back to bounces today.

Brandon

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 5 Mar 2014 19:33:50 +0000

On Wed, Mar 5, 2014 at 7:26 PM, Дилян Палаузов <[email-masked]>wrote:

>
> However, I would be glad if somebody contributes to Sendmail to extend the
> milter-side/API, so that one can send over it arbitrary responses after
> data-end and my milter just uses that API instead of me trying to hack
> sendmail.
>

I could swear that was made available as part of milter as an FFR, but
maybe I'm wrong.  Maybe I just saw a patch that never got adopted and
released.

Either way, it's come up enough times over the years that maybe it's time
to at least publish the spec.

-MSK

---

From: Tony Finch <[email-masked]>
Date: Wed, 5 Mar 2014 19:44:20 +0000

> yesterday that are advertising PRDR.

It is an experimental compile-time option in Exim.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
South Utsire, Southeast Forties: Southerly 5 to 7, veering southwesterly 4 or
5 later. Moderate or rough. Occasional rain. Moderate or good.

---

From: Paul Smith <[email-masked]>
Date: Wed, 05 Mar 2014 20:22:19 +0000

On 05/03/2014 18:20, Murray S. Kucherawy wrote:
> How come this never got adoption?  It comes up from time to time in 
> "We really should do this" sorts of discussions, but it doesn't seem 
> like anyone ever took the plunge and it just expired.  Is it just that 
> nobody does it because nobody else does it?
>

Hmm, this one's a tricky one. With our mail server we've been quite 
careful to do things which would either accept the "whole message", or 
reject it all.  I can see places where we could change functionality to 
use PRDR, but the problem is that the user would expect it to always 
work, not be dependent on the sending MTA. So, we would have to 'fake' 
PRDR, by falling back to accepting the message and generating bounce 
messages, which isn't nice, and will lead to backscatter.

(I suppose we could 'fake PRDR' by only allowing one recipient per 
message, but that has bad side-effects as well)

So, I'd be reluctant to implement it on the server-side until PRDR 
support was widely available, even though this is where the benefits 
would be.

OTOH, implementing it on the client-side should have minimal bad 
side-effects, but would also give minimal direct benefits, so there's 
little pressure to do that.

-

Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

---

From: Gilles Chehade <[email-masked]>
Date: Wed, 5 Mar 2014 22:25:03 +0100

On Wed, Mar 05, 2014 at 08:22:19PM +0000, Paul Smith wrote:
> >How come this never got adoption?  It comes up from time to time
> >in "We really should do this" sorts of discussions, but it doesn't
> >seem like anyone ever took the plunge and it just expired.  Is it
> >just that nobody does it because nobody else does it?
> >
> careful to do things which would either accept the "whole message",
> or reject it all.  I can see places where we could change
> functionality to use PRDR, but the problem is that the user would
> expect it to always work, not be dependent on the sending MTA. So,
> we would have to 'fake' PRDR, by falling back to accepting the
> message and generating bounce messages, which isn't nice, and will
> lead to backscatter.
> 

Correct me if I'm wrong, I may have missed something in the draft:

PRDR implies that deliveries be attempted during the SMTP transaction so
this cannot really be implemented by MTA that use a two-step approach to
first commit to queue and acknowledge responsibility, then later deliver
the message to the mailbox ?

-- 
Gilles Chehade

https://www.poolp.org                                          @poolpOrg

---

From: Ned Freed <[email-masked]>
Date: Wed, 05 Mar 2014 13:30:28 -0800 (PST)

> the message to the mailbox ?

No, that's incorrect. From the draft:

           Furthermore, positive responses are not a
           guarantee that any subsequent transfer or delivery operations
           will also succeed, but instead only indicate that the message
           appears to be acceptable for the recipient according to the
           rules and policies that are known to the current server.

You should do as much vaidation as possible before accepting an address or
message, but that's true in general; it has nothing to do with this
extension.

More generally, any notion that this extension eliminates backscatter is 
incorrect for a whole bunch of reasons, this included. At most it limits it a
bit.

				Ned

---

From: Paul Smith <[email-masked]>
Date: Wed, 05 Mar 2014 21:37:46 +0000

On 05/03/2014 21:25, Gilles Chehade wrote:
> PRDR implies that deliveries be attempted during the SMTP transaction 
> so this cannot really be implemented by MTA that use a two-step 
> approach to first commit to queue and acknowledge responsibility, then 
> later deliver the message to the mailbox ? 
No, that's correct (AIUI) - all is fine if the sender support PRDR.

The problem is that if users expect PRDR functionality to be there, you 
can't have it work sometimes and not others. So, when the sender doesn't 
support PRDR, the receiver needs to fake it by first accepting the 
message for all recipients and then sending delivery failure reports back.

Otherwise, one user could say "reject a message if it it contains the 
word 'viagra'" and another could want to accept i. Then, sometimes the 
first user would have the message rejected (if it is just to them, or 
the sender supports PRDR), and sometimes they would receive the message 
(if it is to both users, and the sender doesn't support PRDR). Users 
wouldn't be happy about this inconsistency, and the support load would 
be unacceptable from our PoV.

So, if the sender didn't support PRDR, we'd have to accept the message, 
deliver it to the second user, and send a bounce message back (with risk 
of backscatter) for the first user.

In any case, spammers would probably not support PRDR, and that'd 
probably be the most common case people would use it, and that would be 
the most likely case to cause backscatter from accepting then sending 
bounce messages.

-

---

From: Gilles Chehade <[email-masked]>
Date: Wed, 5 Mar 2014 23:04:18 +0100

>            rules and policies that are known to the current server.

Well, my understanding of this paragraph is that if I do get a positive
response for a recipient, it is for this transaction only. Trying again
to transfer/deliver to the same recipient will not necessarily mean the
response will be positive again.

If PRDR allows you to answer positively to mails that you will possibly
fail later down the chain doesn't it mean that any PRDR positive answer
can basically be ignored because it carries no guarantees (as in people
will fake positive replies because it is easier in some cases, and they
can bounce later anyway) ?

> message, but that's true in general; it has nothing to do with this

agreed

-- 
Gilles Chehade

---

From: Steve Atkins <[email-masked]>
Date: Wed, 5 Mar 2014 15:02:07 -0800

On Mar 5, 2014, at 10:20 AM, Murray S. Kucherawy <[email-masked]> wrote:

> How come this never got adoption?  It comes up from time to time in "We really should do this" sorts of discussions, but it doesn't seem like anyone ever took the plunge and it just expired.  Is it just that nobody does it because nobody else does it?
> 

What sort of smtp clients /senders of email are the target market for this?

Most of the obvious users I can think of (send lots of similar email on an ongoing fashion, care deeply enough about bounce management to really want to track which recipients are bouncing, technically competent enough to both implement and make good use of PRDR) don’t tend to batch and blast, rather they tend to send unique emails to each recipient.

(I’m sure it’s been discussed before, but I don’t recall what the answer was and there’s no mention of it in the draft.)

Cheers,
  Steve

---

From: Brandon Long <[email-masked]>
Date: Wed, 5 Mar 2014 15:49:43 -0800

> (I’m sure it’s been discussed before, but I don’t recall what the answer was and there’s no mention of it in the draft.)

For us, the most obvious use case is for enterprise routing rules.
For example, you can imagine that a school might implement an
objectionable content filter for students, but not for faculty.  In
that case, a message containing such content that was in the same smtp
transaction could now be allowed to the teacher and rejected by the
student.  Today, we have to accept for both and then generate a
bounce.

As for ESPs, yes, if you're already doing single recipients per
transaction, this isn't especially useful.

As for spammers, not all spammers run their own software.  Even those
that do tend to just throw things together using whatever parts they
find, so they could grab some library which does this without thinking
too much about it.  Others abuse existing infrastructure by hijacking
accounts and such, so they'll get whatever impl the hijacked infra
already has.  I could even imagine that spammers would find this
useful if it increased their immediate knowledge of block rates given
they almost certainly are using forged senders or aren't checking the
bounces.

Anyways, the obvious use case is per-user filters (spam or rule
based), neither of which are really that strong, I'd think.

Brandon

---

From: "Carl S. Gutekunst" <[email-masked]>
Date: Wed, 05 Mar 2014 16:27:52 -0800

Brandon Long <[email-masked]> wrote:
> ....

At my two most recent employers, we referred to this situation as a 
"disposition conflict": one Email message, but different policies 
configured by different recipients created a situation where it wasn't 
possible to convey a single response (250, 450, or 550) to the sender. 
There are many more ways to get in this state than just spam filters: 
policy rules that allow message bifurcation, for example, or data loss 
protection filters that defer messages for administrative action or kick 
them back to the sender.

While store-and-forward MTAs have ways to resolve this, SMTP Proxies do 
not: there's no place to queue messages for retries, and no ability to 
send DSNs. So instead you implement a long set of rules ("if this, then 
this") to handle the conflicts in the least surprising way; but the 
result is still unsatisfying and occasionally quite surprising.

If you're responsible for an SMTP proxy, PRDR gets much more interesting.

<csg>

---

From: Ned Freed <[email-masked]>
Date: Wed, 05 Mar 2014 19:25:31 -0800 (PST)

> based), neither of which are really that strong, I'd think.

This is exactly the situation with our MTA - we go to a lot of trouble to
reject as much as possible at the RCPT TO stage, so by the time we get to the
end of DATA the only thing that can generate a per-recipient failure is a sieve
filter result (this includes spam filter results), and then only when that
sieve uses a reject/ereject rather than discard.

And the problem with reject/ereject is they are rarely exposed by sieve
building interfaces, precisely because of the potential to create backscatter.

It also doesn't completely eliminate the need to generate DSNs, e.g., when one
of the RCPT TO addresses bifurcates on our end to a mixture of valid and
invalid recipients.

All that said, here's an idea: A new sieve action "prdrreject" that acts like
reject if PRDR is available and the RCPT TO address didn't bifurcate into
multiple recipients, and like discard otherwise. That would give you the best
possible result while not creating backscatter.

One downside is the least astonishment principle violation, but frankly, being
able to get better messages back at least sometimes would be worth it IMO.

I also dispair of writing down the semantics of such a thing in a way
implementors would understand.

But all in all, this makes PRDR considerably more attractive to me to
implement.

One thing I'm not wild about is the final response. It strikes me as
unnecessary.

Finally, in terms of implementation difficulty, the client side of this would
be easy for us, since it can piggyback off of existing LMTP client support. The
server side is a litttle harder - keeping track of things is a bit more subtle
than it first appears - but quite doable.

				Ned

---

From: Tony Finch <[email-masked]>
Date: Thu, 6 Mar 2014 13:31:22 +0000

> possible result while not creating backscatter.

Rather than discarding I think it is better to file into a quarantine or
spam mailbox. This gives recipients a better way of dealing with questions
about missing messages, without involving the postmaster.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Thames, Dover, Wight: South 4 or 5, veering southwest 5 to 7. Slight or
moderate. Mainly fair. Moderate or good.

---

From: Ned Freed <[email-masked]>
Date: Thu, 06 Mar 2014 08:37:11 -0800 (PST)

> about missing messages, without involving the postmaster.

Also an option, and one we support, but in my experience rarely used in
practice.

				Ned

---

From: "Carl S. Gutekunst" <[email-masked]>
Date: Thu, 06 Mar 2014 09:03:45 -0800

> Also an option, and one we support, but in my experience rarely used in

Dropping the message into the quarantine is the "last choice" action for 
disposition conflicts on the Postini and SonicWALL proxies. It's also 
one that gets calls to customer support. ("I set the policy to reject 
this, how comes it got into my spam folder?")

<csg>

---

From: Paul Smith <[email-masked]>
Date: Thu, 06 Mar 2014 17:11:16 +0000

On 06/03/2014 17:03, Carl S. Gutekunst wrote:
> Dropping the message into the quarantine is the "last choice" action 
> for disposition conflicts on the Postini and SonicWALL proxies. It's 
> also one that gets calls to customer support. ("I set the policy to 
> reject this, how comes it got into my spam folder?")

This is my issue with PRDR.

If a user set 'discard if x,y,z', and the sender doesn't support PRDR, 
we have to either:

- discard the message silently (ugh)
- send a bounce message back (backscatter)
- quarantine the message (support issues, as above)
- fake PRDR by temp rejecting the message, then on the next retry only 
accept one recipient at a time (double-ugh)

At the moment we just don't give the user an option to 'discard'. Their 
bad messages get quarantined - but that's what they know will happen, so 
at least it's deterministic, even if not exactly what the user wants. 
Having a 'sometimes discard, sometimes quarantine' option would be nasty 
from a user's PoV.

-

---

From: Sabahattin Gucukoglu <[email-masked]>
Date: Thu, 06 Mar 2014 18:09:31 +0000

On 6 Mar 2014, at 17:11, Paul Smith <[email-masked]> wrote:
>> Dropping the message into the quarantine is the "last choice" action for disposition conflicts on the Postini and SonicWALL proxies. It's also one that gets calls to customer support. ("I set the policy to reject this, how comes it got into my spam folder?")
> 
> If a user set 'discard if x,y,z', and the sender doesn't support PRDR, we have to either:
> 
> - fake PRDR by temp rejecting the message, then on the next retry only accept one recipient at a time (double-ugh)

There is a final option: reject the message outright at SMTP time, but deliver it to the person that accepted it normally.

This is also the required strategy if PRDR is supported, but a recipient is logically expanded at the receiving proxy into multiple recipients.

PRDR is great in theory, but I'm afraid it's doomed by sheer lack of implementation.

Cheers,
Sabahattin

---

From: "Carl S. Gutekunst" <[email-masked]>
Date: Thu, 06 Mar 2014 10:44:53 -0800

> There is a final option: reject the message outright at SMTP time, but deliver it to the person that accepted it normally.

Ouch. I don't ever want to tell the sender I rejected the message when I 
actually delivered it. My law firm customers would have a cow.

> PRDR is great in theory, but I'm afraid it's doomed by sheer lack of implementation.

Well, that's kinda the case with any new capability in a protocol. The 
whole purpose of this thread seems to be taking a temperature check, 
which so far seems lukewarm. (I'm highly interested, but seem to be the 
exception. If the only people who are "highly" interested are those 
responsible for SMTP proxies, then yeah, it's not going to happen.)

<csg>

---

From: Paul Smith <[email-masked]>
Date: Thu, 06 Mar 2014 19:50:20 +0000

> There is a final option: reject the message outright at SMTP time, but deliver it to the person that accepted it normally.
I'd never even thought of that... Yeuch. I'd have thought that would be 
'illegal' in SMTP.

> PRDR is great in theory, but I'm afraid it's doomed by sheer lack of implementation.
I think it's doomed by there not being a 'good' fallback solution. This 
would be especially a problem when its functionality would virtually 
never be available. If it became more common, so the fallback is the 
exception rather than the rule, then it would be less of a problem.

At the server end, as far as I can see, it would be trivial to 
'implement' PRDR (just announce the capability in the EHLO response, and 
accept PRDR in the MAIL FROM, and that's it). It would gain the server 
nothing, but be virtually free to implement.

If servers did this, it might encourage clients to implement it, which 
would then make it more acceptable for servers to start actually using 
it for partial accepts.

-

---

From: Sabahattin Gucukoglu <[email-masked]>
Date: Thu, 06 Mar 2014 20:10:51 +0000

On 6 Mar 2014, at 19:50, Paul Smith <[email-masked]> wrote:
> I'd never even thought of that... Yeuch. I'd have thought that would be 'illegal' in SMTP.

Well it's not a nice option, but you could make it less awful by just explaining what it means in the message text (which you'll be lucky if the user sees, to judge from the behaviour of some stupid MTAs).

> I think it's doomed by there not being a 'good' fallback solution. This would be especially a problem when its functionality would virtually never be available. If it became more common, so the fallback is the exception rather than the rule, then it would be less of a problem.

Yes, this is what I was really trying to say.  The functionality of PRDR depends so critically on everyone else having it that it's basically impossible to justify implementation without everybody else on board.  Think of it as the IPv6 of SMTP extensions, if you like. :)

I think it's a shame that LMTP left SMTP behind in this regard.  It never seemed to bother people then.  Ironically LMTP is now more "In vogue" than ever (many LDA now support it natively, or can be patched to do it) so an obvious implementation choice for PRDR would simply be in translating LMTP into PRDR.

Cheers,
Sabahattin

---

From: Ned Freed <[email-masked]>
Date: Thu, 06 Mar 2014 12:27:06 -0800 (PST)

> Yes, this is what I was really trying to say.  The functionality of PRDR
> depends so critically on everyone else having it that it's basically impossible
> to justify implementation without everybody else on board.  Think of it as the
> IPv6 of SMTP extensions, if you like. :)

Except that isn't true. As I pointed out previously, there's substantial
benefit and limited downside to implementing a user filter option with "reject
at SMTP time if possible, otherwise discard" semantics. Given the existence of
that option, server side implementation of PRDR becomes a win, and given that
so does client side implementation.

> I think it's a shame that LMTP left SMTP behind in this regard.  It never
> seemed to bother people then.  Ironically LMTP is now more "In vogue" than ever
> (many LDA now support it natively, or can be patched to do it) so an obvious
> implementation choice for PRDR would simply be in translating LMTP into PRDR.

That's actually a significant problem with the current PRDR proposal - its
semantics are different from those of LMTP, and worse, substantially more
complex. Had this been a simple tweak to reuse LMTP support I would have
implemented it years ago. But it isn't.

				Ned

---

From: Paul Smith <[email-masked]>
Date: Thu, 06 Mar 2014 20:41:51 +0000

On 06/03/2014 20:27, Ned Freed wrote:
> so does client side implementation.T
The problem is that we can't do that. Content filters aren't 100% 
reliable, so we won't just discard messages or we would have hell to pay 
from our customers. We either quarantine or reject. Users will complain 
if they have chosen to reject, and we quarantine. They won't understand 
the intricacies of why that is so, they'll just want it to work.

So, if we support PRDR we have to reject or send a bounce when the 
sender doesn't support it. That causes backscatter which isn't nice. 
Since we should be trying to cut down backscatter, I can't see that a 
standard which potentially causes more backscatter is a good idea. If 
everyone (or at least a large majority) supported PRDR, then it would be 
acceptable; when a minority support it, it's not worth it (IMHO).

-

---

From: Steve Atkins <[email-masked]>
Date: Thu, 6 Mar 2014 12:49:09 -0800

On Mar 6, 2014, at 12:27 PM, Ned Freed <[email-masked]> wrote:

> at SMTP time if possible, otherwise discard" semantics.

Sure, lets say there is.

> Given the existence of
> that option, server side implementation of PRDR becomes a win,

Perhaps. But the complexity of doing it is significant, especially
if you want similar user visible behaviour whether or not the client
is playing along or not.

> and given that

The client side can get all the advantages today, without server side support
and without any new code needed, just by sending one body per recipient,
I think?

If so, there’s very little benefit to a client in adding the complexity of supporting
PRDR at all. If it’s functionality they want, they can already get it in all cases,
whether or not the server side supports it.

Other than perhaps non-store-and-forward filtering proxies, I can’t think of a
situation where PRDR would be obviously a sensible thing for a *client* to
deploy.

Cheers,
  Steve

---

From: "Carl S. Gutekunst" <[email-masked]>
Date: Thu, 06 Mar 2014 16:16:18 -0800

Paul Smith <[email-masked]> wrote:*
*
> At the moment we just don't give the user an option to 'discard'. 
> Their bad messages get quarantined - but that's what they know will 
> happen, so at least it's deterministic, even if not exactly what the 
> user wants. Having a 'sometimes discard, sometimes quarantine' option 
> would be nasty from a user's PoV.

Paul, I want to be sure I understand your point.

In your implementation, there is no possibility of a disposition 
conflict because the configuration options exposed to your users do not 
permit it. To provide meaningful support for per-recipient data 
responses, you would need to add a configuration option whose behavior 
would be non-deterministic. Right?

My problem is that the disposition conflict already exists (arguably due 
to Bad Choices by the author of the code ten years ago). Having the 
ability to return per-recipient status to the sender-SMTP after the dot 
would make the behavior of my filter less surprising.

This is distinct from the question of whether the current PRDR draft is 
the correct solution, or whether something more LMTP-like would be 
preferable.

<csg>

---

From: Tony Finch <[email-masked]>
Date: Fri, 7 Mar 2014 09:02:34 +0000

Paul Smith <[email-masked]> wrote:

> Users will complain if they have chosen to reject, and we quarantine.

Isn't this a matter of setting expectations? The options are essentially
reject-or-quarantine, reject-or-discard, and reject-or-bounce. I admit
that it is hard to write a rubric explaining what "reject" means and why
it is not always possible :-)

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Northwest FitzRoy, Sole: Northerly, backing southerly later, 4 or 5,
increasing 6 or 7, perhaps gale 8 later. Rough or very rough. Rain or
drizzle. Moderate occasionally poor.

---

From: Paul Smith <[email-masked]>
Date: Fri, 07 Mar 2014 09:17:39 +0000

> it is not always possible :-)
Yes. The question is, is it simpler from the users' PoV to have two 
options 'quarantine or accept', or three options, 'quarantine, accept or 
reject-or-quarantine'?

If the 'reject-or-quarantine' will reject 75% of the time and quarantine 
25% of the time, it's probably worth it. If it rejects 5% of the time 
and quarantines 95% of the time, it's probably not worth it. I can 
guarantee that people will complain that our software is broken if it 
hardly ever rejects relevant messages.

We actually get very few requests for rejection at this level. 
(Sometimes we even have a hard time persuading people that rejecting 
messages because the recipient address is unknown is a good idea). Our 
users don't want to risk losing messages, so they seem to prefer 
quarantine to reject.

I can see that if we had already offered people 'reject-or-quarantine', 
and could currently only reject if the message was sent only to one 
recipient, then adding PRDR support would be worthwhile, but since we 
don't currently offer that option, I'm not feeling the pressure to add 
PRDR support and make it more complicated for users.

I suspect we may add PRDR 'support' in the server component (advertise 
and understand it, but don't use it - which is allowed, and trivial to 
do), and monitor other servers' support of it in the client component, 
to see if it's worth adding there.

-

---

From: Ned Freed <[email-masked]>
Date: Fri, 07 Mar 2014 11:50:54 -0800 (PST)

> response will be positive again.

I'm quite confidence your reading is incorrect - the ending text clearly
indicates that this is talking about a specific message in the context
of a single server.

> can bounce later anyway) ?

Of course PRDR allows you to respond positively to mail that will fail later.
Nothing says a server offering PRDR is performing final delivery, and given
that mail is a store-and-forward protocol that can involve many hops before
final delivery, how could a server possibly provide such a guarante.

Besides, we have a name for PRDR used for final delivery: We call that LMTP.
And even then message can fail to make to the actual recipient; that's what
failure MDNs are for.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Fri, 07 Mar 2014 11:59:40 -0800 (PST)

> if you want similar user visible behaviour whether or not the client

But that's the entire point of what I'm saying here - by intentionally
making the behavior inconsistent, you end up in a better place.

I also disagree that this is terribly hard to implement. Anyone who
has done LMTP or Sieve ereject - and lots of people have - has the necessary
code mostly in place. It would be better if this were identical
to LMTP in terms of response codes though.

> and without any new code needed, just by sending one body per recipient,

Not even close. An obvious problem is that this tickles spam filters
differently. A less obvious problem is unnecessary duplicates. Once split, it's
very difficult to recombine messages that travel different paths. So splitting
leads to duplicates. And while these may not bother you, it bothers other
people a *lot*, to the point wher it becomes a support call generator.

> whether or not the server side supports it.

That might be true were it not for LMTP. But a client with LMTP support can
accomodate PRDR pretty easily.

				Ned

---

From: "Peter J. Holzer" <[email-masked]>
Date: Sat, 8 Mar 2014 11:35:22 +0100

On 2014-03-05 21:37:46 +0000, Paul Smith wrote:
> So, if the sender didn't support PRDR, we'd have to accept the
> message, deliver it to the second user, and send a bounce message
> back (with risk of backscatter) for the first user.

That's not the only fall-back option, as you noted in another message.

> probably be the most common case people would use it, and that would
> be the most likely case to cause backscatter from accepting then
> sending bounce messages.

I'm not especially worried about spammers. For them, falling back to
"temporary reject and accept one recipient at a time" is acceptable[2].
I'm worried about normal users who send a mail to 100 addresses from
their address book. At normal retry rates such messages could take days
to deliver, which would not be acceptable for the recipients[1].  

But the number of organizations which would send a single (non-spam)
message to 100 of your users is probably low. Most importantly, it's
your own (for which simultaneously implementing PRDR on the client and
server side should be possible) and then there are maybe a handful of
partner organizations.

So you don't need universal deployment of PRDR to make PRDR with a
fallback to "temporary reject and accept one recipient at a time"
acceptable, just isolated pockets plus (maybe) the large ESPs.

	hp

[1] This is why I spent some effort on dynamically creating groups 
    of recipients to minimize the number of retries when I implemented
    per-recipient content filters for qpsmtpd.

[2] Well, maybe not for them, but for me, dealing with spammers.

-- 
   _  | Peter J. Holzer    | Der eigene Verstand bleibt gefühlt messer-
|_|_) |                    | scharf. Aber die restliche Welt blickt's
| |   | [email-masked]         | immer weniger.
__/   | http://www.hjp.at/ |   -- Matthias Kohrs in desd

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 10 Mar 2014 13:23:32 +0100

> possible result while not creating backscatter.

Please don't.  Sieve is not part of SMTP, strictly speaking, but such
new action could further obscure the meaning of PRDR.

Rather, draft prdr-00 ought to be reworked and clarified.  IMHO, the
"temporary reject and accept one recipient at a time" technique
(Paul's double-ugh) should be recommended as a fallback when the
sender doesn't support PRDR _and_ recipient mailboxes are set up so
that they could result in differing 2xx/4xx/5xx behavior.  That would
clarify the meaning, besides prompting implementations.

The draft looks badly written or inconsistent.  It does not make
explicit that reply codes can worsen but never get better.  Without
such statement, phrases like "SMTP clients MUST give priority to the
final response" could be misunderstood by casual readers.  By the same
argument, the draft seems to imply that a client can retry to send the
same message to users who already rejected it, provided that the
server finally quit with "421 Daemon going down for maintenance after
50 of 100 recipients" (at 10 mins each.)  Also, it is not clear why
clients should issue separate NDNs according to the reasons why each
recipient rejected a message, as that doesn't seem to minimize the
number of notifications.

jm2c
Ale

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 10 Mar 2014 09:41:30 -0700

On Wed, Mar 5, 2014 at 11:33 AM, Murray S. Kucherawy <[email-masked]>wrote:

>> the milter-side/API, so that one can send over it arbitrary responses after
I reached out to the author last week to ask if he'd like to work on it
through to publication, but I haven't received a reply yet.

Does this group have enough interest and commitment to pursue it?  Sounds
like at least Ned is interested in implementation for the sake of
experimentation (he should certainly correct me if I'm wrong), a patch
exists for sendmail, and exim already has it as an experimental option.

-MSK

---

From: Ned Freed <[email-masked]>
Date: Mon, 10 Mar 2014 10:18:44 -0700 (PDT)

> new action could further obscure the meaning of PRDR.

I never said, or even hinted, that the definition of such an action would be
part of standardizing PRDR.

> clarify the meaning, besides prompting implementations.

This has been done outside the context of PRDR by servers incapable of
delivering to more than one mailbox at a time. It tends to interact very poorly
with typical message retry policies, to the point that we've had to spend
not-inconsiderable time instructing customers on how to adjust their policies
to accomodate such hosts.

In short, this is a bad idea that has no business even being suggested,
let alone recommended.

> recipient rejected a message, as that doesn't seem to minimize the

Agreed; the draft needs quite a bit of work.

				Ned

---

From: Дилян Палаузов <[email-masked]>
Date: Mon, 10 Mar 2014 20:53:17 +0100

Hello,

about the PRDR current adoption, I see the problem not that much in the 
unimplemented PRDR facility on the server side, but rather in the lack 
of possibility to reject mail at SMTP level after DATA-fullstop, even in 
the simple case where the mail has only one recipient.

My personal preference is to reject the spam intended for me, and inform 
the sender immediately, instead of filling my spam folder, and loosing 
time to check in it, causing by this way huge delays until false 
classified ham is proceeded.  I guess many other users will have the 
same preferences, if it frees them from the obligation to check the spam 
folder.  This does not cause also additional support burden to 
postmasters, as the work to clarify why a mail is rejected because it 
was being evaluated as spam, is the same compared to the work explaining 
why the email ended in the spam folder.

For me a step in the adoption of PRDR is to ask, if the concrete user 
prefers spam mails to go in the spam folder or to be rejected on SMTP 
level.  There are also some variations possible, like emails which are 
very, very, very likely spam to be rejected at SMTP level, if the mail 
envelope has only one recipient, otherwise delivered to the spam or 
inbox folders.

Spam is not the only reason to reject mails after DATA-fullstop. 
Another issue are emails sent to mailing lists, where the rejection can 
be concluded only after evaluating From:, Resent-From:, Sender: and 
Resent-Sender: headers.  If the mailing list preference is to reject 
mails from non subscribers, and some entitled sender chooses by error 
the wrong sender address, then this email will probably have only one 
recipient on the envelope, and the mail can be rejected at SMTP level.

Asking why PRDR is not adopted and asking why in the above 
single-recipient-scenario emails are not rejected as SMTP level after 
DATA-fullstop, is essentially the same.

It is not the problem of the PRDR draft, it is not the problem with the 
lack of its current adoption, it is the design of the SMTP servers, that 
cannot decide during SMTP reception after DATA-fullstop, if the 
recipient wants the email or not, regardless of the possibility to 
reject it (e.g. when there is only one recipient).

I guess for most software it would be trivial to implement the 
client-part (interpret the response after DATA-fullstop-353); and 
currently the sole reason for the limited use of PRDR is the amount of 
work to make the SMTP-server part of the MTA able to decide at SMTP 
level, if the recipient wants the message.

If this logic was already in the servers, they could accept only the 
first recipient from the mail envelope and send temporary rejection to 
all the others.  But on the same time deliver the email promptly to all 
recipients, that want the email.  In this case the sender might get a 
message, that the email is not delivered yet (after four hours), however 
the email will be in the mailboxes of the recipients at that time. 
Under this circumstances only the case has to be dealt with, when the 
recipients have promptly received the emails, but the sender gets the 
wrong message, that her/his message was not delivered yet.

Depending how the temporary rejection is done, there might be no 
administrative burden with such messages.  Some 451-text like this shall 
reduce the questions:
   The sending server does not implement the PRDR-SMTP extension, that's 
why it is not possible to
   tell the sending MTA immediately, if all recipients have already 
received the message.  The
   recipients that wanted to receive the message, have already received 
it.  It takes some time to
   tell the sending MTA, which recipients have rejected the message, 
hence the current notification.

Going a step further, a new enhanced status code can be added to 
draft-hall-prdr-00.txt, that is returned when the temporary reject is 
done, because the sending client does not ask for PRDR.  On the server 
side it shall be also not that difficult to track, if a message is 
delivered for the second time (once when the mail was rejected 
temporary, and once, when the sender retries and the message is 
accepted), as the double delivery suppression functionality shall 
already be there.

As for Sendmail and Postfix, this could be done by milters, which 
evaluate Sieve scripts and enforce SMTP reject.  More generally, this 
can be achieved by tweaking the milter interface to support per 
recipient logic/data/callbacks.  I think this is what the pmilter 
interface of META1 does.

As for the question, about why shall a server print for every recipient 
a response, instead of using one reply text, if all recipients reject 
the message, the answer is, that the rejection text can differ per 
recipient.

Kind regards
   Dilyan

---

From: Brandon Long <[email-masked]>
Date: Mon, 10 Mar 2014 13:04:26 -0700

The longer you hold up the response to receiving the data content, the
more likely you are that some connection level event will occur,
causing a re-send of the message.

So, if you can take up to 30s to do proper spam checking of a message,
how many dups are you willing to put up with?

Also, if you don't let some of the spam in, its much harder for the
system to get false positives marked as not spam and learn.

Certainly, some spam systems do a better job of delineating between
"very very spammy" and not, but that isn't always obvious.

And I think spammers would figure out pretty quickly if they got
different behavior between spamming a single address and multiple.

And I'm really not a fan of the accept but say we didn't 'yet' method,
that sounds pretty awful.

We certainly reject a good percentage of spam at smtp time, but
rejecting all spam at smtp time isn't really possible.  We are guilty
on the mailing list permission backscatter though, as with anything,
its possible to fix it, just hasn't reason high enough on the
priority.

Brandon

On Mon, Mar 10, 2014 at 12:53 PM, Дилян Палаузов
<[email-masked]> wrote:
> unimplemented PRDR facility on the server side, but rather in the lack of
> possibility to reject mail at SMTP level after DATA-fullstop, even in the
> simple case where the mail has only one recipient.
>
> My personal preference is to reject the spam intended for me, and inform the
> sender immediately, instead of filling my spam folder, and loosing time to
> check in it, causing by this way huge delays until false classified ham is
> proceeded.  I guess many other users will have the same preferences, if it
> frees them from the obligation to check the spam folder.  This does not
> cause also additional support burden to postmasters, as the work to clarify
> why a mail is rejected because it was being evaluated as spam, is the same
> compared to the work explaining why the email ended in the spam folder.
>
> prefers spam mails to go in the spam folder or to be rejected on SMTP level.
> There are also some variations possible, like emails which are very, very,
> very likely spam to be rejected at SMTP level, if the mail envelope has only
> one recipient, otherwise delivered to the spam or inbox folders.
>
> Spam is not the only reason to reject mails after DATA-fullstop. Another
> issue are emails sent to mailing lists, where the rejection can be concluded
> only after evaluating From:, Resent-From:, Sender: and Resent-Sender:
> headers.  If the mailing list preference is to reject mails from non
> subscribers, and some entitled sender chooses by error the wrong sender
> address, then this email will probably have only one recipient on the
> envelope, and the mail can be rejected at SMTP level.
>
> It is not the problem of the PRDR draft, it is not the problem with the lack
> of its current adoption, it is the design of the SMTP servers, that cannot
> decide during SMTP reception after DATA-fullstop, if the recipient wants the
> email or not, regardless of the possibility to reject it (e.g. when there is
> only one recipient).
>
> I guess for most software it would be trivial to implement the client-part
> (interpret the response after DATA-fullstop-353); and currently the sole
> reason for the limited use of PRDR is the amount of work to make the
> SMTP-server part of the MTA able to decide at SMTP level, if the recipient
> wants the message.
>
> If this logic was already in the servers, they could accept only the first
> recipient from the mail envelope and send temporary rejection to all the
> others.  But on the same time deliver the email promptly to all recipients,
> that want the email.  In this case the sender might get a message, that the
> email is not delivered yet (after four hours), however the email will be in
> the mailboxes of the recipients at that time. Under this circumstances only
> the case has to be dealt with, when the recipients have promptly received
> the emails, but the sender gets the wrong message, that her/his message was
> not delivered yet.
>
>   The sending server does not implement the PRDR-SMTP extension, that's why
> it is not possible to
>   tell the sending MTA immediately, if all recipients have already received
> the message.  The
>   recipients that wanted to receive the message, have already received it.
> It takes some time to
>   tell the sending MTA, which recipients have rejected the message, hence
> the current notification.
>
> draft-hall-prdr-00.txt, that is returned when the temporary reject is done,
> because the sending client does not ask for PRDR.  On the server side it
> shall be also not that difficult to track, if a message is delivered for the
> second time (once when the mail was rejected temporary, and once, when the
> sender retries and the message is accepted), as the double delivery
> suppression functionality shall already be there.
>
> As for Sendmail and Postfix, this could be done by milters, which evaluate
> Sieve scripts and enforce SMTP reject.  More generally, this can be achieved
> by tweaking the milter interface to support per recipient
> logic/data/callbacks.  I think this is what the pmilter interface of META1
> does.
>
> As for the question, about why shall a server print for every recipient a
> response, instead of using one reply text, if all recipients reject the
> message, the answer is, that the rejection text can differ per recipient.
>
> Kind regards
>   Dilyan
>
>
> _______________________________________________
> ietf-smtp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-smtp

---

From: Ned Freed <[email-masked]>
Date: Tue, 11 Mar 2014 11:31:51 -0700 (PDT)

> That's not the only fall-back option, as you noted in another message.

In fairness, it kind of is the only option if you reject outright the idea of
silently discarding messages.

I think the "never discard" approach is far out on the raggedy edge, but
that does not make the approach ipso facto invalid.

> to deliver, which would not be acceptable for the recipients[1].

Bingo. This is part of why the approach of rejecting all but one recipient at 
RCPT TO time is a nonstarter.

> server side should be possible) and then there are maybe a handful of

Well, all I have to offer is anecdotal evidence, but I work with around
5 local args, and all of them have mailing lists this large or larger.

> acceptable, just isolated pockets plus (maybe) the large ESPs.

You certainly don't need universal deployment of PRDR for it to be useful,
but that doesn't make temporary RCPT TO reject palatable.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 12 Mar 2014 10:49:05 +0100

On Tue 11/Mar/2014 19:31:51 +0100 Ned Freed wrote:
>>> message, deliver it to the second user, and send a bounce
>>> message back (with risk of backscatter) for the first user.
>> 
> In fairness, it kind of is the only option if you reject outright
> the idea of silently discarding messages.
> 
> I think the "never discard" approach is far out on the raggedy
> edge, but that does not make the approach ipso facto invalid.

Not invalid sounds a bit weak, considering that the fall-back of
choice is going to be the server workhorse, at least until PRDR is not
widely deployed.  It is the only way that a server can allow
per-recipient configurations that are meaningful independently of the
technical capabilities of the sender.

>>> In any case, spammers would probably not support PRDR, and
>>> that'd probably be the most common case people would use it,
>>> and that would be the most likely case to cause backscatter
>>> from accepting then sending bounce messages.
>>
>> I'm not especially worried about spammers. For them, falling back
>> to "temporary reject and accept one recipient at a time" is
>> acceptable[2]. I'm worried about normal users who send a mail to
>> 100 addresses from their address book. At normal retry rates such
>> messages could take days to deliver, which would not be
>> acceptable for the recipients[1].
> 
> Bingo. This is part of why the approach of rejecting all but one
> recipient at RCPT TO time is a nonstarter.

Right, done that way it makes no sense.  However, it is possible to
split recipients into two groups, say, those who whitelisted the
message based on the envelope, and those who wanted to examine the
content before making a choice.  Normal rates do provide for a few
quick retries in a row.

For the spammers vs. normal users distinction, I read papers that work
out some kind of score by analyzing how recipients are related to one
another and to the sender.  I never saw an actual how-to for
production servers, though.  Perhaps, the normal spam threshold can be
lowered if the recipients are not well clustered.  Then, the first
recipient of the second group will decide whether to accept the
message also on behalf of the others.  The rationale is that overall
email consistency is improved by honoring CC directives.  Could such
reasoning be refined into a fall-back recommendation?

Ale

---

From: Ned Freed <[email-masked]>
Date: Wed, 12 Mar 2014 18:51:17 -0700 (PDT)

> technical capabilities of the sender.

I'm not sure what this is supposed to mean. I was talking about having an
overarching policy of never discarding a message. Aside from a few small
setups, I don't think I've ever seen anyone implement such a policy in the past
decade or so. Like it or not, a lot of mail believed to be spam is silently
dropped more often than not, and mail believed to to be dangerous in someway is
almost always discarded.

As such, I'm not especially worried about how PDRD interacts with a
never-discard policy. I'm much more interested in how it lets us replace
discard with reject.

> content before making a choice.  Normal rates do provide for a few

Sorry, that's not something you can count on. Our implementation retries
partial failures quickly by default but that's not true in general.

And I don't think non-overrideable envelope whitelisting is all that common. In
fact I'm not even sure a lot of implementations support those semantics in any
useful way.

> reasoning be refined into a fall-back recommendation?

I'm extremely dubious about standardizing use of specific types of data
analytics in the AS/AV space.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 14 Mar 2014 12:56:50 +0100

On Thu 13/Mar/2014 02:51:17 +0100 Ned Freed wrote:
>> choice is going to be the server workhorse, at least until PRDR
>> is not widely deployed.  It is the only way that a server can
>> allow per-recipient configurations that are meaningful
>> independently of the technical capabilities of the sender.
> 
> I'm not sure what this is supposed to mean.

I'm sorry I garbled that paragraph.

> I was talking about having an overarching policy of never
> discarding a message. Aside from a few small setups, I don't think
> I've ever seen anyone implement such a policy in the past decade or
> so. Like it or not, a lot of mail believed to be spam is silently 
> dropped more often than not, and mail believed to to be dangerous
> in someway is almost always discarded.

Indeed, RFC 5321 says:

   As discussed in Section 7.8 and Section 7.9 below, dropping mail
   without notification of the sender is permitted in practice.

I do so for viruses, thanks to the extremely low false-positive rate.

> As such, I'm not especially worried about how PDRD interacts with
> a never-discard policy. I'm much more interested in how it lets us
> replace discard with reject.

Yes, but you can only do so if the client supports it.  That may give
rise to the long-winded road to adoption that was discussed last week:
 Servers avoid discarding when that creates more problems than it
solves.  Rejecting works better, so it allows more room for anti-spam
solutions.  Per-recipient settings individually allow more room too,
but thus far that implies discarding.  So it seems one can improve
along one of two orthogonal directions only.  Is that the reason why
many spammers send to multiple recipients?

> I'm extremely dubious about standardizing use of specific types of
> data analytics in the AS/AV space.

We don't need to "standardize" them, so long as it's clear what we're
talking about.  The case of a server facing a large number of RCPT TO
commands is also in Section 7.8 of RFC 5321.  Since that is the meat
of PRDR, it ought to elaborate on that topic a little bit more, IMHO.

Ale

---

From: John C Klensin <[email-masked]>
Date: Fri, 14 Mar 2014 13:42:43 -0400

Hi.

Since I'm still holding the pen on any possible replacement for
RFC 5321 (and have been waiting for over six months for a
response from the ADs about how to handle that), a few
observations from that perspective...

--On Friday, March 14, 2014 12:56 +0100 Alessandro Vesely
<[email-masked]> wrote:

>> discarding a message. Aside from a few small setups, I don't
>> think I've ever seen anyone implement such a policy in the
>> past decade or so. Like it or not, a lot of mail believed to
>> be spam is silently  dropped more often than not, and mail
>> believed to to be dangerous in someway is almost always
>> discarded.
> 
> Indeed, RFC 5321 says:
> 
>    As discussed in Section 7.8 and Section 7.9 below, dropping
> mail    without notification of the sender is permitted in
> practice.

Yes.  And that particular approach to weasel-wording --
requiring either delivery or rejection (with notices where
rejection cannot be provided at SMTP time) and then explicitly
providing an escape -- reflects the strong consensus of the
group at the time.

Temporarily moving the discussion up several levels, there are
only three possible models for email (or, for that matter,
postal mail) acceptance and delivery:

(i) Delivery is reliable and can be assumed.  Undelivered
messages will be returned or the sender appropriately notified.

(ii) Delivery is unreliable and should not be assumed absent
explicit acknowledgment by the addressee.  For this model to
work well and not segue into the third, the recipient must not
be able to receive and examine the content of a message (and
possibly not even the envelope) without generating a reliable
acknowledgment. 

(iii) Delivery is unpredictable: When one sends a message, one
has no idea whether it will be delivered or not, whether it has
been received or not, etc.  Any acknowledgments or rejection
notices are purely optional to the delivery system, the
recipient, or various intermediaries.

Internet mail was designed around the first assumption.  We are,
for better or worse, sliding toward the third.  It is impossible
(at least for me) to quantify the degree of that effect, but the
more unpredictable things get, the more it drives people toward
use of IM-like technologies in preference to email: those
technologies permit better sender and recipient authentication,
have the advantage of direct (and easily-encrypted) links
without worrying about interference by intermediaries, may be
more immediate than email if the latter is to be intercepted and
scanned/ analyzed in transit, and provide immediate information
about delivery (or non-delivery).  

>> As such, I'm not especially worried about how PDRD interacts
>> with a never-discard policy. I'm much more interested in how
>> it lets us replace discard with reject.
> 
> Yes, but you can only do so if the client supports it.

The client and, to at least some degree, the delivery server or
other systems acting as its surrogate.

>  That
> may give rise to the long-winded road to adoption that was
> discussed last week:  Servers avoid discarding when that
> creates more problems than it solves.  Rejecting works better,
> so it allows more room for anti-spam solutions. 

I'm not persuaded that is true.  I'm not a spammer (and
presumably neither are you) so it may be hard for either of us
to understand how  they think but I'd guess a few things are
obvious:

(i) Their goal is to get as many messages through to the
recipients who will act as they desire on those messages as
possible.  

(ii) Given that the costs of sending an extra message (or 10)
are very low and those of attaching extra addresses to a given
message are even lower, providing the actual spam-senders with
information about which addresses are valid and which are not
doesn't have much benefit for spam-fighting.  It may, however,
be of advantage to those who compile lists of valid,
message-accepting, addresses.  The latter, as well as avoidance
of blcwback, may actually make discarding a better option than
selective rejection.  

(iii) As others have pointed out, making rules that one expects
spammers (and anyone else operating at or over the boundaries of
good taste and good practices) to follow is typically a fool's
errand.

> Per-recipient
> settings individually allow more room too, but thus far that
> implies discarding.  So it seems one can improve along one of
> two orthogonal directions only.  Is that the reason why many
> spammers send to multiple recipients?

See above.
 
>> I'm extremely dubious about standardizing use of specific
>> types of data analytics in the AS/AV space.
> 
> We don't need to "standardize" them, so long as it's clear
> what we're talking about.  The case of a server facing a large
> number of RCPT TO commands is also in Section 7.8 of RFC 5321.
> Since that is the meat of PRDR, it ought to elaborate on that
> topic a little bit more, IMHO.

My current hypothesis is "I don't think so" and, as implied
above, nothing is likely to happen any time soon anyway, but
send text and see if you can get consensus for it.  Better yet,
follow the advice that has been given (not just by me) to many
others who have asked for substantive changes to 5321: create an
I-D that updates it and see if you can get enough traction to
get it approved as a Proposed Standard.  If you can do so and
then demonstrate the interest and interoperability needed to
advance it, the odds of incorporation into some future 5321bis
are pretty high.  For procedural reasons even if there were no
substantive ones, the odds of actual modifications in 5321bis
without such standalone documents are vanishingly small: the
only exceptions would be for clarifying text, obvious
corrections, or editorial changes that have no substantive
impact on the general understanding of the specification.

     john

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 14 Mar 2014 20:48:54 +0100

Hi John,

> providing an escape -- reflects the strong consensus of the

I don't think that particular snippet needs to be revised.  It never
went on the YAM database.

> recipient, or various intermediaries.

The last model is the only possibly real one, given that perfection is
so difficult to attain, especially on large, heterogeneous systems.

> about delivery (or non-delivery).

Agreed.  Of course, neither I can quantify how much we're into model
(iii), but --thanks to Jon Postel, you, and myriad others-- it seems
reliable enough to enjoy continued use.

> other systems acting as its surrogate.

Unlike LMTP, an organization deploying PRDR cannot control how clients
connect to an MX.  The point raised by Paul[1] is that it is difficult
to improve per-recipient solutions without a good fall-back method.

[1] http://www.ietf.org/mail-archive/web/ietf-smtp/current/msg07676.html

> of blowback, may actually make discarding a better option than
> selective rejection.  

That's true.  However, when false positives are too frequent the
resulting behavior falls too deeply into model (iii).  In those cases,
rejection likely gets back to the author.  S/he may or may not want to
retry, but having a chance to understand what went on will not blame
the mail system as a whole for the malfunction.  IOW, false positives
are more endurable if they are notified.  Thus, those who configure
anti-spam filters can get a little more leeway by rejecting rather
than dropping.

> good taste and good practices) to follow is typically a fool's

Yes.

>> spammers send to multiple recipients?

Per-recipient solutions can also be run on the MUA, if dropping is the
only alternative.  That makes them completely orthogonal to boundary
filters.  In any case, since that's in users hands, they are
responsible to find what suits them better.

> send text and see if you can get consensus for it.

The OP[2] apparently suggested to unexpire that 7-year old draft.  Its
introduction makes it clear that PRDR is meant to improve anti-spam
boundary filtering.  My point is that we'd need more insight into the
relationship between a message's spamminess and the presence of
multiple recipients, if there is any.  Such insight may suggest a good
fall-back method, or it may suggest to forget PRDR tout-court.

[2] http://www.ietf.org/mail-archive/web/ietf-smtp/current/msg07655.html

>  Better yet,

I don't think PRDR requires any changes in RFC 5321.  But perhaps
rfc5321bis will happen before prdr-01...

Ale

---

From: "John Levine" <[email-masked]>
Date: 14 Mar 2014 20:52:44 -0000

>to understand how  they think ...

You are correct that spammers don't care whether any individual
message gets delivered.  All they care about is volume.  This is why
the conventional wisdom not to unsubscribe from spam is wrong; they
already have your address, and if the sender happens to be somewhat
legitimate, they might actually stop.

There's only one reason to prefer reject to bounce, which is that in
the case that the message has a bounce address of someone unrelated to
the actual sender, rejections are far less likely to provoke blowback
spam to the bounce address.  This has very little to do with spam
filtering other than that filters that run during the SMTP session
can provoke rejects. and spam is disproportionately likely to have
fake bounce addresses.

R's,
John

---

From: Ned Freed <[email-masked]>
Date: Sat, 15 Mar 2014 17:23:16 -0700 (PDT)

> Yes, but you can only do so if the client supports it.

Of course.

> That may give
> solutions.

You're missing the point. I'm not talking about servers that avoid discard. As
I've said several times now, most servers I'm aware of have a never-reject
policy. Never-discard (or sometimes-discard) policies are the rare exception.

PRDR support lets a server reject messages it would otherwise discard,
providing more information to senders about false positives than they would
otherwise get. That alone makes it advantagous for servers to implement,
because now they can say that it's up to the client to do something if they
want more information.

> Per-recipient settings individually allow more room too,

> of PRDR, it ought to elaborate on that topic a little bit more, IMHO.

The current draft is nowhere near ready for standardization. This is
but one of many details that need to be looked at.

The concern now isn't whether the draft is up to snuff, but rather whether this
idea is worth pursuing. I originally thought it wasn't, but it coupled with a
new handling for suspected spam changes things for me.

				Ned

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Mon, 17 Mar 2014 23:30:19 -0400

At 20:48 +0100 on 03/14/2014, Alessandro Vesely wrote about Re: 

>responsible to find what suits them better.

Note that reliable Per-Recipient reporting to the MUA (so it can 
react to and handle the reports that the message was accepted for 
delivery or the delivery request was rejected) depends on the MUA 
using as its Mail Submission Server for the message one run by the 
Recipient's ISP (and that all the recipients are serviced by THAT 
SMTP Gateway Server). So long as it uses a server that will only 
forward it to a Server responsible for the email domain instead of 
one which would accept responsibility for the delivery then there is 
no way that a Per-Recipient submit time response to the RCPT TO can 
occur (since a MSA Server does not know if the message will be 
accepted).

As an example, if I wanted to send a message to Alessandro, I would 
ONLY be able to get a submission time accept/reject if it was being 
sent by my MUA directly to the tana.it MX server. If I send though MY 
ISP's submission server it will forward to the MX server and I will 
only an email reject message not a reject at submit time.

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Mon, 17 Mar 2014 23:35:58 -0400

At 20:52 +0000 on 03/14/2014, John Levine wrote about Re: [ietf-smtp] 
Per-Recipient Data Responses:

>the conventional wisdom ot to unsubscribe from spam is wrong; they

OTOH: The assumption behind that conventional wisdom is that the 
unsubscribe acts as a active account validation and thus makes the 
address more valuable to spammers since they know that the address 
got accepted as opposed to possibly having bounced.

---

From: "John R Levine" <[email-masked]>
Date: 18 Mar 2014 00:35:18 -0400

> OTOH: The assumption behind that conventional wisdom is that the unsubscribe 
> acts as a active account validation and thus makes the address more valuable 
> to spammers since they know that the address got accepted as opposed to 
> possibly having bounced.

In this case the conventional wisdom is just wrong.  Disregarding exotic 
stuff like spear phishing, all they care about is volume, not list 
quality.  There is no evidence that spammers sell lists of unsub 
addresses.

The FTC did an actual experiment a few years ago, where they set up two 
groups of accounts, clicked all the unsub links in one group's spam, and 
then counted the subsequent mail.  The unsub group got less spam.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

---

From: Russ Allbery <[email-masked]>
Date: Mon, 17 Mar 2014 23:35:53 -0700

> then counted the subsequent mail.  The unsub group got less spam.

Yes.  I've known people who have reproduced that experiment, and I've a
small version of it myself, all with the same results.

It's a pain in the ass, and probably not worth the time investment versus
just improving one's local filters, but it does reduce (although certainly
not eliminate by any stretch) the spam.

Visiting the unsub links in a browser that's relatively immune to malware
is a good idea, of course.

-- 
Russ Allbery ([email-masked])              <http://www.eyrie.org/~eagle/>

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 18 Mar 2014 09:56:45 +0100

On Tue 18/Mar/2014 04:30:19 +0100 Robert A. Rosenberg wrote:
> Note that reliable Per-Recipient reporting to the MUA (so it can react
> to and handle the reports that the message was accepted for delivery
> or the delivery request was rejected) depends on the MUA using as its
> Mail Submission Server for the message one run by the Recipient's ISP
> (and that all the recipients are serviced by THAT SMTP Gateway
> Server). So long as it uses a server that will only forward it to a
> Server responsible for the email domain instead of one which would
> accept responsibility for the delivery then there is no way that a
> Per-Recipient submit time response to the RCPT TO can occur (since a
> MSA Server does not know if the message will be accepted).
> 

Yup, that's regular SMTP behavior.  There is no advantage in /getting/
a report directly at submission time:  There is increased reliability
in /issuing/ a rejection decision online, as that does not risk to
cause backscatter.  There is no reduced reliability in letting a
submission server issue a DSN to the authenticated user it relayed for.

Afterthoughts taking place after acceptance ought to go through a
different path, such as that described in RFC 6650.  For example, is
my MUA had erroneously classified your message as spam, the tana.it
mail server should have noticed the move to the spam folder; then it
should have sent a report to [email-masked], based on the SPF
authentication verified at acceptance time.  That path is very
unreliable, as my server currently doesn't do that (yet), and even if
it did, it is not clear that the report would have reached the
original author.

Ale