Subject: Sieve reject at SMTP time possible with which implementations?

From: "Matthew Elvey (FM)" <[email-masked]>
Date: Wed, 22 Oct 2003 01:06:25 -0700

Sieve reject at SMTP time possible with which implementations?

In other words, what Sieve implementations can be configured such that a 
reject causes the email to be refused/not accepted during the SMTP 
transaction, instead of accepted, and then bounced back to the alleged 
sender (often, in the case of spam, a Joe-Job victim).  Someone just 
told me that Cyrus' implementation can't do this.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 22 Oct 2003 13:32:20 +0200

> Sieve reject at SMTP time possible with which implementations?

None that I know about.

> In other words, what Sieve implementations can be configured such that 
> a reject causes the email to be refused/not accepted during the SMTP 
> transaction, instead of accepted, and then bounced back to the 
> alleged sender (often, in the case of spam, a Joe-Job victim).  
> Someone just told me that Cyrus' implementation can't do this.

That sounds correct.

--Arnt

---

From: Rob Siemborski <[email-masked]>
Date: Wed, 22 Oct 2003 10:23:47 -0400 (EDT)

> told me that Cyrus' implementation can't do this.

Note that Cyrus's implementation is actually LMTP, which presumes an
SMTP server will be sitting right next to it, so even if a 5xx failure was
issued, a bounce would still be generated by the SMTP server.

In any case, RFC3028 specifically says that an MDN is sent back when a
reject action happens.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

---

From: [email-masked]
Date: Thu, 23 Oct 2003 05:10:15 -0700 (PDT)

> In any case, RFC3028 specifically says that an MDN is sent back when a

Which in turn means that handling reject at the SMTP level would be incompliant
with the sieve specification.

Of course nothing prevents an implementation from adding a separate action
other than reject for this. But SMTP level errors don't match up to sieve
actions very well. Why? Because sieve evaluation almost always requires having
the entire message on hand. In SMTP this means the only reponse code that's
left is the last one, where the server accepts or rejects the message on behalf
of all recipients. This becomes a problem when there are multiple recipients
and some reject the message and some do not. The result is that SMTP level
rejection can never be anything but a special case for user sieves.

				Ned

---

From: Rob Siemborski <[email-masked]>
Date: Thu, 23 Oct 2003 08:24:50 -0400 (EDT)

> with the sieve specification.

Yes, thats what I meant ;)

> and some reject the message and some do not.

While difficult/impossible in the SMTP situation, it can be done at the
"SMTP level," provided the last hop is over LMTP instead of SMTP, since
LMTP gives per-recipient status codes.

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

---

From: "Matthew Elvey (FM)" <[email-masked]>
Date: Thu, 23 Oct 2003 15:39:06 -0700

>>and some reject the message and some do not.
I would like to see such an action (Shall we call it "refuse"?) in the 
next version of the spec, or in an extension.  Anyone else?
Something like "refuse MUST reject an email at the SMTP level, instead 
of sending an MDN, where possible, as this would be an anti-Joe-Job feature"
It will otherwise behave like reject, but only permit a single line of 
response text.  Issues of specifying response codes (4xx, 5xx, 4.x.x, 
etc) would need to be hashed out, 'where possible' would address the 
problem Ned raised, above.
MDNs cause rejected joe-job spam to hit the victim's mailbox; SMTP level 
refusals usually don't.
Revising reject's behaviour instead might be problematic as 
long/multiline responses that reject currently allows might be imposible 
to send at SMTP time.  (I'm guessing on this one; maybe it isn't a 
problem to stuff multiline responses as one very long line.)

Thanks for all the comments, everyone!

---

From: [email-masked]
Date: Sat, 25 Oct 2003 21:48:17 -0700 (PDT)

> problem Ned raised, above.

It would have to be done as an extension; adding something like this to
the base specification isn't realistic at this point.

I personally would have no objection to this extension.

> MDNs cause rejected joe-job spam to hit the victim's mailbox; SMTP level

Currently the best way to deal with this sort of trash is to use discard,
not reject. Reject really isn't intended to be used as an antispam tool.
As such, a refuse action is best compared to discard, not reject.

> problem to stuff multiline responses as one very long line.)

Revising reject's behasvior is a nonstarter IMO.

				Ned

---

From: "Matthew Elvey (FM)" <[email-masked]>
Date: Wed, 29 Oct 2003 15:17:01 -0800

> I personally would have no objection to this extension. 

Here goes!

I hope someone here will help shepherd such an extension through the 
IETF process.

The procedure is rather overwhelming; while I'd like to contribute,
perhaps I need not become a certified IETF labyrinth navigator in the 
process. :)
OTOH, there's a son-of-GTUBE EICAR-like standard I'd like to create, so 
if no one is willing,
I can try to kill two birds with one stone. (Learn the skills to 
shepherd two specs, that is.)
If no one is willing, I'll need to figure out stuff like
Is there a WG for handling Sieve extensions?   Do I need to be a paying 
IETF member (prohibitive cost?)
Is there a good editor/emacs mode/Windows app for writing 
Internet-Drafts/RFCs?
Put the following into the boilerplate, etc?  I see Ned is a prolly very 
busy Apps director(!)

First draft!  (With some commentary about open issues mixed in; I know 
it's not in the right format, and it's totally unedited, but I'm 
drained, so here goes.)  Feedback welcomed by email or on-list.

Abstract

MDNs cause returned Joe-job spam to hit the Joe-job victim's mailbox; 
SMTP level refusals usually don't.  'refuse' uses the latter method to 
return unwanted email.

In other words, Sieve should have the capability such that  email may be 
flat out not accepted

during the SMTP transaction, instead of accepted, and then bounced back 
to the alleged sender (often, in the case of spam, a

Joe-Job victim). 

Discussion
   
    The SIEVE mail filtering language [SIEVE] "refuse" extension, if 
supported, permits users to handle unwanted email in a

way that is sometimes preferable to the existing 'discard' and 'reject' 
capabilities.  When a spam-detection system suspects

a message is spam, but isn't certain, discarding the email is considered 
too risky for some users, for example, those who

receive sales leads by email.  They are willing to use the reject 
command.  Users are willing to reject but not discard because the sender 
of an email incorrectly marked as spam will receive a notification that 
the email was refused, and will likely try again to contact the intended 
recipient, perhaps via another method of communication.  Unfortunately, 
using is problematic, because in the usual case, the email is indeed 
spam, and the alleged sender to whom the MDN caused by the reject will 
be sent will often be an innocent "Joe-job" victim.  'Refuse' is 
intended to be superior to 'reject' because it

will be less likely to result in email to an innocent victim (a 
'Joe-job').  'refuse' refuses to accept an email for delivery instead of 
accepting it and then sending an MDN.  Most spam is sent through open 
proxies, so 'refuse' eliminates most Joe-job bounces resulting from 
usage of reject. 'Reject' will also reduce Joe-jobs caused by virus 
self-propagation via emails with false sender information.

Change History (to be removed prior to publication as an RFC)

    Changes from -00 to -01:

Table of Contents
       
     1  Introduction and Overview . . . . . . . . . . . . . . . . . .  3
     2  Conventions Used in This Document  . . . . . . . . . . . . . . 3
     3  SIEVE Extensions  . . . . . . . . . . . . . . . . . . . . . .  4
       3.1  General Considerations . . . . . . . . . . . . . . . . . . 4
       3.2  Test spamtest . . . . . . . . . . . . . . . . . . . . . .  4

     4  Security Considerations . . . . . . . . . . . . . . . . . . . 
     5  IANA Considerations  . . . . . . . . . . . . . . . . . . . . .
       5.1  spamtest registration . . . . . . . . . . . . . . . . . . 
     6  References  . . . . . . . . . . . . . . . . . . . . . . . . . 
       6.1  Normative References . . . . . . . . . . . . . . . . . . .
       6.2  Non-Normative References  . . . . . . . . . . . . . . . . 
     7  Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . .
     8  Author's Address  . . . . . . . . . . . . . . . . . . . . . . 

     9  Intellectual Property Rights Statement . . . . . . . . . . . .
    10  Full Copyright Statement  . . . . . . . . . . . . . . . . . . 

1 Introduction and Overview

'refuse' MUST refuse to accept for an email delivery at the SMTP level, 
instead of sending an MDN, other than for the specified exceptions.  A 
SIEVE implementation that cannot do so MUST NOT claim to support the 
refuse extension.

Where the suspected spam source is believed to be an open relay (e.g. as 
determined by a RBLDNS lookup), the implementation MAY send an MDN when 
'refuse' is indicated, as an MDN allows more opportunity for being 
handled in an automated fashion by its recipient because of its 
prescribed, relatively informative format.  Systems that support the 
extension MUST encourage use of "refuse" over 'discard' and 'reject' 
where it is an appropriate alternative. 
 
"Extensions which define actions MUST state how they interact with
   actions discussed in the base specification." - [SIEVE]

Reject is considered incompatible with most other actions:
For Cyrus, the list is 
:FILEINTO_KEEP_REDIRECT_REJECT_VACATION_SETFLAG_ADDFLAG_REMOVEFLAG__MARK_UNMARK); 
refuse should be consistent.
(I actually think reject should be considered compatible though! What if 
I want to log rejected email?  As a user, I can't!)

The spamtest and virustest extensions SHOULD be supported.

Specifying response codes (4xx, 5xx, 4.x.x, etc)

Refused emails are refused with a response code of 550.

This is a really messy topic.  Should we allow the user to specify the 
response code?  Should they be able to select just one of a set of 
standard codes?  Or should they have no choice?
RFC2821, an Internet Standard, says a 550 is appropriate in response to 
DATA, but in one part frets about 550 as a response code when there are 
multiple recipients, a case that is dealt with in the security portion 
of this document.
550 and 552 are common responses, but 552 is not appropriate per RFC2821.
As originally conceived, 'refuse' is only for use with email identified 
as spam or virus-infested, however users may wish to refuse mail for 
other reasons, such as a wish to only accept encrypted email.  Although 
concievably not all possible reasons may fall under the umbrella of 550, 
the KISS principle suggests that it's good enough.

There does not seem to be an RFC that specifies the way for extended 
status codes of RFC1893 to be provided during an SMTP session.  It seems 
that RFC1891 codes are only for use in emailed DSNs, not as SMTP reply 
codes.  I haven't heard of a usage such as "550 5.7.0 SpamAssassin 
thinks the message is spam."
RFC1891 specifies a way for RFC1894-compliant DSN emails using extended 
status codes of RFC1893 to be requested during an SMTP session, but it 
does NOT specify a way for them to be used during the SMTP session itself.
RFC1893 specifies the extended codes, but not where/how they should be used.
It also doesn't specify codes that seem well well suited to the case of 
email unwanted by the recipient.
The only one the makes sense (and only for a small subset of email one 
might want refused) is 5.6.x for unwanted HTML email.
Email from IPs in blocklists, or that has failed a virus-detection, 
Bayesian or DCC type test doesn't fit well in any category. 5.7.0 
("undefined security status") is the best fit.  Again, the KISS 
principle suggests that a fixed code of 550 is good enough.

SMTP command replies containing response codes can be multi-line, per 
RFC2821, p.44.

   
2 Conventions Used in This Document
   
    Conventions for notations are as in [SIEVE] section 1.1, including
    use of [KEYWORDS].
   
    This document does not attempt to define
    what exactly constitutes a spam or virus containing email or how it 
should be identified, or
    what actions should be taken when detected.

3 SIEVE Extensions
   
3.1 General Considerations
   
    The "refuse" action described below ...
   
3.2 Test spamtest
   
        Syntax: refuse <reason: string>
   
   In the following script, messages that test highly positive for spam 
are refused.

   Example:
    if spamtest :value "ge" :comparator "i;ascii-numeric" "6" {
                refuse text:
         5.7.0 SpamAssassin thinks the message is spam.
     It is therefore being refused.
         Please call 1-900-PAY-US if you want to reach us.
.
                         ;       
    elsif spamtest :value "ge" :comparator "i;ascii-numeric" "4" {
                fileinto "Suspect";
                 }

    SIEVE implementations that implement the "refuse" action have an
    identifier of "refuse" for use with the capability mechanism.

4 Security Considerations
   
    SIEVE implementations SHOULD ensure that 
    can only occur for messages that have gone through a
    legitimate spam or virus check process. 

Sieve evaluation as typically implemented requires having the entire 
message on hand.
In SMTP this means the only response code that's left is the final one, 
where the server accepts or rejects the message on behalf of all 
recipients. This becomes a problem when there are multiple recipients 
and some reject the message and some do not.  In this case, reject MUST 
cause an MDN (multiple MDNs?) to be sent on behalf of the rejecting 
recipients. (Idea: allow MTA to tempfail the rejecting recipients and 
accept the others, until all recipients are rejecting the email; then it 
can be refused.)

    Beyond that, the "refuse" extension does not raise
    any security considerations that are not present in the base [SIEVE]
    protocol, and these issues are discussed in [SIEVE].

5 IANA Considerations
   
    The following templates specify the IANA registration of the Sieve
    extensions specified in this document:

5.1 spamtest registration
   
    To: [email-masked]
    Subject: Registration of new Sieve extension

    Capability name: refuse
    Capability keyword: refuse
    Capability arguments: N/A
    Standards Track/IESG-approved experimental RFC number: this RFC
    Person and email address to contact for further information:

    Matthew Elvey
    3042 Sacramento-ietf St Ste 04
    San Francisco, CA
    U.S.A.

    <mailto:[email-masked]>

    This information should be added to the list of sieve extensions
    given on http://www.iana.org/assignments/sieve-extensions.

6 References
   
6.1 Normative References
   
    [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate
    Requirement Levels", RFC 2119, March 1997.
   
    [SIEVE] Showalter, "Sieve:  A Mail Filtering Language", RFC 3028,
    January 2001.
   
7 Acknowledgments
   
    Thanks to ... for comments and corrections.

8 Author's Addresses
   
    Matthew Elvey
    3042 Sacramento-ietf St Ste 04
    San Francisco, CA
    U.S.A.

    <mailto:[email-masked]>

9 Intellectual Property Rights Statement
    <boilerplate>

10 Full Copyright Statement
    <boilerplate>

---

From: "Nigel Swinson" <[email-masked]>
Date: Fri, 31 Oct 2003 17:30:37 -0000

> told me that Cyrus' implementation can't do this.

Somewhat of a late post I know, but Rockliffe's Mailsite has a Server Sieve
Script where if you use "Reject" it will reject at the SMTP protocol level.
I doubt this is much use to you as presumably you are looking for something
that is available at the end user level (as I can see in your I-Draft).

The problem with filtering before storing in the end users mailbox is as Ned
says, the concept of the mail envelope.  At the SMTP level, the envelope
probably has many recipients, so who's configuration do you use?  It's only
when you are about to put a message in a users INBOX will you know that you
can take action on behalf of all the recipients (cos there is only one).
Hence when MailSite rejects a message at the SMTP level it is using a global
server level script.

A comment on your I-Draft.  I've never heard of "joe-job spam".  Now I know
I only code mail servers, not use them, so perhaps I'm just ignorant, but
I'm wondering if "joe-job spam" is a well established enough term to be
using without describing what it means?  I see you do describe it
eventually, but wonder if you can avoid the term completly in the I-Draft.

Nigel

---

From: "Mark E. Mallett" <[email-masked]>
Date: Fri, 31 Oct 2003 13:06:46 -0500

> Hence when MailSite rejects a message at the SMTP level it is using a global

Yeah- what one would need to do is do an attempted delivery for
each SMTP "mail to" recipient and deal with rejections based on
that attempt.  You would have to have access to each individual's
filter program to do this.  Furthermore you would need something
other than SMTP, since once you have said "OK" to the "mail to"
you can't take it back later.   Or you would have to have a way
of enforcing only one recipient per SMTP session.  (accepting
the first "mail to" and giving 4xx responses for each subsequent
one would be very hackish.)  Here it would be great to have an SMTP
enhancement allowing post-DATA rejections.

> eventually, but wonder if you can avoid the term completly in the I-Draft.

JoeJob is a very common term referring to forging somebody else's
address as a return address, so that unfortunate victim gets bounces.
It's a googlable term.

mm

---

From: Guðbjörn Hreinsson <[email-masked]>
Date: Fri, 31 Oct 2003 18:25:36 -0000

> Hence when MailSite rejects a message at the SMTP level it is using a
global

You can reject per Recipient and accept others. Nothing wrong with it.
You can do this per domain or recipient, the recipient could modify this
configuration option and the data could be in a SQL or LDAP server.

---

From: "Matthew Elvey (FM)" <[email-masked]>
Date: Fri, 31 Oct 2003 19:13:33 -0800

Thanks, Mark, Nigel for comments, and Kjetil for a template (though I'll 
probably use RFC3285's, which I just became aware of).   Alexey and I 
plan to co-author the I-D.

On 10/31/2003 10:06 AM, Mark E. Mallett sent forth electrons to convey:

>>probably has many recipients, 
>>
s/probably/very occasionally/, so it's not as much of an issue.  I think 
a 'refuse' that works perfectly as long as there aren't multiple 
*envelope* recipients (which, keep in mind, is much rarer than multiple 
recipients of any kind) ( and handles the special case by causing an MDN 
to be sent on behalf of the rejecting recipients) is much better than no 
'refuse' at all.

A LONG and often technical discussion of issues you needn't read if 
you're not comfortable with the above:
  In my proto-draft (security section), I suggest this be handled as 
follows:
"Sieve evaluation as typically implemented requires having the entire 
message on hand.
In SMTP this means the only response code that's left is the final one, 
where the server accepts or rejects the message on behalf of all 
recipients. This becomes a problem when there are multiple recipients 
and some reject the message and some do not.  In this case, reject MUST 
cause an MDN to be sent on behalf of the rejecting recipients. (Idea: 
allow MTA to tempfail the rejecting recipients and accept the others, 
until all recipients are rejecting the email; then it can be refused.)"
[I now reject that latter idea:] Is the additional complexity introduced 
by specifying how the tempfail would work to make SMTP rejects work even 
in the relatively rare case of multiple recipients on the envelope 
really worth it?  Making this mandatory would make the implementation of 
this spec considerably more work, I think.   I don't want to make it 
optional, I'd rather leave it out than make it optional.  (By the way, 
tempfail means to send a 4xx SMTP response code)  It would work as 
follows:  when there are multiple recipients and some reject the message 
and some do not, a tempfail response code is sent.  The recipient server 
has to maintain state about the message so it can recognize it when (if) 
delivery is reattempted. (some spamware never reattempts delivery!)  
When delivery is reattempted, the rejecting recipients are tempfailed, 
but the others aren't.  (This is perfectly acceptable, per RFC2821.) 
When the third delivery attempt is made, the previously rejecting 
recipients are not tempfailed and (assuming) all the recipients in this 
delivery are confirmed to be rejecting recipients, the whole email can 
be permanently refused.  (Unfortunately, some spamware reattempts 
delivery even after permanent failures!)  I'd be surprised if a spec 
requiring this would be broadly implemented, even though I think it 
would reliably provide the desired result.  I don't want to write a 
Standards Track document that never gets past the I-D stage!

>>so who's configuration do you use?  It's only
(And technically violating the RFC - but I'm NOT complaining about that! 
- it's effectively crying out for this extension.)

>you can't take it back later. 
>
No.  Actually RFC2821 specifies when the server takes responsibility:
"In sending a positive completion reply to the end of data indication, 
the receiver takes

   full responsibility for the message (see section 6.1).'
Until then, the server can drop the message and connection and be RFC compliant, even if it has said OK to all the recipients.
Also there is no such thing as a "mail to" command; I assume you mean 'rcpt to'.

>  Or you would have to have a way
>one would be very hackish.)  
>
I would be open to something like: an implementation MAY tempfail 
recipients after the first one to facilitate per-user 'refuse' actions.  
An implementation could, for example choose to do this for mail from any 
blacklisted IPs, and safely do so even using aggressive blacklists that 
list IPs that send both ham and spam.  It's not clear whether this would 
violate any RFCs.  RFC 2821 does not allow "Rejection of messages (for 
excessive recipients)" < 100 but a 450 rejection of that sender because 
the mailbox is unavailable for policy reasons seems acceptable to my 
reading.

>Here it would be great to have an SMTP
SMTP already allows this.  It's clearly permitted by RFC 2821 to send a 
negative completion reply to the end of data indication.  Were it not, 
this I-D would be a non-starter!

>address as a return address, so that unfortunate victim gets bounces.
I'll be sure that it's clear if we use it.
http://members.cox.net/joejob/