Subject: Proposed new spam policy

From: Bill Fenner <[email-masked]>
Date: Wed, 12 Oct 2005 13:01:17 -0700

Following up on a long-lost TODO item: I took Steve Bellovin's  
proposed updated spam policy that he circulated in February 2004 on  
this list and added a few updates based on the discussion that  
followed.  I'd like to see what people think of this version; if  
there are no cries of "It'll never work" I'll bring it to the IESG  
telechat to be approved to replace http://www.ietf.org/IESG/ 
STATEMENTS/mail-submit-policy.txt .

Thanks,
   Bill

             Guidance for spam control on IETF mailing lists

1.

    This note provides guidance on balancing the need to limit spam with
    preserving openness of IETF mailing lists.  To preserve the utility
    of mailing lists, some form of spam control MUST be used.
    Uncontrolled lists are a denial of service attack on list members;
    besides, complaints about the spam that arrives has been known to
    shut down such lists.

    We focus on the most common method of spam control: mailing lists
    which use heuristic methods for automated message rejection to  
reduce
    the amount of spam delivered to list subscribers and to protect the
    subscribers from specific types of dangerous content.

    This note has these goals:

    a.  Protect subscribers from disrupted communication due to the
        burden of spam.

    b.  Protect subscribers from dangerous content.

    c.  Minimize the rejection of legitimate messages and allow the list
        to function without undue burden on the administrator.

    d.  Protect the senders by giving them the information and methods
        they need for contacting the list administrator to request an
        override of an automated rejection.

    e.  Require that messages from legitimate senders that are not
        members of the list to be accepted.

    The two extremes of mailing list operation run from a completely  
open
    list in which no submissions are rejected to moderated lists where a
    human reviewer must explicitly approve each submission.  Neither
    approach is desirable in most cases.  The more typical approach  
is to
    allow some types of messages to be distributed with no human check,
    while remaining messages are either rejected outright, or sent to a
    human reviewer for rejection or approval.

    Manual filtering is acceptable, if the approver is able to  
respond to
    postings promptly.  However, because of the massive increase in the
    spam rate, many sites have been forced to resort to automated
    filtering of various types.  Under certain conditions, such  
automated
    filters MAY be used for IETF mailing lists.  Note, though, the
    difference between spam control and moderation.  If there is any
    reasonable doubt about whether or not a message is spam, it almost
    certainly is not.  Moderation is governed by a separate policy,
    http://www.ietf.org/IESG/STATEMENTS/moderated-lists.txt.

    If an automated spam detector is used, it should take several  
factors
    into account.  These include, in decreasing order of importance:

    a.  Whether or not the purported sender is a subscriber to the
        mailing list.  This alone SHOULD NOT be relied upon as evidence
        that a message isn't spam, given the prevalence of "joe jobs"
        (spam that impersonates real users) and spam-sending worms.   
Note
        that the following addresses MUST always be considered to be
        subscribers for this purpose: [email-masked],
        [email-masked], [email-masked],
        [email-masked], [email-masked].  Note also that it is not
        accepable to simply drop messages from non-subscribers.

    b.  The likelihood that the message is spam, based on textual
        evidence, statistical filters, and the like.  Note the danger
        here of false positives;

    c.  Past behavior by that particular sending IP address;

    d.  Appearance of the sending IP address in assorted blacklists.
        This last should be used with great caution, given the high rate
        of false positives.

    A good spam detection mechanism will weight the various factors and
    react accordingly.  A message from a known subscriber with the word
    "mortgage" is probably not spam; a message from a non-subscriber  
on a
    generic mail-hosting service that starts "I am Mariam Abacha" and
    talks about hidden bank accounts almost certainly is; email from a
    non-subscriber with the word "mortgage" may or may not be, and  
should
    be examined more closely.

    Mail containing executable content (Unix shell scripts, Windows .exe
    files, and the like) or virus-infected attachments SHOULD be
    discarded immediately, regardless of other positive or negative
    indications.  Mail containing other types of attachments -- HTML,
    Powerpoint, etc. -- is often inappropriate; posters are cautioned to
    use their own judgment for each such email.  There is a tradeoff  
here
    -- large, bulky files are often best handled by sending their URLs
    instead; on the other hand, such practices mean that there will  
be no
    easily-accessible archival record of their contents.  (Configuration
    of mailers to send HTML alone SHOULD NOT be done.)

    In other cases, it MUST be possible for the sender of a legitimate
    message, whether a mailing list subscriber or not, to determine  
if it
    is has been dropped as apparent spam.  This can be done in several
    ways; all of these have their advantages and disadvantages.

    a.  Provide a Web-accessible archive of postings rejected in the  
past
        few days.  Few, if any, current mailing list packages currently
        do this, though some spam filters can easily be configured to do
        so.  One tool written with this problem in mind is available at
        <http://www.hactrn.net/hacks/spamtools/spamindex.pl>.  This
        alternative is RECOMMENDED.

    b.  Provide an up-to-date archive of accepted postings.
        Unfortunately, while this can show dropped messages, it doesn't
        help if the email is merely delayed, nor does it say why a
        message was dropped.  This MAY be used.

    c.  Have the receiving MTA scan the message and send a 500-class
        rejection to the DATA command.  This can lead to mailer timeouts
        and retransmissions.  This MAY be used, but (b) is a better
        alternative.

    d.  Email back a rejection message.  In the event of forged spam --
        and there's a lot of it -- this can lead to denial of service
        attacks on the purported sender.  This SHOULD NOT be done.

    Implicit in options (a) and (b) is that posting to an IETF mailing
    list is not guaranteed to be a reliable process -- it is up to the
    sender to verify that a message actually appears on the mailing  
list.
    This is not to mean that mailing lists are inherently unreliable,
    just that the burden for verification that postings are being
    delivered falls on the sender, not on the mailing list maintainer.
    Folks who post to lists they are not subscribed to need to pay
    particular attention.

    If any spam filtering is done, there MUST be a bypass path by which
    someone can complain that a message was inappropriately blocked.
    Information about the bypass path SHOULD be available in the same
    place as other mailing list information - i.e., subscription  
address,
    mailing list owner, archive location.  Such a complaint SHOULD be
    responded to within 2 business days.  The usual appeal chain -- WG
    chairs, ADs, etc., per RFC 2026 -- applies.

Author's Address

    Bill Fenner (for the IESG)
    AT&T Labs - Research
    75 Willow Rd.
    Menlo Park, CA  94025
    USA

    Email: [email-masked]

---

From: "Adrian Farrel" <[email-masked]>
Date: Wed, 12 Oct 2005 21:37:08 +0100

Hi,

I know I will get shouted at, but...

I want to object very strongly to the suggestion of using blacklists that
are out of our immediate control. My ISPs relays just found their way onto
a black list that was used by the domain that hosts the mailing list for
one of my working groups. Net result: unable to post to my working group's
mailing list, unable to manage said mailing list, unable to contact domain
manager to get a white list entry.

Clearly solutions to my problem abound, but I don't want to see us go down
a path that introduces this problem to IETF administered mailing lists.
Further, if we do run our own blacklists, they need to be based on direct
experience at the ietf domain and not reports of other people's
experiences somewhere in the Internet.

Currently still fuming.
Cheers,
Adrian

---

From: Bill Fenner <[email-masked]>
Date: Wed, 12 Oct 2005 14:01:55 -0700

I'm generally against blacklists too.  I'm inclined to remove bullet
"d" even though it's at the end of the prioritized list and has a
specific caution.  What do others think?

  Bill

---

From: Jari Arkko <[email-masked]>
Date: Thu, 13 Oct 2005 00:04:14 +0300

Actually, I like the new spam rules. In an ideal world
we can base our decisions on direct information. However,
this is work. Having the list maintainers or chairs manage
spam is also work.  I think its better to leave reliable delivery
to end-to-end mechanisms, i.e., the sender, where it most
of the time happens through natural means anyway (checking
that your comments were addressed in a draft, seeing your
post appear in your inbox, etc). I acknowledge the possibility
that your ISP may get on a black list -- I've been there...
Anyway, getting alternative or temporary e-mail addresses
for such emergencies does not seem hard.

--Jari

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 12 Oct 2005 23:22:55 +0200

> specific caution.  What do others think?

I agree.

>         and retransmissions.  This MAY be used, but (b) is a better

I've personally had good success with this; it has resulted in my incoming
spam being on a clearly decreasing trend over the last year, as it seems that
spammers don't really care for sending mail to addresses which consistently
reject them.  So this method has an important side effect, in addition to
the immediate effect of notifying the sender of a legitimate but erroneously
spam-classified message that the message was rejected. 

Regards,

	Henrik

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 12 Oct 2005 23:23:30 +0200

> specific caution.  What do others think?

I think that specifying "no external blacklists" will greatly reduce the 
number of sites that are willing to host IETF mailing lists.

While ietf.org is always an alternative, I don't like that.

(I've had my personal mailserver appear on blacklists. Not fun....)

                   Harald

---

From: Bill Fenner <[email-masked]>
Date: Wed, 12 Oct 2005 21:08:26 -0700

>number of sites that are willing to host IETF mailing lists.

I don't think that's what removing "d." would accomplish.  The list
is introduced with the text:

>>If an automated spam detector is used, it should take several factors

Keeping blacklists on this list mean that the IETF recommends using
blacklists to filter mailing lists.  Removing blacklists from this
list doesn't mean that you can't use blacklists, it just means it's
not specifically recommended.  ("should", and lack of "and must not use
mechanisms not on this list").

  Bill

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 13 Oct 2005 08:02:45 +0200

> mechanisms not on this list").

OK - read like that, the text (without d.) works for me.

Blacklists have certainly bitten me more often than many other things.

---

From: Dean Willis <[email-masked]>
Date: Sat, 15 Oct 2005 14:51:37 -0500

> specific caution.  What do others think?

While philosophically opposed to blacklists and familiar with the pain 
of recovering from a false-positive listing, I've found that using 
spamhaus blocks more spam than all my other tools combined, currently 
several thousand pieces a day on the server that hosts my email. It also 
does so with significantly less processor impact than my next-line 
defense, spam-assassin.

--
Dean

---

From: Sam Hartman <[email-masked]>
Date: Mon, 17 Oct 2005 21:34:42 -0400

>>>>> "Harald" == Harald Tveit Alvestrand <[email-masked]> writes:

    >>  I'm generally against blacklists too.  I'm inclined to remove
    >> bullet "d" even though it's at the end of the prioritized list
    >> and has a specific caution.  What do others think?

    Harald> I think that specifying "no external blacklists" will
    Harald> greatly reduce the number of sites that are willing to
    Harald> host IETF mailing lists.

Note that a site that rejects messages based on blacklists without
considering other factors fails to comply with current policy, fails
to comply with the policy Bill originally proposed and fails to comply
with the policy with Bill's proposed dropping of item D.