Subject: Rejecting list non-subscribers

From: "Mike McBride (mmcbride)" <[email-masked]>
Date: Thu, 20 Mar 2008 09:25:19 -0700

Is there an RFC section stating that I can't auto reject emails from
non-subscribers? Trying to simplify my list maintenance life.

thanks,
mike

---

From: "Phelan, Tom" <[email-masked]>
Date: Thu, 20 Mar 2008 12:27:10 -0400

Hi Mike,

The DCCP list has been auto-rejecting posts from non-subscribers for
quite some time now.  No one's complained...

Tom P.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Mike McBride (mmcbride)
> Sent: Thursday, March 20, 2008 12:25 PM
> To: [email-masked]
> Subject: Rejecting list non-subscribers
>

---

From: Lars Eggert <[email-masked]>
Date: Thu, 20 Mar 2008 18:41:05 +0200

> quite some time now.  No one's complained...

http://www.ietf.org/IESG/STATEMENTS/mail-submit-policy.txt says: "It  
is not accepable to simply drop messages from non-subscribers."

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 20 Mar 2008 11:43:04 -0500

I'm almost sure that there is some failure condition that this will trip 
over (non-area AD sending comments to a WG mailing list), given current 
expectations.

Having said that - I'd be really excited about discussing this, and figuring 
out what failure conditions we would have to fix, and fixing them.

For instance, perhaps auto-white listing all ADs would fix that specific 
problem.

I'd be a lot more excited about fixing the problems and then requiring 
posters to subscribe, than continuing to delete spam every day or so ("my 
life is too short").

And I'm sure that older group mailing lists get a lot more spam than mine 
does.

Thanks,

Spencer

----- Original Message ----- 
From: "Phelan, Tom" <[email-masked]>
To: "Mike McBride (mmcbride)" <[email-masked]>; <[email-masked]>
Sent: Thursday, March 20, 2008 11:27 AM
Subject: RE: Rejecting list non-subscribers

Hi Mike,

Tom P.

---

From: "Phelan, Tom" <[email-masked]>
Date: Thu, 20 Mar 2008 12:44:28 -0400

Hi Lars,

We don't "simply drop".  We respond with something that tells the sender
to contact the chairs (or sign up for the list) to get the message
posted.

Tom P.

> From: Lars Eggert [mailto:[email-masked]] On Behalf Of
Lars
> Eggert
> Sent: Thursday, March 20, 2008 12:41 PM
> To: Phelan, Tom
> Cc: Mike McBride (mmcbride); [email-masked]
> Subject: Re: Rejecting list non-subscribers
>

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 20 Mar 2008 11:46:52 -0500

> http://www.ietf.org/IESG/STATEMENTS/mail-submit-policy.txt says: "It  is 
> not accepable to simply drop messages from non-subscribers."

Like I said - the expectations are that we don't do this.

... which is why we would have to figure out the reasons why it's a problem 
to drop messages from non-subscribers, as part of discussions about changing 
expectations.

Lars - does auto-rejecting, saying "you must be a subscriber to post, and 
here's how to subscribe", qualify as "simply drop messages"?

I do understand why accepting a post and then dropping it with no indication 
to the poster would not be helpful.

Spencer

---

From: Marshall Eubanks <[email-masked]>
Date: Thu, 20 Mar 2008 12:50:47 -0400

MBONED moderates non-subscriber emails. I add senders of real  
messages to the "accept" list,
so they only have to be dealt with once. (These are generally cross- 
posts or reviews.)

Are you getting a lot of spam ?

Regards
Marshall

---

From: Melinda Shore <[email-masked]>
Date: Thu, 20 Mar 2008 13:02:24 -0400

> to drop messages from non-subscribers, as part of discussions about changing

My recollection of the argument at the time the policy was implemented
(was it really that long ago) is that there are people who have some
formal organizational role (the Ads, Ads from other areas, the internet
drafts administrators, etc.) who will occasionally need to post to a
working group mailing list and that they shouldn't have to subscribe
in order to do so.

I hold all mail from non-subscribers for review, and when someone who
has that role ends up in the held-mail queue I add them to the list of
people who can post without joining.  As I recall, somebody-or-other
posted a starter list of those folks when the policy was first instituted.

Melinda

---

From: Acee Lindem <[email-masked]>
Date: Thu, 20 Mar 2008 13:03:42 -0400

Hi Tom, Mike,
We do the same for the OSPF list and we have gotten complaints.  
However, I do see the alternative of having the chairs handle the  
SPAM as viable.
Thanks,
Acee

---

From: "Phelan, Tom" <[email-masked]>
Date: Thu, 20 Mar 2008 13:07:47 -0400

Hi Spencer,

> For instance, perhaps auto-white listing all ADs would fix that
specific

We did auto-whitelist a bunch of people.

Tom P.

> Behalf Of Spencer Dawkins
> Sent: Thursday, March 20, 2008 12:43 PM
> To: [email-masked]

---

From: James Carlson <[email-masked]>
Date: Thu, 20 Mar 2008 13:19:04 -0400

> posted a starter list of those folks when the policy was first instituted.

I do the same with held messages, but I go a little further: as long
as the posting is obviously not spam, I add the person to the list of
those who can post, and let the sender know what I've done.  I don't
care about "role," and I have no need to determine it; I care only
about discarding obvious trash.

-- 
James Carlson, Solaris Networking              <[email-masked]>
Sun Microsystems / 35 Network Drive        71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 20 Mar 2008 12:40:39 -0500

Hi, Tom,

Understood. I'm kinda asking what can be done for all (ietf.org-hosted, but 
many of us are) working group mailing lists, instead of letting working 
group chairs figure out one working group at a time that Pasi is now an AD, 
and posts from [email-masked].

:-)

I'm hoping that if we have some fairly simple (and hopefully automated) 
procedures in place that solve 90 percent of our "real postings from 
non-subscribers" issues, we can reasonably ask the IESG to change the policy 
that Lars pointed to.

Spencer

Tom P.

---

From: "Wojciech Dec (wdec)" <[email-masked]>
Date: Thu, 20 Mar 2008 18:39:30 +0100

Same thing's done in ANCP. Level of spam appears to be correlated to
activity on the WG alias (still able to distinguish between the two
thankfully ;-) ).

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Marshall Eubanks
> Sent: 20 March 2008 17:51
> To: Mike McBride (mmcbride)
> Cc: [email-masked]
> messages to the "accept" list, so they only have to be dealt 
> with once. (These are generally cross- posts or reviews.)
>

---

From: "Phelan, Tom" <[email-masked]>
Date: Thu, 20 Mar 2008 13:58:18 -0400

Hi Spencer,

A centrally-maintained white list would be very good.

When DCCP did moderator-action for postings from non-subscribers, we
found that we were trashing probably significantly over 1,000 spam
messages for every legitimate non-subscriber post.  This seemed like a
significant waste of our time.  So we set up a reject with message
policy, and added a number of people to a whitelist who weren't already
on the list.  Note that our reject-response message says you can send
the message to the chairs and we'll post it.

Since we started rejecting non-subscriber posts with instructions
(probably well over a year ago), we've only had one contact from someone
who was rejected, and he opted to join the list.  I don't know how many
people have been rejected and then opted to join the list, or felt it
was too much effort and abandoned the post.

As far as I'm concerned, doing this has turned out to be a big success.

Tom P.

> Sent: Thursday, March 20, 2008 1:41 PM
> To: [email-masked]
> Understood. I'm kinda asking what can be done for all
(ietf.org-hosted,
> but

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 20 Mar 2008 19:03:53 +0100

> to the poster would not be helpful.

Umm ... why does this question pop up now, I wonder?  Having seen the spam
volume increase noticeably over the last 3 weeks, I suspect it's because
we're still without the excellent protection against spam which TMDA gave
us earlier.

I'd still like to get TMDA in place again -- the distributed cost (in time
wasted) to list moderators of not having it is aggravating.

	Henrik

---

From: "Brian Rosen" <[email-masked]>
Date: Thu, 20 Mar 2008 14:10:36 -0400

Just BTW on this subject, I notice that most of the spam is getting bounced.
I get a bounce message, and don't have to respond.  I do still see some spam
hitting my list, which I have to handle in the usual way.  

Brian

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Henrik Levkowetz
Sent: Thursday, March 20, 2008 2:04 PM
To: [email-masked]
Subject: Re: Rejecting list non-subscribers

	Henrik

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 20 Mar 2008 12:07:04 -0700

> Are you getting a lot of spam ?

Yes, [email-masked] received 61 spam messages on Wednesday.  Today's incoming
spam to that list looks to be on par with yesterday's.

> A centrally-maintained white list would be very good.

We had one -- TMDA.  It worked well.  It has been disabled.

Henrik Levkowetz wrote:

> I'd still like to get TMDA in place again -- the distributed 
> cost (in time wasted) to list moderators of not having it 
> is aggravating.

I concur.  Spending computer time (a second or two?) to whitelist 
posts to lists is far more desirable than having every working
group chair spend 10 seconds each day (or even each week) looking
through posts of suspected spam and clicking Deny or Accept or
whatever.

-d

---

From: James M Galvin <[email-masked]>
Date: Thu, 20 Mar 2008 16:14:04 -0400

Tom,

You bring up an interesting distinction, i.e., you regard "drop" as 
distinctly different than "reject".

While I understand the distinction I do not believe that was 
intended by the policy, nor is it how it has been interpreted by me 
or anyone I spoke with during the transition.  Of course, to be 
fair, I never posed the distinction in any discussion, so perhaps 
my interpretation is biased.

Jim

-- On Thursday, March 20, 2008 12:44 PM -0400 "Phelan, Tom" 
<[email-masked]> wrote regarding RE: Rejecting list 
non-subscribers --

> Hi Lars,
>
> We don't "simply drop".  We respond with something that tells the
> sender to contact the chairs (or sign up for the list) to get the
> message posted.
>
> Tom P.
>
> > > The DCCP list has been auto-rejecting posts from
> > > non-subscribers for quite some time now.  No one's
> > > complained...
> >
> > http://www.ietf.org/IESG/STATEMENTS/mail-submit-policy.txt
> > says: "It is not accepable to simply drop messages from
> > non-subscribers."
> >
> > >> [mailto:[email-masked]]
> On
> > >> Is there an RFC section stating that I can't auto reject
> > >> emails
> from

---

From: James M Galvin <[email-masked]>
Date: Thu, 20 Mar 2008 16:19:54 -0400

-- On Thursday, March 20, 2008 12:07 PM -0700 Dan Wing 
<[email-masked]> wrote regarding RE: Rejecting list 
non-subscribers --

> Yes, [email-masked] received 61 spam messages on Wednesday.
> Today's incoming spam to that list looks to be on par with
> yesterday's.

Dan,

I just checked your list configuration and it had an error, which I 
fixed.

For everyone, the correct configuration should have the following 
SPAM filter rule:

    X\-Spam\-Level: \*{5,}

With action set to "discard".  The SPAM filter rules are found 
under  "Privacy Options".

Jim

---

From: James M Galvin <[email-masked]>
Date: Thu, 20 Mar 2008 16:26:37 -0400

Unfortunately, Mailman does not lend itself to system wide 
configuration options.  The only way to whitelist senders is on a 
per list basis and, as such, each moderator needs to do this.

As others have described but I'll repeat here more fully, the 
operational model is:

1. Get a summary listing each day of messages in the "moderator" 
queue.  There should be sufficient information here that a quick 
scan will tell you if there is a message that needs attention.

2. If a message needs attention you need to login in to the 
moderator page to take care of it.  If you want to add the person 
to the "auto accept" list you can do that, or not as you prefer.

3. If you want to clear the queue you just need to select "discard 
all messages marked defer" and click "submit".  If you've been 
watching the summary listings each day these two clicks will take 
care of it.

It should be the case that no mailing list gets more than a few 
SPAM messages a day, where a few is less than 10.  If anybody is 
getting more than a few SPAM messages a day then you should report 
that to "[email-masked]".

Jim

-- On Thursday, March 20, 2008 12:40 PM -0500 Spencer Dawkins 
<[email-masked]> wrote regarding Re: Rejecting list 
non-subscribers --

> (ietf.org-hosted, but many of us are) working group mailing
> lists, instead of letting working group chairs figure out one
> working group at a time that Pasi is now an AD, and posts from
> [email-masked].
>
> :-)
>
> I'm hoping that if we have some fairly simple (and hopefully
> automated) procedures in place that solve 90 percent of our "real
> postings from non-subscribers" issues, we can reasonably ask the
> IESG to change the policy that Lars pointed to.
>
> Spencer
>

---

From: James M Galvin <[email-masked]>
Date: Thu, 20 Mar 2008 16:31:55 -0400

I should add that the reference Lars included also says this:

   e.  Require that messages from legitimate senders that are not
       members of the list to be accepted.

That is a stated goal of the discussion of SPAM filtering.  This 
suggests to me that "rejection" is inappropriate.

Jim

-- On Thursday, March 20, 2008 4:14 PM -0400 James M Galvin 
<[email-masked]> wrote regarding RE: Rejecting list 
non-subscribers --

> Tom,
>
> You bring up an interesting distinction, i.e., you regard "drop"
> as distinctly different than "reject".
>
> intended by the policy, nor is it how it has been interpreted by
> me or anyone I spoke with during the transition.  Of course, to
> be fair, I never posed the distinction in any discussion, so
> perhaps my interpretation is biased.
>
> Jim
>
>
>
> > We don't "simply drop".  We respond with something that tells
> > the sender to contact the chairs (or sign up for the list) to
> > get the message posted.
> >
> > Tom P.
> >

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 20 Mar 2008 13:48:49 -0700

> With action set to "discard".  The SPAM filter rules are found 

My previous configuration caused such messages to be Held.  This
configuration was intentional, so that I have a log of messages that 
were classified as spam by SpamAssassin.  That way, in the event a 
poster tells me their mail was not distributed to the list, I 
can figure out what occurred by examining the logs of my automated
script which cleans out the Held messages.  I have a script that 
runs each hour that cleans up the queued messages for me -- I did 
not clean out all of those 61 spam messages manually. 

With the suggested configuration, when post is falsely marked
as 'spam' by SpamAssassin and is quietly discarded I have no way 
to figure out what occurred when a poster complains their message
did not appear on the list.  I could open a case, I suppose, but
that is a waste of everyone's time:  it is much easier for me
to grep the log of my script that cleans out the Held messages.

I have reverted BEHAVE to its previous configuration.

-d

---

From: James M Galvin <[email-masked]>
Date: Thu, 20 Mar 2008 16:56:19 -0400

Fair enough.

However, if someone's email is getting dropped, you can have them 
send a message to "[email-masked]" to get to the bottom of 
it.  We have a mechanism in place to capture messages so we can see 
what SpamAssassin does to them and we will work with folks to get 
to the bottom of any issue they're having.

Or you can handle it.  Either way is fine.

Setting that aside, sounds like you have a great script!  Could it 
be used by others to help them with their lists?  In fact, if it is 
fully automated, perhaps it is something that can be setup on the 
IETF server to run for everyone?

Jim

-- On Thursday, March 20, 2008 1:48 PM -0700 Dan Wing 
<[email-masked]> wrote regarding RE: Rejecting list 
non-subscribers --

> > I just checked your list configuration and it had an error,
> > which I  fixed.
> >
> > For everyone, the correct configuration should have the
> > following  SPAM filter rule:
> >
> >     X\-Spam\-Level: \*{5,}
> >
> configuration was intentional, so that I have a log of messages
> that  were classified as spam by SpamAssassin.  That way, in the
> event a  poster tells me their mail was not distributed to the
> list, I  can figure out what occurred by examining the logs of my
> automated script which cleans out the Held messages.  I have a
> script that  runs each hour that cleans up the queued messages
> for me -- I did  not clean out all of those 61 spam messages
> manually.
>

---

From: "Dan Wing" <[email-masked]>
Date: Thu, 20 Mar 2008 14:07:14 -0700

> to the bottom of any issue they're having.

Ok, that's good to know.

> IETF server to run for everyone?

This is the script (I am not the author of it, I just use it):

  http://www.waider.ie/hacks/workshop/perl/mailman.pl

it runs SpamAssassin locally to determine if a message should
be discarded.  I do a tiny bit of keyword matching (with a custom
user_prefs file) to try to avoid false positives for lists that
I own.  I use this same script with several different mailman
lists that I own on various servers -- not all of the servers
have the facility to have a postmaster help me to troubleshoot 
mis-scored posts.  So I prefer the logging capability.

One drawback is that "ps" can see your command line options 
so using "--password" isn't a good idea on a multi-user system.

The log is like this:

Starting mailman.pl at: Wed Mar 19 00:05:02 PDT 2008
Spam: 17.05/5
(BAYES_99,INVALID_MSGID,LOCAL_B_ALWAYSTRUE,MISSING_MIME_HB_SEP,RAZOR2_CF_RANGE
_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RDNS_NONE,URIBL_BLACK,URIBL_JP_
SURBL,URIBL_OB_SURBL,URIBL_SC_SURBL,URIBL_WS_SURBL)
  Blipping From [email-masked]
  Subj:    ***SPAM*** 83.264 (5) Potenzschwache - wir haben die Losung

Spam: 24.592/5
(BAYES_99,FORGED_MUA_OUTLOOK,INVALID_MSGID,LOCAL_B_ALWAYSTRUE,MISSING_MIME_HB_
SEP,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_BL_S
PAMCOP_NET,RDNS_NONE,URIBL_BLACK,URIBL_JP_SURBL,URIBL_SBL)
  Blipping From [email-masked]
  Subj:    ***SPAM*** 61.899 (5) Mit unserem Produkt haben Sie wieder Spass am
Leben!

-d

---

From: Derek Atkins <[email-masked]>
Date: Fri, 21 Mar 2008 09:50:49 -0400

Hi,

"Spencer Dawkins" <[email-masked]> writes:

> ... which is why we would have to figure out the reasons why it's a
> problem to drop messages from non-subscribers, as part of discussions
> about changing expectations.

As a sec-dir reviewer I frequently get pulled into on-list discussions
of drafts that I've reviewed.  When that happens it would be BAD if my
messages get dropped.  I think it's unreasonable to be required to
subscribe to a WG list just to talk about a review of one draft.

Similarly, when there are cross-group discussions about an issue that
crosses over to multiple lists it would be bad to drop a message
because someone is subscribed to one WG list and not the other.

So yes, there are real reasons not to just drop messages from
non-subscribers.

-derek
-- 
       Derek Atkins                 617-623-3745
       [email-masked]             www.ihtfp.com
       Computer and Internet Security Consultant