Subject: Removing spam from archives

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 6 Jan 2005 12:03:47 -0800

Hi -

I've noticed that spam has gotten into some WG archives.
In some cases, it appears to have been inserted directly
into the archive without ever having gone out on the WG
mailing list.  The MHonArc program used to maintain the
HTML archives provides a facility for removing such crud.
Rather than going through [email-masked] for every
single one of these, would it make sense to extend the
mailman interface (for IETF-maintained lists, anyway)) to
allow mailing list maintainers to delete the garbage?

Randy

---

From: Ralph Droms <[email-masked]>
Date: Thu, 06 Jan 2005 15:20:45 -0500

Hear, hear!!!

I've been complaining about spam going directly into the archives 
(presumably, e-mailed directly to the inbound archive e-mail address) for 
at least a year to no avail.

Over and above giving us a tool to eliminate the crud once it's crept into 
the archive, can we *please* get the system fixed so the crud doesn't get 
in at all?

- Ralph

---

From: Ted Hardie <[email-masked]>
Date: Thu, 6 Jan 2005 15:07:10 -0800

At 3:20 PM -0500 1/6/05, Ralph Droms wrote:
>(presumably, e-mailed directly to the inbound archive e-mail 
>address) for at least a year to no avail.
>
>Over and above giving us a tool to eliminate the crud once it's 
>crept into the archive, can we *please* get the system fixed so the 
>crud doesn't get in at all?
>
>- Ralph

I second the motion to get it fixed so direct mail to the inbound 
e-mail archive
is disabled.

Unfortunately, there is a problem with removing things from the archives once
there; since the mailing list discussions are our official way of doing things,
the ability to remove a mailing list message from the archives implies the
ability to change our official record.  While I understand that these messages
never were part of our discussions (especially since the spam didn't 
even hit the
mailing list), giving someone the ability to edit them out might easily
be taken to include the ability to edit out other sorts of messages.  That
slope is very, very slippery indeed.

Can we first fix the incoming spam problem, and then tackle the cleanup
issue?  It's entirely possible that we can do it as one time cleanup pretty
easily (by recording what was removed, and noting the removal on each
affected WG list).  But on ongoing basis, this would worry me.

			regards,
				Ted Hardie

---

From: Bob Hinden <[email-masked]>
Date: Thu, 06 Jan 2005 15:30:52 -0800

Ted,

>I second the motion to get it fixed so direct mail to the inbound e-mail 
>archive
>is disabled.

Me too!

>never were part of our discussions (especially since the spam didn't even 
>hit the

Good point.  Unless the problem has gotten so bad that there is more SPAM 
than real content, I think it would better to leave it alone.  Editing the 
archive might create more long term problems than leaving the SPAM intact.

Bob

---

From: Pete Resnick <[email-masked]>
Date: Thu, 6 Jan 2005 17:39:06 -0600

On 1/6/05 at 3:30 PM -0800, Bob Hinden wrote:

>>never were part of our discussions (especially since the spam 
>>didn't even hit the
>Good point.  Unless the problem has gotten so bad that there is more 
>SPAM than real content, I think it would better to leave it alone. 
>Editing the archive might create more long term problems than 
>leaving the SPAM intact.

If all we're talking about is removing those messages that were added 
directly to the archive and never went out on the list, I don't think 
we are at all heading down the slope. Editing the archive of what 
actually went out to the list is another matter entirely.

I think messages which were "accidentally" added to the archive 
should be removed, without comment, by the list admin.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: Ralph Droms <[email-masked]>
Date: Thu, 06 Jan 2005 18:42:11 -0500

Bob - except when we get a burst of traffic on the dhcwg mailing list, we 
do archive more spam (couple of messages a day) than real traffic...

- Ralph

---

From: Bob Hinden <[email-masked]>
Date: Thu, 06 Jan 2005 16:09:01 -0800

Ralph,

>do archive more spam (couple of messages a day) than real traffic...

OK, I am convinced.  But lets fix the underlying problem first so the 
editing of the archive can be a one time thing.

Bob

---

From: [email-masked] (Scott Bradner)
Date: Thu, 6 Jan 2005 20:54:55 -0500 (EST)

I think that the mail-direct-to-the-archive is how IETF archives
mail for lists that are not run by IETF - if so it could
be disabled for IETF-run lists bit not teh others

Scott

---

From: Bill Fenner <[email-masked]>
Date: Thu, 6 Jan 2005 18:01:31 -0800

>e-mail archive is disabled.

I'll buy a ticket for that train!  Perhaps run each incoming message
for the archive through procmail (or some other filter) and only archive
messages that have the right List-ID:.  There's probably a shortcut for
lists that are served by the ietf.org installation itself, too.

I agree that having an online button for "remove this from the archive"
could raise questions.  I do think, however, that if we can remove
messages that were sent directly to the archives, we would be well
served by doing so.  If we can identify those messages easily, for
example by looking for Received: headers that show that it went
directly to wg-web-archive@ or wg-archive@ (note: two seperate archives
for ftp and www versions) I think we should.

I'd also love to see an automated process to ensure that the archives
are being fed.  I did a quick check on dhc (since I knew that Ralph had
problems with archives) and found that the ftp.ietf.org archives end in
November 2004, and the www.ietf.org archives are still going but full
of spam.  I'm sure nobody wants to go through all hundred-some active
working groups and make sure everything's going OK with the archives,
thus an automated process would be really an excellent thing.

  Bill

---

From: Aaron Falk <[email-masked]>
Date: Thu, 6 Jan 2005 18:04:17 -0800

> be disabled for IETF-run lists bit not teh others

If so it seems like it could at least made restricted to accept only
from a short list of senders.

--aaron

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 7 Jan 2005 09:52:58 -0500

On Thu, Jan 06, 2005 at 06:01:31PM -0800, Bill Fenner wrote:
> could raise questions.

Having a button that can remove anything could raise question.  

Having a button that says "move this to a spam directory" raises a lot
fewer.  That way things aren't deleted and you can offer a lot more
control to parties that it would be administratively questionable to
give outright delete access to.

-- 
Jeff Haas 
NextHop Technologies