Subject: dealing with email (was Continuation of Thread: UDP Lite)

From: Mark Allman <[email-masked]>
Date: Fri, 03 Sep 1999 18:27:30 -0400

Warning: large digression, please forgive

> (I'm hoping against hope that the end2end and PILC lists are
> somehow syncrhonized so that at least some of us will not all have
> to deal with several copies of every message.  I will continue to
> trim my addressee lists to only the mailing lists, so you won't
> have to see more than 2 copies of my words.  Dealing with 2 or 3
> copies of everything is a pain, but it seems to be necessary to
> get past the forum objections.)

PILC is not syncronized with end2end.  

I, too, find it incredibly annoying to sift through 2 (or more!)
copies of everything.  And, I have found the practice of CCing many
lists I am on has been increasing lately.  So, I hope you'll forgive
me for not practicing what I preach, but below is running code that
will get rid of duplicate messages in a particular folder, assuming
you use mh (and why wouldn't you?).  In case anyone is interested...

allman

---
#!/usr/local/bin/perl

# mh-rm-dups
#     remove duplicate messages from an mh folder
#
# Mark Allman ([email-masked])
# Last Updated: Fri Sep  3, 1999

open (FOLDER,"folder |");
while (<FOLDER>)
{
    print;
}
close (FOLDER);
print " scaninng...\n";
open (SCAN,"scan |");
while (<SCAN>)
{
    ($msgno) = /\s+(\d+)/;
    $active [$msgno] = 1;
    if (!$min)
    {
	$min = $msgno;
    }
    $max = $msgno;
}
close (SCAN);
print " grabbing Message-Id...\n";
for ($i = $min; $i <= $max; $i++)
{
    if ($active [$i])
    {
	$found = 0;
	open (MSG,"show -nocheckmime -showproc cat $i |");
	while (<MSG>)
	{
	    if (/^Message-Id/)
	    {
		$msgid [$i] = $_;
		$found = 1;
	    }
	}
	if (!$found)
	{
	    $active [$i] = 0;
	}
	close (MSG);
    }
}
print " comparing...\n";
for ($i = $min; $i <= $max; $i++)
{
    if ($active [$i])
    {
	for ($j = $min; $j <= $max; $j++)
	{
	    if ($i != $j && $active [$j])
	    {
		if ($msgid [$i] eq $msgid [$j])
		{
		    print "  ", $j, " is a duplicate of ", $i, ", removing\n";
		    $active [$j] = 0;
		    `rmm $j`;
		}
	    }
	}
    }
}
exit (0);

---

From: Randy Bush <[email-masked]>
Date: Sat, 4 Sep 1999 10:30:20 -0700 (PDT)

[ whole lot of crufty perl deleted ]

use procmail

    :0 Wh: msgid.lock
    | formail -D 8192 msgid.cache

works quite well unless you are subscribed to mailing lists exploded by some
really sickware called lyris which changes the message-id: line.

randy