Subject: [Asrg] Collaborative real time spam blocking

From: Jim Whitescarver <[email-masked]>
Date: Fri, 22 Oct 2004 14:28:06 -0400

Greetings,

I do not know if this is the correct forum to address this issue.  
Please let me know if these issues are being addressed elsewhere.  What 
I am seeking is a means for coordinating a grass roots trust network for 
aggressive dynamic blocking of spammer IP addresses.    While there are 
many blacklists out there, they do not provide for mechanisms to 
facilitate real time blocking.  Standards are needed for collaborative 
blocking,  incident reporting and verification.  Spammers move to 
quickly for traditional blacklisting approaches be effective.

My experience blocking spam over the last month suggests that a 
collaborative dynamic approach could be quite effective.  It is 
remarkable what I am able to achieve all by myself.  It seems clear to 
me that  collaboratively we could actually end the spam problem.

With the exponential growth of spam, filtering it is not enough.  The 
spam processing on our email server overloaded the machine to the point 
where it was forced to reboot several times a day.  While our policy of 
discarding invalid addresses rather than rejecting them exasperated the 
problem in our case, it will not be very long, given the growth rate of 
spam, before all email servers on the Internet have to face a similar 
day of recconning.

For just over a month I have waged a personal war against spam.   On a 
single machine with less than 500 email accounts I have identified just 
about 200,000 spam emails from about 60,000 individual IP addresses 
defining about 3000 IP groups (of 256) having 3 or more spammer IP 
addresses.  I am rejecting a total of about 750,000 IP addresses.

All the 'experts' tell me black lists don't work any more because spammers are moving too fast and there are too many of them.  There is no simple solution but I believe an agreesive, combined, scalable, collaborative grass roots approach can work.

While I have succeeded in significantly reducing the spam load on the 
machine, I cannot alone stop spam in a timely manner or  impact the spam 
problem in general.  Every day I identify about 1500 new spammer IP 
addresses.  In many cases I block them immediately after receiving just 
one spam.  This prevents getting hundreds of spams from each address.

The trick is to identify a spammer IP as soon as possible and block it 
as soon as possible.  If this was done widely, the spammers would be out 
of business.    The system must agressively  identify and block 
spammers, but it also must be forgiving and heal as spammers move around 
the net.  If the incentives are there for Internet service providers to 
enforce anti-spam policies to avoid being blocked, and timely blocking 
can significantly reduce the profits from spamming, spam can be stopped.

Spam me once, shame on you!  Spam me twice, shame on me!

Trust networks are needed to avoid malicious blocking.  Participants 
should be able to negotiate the degree of  confirmation desired before 
blocking and choose trusted  and untrusted sources.  There need to be 
mechanisms for sharing black lists, whitelists, incidents, reprieve 
requests.  This should work on a grass roots level without any  central 
authority.  It can include standards for DNSBL services but will require 
additional web services.  It may also suggest extensions to the header 
standards.  It would also be useful to have a means of summarizing spam 
incidents by IP or group in terms of time frame, number of incidents, 
number of email addresses, and perhaps confidence.

In the remainder of this email  I provide additional details of how my 
system is working at present in anticipation of questions some of you 
may have.  You can see the system working in almost real time at:
http://Xanthus.Net/spam.cgi

I also introduce the use of "honey pot" email addresses for spam 
identification, which is an idea I have not seem elsewhere.  However, I 
don't see this as a standards issue at this point, but you may read on 
if you are interested.

Please let me know if there are any relevant activities in Collaborative 
real time spam blocking standards activities.

Thanks,

Jim
-------------------------------------------------------------
Notes on  my current spam blocking activities:
http://Xanthus.Net/spam.cgi

Rapid spam identification is essential for dynamic blocking to be 
effective.  The primary method is what I call "honey pot" email 
addresses mined from the mail logs.  These are non-existent users or 
accounts that have been out of use for many years.  I have collected 
thousands of these as our providers default policy was to not return 
them as undeliverable.  Most were invented by worms or spammers and 
forged from addresses that were subsequently harvested.

Since these users do not exist, they cannot solicit any email.  Any 
email to these addresses is considered unsolicited and the address of 
the SMTP host is blocked.   Spam filtering tools may be used to 
automatically confirm the spam nature of these emails before 
automatically blocking for accounts that might conceivably get a valid 
email.

A second  source is users IMAP folders named SpamBlock.  They save 
emails there to prevent future emails for them or other users.  
SpamBlock folders are processed every few minutes.   Those who use a 
SpamBlock folder must use it responsibly to insure they don't block 
anybody legit. A secondary spam filter could help avoid errors here as 
well.

I check a whitelist and then just add a REJECT to the access database 
for SENDMAIL.  Periodically I add REJECTs for IP groups of 256 with 
three or more spammer IP addresses.

In a month I've only had three user complaints, one that was due to my 
neglecting to put hotmail.com on my whitelist, the other  two due to 
users who filed non-spam emails in their SpamBlock folders.  There is no 
painless way to solve the spam problem. I believe that dealing with 
wrongly blocked addresses is the price we must pay if we really want to 
stop spam.  Affirmative action against spam means guilty until proven 
innocent, but we can be very forgiving, at least on fist offenses.

I realize this is a tough policy, but it is what is needed, I think, to 
stop the exponential growth of spam.  It is dynamic, so that with enough 
participants, spam could be stopped no matter how fast the spammers 
move.  Spamming would become unprofitable and ISP would be motivated to 
enforce anti-spam policies.

I also forgive individual IP addresses after 10 days or so, I may change 
that.  Many seem to never spam again, while others repeat 
periodically.   I do not forgive those with repeated incidents to 
multiple email addresses yet.   I will give them another chance to be 
good sooner or later.

I think a combination of approaches is needed to fight spam.  My 
approach is best suited for blocking the new spambots and spammer 
sites.   If widely employed it could have a major impact on the industry.

There is much more I want to do. 

I want to add auto real time reporting of spammers also.  The standards 
can be important here so I am happy to see an ASRG group in that area.   
I see some similarities in some of the desired functionality that might 
also be employed in supporting dynamic collaborative blocking.

It has been suggested that my lists can be compressed greatly using CIDR 
notation for firewall rules.  That could also allow much greater 
selectivity and effectiveness in IP group blocking using powers of two 
group sizes.   And CIDR would be the preferred format according to our 
Internet provider.   Let me know if you know of any tools that could 
help automate this.

I am not sure how much further I'll get in my own personal war against 
spam, I need to find a way to support my efforts...  In the mean time 
let me know if you see any potential collaboration.

Thanks,

Jim Whitescarver
[email-masked]
973-643-0920

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Devdas Bhagat <[email-masked]>
Date: Sat, 23 Oct 2004 01:00:59 +0530

> aggressive dynamic blocking of spammer IP addresses.    While there are 

The best idea I have seen for this is a PGP/GPG signed post to usenet.

<snip>
> 
> All the 'experts' tell me black lists don't work any more because
> spammers are moving too fast and there are too many of them.  There is
> no simple solution but I believe an agreesive, combined, scalable,
> collaborative grass roots approach can work.

DNSBLs work. They just shouldn't be your only filters. Validating email
addresses, both syntactically and against lists of legitimate recipients,
checking for helo content, matching DNS (forward/reverse with regular 
expressions), DNSBLs, sender verification.....

> one spam.  This prevents getting hundreds of spams from each address.

Nice. Join the club.
 
> can significantly reduce the profits from spamming, spam can be stopped.

The trick is to identify the spammer IP.

> blocking and choose trusted  and untrusted sources.  There need to be 

Why? I choose which DNSBLs to trust, and which not to. Any DNSBL user
needs to do that.

<snip>

> don't see this as a standards issue at this point, but you may read on 

In email related activities, these are known as spamtraps.

Devdas Bhagat

---

From: Markus Stumpf <[email-masked]>
Date: Fri, 22 Oct 2004 22:13:17 +0200

On Fri, Oct 22, 2004 at 02:28:06PM -0400, Jim Whitescarver wrote:
> requests.

This is one of the ideas of DNS-MTAMARK. In reverse DNS mark a IP as a
designated mailserver.

> identification, which is an idea I have not seem elsewhere.

There are thousands of the and they are in wide use.

> real time spam blocking standards activities.

spamcop has a blacklist that is feeded by spam reports and uses a short
time to live for entries, so that active spews can be blocked fast.

However there are a lot of problems involved in blacklists (and
whitelists also):
1) stale entries
   we have our own maintained blacklists with entries dating back to
   2001. The list has about 100 million IP addresses listed and it
   for sure has stale entries.
2) grouping addresses
   remember: some admins at ISPs a fscking dumb.
   I have seen cases where they had one or two official mailservers in
   the same /24 as their dialin accounts. And as they use some automatic
   DNS maintenace tool the rev DNS for that mailservers is even the same
   as for the dialin lines. You have 5 incidents from that subnet, you
   group and block the whole /24 and your tech hotline has some fun with
   angry customers.
3) classification
   we operate a network of some hundred mailservers for our customers.
   These include virus checking. We had the idea to automatically run
   a blacklist of "virus spews" (that are also spam spews in a lot of
   cases) by sending UDP messages to a centralized server that puts the
   IPs from that messages in a blacklist.
   Bad idea.
   Really a lot of the messages going to the mailservers are forwards
   that are routed through mailservers without virus checking. Within
   24 hours we had tons of large, official mailservers of large ISPs and
   organisations on the blocklist. Luckily in the test phase we used it
   for tagging only and didn't block hard.
   Even setting a timeout of 4 hours for the entries didn't change much.
4) the same problem above for viruses applies to spam messages, too.
   For dialup customers we provide outgoing relays (smarthosts). Although
   we have a close monitoring of those servers and abuse is detected
   very fast usually and we block the customer (open relay, virus,
   feedback script on the webserver).
   However even the e.g. 500 messages going through may trigger honey
   pots and have the server blacklisted. Due to some latency (managing
   the blacklists) the entry may even be done after we already have
   fixed the problem.
5) we do artificial backup MX
   spammers and even viruses prefer to connect to backup MX server
   because they have weaker policies than the best MX servers usually.
   The greylisting success principle "spammers fire and forget" makes
   the artifical backup MX servers also very successful.
   Same idea as for the virus detection: feed the hosts connecting to
   the backup MX if the best MX is running to a blacklist.
   Bad idea.
   Even some official mailservers have their own "idea" what to do on
   delivery failures (temp or perm) to the best MX and quite some of
   them connect do MX servers with a longer distance (bigger prio
   number).
   During a four weeks test phase we collected about 800000 unique IP
   addresses, but through the fact - even it if where only very few -
   some of them where from legitimate mailservers it renders this mechanism
   useless. Having a big mailserver that makes 300000 connections a day
   to the mailservers under your control on the block and be it for
   15 minutes does a lot of damage. Even though customers hate spam
   they hate it even to have messages blocked they are waiting for,
   because they need them urgently or they are of big value.

All this automated blocking looks very good and promising at first glance,
but the devil is in the details ...

	\Maex

-- 
SpaceNet AG            | Joseph-Dollinger-Bogen 14 | Fon: +49 (89) 32356-0
Research & Development |       D-80807 Muenchen    | Fax: +49 (89) 32356-299
"The security, stability and reliability of a computer system is reciprocally
 proportional to the amount of vacuity between the ears of the admin"

---

From: Matthew Elvey <[email-masked]>
Date: Fri, 22 Oct 2004 14:14:36 -0700

On 10/22/2004 12:30 PM, Devdas Bhagat sent forth electrons to convey:

>>aggressive dynamic blocking of spammer IP addresses.    
>>    
>>
Such a system is already in operation and wide use.  For more info see
http://www.drbl.croco.net/eng/  (or http://www.drbl.croco.net/ for 
Russian!) and
http://www.drbl.croco.net/eng/faq.html
Just set up a DRBL node, store your BL in it, and announce it.

>The best idea I have seen for this is a PGP/GPG signed post to usenet.
The latency is rather high, but it is highly DDoS-resistant.

---

From: "Mark C. Langston" <[email-masked]>
Date: Fri, 22 Oct 2004 16:56:19 -0700

> quickly for traditional blacklisting approaches be effective.

Have a look at GOSSiP (http://www.sufficiently-advanced.net/ ).  It's a
massively-distributed, peer-to-peer email reputation system.  It could,
in theory, do exactly this sort of thing.  However, I'd need to add some
fairly thoroughly-thought-through identity-aggregation and behavioral
pattern analysis/identification to get it all the way to where you'd
want it to be.

As a quick kludge, you could probably examine all 0 reputation, 0
confidence incoming mail and simply pattern-match GOSSiP ID elements
against other 0/0 identities in the database.  If the total mailcount
from a given identity is, say, <10 (or anything you wish), it's all
tagged as being spam, and the ID is a partial match to other IDs with
similar characteristics, there are assumptions you could make before the
mail is accepted for delivery.  

Since GOSSiP's basically a social network (in the "social network
theory" sense), you'd also have regional variations because of the
scale-free nature of such nets.  The "relay nodes" described at the end
of the spec on the website would allow visibility into these regional
variations, just as "social hubs" act as bridges between otherwise
diverse social groups.

-- 
Mark C. Langston            GOSSiP Project          Sr. Unix SysAdmin
[email-masked]   http://sufficiently-advanced.net    [email-masked]
Systems & Network Admin      Distributed               SETI Institute
http://bitshift.org       E-mail Reputation       http://www.seti.org