Subject: RE: [Asrg] 2.a.1 Analysis of Actual Spam Data - Experimental Desi gn

From: "Sauer, Damon" <[email-masked]>
Date: Mon, 18 Aug 2003 13:37:58 -0500

I can match the message to my gateway logs and pull data from there too. I
just need to be able to extract the data for it to be useful.
 Also, some tool is better than no tool. I can work around short comings as
long as the basic functionality is there.
The rewrites are also very minimal and do not destroy the information that
we am looking for.
It will also allow me to get more people involved in classifying the email.

Regards,
Damon Sauer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Monday, August 18, 2003 2:30 PM
To: Sauer, Damon
Cc: 'Yakov Shafranovich'; 'Terry Sullivan'; [email-masked]'
Subject: Re: [Asrg] 2.a.1 Analysis of Actual Spam Data - Experimental Desi
gn 

Sauer, Damon writes:
> We found this piece of software that will run against an exchange 
> mailbox, gather certain information from email in a folder and put in 
> in a database ... It is very interesting. But it does not go far 
> enough. Could someone look at this? I think that it would be a GREAT 
> help in cataloging the data. Once the vital information is in a sql 
> database- I can match my logs to it.

I would caution against using Exchange as a back-end for mail reception; it
rewrites the messages extensively and many key spam signatures will be lost.

--j.

*****
"The information transmitted is intended only for the person or entity to
which it is addressed and may contain confidential, proprietary, and/or
privileged material.  Any review, retransmission, dissemination or other use
of, or taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited.  If you received
this in error, please contact the sender and delete the material from all
computers."

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 18 Aug 2003 15:47:01 -0400

Can you provide more detailed information as to what kind of blocking did 
you do. Did you reject all incoming emails with 550 and send a challenge 
message?

At 02:32 PM 8/18/2003, Sauer, Damon wrote:
>Just an FYI - My ENTIRE mail volume dropped from an average of 50M a month
>to 20M a month, with the same percentage being blocked at the content
>filters, after implementing address checking (550 given) at the connection.
>
>
>
>Regards,
>Damon Sauer
>
>
>
>From: Tom Thomson [mailto:[email-masked]]
>Sent: Monday, August 18, 2003 10:52 AM
>To: Terry Sullivan; [email-masked]
>Subject: RE: [Asrg] 2.a.1 Analysis of Actual Spam Data - Experimental Design
>
>
> > I feel pretty confident that one box can respond to requests sent to
> > multiple IP addresses, and therefore can serve as home to an
> > arbitrarily large number of different domains.  If these email
> > addresses "live" on 60 different machines, then there will be an
> > additional mechanical step of "synching" the data from each machine.
> > Then too, keeping one machine up for the experimental period strikes
> > me as less "overhead" than keeping 60 machines going.  That I can see,
> > using multiple boxes only serves as a potential confound, because
> > server availability affects spam volume in a systematic way. If one
> > machine (or  worse, two) go(es) "hard down" for a week or two, the
> > results of the larger experiment are placed at risk.  Addressess
> > served by that/those machine(s) will have a lower spam volume, of
> > course, but not because of the indepdendent variable.  But, as I said,
> > this feature qualifies as nice-to-have, but not required.
>
>I think you misread what I was proposing.  Comparisons have to be between
>mail addresses which are identical except for there 550 behaviour when
>determining whether 550 behaviour affects spam volume.  So you don't compare
>mailboxes on two different machines, or in two different TLDs.  What I am
>saying in effect is that the experiment needs to be carried out in a number
>of TLDs, since it may deliver different conclusions in different TLDs.
>
> > However, to the extent that there is some reasonable basis for
> > believing that spammers respond differentially to 550s from different
> > TLDs, then that imposes an additional requirement: keep the number of
> > TLDs small (say, 3: .com/.org/.net), or use a LOT more addresses.
>
>If the one-TLD experiment uses 60 pairs of adresses, then a multi-TLD
>experiment must use 60 pairs for each TLD.  Simple as that. Going to even a
>small number of TLDs (eg 3 TLDs) while keeping the original number of
>addreseses as you suggest is going to be a disaster if the TLD does have
>some effect, as it reduces the amount of data which can tell you about the
>effects of the 550 responses where they are the only independent variable by
>a factor of three. It would be helpful not to restrict the TLDs to those
>where English is thh prime language, as in the three you list.  Maybe use
>.com, .uk, .fr, .de (plus .org and .net maybe).
>
>There are four potential gains to using several TLDs, provided that enough
>data is collected to make a valid experiment within each individual TLD.
>First, we can see whether the 550 method has different effects in different
>domains; second, we can get some idea of the effect of tld on spam volume
>(anecdotal evidence conflicts here, and I've seen no solid numbers);  third,
>if the tld does in fact make no difference we have several times as much
>data to work with; fourth, if the 550 response does indeed have an effect we
>will be able to see if part of that effect is a reduction or increase in the
>unexplained variance.
>
> > >...I think it's perfectly reasonable to measure daily volumes...
> >
> > Knock yerself out.  Devote as much time as you like to analyzing daily
> > volume.  In fact, you can start right now, using Peter's data; it's a
> > large enough sample to permit a reasonably robust estimate of the
> > "true" population variance.  You might find analyzing those data to be
> > a statistically informative exercise; I know I did.
>
>I'm not the least bit interested in trying to do any further analysis on
>daily data.  What bothers me about just collecting (say) 90 day volumes is
>that an appropriate measure might be seven day volumes or 1 month volumes or
>three month volumes or even 90 days (unlikely - 90 days is neither an even
>number of weeks nor an even number of months so it won't properly mask
>periodic effects based on the calendar, which probably will be present).  If
>I have 1 day numbers I can use then to produce the numbers for any period
>which is a multiple of 1 day, and see which multiple of 1 day reduces the
>unexplained variance best (provided the experiment runs for long enough,
>that is) and that way I get to see whether the 550 responses (if they have
>any efect at all) produce a flat reduction or a change in shape or both.
>
>Anyway, you've already seen the comments I made after an initial analysis of
>Peter's data - I think I was the first to point out that there was no
>evidence of a downward tred, not even evidence of the absence of an upward
>trend, and that daily volume is very noisy indeed. And without a data for a
>properly organised control address to compare it with little useful analysis
>can be done except to note that there is a good deal of unexplained variance
>and no visible trend at any reasonable level of significance.
>
>Tom

---

From: "Sauer, Damon" <[email-masked]>
Date: Mon, 18 Aug 2003 16:31:37 -0400

I rejected only those addresses that did not exist in my address database of
users and honeypot addresses.
If the address did not exist the error is "550 user unknown". I cant go into
anymore detail as to what happens to senders that give more than x number of
550s or how that data is compiled and applied.

Regards, 
Damon Sauer 

-----Original Message-----
From: Yakov Shafranovich [mailto:[email-masked]]
Sent: Monday, August 18, 2003 3:47 PM
To: Sauer, Damon; 'Tom Thomson'; 'Terry Sullivan'; [email-masked]'
Subject: RE: [Asrg] 2.a.1 Analysis of Actual Spam Data - Experimental
Desi gn

*****

---

From: Paul Judge <[email-masked]>
Date: Mon, 18 Aug 2003 17:08:45 -0400

As Damon stated in his first and second post on this topic, this is not
challenge-response. It is simply address verification. If the address does
not exist, a 550 error is given.

> From: Brad Knowles [mailto:[email-masked]] 
> Sent: Monday, August 18, 2003 4:29 PM
> To: Sauer, Damon
> Cc: 'Tom Thomson'; 'Terry Sullivan'; [email-masked]'
> Subject: RE: [Asrg] 2.a.1 Analysis of Actual Spam Data - 
> Experimental Desi gn
> 
> 
> At 2:32 PM -0400 2003/08/18, Sauer, Damon wrote:
> 
> >  Just an FYI - My ENTIRE mail volume dropped from an 
> average of 50M a 
> > month  to 20M a month, with the same percentage being 
> blocked at the 
> > content  filters, after implementing address checking (550 
> given) at 
> > the connection.
> 
> 	Keep in mind that there are those of us who simply refuse to 
> communicate with anyone that uses a C-R type system.
> 
> 	So, what's your loss in legitimate volume?  Do you know 
> how many 
> people might have had real e-mail to send you, but decided not to 
> bother?
> 
> -- 
> Brad Knowles, <[email-masked]>
> 
> "They that can give up essential liberty to obtain a little 
> temporary safety deserve neither liberty nor safety."
>      -Benjamin Franklin, Historical Review of Pennsylvania.
> 
> GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ 
> !E-(---) W+++(--) N+
> !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) 
> X++(+++) R+(+++)
> tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* 
> tv+z(+++)
>

---

From: "Sauer, Damon" <[email-masked]>
Date: Tue, 19 Aug 2003 11:13:02 -0400

Our message systems, in the past, in order to do filtering and routing (we
have over 60 domains) accepted all email to those domains and did address
checking later.
 To answer the second question from Kurt... If I am understanding your
question correctly, No. There is no way to really tell because we do not
keep historical data for 550's for the purpose of analysis and my systems,
before we started address checking, didn't know about 550's. So there is
nothing to compare. My anti-spam scripts care more about how many 550's I
get, not which ones I am getting. However, I was able to remove over 8000 ip
addresses and ranges from our untrusted domains list due to these addresses
not trying to attach to us anymore. I can correlate most of the removals to
the fact that we started address checking, but I do not have empirical data
to "prove" that.

Regards, 
Damon Sauer 

-----Original Message-----
From: Jon Kyme [mailto:[email-masked]]
Sent: Tuesday, August 19, 2003 9:16 AM
To: ASRG
Subject: Re: RE: [Asrg] 2.a.1 Analysis of Actual Spam Data -
Experimental Desi gn

> At 4:31 PM -0500 2003/08/18, Sauer, Damon wrote:
> 
> 
> >   It is not C-R.
> 
> 	Sorry, my mistake.  I misunderstood.
> 
> > You give me a RCPT TO: the RCPT TO: is checked against my
> > address database. If it is good I say OK. If it is bad I say 550 No
> such
> >  user.
> 
> 	Okay, now I'm really confused.  I thought that most mail systems 
> did this -- if you try to send mail to an account that doesn't exist, 
> they respond with an error and refuse to accept the message.  Thus, 
> the mail volume goes down.
> 
> 	Is there something additional that has been done here?
> 

Indeed, many systems do not. A customer on a network that I'm familiar with
called the operations team to ask them if they could fiddle with their
firewall (in fact that firewall is managed by someone else that they hired
- which I guess they'd forgotten) to block port 25 for a particular IP
range.
The demand of generating bounces to a spam run (to no doubt forged senders)
had crippled their (exchange) server. When asked why they weren't just
rejecting unknown recipients they said that Mgmt wanted to see
"wrong email addresses".  How we laughed.

And then of course there's Yahoo and many others.

--

*****