Subject: Re: [Asrg] Collecting IP reputation data from many people

From: [email-masked]
Date: Tue, 2 Nov 2010 16:39:24 -0400

I was thinking more about how spammers could game this system, and... I
think obfuscation by reducing the precision of reporting would have
minimal benefit.  The only benefit I was hoping for was minimizing how well
a spammer could tell if he was successfully influencing the system.  And I
don't think it will.

I had been thinking of reporting percentage of email from each IP which is
not spam with maybe only 5 distinct values (10, 30, 50, 70, 90%), and
reporting no information on the actual volume of email from the IP.

Now I'm thinking about providing non-spam percentages, and a number
indicating relative total volume of email from the IP, probably both
with a range of 256 values.  Basically all the precision I think anybody
could find useful.

I'd like opinions on which would be better.

And I'm hesitant to discuss details of why I think obfuscation would not be
useful because I must assume there are spammers intelligent enough to be on
this list :/

If you think it's obvious enough (to spammers) why precision reduction
might have no benefit, go ahead and say it.

On 10/22, Daryl C. W. O'Shea wrote:
> You could avoid bulk amounts of forgery by passing sequence numbers
> back and forth or what not.  I'm thinking we might have been passing

UDP with verification of the sender makes sense.  The Roaring Penguin /
mimedefang implementation, which I think I'll largely copy, uses a
SHA1 HMAC signature using a shared secret (and an 8 byte random report ID).
So I could send back a challenge containing the ID, timestamp, and an
addtional random number (signed), and require them to sign that.
Still better than the overhead of TCP?

-- 
"Don't go around saying the world owes you a living. The world owes you
nothing. It was here first."  - Mark Twain
http://www.ChaosReigns.com

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 04 Nov 2010 20:34:11 +0100

> I'd like opinions on which would be better.

The second format might allow some sort of projective conglomeration of results.

> SHA1 HMAC signature using a shared secret (and an 8 byte random report ID).

For the record, let me annotate the url:
http://tools.ietf.org/html/draft-dskoll-reputation-reporting

---

From: [email-masked]
Date: Thu, 4 Nov 2010 15:49:03 -0400

> The second format might allow some sort of projective conglomeration of results.

What?

-- 
"Blades don't need reloading." - The Zombie Survival Guide by Max Brooks
http://www.ChaosReigns.com

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 05 Nov 2010 09:07:48 +0100

>>  The second format might allow some sort of projective conglomeration of results.

If I have reports from both A and B about S, I may want to add their 
results.  I can extrapolate the real volumes Va and Vb that A and B 
got from S by scaling the relative volumes Ra and Rb by, say, the 
volumes I send to A and B, respectively.  Call Ma and Mb these 
"projections" of A and B on My site, and Pa and Pb the percentages 
found in the report.  Then I can reckon their sum as

(Pa*Va + Pb*Vb) / (Va + Vb) = (Pa*Ra*Ma + Pb*Rb*Mb) / (Ra*Ma + Rb*Mb)

One may want to season the projection factors with more ingredients, 
such as the trust granted to the relevant reporter.

---

From: Douglas Otis <[email-masked]>
Date: Fri, 05 Nov 2010 08:55:35 -0700

On 11/5/10 1:07 AM, Alessandro Vesely wrote:
> >>> Now I'm thinking about providing non-spam percentages, and a
> >>> number indicating relative total volume of email from the IP,
> >>> probably both with a range of 256 values. Basically all the
> >>> precision I think anybody could find useful.

Not all email is bidirectional, where an amount sent will not reflect an 
amount received.  Received rates are poor predictors of sending rates 
due to nonlinear relationships between sending and receiving.  People 
are able to read only so much.  Volume over time can be expressed as 
absolutes using natural log with as little as 4 bits.  As sending volume 
increases, a receiver's relative volume will decrease due to a required 
greater diversity of sending destinations.  Natural log can approximate 
receiver reductions resulting from a greater diversity of destinations.

Summing log volume requires conversion back to linear values adjusted to 
the same time interval.  When not converted, natural log might better 
predict expected individual receive volumes when a very large number of 
receivers have been aggregated to better approximate actual sending rates.

-Doug

---

From: David Nicol <[email-masked]>
Date: Wed, 17 Nov 2010 16:11:17 -0600

On Wed, Oct 20, 2010 at 8:45 PM,  <[email-masked]> wrote:
>   Also, the availability of captcha
> solving for $2 per 1000.

which makes Just Captchas an inadequate pot of sender-pays chili.
Captchas may work very nicely as a non-money entry point though,
that's what I will use them for when I get back around to making my
system go again.