Subject: [Asrg] A paper/project worth considering (found it!)

From: Rich Kulawiec <[email-masked]>
Date: Tue, 2 Dec 2008 20:45:40 -0500

Credence
	http://www.cs.cornell.edu/People/egs/credence/

Quoting:

	Much of the content in peer-to-peer filesharing networks is
	corrupt, damaged, or mislabeled. Such polluted content makes
	it difficult for correctly functioning peers to locate desired
	content, decreasing the overall usefulness of the network.

	Credence is a practical distributed reputation system, designed
	to counteract content pollution in peer-to-peer filesharing
	systems. Credence enables a peer to determine the authenticity of
	online content; that is, how accurate the purported description
	of the object matches the object itself. Participants in the
	Credence network vote on objects; Credence collates these votes,
	and weights them by a novel similarity measure which weighs highly
	votes from like-minded peers while discounting the votes from
	peers engaged in vote-spamming. This novel voter correlation
	scheme provides an incentive for peers to vote honestly and
	mitigates the impact of dishonest peers.

	We have implemented Credence on top of the LimeWire client for
	the Gnutella network. Our client provides a peer-based judgement
	that a given object will possess the properties with which it
	is labeled and enables users to evaluate search results for
	authenticity before downloading.

	Many peer-to-peer reputation schemes have been proposed in
	academia. Credence is the first practical implementation of a
	peer-to-peer reputation scheme. As future systems become more
	connected and endpoints become more reliant on content provided
	by other nodes, reputation systems like Credence can play an
	important role in assessing the validity of remote content.

Consider the possible applicability of this to "systems observed to be
emitting SMTP traffic" where human "voters" are replaced by spamtraps
and similar.

I'm not claiming that this is "the answer" by any means.  I do think
it's an interesting approach to a similar problem.

---Rsk
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: "J.D. Falk" <[email-masked]>
Date: Wed, 03 Dec 2008 15:46:09 +0530

> it's an interesting approach to a similar problem.

That'd be a reputation system focused on the "bad/not bad" axis, based on
spamtraps.  Spamtraps are very, very good at determining "bad," but not very
effective for "not bad" or "good" -- which is why most existing reputation
systems, such as senderscore.org, also mix in user feedback via complaints.

---

From: John Levine <[email-masked]>
Date: 3 Dec 2008 12:20:00 -0000

>	http://www.cs.cornell.edu/People/egs/credence/

I've added an Interesting Papers page to the ASRG wiki for papers
mentioned on the list.  Feel free to get a password and add other
interesting papers relevant to spam even if they're not mentioned
on the list.

http://wiki.asrg.sp.am

---

From: Rich Kulawiec <[email-masked]>
Date: Wed, 3 Dec 2008 09:31:08 -0500

> systems, such as senderscore.org, also mix in user feedback via complaints.

I think user feedback is unreliable and shouldn't be trusted:
if users could reliably tell spam from not-spam, then phishing would not
be a serious problem and spammers hawking products would rarely make sales.
Even clueful, experienced humans are occasionally confused or misled
by sufficiently clever or obfuscated spam.  Besides, I'm not really
comfortable trying to place the burden of the spam problem on the
backs of end users, who have other things to do.  It's not their
"job", it's ours.  (We don't ask them to help configure routers and
firewalls and such, we shouldn't ask them to help configure mail systems.)

But even if you don't buy that argument, here's another reason why
end-user feedback should be discarded: where's it coming from?

It's coming from the same systems that are being used to send spam,
because they're fully-compromised...which means that *nothing* they do
can be trusted, and input from them certainly shouldn't be presumed to
reflect the will of their former owners.  If any such reputation system
was adopted on a large scale, then it would be worth the time of the new
owners of all those systems to use them to try to game the reputation
system.  Now granted, a system like Credence is designed to blunt the
effects of that tactic; but why bother allowing input at all from those
hosts, when it's clear a priori that it will be overwhelmingly bogus?
All it will do is increase scalability issues, generate lots of network
traffic, etc.  Sure, spammers would no doubt try to inject bogus
input into a reputation system from their own hosts (that is: the ones
that really do belong to them), but it seems to me that reducing the
numbers involved might keep the problem tractable.

One of the interesting related questions (for which I don't have an
answer) is "what number and diversity of spamtraps are required to have
X probability of noticing a spam run of volume Y in time Z?"

---Rsk

---

From: "J.D. Falk" <[email-masked]>
Date: Wed, 03 Dec 2008 21:35:13 +0530

> I think user feedback is unreliable and shouldn't be trusted:

As previously discussed, modern reputation algorithms take that into account
-- for the very reasons you've discussed.  However, the logic is almost
always proprietary.

> X probability of noticing a spam run of volume Y in time Z?"

Sounds like an interesting research project.  Go for it.

---

From: Jeff Macdonald <[email-masked]>
Date: Wed, 3 Dec 2008 10:28:14 -0500

>interesting papers relevant to spam even if they're not mentioned

I see that there is a link to CEAS. One of the papers caught my eye,
http://ceas.cc/2008/papers/ceas2008-paper-33.pdf. Since credence is a
reputation system and Repuscore is too (what the ceas paper is talking
about), and at the risk of beating the subject to death, I'd like to
focus on the attributes of a reputation system and why folks think
these attributes are important.

For instance, in the CEAS paper, they modified the calculation for
reputation to include volume (yet the outcome was the same as far as I
could tell) and the desire to have reputation be slowly gained yet to
be able to fall quickly.

-- 
Jeff Macdonald
[email-masked]

---

From: "David Nicol" <[email-masked]>
Date: Wed, 3 Dec 2008 17:06:39 -0600

On Tue, Dec 2, 2008 at 7:45 PM, Rich Kulawiec <[email-masked]> wrote:

>        Credence collates these votes,

I wonder if the credence team ever made the netflixprize leaderboard.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: Rich Kulawiec <[email-masked]>
Date: Thu, 4 Dec 2008 06:50:37 -0500

> -- for the very reasons you've discussed.  However, the logic is almost

<chuckle> I don't see why: "ignore it completely, as it's obviously garbage"
is a pretty simple algorithm.

> Sounds like an interesting research project.  Go for it.

I've actually been slowing working on this for many years, and have
tentatively concluded that it may not be answerable.  The effort hasn't
been entirely wasted, as I've been able to prove to my satisfaction that
list-washing has long since been extended from email addresses to domains,
MX's, networks and other constructs.  This suggests that (unlike perhaps
a decade ago) the utility of "local" spamtraps may be increasing confined
to less sophisticated spammers, as more clueful ones have found it
worth the effort to avoid them.  I've been able to draw a few other
conclusions along the way as well, but I'm becoming convinced that
there may be too many variables in play to effectively answer the
core question.

---Rsk

---

From: Seth <[email-masked]>
Date: Thu, 4 Dec 2008 10:28:59 -0500 (EST)

> is a pretty simple algorithm.

When you're the size of AOL, .1 bits of information per button press
still yields a lot of useful data.

Seth

---

From: "Chris Lewis" <[email-masked]>
Date: Thu, 4 Dec 2008 11:18:47 -0500

> still yields a lot of useful data.

Even when you're the size of us (almost _exactly_ 1000 times smaller in
all numbers), it's still quite useful information.

We have a TIS button.  I have no reason to believe that the error rate
on hitting it is even as bad as 5%.

Do we block an IP on one TIS hit?  No.  We compute good/bad ratios and
have heuristics on when its high enough to do something about.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 04 Dec 2008 08:31:49 -0800

> have heuristics on when its high enough to do something about.

The "bad" number is affirmative.  People hit TIS.  As a measure, the bad number 
therefore has a 100% confidence level of accuracy (as long as we are careful 
about defining badness.)

But where do you get the 'good' number from and is it really equally forceful?

If the basis for obtaining each number is conceptually different -- ie, if the 
two different numbers really do warrant differential faith in their accuracy, 
then how do you balance between them?

In case I'm not being clear:

      The fact that someone hit TIS means that -- independent of whether it is 
actually something that might be called spam -- the message irritated the user. 
  Every single TIS click has a 100% confidence factor, in terms of being a valid 
count of being problematic to the end-user.  (I'll quickly acknowledge that we 
have a derivative issue from the fact that a given user is inconsistent and what 
is irritating to me this morning might not be irritating this afternoon; but we 
have plenty to consider by just looking at first-order issues.)

      In contrast, perhaps you take the 'good' number from something like "no 
one complained".  There can be lots of reasons no one complained, only some of 
which are due to a message's being "good".  So our confidence in the aggregate 
measure of goodness needs to be much less than 100%.

So, how do we factor in differential confidence levels in the final assessment?

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: Steve Atkins <[email-masked]>
Date: Thu, 4 Dec 2008 08:42:50 -0800

On Dec 4, 2008, at 8:31 AM, Dave CROCKER wrote:

> The "bad" number is affirmative.  People hit TIS.  As a measure, the  
> bad number therefore has a 100% confidence level of accuracy (as  
> long as we are careful about defining badness.)
>
> If the basis for obtaining each number is conceptually different --  
> ie, if the two different numbers really do warrant differential  
> faith in their accuracy, then how do you balance between them?
>
>     The fact that someone hit TIS means that -- independent of  
> whether it is actually something that might be called spam -- the  
> message irritated the user.  Every single TIS click has a 100%  
> confidence factor, in terms of being a valid count of being  
> problematic to the end-user.  (I'll quickly acknowledge that we have  
> a derivative issue from the fact that a given user is inconsistent  
> and what is irritating to me this morning might not be irritating  
> this afternoon; but we have plenty to consider by just looking at  
> first-order issues.)

A common approach is to select lots of messages, then hit the TiS  
button. The recipient may not have read, or even seen, any given mail  
in that group.

>
>
>     In contrast, perhaps you take the 'good' number from something  
> like "no one complained".  There can be lots of reasons no one  
> complained, only some of which are due to a message's being "good".   
> So our confidence in the aggregate measure of goodness needs to be  
> much less than 100%.

If an email is moved from the junk folder to the inbox that's usually  
treated as a significant good. There are UI approaches in place that  
discourage using the junk folder as just another mailbox in order to  
encourage that behaviour.

There are also a number of other measures in use, which I'm not going  
to go into on a public mailing list.

> So, how do we factor in differential confidence levels in the final  

We don't. The ISPs running the particular system do so, in an adaptive  
manner, in order to optimize the comfort of their customers.

Cheers,
   Steve

---

From: "Chris Lewis" <[email-masked]>
Date: Thu, 4 Dec 2008 13:15:36 -0500

> But where do you get the 'good' number from and is it really equally forceful?

...

> So, how do we factor in differential confidence levels in the final assessment?

Ironically, I'm in the process of rebuilding this code at the moment ;-)

When you read this keep in mind that this is in _addition_ to all the
other filtering (including DNSBLs, both 3rd party and local) that we use.

Basically what we do is generate a score based on the number of
non-blocked emails, contentblocked emails (not IP-blocked), trap volumes
and complaints, and pick a threshold score.  Each of the numbers is
scaled differently in a computation something like this:

if (((complaints * cf + contentblocked * bf + trap * tf) / non-blocked)
> 1) {
	go block the IP
}

[Notice that we're not factoring in blocked IP.  Specifically to avoid
the thresholder locking up thru positive feedback ;-).  They're blocked
anyway, so it doesn't matter.]

Where cf, bf, and tf are chosen thru experience and experimentation.
There's also some gunk in there to deal with when the numbers are too
small to be significant (especially non-blocked == 0 ;-).  When the
non-blocked numbers are low, it doesn't matter very much whether you
block it or not anyway.

Note that there's also an implicit factor of how long the metrics are
over.  In the past it was 2 days.  Now it's probably going to 7 days
potentially with factoring in abrupt volume increases.

All of the metrics numbers have "100% confidence".  The scaling factors
are a confidence factor for each number.

They're somewhat predictable.

Eg: Our "TIS hit per spam" compliance factor is (currently) about 1 in
50.  Ignoring other factors, assuming smooth distribution, a cf of 25
will cause the IP to block when the output is 50% spam.  We put lots of
headroom in to allow for uneven distribution.

In the past we were using something like 50, 2 and .01 respectively for
cf, bf and tf.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: Seth <[email-masked]>
Date: Thu, 4 Dec 2008 13:29:39 -0500 (EST)

> all numbers), it's still quite useful information.

And your average user probably has 1000 times as much clue, so it
balances out.

Seth

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 04 Dec 2008 10:30:05 -0800

Chris Lewis wrote:
>   Each of the numbers is

Relative to my question, what you've said is that you have a scaling factor 
specifically for the TIS hits.  That lets you tune its specific confidence level.

But you don't cite any of the "good" attributes, nevermind scaling factors, per 
your reference to good/bad.

I'm being pedantic because it was the differential nature of postive vs. 
negative measurements that intrigued me, in terms of practical impact.

d/
-- 

   Dave Crocker

---

From: der Mouse <[email-masked]>
Date: Thu, 4 Dec 2008 13:52:01 -0500 (EST)

> The fact that someone hit TIS means that -- independent of whether it
> is actually something that might be called spam -- the message
> irritated the user.

Not quite.

Haven't you ever hit the wrong button by mistake?  Or performed an
operation on selected items, only to discover the set of selected items
wasn't what you meant it to be?

I sure have.

					Mouse

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 04 Dec 2008 11:01:11 -0800

yeah, but my reference to time of day was intended to cover the vagaries that 
make hitting the button noisy.  But my point is that the user does have to take 
affirmative action, in contrast with the way "goodness" is typically measured.

d/

-- 

   Dave Crocker

---

From: "Chris Lewis" <[email-masked]>
Date: Thu, 4 Dec 2008 14:22:28 -0500

> your reference to good/bad.

Not quite - the counts in the numerator are "bads" and in the
denominator the "goods".  Each of the counts have scaling factors
(normalized so that "passed through"'s scaling factor is 1).

I don't look at them as confidence indicators, but rather as scaling
factors to derive some sort of notion of "what's the probability of a
given email from this IP being spam?".

I form that equation that way strictly thru force of habit, because
given the headroom I want it to have, pendantic accuracy is irrelevant.
 A different equation for "block this IP?" would be something like this:

(complaints * cf + contentblocked * bf + trap * tf)
--------------------------------------------------- > maxratio
(complaints + contentblocked + trap + passthru)

Where if cf, bf and tf were all 1, is a simple ratio of spam/total.

It's probably easier to comprehend, but, might not have the
characteristics I want.  I'll play with it.

---

From: "Chris Lewis" <[email-masked]>
Date: Thu, 4 Dec 2008 14:23:19 -0500

> And your average user probably has 1000 times as much clue, so it

It's not a matter of balancing, it's a matter of "we're still big
enough" for the information to be useful.

I wouldn't count on "more clue".  The typical AOL user is on AOL because
they _want_ to be.  Whereas many of ours are only on here because they
_have_ to be, and aren't necessarily any more savvy.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 04 Dec 2008 12:23:25 -0800

...
> (normalized so that "passed through"'s scaling factor is 1).

damn. apologies.  I missed the right-most part of the equation.

now that I'm seeing it, where is the factor for it (non-blocked * nbf)?

> given email from this IP being spam?".

right. and i did see the difference in a 'factor' vs. a 'confidence' but skipped 
past that, since both are about determining an amount to discount the value.

d/
-- 

   Dave Crocker

---

From: "Chris Lewis" <[email-masked]>
Date: Thu, 4 Dec 2008 15:41:40 -0500

> now that I'm seeing it, where is the factor for it (non-blocked * nbf)?

In that equation, all of the other factors are scaled so that the scale
factor for non-blocked is 0.

Switching over to Perl code ;-)

("prod" is passthru, "trap" and "other" are two different trap lumps,
"content" is content filter hits)

    $factor_complaint = 25 if !defined($factor_complaint);
    $factor_content = 2 if !defined($factor_content);
    $factor_trap = .01 if !defined($factor_trap);
    $factor_other = .005 if !defined($factor_other);

    my $traffic = $prod + $content + $trap + $other;

        if (!$traffic || !$prod) {
            $info->{score} = 1;
        } else {
            $info->{score} =
                $factor_complaint * $compcount / $prod +
                $factor_content * $content / $prod +
                $factor_trap * $trap / $prod +
                $factor_other * $other / $prod;
        }

if scale > .75, the IP gets blocked.

Which sucks.  Immediately blocks some google mail servers.  It's
probably not wrong, but...

I could replace $prod with $traffic in all the denominators, but you
can't get content or complaint hits on the traps.  So that just dilutes
the score for very high trap hits.

Remember, I'm not going for pedantic purity.  I'm going for a function
that has lots of head room and catches the worst offenders.

I'm going to recode to my original function and see how that works.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: "Chris Lewis" <[email-masked]>
Date: Thu, 4 Dec 2008 15:46:48 -0500

> factor for non-blocked is 0.

Urp.  1.

---

From: "J.D. Falk" <[email-masked]>
Date: Fri, 05 Dec 2008 05:25:47 +0530

> that has lots of head room and catches the worst offenders.

Yep, and that's all anyone can do, given the ever-changing nature of spam.

Other reputation systems I've worked on or learned about also mix in other
factors, such as:

- "not spam" votes (which aren't weighted 1:1 with spam votes)
- reporter reputation (not all users' votes get the same weight)
- past behavior of the IP (change thresholds based on recent behavior)
- type of mailstream (different thresholds for gmail vs. sears)

---

From: "David Nicol" <[email-masked]>
Date: Fri, 5 Dec 2008 13:55:17 -0600

So the post-GPTP approach to spam/not-spam was (is?) envisioned as
statistical confidences on several axes, coupled with MUA support for
tuning your own filter (where you are the U in MUA) to provide you an
optimum filter while sharing your feedback ratings on all axes on
which you care to comment.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: Michael Thomas <[email-masked]>
Date: Mon, 08 Dec 2008 17:01:14 -0800

Dave CROCKER wrote:
>      The fact that someone hit TIS means that -- independent of whether 
> it is actually something that might be called spam -- the message 
> irritated the user.  Every single TIS click has a 100% confidence 
> factor, in terms of being a valid count of being problematic to the 
> end-user.  (I'll quickly acknowledge that we have a derivative issue 
> from the fact that a given user is inconsistent and what is irritating 
> to me this morning might not be irritating this afternoon; but we have 
> plenty to consider by just looking at first-order issues.)

   I think as was already pointed out, 100% is not something that
   humans do well in any context. That is, mistakes happen. And it's
   probably worth pointing out again that TIS and actual spam are
   only loosely correlated. People who complain about the
   human evaluators (not you) being imperfect spam judges are
   missing the larger point of what _they_ see that button as.

>      In contrast, perhaps you take the 'good' number from something like 
> "no one complained".  There can be lots of reasons no one complained, 
> only some of which are due to a message's being "good".  So our 
> confidence in the aggregate measure of goodness needs to be much less 
> than 100%.
> 

   I've often wondered whether you could do something with timing
   the longevity of things in people's inboxes as first order
   approximation of "value". For example: if the timing between me
   seeing a piece of mail, and me hitting the delete button is
   consistently short, it's probably an indication that I'm not
   very interested in it. Maybe not enough to killfile them, but
   it probably would yield some clues as to how *I* prioritize some
   traffic over other traffic.

   As you allude to, this is clearly a dynamic system too: my interest
   in some topics is situational, and clearly changes over time. It
   is also context based: even actors that I rarely read may be
   contributing to a subject that I'm very interested, etc, etc.

   What this really points to, IMO, is that the entire way that mail
   -- and the many other emerging or established media -- are presented,
   prioritized, alerted, etc are pretty well borked. Spam is just one
   small -- but important -- part of that problem. But even if spam were
   solved through a divine act tomorrow, it would not address the ever
   increasing fire hose that we're demanded to drink from.

		Mike

---

From: Douglas Otis <[email-masked]>
Date: Tue, 9 Dec 2008 10:06:38 -0800

On Dec 4, 2008, at 3:50 AM, Rich Kulawiec wrote:

> This suggests that (unlike perhaps a decade ago) the utility of  
> "local" spamtraps may be increasing confined to less sophisticated  
> spammers, as more clueful ones have found it worth the effort to  
> avoid them.  I've been able to draw a few other conclusions along  
> the way as well, but I'm becoming convinced that there may be too  
> many variables in play to effectively answer the core question.

The less sophisticated approach still represents a majority of the  
problem.  Have a portion of the detection remain inactive.  A  
differential between the active provides evidence to the level of  
avoidance.  When detection represents less that 1% of the overall,  
larger volumes will appear more diverse, causing non-linear detection  
rates.  The real complexity is with botnets or IP addresses vulnerable  
to BGP hijacking.  They remain dormant over long periods, but may  
appear suddenly.  Unfortunately, the speed of domain registration and  
lack of advanced notice is effectively leveraged as a means to recover  
botnet control nodes and to defeat name based protections.

-Doug

---

From: Rich Kulawiec <[email-masked]>
Date: Sun, 14 Dec 2008 15:24:30 -0500

> on hitting it is even as bad as 5%.

Interesting.  As I mentioned elsewhere, I recently went through nearly
5 years of feedback loop reports from AOL and found that the error
rate was 100.00% -- every report ever filed was wrong.  (I think I
also mentioned that I found cases where users reported *their own
messages* to mailing lists as spam.)

I have no reason to think AOL's users are any better or worse at this
than Comcast's or Yahoo's or any other ISP/mail provider.  (I should
conjecture that Chris's users are better -- well, they'd have to be in
order to keep the error rate that much lower!)

I think at the scale of the Internet, users are awful at telling spam
from not-spam: if they were good at it, phishing would be a non-problem.

But let me put all of these conversation about end-user abilities
aside and look at this a different way.  Anti-spam policy is as much a
security function as, say, firewall configuration; and there's no way
I'd even consider giving users the ability to affect that.  It's all
very populist to give users these controls, but I think it's terribly
misguided and reflects a lack of realization that spam can be as much
of a security threat as malicious packets.  Analyzing such threats
and devising effective counter-measures to them requires trained,
experienced people -- moreover, it requires people who have the
responsibility for doing so.

What I'm arguing (and I've argued this elsewhere) is that it's not
the role of end users to set anti-spam policy (in whole or in part)
any more than it's their role to set firewall policy.  It's not their
job, and they're terrible at it.

---Rsk

---

From: Franck Martin <[email-masked]>
Date: Mon, 15 Dec 2008 09:04:03 +1200 (ANAT)

yes, I think also they are too stupid to choose their government ;) 

slippery slope! 

----- Original Message ----- 
From: "Rich Kulawiec" <[email-masked]> 
To: "Anti-Spam Research Group - IRTF" <[email-masked]> 
Sent: Monday, 15 December, 2008 8:24:30 AM (GMT+1200) Auto-Detected 
Subject: Re: [Asrg] A paper/project worth considering (found it!) 

---Rsk

---

From: "Chris Lewis" <[email-masked]>
Date: Sun, 14 Dec 2008 17:30:24 -0500

> messages* to mailing lists as spam.)

I don't understand this.  I tried to explain this phenomena before.
Didn't you take statistics somewhere?

Your sample is _extremely_ biased.

Let's say for sake of argument, AOL's users have a 5% error rate.  5% of
what they report via TIS isn't spam.  That means, on average, 95 out of
100 reports are accurate and it is spam.

You have a FBL.  But you don't send any spam, right?  You only get your
share of the error rate, and none of the accurate ones - because you
don't send any spam.

So, from your perspective, the TIS button is 100% wrong.  For _you_ it
is.  But it's NOT reflective of TIS hits against a network that sends spam.

> than Comcast's or Yahoo's or any other ISP/mail provider.

Are you contending that Comcast's or Yahoo's FBLs are yielding correct
TIS hits?  Or do you have FBLs with them at all?

> job, and they're terrible at it.

I don't think many would argue against that.  Least of all me or AOL.
Neither Nortel nor AOL lets the users set anti-spam policy.  What the
TIS button does is help highlight situations where the anti-spam filters
aren't working.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: Daniel Feenberg <[email-masked]>
Date: Sun, 14 Dec 2008 18:16:40 -0500 (EST)

> messages* to mailing lists as spam.)

I have to say that this precise sounding figure "100%" comes from using 
an incorrect base. If you sent a million messages and get one incorrect 
spam report, the error rate is .0001%, not 100%. You are getting a 
nonsense number because you are using the wrong base - a common error when 
dealing with percentages.

Dividing by the number of reports rather than the number of messages 
provides no way for AOL to ever have anything other than 100% error rate, 
assuming you do not send spam. Now, if you want to claim AOL spam reports 
are poor evidence, you at least have to tell us the number of good 
messages. Otherwise, the deck is stacked against AOL in a totally unfair 
manner - there is no way for the users to have any error rate other than 
100% (or undefined, if there are no reports).

Daniel Feenberg

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 15 Dec 2008 12:31:02 +0100

Chris Lewis wrote:
> I don't think many would argue against that.

I would, for one.

There is a number of documents about user rights, in categories 
ranging from general human rights, through electronic treatment of 
data, to email specifically. Many of them establish very reasonable 
concepts, some of which are newer than the SMTP model. How are users 
supposed to exercise their rights? When we target anti-spam issues, 
I'd say users may exercise their rights by setting the options in the 
anti-spam policy that are provided for that purpose.

Here's an example: I had a group of users who kept their email 
addresses almost secret, but occasionally my server rejected some 
legit messages sent to them via a blacklisted provider (before I could 
whitelist it.) Thus, they wanted to skip DNSBL checks, and I made it 
optional. What's wrong with that?

In general, the reliance of a reputation system is not an objective 
datum. I'd invite ESPs to let users choose which authorities they wish 
to trust. It is part of the anti-spam policy, which is different from 
the firewall policy: _their_ mail, _my_ server.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: "John Johnson" <[email-masked]>
Date: Mon, 15 Dec 2008 11:08:35 -0600

> the firewall policy: _their_ mail, _my_ server.
   Are you delivering 100% of the email destined for them?  Or are you
treating the e-mail
  as  __my_network_asset__ before delivery?   Other than RBL's, what
other choices do you
  allow your users to tune? SMTP EHLO's ?  Where does being responsive
to your users end,
  and protecting your network begin?  If they choose not to use RBL's,
do they still complain
  about the level of spam? If so, what do you do when they complain?

---

From: Steve Atkins <[email-masked]>
Date: Mon, 15 Dec 2008 09:18:17 -0800

>> I don't think many would argue against that.

I would too. The defining characteristic of objectionable email
is that recipients find it objectionable. ISPs block mail that
recipients find objectionable or (much more rarely) causes
them operational problems.

So, by that definition, recipients drive mail filtering policy.

(anti-spam policy is a very vague and broad term, I'm assuming
Rich is talking about inbound mail filtering).

The fact that competent ISPs can extract extremely useful and
accurate data to drive filtering policy from many data points
where less experienced and informed admins at small outfits
find something wrong with the very, very small and biased
samples of that data they see shouldn't surprise anyone.

That some people are using the tiny, biased samples they
have of the data to drive a broad argument based on the
accuracy of the whole population isn't very surprising either,
but it is a bit of a disappointment that that basic flaw is being
displayed in a research group.

Cheers,
   Steve

---

From: Michael Thomas <[email-masked]>
Date: Mon, 15 Dec 2008 09:45:02 -0800

> So, by that definition, recipients drive mail filtering policy.
And "objectionable" is a superset of "spam". Which is why whines
about users pressing the junk being "incorrect" fundamentally misses
the point: it's not their fault that the receivers of the feedback have
such a procrustean view of what constitutes "objectionable"; the
users are just telling them something they don't want to hear.

The day that we stop thinking in terms of universal truths ("is
objectively spam or not") is the day that we start to make forward
progress from the user's standpoint. Baysian filtering already gets
that, which is why it's so popular even if the sample sizes are tiny.

       Mike

---

From: "Chris Lewis" <[email-masked]>
Date: Mon, 15 Dec 2008 13:59:11 -0500

> So, by that definition, recipients drive mail filtering policy.

I should mention that the conversation that Rich and I were having had a
context that may have been lost or missed along the way - in that we
were talking about corporate, not ISP filters.  Secondly, by "set
policy", I take that to mean not only cross-the-board policy for an
_entire_ receiver population, but _setting_ it.  That is a radically
different thing than "guide", "drive" or "influence" policy for the
entire population or for individual receivers.

> Rich is talking about inbound mail filtering).

Right: corporate, inbound, cross-the-board.

That in no way prevents an ISP (or corporate) choosing to allow users to
set policy for themselves.

Few ISPs or corporates would be crazy enough to allow their users to
diddle the server-level knobs directly.  That is what I think Rich was
speaking to, and what I was.

Because if you do, bad things happen.

> displayed in a research group.

Cuts both ways ;-)

---

From: "J.D. Falk" <[email-masked]>
Date: Mon, 15 Dec 2008 13:32:22 -0700

On 08/12/2008 18:01, "Michael Thomas" <[email-masked]> wrote:

>    approximation of "value".

It's been done.  Not sure if it proved more effective than spam/not-spam
votes, though.

---

From: Michael Thomas <[email-masked]>
Date: Mon, 15 Dec 2008 13:12:53 -0800

> It's been done.  Not sure if it proved more effective than spam/not-spam

Details? This is a pretty open ended subject.

       Mike

---

From: John Levine <[email-masked]>
Date: 15 Dec 2008 21:52:37 -0000

>Details? This is a pretty open ended subject.

There's been papers at CEAS conferences.  See wiki.asrg.sp.am for links
to the conference proceeding pages.

R's,
John

---

From: Steve Atkins <[email-masked]>
Date: Mon, 15 Dec 2008 16:49:09 -0800

> Right: corporate, inbound, cross-the-board.

Ah, OK. I'd crossed my threads, I think.

> speaking to, and what I was.

Many do allow individual recipients to affect mail delivery
decisions, both for that recipient and more generally. Similar to the
way ISPs do. Particularly the ones who are relying on appliances
or services.

> Because if you do, bad things happen.

Yup.

Though less bad things than relying on third parties to set the
policies in some cases. :)

Cheers,
   Steve

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 16 Dec 2008 15:37:41 +0100

John Johnson wrote:
> Alessandro Vesely wrote:
>> In general, the reliance of a reputation system is not an
>> objective datum. I'd invite ESPs to let users choose which
>> authorities they wish to trust. It is part of the anti-spam
>> policy, which is different from the firewall policy: _their_
>> mail, _my_ server.
>  treating the e-mail as  __my_network_asset__ before delivery?

I just deliver mail to recipients. Well, after A-V filtering. Ah, I 
also grab postmaster@, abuse@ and similar addresses.

> Other than RBL's, what other choices do you allow your users to
> tune?

I never meant to give them the root password, if that's what Chris 
means by "allow them to diddle the server-level knobs directly".

What I wanted to say is that we would get better responses from users 
if we were able to put better questions. Bayesian filters are so 
ambiguous that we should be ashamed of proposing them: why would 
people ever use artificial or natural intelligence if some much 
simpler statistical data sufficed? And I don't mention TIS buttons, as 
their ambiguity has been pointed out recently. Doesn't everybody agree 
that we only use such tools because we have nothing better at hands?

> Where does being responsive to your users end, and protecting your 
> network begin?

Hm... I don't work in a large corporate network, so that's possibly 
where I misunderstand what someone else may mean by "policy". AFAICS, 
being responsive and protective are not conflicting tasks. Hence, a 
well planned anti-spam policy should provide for clear-cut user level 
options, if at all possible. Whether setting those options is or is 
not part of a system's anti-spam policy, is a terminological question. 
(And the answer is?)

In particular, reputation systems deserve being at the user level. 
This is rather an observation than an appeal: governments or similar 
authorities in charge of patrolling the territory might have endorsed 
some DNSBLs, say, as part of their job. However, that didn't happen, 
and reputation, as the Credence paper that started this thread holds, 
looks likely to remain based on users' judgment.

---

From: "David Nicol" <[email-masked]>
Date: Tue, 16 Dec 2008 11:25:14 -0600

On Tue, Dec 16, 2008 at 8:37 AM, Alessandro Vesely <[email-masked]> wrote:

> What I wanted to say is that we would get better responses from users if we
> were able to put better questions. Bayesian filters are so ambiguous that we
> should be ashamed of proposing them: why would people ever use artificial or
> natural intelligence if some much simpler statistical data sufficed? And I
> don't mention TIS buttons, as their ambiguity has been pointed out recently.
> Doesn't everybody agree that we only use such tools because we have nothing
> better at hands?

But Paul Graham was /so eloquent/ in his argument for them
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 17 Dec 2008 12:00:17 +0100

> But Paul Graham was /so eloquent/ in his argument for them

Yeah, I guess there is a lesson to be learned: countermeasures don't 
bring any real advancement. To avoid getting stuck in the trenches 
that we built that way, we should apprehend and surmount the system 
weaknesses that spammers exploit. For example, implementing some of 
the requirements of the various privacy laws may be a step toward that 
goal.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: Barry Shein <[email-masked]>
Date: Wed, 17 Dec 2008 16:45:10 -0500

> the requirements of the various privacy laws may be a step toward that 

I'd love to hear from some spammers on some of this, like specifically
how the introduction of certain anti-spam technologies actually
affected their business (as it were.)

That's something really missing in all of this discussion, any input
from (perhaps former) spammers.

All we see is the blind change (mostly positive) in volume and some
change in its character over time.

Something we have almost no input about is how these various spam
technologies affected spammers.

We can guess or extrapolate wildly but my guess is that the view from
the horses' mouths would be surprising and illuminating.

I know, hesitancy of real or former spammers to participate due to
legal or personal safety -- I'd volunteer to help with anonymization
or other concerns...I *KNOW* you know where to find me :-)

-- 
        -Barry Shein

The World              | [email-masked]           | http://www.TheWorld.com
Purveyors to the Trade | Voice: 800-THE-WRLD        | Login: Nationwide
Software Tool & Die    | Public Access Internet     | SINCE 1989     *oo*
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: mathew <[email-masked]>
Date: Wed, 24 Dec 2008 12:52:26 -0600

> from (perhaps former) spammers.

I can't think of many people it would be more pointless to seek  
opinion from, than those who we *know* lie habitually and steal from  
others for a living. Psychotics who don't respond to medication,  
perhaps?

mathew

---

From: Barry Shein <[email-masked]>
Date: Wed, 24 Dec 2008 20:10:37 -0500

> others for a living. Psychotics who don't respond to medication,  

They may be crazy, but they got YOU runnin' on a treadmill, and a
billion others.

After many months of vicious discrediting and back and forth, for some
reason I spent some time on this 15 years ago, I once took a bunch of
holocaust deniers out to dinner.

It was actually quite enlightening.

Like, I found out most didn't believe a word they were saying, one
just had a grudge against some ex's jewish divorce lawyer or similar
nonsense, another got a kick out of getting a rise out of people and
was willing to work hard at it, etc. Some were quite obviously crazy,
and I don't mean that in the schoolyard sense, more in the clinical
sense, crazy, really, barely on the outside.

OTOH I have spoken on the phone to some famous deniers, some were
downright spooky is all I can say. One guy was playing a game where
he'd put out outrageous stuff to bait people, newspapers into a libel
lawsuit and he and a lawyer friend would shake them down for a quick
$100K or thereabouts to settle out of court.

Not everything is what it appears. Actual information is more useful
than armchair bluster.

-- 
        -Barry Shein