Subject: Re: [ietf-dkim] Bayesian filters are the pits

From: Scott Kitterman <[email-masked]>
Date: Tue, 22 Aug 2006 16:26:33 -0400

On Tuesday 22 August 2006 15:56, Hallam-Baker, Phillip wrote:
> ... we need to promote the idea that you should not look for the
> existence or even the validity of a DKIM header as being as important as
> the domain that is claiming responsibility. If you can't correlate the
> domain to some form of additional information you should ignore the record
> entirely.

And I would argue that SSP is a first attempt to provide some of the 
additional information.

What we need is more determinism and fewer heuristics.

If we can get an SSP is adequately expressive, I want to be able to reject 
messages after DATA if the fall outside the scope of the defined sender 
policy.  If the message is real, the sender will get a rejection notification 
and they can try an alternate means of communication.  If it's forged spam, 
no one is bothered by backscatter.  This a path away from messages 
disappearing forever into never reviewed spam folders.

I also want the SSP to work for as broad a set of senders as it can reasonably 
accommodate.  If it just works for the large senders, we will not, in my 
opinion, have done the job we were chartered to do.  Scalability is both up 
and down.

Scott K
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Douglas Otis <[email-masked]>
Date: Tue, 22 Aug 2006 13:57:36 -0700

On Aug 22, 2006, at 12:56 PM, Hallam-Baker, Phillip wrote:

> Third we need to promote the idea that you should not look for the  
> existence or even the validity of a DKIM header as being as  
> important as the domain that is claiming responsibility. If you  
> can't correlate the domain to some form of additional information  
> you should ignore the record entirely.

It is hard to understand what is meant by this suggestion.  One might  
assume that a DKIM signature makes such a claim on its own.  Of  
course this claim has limitations.   Are you suggesting that there  
also be a means to associate the DKIM header with that of the SMTP  
client?

-Doug

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 22 Aug 2006 12:56:26 -0700

I have been looking through some of the responses from the spamming community to SPF. I conclude that the real problem here is that people using naive Bayesian filtering don't have a clue. 
 
The problem with the Bayesisan approach is that it is very vulnerable to counter-programming by spammers. So when SPF started to gain traction the spammers realized that they could deter adoption of SPF by simply introducing SPF data into their systems. After a short while the naive Bayesian schemes would quickly generate a large negative score for having SPF data present. 
 
The solution to this problem is mostly marketting communications rather than technical.

First we need to get across the fact that spam filtering companies do not in general use the naive Bayesian filtering approaches popularized by Paul Graham and promoted by the conference at MIT. 
 
Second we need a simple fix to deter the 'jamming' attack by spammers. The simple fix here is to simply have a rule that says that certain featues can never result in negative scores. SPF/Sender-ID and DKIM should be amongst them. 
 
Third we need to promote the idea that you should not look for the existence or even the validity of a DKIM header as being as important as the domain that is claiming responsibility. If you can't correlate the domain to some form of additional information you should ignore the record entirely.

---

From: "J.D. Falk" <[email-masked]>
Date: Tue, 22 Aug 2006 16:40:43 -0700

On 2006-08-22 12:56, Hallam-Baker, Phillip wrote:

> domain to some form of additional information you should ignore the 
> record entirely.

That's generally true in a simplistic spam / not spam decision.  If 
you're making a forged / not forged decision, the record is still useful.

This has nothing to do with naive Bayes, but everything to do with naive 
mail administrators looking for simple binary spam / not spam criteria.

-- 
J.D. Falk, Anti-Spam Product Manager
Yahoo! Communications Platform Team
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html