Subject: RE: [ietf-dkim] Bayesian filters are the pits

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Wed, 23 Aug 2006 12:27:43 -0700

Just wanted to verify a line of thinking I was planning to include in the overview doc. We have to give some indication to verifiers on how to use the information. 

I think that it is important to point out that you are not going to get good results from naïve learning schemes and make sure that there is sufficient coverage there to defend against the objection.

> -----Original Message-----
> From: Jim Fenton [mailto:[email-masked]] 
> Sent: Wednesday, August 23, 2006 3:12 PM
> To: Hallam-Baker, Phillip
> Cc: J.D. Falk; [email-masked]
> Subject: Re: [ietf-dkim] Bayesian filters are the pits
> 
> While an interesting topic, this seems only peripherally 
> related to DKIM and I wonder whether a different venue, such 
> as the ASRG list, would be more appropriate.
> 
> -Jim
> 
> Hallam-Baker, Phillip wrote:
> > I think that I am working towards a rather broader critique 
> of the way that SpamBayes &ct. are applied.
> >
> > Naïve Bayesian learning schemes are intrinsically 
> vulnerable to counter-programming. They work on a small scale 
> only because there is not a sufficient value to counter-programming.
> >   
> > ...
> >   
> 
> 

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: John Levine <[email-masked]>
Date: 24 Aug 2006 03:03:45 -0000

>I think that it is important to point out that you are not going to
>get good results from naïve learning schemes and make sure that there
>is sufficient coverage there to defend against the objection.

It seems to me the important point is that bad guys can and will
implement any technical spec that anyone can think up.

This means that it is foolish to assume that the mere presence of a
DKIM signature or any other technical detail means anything good about
a message.  This is true whether it is people or software making the
assumption.

R's,
John

---

From: "J.D. Falk" <[email-masked]>
Date: Thu, 24 Aug 2006 13:44:22 -0700

> a message.  This is true whether it is people or software making the

If anyone can turn this into a short (but still accurate) slogan, I'll 
arrange for a small run of t-shirts.

-- 
J.D. Falk, Anti-Spam Product Manager
Yahoo! Communications Platform Team
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "Hector Santos" <[email-masked]>
Date: Thu, 24 Aug 2006 17:24:57 -0400

----- Original Message -----
From: "J.D. Falk" <[email-masked]>
To: <[email-masked]>

> If anyone can turn this into a short (but still accurate) slogan,
> I'll arrange for a small run of t-shirts.

Yeah, about 2-3 t-shirts is all that is needed. <g>

Having experience in AI, diagnostic and expert systems for a large range of
applications from DoD, ARPA, Robotics, Nuclear, and in the business world,
it is often best applied in monitoring and detecting failure; deviations
from the physics, emperical, including expert knowledge.

If you expect something to behave a certain way, the deviation from that
expectation has significant value.  Most control systems work this way.

But with the low tech world of email, DKIM and SSP, we are mixing apples and
oranges.

Mail Content analysis or the true intent of the mail author has nothing to
with the mechanical protocols in question.  The proposed DKIM-BASE protocol
has an expected mode of operation.  The deviation from that expectation is
what is being studied. That is what SSP is about.

It is certainly feasible for the negative results and process entries from
SSP and/or DKIM-BASE validation to be feed into a neural net model for
example.

The arguments here by some to be that the deviations from expected behavior
has no significance.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

---

From: Dave Crocker <[email-masked]>
Date: Thu, 24 Aug 2006 15:05:02 -0700

> arrange for a small run of t-shirts.

      DKIM:

      Signature == Good/Bad

      Signature + Assessment == Good

d/
-- 

  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: <[email-masked]>
Date: Thu, 24 Aug 2006 18:04:03 -0400

DKIM tells you where the mail is from, != whether you want the mail

---

From: "Hector Santos" <[email-masked]>
Date: Fri, 25 Aug 2006 03:38:42 -0400

----- Original Message -----
From: "Dave Crocker" <[email-masked]>
To: "J.D. Falk" <[email-masked]>

>       Signature + Assessment == Good

Dave,  I had to be anal on this, but unless you are saying DKIM IS NOT the
above, I think might want to rethink this one :-)

If you want the t-shirt to say the above, it implies "assessment"
disseminates the bad from the total population of good and bad which is what
I've been advocating with SSP.

Proof:

    signature = {good, bad}
    signature + assessment = {good}

Resolving for assessment:

    assessment = {good} - signature
    assessment = {good} - {good,bad}
    assessment = {good} - {good} - {bad}
    assessment = -{bad}

which implies that your idea for Assessment focuses on subtracting the bad
from the total population of good and bad, and what remains is the good. <g>

So unless you have a database of the bad, you won't know what is bad in
order to filter it.  You are left with a deterministic method of detecting
the bad which is what SSP is about.

If you want your assessment to focus on looking for the good, then your
model implies the "Bad is the anti-good" or "Good is the opposite of bad" or
"Good is the negation of the bad" or modeled as:

    {good} = -{bad}   <--- Good is the anti-bad!

Proof:

    signature = {good, bad}
    signature + assessment = {good}

Assume:

    Assessment = {good}   <--- Database of the good!

Substitute:

    {good, bad} + {good}  = {good}

Therefore, in order for this to be correct, you must have:

    {good} = -{bad}   <--- Good is the anti-bad!

Which says that if you are not in the GOOD database, then you must be a BAD
person!

Of course, I don't believe that is not really what you seek. So
alternatively, to make any sense of it, it could be modeled as such:

    {good, bad} + {good}  = {good*2, bad}

which says that Assessment is using a database of the good in order to add
weight to the population of what is acceptable.

The problem?

You still are passing on the {bad} population.  Your idea for assessment has
not done anything to address the bad.

Hence, you can view this as a simple neural net:

Weight of 1.5 you have a AND gate:

    {good, bad} AND {good}  = {good}

Weight of .5 you have an OR gate:

    {good, bad} OR {good}  = {good*2, bad}

Which of course, implies that the more information you have on the good, you
can use an AND condition to eliminate the noise (bad).   But with less
information, you would need to use an OR condition to order to help built up
the weights.