Subject: spamtestbis

From: Cyrus Daboo <[email-masked]>
Date: Thu, 03 Feb 2005 16:14:34 -0500

FYI the draft for the new version of the spamtest extension is now 
available. The only change between this and the current RFC is the addition 
of a :percent argument to the spamtest test to allow for a numeric range of 
0 - 100. The value -1 is used to indicate a message that was not 
categorised in anyway.

-- 
Cyrus Daboo

---

From: [email-masked]
Date: Thu, 03 Feb 2005 14:08:56 -0800 (PST)

> 0 - 100. The value -1 is used to indicate a message that was not

First, I don't think the -1 is a good idea because of how it interacts with
numeric comparators. I'd much rather retain 0 as the value indicating no test
was done. 

The only other issue I see with the draft is the business of making require
"spamtest" and require "spamtestpercent" mutually exclusive. I certainly
agree that we want require "spamtestpercent" to subsume require "spamtest" -
- require lists in sieve can be really long - but I wonder if making
them mutually exclusive goes too far.

Additionally, one thing that isn't entirely clear with the current text is that
the following is legal:

   require "spamtest"; if spamtest :contains "foo" ...

That is, spamtest without a :percent is allowed when require "spamtestpercent"
is used. I think this should be clarified.

				Ned

---

From: "Nigel Swinson" <[email-masked]>
Date: Fri, 4 Feb 2005 13:01:38 -0000

> > 0 - 100. The value -1 is used to indicate a message that was not

I'm surprised we need a percentage spam score... I would have thought 10 levels was enough.  Indeed in your example you use level 30(%) = 3.
 
> numeric comparators. I'd much rather retain 0 as the value indicating no test

I agree.  I think many existing scripts will test for 0 as a way of finding out if the message was scanned, so why risk breaking them?  Personally I don't mind if "message is clear of spam" = 1%.  It's not really an exact science anyway...

Nigel

---

From: [email-masked]
Date: Fri, 04 Feb 2005 10:41:03 -0800 (PST)

> I'm surprised we need a percentage spam score... I would have thought 10
> levels was enough.  Indeed in your example you use level 30(%) = 3.

FWIW, I personally don't see any need for more levels, but I have seen
some user requests for this and I understand others have seen considerable
demand for it.

> I agree.  I think many existing scripts will test for 0 as a way of finding
> out if the message was scanned, so why risk breaking them?  Personally I don't
> mind if "message is clear of spam" = 1%.  It's not really an exact science
> anyway...

Yeah, "the exact science of spam identification" is indeed an oxymoron. Its
the reason why 10 levels is enough for me at least.

				Ned

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 04 Feb 2005 14:06:56 -0500

Hi [email-masked],

--On February 3, 2005 2:08:56 PM -0800 [email-masked] wrote:

>> available. The only change between this and the current RFC is the
>> addition of a :percent argument to the spamtest test to allow for a
>> numeric range of 0 - 100. The value -1 is used to indicate a message
>> that was not categorised in anyway.
>
> First, I don't think the -1 is a good idea because of how it interacts
> with numeric comparators. I'd much rather retain 0 as the value
> indicating no test was done.

So we would have:

    spamtest    interpretation
    value

   		 0          message was not tested for spam
   		 1          message was tested and is clear of spam
   	2 - 99     message was tested and has a varying likelihood of
   		            containing spam in increasing order
   		 100        message was tested and definitely contains spam

Perhaps those that requested the percent range could state whether this is 
acceptable.

> The only other issue I see with the draft is the business of making
> require "spamtest" and require "spamtestpercent" mutually exclusive. I
> certainly agree that we want require "spamtestpercent" to subsume require
> "spamtest" - - require lists in sieve can be really long - but I wonder
> if making them mutually exclusive goes too far.

I can change the 'MUST NOT' to a 'SHOULD NOT' for that requirement.

> Additionally, one thing that isn't entirely clear with the current text
> is that the following is legal:
>
> That is, spamtest without a :percent is allowed when require
> "spamtestpercent" is used. I think this should be clarified.
>
>

Here is some alternative text for this:

Old:

   SIEVE implementations that implement the "spamtest" test have an
   identifier of "spamtest" for use with the capability mechanism.  If
   the ":percent" argument is used with any spamtest test, then the
   capability idenitifier "spamtestpercent" MUST be present, and the
   "spamtest" capability MUST NOT be present.

New:

   SIEVE implementations that implement the "spamtest" test use an
   identifier of either "spamtest" or "spamtestpercent" for use with
   the capability mechanism.

   If the ":percent" argument is not used with any spamtest test, then
   one of either the "spamtest" or "spamtestpercent" capability
   identifiers MUST be present, though both SHOULD NOT be present together.

   If the ":percent" argument is used with any spamtest test, then
   the "spamtestpercent" capability identifier MUST be present, and
   the "spamtest" capability identitifer SHOULD NOT be present.

-- 
Cyrus Daboo

---

From: [email-masked]
Date: Fri, 04 Feb 2005 11:19:16 -0800 (PST)

>    		 100        message was tested and definitely contains spam

Yes, I think this is what we want.

>    the "spamtest" capability identitifer SHOULD NOT be present.

Much better. I'm happy with this.

				Ned

---

From: "Mark E. Mallett" <[email-masked]>
Date: Sat, 5 Feb 2005 23:52:56 -0500

On Thu, Feb 03, 2005 at 04:14:34PM -0500, Cyrus Daboo wrote:
> available.

Picky comments from mem..

I have a hard time reading this document, mainly because of phrases
that include "returning strings" and overloading of some words to
mean different things.  Perhaps it's just the way my brain works,
or doesn't work.  Specifically:

1.  Introduction and Overview

   > The purpose of this document is to introduce two SIEVE tests that can
   > be used to implement 'generic' tests for spam and viruses in messages
   > processed via SIEVE scripts.  These tests return a string containing
   > a range of numeric values that indicate the severity of spam or
   > viruses in a message, or a string that indicates the message has not
   > passed through any spam or virus checking tools.

When you talk about SIEVE tests, it reads as though you are talking
about SIEVE language elements.  Those elements won't really do those
things, it's the underlying filter implementation that would interpret
the result of spam or virus analysis and produce, conceptually, a string
(or strings) that can be used by those SIEVE language elements.  (The
next sentence says as much.)  Also, that conceptual string probably
contains "a number that lies within a range of numeric values" -- the
string would not contain "a range of numeric values."

3.1  General Considerations

   > The "spamtest" and "virustest" tests described below both return a
   > string that starts with a numeric value, followed by an optional
   > space (%x20) character and optional arbitrary text.

Here's where the trouble really lies.  This pretty clearly is talking
about SIEVE language statements -- and if it isn't, then the use of
those words is even more of a problem.  Those statements do not return a
string; they act upon the string that was returned by the underlying
tests.  (Or, rather, they act as if they did that: no string really has
to be present.)

   > The additional text can be used to carry implementation
   > specific details about the tests performed and descriptive comments
   > about the result.  Tests can be done using standard string
   > comparators against this text if it helps to refine behaviour,

Does "this text" refer to "The additional text" or to the entire string?
The English grammar implies the former, but text in the introduction
implies the latter.  For example, given a underlying spam test result
of:

   "4 too many recipients"

how does:

    if spamtest :is "too many recipients"  { ... }

evaluate?  IMHO testing against the "additional text" would be the most
useable.

   > The "spamtest" test evaluates to true if the spamtest result matches
   > the value.  The type of match is specified by the optional match

Another example where the word "spamtest" means two different things:
the SIEVE statement and the underlying test that produces a string for
use by that statement.  I'd recommend finding different ways to
refer to the SIEVE statement and the underlying analysis of spam and
virus results.

[ Imagine similar comments about similar sections ... ]

BTW I think this is the kind of extension that could be solved with
better fundamental language elements and/or better coordination between
language extensions.

For example, one could build on the proposed "variables" extension by
imagining a new set of read-only variables within specific namespaces
(per the old namespace idea, a la dates and timezones).  Imagine:

   require "variables";
   require [ "relational", "comparator-i;ascii-numeric" ];

   set "spamval" "${spamtest::value}"
   set "spampct" "${spamtest::pctvalue}"
   set "spamtext" "${spamtest::text}"

   if string :value "ge"
             :comparator "i;ascii-numeric"
	     "${spamval}" "3" { ... }
   if string :value "ge"
             :comparator "i;ascii-numeric"
             "${spamtest::pctvalue}" "30" { ... }

Time travel might be required though.

mm

---

From: "Nigel Swinson" <[email-masked]>
Date: Mon, 7 Feb 2005 14:19:47 -0000

> Time travel might be required though.

Or better still why not just refer to the variables direct in the string test?

    if string :value "ge"
              :comparator "i;ascii-numeric"
              "${spamtest.value}" "3" { ... }
    if string :value "ge"
              :comparator "i;ascii-numeric"
              "${spamtest.pctvalue}" "30" { ... }

This is certainly on a similar tack to my post relating to associative arrays and sms notification, so unsurprisingly I would really like to see something along these lines, but I'm not sure how we get there.  Indeed  I think it's this kind of thing that gives the variables extension a large amount of it's value.  The variables spec does make reference to:

   Namespaces are meant for future extensions which make internal state
   available through variables.  These variables SHOULD be put in a
   namespace with the same name as its capability string.  Notice that
   the user can not specify a namespace when setting variables with SET.

I wondered if we would therefore say things like if you require "spamtest", then a namespace called spamtest would be defined containing variables called value and text which contain etc.  It then seems to make sense to specify in the spamtest spec what variables it would define.  Do we have time to add such a section to spamtestbis?

One problem here is that it costs resources to obtain the spam score and virus score, and part of the reason you might be using sieve would be to refine which messages would result in those resources being used up.  So you might still want to somehow specify or recommend that the implementation make the variables in the namespace available in a lazy fashion.

Nigel

---

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 8 Feb 2005 15:26:30 -0500

>               "${spamtest.value}" "3" { ... }

My second example quoted above had that (but with improper namespace
syntax-- somehow I thought that namespaces had been removed and
completely glossed over them in the latest variables draft).

> One problem here is that it costs resources to obtain the spam score
> and virus score, and part of the reason you might be using sieve would
> be to refine which messages would result in those resources being used
> up.  So you might still want to somehow specify or recommend that the
> implementation make the variables in the namespace available in a lazy
> fashion.

Indeed, that's what I had in mind as well in that particular example.

mm

---

From: Matthew Elvey <[email-masked]>
Date: Sat, 12 Feb 2005 10:42:20 -0800

>some user requests for this and I understand others have seen considerable
Any of it compelling?  Sometimes, there's considerable demand for bad 
ideas (e.g. to accept spam from open relays, e.g. from an  EFF founder - 
http://www.toad.com/gnu/).  I thought we discussed this and decided 10 
was enough levels, but I find no mention of it in the archives.

---

From: [email-masked]
Date: Sat, 12 Feb 2005 12:45:42 -0800 (PST)

> Any of it compelling?

The one I find somewhat compelling is the desire to convert to existing uses
and cutoff values for SpamAssassin to sieve and spamtest. SpamAssassin uses
real numbered scores and the obvious mapping to a 1-10 scale is a bit too
grainy. A 1-100 scale, OTOH, is more than sufficient.

> Sometimes, there's considerable demand for bad

This is a long way from being a bad idea.

				Ned

---

From: "Mark E. Mallett" <[email-masked]>
Date: Sat, 12 Feb 2005 16:38:21 -0500

> grainy. A 1-100 scale, OTOH, is more than sufficient.

I'm kind of on the middle on this one.  In general I like to avoid
hardwiring runtime elements into the language, preferring instead to
give options to the script writers.  You just can't always predict what
people want, so trying not to guess is best.  My ideal would be to say
"the spam test returns a value of this type" where "type" could include
a fundamental data type (e.g. an integer, a string, a floating point
number) as well as a range (e.g. integer between 6 and 10, a string
containing one of a number of keywords, and so forth).  Much of that
doesn't fit here, of course.

But indeed I have heard from people who want far greater ranges than
0 to 10.  To quote one anonymously, specifically about spam scores:

    You can't easily use those results from procmail, as it can't do
    numeric comparisons like 'if ( ( prob >= 99.0 ) && ( prob < 99.9 ) )
    ...' to decide where to file; you need to write them in terms of
    pattern matching which gets messy.

So there's at least one person calling for a range of 0 to 1000!

There are other analyzers out there that return a floating point spam
score, although they tend to have their own ability to emit a summary
header based on a threshold.

Personally I'd try to find a way to say "map the spam test into a range
from m to n" rather than hardwiring one or two ranges in advance.  There
would be no problem in having a default (e.g. m=0, n=10).

Yours,
-mm-