Subject: comments on draft 01

From: "Paul Falstad" <[email-masked]>
Date: Thu, 28 Aug 1997 03:46:28 -0700

|      An extension for regular expressions will be written.  While
|      regular expressions are of questionable utility for most users, the
|      programmers writing implementations desperately want regular
|      expressions.

I'd suggest "contains-re" for this..  "matches-re" might be a more obvious
choice, but it implies that the regex is anchored at the beginning
and end of the string.  I can start the regex with ^ and/or end it
with "$" if I want that.

| The following example will fail on any server that  does  not  implement
| the extension known as DWIM.
| 
| Example:    require "dwim";
|             if header ("subject") contains-nocase ("the secret message")
|             {
|                  dwim blurdybloop body;
|             } stop
| 
| OPEN:   I have serious concerns with require; it makes it impossible to
|         separate parsing from evaluation, and introduces some awkward
|         cases.  For instance, a script "if size under 1 { require "foo";
|         do_foo; } else {... }" Even if the test will never happen, this
|         require will prevent the script from working.  Just support
|         seems to make more sense.

It seems like "require" lines should be at the top of the script, and
should be processed as part of parsing.  If the extension is not there,
just forget the whole thing.  If we want to do anything fancier, I think
we'd need to specify explicitly how scripts should "fail" at run-time if
the extension isn't there (log an error?  notify the owner of the
mailbox?  assume "normal"?), and possibly provide some way for the
script to recover from the lack of the extension (perhaps use an alternate
mechanism), but I think this will make things more complex than they need
to be.

A script that works on some incoming messages and fails on others is
useless.  I think the user should not be allowed to install a script that
requires unsupported extensions for some possible inputs; the server
should treat this as a parse error, and preferably notify the user right
away when the script is submitted (if the implementation supports that).

| [OPEN: This section is probably incomplete.  I am not sure that the
| right answer is to make it easy to refuse messages, but secretly keep a
| copy.  Should bounce prevent all other actions from taking affect?]

I think so, since a DSN is an official notification that the message
was not delivered.

| 5.1. all-of
[...]
| 5.2. any-of

is there a reason "and" and "or" were not considered here?  They seem a
lot more readable and easy to use, especially for the novice.

|      If a header listed in  the  header-name-list  argument  exists,  it
|      contains  the  null  key ("").  However, if the named header is not
|      present, it does  not  contain  the  null  key.  So  if  a  message
|      contained the header
| 
|              X-Caffeine: C8H10N4O2
| 
|      these tests on that header evaluate as follows:
| 
|              header ("X-Caffeine") is ("")         => false
|              header ("X-Caffeine") matches ("")    => false
|              header ("X-Caffeine") contains ("")   => true

If a message contains the headers:

  Received: foo
      more foo, on a continuation line
      even more foo
  Received: bar

then what is the string value of header("Received") ?  Does
header ("Received") matches ("*foo*bar*") evaluate to true?

-- 
Paul Falstad                                 Software.com, Inc.
[email-masked]                    805-957-1790 x520
http://www.ttinet.com/pjf/                   http://www.software.com/

---

From: "Paul Falstad" <[email-masked]>
Date: Thu, 28 Aug 1997 04:31:42 -0700

|      This document describes a mail filtering language for filtering
|      messages at time of final delivery.  It is designed to be

We need a language for spam filtering at time of initial acceptance by
the MTA, and we're thinking of using a modified version of sieve to do
this.  Since there's only one filter script per server for us, it's
really an mta configuration issue, and there's no real need for a
standard; but it doesn't make a lot of sense for us to develop a
completely different filtering language, especially since we'll want to
support standard sieve for end-user filtering eventually.

Anyway, here are some of the ideas we had for our version, some of
which might make sense in the standard, or perhaps in an extension:

- tests on the body, like so:

    if body contains "MAKE.MONEY.FAST" ...

- tests on the first few lines of the body:

    if body.top(4) contains "MAKE.MONEY.FAST" ...

  (I think tests on the entire body are best avoided, since it doesn't
  make a lot of sense to do a substring search for "MAKE.MONEY.FAST"
  on a 1-megabyte word document )

- tests on the sender:

    if sender matches "..." ...
    if sender.domain is "lusers.com" ...
    if sender.local-part is "foo" ...

  Of course, the draft specifically says that envelope-matching commands
  are left out intentionally, but they can be useful for spam filtering.
  It's not clear if the "Return-Path:" would get added before or after
  the filter is run; if it's before, then you don't really need these
  extra keywords except for convenience's sake...  (We'd be running
  filters well before it makes sense to add "Return-Path:", assuming
  the mail is even intended for local delivery, so we can't rely on that.)

In case anyone is interested in what other changes we need..  Well, since
we want to run filters before the mail has even been accepted by the
mta, we have a slightly different set of actions.  "bounce" would cause
a failure response code to the SMTP client rather than an actual DSN, and
"fileinto" and "reply" aren't supported.

Also, we need to do tests on the recipient list:

  if recipients contains "[email-masked]" ...
  if recipients.count over 10 ...
  if recipients.count is 1 ...

Of course, tests on the recipient list don't make any sense at
final delivery time, since the recipient list probably isn't available,
and probably shouldn't be visible to the user even if it is..  But
checking for large numbers of recipients can be helpful for weeding out
spam.

From: "Paul Falstad" <[email-masked]>
Date: Thu, 28 Aug 1997 04:31:42 -0700
Reply-To: Paul Falstad <[email-masked]>
To: [email-masked]
Subject: suggestions for spam filtering, etc.
Sender: [email-masked]

|      This document describes a mail filtering language for filtering
|      messages at time of final delivery.  It is designed to be

We need a language for spam filtering at time of initial acceptance by
the MTA, and we're thinking of using a modified version of sieve to do
this.  Since there's only one filter script per server for us, it's
really an mta configuration issue, and there's no real need for a
standard; but it doesn't make a lot of sense for us to develop a
completely different filtering language, especially since we'll want to
support standard sieve for end-user filtering eventually.

Anyway, here are some of the ideas we had for our version, some of
which might make sense in the standard, or perhaps in an extension:

- tests on the body, like so:

    if body contains "MAKE.MONEY.FAST" ...

- tests on the first few lines of the body:

    if body.top(4) contains "MAKE.MONEY.FAST" ...

  (I think tests on the entire body are best avoided, since it doesn't
  make a lot of sense to do a substring search for "MAKE.MONEY.FAST"
  on a 1-megabyte word document )

- tests on the sender:

    if sender matches "..." ...
    if sender.domain is "lusers.com" ...
    if sender.local-part is "foo" ...

  Of course, the draft specifically says that envelope-matching commands
  are left out intentionally, but they can be useful for spam filtering.
  It's not clear if the "Return-Path:" would get added before or after
  the filter is run; if it's before, then you don't really need these
  extra keywords except for convenience's sake...  (We'd be running
  filters well before it makes sense to add "Return-Path:", assuming
  the mail is even intended for local delivery, so we can't rely on that.)

In case anyone is interested in what other changes we need..  Well, since
we want to run filters before the mail has even been accepted by the
mta, we have a slightly different set of actions.  "bounce" would cause
a failure response code to the SMTP client rather than an actual DSN, and
"fileinto" and "reply" aren't supported.

Also, we need to do tests on the recipient list:

  if recipients contains "[email-masked]" ...
  if recipients.count over 10 ...
  if recipients.count is 1 ...

Of course, tests on the recipient list don't make any sense at
final delivery time, since the recipient list probably isn't available,
and probably shouldn't be visible to the user even if it is..  But
checking for large numbers of recipients can be helpful for weeding out
spam.

From: "Paul Falstad" <[email-masked]>
Date: Thu, 28 Aug 1997 03:46:28 -0700
Reply-To: Paul Falstad <[email-masked]>
To: [email-masked]
Subject: comments on draft 01
Sender: [email-masked]

| 5.1. all-of
[...]
| 5.2. any-of

  Received: foo