Subject: question regarding portability

From: "Jack De Winter" <[email-masked]>
Date: Mon, 09 Feb 1998 14:13:58 -0500

was going through the draft again on the weekend and
came up with some interesting points.

In 4.2, there is a specification for FileInto, and
it shows that "INBOX.harassment" is the folder.

1) What should happen if "INBOX.harassment" does not exist?
2) Is it worth introducing an extension to test if "INBOX.harassment"
   exists?
3) Is there any way we can allow for a folder to be created? Or is that
   beyond scope? (i.e. fileinto if exists, create then fileinto if not)
4) Do we want to get into the nasty IMAP folder seperator character
   argument here?  Or is there some way to make the fileinto portable?

The main point for me is #4.  If I want to write a portable script,
something like #4 blow it up.  #3 if interesting for cases where you
want a generic script.

regards,
Jack
-------------------------------------------------
Jack De Winter - Wildbear Consulting, Inc.
(519) 884-4498		http://www.wildbear.on.ca/jacks/

---

From: Tim Showalter <[email-masked]>
Date: 9 Feb 1998 14:37:30 -0500

> Date: Mon, 09 Feb 1998 14:13:58 -0500
> From: "Jack De Winter" <[email-masked]>

> 1) What should happen if "INBOX.harassment" does not exist?

The script shouldn't run, in my opinion.  This can be checked for before
running the script.

Of course, this means I should change the draft, too.

> 2) Is it worth introducing an extension to test if "INBOX.harassment"

I don't think so.

>    beyond scope? (i.e. fileinto if exists, create then fileinto if not)

It's not beyond scope, but I'm against it.  It's not obvious, it could
create a mailbox that the user doesn't know about in the case of a typo,
and if anyone ever introduces anything that allows a script to munge a
string, it could get ugly.

>    argument here?  Or is there some way to make the fileinto portable?

Who says we're limited to IMAP?  This isn't just a hierarchy seperator
argument.  This depends on the naming scheme for folders in a particular
implementation.

I don't see a lot of utility in doing something "smarter", but I could be
wrong.

-- 
                                          Tim Showalter [email-masked]

---

From: Lyndon Nerenberg <[email-masked]>
Date: Tue, 10 Feb 1998 11:16:24 -0700 (MST)

> The script shouldn't run, in my opinion.  This can be checked for before

But you still have to deal with the case where the folder goes away 
after the script has been installed. If this happens the message should 
file into the default mailbox. (Possibly with some sort of diagnostic?)

--lyndon

---

From: Tim Showalter <[email-masked]>
Date: 10 Feb 1998 17:26:14 -0500

> From: Lyndon Nerenberg <[email-masked]>
> 

This can still be checked before running the script; just compile a list of
things that have to be true (mailbox is under quota, all mentioned
mailboxes still exist); if it's not true, don't filter the message and
leave it in the queue.

As long as I can mail the user telling them their script is bogus, I guess
I don't care.  I feel very strongly about giving good diagnostics to
users.  (The old FLAMES system doesn't do this and it makes debugging a
nontrivial script a real pain.)

---

From: Lyndon Nerenberg <[email-masked]>
Date: Tue, 10 Feb 1998 15:46:44 -0700 (MST)

On 10 Feb 1998 17:26:14 -0500 Tim Showalter <[email-masked]> 
wrote:

> mailboxes still exist); if it's not true, don't filter the message 
and > leave it in the queue.

This has the potential for becoming an expensive operation on large or 
complex scripts. I'm not sure that I want to incur the overhead of a 
full parse on every delivery if only 5% of the code path is executed 
95% of the time. You also open a system DOS attack. Someone could 
subscribe to many high-volume mailing lists, then remove the folder 
from a working script. This would start backlogging the system mail 
queue, potentially delaying mail to other users.

If the target missing mailbox is shared with another user, and that 
other  user deletes the mailbox, should this cause mail to the first 
user to be stalled? It would happen in this scenario, and that seems a 
bit drastic to me.

> As long as I can mail the user telling them their script is bogus, I 
guess > I don't care. 

Presumably the user won't see that message, since you're queueing all 
his incoming mail due to the missing mailbox error. 

--lyndon

---

From: Tim Showalter <[email-masked]>
Date: 10 Feb 1998 18:23:21 -0500

> > This can still be checked before running the script; just compile a
> > list of things that have to be true (mailbox is under quota, all
> > mentioned mailboxes still exist); if it's not true, don't filter the
> > message and leave it in the queue.
> 
> 95% of the time.

You don't need to do a full parse.  You need to keep a list of preconditions
that have to be true before you can run the script.

In the case of mailbox lookups, this could be a comparatively expensive
operation, but the mail store should be relatively intelligent about such
things.

> You also open a system DOS attack. Someone could 

Any mailbomb would have basically the same effect.  Ironically, if you did
the same mass-subscribe and don't delete the mailbox, it's worse -- you
still have to ensure the same validation steps, but you have to write all
the messages out, consuming even more I/O bandwidth.

Even if you just file into INBOX if a given mailbox can't be found, you
still lose.

> user to be stalled? It would happen in this scenario, and that seems a 

Even if the number of messages being cross-filed in this way is very large?
It's beneficial to back up in this case.

What happens if we just temporarily fail messages that get filed into a
nonexistant mailbox?

> > guess I don't care. 

Delivery failure messages should be special, and probably shouldn't go back
out to the SMTP server.

I'm not strongly opposed to filing into INBOX on error, but there are cases
to be made either way.

---

From: Lyndon Nerenberg <[email-masked]>
Date: Wed, 11 Feb 1998 10:15:57 -0700 (MST)

[ I've changed the subject to reflect the discussion a bit more 
accurately. ]

> operation, but the mail store should be relatively intelligent about such

If the script references remote folders it could be a *very* expensive 
operation. I think the cost far outweighs the benefit. (What do you do
when a remote folder verification fails because the server is down?)

> Any mailbomb would have basically the same effect.

A mailbomb might put the *user* overquota, if delivery fell back to 
INBOX or some such in the case of a SIEVE error. By holding all mail 
for the user in the system queue in the case of a SIEVE error you can
deny service to the *entire* system. This is unacceptable.

>  Ironically, if you did

But then you're doing what you're supposed to be doing :-)  (And not 
denying service to other users.)

> Even if you just file into INBOX if a given mailbox can't be found, you

How? The mail is still received. Having it in the default mailbox is an
annoyance. Having other users mail get rejected because you plugged the 
system queue is a lose.

> It's beneficial to back up in this case.

A backup won't make any difference. In a shared folder environment, 
user B deleting user A's shared folder is not a bug. It's explicitly 
allowed for in the sharing model. SIEVE scripts should not barf 
arbitrarily as a result of a legitimate action.

> What happens if we just temporarily fail messages that get filed into
> a nonexistant mailbox?

This seems like a reasonable path to follow. I also like Chris 
Bartram's idea of having an "errors" folder that would override the 
default folder.

> Delivery failure messages should be special, and probably shouldn't go back

You cannot count on this being possible.

--lyndon

---

From: Tim Showalter <[email-masked]>
Date: 11 Feb 1998 17:01:14 -0500

> Date: Wed, 11 Feb 1998 10:15:57 -0700 (MST)
> 

I guess I could live with this.  (I don't agree with all of your points,
but this is probably cleaner anyway.)

So in the case of run-time error, the script just files into INBOX (or the
error mailbox), noting that there was a run-time error in a seperate
message.

> You cannot count on this being possible.

Why not?  I think a Sieve implementation running at time of final delivery
has the ability to insert a message into the mail store.*

*deliberate bait

---

From: Lyndon Nerenberg <[email-masked]>
Date: Thu, 12 Feb 1998 09:59:45 -0700 (MST)

> has the ability to insert a message into the mail store.*

What if the default mailbox is on another server?

--lyndon

---

From: Tim Showalter <[email-masked]>
Date: 12 Feb 1998 16:27:38 -0500

> > > > Delivery failure messages should be special, and probably shouldn't
> > > > go back out to the SMTP server.
> > > 

If the time of final delivery for a system is such that the default mailbox
may be on a server different from the one where the message is being
processed, the server is going to have to have the ability to deposit mail
into an arbitrary mailbox, even if the server's down, even if the mailbox
is on another server, then the system is going to have to be able to deal
with it.

If this isn't true, how is the filtering agent going to write to the
mailboxes in the first place?

---

From: Lyndon Nerenberg <[email-masked]>
Date: Thu, 12 Feb 1998 15:39:58 -0700 (MST)

>From the start I've considered the "mailbox" to be an opaque label. If 
a server decides to interpret mailbox names as IMAP URLs (say), SIEVE 
shouldn't care. While I agree that there are silly ways to abuse such a 
facility, as long as it's allowed you can't count on a folder being 
local (short of mandating that in the spec).

--lyndon

---

From: Tim Showalter <[email-masked]>
Date: 12 Feb 1998 17:52:24 -0500

> Cc: MTA Filters List <[email-masked]>
> Date: Thu, 12 Feb 1998 15:39:58 -0700 (MST)
> 

I'm not saying folders are local.  I'm saying the filtering agent has to be
able to write to them in some useful way, and that the filtering agent is
going to have to deal with "mailbox is unavailble" errors.

This doesn't change the fact that the filtering agent has to be able to
write a message to the message store.  I guess I shouldn't say "write" --
it has to have the ability to add a message to the message store without
going through SMTP again.

---

From: Lyndon Nerenberg <[email-masked]>
Date: Thu, 12 Feb 1998 16:28:31 -0700 (MST)

> going through SMTP again.

Aha. It turns out we're in violent agreement.

--lyndon