Subject: [sieve] About editheader

From: Stephan Bosch <[email-masked]>
Date: Wed, 26 Oct 2011 15:44:17 +0200

Hi,

I've recently started implementing the editheader extension and based on 
the specification (RFC 5293) I have a few questions.

(1)

This is the first extension to introduce an action command  with match 
arguments (match type and comparator), apart perhaps from the "denotify" 
command of the long-deprecated notify extension. Some match types such 
as ":matches" introduce the side-effect of setting match variables when 
the variables extension is active (RFC 5229; Section 3.2). So, my 
question is: can deleteheader action of the editheader extension affect 
match variables or is that supposed to be disabled much like the body test?

Intuitively, I would think it is supposed to be disabled, i.e. I feel no 
action command should influence match variables. Also, it is not really 
possible to short-circuit the 'test' as required in RFC 5229 Section 
3.2, because all matching headers must be deleted and thus matching 
continues.

What do you think?

(2)

If I understand the specification correctly, users can use the 
editheader extension to (accidentally) create invalid IMAIL (now RFC 
5322, was RFC 2822) messages. Some header fields defined in the IMAIL 
specification have a restricted number of occurrences and some have a 
specific well-defined structure. The editheader specification does not 
require, recommend, nor mention keeping the message a valid RFC2822 
message. The only syntactic restriction I see is the requirement that 
the field value complies with the "unstructured" nonterminal syntax from 
IMAIL. It is allowed to silently refuse certain modifications (e.g. 
deleting Received headers), but I would have expected some 
recommendations in this respect. For example, is it wise to allow adding 
a second 'Subject:' header, or to allow setting the 'Date:' header to 
some syntactically incorrect nonsense? More complex examples would 
include the requirement for a 'Sender:' header when more than one 
'From:' mailbox is specified.

Syntax problems can be handled the same as deleting a restricted header 
(silently ignore), however, enforcing the uniqueness (and presence) of 
certain headers is a bit more complicated. Bluntly refusing addheader 
when the header already exists makes modifying such headers impossible, 
especially when deleting the header first would not be allowed (e.g. for 
'Date:'; it is unique AND required).  Implicitly deleting the previous 
occurrence of such headers during addheader would be an option, but is 
that a good idea?

Or am I just exaggerating this problem?

Regards,

Stephan.

---

From: Philip Guenther <[email-masked]>
Date: Wed, 26 Oct 2011 19:09:53 -0700

On Wed, 26 Oct 2011, Stephan Bosch wrote:
> I've recently started implementing the editheader extension and based on the
> specification (RFC 5293) I have a few questions.
> 
> (1)
> 
...

It should affect match variables.  The variables extension, when it 
describes the side-effect of the :matches match-type, places no 
restriction on it only affecting its use in tests and not in actions.

As co-author of the RFC I can confirm that it was my intent that they do 
so and that the original implementation, as written by my co-author Jutta, 
certainly does set match variables.

Note that the body test explictly calls out that variables aren't set, 
while no such statement appears in this RFC.

> 'From:' mailbox is specified.

I think this is completely up to local policy.  There are implementations 
that operate in contexts where the 'message' that is being operated on is 
not required to meet all the requirements of RFC 5322.

For a similar scenario, the IMAP standard's APPEND command merely says 
that an appended message "SHOULD be in the format of an [RFC-2822] 
message" and even provides an example of why a message might legitimately 
fail to be in that form later:
           Note: There MAY be exceptions, e.g., draft messages, in
           which required [RFC-2822] header lines are omitted in
           the message literal argument to APPEND.  The full
           implications of doing so MUST be understood and
           carefully weighed.

(IMO, using the RFC 2119 keywords "MAY" and "MUST" in that note was 
pointless.  MUST be understood by whom?  And how does 'understanding the 
implications' affect interoperability?)

Personally, I'm a liberal on this: I think an implementation should let 
you do whatever you want in the script and only when the message goes 
'out' (via a fileinto, redirect, the implicit keep, etc) should the format 
be checked in any way.  The message format standard isn't a straight 
jacket, but rather a statement about what others are expected to accept 
and how they should interpret that.  If you send something outside of that 
spec, you've gone outside the bounds of the standard and left your message 
open to rejection, modification, and misinterpretation...unless you have 
an agreement beyond the base standard.

> Or am I just exaggerating this problem?

In my opinion, yes.

Philip Guenther

---

From: [email-masked]
Date: Wed, 26 Oct 2011 20:18:37 -0700 (PDT)

> match variables or is that supposed to be disabled much like the body test?

I don't see why not. Our implementation certainly does.

> action command should influence match variables.

My intuitation says exactly the opposite. And more to the point, the
specifications seem very clear that :matches combined with variables sets match
variables. It doesn't say only :matches in a test combined with variables set
match variables AFAIK.

> Also, it is not really

Short-circuiting simply means you're supposed to stop as soon as possible and
retain the last match. It's a moot statement when there's no stopping
condition as with deleteheader, so I see no real conflict.

> 5322, was RFC 2822) messages.

It can also be done intentionally. And there can be good reasons for doing so.
Sieves are typically applied around the time of final delivery to the store,
and all kinds of odd things hang off of message stores.

And in point of fact, this is actually permitted by the relevant standards:
Criteria for what's legal in a message store message are a lot more lax than
RFC 5322 specifies.

> Some header fields defined in the IMAIL
> recommendations in this respect.

I wouldn't and would oppose their addition. In fact in hindsight I think the
MUST NOT allow deletion of Received: fields is actually a mistake. (Yes, I know
it can be used to break loop detection, but it also can be used to work around
bogus loop checks which, like it or not, do happen from time to time.)

> For example, is it wise to allow adding
> some syntactically incorrect nonsense?

The problem is figuring out where the boundary between what you should allow
and what you shouldn't is nearly impossible to specify. And even if were,
there's an even bigger problem: A series of editheader actions may at  some
intermediate point leave the header in an "illegal" condition that subsequent
actions fix up. So if you want to try and prevent generation of "illegal"
headers you have to look at the sum of the actions. Good luck with that.

> More complex examples would

A condition that could easily arise as an intermediate step in editing.

> occurrence of such headers during addheader would be an option, but is

No, for the reasons I've given.

> Or am I just exaggerating this problem?

Yes, I'd have to say that you are.

				Ned

---

From: Stephan Bosch <[email-masked]>
Date: Thu, 27 Oct 2011 19:03:26 +0200

> condition as with deleteheader, so I see no real conflict.

Yeah, now that  you explain it like that, I agree. My main concern is 
consensus anyway.

[...]

> Yes, I'd have to say that you are.

And, apparently you're not the only one. Well, it's a good thing anyway; 
saves me quite a bit of work. ;)

Thanks!

Regards,

Stephan