Subject: Sieve base-spec revision I-D

From: Philip Guenther <[email-masked]>
Date: Tue, 15 Feb 2005 16:22:52 -0800

It missed at least one of the deadlines, so the I-D for the revision
of RFC 3028 can be found at:
	http://sendmail.org/draft-ietf-sieve-3028bis-00.txt

Changes from RFC 3028
 1. Split references into normative and informative
 2. Update references to current versions of DSN, IMAP, MDN, and UTF-8 RFCs
 3. Replace "e-mail" with "email"
 4. Incorporate RFC 3028 errata
 5. The "reject" action cancels the implicit keep
 6. Replace references to ACAP with references to the i18n-comparator draft.
    Further work is needed to completely sync with that draft.
 7. Start to update grammar to only permit legal UTF-8 (incomplete)
    and correct various other errors and typos
 8. Update IPR broilerplate

I have some additional fixes for the grammar already queued up for
the -01 rev, so you can probably hold off on picking nits there for
now.  The only open issue I need feedback on there is where bare
CR and LF characters should be allowed.  In comments?  In strings?
In multiline literals?

The other open issues on my list are:

1) removal of the "Tests MUST NOT have side effects" restriction.
   We must permit the behavior of the 'variables' extension; do we
   still want to discourage side-effects in general or should the
   last two paragraphs of section 2.5 simply be dropped?

2) some other clarifications that Cyrus forwarded to me that I
   haven't rolled in yet.  (sorry)

3) are there other changes that should be made to synchronize with
   the i18n-comparator draft?

Respectfully submitted,

Philip Guenther
[email-masked]

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Wed, 16 Feb 2005 02:59:18 +0100

>  3. Replace "e-mail" with "email"

I think this is the wrong way around, no one pronounces "e-mail" in a
manner consistent with the spelling "email" ;-).  however, using _both_
spellings as in 3028 certainly isn't ideal.

>     and correct various other errors and typos

too bad RFC 3629 didn't include a syntax class for UTF-8 sans control
codes, I'm sure this will be needed elsewhere, too.  if you are
pedantic, 0x80..0x9F are also code points used for control codes.  so,
don't use UTF8-2, but define your own

  UTF8-2-NO-CTL = %xC2 %xA0-BF / %C3-DF UTF8-tail  ; excludes U+0080..U
+009F

I think defining

  UTF8-NO-CTL = UTF8-1-NO-CTL / UTF8-2-NO-CTL / UTF8-3 / UTF8-4
  UTF8-1-NO-CTL = %x20-7F

will make the rest of your grammar more readable.

> CR and LF characters should be allowed.  In comments?  In strings?

do we gain anything by supporting bare CR or LF?  I think we should
disallow control characters altogether (except CRLF and HTAB, of
course).  is this too big a change?  I note that 3028 only accepts NUL
inside /**/ comments (gotcha!), so arbitrary binary data isn't supported
currently.

if we keep control characters, solitary CR or LF should be accepted just
as any other odd control character.

you are aware that quoted-string is hopelessly broken, I'm sure.

wrt bracket-comment, I don't think you need to complicate it much more:

bracket-comment = "/*" (*(CHAR-NOT-STAR / ("*" CHAR-NOT-SLASH)) / "*")
"*/"

>    haven't rolled in yet.  (sorry)

a clarification about ":matches" would be useful: the pattern must match
the entire string.

thanks,
-- 
Kjetil T.

---

From: Philip Guenther <[email-masked]>
Date: Tue, 15 Feb 2005 18:42:33 -0800

>On Tue, 2005-02-15 at 16:22 -0800, Philip Guenther wrote:
...
>spellings as in 3028 certainly isn't ideal.

<shrug>  Looking again, I see the various IETF webpages aren't
consist on this, some using "email" or "Email", others using "e-mail"
or "E-mail".  I find Donald Knuth's argument for "email", as
reproduced by W. Richard Stevens at:
	http://www.kohala.com/start/papers.others/knuth.email.html

persuasive, but if there's consensus to go the other way I'll change it.

...
>don't use UTF8-2, but define your own

While the prose of RFC 3028 at least partially addresses the presence
of NUL, HTAB, CR, and LF in scripts, it doesn't ban any other valid
UTF-8 character.  Can you speak more on why you think control codes
should be banned from strings and comments?

...
>as any other odd control character.

<editor hat off>

If I want to search for control characters in a message's subject
(or body, using the body extension) or include them in a vacation
reply or notification message, why shouldn't I be able to?

<editor hat on>

>you are aware that quoted-string is hopelessly broken, I'm sure.

Already fixed in my copy.  :-)

>bracket-comment = "/*" (*(CHAR-NOT-STAR / ("*" CHAR-NOT-SLASH)) / "*")

That, like the version in 3028, permits "*/" in the middle of a
comment.  I know how to write the grammar for C-style comments; I
just need the consensus on what characters are permitted.

...
>a clarification about ":matches" would be useful: the pattern must match

How about the following for 2.7.1, paragraph 5, first sentence:
	The ":matches" match type specifies a wildcard match using
	the characters "*" and "?"; the entire value must be matched.

Philip Guenther
[email-masked]

---

From: Barry Leiba <[email-masked]>
Date: Tue, 15 Feb 2005 21:56:13 -0500

> It missed at least one of the deadlines

Yes... ARGH!  9 *AM* Monday, not 9 *PM* Monday.  Grr.  So I missed it
also, having submitted the "relational" draft Monday afternoon.  :-(
Here's what I submitted, and I'll resubmit it after the IETF meeting.
The only substantive change is in the formal grammar, where I've added
definitions of the comparison terms ("gt", "ge", "lt", "le", "eq",
"ne").

Barry

--
Barry Leiba, Pervasive Computing Technology  ([email-masked])
http://www.research.ibm.com/people/l/leiba
http://www.research.ibm.com/spam

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Wed, 16 Feb 2005 05:36:26 +0100

> persuasive, but if there's consensus to go the other way I'll change it.

I don't find that persuasive.  the natural progression is "electronic
mail" -> "e-mail" -> "mail".  the latter is another keystroke saved. :-)

> should be banned from strings and comments?

"they're ugly and will mostly serve as a projectile ejector which users
can aim at their lower extremities".

but I've already changed my mind :-).  we need to mimic what [IMAIL]
accepts at least for strings and multi-line strings.  to make it easier
to comment out parts of the code, we should not make the syntax for
comments more restrictive.

NUL should be disallowed completely.

> >as any other odd control character.

that is, with no special meaning.  this is more lax than [IMAIL], which
says

   - CR and LF MUST only occur together as CRLF; they MUST NOT appear
     independently in the body.

the grammar makes it clear that the same is true for the headers.  if
you prefer to be as strict as [IMAIL], that's fine by me, too.

> reply or notification message, why shouldn't I be able to?

for vacation, you can use ":mime" and do the quoted-printable thing
yourself.

there is a slight problem since a header can use encoding to include NUL
and other problematic characters, and the Sieve interpreter undoes this
in order to present it all as UTF-8 or plain octets to the user.  I wish
there were an escape for including octets by value in strings, e.g.
"\x00".  that would be an incompatible change, can we do that?

(if so, I'd want "\uXXXX" and "\UXXXXXXXX" for Unicode, too (same syntax
as Java and C99))

> comment.

I don't think so, note the placement of the parentheses.  but no matter.

> 	the characters "*" and "?"; the entire value must be matched.

ok, but I think I prefer s/value/value argument/
-- 
Kjetil T.

---

From: Michael Haardt <[email-masked]>
Date: Wed, 16 Feb 2005 09:42:37 +0100

>From the Exim Sieve implementation:

----------
Sieve scripts can not contain NUL characters in strings, but mail
headers could contain MIME encoded NUL characters, which could never
be matched by Sieve scripts using exact comparisons.  For that reason,
this implementation extends the Sieve quoted string syntax with \0
to describe a NUL character, violating \0 being the same as 0 in
RFC 3028.  Even without using \0, the following tests are all true in
this implementation.  Implementations that use C-style strings will only
evaulate the first test as true.

Subject: =?iso-8859-1?q?abc=00def

header :contains "Subject" ["abc"]
header :contains "Subject" ["def"]
header :matches "Subject" ["abc?def"]

Note that by considering Sieve to be a MUA, RFC 2047 can be interpreted
in a way that NUL characters truncating strings is allowed for Sieve
implementations, although not recommended.  It is further allowed to use
encoded NUL characters in headers, but that's not recommended either.
The above example shows why.  Good code should still be able to deal
with it.
----------

I doubt repeating it makes a difference to anybody here, but I would
really like if RFC 3028 specified behaviour of the above comparisons.

Michael

---

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 22 Feb 2005 13:26:24 -0500

> (if so, I'd want "\uXXXX" and "\UXXXXXXXX" for Unicode, too (same syntax

I agree with wanting to be able to specify octets by value.  I'd hate to
do it with a leading alpha character though, as '\x' is supposed to mean
'x'.  I've done it a la C escapes, with '\0nnn' meaning an octal value,
where you only have to make '0' special, and I don't think \0 sequences
being special would come as as much of a shock as \<alpha> sequences
being special.  I could also see a \0<alpha> sequence having extra
meaning, e.g. '\0xnn' for hex.  

What would be the need for \uXXXX that a series of multiple escaped
bytes couldn't provide?

mm

---

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 22 Feb 2005 13:20:31 -0500

>    last two paragraphs of section 2.5 simply be dropped?

I would either drop them, or isolate the side-effect prohibition to
the message only (i.e., may change the state of the filter, but
must not change the message).

BTW, since the intent is to make accomodation for the variables
extension, perhaps the introductory text about how variables are
not provided (two places) should be amended.

One other minor thing noted:

2.10.2.  Implicit Keep

   > An implicit keep is performed if a message is not written to a
   > mailbox, redirected to a new address, rejected, or explicitly thrown
   > out.  That is, if a fileinto, a keep, a redirect, or a discard is
   > performed, an implicit keep is not.

You've added "rejected" to the first sentence (as well as modifying
4.1), so you should probably also add "reject" to the second sentence.

(of course, this is muddled somewhat by :copy ...)

mm

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Tue, 22 Feb 2005 21:22:55 +0100

On Tue, 2005-02-22 at 13:26 -0500, Mark E. Mallett wrote:
> > I wish
> being special.

perhaps, but I don't think it makes much difference.  FWIW, I did a
quick survey of our user's Sieve scripts (a little more than 2000
scripts), and none of them had extraneous backslashes.  only one user
had (wrongly) written "\." in a regex context.

I don't like \0nnn since it is AFAIK a syntax not used in any other
language, also octal is a bit old-school :-)

(C-style \nnn for octal is problematic since it is variable width.)

> I could also see a \0<alpha> sequence having extra

it's more convenient than having to do the UTF-8 coding yourself.  not
that it's hard, I could even do it in my head, but it's error prone.  it
also puts an additional burden on Sieve script builders, they need to
convert back and forth before presenting the rule for the user.

-- 
Kjetil T.

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 23 Feb 2005 22:06:05 +0000

Kjetil Torgrim Homme wrote:

>that it's hard, I could even do it in my head, but it's error prone.
>
I second this.

>it also puts an additional burden on Sieve script builders, they need to