Subject: Sieve-Notify and potential associative arrays.

From: "Nigel Swinson" <[email-masked]>
Date: Wed, 2 Feb 2005 02:21:49 -0000

Looking at the long since expired Sieve Notify extension, it seems somewhat ugly adjacent to the much newer variables extension, and I would suggest needs a lot of work.  I've had a number of thoughts on the subject, some of which may have an impact on the variables extension, although they are possibly sufficiently separate that they would make an extension on their own.

It seems to me that the notify extension is trying to do too much in supporting notification by email, SMS or indeed any arbitrary notification mechanism.  It was pointed out quickly how many complex internationalization issues you have to deal with when composing emails, but you have completely different concerns when dealing with SMS messages, so I'm not sure it makes sense to bundle them.  I think we should therefore have something more along these lines:

Syntax:   sms [":recipient" <recipient-numbers: string-list>] [:limit <number>] <message: string>

The :recipient tag specifies the target phone numbers to send this SMS to.  If not present, the implementation should try to send the SMS to the owner of the script where the number is held by the sieve implementation, ie a mailbox property.  If present, specifies the target number, and it can also be a list of numbers if more than one recipient is desired.  Each phone number must begin with a + and include the country code to ensure that the script will work regardless of location of server/script.

The message is a string, whereby variable expansion is also permitted. The limit is the maximum number of SMS messages that the server can send.  Given that each message typically has a cost associated with it, the limit by default will be whatever produces the least cost which in today’s terms is 1 message (140 bytes) but may change in the future.  They can use this optional tagged argument to override this such that they will receive a multipart SMS containing more content.  Although to be honest I'm not sure how multipart SMS messages work, but I did find a reference detailing that the standard is 140 bytes http://en.wikipedia.org/wiki/Short_message_service#Technical_details.

Messages are of course therefore in UTF8 and again I'm not sure how you do internationalization in SMS but it does appear to be possible.  If we have our UTF8 message and our limit of the number of SMS messages we need to send, then that hopefully should be enough to do a transformation into the SMS gateway with the right encoding/character set.

The proposal in draft-ietf-sieve-notify-02.txt is this:
 
    Syntax:   notify [":method" string]
               [":id" string]
               [":options" 1*(string-list / number)]
               [<":low" / ":normal" / ":high">]
               ["message:" string]

I'm thinking it's got too many things in it which are trying to be super-generic to cover all uses (:method/:id/:options), but in actual fact we'll quickly regret this and prefer specific extensions for notifying through different channels with well defined and well documented arguments.

Which brings me on to variables.  Each of these different notification types, and also the vacation extension to a certain degree, has the need to author messages, and likely include sections of the triggering message.  So suppose I want to author what I think is a fairly reasonable SMS which looks like this:
           [To:<recipient-addresss>From:<sender-address>] <Subject>\r\n<body>

One way to get his is using the proposed $name$ variables which seem pretty ugly next to what we've worked so hard on with the variables extension, and also is pretty inflexible.  If we use the variables extension as is, then we could do this:

if header :matches “*” “To” {
  set “Recipient” “${1}”
}
if header :matches “*” “From” {
  set “Sender” “${1}”
}
if header :matches “*” “Subject” {
  set “Subject” “${1}”
}
set “Message” “[To: {$Recipient} From: {$Sender}] {$Subject}”

(I note that my intuative use of * won't work cos * in matches according to variables is non-greedy, yet * with regex is greedy, but I thought I'd do that deliberately to make you think...)

This seems like a lot of work to do something pretty "standard".

I wonder if it would make sense for us to add associative arrays containing entries of specific interest to the sieve script author.  So for example a $HEADERS array which contained all the header values.  So suppose my headers were:

Date: Thu, 27 Jan 2005 09:56:56 -0800 (PST)
To: [email-masked]
From: [email-masked]
Subject: Welcome to ietf-mta-filters

Then I would end up with an array like this:

This would produce an array with these entries:
$HEADERS[‘Received’][0] = “from p01m168.mxlogic.net [bla bla] ”
$HEADERS[‘Received’][1] = “from unknown [208.184.76.39] [bla bla] ”
$HEADERS[‘Received’][2] = “from above.proper.com  [bla bla] ”
$HEADERS[‘Received’][3] = “(from majordom@localhost) ”
        .”by above.proper.com (8.12.11/8.12.9/Submit) id j0RHuuq8038984;”
        .“Thu, 27 Jan 2005 09:56:56 -0800 (PST)”
$HEADERS[‘Date’][0] = “Thu, 27 Jan 2005 09:56:56 -0800 (PST)”
$HEADERS[‘To’][0] = “[email-masked]”
$HEADERS[‘From’][0] = “[email-masked]”
$HEADERS[‘Subject’][0] = “Welcome to ietf-mta-filters”

The entries in the array would have RFC2047 encoding removed, stored in UTF8, and header wrapping would be removed such that the headers would have no new lines in them.  The \r\n at the end would be removed also.  (Note the “.” String concatenation operator is not available in Sieve, but used above to demonstrate the removal of header wrapping).  The implementation would be able to evaluate the entries of the associative array in a lazy manner to avoid proceesing.

When doing string expansion, if an array is used the entries in an array would be appended together, which would of course not include the \r\n that was in the original headers, and not do the header wrapping, which means it would be entirely unsuitable for use as email headers. If in future if we need a way of turning an array into a set of headers, then we can offer a parameter to the action that would take the array and the implementation would turn these into a valid header set.

With the above strategy, we now have this:
set “Message” “[To: ${HEADERS[‘To’]} From: ${HEADERS[‘From’]}] ${HEADERS[‘Subject’]}”

I then wonder if we should have other arrays like $ENV or $MESSAGE or $ENVELOPE and could hold information like the name of the mail server, the name of the mailbox, the domain, the time zone, the spam score, or virus score.  It could be the way that extensions provide their variables to us rather than what is suggested in the variables draft:
  
4.  Interpretation of strings
   Namespaces are meant for future extensions which make internal state
   available through variables.  These variables SHOULD be put in a
   namespace with the same name as its capability string.  Notice that
   the user can not specify a namespace when setting variables with SET.

This does mean that we could do header tests by using the string test, and the $HEADERS array, and I guess that's not a bad thing, but I'm not sure if it's a good thing either...

Ah, it feels better to get all that down on paper :o)

Nigel

---

From: [email-masked]
Date: Wed, 02 Feb 2005 14:42:32 -0800 (PST)

> I wonder if it would make sense for us to add associative arrays containing
> entries of specific interest to the sieve script author.  So for example a
> $HEADERS array which contained all the header values.  So suppose my headers
> were:

I'm a fan of associative arrays, having first used them in the Sail quite some
time ago. (Anyone else remember Sail?) However, I have to say I think
associative arrays would be serious overkill for sieve. Yes, I realize they're
powerful, but they introduce significant language and implementation
complexity.

				Ned

---

From: "Nigel Swinson" <[email-masked]>
Date: Fri, 4 Feb 2005 13:45:00 -0000

> powerful, but they introduce significant language and implementation

It's really only read only associative arrays I was proposing, so that 
eliminates a large amount of complexity.  If read-only associative arrays 
are overkill for Sieve then how best can we support authoring of simple 
messages based on the content of the inbound mail?  Do you/we favour the 
$subject$ approach?  I guess I would accept some implementation complexity 
if it means the language is easier to use....

Nigel

---

From: [email-masked]
Date: Fri, 04 Feb 2005 10:00:19 -0800 (PST)

> messages based on the content of the inbound mail?

This begs the question of whether or not complex header transformations are
something we should be doing with sieve. I happen to think the answer to this
particular question is "no". Copying a subject and perhaps a couple of other
fields into a new message is about as far as I think sieve needs to go.

Of course there are all sorts of more complex things people will want to do.
But this is a seriously slippery slope, because lurking not far away are
things like content summaries, keyword extraction, translation services,
content shaping, and so on. This stuff has "research problem" written all
over it, and IMO sieve needs to stay away from this space.

> Do you/we favour the $subject$ approach?

I think it is just sufficient to get the jobs appropriate for sieve done.

> I guess I would accept some implementation complexity

Ease of use is a two-edged sword. One of the things sieve is used for is
as an output format for GUI filter builder utilities. Variables are already
pretty hard to handle in such tools, things like associative arrays
are really over the top.

				Ned

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Mon, 07 Feb 2005 01:10:53 +0100

On Wed, 2005-02-02 at 02:21 +0000, Nigel Swinson wrote:
> It seems to me that the notify extension is trying to do too much in
> supporting notification by email, SMS or indeed any arbitrary
> notification mechanism.  It was pointed out quickly how many complex
> internationalization issues you have to deal with when composing
> emails, but you have completely different concerns when dealing with
> SMS messages, so I'm not sure it makes sense to bundle them.  I think
> we should therefore have something more along these lines:
> 

as a general point, I think it's not worthwhile to standardise
specifically on SMS.  it's not _that_ universal, and it's also being
phased out by MMS, which is just good old MIME documents (transported by
HTTP for the last hops, but SMTP for routing).

a limit on message length is a good option to add, but it shouldn't be
based on SMS chunk size.

> The :recipient tag specifies the target phone numbers to send this SMS
> to.  If not present, the implementation should try to send the SMS to
> the owner of the script where the number is held by the sieve
> implementation, ie a mailbox property.  If present, specifies the
> target number, and it can also be a list of numbers if more than one
> recipient is desired.  Each phone number must begin with a + and
> include the country code to ensure that the script will work
> regardless of location of server/script.

is it important to trap this at compile-time?

> I'm thinking it's got too many things in it which are trying to be
> super-generic to cover all uses (:method/:id/:options), but in actual
> fact we'll quickly regret this and prefer specific extensions for
> notifying through different channels with well defined and well
> documented arguments.

you may be right.  separate extension names for each notify method may
be the way to go.  how many methods are we talking about?  we've been
loosely discussing a Jabber notification method where I work.  (SMS/MMS
costs money :)

> Which brings me on to variables.  Each of these different notification
> types, and also the vacation extension to a certain degree, has the
> need to author messages, and likely include sections of the triggering
> message.  So suppose I want to author what I think is a fairly
> reasonable SMS which looks like this:
> One way to get his is using the proposed $name$ variables which seem
> pretty ugly next to what we've worked so hard on with the variables
> extension, and also is pretty inflexible.  If we use the variables
> extension as is, then we could do this:
> 

(you'll have to switch the arguments, it's header COMPARATOR header-name
pattern)

> (I note that my intuative use of * won't work cos * in matches
> according to variables is non-greedy, yet * with regex is greedy, but
> I thought I'd do that deliberately to make you think...)

greediness doesn't come into play here, as :matches "*" == :regex
"^(.*)$"

> This seems like a lot of work to do something pretty "standard".

agreed.

> I wonder if it would make sense for us to add associative arrays
> containing entries of specific interest to the sieve script author.
> [...]

this is just a different syntax for:

  ${header.received.n1} = “from p01m168.mxlogic.net [bla bla] ”
  ${header.received.n2} = from unknown [208.184.76.39] [bla bla] ”
  ${header.date} = “Thu, 27 Jan 2005 09:56:56 -0800 (PST)”

(the variables draft doesn't allow the last component of the variable
name to be a number.  I think that is an aspect we can change if we want
to, but it must be done now.)

anyhow, doing it via a namespace makes it more natural to present the
information in a parsed manner.  e.g., you could have an
header.date.year == "2005" in addition.

(the variables draft allows header.date and header.date.year to
co-exist, as the namespace is defined as the bit before the last full
stop.)

--

---

From: "Nigel Swinson" <[email-masked]>
Date: Mon, 7 Feb 2005 14:19:50 -0000

> HTTP for the last hops, but SMTP for routing).

I don't yet know enough about MMS sadly to make comment :o(

> a limit on message length is a good option to add, but it shouldn't be

Why are they setting a limit though?  Partly due to how much text they want to scroll through, but more todo with how much it's going to cost them.  Isn't it the standardised SMS chunk size the root of the cost for SMS just now? If we make the limit based on message length, then we require that our users have some knowledge of how long a message they can have for one SMS chunk and therefore keep their costs down.

> > Each phone number must begin with a + and

I think it's similair to the desire to catch invalid variable names at compile time.  So yes, I think so.

> > if header :matches "*" "To" {
> > set "Recipient" "${1}"
> > }
> > if header :matches "*" "From" {
> > set "Sender" "${1}"
> > }
> > if header :matches "*" "Subject" {
> > set "Subject" "${1}"
> > }
> > set "Message" "[To: {$Recipient} From: {$Sender}] {$Subject}"
> 

Ug I never did find the order intuative...

> greediness doesn't come into play here, as :matches "*" == :regex

I'm happy that that is the case but is that obvious to everyone?  It wasn't obvious to me, and I don't recall reading that it was ever specified.  I think we have another thread on this topic though...
 
> > $HEADERS[‘Received’][0] = "from p01m168.mxlogic.net [bla bla] "
> > $HEADERS[‘Received’][1] = "from unknown [208.184.76.39] [bla bla] "
> > $HEADERS[‘Date’][0] = "Thu, 27 Jan 2005 09:56:56 -0800 (PST)"
> 
> ${header.received.n1} = "from p01m168.mxlogic.net [bla bla] "
> ${header.received.n2} = from unknown [208.184.76.39] [bla bla] "
> ${header.date} = "Thu, 27 Jan 2005 09:56:56 -0800 (PST)"

Yes, I would have suggested that instead, but I suppose I wondered if such a mechanism would mean we'd have to predefine all header names we'd ever use.  I thought perhaps the use of the associative array made it clearer that the header might not actually exist in the email, and therefore cope with all future header names.

> to, but it must be done now.)

I think it preferable to allow array syntax instead of variables starting with numbers.

> header.date.year == "2005" in addition.

Indeed, that might work neatly!

If we were to do this then where would such a mechanism be specified?  3028bis, variables, new draft?  I'd love it to be available in the variables spec...

Nigel

---

From: "Nigel Swinson" <[email-masked]>
Date: Mon, 7 Feb 2005 14:19:48 -0000

> particular question is "no". 

I agree.

> Copying a subject and perhaps a couple of other

I agree.  So the question is restated as should the set of "other fields" be predefined and fixed, or generic and able to cope with all current and future needs?  The existing $subject" strategy seems to make a very limited and fixed set of message attributes available, and I'm concerned that this isn't going to meet our future needs and barely meets our current needs.

> over it, and IMO sieve needs to stay away from this space.

Content summaries sound quite useful :o)  I suppose were they to stabalize then I'd like them to be made available through other Sieve extensions which would make new variables available to the script author for testing or use in creating new messages.

> I think it is just sufficient to get the jobs appropriate for sieve done.

If we are to keep it then I hope we at least change it to ${subject} or ${headers.subject}

> pretty hard to handle in such tools, things like associative arrays

Actually I think what I'm proposing will make it easier for a gui, not harder.  I'm imagining a gui with checkboxes along these lines:

Send an SMS to your phone including the following attributes:
- Sender
- Subject
- Message Body
- Spam Score
- Priority

Then depending on which checkboxes were ticked would change what message string we authored, where the checkbox would either add or remove a $HEADERS['Subject'] from the message string.  Without this kind of mechanism, we have to not only add the string to the message, but also use a header test before hand to make sure we have that variable available to us to use in this message.  

I've had to solve this in the past by having large "magic strings" like this:

 if spamtest :matches :comparator "i;ascii-numeric" "*" {set "spamscore" "${1}";}
 addheader "X-Spam-Score" "${spamscore}";

Where the presence or absence of the entire two lines is controlled by a checkbox.  It would have been much easier if I had a ${spamscore.score} or a ${MESSAGE['SpamScore']} available to me.

I would doubt that any gui builder out there can handle every corner case of every part of Sieve, but rather makes the most useful subset available to the user and surrenders to a text editor if the nesting/syntax goes beyond the scope of what can be easily communicated through form controls.

Nigel

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Mon, 07 Feb 2005 16:06:13 +0100

> compile time.  So yes, I think so.

I agree it is desirable, and if we have mechanism specific keywords,
arguments should be syntax checked as much as possible.

> > (you'll have to switch the arguments, it's header COMPARATOR header-name

sorry, this should say "header MATCH-TYPE header-name pattern"

> Ug I never did find the order intuative...

it is intuitive if you use the default match-type ":is".  the
non-intuitive bit is the placement of the match-type, but it is a basis
for the syntax that all tagged arguments appear before positional
arguments.

> I'm happy that that is the case but is that obvious to everyone?  It
> wasn't obvious to me, and I don't recall reading that it was ever
> specified.  I think we have another thread on this topic though...

I think we agreed previously that 3028bis should be more explicit about
the meaning of wildcards in the argument to :matches.  I think it is
non-controversial, though, nobody would expect "abc" to match "b*".

> Yes, I would have suggested that instead, but I suppose I wondered if
> such a mechanism would mean we'd have to predefine all header names
> we'd ever use.  I thought perhaps the use of the associative array
> made it clearer that the header might not actually exist in the email,
> and therefore cope with all future header names.

that's really for the extension to decide.  the variables draft doesn't
rule out having variables spring to life when referenced -- it only
specifies the syntax of the names.   how the variables inside a
namespace work is left for the extension to define.

I would prefer the list of supported header names was explicit, though,
so that the extension could do the relevant decoding for the user.
notice that I omitted the number suffix for header.date, since RFC 2822
explicitly says there should be 1 and only 1 "Date" header.

> I think it preferable to allow array syntax instead of variables

I'm afraid I don't see that happening, the variables draft is too close
publication for such a change to the syntax.

> If we were to do this then where would such a mechanism be specified?
> 3028bis, variables, new draft?  I'd love it to be available in the
> variables spec...

the namespace feature of the variables draft was specifically added to
allow independent extensions to add "magic" variables.
-- 
Kjetil T.

---

From: [email-masked]
Date: Mon, 07 Feb 2005 08:31:14 -0800 (PST)

> I agree.  So the question is restated as should the set of "other fields" be
> predefined and fixed, or generic and able to cope with all current and future
> needs?  The existing $subject" strategy seems to make a very limited and fixed
> set of message attributes available, and I'm concerned that this isn't going to
> meet our future needs and barely meets our current needs.

Oops, I missed the fact that you were talking about the specific (and overly
limited) approach taken in the long-expired notification draft. I have been
assuming that the existing $subject strategy would vanish now that we have
variables. I regard it as much clear to do a

   if header :matches "subject" "*" {set "subject" "${0}"}
   else {set "subject" ""}

sort of thing than to have a couple of magic variables, or associative arrays
of headers for that matter.

> Content summaries sound quite useful :o)

Yes, well, it seemed useful when I first heard a presentation on this general
area by Minski back in 1977. However, at the time I wasn't at all impressed by
the approaches being taken, and here it is, almost 30 years later, and look at
all the progress we've made. Personally, I'm not holding my breath...

> I suppose were they to stabalize then I'd like them to be made available
> through other Sieve extensions which would make new variables available to the
> script author for testing or use in creating new messages.

Again, I'm not holding my breath...

> If we are to keep it then I hope we at least change it to ${subject} or

I'd rather get rid of it entirely and use variables instead.

> Actually I think what I'm proposing will make it easier for a gui, not
> harder.

Perhaps in this specific case, but in general, no. The problem with  the idea
of having "special" variables show up is that it won't be limited to accessing
the header. You yourself used the notion of extending it to spamtest. This
sort of thing requires fairly deep knowledge of each extension in the GUI.
I'd much rather have the binding from various information sources to
variables be explicit. It may not be possible to present this sort of
thing in a very pretty way, but at least it should be possible to accmodate
extensions more easily.

>  addheader "X-Spam-Score" "${spamscore}";

Which I think is fine.

> Where the presence or absence of the entire two lines is controlled by a
> checkbox.  It would have been much easier if I had a ${spamscore.score} or a
> ${MESSAGE['SpamScore']} available to me.

Yeech. This is exactly the sort of thing I think we need to avoid at all
costs.

				Ned

---

From: "Nigel Swinson" <[email-masked]>
Date: Mon, 7 Feb 2005 18:33:05 -0000

> sort of thing requires fairly deep knowledge of each extension in the GUI.

I can see it's a nice goal to have a generic gui for building any old sieve 
script, but I don't think we can really get away from having to do some 
development work for each extension to make it available more neatly to the 
user.

> thing in a very pretty way, but at least it should be possible to 

Could we instead say that the spamscore namespace is populated after the 
first instance of using the spamtest test?  That would make my magic string:

if spamtest :matches "*" {}
addheader "X-Spam-Score" "${spamscore.score}";

Slightly better, but still a bit ugly...  Perhaps instead define some new 
action who's sole purpose is to populate the spamscore or header namespace. 
Something like the getdate (?) action from a much earlier revision of the 
variables spec?

> Yeech. This is exactly the sort of thing I think we need to avoid at all

If you like my existing magic string, then you could rephrase my request to 
say that I'd like to change my magic string to just:

  addheader "X-Spam-Score" "${spamscore.score}";

What's Yeech about that?  It's less than half the syntax and the intent is 
much clearer.

Nigel

---

From: "Nigel Swinson" <[email-masked]>
Date: Mon, 7 Feb 2005 18:33:02 -0000

> non-controversial, though, nobody would expect "abc" to match "b*".

That's a good example, thanks :o)

> explicitly says there should be 1 and only 1 "Date" header.

But all headers are subject to RFC2047 and unwrapping aren't they?  If we 
explicitly list all headers available in the header scope, then that means 
when somebody decides to use a new X- header or something like that, then 
they would need to write their own Sieve extension to make it available in 
the/another namespace.  This sounds overburdensome to me.  While I recognise 
that the standard says that there are limits on certain headers like the 
Data header, I think I'd much rather have the header "namespace" or array or 
whatever generically hold all 2822 headers.

If there is some extra parsing or meaning to a particular header, then I 
could see we could define a new namespace to access the headers in the more 
specific way with access to the more specific parsing.  Perhaps:

headers.content-type = "TEXT/PLAIN; charset=Windows-1252"
mimeheaders.content-type.type = "TEXT"
mimeheaders.content-type.subtype = "PLAIN"
mimeheaders.content-type.charset = "Windows-1252"

> publication for such a change to the syntax.

So you are saying you'd prefer us to publish variables, then later realise 
we needed this and have to have another standard or revision to allow us to 
use array notation?  I guess this is heading in the direction of Mark's 
comments, now it seems we are talking about "stringvariables" not really 
generic variables which will meet all the languages needs.

I suppose I was hoping to hear some technical reasons for why we wouldn't 
want [] and rather have a completely non standard way of providing access to 
what is really an array.

> allow independent extensions to add "magic" variables.

Perhaps if we groups several generic namespaces together which apply to all 
messages then we'd have enough for another spec.  Perhaps headers, envelope, 
message, server?  It seems to be an excellent addition to the variables 
extension if you ask me, but I recognize that this would all take a while to 
hammer out.  I just hope we don't go with a variables spec which limits our 
choices when we get there :o(

Nigel

---

From: [email-masked]
Date: Mon, 07 Feb 2005 15:25:28 -0800 (PST)

> development work for each extension to make it available more neatly to the

For seamless integration, sure, but it would really be nice if basic
integration didn't require such in-depth knowledge.

> first instance of using the spamtest test?

This makes some things better, others worse. Better in the sense that
at least the variables aren't quite as magic. Worse in the sense that 
it creates a magic side effect.

> That would make my magic string:

> action who's sole purpose is to populate the spamscore or header namespace.

That's a much better idea, actually. I could live with this approach.
Something like:

   headervariables list-of-header-fields

We could even make the "header." prefix settable.

> Something like the getdate (?) action from a much earlier revision of the

I believe it was setdate, but yes, I found the setdate thing to be fairly
reasonable.

> What's Yeech about that?  It's less than half the syntax and the intent is

The issue was more the array notation and the creep into multiple extensions.

				Ned

---

From: "Nigel Swinson" <[email-masked]>
Date: Tue, 8 Feb 2005 11:02:53 -0000

> We could even make the "header." prefix settable.

Ok, so what if we have an action which requests certain variables associated with an extension to be made available in it's namespace.

   setvariables <extension-name: string> [<list-of-variables: string-list>]

If you don't supply a list of variables that you want access to, it will activate all variables the extension defines, else it will only activate the list of variables that you were interested in.  This might help with the issue of only wanting to use processing resources (av/as?) for some messages.

If we were all happy with this approach, it would seem to me to be a good addition to the variables spec?

Nigel

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Tue, 08 Feb 2005 16:18:13 +0100

On Tue, 2005-02-08 at 11:02 +0000, Nigel Swinson wrote:
> Ok, so what if we have an action which requests certain variables
> associated with an extension to be made available in it's namespace.
> 
> If you don't supply a list of variables that you want access to, it
> will activate all variables the extension defines, else it will only
> activate the list of variables that you were interested in.  This
> might help with the issue of only wanting to use processing resources
> (av/as?) for some messages.

the analysis of the script to find the variables actually used is so
trivial I don't think it is worthwhile to burden the script writer with
replicating it.

I don't think it is meaningful to add a general function.  consider
SETDATE, which had an optional parameter for the timezone.

for many extensions, I don't think an explicit action is needed, and in
my view that includes the spamscore extension.  there is one advantage
to doing it, though.  if you require only "spamscore", and not
"variables", strings like "${spamscore.value}" will still be legal, they
will just not be interpreted.  a separate extension name which makes a
setter function available may make such errors less likely.
-- 
Kjetil T.

---

From: [email-masked]
Date: Tue, 08 Feb 2005 08:45:58 -0800 (PST)

> SETDATE, which had an optional parameter for the timezone.

I agree. The need for extension-specific parameters makes a general function
infeasible.

> setter function available may make such errors less likely.

I agree here as well. The cost of doing what's necessary to bind a fair
number of different variables to a set of values is the only thing that
makes this whole idea somewhat attractive. But the code of binding
one or two variables, which is what you'd have in the case of spamtest
or virustest, is so short that I just don't see the need for a separate
variable binding action.

				Ned

---

From: "Nigel Swinson" <[email-masked]>
Date: Tue, 8 Feb 2005 17:11:53 -0000

> I agree. The need for extension-specific parameters makes a general 
<snip>
> or virustest, is so short that I just don't see the need for a separate

Great, well as you can guess I'd much rather these variables were just made 
available, and we didn't need a setvariables, so where are we now?  Are we 
saying that spamtest should be revised to specify that it defines certain 
variables in a spamtest namespace, and we have a new extension that if 
requested makes a list of message attributes available as variables, and we 
don't need any special action to ask that they be initialised?  Instead 
sensible implementations will only evaluate costly variables in a lazy 
fashion so as to avoid resource overhead?

So my SMS string has become:
 set "Message" "[To: {$header.to} From: ${header.from}] ${header.subject}"
And my "magic string" has become:
 addheader "X-Spam-Score" "${spamscore.score}";

?

Nigel

---

From: [email-masked]
Date: Tue, 08 Feb 2005 09:35:18 -0800 (PST)

> > I agree. The need for extension-specific parameters makes a general
> > function infeasible.

> available, and we didn't need a setvariables, so where are we now?

My position is that none of this is really needed, but if we have to have it
needs to take the form of a separate extension or set of extensions to define 
the needed actions to set the variables. Additionally, the only case I've seen
where I view the marginal utility as being being sufficient to bother is the
that of setheadervariables (or whatever you want to call it).

> Are we
> don't need any special action to ask that they be initialised?

Absolutely not. IMO none of this has any business being added to spamtest.
Let's please remember that spamtest is defined in a standards-track RFC, and
any enhancements to it have to be done as a separate extension. It could
be incorporated into a revised RFC as a separate extension, but again,
I don't think the benefits in the case of spamtest are really there.

> Instead

>  addheader "X-Spam-Score" "${spamscore.score}";

I don't see any variable setting actions here, so this is IMO incomplete.

				Ned

---

From: Michael Haardt <[email-masked]>
Date: Wed, 16 Feb 2005 14:16:20 +0100

Sorry for the late reply, but I sent this previously with a sender
that's not subscribed to the list.

On Wed, Feb 02, 2005 at 02:21:49AM -0000, Nigel Swinson wrote:
>  It seems to me that the notify extension is trying to do too much
>  in supporting notification by email, SMS or indeed any arbitrary
>  SMS messages, so I'm not sure it makes sense to bundle them.

It may be easier to specify different actions than different extensions
to notify, at least current extension specifications make me think so.
So I don't mind to split things up.

>  to.  If not present, the implementation should try to send the SMS to the
>  owner of the script where the number is held by the sieve implementation,
>  ie a mailbox property.

As long as that's not a "SHOULD try", it is fine with me.  Enforcing a new
mailbox property in order to use this extension would not be nice at all.

So what if no number is specified, either explicit or implicit? Should
the action be ignored or cause the script to fail? The same question
arises on invalid number strings.  I vote for ignoring invalid numbers
and actions without any number at all.

>  If present, specifies the target number, and it
>  can also be a list of numbers if more than one recipient is desired.

Please let the implementation limit the maximum amount of specified
recipients or sms actions.

>  Each phone number must begin with a + and include the country code to
>  ensure that the script will work regardless of location of server/script.

I think that must be a MUST. :) I am sure the TON/NPI for the
international numbers you described is defined by a standard that could
be referenced here, I just don't know which one.

>  The message is a string, whereby variable expansion is also
>  permitted. The limit is the maximum number of SMS messages that the
>  server can send.  Given that each message typically has a cost associated
>  with it, the limit by default will be whatever produces the least cost
>  which in today?s terms is 1 message (140 bytes) but may change in the
>  future.

Over here, up to 160 octets do fit in a single message.  I think that
the EUR currency sign is encoded in two octets, though.  The character
set is IA5, so we have to specify what happens to unicode characters that
can not be translated to IA5.  I am used to gateways just dropping them.

Of course you could apply the length after IA5 translation, but how
about specifying the cost in number of messages (:parts)?

>  types, and also the vacation extension to a certain degree, has the need
>  to author messages, and likely include sections of the triggering message.
>  So suppose I want to author what I think is a fairly reasonable SMS
>  which looks like this:

I prefer this:

               From <sender-address> To <recipient-address>: 
<Subject>\r\n<body>

I would like to allow the implementation to use a different default
format.  In case it has access to a language mailbox property, it should
be allowed to send a message in the users native language by default.

>  extension, and also is pretty inflexible.

Agreed.

There are two remaining issues: Rate limiting and security.  I suggest
a new extension which specifies a condition for rate limiting, e.g.
named token buckets, to keep that out of sms.  It may be useful to forward
mail to expensive gateway addresses as well.  The sms/notify extension
should not offer builtin rate limiting.  With concern to security: The
implementation SHOULD enforce that SMS are only sent to verified numbers.
Otherwise, one typo suffices to annoy someone else like hell, apart
from letting him read part of your mail.

Michael

---

From: "Nigel Swinson" <[email-masked]>
Date: Thu, 17 Feb 2005 14:43:31 -0000

> that's not subscribed to the list.

:o)

> So I don't mind to split things up.

Ok :o)
 
> mailbox property in order to use this extension would not be nice at all.

Good point, I'm fine with this.

> and actions without any number at all.

I think if phone numbers are supplied, then the implementation SHOULD raise a compile error if the supplied string is not valid.  If no phone number is supplied then the implementation SHOULD try use a default, and if this is not available then the implementation SHOULD notify the user of the error but MUST NOT fail the execution of the script.  I had a look to see what was specified in the vacation draft, but didn't find any comments relating to what I think are similair issues.

> recipients or sms actions.

Ok :o)  And if more than the limit are specified, then this would be a compile error?

> be referenced here, I just don't know which one.

Ok, I guess when the time comes we'll need to find it and reference it.

> about specifying the cost in number of messages (:parts)?

The :limit parameter I was proposing was meant to apply after any character set encoding of the message.  I'm not familiar with IA5, I had hoped we could just pipe UTF8 and then truncate at some appropriate point.  So I'm not sure in what way your proposed :parts differs to my :limit?

> be allowed to send a message in the users native language by default.

I wasn't actually proposing we standardize the default message format, just describing what I was going to use, but that is an interesting idea :o)  That could make your sieve action just:

   sms;

If we are to offer a standard, then maybe we should just drop the "To" as well, as in the vast majority of cases I'd imagine users will know who the message was sent to.

> >  extension, and also is pretty inflexible.

:o)
 
> should not offer builtin rate limiting.  

Reminds me of an old post that I can't find just now todo with gathering statistics about how long various extensions were taking so as to limit the capabilities of scripts...

> With concern to security: The

I'm still dreaming of the day when phone numbers don't exist in the user domain at all, and rather we move completely over to using the localpart@domainname format for email, post, phone, sms, fax.  I'm told the model in the US is that the recipient pays for receiving an SMS, and I'm amazed to hear that it works!  Apparently you can disable SMS from certain senders as a way of protecting against error/mallicious use.

Thanks for the comments!

Nigel

---

From: Michael Haardt <[email-masked]>
Date: Thu, 17 Feb 2005 16:32:52 +0100

On Thu, Feb 17, 2005 at 02:43:31PM -0000, Nigel Swinson wrote:
> I think if phone numbers are supplied, then the implementation SHOULD
> raise a compile error if the supplied string is not valid.

You assume that Sieve scripts are always compiled, but in fact RFC 3028
allows for them to be interpreted, too.  Invalid phone numbers are
like invalid header names in comparisons to me.  It's not specified
what happens in this case.  My implementation generates an error to
ease debugging, but does so when interpreting the script.

> Ok :o)  And if more than the limit are specified, then this would be a compile error?

Or a runtime error. :)

> The :limit parameter I was proposing was meant to apply after any
> character set encoding of the message.  I'm not familiar with IA5, I had
> hoped we could just pipe UTF8 and then truncate at some appropriate point.
> So I'm not sure in what way your proposed :parts differs to my :limit?

Again, over here people pay per part.  One SMS part has a fixed cost,
billed (mostly) to the sender.  Very long SMS contain multiple parts.
If other telephone networks follow this model, then it makes more
sense to let people specify the maximum number of parts allowed,
which dirctly translates to money, cutting the message after that.
What good is a limit of 170 characters, if 1-160 characters cost one
unit and 161-320 cost two units? Then again, I may give a message with
many UTF-8 characters that will be dropped when converting to IA5, so
people won't understand why sometimes 200 characters "fit" in 170 octets.
With parts, they indirectly say "you can spend this much money" and do
not need to care about encoding or number of octets.

> I wasn't actually proposing we standardize the default message format,
> just describing what I was going to use, but that is an interesting idea
> :o)  That could make your sieve action just:
> 
>    sms;

Yes, if you have a mailbox property that contains a phone number, but
I wouldn't make that even a SHOULD, but optional.

> If we are to offer a standard, then maybe we should just drop the "To"
> as well, as in the vast majority of cases I'd imagine users will know
> who the message was sent to.

Not if they have multiple mailboxes, like office and private.

Michael

---

From: "Nigel Swinson" <[email-masked]>
Date: Thu, 17 Feb 2005 18:06:59 -0000

> RFC 2806 defines the "tel" URL scheme - is this what you are looking for?
> 
> Daniel

Yeah perfect!  Thanks Daniel!  I found RFC 3966 is the newer version of 2806, and it goes OTT for the purposes for Sieve, but if we pick selected bits of the BNF we get:
 
   telephone-subscriber = global-number / local-number
   global-number        = global-number-digits *par
   local-number         = [I'd suggest we'd not allow this]
   par                  = [I'd suggest we'd not allow this]
   global-number-digits = "+" *phonedigit DIGIT *phonedigit
   phonedigit           = DIGIT / [ visual-separator ]
   visual-separator     = "-" / "." / "(" / ")"

Then it's a fairly trivial case of binning the visual-separators and you should be left with a string of numbers prefixed with a +. :o)

Nigel

---

From: "Nigel Swinson" <[email-masked]>
Date: Thu, 17 Feb 2005 17:56:06 -0000

> not need to care about encoding or number of octets.

I'm in the UK so it's the same for me too.  You are detailing the exact reason why I proposed :limit being a message limit rather than a character limit (which was in the original I-Draft).  My "message" is your "part", we're gunning for the same thing.  :o)

> Not if they have multiple mailboxes, like office and private.

So will that be more common?  With only 140 8 byte characters to play with, using up approximately 40 for what might always be the same might be quite a lot....  Ideally we'd pick a default which had no internationalisation issues, so maybe not even use the word "From"....  But perhaps the subject has no place in the draft anyway...

Nigel