Subject: Re: [Asrg] 4. Consent Framework - General

From: "John Fenley" <[email-masked]>
Date: Thu, 28 Aug 2003 12:09:46 -0600

From: "Andrew Akehurst" <[email-masked]>
>Quoting John Fenley <[email-masked]>:
>
> > It seems to me that no matter what language the consent famework
> > is writen in, the limiting factor is still the tests that can be
> > performed. It does no good to have even a perfect consent > defenition 
>if the tests that must be perfomed are not possible.
>
>We're obviously limited by the bounds of conventional computation.
>
>If AI technology were good enough I could scan my brain, create a
>clone of my mental model for deciding whether e-mail is acceptable
>and run that model against incoming messages.
>
>Utterly impractical of course (although Greg Egan gave a nice
>fictional description of such a process in "Permutation City").
>
>However there are plenty of tests that are readily computable and
>the power of such a framework lies in the ability to logically combine 
>tests and to enforce policy decisions at multiple points
>in the network.

I agree that there are many tests that could be used without Choicelist, but 
those tests are mostly "natural tests" such as content analysis and the use 
of existing systems to clasify mail. I find that these tests are 
insufficient to define the types of consent i feel that people would want to 
define. ie...friends of friends, or personal vs commercial.

> > I have created a consent framework that i feel is simple, complete,
> > and relies on tests that i know would be possible with the > choicelist 
>system I have been working on.
>
>Re-reading your PDF at:
>   http://www.ftc.gov/bcp/workshops/spam/Supplements/fenley.pdf
>
>... I see Choicelist as being an implementation of a specific consent 
>system. (I don't mean that as a criticism of its generality,
>just an observation)
>
>The principle behind it is not inconsistent with the ASRG's
>definition of a generic consent framework:
>
>    http://www.solidmatrix.com/research/asrg/asrg-consent-framework.html
>
>In that context, the Choicelist consent policy rules specify:
>
>  - accept mail from Choicelist senders where the user    has opted in
>
>  - reject mail from Choicelist senders where the user has
>    not opted in
>
>  - filter all other messages using some other spam-filtering
>    tools/techniques
>
>Have I interpreted this correctly?

Yes you have. At the time I wrote that, I believed that seperating 
commercial mail would be suficient to solve most problems with spam. I have 
since changed my view, and now see it as vital that personal mail be 
separated and protected.

I have incorporated many of the views put forth on this list over the last 4 
months into an updated system. as you can imagine it has evolved quite a 
bit. I do not at this time have a single comprehensive overview of the 
current system available on the web though i am working on one.

>
>The "Choicelist Database Query Protocol" you refer to at:
>   http://www.choicelist.com/Personal.html
>
>... would be a kind of consent protocol as defined in section 2.5
>of the ASRG framework.
>
>Choicelist could be one of the implementation options available
>in a standard framework, or else could become part of the
>standard. If it is as powerful as you suggest then perhaps the
>work on XML-CPDL ought to define standard test(s) for checking the
>Choicelist status of a sender.

The database query protocol is mostly a way to speed requests by 
communicating behaviors to the requesting MUA. the protocol would 
automatically refer requesting MUAs to open public mirrors to distribute the 
load. it would also allow a "shimmering mirror" connection between mirror 
servers allowing the list of canceled Ids to propagate almost instantly 
across all servers. all this would happen seamlessly because of the nature 
of the protocol.

I am currently working on creating a working system. in it's most current 
incarnation there realy only need to be 4 types of tests, and these will be 
keyed lookups in a database. In this sense the "tests" have already been 
done and all that is required is a lookup of the test results.

an email address returns a choicelist Id number.
a choicelist Id number returns all info related to that Id... addresses, 
authentication info, etc.
a quick lookup returns a list of all recently canceled Id numbers (since the 
last overall update)
a 3rd party list number returns a list of email addresses.

I realy need to update my stuff and make it more readable... that page is 
something i meant to update a while ago.

> > The attached file would be read as is by a Choicelist MUA.
>
>Thanks, it's worth a read. There's definitely an elegant simplicity
>about the way it looks.
>
>First impressions/comments...
>   - can you clarify the syntax of the "list" command? In the example
>     "list=324bn28v[a][school],3jbnj8bc[a][chemistry class],"
>
>     Is it that "324bn28v" and "3jbnj8bc" are choicelist IDs,
>     that "a" is short for "allow" and the other section is a     comment 
>or description?

Yeah, the random strings are just a representation of choicelist Ids, and 
"a" is short for "allow". the next bracketed field would be the destination 
directory for emails from that group.

I am hoping that groups like this would be constructed by schools and 
teachers to facilitate communication between students, or by companies to 
ensure that company mail is always whitelisted and separated.

>   - in your whitelist/blacklist command entries you specify some
>     IP addresses to match against. Which IP address are you     intending 
>to match? And how would a Choicelist system
>     deal with DHCP or NAT configurations, in which IP addresses
>     are either unstable or not those of the machine you expect?

Ip address compatability here would work the same way that Ip based lists 
are used currently... actually I don't realy know but i wanted to allow the 
option for those who would choose to use it. I personally see little use for 
these as the number of entries in the database increases.

>   - consider using ISO 8601 formats on dates. YYYY-MM-DD is less
>     ambiguous in an international context than MM/DD/YYYY.

I didn't know there was a standard...I will change to this standard format 
for dates, as well as investigate standard formats in the future.

>Thanks
>Andrew

Thank You.
John Fenley

_________________________________________________________________
Get MSN 8 and help protect your children with advanced parental controls.  
http://join.msn.com/?page=features/parental

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Bart Schaefer <[email-masked]>
Date: Fri, 29 Aug 2003 03:55:10 +0000

How would Choicelist or a similar scheme handle mailing lists that use
VERPs?  Specifically those where the VERP token is regenerated at every
list delivery and hence differs even for the same recipient?  (I believe
Yahoo! Groups has this behavior, for example, though I haven't checked
recently.)

---

From: Andrew Akehurst <[email-masked]>
Date: Fri, 29 Aug 2003 16:10:41 +0100 (BST)

Quoting John Fenley <[email-masked]>:

>>> performed. It does no good to have even a perfect consent >
>>> defenition if the tests that must be perfomed are not possible.

>> We're obviously limited by the bounds of conventional computation...
>> the power of such a framework lies in the ability to logically 
>> combine tests and to enforce policy decisions at multiple points 
>> in the network. 

> I agree that there are many tests that could be used without
> Choicelist, but those tests are mostly "natural tests" such as 
> content analysis and the use of existing systems to clasify mail.
> I find that these tests are insufficient to define the types of
> consent i feel that people would want to define. ie...friends of
> friends, or personal vs commercial.

In that case we really should consider how to make Choicelist tests
available to anyone via the standard framework.

There could be a test to determine if the sender has the 
receiver's Choicelist consent via an opt-in. This could be used
for whitelisting messages.

There could also be a test to determine if a given sender is the
subject of a receiver's Choicelist opt-out, which could be used
for blacklisting messages.

In the light of your change of attitude (below) towards the 
commercial versus personal mail issue, maybe these will be
refined in future.

>>> I have created a consent framework that i feel is simple, 
>>> complete,and relies on tests that i know would be possible
>>> with the choicelist system I have been working on.

>> ... I see Choicelist as being an implementation of a specific
>> consent system. (I don't mean that as a criticism of its 
>> generality,just an observation)

...8<...

> commercial mail would be suficient to solve most problems with 
> spam. I have since changed my view, and now see it as vital that
> personal mail be separated and protected.

Out of curiosity, what made you change your opinion?

> I have incorporated many of the views put forth on this list over 
> the last 4 months into an updated system. as you can imagine it 
> has evolved quite a bit. I do not at this time have a single 
> comprehensive overview of the current system available on the web 
> though i am working on one.

When you have another draft of it, please post a URL on the list.
I'm sure it will help us clarify some of the issues around the
consent framework.

> automatically refer requesting MUAs to open public mirrors to 
> distribute the load. it would also allow a "shimmering mirror"
> connection between mirror servers allowing the list of canceled 
> Ids to propagate almost instantly across all servers. all this 
> would happen seamlessly because of the nature of the protocol.

Sounds good. Load balancing is definitely a requirement for 
anything that provides a centralised consent lookup service.

> I am currently working on creating a working system. in it's most
> current incarnation there realy only need to be 4 types of tests,
> and these will be keyed lookups in a database. In this sense the
> "tests" have already been done and all that is required is a
> lookup of the test results.

What were these 4 types of test?

> I realy need to update my stuff and make it more readable... that page
> is something i meant to update a while ago.

Ok, well I'll wait until you have a newer draft.

>> Thanks, it's worth a read. There's definitely an elegant simplicity

...8<...

>>     IP addresses to match against. Which IP address are you
>>     intending to match? And how would a Choicelist system
  
> Ip address compatability here would work the same way that Ip based
> lists are used currently... actually I don't realy know but i wanted 
> to allow the option for those who would choose to use it. 

I can see why it might be useful to have the feature in an 
implementation. However in the context of an overall consent framework
the definition of IP-based blocklisting could (and probably would) be
done using separate tests.

Such tests would probably make no reference to Choicelist data but
could be logically combined with Choicelist tests to produce some
quite sophisticated policies.

I can imagine wanting to have policy rules which state the equivalent
of:
   "Block all mail from 172.254.0.0/16 unless it originated from a
    Choicelist sender I have chosen to opt-in to"

Or if you defer to your favourite blocklist (say "Acme"):
   "Block all mail from Acme-DNSBl addresses unless it originated 
    from a Choicelist sender I have opted-in to"

Admittedly I've not stated these in a form very predicate-like. They
probably resolve to structures like:

 <AND>
   <TEST id="Acme-DNSl" method="Acme()">
   <NOT><TEST id="Choicelist-OptIn" method="Choice_OptIn()"></NOT>
 </AND>

 ("Sender is blocklisted by Acme and is not Choicelist whitelisted")

... which would map onto an action to reject the message.

This is just an example, but that's what it *might* look like in
XML-CPDL. I'm still pondering some ideas about the syntax in this
area of XML-CPDL and will mail out some ideas over the next couple
of days.

You could use the above kind of rule to get around some of the 
problems if a DNSBl service "goes bad". If a whole network IP block
gets a bad reputation because of a few spammers, users would still be
able to receive messages from the senders on that network whom they
have consented (via Choicelist) to communicate with.

And if Choicelist consent is expressed "out of band" (i.e. not via
e-mail messages or SMTP) then the use of e-mail blocking services
would not prevent such innocent senders on "bad" network blocks 
from establishing consent with new receivers.

> I personally see little use for these as the number of entries 
> in the database increases.

Yes, but until it does increase you need other tests in the meantime
to try and control the problem long enough for a full consent-based
system to become effective.

At least you have tried to allow for some short-to-medium term
solution so that the system could be phased in. Too many proposals
require an immediate massive change in internet practice in order
to have any chance of success.

Anyway there's plenty to think about while we await your new 
Choicelist documentation.

Thanks

Andrew