Subject: Re: [Asrg] 4. Consent Framework - General

From: "John Fenley" <[email-masked]>
Date: Tue, 26 Aug 2003 19:24:19 -0600

It seems to me that no matter what language the consent famework is writen 
in, the limiting factor is still the tests that can be performed. It does no 
good to have even a perfect consent defenition if the tests that must be 
perfomed are not possible.

I have created a consent framework that i feel is simple, complete, and 
relies on tests that i know would be possible with the choicelist system I 
have been working on.

The attached file would be read as is by a Choicelist MUA.

John Fenley
www.Choicelist.com

_________________________________________________________________
Get MSN 8 and help protect your children with advanced parental controls.  
http://join.msn.com/?page=features/parental

---

From: Andrew Akehurst <[email-masked]>
Date: Thu, 28 Aug 2003 12:47:42 +0100 (BST)

Quoting John Fenley <[email-masked]>:

> It seems to me that no matter what language the consent famework
> is writen in, the limiting factor is still the tests that can be
> performed. It does no good to have even a perfect consent 
> defenition if the tests that must be perfomed are not possible.

We're obviously limited by the bounds of conventional computation.

If AI technology were good enough I could scan my brain, create a
clone of my mental model for deciding whether e-mail is acceptable
and run that model against incoming messages.

Utterly impractical of course (although Greg Egan gave a nice
fictional description of such a process in "Permutation City").

However there are plenty of tests that are readily computable and
the power of such a framework lies in the ability to logically 
combine tests and to enforce policy decisions at multiple points
in the network.

> I have created a consent framework that i feel is simple, complete,
> and relies on tests that i know would be possible with the 
> choicelist system I have been working on.

Re-reading your PDF at:
  http://www.ftc.gov/bcp/workshops/spam/Supplements/fenley.pdf

... I see Choicelist as being an implementation of a specific 
consent system. (I don't mean that as a criticism of its generality,
just an observation)

The principle behind it is not inconsistent with the ASRG's
definition of a generic consent framework:

   http://www.solidmatrix.com/research/asrg/asrg-consent-framework.html

In that context, the Choicelist consent policy rules specify: 

 - accept mail from Choicelist senders where the user 
   has opted in

 - reject mail from Choicelist senders where the user has
   not opted in

 - filter all other messages using some other spam-filtering
   tools/techniques

Have I interpreted this correctly?

The "Choicelist Database Query Protocol" you refer to at:
  http://www.choicelist.com/Personal.html

... would be a kind of consent protocol as defined in section 2.5
of the ASRG framework.

Choicelist could be one of the implementation options available
in a standard framework, or else could become part of the
standard. If it is as powerful as you suggest then perhaps the
work on XML-CPDL ought to define standard test(s) for checking the
Choicelist status of a sender.

> The attached file would be read as is by a Choicelist MUA.

Thanks, it's worth a read. There's definitely an elegant simplicity
about the way it looks.

First impressions/comments...
  - can you clarify the syntax of the "list" command? In the example
    "list=324bn28v[a][school],3jbnj8bc[a][chemistry class],"

    Is it that "324bn28v" and "3jbnj8bc" are choicelist IDs,
    that "a" is short for "allow" and the other section is a 
    comment or description?

  - in your whitelist/blacklist command entries you specify some
    IP addresses to match against. Which IP address are you 
    intending to match? And how would a Choicelist system
    deal with DHCP or NAT configurations, in which IP addresses
    are either unstable or not those of the machine you expect?

  - consider using ISO 8601 formats on dates. YYYY-MM-DD is less
    ambiguous in an international context than MM/DD/YYYY.

Thanks

Andrew

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: [email-masked]
Date: Fri, 29 Aug 2003 01:26:21 -0400

On Tue, Aug 26, 2003 at 07:24:19PM -0600, John Fenley wrote
> in, the limiting factor is still the tests that can be performed. It does 
> no good to have even a perfect consent defenition if the tests that must be 

  Let's assume that we're not going to scrap SMTP or massively modify
it.  This leaves us with the following available tests...

  1) before the connection; ISP-wide firewalling/null-routing of an
     address or address range.  This would be based on local lists.

  2) after the connection, but before the body is sent, we have the
     following info delivered by the sending MTA
     - HELO (or EHLO) (What the sending machine calls itself)
     - MAIL FROM:     (Who the mail is allegedley from)
     - RCPT:          (Who the mail is addressed to)
     - from the sending IP address, we can obtain hostname, rDNS, and
       whether or not it's in a particular DNSbl

  3) the DATA: stage provides the body *AND THE HEADERS*, at which point
     we can apply header and body filters.  A common misconception I've
     seen is that "Received:", "Subject:", "From:", "To:", and all the
     "X-" headers are somehow separate from the body of the message.
     MUAs display them that way, but in reality, they are sent in the
     DATA section along with "the message body".

  Now let's look at how this is implemented in real life...

  1) An example would be in the web article at...
     http://www.wired.com/news/politics/0,1283,50455,00.html

> Blocking the spam-sending ISPs hasn't alleviated all of the
> problems. Refusing massive amounts of attempted connections also
> puts a strain on servers, in some cases bogging the system down in
> much the same way as a sustained denial-of-service attack.
>
> "British ISP UXN found that simply blocking China Telecom wasn't
> enough because UXN's mail servers still had to deal with hundreds of
> connection requests per minute from Chinese mail servers," Linford
> said. "UXN had to actually firewall China Telecom's IP range from
> connecting to UXN's mail servers to stop the mass of connections
> from clogging UXN's mail service."

  2) Most MTAs will reject email allegedly from a domain that doesn't
     resolve.  There are a couple of good reasons...
     - non-existant/forged sending domain is a strong spamsign
     - the MTA will not be able to send back a bounce message (DSN) to
       the non-existant domain
     Note that if we want the filtering to be user configurable, then it
     *MUST* be done *AFTER* "RCPT:".  That's when we find out who it's
     actually addressed to.

  3) Here's where all the body filters (bayesian, etc) do their thing.
     This includes "Habeas Haiku", etc.

> and relies on tests that i know would be possible with the choicelist
> system I have been working on.
> 

  Your example has some eerie similarities to the system that clss.net
has been using for a while.  Just after the RCPT: stage (when the MTA
finds out who the email is addressed to) their modified Qmail parses a
user-modifiable config file in the user's home directory.  If an email
is rejected, the sending MTA gets a 550 reject message, *NOT* a bounce.
Here's mine file, with minor editing-out (I realize this mailing list is
public record).  I've omitted some of my correspondents and masked the
pointer to my filter-bypass page.  Note that the following is a *REAL
LIFE EXAMPLE IN USE TODAY*

#!/var/qmail/bin/dnsblfilter -f
(This tells the MTA that the following are commands for the filter.  By
way of explanation...
SIACCEPT - accept if MAIL FROM: is the listed email address
SIREJECT - reject if MAIL FROM: is the listed email address
SIACCEPTTAIL - accept if MAIL FROM: ends with the following string
SIREJECTTAIL - reject if MAIL FROM: ends with the following string
PIACCEPT - accept if rDNS is the listed string
PIREJECT - reject if rDNS is the listed string
PIACCEPTTAIL - accept if rDNS ends with the following string
PIREJECTTAIL - reject if rDNS ends with the following string
IACCEPT - accept if IP address is in a given range
IREJECT - reject if IP address is in a given range
ACCEPT - accept if IP address is in a given DNSbl
REJECT - reject if IP address is in a given DNSbl

Note: Rejct commands can be followed by
  - %TXT%, which passes a text string direct from a DNSbl
  - a comment field.  I use this to tell people where to look for a
    temporary unfiltered email address to get in contact with me.
    Spammers tend not to read their logs, and trojaned home machines
    don't keep logs)

SIACCEPTTAIL ietf.org
PIACCEPTTAIL ietf.org
(I don't want to bounce email from this list <g>)

PIACCEPTTAIL grp.scd.yahoo.com
(I'm on a yahoo groups mailing list)

PIREJECTTAIL web80601.mail.yahoo.com
(Used by 419 scammers)

PIREJECTTAIL nj.comcast.net
(One machine on this net was pounding the planet, including me, with
Sobig.F emails)

PIREJECTTAIL nbcs-av.rutgers.edu Hey, #######.  Yes I'm talking to you.  Haven't you ever heard of viruses forging MAIL FROM ?
(Snarky message going to the logfile for some idiot admin who's set up
his "anti-virus filter" to tell me that my linux machine is infected
with a Windows virus... over... and over... and over).

SIREJECTTAIL hinet.net,netian.com,anfmail.com,e-gold.com,pisem.net,itsavvinner.net (Certain envelope-senders)
(Outfits that I do not want to hear from)

SIREJECTTAIL alansis1.com You are receiving this reject message because you have opted me in without my permission to receive messages about financial information through Alansis or a Alansis affiliate.
IREJECT 207.134.96.0/20 You are receiving this reject message because you have opted me in without my permission to receive messages about financial information through Alansis or a Alansis affiliate.
(Snarky reject message for a spammer.  Just to play safe I filter both
the domain name and the address range)

# 0.32 = Argentina; 0.76 = Brazil; 0.152 = Chile 0.156 = China; 0.250 = France;
# 1.88 = Hong Kong; 1.100 = India; 1.104 = Indonesia; 1.120 = Isreal;
# 1.124 Italy; 1.136 = Japan; 1.154 = Korea, South; 1.202 = Malaysia;
# 2.16 = Netherlands; 2.54 = Nigeria; 2.104 = Poland; 2.131 = Russia;
# 2.190 = Singapore; 2.212 = Spain; 0.158 = Taiwan; 3.58 = United Kingdom
REJECT zz.countries.nerd.dk A 127.0.0.32,127.0.0.76,127.0.0.152,127.0.0.156,127.0.0.250,127.0.1.88,127.0.1.100,127.0.1.104,127.0.1.120,127.0.1.124,127.0.1.136,127.0.1.154,127.0.1.202,127.0.2.16,127.0.2.54,127.0.2.104,127.0.2.131,127.0.2.190,127.0.2.212,127.0.0.158,127.0.3.58 %TXT% (zz.countries.nerd.dk) If yours was a legitimate email see http://########.html to bypass block.
(Countries from which I seem to get nothing but spam.  I do give a
pointer to my unfiltered address if the sender is legit)

REJECT rackspace.blackholes.us A 127.0.0.2 Rackspace.  If yours was a legitimate email see http://########.html to bypass block.
REJECT verio.blackholes.us A 127.0.0.2 Verio.  If yours was a legitimate email see http://########.html to bypass block.
REJECT xo.blackholes.us A 127.0.0.2 XO.  If yours was a legitimate email see http://########.html to bypass block.
REJECT cogentco.blackholes.us A 127.0.0.2 Cogentco.  If yours was a legitimate email see http://########.html to bypass block.
(Major spamhausen I do not want to hear from)

REJECT list.dsbl.org A 127.0.0.2 Email rejected on advice of list.dsbl.org.  If yours was a legitimate email see http://########.html to bypass block.
REJECT dynablock.easynet.nl A 127.0.0.2 %TXT% Email rejected on advice of dynablock.easynet.nl.  If yours was a legitimate email see http://########.html to bypass block.
REJECT sbl.spamhaus.org A 127.0.0.2 %TXT% Email rejected on advice of sbl.spamhaus.org.  If yours was a legitimate email see http://########.html to bypass block.
REJECT proxies.relays.monkeys.com A 127.0.0.2 %TXT% (proxies.relays.monkeys.com) If yours was a legitimate email see http://########.html to bypass block.
REJECT blackholes.five-ten-sg.com A 127.0.0.3 Dynamic IP address blocked on advice of blackholes.five-ten-sg.com.  If yours was a legitimate email see http://########.html to bypass block.
REJECT dun.dnsrbl.net A 127.0.0.3 Dynamic IP address blocked on advice of dun.dnsrbl.net.  If yours was a legitimate email see http://########.html to bypass block.
(Various DNSbls targetting specific portions of the spam problem)

REJECTNOHOSTNAME If yours was a legitimate email see http://########.html to bypass block.
(Inconsistent DNS/rDNS is one thing.  But if an MTA has no rDNS
whatsoever, either the admin is grossly incompetent, or else is a
spammer trying to cover his tracks)

PIREJECTTAIL groups.msn.com,zapo.net,dsl.att.net,client.attbi.com,client2.attbi.com,pisem.net,client.comcast.net,dsl.siol.net,cable.mindspring.com,coollist.com,cust.web-sat.com,dial.xnet.ro,ipt.aol.com,charter.com,dip.t-dialin.net,pureserver.de,bayarea.net,newskies.net,prodigy.net.mx,syd.iprimus.net.au,charter.net,tiscali.com,md.comcast.net,mel.aone.net.au,cable.ntl.com,mc.videotron.ca (Certain providers) If yours was a legitimate email see http://########.html to bypass block.
(rDNS associated with spam in my inbox)

IREJECT 200.0.0.0/8 No Hablo E-spam-ol. If yours was a legitimate email see http://########.html to bypass block.
(South America, mostly open proxies used by American spammers)

IREJECT 4.0.0.0/8 Genuity unwelcome here.  If yours was a legitimate email see http://########.html to bypass block.
(Never let it be said that I'm not an "equal opportunity blocker".
Here's a /8 in the USA)

SIREJECT [email-masked] I did not subscribe to any toner cartridge mailing list
(See, I even reject a network from here in Canada)

# E-Dialog is permanent
IREJECT 64.28.75.176/28 No monologues from e-dialog
(Anybody who's ever been a domain contact will understand)

# Accept email from "ogw" outbound.gate.way MTAs at RR; reject direct to MX
PIACCEPTTAIL ogw.rr.com
PIREJECTTAIL rr.com
(Accept email from RoadRunner authorized MTAs, but block direct-to-MX)

PIACCEPTTAIL aol.com
SIREJECTTAIL aol.com I only accept From: @aol.com addresses if the email actually originates from a aol.com server.
(I get very little spam actually from AOL, especially after blocking
ipt.aol.com direct-to-MX.  This two-step algorithm accepts email actually
from AOL.  Any email that was not accepted by the first step is probably
a forgery.  There is a risk that it's from a real AOL user who's sending
from another ISP, but wants the reply to go to his AOL account.  I
haven't run into any false-positives yet)

PIACCEPTTAIL hotmail.com
SIREJECTTAIL hotmail.com I only accept From: @hotmail.com addresses if the email actually originates from a hotmail.com server.
PIACCEPTTAIL msn.com
SIREJECTTAIL msn.com I only accept From: @msn.com addresses if the email actually originates from a msn.com server.
PIACCEPTTAIL yahoo.com
SIREJECTTAIL yahoo.com I only accept From: @yahoo.com addresses if the email actually originates from a yahoo.com server.
(Same as the AOL rule, applied to hotmail and msn and yahoo)

-- 
Walter Dnes <[email-masked]>
Email users are divided into two classes;
1) Those who have effective spam-blocking
2) Those who wish they did