Subject: DEPLOY: Prior Art for PRA

From: Yakov Shafranovich <[email-masked]>
Date: Tue, 31 Aug 2004 13:57:04 -0400

The following is the PRA algorithm from the current draft:

-------------------------------------------------------------------------
1. Locate the first non-empty Resent-Sender header in the message. 
   If no such header is found, continue with step 2.  If it is preceded 
by a non-empty Resent-From header and one or more Received or 
Return-Path headers occur after said Resent-From header and before the 
Resent-Sender header, continue with step 2. Otherwise, proceed to step 5.

2. Locate the first non-empty Resent-From header in the message. If a 
Resent-From header is found, proceed to step 5. Otherwise, continue with 
step 3.

3. Locate all the non-empty Sender headers in the message.  If there are 
no such headers, continue with step 4. If there is exactly one such 
header, proceed to step 5. If there is more than one such header, 
proceed to step 6.

4. Locate all the non-empty From headers in the message. If there is 
exactly one such header, continue with step 5. Otherwise, proceed to step 6.

5. A previous step has selected a single header from the message. If 
that header is malformed (e.g. it appears to contain multiple mailboxes, 
or the single mailbox is hopelessly malformed, or the single mailbox 
does not contain a domain name), continue with step 6. Otherwise, return 
that single mailbox as the Purported Responsible Address.

6. The message is ill-formed, and it is impossible to determine a 
Purported Responsible Address.
-------------------------------------------------------------------------

The algorithm looks at the following headers in order:
1. First non-empty Resent-Sender.
2. First non-empty Resent-From.
3. Sender.
4. From.

The PRA draft states that:
"Note that steps 1 and 2 above extract the Resent-Sender or Resent- From 
header from the first resent block (as defined by section 3.6.6 of 
[RFC2822]) if any.  Steps 3 and 4 above extract the Sender or From 
header if there are no resent blocks."

RFC 2822 is clear that (section 3.6.6):
"Each of the resent fields corresponds to a particular field elsewhere 
in the syntax."

Therefore, Resent-Sender and Resent-From correspond to "Sender" and 
"From" headers. Precendence of "Sender" over "From" and by extension of 
"Resent-Sender" over "Resent-From" is clear from section 3.6.2 of RFC 2822:

"The originator fields of a message consist of the from field, the 
sender field (when applicable)"

and

"The originator fields indicate the mailbox(es) of the source of the 
message.  The "From:" field specifies the author(s) of the message, 
that is, the mailbox(es) of the person(s) or system(s) responsible  for 
the writing of the message.  The "Sender:" field specifies the  mailbox 
of the agent responsible for the actual transmission of the message.
"

As we all know, all RFCs are under intellectual property of the IETF and 
held by ISOC.

More prior art, for example documentation for fetchmail written by ESR 
states as follows (http://catb.org/~esr/fetchmail/fetchmail-man.html):

--------------------------------------------------------------------------
INTERACTION WITH RFC 822
When trying to determine the originating address of a message, fetchmail 
looks through headers in the following order:

         Return-Path:
         Resent-Sender: (ignored if it doesn’t contain an @ or !)
         Sender: (ignored if it doesn’t contain an @ or !)
         Resent-From:
         From:
         Reply-To:
         Apparently-From
--------------------------------------------------------------------------

The only difference from PRA is the fact that Sender appears before 
Resent-From.

RFC 3028 which defined SIEVE has the following (section 5.1):
--------------------------------------------------------------------------
"5.1.     Test address

    Syntax:   address [ADDRESS-PART] [COMPARATOR] [MATCH-TYPE]
              <header-list: string-list> <key-list: string-list>

    The address test matches Internet addresses in structured headers
    that contain addresses.  It returns true if any header contains any
    key in the specified part of the address, as modified by the
    comparator and the match keyword.
....
    Implementations MUST restrict the address test to headers that
    contain addresses, but MUST include at least From, To, Cc, Bcc,
    Sender, Resent-From, Resent-To, and SHOULD include any other header
    that utilizes an "address-list" structured header body.
--------------------------------------------------------------------------

I didn't have the time, but we can check any of the implementations 
listed on http://www.cyrusoft.com/sieve/ to see the exact way they do 
it. I wound guess that they following a similar algorithm.

SpamAssassin configuration file 
(http://spamassassin.apache.org/doc/Mail_SpamAssassin_Conf.html):
--------------------------------------------------------------------------
WHITELIST AND BLACKLIST OPTIONS
whitelist_from [email-masked]

Used to specify addresses which send mail that is often tagged 
(incorrectly) as spam; it also helps if they are addresses of big 
companies with lots of lawyers. This way, if spammers impersonate them, 
they'll get into big trouble, so it doesn't provide a shortcut around 
SpamAssassin. If you want to whitelist your own domain, be aware that 
spammers will often impersonate the domain of the recipient. The 
recommended solution is to instead use whitelist_from_rcvd as explained 
below.
.....
The headers checked for whitelist addresses are as follows: if 
Resent-From is set, use that; otherwise check all addresses taken from 
the following set of headers:

         Envelope-Sender
         Resent-Sender
         X-Envelope-From
         From

In addition, the ``envelope sender'' data, taken from the SMTP envelope 
data where this is available, is looked up
--------------------------------------------------------------------------

Qmail documentation 
(http://qmail.sg.feratech.com/qmail-manual-html/man5/qmail-header.html):
--------------------------------------------------------------------------
SENDER ADDRESSES
      qmail-inject looks for sender address lists in the following
      fields:    Sender,   From,  Reply-To,  Return-Path,  Return-
      Receipt-To, Errors-To, Resent-Sender,  Resent-From,  Resent-
      Reply-To.
--------------------------------------------------------------------------

iPlanet documentation from Sun 
(http://docs.sun.com/source/816-6091-10/contents.htm):
--------------------------------------------------------------------------
The email message's originator address is taken from one of the 
following five sources, shown in decreasing order of preference:
1. Resent-from:	 	4. Sender:	
2. From:	 	5. Envelope From:	
3. Resent-sender:
--------------------------------------------------------------------------

Another piece of code 
(http://email.uoa.gr/projects/fromfilter/doc/fromfilter.html):
--------------------------------------------------------------------------
For now, the only headers we are interested in are From and Sender or, 
if this message has been forwarded and the sender has retained the 
original headers, the Resent-From and Resent-Sender.
--------------------------------------------------------------------------

Another example (http://h71000.www7.hp.com/wizard/wiz_7493.html):
--------------------------------------------------------------------------
(The list of SMTP return addresses that are checked for and used by
   OpenVMS include the following (ordered) list of fields:

    Resent-Reply-To
    Resent-From
    Reply-To
    From
    Resent-Sender
    Sender
    Return-Path)
--------------------------------------------------------------------------

The only differences are order of fields and some semantics. I am sure 
there are more examples.

Yakov

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 31 Aug 2004 16:23:53 -0400

Hi Yakov,

--On Tuesday, August 31, 2004 1:57 PM -0400 Yakov Shafranovich 
<[email-masked]> wrote:

> listed on http://www.cyrusoft.com/sieve/ to see the exact way they do it.
> I wound guess that they following a similar algorithm.

The SIEVE address test is not a PRA-type check that scans possible address 
headers in some well-defined order to extract a particular value. Instead 
it provides a way to test the content of specific address headers - no 
ordering is implied, and the author of the sieve script gets to say which 
headers are tested. So I don't think SIEVE can be used as an example of PRA 
prior art - sorry!

-- 
Cyrus Daboo

---

From: Hadmut Danisch <[email-masked]>
Date: Wed, 1 Sep 2004 00:44:43 +0200

> The following is the PRA algorithm from the current draft:

Why do you call this an algorithm, anyway?

It is just giving header lines a priority or precedence. 

  Use the one with the highest priority, and if there is no 
  such, use the one with the second highest priority, if there
  is still no such use the one with the third highest priority...

is not what I'd call an algorithm. It is just assigning priorities
to header lines. The idea of priorities and precendences is already
known for centuries. You can't patent a definition of priorities.
It has not even been invented or defined, it is just a observation of 
the current practise. 

So what exactly is new and what exactly is to be considered as an 
algorithm?

Hadmut