Subject: What Meng said

From: Andrew Newton <[email-masked]>
Date: Wed, 11 Aug 2004 12:07:32 -0400

In rereading the Jabber logs and reading the recent threads on 
SUBMITTER, I'd like to point everyone to a simple sentence Meng wrote 
in the Jabber session:  
http://www.xmpp.org/ietf-logs/[email-masked]/2004-08-04.html

>  [11:05:42] <mengwong> if submitter is not present, we fall back to 
> mail-from. people are missing this.

In defense of "people", I reread marid-submitter and marid-core this 
morning and could not find the place where this was stated.  However, 
this statement is significant.  I'm not saying it is a good or bad 
idea, but it profoundly changes how SUBMITTER is used, the nexus of 
SUBMITTER and PRA, and even the semantics of the RR version identifier.

-andy

---

From: Tony Finch <[email-masked]>
Date: Wed, 11 Aug 2004 17:36:10 +0100

On Wed, 11 Aug 2004, Andrew Newton wrote:
>
> In rereading the Jabber logs and reading the recent threads on SUBMITTER, I'd
> like to point everyone to a simple sentence Meng wrote in the Jabber session:
> In defense of "people", I reread marid-submitter and marid-core this morning
> and could not find the place where this was stated.  However, this statement
> is significant.  I'm not saying it is a good or bad idea, but it profoundly
> changes how SUBMITTER is used, the nexus of SUBMITTER and PRA, and even the
> semantics of the RR version identifier.

The reason Sender-ID falls back to checking MAIL FROM is because SUBMITTER
may be omitted if it is the same as MAIL FROM. If a Sender-ID
implementation is checking the MAIL FROM in the absence of SUBMITTER,
should it use the v=spf1 record or the v=marid1 record?

Sites must be able to write a sender-ID record that is aware of which
identity is being checked (header sender or bounce address) since these
have different semantics and distinct sets of addresses may be valid in
each context, e.g. if the site uses BATV. This might be acheived through
different record version numbers, or by extending the macro language.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BERWICK ON TWEED TO WHITBY: WEST OR SOUTHWEST 2 OR 3 INCREASING 3 OR 4. FAIR.
GOOD. SLIGHT OR SMOOTH.

---

From: Dave Crocker <[email-masked]>
Date: Wed, 11 Aug 2004 09:34:08 -0700

Andrew,

AN> In defense of "people", I reread marid-submitter and marid-core this
AN> morning and could not find the place where this was stated.  However, 
AN> this statement is significant.  I'm not saying it is a good or bad 
AN> idea, but it profoundly changes how SUBMITTER is used, the nexus of 
AN> SUBMITTER and PRA, and even the semantics of the RR version identifier.

indeed.

submitter has been offered as an efficiency hack on the rfc2822-based
checking of sender-id.  there has been no rfc2821.mailfrom in it.

so meng's statement would seem to change the basic semantics of
sender-id, depending upon whether submitter is present.

d/
--
 Dave Crocker <mailto:[email-masked]>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>, <fax:+1.866.358.5301>

---

From: Rand Wacker <[email-masked]>
Date: Wed, 11 Aug 2004 09:43:30 -0700 (PDT)

> SUBMITTER and PRA, and even the semantics of the RR version identifier.

I'm of the same opinion as Andy upon re-reading the documents, if
submitter is not present then you *have* to grovel through the headers
(this is necessary to support Sender ID filters plugged in to or behind
MTAs that may not yet support SUBMITTER).

There is a minor optimization noted that in the future, if SUBMITTER would
exactly match MAIL FROM, then SUBMITTER could be omitted.  I thought this
paragraph had the possibility for confusion written all over it, but
others wanted to leave it in.

So maybe "people" are reading the specs differently than Meng? ;)

-Rand

---

From: [email-masked]
Date: Wed, 11 Aug 2004 09:53:17 -0700 (PDT)

> checking of sender-id.  there has been no rfc2821.mailfrom in it.

That has been my reading as well. Indeed, the text of the submitter document is
quite clear that the identity involved is the PRA identity. It would also be
rather strange for the SUBMITTER SMTP extension to change the semantics of SMTP
when the extension _isn't_ used.

> sender-id, depending upon whether submitter is present.

The rationale document is also pretty clear that RFC 2821 MAIL FROM isn't
used, and why.

				Ned

---

From: John Levine <[email-masked]>
Date: 11 Aug 2004 17:36:17 -0000

>others wanted to leave it in.

I don't understand how an SMTP server is supposed to tell the difference
between a MAIL FROM that doesn't have SUBMITTER because it's the same as
the bounce address and a MAIL FROM that doesn't have SUBMITTER because it
doesn't implement SUBMITTER.

The EHLO response lets the server tell the client what extensions it
offers, but there's no way for the client to tell the server what
extensions it uses other than by using them.

I've never thought that SUBMITTER was useful, but this makes it both
useless and meaningless.  If it's going to be defined, at least define
it so it means something.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, Mayor
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: Meng Weng Wong <[email-masked]>
Date: Wed, 11 Aug 2004 14:55:08 -0400

On Wed, Aug 11, 2004 at 12:07:32PM -0400, Andrew Newton wrote:
| 
| In rereading the Jabber logs and reading the recent threads on 
| SUBMITTER, I'd like to point everyone to a simple sentence Meng wrote 
| in the Jabber session:  
| http://www.xmpp.org/ietf-logs/[email-masked]/2004-08-04.html
| 
| > [11:05:42] <mengwong> if submitter is not present, we fall back to 
| >mail-from. people are missing this.
| 
| In defense of "people", I reread marid-submitter and marid-core this 
| morning and could not find the place where this was stated.  However, 
| this statement is significant.  I'm not saying it is a good or bad 
| idea, but it profoundly changes how SUBMITTER is used, the nexus of 
| SUBMITTER and PRA, and even the semantics of the RR version identifier.
| 

I should explain the thinking behind that statement.  It is
not my intention to open a can of worms, but since I'm
likely to get yelled at either way I might as well err on
the side of openness.

Sender ID is complicated by the fact that it has "before"
and "after" aspects.

In May, in my meeting with Microsoft when Submitter was
first proposed, we tossed around a few ideas for how it
could be used.

The vision we arrived at (and Harry, Jim, please correct me)
looks something like this:

The core identity in Sender ID is the PRA.

To make the PRA available without the bandwidth and processing
burden of DATA, Submitter is used to provide a sneak preview
of the PRA at MAIL time.

Since the goal is a preview, if Mail-From and Submitter are
identical, there is strictly speaking no need to display
Submitter.

In the early days of Sender ID, you have to check the PRA,
because you can't assume the typical sender is Submitter
aware.  This is the "before" aspect.

But after Submitter has become more widely supported, a
receiver may be able to assume that Submitter was omitted
because the PRA and the Mail-From match.  This is the
"after" aspect, the long-term endgame scenario.  I would
like to get there sooner rather than later.

The language in 4.1 alludes to this assumption.

   4.1 Setting the SUBMITTER Parameter Value

   The purpose of the SUBMITTER parameter is to allow the
   SMTP client to indicate to the server the purported
   responsible address of the message directly in the RFC
   2821 protocol.

   Therefore, SMTP clients that support the Responsible
   Submitter extension MUST include the SUMBITTER parameter
   on all messages where the purported responsible address,
   as defined in section 4 of [SENDER-ID] differs from the
   MAIL FROM address.  This includes messages where the MAIL
   FROM address is empty or "<>".  SMTP clients SHOULD
   include the SUBMITTER parameter on messages where the
   purported responsible address is the same as the MAIL
   FROM address.

I take this to mean that an SMTP client that supports the
Responsible Submitter extension MAY omit the parameter on
messages where the Mail From is the same as the PRA.  And an
SMTP receiver that supports the Responsible Submitter
extension MAY assume that the Mail From is a good predictor
of the PRA.

I think that this line of thinking is useful.

If, however, we don't want to make the assumption that a
missing Submitter means we use the Mail-From, we would
require a Submitter aware sender to always pass Submitter,
even when PRA == Mail-From.  Then the language could be
changed to read:

  SMTP clients that support the Responsible Submitter
  extension MUST always include a SUBMITTER parameter on all
  messages.  This includes messages where the MAIL FROM
  address is empty or "<>".

This may also be useful, but we should consider the tradeoffs.

First, this would remove a receiver's ability to assume that
Mail-From == PRA when Submitter is absent.  Second, this
would also require PRA checks on all mail where Submitter is
not provided, and therefore make it impossible to detect and
reject envelope spoofs before DATA.  I believe that such a
change would have serious implications for the acceptance of
the proposed standard in the marketplace.  I will go into
more detail below, but first, I want to take a look at
Submitter.

I like the idea of Submitter.  Because it is a new
construct, we can load it up with powerful semantics right
out of the gate.

I believe it was Bob Atkinson who pointed out that if a
sender was aware of Submitter, that sender should also be
aware of Sender ID.  This meant that anytime a receiver sees
a Submitter, the SPF check on Submitter should return a PASS
result.  Obviously if it returns a FAIL result you reject.
But if the spf/Submitter test returns an UNKNOWN, you can
also reject!

Why?  Because no legitimate, Submitter-aware MTA should ever
send a Submitter field that doesn't result in a PASS.  If
the purported responsible address doesn't resolve in a PASS,
why bother 

If we can say that for Submitter, UNKNOWN isn't good enough,
then we have a stronger level of assurance than the
spf/Mail-From test, and we make it harder for spammers to
spoof domains in SUBMITTER.  This statement of applicability
may not be in the specification; if it is not too late,
perhaps it could be added.  I would be happy if that
happened.  If it did not happen, I expect spammers would
forge non-SPF-publishing domains in the SUBMITTER field.

We're getting close to last call.  Now that code has been
written, conformant products are shipping, specifications
have been drafted, and the design has been documented,
perhaps it's time to talk about requirements.

The anti-false-positive community wants to easily whitelist
known good senders without hurting deliverability.  This
includes the ESP industry and receiving ISPs.  Because this
is the most modest requirement, spf/Mail-From,
spf/Submitter, spf/PRA, CSV, and DK are all good enough for
this community.  They just want us to pick one and get on
with it.  Apparently S/MIME is not good enough because
people using mail clients from 1995 are an important
demographic who must not be confused by new technology.

The anti-phishing community wants to make it possible for
MUAs to validate what the end-users see, to easily detect
obvious forgeries, to easily detect obvious good mail, and
allow room for accreditation.  If we could construct an
email analogue of the https padlock icon, that would be a
step in the right direction.  Any one of DK, SMG, PGP, or
Sender ID can provide that padlock, at varying levels of
assurance.  Sender ID tries to help by offering the
designated sender approach.  I expect the MASS WG to
complement our work with good cryptographic solutions.

But technical solutions can never perfectly solve the
phishing problem.  The genie is out of the bottle: we can't
keep arbitrary graphical content from being displayed to
end-users.  What we can do is flip toward an AGUPI paradigm.
Once enough reputable senders authenticate themselves, we
can relegate all unauthenticated or unreputable senders to a
second class folder, and create new user expectations for
the differences between first class and second class folder.
Getting out of that folder should be easy for good guys and
hard for bad guys.  We can do this using accreditation and
reputation.

The SPF Classic community wants some level of protection
from blowback; the ability to reject before DATA; and a
strong proposal which can be quickly rolled out at the MTA
level independent of the MUA.  While we appreciate that
BATV/SES are the final word in blowback prevention, because
many spams are overwhelmingly directed at large ISPs, if we
can get those large ISPs to reject at SMTP time instead of
generating bounces or silently swallowing suspected spam,
then we already have a significant benefit.

Sender ID is a compromise between SPF Classic and Caller ID.
The opensource community has been very vocal in its defense
of its requirements.  So has the anti-phishing community.
My goal in exploring the affordances of Submitter is to
build a bridge between the two sets of requirements, which
are, to some degree, in conflict.  I would like to offer the
opensource and SPF Classic communities a strong argument
that their objectives will be met by Sender ID.  I fear that
if we do not, Sender ID will lose their support.

With this in mind, we can come back to the question:

In the absence of Submitter, should a receiver assume that
the Mail From is a good predictor of the PRA?

I would like to live in a world where the answer is yes.
Because of the "before/after" issue we may have to wait a
while before we can safely make that assumption.  This is
not the place to discuss how long, but I would like it to be
sooner rather than later.  (I happen to feel that the spam
problem is pretty urgent.  The baby is drowning in the
bathwater.)

I believe that if we choose a world where the answer is no,
then the SPF Classic community will continue to pursue
Mail-From solutions independent of Sender ID.  SPF code has
been written and has been packaged into the majority of
opensource MTAs.  If I cannot offer a compelling argument
that their requirements are respected in Sender ID, I may
not be able to convince them to move toward SUBMITTER and
PRA.  Of course, I may not need to do anything; when the
RFCs are out, there is a certain expectation that they will
be followed.

I understand that many participants in this working group
have, during the course of the standards process,
accumulated a fair amount of frustration.  I am not
bringing up these issues because to be contrarian.  I am not
bringing up these issues to further some secret agenda.  I
believe that Sender ID is very close to being good enough
for nearly everyone.  I'm just not sure that it's there yet,
and I want to help it go the rest of the way.

If we leave these issues unaddressed, I fear that an
important constituency will simply respond by declining to
adopt our standard.

I fear that if we are not explicit about exactly what the
value proposition is, people who buy into the system
expecting a silver bullet will accuse us of false
advertising.

I fear that if we do not think through the full range of
WWSD scenarios and answer each one convincingly, the
solution will not last.

I am bringing up these issues because I hope that they can
be avoided.

I recognize that perhaps this input is too late, not
appropriate to this forum, or places too high a value on
cooperation between different communities.  If you feel that
this message has not been constructive, I apologize in
advance; please understand that I am doing what I think is
right.  I will not raise these issues again; all you have to
do is not reply, and it will go away.

On a separate note, I have heard that EzMLM and YahooGroups
lists do not pass the PRA test.  The new core draft may
benefit from an applicability statement that waives the PRA
test for recognized mailing lists.

---

From: Dave Crocker <[email-masked]>
Date: Wed, 11 Aug 2004 13:10:02 -0700

Tony,

TF> The reason Sender-ID falls back to checking MAIL FROM is because SUBMITTER
TF> may be omitted if it is the same as MAIL FROM. If a Sender-ID
TF> implementation is checking the MAIL FROM in the absence of SUBMITTER,
TF> should it use the v=spf1 record or the v=marid1 record?

 One of the more interesting errors that we all have made in designing
 protocols is the pseudo-efficiency hack of using defaults.

 With unfortunate frequency, I keep seeing this "assumption" about
 values come back to bite us.  From/Sender is merely one example.
 (Another is "default" character set for MIME, but that's another
 diatribe.)

 Might be nice for a new protocol to reflect something from the learning
 curve, and avoid this mistake. No Submitter field should mean no
 submitter value in rfc2821 headers, rather than 'look at mailfrom'.

d/

---

From: Mark Lentczner <[email-masked]>
Date: Wed, 11 Aug 2004 14:38:38 -0700

First, some comments:

Alas, due to the way ESMTP extensions work, there is no way to know if 
the client supports the extension unless it uses it.  For one reasons 
I'm not aware of, the current state of the draft allows four states:
	The client doesn't understand SUBMITTER
	    1) there is no SUBMITTER=
	The client does understand SUBMITTER and:
	    PRA == MAIL FROM and
	            2) the client leaves SUBMITTER= off
	        or  3) the client includes SUBMITTER= anyway
	    PRA != MAIL FROM
	        4) the client includes SUBMITTER=
Alas, there is no way to distinguish state 1 from 2.  Sure, things 
would be tighter if we disallow state 2.  I'm not sure this is a big 
deal.  If you leave state 2 in, then many environments where PRA always 
equals MAIL FROM needn't change any software and can claim compliance.  
Of course, I'm not sure there are really any such domains.

I also tend to agree that any language about what might happen if we 
require SUBMITTER in the future should be left out of the draft unless 
we actually agree on what will happen, and when we will do it.

Second, In the language of pseudo-code, Here are is what I think 
combined Sender-ID and SUBMITTER requires an MTA to do:

accept body of message
extract PRA from headers
if the client supplied SUBMITTER= {
     if SUBMITTER != PRA { exit with fail }
}
r0 = check PRA against the Sender-ID record
exit with r0

Of course, this doesn't gain any possible advantage from SUBMITTER.  
This is what an aggressive MTA can do:

if client supplies SUBMITTER= {
     r1 = check SUBMITTER against its Sender-ID record
     if r1 is pass {
         if SUBMITTER is on your white-list, or has a high enough 
reputation score {
             accept body of message
             exit with pass, no further checks needed
         }
     }
     else if r1 is fail {
         reject mail (still prior to DATA)
         exit with fail
     }
     -- r1 is either unknown or error at this point --

     accept body of message
     extract PRA from headers
     if PRA != SUBMITTER { exit with fail }
     if PRA == SUBMITTER { exit with r1 }
}
else client supplies just Mail From {
     r2 = check MAIL FROM against its Sender-ID record (hoping it will 
be same as PRA)

     accept body of message
     extract PRA from headers
     if PRA == MAIL FROM { exit with r2 }

     r3 = check PRA against its Sender-ID record
     exit with r3
}

This is more convoluted than the draft requires because it is trying to 
stave off as much work as possible.  Specifically, if the reputation is 
high, or is on a white-list, then the PRA extraction from the headers 
can be skipped (as your local mail policy essentially trusts that 
domain's machines (as authorized via Sender-ID) to do the right thing.)

This code also tries to get as much going in parallel as possible.  Not 
only is it getting the SUBMITTER (if present) check going quickly; a 
site can speculatively try MAIL FROM (in the absence of SUBMITTER), in 
hopes that by the time it gets the headers, the result will be back, 
and MAIL FROM will match PRA.

If we change the spec to eliminate state 2 above (where a client that 
understands SUBMITTER omits it if PRA == MAIL FROM), then really, 
NOTHING in the above code changes.  An MTA can't be any more 
aggressive.

In summary (did you ever think I'd get to one?):
- I think Meng's interpretation is wrong.
- Removing the ability to omit SUBMITTER if PRA equals MAIL FROM 
doesn't really matter
- As defined, this all works to save processing time for a receiver if 
it a) the MTA can take advantage of starting Sender-ID checks in 
parallel with receiving the DATA, and/or b) the receiving domain has a 
white-list or a reputation system it trusts.

	- Mark

Mark Lentczner
http://www.ozonehouse.com/mark/
[email-masked]

---

From: "Michel Bouissou" <[email-masked]>
Date: Thu, 12 Aug 2004 00:24:08 +0200 (CEST)

Meng Weng Wong a écrit :
> lists do not pass the PRA test.

I believe the Sympa MLM (http://www.sympa.org) doesn't pass it either,
because it doesn't add any "Sender:" nor "Resent-From:" nor whatever
header that PRA uses. Although this can probably be easily "fixed in a
future version", this would be "one more thing that will need and upgrade"
to be PRA compliant.

> The new core draft may benefit from an applicability statement that
> waives the PRA test for recognized mailing lists.

Then you have to define "recognized mailing lists", which might be a
matter for yet another draft.

If a candidate standard has to start its life by making exceptions for a
number of situations, I believe it shows that it is basically flawed.

-- 
Michel Bouissou <[email-masked]> OpenPGP ID 0xDDE8AC6E

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 11 Aug 2004 16:33:05 -0700 (PDT)

> So maybe "people" are reading the specs differently than Meng? ;)

Then the intent and the letter of the spec differ.  This has to be fixed,
or this will fall flat.

Your average implementor is not a mind-reader and is probably lazy enough
that she/he would not go through the list archive, and thus may (read:
likely would) miss such a subtlety.

Please revise the spec.

---

From: [email-masked]
Date: Wed, 11 Aug 2004 16:22:57 -0700 (PDT)

> Alas, there is no way to distinguish state 1 from 2.

I was about to point this out... Thanks for doing the analysis.

> Sure, things

Mailings lists are pretty common.

>          if SUBMITTER is on your white-list, or has a high enough reputation score {
>              accept body of message

Nit: r1 can also be pass at this point.

>      r2 = check MAIL FROM against its Sender-ID record (hoping it will be same as PRA)

As you note below, this part isn't exactly required, but it does offer the
highest degree of operational overlap.

> - I think Meng's interpretation is wrong.

Agreed.

> - Removing the ability to omit SUBMITTER if PRA equals MAIL FROM

Agreed.

> white-list or a reputation system it trusts.

Yep.

				Ned

---

From: Jim Fenton <[email-masked]>
Date: Wed, 11 Aug 2004 17:10:05 -0700

Mark, thanks; great analysis of how this works.

At 02:38 PM 8/11/2004 -0700, Mark Lentczner wrote:
>This is more convoluted than the draft requires because it is trying to stave off as much work as possible.  Specifically, if the reputation is high, or is on a white-list, then the PRA extraction from the headers can be skipped (as your local mail policy essentially trusts that domain's machines (as authorized via Sender-ID) to do the right thing.)

Just to emphasize the converse, PRA extraction from the headers will still be required unless the reputation is high or on a white-list.  Some of the comments I have seen on this list would lead one to believe that extracting the PRA and checking Submitter against it is optional.  In fact, not to do so with a sender you don't know well would greatly decrease the effectiveness of Sender ID by spoofing the PRA while using Sender with an authorized domain for the source address of the SMTP client.

-Jim

---

From: Tony Finch <[email-masked]>
Date: Thu, 12 Aug 2004 11:43:39 +0100

> Alas, there is no way to distinguish state 1 from 2.  Sure, things would be
> tighter if we disallow state 2.  I'm not sure this is a big deal.

It makes it much more likely that there will be bugs and interop problems
that prevent people from using Sender-ID in interesting ways. I'm working
on techniques like BATV that rely on totally different verification rules
for the PRA and the bounce address. A clever SPF record coupled with a
custom DNS server allows SPF-aware servers (i.e. which check the bounce
address as opposed to the PRA) to verify signed bounce addresses without
an SMTP callback. If there is confusion in the protocol about whether the
PRA or bounce address is being checked then this becomes hopelessly
unreliable.

> I also tend to agree that any language about what might happen if we require
> SUBMITTER in the future should be left out of the draft unless we actually
> agree on what will happen, and when we will do it.

It's utterly foolish to expect that the Internet will follow your
timetable.

Tony.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 12 Aug 2004 08:33:05 -0700

Mark,

ML> I also tend to agree that any language about what might happen if we 
ML> require SUBMITTER in the future should be left out of the draft unless 
ML> we actually agree on what will happen, and when we will do it.

just to flog this issue mercilessly:

  Since "we" (the IETF community) have no control over rates of adoption
  by the Internet product and services community, any attempt to write a
  schedule for adoption will be wrong. Certainly the IETF community's
  track record at predicting adoption rate has always been wrong.

  The one exception is the assumption that adoption will take much
  longer than we assume...

d/

---

From: Meng Weng Wong <[email-masked]>
Date: Fri, 13 Aug 2004 02:56:04 -0400

I agree with everything Mark says, so I must have
communicated my position poorly :)

If I could have one more chance to explain my thinking,
let's cut to the chase:

On Wed, Aug 11, 2004 at 02:38:38PM -0700, Mark Lentczner wrote:
| else client supplies just Mail From {
|     r2 = check MAIL FROM against its Sender-ID record (hoping it will 
| be same as PRA)
| 
|     accept body of message
|     extract PRA from headers
|     if PRA == MAIL FROM { exit with r2 }
| 
|     r3 = check PRA against its Sender-ID record
|     exit with r3
| }

I agree that this is how Sender ID should work.

My question is: can you reject if the r2 result is FAIL?

I think you can.  Here's why.

The forwarding scenario is, of course, our weak link.

Under SPF Classic, noncompliant forwarders who didn't do SRS
could expect to get their mail rejected.

Under Sender ID, noncompliant forwarders who don't prepend a
Resent-From header could expect to get their mail rejected.

Under Sender ID, noncompliant forwarders who don't offer a
SUBMITTER parameter could expect to get their mail rejected.

In all cases, noncompliant forwarders get their mail
rejected.

Therefore, immediately rejecting after MAIL if (the
SUBMITTER value is not provided and r2 is FAIL), is no worse
than continuing to accept the message and extracting the
PRA, because the PRA can be expected to fail anyway if it's
forwarding.

This means that we *can* safely assume,
in the absence of SUBMITTER, that MAIL-FROM == PRA,
whether or not we think the sender is omitting it because
1) they are not Sender-ID aware, or
2) they are aware and Mail-From == PRA.

If we do this, then the essential requirements of the SPF
Classic community are respected in Sender ID.

I do not think this course of action harms the essential
requirements of Caller ID so why not make both sides happy?

Back to the algorithm:

| else client supplies just Mail From {
|     r2 = check MAIL FROM against its Sender-ID record (hoping it will 
| be same as PRA)

so, if the r2 is FAIL, reject.  if the r2 is PASS, proceed.

|     accept body of message
|     extract PRA from headers
|     if PRA == MAIL FROM { exit with r2 }
| 
|     r3 = check PRA against its Sender-ID record
|     exit with r3
| }

the important requirement for me is being able to, when
SUBMITTER is absent, reject based on a MAIL FROM fail
result, and being able to, when SUBMITTER is absent, accept
based on a MAIL FROM pass result.

If course if SUBMITTER is provided that overrides the
return-path, but i really would like to be able to avoid
going into the headers upon FAIL.

If the result is a PASS then going into the headers is fine
by me; we would then check that the PRA either matches the
MAIL FROM or passes an SPF check directly.

(non-SUBMITTER compliant VERP mailing lists with Sender:
owner-listname are a good case where PRA != MAIL-FROM but
spf_check(PRA) == PASS == spf_check(MAIL-FROM)).

perhaps some data can tell whether such an algorithm would
be more or less prone to error in the field.

---

From: "Andrew W. Donoho" <[email-masked]>
Date: Fri, 13 Aug 2004 10:30:08 -0500

Meng and MARID Community,

> requirements of Caller ID so why not make both sides happy?

	As a new participant who attened the MARID F2F, I would like to 
suggest that Meng's interpretation should be cast in the Sender-ID 
specification. It currently is not. Judging by Mr. Lyon's (jimbo, 
[email-masked]) comments, he has a very different 
interpretation and expects Meng's Mail-From checking to occur some time 
in the vague future, if at all. This does not sound like the parties to 
the merging of the SPF and Caller-ID specs have the same 
interpretation. Because premature marketing activities are being 
introduced today before the Sender-ID spec is finalized nor the IPR 
issues determined, there is a very real chance that marketing may 
overwhelm the standards process. We should tread carefully here. The 
rush to specification for Sender-ID seems ill advised.

	The SPF community has some operational experience with the 
effectiveness of Mail-From checking. (I volunteered to work with the 
SMTP-Verify group in ASRG to document this experience. Please join me 
and help perform this important test of engineering assertions.) 
Furthermore, because Mail-From checking is conceptually simple, it 
offers implementation simplicity and a good method to harden the 
mapping between domain names and IP addresses for email. It is not the 
only solution. At the MARID meeting, I encouraged this community that 
if they were willing to extend SMTP with SUBMITTER then they should, at 
the same time, consider tightening up HELO/EHLO checking. From this 
list, Meng has also proposed that we eliminate sending to non-MX 
advertised hosts. Both of these issues should be considered before 
embracing the addition of new tags. Perhaps an ESMTP extension, say 
STRICT, could be used to announce both support of and adherence to 
HELO/EHLO checking and only sending to MX advertised hosts. (As I am, 
obviously, not an SMTP expert, I encourage others to make a real 
proposal.) I am just identifying that these loose interpretations of 
the SMTP spec have contributed to the abuse of the email system. 
Shouldn't we try to close those holes and exploit DNS as an important 
"third party" identifier/authenticator service to improve email?

	I am saying that simple mechanisms without IPR encumbrances should be 
exploited by this community. SPF-Classic and some form of stricter SMTP 
checking meet this criterion. The PRA and other mechanisms may offer 
value but, because of their complexity, should be deferred until these 
simpler mechanism's specifications are finalized. As the difference 
between Mr. Wong's and Mr. Lyon's interpretation of the Sender-ID spec 
shows, there is still some important conceptual work to be done here. A 
rush to deploy a complex implementation that may not have the desired 
anti-spam/anti-phishing effect is not good engineering practice. In 
other words, while we have an extremely rough consensus, the running 
code may not actually solve the identification problem. We should look 
to simplify and refine our solution.

Andrew

____________________________________
Andrew W. Donoho
[email-masked], PGP Key ID: 0x81D0F250
+1 (512) 453-6652 (o), +1 (512) 750-7596 (m)

---

From: "Carl S. Gutekunst" <[email-masked]>
Date: Fri, 13 Aug 2004 10:05:50 -0700

Andrew W. Donoho wrote:

> .... there is a very real chance that marketing may overwhelm the 
> standards process. We should tread carefully here. The rush to 
> specification for Sender-ID seems ill advised.
>
> ... The PRA and other mechanisms may offer value but, because of their 
> complexity, should be deferred until these simpler mechanism's 
> specifications are finalized. 

Thank you for the most rational and sensible post I've seen on this list 
in the past two months. Please, folks, can we pick some of the 
low-hanging fruit first?

Incidentally, Andrew, I tried to reply to you directly, but your SPF 
check bounces my mail. It does not appear that there is any way for me 
to send E-mail to you from any of my legitimate accounts.

<csg>

---

From: "Carl S. Gutekunst" <[email-masked]>
Date: Fri, 13 Aug 2004 10:29:47 -0700

> to send E-mail to you from any of my legitimate accounts. 

Sending to the whole list because I implied here Andrew was doing 
something wrong.

(*BLUSH*) My bad Andrew. The habeas.com SPF record is doing exactly what 
it's supposed to do, blocking anyone from sending E-mail with a MAIL 
FROM of habeas.com if it isn't sent out via the Habeas MX host.

I really don't think that our IT guy realized the consequences for those 
of us who mostly work remotely, but that's certainly not your problem.

<csg>

---

From: "John Glube" <[email-masked]>
Date: Fri, 13 Aug 2004 14:24:03 -0400

Meng wrote:

>SUBMITTER value is not provided and r2 is FAIL), is no
>worse than continuing to accept the message and extracting
>the PRA, because the PRA can be expected to fail anyway if
>it's forwarding.

>This means that we *can* safely assume, in the absence of
>SUBMITTER, that MAIL-FROM == PRA, whether or not we think
>the sender is omitting it because 

>requirements of Caller ID so why not make both sides happy?

I have two concerns:

* Technical - 

Meng, if we are going to say:

MAIL FROM = SUBMITTER

Therefore in the absence of SUBMITTER do checks on MAIL FROM

Why not:

* require malformed SMTP mail from checks as a pre-cursor
to avoid spammers from sending IP to recipient MX as was
the case in the first US Bank phish?

* allow for EHELO/HELO and PTR checks as well?

I realize my questions are a bit awkward especially given
Andy's previous comments, but if we are going to appease
the classic SPF community to solve a problem, why not also
appease the unified SPF community?

Also, how is the classic SPF or unified SPF community
appeased if MS has yet to commit on the license format?

(On this point, I acknowledge it was agreed the WG was
going to do the engineering work first and then deal with
the license issue, depending on the MS proposal, but ...)

In other words, from an engineering/design perspective,
either:

* the Sender-ID design is amended to fully meet the 'social
concern' and the 'redundancy design concerns;' and,

* MS fully commits on the license issue to meet the
concerns of the open source community; or,

* don't meet them at all and allow the design to stand as
it is, acknowledge that MS is going to file a defensive IPR
claim, while requiring a royalty free license in the format
it feels appropriate, then everyone can clearly understand
exactly what this means and the WG can make an informed
decision.

(I suspect my comments are a bit pre-mature and if they
are, I apologize.)

My own view is that:

* the design should be amended to fully meet the social and
redundancy design concerns.

* MS should commit to meet the concerns of the open source
community, while protecting its own interests.

Meng, I appreciate your position is a compromise. 

I both respect and applaud your effort. Hopefully something
is worked out which keeps the open source and corporate
communities in the same tent.

If what you propose works, great ...

* Implementation/Non technical - 

This question is for Harry,

Sender-ID is going to require some 're-jigging' of message
headers.

The US Federal Government, Australia and the European Union
all have written laws regulating commercial email.

All these laws tell people in essence not to use 'deceptive
headers.' 

The US Federal Law in particular has some very specific
prohibitions, especially as to what can and cannot go in
the from line, etc. 

Both Australia and the US are in the middle of their
process to write specific regulations.

In the EU, most nation states have transposed the EU guide.

Sender-ID is creating a new identity which did not exist
when the Australian, US and EU laws were written.

I have particular concerns, especially given some proposed
suggestions I have seen involving how ESPs should proceed
and whether these are or are not off side with the US
Federal law.

Has anybody bothered to check to make sure we are not
creating the ultimate catch-22? 

Yes, you can authenticate your identity as a sender, but by
complying with the Sender-ID protocol, you put yourself off
side with local law. Woops.

Harry, could you raise this with the lawyers at MS? I
appreciate this may be a 'ghost' concern, but ...

Implementation - PR

On the PR side, yes, it is disconcerting and in some cases
amusing to read a series of PR releases concerning
Sender-ID published yesterday and today in the online media
in which folks are making all kinds of statements,
including one ESP saying "we are Sender-ID compliant," when
the draft protocol is not finished and nobody is using
Sender-ID in the wild. 

But ... I am not sure there is much this group can do,
unless the WG chairs decide to issue a press release. Given
the last brouhaha which occurred after one of the chairs
was interviewed ... somehow I suspect silence is the rule
of thumb.

John

John Glube
Toronto, Canada

The FTC Calls For Sender Authentication
http://www.learnsteps4profit.com/dne.html

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.734 / Virus Database: 488 - Release Date: 04/08/2004

---

From: "Andrew W. Donoho" <[email-masked]>
Date: Fri, 13 Aug 2004 13:50:13 -0500

On Aug 13, 2004, at 12:05, Carl S. Gutekunst wrote:
>> ... The PRA and other mechanisms may offer value but, because of 
>> their complexity, should be deferred until these simpler mechanism's 
> Thank you for the most rational and sensible post I've seen on this 
> list in the past two months. Please, folks, can we pick some of the 
> low-hanging fruit first?
>

Carl,

	Thank you for the kind words.

	Here are the offending entries from my mail log.

11:58:46.69 4 SMTPI-80515(conure.mail.pas.earthlink.net) cmd: MAIL 
FROM:<[email-masked]> SIZE=2651
11:58:46.69 4 SMTPI-80515(conure.mail.pas.earthlink.net) 
SPF(habeas.com) checking
11:58:47.03 1 SMTPI-80515(conure.mail.pas.earthlink.net) Return-Path 
[email-masked]' rejected: sender domain does not match SPF records
11:58:49.03 4 SMTPI-80515(conure.mail.pas.earthlink.net) rsp: 587 
[email-masked] sender domain does not match SPF records
11:58:49.03 4 SMTPI-80515(conure.mail.pas.earthlink.net) cmd: RCPT 
TO:<[email-masked]>
11:58:51.03 4 SMTPI-80515(conure.mail.pas.earthlink.net) rsp: 501 
Unknown command
11:58:51.03 4 SMTPI-80515(conure.mail.pas.earthlink.net) cmd: DATA

Here is your SPF record:

habeas.com.             76012   IN      TXT     "v=spf1 a mx ptr 
ip4:66.120.210.129/27 -all"

In other words, my mail server did what your policy said to do - 
reject. You may want to talk to your DNS administrator.

Best Regards,
Andrew

---

From: "John Glube" <[email-masked]>
Date: Fri, 13 Aug 2004 20:15:06 -0400

Harry,

If one goes to:

http://www.anti-spamtools.org/SenderIDEmailPolicyTool/Default.asp
x

A wizard has been set up to assist folks in publishing what
is called an 'E-mail policy document' which can be used by
people who have implemented Sender-ID

I found this link by going to:

http://www.microsoft.com/mscorp/twc/privacy/spam_senderid.mspx

(It is referenced under related links.)

If you run through the wizard, at the last page, you will
find the following statement:

Your E-Mail Policy Document must be published in DNS
records of type TXT in the subdomain "_ep" under your
domain. Consult the documentation of your DNS
administration tools for further details on publishing this.

It is my understanding:

* the marid protocol for SPF (to be renamed Sender-ID) does
not require publishing the 'E-Mail Policy Document' in the
subdomain "_ep." 

* this is a carry over from the Caller-ID protocol.

* with the merger of SPF and Sender-ID this requirement was
dropped

With the media drive now underway to get the word out about
Sender-ID:

* if people go to the SPF site they will receive one
instruction for publishing an Sender Policy Framework in
DNS records of type TXT;

* if people go to the anti-spamtools.org site, people are
going to be lead to believe they have to publish their
'E-Mail Policy Document' in DNS records of type TXT in the
subdomain "_ep."

With the ongoing issue of a change in version string and
whether you have to publish one or two records, this raises
concerns.

I am not wanting to suggest any malice on anyone's part. 

I suspect what happened is that someone in setting up the
wizard simply read the wrong document.

If you scroll down the page at:

http://www.microsoft.com/mscorp/twc/privacy/spam_senderid.mspx

you will note that the: 

>Sender ID Draft Specification: MTA Authentication Records
>in DNS" 

is the version dated June 23, 2004 which is now out of date.

Can we get this rectified so that the wizard is based on
the most recent version of the protocol? 

Otherwise I suspect people are going to be publishing the
wrong records in the wrong place.

John

P.S. I also note the following additional statements on
this page:

http://www.microsoft.com/mscorp/twc/privacy/spam_senderid.mspx

concerning the wizard.

>Note: This tool is in beta, pending IETF approval. SPF
>records created with this tool will require later revisions.

>To check for the SPF record in incoming mail, your ISP or
>system administrator will need to update to Sender
>ID–compliant software to check for the Purported
>Responsible Address (PRA) of incoming mail.

>If you are a software developer and are interested in
>implementing the PRA check in your software, please review
>the terms of the Caller ID for E-Mail implementation
>license agreement before you begin. Please note that
>Microsoft does not require a license agreement for
>individuals, companies, or ISPs who wish only to publish
>their Sender ID SPF text records.

I simply reference this material in passing.

John Glube
Toronto, Canada

---

---

From: Margaret Olson <[email-masked]>
Date: Sat, 14 Aug 2004 17:39:25 -0400

Here's how I'm interpreting the practical implications of this, please 
tell me if I'm just misunderstanding you:

If I want my record  to be evaluated based on the PRA algorithm then I 
MUST use SUBMITTER
If I want my record to be evaluated based on MAIL FROM then I MUST NOT 
use SUBMITTER and my PRA MUST be the same as the MAIL FROM.

How does this work when the receiving side doesn't understand SUBMITTER?

Margaret.

---

From: "John Glube" <[email-masked]>
Date: Sun, 15 Aug 2004 13:11:10 -0400

Margaret Olson wrote on August 14, 2004 at 5:39 PM

>Here's how I'm interpreting the practical implications of this,
>please tell me if I'm just misunderstanding you:

>If I want my record to be evaluated based on the PRA algorithm
>then I MUST use SUBMITTER

>If I want my record to be evaluated based on MAIL FROM then I
>MUST NOT use SUBMITTER and my PRA MUST be the same as the MAIL
>FROM.

>How does this work when the receiving side doesn't understand

Okay, let me venture forth ... and kick the ball a bit
further down the field and ask some additional questions:

* Based on the newest drafts, is it correct to state that
Sender-ID will only read the spf/pra02 version string. 

* Prudent senders will therefore have to publish a v=spf1
version string in the TXT record and spf/pra02 in the DNS
RR type SPF2.

* Will Submitter compliant Senders only use Sender, or will
you want to include SMTP Mail From in case the receiver
can't read Submitter and is only using SPF?

* Would the best practice be to have the domain in Sender
and SMTP mail from the same?

* What about CSV? 

Should prudent senders also be publishing the version
string for CSV?

How will Submitter compliant senders relate to receivers
for example who are using CSV and SPF, but not Sender-ID
for licensing reasons?

I raise these questions as:

* The WG has apparently decided to go down the consecutive
as opposed to the concurrent path.

* This means each protocol will use its own version string,
as opposed to one version string serving all protocols.

* We are now confronted which inter-operability issues, so
making things even more complex.

* These are questions which need to be answered.

In my view, while it is all very nice to say:

* Sender-ID will only read sender-id records and CSV will
only read CSV records, 

We need to ensure a sender can transmit to someone who is
relying on any one or more of:

SPF, 

(Even if SPF is not a formal protocol, it exists in the
wild and unfortunately the most recent Sender-ID protocol
version does not to the best of my understanding give even
the minimal compromise that Meng was asking for to carry
the SPF community forward with Sender-ID. Therefore, please
don't be surprised if ultimately many in the SPF open
source community simply respond by rejecting Sender-ID.),

Sender-ID and CSV, 

(which don't yet exist in the wild to any significant
extent), 

Otherwise we will end up with a mess.

Thanks

John

John Glube
Toronto, Canada

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.737 / Virus Database: 491 - Release Date: 11/08/2004

---

From: "Chris Haynes" <[email-masked]>
Date: Sun, 15 Aug 2004 19:36:12 +0100

Forgive me for suggesting a clarification to one of your assertions.

Your assertion

    " If I want my record to be evaluated based on MAIL FROM
     then I MUST NOT use SUBMITTER
     and my PRA MUST be the same as the MAIL FROM."

implies that Sender-ID contains a distinct process for evaluating MAIL-FROM (as
in SPF).

I believe this not to be the case.

In an off-list communication Mark Lentczner pointed out that:

"The MAIL-FROM domain check is only a speculative optimization.  If it doesn't
match PRA, it isn't used."

As I understand it, if there is no SUBMITTER, the domain of a MAIL-FROM is used
as the domain from which the record is extracted, but the way in which this
record is then processed is as if this were the PRA-derived domain.

When the PRA later becomes available, its domain is compared with that derived
from the MAIL-FROM.

If they are different, the record and any provisional test results derived from
the MAIL-FROM are discarded, and the record for the domain identified by the PRA
is now collected and tested.

In other words that is no sense in which any record is ever "evaluated based on
MAIL-FROM"; the tests / algorithms applied will always be those associated with
the PRA.

The optimization derived from using MAIL-FROM is the opportunity to fetch and
start testing the record, in the hope that, when the PRA is eventually
determined, the correct record is found to have been used.

No definitive tests are concluded at 2821-time, the information derived from
MAIL-FROM can never result in a decision at 2821-time to reject the message, so
the SMTP DATA phase and PRA establishment will always take place.

If the domain of MAIL-FROM is different from that of the PRA, its record has no
functional impact whatsoever on the evaluation of the message - which is, I
trust, a fair re-phrasing of Mark's observation.

If SUBMITTER is used, it is possible to reject some messages at 2821-time (see
section 4.2 of draft submitter-03.txt) and thereby save the bandwidth / time of
the DATA phase - so I can see no benefit whatsoever in a client deliberately
omitting the SUBMITTER when the receiver is prepared to accept it.

Chris Haynes

---

From: John Leslie <[email-masked]>
Date: Sun, 15 Aug 2004 18:55:12 -0400

>   Sender-ID will only read the spf/pra02 version string. 

   Except for the spelling, which is "v=spf2.0/pra", yes.

>   version string in the TXT record and spf/pra02 in the DNS

   I have no opinion on whether spf1 check_host() will ever read something
other than TXT records; but this is a likely scenario, yes.

>   can't read Submitter and is only using SPF?

   These seem to me to be quite separate issues; but no doubt some will.

>   and SMTP mail from the same?

   I have no opinion.

> * What about CSV? 

   There, I _have_ an opinion. ;^)

> Should prudent senders also be publishing the version

   So far, I've been telling people to stay tuned: I'm not yet sure
whether the format of CSV publishing in draft-ietf-marid-csv-csa-01.txt
will be the final output of this Working Group.

   But once we agree on the final format, yes, operators of MTAs should
IMHO be publishing CSV as well as Sender-ID.

   CSV will verify that the MTA is specifically authorized to act as
a SMTP client, in such a manner that reputation services can closely
track whether the domain publishing CSV record(s) has policies and
practices which minimize email problems such as spam and viruses; and
reputation reports can quickly reflect both problems and their
resolutions.

   So, IMHO, anyone wishing to publish Sender-ID records _should_ also
be publishing CSV (once we've agreed upon it, of course).

> for example who are using CSV and SPF, but not Sender-ID

   At first blush, I'd assume they'll simply continue publishing CSV
and "v=spf1". (It's nearly certain there _will_ be such receivers.)

>   as opposed to the concurrent path.

   This is clearly what the Chairs are asking us to do.

>   as opposed to one version string serving all protocols.

   That seems hard to avoid...

>   making things even more complex.

   IMHO, inter-operability is probably easiest this way.

> * These are questions which need to be answered.

   I really don't think the answers I've given are controversial.

   What is controversial is that we find ourselves complicating the
task of publishing MARID-appropriate DNS records (which is quite
contrary to the original aim of SPF).

> SPF, ...
> Sender-ID and CSV, ...

   The current path, with three different records being published, is
fully workable; and IMHO is nothing to be ashamed of as Working Group
output.

--
John Leslie <[email-masked]>

---

From: "John Glube" <[email-masked]>
Date: Sun, 15 Aug 2004 22:01:59 -0400

John,

I appreciate your taking the time to confirm and in some
cases clarify my understandings.

One follow up question. You wrote:

>What is controversial is that we find ourselves
>complicating the task of publishing MARID-appropriate DNS
>records (which is quite contrary to the original aim of
>SPF).

If we accept the position of separate DNS records for each
protocol, can you elaborate on the controversy as you see
it?

I am not wanting to put you on the spot. So let me outline
the situation as I see it and then perhaps you could
provide corrections, comments and so forth.

Here goes:

Prior to the present draft of the Marid protocol, marid -
appropriate DNS records were published using v=spf1 in the
DNS IN TXT file.

(My apologies if I have not stated this correctly.)

A receiving MTA could either with or without Submitter: 

* Use these DNS records to run an SPF check, although not
specified in the Marid core protocol; 

* If a negative result was returned, reject the message and
send a rejection notice in accordance with the SPF protocol;

This could be done at the DATA stage, so arguably saving
bandwidth?

* If a pass or neutral result was returned, proceed to
'swallow' the message and run a Marid core check.

This gave the receiving MTA the benefit of SPF, along with
PRA checks.

Under the present draft of the Marid protocol, marid -
appropriate DNS records should be published using
v=spf2.0/pra in the DNS RR type SPF2 file, although the
protocol does allow for TXT type records in the SPF2 file.

However, by changing the version string, although the
receiving MTA may decide to extract the domain in SMTP mail
from and run a check against the SPF record in the DNS IN
TXT file, if the domain has not published an SPF record,
because it is not required:

* Have we lost the potential optimization which was
previously available?

* Does the optimization suggested by Chris in his note to
Margaret, based on his offline discussion with Mark fully
replace this earlier optimization? 

http://www.imc.org/ietf-mxcomp/mail-archive/msg03299.html

(There is a bad pun in there, but I'll skip it.)

It seems it could, but only if the domain extracted from
the SMTP mail from is the same as the purported responsible
domain as identified in the marid appropriate DNS records?

Is this the controversy you speak about, or is it something
else?

I apologize if my questions are putting anyone on the spot.

I simply want to ensure the understandings are clear, so
that receivers can optimize their results and prudent
senders can accommodate this objective.

I appreciate your taking time to respond.

John

John Glube
Toronto, Canada

---

---

From: Mark Lentczner <[email-masked]>
Date: Sun, 15 Aug 2004 19:02:19 -0700

>    Except for the spelling, which is "v=spf2.0/pra", yes.

Actually, "spf2.0/pra" -- we dropped the "v=".

	- Mark

---

From: "Chris Haynes" <[email-masked]>
Date: Mon, 16 Aug 2004 11:26:39 +0100

Meng,

Please may I ask a brutally simple question.

I think it is generally agreed that where MAIL-FROM == PRA any decision is based
on the PRA.

My question below applies only to those messages in which the domain of the
MAIL-FROM is different from the domain of the PRA.

Where, in the collection of drafts issued on Friday, does Sender-ID refer to the
rejection of a message as a result of evaluating the record associated with its
MAIL-FROM domain?

Chris Haynes

At Friday, August 13, 2004 7:56 AM "Meng Weng Wong" said

---

From: John Leslie <[email-masked]>
Date: Mon, 16 Aug 2004 15:00:19 -0400

John Glube <[email-masked]> wrote:
> "'John Leslie'" <[email-masked]> [wrrte:]
> 

   The controversy is pretty quiet actually; but the pain level is
obvious. Any number of people would like to avoid the MARID WG
recommending publication of multiple records.

> This could be done at the DATA stage, so arguably saving

   I'd rather not get into that argument. Regardless of bandwidth
savings, an error returned during the SMTP connection is "better"
because it's more likely to reach a responsible party.

> v=spf2.0/pra in the DNS RR type SPF2 file, although the
  ^^^^^^^^^^^^
   Mark already corrected me on that: there is in fact no "v=".

> protocol does allow for TXT type records in the SPF2 file.

   Yes, it should be understood that SPF2 is a "text" type, and
most domains will publish TXT RRs as well as SPF2.

> * Have we lost the potential optimization which was

   I'm not clear what "optimization" you mean, but we've probably
lost it.

>   replace this earlier optimization? 

   I'm not sure I understand what you refer to here. The obvious
sentence is:
] 
] The optimization derived from using MAIL-FROM is the opportunity to
] fetch and start testing the record, in the hope that, when the PRA
] is eventually determined, the correct record is found to have been
] used.

   This seems to imply that sender-ID checks could be run against
the MAIL-FROM when there is no submitter _before_ accepting the DATA.
I don't believe this is recommended in any of the current MARID I-Ds
(though it isn't actually prohibited).

   The potential for saving any appreciable bandwidth escapes me,
unless we're talking about serious tar-pitting, which IMHO is quite
out of scope for MARID discussions.

   Somebody may have an opinion on whether this replaces the earlier
optimization, but I certainly don't.

> domain as identified in the marid appropriate DNS records?

   It's more restrictive than that actually. Only if the PRA turns
out to be exactly the same as the MAIL-FROM are you on safe ground.
(There is some hand-waving about cacheing SPF2 results if no macros
are involved...)

> Is this the controversy you speak about, or is it something

   No, this is not what I spoke of.

   The visible part of the iceberg of controversy is the numerous
occasions where the CSV folks have been asked why the SPF record
isn't "good enough" to serve the function of the CSV SRV record.
Also, there has been _extensive_ discussion of whether we actually
need to change the version string when moving from SPF to caller-ID.
That's the controversy I refer to.

---

From: "Chris Haynes" <[email-masked]>
Date: Mon, 16 Aug 2004 21:08:34 +0100

> > "'John Leslie'" <[email-masked]> [wrrte:]

<snip>

> ] is eventually determined, the correct record is found to have been

The wording above is my understanding / interpretation of the optimisation
described by Mark Lentczner in his pseudo-code in
http://www.imc.org/ietf-mxcomp/mail-archive/msg03230.html
and which is applicable when no SUBMITTER has been used.

Inspect carefully his use of test 'r2' and note that this test result cannot be
acted upon until after the PRA is available and the initial 'guess' has been
proved correct.

> out of scope for MARID discussions.

All I believe it can save is time - the record can be being fetched from DNS and
evaluated in parallel with the SMTP DATA phase.

If it turns out that MAIL-FROM != PRA, bandwidth (and DNS service resource) will
have been wasted, as the first test result will be discarded.

> optimization, but I certainly don't.

As I understand it, the central reason for introducing SUBMITTER was to give
Sender-ID the opportunity of matching the SPF optimisation of  being able to
reject some messages before the SMTP DATA phase - saving bandwidth.  I presume
this is what you refer to as the 'earlier optimisation'.

If no SUBMITTER is used, I believe Sender-ID must always proceed to the DATA
phase, as this is when the information defining the PRA is supplied.

<snip>

HTH

Chris Haynes

---

From: "John Glube" <[email-masked]>
Date: Mon, 16 Aug 2004 16:54:07 -0400

From: John Leslie Sent: August 16, 2004 3:00 PM

John Leslie wrote in part:

>The visible part of the iceberg of controversy is the
>numerous occasions where the CSV folks have been asked why
>the SPF record isn't "good enough" to serve the function of
>the CSV SRV record. Also, there has been _extensive_
>discussion of whether we actually need to change the version
>string when moving from SPF to caller-ID. [Sender-ID] That's
>the controversy I refer to.

My square brackets added. 

What? You mean the CSV folks were accused of being a bunch
of slaveens by the SPF folks for not sharing. Those uppity
come from aways. Flat on the back with that:-)

(I lived in Newfoundland for 18 years, affectionately known
as the Rock. Canada's distinct 'English' province, being a
former Dominion before 1949. Newfoundland English or
Newfinese is the native tongue.)

Appreciate the clarification. I am familiar with the
Sender-ID change of version string controversy. I can go
through the archives to review the CSV/SPF discussions.

John

John Glube
Toronto, Canada

---

---

From: Mark Lentczner <[email-masked]>
Date: Mon, 16 Aug 2004 13:52:20 -0700

> trust, a fair re-phrasing of Mark's observation.

Just to be technically precise: In general the test must be that 
MAIL-FROM must equal PRA, not just the domains.  If, when checking the 
MAIL-FROM address, it is determined that the Sender-ID records for the 
domain of MAIL-FROM don't rely on the local-part, then merely the 
domains of MAIL-FROM and PRA can be compared.

	- Mark

---

From: Meng Weng Wong <[email-masked]>
Date: Tue, 17 Aug 2004 10:25:35 -0400

On Mon, Aug 16, 2004 at 11:26:39AM +0100, Chris Haynes wrote:
| 
| Please may I ask a brutally simple question.
| 
| My question below applies only to those messages in which the domain of the
| MAIL-FROM is different from the domain of the PRA.
| 
| Where, in the collection of drafts issued on Friday, does Sender-ID refer to the
| rejection of a message as a result of evaluating the record associated with its
| MAIL-FROM domain?

Sender ID as currently specified doesn't do that.

If people want the MAIL FROM to play a role in Sender ID,
they should say so.  If they do not say so, the working
assumption is that nobody cares.

I personally think that if the PRA lookup returns "none" or
"unknown", MAIL FROM should be checked, and if that test
returns "fail" then the message should be rejected.

This is in line with the "unified SPF" model.

The converse is the scenario where a MAIL FROM pass is
returned, and a receiver arrives at a generally positive
feeling about the sender; that would allow EzMLM and similar
MLMs to get their mail through, and reduce the false
positives that would occur on a PRA check alone.

---

From: Rand Wacker <[email-masked]>
Date: Tue, 17 Aug 2004 08:03:25 -0700 (PDT)

> returns "fail" then the message should be rejected.

You mean if the message has *no* From, Sender, Resent-From, or
Resent-Sender headers, or if there is no SUBMITTER argument?

-Rand

---

From: Meng Weng Wong <[email-masked]>
Date: Tue, 17 Aug 2004 11:11:14 -0400

On Tue, Aug 17, 2004 at 08:03:25AM -0700, Rand Wacker wrote:
| On Tue, 17 Aug 2004, Meng Weng Wong wrote:
| 
| > I personally think that if the PRA lookup returns "none" or
| > "unknown", MAIL FROM should be checked, and if that test
| > returns "fail" then the message should be rejected.
| 
| You mean if the message has *no* From, Sender, Resent-From, or
| Resent-Sender headers, or if there is no SUBMITTER argument?
| 

I mean:

  MAIL FROM:<[email-masked]>
  Resent-From: <[email-masked]>
  From: <[email-masked]>

Given that clueless.com does not publish an SPF record, an
MTA operating solely on a PRA algorithm will let the message
through because the result of the PRA evaluation is "none".

Given that the installed base of MUAs cannot be expected to
display "Allegedly from [email-masked] via
[email-masked]", for the next few years the
average user will see in their MUA only

  From: <[email-masked]>

If an MTA were allowed to do SPF checks on the MAIL FROM
when the spf/PRA result is "none" or "unknown", this
scenario would be defeated.  Upgrading MTAs is quicker than
upgrading MUAs.

That shifts the spammer position to:

  MAIL FROM:<[email-masked]>
  Resent-From: <[email-masked]>
  From: <[email-masked]>

That is a small but significant improvement:

1) it tells clueless.com that they need to set up SPF records

2) it keeps the existing SPF Classic userbase happy, and
   removes the need for a dual-standard deployment plan.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 17 Aug 2004 08:25:57 -0700

Rand,

>> returns "fail" then the message should be rejected.
RW> You mean if the message has *no* From, Sender, Resent-From, or
RW> Resent-Sender headers, or if there is no SUBMITTER argument?

The new PRA document makes clear that PRA derivation is based on
Re-sent, Sender and From.  Indeed, that is exactly what a reasonable
reading of RFC2822 (and RFC822 and RFC733) should cause one to expect.

What would be more than a little strange would be to impart additional
"source responsibility" meaning to a return address (rfc2821.mailfrom),
if re-sent,sender,from were problematic.

d/
--
 Dave Crocker <dcrocker-at-brandenburg-dot-com>
 Brandenburg InternetWorking <www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>

---

From: Roy Badami <[email-masked]>
Date: Tue, 17 Aug 2004 16:37:42 +0100

>>>>> "Meng" == Meng Weng Wong <[email-masked]> writes:

    Meng> Sender ID as currently specified doesn't do that.

    Meng> If people want the MAIL FROM to play a role in Sender ID,
    Meng> they should say so.  If they do not say so, the working
    Meng> assumption is that nobody cares.

Well, FWIW I intend to continue to publish SPFv1 records along side
Sender ID records for my personal domain, and will probably do
likewise with my employer's domains when I start publishing records
for them.

If people don't wish to check the MAIL FROM (eg because there is mail
forwarded into their domain through forwarders which are Sender ID
compliant but not SRS-compliant, or just because they don't like
SPFv1) then that's their choice.  But I no reason to stop publishing
the information necessary to allow MAIL FROM checks on my domains by
those who wish to perform them.

    Meng> I personally think that if the PRA lookup returns "none" or
    Meng> "unknown", MAIL FROM should be checked, and if that test
    Meng> returns "fail" then the message should be rejected.

Well, if you're only going to check the MAIL FROM when the PRA check returns 
has no Sender ID record, then that means I'll still have to continue
publishing SPFv1 records if I want to tell the world that *all* mail
with an envelope sender in my domain originates from one of my MTAs.

    Meng> This is in line with the "unified SPF" model.

Personally I think we should leave this as "for future study".  I can
live with Sender ID in its current form, and reopening the 'identity'
can of worms at this late stage will just delay Sender ID.

Let's get Sender ID out the door, and then we can think about
validating other identities.  It would be relatively easy to add other
identities to Sender ID in a backward-compatible way at a later date,
particularly given the new versioning mechanism.

      -roy

---

From: Tripp Cox <[email-masked]>
Date: Tue, 17 Aug 2004 11:44:31 -0400

>    removes the need for a dual-standard deployment plan.

I think Meng has highlighted an incredibly important practical issue 
with PRA and marid-core as presently defined.  I have recently been 
struggling to determine the ultimate utility of MARID to the service 
provider, without much to show for it.  Consider the following 
postulate:

A policy for authorizing a given message to be transported by a given 
agent cannot be sufficiently expressed by a single domain in many 
cases.  The PRA "algorithm" is a means to derive a single pointer to an 
expression of policy.  As such, the chances that a receiver can 
conclude an explicit permit/deny result is greatly diminished (i.e. the 
only potentially conclusive, explicit result is that the PRA points to 
an explicit deny policy).

By considering and respecting the policies of both (or perhaps even 
all) domains purporting to be senders, one could apply logical 
equations to yield greater conclusive results (e.g. the table I provide 
below for applying policies to the 2821 and 2822 From domains).

2821		2822
MAIL FROM	From:
-			-		=	Don't accept.
+			-		=	Don't accept.
-			+		=	Don't accept.
-			?~		=	Don't accept.
?~			-		=	Don't accept.
?~			?~		=	Inconclusive.
+			?~		=	Inconclusive.
?~			+		=	Inconclusive.
+			+		=	Accept.

This could be extended to consider the policies of Sender, Resent-From, 
or any other purported senders.

Tripp

---

From: Margaret Olson <[email-masked]>
Date: Tue, 17 Aug 2004 12:06:59 -0400

>    removes the need for a dual-standard deployment plan.

So this has nothing to do with the presence or absence of the SUBMITTER 
parameter, but rather with the presence of a SPF v2/PRA record for the 
PRA domain.

I'm in favor - unless I'm missing something it doesn't diminish the 
PRA, getting rid of dual standards is a good thing, and more net 
happiness in the world is also a good thing :-).

Margaret.

---

From: "Andrew W. Donoho" <[email-masked]>
Date: Tue, 17 Aug 2004 11:11:49 -0500

Meng,

	This statement does not make any sense to me. SPF is a 2821 time 
check. While the Sender-ID spec tries to eliminate it, Sender-ID is 
really a 2822 time check and it performs a different function than SPF. 
There is no reason that Sender-ID precludes an SPF check. In fact, if 
you think of the anti-spam problem as a security problem, then you want 
defense in depth. You want any mail to pass multiple checks. Therefore, 
I think many operators will check SPF before the DATA phase and 
Sender-ID, along with other anti-spam filters, after receipt. That is 
the beauty of SPF - it is simple and, hence, performs one thing well. 
While you made a deal to combine the two specs, that is not necessary 
and I think that the SPF community is uninterested in dropping a 
working simple protocol. They can coexist.

Andrew

---

From: Dave Crocker <[email-masked]>
Date: Tue, 17 Aug 2004 09:25:46 -0700

Meng,

MWW>   MAIL FROM:<[email-masked]>
MWW>   Resent-From: <[email-masked]>
MWW>   From: <[email-masked]>

MWW> Given that clueless.com does not publish an SPF record, an
MWW> MTA operating solely on a PRA algorithm will let the message
MWW> through because the result of the PRA evaluation is "none".

 That's not a PRA issue.

 You are suggesting a behavior that goes far outside of the PRA
 specification and is strictly within the spf/sender-id specification.

d/

---

From: Dave Crocker <[email-masked]>
Date: Tue, 17 Aug 2004 09:57:16 -0700

Tripp,

TC> A policy for authorizing a given message to be transported by a given 
TC> agent cannot be sufficiently expressed by a single domain in many 
TC> cases.  The PRA "algorithm" is a means to derive a single pointer to an 
TC> expression of policy.  As such, the chances that a receiver can 

1. The underlying problem that you are reacting to has to do with having
the entity that is posting a message be the one to authorize the path
the message follows.

2. What PRA does is to formalize a reality: there is only one entity
that did the latest posting of the message.  All of the other fields
that might be present and refer to a poster refer to entities that are
not relevant to the CURRENT transmission.

d/

---

From: "John Glube" <[email-masked]>
Date: Tue, 17 Aug 2004 15:22:09 -0400

From: Meng Weng Wong Sent: August 17, 2004 10:26 AM

>>My question below applies only to those messages in which
>>the domain of the MAIL-FROM is different from the domain of
>>the PRA.  Where, in the collection of drafts issued on
>>Friday, does Sender-ID refer to the rejection of a message
>>as a result of evaluating the record associated with its
MAIL->>FROM domain?"

>"unknown", MAIL FROM should be checked, and if that test returns
>"fail" then the message should be rejected.

>The converse is the scenario where a MAIL FROM pass is returned,
>and a receiver arrives at a generally positive feeling about the
>sender; that would allow EzMLM and similar MLMs to get their
>mail through, and reduce the false positives that would occur on
>a PRA check alone.

My comments:

* At present according to its site and related information filed
with the IETF, MS is claiming an IPR in Sender-ID, proposes to
file a defensive patent and is requesting software developers to
sign a royalty free license.

* We will not know whether MS will accommodate its required
license to meet any expressed concerns of the open source
community until August 23.

* Based on this, the only workable presumption, given
previous comments made to this list concerning the
licensing issue, is that some open source mail software
will not be able to accept the license and therefore will
not be able to alter their software to be compatible with
Sender-ID, as the protocol is presently on the table.

The change that Meng is saying he would like to see in the
protocol documents, which others have requested, brings with it a
number of benefits:

* It removes an apparent design flaw in Sender-ID by reducing the
risk of false positives.

This would seem to be a fundamental argument in support of Meng's
position, given the mandate of this WG.

As to the argument about mail forwarding, Meng has used the word
"should" not "must" in his proposal.

* It allows software developers within the open source community
to design plug-ins for mailing software that support Sender-ID
without having to adopt or support the PRA algorithm, simply
relying on the SMTP mail from check and by adopting the plug ins
available to support mail forwarding.

This means the license request by MS becomes a non-issue, except
for those in the open source community who wish to implement
Submitter and support PRA at the data transmission stage.

I simply point this out so that as requested by Harry Katz,
MS can take this matter into consideration as it works on
its final IPR/license proposal.

* It means that senders are not compelled to publish an SPF
record and a Sender-ID record.

Nice, but not fundamental.

* Most importantly, it strengthens the protocol from the
receiver's perspective, particularly large mail box providers and
large corporate network providers who are facing the greatest
crush and cost in the ongoing war against spammers.

Why? It means large mail providers can require PRA and SMTP mail
from checks using Sender-ID and EHELO/HELO checks using CSV.

This strengthens the receiver's hand, while further
reducing the risk of false positives.

What happens if this change is not made?

SPF is not before this WG as a formal protocol.

The approach put forward by Meng overcomes the problem
which many organizations will face if Sender-ID does not include
a formal SMTP mail from check in the absence of PRA at the data
stage.

Should they or should they not adopt and use a protocol,
namely SPF which allows for an SMTP mail from check, so reducing
the risk of false positives, but that has not been formally
approved by the IETF.

Without the requested change, organizations are put in a
box.

We want to follow the rules. However, by following the
rules we adopt a protocol which according to one of the designers
may result in an increased false positive rate. This is not in
our interest.

We can avoid the problem by implementing SPF and Sender-ID,
but then we are not following the rules and this puts us
into never, never land.

* Design Perspective. The underlying reason for having
syntax strings for each design is this allows each protocol
to focus on one issue. In the case of Sender-ID this is
PRA.

By writing a design for Sender-ID which focuses on PRA and
includes SMTP mail from this is a change in design perspective.

The question becomes, does this change bring with it any
negatives?

Given PRA is a new model version of SMTP Mail From (Both
SPF and Sender-ID break mail forwarding) I don't see any.

Having said this, I am not a design, security or
operational expert and there may be one or more actual
reasons which require consideration.

* Delay. By accommodating this request, will it further
delay the process of getting Sender-ID out of the box?

The design work has been done on running an SMTP mail from check.
Draft protocols are readily available. Open source plug ins are
available both for mail from and re-sent mail from. There is a
very lively community available to provide support.

* The merger of SPF with Caller-ID. Some are critical of
this.

What I do know is:

* We are where we are; and

* We have an opportunity to make the present proposal
better, without any downside.

As Margaret pointed out:

"and more net happiness in the world is also a good thing
:-).

For all these reasons, I support Meng's request.

John

John Glube
Toronto, Canada

---

---

From: Roy Badami <[email-masked]>
Date: Tue, 17 Aug 2004 22:15:08 +0100

>>>>> "Roy" == Roy Badami <[email-masked]> writes:

    Roy> Let's get Sender ID out the door, and then we can think about
    Roy> validating other identities.  It would be relatively easy to
    Roy> add other identities to Sender ID in a backward-compatible
    Roy> way at a later date, particularly given the new versioning
    Roy> mechanism.

Actually, the one change that might be worth considering for Sender ID
is to add an identity macro which always expands to the string
'pra'. This will make it less painful to produce backward compatible
extensions later.  (Sorry, I know Unified SPF has something like this
but I can't remember what it's called.)

    -roy