Subject: DEPLOY: Over-running TXT dataspace in FQDN (-protocol I believe)

From: Rand Wacker <[email-masked]>
Date: Wed, 25 Aug 2004 20:44:16 -0700 (PDT)

With regards to the Sender ID record being published as a TXT record for
the FQDN hostname (in a TXT record for domain.com), real world usage is
already showing that this dataspace is becoming populated with spfv1
records deployed by many sites in order to increase deliverability to
certain very large ISPs and others.  By publishing spfv2/pra records in
the same dataspace (domain.com), we are effectively halving the usable UDP
packet size for responses, since a TXT query for domain.com will try to
return *all* TXT records for domain.com in one record.

Real world deployment experience is that most network firewall operators
do not allow DNS queries over TCP except to trusted secondaries.  The
indicated wishes of large ISPs is that they do not want to have to fall
back to a TCP connection anyways for verifying incoming mail because it
will cause too much overhead.

My concern was mollified by people pointing out that "even AOL's SPF
record can fit in 160 bytes, so they can almost squeeze three parallel
records in to the same TXT UDP response packet."  In the past couple of
weeks I have talked to a few *major* .com sites (financial, media,
consumer) whose real-world outbound architecture is waaaaaaaay more
complex than AOLs.  In particular, with the number of global sites they
run, the number of third party delivery services they contract with, and
their general focus being on running their business instead of their
email, many of these large companies are going to have trouble fitting a
single SPF-like record in a UDP-sized query, let alone fitting it in half
that size.

My concern for Sender ID is that since SPF has already claimed the FQDN
TXT dataspace for itself (and since SPF is seeing parallel adoption
regardless of where Sender ID goes), trying to stuff a second Sender ID
record in the same data space will be problematic for some, and if we
ever try to evolve this to a third version then it will *never* fit.

The solution I would suggest is to put spfv2/pra records in a sub-domain
such as _marid.company.com.  While it would be nice to recommend that
people begin allowing TCP DNS queries, it is unlikely that the highest
volume sites would ever want to implement such.

This is a concern that has been voiced to me by several major sites that
are exactly the same set of sites we use as examples for where we need to
stop phishing from.  They are most likely going to publish *both* spfv1
and spfv2/pra records, and their initial investigation of their email
architectures has indicated that those records are going to be pushing
this real-world 240-byte limit (a 480 byte effective UDP packet split in
two)

-Rand

---

From: "John Glube" <[email-masked]>
Date: Thu, 26 Aug 2004 03:43:32 -0400

Rand,

From your discussions with managers of large email
infrastructures, who are seeing a problem with packet size;
along with the big ISPs who are not interested in using TCP
to receive email because of cost, does it make more sense
not to change the version string?

Let me phrase the question this way. My understanding from
listening to the audio of the face to face Marid in early
August, one of the major rationales for a version string
change was implementation. 

It was felt folks who implemented SPF were early adopters.
By changing the version string, this would freeze SPF and
folks would move ahead with Sender-ID.

What you are telling us is the direct opposite. 

Some of the Big ISPs are using SPF to detect email forgery
at the pre data stage, while telling folks, do you want
less content filtering? Publish an SPF record.

(Of course this achieves the desired objective of sender
authentication, thwarting spoofing and other steps taken to
hide identity, by getting everyone out in the open.)

With the premise of enhanced email delivery, along with
protecting against domain spoofing, SPF is gaining
implementation momentum among domain owners.

This leads one to the conclusion the solution is instead of
creating a sub domain for Sender-ID records, rather it is
not to change the version string.

Why? It seems the theory behind the justification for the
change does not reflect reality. Thoughts? Comments?

John

P.S. Another question. Since the Big ISPs do not want to
use TCP to check authentication for email delivery, does
this mean from a practical perspective, the Big ISPs would
prefer to create a mirror data base of authentication
records within their own networks and when a domain owner
makes a change, the domain owner's DNS server could
propagate the change through out the mirrored sites?

Please excuse my naiveté if I have asked a dumb question.

John Glube
Toronto, Canada
 
The FTC Calls For Sender Authentication
http://www.learnsteps4profit.com/dne.html

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Rand Wacker
Sent: August 25, 2004 11:44 PM
To: [email-masked]
Subject: DEPLOY: Over-running TXT dataspace in FQDN (-protocol I
believe)

With regards to the Sender ID record being published as a
TXT record for the FQDN hostname (in a TXT record for
domain.com), real world usage is already showing that this
dataspace is becoming populated with spfv1 records deployed
by many sites in order to increase deliverability to
certain very large ISPs and others.  By publishing
spfv2/pra records in the same dataspace (domain.com), we
are effectively halving the usable UDP packet size for
responses, since a TXT query for domain.com will try to
return *all* TXT records for domain.com in one record.

Real world deployment experience is that most network
firewall operators do not allow DNS queries over TCP except
to trusted secondaries.  The indicated wishes of large ISPs
is that they do not want to have to fall back to a TCP
connection anyways for verifying incoming mail because it
will cause too much overhead.

My concern was mollified by people pointing out that "even
AOL's SPF record can fit in 160 bytes, so they can almost
squeeze three parallel records in to the same TXT UDP
response packet."  In the past couple of weeks I have
talked to a few *major* .com sites (financial, media,
consumer) whose real-world outbound architecture is
waaaaaaaay more complex than AOLs.  In particular, with the
number of global sites they run, the number of third party
delivery services they contract with, and their general
focus being on running their business instead of their
email, many of these large companies are going to have
trouble fitting a single SPF-like record in a UDP-sized
query, let alone fitting it in half that size.

My concern for Sender ID is that since SPF has already
claimed the FQDN TXT dataspace for itself (and since SPF is
seeing parallel adoption regardless of where Sender ID
goes), trying to stuff a second Sender ID record in the
same data space will be problematic for some, and if we
ever try to evolve this to a third version then it will
*never* fit.

The solution I would suggest is to put spfv2/pra records in
a sub-domain such as _marid.company.com.  While it would be
nice to recommend that people begin allowing TCP DNS
queries, it is unlikely that the highest volume sites would
ever want to implement such.

This is a concern that has been voiced to me by several
major sites that are exactly the same set of sites we use
as examples for where we need to stop phishing from.  They
are most likely going to publish *both* spfv1 and spfv2/pra
records, and their initial investigation of their email
architectures has indicated that those records are going to
be pushing this real-world 240-byte limit (a 480 byte
effective UDP packet split in two)

-Rand

---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.737 / Virus Database: 491 - Release Date: 11/08/2004

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.737 / Virus Database: 491 - Release Date: 11/08/2004

---

From: Jim Fenton <[email-masked]>
Date: Thu, 26 Aug 2004 05:12:48 -0700

>ever try to evolve this to a third version then it will *never* fit.

I share this concern, but...

>volume sites would ever want to implement such.

I think it was brought up earlier that using a prefix like _marid would break the use of wildcards (can't do _marid.*.example.com).  But it wasn't clear (to me, at least) whether wildcards would work in this application, even without the prefix.  Wildcard support would be very nice to have, to provide symmetry with wildcard MX records for incoming mail.  Can anyone clear up whether adding the prefix breaks wildcards, or were they already broken?

Of course, a new record type would be preferable, but there are real-world deployment issues there.

-Jim

---

From: Meng Weng Wong <[email-masked]>
Date: Thu, 26 Aug 2004 09:40:09 -0400

On Wed, Aug 25, 2004 at 08:44:16PM -0700, Rand Wacker wrote:
| 
| This is a concern that has been voiced to me by several major sites that
| are exactly the same set of sites we use as examples for where we need to
| stop phishing from.  They are most likely going to publish *both* spfv1
| and spfv2/pra records, and their initial investigation of their email
| architectures has indicated that those records are going to be pushing
| this real-world 240-byte limit (a 480 byte effective UDP packet split in
| two)
| 

The divergence between v=spf1 and spf2.0/pra records was
motivated by the desire to accommodate those domains whose
sets of permitted IPs differed according to context.

Example.com might wish to permit one set of MTAs to use its
name in mail-from, and another set of MTAs to use its domain
in the PRA.

My initial response to this requirement was to say,
"example.com should just take the union of both those sets
and publish that."

I believe that some people have spoken up and said that a
union solution was unappealing.

As things stand, the current draft means most people would
do:

 example.com TXT "v=spf1     blah blah blah"
 example.com TXT "spf2.0/pra blah blah blah"

I believe that the vast majority of all domains will have
exactly the same "blah blah blah".

Rand's objection appears to share this belief.

Therefore, I suggest that we introduce the %{e} scope macro
to accommodate the very small fraction of domains who need
to distinguish the scopes, and allow everyone else to leave
things alone.

At the Microsoft event earlier this month, we spent some
time discussing this issue.  The audience represented the
ESP community, who I believe are the folks who would most
want to assert the distinction between scopes.  The audience
felt that changing the prefix would be fine, but doing a
scope macro would also be fine.

---

From: David Blacka <[email-masked]>
Date: Thu, 26 Aug 2004 10:19:04 -0400

On Thursday 26 August 2004 8:12 am, Jim Fenton wrote:
> Can anyone
> clear up whether adding the prefix breaks wildcards, or were they already
> broken?

Since I don't know what you mean by "broken", I can't comment to whether or 
not wildcards are "already broken".  BUT,  if you publish:
 
  *.example.com. IN TXT "v=spf1 ..."
  *.example.com. IN TXT"spf2.0/pra ..."

Then, if a query for "host.example.com IN TXT" would match the wildcard, the a 
query for "_marid.host.example.com IN TXT" would ALSO match the wildcard.

So, the problem isn't that wildcards and marid records don't work, it is just 
that when using wildcards, the _marid prefix no longer gets you *just* the 
spf2.0/pra record.  I.e. the current status quo.

-- 
David Blacka    <[email-masked]> 
Sr. Engineer    VeriSign Applied Research

---

From: "william(at)elan.net" <[email-masked]>
Date: Thu, 26 Aug 2004 08:23:17 -0700 (PDT)

> sets of permitted IPs differed according to context.

This was unfortunete decision from IETF meeting largely done to 
distinguish MARID records from existing SPF base. It was felt by some
that existing records are published for mail-from and may not apply
for MARID PRA and by doing new version of SPF it can be insured that
all records used are those that wanted to published MARID records.

I have to say that I share opinion that records might be different
for different identities, but I did not think there was enough
reason to force duplication as per current proposal. I had to think
about it when this was talked during MARID meeting (before ideas
of scoping modifier that me and others proposed were rejected)
and I did not immediatly object.

However the way I see it now is that although this does introduce
scoping it did in a way that will not allow for two scope records
to have shared data parts which will cause large amount of dupliction
and increase of dns records. I note specifically that records do not
even allow for different scopes to even refer to the same shared portion
of dns record (though for example include) which could potentially
provide a solution in case of large SPF record size. Overeall I think
the way the scoping was introduced with SPF2.0/PRA carries larger
negative then positive value.

Both scope macro and new prefix have their own advantage and disadvantages 
and I do not like either alternative. With prefix you gain ability
to be certain that record published is indeed for MARID PRA (as spf1
records are entered directly in TXT for domain) as is with SPF2.0/PRA
but in case of shared records, it requires separate lookup which I
suppose would be done with "include". This can be avoided by using
wildcards but in that case the same record applies for all subdomains
and this may not have been the original intent. At the same time
when you do want to use wildcards, you can no longer proper publish
scope, which is actually a bigger concern.

With scoping macro, it easily solves the problem for SPF1 & MARID PRA
records having same data but at the same time main objection of new 
version of SPF2.0/PRA was to distinguish when publisher intended
to use PRA or not and in case of completely same data, you may never
know. At the same time if data is not the same, this will ALWAYS require
2nd dns lookup which is even worth, so I think spf macro is worst choice.

I do have an alternative idea though (and its not anything knew that was
not already mentioned before). This involves introducing new operator
that specifies scope (i.e. "scope=PRA") and requirying that to be present 
if record is to be considered valid for MARID PRA. If somebody needs to 
specify portion of SPF record that is not covered by PRA but should be
understood by old SPF1 complient client, then such record would go
before "scope=PRA", the shared portion would have to be proceeded by 
"scope=PRA,MFROM". In rare cases when you have record that should
be covered only by PRA but should not be ignored by SPF1 client, I
propose that "scope=MFROM ?all" (or ~all or -all, etc) be placed
followed by "scope=PRA" and rest of the record for PRA and repeat
of the ending ?all. To give an example:

 example.com. IN TXT "v=spf1 scope=MFROM +ip4:192.0.0.1 
			     scope=PRA,MFROM +ipf:192.0.0.2
			     scope=MFROM ~all
			     scope=PRA +ip4:192.0.0.3 -all"

In above current (SPF1 only clients that don't know about PRA and scoping)
would see above as only: "v=spf1 +ip4=192.0.0.1 +ip4:192.0.0.2 ~all"
where as new scope aware clients will see this same way for old SPF1 
scope of enevelope "mail from" but will see this as 
"+ip4:192.0.0.2 +ip4:192.0.0.3 -all" for PRA. 

Now what I actually proposed before is short versions for operator names 
(in order to have shorter dns records) where each scope is represented by 
one letter so instead of long "scope=PRA,MFROM" we can have "sc=pm" or 
possibly "sc=p,m". 

I would appreciate if these ideas and this form of scoping identifier are 
reconsidered especially if decide to go with anything like unified SPF 
instead of (or in addition to) current MARID PRA proposal.

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: Rand Wacker <[email-masked]>
Date: Thu, 26 Aug 2004 08:24:20 -0700 (PDT)

> union solution was unappealing.

I feel this way too.  MAIL FROM and the PRA are fundamentally different
and the records need a way to distinguish them.

> exactly the same "blah blah blah".

From the exact same conference of outsourced senders you were at my belief
is that the "blah blah blah" would be very different for those senders.
Many large commercial sites are beginning to indicate they will have a
different blah blah blah.

> to distinguish the scopes, and allow everyone else to leave

I would like to refresh my memory on this.  Do you have a pointer to a
thread where this was discussed?

-Rand

---

From: Rand Wacker <[email-masked]>
Date: Thu, 26 Aug 2004 08:19:08 -0700 (PDT)

> real-world deployment issues there.

It is a good point of encouragement.

-Rand

---

From: David Blacka <[email-masked]>
Date: Thu, 26 Aug 2004 10:19:04 -0400

---

From: Rand Wacker <[email-masked]>
Date: Thu, 26 Aug 2004 08:48:57 -0700 (PDT)

John, thanks for the reply.  One thing I want to start by saying (and then
I'm going to leave the discussion for after last call), its becoming clear
that the most fundamental disagreement on this list has to do with the
approach of doing *channel* authentication versus *message*
authentication.  I say this because you tend to hold SPF Classic and
Sender ID as equivalent in your response, which I do not.

> not to change the version string?

No.  Both I and the people looking to implement these records feel quite
strongly that using the same record for both is very dangerous.  In fact,
these largest sites are the ones who are *most* likely to have different
records (bit of a Catch 22 there).

> What you are telling us is the direct opposite.

The truth is that there are some very large sites which have already begun
asking senders to publish SPF records.  There are other large sites that
will be asking senders to publish Sender ID records.  (I can think of a
third large site that will ask for DomainKeys later ;)  The goal of all of
these senders is to get their mail through, they don't care so much how
they have to do that.

> less content filtering? Publish an SPF record.

^email forgery^channel forgery.  Also for most receivers I've talked to,
"pre data" doesn't matter since for the worst spam the packets are already
wedged in to their TCP receive buffers.

> hide identity, by getting everyone out in the open.)

No offense John, but this is a sub-concious troll to bring the debate back
to SPF Classic vs Sender ID. ;)

> implementation momentum among domain owners.

Enhanced email delivery, yes, but no-one I've talked to is under the
delusion that SPF Classic protects against domain spoofing in the "what
the user sees" context

> not to change the version string.

Incorrect conclusion.

> propagate the change through out the mirrored sites?

This is functionality already inherent in DNS, which is one of the things
that makes it appealing for a task like this.

Cheers - Rand

---

From: Jeff Macdonald <[email-masked]>
Date: Thu, 26 Aug 2004 13:06:03 -0400

On Thu, 2004-08-26 at 11:48, Rand Wacker wrote:
<snip/>
> records (bit of a Catch 22 there).

Could you give an example? Or point to one is such an example exists?

---

From: "John Glube" <[email-masked]>
Date: Thu, 26 Aug 2004 13:33:37 -0400

Hi Rand,

A couple of quick comments:

* I appreciate your clarifying that the need is to have
separate records, because of the senders' configuration.

* You suggest I am putting SPF classic and Sender ID on the
same level, or wanting to get into the SPF classic v Sender
ID debate.

You base this in part on my statement:

> authentication, thwarting spoofing and other steps taken
to > hide identity, by getting everyone out in the open.)

The statement reflects a synopsis of the FTC's view of the
value of sender authentication, which is the perspective I
have been using to view what is going on in an effort to
stay objective in my own thinking.

My view from a security perspective is that the best
approach is to not rely on either one approach or one data
set, but to do both message and channel authentication
using a variety of data sets. 

I agree this discussion concerns last call for Sender-ID. 

I am not wanting to get into a debate over one approach
versus another. 

The issue is:

* Whether within Sender-ID it made more sense to go with
the same version of records as SPF.

You have raised a catch 22 problem for larger organizations
with complex records. Going the sub-domain route raises
catch 22 problems for the vast number of individual domain
holders.

Needless to say, I am sure the design and security experts
will come forward with a solution.

From my own perspective, although some may not consider it
directly relevant to last call of any particular proposal,
given the issues, infrastructure and underlying import of
email, the question in my mind always remains:

* How would the FTC view what is decided given the FTC
mandate to protect the US consumer's interest, ensure
competition and protect against unfair trade?

I also keep in the back of my mind that the FTC has clearly
stated, if industry does not get it right, (what is right
of course being unclear as it helps to keep people on their
toes) the Feds reserve the right to step in and do the job.

As to the rest of your comments, I greatly appreciate your
elaborations.

Cheers, John

John Glube
Toronto, Canada

---

---

From: Rand Wacker <[email-masked]>
Date: Thu, 26 Aug 2004 08:48:57 -0700 (PDT)

Incorrect conclusion.

Cheers - Rand

---

From: Meng Weng Wong <[email-masked]>
Date: Thu, 26 Aug 2004 09:40:09 -0400

---

From: Rand Wacker <[email-masked]>
Date: Thu, 26 Aug 2004 13:14:13 -0700 (PDT)

> Could you give an example? Or point to one is such an example exists?

Example of large sites that will have spfv1 records that would differ from
spfv2.0/pra?  Sure:

A number of very large commercial banks run fairly complex outbound mail
systems.  More complex even than AOL -- which seems to be the yardstick --
because their email systems have evolved over the years through various
business actions.  It's not uncommon for a single bank to send outbound
mail from more than a *dozen* different systems scattered all over the
world (literally, more than a dozen).  In ip4: notation this would take
something like 12 x 21 bytes = 252.  (Note that these outbound sites are
completely separate from their inbound servers, so mx: wouldn't work).

Add to this the fact that these corporations regularly outsource marketing
and other material delivery to third party firms.  Since they want these
messages to be branded as coming from their primary domain, the spv2.0/pra
record would include many include: statements like "include:esp1.com
include:esp2.com include:esp3.com"  It doesn't take many additions here to
completely blow that 240 byte-ish limit that we're bumping up against by
inserting *two* records in one UDP response packet.

How would this differ from their spfv1 record?  Well, since most
outsourced mailings use an ESP-controlled MAIL FROM address (for bounce
handling purposes), the spfv1 record would not necessarily include those
additional ESP references.  So for a specific bank I am thinking of (13
outbound sites + 8 identified ESP partners), the spfv2.0/pra record would
be something like 401 bytes and the spfv1 record would be something like
252 bytes, for a total of 653 bytes, blowing the cap on the UDP response
packet.

Note that this doesn't apply just to banks, I've talk to numerous online
consumer sites and various media companies that will have the same
problem.  They didn't have concerns when it looked like Sender ID would
re-use the same records as spfv1, but now they are starting to worry.  The
complexity of global corporate mail systems is not to be believed and to
assume that Hotmail/AOL/Yahoo have the biggest outbound delivery system is
being a little naive.

-Rand

---

From: Rand Wacker <[email-masked]>
Date: Thu, 26 Aug 2004 13:27:24 -0700 (PDT)

> using a variety of data sets.

I totally agree.  It has been Sendmail's stated belief for quite some time
that multiple technologies will arise to address these issues and it is
our intention to support those that work best to meet the needs of the
Internet community at large.

> catch 22 problems for the vast number of individual domain

Hmm, in addition to working with a number of globally large enterprise
customers, Sendmail also represents a huge number of those individual
domain holders (I for one manage a "vast number" of individual domains in
my spare time ;)  I haven't seen much that presents a problem deploying to
sub-domains.  In fact, our operational experience with DomainKeys so far
has shown using a whole separate sub-domain namespace gives you a
tremendous amount of flexibility in both record depth and server
deployment.

> competition and protect against unfair trade?

Not speaking for the FTC, but the universal core concern for the large
enterprises I've talked to (the ones whose customers are targets of
phishing campains) are as follows:

- Firstly, to provide end-users the information to protect themselves by
  authenticating something the end user will see
- Secondly, to ensure reliable delivery of their customer relationship
  email

Because of these dual requirements -- and because of the already
well-publicized recommendations by many large receivers that volume
senders publish one/the other/both -- senders are going to deploy records
for Sender ID /and/ for SPF, and almost all of them are looking at what
its going to take to put a cryptographic signer in place in the future as
well.

My whole point of this thread is to point out the real-world deployment
issues that utlizing the top-level FQDN TXT space is going to pose.  I
fully support making some type of indication of what records apply to what
message addresses, I'm looking for different alternatives.

-Rand

---

From: Jeff Macdonald <[email-masked]>
Date: Thu, 26 Aug 2004 17:06:49 -0400

> Example of large sites that will have spfv1 records that would differ from

Actually I meant "using the same record for both is very dangerous". I
suppose you mean hitting the UDP limits. But since you detailed the 2
records scenario I'm kinda confused.

<snip>complex bank scenario</snip>

> inserting *two* records in one UDP response packet.

I'm sure there is a real concern about the size. However, these records
would be in the the Bank's primary domain space.

> additional ESP references.

I think you meant the Bank's IPs listed (SPF2) in the Bank's primary
domain won't/shouldn't be listed in the ESP's spf records. However,
since the MAIL FROM is in the ESP's domain space, is this not a separate
DNS record anyway?

See my confusion? If MAIL FROM and Header FROM are different in the
domain portion that should mean that different DNS records need to be
looked up. I realize with Sender-ID doubles the amount of data. The SPF
and SPF2 records would indeed be different.

However, help me understand how the content of a SPF/SPF2 record could
be different if the MAIL FROM and the PRA have the same domain portion.

-- 
:: Jeff Macdonald | Senior Software Engineer
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

---

From: Mark Lentczner <[email-masked]>
Date: Thu, 26 Aug 2004 14:13:41 -0700

I'm not going to weigh in on the underlying debate here, but I'm going 
to inject some information that might be useful:

1) Domains with large records will overflow the UDP limit:
Domains with large records that want to publish large, different 
records for under both v=spf1 and spf2.0/pra can do so as follows:

	example.com.             IN TXT "v=spf1 redirect=_spf1.%{d}"
	example.com.             IN TXT "spf2.0/pra redirect=_spf2.%{d}"
	_spf1.example.com.       IN TXT "v=spf1 ... some very long record here 
..."
	_spf2.example.com.       IN TXT "spf2.0/pra ... some different very 
long record here ..."

Yes, this will require two DNS lookups (though both can be cached the 
same as the one, so any reasonable TTL makes this less of an issue).  
Neither query will fall over to TCP (assuming the long records are 
individually under the UDP limit)

2) Wildcards
All the major points about wildcards have been made: Essentially, they 
don't work the way you'd like them to whether or not there is a prefix 
-- it is the nature of DNS wildcards.

On the other hand, if a domain uses wildcard MX records, then wildcard 
TXT and/or SPF2 records can be used to achieve a matching effect.  See 
the protocol-03 draft for an example and discussion of just this point. 
  I will only add here that publishing under a prefix or not makes no 
difference - the use of wildcards has the same caveats and 
difficulties.

3) Incorporating scope into the record format
Several schemes have been proposed on this list for incorporating 
scoping into the protocol syntax.  I will summarize them here:

3a) Put scopes in the <ver-ext> field
	Proposed by Wayne in 
http://www.imc.org/ietf-mxcomp/mail-archive/msg03441.html
This scheme replaces the "/pra" and <ver-ext> field in the version 
string with a list of scopes.  For example: "spf2.0/pra,mailfrom" or 
"spf2.0/mailfrom".  It applies to the whole record, so it only helps 
domains with very large records if their v=spf1 and spf2.0/pra records 
are otherwise the same.  If the above two large records were the same, 
this would look like:

	example.com.             IN TXT "spf2.0/pra,mailfrom ... some very 
long record here ..."

If the large records are different, they'd still have to resort to the 
method presented in (1) above.

3b) Add a scope macro %{e}
	Originally in the Unified-SPF concept, but recently proposed by Meng 
in http://www.imc.org/ietf-mxcomp/mail-archive/msg03663.html
This scheme adds a macro that is replaced with the identifier for the 
scope being checked: "pra" or "mailfrom".  It can be used with the 
include: mechanism or redirect= modifier to incorporate different parts 
into mostly common record:

	example.com.             IN TXT "spf2.0/pra ...common... 
redirect=_%{e}.%{d}"
	_pra.example.com.        IN TXT "spf2.0/pra ... pra only stuff ..."
	_mailfrom.example.com.   IN TXT "spf2.0/pra ... mailfrom only stuff 
..."

If the common mechanisms work, then only one DNS query needs to be 
done.  Otherwise, this basically devolves into the method presented in 
(1) above.

3c) Add a scope modifier
	Variations discussed in the past, recently proposed by William in 
http://www.imc.org/ietf-mxcomp/mail-archive/msg03677.html
This scheme adds a modifier that alters the applicable scope for the 
following mechanisms until the next scope modifier.  For the big 
company case, this is only a help if bulk of the records for PRA and 
MAILFROM are the same, and the common, plus all the difference, plus 
the overhead of the scope modifier still fit in one record:

	example.com.             IN TXT "v=spf1 ... mailfrom only stuff ... "
	                                "sc=pm ...common... "
	                                "sc=m -all "
	                                "sc=p ... pra only stuff ..."

If it doesn't fit, then it devolves into the method presented in (1) 
above.

This idea has several structural problems: Modifiers would have to be 
substantially reworked to achieve this "environment variable" like 
behavior.  The current document doesn't support such modifiers.  Also, 
this scheme has several important subtle restrictions to achieve 
backward compatibility with existing SPF systems.

3-Summary)
For sites where records for Sender-ID and SPF are large, similar though 
not the same, none of these methods is a win over the method in (1).  
For sites with small, similar records, only (3c) has any advantage over 
(1).  For sites where the records are the same (claimed by many to be 
the vast majority of sites), (3a) is the biggest win.

	- Mark

Mark Lentczner
http://www.ozonehouse.com/mark/
[email-masked]

---

From: "william(at)elan.net" <[email-masked]>
Date: Thu, 26 Aug 2004 16:07:05 -0700 (PDT)

> difference - the use of wildcards has the same caveats and 

I disagree that publishing under prefix makes no difference as to 
workability of wildcards. As noted wildcards can be used to 
extend spf policy to all subdomains under example.com which are not
specifically defined as separate records in dns, i.e.:
 a.example.com. IN TXT "v=spf1 ip4:192.168.0.0/16 ~all"
 *.example.com.	IN TXT "v=spf1 ip4:10.0.0.0/8 ~all"
will apply to 
 b.example.com and c.example.com and d.example.com, etc
 but not to a.example.com which has its own record

In case we decide to have main record be located within subdomain prefix 
depending on the identity, then in order to achieve same effect with 
wildcards, we'd still have to place the record in root of the domain, i.e.
 a._marid_pra.example.com. IN TXT "v=spf1 ip4:192.168.0.0/16 ~all"
 *.example.com.	IN TXT "v=spf1 ip4:10.0.0.0/8 ~all"

In this case wildcard record will apply to b._marid_pra.example.com and
c._marid_pra.example.com, etc but not to a._marid_pra.example.com. But
at the same time here the record from the wildcard (i.e. 10.0.0.0/8) will 
also apply to a._spf_mfrom.example.com. eventhough this might be different
identity and while for specific record we can do:
 a._marid_pra.example.com. IN TXT "v=spf1 ip4:192.168.0.0/16 ~all"
 b._spf_mfrom.example.com. IN TXT "v=spf1 ip4:192.168.10.0/24 ~all"
The same can not be done for wildcard records as there identity you only
have one place to put such record and it would have to be same for any kind
of identity (even ones not yet defined...). 

In my view this makes wildcards are bad choice for achieving scoping and 
identity separation. That is unless we decide that we don't need to support
wildcards at all and in that case to avoid problems (as some will still
do it seeing how its possiblems), the document would have to specifically
say that you CAN NOT use wildcards or CAN NOT place records in anything
but the specified scope/identity prefix.

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: "william(at)elan.net" <[email-masked]>
Date: Thu, 26 Aug 2004 17:03:53 -0700 (PDT)

> backward compatibility with existing SPF systems.

I disagree about "substantially reworked" part. Its true that currently 
there are no modifiers that work like as enviroment variable, but creating 
is not that hard or at least not any harder to write down in draft document
then any other scoping description. Implementing it is pretty easy too, 
all programming languages allow for local and global variables and spf
scope is likely to be global variable for the library doing spf record 
processing anyway.
 
> not the same, none of these methods is a win over the method in (1). 
Not quite. If records for different identies are 100% different and both
are very large, then method in (1) is the only way of not overruning
DNS UDP record limit. And I want to point out too that if records are so 
large as not not fit one record, you'd have to split it among two or more 
record by using include or redirect anyway.

At the same time if records are large but contain primarily shared/same 
portion the in 3c you can have this shared portion as part of the main
and if differences are small, the total would not overrun UDP limit.
 
> For sites with small, similar records, only (3c) has any advantage over (1). 
That would be for "large, similar records", see above. For small similar
records its also true, though there you can just enter records side-side
and not worry about overrunning UDP limit (although I personally think
that is bad and duplication should be avoided if possible).

> For sites where the records are the same (claimed by many to be 
3(c) offers exactly same advantages and disadvantages as 3(a) if records 
are the same (and with shortered scoping names, its even smaller is size)

example.com. IN TXT "spf2.0/pra,mailfrom ... some very long record"
 vs
example.com. IN TXT "v=spf1 sc=pm ... some very long record"

-------

I want to point out another important point for comparing different record
formats that originally caused new spf version string to be incorporated 
into current version of MARID documents. That is if we want separate
MARID PRA identity in record format in such a way that current base of
SPF record do not apply so as not to create confusion for different 
clients.

In case of 3(a) this is true as we have new version string (same as has 
been achieved now), however to support this new version with mailfrom
identity ALL existing SPF records would have to be republished and 
duplicated (so that both SPF1 and SPF2 clients can see them).

In case of 3(b) if main SPF version stays the same "v=SPF1" and record
is the same for different identities, then and MARID-aware client can not 
tell the difference between that being origional SPF1 record that was 
never supposed to apply for MARID and record that is shared between
MARID PRA and Mail-From identities, so original goal is not meat. However
if we do change version string, then we have same problem as what I 
described above where existing SPF base has to publish both new version
records and old SPF1 records for compatibility with different clients.

In case of 3(c) the objective is meat because if there is no scope modifier
then client can assume to be old not-yet updated SPF1 record that applies
only to mail-from identity. At the same time when domain owner does update
the record to support both mail-from and pra, he would not have to duplicate
the mail-from part of the record to support both old spf1 and new spf2 
clients (as is likely the case with 3a and 3b).

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 27 Aug 2004 01:59:11 +0200

Mark Lentczner wrote:

> _spf1.example.com. IN TXT "v=spf1 ... some very long [...]"
> _spf2.example.com. IN TXT "spf2.0/pra some different [...]"

Nice idea.

 [wildcards]
> See the protocol-03 draft for an example and discussion of
> just this point.

Where can I get protocol-03 ?
  
> 3-Summary)
> For sites where records for Sender-ID and SPF are large,
> similar though not the same, none of these methods is a win
> over the method in (1).

True.

> For sites where the records are the same (claimed by many to
> be the vast majority of sites), (3a) is the biggest win.

Looking at your summary (3b) could be more flexible than (3a),
but (3a) is probably clearer.  At the moment "SPF classic" and
"SenderId / PRA" still use almost the same "protocol" (= format
of the sender policy), although I'm not sure what results like
UNKNOWN or SOFTFAIL _mean_ in conjuction with "SenderId / PRA".

In fact I don't understand SOFTFAIL at all.  But assuming that
this is only my problem it would be nice to have _one_ format
for the three kinds of sender policies:

"SPF classic" (mailfrom) only, "SenderId / PRA" only, and both.

Either (3a) or (3b) offer this feature.  And in theory it's
still possible to fix the PRA algorithm for compatibility with
RfC 2476 (when an MSA "forgot" to add a Sender), or when a
mailing list uses Errors-To instead of Sender.

                            Bye, Frank

---

From: Mark Lentczner <[email-masked]>
Date: Thu, 26 Aug 2004 20:20:55 -0700

On Aug 26, 2004, at 2:13 PM, Mark Lentczner wrote:
> See the protocol-03 draft for an example and discussion...

Whoops!  Typo!  The currently published draft, 
draft-ietf-marid-protocol-02.txt has this example and discussion.

	- Mark

---

From: "John Glube" <[email-masked]>
Date: Fri, 27 Aug 2004 02:44:24 -0400

Rand,

>Hmm, in addition to working with a number of globally large
>enterprise customers, Sendmail also represents a huge
>number of those individual domain holders (I for one manage
>a "vast number" of individual domains in my spare time ;) I
>haven't seen much that presents a problem deploying to
>sub-domains.  In fact, our operational experience with
>DomainKeys so far has shown using a whole separate
>sub-domain namespace gives you a tremendous amount of
>flexibility in both record depth and server deployment.

I should have been clearer. Web hosting services offer sub
domains. However, there is a threshold cost involved, which
can sometimes be an incremental amount on a monthly basis. 

The catch? Pay the cost only to find the particular sender
authentication scheme does not work when implemented on a
large scale.

This is not meant as a specific criticism of Sender-ID, but
rather an expression of general concern.

The real world concerns you are raising are extremely
valid. 

Perhaps it is my innate sense of irony. Here we are, 4 days
into last call, when someone like yourself comes in from
the field and says hey guys "she just don't fit." :-) 

It just makes me a bit jittery. At the same time, given the
time constraints everyone is working under, perhaps it is a
function of the beast.

>... the universal core concern for the large enterprises
>I've talked to (the ones whose customers are targets of
>phishing campaigns) are as follows:

>- Firstly, to provide end-users the information to protect
>themselves by authenticating something the end user will see

These statements raise fundamental questions. It strikes me
there are two underlying issues:

* Implementation of one or more methodologies which can aid
in thwarting volume senders of unwanted email from hiding
identity.

Here it would seem we require a solution which protects one
or more delivery channels through authentication,
(thwarting the hiding of identity) plus reputation
(allowing the sorting of good from bad mail) and
accreditation for those who would otherwise be grey listed.

* Whether the correct response to phishing, which is
spoofed domains, forged header and message content, along
with the use of fake web sites is authenticating the domain
in the message from line and requiring the upgrading of
email client software across the board, so end user's can
see the result.

To respond to this specific form of crime, would it be more
efficient and focused to allow the end user to authenticate
the sender's message content through some form of light
weight cryptographic scheme?

>- Secondly, to ensure reliable delivery of their customer
>relationship email

This is a concern across the board, with the present state
of email delivery, due in large part to coping with ever
spiralling levels of unsolicited bulk mail.

In making these comments, I am not attacking the Sender-ID
design. It is as you point out a response in part to very
specific complaints. 

I am simply suggesting the concern you raise of
over-running of TXT data space in FQDN is valid. On
balance, the use of sub-domains may well be a practical
solution to the specific issue as raised. But, given some
of the side effects to the vast numbers of others involved
with email perhaps the issue highlights a more fundamental
concern.

Whether the financial institutions' design criteria, which
I gather from your comments has stamped the creation of
certain key aspects of Sender-ID is the right approach to
dealing with the phishing problem?

Answering this question may lead to a better answer to the
key practical concerns you raise.

John

John Glube
Toronto, Canada

---

---

From: Graham Murray <[email-masked]>
Date: Fri, 27 Aug 2004 10:47:07 +0100

Rand Wacker <[email-masked]> writes:

> messages to be branded as coming from their primary domain,

While this may be off-topic for this group, I believe that companies
doing this are not helping themselves at all when it comes to
protecting against phishing and other impersonation attacks. I know
that most people do not look at headers, but some of us do. Because I
am aware of phishing, when I receive an email which claims to come
from a financial institution or e-commerce site etc, I give it far
more scrutiny than I would most other emails. This includes looking
at the headers. Without Sender-ID, PRA, or other authentication
scheme (which is the status quo) it is almost impossible for the
recipient to determine whether or not a particular mail is genuine
when it is sent from a third party firm. So if they ensure that all
emails to customers (and potential customers) are sent from their own
servers they will make phishing detection and avoidance much simpler
both pre- and post- introduction of MARID checks.

---

From: David Blacka <[email-masked]>
Date: Fri, 27 Aug 2004 09:17:13 -0400

>  *.example.com.	IN TXT "v=spf1 ip4:10.0.0.0/8 ~all"

You seem to be discussing a plan that I, at least, have not seen before.  All 
of the discussion that I have seen has been about actual *prefixes*, which 
isn't what you are showing here.

However, you are illustrating an important point about naming and wildcards 
which holds true even if your examples seem a bit odd to me.

IF scope is part of the prefix scheme (i.e., _pra._marid.a.example.com IN TXT 
"spf2.0/..."), that scope MUST also be present in the RDATA of the record: 
_pra._marid.a.example.com IN TXT "spf2.0/pra ...".  Any information important 
to the interpretation of the record must not solely rest within the name, 
because of this wildcard issue.  This is a general problem with DNS record 
subtyping and wildcards.

To put this another way: clients, when querying for a TXT record using a 
prefix scheme MAY get back TXT records for different protocols and scopes, 
and must be able to pick the correct record from the set.

> but the specified scope/identity prefix.

What you are seeing is that prefixes (or subdomains, whatever) are a bad 
choice for *solely* dealing with scoping and identity separation.

---

From: "william(at)elan.net" <[email-masked]>
Date: Fri, 27 Aug 2004 07:05:01 -0700 (PDT)

> isn't what you are showing here.

My mistake, that example was supposed to be
   _marid_pra.a.example.com. IN TXT "v=spf1 ip4:192.168.0.0/16 ~all"
    *.example.com.	IN TXT "v=spf1 ip4:10.0.0.0/8 ~all"
 
> and must be able to pick the correct record from the set.
Yes, exactly.

> > In my view this makes wildcards are bad choice for achieving scoping and
			  ^-- another typo, should have been subdomain/prefix
> > do it seeing how its possible), the document would have to specifically

Basicly I'm saying that we can not rely on using prefixes for scoping 
separation unless we're certain that no wildcard records would exist
(which is probably an impossible assumption) or otherwise we have a 
problem with different scoping records that use wildcards and we can not 
distinguish between. And since we can not achieve scoping by using 
prefix/subdomain system on its own, we have to include scoping identifier
in the data part. But then we might as well just rely on the data part and 
create appropriate and extendeable scoping system there.

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: David Blacka <[email-masked]>
Date: Fri, 27 Aug 2004 10:07:41 -0400

> create appropriate and extendeable scoping system there.

The motivation for using prefixes was (I thought) to ameliorate the response 
size issue when Sender-ID is deployed alongside SPFv1 (or other unrelated TXT 
records).  Which it appears to do in all cases except the use of wildcards.

The fact the the prefix technique does not work in all cases does not make it 
useless in my mind.

So I agree with your statement except for the last sentence.

---

From: Rand Wacker <[email-masked]>
Date: Fri, 27 Aug 2004 11:09:50 -0700 (PDT)

> since the MAIL FROM is in the ESP's domain space, is this not a separate

Well, I meant that since the ESP-delivered messages would use a bank
labelled From: but an ESP labelled MAILFROM, then the *banks* spfv1
records would not need to include the ESP relay IP addresses.  The banks'
spfv2 record *would* however need to include the ESP relay IP addresses.

> and SPF2 records would indeed be different.

Sorry, this doesn't parse.

> be different if the MAIL FROM and the PRA have the same domain portion.

If the MAIL FROM and the PRA have the same domain portion then you are
right, the spfv1 and spfv2 records would definitly be the same.  But what
I am telling you is that for a lot of these examples, the domain of the
MAIL FROM is /not/ the same as the domain of the PRA for the same
inidividual message.

-Rand

---

From: Rand Wacker <[email-masked]>
Date: Fri, 27 Aug 2004 11:27:14 -0700 (PDT)

> can sometimes be an incremental amount on a monthly basis.

We're arguing about a theoretical incremental cost for someone to publish
a subdomain record through their ISP?  If that ISP provides email service
they are going to figure out they have to just do this for their
customers, in a sub-domain or not in a sub-domain, otherwise those
customers are going to another ISP that does.  The incremental cost of
publishing a record in a sub-domain is inconsequential when compared with
publishing a new data record and re-tooling the SMTP infrastructure to
determine where mail should actually be coming from.

> the field and says hey guys "she just don't fit." :-)

I brought up the point at IETF when the proposal to split was made, then
shopped it around with a few of our major customers over the past couple
of weeks, and here I am with more details.

> dealing with the phishing problem?

Yes, they feel that Sender ID is a useful band-aid for the immediate
problems, and recognize the need to develop a more robust solution (ie
cryptographic).

*Any* IP based authentication solution is going to have limitations,
primarily that they can only authenticate the most recent hop that a
message takes.  For these large customers I'm talking to, a large portion
(usually more than 50%) of their emails go precisely one hop from the
sender relay to one of either AOL, Yahoo, or Hotmail.  And unknown but
expectedly large portion of the other mails go one hop as well, but its
impossible to tell how many others may be forwarded.

So, for a large portion of the customers these senders are trying to send
authenticated emails to, Sender ID will help.  There is going to be
friction in getting mid-points for forwarding solutions upgraded to add
Resent-From and Resent-Sender headers, but those changes are much less
invasive than SRS rewriting.  Who knows, we may skip the whole
upgrade-the-middle-part and move straight to deployment of end-to-end
crypto auth, Sendmail is working hard on that as well.

But in the short term, Sender ID offers as many good benefits as any
IP-based message authentication scheme can.

-Rand

---

From: Mark Lentczner <[email-masked]>
Date: Tue, 31 Aug 2004 13:49:21 -0700

Re-reading this thread, looking for common ground, I see

1) There is near universal agreement that domains will still want to 
publish records about the 2821 Mail From identity.

2) There is still concern on where the TXT records should be published: 
Either at something like _pra._marid.example.com vs. at example.com 
itself.

3) It has been shown that even if records go into sub-domains, due to 
the way DNS wildcards work, records still need to be distinguishable 
based solely on their content, not their location.

> 	example.com.             IN TXT "spf2.0/pra,mailfrom ... some very 

The only counter to this scheme (other than some preference for the 
others) was that it would require existing v=spf1 domains to 
re-publish.  I don't beleive this would be the case::  All existing SPF 
implementations are likely to accept v=spf1 for quite some time, if not 
forever.  And, because it is also likely that all existing SPF 
implementations would be rev'd quickly[1] to support spf2.0/mailfrom, 
then new sites can simply publish the new version.

	- Mark

[1]: There aren't that many existing implementations, and everyone 
running one knows that they are running less than 1.0 code and 
generally is keeping an eye on updates.

---

From: "william(at)elan.net" <[email-masked]>
Date: Tue, 31 Aug 2004 15:35:25 -0700 (PDT)

> publish records about the 2821 Mail From identity.
That was quite clear from the start. In fact when I first talked to Meng 
on the interim meeting about combination of SPF and Caller-ID he was quite 
clear that this is being done so that existing SPF records would be 
supported by wider range of software including from Microsoft, and that
at the same time SPF libraries would also be upgraded to support new
type records for Caller-ID but that both would continue to exist.
 
> based solely on their content, not their location.
Yes, I made this quite clear.

But after rereading David Blacka's last email on the subject:
 http://www.imc.org/ietf-mxcomp/mail-archive/msg03763.html
I will no longer object to putting records in subdomains, I made wrong 
logical conclusion on it being useless solely because it does not achieve 
scoping by itself. In fact while providing scoping as part of spf data 
part is required, we can still benefit from subdomain/prefix placement
of records in case when records are in fact completely different, then
when somebody looks for one particular record type, they no longer get
the other record and we can have larger size of records in dns and do
not need to use approach of redirection that Mark previously desciribed.

But at the same time there is a downside to doing it this way too as
application, may end up doing more dns lookups. For example if application
checks for PRA record first and then for mailfrom and first one is not 
published, it ends up doing second lookup for second record. While if
records were both contained at the root of the domain, the application
would only need to do one lookup.

So choose what is worth - possible problems with larger dns record size or
possible problems with larger number of dns lookups... 
 
> then new sites can simply publish the new version.

You're assuming that existing SPF implementors want to support PRA which
maybe an issue because of Microsoft licese requirements. If implementors
don't want to support PRA, they have no reason to not coninue to only work
with SPF1 records. 

There is a solution available which completely supports existing SPF base 
and as I already posted before:
  example.com.      IN TXT "spf2.0/pra,mailfrom ... very long record ~all"
is completely (100%) equivalent to:
  example.com.      IN TXT "v=spf1 sc=p,m ... very long record ~all"

Because in case of records being equivalent a scoping modifier is 
interpreted exactly the same way as version modifier for entire record.

> generally is keeping an eye on updates.
I hope you're right, but if I remember the talk about merging SPF and 
CallerID was so that existing SPF record could be supported by both.

-- 
William Leibzon
Elan Networks
[email-masked]