Subject: Re: SPF I-D for review: draft-schlitt-spf-classic-00.txt

From: Paul Vixie <[email-masked]>
Date: Wed, 12 Jan 2005 04:19:37 +0000

> > ...  SPF RRs will only be used when there is an extant LHS@RHS, and
> > the RHS has to have either an A or MX RR which is at either a real
> > or synthesized (wildcard) name.
> 
> The problem is that spammers and phishers can easily use
> nonexistant.subdomain.example.com to get around example.com's sender
> policy.

an spf subscriber can trivially detect this.  and probably already is.
postfix calls this functionality by many names, such as:

        reject_unknown_sender_domain,
        reject_unknown_recipient_domain,
        reject_non_fqdn_sender,
        reject_non_fqdn_recipient,
        reject_invalid_hostname,
        reject_unauth_pipelining,
        reject_unauth_destination,
        reject_unknown_hostname,
        reject_non_fqdn_hostname,

and anyone who is interested enough in preventing forgery to install an
spf-capable mailer has the option of turning on features related to
nonexistent subdomains of spf-publishing domains.

since this can easily be handled on the client using the existing
protocol and well-tested features of common implementations, it should
not be made into an spf protocol requirement.

> This has to work for both the RFC2821 MAIL FROM identity and the
> RFC2821 HELO identity.  Sure the email receiver can have a policy of
> rejecting email from non-existant domains or domains without an A
> record, but in the world of email, there is far too much garbage sent
> by legitimate mailers to make such a receiver policy universal.  There
> needs to be a way for example.com to publish something for the entire
> zone that it controls, even the non-existant parts.

if a legitimate mailer is going to send garbage, they shouldn't publish
spf records.

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/namedroppers/>

---

From: Paul Vixie <[email-masked]>
Date: Wed, 12 Jan 2005 04:26:39 +0000

> ... The current SPF spec calls for checking SPF records in two places,
> and two places only: at the given domain, and at the top of the zone
> (just below the zone cut).  As such, any conforming SPF implementation
> would never know about any SPF records at co.uk, or .com.

in what way are you discovering the zone cut?  it sounds as if you're
committing the "novice dns" mistake of assuming that every label cut
is also a zone cut.  if GW.HOME.VIX.COM is the mail sender (A and AAAA
exist), but its name is really GW.HOME in the zone VIX.COM (and there
is therefore no HOME.VIX.COM zone cut, NS RRset, or SOA RR), how would
you propose to discover the "zone cut" you're speaking of?

i know how i had to do it in bind8's res_findzonecut() (for RFC 2136
dynamic update clients to use when finding out where to send their
updates) and it's _hard_.  surely you aren't expecting every SMTP
listener to do this on every SMTP session they process.

---

From: "william(at)elan.net" <[email-masked]>
Date: Tue, 11 Jan 2005 20:56:28 -0800 (PST)

> you propose to discover the "zone cut" you're speaking of?

By checking the authority section:
-------------------------------------------------------------------------
$ dig soa gw.home.vix.com
; <<>> DiG 9.2.4 <<>> soa gw.home.vix.com
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 49803
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;gw.home.vix.com.               IN      SOA

;; AUTHORITY SECTION:
vix.com.                0       IN      SOA     ns.lah1.vix.com. 
hostmaster.vix.com. 2005010200 3600 1800 604800 3600
-------------------------------------------------------------------------

So zonecut is "vix.com." since that is the start of authority. Current
SPF draft would call for doing SPF and/or TXT lookup at that zonecut.
Personally I'd prefer lookup to be madeto "**".<zonecut> instead.

The are some assumptions that are not in the draft but that we talked
about, in particular its assumed that if DNS RR is not available that DNS
server would return authority info in ADDITIONAL section of dns response
packet (which all BIND servers apparently do, if some other server does
not then additional dns lookup would be needed), i.e.:

-------------------------------------------------------------------------
$ dig gw.home.vix.com
; <<>> DiG 9.2.4 <<>> gw.home.vix.com
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 13468
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;gw.home.vix.com.               IN      A

;; AUTHORITY SECTION:
vix.com.                3600    IN      SOA     ns.lah1.vix.com. 
hostmaster.vix.com. 2005010200 3600 1800 604800 3600
-------------------------------------------------------------------------

> listener to do this on every SMTP session they process.

Personally I'd not mind with having that function becoming part of resolver,
that will sure help us out ...

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: Paul Vixie <[email-masked]>
Date: Wed, 12 Jan 2005 06:24:47 +0000

> Although I've been reading posts from you since the late 80s, I'm
> afraid I haven't read enough to know for sure whether you are speaking
> sarcastically or not, so please excuse me if I respond seriously to a
> joke.

no worries.  i'm not joking.  spf has been a marketing-driven juggernaut
since its first day, and nothing anybody has been able to say has changed
its course by even a degree.

> Yes, it is now very hard to slow down or change the direction of SPF.
> The spec has been pretty slushy for over a year, and incompatible
> changes are going to cause a lot of problems.

s/for over a year/since it was first proposed/, please.  i'm assuming that
you've read <http://www.circleid.com/article/634_0_1_0_C/>, and my response.

however, we've all been advised to limit our discussions about this draft
to dns-specific issues, so we should take this part offline.

---

From: wayne <[email-masked]>
Date: Wed, 12 Jan 2005 06:21:48 -0600

[ Moderators note: Post was moderated, either because it was posted by 
   a non-subscriber, or because it was over 20K.  
   With the massive amount of spam, it is easy to miss and therefore 
   delete relevant posts by non-subsrcibers. Please fix your 
   subscription addresses. ]

In <[email-masked]> Paul Vixie <[email-masked]> writes:

> an spf subscriber can trivially detect this.  [...]

I'm not sure what you mean by an "spf subscriber", I'm guessing you
are talking about the system that checks for SPF records.  (This is
called the "spf client" in the I-D.)

> postfix calls this functionality by many names, such as: [list snipped]

Yes, and other mailers have similar options, but those are receiver
policies, not sender policies.  

> nonexistent subdomains of spf-publishing domains.

I don't see how you can detect if a domain owner has published SPF
records without checking for them in the DNS.  Nonexistant subdomains
won't return SPF records.  Hence, the default at the zone cut.

I really don't see how the email receiver (SMTP server) options that
you mentioned above will work for the "spf subscriber".

-wayne

---

From: wayne <[email-masked]>
Date: Wed, 12 Jan 2005 05:57:57 -0600

In <[email-masked]> Paul Vixie <[email-masked]> writes:

>> (just below the zone cut).
>
> in what way are you discovering the zone cut?

See: http://www.schlitt.net/spf/spf_classic/draft-schlitt-spf-classic-00.html#version

>                                                it sounds as if you're
> is also a zone cut.

It sounds as if you're assuming instead of reviewing the I-D.

> listener to do this on every SMTP session they process.

Actually, res_findzonecut() is what I've based my code on.  See this
post I made to spf-discuss almost a year ago:

http://archives.listbox.com/[email-masked]/200403/0026.html

I don't claim to be an expert on DNS.  As others have pointed out, the
zone cut is one of the newest features of SPF (introducted about May
'03).  I certainly could be convinced to drop it and/or modify it to
make it work better.

-wayne

---

From: Paul Vixie <[email-masked]>
Date: Wed, 12 Jan 2005 17:24:45 +0000

> called the "spf client" in the I-D.)

ok, s/subscriber/client/, i'll try to remember.

> policies, not sender policies.  

so far so good.

> > and anyone who is interested enough in preventing forgery to install
> > an spf-capable mailer has the option of turning on features related
> > to nonexistent subdomains of spf-publishing domains.
> 

this is still asking the recipient to pay for the sender's sins.  if an
SPF publisher wants their SPF data to be relevant then you have to deal
with the fact that SPF clients will, in addition to checking for SPF,
reject mail that has nonexistent RHS's or HELO names.

the way this would look in your draft is something like:

"SPF Clients should take precautions to reject e-mail containing
nonexistent domain names; SPF Publishers should be aware that this
will occur, and avoid emitting legitimate e-mail containing such
domain names."

> you mentioned above will work for the "spf subscriber".

see above.  with this simple rule, SPF Clients and SPF Publishers can
cooperate reliably in matters of e-mail authenticity, yet shift none of
the costs of this cooperation onto those who do not publish or consume
SPF data.  a forger who tried to use a nonexistent domain name as a way
to defeat the SPF checking would find their mail rejected due to the
nonexistence of the domain name rather than due to the presence of SPF.

i'm aware of dns installations who conform to rfc1034/rfc1035 in every
way, even though they only answer SOA queries from locally trusted hosts
such as dynamic update clients or IXFR/AXFR slave servers.  not even the
logic and commentary in BIND8's res_findzonecut() will save you from that.
it is well and truly too late to mandate that SOA's be sent, and even if
we could mandate it, we wouldn't want every mail server querying for them
since they aren't cached.  the performance and information-leak problems
from that would scream "bad engineering" even if the protocol could be
changed to require that zone cuts be visible to outsiders.

on a personal note, i'm struck by the similarity of this discussion to
others i've had about SPF in the past.  there's a "proof by vigorous
assertion" feeling about all this.  so let me play rough for a minute:

	   "DNS doesn't work the way you want it to.  Get over it."

---

From: wayne <[email-masked]>
Date: Wed, 12 Jan 2005 15:14:15 -0600

> 	   "DNS doesn't work the way you want it to.  Get over it."

I'm quoting last things first to get this out of the way.

No problems playing rough, I've around the net since the late 80s and
have a very thick skin.  I will, however, assume the same for you and
say that there are a lot of similarities with what you are saying now
with previous comments I've seen from you about SPF.  A lot of what
you have been saying smacks of condescension, sour grapes and, yes,
ignorance.

In your reply to Meng's interview on CircleID[1], you said you very
much wish you would have jumped on this idea back in 1998.  I agree,
you should have.  You didn't.  Get over it.

Your claim last fall[2] that "he" (Meng?  me?  someone else?) didn't
know any better than to use TXT RRs is blatantly false and shows your
ignorance.  Meng started out by merging RMX and DMP, and RMX tried to
use a new RR.  We investigated and discussed using a new RR and
decided that trying to get a new RR would take years, would hurt
deployment, and in our humble opinions, we could design SPF records so
that problems with using TXT (as outline in, e.g. dns-choices) could
be largely avoided.

Your question[3] about how we intend to find the zone cut again shows
your ignorance and that you have not even bothered to read the draft.
I thought there was a rule in the IETF that those that haven't read
the drafts shouldn't comment on them.  Even if there isn't, I think we
would be a lot more productive if you did.

The people involved in developing SPF are not world experts in all
things email, DNS and/or spam.  Many of us, however, are very
knowledgeable about several of these areas.  In general, our overriding
goal has been to create a good (but not necessarily perfect) system
to slow email forgery and get as rapid a deployment as possible.
Rapid deployment has, for better or worse, influenced many of our
decisions, including using DNS instead of a new protocol, using TXT
RRs, not using a subdomain for subtyping, not requiring upgrades to
DNS servers, etc.  We have concentrated on the practical, rather than
*just* the theoretical.

We are trying to give domain owners a voice that email receivers can
listen to and determine if a given email is forged/authorized.  Many
of us feel that it is important to be able to easily give the domain
owner a voice about their entire domain.  The zone cut idea is one
possibility, creating a new kind of DNS wildcard system is another.
The zone cut stuff is working, in practice, for me in my beta-code.
If it can't be made to work, or we can find another idea that would
work better, I'm all for dropping it.

Ok, sorry for the long rant, but I think it is important for the
namedropper folks who have not been involved with email/spam to the
degree that Paul or the SPF folks to understand where SPF is coming
from. 

> will occur, and avoid emitting legitimate e-mail containing such

Again, always rejecting email from non-existent domains doesn't solve
all of the problem that the zone cut tries to solve.  You are
requiring that all email senders fix their systems to not issue bad
MAIL FROM/HELO domains, rather than just those domain owners that want
to use SPF.  (Or, another way to look at it is that you are requiring
SPF clients to reject legitimate email just because they check SPF
records.)

That won't fly, deployment wise.

Also, the zone cut language doesn't just deal with cases of
non-existent domains.  It also means that you don't have to publish
SPF records for all the hosts in your domain.  You just need to
publish the ones that are involved with non-forged email.

This is getting somewhat out of the area of DNS and into email.  What
would be extremely useful for SPF, and other anti-forgery systems such
as Yahoo's DomainKeys or Cisco's IIM, is the ability for a domain
owner to be able to speak for their entire domain.  Can we find a DNS
way of doing that?

> changed to require that zone cuts be visible to outsiders.

I have not hit those cases that you speak of in my, admittedly,
limited testing.  I think it would be reasonable to require domain
owners that wish to use the zone default SPF record to have a working
zone.  As long as broken zones don't cause problems for SPF checkers,
that shouldn't be too bad.

-wayne

[1] http://www.circleid.com/article/634_0_1_0_C/
[2] http://ops.ietf.org/lists/namedroppers/namedroppers.2004/msg01923.html
[3] http://ops.ietf.org/lists/namedroppers/namedroppers.2005/msg00028.html

---

From: Paul Vixie <[email-masked]>
Date: Wed, 12 Jan 2005 22:08:36 +0000

i'm trying hard to limit my comments on this thread to the dns-relevant bits.

> Rapid deployment has, for better or worse, influenced many of our decisions

if you're basically just pandering to your installed base, why not admit this,
tell the IETF what you've done in the form of an FYI, and give yourself a rest?

> work better, I'm all for dropping it.

anything can be made to work in beta-code.  when you seek universal deployment,
the size of that universe dictates that you work within the letter and spirit
of the spec rather than within the loopholes and enforceability limits.

> to use SPF.

i have been running this way for a couple of years.  rarely i'll get a phone
call from somebody who tried to HELO me from a NAT'd host whose hostname was
not visible in external dns.  i always apologize and then give them my "fax"
number.  this hasn't happened since spring 2004, and i get a LOT of mail from
a LOT of places.  are you sure your information on this is as current as mine?

> (Or, another way to look at it is that you are requiring SPF clients
> to reject legitimate email just because they check SPF records.)

here, we're quibbling over what it means for e-mail to be "legitimate".  the
spirit of HELO is "this is my hostname".  while "HELO localhost" is legitimate
under the letter of RFC 2821, i'd argue that it's a violation of the spirit of
RFC 2821.  given that you're asking RFC 2821 server-side implementations to
consider rejecting e-mail for reasons not contemplated by RFC 2821, it's hard
for me to understand you when you say...

> That won't fly, deployment wise.

but whatever.  let's move on.

> publish the ones that are involved with non-forged email.

i was thinking that you'd publish SPF records for things that had MX
records, or at worst, from things that had (MX or A or AAAA) records.
if you're having trouble doing this, take a look at the "m4" or "cpp"
preprocessors, or ask your DNS vendor for a built in macro processing
facility.

> owner to be able to speak for their entire domain.  Can we find a DNS

ultimately your concern is about nonexistent subdomains.  if a forger can
get more value out of spoofing nonexistent subdomains than from spoofing
nonexistent unrelated domains, then i've been out of the anti-spam world
longer than i thought.  here's what it sounds like you ought to do.

1. codify current behaviour (not the beta-code behaviour; the global one)
2. explain the good and bad aspects of the current dns wildcard features
3. launch a separate effort to add a rrtype-specific wildcard dns feature

> ...  I think it would be reasonable to require domain owners that wish
> to use the zone default SPF record to have a working zone.  As long as
> broken zones don't cause problems for SPF checkers, that shouldn't be
> too bad.

i'm thinking of all dns queries that will be received by nonparticipants,
and sent by SPF Clients to those nonparticipants upon every inbound SMTP
session.

(one non-dns-related side note: somebody around here told me recently that
they had had to delete their SPF records because one or more SPF Clients
had implemented SPF improperly, and were rejecting mailing list e-mail
based on the mismatch between the header From: and the list exploder.  all
this proves is that no matter how careful a specification is authored, it
still has to run the gauntlet of wide scale implementation.)

---

From: Paul Vixie <[email-masked]>
Date: Thu, 13 Jan 2005 16:45:47 +0000

> > the caution this suggestions for SPF is that if this SOA-surfing idea goes
> > forward, something has GOT TO BE SAID about protecting TLD and root servers
> > from queries about these names.
> 
> I have already said that I thought that putting an something in about
> never querying domains with less than two labels is a good idea.  I
> think it is a good idea whether the zone cut stuff remains or not.

in the time since i first proposed the two-label rule, someone pointed out
CO.UK to me, where it's actually a three-label rule.  i don't think this
kind of rule can work.  i don't think that anything like this can work.  if
someone can force you to query for F(name) simply by mentioning a nonexistent
"name", where F() is a function returning the parent, grandparent, child,
brother, or indeed any name other than "name", then it's a new DDoS vector.