Subject: Re: [spf-help] Re: SPF and SenderID

From: Greg Connor <[email-masked]>
Date: Tue, 19 Jul 2005 22:59:14 -0700

>> On 6/15/05, Douglas Otis <[email-masked]> wrote:
>>
>> > For "Sender Policy Framework (SPF) for Authorizing Use of Domains in
>> > E-MAIL, version 1", the listed authors are M. Wong, and W. Schlitt.
>> > For "Sender ID: Authenticating E-Mail" the listed authors are J. Lyon,
>> > and M. Wong.  The "spf2.0/" record syntax was a product of the MARID WG
>> > specifically addressing this very issue.  As all the authors
>> > participated in the MARID WG that identified this scope related
>> > problem, and as both of these drafts also credit common authorship, it
>> > would be impossible to say which draft is now authoritative for the
>> > "v=spf1" record.
>>
> On Fri, 2005-06-17 at 14:27 -0400, Michael Hammer wrote:
>> I would certainly agree with you that the waters have been muddied. I
>> have stated my opinion on that in other threads/locations so I won't
>> go there. The issue at hand though is whether IETF/IESG should accept
>> an ID (even experimental) which proposes to break the implementation
>> (existing) of a competing ID where the purported reason for doing so
>> comes down to "people won't publish the record we want (and
>> participate in our experiment) so we will force them to publish our
>> record, suffer bad outcomes or remove existing (SPF1) records all
>> together.

--Douglas Otis <[email-masked]> wrote:
> From the day of the SPF/Sender-ID merger in the interim meeting at
> Qualcomm, it has been clear the SPF record would be utilized to evaluate
> the PRA.  Microsoft wanted to use their XML record, but the WG convinced
> them to use the existing record syntax.  This is why Meng Wong
> eventually suggested the use of the "spf2.0/mfrom ..." record once the
> licensing issue became a concern.  This is history that can not be
> changed.  Especially by a third-party like IETF/IESG.  You want the IESG
> to say who made the mistake?  There were so many in sequence.

As one who spent a fair amount of time working in the MARID working group, 
I can honestly say I'm still disappointed that there was never a proposed 
standard, only drafts and talking.

I wouldn't call any of those drafts or discussions "mistakes" but I 
wouldn't credit them as being standards either.  My belief, in the few 
conversations I did have with MS folks, was that we were working toward a 
common goal that was, in the end, never realized.  The WG failed to produce 
the output we were all expecting... we were left only with "Go ye forth and 
experiment some more."

Technically, there is no official functional definition of what a TXT 
record means other than "it contains text".  So, no draft writer can claim 
it's an exclusive domain.  However, if I'm looking for propriety in whose 
draft should or should not claim to make use of the records, I'm going to 
look for prior art.  I'm going to look for what people who actually 
published the records *intended* them to be used for.  It seems pretty 
clear to me that people who published TXT records starting with v=spf1 
intended them to be used *FOR SPF*.  Any other interpretation seems 
specious to me.  Nothing output by MARID has reached any level of adoption 
even approaching the level SPF was at even before MARID was convened.

Probably the worst "mistake" we made with MARID was allowing it to be 
subjected to denial-of-service attacks, and I'm not talking about the DNS 
variety.  I'm talking about people who continually trotted out the same FUD 
over and over, blatantly going over posting limits and outside of topics 
requested by the chair, and repeatedly questioning what had already been 
declared a rough consensus.

>> > It has been nearly a year to consider the impact of this problem.  Only
>> > those that don't care about the scope of the record should publish
>> > "v=spf1".  Those that care, should publish "spf2.0" records that
>> > declare the assured scope.

...
> This depends upon which era is considered, and which draft the publisher
> may have been reading.

...
> The SPF spec is frozen, which is why it does not offer a solution to
> your dilemma.  You would need to thaw it out before you could use the
> new and scope-safe record.

...
> You should review the histories of these two drafts.  It is clear to me
> the authors of the "frozen in time" draft have not allowed themselves to
> deal with this conflict, as the PRA did not exist back then.  This draft
> is just to document a distant point in history, nothing more.  Reading
> the white paper, you would be hard pressed to even discern there was a
> problem.

...
> I am posting to the MARID reflector in order to express my views
> concerning the two drafts published.  Your view of history is simply
> biased.  What is preventing the use of "spf2.0/mform"?  Why battle when
> there is a solution?  You are talking about a mistake made by the group
> as a whole.  We all make mistakes.  Watch for the camel's nose next
> time.

Speaking of FUD, thank you for the eloquent reminders as to why I haven't 
opened the MXCOMP mail folder for nearly a year.

MARID failed it's primary goal, but most of the people involved are moving 
ahead with other proposals.  More power to them.  I don't even think of 
them as "competing" proposals.  The best case would be if they all move 
forward in some form or another.

I don't think it's productive to point to the actions taken by MARID and 
say that those are the "official" versions of anything.

Anyway, best of luck with your... whatever it is you're doing.  The only 
real advice I have is, resist the temptation to tear down what other people 
are building, and go build something interesting of your own.  It's a lot 
harder to build than to tear down, but you have a better chance of earning 
respect in the long run.

Be well.

gregc

--
Greg Connor <[email-masked]>

---

From: Douglas Otis <[email-masked]>
Date: Wed, 20 Jul 2005 08:12:25 -0700

> opened the MXCOMP mail folder for nearly a year.

I am surprised you consider a suggestion to permit use of a compatible
record (spf2.0) by SPF Classic (and Sender-ID) to be FUD.  I have just
returned from a large promotional meeting attended by SPF proponents,
such as Meng Wong and Wayne Schlitt.  On more than one occasion an
explanation was made that publishing the v=spf1 record would be used to
evaluate the PRA.  I never heard anyone object, so I don't understand
how one can conclude what people intend when they publish this record.
I would have hoped you could understand why not permitting a record that
declares its scope has become a problem.        

> say that those are the "official" versions of anything.

There are experimental versions of SPF Classic and Sender-ID with a few
still trying to find solutions for problems that remain with these
approaches.  Two areas still wanting is a means to assert scope in SPF
classic to indicate that the PRA has not been evaluated and should not
be used.  The other is a means to indicate the exclusive use of a domain
is or is not being assured by the server.

At this meeting there were companies that acknowledge the concern
created by such oversight and were providing domain owners their own
unique outbound IP address.  When considering the shared server problem
together with the loss of forwarded email, it is hard not to think that
DomainKeys and DKIM are already well ahead on the solutions curve.

There is no spam or phishing solution without the use of email
reputation.  The accrual of reputation demands authentication and not
just authorization, if to be fair.  Saying that authorization is good
enough sounds too much to me like "let them eat cake" when knowing those
that care will have their own servers.  Getting this right requires an
understanding that neither authorization nor authentication alone is a
solution. 

The next step in this process is reputation, however this step can only
be safely made when there is an ability to authenticate the entity being
held accountable.  Just calling it authentication is not productive,
when it is based upon an assumption that the server is not shared.  I
find this a bad assumption and one that will create endless support
issues when exploited by the millions of zombies known to exist.

-Doug

---

From: "Greg Connor" <[email-masked]>
Date: Wed, 20 Jul 2005 12:55:25 -0700 (PDT)

> is or is not being assured by the server.

I don't know of a good way to objectively tell whether something is
experimental.  Quite a number of people use SPF and don't intend to stop,
but it's still a minority in terms of the world population of domains and
mail servers.  So, I guess the "experimental" label can be applied to
either usage, but in terms of numbers, SPF classic is clearly ahead in
terms of adoption.

I'm not sure exactly what is meant by "a means to indicate the exclusive
use of a domain is or is not being assured by the server".  The spec is
pretty clear on what constitutes a "pass" vs. "unknown".  I have been
suggesting to people for quite some time that they should use the "+" mode
if they control the MTA, or if they trust the MTA operators enough to take
responsibility for the MTA's actions.

For some domain owners, this could mean "the site uses SMTP AUTH and
doesn't allow spoofing of other domains".  For others, they may be content
with "the site doesn't currently have an ongoing forgery problem".  This
is a tradeoff the domain owner must decide for himself.  If they don't
wish to take responsibility for the MTA, they should use "?" instead of
"+" to add it.  We want to encourage everyone to move to the "+" column
eventually, but there is still quite a bit of value in narrowing the "?"
space from ?all to ?certain-MTAs -all.  Every little bit helps.

I think we have been over the difference between "+" and "?" in the spec
before, so I'm puzzled as to why this is still often (very often) brought
up as a "fault".  Do you really see it as a fault, or is it just a
convenient place to attack SPF because people often get confused about it?
 Do you understand the difference between Pass and Neutral results?

(Last I checked CSV doesn't have a neutral or unknown mode.  If I remember
right, the only way to signify "unknown" under CSV is not to publish.  Of
course, you didn't say whether you are comparing SPF to CSV, or just
railing on SPF without a constructive counter-proposal.)

> understanding that neither authorization nor authentication alone is a

I think I would agree with that.  But if there's any lesson I take away
from MARID, it's that the best is often the enemy of the good.  Waiting to
solve one problem until we have enough tools to solve a larger number of
problems doesn't seem to be the best way forward.  Some would look at the
problem space and say "This is huge and complicated... We have to approach
it in just the right way."  I look at the problem space and say "This is
huge and complicated, so let's get started."

> issues when exploited by the millions of zombies known to exist.

I understand what you mean here, and I do believe you are sincere, but I
don't agree with your characterization nor your conclusion.  I should
probably go back and read the spec and see if the language explaining pass
vs. neutral is clear enough.  If you have suggestions on how to make this
clearer so people aren't confused in the way you suspect they currently
are, I'm sure Wayne would welcome the feedback.

Thanks
gregc

--
Greg Connor <[email-masked]>
on my squirrelmail right now

---

From: Douglas Otis <[email-masked]>
Date: Wed, 20 Jul 2005 22:18:12 -0700

On Wed, 2005-07-20 at 12:55 -0700, Greg Connor wrote:

> experimental.

This is the category assigned by the IETF.

> use of a domain is or is not being assured by the server".

While there have been several providers to offer domain owners their own
IP address for reputation protection, such providers will likely need to
acquire a server per 8 such domain owners, to justify the IP address
use.

While perhaps as much as 20% of the domain owners could acquire their
own IP address, this method of constraining outbound SMTP authorization
is expensive.  What is not apparent is whether a server ensures that
shared domain use is not abused.  At the meeting in the technical
section, an example had something similar to 'include:isp.net'.  If
servers at isp.net handle large numbers of domains, and do not constrain
both the PRA and the bounce-address, then such an 'include' could become
reputation suicide.

> The spec is pretty clear on what constitutes a "pass" vs. "unknown".

As there are many functions possible within the SPF script, it is
difficult to know what basic function is being attempted.  Without
knowing this, what does 'pass' really mean?  I would recommend creating
a structure that describes returning results in three categories based
upon the actual function completed: 

 - authorization
 - authentication
 - compliance

> I have been suggesting to people for quite some time that they should
> use the "+" mode if they control the MTA, or if they trust the MTA
> operators enough to take responsibility for the MTA's actions.

You are advocating an undocumented convention that could not be
differentiated from existing use, but okay.  If the mailbox-domain is
not constrained as indicated by the absence of '+', then results should
fall within the 'authorization' category.  If the domain is constrained,
perhaps by specialized software, or exclusive use of an IP address, then
results could fall within the 'authentication' category. 

Without knowing whether the domain is constrained, which normally it is
not, then holding the domain accountable for abuse would be unfair.  A
'failure' result within the 'authorization' category could be used to
repudiate the source of the message only.  However, a 'success' result
could not be used as a basis for reputation accrual.  Alas, it is not
human nature to follow this practice.  A recipient will still want to do
_something_ about abuse, even when it is only being 'authorized' by a
domain owner.  As a result, these domain owners may become unfairly
blocked.

The domain owner should not have been held accountable, as they are
unable to protect themselves.  Nor will the SPF system allow everyone to
protect themselves, which suggests it is okay to create a type of second
class email domain.  This is where the phrase "let them eat cake" comes
to mind.

> doesn't allow spoofing of other domains".

Yes that is possible, but this system keeps the domain owner in the
dark, and the administrator that MUST provide the security, anonymous
and in control of all evidence.  Again human nature seems to suggest
that there will be many domain owners unfairly treated.

> For others, they may be content with "the site doesn't currently have
> an ongoing forgery problem".  This is a tradeoff the domain owner must
> decide for himself.  If they don't wish to take responsibility for the
> MTA, they should use "?" instead of "+" to add it.  We want to
> encourage everyone to move to the "+" column eventually, but there is
> still quite a bit of value in narrowing the "?" space from ?all to ?
> certain-MTAs -all.  Every little bit helps.

I would say there is another option that should be considered.  Not
publishing SPF records.  Many recipients are rejecting '~' and '?' which
creates an immediate problem for the sender with forwarding issues that
depend upon this exploited feature.  I doubt there are many service
providers willing to offer domain owners indemnification, should their
domain become abused, and the domain's reputation is subsequently
damaged.

> Do you understand the difference between Pass and Neutral results?

I am attempting to clarify what is needed for the next step, while you
are focused on the mechanics that look past the basics.  Step back and
think about the basic functions: authorization, authentication and
policy compliance.  As I said, it appears the '~' and '?' are already
too heavily exploited to be of much use as shades of failure.

> railing on SPF without a constructive counter-proposal.)

I see DKIM and CSV as fair as safe paths to enabling name-based
reputation.  Consider what is needed to perform the basic functions.
My comments come from a perspective different from most providers.  A
perspective found dealing with angry senders; where error messages often
point to the wrong reputation service; for the sender, it does not
matter who created the reputation accrual error, they want someone to
call for restitution.

> problems doesn't seem to be the best way forward. 

I still see CSV and DKIM as a way forward.  I would have expect CSV to
come first however.  CSV protects the network resources, and DKIM
protects the domain owners.  There are different incentives for each
method, but in the end, they both can safely support a name-based
reputation system without unfairly causing harm.  

> are, I'm sure Wayne would welcome the feedback.

If you look at the language for pass, you will find the word 'authorize'
accompanied by language that suggest this is good enough to hold the
entity accountable and to accrue a reputation.  As I have said many
times, authorization is not good enough.  HELO authentication carries to
much baggage to be really useful at defending resources, which is the
role it should play.  I have never been shy at offering feedback. : )

-Doug

---

From: Greg Connor <[email-masked]>
Date: Wed, 20 Jul 2005 23:50:14 -0700

> both the PRA and the bounce-address, then such an 'include' could become

I'm not sure what having "your own IP address" has to do with either 
authentication or authorization.  Are you working under the misapprehension 
that you need a separate IP address per domain name in order to turn on 
SMTP AUTH and enforce proper use of identities?

A casual reader of the above might make the mistake of assuming that SPF 
requires a static IP in order to work.  It is strongly implied.  If you 
actually believe this, I can take the time to explain how SMTP AUTH and SPF 
Pass results are related.

If you don't understand what I meant when I said "the site uses SMTP AUTH 
and doesn't allow spoofing of other domains" then please ask for 
clarification.  If you DO understand what I meant, please stop spreading 
misleading information.

> upon the actual function completed:

The SPF spec describes "pass" as "the action is authorized".  It doesn't 
describe authentication or compliance, as far as I know.  I believe SPF is 
a great tool for what it does, but it doesn't do everything.  I think this 
is OK... any tool claiming to do all of the above should be viewed with 
skepticism...   To me it seems a lot more likely that the problem space 
will be covered by multiple tools.

> results could fall within the 'authentication' category.

I'm sorry, I think my initial explanation was not detailed enough.  I was 
assuming that since you spend a fair amount of time pointing out SPF's 
faults, that you were familiar enough with the mechanisms in question to 
understand my statement.

Here is some background necessary in order to understand the significance 
of "pass" vs. "unknown" in the context of reputation.

"+" indicates that the mechanism results in a "pass".  "+" is also the 
default, so if you have a mechanism with no prefix, the "+" is understood. 
+include:isp.net and include:isp.net are the same.  If the mechanism 
matches the situation, the result is "Pass".  The receiver can proceed with 
confidence in the identity (mfrom or helo).  The action is considered 
authorized, and the domain owner should be held accountable for the action 
of the MTA.

"?" indicates that the mechanism results in a "neutral" result.  The action 
is neither explicitly authorized nor explicitly denied.  The receiver 
should proceed with whatever the default action is for domains not 
protected by SPF.

> domain owner.  As a result, these domain owners may become unfairly

Right, again I apologize for not explaining what I meant well enough.  I 
believe your first paragraph above is consistent with the "Pass" result and 
the "+" modifier (or no modifier), and your second paragraph applies well 
to the "neutral" result and the "?" modifier.

The difference between these two modes is as you say.  "Pass" is the only 
result I would really describe as "success".  It implies that the domain 
owner wants the privileges conferred by a Pass, and is willing to defend 
the messages coming from that MTA with his reputation.  They are one and 
the same as far as SPF is concerned... it is meaningless to authorize a 
message if you're not willing to stake your reputation.

> that there will be many domain owners unfairly treated.

I don't really draw a distinction between equipment I own or control, and 
stuff done on my behalf by a vendor.  The important distinction is not who 
owns the equipment, but whether I trust the vendor/partner/employee/robot 
in charge of doing it, and whether I trust them enough to stake my own 
reputation.

A lot of ISPs are now requiring the use of SMTP AUTH within their networks, 
but my experience has been that they don't block messages claiming to be 
from various domains.  Hopefully if enough users ask for this, they will 
start matching up domains that I own and control with my name and password 
for SMTP AUTH, and disallowing others from using domains they don't own.  I 
don't think it's a tough problem to solve, but currently they have no real 
incentive to solve it.  I think wider use of ip-based authorization will 
put more pressure on them.

In the meantime, some domain owners may choose to give them a PASS anyway, 
in effect staking their reputation on a service that may be mostly 
spam-free and forgery-free right now, but may develop such problems in the 
future, and they will deal with that if and when it happens.

>> > be safely made when there is an ability to authenticate the entity
>> > being held accountable.  Just calling it authentication is not
>> > productive, when it is based upon an assumption that the server is not
>> > shared.  I find this a bad assumption and one that will create endless
>> > support issues when exploited by the millions of zombies known to
>> > exist.
>>
>> probably go back and read the spec and see if the language explaining
>> pass vs. neutral is clear enough.  If you have suggestions on how to
>> make this clearer so people aren't confused in the way you suspect they
>> currently are, I'm sure Wayne would welcome the feedback.
>

You stated a couple of times (though I snipped them for space conservation) 
that you see CSV and DKIM as a valid path to authorization, authentication, 
and reputation.  Why do you feel that CSV+DKIM fulfills this goal, but 
SPF-HELO+DKIM would not?

I readily admit that I don't understand CSV well enough to spend a lot of 
time bashing it on various lists.  Therefore, please don't assume I'm 
bashing on CSV here.  If the point of CSV is to provide confidence in the 
HELO name, how is that functionally different from SPF used to check HELO?
  - A domain owner publishes a record in DNS
  - A receiver has a name and wishes to check it against the IP that 
offered the name
  - The receiver gets the DNS record and checks to see if the IP is 
authorized
  - If the IP is authorized, the receiver can proceed with confidence in 
the HELO identity
It's my understanding that the above requirements can apply to either CSV 
or SPF-HELO.  I can guess that you probably don't agree with this, but I 
still cannot fathom the reasons.  Since SPF has been a frequent target for 
folks to take pot-shots at such as "authorization without authentication is 
not good enough", it's rather important to me to try and understand the 
essence of this issue.

Thanks for taking the time to write.
gregc

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Thu, 21 Jul 2005 11:16:54 +0200

[Greg Connor]:
>
>   I'm not sure what having "your own IP address" has to do with
>   either authentication or authorization.  Are you working under the
>   misapprehension that you need a separate IP address per domain
>   name in order to turn on SMTP AUTH and enforce proper use of
>   identities?

excuse my jumping in, but it seems like you and Doug can't understand
each other very well :-)

no.  SPF authorises an IP-address to use a domain.  that IP-address
may be authorised to use several other domains as well.  it can also
handle other domains which haven't deployed SPF.  these domains can be
owned by different entities, and SPF offers no protection against
abuse from the entities sharing your server.  in fact, they will be
fully authorised to spew junk and destroy your reputation.  so you
take your business elsewhere -- but it won't help.  the reputation is
tied to your domain name.  the only solution is to plea with others,
to wait, or to switch domain name.

contrast this with CSV.  CSV authorises a server to send e-mail in
general, its administrator claims responsibility for handling abuse
complaints in a timely manner and so on.  CSV says _nothing_ about the
domains used as sender addresses in the e-mail sent out by that
server.  as a consequence there are no issues with forwarding, or
people using their Hotmail address even when sending via the
university server etc.  it also means that if you happen to be hosted
by an irresponisible company, you can take your business elsewhere, to
a host which may already have good reputation.  the bad reputation is
left behind with the bad server.

>   A casual reader of the above might make the mistake of assuming
>   that SPF requires a static IP in order to work.

well, it does require at least one IP address per entity wanting
distinct reputation.  I'd say that's pretty close to requiring an IP
address per domain, but of course that's a judgement for each domain
operator to make.

>   You stated a couple of times (though I snipped them for space
>   conservation) that you see CSV and DKIM as a valid path to
>   authorization, authentication, and reputation.  Why do you feel
>   that CSV+DKIM fulfills this goal, but SPF-HELO+DKIM would not?

DKIM handles the other piece, authorising the use of the domain (or
individual e-mail address) as a sender of e-mail.  this is done by
adding cryptographically secure authentication to the message itself.
this leaves the authentication to the system generating the e-mail,
either at MTA or MUA level, and it survives forwarding through a chain
of servers (although there are some caveats still being worked out.)

-- 
hope this helps,
Kjetil T.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 21 Jul 2005 12:31:54 +0200

Kjetil Torgrim Homme wrote:

> SPF authorises an IP-address to use a domain.
[...]
> SPF offers no protection against abuse from the entities
> sharing your server.

Yes, for that you would need "enforced submission rights"
as specified in RfC 2476(bis) option 6.1, or your own MTA.

The spec. is rather clear about this issue (chapter 10.4).

> take your business elsewhere -- but it won't help.  the
> reputation is tied to your domain name.  the only solution
> is to plea with others, to wait, or to switch domain name.

If you're worried about this problem you can use the "?"
(NEUTRAL) qualifier, and still have "-" (FAIL) for the rest
of the world.

> it does require at least one IP address per entity wanting
> distinct reputation.

No, that's not the case.  And for the HELO-part of SPF it's
very similar to what you said about CSV, for simple policies
mta.example.com. IN SPF "v=spf1 a -all"

CSV is unrelated to forgeries, it's a solid base to reputation
systems - with SPF hoping that MTAs really limit themselves to
simple HELO-policies is shaky (but not impossible).

DKIM has a problem with forgeries, you've to declare that you
_always_ sign your headers.  SPF is in theory more flexible:

You only need to know the IPs.  In fact excluding the worst
offenders with "-" (FAIL) and use "?" (NEUTRAL) for the rest
of the world could also help.  I've not tested this approach,
I have a classic no-nonsense CIDR-based PASS or FAIL policy,
the original RMX idea, no special SPF-features.  Bye, Frank

---

From: "Scott Kitterman" <[email-masked]>
Date: Thu, 21 Jul 2005 10:43:43 -0400

>-----Original Message-----
>From: [email-masked]
>[mailto:[email-masked]]On Behalf Of Douglas Otis
>Sent: Thursday, July 21, 2005 1:18 AM

>Subject: Re: [spf-help] Re: SPF and SenderID

>depend upon this exploited feature.

I'm curious what the basis for this statement is.  Do you have statistics?

As a domain owner that relies entirely on shared MTAs, many of which are
vulnerable to cross user forgery, almost all mail that comes from my domain
(including this message) gets a Neutral ("?") result.  It's been this way
for more than a year.  So far, I've had one mail rejection as a result.  I
contacted that person via another channel and they thanked me for pointing
out that they had set up their system wrong.  Based on my experience, I
would say that no one is rejecting mail due to a Neutral result.

In addition to my personal experience, I also help handle submissions to the
SPF web site.  In the approximately 1,000 submissions we've had in the last
6 months there haven't been ANY that were caused by a Neutral rejection.  If
this were a common phenomenon, I believe that people would be complaining.

WRT to softfail ("~"), it's a little different, there we are seeing
submissions that relate to rejections on a softfail result.  The numbers are
small (I find 15 out of 1,073), but it does happen.  This is also entirely
unrelated to the question of should a domain owner that is using a shared
server that has the potential to be subject to cross-user forgery give a
Neutral result.  I thought that was the topic we were on.

I've done this for a long time and it works for me.

Scott Kitterman

---

From: "Alan DeKok" <[email-masked]>
Date: Thu, 21 Jul 2005 14:15:24 -0400

Kjetil Torgrim Homme <[email-masked]> wrote:
> in fact, they will be fully authorised to spew junk and destroy your
> reputation.  so you take your business elsewhere -- but it won't
> help.  the reputation is tied to your domain name.

  So your reputation doesn't recover when you move the domain
elsewhere?

  Wow, you hold grudges for a *long* time.

> contrast this with CSV.  

  Which does something else entirely, so comparisons aren't really
appropriate.

> it also means that if you happen to be hosted by an irresponisible
> company, you can take your business elsewhere, to a host which may
> already have good reputation.  the bad reputation is left behind
> with the bad server.

  And no one, of coure, will make the connection between "bad IP" and
"your domain is at that IP", so it will be impossible for anyone to
decide that the IP's bad reputation will affect your domain's
reputation.

  Where SPF ties reputations to domains, CSV ties it to IP's.  Since
domains are hosted at IP's, it's not a terribly large leap of faith to
use CSV to tie reputation to domains, too.  And CSV can't prevent this
leap, so we can guarantee that people will be using CSV to affect
domain reputation.

  At that point, much of your argument against SPF applies to CSV.

  Alan DeKok.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 21 Jul 2005 21:35:16 +0200

Alan DeKok writes:
>   So your reputation doesn't recover when you move the domain elsewhere?
>

If you were a reputation service provider, would you wipe a domain's 
slate clean just because it changed outgoing MX? If I were a spammer, 
I'd get n+1 outgoing MXes where n is the number of changes you require 
to forget my past misbehaviour, then I'd rotate them.

Arnt

---

From: "Alan DeKok" <[email-masked]>
Date: Thu, 21 Jul 2005 16:51:04 -0400

Arnt Gulbrandsen <[email-masked]> wrote:
> slate clean just because it changed outgoing MX?

  If reputation doesn't change over time, it's abusive.

  Ask the people who register new domains, and *then* discover that
the domain was used 3 years ago to send spam.  They often find that
"reputation services" don't care that the domain has changed hands,
for precisely the reasons you've articulated.

  The same applies to people who get assigned networks that were
previously used to send spam.  Their traffic is blocked, and they get
told "too bad".

  Abusive reputation services are not much better than spammers.

  Alan DeKok.

---

From: John Leslie <[email-masked]>
Date: Thu, 21 Jul 2005 18:42:58 -0400

> So your reputation doesn't recover when you move the domain elsewhere?

   Exactly. A 60-day review cycle is considered fast.

> Which does something else entirely, so comparisons aren't really

   Comparison of effects _is_ valid...

> decide that the IP's bad reputation will affect your domain's

   True, various reputation services _will_ screw things up in any
way we can imagine, and some we can't...

> Where SPF ties reputations to domains, CSV ties it to IP's. 

   There's really no basis for such a statement.

   CSV contains a well-defined structure for reputation services.
Authorization is _by_ the domain of the HELO string, and _of_ any
actions of the MTA using that HELO string. The IP address(es) are
only for authentication that he MTA using that HELO string is one
operating under the control of that domain.

   The absolutely clear intent of CSV is to provide information
showing authorization and authentication by a _domain_ -- not by
some ISP which assigns IP addresses.

   CSV cannot stop reputation services from tieing reputation to
IP addresses, but the information it provides is tied to domains.

> Since domains are hosted at IP's, it's not a terribly large leap of
> faith to use CSV to tie reputation to domains, too. 

   It is no leap at all: CSV information shows a responsible domain
(not a responsible IP address).

> And CSV can't prevent this leap, so we can guarantee that people will
> be using CSV to affect domain reputation.

   Thank you. That is exactly our intent.

> At that point, much of your argument against SPF applies to CSV.

   I'll let others judge that. I read Kjetil's argument to be that
with SPF, your reputation will not quickly recover, and will inhibit
email "From" your domain for quite a while, whereas with CSV the
reputation which will suffer is that of the MTA sending it, and you
can immediately send email "From" your domain through a different
domain's MTA.

   (Since the MTA is generally not visible to readers of your email,
they will be unaware of any change.)

--
John Leslie <[email-masked]>

---

From: Dave Crocker <[email-masked]>
Date: Thu, 21 Jul 2005 16:59:16 -0700

>  domains are hosted at IP's

CSV uses host domain names, to identify client MTAs.

That client's IP address is derived during the authentication phase, but the 
domain name is the identity used for CSV processing, including the Domain Name 
Accreditation portion.

  d/
  ---
  Dave Crocker
  Brandenburg InternetWorking
  +1.408.246.8253
  dcrocker  a t ...
  WE'VE MOVED to:  www.bbiw.net

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 22 Jul 2005 01:48:52 +0200

Arnt Gulbrandsen wrote:

> If I were a spammer, I'd get n+1 outgoing MXes where n is the
> number of changes you require to forget my past misbehaviour,
> then I'd rotate them.

Makes sense.  You'd also manage to change the FQDN used in HELO
when the old FQDN is "burnt".  That's independent of CSV or SPF,
how do reputation systems handle it, start with "assume guilty"
for new FQDNs ?
                  Bye, Frank

P.S. FWIW, John's eplanation matches what I think how it could
work based on CSV or SPF-HELO.  Won't help against backscatter
of course, but that's no bug, only a feature covered by one of
the other schemes like an SPF MAIL FROM FAIL policy / SES / ...

---

From: gconnor <[email-masked]>
Date: Thu, 21 Jul 2005 17:03:58 -0700 (PDT)

> some ISP which assigns IP addresses.

The version of CSV that I reviewed (though it was some time ago) seemed pretty
clearly aimed at establishing a connection between the actions of an MTA and a
name -- the name used in HELO.  However, there was no information about how to
generalize behavior of many hosts with different hostnames, all in the same
domain.

Thus you can build up a good reputation for "mail1.example.net" and a bad one
for "mail2.example.net", and neither of those is connected to the other.  In
that scenario, "example.net" has no reputation, unless there are one or more
MTAs that announce themselves with "EHLO example.net".

Your message seems to suggest that you can take several hosts in the same
domain (as in "the domain of the HELO string" and not just "the HELO string")
so I'm wondering if this is now defined in CSV or if there's some other
document that tells how to do it.  If my machine is called mail1.example.net,
it makes sense to consolidate the reputation info under "example.net"... but
if my machine is called mydomain.com, do my actions affect the reputation of
"com."?  what about mail1.bbc.co.uk vs. client1.demon.co.uk?

If the reputation attaches to a single name and doesn't get consolidated over
multiple MTAs, what value is being added over just a list of IPs with their
own reputations?  Actually, CSV would be worse in that case because the
spammer can use the current date in seconds as his HELO name, even with the
same domain suffix, and have a clean reputation every time.  hmmm..

(I am of course open to the idea that I've missed something... if so please
just point me in the right direction...)

--
Greg Connor
[email-masked]

Everyone says that having power is a great responsibility.  This is a lot
of bunk.  Responsibility is when someone can blame you if something goes
wrong.  When you have power you are surrounded by people whose job it is
to take the blame for your mistakes.  If they're smart, that is.
                -- Cerebus, "On Governing"

---

From: "Alan DeKok" <[email-masked]>
Date: Thu, 21 Jul 2005 21:36:45 -0400

> CSV uses host domain names, to identify client MTAs.

  The comment I was addressing discussed CSV tying reputation to IP's,
which is all I was responding to.

> That client's IP address is derived during the authentication phase,
> but the domain name is the identity used for CSV processing,
> including the Domain Name Accreditation portion.

  Which sort of negates the original comment that claimed SPF has a
problem because it ties reputation to domains, whereas CSV doesn't
have that problem.

  Alan DeKok.

---

From: "Alan DeKok" <[email-masked]>
Date: Thu, 21 Jul 2005 22:11:10 -0400

>    Exactly. A 60-day review cycle is considered fast.

  Any review is better than no review.  The original comment was
insinutating that no review was done.

>    Comparison of effects _is_ valid...

  Agreed.

> I read Kjetil's argument to be that with SPF, your reputation will
> not quickly recover, and will inhibit email "From" your domain for
> quite a while,

  I'm not sure why that is, as the explanation wasn't clear to me.
Nothing in CSV or SPF that I can see indicates how quickly reputation
will recover.  So assertions that reputation will recover more quickly
for one than the other are unwarranted.

...
> whereas with CSV the reputation which will suffer is
> that of the MTA sending it, and you can immediately send email
> "From" your domain through a different domain's MTA.

  So... CSV ties reputations to domains, or to MTA's?

  The confusion here is that there are multiple domains that may be
used for authorization.  EHLO, and MAIL FROM.  When CSV uses EHLO,
sending "MAIL FROM" a particular domain is invisible to CSV, because
it's not looking at MAIL FROM.  And in that case, reputation is tied
to MTA, not to the "MAIL FROM" domain.

  If CSV uses "MAIL FROM" for reputation, then it has pretty much the
same issue as any other proposal using that field.

  Alan DeKok.

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Fri, 22 Jul 2005 12:24:28 +0200

[gconnor]:
>
>   The version of CSV that I reviewed (though it was some time ago)
>   seemed pretty clearly aimed at establishing a connection between
>   the actions of an MTA and a name -- the name used in HELO.
>   However, there was no information about how to generalize behavior
>   of many hosts with different hostnames, all in the same domain.
>   
>   Thus you can build up a good reputation for "mail1.example.net"
>   and a bad one for "mail2.example.net", and neither of those is
>   connected to the other.  In that scenario, "example.net" has no
>   reputation, unless there are one or more MTAs that announce
>   themselves with "EHLO example.net".

in my experience it is no longer common for the outgoing SMTP server
to have the name of an e-mail domain in use.  we had such a setup ten
years ago, but even then many thought it old fashioned to have a host
with the same name as the delegated domain.  from RFC 2821:

   -  The domain name given in the EHLO command MUST BE either a
      primary host name (a domain name that resolves to an A RR) or,
      if the host has no name, an address literal as described in
      section 4.1.1.1.

(address literals obviously can't be used with CSV.)  I don't think
this aspect of SMTP is going to be changed, and there is little
incentive to do so.

>   Your message seems to suggest that you can take several hosts in
>   the same domain (as in "the domain of the HELO string" and not
>   just "the HELO string") so I'm wondering if this is now defined in
>   CSV or if there's some other document that tells how to do it.  If
>   my machine is called mail1.example.net, it makes sense to
>   consolidate the reputation info under "example.net"... but if my
>   machine is called mydomain.com, do my actions affect the
>   reputation of "com."?  what about mail1.bbc.co.uk
>   vs. client1.demon.co.uk?

this is where SPF tries to find the zone cut and uses dozens of DNS
queries, but that won't help, since a spammer can just as easily
delegate himself arbitrarily many subdomains.

>   If the reputation attaches to a single name and doesn't get
>   consolidated over multiple MTAs, what value is being added over
>   just a list of IPs with their own reputations?

it's a practical issue.  doing a SRV lookup based on the IP would mean
a lookup in in-addr.arpa.  for many sites, adding non-PTR records to
their reverse zones would be excedingly difficult.  many corporate
users can't even get their IP provider to update their PTR records to
be consistent.  since CSV uses the HELO name as the basis, only
administrative access to the domain with the HELO name is needed to
publish CSV.

>   Actually, CSV would be worse in that case because the spammer can
>   use the current date in seconds as his HELO name, even with the

he would need to add SRV records for each and every HELO name, but a
suitably savvy spammer (if such exists) could make a magic wildcard
syntax in the DNS server for this purpose.  actually, _I_ want such a
wildcard to publish _negative_ SRV records for our workstations.  as
it is, we would need to add 35296 negative SRV records to our zone to
fully implement CSV...

>   (I am of course open to the idea that I've missed something... if
>   so please just point me in the right direction...)

likewise.
-- 
Kjetil T.

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Fri, 22 Jul 2005 12:51:50 +0200

>   insinutating that no review was done.

for many users, 60 days might as well be never.

no one can tell how the reputation services for CSV will operate, but
we can look at the DNSBLs in use today.  several of these have a
stated policy of never removing any IP addresses used for spamming.
e.g., SORBS requires the netblock to switch owners, _and_ the new
owner to donate USD 50 per IP address to charity to remove the
listing.

>   > I read Kjetil's argument to be that with SPF, your reputation
>   > will not quickly recover, and will inhibit email "From" your
>   > domain for quite a while,
>   
>   Nothing in CSV or SPF that I can see indicates how quickly
>   reputation will recover.  So assertions that reputation will
>   recover more quickly for one than the other are unwarranted.

as the two reputations in CSV aren't connected, it is safe to say that
the recovery will be quick...  for SPF, there is no telling, it is out
of the domain administrator's hands.

>     The confusion here is that there are multiple domains that may
>   be used for authorization.  EHLO, and MAIL FROM.  When CSV uses
>   EHLO, sending "MAIL FROM" a particular domain is invisible to CSV,
>   because it's not looking at MAIL FROM.  And in that case,
>   reputation is tied to MTA, not to the "MAIL FROM" domain.

correct.  tying it to the MTA is what makes CSV deployable for just
about any e-mail provider, whereas SPF in most cases require large
changes both in infrastructure and in customer behaviour.

>     If CSV uses "MAIL FROM" for reputation, then it has pretty much
>   the same issue as any other proposal using that field.

yes, I think it has been established by now that "MAIL FROM" can't be
used without breaking e-mail or being vulnerable to replay attacks.
-- 
Kjetil T.

---

From: John Leslie <[email-masked]>
Date: Fri, 22 Jul 2005 07:37:33 -0400

> The version of CSV that I reviewed (though it was some time ago)

   (It really hasn't changed much.)

> seemed pretty clearly aimed at establishing a connection between the
> actions of an MTA and a name -- the name used in HELO. 

   Exactly.

> However, there was no information about how to generalize behavior of
> many hosts with different hostnames, all in the same domain.

   We do not consider that issue in-scope.

   Undoubtedly, some reputation services will take on that task; and
a number of approaches come to mind. The most useful approaches would
seem to be those which merely gather domain-wide information to be
used when name-specific information is sparse.

> Thus you can build up a good reputation for "mail1.example.net" and
> a bad one for "mail2.example.net", and neither of those is connected
> to the other. 

   This is exactly true, as far as CSV is concerned.

> In that scenario, "example.net" has no reputation, unless there are
> one or more MTAs that announce themselves with "EHLO example.net".

   Correct, as far as CSV is concerned. Reputation services, however,
may aggregate available information for subdomains, and use it in
some fashion.

> the same domain (as in "the domain of the HELO string" and not just
> "the HELO string") so I'm wondering if this is now defined in CSV
> or if there's some other document that tells how to do it. 

   There has been no change: we still consider it out-of-scope.

   My wording was (deliberately) vague, so as to cover both the
exact CSV meaning and some possible interpretations by reputation
services.

> If my machine is called mail1.example.net, it makes sense to
> machine is called mydomain.com, do my actions affect the reputation of

   Though we consider this issue out-of-scope, a likely implementation
by reputation services would be to carry it upwards to the registered
entity: thus stopping at mydomain.com, demon.co.uk, or (hopefully)
somecompany.city.st.us.

> just a list of IPs with their own reputations? 

   The granulatity match is better; and the (out-of-scope) aggregation
of subdomains is far easier.

> same domain suffix, and have a clean reputation every time.

   Correction: with that practice, the spammer would have "no
reputation" every time. CSV intends that other measures be applied
in cases of "no reputation" -- one example being IP blacklists.

---

From: John Leslie <[email-masked]>
Date: Fri, 22 Jul 2005 07:43:42 -0400

> for one than the other are unwarranted.

   Agreed. The essential argument has been that CSV's reputation is
more granular: it's the reputation of the MTA (or group of MTAs)
which will suffer, not the reputation of the domain listed in the
"From" address. Thus, you don't need to wait for the reputation to
recover: you can simply change MTAs to one with a good reputation.

   Exactly.

> same issue as any other proposal using that field.

   CSV never pays any attention to MAIL-FROM.

---

From: John Leslie <[email-masked]>
Date: Fri, 22 Jul 2005 07:54:46 -0400

Kjetil Torgrim Homme <[email-masked]> wrote:
> 
> actually, _I_ want such a wildcard to publish _negative_ SRV records
> for our workstations.  as it is, we would need to add 35296 negative
> SRV records to our zone to fully implement CSV...

   CSV _has_ been updated to allow a domain to specify that any
subdomain lacking a specific SRV record is "not authorized". This,
however, is outside the central purpose of CSV -- which is to enable
bypassing "expensive" and sometimes-inaccurate other mechanisms.

   (The revised spec does not require every CSV-compliant MTA to
search for that bit: we expect mechanisms to evolve for querying
databases of domains which set it.)

---

From: "Hector Santos" <[email-masked]>
Date: Fri, 22 Jul 2005 08:07:23 -0400

----- Original Message -----
From: "Kjetil Torgrim Homme" <[email-masked]>

> changes both in infrastructure and in customer behaviour.

hmmm, I hate to get into dumb debates.  But I can't help myself.

CSV does not address the TARGET problem of malicious transactions.  It has
no practical value other than ENFORCE a new level of secured transactions
and this is only DOABLE within compliant systems.

In other words,  the problem is the bad people.  Not the Good people.  This
was proven over and over again by implementators and data research
organizations.  If you take any protocol and you analyze the results, you
will find that at both ends of spectrum, it will have low false positives
and low false negatives.  See  David Silver, VP of MarkMonitor presentation
at:

http://metahost.savvislive.com/microsoft/20050712/GS6_auth_scorecard.asx

In short:

    Bad systems do not comply.
    Good systems comply.

This is the fundamental issue that the "spammers, spoofers", etc, rely on.
What makes phishers a touch problem is that they LOOK like compliant systems
at the SMTP level.   This is part of the "middle" segment, the ones you
can't catch, that David was referring to and it matches with all our
emperical statistics we have done as well.

> used without breaking e-mail or being vulnerable to replay attacks.

Not quite.

Our success with our AVS system is fundamental based on "SMTP compliance."

Over 60-80% of all transactions are BAD - that's a fact. there is no dispute
here and if anyone says otherwise at this point of time, they simply don't
know what they are talking about.

If SMTP requires that the MAIL FROM: be fundamentally required and correct,
then it should go without saying that it is INDEED a verifiable entity -
regardless of how you go about it and there are quite a few ways to do it.

This is different from the client domain (HELO/EHLO) where it is WRITTEN in
stone that it could be BAD and thus it not required to be correct for
accepting a transaction.   But even then, there are new levels of
consideration that one can implement for client domain validation - i.e,
domain literal compliance, local machine domain spoofing.

We will implement any GOOD idea and their many reasons we haven't
implemented CSV.  It just isn't a practical solution today.  However, that
does not say that a closed net or group of related mail networks (vendor to
vendor, etc) can not use it, but we already have ways to address trusted
relationships.  So it would be a WASTE of time to use CVS today.

Now if we totally revamped the industry and you enforced a new LEVEL of
client domain requirements, including A/R, then we can begin to use it.

But then we are still back at square one:

  - How to address the TARGET problem of legacy, non-compliant
    systems that the BAD people are basing their distribution on.

and that is the PROBLEM.  If it was not, we would not be here today.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Fri, 22 Jul 2005 15:18:02 +0200

[Hector Santos]:
>   > correct.  tying it to the MTA is what makes CSV deployable for
>   > just about any e-mail provider, whereas SPF in most cases
>   > require large changes both in infrastructure and in customer
>   > behaviour.
>   

oh well, one man's dumb debate is enlightening for others (such as me :)

>   CSV does not address the TARGET problem of malicious transactions.
>   It has no practical value other than ENFORCE a new level of
>   secured transactions and this is only DOABLE within compliant
>   systems.

I don't think anyone believes that CSV will be much help in the
struggle against spam, but it will help establish clearer
responsibilities and accountability.  we already reject half a million
delivery attempts daily based on bogus HELO values.  CSV will increase
that hit rate (probably not by much, though), and remove some of the
ad hoc rules in our system.

>   In other words, the problem is the bad people.  Not the Good
>   people.  This was proven over and over again by implementators and
>   data research organizations.  If you take any protocol and you
>   analyze the results, you will find that at both ends of spectrum,
>   it will have low false positives and low false negatives.  See
>   David Silver, VP of MarkMonitor presentation at:
>   

that's not what I heard in the above talk.  smart bad systems comply,
too.  one example from the talk is that both "pfizer.com" and
"pfizre.com" publish SPF records.

the talk also emphasises a number of problems with managing SPF for
corporate users, e.g., coordinating the use of multiple domains across
departments.  this is interesting since CSV and DKIM has no problems
with this and allows easy delegation of responsibility.

>   This is the fundamental issue that the "spammers, spoofers", etc,
>   rely on.  What makes phishers a touch problem is that they LOOK
>   like compliant systems at the SMTP level.  This is part of the
>   "middle" segment, the ones you can't catch, that David was
>   referring to and it matches with all our emperical statistics we
>   have done as well.

yes, until we get full deployment, the middle section will be a
problem.  this is why it is critical to agree on a standard which is
deployable by everyone.

>   > yes, I think it has been established by now that "MAIL FROM"
>   > can't be used without breaking e-mail or being vulnerable to
>   > replay attacks.
>   
>   Not quite.
>   
>   Over 60-80% of all transactions are BAD - that's a fact. there is
>   no dispute here and if anyone says otherwise at this point of
>   time, they simply don't know what they are talking about.

no disagreement here!

>   If SMTP requires that the MAIL FROM: be fundamentally required and
>   correct, then it should go without saying that it is INDEED a
>   verifiable entity - regardless of how you go about it and there
>   are quite a few ways to do it.

sure.  I was talking about going _beyond_ SMTP.  we require the domain
name used in MAIL FROM to be e-mail routable (ie. have valid A or MX
record).  you might say this goes beyond RFC 2821, but in any case
there is no need to formalise this behaviour.  I think most will think
it is the natural thing to do.

>   This is different from the client domain (HELO/EHLO) where it is
>   WRITTEN in stone that it could be BAD and thus it not required to
>   be correct for accepting a transaction.

exactly, which is why we need an explicit method to cover for the
lee-way RFC 821 provided.

>   implemented CSV.  It just isn't a practical solution today.

do you have any technical problems doing it?

>   However, that does not say that a closed net or group of related
>   mail networks (vendor to vendor, etc) can not use it, but we
>   already have ways to address trusted relationships.  So it would
>   be a WASTE of time to use CVS today.

CSV is still in draft, so I understand fully that you're holding off.

>   and that is the PROBLEM.  If it was not, we would not be here

CSV will help somewhat (see number of bad HELO above), but DKIM is
much closer to be a solution for that problem.
-- 
Kjetil T.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 21 Jul 2005 21:55:03 -0700

>  which is all I was responding to.

CSV uses domain names, to refer to accreditation information.  It does not use 
IP Addresses.

  d/
  ---
  Dave Crocker
]

---

From: Douglas Otis <[email-masked]>
Date: Fri, 22 Jul 2005 10:04:50 -0700

> I'm curious what the basis for this statement is.  Do you have  

No statistics.  I am repeating comments I have heard from  
administrators at the meeting.  When perhaps a fraction of a percent  
of non-abusive domains publish these records, saying 'many' would of  
course be like describing the fleas of a flea.  It was not clear if  
this was the result of some type of reputation response due to abuse  
of the '?', which effectively says 'never mind'.  If you feel this to  
be in error then '?'.  It does follow that I had expected and would  
be a natural response when seeing this as means to prevent the  
exploit of a neutral result.

-Doug

---

From: Douglas Otis <[email-masked]>
Date: Fri, 22 Jul 2005 10:01:39 -0700

> ad hoc rules in our system.

Reputation systems will move away from reliance upon using the IP  
address in favor of the domain name when made possible by DKIM, for  
example.  DKIM clearly provides authentication without requiring many  
infrastructure alterations.  One thing not protected is the network  
resources.  A reputation service can and often does offer a  
significant level of resource protection however.  With either DKIM  
or Sender-ID there can be no resource protect that results from the  
application of reputation following the acceptance of the message.

One key, fair, and safe requirement for any reputation system would  
be the accrual of behavior be based upon authenticated identities.   
Here is where CSV becomes vital.  CSV allows the extension of name  
based reputation into also protecting the network resources.   I know  
that SPF claims to be able to "authenticate" the HELO, but with SPF  
Classic, it is all (which includes a list of problems) or nothing.   
The high required minimum number of lookups for the SPF records  
absolutely prevents this record from playing a resource protective  
role.  CSV not only does NOT need wildcards to express a domain-wide  
policy, but expects this function can be done using a single lookup  
prior to any further exchanges, including the message.

When there is a failure to find the CSV based off the HELO, then, as  
an option, there is a means to quickly discover whether the domain  
offers a CSV policy.  A domain name provides a fair amount of  
information.  Reputation can assessed based upon this ancillary  
information to hold the appropriate level accountable.  There could  
be methods that propagate accountability up the name tree, and I  
would say that bad reputation always would move down the tree.

A school on a single T1 may tax the budget $450/month.  Without  
resource protection, the school may find that that this resource is  
being consumed by a flood of spam in effect wasting this investment.   
Here resource protection becomes vital.  When perhaps 80% of domain  
owners share servers, the use of IP addresses tends to paint with too  
broad of a brush.  To move into the use of name reputation to be more  
selective about who the bad actors are, then CSV offers that  
possibility.  A CSV lookup failure will force the fail-over to the  
use of IP address to assess reputation.  It could be to your  
customers advantage to not be tied to the IP address reputation.  CSV  
becomes a way to offer an alternative reputation specific for the  
client.  Even when the HELO is held static, this still permits an  
authenticated name as a basis for reputation accrual.

CSV will hold the domain granted access or the administrator granting  
access accountable.  SPF however holds whoever authorized the server  
accountable.  Those that authorize the server are likely not  
responsible for who has access or who uses their domain.  Yes, it is  
possible the server may force this restriction.  There are also  
domain owners that will not trust such restrictions, and will want  
their own IP address that is never shared with other domains.  I want  
to see what email service provider will indemnify there customers  
from others using their domain on the server.

-Doug

---

From: "Alan DeKok" <[email-masked]>
Date: Fri, 22 Jul 2005 13:15:04 -0400

Dave Crocker <[email-masked]> wrote:
> CSV uses domain names, to refer to accreditation information.  It does not
>  use IP Addresses.

  Once again: using the assumptions of the original poster (however
right or wrong), I showed that the original argument was invalid.

  You appear to have concluded that my quoting of those assumptions
means I believe them.  This conclusion is illogical.

  If I quote the Bible, will you tell people I think I'm Jesus Christ?

  Alan DeKok.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 22 Jul 2005 22:23:49 +0200

Kjetil Torgrim Homme wrote:

> address literals obviously can't be used with CSV.

Nor with SPF, but MTAMARK could help (directly or
indirectly as source for SIQ or traditional DNS?L)

[...]
> this is where SPF tries to find the zone cut

That was never implemented and finally removed from
the spec. => no more zone cut in the SPF RfC

> dozens of DNS queries

More like one (q=ns), but as I said it was removed.
CSV uses in the worst case six queries instead of a
zone cut.

> a spammer can just as easily delegate himself arbitrarily
> many subdomains.

Yes.  The CSV idea is pretty irrelevant to fight abuse, it
is more about accelerated processing of white / grey MTAs,
not about intentionally black hats...  Unless you're ready
to reject all mails from unidentified sources.  Bye, Frank

---

From: "Alan DeKok" <[email-masked]>
Date: Fri, 22 Jul 2005 17:58:04 -0400

> stated policy of never removing any IP addresses used for spamming.

  The solution to abusive reputation providers is well-known, at least
outside of the computer world.  See various "eco-friendly" logos, and
how their chain of responsibility (i.e. certification) is structured.

  You need at least a two-level reputation service.  One which
interacts with customers, and the other which certifies the reputation
services.  This structure has been demonstrated to solve a lot of
issues with abusive and/or corrupt reputation services.

  Alan DeKok.

---

From: Douglas Otis <[email-masked]>
Date: Sat, 23 Jul 2005 04:54:36 -0700

> issues with abusive and/or corrupt reputation services.

There is a conflict of interests which may erode the integrity of what
is typically described as accreditation.  When the revenue stream is
primarily from senders, then these services will likely respond to the
desires of the senders.  When the revenue stream is primarily from the
recipients, then services will likely respond to the desires of the
recipients.  There is often a large difference between these two
desires.

Having said that, there may be an additional role to supplement the
basic trustworthiness of the communication.  Such a service could be
used in conjunction with recipient based services, as a check against
the sender based service.  Verifying keys comes to mind which is
typically the role of a CA.  A CA will normally worry about getting
paid, and by whom.  They are typically more concerned about getting the
'by whom' part right to protect their reputation as a CA.

Rather than just using a lock icon to indicate security, each CA should
offer their own short-cut icon that prefixes the name being validated.
                       |
   ---------------------
   | CA |  example.com |
------------------------

This information should appear without expecting the user to investigate
the source of the certificate.  These two pieces of information can be
used together.  A CA run by the FDIC for example, should know who are
the banks.  You as a consumer, may want to know what is the better bank
to do business with, which will require a different service that offers
information on behalf of the consumer rather than the bank.

-Doug

---

From: "Alan DeKok" <[email-masked]>
Date: Sat, 23 Jul 2005 13:12:21 -0400

Douglas Otis <[email-masked]> wrote:
> desires of the senders. ...

  Exactly.  Which is why a two-level system solves a lot of those issues.

  Did you have comments about a two-level system, or were you just
agreeing that one-level reputation systems are problematic?

  Tying multiple first-tier reputation services together won't help.
The result is still a first-tier reputation service.

  Alan DeKok.

---

From: Dave Crocker <[email-masked]>
Date: Sat, 23 Jul 2005 23:52:05 -0700

>  stated policy of never removing any IP addresses used for spamming.

I would hope that reputation services accessed with CSV's DNA are not "for" CSV. 

CSV provides a query mechanism, but what is significant about reputation ratings 
is not the access method but the nature of the identity being assessed and the 
policies for assessing it.

The identity that CSV covers is the administrator of a host name that is cited 
by the client SMTP.  More simply, CSV involves the identity of an MTA operator.

It is not an accident that CSV targets the same kinds of identities for which 
there is the most experience with white- and black-lists.  Although they use IP 
Addresses, they pertain to operations identities, not author identities.

Hence, it is certainly reasonable to look at the range of existing white- and 
black-list operations and project that their considerable variation in 
assessment policies would be similar to the listing services that might be 
queried with CSV's DNA.

>  the recovery will be quick...  

No doubt my confusion is from not reading this thread carefully enough, but I do 
not know what "two" reputations are being referenced.  CSV deals with one 
identity, namely the host name provided in the client smtp's helo/ehlo command.

>  used without breaking e-mail or being vulnerable to replay attacks.

At some point, having detailed discussion about a specification ought to presume 
that participants have read the specifications.

  d/
  ---
  Dave Crocker

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sun, 24 Jul 2005 15:17:33 +0200

On Sat, 2005-07-23 at 23:52 -0700, Dave Crocker wrote:
> >  no one can tell how the reputation services for CSV will operate, [...]
> 

well, perhaps.  it seems to me that the CSV-DNA spec is a bit
schizophrenic on this point.  it makes sure there will be no future
conflicts with similar schemes for other services (e.g., HTTP servers),
but on the other hand it's specifically tailored to client SMTP
("_VOUCH._SMTP" etc.).  the EHLO domain name can be chosen relatively
arbitrarily, and you should be careful not to claim that reputable use
of the EHLO name extends to other services.  the method of ascertaining
the name is an inextricable part of the authenticated identity.

more relevant to the discussion, if DNA can be applied to identities
established with mechanisms other than the implied CSV-CSA, this MUST in
my humble opinion be tagged in the accreditation record with a service
different from "MARID", and as such the interpration will be out of
scope for the CSV-DNA specification.  we do not want to (potentially)
repeat the MAIL FROM vs. PRA mistake, do we?

> identity, namely the host name provided in the client smtp's helo/ehlo command.

the scenario was that a domain owner had chosen an e-mail provider with
bad reputation for its MTA name.  if the domain owner switches e-mail
provider, the new provider's MTA name will be completely independent,
and the domain owner's sent e-mail will instaneously be rid of the bad
reputation.

> that participants have read the specifications.

I'm sorry, I have only read the specifications, and have not followed
the discussion on mailing lists.  I guess I should jump in and ask for
clarifications to the text where I feel it's needed.
-- 
Kjetil T.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 24 Jul 2005 10:23:36 -0700

>  of the EHLO name extends to other services.

that might be why the _vouch qualifies (is specifically linked to) _smtp.

>  if DNA can be applied to identities

I've read this paragraph several times.  I am simply not understanding it.  At 
the least, I have no idea what the reference to tagging with a service different 
from marid means.

>  and the domain owner's sent e-mail will instaneously be rid of the bad

That's why the helo/ehlo name is associated with the MTA operator.  Your 
phrasing suggests that the "domain owner" is somehow separate from the "email 
provider", whereas the ehlo name IS the email provider.

  d/
  ---
  Dave Crocker

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sun, 24 Jul 2005 20:10:00 +0200

> that might be why the _vouch qualifies (is specifically linked to) _smtp.

yes, but it doesn't link it to EHLO.  there are other identities which
can be used to for SMTP clients.  if there isn't, why did you object
when I implied CSV-DNA only can be used with CSV-CSA?

> the least, I have no idea what the reference to tagging with a service different 

since I clearly isn't very good at explaining myself, let's start with
first principles.  we have an domain name which is authenticated somehow
whose reputation we want to check.  we look up PTR records, and as per
CSV-DNA, these begin with "_vouch._smtp".  at this point, we know the
pointers should link to CSV-DNA compliant reputation services.  we do
not know whether the name space of our authenticated domain name matches
the name space of identities employed by the reputation service.  this
name space must be declared in the TXT RR, and it's the very first field
in a record.  the draft uses the term "service" rather than  "name
space", however.  draft -02 says the service name MUST be "MARID", but
it does not identify the name space used.  being part of the CSV suite,
it is natural to ASSUME the CSV-CSA name space is used, but as far as I
can tell, the specification is entirely silent on this subject.

now, you were claiming CSV-DNA could be used with other mechanisms, and
therefore name spaces, than CSV-CSA:  "I would hope that reputation
services accessed with CSV's DNA are not "for" CSV."

I don't think that is true with -02 of the draft, since those
accreditation records necessarily would have to break a MUST, namely the
name of the service, or else risk misinterpretation of the record.

avoiding this problem is easy, the accreditation records should include
a fixed word identifier as a preamble (say "DNA"), so that it is clear
that
    DNA,MARID,1,A
and
    DNA,MX,1,A
both should be handled with the semantics of CSV-DNA (the "MX" name
space is just an example and at this time unspecified).  the current
alternative is
    MARID,1,A
and
    MX,1,A
leaving the latter completely undefined, requiring its specification to
copy elements from the CSV-DNA document rather than just build upon it.

(the service name "MARID" is not appropriate, btw, it really should be
"CSA" to avoid unnecessary confusion.)

> provider", whereas the ehlo name IS the email provider.

I don't think it's worthwhile to try explain this further, I suggest you
just go back and read the thread rather than have me rehash it here.
-- 
Kjetil T.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 24 Jul 2005 11:39:31 -0700

>  a fixed word identifier as a preamble (say "DNA"), so that it is clear

I think that you are confusing the semantics with the mechanism.  DNA provides a 
mechanism for reporting some information.  The semantics exist outside the 
mechanism.

1. The client-smtp reputation/accreditation assessment that is associated with a 
particular domain name exists in its own right, independent of the mechanism 
used (eg, DNA) to report it.

2. A particular client choose to declare itself associated with that domain 
name. CSA is one way to make that declaration.

You are suggesting that the assessment information be labeled according to the 
mechanism used to communicate it.  That's not a good idea.  

The assessment does not depend on the reporting mechanism.  The assessment 
information certainly *does* need to be careful in the way it specifies the 
identity and labeling what is being assessed, but it should not specify how it 
is reported.

  d/
  ---
  Dave Crocker

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sun, 24 Jul 2005 21:10:41 +0200

> mechanism for reporting some information.  The semantics exist outside the 

okay, let's say "MARID" denotes "DNA", but that leaves no method of
declaring name space.

> used (eg, DNA) to report it.

there is no single correct definition of "domain name", just look at the
discussions here suggesting it can be determined based on reverse DNS,
MAIL FROM, PRA or EHLO, or any number of other methods.  declaring that
_any_ method of authentication is equally valid is a recipe for
disaster.  the accreditation record must declare stringently what it
covers.

> name. CSA is one way to make that declaration.

yes.  when there are many methods, the client may choose to point to a
reputation service which has accrued reputation based on some other name
space.

> identity and labeling what is being assessed, but it should not specify how it 

not the reporting mechanism per se, but the name space used is a crucial
part of the identity.

CSV-DNA must either be locked to CSV-CSA, or it must explicitly cater
for other name spaces.
-- 
Kjetil T.

---

From: "Scott Kitterman" <[email-masked]>
Date: Sun, 24 Jul 2005 22:16:59 -0400

>Sent: Friday, July 22, 2005 1:05 PM
>To: Scott Kitterman
>Cc: MARID
Some recent statistics posted on spf-discuss showed, IIRC, that Neutral was
not a strong spam sign.  Roughly half of Neutral results were classified Ham
and roughtly half as Spam.  Given that most e-mail being sent today is Spam,
Neutral results actually make a very poor indicator for Spam.

There are some popularly forged domains for which this is not true.  There
was also some recent discussion about strategies to deal with this.  I don't
recall anyone advocating outright rejection of all messages with Neutral
results.  The sensible strategy seemed to set thresholds and start to reject
once a certain number had been received in a certain period.

Softfail was a decent indicator of forged spam, much different than Neutral
as I recall.

Scott Kitterman

---

From: John Leslie <[email-masked]>
Date: Tue, 26 Jul 2005 18:31:58 -0400

I feel this deserves a reply, though I don't know exactly what to say:

Kjetil Torgrim Homme <[email-masked]> wrote:
>> I think that you are confusing the semantics with the mechanism. 
>> DNA provides a mechanism for reporting some information.  The
>> semantics exist outside the mechanism.
> 

   "MARID", you should recall, stands for "MTA Authorization Records
in DNS", which is exactly the subject of DNA. So "MARID" denotes the
subject matter; DNA specifies how the reporting is done.

>> 1. The client-smtp reputation/accreditation assessment that is
>>    associated with a particular domain name exists in its own right,
>>    independent of the mechanism used (eg, DNA) to report it.
> 
> there is no single correct definition of "domain name",

   Actually, DNS defines it quite exactly. That is the context here.

> just look at the discussions here suggesting it can be determined
> based on reverse DNS, MAIL FROM, PRA or EHLO, or any number of other
> methods. 

   There are a number of fields which contain domain-names. The DNA
mechanism could in principle be applied to any of them. In fact, we
initially wrote DNA such that it could be widely applied; only later
did we limit it to the HELO field and CSV.

> declaring that _any_ method of authentication is equally valid is
> a recipe for disaster. 

   I can't think of anyplace that has been declared.

> the accreditation record must declare stringently what it covers.

   DNA has declared that as stringently as we believe appropriate.
Beyond that, it's up to each accreditation service to declare the
exact meaning of their accreditation.

   (There is some ongoing discussion about recommending temporary
errors, but it's unlikely to reach consensus. OTOH, there is consensus
that _reputation_ services should have a way to recommend temporary
errors and/or greylisting.)

>> 2. A particular client choose to declare itself associated with
>>    that domain name. CSA is one way to make that declaration.
> 
> yes.  when there are many methods, the client may choose to point
> to a reputation service which has accrued reputation based on some
> other name space.

   In CSV, sending SMTP client domains point to _accreditation_ services.
The receiving SMTP server would be the party using reputation services.
We've tried to keep that distinction clear through the CSV specs.

>> You are suggesting that the assessment information be labeled
>> according to the mechanism used to communicate it.  That's not a
>> good idea.
>> 
>> The assessment does not depend on the reporting mechanism.  The
>> assessment information certainly *does* need to be careful in the
>> way it specifies the identity and labeling what is being assessed,
>> but it should not specify how it is reported.
> 
> not the reporting mechanism per se, but the name space used is a
> crucial part of the identity.
> 

   DNA is locked to CSA, though a similar mechanism could easily cater
to other name-spaces.

   It might help to point out that in section 6 of the DNA spec, we
intended that in "Name: <domain being checked>.<Accreditation Service>"
"<Accreditation Service>" is not necessarily the name of the service
provider: if the same provider provides different accreditation services
they might well choose to use subdomains for different services. We
believe there is ample room for DNA-like mechanisms covering names
obtained from other sources than HELO strings. We do not believe there
is any need to limit how these might be specified and reported.

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Wed, 27 Jul 2005 14:23:45 +0200

> subject matter; DNA specifies how the reporting is done.

okay, I thought "MARID" was dead and that references to it were
obsolete.  in any case, I think it's better to use "how it is done" as
the tag, since the tag identifies exactly that for an implementation.
if you envision other methods later, it's prudent to change it to say
both "MARID" and "DNA".  or just "DNA".

>    Actually, DNS defines it quite exactly. That is the context here.

the draft does not say so.  it is common usage to say "domain name"
about the name of the zone, and "host name" about entries in that zone.
I therefore think "domain name" should be explicitly defined (with a
reference to RFC 1034).

> did we limit it to the HELO field and CSV.

that's fine.  the problem is that the accreditation service can't say
which are valid, or even which service it has evaluated.  an SMTP client
can point to an accreditation service which validates web content on the
same domain name.  "_VOUCH._SMTP" allows the client system to point to a
accreditation service specific to SMTP, and it could also add a
(hypothetical) "_VOUCH._HTTP" in cases where there's running a web
server on the same host.  which of these records were used, is not known
to the accreditation service, it operates blind.

>    I can't think of anyplace that has been declared.

it has been declared by omission.

> exact meaning of their accreditation.

then you need to give them the capability.

> We've tried to keep that distinction clear through the CSV specs.

sorry about the sloppy usage, but please explain the difference.  the
word "reputation" is only used twice in the specs, and not in what
appears to be a terminology establishing context.

>    DNA is locked to CSA, though a similar mechanism could easily cater

Dave Crocker doesn't seem to agree with you, which to me clearly
indicates the spec needs to be more explicit on this point.

> is any need to limit how these might be specified and reported.

since the DNA spec disallows reuse of the spec with a different name
space (the record MUST say "MARID", and you now clarified "DNA is locked
to CSA"), I'd say it limits future usage significantly.

the fix is simple: introduce a name space identifier to the
accreditation record.
-- 
Kjetil T.

---

From: John Leslie <[email-masked]>
Date: Thu, 28 Jul 2005 06:11:35 -0400

It seems to me this discussion belongs on the CLEAR mailing-list:

Kjetil Torgrim Homme <[email-masked]> wrote:
> obsolete. 

   The MARID Working Group has been closed. That doesn't automatically
make references to it obsolete; however, it does suggest that there
might be a more appropriate string to tag the DNA accreditation TXT RR.

> in any case, I think it's better to use "how it is done" as the tag,
> since the tag identifies exactly that for an implementation.

   "How it is done" doesn't covey much (to me, anyway). It's really
just a validity check, anyway, since the position of the TXT record
in DNS basically defines its meaning. And, as I said, the exact meaning
must be assigned by the accrediting organization.

   (Recall that CSA completely answers the question of whether the
sending SMTP client is authorized by the domain cited in the HELO: the
question addressed by accreditation is whether that domain has policies
and practices which "adequately" protect against abusive emails being
sent.)

> I therefore think "domain name" should be explicitly defined (with a

   Someone else is going to have to weigh in here: I don't understand
Kjetil's point, so I can't propose text.

> which are valid, or even which service it has evaluated. 

   Quite the contrary: only the accreditation service _can_ say what
service it has evaluated. The expectation in DNA is that the operation
of a sending SMTP client is being evaluated; but DNA cannot enforce it.
It is up to reputation services to determine whether the evaluation by
each accreditation service is useful for its purposes.

   Where the domain-name may have been found is quite immaterial. DNA
states that the HELO string shall be treated as identification of the
management of the SMTP client which sends it, subject to verification
through DNS lookup of SRV records that the domain in question authorizes
a client at that IP address to claim that association.

   But for purposes of accreditation, the service doing accreditation
is expected only to accredit the policies and practices of the
management. If it is asked about a name obtained some other way, it
cannot be aware of how that name was obtained.

   We are well aware that accreditation/reputation issues arise for
the various flavors of SPF: the DNA spec does not intend to make any
statement about how such accreditation/reputation issues might be
handled.

   (IMHO, if there is an issue to be raised of confusion with SPF
reputation, it sits with the "_VOUCH._SMTP" substring in the PTR lookup,
not with the tag string of the accreditation record itself. And I'm
prepared to defend the "_VOUCH.-SMTP" choice, since the SMTP protocol
covers transport, not message content.)

> an SMTP client can point to an accreditation service which validates
> web content on the same domain name. "_VOUCH._SMTP" allows the client
> system to point to a accreditation service specific to SMTP, and it
> could also add a (hypothetical) "_VOUCH._HTTP" in cases where there's
> running a web server on the same host.  which of these records were
> used, is not known to the accreditation service, it operates blind.

   I entirely agree. (You make my point, I think...)

   (It is possible that something within the text of Section 4 of the
DNA spec is confusing folks here. I will point out that the "_VOUCH._SMTP"
substring of the Target clarifies that the PTR record in question is to
recommend an accreditation service to check for DNA purposes, and that
other PTR records may also exist at the same position in the DNS tree.
DNA does not require that any actual query be done to that Target string,
but such a query would be expected to aid a in finding the explanation
of what is being accredited.)

> then you need to give them the capability.

   I fail to see how we could "give" or "take away" such a capability.

>>> CSV-DNA must either be locked to CSV-CSA, or it must explicitly
>>> cater for other name spaces.
>> 

   Someone else will have to propose text to make this more explicit.
(Dave can speak for himself if he disagrees with anything I've posted.)

>> provider: if the same provider provides different accreditation
>> services they might well choose to use subdomains for different
>> services. We believe there is ample room for DNA-like mechanisms
>> covering names obtained from other sources than HELO strings. We
>> do not believe there is any need to limit how these might be specified
>> and reported.
> 
> space (the record MUST say "MARID", and you now clarified "DNA is
> locked to CSA"), I'd say it limits future usage significantly.

   The meaning of the accreditation record is delineated by its position
in the DNS tree, not its introductory tag.

> the fix is simple: introduce a name space identifier to the

   I can't imagine any way to define such a "name space identifier"
which would do the job half as well as its position in the DNS tree.
Nor, IMHO, should we make any pretense we can do so.

---

From: John Leslie <[email-masked]>
Date: Thu, 28 Jul 2005 06:52:35 -0400

(Second try: I'm trying to adjust myself to Paris time; it's not
working well ;^)