Subject: Re: [ietf-dkim] Updated implementation report

From: Jeff Macdonald <[email-masked]>
Date: Fri, 1 Oct 2010 10:33:35 -0400

On Wed, Sep 29, 2010 at 5:22 PM, Hector Santos <[email-masked]> wrote:
> Murray S. Kucherawy wrote:
>> I've posted a new issue of the DKIM implementation report.  The most interesting changes are the inclusion of a day of sample data from AOL and a revision of the data summary reported by the OpenDKIM stats project using the updated schema, which allows a few interesting new observations.
>>
>
> Thanks for the report.
>
> I believe the most outstanding data point here is the extremely high
> original signing of mail.  AOL's 1.2 out of 1.4 billion or 86%
> signatures were 1st party and your own statistics showed 73% were 1st
> party.
>
> This validates what I have always been felt is the high promise for
> DKIM exclusive (1st party or passive 3rd party) operations.  Policy is
> inevitable to facilitate policy (domain expectation) fault detection.

Yet 55.8% of the keys were tagged as being in test mode. I also
suspect that many implementations don't offer any alternative than 1st
party. I believe this is true for IronPorts. It be interesting to see
a MTA breakdown in respect to 1st and 3rd party signatures.

-- 
Jeff Macdonald
Ayer, MA

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 01 Oct 2010 09:28:04 -0700

On 9/29/2010 4:09 PM, MH Michael Hammer (5304) wrote:
> This isn't particularly surprising to me. I've always believed there is
> a significantly higher value for 1st party signing compared to 3rd
> party. The value proposition has always seemed to be much more
> compelling to me.

You automatically jump to the conclusion that a dominance in a statistic means 
higher importance.

There are far more people who are in the "average" range of intelligence than 
there are who have an IQ over 140.  Are the people with higher IQs less important?

In an average audience of 100, I usually found that roughly 3 people had had 
their house broken into.  Since breakins are so rare, does this mean that 
locking your door really isn't necessary?  (The other side of that question is 
that burglars seem to be able to get into a house if they wish, and does this 
mean that locking your door really isn't effective?)

The nature of statistical analysis inherently constraints their actual meaning. 
  Let's be careful not to go outside those constraints.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Fri, 1 Oct 2010 12:58:09 -0400

> -----Original Message-----
> From: Dave CROCKER [mailto:[email-masked]]
> Sent: Friday, October 01, 2010 12:28 PM
> To: MH Michael Hammer (5304)
> Cc: [email-masked]
> Subject: Re: [ietf-dkim] Updated implementation report
> 
> 
> 
> You automatically jump to the conclusion that a dominance in a
statistic
> means
> higher importance.
> 
> There are far more people who are in the "average" range of
intelligence
> than
> there are who have an IQ over 140.  Are the people with higher IQs
less
> important?
> 
> In an average audience of 100, I usually found that roughly 3 people
had
> had
> that burglars seem to be able to get into a house if they wish, and
does
> this

You misinterpret what I wrote (or perhaps I wasn't clear enough) or
intended to write.

What I was trying to communicate is that the value proposition for DKIM
signing appears (At least to me) stronger for the owner/administrator of
a given domain than it would be for a given 3rd party. 

As far as your example of intelligence, your question regarding
"importance" is incomplete. Important to whom and in what context? 

Note, I didn't say that 3rd party signing was less important generally.
What I wrote (or intended to write)  was that my belief is that 1st
party signing represents a higher value proposition to 1st party signers
than 3rd party signing represents to 3rd party signers.

For the most part, I have only heard a lot of interest from ESPs to jump
in and do 3rd party signing. Their value proposition appears to be
deliverability (and that is probably on the margin).

Most of the 1st party signers I have spoken with are more focused on
abuse issues.

Mike

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 01 Oct 2010 10:05:54 -0700

> You misinterpret what I wrote (or perhaps I wasn't clear enough) or

Well, I do tend to overinterpret single data points...

> a given domain than it would be for a given 3rd party.

And my point is that the this set of data don't provide a basis for making such 
a fundamental assertion.  I'm not saying your opinion is wrong -- although I do 
think it is misguided -- but rather than this data on their own don't 
substantiate that assessment.  There are too many other possible explanations 
for this data.

(As for misguided, I think that it is a mistake to discuss DKIM 'value' in terms 
of author-vs-non-author, since DKIM is not designed to make assertions about 
authorship.  It is design to enable making assertions about handlers of email 
and it is designed to make them differentially for different streams.)

> "importance" is incomplete. Important to whom and in what context?

Exactly.  Please re-apply this point to the current topic...

> than 3rd party signing represents to 3rd party signers.

Oh.  Sorry.  I didn't get that.  It's an interesting idea but I'd want to hear 
it explored quite a lot, since the idea of value is pretty broad.  For example, 
if 3rd party signatures allow an ESP to get mail delivered better and, 
therefore, to stay in business, I'd be hard-pressed to call DKIM's 'value' lower 
than for a first-party signer.

> Most of the 1st party signers I have spoken with are more focused on

Well, it's certainly true that first party signers have an easier opportunity to 
conflate goals.

d/

-- 

   Dave Crocker

---

From: Hector Santos <[email-masked]>
Date: Fri, 01 Oct 2010 13:55:19 -0400

Jeff Macdonald wrote:
>> DKIM exclusive (1st party or passive 3rd party) operations. �Policy is

Lets keep in my that this report includes an interoperability testing 
event that took place 3 years ago, when POLICY still on the table.

Yet, at the time the main two libraries in year, in particular ALT-N, 
the host of the event, had SSP support built-in in its open source 
library!  The sendmail library also had SSP support.  They both 
followed RFC 5016 "Requirements for a DKIM Policy."

In any case, things has changed drastically in the last three years, 
and we will need "three" more years before more good information can 
be used.

Sure, millions of domains are signing but with little wide-spread payoff.

That is whats missing in this report - the PAYOFF.

The fact is this - 1st party signatures is dominant and the continue 
effort to nullify that "tidbit" doesn't seen to help anything by deny 
a reality.

As long as 100% of the domains sign with a 5322.FROM there will also 
be an binding and none separating association regardless on how much 
we want intermediary freedom to sign mail.

-- 
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

---

From: Jeff Macdonald <[email-masked]>
Date: Fri, 1 Oct 2010 16:19:20 -0400

> than for a first-party signer.

I find this exchange very interesting. I though the value of DKIM was
to provide a stable identifier. I find 1st party signing to be rather
constrained. It seems to defeat the purpose of DKIM. One might as well
resurrect DomainKeys, because it seems to have the same goals as 1st
party signers.

I'd like to propose Author Domain Signatures as signatures that the
author domain authorized. The ATPS and ALS proposals are ways of doing
that. Update ADSP to use this definition instead of "d= matches the
RFC5322:From domain".

I believe this allows everyone to get the best value of DKIM.

-- 
Jeff Macdonald
Ayer, MA

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Fri, 01 Oct 2010 23:24:10 +0200

On 09/29/2010 10:58 PM, Murray S. Kucherawy wrote:
> (You'll just have to read it to find out what they are!)

excellent work, Murray! One remark, one question.

Remark: I'd suggest to transform the AOL data to also contain 
percentages, just like the results of the other (OpenDKIM statistics) 
project. It will make it easier to compare (the results of) the two 
'projects' and in general, percentages give a better view on what was 
measured.

Question: regarding what you wrote:

    That the "To" field is the one most often associated with
    verification failures suggests some MTAs handling the message are
    correcting cases where the field is improperly formed.  A common case
    is failing to quote the comment portion of that field when required
    to do so by [MAIL].  Such corrections cause signatures to become

If this is the #1 reason that verifications fail, would there be room 
for a new canonicalization scheme, to improve verification rates? I know 
there are MTA's, implementing the principle of 'garbage in - garbage 
out', just like there are MTA's implementing the principle of 'be 
liberal in what you accept, be strict in what you emit'; the latter may 
add a missing Date field, or correct a syntactically wrong Date field, 
or modify To fields to match RFC5322 etc. This has been discussed 
before, and it is impossible to come up with a canonicalization scheme 
that addresses 100% of these modifications, but if we can address the 
top 5 or top 10 types of modifications (and hence reasons for 
verification failure), we might be able to further improve the 
verification score, dont' we? Murray, do we have any figures about the 
total percentage of DKIM signatures, which were invalidated by header 
modifications, and a complete list of which headers were modified? Are 
we talking about 5%, 1%, 0.1%, 0.01%?

This raises another question: a DKIM verification failure in itself is 
not a problem: a spammer signing with an incorrect signature, or 
replaying old (DKIM) message headers with new spam content will cause 
verification failures, which is how it is supposed to be. However, 
another category of DKIM verification failures may have to do with 
header modifications by downstream MTA's, invalidating DKIM signatures. 
The question here is: how can we gather statistics about these two (and 
maybe there are more) different categories of verification failures, or 
how can we differentiate between these two (or more) categories?

IMHO working on improvements to have more correct DKIM signatures 
survive 'transport' is important to get industry momentum for DKIM. The 
MLM problem is only one aspect of that problem. This is in line with a 
question I got this week from a customer: what is the value of a 
protocol (and why should I invest in it) if it is reliable in only 90% 
of all cases? And even if we tell him, the real figure is maybe 95%, 
then he's still asking for something in the order of 99% or 99,5%.

/rolf

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Fri, 1 Oct 2010 17:40:35 -0400

> From: [email-masked] [mailto:ietf-dkim-
> [email-masked]] On Behalf Of Jeff Macdonald
> Sent: Friday, October 01, 2010 4:19 PM
> To: [email-masked]

I find the exchange interesting as well. Of course the purpose of DKIM is to provide a stable identifier.... that does not mean that all stable identifiers should be given the same weight.

Warren Buffet is a stable identifier. Michael Hammer is a stable identifier. Which stable identifier are you going to give more weight to if the message associated with the stable identifier relates to investing? Which stable identifier are you going to give more weight to if the message associated with the stable identifier is about email authentication?

Context is always important and trying to say that a 3rd party signature/stable identifier is absolutely no different than a 1st party signature/stable identifier brings us into Animal Farm territory......

All stable identifiers are equal but some are more equal than others.

There is an inherent difference between a domain signing a message for itself and a 3rd party signing a message. It may not be stated in the RFC but it is there nonetheless. 

What you are proposing Jeff is a means to delegate signing to a 3rd party by the first party. That is different than a 3rd party who handled the message signing on the basis that it handled the message (what we have today). That is, you wish to add delegated authorization.

The domain is the domain is the domain. 3rd parties may come and go. We do not know how stable an identifier 3rd party signing is in the wild. It will be good to have more data points before engaging in this discussion. -

To pick on you a little, if a domain owner uses your approach to authorize signing by an ESP1, what is the stable identifier we are talking about? Is it specific to this customer or is it shared across customers? Does the domain owner understand potential impacts on their reputation (assuming domain based reputation systems ever get off the ground in our lifetime)?

What happens when the domain owner dumps ESP1 and goes to ESP2? Do they lose whatever (We assume fantastic) reputation they had? Do they go to square 1 or are they borrowing/renting reputation from ESP2? If they are borrowing/renting reputation from ESP2, how do they know that ESP2 isn't using their domains good reputation to help other not so good senders at ESP2? Diluting the badness so to speak.

I'm assuming this desire for 3rd party signing to have the same weight as 1st party signing is somehow related to deliverability and not abuse. I've never been a big fan of the reputation bandwagon. I view reputation as "What have you done to me today". We can all sing that tune but not do the dance in spike heels.

>From my perspective, senders generally get the reputation they deserve. It doesn't matter whether it is IP based or domain based. Mailbox providers are not stupid. They can see the practices of senders as well as the response of recipients. Many mailbox providers have better insight then the emitters of mail streams.

Just a few random thoughts on a Friday afternoon.

Mike

---

From: Douglas Otis <[email-masked]>
Date: Fri, 01 Oct 2010 14:43:33 -0700

On 10/1/10 1:19 PM, Jeff Macdonald wrote:
> On Fri, Oct 1, 2010 at 1:05 PM, Dave CROCKER<[email-masked]>  wrote:
Jeff,

The ALS extension is fine for signing with sub-domains just to isolate 
different mail streams.  You should also consider the TPA-Label draft 
which includes authentication and header field requirements in addition 
to the third-party domain authorization, as found in the simpler ATPS draft.

The ability to include added compliance requirements provides two 
significant benefits:

1) When DKIM signatures are required by policy (i.e. ADSP), then when 
another domain offers authentication based upon DKIM, SPF, TLS, EHLO, 
etc.  the Author Domain would be able to rely upon these other methods 
to safely recover from damaged or missing signatures.  As it now stands, 
there is a good chance these messages will be otherwise discarded when 
the Author Domain attempts to keep their recipients from being flooded 
with spoofed messages by using the "discardable" assertion.

2) When the Author Domain wishes to exchange messages with typical 
mailing-lists, additional header field and authentication requirements 
communicates to receivers which specific methods should be used, and 
when these are not met, the message can be rejected.  This approach 
retains protection for the entire domain otherwise exposed to phishing 
attempts.

Murray expressed a concern that these options are not allowed in a DKIM 
working group document.  However, implementation of these alternative 
methods are not specified by the working group, and the TPA-Label draft 
and can simply cite existing work.  For example RFC 5321 section 4.1.4 
states:

,---
An SMTP server MAY verify that the domain name argument in the EHLO
command actually corresponds to the IP address of the client.
However, if the verification fails, the server MUST NOT refuse to
accept a message on that basis.

'---

In other words, rather than seeing third-party policy only in terms of 
DKIM verification, robust handling must not exclude other existing 
methods.  The ability to specify these additional compliance options 
allows improved delivery rates and can extend policy specifications to 
non-participating mailing-lists not otherwise covered by ADSP.

-Doug

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 1 Oct 2010 16:21:05 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Rolf E. Sonneveld
> Sent: Friday, October 01, 2010 2:24 PM

OK, I'll look into that for the next version.

> we talking about 5%, 1%, 0.1%, 0.01%?

I don't think the data support any conclusions about possible canonicalization improvements yet.  I also can't (after admittedly spending only about 90 seconds on it) imagine what that might look like, short of something like "relaxed plus no punctuation outside angle brackets" or the like.

The OpenDKIM statistics collected so far show that of 135549 signatures received, 121821 passed (meaning 13728 failed).  There were also 6194 that passed (inasmuch as the header hashes lined up) but the body hash didn't.  So we see that failures caused by header changes beat body changes by just over two-to-one.

What we don't currently collect is a list of signed fields that were deleted in transit, nor do we collect a list of fields that were signed but not actually present when signing to prevent their later addition.  We also don't collect what exactly the various in-transit modifications were.  We also do only a limited approximate matching; if a field changes a great deal, we don't count it in order to avoid false positives (e.g. two Received: header fields could be compared and cause a false positive).  And since the contents of "z=" are assumed to be an accurate reflection of the signed header fields for the purposes of this study, we rely on people to add them and do so correctly.  So it's not a precise study, and we certainly don't collect enough information to say what the precise changes are that cause failures, but it still does reveal some interesting stuff.

> how can we differentiate between these two (or more) categories?

I suspect the best you can do, short of getting everyone along the way to save copies of all signed traffic for a while, is to encourage wider deployment of "z=" by both signers and verifiers.  That'll also be hard to do though since it doesn't actually help anyone achieve anything, other than those of us interested in advancing the protocol, and it just makes headers bigger which sometimes runs into processing limits.  And several implementations probably didn't even bother to add support for it.

I think it's rare to find anyone validating signatures except at the sender and the receiver, so any mid-stream rewrites of stuff are happening silently.

Another thing that might be interesting to collect is a study of which MTAs are making which changes that break signatures, or which signature-breaking MLM actions are the most common.  These studies would be a lot harder to conduct, however.

---

From: "John R. Levine" <[email-masked]>
Date: 1 Oct 2010 23:04:24 -0400

>> for a new canonicalization scheme, to improve verification rates?

Seems to me it would be more appropriate to add a note saying something 
like be sure your headers are all RFC 5322 compliant before signing, 
including arcana such as quoting rules in address fields, to avoid 
signature failures due to helpful relay MTAs fixing the quoting errors on 
the way through.

As far as figuring out who's doing what, it's hard to think of anything 
better than running a bunch of deliberately marginal messages through a 
variety of MTAs and see what happens.  A couple of years ago I set up a 
forwarding project, in which I asked people to set up different MTAs to 
forward mail back to me, just so we could find out about this kind of 
stuff.  The code has suffered severe bit rot but if people really wanted 
to use it, I could probably resuscitate it.

R's,
John

---

From: Barry Leiba <[email-masked]>
Date: Sat, 2 Oct 2010 10:23:52 -0400

I'd like to rein in discussion a bit, here.  Or, perhaps more
accurately, separate the threads.  It's fine if folks want to talk
about the relative value of some kinds of signatures relative to
others, and so on -- that sort of thing actually is on our charter.
But it's a separate item from getting an implementation and
interoperability report done.

To that end, let's make sure we keep the threads separate.  John's
comment, to which I'm replying here (see below), is the sort of
comment that will help finish the implementation report.  Let's take
comments about what we think the report tells us about the efficacy of
DKIM... to separate threads.  Thanks.

Barry, as chair

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Sat, 2 Oct 2010 12:13:50 -0400

I can provide one example (case) of an implementation breaking signing. In the case where a transmitted message does not specify a mime type, Ironport appliances will do the DKIM signing and afterwards add the mime type header ASCII.

This is a bit of an edge case but is illustrative of a way in which a signature might be broken.

I identified this issue (and submitted to Ironport) back in late 2007 or early 2008 (I would have to do some digging to nail it down exactly). The problem was created when the "dot stuffing" issue in their implementation was fixed).

RFC1521 (and subsequent) indicate (wording varies slightly over time/RFC):

Default RFC 822 messages are typed by this protocol as plain text in
   the US-ASCII character set, which can be explicitly specified as
   "Content-type: text/plain; charset=us-ascii".  If no Content-Type is
   specified, this default is assumed.  In the presence of a MIME-
   Version header field, a receiving User Agent can also assume that
   plain US-ASCII text was the sender's intent.  In the absence of a
   MIME-Version specification, plain US-ASCII text must still be
   assumed, but the sender's intent might have been otherwise.

So there isn't really a need to "fix" this because the RFC indicates US-ASCII is assumed if not specified - but if it were to be "fixed" by a helpful MTA, it should be done before DKIM signing rather than after.

The only reason I found this issue is that I do testing for various things using manual SMTP sessions and lazy person that I am, I do not type in the extra characters to specify content type.

I haven't posted this publicly before because it seemed (as an implementation issue) something that was somewhat esoteric.

Mike

> [email-masked]] On Behalf Of Barry Leiba
> Sent: Saturday, October 02, 2010 10:24 AM

---

From: Jeff Macdonald <[email-masked]>
Date: Mon, 4 Oct 2010 11:00:42 -0400

I'm catching up to the list here, so I'm answering these in the order
I see. Someone else may have covered some points and I apologize if it
has been beaten to death.

comments below:

> To pick on you a little, if a domain owner uses your approach to authorize signing by an ESP1, what is the stable identifier we are talking about? Is it specific to this customer or is it shared ac…

We are advising our clients to create a sub-domain for DKIM
identifiers. Something they control. Something not tied to their
From:. For those that have multiple streams, we'd recommend creating
DKIM identifiers for each stream. Using DKIM as it was intended.

> What happens when the domain owner dumps ESP1 and goes to ESP2? Do they lose whatever (We assume fantastic) reputation they had? Do they go to square 1 or are they borrowing/renting reputation from…

There are of course bad ways to implement things. Some ESPs use shared
IPs for new clients in order to meet the sending rates clients ask for
on day one. We do not have that strategy. Client's go through a
warming phase with IPs dedicated to them. With DKIM, they will either
come with their own identifier and be able to continue business as
usual or start with a new DKIM identifier. I'd advise against using an
e-Dialog DKIM identifier.

> I'm assuming this desire for 3rd party signing to have the same weight as 1st party signing is somehow related to deliverability and not abuse.

Abusers can be 1st party signers. So I don't understand why there is a
distinction being made between 1st and 3rd party.

-- 
Jeff Macdonald
Ayer, MA

---

From: Douglas Otis <[email-masked]>
Date: Mon, 04 Oct 2010 11:59:44 -0400

On 10/4/10 11:00 AM, Jeff Macdonald wrote:
> > To pick on you a little, if a domain owner uses your approach to
> > authorize signing by an ESP1, what is the stable identifier we are
> > talking about? Is it specific to this customer or is it shared
> > across customers? Does the domain owner understand potential
> > impacts on their reputation (assuming domain based reputation
> > systems ever get off the ground in our lifetime)?
>

Only when signed by the same domain (Author Domain)  matched exactly 
with the From email address domain can ADSP policy apply.  Retaining 
customers when your domain is heavily phished means having non-complaint 
messages rejected becomes a greater concern.

>  What happens when the domain owner dumps ESP1 and goes to ESP2? Do
>  they lose whatever (We assume fantastic) reputation they had? Do they
>  go to square 1 or are they borrowing/renting reputation from ESP2? If
>  they are borrowing/renting reputation from ESP2, how do they know
>  that ESP2 isn't using their domains good reputation to help other not
>  so good senders at ESP2? Diluting the badness so to speak.

This assumes reputation assignment ignores who registered the parent domain.

>  There are of course bad ways to implement things. Some ESPs use
>  shared IPs for new clients in order to meet the sending rates clients
>  ask for on day one. We do not have that strategy. Client's go through
>  a warming phase with IPs dedicated to them. With DKIM, they will
>  either come with their own identifier and be able to continue
>  business as usual or start with a new DKIM identifier. I'd advise
>  against using an e-Dialog DKIM identifier.

This also ignores the ability of bad actors to send themselves signed 
messages they can then be redistribute because DKIM is not _required_ to 
capture the SMTP RCPT TO.  Any expectation of such would cause many 
messages to be lost.

> > I'm assuming this desire for 3rd party signing to have the same
> > weight as 1st party signing is somehow related to deliverability
> > and not abuse.
>
>  Abusers can be 1st party signers. So I don't understand why there is
>  a distinction being made between 1st and 3rd party.

The difference relates to compliance related with signing practices.  
With replay ability, DKIM is not easily leveraged as a basis for 
reputation.  Reputation will likely need something developed along the 
lines of public keys or certificates offered by SMTP clients, because 
these services can track where messages are sent.  Where messages are 
sent is often the basis for identifying what is abusive.

-Doug