Subject: [ietf-dkim] Statistics about DKIM and MIME

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 24 Oct 2010 21:21:32 -0700

OpenDKIM now has enough data to make some interesting observations about signatures and MIME.

As far as MIME encodings go (only the "outermost" encoding was counted), there was a pretty common theme:

binary failed 4% of the time
quoted-printable failed 4% of the time
7bit failed 7.7% of the time
base64 failed 7.8% of the time
8bit failed 14% of the time
16bit (?!) never failed (though there was only one attempt)

I expected 8bit to fail more for some reason.

As far as MIME parts go (again, only the "outermost" MIME type was counted), most of them have about a 90-93% survival rate which is about in line with general signature survival rates.  The one that stands out is "multipart/signed" (from RFC1847) which drops to about a 65% survival rate.  I don't know much about how this is typically formatted or treated enroute, but it was easily the biggest outlier in the report.  Not sure if that should be a surprise to us or not.

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Mon, 25 Oct 2010 13:31:59 +0200

Hi, Murray,

On 10/25/10 6:21 AM, Murray S. Kucherawy wrote:
> As far as MIME encodings go (only the "outermost" encoding was 
> counted), there was a pretty common theme:
>

Interesting figures. Especially the 16bit ;-)

> As far as MIME parts go (again, only the "outermost" MIME type was 
> counted), most of them have about a 90-93% survival rate which is 
> about in line with general signature survival rates.
>

This still leaves the question open whether there is any relation 
between MIME labelling and -content transfer encoding, or none at all.

> The one that stands out is "multipart/signed" (from RFC1847) which 
> drops to about a 65% survival rate.
>

I'm not sure whether 'survival' is the correct term in your report. I 
assume you mean percentages of DKIM signatures that verify correctly as 
seen by the verifier? The other 7-10% of signatures can also come from 
Bad Actors who replay signatures with different content of the message. 
It is possible they arrive unchanged at the verifier and then fail 
verification, but that doesn't mean the (replayed) DKIM signature did 
not 'survive'.

> I don't know much about how this is typically formatted or treated 
> enroute, but it was easily the biggest outlier in the report.  Not 
> sure if that should be a surprise to us or not.
>

In general the fundamental question here is indeed about survival rate: 
what is the real and 'exact' percentage of messages, signed by domain 
example.com that still verifies correctly after n hops by the verifier 
where n = 1,2,3,4...

/rolf

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Mon, 25 Oct 2010 15:31:09 +0200

> between MIME labelling and -content transfer encoding, or none at all.

Clarification: it was my intention to say:

[...] between MIME labelling and -content transfer encoding on one side, 
and DKIM signature 'survival' on the other side. Or no relationship at all.

/rolf

---

From: "John R. Levine" <[email-masked]>
Date: 25 Oct 2010 11:07:05 -0400

> The one that stands out is "multipart/signed" (from RFC1847) which drops 
> to about a 65% survival rate.  I don't know much about how this is 
> typically formatted or treated enroute, but it was easily the biggest 
> outlier in the report.  Not sure if that should be a surprise to us or not.

I'm surprised.  That suggests something often adds the S/MIME signature 
after the DKIM signature, but as far as I know, S/MIME signatures are 
usually applied by the MUA.

Do the stats say what kind of failure it was, e.g. body hash or header 
hash?

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 25 Oct 2010 10:01:29 -0700

> -----Original Message-----
> From: John R. Levine [mailto:[email-masked]]
> Sent: Monday, October 25, 2010 8:07 AM
> To: Murray S. Kucherawy
> Cc: [email-masked]
> Subject: Re: [ietf-dkim] Statistics about DKIM and MIME
> 

Actually it's worse than I said originally.  We track pass/fail in two bits, one being whether or not the crypto lined up and the other being whether or not the body hashes matched.  Thus, it's possible to get a "pass" coupled with a body hash change.  I had only selected for the first bit.

So here are the stats again.  The first column is obviously the media type; the second is the count of signatures covering a message with that type as the outermost MIME part; the third column is the number of those that passed in both the crypto and the body hash sense, and the fourth is the pass percentage.

application/ms-tnef               26      23      88.5%
application/pdf                   16      16      100%
message/disposition-notification  10      10      100%
message/rfc822                    2       2       100%
multipart/alternative             290865  265270  91.2%
multipart/mixed                   38509   35370   91.8%
multipart/related                 7959    7149    89.8%
multipart/report                  958     883     92.2%
multipart/signed                  314     86      27.4%
text                              13      13      100%
text/calendar                     34      32      94.1%
text/html                         63144   55880   88.5%
text/plain                        72195   55415   76.8%

In the particular case of multipart/signed there were 106 messages where the RSA verification failed, but 122 where it passed but the body hash at the verifier didn't match the one in the signature.  So more failures occur from body changes than do from header changes.

---

From: Steve Atkins <[email-masked]>
Date: Mon, 25 Oct 2010 12:00:46 -0700

On Oct 25, 2010, at 8:07 AM, John R. Levine wrote:

>> The one that stands out is "multipart/signed" (from RFC1847) which drops to about a 65% survival rate.  I don't know much about how this is typically formatted or treated enroute, but it was easily the biggest outlier in the report.  Not sure if that should be a surprise to us or not.
> 
> I'm surprised.  That suggests something often adds the S/MIME signature after the DKIM signature, but as far as I know, S/MIME signatures are usually applied by the MUA.

Someone was hawking a box to sign and verify RFC 2015 PGP signatures at the enterprise firewall a few years back. That'd break DKIM signatures with a multipart/signed outermost type.

Cheers,
  Steve

---

From: Michael Thomas <[email-masked]>
Date: Mon, 25 Oct 2010 10:33:59 -0700

On 10/25/2010 10:01 AM, Murray S. Kucherawy wrote:
  In the particular case of multipart/signed there were 106 messages where the RSA verification failed, but 122 where it passed but the body hash at the verifier didn't match the one in the signature.  So more failures occur from body changes than do from header changes.

This smells suspiciously like middle boxen doing HIPAA-like things
with SMIME. Let's just blame Jon Callas :)

Mike