Subject: [ietf-dkim] Working group last call on draft-ietf-dkim-implementation-report

From: Barry Leiba <[email-masked]>
Date: Mon, 4 Oct 2010 16:41:29 -0400

Thus begins working group last call on the DKIM implementation and
interoperability report, draft-ietf-dkim-implementation-report-02:
  http://tools.ietf.org/html/draft-ietf-dkim-implementation-report
The working group last call will run through Friday, 22 October, 2010.

This implementation report will be used to advance the DKIM base spec
to Draft Standard.  Everyone please review it, and post
comments/issues. Please also post here if you've reviewed it and think
it's ready to go.

Barry, as chair
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "John R. Levine" <[email-masked]>
Date: 4 Oct 2010 17:06:10 -0400

> comments/issues. Please also post here if you've reviewed it and think

I have reviewed it, and it looks ready to go.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 4 Oct 2010 14:09:05 -0700

I wrote it, and it looks ready to go.
________________________________________
From: [email-masked] [[email-masked]] On Behalf Of John R. Levine [[email-masked]]
Sent: Monday, October 04, 2010 2:06 PM
To: Barry Leiba
Cc: DKIM Mailing List
Subject: Re: [ietf-dkim] Working group last call on draft-ietf-dkim-implementation-report

R's,
John

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Mon, 4 Oct 2010 17:17:13 -0400

> -----Original Message-----
> From: [email-masked] [mailto:ietf-dkim-
> [email-masked]] On Behalf Of John R. Levine
> Sent: Monday, October 04, 2010 5:06 PM
> To: Barry Leiba
> Cc: DKIM Mailing List

I've reviewed it and agree that it looks ready to go.

Mike

---

From: Hector Santos <[email-masked]>
Date: Mon, 04 Oct 2010 18:02:33 -0400

> comments/issues. Please also post here if you've reviewed it and think

I have only one comment.  The removal of very significant data points 
from this last revision:

   Author vs. Third-Party:  73% of the signatures observed were author
        signatures, meaning the "d=" value in the signature matched the
        domain found in the From: header field.  The remainder, therefore,
        were third-party signatures.

   Originator signatures:  1.2 billion
   Third-party signatures:  184 million

This is signification information.

Why was it removed?  Why hide this significant fact?

-- 
HLS

---

From: Michael Thomas <[email-masked]>
Date: Mon, 04 Oct 2010 14:21:06 -0700

> I wrote it, and it looks ready to go.

Dear g*d, it's the mailing list equivalent of people pressing the
"like" button on their own posts :)

Mike

---

From: "J.D. Falk" <[email-masked]>
Date: Mon, 4 Oct 2010 19:40:43 -0400

> I have reviewed it, and it looks ready to go.

+1

Regarding Hector's complaint, I think a separate usage report focused on 1st/3rd party signing practices may be appropriate -- but I don't think it makes sense to hold up the advancement of the DKIM base spec for that.

---

From: <[email-masked]>
Date: Mon, 4 Oct 2010 18:10:34 -0400

I would be curious also but would be happy with a

73% of the signatures were author signatures meaning the "d=" value in the signature matched the domain found in the From:header field

and let the reader draw their own conclusions

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 4 Oct 2010 19:30:07 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of J.D. Falk
> Sent: Monday, October 04, 2010 4:41 PM
> To: DKIM List
> Regarding Hector's complaint, I think a separate usage report focused
> on 1st/3rd party signing practices may be appropriate -- but I don't
> think it makes sense to hold up the advancement of the DKIM base spec
> for that.

Actually the data are still there.  It was just reworded, as per Jeff's suggestion:

   Identities:  73.8% of the signatures observed included a "d=" value
      matching the domain in the From: field.

...so this is a non-issue.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 4 Oct 2010 19:36:33 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
> Sent: Monday, October 04, 2010 3:11 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [ietf-dkim] Issue: implementation Report v02 - Removal of 1st vs 3rd party statistics
> 
> 73% of the signatures were author signatures meaning the "d=" value in
> the signature matched the domain found in the From:header field
> 

And that's what's still there.  First half of page 10.

The term "third-party" was removed because DKIM itself doesn't say anything about a binding between "d=" and anything else in the message.  That concept is first presented in ADSP.  Since the implementation report is only about DKIM itself, not ADSP, discussing author vs. third party is actually irrelevant.

---

From: Hector Santos <[email-masked]>
Date: Mon, 04 Oct 2010 22:54:26 -0400

> ...so this is a non-issue.

Sure it is.  Was it 78.9% for all implementations?  AOL and GOOGLE, 
the 2006 Interoperability Event participants,  your OpenDKIM network 
of implementators?  Or was it just for AOL 78.9%

In fact, one can take a swag that google's data is 100% third party 
because it was there reception analysis - unless one suggest google 
allows others to sign as GMAIL to send to their servers?

Stop skewing DATA - its unethical. We might not be smart but we are 
not STUPID.

-- 
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

---

From: Hector Santos <[email-masked]>
Date: Mon, 04 Oct 2010 23:15:05 -0400

Murray S. Kucherawy wrote:

> The term "third-party" was removed because DKIM itself 
> doesn't say anything about a binding between "d=" and anything 
> else in the message.  That concept is first presented in ADSP.  
> Since the implementation report is only about DKIM itself, not 
> ADSP, discussing author vs. third party is actually irrelevant.

-1

It is extremely relevant.

Its an implementation data report about observed operations and
consistent per chapter itemized goals:

   2. Collect data on the deployment, interoperability, and
   effectiveness of the base DKIM protocol, with consideration
   toward updating the working group's informational documents.

   3. Collect data on the deployment, interoperability, and
   effectiveness of the Author Domain Signing Practices protocol
   (RFC 5617), and determine if/when it's ready to advance on the
   standards track. Update it at Proposed Standard, advance it to
   Draft Standard, deprecate it, or determine another disposition,
   as appropriate.

   4. Taking into account the data collected in (2) and (3), update
   the overview and deployment/operations documents. These are
   considered living documents, and should be updated periodically,
   as we have more real-world experience.

The empirical data is on par with #2, #3 and thus #4.   It provides 
the field testing and engineering insights and information people need 
to progress with DKIM in a better way without blinders.

I don't get you guys, doing this to push a standard.  If you think 
this is kolsher - its not.

---

From: <[email-masked]>
Date: Mon, 4 Oct 2010 20:49:06 -0400

We are not holding up the dkim spec, we are wanting a datapoint to be kept in the draft-ietf-dkim-implementation-report

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 5 Oct 2010 04:48:00 -0700

> Sent: Monday, October 04, 2010 5:49 PM
> To: [email-masked]
> We are not holding up the dkim spec, we are wanting a datapoint to be
> kept in the draft-ietf-dkim-implementation-report

But:

a) It's technically out of scope, i.e. it's not a data point relevant to DKIM itself since RFC4871 doesn't say anything at all about a binding between "d=" and anything else in the message, so its removal would actually be justified; however

b) It was not actually removed from the draft, it was simply reworded to be more precise as suggested by one of the other participants[1].  It's right there on page 10, at least on the version I'm looking at on the IETF web site.

So are you saying you want the old wording back for some reason?  If not, I don't see what the complaint is.

It looks like I dropped the AOL version of the same statistic by mistake.  I'll add it back in after last call completes.

Gmail did not provide me with their version of that statistic.  Since it's of such interest, I'll ask for it, but I may not get it.

-MSK

[1] http://mipassoc.org/pipermail/ietf-dkim/2010q4/014572.html

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 05 Oct 2010 12:55:38 +0100

> It is extremely relevant.

The data is there. The numbers can be calculated from the sample size 
(~500k) and the proportions. They're nowhere near the numbers ("Originator 
signatures: 1.2 billion Third-party signatures:  184 million") that you 
quoted in another email, which also don't match the proportions that you 
quoted. Where did 1.2 billion come from?

"Third party" is somewhat of a leap from "the domains don't match". For 
example, if the from header is in the domain "example.com" and we see 
"d=foo.example.com", is that really a third party signature? Perhaps some 
clarity of whether subdomains were permitted to match would be useful.*

Oh, and are you thinking this is about implementation of ADSP? I think it's 
supposed to be about implementation of DKIM, so that DKIM can be 
progressed. Please don't let a misunderstanding hold that process up.

> this is kolsher - its not.

* It would be interesting to know what proportions of author addresses were 
subdomains of the d= value, and vice-versa. Even to know if the domains 
share common whois registrations (like foo.example.com and bar.example.com) 
would be nice, though harder to do. Having said all that, I have my own log 
files that I could analyze, so I'll shut up.

-- 
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

---

From: <[email-masked]>
Date: Tue, 5 Oct 2010 08:49:33 -0400

sorry, jumped a passing bandwagon, good to go then

---

From: Hector Santos <[email-masked]>
Date: Tue, 05 Oct 2010 08:50:43 -0400

Ian Eiloart wrote:

> (~500k) and the proportions. They're nowhere near the numbers 
> ("Originator signatures: 1.2 billion Third-party signatures:  184 
> million") that you quoted in another email, which also don't match the 
> proportions that you quoted. Where did 1.2 billion come from?

Sounds like revision v02 is already having its intended effect.

Ian, see the previous revision v01 section 4.2

http://tools.ietf.org/html/draft-ietf-dkim-implementation-report-01#section-4.2

In fact, what was left in rev 02 was Murry's 78.9% for the OpenDKIM 
observation of 1st vs 3rd.  What was removed was the AOL data point. I 
stated it as 86% here:

      http://mipassoc.org/pipermail/ietf-dkim/2010q3/014556.html

> "Third party" is somewhat of a leap from "the domains don't match".

Third party per RFC 5016 is well defined.

> For example, if the from header is in the domain "example.com" and we see 
> "d=foo.example.com", is that really a third party signature? Perhaps 
> some clarity of whether subdomains were permitted to match would be 
> useful.*

It doesn't matter.  The Observed data is what counts. Per RFC 5016 
definitions, this is what we got X for that, Y for this.

> Oh, and are you thinking this is about implementation of ADSP? 

As an engineer I look at data, look for patterns, see how they 
correlate to logical protocols and even justify experiments and 
problem solving.

To me, the data points show there is a strong 1st party stream of 
mail.  POLICY would be important here.  But that is not what the 
report is about.

For example, if the report showed the opposite, over 70% of the mail 
stream was 3rd party (5322.From != DKIM.d per RFC 5016), rest assured, 
we would be hearing how much POLICY or ADSP is insignificant and 
should be deprecated - and I would AGREE.

The reality is the overwhelming 1st party mail continues to justify a 
need for policy.  But that is my interpretation, not what the report 
is about.

> I think 
> it's supposed to be about implementation of DKIM, so that DKIM can be 

Its not an mis-understanding.  There is nothing holding back DKIM but 
this constant interference with the reality.  Embrace and see how 
things change. What the factoid removal does is goes against chartered 
itemize goals of #2, #3 and #4.

> * It would be interesting to know what proportions of author addresses 
> were subdomains of the d= value, and vice-versa. Even to know if the 
> domains share common whois registrations (like foo.example.com and 
> bar.example.com) would be nice, though harder to do. Having said all 
> that, I have my own log files that I could analyze, so I'll shut up.

Your, all data would be welcomed too.

Soon I will have accumulated data as well.  Currently working out how 
to present them in our web-view of the statistics.  IOW, adding 
DKIM/POLICY related columns to these statistics:

      http://www.winserver.com/public/spamstats.wct

-- 
HLS

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Tue, 05 Oct 2010 15:26:51 +0200

> comments/issues. Please also post here if you've reviewed it and think

I've reviewed it and came across the following:

Chapter 2: as for DKIM specific definitions used in this document, they 
cannot be found in [EMAIL-ARCH]. Please add [DKIM] here as source of 
definitions.

Par. 3.2 - 3.4: these paragraphs are a bit 'vague', due to terms like 
'Some implementations [...]', 'Some test cases [...]', 'at least two 
implementations...'. Presumably this has to do with the fact that the 
interoperability event took place long time ago? If so, I'd suggest to 
include a sentence to describe the difficulty to gather all relevant 
information, three years after the event took place. To make these 
paragraphs a bit less 'vague' I would like to suggest to include the 
original set(s) of test messages and the original interoperability test 
plan(s), if still around somewhere.

Par. 4.1.2: [...] Where "list mail" is defined as [...]
Should this not be: [...] Where "Non-List Mail" is defined as [...]?

Murray, thanks for the work!

/rolf

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 5 Oct 2010 06:52:20 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Ian Eiloart
> Sent: Tuesday, October 05, 2010 4:56 AM
> To: Hector Santos; [email-masked]

Yes, that's precisely right.  The purpose of the implementation report is to discuss DKIM's interoperability only, to satisfy certain IESG requirements.  Nothing more.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 5 Oct 2010 07:07:06 -0700

Hi Rolf,

> From: Rolf E. Sonneveld [mailto:[email-masked]]
> Sent: Tuesday, October 05, 2010 6:27 AM
> To: Barry Leiba
> Cc: DKIM Mailing List; Murray S. Kucherawy

The top of chapter 2 contains "[DKIM]" which is the formal reference to that specification.

"[EMAIL-ARCH]" is a general tutorial/overview about email architecture, included to help any newcomers to understand the context of the whole work.

> interoperability event took place long time ago?

The language of that section is just a general description of the participants (the software, specifically) in the interop event and what they brought to the party.  The thrust is actually to show that they were all diverse, i.e. that a number of different parties wrote code per the RFC and then tried to send mail to each other to see what happens.  It would be uninteresting to have 20 parties come together all running different installations of the same package.  Showing diversity in an interoperability report is quite important.

Changing "Some test cases" to "Test cases" seems fine to me.

"At least two implementations" is a specific IETF requirement for an interop report.  I actually am pretty sure all implementations there implemented even all of the optional stuff, except "z=", but I don't have any specific record saying that so I'm making only the weaker of the two statements.

> If so, I'd suggest to

There wasn't a formal plan.  It was basically "send all your tests at all the participants, then work with each of them to figure out what failed and why".  Each participant brought its own corpus of test messages.

Unfortunately very little of the actual test data and results documentation still exist, mainly because of a promise among the participants to keep the results about specific implementations private.  The public result, appropriately, was the group's evaluation of the quality of the RFC and the published list of errata.

> Should this not be: [...] Where "Non-List Mail" is defined as [...]?

Sorry, yes.  Fixed for the next version.

> Murray, thanks for the work!

Thanks for the feedback!

-MSK

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Tue, 05 Oct 2010 16:35:50 +0200

> The top of chapter 2 contains "[DKIM]" which is the formal reference to that specification.

Are we looking at the same report? I used 
http://tools.ietf.org/html/draft-ietf-dkim-implementation-report which 
brings me to version 02 of the document. I see [DKIM] mentioned here and 
there, throughout the document, but not in chapter 2? Or do you mean the 
page header, which mentions RFC4871 and of course presents the same 
referral as [DKIM] gives?

> "[EMAIL-ARCH]" is a general tutorial/overview about email architecture, included to help any newcomers to understand the context of the whole work.

I know [EMAIL-ARCH], it's definitely a good choice to refer in your 
document to this architecture RFC.

> "At least two implementations" is a specific IETF requirement for an interop report.  I actually am pretty sure all implementations there implemented even all of the optional stuff, except "z=", bu…

OK.
> Unfortunately very little of the actual test data and results documentation still exist, mainly because of a promise among the participants to keep the results about specific implementations private.

OK, then maybe that's something that needs to be mentioned, to explain 
why that part of the report is less exact than for example the data 
points about OpenDKIM and AOL.

>    The public result, appropriately, was the group's evaluation of the quality of the RFC and the published list of errata.

Apart from these minor remarks, in my view, it's ready to go.

/rolf

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 5 Oct 2010 08:05:14 -0700

> Sent: Tuesday, October 05, 2010 7:36 AM
> To: Murray S. Kucherawy
> Cc: DKIM Mailing List

Sorry, you're right; the first [DKIM] reference is in Section 1.  For completeness, I'll add it to Section 2 as well.

> points about OpenDKIM and AOL.

Yep, that seems reasonable.  I'll do that as well.

-MSK

---

From: Jeff Macdonald <[email-masked]>
Date: Tue, 5 Oct 2010 15:20:57 -0400

> comments/issues. Please also post here if you've reviewed it and think

I find sections 4.2 and 4.3 hard to compare. It could be just me. That
is just my quick observation.

-- 
Jeff Macdonald
Ayer, MA

---

From: SM <[email-masked]>
Date: Tue, 05 Oct 2010 14:22:46 -0700

>interoperability report, draft-ietf-dkim-implementation-report-02:

I read the implementation report for RFC 4871 
(draft-ietf-dkim-implementation-report-02).  It addresses the 
comments I raised previously.

Regards,
-sm