Subject: Re: [ietf-dkim] Proposed new charter

From: Franck Martin <[email-masked]>
Date: Wed, 24 Feb 2010 01:57:47 -0800 (PST)

Shouldn't we move forward Murray's draft "quickly" that allows to report back broken DKIM signature to the validating domain?

This would allow to collect information on why signature gets broken making the DKIM draft stronger.

----- Original Message -----
From: "Barry Leiba" <[email-masked]>
To: "IETF DKIM WG" <[email-masked]>
Sent: Tuesday, 23 February, 2010 12:40:15 PM
Subject: Re: [ietf-dkim] Proposed new charter

> Here is a charter proposal for us to bash.  It covers the items we
> voted on.  As a working group, we'll need to agree on the wording, and
> come up with a few reasonable milestones with dates.

It's only been a few days, but so far we're getting good agreement on
the charter text, but no milestones and dates.  We'd really like the
working group -- not the chairs -- to decide on the deliverables and
the dates.  If folks are willing to agree to text, but not to be clear
about what we'll produce when, we think that's a sign of insufficient
energy to move forward.

If y'all want to do this, we'll need some proposals for what gets
accomplished and when, so we can discuss that too.

Barry, as chair

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Suresh Ramasubramanian <[email-masked]>
Date: Wed, 24 Feb 2010 15:47:41 +0530

I support this. The rest of Barry's charter proposal is OK by me.

thanks
suresh

-- 
Suresh Ramasubramanian ([email-masked])
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Michael Thomas <[email-masked]>
Date: Wed, 24 Feb 2010 05:51:19 -0800

I'm sort of dubious about this. Unless you're using z=, your chances of
figuring out why something broke are slim to none. With z=, your chances
of figuring it out are merely slim.

Mike, with far too much experience at that

On 02/24/2010 02:17 AM, Suresh Ramasubramanian wrote:
> On Wed, Feb 24, 2010 at 3:27 PM, Franck Martin<[email-masked]>  wrote:

---

From: Mark Delany <[email-masked]>
Date: Wed, 24 Feb 2010 08:54:57 -0800

> Mike, with far too much experience at that

It got a luke-warm response a few years back, but now that a lot more  
people are having to deal with "why did the verify failed", is it  
worth re-vivifying the DKIM-Trace stuff, or whatever it was called  
back then? We found it very useful for our early days of interop  
testing.

The idea is pretty simple: The signer adds a header that characterizes  
the content before and after canonicalization. The verifier performs  
the same characterization and compares the differences. The  
characterizations we used at the time were simple character counts  
represented in a relatively compressed form (27 a's, 60 b's, 40LFs,  
50spaces, etc)

The form we used is kinda ugly as http://www.ietf.org/mail-archive/web/ietf/current/msg39488.html 
  shows, but it was very illuminating as things like mis-match of  
white-space counts, or case-conversions, etc, identified what changed  
and where it changed (canonicalization vs transit).

Mark.

---

From: Michael Thomas <[email-masked]>
Date: Wed, 24 Feb 2010 09:47:01 -0800

> represented in a relatively compressed form (27 a's, 60 b's, 40LFs,

The thing that I'm skeptical about is that an automaton can be programmed
to do this sort of analysis with any sort of accuracy. We're talking about
a potential flood of reports coming in, I assume, so I doubt we're all going
to be putting out job reqs for "DKIM Signature Breakage Analysis Engineer".
There were far too many breakages even with tools and hunches that were very
difficult to figure out, and even then there were lots of mysteries.

And of course, there's an open question about what you do with this sort of
forensic data... it can be gamed, after all. So if there's any advantage for
bad guys to game it, it probably will be.

But I guess this all rather begs the question about what people intend to
do with those breakage stats and/or analysis.

Mike

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 24 Feb 2010 10:06:26 -0800

> -----Original Message-----
> From: [email-masked] [mailto:ietf-dkim-
> [email-masked]] On Behalf Of Michael Thomas
> Sent: Wednesday, February 24, 2010 9:47 AM
> To: Mark Delany
> Cc: IETF DKIM WG
> Subject: Re: [ietf-dkim] Broken signature analysis
> 

I've been using several approaches to solve these problems.

Basically I have two debugging modes.  In the first mode I enable "z=" tags on signing (or ask the sender to do so) and then wait for a failure; if the "bh" matches then "z=" will contain enough information to identify the problem, otherwise you have to move to the second mode.  OpenDKIM can be compiled to make some attempt to figure out what changed in this case by using the library equivalent of "agrep" (approximate grep) to try to find similar header fields in the "z=" block and what the receiver actually got.

In the second mode, I enable debugging flags on both ends of the connection that save the canonicalized header and body, and then collect and "diff" to see what changed (or spot canonicalization errors).  This can be automated, but the data collection part is tough because it requires co-operation of both ends.  The "r=" (reporting address) tag that the WG chose not to pursue has been extremely useful here.

And in terms of general deployment statistics and observations about overall success, OpenDKIM has a statistics mode that we will be encouraging people to enable and report back to us periodically, and we will relay data from willing participants to the working group once the WG gets to that point.

---

From: Michael Thomas <[email-masked]>
Date: Wed, 24 Feb 2010 10:13:23 -0800

I understand that you, me and Mark have a vested interest in DKIM qua DKIM,
but I think we're outliers. What's unclear to me is what mailops and/or
ARF-consuming boxen intend to do with this data.

Mike, and you passed on BARF as the name of the WG? Shame!

---

From: Steve Atkins <[email-masked]>
Date: Wed, 24 Feb 2010 09:41:06 -0800

> and where it changed (canonicalization vs transit).

Neat. A nice way of fingerprinting.

I'm guessing that the verifier typically won't particularly care why it failed,
certainly not to the extent they'll want to expend much effort to characterize
it mechanically.

The signer will already have (or be able to regenerate) a byte accurate
copy of what was sent, if they care to diagnose things. If the verifier
simply returns a copy of what they received back to the signer, is
that adequate for any forensics needed, or does more metadata need
to be sent along with the message?

Cheers,
  Steve

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 24 Feb 2010 10:23:05 -0800

Sounds like there are 3 things to discuss:

1. How to use existing stuff, per Murray's description.

2. What additional set of information should be obtained

3. How a signer can signal the need for #2.

The reason for separating 2 and 3 is that having 2 can allow uncoordinated 
logging, with post hoc reconciliation, if both sides happen to be doing logging.

Whereas 3 is the new header fied and is a protocol enhancement -- albeit a small 
one -- that requires changes to the protocol engines on both sides.  A good 
idea, but also nice to keep out of the critical path, if possible.

Just being able to get everyone's loggers to record the right set of information 
is likely to allow useful diagnostics more quickly.  Or perhaps I'm missing how 
to do this more easily?

 From Mark's note, I can't quite figure out enough of the diagnostic data 
specification to do the ABNF for it...

All of this should be written into an I-D draft.  Probably does not need to go 
to publication, but it would be good to make sure everyone is on the same page 
(so to speak) about the details.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: SM <[email-masked]>
Date: Wed, 24 Feb 2010 11:04:39 -0800

>difficult to figure out, and even then there were lots of mysteries.

I guess that it is all about determining whether the number of 
reports points to a significant problem.  Murray mentioned the z= tag 
( http://mipassoc.org/pipermail/ietf-dkim/2010q1/012988.html ).  It's 
useful to catch obvious causes of DKIM verification failures.  I 
agree that there is too much breakage for automated analysis.

>do with those breakage stats and/or analysis.

Personally, it is useful to fix bugs and improve the implementation.

>back then? We found it very useful for our early days of interop

I recall seeing the header for DomainKeys.

>represented in a relatively compressed form (27 a's, 60 b's, 40LFs,

Do you have any documentation for that?  As a non-WG comment, I doubt 
that the before and after capture would catch the useful cases due to 
rewrites and other changes done in some environments.

Regards,
-sm

---

From: Mark Delany <[email-masked]>
Date: Wed, 24 Feb 2010 11:00:16 -0800

> bad guys to game it, it probably will be.

I wasn't thinking of wide activation necessarily, it might be  
something that, eg, just MAAWG members and implementors might  
selective enable over an interop testing period.

Mark.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 24 Feb 2010 11:43:26 -0800

> [email-masked]] On Behalf Of Mark Delany
> Sent: Wednesday, February 24, 2010 11:00 AM
> To: Michael Thomas
> Cc: IETF DKIM WG

This isn't the first time I've heard DKIM interop mentioned recently, so now I'm curious: Are we interested in repeating the work done in Dallas?  If so, what are we hoping to learn from a second round?

Apart from ADSP, I don't think DKIM has changed enough to make that effort interesting right now.

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 24 Feb 2010 11:53:11 -0800

> Apart from ADSP, I don't think DKIM has changed enough to make that effort interesting right now.

Thanks for raising this.  I meant to follow up on it.  There were a couple of 
comments made during sessions at MAAWG and my sense was that the audience(s) did 
think there would be utility in more testing.

The previous interop event targeted basic, direct signing and validation.  My 
sense is that the interest now is extending utility, such as:

    1)  exploring interesting scenarios that seem to be causing failures.

    2)  perhaps exploring use of d= subdomains for differentially labeling 
different sub-streams

    3)  exploring the use of multiple signatures

If the interest is in these types of issues, we might find it easier and more 
productive to have a distributed event, over a couple of weeks.  Still needs 
planning, scenarios and interaction, but not necessarily a single room with 
people in it.

Thoughts?

d/
-- 

   Dave Crocker

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 24 Feb 2010 11:55:37 -0800

> From: Dave CROCKER [mailto:[email-masked]]
> Sent: Wednesday, February 24, 2010 11:53 AM
> To: Murray S. Kucherawy
> Cc: Mark Delany; IETF DKIM WG

Now you're talking.  And throw re-signing by lists in there as well.  Oh, and signing A-R header fields during relaying.

---

From: Franck Martin <[email-masked]>
Date: Wed, 24 Feb 2010 12:28:44 -0800 (PST)

I spoke recently to someone which I think will join this group soon.

But basically his idea of being alerted of a broken signature was also to catch people who are trying to fake the DKIM signature, and see the extent of it.

Also personally, I think the sender is more motivated to fix a broken DKIM signature than the receiver.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Michael Thomas <[email-masked]>
Date: Wed, 24 Feb 2010 12:56:51 -0800

> But basically his idea of being alerted of a broken signature was also to catch people who are trying to fake the DKIM signature, and see the extent of it.

Faking DKIM signatures shouldn't help *anybody*. If there's any incentive to
make a fake DKIM signature by bad guys, somebody's software is horribly broken.

> Also personally, I think the sender is more motivated to fix a broken DKIM signature than the receiver.

Sure, but I think the question here is whether a huge hose of ARF reports from
potentially unknown and not very trustworthy sources is the right way to go about
sniffing out forwarding oddities, etc.

I guess a lot of my uncomfort here is that abuse reporting could end up being
its own abuse vector as well as something that take on a life of its own. The
potential volume of traffic could be very large versus the benefit of... what?
It seems to me that the problem space for this should be extremely constrained
to solve a minimal set of very explicit existing problems, and not feature
creep beyond that. WRT DKIM, I'm not sure what that problem set is.

Mike

---

From: SM <[email-masked]>
Date: Wed, 24 Feb 2010 13:09:20 -0800

Hi Franck,
At 12:28 24-02-10, Franck Martin wrote:
>But basically his idea of being alerted of a broken signature was 
>also to catch people who are trying to fake the DKIM signature, and 
>see the extent of it.

There are a lot of broken signatures for messages going through 
mailing lists.

>broken DKIM signature than the receiver.

See http://lists.opendkim.org/archive/opendkim/users/2010/02/0116.html

Regards,
-sm

---

From: <[email-masked]>
Date: Wed, 24 Feb 2010 12:57:57 -0500

Forensics and reporting is about all one can do with a broken signature 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Michael Thomas
Sent: Wednesday, February 24, 2010 12:47 PM
To: Mark Delany
Cc: IETF DKIM WG
Subject: Re: [ietf-dkim] Broken signature analysis

Mike

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 24 Feb 2010 18:45:57 -0500

>creep beyond that. WRT DKIM, I'm not sure what that problem set is.

Well said.   Much better than the reply I'd started drafting.

Scott K

---

From: Suresh Ramasubramanian <[email-masked]>
Date: Thu, 25 Feb 2010 07:13:02 +0530

I was thinking that specific email that was received might be
retrieved, stripped of its dkim headers and resent under controlled
conditions to see what broke.

I've seen two emails recently - on a moderately busy server that hosts
some high traffic discussion lists, as well as personal mail for half
a dozen people who are "heavy" users of email .. that failed dkim
validation and signature verification was disabled .. on the latest
exim and its libdkim, on debian.  Both seem to be random spam - one
forwarded through a friend's pobox account and sent originally from a
webmail .. and the second being some kind of send to a friend feature
on a video site, that's being abused to send nigerian spam to a
mailing list address.

As spam isn't particularly noted for good construction ..

/var/log/exim4/mainlog.6.gz:2010-02-18 19:23:55 1NiJT8-0001OV-VA DKIM:
Error while running this message through validation, disabling
signature verification.
/var/log/exim4/mainlog.6.gz:2010-02-18 19:23:55 1NiJT8-0001OV-VA <=
SRS0=KHZi=KD=sify.com=[email-masked]
H=sienna.pobox.com [64.74.157.51]:56699 I=[204.74.68.40]:25 P=esmtp
S=31396 id=[email-masked] T="Great
hotel deals from India and across the globe" from
<SRS0=KHZi=KD=sify.com=[email-masked]> for
[email-masked]
/var/log/exim4/mainlog.6.gz:2010-02-18 19:23:55 cwd=/var/spool/exim4 3
args: /usr/sbin/exim4 -Mc 1NiJT8-0001OV-VA
/var/log/exim4/mainlog.6.gz:2010-02-18 19:23:55 1NiJT8-0001OV-VA =>
xyz <[email-masked]>
F=<SRS0=KHZi=KD=sify.com=[email-masked]> R=localuser
T=local_delivery S=31556
/var/log/exim4/mainlog.6.gz:2010-02-18 19:23:55 1NiJT8-0001OV-VA Completed

/var/log/exim4/mainlog.9.gz:2010-02-16 03:52:27 1NhLyd-0000Wr-7z DKIM:
Error while running this message through validation, disabling
signature verification.
/var/log/exim4/mainlog.9.gz:2010-02-16 03:52:27 1NhLyd-0000Wr-7z <=
[email-masked] H=(livevideo.com) [207.7.146.81]:4201
I=[204.74.68.40]:25 P=smtp S=75499 id=634018895491215047@LVML11
T="Join kufo.george on LiveVideo!" from <[email-masked]> for
[email-masked]
/var/log/exim4/mainlog.9.gz:2010-02-16 03:52:27 cwd=/var/spool/exim4 3
args: /usr/sbin/exim4 -Mc 1NhLyd-0000Wr-7z
/var/log/exim4/mainlog.9.gz:2010-02-16 03:52:27 1NhLyd-0000Wr-7z =>
listname <[email-masked]> F=<[email-masked]>
R=mailman_router T=mailman_transport S=75553
/var/log/exim4/mainlog.9.gz:2010-02-16 03:52:27 1NhLyd-0000Wr-7z Completed

On Wed, Feb 24, 2010 at 7:21 PM, Michael Thomas <[email-masked]> wrote:
>>> Shouldn't we move forward Murray's draft "quickly" that allows to report
>>> back broken DKIM signature to the validating domain?
>>>

---

From: Jim Fenton <[email-masked]>
Date: Wed, 24 Feb 2010 22:35:56 -0800

There are really two parts to this:  (1) How to do the analysis of what 
failed, and (2) How to report the failure to the signing domain.

(1) Assuming it's a signature failure and not a body hash failure, 
analyzing z= gives the best chance of finding the cause.  There are 
going to be a lot of failures that one can't diagnose that way, of 
course, but z= should help with enough of them to be worth trying.  Some 
tools might help, like something that does an automated comparison of 
the z= value in a signature with the corresponding header fields and 
reports any discrepancy.

The best thing about this is that z= is already in the spec.

(2) I wouldn't give up on the reporting mechanism too quickly due to the 
potential for abuse.  Perhaps the use of the reporting address could be 
constrained in such a way that abuse gets dropped more easily, since 
this isn't a general-purpose address that has to deal with all sorts of 
use cases.  So perhaps the messages get dropped if they're not 
authenticated (maybe use SPF here to drop messages as they're received, 
and tell admins not to do forwarding on the reporting address).  Maybe 
additionally rate limit by domain.  I'm just brainstorming; other ideas?

-Jim

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 25 Feb 2010 11:18:13 -0000

> Forensics and reporting is about all one can do with a broken signature

It's up to recipients. I they really care about whether some mesage was  
genuine, they might find it worthwhile to try to deduce from the  
diagnostic headers what had got lost/added, and then try to reverify a  
corrected version.

A lot of work, but useful in some cases, and always useful to identify  
sites that had caused the problem and for issuing LARTs accordingly.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 27 Feb 2010 13:22:05 +0100

On 24/Feb/10 20:55, Murray S. Kucherawy wrote:
> [...] Oh, and signing A-R header fields during relaying.

Doesn't that conflict with the practice of removing A-Rs?
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: John Levine <[email-masked]>
Date: 27 Feb 2010 14:54:27 -0000

>Doesn't that conflict with the practice of removing A-Rs?

You only need to remove A-Rs that claim to be from you.

R's,
John