Subject: Re: [ietf-dkim] list vs contributor signatures, was Wrong Discussion

From: Scott Kitterman <[email-masked]>
Date: Thu, 27 May 2010 22:39:11 -0400

"Brett McDowell" <[email-masked]> wrote:
...
>As a newbie to this list, I have to say I agree.  This has been a far less collegial debate than what I'm used to.  That said, I may be guilty of reciprocating, and if anyone feels they have been on the receiving end of such, I apologize.
...

I think your only offense is presenting a perspective based on operational experience that varies from the preconceived notions of a substantial fraction of the participants of this working group.

There has been considerable resistance to doing any standardized policy work relative to DKIM.  It's unfortunate that this resulted in policy having to be bolted on to DKIM after it was designed because we were prohibited from doing policy work as a part of DKIM development. 

As far as I can see, the only problem with ADSP and your discussion of it is that ADSP is guilty of doing exactly what it was designed to do with exactly the limitations we said it would have when we designed it.  The same people that didn't like it then, don't like it now.

Scott K
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Steve Atkins <[email-masked]>
Date: Thu, 27 May 2010 21:01:54 -0700

> As far as I can see, the only problem with ADSP and your discussion of it is that ADSP is guilty of doing exactly what it was designed to do with exactly the limitations we said it would have when…

That's because it was broken then, and it's still broken now. :)

This explanation of why I think it's broken, and what might be done to fix it, is fairly terse, though, so please don't grab onto a single phrase and shake that phrase to death like a terrier with a rat. Really, it's worth reading all the way through.

There are, I think, two problems that are intrinsic to the use of ADSP in the context of mitigating phishing email.

One underlying problem is that ADSP is based on the inverse of an intentionally unreliable positive assertion (DKIM). That maps the non-problem of a mail with a broken DKIM signature being treated as unsigned to the serious problem (72% false positive rate) of mail with a broken signature being discarded.

The only reason that DKIM is intentionally unreliable is that it's primary design constraint was that it would be something that could be added by a smarthost, without the sending MUA needing to be aware of it, and received by an MX and delivered to a receiving MUA without either needing to be aware of it.

If ADSP were based on a reliable signing algorithm (such as S/MIME) then my objections to it based on the fragility of DKIM would go away (fundamentally, that it doesn't work reliably and causes legitimate email to be lost). Given it's primary value is when applied to B2C bulk mail, the constraints that apply to DKIM signing wouldn't apply and pretty much every MUA renders S/MIME.

While I believe that would be a much smarter direction to go in, we've chosen not to. As ADSP is based on DKIM, and DKIM is always going to have a significant level of broken signatures, ADSP is always going to have a significant false positive rate - making it unusable for mail that it's important be delivered.

All we can do is mitigate in an attempt to reduce that false positive rate. I'm sure we can come up with something that'll bring it down to single digit percentages of legitimate mail lost, or even to less than 1%, at least in some cases with very limited use of email (bulk mailer smarthost directly to major ISP MX, for instance). And there are situations where the mail that's being sent is sufficiently worthless or redundant that losing one mail in every few hundred might be acceptable.

The other underlying problem, as applied to phishing email, is that ADSP will currently, if implemented perfectly by all senders and all receivers, reduce the volume of phishing mail delivered to the inbox by less than 50%, and that reduction is likely to decrease rapidly. Given the volume of phishing mail sent, and the volume of it that's already blocked much more reliably by existing filters, that means it is likely to have minimal effect on the number of phishing emails delivered to the recipients inbox.

So the suggested use model is one where the security of the mail is so critical that it's worth going to this much effort in order to discard <50% of phish messages, yet it isn't critical enough that losing one in every few hundred legitimate messages isn't a problem. This doesn't seem like a use case most informed security or bulk email experts would consider interesting. If that's the low bar we're aiming for, then we're fine, but it would be dishonest not to document the limitations in a way that's clear to even a casual reader.

OTOH, if that isn't acceptable then we need to change things. We cannot fix the second issue, ineffectiveness against phishing mail, both intrinsically and because the chairs have ruled that it's not a valid topic for discussion. So all we can do is improve the first issue - high levels of false positives leading to legitimate email being rejected. There are a few different places we could look at doing that, and some of the questions we need to ask are:

  1. Do we want to reduce the DKIM broken signature rate or do we want to make ADSP less vulnerable to it. Or both, I guess.

  2. If we want to reduce the DKIM broken signature rate, do we need to rework DKIM at all, or do we need to make operational recommendations to the generator and signer of the mail, or do we need to provide operational advice to forwarders and mailing list managers. For any of those we need to balance the improvement against the reduction in deployment and reduction in correct deployment the increased complexity will cause. The history of SPF-all and SRS is likely relevant there.

  3. If we want to make ADSP less vulnerable to it, this basically means reworking ADSP such that it says that receivers should accept unsigned mail in some cases - the various suggestions about accepting unsigned mail from "trusted" mailing lists or forwarders. This is bound to open up a bunch of theoretical security issues, and probably some real ones.

  3a.  If we go in that direction we're looking at making some fairly tricky security related decisions. We'd need a proper analysis of the security risks and operational benefits, which would require us to be more honest than we have been in the past about what the end goals are, and how some of the more obvious attack trees would affect those goals.

Any of those changes - i.e. doing anything other than accepting the seriously broken behaviour and just documenting it - would require buy-in from the chairs and other participants, and likely a charter clarification.

Cheers,
  Steve

---

From: "Scott Kitterman" <[email-masked]>
Date: Fri, 28 May 2010 01:02:46 -0400

...
>   1. Do we want to reduce the DKIM broken signature rate or do we want to
> make ADSP less vulnerable to it. Or both, I guess.
>
>   2. If we want to reduce the DKIM broken signature rate, do we need to
> rework DKIM at all, or do we need to make operational recommendations to
> the generator and signer of the mail, or do we need to provide
> operational advice to forwarders and mailing list managers. For any of
> those we need to balance the improvement against the reduction in
> deployment and reduction in correct deployment the increased complexity
> will cause. The history of SPF-all and SRS is likely relevant there.
...

I'm curious what level of reliability you think DKIM signatures have
currently?  And are you measuring at a border MTA or somewhere later in
the delivery chain?

Scott K

---

From: Douglas Otis <[email-masked]>
Date: Thu, 27 May 2010 23:15:50 -0700

> While I believe that would be a much smarter direction to go in, we've chosen not to. As ADSP is based on DKIM, and DKIM is always going to have a significant level of broken signatures, ADSP is al…
One objection to S/MIME was its poor presentation, where a failure as 
appearing to be phish is less tangible.  S/MIME is also not compatible 
with provider practices of modifying presentations.  In this case, the 
provider is expected to track its own brutal message handling,  the 
motivation for Authentication-Results headers.  Providers often consider 
themselves responsible for a user's impression of a message.  Adding 
virtual icons, colored borders and the like can be effective strategies 
at combating phish, where presentation plays a critical role not 
compatible with S/MIME.
> The other underlying problem, as applied to phishing email, is that ADSP will currently, if implemented perfectly by all senders and all receivers, reduce the volume of phishing mail delivered to t…
This conclusion overlooks the role message presentation plays.  A role 
that represents a highly effective deterrent.
> So the suggested use model is one where the security of the mail is so critical that it's worth going to this much effort in order to discard<50% of phish messages, yet it isn't critical enough that losing one in every few hundred legitimate messages isn't a problem. This doesn't seem like a use case most informed security or bulk email experts would consider interesting. If that's the low bar we're aiming for, then we're fine, but it would be dishonest not to document the limitations in a way that's clear to even a casual reader.
>    
Where signatures are damaged beyond the provider's control is where 
attention is needed.  This often represents third-party services.

There are two possible approaches for handling these services.

A) Have senders publish, in a scalable lightweight fashion, which 
third-party services are used.

B) Depend upon recipient funded services that assess these services.

Organizations most affected by fraud should be able to easily comply with A.

In the case of public domains where fraud is less of an issue (ignoring 
successful ploys that leverage social networking) then B is needed.

The third-party authorization scheme suggested can handle either 
approach.  In the case of B,  DNAME could vector to a consortium 
policing these services by publishing their own list of acceptable 
domains.  These services will likely specialize on handling different 
regions.

The suggested third-party authorization scheme attempts to include 
several qualifiers to further limit acceptance where possible, such as 
requiring LIST-ID, or authenticated Mail-From.   It seems that this 
should be kept flexible, until it become understood what is needed.

-Doug

---

From: Steve Atkins <[email-masked]>
Date: Thu, 27 May 2010 23:35:17 -0700

On May 27, 2010, at 10:02 PM, Scott Kitterman wrote:

> currently?

I just measured 72% broken in one case, and it's one that's
not that atypical.

It'll depend on the sort of mail you're talking about. For mail that's
drafted carefully, and avoids doing anything (mime encoding,
charsets) that might trigger MIME level rewrites during delivery, that's
sent by a competent bulk mailer from well configured smarthosts
directly to the MXes of a major consumer ISP I'd expect them to
be extremely robust as measured at the MX. 

If your favorite use case of ADSP is bulk business-to-consumer mail
sent from domains dedicated to sending solely that sort of mail,
with no usage by employees, role accounts, customer support
and so on then that's really good news. You'd need to start your
email business unit pretty much from scratch to be able to do
that, though. Also, I work with quite a lot of bulk mailers, and
consistent operational competence of the level required is much
rarer than you might hope.

For mail that's being sent through any less well understood path
there's risk of the mail being rewritten or lightly corrupted to an
extent that it'll break the signature simply as part of SMTP forwarding.
A common cause of this is mail being forwarded from a
mail filtering appliance to an internal Exchange server. I've seen
a bunch of obscure MIME rewriting and corruption issues in that
case (our app used to express the same strong opinions at an
ESMTP level that Exchange does, so I spent a long time tracking
down a bunch of oddities there). I've no idea how widespread this
sort of issue is, but I've seen problems that I think might cause
this sort of issue at maybe 10% of my customers.

For mail that's being sent through actual forwarders it's somewhat
worse, as they're even more likely to annotate headers or body,
modify headers or redraft MIME structure. Even "invisible" forwarding
such as .forward files can break things when the mail is annotated
by spam filters, then forwarded.

All of the above can be mitigated somewhat by being very
cautious both in generating mail content and by choosing
which header not to sign, I'm sure.

There there are mailing list managers - all bets are off, they tend
to rewrite most everything and will break signatures more often
than not. The only good point there is that the sender probably
knows they're sending to a mailing list.

>  And are you measuring at a border MTA or somewhere later in
> the delivery chain?

I've been considering the state at the border MTA. I suspect it'll be a
little worse later in the delivery chain, typically, but that's going to
vary significantly based on the delivery path. I think there's going
to be a large systemic component and a much smaller random
component.

And with that, I declare it the end of the week. Have a good
slow Friday and Memorial day long weekend, everyone, if
you're in a bit of the world that observes that.

Cheers,
  Steve