Subject: [ietf-dkim] Clarifying DKIM (etc.) expectations for mailing lists in the face of digests

From: Dave CROCKER <[email-masked]>
Date: Wed, 04 Aug 2010 10:07:36 -0700

Folks,

As long as there is some effort to review what goals are being pursued, with 
respect to mailing lists, I'd like to raise an additional question:

For typical, modern mailing lists, a subscriber can choose between delivery of 
each message, as posted individually, versus delivery of batches of messages in 
a digest.

In the latter case, no one would reasonably expect a DKIM signature from a first 
(author/originator) sequence to survive.  Yet there seems to be some strong 
expectation that it will or should survive if the recipient happens to choose 
delivery on a per-message basis.

What is the security model that makes this expectation of preservation important 
and reasonable, given that it is so easily and whimsically violated by a common 
recipient-selectable setting?

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 4 Aug 2010 10:16:35 -0700

> -----Original Message-----
> From: [email-masked] [mailto:ietf-dkim-
> [email-masked]] On Behalf Of Dave CROCKER
> Sent: Wednesday, August 04, 2010 10:08 AM
> To: DKIM IETF WG
> Subject: [ietf-dkim] Clarifying DKIM (etc.) expectations for mailing
> lists in the face of digests
> 

In theory they could survive, if the construction of the multipart/digest from the MLM preserved all of the signed data from the original submission.  Then each contained message/rfc822 part could be DKIM-evaluated on its own.  And the MLM could sign the overall message.

I've never heard of that stated as a desired thing, though.

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 04 Aug 2010 10:20:38 -0700

On 8/4/2010 10:16 AM, Murray S. Kucherawy wrote:
>> In the latter case, no one would reasonably expect a DKIM signature from a
>> first (author/originator) sequence to survive.
...
>
> In theory they could survive, if the construction of the multipart/digest

Yup.  But that's why I chose my wording pretty carefully, making a statement 
about likelihood rather than theory...

I'll admit that it was phrased to represent my own judgement of 'reasonable', 
but I'd claim that anyone disagreeing would have the first task of explaining 
why it would be a "reasonable" expectation, before anyone should consider 
solving it.

d/

-- 

   Dave Crocker

---

From: Michael Thomas <[email-masked]>
Date: Wed, 04 Aug 2010 10:22:52 -0700

> recipient-selectable setting?

The original 822 From: address is not preserved in a digest.

Mike

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 4 Aug 2010 10:27:51 -0700

> From: Dave CROCKER [mailto:[email-masked]]
> Sent: Wednesday, August 04, 2010 10:21 AM
> To: Murray S. Kucherawy
> Cc: DKIM IETF WG
> Subject: Re: [ietf-dkim] Clarifying DKIM (etc.) expectations for
> mailing lists in the face of digests
> 

Hence my ending comment that nobody's actually asked for that yet.  I'm just observing that the technology exists to enable it.

Also: Didn't we say on the list, if not in the RFC, that signatures inside parts shouldn't be processed?

---

From: Martijn Grooten <[email-masked]>
Date: Wed, 4 Aug 2010 18:30:09 +0100

> recipient-selectable setting?

There's a scenario where a spammer/phisher sets up a mailing list, adds a bunch of addresses to the list and then sends a message with a paypal.com From: address through the list. The DKIM signature will obviously be invalid, but a MTA/spam filter won't be able to decide whether this is because the message didn't really come from Paypal, or because it did but the mailing list broke it.

Martijn.

Virus Bulletin Ltd, The Pentagon, Abingdon, OX14 3YP, England.
Company Reg No: 2388295. VAT Reg No: GB 532 5598 33.

---

From: Martijn Grooten <[email-masked]>
Date: Wed, 4 Aug 2010 19:15:24 +0100

I've seen spam with List-ID: headers, but I don't think they were of the type I described. If messages that (look like they) come from mailing lists are less likely to be blocked by filters (whether this is because of DKIM or for some other reason), I'm sure spammers will one day make use of it.

To me this means two things:
a) as a receiver (i.e. filter) you should never second guess why a DKIM signature is broken (I don't think anyone ever suggested one should);
b) as a sender, if your mail is so important that it should be discarded if the DKIM signature is broken, don't send it through systems that are likely to break it.

Martijn.

> From: Dave CROCKER [mailto:[email-masked]]
> Sent: 04 August 2010 18:35
> To: Martijn Grooten
> That sounds like a theoretical attack.  Are there examples of this
> being done in
> the wild?  Do the examples demonstrate actual utility?
>
> d/
>
> On 8/4/2010 10:29 AM, Martijn Grooten wrote:
> >> What is the security model that makes this expectation of
> preservation
> >> important
> >> and reasonable, given that it is so easily and whimsically violated
> by
> >> a common
> > There's a scenario where a spammer/phisher sets up a mailing list,
> adds a bunch of addresses to the list and then sends a message with a
> paypal.com From: address through the list. The DKIM signature will
> obviously be invalid, but a MTA/spam filter won't be able to decide
> whether this is because the message didn't really come from Paypal, or
> because it did but the mailing list broke it.
> >
> > Martijn.
> >

---

From: Hector Santos <[email-masked]>
Date: Wed, 04 Aug 2010 15:14:24 -0400

Dave CROCKER wrote:
> Folks,
> 
> As long as there is some effort to review what goals are being pursued, 
> with respect to mailing lists, I'd like to raise an additional question:
> 
> For typical, modern mailing lists, a subscriber can choose between 
> delivery of each message, as posted individually, versus delivery of 
> batches of messages in a digest.
> 
> In the latter case, no one would reasonably expect a DKIM signature 
> from a first (author/originator) sequence to survive.  Yet there seems 
> to be some strong expectation that it will or should survive if the 
> recipient happens to choose delivery on a per-message basis.
> 
> preservation important and reasonable, given that it is so easily 
> and whimsically violated by a common recipient-selectable setting?

Good point Dave.

DIGEST MESSAGE

The digest message itself should not present any security issues for 
DKIM signing considerations by the list.  The digest 5322.From is set 
to the list address and the digest body is a unique content creation 
owned and authored by the list itself. Signing the digest should 
result as a valid 1st party for all digest recipients.

INDIVIDUAL MESSAGE

How each message is added to the digest content is implementation based.

In our case, a individual DKIM signed message will not be shown as a 
signed message because the digest format includes  a summary index 
table at the top followed by the text only display of each one showing 
only the necessary primary headers to distinguish the specific message:

     Summary index Table

     --- Digest Message #1 -----
     Date:
     From:
     Subject:

     text/plain body only, no attachments

     --- Digest Message #2 -----
     Date:
     From:
     Subject:

     text/plain body only, no attachments

     ....

So what will it take for individual signed messages to survive a digest?

Since each message submitted to the list is stored in its original 
integrity, the only way I see is to add the validated (with AR header) 
signed message to the digest as a message attachment, probably as a 
content type of message/rfc822.

Most MUAs (like ThunderBird) will display the message/rfc822 
attachment icon and when clicked, a new Message Window is shown. 
This message will be viewed as a signed message by the original author 
and not the distributing list domain.

So I think the individual DKIM message can survive with it original 
signature when added to the digest as a message/rs822 attachment. 
When viewed by the user, it would appear as if the author send a 
direct private message to the user.

Security issues?

I don't see any for the DIGEST signature itself. It would be a more 
trusted 1st party signature by the list domain.

For the individual message views, if saved as an attachments, the list 
should at least validate it and add the AR to it before adding it to 
the digest as a message attachment.  The AR will most likely already 
be in the list message submission added by the edge point WCSMTP 
receiver when it does its DKIM (and ADSP) validation.  So the list 
would not have to worry about doing another DKIM (and ADSP) validation.

-- 
HLS

---

From: John Levine <[email-masked]>
Date: 4 Aug 2010 21:01:34 -0000

>whether this is because the message didn't really come from Paypal,
>or because it did but the mailing list broke it.

I don't see how this poses any new problems.

If you believe in ADSP or manual drop lists, you drop the message
because it's from paypal.com and it's unsigned.  I think we can expect
that we won't see any real paypal.com mail coming through lists.

Otherwise, it's just spam.  Does anyone treat List-ID: or other list
headers as a not-spam indicator unless it's from a list that you have
reason to think has local subscribers?  I certainly don't.

R's,
John

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 04 Aug 2010 14:10:10 -0700

...
> I don't see how this poses any new problems.

More to the point is that this attack does not appear to be relevant to the 
question I asked.

Phrased differently, the question I am asking is:

    A mailing list digest does not preserve DKIM signatures from (any of) the 
original messages, and this appears to be acceptable to the community.

    Given that, why is it important to have those same signatures preserved, on 
the whim of a recipient's happening to decide to receive postings one at a time?

d/

-- 

   Dave Crocker

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Wed, 04 Aug 2010 23:44:59 +0200

Dave,

> original messages, and this appears to be acceptable to the community.

Are you sure it is acceptable to everyone, or does the community take it 
as it is? I agree with you that there should be no difference regarding 
the treatment of the original DKIM signature, whether the message 
arrives in digest form or not. I'm still not convinced that the original 
DKIM signature is not relevant for the verifier of the message at the 
receiver side.

The tension that there is between the MLM being a User Actor and being a 
Mediator is illustrated with the following text you wrote in RFC5598:

>        RFC5322  <http://tools.ietf.org/html/rfc5322>.Reply-To:  Set by - Mediator or original Author
>
>           Although problematic, it is common for a Mailing List to assign
>           its own addresses to the Reply-To: header field of messages
>           that it posts.  This assignment is intended to ensure that
>           replies go to all list members, rather than to only the
>           original Author.  As a User Actor, a Mailing List is the Author
>           of the new message and can legitimately set the Reply-To:
>           value.  As a Mediator attempting to represent the message on
>           behalf of its original Author, creating or modifying a
>           Reply-To: field can be viewed as violating that Author's
>           intent.

If we look at the MLM as being a User Actor, then I agree that we should 
not care about the original DKIM signature. If however we consider the 
MLM as a Mediator, we should probably care about the original DKIM 
signature.

Is there consensus that in the context of an MLM the original DKIM 
signature can be dropped and we should not care about it?

/rolf

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 4 Aug 2010 14:47:44 -0700

> Sent: Wednesday, August 04, 2010 2:10 PM
> To: [email-masked]

Does "acceptable" here mean the community has considered the issue and elected to accept it the way it is, or the community is unaware of this as a consideration and operates calmly unaware of it as a possibility?

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 04 Aug 2010 14:52:00 -0700

On 8/4/2010 2:44 PM, Rolf E. Sonneveld wrote:
> Are you sure it is acceptable to everyone, or does the community take it as it
> is?

That's a fair question, and frankly I doubt much of the community is even aware 
of the issue.  So really I'm making an assumption.

Given that it is impossible to preserve the signature, when the message is 
embedded in another message, I'd be inclined to say that we need to see evidence 
from the community that that's not acceptable.

 >   I agree with you that there should be no difference regarding the treatment
> of the original DKIM signature, whether the message arrives in digest form or
> not. I'm still not convinced that the original DKIM signature is not relevant
> for the verifier of the message at the receiver side.

If they cannot verify the signature and the specification says to treat 
unverified signature the same as having no signature, then anything else the 
receiver chooses to do is outside of the specification.

> Mediator is illustrated with the following text you wrote in RFC5598:

I don't understand what you mean by "tension".  A Mediator is a type of User 
Actor.  It is not a Relay.

> If we look at the MLM as being a User Actor, then I agree that we should not
> care about the original DKIM signature. If however we consider the MLM as a
> Mediator, we should probably care about the original DKIM signature.
>
> Is there consensus that in the context of an MLM the original DKIM signature can
> be dropped and we should not care about it?
>
> /rolf

-- 

   Dave Crocker

---

From: Steve Atkins <[email-masked]>
Date: Wed, 4 Aug 2010 15:20:40 -0700

> Does "acceptable" here mean the community has considered the issue and elected to accept it the way it is, or the community is unaware of this as a consideration and operates calmly unaware of it a…

I'd be surprised if most of the mailing list using or operating community
even considered it a meaningful question.

A DKIM signature is used to track reputation of a mail stream, in order
to make delivery decisions or to make it possible in some cases to
identify someone to contact in case of complaints.

In both cases the only entity anyone is going to care about is the
mailing list, so the question is whether the mailing list DKIM signs
mail, or not. "Maintaining a DKIM signature" isn't a meaningful
concept for digests which are _sent by the list_ and have no other
author. (I'm not touching on whether the same is true for non-digest
mail here).

OTOH, multipart/digest mail attaches the messages as message/rfc822
attachments. I don't see any fundamental reason why you couldn't
pull those out and check the DKIM signatures if you felt so inclined
(though I don't see the value of doing so).

Cheers,
  Steve

---

From: Douglas Otis <[email-masked]>
Date: Wed, 04 Aug 2010 15:24:50 -0700

> that we won't see any real paypal.com mail coming through lists.
Clearly, paypal.com did not initially internalize the significance of 
ADSP dkim=discardable.  Perhaps now they have.  I seriously doubt they 
were alone in that regard.  It is not inconceivable that A-R headers 
were seen as a remedy that might allow such use.   There are many 
similar companies where employees exchange messages using the 
recognizable domain of their organization.  ADSP dkim=discardable is not 
a guarantee of messages never reaching an informal mailing list, and the 
appendix in RFC5617 only alluded to problems caused when the Author 
Domain signature is not valid.
> reason to think has local subscribers?  I certainly don't.
Most people are unable to deal with the typical email volumes generated 
by informal mailing lists, without reliance upon message sorting.  In 
such a case, it seems unlikely a recipient would consider sorted mail as 
representing some type of individual transaction.  A message considered 
to have been emitted from an informal mailing-list is unlikely effective 
as a phishing lure, where most lists will also unsubscribe members who 
make unrelated solicitations.

While phishing remains a serious matter, disruptions created by ADSP 
should be resolved by modifying its assertions in a manner still 
effective against phishing.  A community compiled list of informal 
third-party services could greatly facilitate these efforts.

-Doug

---

From: Hector Santos <[email-masked]>
Date: Wed, 04 Aug 2010 19:18:34 -0400

> ADSP dkim=discardable.  Perhaps now they have.  

The evidence is there they want hard failures via SPF and ADSP. Having 
those public records is a strong conscious decision of their intent 
for exclusive paypal.com mail operations. No Outsiders.

While I have no direct confirmation, I believe PAYPAL.COM is looking
for faults (BAD) more than anything else:

   If the mail didn't come from our machines, its not our mail.
   Get rid of it. (SPF)

   If the mail does not a valid 1st DKIM signature, its not out mail.
   Get rid of it. (DKIM+ADSP)

Fault detection is clearly the current biggest possible realistic
practical benefit you can get from DKIM+ADSP - if and only if 
receivers supported ADSP.  It is also the easiest and most practical 
logic you can to software today - no batteries (REPUTATION databases, 
do they exist?) required.

Whether the receiver actually gets rid/reject the faulty mail is 
another matter.  Even if the WHITE LIST model is being sought, you can 
only determine a valid good message after you determine it isn't a bad 
message in the first place. In addition, SPF, ADSP allows for 
detection of faults with anonymous transactions.  Look for good mail 
requires batteries - reputations databases (which one?)

> I seriously doubt they 
> recognizable domain of their organization.  

Maybe they decided to use subdomains (i.e. corp.paypal.com, 
sales.paypal.com, etc).  Its not hard to accept this practice of 
separating the highly abused paypal.com brand domain from everything else.

Facebook.com, facebookmail also has a hard SPF policy, all messages 
seem to be 1st party signed only but there is no explicit ASDP saying so.

-- 
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

---

From: Douglas Otis <[email-masked]>
Date: Wed, 04 Aug 2010 18:09:01 -0700

> > Clearly, paypal.com did not initially internalize the significance
> > of ADSP dkim=discardable.  Perhaps now they have.
>
>  The evidence is there they want hard failures via SPF and ADSP.
>  Having those public records is a strong conscious decision of their
>  intent for exclusive paypal.com mail operations. No Outsiders.

By internalize, I meant paypal.com users were not fully cognizant of 
issues related to their ADSP assertions.  I don't think paypal set out 
to cause mailing-list related problems or failures.

>  do they exist?) required.

Agreed.  ADSP represents a proactive measure against phishing where 
reactive solutions have been ineffective primarily due to the rapidity 
demonstrated with bot-nets.

>  another matter.

Which is why ADSP needs to accommodate normal use.  Otherwise, with 
these issues, broader adoption is less likely.

>  Even if the WHITE LIST model is being sought, you
>  can only determine a valid good message after you determine it isn't
>  a bad message in the first place. In addition, SPF, ADSP allows for

As the Internet transitions into accepting email from IPv6 sources, 
associating an IP address seen at receiving MTAs with some DNS response 
becomes increasingly problematic.  With IPv6 not being a superset of 
IPv4, the resulting CGNs, tunnels, and translation services will cause 
such methods to be unreliable.  I am about to write an ID to describe 
how DKIM keys could be used to authenticate outbound servers, and not 
just the message.  Reputation will not work at the message using 
cryptographic schemes that can be replayed.  Any such scheme can be 
easily gamed.  The TPA-Label scheme is able to authorize informal 
third-party services based upon this type of authentication.  A 
cryptograhic name based approach to authenticating outbound servers 
represents a sound bases for defending email.

> > I seriously doubt they were alone in that regard.  It is not
> > inconceivable that A-R headers were seen as a remedy that might
> > allow such use.   There are many similar companies where employees
> > exchange messages using the recognizable domain of their
> > organization.
>

Using subdomains or look-alike domains are not very helpful at 
preventing phishing.  Such tactics make phishing more productive by 
increasing recipient confusion.  There is paypal-inc.com that lacks any 
ADSP assertion, where one hopes this domain does not become targeted.

>  seem to be 1st party signed only but there is no explicit ASDP saying

Paypal.com is already at the limit of 10 spf records (that are 
replicated for version 2) to define their IPv4 address space.  However 
their version 1 and 2 record sets return a neutral result when a source 
IP address does not match.  Facebook, on the other hand, offers a FAIL.  
Clearly, the spf approach will prove frustrating when attempting to 
connect to IPv6 servers.

-Doug

---

From: Martijn Grooten <[email-masked]>
Date: Thu, 5 Aug 2010 15:35:30 +0100

> the whim of a recipient's happening to decide to receive postings one

In my understanding, The Problem isn't that recipients want emails sent through a mailing list to preserve DKIM signatures. As you point out, it'd be a bit silly to expect that.

Rather, The Problem is that as a sender who DKIM signs their messages, you would _ideally_ want all your messages to arrive at the final recipients with a valid signature. If this were the case (and if you could be sure all your mail left your systems with a valid DKIM signature) you could publish an ADSP dkim=discardable record.

Mailing lists are one of the reasons why this isn't the case and thus I believe that only in exceptional cases one should publish an ADSP dkim=discardable record. However, if we could somehow find a way for DKIM signatures to survive list servers, it would be worth looking into. (Mind you, I don't think we can or at least not unless we reinvent email or something.)

Even if we will still have the "problem" of unsigned messages to arrive in mailing list digests. Because -- and that's why I mentioned the scenario -- there are phishing attacks possible that work through lists but are extremely unlikely to work when the message is part of a digest.

Martijn.

---

From: John Levine <[email-masked]>
Date: 5 Aug 2010 15:19:51 -0000

> there are phishing attacks possible that work through lists but are
>extremely unlikely to work when the message is part of a digest.

Could you give some examples of phishing attacks that work through
lists?  Real ones you've seen would be much more helpful than
hypothetical ones.

The only recent phish I can think of on my lists is the "I've been
robbed in a foreign country and need money" scam.  But that invariably
comes from a compromised account, where the crook has stolen the
purported sender's credentials.  How could DKIM or any signature
scheme address that?

R's,
John

PS: my apologies to anyone who's finding this repetitive
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 09 Aug 2010 12:35:10 +0100

--On 4 August 2010 10:07:36 -0700 Dave CROCKER <[email-masked]> wrote:

>
> What is the security model that makes this expectation of preservation
> important  and reasonable, given that it is so easily and whimsically
> violated by a common  recipient-selectable setting?

I think the identified problem is one of deliverability, not security.

However, if there's a need to trust the original sender, and you don't 
quite trust the list to get that right for you, then the implication is 
that the recipient (the person with the need) should not select a digest 
mode.

-- 
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/