Subject: [dmarc-ietf] Reports helping spammers? (#81)

From: "Brotman, Alex" <[email-masked]>
Date: Thu, 21 Jan 2021 20:59:41 +0000

Hello folks,

Thought I'd see if we could come to a conclusion on this ticket.  The gist is that the reporter believes that (aggregate?) reports can help spammers to determine some effectiveness of their message attempts.

Full Text:
-------------
Spammers could use DMARC reports to monitor the effectiveness of their campaigns, and we do not want to help them. Do existing implementations send reports to any domain that requests them, or only to those domains that are considered "acceptable"? If reports are only sent to acceptable domains, what sort of criteria have been useful?

System administrators will appreciate such advice. Product developers will need guidance about the features they should provide so that a system administrator can control which domains do not receive reports.
-------------

>From an operator side, I don't agree with this assessment.  The reports do not show if/why a MBP may place a message in the Junk folder.  Could it be DMARC quarantine?  Sure.  It could also be any number of things from a large matrix of decisions, none of which are shown in a DMARC report.  Also, the reports are typically sent once per day (seems like most ignore the 'ri'), quite likely some time after the end of the reporting period.  Additionally, they probably have more efficient/immediate methods of evaluating their success rate.

If you believe something has been overlooked, please feel free to share.

--
Alex Brotman
Sr. Engineer, Anti-Abuse & Messaging Policy
Comcast

---

From: Seth Blank <[email-masked]>
Date: Thu, 21 Jan 2021 13:42:25 -0800

I don't understand this concern. The data in a DMARC report speaks to the
underlying authentication of a message on receipt, and nothing about the
"spaminess" or not of a message as it's processed.

On Thu, Jan 21, 2021 at 1:00 PM Brotman, Alex <Alex_Brotman=
[email-masked]> wrote:

> Hello folks,
>
> Thought I'd see if we could come to a conclusion on this ticket.  The gist
> is that the reporter believes that (aggregate?) reports can help spammers
> to determine some effectiveness of their message attempts.
>
> Full Text:
> -------------
> Spammers could use DMARC reports to monitor the effectiveness of their
> campaigns, and we do not want to help them. Do existing implementations
> send reports to any domain that requests them, or only to those domains
> that are considered "acceptable"? If reports are only sent to acceptable
> domains, what sort of criteria have been useful?
>
> System administrators will appreciate such advice. Product developers will
> need guidance about the features they should provide so that a system
> administrator can control which domains do not receive reports.
> -------------
>
> >From an operator side, I don't agree with this assessment.  The reports
> do not show if/why a MBP may place a message in the Junk folder.  Could it
> be DMARC quarantine?  Sure.  It could also be any number of things from a
> large matrix of decisions, none of which are shown in a DMARC report.
> Also, the reports are typically sent once per day (seems like most ignore
> the 'ri'), quite likely some time after the end of the reporting period.
> Additionally, they probably have more efficient/immediate methods of
> evaluating their success rate.
>
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

-- 

*Seth Blank* | VP, Standards and New Technologies
*e:* [email-masked]
*p:* 415.273.8818

This email and all data transmitted with it contains confidential and/or
proprietary information intended solely for the use of individual(s)
authorized to receive it. If you are not an intended and authorized
recipient you are hereby notified of any use, disclosure, copying or
distribution of the information included in this transmission is prohibited
and may be unlawful. Please immediately notify the sender by replying to
this email and then delete it from your system.

---

From: Todd Herr <[email-masked]>
Date: Thu, 21 Jan 2021 18:03:00 -0500

I would submit that it would be in the best interests of mailbox providers
and others who are in the business of making acceptance and filtering
decisions to ensure that spammers properly authenticate their email, so as
to better and more reliably assign a poor reputation to the associated
domains and make it easier to identify them.

On Thu, Jan 21, 2021 at 4:42 PM Seth Blank <seth=
[email-masked]> wrote:

>> Thought I'd see if we could come to a conclusion on this ticket.  The
>> gist is that the reporter believes that (aggregate?) reports can help
>> spammers to determine some effectiveness of their message attempts.
>>
>> Full Text:
>> -------------
>> System administrators will appreciate such advice. Product developers
>> will need guidance about the features they should provide so that a system

-- 

*Todd Herr* | Sr. Technical Program Manager
*e:* [email-masked]
*p:* 703.220.4153

---

From: Emil Gustafsson <[email-masked]>
Date: Thu, 21 Jan 2021 20:11:37 -0800

I think spammers would only get two things from the DMARC reports:
1) Have they setup their configuration correctly?
2) Consume resources from the mailbox provider to generate reports.

Only if #2 is a concern for the mailbox provider I think there is a need to
filter who gets a report and who does not.

/E

On Thu, Jan 21, 2021 at 3:03 PM Todd Herr <todd.herr=
[email-masked]> wrote:

>> *p:* 415.273.8818 <(415)%20273-8818>
>>
>>
> *p:* 703.220.4153 <(703)%20220-4153>
>
>

---

From: Douglas Foster <[email-masked]>
Date: Fri, 22 Jan 2021 07:00:30 -0500

My specific concern is with disposition information.    Whether the message
presents with acceptable credentials is the sender's responsibility, and I
have no problem with documenting whether it passed SPF or DKIM or both,
including which DKIM scope was used for the evaluation.     I have no
problem with reporting the sender policy that I retrieved from DNS.

But should I report the disposition applied, and the reason that my
disposition was different from the sender policy?    That seems like
information leakage about my defenses which should only be revealed to
highly trusted correspondent domains.

Consider what is likely to happen If I call an organization's help desk and
say,
     "My emails to Sally are not getting delivered, and I want to know why?"
The answer should be, and usually is,
     "Have Sally open a ticket and we will discuss the problem with her!"
Should not the same policy apply here?

Possible misuse of disposition information:
- DMARC=(Fail), Disposition = (120 delivered) -- probably means that my
system does not enforce DMARC at all
- DMARC=(Pass), Disposition = (20 delivered, 100 rejected)  -- possibly
means that my system needs 20 messages to learn how to identify bad content

I suggest that disposition information should be redacted by default, and
only included on an exception basis for highly trusted source domains.

Doug Foster

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 22 Jan 2021 14:15:48 +0100

On Fri 22/Jan/2021 13:00:30 +0100 Douglas Foster wrote:
> - DMARC=(Fail), Disposition = (120 delivered) -- probably means that my system 
> does not enforce DMARC at all
> - DMARC=(Pass), Disposition = (20 delivered, 100 rejected)  -- possibly means 
> that my system needs 20 messages to learn how to identify bad content
> 
> I suggest that disposition information should be redacted by default, and only 
> included on an exception basis for highly trusted source domains.

There is a paragraraph in RFC 7489 that I cannot find in the draft:

    Aggregate reports are limited in scope to DMARC policy and
    disposition results, to information pertaining to the underlying
    authentication mechanisms, and to the identifiers involved in DMARC
    validation.

Can that be clarified better?  Se spec sometimes uses the term "final 
disposition" to mean what Doug calls "disposition information" in the text 
quoted above.

To wit, a DMARC filter acting between DATA and end-of-data doesn't actually 
know about final disposition.  It can reject, but cannot deliver to Inbox.

In fact, the <disposition> field refers to what the filter is configured to do, 
except that quarantine can only be signaled to downstream processes.  It 
reports "quarantine" if it did that signaling.  It reports "reject" if it 
rejects.  Otherwise reports "none".  (Did we conclude we want "pass"?)

Identifying bad content, possibly based on DMARC having identified a bad actor, 
or after actual content inspection, happens downstream.  A DMARC filter doesn't 
know and doesn't want to know the final outcome, and doesn't report it.

It is important to be very clear on this point, to avoid that receivers fail to 
enable aggregate reporting for fear of helping spammers.

Best
Ale
--

---

From: Todd Herr <[email-masked]>
Date: Fri, 22 Jan 2021 08:40:11 -0500

On Fri, Jan 22, 2021 at 7:00 AM Douglas Foster <
[email-masked]> wrote:

> My specific concern is with disposition information.    Whether the
> message presents with acceptable credentials is the sender's
> responsibility, and I have no problem with documenting whether it passed
> SPF or DKIM or both, including which DKIM scope was used for the
> evaluation.     I have no problem with reporting the sender policy that I
> retrieved from DNS.
>
> Consider what is likely to happen If I call an organization's help desk
> and say,

> only included on an exception basis for highly trusted source domains.
You're using the phrase "disposition was different from the sender policy"
in a way that I don't understand.

Sender policy is a request for handling when a message fails DMARC
validation checks. In your examples of possible misuse of disposition
information, one you're citing is 100 rejected messages when DMARC passes;
that's not a disposition that's different from sender policy, because DMARC
pass is no guarantee of a message being accepted, and again, sender policy
only concerns the state of DMARC failing validation checks. The DMARC
policy statement isn't a vehicle for requesting handling when the message
passes DMARC checks. Beyond that, I'm not even sure that a condition exists
where a message would have a disposition of "rejected due to DMARC" when
the DMARC validation result is pass, but I've been accused in the past of
lacking imagination, so perhaps it could happen.

For your example of DMARC failing and all 120 messages being delivered,
I've never personally met a spammer (every conversation I ever had started
with "I'm not a spammer")  but I can't conceive of a spammer configuring
his domain with a DMARC policy of p=reject and sending mail that doesn't
authenticate as a way of probing things, but I suppose it could happen.
Because aggregate reports only come in once every 24 hours from most
places, he's not going to get immediate gratification like he would simply
by having a few test or seed accounts at the target domain, but maybe he's
patient. Of course, DMARC isn't the sole arbiter of whether or not the
message made it to the Inbox and not the Junk or Spam folder, so results
would be inconclusive at best. His test accounts will tell him much more
than DMARC reports will tell him.

I can't speak for any mailbox providers, but I suspect that the work of
updating their reporting tools to handle an exception list and curating
such a list is more expensive to them than whatever risk might exist in
generating a DMARC aggregate report for a few spam sending domains. Maybe a
note in Security Considerations or something? *shrug*

--

---

From: Douglas Foster <[email-masked]>
Date: Fri, 22 Jan 2021 09:02:35 -0500

Not sure what is unclear about my concern.

The spec provides for reporting whether the actual disposition was
different from the sender policy request, and the reason that this was done.

Ale suggests that "disposition" must be narrowly defined to mean only the
disposition based on DMARC staus.  This still means that local policy is
revealed under some circumstances.

I do not see why local policy should be revealed at all.

---

From: Todd Herr <[email-masked]>
Date: Fri, 22 Jan 2021 09:15:29 -0500

> different from the sender policy request, and the reason that this was done.
"DMARC=(Pass), Disposition = (20 delivered, 100 rejected)  -- possibly
means that my system needs 20 messages to learn how to identify bad content"

As I said, the above is not an example of the 'actual disposition being
different from the sender policy request', because the sender policy
request does not cover the scenario where DMARC passed. It only applies to
those cases where DMARC fails. There is no facility for a domain owner to
use DMARC to request message treatment when the DMARC validation result is
'pass', nor should a domain owner assume that a message that passes DMARC
checks will be accepted solely based on that information.

> I do not see why local policy should be revealed at all.

Consider the opposite case, one where a hypothetical financial institution
publishes a policy of p=reject and receives aggregate reports showing mail
that it did not originate failing DMARC but not being rejected. That is
information that might prompt a conversation between the financial
institution and the DMARC validator, in an effort to ensure that its mutual
customers aren't exposed to possible abuse vectors.

Yes, that would still be communicated in your scenario of highly trusted
correspondent domains, but I don't believe that the work necessary to
curate such a list provides enough ROI for the report generator to override
whatever minimal risk there might be in revealing its DMARC handling
policies to miscreants. DMARC aggregate reports are not the only source of
such information for the miscreant; the bad guys already have accounts at
their target mailbox providers, and they're sending mail to those accounts
and learning what happens from those accounts and from their own bounce
logs.

--

---

From: Douglas Foster <[email-masked]>
Date: Fri, 22 Jan 2021 10:19:02 -0500

Your second point seems to address my question, asserting that the value of
information sharing exceeds the risk.   This is debatable, so I think it
should be documented in security considerations and reacting options should
be articulated.

The first point still escapes me.  If we are providing information about
all messages received, and are providing disposition information about all
those messages, then it includes information about messages that were
acceptably authorized.  Sender policy is irrelevant.  All that matters is
which messages are reported with disposition results.

---

From: Seth Blank <[email-masked]>
Date: Fri, 22 Jan 2021 07:31:18 -0800

Douglas, what I'm hearing in this thread is that the information in a DMARC
report is well understood, and mailbox receivers are especially sensitive
to information leakages to spammers who could abuse that, and after seven
years of operational experience, are telling you such leakage does not
occur from DMARC aggregate reports.

I'm not opposed to saying such a thing in the considerations if there's
group consensus to do so, but let's wrap this thread and move on to others
tickets now, please.

Seth, as Chair

On Fri, Jan 22, 2021 at 7:19 AM Douglas Foster <
[email-masked]> wrote:

> Your second point seems to address my question, asserting that the value
> of information sharing exceeds the risk.   This is debatable, so I think it
>>> Ale suggests that "disposition" must be narrowly defined to mean only
>>> the disposition based on DMARC staus.  This still means that local policy
>>> is revealed under some circumstances.
>>>
>> Consider the opposite case, one where a hypothetical financial
>> institution publishes a policy of p=reject and receives aggregate reports
>> showing mail that it did not originate failing DMARC but not being
>> rejected. That is information that might prompt a conversation between the
>> financial institution and the DMARC validator, in an effort to ensure that
>> its mutual customers aren't exposed to possible abuse vectors.
>>

--

---

From: Douglas Foster <[email-masked]>
Date: Fri, 22 Jan 2021 10:40:35 -0500

Ok

On Fri, Jan 22, 2021, 10:31 AM Seth Blank <[email-masked]> wrote:

> Douglas, what I'm hearing in this thread is that the information in a
> DMARC report is well understood, and mailbox receivers are especially
> sensitive to information leakages to spammers who could abuse that, and
> after seven years of operational experience, are telling you such leakage
> does not occur from DMARC aggregate reports.
>

---

From: "John Levine" <[email-masked]>
Date: 22 Jan 2021 17:40:18 -0500

In article <[email-masked]> you write:
>-=-=-=-=-=-
>-=-=-=-=-=-

We have a decade of experience with DMARC and I don't ever recall
anyone having a practical issue with who they send reports to.

As someone said, the better the spammers align their stuff, the better
we can filter it.

Close this, please.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 23 Jan 2021 12:20:50 +0100

> As someone said, the better the spammers align their stuff, the better

100% agreed.

> Close this, please.

Please don't.  That such a doubt can cross the minds even of knowledgeable 
people is a real issue.  At a minimum, the paragraph I cited[*] should be 
restored.  A crispy further clarification is welcome.

Best
Ale
-- 

[*] https://mailarchive.ietf.org/arch/msg/dmarc/HRAR3hSdckw3mU_Gebh8vcYOjm4

---

From: Douglas Foster <[email-masked]>
Date: Sat, 23 Jan 2021 09:13:53 -0500

The zeal with which the status quo is being defended suggests that
disposition reporting is pretty valuable to legitimate sending domains.
However, this situation complicates any effort to argue that such
information is useless and harmless for malicious domains.   Such a fine
distinction has not yet been justified.

I can fully endorse Murray's position that alignment reporting is
beneficial, even when the sending domain is malicious.   However, it is
also off-topic.   My focus is on disposition reporting, not alignment
reporting.

Bottom line:  The perceived risk of disposition reporting will differ with
each person, and therefore with each reporting domain.    The specification
would be improved by providing a way for skeptical domain owners to redact
information that they do not wish to disclose.   Currently, the options are
to (a) not report at all, or (b) report ambiguous and slightly misleading
information such as "dispostion=quarantine, overridereason=other".   A
better option would be to have options to state
"dispositioin=not specified, overridereason=not specified".

On Sat, Jan 23, 2021 at 6:21 AM Alessandro Vesely <[email-masked]> wrote:

> [*]
> https://mailarchive.ietf.org/arch/msg/dmarc/HRAR3hSdckw3mU_Gebh8vcYOjm4
>
>
>
>
>
>
>
>
>
>
>
>
>
>

---

From: "Brotman, Alex" <[email-masked]>
Date: Mon, 25 Jan 2021 00:31:58 +0000

Ale,

I didn't incorporate that section as I thought that section about Privacy Considerations might be in the main document as it applies to both rua/ruf.  I'll work to add something similar for the coming revision, thank you for the note.

> -----Original Message-----
> From: dmarc <[email-masked]> On Behalf Of Alessandro Vesely
> Sent: Saturday, January 23, 2021 6:21 AM
> To: John Levine <[email-masked]>; [email-masked]; [email-masked]
> Subject: Re: [dmarc-ietf] Reports helping spammers? (#81)
>
> https://urldefense.com/v3/__https://mailarchive.ietf.org/arch/msg/dmarc/HRA
> R3hSdckw3mU_Gebh8vcYOjm4__;!!CQl3mcHX2A!VZ89uCnai2kVE35KuFXiBNx_0
> VGYmUVnYzQbjqBfthrbcZ3wazcjrZ0hgM9i2T4iF5MP$
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/dmarc__;!
> !CQl3mcHX2A!VZ89uCnai2kVE35KuFXiBNx_0VGYmUVnYzQbjqBfthrbcZ3wazcjrZ0
> hgM9i2d5OX4cb$

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 25 Jan 2021 12:03:41 +0100

On Sat 23/Jan/2021 15:13:53 +0100 Douglas Foster wrote:
> 
> I can fully endorse Murray's position that alignment reporting is beneficial, 
> even when the sending domain is malicious.   However, it is also off-topic.  
>   My focus is on disposition reporting, not alignment reporting.

I see.

> Bottom line:  The perceived risk of disposition reporting will differ with each 
> person, and therefore with each reporting domain.    The specification would be 
> improved by providing a way for skeptical domain owners to redact 
> information that they do not wish to disclose.   Currently, the options are to 
> (a) not report at all, or (b) report ambiguous and slightly misleading 
> information such as "dispostion=quarantine, overridereason=other".   A better 
> option would be to have options to state "dispositioin=not specified, 
> overridereason=not specified".

The information that reports actually disclose is when they say why the 
disposition differs from what the author's domain asked.  That info is given in 
the comment field, which can be "forwarded", "sampled_out", 
"trusted_forwarder", "mailing_list", "local_policy", or any other reason.  This 
field is already optional.

Saying "none" without explaining why doesn't really disclose much, does it?

Best
Ale
--

---

From: Douglas Foster <[email-masked]>
Date: Mon, 25 Jan 2021 20:33:42 -0500

---------- Forwarded message ---------
From: Douglas Foster <[email-masked]>
Date: Mon, Jan 25, 2021 at 8:32 PM
Subject: Re: [dmarc-ietf] Reports helping spammers? (#81)
To: Alessandro Vesely <[email-masked]>

Yes, I think you are right, the information loss to bad actors is limited,
while the benefits of information release may be pretty valuable to good
actors.

For reporting users, a "non-aligned" raises important questions:   If it is
a legitimate forward, the sender probably wants it accepted.   If it is a
legitimate forward that the sender wants delivered, but the recipient
blocked because of DMARC, then the sender has to decide whether to drop
back to p=none or From rewrite.  All of this is much easier to evaluate if
the sender provides some disposition feedback.   I think we can say
something along these lines in the security considerations.

You said that the disposition="none" or "sampled out" should only mean "Not
blocked because of DMARC evaluation", without making any indication of
whether the message was blocked for any other reason.   I will rest easier
if this concept is articulated clearly in the document.  It is a big
document and I tend to read it in pieces, so perhaps it is there and I
missed it.

To my original question, I don't think I would ever send a report to a
domain that has a negative reputation, or a domain with no reputation that
came from a source with negative reputation.   I don't want those people
talking to me, so I have no intention of talking to them.   That still
allows for notifying positive-reputation domains if a negative-reputation
source abuses their domain.

Doug Foster

On Mon, Jan 25, 2021 at 6:03 AM Alessandro Vesely <[email-masked]> wrote:

> beneficial,
> better