Subject: Re: [dmarc-ietf] Clarification about data integrity within Aggregate Reports (Ticket #40)

From: "Brotman, Alex" <[email-masked]>
Date: Mon, 4 Jan 2021 15:53:20 +0000

--
Alex Brotman
Sr. Engineer, Anti-Abuse & Messaging Policy
Comcast

> -----Original Message-----
> From: dmarc <[email-masked]> On Behalf Of Alessandro Vesely
> Sent: Thursday, December 31, 2020 10:27 AM
> To: [email-masked]
> Subject: Re: [dmarc-ietf] Clarification about data integrity within Aggregate
> Reports (Ticket #40)
>
> On Wed 30/Dec/2020 23:18:35 +0100 Brotman, Alex wrote:
> > Hello folks,
> >
> > There's an open ticket
> (https://urldefense.com/v3/__https://trac.ietf.org/trac/dmarc/ticket/40__;!
> !CQl3mcHX2A!VCWZO-
> 6THaYEt8r1yDE9TgQzeJyuFmGpwSepa3SWwao3ViUQfPaxJ5FrbnTPKuzHLj6b
> $ ) noting that we should clarify what constitutes valid data in a report. For
> example, the report cannot state that DMARC-DKIM was a "pass" when
> DKIM itself was a failure.  See the original thread here:
> >
> > https://urldefense.com/v3/__https://mailarchive.ietf.org/arch/msg/dmar
> > c/Ii_dLXFzBNnRP361F922ty789I8/__;!!CQl3mcHX2A!VCWZO-
> 6THaYEt8r1yDE9TgQz
> > eJyuFmGpwSepa3SWwao3ViUQfPaxJ5FrbnTPKvlFgYhq$
> >
> > It seems like the gist is that within the report it should never happen that
> DKIM or SPF are noted to have passed in the context of DMARC if they have
> not passed on their own.  It should also be properly noted by the reporter if
> they override with local policy.  Not by overriding the SPF/DKIM failures (and
> showing them as incorrectly passing), but instead by noting that local policy
> overrides properly (regardless of whether that override is higher or lower).
> >
> > Does that seem properly summarized?
>
>
> If the aggregate report content, Section 2.2, was well explained, the above
> text would be redundant.  The point is that Section 2.2 looks like a high level
> list of features.  It is completely useless for implementing a report producer,
> let alone a consumer.  We have to rewrite that section, possibly trying to re-
> use the same wording and the same order of appearance of concepts, so as
> to minimize readers' confusion, but strictly matching the content of Appendix
> A (was Appendix C).
>

I don't think I disagree here, but I want to be clear on what you're requesting.  You'd like to see a more verbose description of the goal of the aggregate report, as well as the contents?

"... Each report MUST contain data for only one 5322 domain.  The values reported MUST be as evaluated from the original message, not from the local policy overrides ..." and so on?

> The consistency checks above can be useful for building verification tools.
>
> Let me take this occasion to recall that there are XML syntax check tools that
> can be used to automatically verify the syntax based on the schema.  We
> should write a more compliant XML in order to use them.

I've written an RNC (Relax NG Compact) previously.  As we get closer to finalizing any format changes, we can create that, and then use jing/pyjing to validate XML reports.  You'd want to use this to enforce report contents as specified above?  Not that I'm opposed, though, it can't enforce that the sender is using the proper values for DKIM pass/fail, only that the value is one of "pass"/"fail" (at least for RNC).  I don't believe there's a way to create interrelated dependencies where we could say that "DMARC can't possibly pass if both SPF and DKIM fail, and the report should be bogus if that's the case".

>
>
> Best
> Ale
> --
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/dmarc
> __;!!CQl3mcHX2A!VCWZO-
> 6THaYEt8r1yDE9TgQzeJyuFmGpwSepa3SWwao3ViUQfPaxJ5FrbnTPKvtKzeGr
> $

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 5 Jan 2021 12:20:37 +0100

On Mon 04/Jan/2021 16:53:20 +0100 Brotman, Alex wrote:
> -----Original Message----- From: dmarc <[email-masked]> On Behalf Of Alessandro Vesely
>>> There's an open ticket (https://trac.ietf.org/trac/dmarc/ticket/40) noting
>>> that we should clarify what constitutes valid data in a report. For 
>>> example, the report cannot state that DMARC-DKIM was a "pass" when DKIM
>>> itself was a failure. >>>
>>> not passed on their own.  [...]
>>>

Yes, more or less.

Section 2.2 is good until "The format for these reports is defined in Appendix 
C."  (Ok, that should be Appendix A)  Following that, it should describe what 
the content actually must be.  I'd start, for example, like so:

     Aggregate reports (feedback) consist of two parts, a header and a set of
     records.  The header holds the report metadata (report_metadata) as well as
     the policy discovered for the given domain, subject of the report
     (policy_published).  Each report MUST contain data for only one domain.
     That policy MUST correspond to the DMARC record that the subject domain
     published during the reporting period.  If the subject domain modified its
     DMARC record during the period, a report generator MAY create multiple
     reports for the same domain, each for the periods during which the record
     didn't vary.  Otherwise, a generator SHOULD report the last policy it
     found.

     The set of records (record)...

I think we can skip elements whose content is obvious, but say something about 
various cases.  I can write more text after we find the right tone.  Is the 
above too lengthy?

> I've written an RNC (Relax NG Compact) previously.  As we get closer to finalizing any format changes, we can create that, and then use jing/pyjing to validate XML reports.

jing is a nice utility.  I had used svalidate and xmlstarlet, which provide 
similar validation.

>  You'd want to use this to enforce report contents as specified above?

Absolutely.  Let me note that, to pass tests using the utilities mentioned 
above, reports should start something like so:

     <?xml version="1.0" encoding="UTF-8"?>
     <dmarc:feedback xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
         xmlns:dmarc="http://dmarc.org/dmarc-xml/0.1"
         xs:schemaLocation="http://dmarc.org/dmarc-xml/0.1 rua.xsd">
         <report_metadata>
         ...

(replace schemaLocation as appropriate)

The above form is currently not used.  Note that the "version" element becomes 
irrelevant when a schema is given.

It is easy, as a developer upgrades her report producer, to add the correct 
boilerplate.  To wit, it's more difficult to add selectors or to switch from 
.zip to .gz.

>  Not that I'm opposed, though, it can't enforce that the sender is using the proper values for DKIM pass/fail, only that the value is one of "pass"/"fail" (at least for RNC).  I don't believe there's a way to create interrelated dependencies where we could say that "DMARC can't possibly pass if both SPF and DKIM fail, and the report should be bogus if that's the case".

Right.  On the other hand, it makes little sense to check consistency if the 
syntax is bad.

We could create a web-based DMARC checking tool.  You get there and request a 
check, authenticating yourself and providing a few email addresses for the 
test.  The tool will then send you various messages from various IPs/ domains 
and then check the aggregate reports you produce.  That would include the 
consistency checks of ticket #40.

Best
Ale
--