Subject: [dmarc-ietf] Extensions in Aggregate Reporting - Feedback Requested

From: "Brotman, Alex" <[email-masked]>
Date: Thu, 3 Jun 2021 12:47:21 +0000

Hello folks,

During our interim call last week the topic of extensions within the DMARC aggregate report came up.  There was a discussion about how to best introduce these, but also how they might be best used.  I noted three cases that I could see today; ARC, PSD, and BIMI.   And indeed we have tickets relating to the first two.  The original thought was that the aggregate draft would allow a place for extensions, and then additional drafts would define those within the IETF.  When -02 was originally being worked on, there was a thread about how we might like to see this, though not many responses.  The result is in section 4 of the -02 draft [1]. and I thought we'd enhance that as we progressed.  At the time, I didn't intend to limit the extensions to IETF-approved extensions, though wasn't sure how else this might be used by reporting entities (I mentioned domain reputation-ish things during the call).  I'd consider that if we don't enforce IETF-registered extensions, the receivers could still ignore extensions they don't want to handle.  I'm also aware this could bloat a report in terms of size, though we've already indicated we don't seem overly concerned with the size of the XML body.  A few things I'd like to see the group reach consensus on are:

1) Extensions in their own section (as it is now) or within each <row> element
2) Must extensions be IETF-approved
3) If (2) is true, do we want to define any during the DMARCbis process (essentially a demonstration of how it is to be done)

Thank you for your continued feedback

1: https://datatracker.ietf.org/doc/html/draft-ietf-dmarc-aggregate-reporting-02#section-4

--
Alex Brotman
Sr. Engineer, Anti-Abuse & Messaging Policy
Comcast

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 4 Jun 2021 11:26:36 +0200

On Thu 03/Jun/2021 14:47:21 +0200 Brotman, Alex wrote:
> 
> During our interim call last week the topic of extensions within the DMARC aggregate report came up.  There was a discussion about how to best introduce these, but also how they might be best used.  I noted three cases that I could see today; ARC, PSD, and BIMI.   And indeed we have tickets relating to the first two.  The original thought was that the aggregate draft would allow a place for extensions, and then additional drafts would define those within the IETF.  When -02 was originally being worked on, there was a thread about how we might like to see this, though not many responses.  The result is in section 4 of the -02 draft [1].

I have some comments about that attempt.  First, it shows extensions right below <feedback>, while it seems more useful to have them as child of <record>.  Second, I'm not sure we need an <extensions> container.  I'd go for an example like, say, so:

<feedback xmlns="http://ietf.org/xml-namesapaces/dmarc-xml/1.0">
    <report_metadata>
       ...
    </report_metadata>
    <policy_published>
       ...
    </policy_published>
    <extension_metadata name="bimi" xmlns="http://ietf.org/xml-namesapaces/bimi-xml?/1.0">
       ...
    </extension_metadata>
    <record>
       <row>
          ...
       </row>
       <identifiers>
          ...
       </identifiers>
       <auth_results>
          ...
       </auth_results>
          ...
       <extension name="bimi" xmlns="http://ietf.org/xml-namesapaces/bimi-xml??/1.0">
          ...
       </extension>
    </record>
    <record>
       ...
    </record>
</feedback>

Third, we need to grasp how XML grammars can be composed, and insert it in Appendix A.

>  At the time, I didn't intend to limit the extensions to IETF-approved extensions, though wasn't sure how else this might be used by reporting entities (I mentioned domain reputation-ish things during the call).  I'd consider that if we don't enforce IETF-registered extensions, the receivers could still ignore extensions they don't want to handle.

I assume no one reads the XML directly, except for debugging.  If report consumers don't know about an extension, its content will never reach human eyeballs.  Extension existence will have to be advertised, and a IANA page could be a decent means of doing that.

>  I'm also aware this could bloat a report in terms of size, though we've already indicated we don't seem overly concerned with the size of the XML body.  A few things I'd like to see the group reach consensus on are:
> 

Both, and both optional.  An extension can have some data to add in some <record>, but not necessarily in all of them.

> 2) Must extensions be IETF-approved

We cannot stop non-registered extensions.  Yet, developers may want to see an RFC before implementing code that extracts a given extension's content.

> 3) If (2) is true, do we want to define any during the DMARCbis process (essentially a demonstration of how it is to be done)

It would be a good way to show how to define them.  Not our primary task, though.

Best
Ale
--

---

From: Trent Adams <[email-masked]>
Date: Fri, 4 Jun 2021 15:45:08 +0000

I agree with Alessandro's suggestions, and really like the flexibility of self-labeled elements so they can be located where appropriate rather than forcing all extensions into a single bucket.

My $0.02,
Trent

On 6/4/21, 3:27 AM, "dmarc on behalf of Alessandro Vesely" <[email-masked] on behalf of [email-masked]> wrote:

    <feedback xmlns="https://urldefense.com/v3/__http://ietf.org/xml-namesapaces/dmarc-xml/1.0__;!!ORgEfCBsr282Fw!6SA4ihYzl7xfKdVfYDefKIsr4PotRb5Nkjs2hXHPyIU5KTpmffBFLkJEKqvwSJvF$ ">
        <report_metadata>
           ...
        </report_metadata>
        <policy_published>
           ...
        </policy_published>
        <extension_metadata name="bimi" xmlns="https://urldefense.com/v3/__http://ietf.org/xml-namesapaces/bimi-xml?*1.0__;Lw!!ORgEfCBsr282Fw!6SA4ihYzl7xfKdVfYDefKIsr4PotRb5Nkjs2hXHPyIU5KTpmffBFLkJEKvQ2pQn-$ ">
           ...
        </extension_metadata>
        <record>
           <row>
              ...
           </row>
           <identifiers>
              ...
           </identifiers>
           <auth_results>
              ...
           </auth_results>
              ...
           <extension name="bimi" xmlns="https://urldefense.com/v3/__http://ietf.org/xml-namesapaces/bimi-xml?**A1.0__;Py8!!ORgEfCBsr282Fw!6SA4ihYzl7xfKdVfYDefKIsr4PotRb5Nkjs2hXHPyIU5KTpmffBFLkJEKtkFNC6p$ ">
              ...
           </extension>
        </record>
        <record>
           ...
        </record>
    </feedback>

    Best
    Ale
    -- 

    > 1: https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draft-ietf-dmarc-aggregate-reporting-02*section-4__;Iw!!ORgEfCBsr282Fw!6SA4ihYzl7xfKdVfYDefKIsr4PotRb5Nkjs2hXHPyIU5KTpmffBFLkJEKtSg8JaH$ 

    _______________________________________________
    dmarc mailing list
    [email-masked]
    https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/dmarc__;!!ORgEfCBsr282Fw!6SA4ihYzl7xfKdVfYDefKIsr4PotRb5Nkjs2hXHPyIU5KTpmffBFLkJEKn_Uzf5Y$

---

From: Dotzero <[email-masked]>
Date: Fri, 4 Jun 2021 14:53:20 -0400

On Thu, Jun 3, 2021 at 8:48 AM Brotman, Alex <Alex_Brotman=
[email-masked]> wrote:

> Hello folks,
>
> During our interim call last week the topic of extensions within the DMARC
> aggregate report came up.  There was a discussion about how to best
> introduce these, but also how they might be best used.  I noted three cases
> that I could see today; ARC, PSD, and BIMI.   And indeed we have tickets
> relating to the first two.  The original thought was that the aggregate
> draft would allow a place for extensions, and then additional drafts would
> define those within the IETF.  When -02 was originally being worked on,
> there was a thread about how we might like to see this, though not many
> responses.  The result is in section 4 of the -02 draft [1]. and I thought
> we'd enhance that as we progressed.  At the time, I didn't intend to limit
> the extensions to IETF-approved extensions, though wasn't sure how else
> this might be used by reporting entities (I mentioned domain reputation-ish
> things during the call).  I'd consider that if we don't enforce
> IETF-registered extensions, the receivers could still
>   ignore extensions they don't want to handle.  I'm also aware this could
> bloat a report in terms of size, though we've already indicated we don't
> seem overly concerned with the size of the XML body.  A few things I'd like
> to see the group reach consensus on are:
>
> 3) If (2) is true, do we want to define any during the DMARCbis process
> (essentially a demonstration of how it is to be done)
>
> 1:
> https://datatracker.ietf.org/doc/html/draft-ietf-dmarc-aggregate-reporting-02#section-4
>
> --
> Alex Brotman

1) Extensions should be in their own extension and seperate from the core
reporting. There is a reason we call them "extensions".
2) Extensions used in reporting under the standard should be IETF-approved.
This is a standards body. Anyone can leverage standards for private use but
our focus as an IETF working group is interoperability. By requiring IETF
approval/registration it puts the extensions under IETF "Note Well" and
makes the extension public.

Michael Hammer

---

From: Matthäus Wander <[email-masked]>
Date: Sat, 5 Jun 2021 13:45:00 +0200

ARC shows the need for extension data within <record> elements:
<https://datatracker.ietf.org/doc/html/rfc8617#section-7.2.2>

Example from RFC 8617:
<policy_evaluated>
  <disposition>none</disposition>
  <dkim>fail</dkim>
  <spf>fail</spf>
  <reason>
   <type>local_policy</type>
   <comment>arc=pass as[2].d=d2.example as[2].s=s2
	as[1].d=d1.example as[1].s=s3
	remote-ip[1]=2001:DB8::1A</comment>
  </reason>
</policy_evaluated>

With proper extension support, this example could look like this:
<row>
   <policy_evaluated>
     <disposition>none</disposition>
     <dkim>fail</dkim>
     <spf>fail</spf>
     <reason>
       <type>extension</type>
       <comment>arc=pass</comment>
     </reason>
   </policy_evaluated>
</row>
<auth_results>...</auth_results>
<extensions>
   <arc xmlns="(...)">
     <result>pass</result>
     <sets>
       <as>
         <instance>2</instance>
         <domain>d2.example</domain>
         <selector>s2</domain>
       </as>
       <as>
         <instance>1</instance>
         <domain>d1.example</domain>
         <selector>s3</domain>
         <remote_ip>2001:DB8::1A</remote_ip>
       </as>
     </sets>
   </arc>
</extensions>

An IANA registry could serve as repository of IETF-approved extensions.

It's worth considering whether certain elements should be defined as 
extensible. In the above example, the <arc> element could be placed 
below <auth_results> next to the existing <dkim> and <spf> elements. 
Implementations support this behavior by simply ignoring unknown 
elements. XML Schema supports extensibility in well-defined places with 
the <xsd:any> element.

Regards,
Matt

---

From: Matthäus Wander <[email-masked]>
Date: Sat, 5 Jun 2021 13:56:04 +0200

Alessandro Vesely wrote on 2021-06-04 11:26:
> Second, I'm not sure we need an <extensions> container.  
> I'd go for an example like, say, so:
> 
> [...]
>     <extension_metadata name="bimi" 
> xmlns="http://ietf.org/xml-namesapaces/bimi-xml?/1.0">
 > [...]
>        <extension name="bimi" 
> xmlns="http://ietf.org/xml-namesapaces/bimi-xml??/1.0">

If we use an attribute for the extension name, then we don't the 
container section.
As the current schema does not use attributes at all, I'm more inclined 
to define the extension name in a different way for consistent non-use 
of attributes. But that's a minor detail.

> Both, and both optional.  An extension can have some data to add in some 
> <record>, but not necessarily in all of them.

+1

Regards,
Matt

---

From: "Brotman, Alex" <[email-masked]>
Date: Mon, 14 Jun 2021 12:41:44 +0000

To summarize,

We'd like to see extensions available both below the "feedback" and "record" elements.  The -02 draft only has it below the "feedback" element.  I agree that all elements, each time they are utilized, should mention a reference as to how they are to be utilized.

We'd also like to have extensions go through an IETF process, however, we also understand that we cannot exclude third parties from defining/deploying their own extensions.  I suppose we could tell report receivers they "MUST" ignore any extensions which are not IETF-approved, though that seems a bit heavy-handed.

So, a sample report may look something like:

   <feedback>
     <version>2.0</version>
     <report_metadata>
       <version>2</version>
       <org_name>Sample Reporter</org_name>
       <email>[email-masked]</email>
       <extra_contact_info>...</export_contact_info>
       <report_id>3v98abbp8ya9n3va8yr8oa3ya</report_id>
       <date_range>
         <begin>161212415</begin>
         <end>161221511</end>
       </date_range>
     </report_metadata>
     <policy_published>
       <domain>example.com</domain>
       <p>quarantine</p>
       <sp>none</sp>
       <pct>100</pct>
     </policy_published>
     <record>
       <row>
         <source_ip>192.168.4.4</source_ip>
         <count>123</count>
         <policy_evaluated>
           <disposition>quarantine</disposition>
           <dkim>pass</dkim>
           <spf>fail</spf>
         </policy_evaluated>
       </row>
       <identifiers>
         <header_from>example.com</header_from>
       </identifiers>
       <auth_results>
         <dkim>
           <domain>example.com</domain>
           <result>pass</result>
           <selector>abc123</selector>
         </dkim>
         <spf>
           <domain>example.com>
           <result>fail</result>
         </spf>
       </auth_results>
        <extensions>
          <extension_name definition="url">
             .........
          </extension_name>
       </extensions>
     </record>
    <extensions>
       <extension_name definition="url">
        ....
       </extension_name>
    </extensions>
   </feedback>

The goal being to allow extensions to live either at the reported-IP level, or at the domain level.

Does this seem reasonable?

> -----Original Message-----
> From: dmarc <[email-masked]> On Behalf Of Matthäus Wander
> Sent: Saturday, June 5, 2021 7:56 AM
> To: [email-masked]
> Subject: Re: [dmarc-ietf] Extensions in Aggregate Reporting - Feedback
> Requested
>
> If we use an attribute for the extension name, then we don't the container
> section.
> As the current schema does not use attributes at all, I'm more inclined to define
> the extension name in a different way for consistent non-use of attributes. But
> that's a minor detail.
>
> > Both, and both optional.  An extension can have some data to add in
> > some <record>, but not necessarily in all of them.
>
> +1
>
> Regards,
> Matt

---

From: Trent Adams <[email-masked]>
Date: Mon, 14 Jun 2021 15:35:01 +0000

+1

On 6/14/21, 6:42 AM, "dmarc on behalf of Brotman, Alex" <[email-masked] on behalf of Alex_Brotman=[email-masked]> wrote:

    To summarize,

       <feedback>
         <version>2.0</version>
         <report_metadata>
           <version>2</version>

    > > xmlns="https://urldefense.com/v3/__http://ietf.org/xml-namesapaces/bimi-xml?*1.0__;Lw!!ORgEfCBsr282Fw!5jPsNDxnYoaeo_HCTDHZOSs5McSKUik5frtgdFxHB_wL9O6VDywrZhokLOwusINU$ ">
    >  > [...]
    > >        <extension name="bimi"
    > > xmlns="https://urldefense.com/v3/__http://ietf.org/xml-namesapaces/bimi-xml?**A1.0__;Py8!!ORgEfCBsr282Fw!5jPsNDxnYoaeo_HCTDHZOSs5McSKUik5frtgdFxHB_wL9O6VDywrZhokLNemKNFV$ ">
    >

    _______________________________________________
    dmarc mailing list
    [email-masked]
    https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/dmarc__;!!ORgEfCBsr282Fw!5jPsNDxnYoaeo_HCTDHZOSs5McSKUik5frtgdFxHB_wL9O6VDywrZhokLIcIrQ-o$

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 14 Jun 2021 18:27:54 +0200

On Mon 14/Jun/2021 14:41:44 +0200 Brotman, Alex wrote:
> 
> I agree that all elements, each time they are utilized, should mention a reference as to how they are to be utilized.
> 
> [...]
> 

So why doesn't <feedback> mention a reference to how it is utilized?

About that overabundance of <version>'s, the 1st entry, right below <feedback>, is the aggregate report version.  Thus far, we agreed that it is useless as a grammar indication if <feedback> contains its namespace declaration.  However, Matt noted that there may be parsers that consider reports to be pre-IETF drafts if they miss the <version> element.  In this case, it makes sense to keep <version>1.0</version> for backward compatibility, especially if we try not to break existing parsers.

The second <version> entry, inside <report_metadata>, had better wear a different name, to avoid confusion.  I assume it's meant to be the report reference version.  A couple of alternative examples:

     <report_metadata>
       <generator>https://github.com/trusteddomainproject/OpenDMARC/releases/tag/rel-opendmarc-1-4-1</generator>

or

     <report_metadata>
       <generator>
         <url>http://www.trusteddomain.org/opendmarc/</url>
         <version>1.4.1</version>
       </generator>

Best
Ale
--

---

From: "John Levine" <[email-masked]>
Date: 14 Jun 2021 15:50:54 -0400

It appears that Brotman, Alex <[email-masked]> said:
>To summarize,
>
>We'd like to see extensions available both below the "feedback" and "record" elements.  The -02 draft only has it below the "feedback" element.  I agree that all
>elements, each time they are utilized, should mention a reference as to how they are to be utilized.
>
>We'd also like to have extensions go through an IETF process, however, we also understand that we cannot exclude third parties from defining/deploying their own
>extensions.  I suppose we could tell report receivers they "MUST" ignore any extensions which are not IETF-approved, though that seems a bit heavy-handed.

Remember, IETF standards are about how to interoperate.  If people want to interpret private extensions,
that is fine so long as the private ones don't interfere with public ones.

While we would like it if people published the details of their private extensions, just knowing the
names is useful since that keeps names from colliding.  So we should set up a FCFS registry for
extensions, specification would be nice but not required.

R's,
John