Subject: Re: [mile] Including Mail fields in IODEF

From: "Panos Kampanakis (pkampana)" <[email-masked]>
Date: Fri, 1 Mar 2013 21:52:31 +0000

I think MARF provides more functionality and should be leverage for emails in IODEF.
I also think we need to resurrect http://tools.ietf.org/html/draft-vesely-mile-mail-abuse-00 within MILE since MARF was concluded..
Panos

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Moriarty, Kathleen
Sent: Thursday, February 21, 2013 5:19 AM
To: [email-masked]; [email-masked]
Subject: [mile] Including Mail fields in IODEF

Hello,

Cross posting with MAIL and MARF - 

In MILE related work, I have come across use cases that would like to include DKIM and SPF information in addition to specific mail fields (like the ones Chris lists below).  We would like some help to figure out the best approach.  Should we embed ARF and MARF RFC extensions to accommodate this need or should we look at updating RFC5901?  Both take the approach of including an email message as opposed to using XML to tag each field and allow for this in the data model (in my opinion, that is fine and reduces bloat, but there may be other opinions).

There was a draft published last year (link included below) that includes MARF in an IODE extension.

Thanks,
Kathleen
________________________________________
From: Harrington, Christopher
Sent: Wednesday, February 20, 2013 2:57 PM
To: Moriarty, Kathleen; [email-masked]
Subject: RE: Mail fields

I'm for the simplest solution as always. These are the indicator types that we routinely share. I would use these as a base:

Email address (denoting if it is to or from) Email Subject Email attachment name Email attachment hash X-Mailer (from header) Hyperlink in email

It's also very common to share the whole header. Bad guys routinely forge them and put extra header items that can be used as indicators.  Although not an indicator sharing the entire email as an .eml or .msg file is also pretty common.

Thanks,

--Chris

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Moriarty, Kathleen
Sent: Wednesday, February 20, 2013 2:58 AM
To: [email-masked]
Subject: [mile] Mail fields

Hi,

In looking at the updated rfc5070bis and coming across some requests for handling certain types of exchanges, I am curious to hear how others think we should handle mail related indicators and incidents.  A couple of commonly exchanged fields were added into the Record class.  You can still extend out using RFC5901 and include a full mail message, but if you wanted to include DKIM or Sender Policy Framework, you need something else.  The IETF group MARF already solved these issues.

MARF uses the email tags rather than XML and there was a draft that embedded MARF content into IODEF (contains an example), can be found here:
http://tools.ietf.org/html/draft-vesely-mile-mail-abuse-00

Since mail is already marked and can be parsed, would this be a better option to use what MARF has already done to solve the question on how to exchange this data?  Other options would be to update RFC5901 or to extend IODEF further.  I prefer the use of MARF.  It is already in use by mail operators, so there is adoption.

Thanks,
Kathleen
_______________________________________________
mile mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mile
_______________________________________________
mile mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mile

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 3 Mar 2013 01:00:16 -0800

Hi Kathleen, sorry for the delay replying to this.  It seems to me that, as
you suggested, your options are:

1) As I read RFC5901, it's a set of XML tags used to extract specific
pieces of message data or meta-data and include them in an IODEF report.
You could simply add some DKIM and SPF fields as well.  Keep in mind that
SPF applies once per message (because there's always exactly one envelope)
but any number of DKIM results (including zero) can be present.  There's
also some religion around whether a failed DKIM signature is even worth
reporting, but we can get into that separately if needed.

2) RFC5901 also adds the capability to include a complete email message in
the report payload.  This could be an ARF message (using the SPF and DKIM
extensions), which itself can contain the original.

I would suggest going with (1) personally; it's more work procedurally, but
the result is cleaner.  With (2) you're able to capitalize on what MARF
did, but the receiver has to know that the payload message is actually an
abuse report that contains part or all of the problem message, and is not
itself the problem message.

Let me know if you'd like more detail on any of that.

-MSK

On Thu, Feb 21, 2013 at 2:19 AM, Moriarty, Kathleen <
[email-masked]> wrote:

> In MILE related work, I have come across use cases that would like to
> include DKIM and SPF information in addition to specific mail fields (like
> the ones Chris lists below).  We would like some help to figure out the
> best approach.  Should we embed ARF and MARF RFC extensions to accommodate
> this need or should we look at updating RFC5901?  Both take the approach of
> including an email message as opposed to using XML to tag each field and
> allow for this in the data model (in my opinion, that is fine and reduces
> bloat, but there may be other opinions).
>
> I'm for the simplest solution as always. These are the indicator types that
> we routinely share. I would use these as a base:
>
> Email address (denoting if it is to or from)
> Email Subject
> Email attachment name
> Email attachment hash
> X-Mailer (from header)
> Hyperlink in email
>
> It's also very common to share the whole header. Bad guys routinely forge
> them and put extra header items that can be used as indicators.  Although
> not an indicator sharing the entire email as an .eml or .msg file is also
> pretty common.
>
> Thanks,
>
> --Chris
>
>
> In looking at the updated rfc5070bis and coming across some requests for
> handling certain types of exchanges, I am curious to hear how others think
> we should handle mail related indicators and incidents.  A couple of
> commonly exchanged fields were added into the Record class.  You can still
> extend out using RFC5901 and include a full mail message, but if you wanted
> to include DKIM or Sender Policy Framework, you need something else.  The
> IETF group MARF already solved these issues.
>
> MARF uses the email tags rather than XML and there was a draft that
> embedded
> MARF content into IODEF (contains an example), can be found here:
> Since mail is already marked and can be parsed, would this be a better
> option to use what MARF has already done to solve the question on how to
> exchange this data?  Other options would be to update RFC5901 or to extend
> IODEF further.  I prefer the use of MARF.  It is already in use by mail
> operators, so there is adoption.
>
> Thanks,
> Kathleen
> marf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/marf
>

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 3 Mar 2013 01:09:32 -0800

The issue with MARF inside IODEF is that the receiver needs to know that
the payload being provided inside an EmailMessage element is itself an ARF
report, and not the message that caused the report in the first place.  You
certainly could crack open the EmailMessage content and see if conforms to
the ARF specification to tell which kind of report you've gotten, but that
seems inelegant.

I suppose then another option is an extension element that indicates you've
received an ARF payload rather than the actual offending message.

Also of note: An ARF can contain the offending message or only the
offending message's header, and still be compliant.  If your application
needs the whole message, you'll have to add some additional stipulations
someplace.

-MSK

On Fri, Mar 1, 2013 at 1:52 PM, Panos Kampanakis (pkampana) <
[email-masked]> wrote:

> I also think we need to resurrect
> http://tools.ietf.org/html/draft-vesely-mile-mail-abuse-00 within MILE
> since MARF was concluded..
> Panos
>
>
> I'm for the simplest solution as always. These are the indicator types
> that we routinely share. I would use these as a base:
>
> Email address (denoting if it is to or from) Email Subject Email
> attachment name Email attachment hash X-Mailer (from header) Hyperlink in
> email
>
> embedded MARF content into IODEF (contains an example), can be found here:

---

From: "Panos Kampanakis (pkampana)" <[email-masked]>
Date: Sun, 3 Mar 2013 22:37:01 +0000

Thank you Murray.

The "<arf:EmailMessage>
   Received: from mailserver.example.net
        (mailserver.example.net [192.0.2.1])
        by example.com with ESMTP id M63d4137594e46;
        Thu, 08 Mar 2005 14:00:00 -0400
   From: &lt;[email-masked]&gt;
   To: &lt;Undisclosed Recipients&gt;
   Subject: Earn money
   MIME-Version: 1.0
   Content-type: text/plain
   Message-ID: [email-masked]
   Date: Thu, 02 Sep 2004 12:31:03 -0500

   Spam Spam Spam
   Spam Spam Spam
   Spam Spam Spam
   Spam Spam Spam
         </arf:EmailMessage>"

that I see in http://bgp.potaroo.net/ietf/all-ids/draft-vesely-mile-mail-abuse-00.txt looks like just an email message. I don't see "feedback-type" or other ARF fields for example that would make it a ARF.

draft-vesely-mile-mail-abuse-00.txt seems to define a header and then have the option for the actual message (EmailMessage). Am I reading it wrong?

Panos

From: Murray S. Kucherawy [mailto:[email-masked]]
Sent: Sunday, March 03, 2013 4:10 AM
To: Panos Kampanakis (pkampana)
Cc: Moriarty, Kathleen; [email-masked]; [email-masked]
Subject: Re: [marf] Including Mail fields in IODEF

The issue with MARF inside IODEF is that the receiver needs to know that the payload being provided inside an EmailMessage element is itself an ARF report, and not the message that caused the report in the first place.  You certainly could crack open the EmailMessage content and see if conforms to the ARF specification to tell which kind of report you've gotten, but that seems inelegant.
I suppose then another option is an extension element that indicates you've received an ARF payload rather than the actual offending message.

Also of note: An ARF can contain the offending message or only the offending message's header, and still be compliant.  If your application needs the whole message, you'll have to add some additional stipulations someplace.
-MSK

On Fri, Mar 1, 2013 at 1:52 PM, Panos Kampanakis (pkampana) <[email-masked]<mailto:[email-masked]>> wrote:
I think MARF provides more functionality and should be leverage for emails in IODEF.
I also think we need to resurrect http://tools.ietf.org/html/draft-vesely-mile-mail-abuse-00 within MILE since MARF was concluded..
Panos

-----Original Message-----
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Moriarty, Kathleen
Sent: Thursday, February 21, 2013 5:19 AM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [mile] Including Mail fields in IODEF

Hello,

Thanks,
Kathleen
________________________________________
From: Harrington, Christopher
Sent: Wednesday, February 20, 2013 2:57 PM
To: Moriarty, Kathleen; [email-masked]<mailto:[email-masked]>
Subject: RE: Mail fields

Thanks,

--Chris

-----Original Message-----
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Moriarty, Kathleen
Sent: Wednesday, February 20, 2013 2:58 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [mile] Mail fields

Hi,

Thanks,
Kathleen
_______________________________________________
mile mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mile
_______________________________________________
mile mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mile
_______________________________________________
marf mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/marf

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 3 Mar 2013 19:46:26 -0800

Hi Panos,

The "feedback-type" would be part of the ArfHeader object, I would
imagine.  It appears immediately before the portion of the example you
cited.

This might also be a viable way to add ARF capability to IODEF, though I
don't think that was the original problem statement (which was only to
include DKIM and SPF details).

At any rate, I don't think you're reading it wrong.

-MSK

On Sun, Mar 3, 2013 at 2:37 PM, Panos Kampanakis (pkampana) <
[email-masked]> wrote:

>  Thank you Murray.****
>
> ** **
>
> The “<arf:EmailMessage>****
>
>    Received: from mailserver.example.net****
>
>         (mailserver.example.net [192.0.2.1])****
>
>         by example.com with ESMTP id M63d4137594e46;****
>
>         Thu, 08 Mar 2005 14:00:00 -0400****
>
>    From: &lt;[email-masked]&gt;****
>
>    To: &lt;Undisclosed Recipients&gt;****
>
>    Subject: Earn money****
>
>    MIME-Version: 1.0****
>
>    Content-type: text/plain****
>
>    Message-ID: [email-masked]****
>
>    Date: Thu, 02 Sep 2004 12:31:03 -0500****
>
> ** **
>
>    Spam Spam Spam****
>
>    Spam Spam Spam****
>
>    Spam Spam Spam****
>
>    Spam Spam Spam****
>
>          </arf:EmailMessage>”****
>
> that I see in http://bgp.potaroo.net/ietf/all-ids/draft-vesely-mile-mail-abuse-00.txt looks like just an email message. I don’t see “feedback-type" or other ARF fields for example that would make it a ARF.****
>
> ** **
>
> draft-vesely-mile-mail-abuse-00.txt seems to define a header and then have the option for the actual message (EmailMessage). Am I reading it wrong?****
>
> ** **
>
> Panos****
>
> ** **
>
> ** **
>
> ** **
>
> *From:* Murray S. Kucherawy [mailto:[email-masked]]
> *Sent:* Sunday, March 03, 2013 4:10 AM
> *To:* Panos Kampanakis (pkampana)
> *Cc:* Moriarty, Kathleen; [email-masked]; [email-masked]
> *Subject:* Re: [marf] Including Mail fields in IODEF****
>
> ** **
>
> seems inelegant.****
> I suppose then another option is an extension element that indicates
> you've received an ARF payload rather than the actual offending message.
>
> someplace.****
> -MSK****
> ** **
> [email-masked]> wrote:****
>
> Panos****
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Moriarty, Kathleen****
>
> https://www.ietf.org/mailman/listinfo/marf****
>
> ** **
>

---

From: "Moriarty, Kathleen" <[email-masked]>
Date: Tue, 5 Mar 2013 23:57:56 -0500

Hi Murray,

I may not have been clear enough in my first message.  I was asking about including ARF to represent any/all mail fields useful in exchanges by including ARF.  The option of using RFC5901 with the full mail message would work as well (as you pointed out).  If ARF has wider adoption, it may make sense to use existing standards and tools to accomplish the task by embedding ARF in IODEF.  The format covers some data type representations not in IODEF, but specific to mail (which makes sense to extend when you get that deep).

Thanks,
Kathleen

Sent: Sunday, March 03, 2013 10:46 PM

Hi Panos,
The "feedback-type" would be part of the ArfHeader object, I would imagine.  It appears immediately before the portion of the example you cited.

This might also be a viable way to add ARF capability to IODEF, though I don't think that was the original problem statement (which was only to include DKIM and SPF details).
At any rate, I don't think you're reading it wrong.
-MSK

On Sun, Mar 3, 2013 at 2:37 PM, Panos Kampanakis (pkampana) <[email-masked]<mailto:[email-masked]>> wrote:
Thank you Murray.

The "<arf:EmailMessage>
   Received: from mailserver.example.net<http://mailserver.example.net>
        (mailserver.example.net<http://mailserver.example.net> [192.0.2.1])
        by example.com<http://example.com> with ESMTP id M63d4137594e46;
        Thu, 08 Mar 2005 14:00:00 -0400
   From: &lt;[email-masked]<mailto:[email-masked]>&gt;
   To: &lt;Undisclosed Recipients&gt;
   Subject: Earn money
   MIME-Version: 1.0
   Content-type: text/plain
   Message-ID: [email-masked]<mailto:[email-masked]>
   Date: Thu, 02 Sep 2004 12:31:03 -0500

   Spam Spam Spam
   Spam Spam Spam
   Spam Spam Spam
   Spam Spam Spam
         </arf:EmailMessage>"

Panos

From: Murray S. Kucherawy [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Sunday, March 03, 2013 4:10 AM
To: Panos Kampanakis (pkampana)
Cc: Moriarty, Kathleen; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [marf] Including Mail fields in IODEF

-----Original Message-----
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Moriarty, Kathleen
Sent: Thursday, February 21, 2013 5:19 AM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [mile] Including Mail fields in IODEF

Hello,

Thanks,
Kathleen

Thanks,

--Chris

-----Original Message-----
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Moriarty, Kathleen
Sent: Wednesday, February 20, 2013 2:58 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [mile] Mail fields

Hi,

Thanks,
Kathleen