Subject: [mile] IODEFv2 suggestions

From: Lagadec Philippe <[email-masked]>
Date: Fri, 25 Jul 2014 13:35:17 +0000

Hello,

A few years ago I studied IODEFv1 for some projects and wrote a small parser (iodeflib) to learn more about IODEF.
At the time I noted a few issues in the RFC 5070 specification (especially in the context of incident sharing between governmental organizations), but I only discovered the work on IODEFv2 very recently. 
So here is a list of suggestions that you might consider for IODEFv2. Some may have already been addressed in the current draft or in the TRAC system.

1. in RFC5070, Impact.type section 3.10.1 page 28 has a default value of "other", but that value is not in the enum list nor in the XML schema. In the XML schema, the default value is "unknown".  This is still the case in the IODEFv2 draft-07.

2. DATETIME (RFC5070 section 2.8): it is not clear to me whether RFC3339 allows timestamps without explicit timezone (i.e. expressed in local time without UTC offset). Maybe it would be useful to mention in the IODEF specifications that an explicit timezone is mandatory (either "Z" or UTC time offset) in every timestamp, for interoperability reasons?

3. HistoryItem.action (RFC5070 section 3.11.1): the description says it is the same enumeration as Expectation.category. However, Expectation has no category attribute, so I think it should be "action". Same issue in IODEFv2 draft-07.

4. The Incident class does not have an attribute to specify a title or a short description for the incident. I think that would be useful when displaying a list of incidents.

5. I have not found any field to describe the current status of the incident, e.g. if it is open, closed, being managed, etc.

6. Also I have not found an attribute to specify if the incident is under investigation, confirmed to be an actual incident, or confirmed to be a false positive. (which might be part of the incident status)

7. The IODEF-Document class does not have attributes such as description, name/title, timestamp to describe the purpose of the overall package which may contain one or several incidents. I think it might be useful to add a few general attributes like this at the root level. IODEFv2 adds AdditionalData which was missing in IODEFv1, that is very useful to store things like XML-DSIG.

8. Expectation class: I think the severity attribute should be named "priority" to better match its purpose. 

9. The Incident class has no timestamp to specify when the IODEF content was created (i.e. when the incident report was created), and more important, when the IODEF Incident content was modified/updated for the last time. I think this is necessary when several versions of an IODEF incident are exchanged over time, it would help to know which version is the most recent and supersedes the previous ones. I think we need a modification timestamp at the Incident level, to be set whenever any field changes in the whole Incident tree. It might also be useful to have an optional revision number, for some use cases. 

10. I think it would be useful to have AdditionalData available in every class, because it is difficult to foresee all future and custom use cases. For example, there is no AdditionalData in the Impact classes, in Confidence, in the subclasses of Contact, in Expectation, Flow, etc. There is AdditionalData already in many classes, so I do not think it is too complicated to handle it everywhere for IODEF applications.

11. For governmental usage (and possibly for any company/organization with similar security schemes), it would be very useful to have an attribute in the Incident class to specify the security classification level of the Incident report (e.g. "NATO Unclassified" or "FOUO" - see http://en.wikipedia.org/wiki/Classified_information). It may be as simple as a string, or something more structured (an enumeration covering all cases would be too difficult). It would also be useful to have the same attribute in the System, Node and Record classes.

12. When displaying incidents on a geographical map, it is useful to have a way to specify a geographical location for the incident itself. When there is only one target or when all targets are in the same physical place, then it is quite straightforward to use that location. When targets are in different places, then you may display several icons for one incident. In my opinion, it would be useful to have an attribute in the Incident class itself to specify one or several geographical locations where it happened, when applicable. A geographical location should be expressed either as a postal address (as proposed in the TRAC issue #11), or as latitude/longitude. There are cases when no specific address is relevant, or when an application does not have a proper database to convert postal addresses to coordinates for display. Then latitude/longitude should be an option. Unless there is already an IETF standard for geographical coordinates, it would be possible to use something like OGC KML "kml:coordinatesType" (see http://portal.opengeospatial.org/files/?artifact_id=27810 page 210). It could look like this:
<Incident>
  <GeoLocations>
     <coordinates>-74.006393,40.714172,0</coordinates>
     <coordinates>-122.0822035425683,37.42228990140251</coordinates>
     <PostalAddress>15 Main St.$Ottawa$Canada</PostalAddress>
  </GeoLocations>
</Incident>

13. Then it would make sense to add the possibility to specify geographical coordinates for a Node, a Contact, maybe System and others.

14. PostalAddress.meaning is defined as enum, but it is actually a free-text string.

15. The definition of ML_STRING in the text (RFC5070 sections 2.4 and 6) is quite vague and no examples are provided, although ML_STRING is used in many places. Only the XML schema shows clearly that ML_STRING can have a "lang" attribute. Then by analyzing section 6, it can be inferred that there are three main cases:
- case 1: a single ML_STRING, using the same language as declared in IODEF_Document or Incident. Then the lang attribute is not required in the ML_STRING.
- case 2: a single ML_STRING, using a different language from IODEF_Document or Incident. Then according to section 6, I think the ML_STRING.lang attribute is required.
- case 3: several ML_STRINGs in the same object (e.g. Incident.Description). According to section 6, these ML_STRINGs should have the same content expressed in different languages. Two ML_STRINGs cannot have the same language, but they must have the same meaning. So only one can have no lang attribute, all the others must have the lang attribute with different languages. 
Example:
<Incident lang="en">
  <Description>Denial of service attack on www.acme.com</Description>
  <Description lang="fr">Attaque en deni de service sur www.acme.com</Description>
</Incident>

I think this should be clarified in the definition of ML_STRING. And this definition should point to the section about Internationalization Issues for background explanations.

16. Last but not least, IODEFv1 does not provide a native way to specify the incident type. We can use Impact.type to describe the type of impact, but that is different from the type of the incident itself. For example a denial of service can be caused by different types of incidents (e.g. a DDoS, the exploitation of a vulnerability, a worm, a surge in legitimate requests, a physical outage, a unhandled bug, etc).
Even if it is hard to find a proper/generic way to categorize incidents that everybody agrees upon, I think it would be good to add an attribute with an enumeration for it in the Incident class. 
Potential sources of inspiration:
https://www.us-cert.gov/government-users/reporting-requirements
http://veriscommunity.net/enums.html#section-actions
https://alienvault.bloomfire.com/posts/653702-open-source-security-event-taxonomy/public
https://capec.mitre.org/about/index.html (although this might be too detailed)
And surely many others.
I think it should be possible to assign several categories to an incident, because it is often a combination of several types (e.g. malware and social engineering).

Regards,

Philippe Lagadec.

---

From: "Roman D. Danyliw" <[email-masked]>
Date: Wed, 30 Jul 2014 02:19:06 +0000

Hello Lagadec!

Thank you for your detailed feedback!  An issue by issue response is inline.

> -----Original Message-----
> From: mile [mailto:[email-masked]] On Behalf Of Lagadec Philippe
> Sent: Friday, July 25, 2014 9:35 AM
> To: [email-masked]
> Subject: [mile] IODEFv2 suggestions
> 
> Hello,
> 

I'm sure the community would like to hear about it.  Perhaps your experience can be added to the implementation report, draft-ietf-mile-implementreport-00.

> At the time I noted a few issues in the RFC 5070 specification (especially in
> the context of incident sharing between governmental organizations), but I
> only discovered the work on IODEFv2 very recently.
> So here is a list of suggestions that you might consider for IODEFv2. Some
> may have already been addressed in the current draft or in the TRAC system.
> 
> 1. in RFC5070, Impact.type section 3.10.1 page 28 has a default value of
> "other", but that value is not in the enum list nor in the XML schema. In the
> XML schema, the default value is "unknown".  This is still the case in the
> IODEFv2 draft-07.

That's a typo.  The text should be consistent with the schema and set to "unknown".  It will fixed in -08.

> 2. DATETIME (RFC5070 section 2.8): it is not clear to me whether RFC3339
> allows timestamps without explicit timezone (i.e. expressed in local time
> without UTC offset). Maybe it would be useful to mention in the IODEF
> specifications that an explicit timezone is mandatory (either "Z" or UTC time
> offset) in every timestamp, for interoperability reasons?

In rereading RFC5070 Section 2., RFC 3339, and the xs:dateTime definition I too see the possibility for confusion.  My read is that UTC-offsets are optional in xs:dateTime, but RFC 3339 references a subset of ISO 8601 and allows only a "date-time" format (per Section 5.6) with a UTC-offset.  As you point out, this distinction isn't very clear in the text but I think that the end result is that UTC offsets are required.

I think we have a few options:
(1) Provide clarifying text in the DATETIME definition saying that this format is a "date-time" (which requires an offset) as defined by RFC3339 Section 5.6 , and removing the reference to ISO 8601.
(2) Everything in #1 but then enforce this in the schema 
(3) Relax the requirement for UTC offset and not reference RFC3339, and only use the xs:dateTime definition (also derived from ISO 8601).  Then we need to decide how timestamps without an offset should be interpreted -- as UTC, local time of the receiver, or guess the local time of the sender?

> 3. HistoryItem.action (RFC5070 section 3.11.1): the description says it is the
> same enumeration as Expectation.category. However, Expectation has no
> category attribute, so I think it should be "action". Same issue in IODEFv2
> draft-07.

That's a typo.  The HistoryItem@action text reference Expecation@action.  It will fixed in -08.

> 4. The Incident class does not have an attribute to specify a title or a short
> description for the incident. I think that would be useful when displaying a list
> of incidents.

Would using multiple Incident/Description elements as is currently permitted be sufficient?

> 5. I have not found any field to describe the current status of the incident,
> e.g. if it is open, closed, being managed, etc.

Agreed.  This short coming was tracked in Issue #46 and the discussion thread is at http://www.ietf.org/mail-archive/web/mile/current/msg01377.html.  

Not all of the suggestions in Issue #46 were commented on yet, so I'll take your feedback as an endorsement at least part of this list.  If you have suggestions on a taxonomy, the current proposal considers two different taxonomies because two separate attributes might be needed to distinguish between the state of the incident and how it is being handled.

** incident-handling-status = New/In Progress/Ended/Resolved/Unknown
** incident-status = Occurring/Contained/Concluded/Future Threat

> 6. Also I have not found an attribute to specify if the incident is under
> investigation, confirmed to be an actual incident, or confirmed to be a false
> positive. (which might be part of the incident status)

Any recommendations on combining this with the above lists, or do you feel a separate attribute is required?

> 7. The IODEF-Document class does not have attributes such as description,
> name/title, timestamp to describe the purpose of the overall package which
> may contain one or several incidents. I think it might be useful to add a few
> general attributes like this at the root level. IODEFv2 adds AdditionalData
> which was missing in IODEFv1, that is very useful to store things like XML-
> DSIG.

To clarify, you'd like to add IODEF-Document/{Description, ReportTime, @purpose}.  Anything else?

This begs the question, does IODEF-Document need a unique identifier scheme like Incident/IncidentID or Incident/Indicator/IndicatorID?

> to better match its purpose.

Upon rereading, you're right, Expectation@severity seems misnamed.

To the community: does the clarity in renaming Expectation@severity to @priority worth the backward compatibility cost?

> 9. The Incident class has no timestamp to specify when the IODEF content
> was created (i.e. when the incident report was created), and more
> important, when the IODEF Incident content was modified/updated for the
> last time. I think this is necessary when several versions of an IODEF incident
> are exchanged over time, it would help to know which version is the most
> recent and supersedes the previous ones. I think we need a modification
> timestamp at the Incident level, to be set whenever any field changes in the
> whole Incident tree. It might also be useful to have an optional revision
> number, for some use cases.

Issue #46, sub-bullet #4 is tracking a discussion (http://www.ietf.org/mail-archive/web/mile/current/msg01377.html) to add reporting time vs. an IODEF-Document generation time.  Does this generation time address your need for a "timestamp to specify when the IODEF content was created"?

This past Fall we discussed how to deal with update semantics at http://www.ietf.org/mail-archive/web/mile/current/msg01196.html.  The result was a lack of interest due to the perceived complexity.  Is your suggestion to add a version number to the Incident class whereby every time an updated instance is sent out this version number is incremented?  The originating CSIRT would keep the associated state.
 
> 10. I think it would be useful to have AdditionalData available in every class,
> because it is difficult to foresee all future and custom use cases. For example,
> there is no AdditionalData in the Impact classes, in Confidence, in the
> subclasses of Contact, in Expectation, Flow, etc. There is AdditionalData
> already in many classes, so I do not think it is too complicated to handle it
> everywhere for IODEF applications.

I concur that AdditionalData is added to certain classes but others lack it for no apparent reason.  It's worth thinking through where else to add it.  I have some reservation to adding AdditionalData to all classes.  Introducing a child elements to those classes that currently have simple element content (e.g., Confidence) would require a lot of redesign. 

I think that all aggregate classes (those that have child classes) would be candidates for consideration.  These are all the aggregate classes (i.e., have child classes) that don't have AdditionalData:

** RelatedActivity
** ThreatActor
** Campaign
** History
** HistoryItem
** Expectation
** Discovery
** DetectionPattern
** Reference
** Assessment
** Flow
** Node
** Service
** EmailData
** DomainData
** NameServers
** DomainContacts
** Record
** RecordData
** WindowsRegistryKeysModified
** Key
** HashData
** IndicatorData
** Indicator
** IndicatorExpression

Any thoughts from the community?

> 11. For governmental usage (and possibly for any company/organization with
> similar security schemes), it would be very useful to have an attribute in the
> Incident class to specify the security classification level of the Incident report
> (e.g. "NATO Unclassified" or "FOUO" - see
> http://en.wikipedia.org/wiki/Classified_information). It may be as simple as
> a string, or something more structured (an enumeration covering all cases
> would be too difficult). It would also be useful to have the same attribute in
> the System, Node and Record classes.

Would an xs:string attribute in Incident, EventData, and IndicatorData be sufficient?  If a System, Node, or Record had a different  classification, it could be nested appropriately with an EventData.  Or would this attribute be required everywhere a @restriction is currently found?

> 12. When displaying incidents on a geographical map, it is useful to have a
> way to specify a geographical location for the incident itself. When there is
> only one target or when all targets are in the same physical place, then it is
> quite straightforward to use that location. When targets are in different
> places, then you may display several icons for one incident. In my opinion, it
> would be useful to have an attribute in the Incident class itself to specify one
> or several geographical locations where it happened, when applicable. A
> geographical location should be expressed either as a postal address (as
> proposed in the TRAC issue #11), or as latitude/longitude. There are cases
> when no specific address is relevant, or when an application does not have a
> proper database to convert postal addresses to coordinates for display. Then
> latitude/longitude should be an option. Unless there is already an IETF
> standard for geographical coordinates, it would be possible to use
> something!
>   like OGC
>   KML "kml:coordinatesType" (see
> http://portal.opengeospatial.org/files/?artifact_id=27810 page 210). It could
> look like this:
> <Incident>
>   <GeoLocations>
> 13. Then it would make sense to add the possibility to specify geographical
> coordinates for a Node, a Contact, maybe System and others.

We discussed adding location information in the Fall survey as question #9(d) (http://www.ietf.org/mail-archive/web/mile/current/msg01189.html).  The subsequent discussion was scoped to adding geographic location information to a given Node.  At the time there was insufficient interest in pursuing a change.  To the broader group, should we reopen this decision now that there is interest?  I think so.

If we do adopt this change, the KML example is illustrative, but due to intellectual property issues, it's unlikely we could take it outright.

I need help understanding item #12.  Is the intent to represent location information (Postal Address/Coordinates) in addition to what might be described in the Node class?  Would the location information in the Incident ever be different than at the Node, or would this support a use case where no Node-level location information is provided?

If the location information was in Node, do you have an illustrative use case where it be would require it in System as well?

I concur that if we add geographic coordinates to Node it makes sense to also be in Contact.

> 14. PostalAddress.meaning is defined as enum, but it is actually a free-text

That's a typo.  It will be fixed in -08.

> 15. The definition of ML_STRING in the text (RFC5070 sections 2.4 and 6) is
> quite vague and no examples are provided, although ML_STRING is used in
> many places. Only the XML schema shows clearly that ML_STRING can have a
> "lang" attribute. Then by analyzing section 6, it can be inferred that there are
> three main cases:
> - case 1: a single ML_STRING, using the same language as declared in
> IODEF_Document or Incident. Then the lang attribute is not required in the
> ML_STRING.
> - case 2: a single ML_STRING, using a different language from
> IODEF_Document or Incident. Then according to section 6, I think the
> ML_STRING.lang attribute is required.
> - case 3: several ML_STRINGs in the same object (e.g. Incident.Description).
> According to section 6, these ML_STRINGs should have the same content
> expressed in different languages. Two ML_STRINGs cannot have the same
> language, but they must have the same meaning. So only one can have no
> lang attribute, all the others must have the lang attribute with different
> languages.
> Example:
> <Incident lang="en">
>   <Description lang="fr">Attaque en deni de service sur
> www.acme.com</Description> </Incident>
> 
> I think this should be clarified in the definition of ML_STRING. And this
> definition should point to the section about Internationalization Issues for
> background explanations.

I completely agree that ML_STRING is underspecified.  Outside internalization review has also stated that the current approach is problematic.  We'll add this feedback to the internationalization review that will be occurring in August.

> 16. Last but not least, IODEFv1 does not provide a native way to specify the
> incident type. We can use Impact.type to describe the type of impact, but
> that is different from the type of the incident itself. For example a denial of
> service can be caused by different types of incidents (e.g. a DDoS, the
> exploitation of a vulnerability, a worm, a surge in legitimate requests, a
> physical outage, a unhandled bug, etc).
> Even if it is hard to find a proper/generic way to categorize incidents that
> everybody agrees upon, I think it would be good to add an attribute with an
> enumeration for it in the Incident class.
> https://capec.mitre.org/about/index.html (although this might be too
> detailed) And surely many others.
> I think it should be possible to assign several categories to an incident,
> because it is often a combination of several types (e.g. malware and social
> engineering).

Agreed.  Issue #46, sub-bullet #2 is tracking that some approach to categorize the incidents is necessary.  Your point is well taken that whatever taxonomy or free-form approach that is taken, multiple categories for a given incident should be possible.

> Philippe Lagadec.

Again, thank you for this thorough review!

Roman

> _______________________________________________
> mile mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mile