Subject: Re: [sacm] Comments to SACM Vulnerability Assessment Scenario

From: "Wolfkiel, Joseph L CIV DISA ID (US)" <[email-masked]>
Date: Mon, 7 Dec 2015 22:21:31 +0000

Looks like I left off the SACM e-mail accidently.  Adding them back in and including response to your last question.

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52
Fort Meade DISA Acquisiton Bldg Cube A4A58E
Work: (301) 225-8820
Gov Cell: (571) 814-8231
[email-masked]

-----Original Message-----
From: Haynes, Dan [mailto:[email-masked]] 
Sent: Monday, December 07, 2015 1:14 PM
To: Wolfkiel, Joseph L CIV DISA ID (US)
Subject: RE: [Non-DoD Source] [sacm] Comments to SACM Vulnerability Assessment Scenario

Thanks Joe.  More comments inline below.

-Danny

 -----Original Message-----
From: Wolfkiel, Joseph L CIV DISA ID (US) [Caution-mailto:[email-masked]] 
Sent: Monday, December 07, 2015 8:38 AM
To: Haynes, Dan <[email-masked]>
Subject: RE: [Non-DoD Source] [sacm] Comments to SACM Vulnerability Assessment Scenario

Here are responses back to your replies.  In some of your responses, you provided explanations.  If you believe that others may misunderstand or be confused by verbiage, it may be better to clarify the document instead.

Specific responses follow:

[danny]: Seems like a reasonable suggestion.  We could either remove the "basic" qualifier or separate them out into two different lists ("basic" and "advanced").  Do you have a preference?

[joe] My preference is to start the discussion on what "basic" means, so if some of the data is basic and some isn't, then I like the idea of dividing it into two lists.  My opinion is that SACM will ultimately need to define some minimum required data set to be collected for interoperability and device identification/characterization, so it's a good concept to address in the use case documents. That is, of course, assuming that "minimum required" and "basic" are synonymous.  I'm struggling to understand what you mean when you say "basic".

[danny]:  Thanks for the additional information.  You are absolutely correct in your opinion.  SACM does need to define some set of mandatory-to-implement information to ensure interoperability.  This information should and will be captured in the information model and will drive the development of data models (or at least data models will be required to support the information needs in the information model).

*****************

 Comment: Recommend consideration of placing attributes on the endpoints so they can be discovered by multiple sensors and also viewed and edited locally.  This would enhance accuracy and ensure a common assessment by all sensors collecting data from an endpoint.

[danny]: I could potentially see this as being an implementation detail.  Can your provide a little more information on how you envision this working?  Would organizations just place whatever human-assigned data they want on the endpoint wherever they want?  Would there be mandated data models and locations where the human-assigned information would be stored?  I am just trying to think through the situation as this information would likely be organization specific and unless an organization is developing its own collection tools there is a good chance COTs collection tools would not know about this information nor how to collect it.  Anyways, I think we can add some more text to discuss this topic once we have a better understanding of what SACM needs here.

[joe] I agree this is an immature topic.  I've been doing some design work in this area so my organization can move this way in the future.  Current thought is that the data elements you need to "tag" devices with characterization data include (for each characteristic), the URI of the characteristic definition (e.g. "http://ietf.org/data_sensitivity), the value selected  (e.g. "medium") or the ID of the value selected (e.g. "3"), the timestamp of when the attribute was assigned, and the status of the attribute (true=active, false=deprecated).

Here's an example using the DoD ARF "taggedString" format.  This is not a recommendation or requirement that SACM use XML as transport, just an example of how we're documenting device attributes in the DoD ARF implementation.

<ns01:taggedString name="https://ietf.org?name="data_sensitivity" value="3"  timestamp="2015-05-23T01:42:05.000-04:00" source="jwolfkiel" status="true">

[danny]: This example is very helpful.  Just curious, how does a tool know where to find this tag?  Are they all stored in a standard location?  Does the location vary depending on the type of tag?  Does it require the tool to do an exhaustive search to find the tags on the system?

[joe] We're planning to designate specific registry hives and keys for microsoft and/or file locations in the "..ix" file systems to store the tag values.  I would expect to put all the tags in the same logical location.

 ***********************

Comment: Why is this list constrained to TCP and UDP?  I would expect GRE, IPsec, and other protocols to also be of interest

 [danny]: This wasn't intended to be an exhaustive list.  TCP and UDP were just provided as common examples.

[joe]: For lists of examples, it's traditional to include the "e.g." qualifier, which will clarify that the list is a set of examples, not an exhaustive definition.

 [danny]:  Sounds good.  We can make this update.

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52
Fort Meade DISA Acquisiton Bldg Cube A4A58E
Work: (301) 225-8820
Gov Cell: (571) 814-8231
[email-masked] < Caution-mailto:[email-masked] > 

-----Original Message-----

From: Haynes, Dan [Caution-mailto:[email-masked] < Caution-mailto:[email-masked] > ] 

Sent: Sunday, December 06, 2015 8:47 PM

To: Wolfkiel, Joseph L CIV DISA ID (US); [email-masked] < Caution-mailto:[email-masked] > 

Subject: RE: [Non-DoD Source] [sacm] Comments to SACM Vulnerability Assessment Scenario

All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser. 

________________________________

 

Hi Joe,

Thanks for the feedback!

Comments inline below.  Let me know if I am misunderstanding anything.

Thanks,

Danny

-----Original Message-----

From: Wolfkiel, Joseph L CIV DISA ID (US) [Caution-Caution-mailto:[email-masked]] 

Sent: Thursday, November 19, 2015 4:43 PM

To: [email-masked] < Caution-mailto:[email-masked] > 

Cc: Haynes, Dan <[email-masked] < Caution-mailto:[email-masked] > >

Subject: RE: [Non-DoD Source] [sacm] Comments to SACM Vulnerability Assessment Scenario

Here are my comments to the scenario document:

Section 1, 2nd paragraph: " whether or not the flaw is present within an enterprise, when

compared to information about the state of the enterprise's   endpoints"

Comment: Does it say which vulnerabilities were discovered on what software on which endpoints?  There's not a definition or example of what "vulnerability report" means.

[danny]: The input vulnerability report would likely tell you what software, version, on which platform was affected and any other applicable data for any endpoint.  After the assessment, you will know which vulnerabilities in which software on which endpoints.

Section 2, bulleted list:

"The document begins with the assumption that the enterprise has

      received a vulnerability report , and that the report data has

      already been processed into a format that the enterprise's

      security software tools can understand and use.  In particular,

      this document:"

Comment: The text alludes to a requirement that the report be machine readable, but never states it.

[danny]: The document does not specify whether or not the vulnerability report is in a machine-readable format just that the necessary information has been extracted and put in a format tool's can use.  With that said, I think it would be great if there was some machine-readable format for vulnerability reports.

Section 2, bulleted list:

"Does not discuss how the enterprise identifies a potentially

         relevant  vulnerability report."

Comment: It's not clear what a "potentially relevant" report is.

[danny]: This refers to the initial process of determining which vulnerability reports are applicable to an enterprise and assumes:

    * an enterprise obtains applicable vulnerability reports in some manner as they become available

    * non-applicable vulnerability reports are filtered out in some fashion

    * this is done prior to this scenario beginning

Given this, we should update the assumption to state "The document begins with the assumption that the enterprise has received a vulnerability report that is applicable to their environment, and the report data has already.".

Section 2, bulleted list:

" Does not discuss how the enterprise collects that report"

Comment: Does this address how the report was acquired, or how the data in the report was acquired?

[danny]: No, it does not address how the enterprise collects the vulnerability report from the publishing vendor.

Section 2, bulleted list:

   o  The document assumes that the enterprise has a means of

      identifying enterprise endpoints.  This could mean identifying

     endpoints as they join the network, actively scanning for

      connected endpoints, passive scanning of network traffic to

      identify connected endpoints, or some other method of accounting

     for the presence of all endpoints in the enterprise.

Comment: Does the statement about endpoint identification imply that the report contains data about endpoints?

[danny]: No, this just means the enterprise has some way to know what endpoints are on its network.  The vulnerability report only provides information about what software and configurations result in a vulnerable state.  The vulnerability report could then be used as input into an assessment to determine if it is in fact vulnerable.

Section 2, bulleted list:

*  Does not specify how extracted endpoint information and

         vulnerability alert information are normalized to be

         compatible.

Comment: What are alerts normalized to be compatible with?

[danny]: Compatible here just means that the information about an endpoint as well as the relevant data from the vulnerability report are normalized in some way such that the data is comparable and an assessment can be performed.

Section 2, bulleted list:

o  The document assumes that the enterprise has a policy by which

      assessment of endpoints based on vulnerability reports  are

      prioritized.  The document:

Comment: It appears that an organization will "assess" something on receipt of a vulnerability report.  What does it assess?

[danny]: Yes. Based on the vulnerability report, an enterprise will determine if the vulnerability report is applicable to an endpoint (not to be confused with applicable to the enterprise during initial report retrieval) and then carry out an assessment. The priority of that assessment would be determined by the enterprise policy.  The priority would likely affect the timeliness of the assessment, and even how often any reassessments occur.  The priority of the assessments is given a nod in the section titled Priority, but is not discussed as part of the overall scenario.

Section 3, first paragraph:

"The first step in this scenario involves identifying endpoints and

collecting basic system information from them.  This occurs prior to

   the receipt of any specific vulnerability report and is part of the

   regular, ongoing monitoring of endpoints within an enterprise.  This

   process is not meant to report on, or gather data for any specific

   vulnerabilities.  The information gathered during this step could be

   applied in many enterprise automation efforts.  Specifically, in

   addition to vulnerability management, it could be used by

   configuration and license management tasks.  All of the information

   collected during this step is stored in a central location such as a

  CMDB ."

Comment: Please spell out "CMDB" on first use and provide a reference that explains what the authors think a CMDB would contain.  A configuration management database (CMDB) isn't necessarily assumed to contain a comprehensive device inventory.  Arguably, it stores baseline configurations and deviations to them.

[danny]: We just used this as a general place where we can store/retrieve collection data (including inventory), guidance, assessment results (including baseline configurations and deviations), etc.  It sounds like we may need to provide a little more context on this.  We may also want to call it something other than CMDB to avoid any of these assumptions (e.g. repository with data stores which can hold all this information).

Section 3.1, first paragraph:

Unique endpoint IDs are used to

   allow for systems to be tracked over time  and also allow for proper

   counts of assets during inventories and other similar collections.

Comment: Is this the scope of how/what endpoint IDs are used for?  Are IDs used to correlate information between sensors?

[danny]: This text was not meant to exclude between sensors.  It would be fair to include across sensors as well.

Section 3.1, second paragraph:

This sub-step aligns with the Endpoint Discovery, Endpoint

  Characterization (automated collection data only) , and Endpoint

   Target Identification building block capabilities.

Comment: Please provide rationale on why manually collected data is prohibited

[danny]: At the moment, I don't recall exactly why we included that restriction :).  Let me find out and follow up on this one.

Section 3.3, bulleted list:

Suggest changing " Endpoint type- e.g., standard computer, printer, router, mobile

      device, tablet, etc." to read " Endpoint function or general role - e.g., standard computer, printer, router, mobile

     device, tablet, etc."

Comment: "type" is weakly defined and ambiguous.

[danny]: Another way we might want to classify endpoint type is by traditional (workstation, server, etc.), mobile (cell phones, laptops, tablets, etc.), network infrastructure (switches, routers, etc.), and constrained (IoT, ICS, printer, etc.).

Section 3.3, bulleted list:

o  Hardware version/firmware

Comment: version or firmware of what?  The BIOS, cellular modem, hard drive?

[danny]: We did not specify anything in particular.  It was just an example of the type of information an enterprise might want to collect about an endpoint in its pre-assessment data collection activities.  An enterprise could potentially collect all of the information you listed above.  With that said, we think it would be fair to include BIOS versions and firmware revisions to make it a little more clear.  The main point here, however, is that the scenario is only talking about basic information about the endpoint.

Section 3.3, bulleted list:

"o  Open ports/running services 

   o  Operating system optional component inventory - some OS' have

      optional components that can be installed which may not show up as

      separate pieces of software (e.g., web and ftp servers, demo web

      pages, shared libraries, etc.).  Note that this could also occur

      within third-party applications as well. "

Comment:  Suggest that neither of these bullets describes "basic" data, since they are long, detailed lists that aren't specifically required to identify or characterize a device.

[danny]: Seems like a reasonable suggestion.  We could either remove the "basic" qualifier or separate them out into two different lists ("basic" and "advanced").  Do you have a preference?

Section 3.3, bulleted list:

"Human-assigned endpoint attributes are yet another type of data item

   that could be collected here.  This would include any attributes that

   cannot be collected programmatically.  The attributes could be

   manually entered into a CMDB by a human after the initial data

   collection occurs.   The attributes could help with system

   categorization and may have influence on later prioritizations.  The

   human-assigned attributes could include:"

Comment: Recommend consideration of placing attributes on the endpoints so they can be discovered by multiple sensors and also viewed and edited locally.  This would enhance accuracy and ensure a common assessment by all sensors collecting data from an endpoint.

[danny]: I could potentially see this as being an implementation detail.  Can your provide a little more information on how you envision this working?  Would organizations just place whatever human-assigned data they want on the endpoint wherever they want?  Would there be mandated data models and locations where the human-assigned information would be stored?  I am just trying to think through the situation as this information would likely be organization specific and unless an organization is developing its own collection tools there is a good chance COTs collection tools would not know about this information nor how to collect it.  Anyways, I think we can add some more text to discuss this topic once we have a better understanding of what SACM needs here.

Section 3.3, 5th paragraph:

" There is direct alignment with the Endpoint Component

   Inventory, Posture  Attribute Identification, and Posture Attribute

   Value Collection building block capabilities since the purpose of

this sub-step is to perform an initial inventory of the endpoint and

   collect basic attributes and their values."

Comment: Please define "posture"

[danny]: From the Terminology document (Caution-Caution-https://github.com/sacmwg/draft-ietf-sacm-terminology/blob/master/draft-ietf-sacm-terminology.md < Caution-Caution-https://github.com/sacmwg/draft-ietf-sacm-terminology/blob/master/draft-ietf-sacm-terminology.md > ), "posture" is defined in the terminology document as:

"Defined in {{RFC5209}} as "configuration and/or status of hardware or software on an endpoint as it pertains to an organization's security policy." This term is used within the scope of SACM to represent the configuration and state information that is collected from a target endpoint in the form of endpoint attributes (e.g. software/hardware inventory, configuration settings, dynamically assigned addresses). This information may constitute one or more posture attributes."

Also, "posture attribute" is defined as:

"Defined in {{RFC5209}} as "attributes describing the configuration or status (posture) of a feature of the endpoint. A Posture Attribute represents a single property of an observed state. For example, a Posture Attribute might describe the version of the operating system installed on the system."  Within this document this term represents a specific assertion about endpoint configuration or state (e.g. configuration setting, installed software, hardware) represented via endpoint attributes. The phrase "features of the endpoint" highlighted above refers to installed software or software components."

Section 4, first paragraph:

"These stored vulnerability reports can be used and compared

   with later vulnerability reports for the purpose of duplicate

   detection and in some cases guidance on how to handle similar issues."

Comment: storing all individual reports in a data store and trying to dynamically combine and collate them at run time shouldn't be a prescribed behavior.

[danny]: Agree, we think this is an implementation detail.  An enterprise could store all the reports in a single CMDB or they could store them as part of the device history.  This could further be broken up into external vulnerability report received from the vendor as well as the final format used by tools.  We will rework this to make it clear that this is an implementation detail. 

Section 4, second paragraph:

   All vulnerability reports should be assigned an internal tracking ID

   by the enterprise as a first step.  Incoming vulnerability reports

might not have a global identifier when they are received, and might

   never have one assigned.

Comment: Suggest reports should be assigned a unique ID by the transmitter so the receiver isn't tasked with providing the ID

[danny]: It is true that many larger vendors use an ID (bug or even vulnerability) to identify their vulnerability reports.  However, there are many smaller vendors who do not provide an ID.  As well, some vendors may not even acknowledge a vulnerability report generated by a third-party security researcher.  In addition, there would be no consistency when using the IDs across multiple vendors and third-parties.  The enterprise would need to assign some kind of ID (maybe a GUID) within their CMDB to track the vulnerability reports internally.  Other IDs assigned by third-parties (e.g., CVE) could also be associated and used as needed.

Section 4, third paragraph, bulleted list:

o  Ingest Date - the date that the report was received by the enterprise .

Comment 1: Add "and time" so if two reports are received on the same day, you can tell which one was first

[danny]: Agree, this change makes sense.

Comment 2: Is "enterprise" the same as "CMDB", since the report is apparently machine readable and received by an application?

[danny]: I don't think this necessarily synonymous with a CMDB, but, an enterprise could assign the ingest date and time when the report is added to the CMDB.  With that said, I don't think an enterprise is required to do it then.  An enterprise could decide to assign it in some other way.

Section 4, third paragraph, bulleted list:

Date and Report Release

Comment: Change to read " Date and time of Report Release"

[danny]: Agree, this change makes sense.

Section 4, third paragraph, bulleted list:

identify the period when a particular endpoint was first vulnerable

Change to read: " identify the period when a particular endpoint was first assessed as vulnerable" - because you only know the first time a sensor detected a vulnerability, which is dependent on someone discovering the vulnerable state and a sensor looking for it on a given endpoint.  The endpoint may have been vulnerable for years prior to the detection.

[danny]: Agree, this change makes sense.

Section 4, third paragraph, bulleted list:

o  Severity Score (if available) - these may be useful for later mitigation prioritization.

Comment: Recommend that severities not be transmitted with the vulnerability since they change over time and should be obtained from a continuously updated lookup source

[danny]: Maybe there is a way a vulnerability report could externally reference a vulnerability score somewhere so they can change as needed and over time?  With that said, if we do something like this or out-of-band, there is the potential we will lose the ability track the severity scores over time.  We should discuss this further.  

Section 5, first paragraph:

If this is not sufficient to determine endpoint applicability, a secondary data

  collection for additional data and attributes may be performed to

   determine status with regard to the vulnerability report.

Comment: This doesn't consider current remediation methodologies where remediation packages that contain multiple patches are sent out and determine whether/how to "fix" an endpoint at runtime.

[danny]: All we were trying to saying here is that there may be cases where you need to do further assessment beyond what is possible with pre-assessment collected data in order to determine if an endpoint is vulnerable.  We also did not consider remediation because it is out-of-scope.

Section 5.1, first paragraph:

This data would  already have been collected in the pre-assessment data collection.

Change to read: " This data may have been collected in the pre-assessment data collection."  There is no guarantee this has happened.

[danny]: Agree, this change makes sense.

Section 5, second paragraph, bulleted list:

o  Date of assessment - The date that the assessment was performed.

Change to read: " o  Date and time  of assessment - The date that the assessment was performed."

[danny]: Agree, this change makes sense.

Section C.2 "Endpoint" bulleted list:

Endpoint type - the device type of the endpoint (e.g., standard

      computer, printer, router, mobile device, tablet, etc.)

Comment: This is ambiguous.  Typically a "tablet" is a type of "mobile device", that may also function as a WIFI router when used as a hot spot.  I think this list has a bunch of different, incompatible attributes jumbled together.  Suggest it be broken out into multiple lists, each describing the same "type" of 

characteristic (e.g. functionality, color, portability, form factor, etc)

[danny]: Another way we might want to classify endpoint type is by traditional (workstation, server, etc.), mobile (cell phones, laptops, tablets, etc.), network infrastructure (switches, routers, etc.), and constrained (IoT, ICS, printer, etc.).

Section C.2 "Endpoint" bulleted list:

o  Hardware version/firmware - the hardware or firmware version (if applicable)

Comment: This is meaningless without describing which hardware component you're collecting this data for

[danny]: This is just intended to be a general definition and placeholder for hardware/firmware information.  However, we can provide a couple basic examples to make it more clear.

Section C.2 "Endpoint" bulleted list:

o  Operating system - Operating system name

Comment: Please state which part of the operating system descriptive text you consider to be the "name" (e.g. Windows?  Windows 10?  Windows 10 Home?)

[danny]: This is just a high-level definition place holder for now.  If we try to create data models and protocols around this work, we will definitely need to be more specific here.

Section C.2 "Endpoint" bulleted list:

o  Installed software name  - List of all installed software packages

      (i.e., software inventory).  May or may not include software

      installed by the operating system.

Comment: Same issue for software names as for OS names

[danny]: Same answer as OS names :).

Section C.2 "Endpoint" bulleted list:

o  Open ports/services - Listening network ports (TCP and UDP ) and

      running services

Comment: Why is this list constrained to TCP and UDP?  I would expect GRE, IPsec, and other protocols to also be of interest

[danny]: This wasn't intended to be an exhaustive list.  TCP and UDP were just provided as common examples.

Section C.2 "Endpoint" bulleted list:

   o  Operating system optional component inventory - Operating system

      specific components and software (when NOT already included in the

      general software inventory)

Comment: Give examples or define a list.  The OS is made up of thousands of "components"

[danny]: We provided a couple of examples on page 6 of the document "(e.g., web and ftp servers, demo web

      pages, shared libraries, etc.)", but, we will restate them here for clarity.

Section C.2 "Endpoint" bulleted list:

  o  Location  - The physical location of an enterprise endpoint

      (department, room, etc.)

Comment: Describe how the location would be quantified.  Is it GPS coordinates, is it a region name, is it a name pulled from an enumeration defined at a URL?

[danny]: Like some of the other examples this was more of a general place holder.  What this actually means will likely be determined in a data model that would contain location information.

Section C.2 "Endpoint" bulleted list:

o  Role - The function  that the endpoint serves within the enterprise

      (e.g., end user system, database server, public web server, etc.)

Comment: Defining a role as a function is circular and simply makes the two words synonyms.  I suggest keeping "function" as a separate item from role.  For example, a role may be "workstation", while the functions would be office automation, video playback, e-mail client, etc.

[danny]: This term is just thinking along the lines of the enterprise.  Maybe we can just use "purpose" instead of "function" to make sure that it's not confused with any SACM terms (e.g. SACM Function)?  We may also want to think about "role" since there is a "SACM Role".

Section C.2 "Endpoint" bulleted list:

o  File system attributes - Attributes that describe the state of a

      file or directory (e.g., versions, size, write date, modified

      date, checksum, etc.)

Comment: Remove "the state of" since other attributes that aren't state-based are also of interest

[danny]: Agree, this change makes sense.

Section C.2 "External Vulnerability Report"

Comment: Prior to listing properties of an external vulnerability report, please describe what you mean.  Is a report "external" because it was created by someone outside of your organization?  Is it a report about a newly discovered vulnerability, or a vulnerability discovered on one of your endpoints by an external entity?

[danny]: Sure, we will work to improve this.

Overall Comment: Please provide a description of the different "reports" described in this paper in the introduction.  After reading through twice, I think you are using vulnerability "report" and vulnerability "definition" to mean the same thing.  A vulnerability assessment is apparently the process of detecting vulnerabilities on specific endpoints leading to a "vulnerability assessment report".  This is never explicitly defined or addressed, so it is confusing to try to build a mental model of the ideas in the scenario.

[danny]: We will work to clarify this.

I also found a few other comments not listed here that were in the original word document.  Here are responses to those comments.

Section 3.3 Paragraph 4 bulleted list "Criticality"

Comment: Possibly broken out by C/I/A or MAC and Conf levels.

[danny]: Yes, those are all possibilities.  In this document, we didn't try to dive down to that level of detail.  However, if the WG uses this document to drive the development of data models and protocols to support this scenario, we will need to discuss options and these should be considered.

Appendix A. Continuous Vulnerability Assessment

Comment: You generally can't fix vulnerabilities in a BYOD environment, so It's not clear what value knowing about them brings.

[danny]: Yeah, this is likely dependent on an organization's BYOD policy.  An organization might not have access or as part of the BYOD policy require users' provide access to BYOD devices.  We can better figure this out depending on whether or not we want BYOD to be in scope for this first iteration of work.

Appendix B. Priority

Comment: For instance, a vulnerability affecting an operating system or application used through the enterprise would likely be prioritized higher than a vulnerability in an application which is used only on a few, low-criticality endpoints.

[danny]: I think a system administrator would be able to do this with a general understanding of what operating systems and software they use.  For example, they probably know if Windows 7, RHEL 6, Microsoft Office, Apache Webserver, etc. are widely deployed throughout their enterprise.  This might be good for a high-level triage of vulnerability reports.  With that said, you would probably need to look to the pre-collection data in order to determine a more granular and comprehensive priority.  The severity score of the vulnerability report could also be used in determining priority.

Section C.2 "Criticality - An enterprise-defined rating (possibly a score) that helps determine the criticality of the endpoint.  If this endpoint is attacked or lost, what is the impact to the overall enterprise?"

Comment: "possibly a score" should be "set of scores".

[danny]: Agree, this change makes sense.

Section C.2 "External Vulnerability Report"

Comment: So these are required attributes of an external vulnerability report?

[danny]: Yes, these are the information needs (defined in Section 4) that would be useful for a vulnerability report.  Is there anything that you think needs to be added or subtracted?

Section C.2 "Assessment Report"

Comment: I would think a vulnerability report is a type of assessment report.

[danny]: I think there may be some confusion due to us using the term "report".  Vulnerability report is essentially a security advisory where the information necessary to assess an endpoint to see if it is vulnerable is available.  Assessment report is the results of the assessment possibly in human form.  Based on the IETF 94 Information Model Update session, there seemed to be some consensus that "reports" are out-of-scope for SACM.  Given this, we probably want to reconsider this terminology to eliminate confusion.

---END OF RESPONSES TO COMMENTS---

Joseph L. Wolfkiel

SCM Engineering Lead

DISA ID52

Fort Meade DISA Acquisiton Bldg Cube A4A58E

Work: (301) 225-8820

Gov Cell: (571) 814-8231

[email-masked] < Caution-mailto:[email-masked] >  < Caution-Caution-mailto:[email-masked] > 

 

-----Original Message-----

From: sacm [Caution-Caution-mailto:[email-masked] < Caution-Caution-mailto:[email-masked] > ] On Behalf Of Wolfkiel, Joseph L CIV DISA ID (US)

Sent: Tuesday, November 17, 2015 9:26 AM

To: [email-masked] < Caution-mailto:[email-masked] >  < Caution-Caution-mailto:[email-masked] > 

Cc: Haynes, Dan

Subject: [Non-DoD Source] [sacm] Comments to SACM Vulnerability Assessment Scenario

All active links contained in this email were disabled.  Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser.  

 

----

I sent Danny Haynes comments on the vulnerability assessment document at: Caution-Caution-Caution-https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/  Since they may be relevant to some of our discussions, I'm attaching a document with the comments.  With my apologies, it was easier to insert comments in a word document for review, so I'm attaching that.

Joseph L. Wolfkiel

SCM Engineering Lead

DISA ID52

Fort Meade DISA Acquisiton Bldg Cube A4A58E

Work: (301) 225-8820

Gov Cell: (571) 814-8231

[email-masked] < Caution-mailto:[email-masked] >  < Caution-Caution-mailto:[email-masked] >

---

From: "Haynes, Dan" <[email-masked]>
Date: Tue, 8 Dec 2015 18:51:48 +0000

Hi Joe,

Regarding the text from Section 3.1 Paragraph 2:

    "This sub-step aligns with the Endpoint Discovery, Endpoint
Characterization (automated collection data only), and Endpoint Target
Identification building block capabilities."

And your comment: 

    "Please provide rationale on why manually collected data is prohibited."

I checked in with the team and this restriction can be removed allowing any
type of data.

Thanks,

Danny

-----Original Message-----
From: Wolfkiel, Joseph L CIV DISA ID (US)
[mailto:[email-masked]] 
Sent: Monday, December 07, 2015 5:22 PM
To: Haynes, Dan <[email-masked]>
Cc: [email-masked]
Subject: RE:[sacm] Comments to SACM Vulnerability Assessment Scenario

Looks like I left off the SACM e-mail accidently.  Adding them back in and
including response to your last question.

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52

-----Original Message-----
From: Haynes, Dan [mailto:[email-masked]] 
Sent: Monday, December 07, 2015 1:14 PM
To: Wolfkiel, Joseph L CIV DISA ID (US)
Subject: RE: [Non-DoD Source] [sacm] Comments to SACM Vulnerability
Assessment Scenario

-Danny

 -----Original Message-----
From: Wolfkiel, Joseph L CIV DISA ID (US)
[Caution-mailto:[email-masked]] 
Sent: Monday, December 07, 2015 8:38 AM
To: Haynes, Dan <[email-masked]>
Subject: RE: [Non-DoD Source] [sacm] Comments to SACM Vulnerability
Assessment Scenario

Here are responses back to your replies.  In some of your responses, you
provided explanations.  If you believe that others may misunderstand or be
confused by verbiage, it may be better to clarify the document instead.

[danny]: Seems like a reasonable suggestion.  We could either remove the
"basic" qualifier or separate them out into two different lists ("basic" and
"advanced").  Do you have a preference?

[joe] My preference is to start the discussion on what "basic" means, so if
some of the data is basic and some isn't, then I like the idea of dividing
it into two lists.  My opinion is that SACM will ultimately need to define
some minimum required data set to be collected for interoperability and
device identification/characterization, so it's a good concept to address in
the use case documents. That is, of course, assuming that "minimum required"
and "basic" are synonymous.  I'm struggling to understand what you mean when
you say "basic".

[danny]:  Thanks for the additional information.  You are absolutely correct
in your opinion.  SACM does need to define some set of
mandatory-to-implement information to ensure interoperability.  This
information should and will be captured in the information model and will
drive the development of data models (or at least data models will be
required to support the information needs in the information model).

*****************

 Comment: Recommend consideration of placing attributes on the endpoints so
they can be discovered by multiple sensors and also viewed and edited
locally.  This would enhance accuracy and ensure a common assessment by all
sensors collecting data from an endpoint.

[danny]: I could potentially see this as being an implementation detail.
Can your provide a little more information on how you envision this working?
Would organizations just place whatever human-assigned data they want on the
endpoint wherever they want?  Would there be mandated data models and
locations where the human-assigned information would be stored?  I am just
trying to think through the situation as this information would likely be
organization specific and unless an organization is developing its own
collection tools there is a good chance COTs collection tools would not know
about this information nor how to collect it.  Anyways, I think we can add
some more text to discuss this topic once we have a better understanding of
what SACM needs here.

[joe] I agree this is an immature topic.  I've been doing some design work
in this area so my organization can move this way in the future.  Current
thought is that the data elements you need to "tag" devices with
characterization data include (for each characteristic), the URI of the
characteristic definition (e.g. "http://ietf.org/data_sensitivity), the
value selected  (e.g. "medium") or the ID of the value selected (e.g. "3"),
the timestamp of when the attribute was assigned, and the status of the
attribute (true=active, false=deprecated).

Here's an example using the DoD ARF "taggedString" format.  This is not a
recommendation or requirement that SACM use XML as transport, just an
example of how we're documenting device attributes in the DoD ARF
implementation.

<ns01:taggedString name="https://ietf.org?name="data_sensitivity" value="3"
timestamp="2015-05-23T01:42:05.000-04:00" source="jwolfkiel" status="true">

[danny]: This example is very helpful.  Just curious, how does a tool know
where to find this tag?  Are they all stored in a standard location?  Does
the location vary depending on the type of tag?  Does it require the tool to
do an exhaustive search to find the tags on the system?

[joe] We're planning to designate specific registry hives and keys for
microsoft and/or file locations in the "..ix" file systems to store the tag
values.  I would expect to put all the tags in the same logical location.

 ***********************

Comment: Why is this list constrained to TCP and UDP?  I would expect GRE,
IPsec, and other protocols to also be of interest

 [danny]: This wasn't intended to be an exhaustive list.  TCP and UDP were
just provided as common examples.

[joe]: For lists of examples, it's traditional to include the "e.g."
qualifier, which will clarify that the list is a set of examples, not an
exhaustive definition.

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52
Fort Meade DISA Acquisiton Bldg Cube A4A58E
Work: (301) 225-8820
Gov Cell: (571) 814-8231
[email-masked] <
Caution-mailto:[email-masked] > 

-----Original Message-----

From: Haynes, Dan [Caution-mailto:[email-masked] <
Caution-mailto:[email-masked] > ] 

To: Wolfkiel, Joseph L CIV DISA ID (US); [email-masked] <
Caution-mailto:[email-masked] > 

Subject: RE: [Non-DoD Source] [sacm] Comments to SACM Vulnerability
Assessment Scenario

All active links contained in this email were disabled. Please verify the
identity of the sender, and confirm the authenticity of all links contained
within the message prior to copying and pasting the address to a Web
browser. 

 

Hi Joe,

Thanks for the feedback!

 

Thanks,

Danny

-----Original Message-----

From: Wolfkiel, Joseph L CIV DISA ID (US)
[Caution-Caution-mailto:[email-masked]] 

Subject: RE: [Non-DoD Source] [sacm] Comments to SACM Vulnerability
Assessment Scenario

 

Section 1, 2nd paragraph: " whether or not the flaw is present within an
enterprise, when

 

Comment: Does it say which vulnerabilities were discovered on what software
on which endpoints?  There's not a definition or example of what
"vulnerability report" means.

[danny]: The input vulnerability report would likely tell you what software,
version, on which platform was affected and any other applicable data for
any endpoint.  After the assessment, you will know which vulnerabilities in
which software on which endpoints.

      this document:"

Comment: The text alludes to a requirement that the report be machine
readable, but never states it.

[danny]: The document does not specify whether or not the vulnerability
report is in a machine-readable format just that the necessary information
has been extracted and put in a format tool's can use.  With that said, I
think it would be great if there was some machine-readable format for
vulnerability reports.

 

 

 

[danny]: This refers to the initial process of determining which
vulnerability reports are applicable to an enterprise and assumes:

    * an enterprise obtains applicable vulnerability reports in some manner
as they become available

 

Given this, we should update the assumption to state "The document begins
with the assumption that the enterprise has received a vulnerability report
that is applicable to their environment, and the report data has already.".

 

Comment: Does this address how the report was acquired, or how the data in
the report was acquired?

[danny]: No, it does not address how the enterprise collects the
vulnerability report from the publishing vendor.

 

Comment: Does the statement about endpoint identification imply that the
report contains data about endpoints?

[danny]: No, this just means the enterprise has some way to know what
endpoints are on its network.  The vulnerability report only provides
information about what software and configurations result in a vulnerable
state.  The vulnerability report could then be used as input into an
assessment to determine if it is in fact vulnerable.

         compatible.

 

[danny]: Compatible here just means that the information about an endpoint
as well as the relevant data from the vulnerability report are normalized in
some way such that the data is comparable and an assessment can be
performed.

 

Comment: It appears that an organization will "assess" something on receipt
of a vulnerability report.  What does it assess?

[danny]: Yes. Based on the vulnerability report, an enterprise will
determine if the vulnerability report is applicable to an endpoint (not to
be confused with applicable to the enterprise during initial report
retrieval) and then carry out an assessment. The priority of that assessment
would be determined by the enterprise policy.  The priority would likely
affect the timeliness of the assessment, and even how often any
reassessments occur.  The priority of the assessments is given a nod in the
section titled Priority, but is not discussed as part of the overall
scenario.

  CMDB ."

Comment: Please spell out "CMDB" on first use and provide a reference that
explains what the authors think a CMDB would contain.  A configuration
management database (CMDB) isn't necessarily assumed to contain a
comprehensive device inventory.  Arguably, it stores baseline configurations
and deviations to them.

[danny]: We just used this as a general place where we can store/retrieve
collection data (including inventory), guidance, assessment results
(including baseline configurations and deviations), etc.  It sounds like we
may need to provide a little more context on this.  We may also want to call
it something other than CMDB to avoid any of these assumptions (e.g.
repository with data stores which can hold all this information).

 

Comment: Is this the scope of how/what endpoint IDs are used for?  Are IDs
used to correlate information between sensors?

[danny]: This text was not meant to exclude between sensors.  It would be
fair to include across sensors as well.

 

Comment: Please provide rationale on why manually collected data is
prohibited

[danny]: At the moment, I don't recall exactly why we included that
restriction :).  Let me find out and follow up on this one.

Suggest changing " Endpoint type- e.g., standard computer, printer, router,
mobile

      device, tablet, etc." to read " Endpoint function or general role -
e.g., standard computer, printer, router, mobile

     device, tablet, etc."

 

[danny]: Another way we might want to classify endpoint type is by
traditional (workstation, server, etc.), mobile (cell phones, laptops,
tablets, etc.), network infrastructure (switches, routers, etc.), and
constrained (IoT, ICS, printer, etc.).

 

 

[danny]: We did not specify anything in particular.  It was just an example
of the type of information an enterprise might want to collect about an
endpoint in its pre-assessment data collection activities.  An enterprise
could potentially collect all of the information you listed above.  With
that said, we think it would be fair to include BIOS versions and firmware
revisions to make it a little more clear.  The main point here, however, is
that the scenario is only talking about basic information about the
endpoint.

 

 

 

Comment:  Suggest that neither of these bullets describes "basic" data,
since they are long, detailed lists that aren't specifically required to
identify or characterize a device.

[danny]: Seems like a reasonable suggestion.  We could either remove the
"basic" qualifier or separate them out into two different lists ("basic" and
"advanced").  Do you have a preference?

 

Comment: Recommend consideration of placing attributes on the endpoints so
they can be discovered by multiple sensors and also viewed and edited
locally.  This would enhance accuracy and ensure a common assessment by all
sensors collecting data from an endpoint.

[danny]: I could potentially see this as being an implementation detail.
Can your provide a little more information on how you envision this working?
Would organizations just place whatever human-assigned data they want on the
endpoint wherever they want?  Would there be mandated data models and
locations where the human-assigned information would be stored?  I am just
trying to think through the situation as this information would likely be
organization specific and unless an organization is developing its own
collection tools there is a good chance COTs collection tools would not know
about this information nor how to collect it.  Anyways, I think we can add
some more text to discuss this topic once we have a better understanding of
what SACM needs here.

 

[danny]: From the Terminology document
(Caution-Caution-https://github.com/sacmwg/draft-ietf-sacm-terminology/blob/
master/draft-ietf-sacm-terminology.md <
Caution-Caution-https://github.com/sacmwg/draft-ietf-sacm-terminology/blob/m
aster/draft-ietf-sacm-terminology.md > ), "posture" is defined in the
terminology document as:

"Defined in {{RFC5209}} as "configuration and/or status of hardware or
software on an endpoint as it pertains to an organization's security
policy." This term is used within the scope of SACM to represent the
configuration and state information that is collected from a target endpoint
in the form of endpoint attributes (e.g. software/hardware inventory,
configuration settings, dynamically assigned addresses). This information
may constitute one or more posture attributes."

"Defined in {{RFC5209}} as "attributes describing the configuration or
status (posture) of a feature of the endpoint. A Posture Attribute
represents a single property of an observed state. For example, a Posture
Attribute might describe the version of the operating system installed on
the system."  Within this document this term represents a specific assertion
about endpoint configuration or state (e.g. configuration setting, installed
software, hardware) represented via endpoint attributes. The phrase
"features of the endpoint" highlighted above refers to installed software or
software components."

 

Comment: storing all individual reports in a data store and trying to
dynamically combine and collate them at run time shouldn't be a prescribed
behavior.

[danny]: Agree, we think this is an implementation detail.  An enterprise
could store all the reports in a single CMDB or they could store them as
part of the device history.  This could further be broken up into external
vulnerability report received from the vendor as well as the final format
used by tools.  We will rework this to make it clear that this is an
implementation detail. 

   never have one assigned.

Comment: Suggest reports should be assigned a unique ID by the transmitter
so the receiver isn't tasked with providing the ID

[danny]: It is true that many larger vendors use an ID (bug or even
vulnerability) to identify their vulnerability reports.  However, there are
many smaller vendors who do not provide an ID.  As well, some vendors may
not even acknowledge a vulnerability report generated by a third-party
security researcher.  In addition, there would be no consistency when using
the IDs across multiple vendors and third-parties.  The enterprise would
need to assign some kind of ID (maybe a GUID) within their CMDB to track the
vulnerability reports internally.  Other IDs assigned by third-parties
(e.g., CVE) could also be associated and used as needed.

 

Comment 1: Add "and time" so if two reports are received on the same day,
you can tell which one was first

 

Comment 2: Is "enterprise" the same as "CMDB", since the report is
apparently machine readable and received by an application?

[danny]: I don't think this necessarily synonymous with a CMDB, but, an
enterprise could assign the ingest date and time when the report is added to
the CMDB.  With that said, I don't think an enterprise is required to do it
then.  An enterprise could decide to assign it in some other way.

Date and Report Release

 

 

Change to read: " identify the period when a particular endpoint was first
assessed as vulnerable" - because you only know the first time a sensor
detected a vulnerability, which is dependent on someone discovering the
vulnerable state and a sensor looking for it on a given endpoint.  The
endpoint may have been vulnerable for years prior to the detection.

 

o  Severity Score (if available) - these may be useful for later mitigation
prioritization.

Comment: Recommend that severities not be transmitted with the vulnerability
since they change over time and should be obtained from a continuously
updated lookup source

[danny]: Maybe there is a way a vulnerability report could externally
reference a vulnerability score somewhere so they can change as needed and
over time?  With that said, if we do something like this or out-of-band,
there is the potential we will lose the ability track the severity scores
over time.  We should discuss this further.  

If this is not sufficient to determine endpoint applicability, a secondary
data

 

Comment: This doesn't consider current remediation methodologies where
remediation packages that contain multiple patches are sent out and
determine whether/how to "fix" an endpoint at runtime.

[danny]: All we were trying to saying here is that there may be cases where
you need to do further assessment beyond what is possible with
pre-assessment collected data in order to determine if an endpoint is
vulnerable.  We also did not consider remediation because it is
out-of-scope.

This data would  already have been collected in the pre-assessment data
collection.

Change to read: " This data may have been collected in the pre-assessment
data collection."  There is no guarantee this has happened.

 

Change to read: " o  Date and time  of assessment - The date that the
assessment was performed."

 

 

Comment: This is ambiguous.  Typically a "tablet" is a type of "mobile
device", that may also function as a WIFI router when used as a hot spot.  I
think this list has a bunch of different, incompatible attributes jumbled
together.  Suggest it be broken out into multiple lists, each describing the
same "type" of 

 

[danny]: Another way we might want to classify endpoint type is by
traditional (workstation, server, etc.), mobile (cell phones, laptops,
tablets, etc.), network infrastructure (switches, routers, etc.), and
constrained (IoT, ICS, printer, etc.).

o  Hardware version/firmware - the hardware or firmware version (if
applicable)

Comment: This is meaningless without describing which hardware component
you're collecting this data for

[danny]: This is just intended to be a general definition and placeholder
for hardware/firmware information.  However, we can provide a couple basic
examples to make it more clear.

 

Comment: Please state which part of the operating system descriptive text
you consider to be the "name" (e.g. Windows?  Windows 10?  Windows 10 Home?)

[danny]: This is just a high-level definition place holder for now.  If we
try to create data models and protocols around this work, we will definitely
need to be more specific here.

 

 

 

      running services

Comment: Why is this list constrained to TCP and UDP?  I would expect GRE,
IPsec, and other protocols to also be of interest

[danny]: This wasn't intended to be an exhaustive list.  TCP and UDP were
just provided as common examples.

 

Comment: Give examples or define a list.  The OS is made up of thousands of
"components"

[danny]: We provided a couple of examples on page 6 of the document "(e.g.,
web and ftp servers, demo web

      pages, shared libraries, etc.)", but, we will restate them here for
clarity.

      (department, room, etc.)

Comment: Describe how the location would be quantified.  Is it GPS
coordinates, is it a region name, is it a name pulled from an enumeration
defined at a URL?

[danny]: Like some of the other examples this was more of a general place
holder.  What this actually means will likely be determined in a data model
that would contain location information.

 

Comment: Defining a role as a function is circular and simply makes the two
words synonyms.  I suggest keeping "function" as a separate item from role.
For example, a role may be "workstation", while the functions would be
office automation, video playback, e-mail client, etc.

[danny]: This term is just thinking along the lines of the enterprise.
Maybe we can just use "purpose" instead of "function" to make sure that it's
not confused with any SACM terms (e.g. SACM Function)?  We may also want to
think about "role" since there is a "SACM Role".

      date, checksum, etc.)

Comment: Remove "the state of" since other attributes that aren't
state-based are also of interest

 

 

Comment: Prior to listing properties of an external vulnerability report,
please describe what you mean.  Is a report "external" because it was
created by someone outside of your organization?  Is it a report about a
newly discovered vulnerability, or a vulnerability discovered on one of your
endpoints by an external entity?

 

Overall Comment: Please provide a description of the different "reports"
described in this paper in the introduction.  After reading through twice, I
think you are using vulnerability "report" and vulnerability "definition" to
mean the same thing.  A vulnerability assessment is apparently the process
of detecting vulnerabilities on specific endpoints leading to a
"vulnerability assessment report".  This is never explicitly defined or
addressed, so it is confusing to try to build a mental model of the ideas in
the scenario.

 

I also found a few other comments not listed here that were in the original
word document.  Here are responses to those comments.

 

 

[danny]: Yes, those are all possibilities.  In this document, we didn't try
to dive down to that level of detail.  However, if the WG uses this document
to drive the development of data models and protocols to support this
scenario, we will need to discuss options and these should be considered.

 

Comment: You generally can't fix vulnerabilities in a BYOD environment, so
It's not clear what value knowing about them brings.

[danny]: Yeah, this is likely dependent on an organization's BYOD policy.
An organization might not have access or as part of the BYOD policy require
users' provide access to BYOD devices.  We can better figure this out
depending on whether or not we want BYOD to be in scope for this first
iteration of work.

Appendix B. Priority

Comment: For instance, a vulnerability affecting an operating system or
application used through the enterprise would likely be prioritized higher
than a vulnerability in an application which is used only on a few,
low-criticality endpoints.

[danny]: I think a system administrator would be able to do this with a
general understanding of what operating systems and software they use.  For
example, they probably know if Windows 7, RHEL 6, Microsoft Office, Apache
Webserver, etc. are widely deployed throughout their enterprise.  This might
be good for a high-level triage of vulnerability reports.  With that said,
you would probably need to look to the pre-collection data in order to
determine a more granular and comprehensive priority.  The severity score of
the vulnerability report could also be used in determining priority.

Section C.2 "Criticality - An enterprise-defined rating (possibly a score)
that helps determine the criticality of the endpoint.  If this endpoint is
attacked or lost, what is the impact to the overall enterprise?"

 

 

 

Comment: So these are required attributes of an external vulnerability
report?

[danny]: Yes, these are the information needs (defined in Section 4) that
would be useful for a vulnerability report.  Is there anything that you
think needs to be added or subtracted?

 

Comment: I would think a vulnerability report is a type of assessment
report.

[danny]: I think there may be some confusion due to us using the term
"report".  Vulnerability report is essentially a security advisory where the
information necessary to assess an endpoint to see if it is vulnerable is
available.  Assessment report is the results of the assessment possibly in
human form.  Based on the IETF 94 Information Model Update session, there
seemed to be some consensus that "reports" are out-of-scope for SACM.  Given
this, we probably want to reconsider this terminology to eliminate
confusion.

---END OF RESPONSES TO COMMENTS---

Joseph L. Wolfkiel

SCM Engineering Lead

DISA ID52

Work: (301) 225-8820

Gov Cell: (571) 814-8231

[email-masked] <
Caution-mailto:[email-masked] >  <
Caution-Caution-mailto:[email-masked] > 

 

-----Original Message-----

From: sacm [Caution-Caution-mailto:[email-masked] <
Caution-Caution-mailto:[email-masked] > ] On Behalf Of Wolfkiel,
Joseph L CIV DISA ID (US)

To: [email-masked] < Caution-mailto:[email-masked] >  <
Caution-Caution-mailto:[email-masked] > 

Cc: Haynes, Dan

Subject: [Non-DoD Source] [sacm] Comments to SACM Vulnerability Assessment
Scenario

All active links contained in this email were disabled.  Please verify the
identity of the sender, and confirm the authenticity of all links contained
within the message prior to copying and pasting the address to a Web
browser.  

 

----

I sent Danny Haynes comments on the vulnerability assessment document at:
Caution-Caution-Caution-https://datatracker.ietf.org/doc/draft-coffin-sacm-v
uln-scenario/  Since they may be relevant to some of our discussions, I'm
attaching a document with the comments.  With my apologies, it was easier to
insert comments in a word document for review, so I'm attaching that.

Joseph L. Wolfkiel

SCM Engineering Lead

DISA ID52

Work: (301) 225-8820

Gov Cell: (571) 814-8231

[email-masked] <
Caution-mailto:[email-masked] >  <
Caution-Caution-mailto:[email-masked] >