Subject: Re: [sacm] SACM Vulnerability Assessment Scenario Review

From: "Haynes, Dan" <[email-masked]>
Date: Wed, 2 Dec 2015 17:42:59 +0000

Thanks for the feedback Adam!  Glad to hear they believed it was a fairly accurate representation of vulnerability assessment.

Yes, the scenario is more generally applicable than an exact scenario.

Thanks,

Danny

From: Adam Montville [mailto:[email-masked]]
Sent: Wednesday, November 18, 2015 8:29 AM
To: Haynes, Dan <[email-masked]>
Cc: [email-masked]
Subject: Re: [sacm] SACM Vulnerability Assessment Scenario Review

Hi Danny,

I asked some people at CIS who work with various US state, local, and tribal governments in this capacity to review the draft.  They felt that the draft was a fairly accurate representation of vulnerability assessment and had no specific comments.

When I asked for their review, I specifically asked them to opine as to the accuracy of the draft over its precision - I believe the goal is to depict a generally applicable, rather than exact, scenario.

Kind regards,

Adam

PS: I expect further reviews from within my organization to be forthcoming.

On Nov 17, 2015, at 8:28 AM, Haynes, Dan <[email-masked]<mailto:[email-masked]>> wrote:

Just to follow up on Adam’s request for the WG to review the Vulnerability Assessment Scenario document by November 20th, it would be great to hear any feedback that you may have on the document as well as how well it aligns with the way your organization does vulnerability assessment.

For your convenience, you may find the document here (https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/).

Feel free to provide any feedback over the list.

Thanks,

Danny
_______________________________________________
sacm mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/sacm

---

From: "Haynes, Dan" <[email-masked]>
Date: Mon, 7 Dec 2015 13:30:48 +0000

Hi Sherif,

Thanks for the feedback!

Comments inline below.  Please let me know if I am misunderstanding anything.

Thanks,

Danny

From: Sherif Mansour Farag [mailto:[email-masked]]
Sent: Wednesday, November 18, 2015 8:40 AM
To: Haynes, Dan <[email-masked]>; [email-masked]
Subject: Re: SACM Vulnerability Assessment Scenario Review

Hi Danny,

I have taken a look at https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/ and I have some feedback.

How do we scan?

We tend to run both credentialed and non-credentialed scans on our internal network ranges, as well as ASV scans for our external network ranges.

For context a credentialed scan is when a vulnerability scanner logs in over WMI/SSH and runs some checks on the box. A non credentialed scans depends on port scanning and a few passive scanning techniques.

The credentialed scans give us more detailed information, while the network scans helps us identify any deltas or discrepancies between the two.

Example, the internal scanner detected 100 systems via port scanning but was only able to log in and check 85 of those systems.

[danny]:  This seems to further highlight the need to consider both authenticated and unauthenticated scans, the data these scans collect, and the implications of that data (such as those Josh mentioned in his feedback).

What do we check for ?

Aside from the issues mentioned in the document, with regards to actual checks, we also look for any virtualization on these systems. Are there any docker containers for VMware/Virtualbox guests for example? Additionally if there is a web-application listening on a port we run some light web app tests as well.

[danny]:  Good to know.  This is definitely something SACM will need to consider.

​How do we use the reporting?

Ah! this is the tricky bit, its manual at the moment but we look to use an API to automate the process. In general we look for a few things

  *   ​We like CSV/spreadsheets because we could filteri either by a vulnerability across machines OR all issues on one machine.
     *   ​Some sys admins would want to check if their patched "Canary" server has all issues resolved before pushing a fix across the enviroment
     *   Equally some security engineers would want to know which systems are impacted by a specific issue

[danny]: I think this should all be possible if we have vulnerability report data, collection data, and assessment results in a data store that can be queried.

  *   Additionally there are multiple stakeholders for example:
     *   ​Sys Admins patch the version of Java installed on a system, but not before developers can test and confirm their code can work with a later version of Java.

[danny]: This makes sense and is good to know.  However, I think it might be currently out-of-scope for the scenario since it ends at identifying which endpoints are impacted by a particular vulnerability report.

  *   ​We proceed three types of reports per Asset group owners (i.e. the sys admins that would normally patch the boxes).
     *   ​Emergency - Drop everything you are doing and fix all issues in this report.
     *   High - You have less than a month to patch these issues (one of our counterparts has a 24 hour SLA for this).
     *   Medium/Low - You need to fix these issues by the end of the quarter.

[danny]:  Would it be fair to call this the priority of how the impacted endpoints should be remediated (although remediation is out-of-scope for SACM)?  Also, do you different types of information, or different levels of information, for each type of report?

The challenge is with containers like docker now, because while the base image has instructions to load the latest updates before building the container, that container might be out in the environment for a while and forgotten, which is why we are looking for a solution that interrogates docker containers for example (https://coreos.com/blog/vulnerability-analysis-for-containers/)

[danny]: Thanks for the additional information.  This was very helpful for me.  I also agree that this is something that needs to be considered by SACM.  On a related note, in the SCAP space, I know Red Hat is working to address containers with their OpenSCAP tool (http://www.open-scap.org/resources/documentation/security-compliance-of-rhel7-docker-containers/).

Hope that helps.

Sherif Mansour
Enterprise Information Security
Sr Security Engineer | EIS – Secure Design & Consulting

T + 44 (0) 203 194 5245 |  M + 44 (0) 7860 782936

Expedia.com Ltd – 407 St John St, London EC1V4EX

________________________________
From: sacm <[email-masked]<mailto:[email-masked]>> on behalf of Haynes, Dan <[email-masked]<mailto:[email-masked]>>
Sent: Tuesday, November 17, 2015 2:28 PM
To: [email-masked]<mailto:[email-masked]>
Subject: [sacm] SACM Vulnerability Assessment Scenario Review

Thanks,

Danny

---

From: Sherif Mansour Farag <[email-masked]>
Date: Thu, 10 Dec 2015 01:17:01 +0000

Hi Danny,

I think you pretty much got it.

  *   ​For the Java example below, it just means that we sometimes filter reports by the time of issues as different stake holders would need to remediate the problem.
  *   As for the actually reporting, the same type of information is sent to the different teams (What is the problem, what endpoints is it on, how to fix it, how bad is the problem etc..).
     *   ​One caveat is metrics. Eventually we would generate data to present to leadership so the quantity of issues as well as time to remediation is important. Its important because we are only as secure as our weakest link in the chain, so knowing how long we are exposed before an issue is fixed is an important metric.

 -Sherif
​​
________________________________
From: Haynes, Dan <[email-masked]>
Sent: Monday, December 07, 2015 1:30 PM
To: Sherif Mansour Farag; [email-masked]
Subject: RE: SACM Vulnerability Assessment Scenario Review

Hi Sherif,

Thanks for the feedback!

Thanks,

Danny

Hi Danny,

How do we scan?

What do we check for ?

Hope that helps.

Sherif Mansour

Thanks,

Danny

---

From: "Haynes, Dan" <[email-masked]>
Date: Thu, 10 Dec 2015 13:34:11 +0000

Thanks Sherif!

Bullet 1: Thanks for the clarification.  That makes sense.

Bullet 2: Good to know.  I think this information would be available by examining the history of vulnerability assessments for each endpoint across the network correct?

Thanks,

Danny

From: Sherif Mansour Farag [mailto:[email-masked]]
Sent: Wednesday, December 09, 2015 8:17 PM
To: Haynes, Dan <[email-masked]>; [email-masked]
Subject: Re: SACM Vulnerability Assessment Scenario Review

Hi Danny,

  *   ​For the Java example below, it just means that we sometimes filter reports by the time of issues as different stake holders would need to remediate the problem.
  *   As for the actually reporting, the same type of information is sent to the different teams (What is the problem, what endpoints is it on, how to fix it, how bad is the problem etc..).
     *   ​One caveat is metrics. Eventually we would generate data to present to leadership so the quantity of issues as well as time to remediation is important. Its important because we are only as secure as our weakest link in the chain, so knowing how long we are exposed before an issue is fixed is an important metric.
 -Sherif
​​
________________________________
From: Haynes, Dan <[email-masked]<mailto:[email-masked]>>
Sent: Monday, December 07, 2015 1:30 PM
To: Sherif Mansour Farag; [email-masked]<mailto:[email-masked]>
Subject: RE: SACM Vulnerability Assessment Scenario Review

Hi Sherif,

Thanks for the feedback!

Thanks,

Danny

From: Sherif Mansour Farag [mailto:[email-masked]]
Sent: Wednesday, November 18, 2015 8:40 AM
To: Haynes, Dan <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: SACM Vulnerability Assessment Scenario Review

Hi Danny,

How do we scan?

What do we check for ?

Hope that helps.

Sherif Mansour

Thanks,

Danny

---

From: Sherif Mansour Farag <[email-masked]>
Date: Thu, 10 Dec 2015 14:19:28 +0000

​Yes, when it was fist discovered, last time it was detected, has it been resolved then reopened etc..

Sent: Thursday, December 10, 2015 1:34 PM

Thanks Sherif!

Thanks,

Danny

Hi Danny,

Hi Sherif,

Thanks for the feedback!

Thanks,

Danny

Hi Danny,

How do we scan?

What do we check for ?

Hope that helps.

Sherif Mansour

Thanks,

Danny

---

From: "Haynes, Dan" <[email-masked]>
Date: Thu, 10 Dec 2015 14:40:31 +0000

Great, thanks for the quick reply!

-Danny

From: Sherif Mansour Farag [mailto:[email-masked]]
Sent: Thursday, December 10, 2015 9:19 AM
To: Haynes, Dan <[email-masked]>; [email-masked]
Subject: Re: SACM Vulnerability Assessment Scenario Review

Thanks Sherif!

Thanks,

Danny

Hi Danny,

Hi Sherif,

Thanks for the feedback!

Thanks,

Danny

Hi Danny,

How do we scan?

What do we check for ?

Hope that helps.

Sherif Mansour

Thanks,

Danny

---

From: "Wolfkiel, Joseph L CIV DISA ID (US)" <[email-masked]>
Date: Thu, 10 Dec 2015 14:52:37 +0000

Danny,

Is this within scope of your use case?  I assumed the vulnerability assessment scenario was just a limited scope document that didn't really address how vulnerability scanning is done in a day-to-day environment.  In general, we have vulnerability scanners deployed, the scanner vendors embed the "vulnerability reports" into the content they automatically distribute, scans are run, then open vulnerabilities are evaluated to see why our automated patching solutions didn't fix them and to have the appropriate persons do remediation.  In that kind of environment, it's more important to track first seen, last seen type data versus the targeted one-vulnerability-at-a-time scenario described in the use case.

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52
Fort Meade DISA Acquisiton Bldg Cube A4A58E
Work: (301) 225-8820
Gov Cell: (571) 814-8231
[email-masked]

-----Original Message-----
From: sacm [mailto:[email-masked]] On Behalf Of Sherif Mansour Farag
Sent: Thursday, December 10, 2015 9:19 AM
To: Haynes, Dan; [email-masked]
Subject: [Non-DoD Source] Re: [sacm] SACM Vulnerability Assessment Scenario Review

All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser. 

Thanks Sherif! 

Thanks,

Danny

From: Sherif Mansour Farag [Caution-mailto:[email-masked]]
Sent: Wednesday, December 09, 2015 8:17 PM
To: Haynes, Dan <[email-masked]>; [email-masked]
Subject: Re: SACM Vulnerability Assessment Scenario Review

Hi Danny,

 -Sherif
​​ 

From: Haynes, Dan <[email-masked] < Caution-mailto:[email-masked] > >
Sent: Monday, December 07, 2015 1:30 PM
To: Sherif Mansour Farag; [email-masked] < Caution-mailto:[email-masked] > 
Subject: RE: SACM Vulnerability Assessment Scenario Review

Hi Sherif,

Thanks for the feedback!  

Thanks,

Danny

From: Sherif Mansour Farag [Caution-mailto:[email-masked] < Caution-mailto:[email-masked] > ]
Sent: Wednesday, November 18, 2015 8:40 AM
To: Haynes, Dan <[email-masked] < Caution-mailto:[email-masked] > >;[email-masked] < Caution-mailto:[email-masked] > 
Subject: Re: SACM Vulnerability Assessment Scenario Review

Hi Danny,

I have taken a look at Caution-https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/ < Caution-https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/ >  and I have some feedback.

How do we scan?

What do we check for ?

The challenge is with containers like docker now, because while the base image has instructions to load the latest updates before building the container, that container might be out in the environment for a while and forgotten, which is why we are looking for a solution that interrogates docker containers for example (Caution-https://coreos.com/blog/vulnerability-analysis-for-containers/ < Caution-https://coreos.com/blog/vulnerability-analysis-for-containers/ > )

[danny]: Thanks for the additional information.  This was very helpful for me.  I also agree that this is something that needs to be considered by SACM.  On a related note, in the SCAP space, I know Red Hat is working to address containers with their OpenSCAP tool (Caution-http://www.open-scap.org/resources/documentation/security-compliance-of-rhel7-docker-containers/ < Caution-http://www.open-scap.org/resources/documentation/security-compliance-of-rhel7-docker-containers/ > ).

Hope that helps.

Sherif Mansour

From: sacm <[email-masked] < Caution-mailto:[email-masked] > > on behalf of Haynes, Dan <[email-masked] < Caution-mailto:[email-masked] > >
Sent: Tuesday, November 17, 2015 2:28 PM
To: [email-masked] < Caution-mailto:[email-masked] > 
Subject: [sacm] SACM Vulnerability Assessment Scenario Review

For your convenience, you may find the document here (Caution-https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/ < Caution-https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/ > ). 

Thanks,

Danny

---

From: "Haynes, Dan" <[email-masked]>
Date: Thu, 10 Dec 2015 19:40:34 +0000

Hi Joe,

You are correct, the document is limited in scope and is not intended to prescribe how to carry out day-to-day vulnerability scanning.  With that said, given the information that would be available (or could be available) in this scenario, it seems to me like Sherif's use case would be feasible.

Maybe I am misunderstanding what you are asking :)? 

Thanks,

Danny

-----Original Message-----
From: Wolfkiel, Joseph L CIV DISA ID (US) [mailto:[email-masked]] 
Sent: Thursday, December 10, 2015 9:53 AM
To: Sherif Mansour Farag <[email-masked]>; Haynes, Dan <[email-masked]>; [email-masked]
Subject: RE: [Non-DoD Source] Re: [sacm] SACM Vulnerability Assessment Scenario Review

Danny,

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52

-----Original Message-----
From: sacm [mailto:[email-masked]] On Behalf Of Sherif Mansour Farag
Sent: Thursday, December 10, 2015 9:19 AM
To: Haynes, Dan; [email-masked]
Subject: [Non-DoD Source] Re: [sacm] SACM Vulnerability Assessment Scenario Review

Thanks Sherif! 

Thanks,

Danny

Hi Danny,

 -Sherif
​​ 

Hi Sherif,

Thanks for the feedback!  

Thanks,

Danny

Hi Danny,

How do we scan?

What do we check for ?

Hope that helps.

Sherif Mansour

Thanks,

Danny

---

From: "Wolfkiel, Joseph L CIV DISA ID (US)" <[email-masked]>
Date: Thu, 10 Dec 2015 19:46:39 +0000

It's probably that we don't share a common understanding of the use case.  When I read it, I inferred that it assumed you would do a dedicated scan for the newly received "vulnerability report" and only one big scan after the initial receipt.  The use case didn't really address "future" type of issues, such as finding devices that weren't scanned in the initial assessment, or identifying devices where a patch doesn't "stick" over time.

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52

-----Original Message-----
From: Haynes, Dan [mailto:[email-masked]] 
Sent: Thursday, December 10, 2015 2:41 PM
To: Wolfkiel, Joseph L CIV DISA ID (US); Sherif Mansour Farag; [email-masked]
Subject: RE: [Non-DoD Source] Re: [sacm] SACM Vulnerability Assessment Scenario Review

Hi Joe,

Thanks,

Danny

-----Original Message-----
From: Wolfkiel, Joseph L CIV DISA ID (US) [mailto:[email-masked]] 
Sent: Thursday, December 10, 2015 9:53 AM
To: Sherif Mansour Farag <[email-masked]>; Haynes, Dan <[email-masked]>; [email-masked]
Subject: RE: [Non-DoD Source] Re: [sacm] SACM Vulnerability Assessment Scenario Review

Danny,

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52

-----Original Message-----
From: sacm [mailto:[email-masked]] On Behalf Of Sherif Mansour Farag
Sent: Thursday, December 10, 2015 9:19 AM
To: Haynes, Dan; [email-masked]
Subject: [Non-DoD Source] Re: [sacm] SACM Vulnerability Assessment Scenario Review

Thanks Sherif! 

Thanks,

Danny

Hi Danny,

 -Sherif
​​ 

Hi Sherif,

Thanks for the feedback!  

Thanks,

Danny

Hi Danny,

How do we scan?

What do we check for ?

Hope that helps.

Sherif Mansour

Thanks,

Danny