Subject: [sacm] Gut Check?

From: Gunnar Engelbach <[email-masked]>
Date: Mon, 02 Feb 2015 13:56:57 -0500

As of late I haven't been as engaged in this working group so I have 
likely missed decision points that render the following observations 
moot, but there have been some things I've seen on the list that I felt 
I needed to pause to point out.

1) Scope creep.
There has been some good work going on with items that are important to 
security management, but I have to wonder how many of these are 
completely necessary to standardize the ability to automate a security 
assessment?

In the very simplest case an assessment requires have the ability to 
express the desired state of an endpoint in an automatable format, the 
ability to collect the set of attributes necessary for that computation, 
and the ability to unambiguously interpret the set of attributes values.

As an example, I will pick on I-D -- and the only reason I am using that 
one is because it's the most recent topic I read a message on.

Strictly speaking, it's not necessary to be able to positively identify 
and endpoint in order to automate a security assessment of it. 
Identification does because important when it comes to network management.

Now, recognizing that SACM exists in a world of enterprise management is 
reasonable, but so is the allowance that any component will be needed to 
be replaced with something better in the future.

So this really leads to two questions:

   A) What are the fundamental set of needs for addressing the charter 
of this working group?

   B) Will picking a particular solution now for any of the ancillary 
components lead to a design that makes it harder to replace them later?

2) Some comments I have seen recently imply that the idea that there is 
a known and finite set of endpoint attributes to be collected.  But this 
is only true within the scope of a particular policy[1].  The collection 
of all policies that are applicable to and endpoint determine the final 
set of attributes, and this set will change as the policies are updated 
and new policies are added.

With a couple of additional points:

   A) On any given day a new zero-day exploit can require the collection 
of an endpoint attribute that had not been previously known.

   B) The final set of attributes is not known until the set of 
applicable policies is known.

it seems apparent that security automation is an inherently top-down and 
dynamic process.

3) There has been quite a bit of talk about the data format and 
information model, and I think these conversations are premature.

What enables automated assessments is the ability to express the 
assessment goal in a machine-executable format.  I would go so far as to 
say that this top-level format is the most immutable item in the SACM 
ecosystem.  That is to say, replacing this format in the future will be 
more costly than updating any other part of SACM.

Starting with that assumption, the definition of that top-level format 
should be the starting point and addressing the needs of that is what 
will drive the data and information models.

Comments?

--gun

[1] For the sake of expediency I use the term "policy" to refer to any 
top-level security automation document.  It might be a security 
compliance policy, such as HIPAA or PCI, known vulnerability sweep, 
malware detection, etc.

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 3 Feb 2015 10:36:21 +0000

> -----Original Message-----
> From: sacm [mailto:[email-masked]] On Behalf Of Gunnar Engelbach
> Sent: Monday, February 02, 2015 8:57 PM
> 
> 

Gunnar,

Thanks for your comments. 

Can you clarify - what is the difference between 'that top level format' and the Information Model? I thought (and I believe this was the consensus of the WG) that the IM IS the (protocol-independent) top level format and data model (which can be expressed in different DMLs) derived from it. What am I missing? 

Regards,

Dan

---

From: Gunnar Engelbach <[email-masked]>
Date: Tue, 03 Feb 2015 12:03:29 -0500

> Can you clarify - what is the difference between 'that top level format' and the Information Model? I thought (and I believe this was the consensus of the WG) that the IM IS the (protocol-independe…

Sure, maybe that is the intent.  If so, I totally missed it.

I read every message that comes across this list, and the impression I 
have been getting is that the Information Model and Data Model apply to 
the collection and distribution of endpoint attributes and not to a 
higher level goal that they are in service to.

--gun

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 3 Feb 2015 17:53:37 +0000

> From: Gunnar Engelbach [mailto:[email-masked]]
> Sent: Tuesday, February 03, 2015 7:03 PM
> To: Romascanu, Dan (Dan); [email-masked]
> Subject: Re: [sacm] Gut Check?
> 
> 
> 
> >> assessment goal in a machine-executable format.  I would go so far as
> >> to say that this top-level format is the most immutable item in the
> >> SACM ecosystem.  That is to say, replacing this format in the future
> >> will be more costly than updating any other part of SACM.
> >>
> >> Starting with that assumption, the definition of that top-level
> >> format should be the starting point and addressing the needs of that
> >> is what will drive the data and information models.
> >>
> >>
> >>
> >> Comments?
> >>
> >
> > Gunnar,
> >
> > Can you clarify - what is the difference between 'that top level format' and
> the Information Model? I thought (and I believe this was the consensus of
> the WG) that the IM IS the (protocol-independent) top level format and data
> model (which can be expressed in different DMLs) derived from it. What am I
> missing?
> 
> 
> I read every message that comes across this list, and the impression I have
> been getting is that the Information Model and Data Model apply to the
> collection and distribution of endpoint attributes and not to a higher level
> goal that they are in service to.
> 
> 

Let me try to focus better my question. What is missing from the current IM proposal in order to reach that 'higher level goal'?

Thanks and Regards,

Dan

---

From: "Waltermire, David A." <[email-masked]>
Date: Tue, 3 Feb 2015 18:00:56 +0000

Comments inline.

-----Original Message-----
From: sacm [mailto:[email-masked]] On Behalf Of Romascanu, Dan (Dan)
Sent: Tuesday, February 03, 2015 12:54 PM
To: Gunnar Engelbach; [email-masked]
Subject: Re: [sacm] Gut Check?

> >> assessment goal in a machine-executable format.  I would go so far 
> >> as to say that this top-level format is the most immutable item in 
> >> the SACM ecosystem.  That is to say, replacing this format in the 
> >> future will be more costly than updating any other part of SACM.
> >>
> >> format should be the starting point and addressing the needs of 
> >> that is what will drive the data and information models.
> >>
> >>
> >>
> >> Comments?
> >>
> >
> > Gunnar,
> >
> the Information Model? I thought (and I believe this was the consensus 
> of the WG) that the IM IS the (protocol-independent) top level format 
> and data model (which can be expressed in different DMLs) derived from 
> it. What am I missing?
> 
> 
> have been getting is that the Information Model and Data Model apply 
> to the collection and distribution of endpoint attributes and not to a 

[daw: I would also like to know what is missing from the charter, use cases, and requirements to define these higher-level goals?]

Thanks and Regards,

Dan

_______________________________________________
sacm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/sacm

---

From: Clifford Kahn <[email-masked]>
Date: Thu, 5 Feb 2015 18:46:28 +0000

Gunnar,
Thanks for raising this. I find it clarifying. Some of us have been down in the weeds, with caterpillars. 

You write:

> assessment goal in a machine-executable format.  

In the SACM model, assessment goals would be expressed as Guidance objects. 
We have not talked much about what is in a Guidance object. Gunnar, I think
that is your concern -- am I right?

However, there are existing standards for expressing assessment goals, including:
	- OVAL (http://oval.mitre.org/) 
	- XCCDF (http://scap.nist.gov/specifications/xccdf/) 
	- Other members of the SCAP family

I think SACM should complement these standards. I think it makes sense to use such notations in Guidance instances. If so, we should make sure that would work: for example, that the information model expresses the inputs that an XCCDF engine needs.

I don't think SACM is about creating a new standard for assessment goals. I think SACM is largely about  *assessment agility*. In response to an emergent threat:

	- One can update guidance, and the new guidance will take immediate effect everywhere it needs to, so that the right attributes are collected and the evaluators check the right things.

	- If that isn't sufficient, one can deploy a new evaluator, an antidote created for the threat. It's not necessary that this new evaluator take its guidance in a standard notation. For agility, we need to encourage both innovation and standardization. 

Today, evaluators are largely bound up with collectors. If I want to apply a new evaluator to my 100,000 endpoints, I need to install it on 100,000 endpoints. That's a massive deployment effort. Because of the variety of platforms in use, I may need a Windows version, a Mac version, an Android version, etc. That's a big software development effort. Some of these platforms accept updates only from app stores. Getting software into an app store takes time. The current reality is far from agile. Decoupling evaluators from collectors will help. 

In conclusion, a good standard for collecting and communicating endpoint attributes will fill a gap. 

But as you say, we need to keep our eye on the higher level goal they are in service to. And so, it would be good to check what endpoint attributes an XCCDF engine (for example) needs, and make sure our model has them. 

Best,
			Cliff

---

From: Kathleen Moriarty <[email-masked]>
Date: Thu, 5 Feb 2015 15:01:42 -0500

>         - Other members of the SCAP family

YANG modules are another possibility and are used everywhere else.
There is a bit of an explosion of them in the IETF and in the open
source community at the moment.  If YANG can be used, it should be
considered.  Not all devices will want to support SCAP.

-Kathleen
contributor hat

-- 

Best regards,
Kathleen

---

From: Gunnar Engelbach <[email-masked]>
Date: Thu, 05 Feb 2015 14:40:24 -0500

Yes, XCCDF and OVAL are examples of what I am referring to, but I don't 
want to presuppose that those are the best way to solve this problem.

We have a nice set of requirements -- let's talk about those and let 
them drive us toward the best solution.

My point is that that decision needs to be made before we talk about 
collection in any detail -- otherwise you don't know how to respond to 
collection requests, what format the collected attributes need to be in 
so they can be properly interpreted, and how to unambiguously identify 
those attributes.

Something else I will point out about OVAL.  It doesn't just specify a 
flat list of attributes to collect, it can also contain logic structures 
that use a collected values to determine what to collect.

As a simple example, if you are interested in what minimal password 
length is being enforced by an endpoint, the correct place to look for 
that setting depends on whether the endpoint is using local 
authentication, YP, NIS/NIS+, Kerberos, etc.

If you develop a collector that is not aware of OVAL as the collection 
language, then it will not be capable of doing that collection.

As far as the cost of updating software on the endpoints goes, I 
entirely agree, and that's why starting at the top before moving down to 
the attributes is so important.

The set of attributes to collect is not a finite known set.  The list 
will change as new vulnerabilities are discovered, new software 
installed, the role of the endpoint changes, and so on.  In order to 
minimize the frequency with which the collection software needs to be 
updated, collection needs to be able to be done in a dynamic manner 
where what to collect and how to collect it is driven by the Guidance.

A secondary factor in that process is accuracy.  Collectors will be 
built by multiple vendors and their results can easily be different if 
the "what and how" of collection is not kept consistent.  If that "what 
and how" is a set of instructions provided by the Guidance, consistency 
between implementations becomes much less of an issue.