Subject: Re: [sacm] BOF Status

From: Gunnar Engelbach <[email-masked]>
Date: Sat, 09 Feb 2013 22:28:54 -0500

Finally getting back to this, I scribbled together some basic drawings 
to visualize my cheese text table.  The PNG images are duplicates of the 
SVG images, but more likely to be viewable whereas the SVG images 
preserve the ability to scale parts of the drawing if anybody else wants 
to muck with them.

1:  Represents the items in the table with connectors 
to show communication paths.  The two dashed connectors are the options 
indicated by notes 1 & 2 in the original table.

2:  The same drawing by showing a possible 
software implementation where multiple entities from (1) are grouped 
together in two separate software components in a manner that closely 
resembles Dave's architecture.

3:  A much busier image where the rounded green boxes 
represent individual computers.  Intended to show how entities from (1) 
can be mixed together in various combinations to produce a full 
solution.  Communication connections on the same computer are removed 
for clarity, but all connectors indicated are the same as in (1) with 
the exception of the dotted connectors for agentless data collection. 
Note also that there is a degree of duplication in this drawing. 
Software entities at lower levels in this drawing are not necessarily 
intended to be the same implementation as others, but may be different 
implementations depending on the scale of use at that level.

In all images I have again omitted interactions with content 
repositories because it was just getting too busy in there, but I would 
expect them to also live at various levels in the enterprise, from a 
simple one for caching applicable content on an independent 
workstations, ones local to a subnet, master ones for an enterprise, as 
well as public aggregators and top-level definitive public repositories.

On 1/31/2013 5:28 PM, Waltermire, David A. wrote:
> I am working on incorporating our discussions on requirements and definitions into the use case document.  I also have a stubbed out architecture document that I plan to put the architecture discussion content into.  I hope to have an update posted of the use case document next week and a first draft of the architecture document on the 13th (this beats the 2/18 deadline for initial drafts).  My hope is that we can turn around at least one more revision of each based on additional conversation before 2/25 which is the I-D cutoff date.
>
> I am planning to base the architecture document on the following discussion threads and the notional picture (attached):
>
> http://www.ietf.org/mail-archive/web/sacm/current/msg00731.html (plus thread)
> http://www.ietf.org/mail-archive/web/sacm/current/msg00884.html
>
> If you have any comments/concerns with what is described in these threads, please raise them now.
>
> Does this sound like a workable plan?
>
> Thanks,
> Dave
>
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Romascanu, Dan (Dan)
> Sent: Thursday, January 31, 2013 5:10 PM
> To: Sean Turner; [email-masked]
> Subject: Re: [sacm] BOF Status
>
> Great news and thanks for supporting this, Sean!
>
> Now we need to do some serious work as the second BOF (under the same scope) is also the last one allowed under the current rules - so failure is not an option.
>
> Updating the Internet-Drafts seems to be the next most important step in the process, so that the discussions in Orlando are based on well-written documents that also reflect the discussions on architecture and requirement and the better focused achieved by the discussions on the list.
>
> Regards,
>
> Dan
>
> ________________________________________
> From: [email-masked] [[email-masked]] on behalf of Sean Turner [[email-masked]]
> Sent: Thursday, January 31, 2013 8:18 PM
> To: [email-masked]
> Subject: [sacm] BOF Status
>
> The SACM BOF for IETF 86 was approved.  Chairs are Kathleen Moriarty and Dan Romascanu (same as last time).  I added additional conflicts to allow those involved in SCIM/NETMOD can make it if they so choose.
>
> Scheduling of the actual meeting slot occurs later in the process.
>
> Looking forward to seeing you all there (or in the jabber room).
>
> spt
> _______________________________________________
> sacm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sacm
> _______________________________________________
> sacm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sacm
>
>
>
> _______________________________________________
> sacm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sacm
>

---

From: "Waltermire, David A." <[email-masked]>
Date: Sun, 10 Feb 2013 16:53:18 -0500

Gunnar,

This is helpful. I am working on the architecture document today and I have a comments on what you provided. Looking at your diagrams, there is a built-in assumption that an evaluator is responsible for requesting and processing any collected data before it is persisted. In my mind, it would work the other way around. 

The controller would potentially receive requests from an evaluator indicating any information that should be collected (assuming existing data is insufficient).  The controller would orchestrate collection of the data and then the evaluator would access that data. The advantage of this approach is that multiple evaluators can use the same data for analysis and the controller would be in a situation where it could optimize the degree of data collection to fit the needs of many evaluators.

In a simple example the controller could be an application managing collection activities.  In more advanced cases it could be a Policy Decision Point that is making authorization decisions based on collected data. In the latter case you could say that the evaluation and controller functions are being combined into the same capability. In either case, my assumption is that any collected data is persisted allowing for further analysis processes to be performed by external capabilities. In general, I see this as a good thing.

This approach would require somewhere in the order of 4 interfaces (data flows) to be standardized (i.e. content retrieval, collection tasking, collected data publication, and data query).

What do you or others think of this approach?

Sincerely,
Dave

-----Original Message-----
From: Gunnar Engelbach [mailto:[email-masked]] 
Sent: Saturday, February 09, 2013 10:29 PM
To: Waltermire, David A.
Cc: Romascanu, Dan (Dan); Sean Turner; [email-masked]
Subject: Re: [sacm] BOF Status

Finally getting back to this, I scribbled together some basic drawings to visualize my cheese text table.  The PNG images are duplicates of the SVG images, but more likely to be viewable whereas the SVG images preserve the ability to scale parts of the drawing if anybody else wants to muck with them.

1:  Represents the items in the table with connectors to show communication paths.  The two dashed connectors are the options indicated by notes 1 & 2 in the original table.

2:  The same drawing by showing a possible software implementation where multiple entities from (1) are grouped together in two separate software components in a manner that closely resembles Dave's architecture.

3:  A much busier image where the rounded green boxes represent individual computers.  Intended to show how entities from (1) can be mixed together in various combinations to produce a full solution.  Communication connections on the same computer are removed for clarity, but all connectors indicated are the same as in (1) with the exception of the dotted connectors for agentless data collection. 
Note also that there is a degree of duplication in this drawing. 
Software entities at lower levels in this drawing are not necessarily intended to be the same implementation as others, but may be different implementations depending on the scale of use at that level.

In all images I have again omitted interactions with content repositories because it was just getting too busy in there, but I would expect them to also live at various levels in the enterprise, from a simple one for caching applicable content on an independent workstations, ones local to a subnet, master ones for an enterprise, as well as public aggregators and top-level definitive public repositories.

---

From: Gunnar Engelbach <[email-masked]>
Date: Sun, 10 Feb 2013 20:07:51 -0500

> This is helpful. I am working on the architecture document today and I have a comments on what you provided. Looking at your diagrams, there is a built-in assumption that an evaluator is responsibl…

Not necessarily.  The communications are two-way.  The originator of the 
communications might be the Data Collector, such as in a system that is 
event-driven where detected changes on the endpoint are evaluated and 
results made available in NRT.  An example of that type of usage might 
be to trigger the network to isolate the endpoint for remediation or 
terminate a VPN connection.

But there are also on-demand assessments, such as scheduled, 
user-initiated, automatically triggered by the reporting of a new CVE 
entry, etc, in which case the data collection could be initiated by the 
Evaluator which in turn received its marching orders from the Controller.

Also, a Data Collector might not necessarily be something that collects 
artifacts directly from an endpoint.  It might be a software component 
that can query a CMDB and translate that result set into the proper SACM 
protocol.  Or it might be a human being poking in data from a 
vendor-provided data sheet.

Whether it's [Controller <--> Evaluator <--> Data Collector] or 
[Evaluator <---> Controller <--> Data Collector], I think both can be 
made to work.

The important thing to me at this point is to have at least the minimum 
set of entity roles I've broken out.  But I've got no loyalty to the 
communication lines I've indicated.  So everything I say in this 
response is not to push a particular vision, but just trying to talk 
through the implications.

Yes, the ability to persist the collected data and use it other places 
would be very useful.  That's a good argument for flowing collected 
items through the Controller, although an alternative would be to 
include the collected items along with the results produced by the 
evaluator.  On the other hand, in the simple case such as an all-in-one 
desktop application, that's an unnecessary indirection.  In that 
particular case, though, I could see the controller and evaluator being 
the same thing.

While a data store for collected endpoint artifacts is not part of the 
current charter, I completely agree that that's an additional component 
we need to be able to feed out of this.

In favor of having collected data flow through the Evaluator is that we 
typically are concerned with the level of compliance to a policy (HIPAA, 
SOX, USGCB, local security policy, whatever).  A policy is a set of 
multiple rules where quite often the result of one rule depends on the 
outcome of a different rule.  All data items must be present to 
completely the policy analysis but that set of items might be different 
from machine to machine depending on the outcome of one of the rules. 
Collecting everything that could possibly be needed is overkill, missing 
something results in an undetermined level of policy compliance.  The 
best way I can see to optimize this is to make evaluation interactive 
with collection and that means putting control of collection in the 
hands of the Evaluator.

A few specific comments about your version:

1) You show the Results Viewer communicating only with the evaluator 
with no indication of permanent storage.  Implying the only thing able 
to be viewed is the results of a live assessment (historical assessments 
being provided by doing a replay of an assessment using collected data 
from a prior point in time).  Is that the intent?

2) You have the Data Collector talking directly to the Content 
Repository.  If the idea of the benchmarks is to remain high-level (for 
example "The chargen service must be inactive") and upon receiving that 
direction the collector uses the Content Repository to get the set of 
instructions for how to do that on its platform of interest (id est, 
Windows vice OSX vice XBOX ...) -- that's an interesting approach worth 
talking about, but one I think would be needlessly inefficient and 
complex and require some sophistication in the data collector. 
Personally, I think the data collector should be as dead simple as possible.

3) You show the data collector possibly talking directly to Data 
Storage.  Again, I think this would require some level of sophistication 
on the part of the collector and increases the number of data paths as a 
solution is implemented -- especially if there are multiple data stores 
to feed.  Considering that authentication and data integrity should be a 
part of every communication and that collection happens for every single 
endpoint, that would mean being able to push data store details (network 
address/port, description of data subscribed to) and authentication 
information out to every point of collection for every data store.

4) You show the Evaluator making Collection Requests to the Controller 
and getting data back.  That, by itself, might be a good thing, but then 
there isn't really a need for the Evaluator to communicate directly with 
the artifact data store.  By itself that's just an extra line of 
communicate, but consider a case where there are multiple data stores 
and evaluation of a policy requires getting items from each (and yes, I 
can think of good examples where that could be the case, but I'll forgo 
that because I've blabbed on enough already).  In that case each 
evaluator has to know about each data store, how to communicate with it, 
and what the scope of data each contains is.  I think that type of 
knowledge belongs at the Controller and not the Evaluator level.

4) I also see things like scheduling living at the level of the 
controller for recurring assessments.  As you show, the Controller could 
certain push collection data to the evaluator on a set schedule in order 
to re-evaluate a policy (or do an on-demand policy evaluation) but the 
knowledge of what items need to be collected to do that are determined 
by the policy as well as the target platform and it would be the 
Evaluator that is capable of making that determination.

I've also been thinking about another way to redefine all this in terms 
of interfaces.  An Evaluator, for example, can be a Data Item Consumer, 
a Data Item Requester, a Controllable, and a Results Producer.  A 
Controller could also implement Data Item Requester and Data Item 
Provider, and a Data Collector would implement Data Item Provider at a 
minimum.

Going with that approach, the actual implementation of each entity can 
vary and both architectures we've shown could be supported, depending on 
which implementations were mixed and matched together.

So if the implementation of a Content Repository also implemented the 
Data Item Consumer and and Results Producer interfaces it could also be 
used as an Evaluator.

Where this approach gets difficult is for the poor enterprise admin who 
has to fit together an assortment of software components to build a 
complete system, but I think we're going to end up with that problem 
regardless.