Subject: [sacm] review of draft-camwinget-sacm-requirements-01.txt

From: "ietfdbh" <[email-masked]>
Date: Mon, 21 Oct 2013 19:15:21 -0400

Hi,

I have reviewed this document and have some comments.

1) section 2 - update to reference the ietf-sacm-terminology draft.

2) section 3.1 uses Asset in the diagram. Since we are focusing on endpoint
assessment, should the diagram utilize endpoint rather than asset? Asset
evaluation might be a larger scope than the endpoints we are discussing.

3)"Evaluator is the service that affects ..."; Dave and I have been trying
to make the language less ambiguous by using "collects" and "evaluates". I
don't know what "affects" means here. There is also the question of whether
the collector and the evaluator are necessarily the same entity. 

4) section 3.1 states that the focus will be the data models, protocols, and
transport for B. I haven't seen WG consensus that this is the limit of our
scope. I have recently edited quite a few use cases that cover more scope
than B.

5) There is an "Abbreviated Title" on every page. Please edit the xml2rfc
<title> field (assuming the source is in xml2rfc format).

6) In 3.3, there are places with "> characters at the end of the lines. 

7) We also should run a spellchecker when  possible. s/recieve/receive/
See http://tools.ietf.org/tools/idspell/webservice

8) section 3.2 states a number of requirements that I have not seen
discussed on the WG mailing list, or in the use cases submitted for
inclusion in the use-cases document. I thought the goal was to have the use
cases drive the requirements. 

9) section 3.3 Discovery says that evaluators and consumers should be
registered. I haven't seen use cases that claim a need for such a thing. Is
this requirement about a central registration database or something? Who
would be responsible for maintaining such a database? I'm not sure we'll get
IETF consensus for such a central authority for sacm. Or is this thinking in
terms of existing standards, such as dns or dhcp, or something similar? 

10) There are requirements for "lightweight representations" and "bulk
updates". I wonder if these "requirements" might be better described in use
cases, rather than in requirements. Part of the problem I see is that large
databases might be repositories used by an evaluator or consumer, where the
large database would  likely be local to the consumer. But the collector of
attributes from an endpoint might utilize one or more lightweight
protocol/data model to collect from endpoints, and the large database might
represent the aggregation of multiple lightweight queries. Until we begin to
look at specific data models, scaling might not become a requirement. For
example, SNMP is a lightweight protocol, and MIB modules are typically
lightweight data models, but they are often used to manage very large
networks, aggregating data into large local databases. Where bulk updates of
large collections of attributes, such as routing tables, are needed,
protocols such as Netconf might be more effective. Where speed is of the
essence, a lightweight notification protocol such a syslog might be
effective. But these features are not required of all
implementations/deployments (i.e. use cases). So I recommend describing the
use cases that require these features, rather than simply declaring them
requirements, so we can discuss which "requirements" apply to which use
cases, and why.

11) Some of these requirements seem to be "Mom and Apple Pie" types of
requirements. We might want to simply build a list of requirements that MUST
be met if we expect the IETF/IESG to approve data models or protocols or
transports recommended by sacm WG. The thing is, most existing IETF
standards already support internationalization (where appropriate),
scalability, interoperability, implementation of strong security, deployment
of varied levels of security (as applicable to the environment), discovery
(when appropriate), extensible data modeling, and so on. So I'm not sure we
need to list these, but it might be helpful to readers coming from outside
the IETF.

These are normal requirements or recommendations for IETF standardization,
and are typically documented in some RFCs (BCPs) already. I can help to
identify a number of them, such as RFC 3365, 5706, 7013, 6684, 6087, 4290,
4181, 3439, 5405, 6365, and so on. (Some of these are guidelines for using
specific protocols or designing data models, but could be applicable to new
designs as well).

What we really need to work on understanding are requirements of specific
use cases, such as endpoints reporting information subject to privacy
regulations such as medical databases, or querying attributes from
endpoints that have constrained resources like building automation sensors,
or querying information from endpoints with slow or intermittent
connectivity, such as an arctic research station or a space station.

--
Maybe we need to extract the architectural requirements from the use cases
to start, then develop a general proposed architecture to model the
many-to-many relationships and potential domain crossings, and so on
discussed in the use cases. Then we could move onto requirements for the
application protocols between architectural entities (e.g., bulk retrieval,
asynchronous notifications,etc.), the requirements for the underlying
transports (e.g., security, reliability), and the requirements for data
models (e.g., internationalization)

David Harrington
[email-masked]
+1-603-828-1401

---

From: "Watson, Kimberly" <[email-masked]>
Date: Tue, 22 Oct 2013 20:17:57 +0000

I strongly agree with points 3 and 4 below.

Once you separate the actions of collection and evaluation - and I think you must to ensure flexibility and scalability in posture assessment processes and capabilities - you must extend the scope to data models for A and possibly even C/repository at a minimum. Most of the use cases expound on the need to evaluate data already collected to assess a newly-defined posture requirement/concern. If you do not extend to A (and maybe C - I can't tell if defining A is enough to allow for querying/use of data in the evaluator repository by other processes) then you can indirectly tie deploying/implementing  a new evaluator to collect the information required for a new assessment even if you already have collected that data in a different evaluator. 

If I look at your simplistic architecture picture, all data collected and stored/passed to an evaluator must use standard data models such that all evaluators store this data the same for multiple assessment consumers to request evaluation from multiple evaluators and get consistent answers. In my understanding B is about asking questions/directing evaluation in a standardized way and receiving standardized answers. I need to know that the conversion of information collected using A has been translated/stored in a standardized fashion to ensure that the evaluation performed by different evaluators collecting the same data will produce the same assessment result.

Being new to this forum and having read only the three new documents, it is quite possible that I am misunderstanding something and expressing a concern that isn't valid...but I was very surprised after reading all the use cases to see the architecture picture expressly state evaluation but not collection, and limit scope to just B. my current operational issues are in standardizing the collection and storage of endpoint posture attributes for reuse by multiple assessment processes that are constantly defining new criteria and policy for evaluation and posture validation. Most of the use cases appear to have this same need at their core.
--
Kim Watson
Office of Cybersecurity and Communications
Federal Network Resilience
Blackberry: (202) 494-2534

-----Original Message-----
From: ietfdbh [mailto:[email-masked]] 
Sent: Monday, October 21, 2013 7:15 PM
To: [email-masked]
Subject: [sacm] review of draft-camwinget-sacm-requirements-01.txt

Hi,

2) section 3.1 uses Asset in the diagram. Since we are focusing on endpoint assessment, should the diagram utilize endpoint rather than asset? Asset evaluation might be a larger scope than the endpoints we are discussing.

3)"Evaluator is the service that affects ..."; Dave and I have been trying to make the language less ambiguous by using "collects" and "evaluates". I don't know what "affects" means here. There is also the question of whether the collector and the evaluator are necessarily the same entity. 

4) section 3.1 states that the focus will be the data models, protocols, and transport for B. I haven't seen WG consensus that this is the limit of our scope. I have recently edited quite a few use cases that cover more scope than B.

5) There is an "Abbreviated Title" on every page. Please edit the xml2rfc <title> field (assuming the source is in xml2rfc format).

7) We also should run a spellchecker when  possible. s/recieve/receive/ See http://tools.ietf.org/tools/idspell/webservice

8) section 3.2 states a number of requirements that I have not seen discussed on the WG mailing list, or in the use cases submitted for inclusion in the use-cases document. I thought the goal was to have the use cases drive the requirements. 

9) section 3.3 Discovery says that evaluators and consumers should be registered. I haven't seen use cases that claim a need for such a thing. Is this requirement about a central registration database or something? Who would be responsible for maintaining such a database? I'm not sure we'll get IETF consensus for such a central authority for sacm. Or is this thinking in terms of existing standards, such as dns or dhcp, or something similar? 

10) There are requirements for "lightweight representations" and "bulk updates". I wonder if these "requirements" might be better described in use cases, rather than in requirements. Part of the problem I see is that large databases might be repositories used by an evaluator or consumer, where the large database would  likely be local to the consumer. But the collector of attributes from an endpoint might utilize one or more lightweight protocol/data model to collect from endpoints, and the large database might represent the aggregation of multiple lightweight queries. Until we begin to look at specific data models, scaling might not become a requirement. For example, SNMP is a lightweight protocol, and MIB modules are typically lightweight data models, but they are often used to manage very large networks, aggregating data into large local databases. Where bulk updates of large collections of attributes, such as routing tables, are needed, protocols such as Netconf might be more effective. Where speed is of the essence, a lightweight notification protocol such a syslog might be effective. But these features are not required of all implementations/deployments (i.e. use cases). So I recommend describing the use cases that require these features, rather than simply declaring them requirements, so we can discuss which "requirements" apply to which use cases, and why.

11) Some of these requirements seem to be "Mom and Apple Pie" types of requirements. We might want to simply build a list of requirements that MUST be met if we expect the IETF/IESG to approve data models or protocols or transports recommended by sacm WG. The thing is, most existing IETF standards already support internationalization (where appropriate), scalability, interoperability, implementation of strong security, deployment of varied levels of security (as applicable to the environment), discovery (when appropriate), extensible data modeling, and so on. So I'm not sure we need to list these, but it might be helpful to readers coming from outside the IETF.

These are normal requirements or recommendations for IETF standardization, and are typically documented in some RFCs (BCPs) already. I can help to identify a number of them, such as RFC 3365, 5706, 7013, 6684, 6087, 4290, 4181, 3439, 5405, 6365, and so on. (Some of these are guidelines for using specific protocols or designing data models, but could be applicable to new designs as well).

What we really need to work on understanding are requirements of specific use cases, such as endpoints reporting information subject to privacy regulations such as medical databases, or querying attributes from endpoints that have constrained resources like building automation sensors, or querying information from endpoints with slow or intermittent connectivity, such as an arctic research station or a space station.

--
Maybe we need to extract the architectural requirements from the use cases to start, then develop a general proposed architecture to model the many-to-many relationships and potential domain crossings, and so on discussed in the use cases. Then we could move onto requirements for the application protocols between architectural entities (e.g., bulk retrieval, asynchronous notifications,etc.), the requirements for the underlying transports (e.g., security, reliability), and the requirements for data models (e.g., internationalization)

David Harrington
[email-masked]
+1-603-828-1401

---

From: Gunnar Engelbach <[email-masked]>
Date: Tue, 22 Oct 2013 19:27:34 -0400

It sounds like your perspective on this is very close to what most of us 
have been talking about since the inception of the SACM working group. 
Some of the early discussions and architecture suggestions might be 
worth a review.

Right now we're trying to not let those initial preconceptions bias us, 
instead focusing on consensus of the use cases, then letting those drive 
requirements and the eventual architecture.

What Nancy submitted is a necessary placeholder to meet documentation 
deadlines, but it will change as this requirements process bobs down the 
waterfall.

And with those statements out of the way, I'd like to make special note 
of two statements you made in particular, paraphrased, because I don't 
think they are apparent from the existing use cases:

1) Separation of collection, evaluation [, etc.], is key to flexibility 
and scalability -- and I would add increases the ability to interact 
with existing software.

2) It's not just standardizing the data flows between components [id 
est, collector, evaluator, controller, etc.] that is important, but also 
standardization of the content that drives collection and evaluation. 
For exactly the reasons you give, among others.  I'd also go further and 
say the content needs to be public -- otherwise it might as well be 
proprietary.

On 10/22/2013 4:17 PM, Watson, Kimberly wrote:
> 10) There are requirements for "lightweight representations" and "bulk updates". I wonder if these "requirements" might be better described in use cases, rather than in requirements. Part of the problem I see is that large databases might be repositories used by an evaluator or consumer, where the large database would  likely be local to the consumer. But the collector of attributes from an endpoint might utilize one or more lightweight protocol/data model to collect from endpoints, and the large database might represent the aggregation of multiple lightweight queries. Until we begin to look at specific data models, scaling might not become a requirement. For example, SNMP is a lightweight protocol, and MIB modules are typically lightweight data models, but they are often used to manage very large networks, aggregating data into large local databases. Where bulk updates of large collections of attributes, such as routing tables, are needed, protocols such as Netconf might b
 e
>   more effective. Where speed is of the essence, a lightweight notification protocol such a syslog might be effective. But these features are not required of all implementations/deployments (i.e. use cases). So I recommend describing the use cases that require these features, rather than simply declaring them requirements, so we can discuss which "requirements" apply to which use cases, and why.
>
> _______________________________________________
> sacm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sacm
>

---

From: "Watson, Kimberly" <[email-masked]>
Date: Wed, 23 Oct 2013 19:47:29 +0000

I think going from use case to requirements to architecture makes sense, and in many ways why I didn't read the current architecture draft. 

As for your comments that my statements are not apparent from the use cases, I went back to see why I felt they were ...and I noticed that my assumptions/assertions are derived from the discussion and workflow of end point posture assessments and not the use cases.

Separation of Collection and Evaluation is explicit in the statement about what an endpoint posture assessment typically includes and in the fact that the workflow (WF) has 5, 6, & 7 as steps. While the use cases do not identify 3 versions of the same use case where you have them separate (WF 5), combined (WF 6), or accessible by multiple evaluators or other processes (WF 7), the breakout implied to me that it is a requirement that the two are not bound together.

When you break that binding, then the expression of: what to collect, how to refer to/store posture attributes, what to evaluate against, how to evaluate, and how to provide results MUST be standardized and public. I felt that B can handle the last three, but ignored the first two. I'm not sure what you mean by content (it is a very loaded term in security automation discussions), but I completely agree that it is more than the data flow that needs to be standardized. This appears to be a higher level requirement tied to the separation of collection and evaluation, and to serve the purpose as defined early in the document: Typically, endpoint posture information is gathered and then
published to appropriate data repositories to make collected information available for further analysis supporting organizational security processes.

So neither of these are articulated in the use cases, but are necessary to support the larger goal of scalable, flexible, and evolvable endpoint posture assessment as defined earlier in the document.

Do they need to be expressly in the use cases? Or do we just need to agree that they are necessary characteristics or attributes of solutions to the problem? Can we derive requirements from the description of what endpoint posture assessment is or the workflow? Or can they only come from the use cases?
--
Kim Watson
Office of Cybersecurity and Communications
Federal Network Resilience
Blackberry: (202) 494-2534

-----Original Message-----
From: Gunnar Engelbach [mailto:[email-masked]] 
Sent: Tuesday, October 22, 2013 7:28 PM
To: Watson, Kimberly; [email-masked]
Subject: Re: [sacm] review of draft-camwinget-sacm-requirements-01.txt

It sounds like your perspective on this is very close to what most of us have been talking about since the inception of the SACM working group. 
Some of the early discussions and architecture suggestions might be worth a review.

Right now we're trying to not let those initial preconceptions bias us, instead focusing on consensus of the use cases, then letting those drive requirements and the eventual architecture.

What Nancy submitted is a necessary placeholder to meet documentation deadlines, but it will change as this requirements process bobs down the waterfall.

And with those statements out of the way, I'd like to make special note of two statements you made in particular, paraphrased, because I don't think they are apparent from the existing use cases:

1) Separation of collection, evaluation [, etc.], is key to flexibility and scalability -- and I would add increases the ability to interact with existing software.

2) It's not just standardizing the data flows between components [id est, collector, evaluator, controller, etc.] that is important, but also standardization of the content that drives collection and evaluation. 
For exactly the reasons you give, among others.  I'd also go further and say the content needs to be public -- otherwise it might as well be proprietary.

On 10/22/2013 4:17 PM, Watson, Kimberly wrote:
> Office of Cybersecurity and Communications Federal Network Resilience
> 10) There are requirements for "lightweight representations" and "bulk updates". I wonder if these "requirements" might be better described in use cases, rather than in requirements. Part of the problem I see is that large databases might be repositories used by an evaluator or consumer, where the large database would  likely be local to the consumer. But the collector of attributes from an endpoint might utilize one or more lightweight protocol/data model to collect from endpoints, and the large database might represent the aggregation of multiple lightweight queries. Until we begin to look at specific data models, scaling might not become a requirement. For example, SNMP is a lightweight protocol, and MIB modules are typically lightweight data models, but they are often used to manage very large networks, aggregating data into large local databases. Where bulk updates of large collections of attributes, such as routing tables, are needed, protocols such as Netconf might be

---

From: Gunnar Engelbach <[email-masked]>
Date: Wed, 23 Oct 2013 17:30:19 -0400

Comments inline...

> published to appropriate data repositories to make collected information available for further analysis supporting organizational security processes.

For now, how about we say that content is the document or documents that 
provide the instructions for an automated assessment to occur -- 
generally organized to implement a discrete action such as compliance 
policy adherence or a vulnerability assessment.

But this makes me wonder if it is necessary to know the disposition of 
the content in order to define the data flows and components in SACM. 
Are they interdependent, or will the SACM data flows be effective while 
treating the content as generic payload?

Further, I indicated earlier my impression that most of the active 
participants have a similar view of what SACM looks like.  I would like 
to hear more from others who do not because I think that's where were 
more likely to find considerations that have thus far escaped notice.

> Do they need to be expressly in the use cases? Or do we just need to agree that they are necessary characteristics or attributes of solutions to the problem? Can we derive requirements from the des…

That works for me, but that's easy for me to say because I already agree 
with them as requirements.

Having a use case exemplifying the need would be useful.  Would you care 
to submit one?

---

From: "Watson, Kimberly" <[email-masked]>
Date: Thu, 24 Oct 2013 13:24:55 +0000

I can try to develop a use case...but what I realized last night is that a use case that would highlight these requirements will be expressing a "How" not a "What" which is normally a bad thing because it can bias the requirements/architecture derived from those use cases to a particular solution. 

I personally think it is best to keep the "How" part out of the use cases, but define characteristics that any/all solutions must have to ensure scalability, flexibility, and support incremental improvement without redesign (what I tend to call evolution).

If we specify that any solution to a use case must be able to: 
1. perform the assessment from collection to evaluation in one request
2. perform the collection at a previous point in time and then have the associated evaluator assess upon request
3. perform the collection at a previous point in time and then have ANY evaluator assess upon request

This creates 2 high level requirements (probably more, but we are talking about these two):

1. Collection and Evaluation are decoupled
2. there must be standardization and transparency in the expression of collection as well as in the expression of evaluation

At this point it is probably best to find out if anyone disagrees with the assertion that separating collection from evaluation is a necessary characteristic of solutions that address the use cases.

-----Original Message-----
From: Gunnar Engelbach [mailto:[email-masked]] 
Sent: Wednesday, October 23, 2013 5:30 PM
To: Watson, Kimberly; [email-masked]
Subject: Re: [sacm] review of draft-camwinget-sacm-requirements-01.txt

Comments inline...

> Do they need to be expressly in the use cases? Or do we just need to agree that they are necessary characteristics or attributes of solutions to the problem? Can we derive requirements from the des…

That works for me, but that's easy for me to say because I already agree with them as requirements.

Having a use case exemplifying the need would be useful.  Would you care to submit one?

---

From: Adam Montville <[email-masked]>
Date: Thu, 24 Oct 2013 14:07:52 +0000

This has been a very good discussion to track.  I agree with the notion that use cases shouldn't really consider the "how" (and, IMO, the use case document could stand to be cleaned up a bit in this way, though it may not be necessary at this point).  Perhaps a use case that would (seem to) clearly require would be one that considers assessment of embedded devices which may have the ability to report configuration but not necessarily perform evaluation on-board.  I'm thinking in terms of the "Internet of things."  The Ice Station Zebra use case might be another, where an evaluation component may not be permissible in the environment (separation of duties, least priv, that sort of thing), but the collection still has to happen.  Just throwing out ideas.

Either way, the Requirements document is one where we're going to see most of this stuff shake out, and I think it will serve to help us refine the use cases.

At present, I'm working on a detailed review of the Use Case document and trying to derive some potential requirements from it.  Once I'm done with that, I'll float it to the list here as something to complement the existing Requirements draft and kindle some discussion.

Adam

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Watson, Kimberly
> Sent: Thursday, October 24, 2013 8:25 AM
> To: Gunnar Engelbach; [email-masked]
> I can try to develop a use case...but what I realized last night is that a use case
> that would highlight these requirements will be expressing a "How" not a
> "What" which is normally a bad thing because it can bias the
> requirements/architecture derived from those use cases to a particular
> solution.
> 
> I personally think it is best to keep the "How" part out of the use cases, but
> define characteristics that any/all solutions must have to ensure scalability,
> flexibility, and support incremental improvement without redesign (what I
> tend to call evolution).
> 
> 1. perform the assessment from collection to evaluation in one request 2.
> perform the collection at a previous point in time and then have the
> associated evaluator assess upon request 3. perform the collection at a
> previous point in time and then have ANY evaluator assess upon request
> 
> 1. Collection and Evaluation are decoupled 2. there must be standardization
> and transparency in the expression of collection as well as in the expression
> of evaluation
> 
> At this point it is probably best to find out if anyone disagrees with the
> assertion that separating collection from evaluation is a necessary
> characteristic of solutions that address the use cases.
> 
> --
> Kim Watson
> > So neither of these are articulated in the use cases, but are necessary to
> support the larger goal of scalable, flexible, and evolvable endpoint posture
> assessment as defined earlier in the document.
> >
> > Do they need to be expressly in the use cases? Or do we just need to agree
> that they are necessary characteristics or attributes of solutions to the
> problem? Can we derive requirements from the description of what
> endpoint posture assessment is or the workflow? Or can they only come
> from the use cases?
> 
> ...

. . .

---

From: Adam Montville <[email-masked]>
Date: Thu, 24 Oct 2013 14:59:29 +0000

> Adam Montville
> Sent: Thursday, October 24, 2013 9:09 AM
> To: Watson, Kimberly; Gunnar Engelbach; [email-masked]
> This has been a very good discussion to track.  I agree with the notion that
> use cases shouldn't really consider the "how" (and, IMO, the use case
> document could stand to be cleaned up a bit in this way, though it may not
> be necessary at this point). 

For the sake of providing an example with respect to my comment about how the use case document could be cleaned up a bit, I happened to be reviewing 2.12 (Traditional Endpoint Assessment with Stored Results) just now.  It reads:

   "An external trigger initiates an assessment of an endpoint.  The
   Controller uses the data in the Datastore to look up authentication
   information for the endpoint and passes that along with the
   assessment request details to the Evaluator.  The Evaluator uses the
   Endpoint information to request taxonomy information from the
   Collector on the endpoint, which responds with those attributes.  The
   Evaluator uses that taxonomy information along with the information
   in the original request from the Controller to request the
   appropriate content from the Content Repository.  The Evaluator uses
   the content to derive the minimal set of endpoint attributes needed
   to perform the assessment and makes that request.  The Evaluator uses
   the Collector response to do the assessment and returns the results
   to the Controller.  The Controller puts the results in the Datastore."

IMO, there should probably be no discussion in this use case of a Datastore, Controller, Content Repository.  By my read, this use case intends to say: "The system, triggered externally, targets an endpoint for characterization (inventory?), so that the system can determine the minimal set of endpoint attributes to collect and store for subsequent use."

>From here, we can derive requirements such as:

- The system must support assessment triggers from external sources
- The system should support authentication for external assessment triggers
- The system should support authorization for external assessment triggers
- The system must support target endpoint inventory (I think this is what was intended by requesting "taxonomy")
- The system must support checklist composition based on target endpoint inventory
- The system must support composite checklist optimization (i.e. remove duplicate collection expressions)
- The system must make assessment results available for subsequent use

Architectures would then get into the "how" of it all, which is where I would see Collectors, Datastores, Evaluators, etc. come into play.  There's more than one way to satisfy the requirements which are derived from this particular use case. 

The Use Case, as written, gets the point across, but I think it does so superfluously.  Maybe it's just a matter of personal taste or preference, but I think a sentence or two would capture the intent of this use case without implying or assuming any particular architectural solution.  This may or may not be the case with each other use case, though from what I've been through so far, I think most could stand to be pruned.

Adam

. . .

---

From: Gunnar Engelbach <[email-masked]>
Date: Thu, 24 Oct 2013 12:05:08 -0400

We've been getting a bit pedantic lately in having use cases drive 
requirements -- a step backward into the old waterfall development 
model, but a reasonable regression given the unique way development 
progresses in this working group.

But, bottom line, even without a use case we can come up with reasonable 
examples showing the need for these requirements.  It's just a matter of 
consensus on whether these examples/requirements fall within the scope 
of this working group.

Still, I'd like to see the progression documented in some manner and not 
have new things appearing parthenogenically in the requirements 
document.  Perhaps a section in the use cases document for expected 
usage scenarios not already covered by the use cases?

On 10/24/2013 10:07 AM, Adam Montville wrote:
> This message and attachments may contain confidential information. If it appears that this message was sent to you by mistake, any retention, dissemination, distribution or copying of this message and attachments is strictly prohibited. Please notify the sender immediately and permanently delete the message and any attachments.
>
> . . .
>

---

From: "ietfdbh" <[email-masked]>
Date: Thu, 24 Oct 2013 12:31:07 -0400

Hi,

>From the Intro to use-cases:
   Our goal with this document is to improve our agreement on which
   problems we're trying to solve.  We need to start with short, simple
   problem statements and discuss those by email and in person.  Once we
   agree on which problems we're trying to solve, we can move on to
   propose various solutions and decide which ones to use.

So one of the issues I faced as editor of use-cases was whether to prune and
rewrite use cases.
The use cases have been submitted to describe how somebody uses security
automation.
If I rewrite the use case to remove concepts such as repository, controller,
collectors, etc., then I am basically changing the use case.

I slightly modified one of Gunnar's use cases to use the terms collect and
evaluate, rather than assess, because I thought we had something of a WG
consensus to break those concepts out, and avoid the ambiguous "assess". See
my QUESTION in 2.13.

If somebody has an application/system that automates an assessment process,
and divides that process into a collection aspect, an evaluation aspect, and
a reporting aspect, and uses another process to coordinate those aspects,
and  I rewrote the use case to describe it from my point of view as  "well,
it did an assessment" or "the system collects, evaluates, and reports on an
assessment", I could be hiding the important details of the use case - that
these might be separate entities in the network, produced by different
vendors, and might need standardized protocols and standardized data models
to transport the information between entities. I don't think such an
editorial rewrite is to anybody's benefit, because when we design a sacm
solution, we better take into considerations that, in some use cases, these
aspects may need to be separated and coordinated, and we may need
standardized protocols and data models between them.

For the most part, I decided that as an editor, I should allow the use cases
to be published mostly as-submitted, to ensure that the contributors'
description of *their* use case isn't mangled by editing. Gunnar's use cases
tend to use his preferred terminology because they represent important
aspects of his usage. We can try to coalesce the terminology and
architectural entities as we extract the requirements. Some of those
requirements will be architectural requirements, and I think we want to
understand the use cases that might drive certain architectural
requirements.

Certainly, if the WG would prefer to use "collect", "evaluate", and "report"
rather than the all-inclusive "assess", we can ask the contributors to
rewrite their use cases (or the editors can rewrite them and ask the
contributor to agree it is still an accurate representation). I think it is
important to capture the original in at least one revision, so it is clear
what we changed from the original submissions.

It might be a good idea to identify who submitted each use case, so we know
who to discuss that particular use case with, to better understand that
person's usage. We might also allow "me too" statements, if an existing use
case already describes another's use case.

I think at this point in time, we should be looking at the original use
cases and trying to understand the important aspects of the use case, so we
can extract requirements for sacm, without rewriting the use cases, so we
can try to better understand the requirements.

David Harrington
[email-masked]
+1-603-828-1401

> Sent: Thursday, October 24, 2013 10:59 AM
> For the sake of providing an example with respect to my comment about
> how the use case document could be cleaned up a bit, I happened to be
> reviewing 2.12 (Traditional Endpoint Assessment with Stored Results) just
> now.  It reads:
> 
> IMO, there should probably be no discussion in this use case of a
Datastore,
> Controller, Content Repository.  By my read, this use case intends to say:
> "The system, triggered externally, targets an endpoint for
characterization
> (inventory?), so that the system can determine the minimal set of endpoint
> attributes to collect and store for subsequent use."
> 
> - The system must support target endpoint inventory (I think this is what
was
> intended by requesting "taxonomy")
> Architectures would then get into the "how" of it all, which is where I
would
> see Collectors, Datastores, Evaluators, etc. come into play.  There's more
> than one way to satisfy the requirements which are derived from this
> particular use case.
> 
> The Use Case, as written, gets the point across, but I think it does so
> superfluously.  Maybe it's just a matter of personal taste or preference,
but I
> think a sentence or two would capture the intent of this use case without
> implying or assuming any particular architectural solution.  This may or
may
> not be the case with each other use case, though from what I've been
> through so far, I think most could stand to be pruned.
> 
> Adam
> 
> . . .

---

From: Gunnar Engelbach <[email-masked]>
Date: Thu, 24 Oct 2013 12:46:49 -0400

On 10/24/2013 10:59 AM, Adam Montville wrote:
> IMO, there should probably be no discussion in this use case of a Datastore, Controller, Content Repository.
> By my read, this use case intends to say: "The system, triggered externally, targets an endpoint for characterization (inventory?), so that the system can determine the minimal set of endpoint attributes to collect and store for subsequent use."

+1

With the exception of the taxonomy step, this use case is covered in 
others such as 2.2.

Taxonomy wouldn't be a full inventory.  Just enough to determine what 
content applies to an endpoint.  CPE vice AI.

Similarly, 2.13 and 2.14 can be merged with the understanding that the 
system might trust the connecting endpoint to submit its own 
self-evaluation results, whereas others might only request specific 
system characteristics and do the evaluation internally.

> The Use Case, as written, gets the point across, but I think it does so superfluously.  Maybe it's just a matter of personal taste or preference, but I think a sentence or two would capture the int…

Brevity is good.  Then again, compression means data loss.

---

From: Adam Montville <[email-masked]>
Date: Thu, 24 Oct 2013 17:06:50 +0000

> Sent: Thursday, October 24, 2013 11:31 AM
> To: Adam Montville; 'Watson, Kimberly'; 'Gunnar Engelbach'; [email-masked]
> Subject: RE: [sacm] review of draft-camwinget-sacm-requirements-01.txt
> 
> Hi,
> 
> consensus to break those concepts out, and avoid the ambiguous "assess".
> See my QUESTION in 2.13.
> 
> If somebody has an application/system that automates an assessment
> process, and divides that process into a collection aspect, an evaluation
> aspect, and a reporting aspect, and uses another process to coordinate those
> aspects, and  I rewrote the use case to describe it from my point of view as
> "well, it did an assessment" or "the system collects, evaluates, and reports
> on an assessment", I could be hiding the important details of the use case -
> that these might be separate entities in the network, produced by different
> vendors, and might need standardized protocols and standardized data
> models to transport the information between entities. I don't think such an
> For the most part, I decided that as an editor, I should allow the use cases to
> be published mostly as-submitted, to ensure that the contributors'
> aspects of his usage. We can try to coalesce the terminology and architectural
> entities as we extract the requirements. Some of those requirements will be
> architectural requirements, and I think we want to understand the use cases
> that might drive certain architectural requirements.
> 
> rather than the all-inclusive "assess", we can ask the contributors to rewrite
> their use cases (or the editors can rewrite them and ask the contributor to
> agree it is still an accurate representation). I think it is important to capture
> the original in at least one revision, so it is clear what we changed from the
> original submissions.
> 

Good idea.

Here's a list of the use cases.  Can the submitter of each use case indicate so?

2.1	Definition and Publication of Automatable Configuration Guides
2.2	Automated Checklist Verification
2.3	Organizational Software Policy Compliance
2.4	Detection of Posture Deviations
2.5	Search of Signs of Infection
2.6	Remediation and Mitigation
2.7	Endpoint Information Analysis and Reporting
2.8	Asynchronous Compliance/Vulnerability Assessment at Ice Station Zebra
2.9	Vulnerable Endpoint Identification
2.10	Compromised Endnpoint Identification
2.11	Suspicious Endpoint Behavior
2.12	Traditional endpoint assessment with stored results
2.13	NAC/NAP connection with no stored results using an endpoint evaluator
2.14	NAC/NAP connection with no stored results using a third-party evaluator
2.15	Repository Interaction - A Full Assessment
2.16	Repository Interaction - Filtered Delta Assessment
2.17	Direct Human Retrieval of Ancillary Materials
2.18	Register with repository for immediate notification of new security vulnerability content that match a selection filter

> 
> I think at this point in time, we should be looking at the original use cases and
> trying to understand the important aspects of the use case, so we can extract
> requirements for sacm, without rewriting the use cases, so we can try to
> better understand the requirements.

I agree.  It may be helpful to understand the important aspects by getting the use cases to be a bit more concise.

> > From: [email-masked] [mailto:[email-masked]] On Behalf
> > Of Adam Montville
> > > From: [email-masked] [mailto:[email-masked]] On
> Behalf
> > Of
> > > Adam Montville
> > > This has been a very good discussion to track.  I agree with the
> > > notion
> that
> > > use cases shouldn't really consider the "how" (and, IMO, the use
> > > case document could stand to be cleaned up a bit in this way, though
> > > it may
> not
> > reviewing 2.12 (Traditional Endpoint Assessment with Stored Results)
> > just now.  It reads:
> >
> > (inventory?), so that the system can determine the minimal set of
> > endpoint attributes to collect and store for subsequent use."
> >
> > see Collectors, Datastores, Evaluators, etc. come into play.  There's
> > more than one way to satisfy the requirements which are derived from
> > this particular use case.
> >
> > The Use Case, as written, gets the point across, but I think it does
> > so superfluously.  Maybe it's just a matter of personal taste or
> > preference,
> but I
> > think a sentence or two would capture the intent of this use case
> > without implying or assuming any particular architectural solution.
> > This may or
> may

. . .

---

From: Gunnar Engelbach <[email-masked]>
Date: Thu, 24 Oct 2013 14:26:36 -0400

2.1     Definition and Publication of Automatable Configuration Guides
  2.2     Automated Checklist Verification
  2.3     Organizational Software Policy Compliance
  2.4     Detection of Posture Deviations
  2.5     Search of Signs of Infection
  2.6     Remediation and Mitigation
  2.7     Endpoint Information Analysis and Reporting
* 2.8     Asynchronous Compliance/Vulnerability Assessment at Ice 
Station Zebra
  2.9     Vulnerable Endpoint Identification
  2.10    Compromised Endnpoint Identification
  2.11    Suspicious Endpoint Behavior
* 2.12    Traditional endpoint assessment with stored results
* 2.13    NAC/NAP connection with no stored results using an endpoint 
evaluator
* 2.14    NAC/NAP connection with no stored results using a third-party 
evaluator
* 2.15    Repository Interaction - A Full Assessment
* 2.16    Repository Interaction - Filtered Delta Assessment
* 2.17    Direct Human Retrieval of Ancillary Materials
* 2.18    Register with repository for immediate notification of new 
security vulnerability content that match a selection filter

* These are the ones that I remember.

---

From: "Waltermire, David A." <[email-masked]>
Date: Thu, 24 Oct 2013 18:34:40 +0000

> Here's a list of the use cases.  Can the submitter of each use case indicate so?

I wrote or refined use cases 2.1 - 2.4. Much of this was based on feedback and discussion from the list so I hesitate to take full credit for submission.

> 2.18	Register with repository for immediate notification of new security
> vulnerability content that match a selection filter

---

From: Stephen Hanna <[email-masked]>
Date: Thu, 24 Oct 2013 20:39:35 +0000

> Here's a list of the use cases. Can the submitter of each

I wrote use cases 2.5 - 2.7.

Let me say that I completely endorse Adam's suggestion to
pare the use cases down to a small set of brief ones that
succinctly spans and explains the problems we must address.

Dave Harrington expressed concern that editing use cases
involves changing the use cases. True but that's OK. The
goal here should not be to preserve intact every person's
ideas but to arrive at a short, understandable document
that spans and explains the essential use cases.

Some of the current use case text refers to architectural
components like Datastore, Controller, and Registry. Again,
I agree with Adam who said those do not belong in a use case.
The use case should describe the problem not how it's solved.

Thanks,

Steve