Subject: [sacm] Matching Algorithm Requirements

From: "Jim Schaad" <[email-masked]>
Date: Thu, 28 Jan 2016 14:41:01 -0800

There has been a discussion on matching algorithms with respect to the
requirements draft.  I would like to get feedback for myself on what the
requirements of this matching algorithm is supposed to be doing.  To that
end I would like to start with some questions.

1.  What are the targets that a matching algorithm are supposed to match?
One target is going to be endpoints (for some definition of endpoints).
However are there other targets that need to be matched as well?  Do we
match Software packages?  SACM components? 

2.  What are the outputs of the matching algorithm?  Is it a binary YES/NO?
Is it some type of confidence level? 90% confidence match?  Are there
historical notations? Yesterday I would have said that they matched, but
today I say they do not match.

3.  The Friday morning cabal was talking about matching algorithms at one
point, I did not always read the minutes from that group, however did they
ever come up with the basics/starting point for this algorithm so we can
start discussing what it does or does not do and what it is based on from
the IM?

4.  Is the matching algorithm restricted to just the inputs given to it, or
is it allowed to perform additional queries of SACM entities to get more
data.  If it gets more data, is it restricted to the ACLS of the questor?
If it is not restricted are there security problems?  If it is restricted
and there reproducibility problems?

5.  Are there currently any matching algorithms from the prospective
solution space that the SACM group needs to be looking at?

Jim

---

From: "Wolfkiel, Joseph L CIV DISA ID (US)" <[email-masked]>
Date: Fri, 29 Jan 2016 21:28:52 +0000

My opinions:

I would expect the primary matching algorithm to address matching IP networked device endpoints between SACM data collectors and within the same data collector at different points in time.

Matching applications and patches is also of concern.  For example if the same application installs on Windows, Linux, Solaris, IOS, and other operating systems using different names, can a SACM system tell that the deployed instances are all instances of the same application?  I would treat this as a separate problem given the SWID overlap.

I like the confidence level approach, but it will ultimately probably need to be treated as a binary.  The way the matching algorithms I've designed work is to rank different matching criteria in order of confidence (ordinal, not cardinal) and establish a cutoff on which matches will be used to combine data from different sensors into the same endpoint record.  I generally worry more about the history in terms of when were the matching attributes collected.  For example an two IP addresses, collected on the same network within 10 minutes of each other would  be a much better indicator of a matching endpoint than IP addresses collected 7 days apart.

I'm not sure if they (we?) ever did.  I'm happy to share the matching algorithms we've used in the DoD, along with the cases where they didn't work.  I'd offer to "contribute" the algorithms, but not sure I'm up to that level of effort.

I would restrict it to just the inputs provided.  Building additional complexity might make the problem close to unsolvable.

Not sure who gets to decide this.

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52
Fort Meade DISA Acquisiton Bldg Cube A4A58E
Work: (301) 225-8820
Gov Cell: (571) 814-8231
[email-masked]

-----Original Message-----
From: sacm [mailto:[email-masked]] On Behalf Of Jim Schaad
Sent: Thursday, January 28, 2016 5:41 PM
To: [email-masked]
Subject: [Non-DoD Source] [sacm] Matching Algorithm Requirements

Jim

_______________________________________________
sacm mailing list
[email-masked]
Caution-https://www.ietf.org/mailman/listinfo/sacm

---

From: Gunnar Engelbach <[email-masked]>
Date: Fri, 29 Jan 2016 17:55:32 -0500

Good questions.

But before getting too deep in them, let's consider the cases under 
which such an algorithm will be necessary:

A) Correlation needs to be performed consistently between products from 
separate vendors

-- and/or --

B) Correlation is a federated function

If we make the assumption that information about an endpoint will be 
communicated including attributes for identification (as opposed to 
resolving to an identifier), then you have to get pretty creative to 
come up with a case where either is the case.

Even if we choose to go with the identifier route, that identifier is 
most likely to be for a specific data store (so the identifier would 
actually be store + ID).  As long as that identifier is created by that 
data store, then the same algorithm will have been used and consistency 
is maintained.

A consequence of this is that the quality of correlation will vary from 
product to product, which in turn becomes a marketing point for various 
solutions.  As long as this is an acceptable state, then I don't think 
having a correlation algorithm is important to what SACM is trying to 
accomplish right now.

I would, however, like to hear if anybody else can think of a use case 
that makes this of more immediate importance.  'Cuz it's Friday and I've 
already started on weekend mode.

--gun

On 1/28/2016 5:41 PM, Jim Schaad wrote:
> https://www.ietf.org/mailman/listinfo/sacm