Subject: RE: IDMEF Assessment, Impact, Action, Confidence

From: "Eric D. Williams" <[email-masked]>
Date: Thu, 8 Nov 2001 09:51:10 -0500

Since we are looking at the cautionary statement...

As to what I recall concerning the original discussion on Confidence
values, I agree the cautionary statement would be the correct
approach.  There was much consternation on whether the value could be
even described appropriately in a consistent way, however that does
not preclude convergence on a standard metric for confidence.  Given
that logic, my nit would be with sentence 1.  I would modify as
follows to capture the 'possibility' that later
revisions/implementations may converge on methods to compute and
represent confidence values.

Sentence one:  Caution: It should be noted that different types of
analyzers may compute confidence values in different ways and that in
many cases, confidence values from different analyzers should not be
compared (for example, if the analyzers use different methods for
computing or representing confidence or are of different types or
configurations).

I think this way we do not, through the caution, give developers the
"idea" that confidence from varying analyzers can never be compared
(which is what I perceived from the current version).  It may also be
prudent to give some addl.' language on documenting confidence
algorithms or passing a confidence algorithm designator - in a later
draft of course :).

My $.02

Eric Williams
PGP Public Key: http://new.infobro.com/KeyServ/EricDWilliams.asc
Finger Print: 1055 8AED 9783 2378 73EF  7B19 0544 A590 FF65 B789

On Wednesday, November 07, 2001 4:08 PM, David A. Curry
[SMTP:[email-masked]] wrote:
> 
> 
> >>>>> On Wed, 7 Nov 2001, Lansing Sloan, identified as "Lansing"
> >>>>> below, wrote:  
> >>>>> 
> 
>   Lansing> I agree that the caution paragraph is warranted, and I
> haven't seen 
>   Lansing> any unacceptable proposal yet, so your best judgement
> should be 
>   Lansing> fine.
> 
> Actually, I like your sentence best, except that I joined it to the
> previous one which (to me, anyway) improves the flow.  So here's
> how it reads now:  
> 
>     Caution:  It should be noted that different types of analyzers
> compute 
>       confidence values in different ways and that in many cases,
> confidence 
>       values from different analyzers cannot be compared to each
> other (for 
>       example, if the analyzers are of different types or are of
> the same 
>       type but differently configured).  Care should be taken when
> implementing 
>       systems that process confidence values (such as event
> correlators) not 
>       to make comparisons or assumptions that cannot be supported
> by the 
>       system's knowledge of the environment in which it is working.
> 
> Comments?
> 
> --Dave