Subject: RE: [IDMEF][Issue 10] Severity scale too narrow

From: "Matthew F. Caldwell" <[email-masked]>
Date: Mon, 12 Jan 2004 19:47:54 -0500

Greg, 

	I agree using an integer to describe the level of severity is a
necessity. Our SIM/SEM solution already reassign the priority into
numeric form in the aggreagation phase. It could be made better for
everyone if assignment was already in the logging format. For example:
HIGH = 255 , MEDIUM = 128 LOW = 64. It is faster and more efficient to
use an integer format in correlation primarily because string
comparisons are costly CPU wise. When you compute/correlate/classify
3000/eps any CPU cycles you can spare increases the effectiveness of the
SIM/SEM solution and or analysts running queries on databases.  

Matt

Matthew F. Caldwell, CISSP
Founder and Chief Security Officer
GuardedNet, Inc. (neuSECURE)
Web:www.guarded.net
Email:[email-masked]
Phone: 404-591-8250

-----Original Message-----
From: Greg Shipley [mailto:[email-masked]] 
Sent: Monday, January 12, 2004 5:50 PM
To: David A. Curry
Cc: [email-masked]; intrusion detection wg
Subject: Re: [IDMEF][Issue 10] Severity scale too narrow 

On Sun, 11 Jan 2004, David A. Curry wrote:

> I am way skeptical of any scheme that purports to assign numerical 
> values to this type of data and claim that they are "accurate" in any 
> meaningful way.
>
> As I recall, the group was similarly skeptical when the idea was 
> proposed in the past (and it was proposed more than once).

If it's a done deal, it's a done deal, and please view my ramblings as
nothing more than comments from the peanut gallery.  (and please excuse
me and my monkey wrench!) I just know from EXPERIENCE that simple
classification metrics (e.g. 5 ratings) are not going to cut it in many
real world scenarios.  For example, when we were doing our SIM testing
last year in conjunction with Syracuse University we had days when we
hit 2600 events PER SECOND from our IDS/FW sensors.  Without some
ability to classify/rank those events more granularly you can forget
about ever really sifting through the data, much less acting on it.

IMO, granular classification mechanisms are mandatory when dealing with
large data volumes.  Now, whether or not that classification falls under
the IDWG charter is, well, for you folks to decide!

As for your skepticism, I totally agree with your concern about
accuracy.
However, we're watching the NIDS space evolve right under our noses, and
I'm going to speculate that confidence levels will be appearing in
commercial products in less than 12 months.  Take, for example,
SourceFire's RNA product; they are using passive vulnerability
identification techniques to gather data for alert classifications.
(e.g.  That MS RPC/DCOM alert is critical because the target is indeed
vulnerable to that MS RPC attack)  I believe SourceFire's RNA is
shipping, TODAY, and other vendors are not far behind.

(Perhaps one leaves the classification system intact, and inserts an
additional/optional confidence variable, instead?)

How does this relate to the IDWG?  Again, that's for you folks to
decide, and I really don't intend to cause problems here.  But, IMO, I'm
guessing that granular classification systems and/or confidence levels
will be a factor for commercial implementations.  The commercial market
is already moving there NOW.  (And I'd be curious to hear what any of
them think, if they are on this list, but I digress...)

My .02,

-Greg

P.S. As for the ascending vs. descending, as long as its documented and
consistent, I don't think it matters...  (Herve's choice?  :)

---

From: Herve Debar <[email-masked]>
Date: Tue, 27 Jan 2004 14:54:33 +0100

>>proposed in the past (and it was proposed more than once).

I think in similar terms. I'd like to propose an enumerated, textual 
list, and let people assign whatever internal numerical values they 
want. Maybe this "numerical" value could be provided for as an option, 
but the enumerated string would be mandatory.

Greg Shipley wrote:
 > nothing more than comments from the peanut gallery.  (and please 
excuse me
 > and my monkey wrench!) I just know from EXPERIENCE that simple
 > last year in conjunction with Syracuse University we had days when we hit
 > 2600 events PER SECOND from our IDS/FW sensors.  Without some ability to
 > classify/rank those events more granularly you can forget about ever
 > really sifting through the data, much less acting on it.

First of all, it's an open discussion, so any input is valid.

In fact, I have a completely opposite feeling apout integer values. What 
I want in an alert is
1) do I have to do something about it
	and if 1)==TRUE
2) how much time do I have to comply with my business objectives.

Using integer values is akin to setting a threshold, and this depends on 
organizations. That's why I like the enumerated list, and people then 
set their own sensitivity to it.

> SIM/SEM solution and or analysts running queries on databases.  

I remember the old CSIDS sensor doing that. We just never got around to 
using something else than the 1-5 scale. As much as I understand how 
this is important for the reasoning of your engine, I'm not sure that 
this is information that will be transported as is by IDMEF messages.

I'm still mulling this one around, and am not up for a definite proposal.

Hervé
-- 
Hervé Debar             <mailto:[email-masked]>
Tel: +33 (0)2 31 75 92 61            GSM: +33 (0)6 74 09 09 66
France Télécom R&D                   Fax: +33 (0)2 31 75 93 13
42 rue des Coutures  (--)  BP 6243  (--)  F-14066 Caen Cedex 4

---

From: Yoann Vandoorselaere <[email-masked]>
Date: Tue, 27 Jan 2004 17:52:05 +0000

> but the enumerated string would be mandatory.

Wouldn't numerical value be a problem for interoperability between
products ?

-- 
Yoann Vandoorselaere, http://www.prelude-ids.org

"Programming is a race between programmers, who try and make more and 
 more idiot-proof software, and universe, which produces more and more 
 remarkable idiots. Until now, universe leads the race"  -- R. Cook