Subject: Vendors, IODEF, and response mechanisms

From: "Stewart, William R." <[email-masked]>
Date: Fri, 30 Nov 2001 09:50:03 -0500

A couple of questions;

1. It's my understanding that Tivoli is using an early version of IDMEF in
one of their products.  Are any other vendors implementing IDMEF messages
yet, in one form or another?

2. Are there any plans for a IDMEF/IODEF mapping?

3. It is unclear to me how the breadth of responses to alerts will be
implemented with the current message set.  For example, if an IDS contains
business rules to shut down certain ports or even counterattack, which
messages would be used?  In the requirements document, the analyzer seems to
be tasked with also making some automated reactions, though the diagram seem
to imply the manager would be involved.  Should these features be best
represented by another virtual component, such as a "responder"?

Undoubtably, the above has been covered in working group meetings, though I
haven't been able to discern this from the artifacts.

Will Stewart
Sr. System Engineer
[email-masked]
(703)465-7742

---

From: Yuri Demchenko <[email-masked]>
Date: Fri, 30 Nov 2001 16:24:36 +0100

WIlliam,

> 2. Are there any plans for a IDMEF/IODEF mapping?

Compatibility with IDMEF is a mandatory requirement to IODEF (RFC 3067)

Current version of IODEF 
> http://www.terena.nl/task-forces/tf-csirt/iodef/docs/draft-terena-itdwg-iodef-xml-002.txt
will be updated before the IETF and will include: 

(from IODEF side) extended/changed description of Evidence and
EvidenceData with prospect to describe correlation between Incidents and
EvidenceData (something more to be done here)

(from IDMEF side) new elements which are introduced in the last version
IDMEF-05

There are two (intended) options in IODEF to use IDMEF/IDS message or
information:

1) include complete IDMEF Alert message into IncidentAlert/IODEF 

2) decompose it into IODEF Elements/Structure; 
and here we have also possibility to use all IDMEF classes "as-is" or
decompose some of them.

Compatibility issue will the one to discuss at the INCH BOF (Monday
13.00-15.00) and inside IODEF design team. 

Your participation and contribution is welcome.

Regards, 

Yuri

-- 
-----------------------------------------------------------------------
Yuri Demchenko, TERENA, Singel 468D, 1017 AW Amsterdam, The Netherlands 
Tel: +31 20 530 4488  Fax: +31 20 530 4499  E-mail: [email-masked]
-----------------------------------------------------------------------

---

From: "David A. Curry" <[email-masked]>
Date: Fri, 30 Nov 2001 12:44:10 -0500

>>>>> On Fri, 30 Nov 2001, Stewart, William R., identified as "William" below,
>>>>> wrote:

  William> 1. It's my understanding that Tivoli is using an early version of
  William> IDMEF in one of their products.  Are any other vendors implementing
  William> IDMEF messages yet, in one form or another?

Tivoli is using a very early version of IDMEF (from when it was still called
IDEF), and really what they're using is a superset of that.  If you know all
the drafts we've had it'll look familiar to you, but it won't look like what
is in the IDMEF today.

Silicon Defense has a Snort IDMEF plug-in available, along with a C library
implementation of the IDMEF.  This, along with all the other IDMEF software
the group is aware of, is available from Silicon Defense's IDWG web site:

	http://www.silicondefense.com/idwg/

I know one IDS vendor that was planning to do "send alerts via IDMEF" in the
same way that they support "send alerts via SNMP" today.  They weren't going
to replace their current proprietary sensor communications mechanisms with
it, though.  I will decline to name the vendor, as my information is old, and
I have no idea whether or not they still have plans to do this, or what the
time frame might be.

--Dave

---

From: "Azim, Ozakil" <[email-masked]>
Date: Fri, 30 Nov 2001 13:07:32 -0500

Warning : Vendor Update

At netForensics, we have a security information management system using
an XML based Event schema for communication between our sensors & analysers.

Like Tivoli Risk Manager, this was initially based on IDMEF but over a
period of time, we have moved away from IDMEF for two reasons:
1) IDMEF seems to be focussed only on IDS messages communicated only
    between analysers and managers. (The Event element is apparently
    out of the IDMEF scope).
2) IDMEF is very verbose and is simply not suitable for handling security
    events at the very high rates generated by some security devices.

-azim

-- 
Ozakil Azim, [email-masked], 732-393-6030

---

From: Stuart Staniford <[email-masked]>
Date: Fri, 30 Nov 2001 11:27:33 -0800

> represented by another virtual component, such as a "responder"?
We decided early on just to work on IDS alert notifications, not 
configuration or response commands, and not a general event format .  
The idea was to do the easiest thing first, focussing on an achievable 
target, and then possibly do more in future documents if it seemed 
justified.  (Actually, because of the way IDMEF handles signatures, it 
is in fact feasible to use it for much more general events than just IDS 
alerts.)

Partly this was based on the experience several of us had had with CIDF, 
where we tried to make an all-singing all-dancing event format, which 
proved to be incredibly complex and hard to interpret reliably.  It was 
also binary and much more compact than IDMEF.  At that time, everyone 
told us we should have done it in XML instead :-)

Stuart.

---

From: "Azim, Ozakil" <[email-masked]>
Date: Fri, 30 Nov 2001 16:22:05 -0500

Stuart,

Let me start by saying that the discussions on the IDWG mailing and
the IDMEF has been a real good source of information and standardization
in our work. You guys have been doing a great job!

And, XML certainly is the way to go (which is why we use it despite the
extra costs). What I dont always get is why the xml data format has
to be so verbose.

Let say for example that we wanted to send an additional piece of
data in an IDMEF Alert. say it is the direction of a connection across
a firewall with two values (0/1) for (IN/OUT). we could represent it as:
1 - <d>0</d>
2 - <dir>0</dir>
3 - <direction>0</direction>
4 - <AdditionalData type="integer" meaning="direction">0</AdditionalData>
5 - <Data type="int" name="dir">0</Data>
6 - <ad nm="dir">0</ad>

(I accept that some of these samples are extreme. but bear with me :)

my question is, why does IDMEF have to be so explicit? if it is more
cryptic, wouldn't it still be as useful? it might be a little less
humanly readable - but how many humans are going to be reading IDMEF
xml directly other than for debugging?

-azim

Stuart Staniford wrote:
    <snip>

---

From: Stuart Staniford <[email-masked]>
Date: Fri, 30 Nov 2001 13:35:44 -0800

Azim:

Thanks for the comments - I see your point.  Speaking strictly for 
myself, I never liked the extension mechanism anyway - AdditionalData 
always struck me as clumsy.  I will confess that in stuff I have built, 
I have just stuck in extra fields that make my IDMEF invalid but 
well-formed and relied on the fact that the parser will ignore the tags 
(I know, that's evil).

The IDMEF document is in last call right now!  Why don't you propose an 
alternative that you think would work better?

Stuart.

On Friday, November 30, 2001, at 01:22 PM, Azim, Ozakil wrote:

> -- Ozakil Azim, [email-masked], 732-393-6030
>
>
>
>

---

From: "David A. Curry" <[email-masked]>
Date: Fri, 30 Nov 2001 18:08:58 -0500

>>>>> On Fri, 30 Nov 2001, Azim, Ozakil, identified as "Ozakil" below, wrote:

  Ozakil> my question is, why does IDMEF have to be so explicit? if it is more
  Ozakil> cryptic, wouldn't it still be as useful? it might be a little less
  Ozakil> humanly readable - but how many humans are going to be reading IDMEF
  Ozakil> xml directly other than for debugging?

Many, many moons ago, at one of the IDWG/IETF meetings, it was decided that
human readability was desirable, at the expense of verbosity, so that's why
we don't use <d> tags (to take your most extreme example).

Two other comments:

 1. It's not completely fair to pick on <AdditionalData>, because in theory,
    it *shouldn't* get used that much -- if we've done a good job of putting
    the other necessary tags in place.  And anything that does get put into
    <AdditionalData>, in theory, is only there until it gets adopted into the
    next version of the standard, when its verbosity will decrease.

 2. The "preferred" channel for sending these things is IDXP, which supports
    compression.  SSL supports it too.  And if you run any kind of halfway
    decent compression algorithm on this stuff, it gets really small (if I
    recall correctly, compression was something like 70-80%, but it's been
    a while since I've checked).  So while it's verbose, it's not necessarily
    a bandwidth hog.

That's not to say your points aren't accurate, they are.  But this topic has
not come up on the list for a long, long time.  Maybe it should have come up,
but it didn't, and as we all know, the squeaky wheel gets the grease...

--Dave

---

From: "Azim, Ozakil" <[email-masked]>
Date: Fri, 30 Nov 2001 18:51:52 -0500

Stuart/David/Lam,

I wouldn't presume to propose an alternative. This is a tricky problem
and would, otherwise, probably have been addressed already. And it is safer
sitting on the critics' bench :).

But regarding compression, I remember a similar argument used by W3C
against the proposed WAP Binary XML Spec submission that, I believe,
proposed using a binary tokenized format. The W3C's comment
at that time, too, was basically - 'use standard compression'.

Standard compression is a good solution when you have both processing
power and batched operations (compressing and decompressing individual
events is not very effective). In the kind of environments that
we operate in, we find that a firewall can easy generate a few hundred
messages per second; and if we are managing multiple firewalls, the
message rates goes up to thousands/sec. Even though both batching of
events and increasing the processing power is doable, there is a
definite price to be paid!

When we were first exploring using IDMEF, we had encoded CiscoIDS messages
into IDMEF - a 200 byte message became about 2000 bytes. That wasn't
acceptable for us. So we basically reduced tag names (since we know
the tags conform to a DTD that's valid, and well-documented) and we flattened
the levels of inherited/aggregated relationships. This, we know, is no
longer even IDMEF compliant :(. But we figured that since IDMEF is XML
anyway, all our  agents needed to do was a simple translation from IDMEF
to our Event format when talking to IDMEF compliant IDSes.

regards,

Azim

---

From: Stuart Staniford <[email-masked]>
Date: Fri, 30 Nov 2001 16:10:44 -0800

> sitting on the critics' bench :).

You've got that right!

> definite price to be paid!

Yeah - handling firewall logs was definitely not part of our 
requirements (no firewall people were involved in the WG for one).  At 
the requirements phase, it was pretty clear that no-one was 
contemplating using this for thousands of alerts per second, so that 
didn't drive the thinking.

So is the WAP binary idea dead?  (I haven't followed it).  Some of the 
Cisco guys who were implementing IDMEF a while back had the same idea.  
(Are you still here Paul?)

> to our Event format when talking to IDMEF compliant IDSes.

There's nothing to stop us from writing another document that says 
"Here's the approved standard way to send compressed IDMEF" if that's 
what we need to do to make this work.

Stuart.