Subject: Glenn's last set of comments

From: Mike Erlinger <[email-masked]>
Date: Sun, 18 Mar 2001 06:28:10 -0800

hi all

what follows is Glenn's last set of comments and my responses.  
this is what i will go over at the ietf meeting.  my general comment
is that without specific recommendations, it is difficult to determine
what, if anything should be changed.

mike

======================================================================
Legends:
[Nothing in Margin] Excerpts from  idwg-requirements-04.txt
      ==>           Glenn's comments on idwg-requirements-04.txt 
      XXXX          Mike's  response to ==>
      GLEN          Glenns  response to XXXX
      Mike          Mike's response to GLEN
      Change        Proposed Change to Document

Note:
      I have deleted parts on which there is agreement,
      and/or irreconcilable disagreement :-)  
======================================================================
                                 Part-1.
                                 ======
1. High level issues:
   - The document presupposes that "activity", i.e., what gets reported
     via IDEF, directly reflects attacks or at least probable attacks.
     But it seems fundamentally more flexible for IDEF to encompass
     may not reflect an attack, but with additional context available
     more generic events, such as "user login", which by themselves
     to the Analyzer, may.
==> This comment has not been fully incorporated.

XXXX How so?  It seems to me that there are the following sets of things
XXXX defined in section 3.2 and the figure.   Activity is anything of
XXXX interest that a Sensor sees, based on what a sensor is looking for
XXXX it sends an Event to the Analyzer.  In Vern's case of "user login"
XXXX it is Activity, and it is an Event because some Sensor forwards to some
XXXX Analyzer.  The Alert is then "A message from an Analyzer that an
XXXX Event has taken place.
XXXX This all seems pretty logical to me and was worked on by the working
XXXX group and Mark for a very long time.

XXXX What specific changes would you make?
GLEN I would get the definitions straight and check for correct terminology
GLEN throughout the document.

Mike I would argue that the definitions are straight in the document.  
Mike Someone needs to tell me explicitly what definitions are NOT clear
Mike and what needs further refinement.  Most of these were approved by
Mike the WG a long time ago.  Unless there are specifics then no further
Mike changes will be made.

    - The examples ("Scenario:") are quite often unnecessary 
==> This is still a problem.
XXXX How so?   Neither the IESG nor Vern indicated that all Scenarios 
XXXX should be removed.  The changes made were an attempt to include
XXXX only Scenarios that clarified.

XXXX What specific changes would you make?  Namely what Scenarios do
XXXX you want removed and which ones that were removed do you want back?

GLEN We could do well without many of the scenarios/rationales and, with minor
GLEN supplements to the respective requirements subsections itself.

Mike Disagree, without specifics what is there is there.

    - The MUST/SHOULD/etc keywords are generally only used for ....
==> This still remains
XXXX See the introductory sentence of Section 2, the sentence that begins...
XXXX "This is not an IETF standards track document....
XXXX Discussions with the AD and others indicate that the phrase
XXXX "are NOT as in RFC 2119" is all that is needed.
GLEN I don't think that this section is appropriate. But this is editorial
GLEN matter and I will defer to the editors decision.

Mike Yes and No.  There must be some way to list the specification of
Mike requirements.  The normal way is to use MUST, SHOULD, etc.  Without
Mike this statement, we CANNOT use those words.  What words would you
Mike replace them with?

    - There should be explicit discussion of trust models.
==> This is still missing. It needs to be added to the assumptions 
==> section
XXXX The Working Group was asked to specify this when the comments were
XXXX made and before the IESG comments at various meetings.  Neither the WG,
XXXX nor the IESG, had any specific comments to make.  I would argue that
XXXX the trust model is implied by the protocol chosen, e.g. TLS.  As a
XXXX working group we have never specified any more.  

XXXX We could add some words saying that the Trust Model
XXXX is whatever we get, but in my opinion,  since this is a Requirements
XXXX Document, leaving
XXXX the Trust Model unspecified is better.   We could say something like,
XXXX "The Trust Model is not specified as a requirement, but is rather left
XXXX  to the choice of protocol, i.e., a Design Decision"

XXXX What specific changes would you make? 
GLEN I would just add a subsection on the trust model. The requirements
GLEN should be based on the trust model.  That would seem logical to me.

Mike I know you would add a subsection on the trust model, but what would
Mike that subsection say.  In the above I put forth a proposed wording.
Mike How would you change that wording?

Change  to be added to Section 3.3 Architectural Assumptions
Change  "The Trust Model is not specified as a requirement, but is rather
Change  left to the choice of protocol, i.e., a Design Decision.  What is
Change  specified is individual security related requirements, (Section 6)."
  
    - It often isn't clear when requirements are referring to
      specifications or formats; or to products (implementations).
==> I feel that this is still a problem. 
XXXX Where specifically?  Stuart's and other's comments on the IESG comments
XXXX provided  various wording changes that were incorporated in the ID.
XXXX At the time of those proposed wording changes, neither Vern nor the WG
XXXX made any additional comments.

XXXX What specific changes would you make? 
GLEN With the corrected usage of many terms and the new draft is looking 
GLEN much better in this respect. (There are still some grey areas but I 
GLEN guess a few revisions is all that we need) 

Mike So what are those revisions?  It reads fine to me now.  Without some
Mike specifics I have no idea what confuses you or what to change.

==>
==> I think that we need to get the terminology straightened out.
==> In the text the definitions are unclear and terms [alert/event/
==> attack] are used interchangeably. 
==> My take is :
==>    
==>    Sensors look for Events that may have been caused by Attacks.
==>          [There are well known attacks and enumerations of attacks]
XXXX   I would argue Sensors look for Events.  Events are whatever a
XXXX   Sensor looks for.  It does NOT have to be an attack
GLEN  I have no quarrels. I would just like to see a set of consistent
GLEN  definitions. We are not doing well in this respect.

Mike  So what definitions do you NOT understand and where are there
Mike  inconsistences.  If I really made the changes I thought that I did
Mike  then I see no inconsistences.

[DELETED]
==>    Analyzers send Alerts to Managers.
==>
==>    IDMEF defines the format of the Alert
==>
==>    Corollaries:
==>      o An Alert is not an Event or an Event report 
==>      o An Alert it is not an Attack.     
==>      o IDMEF is not a communication mechanism. It is designed to be
==>        independent of the communication mechanism.
==>    
==>    Addendum:
==>      The IDWG also intends to define a communication mechanism
==>      for exchanging Alerts [IDMEF messages] 
==>
XXXX   I think this last statement is a Big area of concern.  It was not
XXXX   an IESG concern, but was one of mine which I felt reluctant to
XXXX   make major changes.  My view is that when the Req Doc was written
XXXX   IDMEF was both the protocol and the Alert format in many people's
XXXX   mines.  As a WG we were defined to specify the message format, and
XXXX   to look for a protocol solution.  There are separate sections in
XXXX   the Req Doc, but there is
XXXX   some ambiguity in many areas.  I have no problem with this being
XXXX   one of those areas that we try to clarify.
XXXX   I believe this all flows from our name IDEF, see the WG web page
XXXX   for further clarification.

XXXX   IDMEF occurs about a 100 times in the document.  I would propose
XXXX   that we change the Intro to specify that the working group is
XXXX   IDWG, that the message format is IDMEF, and that the protocol is
XXXX   an unspecified name (when the Reg doc was created it was still unclear
XXXX   whether we would use an existing protocol or create a new one.
GLEN The IDWG, IDMEF, IDP matter has been sorted out in the new draft.
GLEN I do not see problems there. But as I have mentioned above the
GLEN definitions activity, event, attack, alert, ... and their usage
GLEN are still loose.

Mike  Again, give me specifics.  I see no problem in the current draft.
 
                                   Part-II
                                   =======      
            Intrusion Detection Message Exchange Requirements
                 <draft-ietf-idwg-requirements-04.txt>

[Deleted]
2. Conventions used in this document
     
==> Perhaps we can do away with this section [ s/MUST/must/ etc]
==> I think that was the intent of Vern's mail.

XXXX If you think that was the intent of his email, you should have said
XXXX so when the email was sent, 17 March 2000.  Talking to the AD, this
XXXX was NOT the IESG intent, but rather that we make it clear what our
XXXX MUST, etc., meant.  Requirements documents are generally written with
XXXX must, should, etc.  We just needed to make clear that we were not
XXXX using them in the 'standard' IETF way, which we did.
GLEN I still think that we SHOULD do away with this section :-)
GLEN [ s/MUST/must/ etc]. But this an editorial matter I will leave to the 
GLEN editors discretion. 

Mike If this is removed, then you need to come up with a way to specify
Mike Requirements.  How would you do so?

3.  Introduction

This document defines requirements for the Intrusion Detection Message
Exchange Format (IDMEF), which is the intended work product of the Intrusion
Detection Exchange Format Working Group (IDWG). IDMEF is planned to be
a standard format which automated Intrusion Detection Systems can use
for reporting events which they have deemed to be suspicious or of interest.

==> Note. IDMEF will be used for "reporting events". Compare with
==> diagram below.

XXXX Same issue as above, except change "event" to alert.
XXXX I would proposed doing a grep on 'event' and making sure that it is
XXXX not confused with 'alert'
GLEN That seems to be (mostly) done. There is much less confusion now. Thanks!
[DELETED]

[Deleted]
3.3  Architectural Assumptions
==> Trust Model needs to be discussed here.
XXXX  See the Intro comments above about Trust Models.  If you have some
XXXX  particular wording, please make it known.  I really believe that the
XXXX  WG left this unspecified on purpose.  The Trust Model will come
XXXX  through the protocol which is unspecified.

GLEN If the Editors and WG have decided to leave the trust model unspecified
GLEN on purpose and if the IESG can be convinced of the validity of the 
GLEN purpose, I am still concerned, but my concern can certainly be overridden. 
GLEN If that is not the case, I do not have any wordings ready, I can 
GLEN try to put some wordings together. Should not be a big deal.

Mike I agree it is not a big deal if you already know what the implementation
Mike choices are.  This is a requirements document.  So tell me the wording
Mike to describe the explicit trust model that you think the requirements
Mike document is able to require and whether this trust model is a SHOULD,
Mike MUST or whatever.  Make sure to recognize that if we make the wrong
Mike decision here, e.g., to strong a wording, then we may back ourselves
Mike into a corner on what can actually be implemented.

Change  Above is a proposed change to section 3.3.  It would add some
Change  discussion of the Trust Model

[DELETED]

6.2.1 Rationale: Since it is expected that firewalls will often be
      deployed between IDMEF analyzers and their corresponding managers,
      the ability to send IDMEF messages through firewalls is necessary.
==> Do we need the following ? [I am not sure that is correct.] 
XXXX  I do not see the concern, nor what wording you would like to have
XXXX  nor what you think is incorrect (e.g., the desire to not make
XXXX  changes in the firewall or the POST method or ?).
GLEN  The concern is that changes will be required in the firewall.
GLEN  Whether the change will weaken the security perimeter or not depends 
GLEN  on the changes and circumstances.

Mike  So the requirement is to NOT create a design that forces a weakening
Mike  of the firewall.  I would change your the last sentence in your
Mike  comment to be:  "Whether the change will weaken the security 
Mike  perimeter or not depends on design of the implementation."  All
Mike  this requirements says is that the design MUST "support transmission
Mike  of messages ..without compromising security"  
Mike  The requirement can be used to choose between 2 or more protocol
Mike  designs that purport to work through firewalls.

      Setting up this communication MUST NOT require changes to the
      intervening firewall(s) that weaken the security of the protected
      network(s). Nor SHOULD this be achieved by conflating IDMEF
      messages with other kinds of traffic (e.g., by overloading the
      HTTP POST method) since that would make it difficult for an
      organization to apply separate policies to IDMEF traffic and other
      kinds of traffic.

[DELETED]
6.3.1 Rationale: Since the alert messages are used by a manager to
      direct responses or further investigation related to the security 
      of an enterprise network, it is important that the receiver have
      confidence in the identity of the sender and that the sender have
      confidence in the identity of the receiver. This is peer-to-peer 
      authentication of each party to the other. It MUST NOT be 
      based on authentication of the underlying communications 
      mechanism, for example, because of the risk that this 
      authentication process might be subverted or misconfigured.
==> IESG comments not addressed.
     [This appears to rule out use of IPSEC, which is "authentication 
     of the underlying communications mechanism".  That seems ill-advised, 
     so this should be clarified. ]

XXXX I thought that this wording had changed.  It appears not to have,
XXXX nor do I see the alternate wording that was proposed
XXXX What would you propose?
XXXX In my opinion a single word would suffice:  'solely'
XXXX "It MUST NOT be based 'solely' on authentication....
GLEN Just remove that part. It seems to be alluding to some trust model 
GLEN that we want to avoid discussing !

Mike I am reluctant to remove it without some further discussion because
Mike I see it as clarifying the requirement to include features/functions
Mike beyond IPSEC.  thus, i view your proposed change as a change to a
Mike requirment and without a whole lot of WG discussion, I am reluctant
Mike to make any such change.

[DELETED]
7.2    The content of IDMEF messages MUST contain the identified name of
       the event if it is known. This name MUST be drawn from a 
==>  Do we want to know the name of the event ? 
XXXX  Yes 
==>  Perhaps we want to know the name of the attack ? 
XXXX Events are not attacks, they are what Sensor look for.
GLEN That is confusing again. Did events have names, well known names ?
GLEN I don't know what is the well known name for a user login event,
GLEN or a telnet connection event!

Mike  login and telnet
Mike  You may not at this time, but there are 2 or more registries that
Mike  are supporting such naming conventions for both events and alerts
Mike  (bugtrack, CVE)
Mike  for example, an Event which is a single scan of port 92, may be
Mike  named Scan 92, while an Alert known as a Port Scan, may be defined
Mike  as an Alert caused by N individual scans, where N is defined by the
Mike  analyzer.
Mike  What is going on here is
Mike  that we are requiring that the IDMEG support a field for specifying
Mike  that information, if it is known.
Mike  In this document it is NOT our concern where this information comes from.
Mike

       standardized list of events or will be an implementation-specific 
       name if the event identity has not yet been standardized. It is 
       not known how this list will be defined or updated, although 
       requirements on the creation of this list are presented in the 
       next section of this document.

7.2.1  Rationale: Given that this document presents requirements on 
       standardizing ID message formats so that an ID manager is able to
       receive alerts from analyzers from multiple implementations, it 
       is important that the manager understand the semantics of the 
       reported events. There is, therefore, a need to identify known 
       events and store information concerning their methods and 
       possible fixes to these events. Some events are well known and 
       this recognition can help the operator.
==> The above is not very clear. The "semantics" of reported
==> events and the attack name are directly related? 
XXXX Yes, some events and attack  names are well known.  this is not
XXXX always true.  
GLEN Huh? Event names or attack names or both ? 
GLEN "Events are not attacks" Quote from above. This is coming up again
GLEN and again. In 7.2 we are supposedly referring to event names. And
GLEN there is no clue as to what that name is. 

Mike  I don't care what that name is, ONLY that the message format have
Mike  a field for that Name.  It is NOT our job to define event names only
Mike  to support their occurrence in an IDMEF message.  The WG gave up
Mike  the task of event and alert nomenclature, to whatever other groups
Mike  exist to fulfill that function.  We are just providing a field for
Mike  the end result of that process.  No where does the Requirements Doc
Mike  require that events have names, only that the message format support
Mike  a field that can transfer that information.

7.2.2  Scenario: Intruder launches an attack that is detected by two 
       different analyzers from two distinct implementations. Both 
       report the same event identity to the ID manager, even though the 
       algorithms used to detect the attack by each analyzer might have 
       been different.
==> The term "event identity" is confusing. 
XXXX  the term "event identity" is used in a couple of places to refer
XXXX  to events that are well known and have a name/identity.  i do
XXXX  not see the confusion.  what term would you replace it with?
GLEN  If you mean that it is used in a couple of places in this document
GLEN  - that is incorrect. It appears only in this subsection namely, in
GLEN  7.2, 7.2.1 and 7.2.2.
GLEN  Maybe we should mention "event identity" in the definitions section, 
GLEN  if at all.

Mike  You are correct that this is the only place it is used, but i do
Mike  not think it needs to go in the definitions section.  I would change
Mike  7.2 to read:
Change  The content of IDMEF messages MUST contain the identified name of
Change the event (event identity) if it is known. This name MUST be drawn
Change from a
Change standardized list of events or will be an implementation-specific
Change name if the event identity has not yet been standardized. It is
Change not known how this list will be defined or updated, although
Change requirements on the creation of this list are presented in the
Change next section of this document.

[DELETED]
7.6    The IDMEF message MUST support the representation of different
       types of device addresses.

==> That is a new term that has appeared - "devices". What is a device?
==> a sensor, an analyzer , a manager ? If we need a new term it should 
==> be defined in the terminology section.
XXXX  I think it is overkill.  I would rather say something in the Rationale
XXXX  about a device being anything.  I remember a discussion that devices
XXXX  are not limited to network devices, so the Rationale might include
XXXX  new wording like, "For this Requirement, a device is anything that is
XXXX  involved in intrusion detection." and continue with the rest of
XXXX  the Rationale
GLEN  Well. There are all those tiny  devices on computers and they have 
GLEN  addresses too. (/dev/tty*, /dev/sd*, ...) The scope of "devices" in
GLEN  7.6 is not clear to me. Representing  addresses of all those devices 
GLEN  would be a challenge. I think that our scope is primarily host and 
GLEN  network devices {routers etc}. Even if it is all devices - that needs
GLEN  to be clear from the text.

Mike  I would argue that you missed the discussion of the scope of the
Mike  IDMEF and intrusions.  This was discussed at one of the early meetings,
Mike  where it was decided that the scope goes beyond network devices, e.g.,
Mike  my house.  Where I might have an 'window opening event'
Mike  I would even change what I suggest above to be:

Mike
Change  "For this Requirement, a device is anything that is
Change  involved in an intrusion and/or in intrusion detection." 

Mike  The requirement is that the IDMEF support additional addressing
Mike  mechanisms.  ARP is a protocol (design) that supports a feature
Mike  like this.  ARP which has one field (8 bits) specifying
Mike  the type of address, e.g., ethernet, atm, etc.  And other fields
Mike  (variable length) that indicate the actual address.  Whatever we
Mike  choose for the IDMEF must support such a flexible addressing structure.
Mike  Thus your scope is to small.  The wording should allow any address
Mike  type.  A particular design will limit that selection, e.g., ARP
Mike  is limited to 256 address types.

7.7    The IDMEF message MUST contain an indication of the possible
       impact of this event on the target. The value of this field MUST 
       be drawn from a standardized list of values.

7.7.1  Rationale: Information concerning the possible impact of the
       event on the target system provides an indication of what the
       intruder is attempting to do and is critical data for the
       operator to perform damage assessment. Not all systems will be 
       able to determine this, but it is important data to transmit for 
       those systems that can.
==> IESG concern on properties of the standardized list of values
==> not addressed.
XXXX  Yes, that was missed in one sense.  But as Stuart said, this is
XXXX  a Requirements Document.  Vern's response indicated concern for
XXXX  properties of the list, maintenance, etc.  Our response is that
XXXX  it is NOT a concern for this document, we are just requiring a
XXXX  field in the message format.  
XXXX  Maybe the Rationale should include some wording to the effect that
XXXX  we are making NO requirements on the list itself, only a requirement
XXXX  that the message format contain such a field.
GLEN  But we know what are the properties of this list. Why don't we just
GLEN  write it. (The IDMEF-DTD is defining a enumerated set of impacts
GLEN  that is the way we are going).

Mike  your cheating :-)  the current IDMEF is a (one) possible design of
Mike  the IDMEF.  the requirement provides a specification that that design
Mike  must fulfill.  Ideally, requirements DO NOT come after design.

[DELETED]

7.13   The IDMEF MUST support reporting alert creation date and time in 
       each event. The IDMEF MAY support reporting the date and time the
       event began in addition to the date and time the alert was
       created.
==> The IESG concern was about having an "event onset" time. Which has 
==> got included. But "event detection" time has got dropped. Is this
==> intentional? All three [onset, detection, reporting] are significant 
==> IMHO.

XXXX  See below.

7.13.1 Rationale: Time is important from both a reporting and
       correlation point of view. Event detection time might differ from 
       the alert creation time as it might take some time to actually 
       generate the alert message given that an event has been detected. 
       If the sensing element can determine the time the event occurred 
       it is strongly encouraged to place that information in the alert 
       message as well.
==> The reference to "detection time" is not appropriate as that has been
==> dropped in 7.13.
XXXX  Agree.  This is the whole time issue revisited. I am not sure how
XXXX  to change the requirement.  Any changes need to NOT significantly
XXXX  change the requirement even if we have a better understanding of
XXXX  of what we would want to say.

XXXX  So how would you reword it?
GLEN  I would get the definitions of the various times straight. Probably 
GLEN  use a diagram to explain the various times. That would help.

Mike  So, I liked Paul's suggestion which I think made it into the document.
Mike  What is wrong with that explanation?  Where are the various time
Mike  definitions not clear and how would you reword them? 

[DELETED]

8. Alert Identifiers and the Alert Identifier Definition Process
8.1   The standard list of IDMEF alerts MUST be extensible. As new events
      are defined by the community and as new methods of detecting them
      are available, the IDMEF MUST be able to grow with the technology.
==> Troublesome Terminology! This "Alert Identifier" term is VERY 
==> confusing. Are we talking about "Attack Identifier" ? 
XXXX Don't see the issue.  Alerts are NOT exclusively Attacks.  For example
XXXX courts have decided that Port Scans are NOT attacks, but I sure want
XXXX to know about them. 
GLEN See comments below.

==> Question: do we need this section for IDMEF ? 
XXXX Yes.

GLEN Well then we will need to clean up this section. It is a mess. E.g.
GLEN Alerts are what goes form analyzer to manager. They have identifiers
GLEN - that is in ID parlance (and in the IETF-IDWG parlance ) known as 
GLEN AlertID and/or Alert Identifier. 

Mike  As far as ID parlance, names are used all over the place.  We, IDWG
Mike  defined events and alerts as we wanted them used in this document and
Mike  this document alone.  we are not trying to correct ID parlance.
Mike  So, i do not see any concern with our use of the names.

GLEN This Alert Identifier distinguishes the alert from other alerts
GLEN generated by the same analyzer (and  other analyzers too!). The
GLEN scope and degree of the uniqueness is a topic of hot debate.   

Mike  True.  8.1 just says that whatever list we might use for alerts,
MIke  It must be expandable.  there is NO requirement specified about
Mike  uniqueness - maybe there should, but there is none.
      
Mike  8.2  just says that whatever design we choose for IDMEF, it must
Mike  be extensible.  We do not say how, nor how much flexibility must
Mike  be in the design, just that it needs extensibility.

Mike  8.3  just says that IDS can have their own proprietary list.

GLEN So, the term Alert Identifier cannot be used, as a default, for 
GLEN the purposes of this section. 

Mike  To me, it is well defined here.  if it is not well defined in
Mike  other docs, then define it in those docs.  This doc says nothing
MIke  about uniqueness or anything else, other than the list of Alert
Mike  Identifiers should be expandable.

-- 
mike erlinger, professor computer science
www:      http://www.cs.hmc.edu/~mike
email:    [email-masked]
smail:    computer science dept harvey mudd college,
          301 E. 12th Street, Claremont, CA, 91711 
909-621-8912,   FAX: 909-621-8465