Subject: [OPSAWG] comments on draft-ietf-opsawg-syslog-snmp-00

From: Martin Bjorklund <[email-masked]>
Date: Mon, 23 Feb 2009 21:54:09 +0100 (CET)

Hi,

Some comments on this draft.

The picture on page 6 could benefit from changing some of the lines
into arrows - I think the picture tries to illustrate the flow of
notifications (from right to left).

The grammar in section 3.2 (and also section 5) makes the ctxEngineId
and ctxEngineName SD-ELEMENT parameters optional.  But the text in 3.2
says:

  The first two PARAM-NAME elements are "ctxEngine" and "ctxName".
  They must be present in an SNMPv3 notification and therefore they
  must be present in a SYSLOG message generated by an snmp-to-syslog
  translator.

So the question is if they are optional, or maybe MUST be present?

Further, the text in 3.2 says "the contextName is encoded as a
hexadecimal string." but the grammar specifies it as a UTF8 string.

The grammar in 3.2 also includes an optional VARLABEL, which isn't
further discussed in the text.  I assume the intention is that if this
label is known to the snmp-to-syslog translator, then it may be
included, to make the resulting syslog message easier to read.  But
this makes me wonder a bit of the environments in which this kind of
translator will be used, and the objectives of this draft.  The
mapping as it is defined isn't really useful unless an application on
the receiving side (syslog server) has SNMP / MIB knowledge.
Specifically, a human user cannot easily read those messages and
extract useful information.  I suspect human readability is not the
goal of this mapping.  I think this should be discussed in the intro
of the draft.  If the mapping is supposed to cover both machine and
human readability, then the VARLABEL seems a bit arbitrary (since even
DisplayStrings are encoded in hex - *if* the translator has MIB
knowledge it would seem fine to encode DisplayStrings and Utf8Strings
as utf8)

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 24 Feb 2009 09:00:20 +0100

> notifications (from right to left).

I have added arrow heads.
 
> So the question is if they are optional, or maybe MUST be present?

The idea is that "ctxEngine" and "ctxName" are a MUST for SNMPv3 and
they are optional for other versions of SNMP. This is hinted at on
page 4:

   In the case of SNMPv1 or SNMPv2c notifications, the contextEngineID
   and the contextName parameters are not present in notification
   messages.

I will clarify the sentence as follows:

   The first two PARAM-NAME elements are "ctxEngine" and "ctxName".
   They must be present in an SNMPv3 notification and therefore they
   must be present in a SYSLOG message generated by an snmp-to-syslog
   translator from an SNMPv3 notification.

> hexadecimal string." but the grammar specifies it as a UTF8 string.

Since SnmpAdminString is used in several MIB modules where context
names appear, I assume UTF8 string is the right choice. New text:

   The contexdEngineID is encoded as as hexadecimal string while the
   contextName is encoded as a UTF8 string.
 
> knowledge it would seem fine to encode DisplayStrings and Utf8Strings

The question boils down to whether we can assume MIB knowledge of the
translator or not. I believe in the general case we can not assume
full MIB knowledge since a certain percentage of notifications tends
to be device/vendor specific and it is unrealistic to assume that an
implementation will know all MIB modules. If this is true, then we
have to ship OIDs and hex data for certain notifications to the
receiver. So one extreme approach is to simply keep the translator
dumb and put the burdon on the receiver's side (and it should be
fairly easy to write a filter, e.g. an extended version of smixlate,
that takes SYSLOG messages and translates OIDs and values into
something more human friendly). This is where this document originally
started from.

However, for some device vendors who like to put the translator into
their devices, the situation is different because they can ensure that
they have full MIB knowledge and thus translations can happen on the
device. This is why we added VARLABEL and the question that you bring
up now is whether we should also support alternate value
representations via an 'a' parameter, which contains the value
representation generated by applying DISPLAY-HINTs or replacing
numbers with labels in enumerations and bits so that

     v="1.3.6.1.2.1.1.3.0" l="sysUpTime.0" d="94860"
     v="1.3.6.1.6.3.1.1.4.1.0" l="snmpTrapOID.0" o="1.3.6.1.6.3.1.1.5.4"
     v="1.3.6.1.2.1.2.2.1.1.3" d="3"
     v="1.3.6.1.2.1.2.2.1.7.3" d="1"
     v="1.3.6.1.2.1.2.2.1.8.3" d="1"

becomes

     v="1.3.6.1.2.1.1.3.0" l="sysUpTime.0" d="94860"
     v="1.3.6.1.6.3.1.1.4.1.0" l="snmpTrapOID.0" o="1.3.6.1.6.3.1.1.5.4" a="linkUp"
     v="1.3.6.1.2.1.2.2.1.1.3" l="ifIndex.3" d="3"
     v="1.3.6.1.2.1.2.2.1.7.3" l="ifAdminStatus.3" d="1" a="up"
     v="1.3.6.1.2.1.2.2.1.8.3" l="ifOperStatus.3" d="1" a="up"

The only downside is the obvious increase in length but on the other
hand the operational value of the result might increase significantly.

Comments?

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 24 Feb 2009 11:19:28 +0100 (CET)

>    and the contextName parameters are not present in notification

But the next sentence after the one you quoted says:

   In general, we assume that notifications from an SNMP version
   preceding SNMPv3 are mapped into the notification format used by
   SNMPv3 according to the coexistance rules defined in RFC 3584
   [RFC3584].

After translation, each notification will have a contextEngineID and
contextName.  Otherwise, what is the point of this sentence?

> full MIB knowledge 

Agreed.

> since a certain percentage of notifications tends

Ok.  I think a short note about this should go into the introduction.

But stepping back, is it realistic to assume that an environment
exists where you can have a syslog server with full MIB knowledge, but
not have a SNMP trap sink?

>      a="linkUp"

I think that something like this would be useful, and by making it
optional, the translator can decide if the increased size is
acceptable or not.

The 'a' parameter seems useful, but in many cases it seems the
information is redundant - if the receiver does not have MIB
knowledge, the 'd' parameter is not very useful, but 'a' is.  OTOH if
the receiver has MIB knowledge, the 'd' value can be calculated from
the 'a' value.   As another example, if the type is DisplayString it
seems like a waste of bytes to send the value as a ascii string *and*
as a hexstring.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 24 Feb 2009 13:33:31 +0100

> contextName.  Otherwise, what is the point of this sentence?

Its about the mapping of the notification payload format. In concrete
terms: SNMPv1 traps are mapped using RFC 3584 to the new format. The
text could probably be more specific. The fine detail here is that an
automatic extraction of "ctxEngine" and "ctxName" out of
SNMPv1/SNMPv2c messages is not possible unless the translator has
access to a mapping table (which might be the case if the translator
is integrated into the management software of a device but once the
translator sits outside the device, this information is usually not
available).
 
> not have a SNMP trap sink?

Mappings from SNMP to SYSLOG have been done in the past and I believe
the reasons for having these mappings will persist. Architecturally,
the SNMP to SYSLOG translator is an SNMP notification receiver (an
SNMP trap sink). So I am not sure I understand your question.

> seems like a waste of bytes to send the value as a ascii string *and*

So you are arguing now against having the 'a' format since it is
redundant? I like to have a basic format I can rely on being there all
the time. If there is additional rendering for human beings, fine. But
I would hate to get something like:

  l="asdf.42" a="boogoo"

where I have no chance to figure out what it means - is boogoo an enum
or a string value? And what is asdf.42 anyway? So I like to have
always the OID and a straight mapping of the value and its type
information.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 24 Feb 2009 15:27:15 +0100 (CET)

> translator sits outside the device, this information is usually not

Right.  From the draft I understood that the 3584 transformation is
supposed to be done, but now you write that it might not be possible
to do the conversion.  So what happens when the snmp-to-syslog
translator receives a v1/v2c trap?

> the reasons for having these mappings will persist.

Absolutely!

> Architecturally,

My question is if it will be useful to deploy a translator that
translates into a "binary" form that a human user cannot understand
without a tool that has all MIBs.

> redundant? 

No.  I just wonder if it would make sense to drop the other parameter
(in the case above drop 'd'), if 'a' is provided.

> I like to have a basic format I can rely on being there all
> the time.

There is already a choice in the grammar for VARVALUE; 'a' could be
another choice there.  I.e. even in the format in the draft there is
no single VARVALUE always present.

> If there is additional rendering for human beings, fine. But
> or a string value?

In the worst case you need MIB knowledge to figure it out, and in the
best case you do not need it.  With the "binary" encoding you *always*
need MIB knowledge.

Again, I wonder about the deployment scenario.  If all the detailed
information is important, don't you typically have a proper SNMP
notification receiver?  Isn't one reason for existing snmp->syslog
translators that the notification gets "readable"?

> And what is asdf.42 anyway?

I'm sure there are notifications with objects names like this, but to
be fair - most objects do have better names than this.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 24 Feb 2009 17:05:12 +0100

> translator receives a v1/v2c trap?

I think you are confusing two things and we probably have not done a
good job explaining things well:

(1) There is the translation of the SNMPv1 trap PDU into RFC 3416 PDU
    formats.  This translation is always posssible and does not
    require any additional information. So a pointer to RFC 3584
    solves this issue.

(2) There is the context information that does not exist in
    SNMPv1/SNMPv2c message formats. While RFC 3584 provides a MIB
    module that can do this translation, someone has to configure the
    relevant tables. Since this is not realistic to assume in all
    translator deployments, we made the context objects optional.

Does this make sense now?
 
> without a tool that has all MIBs.

Well, much less binary than BER encoded stuff. ;-) My point is that
for some notifications, a generic translator can't do much more than
producing hex strings.

> no single VARVALUE always present.

This could probably work but needs to be worked out in more detail.
So your proposal is to distinguish between rendered strings and
enumerations by adding more choices to the enumeration? Or do you
think it does not matter to distinguish between the string value "up"
and the enum value "up"? Or do you want to keep d= and a= for enums
but drop the hexadecimal string only for OCTET STRINGS that have an a=
representation?

> translators that the notification gets "readable"?

Perhaps we need to agree on some principles:

(i)   A translator should not loose information and must no make
      information carried in a notification message ambiguous.

(ii)  MIB knowledge is not required by a translator.

(iii) A translator with MIB knowledge may generate an improved human
      readable format as long as (i) and (ii) are met.

Do we agree on this?
 
> be fair - most objects do have better names than this.

My point was that a label is not a unique identifier and this is why I
so far insisted that the OID is always kept and VARLABEL is optionally
added for improved human readability.

Now for OCTET STRINGS where the translator knows about DISPLAY-HINT
rules, I think we can add a special case so that hexadecial
representation of the value can be skipped (assuming that the tag of
the new string representation indicates that the underlying
representation really was on OCTET STRING).

So then the question remains what we do about enumerations and bits or
OBJECT IDENTIFIER constants.

/js

---

From: Rainer Gerhards <[email-masked]>
Date: Tue, 24 Feb 2009 17:24:03 +0100

> for some notifications, a generic translator can't do much more than

As someone who does such generic translators today, I agree to
Juergen's point. There are always cases left where you need to stuff a
hex string into the message because you do not know enough to do
anything else.

Rainer

---

From: "David Harrington" <[email-masked]>
Date: Tue, 24 Feb 2009 12:28:17 -0500

> contextName.  Otherwise, what is the point of this sentence?

I think that converging them into a common format makes sense, and
since that conversion is already defined in RFC3584, it makes sense to
use that.

> not have a SNMP trap sink?

As I think about this, I envision devices only capable of sending
syslog messages to a syslog receiver. But at the syslog receiver,
having a translator to a local SNMP engine (which might also be a trap
sink for devices that can send traps.)

One advantage I can see is congestion control and possible application
acknowledgement. Syslog has no acknowledgements, but SNMP does. So if
a syslog message is sent to a syslog receiver, translated into SNMP,
and then sent as an Inform, then you more-or-less add acknowledgements
to syslog. Granted, the acknowledgement does not go to the syslog
originator, but converting from syslog to Informs means you can
provide better congestion control between the translator/forwarder and
the trap sink. 

If the syslog messages are not received over the wire, but are "sent"
to local logs, then you could have a situation where you have syslog
support and SNMP support on the same device, and use the converter
between daemons, to handle, device drivers that only speak syslog, but
you can convert those to SNMP Informs which gives you system-level
acks, even if the device driver doesn't get the ack. Given such an
ack, you can suppress resends of syslog messages, in much the same way
Informs allow you to limit resends better than traps.

David Harrington
[email-masked]
[email-masked]
[email-masked]

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 25 Feb 2009 10:05:37 +0100 (CET)

> Does this make sense now?

I think the problem is the text:

   In general, we assume that notifications from an SNMP
   version preceding SNMPv3 are mapped into the notification format used
   ^^^^^^^^^^^^^^^^^^^^^^^^
   by SNMPv3 according to the coexistance rules defined in RFC 3584

So it seems to me that you require a v2 notification to be converted
into "the notification format used by v3".  Anyway, it seems we agree
that this text should be clarified.

> for some notifications, a generic translator can't do much more than

I absolutely agree.

> but drop the hexadecimal string only for OCTET STRINGS that have an a=

I must admit that I have not worked out a complete proposal for this -
my first idea was just that the value could be easier to read by
saying that *if* the translator has MIB knowledge then it MAY send
"human readable" strings as utf8, instead of hex.  Then you suggested
that the same could be done for enums and bits, which I think makese
sense as well.

>       readable format as long as (i) and (ii) are met.

This seems very natural.

> added for improved human readability.

Yes, I agree that this follows from (i) above.  I think the document
needs to describe the intended usage of VARLABEL though.

> representation really was on OCTET STRING).

Yes.

> OBJECT IDENTIFIER constants.

One option could be to include add parameters 'E', 'B', 'O' which
would be the textual representation of enums, bits, and OBJECT
IDENTIFIER.  In the case of the OBJECT IDENTIFIER value, I guess it
has to be on the format <module>.<name>, or that the parameter 'o' is
also needed (i.e. that 'O' is just a complement to 'o').

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 25 Feb 2009 12:25:52 +0100

>    version preceding SNMPv3 are mapped into the notification format used

Here is the proposed new text:

   This document assumes that notifications are in the format defined in
   RFC 3416 [RFC3416].  Notifications in the SNMPv1 notification format
   must be translated as described in Section 3.1 of RFC 3584 [RFC3584].

> needs to describe the intended usage of VARLABEL though.

Yes. I will add text.
 
> > representation really was on OCTET STRING).

I just typed "I will do this" when I figured out that there is still a
potential problem. A MIB aware translator able to apply DISPLAY-HINTs
will turn a MacAddress binary value '00E0812E961E'h into
"00:E0:81:2E:96:1E" and a MIB unaware receiver might turn this back
into '30303A45303A38313A32453A39363A3145'h. This would be bad. I see
two options:

a) We always ship the hex value in addition to the rendered value. We
   might consider using base64 instead of hex to save space.

b) We restrict the alternate textual representations to DISPLAY-HINTs
   of the form 'Na' or 'Nt' where N is some number. This implies that
   a MacAddress or a DateAndTime value will be carried in hex / base64
   representation.

Hm, not nice.

> also needed (i.e. that 'O' is just a complement to 'o').

Since we can't expect that the receiver has MIB knowledge to turn a
label such as "up" back to a number (and formally, the receiver would
have to have the same version of the MIB module), I think we have to
send the original value in the case of enumerations / BITS / OBJECT
IDENTIFIER values and the alternate format is an optional
representation. Given this, the simplest solution for the string
mapping problem mentioned above would be to actually do the same.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 25 Feb 2009 13:37:24 +0100 (CET)

>    a MacAddress or a DateAndTime value will be carried in hex / base64

[...]

> Given this, the simplest solution for the string

Yes.  An alternative could be that for the DISPLAY-HINTs in case b)
above, the hex string does not need to be sent.

/martin