Subject: RE: re-review of Notify Log MIB

From: Randy Presuhn <[email-masked]>
Date: Mon, 20 Dec 1999 12:43:25 -0800 (PST)

Hi -

Long message, important question at the bottom!  If your head
hurts already, just jump to the end.  :-)

> From: "Sharon Chisholm" <[email-masked]>
> To: Randy Presuhn <[email-masked]>, [email-masked]
> Subject: RE: re-review of Notify Log MIB
> Date: Mon, 20 Dec 1999 10:52:14 -0500
...
> I'm not sure who is saying what in this discussion, but I
> left to original headers in just in case someone else can
> follow them.
...

It's not important who said what.  What matters is that we
resolve the IESG reviewer comments and finish this document
sensibly.

...
> |[*] Ok. will make the "flush mandatory".
> What!?  What is someone is consuming the log and has not finished consuming
> the
> entries you are flushing?  Doesn't it make more sense and keep things simple
> and
> keep the monotonically increasing index and discourage flushing? What good
> is
> the log if I don't have *any* guarantee that notifications logged within the
> last few minutes are stored?
...

That's an unfortunate consequence of the indexing structure
adopted for the logs and the "flushing" text.  The WG consensus
on the index structure was for simple integers, with folks
fully aware that wrap behaviour would be a problem.  A more
sophisticated approach, with a mib object reporting the index
of the next entry to be added, and more elaborate DESCRIPTION
text describing when entries could be flushed would avoid
this problem.

However, we are so far behind schedule with this document
that it stopped being funny YEARS ago.

We had two choices:
   1) spend even more time on it before doing anything
   2) get it out at proposed,
      get some operational experience with it,
      REVISE AS NEEDED.

The clear preference of the WG was #2.  Our AD has come down
in favor of #2.  I want to get this thing off my plate.

Without adding additional objects to the MIB, the other
way to resolve the reviewer's objection would be to
delete the words "and may flush existing entries"
from the DESCRIPTION of nlmLogIndex.

However, since the persistance (or lack thereof) of log entries 
is not defined, there are no guarantees that stuff won't be lost
due to a reboot, or pushed out by other incoming notifications,
possibly as a result of the attack Wes described.

...
> Actually, nlmStatsLogNotificationsLogged is part of the nlmStatsLogTable 
> which is indexed by nlmLogName.  So, if you added it a notification to
> more than one log, you would increment nlmStatsLogNotificationsLogged for
> each of the logs, not one statistic more than once.  I think you guys were
> thinking of nlmStatsGlobalNotificationsLogged.
> nlmStatsLogNotificationsLogged 
> can more or less be used when consuming the logs.

But there is still no guarantee that the number a notification
receiver application should have received will match the number
that should have gone into a particular log.  Referring to
the same snmpNotifyFilterTable entry does not imply that the
log and the target would have the same credentials.

> |
> |>yup.  I don't think the detection of notification loss was a design
> |>requirement. 
> Perhaps not, but it is still an important use of the MIB in my 
> somewhat biased opinion. 

This design won't do what you need.  The count of TRAPs/INFORMs
sent per security name at each target would need to be tracked.
This would require a new table, supplementing the ones in RFC 2573.
This is not our job.

> |>> > 3. Poll the agent to find out which notification (if any) 
> |>> > you've lost. If
> |>> > the log index
> |>> > were not monotonically, the manager will need to read in the 
> |>> > entire log. If it
> |>> > were monotonically increasing, then all he needs to do is do 
> |>> > a get-next from
> |>> > the last index that he polled for.
> |>> > 
> |>> > Goiung back to the point you raised about "wrapping of the 
> |>> > logIndex back to 1",
> |>> > if we don't delete all the notification logs in the table, 
> |>> > then we'll be
> |>> > unable to determine
> |>> > which notification we lost without polling the ENTIRE table. Ouch!
> |>> > For that alone, I'd say an implementation MUST delete all 
> |>> > notifications in
> |>> > the log at that time.
> |>> > Comments?
> No, I thought randy posted a nice algorithm of how to deal with wrapping
> while consuming the log.  Basically, if the next row in the table does not
> exist you check back at the beginning of the table and check the nlmLogTime
> or nlmLogDateAndTime.
...

If we were to delete the "and may flush existing entries"
words, this would be the only way to figure out what was
recent.  As I posted earlier in this thread, I believe the
processing cost of getting the set of "recent" entries to be
no worse than that entailed by approaches requiring additional
MIB objects.

Since there is a strong objection to "must flush", would there
be any objections to resolving the IESG reviewer comment by
deleting the words "and may flush existing entries" from the
DESCRIPTION of nlmLogIndex?

 ------------------------------------------------------------------------
 Randy Presuhn           [email-masked]       http://www.bmc.com/
 Voice: +1 408 546-1006  BMC Software, Inc.  1-3141  2141 N. First Street
 Fax:   +1 408 965-0359  San Jose, California 95131  USA
 ------------------------------------------------------------------------
 Any relationship between my opinions and BMC's should be coincidental.
 ------------------------------------------------------------------------

---

From: Ramanathan Kavasseri <[email-masked]>
Date: Mon, 20 Dec 1999 14:46:50 -0800

>hurts already, just jump to the end.  :-)

[...snip...]

>no worse than that entailed by approaches requiring additional

Assuming we permit wraps, how will you differentiate an old
entry from a new one? What if the log wraps, and at the same time,
the agent resets - perhaps it's far-fetched, but it's a valid scenario.
As was pointed out above, if the MIB
or the agent gets reset, you'll still need to deal with notification
log loss. 

I would like to see the "MUST be flushed" text, with the rider that
we resolve the issue when we get implementation feedback the
next time around. A solution (which is ugly, and will likely
get shot to bits, but I'll throw it out anyway) is to perhaps
have a "history table" that holds the entries that were
flushed at wrap time...There should be a timelimit
after which entries in this table will be aged out. This way,
we get the "flushed entries" behaviour, while trying to minimize
any possible notification loss. I still consider it ugly
to try any sort of los-prevention as the purpose of the MIB is
not to "guarantee" that logs will remain until read.
Besides, I wonder how long the engine will need to run to generate
that amount of logs...BTW, please note that I'm suggesting this abysmal
solution (or anything else) be reserved till implementation feedback,
so we can get *some* information on this and other possible
issues with the MIB...

>DESCRIPTION of nlmLogIndex?

If we don't say "must flush", we'll then need to modify the DESCRIPTION for
nlmLogIndex to describe the intended behaviour at wrap-time...
And we'll need to note that a "get-next" of the last index read
will not necessarily result in a new log entry - how useful is that?
If there is still strong objection to a "flushing entries" scenario, then I'll
remove the "and may flush existing entries" text, but I'd like to know
what the intended behaviour should be at that point.
 
Ram