Subject: [OPSAWG] guidelines: Performance Management

From: "David Harrington" <[email-masked]>
Date: Tue, 9 Sep 2008 19:19:12 -0400

Hi,

The section on Performance Management in
draft-ietf-opsawg-operations-and-management lumps device mgmt, network
mgmt, and service mgmt together. Should these be separated?

If so, can you provide proposed text changes to separate these?

David Harrington
[email-masked]
[email-masked]
[email-masked]

_______________________________________________
OPSAWG mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/opsawg

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 15 Sep 2008 17:37:42 +0200

(contributor opinion) 

I am not sure that the separation is necessary. We are dealing mostly
with protocol documents that define new protocols or protocol
extensions. We are concerned how the performance of the respective
protocols is being defined and managed. What is important in my opinion
is to list what are the key performance metrics (or mention the need to
define such metrics) and to be clear what hooks for performance
management must be defined within the protocol. 

Here is a slight re-write of the existing text on these lines: 

-----------

   From a manageability point of view it is important to determine how
well
   a network deploying the protocol or technology defined in the
document 
   is doing. In order to do this the network operators need to  
   consider information that would be useful to determine
   the performance characteristics of a deployed system using the target
   protocol. In many cases the performance metrics are generic and 
   already defined by work done in the IPPM WG, or BMWG for example, 
   but in some cases new metrics need to be defined (example [SIP
Performance Metrics] in PMOL).
   It would be useful if the document would identify the need for such
new 
   metrics. 

Here is a list of questions that could be answered and items that can be

Discussed related to performance management in a manageability section: 

   - What are the principal performance factors that need to be looked
at
   when measuring the operational performance of the protocol
   implementations?  Is it important to measure setup times? throughput?
   quality versus throughput? interruptions? end-to-end throughput? end-
   to-end quality? hop-to-hop throughput?

   - Consider scalability, such as whether performance will be affected
by
   the number of protocol connections.  If so, then it might be useful
   to provide information about the maximum number of table entries that
   should be expected to be modeled, how many entries an implementation
   can support, the current number of instances, and the expected
   behavior when the current instances exceed the capacity of the
   implementation.  This should be considered in a data-modeling
   independent manner - what makes managed-protocol sense, not what
   makes management-protocol-sense.  If it is not managed-protocol-
   dependent, then it should be left for the management-protocol data
   modelers to decide.  For example, VLAN identifiers have a range of
   1..4095 because of the VLAN standards.  A MIB implementing a VLAN
   table should be able to support 4096 entries because the content
   being modeled requires it.

   - Consider the capability of determining the 
   operational activity, such as the number of message in and
   the messages out, the number of received messages rejected due to
   format problems, the expected behaviors when a malformed message is
   received.

   - Consider the expected behaviors for counters - what is a reasonable
   maximum value for expected usage? should they stop counting at the
   maximum value and retain the maximum value, or should they rollover?
   How can users determine if a rollover has occurred, and how can users
   determine if more than one rollover has occurred?

   - What performance management information should be maintained 
   across reboots of the device, or restarts of the management system?

   - Could events, such as hot-swapping a blade in a chassis, cause
   discontinuities in information?  Does this make any difference in
   evaluating the performance of a protocol?

   - Consider whether multiple management applications will share
   counters; if so, then no one management application should be allowed
   to reset the value to zero since this will impact other applications.

-------------

I have deleted the last two paragraphs which do not seem essential to
me. 
Of course, it's just my opinion. 

Dan

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of David Harrington
> Sent: Wednesday, September 10, 2008 2:19 AM
> To: [email-masked]
> Subject: [OPSAWG] guidelines: Performance Management
> 
> Hi,
> 
> draft-ietf-opsawg-operations-and-management lumps device 
> mgmt, network mgmt, and service mgmt together. Should these 
> be separated?
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 15 Sep 2008 18:43:16 +0200

On Mon, Sep 15, 2008 at 05:37:42PM +0200, Romascanu, Dan (Dan) wrote:
 
>    - Consider scalability, such as whether performance will be affected by

I am not sure the VLAN identifier space is a good example given the
recent trend in the IEEE to extend this identifier space by treating
1..4095 as global identifiers and numbers > 4095 as local identifiers.
The notion of a "maximum number" has often been wrong for something
that is successful and from a modeling point hard constraints should
really be introduced with great care. This document might not be the
place to spell this out - but the text above can perhaps be
mis-understood in a way that adding constraints makes sense.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
OPSAWG mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/opsawg

---

From: "David Harrington" <[email-masked]>
Date: Mon, 15 Sep 2008 21:32:46 -0400

I think keeping the VLAN modeling could be good for explaining how
models whould make "protocol" sense for the protocol being modeled.
But it would also be useful to point out that constraints could be
misjudged if a protocol becomes more successful than expected. There
is an engineering tradeoff involved. I think both points are valid
considerations.

dbh

> From: Juergen Schoenwaelder 
> [mailto:[email-masked]] 
> Sent: Monday, September 15, 2008 12:43 PM
> To: Romascanu, Dan (Dan)
> Cc: David Harrington; [email-masked]
> Subject: Re: [OPSAWG] guidelines: Performance Management
>

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 16 Sep 2008 08:06:06 +0200

The case is a very good example for both situations, with the evolution
of the VLAN space leading to the need to extend the model with new
objects, as SMI rules do not allow extensions in range or size on the
same object. The recommendation in the document should be data-modeling
language independent, with the core recommendation being that the
protocol document should make clear the limitations implicit within the
protocol and behavior when limits are exceeded. If VLANs were defined in
the IETF I would expect the RFC that define them to mention that from a
version to the other the limits derived from VLAN range and structure
have changes and new/modified objects are probably needed in the data
models. 

Dan

> From: David Harrington [mailto:[email-masked]] 
> Sent: Tuesday, September 16, 2008 4:33 AM
> To: [email-masked]; Romascanu, Dan (Dan)
> Cc: [email-masked]
> Subject: RE: [OPSAWG] guidelines: Performance Management
> 
> I think keeping the VLAN modeling could be good for 
> explaining how models whould make "protocol" sense for the 
> protocol being modeled.
> But it would also be useful to point out that constraints 
> could be misjudged if a protocol becomes more successful than 
> expected. There is an engineering tradeoff involved. I think 
> both points are valid considerations.
> 
> dbh
>