Subject: [PMOL] feedback on draft-morton-perf-metrics-framework-02

From: Benoit Claise <[email-masked]>
Date: Tue, 18 Mar 2008 16:58:40 +0100

Dear all,

Here is my feedback after reading the draft.

- First of all, as mentioned during the WG meeting, the following 3 
items seem to be a structure for the draft.

   There are at least three phases in the development of performance
   standards.  They are:

   1.  Definition of a Performance Metric and its units of measure

   2.  Specification of a Method of Measurement

   3.  Specification of the Reporting Format

I understand it's not the case, so the focus of the document compared to 
these 3 points should be explained
- you mention "Performance Metrics Entity" in the abstract. If 
capitalized, doesn't it imply a definition?
The draft only mentions:

   This memo refers to a Performance Metrics Entity, or PM Entity, which
   may in future be a working group or directorate or a combination of
   these two.

You might to mention what its goal is.
- " (ii) the correlation between the metric and the quality of service / 
experience delivered to the user (person or other application)"
Ideally the metric is the quality of service/experience
- In  section 3.4.2,  an example of the Service Level Contract/Service 
Level Agreement would seem ideal, as the SLA must be clearly, including 
the way of metering it.
- 
- 3.4.4. Metric Definition Template

    Name
    Description
    Measurement
    Method
    Units of measurement
    Measurement or Sampling Interval
    Use and Applications
    Implementation
    Reporting Model

Which ones MUST/SHOULD be present? All of them?
- Section 3.7 "Reporting protocols typically introduce some limitations 
and assumptions with regard to the definition of a metric."
After discussing with Alan after the meeting, I understand what you want 
to say. However, I think it should be expanded.
- Section 3.12. I don't understand why we have the following excerpt in 
there:

   The key areas of variation between different metric users include:

   o  Suitability of passive measurements of live traffic, or active
      measurements using dedicated traffic

   o  Measurement in laboratory environment, or on a network of deployed
      devices

   o  Accuracy of the results

   o  Access to measurement points and configuration information

   o  Measurement topology (point-to-point, point-to-multipoint)

   o  Scale of the measurement system

   o  Measurements conducted on-demand, or continuously

   o  Required reporting formats

   ------------------- end of Al's sections, but more in 4.2 -----

_Editorial:_
- you might want to give a reference to RAQMON
- section 3.1 " Metrics are intended for use in measuring the 
performance of an application, network or service." while section 3.2 
mentions "A metric is a measure of an observable behavior of an 
application, protocol or other system."
- realign

   (i) Description of metric The description should explain (ii) Metric
   use and applications (iii) Implementation (iv) Reporting Model

Regards, Benoit.

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol