Subject: [PMOL] Fwd: New Version Notification for draft-ietf-pmol-metrics-framework-05

From: Benoit Claise <[email-masked]>
Date: Sun, 10 Oct 2010 22:49:27 +0200

Dear all,

Here is a new version of the framework draft.
What is new?
     - improve the Performance Metric definition
     - change the title to RFC5706
     - PM Entity defined as a directorate

No more open issues in the draft. This should be ready for WGLC.

Regards, Benoit.

-------- Original Message --------
Subject: 	New Version Notification for draft-ietf-pmol-metrics-framework-05
Date: 	Sun, 10 Oct 2010 13:38:40 -0700 (PDT)
From: 	IETF I-D Submission Tool <[email-masked]>
To: 	[email-masked]
CC: 	[email-masked]

A new version of I-D, draft-ietf-pmol-metrics-framework-05.txt has been successfully submitted by Benoit Claise and posted to the IETF repository.

Filename:	 draft-ietf-pmol-metrics-framework
Revision:	 05
Title:		 Guidelines for Considering New Performance Metric Development
Creation_date:	 2010-10-10
WG ID:		 pmol
Number_of_pages: 21

Abstract:
This document describes a framework and a process for developing
performance metrics of protocols and applications transported over
over IETF-specified protocols, and that can be used to characterize
traffic on live networks and services.

The IETF Secretariat.

---

From: "Aamer Akhter (aakhter)" <[email-masked]>
Date: Sat, 16 Oct 2010 14:14:42 -0500

Hi Benoit,

Please find my comments below. Overall I found the guidelines very much on the right path – and (unsurprisingly) do mirror the internal work discussions we have done in our definitions process. 

section 1.1

The motivation does not seem to cleanly resolve whether the generation of upper layer performance metrics needs to be separated from the monitoring and transport frameworks (RMON, IPFIX). Unless this is done, there may be overlapping efforts across the WGs. This is somewhat talked about in section 6, but can we have clear guidance between separation between metrics definition, transport, and verification/validation. The verification/validation being something perhaps we can extend in BMWGs charter.

section 2.3

"The quantitative component can be influenced by the

   complete end-to-end system effects (network infrastructure).". 

Not clear what this means. 

What is the difference between application and network infrastructure? Where do network transcoders sit?

Is HD vs. SD video a qualitative or quantitative component?

section 2.4

"specific to an IETF-specified

   protocol or specific to an application transported over an IETF-

   specified protocol." then "database logging time"

does this restrict the evaluation of database logging time to IETF protocols (TCP, HTTP), and not any deeper?

section 3.

Are active and passive measurements widely understood terms? Maybe just a reference.

section 5.1

                "The key areas of variation between different metric users..." 

                Not sure if this covered by the exiting cases but please add a statement regarding the performance context that is being considered. For example, the network operator may find it convenient to look at RTP flows as an indicator of network health rather than a specific interest in the health of the RTP flow. In that regard, some of the QoE aspects specific to the application may not matter at all. 

is time range part of the existing list (real-time, delayed reporting, summarized reporting, etc)

section 5.4.2

"   Performance Metric names MUST be unique within the set of metrics

   being defined and MAY be descriptive." 

   Unclear why a metric name would _not_ be descriptive. Perhaps an example is needed?

section 5.4.2

                "If the measurement is specific to a measurement point this SHOULD be

   defined.  The measurement domain MAY also be defined.  Specifically,

   if measurement points are spread across domains, the measurement

   domain (intra-, inter-) is another factor to consider.

   In some cases, the measurement requires multiple measurement points:

   all measurement points SHOULD be defined, including the measurement

   domain(s)."

   It would be helpful to have references to more discussion on what defines a measurement domain. Is a measurement domain an administrative domain? Is the network portion of a flow leg a measurement domain? What if a NAT is involved? Are there two domains then?

section 5.4.2

Slightly related to measurement timing is the issue of granularity of measurement needed to compute the metric. For example it is possible to sample RTP flows to determine packet loss. However it is not possible to sample packets from a single RTP flow to determine packet loss. 

section 6.2

                "repeatable"

                I think for application performance this might mean a complete description of several workloads to be tested against. How will this be done? ie are we talking about a pcap file-- if so what would that pcap file need to look like? How would it be distributed?

 

From: [email-masked] [mailto:[email-masked]] On Behalf Of Benoit Claise (bclaise)
Sent: Sunday, October 10, 2010 4:49 PM
To: [email-masked]
Subject: [PMOL] Fwd: New Version Notification fordraft-ietf-pmol-metrics-framework-05

Dear all,

Regards, Benoit.

-------- Original Message -------- 

Subject: 

New Version Notification for draft-ietf-pmol-metrics-framework-05

Date: 

Sun, 10 Oct 2010 13:38:40 -0700 (PDT)

From: 

IETF I-D Submission Tool <[email-masked]> <mailto:[email-masked]> 

To: 

[email-masked]

CC: 

[email-masked]

 
Abstract:

The IETF Secretariat.