Subject: [ippm] I-D Action:draft-ietf-ippm-metrictest-00.txt

From: [email-masked]
Date: Mon, 5 Jul 2010 00:45:02 -0700 (PDT)

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the IP Performance Metrics Working Group of the IETF.

	Title           : IPPM standard advancement testing
	Author(s)       : R. Geib, et al.
	Filename        : draft-ietf-ippm-metrictest-00.txt
	Pages           : 36
	Date            : 2010-07-02

This document specifies tests to determine if multiple independent
instantiations of a performance metric RFC have implemented the
specifications in the same way.  This is the performance metric
equivalent of interoperability, required to advance RFCs along the
standards track.  Results from different implementations of metric
RFCs will be collected under the same underlying network conditions
and compared using state of the art statistical methods.  The goal is
an evaluation of the metric RFC itself, whether its definitions are
clear and unambiguous to implementors and therefore a candidate for
advancement on the IETF standards track.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-ippm-metrictest-00.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

---

From: Jerome Benoit <[email-masked]>
Date: Tue, 6 Jul 2010 14:19:33 +0200

> http://www.ietf.org/internet-drafts/draft-ietf-ippm-metrictest-00.txt

Do you plan do add inaccuracy introduced by runtime on each
implementation ? 
Do you plan to you plan to test implementation that
will not follow IETF OWAMP (or TWAMP) RFC but will largely inspired by
them (NAT friendliness of theses RFC make them difficult to follow
when you want to send probes from the network edge et behind a CPE) ?

Regards,

-- 
Jérôme Benoit aka fraggle
La Météo du Net - http://grenouille.com
OpenPGP Key ID : 9FE9161D
Key fingerprint : 9CA4 0249 AF57 A35B 34B3 AC15 FAA0 CB50 9FE9 161D

---

From: Al Morton <[email-masked]>
Date: Tue, 06 Jul 2010 08:34:21 -0400

Jerome,

Thanks for your comments, see below.

Al

>Do you plan do add inaccuracy introduced by runtime on each

Sorry, but it's not clear to me where your comment applies.

Each of the metric RFCs cover error sources and error reporting,
so (if I understand your question) this is one of the areas to
check and see if implementing developers understood the requirements.

>when you want to send probes from the network edge et behind a CPE) ?
Yes, this draft starts from the premise that measurement results from
different implementations will be compared. Neither OWAMP or TWAMP
are required.

---

From: Jerome Benoit <[email-masked]>
Date: Tue, 6 Jul 2010 19:51:00 +0200

Le Tue, 06 Jul 2010 08:34:21 -0400,

> Sorry, but it's not clear to me where your comment applies.

I'm not a math guru but I wanted to know if the proposed draft will
take in account inaccuracy on timestamping that come only from the
runtime. For example, we have some Proof of Concept code (probecraft :
http://github.com/lucasdicioccio/probecraft ) that is able to send most
of the known active probes, but we have chosen on the implementation
side to rely only on lipcap and its timestamping capability on packets :
the wire-time is what lipcap give us. It's a design choice. If the
kernel has a lot of packets to treat, the desired time series for
some sort of probe will not be followed as requested. If lipcap is able
to use some kernel facilities to bypass buffering or routing table,
we can minimize runtime inaccuracy.  

There is other bias that can be introduced on the probing src host that
can invalid the probe (ARP cache, etc.). The probing src host can be
very noisy ... 

> check and see if implementing developers understood the requirements.

Oki. 
 
> different implementations will be compared. Neither OWAMP or TWAMP

Great. 

Thks.