Subject: Proposef RFC 2680 : A One-way Packet Loss Metric for IPPM

From: KERBERENES Jacques CNET/DAC/LAN <[email-masked]>
Date: Wed, 29 Sep 1999 14:28:25 +0200

Hi everybody,

For our internal needs I am currently writing a paper about IP metrics, and
a companion paper that describes some trials.
My concern is about RFC2680 §4 'Some Statistics  for One-way Packet Loss'.
The only statistic  proposed is 'Type-P-one-way-Packet-Loss-Average . 
1)It is noted that this statistic will be meaningless.
2) My understanding of the Surveyor and RIPE-NCC projects that have
implemented this RFC , is that the statistics used in the reports are Median
and Percentiles instead of Average.Am I wrong?
So my proposal is to add these statistics inside §4 of RFC2680.

Best regards 
Jacques

Jacques Kerberenes

e-mail : [email-masked]

Phone : +33  (0)2 96 05 29 13
Fax :      +33  (0)2 96 05 18 52

CNET/DAC/NTR
2, av Pierre Marzin
BP 40 22301 LANNION CEDEX
FRANCE

---

From: [email-masked]
Date: Wed, 29 Sep 1999 16:19:07 +0200

> For our internal needs I am currently writing a paper about IP
> metrics, and a companion paper that describes some trials.  My
> concern is about RFC2680 §4 'Some Statistics for One-way
> Packet Loss'.  The only statistic proposed is 'Type-P-one-way-
> Packet-Loss-Average.'
>
> 1) It is noted that this statistic will be meaningless.

Oh?  Where?  Can you please quote chapter and verse?

Are you certain that you are not mixing up the properties of
"average one-way delay" with "average packet loss"?  The problem
with the first metric is that since some samples are "undefined"
(lost packets), it's in general meaningless to calculate an
average over all the samples (how do you handle the lost
pakcets?).

However, for the samples you calculate 'Type-P-one-way-Packet-
Loss-Average' over, the packets either made it or they didn't --
there's no "undefined" value in the set of samples you use to
calculate this metric.

> 2) My understanding of the Surveyor and RIPE-NCC projects that
> have implemented this RFC, is that the statistics used in the
> reports are Median and Percentiles instead of Average.  Am I
> wrong?  So my proposal is to add these statistics inside §4 of
> RFC2680.

Median and percentiles for delay, yes, but I still think they
calculate average packet loss over a measurement interval.  I
don't think RFC2680 needs amendment in this area.

- Håvard

(...who normally doesn't write much on this list, and hopes he
got this right.  I'll go back to lurking now. :-)

---

From: "Henk Uijterwaal (RIPE-NCC)" <[email-masked]>
Date: Wed, 29 Sep 1999 16:26:44 +0200 (CEST)

Hi Jacques,

> 1)It is noted that this statistic will be meaningless.

No, there are cases where this statistic won't work, but in general it can
be useful.  

The case mentioned in the RFC deals with measuring losses at the 1% level
over short time intervals.  To measure these, one has to send a large
number of packets in a short time.  But, if one sends lots of packets in a
short interval, there is the risk that the measurement devices get
overloaded and start to miss packets.  There is no way to distinguish this
from losses in the Internet, so at that point, the measurement becomes
meaningless.  

If, OTOH, one limits the number of packets sent to something sensible(*),
then the average is a useful quantity.

> and Percentiles instead of Average.Am I wrong?

For the RIPE-NCC project, this is correct.  

Note, however, that median will also become useless in the limit of short
time intervals or large number of packets.

Henk

(*) At the Oslo IETF, we agreed that a BCP to define "sensible" would be
useful.  I even promised to spend some cycles on this in September 1999. 
I guess I still have a day :-) 

------------------------------------------------------------------------------
Henk Uijterwaal                    Email: [email-masked]
RIPE Network Coordination Centre     WWW: http://www.ripe.net/home/henk
Singel 258                         Phone: +31.20.535-4414,  Fax -4445
1016 AB Amsterdam                   Home: +31.20.4195305
The Netherlands                   Mobile: +31.6.55861746  
------------------------------------------------------------------------------

The Committee (...) was unable to reach a consensus that substantial merit was
lacking. Thus, the appeal was deemed meritorious.          (Orlando NABC #19).

---

From: [email-masked] (Sunil Kalidindi)
Date: Wed, 29 Sep 1999 10:42:40 -0400 (EDT)

Hi,

Replies below.

> The only statistic  proposed is 'Type-P-one-way-Packet-Loss-Average . 

The statistic Type-P-One-way-Packet-Loss-Average is only illustrative.

---	RFC 2680 ---
...
4. Some Statistics Definitions for One-way Packet Loss

   Given the sample metric Type-P-One-way-Packet-Loss-Poisson-Stream, we
   now offer several statistics of that sample.  These statistics are
   offered mostly to be illustrative of what could be done.

...
-----

There could be other useful statistics too.

> 1)It is noted that this statistic will be meaningless.

This statistic is NOT meanigless. The RFC is merely saying that if the error
bars are too high the statistic will not be meaningful. I suspect that this
sentence was misunderstood.

> and Percentiles instead of Average.Am I wrong?

I believe you are referring to the delay statistics - not the loss stats.
Both the Surveyor and the RIPE projects have used the percentiles of delay 
statistics instead of averages. Those stats are defined in RFC2679.

> So my proposal is to add these statistics inside §4 of RFC2680.

Thanks,
Sunil

-- 
------------------------------------------------------------------------------
S u n i l  K a l i d i n d i				[email-masked]
S u r v e y  o  r  P r o j e ct				v o i c e: 914-765-1128
A d v a n c e d  N e t w o r k & S v c s, I n c.	F a x:   914-765-1129
------------------------------------------------------------------------------