Subject: [ippm] draft-ietf-ippm-reporting-01.txt

From: stanislav shalunov <[email-masked]>
Date: 08 Nov 2006 21:56:35 -0500

As mentioned during the IPPM meeting, we met today to discuss the
reporting draft.  Al, Emile, Henk, and I came to the meeting.  The
following issues were identified with the draft (mostly Al's notes on
his reading copy of the draft).  Please comment on the issues where
you have an opinion.

1. Should lost packets be included in the calculation of median delay
   (as currently in the draft, with infinite delay), or omitted?

   For including infinite delay: this is specified as the only option
   by section 5.1 of RFC 2679.

   For computing percentiles on received packets only: Al, please
   clarify.

2. Should the name for ``delay'' be more descriptive and say ``median
   delay''?

   For longer version: more descriptive.

   For shorter version: more concise, no statistical terms not
   meaningful to many end-users (thus less confusing).

3. Should name for loss similarly be ``loss ratio'' (or fraction or
   whatnot)?

   For longer version: more descriptive.

   For shorter version: ``ratio'' or ``fraction'' is clearly indicated
   by reporting percentage.

4. Should jitter be called ``delay spread'' instead?

5. ITU draft definition of duplication currently differs from what is
   in the draft.  The IPPM draft has it so that it can't exceed 100%.
   ITU draft allows two options (which differ in the treatment of lost
   packets), with both allowing the measure to exceed 100%.  Henk will
   write a separate IPPM document defining duplication harmonizing
   with ITU.  If there's difficulty, Stas is OK with dropping the
   metric altogether.

6. Reordering definition is not as in the reordering RFC.

7. Round-trip metrics should say how the packets need to be echoed.

8. The middle of the reporting interval is not something that's
   mentioned as the time of the measurement anywhere else (where
   beginning and end are used).  Will change to the end.  Rationale:
   With the end as the time of the measurement, a measurement just
   concluded measures the network as it is now.

9. The draft specified sliding window in passive measurement.  Is this
   right?  Use case: VoIP phone displaying statistics.  Should it
   update continuously or once every 10 seconds?

   For continuous update: most timely statistics, more useful in cases
   like sudden silence (``did the connection just drop or get
   worse?'').

   For discrete updates: the rate of updates contains information.
   The user will easily infer that the measurements aggregate since
   last update.

10. The second paragraph of section 3 (``Scope'') should have a
    subtitle.

11. Mention the difficulty of clock synchronization for one-way delay
    measurements?

-- 
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

Heather has two mommies.  That's nothing.  The Internet has at least
two dozen daddies.

_______________________________________________
ippm mailing list
[email-masked] 
https://www1.ietf.org/mailman/listinfo/ippm

---

From: Henk Uijterwaal <[email-masked]>
Date: Thu, 09 Nov 2006 21:07:32 +0100

Hi Stas, others,

(Personal opinions, chair hat off).

>    (as currently in the draft, with infinite delay), or omitted?

One goal is to make the quantities orthogonal, including lost packets
in the delay metric accomplishes the opposite.  Removing them says
something like "10 packets were sent, 2 were lost, of the 8 that arrived,
the median delay was X".

> 2. Should the name for ``delay'' be more descriptive and say ``median

I think so.  This is what we report, we don't report "average delay"
or "delay of the first packet".

> 3. Should name for loss similarly be ``loss ratio'' (or fraction or

Yes, though here I think the % makes it obvious what is reported.

> 4. Should jitter be called ``delay spread'' instead?

Yes, this is what you are reporting.

>    with ITU.  If there's difficulty, Stas is OK with dropping the

I just went through the ITU document and couldn't find a metric there.
For the IPPM draft, I'm thinking about #duplicates/#sent (or
#received/#sent-1).  This can go over 100% (say if each packet sent
results in 3 copies arriving, the metric will be 200%) but I think
that is OK.  Nobody has problems understanding something like "traffic
increased with 200% in the last month".

Henk

------------------------------------------------------------------------------
Henk Uijterwaal                           Email: henk.uijterwaal(at)ripe.net
RIPE Network Coordination Centre          http://www.amsterdamned.org/~henk
P.O.Box 10096          Singel 258         Phone: +31.20.5354414
1001 EB Amsterdam      1016 AB Amsterdam  Fax: +31.20.5354445
The Netherlands        The Netherlands    Mobile: +31.6.55861746
------------------------------------------------------------------------------

I can't believe grown engineers are afraid of a dot.  (Ed Lewis)

---

From: Al Morton <[email-masked]>
Date: Thu, 09 Nov 2006 15:16:42 -0500

>    by section 5.1 of RFC 2679.

Not quite an option, it is an informal representation of "undefined".
The singleton definition says the Delay of packets that
don't arrive is undefined, and I propose to keep them that way.

from RFC 2679:

3.4. Definition:
...
    >>The *Type-P-One-way-Delay* from Src to Dst at T is undefined
    (informally, infinite)<< means that Src sent the first bit of a
    Type-P packet to Dst at wire-time T and that Dst did not receive that
    packet.
...
Some of the text in Section 5.1 went with the informal representation,
some did not.
...
5.1. Type-P-One-way-Delay-Percentile

    Given a Type-P-One-way-Delay-Poisson-Stream and a percent X between
    0% and 100%, the Xth percentile of all the dT values in the Stream.
    In computing this percentile, undefined values are treated as
    infinitely large.  Note that this means that the percentile could
    thus be undefined (informally, infinite).  In addition, the Type-P-
    One-way-Delay-Percentile is undefined if the sample is empty.

>    For computing percentiles on received packets only: Al, please

Compute on the delay distribution conditioned on packet arrival within
the timeout, and then the delay metric will be "as orthogonal as possible"
to loss.

Al

---

From: Al Morton <[email-masked]>
Date: Thu, 09 Nov 2006 22:49:53 -0500

At 03:07 PM 11/9/2006, Henk Uijterwaal wrote:
>increased with 200% in the last month".

Right, that was the draft from the meeting last June.
I've done some work since then, but I have yet to start
editing the new draft (one meeting at a time!)

The structure of the draft is to first define the
primary outcomes, then parameters (metrics) based on those outcomes.

>IIRC, you said something like: duplicates/packets_sent
>yesterday.  In other words, if one packet sent results in 2 packets
>received, it will be 1/1=100%.  If one packet results in 3 packets
>received, it will be 2/1=200%.  Is this correct, is it in the doc?
>(If not, I'll invent a metric for the IPPM doc).

I had two metrics on paper yesterday, the other is:
duplicates /non-duplicate packets received.

I think this may be better because it does not mix loss and
duplication in the denominator by using packets_sent.

So, if one packet sent results in 2 packets of the same
received, it will be 1/1=100%.  If one packet results in 3 packets
received, it will be 2/1=200%.  If two packets are sent, 1 is lost
and 3 copies of the other are received, it will be 2/1=200%.

Al

---

From: "Joseph Kopena" <[email-masked]>
Date: Fri, 10 Nov 2006 00:20:29 -0500

> Yes, this is what you are reporting.

One small comment is that if you show users "delay spread" on a
readout, there's probably going to be some number of them that will
take it as the range, the difference from maximum to minimum, rather
than the interquartile.

-- 
- joe kopena

---

From: Al Morton <[email-masked]>
Date: Fri, 10 Nov 2006 14:08:16 -0500

>take it as the range, the difference from maximum to minimum, rather

So, it may be clearer to say "Inter-Quartile Delay Spread".
At least that relives ambiguity w.r.t. the range, and I want
to expand these names to be sufficiently descriptive.

Al

---

From: Henk Uijterwaal <[email-masked]>
Date: Mon, 13 Nov 2006 07:51:33 +0100

> to expand these names to be sufficiently descriptive.

While this is technically the correct name, I think it misses the
point of being understandable by non-experts.

Henk

---

From: Al Morton <[email-masked]>
Date: Mon, 13 Nov 2006 08:57:24 -0500

At 01:51 AM 11/13/2006, Henk Uijterwaal wrote:
>>>>Stas wrote:

If so, maybe "75 percentile - 25 percentile Delay Spread"
(possibly using the shorthand, %-ile) would be clear to everyone.

Al