Subject: [ippm] Draft IPPM Minutes

From: Merike Kaeo <[email-masked]>
Date: Fri, 07 Sep 2001 13:39:38 -0700

Below is a final draft of the meeting minutes from the last IETF meeting 
.  The final is due on Monday.  Sorry for late distribution on the 
list......however, if anyone has anything to add or any modifications, 
please let myself and Matt know by Monday morning.  Thanks.

-merike

Minutes of the IP Performance Metrics WG (ippm)
Tuesday, 07 August 2001, 13:00 - 14:00
===================================================

The meeting was chaired by the working group chairs, Matt Zekauskas and
Merike Kaeo.  Reported by James Stevens and Paul Love; these minutes
were edited by the chairs.

AGENDA

1. Agenda bashing
2. Small case-study on advancing metrics through the IETF standards process;
      comparing RIPE-NCC and Surveyor data as per the internet-draft.
3. One-way Active Measurements Protcol Requirements
4. Measuring Packet Reordering
5. Potential new work items
     Sampling Methodology
     MIB for IPPM

IETF home page: http://www.ietf.org/html.charters/ippm-charter.html
IPPM home page: http://www.advanced.org/IPPM/

2. Comparing Metrics based on draft-bradner-metricstest-00.txt
      --Matt Zekauskas

    Matt presented a small case-study on advancing metrics through
    the IETF standards process, comparing RIPE-NCC and Surveyor data as
    per the I-D.  He noted some issues: choosing which values you used for
    the mean & standard deviation, the fact that you might end up with a
    standard deviation that is less than the calibration error, that you
    need to be explicit about which values are included, or not included
    (for example, holes due to GPS loss), and that comparing
    singleton measurements for streams of delay values is not well
    defined.

    Questions were raised on how were intervals in which no data from
    metrics was recorded dealt with. This had the potential to make the
    data set recorded appear in statistical terms better than it actually
    was. This was highlighted as a measurement implementation issue
    e.g. GPS fading due to rain, which could coincide with network
    degradation on a microwave link.

3. One-way Active Measurements Protocol Requirements
      --Merike Kaeo

    Although we received no comments on the requirements draft, several
    people in the meeting indicated that they did read the draft.

    There was some discussion on post-processing of data, and whether
    the requirements draft should forbid it.  The current draft appears
    to forbid it, but several commented that post-processing, at least in
    some sense, is required to detect loss and errors.  Section 4.1
    of the draft is ambiguous -- does it forbid it or require it?
    Matt pointed out that his understanding of the authors intent
    was that significant post-processing should not be required
    (but be allowed).

    Merike requested that those reviewing the drafts provide sample
    text for revisions rather than just pointing out that revisions are
    needed.

    The scalability and security of protocol were discussed . Comments
    were solicited, and Merike noted that security aspects would
    need to be reviewed by relevant IETF experts.

    The types of measurement machines that could be controlled can
    vary in their capability (for example, precision).  There should
    be some ability for the control node to know the capabilities.
    There was brief discussion to put this in general terms in the
    requirements, and work out details in the protocol.

    A participant noted that there was no need to restrict the
    measurements to be one-way delay, or even one-way.  Merike
    noted that was why the name was proposed to be changed
    from one-way delay protocol requirements to one-way active
    measurement requirements.

    There was some discussion of additional feature reporting that
    should be added.  The following were mentioned:

    - Any known time-varying aspect of metrics or the implementation
      should be reported at the outset.
    - The ability to have a maximum number of measurement packets
      or sequence numbers should be provided.
    - The ToS/DSCP should be reported.

    It was also noted that in some cases the requirements document
    switched to implementation instead of requirements, for example,
    specifiying UDP is not approprate; it is one solution, not a
    requirement.

    Another participant noted that it would be useful to specify that
    parameters (say of a Poisson stream) should change over time,
    for example, set changes based on diurnal or weekly patterns
    in traffic streams.  Right now, you must start a new session.

    There was also a comment that there should be some way for vendors
    to include optional vendor-specific information.

    Another participant asked about the timeframe for progressing
    this document; the hope is that the authors can get a revision
    out in the next month.  The requirements document should be
    finalized before work on OWDP can officially continue.

    Those that commented should send notes and comments to the list
    to provoke more discussion and allow us to reach consensus.

    The working group had unanimous consensus that the name for the OWDP
    should be made more generic (so as to also include metrics for IPDV,
    Loss, etc).  Proposed was the One-Way Active Measurement Protocol.

    Matt briefly presented a slide from Stanislav Shalunov asking if
    the new send schedule proposed for OWDP was sufficient for
    the requirements.  A pointer to the slide will be posted to
    the mailing list, and working group members were requested to
    comment on it.

4. The Packet Reordering Problem
     -- Dale Pullin, California Institute of Technology

    Dale Pullin presented his work on a metric for  packet reordering
    that would result in a single number representing the "orderedness".
    He focussed on a using a value known as "Minimum Longest Ascending
    Sequence"  (MLAS).  A website (http://www.tiac.net/users/cri/mlas.html)
    contains the basic algorithm.  However, Dale noted that the
    sample code on the website contains some errors.
    His metric, omega, is the length of the MLAS, divided by
    the number of elements.  It can be computed in NlogN time in
    O(N) space.

    Matt Mathis noted that for sequences which are nearly always in order,
    the MLAS algorithm doesn't seem to characterize the measure of the number
    of packets not in the expected location.  He also wondered that in a
    practical sense, was an  NlogN algorithm really needed?

    Comment by Henk that MLAS algorithm represented a measure of
    orderness, not distance between out-of-order packets, and wondered if
    this showed anything interesting about real network behavior.  For
    example, if a sequence has 10 elements, and the first two elements are
    flipped, omega is .9; If you flip the first and last elements, omega is
    still .9.   MLAS provides fraction of packets that are out of sequence,
    but does not state how far out they are.  Dale thought this could be
    determined if needed.

    Clarification was asked for what was meant by the "terminal element":
    it is the last element.

    There was a question regarding similarity of this algorithm and the
    results it provides with other sort based algorithms.  In those
    algorithms the measure is number of sequential inversions (SI).  Is
    this different from that, or just a different way to come up the same
    thing.  Which is better?  SI can be done in linear time.
    Has thought been given to the relative value of information provided?

    Matt Mathis commented that the number of backward steps might be
    sufficient, i.e. that the simple number of swaps metric might be better
    suited for TCP (detecting when TCP will have major problems).  He also
    pointed out that he thought that the presented algorithm would see more
    reordering than one that just did backward swaps.

    Dale will check this and follow up, but verified that the MLAS
    algorithm was different.  Dale also commented that he investigated
    some faster algorithms, but found that these were not always
    representative.

    Someone asked if the algorithm could be used incrementally, rather
    than operating on an entire sequence at once.  Dale confirmed that
    MLAS algorithm could take updates on packet arrivals for use in
    "real-time".

    The group felt some out-of-order metric (packet reordering metric) was
    needed, but that this particular metric would require some
    relationship to other network phenomena (like bulk transport capacity)
    to be really useful, before it would be taken up as a WG document.

5. Other potential new work

    Matt Z. briefly presented an outline slide on Sampling Methodology
    work by Mihailo Zilovic and Lubomir Citkusev.  They have performed
    some simulations to see what parameters might be suitable for
    detecting the underlying network traffic.  The initial observation
    is that the sample size required depends on the traffic type.
    There will be a post to the mailing list; please comment on it.

    Finally, there was a personal submission on developing a MIB for the
    IPPM metrics.  Emile Stephan wanted to present a personal draft on a
    MIB for IPPM.  An outline and introduction was given, but time
    constraints prevented full discussion.  The chairs requested that
    consideration of whether this belongs in the IPPM WG needs to be
    addressed (including the relationship to work in RMONMIB and maybe
    other ops MIB activity) and that comments to mailing list are required
    on acceptance for this work

_______________________________________________
ippm mailing list
[email-masked]
http://mailhost.advanced.org/mailman/listinfo/ippm

---

From: Dan Romascanu <[email-masked]>
Date: Sat, 8 Sep 2001 16:25:29 +0300

on it.
>     on acceptance for this work
I think it would be useful if Emile can send to the list the argument had he
intended to make in the London meeting. There was very little time in the
meeting, and the idea of an IPPM MIB, and the rationale of such a work
relative to the IPPM and RMON MIB work may be worth being brought to the
list.

Regards,

Dan

---

From: STEPHAN Emile FTRD/DAC/LAN <[email-masked]>
Date: Tue, 11 Sep 2001 09:58:08 +0200

Hi all,
 
I propose to comment each slide in a mail. Then we may have a discussion on
the list.
does that fit ?
 
regards
emile

-----Message d'origine-----
De : Dan Romascanu [mailto:[email-masked]]
Envoye : samedi 8 septembre 2001 15:25
A : Merike Kaeo; [email-masked]
Cc : Matt Zekauskas
Objet : RE: [ippm] Draft IPPM Minutes

on it. 

Regards, 

Dan