Subject: Re: Last Call : L3 QoS Benchmarking Workplan ???

From: Scott Bradner <[email-masked]>
Date: Fri, 25 Feb 2000 16:20:41 -0500 (EST)

I aslo agree that it would be quite good if there were some
standardized ways to look at testing in this area

Scott

---

From: "Steve Bell" <[email-masked]>
Date: Fri, 25 Feb 2000 16:15:32 -0800

Agilent would also support the effort, definitely needed.

Steve Bell
The Silicon Valley Networking Lab (SVNL)
     of Agilent Technologies
2681-A Zanker Rd.
San Jose, CA 95134
(408)432-3840 x111
www.svnl.com

                  S V N L  of  A G I L E N T
     T H E    I N D U S T R Y ' S    T E S T    C E N T E R

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Scott Bradner
Sent: Friday, February 25, 2000 1:21 PM
To: [email-masked]; [email-masked]
Subject: Re: Last Call : L3 QoS Benchmarking Workplan ???

Scott

---

From: Randy Bush <[email-masked]>
Date: Fri, 25 Feb 2000 19:09:08 -0800

> Agilent would also support the effort, definitely needed.

that counts as 10,000 votes against.  the ietf is an organization of
individuals.  

randy

---

From: "Steve Bell" <[email-masked]>
Date: Sat, 26 Feb 2000 10:59:49 -0800

>> that counts as 10,000 votes against.  

darn... Kevin, looks like you are going to have to work
much harder to find the 9,997 votes to overcome this setback! :)

Steve Bell / SVNL of Agilent

---

From: "David Newman" <[email-masked]>
Date: Sat, 26 Feb 2000 15:23:19 -0500

> that counts as 10,000 votes against.  the ietf is an organization of

In ddc's immortal phrase, the IETF is also an organization that rejects
kings, presidents, and voting. Your 10,000 votes are hereby nullified. ;-)

While ddc rejected voting he affirmed the need for rough consensus. As was
dicussed briefly in Washington, and has been made amply clear by individuals
posting to this list in the past 12 hours, the consensus thus far favors
proceeding with, not killing, the QOS workplan.

David Newman

---

From: Geoff Huston <[email-masked]>
Date: Sun, 27 Feb 2000 21:27:10 +1100

>standardized ways to look at testing in this area

Can I cut from the testamonials of 'doing this would be great'
and the 'here's my technical input the the working group's work'
to respond to Scott's message and look at what this is really
proposing...

The issue here is that there are a wide variety of perceptions on
what should be measured here, who should be undertaking the
measurements (customer, provider or third party), and why such
a measurement should be done - and thats all a precursor to
figuring out if the collected data has any damn relevance to
delivered QoS in any case!

This is not a simple network element MIB definition - or if someone
thinks it is, then I'd offer the response that such a MIB-oriented
view of QoS is an attempt to describe a beach by looking at the
state of each and every grain of sand.

Its also not simply an end-to-end problem where the sender receives
QoS reports from receiver(s) - while such an approach may inform a
source how well a signal is being received it is only half the story.
In theory the report should be able to identify the receiver's
quality relative to whatever quality vector is being exploited by
the source.

Also, it would be reasonable to expect that a QoS benchmark metric
should be able to be used in a 'what if' scenario, attempting to
take a broader view of a network's ability to sustain a certain
profile of traffic load within a set of associated performance
quality parameters.

Now, we know a lot a about network element state measurement.

Thanks to the careful IPPM work we now know a damn sight more than we
did about methodology measurement of the basic network per-path
behaviours of one way delay, one way delay variation and one way
packet loss.

But QoS benchmarking is a larger and significantly more challenging
topic in my view.  I'm not sure whether the QoS metric issue is an
IETF or IRTF topic. It still feels to me that it has strong research
inputs as the task is not all that well defined as I see it, and the
approaches that we will need to use to provide QoS metrics are
still very much up in the air. It feels to me that this will straddle
both areas, and that will require some considerable care in managing the
efforts of a working group to produce high quality outputs.

regards,

   Geoff Huston

---

From: "David Newman" <[email-masked]>
Date: Sun, 27 Feb 2000 12:25:43 -0500

> measurements (customer, provider or third party)

Before we can answer either of these questions, we need consensus on what it
is we need to measure in the first place. Defining some of those benchmarks
is the goal of the current proposal.

> This is not a simple network element MIB definition

There's nothing in the current workplan about MIB definitions, and I'd
strongly recommend we *not* get into that.

- or if someone
> view of QoS is an attempt to describe  beach by looking at the

You need only to suffer through a botched software upgrade or a severe route
flap to recognize the grain-of-sand-vs.-entire-beach analogy is less than
perfect. At least in the IP world, the behavior of one node can have a very
significant impact on performance of even the largest network.

It is correct to state that the focus here is on devices rather than
networks -- as is all work done within the bmwg.

> profile of traffic load within a set of associated performance

I agree. This is essentially a micro-vs.-macro argument. There's no right
answer to which approach is more correct. Both are necessary.

As noted, the bmwg's charter is to define device benchmarks. Network/service
measurements are out of scope for our proposal.

> IETF or IRTF topic.

That's interesting. One concern I'd raise is that unlike the current
proposal future QoS metrics may involve criteria above the network layer
(just hypothetically, imagine a QOS mechanism that uses port numbers or
application header data to classify flows). In such cases, I wonder whether
IRTF input would be meaningful.

> I still feels to me that it has strong research

Ah, but which WG? Adopting the "broader view" you advocate certainly would
produce interesting and useful measurements. But it carries the danger of
ignoring the "grains of sand." Given the ability of one IP element to
dramatically affect even the largest network as a whole, it does seem
reasonable to be able to describe the QOS characteristics on a per-element
as well as a per-network basis. And that's what we're attempting to do.

BTW, I agree fully that 1) network-level measurements are good and necessary
and 2) there should be close coordination among bmwg, ippm, and whatever
other groups may be involved.

Until someone puts forth some metrics, though, it's unclear what these
groups would have to talk about.

David Newman