Subject: Re: Last Call : L3 QoS Benchmarking Workplan ???

From: "John Dawson" <[email-masked]>
Date: Fri, 25 Feb 2000 16:09:53 -0800

Kevin,

I didn't understand this was a vote required issue.  I would agree that the
effort is needed.  I also tried to post a concern to the reflector but it
doesn't appear to have made it (see below).

John
=========
I did not see any placeholder for throughput related tests in the proposal.
 One thing that really makes me nervous about QOS is the expectations with
respect to packet loss rate.  Although forwarding rate tests should be able
to characterize how many packets per second a system can handle, I don't
subscribe to the concept that all protocols will run efficiently even in
the presence of high packet loss rates.

>From a QOS standpoint, I would like to know that I can reserve a
high-priority channel and get a certain amount of data through that channel
without loss (even though the rest of the network is under heavy load).

If we do not include any zero loss rate type tests, is the expectation that
all applications built on top of the proposed QOS model will function
equally well in the presence of arbritary loss?  

Will this effort try to put any hard numbers behind what can be considered
acceptable levels of jitter, delay, and loss for a defined "level of
service" (i.e. graded channel capacities)? 

or is QOS just another way of marketing relative (non-guaranteed) levels of
service that change drastically depending on the current system load?

---

From: "David Newman" <[email-masked]>
Date: Sat, 26 Feb 2000 22:50:04 -0500

Hi John,

> equally well in the presence of arbritary loss?

Strongly agree that zero-loss tests should be included once we get to the
methodology.

I may be to blame for some confusion on this point. As noted in section 3 of
the proposal, the four main metrics we plan to define are forwarding rate,
packet loss, latency, and jitter. Scott originally suggested throughput (per
RFC 1242) rather than forwarding rate (per RFC 2285) but I objected on
grounds opposite to yours -- that a throughput metric wouldn't tell us
anything about forwarding rates once packet loss begins.

The gist of my argument for using forwarding rate was that 1) we were
already measuring packet loss and 2) we'd *gain* thhe ability to measure
under lossy scenarios and we wouldn't *lose* any information we might have
gleaned had we used a throughput metric instead.

That said, I would be glad to consider arguments for using a throughput
metric instead of or in addition to forwarding rate.

> service" (i.e. graded channel capacities)?

I am allergic to labeling any measurement a "good" or "bad" one. The goal of
this effort is to define performance benchmarks, not to place judgments on
the acceptability (or otherwise) of those benchmarks.

> service that change drastically depending on the current system load?

Four comments here:

--even deterministic schemes like ATM have some non-guaranteed service
levels
--we all do step testing today to determine device performance at various
load levels
--this effort seeks to define various benchmarks, and I performance
(tput/fwding rate/loss/latency/jitter) under various load levels to be one
of those benchmarks
--determining whether a device performance corresponds to predefined
IP-based service parameters is something for engineers -- not marketeers --
to determine. That's why we're doing this

Thanks again your feedback.

David Newman

.