Subject: Re: Qos Benchmarking Proposal

From: Kevin Dubray <[email-masked]>
Date: Thu, 10 Feb 2000 15:05:29 -0500

Regarding the QoS benchmarking proposal, 

1) It would be important to know how ITU I.380 maps to the proposed work.  Any
ideas?  (Last I knew, I.380 was closer to the IPPM's purview...) 
It wouldn't be productive to replicate work needlessly.

2) The phrase:

"The terminology document also will define QoS concepts and parameters that 
will be used in the methodology document."

appears to be open-ended.  What concepts and parameters do you anticipate
defining?  (The devil is always in the details... ;-)

-Kevin

---

From: "Cynthia E. Martin" <[email-masked]>
Date: Sun, 27 Feb 2000 19:37:09 -0500

All,

I have a problem with Layer 3 QOS benchmarking.  I haven't read any IETF RFC's
lately that describe Layer 3 QOS guarantees.  If someone wants to address QOS, why
not IP over ATM QOS?  TM 4.0 is very clear and it is straight forward to define
metrics based on it.  How about IP over FR based on BICI and FRF.13?  That is also
very clear.  If people want to work on the document, narrow the scope or we will
all be working on this document until we retire.

Regards,
Cynthia

---

From: "David Newman" <[email-masked]>
Date: Sun, 27 Feb 2000 21:40:35 -0500

> all be working on this document until we retire.

*Precisely* to avoid this sort of mission creep, this current proposal is
limited in scope to just two mechansims, both in the IP header -- IP
precedence and DSCPs. The use of both mechanisms is discussed in RFC 2474.

There is no suggestion made, explicitly or implicitly, in the current
proposal to attempt anything other than the definition and use of
performance benchmarks for devices that implement either of these two
mechanisms.

David Newman

---

From: Jeffrey Dunn <[email-masked]>
Date: Sun, 27 Feb 2000 23:13:59 -0500

David,

I think that what is at issue here, with respect to RFC 2474 is the
following sentence in the pre-amble:

"Our focus is on the general semantics of the behaviors rather than the
specific mechanisms used to implement them since these behaviors will
evolve less rapidly than the mechanisms."

Semantics are wonderful building blocks; however, they cannot be
expected to take the place of hard metrics.  This view is further
strengthened by the following excerpt from RFC 2474:

"The configuration of network elements with respect to which packets get
special treatment and what kinds of rules are to be applied to the use
of resources is much less well-understood."

This, I believe, is essentially Cynthia's point.  We have hard and fast
rules for such services as ATM, FR and ISDN, but very little real
enforceable policy for IP over foo.  As a result, to attempt to
characterize the conformance of IP based QoS to a purely IP based QoS
contract is a Sisyphean labor.  I would like to include one final RFC
quote, which I believe bears out my point.  In the immortal words of Jon
Postel regarding IPv4:

"There are no mechanisms to augment end-to-end data reliability, flow
control, sequencing, or other services commonly found in host-to-host
protocols."

Regards,

Jeff

>David Newman wrote:

---

From: "David Newman" <[email-masked]>
Date: Mon, 28 Feb 2000 08:25:22 -0500

> expected to take the place of hard metrics.

You're tarring us with the wrong brush. That is the view of the authors of
RFC 2474. It is not our view. We propose to define hard metrics.

BTW, I cited RFC 2474 because Cynthia said she hadn't "read any IETF RFC's
lately that describe Layer 3 QOS guarantees." There has been quite a lot of
discussion since at least the Munich IETF about the two L3 mechanisms
discussed in RFC 2474.

> I would like to include one final RFC

Please reread our proposal. We do not attempt end-to-end anything. Indeed,
any such attempt would be out of scope for the bmwg.

dn

---

From: Scott Poretsky <[email-masked]>
Date: Mon, 28 Feb 2000 09:05:11 -0500

>evolve less rapidly than the mechanisms."

This is exactly our point.  RFC 2474 says here is how it works, but it 
provides no guidelines for the comparison of implementations.  However, it 
shouldn't because that is the responsibility of bmwg.

>of resources is much less well-understood."

The proposed work should make it more understood.

>contract is a Sisyphean labor.

You are right that this is a very challenging task.  However, it is 
becoming increasingly necessary because routers implement QoS mechanisms 
such as WRED and WFQ with separate parameter sets assigned to IP Precedence 
Levels and DSCPs.  The current shortcoming for the Network Provider is lack 
of common test scenarios and methodologies to compare the various QoS 
implementations.

>control, sequencing, or other services commonly found in host-to-host

I agree with Mr. Postel's statement.  However, in a *congested* network we 
need more than just TCP.  Sally Floyd's published RED work in 1993 proved this.

---

From: Geoff Huston <[email-masked]>
Date: Tue, 29 Feb 2000 10:51:08 +1100

I keep on seeing a diverse set of motivations for this work, and
they are difficult to reconcile:....

"Network Operators need bench top techniques and metrics to assess a vendor's
  implementation"

"Network Operators need methodologies to probe operating devices to
  work out if the device is performing the way it is claimed to."

"Network Operators need methodologies to probe operating devices to
  work out if their configuration is working as intended."

"Network Operators need methodologies to probe operating devices to
  work out if the device is performing correctly within the parameters
  of existing load."

"Network Operators need border-to-border tools to probe network paths
  to work out if they are meeting their QoS commitments."

"Network Operators need any form of tool to add some weight to their
  QoS marketing hyperbole."

"Network vendors need any form of tool to add some weight to their
  QoS marketing hyperbole."

"Network service customers need methodologies to probe operating
  networks to see if a claimed service metric is being delivered."

and so on.

Is anyone willing to commit, on one sentence what the intended motivation
is for this activity? It would be really helpful!

Geoff

---

From: "David Newman" <[email-masked]>
Date: Tue, 29 Feb 2000 08:57:13 -0500

> is for this activity?

Yes. If vendors X, Y, and Z all claim their boxes classify traffic based on
IP precedence or DSCP criteria, how can I verify those mechanisms actually
do what the vendors' sales droids tell me they do?

That is the motivation of this effort. Many of the other quotes you cited
raise interesting questions, but they do not reflect representations we have
made.

David Newman