Subject: RE: Workplan Proposal: Benchmarking Differentiated Services Mecha nism - Version 2

From: "John Tinsley" <[email-masked]>
Date: Tue, 5 Sep 2000 15:24:27 -0700

Jerry,
 
Thanks for working on this proposal.  The industry desperately needs to have
a common set of benchmarks for DiffServ devices.  This technology will
become more and more important as time goes by.  These RFC defined and
sanctioned benchmarks will aid customers as well as developers of said
devices.  
 
John

-----Original Message-----
From: Perser, Jerry [mailto:[email-masked]]
Sent: Friday, August 25, 2000 11:43 AM
To: BMWG (E-mail)
Subject: Workplan Proposal: Benchmarking Differentiated Services Mechanism -
Version 2

Workplan Proposal: Benchmarking Differentiated Services Mechanisms 
J. Perser, Spirent Communications 
D. Newman, Network Test 
S. Poretsky, Avici 

July 2000 

This memo proposes the development of practices for benchmarking the
performance of devices that implement differentiated services mechanisms.
The objective of such mechanisms is to prioritize certain traffic classes
during periods of device or network congestion. Accordingly, the objective
of this effort is to develop terminology and methodology for evaluating the
effectiveness of those mechanisms.

We propose that this effort be conducted within the Benchmarking Working
Group (bmwg) of the Internet Engineering Task Force (IETF).

1. Effort's Motivation 

Driven by exponential traffic growth, demand for bandwidth-hungry
applications, and a desire for guaranteed service levels, vendors of
Internet routers and switches had added mechanisms intended to ensure
different service levels for different classes of IP traffic.  There is
currently no standard practice for evaluating whether these Differentiated
Services (DS) mechanisms actually work as advertised.  As a result, there
are widely varying definitions of terms and methodologies for evaluating DS
performance.

The lack of an agreed-upon terminology and methodology makes it difficult to
perform meaningful comparisons of different vendors' implementations of the
same mechanism.

2. Effort's Goals 

The general goal of this effort is to develop terminology and methodology
documents for benchmarking the per-hop behavior characteristics of devices
implementing DS mechanisms.

The specific focus of this effort is to identify terms and practices for
benchmarking devices that examine either the precedence field as defined in
RFC 791 or the differentiated services code point as defined in RFC 2474. 

The authors' intent with these documents is to enable comparison of
routing/switching devices that use DS enabled hardware or software.

3. Plan and Milestones 

This effort will follow the standard BMWG procedure of producing separate
terminology and methodology documents. 

3.1 Terminology Document 

The terminology document will serve as the foundation for the methodology
document by defining DS test metrics and other concepts related to IP
precedence and DSCP settings.  The main metrics to be addressed in the
terminology document include:

 --forwarding rate 
 --packet loss 
 --latency 
 --jitter 

The terminology document also will define concepts and parameters for a
congesting a network.  Differentiated services may not become active until
the traffic exceeds the DUT/SUT throughput.  Describing how a test congests
the network is as important as how the device under test prioritizes traffic
during periods of congestion.

3.1.1 Terminology Milestones 
  
We propose the following timetable for completion of the terminology
document: 

  October  2000    First draft presented to BMWG 
  April    2001    WG Last Call on DS Benchmarking Terminology I-D. 

3.2 Methodology Documents 

After the BMWG reaches rough consensus on the terminology document, the
authors propose to write companion methodology document.  The authors
propose a single methodology document for measuring performance of devices
using IP precedence and DSCP field mechanisms.  If a rough consensus can not
be made on a single document, then separate methodologies will be written
for each of the two mechanisms.

3.2.1 Methodology Milestones 

We propose the following timetable for completion of the DS test methodology
documents. 

  March    2001    First draft of Benchmarking Methodology 
  July     2001    Consensus on a single Methodology document 
  November 2001    WG Last Call on Benchmarking Methodology 

___________________________________________________________ 
Jerry Perser               Phone: 818.676.2320 
SmartLab Manager           Lab:   818.676.2337 
SmartBits Systems          Fax:   818.880.9165     
26750 Agoura Road          Corp:  818.676.2300 
Calabasas, CA, 91302       Pager: [email-masked]