Subject: Re: Workplan Proposal: Benchmarking Differentiated Services Mechanism - Version 2

From: Sumit Khurana <[email-masked]>
Date: Wed, 6 Sep 2000 15:08:21 -0400 (EDT)

Jerry,
	We have submitted an  Internet Draft
draft-khurana-bmwg-diffservm-00.txt, "Benchmarking Methodology for
Devices Implementing Differentiated Services"  to the bmwg. This draft was
also discussed at the Pittsburgh IETF and we had requested that it be
accepted as a working group document. 

Thanks,
-Sumit

On Fri, 25 Aug 2000, Perser, Jerry wrote:

> Workplan Proposal: Benchmarking Differentiated Services Mechanisms
> J. Perser, Spirent Communications 
> D. Newman, Network Test
> S. Poretsky, Avici
> 
> July 2000
> 
> This memo proposes the development of practices for benchmarking the
> performance of devices that implement differentiated services mechanisms.
> The objective of such mechanisms is to prioritize certain traffic classes
> during periods of device or network congestion. Accordingly, the objective
> of this effort is to develop terminology and methodology for evaluating the
> effectiveness of those mechanisms.
> We propose that this effort be conducted within the Benchmarking Working
> Group (bmwg) of the Internet Engineering Task Force (IETF).
> 
> 1. Effort's Motivation
> 
> Driven by exponential traffic growth, demand for bandwidth-hungry
> applications, and a desire for guaranteed service levels, vendors of
> Internet routers and switches had added mechanisms intended to ensure
> different service levels for different classes of IP traffic.  There is
> currently no standard practice for evaluating whether these Differentiated
> Services (DS) mechanisms actually work as advertised.  As a result, there
> are widely varying definitions of terms and methodologies for evaluating DS
> performance.
> 
> The lack of an agreed-upon terminology and methodology makes it difficult to
> perform meaningful comparisons of different vendors' implementations of the
> same mechanism.
> 
> 2. Effort's Goals
> 
> The general goal of this effort is to develop terminology and methodology
> documents for benchmarking the per-hop behavior characteristics of devices
> implementing DS mechanisms.
> 
> The specific focus of this effort is to identify terms and practices for
> benchmarking devices that examine either the precedence field as defined in
> RFC 791 or the differentiated services code point as defined in RFC 2474. 
> 
> The authors' intent with these documents is to enable comparison of
> routing/switching devices that use DS enabled hardware or software.
> 
> 3. Plan and Milestones
> 
> This effort will follow the standard BMWG procedure of producing separate
> terminology and methodology documents.
> 
> 3.1 Terminology Document
> 
> The terminology document will serve as the foundation for the methodology
> document by defining DS test metrics and other concepts related to IP
> precedence and DSCP settings.  The main metrics to be addressed in the
> terminology document include:
> 
>  --forwarding rate
>  --packet loss
>  --latency
>  --jitter
> 
> The terminology document also will define concepts and parameters for a
> congesting a network.  Differentiated services may not become active until
> the traffic exceeds the DUT/SUT throughput.  Describing how a test congests
> the network is as important as how the device under test prioritizes traffic
> during periods of congestion.
> 
> 3.1.1 Terminology Milestones
>  
> We propose the following timetable for completion of the terminology
> document:
> 
>   October  2000    First draft presented to BMWG 
>   April    2001    WG Last Call on DS Benchmarking Terminology I-D.
> 
> 3.2 Methodology Documents
> 
> After the BMWG reaches rough consensus on the terminology document, the
> authors propose to write companion methodology document.  The authors
> propose a single methodology document for measuring performance of devices
> using IP precedence and DSCP field mechanisms.  If a rough consensus can not
> be made on a single document, then separate methodologies will be written
> for each of the two mechanisms.
> 
> 3.2.1 Methodology Milestones
> 
> We propose the following timetable for completion of the DS test methodology
> documents.
> 
>   March    2001    First draft of Benchmarking Methodology
>   July     2001    Consensus on a single Methodology document
>   November 2001    WG Last Call on Benchmarking Methodology
> 
> ___________________________________________________________ 
> Jerry Perser               Phone: 818.676.2320 
> SmartLab Manager           Lab:   818.676.2337 
> SmartBits Systems          Fax:   818.880.9165     
> 26750 Agoura Road          Corp:  818.676.2300 
> Calabasas, CA, 91302       Pager: [email-masked]
> 
>   
>