Subject: [bmwg] FW: IETF BMWG Work Item Proposal for SIP Performance Benchmarking

From: [email-masked]
Date: Tue, 23 May 2006 17:50:06 -0400

Fellow BMWG-ers,

Please find this IETF Work Item Proposal for your review and comment.  The
authors will be submitting a first draft Terminology and Methodology for the
Montreal meeting.

Scott

>  -----Original Message-----
> From: 	Poretsky, Scott  
> Sent:	Thursday, May 11, 2006 12:44 PM
> To:	[email-masked]'
> Subject:	IETF BMWG Work Item Proposal for SIP Performance
> Benchmarking
> Importance:	High
> 
> Hi Al,
> 
[snip] 

> Thanks,
> Scott
> 
> ########################
> BMWG Work Item Proposal
> 
> TITLE: Session Initiation Protocol (SIP) Performance Benchmarking
> 
> AUTHOR TEAM:
> Dr. Carol Davids, Illinois Institute of Technology
> Dr. Vijay Gurbani, Lucent Technologies
Scott Poretsky, Reef Point Systems
>  
> GOALS
> The purpose of this work item is to provide a single terminology and
> methodology from which SIP equipment vendors and VoIP service providers
> can measure performance benchmarking metrics for comparison and selection.
> It is intended to develop terms, benchmarks, and methodologies that can be
> applied to any type of IP device including SIP Servers, Session Border
> Controllers (SBCs), and Security Gateways (SEGs).
> 
> MOTIVATION
> Service Providers are now planning VoIP and Multimedia network deployments
> using the IETF developed Session Initiation Protocol (SIP).  Through SIP,
> service providers will be able to have rich service offerings and many
> even plan to turn off their Public Switch Telephone Network within five
> years.   VoIP has led to development of new networking devices including
> SIP Servers, Session Border Controllers, and Security Gateways.  The mix
> of voice and IP functions in this variety of devices has produced
> inconsistencies in vendor reported performance metrics and has caused
> confusion in the service provider community. 
> 
> SCOPE:
> This work item will provide terms, benchmarks, and methodologies for
> performance benchmarking the SIP control and media planes.  The
> methodologies can be used for benchmarking SIP performance of SIP servers,
> SBCs, and SEGs.  The media used for the benchmarking will be the IETF
> defined Real-Time Protocol (RTP).  Test cases will allow testing of VoIP
> or Multimedia over IP by varying the number of media streams per SIP call
> from 1 to higher.  There will be at least one test case to measure the
> impact of a SIP DOS Attack on the performance metrics.  All SIP traffic
> will be in the clear; no encryption using TLS or IPsec will be used.
> Similar benchmarks and methodologies using encryption is a potential work
> item for the BMWG that could be considered in the future.
> 
> There will be separate test cases in the methodology document for
> obtaining measurements for each of the Benchmarking Metrics defined in the
> Terminology.  The benchmarking metrics to be measured for the SIP Control
> Plane and SIP Media Plane are as follow:
> 
> 	Benchmark Metrics - SIP Control Plane 
> 	Standing Calls, maximum (calls)
> 	Calls Per Second, maximum (CPS)
> 	Call Attempts Per Second, maximum (CAPS)
> 	Busy Hour Call Attempts, maximum (BHCA)
> 	Busy Hour Call Connects, maximum (BHCC)
> 	Call Completion Rate (%)
> 	Call Setup Delay, average (msec)
> 	Call Teardown Delay, average (msec)
> 
> 	Benchmark Metrics- SIP Media Plane
> 	RTP Media Throughput, per Media Stream (pps)
> 	RTP Media Throughput, Aggregate (pps)
> 	RTP Packet Loss, average (pps)
> 	RTP Packet Delay, average (msec)
> 	RTP Packet Jitter, average (msec)
> 
> In order to obtain measurements for the benchmarking metrics it will be
> necessary to configure the test setup with certain test parameters.  Some
> of these test parameters may also be the benchmarking metric being
> measured for another test case.  The test parameters to be configured for
> the SIP Control Plane and SIP Media Plane are as follow:
> 
> 	Test Parameters- SIP Control Plane 
> 	Call Duration (msec)
> 	Call Per Second (CPS)
> 	Call Attempts Per Second (CAPS)
> 
> 	Test Parameters - SIP Media Plane
> 	RTP Media Streams per Call (streams per call)
> 	RTP Packet Size (bytes)
> 	RTP Media Offered Load, per Media Stream (pps)
> 	RTP Media Offered Load, Aggregate (pps)
> 
> The basic test topology to be used for this benchmarking is as follows:
> 
> Emulated Agents<--> SEG (optional) <--> SIP Proxy Server (or SBC) <--> SIP
> Server (optional) <--> Emulated Agents
> 
> PROPOSED MILESTONES
> 06/07 First Draft Terminology and Methodology
> 02/07 First WG Last Call
> 05/07 SIP WG and SIPPING WG Review
> 06/07 Final BMWG Last Call for Terminology and Methodology
> 08/07 Submittal for IESG review
> 
> 
> 

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg