Subject: RE: [bmwg] RE: [Sipping] Draft: SIP Performance Metrics

From: [email-masked]
Date: Sun, 25 Jun 2006 08:07:35 -0400

The authors of this work have been in regular communication from Erich M.
Nahum of SPEC.  Here is a snip from one of the emails that well summmarizes
the relationship:

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Friday, May 26, 2006 2:51 PM
> To: Vijay K. Gurbani
> Cc: Malas, Daryl; [email-masked]; [email-masked]; [email-masked]
> Subject: [bmwg] SPEC SIP Benchmarking Effort
>
> [snip]
>
> I have been organizing the SPEC SIP benchmarking effort, and would
> love to get participation from IETF folks.  It seems like the two
> activities could be very complementary.  
> 
> Based on what I've seen, it seems like the IETF BMWG and Daryl Malas' 
> draft posted on SIPPING a few days ago are more oriented towards defining 
> the proper methods and metrics to measure (e.g., call success rate,
defects 
> per million, etc.).  
> 
> In contrast, SPEC wants to develop and distribute common code for 
> benchmarking, as is done with SPECWeb and SPECJAppServer.  That code can 
> and should use the standardized peformance definitions agreed to by 
> SIPPING and/or BMWG.  

Scott

-----Original Message-----
From: Romascanu, Dan (Dan) [mailto:[email-masked]]
Sent: Sunday, June 25, 2006 6:00 AM
To: Burger, Eric; Malas, Daryl
Cc: [email-masked]; [email-masked]
Subject: RE: [bmwg] RE: [Sipping] Draft: SIP Performance Metrics

Eric,

>From the bmwg charter: 

> The major goal of the Benchmarking Methodology Working Group is to
make
a series of recommendations concerning the measurement of the
performance characteristics of various internetworking technologies;

To the extend that we consider SIP an internetworking technology, I
believe that the scope of the 'SIP Performance Metrics' draft is within
the scope of what bmwg is doing for a while, quite successfully, some
say. 

On a more 'philosophical plan', there is nothing that says that the IETF
work must strictly deal with defining the bits in the Internet Protocols
- see http://www.ietf.org/internet-drafts/draft-hoffman-taobis-08.txt.
And in any case, measuring how a protocol or a device implementing a
protocol behaves can be considered also 'DIRECTLY related to protocol
development'.

Dan

PS - This does not mean that I do not have my own questions about the
I-D :-)

PPS - If we end by deciding that we do this work in the IETF we should
certainly coordinate with other organizations developing standards,
performance measurement and test methodologies in this space. 

> -----Original Message-----
> From: Burger, Eric [mailto:[email-masked]] 
> Sent: Friday, June 23, 2006 11:18 PM
> To: Malas, Daryl
> Cc: [email-masked]; [email-masked]
> Subject: [bmwg] RE: [Sipping] Draft: SIP Performance Metrics
> 
> More to the point, is this work appropriate for the IETF at 
> all?  I would offer that if we needed additions to SIP to 
> instrument it for monitoring, then sure.  However, I think 
> this would much better be done in another venue such as the 
> SIP Forum or IMS Forum or even SPEC.  If it turns out that 
> there need to be some SIP changes, then we could bring that 
> to the IETF.
> 
> We have way too much on our plate as it is.
> 
> 
> 
> Disclaimer: I'm on the boards of both the SIP Forum and IMS Forum.
> However, this isn't make-work for those groups.  Rather, this 
> is a plea to NOT make more work in the IETF that is not 
> DIRECTLY related to protocol development.
> 
> -----Original Message-----
> From: Henning Schulzrinne [mailto:[email-masked]]
> Sent: Wednesday, May 24, 2006 2:44 PM
> To: Vijay K. Gurbani
> Cc: Malas, Daryl; [email-masked]; [email-masked]; [email-masked]
> Subject: Re: [Sipping] Draft: SIP Performance Metrics
> 
> A working group has just been formed (this week) in the SPEC 
> consortium.
> 
> Vijay K. Gurbani wrote:
> > Daryl: There is work starting in the BMWG along SIP performance 
> > metrics, being worked by Scott Poretsky and me.  At the 
> very least, we 
> > should co-ordinate your draft with the thoughts there, and then can 
> > figure out with the help of chairs and ADs as to where the 
> work should 
> > proceed.
> > 
> > I have also heard about some performance metrics work in TISPAN and 
> > SPEC, but I do not have any more details about these.
> > 
> 
> _______________________________________________
> Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> This list is for NEW development of the application of SIP 
> Use [email-masked] for questions on current 
> sip Use [email-masked] for new developments of core SIP
> 
> _______________________________________________
> bmwg mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/bmwg
> 

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: [email-masked]
Date: Sun, 25 Jun 2006 17:23:07 -0400

> Nahum of SPEC.  Here is a snip from one of the emails that well summmarizes

Actually, I'm from IBM, not SPEC.  SPEC is just a consortium of its member
companies/organizations.  Sort of like IETF.

-Erich

-- 
Erich M. Nahum                  IBM T.J. Watson Research Center
Research Staff Member           P.O. Box 704
[email-masked]            Yorktown Heights NY 10598

---

From: [email-masked]
Date: Mon, 26 Jun 2006 15:05:42 -0400

From: [email-masked]

   Actually, I'm from IBM, not SPEC.  SPEC is just a consortium of its
   member companies/organizations.  Sort of like IETF.

IETF is a consortium of its member *individuals*, which makes a great
deal of difference.

Dale