Subject: [Sipping] New Draft: SIP Performance and Reliability Metrics

From: "Malas, Daryl" <[email-masked]>
Date: Tue, 23 May 2006 14:07:14 -0600

Sipping WG,

I have submitted a new draft for review regarding performance metrics.

Synopsis:

"SIP has become a standard among many service providers, vendors, and
end users.  Although there are many different standards for measuring
the reliability and performance of signaling protocols, none of these
have been adapted for use with SIP.  This document is intended for
providing a guideline for the above listed entities in providing a
standard approach for measuring and reporting SIP reliability and
performance metrics.  This will allow a common approach and
understanding of expectations between service providers, vendors, and
the users of those services."

http://www.ietf.org/internet-drafts/draft-malas-performance-metrics-00.t
xt

I have many conversations, seems like daily, regarding this topic.
Usually, it involves how many different methods and definitions there
are to understand the performance of SIP.  I would really like for this
working group to work together to create a subset of metrics, which can
be perceived as a standard (starting point) when discussing a minimum
set of performance and reliability metrics.  This standard could then be
referenced in almost every SIP-related IETF document in the QoS/Network
Management (or similar) section.

As you read through the document, you will notice common terms usually
referred to in a telephony application of SIP.  Although, I could
probably sit here and come up with a number of clever acronyms to
describe the metrics, I believe the method and definition remains the
same.

I am asking this WG for help in providing suggestions for any other
metrics; however, I do not think this list should be exhaustive.  I
think it should provide a common, familiar baseline of metrics.
Vendors, enterprises, federations, etc., will continue to develop unique
metrics as necessary; but, I believe this document will provide a
standard to baseline against.

Please let me know what you think.  I believe this is critical to
provide uniformity among all companies and standards bodies utilizing
the SIP.

Thanks...

Daryl
[email-masked]

BTW, if the hyperlink screws up...then will someone please tell me how
to insert it without it breaking up, and making it difficult to simply
click?

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: Prakasha <[email-masked]>
Date: Thu, 25 May 2006 10:23:16 +0530

Copying earlier mail sent to Daryl...

-----Original Message-----
From: Prakasha [mailto:[email-masked]] 
Sent: Wednesday, May 24, 2006 10:56 AM
To: 'Malas, Daryl'
Subject: RE: [Sipping] New Draft: SIP Performance and Reliability Metrics

An interesting document and I guess with more and more operators embracing
SIP and SIP based application servers, this document would very well be used
as reference by those operators. It would also help them to set benchmark in
their network while multiple vendors are involved in the network.

Couple of comments:

- I guess SIPStone already has identified couple of other metrics like
INVITE
  processing time etc. Can they be included here? 
- Is there any metrics on how many SIP messages can be processed per second
  in a particular configuration of the hardware (or any method to arrive
this figure)?
- For the CHT, won't using ACK provide accurate measure?
- I am not convinced about the 5XX beeing used as defects per million
measure. 5XX responses
  could also be sent back when the server is overloaded because of some
network
  condition. So in such cases, should the "defect" be attributed to the
server?
- I am not sure if the metrics that you have identified are to be reported
to the OSS,
  then these may have to be included in the SIP MIB draft
(http://www.ietf.org/internet-drafts/draft-ietf-sip-mib-10.txt)
- How about some metrics like - number of media updates with UPDATE, number
  of retransmissions, number of calls failed in call setup stage

My 2 cents,

Regards,
Prakash

Huawei Technologies India Pvt. Ltd
Bangalore, India
Tele: +91-80-41781770 ext: 1138
Mobile: +91-9448455131

http://www.huawei.com

**********************************************************************
This  e-mail and its  attachments  contain  confidential information 
from HUAWEI,  which is intended only for the  person or entity whose 
address is listed above.  Any use of the information contained herein 
in any way (including, but not limited to, total or partial disclosure, 
reproduction, or  dissemination) by  persons other  than the intended 
recipient(s) is prohibited. If you receive this e-mail in error, please 
notify the sender by phone or email immediately and delete it!
***********************************************************************

-----Original Message-----
From: Malas, Daryl [mailto:[email-masked]] 
Sent: Wednesday, May 24, 2006 1:37 AM
To: [email-masked]
Subject: [Sipping] New Draft: SIP Performance and Reliability Metrics

Sipping WG,

Synopsis:

"SIP has become a standard among many service providers, vendors, and end
users.  Although there are many different standards for measuring the
reliability and performance of signaling protocols, none of these have been
adapted for use with SIP.  This document is intended for providing a
guideline for the above listed entities in providing a standard approach for
measuring and reporting SIP reliability and performance metrics.  This will
allow a common approach and understanding of expectations between service
providers, vendors, and the users of those services."

I have many conversations, seems like daily, regarding this topic. Usually,
it involves how many different methods and definitions there are to
understand the performance of SIP.  I would really like for this working
group to work together to create a subset of metrics, which can be perceived
as a standard (starting point) when discussing a minimum set of performance
and reliability metrics.  This standard could then be referenced in almost
every SIP-related IETF document in the QoS/Network Management (or similar)
section.

As you read through the document, you will notice common terms usually
referred to in a telephony application of SIP.  Although, I could probably
sit here and come up with a number of clever acronyms to describe the
metrics, I believe the method and definition remains the same.

I am asking this WG for help in providing suggestions for any other metrics;
however, I do not think this list should be exhaustive.  I think it should
provide a common, familiar baseline of metrics. Vendors, enterprises,
federations, etc., will continue to develop unique metrics as necessary;
but, I believe this document will provide a standard to baseline against.

Please let me know what you think.  I believe this is critical to provide
uniformity among all companies and standards bodies utilizing the SIP.

Thanks...

Daryl
[email-masked]

BTW, if the hyperlink screws up...then will someone please tell me how to
insert it without it breaking up, and making it difficult to simply click?

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip Use
[email-masked] for new developments of core SIP