One document matched: draft-malas-performance-metrics-03.txt
Differences from draft-malas-performance-metrics-02.txt
SIPPING Working Group D. Malas
Internet Draft Level 3 Communications
Expires: December 2006 June 22, 2006
SIP Performance Metrics
draft-malas-performance-metrics-03.txt
Status of this Memo
By submitting this Internet-Draft, each author represents that
any applicable patent or other IPR claims of which he or she is
aware have been or will be disclosed, and any of which he or she
becomes aware will be disclosed, in accordance with Section 6 of
BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on December 22, 2006.
Abstract
This document defines the use of industry recommended reliability
metrics for use with the SIP.
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119 [1].
Table of Contents
1. Introduction...................................................2
Malas Expires December 22, 2006 [Page 1]
Internet-Draft SIP Performance Metrics June 2006
2. SIP Performance Metrics........................................3
2.1. Session Request Delay (SRD)...............................3
2.2. Session Completion Delay (SCD)............................4
2.3. Average Hops per INVITE (AHI).............................4
2.4. Average Hops per Session (AHS)............................5
2.5. Session Duration Time (SDT)...............................5
2.6. Session Establishment Rate (SER)..........................7
2.7. Session Defects (SD)......................................7
2.8. Ineffective Session Attempts (ISA)........................8
2.9. Session Disconnect Failures (SDF).........................8
2.10. Session Completion Rate (SCR)............................9
2.11. Session Success Rate (SSR)..............................10
2.12. Additional metrics under consideration..................11
3. Back-to-back User Agent (B2BUA) Considerations................11
4. Data Collection Considerations................................11
5. Metric Correlations...........................................12
6. Security Considerations.......................................12
7. IANA Considerations...........................................12
8. Conclusions...................................................12
9. Acknowledgments...............................................12
10. References...................................................13
10.1. Normative References....................................13
10.2. Informative References..................................13
Author's Addresses...............................................13
Intellectual Property Statement..................................13
Disclaimer of Validity...........................................14
Copyright Statement..............................................14
Acknowledgment...................................................14
1. Introduction
SIP has become a standard among many service providers, vendors, and
end users. Although there are many different standards for measuring
the performance of signaling protocols, none of these have been
adapted for use with SIP. This document is intended for providing a
guideline for the above listed entities in providing a standard
approach for measuring and reporting SIP performance metrics in a
production environment with an end-to-end perspective. This will
allow a common approach and understanding of expectations between
service providers, vendors, and the users of those services.
Not all metrics for performance map to all applications of the SIP.
This document provides an overview of many different metrics, which
may be used as an individual or set of metrics necessary based on the
use of SIP.
Malas Expires December 22, 2006 [Page 2]
Internet-Draft SIP Performance Metrics June 2006
There are many metrics available for determining performance.
Although this document contains a number of them, it is not intended
to be exhaustive. Instead, it is designed to provide a common sub-
set with a common agreed upon definition. This document does not
provide any standard or benchmark information regarding IETF
recommended performance criteria to compare any output value derived
from the following described metrics.
2. SIP Performance Metrics
The following metrics may be utilized for many different SIP
applications. In regards to all of the following metrics, message
re-transmissions must be excluded in order to provide accurate metric
results.
Some metrics are calculated based on the final message responses.
These metrics do not take into consideration route advances to
additional signaling functions based on "final" failure responses.
In these unique cases, the final response related to the initial
setup attempt should be utilized for input to the metric.
2.1. Session Request Delay (SRD)
It is important session request delay is calculated for both sessions
ending in failure and success. In a successful request attempt, SRD
is defined as the time interval from the moment the INVITE message
containing the necessary information is passed by the originating
agent or user to the intended mediation or destination agent until
the first provisional response is received indicating an audible or
visual status of the INVITE request. In SIP, the message indicating
status would be a non-100 Trying provisional message received in
response to an INVITE request. In some cases, a non-100 Trying
provisional message is not received, but rather a 200 message is
received as the first status message instead. In these situations,
the 200 message would be used to calculate the interval. In a failed
request attempt, the interval is defined from the INVITE request and
a non-100 Trying provisional message or a failure indication
response. A failure response is described as a 4XX, 5XX, or possible
6XX message. SRD may be used to detect problems in downstream
signaling functions, which may be impairing the INVITE message from
reaching the intended UA or UAS.
SRD = Time of Status Indicative Response – Time of INVITE
Malas Expires December 22, 2006 [Page 3]
Internet-Draft SIP Performance Metrics June 2006
SUM (Time of Status Indicative Response – Time of INVITE)
ASRD = ---------------------------------------------------------
SUM # of INVITE Requests
ASRD = Average SRD
The following flow provides an example of Session Request Delay:
UA1 UA2
| |
|INVITE |
|---------------------> |
| /\ 100|
| <---------||----------|
| SRD |
| || |
| \/ 180|
| <---------------------|
| |
2.2. Session Completion Delay (SCD)
SCD is defined as the interval between sending a session completion
message, such as a BYE, and receiving the subsequent 2XX
acknowledgement. The following flow provides an example of this
metric:
UA1 UA2
| |
|INVITE |
|---------------------> |
| 100|
| <---------------------|
| 180/200|
| <---------------------|
| BYE|
|---------------------->|
| /\ |
| || |
| SCD |
| || |
| \/ 200|
|<----------------------|
2.3. Average Hops per INVITE (AHI)
AHI is calculated as an average and is defined as the number of hops
per INVITE request. This metric is used to indicate potential
Malas Expires December 22, 2006 [Page 4]
Internet-Draft SIP Performance Metrics June 2006
inefficient routing and/or help an operator detect and/or prevent
routing loops.
Variables =
a = # of INVITE requests per session attempt
b = SUM of a "Max Forwards" value
c = Max Forwards value in originating INVITE
(a * c) – (b)
AHI = -----------------
a
In order for the results of this and the following metric to be
accurate, the Max Forwards value should remain consistent throughout
the measured end-to-end network.
2.4. Average Hops per Session (AHS)
AHS is calculated in a similar manner to AHI; however, the "Max
Forwards" value is taken from each request associated with the entire
session as described in the following section 2.5. This metric is
also used in a similar manner as AHI.
Variables =
a = # of SIP requests
b = "Max Forwards" value in originating message
c = # of completed sessions
d = SUM of a "Max Forwards" value
(a * b) – (d)
AHS = -----------------
c
2.5. Session Duration Time (SDT)
SDT is usually calculated as an average and is defined as the
duration of a dialog from receipt of a 200 OK response to an INVITE
and an associated BYE message indicating dialog completion. This
metric is used to detect problems causing short session durations.
Malas Expires December 22, 2006 [Page 5]
Internet-Draft SIP Performance Metrics June 2006
SDT = Time of BYE – Time of 200 OK response to INVITE
SUM (Time of BYE – Time of 200 OK response to INVITE)
ASDT = ------------------------------------------------------
SUM # of INVITE w/ 200OK & BYE
ASDT = Average SDT
The following flow represents an example of the determination of this
metric:
UA1 UA2
| |
|INVITE |
|---------------------> |
| 100|
| <---------------------|
| 180|
| <---------------------|
| 200|
| <---------------------|
| /\ |
| || |
| SDT |
| || |
|BYE \/ |
|---------------------> |
Malas Expires December 22, 2006 [Page 6]
Internet-Draft SIP Performance Metrics June 2006
2.6. Session Establishment Rate (SER)
SER is defined as the number of INVITE requests resulting in a 200 OK
response, to the total number of attempted INVITE requests. This
metric is used to detect the ability of a terminating UA or UAS’s
ability to successfully establish sessions per INVITE request.
# of INVITE Requests w/ associated 200OK
SER = ----------------------------------------
Total # of INVITE Requests
The following flow represents session establishment as described
above:
UA1 UA2
| |
|INVITE |
|---------------------> |
| /\ 100|
| <---------||----------|
| || |
| Session Established |
| || 180|
| <---------||----------|
| \/ 200|
| <---------------------|
| |
| |
2.7. Session Defects (SD)
Session defects provide a subset of SIP failure responses, which
consistently indicate a failure in dialog processing. Defects are
necessary to provide input to calculations such as Defects per
Million (DPM) or other similar metrics. These failure responses are
in response to initial session setup requests, such as a new INVITE.
The following failure responses provide a guideline for defective
criterion:
. 500 Server Internal Error
. 503 Service Unavailable
. 504 Server Timeout
This set of failure responses was derived through correlating more
granular ISUP failure responses as described in RFC 3398.
Malas Expires December 22, 2006 [Page 7]
Internet-Draft SIP Performance Metrics June 2006
2.8. Ineffective Session Attempts (ISA)
Ineffective session attempts occur when a proxy or agent internally
releases a setup request with a failed or congested condition. The
following failure responses provide a guideline for this criterion:
. 408 Request Timeout
. 500 Server Internal Error
. 503 Service Unavailable
. 504 Server Timeout
This set was derived in a similar manner as described in Section 2.7,
in addition 408 failure responses can be indicative a congested state
with a downstream element.
This metric is calculated as a percentage of total session setup
requests. The following calculation provides a guideline:
# of ISA
ISA % = --------------------------
Total # of INVITE Requests
2.9. Session Disconnect Failures (SDF)
Session disconnect failures occur when an active session is
terminated due to a failure condition that can be identified by a
REASON header [5] in a BYE. This occurs, for example, when a user
agent server (UAS) is controlling an IP or TDM (Time Division
Multiplexing) media gateway, and the media gateway notifies the UAS
of a failure condition causing the loss of media related to an
established session. The UAS will release the session with a BYE,
but should include a REASON header indicating the session was
disconnected abnormally. The REASON value is utilized to determine
the disconnect was a failure.
This metric is calculated as a percentage of total session completed
successfully as defined in Section 2.6. The following calculation
provides a guideline:
# of SDF's
SDF % = -------------------------------------------
Total # of Successfully Set-up Sessions
Malas Expires December 22, 2006 [Page 8]
Internet-Draft SIP Performance Metrics June 2006
2.10. Session Completion Rate (SCR)
A session completion, as described in this metric, is defined as a
SIP dialog, which completes without failing due to a lack of response
from an intended proxy, UAS, or UA. A session completes successfully
when it begins with a setup request and ends with a session
completion message. This metric is only used when at least one proxy
is involved in the dialog.
The following dialog [4] describes a successful session completion:
Alice Proxy 1 Proxy 2 Bob
| | | |
| INVITE | | |
|--------------->| | |
| 407 | | |
|<---------------| | |
| ACK | | |
|--------------->| | |
| INVITE | | |
|--------------->| INVITE | |
| 100 |--------------->| INVITE |
|<---------------| 100 |--------------->|
| |<---------------| |
| | | 180 |
| | 180 |<---------------|
| 180 |<---------------| |
|<---------------| | 200 |
| | 200 |<---------------|
| 200 |<---------------| |
|<---------------| | |
| ACK | | |
|--------------->| ACK | |
| |--------------->| ACK |
| | |--------------->|
| Both Way RTP Media |
|<================================================>|
| | | BYE |
| | BYE |<---------------|
| BYE |<---------------| |
|<---------------| | |
| 200 | | |
|--------------->| 200 | |
| |--------------->| 200 |
| | |--------------->|
| | | |
Malas Expires December 22, 2006 [Page 9]
Internet-Draft SIP Performance Metrics June 2006
The following dialog describes an unsuccessful session completion:
Alice Proxy 1 Proxy 2 Bob
| | | |
| INVITE | | |
|--------------->| | |
| 407 | | |
|<---------------| | |
| ACK | | |
|--------------->| | |
| INVITE | | |
|--------------->| INVITE | |
| 100 |--------------->| INVITE |
|<---------------| 100 |--------------->|
| |<---------------| |
| | | INVITE |
| | |--------------->|
| | | |
| | | INVITE |
| | |--------------->|
| | | |
| | 408 | |
| 408 |<---------------| |
|<---------------| ACK | |
| |--------------->| |
| ACK | | |
|--------------->| | |
This metric is calculated as a percentage of total sessions completed
successfully. The following calculation provides a guideline:
# of Successfully Completed Sessions
SCR % = ---------------------------------------
Total # of Session Attempts
2.11. Session Success Rate (SSR)
Session success rate is included for usage to combine metrics
providing a description of the overall service perspective a vendor
or provider. It is defined as the percentage of successfully
completed sessions compared to sessions, which fail due to ISA or
SDF. The following calculation provides a guideline:
SSR = 100% - (ISA% + SDF%)
Malas Expires December 22, 2006 [Page 10]
Internet-Draft SIP Performance Metrics June 2006
2.12. Additional metrics under consideration
The following metrics have been suggested, but need to be
determined as necessary for inclusion in this document.
Retries per Session - This metric will detect the number of
message retry attempts per session attempt or establishment.
Average Contact User Selection – This metric will determine the
average selected contact per 300 response.
Refers per Session – This metric will determine the number of
Refers per established session.
Re-INVITE's per Session – This metric will determine the number
of RE-INVITE’s per established session.
3. Back-to-back User Agent (B2BUA) Considerations
A B2BUA may impact the ability to collect these metrics with an end-
to-end perspective. It is necessary to realize a B2BUA may act as an
originating and terminating UA or it may act as a proxy. In some
cases, it may be necessary to consider information collected from
both sides of the B2BUA in order to determine the end-to-end
perspective. In other cases, the B2BUA may act simply as a proxy
allowing data to be derived as necessary for the input into any of
the listed calculations.
4. Data Collection Considerations
The input necessary for these calculations may be collected in a
number of different manners. It may be collected or retrieved from
call detail records (CDR) or raw signaling information generated by a
proxy, UA, or UAS.
The information may also be transmitted through use of SNMP traps as
described in the work in progress SIP MIB draft [6], or through a
potential undefined new performance metric event package [3]
retrieved via SUBSCRIBE requests.
Data may be collected for a sample of calls or all calls, and may
also be derived from test call scenarios. These metrics are flexible
based on the needs of the application.
Malas Expires December 22, 2006 [Page 11]
Internet-Draft SIP Performance Metrics June 2006
5. Metric Correlations
These metrics may be used to determine the performance of a domain
and/or user. This would be to provide a metric relative to one or
more dimensions. The following is a subset of dimensions for
providing further granularity per metric:
- To "user"
- From "user"
- Bi-directional To/From "user"
- To "domain"
- From "domain"
- Bi-directional To/From "domain"
Example: The SCR of domain A is 99.97%.
6. Security Considerations
Security should be considered in the aspect of securing the relative
data utilized in providing input to the above calculations. All
other aspects of security should be considered as described in [2].
7. IANA Considerations
There are no IANA considerations at this time.
8. Conclusions
The proposed guideline provides a description of common performance
metrics, and their defined use with SIP. The use of these metrics
will provide a common viewpoint across all vendors, service
providers, and customers. These metrics will likely be utilized in
production SIP environments for providing input regarding Key
Performance Indicators (KPI) and Service Level Agreement (SLA)
indications.
9. Acknowledgments
I would like to thank John Hearty for his efforts in scrubbing
through the draft and providing insight regarding clarification of
certain aspects described throughout the document.
Malas Expires December 22, 2006 [Page 12]
Internet-Draft SIP Performance Metrics June 2006
10. References
10.1. Normative References
[1] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
[2] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A.,
Peterson, J., Sparks, R., Handley, M. and E. Schooler, "SIP:
Session Initiation Protocol", RFC 3261, June 2002.
[3] Roach, A., "Session Initiation Protocol (SIP)-Specific Event
Notification", RFC 3265, June 2002.
[4] Johnston, A., Donovan, S., Sparks, R., Cunningham, C., and K.
Summers, "Session Initiation Protocol (SIP) Basic Call
Flow Examples", BCP 75, RFC 3665, December 2003.
[5] Schulzrinne, H., Oran, D., Camarillo, G., "The Reason Header
Field for the Sessions Initiation Protocol (SIP)", RFC 3326,
December 2002.
[6] Lingle, K., Mule, J., Maeng, J., Walker, D., "Management
Information Base for the Session Initiation Protocol (SIP)",
draft-ietf-sip-mib-10, Work in Progress.
10.2. Informative References
Author's Addresses
Daryl Malas
Level 3 Communications LLC
1025 Eldorado Blvd.
Broomfield, CO 80021
USA
EMail: daryl.malas@level3.com
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
Malas Expires December 22, 2006 [Page 13]
Internet-Draft SIP Performance Metrics June 2006
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Disclaimer of Validity
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement
Copyright (C) The Internet Society (2006).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
Malas Expires December 22, 2006 [Page 14]
| PAFTECH AB 2003-2026 | 2026-04-24 02:45:58 |