One document matched: draft-ietf-pmol-sip-perf-metrics-06.xml
<?xml version="1.0" encoding="US-ASCII"?>
<!-- This template is for creating an Internet Draft using xml2rfc,
which is available here: http://xml.resource.org. -->
<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
<!-- One method to get references from the online citation libraries.
There has to be one entity for each item to be referenced.
An alternate method (rfc include) is described in the references. -->
<!ENTITY RFC2119 SYSTEM "reference.RFC.2119.xml">
<!ENTITY RFC3261 SYSTEM "reference.RFC.3261.xml">
<!ENTITY RFC3265 SYSTEM "reference.RFC.3265.xml">
<!ENTITY RFC3665 SYSTEM "reference.RFC.3665.xml">
<!ENTITY RFC3326 SYSTEM "reference.RFC.3326.xml">
<!ENTITY RFC3428 SYSTEM "reference.RFC.3428.xml">
<!ENTITY RFC4780 SYSTEM "reference.RFC.4780.xml">
<!ENTITY RFC3398 SYSTEM "reference.RFC.3398.xml">
<!ENTITY RFC2330 SYSTEM "reference.RFC.2330.xml">
<!ENTITY RFC3262 SYSTEM "reference.RFC.3262.xml">
<!ENTITY RFC5486 SYSTEM "reference.RFC.5486.xml">
]>
<?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?>
<!-- used by XSLT processors -->
<!-- For a complete list and description of processing instructions (PIs),
please see http://xml.resource.org/authoring/README.html. -->
<!-- Below are generally applicable Processing Instructions (PIs) that most I-Ds might want to use.
(Here they are set differently than their defaults in xml2rfc v1.32) -->
<?rfc strict="no" ?>
<!-- give errors regarding ID-nits and DTD validation -->
<!-- control the table of contents (ToC) -->
<?rfc toc="yes"?>
<!-- generate a ToC -->
<?rfc tocdepth="4"?>
<!-- the number of levels of subsections in ToC. default: 3 -->
<!-- control references -->
<?rfc symrefs="yes"?>
<!-- use symbolic references tags, i.e, [RFC2119] instead of [1] -->
<?rfc sortrefs="yes" ?>
<!-- sort the reference entries alphabetically -->
<!-- control vertical white space
(using these PIs as follows is recommended by the RFC Editor) -->
<?rfc compact="yes" ?>
<!-- do not start each main section on a new page -->
<?rfc subcompact="no" ?>
<!-- keep one blank line between list items -->
<!-- end of list of popular I-D processing instructions -->
<rfc category="std" docName="draft-ietf-pmol-sip-perf-metrics-06"
ipr="trust200902">
<!-- ***** FRONT MATTER ***** -->
<front>
<!-- The abbreviated title is used in the page header - it is only necessary if the
full title is longer than 39 characters -->
<title abbrev="SIP End-to-End Performance Metrics">Basic Telephony SIP End-to-End
Performance Metrics</title>
<!-- add 'role="editor"' below for the editors if appropriate -->
<!-- Another author who claims to be an editor -->
<author fullname="Daryl Malas" initials="D.M." surname="Malas">
<organization>CableLabs</organization>
<address>
<postal>
<street>858 Coal Creek Circle</street>
<!-- Reorder these if your country does things differently -->
<city>Louisville</city>
<region>CO</region>
<code>80027</code>
<country>US</country>
</postal>
<phone>+1 303 661 3302</phone>
<email>d.malas@cablelabs.com</email>
<!-- uri and facsimile elements may also be added -->
</address>
</author>
<author fullname="Al Morton" initials="A.M." surname="Morton">
<organization>AT&T Labs</organization>
<address>
<postal>
<street>200 Laurel Avenue South</street>
<!-- Reorder these if your country does things differently -->
<city>Middletown</city>
<region>NJ</region>
<code>07748</code>
<country>US</country>
</postal>
<phone>+1 732 420 1571</phone>
<email>acmorton@att.com</email>
<!-- uri and facsimile elements may also be added -->
</address>
</author>
<date month="July" year="2010" />
<!-- If the month and year are both specified and are the current ones, xml2rfc will fill
in the current day for you. If only the current year is specified, xml2rfc will fill
in the current day and month for you. If the year is not the current one, it is
necessary to specify at least a month (xml2rfc assumes day="1" if not specified for the
purpose of calculating the expiry date). With drafts it is normally sufficient to
specify just the year. -->
<!-- Meta-data Declarations -->
<area>Operations and Management</area>
<workgroup>PMOL</workgroup>
<!-- WG name at the upperleft corner of the doc,
IETF is fine for individual submissions.
If this element is not present, the default is "Network Working Group",
which is used by the RFC Editor as a nod to the history of the IETF. -->
<keyword>template</keyword>
<!-- Keywords will be incorporated into HTML output
files in a meta tag but they have no effect on text or nroff
output. If you submit your draft to the RFC Editor, the
keywords will be used for the search engine. -->
<note title="">
<t>This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November 10,
2008. The person(s) controlling the copyright in some of this material
may not have granted the IETF Trust the right to allow modifications of
such material outside the IETF Standards Process. Without obtaining an
adequate license from the person(s) controlling the copyright in such
materials, this document may not be modified outside the IETF Standards
Process, and derivative works of it may not be created outside the IETF
Standards Process, except to format it for publication as an RFC or to
translate it into languages other than English.</t>
</note>
<note title="Abstract">
<t>This document defines a set of metrics and their usage to evaluate
the performance of end-to-end Session Initiation Protocol (SIP) for
telephony services in both production and testing environments. The purpose of
this document is to combine a standard set of common metrics, allowing
interoperable performance measurements, easing the comparison of
industry implementations.</t>
</note>
</front>
<middle>
<section title="Introduction and Scope">
<t>SIP has become a widely-used standard among many service providers,
vendors, and end users in the telecommunications industry. Although there
are many different standards for measuring the performance of telephony
signaling protocols, such as SS7, none of the metrics specifically address SIP.</t>
<t>The scope of this document is limited to the definitions of a
standard set of metrics for measuring and reporting SIP performance from
an end-to-end perspective in a telephony environment. The metrics introduce
a common foundation for understanding and quantifying performance expectations
between service providers, vendors, and the users of services based on SIP.
The intended audience for this document can be found among network operators, who
often collect information on the responsiveness of the network to
customer requests for services.</t>
<t>Measurements of the metrics described in this document are affected
by variables external to SIP. The following is a non-exhaustive list of
examples:</t>
<t><list style="symbols">
<t>Network connectivity</t>
<t>Switch and router performance</t>
<t>Server processes and hardware performance</t>
</list></t>
<t>This document defines a list of pertinent metrics for varying aspects of a
telephony environment. They may be used individually or as a set based on the
usage of SIP within the context of a given telecommunications service.</t>
<t>The metrics defined in this document DO NOT take into consideration
the impairment or failure of actual application processing of a request
or response. The metrics do not distinguish application processing time
from other sources of delay, such as packet transfer delay.</t>
<t>Metrics designed to quantify single device application processing
performance are beyond the scope of this document.</t>
<t>This document does not provide any numerical objectives or acceptance
threshold values for the SIP performance metrics defined below, as these
items are beyond the scope of IETF activities, in general.</t>
<t>The metrics defined in this document are applicable in scenarios
where the SIP messages launched (into a network under test) are
dedicated messages for testing purposes, or where the messages are
user-initiated and a portion of the live traffic present. These two
scenarios are sometimes referred to as active and passive measurement,
respectively.</t>
</section>
<section title="Terminology">
<t>The following terms and conventions will be used throughout this
document:</t>
<t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in <xref
target="RFC2119">RFC 2119</xref>.</t>
<t>End-to-End - This is described as two or more elements utilized for
initiating a request, receiving the request, and responding to the
request. It encompasses elements as necessary to be involved in a
session dialog between the originating user agent client (UAC),
destination user agent server (UAS), and any interim proxies (may also
include back-to-back user agent's (B2BUA's)). This may be relative to a
single operator's set of elements or extend to encompass all elements
(if beyond a single operator's network) associated with a session.</t>
<t>Session - As described in <xref target="RFC3261">RFC 3261</xref>, SIP
is used primarily to request, create, and conclude sessions. "These
sessions include Internet telephone calls, multimedia distribution, and
multimedia conferences." The metrics within this document measure the
performance associated with the SIP dialogs necessary to establish these
sessions; therefore, they are titled as: Session Request Delay, Session
Disconnect Delay, etc. Although the titles of many of the metrics
include this term, they are specifically measuring the signaling aspects
only. Each session is identified by a unique Call-ID, "To" and
"From" header field tag.</t>
<t>Session Establishment - Session establishment occurs when a 200 OK
response from the target UA has been received, in response to the originating
UA's INVITE setup request, indicating the session setup request was
successful.</t>
<t>Session Setup - As referenced within the sub-sections of 4.2 in this
document, session setup is the set of messages and included parameters
directly related to the process of a UA requesting to establish a
session with a corresponding UA. This is also described as a set of
steps in order to establish "ringing" <xref
target="RFC3261"></xref>.</t>
</section>
<section title="Time Interval Measurement and Reporting">
<t>Many of the metrics defined in this memo utilize a clock to assess
the time interval between two events. This section defines time- related
terms and reporting requirements.</t>
<t>t1 - start time</t>
<t>This is the time instant (when a request is sent) that begins a
continuous time interval. T1 occurs when the designated request has been
processed by the SIP application and the first bit of the request packet
has been sent from the UA or proxy (and is externally observable at some
logical or physical interface).</t>
<t>T1 represents the time at which each request-response test begins,
and SHALL be used to designate the time-of-day when a particular
measurement was conducted (e.g., The Session Request Delay at "t1" and
(some specific UA interface) was measured to be X ms.)</t>
<t>t4 - end time</t>
<t>This is the time instant that concludes the continuous time interval
begun when the related request is sent. t4 occurs when the last bit of
the designated response is received by the SIP application at the
requesting device (and is externally observable at some logical or
physical interface).</t>
<t>Note: The designations t2 and t3 are reserved for future use at
another interface involved in satisfying a request.</t>
<t>Section 10.1 of <xref target="RFC2330"></xref> describes
time-related issues in measurements, and defines the errors that can be
attributed to the clock themselves. These definitions are used in the
material below.</t>
<t>Time of Day Accuracy</t>
<t>As defined above, t1 is associated with the start of a request and
also serves as the time-of-day stamp associated with a single specific
measurement. The clock offset <xref target="RFC2330"></xref> is the
difference between t1 and a recognized primary source of time, such as
UTC (offset = t1 - UTC).</t>
<t>When measurement results will be correlated with other results or
information using time-of-day stamps, then the time clock that supplies
t1 SHOULD be synchronized to a primary time source, to minimize the
clock's offset. The clocks used at the different measurement points
SHOULD be synchronized to each other, to minimize the relative offset
(as defined in RFC2330). The clock's offset and the relative offset MUST
be reported with each measurement.</t>
<t>Time Interval Accuracy</t>
<t>The accuracy of the t4-t1 interval is also critical to maintain and
report. The difference between a clock's offsets at t1 and t4 is one
source of error for the measurement and is associated with the clock's
skew <xref target="RFC2330"></xref>.</t>
<t>A stable and reasonably accurate clock is needed to make the time
interval measurements required by this memo. This source of error SHOULD
be constrained to less than +/- 1 ms, implying 1 part per 1000 frequency
accuracy for a 1 second interval. This implies greater stability is
required as the length of the t4-t1 increases, in order to constrain the
error to be less than +/- 1ms.</t>
<t>There are several other important aspects of clock operation:</t>
<t><list style="numbers">
<t>Synchronization protocols require some ability to make
adjustments to the local clock. However, these adjustments (clock
steps or slewing) can cause large errors if they occur during the t1
to t4 measurement interval. Clock correction SHOULD be suspended
during a t1 to t4 measurement interval, unless the time interval
accuracy requirement above will be met. Alternatively, a measurement
SHOULD NOT be performed during clock correction, unless the time
interval accuracy requirement above will be met.</t>
<t>If a free-running clock is used to make the time interval
measurement, then the time of day reported with the measurement
(which is normally timestamp t1) SHOULD be derived from a different
clock that meets the time of day accuracy requirements described
above.</t>
</list></t>
<t>The physical operation of reading time from a clock may be
constrained by the delay to service the interrupt. Therefore, if the
accuracy of the time stamp read at t1 or t4 includes the interrupt
delay, this source of error SHOULD be known and included in the error
assessment.</t>
</section>
<section title="SIP Performance Metrics">
<t>In regards to all of the following metrics, t1 begins with the first
associated SIP message sent by either UA, and is not reset if the
UA must retransmit the same message, within the same transaction,
multiple times. The first associated SIP message indicates the t1
associated with the user or application expectation relative to the
request.</t>
<t>Some metrics are calculated using messages from different transactions
in order to measure across actions such as redirection and failure recovery.
The end time is typically based on a successful end-to-end provisional response,
a successful final response, or a failure final response for which there is no
recovery. The individual metrics detail which message to base the end time on.</t>
<t>The authentication method used to establish the SIP dialog will change
the message exchanges. The example message exchanges used do not attempt to
describe all of the various authentication types. Since, authentication is
frequently used, SIP Digest authentication was used for example purposes.
</t>
<t>In regards to all of the metrics, the accuracy and granularity of the
output values are related to the accuracy and granularity of the input
values. Some of the metrics below are defined by a ratio. When the denominator of this
ratio is 0, the metric is undefined. </t>
<t>While these metrics do not specify the sample size. This should be taken
into consideration. These metrics will provide a better indication of performance with
larger sample sets. For example, some SIP Service Providers (SSPs) may choose to collect
input over an hour, daily, weekly or monthly timeframe, while another SSP may choose to
perform metric calculations over a varying set of SIP dialogs.
</t>
<section title="Registration Request Delay (RRD)">
<t>Registration Request Delay (RRD) is a measurement of the delay in
responding to a UA REGISTER request. RRD SHALL be measured and
reported only for successful REGISTER requests, while Ineffective
Registration Attempts (Section 4.2) SHALL be reported for failures.
This metric is measured at the originating UA. The output value of this metric is
numerical and SHOULD be stated in units of milliseconds. The RRD is
calculated using the following formula:</t>
<t>RRD = Time of Final Response - Time of REGISTER Request</t>
<t>In a successful registration attempt, RRD is defined as the time
interval from the first bit of the initial REGISTER message containing
the necessary information is passed by the originating UA to the
intended registrar until the last bit of the 200 OK is received
indicating the registration attempt has completed successfully. This
dialog includes an expected authentication challenge prior to
receiving the 200 OK as described in the following registration flow
examples.</t>
<t>The following message exchange provides an example of identifiable events
necessary for inputs in calculating RRD during a successful
registration completion:</t>
<figure>
<artwork><![CDATA[
UA1 Registrar
| |
|REGISTER |
t1---->|--------------------->|
/\ | 401|
|| |<---------------------|
RRD |REGISTER |
|| |--------------------->|
\/ | 200|
t4---->|<---------------------|
| |
]]></artwork>
</figure>
<t>Note: Networks with elements using primarily Digest authentication will exhibit
different RRD characteristics than networks with elements primarily using
other authentication mechanisms (such as Identity). Operators monitoring
RRD in networks with a mixture of authentications schemes should take note
that the RRD measurements will likely have a multimodal distribution.
</t>
</section>
<section title="Ineffective Registration Attempts (IRA)">
<t>Ineffective registration attempts are utilized to detect failures
or impairments causing an inability for a registrar to receive a UA
REGISTER request. This metric is measured at the originating UA. The output value
of this metric is numerical and SHOULD be reported as a percentage of
registration attempts.</t>
<t>This metric is calculated as a percentage of total REGISTER
requests. The IRA is calculated using the following formula:</t>
<figure>
<artwork><![CDATA[
# of IRA
IRA % = ----------------------------- x 100
Total # of REGISTER Requests
]]></artwork>
</figure>
<t>A failed registration attempt is defined as a final failure
response to the initial REGISTER request. It usually indicates a
failure received from the destination registrar, interim proxies, or
due to a timeout of the REGISTER request at the originating UA. A
failure response is described as a 4XX (excluding 401, 402, and 407
non- failure challenge response codes), 5XX, or possible 6XX message.
A timeout failure is identified by the timer F expiring. IRA may be
used to detect problems in downstream signaling functions, which may
be impairing the REGISTER message from reaching the intended
registrar; or, it may indicate a registrar has become overloaded and
is unable to respond to the request.</t>
<t>The following message exchange provides a timeout example of an identifiable
event necessary for input as a failed registration attempt:</t>
<figure>
<artwork><![CDATA[
UA1 Registrar
| |
|REGISTER |
|--------------------->|
|REGISTER |
|--------------------->|
|REGISTER |
|--------------------->|
| |
Failure ---->|***Timer F Expires |
| |
]]></artwork>
</figure>
<t>In the previous message exchange UA1 retries a REGISTER request
multiple times before the timer, indicating the failure, expires. Only
the first REGISTER request MUST used for input to the calculation and
an IRA. Subsequent REGISTER retries are identified by the same transaction
identifier (same topmost Via header field branch parameter value)
and MUST be ignored for purposes of metric calculation. This ensures
an accurate representation of the metric output.</t>
<t>The following message exchange provides a registrar servicing failure example
of an identifiable event necessary for input as a failed registration
attempt:</t>
<figure>
<artwork><![CDATA[
UA1 Registrar
| |
|REGISTER |
|--------------------->|
| |
| |
| |
| |
| 503|
Failure ---->|<---------------------|
| |
]]></artwork>
</figure>
</section>
<section title="Session Request Delay (SRD)">
<t>Session Request Delay (SRD) is utilized to detect failures or impairments
causing delays in responding to a UA session request. SRD is measured
for both successful and failed session setup requests as this metric
usually relates to a user experience; however, SRD for session
requests ending in a failure MUST NOT be combined in the same result
with successful requests. The duration associated with success and
failure responses will likely vary substantially, and the desired
output time associated with each will be significantly different in
many cases. This metric is similar to Post-Selection Delay defined in <xref
target="E.721"></xref>, and it is measured at the originating UA only. The output
value of this metric MUST indicate whether the output is for
successful or failed session requests and SHOULD be stated in units of
seconds. The SRD is calculated using the following formula:</t>
<t>SRD = Time of Status Indicative Response - Time of INVITE</t>
<section title="Successful Session Setup SRD">
<t>In a successful request attempt, SRD is defined as the time
interval from the first bit of the initial INVITE message containing
the necessary information is sent by the originating user agent
to the intended mediation or destination agent until the last bit of
the first provisional response is received indicating an audible or
visual status of the initial session setup request. (Note: In some
cases, the initial INVITE may be forked. Section 5.4 provides
information for consideration on forking.) In SIP, the
message indicating status would be a non-100 Trying provisional
message received in response to an INVITE request. In some cases, a
non-100 Trying provisional message is not received, but rather a 200
message is received as the first status message instead. In these
situations, the 200 message would be used to calculate the
interval. In most circumstances, this metric relies on receiving a non-100 Trying
message. The use of PRACK <xref target="RFC3262"></xref> MAY improve the
quality and consistency of the results.</t>
<t>The following message exchange provides an example of identifiable events
necessary for inputs in calculating SRD during a successful session
setup request without a redirect (i.e. 3XX message):</t>
<figure>
<artwork><![CDATA[
UA1 UA2
| |
|INVITE |
t1---->|--------------------->|
/\ | |
|| | |
SRD | |
|| | |
\/ | 180|
t4---->|<---------------------|
| |
]]></artwork>
</figure>
<t>The following message exchange provides an example of identifiable events
necessary for inputs in calculating SRD during a successful session
setup with a redirect (e.g. 302 Moved Temporarily):</t>
<figure>
<artwork><![CDATA[
UA1 Redirect Server UA2
| | |
|INVITE | |
t1---->|--------------------->| |
/\ | 302| |
|| |<---------------------| |
|| |ACK | |
SRD |--------------------->| |
|| |INVITE |
|| |------------------------------------------->|
\/ | 180|
t4---->|<-------------------------------------------|
]]></artwork>
</figure>
</section>
<section title="Failed Session Setup SRD">
<t>In a failed request attempt, SRD is defined as the time interval
from the first bit of the initial INVITE message containing the
necessary information sent by the originating agent or user to the
intended mediation or destination agent until the last bit of the
first provisional response or a failure indication response. A
failure response is described as a 4XX (excluding 401, 402, and 407
non-failure challenge response codes), 5XX, or possible 6XX message.
A change in the metric output might indicate problems in downstream signaling
functions, which may be impairing the INVITE message from reaching
the intended UA or may indicate changes in end-point behavior. While
this metric calculates the delay associated with a failed session request,
the metric Ineffective Session Attempts (Section 4.10) is used for calculating
a ratio of session attempt failures.</t>
<t>The following message exchange provides an example of identifiable events
necessary for inputs in calculating SRD during a failed session
setup attempt without a redirect (i.e. 3XX message):</t>
<figure>
<artwork><![CDATA[
UA1 UA2
| |
|INVITE |
t1---->|--------------------->|
/\ | |
|| | |
SRD | |
|| | |
\/ | 480|
t4---->|<---------------------|
| |
]]></artwork>
</figure>
<t>The following message exchange provides an example of identifiable events
necessary for inputs in calculating SRD during a failed session
setup attempt with a redirect (e.g. 302 Moved Temporarily):</t>
<figure>
<artwork><![CDATA[
UA1 Redirect Server UA2
| | |
|INVITE | |
t1---->|--------------------->| |
/\ | 302| |
|| |<---------------------| |
|| |ACK | |
SRD |--------------------->| |
|| |INVITE |
|| |------------------------------------------->|
\/ | 480|
t4---->|<-------------------------------------------|
]]></artwork>
</figure>
</section>
</section>
<section title="Session Disconnect Delay (SDD)">
<t>This metric is utilized to detect failures or impairments delaying
the time necessary to end a session. It can be measured from either end-point UAs
involved in the SIP dialog. SDD is measured for both successful and
failed session completions. The output value of this metric is
numerical and SHOULD be stated in units of milliseconds. The SDD is
calculated using the following formula:</t>
<t>SDD = Time of 2XX or Timeout - Time of Completion Message (BYE)</t>
<t>SDD is defined as the interval between the first bit of the sent session completion
message, such as a BYE, and the last bit of the subsequently
received 2XX response. In some cases a recoverable error response,
such as a 503 retry-after, may be received. In such situations, these responses
should not be used as the end time for this metric calculation. Instead the successful (2XX)
response related to the recovery message is used. The following
message exchanges provide an example of identifiable events necessary for
inputs in calculating SDD during a successful session completion:</t>
<t>Measuring SDD at the originating UA (UA1) -</t>
<figure>
<artwork><![CDATA[
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|
|ACK |
|--------------------->|
|BYE |
t1---->|--------------------->|
/\ | |
|| | |
SDD | |
|| | |
\/ | 200|
t4---->|<---------------------|
]]></artwork>
</figure>
<t>Measuring SDD at the target UA (UA2) -</t>
<figure>
<artwork><![CDATA[
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|
|ACK |
|--------------------->|
| BYE|
|<---------------------|<----t1
| | /\
| | ||
| | SDD
| | ||
|200 | \/
|--------------------->|<----t4
]]></artwork>
</figure>
<t>In some cases, no response is received after a session completion
message is sent and potentially retried. In this case, the
completion message, such as a BYE, results in a Timer F
expiration. Sessions ending in this manner SHOULD be excluded from the
metric calculation.</t>
</section>
<section title="Session Duration Time (SDT)">
<t>This metric is used to detect problems (e.g. poor audio quality)
causing short session durations. SDT is measured for both successful
and failed session completions. It can be measured from either end-point UAs
involved in the SIP dialog. This metric is similar to Call Hold Time, and it is
traditionally calculated as Average Call Hold Time (ACHT) in telephony
applications of SIP. The output value of this metric is numerical and
SHOULD be stated in units of seconds. The SDT is calculated using the
following formula:</t>
<t>SDT = Time of BYE or Timeout - Time of 200 OK response to
INVITE</t>
<t>This metric does not calculate the duration of sessions leveraging early media.
For example, some automated response systems only use early media by responding with a
SIP 183 Session in Progress message with SDP connecting the originating UA with the
the automated message. Usually, in these sessions the originating UA never receives a 200 OK,
and the message exchange ends with the originating UA sending a CANCEL.
</t>
<section title="Successful session duration SDT">
<t>In a successful session completion, SDT is calculated as an
average and is defined as the duration of a dialog defined by the
interval from receipt of the first bit of a 200 OK response to an
INVITE and receipt of the last bit of an associated BYE message
indicating dialog completion. Retransmissions of the 200 OK and ACK messages
due to network impairments do not reset the metric timers.</t>
<t>The following message exchanges provide an example of identifiable events
necessary for inputs in calculating SDT during a successful session
completion (The message message exchanges are changed between the originating
and target UAs to provide varying examples.):</t>
<t>Measuring SDT at the originating UA (UA1) -</t>
<figure>
<artwork><![CDATA[
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
t1---->|<---------------------|
/\ |ACK |
|| |--------------------->|
|| | |
SDT | |
|| | |
|| | |
\/ | BYE|
t4---->|<---------------------|
| |
]]></artwork>
</figure>
<t>When measuring SDT at the target UA (UA2), it is defined by the interval from
sending the first bit of a 200 OK response to an INVITE and receipt
of the last bit of an associated BYE message indicating dialog
completion. If UA2 initiates the BYE, then it is defined by the
interval from sending the first bit of a 200 OK response to an
INVITE and sending the first bit of an associated BYE message
indicating dialog completion. This is illustrated in the following
example message exchange -</t>
<figure>
<artwork><![CDATA[
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|<----t1
|ACK | /\
|--------------------->| ||
| | ||
| | SDT
| | ||
| | ||
| BYE| \/
|<---------------------|<----t4
| |
]]></artwork>
</figure>
<t>(In these two examples, t1 is the same even if either UA
receives the BYE instead of sending it.)</t>
</section>
<section title="Failed session completion SDT">
<t>In some cases, no response is received after a session completion
message is sent and potentially retried. In this case, SDT is
defined as the interval between receiving the first bit of a 200 OK
response to an INVITE, and the resulting Timer F expiration. The
following message exchanges provide an example of identifiable events necessary
for inputs in calculating SDT during a failed session completion
attempt:</t>
<t>Measuring SDT at the originating UA (UA1) -</t>
<figure>
<artwork><![CDATA[
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
t1---->|<---------------------|
/\ |ACK |
|| |--------------------->|
|| |BYE |
SDT |--------------------->|
|| |BYE |
|| |--------------------->|
\/ | |
t4---->|***Timer F Expires |
]]></artwork>
</figure>
<t>When measuring SDT at UA2, SDT is defined as the interval
between sending the first bit of a 200 OK response to an INVITE, and
the resulting Timer F expiration. This is illustrated in the
following example message exchange -</t>
<figure>
<artwork><![CDATA[
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|<----t1
| ACK| /\
|--------------------->| ||
| BYE| ||
|<---------------------| SDT
| BYE| ||
|<---------------------| ||
| | \/
| Timer F Expires***|<----t4
]]></artwork>
</figure>
<t>Note that in the presence of message loss and
retransmission, the value of this metric measured at UA1 may differ from
the value measured at UA2 up to the value of Timer F.
</t>
</section>
</section>
<section title="Session Establishment Ratio (SER)">
<t>This metric is used to detect the ability of a terminating UA or
downstream proxy to successfully establish sessions per new session
INVITE requests. SER is defined as the number of new session INVITE
requests resulting in a 200 OK response, to the total number of
attempted INVITE requests less INVITE requests resulting in a 3XX
response. This metric is similar to Answer Seizure Ratio (ASR) defined
in <xref target="E.411"></xref>. It is
measured at the originating UA only. The output value of this metric is numerical
and SHOULD be adjusted to indicate a percentage of successfully
established sessions. The SER is calculated using the following
formula metric:</t>
<figure>
<artwork><![CDATA[
# of INVITE Requests w/ associated 200
SER = --------------------------------------------------------- x 100
(Total # of INVITE Requests)-(# of INVITE Requests w/ 3XX Response)
]]></artwork>
</figure>
<t>The following message exchange provides an example of identifiable events
necessary for inputs in determining session establishment as described
above:</t>
<figure>
<artwork><![CDATA[
UA1 UA2
| |
|INVITE |
+----------->|------------------>|
| | 180|
| |<------------------|
Session Established | |
| | |
| | 200|
+----------->|<------------------|
| |
]]></artwork>
</figure>
<t>The following is an example message exchange including a SIP 302 Redirect response.
</t>
<figure>
<artwork><![CDATA[
UA1 UA2 UA3
| | |
|INVITE | |
+----------->|------------------>| |
| | | |
INVITE w/ 3XX Response | | |
| | 302| |
+----------->|<------------------| |
| | |
|INVITE |
+----------->|-------------------------------------->|
| | |
| | 180|
Session Established |<--------------------------------------|
| | |
| | 200|
+----------->|<--------------------------------------|
| |
]]></artwork>
</figure>
</section>
<section title="Session Establishment Effectiveness Ratio (SEER)">
<t>This metric is complimentary to SER, but is intended to exclude the
potential effects of an individual user of the target UA from the metric. SER is
defined as the number of INVITE requests resulting in a 200 OK
response and INVITE requests resulting in a 480, 486, 600 or 603; to the
total number of attempted INVITE requests less INVITE requests
resulting in a 3XX response. The response codes 480, 486, 600 and 603 were
chosen, because they clearly indicate the effect of an individual user of the UA.
It is possible an individual user could cause a negative effect on the UA. For example,
they may have misconfigured the UA causing a response code not directly related to a
SSP, but this cannot be easily
determined from an intermediary B2BUA somewhere inbetween the originating and terminating UA.
With this in consideration, response codes such as 401, 407 and 420 (not an exhaustive list)
were not included in the numerator of the metric. This metric is similar to Network Effectiveness
Ratio (NER) defined in <xref target="E.411"></xref>. It is measured at the originating UA only. The
output value of this metric is numerical and SHOULD be adjusted to
indicate a percentage of successfully established sessions less common
UAS failures.</t>
<t>The SEER is calculated using the following formula:</t>
<figure>
<artwork><![CDATA[
# of INVITE Requests w/ associated 200, 480, 486, 600 or 603
SEER = -------------------------------------------------------- x 100
(Total # of INVITE Requests)-(# of INVITE Requests w/ 3XX Response)
]]></artwork>
</figure>
<t>Reference the example flows is Section 4.6.</t>
</section>
<section title="Ineffective Session Attempts (ISA)">
<t>Ineffective session attempts occur when a proxy or agent internally
releases a setup request with a failed or overloaded condition. This
metric is similar to Ineffective Machine Attempts (IMA) in telephony
applications of SIP, and was adopted from Telcordia GR-512-CORE <xref
target="GR-512"></xref>. The output value of this metric is numerical
and SHOULD be adjusted to indicate a percentage of ineffective session
attempts. The following failure responses provide a guideline for this
criterion:</t>
<t><list style="symbols">
<t>408 Request Timeout</t>
<t>500 Server Internal Error</t>
<t>503 Service Unavailable</t>
<t>504 Server Timeout</t>
</list></t>
<t>This set was derived in a similar manner as described in Section
4.9, in addition 408 failure responses may indicate an overloaded
state with a downstream element; however, there are situations other
than overload, which may cause an increase in 408 responses.</t>
<t>This metric is calculated as a percentage of total session setup
requests. The ISA is calculated using the following formula:</t>
<figure>
<artwork><![CDATA[
# of ISA
ISA % = ----------------------------- x 100
Total # of Session Requests
]]></artwork>
</figure>
</section>
<section title="Session Completion Ratio (SCR)">
<t>A session completion is defined as a SIP dialog, which completes
without failing due to a lack of response from an intended proxy or
UA. This metric is only used when at least one proxy is involved in
the dialog. This metric is similar to Call Completion Ratio (CCR) in
telephony applications of SIP. The output value of this metric is
numerical and SHOULD be adjusted to indicate a percentage of
successfully completed sessions.</t>
<t>This metric is calculated as a percentage of total sessions
completed successfully. The SCR is calculated using the following
formula:</t>
<figure>
<artwork><![CDATA[
# of Successfully Completed Sessions
SCR % = --------------------------------------- x 100
Total # of Session Requests
]]></artwork>
</figure>
<section title="Successful Session Completion">
<t>A session completes successfully when it begins with a setup
request and ends with a session completion message.</t>
<t>The following dialog <xref target="RFC3665"></xref> provides an
example describing the necessary message exchanges of a successful session
completion:</t>
<figure>
<artwork><![CDATA[
UA1 Proxy 1 Proxy 2 UA2
| | | |
|INVITE | | |
|--------------->| | |
| 407| | |
|<---------------| | |
|ACK | | |
|--------------->| | |
|INVITE | | |
|--------------->|INVITE | |
| 100|--------------->|INVITE |
|<---------------| 100|--------------->|
| |<---------------| |
| | | 180|
| | 180 |<---------------|
| 180|<---------------| |
|<---------------| | 200|
| | 200|<---------------|
| 200|<---------------| |
|<---------------| | |
|ACK | | |
|--------------->|ACK | |
| |--------------->|ACK |
| | |--------------->|
| Both Way RTP Media |
|<================================================>|
| | | BYE|
| | BYE|<---------------|
| BYE|<---------------| |
|<---------------| | |
|200 | | |
|--------------->|200 | |
| |--------------->|200 |
| | |--------------->|
| | | |
]]></artwork>
</figure>
</section>
<section title="Failed Session Completion">
<t>Session completion fails when an INVITE is sent from the originating UA, but
there is no indication the INVITE reached the target UA. This can
also occur if the target UA does not respond to the INVITE or the
response never reaches the target UA associated with the session.</t>
<t>The following dialog provides an example describing the necessary
message exchanges of an unsuccessful session completion:</t>
<figure>
<artwork><![CDATA[
UA1 Proxy 1 Proxy 2 UA2
| | | |
|INVITE | | |
|--------------->| | |
| 407| | |
|<---------------| | |
|ACK | | |
|--------------->| | |
|INVITE | | |
|--------------->|INVITE | |
| 100|--------------->|INVITE |
|<---------------| 100|--------------->|
| |<---------------| |
| | |INVITE |
| | |--------------->|
| | | |
| | |INVITE |
| | |--------------->|
| | | |
| | 408| |
| 408|<---------------| |
|<---------------|ACK | |
| |--------------->| |
|ACK | | |
|--------------->| | |
]]></artwork>
</figure>
</section>
</section>
</section>
<section title="Additional Considerations">
<section title="Metric Correlations">
<t>These metrics may be used to determine the performance of a domain
and/or user. This would be to provide a metric relative to one or more
dimensions. The following is an example subset of dimensions for providing
further granularity per metric:</t>
<t><list style="symbols">
<t>To "user"</t>
<t>From "user"</t>
<t>Bi-direction "user"</t>
<t>To "domain"</t>
<t>From "domain"</t>
<t>Bi-direction "domain"</t>
</list></t>
</section>
<section title="Back-to-back User Agent (B2BUA)">
<t>A B2BUA may impact the ability to collect these metrics with an
end- to-end perspective. It is necessary to realize a B2BUA may act as
an originating UAC and terminating UAS or it may act as a proxy. In
some cases, it may be necessary to consider information collected from
both sides of the B2BUA in order to determine the end-to-end
perspective. In other cases, the B2BUA may act simply as a proxy
allowing data to be derived as necessary for the input into any of the
listed calculations.</t>
</section>
<section title="Authorization and Authentication">
<t>During the process of setting up a SIP dialog, various
authentication methods may be utilized. These authentication methods
will add to the duration as measured by the metrics, and the length of
time will vary based on those methods. The failures of these
authentication methods will also be captured by these metrics, since
SIP is ultimately used to indicate the success or failure of the
authorization and/or authentication attempt. The metrics in section 3
are inclusive of the duration associated with this process, even if
the method is external to the SIP protocol. This was included
purposefully, due to its inherent impact on the protocol and the
subsequent SIP dialogs.</t>
</section>
<section title="Forking">
<t>Forking SHOULD be considered when determining the messages
associated with the input values for the described metrics. If all of
the forked dialogs were used in the metric calculations, the numbers
would skew dramatically. There are two different points of forking,
which MUST be considered. First, forking may occur at a proxy
downstream from the UA that is being used for metric input values.
The downstream proxy is responsible for forking a message. Then, this proxy
will send provisional (e.g. 180) messages received from the requests and
send the accepted (e.g. 200) response to the UA. Second, in the
cases where the originating UA or proxy is forking the messages, then it
MUST parse the message exchanges necessary for input into the metrics
For example, it MAY utilize the first INVITE or set of INVITE messages sent and the
first accepted 200 OK. Tags will identify this dialog as distinct
from the other 200 OK responses, which are acknowledged and an
immediate BYE is sent. The application responsible for capturing
and/or understanding the input values MUST utilize these tags to
distinguish between dialog requests.</t>
<t>Note that if an INVITE is forked before reaching its destination, multiple
early dialogs are likely and multiple confirmed dialogs are possible (though unlikely).
When this occurs, an SRD measurement should be taken for each dialog that is created (early or confirmed).
</t>
</section>
<section title="Data Collection">
<t>The input necessary for these calculations may be collected in a
number of different manners. It may be collected or retrieved from
call detail records (CDR) or raw signaling information generated by a
proxy or UA. When using records, time synchronization MUST be
considered between applicable elements.</t>
<t>If these metrics are calculated at individual elements (such as proxies or endpoints) instead
of by a centralized management system and the individual elements use different measurement sample
sizes, then the metrics reported for the same event at those elements may differ significantly.
</t>
<t>The information may also be transmitted through the use of network
management protocols like Simple Network Management Protocol (SNMP)
and via future extensions to the SIP Management Information Base (MIB)
modules <xref target="RFC4780"></xref>, or through a potential
undefined new performance metric event package <xref
target="RFC3265"></xref> retrieved via SUBSCRIBE requests.</t>
<t>Data may be collected for a sample of calls or all calls, and may
also be derived from test call scenarios. These metrics are flexible
based on the needs of the application.</t>
<t>For consistency in calculation of the metrics, elements
should expect to reveal event inputs for use by a centralized management
system, which would calculate the metrics based on a varying set sample
size of inputs received from elements compliant with this specification.
</t>
</section>
<section title="Testing Documentation">
<t>In some cases, these metrics will be used to provide output values
to signify the performance level of a specific SIP-based element. When
using these metrics in a test environment, the environment MUST be
accurately documented for the purposes of replicating any output
values in future testing and/or validation.</t>
</section>
</section>
<section title="Conclusions">
<t>This document provides a description of common performance
metrics, and their defined use with SIP. The use of these metrics will
provide a common viewpoint across all vendors, service providers, and
users. These metrics will likely be utilized in production telephony SIP
environments for providing input regarding Key Performance Indicators
(KPI) and Service Level Agreement (SLA) indications; however, they may
also be used for testing end-to-end SIP-based service environments.</t>
</section>
<section anchor="IANA" title="IANA Considerations">
<t>There are no IANA considerations at this time.</t>
</section>
<section anchor="Security" title="Security Considerations">
<t>Security SHOULD be considered in the aspect of securing the relative
data utilized in providing input to the above calculations. All other
aspects of security SHOULD be considered as described in <xref
target="RFC3261">RFC 3261</xref>.</t>
<t>Implementers of these metrics MUST realize that these metrics
could be used to describe characteristics of customer and user usage patterns,
and privacy should be considered when collecting, transporting and storing them.
</t>
</section>
<section anchor="Contributors" title="Contributors">
<t>The following people made substantial contributions to this work:</t>
<figure>
<artwork><![CDATA[
Carol Davids Illinois Institute of Technology
Marian Delkinov Ericsson
Adam Uzelac Global Crossing
Jean-Francois Mule CableLabs
Rich Terpstra Level 3 Communications
]]></artwork>
</figure>
</section>
<section anchor="Acknowledgements" title="Acknowledgements">
<t>We would like to thank Robert Sparks, John Hearty and Dean Bayless for their efforts
in reviewing the draft and providing insight regarding clarification of
certain aspects described throughout the draft. We also thank Dan
Romascanu for his insightful comments and Vijay Gurbani for agreeing to
perform the role of document shepherd.</t>
</section>
</middle>
<back>
<!-- References split into informative and normative -->
<references title="Normative References">
<!--?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.2119.xml"?-->
&RFC2119;
&RFC3261;
&RFC3265;
&RFC3665;
&RFC3326;
&RFC3428;
&RFC4780;
&RFC3262;
</references>
<references title="Informative References">
<!-- Here we use entities that we defined at the beginning. -->
<reference anchor="GR-512">
<front>
<title>LSSGR: Reliability, Section 12</title>
<author>
<organization>Telcordia</organization>
</author>
<date month="January" year="1998" />
</front>
<seriesInfo name="GR-512-CORE" value="Issue 2" />
</reference>
<reference anchor="E.411">
<front>
<title>Series E: Overall Network Operation, Telephone Service,
Service Operation and Human Factors</title>
<author>
<organization>ITU-T</organization>
</author>
<date month="March" year="2000" />
</front>
<seriesInfo name="E.411" value="" />
</reference>
<reference anchor="E.721">
<front>
<title>Series E: Overall Network Operation, Telephone Service,
Service Operation and Human Factors</title>
<author>
<organization>ITU-T</organization>
</author>
<date month="May" year="1999" />
</front>
<seriesInfo name="E.411" value="" />
</reference>
&RFC3398;
&RFC2330;
&RFC5486;
</references>
</back>
</rfc>
| PAFTECH AB 2003-2026 | 2026-04-24 04:39:30 |