One document matched: draft-morton-ippm-delay-var-as-04.xml
<?xml version="1.0" encoding="US-ASCII"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd">
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="3"?>
<?rfc tocindent="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc comments="yes"?>
<?rfc inline="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?>
<rfc category="info" docName="draft-morton-ippm-delay-var-as-04"
ipr="full3978">
<front>
<title abbrev="Delay Variation AS">Packet Delay Variation Applicability
Statement</title>
<author fullname="Al Morton" initials="A." surname="Morton">
<organization>AT&T Labs</organization>
<address>
<postal>
<street>200 Laurel Avenue South</street>
<city>Middletown,</city>
<region>NJ</region>
<code>07748</code>
<country>USA</country>
</postal>
<phone>+1 732 420 1571</phone>
<facsimile>+1 732 368 1192</facsimile>
<email>acmorton@att.com</email>
<uri>http://home.comcast.net/~acmacm/</uri>
</address>
</author>
<author fullname="Benoit Claise" initials="B." surname="Claise">
<organization>Cisco Systems, Inc.</organization>
<address>
<postal>
<street>De Kleetlaan 6a b1</street>
<city>Diegem</city>
<region></region>
<code>1831</code>
<country>Belgium</country>
</postal>
<phone>+32 2 704 5622</phone>
<facsimile></facsimile>
<email>bclaise@cisco.com</email>
<uri></uri>
</address>
</author>
<date day="18" month="November" year="2007" />
<abstract>
<t>Packet delay variation metrics appear in many different standards
documents. The metric definition in RFC 3393 has considerable
flexibility, and it allows multiple formulations of delay variation
through the specification of different packet selection functions.</t>
<t>Although flexibility provides wide coverage and room for new ideas,
it can make comparisons of independent implementations more difficult.
Two different formulations of delay variation have come into wide use in
the context of active measurements. This memo examines a range of
circumstances for active measurements of delay variation and their uses,
and recommends which of the two forms is best matched to particular
conditions and tasks.</t>
</abstract>
<note title="Requirements Language">
<t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in <xref
target="RFC2119">RFC 2119</xref>.</t>
</note>
</front>
<middle>
<section title="Introduction">
<t>There are many ways to formulate packet delay variation metrics for
the Internet and other packet-based networks. The IETF itself has
several specifications for delay variation <xref
target="RFC3393"></xref>, sometimes called jitter <xref
target="RFC3550"></xref> or even inter-arrival jitter <xref
target="RFC3550"></xref>, and these have achieved wide adoption. The
International Telecommunication Union - Telecommunication
Standardization Sector (ITU-T) has also recommended several delay
variation metrics (called parameters in their terminology) <xref
target="Y.1540"></xref> <xref target="G.1020"></xref>, and some of these
are widely cited and used. Most of the standards above specify more than
one way to quantify delay variation, so one can conclude that
standardization efforts have tended to be inclusive rather than
selective.</t>
<t>This memo uses the term "delay variation" for metrics that quantify a
path's ability to transfer packets with consistent delay. <xref
target="RFC3393"></xref> and <xref target="Y.1540"></xref> both prefer
this term. Some refer to this phenomenon as "jitter" (and the buffers
that attempt to smooth the variations as de-jitter buffers).
Applications of the term "jitter" are much broader than packet transfer
performance, with "unwanted signal variation" as a general definition.
"Jitter" has been used to describe frequency or phase variations, such
as data stream rate variations or carrier signal phase noise. The phrase
"delay variation" is almost self-defining and more precise, so it is
preferred in this memo.</t>
<t>Most (if not all) delay variation metrics are derived metrics, in
that their definitions rely on another fundamental metric. In this case,
the fundamental metric is one-way delay, and variation is assessed by
computing the difference between two individual one-way delay
measurements, or a pair of singletons. One of the delay singletons is
taken as a reference, and the result is the variation with respect to
the reference. The variation is usually summarized for all packets in a
stream using statistics.</t>
<t>The industry has predominantly implemented two specific formulations
of delay variation (for one survey of the situation, see<xref
target="Krzanowski"> </xref>):</t>
<t><list style="numbers">
<t>Inter-Packet Delay Variation, IPDV, where the reference is the
previous packet in the stream (according to sending sequence), and
the reference changes for each packet in the stream. Properties of
variation are coupled with packet sequence in this formulation. This
form was called Instantaneous Packet Delay Variation in early IETF
contributions.</t>
<t>Packet Delay Variation, PDV, where a single reference is chosen
from the stream based on specific criteria. The most common
criterion for the reference is the packet with the minimum delay in
the sample. This term derives its name from a similar definition for
Cell Delay Variation, an ATM performance metric.</t>
</list></t>
<t>It is important to note that the authors of relevant standards for
delay variation recognized there are many different users with varying
needs, and allowed sufficient flexibility to formulate several metrics
with different properties. Therefore, the comparison is not so much
between standards bodies or their specifications as it is between
specific formulations of delay variation. Both Inter-Packet Delay
Variation and Packet Delay Variation are compliant with <xref
target="RFC3393"></xref>, because different packet selection functions
will produce either form.</t>
<section title="Background Literature in IPPM and Elsewhere">
<t>With more people joining the measurement community every day, it is
possible this memo is the first from the IP Performance Metrics (IPPM)
Working Group that the reader has consulted. This section provides a
brief roadmap and background on the IPPM literature, and the published
specifications of other relevant standards organizations.</t>
<t>The IPPM framework <xref target="RFC2330"></xref> provides a
background for this memo and other IPPM RFCs. Key terms such as
singleton, sample, and statistic are defined there, along with methods
of collecting samples (Poisson streams), time related issues, and the
"packet of Type-P" convention.</t>
<t>There are two fundamental and related metrics that can be applied
to every packet transfer attempt: one-way loss <xref
target="RFC2680"></xref> and one-way delay <xref
target="RFC2679"></xref>. Lost and delayed packets are separated by a
waiting time threshold. Packets that arrive at the measurement
destination within their waiting time have finite delay and are not
lost. Otherwise, packets are designated lost and their delay is
undefined. Guidance on setting the waiting time threshold may be found
in <xref target="RFC2680"></xref> and <xref
target="I-D.morton-ippm-reporting-metrics"></xref>.</t>
<t>Another fundamental metric is packet reordering as specified in
<xref target="RFC4737"></xref>. The reordering metric was defined to
be "orthogonal" to packet loss. In other words, the gap in a packet
sequence caused by loss does not result in reordered packets, but a
re-arrangement of packet arrivals from their sending order constitutes
reordering.</t>
<t>Derived metrics are based on the fundamental metrics. The metric of
primary interest here is delay variation <xref
target="RFC3393"></xref>, a metric which is derived from one-way delay
<xref target="RFC2680"></xref>. Another derived metric is the loss
patterns metric <xref target="RFC3357"></xref>, which is derived from
loss.</t>
<t>The measured values of all metrics (both fundamental and derived)
depend to great extent on the stream characteristics used to collect
them. Both Poisson streams <xref target="RFC3393"></xref> and Periodic
streams <xref target="RFC3432"></xref> have been used with the IPDV
and PDV metrics. The choice of stream specifications for active
measurement will depend on the purpose of the characterization and the
constraints of the testing environment. Periodic streams are
frequently chosen for use with IPDV and PDV, because the application
streams that are most sensitive to delay variation exhibit
periodicity. Additional details that are method-specific are discussed
the section on Measurement Considerations.</t>
<t>In the ITU-T, the framework, fundamental metrics and derived
metrics for IP performance are specified in Recommendation Y.1540
<xref target="Y.1540"></xref>. <xref target="G.1020"></xref> defines
additional delay variation metrics, analyses the operation of fixed
and adaptive de-jitter buffers, and describes an example adaptive
de-jitter buffer emulator. Appendix II of <xref
target="G.1050"></xref> describes the models for network impairments
(including delay variation) that are part of standardized IP network
emulator which may be useful when evaluating measurement
techniques.</t>
</section>
<section title="Organization of the Memo">
<t>The Purpose and Scope follows in Section 2. We then give a summary
of the main tasks for delay variation metrics in section 3. Section 4
defines the two primary forms of delay variation, and section 5
presents summaries of four earlier comparisons. Section 6 adds new
comparisons to the analysis, and section 7 reviews the applicability
and recommendations for each form of delay variation. Section 8 then
looks at many important delay variation measurement considerations.
Following the IANA and Security Considerations, there are two
Appendices. One presents guidance on reducing delay variation in
networks, and the other calculation of the minimum delay for the PDV
form.</t>
</section>
</section>
<section title="Purpose and Scope">
<t>The IPDV and PDV formulations have certain features that make them
more suitable for one circumstance and less so for another. The purpose
of this memo is to compare two forms of delay variation, so that it will
be evident which of the two is better suited for each of many possible
uses and their related circumstances.</t>
<t>The scope of this memo is limited to the two forms of delay variation
briefly described above (Inter-Packet Delay Variation and Packet Delay
Variation), circumstances related to active measurement, and uses that
are deemed relevant and worthy of inclusion here through IPPM Working
Group consensus.</t>
<t>It is entirely possible that the analysis and conclusions drawn here
are applicable beyond the intended scope, but the reader is cautioned to
fully appreciate the circumstances of active measurement on IP networks
before doing so.</t>
<t>The scope excludes assessment of delay variation for packets with
undefined delay. This is accomplished by conditioning the delay
distribution on arrival within a reasonable waiting time based on an
understanding of the path under test and packet lifetimes. The waiting
time is sometimes called the loss threshold <xref
target="RFC2680"></xref>: if a packet arrives beyond this threshold, it
may as well have been lost because it is no longer useful. This is
consistent with <xref target="RFC3393"></xref>, where the
Type-P-One-way-ipdv is undefined when the destination fails to receive
one or both packets in the selected pair. Furthermore, it is consistent
with application performance analysis to consider only arriving packets,
because a finite waiting time-out is a feature of many protocols.</t>
</section>
<section title="Brief Descriptions of Delay Variation Uses">
<t>This section presents a set of tasks that call for delay variation
measurements. Here, the memo provides several answers to the question,
"How will the results be used?" for the delay variation metric.</t>
<section title="Inferring Queue Occupation on a Path">
<t>As packets travel along the path from source to destination, they
pass through many network elements, including a series of router
queues. Some types of the delay sources along the path are constant,
such as links between two locations. But the latency encountered in
each queue varies, depending on the number of packets in the queue
when a particular packet arrives. If one assumes that at least one of
the packets in a test stream encounters virtually empty queues all
along the path (and the path is stable), then the additional delay
observed on other packets can be attributed to the time spent in one
or more queues. Otherwise, the delay variation observed is the
variation in queue time experienced by the test stream.</t>
</section>
<section title="Determining De-jitter Buffer Size">
<t>Note - while this memo and other IPPM literature prefer the term
delay variation, the terms "jitter buffer" and the more accurate
"de-jitter buffer" are widely adopted names for a component of packet
communication systems, and they will be used here to designate that
system component.</t>
<t>Most Isochronous applications (a.k.a. real-time applications)
employ a buffer to smooth out delay variation encountered on the path
from source to destination. The buffer must be big enough to
accommodate the expected variation of delay, or packet loss will
result. However, if the buffer is too large, then some of the desired
spontaneity of communication will be lost and conversational dynamics
will be affected. Therefore, application designers need to know the
range of delay variation they must accommodate, whether they are
designing fixed or adaptive buffer systems.</t>
<t>Network service providers also attempt to constrain delay variation
to ensure the quality of real-time applications, and monitor this
metric (possibly to compare with a numerical objective or Service
Level Agreement).</t>
<t>De-jitter buffer size can be expressed in units of octets of
storage space for the packet stream, or in units of time that the
packets are stored. It is relatively simple to convert between octets
and time when the buffer read rate (in octets per second) is
constant:</t>
<t>read_rate * storage_time = storage_octets</t>
<t>Units of time are used in the discussion below.</t>
<t>The objective of a de-jitter buffer is to compensate for all prior
sources of delay variation and produce a packet stream with constant
delay. Thus, a packet experiencing the minimum transit delay from
source to destination, D_min, should spend the maximum time in a
de-jitter buffer, B_max. The sum of D_min and B_max should equal the
sum of the maximum transit delay (D_max) and the minimum buffer time
(B_min). We have</t>
<t>Constant = D_min + B_max = D_max + B_min,</t>
<t>after rearranging terms,</t>
<t>B_max - B_min = D_max - D_min = range(B) = range(D)</t>
<t>where range(B) is the range of packet buffering times, and range(D)
is the range of packet transit delays from source to destination.</t>
<t>Packets with transit delay between the max and min spend a
complimentary time in the buffer and also see the constant delay.</t>
<t>In practice, the minimum buffer time, B_min, may not be zero, and
the maximum transit delay, D_max may be a high percentile (99.9%-ile)
instead of the maximum.</t>
<t>Note that B_max - B_min = range(B) is the range of buffering times
needed to compensate for delay variation. The actual size of the
buffer may be larger (where B_min > 0) or smaller than
range(B).</t>
<t>There must be a process to align the de-jitter buffer time with
packet transit delay. This is a process to identify the packets with
minimum delay and schedule their play-out time so that they spend the
maximum time in the buffer. The error in the alignment process can be
accounted for by a factor, A. In the equation below, the range of
buffering times *available* to the packet stream, range(b), depends on
buffer alignment with the actual arrival times of D_min and D_max.</t>
<t>range(b) = b_max - b_min = D_max - D_min + A</t>
<t>When A is positive, the de-jitter buffer applies more delay than
necessary (where Constant = D_max+b_min+A represents one possible
alignment). When A is negative, there is insufficient buffer time
available to compensate for range(D) because of mis-alignment. Packets
with D_min may be arriving too early and encountering a full buffer,
or packets with D_max may be arriving too late, and in either case the
packets would be discarded.</t>
<t>In summary, the range of transit delay variation is a critical
factor in the determination of de-jitter buffer size.</t>
</section>
<section title="Spatial Composition">
<t>In Spatial Composition, the tasks are similar to those described
above, but with the additional complexity of a multiple network path
where several sub-paths are measured separately and no source to
destination measurements are available. In this case, the source to
destination performance must be estimated, using Composed Metrics as
described in <xref target="I-D.ietf-ippm-framework-compagg"></xref>
and <xref target="Y.1541"></xref>. Note that determining the composite
delay variation is not trivial: simply summing the sub-path variations
is not accurate.</t>
</section>
<section title="Service Level Comparison">
<t>IP performance measurements are often used as the basis for
agreements (or contracts) between service providers and their
customers. The measurement results must compare favorably with the
performance levels specified in the agreement.</t>
<t>Packet delay variation is usually one of the metrics specified in
these agreements. In principle, any formulation could be specified in
the Service Level Agreement (SLA). However, the SLA is most useful
when the measured quantities can be related to ways in which the
communication service will be utilized by the customer, and this can
usually be derived from one of the tasks described above.</t>
</section>
<section title="<your favorite here>">
<t>Note: At the IETF-68 IPPM session, Alan Clark suggested another
possible task for DV measurements, that of detecting and somehow
removing the delay variation associated with a smoothing buffer used
with a video codec. Further work is needed to define the problem and
to investigate the applicability of IPDV and PDV.</t>
</section>
</section>
<section title="Formulations of IPDV and PDV">
<t>This section presents the formulations of IPDV and PDV, and provides
some illustrative examples. We use the basic singleton definition in
<xref target="RFC3393"></xref> (which itself is based on <xref
target="RFC2679"></xref>):</t>
<t>"Type-P-One-way-ipdv is defined for two packets from Src to Dst
selected by the selection function F, as the difference between the
value of the Type-P-One-way-delay from Src to Dst at T2 and the value of
the Type-P-One-Way-Delay from Src to Dst at T1."</t>
<section title="IPDV: Inter-Packet Delay Variation">
<t>If we have packets in a stream consecutively numbered i = 1,2,3,...
falling within the test interval, then IPDV(i) = D(i)-D(i-1) where
D(i) denotes the one-way-delay of the ith packet of a stream.</t>
<t>One-way delays are the difference between timestamps applied at the
ends of the path, or the receiver time minus the transmission time. So
D(2) = R2-T2. With this timestamp notation, it can be shown that IPDV
also represents the change in inter-packet spacing between
transmission and reception:</t>
<t>IPDV(2) = D(2) - D(1) = (R2-T2) - (R1-T1) = (R2-R1) - (T2-T1)</t>
<t>An example selection function given in <xref
target="RFC3393"></xref> is "Consecutive Type-P packets within the
specified interval." This is exactly the function needed for IPDV. The
reference packet in the pair is always the previous packet in the
sending sequence.</t>
<t>Note that IPDV can take on positive and negative values (and zero).
One way to analyze the IPDV results is to concentrate on the positive
excursions. However, approach has limitations that are discussed in
more detail below (see section 5.3).</t>
<t>The mean of all IPDV(i) for a stream is usually zero. However, a
slow delay change over the life of the stream, or a frequency error
between the measurement system clocks, can result in a non-zero
mean.</t>
</section>
<section title="PDV: Packet Delay Variation">
<t>The name Packet Delay Variation is used in <xref
target="Y.1540"></xref> and its predecessors, and refers to a
performance parameter equivalent to the metric described below.</t>
<t>The Selection Function for PDV requires two specific roles for the
packets in the pair. The first packet is any Type-P packet within the
specified interval. The second, or reference packet is the Type-P
packet within the specified interval with the minimum
one-way-delay.</t>
<t>Therefore, PDV(i) = D(i)-D(min) (using the nomenclature introduced
in the IPDV section). D(min) is the delay of the packet with the
lowest value for delay (minimum) over the current test interval.
Values of PDV may be zero or positive, and quantiles of the PDV
distribution are direct indications of delay variation.</t>
<t>PDV is a version of the one-way delay distribution, shifted to the
origin by normalizing to the minimum delay.</t>
</section>
<section title="A "Point" about Measurement Points">
<t>Both IPDV and PDV are derived from the one-way delay metric. One
way delay requires knowledge of time at two points, e.g., the source
and destination of an IP network path in end-to-end measurement.
Therefore, both IPDV and PDV can be categorized as 2-point metrics
because they are derived from one-way delay. Specific methods of
measurement may make assumptions or have a priori knowledge about one
of the measurement points, but the metric definitions themselves are
based on information collected at two measurement points.</t>
</section>
<section title="Examples and Initial Comparisons">
<t>Note: This material originally presented in slides 2 and 3 of <xref
target="Morton06"></xref>.</t>
<t>The Figure below gives a sample of packet delays and calculates
IPDV and PDV values and depicts a histogram for each one.</t>
<t><figure anchor="FigA4.3" title="IPDV and PDV Comparison">
<preamble></preamble>
<artwork align="center"><![CDATA[ Packet # 1 2 3 4 5
-------------------------------
Delay, ms 20 10 20 25 20
IPDV U -10 10 5 -5
PDV 10 0 10 15 10
| |
4| 4|
| |
3| 3| H
| | H
2| 2| H
| | H
H H 1| H H 1|H H H
H H | H H |H H H
---------+-------- +---------------
-10 -5 0 5 10 0 5 10 15
IPDV Histogram PDV Histogram
]]></artwork>
<postamble></postamble>
</figure></t>
<t>The sample of packets contains three packets with "typical" delays
of 20ms, one packet with a low delay of 10ms (the minimum of the
sample) and one packet with 25ms delay.</t>
<t>As noted above, this example illustrates that IPDV may take on
positive and negative values, while the PDV values are greater than or
equal to zero. The Histograms of IPDV and PDV are quite different in
general shape, and the ranges are different, too (IPDV range = 20ms,
PDV range = 15 ms). Note that the IPDV histogram will change if the
sequence of delays is modified, but the PDV histogram will stay the
same. PDV normalizes the one-way delay distribution to the minimum
delay and emphasizes the variation independent from the sequence of
delays.</t>
</section>
</section>
<section title="Survey of Earlier Comparisons">
<t>This section summarizes previous work to compare these two forms of
delay variation.</t>
<section title="Demichelis' Comparison ">
<t>In <xref target="Demichelis"></xref>, Demichelis compared the early
draft versions of two forms of delay variation. Although the IPDV form
would eventually see widespread use, the ITU-T work-in-progress he
cited did not utilize the same reference packets as PDV. Demichelis
compared IPDV with the alternatives of using the delay of the first
packet in the stream and the mean delay of the stream as the PDV
reference packet. Neither of these alternative references were used in
practice, and they are now deprecated in favor of the minimum delay of
the stream <xref target="Y.1540"></xref>.</t>
<t>Active measurements of a transcontinental path (Torino to Tokyo)
provided the data for the comparison. The Poisson test stream had
0.764 second average inter-packet interval, with more than 58 thousand
packets over 13.5 hours. Among Demichelis' observations about IPDV are
the following:</t>
<t><list style="numbers">
<t>IPDV is a measure of the network's ability to preserve the
spacing between packets.</t>
<t>The distribution of IPDV is usually symmetrical about the
origin, having a balance of negative and positive values (for the
most part). The mean is usually zero, unless some long-term delay
trend is present.</t>
<t>IPDV singletons distinguish quick delay variations (short-term,
on the order of the interval between packets) from longer term
variations.</t>
<t>IPDV places reduced demands on the stability and skew of
measurement clocks.</t>
</list>He also notes these features of PDV:</t>
<t><list style="numbers">
<t>The PDV distribution does not distinguish short-term variation
from variation over the complete test interval. (Comment: PDV can
be determined over any sub-intervals when the singletons are
stored.)</t>
<t>The location of the distribution is very sensitive to the delay
of the first packet, IF this packet is used as the reference. This
would be a new formulation that differs from the PDV definition in
this memo (PDV references the packet with minimum delay, so it
does not have this drawback).</t>
<t>The shape of the PDV distribution is identical to the delay
distribution, but shifted by the reference delay.</t>
<t>Use of a common reference over measurement intervals that are
longer than a typical session length may indicate more PDV than
would be experienced by streams that support such sessions.
(Ideally, the measurement interval should be aligned with the
session length of interest, and this influences determination of
the reference delay, D(min).)</t>
<t>The PDV distribution characterizes the range of queue
occupancies along the measurement path (assuming the path is
fixed), but the range says nothing about how the variation took
place.</t>
</list>The summary metrics used in this comparison were the number
of values exceeding a +/-50ms range around the mean, the Inverse
Percentiles, and the Inter-Quartile Range.</t>
</section>
<section title="Ciavattone et al.">
<t>In <xref target="Cia03"></xref>, the authors compared IPDV and PDV
(referred to as delta) using a periodic packet stream conforming to
<xref target="RFC3432"></xref> with inter-packet interval of 20
ms.</t>
<t>One of the comparisons between IPDV and PDV involves a laboratory
set-up where a queue was temporarily congested by a competing packet
burst. The additional queuing delay was 85ms to 95ms, much larger than
the inter-packet interval. The first packet in the stream that follows
the competing burst spends the longest time queued, and others
experience less and less queuing time until the queue is drained.</t>
<t>The authors observed that PDV reflects the additional queuing time
of the packets affected by the burst, with values of 85, 65, 45, 25,
and 5ms. Also, it is easy to determine (by looking at the PDV range)
that a de-jitter buffer of >85 ms would have been sufficient to
accommodate the delay variation. Again, the measurement interval is a
key factor in the validity of such observations (it should have
similar length to the session interval of interest).</t>
<t>The IPDV values in the congested queue example are very different:
85, -20, -20, -20, -20, -5ms. Only the positive excursion of IPDV
gives an indication of the de-jitter buffer size needed. Although the
variation exceeds the inter-packet interval, the extent of negative
IPDV values is limited by that sending interval. This preference for
information from the positive IPDV values has prompted some to ignore
the negative values, or to take the absolute value of each IPDV
measurement (sacrificing key properties of IPDV in the process, such
as its ability to distinguish delay trends).</t>
<t>Note that this example illustrates a case where the IPDV
distribution is asymmetrical, because the delay variation range (85ms)
exceeds the inter-packet spacing (20ms). We see that the IPDV values
85, -20, -20, -20, -20, -5ms have zero mean, but the left side of the
distribution is truncated at -20ms.</t>
<t>Elsewhere, the authors considered the range as a summary statistic
for IPDV, and the 99.9%-ile minus the minimum delay as a summary
statistic for delay variation, or PDV.</t>
</section>
<section title="IPPM List Discussion from 2000">
<t>Mike Pierce made many comments in the context of the 05 version of
draft-ietf-ippm-ipdv. One of his main points was that a delay
histogram is a useful approach to quantifying variation. Another point
was that the time duration of evaluation is a critical aspect.</t>
<t>Carlo Demichelis then mailed his comparison paper to the IPPM list,
<xref target="Demichelis"></xref> as discussed in more detail
above.</t>
<t>Ruediger Geib observed that both IPDV and the delay histogram (PDV)
are useful, and suggested that they might be applied to different
variation time scales. He pointed out that loss has a significant
effect on IPDV, and encouraged that the loss information be retained
in the arrival sequence.</t>
<t>Several example delay variation scenarios were discussed,
including:</t>
<t><figure anchor="Fig0" title="Delay Examples">
<preamble></preamble>
<artwork align="center"><![CDATA[Packet # 1 2 3 4 5 6 7 8 9 10 11
-------------------------------------------------------
Ex. A
Lost
Delay, ms 100 110 120 130 140 150 140 130 120 110 100
IPDV U 10 10 10 10 10 -10 -10 -10 -10 -10
PDV 0 10 20 30 40 50 40 30 20 10 0
-------------------------------------------------------
Ex. B
Lost L
Delay, ms 100 110 150 U 120 100 110 150 130 120 100
IPDV U 10 40 U U -10 10 40 -20 -10 -20
PDV 0 10 50 U 20 0 10 50 30 20 0]]></artwork>
<postamble></postamble>
</figure></t>
<t>Clearly, the range of PDV values is 50 ms in both cases above, and
this is the statistic that determines the size of a de-jitter buffer.
The IPDV range is minimal in response to the smooth variation in
Example A (20 ms). However, IPDV responds to the faster variations in
Example B (60 ms range from 40 to -20). Here the IPDV range is larger
than the PDV range, and over-estimates the buffer size
requirements.</t>
<t>A heuristic method to estimate buffer size using IPDV is to sum the
consecutive positive or zero values as an estimate of PDV range.
However, this is more complicated to assess than the PDV range, and
has strong dependence on the actual sequence of IPDV values (any
negative IPDV value stops the summation, and again causes an
underestimate).</t>
<t>IPDV values can be viewed as the adjustments that an adaptive
de-jitter buffer would make, IF it could make adjustments on a
packet-by-packet basis. However, adaptive de-jitter buffers don't make
adjustments this frequently, so the value of this information is
unknown. The short-term variations may be useful to know in some other
cases.</t>
</section>
<section title="Y.1540 Appendix II">
<t>Appendix II of <xref target="Y.1540"></xref> describes a secondary
terminology for delay variation. It compares IPDV, PDV (referred to as
2-point PDV), and 1-point packet delay variation (which assumes a
periodic stream and assesses variation against an ideal arrival
schedule constructed at a single measurement point). This early
comparison discusses some of the same considerations raised in section
6 below.</t>
</section>
<section title="Clark's ITU-T SG 12 Contribution">
<t>Alan Clark's contribution to ITU-T Study Group 12 in January 2003,
provided an analysis of the root causes of delay variation and
investigated different techniques for measurement and modeling of
"jitter" <xref target="COM12.D98"></xref>. Clark compared a metric
closely related to IPDV, Mean Packet-to-Packet Delay Variation, MPPDV
= mean(abs(D(i)-D(i-1))) to the newly proposed Mean Absolute Packet
Delay Variation (MAPDV2, see <xref target="G.1020"></xref>). One of
the tasks for this study was to estimate the number of packet discards
in a de-jitter buffer. Clark concluded that MPPDV did not track the
ramp delay variation he associated access link congestion (similar to
Figure 2, Example A above), but MAPDV2 did.</t>
<t>Clark also briefly looked at PDV (as described in the 2002 version
of<xref target="Y.1541"></xref>). He concluded that if PDV was applied
to a series of very short measurement intervals (e.g., 200ms), it
could be used to determine the fraction of intervals with high packet
discard rates.</t>
</section>
</section>
<section title="Additional Properties and Comparisons">
<t>This section treats some of the earlier comparison areas in more
detail, and introduces new areas for comparison.</t>
<section title="Packet Loss">
<t>The measurement packet loss is of great influence for the delay
variation results, as displayed in the figures 3 and 4 (L means Lost
and U means undefined). Figure 3 shows that in the extreme case of
every other packet loss, the IPDV doesn't produce any results, while
the PDV produces results for all arriving packets.</t>
<t><figure anchor="Fig3" title="Path Loss Every Other Packet">
<preamble></preamble>
<artwork align="center"><![CDATA[Packet # 1 2 3 4 5 6 7 8 9 10
Lost L L L L L
---------------------------------------
Delay, ms 3 U 5 U 4 U 3 U 4 U
IPDV U U U U U U U U U U
PDV 0 U 2 U 1 U 0 U 1 U]]></artwork>
<postamble></postamble>
</figure></t>
<t>In case of a burst of packet loss, as displayed in figure 3, both
the IPDV and PDV produces some results. Note that PDV still produces
more values than IPDV.</t>
<t><figure anchor="Fig4" title="Burst of Packet Loss">
<preamble></preamble>
<artwork align="center"><![CDATA[Packet # 1 2 3 4 5 6 7 8 9 10
Lost L L L L L
---------------------------------------
Delay, ms 3 4 U U U U U 5 4 3
IPDV U 1 U U U U U U -1 -1
PDV 0 1 U U U U U 2 1 0]]></artwork>
<postamble></postamble>
</figure></t>
<t>In conclusion, the PDV results are affected by the packet loss
ratio. The IPDV results are affected by both the packet loss ratio and
the packet loss distribution. In the extreme case of loss of every
other packet, IPDV doesn't provide any results.</t>
</section>
<section title="Path Changes">
<t>When there is little or no stability in the network under test,
then the devices that attempt to characterize the network are equally
stressed, especially if the results displayed are used to make
inferences which may not be valid.</t>
<t>Sometimes the path characteristics change during a measurement
interval. The change may be due to link or router failure,
administrative changes prior to maintenance (e.g., link cost change),
or re-optimization of routing using new information. All these causes
are usually infrequent, and network providers take appropriate
measures to ensure this. Automatic restoration to a back-up path is
seen as a desirable feature of IP networks.</t>
<t>Frequent path changes and prolonged congestion with substantial
packet loss clearly make delay variation measurements challenging.
Path changes are usually accompanied by a sudden, persistent increase
or decrease in one-way-delay. <xref target="Cia03"></xref> gives one
such example. We assume that a restoration path either accepts a
stream of packets, or is not used for that particular stream (e.g., no
multi-path for flows).</t>
<t>In any case, a change in the TTL (or Hop Limit) of the received
packets indicates that the path is no longer the same. Transient
packet reordering may also be observed with path changes, due to use
of non-optimal routing while updates propagate through the network
(see <xref target="Casner"></xref> and <xref target="Cia03"></xref>
)</t>
<t>Many, if not all, packet streams experience packet loss in
conjunction with a path change. However, it is certainly possible that
the active measurement stream does not experience loss. This may be
due to use of a long inter-packet sending interval with respect to the
restoration time, and it becomes more likely as "fast restoration"
techniques see wider deployment (e.g., <xref
target="RFC4090"></xref>.</t>
<t>Thus, there are two main cases to consider, path changes
accompanied by loss, and those that are lossless from the point of
view of the active measurement stream. The subsections below examine
each of these cases.</t>
<section title="Lossless Path Change">
<t>In the lossless case, a path change will typically affect only
one IPDV singleton. For example, the delay sequence below always
produces IPDV=0 except in the one case where the value is 5:</t>
<t>(...10, 10, 10, 10, 15, 15, 15, ...)</t>
<t>produces IPDV singletons</t>
<t>(..., 0, 0, 0, 5, 0, 0, ...).</t>
<t>However, if the change in delay is negative and larger than the
inter-packet sending interval, then more than one IPDV singleton may
be affected because packet reordering is also likely to occur.</t>
<t>The use of the new path and its delay variation can be quantified
by treating the PDV distribution as bi-modal, and characterizing
each mode separately. This would involve declaring a new path within
the sample, and using a new local minimum delay as the PDV reference
delay for the sub-sample (or time interval) where the new path is
present.</t>
<t>The process of detecting a bi-modal delay distribution is made
difficult if the typical delay variation is larger than the delay
change associated with the new path. However, information on TTL (or
Hop Limit) change or the presence of transient reordering can assist
in an automated decision.</t>
<t>The effect of path changes may also be reduced by making PDV
measurements over short intervals (minutes, as opposed to hours).
This way, a path change will affect one sample and its PDV values.
Assuming that the mean or median one-way-delay changes appreciably
on the new path, then subsequent measurements can confirm a path
change and trigger special processing on the interval to revise the
PDV result.</t>
<t>Alternatively, if the path change is detected, by monitoring the
test packets TTL or Hop Limit, or monitoring the change in the IGP
link-state database, the results of measurement before and after the
path change could be kept separated, presenting two different
distributions. This avoids the difficult task of determining the
different modes of a multi-modal distribution.</t>
</section>
<section title="Path Change with Loss">
<t>If the path change is accompanied by loss, such that the are no
consecutive packet pairs that span the change, then no IPDV
singletons will reflect the change. This may or may not be
desirable, depending on the ultimate use of the delay variation
measurement. The Figure 5, in which L means Lost and U means
undefined, illustrates this case.</t>
<t><figure anchor="Fig5" title="Path Change with Loss">
<preamble></preamble>
<artwork align="center"><![CDATA[Packet # 1 2 3 4 5 6 7 8 9
Lost L L
------------------------------------
Delay, ms 3 4 3 3 U U 8 9 8
IPDV U 1 -1 0 U U U 1 -1
PDV 0 1 0 0 U U 5 6 5]]></artwork>
<postamble></postamble>
</figure></t>
<t>PDV will again produce a bimodal distribution. But here, the
decision process to define sub-intervals associated with each path
is further assisted by the presence of loss, in addition to TTL,
reordering information, and use of short measurement intervals
consistent with the duration of user sessions. It is reasonable to
assume that at least loss and delay will be measured simultaneously
with PDV and/or IPDV.</t>
</section>
<t></t>
</section>
<section title="Clock Stability and Error">
<t>Low cost or low complexity measurement systems may be embedded in
communication devices that do not have access to high stability
clocks, and time errors will almost certainly be present. However,
larger time-related errors (~1ms) may offer an acceptable trade-off
for monitoring performance over a large population (the accuracy
needed to detect problems may be much less than required for a
scientific study, ~0.01ms for example).</t>
<t>Maintaining time accuracy <<1ms has typically required access
to dedicated time receivers at all measurement points. Global
positioning system (GPS) receivers have often been installed to
support measurements. The GPS installation conditions are fairly
restrictive, and many prospective measurement efforts have found the
deployment complexity and system maintenance too difficult.</t>
<t>As mentioned above, <xref target="Demichelis"></xref> observed that
PDV places greater demands on clock synchronization than for IPDV.
This observation deserves more discussion. Synchronization errors have
two components: time of day errors and clock frequency errors
(resulting in skew).</t>
<t>Both IPDV and PDV are sensitive to time-of-day errors when
attempting to align measurement intervals at the source and
destination. Gross mis-alignment of the measurement intervals can lead
to lost packets, for example if the receiver is not ready when the
first test packet arrives. However, both IPDV and PDV assess delay
differences, so the error present in two one-way-delay singletons will
cancel as long as it is constant. So, NTP or GPS synchronization is
not required to correct the time-of-day error in case the delay
variation measurement, while it is required for the one-way delay
measurement.</t>
<t>Skew is a measure of the change in clock time over an interval
w.r.t. a reference clock. Both IPDV and PDV are affected by skew, but
the error sensitivity in IPDV singletons is less because the intervals
between consecutive packets are rather small, especially when compared
to the overall measurement interval. Since PDV computes the difference
between a single reference delay (the sample minimum) and all other
delays in the measurement interval, the constraint on skew error is
greater to attain the same accuracy as IPDV. Again, use of short PDV
measurement intervals (on the order of minutes, not hours) provides
some relief from the effects of skew error.</t>
<t>If skew is present in a sample of one-way-delays, its symptom is
typically a linear growth or decline over all the one-way-delay
values. As a practical matter, if the same slope appears consistently
in the measurements, then it may be possible to fit the slope and
compensate for the skew in the one-way-delay measurements, thereby
avoiding the issue in the PDV calculations that follow. See <xref
target="RFC3393"></xref> for additional information on compensating
for skew.</t>
<t>Values for IPDV may have non-zero mean over a sample when clock
skew is present, and this tends to complicate IPDV analysis using the
assumptions of zero mean and symmetric distribution. A requirement of
zero mean with IPDV raises the clock skew-derived error requirement to
the same order as for PDV, because skew must be constrained over the
entire measurement interval to ensure a zero IPDV mean.</t>
<t>There is a third factor related to clock error and stability: this
is the presence of a clock synchronization protocol (e.g., NTP) and
the time adjustment operations that result. When a time error is
detected (typically on the order of a few milliseconds), the host
clock frequency is continuously adjusted to reduce the time error. If
these adjustments take place during a measurement interval, they may
appear as delay variation when none was present, and therefore are a
source of error (regardless of the DV form considered).</t>
</section>
<section title="Spatial Composition">
<t>ITU-T Recommendation <xref target="Y.1541"></xref> gives a
provisional method to compose a PDV metric using PDV measurement
results from two or more sub-paths. Additional methods are considered
in <xref target="I-D.ietf-ippm-spatial-composition"></xref>.</t>
<t>PDV has a clear advantage at this time, since there is no validated
method to compose an IPDV metric. In addition, IPDV results depend
greatly on the exact sequence of packets and may not lend themselves
easily to the composition problem.</t>
</section>
<section title="Reporting a Single Number (SLA)">
<t>Despite the risk of over-summarization, measurements must often be
displayed for easy consumption. If the right summary report is
prepared, then the "dashboard" view correctly indicates whether there
is something different and worth investigating further, or that the
status has not changed. The dashboard model restricts every instrument
display to a single number. The packet network dashboard could have
different instruments for loss, delay, delay variation, reordering,
etc., and each must be summarized as a single number for each
measurement interval. The single number summary statistic is a key
component of SLAs, where a threshold on that number must be met x% of
the time.</t>
<t>The simplicity of the PDV distribution lends itself to this
summarization process (including use of the percentiles, median or
mean). An SLA of the form "no more than x% of packets in a measurement
interval shall have PDV >= y ms, for no less than z% of time" is
relatively straightforward to specify and implement. <xref
target="Y.1541"></xref> introduced the notion of a pseudo-range when
setting an objective for the 99.9%-ile of PDV. The conventional range
(max-min) was avoided for several reasons, including stability of the
maximum delay. The 99.9%-ile of PDV is helpful to performance planners
(seeking to meet some user-to-user objective for delay) and in design
of de-jitter buffer sizes, even those with adaptive capabilities.</t>
<t>IPDV does not lend itself to summarization so easily. The mean IPDV
is typically zero. As the IPDV distribution will have two tails
(positive and negative) the range or pseudo-range would not match the
needed de-jitter buffer size. Additional complexity may be introduced
when the variation exceeds the inter-packet sending interval, as
discussed above. Should the Inter-Quartile Range be used? Should the
singletons beyond some threshold be counted (e.g., mean +/- 50ms)? A
strong rationale for one of these summary statistics has yet to
emerge.</t>
<t>When summarizing IPDV, some prefer the simplicity of the
single-sided distribution created by taking the absolute value of each
singleton result, abs(D(i)-D(i-1)). This approach sacrifices the
two-sided inter-arrival spread information in the distribution. It
also makes the evaluation using percentiles more confusing, because a
single late packet that exceeds the variation threshold will cause two
singleton measurement pairs to fail the criteria (one positive, the
other negative converted to positive). The single-sided PDV
distribution is an advantage in this category.</t>
</section>
<section title="Jitter in RTCP Reports">
<t><xref target="RFC3550"></xref> gives the calculation of the
inter-arrival Jitter field for the RTCP report, with a sample
implementation in an Appendix.</t>
<t>The RTCP Jitter value can be calculated using IPDV singletons. If
there is packet reordering, as defined in <xref
target="RFC4737"></xref>, then estimates of Jitter based on IPDV may
vary slightly, because <xref target="RFC3550"></xref> specifies the
use of receive packet order.</t>
<t>Just as there is no simple way to convert PDV singletons to IPDV
singletons without returning to the original sample of delay
singletons, there is no clear relationship between PDV and <xref
target="RFC3550"></xref> Jitter.</t>
</section>
<section title="MAPDV2">
<t>MAPDV2 stands for Mean Absolute Packet Delay Variation (version) 2,
and is specified in <xref target="G.1020"></xref>. The MAPDV2
algorithm computes a smoothed running estimate of the mean delay using
the one-way delays of 16 previous packets. It compares the current
one-way-delay to the estimated mean, separately computes the means of
positive and negative deviations, and sums these deviation means to
produce MAPVDV2. In effect, there is a MAPDV2 singleton for every
arriving packet, so further summarization is usually warranted.</t>
<t>Neither IPDV or PDV forms assist in the computation of MAPDV2.</t>
</section>
<section title="Load Balancing">
<t>Network traffic load balancing is a process to divide packet
traffic in order to provide a more even distribution over two or more
equally viable paths. The paths chosen are based on the IGP cost
metrics, while the delay depends on the path's physical layout.
Usually, the balancing process is performed on a per-flow basis to
avoid delay variation experienced when packets traverse different
physical paths.</t>
<t>If the sample includes test packets with different characteristics
such as IP addresses/ports, there could be multi-modal delay
distributions present. The PDV form makes the identification of
multiple modes possible. IPDV may also reveal that multiple paths are
in use with a mixed flow sample, but the different delay modes are not
easily divided and analyzed separately.</t>
<t>Should the delay singletons using multiple addresses/ports be
combined in the same sample? Should we characterize each mode
separately? (This question also applies to the Path Change case.) It
depends on the task to be addressed by the measurement.</t>
<t>For the task of de-jitter buffer sizing or assessing queue
occupation, the modes should be characterized separately because flows
will experience only one mode on a stable path. Use of a single flow
description (address/port combination) in each sample simplifies this
analysis. Multiple modes may be identified by collecting samples with
different flow attributes, and characterization of multiple paths can
proceed with comparison of the delay distributions from each
sample.</t>
<t>For the task of capacity planning and routing optimization,
characterizing the modes separately could offer an advantage. Network
wide capacity planning (as opposed to link capacity planning) takes as
input the core traffic matrix, which corresponds to a matrix of
traffic transferred from every source to every destination in the
network. Applying the core traffic matrix along with the routing
information (typically the link state database of a routing protocol)
in a capacity planning tool offers the possibility to visualize the
paths where the traffic flows and to optimize the routing based on the
link utilization. In the case where equal cost multiple paths (ECMP)
are used, the traffic will be load balanced onto multiple paths. If
each mode of the IP delay multi-modal distribution can be associated
with a specific path, the delay performance offers an extra
optimization parameter, i.e. the routing optimization based on the IP
delay variation metric. As an example, the load balancing across ECMPs
could be suppressed so that the VoIP calls would only be routed via
the path with the lower IP delay variation. Clearly, any modifications
can result in new delay performance measurements, so there must be a
verification step to ensure the desired outcome.</t>
</section>
</section>
<section title="Applicability of the Delay Variation Forms and Recommendations">
<t>Based on the comparisons of IPDV and PDV presented above, this
section matches the attributes of each form with the tasks described
earlier. We discuss the more general circumstances first.</t>
<section title="Uses">
<t></t>
<section title="Inferring Queue Occupancy">
<t>The PDV distribution is anchored at the minimum delay observed in
the measurement interval. When the sample minimum coincides with the
true minimum delay of the path, then the PDV distribution is
equivalent to the queuing time distribution experienced by the test
stream. If the minimum delay is not the true minimum, then the PDV
distribution captures the variation in queuing time and some
additional amount of queuing time is experienced, but unknown. One
can summarize the PDV distribution with the mean, median, and other
statistics.</t>
<t>IPDV can capture the difference in queuing time from one packet
to the next, but this is a different distribution from the queue
occupancy revealed by PDV.</t>
</section>
<section title="Determining De-jitter Buffer Size">
<t>This task is complimentary to the problem of inferring queue
occupancy through measurement. Again, use of the sample minimum as
the reference delay for PDV yields a distribution that is very
relevant to de-jitter buffer size. This is because the minimum delay
is an alignment point for the smoothing operation of de-jitter
buffers. A de-jitter buffer that is ideally aligned with the delay
variation adds zero buffer time to packets with the longest
accommodated network delay (any packets with longer delays are
discarded). Thus, a packet experiencing minimum network delay should
be aligned to wait the maximum length of the de-jitter buffer. With
this alignment, the stream is smoothed with no unnecessary delay
added. <xref target="G.1020"></xref> illustrates the ideal
relationship between network delay variation and buffer time.</t>
<t>The PDV distribution is also useful for this task, but different
statistics are preferred. The range (max-min) or the 99.9%-ile of
PDV (pseudo-range) are closely related to the buffer size needed to
accommodate the observed network delay variation.</t>
<t>In some cases, the positive excursions (or series of positive
excursions) of IPDV may help to approximate the de-jitter buffer
size, but there is no guarantee that a good buffer estimate will
emerge, especially when the delay varies as a positive trend over
several test packets.</t>
</section>
<section title="Spatial Composition">
<t>PDV has a clear advantage at this time, since there is no
validated method to compose an IPDV metric.</t>
</section>
<section title="Service Level Specification: Reporting a Single Number">
<t>The one-sided PDV distribution can be constrained with a single
statistic, such as an upper percentile, so it is preferred. The IPDV
distribution is two-sided, usually has zero mean, and no universal
summary statistic that relates to a physical quantity has emerged in
years of experience.</t>
</section>
</section>
<section title="Challenging Circumstances">
<t>Note that measurement of delay variation may not be the primary
concern under unstable and unreliable circumstances.</t>
<section title="Clock and Storage Issues">
<t>When appreciable skew is present between measurement system
clocks, then IPDV has an advantage because PDV would require
processing over the entire sample to remove the skew error. However,
significant skew can invalidate IPDV analysis assumptions, such as
the zero mean and symmetric distribution characteristics. Small skew
may well be within the error tolerance, and both PDV and IPDV
results will be usable. There may be a portion of the skew,
measurement interval, and required accuracy 3-D space where IPDV has
an advantage, depending on the specific measurement
specifications.</t>
<t>Neither form of delay variation is more suited than the other to
on-the-fly summarization without memory, and this may be one of the
reasons that <xref target="RFC3550"></xref> RTCP Jitter and MAPDV2
in <xref target="G.1020"></xref> have attained deployment in
low-cost systems.</t>
</section>
<section title="Frequent Path Changes">
<t>If the network under test exhibits frequent path changes, on the
order of several new routes per minute, then IPDV appears to isolate
the delay variation on each path from the transient effect of path
change (especially if there is packet loss at the time of path
change). However, if one intends to use IPDV to indicate path
changes, it cannot do this when the change is accompanied by loss.
It is possible to make meaningful PDV measurements when paths are
unstable, but great importance would be placed on the algorithms
that infer path change and attempt to divide the sample on path
change boundaries.</t>
<t>When path changes are frequent and cause packet loss, delay
variation is probably less important than the loss episodes and
attention should be turned to the loss metric instead.</t>
</section>
<section title="Frequent Loss">
<t>If the network under test exhibits frequent loss, then PDV may
produce a larger set of singletons for the sample than IPDV. This is
due to IPDV requiring consecutive packet arrivals to assess delay
variation, compared to PDV where any packet arrival is useful. The
worst case is when no consecutive packets arrive, and the entire
IPDV sample would be undefined. PDV would successfully produce a
sample based on the arriving packets.</t>
</section>
<section title="Load Balancing">
<t>PDV distributions offer the most straightforward way to identify
that a sample of packets have traversed multiple paths. The tasks of
de-jitter buffer sizing or assessing queue occupation with PDV
should be use a sample with a single flow because flows will
experience only one mode on a stable path, and it simplifies the
analysis.</t>
</section>
</section>
<section title="Summary">
<t></t>
<texttable title="Summary of Comparisons">
<preamble></preamble>
<ttcol align="left">Comparison Area</ttcol>
<ttcol align="left">PDV</ttcol>
<ttcol align="left">IPDV</ttcol>
<c>Challenging Circumstances</c>
<c>Less sensitive to packet loss, and simplifies analysis when Load
balancing or multiple paths are present</c>
<c>Preferred when path changes are frequent or when measurement
clocks exhibit some skew</c>
<c>Spatial Composition of DV metric</c>
<c>All validated methods use this form</c>
<c>Has sensitivity to sequence and spacing changes, which tend to
break the segment IID requirement</c>
<c>Determine De-Jitter Buffer Size Required</c>
<c>"Pseudo-range" reveals this property by anchoring the
distribution at the minimum delay</c>
<c>No reliable relationship, but some heuristics</c>
<c>Estimate of Queuing Time and Variation</c>
<c>Distribution has one-to-one relationship on a stable path,
especially when sample min = true min</c>
<c>No reliable relationship</c>
<c>Specification Simplicity: Single Number SLS</c>
<c>One constraint needed for single-sided distribution, and easily
related to quantities above</c>
<c>Distribution is two-sided, usually has zero mean, and no
universal summary statistic that relates to a physical quantity</c>
<postamble></postamble>
</texttable>
</section>
</section>
<section title="Measurement Considerations">
<t>TO DO: Add info comparing methodological approximations for each
form, including on-the-fly statistics, memory requirements, implications
on the reference value (D(min)), quantiles not available as a running
measure, (possibly in a new subsection)</t>
<section title="Measurement Stream Characteristics">
<t>As stated in the background section, there is a strong dependency
between the active measurement stream characteristics and the results.
The IPPM literature includes two primary methods for collecting
samples: Poisson sampling described in <xref target="RFC2330"></xref>,
and Periodic sampling in<xref target="RFC3432"> </xref>. The Poisson
method was intended to collect an unbiased sample of performance,
while the Periodic method addresses a "known bias of interest".
Periodic streams are required to have random start times and limited
stream duration, in order to avoid unwanted synchronization with some
other periodic process, or cause congestion-aware senders to
synchronize with the stream and produce atypical results. The random
start time should be different for each new stream.</t>
<t>It is worth noting that <xref target="RFC3393"></xref> was
developed in parallel with <xref target="RFC3432"></xref>. As a
result, all the stream metrics defined in <xref
target="RFC3393"></xref> specify the Poisson sampling method.</t>
<t>Periodic sampling is frequently used in measurements of delay
variation. Several factors foster this choice:</t>
<t><list style="numbers">
<t>Many application streams that are sensitive to delay variation
also exhibit periodicity, and so exemplify the bias of interest.
If the application has a constant packet spacing, this constant
spacing can be the inter-packet gap for the test stream. VoIP
streams often use 20ms spacing, so this is an obvious choice for
an Active stream. This applies to both IPDV and PDV forms.</t>
<t>The spacing between packets in the stream will influence
whether the stream experiences short-range dependency, or only
long-range dependency, as investigated in <xref
target="Li.Mills"></xref>. The packet spacing also influences the
IPDV distribution and the stream's sensitivity to reordering. For
example, with a 20 ms spacing the IPDV distribution cannot go
below -20ms without packet reordering.</t>
<t>The measurement process may make several simplifying
assumptions when the send spacing and send rate are constant. For
example, the inter-arrival times at the destination can be
compared with an ideal sending schedule, and allowing a one-point
measurement of delay variation (described in <xref
target="Y.1540"></xref>) that approximates the IPDV form.
Simplified methods that approximate PDV are possible as well (some
are discussed in Appendix II of <xref
target="Y.1541"></xref>).</t>
<t>Analysis of truncated, or non-symmetrical IPDV distributions is
simplified. Delay variations in excess of the periodic sending
interval can cause multiple singleton values at the negative limit
of the packet spacing (see section 5.2 and <xref
target="Cia03"></xref>). Only packet reordering can cause the
negative spacing limit to be exceeded.</t>
</list>Despite the emphasis on inter-packet delay differences with
IPDV, both Poisson <xref target="Demichelis"></xref> and Periodic
<xref target="Li.Mills"></xref> streams have been used, and these
references illustrate the different analyses that are possible.</t>
<t>The advantages of using a Poisson distribution are discussed in
<xref target="RFC2330"></xref>. The main properties are to avoid
predicting the sample times, avoid synchronization with periodic
events that are present in networks, and avoid inducing
synchronization with congestion-aware senders. When a Poisson stream
is used with IPDV, the distribution will reflect inter-packet delay
variation on many different time scales (or packet spacings). The
unbiased Poisson sampling brings a new layer of complexity in the
analysis of IPDV distributions.</t>
</section>
<section title="Measurement Devices">
<t>One key aspect of measurement devices is their ability to store
singleton measurements. This feature usually is closely related to
local calculation capabilities. For example, an embedded measurement
device with limited storage will like provide only a few statistics on
the delay variation distribution, while dedicated measurement systems
store all the singletons and allow detailed analysis (later
calculation of either form of delay variation is possible with the
original singletons).</t>
<t>Therefore, systems with limited storage must choose their metrics
and summary statistics in advance. If both IPDV and PDV statistics are
desired, the supporting information must be collected as packets
arrive. For example, the PDV range and high percentiles can be
determined later if the minimum and several of the largest delays are
stored while the measurement is in-progress.</t>
</section>
<section title="Units of Measurement">
<t>Both IPDV and PDV can be summarized as a range in milliseconds.</t>
<t>With IPDV, it is interesting to report on a positive percentile,
and an inter-quantile range is appropriate to reflect both positive
and negative tails (e.g., 5% to 95%). If the IPDV distribution is
symmetric around a mean of zero, then it is sufficient to report on
the positive side of the distribution.</t>
<t>With PDV, it is sufficient to specify the upper percentile (e.g.,
99.9%).</t>
</section>
<section title="Test Duration">
<t>At several points in this memo, we have recommended use of test
intervals on the order of minutes. In their paper examining the
stability of Internet path properties<xref target="Zhang.Duff">
</xref>, Zhang et al. concluded that consistency was present on the
order of minutes for the performance metrics considered (loss, delay,
and throughput) for the paths they measured.</t>
<t>The topic of temporal aggregation of performance measured in small
intervals to estimate some larger interval is described in the Metric
Composition Framework <xref
target="I-D.ietf-ippm-framework-compagg"></xref>.</t>
<t>The primary recommendation here is to test using durations that are
similar in length to the session time of interest. This applies to
both IPDV and PDV, but is possibly more relevant for PDV since the
duration determines how often the D_min will be determined, and the
size of the associated sample.</t>
</section>
<section title="Clock Sync Options">
<t>As with one-way delay measurements, local clock synchronization is
an important matter for delay variation measurements.</t>
<t>There are several options available:</t>
<t><list style="numbers">
<t>Global Positioning System receivers</t>
<t>In some parts of the world, Cellular Code Division Multiple
Access (CDMA) systems distribute timing signals that are derived
from GPS and traceable to UTC.</t>
<t>Network Time Protocol <xref target="RFC1305"></xref> is a
convenient choice in many cases, but usually offers lower accuracy
than the options above.</t>
</list></t>
</section>
<section title="Distinguishing Long Delay from Loss">
<t>Lost and delayed packets are separated by a waiting time threshold.
Packets that arrive at the measurement destination within their
waiting time have finite delay and are not lost. Otherwise, packets
are designated lost and their delay is undefined. Guidance on setting
the waiting time threshold may be found in <xref
target="RFC2680"></xref> and <xref
target="I-D.morton-ippm-reporting-metrics"></xref>.</t>
<t>In essence, <xref
target="I-D.morton-ippm-reporting-metrics"></xref> suggests to use a
long waiting time to serve network characterization and revise results
for specific application delay thresholds as needed.</t>
</section>
<section title="Accounting for Packet Reordering">
<t>Packet reordering, defined in <xref target="RFC4737"></xref>, is
essentially an extreme form of delay variation where the packet stream
arrival order differs from the sending order. </t>
<t>PDV results are not sensitive to packet arrival order, and are not
affected by reordering other than to reflect the more extreme
variation.</t>
<t>IPDV results will change if reordering is present because they are
sensitive to the sequence of delays of arriving packets. The main
example of this sensitivity is in the truncation of the negative tail
of the distribution.</t>
<t><list style="symbols">
<t>When there is no reordering, the negative tail is limited by
the sending time spacing between packets.</t>
<t>If reordering occurs, the negative tail can take on any value
(in principal).</t>
</list>In general, measurement systems should have the capability to
detect when sequence has changed. If IPDV measurements are made
without regard to packet arrival order, the IPDV will be
under-reported when reordering occurs.</t>
</section>
<section title="Results Representation and Reporting">
<t>All of the references that discuss or define delay variation
suggest ways to represent or report the results, and interested
readers should review the various possibilities.</t>
<t>For example, <xref
target="I-D.morton-ippm-reporting-metrics"></xref> suggests to report
a pseudo range of delay variation based on calculating the difference
between a high percentile of delay and the minimum delay. The
99.9%-ile minus the minimum will give a value that can be compared
with objectives in <xref target="Y.1541"></xref>.</t>
</section>
</section>
<section anchor="IANA" title="IANA Considerations">
<t>This document makes no request of IANA.</t>
<t>Note to RFC Editor: this section may be removed on publication as an
RFC.</t>
</section>
<section anchor="Security" title="Security Considerations">
<t>The security considerations that apply to any active measurement of
live networks are relevant here as well. See <xref
target="RFC4656"></xref></t>
</section>
<section anchor="Acknowledgements" title="Acknowledgements">
<t>The authors would like to thank Phil Chimento for his suggestion to
employ the convention of conditional distributions for Delay to deal
with packet loss, and his encouragement to "write the memo" after
hearing the talk on this topic at IETF-65. We also acknowledge
constructive comments from Alan Clark, Loki Jorgenson, Carsten Schmoll,
and Robert Holley.</t>
</section>
<section title="Appendix on Reducing Delay Variation in Networks">
<t>This text is both preliminary and generic but we want to explain the
basic troubleshooting.</t>
<t>If there is a DV problem, it may be because:</t>
<t>1. there is congestion. Find where the bottleneck is, and increase
the buffer Alternatively, increase the bandwidth Alternatively, remove
some applications from that class of service</t>
<t>2. there is a variability of the traffic Discover that traffic, then
change/apply QoS (for example, rate-limiting)</t>
</section>
<section title="Appendix on Calculating the D(min) in PDV">
<t>Practitioners have raised questions several questions that this
section intends to answer:</t>
<t>- how is this D_min calculated? Is it DV(99%) as mentioned in <xref
target="Krzanowski"></xref>?</t>
<t>- do we need to keep all the values from the interval, then take the
minimum? Or do we keep the minimum from previous intervals?</t>
<t>The value of D_min used as the reference delay for PDV calculations
is simply the minimum delay of all packets in the current sample. The
usual single value summary of the PDV distribution is D_99.9%-ile minus
D_min.</t>
<t>It may be appropriate to segregate sub-sets and revise the minimum
value during a sample. For example, if it can be determined with
certainty that the path has changed by monitoring the Time to Live or
Hop Count of arriving packets, this may be sufficient justification to
reset the minimum for packets on the new path. There is also a simpler
approach to solving this problem: use samples collected over short
evaluation intervals (on the order of minutes). Intervals with path
changes may be more interesting from the loss or one-way delay
perspective (possibly failing to meet one or more SLAs), and it may not
be necessary to conduct delay variation analysis. Short evaluation
intervals are preferred for measurements that serve as a basis for
troubleshooting, since the results are available to report soon after
collection.</t>
<t>It is not necessary to store all delay values in a sample when
storage is a major concern. D_min can be found by comparing each new
singleton value with the current value and replacing it when required.
In a sample with 5000 packets, evaluation of the 99.9%-ile can also be
achieved with limited storage. One method calls for storing the top 50
delay singletons and revising the top value list each time 50 more
packets arrive.</t>
</section>
</middle>
<back>
<references title="Normative References">
<?rfc include="reference.RFC.2119"?>
<?rfc include='reference.RFC.2330'?>
<?rfc include='reference.RFC.2679'?>
<?rfc include='reference.RFC.2680'?>
<?rfc include='reference.RFC.3393'?>
<?rfc include='reference.RFC.3432'?>
<?rfc include='reference.RFC.3550'?>
<?rfc include='reference.RFC.4090'?>
<?rfc include='reference.RFC.4656'?>
<?rfc include='reference.RFC.4737'?>
<?rfc ?>
</references>
<references title="Informative References">
<?rfc include='reference.I-D.ietf-ippm-framework-compagg'?>
<?rfc include='reference.I-D.ietf-ippm-spatial-composition'?>
<?rfc include='reference.I-D.morton-ippm-reporting-metrics'?>
<?rfc include='reference.RFC.1305'?>
<?rfc include='reference.RFC.3550'?>
<?rfc include='reference.RFC.3357'?>
<reference anchor="Casner">
<front>
<title>A Fine-Grained View of High Performance Networking, NANOG 22
Conf.; http://www.nanog.org/mtg-0105/agenda.html</title>
<author fullname="S. Casner, C. Alaettinoglu, and C. Kuan,"
surname="">
<organization></organization>
</author>
<date month="May 20-22" year="2001" />
</front>
</reference>
<reference anchor="Cia03">
<front>
<title>Standardized Active Measurements on a Tier 1 IP Backbone,
IEEE Communications Mag., pp 90-97.</title>
<author fullname="L.Ciavattone, A.Morton, and G.Ramachandran">
<organization></organization>
</author>
<date month="June" year="2003" />
</front>
</reference>
<reference anchor="Demichelis">
<front>
<title>Packet Delay Variation Comparison between ITU-T and IETF
Draft Definitions</title>
<author fullname="C.Demichelis"
surname="http://www.advanced.org/ippm/archive.3/att-0075/01-pap02.doc">
<organization>CSELT</organization>
</author>
<date month="November" year="2000" />
</front>
</reference>
<reference anchor="Krzanowski">
<front>
<title>Jitter Definitions: What is What?</title>
<author fullname="R.Krzanowski"
surname="Presentation at IPPM, IETF-64">
<organization></organization>
</author>
<date month="November" year="2005" />
</front>
</reference>
<reference anchor="Morton06">
<front>
<title>"A Brief Jitter Metrics Comparison, and not the last word, by
any means…", Slide Presentation at IETF-65, IPPM
Session,</title>
<author fullname="Al Morton" initials="A." surname="Morton">
<organization>AT&T Labs</organization>
</author>
<date day="21" month="March" year="2006" />
</front>
</reference>
<reference anchor="G.1020">
<front>
<title>"Performance parameter definitions for the quality of speech
and other voiceband applications utilizing IP networks"</title>
<author surname="ITU-T Recommendation G.1020">
<organization></organization>
</author>
<date year="2006" />
</front>
</reference>
<reference anchor="G.1050">
<front>
<title>"Network model for evaluating multimedia transmission
performance over Internet Protocol"</title>
<author initials="" surname="ITU-T Recommendation G.1050">
<organization></organization>
</author>
<date month="November" year="2005" />
</front>
</reference>
<reference anchor="Y.1540">
<front>
<title>Internet protocol data communication service - IP packet
transfer and availability performance parameters</title>
<author fullname="" surname="ITU-T Recommendation Y.1540">
<organization></organization>
</author>
<date month="December " year="2002" />
</front>
</reference>
<reference anchor="Y.1541">
<front>
<title>Network Performance Objectives for IP-Based Services</title>
<author fullname="" surname="ITU-T Recommendation Y.1540">
<organization></organization>
</author>
<date month="February " year="2006" />
</front>
</reference>
<reference anchor="COM12.D98">
<front>
<title>ITU-T Delayed Contribution COM 12 - D98, "Analysis,
measurement and modelling of Jitter"</title>
<author fullname="Alan Clark" initials="Alan" surname="Clark">
<organization>Telchemy Inc.</organization>
</author>
<date day="27-31" month="January" year="2003" />
</front>
</reference>
<reference anchor="Li.Mills">
<front>
<title>"The Implications of Short-Range Dependency on Delay
Variation Measurement", Second IEEE Symposium on Network Computing
and Applications</title>
<author fullname="Quong Li" initials="Quong" surname="Li">
<organization>Philips</organization>
</author>
<author fullname="David L. Mills" initials="David" surname="Mills">
<organization>University of Delaware</organization>
</author>
<date day="" month="" year="2003" />
</front>
</reference>
<reference anchor="Zhang.Duff">
<front>
<title>"On the Constancy of Internet Path Properties", Proceedings
of ACM SIGCOMM Internet Measurement Workshop,</title>
<author fullname="Yin Zhang" initials="Yin" surname="Zhang">
<organization>AT&T Labs</organization>
</author>
<author fullname="Nick Duffield" initials="Nick" surname="Duffield">
<organization>DuffielAT&T Labs</organization>
</author>
<author fullname="Vern Paxson" initials="Vern" surname="Paxson">
<organization>ACIRI</organization>
</author>
<author fullname="Scott Shenker" initials="Scott" surname="Shenker">
<organization>ACIRI</organization>
</author>
<date day="" month="November" year="2001" />
</front>
</reference>
<?rfc ?>
</references>
</back>
</rfc>| PAFTECH AB 2003-2026 | 2026-04-24 05:56:25 |