One document matched: draft-ietf-tsvwg-circuit-breaker-14.xml
<?xml version="1.0" encoding="US-ASCII"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd">
<rfc category="bcp" docName="draft-ietf-tsvwg-circuit-breaker-14"
ipr="trust200902" obsoletes="" updates="">
<!-- Updates -->
<?rfc toc="yes"?>
<!-- generate a table of contents -->
<?rfc symrefs="yes"?>
<!-- use anchors instead of numbers for references -->
<?rfc sortrefs="yes" ?>
<!-- alphabetize the references -->
<?rfc compact="yes" ?>
<!-- conserve vertical whitespace -->
<?rfc subcompact="no" ?>
<!-- but keep a blank line between list items -->
<front>
<title abbrev="">Network Transport Circuit Breakers</title>
<author fullname="Godred Fairhurst" initials="G." surname="Fairhurst">
<organization>University of Aberdeen</organization>
<address>
<postal>
<street>School of Engineering</street>
<street>Fraser Noble Building</street>
<city>Aberdeen</city>
<region>Scotland</region>
<code>AB24 3UE</code>
<country>UK</country>
</postal>
<email>gorry@erg.abdn.ac.uk</email>
<uri>http://www.erg.abdn.ac.uk</uri>
</address>
</author>
<date day="20" month="March" year="2016" />
<area>Transport</area>
<workgroup>TSVWG Working Group</workgroup>
<keyword></keyword>
<keyword></keyword>
<abstract>
<t>This document explains what is meant by the term "network transport
Circuit Breaker" (CB). It describes the need for circuit breakers for
network tunnels and applications when using non-congestion-controlled
traffic, and explains where circuit breakers are, and are not, needed.
It also defines requirements for building a circuit breaker and the
expected outcomes of using a circuit breaker within the Internet.</t>
</abstract>
</front>
<middle>
<!-- text starts here -->
<section title="Introduction" toc="include">
<t>The term "Circuit Breaker" originates in electricity supply, and has
nothing to do with network circuits or virtual circuits. In electricity
supply, a Circuit Breaker is intended as a protection mechanism of last
resort. Under normal circumstances, a Circuit Breaker ought not to be
triggered; it is designed to protect the supply network and attached
equipment when there is overload. People do not expect an electrical
circuit-breaker (or fuse) in their home to be triggered, except when
there is a wiring fault or a problem with an electrical appliance.</t>
<t>In networking, the Circuit Breaker (CB) principle can be used as a
protection mechanism of last resort to avoid persistent excessive
congestion impacting other flows that share network capacity. Persistent
congestion was a feature of the early Internet of the 1980s. This
resulted in excess traffic starving other connections from access to the
Internet. It was countered by the requirement to use congestion control
(CC) in the Transmission Control Protocol (TCP) <xref
target="Jacobsen88"></xref>. These mechanisms operate in Internet hosts
to cause TCP connections to "back off" during congestion. The addition
of a congestion control to TCP (currently documented in <xref
target="RFC5681"></xref> ensured the stability of the Internet, because
it was able to detect congestion and promptly react. This was effective
in an Internet where most TCP flows were long-lived (ensuring that they
could detect and respond to congestion before the flows terminated).
Although TCP was by far the dominant traffic, this is no longer the
always the case, and non-congestion-controlled traffic, including many
applications using the User Datagram Protocol (UDP), can form a
significant proportion of the total traffic traversing a link. The
current Internet therefore requires that non-congestion-controlled
traffic is considered to avoid persistent excessive congestion.</t>
<t>A network transport Circuit Breaker is an automatic mechanism that is
used to continuously monitor a flow or aggregate set of flows. The
mechanism seeks to detect when the flow(s) experience persistent
excessive congestion. When this is detected, a Circuit Breaker
terminates (or significantly reduce the rate of) the flow(s). This is a
safety measure to prevent starvation of network resources denying other
flows from access to the Internet. Such measures are essential for an
Internet that is heterogeneous and for traffic that is hard to predict
in advance. Avoiding persistent excessive congestion is important to
reduce the potential for "Congestion Collapse" <xref
target="RFC2914"></xref>.</t>
<t>There are important differences between a transport Circuit Breaker
and a congestion control method. Congestion control (as implemented in
TCP, SCTP, and DCCP) operates on a timescale on the order of a packet
round-trip-time (RTT), the time from sender to destination and return.
Congestion at a network link can also be detected using Explicit
Congestion Notification (ECN) <xref target="RFC3168"></xref>, which
allows the network to signal congestion by marking ECN-capable packets
with a Congestion Experienced (CE) mark. Both loss and reception of
CE-marked packets are treated as congestion events. Congestion control
methods are able to react to a congestion event by continuously adapting
to reduce their transmission rate. The goal is usually to limit the
transmission rate to a maximum rate that reflects a fair use of the
available capacity across a network path. These methods typically
operate on individual traffic flows (e.g., a 5-tuple that includes the
IP addresses, protocol, and ports).</t>
<t>In contrast, Circuit Breakers are recommended for
non-congestion-controlled Internet flows and for traffic aggregates,
e.g., traffic sent using a network tunnel. They operate on timescales
much longer than the packet RTT, and trigger under situations of
abnormal (excessive) congestion. People have been implementing what this
document characterizes as circuit breakers on an ad hoc basis to protect
Internet traffic. This document therefore provides guidance on how to
deploy and use these mechanisms. Later sections provide examples of
cases where circuit-breakers may or may not be desirable.</t>
<t>A Circuit Breaker needs to measure (meter) some portion of the
traffic to determine if the network is experiencing congestion and needs
to be designed to trigger robustly when there is persistent excessive
congestion.</t>
<t>A Circuit Breaker trigger will often utilize a series of successive
sample measurements metered at an ingress point and an egress point
(either of which could be a transport endpoint). The trigger needs to
operate on a timescale much longer than the path round trip time (e.g.,
seconds to possibly many tens of seconds). This longer period is needed
to provide sufficient time for transport congestion control (or
applications) to adjust their rate following congestion, and for the
network load to stabilize after any adjustment. Congestion events can be
common when a congestion-controlled transport is used over a network
link operating near capacity. Each event results in reduction in the
rate of the transport flow experiencing congestion. The longer period
seeks to ensure that a Circuit Breaker does not accidentally trigger
following a single (or even successive) congestion events.</t>
<t>Once triggered, the Circuit Breaker needs to provide a control
function (called the "reaction"). This removes traffic from the network,
either by disabling the flow or by significantly reducing the level of
traffic. This reaction provides the required protection to prevent
persistent excessive congestion being experienced by other flows that
share the congested part of the network path.</t>
<t><xref target="Require"></xref> defines requirements for building a
Circuit Breaker.</t>
<t>The operational conditions that cause a Circuit Breaker to trigger
ought to be regarded as abnormal. Examples of situations that could
trigger a Circuit Breaker include:<list style="symbols">
<t>anomalous traffic that exceeds the provisioned capacity (or whose
traffic characteristics exceed the threshold configured for the
Circuit Breaker);</t>
<t>traffic generated by an application at a time when the
provisioned network capacity is being utilised for other
purposes;</t>
<t>routing changes that cause additional traffic to start using the
path monitored by the Circuit Breaker;</t>
<t>misconfiguration of a service/network device where the capacity
available is insufficient to support the current traffic
aggregate;</t>
<t>misconfiguration of an admission controller or traffic policer
that allows more traffic than expected across the path monitored by
the Circuit Breaker.</t>
</list></t>
<t>Other mechanisms could also be available to network operators to
detect excessive congestion (e.g., an observation of excessive
utilisation for a port on a network device). Utilising such information,
operational mechanisms could react to reduce network load over a shorter
timescale than those of a network transport Circuit Breaker. The role of
the Circuit Breaker over such paths remains as a method of last resort.
Because it acts over a longer timescale, the Circuit Breaker ought to
trigger only when other reactions did not succeed in reducing persistent
excessive congestion.</t>
<t>In many cases, the reason for triggering a Circuit Breaker will not
be evident to the source of the traffic (user, application, endpoint,
etc). A Circuit Breaker can be used to limit traffic from applications
that are unable, or choose not, to use congestion control, or where the
congestion control properties of the traffic cannot be relied upon
(e.g., traffic carried over a network tunnel). In such circumstances, it
is all but impossible for the Circuit Breaker to signal back to the
impacted applications. In some cases applications could therefore have
difficulty in determining that a Circuit Breaker has triggered, and
where in the network this happened.</t>
<t>Application developers are therefore advised, where possible, to
deploy appropriate congestion control mechanisms. An application that
uses congestion control will be aware of congestion events in the
network. This allows it to regulate the network load under congestion,
and is expected to avoid triggering a network Circuit Breaker. For
applications that can generate elastic traffic, this will often be a
preferred solution.</t>
<section title="Types of Circuit Breaker">
<t>There are various forms of network transport circuit breaker. These
are differentiated mainly on the timescale over which they are
triggered, but also in the intended protection they offer:<list
style="symbols">
<t>Fast-Trip Circuit Breakers: The relatively short timescale used
by this form of circuit breaker is intended to provide protection
for network traffic from a single flow or related group of
flows.</t>
<t>Slow-Trip Circuit Breakers: This circuit breaker utilizes a
longer timescale and is designed to protect network traffic from
congestion by traffic aggregates.</t>
<t>Managed Circuit Breakers: Utilize the operations and management
functions that might be present in a managed service to implement
a circuit breaker.</t>
</list>Examples of each type of circuit breaker are provided in
section 4.</t>
</section>
</section>
<section title="Terminology" toc="include">
<t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in <xref
target="RFC2119"></xref>.</t>
</section>
<section title="Design of a Circuit-Breaker (What makes a good circuit breaker?)"
toc="include">
<t>Although circuit breakers have been talked about in the IETF for many
years, there has not yet been guidance on the cases where circuit
breakers are needed or upon the design of circuit breaker mechanisms.
This document seeks to offer advice on these two topics.</t>
<t>Circuit Breakers are RECOMMENDED for IETF protocols and tunnels that
carry non-congestion-controlled Internet flows and for traffic
aggregates. This includes traffic sent using a network tunnel. Designers
of other protocols and tunnel encapsulations also ought to consider the
use of these techniques as a last resort to protect traffic that shares
the network path being used.</t>
<t>This document defines the requirements for design of a Circuit
Breaker and provides examples of how a Circuit Breaker can be
constructed. The specifications of individual protocols and tunnel
encapsulations need to detail the protocol mechanisms needed to
implement a Circuit Breaker.</t>
<t>Section 3.1 describes the functional components of a circuit breaker
and section 3.2 defines requirements for implementing a Circuit
Breaker.</t>
<section title="Functional Components" toc="include">
<t>The basic design of a Circuit Breaker involves communication
between an ingress point (a sender) and an egress point (a receiver)
of a network flow or set of flows. A simple picture of operation is
provided in figure 1. This shows a set of routers (each labelled R)
connecting a set of endpoints.</t>
<t>A Circuit Breaker is used to control traffic passing through a
subset of these routers, acting between the ingress and a egress point
network devices. The path between the ingress and egress could be
provided by a tunnel or other network-layer technique. One expected
use would be at the ingress and egress of a service, where all traffic
being considered terminates beyond the egress point, and hence the
ingress and egress carry the same set of flows.</t>
<figure>
<artwork><![CDATA[
+--------+ +--------+
|Endpoint| |Endpoint|
+--+-----+ >>> circuit breaker traffic >>> +--+-----+
| |
| +-+ +-+ +---------+ +-+ +-+ +-+ +--------+ +-+ +-+ |
+-+R+--+R+->+ Ingress +--+R+--+R+--+R+--+ Egress |--+R+--+R+-+
+++ +-+ +------+--+ +-+ +-+ +-+ +-----+--+ +++ +-+
| ^ | | |
| | +--+------+ +------+--+ |
| | | Ingress | | Egress | |
| | | Meter | | Meter | |
| | +----+----+ +----+----+ |
| | | | |
+-+ | | +----+----+ | | +-+
|R+--+ | | Measure +<----------------+ +--+R|
+++ | +----+----+ Reported +++
| | | Egress |
| | +----+----+ Measurement |
+--+-----+ | | Trigger + +--+-----+
|Endpoint| | +----+----+ |Endpoint|
+--------+ | | +--------+
+---<---+
Reaction
]]></artwork>
</figure>
<t>Figure 1: A CB controlling the part of the end-to-end path between
an ingress point and an egress point. (Note: In some cases, the
trigger and measurement functions could alternatively be located at
other locations (e.g., at a network operations centre.)</t>
<t>In the context of a Circuit Breaker, the ingress and egress
functions could be implemented in different places. For example, they
could be located in network devices at a tunnel ingress and at the
tunnel egress. In some cases, they could be located at one or both
network endpoints (see figure 2), implemented as components within a
transport protocol.</t>
<t><figure>
<artwork><![CDATA[
+----------+ +----------+
| Ingress | +-+ +-+ +-+ | Egress |
| Endpoint +->+R+--+R+--+R+--+ Endpoint |
+--+----+--+ +-+ +-+ +-+ +----+-----+
^ | |
| +--+------+ +----+----+
| | Ingress | | Egress |
| | Meter | | Meter |
| +----+----+ +----+----+
| | |
| +--- +----+ |
| | Measure +<-----------------+
| +----+----+ Reported
| | Egress
| +----+----+ Measurement
| | Trigger |
| +----+----+
| |
+---<--+
Reaction
]]></artwork>
</figure></t>
<t>Figure 2: An endpoint CB implemented at the sender (ingress) and
receiver (egress).</t>
<t>The set of components needed to implement a Circuit Breaker
are:</t>
<t><list style="numbers">
<t>An ingress meter (at the sender or tunnel ingress) that records
the number of packets/bytes sent in each measurement interval.
This measures the offered network load for a flow or set of flows.
For example, the measurement interval could be many seconds (or
every few tens of seconds or a series of successive shorter
measurements that are combined by the Circuit Breaker Measurement
function).</t>
<t>An egress meter (at the receiver or tunnel egress) that records
the number/bytes received in each measurement interval. This
measures the supported load for the flow or set of flows, and
could utilize other signals to detect the effect of congestion
(e.g., loss/congestion marking <xref target="RFC3168"></xref>
experienced over the path). The measurements at the egress could
be synchronised (including an offset for the time of flight of the
data, or referencing the measurements to a particular packet) to
ensure any counters refer to the same span of packets.</t>
<t>A method that communicates the measured values at the ingress
and egress to the Circuit Breaker Measurement function. This could
use several methods including: Sending return measurement packets
(or control messages) from a receiver to a trigger function at the
sender; an implementation using Operations, Administration and
Management (OAM); or sending an in-band signalling datagram to the
trigger function. This could also be implemented purely as a
control plane function, e.g., using a software-defined network
controller.</t>
<t>A measurement function that combines the ingress and egress
measurements to assess the present level of network congestion.
(For example, the loss rate for each measurement interval could be
deduced from calculating the difference between ingress and egress
counter values.) Note the method does not require high accuracy
for the period of the measurement interval (or therefore the
measured value, since isolated and/or infrequent loss events need
to be disregarded.)</t>
<t>A trigger function that determines whether the measurements
indicate persistent excessive congestion. This function defines an
appropriate threshold for determining that there is persistent
excessive congestion between the ingress and egress. This
preferably considers a rate or ratio, rather than an absolute
value (e.g., more than 10% loss, but other methods could also be
based on the rate of transmission as well as the loss rate). The
Circuit Breaker is triggered when the threshold is exceeded in
multiple measurement intervals (e.g., 3 successive measurements).
Designs need to be robust so that single or spurious events do not
trigger a reaction.</t>
<t>A reaction that is applied at the Ingress when the Circuit
Breaker is triggered. This seeks to automatically remove the
traffic causing persistent excessive congestion.</t>
<t>A feedback control mechanism that triggers when either the
receive or ingress and egress measurements are not available,
since this also could indicate a loss of control packets (also a
symptom of heavy congestion or inability to control the load).</t>
</list></t>
</section>
<section title="Other network topologies">
<t>A Circuit Breaker can be deployed in networks with topologies
different to that presented in figures 1 and 2. This section describes
examples of such usage, and possible places where functions can be
implemented.</t>
<section title="Use with a multicast control/routing protocol">
<t><figure>
<artwork><![CDATA[
+----------+ +--------+ +----------+
| Ingress | +-+ +-+ +-+ | Egress | | Egress |
| Endpoint +->+R+--+R+--+R+--+ Router |--+ Endpoint +->+
+----+-----+ +-+ +-+ +-+ +---+--+-+ +----+-----+ |
^ ^ ^ ^ | ^ | |
| | | | | | | |
+----+----+ + - - - < - - - - + | +----+----+ | Reported
| Ingress | multicast Prune | | Egress | | Ingress
| Meter | | | Meter | | Measurement
+---------+ | +----+----+ |
| | |
| +----+----+ |
| | Measure +<--+
| +----+----+
| |
| +----+----+
multicast | | Trigger |
Leave | +----+----+
Message | |
+----<----+
]]></artwork>
</figure>Figure 3: An example of a multicast CB controlling the
end-to-end path between an ingress endpoint and an egress
endpoint.</t>
<t>Figure 3 shows one example of how a multicast Circuit Breaker
could be implemented at a pair of multicast endpoints (e.g., to
implement a Fast-Trip Circuit Breaker, <xref target="FCB"></xref>).
The ingress endpoint (the sender that sources the multicast traffic)
meters the ingress load, generating an ingress measurement (e.g.,
recording timestamped packet counts), and sends this measurement to
the multicast group together with the traffic it has measured.</t>
<t>Routers along a multicast path forward the multicast traffic
(including the ingress measurement) to all active endpoint
receivers. Each last hop (egress) router forwards the traffic to one
or more egress endpoint(s).</t>
<t>In this figure, each endpoint includes a meter that performs a
local egress load measurement. An endpoint also extracts the
received ingress measurement from the traffic, and compares the
ingress and egress measurements to determine if the Circuit Breaker
ought to be triggered. This measurement has to be robust to loss
(see previous section). If the Circuit Breaker is triggered, it
generates a multicast leave message for the egress (e.g., an IGMP or
MLD message sent to the last hop router), which causes the upstream
router to cease forwarding traffic to the egress endpoint <xref
target="RFC1112"></xref>.</t>
<t>Any multicast router that has no active receivers for a
particular multicast group will prune traffic for that group,
sending a prune message to its upstream router. This starts the
process of releasing the capacity used by the traffic and is a
standard multicast routing function (e.g., using Protocol
Independent Multicast Sparse Mode (PIM-SM) routing protocol <xref
target="RFC4601"></xref>). Each egress operates autonomously, and
the Circuit Breaker "reaction" is executed by the multicast control
plane (e.g., by PIM) requiring no explicit signalling by the Circuit
Breaker along the communication path used for the control messages.
Note: there is no direct communication with the Ingress, and hence a
triggered Circuit Breaker only controls traffic downstream of the
first hop multicast router. It does not stop traffic flowing from
the sender to the first hop router; this is common practice for
multicast deployment.</t>
<t>The method could also be used with a multicast tunnel or
subnetwork (e.g., <xref target="SCB"></xref>, <xref
target="MCB"></xref>), where a meter at the ingress generates
additional control messages to carry the measurement data towards
the egress where the egress metering is implemented.</t>
</section>
<section title="Use with control protocols supporting pre-provisioned capacity">
<t>Some paths are provisioned using a control protocol, e.g., flows
provisioned using the Multi-Protocol Label Switching (MPLS)
services, paths provisioned using the resource reservation protocol
(RSVP), networks utilizing Software Defined Network (SDN) functions,
or admission-controlled Differentiated Services. Figure 1 shows one
expected use case, where in this usage a separate device could be
used to perform the measurement and trigger functions. The reaction
generated by the trigger could take the form of a network control
message sent to the ingress and/or other network elements causing
these elements to react to the Circuit Breaker. Examples of this
type of use are provided in section <xref target="MCB"></xref>.</t>
</section>
<section anchor="unidir"
title="Unidirectional Circuit Breakers over Controlled Paths">
<t>A Circuit Breaker can be used to control uni-directional UDP
traffic, providing that there is a communication path that can be
used for control messages to connect the functional components at
the Ingress and Egress. This communication path for the control
messages can exist in networks for which the traffic flow is purely
unidirectional. For example, a multicast stream that sends packets
across an Internet path and can use multicast routing to prune flows
to shed network load. Some other types of subnetwork also utilize
control protocols that can be used to control traffic flows.</t>
</section>
</section>
</section>
<section anchor="Require"
title="Requirements for a Network Transport Circuit Breaker">
<t>The requirements for implementing a Circuit Breaker are:</t>
<t><!--The order of requirments was updated in -01.
--></t>
<t><list style="numbers">
<t>There needs to be a communication path for control messages to
carry measurement data from the ingress meter and from the egress
meter to the point of measurement. (Requirements 16-18 relate to the
transmission of control messages.)</t>
<t>A CB is REQUIRED to define a measurement period over which the CB
Measurement function measures the level of congestion or loss. This
method does not have to detect individual packet loss, but MUST have
a way to know that packets have been lost/marked from the traffic
flow.</t>
<t>An egress meter can also count ECN <xref target="RFC3168"></xref>
congestion marks as a part of measurement of congestion, but in this
case, loss MUST also be measured to provide a complete view of the
level of congestion. For tunnels, <xref
target="ID-ietf-tsvwg-tunnel-congestion-feedback"></xref> describes
a way to measure both loss and ECN-marking; these measurements could
be used on a relatively short timescale to drive a congestion
control response and/or aggregated over a longer timescale with a
higher trigger threshold to drive a CB. Subsequent bullet items in
this section discuss the necessity of using a longer timescale and a
higher trigger threshold.</t>
<t>The measurement period used by a CB Measurement function MUST be
longer than the time that current Congestion Control algorithms need
to reduce their rate following detection of congestion. This is
important because end-to-end Congestion Control algorithms require
at least one RTT to notify and adjust the traffic when congestion is
experienced, and congestion bottlenecks can share traffic with a
diverse range of RTTs. The measurement period is therefore expected
to be significantly longer than the RTT experienced by the CB
itself.</t>
<t>If necessary, a CB MAY combine successive individual meter
samples from the ingress and egress to ensure observation of an
average measurement over a sufficiently long interval. (Note when
meter samples need to be combined, the combination needs to reflect
the sum of the individual sample counts divided by the total
time/volume over which the samples were measured. Individual samples
over different intervals can not be directly combined to generate an
average value.)</t>
<t>A CB MUST be constructed so that it does not trigger under light
or intermittent congestion (see requirements 7-10).</t>
<t>A CB is REQUIRED to define a threshold to determine whether the
measured congestion is considered excessive.</t>
<t>A CB is REQUIRED to define the triggering interval, defining the
period over which the trigger uses the collected measurements. CBs
need to trigger over a sufficiently long period to avoid
additionally penalizing flows with a long path RTT (e.g., many path
RTTs).</t>
<t>If necessary, a CB MAY combine successive individual meter
samples from the ingress and egress to ensure observation of an
average measurement over a sufficiently long interval. (Note when
meter samples need to be combined, the combination needs to reflect
the sum of the individual sample counts divided by the total
time/volume over which the samples were measured. Individual samples
over different intervals can not be directly combined to generate an
average value.)</t>
<t>A CB MUST be robust to multiple congestion events. This usually
will define a number of measured persistent congestion events per
triggering period. For example, a CB MAY combine the results of
several measurement periods to determine if the CB is triggered
(e.g., it is triggered when persistent excessive congestion is
detected in 3 of the measurements within the triggering
interval).</t>
<t>The normal reaction to a trigger SHOULD disable all traffic that
contributed to congestion (otherwise, see requirements 12,13).</t>
<t>The reaction MUST be much more severe than that of a Congestion
Control algorithm (such as TCP's congestion control <xref
target="RFC5681"></xref> or TCP-Friendly Rate Control, TFRC <xref
target="RFC5348"></xref>), because the CB reacts to more persistent
congestion and operates over longer timescales (i.e., the overload
condition will have persisted for a longer time before the CB is
triggered).</t>
<t>A reaction that results in a reduction SHOULD result in reducing
the traffic by at least an order of magnitude. A response that
achieves the reduction by terminating flows, rather than randomly
dropping packets, will often be more desirable to users of the
service. A CB that reduces the rate of a flow, MUST continue to
monitor the level of congestion and MUST further react to reduce the
rate if the CB is again triggered.</t>
<t>The reaction to a triggered CB MUST continue for a period that is
at least the triggering interval. Operator intervention will usually
be required to restore a flow. If an automated response is needed to
reset the trigger, then this needs to not be immediate. The design
of an automated reset mechanism needs to be sufficiently
conservative that it does not adversely interact with other
mechanisms (including other CB algorithms that control traffic over
a common path). It SHOULD NOT perform an automated reset when there
is evidence of continued congestion.</t>
<t>A CB trigger SHOULD be regarded as an abnormal network event. As
such, this event SHOULD be logged. The measurements that lead to
triggering of the CB SHOULD also be logged.</t>
<t>The control communication needs to carry measurements
(requirement 1) and, in some uses, also needs to transmit trigger
messages to the ingress. This control communication may be in-band
or out-of-band. The use of in-band communication is RECOMMENDED when
either design would be possible. The preferred CB design is one that
triggers when it fails to receive measurement reports that indicate
an absence of congestion, in contrast to relying on the successful
transmission of a "congested" signal back to the sender. (The
feedback signal could itself be lost under congestion).<list
style="hanging">
<t hangText="in-Band:">An in-band control method SHOULD assume
that loss of control messages is an indication of potential
congestion on the path, and repeated loss ought to cause the CB
to be triggered. This design has the advantage that it provides
fate-sharing of the traffic flow(s) and the control
communications. This fate-sharing property is weaker when some
or all of the measured traffic is sent using a path that differs
from the path taken by the control traffic (e.g., where traffic
and control messages follow a different path due to use of
equal-cost multipath routing, traffic engineering, or tunnels
for specific types of traffic).</t>
<t hangText="Out-of-Band:">An out-of-band control method SHOULD
NOT trigger CB reaction when there is loss of control messages
(e.g., a loss of measurements). This avoids failure
amplification/propagation when the measurement and data paths
fail independently. A failure of an out-of-band communication
path SHOULD be regarded as abnormal network event and be handled
as appropriate for the network, e.g., this event SHOULD be
logged, and additional network operator action might be
appropriate, depending on the network and the traffic
involved.</t>
</list></t>
<t>The control communication MUST be designed to be robust to packet
loss. A control message can be lost if there is a failure of the
communication path used for the control messages, loss is likely to
also be experienced during congestion/overload. This does not imply
that it is desirable to provide reliable delivery (e.g., over TCP),
since this can incur additional delay in responding to congestion.
Appropriate mechanisms could be to duplicate control messages to
provide increased robustness to loss, or/and to regard a lack of
control traffic as an indication that excessive congestion could be
being experienced <xref target="ID-ietf-tsvwg-RFC5405.bis"></xref>.
If control messages traffic are sent over a shared path, it is
RECOMMENDED that this control traffic is prioritized to reduce the
probability of loss under congestion. Control traffic also needs to
be considered when provisioning a network that uses a Circuit
Breaker.</t>
<t>There are security requirements for the control communication
between endpoints and/or network devices (<xref
target="sec"></xref>). The authenticity of the source and integrity
of the control messages (measurements and triggers) MUST be
protected from off-path attacks. When there is a risk of on-path
attack, a cryptographic authentication mechanism for all
control/measurement messages is RECOMMENDED.</t>
</list></t>
</section>
<section title="Examples of Circuit Breakers">
<t>There are multiple types of Circuit Breaker that could be defined for
use in different deployment cases. There could be cases where a flow
become controlled by multiple Circuit Breakers (e.g., when the traffic
of an end-to-end flow is carried in a tunnel within the network). This
section provides examples of different types of Circuit Breaker:</t>
<section anchor="FCB" title="A Fast-Trip Circuit Breaker">
<t><xref target="RFC2309"></xref> discusses the dangers of
congestion-unresponsive flows and states that "all UDP-based streaming
applications should incorporate effective congestion avoidance
mechanisms". Some applications do not use a full-featured transport
(TCP, SCTP, DCCP). These applications (e.g., using UDP and its
UDP-Lite variant) need to provide appropriate congestion avoidance.
Guidance for applications that do not use congestion-controlled
transports is provided in <xref
target="ID-ietf-tsvwg-RFC5405.bis"></xref>. Such mechanisms can be
designed to react on much shorter timescales than a Circuit Breaker,
that only observes a traffic envelope. Congestion control methods can
also interact with an application to more effectively control its
sending rate.</t>
<t>A fast-trip Circuit Breaker is the most responsive form of Circuit
Breaker. It has a response time that is only slightly larger than that
of the traffic that it controls. It is suited to traffic with
well-understood characteristics (and could include one or more trigger
functions specifically tailored the type of traffic for which it is
designed). It is not suited to arbitrary network traffic and could be
unsuitable for traffic aggregates, since it could prematurely trigger
(e.g., when the combined traffic from multiple congestion-controlled
flows leads to short-term overload).</t>
<t>Although the mechanisms can be implemented in RTP-aware network
devices, these mechanisms are also suitable for implementation in
endpoints (e.g., as a part of the transport system) where they can
also compliment end-to-end congestion control methods. A shorter
response time enables these mechanisms to triggers before other forms
of Circuit Breaker (e.g., Circuit Breakers operating on traffic
aggregates at a point along the network path).</t>
<section title="A Fast-Trip Circuit Breaker for RTP">
<t>A set of fast-trip Circuit Breaker methods have been specified
for use together by a Real-time Transport Protocol (RTP) flow using
the RTP/AVP Profile <xref target="RTP-CB"></xref>. It is expected
that, in the absence of severe congestion, all RTP applications
running on best-effort IP networks will be able to run without
triggering these Circuit Breakers. A fast-trip RTP Circuit Breaker
is therefore implemented as a fail-safe that when triggered will
terminate RTP traffic.</t>
<t>The sending endpoint monitors reception of in-band RTP Control
Protocol (RTCP) reception report blocks, as contained in SR or RR
packets, that convey reception quality feedback information. This is
used to measure (congestion) loss, possibly in combination with ECN
<xref target="RFC6679"></xref>.</t>
<t>The Circuit Breaker action (shutdown of the flow) is triggered
when any of the following trigger conditions are true:</t>
<t><list style="numbers">
<t>An RTP Circuit Breaker triggers on reported lack of
progress.</t>
<t>An RTP Circuit Breaker triggers when no receiver reports
messages are received.</t>
<t>An RTP Circuit Breaker triggers when the long-term RTP
throughput (over many RTTs) exceeds a hard upper limit
determined by a method that resembles TCP-Friendly Rate Control
(TFRC).</t>
<t>An RTP Circuit Breaker includes the notion of Media
Usability. This Circuit Breaker is triggered when the quality of
the transported media falls below some required minimum
acceptable quality.</t>
</list></t>
</section>
</section>
<section anchor="SCB" title="A Slow-trip Circuit Breaker">
<t>A slow-trip Circuit Breaker could be implemented in an endpoint or
network device. This type of Circuit Breaker is much slower at
responding to congestion than a fast-trip Circuit Breaker. This is
expected to be more common.</t>
<t>One example where a slow-trip Circuit Breaker is needed is where
flows or traffic-aggregates use a tunnel or encapsulation and the
flows within the tunnel do not all support TCP-style congestion
control (e.g., TCP, SCTP, TFRC), see <xref
target="ID-ietf-tsvwg-RFC5405.bis"></xref> section 3.1.3. A use case
is where tunnels are deployed in the general Internet (rather than
"controlled environments" within an Internet service provider or
enterprise network), especially when the tunnel could need to cross a
customer access router.</t>
</section>
<section anchor="MCB" title="A Managed Circuit Breaker">
<t>A managed Circuit Breaker is implemented in the signalling protocol
or management plane that relates to the traffic aggregate being
controlled. This type of Circuit Breaker is typically applicable when
the deployment is within a "controlled environment".</t>
<t>A Circuit Breaker requires more than the ability to determine that
a network path is forwarding data, or to measure the rate of a path -
which are often normal network operational functions. There is an
additional need to determine a metric for congestion on the path and
to trigger a reaction when a threshold is crossed that indicates
persistent excessive congestion.</t>
<t>The control messages can use either in-band or out-of-band
communications.</t>
<section title="A Managed Circuit Breaker for SAToP Pseudo-Wires">
<t><xref target="RFC4553"></xref>, SAToP Pseudo-Wires (PWE3),
section 8 describes an example of a managed Circuit Breaker for
isochronous flows.</t>
<t>If such flows were to run over a pre-provisioned (e.g.,
Multi-Protocol Label Switching, MPLS) infrastructure, then it could
be expected that the Pseudowire (PW) would not experience
congestion, because a flow is not expected to either increase (or
decrease) their rate. If, instead, PW traffic is multiplexed with
other traffic over the general Internet, it could experience
congestion. <xref target="RFC4553"></xref> states: "If SAToP PWs run
over a PSN providing best-effort service, they SHOULD monitor packet
loss in order to detect "severe congestion". The currently
recommended measurement period is 1 second, and the trigger operates
when there are more than three measured Severely Errored Seconds
(SES) within a period. If such a condition is detected, a SAToP PW
ought to shut down bidirectionally for some period of time...".</t>
<t>The concept was that when the packet loss ratio (congestion)
level increased above a threshold, the PW was by default disabled.
This use case considered fixed-rate transmission, where the PW had
no reasonable way to shed load.</t>
<t>The trigger needs to be set at the rate that the PW was likely to
experience a serious problem, possibly making the service
non-compliant. At this point, triggering the Circuit Breaker would
remove the traffic preventing undue impact on congestion-responsive
traffic (e.g., TCP). Part of the rationale, was that high loss
ratios typically indicated that something was "broken" and ought to
have already resulted in operator intervention, and therefore need
to trigger this intervention.</t>
<t>An operator-based response to triggering of a Circuit Breaker
provides an opportunity for other action to restore the service
quality, e.g., by shedding other loads or assigning additional
capacity, or to consciously avoid reacting to the trigger while
engineering a solution to the problem. This could require the
trigger function to send a control message to a third location
(e.g., a network operations centre, NOC) that is responsible for
operation of the tunnel ingress, rather than the tunnel ingress
itself.</t>
</section>
<section title="A Managed Circuit Breaker for Pseudowires (PWs)">
<t>Pseudowires (PWs) <xref target="RFC3985"></xref> have become a
common mechanism for tunneling traffic, and could compete for
network resources both with other PWs and with non-PW traffic, such
as TCP/IP flows.</t>
<t><xref target="ID-ietf-pals-congcons"></xref> discusses congestion
conditions that can arise when PWs compete with elastic (i.e.,
congestion responsive) network traffic (e.g, TCP traffic). Elastic
PWs carrying IP traffic (see <xref target="RFC4488"></xref>) do not
raise major concerns because all of the traffic involved responds,
reducing the transmission rate when network congestion is
detected.</t>
<t>In contrast, inelastic PWs (e.g., a fixed bandwidth Time Division
Multiplex, TDM) <xref target="RFC4553"></xref> <xref
target="RFC5086"></xref> <xref target="RFC5087"></xref>) have the
potential to harm congestion responsive traffic or to contribute to
excessive congestion because inelastic PWs do not adjust their
transmission rate in response to congestion. <xref
target="ID-ietf-pals-congcons"></xref> analyses TDM PWs, with an
initial conclusion that a TDM PW operating with a degree of loss
that could result in congestion-related problems is also operating
with a degree of loss that results in an unacceptable TDM service.
For that reason, the document suggests that a managed Circuit
Breaker that shuts down a PW when it persistently fails to deliver
acceptable TDM service is a useful means for addressing these
congestion concerns.</t>
</section>
</section>
</section>
<section title="Examples where circuit breakers may not be needed. ">
<t>A Circuit Breaker is not required for a single congestion-controlled
flow using TCP, SCTP, TFRC, etc. In these cases, the congestion control
methods are already designed to prevent persistent excessive
congestion.</t>
<section title="CBs over pre-provisioned Capacity">
<t>One common question is whether a Circuit Breaker is needed when a
tunnel is deployed in a private network with pre-provisioned
capacity.</t>
<t>In this case, compliant traffic that does not exceed the
provisioned capacity ought not to result in persistent congestion. A
Circuit Breaker will hence only be triggered when there is
non-compliant traffic. It could be argued that this event ought never
to happen - but it could also be argued that the Circuit Breaker
equally ought never to be triggered. If a Circuit Breaker were to be
implemented, it will provide an appropriate response if persistent
congestion occurs in an operational network.</t>
<t>Implementing a Circuit Breaker will not reduce the performance of
the flows, but in the event that persistent excessive congestion
occurs it protects network traffic that shares network capacity with
these flows. A Circuit Breaker also protects network traffic using a
non-pre-provisioned path from a failure caused by additional network
load resulting when Circuit Breaker traffic is routed over this
path.</t>
</section>
<section title="CBs with tunnels carrying Congestion-Controlled Traffic">
<t>IP-based traffic is generally assumed to be congestion-controlled,
i.e., it is assumed that the transport protocols generating IP-based
traffic at the sender already employ mechanisms that are sufficient to
address congestion on the path. A question therefore arises when
people deploy a tunnel that is thought to only carry an aggregate of
TCP traffic (or traffic using some other congestion control method):
Is there advantage in this case in using a Circuit Breaker?</t>
<t>TCP (and SCTP) traffic in a tunnel is expected to reduce the
transmission rate when network congestion is detected. Other
transports (e.g, using UDP) can employ mechanisms that are sufficient
to address congestion on the path <xref
target="ID-ietf-tsvwg-RFC5405.bis"></xref>. However, even if the
individual flows sharing a tunnel each implement a congestion control
mechanism, and individually reduce their transmission rate when
network congestion is detected, the overall traffic resulting from the
aggregate of the flows does not necessarily avoid persistent
congestion. For instance, most congestion control mechanisms require
long-lived flows to react to reduce the rate of a flow. An aggregate
of many short flows could result in many flows terminating before they
experience congestion. It is also often impossible for a tunnel
service provider to know that the tunnel only contains
congestion-controlled traffic (e.g., Inspecting packet headers could
not be possible). Some IP-based applications might be implement
adequate mechanisms to address congestion. The important thing to note
is that if the aggregate of the traffic does not result in persistent
excessive congestion (impacting other flows), then the Circuit Breaker
will not trigger. This is the expected case in this context - so
implementing a Circuit Breaker ought not to reduce performance of the
tunnel, but in the event that persistent excessive congestion occurs
the Circuit Breaker protects other network traffic that shares
capacity with the tunnel traffic.</t>
</section>
<section title="CBs with Uni-directional Traffic and no Control Path">
<t>A one-way forwarding path could have no associated communication
path for sending control messages, and therefore cannot be controlled
using a Circuit Breaker (compare with <xref
target="unidir"></xref>).</t>
<t>A one-way service could be provided using a path that has dedicated
capacity and does not share this capacity with other elastic Internet
flows (i.e., flows that vary their rate). One way to mitigate the
impact on other flows that share the same capacity is to manage the
traffic envelope by using ingress policing. Supporting this type of
traffic in the general Internet requires operator monitoring to detect
and respond to persistent excessive congestion.</t>
</section>
</section>
<section anchor="sec" title="Security Considerations" toc="include">
<t>All Circuit Breaker mechanisms rely upon coordination between the
ingress and egress meters and communication with the trigger function.
This is usually achieved by passing network control information (or
protocol messages) across the network. Timely operation of a Circuit
Breaker depends on the choice of measurement period. If the receiver has
an interval that is overly long, then the responsiveness of the Circuit
Breaker decreases. This impacts the ability of the Circuit Breaker to
detect and react to congestion. If the interval is too short the Circuit
Breaker could trigger prematurely resulting in insufficient time for
other mechanisms to act, potentially resulting in unnecessary disruption
to the service.</t>
<t>A Circuit Breaker could potentially be exploited by an attacker to
mount a Denial of Service (DoS) attack against the traffic being
controlled by the Circuit Breaker. Mechanisms therefore need to be
implemented to prevent attacks on the network control information that
would result in DoS.</t>
<t>The authenticity of the source and integrity of the control messages
(measurements and triggers) MUST be protected from off-path attacks.
Without protection, it could be trivial for an attacker to inject fake
or modified control/measurement messages (e.g., indicating high packet
loss rates) causing a Circuit Breaker to trigger and to therefore mount
a DoS attack that disrupts a flow.</t>
<t>Simple protection can be provided by using a randomized source port,
or equivalent field in the packet header (such as the RTP SSRC value and
the RTP sequence number) expected not to be known to an off-path
attacker. Stronger protection can be achieved using a secure
authentication protocol to mitigate this concern.</t>
<t>An attack on the control messages is relatively easy for an attacker
on the control path when the messages are neither encrypted nor
authenticated. Use of a cryptographic authentication mechanism for all
control/measurement messages is RECOMMENDED to mitigate this concern,
and would also provide protection from off-path attacks. There is a
design trade-off between the cost of introducing cryptographic security
for control messages and the desire to protect control communication.
For some deployment scenarios the value of additional protection from
DoS attack will therefore lead to a requirement to authenticate all
control messages.</t>
<t>Transmission of network control messages consumes network capacity.
This control traffic needs to be considered in the design of a Circuit
Breaker and could potentially add to network congestion. If this traffic
is sent over a shared path, it is RECOMMENDED that this control traffic
is prioritized to reduce the probability of loss under congestion.
Control traffic also needs to be considered when provisioning a network
that uses a Circuit Breaker.</t>
<t>The Circuit Breaker MUST be designed to be robust to packet loss that
can also be experienced during congestion/overload. Loss of control
messages could be a side-effect of a congested network, but also could
arise from other causes <xref target="Require"></xref>.</t>
<t>The security implications depend on the design of the mechanisms, the
type of traffic being controlled and the intended deployment scenario.
Each design of a Circuit Breaker MUST therefore evaluate whether the
particular Circuit Breaker mechanism has new security implications.</t>
</section>
<section title="IANA Considerations" toc="include">
<t>This document makes no request from IANA.</t>
</section>
<section title="Acknowledgments">
<t>There are many people who have discussed and described the issues
that have motivated this document. Contributions and comments included:
Lars Eggert, Colin Perkins, David Black, Matt Mathis, Andrew McGregor,
Bob Briscoe and Eliot Lear. This work was part-funded by the European
Community under its Seventh Framework Programme through the Reducing
Internet Transport Latency (RITE) project (ICT-317700).</t>
</section>
<section title="Revision Notes">
<t>XXX RFC-Editor: Please remove this section prior to publication
XXX</t>
<t>Draft 00</t>
<t>This was the first revision. Help and comments are greatly
appreciated.</t>
<t>Draft 01</t>
<t>Contained clarifications and changes in response to received
comments, plus addition of diagram and definitions. Comments are
welcome.</t>
<t>WG Draft 00</t>
<t>Approved as a WG work item on 28th Aug 2014.</t>
<t>WG Draft 01</t>
<t>Incorporates feedback after Dallas IETF TSVWG meeting. This version
is thought ready for WGLC comments. Definitions of abbreviations.</t>
<t>WG Draft 02</t>
<t>Minor fixes for typos. Rewritten security considerations section.</t>
<t>WG Draft 03</t>
<t>Updates following WGLC comments (see TSV mailing list). Comments from
C Perkins; D Black and off-list feedback.</t>
<t>A clear recommendation of intended scope.</t>
<t>Changes include: Improvement of language on timescales and minimum
measurement period; clearer articulation of endpoint and multicast
examples - with new diagrams; separation of the controlled network case;
updated text on position of trigger function; corrections to RTP-CB
text; clarification of loss v ECN metrics; checks against submission
checklist 9use of keywords, added meters to diagrams).</t>
<t>WG Draft 04</t>
<t>Added section on PW CB for TDM - a newly adopted draft (D.
Black).</t>
<t>WG Draft 05</t>
<t>Added clarifications requested during AD review.</t>
<t>WG Draft 06</t>
<t>Fixed some remaining typos.</t>
<t>Update following detailed review by Bob Briscoe, and comments by D.
Black.</t>
<t>WG Draft 07</t>
<t>Additional update following review by Bob Briscoe.</t>
<t>WG Draft 08</t>
<t>Updated text on the response to lack of meter measurements with
managed circuit breakers. Additional comments from Eliot Lear (APPs
area).</t>
<t>WG Draft 09</t>
<t>Updated text on applications from Eliot Lear. Additional feedback
from Bob Briscoe.</t>
<t>WG Draft 10</t>
<t>Updated text following comments by D Black including a rewritten ECN
requirements bullet with of a reference to a tunnel measurement method
in [ID-ietf-tsvwg-tunnel-congestion-feedback].</t>
<t>WG Draft 11</t>
<t>Minor corrections after second WGLC.</t>
<t>WG Draft 12</t>
<t>Update following Gen-ART, RTG, and OPS review comments.</t>
<t>WG Draft 13</t>
<t>Fixed a typo.</t>
<t>WG Draft 14</t>
<t>Update after IESG discussion, including:</t>
<t>Reworded introduction. Added definition of ECN.</t>
<t>Requirement</t>
<t>Addressed inconsistency between requirements for control messages. -
Removed a "MUST" - following WG feedback on a anearlier version of the
draft that "SHOULD" is more appropriate.</t>
<t>Addressed comment about grouping requirements to help show they were
inter-related. This reordered some requirements.</t>
<t>Reworded the security considerations.</t>
<t>Corrections to wording to improve clarity.</t>
</section>
</middle>
<!-- *****BACK MATTER ***** -->
<back>
<!-- -->
<references title="Normative References">
<?rfc sortrefs="yes"?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.2119.xml"?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.3168.xml"
?>
<reference anchor="ID-ietf-tsvwg-RFC5405.bis">
<front>
<title>UDP Usage Guidelines (Work-in-Progress)</title>
<author fullname="Lars" initials="L" surname="Eggert">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<author fullname="Gorry" initials="G" surname="Fairhurst">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<author fullname="Greg" initials="G" surname="Shepherd">
<organization></organization>
</author>
<date year="2015" />
</front>
</reference>
</references>
<references title="Informative References">
<reference anchor="Jacobsen88">
<front>
<title>Congestion Avoidance and Control", SIGCOMM Symposium
proceedings on Communications architectures and protocols</title>
<author fullname="Jacobson, V.">
<organization>European Telecommunication Standards, Institute
(ETSI)</organization>
</author>
<date month="August" year="1998" />
</front>
</reference>
<reference anchor="RTP-CB">
<front>
<title>Multimedia Congestion Control: Circuit Breakers for Unicast
RTP Sessions (draft-ietf-avtcore-rtp-circuit-breakers)</title>
<author fullname="C. S. Perkins" initials="C.S." surname="Perkins">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<author fullname="V. Singh" initials="V." surname="Singh">
<organization></organization>
</author>
<date month="February" year="2014" />
</front>
</reference>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.1112.xml"
?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.2309.xml"
?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.2914.xml"
?>
<?rfc ?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.3985.xml"?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.4488.xml"?>
<?rfc ?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.4553.xml"
?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.4601.xml"?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.5086.xml"
?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.5087.xml"
?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.5348.xml"?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.5681.xml"?>
<?rfc include="http://xml.resource.org/public/rfc/bibxml/reference.RFC.6679.xml"?>
<reference anchor="ID-ietf-pals-congcons">
<front>
<title>Pseudowire Congestion Considerations
(Work-in-Progress)</title>
<author fullname="Yaakov" initials="YJ" surname="Stein">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<author fullname="David" initials="D" surname="Black">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<author fullname="Bob" initials="B" surname="Briscoe">
<organization></organization>
</author>
<date year="2015" />
</front>
</reference>
<reference anchor="ID-ietf-tsvwg-tunnel-congestion-feedback">
<front>
<title>Tunnel Congestion Feedback (Work-in-Progress)</title>
<author fullname="X Wei" initials="X" surname="Wei">
<organization></organization>
</author>
<author fullname="L Zhu" initials="L" surname="Zhu">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<author fullname="L Deng" initials="L" surname="Dend">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<date year="2015" />
</front>
</reference>
<?rfc ?>
</references>
</back>
</rfc>
| PAFTECH AB 2003-2026 | 2026-04-23 05:51:04 |