One document matched: draft-ietf-mptcp-experience-01.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="rfc2629.xslt" ?>
<!-- generated by https://github.com/cabo/kramdown-rfc2629 version 1.0.23 -->
<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
]>
<?rfc toc="yes"?>
<?rfc sortrefs="yes"?>
<?rfc symrefs="yes"?>
<rfc ipr="trust200902" docName="draft-ietf-mptcp-experience-01" category="info">
<front>
<title abbrev="MPTCP Experience">Experience with Multipath TCP</title>
<author initials="O." surname="Bonaventure" fullname="Olivier Bonaventure">
<organization>UCLouvain</organization>
<address>
<email>Olivier.Bonaventure@uclouvain.be</email>
</address>
</author>
<author initials="C." surname="Paasch" fullname="Christoph Paasch">
<organization>UCLouvain</organization>
<address>
<email>Christoph.Paasch@uclouvain.be</email>
</address>
</author>
<author initials="G." surname="Detal" fullname="Gregory Detal">
<organization>UCLouvain and Tessares</organization>
<address>
<email>Gregory.Detal@tessares.net</email>
</address>
</author>
<date year="2015" month="March" day="08"/>
<area>Transport</area>
<workgroup>MPTCP Working Group</workgroup>
<keyword>Internet-Draft</keyword>
<abstract>
<t>This document discusses operational experiences of using Multipath TCP in real world networks. It lists several prominent use cases for which Multipath TCP has been considered and is being used. It also gives insight in some heuristics and decisions that have helped to realize these use cases. Further, it presents several open issues that are yet unclear on how they can be solved.</t>
</abstract>
</front>
<middle>
<section anchor="introduction" title="Introduction">
<t>Multipath TCP was standardized in <xref target="RFC6824"/> and four implementations have been developed <xref target="I-D.eardley-mptcp-implementations-survey"/>. Since the publication of <xref target="RFC6824"/>, some experience has been gathered by various network researchers and users about the issues that arise when Multipath TCP is used in the Internet.</t>
<t>Most of the experience reported in this document comes from the
utilization of the Multipath TCP implementation in the Linux kernel
<xref target="MultipathTCP-Linux"/>. It has been downloaded and is used by
thousands of users all over the world. Many of these users have
provided direct or indirect feedback by writing documents (scientific
articles or blog messages) or posting to the mptcp-dev mailing list ( https://listes-2.sipr.ucl.ac.be/sympa/arc/mptcp-dev ) . This Multipath TCP implementation is actively maintained and continuously improved. It is used on various types of hosts, ranging from smartphones or embedded systems to high-end servers.</t>
<t>This is not, by far, the most widespread deployment of Multipath
TCP. Since September 2013, Multipath TCP is also supported on
smartphones and tablets running iOS7 <xref target="IOS7"/>. There are likely
hundreds of millions of Multipath TCP enabled devices. However, this particular Multipath TCP implementation is currently only used to support a single application. Unfortunately, there is no public information about the lessons learned from this large scale deployment.</t>
<t>This document is organized as follows. We explain in Section <xref target="mbox"/>
which types of middleboxes the Linux Kernel implementation of
Multipath TCP supports and how it reacts upon encountering
these. Next, we list several use cases of Multipath TCP in Section
{{usecases}. Section {{congestion} summarises the MPTCP specific
congestion controls that have been implemented. Sections <xref target="pm"/> and
<xref target="scheduler"/> discuss heuristics and issues with respect to subflow
management as well as the scheduling across the subflows. Section
<xref target="mss"/> explains some problems that occurred with subflows having
different MSS values. Section <xref target="cdn"/> presents issues with respect to content delivery networks and suggests a solution to this issue. Finally, Section <xref target="wifi"/> shows an issue with captive portals where MPTCP will behave suboptimal.</t>
</section>
<section anchor="mbox" title="Middlebox interference">
<t>The interference caused by various types of middleboxes has been an important concern during the design of the Multipath TCP protocol. Three studies on the interactions between Multipath TCP and middleboxes are worth being discussed.</t>
<t>The first analysis was described in <xref target="IMC11"/>. This paper was the main motivation for including inside Multipath TCP various techniques to cope with middlebox interference. More specifically, Multipath TCP has been designed to cope with middleboxes that :
- change source or destination addresses
- change source or destination port numbers
- change TCP sequence numbers
- split or coalesce segments
- remove TCP options
- modify the payload of TCP segments</t>
<t>These middlebox interferences have all been included in the MBtest suite <xref target="MBTest"/>. This test suite has been used <xref target="HotMiddlebox13"/> to verify the reaction of the Multipath TCP implementation in the Linux kernel when faced with middlebox interference. The test environment used for this evaluation is a dual-homed client connected to a single-homed server. The middlebox behavior can be activated on any of the paths. The main results of this analysis are :</t>
<t><list style="symbols">
<t>the Multipath TCP implementation in the Linux kernel is not affected by a middlebox that performs NAT or modifies TCP sequence numbers</t>
<t>when a middlebox removes the MP_CAPABLE option from the initial SYN segment, the Multipath TCP implementation in the Linux kernel falls back correctly to regular TCP</t>
<t>when a middlebox removes the DSS option from all data segments, the Multipath TCP implementation in the Linux kernel falls back correctly to regular TCP</t>
<t>when a middlebox performs segment coalescing, the Multipath TCP implementation in the Linux kernel is still able to accurately extract the data corresponding to the indicated mapping</t>
<t>when a middlebox performs segment splitting, the Multipath TCP implementation in the Linux kernel correctly reassembles the data corresponding to the indicated mapping. <xref target="HotMiddlebox13"/> documents a corner case with segment splitting that may lead to desynchronisation between the two hosts.</t>
</list></t>
<t>The interactions between Multipath TCP and real deployed middleboxes is also analyzed in <xref target="HotMiddlebox13"/> and a particular scenario with the FTP application level gateway running on a NAT is described.</t>
<t>From an operational viewpoint, knowing that Multipath TCP can cope
with various types of middlebox interference is important. However,
there are situations where the network operators need to gather
information about where a particular middlebox interference
occurs. The tracebox software <xref target="tracebox"/> described in <xref target="IMC13a"/> is
an extension of the popular traceroute software that enables network
operators to check at which hop a particular field of the TCP header
(including options) is modified. It has been used by several network
operators to debug various middlebox interference problems. tracebox
includes a scripting language that enables its user to specify
precisely which packet is sent by the source. tracebox sends packets
with an increasing TTL/HopLimit and compares the information returned
in the ICMP messages with the packet that it sends. This enables
tracebox to detect any interference caused by middleboxes on a given
path. tracebox works better when routers implement the ICMP extension
defined in <xref target="RFC1812"/>.</t>
<t>Users of the Multipath TCP implementation have reported some
experience with middlebox interference. The strangest scenario has
been a middlebox that accepts the Multipath TCP options in the SYN
segment but later replaces Multipath TCP options with a TCP EOL
option <xref target="StrangeMbox"/>. This causes Multipath TCP to perform a
fallback to regular TCP without any impact on the application.</t>
</section>
<section anchor="usecases" title="Use cases">
<t>Multipath TCP has been tested in several use cases. Several of the papers published in the scientific litterature have identified possible improvements that are worth being discussed here.</t>
<t>A first, although initially unexpected, documented use case for
Multipath TCP has been the datacenters
<xref target="HotNets"/><xref target="SIGCOMM11"/>. Today’s datacenters are designed to provide
several paths between single-homed servers. The multiplicity of these
paths comes from the utilization of Equal Cost Multipath (ECMP) and
other load balancing techniques inside the datacenter. Most of the
deployed load balancing techniques in these datacenters rely on hashes
computed or the five tuple to ensure that all packets from the same
TCP connection will follow the same path to prevent packet
reordering. The results presented in <xref target="HotNets"/> demonstrate by
simulations that Multipath TCP can achieve a better utilization of the
available network by using multiple subflows for each Multipath TCP
session. Although <xref target="RFC6182"/> assumes that at least one of the
communicating hosts has several IP addresses, <xref target="HotNets"/> demonstrates
that there are also benefits when both hosts are single-homed. This
idea was pursued further in <xref target="SIGCOMM11"/> where the Multipath TCP
implementation in the Linux kernel was modified to be able to use
several subflows from the same IP address. Measurements performed in a
public datacenter showed performance improvements with Multipath TCP
<xref target="SIGCOMM11"/>.</t>
<t>Although ECMP is widely used inside datacenters, this is not the only environment where there are different paths between a pair of hosts. ECMP and other load balancing techniques such as LAG are widely used in today’s network and having multiple paths between a pair of single-homed hosts is becoming the norm instead of the exception. Although these multiple paths have often the same cost (from an IGP metrics viewpoint), they do not necessarily have the same performance. For example, <xref target="IMC13c"/> reports the results of a long measurement study showing that load balanced Internet paths between that same pair of hosts can have huge delay differences.</t>
<t>A second use case that has been explored by several network researchers is the cellular/WiFi offload use case. Smartphones or other mobile devices equipped with two wireless interfaces are a very common use case for Multipath TCP. As of this writing, this is also the largest deployment of Multipath-TCP enabled devices <xref target="IOS7"/>. Unfortunately, as there are no public measurements about this deployment, we can only rely on published papers that have mainly used the Multipath TCP implementation in the Linux kernel for their experiment.</t>
<t>The performance of Multipath TCP in wireless networks was briefly
evaluated in <xref target="NSDI12"/>. One experiment analyzes the performance of
Multipath TCP on a client with two wireless interfaces. This
evaluation shows that when the receive window is large, Multipath TCP
can efficiently use the two available links. However, if the window
becomes smaller, then packets sent on a slow path can block the
transmission of packets on a faster path. In some cases, the
performance of Multipath TCP over two paths can become lower than the
performance of regular TCP over the best performing path. Two
heuristics, reinjection and penalization, are proposed in <xref target="NSDI12"/>
to solve this identified performance problem. These two heuristics
have since been used in the Multipath TCP implementation in the Linux
kernel. <xref target="CONEXT13"/> explored the problem in more details and revealed
some other scenarios where Multipath TCP can have difficulties in
efficiently pooling the available paths. Improvements to the Multipath
TCP implementation in the Linux kernel are proposed in <xref target="CONEXT13"/> to cope with some of these problems.</t>
<t>The first experimental analysis of Multipath TCP in a public wireless environment was presented in <xref target="Cellnet12"/>. These measurements explore the ability of Multipath TCP to use two wireless networks (real WiFi and 3G networks). Three modes of operation are compared. The first mode of operation is the simultaneous use of the two wireless networks. In this mode, Multipath TCP pools the available resources and uses both wireless interfaces. This mode provides fast handover from WiFi to cellular or the opposite when the user moves. Measurements presented in <xref target="CACM14"/> show that the handover from one wireless network to another is not an abrupt process. When a host moves, it does not experience either excellent connectivity or no connectivity at all. Instead, there are regions where the quality of one of the wireless networks is weaker than the other, but the host considers this wireless network to still be up. When a mobile host enters such regions, its ability to send packets over another wireless network is important to ensure a smooth handover. This is clearly illustrated from the packet trace discussed in <xref target="CACM14"/>.</t>
<t>Many cellular networks use volume-based pricing and users often prefer to use unmetered WiFi networks when available instead of metered cellular networks. <xref target="Cellnet12"/> implements the support for the MP_PRIO option to explore two other modes of operation.</t>
<t>In the backup mode, Multipath TCP opens a TCP subflow over each interface, but the cellular interface is configured in backup mode. This implies that data only flows over the WiFi interface when both interfaces are considered to be active. If the WiFi interface fails, then the traffic switches quickly to the cellular interface, ensuring a smooth handover from the user’s viewpoint <xref target="Cellnet12"/>. The cost of this approach is that the WiFi and cellular interfaces likely remain active all the time since all subflows are established over the two interfaces.</t>
<t>The single-path mode is slightly different. This mode benefits from the break-before-make capability of Multipath TCP. When an MPTCP session is established, a subflow is created over the WiFi interface. No packet is sent over the cellular interface as long as the WiFi interface remains up <xref target="Cellnet12"/>. This implies that the cellular interface can remain idle and battery capacity is preserved. When the WiFi interface fails, new subflows are established over the cellular interface in order to preserve the established Multipath TCP sessions. Compared to the backup mode described earlier, this mode of operation is characterized by a throughput drop while the cellular interface is brought up and the subflows are reestablished. During this time, no data packet is transmitted.</t>
<t>From a protocol viewpoint, <xref target="Cellnet12"/> discusses the problem posed
by the unreliability of the ADD_ADDR option and proposes a small
protocol extension to allow hosts to reliably exchange this
option. It would be useful to analyze packet traces to understand
whether the unreliability of the REMOVE_ADDR option poses an
operational problem in real deployments.</t>
<t>Another study of the performance of Multipath TCP in wireless networks was reported in <xref target="IMC13b"/>. This study uses laptops connected to various cellular ISPs and WiFi hotspots. It compares various file transfer scenarios and concludes based on measurements with the Multipath TCP implementation in the Linux kernel that “MPTCP provides a robust data transport and reduces variations in download latencies”.</t>
<t>A different study of the performance of Multipath TCP with two wireless networks is presented in <xref target="INFOCOM14"/>. In this study the two networks had different qualities : a good network and a lossy network. When using two paths with different packet loss ratios, the Multipath TCP congestion control scheme moves traffic away from the lossy link that is considered to be congested. However, <xref target="INFOCOM14"/> documents an interesting scenario that is summarised in the figure below.</t>
<figure title="Simple network topology" anchor="figsimple"><artwork><![CDATA[
client ----------- path1 -------- server
| |
+--------------- path2 ------------+
]]></artwork></figure>
<t>Initially, the two paths have the same quality and Multipath TCP
distributes the load over both of them. During the transfer, the
second path becomes lossy, e.g. because the client moves. Multipath
TCP detects the packet losses and they are retransmitted over the
first path. This enables the data transfer to continue over the first
path. However, the subflow over the second path is still up and
transmits one packet from time to time. Although the N packets have
been acknowledged over the first subflow (at the MPTCP level), they
have not been acknowledged at the TCP level over the second
subflow. To preserve the continuity of the sequence numbers over the
second subflow, TCP will continue to retransmit these segments until
either they are acknowledged or the maximum number of retransmissions
is reached. This behavior is clearly inefficient and may lead to
blocking since the second subflow will consume window space to be able
to retransmit these packets. <xref target="INFOCOM14"/> proposes a new Multipath
TCP option to solve this problem. In practice, a new TCP option is
probably not required. When the client detects that the data
transmitted over the second subflow has been acknowledged over the
first subflow, it could decide to terminate the second subflow by
sending a RST segment. If the interface associated to this subflow is
still up, a new subflow could be immediately reestablished. It would then be immediately usable to send new data and would not be forced to first retransmit the previously transmitted data. As of this writing, this dynamic management of the subflows is not yet implemented in the Multipath TCP implementation in the Linux kernel.</t>
<t>A third use case has been the coupling between software defined networking techniques such as Openflow and Multipath TCP. Openflow can be used to configure different paths inside a network. Using an international network, <xref target="TNC13"/> demonstrates that Multipath TCP can achieve high throughput in the wide area. An interesting point to note about the measurements reported in <xref target="TNC13"/> is that the measurement setup used four paths through the WAN. Only two of these paths were disjoint. When Multipath TCP was used, the congestion control scheme ensured that only two of these paths were actually used.</t>
</section>
<section anchor="congestion" title="Congestion control">
<t>Congestion control has been an important problem for Multipath TCP. The standardised congestion control scheme for Multipath TCP is defined in <xref target="RFC6356"/> and <xref target="NSDI11"/>. This congestion control scheme has been implemented in the Linux implementation of Multipath TCP. Linux uses a modular architecture to support various congestion control schemes. This architecture is applicable for both regular TCP and Multipath TCP. While the coupled congestion control scheme defined in <xref target="RFC6356"/> is the default congestion control scheme in the Linux implementation, other congestion control schemes have been added. The second congestion control scheme is OLIA <xref target="CONEXT12"/>. This congestion control scheme is also an adaptation of the NewReno single path congestion control scheme to support multiple paths. Simulations and measurements have shown that it provides some performance benefits compared to the the default congestion control scheme <xref target="CONEXT12"/>. Measurement over a wide range of parameters reported in <xref target="CONEXT13"/> also indicate some benefits with the OLIA congestion control scheme. Recently, a delay-based congestion control scheme has been ported to the Multipath TCP implementation in the Linux kernel. This congestion control scheme has been evaluated by using simulations in <xref target="ICNP12"/>.</t>
</section>
<section anchor="pm" title="Subflow management">
<t>The multipath capability of Multipath TCP comes from the utilization of one subflow per path. The Multipath TCP architecture <xref target="RFC6182"/> and the protocol specification <xref target="RFC6824"/> define the basic usage of the subflows and the protocol mechanisms that are required to create and terminate them. However, there are no guidelines on how subflows are used during the lifetime of a Multipath TCP session. Most of the experiments with Multipath TCP have been performed in controlled environments. Still, based on the experience running them and discussions on the mptcp-dev mailing list, interesting lessons have been learned about the management of these subflows.</t>
<t>From a subflow viewpoint, the Multipath TCP protocol is completely symmetrical. Both the clients and the server have the capability to create subflows. However in practice the existing Multipath TCP implementations <xref target="I-D.eardley-mptcp-implementations-survey"/> have opted for a strategy where only the client creates new subflows. The main motivation for this strategy is that often the client resides behind a NAT or a firewall, preventing passive subflow openings on the client. Although there are environments such as datacenters where this problem does not occur, as of this writing, no precise requirement has emerged for allowing the server to create new subflows.</t>
<section anchor="implemented-subflow-managers" title="Implemented subflow managers">
<t>The Multipath TCP implementation in the Linux kernel includes several strategies to manage the subflows that compose a Multipath TCP session. The basic subflow manager is the full-mesh. As the name implies, it creates a full-mesh of subflows between the communicating hosts.</t>
<t>The most frequent use case for this subflow manager is a multihomed client connected to a single-homed server. In this case, one subflow is created for each interface on the client. The current implementation of the full-mesh subflow manager is static. The subflows are created immediately after the creation of the initial subflow. If one subflow fails during the lifetime of the Multipath TCP session (e.g. due to excessive retransmissions, or the loss of the corresponding interface), it is not always reestablished. There is ongoing work to enhance the full-mesh path manager to deal with such events.</t>
<t>When the server is multihomed, using the full-mesh subflow manager may lead to a large number of subflows being established. For example, consider a dual-homed client connected to a server with three interfaces. In this case, even if the subflows are only created by the client, 6 subflows will be established. This may be excessive in some environments, in particular when the client and/or the server have a large number of interfaces. It should be noted that there have been reports on the mptcp-dev mailing indicating that users rely on Multipath TCP to aggregate more than four different interfaces. Thus, there is a need for supporting many interfaces efficiently.</t>
<t>It should be noted that creating subflows between multihomed clients and servers may sometimes lead to operational issues as observed by discussions on the mptcp-dev mailing list. In some cases the network operators would like to have a better control on how the subflows are created by Multipath TCP. This might require the definition of policy rules to control the operation of the subflow manager. The two scenarios below illustrate some of these requirements.</t>
<figure title="Simple switched network topology" anchor="figswitched"><artwork><![CDATA[
host1 ---------- switch1 ----- host2
| | |
+-------------- switch2 --------+
]]></artwork></figure>
<t>Consider the simple network topology shown in <xref target="figswitched"/>. From an operational viewpoint, a network operator could want to create two subflows between the communicating hosts. From a bandwidth utilization viewpoint, the most natural paths are host1-switch1-host2 and host1-switch2-host2. However, a Multipath TCP implementation running on these two hosts may sometimes have difficulties to achieve this result.</t>
<t>To understand the difficulty, let us consider different allocation strategies for the IP addresses. A first strategy is to assign two subnets : subnetA (resp. subnetB) contains the IP addresses of host1’s interface to switch1 (resp. switch2) and host2’s interface to switch1 (resp. switch2). In this case, a Multipath TCP subflow manager should only create one subflow per subnet. To enforce the utilization of these paths, the network operator would have to specify a policy that prefers the subflows in the same subnet over subflows between addresses in different subnets. It should be noted that the policy should probably also specify how the subflow manager should react when an interface or subflow fails.</t>
<t>A second strategy is to use a single subnet for all IP addresses. In this case, it becomes more difficult to specify a policy that indicates which subflows should be established.</t>
<t>The second subflow manager that is currently supported by the Multipath TCP implementation in the Linux kernel is the ndiffport subflow manager. This manager was initially created to exploit the path diversity that exists between single-homed hosts due to the utilization of flow-based load balancing techniques. This subflow manager creates N subflows between the same pair of IP addresses. The N subflows are created by the client and differ only in the source port selected by the client.</t>
</section>
<section anchor="subflow-destination-port" title="Subflow destination port">
<t>The Multipath TCP protocol relies on the token contained in the MP_JOIN option to associate a subflow to an existing Multipath TCP session. This implies that there is no restriction on the source address, destination address and source or destination ports used for the new subflow. The ability to use different source and destination addresses is key to support multihomed servers and clients. The ability to use different destination port numbers is worth being discussed because it has operational implications.</t>
<t>For illustration, consider a dual-homed client that creates a second subflow to reach a single-homed server as illustrated in the <xref target="figmultihomed"/>.</t>
<figure title="Multihomed-client connected to single-homed server" anchor="figmultihomed"><artwork><![CDATA[
client ------- r1 --- internet --- server
| |
+----------r2-------+
]]></artwork></figure>
<t>When the Multipath TCP implementation in the Linux kernel creates the second subflow it uses the same destination port as the initial subflow. This choice is motivated by the fact that the server might be protected by a firewall and only accept TCP connections (including subflows) on the official port number. Using the same destination port for all subflows is also useful for operators that rely on the port numbers to track application usage in their network.</t>
<t>There have been suggestions from Multipath TCP users to modify the implementation to allow the client to use different destination ports to reach the server. This suggestion seems mainly motivated by traffic shaping middleboxes that are used in some wireless networks. In networks where different shaping rates are associated to different destination port numbers, this could allow Multipath TCP to reach a higher performance. As of this writing, we are not aware of any implementation of this kind of tweaking.</t>
<t>However, from an implementation point-of-view supporting different destination ports for the same Multipath TCP connection introduces a new performance issue. A legacy implementation of a TCP stack creates a listening socket to react upon incoming SYN segments. The listening socket is handling the SYN segments that are sent on a specific port number. Demultiplexing incoming segments can thus be done solely by looking at the IP addresses and the port numbers. With Multipath TCP however, incoming SYN segments may have an MP_JOIN option with a different destination port. This means, that all incoming segments that did not match on an existing listening-socket or an already established socket must be parsed for an eventual MP_JOIN option. This imposes an additional cost on servers, previously not existent on legacy TCP implementations.</t>
</section>
<section anchor="closing-subflows" title="Closing subflows">
<figure title="Multipath TCP may not be able to avoid time-wait state (even if enforced by the application)." anchor="figtimewait"><artwork><![CDATA[
client server
| |
MPTCP: established | | MPTCP: established
Sub: established | | Sub: established
| |
| DATA_FIN |
MPTCP: close-wait | <------------------------ | close() (step 1)
Sub: established | DATA_ACK |
| ------------------------> | MPTCP: fin-wait-2
| | Sub: established
| |
| DATA_FIN + subflow-FIN |
close()/shutdown() | ------------------------> | MPTCP: time-wait
(step 2) | DATA_ACK | Sub: close-wait
MPTCP: closed | <------------------------ |
Sub: fin-wait-2 | |
| |
| subflow-FIN |
MPTCP: closed | <------------------------ | subflow-close()
Sub: time-wait | subflow-ACK |
(step 3) | ------------------------> | MPTCP: time-wait
| | Sub: closed
| |
]]></artwork></figure>
<t><xref target="figtimewait"/> shows a very particular issue within Multipath TCP. Many high-performance applications try to avoid Time-Wait state by deferring the closure of the connection until the peer has sent a FIN. That way, the client on the left of <xref target="figtimewait"/> does a passive closure of the connection, transitioning from Close-Wait to Last-ACK and finally freeing the resources after reception of the ACK of the FIN. An application running on top of a Multipath TCP enabled Linux kernel might also use this approach. The difference here is that the close() of the connection (Step 1 in <xref target="figtimewait"/>) only triggers the sending of a DATA_FIN. Nothing guarantees that the kernel is ready to combine the DATA_FIN with a subflow-FIN. The reception of the DATA_FIN will make the application trigger the closure of the connection (step 2), trying to avoid Time-Wait state with this late closure. This time, the kernel might decide to combine the DATA_FIN with a subflow-FIN. This decision will be fatal, as the subflow’s state machine will not transition from Close-Wait to Last-Ack, but rather go through Fin-Wait-2 into Time-Wait state. The Time-Wait state will consume resources on the host for at least 2 MSL (Maximum Segment Lifetime). Thus, a smart application, that tries to avoid Time-Wait state by doing late closure of the connection actually ends up with one of its subflows in Time-Wait state. A high-performance Multipath TCP kernel implementation should honor the desire of the application to do passive closure of the connection and successfully avoid Time-Wait state - even on the subflows.</t>
<t>The solution to this problem lies in an optimistic assumption that a host doing active-closure of a Multipath TCP connection by sending a DATA_FIN will soon also send a FIN on all its in subflows. Thus, the passive closer of the connection can simply wait for the peer to send exactly this FIN - enforcing passive closure even on the subflows. Of course, to avoid consuming resources indefinitely, a timer must limit the time our implementation waits for the FIN.</t>
</section>
</section>
<section anchor="scheduler" title="Packet schedulers">
<t>In a Multipath TCP implementation, the packet scheduler is the algorithm that is executed when transmitting each packet to decide on which subflow it needs to be transmitted. The packet scheduler itself does not have any impact on the interoperability of Multipath TCP implementations. However, it may clearly impact the performance of Multipath TCP sessions. It is important to note that the problem of scheduling Multipath TCP packets among subflows is different from the problem of scheduling SCTP messages. SCTP implementations also include schedulers, but these are used to schedule the different streams. Multipath TCP uses a single data stream.</t>
<t>Various researchers have explored theoretically and by simulations the problem of scheduling packets among Multipath TCP subflows <xref target="ICC14"/>. Unfortunately, none of the proposed techniques have been implemented and used in real deployment. A detailed analysis of the impact of the packet scheduler will appear in <xref target="CSWS14"/>. This article proposes a pluggable architecture for the scheduler used by the Multipath TCP implementation in the Linux kernel. This architecture allows researchers to experiment with different types of schedulers. Two schedulers are compared in <xref target="CSWS14"/> : round-robin and lowest-rtt-first. The experiments and measurements described in <xref target="CSWS14"/> show that the lowest-rtt-first scheduler appears to be the best compromise from a performance viewpoint.</t>
<t>Another study of the packet schedulers is presented in <xref target="PAMS2014"/>. This study relies on simulations with the Multipath TCP implementation in the Linux kernel. The simulation scenarios discussed in <xref target="PAMS2014"/> confirm the impact of the packet scheduler on the performance of Multipath TCP.</t>
</section>
<section anchor="mss" title="Segment size selection">
<t>When an application performs a write/send system call, the kernel
allocates a packet buffer (sk_buff in Linux) to store the data the
application wants to send. The kernel will store at most one MSS
(Maximum Segment Size) of data per buffer. As MSS can differ amongst
subflows, an MPTCP implementation must select carefully the MSS used
to generate application data. The Linux kernel implementation
had various ways of selecting the MSS: minimum or maximum amongst the
different subflows. However, these heuristics of MSS selection can cause significant
performances issues in some environment. Consider the following
example. An MPTCP connection has two established subflows that
respectively use a MSS of 1420 and 1428 bytes. If MPTCP selects the maximum,
then the application will generate segments of 1428 bytes of data. An
MPTCP implementation will have to split the segment in two (a
1420-byte and 8-byte segments) when pushing on the subflow with the
smallest MSS. The latter segment will introduce a large overhead as
for a single data segment 2 slots will be used in the congestion
window (in packets) therefore reducing by ~2 the potential throughput
(in bytes/s) of this subflow. Taking the smallest MSS does not solve
the issue as there might be a case where the sublow with the smallest
MSS will only participate marginally to the overall performance
therefore reducing the potential throughput of the other subflows.</t>
<t>The Linux implementation recently took another approach <xref target="DetalMSS"/>. Instead of
selecting the minimum and maximum values, it now dynamically adapts
the MSS based on the contribution of all the subflows to the
connection’s throughput. For this it computes, for each subflow, the potential
throughput achieved by selecting each MSS value and by taking into
account the lost space in the cwnd. It then selects the MSS that allows
to achieve the highest potential throughput.</t>
</section>
<section anchor="cdn" title="Interactions with the Domain Name System">
<t>Multihomed clients such as smartphones could lead to operational problems when interacting with the Domain Name System. When a single-homed client performs a DNS query, it receives from its local resolver the best answer for its request. If the client is multihomed, the answer returned to the DNS query may vary with the interface over which it has been sent.</t>
<figure title="Simple network topology" anchor="figcdn"><artwork><![CDATA[
cdn1
|
client -- cellular -- internet -- cdn3
| |
+----- wifi --------+
|
cdn2
]]></artwork></figure>
<t>If the client sends a DNS query over the WiFi interface, the answer
will point to the cdn2 server while the same request sent over the
cellular interface will point to the cdn1 server. This might cause
problems for CDN providers that locate their servers inside ISP
networks and have contracts that specify that the CDN server will only
be accessed from within this particular ISP. Assume now that both the
client and the CDN servers support Multipath TCP. In this case, a
Multipath TCP session from cdn1 or cdn2 would potentially use both the cellular network and the WiFi network. This would violate the contract between the CDN provider and the network operators. A possible solution to prevent this problem would be to modify the DNS resolution on the client. The client subnet EDNS extension defined in <xref target="I-D.vandergaast-edns-client-subnet"/> could be used for this purpose. When the client sends a DNS query from its WiFi interface, it should also send the client subnet corresponding to the cellular interface in this request. This would indicate to the resolver that the answer should be valid for both the WiFi and the cellular interfaces (e.g., the cdn3 server).</t>
</section>
<section anchor="wifi" title="Captive portals">
<t>Multipath TCP enables a host to use different interfaces to reach a server. In theory, this should ensure connectivity when at least one of the interfaces is active. In practice however, there are some particular scenarios with captive portals that may cause operational problems. The reference environment is the following :</t>
<figure title="Issue with captive portal" anchor="figcaptive"><artwork><![CDATA[
client ----- network1
|
+------- internet ------------- server
]]></artwork></figure>
<t>The client is attached to two networks : network1 that provides
limited connectivity and the entire Internet through the second
network interface. In practice, this scenario corresponds to an open
WiFi network with a captive portal for network1 and a cellular service
for the second interface. On many smartphones, the WiFi interface is
preferred over the cellular interface. If the smartphone learns a
default route via both interfaces, it will typically prefer to use the
WiFi interface to send its DNS request and create the first subflow. This is not optimal with Multipath TCP. A better approach would probably be to try a few attempts on the WiFi interface and then try to use the second interface for the initial subflow as well.</t>
</section>
<section anchor="conclusion" title="Conclusion">
<t>In this document, we have documented a few years of experience with Multipath TCP. The information presented in this document was gathered from scientific publications and discussions with various users of the Multipath TCP implementation in the Linux kernel.</t>
</section>
<section anchor="acknowledgements" title="Acknowledgements">
<t>This work was partially supported by the FP7-Trilogy2 project. We would like to thank all the implementers and users of the Multipath TCP implementation in the Linux kernel.</t>
</section>
<section anchor="changelog" title="Changelog">
<t><list style="symbols">
<t>initial version : September 16th, 2014 : Added section <xref target="mss"/> that
discusses some performance problems that appeared with the Linux
implementation when using subflows having different MSS values</t>
<t>update with a description of the middlebox that replaces an unknown
TCP option with EOL <xref target="StrangeMbox"/></t>
</list></t>
</section>
</middle>
<back>
<references title='Informative References'>
<reference anchor='RFC1812'>
<front>
<title>Requirements for IP Version 4 Routers</title>
<author initials='F.' surname='Baker' fullname='Fred Baker'>
<organization>Cisco Systems</organization>
<address>
<postal>
<street>519 Lado Drive</street>
<city>Santa Barbara</city>
<region>CA</region>
<code>93111</code>
<country>US</country></postal>
<phone>+1 805 681 0115</phone>
<email>fred@cisco.com</email></address></author>
<date year='1995' month='June' /></front>
<seriesInfo name='RFC' value='1812' />
<format type='TXT' octets='415740' target='http://www.rfc-editor.org/rfc/rfc1812.txt' />
</reference>
<reference anchor='RFC6182'>
<front>
<title>Architectural Guidelines for Multipath TCP Development</title>
<author initials='A.' surname='Ford' fullname='A. Ford'>
<organization /></author>
<author initials='C.' surname='Raiciu' fullname='C. Raiciu'>
<organization /></author>
<author initials='M.' surname='Handley' fullname='M. Handley'>
<organization /></author>
<author initials='S.' surname='Barre' fullname='S. Barre'>
<organization /></author>
<author initials='J.' surname='Iyengar' fullname='J. Iyengar'>
<organization /></author>
<date year='2011' month='March' />
<abstract>
<t>Hosts are often connected by multiple paths, but TCP restricts communications to a single path per transport connection. Resource usage within the network would be more efficient were these multiple paths able to be used concurrently. This should enhance user experience through improved resilience to network failure and higher throughput.</t><t> This document outlines architectural guidelines for the development of a Multipath Transport Protocol, with references to how these architectural components come together in the development of a Multipath TCP (MPTCP). This document lists certain high-level design decisions that provide foundations for the design of the MPTCP protocol, based upon these architectural requirements. This document is not an Internet Standards Track specification; it is published for informational purposes.</t></abstract></front>
<seriesInfo name='RFC' value='6182' />
<format type='TXT' octets='68772' target='http://www.rfc-editor.org/rfc/rfc6182.txt' />
</reference>
<reference anchor='RFC6356'>
<front>
<title>Coupled Congestion Control for Multipath Transport Protocols</title>
<author initials='C.' surname='Raiciu' fullname='C. Raiciu'>
<organization /></author>
<author initials='M.' surname='Handley' fullname='M. Handley'>
<organization /></author>
<author initials='D.' surname='Wischik' fullname='D. Wischik'>
<organization /></author>
<date year='2011' month='October' />
<abstract>
<t>Often endpoints are connected by multiple paths, but communications are usually restricted to a single path per connection. Resource usage within the network would be more efficient were it possible for these multiple paths to be used concurrently. Multipath TCP is a proposal to achieve multipath transport in TCP.</t><t> New congestion control algorithms are needed for multipath transport protocols such as Multipath TCP, as single path algorithms have a series of issues in the multipath context. One of the prominent problems is that running existing algorithms such as standard TCP independently on each path would give the multipath flow more than its fair share at a bottleneck link traversed by more than one of its subflows. Further, it is desirable that a source with multiple paths available will transfer more traffic using the least congested of the paths, achieving a property called "resource pooling" where a bundle of links effectively behaves like one shared link with bigger capacity. This would increase the overall efficiency of the network and also its robustness to failure.</t><t> This document presents a congestion control algorithm that couples the congestion control algorithms running on different subflows by linking their increase functions, and dynamically controls the overall aggressiveness of the multipath flow. The result is a practical algorithm that is fair to TCP at bottlenecks while moving traffic away from congested links. This document defines an Experimental Protocol for the Internet community.</t></abstract></front>
<seriesInfo name='RFC' value='6356' />
<format type='TXT' octets='27961' target='http://www.rfc-editor.org/rfc/rfc6356.txt' />
</reference>
<reference anchor='RFC6824'>
<front>
<title>TCP Extensions for Multipath Operation with Multiple Addresses</title>
<author initials='A.' surname='Ford' fullname='A. Ford'>
<organization /></author>
<author initials='C.' surname='Raiciu' fullname='C. Raiciu'>
<organization /></author>
<author initials='M.' surname='Handley' fullname='M. Handley'>
<organization /></author>
<author initials='O.' surname='Bonaventure' fullname='O. Bonaventure'>
<organization /></author>
<date year='2013' month='January' />
<abstract>
<t>TCP/IP communication is currently restricted to a single path per connection, yet multiple paths often exist between peers. The simultaneous use of these multiple paths for a TCP/IP session would improve resource usage within the network and, thus, improve user experience through higher throughput and improved resilience to network failure.</t><t> Multipath TCP provides the ability to simultaneously use multiple paths between peers. This document presents a set of extensions to traditional TCP to support multipath operation. The protocol offers the same type of service to applications as TCP (i.e., reliable bytestream), and it provides the components necessary to establish and use multiple TCP flows across potentially disjoint paths. This document defines an Experimental Protocol for the Internet community.</t></abstract></front>
<seriesInfo name='RFC' value='6824' />
<format type='TXT' octets='164866' target='http://www.rfc-editor.org/rfc/rfc6824.txt' />
</reference>
<reference anchor='I-D.vandergaast-edns-client-subnet'>
<front>
<title>Client Subnet in DNS Requests</title>
<author initials='C' surname='Contavalli' fullname='Carlo Contavalli'>
<organization />
</author>
<author initials='W' surname='Gaast' fullname='Wilmer van der Gaast'>
<organization />
</author>
<author initials='S' surname='Leach' fullname='Sean Leach'>
<organization />
</author>
<author initials='E' surname='Lewis' fullname='Edward Lewis'>
<organization />
</author>
<date month='July' day='5' year='2013' />
<abstract><t>This draft defines an EDNS0 extension to carry information about the network that originated a DNS query, and the network for which the subsequent reply can be cached.</t></abstract>
</front>
<seriesInfo name='Internet-Draft' value='draft-vandergaast-edns-client-subnet-02' />
<format type='TXT'
target='http://www.ietf.org/internet-drafts/draft-vandergaast-edns-client-subnet-02.txt' />
</reference>
<reference anchor='I-D.eardley-mptcp-implementations-survey'>
<front>
<title>Survey of MPTCP Implementations</title>
<author initials='P' surname='Eardley' fullname='Philip Eardley'>
<organization />
</author>
<date month='July' day='12' year='2013' />
<abstract><t>This document presents results from the survey to gather information from people who have implemented MPTCP, in particular to help progress the protocol from Experimental to Standards track. The document currently includes answers from four teams: a Linux implementation from UCLouvain, a FreeBSD implementation from Swinburne, an anonymous implementation in a commercial OS, and a NetScalar Firmware implementation from Citrix Systems, Inc. Thank- you! In summary, we have four independent implementations of all the MPTCP signalling messages, with the exception of address management, and some interoperabiity testing has been done by the other three implementations with the 'reference' Linux implementation. So it appears that the RFC is (at least largely) clear and correct. On address management, we have only one implementation of ADD_ADDR with two teams choosing not to implement it. We have one implementation of the working group's coupled congestion control (RFC6356) and none of the MPTCP-aware API (RFC6897). The main suggested improvements are around o how MPTCP falls back (if the signalling is interrupted by a middlebox): (1) corner cases that are not handled properly, (2) at the IETF, the MPTCP community should work with middlebox vendors, either to reduce or eliminate the need for fallback or to understand the middlebox interactions better. o security: both better MPTCP security (perhaps building on SSL) and a lighter weight mechanism, preferably both in one mechanism. It is hoped that the next version can include information from any other implementations. If you are an implementer and want to contribute your answers, please see the -01 version of this document for a blank survey ready to be filled in.</t></abstract>
</front>
<seriesInfo name='Internet-Draft' value='draft-eardley-mptcp-implementations-survey-02' />
<format type='TXT'
target='http://www.ietf.org/internet-drafts/draft-eardley-mptcp-implementations-survey-02.txt' />
</reference>
<reference anchor="MBTest" target="https://bitbucket.org/bhesmans/mbtest">
<front>
<title>MBTest</title>
<author initials="B." surname="Hesmans">
<organization></organization>
</author>
<date year="2013"/>
</front>
</reference>
<reference anchor="tracebox" target="http://www.tracebox.org">
<front>
<title>tracebox</title>
<author initials="G." surname="Detal">
<organization></organization>
</author>
<date year="2013"/>
</front>
</reference>
<reference anchor="HotMiddlebox13" target="http://inl.info.ucl.ac.be/publications/are-tcp-extensions-middlebox-proof">
<front>
<title>Are TCP Extensions Middlebox-proof?</title>
<author initials="B." surname="Hesmans">
<organization></organization>
</author>
<author initials="F." surname="Duchene">
<organization></organization>
</author>
<author initials="C." surname="Paasch">
<organization></organization>
</author>
<author initials="G." surname="Detal">
<organization></organization>
</author>
<author initials="O." surname="Bonaventure">
<organization></organization>
</author>
<date year="2013" month="December"/>
</front>
<seriesInfo name="CoNEXT workshop HotMiddlebox" value=""/>
</reference>
<reference anchor="HotNets" target="http://doi.acm.org/10.1145/1868447.1868457">
<front>
<title>Data center networking with multipath TCP</title>
<author initials="C." surname="Raiciu">
<organization></organization>
</author>
<author initials="C." surname="Pluntke">
<organization></organization>
</author>
<author initials="S." surname="Barre">
<organization></organization>
</author>
<author initials="A." surname="Greenhalgh">
<organization></organization>
</author>
<author initials="D." surname="Wischik">
<organization></organization>
</author>
<author initials="M." surname="Handley">
<organization></organization>
</author>
<date year="2010"/>
</front>
<seriesInfo name="Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks (Hotnets-IX)" value=""/>
</reference>
<reference anchor="IMC11" target="http://doi.acm.org/10.1145/2068816.2068834">
<front>
<title>Is it still possible to extend TCP?</title>
<author initials="M." surname="Honda">
<organization></organization>
</author>
<author initials="Y." surname="Nishida">
<organization></organization>
</author>
<author initials="C." surname="Raiciu">
<organization></organization>
</author>
<author initials="A." surname="Greenhalgh">
<organization></organization>
</author>
<author initials="M." surname="Handley">
<organization></organization>
</author>
<author initials="H." surname="Tokuda">
<organization></organization>
</author>
<date year="2011"/>
</front>
<seriesInfo name="Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference (IMC '11)" value=""/>
</reference>
<reference anchor="IMC13a" target="http://inl.info.ucl.ac.be/publications/revealing-middlebox-interference-tracebox">
<front>
<title>Revealing Middlebox Interference with Tracebox</title>
<author initials="G." surname="Detal">
<organization></organization>
</author>
<author initials="B." surname="Hesmans">
<organization></organization>
</author>
<author initials="O." surname="Bonaventure">
<organization></organization>
</author>
<author initials="Y." surname="Vanaubel">
<organization></organization>
</author>
<author initials="B." surname="Donnet">
<organization></organization>
</author>
<date year="2013"/>
</front>
<seriesInfo name="Proceedings of the 2013 ACM SIGCOMM conference on Internet measurement conference" value=""/>
</reference>
<reference anchor="IMC13b" target="http://doi.acm.org/10.1145/2504730.2504751">
<front>
<title>A measurement-based study of MultiPath TCP performance over wireless network</title>
<author initials="Y." surname="Chen">
<organization></organization>
</author>
<author initials="Y." surname="Lim">
<organization></organization>
</author>
<author initials="R." surname="Gibbens">
<organization></organization>
</author>
<author initials="E." surname="Nahum">
<organization></organization>
</author>
<author initials="R." surname="Khalili">
<organization></organization>
</author>
<author initials="D." surname="Towsley">
<organization></organization>
</author>
<date year="n.d."/>
</front>
<seriesInfo name="Proceedings of the 2013 conference on Internet measurement conference (IMC '13)" value=""/>
</reference>
<reference anchor="IMC13c" target="http://doi.acm.org/10.1145/2504730.2504765">
<front>
<title>From Paris to Tokyo on the suitability of ping to measure latency</title>
<author initials="C." surname="Pelsser">
<organization></organization>
</author>
<author initials="L." surname="Cittadini">
<organization></organization>
</author>
<author initials="S." surname="Vissicchio">
<organization></organization>
</author>
<author initials="R." surname="Bush">
<organization></organization>
</author>
<date year="2013"/>
</front>
<seriesInfo name="Proceedings of the 2013 conference on Internet measurement conference (IMC '13)" value=""/>
</reference>
<reference anchor="Cellnet12" target="http://inl.info.ucl.ac.be/publications/exploring-mobilewifi-handover-multipath-tcp">
<front>
<title>Exploring Mobile/WiFi Handover with Multipath TCP</title>
<author initials="C." surname="Paasch">
<organization></organization>
</author>
<author initials="G." surname="Detal">
<organization></organization>
</author>
<author initials="F." surname="Duchene">
<organization></organization>
</author>
<author initials="C." surname="Raiciu">
<organization></organization>
</author>
<author initials="O." surname="Bonaventure">
<organization></organization>
</author>
<date year="2012"/>
</front>
<seriesInfo name="ACM SIGCOMM workshop on Cellular Networks (Cellnet12)" value=""/>
</reference>
<reference anchor="CSWS14" >
<front>
<title>Experimental Evaluation of Multipath TCP Schedulers</title>
<author initials="C." surname="Paasch">
<organization></organization>
</author>
<author initials="S." surname="Ferlin">
<organization></organization>
</author>
<author initials="O." surname="Alay">
<organization></organization>
</author>
<author initials="O." surname="Bonaventure">
<organization></organization>
</author>
<date year="2014" month="August"/>
</front>
<seriesInfo name="SIGCOMM CSWS2014 workshop" value=""/>
</reference>
<reference anchor="CACM14" target="http://inl.info.ucl.ac.be/publications/multipath-tcp">
<front>
<title>Multipath TCP</title>
<author initials="C." surname="Paasch">
<organization></organization>
</author>
<author initials="O." surname="Bonaventure">
<organization></organization>
</author>
<date year="2014" month="April"/>
</front>
<seriesInfo name="Communications of the ACM, 57(4):51-57" value=""/>
</reference>
<reference anchor="CONEXT13" target="http://inl.info.ucl.ac.be/publications/benefits-applying-experimental-design-improve-multipath-tcp">
<front>
<title>On the Benefits of Applying Experimental Design to Improve Multipath TCP</title>
<author initials="C." surname="Paasch">
<organization></organization>
</author>
<author initials="R." surname="Khalili">
<organization></organization>
</author>
<author initials="O." surname="Bonaventure">
<organization></organization>
</author>
<date year="2013" month="December"/>
</front>
<seriesInfo name="Conference on emerging Networking EXperiments and Technologies (CoNEXT)" value=""/>
</reference>
<reference anchor="NSDI11" >
<front>
<title>Design, implementation and evaluation of congestion control for Multipath TCP</title>
<author initials="D." surname="Wischik">
<organization></organization>
</author>
<author initials="C." surname="Raiciu">
<organization></organization>
</author>
<author initials="A." surname="Greenhalgh">
<organization></organization>
</author>
<author initials="M." surname="Handley">
<organization></organization>
</author>
<date year="2011"/>
</front>
<seriesInfo name="In Proceedings of the 8th USENIX conference on Networked systems design and implementation (NSDI11)" value=""/>
</reference>
<reference anchor="NSDI12" target="http://inl.info.ucl.ac.be/publications/how-hard-can-it-be-designing-and-implementing-deployable-multipath-tcp">
<front>
<title>How Hard Can It Be? Designing and Implementing a Deployable Multipath TCP</title>
<author initials="C." surname="Raiciu">
<organization></organization>
</author>
<author initials="C." surname="Paasch">
<organization></organization>
</author>
<author initials="S." surname="Barre">
<organization></organization>
</author>
<author initials="A." surname="Ford">
<organization></organization>
</author>
<author initials="M." surname="Honda">
<organization></organization>
</author>
<author initials="F." surname="Duchene">
<organization></organization>
</author>
<author initials="O." surname="Bonaventure">
<organization></organization>
</author>
<author initials="M." surname="Handley">
<organization></organization>
</author>
<date year="2012" month="April"/>
</front>
<seriesInfo name="USENIX Symposium of Networked Systems Design and Implementation (NSDI12)" value=""/>
</reference>
<reference anchor="MultipathTCP-Linux" target="http://www.multipath-tcp.org">
<front>
<title>Multipath TCP implementation in the Linux kernel</title>
<author initials="C." surname="Paasch">
<organization></organization>
</author>
<author initials="S." surname="Barre">
<organization></organization>
</author>
<author initials="." surname="et al">
<organization></organization>
</author>
<date year="n.d."/>
</front>
</reference>
<reference anchor="INFOCOM14" >
<front>
<title>Cross-Layer Path Management in Multi-path Transport Protocol for Mobile Devices</title>
<author initials="Y." surname="Lim">
<organization></organization>
</author>
<author initials="Y." surname="Chen">
<organization></organization>
</author>
<author initials="E." surname="Nahum">
<organization></organization>
</author>
<author initials="D." surname="Towsley">
<organization></organization>
</author>
<author initials="K." surname="Lee">
<organization></organization>
</author>
<date year="2014"/>
</front>
<seriesInfo name="IEEE INFOCOM'14" value=""/>
</reference>
<reference anchor="SIGCOMM11" target="http://doi.acm.org/10.1145/2018436.2018467">
<front>
<title>Improving datacenter performance and robustness with multipath TCP</title>
<author initials="C." surname="Raiciu">
<organization></organization>
</author>
<author initials="S." surname="Barre">
<organization></organization>
</author>
<author initials="C." surname="Pluntke">
<organization></organization>
</author>
<author initials="A." surname="Greenhalgh">
<organization></organization>
</author>
<author initials="D." surname="Wischik">
<organization></organization>
</author>
<author initials="M." surname="Handley">
<organization></organization>
</author>
<date year="n.d."/>
</front>
<seriesInfo name="Proceedings of the ACM SIGCOMM 2011 conference" value=""/>
</reference>
<reference anchor="TNC13" >
<front>
<title>Experiences with MPTCP in an intercontinental multipathed OpenFlow network</title>
<author initials="R." surname="van der Pol">
<organization></organization>
</author>
<author initials="M." surname="Bredel">
<organization></organization>
</author>
<author initials="A." surname="Barczyk">
<organization></organization>
</author>
<date year="2013"/>
</front>
<seriesInfo name="TNC2013" value=""/>
</reference>
<reference anchor="PAMS2014" >
<front>
<title>Impact of Path Selection and Scheduling Policies on MPTCP Performance</title>
<author initials="B." surname="Arzani">
<organization></organization>
</author>
<author initials="A." surname="Gurney">
<organization></organization>
</author>
<author initials="S." surname="Cheng">
<organization></organization>
</author>
<author initials="R." surname="Guerin">
<organization></organization>
</author>
<author initials="B." surname="Loo">
<organization></organization>
</author>
<date year="2014"/>
</front>
<seriesInfo name="PAMS2014" value=""/>
</reference>
<reference anchor="IOS7" target="http://support.apple.com/kb/HT5977">
<front>
<title>Multipath TCP Support in iOS 7</title>
<author fullname="Apple">
<organization></organization>
</author>
<date year="2014" month="January"/>
</front>
</reference>
<reference anchor="CONEXT12" >
<front>
<title>MPTCP is not pareto-optimal performance issues and a possible solution</title>
<author initials="R." surname="Khalili">
<organization></organization>
</author>
<author initials="N." surname="Gast">
<organization></organization>
</author>
<author initials="M." surname="Popovic">
<organization></organization>
</author>
<author initials="U." surname="Upadhyay">
<organization></organization>
</author>
<author initials="J.-Y." surname="Leboudec">
<organization></organization>
</author>
<date year="2012"/>
</front>
<seriesInfo name="Proceedings of the 8th international conference on Emerging networking experiments and technologies (CoNEXT12)" value=""/>
</reference>
<reference anchor="ICNP12" >
<front>
<title>Delay-based congestion control for multipath TCP</title>
<author initials="Y." surname="Cao">
<organization></organization>
</author>
<author initials="M." surname="Xu">
<organization></organization>
</author>
<author initials="X." surname="Fu">
<organization></organization>
</author>
<date year="2012"/>
</front>
<seriesInfo name="20th IEEE International Conference on Network Protocols (ICNP)" value=""/>
</reference>
<reference anchor="ICC14" >
<front>
<title>DAPS Intelligent Delay-Aware Packet Scheduling For Multipath Transport</title>
<author initials="N." surname="Kuhn">
<organization></organization>
</author>
<author initials="E." surname="Lochin">
<organization></organization>
</author>
<author initials="A." surname="Mifdaoui">
<organization></organization>
</author>
<author initials="G." surname="Sarwar">
<organization></organization>
</author>
<author initials="O." surname="Mehani">
<organization></organization>
</author>
<author initials="R." surname="Boreli">
<organization></organization>
</author>
<date year="2014"/>
</front>
<seriesInfo name="IEEE ICC 2014" value=""/>
</reference>
<reference anchor="DetalMSS" target="https://listes-2.sipr.ucl.ac.be/sympa/arc/mptcp-dev/2014-09/msg00130.html">
<front>
<title>Adaptive MSS value</title>
<author initials="G." surname="Detal">
<organization></organization>
</author>
<date year="2014" month="September"/>
</front>
<seriesInfo name="Post on the mptcp-dev mailing list" value=""/>
</reference>
<reference anchor="StrangeMbox" target="http://blog.multipath-tcp.org/blog/html/2015/01/30/multipath_tcp_through_a_strange_middlebox.html">
<front>
<title>Multipath TCP through a strange middlebox</title>
<author initials="O." surname="Bonaventure">
<organization></organization>
</author>
<date year="2015" month="January"/>
</front>
<seriesInfo name="Blog post" value=""/>
</reference>
</references>
</back>
</rfc>
| PAFTECH AB 2003-2026 | 2026-04-24 02:55:21 |