One document matched: draft-penno-ledbat-app-practices-recommendations-00.txt
Transport Working Group R. Penno
Internet Draft S. Raghunath
Intended status: Informational Juniper Networks
Expires: September 2009 J. Iyengar
Franklin & Marshall College
March 3, 2009
LEDBAT Practices and Recommendations
draft-penno-ledbat-app-practices-recommendations-00
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on September 3, 2009.
Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document.
Penno Expires September 3, 2009 [Page 1]
Internet-Draft LEDBAT Practices and Recommendations March 2009
Abstract
Applications routinely open multiple TCP connections. For example,
P2P applications maintain connections to a number of different peers
while web browsers perform concurrent download from the same web
server. Application designers pursue different goals when doing so:
P2P apps need to maintain a well-connected mesh in the swarm while
web browsers mainly use multiple connections to parallelize requests
that involve application latency on the web server side. But this
practice also has impacts to the host and the network as a whole. For
example, an application can obtain a larger fraction of the
bottleneck than if it had used fewer connections. Although capacity
is the most commonly considered bottleneck resource, middlebox state
table entries are also an important resource for an end system
communication.
This documents clarifies the current practices of application design
and reasons behind them, and discusses the tradeoffs surrounding the
use of many concurrent TCP connections to one destination and/or to
different destinations. Other resource types may exist, and the
guidelines are expected to comprehensively discuss them.
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119
Table of Contents
1. Introduction 3
2. Terminology 4
3. Multiple TCP Connections Advantages 4
3.1. Avoiding head-of-line blocking 4
3.2. Logical partitioning at application level 5
3.3. Multiple streams with different properties 5
3.4. Signaling application layer request completion 5
3.5. High bandwidth-delay links 5
3.6. Error resiliency and reliability 5
3.7. Leveraging multiple processors in a system 5
4. Multiple TCP connections Disadvantages 5
4.1. Additional connection setup overhead 6
4.2. Memory Space 6
4.3. Link Bandwidth 6
4.4. Middleboxes 7
5. Recommendations For Application Designers 7
Penno Expires September 3, 2009 [Page 2]
Internet-Draft LEDBAT Practices and Recommendations March 2009
5.1. Diffserv 7
5.2. Window scale negotiation 8
5.3. Choice of number of multiple connections 8
6. Security Considerations 8
7. IANA Considerations 8
8. Conclusions 8
9. Acknowledgments 8
10. References 9
10.1. Normative References 9
10.2. Informative References 9
Author's Addresses 10
1. Introduction
The use of P2P protocols by end users is widespread. These protocols
are meant to exchange, replicate, stream or download files with
little human intervention, trying at the same time to minimize the
download time of the files requested by any single peer. This is done
by opening several connections to multiple peers and downloading one
or more chunks of the file from each one, selecting faster peers,
amongst others.
If we assume that in any file transfer the bottleneck is on the
uploading peer or server side, end users that utilize P2P clients in
general download the file faster and consume more bandwidth within a
specific timeframe than traditional client-server applications. P2P
clients can overcome the server side bottleneck by opening multiple
connections to different peers. Users of P2P applications also
consume bandwidth throughout the whole day since even after a file is
fully downloaded it will continue to be shared with others users
increasing the upstream bandwidth.
We can see then that the advantages of P2P applications come from the
fact that they open multiple TCP connection to different peers in
order to download multiple pieces of a file in parallel, always look
for faster peers.
But the use of multiple TCP connections by an application is not new.
Web Browsers have being it for a decade. But these are usually short-
lived connections as opposed to long-lived connections. A long-lived
connection in this document should be interpreted as strictly
defined, i.e., a TCP connection that is simply in the established
state, but not necessarily continuously transferring data. In the
case of P2P protocols, e.g. Bittorrent, at any point in time a
fraction of these connections is actually sending or receiving data,
the others are idle or exchange occasional control information.
Penno Expires September 3, 2009 [Page 3]
Internet-Draft LEDBAT Practices and Recommendations March 2009
With the popularity of P2P applications, which maintain hundreds of
long-lived TCP connections to multiple hosts, the issue applications
making use of multiple TCP connections has been gaining attention.
This documents clarifies the current practices of application design
and reasons behind them, and discusses the tradeoffs surrounding the
use of many concurrent TCP connections to one destination and/or to
different destinations. Other resource types may exist, and the
guidelines are expected to comprehensively discuss them.
2. Terminology
Bandwidth: A measure of the amount of data that can be transferred
per second. So, if a 1Gb file were transferred within one second the
bandwidth consumption during the transfer would be 1Gb/s. If it were
transferred within a day, it would be approximately 0.0002Gb/s.
Volume: The total number of bytes transferred during a long time
period. In both examples above the volume within a day would have
been 1Gb.
Capacity: The maximum bandwidth a link can sustain continuously.
Long-lived connection: A TCP connection that is in the established
state but not necessarily continuously transferring data.
3. Multiple TCP Connections Advantages
There are good reasons for an application to use multiple TCP
connections. P2P apps need to maintain a well-connected mesh in the
swarm while web browsers mainly use multiple connections to
parallelize requests that involve application latency on the web
server side
But from a P2P standpoint multiple TCP connections are at the heart
of its functionality. Multiple connections allow for multiple
simultaneous downloads, which improve reliability and speed. Multiple
connections also allow more effective discovery of new peers, and
effective peer-to-peer communication, which allows exchange of
information such as which pieces of a file a client has and is
available.
3.1. Avoiding head-of-line blocking
Web browsers started using multiple TCP connections partly because of
this reason [STEVENS]. This is especially true when the multiple TCP
connections are between a pair of hosts.
Penno Expires September 3, 2009 [Page 4]
Internet-Draft LEDBAT Practices and Recommendations March 2009
3.2. Logical partitioning at application level
Some applications such as FTP use a separate connection for control
and data transfers. The advantage is that this allows a model where
the data transfer is actually happening between hosts that are not
local (see [RFC959], sections 2.3 & 5.2).
3.3. Multiple streams with different properties
The application may need different properties on multiple streams of
data (e.g., Nagle's algorithm, socket buffer sizes etc).
3.4. Signaling application layer request completion
If the application assumes that connection close indicates the
completion of a request, it becomes necessary to have new connections
for multiple requests. This was a reason for multiple connections in
HTTP 1.0.
3.5. High bandwidth-delay links
In the presence of a large bandwidth-delay product, the 16-bit window
size parameter in TCP header does not allow the application to fully
utilize the link. In such situations, one option is to negotiate the
Window Scale Option [RFC1323] In addition multiple TCP connections
allow the application to achieve an effectively larger window size so
that it can better utilize a link with high bandwidth-delay product
(e.g. iSCSI [SCSIREF]).
3.6. Error resiliency and reliability
When multiple connections are used to download a single file or
webpage, for instance, there is lesser chance of a single failure on
one connection having a negative impact on the whole download.
Especially with P2P applications, this makes the network robust to
failures and churn in participants.
3.7. Leveraging multiple processors in a system
With multiple processor systems, there can be higher performance with
parallelism and multiple connections spread over different
processors.
4. Multiple TCP connections Disadvantages
Every connected application on the Internet competes for resources.
This is not specific to applications that open multiple TCP
Penno Expires September 3, 2009 [Page 5]
Internet-Draft LEDBAT Practices and Recommendations March 2009
connections. The use of multiple TCP connections just amplifies the
issue. In the following sections we discuss these resources and how
they are amplified by an application opening multiple connections.
4.1. Additional connection setup overhead
TCP's mechanisms for starting up the connection and then probing the
available bandwidth have to be repeated for each new connection. So
there may be lesser leverage of network information. There is also
the overhead of additional control traffic that may have been
avoided.
4.2. Memory Space
Each TCP connection needs a TCP control block (TCB) or equivalent to
keep state about its connection. In operating systems where the TCP
stack is part of the kernel, this would come from the kernel memory
space, otherwise from userland memory.
But irrespective from where the memory comes from a TCP control block
requires a significant amount of memory. This is significant issue
for devices that terminate TCP connections from multiple end hosts to
provide functions such as Load-Balancing, Gateway and Tunneling.
Some proposals have been put forward to reduce the amount of memory
occupied by each TCP control block [RFC2140], but the issue remains
significant and is amplified by applications that use multiple TCP
connections.
4.3. Link Bandwidth
The bottlenecks for these N multiple connections could be shared or
separate. If separate, there's no specific bottleneck where the
connections are hogging bandwidth. But from a network resource point
of view, the application download still gets multiple shares.
If some/all bottlenecks are shared, then two possibilities exist for
shared bottleneck
- bottleneck is a last-hop link (user traffic dominates link), OR
- bottleneck is in-network wide-area link (background traffic
dominates link)
If bottleneck is the last-hop, then n transport connections compete
with each other and share link bandwidth.
Penno Expires September 3, 2009 [Page 6]
Internet-Draft LEDBAT Practices and Recommendations March 2009
Although these connections might impact delay-sensitive traffic and
increase delay, in the last hop they only affect the end end-user,
which is in control of which applications run on its host. In this
case the user has the option of manually choosing when to run each
application, configuring the end host, amongst others. Alternatively,
or in conjunction with the above, the application can be enhanced to
use Diffserv and new delay sensitive congestion mechanisms.
If the shared bottleneck is in-network, then the application gets
unfair share of bottleneck bandwidth. This impacts flows belonging to
other users in general, and most importantly delay-sensitive traffic.
4.4. Middleboxes
Middleboxes are defined as any intermediary box performing functions
apart from normal, standard functions of an IP router on the data
path between a source host and destination host [RFC3234].
Middleboxes can be stand-alone or integrated in another device such
as a router or modem.
The functions that are relevant to this discussion are those that
require the middlebox to keep per session state, sometimes referred
as transformation services. Some of these functions are, for example,
NAT, Intrusion Detection and Load-Balancing.
It is easy to see that the more sessions a host initiates, the more
state the middlebox will have to keep. The relationship is at least
1:1 but due asymmetric traffic, routing changes and others, this can
be 1:N.
Although application traffic from most broadband subscribers today go
through at least one middlebox (integrated into the broadband modem),
it can traverse other middleboxes that reside within the ISP's
network or close the destination. These middleboxes aggregate traffic
from multiple subscribers and state tables within these devices can
become a premium.
5. Recommendations For Application Designers
5.1. Diffserv
Applications involved in bulk data transfer with low priority in time
could mark their packets according with the guidelines of RFC 3662
[RFC2662].
Penno Expires September 3, 2009 [Page 7]
Internet-Draft LEDBAT Practices and Recommendations March 2009
5.2. Window scale negotiation
Where appropriate, sender & receiver window should be scaled using
RFC1323 based negotiation in order to make the best use of network
resources. Recommendations to adjust window size are not new and have
been recommended in networks where the BDP (Bandwidth Delay Product)
is large [RFC3481].
5.3. Choice of number of multiple connections
In the event that an application decides to employ multiple parallel
TCP connections, it is recommended empirical investigations be
carried out to find a threshold beyond which additional connections
yield little benefit.
This kind of investigation is also worth having with existing
applications that employ multiple connections. In particular with P2P
systems, it would be useful to understand the impact on the health of
the swarm with the number of connections.
6. Security Considerations
None at this time
7. IANA Considerations
None at this time
8. Conclusions
TBD
9. Acknowledgments
Joe Touch
Penno Expires September 3, 2009 [Page 8]
Internet-Draft LEDBAT Practices and Recommendations March 2009
10. References
10.1. Normative References
[RFC959] J. Postel and J. Reynolds, "File Transfer Protocol (FTP)",
RFC 959 (1985).
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
and W. Weiss, "An Architecture for Differentiated
Services", RFC 2475, December 1998.
10.2. Informative References
[RFC1323] V. Jacobson, B. Braden, D. Borman, TCP Extensions for High
Performance, RFC 1323, May 1992
[RFC2140] J. Touch, TCP Control Block Interdependence, RFC 2140
(1997)
[RFC2616] R. Fielding et al, Hypertext Transfer Protocol - HTTP/1.1,
RFC 2616 (1999)
[RFC3481] Inamura, H., Montenegro, G., Ludwig, R., Gurtov, A., and F.
Khafizov, "TCP over Second (2.5G) and Third (3G) Generation
Wireless Networks", BCP 71, RFC 3481, February 2003.
[RFC3234] Carpenter, B. and S. Brim, "Middleboxes: Taxonomy and
Issues", RFC 3234, February 2002.
[RFC3662] Bless, R., Nichols, K., and K. Wehrle, "A Lower Effort
Per-Domain Behavior (PDB) for Differentiated Services", RFC
3662, December 2003.
[SCSIREF] K.Z. Meth, J. Satran, Design of the iSCSI protocol, Storage
Conference (2003)
[STEVENS] W. Richard Stevens et al, "Unix Network Programming, The
Sockets Networking API", Volume 1, Third Edition (2003),
section 10.5, page 293.
Penno Expires September 3, 2009 [Page 9]
Internet-Draft LEDBAT Practices and Recommendations March 2009
Author's Addresses
Reinaldo Penno
Juniper Networks
1194 N Mathilda Aveue
Sunnyvale, CA
Email: rpenno@juniper.net
Satish Raghunath
Juniper Networks
1194 N Mathilda Aveue
Sunnyvale, CA
Email: satishr@juniper.net
Jana Iyengar
Franklin & Marshall College
Email: jiyengar@fandm.edu
Penno Expires September 3, 2009 [Page 10]
| PAFTECH AB 2003-2026 | 2026-04-24 05:57:17 |