One document matched: draft-ietf-tcpimpl-tools-02.txt
Differences from draft-ietf-tcpimpl-tools-01.txt
Network Working Group Steve Parker
Internet Draft Chris Schmechel
Expiration Date: May 1998 Sun Microsystems, Inc.
November 1997
Some Testing Tools for TCP Implementors
<draft-ietf-tcpimpl-tools-02.txt>
1. Status of this Memo
This document is an Internet Draft. Internet Drafts are working
documents of the Internet Engineering Task Force (IETF), its
areas, and its working groups. Note that other groups may also
distribute working documents as Internet Drafts.
Internet Drafts are draft documents valid for a maximum of six
months, and may be updated, replaced, or obsoleted by other docu-
ments at any time. It is inappropriate to use Internet Drafts as
reference material or to cite them other than as ``work in pro-
gress''.
To learn the current status of any Internet Draft, please check
the ``1id-abstracts.txt'' listing contained in the Internet
Drafts shadow directories on ftp.is.co.za (Africa),
nic.nordu.net (Europe), munnari.oz.au (Pacific Rim),
ds.internic.net (US East Coast), or ftp.isi.edu (US West
Coast).
This memo provides information for the Internet community. This
memo does not specify an Internet standard of any kind. Distri-
bution of this memo is unlimited.
2. Introduction
Available tools for testing TCP implementations are catalogued by
this memo. Hopefully disseminating this information will
encourage those responsible for building and maintaining TCP to
make the best use of available tests. The type of testing the
tool provides, the type of tests it is capable of doing, and its
availability is enumerated. This document lists only tools which
can evaluate one or more TCP implementations, or which can privde
some specific results which describe or evaluate the TCP being
tested.
Parker, Schmechel, Editors [Page 1]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
Each tools is defined as follows:
Name
The name associated with the testing tool.
Category
One or more categories of tests which the tools is capable of
providing. Categories used so far: functional correctness, per-
formance, stress.
Description
A description of the tools construction, and the implementation
methodology of the tests.
Automation
What steps are required to complete the test? What human inter-
vention is required?
Availability
How do you retrieve this tool and get more information about it?
Required Environment
Compilers, OS version, etc. required to build and/or run the
associated tool.
References
A list of publications relating to the tool, if any.
3. Tools
3.1. Dbs
Author
Yukio Murayama
Parker, Schmechel, Editors [Page 2]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
Category
Performance / Stress
Description
Dbs is a tool which allows multiple data transfers to be coordi-
nated, and the resulting TCP behavior to be reviewed. Results
are presented as ASCII log files.
Automation
Command of execution is driven by a script file.
Availability
See http://www.ai3.net/products/dbs for details of precise OS
versions supported, and for download of the source code. Current
implementation supports BSDI BSD/OS, Linux, mkLinux, SunOS, IRIX,
Ultrix, NEWS OS, HP-UX. Other environments are likely easy to
add.
Required Environment
C language compiler, UNIX-style socket API support.
3.2. Dummynet
Author
Luigi Rizzo
Category
Functional Correctness / Performance
Description
Dummynet is a tool which simulates the presence of finite size
queues, bandwidth limitations, and communication delays. Dum-
mynet inserts between two layers of the protocol stack (in the
current implementation between TCP and IP), simulating the above
effects in an operational system. This way experiments can be
done using real protocol implementations and real applications,
even running on the same host (dummynet also intercepts communi-
cations on the loopback interface). Reconfiguration of dummynet
parameters (delay, queue size, bandwidth) can be done on the fly
Parker, Schmechel, Editors [Page 3]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
by using a sysctl call. The overhead of dummynet is extremely
low.
Automation
Requires merging diff files with kernel source code. Command-
line driven through the sysctl command to modify kernel vari-
ables.
Availability
See http://www.iet.unipi.it/~luigi/research.html or e-mail Luigi
Rizzo (l.rizzo@iet.unipi.it). Source code is available for
FreeBSD 2.1 and FreeBSD 2.2 (easily adaptable to other BSD-
derived systems).
Required Environment
C language compiler, BSD-derived system, kernel source code.
References
[Riz97]
3.3. Netperf
Author
Rick Jones
Category
Performance
Description
Single connection bandwidth or latency tests for TCP, UDP, and
DLPI. Includes provisions for CPU utilization measurement.
Automation
Requires compilation (K&R C sufficient for all but -DHISTOGRAM,
may require ANSI C in the future) if starting from source. Execu-
tion as child of inetd requires editing of /etc/services and
/etc/inetd.conf. Scripts are provided for a quick look
(snapshot_script), bulk throughput of TCP and UDP, and latency
Parker, Schmechel, Editors [Page 4]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
for TCP and UDP. It is command-line driven.
Availability
See http://www.cup.hp.com/netperf/NetperfPage.html or e-mail Rick
Jones (raj@cup.hp.com). Binaries are available here for HP/UX
Irix, Solaris, and Win32.
Required Environment
C language compiler, POSIX.1, sockets.
3.4. Orchestra
Author
Scott Dawson, Farnam Jahanian, and Todd Mitton
Category
Functional Correctness / Performance
Description
This tool is a library which provides the user with an ability to
build a protocol layer capable of performing fault injection on
protocols. Several fault injection layers have been built using
this library, one of which has been used to test different vendor
implementations of TCP. This is accomplished by probing the ven-
dor implementation from one machine containing a protocol stack
that has been instrumented with Orchestra. A connection is
opened from the vendor TCP implementation to the machine which
has been instrumented. Faults may then be injected at the
Orchestra side of the connection and the vendor TCP's response
may be monitored. The most recent version of Orchestra runs
inside the X-kernel protocol stack on the OSF MK operating sys-
tem.
When using Orchestra to test a protocol, the fault injection
layer is placed below the target protocol in the protocol stack.
This can either be done on one machine on the network, if proto-
col stacks on the other machines cannot be modified (as in the
case of testing TCP), or can be done on all machines on the net-
work (as in the case of testing a protocol under development).
Once the fault injection layer is in the protocol stack, all mes-
sages sent by and destined for the target protocol pass through
Parker, Schmechel, Editors [Page 5]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
it on their way to/from the network. The Orchestra fault injec-
tion layer can manipulate these messages. In particular, it can
drop, delay, re-order, duplicate, or modify messages. It can
also introduce new messages into the system if desired.
The actions of the Orchestra fault injection layer on each mes-
sage are determined by a script, written in Tcl. This script is
interpreted by the fault injection layer when the message enters
the layer. The script has access to the header information about
the message, and can make decisions based on header values. It
can also keep information about previous messages, counters, or
any other data which the script writer deems useful. Users of
Orchestra may also define their own actions to be taken on mes-
sages, written in C, that may be called from the fault injection
scripts.
Automation
Scripts can be specified either using a graphical user interface
which generates Tcl, or by writing Tcl directly. At this time,
post-analysis of the results of the test must also be performed
by the user. Essentially this consists of looking at a packet
trace that Orchestra generates for (in)correct behavior. Must
compile and link fault generated layer with the protocol stack.
Availability
See http://www.eecs.umich.edu/RTCL/projects/orchestra/ or e-mail
Scott Dawson (sdawson@eecs.umich.edu).
Required Environment
OSF MK operating system, or X-kernel like network architecture,
or adapted to network stack.
References
[DJ94], [DJM96a], [DJM96b]
3.5. Packet Shell
Author
Steve Parker and Chris Schmechel
Parker, Schmechel, Editors [Page 6]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
Category
Functional Correctness / Performance
Description
An extensible Tcl/Tk based software toolset for protocol develop-
ment and testing. Tcl (Tool Command Language) is an embeddable
scripting language and Tk is a graphical user interface toolkit
based on Tcl. The Packet Shell creates Tcl commands that allow
you to create, modify, send, and receive packets on networks.
The operations for each protocol are supplied by a dynamic linked
library called a protocol library. These libraries are silently
linked in from a special directory when the Packet Shell begins
execution. The current protocol libraries are: IP, IPv6, IPv6
extensions, ICMP, ICMPv6, Ethernet layer, data layer, file layer
(snoop and tcpdump support), socket layer, TCP, TLI.
It includes harness, which is a Tk based graphical user interface
for creating test scripts within the Packet Shell. It includes
tests for no initial slow start, and retain out of sequence data
as TCP test cases mentioned in [PADHV98].
It includes tcpgraph, which is used with a snoop or tcpdump cap-
ture file to produce a TCP time-sequence plot using xplot.
Automation
Command-line driven through Tcl commands, or graphical user
interface models are available through the harness format.
Availability
See http://playground.sun.com/psh/ or e-mail Steve Parker
(sparker@Eng.Sun.COM) or Chris Schmechel (cschmec@Eng.Sun.COM).
Required Environment
Solaris 2.4 or higher. Porting required for other operating sys-
tems.
3.6. Tcpanaly
Parker, Schmechel, Editors [Page 7]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
Author
Vern Paxson
Category
Functional Correctness / Performance
Description
This is a tool for automatically analyzing a TCP implementation's
behavior by inspecting packet traces of the TCP's activity. It
does so through packet filter traces produced by tcpdump. It has
coded within it knowledge of a large number of TCP implementa-
tions. Using this, it can determine whether a given trace
appears consistent with a given implementation, and, if so,
exactly why the TCP chose to transmit each packet at the time it
did. If a trace is found inconsistent with a TCP, tcpanaly
either diagnoses a likely measurement error present in the trace,
or indicates exactly whether the activity in the trace deviates
from that of the TCP, which can greatly aid in determining how
the traced implementation behaves.
Tcpanaly's category is somewhat difficult to classify, since it
attempts to profile the behavior of an implementation, rather
than to explicitly test specific correctness or performance
issues. However, this profile identifies correctness and perfor-
mance problems.
Adding new implementations of TCP behavior is possible with tcpa-
naly through the use of C++ classes.
Automation
Command-line driven and only the traces of the TCP sending and
receiving bulk data transfers are needed as input.
Availability
Contact Vern Paxson (vern@ee.lbl.gov).
Required Environment
C++ compiler.
Parker, Schmechel, Editors [Page 8]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
References
[Pax97a]
3.7. Tcptrace
Author
Shawn Ostermann
Category
Functional Correctness / Performance
Description
This is a TCP trace file analysis tool. It reads output trace
files in the formats of : tcpdump, snoop, etherpeek, and netm.
For each connection, it keeps track of elapsed time,
bytes/segments sent and received, retransmissions, round trip
times, window advertisements, throughput, etc from simple to very
detailed output.
It can also produce three different types of graphs:
Time Sequence Graph (shows the segments sent and ACKs returned as
a function of time)
Instantaneous Throughput (shows the instantaneous, averaged over
a few segments, throughput of the connection as a function of
time).
Round Trip Times (shows the round trip times for the ACKs as a
function of time)
Automation
Command-line driven, and uses the xplot program to view the
graphs.
Availability
Source code is available, and Solaris binary along with sample
traces. See
http://jarok.cs.ohiou.edu/software/tcptrace/tcptrace.html or e-
mail Shawn Ostermann (ostermann@cs.ohiou.edu).
Parker, Schmechel, Editors [Page 9]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
Required Environment
C compiler, Solaris, FreeBSD, NetBSD, HPUX, Linux.
3.8. Tracelook
Author
Greg Minshall
Category
Functional Correctness / Performance
Description
This is a Tcl/Tk program for graphically viewing the contents of
tcpdump trace files.
Automation
Command-line driven with a graphical user interface for the
graph.
Availability
See http://www.ipsilon.com/~minshall/sw/tracelook/tracelook.html
or e-mail Greg Minshall (minshall@ipsilon.com).
Required Environment
A modern version of awk, and Tcl/Tk (Tk version 3.6 or higher).
The program xgraph is required to view the graphs under X11.
3.9. TReno
Author
Matt Mathis and Jamshid Mahdavi
Category
Performance
Parker, Schmechel, Editors [Page 10]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
Description
This is a TCP throughput measurement tool based on sending UDP or
ICMP packets in patterns that are controlled at the user-level so
that their timing reflects what would be sent by a TCP that
observes proper congestion control (and implements SACK). This
allows it to measure throughput independent of the TCP implemen-
tation of end hosts and serve as a useful platform for prototyp-
ing TCP changes.
Automation
Command-line driven. No "server" is required, and it only
requires a single argument of the machine to run the test to.
Availability
See http://www.psc.edu/networking/treno_info.html or e-mail Matt
Mathis (mathis@psc.edu) or Jamshid Mahdavi (mahdavi@psc.edu).
Required Environment
C compiler, POSIX.1, raw sockets.
3.10. Ttcp
Author
Unknown
Category
Performance
Description
Originally written to move files around, ttcp became the classic
throughput benchmark or load generator, with the addition of sup-
port for sourcing to/from memory. It has spawned many variants,
recent ones include support for UDP, data pattern generation,
page alignment, and even alignment offset control.
Automation
Command-line driven.
Parker, Schmechel, Editors [Page 11]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
Availability
See ftp://ftp.arl.mil/pub/ttcp/ or e-mail ARL (ftp@arl.mil) which
include the most common variants available.
Required Environment
C compiler, BSD sockets.
3.11. Xplot
Author
Tim Shepard
Category
Functional Correctness / Performance
Description
This is a fairly conventional graphing/plotting tool (xplot
itself), a script to turn tcpdump output into xplot input, and
some sample code to generate xplot commands to plot collected TCP
data.
Automation
Command-line driven with a graphical user interface for the plot.
Availability
See ftp://mercury.lcs.mit.edu/pub/shep/xplot.tar.gz or e-mail Tim
Shepard (shep@lcs.mit.edu).
Required Environment
C compiler, X11.
References
[She91]
Parker, Schmechel, Editors [Page 12]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
4. Summary
This draft lists all TCP tests and testing tools reported to the
authors as part of TCP Implementer's working group and is not
exhaustive. These tools have been verified as available by the
authors.
5. Security Considerations
This version of this memo does not discuss any security-related
issues though general packet manipulation and packet tracing can
raise significant security issues. These issues may be covered
in a later draft.
6. References
[DJ94] Scott Dawson and Farnam Jahanian, "Probing and Fault
Injection of Distributed Protocol Implementations",
University of Michigan Technical Report CSE-TR-217-94,
EECS Department.
[DJM96a] Scott Dawson, Farnam Jahanian, and Todd Mitton,
"ORCHESTRA: A Fault Injection Environment for Distri-
buted Systems", University of Michigan Technical Report
CSE-TR-318-96, EECS Department.
[DJM96b] Scott Dawson, Farnam Jahanian, and Todd Mitton, "Exper-
iments on Six Commercial TCP Implementations Using a
Software Fault Injection Tool", University of Michigan
Technical Report CSE-TR-298-96, EECS Department.
[Pax97a] Vern Paxson, "Automated Packet Trace Analysis of TCP
Implementations", ACM SIGCOMM '97, September 1997,
Cannes, France.
[PADHV98] V. Paxson, M. Allman, S. Dawson, I. Heavens, and B.
Volz, "Known TCP Implementation Problems", Work In Pro-
gress, March, 1998.
[Riz97] Luigi Rizzo, "Dummynet: a simple approach to the
evaluation of network protocols", ACM Computer Communi-
cation Review, Vol. 27, N. 1, January 1997, pp. 31-41.
[She91] Tim Shepard, "TCP Packet Trace Analysis", MIT Labora-
tory for Computer Science MIT-LCS-TR-494, February,
Parker, Schmechel, Editors [Page 13]
INTERNET-DRAFT Some Testing Tools for TCP Implementors November 1997
1991.
7. Author's Address
Steve Parker <sparker@Eng.Sun.COM>
Sun Microsystems, Inc.
901 San Antonio Road, UMPK17-202
Palo Alto, CA 94043
USA
Phone: (650) 786-5176
Chris Schmechel <cschmec@Eng.Sun.COM>
Sun Microsystems, Inc.
901 San Antonio Road, UMPK17-202
Palo Alto, CA, 94043
USA
Phone: (650) 786-4053
Parker, Schmechel, Editors [Page 14]
| PAFTECH AB 2003-2026 | 2026-04-23 08:40:33 |