One document matched: draft-ietf-ipfix-file-02.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
<!ENTITY draftIpfixArch PUBLIC "" "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-ipfix-arch.xml">
<!ENTITY draftIpfixAS PUBLIC "" "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-ipfix-as.xml">
<!ENTITY draftIpfixTesting PUBLIC "" "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-ipfix-testing.xml">
<!ENTITY draftIpfixRR PUBLIC "" "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-ipfix-reducing-redundancy.xml">
<!ENTITY draftIpfixET PUBLIC "" "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-ipfix-exporting-type.xml">
<!ENTITY rfc1321 PUBLIC "" "http://xml.resource.org/public/rfc/bibxml/reference.RFC.1321.xml">
<!ENTITY rfc2119 PUBLIC "" "http://xml.resource.org/public/rfc/bibxml/reference.RFC.2119.xml">
<!ENTITY rfc3917 PUBLIC "" "http://xml.resource.org/public/rfc/bibxml/reference.RFC.3917.xml">
<!ENTITY rfc3954 PUBLIC "" "http://xml.resource.org/public/rfc/bibxml/reference.RFC.3954.xml">
<!ENTITY rfc5101 PUBLIC "" "http://xml.resource.org/public/rfc/bibxml/reference.RFC.5101.xml">
<!ENTITY rfc5102 PUBLIC "" "http://xml.resource.org/public/rfc/bibxml/reference.RFC.5102.xml">
<!ENTITY rfc5103 PUBLIC "" "http://xml.resource.org/public/rfc/bibxml/reference.RFC.5103.xml">
]>
<rfc ipr="full3978" category="std" docName="draft-ietf-ipfix-file-02.txt">
<?rfc compact="yes"?>
<?rfc subcompact="no"?>
<?rfc toc="yes"?>
<?rfc symrefs="yes"?>
<front>
<title abbrev="IPFIX Files">
An IPFIX-Based File Format
</title>
<author initials="B." surname="Trammell" fullname="Brian Trammell">
<organization abbrev="Hitachi Europe">
Hitachi Europe
</organization>
<address>
<postal>
<street>c/o ETH Zurich</street>
<street>Gloriastrasse 35</street>
<city>8092 Zurich</city>
<country>Switzerland</country>
</postal>
<phone>+41 44 632 70 13</phone>
<email>brian.trammell@hitachi-eu.com</email>
</address>
</author>
<author initials="E." surname="Boschi" fullname="Elisa Boschi">
<organization abbrev="Hitachi Europe">
Hitachi Europe
</organization>
<address>
<postal>
<street>c/o ETH Zurich</street>
<street>Gloriastrasse 35</street>
<city>8092 Zurich</city>
<country>Switzerland</country>
</postal>
<phone>+41 44 632 70 57</phone>
<email>elisa.boschi@hitachi-eu.com</email>
</address>
</author>
<author initials="L." surname="Mark" fullname="Lutz Mark">
<organization abbrev="Fraunhofer IFAM">
Fraunhofer IFAM
</organization>
<address>
<postal>
<street>Weiner Str. 12</street>
<city>38259 Bremen</city>
<country>Germany</country>
</postal>
<phone>+49 421 2246206</phone>
<email>lutz.mark@ifam.fraunhofer.de</email>
</address>
</author>
<author initials="T." surname="Zseby" fullname="Tanja Zseby">
<organization abbrev="Fraunhofer FOKUS">
Fraunhofer Institute for Open Communication Systems
</organization>
<address>
<postal>
<street>Kaiserin-Augusta-Allee 31</street>
<city>10589 Berlin</city>
<country>Germany</country>
</postal>
<phone>+49 30 3463 7153</phone>
<email>tanja.zseby@fokus.fraunhofer.de</email>
</address>
</author>
<author initials="A." surname="Wagner" fullname="Arno Wagner">
<organization abbrev="ETH Zurich">
Swiss Federal Institute of Technology Zurich
</organization>
<address>
<postal>
<street>Gloriastrasse 35</street>
<city>8092 Zurich</city>
<country>Switzerland</country>
</postal>
<phone>+41 44 632 70 04</phone>
<email>arno@wagner.name</email>
</address>
</author>
<date month="July" day="14" year="2008"/>
<area>Operations</area>
<workgroup>IPFIX Working Group</workgroup>
<abstract>
<t>This document describes a file format for the storage of flow data
based upon the IPFIX Message format. It proposes a set of requirements
for flat-file, binary flow data file formats, then applies the IPFIX
message format to these requirements to build a new file format. This
IPFIX-based file format is designed to facilitate interoperability and
reusability among a wide variety of flow storage, processing, and
analysis tools.</t>
</abstract>
</front>
<middle>
<section title="Introduction" anchor="intro-sec">
<t>This document proposes a file format based upon IPFIX, designed to
facilitiate interoperability and reusability among a wide variety of
flow storage, processing, and analysis tools. It begins with an overview
of the IPFIX File format, and a quick summary of how IPFIX Files work in
<xref target="design-section"/>. It then explores the motivation for
proposing a standardized flow file format and using IPFIX as the basis
for this new file format in <xref target="motivation-section"/>. It
outlines a set of requirements for standardized flow storage in <xref
target="requirements-section"/>, and explores the applicability of such
a format to various specific application areas in <xref
target="applicability-section"/>.</t>
<t>The file format specification follows in <xref
target="detail-section"/>, including specifications of readers and
writers of these files, and additional specifications that apply in
specific use cases. This format makes use of the IPFIX Options mechanism
for additional file metadata, in order to avoid requiring any protocol
or message format extensions, and to minimize the effort required to
adapt IPFIX implementations to use the file format; a detailed
definition of the Options Templates used for storage metedata appears in
<xref target="templates-section"/>.</t>
<t><xref target="errors-section"/> and <xref
target="integration-section"/> provide specific recommendations for
error resilience during long-term storage and integration of IPFIX File
data with other formats. <xref target="examples-section"/> contains a
detailed example IPFIX File.</t>
<section title="IPFIX Documents Overview" anchor="intro-docs">
<t><xref target="RFC5101">"Specification of the IPFIX Protocol for the
Exchange of IP Traffic Flow Information"</xref> and its associated
documents define the IPFIX Protocol, which provides network engineers
and administrators with access to IP traffic flow information.</t>
<t><xref target="I-D.ietf-ipfix-arch">"Architecture for IP Flow
Information Export"</xref> defines the architecture for the export of
measured IP flow information out of an IPFIX Exporting Process to an
IPFIX Collecting Process, and the basic terminology used to describe
the elements of this architecture, per the requirements defined in
<xref target="RFC3917">"Requirements for IP Flow Information
Export"</xref>. <xref target="RFC5101"/> then covers the details of
the method for transporting IPFIX Data Records and Templates via a
congestion-aware transport protocol from an IPFIX Exporting Process to
an IPFIX Collecting Process.</t>
<t><xref target="RFC5102">"Information Model for IP Flow Information
Export"</xref> describes the Information Elements used by IPFIX,
including details on Information Element naming, numbering, and data
type encoding. Finally, <xref target="I-D.ietf-ipfix-as">"IPFIX
Applicability"</xref> describes the various applications of the IPFIX
protocol and their use of information exported via IPFIX, and relates
the IPFIX architecture to other measurement architectures and
frameworks.</t>
<t>In addition, <xref
target="I-D.ietf-ipfix-exporting-type">"Exporting Type Information for
IPFIX Information Elements"</xref> specifies a method for encoding
Information Model properties within an IPFIX Message stream.</t>
<t>This document references <xref target="RFC5101"/> and the
architecture document for terminology, defines IPFIX File Writer and
IPFIX File Reader in terms of the IPFIX Exporting Processes and IPFIX
Collecting Process definitions from <xref target="RFC5101"/>, and
extends the IPFIX Information Model defined in <xref
target="RFC5102"/> to provide new Information Elements for IPFIX File
metadata. It uses the method described in <xref
target="I-D.ietf-ipfix-exporting-type">"Exporting Type Information for
IPFIX Information Elements"</xref> document to support the
self-description of IPFIX Files containing enterprise-specific
Information Elements.</t>
</section>
</section>
<section title="Terminology" anchor="terminology-section">
<t>Terms used in this document that are defined in the Terminology
section of <xref target="RFC5101"/> are to be interpreted as defined
there.</t>
<list style="hanging">
<t hangText="IPFIX File: ">An IPFIX File is a serialized stream of
IPFIX Messages stored on a filesystem. Any IPFIX Message stream that
would be considered valid when transported one or more of the
specified IPFIX transports (SCTP, TCP, or UDP) as defined in <xref
target="RFC5101"/> is considered an IPFIX File for purposes of this
document; however, this document extends that definition with
recommendations on the construction of IPFIX Files that meet the
requirements identified herein.</t>
<t hangText="IPFIX File Reader: ">An IPFIX File Reader is a Process
which reads IPFIX Files from a filesystem, and is analogous to an
IPFIX Collecting Process. An IPFIX File Reader MUST behave as an IPFIX
Collecting Process as outlined in <xref target="RFC5101"/>, except as
modified by this document.</t>
<t hangText="IPFIX File Writer: ">An IPFIX File Writer is a process
which writes IPFIX Files to a filesystem, and is analogous to an IPFIX
Exporting Process. An IPFIX File Writer MUST behave as an IPFIX
Exporting Process as outlined in <xref
target="RFC5101"/>, except
as modified by this document.</t>
</list>
<t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in <xref
target="RFC2119"/>.</t>
</section>
<section title="Design Overview" anchor="design-section">
<t>An IPFIX File, as defined by this document, is simply a stream
containing one or more IPFIX Messages serialized to some filesystem.
Though any set of valid IPFIX Messages can be serialized into an IPFIX
File, the specification proposes guidelines designed to ease storage and
retrieval of flow data using the format.</t>
<t>IPFIX Files contain only IPFIX Messages; any file metadata such as
checksums or export session details are stored using Options within the
IPFIX Message. This design has several advantages, including complete
compatibility with the IPFIX Protocol on the wire and free
manipulability of IPFIX Files through concatenation, appending, and
splitting (on IPFIX Message boundaries). A schematic of a typical file
is shown below:</t>
<figure title="Typical File Structure" anchor="design-structure">
<artwork><![CDATA[
+=======================================+
| IPFIX File |
| +===================================+ |
| | IPFIX Message | |
| | +-------------------------------+ | |
| | | Options Template Set | | |
| | | Options Template Record | | |
| | | . . . | | |
| | +-------------------------------+ | |
| | +-------------------------------+ | |
| | | Template Set | | |
| | | Template Record | | |
| | | . . . | | |
| | +-------------------------------+ | |
| +===================================+ |
| | IPFIX Message | |
| | +-------------------------------+ | |
| | | Data Set | | |
| | | Data Record | | |
| | | . . . | | |
| | +-------------------------------+ | |
| | +-------------------------------+ | |
| | | Data Set | | |
| | | Data Record | | |
| | | . . . | | |
| | +-------------------------------+ | |
| | . . . | |
| +===================================+ |
| . . . |
+=======================================+
]]></artwork>
</figure>
<t>See <xref target="detail-section"/> for details of the implementation
of this design, including specific requirements and guidelines for File
Readers and File Writers, and Information Elements and Options Templates
used for file metadata.</t>
</section>
<section title="Motivation" anchor="motivation-section">
<t>There are a wide variety of applications for the file-based storage
of IP flow data, across a continuum of time scales. Tools used in the
analysis of flow data and creation of analysis products often use files
as a convenient unit of work, with an ephemeral lifetime. A set of flows
relevant to a security investigation may be stored in a file for the
duration of that investigation, and further exchanged among incident
handlers via email or within an external incident handling workflow
application. Sets of flow data relevant to Internet measurement research
may be published as files, much as libpcap packet trace files are, to
provide common data sets for the repeatability of research efforts;
these files would have lifetimes measured in months or years.
Operational flow measurement systems also have a need for long-term,
archival storage of flow data, either as a primary flow data repository,
or as a backing tier for online storage in a relational database
management system (RDBMS).</t>
<t>The variety of applications of flow data, and the variety of
presently deployed storage approaches, would seem to indicate the need
for a standard approach to flow storage with applicability across the
continuum of time scales over which flow data is stored. A storage
format based around flat files would best address the variety of storage
requirements. While much work has been done on structured storage via
RDBMS, relational database systems are not a good basis for format
standardization owing to the fact that their internal data structures
are generally private to a single implementation and subject to change
for internal reasons. Also, there are a wide variety of operations
available on flat files, and external tools and standards can be
leveraged to meet file-based flow storage requirements. Further, flow
data is often not very semantically complicated, and is managed in very
high volume; therefore, an RDBMS-based flow storage system would not
benefit much from the advantages of relational database technology.</t>
<t>The simplest way to create a new file format is simply to serialize
some internal data model to disk, with either textual or binary
representation of data elements, and some framing strategy for
delimiting fields and records. "Ad-hoc" file formats such as this have
several important disadvantages. They impose the semantics of the data
model from which they are derived on the file format, and as such, they
are difficult to extend, describe, and standardize.</t>
<t>Indeed, one de facto standard for the storage of flow data is one of
these ad-hoc formats. A common method of storing data collected via
Cisco NetFlow V5 or V7 is to serialize a stream of raw NetFlow datagrams
into files. These NetFlow PDU files consist of a collection of
header-prefixed blocks (corresponding to the datagrams as received on
the wire) containing fixed-length binary flow records. NetFlow V5 and V7
data may be mixed within a given file, as the header on each datagram
defines the NetFlow version of the records following; there is indeed
very little difference between the two record formats. While this
NetFlow PDU file format has all the disadvantages of an ad-hoc format,
and is not extensible to data models other than that defined by Cisco
NetFlow, it is at least reasonably well-understood due to its
ubiquity.</t>
<t>Over the past decade XML markup has emerged as a new "universal"
representation format for structured data. It is intended to be
human-readable; indeed, that is one reason for its rapid adoption.
However XML has limited usefulness for representing network flow data.
Network flow data has a simple, repetitive, non-hierarchical structure
that does not benefit much from XML. An XML representation of flow data
would be an essentially flat list of the attributes and their values for
each flow record.</t>
<t>The XML approach to data encoding is very heavyweight when compared
to binary flow encoding. XML's use of start- and end-tags, and
plain-text encoding of the actual values, leads to significant
inefficiency in encoding size. Typical network flow datasets can contain
millions or billions of flows per hour of traffic represented. Any
increase in storage size per record can have dramatic impact on flow
data storage and transfer sizes. While data compression algorithms can
partially remove the redundancy introduced by XML encoding, they
introduce additional overhead of their own.</t>
<t>A further problem is that XML processing tools require a full XML
parser. XML parsers are fully general and therefore complex,
resource-intensive and relatively slow, introducing significant
processing time overhead for large network-flow datasets. In contrast,
parsers for typical binary flow data encodings are simply structured,
since they only need to parse a very small header and then have complete
knowledge of all following fields for the particular flow. These can
then be read in a very efficient linear fashion.</t>
<t>This leads us to propose the IPFIX Message format as the basis for a
new flow data file format. The IPFIX working group, in defining the
IPFIX protocol, has already defined an information model and data
formatting rules for representation of flow data. Especially at shorter
time scales, when a file is a unit of data interchange, the filesystem
may be viewed as simply another IPFIX Message transport between
processes. This format is especially well suited to representing flow
data, as it was designed specifically for flow data export; it is easily
extensible unlike ad-hoc serialization, and compact unlike XML. In
addition, IPFIX is an IETF standard for the export and collection of
flow data; using a common format for storage and analysis at the
collection side allows implementors to use substantially the same
information model and data formatting implementation for transport as
well as storage.</t>
</section>
<section title="Requirements" anchor="requirements-section">
<t>In this section, we outline a proposed set of <xref
target="SAINT2007">requirements</xref> for any persistent storage format
for flow data. First and foremost, a flow data file format should
support storage across the continuum of time scales important to flow
storage applications. Each of the requirements enumerated in the
sections below is broadly applicable to flow storage applications,
though each may be more important at certain time scales. For each, we
first identify the requirement, then explain how the IPFIX Message
format addresses it, or briefly outline the changes that must be made in
order for an IPFIX-based file format to meet the requirement.</t>
<section title="Record Format Flexibility">
<t>Due to the wide variety of flow attributes collected by different
network flow attribute measurement systems, the ideal flow storage
format will not impose a single data model or a specific record type
on the flows it stores. The file format must be flexible and
extensible; that is, it must support the definition of multiple record
types within the file itself, and must be able to support new field
types for data within the records in a graceful way.</t>
<t>IPFIX provides record format flexibility through the use of
Templates to describe each Data Record, through the use of an IANA
Registry to define its Information Elements, and through the use of
enterprise-specific Information Elements.</t>
</section>
<section title="Self Description">
<t>Archived data may be read at a time in the future where any
external reference to the meaning of the data may be lost. The ideal
flow storage format should be self-describing; that is, a process
reading flow data from storage should be able to properly interpret
the stored flows without reference to anything other than standard
sources (e.g., the standards document describing the file format) and
the stored flow data itself.</t>
<t>The IPFIX Message format is partially self-describing; that is,
IPFIX Templates containing only IANA-assigned Information Elements can
be completely interpreted according to the IPFIX Information Model
without additional external data.</t>
<t>However, Templates containing private information elements lack
detailed type and semantic information; a Collecting Process receiving
data described by a template containing private Information Elements
it does not understand can only treat the data contained within those
Information Elements as octet arrays. To be fully self-describing,
enterprise-specific Information Elements must be additionally
described via IPFIX Options according to the Information Element
Type Options Template defined in <xref
target="I-D.ietf-ipfix-exporting-type">"Exporting Type Information
for IPFIX Information Elements"</xref>.</t>
</section>
<section title="Data Compression">
<t>Regardless of the representation format, flow data describing
traffic on real networks tends to be highly compressible. Compression
tends to improve the scalability of flow collection systems, by
reducing the disk storage and I/O bandwidth requirement for a given
workload. The ideal flow storage format should support applications
which wish to leverage this fact by supporting compression of stored
data.</t>
<t>The IPFIX Message format has no support for data compression, as
the IPFIX protocol was designed for speed and simplicity of export. Of
course, any flat file is readily compressible using a wide variety of
external data compression tools, formats, and algorithms; therefore,
this requirement can be met externally.</t>
<t>However, a couple of simple optimizations can be made by File
Writers to increase the integrity and usability of compressed IPFIX
data; these are outlined in <xref target="compression-errors" />.</t>
</section>
<section title="Indexing and Searching">
<t>Binary, record stream oriented file formats natively support only
one form of searching, sequential scan in file order. By choosing the
order of records in a file carefully (e.g., by flow start or flow end
time), a file can be indexed by a single key.</t>
<t>Beyond this, properly addressing indexing is an
application-specific problem, as it inherently involves tradeoffs
between storage complexity and retrieval speed, and requirements vary
widely based on time scales and the types of queries used from site to
site. However, a generic standard flow storage format may provide
limited direct support for indexing and searching.</t>
<t>The ideal flow storage format will support a limited table of
contents facility noting that the records in a file contain data
relating only to certain keys or values of keys, in order to keep
multi-file search implementations from having to scan a file for data
it does not contain.</t>
<t>The IPFIX Message format has no direct support for indexing.
However, its template mechanism and the technique described in <xref
target="I-D.ietf-ipfix-reducing-redundancy">"Reducing Redundancy in
IPFIX and PSAMP Reports"</xref> can be used to describe the contents
of a file in a limited way. Additionally, as flow data is often sorted
and divided by time, the start and end time of the flows in a file may
be declared using the File Time Window Options Template defined in <xref
target="ot-time-window" />.</t>
</section>
<section title="Data Integrity">
<t>When storing flow data over long time scales, especially for
archival purposes, it is important to ensure that hardware or software
faults do not introduce errors into the data over time. The ideal flow
storage format will support the detection and correction of
encoding-level errors in the data.</t>
<t>Note that more advanced error correction is almost certainly best
handled at a layer below that addressed by this document. Error
correction is a topic well addressed by the storage industry in
general (e.g. by RAID and other technologies), and by specifying a
flow storage format based upon files, we can leverage these features
to meet this requirement.</t>
<t>However, the ideal flow storage format will be resilient against
errors, providing an internal facility for the detection of errors and
the ability to isolate errors to as few data records as possible.</t>
<t>Note that this requirement interacts with the choice of data
compression or encryption algorithm. The use of block compression
algorithms can serve to isolate errors to a single compression block,
unlike stream compressors, which may fail to resynchronize after a
single bit error, invalidating the entire message stream. Similarly,
the use of a stream cipher can serve to isolate errors in the
plaintext without amplifying them as, for example, a cipher in CBC
mode can. See the "Recommended Compression Error Resilience Strategy"
and "Recommended Encryption Error Resilience Strategy" sections below
for more on this interaction.</t>
<t>The IPFIX Message format does not support data integrity assurance.
It is assumed that advanced error correction will be provided
externally. For simple error detection support, checksums may be
attached to messages via IPFIX Options according to the Message
Checksum Options Template defined in <xref target="ot-checksum"
/>.</t>
</section>
<section title="Creator Authentication and Confidentiality">
<t>Storage of flow data across long time scales may also require
assurance that no unauthorized entity can read or modify the stored
data. Asymmetric-key cryptography can be applied to this problem, by
signing flow data with the private key of the creator, and encrypting
it with the public keys of those authorized to read it. The ideal flow
storage format will support the encryption and signing of flow
data.</t>
<t>As with error correction, this problem has been addressed well at a
layer below that addressed by this document. Instead of specifying a
particular choice of encryption technology, we can leverage the fact
that existing cryptographic technologies work quite well on data
stored in files to meet this requirement.</t>
<t>Beyond support for the use of TLS for transport over TCP or DTLS
for transport over SCTP or UDP, both of which provide transient
authentication and confidentiality, the IPFIX protocol does not
support this requirement directly. It is assumed that this requirement
will be met externally.</t>
</section>
<section title="Anonymization and Obfuscation">
<t>To ensure the privacy of individuals and organizations at the
endpoints of communications represented by flow records, it is often
necessary to obfuscate or anonymize stored and exported flow data. The
ideal flow storage format will provide for a notation that a given
information element on a given record type represents anonymized,
rather than real, data.</t>
<t>The IPFIX Message format presently has no support for anonymization
notation. It should be noted that anonymization is one of the
requirements given for IPFIX in <xref target="RFC3917"/>. The decision
to qualify this requirement with 'MAY' and not 'MUST' in the
requirements document, and its subsequent lack of specification in the
current version of the IPFIX protocol, is due to the fact that
anonymization algorithms are still an open area of research, and that
there currently exist no standardized methods for anonymization.</t>
<t>No support is presently defined in <xref target="RFC5101"/> or this
IPFIX-based File Format for anonymization, as anonymization notation
is an area of open work for the IPFIX working group.</t>
<!--
<t>Simple anonymization notation may be attached to templates via
IPFIX Options according to the Template Anonymization Options Template
defined in <xref target="ot-anon" />.</t>
-->
</section>
<section title="Session Auditability and Replayability">
<t>Certain use cases for archival flow storage require the storage of
collection infrastructure details alongside the data itself. These
details include information about how and when data was received, and
where it was received from, and are useful for auditing as well as for
the replaying received data for testing purposes.</t>
<t>The IPFIX Message format contains no direct support for
auditability and replayability, though the IPFIX Information Model
does define various Information Elements required to represent
collection infrastructure details. These details may be stored in
IPFIX Files using the Export Session Details Options Template defined
in <xref target="ot-session-detail" /> and the Message Details
Options Template defined in <xref target="ot-message-detail" />.</t>
</section>
<section title="Performance Characteristics">
<t>The ideal standard flow storage format will not have a significant
negative impact on the performance of the application generating or
processing flow data stored in the format. This is a non-functional
requirement, but it is important to note that a standard that implies
a significant performance penalty is unlikely to be widely implemented
and adopted.</t>
<t>A static analysis of the IPFIX Message format would seem to suggest
that implementations of it are not particularly prone to slowness;
indeed, a template-based data representation is more easily subject to
optimization for common cases than representations that embed
structural information directly in the data stream (e.g. XML).
However, a full analysis of the impact of using IPFIX Messages as a
basis for flow data storage on read/write performance will require
more implementation experience and performance measurement.</t>
</section>
</section>
<section title="Applicability" anchor="applicability-section">
<t>This section describes the specific applicability of IPFIX Files to
various use cases. IPFIX Files are particularly useful in a flow
collection and processing infrastructure using IPFIX for flow export. We
explore the applicability and provide guidelines for using IPFIX files
for the storage of flow data collected by IPFIX Collecting Processes and
NetFlow V9 collectors, the testing of IPFIX Collecting Processes, and
diagnostics of IPFIX Devices.</t>
<section title="Storage of IPFIX-collected Flow Data" anchor="applicability-collection">
<t>IPFIX Files can naturally be used to store flow data collected by
an IPFIX Collecting Process; indeed, this was one of the primary
initial motivations behind the file format described within this
document. Using IPFIX Files as such provides a single, standard,
well-understood encoding to be used for flow data on disk and on the
wire, and allows IPFIX implementations to leverage substantially the
same code for flow export and flow storage. In addition, the storage
of single Transport Sessions in IPFIX Files is particularly important
for network measurement research, allowing repeatability of
experiments by providing a format for the storage and exchange of
IPFIX flow trace data much as the libpcap format is used for
experiments on packet trace data.</t>
</section>
<section title="Storage of NetFlow V9-collected Flow Data">
<t>Although the IPFIX protocol is based on the Cisco Netflow Services,
Version 9 (NetFlow V9) <xref target="RFC3954">protocol</xref>, the two
have diverged since work began on IPFIX. However, since the NetFlow V9
information model is a compatible subset of the IPFIX information
model, it is possible to use IPFIX files to store collected NetFlow V9
flow data. This approach may be particularly useful in multi-vendor,
multi-protocol collection infrastructures using both NetFlow V9 and
IPFIX to export flow data.</t>
<t>The applicability of IPFIX Files to this use case is outlined in
<xref target="v9-applicability" />.</t>
</section>
<section title="Testing IPFIX Collecting Processes">
<t>IPFIX Files can be used to store IPFIX Messages for the testing of
IPFIX Collecting Processes. A variety of test cases may be stored in
IPFIX Files. First, IPFIX data sets collected in real network
environments and stored in an IPFIX File can be used as input to check
the behavior of new or extended implementations of IPFIX Collectors.
Furthermore, IPFIX Files can be used to validate the operation of a
given IPFIX Collecting Process in a new environment, i.e., to test
with recorded IPFIX data from the target network before installing the
Collecting Process in the network.</t>
<t>The IPFIX File format can also be used to store artificial,
non-compliant reference messages for specific Collecting Process test
cases. Examples for such test cases are sets of IPFIX records with
undefined Information Elements, Data Records described by missing
Templates, or incorrectly framed messages or data sets. Representative
error handling test cases are defined in <xref
target="I-D.ietf-ipfix-testing">"IPFIX Testing"</xref>.</t>
<t>Furthermore, fast replay of IPFIX records stored in a file can be
used for stress/load tests (e.g., high rate of incoming Data Records,
large Templates with high Information Element counts), as described in
<xref target="I-D.ietf-ipfix-testing">"IPFIX Testing"</xref>. The
provisioning and use of a set of reference files for testing
simplifies the performance of tests and increases the comparability of
test results.</t>
</section>
<section title="IPFIX Device Diagnostics">
<t>As an IPFIX File can be used store any collection of flows, the
format may also be used for dumping and storing various types of flow
data for IPFIX Device diagnostics (e.g., the open flow cache of a
Metering Process or the flow backlog of an Exporting or Collecting
Process at the time of a process reset or crash). File-based storage
is preferable to remote transmission in such error-recovery
situations.</t>
</section>
</section>
<section title="Detailed Description" anchor="detail-section">
<t>An IPFIX File, as introduced in <xref target="design-section"/> and
elaborated below, is at its core simply an IPFIX Message stream
serialized to some filesystem. Any valid serialized IPFIX Message stream
MUST be accepted by a File Reader as a valid IPFIX file. In this way,
the filesystem is simply treated as another IPFIX transport alongside
SCTP, TCP, and UDP. In contrast to normal IPFIX operation, the time
between a File Writer writing an IPFIX Message stream to a File and a
File Reader reading it can be extremely variable. In other words, this
notional file transport has unusually high latency, as the File Reader
and File Writer do not necessarily run at the same time.</t>
<t>This section specifies the detailed actions of File Readers and File
Writers in handling IPFIX Files, and further specifies actions of File
Writers in specific use cases. Unless otherwise specified herein, where
appropriate IPFIX File Writers MUST behave as IPFIX Exporting Processes,
and IPFIX File Readers MUST behave as IPFIX Collecting Processes.</t>
<section title="File Reader Specification">
<t>An IPFIX File Reader MUST accept as valid any serialized IPFIX
Message stream that would be considered valid by one or more of the
other defined IPFIX transport layers. Practically, this means that the
union of template management features supported by SCTP, TCP, and UDP
MUST be supported in IPFIX Files. The following requirements apply to
IPFIX File Readers:</t>
<list style="symbols">
<t>File Readers MUST accept IPFIX Messages containing Template Sets,
Options Template Sets, and Data Sets within the same message, as
with IPFIX over TCP or UDP.</t>
<t>File Readers MUST accept Template Sets that define templates
already defined within the file, as may occur with template
retransmission when using IPFIX over UDP as described in section
10.3.6 of <xref target="RFC5101"/>. In the
event of a conflict between a resent definition and a previous
definition, the File Reader MUST assume that the new template
replaces the old, as consistent with UDP template expiration and ID
reuse.</t>
<t>File Readers MUST accept Template Withdrawals as described in
section 8 of <xref target="RFC5101"/>,
provided that the Template to be withdrawn is defined, as is the
case with IPFIX over TCP and SCTP.</t>
</list>
<t>Note that some applications, particularly those storing large
collections of data over long periods of time, may benefit from the
ability to treat a collection of IPFIX Files as a single Transport
Session. A File Reader MAY be configurable to treat a collection of
Files (e.g., all the files in a directory) as a single Transport
Session. However, a File Reader MUST NOT treat a single IPFIX File as
containing multiple Transport Sessions.</t>
<t>If an IPFIX File uses the technique described in <xref
target="I-D.ietf-ipfix-reducing-redundancy">"Reducing Redundancy in
IPFIX and PSAMP Reports"</xref> AND all of the non-Options Templates
in the File contain the commonPropertiesId Information Element, a File
Reader MAY assume the set of commonPropertiesId definitions provides a
complete table of contents for the File for searching purposes.</t>
</section>
<section title="File Writer Specification" anchor="writer-spec-section">
<t>While any valid serialized IPFIX Message stream is a valid IPFIX
File, the following recommendations will improve representation
simplicity and read performance in the general case, where
possible.</t>
<t>File Writers SHOULD emit each Template Set or Options Template Set
to appear in the file before any Data Set described by the Templates
within that Set, to ensure the File Reader can decode every Data Set
without waiting to process subsequent Templates or Options
Templates.</t>
<t>File Writers SHOULD emit Data Records described by Options
Templates to appear in the file before any Data Records which depend
on the scopes defined by those options.</t>
<t>File Writers SHOULD use Template Withdrawals to withdraw Templates
if template IDs need to be reused. In this case, the new Templates
reusing those IDs SHOULD appear directly in the file after the
Template Withdrawals making the IDs available for reuse. Template
Withdrawals SHOULD NOT be used unless necessary to reuse template
IDs.</t>
<t>Each IPFIX File is generally synonymous with a single Transport
Session. File Writers SHOULD store the Templates and Options required
to decode the data within the File in the File itself, and File
Readers SHOULD NOT use Templates or Options defined in one file to
decode or interpret Data Sets in another.</t>
<t>File Writers SHOULD write IPFIX Messages within an IPFIX File in
ascending Export Time order.</t>
<t>File Writers MAY write Data Records to an IPFIX File in any order.
However, File Writers that write flow records to an IPFIX File in
flowStartTime or flowEndTime order SHOULD be consistent in this
ordering within each File.</t>
</section>
<section title="Specific File Writer Use Cases">
<t>The specifications in this section apply to specific situations.
Each section below extends or modifies the base File Writer
specification in <xref target="writer-spec-section"/>. Considerations
for collocation of a File Writer with IPFIX Collecting Processes and
Metering Processes are given, as are specific guidelines for using
IPFIX Files for archival storage, or as documents. Also covered are
the use of IPFIX Files in the testing and diagnostics of IPFIX
Devices.</t>
<section title="Collocating a File Writer with a Collecting Process">
<t>When collocating a File Writer with an IPFIX Collecting Process
for archival storage of collected data in IPFIX Files as described
in <xref target="applicability-collection"/>, the following
recommendations may improve the usefulness of the stored data.</t>
<t>The simplest way for a to store the data collected in a single
Transport Session is to simply write the incoming IPFIX Messages to
an IPFIX File as they are collected. However, the resulting files
will lack information about the IPFIX Transport Session used to
export them, such as the network addresses of the Exporting and
Collecting Processes and the protocols used to transport them. In
this case, if information about the Transport Session is required,
the File Writer SHOULD store a single IPFIX Transport Session in an
IPFIX File and SHOULD record information about the Transport Session
using the Export Session Details Options Template described
in <xref target="ot-session-detail"/>.</t>
<t>Additional per-Message information MAY be recorded by the File
Writer using the Message Details Options Template described in <xref
target="ot-message-detail"/>. Per-message information includes the
time at which each IPFIX Message was received at the Collecting
Process, and can be used to resend IPFIX Messages while keeping the
original measurement plane traffic profile.</t>
<t>When collocating a File Writer with a Collecting Process, the
Export Time of each Message SHOULD be the Export Time of the Message
received by the Collecting Process containing the first Data Record
in the Message. Note that File Writers storing IPFIX data collected
from an IPFIX Collecting Process using SCTP as the transport
protocol SHOULD interleave messages from multiple streams in order
to preserve Export Time order, and SHOULD reorder the written
messages as necessary to ensure that each Template Set or Options
Template Set appears in the file before any Data Set described by
the Templates within that Set.</t>
</section>
<section title="Collocating a File Writer with a Metering Process">
<t>Note that File Writers may also be collocated directly with IPFIX
Metering Processes, for writing measured information directly to
disk without intermediate IPFIX Exporting or Collecting Processes.
This arrangement may be particularly useful when providing data to
an analysis environment with an IPFIX File based workflow, or when
testing Metering Processes during development.</t>
<t>When collocating a File Writer with a Metering Process, note
that Information Elements associated with Exporting or Collecting
Processes are meaningless, and SHOULD NOT appear in the Export
Session Details Options Template described in <xref
target="ot-session-detail"/> or the Message Details Options Template
described in <xref target="ot-message-detail"/>.</t>
<t>When collocating a File Writer with an Exporting Process, the
Export Time of each Message SHOULD be the time at which the first
Data Record in the Message was received from the Exporting
Process.</t>
</section>
<section title="Using IPFIX Files for Archival Storage">
<t>While in the general case File Writers should store one Transport
Session per IPFIX File, some applications storing large collections
of data over long periods of time may benefit from the ability to
treat a collection of IPFIX Files as a single Transport Session. A
File Writer MAY be configurable to write data from a single
Transport Session into multiple IPFIX Files; however, File Writers
supporting such a configuration option MUST provide a configuration
option to support one-file-per-session behavior for interoperability
purposes.</t>
<t>File Writers compressing or encrypting archival data and File
Readers reading compressed or encrypted archival data SHOULD follow
the recommendations in <xref target="errors-section"/>.</t>
</section>
<section title="Using IPFIX Files as Documents">
<t>When IPFIX Files are used as documents, to store a set of flows
relevant to query, investigation, or other common context, or for
the publication of flow data sets relevant to network research, each
File MUST be readable as a single Transport Session, self-contained
and making no reference to metadata stored in separate Files, in
order to ensure interoperability.</t>
<t>When writing Files to be used as documents, File Writers may emit
the special Data Records described by Options Templates before any
other Data Records in the File, in the following order, to ease the
inspection and use of documents by File Readers:</t>
<list style="symbols">
<t>Time Window records described by the File Time Window Options
Template as defined in <xref target="ot-time-window" /> below;
followed by</t>
<t>commonPropertiesId definitions as described in <xref
target="I-D.ietf-ipfix-reducing-redundancy">"Reducing Redundancy
in IPFIX and PSAMP Reports"</xref>; followed by</t>
<t>Information Element Type Records as described in <xref
target="I-D.ietf-ipfix-exporting-type">"Exporting Type Information
for IPFIX Information Elements"</xref>; followed by</t>
<t>Export Session details records described by the Export Session
Details Options Template as defined in <xref
target="ot-session-detail" /> below.</t>
</list>
<t>The Export Time of each Message within a File used as a document
SHOULD be the time at which the Message was written by the File
Writer.</t>
<t>If an IPFIX File used as a document uses the technique described
in <xref target="I-D.ietf-ipfix-reducing-redundancy">"Reducing
Redundancy in IPFIX and PSAMP Reports"</xref> AND all of the
non-Options Templates in the File contain the commonPropertiesId
Information Element, a File Reader MAY assume the set of
commonPropertiesId definitions provides a complete table of contents
for the File for searching purposes.</t>
</section>
<section title="Using IPFIX Files for Testing">
<t>IPFIX Files can be used for testing IPFIX Collecting Processes in
two ways. First, IPFIX Files can be used to store specific flow data
for regression and stress testing of Collectors; there are no
special considerations for IPFIX Files used in this way.</t>
<t>Second, IPFIX Files are useful for storing reference messages
which do not comply to the IPFIX Protocol in order to test the error
handling and recovery behavior of Collectors. Of course, IPFIX Files
intended to be used in this application necessarily MAY violate any
of of the specifications in this document or in <xref
target="RFC5101"/>, and such files MUST NOT be transmitted to
Collecting Processes or given as input File Readers not under
test.</t>
<t>Note that an extremely simple IPFIX Exporting Process may be
crafted for testing purposes by simply reading an IPFIX File and
transmitting it directly to a Collecting Process. Similarly, an
extremely simple Collecting Process may be crafted for testing
purposes by simply accepting connections and/or IPFIX Messages from
Exporting Processes and writing the session's message stream to an
IPFIX File.</t>
</section>
<section title="Writing IPFIX Files for Device Diagnostics">
<t>IPFIX Files can be used in the debugging of devices which use
flow data as internal state, as a common format for the
representation of flow tables. In such situations, the opaueOctets
information element can be used to store additional non-IPFIX
encoded, non-flow information (e.g., stack backtraces, process
state, etc.) within the IPFIX File as in <xref
target="encapsulation-int" />; the IPFIX flow table information
could also be embedded in a larger proprietary diagnostic format
using delimiters as in <xref target="encapsulation-ext"/></t>
</section>
</section>
</section>
<section title="File Format Metadata Specification" anchor="templates-section">
<t>This section defines the Options Templates used for IPFIX File
metadata, and the Information Elements they require.</t>
<section title="Recommended Options Templates for IPFIX Files" anchor="ot-section">
<t>The following Options Templates allow IPFIX Message streams to meet
the requirements outlined above without extension to the message
format or protocol. They are defined in terms of existing Information
Elements defined in <xref target="RFC5102"/>, the Information Elements
defined in <xref target="I-D.ietf-ipfix-exporting-type">"Exporting
Type Information for IPFIX Information Elements"</xref>, as well as
Information Elements defined in <xref target="ie-section"/>. IPFIX
File Readers and Writers SHOULD support these options templates as
defined below.</t>
<t>In addition, IPFIX File Readers and Writers SHOULD support the
Options Templates defined in <xref
target="I-D.ietf-ipfix-exporting-type">"Exporting Type Information
for IPFIX Information Elements"</xref> in order to support
self-description of enterprise-specific Information Elements.</t>
<section anchor="ot-checksum"
title="Message Checksum Options Template">
<t>The Message Checksum Options Template specifies the structure of
a Data Record for attaching an MD5 message checksum to an IPFIX
Message. An MD5 message checksum as described MAY be used if
long-term data integrity is important to the application. The
described Data Record MUST appear only once per IPFIX Message, but
MAY appear anywhere within the Message.</t>
<t>The template SHOULD contain the following Information
Elements:</t>
<texttable>
<ttcol align="left">IE</ttcol>
<ttcol align="left">Description</ttcol>
<c>messageScope [scope]</c>
<c>A marker denoting this Option applies to the whole IPFIX
Message; content is ignored. This Information Element MUST be
defined as a Scope Field.</c>
<c>messageMD5Checksum</c>
<c>The MD5 checksum of the containing IPFIX Message.</c>
</texttable>
</section>
<section anchor="ot-time-window"
title="File Time Window Options Template">
<t>The File Time Window Options Template specifies the structure of
a Data Record for attaching a time window to an IPFIX File; this
Data Record is referred to as a time window record. A time window
record defines the earliest flow start time and the latest flow end
time of the flow records within a File. One and only one time window
record MAY appear within an IPFIX File if the time window
information is available; a File Writer MUST NOT write more than one
time window record to an IPFIX File. A File Writer that writes a
time window record to a File MUST NOT write any Flow with a start
time before the beginning of the window or an end time after the end
of the window to that File.</t>
<t>The template SHOULD contain the following Information
Elements:</t>
<texttable>
<ttcol align="left">IE</ttcol>
<ttcol align="left">Description</ttcol>
<c>sessionScope [scope]</c>
<c>A marker denoting this Option applies to the whole IPFIX
Transport Session (i.e., IPFIX File); content is ignored. This
Information Element MUST be defined as a Scope Field.</c>
<c>minFlowStartSeconds</c>
<c>The start time of the earliest flow in the Transport Session
(i.e., File) in epoch seconds.</c>
<c>maxFlowEndSeconds</c>
<c>The end time of the latest flow in the Transport Session
(i.e., File) in epoch seconds.</c>
</texttable>
</section>
<section anchor="ot-session-detail"
title="Export Session Details Options Template">
<t>The Export Session Details Options Template specifies the
structure of a Data Record for recording the details of an IPFIX
Transport Session in an IPFIX File. It is intended for use in
storing a single complete IPFIX Transport Session in a single IPFIX
File. The described Data Record SHOULD appear only once in a given
IPFIX File.</t>
<t>The template SHOULD contain the following Information Elements,
subject to applicability as noted on each Information Element:</t>
<texttable>
<ttcol align="left">IE</ttcol>
<ttcol align="left">Description</ttcol>
<c>sessionScope [scope]</c>
<c>A marker denoting this Option applies to the whole IPFIX
Transport Session (i.e., IPFIX File); content is ignored. This
Information Element MUST be defined as a Scope Field.</c>
<c>exporterIPv4Address</c>
<c>IPv4 address of the IPFIX Exporting Process from which the
Messages in this Transport Session were received. Present only for
Exporting Processes with an IPv4 interface. For multi-homed SCTP
associations, this SHOULD be the primary path endpoint address of
the Exporting Process.</c>
<c>exporterIPv6Address</c>
<c>IPv6 address of the IPFIX Exporting Process from which the
Messages in this Transport Session were received. Present only for
Exporting Processes with an IPv6 interface. For multi-homed SCTP
associations, this SHOULD be the primary path endpoint address of
the Exporting Process.</c>
<c>exporterTransportPort</c>
<c>The source port from which the Messages in this Transport
Session were received.</c>
<c>collectorIPv4Address</c>
<c>IPv4 address of the IPFIX Collecting Process which received the
Messages in this Transport Session. Present only for Collecting
Processes with an IPv4 interface. For multi-homed SCTP
associations, this SHOULD be the primary path endpoint address of
the Collecting Process.</c>
<c>collectorIPv6Address</c>
<c>IPv6 address of the IPFIX Collecting Process which received
the Messages in this Transport Session. Present only for
Collecting Processes with an IPv6 interface. For multi-homed SCTP
associations, this SHOULD be the primary path endpoint address of
the Collecting Process.</c>
<c>collectorTransportPort</c>
<c>The destination port on which the Messages in this Transport
Session were received.</c>
<c>collectorTransportProtocol</c>
<c>The IP Protocol Identifier of the transport protocol used to
transport Messages within this Transport Session.</c>
<c>collectorProtocolVersion</c>
<c>The version of the IPFIX Protocol used to transport Messages
within this Transport Session.</c>
<c>minExportSeconds</c>
<c>The Export Time of the first Message in the Transport
Session.</c>
<c>maxExportSeconds</c>
<c>The Export Time of the last Message in the Transport
Session.</c>
</texttable>
</section>
<section anchor="ot-message-detail"
title="Message Details Options Template">
<t>The Message Details Options Template specifies the structure of a
Data Record for attaching additional export details to an IPFIX
Message. These details include the time at which a message was
received and information about the export and collection
infrastructure used to transport the Message. This Options Template
also allows the storage of the export session metadata provided the
Export Session Details Options Template, for storing information
from multiple Transport Sessions in the same IPFIX File.</t>
<t>The template SHOULD contain the following Information Elements,
subject to applicability as noted for each Information Element. Note
that when used in conjunction with the Export Session Details
Options Template, when storing a single complete IPFIX Transport
Session in an IPFIX File, this template SHOULD contain only the
messageScope and collectionTimeMilliseconds Information Elements,
and the exportSctpStreamId Information Element for Messages
transported via SCTP.</t>
<texttable>
<ttcol align="left">IE</ttcol>
<ttcol align="left">Description</ttcol>
<c>messageScope [scope]</c>
<c>A marker denoting this Option applies to the whole IPFIX
message; content is ignored. This Information Element MUST be
defined as a Scope Field.</c>
<c>collectionTimeMilliseconds</c>
<c>The absolute time at which this Message was received by the
IPFIX Collecting Process.</c>
<c>exporterIPv4Address</c>
<c>IPv4 address of the IPFIX Exporting Process from which this
Message was received. Present only for Exporting Processes with an
IPv4 interface, and if this information is not available via the
Export Session Details Options Template. For multi-homed SCTP
associations, this SHOULD be the primary path endpoint address of
the Exporting Process.</c>
<c>exporterIPv6Address</c>
<c>IPv6 address of the IPFIX Exporting Process this Message was
received. Present only for Exporting Processes with an IPv6
interface, and if this information is not available via the Export
Session Details Options Template. For multi-homed SCTP
associations, this SHOULD be the primary path endpoint address of
the Exporting Process.</c>
<c>exporterTransportPort</c>
<c>The source port from which this Message received. Present only
if this information is not available via the Export Session
Details Options Template.</c>
<c>collectorIPv4Address</c>
<c>IPv4 address of the IPFIX Collecting Process which received
this Message. Present only for Collecting Processes with an IPv4
interface, and if this information is not available via the Export
Session Details Options Template. For multi-homed SCTP
associations, this SHOULD be the primary path endpoint address of
the Collecting Process.</c>
<c>collectorIPv6Address</c>
<c>IPv6 address of the IPFIX Collecting Process which received
this Message. Present only for Collecting Processes with an IPv6
interface, and if this information is not available via the Export
Session Details Options Template. For multi-homed SCTP
associations, this SHOULD be the primary path endpoint address of
the Collecting Process.</c>
<c>collectorTransportPort</c>
<c>The destination port on which this Message was received.
Present only if this information is not available via the Export
Session Details Options Template.</c>
<c>collectorTransportProtocol</c>
<c>The IP Protocol Identifier of the transport protocol used to
transport this Message. Present only if this information is not
available via the Export Session Details Options Template.</c>
<c>collectorProtocolVersion</c>
<c>The version of the IPFIX Protocol used to transport this
Message. Present only if this information is not available via the
Export Session Details Options Template.</c>
<c>exportSctpStreamId</c>
<c>The SCTP stream used to transport this Message. Present only if
the Message was transported via SCTP.</c>
</texttable>
</section>
</section>
<section title="Recommended Information Elements for IPFIX Files" anchor="ie-section">
<t>The following Information Elements are used by the options
templates in <xref target="ot-section"/> to allow IPFIX Message
streams to meet the requirements outlined above without extension of
the message format or protocol. IPFIX File Readers and Writers SHOULD
support these Information Elements as defined below.</t>
<t>In addition, IPFIX File Readers and Writers SHOULD support the
Information Elements defined in <xref
target="I-D.ietf-ipfix-exporting-type">"Exporting Type Information
for IPFIX Information Elements"</xref> in order to support
full self-description of Information Elements.</t>
<section anchor="ie-collectionTimeMilliseconds"
title="collectionTimeMilliseconds">
<list style="hanging">
<t hangText="Description: ">The absolute timestamp at which the
data within the scope containing this Information Element was
received by a Collecting Process. This Information Element SHOULD
be bound to its containing IPFIX Message via IPFIX Options and
the messageScope Information Element, as defined below.</t>
<t hangText="Abstract Data Type: ">dateTimeMilliseconds</t>
<t hangText="ElementId: ">TBD1</t>
<t hangText="Status: ">current</t>
</list>
</section>
<section anchor="ie-exportSctpStreamId" title="exportSctpStreamId">
<list style="hanging">
<t hangText="Description: ">The value of the SCTP Stream Identifier
used by the Exporting Process for exporting IPFIX Message data.
This is carried in the Stream Identifier field of the header of the
SCTP DATA chunk containing the IPFIX Message(s).</t>
<t hangText="Abstract Data Type: ">unsigned16</t>
<t hangText="Data Type Semantics: ">identifier</t>
<t hangText="ElementId: ">TBD2</t>
<t hangText="Status: ">current</t>
</list>
</section>
<section anchor="ie-maxExportSeconds" title="maxExportSeconds">
<list style="hanging">
<t hangText="Description: ">The absolute Export Time of the latest
IPFIX Message within the scope containing this Information
Element. This Information Element SHOULD be bound to its
containing IPFIX Transport Session (i.e., File) via an options
record and the sessionScope Information Element, as defined below,
and SHOULD appear only once in a given IPFIX File.</t>
<t hangText="Abstract Data Type: ">dateTimeSeconds</t>
<t hangText="ElementId: ">TBD3</t>
<t hangText="Status: ">current</t>
<t hangText="Units: ">seconds</t>
</list>
</section>
<section anchor="ie-maxFlowEndSeconds" title="maxFlowEndSeconds">
<list style="hanging">
<t hangText="Description: ">The latest absolute timestamp of the
last packet within any Flow within the scope containing this
Information Element, rounded up to the second. This Information
Element SHOULD be bound to its containing IPFIX Transport Session
(i.e., File) via an IPFIX Options and the sessionScope
Information Element, as defined below, and SHOULD appear only once
in a given IPFIX File.</t>
<t hangText="Abstract Data Type: ">dateTimeSeconds</t>
<t hangText="ElementId: ">TBD4</t>
<t hangText="Status: ">current</t>
<t hangText="Units: ">seconds</t>
</list>
</section>
<section anchor="ie-messageMD5Checksum" title="messageMD5Checksum">
<list style="hanging">
<t hangText="Description: ">The MD5 checksum of the IPFIX Message
containing this record. This Information Element SHOULD be bound
to its containing IPFIX Message via an options record and the
messageScope Information Element, as defined below, and SHOULD
appear only once in a given IPFIX Message. To calculate the value
of this Information Element, first buffer the containing IPFIX
Message, setting the value of this Information Element to all
zeroes. Then caluclate the MD5 checksum of the resulting buffer as
defined in <xref target="RFC1321"/>, place the
resulting value in this Information Element, and export the
buffered message.</t>
<t hangText="Abstract Data Type: ">octetArray (16 bytes)</t>
<t hangText="ElementId: ">TBD5</t>
<t hangText="Status: ">current</t>
<t hangText="Reference: ">
<xref target="RFC1321">
RFC 1321, The MD5 Message-Digest Algorithm
</xref>
</t>
</list>
</section>
<section anchor="ie-messageScope" title="messageScope">
<list style="hanging">
<t hangText="Description: ">The presence of this Information
Element as scope in an Options Template signifies that the options
described by the Template apply to the IPFIX Message that contains
them. It is defined for general purpose message scoping of
options, and proposed specifically to allow the attachment a
checksum to a message via IPFIX Options. The value of this
Information Element MUST be written as 0 by the File Writer or
Exporting Process. The value of this Information Element MUST be
ignored by the File Reader or the Collecting Process.</t>
<t hangText="Abstract Data Type: ">octet</t>
<t hangText="ElementId: ">TBD6</t>
<t hangText="Status: ">current</t>
</list>
</section>
<section anchor="ie-minExportSeconds" title="minExportSeconds">
<list style="hanging">
<t hangText="Description: ">The absolute Export Time of the
earliest IPFIX Message within the scope containing this
Information Element. This Information Element SHOULD be bound to
its containing IPFIX Transport Session (i.e., File) via an options
record and the sessionScope Information Element, as defined below,
and SHOULD appear only once in a given IPFIX File.</t>
<t hangText="Abstract Data Type: ">dateTimeSeconds</t>
<t hangText="ElementId: ">TBD7</t>
<t hangText="Status: ">current</t>
<t hangText="Units: ">seconds</t>
</list>
</section>
<section anchor="ie-minFlowStartSeconds" title="minFlowStartSeconds">
<list style="hanging">
<t hangText="Description: ">The earliest absolute timestamp of the
first packet within any Flow within the scope containing this
Information Element, rounded down to the second. This Information
Element SHOULD be bound to its containing IPFIX Transport Session
(i.e., File) via an options record and the sessionScope
Information Element, as defined below, and SHOULD appear only once
in a given IPFIX File.</t>
<t hangText="Abstract Data Type: ">dateTimeSeconds</t>
<t hangText="ElementId: ">TBD8</t>
<t hangText="Status: ">current</t>
<t hangText="Units: ">seconds</t>
</list>
</section>
<section anchor="ie-opaqueOctets" title="opaqueOctets">
<list style="hanging">
<t hangText="Description: ">This Information Element is used to
encapsulate non-IPFIX data into an IPFIX Message stream, for the
purpose of allowing a non-IPFIX data processor to store a data
stream inline within an IPFIX file. A Collecting Process or File
Writer MUST NOT try to interpret this binary data. This
Information Element differs from paddingOctets as its contents are
meaningful in some non-IPFIX context, while the contents of
paddingOctets MUST be 0x00 and are intended only for Information
Element alignment.</t>
<t hangText="Abstract Data Type: ">octet</t>
<t hangText="ElementId: ">TBD9</t>
<t hangText="Status: ">current</t>
</list>
</section>
<section anchor="ie-sessionScope" title="sessionScope">
<list style="hanging">
<t hangText="Description: ">The presence of this Information
Element as scope in an Options Template signifies that the options
described by the Template apply to the IPFIX Transport Session
that contains them. Note that as all options are implicitly scoped
to Transport Session and Observation Domain, this Information
Element is equivalent to a "null" scope. It is defined for general
purpose session scoping of options, and proposed specifically to
allow the attachment of time window to a file via IPFIX Options.
The value of this Information Element MUST be written as 0 by the
File Writer or Exporting Process. The value of this Information
Element MUST be ignored by the File Reader or the Collecting
Process.</t>
<t hangText="Abstract Data Type: ">octet</t>
<t hangText="ElementId: ">TBD10</t>
<t hangText="Status: ">current</t>
</list>
</section>
</section>
</section>
<section title="Recommended Error Resilience Strategies" anchor="errors-section">
<t>This section describes recommended methods for making IPFIX Files
resilient to errors during storage. It is intended primarily for
applications using IPFIX Files for long-term archival storage of flow
data.</t>
<section title="Compression Error Resilience"
anchor="compression-errors">
<t>Note that, since any file may be compressed and
decompressed with a variety of widely available tools
implementing a variety of compression standards (both
specified and de facto), compression of IPFIX File data can be
accomplished externally. However, compression at the file
level is not particularly resilient to errors; in the worst
case, a single bit error in a stream-compressed file may
result in the loss of the entire file.</t>
<t>To limit the impact of errors on the recoverability of
compressed data, we recommend the use of block compression
where possible. Ideally, the block compression algorithm
should support the identification and isolation of blocks
containing errors; bzip2 is an example of such a block
compressor.</t>
<t>Since the block boundary of a block-compressed IPFIX File
may fall in the middle of an IPFIX Message, resynchronization
of an IPFIX Message stream by a File Reader after a
compression error requires some care. The beginning of an
IPFIX Message may be identified by its header signature (the
Version field of the Message Header, 0x00 0x0A, followed by a
16-bit Message Length), but simply searching for the first
occurance of the Version field is insufficient, since these
two bytes may occur in valid IPFIX Template or Data Sets.</t>
<t>Therefore, we propose the following algorithm for File
Readers to resynchronize an IPFIX Message Stream after
skipping a compressed block containing errors:</t>
<list style="numbers">
<t>Search after the error for the first occurrence of the
octet string 0x00, 0x0A (the IPFIX Message Header Version
field.)</t>
<t>Treat this field as the beginning of a candidate IPFIX
Message. Read the two bytes following the Version field as a
Message Length, and seek to that offset from the beginning
of the candidate IPFIX Message.</t>
<t>If the first two octets after the candidate IPFIX Message
are 0x00, 0x0A (i.e., the IPFIX Message Header Version field
of the next message in the stream), or if the end of the
file is reached precisely at the end of the candidate IPFIX
Message, presume that the candidate IPFIX Message is valid,
and begin reading the IPFIX File from the start of the
candidate IPFIX Message.</t>
<t>If not, or if the seek reaches end-of-file or another
block containing errors before finding the end of the
candidate message, go back to step 1, starting the search
two bytes from the start of the candidate IPFIX Message.</t>
</list>
<t>The algorithm above will improperly identify a non-message
as a message approximately 1 in 2^32 times, assuming random
IPFIX data. It may be expanded to consider multiple candidate
IPFIX Messages in order to increase reliability.</t>
<t>In applications (e.g. archival storage) in which error
resilience is very important, File Writers SHOULD use block
compression algorithms, and MAY attempt to align IPFIX
Messages within compression blocks to ease resynchronization
after errors, if such is supported by the chosen block
compressor. File Readers SHOULD use the resynchronization
algorithm above to minimize data loss due to compression
errors.</t>
</section>
<section title="Encryption Error Resilience"
anchor= "encryption-errors">
<t>File-level encryption has error resilience issues similar to
file-level compression. Single bit errors in the encrypted data stream
can result in unreadability of the entire remaining file, dependent on
the encryption method used. The use of CBC (Cipher Block Chaining)
mode, which suffers from this low error resilience, is relatively
common.</t>
<t>In applications (e.g. archival storage) in which error resilience
is very important, File Writers SHOULD use a stream cipher, for
example a block cipher in OFB (Output Feedback) mode (often referred
to as stream mode) instead of modes like CBC when encrypting, since
errors are not amplified by stream ciphers: A single-bit error in the
ciphertext results in a single bit error in the plaintext.
Alternatively File Writers SHOULD use any other cipher which can
resynchronize after bit errors. An example is a block cipher in CBC
mode that is reinitialized after a specific amount of data has been
encrypted. The maximum data loss per bit-error is then up to the next
reinitialization point. In this case, File Writers SHOULD also use the
Message Checksum Options Template to attach a checksum to each IPFIX
Message in the IPFIX File, in order to support the recognition of
errors in the decrypted data.</t>
</section>
</section>
<section title="Recommended File Integration Strategies" anchor="integration-section">
<t>This section describes methods for integrating IPFIX File data with
other file formats.</t>
<section title="Encapsulation of Non-IPFIX Data in IPFIX Files"
anchor= "encapsulation-int">
<t>At times it may be useful to export or store non-IPFIX data inline
in an IPFIX File or Message stream. To do this cleanly, this data must
be encapsulated into IPFIX Messages so that an IPFIX File Reader or
Collecting Process can handle it without any need to interpret it. At
the same time, this data must not be changed during transmission or
storage. The opaqueOctets Information Element as defined in <xref
target="ie-opaqueOctets"/> is provided for this encapsulation.</t>
<t>Processing the encapsulated non-IPFIX data is left to a separate
processing mechanisms that can identify encapsulated non-IPFIX data in
an IPFIX message stream, but need not have any other IPFIX handling
capability, except the ability to skip over all IPFIX messages that do
not encapsulate non-IPFIX data.</t>
<t>The Message Checksum Options Template, described in <xref
target="ot-checksum"/> may be used as a uniform mechanism to identify
errors within encapsulated data.</t>
<t>Note that this mechanism can only encapsulate data objects up to
65,515 octets in length. If the space available in one IPFIX Message
is not enough for the amount of data to be encapsulated, then the data
must be broken into smaller segments that are encapsulated into
consecutive IPFIX Messages. Any additional structuring or semantics of
the raw data is outside the scope of IPFIX and must be implemented
within the encapsulated binary data itself. Furthermore, the raw
encapsulated data cannot be assumed by an IPFIX File Reader to have
any specific format.</t> </section>
<section title="Encapsulation of IPFIX Files within Other File Formats"
anchor="encapsulation-ext">
<t>Consequently, it may also be useful to reverse the encapsulation,
that is, to export or store IPFIX data inline within a non-IPFIX file
or data stream. This makes sense when the other file format is not
compatible with the encapsulation described above in <xref
target="encapsulation-int"/>. Generally speaking, the encapsulation
here will be specific to the format of the containing file. For
example, IPFIX files may be embedded in XML elements using hex or
Base64 encoding, or in raw binary files using start and end delimiters
or some form of run-length encoding. As there are as many potential
encapsulations here as there are potential file formats, the specifics
of each are out of scope for this specification.</t>
</section>
</section>
<section title="Security Considerations" anchor="security-section">
<t>The IPFIX-based file format itself does not directly introduce
security issues. Rather it is used to store information which may for
privacy or business reasons be considered sensitive. The file format
must therefore provide appropriate procedures to guarantee the integrity
and confidentiality of the stored information.</t>
<t>The underlying protocol used to exchange the information that will be
stored using the format proposed in this document must as well apply
appropriate procedures to guarantee the integrity and confidentiality of
the exported information. Such issues are addressed in <xref
target="RFC5101"/>.</t>
<t>Implementors of IPFIX File Writers which store data taken from an
IPFIX Collecting Process using TLS or DTLS for transport security should
note that IPFIX Files may present a potential breach of confidentiality
if IPFIX data collected using TLS or DTLS is stored in unencrypted
files, and should consider providing an external file encryption option
to mitigate this risk.</t>
</section>
<section title="IANA Considerations" anchor="iana-section">
<t>This document specifies the creation of several new IPFIX Information
Elements in the IPFIX Information Element registry located at
http://www.iana.org/assignments/ipfix, as defined in <xref
target="ie-section" /> above. IANA has assigned the following
Information Element numbers for their respective Information Elements as
specified below:</t>
<list style="symbols">
<t>Information Element number TBD1 for the
collectionTimeMilliseconds Information Element.</t>
<t>Information Element number TBD2 for the exportSctpStreamId
Information Element.</t>
<t>Information Element number TBD3 for the maxExportSeconds
Information Element.</t>
<t>Information Element number TBD4 for the maxFlowEndSeconds
Information Element.</t>
<t>Information Element number TBD5 for the messageMD5Checksum
Information Element.</t>
<t>Information Element number TBD6 for the messageScope
Information Element.</t>
<t>Information Element number TBD7 for the minExportSeconds
Information Element.</t>
<t>Information Element number TBD8 for the minFlowStartSeconds
Information Element.</t>
<t>Information Element number TBD9 for the opaqueOctets
Information Element.</t>
<t>Information Element number TBD10 for the sessionScope
Information Element.</t>
</list>
<t>[NOTE for IANA: The text TBDn should be replaced
with the respective assigned Information Element numbers where they
appear in this document.]</t>
</section>
<section title="Acknowledgements">
<t>Thanks to Maurizio Molina, Tom Kosnar, and Andreas Kind for technical
assistance with the requirements for a standard flow storage format.
Thanks to Benoit Claise, Paul Aitken, and Andrew Johnson for their
reviews and feedback.</t>
</section>
</middle>
<back>
<references title="Normative References">
&rfc5101;
&rfc5102;
&draftIpfixRR;
&draftIpfixET;
&rfc1321;
</references>
<references title="Informative References">
&draftIpfixArch;
&draftIpfixAS;
&rfc5103;
&draftIpfixTesting;
&rfc3954;
&rfc3917;
&rfc2119;
<reference anchor='SAINT2007'>
<front>
<title>Requirements for a standardized flow storage solution</title>
<author initials='B' surname='Trammell' fullname='Brian Trammell'>
<organization />
</author>
<author initials='E' surname='Boschi' fullname='Elisa Boschi'>
<organization />
</author>
<author initials='L' surname='Mark' fullname='Lutz Mark'>
<organization />
</author>
<author initials='T' surname='Zseby' fullname='Tanja Zseby'>
<organization />
</author>
<date month='January' day='15' year='2007' />
<abstract/>
</front>
<seriesInfo name='' value='in Proceedings of the SAINT 2007 workshop on Internet Measurement Technology, Hiroshima, Japan'/>
</reference>
</references>
<section title="Example IPFIX File" anchor="examples-section">
<t>In this section we will explore an example IPFIX File which
demonstrates the various features of the IPFIX File format. This file
contains flow records described by a single Template. This file also
contains a File Time Window record to note the start and end time of the
data, and an Export Session Details record to record collection
infrastructure information. Each Message within this File also contains
a Message Checksum record, as this file may be externally encrypted
and/or stored as an archive. The structure of this file is shown in
<xref target="ex-structure" />.</t>
<figure title="File Example Structure" anchor="ex-structure">
<artwork><![CDATA[
+=================================================+
| IPFIX Message seq. 0 |
| +---------------------------------------------+ |
| | Template Set (id 2) 1 rec | |
| | Data Tmpl. id 256 | |
| +---------------------------------------------+ |
| | Options Template Set (id 3) 3 recs | |
| | File Time Window Opt. Tmpl. id 257 | |
| | Message Checksum Opt. Tmpl. id 259 | |
| | Export Session Details Opt. Tmpl. id 258 | |
| +---------------------------------------------+ |
| | Data Set (id 259) [Message Checksum] 1 rec | |
| +---------------------------------------------+ |
+=================================================+
| IPFIX Message seq. 1 |
| +---------------------------------------------+ |
| | Data Set (id 257) [File Time Window] 1 rec | |
| +---------------------------------------------+ |
| | Data Set (id 258) [Export Session] 1 rec | |
| +---------------------------------------------+ |
| | Data Set (id 259) [Message Checksum] 1 rec | |
| +---------------------------------------------+ |
+=================================================+
| IPFIX Message seq. 4 |
| +---------------------------------------------+ |
| | Data Set (id 256) 50 recs | |
| | contains flow data | |
| +---------------------------------------------+ |
| | Data Set (id 259) [Message Checksum] 1 rec | |
| +---------------------------------------------+ |
+=================================================+
| IPFIX Message seq. 55 |
| . . . |
]]></artwork>
</figure>
<t>The template describing the data records contains a flow start
timestamp, an IPv4 5-tuple, and packet and octet total counts. The
Template Set defining this is as shown in <xref
target="ex-data-template" /> below:</t>
<figure title="File Example Data Template" anchor="ex-data-template">
<artwork><![CDATA[
1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 2 | Length = 40 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 256 | Field Count = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| flowStartSeconds = 150 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| sourceIPv4Address = 8 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| dest.IPv4Address = 12 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| sourceTransportPort = 7 | Field Length = 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| dest.TransportPort = 11 | Field Length = 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| protocolIdentifier = 4 | Field Length = 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| octetTotalCount = 85 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| packetTotalCount = 86 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
<section title="Example Options Templates">
<t>This is followed by an Options Template Set containing the options
templates required to read the File: the File Time Window Options
Template defined in <xref target="ot-time-window" /> above, the Export
Session Details Options Template defined in <xref
target="ot-session-detail" /> above, and the Message Checksum Options
Template defined in <xref target="ot-checksum" /> above. This Options
Template Set is shown in <xref target="ex-opt-template-1" /> and <xref
target="ex-opt-template-2" /> below:</t>
<figure title="File Example Options Templates (Time Window and Checksum)" anchor="ex-opt-template-1">
<artwork><![CDATA[
1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length = 80 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 257 | Field Count = 3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count = 1 |0| sessionScope = TBD10 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 1 |0| minFlowStartSeconds = TBD8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 |0| maxFlowEndSeconds = TBD4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 | Template ID = 259 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Count = 2 | Scope Field Count = 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| messageScope = TBD6 | Field Length = 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| messageMD5Checksum = TBD5 | Field Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
<figure title="File Example Options Templates, Continued (Session Details)" anchor="ex-opt-template-2">
<artwork><![CDATA[
1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 258 | Field Count = 9 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count = 1 |0| sessionScope = TBD10 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 1 |0| exporterIPv4Address = 130 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 |0| collectorIPv4Address = 211 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 |0| exporterTransportPort = 217 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 2 |0| col.TransportPort = 216 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 2 |0| col.TransportProtocol = 215 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 1 |0| col.ProtocolVersion = 214 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 1 |0| minExportSeconds = TBD7 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 |0| maxExportSeconds = TBD3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 | set padding (2 octets) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
</section>
<section title="Example Supplemental Options Data">
<t>Following the templates required to decode the file is the
supplemental options information used to describe the file's contents
and type information. First comes the File Time Window record; it
notes that the file contains data from 9 October 2007 between 00:01:13
and 23:56:27 UTC, and appears as in <xref target="ex-timewindow"
/>:</t>
<figure title="File Example Time Window" anchor="ex-timewindow">
<artwork><![CDATA[
1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 257 | Length = 13 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sessionScope | minFlowStartSeconds
| 0 | 2007-10-09 00:01:13 UTC . . .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| maxFlowEndSeconds
. . . | 2007-10-09 23:56:27 UTC . . .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
. . . |
+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
<t>This is followed by information about how the data in the file was
collected, in the Export Session Details record. This record notes
that the session stored in this file was sent via SCTP from an
exporter at 192.0.2.30 port 32769 to an collector at 192.0.2.40 port
4739, and contains messages exported between 00:01:57 and 23:57:12 UTC
on 9 October 2007; it is represented in its Data Set as in <xref
target="ex-session-detail" />:</t>
<figure title="File Example Export Session Details"
anchor="ex-session-detail">
<artwork><![CDATA[
1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 258 | Length = 27 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sessionScope | exporterIPv4Address
| 0 | 192.0.2.30 . . .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| collectorIPv4Address
. . . | 192.0.2.31 . . .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| exporterTransportPort | cTPort
. . . | 32769 | 4739 . . .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| cTProtocol | cPVersion |
. . . | 132 | 10 | . . .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
minExportSeconds |
. . . 2007-10-09 00:01:57 UTC | . . .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
maxExportSeconds |
. . . 2007-10-09 23:57:12 UTC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
</section>
<section title="Example Message Checksum">
<t>Each IPFIX Message within the file is completed with a Message
Checksum record; the structure of this record within its Data Set is
as in <xref target="ex-checksum" />:</t>
<figure title="File Example Message Checksum" anchor="ex-checksum">
<artwork><![CDATA[
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 259 | Length = 24 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| messageScope | |
| 0 | |
+-+-+-+-+-+-+-+-+ |
| messageMD5Checksum |
| (16 byte MD5 checksum of options message) |
| |
| |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | set padding (3 octets) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
</section>
<section title="File Example Data Set">
<t>After the templates and supplemental options information comes the
data itself. The first record of an example Data Set is shown with its message and set headers in <xref target="ex-data" />:</t>
<figure title="File Example Data Set" anchor="ex-data">
<artwork><![CDATA[
1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version = 10 | Length = 1296 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Export Time = 2007-10-09 00:01:57 UTC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Observation Domain ID = 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 256 | Length = 1254 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| flowStartSeconds |
| 2007-10-09 00:01:13 UTC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sourceIPv4Address |
| 192.0.2.2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| destinationIPv4Address |
| 192.0.2.3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sourceTransportPort | destinationTransportPort |
| 32770 | 80 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| protocolId | totalOctetCount
| 6 | 18000 . . .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| totalPacketCount
. . . | 65 . . .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| (49 more records)
. . . |
+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
</section>
<section title="Complete File Example">
<t>Bringing together the examples above and adding message headers as
appropriate, a hex dump of the first 317 bytes of the example file
constructed above would appear as in the annotated <xref
target="ex-hex" /> below. [EDITOR'S NOTE: In this figure, xx refers to
unassigned IANA IE numbers as in the IANA Considerations section above;
cs refers to message checksum bytes that depend on
the rest of the message contents. These will have to be replaced if we
keep this example once the IE numbers are assigned.] </t>
<figure title="File Example Hex Dump" anchor="ex-hex">
<artwork><![CDATA[
0:|00 0A 00 A0 47 0A B6 E5 00 00 00 00 00 00 00 01
[^ first message header (length 160 bytes) -->
16:|00 02 00 28 01 00 00 08 00 96 00 04 00 08 00 04
[^ data template set -->
32: 00 0C 00 04 00 07 00 02 00 0B 00 02 00 04 00 01
48: 00 55 00 04 00 56 00 04|00 03 00 50 01 01 00 03
[^ opt template set -->
64: 00 01 xx xx 00 01 xx xx 00 04 xx xx 00 04 01 03
80: 00 02 00 01 xx xx 00 01 xx xx 00 10 01 02 00 09
96: 00 01 xx xx 00 01 00 82 00 04 00 D3 00 04 00 D9
112: 00 02 00 D8 00 02 00 D7 00 01 00 D0 00 01 xx xx
128: 00 04 xx xx 00 04 00 00|01 03 00 18 00 cs cs cs
[^ checksum record -->
144: cs cs cs cs cs cs cs cs cs cs cs cs cs 00 00 00
176:|00 0A 00 50 47 0A B6 E5 00 00 00 01 00 00 00 01
[^ second message header (length 80 bytes) -->
192:|01 01 00 0E 00 47 0A B6 B9 47 0C 07 1B 00|01 02
[^ time window rec -> [ session detail rec ^ -->
208: 00 1C 00 C0 00 02 1E 0C 00 02 1F 80 01 12 83 84
224: 0A 47 0A B6 E5 47 0C 07 48 00|01 03 00 18 00 cs
[ message checksum rec ^ -->
240: cs cs cs cs cs cs cs cs cs cs cs cs cs cs cs 00
256:|00 0A 05 10 47 0A B6 E5 00 00 00 06 00 00 00 01
[^ third message header (length 1296 bytes) -->
272:|01 00 04 E6|47 0A B6 B9 C0 00 02 02 C0 00 02 03
[^ set hdr ][^ first data rec -->
288: 80 02 00 50 06 00 00 46 50 00 00 00 41
]]></artwork>
</figure>
</section>
</section>
<section title="Applicability of IPFIX Files to NetFlow V9 flow storage"
anchor="v9-applicability">
<t>As the IPFIX Message format is nearly a superset of the NetFlow V9
packet format, IPFIX Files can be used for store NetFlow V9 data
relatively easily. This section describes a method for doing so. The
differences between the two protocols are outlined in <xref
target="v9-comparison" /> below. A simple, lightweight,
message-for-message translation method for transforming V9 Packets into
IPFIX Messages for storage within IPFIX Files is described in <xref
target="v9-transform" />. An example of this translation method is given
in <xref target="v9-example"/>.</t>
<section title="Comparing NetFlow V9 to IPFIX" anchor="v9-comparison">
<t>With a few caveats, the IPFIX Protocol is a superset of the NetFlow
V9 protocol, having evolved from it largely through a process of
feature addition to bring it into compliance with the IPFIX
Requirements and the needs of stakeholders within the IPFIX Working
Group. This appendix outlines the differences between the two
protocols. It is informative only, and presented as an exploration of
the two protocols to motivate the usage of IPFIX Files to store
V9-collected flow data.</t>
<section title="Message Header Format">
<t>Both NetFlow V9 and IPFIX use streams of messages prefixed by
a message header, though the message header differs
significantly between the two. Note that in NetFlow V9
terminology, these messages are called packets, and messages must
be delimited by datagram boundaries. IPFIX does not have this
constraint. The header formats are detailed below:</t>
<figure title="NetFlow V9 Packet Header Format" anchor="v9-packhdr">
<artwork><![CDATA[
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version Number | Count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sysUpTime |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| UNIX Secs |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
<figure title="IPFIX Message Header Format" anchor="v9-msghdr">
<artwork><![CDATA[
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version Number | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Export Time |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Observation Domain ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
<list style="hanging">
<t hangText="Version Number: ">The IPFIX Version Number MUST be
10, while the NetFlow V9 Version Number MUST be 9.</t>
<t hangText="Length vs. Count: ">The Count field in the NetFlow V9
packet header counts records in the message (including data and
template records), while the Length field in the IPFIX Message
Header counts octets in the message. Note that this implies that
NetFlow V9 collectors must rely on datagram boundaries or some
other external delimeter; or otherwise must completely consume a
message before finding its end.</t>
<t hangText="System Uptime: ">System uptime in milliseconds is
exported in the NetFlow V9 packet header. This field is not
present in the IPFIX Message Header, and must be exported using an
IPFIX Option if required.</t>
<t hangText="Export Time: ">Aside from being called UNIX Secs in
the NetFlow V9 packet header specification, the export time in
seconds since 1 January 1970 at 0000 UTC appears in both NetFlow
V9 and IPFIX message headers.</t>
<t hangText="Sequence Number: ">The NetFlow V9 Sequence Number
counts packets, while the IPFIX Sequence Number counts records in
Data Sets. Both are scoped to Observation Domain.</t>
<t hangText="Observation Domain ID: ">Similarly, the NetFlow V9
sourceID has become the IPFIX Observation Domain ID.</t>
</list>
</section>
<section title="Set Header Format">
<t>Set headers are identical between NetFlow V9 and IPFIX; that is,
each Set (FlowSet in NetFlow V9 terminology) is prefixed by a 4-byte
set header containing the Set ID and the length of the set in
octets.</t>
<t>Note that the special Set IDs are different between IPFIX and
NetFlow V9. IPFIX Template Sets are identified by Set ID 2, while
NetFlow V9 Template FlowSets are identified by Set ID 0. Similarly,
IPFIX Options Template Sets are identified by Set ID 3, while
NetFlow V9 Options Template FlowSets are identified by Set ID 1.</t>
<t>Both protocols reserve Set IDs 0-255, and use Set IDs 256-65535
for Data Sets (or FlowSets, in NetFlow V9 terminology).</t>
</section>
<section title="Template Format">
<t>Template FlowSets in NetFlow V9 support a subset of functionality
of those in IPFIX. Specifically, NetFlow V9 does not have any
support for vendor-specific Information Elements as IPFIX does, so
there is no enterprise bit or facility for associating a private
enterprise number with an information element.</t>
<t>Options Template FlowSets in NetFlow V9 are similar to Options
Template Sets in IPFIX in the same way.</t>
</section>
<section title="Information Model">
<t>The NetFlow V9 field type definitions are a compatible subset of,
and have evolved in concert with, the IPFIX Information Model. IPFIX
Information Element numbers in the range 1-127 are defined by <xref
target="RFC5102">the IPFIX Information Model</xref> to be
compatible with the corresponding NetFlow V9 field types.</t>
</section>
<section title="Template Management">
<t>NetFlow V9 has no concept of a Transport Session as in IPFIX, as
NetFlow V9 was designed with a connectionless transport in mind.
Template IDs are therefore scoped to an Exporting Process lifetime
(i.e., an Exporting Process instance between restarts). There is no
facility in NetFlow V9 as in IPFIX for Template withdrawal or
Template ID reuse. Template retransmission at the Exporter works as
in UDP-based IPFIX Exporting Processes.</t>
</section>
<section title="Transport">
<t>In practice, though NetFlow V9 is designed to be
transport-independent, it is transported only over UDP. There is no
facility as in IPFIX for full connection-oriented transport without
datagram boundaries, due to the use of a record count field as
opposed to a message length field in the packet header. There is no
support in NetFlow V9 for transport layer security via TLS or
DTLS.</t>
</section>
</section>
<section title="A Method for Transforming NetFlow V9 messages to IPFIX"
anchor="v9-transform">
<t>This appendix describes a method for transforming NetFlow V9
Packets into IPFIX Messages, which can be used to store NetFlow V9
data in IPFIX Files. A process transforming NetFlow V9 Packets into
IPFIX Messages must handle the fact that NetFlow V9 Packets and IPFIX
Messages are framed differently, that sequence numbering works
differently, and that the NetFlow V9 field type definitions are only
compatible with the IPFIX Information Model field and/or information
element numbers below Information Element number 128.</t>
<t>For each incoming NetFlow V9 packet, the transformation process
must:</t>
<list style="numbers">
<t>Verify that the Version field of the packet header is 9.</t>
<t>Verify that the Sequence Number field of the packet header is
valid.</t>
<t>Scan the packet to:</t>
<list style="numbers">
<t>verify that it contains no Templates with field numbers outside
the range 1-127;</t>
<t>verify that it contains no FlowSets with Set IDs between 2 and
255 inclusive;</t>
<t>verify that it contains the number of records in FlowSets,
Template FlowSets, and Options Template FlowSets declared in the
Count field of the packet header; and</t>
<t>count the number of records in FlowSets for calculating the
IPFIX Sequence number.</t>
</list>
<t>Calculate a Sequence Number for each IPFIX Observation Domain by
storing the last Sequence Number sent for each Observation Domain
plus the count of records in FlowSets in the previous step to be
sent as the Sequence Number for the IPFIX Message within that
Observation Domain following this one.</t>
<t>Generate a new IPFIX Message Header with:</t>
<list style="numbers">
<t>a Version field of 10; </t>
<t>a Length field with the number of octets in the IPFIX Message,
generally available by subtracting 4 from the length of the
NetFlow V9 packet as returned from the transport layer (accounting
for the difference in message header lengths);</t>
<t>the Sequence Number calculated for this message by the Sequence
Number calculation step; and</t>
<t>Export Time and Observation Domain ID taken from the UNIX secs
and Source ID fields of the NetFlow V9 packet header,
respectively.</t>
</list>
<t>Copy each FlowSet from the Netflow V9 packet to the IPFIX Message
after the header. Replace Set ID 0 with Set ID 2 for Template Sets,
and Set ID 1 with Set ID 3 for Options Template Sets.</t>
</list>
<t>Note that this process loses system uptime information; if such
information is required, the transformation process will have to
export that information using IPFIX Options. This may require a more
sophisticated transformation process structure.</t> </section>
<section title="NetFlow V9 Transformation Example" anchor="v9-example">
<t>The following two figures show a single NetFlow V9 packet with
templates and the corresponding IPFIX Message, exporting a single
flow record representing 60,303 octets sent from 192.0.2.2 to
192.0.2.3. This would be the 3rd packet exported in Observation
Domain 33 from the NetFlow V9 exporter, containing records starting
with the 12th record (packet and record sequence numbers count from
0).</t>
<t>The ** symbol in the IPFIX example shows those fields that required
modification from the NetFlow V9 packet by the transformation
process.</t>
<figure title="Example NetFlow V9 Packet" anchor="v9-expacket">
<artwork><![CDATA[
1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version = 9 | Count = 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Uptime = 3750405 ms (1:02:30.405) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Export Time = 1171557627 epoch sec (2007-02-15 16:40:27) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number = 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Observation Domain ID = 33 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 0 | Set Length = 20 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 256 | Field Count = 3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPV4_SRC_ADDR = 8 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPV4_DST_ADDR = 12 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IN_BYTES = 1 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 256 | Set Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPV4_SRC_ADDR |
| 192.0.2.2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPV4_DST_ADDR |
| 192.0.2.3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IN_BYTES |
| 60303 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
<figure title="Corresponding Example IPFIX Message"
anchor="v9-exmessage">
<artwork><![CDATA[
1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ** Version = 10 | ** Length = 52 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Export Time = 1171557627 epoch sec (2007-02-15 16:40:27) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ** Sequence Number = 11 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Observation Domain ID = 33 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ** Set ID = 2 | Set Length = 20 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 256 | Field Count = 3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| sourceIPv4Address = 8 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| destinationIPv4Address = 12 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| octetDeltaCount = 1 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 256 | Set Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sourceIPv4Address |
| 192.0.2.2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| destinationIPv4Address |
| 192.0.2.3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| octetDeltaCount |
| 60303 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
</section>
</section>
</back>
</rfc>
| PAFTECH AB 2003-2026 | 2026-04-23 14:29:23 |