One document matched: draft-ietf-ipfix-flow-selection-tech-04.txt
Differences from draft-ietf-ipfix-flow-selection-tech-03.txt
Internet Engineering Task Force L. Peluso
Internet-Draft University of Napoli
Intended status: Standards Track T. Zseby
Expires: July 11, 2011 Fraunhofer Institute FOKUS
S. D'Antonio
CINI Consortium/University of
Napoli "Parthenope"
C. Henke
Fraunhofer Institute FOKUS
January 07, 2011
Flow Selection Techniques
draft-ietf-ipfix-flow-selection-tech-04.txt
Abstract
Flow selection is the process of selecting a subset of flows from all
flows observed at an observation point. The objective of flow
selection is to reduce the effort of post-processing flow data and
transferring flow records. The flow selection process can be enabled
at different stages of the measurement process. It can be applied
either directly after classification or at recording/exporting time
by limiting the number of flows to be stored and/or exported to the
collecting process. This document describes motivations for flow
selection and presents flow selection techniques. It furthermore
provides an information model for configuring flow selection
techniques and discusses what information about a flow selection
process is worth exporting through a suitable information model.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
Peluso, et al. Expires July 11, 2011 [Page 1]
Internet-Draft Flow Selection Techniques January 2011
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on July 11, 2011.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Peluso, et al. Expires July 11, 2011 [Page 2]
Internet-Draft Flow Selection Techniques January 2011
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5
2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6
4. Flow selection as a function of the IPFIX Exporter . . . . . . 7
4.1. Flow selection in the metering process . . . . . . . . . . 9
4.2. Flow selection in the flow recording process . . . . . . . 9
4.3. Flow selection during the exporting process . . . . . . . 11
5. Flow selection as a function of the IPFIX Mediator . . . . . . 12
6. Flow selection techniques . . . . . . . . . . . . . . . . . . 14
6.1. Flow selection based on flow record content . . . . . . . 14
6.2. Flow selection based on flow record arrival time or
sequence . . . . . . . . . . . . . . . . . . . . . . . . . 14
6.3. Flow selection on external events . . . . . . . . . . . . 14
7. Information model for flow selection information exporting . . 14
7.1. Meter process related (TBD1-TBD3) . . . . . . . . . . . . 16
7.1.1. fsMeterUnmeasPacketCountTsFirst . . . . . . . . . . . 17
7.1.2. fsMeterUnmeasPacketCountTsLast . . . . . . . . . . . . 17
7.1.3. fsMeterUnmeasBytesCount . . . . . . . . . . . . . . . 18
7.2. Flow recording process related (TBD4-TBD11) . . . . . . . 18
7.2.1. fsFrecPacketInDroppedRecsCountTsFirst . . . . . . . . 18
7.2.2. fsFrecPacketInDroppedRecsCountTsLast . . . . . . . . . 19
7.2.3. fsFrecByteInDroppedRecsCount . . . . . . . . . . . . . 19
7.2.4. fsFrecFrecDroppedCountTsFirst . . . . . . . . . . . . 20
7.2.5. fsFrecFrecDroppedCountTsLast . . . . . . . . . . . . . 20
7.2.6. fsFrecUnexportedFrecCount . . . . . . . . . . . . . . 20
7.2.7. fsFrecUnexportedPacketInFrecCount . . . . . . . . . . 21
7.2.8. fsFrecUnexportedBytesInFrecCount . . . . . . . . . . . 21
7.3. Flow exporting process related (TBD12-TBD19) . . . . . . . 21
7.3.1. fsExpPacketInDroppedRecsCountTsFirst . . . . . . . . . 22
7.3.2. fsExpPacketInDroppedRecsCountTsLast . . . . . . . . . 22
7.3.3. fsExpByteInDroppedRecsCount . . . . . . . . . . . . . 23
7.3.4. fsExpFrecDroppedCountTsFirst . . . . . . . . . . . . . 23
7.3.5. fsExpFrecDroppedCountTsLast . . . . . . . . . . . . . 24
7.3.6. fsExpUnexportedCount . . . . . . . . . . . . . . . . . 24
7.3.7. fsExpUnexportedPacketCount . . . . . . . . . . . . . . 24
7.3.8. fsExpUnexportedByteInExpCount . . . . . . . . . . . . 25
8. Implementation requirements . . . . . . . . . . . . . . . . . 25
9. Information Model for Configuration of Flow Selection
Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . 25
9.1. selectorMethod . . . . . . . . . . . . . . . . . . . . . . 26
9.2. flowMaxAdmitFlowRecords . . . . . . . . . . . . . . . . . 27
9.3. flowRecordBytesSize . . . . . . . . . . . . . . . . . . . 27
9.4. flowRecordPacketsSize . . . . . . . . . . . . . . . . . . 28
9.5. flowInactivityTime . . . . . . . . . . . . . . . . . . . . 28
9.6. flowIPAddressVersion . . . . . . . . . . . . . . . . . . . 29
9.7. flowIPv4SourceAddress . . . . . . . . . . . . . . . . . . 29
Peluso, et al. Expires July 11, 2011 [Page 3]
Internet-Draft Flow Selection Techniques January 2011
9.8. flowIPv4DestinationAddress . . . . . . . . . . . . . . . . 30
9.9. flowIPv6SourceAddress . . . . . . . . . . . . . . . . . . 30
9.10. flowIPv6DestinationAddress . . . . . . . . . . . . . . . . 30
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 31
11. Security Considerations . . . . . . . . . . . . . . . . . . . 31
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 31
12.1. Normative References . . . . . . . . . . . . . . . . . . . 31
12.2. Informative References . . . . . . . . . . . . . . . . . . 32
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 33
Peluso, et al. Expires July 11, 2011 [Page 4]
Internet-Draft Flow Selection Techniques January 2011
1. Introduction
This document describes flow selection techniques for traffic
measurements. As stated in [RFC5475], packet selection is the
process of selecting a subset of packets among those collected at an
observation point. The element on which this selection mechanism is
performed is a single packet and the selection decision is random,
deterministic or based on packet properties. In contrast to this,
flow selection techniques consider flows as the basic elements on
which a selection process is performed. A flow is defined as a set
of packets with common properties [IPFIX-REQ]. Flow selection
reduces the resource demands for capturing, storing, exporting and
post-processing flow-based measurement results. Maintaining and
exporting all flow records to the collecting process can exhaust
available resources with the result that measurement data is
discarded which reduces the utility of the measurement data and makes
accurate estimations impossible. Flow Selection can be applied to
only select flows that are of interest for a certain application or
to select a representative subset of the flows. Selecting
representative subsets of flows allows for flow characteristic
estimation while reducing the measurement data. Example applications
in which flow selections can be applied are accounting, attack and
intrusion detection, traffic engineering, traffic classification,
etc.. In many networks few large flows contribute to the majority of
the overall traffic volume [DuLT01a], [DuLT01b], which is also
referred to as "Quasi-Zipf-Law" [KuXW04]. This "elephant and mice"
phenomenon plays an important role in flow based measurements as it
poses challenges on the flow selection process depending on the
application and the available resources. For example in accounting
purposes it is useful to concentrate on the so-called "heavy hitter"
flows to cope with a limited flow cache size or limited transmission
capacity in times when resources are scarce.
2. Scope
This document describes flow selection techniques and their
parameters. It addresses the configuration of flow selection
techniques and defines which information should be reported by
devices that perform flow selection. It only describes processes
directly acting on traffic flows during the metering phase and/or the
exporting phase. Therefore it is assumed that flow selection is
performed after packets are classified into flows. This document
does not address the flow selection effects that might arise from
sampling or filtering packets in the metering process before the
classification process is performed. Such packet selection
techniques are described in [RFC5475] and, therefore, are out of the
scope of this document.
Peluso, et al. Expires July 11, 2011 [Page 5]
Internet-Draft Flow Selection Techniques January 2011
3. Terminology
This document is consistent with the terminology introduced in
[RFC5470], [RFC5475] and [IPFIX-REQ]. Further some additional terms
are presented which extend the terminology.
* Flow
A Flow is defined as a set of packets with common properties
passing an Observation Point in the network during a certain time
interval.
* Flow Record
A Flow Record contains information about a specific flow that was
metered at an observation point. A Flow Record contains measured
properties of the flow (e.g., the total number of bytes of all
packets of the flow) and usually characteristic properties of the
flow (e.g., source IP address).
* Classification
Is a filtering process in which packets are mapped to specific
flow records based on the packet properties. These properties
make up the flow key (e.g. header information, packet content, AS
number). In case a flow record for this specific flow key already
exist the flow record is updated, otherwise a new flow record is
created.
* Flow Selection Process
A Flow Selection Process takes a set of Flow Records as its input
and selects a subset of that set as its output.
* Flow Selection State
A Flow Selection Process may maintain state information for use by
the Flow Selection Process. At a given time, the Flow Selection
State may depend on flows observed at and before that time, as
well as other variables. Examples include:
(i) number of accounted flow records;
(ii) memory space available for flow recording;
Peluso, et al. Expires July 11, 2011 [Page 6]
Internet-Draft Flow Selection Techniques January 2011
(iii) state of the pseudorandom number generators;
(iv) hash values calculated during the selection.
(v) timestamps of flow observation (e.g. first or last packet
of flow) and flow duration
* Flow Selector
A Flow Selector defines the action of a Flow Selection Process on
a single flow of its input. The Flow Selector can make use of the
following information in order to establish whether or not a flow
has to be selected or not:
(i) the content of the flow record;
(ii) any information state related to the flow recording
process;
(iii) any flow selection state that may be maintained by the
Flow Selection Process.
* IPFIX Exporter
The IPFIX exporter is a software that receives measurement data
from e.g. the Metering Process and exports this data to a
collector.
* IPFIX Mediator
The IPFIX Mediator receives flow records from IPFIX Exporters and,
after some post-processing (e.g. more fine-grained selection),
sends them to one or multiple Collectors. This selection can be
driven based on the information requested by the individual
Collector.
4. Flow selection as a function of the IPFIX Exporter
Figure 1 shows the IPFIX reference model as defined in [RFC5470], and
extends it by introducing the functional components where flow
selection can take place.
Peluso, et al. Expires July 11, 2011 [Page 7]
Internet-Draft Flow Selection Techniques January 2011
Packet(s) coming in to Observation Point(s)
| |
v v
+----------------+---------------------------+ +-----+-------+
| Metering Process on an | | |
| Observation Point | | |
| | | |
| packet header capturing | | |
| | |...| Metering |
| timestamping | | Process N |
| | | | |
| packet selection | | |
| | | | |
| classification | | |
| | | | |
| flow state dependent packet sampling (*) | | |
| | | | |
| aggregation | | |
| | | | |
| flow recording (*) | | |
| | | | |
| | Timing out Flows | | |
| | Handle resource overloads | | |
+--------|-----------------------------------+ +-----|-------+
| |
Flow Records (selected by Observation Domain) Flow Records
| |
+----------------------+----------------------+
|
+----------------------|---------------+
| Exporting Process | |
| +---------------+-----------+ |
| | flow exporting (*) | |
| +---------------+-----------+ |
| | |
+----------------------+---------------+
|
v
Export packet to Collector
(*) indicates where flow selection can take place.
Figure 1: Flow selection as a function of the IPFIX Exporter
In contrast to packet selection, flow selection is always applied
after the packets are classified into flows. Flows can be selected
at different stages of the measurement chain:
Peluso, et al. Expires July 11, 2011 [Page 8]
Internet-Draft Flow Selection Techniques January 2011
1. during metering [RFC5475];
2. during flow recording;
3. during flow exporting.
4.1. Flow selection in the metering process
The main reason for applying flow state dependent sampling during the
metering process is that the flow recording process may not have, at
a certain point in time, enough memory positions to record all
observable flows. Another reason may be that there might not be
enough processing resources to create and manage a new flow record.
To overcome these limitations, a number of possible policies can be
applied, the simplest one being to discard new packets which cannot
be assigned to existing flow records (i.e. that would require the
creation of a new flow record). More complex policies are
applicable, mainly aimed at detecting the so called elephant flows,
so as to prioritize flows carrying a higher traffic volume in the
flow recording process. For instance, [EsVa01] proposes criteria to
define a packet as eligible to create a new flow record (sample and
hold, multistage filters).
Regardless of specific algorithms, we are concerned about identifying
what information on the flow state dependent packet sampling is worth
keeping and making available to applications (by exporting it out of
an IPFIX device). An option could be to keep a cumulative counter of
the total number of packets and bytes that were not taken into
account for measurement purposes because of flow state dependent
sampling. Furthermore, it is possible to keep a timestamp for the
first and last of these discarded packets. In practice, this implies
aggregating all these packets in a single macro flow, and keeping
track of its volume and duration. Storing more detailed information
about packets which have not been measured because of flow state
dependent sampling would contradict the fact that the sampling is
done because of lack of memory and/or processing resources.
4.2. Flow selection in the flow recording process
As described in the previous section, because of lack of memory
positions in the flow recording process some incoming packets might
be discarded if they lead to the creation of a new flow record.
However, under certain circumstances, it may be advantageous to
discard an existing flow record during the flow recording process in
order to make room for a new one which has been created upon arrival
of a new packet. For example, an algorithm to make the decision
whether to discard a new incoming packet or an existing flow record
Peluso, et al. Expires July 11, 2011 [Page 9]
Internet-Draft Flow Selection Techniques January 2011
is described in [Moli03].
In this section we focus on the selection of the information to be
stored regarding the record removal rather than on the details of the
decision making algorithm. For the reasons we mentioned above, it
does not make sense to store separate information for each discarded
flow record, as it would contradict the motivation why discarding is
done (i.e. lack of memory resources). The information that can be
kept with a limited overhead is the cumulative counter of the total
number of not yet exported packets and bytes belonging to flow
records that were removed during the flow recording process.
Ideally, we would like to keep also a timestamp for the first (T_fd)
and last (T_ld) not yet exported packets belonging to every discarded
flow record. This would mean aggregating all these packets in a
macro flow, and keeping track of its volume and duration. To do so,
we would need to maintain a timestamp of the first and last non-
exported packets in each flow record. The values of such timestamps
would be checked whenever a record is discarded in order to verify
whether they are smaller or larger than T_fd and T_ld, respectively.
If so, the timestamps would be updated.
Another information that can be easily maintained is the number of
discarding actions, along with the timestamps of the first and last
action. This information SHOULD not be used by applications to re-
normalize the received per flow statistics (because a flow may be
discarded and created multiple times), but rather to monitor and
control the performance of the implemented policy. Note that we take
into account a discarding event only when the discarded flow record
contains data about traffic which has not been exported.
Differently, the removal of a record whose traffic was exported
(either after a timeout or after the arrival of specific packets,
e.g. TCP FIN or RST) is part of the normal operation of an IPFIX
flow metering system. Note also that we consider only the case when
the elimination of a flow record during the flow recording process
leads to the complete loss of all the information contained in the
flow record itself.
The implementation of a different policy, such as immediate exporting
of the flow record before elimination, or freezing of the flow record
and moving it to another area of memory for later exporting, is not
considered as an elimination and therefore it is out of the scope of
this document.
Along with the information about the number of discarded flow records
and associated packets and bytes, it is useful to keep cumulative
information about the number of flow records containing not yet
exported traffic and being currently handled by the flow recording
Peluso, et al. Expires July 11, 2011 [Page 10]
Internet-Draft Flow Selection Techniques January 2011
process. Another information worth keeping is the cumulative number
of not exported packets and bytes contained in them.
4.3. Flow selection during the exporting process
The exporting process may implement policies for exporting only a
subset of the flow records which have been stored in the system
memory. The decision to export only a subset of the flow records can
be motivated by the existence of an explicit policy which filters out
the flow records to be exported. An example of such a policy could
be to export only the flow records associated to flows whose
accounted traffic is below a certain threshold, or to implement a
more complex mechanism such as the one described in [DuLT01a] or
[DuLT01b].
Another motivation which might bring to the exporting of a subset of
stored flow records is resource limitation. For example, the
exporting process has been assigned a limited time slot to operate or
it exports only a predefined number of packets. Hybrid cases can
happen where the exporting of a subset of the flow records is
motivated by the co-existence of resource limitations and ad-hoc
policies which are applied in order to optimize the exporting
process. For example, given that the exporting process applies to a
subset of the flow records, such subset is selected so that the
overall number of exported packets and bytes belonging to the subset
is maximized.
Selecting flow records during the exporting process raises the issue
of identifying the information which is worth keeping about the flow
selection process. Two different scenarios cab be envisaged. If a
flow record is not exported and then it does not feed the flow
recording process, the scenario is the same as when the deletion of
the flow record is caused by the need to make room to another record.
The metrics to be kept are cumulative packets and bytes associated
with non-exported flow records, timestamps of the first and last
packets belonging to non-exported flow records, counter of dropping
events and timestamps of first and last dropping event.
If a record eligible for exporting is not exported and it enters the
flow recording process it has a chance of being exported in the
future. It would be beneficial for an application to get
information, in terms of number of generated packets and bytes, about
the flow records which are not being exported due to the existence of
exporting policies and/or resource limitations. This is intended to
make it possible to detect potential pathologic conditions, like the
missed exporting of a large number of flow records and/or associated
traffic, or the growing number of flow records being involved in the
flow recording process.
Peluso, et al. Expires July 11, 2011 [Page 11]
Internet-Draft Flow Selection Techniques January 2011
The selection of the flow records to be exported implies performing a
complete scanning of the memory area where flow information is
stored, thus jeopardizing the efficiency of the overall exporting
process. For this reason, flow exporting protocol specification does
not include flow selection during the recording process as a
mandatory function even if the information model has been designed to
enable such function.
5. Flow selection as a function of the IPFIX Mediator
As shown in Figure 2, flow selection can be performed as an
intermediate process within an IPFIX Mediator. This process selects
the flow records from a sequence which meets pre-defined criteria and
exports them to an IPFIX Collector. This selection function can be
seen as a more fine-grained process with respect to the selection
performed by an IPFIX Exporter. The criteria used to drive the
selection process at Mediator's level might be applied to the set of
flow records coming from the IPFIX Exporter, thus triggering a
further flow selection process.
Peluso, et al. Expires July 11, 2011 [Page 12]
Internet-Draft Flow Selection Techniques January 2011
Packet(s) coming in to Observation Point(s)
|
IPFIX Original |
Exporter v
+------------------+-------------------+
| |
| Metering Process on an |
| Observation Point |
| | |
| Flow metering and selection |
| | |
| Flow recording and selection |
| | |
| Flow exporting and selection |
| | |
+------------------+-------------------+
|
v
Flow Records (selected by Observation Domain)
|
IPFIX Mediator v
+------------------+-------------------+
| |
| Collecting process |
| | |
| Flow selection (*) |
| | |
| Exporting process |
| | |
+------------------+-------------------+
|
v
Flow Records
Figure 2: Flow selection as a function of the IPFIX Mediator
As an example, if an IPFIX Mediator interacts with a set of IPFIX
Collectors, flow records arriving at the IPFIX Mediator might be
selected based on the IPFIX Collector requesting flow information.
As described in previous sections, flow selection can take place
during metering, recording, and exporting processes of an IPFIX
exporter depending on the policies which are implemented to meet
application requirements. In case flow selection is performed at
Mediator's level, we envisage the use of flow selection techniques as
a step of the exporting process aimed to identify the flow records to
be exported among those stored in the system's memory. This is
because the lighter is the intermediate selection process, the better
Peluso, et al. Expires July 11, 2011 [Page 13]
Internet-Draft Flow Selection Techniques January 2011
is the performance of the mediation framework.
6. Flow selection techniques
We can distinguish the following selection techniques:
1. based on flow record content (i.e. all reported flow
characteristics);
2. based on flow record arrival time;
3. based on external events like the exhaustion of local resources.
6.1. Flow selection based on flow record content
Flow selection can be done based on fields in an IPFIX flow record.
This can be done similarly to field match filtering for packet
selection described in [RFC5475]. The difference is that instead of
packet fields flow record fields are here used to drive the selection
decision. An example would be to select flow records based on
parameters like the flow size in bytes or the number of packets.
Another application would be to select flow records based on either
flow start time or flow keys (IP addresses, ports) of the stored flow
record.
6.2. Flow selection based on flow record arrival time or sequence
Flow records can be selected based on their arrival time at the
exporting process. An example would be to select a number of flow
records during certain time intervals. Another option is to select
flow records based on the order they arrive at the exporting process.
In such case, one can either periodically select the k-th records or
select randomly a set of flow records.
6.3. Flow selection on external events
The selection of flow records can be also triggered by external
events. An example would be to activate flow selection based on the
value of a router state parameter like the number of entries in the
flow cache.
7. Information model for flow selection information exporting
In this section we define the elements devoted to containing the
information described in the previous sections. Some elements have
been associated with a pair of timestamps, which are referred to as
Peluso, et al. Expires July 11, 2011 [Page 14]
Internet-Draft Flow Selection Techniques January 2011
element_nameTsFirst and element_nameTsLast. We would like to point
out that only packet or flow related counts have associated
timestamps, while bytes related counts do not. The reason is that
element_nameTsFirst and element_nameTsLast are referred to the
timestamp of the first and last received packets belonging to not
exported flows, while timestamp is meanliness with regard to bytes
information.
Note that all the following information elements are aimed at
describing macro flows parameters (e.g. the total number of packets
and bytes contained in all dropped or not created flow records).
Some of these macro flows parameters are additive, in the sense that
it is only possible to add contributions to them, but never subtract
some amount. For example, the macro flow of the packets contained in
flow records that are discarded from the flow reporting process
receives an addition when a flow record is discarded, and this
addition can never be subtracted. On the other side, some macro flow
parameters can dynamically receive and loose additions. For example,
the macro flows of packets not yet exported receives an addition when
a new packet arrives, and looses addition as an exporting event takes
place. Associating a timestamp to the oldest and most recent
additions in case of additive flows is an easy task, while it is
complicated for the others (it would require to maintain full state
information) and that is why we did not define timestamps for these
information elements.
The information elements herein introduced are defined in accordance
with the IPFIX information model [RFC5102]. Furthermore, the data
types used to represent the Flow Selection-related information
elements are those defined in section 3.1 of the IPFIX information
model [RFC5102].
List of additional Flow Selection information elements:
Peluso, et al. Expires July 11, 2011 [Page 15]
Internet-Draft Flow Selection Techniques January 2011
+-------+---------------------------------------+
| ID | Name |
+-------+---------------------------------------+
| TBD1 | fsMeterUnmeasPacketCountTsFirst |
+-------+---------------------------------------+
| TBD2 | fsMeterUnmeasPacketCountTsLast |
+-------+---------------------------------------+
| TBD3 | fsMeterUnmeasBytesCount |
+-------+---------------------------------------+
| TBD4 | fsFrecPacketInDroppedRecsCountTsFirst |
+-------+---------------------------------------+
| TBD5 | fsFrecPacketInDroppedRecsCountTsFirst |
+-------+---------------------------------------+
| TBD6 | fsFrecByteInDroppedRecsCount |
+-------+---------------------------------------+
| TBD7 | fsFrecFrecDroppedCountTsFirst |
+-------+---------------------------------------+
| TBD8 | fsFrecFrecDroppedCountTsLast |
+-------+---------------------------------------+
| TBD9 | fsFrecUnexportedFrecCount |
+-------+---------------------------------------+
| TBD10 | fsFrecUnexportedPacketInFrecCount |
+-------+---------------------------------------+
| TBD11 | fsFrecUnexportedBytesInFrecCount |
+-------+---------------------------------------+
| TBD12 | fsExpPacketInDroppedRecsCountTsFirst |
+-------+---------------------------------------+
| TBD13 | fsExpPacketInDroppedRecsCountTsLast |
+-------+---------------------------------------+
| TBD14 | fsExpBytesInDroppedRecsCount |
+-------+---------------------------------------+
| TBD15 | fsExpFrecDroppedCountTsFirst |
+-------+---------------------------------------+
| TBD16 | fsExpFrecDroppedCountTsLast |
+-------+---------------------------------------+
| TBD17 | fsExpUnexportedCount |
+-------+---------------------------------------+
| TBD18 | fsExpUnexportedPacketCount |
+-------+---------------------------------------+
| TBD19 | fsExpUnexportedByteInExpCount |
+-------+---------------------------------------+
7.1. Meter process related (TBD1-TBD3)
Information Elements presented in this section are related to Flow
Selection performed during the Metering Process.
Peluso, et al. Expires July 11, 2011 [Page 16]
Internet-Draft Flow Selection Techniques January 2011
+------+---------------------------------+
| ID | Name |
+------+---------------------------------+
| TBD1 | fsMeterUnmeasPacketCountTsFirst |
+------+---------------------------------+
| TBD2 | fsMeterUnmeasPacketCountTsLast |
+------+---------------------------------+
| TBD3 | fsMeterUnmeasBytesCount |
+------+---------------------------------+
7.1.1. fsMeterUnmeasPacketCountTsFirst
Description:
Specifies the timestamp of the first packet not measured because
of the use of the flow state dependent sampling. Together with
the IE fsMeterUnmeasPacketCountTsLast it allows to evaluate the
count of packets that were not measured because of the use of the
flow sampling.
Abstract Data Type: dateTimeSeconds
ElementId: TBD2
Status: Proposed
Units: seconds
7.1.2. fsMeterUnmeasPacketCountTsLast
Description:
Specifies the timestamp of the last packet not measured because of
the use of the flow state dependent sampling. Together with the
IE fsMeterUnmeasPacketCountTsFirst it allows to evaluate the count
of packets that were not measured because of the use of the flow
sampling.
Abstract Data Type: dateTimeSeconds
ElementId: TBD2
Status: Proposed
Units: seconds
Peluso, et al. Expires July 11, 2011 [Page 17]
Internet-Draft Flow Selection Techniques January 2011
7.1.3. fsMeterUnmeasBytesCount
Description:
This Information Element specifies the count of bytes that were
not measured because of the use of the flow state dependent
sampling.
Abstract Data Type: unsigned64
Data Type Semantics: deltaCounter
ElementId: TBD3
Status: Proposed
Units: bytes
7.2. Flow recording process related (TBD4-TBD11)
Information Elements presented in this section are related to Flow
Selection performed during the Flow Recording Process.
+-------+---------------------------------------+
| ID | Name |
+-------+---------------------------------------+
| TBD4 | fsFrecPacketInDroppedRecsCountTsFirst |
+-------+---------------------------------------+
| TBD5 | fsFrecPacketInDroppedRecsCountTsLast |
+-------+---------------------------------------+
| TBD6 | fsFrecByteInDroppedRecsCount |
+-------+---------------------------------------+
| TBD7 | fsFrecFrecDroppedCountTsFirst |
+-------+---------------------------------------+
| TBD8 | fsFrecFrecDroppedCountTsLast |
+-------+---------------------------------------+
| TBD9 | fsFrecUnexportedFrecCount |
+-------+---------------------------------------+
| TBD10 | fsFrecUnexportedPacketInFrecCount |
+-------+---------------------------------------+
| TBD11 | fsFrecUnexportedBytesInFrecCount |
+-------+---------------------------------------+
7.2.1. fsFrecPacketInDroppedRecsCountTsFirst
Description:
Peluso, et al. Expires July 11, 2011 [Page 18]
Internet-Draft Flow Selection Techniques January 2011
Specifies the timestamp of the first non-exported packet that was
contained in the flow records eliminated from the flow recording
process because of resource limitations/policies in the flow
recording process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD4
Status: Proposed
Units: seconds
7.2.2. fsFrecPacketInDroppedRecsCountTsLast
Description:
Specifies the timestamp of the last non-exported packet that was
contained in the flow records eliminated from the flow recording
process because of resource limitations/policies in the flow
recording process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD5
Status: Proposed
Units: seconds
7.2.3. fsFrecByteInDroppedRecsCount
Description:
This Information Element specifies the count of the non-exported
bytes that were contained in the flow records eliminated from the
flow recording process because of resource limitations/policies in
the flow recording process.
Abstract Data Type: unsigned64
Data Type Semantics: deltaCounter
ElementId: TBD6
Status: Proposed
Units: bytes
Peluso, et al. Expires July 11, 2011 [Page 19]
Internet-Draft Flow Selection Techniques January 2011
7.2.4. fsFrecFrecDroppedCountTsFirst
Description:
Specifies the timestamp of the first flow record elimination event
occurring during the flow recording process. Together with the IE
fsFrecFrecDroppedCountTsLast it allows to estimate the count of
flow records containing non-exported packets eliminated from the
flow recording process because of resources limitations/policies
in the flow recording process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD7
Status: Proposed
Units: seconds
7.2.5. fsFrecFrecDroppedCountTsLast
Description:
Specifies the timestamp of the last flow record elimination event
occurring during the flow recording process. Together with the IE
fsFrecFrecDroppedCountTsFirst it allows to estimate the count of
flow records containing non-exported packets eliminated from the
flow recording process because of resources limitations/policies
in the flow recording process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD8
Status: Proposed
Units: seconds
7.2.6. fsFrecUnexportedFrecCount
Description:
This Information Element specifies the count of the flow records
currently existing in the flow recording process containing at
least one non-exported packet.
Abstract Data Type: unsigned32
Peluso, et al. Expires July 11, 2011 [Page 20]
Internet-Draft Flow Selection Techniques January 2011
Data Type Semantics: deltaCounter
ElementId: TBD9
Status: Proposed
Units: flow records
7.2.7. fsFrecUnexportedPacketInFrecCount
Description:
This Information Element specifies the count of non-exported
packets contained in flow records of the flow recording process.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
ElementId: TBD10
Status: Proposed
Units: packets
7.2.8. fsFrecUnexportedBytesInFrecCount
Description:
This Information Element specifies the count of non-exported bytes
contained in flow records of the flow recording process.
Abstract Data Type: unsigned64
Data Type Semantics: deltaCounter
ElementId: TBD11
Status: Proposed
Units: bytes
7.3. Flow exporting process related (TBD12-TBD19)
Information Elements presented in this section are related to Flow
Selection performed during the Flow Exporting Process.
Peluso, et al. Expires July 11, 2011 [Page 21]
Internet-Draft Flow Selection Techniques January 2011
+-------+--------------------------------------+
| ID | Name |
+-------+--------------------------------------+
| TBD12 | fsExpPacketInDroppedRecsCountTsFirst |
+-------+--------------------------------------+
| TBD13 | fsExpPacketInDroppedRecsCountTsLast |
+-------+--------------------------------------+
| TBD14 | fsExpByteInDroppedRecsCount |
+-------+--------------------------------------+
| TBD15 | fsExpFrecDroppedCountTsFirst |
+-------+--------------------------------------+
| TBD16 | fsExpFrecDroppedCountTsLast |
+-------+--------------------------------------+
| TBD17 | fsExpUnexportedCount |
+-------+--------------------------------------+
| TBD18 | fsExpUnexportedPacketCount |
+-------+--------------------------------------+
| TBD19 | fsExpUnexportedByteInExpCount |
+-------+--------------------------------------+
7.3.1. fsExpPacketInDroppedRecsCountTsFirst
Description:
Specifies the timestamp of the first non-exported packet belonging
to an eliminated flow record. Together with the IE
fsExpPacketInDroppedRecsCountTsLast it allows to estimate the
count of non-exported packets that were contained in the flow
records eliminated from the flow recording process because of
resource limitations/policies in the exporting process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD12
Status: Proposed
Units: seconds
7.3.2. fsExpPacketInDroppedRecsCountTsLast
Description:
Specifies the timestamp of the last non-exported packet belonging
to an eliminated flow record. Together with the IE
fsExpPacketInDroppedRecsCountTsFirst it allows to estimate the
count of non-exported packets that were contained in the flow
records eliminated from the flow recording process because of
Peluso, et al. Expires July 11, 2011 [Page 22]
Internet-Draft Flow Selection Techniques January 2011
resource limitations/policies in the exporting process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD13
Status: Proposed
Units: seconds
7.3.3. fsExpByteInDroppedRecsCount
Description:
This Information Element specifies the count of non-exported bytes
that were contained in the flow records eliminated from the flow
recording process because of resource limitations/policies in the
exporting process.
Abstract Data Type: unsigned64
Data Type Semantics: deltaCounter
ElementId: TBD14
Status: Proposed
Units: bytes
7.3.4. fsExpFrecDroppedCountTsFirst
Description:
Specifies the timestamp of the first flow record elimination event
occurring during the flow recording process. Together with the IE
fsExpFrecDroppedCountTsLast it allows to estimate the count of
flow records containing non-exported packets eliminated from the
flow recording process because of resource limitations/policies in
the exporting process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD15
Status: Proposed
Units: seconds
Peluso, et al. Expires July 11, 2011 [Page 23]
Internet-Draft Flow Selection Techniques January 2011
7.3.5. fsExpFrecDroppedCountTsLast
Description:
Specifies the timestamp of the last flow record elimination event
occurring during the flow recording process. Together with the IE
fsExpFrecDroppedCountTsFirst it allows to estimate the count of
flow records containing non-exported packets eliminated from the
flow recording process because of resource limitations/policies in
the exporting process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD16
Status: Proposed
Units: seconds
7.3.6. fsExpUnexportedCount
Description:
This Information Element specifies the count of the flow records
currently existing in the flow recording process, containing non-
exported traffic and not being exported because of exporting
process resource limitations/policies.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
ElementId: TBD17
Status: Proposed
Units: flow records
7.3.7. fsExpUnexportedPacketCount
Description:
This Information Element specifies the count of non-exported
packets contained in the flow records of the flow recording
process not being exported because of exporting process resource
limitations/policies.
Abstract Data Type: unsigned32
Peluso, et al. Expires July 11, 2011 [Page 24]
Internet-Draft Flow Selection Techniques January 2011
Data Type Semantics: deltaCounter
ElementId: TBD18
Status: Proposed
Units: packets
7.3.8. fsExpUnexportedByteInExpCount
Description:
This Information Element specifies the count of non-exported bytes
contained in the flow records of the flow recording process not
being exported because of exporting process resource limitations/
policies.
Abstract Data Type: unsigned64
Data Type Semantics: deltaCounter
ElementId: TBD19
Status: Proposed
Units: bytes
8. Implementation requirements
In order to implement the described information model counters for
non-exported packets and bytes have to be inserted in the flow
records as flow metrics. Sometimes these counters are referred to as
delta counts. An implementation may also keep absolute counts for
purposes not being specified in this information model (both delta
and absolute counters can be exported in the IPFIX information model,
see [RFC5102]). In addition, to fully support this information
model, it would be REQUIRED to keep in a flow record the timestamps
of the first and last non-exported packets. An implementation may
need to keep timestamps of the first and last exported packets for
other purposes than those of this information model, or to join the
two timers for the last exported and first exported packets, or even
to approximate them with the time of the exporting event.
9. Information Model for Configuration of Flow Selection Techniques
This section aims at describing the representative parameters of the
Peluso, et al. Expires July 11, 2011 [Page 25]
Internet-Draft Flow Selection Techniques January 2011
flow selection techniques presented above. To this regard, it
provides the basis of an information model to be adopted in order to
configure the flow selection process within an IPFIX device. The
information elements herein introduced are defined in accordance with
the IPFIX information model [RFC5102]. Furthermore, the data types
used to represent the Flow Selection-related information elements are
those defined in section 3.1 of the IPFIX information model
[RFC5102].
List of additional Flow Selection information elements:
+-------+-------------------------+-------+-----------------------+
| ID | Name | ID | Name |
+-------+-------------------------+-------+-----------------------+
| TBD20 | selectorMethod | TBD21 | flowRecordPacketsSize |
+-------+-------------------------+-------+-----------------------+
| TBD22 | flowMaxAdmitFlowRecords | TBD23 | flowInactivityTime |
+-------+-------------------------+-------+-----------------------+
| TBD24 | flowRecordBytesSize | | |
+-------+-------------------------+-------+-----------------------+
9.1. selectorMethod
Description:
This Information Element identifies the flow selection method that
is applied by the Flow Selection process, in accordance with what
is described in section 5 of this document.
Some of these methods MAY have parameters in order to fully
support the selected technique. For that reason, further
Information Elements are defined in the following subsections.
Flow selection methods identifiers are herein defined:
+----+-----------------+--------------------------------------------+
| ID | Method | Parameters |
+----+-----------------+--------------------------------------------+
| 1 | Selection based | flowMaxAdmitFlowRecords |
| | on flow size | flowRecordBytesSize flowRecordPacketsSize |
| | count | |
+----+-----------------+--------------------------------------------+
| 2 | Selection based | flowMaxAdmitFlowRecords |
| | on flow content | flowIPAddressVersion flowSourceIPv4Address |
| | property match | flowDestinationIPv4Address |
| | | flowSourceIPv6Address |
| | | flowDestinationIPv6Address |
| | | flowDestinationPort |
Peluso, et al. Expires July 11, 2011 [Page 26]
Internet-Draft Flow Selection Techniques January 2011
| 3 | Selection based | flowMaxAdmitFlowRecords flowInactivityTime |
| | on flow record | |
| | arrival time or | |
| | sequence | |
+----+-----------------+--------------------------------------------+
| 4 | Selection based | flowMaxAdmitFlowRecords |
| | on external | |
| | events | |
+----+-----------------+--------------------------------------------+
Abstract Data Type: unsigned16
Data Type Semantics: identifier
ElementId: TBD20
Status: Proposed
9.2. flowMaxAdmitFlowRecords
Description:
This Information Element specifies the maximum number of eligible
flow records which might be created in the flow cache. It is used
by the Selector Process in order to identify the time when flow
selection should be triggered. A value of 0 means that the Flow
Selection State related to the memory space available for flow
recording MUST be used to estimate the max flow cache size.
For example, this Information Element MAY be used to set the
configuration of a flow size count Flow Selector.
Abstract Data Type: unsigned32
Data Type Semantics: quantity
ElementId: TBD21
Status: Proposed
Units: flow records
9.3. flowRecordBytesSize
Description:
This Information Element specifies the minimum number of bytes
contained in a flow record to be considered as not eligible for
Peluso, et al. Expires July 11, 2011 [Page 27]
Internet-Draft Flow Selection Techniques January 2011
removal. It MAY be used for elephant flows identification.
For example, this Information Element MAY be used to describe the
configuration of a flow size count Flow Selector.
Abstract Data Type: unsigned64
Data Type Semantics: quantity
ElementId: TBD22
Status: Proposed
Units: bytes
9.4. flowRecordPacketsSize
Description:
This Information Element specifies the minimum number of packets
contained in a flow record to be considered as not eligible for
removal. It MAY be used for elephant flows identification.
For example, this Information Element MAY be used to describe the
configuration of a flow size count Flow Selector.
Abstract Data Type: unsigned32
Data Type Semantics: quantity
ElementId: TBD23
Status: Proposed
Units: packets
9.5. flowInactivityTime
Description:
This Information Element specifies the time interval in
microseconds during which the corresponding flow record may be
considered as still active. It is used by the metering process
and/or the flow recording process in order to make the decision on
whether to discard an existing flow to make room for a new one.
For example, this Information Element may be used to describe the
configuration of a flow arrival time Flow Selector.
Peluso, et al. Expires July 11, 2011 [Page 28]
Internet-Draft Flow Selection Techniques January 2011
Abstract Data Type: dateTimeMicroseconds
Data Type Semantics: quantity
ElementId: TBD24
Status: Proposed
Units: microseconds
9.6. flowIPAddressVersion
Description:
This Information Element specifies the IP version field of the
packets belonging to flow records which SHOULD be considered as
not eligible for removal.
Abstract Data Type: unsigned8
Data Type Semantics: identifier
ElementId: 60
Status: current
Reference: see [RFC5102] for the definition of the ipVersion
information element and its Id.
9.7. flowIPv4SourceAddress
Description:
This Information Element specifies the IPv4 source address which
is the parameter to be used as flow key to access the flow record
not eligible for removal.
Abstract Data Type: ipv4Address
Data Type Semantics: identifier
ElementId:8
Status: current
Reference: see [RFC5102] for the definition of the sourceIPv4Address
information element and its Id.
Peluso, et al. Expires July 11, 2011 [Page 29]
Internet-Draft Flow Selection Techniques January 2011
9.8. flowIPv4DestinationAddress
Description:
This Information Element specifies the IPv4 destination address
which is the parameter to be used as flow key to access the flow
record not eligible for removal.
Abstract Data Type: ipv4Address
Data Type Semantics: identifier
ElementId:12
Status: current
Reference: see [RFC5102] for the definition of the
destinationIPv4Address information element and its Id.
9.9. flowIPv6SourceAddress
Description:
This Information Element specifies the IPv6 source address which
is the parameter to be used as flow key to access the flow record
not eligible for removal.
Abstract Data Type: ipv6Address
Data Type Semantics: identifier
ElementId:27
Status: current
Reference: see [RFC5102] for the definition of the sourceIPv6Address
information element and its Id.
9.10. flowIPv6DestinationAddress
Description:
This Information Element specifies the IPv6 destination address
which is the parameter to be used as flow key to access the flow
record not eligible for removal.
Abstract Data Type: ipv6Address
Peluso, et al. Expires July 11, 2011 [Page 30]
Internet-Draft Flow Selection Techniques January 2011
Data Type Semantics: identifier
ElementId:28
Status: current
Reference: see [RFC5102] for the definition of the
destinationIPv6Address information element and its Id.
10. IANA Considerations
This document introduces several new information elements as an
extension to the IPFIX information model. Values TBD1-TBD19 in this
document should be replaced with the assigned numbers by IANA.
11. Security Considerations
In this section security issues concerning an IPFIX device performing
flow selection are pointed out. In case the flow selection function
is activated an IPFIX device might be exposed to security threats.
Since flow selection implies analysing flow packets, associating them
to a specific traffic flow and selecting flow records, a malicious
user who was able to gain control of an IPFIX device might access
both packet and flow data, thus violating their confidentiality.
Furthermore, the intruder might be attracted by the possibility of
altering the flow selection process by modifying the criteria used to
select flow records. In this case, the IPFIX device would export
flow data which are different from the ones that the Collector
expects to receive.
It is apparent that these security threats can be mitigated by
authenticating entities that interact with the IPFIX device and
keeping information for flow selection configuration confidential.
12. References
12.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
Peluso, et al. Expires July 11, 2011 [Page 31]
Internet-Draft Flow Selection Techniques January 2011
12.2. Informative References
[DuLT01a] Duffield, N., Lund, C., and M. Thorup, "Charging from
Sampled Network Usage", ACM Internet Measurement Workshop
IMW 2001, San Francisco, USA, November 2001.
[DuLT01b] Duffield, N., Lund, C., and M. Thorup, "Properties and
Prediction of Flow Statistics from Sampled Packet
Streams", ACM SIGCOMM Internet Measurement Workshop 2002,
November 2002.
[EsVa01] Estan, C. and G,. Varghese, "New Directions in Traffic
Measurement and Accounting: Focusing on the Elephants,
Ignoring the Mice", ACM SIGCOMM Internet Measurement
Workshop 2001, San Francisco (CA), November 2001.
[IPFIX-REQ]
Quittek, J., Zseby, T., Claise, B., and S. Zander,
"Requirements for IP Flow Information Export",
RFC3917 Requirements for IP Flow Information Export
(IPFIX), July 2008.
[KuXW04] Kumar, K., Xu, J., Wang, J., Spatschek, O., and L. Li,
"Space-code bloom filter for efficient per-flow traffic
measurement", INFOCOM 2004 Twenty-third AnnualJoint
Conference of the IEEE Computer and Communications
Societies, March 2004.
[Moli03] Molina, M., "A scalable and efficient methodology for flow
monitoring in the Internet", International Teletraffic
Congress (ITC-18), Berlin, September 2003.
[RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
Meyer, "Information Model for IP Flow Information Export",
RFC 5102, January 2008.
[RFC5470] Sadasivan, G., Bownlee, N., Claise, B., and J. Quittek,
"Architecture for IP Flow Information Export",
RFC5470 Architecture for IP Flow Information Export,
September 2006.
[RFC5475] Zseby, T., Molina, M., Raspall, F., Duffield, N., and S.
Niccolini, "Sampling and Filtering techniques for IP
Packet Selection", RFC5475 Sampling and Filtering
techniques for IP Packet Selection, July 2008.
Peluso, et al. Expires July 11, 2011 [Page 32]
Internet-Draft Flow Selection Techniques January 2011
Authors' Addresses
Lorenzo Peluso
University of Napoli
Via Claudio 21
Napoli 80125
Italy
Phone: +39 081 7683821
Email: lorenzo.peluso@unina.it
Tanja Zseby
Fraunhofer Institute FOKUS
Kaiserin-Augusta-Allee 31
Berlin 10589
Germany
Phone: +49 30 3463 7153
Email: tanja.zseby@fokus.fraunhofer.de
Salvatore D'Antonio
CINI Consortium/University of Napoli "Parthenope"
Monte S.Angelo, Via Cinthia
Napoli 80126
Italy
Phone: +39 081 679944
Email: saldanto@unina.it
Christian Henke
Fraunhofer Institute FOKUS
Kaiserin-Augusta-Allee 31
Berlin 10589
Germany
Phone: +49 30 3463 7366
Email: christian.henke@fokus.fraunhofer.de
Peluso, et al. Expires July 11, 2011 [Page 33]
| PAFTECH AB 2003-2026 | 2026-04-23 14:32:40 |