One document matched: draft-ietf-ipfix-flow-selection-tech-05.txt
Differences from draft-ietf-ipfix-flow-selection-tech-04.txt
Internet Engineering Task Force L. Peluso
Internet-Draft University of Napoli
Intended status: Standards Track T. Zseby
Expires: September 15, 2011 Fraunhofer Institute FOKUS
S. D'Antonio
CINI Consortium/University of
Napoli "Parthenope"
C. Henke
Fraunhofer Institute FOKUS
March 14, 2011
Flow Selection Techniques
draft-ietf-ipfix-flow-selection-tech-05.txt
Abstract
Flow selection is the process of selecting a subset of flows from all
flows observed at an observation point. The objective of flow
selection is to reduce the effort of post-processing flow data and
transferring flow records. The flow selection process can be enabled
at different stages of the measurement process. It can be applied
either directly after classification or at recording/exporting time
by limiting the number of flows to be stored and/or exported to the
collecting process. This document describes motivations for flow
selection and presents flow selection techniques. It furthermore
provides an information model for configuring flow selection
techniques and discusses what information about a flow selection
process is worth exporting through a suitable information model.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
Peluso, et al. Expires September 15, 2011 [Page 1]
Internet-Draft Flow Selection Techniques March 2011
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 15, 2011.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November
10, 2008. The person(s) controlling the copyright in some of this
material may not have granted the IETF Trust the right to allow
modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other
than English.
Peluso, et al. Expires September 15, 2011 [Page 2]
Internet-Draft Flow Selection Techniques March 2011
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5
2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6
4. Flow selection as a function of the IPFIX Exporter . . . . . . 7
4.1. Flow selection in the metering process . . . . . . . . . . 9
4.2. Flow selection in the Flow Recording Process . . . . . . . 9
4.3. Flow selection during the export process . . . . . . . . . 11
5. Flow selection as a function of the IPFIX Mediator . . . . . . 12
6. Flow selection techniques . . . . . . . . . . . . . . . . . . 14
6.1. Flow selection based on flow record content . . . . . . . 14
6.2. Flow selection based on flow record arrival time or
sequence . . . . . . . . . . . . . . . . . . . . . . . . . 14
6.3. Flow selection on external events . . . . . . . . . . . . 14
7. Information model for flow selection information export . . . 14
7.1. Meter process related (TBD1-TBD3) . . . . . . . . . . . . 16
7.1.1. fsNotMetPacketTsFirst . . . . . . . . . . . . . . . . 17
7.1.2. fsNotMetPacketTsLast . . . . . . . . . . . . . . . . . 17
7.1.3. fsNotMetBytesCount . . . . . . . . . . . . . . . . . . 18
7.2. Flow Recording Process related (TBD4-TBD11) . . . . . . . 18
7.2.1. fsNotExpPacketDroppedRecPrTsFirst . . . . . . . . . . 18
7.2.2. fsNotExpPacketDroppedRecPrTsLast . . . . . . . . . . . 19
7.2.3. fsNotExpByteInDroppedRecPrCount . . . . . . . . . . . 19
7.2.4. fsFlowRecDroppedRecPrTsFirst . . . . . . . . . . . . . 20
7.2.5. fsFlowRecDroppedRecPrTsLast . . . . . . . . . . . . . 20
7.2.6. fsFlowRecNotExpRecPrCount . . . . . . . . . . . . . . 20
7.2.7. fsPacketNotExpRecPrCount . . . . . . . . . . . . . . . 21
7.2.8. fsBytesNotExpRecPrCount . . . . . . . . . . . . . . . 21
7.3. Flow export process related (TBD12-TBD19) . . . . . . . . 21
7.3.1. fsPacketNotExpInDroppedFlowRecTsFirst . . . . . . . . 22
7.3.2. fsPacketNotExpInDroppedFlowRecTsLast . . . . . . . . . 22
7.3.3. fsBytesNotExpInDroppedFlowRecCount . . . . . . . . . . 23
7.3.4. fsFlowRecDroppedExpPrTsFirst . . . . . . . . . . . . . 23
7.3.5. fsFlowRecDroppedExpPrTsLast . . . . . . . . . . . . . 24
7.3.6. fsFlowRecNotExpCount . . . . . . . . . . . . . . . . . 24
7.3.7. fsPacketNotExpCount . . . . . . . . . . . . . . . . . 24
7.3.8. fsBytesNotExpCount . . . . . . . . . . . . . . . . . . 25
8. Implementation requirements . . . . . . . . . . . . . . . . . 25
9. Information Model for Configuration of Flow Selection
Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . 25
9.1. flowSelectorMethod . . . . . . . . . . . . . . . . . . . . 26
9.2. flowMaxAdmitFlowRecords . . . . . . . . . . . . . . . . . 27
9.3. flowRecordBytesSize . . . . . . . . . . . . . . . . . . . 27
9.4. flowRecordPacketsSize . . . . . . . . . . . . . . . . . . 28
9.5. flowInactivityTime . . . . . . . . . . . . . . . . . . . . 28
9.6. ipVersion . . . . . . . . . . . . . . . . . . . . . . . . 29
9.7. sourceIPv4Address . . . . . . . . . . . . . . . . . . . . 29
Peluso, et al. Expires September 15, 2011 [Page 3]
Internet-Draft Flow Selection Techniques March 2011
9.8. destinationIPv4Address . . . . . . . . . . . . . . . . . . 29
9.9. sourceIPv6Address . . . . . . . . . . . . . . . . . . . . 30
9.10. destinationIPv6Address . . . . . . . . . . . . . . . . . . 30
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 30
11. Security Considerations . . . . . . . . . . . . . . . . . . . 30
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 31
12.1. Normative References . . . . . . . . . . . . . . . . . . . 31
12.2. Informative References . . . . . . . . . . . . . . . . . . 31
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 32
Peluso, et al. Expires September 15, 2011 [Page 4]
Internet-Draft Flow Selection Techniques March 2011
1. Introduction
This document describes flow selection techniques for traffic
measurements. As stated in [RFC5475], packet selection is the
process of selecting a subset of packets among those collected at an
observation point. The element on which this selection mechanism is
performed is a single packet and the selection decision is random,
deterministic or based on packet properties. In contrast to this,
flow selection techniques consider flows as the basic elements on
which a selection process is performed. A flow is defined as a set
of packets with common properties [RFC5101]. Flow selection reduces
the resource demands for capturing, storing, exporting and post-
processing flow-based measurement results. Flow state dependent
packet sampling allows for reducing reource usage while capturing
traffic packets since only a subset of the observed packets is
captured depending on the values of variables describing flow state.
In case an elephant flow has to be selected, then all packets have to
be captured and, therefore, there is no gain deriving from the use of
flow state dependent packte sampling. In this case a dynanic
extraction of flow definitions corresponding to large flows would be
beneficial [EsVa01]. Maintaining and exporting all flow records to
the collecting process can exhaust available resources with the
result that measurement data is discarded which reduces the utility
of the measurement data and makes accurate estimations impossible.
Flow Selection can be applied to only select flows that are of
interest for a certain application or to select a representative
subset of the flows. Selecting representative subsets of flows
allows for flow characteristic estimation while reducing the
measurement data. Example applications in which flow selections can
be applied are accounting, attack and intrusion detection, traffic
engineering, traffic classification, etc.. In many networks few
large flows contribute to the majority of the overall traffic volume
[DuLT01a], [DuLT01b], which is also referred to as "Quasi-Zipf-Law"
[KuXW04]. This "elephant and mice" phenomenon plays an important
role in flow based measurements as it poses challenges on the flow
selection process depending on the application and the available
resources. For example in accounting purposes it is useful to
concentrate on the so-called "heavy hitter" flows to cope with a
limited flow cache size or limited transmission capacity in times
when resources are scarce.
2. Scope
This document describes flow selection techniques and their
parameters. It addresses the configuration of flow selection
techniques and defines which information should be reported by
devices that perform flow selection. It only describes processes
Peluso, et al. Expires September 15, 2011 [Page 5]
Internet-Draft Flow Selection Techniques March 2011
directly acting on traffic flows during the metering phase and/or the
export phase. Therefore it is assumed that flow selection is
performed after packets are classified into flows. This document
does not address the flow selection effects that might arise from
sampling or filtering packets in the metering process before the
classification process is performed. Such packet selection
techniques are described in [RFC5475] and, therefore, are out of the
scope of this document.
3. Terminology
In this document, as in [RFC5101] and [RFC5476], the first letter of
each IPFIX-specific and PSAMP-specific term is capitalized along with
the IPFIX Mediation-specific terms defined here. This document is
consistent with the terminology introduced in [RFC5101], [RFC5470],
[RFC5475] and [RFC3917]. Further some additional terms are presented
which extend the terminology.
* Classification
Is a filtering process in which packets are mapped to specific
flow records based on the packet properties. These properties
make up the flow key (e.g. header information, packet content, AS
number). In case a flow record for this specific flow key already
exist the flow record is updated, otherwise a new flow record is
created.
* Flow Recording Process
The Flow Recording Process maintains the flow records during the
Metering Process. It is responsible for creating new Flow
Records, updating and deleting existing ones, computing Flow
statistics, detecting Flow expiration times and passing the Flow
Records to the Export Process.
* Flow Selection Process
A Flow Selection Process takes classified packets or flow records
as its input and selects a subset of that set as its output. A
Flow Selection Process MAY run on several instances within the
IPFIX architecture. A Flow Selection Process SHOULD BE used by
the Metering Process to reduce the amount of resources for flow
recording. A Flow Selection Process MAY also be part of an IPFIX
export process and MAY be available as an Intermediate Selection
Process running on an IPFIX Mediator.
* Flow Selection State
Peluso, et al. Expires September 15, 2011 [Page 6]
Internet-Draft Flow Selection Techniques March 2011
A Flow Selection Process may maintain state information for use by
the Flow Selection Process. At a given time, the Flow Selection
State may depend on flows observed at and before that time, as
well as other variables. Examples include:
(i) number of accounted flow records;
(ii) memory space available for flow recording;
(iii) state of the pseudorandom number generators;
(iv) hash values calculated during the selection.
(v) timestamps of flow observation (e.g. first or last packet
of flow) and flow duration
* Flow Selector
A Flow Selector defines the action of a Flow Selection Process on
a single flow of its input. The Flow Selector can make use of the
following information in order to establish whether or not a flow
has to be selected or not:
(i) the content of the flow record;
(ii) any information state related to the flow recording
process;
(iii) any flow selection state that may be maintained by the
Flow Selection Process.
4. Flow selection as a function of the IPFIX Exporter
Figure 1 shows the IPFIX reference model as defined in [RFC5470], and
extends it by introducing the functional components where flow
selection can take place.
Peluso, et al. Expires September 15, 2011 [Page 7]
Internet-Draft Flow Selection Techniques March 2011
Packet(s) coming in to Observation Point(s)
| |
v v
+----------------+---------------------------+ +-----+-------+
| Metering Process on an | | |
| Observation Point | | |
| | | |
| packet header capturing | | |
| | |...| Metering |
| timestamping | | Process N |
| | | | |
| packet selection | | |
| | | | |
| classification | | |
| | | | |
| flow state dependent packet sampling (*) | | |
| | | | |
| aggregation | | |
| | | | |
| flow recording (*) | | |
| | | | |
| | Timing out Flows | | |
| | Handle resource overloads | | |
+--------|-----------------------------------+ +-----|-------+
| |
Flow Records (selected by Observation Domain) Flow Records
| |
+----------------------+----------------------+
|
+----------------------|---------------+
| Export Process v |
| +---------------+-----------+ |
| | flow export (*) | |
| +---------------+-----------+ |
| | |
+----------------------+---------------+
|
v
IPFIX export packet to Collector
(*) indicates where flow selection can take place.
Figure 1: Flow selection as a function of the IPFIX Exporter
In contrast to packet selection, flow selection is always applied
after the packets are classified into flows. Flows can be selected
at different stages of the measurement chain:
Peluso, et al. Expires September 15, 2011 [Page 8]
Internet-Draft Flow Selection Techniques March 2011
1. during metering [RFC5475];
2. during flow recording;
3. during flow export.
4.1. Flow selection in the metering process
The main reason for applying flow selection during the metering
process is that the Flow Recording Process may not have, at a certain
point in time, enough memory and processing resources to record and
manage all observable flows. Further the measurement application may
only be interested in certain flows, depending on Flow properties
(e.g. flow key, flow duration, flow size).
Therefore a number of policies can be applied during the metering
process, the more simpler ones being to discard packets of
uninteresting flows or to discard new packets which cannot be
assigned to existing flow records. More complex policies are
applicable, mainly aimed at detecting the so called elephant flows,
so as to prioritize flows carrying a higher traffic volume in the
Flow Recording Process. For instance, [EsVa01] proposes criteria to
define a packet as eligible to create a new flow record (sample and
hold, multistage filters).
Regardless of specific algorithms, we are concerned about identifying
what information on the flow state dependent packet sampling is worth
keeping and making available to applications (by exporting it out of
an IPFIX device). An option could be to keep a cumulative counter of
the total number of packets and bytes that were not taken into
account for measurement purposes because of flow state dependent
sampling. Furthermore, it is possible to keep a timestamp for the
first and last of these discarded packets. In practice, this implies
aggregating all these packets in a single macro flow, and keeping
track of its volume and duration. Storing more detailed information
about packets which have not been measured because of flow state
dependent sampling would contradict the fact that the sampling is
done because of lack of memory and/or processing resources.
4.2. Flow selection in the Flow Recording Process
Within the Metering Process the flow selection during the Flow
Recording Process has a special role. The Flow Recording Process can
use memory information to derive if a packet which would create a new
flow record should be discarded or not. Under certain circumstances,
it may be advantageous to discard an existing flow record during the
flow recording process in order to make room for a new one which has
been created upon arrival of a new packet. For example, an algorithm
Peluso, et al. Expires September 15, 2011 [Page 9]
Internet-Draft Flow Selection Techniques March 2011
to make the decision whether to discard a new incoming packet or an
existing flow record is described in [Moli03].
In this section we focus on the selection of the information to be
stored regarding the record removal rather than on the details of the
decision making algorithm. For the reasons we mentioned above, it
does not make sense to store separate information for each discarded
flow record, as it would contradict the motivation why discarding is
done (i.e. lack of memory resources). The information that can be
kept with a limited overhead is the cumulative counter of the total
number of not yet exported packets and bytes belonging to flow
records that were removed during the Flow Recording Process.
Ideally, we MAY keep also a timestamp for the first (T_fd) and last
(T_ld) not yet exported packets belonging to every discarded flow
record. This would mean aggregating all these packets in a macro
flow, and keeping track of its volume and duration. To do so, we
would need to maintain a timestamp of the first and last non-
exported packets in each flow record. The values of such timestamps
would be checked whenever a record is discarded in order to verify
whether they are smaller or larger than T_fd and T_ld, respectively.
If so, the timestamps would be updated.
Another information that MAY be easily maintained is the number of
discarding actions, along with the timestamps of the first and last
action. This information SHOULD NOT be used by applications to re-
normalize the received per flow statistics (because a flow may be
discarded and created multiple times), but rather to monitor and
control the performance of the implemented policy. Note that we take
into account a discarding event only when the discarded flow record
contains data about traffic which has not been exported.
Differently, the removal of a record whose traffic was exported
(either after a timeout or after the arrival of specific packets,
e.g. TCP FIN or RST) is part of the normal operation of an IPFIX
flow metering system. Note also that we consider only the case when
the elimination of a flow record during the Flow Recording Process
leads to the complete loss of all the information contained in the
flow record itself.
The implementation of a different policy, such as immediate export of
the flow record before elimination, or freezing of the flow record
and moving it to another area of memory for later exporting, is not
considered as an elimination and therefore it is out of the scope of
this document.
Along with the information about the number of discarded flow records
and associated packets and bytes, it is useful to keep cumulative
information about the number of flow records containing not yet
Peluso, et al. Expires September 15, 2011 [Page 10]
Internet-Draft Flow Selection Techniques March 2011
exported traffic and being currently handled by the Flow Recording
Process. Another information worth keeping is the cumulative number
of not exported packets and bytes contained in them.
4.3. Flow selection during the export process
The export process may implement policies for exporting only a subset
of the flow records which have been stored in the system memory. The
decision to export only a subset of the flow records can be motivated
by the existence of an explicit policy which filters out the flow
records to be exported. An example of such a policy could be to
export only the flow records associated to flows whose accounted
traffic is below a certain threshold, or to implement a more complex
mechanism such as the one described in [DuLT01a] or [DuLT01b].
Another motivation which might bring to the export of a subset of
stored flow records is resource limitation. For example, the export
process has been assigned a limited time slot to operate or it
exports only a predefined number of packets. Hybrid cases can happen
where the export of a subset of the flow records is motivated by the
co-existence of resource limitations and ad-hoc policies which are
applied in order to optimize the export process. For example, given
that the export process applies to a subset of the flow records, such
subset is selected so that the overall number of exported packets and
bytes belonging to the subset is maximized.
Selecting flow records during the export process raises the issue of
identifying the information which is worth keeping about the flow
selection process. Two different scenarios cab be envisaged. If a
flow record is not exported and then it does not feed the Flow
Recording Process, the scenario is the same as when the deletion of
the flow record is caused by the need to make room to another record.
The metrics to be kept are cumulative packets and bytes associated
with non-exported flow records, timestamps of the first and last
packets belonging to non-exported flow records, counter of dropping
events and timestamps of first and last dropping event.
If a record eligible for exporting is not exported and it enters the
Flow Recording Process it has a chance of being exported in the
future. It would be beneficial for an application to get
information, in terms of number of generated packets and bytes, about
the flow records which are not being exported due to the existence of
exporting policies and/or resource limitations. This is intended to
make it possible to detect potential pathologic conditions, like the
missed export of a large number of flow records and/or associated
traffic, or the growing number of flow records being involved in the
Flow Recording Process.
Peluso, et al. Expires September 15, 2011 [Page 11]
Internet-Draft Flow Selection Techniques March 2011
The selection of the flow records to be exported implies performing a
complete scanning of the memory area where flow information is
stored, thus jeopardizing the efficiency of the overall export
process. For this reason, flow export protocol specification does
not include flow selection during the Flow Recording Process as a
mandatory function even if the information model has been designed to
enable such function.
5. Flow selection as a function of the IPFIX Mediator
As shown in Figure 2, flow selection can be performed as an
intermediate process within an IPFIX Mediator [IPFIX-MED]. This
process selects the flow records from a sequence which meet pre-
defined criteria and exports these flow records to an IPFIX
Collector. This selection function can be seen as a more fine-
grained process with respect to the selection performed by an
Exporter. In other terms, the criteria used to drive the selection
process on an IPFIX Mediator might be applied to the set of flow
records coming from the Exporter, thus triggering a further flow
selection process.
Peluso, et al. Expires September 15, 2011 [Page 12]
Internet-Draft Flow Selection Techniques March 2011
Packet(s) coming in to Observation Point(s)
|
IPFIX Original |
Exporter v
+------------------+-------------------+
| |
| Metering Process on an |
| Observation Point |
| | |
| Flow metering and selection |
| | |
| Flow recording and selection |
| | |
| Flow export and selection |
| | |
+------------------+-------------------+
|
v
Flow Records (selected by Observation Domain)
|
IPFIX Mediator v
+------------------+-------------------+
| |
| Collecting process |
| | |
| Flow selection (*) |
| | |
| Export process |
| | |
+------------------+-------------------+
|
v
Flow Records
Figure 2: Flow selection as a function of the IPFIX Mediator
As an example, if an IPFIX Mediator interacts with a set of IPFIX
Collectors, flow records arriving at the IPFIX Mediator might be
selected based on the IPFIX Collector requesting flow information.
As described in previous sections, flow selection can take place
during metering, recording, and export processes of an Exporter
depending on the policies which are implemented to meet application
requirements. In case flow selection is performed at Mediator's
level, we envisage the use of flow selection techniques as a step of
the export process aimed to identify the flow records to be exported
among those stored in the system's memory. This is because the
lighter is the Intermediate Selection Process, the better is the
Peluso, et al. Expires September 15, 2011 [Page 13]
Internet-Draft Flow Selection Techniques March 2011
performance of the mediation framework.
6. Flow selection techniques
We can distinguish the following selection techniques:
1. based on flow record content (i.e. all reported flow
characteristics);
2. based on flow record arrival time;
3. based on external events like the exhaustion of local resources.
6.1. Flow selection based on flow record content
Flow selection can be done based on fields in an IPFIX flow record.
This can be done similarly to field match filtering for packet
selection described in [RFC5475]. The difference is that instead of
packet fields flow record fields are here used to drive the selection
decision. An example would be to select flow records based on
parameters like the flow size in bytes or the number of packets.
Another application would be to select flow records based on either
flow start time or flow keys (IP addresses, ports) of the stored flow
record.
6.2. Flow selection based on flow record arrival time or sequence
Flow records can be selected based on their arrival time at the
export process. An example would be to select a number of flow
records during certain time intervals. Another option is to select
flow records based on the order they arrive at the export process.
In such case, one can either periodically select the k-th records or
select randomly a set of flow records.
6.3. Flow selection on external events
The selection of flow records can be also triggered by external
events. An example would be to activate flow selection based on the
value of a router state parameter like the number of entries in the
flow cache.
7. Information model for flow selection information export
In this section we define the elements devoted to containing the
information described in the previous sections. Some elements have
been associated with a pair of timestamps, which are referred to as
Peluso, et al. Expires September 15, 2011 [Page 14]
Internet-Draft Flow Selection Techniques March 2011
element_nameTsFirst and element_nameTsLast. We would like to point
out that only packet or flow related counts have associated
timestamps, while bytes related counts do not. The reason is that
element_nameTsFirst and element_nameTsLast are referred to the
timestamp of the first and last received packets belonging to not
exported flows, while timestamp is meanliness with regard to bytes
information.
Note that all the following information elements are aimed at
describing macro flows parameters (e.g. the total number of packets
and bytes contained in all dropped or not created flow records).
Some of these macro flows parameters are additive, in the sense that
it is only possible to add contributions to them, but never subtract
some amount. For example, the macro flow of the packets contained in
flow records that are discarded from the flow reporting process
receives an addition when a flow record is discarded, and this
addition can never be subtracted. On the other side, some macro flow
parameters can dynamically receive and loose additions. For example,
the macro flows of packets not yet exported receives an addition when
a new packet arrives, and looses addition as an export event takes
place. Associating a timestamp to the oldest and most recent
additions in case of additive flows is an easy task, while it is
complicated for the others (it would require to maintain full state
information) and that is why we did not define timestamps for these
information elements.
The information elements herein introduced are defined in accordance
with the IPFIX information model [RFC5102]. Furthermore, the data
types used to represent the Flow Selection-related information
elements are those defined in section 3.1 of the IPFIX information
model [RFC 2051].
List of additional Flow Selection information elements:
Peluso, et al. Expires September 15, 2011 [Page 15]
Internet-Draft Flow Selection Techniques March 2011
+-------+---------------------------------------+
| ID | Name |
+-------+---------------------------------------+
| TBD1 | fsNotMetPacketTsFirst |
+-------+---------------------------------------+
| TBD2 | fsNotMetPacketTsLast |
+-------+---------------------------------------+
| TBD3 | fsNotMetBytesCount |
+-------+---------------------------------------+
| TBD4 | fsNotExpPacketDroppedRecPrTsFirst |
+-------+---------------------------------------+
| TBD5 | fsNotExpPacketDroppedRecPrTsLast |
+-------+---------------------------------------+
| TBD6 | fsNotExpByteInDroppedRecPrCount |
+-------+---------------------------------------+
| TBD7 | fsFlowRecDroppedRecPrTsFirst |
+-------+---------------------------------------+
| TBD8 | fsFlowRecDroppedRecPrTsLast |
+-------+---------------------------------------+
| TBD9 | fsFlowRecNotExpRecPrCount |
+-------+---------------------------------------+
| TBD10 | fsPacketNotExpRecPrCount |
+-------+---------------------------------------+
| TBD11 | fsBytesNotExpRecPrCount |
+-------+---------------------------------------+
| TBD12 | fsPacketNotExpInDroppedFlowRecTsFirst |
+-------+---------------------------------------+
| TBD13 | fsPacketNotExpInDroppedFlowRecTsLast |
+-------+---------------------------------------+
| TBD14 | fsBytesNotExpInDroppedFlowRecCount |
+-------+---------------------------------------+
| TBD15 | fsFlowRecDroppedExpPrTsFirst |
+-------+---------------------------------------+
| TBD16 | fsFlowRecDroppedExpPrTsLast |
+-------+---------------------------------------+
| TBD17 | fsFlowRecNotExpCount |
+-------+---------------------------------------+
| TBD18 | fsPacketNotExpCount |
+-------+---------------------------------------+
| TBD19 | fsBytesNotExpCount |
+-------+---------------------------------------+
7.1. Meter process related (TBD1-TBD3)
Information Elements presented in this section are related to Flow
Selection performed during the Metering Process.
Peluso, et al. Expires September 15, 2011 [Page 16]
Internet-Draft Flow Selection Techniques March 2011
+------+-----------------------+
| ID | Name |
+------+-----------------------+
| TBD1 | fsNotMetPacketTsFirst |
+------+-----------------------+
| TBD2 | fsNotMetPacketTsLast |
+------+-----------------------+
| TBD3 | fsNotMetBytesCount |
+------+-----------------------+
7.1.1. fsNotMetPacketTsFirst
Description:
Specifies the timestamp of the first packet not metered because of
the use of the flow state dependent sampling. Together with the
IE fsMeterUnmeasPacketCountTsLast it allows to evaluate the count
of packets that were not metered by the Metering Process because
of the use of the flow sampling.
Abstract Data Type: dateTimeSeconds
ElementId: TBD1
Status: Proposed
Units: seconds
7.1.2. fsNotMetPacketTsLast
Description:
Specifies the timestamp of the last packet not metered because of
the use of the flow state dependent sampling. Together with the
IE fsMeterUnmeasPacketCountTsFirst it allows to evaluate the count
of packets that were not metered by the Metering Process because
of the use of the flow sampling.
Abstract Data Type: dateTimeSeconds
ElementId: TBD2
Status: Proposed
Units: seconds
Peluso, et al. Expires September 15, 2011 [Page 17]
Internet-Draft Flow Selection Techniques March 2011
7.1.3. fsNotMetBytesCount
Description:
This Information Element specifies the count of bytes that were
not metered by the Metering Process because of the use of the flow
state dependent sampling.
Abstract Data Type: unsigned64
Data Type Semantics: deltaCounter
ElementId: TBD3
Status: Proposed
Units: bytes
7.2. Flow Recording Process related (TBD4-TBD11)
Information Elements presented in this section are related to Flow
Selection performed during the Flow Recording Process.
+-------+-----------------------------------+
| ID | Name |
+-------+-----------------------------------+
| TBD4 | fsNotExpPacketDroppedRecPrTsFirst |
+-------+-----------------------------------+
| TBD5 | fsNotExpPacketDroppedRecPrTsLast |
+-------+-----------------------------------+
| TBD6 | fsNotExpByteInDroppedRecPrCount |
+-------+-----------------------------------+
| TBD7 | fsFlowRecDroppedRecPrTsFirst |
+-------+-----------------------------------+
| TBD8 | fsFlowRecDroppedRecPrTsLast |
+-------+-----------------------------------+
| TBD9 | fsFlowRecNotExpRecPrCount |
+-------+-----------------------------------+
| TBD10 | fsPacketNotExpRecPrCount |
+-------+-----------------------------------+
| TBD11 | fsBytesNotExpRecPrCount |
+-------+-----------------------------------+
7.2.1. fsNotExpPacketDroppedRecPrTsFirst
Description:
Peluso, et al. Expires September 15, 2011 [Page 18]
Internet-Draft Flow Selection Techniques March 2011
Specifies the timestamp of the first non-exported packet that was
contained in the flow records eliminated from the Flow Recording
Process because of resource limitations/policies in the Flow
Recording Process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD4
Status: Proposed
Units: seconds
7.2.2. fsNotExpPacketDroppedRecPrTsLast
Description:
Specifies the timestamp of the last non-exported packet that was
contained in the flow records eliminated from the Flow Recording
Process because of resource limitations/policies in the Flow
Recording Process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD5
Status: Proposed
Units: seconds
7.2.3. fsNotExpByteInDroppedRecPrCount
Description:
This Information Element specifies the count of the non-exported
bytes that were contained in the flow records eliminated from the
Flow Recording Process because of resource limitations/policies in
the Flow Recording Process.
Abstract Data Type: unsigned64
Abstract Data Type: deltaCounter
ElementId: TBD6
Status: Proposed
Units: bytes
Peluso, et al. Expires September 15, 2011 [Page 19]
Internet-Draft Flow Selection Techniques March 2011
7.2.4. fsFlowRecDroppedRecPrTsFirst
Description:
Specifies the timestamp of the first flow record elimination event
occurring during the Flow Recording Process. Together with the IE
fsFlowRecDroppedRecPrTsLast it allows to estimate the count of
flow records containing non-exported packets eliminated from the
Flow Recording Process because of resources limitations/policies
in the Flow Recording Process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD7
Status: Proposed
Units: seconds
7.2.5. fsFlowRecDroppedRecPrTsLast
Description:
Specifies the timestamp of the last flow record elimination event
occurring during the Flow Recording Process. Together with the IE
fsFlowRecDroppedRecPrTsFirst it allows to estimate the count of
flow records containing non-exported packets eliminated from the
Flow Recording Process because of resources limitations/policies
in the Flow Recording Process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD8
Status: Proposed
Units: seconds
7.2.6. fsFlowRecNotExpRecPrCount
Description:
This Information Element specifies the count of the flow records
currently existing in the Flow Recording Process containing at
least one non-exported packet.
Abstract Data Type: unsigned32
Peluso, et al. Expires September 15, 2011 [Page 20]
Internet-Draft Flow Selection Techniques March 2011
Data Type Semantics: deltaCounter
ElementId: TBD9
Status: Proposed
Units: flow records
7.2.7. fsPacketNotExpRecPrCount
Description:
This Information Element specifies the count of non-exported
packets contained in flow records of the Flow Recording Process.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
ElementId: TBD10
Status: Proposed
Units: packets
7.2.8. fsBytesNotExpRecPrCount
Description:
This Information Element specifies the count of non-exported bytes
contained in flow records of the Flow Recording Process.
Abstract Data Type: dateTimeSeconds
Data Type Semantics: deltaCounter
ElementId: TBD11
Status: Proposed
Units: bytes
7.3. Flow export process related (TBD12-TBD19)
Information Elements presented in this section are related to Flow
Selection performed during the Flow Export Process.
Peluso, et al. Expires September 15, 2011 [Page 21]
Internet-Draft Flow Selection Techniques March 2011
+-------+---------------------------------------+
| ID | Name |
+-------+---------------------------------------+
| TBD12 | fsPacketNotExpInDroppedFlowRecTsFirst |
+-------+---------------------------------------+
| TBD13 | fsPacketNotExpInDroppedFlowRecTsLast |
+-------+---------------------------------------+
| TBD14 | fsBytesNotExpInDroppedFlowRecCount |
+-------+---------------------------------------+
| TBD15 | fsFlowRecDroppedExpPrTsFirst |
+-------+---------------------------------------+
| TBD16 | fsFlowRecDroppedExpPrTsLast |
+-------+---------------------------------------+
| TBD17 | fsFlowRecNotExpCount |
+-------+---------------------------------------+
| TBD18 | fsPacketNotExpCount |
+-------+---------------------------------------+
| TBD19 | fsBytesNotExpCount |
+-------+---------------------------------------+
7.3.1. fsPacketNotExpInDroppedFlowRecTsFirst
Description:
Specifies the timestamp of the first non-exported packet belonging
to an eliminated flow record. Together with the IE
fsPacketNotExpInDroppedFlowRecTsLast it allows to estimate the
count of non-exported packets that were contained in the flow
records eliminated from the Flow Recording Process because of
resource limitations/policies in the export process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD12
Status: Proposed
Units: seconds
7.3.2. fsPacketNotExpInDroppedFlowRecTsLast
Description:
Specifies the timestamp of the last non-exported packet belonging
to an eliminated flow record. Together with the IE
fsPacketNotExpInDroppedFlowRecTsFirst it allows to estimate the
count of non-exported packets that were contained in the flow
records eliminated from the Flow Recording Process because of
Peluso, et al. Expires September 15, 2011 [Page 22]
Internet-Draft Flow Selection Techniques March 2011
resource limitations/policies in the export process.
Abstract Data Type: unsigned64
ElementId: TBD13
Status: Proposed
Units: bytes
7.3.3. fsBytesNotExpInDroppedFlowRecCount
Description:
This Information Element specifies the count of non-exported bytes
that were contained in the flow records eliminated from the Flow
Recording Process because of resource limitations/policies in the
export process.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
ElementId: TBD14
Status: Proposed
Units: bytes
7.3.4. fsFlowRecDroppedExpPrTsFirst
Description:
Specifies the timestamp of the first flow record elimination event
occurring during the Flow Recording Process. Together with the IE
fsFlowRecDroppedExpPrTsLast it allows to estimate the count of
flow records containing non-exported packets eliminated from the
Flow Recording Process because of resource limitations/policies in
the export process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD15
Status: Proposed
Units: seconds
Peluso, et al. Expires September 15, 2011 [Page 23]
Internet-Draft Flow Selection Techniques March 2011
7.3.5. fsFlowRecDroppedExpPrTsLast
Description:
Specifies the timestamp of the last flow record elimination event
occurring during the Flow Recording Process. Together with the IE
fsFlowRecDroppedExpPrTsFirst it allows to estimate the count of
flow records containing non-exported packets eliminated from the
Flow Recording Process because of resource limitations/policies in
the export process.
Abstract Data Type: dateTimeSeconds
ElementId: TBD16
Status: Proposed
Units: seconds
7.3.6. fsFlowRecNotExpCount
Description:
This Information Element specifies the count of the flow records
currently existing in the Flow Recording Process, containing non-
exported traffic and not being exported because of exporting
process resource limitations/policies.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
ElementId: TBD17
Status: Proposed
Units: flow records
7.3.7. fsPacketNotExpCount
Description:
This Information Element specifies the count of non-exported
packets contained in the flow records of the Flow Recording
Process not being exported because of exporting process resource
limitations/policies.
Abstract Data Type: unsigned32
Peluso, et al. Expires September 15, 2011 [Page 24]
Internet-Draft Flow Selection Techniques March 2011
Data Type Semantics: deltaCounter
ElementId: TBD18
Status: Proposed
Units: packets
7.3.8. fsBytesNotExpCount
Description:
This Information Element specifies the count of non-exported bytes
contained in the flow records of the Flow Recording Process not
being exported because of exporting process resource limitations/
policies.
Abstract Data Type: unsigned64
Data Type Semantics: deltaCounter
ElementId: TBD19
Status: Proposed
Units: bytes
8. Implementation requirements
In order to implement the described information model counters for
non-exported packets and bytes have to be inserted in the flow
records as flow metrics. Sometimes these counters are referred to as
delta counts. An implementation may also keep absolute counts for
purposes not being specified in this information model (both delta
and absolute counters can be exported in the IPFIX information model,
see [RFC5102]). In addition, to fully support this information
model, it would be REQUIRED to keep in a flow record the timestamps
of the first and last non-exported packets. An implementation may
need to keep timestamps of the first and last exported packets for
other purposes than those of this information model, or to join the
two timers for the last exported and first exported packets, or even
to approximate them with the time of the export event.
9. Information Model for Configuration of Flow Selection Techniques
This section aims at describing the representative parameters of the
Peluso, et al. Expires September 15, 2011 [Page 25]
Internet-Draft Flow Selection Techniques March 2011
flow selection techniques presented above. To this regard, it
provides the basis of an information model to be adopted in order to
configure the flow selection process within an IPFIX device. The
information elements herein introduced are defined in accordance with
the IPFIX information model [RFC5102].. Furthermore, the data types
used to represent the Flow Selection-related information elements are
those defined in section 3.1 of the IPFIX information model [RFC
2051].
List of Flow Selection information elements:
+-------+-------------------------+-------+-----------------------+
| ID | Name | ID | Name |
+-------+-------------------------+-------+-----------------------+
| 304 | flowSelectorMethod | TBD21 | flowRecordPacketsSize |
+-------+-------------------------+-------+-----------------------+
| TBD22 | flowMaxAdmitFlowRecords | TBD23 | flowInactivityTime |
+-------+-------------------------+-------+-----------------------+
| TBD24 | flowRecordBytesSize | | |
+-------+-------------------------+-------+-----------------------+
9.1. flowSelectorMethod
Description:
This Information Element identifies the flow selection method that
is applied by the Flow Selection process, in accordance with what
is described in section 5 of this document.
Some of these methods may have parameters in order to fully
support the selected technique. For that reason, further
Information Elements are defined in the following subsections.
Flow selection methods identifiers are herein defined:
+----+-------------------+------------------------------------------+
| ID | Method | Parameters |
+----+-------------------+------------------------------------------+
| 1 | Selection based | flowMaxAdmitFlowRecords |
| | on flow size | flowRecordBytesSize |
| | count | flowRecordPacketsSize |
+----+-------------------+------------------------------------------+
| 2 | Selection based | flowMaxAdmitFlowRecords ipVersion |
| | on flow content | sourceIPv4Address destinationIPv4Address |
| | property match | sourceIPv6Address destinationIPv6Address |
| | | flowDestinationPort |
+----+-------------------+------------------------------------------+
Peluso, et al. Expires September 15, 2011 [Page 26]
Internet-Draft Flow Selection Techniques March 2011
+----+-------------------+------------------------------------------+
| 3 | Selection based | flowMaxAdmitFlowRecords |
| | on flow record | flowInactivityTime |
| | arrival time or | |
| | sequence | |
+----+-------------------+------------------------------------------+
| 4 | Selection based | flowMaxAdmitFlowRecords |
| | on external | |
| | events | |
+----+-------------------+------------------------------------------+
Abstract Data Type: selectorAlgorithm
Data Type Semantics: identifier
ElementId: 304
Status: current
9.2. flowMaxAdmitFlowRecords
Description:
This Information Element specifies the maximum number of eligible
flow records which might be created in the flow cache. It is used
by the Selector Process in order to identify the time when flow
selection should be triggered. A value of 0 means that the Flow
Selection State related to the memory space available for flow
recording MUST be used to estimate the max flow cache size.
For example, this Information Element MAY be used to set the
configuration of a flow size count Flow Selector.
Abstract Data Type: unsigned32
Data Type Semantics: quantity
ElementId: TBD21
Status: Proposed
Units: flow records
9.3. flowRecordBytesSize
Description:
Peluso, et al. Expires September 15, 2011 [Page 27]
Internet-Draft Flow Selection Techniques March 2011
This Information Element specifies the minimum number of bytes
contained in a flow record to be considered as not eligible for
removal. It MAY be used for elephant flows identification.
For example, this Information Element MAY be used to describe the
configuration of a flow size count Flow Selector.
Abstract Data Type: unsigned64
Data Type Semantics: quantity
ElementId: TBD22
Status: Proposed
Units: bytes
9.4. flowRecordPacketsSize
Description:
This Information Element specifies the minimum number of packets
contained in a flow record to be considered as not eligible for
removal. It MAY be used for elephant flows identification.
For example, this Information Element MAY be used to describe the
configuration of a flow size count Flow Selector.
Abstract Data Type: unsigned32
Data Type Semantics: quantity
ElementId: TBD23
Status: Proposed
Units: packets
9.5. flowInactivityTime
Description:
This Information Element specifies the time interval in
microseconds during which the corresponding flow record MAY be
considered as still active. It is used by the metering process
and/or the flow recording process in order to make the decision on
whether to discard an existing flow to make room for a new one.
Peluso, et al. Expires September 15, 2011 [Page 28]
Internet-Draft Flow Selection Techniques March 2011
For example, this Information Element MAY be used to describe the
configuration of a flow arrival time Flow Selector.
Abstract Data Type: dateTimeMicroseconds
Data Type Semantics: quantity
ElementId: TBD24
Status: Proposed
Units: microseconds
9.6. ipVersion
Description:
The IP version field in the IP packet header.
Abstract Data Type: unsigned8
Data Type Semantics: identifier
ElementId: 60
Status: current
9.7. sourceIPv4Address
Description:
The IPv4 source address in the IP packet header.
Abstract Data Type: ipv4Address
Data Type Semantics: identifier
ElementId:8
Status: current
9.8. destinationIPv4Address
Description:
The IPv4 destination address in the IP packet header.
Abstract Data Type: ipv4Address
Peluso, et al. Expires September 15, 2011 [Page 29]
Internet-Draft Flow Selection Techniques March 2011
Data Type Semantics: identifier
ElementId:12
Status: current
9.9. sourceIPv6Address
Description:
The IPv6 source address in the IP packet header.
Abstract Data Type: ipv6Address
Data Type Semantics: identifier
ElementId:27
Status: current
9.10. destinationIPv6Address
Description:
The IPv6 destination address in the IP packet header.
Abstract Data Type: ipv6Address
Data Type Semantics: identifier
ElementId:28
Status: current
10. IANA Considerations
This document introduces several new information elements as an
extension to the IPFIX information model. Values TBD1-TBD19 in this
document should be replaced with the assigned numbers by IANA.
11. Security Considerations
In this section security issues concerning an IPFIX device performing
flow selection are pointed out. In case the flow selection function
is activated an IPFIX device might be exposed to security threats.
Since flow selection implies analysing flow packets, associating them
Peluso, et al. Expires September 15, 2011 [Page 30]
Internet-Draft Flow Selection Techniques March 2011
to a specific traffic flow and selecting flow records, a malicious
user who was able to gain control of an IPFIX device might access
both packet and flow data, thus violating their confidentiality.
Furthermore, the intruder might be attracted by the possibility of
altering the flow selection process by modifying the criteria used to
select flow records. In this case, the IPFIX device would export
flow data which are different from the ones that the Collector
expects to receive.
It is apparent that these security threats can be mitigated by
authenticating entities that interact with the IPFIX device and
keeping information for flow selection configuration confidential.
12. References
12.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
12.2. Informative References
[DuLT01a] Duffield, N., Lund, C., and M. Thorup, "Charging from
Sampled Network Usage", ACM Internet Measurement Workshop
IMW 2001, San Francisco, USA, November 2001.
[DuLT01b] Duffield, N., Lund, C., and M. Thorup, "Properties and
Prediction of Flow Statistics from Sampled Packet
Streams", ACM SIGCOMM Internet Measurement Workshop 2002,
November 2002.
[EsVa01] Estan, C. and G,. Varghese, "New Directions in Traffic
Measurement and Accounting: Focusing on the Elephants,
Ignoring the Mice", ACM SIGCOMM Internet Measurement
Workshop 2001, San Francisco (CA), November 2001.
[IPFIX-MED]
Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi,
"IPFIX Mediation: Framework", Internet
Draft draft-ietf-ipfix-mediators-framework-09,
October 2010.
[KuXW04] Kumar, K., Xu, J., Wang, J., Spatschek, O., and L. Li,
"Space-code bloom filter for efficient per-flow traffic
measurement", INFOCOM 2004 Twenty-third AnnualJoint
Conference of the IEEE Computer and Communications
Peluso, et al. Expires September 15, 2011 [Page 31]
Internet-Draft Flow Selection Techniques March 2011
Societies, March 2004.
[Moli03] Molina, M., "A scalable and efficient methodology for flow
monitoring in the Internet", International Teletraffic
Congress (ITC-18), Berlin, September 2003.
[RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander,
"Requirements for IP Flow Information Export",
RFC3917 Requirements for IP Flow Information Export
(IPFIX), July 2008.
[RFC5101] Claise, B., "Specification of the IP Flow Information
Export (IPFIX) Protocol for the Exchange of IP Traffic
Flow Information", RFC5101 Specification of the IP Flow
Information Export (IPFIX) Protocol for the Exchange of IP
Traffic Flow Information, January 2008.
[RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
Meyer, "Information Model for IP Flow Information Export",
RFC 5102, January 2008.
[RFC5470] Sadasivan, G., Bownlee, N., Claise, B., and J. Quittek,
"Architecture for IP Flow Information Export",
RFC5470 Architecture for IP Flow Information Export,
September 2006.
[RFC5475] Zseby, T., Molina, M., Raspall, F., Duffield, N., and S.
Niccolini, "Sampling and Filtering techniques for IP
Packet Selection", RFC5475 Sampling and Filtering
techniques for IP Packet Selection, July 2008.
[RFC5476] Claise, B., Johnson, A., and B. Claise, "Packet Sampling
(PSAMP) Protocol Specifications", RFC5476 Packet Sampling
(PSAMP) Protocol Specifications (IPFIX), March 2009.
Authors' Addresses
Lorenzo Peluso
University of Napoli
Via Claudio 21
Napoli 80125
Italy
Phone: +39 081 7683821
Email: lorenzo.peluso@unina.it
Peluso, et al. Expires September 15, 2011 [Page 32]
Internet-Draft Flow Selection Techniques March 2011
Tanja Zseby
Fraunhofer Institute FOKUS
Kaiserin-Augusta-Allee 31
Berlin 10589
Germany
Phone: +49 30 3463 7153
Email: tanja.zseby@fokus.fraunhofer.de
Salvatore D'Antonio
CINI Consortium/University of Napoli "Parthenope"
Monte S.Angelo, Via Cinthia
Napoli 80126
Italy
Phone: +39 081 679944
Email: salvatore.dantonio@uniparthenope.it
Christian Henke
Fraunhofer Institute FOKUS
Kaiserin-Augusta-Allee 31
Berlin 10589
Germany
Phone: +49 30 3463 7366
Email: christian.henke@fokus.fraunhofer.de
Peluso, et al. Expires September 15, 2011 [Page 33]
| PAFTECH AB 2003-2026 | 2026-04-23 08:40:47 |