One document matched: draft-ietf-ipfix-flow-selection-tech-00.txt
Internet Engineering Task Force L. Peluso
Internet-Draft University of Napoli
Intended status: Standards Track T. Zseby
Expires: April 21, 2010 Fraunhofer Institute FOKUS
S. D'Antonio
CINI Consortium/University of
Napoli "Parthenope"
M. Molina
DANTE
October 18, 2009
Flow Selection Techniques
draft-ietf-ipfix-flow-selection-tech-00.txt
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79. This document may contain material
from IETF Documents or IETF Contributions published or made publicly
available before November 10, 2008. The person(s) controlling the
copyright in some of this material may not have granted the IETF
Trust the right to allow modifications of such material outside the
IETF Standards Process. Without obtaining an adequate license from
the person(s) controlling the copyright in such materials, this
document may not be modified outside the IETF Standards Process, and
derivative works of it may not be created outside the IETF Standards
Process, except to format it for publication as an RFC or to
translate it into languages other than English.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on April 21, 2010.
Peluso, et al. Expires April 21, 2010 [Page 1]
Internet-Draft Flow Selection Techniques October 2009
Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents in effect on the date of
publication of this document (http://trustee.ietf.org/license-info).
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document.
Abstract
Flow selection is the process in charge of electing a limited number
of flows from all of those observed at an observation point to be
considered into the measurement process chain. The flow selection
process can be enabled at different stages of the monitoring
reference model. It can be performed at metering time once the
packet classification has been executed, i.e. flow state dependent
packet selection, or at recording/exporting time by limiting the
number of flows to be stored and/or exported to the collector
applications. This document illustrates the motivations which might
lead flow selection to be performed and presents a classification of
the related techniques. The document furthermore provides an
information model for configuring flow selection techniques and
discusses what information about the flow selection process is
beneficial to be exported by adopting a suitable information model.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Peluso, et al. Expires April 21, 2010 [Page 2]
Internet-Draft Flow Selection Techniques October 2009
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5
4. Position of the Flow Selection Process . . . . . . . . . . . . 6
5. Flow selection techniques . . . . . . . . . . . . . . . . . . 8
5.1. Flow selection based on flow record content . . . . . . . 8
5.2. Flow selection based on flow record arrival time or
sequence . . . . . . . . . . . . . . . . . . . . . . . . . 9
5.3. Flow selection on external events . . . . . . . . . . . . 9
6. Reporting of Flow Selection Information . . . . . . . . . . . 9
6.1. Flow selection in the metering process . . . . . . . . . . 9
6.2. Flow selection in the flow recording process . . . . . . . 10
6.3. Flow selection in the exporting process . . . . . . . . . 11
7. Information model for flow selection information exporting . . 12
7.1. Meter process related (TBD1-TBD2) . . . . . . . . . . . . 13
7.1.1. FsMeter_UnmeasPacketCount . . . . . . . . . . . . . . 14
7.1.2. FsMeter_UnmeasBytesCount . . . . . . . . . . . . . . . 14
7.2. Flow recording process related (TBD3-TBD8) . . . . . . . . 14
7.2.1. FsFrec_PacketInDroppedRecsCount . . . . . . . . . . . 15
7.2.2. FsFrec_ByteInDroppedRecsCount . . . . . . . . . . . . 15
7.2.3. FsFrec_FrecDroppedCount . . . . . . . . . . . . . . . 15
7.2.4. FsFrec_UnexportedFrecCount . . . . . . . . . . . . . . 16
7.2.5. FsFrec_UnexportedPacketInFrecCount . . . . . . . . . . 16
7.2.6. FsFrec_UnexportedBytesInFrecCount . . . . . . . . . . 16
7.3. Flow exporting process related (TBD9-TBD14) . . . . . . . 17
7.3.1. FsExp_PacketInDroppedRecsCount . . . . . . . . . . . . 17
7.3.2. FsExp_ByteInDroppedRecsCount . . . . . . . . . . . . . 17
7.3.3. FsExp_FrecDroppedCount . . . . . . . . . . . . . . . . 18
7.3.4. FsExp_UnexportedCount . . . . . . . . . . . . . . . . 18
7.3.5. FsExp_UnexportedPacketCount . . . . . . . . . . . . . 18
7.3.6. FsExp_UnexportedByteInExpCount . . . . . . . . . . . . 19
8. Requirements put on implementations . . . . . . . . . . . . . 19
9. Information Model for Configuration of Flow Selection
Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . 20
9.1. selectorMethod . . . . . . . . . . . . . . . . . . . . . . 20
9.2. flowMaxAdmitFlowRecords . . . . . . . . . . . . . . . . . 21
9.3. flowRecordBytesSize . . . . . . . . . . . . . . . . . . . 22
9.4. flowRecordPacketsSize . . . . . . . . . . . . . . . . . . 22
9.5. flowInactivityTime . . . . . . . . . . . . . . . . . . . . 22
10. Security Considerations . . . . . . . . . . . . . . . . . . . 23
11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 23
12.1. Normative References . . . . . . . . . . . . . . . . . . . 23
12.2. Informative References . . . . . . . . . . . . . . . . . . 24
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25
Peluso, et al. Expires April 21, 2010 [Page 3]
Internet-Draft Flow Selection Techniques October 2009
1. Introduction
This document describes flow selection techniques for traffic
measurements. As stated in [RFC5475], packet selection is in charge
of electing a representative subset of packets that allow accurate
estimates of properties of the unsampled traffic to be formed. Its
main application consists in performing some forms of data reduction
on observed Internet traffic in order to limit the processing
overhead at measurement devices. Despite its proven ability in
achieving this objective, the mechanism responsible for steering the
selection process is generally driven by a packet-based decision
strategy. It means that, the element on which this selection
mechanism is performed is a packet and the decision of which packets
are suitable to be sampled depends on packets only. As a
consequence, depending on the specific adopted selection strategy,
packet selection may not take in consideration potential effects of
its actions on subsequent measurement tasks, such as flow recording
and exporting processes, which are instead considering flows rather
than packets. In this perspective, flow selection differs from
packet selection in that the basis elements on which the selection
process is applied is not a packet but a flow. In the IPFIX
architecture the object of the selection process would be the so-
colled flow records. For several applications it makes sense to
select only the flows of interest id resources are scarce. Examples
are accounting or attack detection applications. It has been
observed that the distribution of the number of packets per flow or
the number of bytes per flow are heavy-tailed. That means, most
flows consist only of a small number of packets and only few flows
have a large number of packets. The few large flows contribute to
the majority of the overall traffic volume [DuLT01a], [DuLT01b].
This observation on the flow size distributions in Internet traffic
is also referred to as "Quasi-Zipf-Law" [KuXW04] or as "elephant and
mice phenomenon". The large flows are referred to as elephant flows
or heavy hitters. Obviously, distributions characterizing the flow
size strictly depends on the flow definition in use and can change
with regard to the profile of future applications.
2. Scope
This document describes flow selection techniques and its parameters.
It adresses the configuration of flow selection techniques and
defines which information should be reported by decives that perform
flow selection. It only describes processes directly acting on
traffic flows during the metering phase and/or the exporting phase.
Therefore it is assumed that flow selection is performed after
packest are classified into flows. This document does not address
the flow selection effects that might result from the sampling or
Peluso, et al. Expires April 21, 2010 [Page 4]
Internet-Draft Flow Selection Techniques October 2009
filtering of packets in the metering process before the
classification process is performed. Packet selection techniques are
described in [RFC5475] and, therefore, outside the scope of this
document.
3. Terminology
This document uses the terminology introduced in [RFC5470] and
[RFC5475] In this section, some additional terms are presented which
extend the terminology introduced in [RFC5475].
* Flow Selection Process
A Flow Selection Process takes a set of Flow Records as its input
and selects a subset of that set as its output.
* Flow Selection State
A Flow Selection Process may maintain state information for use by
the Flow Selection Process. At a given time, the Flow Selection
State may depend on flows observed at and before that time, and
other variables. Examples include:
(i) number of accounted flow records;
(ii) memory space available for flow recording;
(iii) state of the pseudorandom number generators;
(iv) hash values calculated during selection.
* Flow Selector
A Flow Selector defines the action of a Flow Selection Process on
a single flow of its input. The Flow Selector can make use of the
following information in determining whether a flow is selected:
(i) the content of the flow record;
(ii) any information state related to the flow recording;
(iii) any selection state that may be maintained by the Flow
Selection Process.
Peluso, et al. Expires April 21, 2010 [Page 5]
Internet-Draft Flow Selection Techniques October 2009
4. Position of the Flow Selection Process
Figure 1 shows the IPFIX reference model as defined in [RFC5470], and
extends it by introducing the functional components where flow
selection can take place. Traffic flows can be selected at different
stages of the measurement chain. The first possibility is to perform
flow selection by analysing flow packets during the metering process.
The second option is to act directly on the traffic flows during the
flow recording process and/or the flow exporting process.
Peluso, et al. Expires April 21, 2010 [Page 6]
Internet-Draft Flow Selection Techniques October 2009
Packet(s) coming in to Observation Point(s)
| |
v v
+----------------+---------------------------+ +-----+-------+
| Metering Process on an | | |
| Observation Point | | |
| | | |
| packet header capturing | | |
| | |...| Metering |
| timestamping | | Process N |
| | | | |
| packet selection | | |
| | | | |
| classification | | |
| | | | |
| flow state dependent packet sampling (*) | | |
| | | | |
| aggregation | | |
| | | | |
| flow recording (*) | | |
| | | | |
| | Timing out Flows | | |
| | Handle resource overloads | | |
+--------|-----------------------------------+ +-----|-------+
| |
Flow Records (selected by Observation Domain) Flow Records
| |
+----------------------+----------------------+
|
+----------------------|---------------+
| Exporting Process v |
| +---------------+-----------+ |
| | flow export (*) | |
| +---------------+-----------+ |
| | |
+----------------------+---------------+
|
v
IPFIX export packet to Collector
(*) indicates where flow selection can take place.
Figure 1
In case the flow selection is performed during the metering process,
then it consists in accounting only a subset of all the incoming
packets which are collected at the observation point. However,
unlike the selection process executed before the packet
Peluso, et al. Expires April 21, 2010 [Page 7]
Internet-Draft Flow Selection Techniques October 2009
classification is performed, the flow selection applies to only the
incoming packets which somehow satisfy certain conditions regarding
flows state information available from the flow recording process.
This kind of selection is considered as a packet sampling technique,
in accordance with [RFC5475] where such technique is referred to as
flow state dependent sampling. The state of the stored flow records
is thus taken into account while performing packet selection, so that
the process responsible for generating or updating flow records might
be influenced by the selective collection of packets which feed it.
Under this perspective, unlike the flow selection performed at the
flow recording and exporting processes, this flow selection technique
operates at a very early stage of the flow monitoring process, as it
acts at packet level. The adoption of such technique allows to
prevent that some observed/observable packets might enforce the flow
recording process to account, for instance, not representative or not
expected flow records. The flow selection that might be provided is
executed during the flow recording and/or exporting processes, it is
done at flow level, once packets are classified and assigned to the
correspondent flows. More exactly, the flow selection process can be
carried out by storing new flow records only in those cases whem
enough resources are available at the monitoring device or by
discarding already accounted records which, under certain
circumstances and at a certain point in time, are not anymore
significant. Finally, at the flow exporting time it might be
required that not all of the stored flow records are actually
exported to the collectors.
5. Flow selection techniques
We can distinguish the following selection techniques:
1. based on flow record content (i.e. all reported flow
characteristics);
2. based on flow record arrival time;
3. based on external events like the exhaustion of local resources.
5.1. Flow selection based on flow record content
Flow selection can be done based on fields in an IPFIX flow record.
This can be done analogous to field match filtering for packet
selection described in [RFC5475]. The difference here is that
instead of packets here field of the flow record content are used for
the selection decision. An example would be to select flow records
with regard to the flow size in bytes or number of packets. Another
application would be to select flow records based on flow start time.
Peluso, et al. Expires April 21, 2010 [Page 8]
Internet-Draft Flow Selection Techniques October 2009
5.2. Flow selection based on flow record arrival time or sequence
Flow records can be selected based on their arrival time at the
collector. One example would be to select a number of flow records
for certain periods of time. Another option is to select flow
records based on the order at which they arrive at the collector.
With this one can select systematically every kth record or select
randomly a set of flow records.
5.3. Flow selection on external events
The selection of flow records can be alsio triggered by external
events. An example would be router state like number of entries in
flow cache.
6. Reporting of Flow Selection Information
In this section we identify and describe in more detail some possible
causes of flow selection, along with the information that can be
beneficial to make available to applications about it.
6.1. Flow selection in the metering process
The main reason for applying in the metering process a flow state
dependent sampling is that the flow recording process may not have,
at a certain point in time, enough positions to record all observable
flows. Another reason may be that there may not be enough processing
resources to create and manage a new flow record. To overcome with
these limitations, a number of possible policies can be applied, the
simplest one being not to consider for measurement the new packets
that do not belong to already existing flow records (i.e. that would
require the creation of a new one). More refined policies are
however possible, mainly aimed at the so called elephant flow
detection, i.e. to give priority in the flow recording process to
flows carrying more traffic. For instance, [EsVa01] proposes
criteria to define a packet eligible to create a new flow record
(sample and hold, multistage filters). Independently of the specific
algorithms, we are concerned here about defining what information it
makes sense to keep about the flow state dependent packet sampling
and make available to applications (by exporting it out of an IPFIX
device). It is certainly possible to keep a cumulative counter of
the total number of packets and bytes that were not considered for
measurement because of flow state dependent sampling. Also, it is
possible to keep a timestamp for the first and last of these non
measured packets. This means, in practice, to aggregate all these
packets in a macro flow, and keep track of its volume and duration.
Imagining keeping more detailed information about packets not
Peluso, et al. Expires April 21, 2010 [Page 9]
Internet-Draft Flow Selection Techniques October 2009
measured because of flow state dependent sampling would contradict
the fact that the sampling is done because of lack of memory and/or
processing resources.
6.2. Flow selection in the flow recording process
This block is optional in the IPFIX framework architecture. However,
we address here the case where it is present. We already described
in the previous section that because of lack of memory positions in
the flow recording process some incoming packets may be discarded if
they lead to the opening of a new flow record. However, under
certain circumstances, it may be advantageous to discard an existing
flow record in the flow recording process to make room for the new
record opened by an arriving packet. For example, an algorithm for
taking the decision whether to discard the new arriving packet or an
existing flow record is described in [Moli03]. In this section we
are not concerned about the algorithm details but about what
information to store about this record removal. For the same reasons
expressed before, we argue that it does not make sense to store
separate information for each discarded flow record, as it would
contradict the motivation itself for which the discarding is done
(i.e. lack of memory resources). The information that is certainly
possible to keep with a limited effort is a cumulative counter of the
total number of not yet exported packets and bytes belonging to flow
records that were eliminated from the flow recording process.
Ideally, we would also like to keep a timestamp for the first (T_fd)
and last (T_ld) not yet exported packets belonging to all these
discarded flow records. This would mean, in practice, to aggregate
all these packets in a macro flow, and keep track of its volume and
duration. To do so precisely, we would need to keep in each flow
record a timestamp for the first and last non-exported packets, and
whenever a record is discarded look at these timestamps to see if
they are smaller or larger (respectively) of T_fd and T_ld and if yes
update them. Another information that can be easily kept is the
number of these discarding events, along with a timestamp of the
first and last of them. This information should not be used by
applications to re- normalize their received per flow statistics
(because a flow may be discarded and re-created multiple times) but
rather to keep under control the good functioning of the implemented
policy. Note that we consider a discarding event only when the
discarded flow record contains some not exported traffic. Otherwise,
the removal of a record whose traffic was fully exported (after a
timeout or after the arrival of specific packets, e.g. TCP FIN or
RST) is part of the normal functioning of an IPFIX flow metering
system. Note also that we consider only the case when an elimination
of a flow record from the flow recording process leads to the
complete loss of all the information contained in the flow record.
If on the contrary another policy is implemented, like immediate
Peluso, et al. Expires April 21, 2010 [Page 10]
Internet-Draft Flow Selection Techniques October 2009
exporting of the flow record before elimination, or freezing of the
flow record and moving it in an area of memory different from which
is considered the flow recording process for later exporting, this is
not considered an elimination and therefore is out of the scope of
this document. In parallel to the information about the number of
discarded flow records and associated packets and bytes, it is useful
to keep cumulative information about the number of flow records
containing not yet exported traffic that exist in the flow recording
process, along with the cumulative number of not exported packets and
bytes contained in them. This information is useful also for
exporting process related reasons, as clarified in the following
paragraph.
6.3. Flow selection in the exporting process
The exporting process may implement policies for not exporting the
whole set of flow records of the flow recording process. In case of
absence of the flow recording process, when the metering process
directly feeds the exporting process (i.e. directly put the exported
packets in the IPFIX format), the following reasoning does not apply.
The motivations for not exporting some flow records (containing non
exported traffic) can be two: there are explicit configured policies
or the exporting process faces resource limitation. An example of
explicit policy can be not to export the flows whose accounted
traffic is below a certain threshold, or a more complex mechanism
such as the one described in [DuLT01a] or [DuLT01b]. An example of
resource limitation is that the exporting process has an assigned,
limited time slot to operate or a limited predefined number of export
packets that it can send. There can also be hybrid cases where there
are resource limitations and policies are applied in order to
optimize the exported information (e.g. given that we want to export
only N flow records, select a subset so that the overall number of
reported packets and bytes belonging to the subset is maximized).
Coming to the issue of which information it makes sense to keep about
this flow selection, there are two cases to consider. If a flow is
not exported and because of this decision is deleted from the flow
recording process, we are in the same case described before (where
the deletion was triggered by the need to make room for another
record). The information to keep is then naturally the same as
described before (cumulative packets and bytes for all the flows not
exported, timestamps of the first and last packets belonging to non
exported flow records, counter of dropping events and timestamp of
first and last dropping event). Only the reason for this removal is
different. If on the contrary a record eligible for exporting is not
exported but it remains in the flow recording process it has always a
chance to be exported in the future. For an application, however, it
would be beneficial to know what it is not currently being exported
because of exporting process policies/resource limitations, in terms
Peluso, et al. Expires April 21, 2010 [Page 11]
Internet-Draft Flow Selection Techniques October 2009
of flow records, packets and bytes. This, not to re-normalize its
estimates (it would be dangerous and error prone because the
exporting of these records may be simply delayed), but rather to keep
under control what is happening: for example, understand if there are
pathologic situations where a large number of flow records and/or
associated traffic are never exported, or if the number of flow
records in the flow recording process is growing, etc. When it comes
to understanding if this information can be easily available,
however, we recognize that there is the problem that in order to be
aware that it has not exported a flow record, an exporting process
should at least have browsed through it. In other words, we would
have to assume that there is always a full scanning of the flow
recording process associated to the exporting process selection
decision. However, there may be more efficient implementations where
this does not happen. Therefore, even if we provide support in the
information model for this information, defining it as mandatory in
the protocol definition would put a constraint on the exporting
process implementation, which is undesirable.
7. Information model for flow selection information exporting
We formally define the elements to contain the information described
in the previous section. Some elements have an associated couple of
timestamps, which we reference for brevity (when it is not ambiguous)
as Tfirst and Tlast (instead of element_nameTfirst,
element_nameTlast). Note that all the following information elements
are aimed at describing macro flows (e.g. the total number of packets
and bytes contained in all dropped or not created flow records).
Some of these macro flows are additive only, in the sense that they
only add contributions to them, but never subtract. E.g. the macro
flow of the packets contained in flow records that are discarded from
the flow reporting process receives a contribution when a flow record
is discarded, and this contribution can never be subtracted. On the
contrary, some of the macro flows can dynamically receive and loose
contributions. E.g. the macro flows of packets not yet exported
receives a contribution when a new packets arrives, and looses some
contribution when there is an exporting event. Associating a
timestamp for the oldest and most recent contributions to additive
only flow is easy, while for the others is not (would require to
maintain full state) and that is why we did not define timestamps for
these information elements.
The information elements here introduced are defined in accordance
with the IPFIX information model [RFC5102] to which reference should
be made for more detailed information. Furthermore, the data types
used to formally rappresent the Flow Selection related information
elements are those defined in section 3.1 of the IPFIX information
Peluso, et al. Expires April 21, 2010 [Page 12]
Internet-Draft Flow Selection Techniques October 2009
model [RFC 2051]. For that reason, they are not redefined in this
section.
List of additional Flow Selection information elements:
+-------+------------------------------------+
| ID | Name |
+-------+------------------------------------+
| TBD1 | FsMeter_UnmeasPacketCount |
+-------+------------------------------------+
| TBD2 | FsMeter_UnmeasBytesCount |
+-------+------------------------------------+
| TBD3 | FsFrec_PacketInDroppedRecsCount |
+-------+------------------------------------+
| TBD4 | FsFrec_ByteInDroppedRecsCount |
+-------+------------------------------------+
| TBD5 | FsFrec_FrecDroppedCount |
+-------+------------------------------------+
| TBD6 | FsFrec_UnexportedFrecCount |
+-------+------------------------------------+
| TBD7 | FsFrec_UnexportedPacketInFrecCount |
+-------+------------------------------------+
| TBD8 | FsRec_UnexportedBytesInFrecCount |
+-------+------------------------------------+
| TBD9 | FsExp_PacketInDroppedRecsCount |
+-------+------------------------------------+
| TBD10 | FsExp_BytesInDroppedRecsCount |
+-------+------------------------------------+
| TBD11 | FsExp_FrecDroppedCount |
+-------+------------------------------------+
| TBD12 | FsExp_UnexportedCount |
+-------+------------------------------------+
| TBD13 | FsExp_UnexportedPacketCount |
+-------+------------------------------------+
| TBD14 | FsExp_UnexportedByteInExpCount |
+-------+------------------------------------+
7.1. Meter process related (TBD1-TBD2)
Information Elements in this section are related to Flow Selection at
the Matering Process.
+------+---------------------------+
| ID | Name |
+------+---------------------------+
| TBD1 | FsMeter_UnmeasPacketCount |
+------+---------------------------+
Peluso, et al. Expires April 21, 2010 [Page 13]
Internet-Draft Flow Selection Techniques October 2009
+------+---------------------------+
| TBD2 | FsMeter_UnmeasBytesCount |
+------+---------------------------+
7.1.1. FsMeter_UnmeasPacketCount
Contains the count of packets that were not measured because of flow
state dependent sampling, in terms of:
TsFirst: timestamp of the first packet not measured because of flow
state dependent sampling (Type: dateTime)
TsLast: timestamp of the last packet not measured because of flow
state dependent sampling (Type: dataTime)
7.1.2. FsMeter_UnmeasBytesCount
Description:
This Information Elements contains the count of bytes that were
not measured because of flow state dependent sampling
Abstract Data Type: unsigned64
Data Type Semantics: quantity
ElementId: TBD2
Status: Proposed
Units: bytes
7.2. Flow recording process related (TBD3-TBD8)
Information Elements in this section are related to Flow Selection at
the Flow Recording Process if present.
+------+------------------------------------+
| ID | Name |
+------+------------------------------------+
| TBD3 | FsFrec_PacketInDroppedRecsCount |
+------+------------------------------------+
| TBD4 | FsFrec_ByteInDroppedRecsCount |
+------+------------------------------------+
| TBD5 | FsFrec_FrecDroppedCount |
+------+------------------------------------+
| TBD6 | FsFrec_UnexportedFrecCount |
+------+------------------------------------+
Peluso, et al. Expires April 21, 2010 [Page 14]
Internet-Draft Flow Selection Techniques October 2009
+------+------------------------------------+
| TBD7 | FsFrec_UnexportedPacketInFrecCount |
+------+------------------------------------+
| TBD8 | FsFrec_UnexportedBytesInFrecCount |
+------+------------------------------------+
7.2.1. FsFrec_PacketInDroppedRecsCount
Contains the count of non exported packets that were contained in
flow records eliminated from the flow recording process because of
resource limitations/policies in the flow recording process. It is
defined in terms of:
TsFirst: timestamp of the first non-exported packet belonging to a
eliminated flow record (Type: dateTime)
TsLast: timestamp of the last non-exported packet belonging to a
eliminated flow record (Type: dateTime)
7.2.2. FsFrec_ByteInDroppedRecsCount
Description:
This Information Elements contains the count of non exported bytes
that were contained in flow records eliminated from the flow
recording process because of resource limitations/policies in the
flow recording process.
Abstract Data Type: unsigned64
Data Type Semantics: quantity
ElementId: TBD4
Status: Proposed
Units: bytes
7.2.3. FsFrec_FrecDroppedCount
Contains the count of flow records containing non exported packets
eliminated from the flow recording process because of resources
limitations/policies in the flow recording process. It is defined in
terms of:
TsFirst: timestamp of the first flow record elimination event from
the flow recording process (Type: dateTime)
Peluso, et al. Expires April 21, 2010 [Page 15]
Internet-Draft Flow Selection Techniques October 2009
TsLast: timestamp of the last flow record elimination event from the
flow recording process (Type: dateTime)
7.2.4. FsFrec_UnexportedFrecCount
Description:
This Information Elements contains the count of the flow records
currently existing in the flow recording process containing at
least one non exported packet.
Abstract Data Type: unsigned32
Data Type Semantics: quantity
ElementId: TBD6
Status: Proposed
Units: flow records
7.2.5. FsFrec_UnexportedPacketInFrecCount
Description:
This Information Elements contains the count of non exported
packets contained in flow records of the flow recording process.
Abstract Data Type: unsigned32
Data Type Semantics: quantity
ElementId: TBD7
Status: Proposed
Units: packets
7.2.6. FsFrec_UnexportedBytesInFrecCount
Description:
This Information Elements contains the count of non exported bytes
contained in flow records of the flow recording process.
Abstract Data Type: unsigned64
Data Type Semantics: quantity
Peluso, et al. Expires April 21, 2010 [Page 16]
Internet-Draft Flow Selection Techniques October 2009
ElementId: TBD8
Status: Proposed
Units: bytes
7.3. Flow exporting process related (TBD9-TBD14)
Information Elements in this section are related to Flow Selection at
the Flow Exporting Process.
+-------+--------------------------------+
| ID | Name |
+-------+--------------------------------+
| TBD9 | FsExp_PacketInDroppedRecsCount |
+-------+--------------------------------+
| TBD10 | FsExp_ByteInDroppedRecsCount |
+-------+--------------------------------+
| TBD11 | FsExp_FrecDroppedCount |
+-------+--------------------------------+
| TBD12 | FsExp_UnexportedCount |
+-------+--------------------------------+
| TBD13 | FsExp_UnexportedPacketCount |
+-------+--------------------------------+
| TBD14 | FsExp_UnexportedByteInExpCount |
+-------+--------------------------------+
7.3.1. FsExp_PacketInDroppedRecsCount
Contains the count of non exported packets that were contained in
flow records eliminated from the flow recording process because of
resource limitations/policies in the exporting process. It is
defined in terms of:
TsFirst: timestamp of the first non exported packet belonging to a
eliminated flow record (Type: dateTime)
TsLast: timestamp of the last non exported packet belonging to a
eliminated flow record (Type: dateTime)
7.3.2. FsExp_ByteInDroppedRecsCount
Description:
This Information Elements contains the count of non exported bytes
that were contained in flow records eliminated from the flow
recording process because of resource limitations/policies in the
exporting process.
Peluso, et al. Expires April 21, 2010 [Page 17]
Internet-Draft Flow Selection Techniques October 2009
Abstract Data Type: unsigned64
Data Type Semantics: quantity
ElementId: TBD10
Status: Proposed
Units: bytes
7.3.3. FsExp_FrecDroppedCount
Contains the count of flow records containing non exported packets
eliminated from the flow recording process because of resource
limitations/policies in the exporting process. It is defined in
terms of:
TsFirst: timestamp of the first flow record elimination event from
the flow recording process (Type: dateTime)
TsLast: timestamp of the last flow record elimination event from the
flow recording process (Type: dateTime)
7.3.4. FsExp_UnexportedCount
Description:
This Information Elements contains the count of the flow records
currently existing in the flow recording process containing non-
exported traffic and not being exported because of exporting
process resource lmitations/policies.
Abstract Data Type: unsigned32
Data Type Semantics: quantity
ElementId: TBD12
Status: Proposed
Units: flow records
7.3.5. FsExp_UnexportedPacketCount
Description:
This Information Elements contains the count of non exported
packets contained in flow records of the flow recording process
Peluso, et al. Expires April 21, 2010 [Page 18]
Internet-Draft Flow Selection Techniques October 2009
not being exported because of exporting process resource
limitations/policies.
Abstract Data Type: unsigned32
Data Type Semantics: quantity
ElementId: TBD13
Status: Proposed
Units: packets
7.3.6. FsExp_UnexportedByteInExpCount
Description:
This Information Elements contains the count of non exported bytes
contained in flow records of the flow recording process not being
exported because of exporting process resource limitations/
policies.
Abstract Data Type: unsigned64
Data Type Semantics: quantity
ElementId: TBD14
Status: Proposed
Units: bytes
8. Requirements put on implementations
To support the described information model an implementation must
keep, in the flow records, counts for non-exported packets and bytes.
Sometimes these are referred as delta counts. An implementation may
also keep absolute counts for scopes not specified in this
information model (it appears that both delta and absolute counters
can be exported in the IPFIX information model, see [RFC5102]). In
addition, to fully support this information model, it would be
required to keep in a flow record a timestamp for the first and last
non-exported packets. An implementation may need to keep timestamps
for the first and last exported packets as well for scopes not
specified in this information model, or to join the two timers for
the last exported and first exported packets (which is of course an
approximation) or to approximate them with the time of the exporting
Peluso, et al. Expires April 21, 2010 [Page 19]
Internet-Draft Flow Selection Techniques October 2009
event.
9. Information Model for Configuration of Flow Selection Techniques
This section aims at describing the representative parameters of the
above presented flow selection techniques. To this regard, it
provides the basis for an information model to adopt in order to
configure the flow selection process at an IPFIX device. The
information elements here introduced are defined in accordance with
the IPFIX information model [RFC5102] to which reference should be
made for more detailed information. Furthermore, the data types used
to formally rappresent the Flow Selection related information
elements are those defined in section 3.1 of the IPFIX information
model [RFC 2051]. For that reason, they are not redefined in this
section.
List of additional Flow Selection information elements:
+-------+-------------------------+-------+-----------------------+
| ID | Name | ID | Name |
+-------+-------------------------+-------+-----------------------+
| TBD15 | selectorMethod | TBD18 | flowRecordPacketsSize |
+-------+-------------------------+-------+-----------------------+
| TBD16 | flowMaxAdmitFlowRecords | TBD19 | flowInactivityTime |
+-------+-------------------------+-------+-----------------------+
| TBD17 | flowRecordBytesSize | ... | ... |
+-------+-------------------------+-------+-----------------------+
9.1. selectorMethod
Description:
This Information Element identifies the flow selection method that
are applied by the Flow Selection process, in accordance to what
described in the section 5 of this document.
Same of these methods may have parameters in order to fully
support the selected technique. For that reason, further
Information Elements are defined in the following subsections.
The following flow selection methods identifiers are defined here:
Peluso, et al. Expires April 21, 2010 [Page 20]
Internet-Draft Flow Selection Techniques October 2009
+----+----------------------------+---------------------------------+
| ID | Method | Parameters |
+----+----------------------------+---------------------------------+
| 1 | Selection based on flow | flowMaxAdmitFlowRecords |
| | size count | flowRecordBytesSize |
| | | flowRecordPacketsSize |
+----+----------------------------+---------------------------------+
| 2 | Selection based on flow | flowMaxAdmitFlowRecords |
| | content property match | ........... |
+----+----------------------------+---------------------------------+
| 3 | Selection based on flow | flowMaxAdmitFlowRecords |
| | record arrival time or | flowInactivityTime |
| | sequence | |
+----+----------------------------+---------------------------------+
| 4 | Selection based on | flowMaxAdmitFlowRecords |
| | external events | ........... |
+----+----------------------------+---------------------------------+
Abstract Data Type: unsigned16
Data Type Semantics: identifier
ElementId: TBD15
Status: Proposed
9.2. flowMaxAdmitFlowRecords
Description:
This Information Element specifies the maximum number of elegible
flow records which might be created in to the flow cache. It is
used by the Selector Process in order to identify the time when
flow selection should be triggered. A value of 0 means that the
Flow Selection State related to the memory space available for
flow recording must be used to estimate the max flow cache size.
For example, this Information Element may be used to describe the
configuration of a flow size count Flow Selector.
Abstract Data Type: unsigned32
Data Type Semantics: quantity
ElementId: TBD16
Status: Proposed
Peluso, et al. Expires April 21, 2010 [Page 21]
Internet-Draft Flow Selection Techniques October 2009
Units: flow records
9.3. flowRecordBytesSize
Description:
This Information Element specifies the minimum number of bytes
contained in a flow record to be considered not elegible for
removal. It may be used in order to identify elephant flows.
For example, this Information Element may be used to describe the
configuration of a flow size count Flow Selector.
Abstract Data Type: unsigned64
Data Type Semantics: quantity
ElementId: TBD17
Status: Proposed
Units: bytes
9.4. flowRecordPacketsSize
Description:
This Information Element specifies the minimum number of packets
contained in a flow record to be considered not elegible for
removal. It may be used in order to identify elephant flows.
For example, this Information Element may be used to describe the
configuration of a flow size count Flow Selector.
Abstract Data Type: unsigned32
Data Type Semantics: quantity
ElementId: TBD18
Status: Proposed
Units: packets
9.5. flowInactivityTime
Description:
Peluso, et al. Expires April 21, 2010 [Page 22]
Internet-Draft Flow Selection Techniques October 2009
This Information Element specifies the time interval in
microseconds during which the corresponding flow record may be
considered still active. It is used by the metering process
and/or the flow recording process in order to take the decision
whether to discard an existing flow to make room for a new one.
For example, this Information Element may be used to describe the
configuration of a flow arrival time Flow Selector.
Abstract Data Type: dateTimeMicroseconds
Data Type Semantics: quantity
ElementId: TBD19
Status: Proposed
Units: microseconds
10. Security Considerations
This document descirbes methods for flow selection techniques that
are applied in network measurements. If users know or can guess the
selection policies they may craft flows in a way to avoid beeing
selected. Furthermore network measurements are often used for the
detecction of network attacks. Therefore it has to be taken into
account that flow selection may remove flows that are of interest for
the detection taks. [more here]
11. IANA Considerations
This document introduces several new information elements as an
extension to the IPFIX information model. IANA assignments should be
created for the information elements described in this document.
12. References
12.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
Peluso, et al. Expires April 21, 2010 [Page 23]
Internet-Draft Flow Selection Techniques October 2009
12.2. Informative References
[DuLT01a] Duffield, N., Lund, C., and M. Thorup, "Charging from
Sampled Network Usage", ACM Internet Measurement Workshop
IMW 2001, San Francisco, USA, November 2001.
[DuLT01b] Duffield, N., Lund, C., and M. Thorup, "Properties and
Prediction of Flow Statistics from Sampled Packet
Streams", ACM SIGCOMM Internet Measurement Workshop 2002,
November 2002.
[DuLT01c] Duffield, N., Lund, C., and M. Thorup, "Learn More, sample
less: control of volume and variance in network
measurement", IEEE Transactions on Information Theory,
May 2005.
[DuLT01d] Duffield, N., Lund, C., and M. Thorup, "Flow Sampling
under Hard Resource Constraints", ACM IFIP Conference on
Measurement and Modeling of Computer Systems SIGMETRICS,
June 2004.
[EsVa01] Estan, C. and G,. Varghese, "New Directions in Traffic
Measurement and Accounting: Focusing on the Elephants,
Ignoring the Mice", ACM SIGCOMM Internet Measurement
Workshop 2001, San Francisco (CA), November 2001.
[FeGL98] Feldmann, A., Rexford, J., and R. Caceres, "Efficient
Policies for Carrying Web Traffic over Flow-Switched
Networks", IEEE/ACM Transaction on Networking,
December 1998.
[KuXW04] Kumar, K., Xu, J., Wang, J., Spatschek, O., and L. Li,
"Space-code bloom filter for efficient per-flow traffic
measurement", INFOCOM 2004 Twenty-third AnnualJoint
Conference of the IEEE Computer and Communications
Societies, March 2004.
[Moli03] Molina, M., "A scalable and efficient methodology for flow
monitoring in the Internet", International Teletraffic
Congress (ITC-18), Berlin, September 2003.
[RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
Meyer, "Information Model for IP Flow Information Export",
RFC 5102, January 2008.
[RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
"Architecture for IP Flow Information Export", RFC 5470,
March 2009.
Peluso, et al. Expires April 21, 2010 [Page 24]
Internet-Draft Flow Selection Techniques October 2009
[RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
Raspall, "Sampling and Filtering Techniques for IP Packet
Selection", RFC 5475, March 2009.
Authors' Addresses
Lorenzo Peluso
University of Napoli
Via Claudio 21
Napoli 80125
Italy
Phone: +39 081 7683821
Email: lorenzo.peluso@unina.it
Tanja Zseby
Fraunhofer Institute FOKUS
Kaiserin-Augusta-Allee 31
Berlin 10589
Germany
Phone: +49 30 3463 7153
Email: tanja.zseby@fokus.fraunhofer.de
Salvatore D'Antonio
CINI Consortium/University of Napoli "Parthenope"
Monte S.Angelo, Via Cinthia
Napoli 80126
Italy
Phone: +39 081 679944
Email: saldanto@unina.it
Maurizio Molina
DANTE
Hills Road 126-130
Cambridge CB2 1PQ
United Kingdom
Phone: +44 1223 371300
Email: maurizio.molina@dante.org.uk
Peluso, et al. Expires April 21, 2010 [Page 25]
| PAFTECH AB 2003-2026 | 2026-04-23 08:48:46 |