One document matched: draft-ietf-ipfix-flow-selection-tech-07.xml
<?xml version="1.0" encoding="US-ASCII"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd">
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="3"?>
<?rfc tocindent="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc comments="yes"?>
<?rfc inline="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?>
<rfc category="std" docName="draft-ietf-ipfix-flow-selection-tech-07.txt"
ipr="pre5378Trust200902">
<front>
<title abbrev="Flow Selection Techniques">Flow Selection
Techniques</title>
<author fullname="Salvatore D'Antonio" initials="S." surname="D'Antonio">
<organization>University of Napoli
"Parthenope"</organization>
<address>
<postal>
<street>Centro Direzionale di Napoli Is. C4</street>
<city>Naples</city>
<code>80143</code>
<country>Italy</country>
</postal>
<phone>+39 081 5476766</phone>
<email>salvatore.dantonio@uniparthenope.it</email>
</address>
</author>
<author fullname="Tanja Zseby" initials="T." surname="Zseby">
<organization>Fraunhofer Institute FOKUS</organization>
<address>
<postal>
<street>Kaiserin-Augusta-Allee 31</street>
<city>Berlin</city>
<code>10589</code>
<country>Germany</country>
</postal>
<phone>+49 30 3463 7153</phone>
<email>tanja.zseby@fokus.fraunhofer.de</email>
</address>
</author>
<author fullname="Christian Henke" initials="C." surname="Henke">
<organization>Technische Universitat Berlin</organization>
<address>
<postal>
<street>Straße des 17. Juni 135</street>
<city>Berlin</city>
<code>10623</code>
<country>Germany</country>
</postal>
<phone>+49 30 3463 7366</phone>
<email>c.henke@tu-berlin.de</email>
</address>
</author>
<author fullname="Lorenzo Peluso" initials="L." surname="Peluso">
<organization>University of Napoli</organization>
<address>
<postal>
<street>Via Claudio 21</street>
<city>Napoli</city>
<code>80125</code>
<country>Italy</country>
</postal>
<phone>+39 081 7683821</phone>
<email>lorenzo.peluso@unina.it</email>
</address>
</author>
<date day="11" month="July" year="2011" />
<workgroup>Internet Engineering Task Force</workgroup>
<abstract>
<t>Flow selection is the process of selecting a subset of flows from all
flows observed at an observation point. Flow selection reduces the effort of
post-processing flow data and transferring Flow Records.
This document describes motivations for flow
selection and presents flow selection techniques. It
provides an information model for configuring flow selection
techniques and discusses what information about a flow selection
process should be exported.</t>
</abstract>
<note title="Requirements Language">
<t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in <xref
target="RFC2119">RFC 2119</xref>.</t>
</note>
</front>
<middle>
<section title="Scope" toc="default">
<t>This document describes flow selection techniques for network traffic
measurements. A flow is defined as a set of packets with common properties as described in <xref target="RFC5101"></xref>.
Flow selection can be done to limit the resource demands for capturing, storing, exporting and
post-processing of Flow Records. It also can be used to select a particular set of flows that are of interest to a specific application. This document provides a categorization of flow selection techniques and describes configuration and reporting parameters for them. In order to be compliant with this document, at least one of the flow selection schemes MUST be implemented. That means that the configuration parameters as well as the reporting Information Elements for this particular scheme MUST be supported.
</t>
<t>This document also addresses configuration and reporting parameters for flow-state dependent packet selection as described in <xref target="RFC5475"></xref>, although this technique is categorized as packet selection. The reason is, that flow-state dependent packet selection techniques often aim at the reduction of resources for flow capturing and flow processing. Furthermore, they were only briefly discussed in <xref target="RFC5475"></xref>. Therefore we included configuration and reporting considerations for such techniques in this document.
</t>
</section>
<section title="Terminology">
<t>This document is consistent with the terminology introduced in <xref target="RFC5101"></xref>, <xref target="RFC5470"></xref>, <xref target="RFC5475"></xref> and <xref target="RFC3917"></xref>.
As in <xref target="RFC5101"></xref> and <xref target="RFC5476"></xref>, the first letter of each IPFIX-specific and PSAMP-specific term is capitalized along with the flow selection specific terms defined here.</t>
<t>* Packet Classification </t>
<t><list style="empty">
<t> Packet Classification is a process by which packets are mapped to specific
Flow Records based on packet properties or external properties (e.g. interface). The properties
make up the Flow Key (e.g. header information, packet content, AS
number). In case a Flow Record for a specific Flow Key already
exists the Flow Record is updated, otherwise a new Flow Record is
created.
</t>
</list></t>
<t>* Packet Aggregation Process</t>
<t><list style="empty">
<t> In the IPFIX Metering Process the Packet Aggregation Process aggregates packet data into flow data and forms the Flow Records. After the aggregation step only the aggregated flow information is available. Information about individual packets is lost. </t>
</list></t>
<t>* Flow Selection Process</t>
<t><list style="empty">
<t>A Flow Selection Process takes Flow Records as its input and selects a
subset of this set as its output. A Flow Selection Process MAY run on several instances within
the IPFIX architecture. A Flow Selection Process MAY be part of an IPFIX Metering Process,
Exporting Process or as an Intermediate Selection Process as defined for the IPFIX Mediator <xref target="RFC6183"></xref>.</t>
</list></t>
<t>* Flow Selection State</t>
<t><list style="empty">
<t>A Flow Selection Process SHOULD maintain state information for use
by the Flow Selector. At a given time, the Flow Selection State may depend on
flows and packets observed at and before that time, as well as other variables. Examples include:<list hangIndent="8"
style="hanging">
<t hangText=" (i)">sequence number of packets and accounted Flow Records;</t>
<t hangText=" (ii)">number of selected flows;</t>
<t hangText=" (iii)">number of observed flows;</t>
<t hangText=" (iv)">current flow cache occupancy;</t>
<t hangText=" (v)">flow specific counters, lower und upper bounds</t>
<t hangText=" (vi)">flow selection timeout intervals</t>
</list></t>
</list></t>
<t>* Flow Selector</t>
<t><list style="empty">
<t>A Flow Selector defines the action of a Flow Selection Process on
a single flow of its input. The Flow Selector can make use of the
following information in order to establish whether a flow has to be
selected or not:<list hangIndent="8" style="hanging">
<t hangText=" (i)">the content of the Flow Record;</t>
<t hangText=" (ii)">any state information related to the Metering Process or Exporting Process;</t>
<t hangText=" (iii)">any Flow Selection State that may be maintained by the Flow Selection Process.</t>
</list></t>
</list></t>
<t>* Complete Flow </t>
<t><list style="empty">
<t> A Complete Flow consists of all packets within the flow time-out interval that enter the Flow Selection Process and belong to the same flow as defined by the flow definition in <xref target="RFC5470"></xref>. For this definition only packets that arrive at the Flow Selection Process are considered. That means, packets that are not observed at the Flow Selection Process because of prior packet selection or packet loss are not considered as belonging to the Complete Flow. </t>
</list></t>
<t>* Flow Filtering</t>
<t><list style="empty">
<t> Flow Filtering selects flows based on a deterministic function on the Flow Record content, flow state, external properties (e.g. ingress interface) or external events (e.g violated Access Control List). If the relevant parts of the Flow Record content can be already observed at packet level (e.g. Flow Keys from packet header fields) Flow Filtering can be performed at packet level by Property Match Filtering as described in <xref target="RFC5475"></xref>.</t>
</list></t>
<t>* Hash-based Flow Filtering</t>
<t><list style="empty">
<t> Hash-based Flow Filtering is a deterministic flow filter function that selects flows based on a Hash Function which is calculated over parts of the Flow Record content or external properties. If the hash value falls into a predefined Hash Selection Range the flow is selected. </t>
</list></t>
<t>* Flow-state Dependent Flow Selection</t>
<t><list style="empty">
<t> Flow-state Dependent Flow Selection is a selection function that selects or drops flows based on the current flow state. The selection can be either deterministic, random or non-uniform random. </t>
</list></t>
<t>* Flow-state Dependent Packet Selection</t>
<t><list style="empty">
<t> Flow-state Dependent Packet Selection is a selection function that selects or drops packets based on the current flow state. The selection can be either deterministic, random or non-uniform random. Flow-state Dependent Packet Selection can be used to prefer the selection of packets belonging to specific flows (e.g. large or small flows).
</t>
</list></t>
<t>* Flow Sampling</t>
<t><list style="empty">
<t> Flow Sampling selects flows based on Flow Record sequence or arrival times (e.g. entry in flow cache, arrival time at Exporter or Mediator). The selection can be systematic (e.g. every n-th flow) or based on a random function (e.g. select each Flow Record with probability p, or randomly select n out of N Flow Records). </t>
</list></t>
</section>
<section title="Difference between Flow Selection and Packet Selection">
<t> Flow selection differs from packet selection described in <xref target="RFC5475"></xref>. Packet selection techniques consider packets as basic element and the parent population consists of all packets observed at an observation point. In contrast to this the basic elements
in flow selection are the flows. The parent population consists of all observed flows and the selection process operates on the flows. The major characteristics of flow selection are the following: </t>
<t>
<list hangIndent="8" style="hanging">
<t hangText="- ">Flow selection takes flows as basic elements. For packet selection, packets are considered as basic elements.</t>
<t hangText="- ">Flow selection can only take place after Packet Classification, because the classification rules determine to which flow a packet belongs. Packet selection can be applied before and after Packet Classification.</t>
<t hangText="- ">Flow selection operates on Complete Flows. That means that after the Flow Selection Process either all packets of the flow are kept or all packets of the flow are discarded. All packets of the flow here means all packets that enter the Flow Selection Process. That means that if the flow selection is preceded by a packet selection process the Complete Flow consists only of the packets that were not discarded during the packet selection.</t>
</list>
</t>
<t>There are some techniques that are difficult to unambiguously categorize into one of the categories. Here we give some guidance how to categorize such techniques:
<list hangIndent="8" style="hanging">
<t hangText="- ">Techniques that can be considered as both packet and flow selection: some packet selection techniques result in the selection of Complete Flows and therefore can be considered as packet or as flow selection at the same time. An example is Property Match Filtering of all packets to a specific destination address. If flows are defined based on destination addresses, such a packet selection also results in a flow selection and can be considered as packet or flow selection.</t>
<t hangText="- "> Flow-state Dependent Packet Selection (as described in <xref target="RFC5475"></xref>): there exist techniques that select packets based on the flow state, e.g. based on the number of already observed packets belonging to the flow. Examples of these techniques from the literature are "Sample and Hold" <xref target="EsVa01"></xref> "Fast Filtered Sampling" <xref target="MSZC10"></xref> or the "Sticky Sampling" algorithm presented in <xref target="MaMo02"></xref>. Such techniques can be used to influence which flows are captured (e.g. increase the selection of packets belonging to large flows) and reduce the number of flows that need to be stored in the flow cache. Nevertheless, such techniques do not necessarily select Complete Flows, because they do not ensure that all packets of a selected flow are captured. Therefore Flow-state Dependent Packet Selection methods that do not ensure that either all or no packets of a flow are selected strictly speaking have to be considered as packet selection techniques and not as flow selection techniques.
</t>
</list>
</t>
</section>
<section title="Flow selection as a Function in the IPFIX Architecture">
<t><xref target="flowselection"></xref> shows the IPFIX reference model
as defined in <xref target="RFC5470"></xref> and shows the Packet Classification
and Packet Aggregation Process in the Metering Process.
<figure anchor="flowselection"
title="Flow selection in the IPFIX Architecture">
<artwork>
Packet(s) coming in to Observation Point(s)
| |
v v
+----------------+---------------------------+ +-----+-------+
| Metering Process | | |
| | | |
| packet header capturing | | |
| | |...| Metering |
| timestamping | | Process N |
| | | | |
| packet sampling | | |
| | | | |
| (packet classification) | | |
| | | | |
| packet filtering* | | |
| | | | |
| (packet aggregation)* | | |
| | | | |
+--------|-----------------------------------+ +-----|-------+
Flow Records Flow Records
| |
+----------------------+----------------------+
|
+----------------------|-----------------+
| Exporting Process* |
+----------------------+-----------------+
| IPFIX (Flow Records)
v
+-------------------------|-----------------------+
| IPFIX Mediator | |
| v |
| Collecting Process(es) |
| | |
| Intermediate Flow Selection Process (*) |
| | |
| Exporting Process(es) |
+-------------------------|-----------------------+
v
IPFIX
(*) indicates where flow selection can take place.</artwork>
</figure>
In contrast to packet selection, flow selection is always
applied after the packets are classified into flows. Flows can be
selected at different stages of the measurement chain:
<list style="numbers">
<t>during the Metering Process</t>
<t>during Exporting Process</t>
<t>during an Intermediate Selection Process on a Mediator</t>
</list></t>
<section title="Flow selection during the Metering Process">
<t> In the Packet Aggregation Process the packet information is used to update the Flow Records in the flow cache. Flow selection that is applied before aggregation equals a packet selection process. The flow still consists of individual packets. Those are then selected based on the classification information, i.e. based on the flow they belong to. Flow selection before aggregation can be based on the fields of the Flow Key (also on a hash value over these fields), but not based on characteristics that are only available after packet aggregation (e.g. flow size, flow duration). Flow selection during the Metering Process is applied to reduce resources for all succeeding processes or to select specific flows of interest in case such flow characteristics are already observable at packet level (e.g. flows to specific IP addresses). In contrast, Flow-state Dependent Packet Selection is a packet selection method, because it does not necessarily select Complete Flows.
</t>
</section>
<section title="Flow selection during the Exporting Process">
<t> The Flow Selection Process at the Exporter is similar to an Intermediate Selection Process as described in <xref target="RFC6183"></xref> and works on Flow records.
Flow selection during the Exporting Process can therefore also depend on flow characteristics that are only visible after the aggregation of packets, such as flow size and flow duration.
The Exporting Process may implement policies for exporting only a subset of the Flow Records which have been stored in the system memory in order to unload flow export
and flow postprocessing. Flow selection during the Exporting Process may select only the subset of Flow Records which are of interest to the users application,
or select only as many Flow Records as can be handled by the available resources (e.g. limited flow cache size and export link capacity).
</t>
</section>
<section title="Flow selection as a function of the IPFIX Mediator">
<t>As shown in <xref target="flowselection"></xref>, flow
selection can be performed as an Intermediate Process within an IPFIX
Mediator <xref target="RFC6183"></xref>. The Intermediate Selection Process takes Flow Record stream as its input and selects a flow record stream.
The Intermediate Selection Process can again apply a flow selection technique to obtain flows of interest to the application. Further the Intermediate Selection Process can
base its selection decision on the correlation of data from different observation points, e.g. by only selecting flows that were at least recorded on two observation points.
</t>
</section>
</section>
<section title="Flow Selection Techniques">
<t> A flow selection technique selects either all or none of the packets of a flow, otherwise the technique has to be considered as packet selection. We distinguish between Flow Filtering and Flow Sampling.
</t>
<section title="Flow Filtering">
<t>
Flow Filtering is a deterministic function on the IPFIX Flow Record content. In case that the relevant flow characteristics are already observable at packet level (e.g. Flow Keys) Flow Filtering can be applied before aggregation at packet level.
</t>
<section title="Property Match Filtering">
<t>Flow Filtering can be done similarly to Property Match Filtering for packet selection described in <xref target="RFC5475"></xref>.
The difference is that, instead of packet fields, Flow Record fields are here used to derive the selection decision. Property Match Filtering is typically used to select a specific subset of the flows that are of interest to a particular application (e.g. all flows to a specific destination, all large flows, etc.). Properties on which the filtering is based can be for example Flow Keys, the flow size in bytes,
the number of packets in the flow, the observation time of the first or last packet, or the maximum packet length. The selection criteria can be a specific value or an interval. Property Match Filtering can be applied during the Metering Process if the properties are already observable at the packet level (e.g. Flow Key fields).
</t>
<t>There are content-based Property Match Filtering techniques that require a computation on the current flow cache. An example is the selection of the k largest flows or a percentage of flows with the longest lifetime. This type of Property Match Filtering is also used in flow selection techniques that react to external events (e.g. resource constraint). For example in case the flow cache is full, the Flow Record with the lowest flow volume per current flow life time is deleted.
</t>
</section>
<section title="Hash-based Flow Filtering">
<t> Hash-based Flow Filtering uses a Hash Function h to map the Flow Key c onto a Hash Range R. A flow is selected if the hash value h(c) is within the Hash Selection Range S, which is
a subset of R. Hash-based Flow Filtering can be used to emulate a random sampling process but still enable the correlation between selected flow subset at different
observation points. Hash-based Flow Filtering is similar to Hash-based Packet Selection, and in fact is identical when Hash-based Packet Selection uses the Flow Key that defines the flow as the hash input.
Nevertheless there MAY be the incentive to apply Hash-based Flow Filtering not on the packet level during the Metering Process, for example when the size of the selection range and therefore the
sampling probability is dependent on the number of observed flows.
</t>
</section>
</section>
<section title="Flow Sampling">
<t>
Flow Sampling operates on Flow Record sequence or arrival times. It can use either a systematic or a random function for the selection process. Flow Sampling usually aims at the selection of a representative subset of all flows in order to estimate characteristics of the whole set (e.g. mean flow size in the network).
</t>
<section title="Systematic sampling">
<t>Systematic sampling is a deterministic selection function. Systematic sampling may be a periodic
selection of the k-th Flow Record which arrives at the Exporting or Intermediate Selection Process. Systematic sampling can also be applied during the Metering Process. An example would be to use an additional data structure that saves the Flow Keys of the non-selected flows.
</t>
<t>
Systematic sampling can also be time-based. Systematic sampling is applied by only creating flows that are observed between time-based start and stop triggers. The time interval may be applied at packet level during the Metering Process or after aggregation on flow level, e.g. by selecting a flow arriving at the Exporting Process every k seconds.
</t>
</section>
<section title="Random sampling">
<t>
Random flow sampling is based on a random process which requires the calculation of random numbers. One can differentiate between n-out-N and probabilistic flow sampling.
The sampling probability of individual Flows Records MAY be adjusted according to the Flow Record content or external events like the available export resources.
Non-uniform random sampling approaches can be applied similar to the ones defined in <xref target="RFC5475"></xref>. An example would be to increase the selection probability of large volume flows over small volume flows as described in the Smart Sampling technique <xref target="DuLT01"></xref>. Random flow sampling can also be applied before the Packet Aggregation Process when additional flow state about non selected flows is kept.
</t>
</section>
</section>
<section title="Flow-state Dependent Flow Selection"><t> Flow-state Dependent Flow Selection can be a deterministic or random flow selection process based on the Flow Record content and the flow state which may be kept additionally for each of the flows. External processes may update counters, bounds and timers for each of the Flow Records and the Flow Selection Process utilises this information for the selection decision. A review of Flow-state Dependent Flow Selection techniques that aim at the selection of the most frequent items by keeping additional flow state information can be found in <xref target="CoHa08"></xref>. Flow-state Dependent Flow Selection can only be applied after packet aggregation, when a packet has been assigned to a flow. The selection process then decides based upon the flow state for each flow if it is kept in the flow cache or not.
Two Flow State Dependent Flow Selection are here described: </t>
<t> The frequent algorithm <xref target="KaPS03"></xref> is a technique that aims at the selection of all flows that at least exceed a 1/k fraction of the observed packet stream. The algorithm has only a flow cache of size k-1 and each flow in the cache has an additional counter. The counter is incremented each time a packet belonging to the flow in the flow cache is observed. In case the observed packet does not belong to any flow all counters are decremented and if any of the flow counters has a value of zero the flow is replaced with a flow formed from the new packet. </t>
<t>Lossy counting is a selection technique that identifies all flows whose packet count exceeds a certain percentage of the whole observed packet stream (e.g. 5% of all packets) with a certain estimation error e. Lossy counting separates the observed packet stream in windows of size N=1/e, where N is an amount of consecutive packets. For each observed flow an additional counter will be held in the flow state. The counter is incremented each time a packet belonging to the flow is observed and all counters are decremented at the end of each window and all flows with a counter of zero will be removed from the flow cache. </t>
</section>
<section title="Flow-state Dependent Packet Selection">
<t> Flow-state Dependent Packet Selection is not a flow selection technique but a packet selection technique. Nevertheless we will describe configuration and reporting parameters for this technique in this document. An example is the "Sample and Hold" algorithm <xref target="EsVa01"></xref> that tries to prefer large volume flows in the selection. When a packet arrives it is selected when a Flow Record for this packet already exists. In case there is no Flow Record, the packet is selected by a certain probability that is dependent on the packet size.
</t>
</section>
</section>
<section title="Configuration of Flow Selection Techniques">
<t>This section describes the configuration parameters of the flow selection techniques presented above. It
provides the basis of an information model to be adopted in order to configure the Flow Selection Process within an IPFIX Device. The following table gives an overview of the defined selection techniques, where they can be applied and what their input parameters are. Dependent on where the flow selection techniques are applied different input parameters can be configured. </t>
<t>Overview of Flow Selection Techniques:</t>
<texttable align="left" style="all">
<ttcol>Location</ttcol>
<ttcol>Selection Method</ttcol>
<ttcol>Selection Input</ttcol>
<c>During the Metering Process based on Packets </c>
<c>Flow-state Dependent Packet Selection</c>
<c>packet sampling probabilities, flow state, packet properties</c>
<c></c>
<c>Property Match Flow Filtering</c>
<c>Flow Key fields, filter function</c>
<c></c>
<c>Hash-based Flow Filtering</c>
<c>selection range, Hash Function, Flow Key</c>
<c></c>
<c>Time-based Systematic Flow Sampling</c>
<c>flow position (derived from arrival time of packets), flow state</c>
<c></c>
<c>Sequence-based Systematic Flow Sampling</c>
<c>flow position (derived from packet position), flow state</c>
<c></c>
<c>Random Flow Sampling</c>
<c>random number generator or list and packet position, flow state</c>
<c>Exporting / Intermediate Selection Process</c>
<c>Property Match Flow Filtering</c>
<c>Flow Record content, filter function</c>
<c></c>
<c>Hash-based Flow Filtering</c>
<c>selection range, Hash Function, hash input (Flow Keys and other flow properties)</c>
<c></c>
<c>Flow-state Dependent Flow Selection</c>
<c>flow state parameters, random number generator or list</c>
<c></c>
<c>Time-based Systematic Flow Sampling</c>
<c>flow arrival time, flow state</c>
<c></c>
<c>Sequence-based Systematic Flow Sampling</c>
<c>flow position, flow state</c>
<c></c>
<c>Random Flow Sampling</c>
<c>random number generator or list and flow position, flow state</c>
</texttable>
<section title="Description of Flow Selection Techniques">
<t>In this section, we define what parameters are required to describe the most common Flow Selection techniques. </t>
<t>Flow Selection Parameters: </t>
<t>For Property Match Filtering:
<list hangIndent="4" style="hanging">
<t hangText="- "> Information Element (from <xref target="RFC5102"></xref>): <vspace blankLines='0' /> Specifies the Information Element which is used as the property in the filter expression. </t>
<t hangText="- "> Selection Value or Value Interval: <vspace blankLines='0' /> Specifies the value or interval of the filter expression. Packets and Flow Record that have a value equal to the Selection Value or within the Interval will be selected. </t>
</list></t>
<t>For Hash-based Flow Filtering:
<list hangIndent="4" style="hanging">
<t hangText="- "> Hash Domain: <vspace blankLines='0' />
Specifies the bits from packet (IPv4 or IPv6) which are taken as the hash input to the Hash Function. </t>
<t hangText="- "> Hash Function: <vspace blankLines='0' /> Specifies the name of the Hash Function that is used to calculate the hash value. Possible Hash Functions are BOB, IPSX, CRC-32 </t>
<t hangText="- "> Hash Selection Range: <vspace blankLines='0' /> Flows that have a hash value within the Hash Selection Range are selected. The Hash Selection Range can be a value interval or arbitrary hash values within the Hash Range of the Hash Function. </t>
<t hangText="- "> Random Seed or Initializer Value: <vspace blankLines='0' /> Some Hash Functions require an initializing value. In order to make the selection decision more secure one can choose a random seed that configures the hash function. </t>
</list></t>
<t>For Flow-state Dependent Flow Selection:
<list hangIndent="4" style="hanging">
<t hangText="- ">frequency threshold: <vspace blankLines='0' /> Specifies the frequency threshold s for flow state dependent flow selection techniques that try to find the most frequent items within a dataset. All flows which exceed the defined threshold will be selected.
</t>
<t hangText="- ">accuracy parameter: <vspace blankLines='0' /> specifies the accuracy parameter e for techniques that deal with the frequent items problems. The accuracy paramter defines the maximum error, i.e. no flows that have a true frequency less than (s- e) N is selected, where s is the frequency threshold and N is the total number of packets.
</t>
</list></t>
<t>The above list of parameters for Flow-state Dependent Flow Selection techniques is suitable for the presented frequent item and lossy counting algorithm. Nevertheless there exist a variety of techniques with very specific parameters which are not defined here. </t>
<t>For Systematic time-based Flow Sampling:
<list hangIndent="4" style="hanging">
<t hangText="- ">Interval length (in usec) <vspace blankLines='0' /> Defines the length of the sampling interval during which flows are selected.
</t>
<t hangText="- ">Spacing (in usec) <vspace blankLines='0' /> The spacing parameter defines the spacing in usec between the end of one sampling interval and the start of the next succeeding interval.
</t>
</list> </t>
<t>For Systematic count-based Flow Sampling:
<list hangIndent="4" style="hanging">
<t hangText="- "> Interval length <vspace blankLines='0' /> Defines the number of flows that are selected within the sampling interval.
</t>
<t hangText="- ">Spacing <vspace blankLines='0' />
The spacing parameter defines the spacing in number of observed flows between the end of one sampling interval and the start of the next succeeding interval.
</t>
</list> </t>
<t>For random n-out-of-N Flow Sampling:
<list hangIndent="4" style="hanging">
<t hangText="- ">Population Size N <vspace blankLines='0' />
The Population Size N is the number of all flows in the Population from which the sample is drawn.
</t>
<t hangText="- ">Sample size n <vspace blankLines='0' />
The sample size n is the number of flows that are randomly drawn from the population N.
</t>
</list> </t>
<t>For probabilistic Flow Sampling:
<list hangIndent="4" style="hanging">
<t hangText="- ">Sampling probability p <vspace blankLines='0' />
The sampling probability p defines the probability by which each of the observed flows is selected.
</t>
</list> </t>
</section>
<section title="Description of Flow-state Dependent Packet Selection">
<t>
The configuration of Flow-state Dependent Packet Selection has not been described in <xref target="RFC5475"></xref> therefore the parameters are defined here:
</t>
<t>For Flow-state Dependent Packet Selection:
<list hangIndent="4" style="hanging">
<t hangText="- "> packet selection probability per possible flow state interval <vspace blankLines='0' />
Defines multiple [flow interval, packet selection probability] value pairs that configure the sampling probability dependent on the current flow state.
</t>
<t hangText="- "> additional parameters <vspace blankLines='0' />
For the configuration of flow state dependent packet selection additional parameters or packet properties may be required for the configuration, e.g. the packet size (<xref target="EsVa01"></xref>)</t>
</list></t>
</section>
</section>
<section title="Information Model for Flow Selection Reporting">
<t>In this section we describe Information Elements (IEs) that SHOULD be exported by a flow selection process in order to support the interpretation of measurement results from flow measurements where only some flows are selected. The information is mainly used to report how many packets and flows have been observed in total and how many of them were selected. This helps for instance to calculate the Attained Selection Fraction, which is an important parameter to provide an accuracy statement.
The IEs can provide reporting information about Flow Records, packets or bytes. The reported metrics are number of total and the number of selected elements. From this the number of dropped elements can be derived. All counters SHOULD be exported and reset when a new measurement interval starts. Additional IEs may be useful for future flow selection techniques. Those can be defined additionally if needed.
</t>
<t>List of additional Flow Selection Information Elements:</t>
<texttable align="center" style="all">
<ttcol>ID</ttcol>
<ttcol>Name</ttcol>
<c>TBD1</c>
<c>fsFlowRecordTotalCount</c>
<c>TBD2</c>
<c>fsFlowRecordSelectedCount</c>
<c>TBD3</c>
<c>fsPacketTotalCount</c>
<c>TBD4</c>
<c>fsPacketSelectedCount</c>
<c>TBD5</c>
<c>fsOctetTotalCount</c>
<c>TBD6</c>
<c>fsOctetSelectedCount</c>
</texttable>
<section title="fsFlowRecordTotalCount">
<t>Description:<list>
<t>This Information Element specifies the current number of all Flow Records that form the parent population as input to the Flow Selection Process.
</t>
</list></t>
<t>Abstract Data Type: unsigned64</t>
<t>ElementId: TBD1</t>
<t>Units: Flows</t>
</section>
<section title="fsFlowRecordSelectedCount">
<t>Description:<list>
<t>This Information Element specifies the current number of Flow Records that were selected during the Flow Selection Process.
</t>
</list></t>
<t>Abstract Data Type: unsigned64</t>
<t>ElementId: TBD2</t>
<t>Units: Flows</t>
</section>
<section title="fsPacketTotalCount">
<t>Description:<list>
<t>This Information Element specifies the current number of packets in all flows that form the parent population as input to the Flow Selection Process.
</t>
</list></t>
<t>Abstract Data Type: unsigned64</t>
<t>ElementId: TBD7</t>
<t>Units: Packets</t>
</section>
<section title="fsPacketSelectedCount">
<t>Description:<list>
<t>This Information Element specifies the current number packets in all flows that were selected during the Flow Selection Process.
</t>
</list></t>
<t>Abstract Data Type: unsigned64</t>
<t>ElementId: TBD8</t>
<t>Units: Packets</t>
</section>
<section title="fsOctetTotalCount">
<t>Description:<list>
<t>This Information Element specifies the current number of all bytes in all flows that form the parent population as input to the Flow Selection Process.
</t>
</list></t>
<t>Abstract Data Type: unsigned64</t>
<t>ElementId: TBD9</t>
<t>Units: Octets</t>
</section>
<section title="fsOctetSelectedCount">
<t>Description:<list>
<t>This Information Element specifies the current number of bytes in all flows that were selected during the Flow Selection Process.
</t>
</list></t>
<t>Abstract Data Type: unsigned64</t>
<t>ElementId: TBD10</t>
<t>Units: Octets</t>
</section>
</section>
<section title="IANA Considerations">
<t>This document introduces several new Information Elements as an
extension to the IPFIX information model. Values TBD1-TBD10 in section 7 of this document
should be replaced with the assigned numbers by IANA.</t>
</section>
<section title="Security Considerations">
<t>In this section security issues concerning an IPFIX Device performing
flow selection are pointed out. In case the flow selection function
is activated an IPFIX Device might be exposed to security threats.
Since flow selection implies analysing flow packets, associating them
to a specific traffic flow and selecting Flow Records, a malicious
user who was able to gain control of an IPFIX Device might access
both packet and flow data, thus violating their confidentiality.</t>
<t>Furthermore, the intruder might be attracted by the possibility of
altering the Flow Selection Process by modifying the criteria used to
select Flow Records. In this case, the IPFIX Device would export
flow data which are different from the ones that the Collector
expects to receive.</t>
<t>It is apparent that these security threats can be mitigated by
authenticating entities that interact with the IPFIX Device and
keeping information for flow selection configuration confidential.</t>
</section>
</middle>
<back>
<references title="Normative References">
<?rfc include="reference.RFC.2119"?>
</references>
<references title="Informative References">
<!--
<reference anchor="DuLT01b">
<front>
<title>Properties and Prediction of Flow Statistics from Sampled
Packet Streams</title>
<author fullname="N.G. Duffield" initials="N.G." surname="Duffield">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<author fullname="C. Lund" initials="C." surname="Lund">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<author fullname="M. Thorup" initials="M." surname="Thorup">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<date day="6-8" month="November" year="2002" />
<abstract>
<t></t>
</abstract>
</front>
<seriesInfo name="ACM SIGCOMM"
value="Internet Measurement Workshop 2002" />
</reference>
-->
<reference anchor="DuLT01">
<front>
<title>Charging from Sampled Network Usage</title>
<author fullname="N.G. Duffield" initials="N.G." surname="Duffield">
</author>
<author fullname="C. Lund" initials="C." surname="Lund">
<organization></organization>
</author>
<author fullname="M. Thorup" initials="M." surname="Thorup">
<organization></organization>
</author>
<date day="1-2" month="November" year="2001" />
<abstract>
<t></t>
</abstract>
</front>
<seriesInfo name="ACM"
value="Internet Measurement Workshop IMW 2001, San Francisco, USA" />
</reference>
<!--
<reference anchor="Moli03">
<front>
<title>A scalable and efficient methodology for flow monitoring in
the Internet</title>
<author fullname="Maurizio Molina" initials="M." surname="Molina">
<organization></organization>
</author>
<date month="September" year="2003" />
</front>
<seriesInfo name="International Teletraffic Congress"
value="(ITC-18), Berlin" />
</reference>
-->
<reference anchor="CoHa08">
<front>
<title>Finding frequent items in data streams</title>
<author fullname="Graham Cormode" initials="G." surname="Cormode">
<organization></organization>
</author>
<author fullname="Marios Hadjieleftheriou" initials="M." surname="Hadjieleftheriou">
<organization></organization>
</author>
<date month="August" year="2008" />
</front>
<seriesInfo name="Journal, Proceedings of the Very Large DataBase Endowment"
value="VLDB Endowment, Volume 1 Issue 2, August 2008" />
</reference>
<reference anchor="KaPS03">
<front>
<title>A simple algorithm for finding frequent elements in sets and bags.</title>
<author fullname=" R. Karp" initials="R." surname="Karp">
<organization></organization>
</author>
<author fullname="C. Papadimitriou" initials="C." surname="Papadimitriou">
<organization></organization>
</author>
<author fullname="S. Shenker" initials="S." surname="S. Shenker">
<organization></organization>
</author>
<date month="March" year="2003" />
</front>
<seriesInfo name="ACM Transactions on Database Systems,"
value="Volume 28, 51-55, 2003" />
</reference>
<reference anchor="MaMo02">
<front>
<title>Approximate Frequency Counts over Data Streams</title>
<author fullname="Gurmeet Singh Manku " initials="G.S." surname="Manku">
<organization></organization>
</author>
<author fullname="Rajeev Motwani " initials="R." surname="Motwani">
<organization></organization>
</author>
<date year="2002" />
</front>
<seriesInfo name="Proceedings of the Internation Conference on Very large DataBases (VLDB)"
value="pages 346--357, 2002, Hong Kong, China" />
</reference>
<reference anchor="MSZC10">
<front>
<title>Fast Filtered Sampling</title>
<author fullname="Jianning Mai" initials="J." surname="Mai">
<organization></organization>
</author>
<author fullname="Ashwin Sridharan" initials="A." surname="Sridharan">
<organization></organization>
</author>
<author fullname="Hui Zang" initials="H." surname="Zang">
<organization></organization>
</author>
<author fullname="Chen-Nee Chuah" initials="C.-N." surname="Chuah">
<organization></organization>
</author>
<date month="January" year="2010" />
</front>
<seriesInfo name="Computer Networks"
value="Volume 54, Issue 11, Pages 1885-1898, ISSN 1389-1286" />
</reference>
<reference anchor="EsVa01">
<front>
<title>New Directions in Traffic Measurement and Accounting:
Focusing on the Elephants, Ignoring the Mice</title>
<author fullname="C. Estan" initials="C." surname="Estan">
<organization></organization>
</author>
<author fullname="G. Varghese" initials="G," surname="Varghese">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<date month="November" year="2001" />
</front>
<seriesInfo name="ACM SIGCOMM"
value="Internet Measurement Workshop 2001, San Francisco (CA)" />
</reference>
<!--
<reference anchor="KuXW04">
<front>
<title>Space-code bloom filter for efficient per-flow traffic measurement</title>
<author fullname="K. Kumar" initials="K." surname="Kumar">
<organization></organization>
</author>
<author fullname="J. Xu" initials="J." surname="Xu">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<author fullname="Jia Wang" initials="J." surname="Wang">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<author fullname="O. Spatschek" initials="O." surname="Spatschek">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<author fullname="L. Li" initials="L." surname="Li">
<organization></organization>
<address>
<postal>
<street></street>
<city></city>
<region></region>
<code></code>
<country></country>
</postal>
<phone></phone>
<facsimile></facsimile>
<email></email>
<uri></uri>
</address>
</author>
<date day="7-11" month="March" year="2004" />
</front>
<seriesInfo name="INFOCOM 2004"
value="Twenty-third AnnualJoint Conference of the IEEE Computer and Communications Societies" />
</reference>
-->
<?rfc include="reference.RFC.3917"?>
<?rfc include="reference.RFC.5101"?>
<?rfc include="reference.RFC.5102"?>
<?rfc include="reference.RFC.5470"?>
<?rfc include="reference.RFC.5475"?>
<?rfc include="reference.RFC.5476"?>
<?rfc include="reference.RFC.6183"?>
</references>
</back>
</rfc>
| PAFTECH AB 2003-2026 | 2026-04-23 14:16:40 |