One document matched: draft-djernaes-netflow-9-transport-00.txt

   Internet Draft                                                       
   Expiration: August 2003                              Martin Djernaes
   Document: draft-djernaes-netflow-9-transport-00.txt    Cisco Systems 
   Category: Informational                               February  2003 
    
    
              Cisco Systems NetFlow Services Export 

	               Version 9 Transport
                                      

   
Status of this Memo 
    
   This document is an Internet-Draft and is in full conformance with  
   all provisions of Section 10 of RFC2026.  
    
   Internet-Drafts are working documents of the Internet Engineering  
   Task Force (IETF), its areas, and its working groups.  Note that  
   other groups may also distribute working documents as Internet-  
   Drafts. Internet-Drafts are draft documents valid for a maximum of  
   six months and may be updated, replaced, or obsolete by other  
   documents at any time. It is inappropriate to use Internet-Drafts as  
   reference material or to cite them other than as "work in progress."  
    
   The list of current Internet-Drafts can be accessed at  
   http://www.ietf.org/ietf/1id-abstracts.txt  
   The list of Internet-Draft Shadow Directories can be accessed at   
   http://www.ietf.org/shadow.html. 


     
Abstract 
    
   NetFlow Export Version 9 Export is defined in the internet draft
   draft-claise-netflow-9 [NFv9], which describes how to transport
   NetFlow over UDP [RFC768]. This document describes how NetFlow
   would use SCTP [RFC2960] as a transport protocol.

   Traditionally NetFlow was transmitted over UDP to a physically
   closely located collector. As the networks grow this is not
   necessarily the case any longer, so issues like congestion
   avoidance and reliability are becoming an increasing issue for
   applications using NetFlow. This document addresses these issues in
   a simple way using the SCTP protocol.


Conventions used in this document 
    
   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in  
   this document is to be interpreted as described in RFC 2119. 


    


Djernaes                         Draft                          [Page 1]

Cisco NetFlow v9 transport                                 February 2003

Table of Contents 
    
   1. Introduction...................................................2 
      1.1 Overview...................................................2 
      1.2 Congestion Avoidance.......................................2
      1.3 Reliability................................................2
   2. NetFlow data...................................................3
      2.1 Packet size................................................3
      2.2 Source ID..................................................3
   3. Exporter.......................................................3
      3.1 Observation Domain.........................................3
      3.2 Association................................................4
      3.3 Stream.....................................................4
      3.4 Template...................................................4
      3.5 Stream usage...............................................5
   4. Collector......................................................5
      4.1 Listen.....................................................5
      4.2 Receive....................................................5
   5. References.....................................................6
   6. Acknowledgments................................................6
   7. Authors Addresses..............................................6



1. Introduction 
    
1.1 Overview 

   This document describes how Cisco NetFlow version 9[NFv9] can be
   transported over SCTP[RFC2960] using traditional reliable mode, but
   also use the partial reliable or unreliable mode[PR-SCTP].

   Weight is especially put on the freedom to use SCTP as a reliable,
   single stream transport, as well as multiple streams with different
   properties, for example in terms of reliability, carrying different
   data types dependant on their importance for the system.

   This document does not describe how to classify importance of data,
   and does therefore not describe how an exporter selects what kind
   of stream to use for a given set of data.

1.2 Congestion Avoidance

   The SCTP transport protocol provides the required level of
   congestion avoidance by design.

1.3 Reliability

   The SCTP transport protocol is by default reliable, but has the
   capability to operate in unreliable and partially reliable modes
   [PR-SCTP].

   Using reliable SCTP streams for NetFlow v9 export is not in itself
   a guarantee that all records are delivered. If there is congestion
   on the link from the exporter to the collector, or if a significant

Djernaes                         Draft                          [Page 2]

Cisco NetFlow v9 transport                                 February 2003

   amount of retransmissions are needed, the send queues on the
   exporter may fill up. In that case it's up to the exporter to
   decide what to do. It may either halt export (buffer the data until
   there is space in the send queues again) or just throw export
   packets away instead of inserting them into the send queue. If
   any data is not inserted into the send queues, the sequence numbers
   used for export must reflect the loss of data.

   Unreliable or partial reliability may be chosen for one or more
   streams inside an association. Unreliable transport may be
   preferred where large amount of data is to be exported and keeping
   send queues is either an unnecessary overhead or
   impractical. Partial reliability may be chosen where a small
   amount of queuing is possible.



2. NetFlow data

2.1 Packet size

   When transmitting NetFlow data it should be formatted as described
   in the NetFlow draft[NFv9] with one Packet Header and a number of
   Flow Sets per export packet. Each NetFlow export packet should be
   equal to or less than the local MTU in size. Where a NetFlow packet
   is transmitted over a network with an MTU smaller than the local
   MTU, IP fragmentation may be used.

2.2 Source ID

   The NetFlow draft states that each NetFlow export packet must
   contain a Packet Header, which includes a source id (SID). The SID
   indicate from which Observation Domain inside the exporter the data
   is being exported, and should be keept unique for each such domain.



3. Exporter

3.1 Observation Domain

3.1.1 Single Observation Domain

   If an exporter consists of a single observation domain, a single
   SID value must be used for all export packets. The exporter will
   typically open one association to the collector, but more are
   possible, in which one or more streams can be used.

   The exporter has the choice of transmitting parts of the export
   data in separate streams or all data in one stream.






Djernaes                         Draft                          [Page 3]

Cisco NetFlow v9 transport                                 February 2003

3.1.2 Multiple Observation Domains

   If an exporter consists of multiple observation domains, one SID
   value for each observation domain must be used. The exporter will
   typically open one association, but more are possible, in which at
   least one stream per observation domain is used.

   The exporter have the choice of using more than one stream per
   observation domain, but data from multiple observation domains
   should not be transmitted over the same stream.

3.2 Association

   The exporter may create one or more associations (connection
   "bundle" in SCTP terminology) from the NetFlow exporter to the
   NetFlow collector. The NetFlow collector may not initiate the
   connection. Inside each association one or more streams may be
   requested by the exporter. If the collector can not support the
   requested number of streams, it may choose to refuse the connection
   and the exporter should try to reduce, if possible, the number of
   streams needed to perform the export.

3.3 Stream

   An Observation Domain must use at least one stream, but may use
   multiple streams, to export NetFlow data. The Observation Domain
   must use the same SID value for all streams used.

   An exporter must not transmit packets with different SID values
   in one stream, the collector should however verify that the SID
   values are the expected values.

3.4 Template

   Since the SCTP association is connection oriented the available
   templates must be transmitted from each Observation Domain to the
   collector immediately after the association is established.

   As a minimum the templates must be transmitted immediately after
   they start to exist on the exporter and should preferably be
   transmitted before any data, using the new template, have been
   transmitted. The collector should however accept data without a
   template and as with UDP it should store the data until the
   template arrives.

   When using a reliable mode for template export, or if the exporter
   knows that the export packet containing the templates was
   positively acknowledged by the SCTP layer, it is not necessary to
   periodically export the templates as described in the NetFlow v9
   Export draft [NFv9].






Djernaes                         Draft                          [Page 4]

Cisco NetFlow v9 transport                                 February 2003

3.5 Stream usage

   The decision of what to send over what kind of stream is an
   application choice and outside the scope of this
   document. Naturally it is better to send templates over a reliable
   stream and send the data on an unreliable (or partial reliable)
   stream. When an exporter handles data with different properties it
   might even be preferable to send them over different streams
   according to those properties.
   
3.5.1 Example: One stream for everything

   If an exporter contain only one Observation Domain the NetFlow v9
   export packets can be inserted unmodified onto a single SCTP
   stream. This stream can either be reliable, partial reliable or
   just unreliable. Using an unreliable stream gives only few
   advantages over a partial reliable stream.

3.5.2 Example: Two streams per Observation Domain

   An exporter can use two streams per Observation Domain. A reliable
   stream could be used for exporting templates, to reduce the
   likelihood of loss and to remove the need for blind
   retransmissions, and a partial or unreliable stream for data, to
   avoid buffering of large amounts of data.



4. Collector

4.1 Listen

   The collector should listen for a new association request from the
   exporter. The exporter will request a number of streams to use for
   export. If the collector doesn't support the number of streams
   inside the association, the collector must refuse the connection
   and continue listen for a new request.

4.2 Receive

   When data is received from an association, the collector must
   correlate data, with the same SID value, from multiple streams into
   one export flow from an Observation Domain. This allow the
   Observation Domain to use separate streams for different types of
   information.

   The collector should verify that the received export packets inside
   one stream does not have diverting SID values. The exporter must
   not export packets inside one stream with multiple SID values.

   The correlated export flow are then treated like a normal export
   flow as described in [NFv9].




Djernaes                         Draft                          [Page 5]

Cisco NetFlow v9 transport                                 February 2003

5. References

  [NFv9]    Claise, B, "Cisco Systems NetFlow Services Export Version
            9", draft-claise-netflow-9-01.txt

  [RFC768]  Postel, J. (ed.), "User Datagram Protocol", STD 6, RFC
            768, August 1980.

  [RFC2960] Stewart, R. (ed.) "Stream Control Transmission Protocol",
            RFC 2960, October 2000 

  [PR-SCTP] Stewart, R, "SCTP Partial Reliability Extension",
	    draft-stewart-tsvwg-prsctp-01.txt


6. Acknowledgments 

   Thanks to Benoit Claise, Vamsi Valluri, Randall Stewart and Stewart
   Bryant for their valuable comments.


7. Authors Addresses 

   Martin Djernaes 
   Cisco Systems, Inc. 
   170 W Tasman Drive
   San Jose, CA 95134 
   USA 
   Phone:  +1 (408) 853-1676 
   Email:  djernaes@cisco.com 
 

























Djernaes                         Draft                          [Page 6]