One document matched: draft-ietf-tewg-measure-06.txt

Differences from draft-ietf-tewg-measure-05.txt



Traffic Engineering Working Group                           Wai Sum Lai 
Internet Draft                                                AT&T Labs 
Document: <draft-ietf-tewg-measure-06.txt>                              
Category: Informational                                Richard W. Tibbs 
                                                    Oak City Networks & 
                                                              Solutions 
                                                                        
                                                  Steven Van den Berghe 
                                                  Ghent University/IMEC 
                                                                        
                                                              July 2003 
    
    
         Requirements for Internet Traffic Engineering Measurement 
    
    
Status of this Memo 
 
   This document is an Internet-Draft and is in full conformance with 
   all provisions of Section 10 of RFC2026.  
    
   Internet-Drafts are working documents of the Internet Engineering 
   Task Force (IETF), its areas, and its working groups. Note that 
   other groups may also distribute working documents as Internet-
   Drafts. Internet-Drafts are draft documents valid for a maximum of 
   six months and may be updated, replaced, or obsoleted by other 
   documents at any time. It is inappropriate to use Internet- Drafts 
   as reference material or to cite them other than as "work in 
   progress." 
    
   The list of current Internet-Drafts can be accessed at 
   http://www.ietf.org/ietf/1id-abstracts.txt 
     
   The list of Internet-Draft Shadow Directories can be accessed at 
   http://www.ietf.org/shadow.html. 
 
Abstract 
    
   In this document, we identify requirements for supporting the 
   traffic engineering of IP networks.  Requirements for traffic 
   measurement in service provider environments are presented and 
   justified, and related issues are discussed. 
    
   Highlights of requirements are: 
   1. To aid network dimensioning, mechanisms to collect node-pair-
   based traffic data are required to facilitate the derivation of per-
   service-class traffic matrix statistics.  
   2. For service assurance, the use of higher-order statistics is 
   required. 
   3. To preserve representative traffic detail at manageable sample 
   volumes, packet-sampled measurements are required. 
   4. To manage large volumes of measured data, use of bulk transfer 
   and filtering/aggregation mechanisms are required. 
    
  
Lai, et al              Category - Expiration                [Page 1] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
    
Table of Contents 
    
   Status of this Memo................................................1 
   Abstract...........................................................1 
   Conventions used in this document..................................3 
   1. Introduction....................................................3 
   2. Conclusions and Recommendations.................................4 
   3. Requirements for TE Measurement and Its Uses....................4 
   3.1 Requirements for Traffic characterization......................5 
   3.2 Requirements for Network monitoring............................5 
   3.3 Requirements for Traffic Matrix Statistics.....................6 
   3.4 Requirements for Performance Monitoring........................6 
   3.5 Requirements for Path Characterization.........................7 
   4. Requirements Summary for TE Measurement Types...................7 
   4.1 Measurement types related to traffic or performance............8 
   4.2 Measurement types related to resource usage....................8 
   5. Requirements for a TE Measurement Information Model.............9 
   6. Measurement Definitions........................................11 
   6.1 Active, passive measurements..................................11 
   6.2 Route, path...................................................11 
   6.3 Throughput, traffic volume....................................11 
   APPENDICES........................................................12 
   APPENDIX A........................................................12 
   A. Measurement Bases..............................................12 
   A.1 Flow-based....................................................14 
   A.2 Interface-based, link-based, node-based.......................14 
   A.3 Node-pair-based...............................................15 
   A.4 Path-based....................................................15 
   APPENDIX B........................................................16 
   B. Measurement Entities...........................................16 
   B.1 Entities related to traffic and performance...................16 
   B.2 Entities related to establishment of connection or path.......18 
   APPENDIX C........................................................18 
   C. Packet Sampling and Estimation.................................18 
   C.1 Packet Sampling...............................................19 
   C.2 Sampling Issues...............................................19 
   C.3 Engineering methods for statistical estimation of measures....20 
   APPENDIX D........................................................20 
   D. Read-Out Periods...............................................20 
   D.1 Data Reduction................................................21 
   D.2 Measurement Interval..........................................21 
   D.3 Summarization.................................................21 
   APPENDIX E........................................................22 
   E. Time Scales for Network Operations.............................22 
   APPENDIX F........................................................23 
   F. Use of Traffic Measurement for Traffic control.................23 
   16. Security Considerations.......................................23 
   17. References....................................................23 
   18. Intellectual Property Statement...............................26 
   19. Acknowledgments...............................................26 
   20. Author's Addresses............................................26 
   Full Copyright Statement..........................................27 
  
Lai, et al              Category - Expiration                [Page 2] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
    
Conventions used in this document 
    
   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in 
   this document are to be interpreted as described in RFC-2119. 
 
1. Introduction 
    
   In this document, we identify requirements for supporting IP network 
   traffic engineering (TE) [1].  Requirements for traffic measurement 
   in service provider environments are presented and justified, and 
   related issues are discussed.  To aid network dimensioning, 
   mechanisms to collect node-pair-based traffic data are required to 
   facilitate the derivation of per-service-class traffic matrix 
   statistics.  For service assurance, the use of higher-order 
   statistics is required.  To preserve representative traffic detail 
   at manageable sample volumes, packet-sampled measurements are 
   required.  To manage large volumes of measured data, use of bulk 
   transfer and filtering/aggregation mechanisms are required. 
    
   Requirements for TE measurement are motivated by the needs for 
   consistency, precision, and effectiveness of the overall TE 
   function.  TE includes measurements, forecasting, planning, 
   dimensioning, control, and performance monitoring.  TE measurement 
   plays a key role to assure the quality of the other aspects of TE. 
    
   Uses of traffic measurement in traffic characterization, network 
   monitoring, and traffic control are first described.  Depending on 
   the network operations to be performed in these tasks, three 
   different time scales can be identified, ranging from months, 
   through days or hours, to minutes or less.  To support these 
   operations, traffic measurement must be able to capture accurately, 
   within a given confidence interval, the traffic variations and peaks 
   without degrading network performance and without generating an 
   immense amount of data.  As one consequence of the need to avoid 
   network performance degradation, specification of a suitable read-
   out period (i.e., summarization or aggregation interval) is 
   essential. 
    
   Traffic measurement can be performed on the basis of flows, 
   interfaces, links, nodes, node-pairs, or paths.  Based on these 
   objects, different measurement entities can be defined, such as 
   traffic volume, average holding time, bandwidth availability, 
   throughput, delay, delay variation, packet loss, and resource usage.  
   Using these measured traffic data, in conjunction with other network 
   data such as topological data and router configuration data, traffic 
   matrix and other relevant statistics can be derived for TE purposes. 
    
   IP multicast traffic measurement is not explicitly addressed in this 
   document.  Nonetheless, given additional elaboration on tree-based 
   measurement principles, most of the considerations for different 
   measurement types (see Appendices A and B) could be applied to IP 
  
Lai, et al              Category - Expiration                [Page 3] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   multicast traffic.  Such elaboration may be dealt with in a 
   subsequent document for specific IP multicast-inferred Internet 
   traffic measurement. 
    
   Relevant work done in measurements by other standards organizations 
   will be applied or adapted, and references to them will be made.  
   These include, in particular, 
    
   . IP Performance Metrics (IPPM) Working Group of the IETF: its 
     framework document [2] and the associated documents on individual 
     metrics [3, 4, 5, 6, 7, 8, 9, 10, 11] 
    
   . ITU-T: Recommendation I.380/Y.1540 [12] and Recommendation Y.1541 
     [13] 
    
2. Conclusions and Recommendations 
    
   Requirements are given in this document for traffic metrics needed 
   for successful TE.  Principles of best practice in traffic 
   characterization and performance characterization are described in 
   the Appendices.  For interoperable compatibility and consistency, 
   requirements for traffic measurement recommended for standardization 
   include: 
    
   (1) Requirements for specific TE measurements 
    
   . Node-pair-based traffic data to derive per-service-class traffic 
     matrix statistics, including statistics of carried load and 
     offered load (Sections 3.3 and Appendix A) 
   . Statistics of achieved performance and throughput (Section 3.4) 
   . A standardized method to detect and record label binding changes 
     for LDP-signaled label-switched paths, at the ingress-egress pair 
     level (Section 3.5) 
    
   (2) Requirements for traffic data collection methods 
 
   . Standardization of measurement definitions and sampling methods, 
     to achieve uniformity across vendors and operators, and to 
     preserve sufficient traffic detail at manageable sample volumes 
     (Section 6 and Appendix C) 
   . Higher-order statistics to facilitate service assurance (Section 
     3.1) 
   . Offline bulk file transfer and standardized filtering/aggregation 
     mechanisms to manage large volumes of measured traffic data 
     (Section 5 and Appendix D) 
   . Linkage between policy mechanisms and TE measurement, possibly 
     triggered by a measurement-driven event notification (Section 5) 
   . Standardization of information models for TE measurement (Section 
     5) 
    
3. Requirements for TE Measurement and Its Uses 
    

  
Lai, et al              Category - Expiration                [Page 4] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   TE measurement is used to collect traffic data for the following 
   purposes: 
   . Traffic characterization 
   . Network monitoring 
   . Traffic matrix Statistics 
   . Performance monitoring 
   . Path characterization 
    
3.1 Requirements for Traffic characterization 
    
   . Requirement 1: Standardization of higher-order statistics to 
     facilitate service assurance.  
   . Requirement 2: Identifying traffic patterns, particularly traffic 
     peak patterns, and their variations in statistical analysis; this 
     includes developing traffic profiles to capture daily, weekly, or 
     seasonal variations. 
   . Requirement 3: Determining traffic distributions in the network on 
     the basis of flows, interfaces, links, nodes, node-pairs, paths, 
     or destinations.  (These bases are discussed in Appendix A.) 
   . Requirement 4: Estimation of the traffic load according to service 
     classes in different routers and the network. 
   . Requirement 5: Observing trends for traffic growth and forecasting 
     of traffic demands. 
    
   For example, TE can use measurements to determine the statistical 
   moments of a traffic flow.  As suggested in [14], given the time 
   series of packet arrivals, a suitable parametric stochastic model 
   based on the mean and variance of the time series can be 
   constructed.  This traffic model is then used in the ensuing phases 
   of TE, such as link dimensioning to meet service objectives. 
    
3.2 Requirements for Network monitoring 
    
   . Requirement 1: Determining the operational state of the network, 
     including fault detection. 
   . Requirement 2: Monitoring the continuity and quality of network 
     services, to ensure that QoS/CoS objectives are met for various 
     classes of traffic, to verify the performance of delivered 
     services, or to serve as a means of sectionalizing performance 
     issues seen by a customer.  [Note 1.  QoS reflects the performance 
     perceivable by a user of a service, while CoS (class of service) 
     is used by a service provider for internal design and operation of 
     a network.]  [Note 2.  Mechanisms for monitoring service 
     continuity may be service-specific and are not discussed here.] 
   . Requirement 3: Evaluating the effectiveness of TE policies, or 
     triggering certain policy-based actions (such as alarm generation, 
     or path preemption) upon threshold crossing; this may be based on 
     the use of performance history data. 
   . Requirement 4: Verifying peering agreements between service 
     providers by monitoring/measuring the traffic flows over 
     interconnecting links at border routers (note that peers are in 
     general not willing to divulge detailed traffic picture inside 
     their autonomous systems); this includes the estimation of inter- 
  
Lai, et al              Category - Expiration                [Page 5] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
     and intra-domain traffic, as well as originating, terminating, and 
     transit traffic that are being exchanged between peers. 
    
   An example of using TE measurements in this area might be monitoring 
   packet loss rates at various points in a network to detect apparent 
   link failure.  Another example is observing traffic at peering 
   points to ensure that peering agreements are met. 
    
3.3 Requirements for Traffic Matrix Statistics 
    
   Requirement 1 
   Standardization of node-pair-based traffic data to derive per-
   service-class traffic matrix statistics, including statistics of 
   carried load and offered load. 
    
   An important set of data for TE is point-to-point or point-to-
   multipoint demands.  This data may be of significant use in the 
   provisioning of traffic-engineered intra-domain paths and external 
   peering in the existing network, as well as planning for the 
   placement and sizing of new links, routers, or peers. 
    
   In current practice, estimates for traffic demands are usually 
   determined from a combination of traffic projections, customer 
   prescriptions, and service level agreements.  Using the facilities 
   of SNMP (Simple Network Management Protocol), it is not easy to 
   obtain network-wide traffic demands from the local interface 
   measurements taken by different IP routers.  As explained in [15, 
   16], information from diverse network measurements, including flow-
   based measurements, and various topological data and router 
   configuration files are needed to infer the traffic volume. 
    
   Some shortcomings in today's method to derive traffic matrix 
   statistics as above include the volume of data from flow-based 
   measurement, the lack of sufficient routing control information, and 
   the need to correlate data from a variety of sources.  To avoid some 
   of these deficiencies and to take advantage of the routing control 
   offered by MPLS, node-pair-based passive measurement should be 
   developed. 
    
3.4 Requirements for Performance Monitoring 
    
   Requirement 1 
   Standardization of statistics of achieved performance and 
   throughput. 
    
   Requirement 2 
   Performance monitoring as a means to trigger LSP restoration 
   activities. 
    
   A major component of performance management is performance 
   monitoring, i.e., continuous real-time monitoring of the quality or 
   health of the network and its various elements to ensure a 
   sustained, uninterrupted delivery of quality service. 
  
Lai, et al              Category - Expiration                [Page 6] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
    
   General aspects of measurements required to support the operation, 
   administration, and maintenance of a network are outside the scope 
   of this document (see [17, 18, 19, 20] for a discussion of MPLS 
   OAM).  However, performance monitoring is required for TE 
   measurement since monitoring the quality of delivered services is 
   essential feedback to the TE function. 
    
   This requires the use of measurement, either passively or actively, 
   to collect information about the operational state of the network 
   and to track its performance.  For a discussion of passive 
   monitoring and the use of synthetic traffic sources in active 
   probing, see [21, 22]. 
    
   Performance degradation can occur as a result of routing 
   instability, congestion, or failure of network components.  Periods 
   of congestion may be detected when the resource usage of a network 
   segment consistently exceeds a certain threshold, or when the cross-
   router delay is unexpectedly high.  Unexpected excessive loss of 
   packets or throughput drops may be used as a means of fault 
   detection, and may result in restoration activities. 
    
3.5 Requirements for Path Characterization 
    
   Requirement 1 
   Standardization of a method to detect and record label binding 
   changes for LDP-signaled label-switched paths, at the ingress-egress 
   pair level. 
    
   In the case of hop-by-hop routed label-switched paths that are 
   established by Label Distribution Protocol (LDP) signaling, there is 
   no explicit binding between path end points.  This will result in 
   the use of different label bindings at both the ingress and egress 
   nodes over time as network topology changes.  Although the 
   forwarding equivalence class (FEC) to label binding information 
   already exists in the MPLS FTN and LSR MIBs [23, 26], a mechanism is 
   needed to keep track of binding changes.  An example of such a 
   mechanism may be the periodic exchange of FEC to label binding 
   information for each ingress-egress pair. 
    
   Internet utilities such as ping and traceroute have been useful to 
   help diagnose network problems and performance debugging.  Utilities 
   with similar functions would be essential for path-oriented 
   operations like in MPLS.  This would include the capability to list, 
   at any time, (1) for a given path, all the nodes traversed by it, 
   and (2) for a given node, all the paths originating from it, 
   transiting through it, and/or terminating on it.  A proposal for 
   path tracing is described in [24].  A proposal to establish basic 
   MPLS data plane liveness is described in [25]. 
    
4. Requirements Summary for TE Measurement Types 
    

  
Lai, et al              Category - Expiration                [Page 7] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   A measurement type is a meaningful and measurable combination of a 
   measurement basis (Appendix A) and a measurement entity (Appendix 
   B).  Two sets of measurement types, organized in the form of 
   matrices, are presented in the following two subsections. 
    
4.1 Measurement types related to traffic or performance 
    
   The following measurement matrix summarizes the measurement types 
   related to traffic or performance.  Potentially, there can be one 
   such matrix for each service class. 
    
            Bases:     Flow     Interface,   Node Pair      Path 
                                   Node                       
  Entities:          (passive)   (passive)     (both)      (both) 

  Traffic Volume        x(1)         x          x(3)        x(3) 
  Avg. Hold. Time        x                                  x(3) 
  Avail. Bandwidth                   x                      x(3) 
  Throughput             x                      x(4)        x(4) 
  Delay                             x(2)        x(4)        x(4) 
  Delay Variation                   x(2)        x(4)        x(4) 
  Packet Loss            x           x          x(5)        x(5) 
    
   Notes: 
   (1) This measurement type can be used to derive flow size 
   statistics. 
   (2) These are 1-point measurements.  For a discussion on 1-point 
   packet delay variation, see [12], Appendix II. 
   (3) As a starting point, statistics collected by passive measurement 
   through the MIBs useful for TE [26, 27, 28] may be used. 
   (4) Active measurements based on IPPM metrics are currently in use 
   for node-pairs; they may be developed for paths.  
   (5) Besides active measurements based on IPPM, path loss may 
   possibly be inferred from the difference between ingress and egress 
   traffic statistics at the two endpoints of a path.  However, such 
   inference for the cumulative losses between a given node pair over 
   multiple routes may be less useful, since different routes may have 
   different loss characteristics. 
    
4.2 Measurement types related to resource usage 
    
   Another measurement matrix summarizes measurement types comprising 
   the different usage, one for each network resource object such as 
   router (processor and memory), link, and buffer, by different 
   classes of traffic: 
    
   . control (e.g., routing control) traffic 
   . signaling traffic 
   . user traffic from different service classes 
    
                        Bases:   Node     Link    Buffer 
           Entities: 
           Control Util.           x        x        x 
  
Lai, et al              Category - Expiration                [Page 8] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 

           Signaling Util.         x        x        x 
           Service Class Util.     x        x        x 
    
   The amount of control and signaling traffic carried by a network is 
   a function of many factors.  To name a few, they include the size 
   and topology of the network, the control and signaling protocols 
   used, the amount of user traffic carried, the number of failure 
   events, etc.  Also, flooding of link-state advertisement (LSA) 
   messages in Interior Gateway Protocols (IGP, such as OSPF or IS-IS) 
   may cause significant routing control traffic during events such as 
   an LSA storm as a result of failures due to fiber cuts or failed 
   power supply.  The above utilization measurements for control and 
   signaling traffic are intended to help develop guidelines for the 
   proper dimensioning and apportionment of network resources so that a 
   given level of user traffic can be adequately supported. 
    
5. Requirements for a TE Measurement Information Model 
    
   Requirement 1 
   Standardization of an information model for TE measurement. 
    
   Requirement 2 
   Standardization of offline bulk file transfer and standardized 
   filtering/aggregation mechanisms to manage large volumes of measured 
   traffic data (see Appendix D for further discussion). 
    
   Several approaches and options for repository technology are now 
   broadly discussed.  Relationships between TE measure information 
   models on other information models (e.g., the COPS Policy Information 
   Base, PIB) that drive network outcomes are of particular importance.  
   For an example of a PIB, see [29]. 
    
   Linkages should be considered between policy mechanisms and TE 
   measures.  This is useful because, while policy-driven networking is 
   well-developed between the policy repositories, policy decision 
   points and policy enforcement points, policy content is very likely 
   the output of TE applications [30].  **Since TE applications are 
   dependent upon TE measures, it is advantageous to provide 
   traceability between the measures and the engineering changes made as 
   a consequence of them.**  An example of a client application that 
   might be driven by TE measures through a PIB is found in [31, 32]. 
    
   Measures (represented by their estimates) should be centrally stored 
   and collected in a logical sense.  This does not preclude distributed 
   storage for purposes of volume management or security/survivability, 
   but alludes to the need for a consistent retrieval mechanism (e.g., 
   NFS).  Two methods are: (1) extend MIBs with new definitions for TE 
   measure estimates or extend PIBs with new objects and use the COPS 
   feedback extensions to get statistics stored at the policy 
   enforecement points, and (2) create data depositories through more 
   centralized facilities, such as relational databases or LDAP (see 
   [29]).  Both methods have merits as collection processes for TE 

  
Lai, et al              Category - Expiration                [Page 9] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   measures, and are simple examples spanning a wide spectrum of 
   solutions.  These two methods are discussed here for expository 
   purposes, not to exclude other solutions. 
    
   Using MIBs allows well-established SNMP protocol and related 
   applications to retrieve data from the network elements being 
   measured.  This is inherently "vendor-neutral," allowing commonly 
   defined TE measurements to be stored for retrieval in a common MIB 
   definition, regardless of network element vendor, technology or other 
   differences. 
    
   A centralized data storage facility has the advantage that TE 
   applications (such as offline and online TE, or measurement-based 
   admission control) can be performed without invasive retrieval of 
   data from network-wide MIBs. 
    
   It is possible that both the distributed MIB-based and centralized 
   repository-based approaches (or another approach altogether) should 
   be considered jointly.  However, this is not mandatory: TE systems 
   could rely solely on events from distributed measurement points, 
   e.g., based on threshold checking in every device.  Even in this 
   case, a centralized storage should be in place to log these events so 
   to provide a linkage between the observed behavior and resulting 
   configuration. 
    
   Although this document focuses on the motivation for providing TE 
   measurement information, it is assumed that this information should 
   be provided to the participating devices by means of a communication 
   protocol that would be used between the aforementioned participating 
   devices and a presumably centralized entity that would aim at 
   storing, maintaining and updating this information, as well as 
   making appropriate decisions at the right time and under various 
   conditions. 
    
   This communication protocol should have the following 
   characteristics: 
    
   1. The protocol should make use of a reliable transport mode, given 
   the importance of configuration information. 
   2. The protocol architecture should provide a means for dynamically 
   provisioning the configuration information to the participating 
   devices, so that it may introduce/contribute to a high level of 
   automation in the actual TE measurement operation. 
   3. The protocol should support one or more reporting mechanisms that 
   may be used for statistical information retrieval.  Reporting 
   mechanisms can be either polling-based (explicit requests) or event-
   based (asynchronous reports). 
   4. The protocol should support the appropriate security mechanisms 
   to provide some guarantees as far as the preservation of the 
   confidentiality of the configuration information is concerned. 
   5. The protocol should support reporting at regular intervals, and 
   can optionally support asynchronous conditional reporting (e.g., 
   whenever a value crosses a threshold). 
  
Lai, et al              Category - Expiration               [Page 10] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
    
6. Measurement Definitions 
    
   Requirement 1 
   Standardization of measurement definitions and sampling methods, to 
   achieve uniformity across vendors and operators and to preserve 
   sufficient traffic detail at manageable sample volumes. 
    
   It is critical to minimize the possibilities of inconsistencies 
   arising from, e.g., differing statistical definitions, overlapping 
   data collection, processing at different protocol levels, and 
   similar inconsistencies by different vendors or network operators.  
   Uniform measurement definitions across vendors and operators should 
   be enforced as far as possible. 
    
   As this is a requirements document and not a document for 
   measurement definitions, the intent of this section is not to 
   provide definition of terms used.  Rather, it is to highlight the 
   difference in usage of closely related terms and describe terms used 
   herein.  Nor is this section exhaustive, since needs for other 
   measures may arise in practice (for examples of other closely-
   related metrics see [33]). 
    
6.1 Active, passive measurements 
    
   These terms are used in the sense of [2].  In an active measurement, 
   test packets, or probes, are injected into the network.  Data 
   collected about these packets are taken as representative of the 
   behavior of the network.  Passive measurements are in-service, non-
   intrusive, and so can be performed directly on the user traffic.  
   For a discussion of sampling issues related to both active and 
   passive measurements, see Appendix C. 
    
6.2 Route, path 
    
   A route is any unidirectional sequence of nodes and links, for 
   sending packets from a source node to a destination node.  A path 
   refers to an MPLS tunnel, i.e., a label-switched path (LSP) [34], 
   this LSP possibly being a traffic-engineered LSP.  Measurements on 
   non-traffic-engineered LSPs may be collected to support the possible 
   future traffic-engineering of those LSPs.  (Note:  What is defined 
   as a route here is referred to as a path in [2].  The route/path 
   distinction is made here to facilitate applications to MPLS.) 
    
   It should be pointed out that there are also methods for creating 
   paths with other technologies such as frame relay or ATM.  The 
   measurement described in this document may apply to these 
   technologies with suitable adaptation.  To simplify description, 
   reference is made to MPLS only in what follows. 
    
6.3 Throughput, traffic volume 
    

  
Lai, et al              Category - Expiration               [Page 11] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   Both quantities can be applied to a network, a network segment, or 
   an individual network element.  Thus, measurement points need to be 
   appropriately defined when a specific measurement is to be performed 
   (e.g., from a given ingress node to another egress or a set of 
   egress nodes). 
    
   Throughput of a network, as a measure of delivered performance, 
   refers to the maximum sustainable rate of transferring packets 
   successfully across the network, under given network conditions, 
   e.g., a given traffic mix, while meeting quality of service (QoS) 
   objectives.  This usage is consistent with the definition of 
   throughput for a network interconnect device as specified in [35].  
   For real-time network control, active measurement of throughput by 
   probing may be used to determine the currently available capacity of 
   a network to carry additional traffic.  (Note: Goodput is a related 
   term referring to a proportion of the traffic successfully 
   transmitted; similarly, badput refers to a proportion of the traffic 
   lost or being corrupted.) 
    
   Traffic volume, reflecting the traffic carried, is the amount of 
   traffic measured during a given period of time.  Passive measurement 
   of the traffic volume is usually used to estimate the long-term 
   offered traffic for the purposes of network dimensioning in the 
   capacity-management and network-planning processes (see Appendix E 
   on Time Scales for Network Operations).  A network should be 
   properly dimensioned so that its throughput is adequate to handle 
   the expected traffic volume.  Hence, traffic volume measurement 
   should be performed on a regular basis. 
    
   Throughput at a cross-section, or specific point in the network, is 
   expressed in terms of number of data units per time unit.  Traffic 
   volume is expressed in data units with reference to a read-out 
   period (see Appendix D on Read-Out Periods).  For transmission 
   systems, the data unit is usually a multiple of either bits or 
   bytes.  For processing systems, the data unit is usually a multiple 
   of packets. 
    
APPENDICES 
    
APPENDIX A 
    
A. Measurement Bases 
    
   Measurements can be classified on the basis of where, and at which 
   level of aggregation the traffic data are gathered.  This is similar 
   to the concept of a *population of interest* as specified in ITU-T 
   Recommendation I.380/Y.1540.  As defined therein, this refers to a 
   set of packets, possibly relative to a particular pair of source and 
   destination hosts, for the purposes of defining performance 
   parameters.  However, measurement bases as used here may not have 
   any association with a source-destination pair.  This is to be 
   described in more details below.  Currently, the different 

  
Lai, et al              Category - Expiration               [Page 12] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   measurement bases to be defined below have not been explicitly 
   specified in the IPPM Framework [2]. 
    
   In this document, the focus is on service providers as organizations 
   requiring traffic and performance measurements.  (However, customer-
   based measurements of enterprise networks may have similar issues.)  
   Service providers will make decisions on how to perform the 
   measurements needed, and there are various tradeoffs involved.  One 
   option is to obtain the measurements directly from the network 
   elements themselves, e.g., via SNMP.  Collecting the measurements on 
   the operational network elements such as routers is sometimes a 
   performance concern.  Currently, there is a number of third-party 
   measurement/monitoring products available.  Hence, another option is 
   to deploy such equipment, which might have performance advantages 
   but also introduces additional cost. 
    
   Regardless of the type of measurement source, either a network 
   element or a third-party product, measurements should be collected, 
   as far as possible, by a measurement source without requiring 
   coordination with other measurement sources.  Thus, it is desirable 
   to perform those measurements that do not require the use of 
   specialized monitoring equipment connected to the network at 
   multiple locations.  While each measurement source may act 
   autonomously with regard to taking measurements, a network operator 
   may specify some network-wide policy regarding measurement 
   scheduling.  Such policy may be, say, the use of the same time of 
   day, the same measurement interval, or measurement intervals that 
   are multiples of each other (e.g., nested intervals with 
   synchronized boundaries).  A schedule therefore should include such 
   time information as the start, the duration, and periodicity of a 
   certain measurement.  Also note that the accuracy of traffic 
   measurement is highly dependent on the synchronization capabilities 
   of the measurement devices that will be involved in the measurement 
   procedures.  While synchronization issues are out of the scope of 
   this document, they should be explicitly addressed whenever a 
   measurement campaign is to be launched, whatever its scope and its 
   frequency. 
    
   The following measurement bases are considered in this document: 
   . Flow-based 
   . Interface-based, link-based, node-based 
   . Node-pair-based 
   . Path-based 
    
   Passive measurements are prevalent today for TE purposes.  However, 
   the above measurement bases may result in active or passive 
   measurements.  For example, an active measurement may be a two-point 
   delay metric such as type-P-one-way-delay defined in [4], and 
   obtained by time-stamping probe packets at selected ingress and 
   egress points; a passive measurement may be to obtain packet inter-
   arrival times by time-stamping successive packets of the traffic at 
   a selected point in the network.  Note that both active and passive 

  
Lai, et al              Category - Expiration               [Page 13] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   measurements are subject to the same sampling and time-source 
   accuracy concerns. 
    
   MPLS has certain advantages when compared with conventional IP 
   networks, from the perspective of the difficulty involved in 
   obtaining unambiguous measurements.  **As different service 
   providers will adopt different technologies, technology-neutral 
   solutions to the problem of obtaining measurements are needed as far 
   as possible.** 
    
   Applicability of traffic measurements to the derivation of traffic 
   matrix statistics and performance monitoring has been described in 
   Section 3. 
    
A.1 Flow-based 
    
   This is conceptually similar to the call detail record (CDR) in 
   circuit-switched telecommunications networks.  It is primarily used 
   on interfaces at access routers, edge routers, or aggregation 
   routers, rather than on backbone routers in the core network.  Like 
   CDR measurements, flow-based records are used to collect detailed 
   information about a flow.  This includes such information as source 
   and destination IP addresses/port numbers, protocol, type of 
   service, timestamps for the start and end of a flow, packet count, 
   octet count, etc. 
    
   As flow is a fine-grained object, measuring every flow that passes 
   through all the edge devices may not be scalable or feasible.  
   Hence, per-flow data are usually used in a special study conducted 
   on a non-continuous schedule and on selected routers only.  Sampling 
   of flow-based measurements may also be needed to reduce both the 
   amount of data collected and the associated overhead. 
    
A.2 Interface-based, link-based, node-based 
    
   While active measurements are often not useful at a single point, 
   passive measurements can be taken at each network element.  For 
   example, SNMP uses passive monitoring to collect raw data on an 
   interface at an edge or backbone router.  These data are stored in 
   MIBs (Management Information Bases) and include counts on packets 
   and octets sent/received, packet discards, errored packets.  Such 
   measurements may have the disadvantage that the identity of each 
   flow is lost. 
    
   To reduce the overhead in managing multiple links between the same 
   ingress and egress points, there is proposal to aggregate links for 
   network optimization [36].  Component links in such a bundled link 
   will have the same routing constraints, resource classes, and 
   attributes.  Multiple links are treated as a single IP link.  
   Traffic measurements, such as bandwidth availability, throughput, 
   etc., should consider the measurement implications for bundled 
   links, and should not inhibit link bundling.  (For example, a single 
   IP link may presumably be referenced as a pair of IP addresses that 
  
Lai, et al              Category - Expiration               [Page 14] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   are assigned to both extremities of the link.  An implicit issue 
   that may need to be resolved relates to the exact characterization 
   of the traffic that will be conveyed in each component link, since a 
   couple of IP addresses may not be sufficient for such link-based 
   measurement.)  Also, such measurements should be protocol 
   independent and media independent to ensure portability and 
   commonality in the measurements. 
    
A.3 Node-pair-based 
    
   Active measurements by probing, as specified in the IPPM framework 
   for example, can be conducted between each pair of (major) routing 
   hubs for determining edge-to-edge performance of a core network.  
   This complements the passive measurements of the previous sub-
   section, which provide local views of the performance of individual 
   network elements. 
    
   In contrast to performance statistics, traffic loading statistics 
   require passive measurements of the actual traffic.  In circuit-
   switched telecommunications networks, each established call has an 
   associated source/destination node-pair.  By maintaining a set of 
   node-pair data registers [usage, call attempts (so-called "peg 
   count" in telephony operation and management), overflow, etc.] in 
   each switch, node-pair-based measurements for traffic statistics 
   such as the load between a given node pair are taken directly.  In 
   IP networks, currently such node-pair-based measurements are 
   difficult to establish due to the dynamic and asymmetric properties 
   of IP routing.  However, it is possible to infer them from flow-
   based passive measurements and other network information, such as 
   routing table snapshots.  A problem with this approach is that flow-
   based measurement data are voluminous.  Also, another problem that 
   must be accounted for is the routing changes among the multiple 
   routes due to, e.g., a change in the configuration of intra-domain 
   routing, or a change in inter-domain policies made by another 
   autonomous system.  These issues were discussed in Section 3.4 on 
   Traffic Matrix Statistics. 
    
A.4 Path-based 
    
   The ability of MPLS to use fixed preferred paths for routing traffic 
   gives the means to develop path-based measurements.  This may enable 
   the development of methodologies for such functions as admission 
   control and performance verification of delivered service. 
    
   Like a flow, a path is associated with a pair of nodes.  However, 
   path is a more coarse-grained object than flow, as paths are usually 
   used to carry aggregated traffic (from different flows).  In 
   addition, when routing changes occur, the amount of traffic to be 
   carried by a path will either not be affected or be merged with that 
   of another path.  Because of these properties, path-based 
   measurements are more scalable and may be used to provide more 
   readily an accurate, network-wide, view of the traffic demands.  For 
   example, the traffic between a given pair of nodes may be inferred 
  
Lai, et al              Category - Expiration               [Page 15] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   from the aggregate of the traffic carried by all paths either 
   terminated by or passed through the same node-pair. 
    
APPENDIX B 
    
B. Measurement Entities 
    
   A measurement entity defines what is measured: it is a quantity for 
   which data collection must be performed with a certain measurement.  
   A measurement type can be specified by a (meaningful) combination of 
   a measurement entity with the measurement basis described in 
   Appendix A. 
    
   An important issue with any measurement is measurement precision 
   and/or accuracy.  However, this issue is not dealt with here since 
   each measurement type will potentially have its own unique 
   requirements.  For example, see [4], Section 3.7, for a discussion 
   on error issues for one-way delay. 
    
B.1 Entities related to traffic and performance 
    
   Some of the measurement entities listed below, such as throughput, 
   delay, delay variation, and packet loss, are related to the 
   respective IPPM performance metrics or the I.380/Y.1540 performance 
   parameters. 
    
   . Traffic volume (mean and variance, in number of bits, bytes, or 
     packets transferred, as counted over a given time interval), on a 
     per service class basis, at various aggregation levels (IP address 
     prefix, interface, link, node, node-pair, path, network edge, 
     customer, or autonomous system) 
     Note:  (1) This is a measurement for the traffic carried by a 
     network, a network segment, or an individual network element; it 
     is used to derive the carried load or carried traffic intensity 
     [37].  When measured during the busy period, this entity is 
     normally used to estimate the traffic offered.  However, the 
     estimation procedure should take into account such factors as 
     congestion, which may result in a decreased volume of carried 
     traffic.  In addition, congestion may lead to user behavior such 
     as reattempt or abandonment, which may affect the actual traffic 
     offered.  (2) To reduce uncertainty in traffic estimation, second-
     order measures may need to be developed.  Beyond the use of 
     variance as in current practice, further study is needed for the 
     feasibility of other second-order techniques.  (3) Measurement of 
     traffic volumes over interconnecting links at border routers can 
     be used to estimate the traffic exchange between peers for 
     contract verification. 
      
   . Average holding time (e.g., flow duration or lifetime, duration of 
     an MPLS path), on a per service class basis 
     Note:  (1) When MPLS TE is used, this is similar to call holding 
     time in telecommunications networks.  Call attempts, usage, and 
     call holding time are three busy-hour entities that should be 
  
Lai, et al              Category - Expiration               [Page 16] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
     independently measured for both call-dependent and load-dependent 
     engineering.  This is important especially when the call busy hour 
     and the load busy hour during a day are non-coincident, due to the 
     hour-to-hour variation of call holding times.  (2) The holding 
     time statistics of long-living static paths reflect the effect of 
     network equipment failures, link outages, or scheduled 
     maintenance, and hence may be used to derive information about up-
     time or service availability.  (3) It is desirable to be able to 
     gather, by passive means, the up-time durations for each pair of 
     label bindings in the label-forwarding information base for labels 
     distributed by different protocols (such as LDP, RSVP-TE, MP-BGP, 
     or BGP).  Then, the derivation of LSP average holding time does 
     not need to be finely correlated with network events such as 
     link/node failures.  (Note that routers measure only the holding 
     times, with their averages being typically computed offline.) 
      
   . Available bandwidth of a link or path - useful for load balancing, 
     measurement-based admission control to determine the feasibility 
     of creating a new MPLS tunnel (real-time information can be used 
     for dynamic establishment) 
     For more information on available bandwidth see [38]. 
    
   . Throughput (in bits per second, bytes per second, or packets per 
     second) 
     Note:  (1) This is the rate at which a given amount of traffic 
     excluding lost, misdelivered, or errored packets, that passes 
     between a set of end points, where end points can be logically or 
     physically defined.  The condition of the network, e.g., normal or 
     high load, under which the measurement is taken should be noted.  
     (2) The protocol level at which a throughput measurement is taken 
     must be specified, as the packet payload and packet overheads are 
     protocol dependent.  (3) The average packet size may be inferred 
     from the bit rate and packet rate measurements, when performed on 
     the basis of an individual router.  This quantity is useful to 
     gauge router performance, since router operations are typically 
     packet-oriented and small packets are more processing-intensive. 
      
   . Delay (e.g., cross-router delay from node-based measurement may be 
     used to measure queueing delay within a router; end-to-end one-way 
     or round-trip packet delay can be obtained by node-pair-based 
     measurement) 
     Note: The condition of the network, e.g., normal or high load, 
     under which the measurement is taken should be noted.  This is 
     useful to determine if delay objectives are met. 
      
   . Delay variation 
     Note:  There are several methods to measure this quantity as 
     specified in ITU-T and IPPM.  (1) In Y.1540, measurements are 
     defined for both 2-point and 1-point IP packet delay variation.  
     However, 2-points methods are being specified as normative.  (2) 
     In IPPM [9], the concept of a selection function is introduced 
     that allows for the explicit designation of selected packets whose 
     one-way delay values are compared to compute one-way delay 
  
Lai, et al              Category - Expiration               [Page 17] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
     variation.  For example, to define a method of measurement, a 
     selection function can be specified to select the consecutive 
     packets within a specified interval, or to select the maximum and 
     minimum one-way delays within a specified interval. 
      
   . Packet loss 
     Note:  (1) While packet losses due to transmission and/or protocol 
     errors may not be traffic related, unexpected excessive loss may 
     be used as a means of fault detection.  (2) In most active 
     measurements, the cause of packet loss is not distinguished.  
     However, it may be desirable to distinguish (e.g., by passive 
     means) packet losses due to policing or network congestion.  The 
     former is a result of user violation of service contract and the 
     network operator should not be penalized for it.  The latter, 
     whether intentional or unintentional, is caused by network 
     conditions such as buffer overflow, router forwarding process 
     busy, and may not be the user's fault.  When policing is done by a 
     network, measurement of non-conforming packets at the edge 
     provides an indication on the extent to which the network is 
     carrying this type of packets (which can potentially be dropped if 
     network gets congested).  Loss due to congestion of any packets, 
     including loss of non-conforming packets, is a useful measure in 
     TE to account for resource management.  (3) Long-term averages can 
     be measured by the I.380/Y.1540 IP packet loss ratio or by the 
     IPPM Poisson sampling of one-way loss.  However, during the 
     convergence times associated with routing updating, the loss may 
     be high enough as to cause service unavailability.  This effect 
     needs to be captured and statistics such as loss patterns, burst 
     loss, or severe loss ratio may be useful. 
      
   . Resource usage, such as link/router utilization, buffer occupancy 
     (e.g., fraction of arriving packets finding the buffer above a 
     given set of thresholds) 
     Note:  (1) Depending on the architecture of a router, router 
     utilization measurements may include processor and memory (e.g., 
     forwarding tables) utilization for each of the line cards and/or 
     the central unit.  (2) Trigger points may be set when resource 
     usage consistently exceeds a certain threshold. 
      
B.2 Entities related to establishment of connection or path 
    
   Where connection admission control is used, a measurement entity for 
   monitoring network performance may be the proportion of connections 
   denied admission.  Also, it may be useful to score the requested 
   bandwidth within the traffic parameters for the setup request.  
   Corresponding to the number of call attempts (i.e., peg count) in 
   telecommunications networks, the number of connection requests, the 
   number of flows, etc., may be measured in given read-out periods to 
   characterize the traffic. 
    
APPENDIX C 
    
C. Packet Sampling and Estimation 
  
Lai, et al              Category - Expiration               [Page 18] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
    
C.1 Packet Sampling 
    
   A wide spectrum of operational applications can be built on traffic 
   measurement.  However, different applications usually require 
   traffic measurements at different levels of temporal and spatial 
   granularity.  To achieve an effective tradeoff between 
   implementation complexity and the range of operational tasks to be 
   enabled, a passive measurement framework based on packet sampling is 
   proposed in [39].   
    
   The use of packet sampling has two motivations.  First, the enormous 
   volumes of traffic require that some form of data reduction to be 
   used.  Second, simple data reduction by aggregation at the 
   measurement point will not provide sufficiently detailed views for 
   all network management applications or exploratory studies.  For 
   this reason, packet sampling is proposed as a means to reduce data 
   volume while still retaining representative detail. 
    
   The primary aim of the proposal [39] is to define a minimal set of 
   primitive packet selection operations out of which all sampling 
   operations that are necessary to support measurement-based 
   applications can be composed.  Operations currently under 
   consideration include filtering and statistical sampling, and also 
   hash-based packet selection, a method that can be used to support 
   the determination of spatial traffic flows across a domain [40].  
   Whichever method is used, the interpretation of the stream of 
   measurements arising from sampled packets must be both transparent 
   and standard.  Other goals are to specify a means to format and 
   export measurements, and a means to manage the configuration of the 
   sampling and export operations. 
    
   The proposal positions these functions to provide a basic packet 
   sampled measurement service to higher level "consumers."  A typical 
   consumer is a network management application that sits behind a 
   remote measurement collector.  Such measurements can support 
   applications for a number of tasks: troubleshooting, demand 
   characterization, scenario evaluation and what-ifs.  Another type of 
   consumer is a higher level on-router measurement application.  One 
   potential class of examples is composite measurements (e.g., inter-
   packet delay statistics) formed from a number of individual packet 
   measurements.  Another class is network security applications, e.g., 
   IP traceback [41].  For some applications, the ability to have low 
   latency between packet measurement and reporting will be 
   particularly useful.  
    
C.2 Sampling Issues 
    
   The concept of read-out periods applies to both active and passive 
   measurements.  This concept is consistent with the sampling issues 
   for a series of measurements as developed in [2], for example.  See 
   sections 10 and 11 of that document for important distinctions 
   between "singletons, samples, and statistics."  The procedure of 
  
Lai, et al              Category - Expiration               [Page 19] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   Poisson sampling, for example, may be used within a read-out period 
   to select a subset of total packet events that are chosen as the 
   sample.  Then a statistic (e.g., mean or variance) can be computed 
   over that sample and associated with the read-out period.  Although 
   [2] does not discuss traffic volume measures such as a traffic 
   matrix, the same sampling issues arise for the traffic matrix and 
   other passive measurements. 
    
C.3 Engineering methods for statistical estimation of measures 
    
   The use of the well-established methods of optimal estimation [42, 
   43, 44, 45] to obtain estimates of the measures for TE is 
   recommended.  This draws upon several facts: 
    
   . Internet traffic is inherently band-limited, but non-stationary;  
   . Internet traffic may be heavy-tailed and possess strong short-term 
     correlations; 
   . A stationary, band-limited process can be approximated arbitrarily 
     closely by optimal estimation methods based on a finite number of 
     past samples. 
    
   Standard procedures for de-trending the raw data to provide "trend + 
   stationary" decompositions should be adopted.  An example is the use 
   of Autoregressive Integrated Moving Average (ARIMA) models, where 
   first differences are applied to the raw (non-stationary) data, 
   yielding a stationary derived process.  Then, the methods of optimal 
   estimation can be applied in a practical setting (e.g., finite sample 
   counts) to the derived stationary process to produce quality 
   estimates of the measures defined herein.  As the original raw 
   process may be any of the measurements discussed in this document, 
   the above procedure may be applied without loss of generality to 
   measures of delay, loss, or complex measures of network state such as 
   path characteristics, etc. 
    
   In addition, these methods need to be applied across multiple time-
   scales, so that TE applications can work with measures related to: 
   . long-term trends over days, weeks, and months; 
   . busy-hour characterizations; and 
   . statistics and correlation properties on the order of seconds [46]. 
    
   The above estimation procedures apply equally to traffic workload, 
   traffic performance, or other estimates of network state, such as the 
   state of routes. 
    
APPENDIX D 
    
D. Read-Out Periods 
    
   A measurement infrastructure must be able to scale with the size and 
   the speed of a network as it evolves.  Hence, it is important to 
   minimize the amount of data to be collected, and to condense the 
   collected data by periodic summarization over read-out periods. 
    
  
Lai, et al              Category - Expiration               [Page 20] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
D.1 Data Reduction 
    
   Techniques to manage large volumes of measured data are needed to 
   prevent network performance from being adversely affected by the 
   unnecessarily excessive loading of router control processors, router 
   memories, transmission facilities, and the administrative support 
   systems.   
    
   For example, offline bulk file transfer may be used as a method to 
   manage large volumes of measured traffic data.  Bulk transfer from 
   routers to collection devices can help reduce the packet processing 
   overhead experienced by using other management interfaces.  Also, 
   data correlation or filtering rules may be set up to suppress 
   redundant data, or to aggregate flows into suitable classes with the 
   corresponding aggregation of statistics.  These types of data 
   reduction may be used as an appropriate or acceptable means for 
   pruning down the overall volume of traffic data that a TE system may 
   ultimately have to store, maintain, and process. 
    
D.2 Measurement Interval 
    
   A measurement interval is the time interval over which measurements 
   are taken.  Some traffic data must be collected continuously, while 
   others by sampling, or on a scheduled basis.  For example, peak 
   loads and peak periods can be identified only by continuous 
   measurement as traffic typically fluctuates irregularly during the 
   whole day.  If traffic variations are regular and predictable, it 
   may be possible to measure the expected normal load on pre-
   determined portions of the day.  Such duration of peak traffic is 
   referred to as a busy period.  Special studies on selected segments 
   of the network may be conducted on a scheduled basis.  Occasionally 
   unexpected events or other decision support needs may arise that 
   require ad-hoc, unscheduled measurement, with the involvement of the 
   network operator, and in such a case measurements may be activated 
   manually.  For instance, active throughput measurement may be used 
   to identify alternate routes for spreading traffic to avoid future 
   periods of network congestion, based on observations of current 
   local congestion events. 
    
D.3 Summarization 
    
   A measurement interval consists of a sequence of consecutive read-
   out periods.  Summarization is usually done by integrating the raw 
   data over a pre-specified read-out period.  The granularity of this 
   period must be suitably chosen.  It should be short enough to 
   capture, with acceptable accuracy, the bursty nature of the traffic, 
   i.e., the traffic variations and peaks.  Since measurements 
   represent a load for the router (if third-party measurement devices 
   are not employed), the read-out period should not be so short that 
   router performance is degraded while a voluminous quantity of data 
   is produced.   Also, read-out may be started when the measured data 
   exceeds a preset threshold, or when the space allocated for 
   temporarily holding the data in a router is exhausted. 
  
Lai, et al              Category - Expiration               [Page 21] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
    
   For a multi-service IP network, each service typically has its own 
   traffic characteristics and performance objectives.  To ensure that 
   CoS-specific features are reflected in the measurement process, 
   different read-out periods may be needed for different classes of 
   service. 
    
APPENDIX E 
    
E. Time Scales for Network Operations 
    
   The information collected by traffic measurement can be provided to 
   the end user or application either in real time, or for record 
   (i.e., data retention) in non-real time, depending on the activities 
   to be performed and the network actions to be taken.  Traffic 
   control will generally require real-time information.  For network 
   planning and capacity management as described below, information may 
   be provided in non-real time after the processing of raw data. 
    
   Broadly speaking, the following three time scales can be classified, 
   according to the use of observed traffic information for network 
   operations [14]. 
    
   Network planning 
   Information that changes on the order of months is used to make 
   traffic forecasts as a basis for network extensions and long-term 
   network configuration.  That is, for planning the topology of the 
   network, planning alternative routes to survive failures or 
   determining where capacity must be augmented in advance of projected 
   traffic growth.  Long-term planning includes the selection and 
   timing of the introduction of new architectures, technologies and 
   vendors, in alignment with financial forecasts and market 
   assessments. 
    
   Capacity management 
   Intermediate-scale (e.g., six months or less) capacity planning 
   deals with detailed implementation of the build plan, short-lead-
   time activities and out-of-plan events.  It typically uses a 
   rolling-month forecast of traffic and demand.  Information that 
   changes on the order of days or hours is used to manage the deployed 
   facilities, by taking appropriate maintenance or engineering actions 
   to optimize utilization.  For example, new MPLS paths may be set up 
   or existing paths modified while meeting service level agreements.  
   Also, load balancing may be performed, or traffic may be rerouted 
   for re-optimization after a failure. 
    
   Real-time network control 
   Information that changes on the order of minutes or less is used to 
   adapt to the current network conditions in near real time.  Thus, to 
   combat localized congestion, traffic management actions may perform 
   temporary rerouting to redistribute the load.  Upon detecting a 
   failure, traffic may be diverted to pre-established, secondary 
   routes until more optimized routes can be arranged. 
  
Lai, et al              Category - Expiration               [Page 22] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
    
APPENDIX F 
    
F. Use of Traffic Measurement for Traffic control 
    
   Destination-based per-hop IP routing and forwarding provides a 
   network operator with primitive and limited control over the routing 
   of traffic flows.  The routing control offered by MPLS can be used 
   to avoid some of the deficiencies of IP routing.  In this context, a 
   primary use of traffic measurement is to engineer the use of label-
   switched paths to achieve service goals for the network. 
    
   Examples of traffic control are: 
   . Adaptively optimizing network performance in response to network 
     events, e.g., rerouting to work around congestion or failures. 
   . Providing a feedback mechanism in the reverse flow messaging of 
     RSVP-TE [47] or CR-LDP [48] signaling in MPLS to report on actual 
     topology state information such as link bandwidth availability.  
     (An example of such a feedback mechanism is described in [49]; 
     however, care should be exercised to ensure network stability and 
     consistency for any mechanism that makes direct operational use of 
     measurement.) 
   . Support of measurement-based admission control, adaptive resource 
     management, or other techniques, e.g., by predicting the future 
     demands of the aggregate of existing flows so that admission 
     decisions can be made on new flows. 
    
   An example of TE measurements used to enable a traffic control 
   mechanism is to configure policing mechanisms in response to traffic 
   load and performance measurements.  A network operator could 
   selectively throttle low-priority flows to improve near-real-time 
   performance of higher-priority flows, and maintain tighter QoS 
   envelopes.  Another example would be to use measurement results for 
   feedback into IGP routing decisions, e.g., for adjusting the link 
   weights based on them. 
    
16. Security Considerations 
    
   The principles and concepts related to Internet traffic measurement 
   as discussed in this document do not by themselves affect the 
   security of the Internet.  However, it is assumed that any 
   measurement systems that are developed or deployed by a service 
   provider are responsible for providing sufficient data integrity 
   (e.g., to prevent forgery of measurement records) and 
   confidentiality (e.g., by restricting attention only to the packet 
   headers of interest).  It is also assumed that a service provider 
   will take proper precautions to ensure that access to its 
   measurement systems and all associated data is secure by using 
   appropriate authentication techniques.  Methods to achieve these 
   security considerations are not addressed in this document. 
    
17. References 
    
  
Lai, et al              Category - Expiration               [Page 23] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   Normative References 
    
   References 1, 2, and 34 below are considered normative. 
    
   Informative References 
 
   1  D.O. Awduche, A. Chiu, A. Elwalid, I. Widjaja, and X. Xiao, 
      "Overview and Principles of Internet Traffic Engineering," RFC 
      3272, May 2002. 
   2  V. Paxson, G. Almes, J. Mahdavi, and M. Mathis, "Framework for IP 
      Performance Metrics," RFC 2330, May 1998. 
   3  J. Mahdavi and V. Paxson, "IPPM Metrics for Measuring 
      Connectivity," RFC 2678, September 1999. 
   4  G. Almes, S. Kalidindi, and M. Zekauskas, "A One-way Delay Metric 
      for IPPM," RFC 2679, September 1999. 
   5  G. Almes, S. Kalidindi, and M. Zekauskas, "A One-way Packet Loss 
      Metric for IPPM," RFC 2680, September 1999. 
   6  G. Almes, S. Kalidindi, and M. Zekauskas, "A Round-trip Delay 
      Metric for IPPM," RFC 2681, September 1999. 
   7  M. Mathis and M. Allman, "A Framework for Defining Empirical Bulk 
      Transfer Capacity Metrics," RFC 3148, July 2001. 
   8  R. Koodli and R. Ravikanth, "One-way Loss Pattern Sample 
      Metrics," RFC 3357, August 2002. 
   9  C. Demichelis and P. Chimento, "IP Packet Delay Variation Metric 
      for IP Performance Metrics (IPPM)," RFC 3393, November 2002. 
   10 V. Raisanen, G. Grotefeld, and A. Morton, "Network performance 
      measurement with periodic streams," RFC 3432, November 2002. 
   11 H. Uijterwaal and M. Kaeo, "One-way Metric Applicability 
      Statement," Internet-Draft, Work in Progress, November 2002. 
   12 ITU-T Recommendation I.380/Y.1540, "Internet Protocol Data 
      Communication Service -- IP Packet Transfer and Availability 
      Performance Parameters," First Issued February 1999, Revised 
      December 2002. 
   13 ITU-T Recommendation Y.1541, "Network Performance Objectives for 
      IP-Based Services," May 2002. 
   14 G. Ash, "Traffic Engineering & QoS Methods for IP-, ATM-, & TDM-
      Based Multiservice Networks," Internet-Draft, Work in Progress, 
      October 2001. 
   15 A. Feldmann, A. Greenberg, C. Lund, N. Reingold, J. Rexford, and 
      F. True, "Deriving Traffic Demands for Operational IP Networks: 
      Methodology and Experience," Proc. ACM SIGCOMM 2000, Stockholm, 
      Swedan. 
   16 A. Feldmann, A. Greenberg, C. Lund, N. Reingold, and J. Rexford, 
      "NetScope: Traffic Engineering for IP Networks," IEEE Network, 
      March/April 2000. 
   17 T.D. Nadeau, M. Morrow, G. Swallow, and D. Allan, " OAM 
      Requirements for MPLS Networks," Internet-Draft, Work in 
      Progress. 
   18 N. Harrison, P. Willis, S. Davari, E. Cuevas, B. Mack-Crane, E. 
      Franze, H. Ohta, T. So, S. Goldfless, and F. Chen, "Requirements 
      for OAM in MPLS Networks," Internet-Draft, Work in Progress, May 
      2001. 
 

  
Lai, et al              Category - Expiration               [Page 24] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
 
   19 ITU-T Draft Recommendation Y.1710, "Requirements for OAM 
      Functionality for MPLS Networks," May 2001. 
   20 ITU-T Draft Recommendation Y.1711, "OAM Mechanisms for MPLS 
      Networks," May 2001. 
   21 S. Waldbusser, R.G. Cole, C. Kalbfleisch, and D. Romascanu, "An 
      Introduction to the RMON Family of MIB Modules," Internet-Draft, 
      Work in Progress, Jan 2003. 
   22 C. Kalbfleisch, R.G. Cole, and D. Romascanu, "Definition of 
      Managed Objects for Synthetic Sources for Performance Monitoring 
      Algorithms," Internet-Draft, Work in Progress, Sept 2002. 
   23 T.D. Nadeau, C. Srinivasan, and A. Viswanathan, "Multiprotocol 
      Label Switching (MPLS) FEC-To-NHLFE (FTN) Management Information 
      Base," Internet-Draft, Work in Progress, January 2002. 
   24 R. Bonica, K. Kompella, and D. Meyer, "Tracing Requirements for 
      Generic Tunnels," Internet-Draft, Work in Progress, August 2002. 
   25 K. Kompella, P. Pan, N. Sheth, D. Cooper, G. Swallow, S. Wadhwa, 
      and R. Bonica, "Detecting MPLS Data Plane Liveness," Internet-
      Draft, Work in Progress, October 2002. 
   26 C. Srinivasan, A. Viswanathan, and T.D. Nadeau, "Multiprotocol 
      Label Switching (MPLS) Label Switch Router (LSR) Management 
      Information Base," Internet-Draft, Work in Progress, January 
      2002. 
   27 C. Srinivasan, A. Viswanathan, and T.D. Nadeau, "Multiprotocol 
      Label Switching (MPLS) Traffic Engineering Management Information 
      Base," Internet-Draft, Work in Progress, January 2002. 
   28 K. Kompella, "A Traffic Engineering MIB," Internet-Draft, Work in 
      Progress, September 2002. 
   29 R. Yavatkar, D. Pendarakis, and R. Guerin, "A Framework for 
      Policy-based Admission Control," RFC 2753, January 2000. 
   30 D. Rawlins, A. Kulkarni, M. Bokaemper, and K.H. Chan, "Framework 
      for Policy Usage Feedback for Common Open Policy Service with 
      Policy Provisioning (COPS-PR)," Internet-Draft, Work in Progress, 
      December 2002. 
   31 C. Jacquenet, "An IP Forwarding Policy Information Base," 
      Internet-Draft, Work in Progress, January 2003. 
   32 C. Jacquenet, "A COPS client-type for IP traffic engineering," 
      Internet-Draft, Work in Progress, January 2003. 
   33 S. Sen and J. Wang, "Analyzing Peer-to-Peer traffic Across Large 
      Networks," Internnet Measurement Workshop, 2002. 
   34 E. Rosen, A. Viswanathan, and R. Callon, "Multiprotocol Label 
      Switching Architecture," RFC 3031, January 2001. 
   35 S. Bradner (Editor), "Benchmarking Terminology for Network 
      Interconnection Devices," RFC 1242, July 1991. 
   36 K. Kompella, Y. Rekhter, and L. Berger, "Link Bundling in MPLS 
      Traffic Engineering," Internet-Draft, Work in Progress, February 
      2001. 
   37 W.S. Lai, "Traffic Measurement for Dimensioning and Control of IP 
      Networks," Internet Performance and Control of Network Systems II 
      Conference, SPIE Proceedings, Vol. 4523, Denver, Colorado, 21-22 
      August 2001, pp. 359-367. 
 


  
Lai, et al              Category - Expiration               [Page 25] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
 
   38 M. Jain and C. Dovrolis, "End-to-End Available Bandwidth: 
      Measurement Methodology, Dynamics, and Relation with TCP 
      Throughput," Proc. ACM SIGCOMM'2002, August 19-23, 2002, 
      Pittsburgh, Pennsylvania. 
   39 N.G. Duffield (Editor), "A Framework for Passive Packet 
      Measurement," Internet-Draft, Work in Progress, September 2002. 
   40 N.G. Duffield and M. Grossglauser, "Trajectory Sampling for 
      Direct Traffic Observation," IEEE/ACM Trans. on Networking, 9(3), 
      pp. 280-292, June 2001. 
   41 C. Partridge, C. Jones, D. Waitzman, and A. Snoeren, "New 
      Protocols to Support Internet Traceback," Internet-Draft, Work in 
      Progress, November 2001. 
   42 S. Haykin, Ed., "Kalman Filtering and Neural Networks," Wiley 
      Interscience, 2001. 
   43 A. Papoulis, "Probability, Random Variables and Stochastic 
      Processes," 3rd Ed., McGraw-Hill, 1991. 
   44 A. Gelb, Ed., "Applied Optimal Estimation," MIT Press, 1974. 
   45 I. R. Petersen, V. A. Ugrinovskii, A. V. Savkin, "Robust Control 
      Design Using H<\infinity> Methods," Springer, 2000. 
   46 V. Bolotin, J. Coombs-Reyes, D. Heyman, Y. Levy, and D. Liu, "IP 
      Traffic Characterization for Planning and Control," Proc. ITC16, 
      Edinburgh, Scotland, June 1999. 
   47 D. Awduche, L. Berger, D. Gan, T. Li, V. Srinivasan, and G. 
      Swallow, "RSVP-TE: Extensions to RSVP for LSP Tunnels," RFC 3209, 
      December 2001. 
   48 B. Jamoussi (Editor), "Constraint-Based LSP Setup using LDP," RFC 
      3212, January 2002. 
   49 P. Ashwood-Smith, B. Jamoussi, D. Fedyk, and D. Skalecki, 
      "Improving Topology Data Base Accuracy with Label Switched Path 
      Feedback in Constraint Based Label Distribution Protocol," 
      Internet-Draft, Work in Progress, November 2002. 
    
18. Intellectual Property Statement 
    
   AT&T Corp. may own intellectual property applicable to packet 
   sampling as presented in references [39, 40] and summarized in 
   Appendix C.1.  AT&T is currently reviewing its licensing intent 
   relative to the Intellectual Property and will notify the IETF when 
   AT&T has made a determination of that intent. 
    
19. Acknowledgments 
    
   Thanks to the inputs from Gerald Ash, Jim Boyle, Robert Cole, 
   Enrique Cuevas, Ruediger Geib, Christian Jacquenet, Merike Kaeo, Ed 
   Kern, Spyros Kontogiorgis, Alfred Morton, Thomas Nadeau, Dimitri 
   Papadimitriou, Moshe Segal, Jing Shen, Bert Wijnen, Raymond Zhang, 
   and the Scampi and Tequila projects.  Special thanks to Blaine 
   Christian for starting this work and contributing to the initial 
   versions.  Nick Duffield provided Appendix C.1 on packet sampling. 
    
20. Author's Addresses 
    
  
Lai, et al              Category - Expiration               [Page 26] 
Internet-Draft  Framework for Internet Traffic Measurement   July 2003 
 
 
   Wai Sum Lai 
   AT&T Labs 
   Room D5-3D18 
   200 Laurel Avenue 
   Middletown, NJ 07748, USA 
   Phone: +1 732-420-3712 
   Email: wlai@att.com 
    
   Richard W. Tibbs 
   Oak City Networks & Solutions 
   304 Harvey St. 
   Radford, VA 24141, USA 
   Phone: +1 540 639 2145 
   Email: drtibbs@oakcitysolutions.com 
    
   Steven Van den Berghe 
   Ghent University/IMEC 
   St. Pietersnieuwsstraat 41 
   B-9000 Ghent, Belgium 
   Phone: ++32 9 264 99 86 
   E-mail: steven.vandenberghe@intec.ugent.be 
    
    
Full Copyright Statement 
 

   "Copyright (C) The Internet Society (date). All Rights Reserved. 
   This document and translations of it may be copied and furnished to 
   others, and derivative works that comment on or otherwise explain it 
   or assist in its implementation may be prepared, copied, published 
   and distributed, in whole or in part, without restriction of any 
   kind, provided that the above copyright notice and this paragraph 
   are included on all such copies and derivative works. However, this 
   document itself may not be modified in any way, such as by removing 
   the copyright notice or references to the Internet Society or other 
   Internet organizations, except as needed for the purpose of 
   developing Internet standards in which case the procedures for 
   copyrights defined in the Internet Standards process must be 
   followed, or as required to translate it into languages other than 
   English. 
    
   The limited permissions granted above are perpetual and will not be 
   revoked by the Internet Society or its successors or assigns. 
    
   This document and the information contained herein is provided on an 
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING 
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING 
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION 
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 





  
Lai, et al              Category - Expiration               [Page 27] 

PAFTECH AB 2003-20262026-04-22 19:14:03