One document matched: draft-morton-bmwg-virtual-net-00.xml

<?xml version="1.0" encoding="US-ASCII"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd">
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="3"?>
<?rfc tocindent="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc comments="yes"?>
<?rfc inline="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?>
<rfc category="info" docName="draft-morton-bmwg-virtual-net-00"
     ipr="trust200902">
  <front>
    <title abbrev="Round-trip Loss">Considerations for Benchmarking Virtual
    Network Functions and Their Infrastructure</title>

    <author fullname="Al Morton" initials="A." surname="Morton">
      <organization>AT&T Labs</organization>

      <address>
        <postal>
          <street>200 Laurel Avenue South</street>

          <city>Middletown,</city>

          <region>NJ</region>

          <code>07748</code>

          <country>USA</country>
        </postal>

        <phone>+1 732 420 1571</phone>

        <facsimile>+1 732 368 1192</facsimile>

        <email>acmorton@att.com</email>

        <uri>http://home.comcast.net/~acmacm/</uri>
      </address>
    </author>

    <date day="14" month="February" year="2014"/>

    <abstract>
      <t>BMWG has traditionally conducted laboratory characterization of
      dedicated physical implementations of internetworking functions. This
      memo investigates additional considerations when network functions are
      virtualized and performed in commodity off-the-shelf hardware.</t>
    </abstract>

    <note title="Requirements Language">
      <t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
      "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
      document are to be interpreted as described in <xref
      target="RFC2119">RFC 2119</xref>.</t>

      <t/>
    </note>
  </front>

  <middle>
    <section title="Introduction">
      <t>BMWG has traditionally conducted laboratory characterization of
      dedicated physical implementations of internetworking functions. The
      Black-box Benchmarks of Throughput, Latency, Forwarding Rates and others
      have served our industry for many years. <xref target="RFC1242"/> and
      <xref target="RFC2544"/> are the cornerstones of the work.</t>

      <t>A set of development goals is to reduce costs while increasing
      flexibility of network devices, and drastically accelerate their
      deployment. Network Function Virtualization has the promise to achieve
      these goals, and therefore has garnered much attention. It now seems
      certain that some network functions will be virtualized following the
      success of cloud computing and virtual desktops supported by sufficient
      network path capacity, performance,and widespread deployment; many of
      the same techniques will be brought to bear.</t>

      <t>See http://www.etsi.org/technologies-clusters/technologies/nfv for
      more background, for example, the white papers there may be a useful
      starting place.</t>
    </section>

    <section title="Scope">
      <t>This memo investigates additional methodological considerations
      necessary when benchmarking Virtual Network Functions (VNF) instantiated
      and hosted in commodity off-the-shelf hardware (COTS).</t>

      <t>A clearly related goal: the benchmarks for the capacity of COTS to
      host a plurality of VNF instances should be investigated.</t>

      <t>A non-goal is any overlap with traditional computer benchmark
      development and their specific metrics (SPECmark suites such as
      SPECCPU).</t>
    </section>

    <section title="New Considerations">
      <t>This section lists the new considerations which must be addressed to
      benchmark VNF(s) and their supporting infrastructure.</t>

      <section title="Hardware Components">
        <t>New Hardware devices will become part of the test set-up.<list
            style="numbers">
            <t>High volume server platforms (COTS, possibly with virtual
            technology enhancements).</t>

            <t>Large capacity, and high speed, high reliability storage
            systems.</t>

            <t>Network Interface ports specially designed for efficient
            service of many virtual NICs.</t>

            <t>High capacity Ethernet Switches.</t>
          </list>Labs conducting comparisons of different VNFs may be able to
        use the same hardware platform over many studies, until the steady
        march of innovations overtakes their capabilities (as happens with the
        lab's traffic generation and testing devices today). </t>
      </section>

      <section title="Configuration Parameters">
        <t>It will be necessary to configure and document the settings for the
        entire COTS platform, including:</t>

        <t><list style="symbols">
            <t>number of server blades (shelf occupation)</t>

            <t>CPUs</t>

            <t>caches</t>

            <t>storage system</t>

            <t>I/O</t>
          </list>as well as configurations that support the devices which host
        the VNF itself: <list style="symbols">
            <t>Hypervisor</t>

            <t>Virtual Machine</t>

            <t>Infrastructure Virtual Network</t>
          </list>and finally, the VNF itself, with items such as:<list
            style="symbols">
            <t>specific function being implemented in VNF</t>

            <t>number of VNF components in the service function chain</t>

            <t>number of physical interfaces and links transited in the
            service function chain</t>
          </list></t>

        <t/>
      </section>

      <section title="Testing Strategies">
        <t>The concept of characterizing performance at capacity limits may
        change. For example:</t>

        <t><list style="numbers">
            <t>It may be more representative of system capacity to
            characterize the case where Virtual Machines (VM, hosting the VNF)
            are operating at 50% Utilization, and therefore sharing the "real"
            processing power across many VMs.</t>

            <t>Another important case stems from the need for partitioning
            functions. A noisy neighbor (VM hosting a VNF in an infinite loop)
            would ideally be isolated and the performance of other VMs would
            continue according to their specifications. </t>

            <t>System errors will likely occur as transients, implying a
            distribution of performance characteristics with a long tail (like
            latency), leading to the need for longer-term tests of each set of
            configuration and test parameters.</t>

            <t>The desire for Elasticity and flexibility among network
            functions will include tests where there is constant flux in the
            VM instances. Requests for new VMs and Releases for VMs hosting
            VNFs no longer needed would be an normal operational condition.
            </t>

            <t>All physical things can fail, and benchmarking efforts can also
            examine recovery aided by the virtual architecture with different
            approaches to resiliency.</t>
          </list></t>
      </section>
    </section>

    <section title="Security Considerations">
      <t>Benchmarking activities as described in this memo are limited to
      technology characterization using controlled stimuli in a laboratory
      environment, with dedicated address space and the constraints specified
      in the sections above. </t>

      <t>The benchmarking network topology will be an independent test setup
      and MUST NOT be connected to devices that may forward the test traffic
      into a production network, or misroute traffic to the test management
      network. </t>

      <t>Further, benchmarking is performed on a "black-box" basis, relying
      solely on measurements observable external to the DUT/SUT. </t>

      <t>Special capabilities SHOULD NOT exist in the DUT/SUT specifically for
      benchmarking purposes. Any implications for network security arising
      from the DUT/SUT SHOULD be identical in the lab and in production
      networks. </t>
    </section>

    <section anchor="IANA" title="IANA Considerations">
      <t>No IANA Action is requested at this time.</t>
    </section>

    <section title="Acknowledgements">
      <t>The author acknowledges an encouraging conversation on this topic
      with Mukhtiar Shaikh and Ramki Krishnan in November 2013.</t>
    </section>
  </middle>

  <back>
    <references title="Normative References">
      <?rfc include="reference.RFC.1242"?>

      <?rfc include="reference.RFC.2119"?>

      <?rfc include="reference.RFC.2330"?>

      <?rfc include='reference.RFC.2544'?>

      <?rfc include="reference.RFC.2679"?>

      <?rfc include='reference.RFC.2680'?>

      <?rfc include='reference.RFC.3393'?>

      <?rfc include='reference.RFC.3432'?>

      <?rfc include='reference.RFC.2681'?>

      <?rfc include='reference.RFC.5905'?>

      <?rfc include='reference.RFC.4737'?>

      <?rfc include='reference.RFC.5357'?>
    </references>

    <references title="Informative References">
      <?rfc include='reference.RFC.1242'?>

      <?rfc include='reference.RFC.5481'?>

      <?rfc include='reference.RFC.6248'?>

      <?rfc include='reference.RFC.6390'?>
    </references>
  </back>
</rfc>