One document matched: draft-ietf-rddp-security-00.txt
Internet Draft James Pinkerton
Document: draft-ietf-rddp-security-00.txt Microsoft Corporation
Expires: April, 2004 Ellen Deleganes
Intel Corporation
Allyn Romanow
Cisco Systems
Bernard Aboba
Microsoft Corporation
October 2003
DDP/RDMAP Security
1 Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
2 Abstract
This document analyzes security issues around implementation and use
of the Direct Data Placement Protocol(DDP) and Remote Direct Memory
Access Protocol (RDMAP). It first defines an architectural model for
an RDMA Network Interface Card (RNIC), which can implement DDP or
RDMAP and DDP. The model includes a definition of resources that can
be attacked. This document then introduces various Trust Models
between a local peer and a remote peer and the tools that can be
used to create countermeasures against attacks. Finally, the
document reviews various attacks and the countermeasures to be used
against them, grouping the attacks into spoofing, tampering,
information disclosure, denial of service, and elevation of
privilege.
J. Pinkerton, et al. Expires April 2004 1
Internet-Draft RDDP/RDMAP Security October 2003
Table of Contents
1 Status of this Memo.........................................1
2 Abstract....................................................1
2.1 Issues......................................................3
3 Introduction................................................5
4 Architectural Model.........................................7
4.1 Components..................................................8
4.2 Resources..................................................10
4.2.1 Connection Context Memory.................................10
4.2.2 Data Buffers..............................................10
4.2.3 Page Translation Tables...................................11
4.2.4 STag Namespace............................................11
4.2.5 Completion Queues.........................................11
4.2.6 RDMA Read Request Queue...................................11
4.2.7 RDMA Asynchronous Event Queue.............................11
4.2.8 RNIC Control Interactions.................................12
4.2.9 Initialization of RNIC Data Structures for Data Transfer..12
4.2.10 RNIC Data Transfer Interactions.........................13
4.3 System Properties..........................................14
5 Trust Models...............................................15
6 Attacker Capabilities......................................17
7 Attacks and Countermeasures................................18
7.1 Tools for Countermeasures..................................18
7.1.1 Protection Domain (PD)....................................18
7.1.2 Limiting STag Scope.......................................19
7.1.3 Access Rights.............................................19
7.1.4 Limiting the Scope of the Completion Queue................20
7.1.5 Limiting the Scope of an Error............................20
7.2 Spoofing...................................................20
7.2.1 Connection Hijacking......................................20
7.2.2 Using an STag on a different connection...................21
7.3 Tampering..................................................22
7.3.1 RDMA Write or Read Response to Memory Outside the Buffer..22
7.3.2 Modifying a Buffer After Indicating Contents Are Ready....23
7.3.3 Using Multiple Stags to access the same buffer............23
7.4 Information Disclosure.....................................23
7.4.1 Probing memory outside of the buffer bounds...............24
7.4.2 Using RDMA Read to Access Stale Data......................24
7.4.3 Accessing a buffer after the transfer is over.............24
7.4.4 Accessing data within a valid STag that was unintended....24
7.4.5 Using RDMA Read on a buffer meant only for RDMA Write.....25
7.4.6 Using Multiple Stags to access the same buffer............25
7.4.7 Remote node loading firmware onto the RNIC................26
7.4.8 Controlling Access to Page Translation Table and STag Mapping
26
7.5 Denial of Service (DOS)....................................26
7.5.1 RNIC Resource Consumption.................................27
7.5.2 Resource Consumption By Active Applications...............27
7.5.2.1 Receive Data Buffers..................................28
J. Pinkerton, et al. Expires - April 2004 [Page 2]
Internet-Draft RDDP/RDMAP Security October 2003
7.5.2.2 Completion Queue (CQ) Resource Consumption............29
7.5.2.3 RDMA Read Request Queue...............................31
7.5.3 Resource Consumption by Idle Applications.................32
7.5.4 Exercise of non-optimal code paths........................32
7.5.5 Remote Invalidation of an STag Shared Across Multiple
Connections......................................................32
7.5.6 Remote Peer Consumes too many Untagged Receive Buffers....33
7.6 Elevation of Privilege.....................................33
7.6.1 Loading Firmware into the RNIC............................34
8 Security Services for RDDP.................................35
8.1 Introduction to Security Options...........................35
8.1.1 Introduction to IPsec.....................................35
8.1.2 Introduction to SSL Limitations on RDMAP..................36
8.1.3 Applications Which Provide Security.......................36
8.1.4 Authentication Only.......................................36
8.1.5 Privacy...................................................36
8.2 Recommendations for IPsec Encapsulation of RDDP............36
9 Summary Table of Attacks and Trust Models..................38
10 References.................................................40
10.1 Normative References......................................40
10.2 Informative References....................................40
11 AuthorÆs Addresses.........................................41
12 Acknowledgments............................................42
13 Full Copyright Statement...................................43
Table of Figures
Figure 1 - RDMA Security Model....................................8
Figure 2 - Summary Attacks and Trust Model Table.................39
2.1 Issues
This section is temporary and will go away when all issues have been
resolved.
Note: this is far from a complete list of issues; as more are
raised, they will be added to this list until some sort of consensus
is reached. They are in the order found in the specification.
Issue: Discuss issues in allowing the non-privileged consumer to
control mapping the Page Translation Table and Stag..............26
Issue: Need to analyze the case of sharing a queue of Untagged
receive buffers across multiple connections, and that the Remote
Peer can mount a denial of service attack........................33
Issue: Security Services section is a placeholder for now........35
Issue: Guidance for application protocols like NFS which implement
security.........................................................36
J. Pinkerton, et al. Expires - April 2004 [Page 3]
Internet-Draft RDDP/RDMAP Security October 2003
Issue: IPsec recommendations for RDMAP/DDP.......................36
Issue: Finish Summary table of Attacks/Trust Models..............38
J. Pinkerton, et al. Expires - April 2004 [Page 4]
Internet-Draft RDDP/RDMAP Security October 2003
3 Introduction
RDMA enables new levels of flexibility when communicating between
two parties compared to current conventional networking practice
(e.g. a stream-based model or datagram model). This flexibility
brings new security issues that must be carefully understood when
designing application protocols utilizing RDMA and when implementing
RDMA-aware NICs (RNICs). Note that for the purposes of this security
analysis, an RNIC may implement RDMAP and DDP, or just DDP.
The specification first develops an architectural model that is
relevant for the security analysis - it details components,
resources, and system properties that may be attacked. The
specification then defines Partial Trust Models. Partial Trust is
defined as:
Partial Trust - When one party is willing to assume that
another party will refrain from a specific attack or set of
attacks, the parties are said to be in a state of Partial
Trust. Note that the partially trusted peer may attempt a
different set of attacks. This may be appropriate for many
applications where any adverse effects of the betrayal is
easily confined and does not place other clients or
applications at risk.
An Untrusted Peer relationship is appropriate when an application
wishes to ensure that it will be robust and uncompromised even in
the face of a deliberate attack by its peer. For example, a single
application that concurrently supports multiple unrelated sessions
would presumably treat each of its peers as an Untrusted Peer.
For the Untrusted peer, a brief list of capabilities is enumerated.
The rest of the specification is focused on analyzing attacks.
First, the tools for mitigating attacks are listed, and then a
series of attacks on components, resources, or system properties is
enumerated. For each attack, possible countermeasures are reviewed.
Applications within a host are divided into two categories -
Privileged and Non-Privileged. Both application types can send and
receive data and request resources. The key differences between the
two are:
The Privileged Application is Partially Trusted. It is assumed
that the Privileged Application will not intentionally attack
the system (e.g., it is a kernel application), although it may
be greedy for resources.
A Non-Privileged ApplicationÆs capabilities are a logical sub-
set of the Privileged ApplicationÆs. It is assumed by the local
host infrastructure that a Non-Privileged Application is
J. Pinkerton, et al. Expires - April 2004 [Page 5]
Internet-Draft RDDP/RDMAP Security October 2003
Untrusted. All Non-Privileged Application interactions with the
RNIC Engine that could affect other applications need to be
done through a Trusted intermediary that can verify the Non-
Privileged Application requests.
J. Pinkerton, et al. Expires - April 2004 [Page 6]
Internet-Draft RDDP/RDMAP Security October 2003
4 Architectural Model
This section describes an RDMA architectural reference model that is
used as security issues are examined. It introduces the components
of the model, the resources that can be attacked, the types of
interactions possible between components and resources, and the
system properties, which should be preserved when under attack.
Figure 1 shows the components comprising the architecture and the
interfaces where potential security attacks could be launched.
External attacks can be injected into the system from an application
that sits above the RI or from the Internet.
The intent here is to describe high level components and
capabilities which affect threat analysis, and not focus on specific
implementation options. Also note that the architectural model is an
abstraction, and an actual implementation may choose to subdivide
its components along different boundary lines than defined here. For
example, the Privileged Resource Manager may be partially or
completely encapsulated in the Privileged Application. Regardless,
it is expected that the security analysis of the potential threats
and countermeasures still apply.
J. Pinkerton, et al. Expires - April 2004 [Page 7]
Internet-Draft RDDP/RDMAP Security October 2003
+-------------+ Request Proxy Interface
| Privileged |<--------------------------------+
| Resource | |
Admin<-->| Manager | App Control Interface |
| |<------+-------------------+ |
+-------------+ | | |
^ v v v
| +-------------+ +-----------------+
| | Privileged | | Non-Privileged |
| | Application | | Application |
| +-------------+ +-----------------+
| ^ ^
|Privileged |Privileged |Non-Privileged
|Control |Data |Data
|Interface |Interface |Interface
RNIC | | |
Interface(RI) v v v
=================================================================
+-----------------------------------------+
| |
| RNIC Engine | <------ Firmware
| |
+-----------------------------------------+
^
|
v
Internet
Figure 1 - RDMA Security Model
4.1 Components
The components shown in Figure 1 - RDMA Security Model are:
* RNIC Engine - the component that implements the RDMA
protocol and/or DDP protocol.
* Privileged Resource Manager - the component responsible for
managing and allocating resources associated with the RNIC
Engine. The Resource Manager does not send or receive data.
Note that whether the Resource Manager is an independent
component, part of the RNIC, or part of the application is
implementation dependent. If a specific implementation does
not wish to address security issues resolved by the Resource
Manager, there may in fact be no resource manager at all.
* Privileged Application - See Section 3 Introduction for a
definition of Privileged Application. The local host
J. Pinkerton, et al. Expires - April 2004 [Page 8]
Internet-Draft RDDP/RDMAP Security October 2003
infrastructure can enable the Privileged Application to map
a data buffer directly from the RNIC Engine to the host
through the RNIC Interface, but it does not allow the
Privileged Application to directly consume RNIC Engine
resources.
* Non-Privileged Application - See Section 3 Introduction for
a definition of Non-Privileged Application. All Non-
Privileged Application interactions with the RNIC Engine
that could affect other applications MUST be done using the
Privileged Resource Manager as a proxy.
A design goal of the DDP and RDMAP protocols is to allow, under
constrained conditions, Non-Privileged applications to send and
receive data directly to/from the RDMA Engine without Privileged
Resource Manager intervention - while ensuring that the host remains
secure. Thus, one of the primary goals of this paper is to analyze
this usage model for the enforcement that is required in the RNIC
Engine to ensure the system remains secure.
The host interfaces that could be exercised include:
* Control Interface - A Privileged Resource Manager uses the
RI to allocate and manage RNIC Engine resources, control the
state within the RNIC Engine, and monitor various events
from the RNIC Engine. It also uses this interface to act as
a proxy for some operations that a Non-Privileged
Application may require (after performing appropriate
countermeasures).
* Non-Privileged Data Transfer Interface - A Non-Privileged
Application uses this interface to initiate and to check the
status of data transfer operations.
* Privileged Data Transfer Interface - A superset of the
functionality provided by the Non-Privileged Data Transfer
Interface. The application is allowed to directly manipulate
RNIC Engine mapping resources to map an STag to an
application data buffer.
* Request Proxy Interface - a Non-Privileged Application uses
this interface to control RNIC Engine resources that could
affect other applications - such as manipulating the RNIC
Engine's mapping of an STag to an application data buffer.
The Privileged Resource Manager implements countermeasures
to ensure that if the Non-Privileged Application launches an
attack it can prevent the attack from affecting other
applications.
J. Pinkerton, et al. Expires - April 2004 [Page 9]
Internet-Draft RDDP/RDMAP Security October 2003
* Figure 1 also shows the ability to load new firmware in the
RNIC Engine. Not all RNICs will support this, but it is
shown for completeness and is also reviewed under potential
attacks.
If Internet control messages, such as ICMP, ARP, RIPv4, etc. are
processed by the RNIC Engine, the threat analyses for those
protocols is also applicable, but outside the scope of this paper.
4.2 Resources
This section describes the primary resources in the RNIC Engine that
could be affected if under attack. For RDMAP, all of the defined
resources apply. For DDP, all of the resources except the RDMA Read
Queue apply.
4.2.1 Connection Context Memory
The state information for each connection is maintained in memory,
which could be located in a number of places - on the NIC, inside
RAM attached to the NIC, in host memory, or in any combination of
the three, depending on the implementation.
Connection Context Memory includes state associated with Data
Buffers. For Tagged Buffers, this includes how STag names, Data
Buffers, and Page Translation Tables inter-relate. It also includes
the FIFO list of Untagged Data Buffers posted for reception of
Untagged Messages (referred to in some contexts as the Receive
Queue), and a list of operations to perform to send data (referred
to in some contexts as the Send Queue).
4.2.2 Data Buffers
There are two different ways to expose a data buffer; a buffer can
be exposed for receiving RDMAP Send Type Messages (a.k.a. DDP
Untagged Messages) on DDP Queue zero or the buffer can be exposed
for remote access through STags (a.k.a. DDP Tagged Messages). This
distinction is important because the attacks and the countermeasures
used to protect against the attack are different depending on the
method for exposing the buffer to the Internet.
For the purposes of the security discussion, a single logical Data
Buffer is exposed with a single STag. Actual implementations may
support scatter/gather capabilities to enable multiple physical data
buffers to be accessed with a single STag, but from a threat
analysis perspective it is assumed that a single STag enables access
to a single logical Data Buffer.
J. Pinkerton, et al. Expires - April 2004 [Page 10]
Internet-Draft RDDP/RDMAP Security October 2003
In any event, it is the responsibility of the RI to ensure that no
STag can be created that exposes memory that the consumer had no
authority to expose.
4.2.3 Page Translation Tables
Page Translation Tables are the structures used by the RNIC to be
able to access application memory for data transfer operations. Even
though these structures are called "Page" Translation Tables, they
may not reference a page at all - conceptually they are used to map
an application address space representation of a buffer to the
physical addresses that are used by the RNIC Engine to move data. If
on a specific system, a mapping is not used, then a subset of the
attacks examined may be appropriate.
4.2.4 STag Namespace
The DDP specification defines a 32-bit namespace for the STag.
Implementations may vary in terms of the actual number of STags that
are supported. In any case, this is a bounded resource that can come
under attack. Depending upon Stag namespace allocation algorithms,
the actual name space to attack may be significantly less than 2^32.
4.2.5 Completion Queues
Completion Queues are used in this specification to conceptually
represent how the RNIC Engine notifies the Application of the
completion of the transmission of data, or the completion of the
reception of data through the Data Transfer Interface. Because there
could be many transmissions or receptions in flight at any one time,
completions are modeled as a queue rather than a single event. An
implementation may also use the Completion Queue to notify the
application of other activities, for example, the completion of a
mapping of an STag to a specific application buffer.
4.2.6 RDMA Read Request Queue
The RDMA Read Request Queue is the memory holding state information
for one or more RDMA Read Request Messages that have arrived, but
for which the RDMA Read Response Messages have not yet been
completely sent.
4.2.7 RDMA Asynchronous Event Queue
The Asynchronous Event Queue is a queue from the RNIC to the
Privileged Resource Manager of bounded size. It is used by the RNIC
to notify the host of various events which might require management
action, including protocol violations, connection state changes,
local operation errors, low water marks on receive queues, and
possibly other events.
J. Pinkerton, et al. Expires - April 2004 [Page 11]
Internet-Draft RDDP/RDMAP Security October 2003
The RDMA Event Queue is a resource that can be attacked because
Remote or Local Peers can cause events to occur which have the
potential of overflowing the queue.
Note that an implementation is at liberty to implement the functions
of the RDMA Asynchronous Event Queue in a variety of ways, including
multiple queues or even simple callbacks. All vulnerabilities
identified for a single Asynchronous Event Queue apply to specific-
purpose subsets. A callback function is simply a very short queue.
4.2.8 RNIC Control Interactions
With RNIC resources and interfaces defined, it is now possible to
examine the interactions supported by the generic RNIC functional
interfaces through each of the 3 interfaces - Privileged Control
Interface, Privileged Data Interface, and Non-Privileged Data
Interface.
Generically, the Privileged Control Interface controls the RNICÆs
allocation, deallocation, and initialization of RNIC global
resources. This includes allocation and deallocation of Connection
Context Memory, Page Translation Tables, STag names, Completion
Queues, and RDMA Read Request Queues.
Event Queue
Initialization and removal of Page Translation Table resources.
4.2.9 Initialization of RNIC Data Structures for Data Transfer
Initialization of the mapping between an STag and a Data Buffer can
be viewed in the abstract as two separate opertions:
a. Initialization of the allocated Page Translation Table
entries with the location of the Data Buffer, and
b. Initialization of a mapping from an allocated STag name to a
set of Page Translation Table entry(s) or partial-entries.
Note that an implementation may not have a Page Translation Table
(i.e. it may support a direct mapping between an STag and a Data
Buffer). In this case threats and mitigations associated with the
Page Translation Table are not relevent.
Initialization of the contents of the Page Translation Table can be
done by either the Privileged Application or by the Privileged
Resource Manager as a proxy for the Non-Privileged Application. By
definition the Non-Privileged Application is not trusted to directly
J. Pinkerton, et al. Expires - April 2004 [Page 12]
Internet-Draft RDDP/RDMAP Security October 2003
manipulate the Page Translation Table. In general the concern is
that the Non-Privileged application may try to maliciously
initialize the Page Translation Table to access a buffer for which
it does not have permission.
The exact resource allocation algorithm for the Page Translation
Table is outside the scope of this specification. It may be
allocated for a specific Data Buffer, or be allocated as a pooled
resource to be consumed by potentially multiple Data Buffers, or be
managed in some other way. This paper attempts to abstract
implementation dependent issues, and focus on higher level security
issues such as resource starvation and sharing of resources between
connections.
The next issue is how an STag name is associated with a Data Buffer.
For the case of an Untagged Data Buffer, there is no wire visible
mapping between an STag name and a Data Buffer. Note that there may,
in fact, be a mapping that is not visible from the wire, but this is
a local host specific issue which should be analyzed in the context
of local host implementation specific security analysis, and thus is
outside the scope of this paper.
For a Tagged Data Buffer, either the Privileged Application, the
Non-Privileged Application, or the Privileged Resource Manager
acting on behalf of the Non-Privileged Resource Manager may
initialize a mapping from an STag to a Page Translation Table, or
may have the ability to simply enable/disable an existing STag to
Page Translation Table mapping. There may also be multiple STag
names which map to a specific group of Page Translation Table
entries (or sub-entries). Specific security issues with this level
of flexibility are examined later.
There are a variety of implementation options for initialization of
Page Translation Table entries and mapping an STag to a group of
Page Translation Table entries which have security repercussions.
This includes support for separation of Mapping an STag verses
mapping a set of Page Translation Table entries, and support for
Applications directly manipulating STag to Page Translation Table
entry mappings (verses requiring access through the Privileged
Resource Manager).
4.2.10 RNIC Data Transfer Interactions
RNIC Data Transfer operations can be subdivided into send operations
and receive operations.
For send operations, there is typically a queue that enables the
Application to post multiple operations. Depending upon the
implementation, Data Buffers used in the operations may or may not
have Page Translation Table entries associated with them, and may or
J. Pinkerton, et al. Expires - April 2004 [Page 13]
Internet-Draft RDDP/RDMAP Security October 2003
may not have STags associated with them. Because this is a local
host specific implementation issue rather than a protocol issue, the
security analysis of threats and mitigations is left to the host
implementation.
Receive operations are different for Tagged Data Buffers verses
Untagged Data Buffers. If more than one Untagged Data Buffer can be
posted by the Application, the DDP specification requires that they
be consumed in FIFO order. Thus the most general implementation is
that there is a FIFO queue of receive Untagged Data Buffers. Some
implementations may also support sharing of the FIFO queue between
multiple connections. In this case defining ææFIFOÆÆ becomes non-
trivial - in general the buffers for a single stream are consumed
from the queue in the order that they were placed on the queue, but
there is no order guarantee between streams.
For receive Tagged Data Buffers, at some time prior to data
transfer, the mapping of the STag to specific Page Translation Table
entries (if present) and the mapping from the Page Translation Table
entries to the Data Buffer must have been initialized (see the prior
section for interaction details).
4.3 System Properties
System properties that can be attacked included system integrity,
system stability (liveness, large fluctuations in performance), and
confidentiality.
J. Pinkerton, et al. Expires - April 2004 [Page 14]
Internet-Draft RDDP/RDMAP Security October 2003
5 Trust Models
The Trust Models described in this section have three primary
distinguishing characteristics.
* Local Resource Sharing (yes/no) - When local resources are
shared, they are shared across a grouping of RDMAP/DDP
Streams. If local resources are not shared, the resources
are dedicated on a per Stream basis. Resources are defined
in Section 4.2 - Resources on page 10. The advantage of not
sharing resources between Streams is that it reduces the
types of attacks that are possible. The disadvantage is that
applications might run out of resources.
* Local Partial Trust (yes/no) - Local Partial Trust is
determined based on whether the local grouping of RDMAP/DDP
Streams (which typically equates to one application or group
of applications) mutually trust each other to not perform a
specific set of attacks.
* Remote Partial Trust (yes/no) - The Remote Partial Trust
level is determined based on whether the Local Peer of a
specific RDMAP/DDP Stream partially trusts the Remote Peer
of the Stream (see the definition of Partial Trust in
Section 3 Introduction).
It is assumed in this paper that the component that implements the
mechanism to control sharing of RNIC Engine resources is the
Privileged Resource Manager. The RNIC Engine exposes its resources
through the RI to the Privileged Resource Manager. All Privileged
and Non-Privileged applications request resources from the Resource
Manager. The Resource Manager implements resource management
policies to ensure fair access to resources. The Resource Manager
should be designed to take into account security attacks detailed in
this specification.
The sharing of resources across connections should be under the
control of the application, both in terms of the Trust Model the
application wishes to operate under, as well as the level of
resource sharing the application wishes to give Local Peer
processes.
Not all of the combinations of the trust characteristics are
expected to be used by applications. This paper specifically
analyzes five application Trust Models that are expected to be in
common use. The Trust Models are as follows:
1. NS-NT - Non-Shared Local Resources, no Local Trust, no Remote
Trust - typically a server application that wants to run in a
mode that has the least number of potential attacks.
J. Pinkerton, et al. Expires - April 2004 [Page 15]
Internet-Draft RDDP/RDMAP Security October 2003
2. NS-RT - Non-Shared Local Resources, no Local Trust, Remote
Partial Trust - typically a peer-to-peer application, which has,
by some method outside of the scope of this specification,
authenticated the Remote Peer. Note that unless some form of key
based authentication is used on a per packet basis, it may not
be possible to bind the authentication result to the RDMA
packet.
3. S-NT - Shared Local Resources, no Local Trust, no Remote Trust -
typically a server application that runs in an untrusted
environment where the amount of resources required is either too
large or too dynamic to dedicate for each RDMAP/DDP Stream.
4. S-LT - Shared Local Resources, Local Partial Trust, no Remote
Trust - typically an application, which provides a session layer
and uses multiple Streams, to provide additional throughput or
fail-over capabilities. All of the Streams within the local
application partially trust each other, but do not trust the
remote peer.
5. S-T - Shared Local Resources, Local Partial Trust, Remote
Partial Trust - typically a distributed application, such as a
distributed database application or a High Performance Computer
(HPC) application, which is intended to run on a cluster. Due to
extreme resource and performance requirements, the application
typically authenticates with all of its peers and then runs in a
highly trusted environment. The application peers are all in a
single application fault domain and depend on one another to be
well-behaved when accessing data structures. If a trusted Remote
Peer has an implementation defect that results in poor behavior,
the entire application could be corrupted.
Models NS-NT and S-NT above are typical for Internet networking -
neither Local Peers nor the Remote Peer is trusted. Sometimes
optimizations can be done that enable sharing of Page Translation
Tables across multiple Local Peers, thus Model S-LT can be
advantageous. Model S-T is typically used when resource scaling
across a large parallel application makes it infeasible to use any
other model. Resource scaling issues can either be due to
performance around scaling or because there simply are not enough
resources. Model NS-RT is probably the least likely model to be
used, but is presented for completeness.
J. Pinkerton, et al. Expires - April 2004 [Page 16]
Internet-Draft RDDP/RDMAP Security October 2003
6 Attacker Capabilities
An attackerÆs capabilities delimit the types of attacks that
attacker is able to launch. RDMAP and DDP require that the initial
LLP Stream (and connection) be set up prior to transferring
RDMAP/DDP Messages. For the attacker to actively generate an
RDMAP/DDP protocol attack, it must have the capability to both send
and receive messages. Attackers with send only capabilities should
be addressed by the LLP, not by RDMAP/DDP.
J. Pinkerton, et al. Expires - April 2004 [Page 17]
Internet-Draft RDDP/RDMAP Security October 2003
7 Attacks and Countermeasures
This section describes the attacks that are possible against the
RDMA system defined in Figure 1 - RDMA Security Model and the RNIC
Engine resources defined in Section 4.2. The analysis includes a
detailed description of each attack, the Trust Models the attack
applies to (see Section 5 for a description of the Trust Models),
and a description of the countermeasures appropriate to the Trust
Model(s) that can be taken to thwart the attack.
Note that, connection setup and teardown is presumed to be done in
stream mode (i.e. no RDMA encapsulation of the payload), so there
are no new attacks related to connection setup/teardown beyond what
is already present in the LLP (e.g. TCP or SCTP). Consequently, any
existing analysis of Spoofing, Tampering, Repudiation, Information
Disclosure, Denial of Service, or Elevation of Privilege continues
to apply. Thus, the analysis in this section focuses on attacks that
are present regardless of the LLP Stream type.
7.1 Tools for Countermeasures
The tools described in this section are the primary mechanisms that
can be used to provide countermeasures to potential attacks.
7.1.1 Protection Domain (PD)
Protection Domains are associated with two of the resources of
concern, connection context memory and STags associated with Page
Translation Table entries and data buffers. Protection Domains are
used mainly to ensure that an STag can only be used to access the
associated data buffer through connections in the same Protection
Domain as that STag.
For the Trust Models that are defined to have non-shared resources
(Trust Models NS-NT and NS-RT), it is recommended that each Stream
be associated with its own, unique Protection Domain. For those
Trust Models where resources are shared (Trust Models S-NT, S-LT and
S-T), it is recommended that Protection Domain be limited to the
number of Streams that share the same Trust Model.
Note that an application (either Privileged or Non-Privileged) can
potentially have multiple Protection Domains. This could be used,
for example, to ensure that multiple clients of a server do not have
the ability to corrupt each other. This can apply to any combination
of the Trust Models, because Partial Trust does not imply complete
trust.
J. Pinkerton, et al. Expires - April 2004 [Page 18]
Internet-Draft RDDP/RDMAP Security October 2003
7.1.2 Limiting STag Scope
The key to protecting a local data buffer is to limit the scope of
its STag to the level appropriate for the Trust Model. The scope of
the STag can be measured in multiple ways.
* Number of Connections and/or Streams on which the STag is
valid. One way to limit the scope of the STag is to limit
the connections and/or Streams that are allowed to use the
STag. As noted in the previous section, use of Protection
Domains appropriately can limit the scope of the STag. It is
also possible to create an STag that is valid only on a
single connection, even in the case where several
connections are associated with the Protection Domain of the
STag.
* Limit the time an STag is valid. By Invalidating an
Advertised STag (e.g., revoking remote access to the buffers
described by an STag when done with the transfer), an entire
class of attacks can be eliminated.
* Limit the buffer the STag can reference. Limiting the scope
of an STag access to *just* the intended buffers to be
exposed is critical to prevent certain forms of attacks.
* Allocating Stag numbers in an unpredictable way. If STags
are allocated using an algorithm which makes it hard for the
Remote Peer to guess which STag(s) are currently in use, it
makes it more difficult for an attacker to guess the correct
value.
7.1.3 Access Rights
Access Rights associated with a specific Advertised STag or
RDMAP/DDP Stream provide another mechanism for applications to limit
the attack capabilities of the Remote Peer. The Local Peer can
control whether a data buffer is exposed for local only, or local
and remote access, and assign specific access privileges (read,
write, read and write).
For DDP, when an STag is advertised, the Remote Peer is presumably
given write access rights to the data (otherwise there was not much
point to the advertisement). For RDMAP, when an application
advertises an STag, it can enable write-only, read-only, or both
write and read access rights.
Similarly, some applications may wish to provide a single buffer
with different access rights on a per-connection or per-Stream
basis. For example, some connections may have read-only access, some
J. Pinkerton, et al. Expires - April 2004 [Page 19]
Internet-Draft RDDP/RDMAP Security October 2003
may have remote read and write access, while on other connections
only the Local Peer is allowed access.
7.1.4 Limiting the Scope of the Completion Queue
Completions associated with sending and receiving data, or setting
up buffers for sending and receiving data, could be accumulated in a
shared Completion Queue for a group of RDMAP/DDP Streams, or a
specific RDMAP/DDP Stream could have a dedicated Completion Queue.
Limiting Completion Queue association to one, or a small number of
RDMAP/DDP Streams can prevent several forms of Denial of Service
attacks.
7.1.5 Limiting the Scope of an Error
To prevent a variety of attacks, it is important that an RDMAP/DDP
implementation be robust in the face of errors. If an error on a
specific Stream can cause other unrelated Streams to fail, then a
broad class of attacks are enabled against the implementation.
7.2 Spoofing
Because the RDMAP Stream is only offloaded if it is in the
ESTABLISHED state, certain types of traditional forms of wire
attacks do not apply -- an end-to-end handshake must have occurred
to establish the RDMAP Stream. So, the only form of spoofing that
applies is one when a remote node can both send and receive packets.
7.2.1 Connection Hijacking
If a man-in-the-middle attacker has the ability to inject packets
which will be accepted by the LLP (e.g., TCP sequence number is
correct) then the connection can essentially be hijacked. One style
of attack is for the man-in-the-middle to send Tagged Messages
(either RDMAP or DDP). If it can discover a buffer that has been
exposed for STag enabled access, then the man-in-the-middle can use
an RDMA Read operation to read the contents of the associated data
buffer, perform an RDMA Write Operation to modify the contents of
the associated data buffer, or invalidate the STag to disable
further access to the buffer. The only countermeasure for this form
of attack is to either secure the RDMAP/DDP Stream or attempt to
provide physical security to prevent man-in-the-middle type access.
The best protection against this form of attack is end-to-end
integrity protection and authentication, such as IPsec (see Section
8 Security Services for RDDP on page 35), to prevent spoofing or
tampering. If authentication is not used, then a man-in-the-middle
attack can occur, enabling spoofing, tampering, and repudiation.
J. Pinkerton, et al. Expires - April 2004 [Page 20]
Internet-Draft RDDP/RDMAP Security October 2003
Because the connection itself is established by the LLP, some LLPs
are more difficult to hijack than others. Please see the relevant
LLP documentation on security issues around connection hijacking
<TBD: references for SCTP and TCP on connection hijacking>.
Another approach is to restrict access to only the local
subnet/link, and provide some mechanism to limit access, such as
physical security or 802.1.x. This model is an extremely limited
deployment scenario, and will not be further examined here.
7.2.2 Using an STag on a different connection
One style of attack from the Remote Peer is for it to attempt to use
STag values that it is not authorized to use. Note that, if the
Remote Peer sends an invalid STag to the Local Peer, per the DDP and
RDMAP specifications, the connection must be torn down. Thus, the
threat exists if a STag has been enabled for Remote Access on one
connection and a Remote Peer is able to use it on an unrelated
connection. If the attack is successful, the attacker could
potentially be able to perform either RDMA Read Operations to read
the contents of the associated data buffer, perform RDMA Write
Operations to modify the contents of the associated data buffer, or
to Invalidate the STag to disable further access to the buffer.
An attempt by a Remote Peer to access a buffer with an STag on a
different connection in the same Protection Domain may or may not be
an attack depending on the Trust Model employed by the application.
For Trust Model S-T, where resources are shared between connections,
and both Local and Remote Peers are Trusted, using an STag on
multiple connections within the same Protection Domain is allowed,
and could be desired behavior. For the other four Trust Models where
the Remote Peer is not Trusted, and/or resources are not intended to
be shared, attempting to use an STag on a different connection could
be considered to be an attack.
In the case where the Trust Model is defined with no shared
resources between connections (Trust Models NS-NT and NS-RT), this
attack can be defeated by assigning each connection to a different
Protection Domain. Before allowing remote access to the buffer, the
Protection Domain of the connection where the access attempt was
made is matched against the Protection Domain of the STag. If the
Protection Domains do not match, access to the buffer is denied, an
error is generated, and the RDMAP Stream associated with the
attacking connection should be terminated. Thus, for Trust Models
NS-NT and NS-RT, it is RECOMMENDED that each connection be in a
separate Protection Domain.
For Trust Models S-NT and S-LT, where resources are shared, but the
Remote Peers are Untrusted, it may not be practical to separate each
connection into its own Protection Domain. In this case, the
J. Pinkerton, et al. Expires - April 2004 [Page 21]
Internet-Draft RDDP/RDMAP Security October 2003
application can still limit the scope of any of the STags it is
enabling for remote access to a single connection. If the STag scope
has been limited to a single connection, any attempt to use that
STag on a different connection will result in an error, and the RDMA
Stream associated with that connection should be terminated. Thus,
for Trust Models S-NT and S-LT, it is RECOMMENDED that the scope of
an STag be limited to a single connection.
For Trust Models S-NT and S-LT (Untrusted Remote Peers), if it is
not possible to use Protection Domains or to limit the scope of an
STag to a single connection, it is RECOMMENDED that STag allocators
select an STag using an algorithm which makes it difficult to guess
the next allocated STag number. This approach is good practice in
general. Allocation methods which always start with the same number
(e.g. zero) after Stream initialization or simply allocate the next
STag in a monotonically increasing namespace should be avoided.
7.3 Tampering
A Remote Peer can attempt to tamper with the contents of data
buffers on a Local Peer that have been enabled for remote write
access. The types of tampering attacks that are possible are
outlined in the sections that follow.
7.3.1 RDMA Write or Read Response to Memory Outside the Buffer
This attack is an attempt by the Remote Peer to perform an RDMA
Write or RDMA Read Response to memory outside of the valid length
range of the data buffer enabled for remote write access. This
attack applies primarily to Trust Models with Untrusted Remote Peers
(NS-NT, S-NT and S-LT), and can occur even when no resources are
shared across connections. This issue can also arise for Trust
Models NS-RT and S-T, which assume remote Partial Trust, if the
application has a bug. Thus it is RECOMMENDED that all Trust Models
ensure this countermeasures are in place against this form of
attack.
The countermeasure for this type of attack must be in the RNIC
implementation, using the STag. When the Local Peer specifies to the
RI, the base and the number of bytes in the buffer that it wishes to
make accessible, the RI must ensure that the base and bounds check
are applied to any access to the buffer referenced by the STag
before the STag is enabled for access. When an RDMA data transfer
operation (which includes an STag) arrives on a connection, a base
and bounds byte granularity access check must be performed to ensure
the operation accesses only memory locations within the buffer
described by that STag.
Thus, it is RECOMMENDED that an RI implementation ensure that a
Remote Peer, regardless of Trust Model, will not be able to access
J. Pinkerton, et al. Expires - April 2004 [Page 22]
Internet-Draft RDDP/RDMAP Security October 2003
memory outside of the buffer specified when the STag was enabled for
remote access.
7.3.2 Modifying a Buffer After Indicating Contents Are Ready
This attack occurs if a Remote Peer attempts to modify the contents
by performing an RDMA Write or an RDMA Read Response after it had
indicated to the Local Peer that the data buffer contents were ready
for use.
This attack applies primarily to the Trust Models where the Remote
Peers are not Trusted (Trust Models NS-NT, S-NT and S-LT), and can
occur even when no resources are shared across connections. Note
that, an error on the part of a Trusted Remote Peer could also
result in this problem.
The Local Peer can protect itself from this type of attack by
revoking remote access when the original data transfer has completed
and before it validates the contents of the buffer. The Local Peer
can either do this by explicitly revoking remote access rights for
the STag when the Remote Peer indicates the operation has completed,
or by checking to make sure the Remote Peer Invalidated the STag
through the RDMAP Invalidate capability, and if it did not, the
Local Peer then explicitly revokes the STag remote access rights.
It is RECOMMENDED that the Local Peer follow the above procedure for
Trust Models NS-NT, S-NT, and S-LT to protect the buffer. The Local
Peer MAY also wish to use this procedure for Trust Models NS-RT and
S-T to protect itself from unintended tampering due to an error in
the Remote Peer.
7.3.3 Using Multiple Stags to access the same buffer
See section 7.4.6 on page 25 for this analysis.
7.4 Information Disclosure
The main potential source for information disclosure is through a
local buffer that has been enabled for remote access. If the buffer
can be probed by a Remote Peer on another connection, then there is
potential for information disclosure.
Information disclosure attacks mainly apply to the Trust Models that
include Untrusted Remote Peers (Trust Models NS-NT, S-NT, and S-LT
as defined in Section 5). Trusted Remote Peers are assumed not to
purposely attempt such attacks - any attempt is assumed to be due to
an error or other unexpected failure in the Remote Peer.
The potential attacks that could result in unintended information
disclosure and countermeasures are as follows:
J. Pinkerton, et al. Expires - April 2004 [Page 23]
Internet-Draft RDDP/RDMAP Security October 2003
7.4.1 Probing memory outside of the buffer bounds
This is essentially the same attack as described in Section 7.3.1,
except an RDMA Read Request is used to mount the attack. The same
countermeasure applies.
7.4.2 Using RDMA Read to Access Stale Data
If a buffer is being used for a combination of reads and writes
(either remote or local), and is exposed to the Remote Peer with at
least remote read access rights, the Remote Peer may be able to
examine the contents of the buffer before they are initialized with
the correct data. In this situation, whatever contents were present
in the buffer before the buffer is initialized can be viewed by the
Remote Peer, if the Remote Peer performs an RDMA Read. This threat
applies to Trust Models NS-NT, S-NT, and S-LT.
Because of this, it is RECOMMENDED that the Local Peer ensure that
no stale data is contained in the buffer when remote read access
rights are initially granted (this can be done by zeroing the
contents of the memory, for example).
7.4.3 Accessing a buffer after the transfer is over
If the Remote Peer has remote read access to a buffer, and by some
mechanism tells the Local Peer that the transfer has been completed,
but the Local Peer does not disable remote access to the buffer
before modifying the data, it is possible for the Remote Peer to
retrieve the new data.
This is similar to the attack defined in Section 7.3.2 Modifying a
Buffer After Indicating Contents Are Ready on page 23. The same
countermeasures apply. In addition, it is RECOMMENDED that the Local
Peer should grant remote read access rights only for the amount of
time needed to retrieve the data.
7.4.4 Accessing data within a valid STag that was unintended
If the Local Peer enables remote access to a buffer using an STag
that references the entire buffer, but intends only a portion of the
buffer to be accessed, it is possible for the Remote Peer to access
the other parts of the buffer anyway. This threat applies to Trust
Models NS-NT, S-NT, and S-LT.
To prevent this attack, it is RECOMMENDED that the Local Peer set
the base and bounds of the buffer when the STag is initialized to
expose only the data to be retrieved.
J. Pinkerton, et al. Expires - April 2004 [Page 24]
Internet-Draft RDDP/RDMAP Security October 2003
7.4.5 Using RDMA Read on a buffer meant only for RDMA Write
One form of disclosure can occur if the access rights on the buffer
were set for remote read, when only remote write access was
intended. This attack applies primarily to Trust Models with
Untrusted Remote Peers (NS-NT, S-NT and S-LT). If the buffer
contained application data, or data from a transfer on an unrelated
connection, the Remote Peer could retrieve the data through an RDMA
Read operation.
The most obvious countermeasure for this attack is to not grant
remote read access if the buffer is intended to be write-only. The
Remote Peer would not be able to retrieve data associated with the
buffer. An attempt to do so would result in an error and the RDMAP
Stream associated with the connection would be terminated.
Thus, it is RECOMMENDED that if an application only intends a buffer
to be exposed for remote write access, it set the access rights to
the buffer to only enable remote write access.
7.4.6 Using Multiple Stags to access the same buffer
Multiple STags accessing the same buffer at the same time can result
in unintentional information disclosure if the STags are used by
different Remote Peers. Because an RDMA implementation could allow
an STag to have read, write, or read and write access associated
with an Stag, it is possible to have unintended information
disclosure if the Remote Peers do not share the same Trust Model.
If only read access is enabled, then the Local Peer has complete
control over the information disclosure and multiple Stags to the
same buffer creates no new security issues.
When one STag has remote read access enabled and a different STag
has remote write access enabled to the same buffer, it is possible
for one connection to view the contents that have been written by
another Remote Peer.
If both Stags have remote write access enabled and the two Remote
Peers do not mutually trust each other, it is possible for one
Remote Peer to overwrite the contents that have been written by the
other Remote Peer.
For Trust Models NS-NT, S-NT, S-LT it is RECOMMENDED that multiple
Remote Peers not be granted access to the same buffer through
different STags at the same time. A buffer should be exposed to only
one Untrusted Remote Peer at a time to ensure that no information
disclosure or information tampering occurs between peers.
J. Pinkerton, et al. Expires - April 2004 [Page 25]
Internet-Draft RDDP/RDMAP Security October 2003
7.4.7 Remote node loading firmware onto the RNIC
If the Remote Peer can cause firmware to be loaded onto the RNIC,
there is an opportunity for information disclosure. See Elevation of
Privilege in Section 7.6 for this analysis.
7.4.8 Controlling Access to Page Translation Table and STag Mapping
Issue: Discuss issues in allowing the non-privileged consumer to
control mapping the Page Translation Table and Stag.
It is RECOMMENDED that the Privileged Resource Manager verify that
the Non-Privileged application has the right to access a specific
Data Buffer before allowing an STag for which the application has
access rights to be associated with a specific Data Buffer. This
can be done when the Page Translation Table is initialized to access
the Data Buffer or when the STag is initialized to point to a group
of Page Translation Table entries, or both.
7.5 Denial of Service (DOS)
A DOS attack is one of the primary security risks of RDMAP. This is
because RNIC resources are valuable and scarce, and many application
environments require communication with Untrusted Remote Peers. If
the remote application can be authenticated or encrypted, clearly,
the DOS profile can be reduced. For the purposes of this analysis,
it is assumed that the RNIC must be able to operate in Untrusted
environments, which are open to DOS style attacks.
Denial of service attacks against RI resources are not the typical
unknown party spraying packets at a random host (such as a TCP SYN
attack). Because the connection must be fully established, the
attacker must be able to both send and receive messages over that
connection, or be able to guess a valid packet on an existing RDMAP
Stream.
This section outlines the potential attacks and the countermeasures
available for dealing with each attack. For each attack, the Trust
Model that it applies to is described.
J. Pinkerton, et al. Expires - April 2004 [Page 26]
Internet-Draft RDDP/RDMAP Security October 2003
7.5.1 RNIC Resource Consumption
This section covers attacks that fall into the general category of a
Local Peer attempting to unfairly allocate scarce RNIC resources.
The Local Peer may be attempting to allocate resources on its own
behalf, or on behalf of a Remote Peer. Resources that fall into this
category include: Protection Domains, Connection Context Memory,
Translation and Protection Tables, and STag namespace. These can be
attacks by currently active Local Peers or ones that allocated
resources earlier, but are now idle.
These attacks generally apply to any Trust Model that includes
Untrusted Local Peers (Trust Models NS-NT, NS-RT and S-NT). This
type of attack can occur even when resources are not shared across
connections.
It is RECOMMENDED that the allocation of all scarce resources be
placed under the control of a Resource Manager. This allows the
Resource Manager to:
* prevent a Local Peer from allocating more than its fair
share of resources, and
* detect if a Remote Peer is attempting to launch a DOS attack
by attempting to create an excessive number of connections
and take corrective action (such as refusing the request or
applying network layer filters against the Remote Peer).
Note that, placing scarce resource management under the control of a
Resource Manager also prevents other Trusted Local Peers from
attempting to allocate more than their fair share of resources.
This analysis assumes that the Resource Manager is responsible for
handing out Protection Domains, and RNIC implementations will
provide enough Protection Domains to allow the Resource Manager to
be able to assign a unique Protection Domain for each unrelated,
Untrusted Local Peer (for a bounded, reasonable number of Local
Peers). This analysis further assumes that the Resource Manager
implements policies to ensure that Untrusted Local Peers are not
able to consume all of the Protection Domains through a DOS attack.
Note that Protection Domain consumption cannot result from a DOS
attack launched by a Remote Peer, unless a Local Peer is acting on
the Remote PeerÆs behalf.
7.5.2 Resource Consumption By Active Applications
This section describes DOS attacks from Local and Remote Peers that
are actively exchanging messages. Attacks on each RDMA NIC resource
are examined, including the Trust Models that apply, and the
specific countermeasures. Note that, attacks on Connection Context
J. Pinkerton, et al. Expires - April 2004 [Page 27]
Internet-Draft RDDP/RDMAP Security October 2003
Memory, Page Translation Tables, and STag namespace are covered in
Section 7.5.1 RNIC Resource Consumption, so are not included here.
7.5.2.1 Receive Data Buffers
The Remote Peer can attempt to consume more than its fair share of
receive data buffers (Untagged DDP buffers or for RDMAP buffers
consumed with Send Type Messages). If resources are not shared
across multiple connections (Trust Models NS-NT, NS-RT), then this
attack is not possible because the Remote Peer will not be able to
consume more buffers than were allocated to the Stream. The worst
case scenario is that the Remote Peer can consume more receive
buffers than the Local Peer allowed, resulting in no buffers to be
available, which would cause the Remote PeerÆs connection to the
Local Peer to be torn down.
If local receive data buffers are shared among multiple Streams and
the Remote Peer is not Trusted (Trust Models S-NT, S-LT), then the
Remote Peer can attempt to consume more than its fair share of the
receive buffers, causing a different Stream to be short of receive
buffers, thus possibly causing the other Stream to be torn down.
One method the Local Peer could use is to recognize that a Remote
Peer is attempting to use more than its fair share of resources and
terminate the Stream. However, if the Local Peer is sufficiently
slow, it may be possible for the Remote Peer to still mount a denial
of service attack. An RNIC Engine implementation that enables a more
robust countermeasure is one that provides high and low-water
notifications to enable the Local Peer to detect and prevent DOS
attacks against shared data buffers. If a low-water notification is
enabled, and the Local Peer is able to bound the amount of time that
it takes to replenish receive buffers, and the Local Peer maintains
statistics to determine which Remote Peer is consuming buffers, then
the Local Peer can size the amount of local receive buffers posted
on the receive queue such that the low-water notification can arrive
before resources are depleted and corrective action can be taken
(e.g., terminate the Stream of the attacking Remote Peer). Enabling
the high-water notification can help the Local Peer detect a Remote
Peer that is launching an attack by sending a large number of out-
of-order packets. The notification is generated when more than the
specified number of buffers are in process simultaneously on a
Stream (i.e., packets have started to arrive for the buffer, but
have not yet been delivered to the ULP).
A different countermeasure is for the RNIC Engine to provide the
capability to limit the Remote PeerÆs ability to consume receive
buffers on a per Stream basis. Unfortunately this requires a large
amount of state to be tracked on a per RNIC basis.
J. Pinkerton, et al. Expires - April 2004 [Page 28]
Internet-Draft RDDP/RDMAP Security October 2003
Thus, if an RNIC Engine provides the ability to share receive
buffers across multiple Streams, it is RECOMMENDED that it enable
the Local Peer to detect if the Remote Peer is attempting to consume
more than its fair share of resources so that the application can
apply countermeasures to detect and prevent the attack.
7.5.2.2 Completion Queue (CQ) Resource Consumption
DOS attacks against the Completion Queue can be caused by either the
Local Peer or the Remote Peer if either attempts to cause more
completions than its fair share, thus potentially starving another
unrelated Stream such that no Completion Queue entries are
available.
A Completion Queue entry can potentially be consumed by a completion
from the send queue or a receive completion. In the former, the
attacker is the Local Peer (Trust Models S-NT). In the later, the
attacker is the Remote Peer (S-NT, S-LT).
The potential attacks and the countermeasures for each are described
in the subsections that follow.
7.5.2.2.1 Local Peer Attacking a Shared CQ
A form of attack can occur for Trust Models NS-NT, NS-RT, and S-NT,
where the Local Peers are Untrusted, and Local Peers can consume
resources on the CQ. Sharing a CQ across connections that belong to
different Protection Domains is NOT RECOMMENDED in cases where any
of the Local Peers are Untrusted. A Local Peer that is slow to free
resources on the CQ by not reaping the completion status quick
enough could stall all other Local Peers attempting to use that CQ.
One of two countermeasures can be used to avoid this kind of attack.
The first is to use a different CQ per Untrusted Local Peer. The
other is to use a Trusted Local Peer to act as a third party to free
resources on the CQ and place the status in intermediate storage
until the Untrusted Local Peer reaps the status information.
7.5.2.2.2 Remote Peer Attacking a Shared CQ
The Remote Peer can attack a CQ by consuming more than its fair
share of CQ entries by using one of two methods. The first method
can only be used if the ULP protocol allows the Remote Peer to
reserve a specified number of CQ entries, possibly leaving
insufficient entries for other connections that are sharing the CQ.
The other method is if the Remote Peer can attack the CQ by
overwhelming the CQ with completions, which can affect completion
processing on other Streams sharing that connection.
J. Pinkerton, et al. Expires - April 2004 [Page 29]
Internet-Draft RDDP/RDMAP Security October 2003
The first method of attack can be avoided if the ULP does not allow
a Remote Peer to reserve CQ entries. This is RECOMMENDED
particularly for Trust Models S-NT and S-LT, with shared resources
and Untrusted Remote Peers. If a Local Peer allows this type of
resource allocation, and it has any Untrusted Remote Peers, then the
Local Peer it is RECOMMENDED that the CQ be isolated to connections
within a single Protection Domain.
One way that a Remote Peer can attempt to overwhelm its CQ with
completions is by sending minimum length RDMAP/DDP Messages to cause
as many completions per second as possible. Assuming that the Local
Peer does not run out of receive buffers (if they do, then this is a
different attack, documented in Section 7.5.2.1 Receive Data Buffers
on page 28), then it might be possible for the Remote Peer to
consume more than its fair share of Completion Queue entries.
Depending upon the CQ implementation, this could either cause the CQ
to overflow (if it is not large enough to handle all of the
completions generated) or for another Stream to not be able to
generate CQ entries (if the RNIC had flow control on generation of
CQ entries into the CQ). In either case, the CQ will stop
functioning correctly and any connections expecting completions on
the CQ will stop functioning.
This attack can occur regardless of whether all of the connections
associated with the CQ are in the same Protection Domain or are in
different Protection Domains. Because this attack assumes a shared
local resource and an Untrusted Remote Peer, Trust Models S-NT, S-LT
apply.
The Local Peer can protect itself from this type of attack using
either of the following methods:
* Resize the CQ to the appropriate level(note that resizing
the CQ can fail, so the CQ resize should be done before
sizing the buffers on the connection), OR
* Grant fewer resources than the Remote Peer requested (not
supplying the number of Receive Data Buffers requested).
The proper sizing of the CQ is dependent on the Trust Model. For the
Trust Model described in Section 5, with Trusted Local Peers and
Untrusted Remote Peers (Trust Model S-LT), a correctly sized CQ
means that the CQ is large enough to hold completion status for all
of the outstanding Receive Data Buffers, or:
J. Pinkerton, et al. Expires - April 2004 [Page 30]
Internet-Draft RDDP/RDMAP Security October 2003
CQ_MIN_SIZE = SUM(MaxPostedOnEachRQ)
+ SUM(MaxPostedOnEachS-RQ)
+ SUM(MaxPostedOnEachSQ)
If the Trust Model assumes neither the Local Peer nor the Remote
Peer is trusted (Trust Model S-NT or S-LT), then the CQ must be
sized to accommodate the maximum number of operations or Receive
Data Buffers that it is possible to post at any one time, thus the
equation becomes:
CQ_MIN_SIZE = SUM(SizeOfEachRQ)
+ SUM(SizeOfEachS-RQ)
+ SUM(SizeOfEachSQ)
Where MaxPosted*OnEach*Q and SizeOfEach*Q varies on a per connection
or per Shared Receive Queue basis.
7.5.2.3 RDMA Read Request Queue
Two types of attacks are possible against resources associated with
RDMA Read Request Queues. One style of attack can only occur when
the RDMA Read Request Queue resources are pooled across multiple
connections. This attack occurs when an Untrusted Local Peer
attempts to unfairly allocate RDMA Read Request Queue resources for
its connections.
It is RECOMMENDED that access to interfaces that allocate RDMA Read
Request Queue entries be restricted to a Trusted Local Peer, such as
a Resource Manager. The Resource Manager should prevent a Local Peer
from allocating more than its fair share of resources.
Another form of attack is the Remote Peer sending more RDMA Read
Requests than the depth of the RDMA Read Request Queue at the Local
Peer.
This attack can be prevented by properly configuring the connection
when the connection is established. The Remote PeerÆs end of the
connection should be configured by a trusted agent such that the
RNIC will not transmit RDMA Read Requests that exceed the depth of
the RDMA Read Request Queue at the Local Peer. If the connection is
correctly configured, and if the Remote Peer submits more requests
than the Local PeerÆs RDMA Read Request Queue can handle, the
requests will be queued at the Remote PeerÆs connection until
previous requests complete. If the Remote PeerÆs connection is not
configured correctly, the RDMAP Stream for that connection is
terminated when more RDMA Read Requests arrive at the Local Peer
than the Local Peer can handle. Thus, the Remote Peer is able to
only affect its own connection.
J. Pinkerton, et al. Expires - April 2004 [Page 31]
Internet-Draft RDDP/RDMAP Security October 2003
7.5.3 Resource Consumption by Idle Applications
The simplest form of a DOS attack given a fixed amount of resources
is for the Remote Peer to create a RDMAP Stream to a Local Peer,
request dedicated resources then do no actual work. This allows the
Remote Peer to be very light weight (i.e. only negotiate resources,
but do no data transfer) and consumes a disproportionate amount of
resources in the server.
A general countermeasure for this style of attack is to monitor
active RDMAP Streams and if resources are getting low, reap the
resources from RDMAP Streams that are not transferring data and
possibly terminate the connection. This needs to be under
administrative control, and demonstrates the need for a MIB for
RDMAP so this condition can be detected and acted upon.
Refer to Section 7.5.1 for the analysis and countermeasures for this
style of attack on the following RNIC resources: Connection Context
Memory, Page Translation Tables and STag namespace.
Note that some RNIC resources are not at risk of this type of attack
from a Remote Peer because an attack requires the Remote Peer to
send messages in order to consume the resource. Receive Data
Buffers, Completion Queue, and RDMA Read Request Queue resources are
examples. These resources are, however, at risk form a Local Peer
that attempts to allocate resources, then goes idle. The general
countermeasure described in this section can be used to free
resources allocated by an idle Local Peer.
7.5.4 Exercise of non-optimal code paths
Another form of DOS attack is to attempt to exercise data paths that
can consume a disproportionate amount of resources. An example might
be if error cases are handled on a ææslow pathÆÆ (consuming either
host or RNIC computational resources), and an attacker generates
excessive numbers of errors in an attempt to consume these
resources.
It is RECOMMENDED that an implementation provide the ability to
detect the above condition and allow an administrator to act,
including potentially administratively tearing down the RDMAP Stream
associated with the connection exercising data paths consuming a
disproportionate amount of resources.
7.5.5 Remote Invalidation of an STag Shared Across Multiple
Connections
If a Local Peer has enabled an STag for remote access, the Remote
Peer could attempt to invalidate the STag by using the RDMAP Send
with Invalidate or Send with SE and Invalidate Message. If the STag
J. Pinkerton, et al. Expires - April 2004 [Page 32]
Internet-Draft RDDP/RDMAP Security October 2003
is only valid on the current connection (NS-NT or NS-RT, S-NT), then
the only side effect is that the Remote Peer can no longer use the
STag, thus there are no security issues.
If the STag is valid across multiple connections, then the Remote
Peer can prevent other connections from using that STag by using the
remote invalidate functionality.
Thus for Trust Models where the Remote Peer may attempt to
invalidate the STag prematurely, the application SHOULD NOT allow an
STag to be valid across multiple connections.
7.5.6 Remote Peer Consumes too many Untagged Receive Buffers
Issue: Need to analyze the case of sharing a queue of Untagged
receive buffers across multiple connections, and that the Remote
Peer can mount a denial of service attack.
Below are some notes.
Many ways to attack here. If receive queue is not shared, itÆs a
simple queue overflow attack on a dedicated resource. Make sure when
the queue is empty and a DDP segment arrives nothing bad happens.
For a shared receive queue, one node attacks with single byte
Untagged Messages to consume large Untagged Buffers (this maximizes
packet arrival rate). One node provides DDP segments out of order to
consume out-of-order resources (this is only possible if out-of-
order placement is supported within a merged TCP/SCTP and DDP
implementation).
7.6 Elevation of Privilege
The RDMAP/DDP Security Architecture explicitly differentiates
between three levels of privilege - Non-Privileged, Privileged, and
the Privileged Resource Manager. If a Non-Privileged Application is
able to elevate its privilege level to a Privileged Application,
then mapping a physical address list to an STag can provide local
and remote access to any physical address location on the node. If a
Privileged Mode Application is able to promote itself to be a
Resource Manager, then it is possible for it to perform denial of
service type attacks where substantial amounts of local resources
could be consumed.
There is only one mechanism discovered to date, other than
implementation defects, which would potentially allow an elevation
of privilege.
J. Pinkerton, et al. Expires - April 2004 [Page 33]
Internet-Draft RDDP/RDMAP Security October 2003
7.6.1 Loading Firmware into the RNIC
If the RI implementation, by some insecure mechanism (or
implementation defect), can enable a Remote Peer or un-trusted Local
Peer to load firmware into the RNIC Engine, it is possible to use
the RNIC to attack the host. Thus, it is RECOMMENDED that an
implementation not enable firmware to be loaded on the RNIC Engine
directly from a Remote Peer, unless the update is done via a secure
protocol, such as IPsec (See Section 8 Security Services for RDDP on
page 35). It is RECOMMENDED that an implementation not allow a Non-
Privileged Local Peer to update firmware in the RNIC Engine.
J. Pinkerton, et al. Expires - April 2004 [Page 34]
Internet-Draft RDDP/RDMAP Security October 2003
8 Security Services for RDDP
Issue: Security Services section is a placeholder for now.
8.1 Introduction to Security Options
8.1.1 Introduction to IPsec
IPsec is a protocol suite which is used to secure communication at
the network layer between two peers. The IPsec protocol suite is
specified within the IP Security Architecture [RFC2401], IKE
[RFC2409], IPsec Authentication Header (AH) [RFC2402] and IPsec
Encapsulating Security Payload (ESP) [RFC2406] documents. IKE is
the key management protocol while AH and ESP are used to protect IP
traffic.
An IPsec SA is a one-way security association, uniquely identified
by the 3-tuple: Security Parameter Index (SPI), protocol (ESP) and
destination IP. The parameters for an IPsec security association
are typically established by a key management protocol. These
include the encapsulation mode, encapsulation type, session keys and
SPI values.
IKE is a two phase negotiation protocol based on the modular
exchange of messages defined by ISAKMP [RFC2408],and the IP Security
Domain of Interpretation (DOI) [RFC2407]. IKE has two phases, and
accomplishes the following functions:
1. Protected cipher suite and options negotiation - using keyed
MACs and encryption and anti-replay mechanisms
2. Master key generation - such as via MODP Diffie-Hellman
calculations
3. Authentication of end-points
4. IPsec SA management (selector negotiation, options negotiation,
create, delete, and rekeying)
Items 1 through 3 are accomplished in IKE Phase 1, while item 4 is
handled in IKE Phase 2.
An IKE Phase 2 negotiation is performed to establish both an inbound
and an outbound IPsec SA. The traffic to be protected by an IPsec
SA is determined by a selector which has been proposed by the IKE
J. Pinkerton, et al. Expires - April 2004 [Page 35]
Internet-Draft RDDP/RDMAP Security October 2003
initiator and accepted by the IKE Responder. In IPsec transport
mode, the IPsec SA selector can be a "filter" or traffic classifier,
defined as the 5-tuple: <Source IP address, Destination IP address,
transport protocol (UDP/SCTP/TCP), Source port, Destination port>.
The successful establishment of a IKE Phase-2 SA results in the
creation of two uni-directional IPsec SAs fully qualified by the
tuple <Protocol (ESP/AH), destination address, SPI>.
The session keys for each IPsec SA are derived from a master key,
typically via a MODP Diffie-Hellman computation. Rekeying of an
existing IPsec SA pair is accomplished by creating two new IPsec
SAs, making them active, and then optionally deleting the older
IPsec SA pair. Typically the new outbound SA is used immediately,
and the old inbound SA is left active to receive packets for some
locally defined time, perhaps 30 seconds or 1 minute.
8.1.2 Introduction to SSL Limitations on RDMAP
8.1.3 Applications Which Provide Security
Issue: Guidance for application protocols like NFS which implement
security.
8.1.4 Authentication Only
8.1.5 Privacy
8.2 Recommendations for IPsec Encapsulation of RDDP
Issue: IPsec recommendations for RDMAP/DDP
This is work that is still to be done. Hopefully this wonÆt be
terribly complex. One possible thought on the approach:
a. Use IPsec ESP with authentication to provide authentication,
integrity and replay protection.
b. Use IKE for key management.
J. Pinkerton, et al. Expires - April 2004 [Page 36]
Internet-Draft RDDP/RDMAP Security October 2003
c. (optionally) use a non-null transform for encryption. This
should be something other than DES.
J. Pinkerton, et al. Expires - April 2004 [Page 37]
Internet-Draft RDDP/RDMAP Security October 2003
9 Summary Table of Attacks and Trust Models
Issue: Finish Summary table of Attacks/Trust Models
<editor: This section is under construction, and will be completed
in a future version of this document>
Rows are the attack (grouped into categories)
Columns are the:
* Sec - Section the attack is discussed
* Attack Name - short name for the attack
* Threat - threat type (DOS, etc)
* Columns labeled 1-5 are the Trust Model number (see section
5 Trust Models on page 15). Each entry has a value of +, -,
and R (research).
J. Pinkerton, et al. Expires - April 2004 [Page 38]
Internet-Draft RDDP/RDMAP Security October 2003
+-------+--------------------------+-------+---+---+---+---+---+
| Sec | Attack Name |Threat | 1 | 2 | 3 | 4 | 5 |
+-------+--------------------------+-------+---+---+---+---+---+
| 7.2.1 | STag use on different | Spoof | | | | | |
| | connection in same PD | | | | | | |
+-------+--------------------------+-------+---+---+---+---+---+
| 7.3.1 | Memory write outside of | Tamper| | | | | |
| | buffer range | | | | | | |
| 7.3.2 | Modify Buffer after | Tamper| | | | | |
| | contents ready | | | | | | |
+-------+--------------------------+-------+---+---+---+---+---+
| 7.4.1 | Probe memory outside of | ID | | | | | |
| | buffer bounds | | | | | | |
| 7.4.2 | Access stale data | ID | | | | | |
| 7.4.3 | Access buffer after | ID | | | | | |
| | transfer over | | | | | | |
| 7.4.4 | Unintended data access | ID | | | | | |
| | using valid STag | | | | | | |
| 7.4.5 | Using RDMA Read on a | ID | | | | | |
| | buffer meant only for | | | | | | |
| | RDMA Write | | | | | | |
| 7.4.6 | Using multiple STags to | ID | | | | | |
| | access the same buffer | | | | | | |
| 7.4.7 | Remote node loading | ID | | | | | |
| | firmware onto RNIC | | | | | | |
+-------+--------------------------+-------+---+---+---+---+---+
| 7.5.1 | RNIC resource consumption| DOS | | | | | |
| 7.5.2 | Resource consumption by | DOS | | | | | |
| | active processes | | | | | | |
| 7.5.3 | Resource consumption by | DOS | | | | | |
| | idle processes | | | | | | |
| 7.5.4 | Non-optimal code paths | DOS | | | | | |
+-------+--------------------------+-------+---+---+---+---+---+
| 7.6.1 | Loading firmware onto | Elev | | | | | |
| | RNIC | | | | | | |
+-------+--------------------------+-------+---+---+---+---+---+
Figure 2 - Summary Attacks and Trust Model Table
J. Pinkerton, et al. Expires - April 2004 [Page 39]
Internet-Draft RDDP/RDMAP Security October 2003
10 References
10.1 Normative References
[RFC2828] Shirley, R., "Internet Security Glossary", FYI 36, RFC
2828, May 2000.
[DDP] Shah, H., J. Pinkerton, R.Recio, and P. Culley, "Direct Data
Placement over Reliable Transports", Internet-Draft draft-ietf-
rddp-ddp-01.txt, February 2003.
[RDMAP] Recio, R., P. Culley, D. Garcia, J. Hilland, "An RDMA
Protocol Specification", Internet-Draft draft-ietf-rddp-rdmap-
01.txt, February 2003.
[SEC-CONS] Rescorla, E., B. Korver, IAB, "Guidelines for Writing
RFC Text on Security Considerations", Internet-Draft draft-ab-
sec-cons-03.txt, January 2003.
[RFC2401] Atkinson, R. and Kent, S., "Security Architecture for the
Internet Protocol", RFC 2401, November 1998
[RFC2402] Kent, S., Atkinson, R., "IP Authentication Header", RFC
2402, November 1998
[RFC2404] Madson, C., Glenn, R., "The Use of HMAC-SHA-1-96 within
ESP and AH", RFC 2404, November 1998
[RFC2406] Kent, S., Atkinson, R., "IP Encapsulating Security
Payload (ESP)", RFC 2406, November 1998
[RFC2407] Piper, D., "The Internet IP Security Domain of
Interpretation of ISAKMP", RFC 2407, November 1998
[RFC2408] Maughan, D., Schertler, M., Schneider, M., Turner, J.,
"Internet Security Association and Key Management Protocol
(ISAKMP), RFC 2408, November 1998
[RFC2409] Harkins, D., Carrel, D., "The Internet Key Exchange
(IKE)", RFC 2409, November 1998
10.2 Informative References
[IPv6-Trust] Nikander, P., J.Kempf, E. Nordmark, "IPv6 Neighbor
Discovery trust modelsTrust Models and threats", Internet-Draft
draft-ietf-send-psreq-01.txt, January 2003.
J. Pinkerton, et al. Expires - April 2004 [Page 40]
Internet-Draft RDDP/RDMAP Security October 2003
11 AuthorÆs Addresses
James Pinkerton
Microsoft Corporation
One Microsoft Way
Redmond, WA. 98052 USA
Phone: +1 (425) 705-5442
Email: jpink@windows.microsoft.com
Ellen Deleganes
Intel Corporation
MS JF5-355
2111 NE 25th Ave.
Hillsboro, OR 97124 USA
Phone: +1 (503) 712-4173
Email: ellen.m.deleganes@intel.com
Allyn Romanow
Cisco Systems
170 W Tasman Drive
San Jose, CA 95134 USA
Phone: +1 408 525 8836
Email: allyn@cisco.com
Bernard Aboba
Microsoft Corporation
One Microsoft Way
Redmond, WA. 98052 USA
Phone: +1 (425) 706-6606
Email: bernarda@windows.microsoft.com
J. Pinkerton, et al. Expires - April 2004 [Page 41]
Internet-Draft RDDP/RDMAP Security October 2003
12 Acknowledgments
Catherine Meadows
Naval Research Laboratory
Code 5543
Washington, DC 20375
Email: meadows@itd.nrl.navy.mil
Patricia Thaler
Agilent Technologies, Inc.
1101 Creekside Ridge Drive, #100
M/S-RG10
Roseville, CA 95678
Phone: +1-916-788-5662
email: pat_thaler@agilent.com
James Livingston
NEC Solutions (America), Inc.
7525 166th Ave. N.E., Suite D210
Redmond, WA 98052-7811
Phone: +1 (425) 897-2033
Email: james.livingston@necsam.com
John Carrier
Adaptec, Inc.
691 S. Milpitas Blvd.
Milpitas, CA 95035 USA
Phone: +1 (360) 378-8526
Email: john_carrier@adaptec.com
Caitlin Bestler
Email: cait@asomi.com
J. Pinkerton, et al. Expires - April 2004 [Page 42]
Internet-Draft RDDP/RDMAP Security October 2003
13 Full Copyright Statement
Copyright (C) The Internet Society (2001). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph
are included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Funding for the RFC Editor function is currently provided by the
Internet Society.
J. Pinkerton, et al. Expires - April 2004 [Page 43]
| PAFTECH AB 2003-2026 | 2026-04-19 08:03:52 |