draft-ietf-rddp-security-00.txt   draft-ietf-rddp-security-01.txt 
Internet Draft James Pinkerton Internet Draft James Pinkerton
Document: draft-ietf-rddp-security-00.txt Microsoft Corporation draft-ietf-rddp-security-01.txt Microsoft Corporation
Expires: April, 2004 Ellen Deleganes Expires: September, 2004 Ellen Deleganes
Intel Corporation Intel Corporation
Allyn Romanow Allyn Romanow
Cisco Systems Cisco Systems
Bernard Aboba Sara Bitan
Microsoft Corporation Microsoft Corporation
October 2003 February 2004
DDP/RDMAP Security DDP/RDMAP Security
1 Status of this Memo 1 Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance
all provisions of Section 10 of RFC2026. with all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other
at any time. It is inappropriate to use Internet-Drafts as documents at any time. It is inappropriate to use Internet-
reference material or to cite them other than as "work in progress." Drafts as reference material or to cite them other than as "work
in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
2 Abstract 2 Abstract
This document analyzes security issues around implementation and use This document analyzes security issues around implementation and
of the Direct Data Placement Protocol(DDP) and Remote Direct Memory use of the Direct Data Placement Protocol(DDP) and Remote Direct
Access Protocol (RDMAP). It first defines an architectural model for Memory Access Protocol (RDMAP). It first defines an architectural
an RDMA Network Interface Card (RNIC), which can implement DDP or model for an RDMA Network Interface Card (RNIC), which can
RDMAP and DDP. The model includes a definition of resources that can implement DDP or RDMAP and DDP. The document reviews various
be attacked. This document then introduces various Trust Models attacks against the resources defined in the architectural model
between a local peer and a remote peer and the tools that can be and the countermeasures that can be used to protect the system.
used to create countermeasures against attacks. Finally, the Attacks are grouped into spoofing, tampering, information
document reviews various attacks and the countermeasures to be used disclosure, denial of service, and elevation of privilege.
against them, grouping the attacks into spoofing, tampering, Finally, the document concludes with a summary of security
information disclosure, denial of service, and elevation of services for RDDP, such as IPSec.
privilege.
J. Pinkerton, et al. Expires September 2004 1
Table of Contents Table of Contents
1 Status of this Memo.........................................1 1 Status of this Memo.........................................1
2 Abstract....................................................1 2 Abstract....................................................1
2.1 Issues......................................................3 2.1 Issues......................................................3
3 Introduction................................................5 2.2 Revision History............................................4
4 Architectural Model.........................................7 2.2.1 Changes from the -00 to -01 version........................4
4.1 Components..................................................8 3 Introduction................................................6
4.2 Resources..................................................10 4 Architectural Model.........................................8
4.2.1 Connection Context Memory.................................10 4.1 Components..................................................9
4.2.2 Data Buffers..............................................10 4.2 Resources..................................................11
4.2.1 Stream Context Memory.....................................11
4.2.2 Data Buffers..............................................11
4.2.3 Page Translation Tables...................................11 4.2.3 Page Translation Tables...................................11
4.2.4 STag Namespace............................................11 4.2.4 STag Namespace............................................12
4.2.5 Completion Queues.........................................11 4.2.5 Completion Queues.........................................12
4.2.6 RDMA Read Request Queue...................................11 4.2.6 Asynchronous Event Queue..................................12
4.2.7 RDMA Asynchronous Event Queue.............................11 4.2.7 RDMA Read Request Queue...................................13
4.2.8 RNIC Control Interactions.................................12 4.2.8 RNIC Interactions.........................................13
4.2.9 Initialization of RNIC Data Structures for Data Transfer..12 4.2.8.1 Privileged Control Interface Semantics................13
4.2.10 RNIC Data Transfer Interactions.........................13 4.2.8.2 Non-Privileged Data Interface Semantics...............13
4.3 System Properties..........................................14 4.2.8.3 Privileged Data Interface Semantics...................14
5 Trust Models...............................................15 4.2.9 Initialization of RNIC Data Structures for Data Transfer..14
6 Attacker Capabilities......................................17 4.2.10 RNIC Data Transfer Interactions.........................15
7 Attacks and Countermeasures................................18 5 Trust and Resource Sharing.................................17
7.1 Tools for Countermeasures..................................18 6 Attacker Capabilities......................................18
7.1.1 Protection Domain (PD)....................................18 7 Attacks and Countermeasures................................19
7.1.2 Limiting STag Scope.......................................19 7.1 Tools for Countermeasures..................................19
7.1.3 Access Rights.............................................19 7.1.1 Protection Domain (PD)....................................19
7.1.4 Limiting the Scope of the Completion Queue................20 7.1.2 Limiting STag Scope.......................................20
7.1.5 Limiting the Scope of an Error............................20 7.1.3 Access Rights.............................................21
7.2 Spoofing...................................................20 7.1.4 Limiting the Scope of the Completion Queue................21
7.2.1 Connection Hijacking......................................20 7.1.5 Limiting the Scope of an Error............................21
7.2.2 Using an STag on a different connection...................21 7.2 Spoofing...................................................21
7.3 Tampering..................................................22 7.2.1 Impersonation.............................................22
7.3.1 RDMA Write or Read Response to Memory Outside the Buffer..22 7.2.2 Stream Hijacking..........................................22
7.3.2 Modifying a Buffer After Indicating Contents Are Ready....23 7.2.3 Man in the Middle Attack..................................22
7.3.3 Using Multiple Stags to access the same buffer............23 7.2.4 Using an STag on a Different Stream.......................23
7.4 Information Disclosure.....................................23 7.3 Tampering..................................................24
7.4.1 Probing memory outside of the buffer bounds...............24 7.3.1 Buffer Overrun - RDMA Write or Read Response..............24
7.4.2 Using RDMA Read to Access Stale Data......................24 7.3.2 Modifying a Buffer After Indication.......................25
7.4.3 Accessing a buffer after the transfer is over.............24 7.3.3 Multiple STags to access the same buffer..................25
7.4.4 Accessing data within a valid STag that was unintended....24 7.3.4 Network based modification of buffer content..............25
7.4.5 Using RDMA Read on a buffer meant only for RDMA Write.....25 7.4 Information Disclosure.....................................25
7.4.6 Using Multiple Stags to access the same buffer............25 7.4.1 Probing memory outside of the buffer bounds...............26
7.4.7 Remote node loading firmware onto the RNIC................26 7.4.2 Using RDMA Read to Access Stale Data......................26
7.4.8 Controlling Access to Page Translation Table and STag Mapping 7.4.3 Accessing a Buffer After the Transfer.....................26
26 7.4.4 Accessing Unintended Data With a Valid STag...............26
7.5 Denial of Service (DOS)....................................26 7.4.5 RDMA Read into an RDMA Write Buffer.......................27
7.5.1 RNIC Resource Consumption.................................27 7.4.6 Using Multiple STags to Access One Buffer.................27
7.5.2 Resource Consumption By Active Applications...............27 7.4.7 Remote Node Loading Firmware onto the RNIC................28
7.5.2.1 Receive Data Buffers..................................28 7.4.8 Controlling Access to PTT & STag Mapping..................28
7.5.2.2 Completion Queue (CQ) Resource Consumption............29 7.4.9 Network based eaves dropping..............................28
7.5.2.3 RDMA Read Request Queue...............................31 7.5 Denial of Service (DOS)....................................28
7.5.3 Resource Consumption by Idle Applications.................32 7.5.1 RNIC Resource Consumption.................................29
7.5.4 Exercise of non-optimal code paths........................32 7.5.2 Resource Consumption By Active Applications...............30
7.5.5 Remote Invalidation of an STag Shared Across Multiple 7.5.2.1 Multiple Streams Sharing Receive Buffers..............30
Connections......................................................32 7.5.2.2 Local Peer Attacking a Shared CQ......................31
7.5.6 Remote Peer Consumes too many Untagged Receive Buffers....33 7.5.2.3 Remote Peer Attacking a Shared CQ.....................32
7.6 Elevation of Privilege.....................................33 7.5.2.4 RDMA Read Request Queue...............................34
7.6.1 Loading Firmware into the RNIC............................34 7.5.3 Resource Consumption by Idle Applications.................35
8 Security Services for RDDP.................................35 7.5.4 Exercise of non-optimal code paths........................35
8.1 Introduction to Security Options...........................35 7.5.5 RI an STag Shared on Multiple Streams.....................36
8.1.1 Introduction to IPsec.....................................35 7.5.6 Remote Peer Consumes Untagged Receive Buffers.............36
8.1.2 Introduction to SSL Limitations on RDMAP..................36 7.6 Elevation of Privilege.....................................36
8.1.3 Applications Which Provide Security.......................36 8 Security Services for RDDP.................................38
8.1.4 Authentication Only.......................................36 8.1 Introduction to Security Options...........................38
8.1.5 Privacy...................................................36 8.1.1 Introduction to IPsec.....................................39
8.2 Recommendations for IPsec Encapsulation of RDDP............36 8.1.2 Introduction to SSL Limitations on RDMAP..................40
9 Summary Table of Attacks and Trust Models..................38 8.1.3 Applications Which Provide Security.......................40
10 References.................................................40 8.2 Recommendations for IPsec Encapsulation of RDDP............40
10.1 Normative References......................................40 8.2.1 Transforms................................................41
10.2 Informative References....................................40 8.2.2 IPsec modes...............................................41
11 AuthorĂs Addresses.........................................41 8.2.3 IKE.......................................................41
12 Acknowledgments............................................42 8.2.4 Security Policy Configuration.............................43
13 Full Copyright Statement...................................43 9 Security considerations....................................45
10 References.................................................46
10.1 Normative References......................................46
10.2 Informative References....................................47
11 Appendix A: Implementing Client/Server Protocols...........48
12 Appendix B: Summary Table of Attacks.......................51
12.1 Spoofing..................................................52
12.2 Tampering.................................................52
12.3 Information Disclosure....................................52
12.4 Denial of Service.........................................52
13 Appendix C: Partial Trust Taxonomy.........................54
14 AuthorĂs Addresses.........................................56
15 Acknowledgments............................................57
16 Full Copyright Statement...................................58
Table of Figures Table of Figures
Figure 1 - RDMA Security Model....................................8 Figure 1 - RDMA Security Model....................................9
Figure 2 - Summary Attacks and Trust Model Table.................39 Figure 2 - Summary Attacks and Trust Model Table.................53
2.1 Issues 2.1 Issues
This section is temporary and will go away when all issues have been This section is temporary and will go away when all issues have
resolved. been resolved.
Note: this is far from a complete list of issues; as more are Note: this is far from a complete list of issues; as more are
raised, they will be added to this list until some sort of consensus raised, they will be added to this list until some sort of
is reached. They are in the order found in the specification. consensus is reached. They are in the order found in the
specification.
<TBD ű remove this section: this section was deleted because it
was a duplicate of Section 7.5.2.1 Multiple Streams Sharing
Receive Buffers on page 30) Thus comments on this section were
added to that section.>..........................................36
Issue: The spec currently makes specific IPsec SHOULD
recommendations. Should this be relaxed to not be normative,
since the protocol is just a transport protocol, not an
application protocol?............................................38
Issue: Guidance for application protocols like NFS which
implement security <TBD>.........................................40
Issue: I think we should refer to IPS security considerations.
Most of the issues discussed there are relevant for RDDP/RDMA as
well (exceptions are the discussion on user certificates).<TBD>..45
Issue: Finish Summary table of Attacks/Trust Models <TBD>........51
2.2 Revision History
2.2.1 Changes from the -00 to -01 version
* Added two pages to the architectural model to describe
the Asynchronous Event Queue, and the types of
interactions that can occur between the RNIC and the
modules above it.
* Addressed Mike Krauses comments submitted on 12/8/2003
* Moved "Trust Models" from the body of the document to an
appendix. Removed references to it throughout the
document (including use of "partial trust". Document now
assumes Remote Peer is untrusted. Thus the key issue is
whether local resources are shared, and what the resource
is.
* Misc cleanup throughout the document.
* The Summary of Attacks at the end of the document is now
an Appendix. It also now provides a summary. Cleared
change bars because became unreadable. Also shortened
section names for attacks to fit in table.
* Added a new concept of "Partial Mutual Trust" between a
collection of Streams to better characterize a set of
attacks in a client/server environment.
* Filled in Security Services for RDDP section (almost all
is new, except IPsec overview).
* Globally tried to change "connection" to "Stream". In
some cases it can be either a connection or stream.
Issue: Discuss issues in allowing the non-privileged consumer to
control mapping the Page Translation Table and Stag..............26
Issue: Need to analyze the case of sharing a queue of Untagged
receive buffers across multiple connections, and that the Remote
Peer can mount a denial of service attack........................33
Issue: Security Services section is a placeholder for now........35
Issue: Guidance for application protocols like NFS which implement
security.........................................................36
Issue: IPsec recommendations for RDMAP/DDP.......................36
Issue: Finish Summary table of Attacks/Trust Models..............38
3 Introduction 3 Introduction
RDMA enables new levels of flexibility when communicating between RDMA enables new levels of flexibility when communicating between
two parties compared to current conventional networking practice two parties compared to current conventional networking practice
(e.g. a stream-based model or datagram model). This flexibility (e.g. a stream-based model or datagram model). This flexibility
brings new security issues that must be carefully understood when brings new security issues that must be carefully understood when
designing application protocols utilizing RDMA and when implementing designing application protocols utilizing RDMA and when
RDMA-aware NICs (RNICs). Note that for the purposes of this security implementing RDMA-aware NICs (RNICs). Note that for the purposes
analysis, an RNIC may implement RDMAP and DDP, or just DDP. of this security analysis, an RNIC may implement RDMAP and DDP,
or just DDP.
The specification first develops an architectural model that is The specification first develops an architectural model that is
relevant for the security analysis - it details components, relevant for the security analysis - it details components,
resources, and system properties that may be attacked. The resources, and system properties that may be attacked in Section
specification then defines Partial Trust Models. Partial Trust is 4.
defined as:
Partial Trust - When one party is willing to assume that It then defines what resources a ULP may share locally across
another party will refrain from a specific attack or set of Streams and what resources the ULP may share with the Remote Peer
attacks, the parties are said to be in a state of Partial across Streams in Section 5. In general, intentional sharing of
Trust. Note that the partially trusted peer may attempt a resources between multiple Streams implies a trust model between
different set of attacks. This may be appropriate for many the Streams. This is defined as:
applications where any adverse effects of the betrayal is
easily confined and does not place other clients or
applications at risk.
An Untrusted Peer relationship is appropriate when an application Partial Mutual Trust ű a collection of RDMAP/DDP Streams,
which represent the local and remote end points of the
Stream, are willing to assume that the Streams from the
collection will not perform malicious attacks against any of
the Streams in the collection. Applications have explicit
control of which collection of endpoints is in the
collection through tools discussed in Section 7.1 Tools for
Countermeasures on page 19.
An untrusted peer relationship is appropriate when an application
wishes to ensure that it will be robust and uncompromised even in wishes to ensure that it will be robust and uncompromised even in
the face of a deliberate attack by its peer. For example, a single the face of a deliberate attack by its peer. For example, a
application that concurrently supports multiple unrelated sessions single application that concurrently supports multiple unrelated
would presumably treat each of its peers as an Untrusted Peer. sessions (e.g. a server) would presumably treat each of its peers
as an untrusted peer. For a collection of Streams which share
Partial Mutual Trust, the assumption is that any Stream not in
the collection is untrusted. For the untrusted peer, a brief list
of capabilities is enumerated in Section 6.
For the Untrusted peer, a brief list of capabilities is enumerated.
The rest of the specification is focused on analyzing attacks. The rest of the specification is focused on analyzing attacks.
First, the tools for mitigating attacks are listed, and then a First, the tools for mitigating attacks are listed (Section 7.1),
series of attacks on components, resources, or system properties is and then a series of attacks on components, resources, or system
enumerated. For each attack, possible countermeasures are reviewed. properties is enumerated in the rest of Section 7. For each
attack, possible countermeasures are reviewed. If all recommended
mitigations are in place the implemented usage models, the
RDMAP/DDP protocol can be shown to not expose any new security
vulnerabilities.
Applications within a host are divided into two categories - Applications within a host are divided into two categories -
Privileged and Non-Privileged. Both application types can send and Privileged and Non-Privileged. Both application types can send
receive data and request resources. The key differences between the and receive data and request resources. The key differences
two are: between the two are:
The Privileged Application is Partially Trusted. It is assumed The Privileged Application is trusted by the system to not
that the Privileged Application will not intentionally attack maliciously attack the operating environment, but it is not
the system (e.g., it is a kernel application), although it may trusted to optimize resource allocation globally. For
be greedy for resources. example, the Privileged Application could be a kernel
application, thus the kernel presumably has in some way
vetted the application before allowing it to execute.
A Non-Privileged ApplicationĂs capabilities are a logical sub- A Non-Privileged ApplicationĂs capabilities are a logical
set of the Privileged ApplicationĂs. It is assumed by the local sub-set of the Privileged ApplicationĂs. It is assumed by
host infrastructure that a Non-Privileged Application is the local system that a Non-Privileged Application is
Untrusted. All Non-Privileged Application interactions with the untrusted. All Non-Privileged Application interactions with
RNIC Engine that could affect other applications need to be the RNIC Engine that could affect other applications need to
done through a Trusted intermediary that can verify the Non- be done through a trusted intermediary that can verify the
Privileged Application requests. Non-Privileged Application requests.
4 Architectural Model 4 Architectural Model
This section describes an RDMA architectural reference model that is This section describes an RDMA architectural reference model that
used as security issues are examined. It introduces the components is used as security issues are examined. It introduces the
of the model, the resources that can be attacked, the types of components of the model, the resources that can be attacked, the
interactions possible between components and resources, and the types of interactions possible between components and resources,
system properties, which should be preserved when under attack. and the system properties, which should be preserved when under
attack.
Figure 1 shows the components comprising the architecture and the Figure 1 shows the components comprising the architecture and the
interfaces where potential security attacks could be launched. interfaces where potential security attacks could be launched.
External attacks can be injected into the system from an application External attacks can be injected into the system from an
that sits above the RI or from the Internet. application that sits above the RI or from the Internet.
The intent here is to describe high level components and The intent here is to describe high level components and
capabilities which affect threat analysis, and not focus on specific capabilities which affect threat analysis, and not focus on
implementation options. Also note that the architectural model is an specific implementation options. Also note that the architectural
abstraction, and an actual implementation may choose to subdivide model is an abstraction, and an actual implementation may choose
its components along different boundary lines than defined here. For to subdivide its components along different boundary lines than
example, the Privileged Resource Manager may be partially or defined here. For example, the Privileged Resource Manager may be
completely encapsulated in the Privileged Application. Regardless, partially or completely encapsulated in the Privileged
it is expected that the security analysis of the potential threats Application. Regardless, it is expected that the security
and countermeasures still apply. analysis of the potential threats and countermeasures still
apply.
+-------------+ Request Proxy Interface +-------------+
| Privileged |<--------------------------------+ | Privileged |
| Resource | | | Resource |
Admin<-->| Manager | App Control Interface | Admin<-+>| Manager | App Control Interface
| |<------+-------------------+ | | | |<------+-------------------+
+-------------+ | | | | +-------------+ | |
^ v v v | ^ v v
| +-------------+ +-----------------+ | | +-------------+ +-----------------+
| | Privileged | | Non-Privileged | |---------------->| Privileged | | Non-Privileged |
| | Application | | Application | | | Application | | Application |
| +-------------+ +-----------------+ | +-------------+ +-----------------+
| ^ ^ | ^ ^
|Privileged |Privileged |Non-Privileged |Privileged |Privileged |Non-Privileged
|Control |Data |Data |Control |Data |Data
|Interface |Interface |Interface |Interface |Interface |Interface
RNIC | | | RNIC | | |
Interface(RI) v v v Interface(RI) v v v
================================================================= =================================================================
+-----------------------------------------+ +--------------------------------------+
| | | |
| RNIC Engine | <------ Firmware | RNIC Engine | <-- Firmware
| | | |
+-----------------------------------------+ +--------------------------------------+
^ ^
| |
v v
Internet Internet
Figure 1 - RDMA Security Model Figure 1 - RDMA Security Model
4.1 Components 4.1 Components
The components shown in Figure 1 - RDMA Security Model are: The components shown in Figure 1 - RDMA Security Model are:
* RNIC Engine - the component that implements the RDMA * RNIC Engine - the component that implements the RDMA
protocol and/or DDP protocol. protocol and/or DDP protocol.
* Privileged Resource Manager - the component responsible for * Privileged Resource Manager - the component responsible
managing and allocating resources associated with the RNIC for managing and allocating resources associated with the
Engine. The Resource Manager does not send or receive data. RNIC Engine. The Resource Manager does not send or
Note that whether the Resource Manager is an independent receive data. Note that whether the Resource Manager is
component, part of the RNIC, or part of the application is an independent component, part of the RNIC, or part of
implementation dependent. If a specific implementation does the application is implementation dependent. If a
not wish to address security issues resolved by the Resource specific implementation does not wish to address security
Manager, there may in fact be no resource manager at all. issues resolved by the Resource Manager, there may in
fact be no resource manager at all.
* Privileged Application - See Section 3 Introduction for a * Privileged Application - See Section 3 Introduction for a
definition of Privileged Application. The local host definition of Privileged Application. The local host
infrastructure can enable the Privileged Application to map infrastructure can enable the Privileged Application to
a data buffer directly from the RNIC Engine to the host map a data buffer directly from the RNIC Engine to the
through the RNIC Interface, but it does not allow the host through the RNIC Interface, but it does not allow
Privileged Application to directly consume RNIC Engine the Privileged Application to directly consume RNIC
resources. Engine resources.
* Non-Privileged Application - See Section 3 Introduction for * Non-Privileged Application - See Section 3 Introduction
a definition of Non-Privileged Application. All Non- for a definition of Non-Privileged Application. All Non-
Privileged Application interactions with the RNIC Engine Privileged Application interactions with the RNIC Engine
that could affect other applications MUST be done using the that could affect other applications MUST be done using
Privileged Resource Manager as a proxy. the Privileged Resource Manager as a proxy.
A design goal of the DDP and RDMAP protocols is to allow, under A design goal of the DDP and RDMAP protocols is to allow, under
constrained conditions, Non-Privileged applications to send and constrained conditions, Non-Privileged applications to send and
receive data directly to/from the RDMA Engine without Privileged receive data directly to/from the RDMA Engine without Privileged
Resource Manager intervention - while ensuring that the host remains Resource Manager intervention - while ensuring that the host
secure. Thus, one of the primary goals of this paper is to analyze remains secure. Thus, one of the primary goals of this paper is
this usage model for the enforcement that is required in the RNIC to analyze this usage model for the enforcement that is required
Engine to ensure the system remains secure. in the RNIC Engine to ensure the system remains secure.
The host interfaces that could be exercised include: The host interfaces that could be exercised include:
* Control Interface - A Privileged Resource Manager uses the * Privileged Control Interface - A Privileged Resource
RI to allocate and manage RNIC Engine resources, control the Manager uses the RI to allocate and manage RNIC Engine
state within the RNIC Engine, and monitor various events resources, control the state within the RNIC Engine, and
from the RNIC Engine. It also uses this interface to act as monitor various events from the RNIC Engine. It also uses
a proxy for some operations that a Non-Privileged this interface to act as a proxy for some operations that
Application may require (after performing appropriate a Non-Privileged Application may require (after
countermeasures). performing appropriate countermeasures).
* Application Control Interface ű An application uses this
interface to the Privileged Resource Manager to allocate
RNIC Engine resources. The Privileged Resource Manager
implements countermeasures to ensure that if the Non-
Privileged Application launches an attack it can prevent
the attack from affecting other applications.
* Non-Privileged Data Transfer Interface - A Non-Privileged * Non-Privileged Data Transfer Interface - A Non-Privileged
Application uses this interface to initiate and to check the Application uses this interface to initiate and to check
status of data transfer operations. the status of data transfer operations.
* Privileged Data Transfer Interface - A superset of the * Privileged Data Transfer Interface - A superset of the
functionality provided by the Non-Privileged Data Transfer functionality provided by the Non-Privileged Data
Interface. The application is allowed to directly manipulate Transfer Interface. The application is allowed to
RNIC Engine mapping resources to map an STag to an directly manipulate RNIC Engine mapping resources to map
application data buffer. an STag to an application data buffer.
* Request Proxy Interface - a Non-Privileged Application uses
this interface to control RNIC Engine resources that could
affect other applications - such as manipulating the RNIC
Engine's mapping of an STag to an application data buffer.
The Privileged Resource Manager implements countermeasures
to ensure that if the Non-Privileged Application launches an
attack it can prevent the attack from affecting other
applications.
* Figure 1 also shows the ability to load new firmware in the * Figure 1 also shows the ability to load new firmware in
RNIC Engine. Not all RNICs will support this, but it is the RNIC Engine. Not all RNICs will support this, but it
shown for completeness and is also reviewed under potential is shown for completeness and is also reviewed under
attacks. potential attacks.
If Internet control messages, such as ICMP, ARP, RIPv4, etc. are If Internet control messages, such as ICMP, ARP, RIPv4, etc. are
processed by the RNIC Engine, the threat analyses for those processed by the RNIC Engine, the threat analyses for those
protocols is also applicable, but outside the scope of this paper. protocols is also applicable, but outside the scope of this
paper.
4.2 Resources 4.2 Resources
This section describes the primary resources in the RNIC Engine that This section describes the primary resources in the RNIC Engine
could be affected if under attack. For RDMAP, all of the defined that could be affected if under attack. For RDMAP, all of the
resources apply. For DDP, all of the resources except the RDMA Read defined resources apply. For DDP, all of the resources except the
Queue apply. RDMA Read Queue apply.
4.2.1 Connection Context Memory 4.2.1 Stream Context Memory
The state information for each connection is maintained in memory, The state information for each Stream is maintained in memory,
which could be located in a number of places - on the NIC, inside which could be located in a number of places - on the NIC, inside
RAM attached to the NIC, in host memory, or in any combination of RAM attached to the NIC, in host memory, or in any combination of
the three, depending on the implementation. the three, depending on the implementation.
Connection Context Memory includes state associated with Data Stream Context Memory includes state associated with Data
Buffers. For Tagged Buffers, this includes how STag names, Data Buffers. For Tagged Buffers, this includes how STag names, Data
Buffers, and Page Translation Tables inter-relate. It also includes Buffers, and Page Translation Tables inter-relate. It also
the FIFO list of Untagged Data Buffers posted for reception of includes the FIFO list of Untagged Data Buffers posted for
Untagged Messages (referred to in some contexts as the Receive reception of Untagged Messages (referred to in some contexts as
Queue), and a list of operations to perform to send data (referred the Receive Queue), and a list of operations to perform to send
to in some contexts as the Send Queue). data (referred to in some contexts as the Send Queue).
4.2.2 Data Buffers 4.2.2 Data Buffers
There are two different ways to expose a data buffer; a buffer can There are two different ways to expose a data buffer; a buffer
be exposed for receiving RDMAP Send Type Messages (a.k.a. DDP can be exposed for receiving RDMAP Send Type Messages (a.k.a. DDP
Untagged Messages) on DDP Queue zero or the buffer can be exposed Untagged Messages) on DDP Queue zero or the buffer can be exposed
for remote access through STags (a.k.a. DDP Tagged Messages). This for remote access through STags (a.k.a. DDP Tagged Messages).
distinction is important because the attacks and the countermeasures This distinction is important because the attacks and the
used to protect against the attack are different depending on the countermeasures used to protect against the attack are different
method for exposing the buffer to the Internet. depending on the method for exposing the buffer to the Internet.
For the purposes of the security discussion, a single logical Data For the purposes of the security discussion, a single logical
Buffer is exposed with a single STag. Actual implementations may Data Buffer is exposed with a single STag. Actual implementations
support scatter/gather capabilities to enable multiple physical data may support scatter/gather capabilities to enable multiple
buffers to be accessed with a single STag, but from a threat physical data buffers to be accessed with a single STag, but from
analysis perspective it is assumed that a single STag enables access a threat analysis perspective it is assumed that a single STag
to a single logical Data Buffer. enables access to a single logical Data Buffer.
In any event, it is the responsibility of the RI to ensure that no In any event, it is the responsibility of the RI to ensure that
STag can be created that exposes memory that the consumer had no no STag can be created that exposes memory that the consumer had
authority to expose. no authority to expose.
4.2.3 Page Translation Tables 4.2.3 Page Translation Tables
Page Translation Tables are the structures used by the RNIC to be Page Translation Tables are the structures used by the RNIC to be
able to access application memory for data transfer operations. Even able to access application memory for data transfer operations.
though these structures are called "Page" Translation Tables, they
may not reference a page at all - conceptually they are used to map Even though these structures are called "Page" Translation
an application address space representation of a buffer to the Tables, they may not reference a page at all - conceptually they
physical addresses that are used by the RNIC Engine to move data. If are used to map an application address space representation of a
on a specific system, a mapping is not used, then a subset of the buffer to the physical addresses that are used by the RNIC Engine
attacks examined may be appropriate. to move data. If on a specific system, a mapping is not used,
then a subset of the attacks examined may be appropriate.
4.2.4 STag Namespace 4.2.4 STag Namespace
The DDP specification defines a 32-bit namespace for the STag. The DDP specification defines a 32-bit namespace for the STag.
Implementations may vary in terms of the actual number of STags that Implementations may vary in terms of the actual number of STags
are supported. In any case, this is a bounded resource that can come that are supported. In any case, this is a bounded resource that
under attack. Depending upon Stag namespace allocation algorithms, can come under attack. Depending upon STag namespace allocation
the actual name space to attack may be significantly less than 2^32. algorithms, the actual name space to attack may be significantly
less than 2^32.
4.2.5 Completion Queues 4.2.5 Completion Queues
Completion Queues are used in this specification to conceptually Completion Queues are used in this specification to conceptually
represent how the RNIC Engine notifies the Application of the represent how the RNIC Engine notifies the Application about the
completion of the transmission of data, or the completion of the completion of the transmission of data, or the completion of the
reception of data through the Data Transfer Interface. Because there reception of data through the Data Transfer Interface. Because
could be many transmissions or receptions in flight at any one time, there could be many transmissions or receptions in flight at any
completions are modeled as a queue rather than a single event. An one time, completions are modeled as a queue rather than a single
implementation may also use the Completion Queue to notify the event. An implementation may also use the Completion Queue to
application of other activities, for example, the completion of a notify the application of other activities, for example, the
mapping of an STag to a specific application buffer. completion of a mapping of an STag to a specific application
buffer. Completion Queues may be shared by a group of Streams, or
4.2.6 RDMA Read Request Queue may be designated to handle a specific Stream's traffic.
The RDMA Read Request Queue is the memory holding state information Some implementations may allow this queue to be manipulated
for one or more RDMA Read Request Messages that have arrived, but directly by both Non-Privileged and Privileged applications.
for which the RDMA Read Response Messages have not yet been
completely sent.
4.2.7 RDMA Asynchronous Event Queue 4.2.6 Asynchronous Event Queue
The Asynchronous Event Queue is a queue from the RNIC to the The Asynchronous Event Queue is a queue from the RNIC to the
Privileged Resource Manager of bounded size. It is used by the RNIC Privileged Resource Manager of bounded size. It is used by the
to notify the host of various events which might require management RNIC to notify the host of various events which might require
action, including protocol violations, connection state changes, management action, including protocol violations, Stream state
local operation errors, low water marks on receive queues, and changes, local operation errors, low water marks on receive
possibly other events. queues, and possibly other events.
The RDMA Event Queue is a resource that can be attacked because The Asynchronous Event Queue is a resource that can be attacked
Remote or Local Peers can cause events to occur which have the because Remote or Local Peers can cause events to occur which
potential of overflowing the queue. have the potential of overflowing the queue.
Note that an implementation is at liberty to implement the functions Note that an implementation is at liberty to implement the
of the RDMA Asynchronous Event Queue in a variety of ways, including functions of the Asynchronous Event Queue in a variety of ways,
multiple queues or even simple callbacks. All vulnerabilities including multiple queues or even simple callbacks. All
identified for a single Asynchronous Event Queue apply to specific- vulnerabilities identified are intended to apply regardless of
purpose subsets. A callback function is simply a very short queue. the implementation of the Asynchronous Event Queue. For example,
a callback function is simply a very short queue.
4.2.8 RNIC Control Interactions 4.2.7 RDMA Read Request Queue
The RDMA Read Request Queue is the memory that holds state
information for one or more RDMA Read Request Messages that have
arrived, but for which the RDMA Read Response Messages have not
yet been completely sent. Because potentially more than one RDMA
Read Request can be outstanding at one time, the memory is
modeled as a queue of bounded size.
4.2.8 RNIC Interactions
With RNIC resources and interfaces defined, it is now possible to With RNIC resources and interfaces defined, it is now possible to
examine the interactions supported by the generic RNIC functional examine the interactions supported by the generic RNIC functional
interfaces through each of the 3 interfaces - Privileged Control interfaces through each of the 3 interfaces - Privileged Control
Interface, Privileged Data Interface, and Non-Privileged Data Interface, Privileged Data Interface, and Non-Privileged Data
Interface. Interface.
4.2.8.1 Privileged Control Interface Semantics
Generically, the Privileged Control Interface controls the RNICĂs Generically, the Privileged Control Interface controls the RNICĂs
allocation, deallocation, and initialization of RNIC global allocation, deallocation, and initialization of RNIC global
resources. This includes allocation and deallocation of Connection resources. This includes allocation and deallocation of Stream
Context Memory, Page Translation Tables, STag names, Completion Context Memory, Page Translation Tables, STag names, Completion
Queues, and RDMA Read Request Queues. Queues, RDMA Read Request Queues, and Asynchronous Event Queues.
Event Queue The Privileged Control Interface is also typically used for
managing Non-Privileged Application resources for the Non-
Privileged Application (and possibly for the Privileged
Application as well). This includes initialization and removal of
Page Translation Table resources, and managing RNIC events
(possibly managing all events for the Asynchronous Event Queue).
Initialization and removal of Page Translation Table resources. 4.2.8.2 Non-Privileged Data Interface Semantics
The Non-Privileged Data Interface enables data transfer (transmit
and receive) but does not allow initialization of the Page
Translation Table resources. However, once the Page Translation
Table resources have been initialized, the interface may enable a
specific STag mapping to be enabled and disabled by directly
communicating with the RNIC, or create an STag mapping for a
buffer that has been previously initialized in the RNIC.
For RDMAP, transmitting data means sending RDMAP Send Type
Messages, RDMA Read Requests, and RDMA Writes. For data
reception, for RDMAP it can receive Send Type Messages into
buffers that have been posted on the Receive Queue or Shared
Receive Queue. It can also receive RDMA Write and RDMA Read
Response Messages into buffers that have previously been exposed
for external write access through advertisement of an STag.
For DDP, transmitting data means sending DDP Tagged or Untagged
Messages. For data reception, for DDP it can receive Untagged
Messages into buffers that have been posted on the Receive Queue
or Shared Receive Queue. It can also receive Tagged DDP Messages
into buffers that have previously been exposed for external write
access through advertisement of an STag.
Completion of data transmission or reception generally entails
informing the application of the completed work by placing
completion information on the Completion Queue.
4.2.8.3 Privileged Data Interface Semantics
The Privileged Data Interface semantics are a superset of the
Non-Privileged Data Transfer semantics. The interface can do
everything defined in the prior section, as well as
create/destroy buffer to STag mappings directly. This generally
entails initialization or clearing of Page Translation Table
state in the RNIC.
4.2.9 Initialization of RNIC Data Structures for Data Transfer 4.2.9 Initialization of RNIC Data Structures for Data Transfer
Initialization of the mapping between an STag and a Data Buffer can Initialization of the mapping between an STag and a Data Buffer
be viewed in the abstract as two separate opertions: can be viewed in the abstract as two separate opertions:
a. Initialization of the allocated Page Translation Table a. Initialization of the allocated Page Translation Table
entries with the location of the Data Buffer, and entries with the location of the Data Buffer, and
b. Initialization of a mapping from an allocated STag name to a b. Initialization of a mapping from an allocated STag name
set of Page Translation Table entry(s) or partial-entries. to a set of Page Translation Table entry(s) or partial-
entries.
Note that an implementation may not have a Page Translation Table Note that an implementation may not have a Page Translation Table
(i.e. it may support a direct mapping between an STag and a Data (i.e. it may support a direct mapping between an STag and a Data
Buffer). In this case threats and mitigations associated with the Buffer). In this case threats and mitigations associated with the
Page Translation Table are not relevent. Page Translation Table are not relevant.
Initialization of the contents of the Page Translation Table can be Initialization of the contents of the Page Translation Table can
done by either the Privileged Application or by the Privileged be done by either the Privileged Application or by the Privileged
Resource Manager as a proxy for the Non-Privileged Application. By Resource Manager as a proxy for the Non-Privileged Application.
definition the Non-Privileged Application is not trusted to directly By definition the Non-Privileged Application is not trusted to
manipulate the Page Translation Table. In general the concern is directly manipulate the Page Translation Table. In general the
that the Non-Privileged application may try to maliciously concern is that the Non-Privileged application may try to
initialize the Page Translation Table to access a buffer for which maliciously initialize the Page Translation Table to access a
it does not have permission. buffer for which it does not have permission.
The exact resource allocation algorithm for the Page Translation The exact resource allocation algorithm for the Page Translation
Table is outside the scope of this specification. It may be Table is outside the scope of this specification. It may be
allocated for a specific Data Buffer, or be allocated as a pooled allocated for a specific Data Buffer, or be allocated as a pooled
resource to be consumed by potentially multiple Data Buffers, or be resource to be consumed by potentially multiple Data Buffers, or
managed in some other way. This paper attempts to abstract be managed in some other way. This paper attempts to abstract
implementation dependent issues, and focus on higher level security implementation dependent issues, and focus on higher level
issues such as resource starvation and sharing of resources between security issues such as resource starvation and sharing of
connections. resources between Streams.
The next issue is how an STag name is associated with a Data Buffer. The next issue is how an STag name is associated with a Data
For the case of an Untagged Data Buffer, there is no wire visible Buffer. For the case of an Untagged Data Buffer, there is no wire
mapping between an STag name and a Data Buffer. Note that there may, visible mapping between an STag name and a Data Buffer. Note that
in fact, be a mapping that is not visible from the wire, but this is there may, in fact, be a mapping that is not visible from the
a local host specific issue which should be analyzed in the context wire, but this is a local host specific issue which should be
of local host implementation specific security analysis, and thus is analyzed in the context of local host implementation specific
outside the scope of this paper. security analysis, and thus is outside the scope of this paper.
For a Tagged Data Buffer, either the Privileged Application, the For a Tagged Data Buffer, either the Privileged Application, the
Non-Privileged Application, or the Privileged Resource Manager Non-Privileged Application, or the Privileged Resource Manager
acting on behalf of the Non-Privileged Resource Manager may acting on behalf of the Non-Privileged Resource Manager may
initialize a mapping from an STag to a Page Translation Table, or initialize a mapping from an STag to a Page Translation Table, or
may have the ability to simply enable/disable an existing STag to may have the ability to simply enable/disable an existing STag to
Page Translation Table mapping. There may also be multiple STag Page Translation Table mapping. There may also be multiple STag
names which map to a specific group of Page Translation Table names which map to a specific group of Page Translation Table
entries (or sub-entries). Specific security issues with this level entries (or sub-entries). Specific security issues with this
of flexibility are examined later. level of flexibility are examined later.
There are a variety of implementation options for initialization of There are a variety of implementation options for initialization
Page Translation Table entries and mapping an STag to a group of of Page Translation Table entries and mapping an STag to a group
Page Translation Table entries which have security repercussions. of Page Translation Table entries which have security
This includes support for separation of Mapping an STag verses repercussions. This includes support for separation of Mapping an
mapping a set of Page Translation Table entries, and support for STag verses mapping a set of Page Translation Table entries, and
Applications directly manipulating STag to Page Translation Table support for Applications directly manipulating STag to Page
entry mappings (verses requiring access through the Privileged Translation Table entry mappings (verses requiring access through
Resource Manager). the Privileged Resource Manager).
4.2.10 RNIC Data Transfer Interactions 4.2.10 RNIC Data Transfer Interactions
RNIC Data Transfer operations can be subdivided into send operations RNIC Data Transfer operations can be subdivided into send
and receive operations. operations and receive operations.
For send operations, there is typically a queue that enables the For send operations, there is typically a queue that enables the
Application to post multiple operations. Depending upon the Application to post multiple operations. Depending upon the
implementation, Data Buffers used in the operations may or may not implementation, Data Buffers used in the operations may or may
have Page Translation Table entries associated with them, and may or not have Page Translation Table entries associated with them, and
may not have STags associated with them. Because this is a local may or may not have STags associated with them. Because this is a
host specific implementation issue rather than a protocol issue, the local host specific implementation issue rather than a protocol
security analysis of threats and mitigations is left to the host issue, the security analysis of threats and mitigations is left
implementation. to the host implementation.
Receive operations are different for Tagged Data Buffers verses Receive operations are different for Tagged Data Buffers verses
Untagged Data Buffers. If more than one Untagged Data Buffer can be Untagged Data Buffers. If more than one Untagged Data Buffer can
posted by the Application, the DDP specification requires that they be posted by the Application, the DDP specification requires that
be consumed in FIFO order. Thus the most general implementation is they be consumed in FIFO order. Thus the most general
that there is a FIFO queue of receive Untagged Data Buffers. Some implementation is that there is a FIFO queue of receive Untagged
implementations may also support sharing of the FIFO queue between Data Buffers. Some implementations may also support sharing of
multiple connections. In this case defining ŠŠFIFOĂĂ becomes non- the FIFO queue between multiple Streams. In this case defining
trivial - in general the buffers for a single stream are consumed "FIFO" becomes non-trivial - in general the buffers for a single
from the queue in the order that they were placed on the queue, but stream are consumed from the queue in the order that they were
there is no order guarantee between streams. placed on the queue, but there is no order guarantee between
streams.
For receive Tagged Data Buffers, at some time prior to data For receive Tagged Data Buffers, at some time prior to data
transfer, the mapping of the STag to specific Page Translation Table transfer, the mapping of the STag to specific Page Translation
entries (if present) and the mapping from the Page Translation Table Table entries (if present) and the mapping from the Page
entries to the Data Buffer must have been initialized (see the prior Translation Table entries to the Data Buffer must have been
section for interaction details). initialized (see the prior section for interaction details).
4.3 System Properties
System properties that can be attacked included system integrity,
system stability (liveness, large fluctuations in performance), and
confidentiality.
5 Trust Models 5 Trust and Resource Sharing
The Trust Models described in this section have three primary It is assumed that in general the Local and Remote Peer are
distinguishing characteristics. untrusted, and thus attacks by either should have mitigations in
place.
* Local Resource Sharing (yes/no) - When local resources are A separate, but related issue is resource sharing between
shared, they are shared across a grouping of RDMAP/DDP multiple streams. If local resources are not shared, the
Streams. If local resources are not shared, the resources resources are dedicated on a per Stream basis. Resources are
are dedicated on a per Stream basis. Resources are defined defined in Section 4.2 - Resources on page 10. The advantage of
in Section 4.2 - Resources on page 10. The advantage of not not sharing resources between Streams is that it reduces the
sharing resources between Streams is that it reduces the
types of attacks that are possible. The disadvantage is that types of attacks that are possible. The disadvantage is that
applications might run out of resources. applications might run out of resources.
* Local Partial Trust (yes/no) - Local Partial Trust is It is assumed in this paper that the component that implements
determined based on whether the local grouping of RDMAP/DDP the mechanism to control sharing of RNIC Engine resources is the
Streams (which typically equates to one application or group Privileged Resource Manager. The RNIC Engine exposes its
of applications) mutually trust each other to not perform a resources through the RI to the Privileged Resource Manager. All
specific set of attacks. Privileged and Non-Privileged applications request resources from
the Resource Manager. The Resource Manager implements resource
* Remote Partial Trust (yes/no) - The Remote Partial Trust management policies to ensure fair access to resources. The
level is determined based on whether the Local Peer of a Resource Manager should be designed to take into account security
specific RDMAP/DDP Stream partially trusts the Remote Peer attacks detailed in this specification. Note that for some
of the Stream (see the definition of Partial Trust in systems the Privileged Resource Manager may be implemented within
Section 3 Introduction). the Privileged Application.
It is assumed in this paper that the component that implements the
mechanism to control sharing of RNIC Engine resources is the
Privileged Resource Manager. The RNIC Engine exposes its resources
through the RI to the Privileged Resource Manager. All Privileged
and Non-Privileged applications request resources from the Resource
Manager. The Resource Manager implements resource management
policies to ensure fair access to resources. The Resource Manager
should be designed to take into account security attacks detailed in
this specification.
The sharing of resources across connections should be under the The sharing of resources across Streams should be under the
control of the application, both in terms of the Trust Model the control of the application, both in terms of the trust model the
application wishes to operate under, as well as the level of application wishes to operate under, as well as the level of
resource sharing the application wishes to give Local Peer resource sharing the application wishes to give Local Peer
processes. processes. For more discussion on types of trust models which
combine partial trust and sharing of resources, see Appendix C:
Not all of the combinations of the trust characteristics are Partial Trust Taxonomy on page 54.
expected to be used by applications. This paper specifically
analyzes five application Trust Models that are expected to be in
common use. The Trust Models are as follows:
1. NS-NT - Non-Shared Local Resources, no Local Trust, no Remote
Trust - typically a server application that wants to run in a
mode that has the least number of potential attacks.
2. NS-RT - Non-Shared Local Resources, no Local Trust, Remote
Partial Trust - typically a peer-to-peer application, which has,
by some method outside of the scope of this specification,
authenticated the Remote Peer. Note that unless some form of key
based authentication is used on a per packet basis, it may not
be possible to bind the authentication result to the RDMA
packet.
3. S-NT - Shared Local Resources, no Local Trust, no Remote Trust -
typically a server application that runs in an untrusted
environment where the amount of resources required is either too
large or too dynamic to dedicate for each RDMAP/DDP Stream.
4. S-LT - Shared Local Resources, Local Partial Trust, no Remote
Trust - typically an application, which provides a session layer
and uses multiple Streams, to provide additional throughput or
fail-over capabilities. All of the Streams within the local
application partially trust each other, but do not trust the
remote peer.
5. S-T - Shared Local Resources, Local Partial Trust, Remote
Partial Trust - typically a distributed application, such as a
distributed database application or a High Performance Computer
(HPC) application, which is intended to run on a cluster. Due to
extreme resource and performance requirements, the application
typically authenticates with all of its peers and then runs in a
highly trusted environment. The application peers are all in a
single application fault domain and depend on one another to be
well-behaved when accessing data structures. If a trusted Remote
Peer has an implementation defect that results in poor behavior,
the entire application could be corrupted.
Models NS-NT and S-NT above are typical for Internet networking -
neither Local Peers nor the Remote Peer is trusted. Sometimes
optimizations can be done that enable sharing of Page Translation
Tables across multiple Local Peers, thus Model S-LT can be
advantageous. Model S-T is typically used when resource scaling
across a large parallel application makes it infeasible to use any
other model. Resource scaling issues can either be due to
performance around scaling or because there simply are not enough
resources. Model NS-RT is probably the least likely model to be
used, but is presented for completeness.
6 Attacker Capabilities 6 Attacker Capabilities
An attackerĂs capabilities delimit the types of attacks that An attackerĂs capabilities delimit the types of attacks that
attacker is able to launch. RDMAP and DDP require that the initial attacker is able to launch. RDMAP and DDP require that the
LLP Stream (and connection) be set up prior to transferring initial LLP Stream (and connection) be set up prior to
RDMAP/DDP Messages. For the attacker to actively generate an transferring RDMAP/DDP Messages. For the attacker to actively
RDMAP/DDP protocol attack, it must have the capability to both send generate an RDMAP/DDP protocol attack, it must have the
and receive messages. Attackers with send only capabilities should capability to both send and receive messages. Attackers with send
be addressed by the LLP, not by RDMAP/DDP. only capabilities should be addressed by the LLP, not by
RDMAP/DDP.
7 Attacks and Countermeasures 7 Attacks and Countermeasures
This section describes the attacks that are possible against the This section describes the attacks that are possible against the
RDMA system defined in Figure 1 - RDMA Security Model and the RNIC RDMA system defined in Figure 1 - RDMA Security Model and the
Engine resources defined in Section 4.2. The analysis includes a RNIC Engine resources defined in Section 4.2. The analysis
detailed description of each attack, the Trust Models the attack includes a detailed description of each attack, what is being
applies to (see Section 5 for a description of the Trust Models), attacked, and a description of the countermeasures that can be
and a description of the countermeasures appropriate to the Trust taken to thwart the attack.
Model(s) that can be taken to thwart the attack.
Note that, connection setup and teardown is presumed to be done in Note that connection setup and teardown is presumed to be done in
stream mode (i.e. no RDMA encapsulation of the payload), so there stream mode (i.e. no RDMA encapsulation of the payload), so there
are no new attacks related to connection setup/teardown beyond what are no new attacks related to connection setup/teardown beyond
is already present in the LLP (e.g. TCP or SCTP). Consequently, any what is already present in the LLP (e.g. TCP or SCTP).
existing analysis of Spoofing, Tampering, Repudiation, Information Consequently, any existing analysis of Spoofing, Tampering,
Disclosure, Denial of Service, or Elevation of Privilege continues Repudiation, Information Disclosure, Denial of Service, or
to apply. Thus, the analysis in this section focuses on attacks that Elevation of Privilege continues to apply. Thus, the analysis in
are present regardless of the LLP Stream type. this section focuses on attacks that are present regardless of
the LLP Stream type.
The attacks are classified into five categories: Spoofing,
Tampering, Information Disclosure, Denail of Service (DoS)
attacks, and Elevation of Privileges. Tampering is any
modification of the legitimate traffic (machine internal or
network). Spoofing attack is a special case of tempering; where
the attacker falsifies an identity of the Remote Peer (identity
can be an IP address, machine name, ULP level identity etc.).
7.1 Tools for Countermeasures 7.1 Tools for Countermeasures
The tools described in this section are the primary mechanisms that The tools described in this section are the primary mechanisms
can be used to provide countermeasures to potential attacks. that can be used to provide countermeasures to potential attacks.
7.1.1 Protection Domain (PD) 7.1.1 Protection Domain (PD)
Protection Domains are associated with two of the resources of Protection Domains are associated with two of the resources of
concern, connection context memory and STags associated with Page concern, Stream Context Memory and STags associated with Page
Translation Table entries and data buffers. Protection Domains are Translation Table entries and data buffers. Protection Domains
used mainly to ensure that an STag can only be used to access the are used mainly to ensure that an STag can only be used to access
associated data buffer through connections in the same Protection the associated data buffer through Streams in the same Protection
Domain as that STag. Domain as that STag.
For the Trust Models that are defined to have non-shared resources If an implementation chooses to not share resources between
(Trust Models NS-NT and NS-RT), it is recommended that each Stream Streams, it is recommended that each Stream be associated with
be associated with its own, unique Protection Domain. For those its own, unique Protection Domain. If an implementation chooses
Trust Models where resources are shared (Trust Models S-NT, S-LT and to allow resource sharing, it is recommended that Protection
S-T), it is recommended that Protection Domain be limited to the Domain be limited to the number of Streams that have Partial
number of Streams that share the same Trust Model. Mutual Trust.
Note that an application (either Privileged or Non-Privileged) can Note that an application (either Privileged or Non-Privileged)
potentially have multiple Protection Domains. This could be used, can potentially have multiple Protection Domains. This could be
for example, to ensure that multiple clients of a server do not have used, for example, to ensure that multiple clients of a server do
the ability to corrupt each other. This can apply to any combination not have the ability to corrupt each other. The server would
of the Trust Models, because Partial Trust does not imply complete allocate a Protection Domain per client to ensure that resources
trust. covered by the Protection Domain could not be used by another
(untrusted) client.
7.1.2 Limiting STag Scope 7.1.2 Limiting STag Scope
The key to protecting a local data buffer is to limit the scope of The key to protecting a local data buffer is to limit the scope
its STag to the level appropriate for the Trust Model. The scope of of its STag to the level appropriate for the Streams which share
the STag can be measured in multiple ways. Partial Mutual Trust. The scope of the STag can be measured in
multiple ways.
* Number of Connections and/or Streams on which the STag is * Number of Connections and/or Streams on which the STag is
valid. One way to limit the scope of the STag is to limit valid. One way to limit the scope of the STag is to limit
the connections and/or Streams that are allowed to use the the connections and/or Streams that are allowed to use
STag. As noted in the previous section, use of Protection the STag. As noted in the previous section, use of
Domains appropriately can limit the scope of the STag. It is Protection Domains appropriately can limit the scope of
also possible to create an STag that is valid only on a the STag. The analysis presented in this document assumes
single connection, even in the case where several two mechanisms for limiting the scope of Streams for
connections are associated with the Protection Domain of the which the STag is valid:
STag.
* Protection Domain scope. The STag is valid if used on
any Stream within a specific Protection Domain, and
is invalid if used on any Stream that is not a member
of the Protection Domain.
* Single Stream scope. The STag is valid on a single
Stream, regardless of what the Stream association is
to a Protection Domain. If used on any other Stream,
it is invalid.
* Limit the time an STag is valid. By Invalidating an * Limit the time an STag is valid. By Invalidating an
Advertised STag (e.g., revoking remote access to the buffers Advertised STag (e.g., revoking remote access to the
described by an STag when done with the transfer), an entire buffers described by an STag when done with the
class of attacks can be eliminated. transfer), an entire class of attacks can be eliminated.
* Limit the buffer the STag can reference. Limiting the scope * Limit the buffer the STag can reference. Limiting the
of an STag access to *just* the intended buffers to be scope of an STag access to *just* the intended
exposed is critical to prevent certain forms of attacks. application buffers to be exposed is critical to prevent
certain forms of attacks.
* Allocating Stag numbers in an unpredictable way. If STags * Allocating STag numbers in an unpredictable way. If STags
are allocated using an algorithm which makes it hard for the are allocated using an algorithm which makes it hard for
Remote Peer to guess which STag(s) are currently in use, it the Remote Peer to guess which STag(s) are currently in
makes it more difficult for an attacker to guess the correct use, it makes it more difficult for an attacker to guess
value. the correct value. As stated in the RDMAP specification
[RDMAP], an invalid STag will cause the RDMAP Stream to
be terminated. For the case of [DDP], at a minimum it
must signal an error to the ULP, and commonly this will
cause the DDP stream to be terminated.
7.1.3 Access Rights 7.1.3 Access Rights
Access Rights associated with a specific Advertised STag or Access Rights associated with a specific Advertised STag or
RDMAP/DDP Stream provide another mechanism for applications to limit RDMAP/DDP Stream provide another mechanism for applications to
the attack capabilities of the Remote Peer. The Local Peer can limit the attack capabilities of the Remote Peer. The Local Peer
control whether a data buffer is exposed for local only, or local can control whether a data buffer is exposed for local only, or
and remote access, and assign specific access privileges (read, local and remote access, and assign specific access privileges
write, read and write). (read, write, read and write) on a per stream or session basis.
For DDP, when an STag is advertised, the Remote Peer is presumably For DDP, when an STag is advertised, the Remote Peer is
given write access rights to the data (otherwise there was not much presumably given write access rights to the data (otherwise there
point to the advertisement). For RDMAP, when an application was not much point to the advertisement). For RDMAP, when an
advertises an STag, it can enable write-only, read-only, or both application advertises an STag, it can enable write-only, read-
write and read access rights. only, or both write and read access rights.
Similarly, some applications may wish to provide a single buffer Similarly, some applications may wish to provide a single buffer
with different access rights on a per-connection or per-Stream with different access rights on a per-Stream or per-Stream basis.
basis. For example, some connections may have read-only access, some For example, some Streams may have read-only access, some may
may have remote read and write access, while on other connections have remote read and write access, while on other Streams only
only the Local Peer is allowed access. the Local Peer is allowed access.
7.1.4 Limiting the Scope of the Completion Queue 7.1.4 Limiting the Scope of the Completion Queue
Completions associated with sending and receiving data, or setting Completions associated with sending and receiving data, or
up buffers for sending and receiving data, could be accumulated in a setting up buffers for sending and receiving data, could be
shared Completion Queue for a group of RDMAP/DDP Streams, or a accumulated in a shared Completion Queue for a group of RDMAP/DDP
specific RDMAP/DDP Stream could have a dedicated Completion Queue. Streams, or a specific RDMAP/DDP Stream could have a dedicated
Limiting Completion Queue association to one, or a small number of Completion Queue. Limiting Completion Queue association to one,
RDMAP/DDP Streams can prevent several forms of Denial of Service or a small number of RDMAP/DDP Streams can prevent several forms
attacks. of Denial of Service attacks.
7.1.5 Limiting the Scope of an Error 7.1.5 Limiting the Scope of an Error
To prevent a variety of attacks, it is important that an RDMAP/DDP To prevent a variety of attacks, it is important that an
implementation be robust in the face of errors. If an error on a RDMAP/DDP implementation be robust in the face of errors. If an
specific Stream can cause other unrelated Streams to fail, then a error on a specific Stream can cause other unrelated Streams to
broad class of attacks are enabled against the implementation. fail, then a broad class of attacks are enabled against the
implementation.
For example, an error on a specific RDMAP stream should not cause
the RNIC to stop processing incoming packets, or corrupt a
receive queue for an unrelated stream.
7.2 Spoofing 7.2 Spoofing
Spoofing attacks can be launched by the Remote Peer, or by a
network based attacker. A network based spoofing attack applies
to all Remote Peers.
Because the RDMAP Stream is only offloaded if it is in the Because the RDMAP Stream is only offloaded if it is in the
ESTABLISHED state, certain types of traditional forms of wire ESTABLISHED state, certain types of traditional forms of wire
attacks do not apply -- an end-to-end handshake must have occurred attacks do not apply -- an end-to-end handshake must have
to establish the RDMAP Stream. So, the only form of spoofing that occurred to establish the RDMAP Stream. So, the only form of
applies is one when a remote node can both send and receive packets. spoofing that applies is one when a remote node can both send and
receive packets. Yet even with this limitation the Stream is
still exposed to the following spoofing attacks.
7.2.1 Connection Hijacking 7.2.1 Impersonation
If a man-in-the-middle attacker has the ability to inject packets A network based attacker can impersonate a legal RDMA/RDDP peer
which will be accepted by the LLP (e.g., TCP sequence number is (by spoofing a legal IP address), and establish an RDMA/RDDP
correct) then the connection can essentially be hijacked. One style Stream with the victim. End to end authentication (i.e. IPsec,
of attack is for the man-in-the-middle to send Tagged Messages SSL or ULP authentication) provides protection against this
(either RDMAP or DDP). If it can discover a buffer that has been attack. For additional information see Section 8, Security
exposed for STag enabled access, then the man-in-the-middle can use Services for RDDP, on page 38.
an RDMA Read operation to read the contents of the associated data
buffer, perform an RDMA Write Operation to modify the contents of 7.2.2 Stream Hijacking
the associated data buffer, or invalidate the STag to disable
further access to the buffer. The only countermeasure for this form Stream Hijacking happens when a network based attacker follows
of attack is to either secure the RDMAP/DDP Stream or attempt to the session establishment phase, and waits until the
provide physical security to prevent man-in-the-middle type access. authentication phase (if such a phase exists) is completed
successfully. He can then spoof the IP address and re-direct the
Stream from the victim to its own machine. For example, an
attacker can wait until an iSCSI authentication is completed
successfully, and hijack the iSCSI Stream.
The best protection against this form of attack is end-to-end The best protection against this form of attack is end-to-end
integrity protection and authentication, such as IPsec (see Section session level integrity protection and authentication, such as
8 Security Services for RDDP on page 35), to prevent spoofing or IPsec (see Section 8, Security Services for RDDP, on page 38), to
tampering. If authentication is not used, then a man-in-the-middle prevent spoofing. Another option is to provide physical security.
attack can occur, enabling spoofing, tampering, and repudiation. Discussion of physical security is out of scope for this
document.
Because the connection itself is established by the LLP, some LLPs Because the connection and/or Stream itself is established by the
are more difficult to hijack than others. Please see the relevant LLP, some LLPs are more difficult to hijack than others. Please
LLP documentation on security issues around connection hijacking see the relevant LLP documentation on security issues around
<TBD: references for SCTP and TCP on connection hijacking>. connection and/or Stream hijacking <TBD: references for SCTP and
TCP on connection hijacking>.
7.2.3 Man in the Middle Attack
If a network based attacker has the ability to delete, inject
replay, or modify packets which will still be accepted by the LLP
(e.g., TCP sequence number is correct) then the Stream can be
exposed to a man in the middle attack. One style of attack is for
the man-in-the-middle to send Tagged Messages (either RDMAP or
DDP). If it can discover a buffer that has been exposed for STag
enabled access, then the man-in-the-middle can use an RDMA Read
operation to read the contents of the associated data buffer,
perform an RDMA Write Operation to modify the contents of the
associated data buffer, or invalidate the STag to disable further
access to the buffer. The only countermeasure for this form of
attack is to either secure the RDMAP/DDP Stream (i.e. integrity
protect) or attempt to provide physical security to prevent man-
in-the-middle type attacks.
The best protection against this form of attack is end-to-end
integrity protection and authentication, such as IPsec (see
Section 8 Security Services for RDDP on page 38), to prevent
spoofing or tampering. If Stream or session level authentication
and integrity protection are not used, then a man-in-the-middle
attack can occur, enabling spoofing and tampering.
Because the connection/Stream itself is established by the LLP,
some LLPs are more exposed to man-in-the-middle attack then
others. Please see the relevant LLP documentation on security
issues around connection and/or Stream hijacking <TBD: references
for SCTP and TCP on connection hijacking>.
Another approach is to restrict access to only the local Another approach is to restrict access to only the local
subnet/link, and provide some mechanism to limit access, such as subnet/link, and provide some mechanism to limit access, such as
physical security or 802.1.x. This model is an extremely limited physical security or 802.1.x. This model is an extremely limited
deployment scenario, and will not be further examined here. deployment scenario, and will not be further examined here.
7.2.2 Using an STag on a different connection 7.2.4 Using an STag on a Different Stream
One style of attack from the Remote Peer is for it to attempt to use One style of attack from the Remote Peer is for it to attempt to
STag values that it is not authorized to use. Note that, if the use STag values that it is not authorized to use. Note that if
Remote Peer sends an invalid STag to the Local Peer, per the DDP and the Remote Peer sends an invalid STag to the Local Peer, per the
RDMAP specifications, the connection must be torn down. Thus, the DDP and RDMAP specifications, the Stream must be torn down. Thus
threat exists if a STag has been enabled for Remote Access on one the threat exists if a STag has been enabled for Remote Access on
connection and a Remote Peer is able to use it on an unrelated one Stream and a Remote Peer is able to use it on an unrelated
connection. If the attack is successful, the attacker could Stream. If the attack is successful, the attacker could
potentially be able to perform either RDMA Read Operations to read potentially be able to perform either RDMA Read Operations to
the contents of the associated data buffer, perform RDMA Write read the contents of the associated data buffer, perform RDMA
Operations to modify the contents of the associated data buffer, or Write Operations to modify the contents of the associated data
to Invalidate the STag to disable further access to the buffer. buffer, or to Invalidate the STag to disable further access to
the buffer.
An attempt by a Remote Peer to access a buffer with an STag on a An attempt by a Remote Peer to access a buffer with an STag on a
different connection in the same Protection Domain may or may not be different Stream in the same Protection Domain may or may not be
an attack depending on the Trust Model employed by the application. an attack depending on whether resource sharing is intended
For Trust Model S-T, where resources are shared between connections, (i.e. whether the Streams shared Partial Mutual Trust or not).
and both Local and Remote Peers are Trusted, using an STag on For some applications using an STag on multiple Streams within
multiple connections within the same Protection Domain is allowed, the same Protection Domain could be desired behavior. For other
and could be desired behavior. For the other four Trust Models where applications attempting to use an STag on a different Stream
the Remote Peer is not Trusted, and/or resources are not intended to could be considered to be an attack. Since this varies by
be shared, attempting to use an STag on a different connection could application, an application typically would need to be able to
be considered to be an attack. control the scope of the STag.
In the case where the Trust Model is defined with no shared In the case where an implementation does not share resources
resources between connections (Trust Models NS-NT and NS-RT), this between Streams (including STags), this attack can be defeated by
attack can be defeated by assigning each connection to a different assigning each Stream to a different Protection Domain. Before
Protection Domain. Before allowing remote access to the buffer, the allowing remote access to the buffer, the Protection Domain of
Protection Domain of the connection where the access attempt was the Stream where the access attempt was made is matched against
made is matched against the Protection Domain of the STag. If the the Protection Domain of the STag. If the Protection Domains do
Protection Domains do not match, access to the buffer is denied, an not match, access to the buffer is denied, an error is generated,
error is generated, and the RDMAP Stream associated with the and the RDMAP Stream associated with the attacking Stream should
attacking connection should be terminated. Thus, for Trust Models be terminated.
NS-NT and NS-RT, it is RECOMMENDED that each connection be in a
separate Protection Domain.
For Trust Models S-NT and S-LT, where resources are shared, but the For implementations that share resources between multiple
Remote Peers are Untrusted, it may not be practical to separate each Streams, it may not be practical to separate each Stream into its
connection into its own Protection Domain. In this case, the own Protection Domain. In this case, the application can still
application can still limit the scope of any of the STags it is limit the scope of any of the STags to a single Stream (if it is
enabling for remote access to a single connection. If the STag scope enabling it for remote access). If the STag scope has been
has been limited to a single connection, any attempt to use that limited to a single Stream, any attempt to use that STag on a
STag on a different connection will result in an error, and the RDMA different Stream will result in an error, and the RDMA Stream
Stream associated with that connection should be terminated. Thus, should be terminated.
for Trust Models S-NT and S-LT, it is RECOMMENDED that the scope of
an STag be limited to a single connection.
For Trust Models S-NT and S-LT (Untrusted Remote Peers), if it is Thus for implementations that do not share STags between Streams
not possible to use Protection Domains or to limit the scope of an it is RECOMMENDED that either each Stream be in a separate
STag to a single connection, it is RECOMMENDED that STag allocators Protection Domain or the scope of an STag be limited to a single
select an STag using an algorithm which makes it difficult to guess Stream.
the next allocated STag number. This approach is good practice in
general. Allocation methods which always start with the same number An additional issue may be unintended sharing of STags (i.e. a
(e.g. zero) after Stream initialization or simply allocate the next bug in the application) or a bug in the Remote Peer which causes
STag in a monotonically increasing namespace should be avoided. an off-by-one STag to be used. For additional protection, it is
RECOMMENDED that the allocation of STags be done in such a
fashion that it is difficult to predict the next allocated STag
number. Allocation methods which deterministically allocate the
next STag should be avoided (e.g. a method which always starts
with STag equal to one and monotonically increases it for each
new allocation).
7.3 Tampering 7.3 Tampering
A Remote Peer can attempt to tamper with the contents of data A Remote Peer or a network based attacker can attempt to tamper
buffers on a Local Peer that have been enabled for remote write with the contents of data buffers on a Local Peer that have been
access. The types of tampering attacks that are possible are enabled for remote write access. The types of tampering attacks
outlined in the sections that follow. that are possible are outlined in the sections that follow.
7.3.1 RDMA Write or Read Response to Memory Outside the Buffer 7.3.1 Buffer Overrun - RDMA Write or Read Response
This attack is an attempt by the Remote Peer to perform an RDMA This attack is an attempt by the Remote Peer to perform an RDMA
Write or RDMA Read Response to memory outside of the valid length Write or RDMA Read Response to memory outside of the valid length
range of the data buffer enabled for remote write access. This range of the data buffer enabled for remote write access. This
attack applies primarily to Trust Models with Untrusted Remote Peers attack can occur even when no resources are shared across
(NS-NT, S-NT and S-LT), and can occur even when no resources are Streams. This issue can also arise if the application has a bug.
shared across connections. This issue can also arise for Trust
Models NS-RT and S-T, which assume remote Partial Trust, if the
application has a bug. Thus it is RECOMMENDED that all Trust Models
ensure this countermeasures are in place against this form of
attack.
The countermeasure for this type of attack must be in the RNIC The countermeasure for this type of attack must be in the RNIC
implementation, using the STag. When the Local Peer specifies to the implementation, using the STag. When the Local Peer specifies to
RI, the base and the number of bytes in the buffer that it wishes to the RI the base address and the number of bytes in the buffer
make accessible, the RI must ensure that the base and bounds check that it wishes to make accessible, the RI must ensure that the
are applied to any access to the buffer referenced by the STag base and bounds check are applied to any access to the buffer
before the STag is enabled for access. When an RDMA data transfer referenced by the STag before the STag is enabled for access.
operation (which includes an STag) arrives on a connection, a base When an RDMA data transfer operation (which includes an STag)
and bounds byte granularity access check must be performed to ensure arrives on a Stream, a base and bounds byte granularity access
the operation accesses only memory locations within the buffer check must be performed to ensure the operation accesses only
described by that STag. memory locations within the buffer described by that STag.
Thus, it is RECOMMENDED that an RI implementation ensure that a Thus, it is RECOMMENDED that an RI implementation ensure that a
Remote Peer, regardless of Trust Model, will not be able to access Remote Peer will not be able to access memory outside of the
memory outside of the buffer specified when the STag was enabled for buffer specified when the STag was enabled for remote access.
remote access.
7.3.2 Modifying a Buffer After Indicating Contents Are Ready 7.3.2 Modifying a Buffer After Indication
This attack occurs if a Remote Peer attempts to modify the contents This attack occurs if a Remote Peer attempts to modify the
by performing an RDMA Write or an RDMA Read Response after it had contents by performing an RDMA Write or an RDMA Read Response
indicated to the Local Peer that the data buffer contents were ready after it had indicated to the Local Peer that the data buffer
for use. contents were ready for use.
This attack applies primarily to the Trust Models where the Remote This attack can occur even when no resources are shared across
Peers are not Trusted (Trust Models NS-NT, S-NT and S-LT), and can Streams. Note that a bug in a Remote Peer, or network based
occur even when no resources are shared across connections. Note tampering, could also result in this problem.
that, an error on the part of a Trusted Remote Peer could also
result in this problem.
The Local Peer can protect itself from this type of attack by The Local Peer can protect itself from this type of attack by
revoking remote access when the original data transfer has completed revoking remote access when the original data transfer has
and before it validates the contents of the buffer. The Local Peer completed and before it validates the contents of the buffer. The
can either do this by explicitly revoking remote access rights for Local Peer can either do this by explicitly revoking remote
the STag when the Remote Peer indicates the operation has completed, access rights for the STag when the Remote Peer indicates the
or by checking to make sure the Remote Peer Invalidated the STag operation has completed, or by checking to make sure the Remote
through the RDMAP Invalidate capability, and if it did not, the Peer Invalidated the STag through the RDMAP Invalidate
Local Peer then explicitly revokes the STag remote access rights. capability, and if it did not, the Local Peer then explicitly
revokes the STag remote access rights.
It is RECOMMENDED that the Local Peer follow the above procedure for It is RECOMMENDED that the Local Peer follow the above procedure
Trust Models NS-NT, S-NT, and S-LT to protect the buffer. The Local to protect the buffer before it validates the contents of the
Peer MAY also wish to use this procedure for Trust Models NS-RT and buffer (or uses the buffer in any way).
S-T to protect itself from unintended tampering due to an error in
the Remote Peer.
7.3.3 Using Multiple Stags to access the same buffer 7.3.3 Multiple STags to access the same buffer
See section 7.4.6 on page 25 for this analysis. See section 7.4.6 on page 27 for this analysis.
7.3.4 Network based modification of buffer content
This is actually a man in the middle attack ű but only on the
content of the buffer, as opposed to the man in the middle attack
presented above, where both the signaling and content can be
modified. See Section 7.2.3 Man in the Middle Attack on page 22.
7.4 Information Disclosure 7.4 Information Disclosure
The main potential source for information disclosure is through a The main potential source for information disclosure is through a
local buffer that has been enabled for remote access. If the buffer local buffer that has been enabled for remote access. If the
can be probed by a Remote Peer on another connection, then there is buffer can be probed by a Remote Peer on another Stream, then
potential for information disclosure. there is potential for information disclosure.
Information disclosure attacks mainly apply to the Trust Models that
include Untrusted Remote Peers (Trust Models NS-NT, S-NT, and S-LT
as defined in Section 5). Trusted Remote Peers are assumed not to
purposely attempt such attacks - any attempt is assumed to be due to
an error or other unexpected failure in the Remote Peer.
The potential attacks that could result in unintended information The potential attacks that could result in unintended information
disclosure and countermeasures are as follows: disclosure and countermeasures are detailed in the following
sections.
7.4.1 Probing memory outside of the buffer bounds 7.4.1 Probing memory outside of the buffer bounds
This is essentially the same attack as described in Section 7.3.1, This is essentially the same attack as described in Section
except an RDMA Read Request is used to mount the attack. The same 7.3.1, except an RDMA Read Request is used to mount the attack.
countermeasure applies. The same countermeasure applies.
7.4.2 Using RDMA Read to Access Stale Data 7.4.2 Using RDMA Read to Access Stale Data
If a buffer is being used for a combination of reads and writes If a buffer is being used for a combination of reads and writes
(either remote or local), and is exposed to the Remote Peer with at (either remote or local), and is exposed to the Remote Peer with
least remote read access rights, the Remote Peer may be able to at least remote read access rights, the Remote Peer may be able
examine the contents of the buffer before they are initialized with to examine the contents of the buffer before they are initialized
the correct data. In this situation, whatever contents were present with the correct data. In this situation, whatever contents were
in the buffer before the buffer is initialized can be viewed by the present in the buffer before the buffer is initialized can be
Remote Peer, if the Remote Peer performs an RDMA Read. This threat viewed by the Remote Peer, if the Remote Peer performs an RDMA
applies to Trust Models NS-NT, S-NT, and S-LT. Read.
Because of this, it is RECOMMENDED that the Local Peer ensure that Because of this, it is RECOMMENDED that the Local Peer ensure
no stale data is contained in the buffer when remote read access that no stale data is contained in the buffer before remote read
rights are initially granted (this can be done by zeroing the access rights are granted (this can be done by zeroing the
contents of the memory, for example). contents of the memory, for example).
7.4.3 Accessing a buffer after the transfer is over 7.4.3 Accessing a Buffer After the Transfer
If the Remote Peer has remote read access to a buffer, and by some If the Remote Peer has remote read access to a buffer, and by
mechanism tells the Local Peer that the transfer has been completed, some mechanism tells the Local Peer that the transfer has been
but the Local Peer does not disable remote access to the buffer completed, but the Local Peer does not disable remote access to
before modifying the data, it is possible for the Remote Peer to the buffer before modifying the data, it is possible for the
retrieve the new data. Remote Peer to retrieve the new data.
This is similar to the attack defined in Section 7.3.2 Modifying a This is similar to the attack defined in Section 7.3.2 Modifying
Buffer After Indicating Contents Are Ready on page 23. The same a Buffer After Indicati on page 25. The same countermeasures
countermeasures apply. In addition, it is RECOMMENDED that the Local apply. In addition, it is RECOMMENDED that the Local Peer should
Peer should grant remote read access rights only for the amount of grant remote read access rights only for the amount of time
time needed to retrieve the data. needed to retrieve the data.
7.4.4 Accessing data within a valid STag that was unintended 7.4.4 Accessing Unintended Data With a Valid STag
If the Local Peer enables remote access to a buffer using an STag If the Local Peer enables remote access to a buffer using an STag
that references the entire buffer, but intends only a portion of the that references the entire buffer, but intends only a portion of
buffer to be accessed, it is possible for the Remote Peer to access the buffer to be accessed, it is possible for the Remote Peer to
the other parts of the buffer anyway. This threat applies to Trust access the other parts of the buffer anyway.
Models NS-NT, S-NT, and S-LT.
To prevent this attack, it is RECOMMENDED that the Local Peer set To prevent this attack, it is RECOMMENDED that the Local Peer set
the base and bounds of the buffer when the STag is initialized to the base and bounds of the buffer when the STag is initialized to
expose only the data to be retrieved. expose only the data to be retrieved.
7.4.5 Using RDMA Read on a buffer meant only for RDMA Write 7.4.5 RDMA Read into an RDMA Write Buffer
One form of disclosure can occur if the access rights on the buffer One form of disclosure can occur if the access rights on the
were set for remote read, when only remote write access was buffer enabled remote read, when only remote write access was
intended. This attack applies primarily to Trust Models with intended. If the buffer contained application data, or data from
Untrusted Remote Peers (NS-NT, S-NT and S-LT). If the buffer a transfer on an unrelated Stream, the Remote Peer could retrieve
contained application data, or data from a transfer on an unrelated the data through an RDMA Read operation.
connection, the Remote Peer could retrieve the data through an RDMA
Read operation.
The most obvious countermeasure for this attack is to not grant The most obvious countermeasure for this attack is to not grant
remote read access if the buffer is intended to be write-only. The remote read access if the buffer is intended to be write-only.
Remote Peer would not be able to retrieve data associated with the Then the Remote Peer would not be able to retrieve data
buffer. An attempt to do so would result in an error and the RDMAP associated with the buffer. An attempt to do so would result in
Stream associated with the connection would be terminated. an error and the RDMAP Stream associated with the Stream would be
terminated.
Thus, it is RECOMMENDED that if an application only intends a buffer Thus, it is RECOMMENDED that if an application only intends a
to be exposed for remote write access, it set the access rights to buffer to be exposed for remote write access, it set the access
the buffer to only enable remote write access. rights to the buffer to only enable remote write access.
7.4.6 Using Multiple Stags to access the same buffer 7.4.6 Using Multiple STags to Access One Buffer
Multiple STags accessing the same buffer at the same time can result Multiple STags accessing the same buffer at the same time can
in unintentional information disclosure if the STags are used by result in unintentional information disclosure if the STags are
different Remote Peers. Because an RDMA implementation could allow used by different, mutually untrusted, Remote Peers. This model
an STag to have read, write, or read and write access associated applies specifically to client/server communication, where the
with an Stag, it is possible to have unintended information server is communicating with multiple clients, each of which do
disclosure if the Remote Peers do not share the same Trust Model. not mutually trust each other.
If only read access is enabled, then the Local Peer has complete If only read access is enabled, then the Local Peer has complete
control over the information disclosure and multiple Stags to the control over information disclosure. Thus a server which
same buffer creates no new security issues. intended to expose the same data (i.e. buffer) to multiple
clients by using multiple STags to the same buffer creates no new
security issues beyond what has already been described in this
document. Note that if the server did not intend to expose the
same data to the clients, it should use separate buffers for each
client (and separate STags).
When one STag has remote read access enabled and a different STag When one STag has remote read access enabled and a different STag
has remote write access enabled to the same buffer, it is possible has remote write access enabled to the same buffer, it is
for one connection to view the contents that have been written by possible for one Remote Peer to view the contents that have been
another Remote Peer. written by another Remote Peer.
If both Stags have remote write access enabled and the two Remote If both STags have remote write access enabled and the two Remote
Peers do not mutually trust each other, it is possible for one Peers do not mutually trust each other, it is possible for one
Remote Peer to overwrite the contents that have been written by the Remote Peer to overwrite the contents that have been written by
other Remote Peer. the other Remote Peer.
For Trust Models NS-NT, S-NT, S-LT it is RECOMMENDED that multiple Thus it is RECOMMENDED that multiple Remote Peers which do not
Remote Peers not be granted access to the same buffer through share Partial Mutual Trust not be granted write access to the
different STags at the same time. A buffer should be exposed to only same buffer through different STags. A buffer should be exposed
one Untrusted Remote Peer at a time to ensure that no information to only one untrusted Remote Peer at a time to ensure that no
disclosure or information tampering occurs between peers. information disclosure or information tampering occurs between
peers.
7.4.7 Remote node loading firmware onto the RNIC 7.4.7 Remote Node Loading Firmware onto the RNIC
If the Remote Peer can cause firmware to be loaded onto the RNIC, If the Remote Peer can cause firmware to be loaded onto the RNIC,
there is an opportunity for information disclosure. See Elevation of there is an opportunity for information disclosure. See Elevation
Privilege in Section 7.6 for this analysis. of Privilege in Section 7.6 for this analysis.
7.4.8 Controlling Access to Page Translation Table and STag Mapping 7.4.8 Controlling Access to PTT & STag Mapping
Issue: Discuss issues in allowing the non-privileged consumer to If a Non-Privileged application is able to directly manipulate
control mapping the Page Translation Table and Stag. the RNIC Page Translation Tables (which translate from an STag to
a host address), it is possible that the Non-Privileged
application could point the Page Translation Table at an
unrelated applicationĂs buffers and thereby be able to gain
access to information in the unrelated application.
It is RECOMMENDED that the Privileged Resource Manager verify that As discussed in Section 4 Architectural Model on page 8,
the Non-Privileged application has the right to access a specific introduction of a Privileged Resource Manager to arbitrate the
Data Buffer before allowing an STag for which the application has mapping requests is an effective countermeasure. This enables the
access rights to be associated with a specific Data Buffer. This Privileged Resource Manager to ensure an application can only
can be done when the Page Translation Table is initialized to access initialize the Page Translation Table (PTT)to point to its own
the Data Buffer or when the STag is initialized to point to a group buffers.
of Page Translation Table entries, or both.
Thus it is RECOMMENDED that the Privileged Resource Manager
verify that the Non-Privileged application has the right to
access a specific Data Buffer before allowing an STag for which
the application has access rights to be associated with a
specific Data Buffer. This can be done when the Page Translation
Table is initialized to access the Data Buffer or when the STag
is initialized to point to a group of Page Translation Table
entries, or both.
7.4.9 Network based eaves dropping
An attacker, eaves dropping the network, can read the content of
all read and write access to the peerĂs buffers. To prevent
information disclosure, the read/written data must be encrypted.
The encryption can be done either by the ULP, or by a protocol
that provides security services to the LLP (e.g. IPsec or SSL).
Refer to section 8 for discussion of security services for
RDDP/RDMA.
7.5 Denial of Service (DOS) 7.5 Denial of Service (DOS)
A DOS attack is one of the primary security risks of RDMAP. This is A DOS attack is one of the primary security risks of RDMAP. This
because RNIC resources are valuable and scarce, and many application is because RNIC resources are valuable and scarce, and many
environments require communication with Untrusted Remote Peers. If application environments require communication with untrusted
the remote application can be authenticated or encrypted, clearly, Remote Peers. If the remote application can be authenticated or
the DOS profile can be reduced. For the purposes of this analysis, encrypted, clearly, the DOS profile can be reduced. For the
it is assumed that the RNIC must be able to operate in Untrusted purposes of this analysis, it is assumed that the RNIC must be
environments, which are open to DOS style attacks. able to operate in untrusted environments, which are open to DOS
style attacks.
Denial of service attacks against RI resources are not the typical Denial of service attacks against RI resources are not the
unknown party spraying packets at a random host (such as a TCP SYN typical unknown party spraying packets at a random host (such as
attack). Because the connection must be fully established, the a TCP SYN attack). Because the connection/Stream must be fully
attacker must be able to both send and receive messages over that established, the attacker must be able to both send and receive
connection, or be able to guess a valid packet on an existing RDMAP messages over that connection/Stream, or be able to guess a valid
Stream. packet on an existing RDMAP Stream.
This section outlines the potential attacks and the countermeasures This section outlines the potential attacks and the
available for dealing with each attack. For each attack, the Trust countermeasures available for dealing with each attack.
Model that it applies to is described.
7.5.1 RNIC Resource Consumption 7.5.1 RNIC Resource Consumption
This section covers attacks that fall into the general category of a This section covers attacks that fall into the general category
Local Peer attempting to unfairly allocate scarce RNIC resources. of a Local Peer attempting to unfairly allocate scarce RNIC
The Local Peer may be attempting to allocate resources on its own resources. The Local Peer may be attempting to allocate resources
behalf, or on behalf of a Remote Peer. Resources that fall into this on its own behalf, or on behalf of a Remote Peer. Resources that
category include: Protection Domains, Connection Context Memory, fall into this category include: Protection Domains, Stream
Translation and Protection Tables, and STag namespace. These can be Context Memory, Translation and Protection Tables, and STag
attacks by currently active Local Peers or ones that allocated namespace. These can be attacks by currently active Local Peers
resources earlier, but are now idle. or ones that allocated resources earlier, but are now idle.
These attacks generally apply to any Trust Model that includes This type of attack can occur regardless of whether resources are
Untrusted Local Peers (Trust Models NS-NT, NS-RT and S-NT). This shared across Streams.
type of attack can occur even when resources are not shared across
connections.
It is RECOMMENDED that the allocation of all scarce resources be It is RECOMMENDED that the allocation of all scarce resources be
placed under the control of a Resource Manager. This allows the placed under the control of a Privileged Resource Manager. This
Resource Manager to: allows the Privileged Resource Manager to:
* prevent a Local Peer from allocating more than its fair * prevent a Local Peer from allocating more than its fair
share of resources, and share of resources.
* detect if a Remote Peer is attempting to launch a DOS attack
by attempting to create an excessive number of connections
and take corrective action (such as refusing the request or
applying network layer filters against the Remote Peer).
Note that, placing scarce resource management under the control of a * detect if a Remote Peer is attempting to launch a DOS
Resource Manager also prevents other Trusted Local Peers from attack by attempting to create an excessive number of
attempting to allocate more than their fair share of resources. Streams and take corrective action (such as refusing the
request or applying network layer filters against the
Remote Peer).
This analysis assumes that the Resource Manager is responsible for This analysis assumes that the Resource Manager is responsible
handing out Protection Domains, and RNIC implementations will for handing out Protection Domains, and RNIC implementations will
provide enough Protection Domains to allow the Resource Manager to provide enough Protection Domains to allow the Resource Manager
be able to assign a unique Protection Domain for each unrelated, to be able to assign a unique Protection Domain for each
Untrusted Local Peer (for a bounded, reasonable number of Local unrelated, untrusted Local Peer (for a bounded, reasonable number
Peers). This analysis further assumes that the Resource Manager of Local Peers). This analysis further assumes that the Resource
implements policies to ensure that Untrusted Local Peers are not Manager implements policies to ensure that untrusted Local Peers
able to consume all of the Protection Domains through a DOS attack. are not able to consume all of the Protection Domains through a
Note that Protection Domain consumption cannot result from a DOS DOS attack. Note that Protection Domain consumption cannot result
attack launched by a Remote Peer, unless a Local Peer is acting on from a DOS attack launched by a Remote Peer, unless a Local Peer
the Remote PeerĂs behalf. is acting on the Remote PeerĂs behalf.
7.5.2 Resource Consumption By Active Applications 7.5.2 Resource Consumption By Active Applications
This section describes DOS attacks from Local and Remote Peers that This section describes DOS attacks from Local and Remote Peers
are actively exchanging messages. Attacks on each RDMA NIC resource that are actively exchanging messages. Attacks on each RDMA NIC
are examined, including the Trust Models that apply, and the resource are examined and specific countermeasures are
specific countermeasures. Note that, attacks on Connection Context identified. Note that attacks on Stream Context Memory, Page
Memory, Page Translation Tables, and STag namespace are covered in Translation Tables, and STag namespace are covered in Section
Section 7.5.1 RNIC Resource Consumption, so are not included here. 7.5.1 RNIC Resource Consumption, so are not included here.
7.5.2.1 Receive Data Buffers 7.5.2.1 Multiple Streams Sharing Receive Buffers
The Remote Peer can attempt to consume more than its fair share of The Remote Peer can attempt to consume more than its fair share
receive data buffers (Untagged DDP buffers or for RDMAP buffers of receive data buffers (Untagged DDP buffers or for RDMAP
consumed with Send Type Messages). If resources are not shared buffers consumed with Send Type Messages) if receive buffers are
across multiple connections (Trust Models NS-NT, NS-RT), then this shared across multiple Streams.
attack is not possible because the Remote Peer will not be able to
consume more buffers than were allocated to the Stream. The worst
case scenario is that the Remote Peer can consume more receive
buffers than the Local Peer allowed, resulting in no buffers to be
available, which would cause the Remote PeerĂs connection to the
Local Peer to be torn down.
If local receive data buffers are shared among multiple Streams and If resources are not shared across multiple Streams, then this
the Remote Peer is not Trusted (Trust Models S-NT, S-LT), then the attack is not possible because the Remote Peer will not be able
Remote Peer can attempt to consume more than its fair share of the to consume more buffers than were allocated to the Stream. The
receive buffers, causing a different Stream to be short of receive worst case scenario is that the Remote Peer can consume more
buffers, thus possibly causing the other Stream to be torn down. receive buffers than the Local Peer allowed, resulting in no
buffers to be available, which could cause the Remote PeerĂs
Stream to the Local Peer to be torn down.
If local receive data buffers are shared among multiple Streams,
then the Remote Peer can attempt to consume more than its fair
share of the receive buffers, causing a different Stream to be
short of receive buffers, thus possibly causing the other Stream
to be torn down. For example, if the Remote Peer sent enough one
byte Untagged Messages, they might be able to consume all local
shared receive queue resources with little effort on their part.
One method the Local Peer could use is to recognize that a Remote One method the Local Peer could use is to recognize that a Remote
Peer is attempting to use more than its fair share of resources and Peer is attempting to use more than its fair share of resources
terminate the Stream. However, if the Local Peer is sufficiently and terminate the Stream. However, if the Local Peer is
slow, it may be possible for the Remote Peer to still mount a denial sufficiently slow, it may be possible for the Remote Peer to
of service attack. An RNIC Engine implementation that enables a more still mount a denial of service attack. One countermeasure that
robust countermeasure is one that provides high and low-water can protect against this attack is implementing a low-water
notifications to enable the Local Peer to detect and prevent DOS notification. The low-water notification alerts the application
attacks against shared data buffers. If a low-water notification is if the number of buffers in the receive queue is less than a
enabled, and the Local Peer is able to bound the amount of time that threshold.
it takes to replenish receive buffers, and the Local Peer maintains
statistics to determine which Remote Peer is consuming buffers, then If all of the following conditions are true, then the Local Peer
the Local Peer can size the amount of local receive buffers posted can size the amount of local receive buffers posted on the
on the receive queue such that the low-water notification can arrive receive queue to ensure a DOS attack can be stopped.
before resources are depleted and corrective action can be taken
(e.g., terminate the Stream of the attacking Remote Peer). Enabling * a low-water notification is enabled, and
the high-water notification can help the Local Peer detect a Remote
Peer that is launching an attack by sending a large number of out- * the Local Peer is able to bound the amount of time that
of-order packets. The notification is generated when more than the it takes to replenish receive buffers, and
specified number of buffers are in process simultaneously on a * the Local Peer maintains statistics to determine which
Stream (i.e., packets have started to arrive for the buffer, but Remote Peer is consuming buffers.
have not yet been delivered to the ULP).
The above conditions enable the low-water notification to arrive
before resources are depleted and thus the Local Peer can take
corrective action (e.g., terminate the Stream of the attacking
Remote Peer).
A different, but similar attack is if the Remote Peer sends a
significant number of out-of-order packets and the RNIC has the
ability to use the application buffer as a reassembly buffer. In
this case the Remote Peer can consume a significant number of
application buffers, but never send enough data to enable the
application buffer to be completed to the application.
An effective countermeasure is to create a high-water
notification which alerts the application if there is more than a
specified number of receive buffers "in process" (partially
consumed, but not completed). The notification is generated when
more than the specified number of buffers are in process
simultaneously on a specific Stream (i.e., packets have started
to arrive for the buffer, but the buffer has not yet been
delivered to the ULP).
A different countermeasure is for the RNIC Engine to provide the A different countermeasure is for the RNIC Engine to provide the
capability to limit the Remote PeerĂs ability to consume receive capability to limit the Remote PeerĂs ability to consume receive
buffers on a per Stream basis. Unfortunately this requires a large buffers on a per Stream basis. Unfortunately this requires a
amount of state to be tracked on a per RNIC basis. large amount of state to be tracked in each RNIC on a per Stream
basis.
Thus, if an RNIC Engine provides the ability to share receive Thus, if an RNIC Engine provides the ability to share receive
buffers across multiple Streams, it is RECOMMENDED that it enable buffers across multiple Streams, it is RECOMMENDED that it enable
the Local Peer to detect if the Remote Peer is attempting to consume the Local Peer to detect if the Remote Peer is attempting to
more than its fair share of resources so that the application can consume more than its fair share of resources so that the
apply countermeasures to detect and prevent the attack. application can apply countermeasures to detect and prevent the
attack.
7.5.2.2 Completion Queue (CQ) Resource Consumption
DOS attacks against the Completion Queue can be caused by either the 7.5.2.2 Local Peer Attacking a Shared CQ
Local Peer or the Remote Peer if either attempts to cause more
completions than its fair share, thus potentially starving another
unrelated Stream such that no Completion Queue entries are
available.
A Completion Queue entry can potentially be consumed by a completion DOS attacks against a Shared Completion Queue (CQ) can be caused
from the send queue or a receive completion. In the former, the by either the Local Peer or the Remote Peer if either attempts to
attacker is the Local Peer (Trust Models S-NT). In the later, the cause more completions than its fair share of the number of
attacker is the Remote Peer (S-NT, S-LT). entries, thus potentially starving another unrelated Stream such
that no Completion Queue entries are available.
The potential attacks and the countermeasures for each are described A Completion Queue entry can potentially be consumed by a
in the subsections that follow. completion from the send queue or a receive completion. In the
former, the attacker is the Local Peer. In the later, the
attacker is the Remote Peer.
7.5.2.2.1 Local Peer Attacking a Shared CQ A form of attack can occur where the Local Peers can consume
resources on the CQ. A Local Peer that is slow to free resources
on the CQ by not reaping the completion status quickly enough
could stall all other Local Peers attempting to use that CQ.
A form of attack can occur for Trust Models NS-NT, NS-RT, and S-NT, One of two countermeasures can be used to avoid this kind of
where the Local Peers are Untrusted, and Local Peers can consume attack. The first is to only share a CQ between Streams that
resources on the CQ. Sharing a CQ across connections that belong to share Partial Mutual Trust. The other is to use a trusted Local
different Protection Domains is NOT RECOMMENDED in cases where any Peer to act as a third party to free resources on the CQ and
of the Local Peers are Untrusted. A Local Peer that is slow to free place the status in intermediate storage until the untrusted
resources on the CQ by not reaping the completion status quick Local Peer reaps the status information. For these reason,
enough could stall all other Local Peers attempting to use that CQ. sharing a CQ across Streams that belong to different Protection
Domains is NOT RECOMMENDED.
One of two countermeasures can be used to avoid this kind of attack. 7.5.2.3 Remote Peer Attacking a Shared CQ
The first is to use a different CQ per Untrusted Local Peer. The
other is to use a Trusted Local Peer to act as a third party to free
resources on the CQ and place the status in intermediate storage
until the Untrusted Local Peer reaps the status information.
7.5.2.2.2 Remote Peer Attacking a Shared CQ For an overview of the Shared CQ attack model, see Section
7.5.2.2.
The Remote Peer can attack a CQ by consuming more than its fair The Remote Peer can attack a CQ by consuming more than its fair
share of CQ entries by using one of two methods. The first method share of CQ entries by using one of the following methods:
can only be used if the ULP protocol allows the Remote Peer to
reserve a specified number of CQ entries, possibly leaving
insufficient entries for other connections that are sharing the CQ.
The other method is if the Remote Peer can attack the CQ by
overwhelming the CQ with completions, which can affect completion
processing on other Streams sharing that connection.
The first method of attack can be avoided if the ULP does not allow * The ULP protocol allows the Remote Peer to reserve a
a Remote Peer to reserve CQ entries. This is RECOMMENDED specified number of CQ entries, possibly leaving
particularly for Trust Models S-NT and S-LT, with shared resources insufficient entries for other Streams that are sharing
and Untrusted Remote Peers. If a Local Peer allows this type of the CQ.
resource allocation, and it has any Untrusted Remote Peers, then the
Local Peer it is RECOMMENDED that the CQ be isolated to connections
within a single Protection Domain.
One way that a Remote Peer can attempt to overwhelm its CQ with * If the Remote Peer or Local Peer (or both) can attack the
completions is by sending minimum length RDMAP/DDP Messages to cause CQ by overwhelming the CQ with completions, then
as many completions per second as possible. Assuming that the Local completion processing on other Streams sharing that
Peer does not run out of receive buffers (if they do, then this is a Completion Queue can be affected (e.g. the Completion
different attack, documented in Section 7.5.2.1 Receive Data Buffers Queue overflows and stops functioning).
on page 28), then it might be possible for the Remote Peer to
consume more than its fair share of Completion Queue entries.
Depending upon the CQ implementation, this could either cause the CQ
to overflow (if it is not large enough to handle all of the
completions generated) or for another Stream to not be able to
generate CQ entries (if the RNIC had flow control on generation of
CQ entries into the CQ). In either case, the CQ will stop
functioning correctly and any connections expecting completions on
the CQ will stop functioning.
This attack can occur regardless of whether all of the connections The first method of attack can be avoided if the ULP does not
associated with the CQ are in the same Protection Domain or are in allow a Remote Peer to reserve CQ entries or there is a trusted
different Protection Domains. Because this attack assumes a shared intermediary such as a Privileged Resource Manager. Unfortunately
local resource and an Untrusted Remote Peer, Trust Models S-NT, S-LT it is often unrealistic to not allow a Remote Peer to reserve CQ
apply. entries ű particularly if the number of completion entries is
dependent on other ULP negotiated parameters, such as the amount
of buffering required by the ULP. Thus it is RECOMMENDED that an
implementation require a Privileged Resource Manager to control
the allocation of CQ entries.
One way that a Local or Remote Peer can attempt to overwhelm a CQ
with completions is by sending minimum length RDMAP/DDP Messages
to cause as many completions (receive completions for the Remote
Peer, send completions for the Local Peer) per second as
possible. If it is the Remote Peer attacking, and we assume that
the Local Peer does not run out of receive buffers (if they do,
then this is a different attack, documented in Section 7.5.2.1
Multiple Streams Sharing Receive Buffers on page 30), then it
might be possible for the Remote Peer to consume more than its
fair share of Completion Queue entries. Depending upon the CQ
implementation, this could either cause the CQ to overflow (if it
is not large enough to handle all of the completions generated)
or for another Stream to not be able to generate CQ entries (if
the RNIC had flow control on generation of CQ entries into the
CQ). In either case, the CQ will stop functioning correctly and
any Streams expecting completions on the CQ will stop
functioning.
This attack can occur regardless of whether all of the Streams
associated with the CQ are in the same Protection Domain or are
in different Protection Domains ű the key issue is that the
number of Completion Queue entries is less than the number of all
outstanding operations that can cause a completion.
The Local Peer can protect itself from this type of attack using The Local Peer can protect itself from this type of attack using
either of the following methods: either of the following methods:
* Resize the CQ to the appropriate level(note that resizing * Resize the CQ to the appropriate level(note that resizing
the CQ can fail, so the CQ resize should be done before the CQ can fail, so the CQ resize should be done before
sizing the buffers on the connection), OR sizing the Send Queue and Receive Queue on the Stream),
OR
* Grant fewer resources than the Remote Peer requested (not * Grant fewer resources than the Remote Peer requested (not
supplying the number of Receive Data Buffers requested). supplying the number of Receive Data Buffers requested).
The proper sizing of the CQ is dependent on the Trust Model. For the The proper sizing of the CQ is dependent on whether the Local
Trust Model described in Section 5, with Trusted Local Peers and Peer will post as many resources to the various queues as the
Untrusted Remote Peers (Trust Model S-LT), a correctly sized CQ size of the queue enables or not. If the Local Peer can be
means that the CQ is large enough to hold completion status for all trusted to post a number of resources that is smaller than the
of the outstanding Receive Data Buffers, or: size of the specific resourceĂs queue, then a correctly sized CQ
means that the CQ is large enough to hold completion status for
all of the outstanding Data Buffers (both send and receive
buffers), or:
CQ_MIN_SIZE = SUM(MaxPostedOnEachRQ) CQ_MIN_SIZE = SUM(MaxPostedOnEachRQ)
+ SUM(MaxPostedOnEachS-RQ) + SUM(MaxPostedOnEachS-RQ)
+ SUM(MaxPostedOnEachSQ) + SUM(MaxPostedOnEachSQ)
If the Trust Model assumes neither the Local Peer nor the Remote If the local peer must be able to completely fill the queues, or
Peer is trusted (Trust Model S-NT or S-LT), then the CQ must be can not be trusted to observe a limit smaller than the queues,
sized to accommodate the maximum number of operations or Receive then the CQ must be sized to accommodate the maximum number of
Data Buffers that it is possible to post at any one time, thus the operations that it is possible to post at any one time. Thus the
equation becomes: equation becomes:
CQ_MIN_SIZE = SUM(SizeOfEachRQ) CQ_MIN_SIZE = SUM(SizeOfEachRQ)
+ SUM(SizeOfEachS-RQ) + SUM(SizeOfEachS-RQ)
+ SUM(SizeOfEachSQ) + SUM(SizeOfEachSQ)
Where MaxPosted*OnEach*Q and SizeOfEach*Q varies on a per connection Where MaxPosted*OnEach*Q and SizeOfEach*Q varies on a per Stream
or per Shared Receive Queue basis. or per Shared Receive Queue basis.
7.5.2.3 RDMA Read Request Queue It is RECOMMENDED that the Local Peer implement a mechanism to
ensure that the Completion Queue can not overflow. Note that it
is possible to share CQs even if the Remote Peers accessing the
CQs are untrusted if either of the above two formulas are
implemented. If the Local Peer can be trusted to not post more
than MaxPostedOnEachRQ, MaxPostedOnEachS-RQ, and
MaxPostedOnEachSQ, then the first formula applies. If the Local
Peer can not be trusted to obey the limit, then the second
formula applies.
Two types of attacks are possible against resources associated with 7.5.2.4 RDMA Read Request Queue
RDMA Read Request Queues. One style of attack can only occur when
the RDMA Read Request Queue resources are pooled across multiple
connections. This attack occurs when an Untrusted Local Peer
attempts to unfairly allocate RDMA Read Request Queue resources for
its connections.
It is RECOMMENDED that access to interfaces that allocate RDMA Read If RDMA Read Request Queue resources are pooled across multiple
Request Queue entries be restricted to a Trusted Local Peer, such as Streams, one attack is if the Local Peer attempts to unfairly
a Resource Manager. The Resource Manager should prevent a Local Peer allocate RDMA Read Request Queue resources for its Streams. For
from allocating more than its fair share of resources. example, the Local Peer attempts to allocate all available
resources on a specific RDMA Read Request Queue for its Streams,
thereby denying the resource to applications sharing the RDMA
Read Request Queue. The same type of argument applies even if the
RDMA Read Request is not shared ű but a Local Peer attempts to
allocate all of the RNICs resource when the queue is created.
Another form of attack is the Remote Peer sending more RDMA Read Thus it is RECOMMENDED that access to interfaces that allocate
Requests than the depth of the RDMA Read Request Queue at the Local RDMA Read Request Queue entries be restricted to a trusted Local
Peer. Peer, such as a Privileged Resource Manager. The Privileged
Resource Manager should prevent a Local Peer from allocating more
than its fair share of resources.
This attack can be prevented by properly configuring the connection Another form of attack is if the Remote Peer sends more RDMA Read
when the connection is established. The Remote PeerĂs end of the Requests than the depth of the RDMA Read Request Queue at the
connection should be configured by a trusted agent such that the Local Peer. If the RDMA Read Request Queue is a shared resource,
RNIC will not transmit RDMA Read Requests that exceed the depth of this could corrupt the queue. If the queue is not shared, then
the RDMA Read Request Queue at the Local Peer. If the connection is the worst case is that the current Stream is disabled. One
correctly configured, and if the Remote Peer submits more requests approach to solving the shared RDMA Read Request Queue would be
than the Local PeerĂs RDMA Read Request Queue can handle, the to create thresholds, similar to those described in Section
requests will be queued at the Remote PeerĂs connection until 7.5.2.1 Multiple Streams Sharing Receive Buffers on page 30. A
previous requests complete. If the Remote PeerĂs connection is not simpler approach is to not share RDMA Read Request Queue
configured correctly, the RDMAP Stream for that connection is resources amoung Streams or enforce hard limits of consumption
terminated when more RDMA Read Requests arrive at the Local Peer per Stream. Thus it is RECOMMENDED that RDMA Read Request Queue
than the Local Peer can handle. Thus, the Remote Peer is able to resource consumption be controlled such that RDMAP/DDP Streams
only affect its own connection. which do not share Partial Mutual Trust do not share RDMA Read
Request Queue resources.
If the issue is a bug in the Remote PeerĂs implementation, and
not a malicious attack, the issue can be solved by requiring the
Remote PeerĂs RNIC to throttle RDMA Read Requests. By properly
configuring the Stream at the Remote Peer through a trusted
agent, the RNIC can be made to not transmit RDMA Read Requests
that exceed the depth of the RDMA Read Request Queue at the Local
Peer. If the Stream is correctly configured, and if the Remote
Peer submits more requests than the Local PeerĂs RDMA Read
Request Queue can handle, the requests would be queued at the
Remote PeerĂs RNIC until previous requests complete. If the
Remote PeerĂs Stream is not configured correctly, the RDMAP
Stream is terminated when more RDMA Read Requests arrive at the
Local Peer than the Local Peer can handle (assuming the prior
paragraphĂs recommendation is implemented).
7.5.3 Resource Consumption by Idle Applications 7.5.3 Resource Consumption by Idle Applications
The simplest form of a DOS attack given a fixed amount of resources The simplest form of a DOS attack given a fixed amount of
is for the Remote Peer to create a RDMAP Stream to a Local Peer, resources is for the Remote Peer to create a RDMAP Stream to a
request dedicated resources then do no actual work. This allows the Local Peer, and request dedicated resources then do no actual
Remote Peer to be very light weight (i.e. only negotiate resources, work. This allows the Remote Peer to be very light weight (i.e.
but do no data transfer) and consumes a disproportionate amount of only negotiate resources, but do no data transfer) and consumes a
resources in the server. disproportionate amount of resources in the server.
A general countermeasure for this style of attack is to monitor A general countermeasure for this style of attack is to monitor
active RDMAP Streams and if resources are getting low, reap the active RDMAP Streams and if resources are getting low, reap the
resources from RDMAP Streams that are not transferring data and resources from RDMAP Streams that are not transferring data and
possibly terminate the connection. This needs to be under possibly terminate the Stream. This would presumably be under
administrative control, and demonstrates the need for a MIB for administrative control.
RDMAP so this condition can be detected and acted upon.
Refer to Section 7.5.1 for the analysis and countermeasures for this Refer to Section 7.5.1 for the analysis and countermeasures for
style of attack on the following RNIC resources: Connection Context this style of attack on the following RNIC resources: Stream
Memory, Page Translation Tables and STag namespace. Context Memory, Page Translation Tables and STag namespace.
Note that some RNIC resources are not at risk of this type of attack Note that some RNIC resources are not at risk of this type of
from a Remote Peer because an attack requires the Remote Peer to attack from a Remote Peer because an attack requires the Remote
send messages in order to consume the resource. Receive Data Peer to send messages in order to consume the resource. Receive
Buffers, Completion Queue, and RDMA Read Request Queue resources are Data Buffers, Completion Queue, and RDMA Read Request Queue
examples. These resources are, however, at risk form a Local Peer resources are examples. These resources are, however, at risk
that attempts to allocate resources, then goes idle. The general from a Local Peer that attempts to allocate resources, then goes
countermeasure described in this section can be used to free idle. This could also be created if the ULP negotiates the
resources allocated by an idle Local Peer. resource levels with the Remote Peer, which causes the Local Peer
to consume resources, however the Remote Peer never sends data to
consume them. The general countermeasure described in this
section can be used to free resources allocated by an idle Local
Peer.
7.5.4 Exercise of non-optimal code paths 7.5.4 Exercise of non-optimal code paths
Another form of DOS attack is to attempt to exercise data paths that Another form of DOS attack is to attempt to exercise data paths
can consume a disproportionate amount of resources. An example might that can consume a disproportionate amount of resources. An
be if error cases are handled on a ŠŠslow pathĂĂ (consuming either example might be if error cases are handled on a "slow path"
host or RNIC computational resources), and an attacker generates (consuming either host or RNIC computational resources), and an
excessive numbers of errors in an attempt to consume these attacker generates excessive numbers of errors in an attempt to
resources. consume these resources.
It is RECOMMENDED that an implementation provide the ability to It is RECOMMENDED that an implementation provide the ability to
detect the above condition and allow an administrator to act, detect the above condition and allow an administrator to act,
including potentially administratively tearing down the RDMAP Stream including potentially administratively tearing down the RDMAP
associated with the connection exercising data paths consuming a Stream associated with the Stream exercising data paths consuming
disproportionate amount of resources. a disproportionate amount of resources.
7.5.5 Remote Invalidation of an STag Shared Across Multiple 7.5.5 RI an STag Shared on Multiple Streams
Connections
If a Local Peer has enabled an STag for remote access, the Remote If a Local Peer has enabled an STag for remote access, the Remote
Peer could attempt to invalidate the STag by using the RDMAP Send Peer could attempt to remote invalidate (RI) the STag by using
with Invalidate or Send with SE and Invalidate Message. If the STag the RDMAP Send with Invalidate or Send with SE and Invalidate
is only valid on the current connection (NS-NT or NS-RT, S-NT), then Message. If the STag is only valid on the current Stream, then
the only side effect is that the Remote Peer can no longer use the the only side effect is that the Remote Peer can no longer use
STag, thus there are no security issues. the STag; thus there are no security issues.
If the STag is valid across multiple connections, then the Remote If the STag is valid across multiple Streams, then the Remote
Peer can prevent other connections from using that STag by using the Peer can prevent other Streams from using that STag by using the
remote invalidate functionality. remote invalidate functionality.
Thus for Trust Models where the Remote Peer may attempt to Thus if RDDP Streams do not share Partial Mutual Trust (i.e. the
invalidate the STag prematurely, the application SHOULD NOT allow an Remote Peer may attempt to invalidate the STag prematurely), it
STag to be valid across multiple connections. is NOT RECOMMENDED that the application allow an STag to be valid
across multiple Streams.
7.5.6 Remote Peer Consumes too many Untagged Receive Buffers
Issue: Need to analyze the case of sharing a queue of Untagged
receive buffers across multiple connections, and that the Remote
Peer can mount a denial of service attack.
Below are some notes.
Many ways to attack here. If receive queue is not shared, itĂs a 7.5.6 Remote Peer Consumes Untagged Receive Buffers
simple queue overflow attack on a dedicated resource. Make sure when
the queue is empty and a DDP segment arrives nothing bad happens.
For a shared receive queue, one node attacks with single byte <TBD ű remove this section: this section was deleted because it
Untagged Messages to consume large Untagged Buffers (this maximizes was a duplicate of Section 7.5.2.1 Multiple Streams Sharing
packet arrival rate). One node provides DDP segments out of order to Receive Buffers on page 30) Thus comments on this section were
consume out-of-order resources (this is only possible if out-of- added to that section.>
order placement is supported within a merged TCP/SCTP and DDP
implementation).
7.6 Elevation of Privilege 7.6 Elevation of Privilege
The RDMAP/DDP Security Architecture explicitly differentiates The RDMAP/DDP Security Architecture explicitly differentiates
between three levels of privilege - Non-Privileged, Privileged, and between three levels of privilege - Non-Privileged, Privileged,
the Privileged Resource Manager. If a Non-Privileged Application is and the Privileged Resource Manager. If a Non-Privileged
able to elevate its privilege level to a Privileged Application, Application is able to elevate its privilege level to a
then mapping a physical address list to an STag can provide local Privileged Application, then mapping a physical address list to
and remote access to any physical address location on the node. If a an STag can provide local and remote access to any physical
Privileged Mode Application is able to promote itself to be a address location on the node. If a Privileged Mode Application is
Resource Manager, then it is possible for it to perform denial of able to promote itself to be a Resource Manager, then it is
service type attacks where substantial amounts of local resources possible for it to perform denial of service type attacks where
could be consumed. substantial amounts of local resources could be consumed.
There is only one mechanism discovered to date, other than
implementation defects, which would potentially allow an elevation
of privilege.
7.6.1 Loading Firmware into the RNIC In general, elevation of privilege is a local implementation
specific issue and thus outside the scope of this specification.
If the RI implementation, by some insecure mechanism (or There is one issue worth noting, however. If the RI
implementation defect), can enable a Remote Peer or un-trusted Local implementation, by some insecure mechanism (or implementation
Peer to load firmware into the RNIC Engine, it is possible to use defect), can enable a Remote Peer or un-trusted Local Peer to
the RNIC to attack the host. Thus, it is RECOMMENDED that an load firmware into the RNIC Engine, it is possible to use the
implementation not enable firmware to be loaded on the RNIC Engine RNIC to attack the host. Thus, it is RECOMMENDED that an
directly from a Remote Peer, unless the update is done via a secure implementation not enable firmware to be loaded on the RNIC
protocol, such as IPsec (See Section 8 Security Services for RDDP on Engine directly from a Remote Peer, unless the Remote Peer is
page 35). It is RECOMMENDED that an implementation not allow a Non- properly authenticated (by a mechanism outside the scope of this
specification), and the update is done via a secure protocol,
such as IPsec (See Section 8 Security Services for RDDP on page
38). It is RECOMMENDED that an implementation not allow a Non-
Privileged Local Peer to update firmware in the RNIC Engine. Privileged Local Peer to update firmware in the RNIC Engine.
8 Security Services for RDDP 8 Security Services for RDDP
Issue: Security Services section is a placeholder for now. Issue: The spec currently took the IPSec requirements for iSCSI
and made them a SHOULD recommendation. A different approach would
be to simply outline the issues in this section, but leave IPSec
implementation requirements to be specified by ULP/Application
requirements. The argument here is that RDDP is a transport, and
security requirements ű particularly authentication and
confidentiality requirements, are dictated by application
concerns, not transport protocol concerns. Which approach should
be taken?
RDMA and RDDP are used to control, read and write data buffers
over IP networks. Therefore, the control and the data packets of
these protocols are vulnerable to the spoofing, tampering and
information disclosure attacks listed in Section 7.
Generally speaking, session confidentiality protects against
eaves dropping. Session authentication and integrity protection
is a counter measurement against various spoofing and tampering
attacks. The effectiveness of authentication and integrity
against a specific attack, depend on whether the authentication
is machine level authentication (as the one provided by IPsec and
SSL), or ULP authentication.
8.1 Introduction to Security Options 8.1 Introduction to Security Options
The following security services can be applied to an RDDP/RDMA
session:
1. Session confidentiality - protects against eaves dropping
(section 7.4.9).
2. Per-packet data source authentication - protects against the
following spoofing attacks: network based impersonation
(section 7.2.1), Stream hijacking (section 7.2.2), and man in
the middle (section 7.2.3).
3. Per-packet integrity - protects against tampering done by
network based modification of buffer content (section 7.3.4)
4. Packet sequencing - protects against replay attacks, which is
a special case of the above tampering attack.
If RDDP/RDMA session may be subject to impersonation attacks, or
Stream hijacking attacks, it is RECOMMENDED that the session be
authenticated, integrity protected, and protected from replay
attacks; it MAY use confidentiality protection to protect from
eaves dropping (in case the RDDP/RDMA session traverses a public
network).
Both IPsec and SSL are capable of providing the above security
services for IP and TCP traffic respectively. ULP protocols are
able to provide only part of the above security services. The
next sections describe the different security options.
8.1.1 Introduction to IPsec 8.1.1 Introduction to IPsec
IPsec is a protocol suite which is used to secure communication at IPsec is a protocol suite which is used to secure communication
the network layer between two peers. The IPsec protocol suite is at the network layer between two peers. The IPsec protocol suite
specified within the IP Security Architecture [RFC2401], IKE is specified within the IP Security Architecture [RFC2401], IKE
[RFC2409], IPsec Authentication Header (AH) [RFC2402] and IPsec [RFC2409], IPsec Authentication Header (AH) [RFC2402] and IPsec
Encapsulating Security Payload (ESP) [RFC2406] documents. IKE is Encapsulating Security Payload (ESP) [RFC2406] documents. IKE is
the key management protocol while AH and ESP are used to protect IP the key management protocol while AH and ESP are used to protect
traffic. IP traffic.
An IPsec SA is a one-way security association, uniquely identified An IPsec SA is a one-way security association, uniquely
by the 3-tuple: Security Parameter Index (SPI), protocol (ESP) and identified by the 3-tuple: Security Parameter Index (SPI),
destination IP. The parameters for an IPsec security association protocol (ESP) and destination IP. The parameters for an IPsec
are typically established by a key management protocol. These security association are typically established by a key
include the encapsulation mode, encapsulation type, session keys and management protocol. These include the encapsulation mode,
SPI values. encapsulation type, session keys and SPI values.
IKE is a two phase negotiation protocol based on the modular IKE is a two phase negotiation protocol based on the modular
exchange of messages defined by ISAKMP [RFC2408],and the IP Security exchange of messages defined by ISAKMP [RFC2408],and the IP
Domain of Interpretation (DOI) [RFC2407]. IKE has two phases, and Security Domain of Interpretation (DOI) [RFC2407]. IKE has two
accomplishes the following functions: phases, and accomplishes the following functions:
1. Protected cipher suite and options negotiation - using keyed 1. Protected cipher suite and options negotiation - using keyed
MACs and encryption and anti-replay mechanisms MACs and encryption and anti-replay mechanisms.
2. Master key generation - such as via MODP Diffie-Hellman 2. Master key generation - via Diffie-Hellman calculations.
calculations
3. Authentication of end-points 3. Authentication of end-points (usually machine level
authentication).
4. IPsec SA management (selector negotiation, options negotiation, 4. IPsec SA management (selector negotiation, options
create, delete, and rekeying) negotiation, create, delete, and rekeying).
Items 1 through 3 are accomplished in IKE Phase 1, while item 4 is Items 1 through 3 are accomplished in IKE Phase 1, while item 4
handled in IKE Phase 2. is handled in IKE Phase 2.
An IKE Phase 2 negotiation is performed to establish both an inbound IKE phase 1 defines four authentication methods; three of them
and an outbound IPsec SA. The traffic to be protected by an IPsec require both sides to have certified signature or encryption
SA is determined by a selector which has been proposed by the IKE public keys; the forth require the side to exchange out-of-band a
initiator and accepted by the IKE Responder. In IPsec transport secret random string ű called pre-shared-secret (PSS).
mode, the IPsec SA selector can be a "filter" or traffic classifier,
defined as the 5-tuple: <Source IP address, Destination IP address, An IKE Phase 2 negotiation is performed to establish both an
transport protocol (UDP/SCTP/TCP), Source port, Destination port>. inbound and an outbound IPsec SA. The traffic to be protected by
The successful establishment of a IKE Phase-2 SA results in the an IPsec SA is determined by a selector which has been proposed
creation of two uni-directional IPsec SAs fully qualified by the by the IKE initiator and accepted by the IKE Responder. The
tuple <Protocol (ESP/AH), destination address, SPI>. IPsec SA selector can be a "filter" or traffic classifier,
defined as the 5-tuple: <Source IP address, Destination IP
address, transport protocol (e.g. UDP/SCTP/TCP), Source port,
Destination port>. The successful establishment of a IKE Phase-2
SA results in the creation of two uni-directional IPsec SAs fully
qualified by the tuple <Protocol (ESP/AH), destination address,
SPI>.
The session keys for each IPsec SA are derived from a master key, The session keys for each IPsec SA are derived from a master key,
typically via a MODP Diffie-Hellman computation. Rekeying of an typically via a MODP Diffie-Hellman computation. Rekeying of an
existing IPsec SA pair is accomplished by creating two new IPsec existing IPsec SA pair is accomplished by creating two new IPsec
SAs, making them active, and then optionally deleting the older SAs, making them active, and then optionally deleting the older
IPsec SA pair. Typically the new outbound SA is used immediately, IPsec SA pair. Typically the new outbound SA is used
and the old inbound SA is left active to receive packets for some immediately, and the old inbound SA is left active to receive
locally defined time, perhaps 30 seconds or 1 minute. packets for some locally defined time, perhaps 30 seconds or 1
minute. Optionally, rekeying can use Diffie-Helman for keying
material generation.
8.1.2 Introduction to SSL Limitations on RDMAP 8.1.2 Introduction to SSL Limitations on RDMAP
8.1.3 Applications Which Provide Security SSL and TLS [RFC 2246] provide session authentication, integrity
and confidentiality for TCP based applications. SSL supports one-
way (server only) or mutual certificates based authentication.
Issue: Guidance for application protocols like NFS which implement There are at least two limitations that make SSL less appropriate
security. then IPsec for RDDP/RDMA security:
8.1.4 Authentication Only 1. The maximum length supported by the TLS record layer protocol
is 2^14 bytes, longer packets must be fragmented (as a
comparison, the maximal length of an IPsec packet, is
determined by the maximum length of an IP packet).
8.1.5 Privacy 2. SSL is a connection oriented protocol. If a stream cipher or
block cipher in CBC mode is used for bulk encryption, then a
packet can be decrypted only after all the packets preceding
it have already arrived. If SSL is used to protect RDDP/RDMA
traffic, then RDDP/RDMA must gather all out-of-order packets
before placing them into the ULP buffer, which might cause a
significant decrease in its efficiency.
8.1.3 Applications Which Provide Security
Issue: Guidance for application protocols like NFS which
implement security <TBD>.
8.2 Recommendations for IPsec Encapsulation of RDDP 8.2 Recommendations for IPsec Encapsulation of RDDP
Issue: IPsec recommendations for RDMAP/DDP Since iSCSI is expected to be one of the ULPs running on top of
RDDP, the recommendations in this section follow the lines of
[IPSSEC].
This is work that is still to be done. Hopefully this wonĂt be 8.2.1 Transforms
terribly complex. One possible thought on the approach:
a. Use IPsec ESP with authentication to provide authentication, All RDDP/RDMA security compliant implementations SHOULD support
integrity and replay protection.
b. Use IKE for key management. IPsec ESP [RFC2406] to provide security for both control packets
and data packets, as well as the replay protection mechanisms of
IPsec. When ESP is utilized, per-packet data origin
authentication, integrity and replay protection MUST be used.
c. (optionally) use a non-null transform for encryption. This To provide confidentiality with ESP, ESP with 3DES in CBC mode
should be something other than DES. [RFC2451] SHOULD be supported, and AES in Counter mode, as
described in [AESCTR], SHOULD be supported. To provide data
origin authentication and integrity with ESP, HMAC-SHA1 [RFC2404]
SHOULD be supported, and AES in CBC MAC mode with XCBC extensions
[AESXCBC] SHOULD be supported. DES in CBC mode SHOULD NOT be used
due to its inherent weakness. ESP with NULL encryption SHOULD be
supported for authentication.
9 Summary Table of Attacks and Trust Models 8.2.2 IPsec modes
Issue: Finish Summary table of Attacks/Trust Models Conformant IP RDDP/RDMA security implementations SHOULD support
ESP [RFC2406] in tunnel mode and MAY implement IPsec with ESP in
transport mode.
<editor: This section is under construction, and will be completed 8.2.3 IKE
in a future version of this document>
Rows are the attack (grouped into categories) Conformant RDDP/RDMA security implementations SHOULD support IKE
[RFC2409] for peer authentication, negotiation of security
associations, and key management, using the IPsec DOI [RFC2407].
Manual keying MUST NOT be used since it does not provide the
necessary rekeying support.
Columns are the: Conformant RDDP/RDMA security implementations SHOULD support peer
authentication using a pre-shared secret, and MAY support
certificate-based peer authentication using digital signatures.
Peer authentication using the public key encryption methods
outlined in IKE's sections 5.2 and 5.3 [RFC2409] SHOULD NOT be
used.
* Sec - Section the attack is discussed Conformant RDDP/RDMA security implementations SHOULD support IKE
Main Mode and Aggressive Mode. IKE Main Mode with pre-shared key
authentication SHOULD NOT be used when either of the peers uses a
dynamically assigned IP address. While Main Mode with pre-shared
key authentication offers good security in many cases, situations
where dynamically assigned addresses are used force use of a
group pre-shared key, which is vulnerable to man-in-the-middle
attack. Since IKE Aggressive mode with pre-shared secret
authentication is exposed to off-line dictionary attack if it is
used then the selected pre-shared secrets must be random (or
pseudo-random) strings not shorter than 128 bits.
* Attack Name - short name for the attack When digital signatures are used for authentication, either IKE
Main Mode or IKE Aggressive Mode MAY be used. In all cases,
access to locally stored secret information (pre-shared key, or
private key for digital signing) must be suitably restricted,
since compromise of the secret information nullifies the security
properties of the IKE/IPsec protocols.
* Threat - threat type (DOS, etc) When digital signatures are used to achieve authentication, an
IKE negotiator SHOULD use IKE Certificate Request Payload(s) to
specify the certificate authority (or authorities) that are
trusted in accordance with its local policy. IKE negotiators
SHOULD check the pertinent Certificate Revocation List (CRL)
before accepting a PKI certificate for use in IKE's
authentication procedures.
* Columns labeled 1-5 are the Trust Model number (see section The IPsec DOI [RFC2407] provides for several types of
5 Trust Models on page 15). Each entry has a value of +, -, identification data. Within IKE Phase 1, for use within the IDii
and R (research). and IDir payloads, conformant RDDP/RDMA security implementations
SHOULD support the ID_IPV4_ADDR, ID_IPV6_ADDR (if the protocol
stack supports IPv6) and ID_FQDN Identity Payloads. Identities
other than ID_IPV4_ADDR and ID_IPV6_ADDR (such as ID_FQDN) SHOULD
be employed in situations where Aggressive mode is utilized along
with pre-shared keys and IP addresses are dynamically assigned.
The IP Subnet, IP Address Range, ID_DER_ASN1_DN, ID_DER_ASN1_GN,
and ID_USER_FQDN formats SHOULD NOT be used for RDDP/RDMA
protocol security; The ID_KEY_ID Identity Payload MUST NOT be
used. As described in [RFC2407], within Phase 1 the ID port and
protocol fields MUST be set to zero or to UDP port 500. Also, as
noted in [RFC2407]: When an IKE exchange is authenticated using
certificates (of any format), any ID's used for input to local
policy decisions SHOULD be contained in the certificate used in
the authentication of the exchange.
+-------+--------------------------+-------+---+---+---+---+---+ The Phase 2 Quick Mode exchanges used by RDDP/RDMA protocol
| Sec | Attack Name |Threat | 1 | 2 | 3 | 4 | 5 | implementations SHOULD explicitly carry the Identity Payload
+-------+--------------------------+-------+---+---+---+---+---+ fields (IDci and IDcr). Each Phase 2 IDci and IDcr Payload
| 7.2.1 | STag use on different | Spoof | | | | | | SHOULD carry a single IP address (ID_IPV4_ADDR, ID_IPV6_ADDR) and
| | connection in same PD | | | | | | | SHOULD NOT use the IP Subnet or IP Address Range formats. Other
+-------+--------------------------+-------+---+---+---+---+---+ ID payload formats MUST NOT be used.
| 7.3.1 | Memory write outside of | Tamper| | | | | |
| | buffer range | | | | | | | To support iSCSI PFS requirements [IPSSEC}, conformant RDDP/RDMA
| 7.3.2 | Modify Buffer after | Tamper| | | | | | security implementation SHOULD support PFS in the rekeying
| | contents ready | | | | | | | process (i.e. in the Quick Mode exchange).
+-------+--------------------------+-------+---+---+---+---+---+
| 7.4.1 | Probe memory outside of | ID | | | | | | Since IPsec acceleration hardware may only be able to handle a
| | buffer bounds | | | | | | | limited number of active IKE Phase 2 SAs, Phase 2 delete messages
| 7.4.2 | Access stale data | ID | | | | | | may be sent for idle SAs, as a means of keeping the number of
| 7.4.3 | Access buffer after | ID | | | | | | active Phase 2 SAs to a minimum. The receipt of an IKE Phase 2
| | transfer over | | | | | | | delete message MUST NOT be interpreted as a reason for tearing
| 7.4.4 | Unintended data access | ID | | | | | | down an RDDP/RDMA Stream. Rather, it is preferable to leave the
| | using valid STag | | | | | | | Stream up, and if additional traffic is sent on it, to bring up
| 7.4.5 | Using RDMA Read on a | ID | | | | | | another IKE Phase 2 SA to protect it. This avoids the potential
| | buffer meant only for | | | | | | | for continually bringing Streams up and down.
| | RDMA Write | | | | | | |
| 7.4.6 | Using multiple STags to | ID | | | | | | 8.2.4 Security Policy Configuration
| | access the same buffer | | | | | | |
| 7.4.7 | Remote node loading | ID | | | | | | One of the goals of this specification is to enable a high level
| | firmware onto RNIC | | | | | | | of interoperability without requiring extensive configuration.
+-------+--------------------------+-------+---+---+---+---+---+ This section provides guidelines on setting of IKE parameters so
| 7.5.1 | RNIC resource consumption| DOS | | | | | | as to enhance the probability of a successful negotiation. It
| 7.5.2 | Resource consumption by | DOS | | | | | | also describes how information on security policy configuration
| | active processes | | | | | | | can be provided so as to further enhance the chances of success.
| 7.5.3 | Resource consumption by | DOS | | | | | |
| | idle processes | | | | | | | To enhance the prospects for interoperability, some of the
| 7.5.4 | Non-optimal code paths | DOS | | | | | | actions to consider include:
+-------+--------------------------+-------+---+---+---+---+---+
| 7.6.1 | Loading firmware onto | Elev | | | | | | [1] Transform restriction. Since support for 3DES-CBC and HMAC-
| | RNIC | | | | | | | SHA1 is required of all implementations, offering these
+-------+--------------------------+-------+---+---+---+---+---+ transforms enhances the probability of a successful negotiation.
If AES-CTR [AESCTR] with XCBC-MAC [AESXCBC] is supported, this
transform combination will typically be preferred, with 3DES-
CBC/HMAC-SHA1 as a secondary offer.
[2] Group Restriction. If 3DES-CBC/HMAC-SHA1 is offered, and DH
groups are offered, then it is recommended that a DH group of at
least 1024 bits be offered along with it. If AES-CTR/XCBC-MAC is
the preferred offer, and DH groups are offered, then it is
recommended that a DH group of at least 2048 bits be offered
along with it, as noted in [KeyLen]. If perfect forward secrecy
is required in Quick Mode, then it is recommended that the QM PFS
DH group be the same as the IKE Phase 1 DH group. This reduces
the total number of combinations, enhancing the chances for
interoperability.
[3] Key lifetimes. If a key lifetime is offered that is longer
than desired, then rather than causing the IKE negotiation to
fail, it is recommended that the Responder consider the offered
lifetime as a maximum, and accept it. The key can then use a
lesser value for the lifetime, and utilize a Lifetime Notify in
order to inform the other peer of lifetime expiration.
Even when the above advice is taken, it still may be useful to be
able to provide additional configuration information in order to
enhance the chances of success, and it is useful to be able to
manage security configuration regardless of the scale of the
deployment.
For example, it may be desirable to configure the security policy
of an RDDP/RDMA device. This can be done manually or
automatically via a security policy distribution mechanism.
Alternatively, if the ULP supports a distribution mechanism such
as iSCSI with iSNS or SLPv2, those mechanism can be used to
supply security policy. If an IP block storage endpoint can
obtain the required security policy by other means (manually, or
automatically via a security policy distribution mechanism) then
it need not request this information through the ULP specific
mechanism. However, if the required security policy configuration
is not available via other mechanisms, those mechanismm can be
used.
It may also be helpful to obtain information about the
preferences of the peer prior to initiating IKE. While it is
generally possible to negotiate security parameters within IKE,
there are situations in which incompatible parameters can cause
the IKE negotiation to fail. The following information can be
provided via ULP specific or other mechanisms:
[4] IPsec or cleartext support. The minimum piece of peer
configuration required is whether an RDDP/RDMA endpoint requires
IPsec or cleartext. This cannot be determined from the IKE
negotiation alone without risking a long timeout, which is highly
undesirable for the RDMA/DDP protocol.
[5] Perfect Forward Secrecy (PFS) support. It is helpful to know
whether a peer allows PFS, since an IKE Phase 2 Quick Mode can
fail if an initiator proposes PFS to a Responder that does not
allow it.
[6] Preference for tunnel mode. While it is legal to propose
both transport and tunnel mode within the same offer, not all IKE
implementations will support this. As a result, it is useful to
know whether a peer prefers tunnel mode or transport mode, so
that it is possible to negotiate the preferred mode on the first
try.
[7] Main Mode and Aggressive Mode support. Since the IKE
negotiation can fail if a mode is proposed to a peer that doesn't
allow it, it is helpful to know which modes a peer allows, so
that an allowed mode can be negotiated on the first try.
Since iSNS or SLPv2 can be used to distribute IPsec security
policy and configuration information for use with IP block
storage protocols and RDDP/RDMA, these discovery protocols would
constitute a 'weak link' were they not secured at least as well
as the protocols whose security they configure. Since the major
vulnerability is packet modification and replay, when iSNS or
SLPv2 are used to distribute security policy or configuration
information, at a minimum, per-packet data origin authentication,
integrity and replay protection MUST be used to protect the
discovery protocol.
9 Security considerations
Issue: I think we should refer to IPS security considerations.
Most of the issues discussed there are relevant for RDDP/RDMA as
well (exceptions are the discussion on user certificates).<TBD>
Figure 2 - Summary Attacks and Trust Model Table
10 References 10 References
10.1 Normative References 10.1 Normative References
[RFC2828] Shirley, R., "Internet Security Glossary", FYI 36, RFC [RFC2828] Shirley, R., "Internet Security Glossary", FYI 36, RFC
2828, May 2000. 2828, May 2000.
[DDP] Shah, H., J. Pinkerton, R.Recio, and P. Culley, "Direct Data [DDP] Shah, H., J. Pinkerton, R.Recio, and P. Culley, "Direct
Placement over Reliable Transports", Internet-Draft draft-ietf- Data Placement over Reliable Transports", Internet-Draft
rddp-ddp-01.txt, February 2003. draft-ietf-rddp-ddp-01.txt, February 2003.
[RDMAP] Recio, R., P. Culley, D. Garcia, J. Hilland, "An RDMA [RDMAP] Recio, R., P. Culley, D. Garcia, J. Hilland, "An RDMA
Protocol Specification", Internet-Draft draft-ietf-rddp-rdmap- Protocol Specification", Internet-Draft draft-ietf-rddp-
01.txt, February 2003. rdmap-01.txt, February 2003.
[SEC-CONS] Rescorla, E., B. Korver, IAB, "Guidelines for Writing [SEC-CONS] Rescorla, E., B. Korver, IAB, "Guidelines for Writing
RFC Text on Security Considerations", Internet-Draft draft-ab- RFC Text on Security Considerations", Internet-Draft draft-
sec-cons-03.txt, January 2003. ab-sec-cons-03.txt, January 2003.
[RFC2401] Atkinson, R. and Kent, S., "Security Architecture for the [RFC2246] T. Dierks, C. Allen, "The TLS Protocol Version 1.0",
Internet Protocol", RFC 2401, November 1998 RFC 2246, January 1999.
[RFC2401] Atkinson, R. and Kent, S., "Security Architecture for
the Internet Protocol", RFC 2401, November 1998
[RFC2402] Kent, S., Atkinson, R., "IP Authentication Header", RFC [RFC2402] Kent, S., Atkinson, R., "IP Authentication Header", RFC
2402, November 1998 2402, November 1998
[RFC2404] Madson, C., Glenn, R., "The Use of HMAC-SHA-1-96 within [RFC2404] Madson, C., Glenn, R., "The Use of HMAC-SHA-1-96 within
ESP and AH", RFC 2404, November 1998 ESP and AH", RFC 2404, November 1998
[RFC2406] Kent, S., Atkinson, R., "IP Encapsulating Security [RFC2406] Kent, S., Atkinson, R., "IP Encapsulating Security
Payload (ESP)", RFC 2406, November 1998 Payload (ESP)", RFC 2406, November 1998
[RFC2407] Piper, D., "The Internet IP Security Domain of [RFC2407] Piper, D., "The Internet IP Security Domain of
Interpretation of ISAKMP", RFC 2407, November 1998 Interpretation of ISAKMP", RFC 2407, November 1998
[RFC2408] Maughan, D., Schertler, M., Schneider, M., Turner, J., [RFC2408] Maughan, D., Schertler, M., Schneider, M., Turner, J.,
"Internet Security Association and Key Management Protocol "Internet Security Association and Key Management Protocol
(ISAKMP), RFC 2408, November 1998 (ISAKMP), RFC 2408, November 1998
[RFC2409] Harkins, D., Carrel, D., "The Internet Key Exchange [RFC2409] Harkins, D., Carrel, D., "The Internet Key Exchange
(IKE)", RFC 2409, November 1998 (IKE)", RFC 2409, November 1998
[AESCTR] Housley, R., "Using AES Counter Mode With IPsec
ESP",Internet draft (work in progress), draft-ietf-ipsec-
ciph-aes-ctr-05.txt, July 2003
[KeyLen] Orman, H., Hoffman, P., "Determining Strengths For
Public Keys Used For Exchanging Symmetric Keys", Internet
draft (work in progress), draft-orman-public-key-lengths-
07.txt, January 2004
[AESXCBC] Frankel, S., Herbert, H., "The AES-XCBC-MAC-96
Algorithm and Its Use with IPsec", Internet draft (work in
progress), draft-ietf-ipsec-ciph-aes-xcbc-mac-02.txt, June
2002
[IPSSEC] Aboba B., et al, "Securing Block Storage Protocols over
IP", Internet draft (work in progress), draft-ietf-ips-
security-19.txt, January 2003
[SCTP] R. Stewart et al., "Stream Control Transmission Protocol",
RFC 2960, October 2000.
[TCP] Postel, J., "Transmission Control Protocol - DARPA Internet
Program Protocol Specification", RFC 793, September 1981.
10.2 Informative References 10.2 Informative References
[IPv6-Trust] Nikander, P., J.Kempf, E. Nordmark, "IPv6 Neighbor [IPv6-Trust] Nikander, P., J.Kempf, E. Nordmark, "IPv6 Neighbor
Discovery trust modelsTrust Models and threats", Internet-Draft Discovery trust modelsTrust Models and threats", Internet-
draft-ietf-send-psreq-01.txt, January 2003. Draft draft-ietf-send-psreq-01.txt, January 2003.
11 AuthorĂs Addresses 11 Appendix A: Implementing Client/Server Protocols
The prior sections outlined specific attacks and their
countermeasures. This section summarizes the attacks and
countermeasures defined in the prior section which are applicable
to creation of a secure application server. An application server
is defined as an application which must be able to communicate
with many clients which do not trust each other and ensure that
each client can not attack another client through server
interactions. Further, the server may wish to use multiple
Streams to communicate with a specific client, and those Streams
may share mutual trust.
All of the prior section's details on attacks and countermeasures
to protect a single Stream apply to the server. This section
focuses on security issues where multiple clients are talking
with a single server.
The following list summarizes the relevent attacks that clients
can mount on the shared server, by re-stating the previous
RECOMMENDations to be client/server specific (the following are
just restatements of the prior RECOMMENDations):
* Spoofing
* Section 7.2.4 Using an STag on a Different on page
23. To ensure that one client can not access another
client's data via use of their STag, it is
RECOMMENDED that the server either scope an STag to a
single Stream or use a Protection Domain per client,
or a combination of the two approaches.
* Tampering
* 7.3.3 Multiple STags to access the same buffer on
page 25. See the following bullet's discussion of
Section 7.4.6.
* Information Disclosure
* 7.4.2 Using RDMA Read to Access Stale Data on page
26. It is RECOMMENDED that the server ensure that no
stale data is contained in a buffer before remote
read access rights are granted to a client (this can
be done by zeroing the contents of the memory, for
example).
* 7.4.5 RDMA Read into an RDMA Write Buffer on page 27.
It is RECOMMENDED that if a server only intends a
buffer to be exposed for remote write access, it set
the access rights to the buffer to only enable remote
write access.
* 7.4.6 Using Multiple STags to Access One Buffer on
page 27. It is RECOMMENDED that separate clients not
be granted write access to the same buffer through
different STags. A buffer should be exposed to only
one client at a time to ensure that no information
disclosure or information tampering occurs between
peers.
* Denial of Service
* 7.5.1 RNIC Resource Consumption on page 29. It is
RECOMMENDED that the server place the allocation of
all scarce resources be placed under the control of a
Privileged Resource Manager.
* 7.5.2.1 Multiple Streams Sharing Receive Buffers on
page 30. If an RNIC Engine provides the ability to
share receive buffers across multiple Streams, it is
RECOMMENDED that it enable the server to detect if
the client is attempting to consume more than its
fair share of resources so that the server can apply
countermeasures to detect and prevent the attack.
* 7.5.2.2 Local Peer Attacking a Shared CQ on page 31.
Sharing a CQ across Streams that belong to different
Protection Domains is NOT RECOMMENDED.
* 7.5.2.3 Remote Peer Attacking a Shared CQ on page 32.
If a server allows the client to influence CQ entry
resource allocation, then it is RECOMMENDED that the
CQ be isolated to Streams within a single Protection
Domain (i.e. streams that share Partial Mutual
Trust).
It is RECOMMENDED that the Local Peer implement a
mechanism to ensure that the Completion Queue can not
overflow.
* 7.5.2.4 RDMA Read Request Queue on page 34. It is
RECOMMENDED that access to interfaces that allocate
RDMA Read Request Queue entries be restricted to a
trusted Local Peer, such as a Privileged Resource
Manager.
It is RECOMMENDED that RDMA Read Request Queue
resource consumption be controlled such that
RDMAP/DDP Streams which do not share Partial Mutual
Trust do not share RDMA Read Request Queue resources.
* 7.5.3 Resource Consumption by Idle Applications on
page 35. Refer to Section 7.5.1.
* 7.5.5 RI an STag Shared on Multiple Streams on page
36. If RDDP Streams do not share Partial Mutual Trust
(i.e. the client may attempt to invalidate the STag
prematurely), it is NOT RECOMMENDED that the server
allow an STag to be valid across multiple Streams.
12 Appendix B: Summary Table of Attacks
Issue: Finish Summary table of Attacks/Trust Models <TBD>
<editor: This section is under construction, and will be
completed in a future version of this document>
Rows are the attack (grouped into categories)
Columns are the:
* Sec - Section the attack is discussed
* Attack Name - short name for the attack
* Threat - threat type (Spoof (Spoofing), Tamp (Tampering),
ID (Information Disclosure), and DOS (Denial of Service))
* SH ű Does the threat assume there are shared resources
(yes/no/NA ű not applicable)?
* TR ű Does the threat assume there is Partial Mutual Trust
between Streams (MT), no trust between Streams (NT), or
is this parameter not applicable (NA)?
12.1 Spoofing
+--------+---------------------------------------------+-----+--+--+
| Sec | Attack Name |Sh|TR|
+--------+---------------------------------------------+-----+--+--+
| 7.2.1 | Impersonation |NA|NA|
+--------+---------------------------------------------+-----+--+--+
| 7.2.2 | Stream Hijacking |NA|NA|
+--------+---------------------------------------------+-----+--+--+
| 7.2.3 | Man in the Middle Attack |NA|NA|
+--------+---------------------------------------------+-----+--+--+
| 7.2.4 | Using an STag on a Different |Y |NT|
+--------+---------------------------------------------+-----+--+--+
12.2 Tampering
+--------+---------------------------------------------+-----+--+--+
| 7.3.1 | Buffer Overrun - RDMA Write or Read Response|NA|NT|
+--------+---------------------------------------------+-----+--+--+
| 7.3.2 | Modifying a Buffer After Indication |NA|NT|
+--------+---------------------------------------------+-----+--+--+
| 7.3.3 | Multiple STags to access the same buffer |Y |NT|
+--------+---------------------------------------------+-----+--+--+
| 7.3.4 | Network based modification of buffer content|NA|NA|
+--------+---------------------------------------------+-----+--+--+
12.3 Information Disclosure
+--------+---------------------------------------------+-----+--+--+
| 7.4.1 | Probing memory outside of the buffer bounds |NA|NT|
+--------+---------------------------------------------+-----+--+--+
| 7.4.2 | Using RDMA Read to Access Stale Data |
+--------+---------------------------------------------+-----+--+--+
| 7.4.3 | Accessing a Buffer After the Transfer |
+--------+---------------------------------------------+-----+--+--+
| 7.4.4 | Accessing Unintended Data With a Valid STag |
+--------+---------------------------------------------+-----+--+--+
| 7.4.5 | RDMA Read into an RDMA Write Buffer |
+--------+---------------------------------------------+-----+--+--+
| 7.4.6 | Using Multiple STags to Access One Buffer |
+--------+---------------------------------------------+-----+--+--+
| 7.4.7 | Remote Node Loading Firmware onto the RNIC |
+--------+---------------------------------------------+-----+--+--+
| 7.4.8 | Controlling Access to PTT & STag Mapping |
+--------+---------------------------------------------+-----+--+--+
| 7.4.9 | Network based eaves dropping |
+--------+---------------------------------------------+-----+--+--+
12.4 Denial of Service
+--------+---------------------------------------------+-----+--+--+
| 7.5.1 | RNIC Resource Consumption |
+--------+---------------------------------------------+-----+--+--+
| 7.5.2.1| Multiple Streams Sharing Receive Buffers |
+--------+---------------------------------------------+-----+--+--+
| 7.5.2.2| Local Peer Attacking a Shared CQ |Error!
Reference source not found.
+--------+---------------------------------------------+-----+--+--+
| 7.5.2.3| Remote Peer Attacking a Shared CQ |
+--------+---------------------------------------------+-----+--+--+
| 7.5.2.4| RDMA Read Request Queue |
+--------+---------------------------------------------+-----+--+--+
| 7.5.3 | Resource Consumption by Idle Applications |
+--------+---------------------------------------------+-----+--+--+
| 7.5.4 | Exercise of non-optimal code paths |
+--------+---------------------------------------------+-----+--+--+
| 7.5.5 | RI an STag Shared on Multiple Streams |
+--------+---------------------------------------------+-----+--+--+
| 7.5.6 | Remote Peer Consumes Untagged Receive Buffers|
+--------+---------------------------------------------+-----+--+--+
Figure 2 - Summary Attacks and Trust Model Table
13 Appendix C: Partial Trust Taxonomy
Partial Trust is defined as when one party is willing to assume
that another party will refrain from a specific attack or set of
attacks, the parties are said to be in a state of Partial Trust.
Note that the partially trusted peer may attempt a different set
of attacks. This may be appropriate for many applications where
any adverse effects of the betrayal is easily confined and does
not place other clients or applications at risk.
The Trust Models described in this section have three primary
distinguishing characteristics. The Trust Model refers to a Local
Peer and Remote Peer, which are the local and remote application
instances communicating via RDMA/DDP.
* Local Resource Sharing (yes/no) - When local resources
are shared, they are shared across a grouping of
RDMAP/DDP Streams. If local resources are not shared, the
resources are dedicated on a per Stream basis. Resources
are defined in Section 4.2 - Resources on page 11. The
advantage of not sharing resources between Streams is
that it reduces the types of attacks that are possible.
The disadvantage is that applications might run out of
resources.
* Local Partial Trust (yes/no) - Local Partial Trust is
determined based on whether the local grouping of
RDMAP/DDP Streams (which typically equates to one
application or group of applications) mutually trust each
other to not perform a specific set of attacks.
* Remote Partial Trust (yes/no) - The Remote Partial Trust
level is determined based on whether the Local Peer of a
specific RDMAP/DDP Stream partially trusts the Remote
Peer of the Stream (see the definition of Partial Trust
in Section 3 Introduction).
Not all of the combinations of the trust characteristics are
expected to be used by applications. This paper specifically
analyzes five application Trust Models that are expected to be in
common use. The Trust Models are as follows:
1. NS-NT - Non-Shared Local Resources, no Local Trust, no Remote
Trust - typically a server application that wants to run in
the safest mode possible. All attack mitigations are in place
to ensure robust operation.
2. NS-RT - Non-Shared Local Resources, no Local Trust, Remote
Partial Trust - typically a peer-to-peer application, which
has, by some method outside of the scope of this
specification, authenticated the Remote Peer. Note that
unless some form of key based authentication is used on a per
RDMA/DDP session basis, it may not be possible be possible
for man-in-the-middle attacks to occur. See section 8,
Security Services for RDDP on page 38.
3. S-NT - Shared Local Resources, no Local Trust, no Remote
Trust - typically a server application that runs in an
untrusted environment where the amount of resources required
is either too large or too dynamic to dedicate for each
RDMAP/DDP Stream.
4. S-LT - Shared Local Resources, Local Partial Trust, no Remote
Trust - typically an application, which provides a session
layer and uses multiple Streams, to provide additional
throughput or fail-over capabilities. All of the Streams
within the local application partially trust each other, but
do not trust the Remote Peer. This trust model may be
appropriate for embedded environments.
5. S-T - Shared Local Resources, Local Partial Trust, Remote
Partial Trust - typically a distributed application, such as
a distributed database application or a High Performance
Computer (HPC) application, which is intended to run on a
cluster. Due to extreme resource and performance
requirements, the application typically authenticates with
all of its peers and then runs in a highly trusted
environment. The application peers are all in a single
application fault domain and depend on one another to be
well-behaved when accessing data structures. If a trusted
Remote Peer has an implementation defect that results in poor
behavior, the entire application could be corrupted.
Models NS-NT and S-NT above are typical for Internet networking -
neither Local Peers nor the Remote Peer is trusted. Sometimes
optimizations can be done that enable sharing of Page Translation
Tables across multiple Local Peers, thus Model S-LT can be
advantageous. Model S-T is typically used when resource scaling
across a large parallel application makes it infeasible to use
any other model. Resource scaling issues can either be due to
performance around scaling or because there simply are not enough
resources. Model NS-RT is probably the least likely model to be
used, but is presented for completeness.
14 AuthorĂs Addresses
James Pinkerton James Pinkerton
Microsoft Corporation Microsoft Corporation
One Microsoft Way One Microsoft Way
Redmond, WA. 98052 USA Redmond, WA. 98052 USA
Phone: +1 (425) 705-5442 Phone: +1 (425) 705-5442
Email: jpink@windows.microsoft.com Email: jpink@windows.microsoft.com
Ellen Deleganes Ellen Deleganes
Intel Corporation Intel Corporation
MS JF5-355 MS JF5-355
2111 NE 25th Ave. 2111 NE 25th Ave.
Hillsboro, OR 97124 USA Hillsboro, OR 97124 USA
Phone: +1 (503) 712-4173 Phone: +1 (503) 712-4173
Email: ellen.m.deleganes@intel.com Email: ellen.m.deleganes@intel.com
Allyn Romanow Allyn Romanow
Cisco Systems Cisco Systems
170 W Tasman Drive 170 W Tasman Drive
San Jose, CA 95134 USA San Jose, CA 95134 USA
skipping to change at page 41, line 29 skipping to change at page 56, line 33
Phone: +1 (503) 712-4173 Phone: +1 (503) 712-4173
Email: ellen.m.deleganes@intel.com Email: ellen.m.deleganes@intel.com
Allyn Romanow Allyn Romanow
Cisco Systems Cisco Systems
170 W Tasman Drive 170 W Tasman Drive
San Jose, CA 95134 USA San Jose, CA 95134 USA
Phone: +1 408 525 8836 Phone: +1 408 525 8836
Email: allyn@cisco.com Email: allyn@cisco.com
Bernard Aboba Sara Bitan
Microsoft Corporation Microsoft Corporation
One Microsoft Way Email: sarab@microsoft.com
Redmond, WA. 98052 USA
Phone: +1 (425) 706-6606 15 Acknowledgments
Email: bernarda@windows.microsoft.com
12 Acknowledgments
Catherine Meadows Catherine Meadows
Naval Research Laboratory Naval Research Laboratory
Code 5543 Code 5543
Washington, DC 20375 Washington, DC 20375
Email: meadows@itd.nrl.navy.mil Email: meadows@itd.nrl.navy.mil
Patricia Thaler Patricia Thaler
Agilent Technologies, Inc. Agilent Technologies, Inc.
1101 Creekside Ridge Drive, #100 1101 Creekside Ridge Drive, #100
skipping to change at page 43, line 4 skipping to change at page 57, line 37
John Carrier John Carrier
Adaptec, Inc. Adaptec, Inc.
691 S. Milpitas Blvd. 691 S. Milpitas Blvd.
Milpitas, CA 95035 USA Milpitas, CA 95035 USA
Phone: +1 (360) 378-8526 Phone: +1 (360) 378-8526
Email: john_carrier@adaptec.com Email: john_carrier@adaptec.com
Caitlin Bestler Caitlin Bestler
Email: cait@asomi.com Email: cait@asomi.com
13 Full Copyright Statement
Bernard Aboba
Microsoft Corporation
One Microsoft Way
Redmond, WA. 98052 USA
Phone: +1 (425) 706-6606
Email: bernarda@windows.microsoft.com
16 Full Copyright Statement
Copyright (C) The Internet Society (2001). All Rights Reserved. Copyright (C) The Internet Society (2001). All Rights Reserved.
This document and translations of it may be copied and furnished to This document and translations of it may be copied and furnished
others, and derivative works that comment on or otherwise explain it to others, and derivative works that comment on or otherwise
or assist in its implementation may be prepared, copied, published explain it or assist in its implementation may be prepared,
and distributed, in whole or in part, without restriction of any copied, published and distributed, in whole or in part, without
kind, provided that the above copyright notice and this paragraph restriction of any kind, provided that the above copyright notice
are included on all such copies and derivative works. However, this and this paragraph are included on all such copies and derivative
document itself may not be modified in any way, such as by removing works. However, this document itself may not be modified in any
the copyright notice or references to the Internet Society or other way, such as by removing the copyright notice or references to
Internet organizations, except as needed for the purpose of the Internet Society or other Internet organizations, except as
developing Internet standards in which case the procedures for needed for the purpose of developing Internet standards in which
copyrights defined in the Internet Standards process must be case the procedures for copyrights defined in the Internet
followed, or as required to translate it into languages other than Standards process must be followed, or as required to translate
English. it into languages other than English.
The limited permissions granted above are perpetual and will not be The limited permissions granted above are perpetual and will not
revoked by the Internet Society or its successors or assigns. be revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an This document and the information contained herein is provided on
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR
PURPOSE.
Funding for the RFC Editor function is currently provided by the Funding for the RFC Editor function is currently provided by the
Internet Society. Internet Society.
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/