draft-ietf-rddp-security-02.txt   draft-ietf-rddp-security-03.txt 
Internet Draft James Pinkerton Internet Draft James Pinkerton
draft-ietf-rddp-security-02.txt Microsoft Corporation draft-ietf-rddp-security-03.txt Microsoft Corporation
Expires: January, 2005 Ellen Deleganes Category: Standards Track Ellen Deleganes
Intel Corporation Expires: February, 2005 Intel Corporation
Sara Bitan Sara Bitan
Microsoft Corporation Microsoft Corporation
July 2004 August 2004
DDP/RDMAP Security DDP/RDMAP Security
1 Status of this Memo 1 Status of this Memo
This document is an Internet-Draft and is in full conformance This document is an Internet-Draft and is in full conformance
with all provisions of Section 10 of RFC2026. with all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet- documents at any time. It is inappropriate to use Internet-Drafts
Drafts as reference material or to cite them other than as "work as reference material or to cite them other than as "work in
in progress." progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
2 Abstract 2 Abstract
This document analyzes security issues around implementation and This document analyzes security issues around implementation and
use of the Direct Data Placement Protocol(DDP) and Remote Direct use of the Direct Data Placement Protocol(DDP) and Remote Direct
Memory Access Protocol (RDMAP). It first defines an architectural Memory Access Protocol (RDMAP). It first defines an architectural
model for an RDMA Network Interface Card (RNIC), which can model for an RDMA Network Interface Card (RNIC), which can
implement DDP or RDMAP and DDP. The document reviews various implement DDP or RDMAP and DDP. The document reviews various
attacks against the resources defined in the architectural model attacks against the resources defined in the architectural model
and the countermeasures that can be used to protect the system. and the countermeasures that can be used to protect the system.
Attacks are grouped into spoofing, tampering, information Attacks are grouped into spoofing, tampering, information
disclosure, denial of service, and elevation of privilege. disclosure, denial of service, and elevation of privilege.
Finally, the document concludes with a summary of security Finally, the document concludes with a summary of security
services for RDDP, such as IPSec. services for DDP and RDMAP, such as IPSec.
J. Pinkerton, et al. Expires January 2005 1 J. Pinkerton, et al. Expires February 2005 1
Table of Contents Table of Contents
1 Status of this Memo.........................................1 1 Status of this Memo.........................................1
2 Abstract....................................................1 2 Abstract....................................................1
2.1 Issues......................................................3 2.1 Issues......................................................3
2.2 Revision History............................................4 2.2 Revision History............................................4
2.2.1 Changes from the -01 to the -02 version....................4 2.2.1 Changes from -02 to -03 version............................4
2.2.2 Changes from the -00 to -01 version........................4 2.2.2 Changes from the -01 to the -02 version....................4
2.2.3 Changes from the -00 to -01 version........................4
3 Introduction................................................6 3 Introduction................................................6
4 Architectural Model.........................................8 4 Architectural Model.........................................8
4.1 Components..................................................9 4.1 Components..................................................9
4.2 Resources..................................................11 4.2 Resources..................................................11
4.2.1 Stream Context Memory.....................................11 4.2.1 Stream Context Memory.....................................11
4.2.2 Data Buffers..............................................11 4.2.2 Data Buffers..............................................11
4.2.3 Page Translation Tables...................................11 4.2.3 Page Translation Tables...................................11
4.2.4 STag Namespace............................................12 4.2.4 STag Namespace............................................12
4.2.5 Completion Queues.........................................12 4.2.5 Completion Queues.........................................12
4.2.6 Asynchronous Event Queue..................................12 4.2.6 Asynchronous Event Queue..................................12
skipping to change at page 2, line ? skipping to change at page 2, line ?
7.2 Spoofing...................................................21 7.2 Spoofing...................................................21
7.2.1 Impersonation.............................................22 7.2.1 Impersonation.............................................22
7.2.2 Stream Hijacking..........................................22 7.2.2 Stream Hijacking..........................................22
7.2.3 Man in the Middle Attack..................................22 7.2.3 Man in the Middle Attack..................................22
7.2.4 Using an STag on a Different Stream.......................23 7.2.4 Using an STag on a Different Stream.......................23
7.3 Tampering..................................................24 7.3 Tampering..................................................24
7.3.1 Buffer Overrun - RDMA Write or Read Response..............24 7.3.1 Buffer Overrun - RDMA Write or Read Response..............24
7.3.2 Modifying a Buffer After Indication.......................25 7.3.2 Modifying a Buffer After Indication.......................25
7.3.3 Multiple STags to access the same buffer..................25 7.3.3 Multiple STags to access the same buffer..................25
7.3.4 Network based modification of buffer content..............25 7.3.4 Network based modification of buffer content..............25
7.4 Information Disclosure.....................................25 7.4 Information Disclosure.....................................26
7.4.1 Probing memory outside of the buffer bounds...............26 7.4.1 Probing memory outside of the buffer bounds...............26
7.4.2 Using RDMA Read to Access Stale Data......................26 7.4.2 Using RDMA Read to Access Stale Data......................26
7.4.3 Accessing a Buffer After the Transfer.....................26 7.4.3 Accessing a Buffer After the Transfer.....................26
7.4.4 Accessing Unintended Data With a Valid STag...............26 7.4.4 Accessing Unintended Data With a Valid STag...............26
7.4.5 RDMA Read into an RDMA Write Buffer.......................27 7.4.5 RDMA Read into an RDMA Write Buffer.......................27
7.4.6 Using Multiple STags to Access One Buffer.................27 7.4.6 Using Multiple STags Which Alias to the Same Buffer.......27
7.4.7 Remote Node Loading Firmware onto the RNIC................28 7.4.7 Remote Node Loading Firmware onto the RNIC................28
7.4.8 Controlling Access to PTT & STag Mapping..................28 7.4.8 Controlling Access to PTT & STag Mapping..................28
7.4.9 Network based eaves dropping..............................28 7.4.9 Network based eavesdropping...............................28
7.5 Denial of Service (DOS)....................................28 7.5 Denial of Service (DOS)....................................29
7.5.1 RNIC Resource Consumption.................................29 7.5.1 RNIC Resource Consumption.................................29
7.5.2 Resource Consumption By Active Applications...............30 7.5.2 Resource Consumption By Active Applications...............30
7.5.2.1 Multiple Streams Sharing Receive Buffers..............30 7.5.2.1 Multiple Streams Sharing Receive Buffers..............30
7.5.2.2 Local Peer Attacking a Shared CQ......................31 7.5.2.2 Local Peer Attacking a Shared CQ......................31
7.5.2.3 Remote Peer Attacking a Shared CQ.....................32 7.5.2.3 Remote Peer Attacking a Shared CQ.....................32
7.5.2.4 RDMA Read Request Queue...............................34 7.5.2.4 Attacking the RDMA Read Request Queue.................35
7.5.3 Resource Consumption by Idle Applications.................35 7.5.3 Resource Consumption by Idle Applications.................36
7.5.4 Exercise of non-optimal code paths........................35 7.5.4 Exercise of non-optimal code paths........................36
7.5.5 RI an STag Shared on Multiple Streams.....................36 7.5.5 Remote Invalidate an STag Shared on Multiple Streams......37
7.6 Elevation of Privilege.....................................36 7.6 Elevation of Privilege.....................................37
8 Security Services for RDDP.................................37 8 Security Services for RDMA and DDP.........................38
8.1 Introduction to Security Options...........................37 8.1 Introduction to Security Options...........................38
8.1.1 Introduction to IPsec.....................................38 8.1.1 Introduction to IPsec.....................................38
8.1.2 Introduction to SSL Limitations on RDMAP..................39 8.1.2 Introduction to SSL Limitations on RDMAP..................40
8.1.3 Applications Which Provide Security.......................39 8.1.3 Applications Which Provide Security.......................40
8.2 Recommendations for IPsec Encapsulation of RDDP............39 8.2 Requirements for IPsec Encapsulation of DDP................40
8.2.1 Transforms................................................40 9 Security considerations....................................42
8.2.2 IPsec modes...............................................40 10 References.................................................43
8.2.3 IKE.......................................................40 10.1 Normative References......................................43
8.2.4 Security Policy Configuration.............................42 10.2 Informative References....................................43
9 Security considerations....................................44 11 Appendix A: Implementing Client/Server Protocols...........44
10 References.................................................45 12 Appendix B: Summary Table of Attacks.......................47
10.1 Normative References......................................45 13 Appendix C: Partial Trust Taxonomy.........................48
10.2 Informative References....................................46 14 AuthorĘs Addresses.........................................50
11 Appendix A: Implementing Client/Server Protocols...........47 15 Acknowledgments............................................51
12 Appendix B: Summary Table of Attacks.......................50 16 Full Copyright Statement...................................52
12.1 Spoofing..................................................51
12.2 Tampering.................................................51
12.3 Information Disclosure....................................51
12.4 Denial of Service.........................................51
13 Appendix C: Partial Trust Taxonomy.........................53
14 AuthorĘs Addresses.........................................55
15 Acknowledgments............................................56
16 Full Copyright Statement...................................57
Table of Figures Table of Figures
Figure 1 - RDMA Security Model....................................9 Figure 1 - RDMA Security Model....................................9
Figure 2 - Summary Attacks and Trust Model Table.................52
2.1 Issues 2.1 Issues
This section is temporary and will go away when all issues have This section is temporary and will go away when all issues have
been resolved. been resolved.
Note: this is far from a complete list of issues; as more are
raised, they will be added to this list until some sort of
consensus is reached. They are in the order found in the
specification.
Issue: The spec currently took the IPSec requirements for iSCSI
and made them a SHOULD recommendation. A different approach would
be to simply outline the issues in this section, but leave IPSec
implementation requirements to be specified by ULP/Application
requirements. The argument here is that RDDP is a transport, and
security requirements ū particularly authentication and
confidentiality requirements, are dictated by application
concerns, not transport protocol concerns. Which approach should
be taken?........................................................37
Issue: Guidance for application protocols like NFS which Issue: Guidance for application protocols like NFS which
implement security <TBD>.........................................39 implement security <TBD>.........................................40
Issue: I think we should refer to IPS security considerations.
Most of the issues discussed there are relevant for RDDP/RDMA as
well (exceptions are the discussion on user certificates).<TBD>..44
Issue: Finish Summary table of Attacks/Trust Models <TBD>........50
2.2 Revision History 2.2 Revision History
2.2.1 Changes from the -01 to the -02 version 2.2.1 Changes from -02 to -03 version
Minimal ū some typos, deleted some text previously marked for * ID changed from Informational to Standards Track. This
caused previous RECOMMENDATIONS to be categorized into
the categories of MUST, SHOULD, MAY, RECOMMENDED, and in
one case, "recommended".
* Completed Appendix B: Summary of Attacks to provide a
summary of implementation requirements for applications
using RDDP and for RNICs in Appendix B: Summary of
Attacks.
* Modified intro to better explain when concept of Partial
Mutual Trust is useful.
* Misc minor changes from Tom Talpey's extensive review,
including:
* Send Queue/Receive Queue formally defined/used.
* RI is gone, now use RNIC interface, RNIC, and Remote
Invalidate.
* Clarified attackers capabilities.
* In many cases replaced "session" with "Stream".
* Added definitions for equation variables in section
7.5.2.3.
* Changed section 8.2 to normative xref to IPS Security,
plus comment on the value of end-to-end IPsec.
* Added clarifying example on STag invalidation (e.g. One-
Shot STag discussion).
* Added clarifying text on why SSL is a bad idea.
* Normative statement on mitigation for Shared RQ.
2.2.2 Changes from the -01 to the -02 version
Minimal - some typos, deleted some text previously marked for
deletion. deletion.
2.2.2 Changes from the -00 to -01 version 2.2.3 Changes from the -00 to -01 version
* Added two pages to the architectural model to describe * Added two pages to the architectural model to describe
the Asynchronous Event Queue, and the types of the Asynchronous Event Queue, and the types of
interactions that can occur between the RNIC and the interactions that can occur between the RNIC and the
modules above it. modules above it.
* Addressed Mike Krauses comments submitted on 12/8/2003 * Addressed Mike Krauses comments submitted on 12/8/2003
* Moved "Trust Models" from the body of the document to an * Moved "Trust Models" from the body of the document to an
appendix. Removed references to it throughout the appendix. Removed references to it throughout the
skipping to change at page 5, line 9 skipping to change at page 5, line 27
* The Summary of Attacks at the end of the document is now * The Summary of Attacks at the end of the document is now
an Appendix. It also now provides a summary. Cleared an Appendix. It also now provides a summary. Cleared
change bars because became unreadable. Also shortened change bars because became unreadable. Also shortened
section names for attacks to fit in table. section names for attacks to fit in table.
* Added a new concept of "Partial Mutual Trust" between a * Added a new concept of "Partial Mutual Trust" between a
collection of Streams to better characterize a set of collection of Streams to better characterize a set of
attacks in a client/server environment. attacks in a client/server environment.
* Filled in Security Services for RDDP section (almost all * Filled in Security Services for RDMA and DDP section
is new, except IPsec overview). (almost all is new, except IPsec overview).
* Globally tried to change "connection" to "Stream". In * Globally tried to change "connection" to "Stream". In
some cases it can be either a connection or stream. some cases it can be either a connection or stream.
3 Introduction 3 Introduction
RDMA enables new levels of flexibility when communicating between RDMA enables new levels of flexibility when communicating between
two parties compared to current conventional networking practice two parties compared to current conventional networking practice
(e.g. a stream-based model or datagram model). This flexibility (e.g. a stream-based model or datagram model). This flexibility
brings new security issues that must be carefully understood when brings new security issues that must be carefully understood when
skipping to change at page 6, line 23 skipping to change at page 6, line 23
of this security analysis, an RNIC may implement RDMAP and DDP, of this security analysis, an RNIC may implement RDMAP and DDP,
or just DDP. or just DDP.
The specification first develops an architectural model that is The specification first develops an architectural model that is
relevant for the security analysis - it details components, relevant for the security analysis - it details components,
resources, and system properties that may be attacked in Section resources, and system properties that may be attacked in Section
4. 4.
It then defines what resources a ULP may share locally across It then defines what resources a ULP may share locally across
Streams and what resources the ULP may share with the Remote Peer Streams and what resources the ULP may share with the Remote Peer
across Streams in Section 5. In general, intentional sharing of across Streams in Section 5. Intentional sharing of resources
resources between multiple Streams implies a trust model between between multiple Streams may imply some level of trust between
the Streams. This is defined as: the Streams. However, some types of resource sharing have
unmitigated security attacks which would mandate not sharing a
specific type of resource unless there is some level of trust
between the Streams sharing resources. Partial Mutual Trust is
defined to address this concept:
Partial Mutual Trust ū a collection of RDMAP/DDP Streams, Partial Mutual Trust - a collection of RDMAP/DDP Streams,
which represent the local and remote end points of the which represent the local and remote end points of the
Stream, are willing to assume that the Streams from the Stream, are willing to assume that the Streams from the
collection will not perform malicious attacks against any of collection will not perform malicious attacks against any of
the Streams in the collection. Applications have explicit the Streams in the collection. Applications have explicit
control of which collection of endpoints is in the control of which collection of endpoints is in the
collection through tools discussed in Section 7.1 Tools for collection through tools discussed in Section 7.1 Tools for
Countermeasures on page 19. Countermeasures on page 19.
An untrusted peer relationship is appropriate when an application An untrusted peer relationship is appropriate when an application
wishes to ensure that it will be robust and uncompromised even in wishes to ensure that it will be robust and uncompromised even in
the face of a deliberate attack by its peer. For example, a the face of a deliberate attack by its peer. For example, a
single application that concurrently supports multiple unrelated single application that concurrently supports multiple unrelated
sessions (e.g. a server) would presumably treat each of its peers Streams (e.g. a server) would presumably treat each of its peers
as an untrusted peer. For a collection of Streams which share as an untrusted peer. For a collection of Streams which share
Partial Mutual Trust, the assumption is that any Stream not in Partial Mutual Trust, the assumption is that any Stream not in
the collection is untrusted. For the untrusted peer, a brief list the collection is untrusted. For the untrusted peer, a brief list
of capabilities is enumerated in Section 6. of capabilities is enumerated in Section 6.
The rest of the specification is focused on analyzing attacks. The rest of the specification is focused on analyzing attacks.
First, the tools for mitigating attacks are listed (Section 7.1), First, the tools for mitigating attacks are listed (Section 7.1),
and then a series of attacks on components, resources, or system and then a series of attacks on components, resources, or system
properties is enumerated in the rest of Section 7. For each properties is enumerated in the rest of Section 7. For each
attack, possible countermeasures are reviewed. If all recommended attack, possible countermeasures are reviewed. If all recommended
skipping to change at page 8, line 11 skipping to change at page 8, line 11
the RNIC Engine that could affect other applications need to the RNIC Engine that could affect other applications need to
be done through a trusted intermediary that can verify the be done through a trusted intermediary that can verify the
Non-Privileged Application requests. Non-Privileged Application requests.
4 Architectural Model 4 Architectural Model
This section describes an RDMA architectural reference model that This section describes an RDMA architectural reference model that
is used as security issues are examined. It introduces the is used as security issues are examined. It introduces the
components of the model, the resources that can be attacked, the components of the model, the resources that can be attacked, the
types of interactions possible between components and resources, types of interactions possible between components and resources,
and the system properties, which should be preserved when under and the system properties which must be preserved.
attack.
Figure 1 shows the components comprising the architecture and the Figure 1 shows the components comprising the architecture and the
interfaces where potential security attacks could be launched. interfaces where potential security attacks could be launched.
External attacks can be injected into the system from an External attacks can be injected into the system from an
application that sits above the RI or from the Internet. application that sits above the RNIC Interface or from the
network.
The intent here is to describe high level components and The intent here is to describe high level components and
capabilities which affect threat analysis, and not focus on capabilities which affect threat analysis, and not focus on
specific implementation options. Also note that the architectural specific implementation options. Also note that the architectural
model is an abstraction, and an actual implementation may choose model is an abstraction, and an actual implementation may choose
to subdivide its components along different boundary lines than to subdivide its components along different boundary lines than
defined here. For example, the Privileged Resource Manager may be defined here. For example, the Privileged Resource Manager may be
partially or completely encapsulated in the Privileged partially or completely encapsulated in the Privileged
Application. Regardless, it is expected that the security Application. Regardless, it is expected that the security
analysis of the potential threats and countermeasures still analysis of the potential threats and countermeasures still
skipping to change at page 9, line 21 skipping to change at page 9, line 21
| ^ v v | ^ v v
| | +-------------+ +-----------------+ | | +-------------+ +-----------------+
|---------------->| Privileged | | Non-Privileged | |---------------->| Privileged | | Non-Privileged |
| | Application | | Application | | | Application | | Application |
| +-------------+ +-----------------+ | +-------------+ +-----------------+
| ^ ^ | ^ ^
|Privileged |Privileged |Non-Privileged |Privileged |Privileged |Non-Privileged
|Control |Data |Data |Control |Data |Data
|Interface |Interface |Interface |Interface |Interface |Interface
RNIC | | | RNIC | | |
Interface(RI) v v v Interface v v v
================================================================= =================================================================
+--------------------------------------+ +--------------------------------------+
| | | |
| RNIC Engine | <-- Firmware | RNIC Engine | <-- Firmware
| | | |
+--------------------------------------+ +--------------------------------------+
^ ^
| |
v v
Internet Internet
Figure 1 - RDMA Security Model Figure 1 - RDMA Security Model
4.1 Components 4.1 Components
The components shown in Figure 1 - RDMA Security Model are: The components shown in Figure 1 - RDMA Security Model are:
* RNIC Engine - the component that implements the RDMA * RNIC Engine (RNIC) - the component that implements the
protocol and/or DDP protocol. RDMA protocol and/or DDP protocol.
* Privileged Resource Manager - the component responsible * Privileged Resource Manager - the component responsible
for managing and allocating resources associated with the for managing and allocating resources associated with the
RNIC Engine. The Resource Manager does not send or RNIC Engine. The Resource Manager does not send or
receive data. Note that whether the Resource Manager is receive data. Note that whether the Resource Manager is
an independent component, part of the RNIC, or part of an independent component, part of the RNIC, or part of
the application is implementation dependent. If a the application is implementation dependent. If a
specific implementation does not wish to address security specific implementation does not wish to address security
issues resolved by the Resource Manager, there may in issues resolved by the Resource Manager, there may in
fact be no resource manager at all. fact be no resource manager at all.
skipping to change at page 10, line 26 skipping to change at page 10, line 26
constrained conditions, Non-Privileged applications to send and constrained conditions, Non-Privileged applications to send and
receive data directly to/from the RDMA Engine without Privileged receive data directly to/from the RDMA Engine without Privileged
Resource Manager intervention - while ensuring that the host Resource Manager intervention - while ensuring that the host
remains secure. Thus, one of the primary goals of this paper is remains secure. Thus, one of the primary goals of this paper is
to analyze this usage model for the enforcement that is required to analyze this usage model for the enforcement that is required
in the RNIC Engine to ensure the system remains secure. in the RNIC Engine to ensure the system remains secure.
The host interfaces that could be exercised include: The host interfaces that could be exercised include:
* Privileged Control Interface - A Privileged Resource * Privileged Control Interface - A Privileged Resource
Manager uses the RI to allocate and manage RNIC Engine Manager uses the RNIC Interface to allocate and manage
resources, control the state within the RNIC Engine, and RNIC Engine resources, control the state within the RNIC
monitor various events from the RNIC Engine. It also uses Engine, and monitor various events from the RNIC Engine.
this interface to act as a proxy for some operations that It also uses this interface to act as a proxy for some
a Non-Privileged Application may require (after operations that a Non-Privileged Application may require
performing appropriate countermeasures). (after performing appropriate countermeasures).
* Application Control Interface ū An application uses this * Application Control Interface - An application uses this
interface to the Privileged Resource Manager to allocate interface to the Privileged Resource Manager to allocate
RNIC Engine resources. The Privileged Resource Manager RNIC Engine resources. The Privileged Resource Manager
implements countermeasures to ensure that if the Non- implements countermeasures to ensure that if the Non-
Privileged Application launches an attack it can prevent Privileged Application launches an attack it can prevent
the attack from affecting other applications. the attack from affecting other applications.
* Non-Privileged Data Transfer Interface - A Non-Privileged * Non-Privileged Data Transfer Interface - A Non-Privileged
Application uses this interface to initiate and to check Application uses this interface to initiate and to check
the status of data transfer operations. the status of data transfer operations.
skipping to change at page 11, line 27 skipping to change at page 11, line 27
4.2.1 Stream Context Memory 4.2.1 Stream Context Memory
The state information for each Stream is maintained in memory, The state information for each Stream is maintained in memory,
which could be located in a number of places - on the NIC, inside which could be located in a number of places - on the NIC, inside
RAM attached to the NIC, in host memory, or in any combination of RAM attached to the NIC, in host memory, or in any combination of
the three, depending on the implementation. the three, depending on the implementation.
Stream Context Memory includes state associated with Data Stream Context Memory includes state associated with Data
Buffers. For Tagged Buffers, this includes how STag names, Data Buffers. For Tagged Buffers, this includes how STag names, Data
Buffers, and Page Translation Tables inter-relate. It also Buffers, and Page Translation Tables inter-relate. It also
includes the FIFO list of Untagged Data Buffers posted for includes the list of Untagged Data Buffers posted for reception
reception of Untagged Messages (referred to in some contexts as of Untagged Messages (commonly called the Receive Queue), and a
the Receive Queue), and a list of operations to perform to send list of operations to perform to send data (commonly called the
data (referred to in some contexts as the Send Queue). Send Queue).
4.2.2 Data Buffers 4.2.2 Data Buffers
There are two different ways to expose a data buffer; a buffer There are two different ways to expose a data buffer; a buffer
can be exposed for receiving RDMAP Send Type Messages (a.k.a. DDP can be exposed for receiving RDMAP Send Type Messages (a.k.a. DDP
Untagged Messages) on DDP Queue zero or the buffer can be exposed Untagged Messages) on DDP Queue zero or the buffer can be exposed
for remote access through STags (a.k.a. DDP Tagged Messages). for remote access through STags (a.k.a. DDP Tagged Messages).
This distinction is important because the attacks and the This distinction is important because the attacks and the
countermeasures used to protect against the attack are different countermeasures used to protect against the attack are different
depending on the method for exposing the buffer to the Internet. depending on the method for exposing the buffer to the network.
For the purposes of the security discussion, a single logical For the purposes of the security discussion, a single logical
Data Buffer is exposed with a single STag. Actual implementations Data Buffer is exposed with a single STag. Actual implementations
may support scatter/gather capabilities to enable multiple may support scatter/gather capabilities to enable multiple
physical data buffers to be accessed with a single STag, but from physical data buffers to be accessed with a single STag, but from
a threat analysis perspective it is assumed that a single STag a threat analysis perspective it is assumed that a single STag
enables access to a single logical Data Buffer. enables access to a single logical Data Buffer.
In any event, it is the responsibility of the RI to ensure that In any event, it is the responsibility of the RNIC to ensure that
no STag can be created that exposes memory that the consumer had no STag can be created that exposes memory that the consumer had
no authority to expose. no authority to expose.
4.2.3 Page Translation Tables 4.2.3 Page Translation Tables
Page Translation Tables are the structures used by the RNIC to be Page Translation Tables are the structures used by the RNIC to be
able to access application memory for data transfer operations. able to access application memory for data transfer operations.
Even though these structures are called "Page" Translation Even though these structures are called "Page" Translation
Tables, they may not reference a page at all - conceptually they Tables, they may not reference a page at all - conceptually they
are used to map an application address space representation of a are used to map an application address space representation of a
buffer to the physical addresses that are used by the RNIC Engine buffer to the physical addresses that are used by the RNIC Engine
to move data. If on a specific system, a mapping is not used, to move data. If on a specific system a mapping is not used, then
then a subset of the attacks examined may be appropriate. a subset of the attacks examined may be appropriate. Note that
the Page Translation Table may or may not be a shared resource.
4.2.4 STag Namespace 4.2.4 STag Namespace
The DDP specification defines a 32-bit namespace for the STag. The DDP specification defines a 32-bit namespace for the STag.
Implementations may vary in terms of the actual number of STags Implementations may vary in terms of the actual number of STags
that are supported. In any case, this is a bounded resource that that are supported. In any case, this is a bounded resource that
can come under attack. Depending upon STag namespace allocation can come under attack. Depending upon STag namespace allocation
algorithms, the actual name space to attack may be significantly algorithms, the actual name space to attack may be significantly
less than 2^32. less than 2^32.
skipping to change at page 15, line 7 skipping to change at page 15, line 9
Table is outside the scope of this specification. It may be Table is outside the scope of this specification. It may be
allocated for a specific Data Buffer, or be allocated as a pooled allocated for a specific Data Buffer, or be allocated as a pooled
resource to be consumed by potentially multiple Data Buffers, or resource to be consumed by potentially multiple Data Buffers, or
be managed in some other way. This paper attempts to abstract be managed in some other way. This paper attempts to abstract
implementation dependent issues, and focus on higher level implementation dependent issues, and focus on higher level
security issues such as resource starvation and sharing of security issues such as resource starvation and sharing of
resources between Streams. resources between Streams.
The next issue is how an STag name is associated with a Data The next issue is how an STag name is associated with a Data
Buffer. For the case of an Untagged Data Buffer, there is no wire Buffer. For the case of an Untagged Data Buffer, there is no wire
visible mapping between an STag name and a Data Buffer. Note that visible mapping between an STag and the Data Buffer. Note that
there may, in fact, be a mapping that is not visible from the there may, in fact, be an STag which represents the buffer.
wire, but this is a local host specific issue which should be However, because the STag by definition is not visible on the
wire, this is a local host specific issue which should be
analyzed in the context of local host implementation specific analyzed in the context of local host implementation specific
security analysis, and thus is outside the scope of this paper. security analysis, and thus is outside the scope of this paper.
For a Tagged Data Buffer, either the Privileged Application, the For a Tagged Data Buffer, either the Privileged Application, the
Non-Privileged Application, or the Privileged Resource Manager Non-Privileged Application, or the Privileged Resource Manager
acting on behalf of the Non-Privileged Resource Manager may acting on behalf of the Non-Privileged Resource Manager may
initialize a mapping from an STag to a Page Translation Table, or initialize a mapping from an STag to a Page Translation Table, or
may have the ability to simply enable/disable an existing STag to may have the ability to simply enable/disable an existing STag to
Page Translation Table mapping. There may also be multiple STag Page Translation Table mapping. There may also be multiple STag
names which map to a specific group of Page Translation Table names which map to a specific group of Page Translation Table
entries (or sub-entries). Specific security issues with this entries (or sub-entries). Specific security issues with this
level of flexibility are examined later. level of flexibility are examined in Section 7.3.3 Multiple STags
to access the same buffer on page 25.
There are a variety of implementation options for initialization There are a variety of implementation options for initialization
of Page Translation Table entries and mapping an STag to a group of Page Translation Table entries and mapping an STag to a group
of Page Translation Table entries which have security of Page Translation Table entries which have security
repercussions. This includes support for separation of Mapping an repercussions. This includes support for separation of Mapping an
STag verses mapping a set of Page Translation Table entries, and STag verses mapping a set of Page Translation Table entries, and
support for Applications directly manipulating STag to Page support for Applications directly manipulating STag to Page
Translation Table entry mappings (verses requiring access through Translation Table entry mappings (verses requiring access through
the Privileged Resource Manager). the Privileged Resource Manager).
4.2.10 RNIC Data Transfer Interactions 4.2.10 RNIC Data Transfer Interactions
RNIC Data Transfer operations can be subdivided into send RNIC Data Transfer operations can be subdivided into send
operations and receive operations. operations and receive operations.
For send operations, there is typically a queue that enables the For send operations, there is typically a queue that enables the
Application to post multiple operations. Depending upon the Application to post multiple operations to send data (referred to
implementation, Data Buffers used in the operations may or may as the Send Queue). Depending upon the implementation, Data
not have Page Translation Table entries associated with them, and Buffers used in the operations may or may not have Page
may or may not have STags associated with them. Because this is a Translation Table entries associated with them, and may or may
local host specific implementation issue rather than a protocol not have STags associated with them. Because this is a local host
issue, the security analysis of threats and mitigations is left specific implementation issue rather than a protocol issue, the
to the host implementation. security analysis of threats and mitigations is left to the host
implementation.
Receive operations are different for Tagged Data Buffers verses Receive operations are different for Tagged Data Buffers verses
Untagged Data Buffers. If more than one Untagged Data Buffer can Untagged Data Buffers. If more than one Untagged Data Buffer can
be posted by the Application, the DDP specification requires that be posted by the Application, the DDP specification requires that
they be consumed in FIFO order. Thus the most general they be consumed in sequential order. Thus the most general
implementation is that there is a FIFO queue of receive Untagged implementation is that there is a sequential queue of receive
Data Buffers. Some implementations may also support sharing of Untagged Data Buffers (Receive Queue). Some implementations may
the FIFO queue between multiple Streams. In this case defining also support sharing of the sequential queue between multiple
"FIFO" becomes non-trivial - in general the buffers for a single Streams. In this case defining "sequential" becomes non-trivial -
stream are consumed from the queue in the order that they were in general the buffers for a single stream are consumed from the
placed on the queue, but there is no order guarantee between queue in the order that they were placed on the queue, but there
streams. is no order guarantee between streams.
For receive Tagged Data Buffers, at some time prior to data For receive Tagged Data Buffers, at some time prior to data
transfer, the mapping of the STag to specific Page Translation transfer, the mapping of the STag to specific Page Translation
Table entries (if present) and the mapping from the Page Table entries (if present) and the mapping from the Page
Translation Table entries to the Data Buffer must have been Translation Table entries to the Data Buffer must have been
initialized (see the prior section for interaction details). initialized (see the prior section for interaction details).
5 Trust and Resource Sharing 5 Trust and Resource Sharing
It is assumed that in general the Local and Remote Peer are It is assumed that in general the Local and Remote Peer are
skipping to change at page 17, line 20 skipping to change at page 17, line 20
A separate, but related issue is resource sharing between A separate, but related issue is resource sharing between
multiple streams. If local resources are not shared, the multiple streams. If local resources are not shared, the
resources are dedicated on a per Stream basis. Resources are resources are dedicated on a per Stream basis. Resources are
defined in Section 4.2 - Resources on page 10. The advantage of defined in Section 4.2 - Resources on page 10. The advantage of
not sharing resources between Streams is that it reduces the not sharing resources between Streams is that it reduces the
types of attacks that are possible. The disadvantage is that types of attacks that are possible. The disadvantage is that
applications might run out of resources. applications might run out of resources.
It is assumed in this paper that the component that implements It is assumed in this paper that the component that implements
the mechanism to control sharing of RNIC Engine resources is the the mechanism to control sharing of the RNIC Engine resources is
Privileged Resource Manager. The RNIC Engine exposes its the Privileged Resource Manager. The RNIC Engine exposes its
resources through the RI to the Privileged Resource Manager. All resources through the RNIC Interface to the Privileged Resource
Privileged and Non-Privileged applications request resources from Manager. All Privileged and Non-Privileged applications request
the Resource Manager. The Resource Manager implements resource resources from the Resource Manager. The Resource Manager
management policies to ensure fair access to resources. The implements resource management policies to ensure fair access to
Resource Manager should be designed to take into account security resources. The Resource Manager should be designed to take into
attacks detailed in this specification. Note that for some account security attacks detailed in this specification. Note
systems the Privileged Resource Manager may be implemented within that for some systems the Privileged Resource Manager may be
the Privileged Application. implemented within the Privileged Application.
The sharing of resources across Streams should be under the The sharing of resources across Streams should be under the
control of the application, both in terms of the trust model the control of the application, both in terms of the trust model the
application wishes to operate under, as well as the level of application wishes to operate under, as well as the level of
resource sharing the application wishes to give Local Peer resource sharing the application wishes to give Local Peer
processes. For more discussion on types of trust models which processes. For more discussion on types of trust models which
combine partial trust and sharing of resources, see Appendix C: combine partial trust and sharing of resources, see Appendix C:
Partial Trust Taxonomy on page 53. Partial Trust Taxonomy on page 48.
6 Attacker Capabilities 6 Attacker Capabilities
An attackerĘs capabilities delimit the types of attacks that An attackerĘs capabilities delimit the types of attacks that
attacker is able to launch. RDMAP and DDP require that the attacker is able to launch. RDMAP and DDP require that the
initial LLP Stream (and connection) be set up prior to initial LLP Stream (and connection) be set up prior to
transferring RDMAP/DDP Messages. For the attacker to actively transferring RDMAP/DDP Messages. Attackers with send only
generate an RDMAP/DDP protocol attack, it must have the capabilities must first guess the current LLP Stream parameters
capability to both send and receive messages. Attackers with send before they can attack RNIC resources (e.g. TCP sequence number).
only capabilities should be addressed by the LLP, not by Attackers with both send and receive capabilities have presumably
RDMAP/DDP. setup a valid LLP Stream, and thus have a wider ability to attack
RNIC resources.
7 Attacks and Countermeasures 7 Attacks and Countermeasures
This section describes the attacks that are possible against the This section describes the attacks that are possible against the
RDMA system defined in Figure 1 - RDMA Security Model and the RDMA system defined in Figure 1 - RDMA Security Model and the
RNIC Engine resources defined in Section 4.2. The analysis RNIC Engine resources defined in Section 4.2. The analysis
includes a detailed description of each attack, what is being includes a detailed description of each attack, what is being
attacked, and a description of the countermeasures that can be attacked, and a description of the countermeasures that can be
taken to thwart the attack. taken to thwart the attack.
Note that connection setup and teardown is presumed to be done in Note that connection setup and teardown is presumed to be done in
stream mode (i.e. no RDMA encapsulation of the payload), so there stream mode (i.e. no RDMA encapsulation of the payload), so there
are no new attacks related to connection setup/teardown beyond are no new attacks related to connection setup/teardown beyond
what is already present in the LLP (e.g. TCP or SCTP). what is already present in the LLP (e.g. TCP or SCTP). Note,
Consequently, any existing analysis of Spoofing, Tampering, however, that RDMAP/DDP parameters may be exchanged in stream
Repudiation, Information Disclosure, Denial of Service, or mode, and if they are corrupted by an attacker unintended
Elevation of Privilege continues to apply. Thus, the analysis in consequences will result. Therefore, any existing mitigations for
this section focuses on attacks that are present regardless of LLP Spoofing, Tampering, Repudiation, Information Disclosure,
the LLP Stream type. Denial of Service, or Elevation of Privilege continues to apply
(and is out of scope of this document). Thus the analysis in this
section focuses on attacks that are present regardless of the LLP
Stream type.
The attacks are classified into five categories: Spoofing, The attacks are classified into five categories: Spoofing,
Tampering, Information Disclosure, Denail of Service (DoS) Tampering, Information Disclosure, Denail of Service (DoS)
attacks, and Elevation of Privileges. Tampering is any attacks, and Elevation of Privileges. Tampering is any
modification of the legitimate traffic (machine internal or modification of the legitimate traffic (machine internal or
network). Spoofing attack is a special case of tempering; where network). Spoofing attack is a special case of tempering; where
the attacker falsifies an identity of the Remote Peer (identity the attacker falsifies an identity of the Remote Peer (identity
can be an IP address, machine name, ULP level identity etc.). can be an IP address, machine name, ULP level identity etc.).
7.1 Tools for Countermeasures 7.1 Tools for Countermeasures
skipping to change at page 20, line 44 skipping to change at page 20, line 49
* Limit the time an STag is valid. By Invalidating an * Limit the time an STag is valid. By Invalidating an
Advertised STag (e.g., revoking remote access to the Advertised STag (e.g., revoking remote access to the
buffers described by an STag when done with the buffers described by an STag when done with the
transfer), an entire class of attacks can be eliminated. transfer), an entire class of attacks can be eliminated.
* Limit the buffer the STag can reference. Limiting the * Limit the buffer the STag can reference. Limiting the
scope of an STag access to *just* the intended scope of an STag access to *just* the intended
application buffers to be exposed is critical to prevent application buffers to be exposed is critical to prevent
certain forms of attacks. certain forms of attacks.
* Allocating STag numbers in an unpredictable way. If STags * Allocating and/or advertising STag numbers in an
are allocated using an algorithm which makes it hard for unpredictable way. If STags are allocated/advertised
the Remote Peer to guess which STag(s) are currently in using an algorithm which makes it hard for the attacker
use, it makes it more difficult for an attacker to guess to guess which STag(s) are currently in use, it makes it
the correct value. As stated in the RDMAP specification more difficult for an attacker to guess the correct
[RDMAP], an invalid STag will cause the RDMAP Stream to value. As stated in the RDMAP specification [RDMAP], an
be terminated. For the case of [DDP], at a minimum it invalid STag will cause the RDMAP Stream to be
must signal an error to the ULP, and commonly this will terminated. For the case of [DDP], at a minimum it must
cause the DDP stream to be terminated. signal an error to the ULP, and commonly this will cause
the DDP stream to be terminated.
7.1.3 Access Rights 7.1.3 Access Rights
Access Rights associated with a specific Advertised STag or Access Rights associated with a specific Advertised STag or
RDMAP/DDP Stream provide another mechanism for applications to RDMAP/DDP Stream provide another mechanism for applications to
limit the attack capabilities of the Remote Peer. The Local Peer limit the attack capabilities of the Remote Peer. The Local Peer
can control whether a data buffer is exposed for local only, or can control whether a data buffer is exposed for local only, or
local and remote access, and assign specific access privileges local and remote access, and assign specific access privileges
(read, write, read and write) on a per stream or session basis. (read, write, read and write) on a per stream basis.
For DDP, when an STag is advertised, the Remote Peer is For DDP, when an STag is advertised, the Remote Peer is
presumably given write access rights to the data (otherwise there presumably given write access rights to the data (otherwise there
was not much point to the advertisement). For RDMAP, when an was not much point to the advertisement). For RDMAP, when an
application advertises an STag, it can enable write-only, read- application advertises an STag, it can enable write-only, read-
only, or both write and read access rights. only, or both write and read access rights.
Similarly, some applications may wish to provide a single buffer Similarly, some applications may wish to provide a single buffer
with different access rights on a per-Stream or per-Stream basis. with different access rights on a per-Stream or per-Stream basis.
For example, some Streams may have read-only access, some may For example, some Streams may have read-only access, some may
skipping to change at page 21, line 54 skipping to change at page 22, line 5
For example, an error on a specific RDMAP stream should not cause For example, an error on a specific RDMAP stream should not cause
the RNIC to stop processing incoming packets, or corrupt a the RNIC to stop processing incoming packets, or corrupt a
receive queue for an unrelated stream. receive queue for an unrelated stream.
7.2 Spoofing 7.2 Spoofing
Spoofing attacks can be launched by the Remote Peer, or by a Spoofing attacks can be launched by the Remote Peer, or by a
network based attacker. A network based spoofing attack applies network based attacker. A network based spoofing attack applies
to all Remote Peers. to all Remote Peers.
Because the RDMAP Stream is only offloaded if it is in the Because the RDMAP Stream requires an LLP Stream in the
ESTABLISHED state, certain types of traditional forms of wire ESTABLISHED state, certain types of traditional forms of wire
attacks do not apply -- an end-to-end handshake must have attacks do not apply -- an end-to-end handshake must have
occurred to establish the RDMAP Stream. So, the only form of occurred to establish the RDMAP Stream. So, the only form of
spoofing that applies is one when a remote node can both send and spoofing that applies is one when a remote node can both send and
receive packets. Yet even with this limitation the Stream is receive packets. Yet even with this limitation the Stream is
still exposed to the following spoofing attacks. still exposed to the following spoofing attacks.
7.2.1 Impersonation 7.2.1 Impersonation
A network based attacker can impersonate a legal RDMA/RDDP peer A network based attacker can impersonate a legal RDMA/DDP peer
(by spoofing a legal IP address), and establish an RDMA/RDDP (by spoofing a legal IP address), and establish an RDMA/DDP
Stream with the victim. End to end authentication (i.e. IPsec, Stream with the victim. End to end authentication (i.e. IPsec,
SSL or ULP authentication) provides protection against this SSL or ULP authentication) provides protection against this
attack. For additional information see Section 8, Security attack. For additional information see Section 8, Security
Services for RDDP, on page 37. Services for RDMA and DDP, on page 38.
7.2.2 Stream Hijacking 7.2.2 Stream Hijacking
Stream Hijacking happens when a network based attacker follows Stream hijacking happens when a network based attacker follows
the session establishment phase, and waits until the the Stream establishment phase, and waits until the
authentication phase (if such a phase exists) is completed authentication phase (if such a phase exists) is completed
successfully. He can then spoof the IP address and re-direct the successfully. He can then spoof the IP address and re-direct the
Stream from the victim to its own machine. For example, an Stream from the victim to its own machine. For example, an
attacker can wait until an iSCSI authentication is completed attacker can wait until an iSCSI authentication is completed
successfully, and hijack the iSCSI Stream. successfully, and hijack the iSCSI Stream.
The best protection against this form of attack is end-to-end The best protection against this form of attack is end-to-end
session level integrity protection and authentication, such as integrity protection and authentication, such as IPsec (see
IPsec (see Section 8, Security Services for RDDP, on page 37), to Section 8, Security Services for RDMA and DDP, on page 38), to
prevent spoofing. Another option is to provide physical security. prevent spoofing. Another option is to provide physical security.
Discussion of physical security is out of scope for this Discussion of physical security is out of scope for this
document. document.
Because the connection and/or Stream itself is established by the Because the connection and/or Stream itself is established by the
LLP, some LLPs are more difficult to hijack than others. Please LLP, some LLPs are more difficult to hijack than others. Please
see the relevant LLP documentation on security issues around see the relevant LLP documentation on security issues around
connection and/or Stream hijacking <TBD: references for SCTP and connection and/or Stream hijacking.
TCP on connection hijacking>.
7.2.3 Man in the Middle Attack 7.2.3 Man in the Middle Attack
If a network based attacker has the ability to delete, inject If a network based attacker has the ability to delete, inject
replay, or modify packets which will still be accepted by the LLP replay, or modify packets which will still be accepted by the LLP
(e.g., TCP sequence number is correct) then the Stream can be (e.g., TCP sequence number is correct) then the Stream can be
exposed to a man in the middle attack. One style of attack is for exposed to a man in the middle attack. One style of attack is for
the man-in-the-middle to send Tagged Messages (either RDMAP or the man-in-the-middle to send Tagged Messages (either RDMAP or
DDP). If it can discover a buffer that has been exposed for STag DDP). If it can discover a buffer that has been exposed for STag
enabled access, then the man-in-the-middle can use an RDMA Read enabled access, then the man-in-the-middle can use an RDMA Read
operation to read the contents of the associated data buffer, operation to read the contents of the associated data buffer,
perform an RDMA Write Operation to modify the contents of the perform an RDMA Write Operation to modify the contents of the
associated data buffer, or invalidate the STag to disable further associated data buffer, or invalidate the STag to disable further
access to the buffer. The only countermeasure for this form of access to the buffer. The only countermeasure for this form of
attack is to either secure the RDMAP/DDP Stream (i.e. integrity attack is to either secure the RDMAP/DDP Stream (i.e. integrity
protect) or attempt to provide physical security to prevent man- protect) or attempt to provide physical security to prevent man-
in-the-middle type attacks. in-the-middle type attacks.
The best protection against this form of attack is end-to-end The best protection against this form of attack is end-to-end
integrity protection and authentication, such as IPsec (see integrity protection and authentication, such as IPsec (see
Section 8 Security Services for RDDP on page 37), to prevent Section 8 Security Services for RDMA and DDP on page 38), to
spoofing or tampering. If Stream or session level authentication prevent spoofing or tampering. If Stream or session level
and integrity protection are not used, then a man-in-the-middle authentication and integrity protection are not used, then a man-
attack can occur, enabling spoofing and tampering. in-the-middle attack can occur, enabling spoofing and tampering.
Because the connection/Stream itself is established by the LLP, Because the connection/Stream itself is established by the LLP,
some LLPs are more exposed to man-in-the-middle attack then some LLPs are more exposed to man-in-the-middle attack then
others. Please see the relevant LLP documentation on security others. Please see the relevant LLP documentation on security
issues around connection and/or Stream hijacking <TBD: references issues around connection and/or Stream hijacking.
for SCTP and TCP on connection hijacking>.
Another approach is to restrict access to only the local Another approach is to restrict access to only the local
subnet/link, and provide some mechanism to limit access, such as subnet/link, and provide some mechanism to limit access, such as
physical security or 802.1.x. This model is an extremely limited physical security or 802.1.x. This model is an extremely limited
deployment scenario, and will not be further examined here. deployment scenario, and will not be further examined here.
7.2.4 Using an STag on a Different Stream 7.2.4 Using an STag on a Different Stream
One style of attack from the Remote Peer is for it to attempt to One style of attack from the Remote Peer is for it to attempt to
use STag values that it is not authorized to use. Note that if use STag values that it is not authorized to use. Note that if
skipping to change at page 23, line 42 skipping to change at page 23, line 42
one Stream and a Remote Peer is able to use it on an unrelated one Stream and a Remote Peer is able to use it on an unrelated
Stream. If the attack is successful, the attacker could Stream. If the attack is successful, the attacker could
potentially be able to perform either RDMA Read Operations to potentially be able to perform either RDMA Read Operations to
read the contents of the associated data buffer, perform RDMA read the contents of the associated data buffer, perform RDMA
Write Operations to modify the contents of the associated data Write Operations to modify the contents of the associated data
buffer, or to Invalidate the STag to disable further access to buffer, or to Invalidate the STag to disable further access to
the buffer. the buffer.
An attempt by a Remote Peer to access a buffer with an STag on a An attempt by a Remote Peer to access a buffer with an STag on a
different Stream in the same Protection Domain may or may not be different Stream in the same Protection Domain may or may not be
an attack depending on whether resource sharing is intended an attack depending on whether resource sharing is intended (i.e.
(i.e. whether the Streams shared Partial Mutual Trust or not). whether the Streams shared Partial Mutual Trust or not). For some
For some applications using an STag on multiple Streams within applications using an STag on multiple Streams within the same
the same Protection Domain could be desired behavior. For other Protection Domain could be desired behavior. For other
applications attempting to use an STag on a different Stream applications attempting to use an STag on a different Stream
could be considered to be an attack. Since this varies by could be considered to be an attack. Since this varies by
application, an application typically would need to be able to application, an application typically would need to be able to
control the scope of the STag. control the scope of the STag.
In the case where an implementation does not share resources In the case where an implementation does not share resources
between Streams (including STags), this attack can be defeated by between Streams (including STags), this attack can be defeated by
assigning each Stream to a different Protection Domain. Before assigning each Stream to a different Protection Domain. Before
allowing remote access to the buffer, the Protection Domain of allowing remote access to the buffer, the Protection Domain of
the Stream where the access attempt was made is matched against the Stream where the access attempt was made is matched against
skipping to change at page 24, line 18 skipping to change at page 24, line 18
For implementations that share resources between multiple For implementations that share resources between multiple
Streams, it may not be practical to separate each Stream into its Streams, it may not be practical to separate each Stream into its
own Protection Domain. In this case, the application can still own Protection Domain. In this case, the application can still
limit the scope of any of the STags to a single Stream (if it is limit the scope of any of the STags to a single Stream (if it is
enabling it for remote access). If the STag scope has been enabling it for remote access). If the STag scope has been
limited to a single Stream, any attempt to use that STag on a limited to a single Stream, any attempt to use that STag on a
different Stream will result in an error, and the RDMA Stream different Stream will result in an error, and the RDMA Stream
should be terminated. should be terminated.
Thus for implementations that do not share STags between Streams Thus for implementations that do not share STags between Streams,
it is RECOMMENDED that either each Stream be in a separate each Stream MUST either be in a separate Protection Domain or the
Protection Domain or the scope of an STag be limited to a single scope of an STag be limited to a single Stream.
Stream.
An additional issue may be unintended sharing of STags (i.e. a An additional issue may be unintended sharing of STags (i.e. a
bug in the application) or a bug in the Remote Peer which causes bug in the application) or a bug in the Remote Peer which causes
an off-by-one STag to be used. For additional protection, it is an off-by-one STag to be used. For additional protection, an
RECOMMENDED that the allocation of STags be done in such a implementation SHOULD allocate STags in such a fashion that it is
fashion that it is difficult to predict the next allocated STag difficult to predict the next allocated STag number. Allocation
number. Allocation methods which deterministically allocate the methods which deterministically allocate the next STag should be
next STag should be avoided (e.g. a method which always starts avoided (e.g. a method which always starts with STag equal to one
with STag equal to one and monotonically increases it for each and monotonically increases it for each new allocation, or a
new allocation). method which always uses the same STag for each operation).
7.3 Tampering 7.3 Tampering
A Remote Peer or a network based attacker can attempt to tamper A Remote Peer or a network based attacker can attempt to tamper
with the contents of data buffers on a Local Peer that have been with the contents of data buffers on a Local Peer that have been
enabled for remote write access. The types of tampering attacks enabled for remote write access. The types of tampering attacks
that are possible are outlined in the sections that follow. that are possible are outlined in the sections that follow.
7.3.1 Buffer Overrun - RDMA Write or Read Response 7.3.1 Buffer Overrun - RDMA Write or Read Response
This attack is an attempt by the Remote Peer to perform an RDMA This attack is an attempt by the Remote Peer to perform an RDMA
Write or RDMA Read Response to memory outside of the valid length Write or RDMA Read Response to memory outside of the valid length
range of the data buffer enabled for remote write access. This range of the data buffer enabled for remote write access. This
attack can occur even when no resources are shared across attack can occur even when no resources are shared across
Streams. This issue can also arise if the application has a bug. Streams. This issue can also arise if the application has a bug.
The countermeasure for this type of attack must be in the RNIC The countermeasure for this type of attack must be in the RNIC
implementation, using the STag. When the Local Peer specifies to implementation, using the STag. When the Local Peer specifies to
the RI the base address and the number of bytes in the buffer the RNIC the base address and the number of bytes in the buffer
that it wishes to make accessible, the RI must ensure that the that it wishes to make accessible, the RNIC must ensure that the
base and bounds check are applied to any access to the buffer base and bounds check are applied to any access to the buffer
referenced by the STag before the STag is enabled for access. referenced by the STag before the STag is enabled for access.
When an RDMA data transfer operation (which includes an STag) When an RDMA data transfer operation (which includes an STag)
arrives on a Stream, a base and bounds byte granularity access arrives on a Stream, a base and bounds byte granularity access
check must be performed to ensure the operation accesses only check must be performed to ensure the operation accesses only
memory locations within the buffer described by that STag. memory locations within the buffer described by that STag.
Thus, it is RECOMMENDED that an RI implementation ensure that a Thus an RNIC implementation MUST ensure that a Remote Peer is not
Remote Peer will not be able to access memory outside of the able to access memory outside of the buffer specified when the
buffer specified when the STag was enabled for remote access. STag was enabled for remote access.
7.3.2 Modifying a Buffer After Indication 7.3.2 Modifying a Buffer After Indication
This attack occurs if a Remote Peer attempts to modify the This attack can occur if a Remote Peer attempts to modify the
contents by performing an RDMA Write or an RDMA Read Response contents of an STag referenced buffer by performing an RDMA Write
after it had indicated to the Local Peer that the data buffer or an RDMA Read Response after the Remote Peer has indicated to
contents were ready for use. the Local Peer that the STag data buffer contents are ready for
use. This attack can occur even when no resources are shared
across Streams. Note that a bug in a Remote Peer, or network
based tampering, could also result in this problem.
This attack can occur even when no resources are shared across For example, assume the STag referenced buffer contains ULP
Streams. Note that a bug in a Remote Peer, or network based control information as well as ULP payload, and the ULP sequence
tampering, could also result in this problem. of operation is to first validate the control information and
then perform operations on the control information. If the Remote
Peer can perform an additional RDMA Write or RDMA Read Response
(thus changing the buffer) after the validity checks have been
completed but before the control data is operated on, the Remote
Peer could force the ULP down operational paths that were never
intended.
The Local Peer can protect itself from this type of attack by The Local Peer can protect itself from this type of attack by
revoking remote access when the original data transfer has revoking remote access when the original data transfer has
completed and before it validates the contents of the buffer. The completed and before it validates the contents of the buffer. The
Local Peer can either do this by explicitly revoking remote Local Peer can either do this by explicitly revoking remote
access rights for the STag when the Remote Peer indicates the access rights for the STag when the Remote Peer indicates the
operation has completed, or by checking to make sure the Remote operation has completed, or by checking to make sure the Remote
Peer Invalidated the STag through the RDMAP Invalidate Peer Invalidated the STag through the RDMAP Invalidate
capability, and if it did not, the Local Peer then explicitly capability, and if it did not, the Local Peer then explicitly
revokes the STag remote access rights. revokes the STag remote access rights.
It is RECOMMENDED that the Local Peer follow the above procedure The Local Peer SHOULD follow the above procedure to protect the
to protect the buffer before it validates the contents of the buffer before it validates the contents of the buffer (or uses
buffer (or uses the buffer in any way). the buffer in any way).
7.3.3 Multiple STags to access the same buffer 7.3.3 Multiple STags to access the same buffer
See section 7.4.6 on page 27 for this analysis. See section 7.4.6 on page 27 for this analysis.
7.3.4 Network based modification of buffer content 7.3.4 Network based modification of buffer content
This is actually a man in the middle attack ū but only on the This is actually a man in the middle attack - but only on the
content of the buffer, as opposed to the man in the middle attack content of the buffer, as opposed to the man in the middle attack
presented above, where both the signaling and content can be presented above, where both the signaling and content can be
modified. See Section 7.2.3 Man in the Middle Attack on page 22. modified. See Section 7.2.3 Man in the Middle Attack on page 22.
7.4 Information Disclosure 7.4 Information Disclosure
The main potential source for information disclosure is through a The main potential source for information disclosure is through a
local buffer that has been enabled for remote access. If the local buffer that has been enabled for remote access. If the
buffer can be probed by a Remote Peer on another Stream, then buffer can be probed by a Remote Peer on another Stream, then
there is potential for information disclosure. there is potential for information disclosure.
skipping to change at page 26, line 26 skipping to change at page 26, line 33
If a buffer is being used for a combination of reads and writes If a buffer is being used for a combination of reads and writes
(either remote or local), and is exposed to the Remote Peer with (either remote or local), and is exposed to the Remote Peer with
at least remote read access rights, the Remote Peer may be able at least remote read access rights, the Remote Peer may be able
to examine the contents of the buffer before they are initialized to examine the contents of the buffer before they are initialized
with the correct data. In this situation, whatever contents were with the correct data. In this situation, whatever contents were
present in the buffer before the buffer is initialized can be present in the buffer before the buffer is initialized can be
viewed by the Remote Peer, if the Remote Peer performs an RDMA viewed by the Remote Peer, if the Remote Peer performs an RDMA
Read. Read.
Because of this, it is RECOMMENDED that the Local Peer ensure Because of this, the Local Peer SHOULD ensure that no stale data
that no stale data is contained in the buffer before remote read is contained in the buffer before remote read access rights are
access rights are granted (this can be done by zeroing the granted (this can be done by zeroing the contents of the memory,
contents of the memory, for example). for example).
7.4.3 Accessing a Buffer After the Transfer 7.4.3 Accessing a Buffer After the Transfer
If the Remote Peer has remote read access to a buffer, and by If the Remote Peer has remote read access to a buffer, and by
some mechanism tells the Local Peer that the transfer has been some mechanism tells the Local Peer that the transfer has been
completed, but the Local Peer does not disable remote access to completed, but the Local Peer does not disable remote access to
the buffer before modifying the data, it is possible for the the buffer before modifying the data, it is possible for the
Remote Peer to retrieve the new data. Remote Peer to retrieve the new data.
This is similar to the attack defined in Section 7.3.2 Modifying This is similar to the attack defined in Section 7.3.2 Modifying
a Buffer After Indicati on page 25. The same countermeasures a Buffer After Indicati on page 25. The same countermeasures
apply. In addition, it is RECOMMENDED that the Local Peer should apply. In addition, the Local Peer SHOULD grant remote read
grant remote read access rights only for the amount of time access rights only for the amount of time needed to retrieve the
needed to retrieve the data. data.
7.4.4 Accessing Unintended Data With a Valid STag 7.4.4 Accessing Unintended Data With a Valid STag
If the Local Peer enables remote access to a buffer using an STag If the Local Peer enables remote access to a buffer using an STag
that references the entire buffer, but intends only a portion of that references the entire buffer, but intends only a portion of
the buffer to be accessed, it is possible for the Remote Peer to the buffer to be accessed, it is possible for the Remote Peer to
access the other parts of the buffer anyway. access the other parts of the buffer anyway.
To prevent this attack, it is RECOMMENDED that the Local Peer set To prevent this attack, the Local Peer MUST set the base and
the base and bounds of the buffer when the STag is initialized to bounds of the buffer when the STag is initialized to expose only
expose only the data to be retrieved. the data to be retrieved.
7.4.5 RDMA Read into an RDMA Write Buffer 7.4.5 RDMA Read into an RDMA Write Buffer
One form of disclosure can occur if the access rights on the One form of disclosure can occur if the access rights on the
buffer enabled remote read, when only remote write access was buffer enabled remote read, when only remote write access was
intended. If the buffer contained application data, or data from intended. If the buffer contained application data, or data from
a transfer on an unrelated Stream, the Remote Peer could retrieve a transfer on an unrelated Stream, the Remote Peer could retrieve
the data through an RDMA Read operation. the data through an RDMA Read operation.
The most obvious countermeasure for this attack is to not grant The most obvious countermeasure for this attack is to not grant
remote read access if the buffer is intended to be write-only. remote read access if the buffer is intended to be write-only.
Then the Remote Peer would not be able to retrieve data Then the Remote Peer would not be able to retrieve data
associated with the buffer. An attempt to do so would result in associated with the buffer. An attempt to do so would result in
an error and the RDMAP Stream associated with the Stream would be an error and the RDMAP Stream associated with the Stream would be
terminated. terminated.
Thus, it is RECOMMENDED that if an application only intends a Thus if an application only intends a buffer to be exposed for
buffer to be exposed for remote write access, it set the access remote write access, it MUST set the access rights to the buffer
rights to the buffer to only enable remote write access. to only enable remote write access.
7.4.6 Using Multiple STags to Access One Buffer 7.4.6 Using Multiple STags Which Alias to the Same Buffer
Multiple STags accessing the same buffer at the same time can Multiple STags which alias to the same buffer at the same time
result in unintentional information disclosure if the STags are can result in unintentional information disclosure if the STags
used by different, mutually untrusted, Remote Peers. This model are used by different, mutually untrusted, Remote Peers. This
applies specifically to client/server communication, where the model applies specifically to client/server communication, where
server is communicating with multiple clients, each of which do the server is communicating with multiple clients, each of which
not mutually trust each other. do not mutually trust each other.
If only read access is enabled, then the Local Peer has complete If only read access is enabled, then the Local Peer has complete
control over information disclosure. Thus a server which control over information disclosure. Thus a server which intended
intended to expose the same data (i.e. buffer) to multiple to expose the same data (i.e. buffer) to multiple clients by
clients by using multiple STags to the same buffer creates no new using multiple STags to the same buffer creates no new security
security issues beyond what has already been described in this issues beyond what has already been described in this document.
document. Note that if the server did not intend to expose the Note that if the server did not intend to expose the same data to
same data to the clients, it should use separate buffers for each the clients, it should use separate buffers for each client (and
client (and separate STags). separate STags).
When one STag has remote read access enabled and a different STag When one STag has remote read access enabled and a different STag
has remote write access enabled to the same buffer, it is has remote write access enabled to the same buffer, it is
possible for one Remote Peer to view the contents that have been possible for one Remote Peer to view the contents that have been
written by another Remote Peer. written by another Remote Peer.
If both STags have remote write access enabled and the two Remote If both STags have remote write access enabled and the two Remote
Peers do not mutually trust each other, it is possible for one Peers do not mutually trust each other, it is possible for one
Remote Peer to overwrite the contents that have been written by Remote Peer to overwrite the contents that have been written by
the other Remote Peer. the other Remote Peer.
Thus it is RECOMMENDED that multiple Remote Peers which do not Thus multiple Remote Peers which do not share Partial Mutual
share Partial Mutual Trust not be granted write access to the Trust MUST NOT be granted write access to the same buffer through
same buffer through different STags. A buffer should be exposed different STags. A buffer should be exposed to only one untrusted
to only one untrusted Remote Peer at a time to ensure that no Remote Peer at a time to ensure that no information disclosure or
information disclosure or information tampering occurs between information tampering occurs between peers.
peers.
7.4.7 Remote Node Loading Firmware onto the RNIC 7.4.7 Remote Node Loading Firmware onto the RNIC
If the Remote Peer can cause firmware to be loaded onto the RNIC, If the Remote Peer can cause firmware to be loaded onto the RNIC,
there is an opportunity for information disclosure. See Elevation there is an opportunity for information disclosure. See Elevation
of Privilege in Section 7.6 for this analysis. of Privilege in Section 7.6 for this analysis.
7.4.8 Controlling Access to PTT & STag Mapping 7.4.8 Controlling Access to PTT & STag Mapping
If a Non-Privileged application is able to directly manipulate If a Non-Privileged application is able to directly manipulate
skipping to change at page 28, line 29 skipping to change at page 28, line 35
unrelated applicationĘs buffers and thereby be able to gain unrelated applicationĘs buffers and thereby be able to gain
access to information in the unrelated application. access to information in the unrelated application.
As discussed in Section 4 Architectural Model on page 8, As discussed in Section 4 Architectural Model on page 8,
introduction of a Privileged Resource Manager to arbitrate the introduction of a Privileged Resource Manager to arbitrate the
mapping requests is an effective countermeasure. This enables the mapping requests is an effective countermeasure. This enables the
Privileged Resource Manager to ensure an application can only Privileged Resource Manager to ensure an application can only
initialize the Page Translation Table (PTT)to point to its own initialize the Page Translation Table (PTT)to point to its own
buffers. buffers.
Thus it is RECOMMENDED that the Privileged Resource Manager Thus if Non-Privileged applications are supported, the Privileged
verify that the Non-Privileged application has the right to Resource Manager MUST verify that the Non-Privileged application
access a specific Data Buffer before allowing an STag for which has the right to access a specific Data Buffer before allowing an
the application has access rights to be associated with a STag for which the application has access rights to be associated
specific Data Buffer. This can be done when the Page Translation with a specific Data Buffer. This can be done when the Page
Table is initialized to access the Data Buffer or when the STag Translation Table is initialized to access the Data Buffer or
is initialized to point to a group of Page Translation Table when the STag is initialized to point to a group of Page
entries, or both. Translation Table entries, or both.
7.4.9 Network based eaves dropping 7.4.9 Network based eaves dropping
An attacker, eaves dropping the network, can read the content of An attacker that is able to eavesdrop on the network can read the
all read and write access to the peerĘs buffers. To prevent content of all read and write access to the peerĘs buffers. To
information disclosure, the read/written data must be encrypted. prevent information disclosure, the read/written data must be
The encryption can be done either by the ULP, or by a protocol encrypted. See also Section 7.2.3 Man in the Middle Attack on
that provides security services to the LLP (e.g. IPsec or SSL). page 22. The encryption can be done either by the ULP, or by a
Refer to section 8 for discussion of security services for protocol that provides security services to the LLP (e.g. IPsec
RDDP/RDMA. or SSL). Refer to section 8 for discussion of security services
for DDP/RDMA.
7.5 Denial of Service (DOS) 7.5 Denial of Service (DOS)
A DOS attack is one of the primary security risks of RDMAP. This A DOS attack is one of the primary security risks of RDMAP. This
is because RNIC resources are valuable and scarce, and many is because RNIC resources are valuable and scarce, and many
application environments require communication with untrusted application environments require communication with untrusted
Remote Peers. If the remote application can be authenticated or Remote Peers. If the remote application can be authenticated or
encrypted, clearly, the DOS profile can be reduced. For the encrypted, clearly, the DOS profile can be reduced. For the
purposes of this analysis, it is assumed that the RNIC must be purposes of this analysis, it is assumed that the RNIC must be
able to operate in untrusted environments, which are open to DOS able to operate in untrusted environments, which are open to DOS
style attacks. style attacks.
Denial of service attacks against RI resources are not the Denial of service attacks against RNIC resources are not the
typical unknown party spraying packets at a random host (such as typical unknown party spraying packets at a random host (such as
a TCP SYN attack). Because the connection/Stream must be fully a TCP SYN attack). Because the connection/Stream must be fully
established, the attacker must be able to both send and receive established, the attacker must be able to both send and receive
messages over that connection/Stream, or be able to guess a valid messages over that connection/Stream, or be able to guess a valid
packet on an existing RDMAP Stream. packet on an existing RDMAP Stream.
This section outlines the potential attacks and the This section outlines the potential attacks and the
countermeasures available for dealing with each attack. countermeasures available for dealing with each attack.
7.5.1 RNIC Resource Consumption 7.5.1 RNIC Resource Consumption
This section covers attacks that fall into the general category This section covers attacks that fall into the general category
of a Local Peer attempting to unfairly allocate scarce RNIC of a Local Peer attempting to unfairly allocate scarce (i.e.
resources. The Local Peer may be attempting to allocate resources bounded) RNIC resources. The Local Peer may be attempting to
on its own behalf, or on behalf of a Remote Peer. Resources that allocate resources on its own behalf, or on behalf of a Remote
fall into this category include: Protection Domains, Stream Peer. Resources that fall into this category include: Protection
Context Memory, Translation and Protection Tables, and STag Domains, Stream Context Memory, Translation and Protection
namespace. These can be attacks by currently active Local Peers Tables, and STag namespace. These can be attacks by currently
or ones that allocated resources earlier, but are now idle. active Local Peers or ones that allocated resources earlier, but
are now idle.
This type of attack can occur regardless of whether resources are This type of attack can occur regardless of whether resources are
shared across Streams. shared across Streams.
It is RECOMMENDED that the allocation of all scarce resources be The allocation of all scarce resources MUST be placed under the
placed under the control of a Privileged Resource Manager. This control of a Privileged Resource Manager. This allows the
allows the Privileged Resource Manager to: Privileged Resource Manager to:
* prevent a Local Peer from allocating more than its fair * prevent a Local Peer from allocating more than its fair
share of resources. share of resources.
* detect if a Remote Peer is attempting to launch a DOS * detect if a Remote Peer is attempting to launch a DOS
attack by attempting to create an excessive number of attack by attempting to create an excessive number of
Streams and take corrective action (such as refusing the Streams and take corrective action (such as refusing the
request or applying network layer filters against the request or applying network layer filters against the
Remote Peer). Remote Peer).
skipping to change at page 30, line 27 skipping to change at page 30, line 35
of receive data buffers (Untagged DDP buffers or for RDMAP of receive data buffers (Untagged DDP buffers or for RDMAP
buffers consumed with Send Type Messages) if receive buffers are buffers consumed with Send Type Messages) if receive buffers are
shared across multiple Streams. shared across multiple Streams.
If resources are not shared across multiple Streams, then this If resources are not shared across multiple Streams, then this
attack is not possible because the Remote Peer will not be able attack is not possible because the Remote Peer will not be able
to consume more buffers than were allocated to the Stream. The to consume more buffers than were allocated to the Stream. The
worst case scenario is that the Remote Peer can consume more worst case scenario is that the Remote Peer can consume more
receive buffers than the Local Peer allowed, resulting in no receive buffers than the Local Peer allowed, resulting in no
buffers to be available, which could cause the Remote PeerĘs buffers to be available, which could cause the Remote PeerĘs
Stream to the Local Peer to be torn down. Stream to the Local Peer to be torn down, and all allocated
resources to be released.
If local receive data buffers are shared among multiple Streams, If local receive data buffers are shared among multiple Streams,
then the Remote Peer can attempt to consume more than its fair then the Remote Peer can attempt to consume more than its fair
share of the receive buffers, causing a different Stream to be share of the receive buffers, causing a different Stream to be
short of receive buffers, thus possibly causing the other Stream short of receive buffers, thus possibly causing the other Stream
to be torn down. For example, if the Remote Peer sent enough one to be torn down. For example, if the Remote Peer sent enough one
byte Untagged Messages, they might be able to consume all local byte Untagged Messages, they might be able to consume all local
shared receive queue resources with little effort on their part. shared receive queue resources with little effort on their part.
One method the Local Peer could use is to recognize that a Remote One method the Local Peer could use is to recognize that a Remote
Peer is attempting to use more than its fair share of resources Peer is attempting to use more than its fair share of resources
and terminate the Stream. However, if the Local Peer is and terminate the Stream (causing the allocated resources to be
sufficiently slow, it may be possible for the Remote Peer to released). However, if the Local Peer is sufficiently slow, it
still mount a denial of service attack. One countermeasure that may be possible for the Remote Peer to still mount a denial of
can protect against this attack is implementing a low-water service attack. One countermeasure that can protect against this
notification. The low-water notification alerts the application attack is implementing a low-water notification. The low-water
if the number of buffers in the receive queue is less than a notification alerts the application if the number of buffers in
threshold. the receive queue is less than a threshold.
If all of the following conditions are true, then the Local Peer If all of the following conditions are true, then the Local Peer
can size the amount of local receive buffers posted on the can size the amount of local receive buffers posted on the
receive queue to ensure a DOS attack can be stopped. receive queue to ensure a DOS attack can be stopped.
* a low-water notification is enabled, and * a low-water notification is enabled, and
* the Local Peer is able to bound the amount of time that * the Local Peer is able to bound the amount of time that
it takes to replenish receive buffers, and it takes to replenish receive buffers, and
* the Local Peer maintains statistics to determine which * the Local Peer maintains statistics to determine which
Remote Peer is consuming buffers. Remote Peer is consuming buffers.
The above conditions enable the low-water notification to arrive The above conditions enable the low-water notification to arrive
before resources are depleted and thus the Local Peer can take before resources are depleted and thus the Local Peer can take
corrective action (e.g., terminate the Stream of the attacking corrective action (e.g., terminate the Stream of the attacking
Remote Peer). Remote Peer).
A different, but similar attack is if the Remote Peer sends a A different, but similar attack is if the Remote Peer sends a
significant number of out-of-order packets and the RNIC has the significant number of out-of-order packets and the RNIC has the
skipping to change at page 31, line 35 skipping to change at page 31, line 45
to arrive for the buffer, but the buffer has not yet been to arrive for the buffer, but the buffer has not yet been
delivered to the ULP). delivered to the ULP).
A different countermeasure is for the RNIC Engine to provide the A different countermeasure is for the RNIC Engine to provide the
capability to limit the Remote PeerĘs ability to consume receive capability to limit the Remote PeerĘs ability to consume receive
buffers on a per Stream basis. Unfortunately this requires a buffers on a per Stream basis. Unfortunately this requires a
large amount of state to be tracked in each RNIC on a per Stream large amount of state to be tracked in each RNIC on a per Stream
basis. basis.
Thus, if an RNIC Engine provides the ability to share receive Thus, if an RNIC Engine provides the ability to share receive
buffers across multiple Streams, it is RECOMMENDED that it enable buffers across multiple Streams, the combination of the RNIC
the Local Peer to detect if the Remote Peer is attempting to Engine and the Privileged Resource Manager MUST be able to detect
consume more than its fair share of resources so that the if the Remote Peer is attempting to consume more than its fair
application can apply countermeasures to detect and prevent the share of resources so that the Local Peer can apply
attack. countermeasures to detect and prevent the attack.
7.5.2.2 Local Peer Attacking a Shared CQ 7.5.2.2 Local Peer Attacking a Shared CQ
DOS attacks against a Shared Completion Queue (CQ) can be caused DOS attacks against a Shared Completion Queue (CQ) can be caused
by either the Local Peer or the Remote Peer if either attempts to by either the Local Peer or the Remote Peer if either attempts to
cause more completions than its fair share of the number of cause more completions than its fair share of the number of
entries, thus potentially starving another unrelated Stream such entries, thus potentially starving another unrelated Stream such
that no Completion Queue entries are available. that no Completion Queue entries are available.
A Completion Queue entry can potentially be consumed by a A Completion Queue entry can potentially be consumed by a
completion from the send queue or a receive completion. In the completion from the Send Queue or a Receive Queue completion. In
former, the attacker is the Local Peer. In the later, the the former, the attacker is the Local Peer. In the later, the
attacker is the Remote Peer. attacker is the Remote Peer.
A form of attack can occur where the Local Peers can consume A form of attack can occur where the Local Peers can consume
resources on the CQ. A Local Peer that is slow to free resources resources on the CQ. A Local Peer that is slow to free resources
on the CQ by not reaping the completion status quickly enough on the CQ by not reaping the completion status quickly enough
could stall all other Local Peers attempting to use that CQ. could stall all other Local Peers attempting to use that CQ.
One of two countermeasures can be used to avoid this kind of One of two countermeasures can be used to avoid this kind of
attack. The first is to only share a CQ between Streams that attack. The first is to only share a CQ between Streams that
share Partial Mutual Trust. The other is to use a trusted Local share Partial Mutual Trust (i.e. Streams within the same
Peer to act as a third party to free resources on the CQ and Protection Domain). The other is to use a trusted Local Peer to
place the status in intermediate storage until the untrusted act as a third party to free resources on the CQ and place the
Local Peer reaps the status information. For these reason, status in intermediate storage until the untrusted Local Peer
sharing a CQ across Streams that belong to different Protection reaps the status information. For these reasons, an RNIC MUST NOT
Domains is NOT RECOMMENDED. enable sharing a CQ across Streams that belong to different
Protection Domains. Addtionally, an application SHOULD NOT share
a CQ between Streams which do not share Partial Mutual Trust.
7.5.2.3 Remote Peer Attacking a Shared CQ 7.5.2.3 Remote Peer Attacking a Shared CQ
For an overview of the Shared CQ attack model, see Section For an overview of the Shared CQ attack model, see Section
7.5.2.2. 7.5.2.2.
The Remote Peer can attack a CQ by consuming more than its fair The Remote Peer can attack a CQ by consuming more than its fair
share of CQ entries by using one of the following methods: share of CQ entries by using one of the following methods:
* The ULP protocol allows the Remote Peer to reserve a * The ULP protocol allows the Remote Peer to reserve a
skipping to change at page 32, line 39 skipping to change at page 32, line 51
* If the Remote Peer or Local Peer (or both) can attack the * If the Remote Peer or Local Peer (or both) can attack the
CQ by overwhelming the CQ with completions, then CQ by overwhelming the CQ with completions, then
completion processing on other Streams sharing that completion processing on other Streams sharing that
Completion Queue can be affected (e.g. the Completion Completion Queue can be affected (e.g. the Completion
Queue overflows and stops functioning). Queue overflows and stops functioning).
The first method of attack can be avoided if the ULP does not The first method of attack can be avoided if the ULP does not
allow a Remote Peer to reserve CQ entries or there is a trusted allow a Remote Peer to reserve CQ entries or there is a trusted
intermediary such as a Privileged Resource Manager. Unfortunately intermediary such as a Privileged Resource Manager. Unfortunately
it is often unrealistic to not allow a Remote Peer to reserve CQ it is often unrealistic to not allow a Remote Peer to reserve CQ
entries ū particularly if the number of completion entries is entries - particularly if the number of completion entries is
dependent on other ULP negotiated parameters, such as the amount dependent on other ULP negotiated parameters, such as the amount
of buffering required by the ULP. Thus it is RECOMMENDED that an of buffering required by the ULP. Thus an implementation MUST
implementation require a Privileged Resource Manager to control implement a Privileged Resource Manager to control the allocation
the allocation of CQ entries. of CQ entries. See Section 4.1 Components on page 9 for a
definition of Privileged Resource Manager.
One way that a Local or Remote Peer can attempt to overwhelm a CQ One way that a Local or Remote Peer can attempt to overwhelm a CQ
with completions is by sending minimum length RDMAP/DDP Messages with completions is by sending minimum length RDMAP/DDP Messages
to cause as many completions (receive completions for the Remote to cause as many completions (receive completions for the Remote
Peer, send completions for the Local Peer) per second as Peer, send completions for the Local Peer) per second as
possible. If it is the Remote Peer attacking, and we assume that possible. If it is the Remote Peer attacking, and we assume that
the Local Peer does not run out of receive buffers (if they do, the Local Peer does not run out of receive buffers (if they do,
then this is a different attack, documented in Section 7.5.2.1 then this is a different attack, documented in Section 7.5.2.1
Multiple Streams Sharing Receive Buffers on page 30), then it Multiple Streams Sharing Receive Buffers on page 30), then it
might be possible for the Remote Peer to consume more than its might be possible for the Remote Peer to consume more than its
skipping to change at page 33, line 13 skipping to change at page 33, line 27
implementation, this could either cause the CQ to overflow (if it implementation, this could either cause the CQ to overflow (if it
is not large enough to handle all of the completions generated) is not large enough to handle all of the completions generated)
or for another Stream to not be able to generate CQ entries (if or for another Stream to not be able to generate CQ entries (if
the RNIC had flow control on generation of CQ entries into the the RNIC had flow control on generation of CQ entries into the
CQ). In either case, the CQ will stop functioning correctly and CQ). In either case, the CQ will stop functioning correctly and
any Streams expecting completions on the CQ will stop any Streams expecting completions on the CQ will stop
functioning. functioning.
This attack can occur regardless of whether all of the Streams This attack can occur regardless of whether all of the Streams
associated with the CQ are in the same Protection Domain or are associated with the CQ are in the same Protection Domain or are
in different Protection Domains ū the key issue is that the in different Protection Domains - the key issue is that the
number of Completion Queue entries is less than the number of all number of Completion Queue entries is less than the number of all
outstanding operations that can cause a completion. outstanding operations that can cause a completion.
The Local Peer can protect itself from this type of attack using The Local Peer can protect itself from this type of attack using
either of the following methods: either of the following methods:
* Resize the CQ to the appropriate level(note that resizing * Size the CQ to the appropriate level, as specified below
the CQ can fail, so the CQ resize should be done before (note that if the CQ currently exists, and it needs to be
sizing the Send Queue and Receive Queue on the Stream), resized, resizing the CQ can fail, so the CQ resize
OR should be done before sizing the Send Queue and Receive
Queue on the Stream), OR
* Grant fewer resources than the Remote Peer requested (not * Grant fewer resources than the Remote Peer requested (not
supplying the number of Receive Data Buffers requested). supplying the number of Receive Data Buffers requested).
The proper sizing of the CQ is dependent on whether the Local The proper sizing of the CQ is dependent on whether the Local
Peer will post as many resources to the various queues as the Peer will post as many resources to the various queues as the
size of the queue enables or not. If the Local Peer can be size of the queue enables or not. If the Local Peer can be
trusted to post a number of resources that is smaller than the trusted to post a number of resources that is smaller than the
size of the specific resourceĘs queue, then a correctly sized CQ size of the specific resourceĘs queue, then a correctly sized CQ
means that the CQ is large enough to hold completion status for means that the CQ is large enough to hold completion status for
all of the outstanding Data Buffers (both send and receive all of the outstanding Data Buffers (both send and receive
buffers), or: buffers), or:
CQ_MIN_SIZE = SUM(MaxPostedOnEachRQ) CQ_MIN_SIZE = SUM(MaxPostedOnEachRQ)
+ SUM(MaxPostedOnEachS-RQ) + SUM(MaxPostedOnEachSRQ)
+ SUM(MaxPostedOnEachSQ) + SUM(MaxPostedOnEachSQ)
Where:
MaxPostedOnEachRQ = the maximum number of requests which
can cause a completion that will be posted on a
specific Receive Queue.
MaxPostedOnEachSRQ = the maximum number of requests which
can cause a completion that will be posted on a
specific Shared Receive Queue.
MaxPostedOnEachSQ = the maximum number of requests which
can cause a completion that will be posted on a
specific Send Queue.
If the local peer must be able to completely fill the queues, or If the local peer must be able to completely fill the queues, or
can not be trusted to observe a limit smaller than the queues, can not be trusted to observe a limit smaller than the queues,
then the CQ must be sized to accommodate the maximum number of then the CQ must be sized to accommodate the maximum number of
operations that it is possible to post at any one time. Thus the operations that it is possible to post at any one time. Thus the
equation becomes: equation becomes:
CQ_MIN_SIZE = SUM(SizeOfEachRQ) CQ_MIN_SIZE = SUM(SizeOfEachRQ)
+ SUM(SizeOfEachS-RQ) + SUM(SizeOfEachSRQ)
+ SUM(SizeOfEachSQ) + SUM(SizeOfEachSQ)
Where:
SizeOfEachRQ = the maximum number of requests which
can cause a completion that can ever be posted
on a specific Receive Queue.
SizeOfEachSRQ = the maximum number of requests which
can cause a completion that can ever be posted
on a specific Shared Receive Queue.
SizeOfEachSQ = the maximum number of requests which
can cause a completion that can ever be posted
on a specific Send Queue.
Where MaxPosted*OnEach*Q and SizeOfEach*Q varies on a per Stream Where MaxPosted*OnEach*Q and SizeOfEach*Q varies on a per Stream
or per Shared Receive Queue basis. or per Shared Receive Queue basis.
It is RECOMMENDED that the Local Peer implement a mechanism to The Local Peer MUST implement a mechanism to ensure that the
ensure that the Completion Queue can not overflow. Note that it Completion Queue can not overflow. Note that it is possible to
is possible to share CQs even if the Remote Peers accessing the share CQs even if the Remote Peers accessing the CQs are
CQs are untrusted if either of the above two formulas are untrusted if either of the above two formulas are implemented. If
implemented. If the Local Peer can be trusted to not post more the Local Peer can be trusted to not post more than
than MaxPostedOnEachRQ, MaxPostedOnEachS-RQ, and MaxPostedOnEachRQ, MaxPostedOnEachSRQ, and MaxPostedOnEachSQ,
MaxPostedOnEachSQ, then the first formula applies. If the Local then the first formula applies. If the Local Peer can not be
Peer can not be trusted to obey the limit, then the second trusted to obey the limit, then the second formula applies.
formula applies.
7.5.2.4 RDMA Read Request Queue 7.5.2.4 Attacking the RDMA Read Request Queue
If RDMA Read Request Queue resources are pooled across multiple If RDMA Read Request Queue resources are pooled across multiple
Streams, one attack is if the Local Peer attempts to unfairly Streams, one attack is if the Local Peer attempts to unfairly
allocate RDMA Read Request Queue resources for its Streams. For allocate RDMA Read Request Queue resources for its Streams. For
example, the Local Peer attempts to allocate all available example, the Local Peer attempts to allocate all available
resources on a specific RDMA Read Request Queue for its Streams, resources on a specific RDMA Read Request Queue for its Streams,
thereby denying the resource to applications sharing the RDMA thereby denying the resource to applications sharing the RDMA
Read Request Queue. The same type of argument applies even if the Read Request Queue. The same type of argument applies even if the
RDMA Read Request is not shared ū but a Local Peer attempts to RDMA Read Request is not shared - but a Local Peer attempts to
allocate all of the RNICs resource when the queue is created. allocate all of the RNICs resource when the queue is created.
Thus it is RECOMMENDED that access to interfaces that allocate Thus access to interfaces that allocate RDMA Read Request Queue
RDMA Read Request Queue entries be restricted to a trusted Local entries MUST be restricted to a trusted Local Peer, such as a
Peer, such as a Privileged Resource Manager. The Privileged Privileged Resource Manager. The Privileged Resource Manager
Resource Manager should prevent a Local Peer from allocating more SHOULD prevent a Local Peer from allocating more than its fair
than its fair share of resources. share of resources.
Another form of attack is if the Remote Peer sends more RDMA Read Another form of attack is if the Remote Peer sends more RDMA Read
Requests than the depth of the RDMA Read Request Queue at the Requests than the depth of the RDMA Read Request Queue at the
Local Peer. If the RDMA Read Request Queue is a shared resource, Local Peer. If the RDMA Read Request Queue is a shared resource,
this could corrupt the queue. If the queue is not shared, then this could corrupt the queue. If the queue is not shared, then
the worst case is that the current Stream is disabled. One the worst case is that the current Stream is disabled. One
approach to solving the shared RDMA Read Request Queue would be approach to solving the shared RDMA Read Request Queue would be
to create thresholds, similar to those described in Section to create thresholds, similar to those described in Section
7.5.2.1 Multiple Streams Sharing Receive Buffers on page 30. A 7.5.2.1 Multiple Streams Sharing Receive Buffers on page 30. A
simpler approach is to not share RDMA Read Request Queue simpler approach is to not share RDMA Read Request Queue
resources amoung Streams or enforce hard limits of consumption resources amoung Streams or enforce hard limits of consumption
per Stream. Thus it is RECOMMENDED that RDMA Read Request Queue per Stream. Thus RDMA Read Request Queue resource consumption
resource consumption be controlled such that RDMAP/DDP Streams MUST be controlled such that RDMAP/DDP Streams which do not share
which do not share Partial Mutual Trust do not share RDMA Read Partial Mutual Trust do not share RDMA Read Request Queue
Request Queue resources. resources.
If the issue is a bug in the Remote PeerĘs implementation, and If the issue is a bug in the Remote PeerĘs implementation, and
not a malicious attack, the issue can be solved by requiring the not a malicious attack, the issue can be solved by requiring the
Remote PeerĘs RNIC to throttle RDMA Read Requests. By properly Remote PeerĘs RNIC to throttle RDMA Read Requests. By properly
configuring the Stream at the Remote Peer through a trusted configuring the Stream at the Remote Peer through a trusted
agent, the RNIC can be made to not transmit RDMA Read Requests agent, the RNIC can be made to not transmit RDMA Read Requests
that exceed the depth of the RDMA Read Request Queue at the Local that exceed the depth of the RDMA Read Request Queue at the Local
Peer. If the Stream is correctly configured, and if the Remote Peer. If the Stream is correctly configured, and if the Remote
Peer submits more requests than the Local PeerĘs RDMA Read Peer submits more requests than the Local PeerĘs RDMA Read
Request Queue can handle, the requests would be queued at the Request Queue can handle, the requests would be queued at the
Remote PeerĘs RNIC until previous requests complete. If the Remote PeerĘs RNIC until previous requests complete. If the
Remote PeerĘs Stream is not configured correctly, the RDMAP Remote PeerĘs Stream is not configured correctly, the RDMAP
Stream is terminated when more RDMA Read Requests arrive at the Stream is terminated when more RDMA Read Requests arrive at the
Local Peer than the Local Peer can handle (assuming the prior Local Peer than the Local Peer can handle (assuming the prior
paragraphĘs recommendation is implemented). paragraphĘs recommendation is implemented). Thus an RNIC
implementation MUST provide a mechanism to cap the number of
outstanding RDMA Read Requests.
7.5.3 Resource Consumption by Idle Applications 7.5.3 Resource Consumption by Idle Applications
The simplest form of a DOS attack given a fixed amount of The simplest form of a DOS attack given a fixed amount of
resources is for the Remote Peer to create a RDMAP Stream to a resources is for the Remote Peer to create a RDMAP Stream to a
Local Peer, and request dedicated resources then do no actual Local Peer, and request dedicated resources then do no actual
work. This allows the Remote Peer to be very light weight (i.e. work. This allows the Remote Peer to be very light weight (i.e.
only negotiate resources, but do no data transfer) and consumes a only negotiate resources, but do no data transfer) and consumes a
disproportionate amount of resources in the server. disproportionate amount of resources in the server.
skipping to change at page 35, line 47 skipping to change at page 36, line 44
section can be used to free resources allocated by an idle Local section can be used to free resources allocated by an idle Local
Peer. Peer.
7.5.4 Exercise of non-optimal code paths 7.5.4 Exercise of non-optimal code paths
Another form of DOS attack is to attempt to exercise data paths Another form of DOS attack is to attempt to exercise data paths
that can consume a disproportionate amount of resources. An that can consume a disproportionate amount of resources. An
example might be if error cases are handled on a "slow path" example might be if error cases are handled on a "slow path"
(consuming either host or RNIC computational resources), and an (consuming either host or RNIC computational resources), and an
attacker generates excessive numbers of errors in an attempt to attacker generates excessive numbers of errors in an attempt to
consume these resources. consume these resources. Note that for most RDMAP or DDP errors,
the attacking Stream will simply be torn down. Thus for this form
of attack to be effective, the Remote Peer needs to exercise data
paths which do not cause the Stream to be torn down.
It is RECOMMENDED that an implementation provide the ability to If an RNIC implementation contains "slow paths" which do not
detect the above condition and allow an administrator to act, result in the tear down of the Stream, it is recommended that an
including potentially administratively tearing down the RDMAP implementation provide the ability to detect the above condition
Stream associated with the Stream exercising data paths consuming and allow an administrator to act, including potentially
a disproportionate amount of resources. administratively tearing down the RDMAP Stream associated with
the Stream exercising data paths consuming a disproportionate
amount of resources.
7.5.5 RI an STag Shared on Multiple Streams 7.5.5 Remote Invalidate an STag Shared on Multiple Streams
If a Local Peer has enabled an STag for remote access, the Remote If a Local Peer has enabled an STag for remote access, the Remote
Peer could attempt to remote invalidate (RI) the STag by using Peer could attempt to remote invalidate the STag by using the
the RDMAP Send with Invalidate or Send with SE and Invalidate RDMAP Send with Invalidate or Send with SE and Invalidate
Message. If the STag is only valid on the current Stream, then Message. If the STag is only valid on the current Stream, then
the only side effect is that the Remote Peer can no longer use the only side effect is that the Remote Peer can no longer use
the STag; thus there are no security issues. the STag; thus there are no security issues.
If the STag is valid across multiple Streams, then the Remote If the STag is valid across multiple Streams, then the Remote
Peer can prevent other Streams from using that STag by using the Peer can prevent other Streams from using that STag by using the
remote invalidate functionality. remote invalidate functionality.
Thus if RDDP Streams do not share Partial Mutual Trust (i.e. the Thus if RDDP Streams do not share Partial Mutual Trust (i.e. the
Remote Peer may attempt to invalidate the STag prematurely), it Remote Peer may attempt to invalidate the STag prematurely), the
is NOT RECOMMENDED that the application allow an STag to be valid application MUST NOT allow an STag to be valid across multiple
across multiple Streams. Streams.
7.6 Elevation of Privilege 7.6 Elevation of Privilege
The RDMAP/DDP Security Architecture explicitly differentiates The RDMAP/DDP Security Architecture explicitly differentiates
between three levels of privilege - Non-Privileged, Privileged, between three levels of privilege - Non-Privileged, Privileged,
and the Privileged Resource Manager. If a Non-Privileged and the Privileged Resource Manager. If a Non-Privileged
Application is able to elevate its privilege level to a Application is able to elevate its privilege level to a
Privileged Application, then mapping a physical address list to Privileged Application, then mapping a physical address list to
an STag can provide local and remote access to any physical an STag can provide local and remote access to any physical
address location on the node. If a Privileged Mode Application is address location on the node. If a Privileged Mode Application is
able to promote itself to be a Resource Manager, then it is able to promote itself to be a Resource Manager, then it is
possible for it to perform denial of service type attacks where possible for it to perform denial of service type attacks where
substantial amounts of local resources could be consumed. substantial amounts of local resources could be consumed.
In general, elevation of privilege is a local implementation In general, elevation of privilege is a local implementation
specific issue and thus outside the scope of this specification. specific issue and thus outside the scope of this specification.
There is one issue worth noting, however. If the RI There is one issue worth noting, however. If the RNIC
implementation, by some insecure mechanism (or implementation implementation, by some insecure mechanism (or implementation
defect), can enable a Remote Peer or un-trusted Local Peer to defect), can enable a Remote Peer or un-trusted Local Peer to
load firmware into the RNIC Engine, it is possible to use the load firmware into the RNIC Engine, it is possible to use the
RNIC to attack the host. Thus, it is RECOMMENDED that an RNIC to attack the host. Thus, an implementation MUST NOT enable
implementation not enable firmware to be loaded on the RNIC firmware to be loaded on the RNIC Engine directly from a Remote
Engine directly from a Remote Peer, unless the Remote Peer is Peer, unless the Remote Peer is properly authenticated (by a
properly authenticated (by a mechanism outside the scope of this mechanism outside the scope of this specification. The mechanism
specification. The mechanism presumably entails authenticating presumably entails authenticating that the remote application has
that the remote application has the right to perform the update), the right to perform the update), and the update is done via a
and the update is done via a secure protocol, such as IPsec (See secure protocol, such as IPsec (See Section 8 Security Services
Section 8 Security Services for RDDP on page 37). It is for RDMA and DDP on page 38). Further, an implementation MUST NOT
RECOMMENDED that an implementation not allow a Non-Privileged allow a Non-Privileged Local Peer to update firmware in the RNIC
Local Peer to update firmware in the RNIC Engine. Engine.
8 Security Services for RDDP
Issue: The spec currently took the IPSec requirements for iSCSI 8 Security Services for RDMA and DDP
and made them a SHOULD recommendation. A different approach would
be to simply outline the issues in this section, but leave IPSec
implementation requirements to be specified by ULP/Application
requirements. The argument here is that RDDP is a transport, and
security requirements ū particularly authentication and
confidentiality requirements, are dictated by application
concerns, not transport protocol concerns. Which approach should
be taken?
RDMA and RDDP are used to control, read and write data buffers RDMA and DDP are used to control, read and write data buffers
over IP networks. Therefore, the control and the data packets of over IP networks. Therefore, the control and the data packets of
these protocols are vulnerable to the spoofing, tampering and these protocols are vulnerable to the spoofing, tampering and
information disclosure attacks listed in Section 7. information disclosure attacks listed in Section 7.
Generally speaking, session confidentiality protects against Generally speaking, Stream confidentiality protects against
eaves dropping. Session authentication and integrity protection eavesdropping. Stream and/or session authentication and integrity
is a counter measurement against various spoofing and tampering protection is a counter measurement against various spoofing and
attacks. The effectiveness of authentication and integrity tampering attacks. The effectiveness of authentication and
against a specific attack, depend on whether the authentication integrity against a specific attack, depend on whether the
is machine level authentication (as the one provided by IPsec and authentication is machine level authentication (as the one
SSL), or ULP authentication. provided by IPsec and SSL), or ULP authentication.
8.1 Introduction to Security Options 8.1 Introduction to Security Options
The following security services can be applied to an RDDP/RDMA The following security services can be applied to an RDMAP/DDP
session: Stream:
1. Session confidentiality - protects against eaves dropping 1. Session confidentiality - protects against eaves dropping
(section 7.4.9). (section 7.4.9).
2. Per-packet data source authentication - protects against the 2. Per-packet data source authentication - protects against the
following spoofing attacks: network based impersonation following spoofing attacks: network based impersonation
(section 7.2.1), Stream hijacking (section 7.2.2), and man in (section 7.2.1), Stream hijacking (section 7.2.2), and man in
the middle (section 7.2.3). the middle (section 7.2.3).
3. Per-packet integrity - protects against tampering done by 3. Per-packet integrity - protects against tampering done by
network based modification of buffer content (section 7.3.4) network based modification of buffer content (section 7.3.4)
4. Packet sequencing - protects against replay attacks, which is 4. Packet sequencing - protects against replay attacks, which is
a special case of the above tampering attack. a special case of the above tampering attack.
If RDDP/RDMA session may be subject to impersonation attacks, or If an RDMAP/DDP Stream may be subject to impersonation attacks,
Stream hijacking attacks, it is RECOMMENDED that the session be or Stream hijacking attacks, it is recommended that the Stream be
authenticated, integrity protected, and protected from replay authenticated, integrity protected, and protected from replay
attacks; it MAY use confidentiality protection to protect from attacks; it MAY use confidentiality protection to protect from
eaves dropping (in case the RDDP/RDMA session traverses a public eavesdropping (in case the RDMAP/DDP Stream traverses a public
network). network).
Both IPsec and SSL are capable of providing the above security Both IPsec and SSL are capable of providing the above security
services for IP and TCP traffic respectively. ULP protocols are services for IP and TCP traffic respectively. ULP protocols are
able to provide only part of the above security services. The able to provide only part of the above security services. The
next sections describe the different security options. next sections describe the different security options.
8.1.1 Introduction to IPsec 8.1.1 Introduction to IPsec
IPsec is a protocol suite which is used to secure communication IPsec is a protocol suite which is used to secure communication
at the network layer between two peers. The IPsec protocol suite at the network layer between two peers. The IPsec protocol suite
is specified within the IP Security Architecture [RFC2401], IKE is specified within the IP Security Architecture [RFC2401], IKE
[RFC2409], IPsec Authentication Header (AH) [RFC2402] and IPsec [RFC2409], IPsec Authentication Header (AH) [RFC2402] and IPsec
Encapsulating Security Payload (ESP) [RFC2406] documents. IKE is Encapsulating Security Payload (ESP) [RFC2406] documents. IKE is
the key management protocol while AH and ESP are used to protect the key management protocol while AH and ESP are used to protect
IP traffic. IP traffic.
An IPsec SA is a one-way security association, uniquely An IPsec SA is a one-way security association, uniquely
identified by the 3-tuple: Security Parameter Index (SPI), identified by the 3-tuple: Security Parameter Index (SPI),
protocol (ESP) and destination IP. The parameters for an IPsec protocol (ESP/AH) and destination IP address. The parameters for
security association are typically established by a key an IPsec security association are typically established by a key
management protocol. These include the encapsulation mode, management protocol. These include the encapsulation mode,
encapsulation type, session keys and SPI values. encapsulation type, session keys and SPI values.
IKE is a two phase negotiation protocol based on the modular IKE is a two phase negotiation protocol based on the modular
exchange of messages defined by ISAKMP [RFC2408],and the IP exchange of messages defined by ISAKMP [RFC2408],and the IP
Security Domain of Interpretation (DOI) [RFC2407]. IKE has two Security Domain of Interpretation (DOI) [RFC2407]. IKE has two
phases, and accomplishes the following functions: phases, and accomplishes the following functions:
1. Protected cipher suite and options negotiation - using keyed 1. Protected cipher suite and options negotiation - using keyed
MACs and encryption and anti-replay mechanisms. MACs and encryption and anti-replay mechanisms.
skipping to change at page 38, line 51 skipping to change at page 39, line 37
4. IPsec SA management (selector negotiation, options 4. IPsec SA management (selector negotiation, options
negotiation, create, delete, and rekeying). negotiation, create, delete, and rekeying).
Items 1 through 3 are accomplished in IKE Phase 1, while item 4 Items 1 through 3 are accomplished in IKE Phase 1, while item 4
is handled in IKE Phase 2. is handled in IKE Phase 2.
IKE phase 1 defines four authentication methods; three of them IKE phase 1 defines four authentication methods; three of them
require both sides to have certified signature or encryption require both sides to have certified signature or encryption
public keys; the forth require the side to exchange out-of-band a public keys; the forth require the side to exchange out-of-band a
secret random string ū called pre-shared-secret (PSS). secret random string - called pre-shared-secret (PSS).
An IKE Phase 2 negotiation is performed to establish both an An IKE Phase 2 negotiation is performed to establish both an
inbound and an outbound IPsec SA. The traffic to be protected by inbound and an outbound IPsec SA. The traffic to be protected by
an IPsec SA is determined by a selector which has been proposed an IPsec SA is determined by a selector which has been proposed
by the IKE initiator and accepted by the IKE Responder. The by the IKE initiator and accepted by the IKE Responder. The IPsec
IPsec SA selector can be a "filter" or traffic classifier, SA selector can be a "filter" or traffic classifier, defined as
defined as the 5-tuple: <Source IP address, Destination IP the 5-tuple: <Source IP address, Destination IP address,
address, transport protocol (e.g. UDP/SCTP/TCP), Source port, transport protocol (e.g. UDP/SCTP/TCP), Source port, Destination
Destination port>. The successful establishment of a IKE Phase-2 port>. The successful establishment of a IKE Phase-2 SA results
SA results in the creation of two uni-directional IPsec SAs fully in the creation of two uni-directional IPsec SAs fully qualified
qualified by the tuple <Protocol (ESP/AH), destination address, by the tuple <Protocol (ESP/AH), destination address, SPI>.
SPI>.
The session keys for each IPsec SA are derived from a master key, The session keys for each IPsec SA are derived from a master key,
typically via a MODP Diffie-Hellman computation. Rekeying of an typically via a MODP Diffie-Hellman computation. Rekeying of an
existing IPsec SA pair is accomplished by creating two new IPsec existing IPsec SA pair is accomplished by creating two new IPsec
SAs, making them active, and then optionally deleting the older SAs, making them active, and then optionally deleting the older
IPsec SA pair. Typically the new outbound SA is used IPsec SA pair. Typically the new outbound SA is used immediately,
immediately, and the old inbound SA is left active to receive and the old inbound SA is left active to receive packets for some
packets for some locally defined time, perhaps 30 seconds or 1 locally defined time, perhaps 30 seconds or 1 minute. Optionally,
minute. Optionally, rekeying can use Diffie-Helman for keying rekeying can use Diffie-Helman for keying material generation.
material generation.
8.1.2 Introduction to SSL Limitations on RDMAP 8.1.2 Introduction to SSL Limitations on RDMAP
SSL and TLS [RFC 2246] provide session authentication, integrity SSL and TLS [RFC 2246] provide Stream authentication, integrity
and confidentiality for TCP based applications. SSL supports one- and confidentiality for TCP based applications. SSL supports one-
way (server only) or mutual certificates based authentication. way (server only) or mutual certificates based authentication.
There are at least two limitations that make SSL less appropriate There are at least two limitations that make SSL underneath RDMAP
then IPsec for RDDP/RDMA security: less appropriate then IPsec for DDP/RDMA security:
1. The maximum length supported by the TLS record layer protocol 1. The maximum length supported by the TLS record layer protocol
is 2^14 bytes, longer packets must be fragmented (as a is 2^14 bytes - longer packets must be fragmented (as a
comparison, the maximal length of an IPsec packet, is comparison, the maximal length of an IPsec packet is
determined by the maximum length of an IP packet). determined by the maximum length of an IP packet).
2. SSL is a connection oriented protocol. If a stream cipher or 2. SSL is a connection oriented protocol. If a stream cipher or
block cipher in CBC mode is used for bulk encryption, then a block cipher in CBC mode is used for bulk encryption, then a
packet can be decrypted only after all the packets preceding packet can be decrypted only after all the packets preceding
it have already arrived. If SSL is used to protect RDDP/RDMA it have already arrived. If SSL is used to protect DDP/RDMA
traffic, then RDDP/RDMA must gather all out-of-order packets traffic, then SSL must gather all out-of-order packets before
before placing them into the ULP buffer, which might cause a RDMAP/DDP can place them into the ULP buffer, which might
significant decrease in its efficiency. cause a significant decrease in its efficiency.
If SSL is layered on top of RDMAP or DDP, SSL does not protect
the RDMAP and/or DDP headers. Thus a man-in-the-middle attack can
still occur by modifying the RDMAP/DDP header to incorrectly
place the data into the wrong buffer, thus effectively corrupting
the data stream.
8.1.3 Applications Which Provide Security 8.1.3 Applications Which Provide Security
Issue: Guidance for application protocols like NFS which Issue: Guidance for application protocols like NFS which
implement security <TBD>. implement security <TBD>.
8.2 Recommendations for IPsec Encapsulation of RDDP 8.2 Requirements for IPsec Encapsulation of DDP
Since iSCSI is expected to be one of the ULPs running on top of
RDDP, the recommendations in this section follow the lines of
[IPSSEC].
8.2.1 Transforms
All RDDP/RDMA security compliant implementations SHOULD support
IPsec ESP [RFC2406] to provide security for both control packets
and data packets, as well as the replay protection mechanisms of
IPsec. When ESP is utilized, per-packet data origin
authentication, integrity and replay protection MUST be used.
To provide confidentiality with ESP, ESP with 3DES in CBC mode
[RFC2451] SHOULD be supported, and AES in Counter mode, as
described in [AESCTR], SHOULD be supported. To provide data
origin authentication and integrity with ESP, HMAC-SHA1 [RFC2404]
SHOULD be supported, and AES in CBC MAC mode with XCBC extensions
[AESXCBC] SHOULD be supported. DES in CBC mode SHOULD NOT be used
due to its inherent weakness. ESP with NULL encryption SHOULD be
supported for authentication.
8.2.2 IPsec modes
Conformant IP RDDP/RDMA security implementations SHOULD support
ESP [RFC2406] in tunnel mode and MAY implement IPsec with ESP in
transport mode.
8.2.3 IKE
Conformant RDDP/RDMA security implementations SHOULD support IKE
[RFC2409] for peer authentication, negotiation of security
associations, and key management, using the IPsec DOI [RFC2407].
Manual keying MUST NOT be used since it does not provide the
necessary rekeying support.
Conformant RDDP/RDMA security implementations SHOULD support peer
authentication using a pre-shared secret, and MAY support
certificate-based peer authentication using digital signatures.
Peer authentication using the public key encryption methods
outlined in IKE's sections 5.2 and 5.3 [RFC2409] SHOULD NOT be
used.
Conformant RDDP/RDMA security implementations SHOULD support IKE
Main Mode and Aggressive Mode. IKE Main Mode with pre-shared key
authentication SHOULD NOT be used when either of the peers uses a
dynamically assigned IP address. While Main Mode with pre-shared
key authentication offers good security in many cases, situations
where dynamically assigned addresses are used force use of a
group pre-shared key, which is vulnerable to man-in-the-middle
attack. Since IKE Aggressive mode with pre-shared secret
authentication is exposed to off-line dictionary attack if it is
used then the selected pre-shared secrets must be random (or
pseudo-random) strings not shorter than 128 bits.
When digital signatures are used for authentication, either IKE
Main Mode or IKE Aggressive Mode MAY be used. In all cases,
access to locally stored secret information (pre-shared key, or
private key for digital signing) must be suitably restricted,
since compromise of the secret information nullifies the security
properties of the IKE/IPsec protocols.
When digital signatures are used to achieve authentication, an
IKE negotiator SHOULD use IKE Certificate Request Payload(s) to
specify the certificate authority (or authorities) that are
trusted in accordance with its local policy. IKE negotiators
SHOULD check the pertinent Certificate Revocation List (CRL)
before accepting a PKI certificate for use in IKE's
authentication procedures.
The IPsec DOI [RFC2407] provides for several types of
identification data. Within IKE Phase 1, for use within the IDii
and IDir payloads, conformant RDDP/RDMA security implementations
SHOULD support the ID_IPV4_ADDR, ID_IPV6_ADDR (if the protocol
stack supports IPv6) and ID_FQDN Identity Payloads. Identities
other than ID_IPV4_ADDR and ID_IPV6_ADDR (such as ID_FQDN) SHOULD
be employed in situations where Aggressive mode is utilized along
with pre-shared keys and IP addresses are dynamically assigned.
The IP Subnet, IP Address Range, ID_DER_ASN1_DN, ID_DER_ASN1_GN,
and ID_USER_FQDN formats SHOULD NOT be used for RDDP/RDMA
protocol security; The ID_KEY_ID Identity Payload MUST NOT be
used. As described in [RFC2407], within Phase 1 the ID port and
protocol fields MUST be set to zero or to UDP port 500. Also, as
noted in [RFC2407]: When an IKE exchange is authenticated using
certificates (of any format), any ID's used for input to local
policy decisions SHOULD be contained in the certificate used in
the authentication of the exchange.
The Phase 2 Quick Mode exchanges used by RDDP/RDMA protocol
implementations SHOULD explicitly carry the Identity Payload
fields (IDci and IDcr). Each Phase 2 IDci and IDcr Payload
SHOULD carry a single IP address (ID_IPV4_ADDR, ID_IPV6_ADDR) and
SHOULD NOT use the IP Subnet or IP Address Range formats. Other
ID payload formats MUST NOT be used.
To support iSCSI PFS requirements [IPSSEC}, conformant RDDP/RDMA
security implementation SHOULD support PFS in the rekeying
process (i.e. in the Quick Mode exchange).
Since IPsec acceleration hardware may only be able to handle a
limited number of active IKE Phase 2 SAs, Phase 2 delete messages
may be sent for idle SAs, as a means of keeping the number of
active Phase 2 SAs to a minimum. The receipt of an IKE Phase 2
delete message MUST NOT be interpreted as a reason for tearing
down an RDDP/RDMA Stream. Rather, it is preferable to leave the
Stream up, and if additional traffic is sent on it, to bring up
another IKE Phase 2 SA to protect it. This avoids the potential
for continually bringing Streams up and down.
8.2.4 Security Policy Configuration
One of the goals of this specification is to enable a high level
of interoperability without requiring extensive configuration.
This section provides guidelines on setting of IKE parameters so
as to enhance the probability of a successful negotiation. It
also describes how information on security policy configuration
can be provided so as to further enhance the chances of success.
To enhance the prospects for interoperability, some of the
actions to consider include:
[1] Transform restriction. Since support for 3DES-CBC and HMAC-
SHA1 is required of all implementations, offering these
transforms enhances the probability of a successful negotiation.
If AES-CTR [AESCTR] with XCBC-MAC [AESXCBC] is supported, this
transform combination will typically be preferred, with 3DES-
CBC/HMAC-SHA1 as a secondary offer.
[2] Group Restriction. If 3DES-CBC/HMAC-SHA1 is offered, and DH
groups are offered, then it is recommended that a DH group of at
least 1024 bits be offered along with it. If AES-CTR/XCBC-MAC is
the preferred offer, and DH groups are offered, then it is
recommended that a DH group of at least 2048 bits be offered
along with it, as noted in [KeyLen]. If perfect forward secrecy
is required in Quick Mode, then it is recommended that the QM PFS
DH group be the same as the IKE Phase 1 DH group. This reduces
the total number of combinations, enhancing the chances for
interoperability.
[3] Key lifetimes. If a key lifetime is offered that is longer
than desired, then rather than causing the IKE negotiation to
fail, it is recommended that the Responder consider the offered
lifetime as a maximum, and accept it. The key can then use a
lesser value for the lifetime, and utilize a Lifetime Notify in
order to inform the other peer of lifetime expiration.
Even when the above advice is taken, it still may be useful to be
able to provide additional configuration information in order to
enhance the chances of success, and it is useful to be able to
manage security configuration regardless of the scale of the
deployment.
For example, it may be desirable to configure the security policy
of an RDDP/RDMA device. This can be done manually or
automatically via a security policy distribution mechanism.
Alternatively, if the ULP supports a distribution mechanism such
as iSCSI with iSNS or SLPv2, those mechanism can be used to
supply security policy. If an IP block storage endpoint can
obtain the required security policy by other means (manually, or
automatically via a security policy distribution mechanism) then
it need not request this information through the ULP specific
mechanism. However, if the required security policy configuration
is not available via other mechanisms, those mechanismm can be
used.
It may also be helpful to obtain information about the
preferences of the peer prior to initiating IKE. While it is
generally possible to negotiate security parameters within IKE,
there are situations in which incompatible parameters can cause
the IKE negotiation to fail. The following information can be
provided via ULP specific or other mechanisms:
[4] IPsec or cleartext support. The minimum piece of peer
configuration required is whether an RDDP/RDMA endpoint requires
IPsec or cleartext. This cannot be determined from the IKE
negotiation alone without risking a long timeout, which is highly
undesirable for the RDMA/DDP protocol.
[5] Perfect Forward Secrecy (PFS) support. It is helpful to know
whether a peer allows PFS, since an IKE Phase 2 Quick Mode can
fail if an initiator proposes PFS to a Responder that does not
allow it.
[6] Preference for tunnel mode. While it is legal to propose
both transport and tunnel mode within the same offer, not all IKE
implementations will support this. As a result, it is useful to
know whether a peer prefers tunnel mode or transport mode, so
that it is possible to negotiate the preferred mode on the first
try.
[7] Main Mode and Aggressive Mode support. Since the IKE The IP Storage working group has spent significant time and
negotiation can fail if a mode is proposed to a peer that doesn't effort to define the normative IPSec requirements for IP Storage
allow it, it is helpful to know which modes a peer allows, so [RFC3723]. Portions of that specification are applicable to a
that an allowed mode can be negotiated on the first try. wide variety of protocols, including the RDDP protocol suite. In
order to not replicate this effort, an RNIC implementation MUST
follow the requirements defined in RFC3723 Section 2.3 and
Section 5, including the associated normative references for
those sections.
Since iSNS or SLPv2 can be used to distribute IPsec security Additionally, since IPsec acceleration hardware may only be able
policy and configuration information for use with IP block to handle a limited number of active IKE Phase 2 SAs, Phase 2
storage protocols and RDDP/RDMA, these discovery protocols would delete messages may be sent for idle SAs, as a means of keeping
constitute a 'weak link' were they not secured at least as well the number of active Phase 2 SAs to a minimum. The receipt of an
as the protocols whose security they configure. Since the major IKE Phase 2 delete message MUST NOT be interpreted as a reason
vulnerability is packet modification and replay, when iSNS or for tearing down an DDP/RDMA Stream. Rather, it is preferable to
leave the Stream up, and if additional traffic is sent on it, to
bring up another IKE Phase 2 SA to protect it. This avoids the
potential for continually bringing Streams up and down.
SLPv2 are used to distribute security policy or configuration Note that there are serious security issues if IPSec is not
information, at a minimum, per-packet data origin authentication, implemented end-to-end. For example, if IPSec is implemented as a
integrity and replay protection MUST be used to protect the tunnel in the middle of the network, any hosts between the peer
discovery protocol. and the IPSec tunneling device can freely attack the unprotected
Stream.
9 Security considerations 9 Security considerations
Issue: I think we should refer to IPS security considerations. This entire specification is focused on security considerations.
Most of the issues discussed there are relevant for RDDP/RDMA as
well (exceptions are the discussion on user certificates).<TBD>
10 References 10 References
10.1 Normative References 10.1 Normative References
[RFC2828] Shirley, R., "Internet Security Glossary", FYI 36, RFC [RFC2828] Shirley, R., "Internet Security Glossary", FYI 36, RFC
2828, May 2000. 2828, May 2000.
[DDP] Shah, H., J. Pinkerton, R.Recio, and P. Culley, "Direct [DDP] Shah, H., J. Pinkerton, R.Recio, and P. Culley, "Direct
Data Placement over Reliable Transports", Internet-Draft Data Placement over Reliable Transports", Internet-Draft
draft-ietf-rddp-ddp-01.txt, February 2003. draft-ietf-rddp-ddp-01.txt, February 2003.
[RDMAP] Recio, R., P. Culley, D. Garcia, J. Hilland, "An RDMA [RDMAP] Recio, R., P. Culley, D. Garcia, J. Hilland, "An RDMA
Protocol Specification", Internet-Draft draft-ietf-rddp- Protocol Specification", Internet-Draft draft-ietf-rddp-
rdmap-01.txt, February 2003. rdmap-01.txt, February 2003.
[SEC-CONS] Rescorla, E., B. Korver, IAB, "Guidelines for Writing [RFC3723] Aboba B., et al, "Securing Block Storage Protocols over
RFC Text on Security Considerations", Internet-Draft draft- IP", Internet draft (work in progress), RFC3723, April 2004.
ab-sec-cons-03.txt, January 2003.
[RFC2246] T. Dierks, C. Allen, "The TLS Protocol Version 1.0",
RFC 2246, January 1999.
[RFC2401] Atkinson, R. and Kent, S., "Security Architecture for
the Internet Protocol", RFC 2401, November 1998
[RFC2402] Kent, S., Atkinson, R., "IP Authentication Header", RFC
2402, November 1998
[RFC2404] Madson, C., Glenn, R., "The Use of HMAC-SHA-1-96 within
ESP and AH", RFC 2404, November 1998
[RFC2406] Kent, S., Atkinson, R., "IP Encapsulating Security
Payload (ESP)", RFC 2406, November 1998
[RFC2407] Piper, D., "The Internet IP Security Domain of
Interpretation of ISAKMP", RFC 2407, November 1998
[RFC2408] Maughan, D., Schertler, M., Schneider, M., Turner, J.,
"Internet Security Association and Key Management Protocol
(ISAKMP), RFC 2408, November 1998
[RFC2409] Harkins, D., Carrel, D., "The Internet Key Exchange
(IKE)", RFC 2409, November 1998
[AESCTR] Housley, R., "Using AES Counter Mode With IPsec
ESP",Internet draft (work in progress), draft-ietf-ipsec-
ciph-aes-ctr-05.txt, July 2003
[KeyLen] Orman, H., Hoffman, P., "Determining Strengths For
Public Keys Used For Exchanging Symmetric Keys", Internet
draft (work in progress), draft-orman-public-key-lengths-
07.txt, January 2004
[AESXCBC] Frankel, S., Herbert, H., "The AES-XCBC-MAC-96
Algorithm and Its Use with IPsec", Internet draft (work in
progress), draft-ietf-ipsec-ciph-aes-xcbc-mac-02.txt, June
2002
[IPSSEC] Aboba B., et al, "Securing Block Storage Protocols over
IP", Internet draft (work in progress), draft-ietf-ips-
security-19.txt, January 2003
[SCTP] R. Stewart et al., "Stream Control Transmission Protocol", [SCTP] R. Stewart et al., "Stream Control Transmission Protocol",
RFC 2960, October 2000. RFC 2960, October 2000.
[TCP] Postel, J., "Transmission Control Protocol - DARPA Internet [TCP] Postel, J., "Transmission Control Protocol - DARPA Internet
Program Protocol Specification", RFC 793, September 1981. Program Protocol Specification", RFC 793, September 1981.
10.2 Informative References 10.2 Informative References
[IPv6-Trust] Nikander, P., J.Kempf, E. Nordmark, "IPv6 Neighbor [IPv6-Trust] Nikander, P., J.Kempf, E. Nordmark, "IPv6 Neighbor
Discovery trust modelsTrust Models and threats", Internet- Discovery trust modelsTrust Models and threats", Internet-
Draft draft-ietf-send-psreq-01.txt, January 2003. Draft draft-ietf-send-psreq-01.txt, January 2003.
11 Appendix A: Implementing Client/Server Protocols 11 Appendix A: Implementing Client/Server Protocols
<TBD: This section has not been updated to reflect the new
normative focus of this specification. It will be updated in the
next version.>
The prior sections outlined specific attacks and their The prior sections outlined specific attacks and their
countermeasures. This section summarizes the attacks and countermeasures. This section summarizes the attacks and
countermeasures defined in the prior section which are applicable countermeasures defined in the prior section which are applicable
to creation of a secure application server. An application server to creation of a secure application server. An application server
is defined as an application which must be able to communicate is defined as an application which must be able to communicate
with many clients which do not trust each other and ensure that with many clients which do not trust each other and ensure that
each client can not attack another client through server each client can not attack another client through server
interactions. Further, the server may wish to use multiple interactions. Further, the server may wish to use multiple
Streams to communicate with a specific client, and those Streams Streams to communicate with a specific client, and those Streams
may share mutual trust. may share mutual trust.
All of the prior section's details on attacks and countermeasures All of the prior section's details on attacks and countermeasures
to protect a single Stream apply to the server. This section to protect a single Stream apply to the server. This section
focuses on security issues where multiple clients are talking focuses on security issues where multiple clients are talking
with a single server. with a single server, and what mitigations the server application
must have in place to ensure robust operation.
The following list summarizes the relevent attacks that clients The following list summarizes the relevent attacks that clients
can mount on the shared server, by re-stating the previous can mount on the shared server, by re-stating the previous
RECOMMENDations to be client/server specific (the following are normative statements to be client/server specific:
just restatements of the prior RECOMMENDations):
* General Requirements
* Section 4.1 Components on page 9. To ensure Non-
Privileged applications running on the server can not
create a DOS attack on each other, all Non-Privileged
Application interactions with the RNIC Engine that
could affect other applications MUST be done using
the Privileged Resource Manager as a proxy.
* Spoofing * Spoofing
* Section 7.2.4 Using an STag on a Different on page * For protection against many forms of spoofing
23. To ensure that one client can not access another attacks, enable IPSec.
client's data via use of their STag, it is
RECOMMENDED that the server either scope an STag to a * Section 7.2.4 Using an STag on a Different Stream on
single Stream or use a Protection Domain per client, page 23. To ensure that one client can not access
or a combination of the two approaches. another client's data via use of the other client's
STag, the server MUST either scope an STag to a
single Stream or use a Protection Domain per client.
If a single client has multiple streams that share
Partial Mutual Trust, then the STag can be shared
between the associated Streams by using a single
Protection Domain amoung the associated Streams. To
prevent unintended sharing of STags within the
associated Streams, an implementation SHOULD allocate
STags in such a fashion that it is difficult to
predict the next allocated STag number.
* Tampering * Tampering
* 7.3.1 Buffer Overrun - RDMA Write or Read Response on
page 24. To ensure a client can not intentionally or
accidentally cause a buffer overrun, an RNIC
implementation MUST ensure that a Remote Peer is not
able to access memory outside of the buffer specified
when the STag was enabled for remote access.
* 7.3.3 Multiple STags to access the same buffer on * 7.3.3 Multiple STags to access the same buffer on
page 25. See the following bullet's discussion of page 25. See the following bullet's discussion of
Section 7.4.6. Section 7.4.6.
* Information Disclosure * Information Disclosure
* 7.4.2 Using RDMA Read to Access Stale Data on page * 7.4.2 Using RDMA Read to Access Stale Data on page
26. It is RECOMMENDED that the server ensure that no 26. A server SHOULD ensure that no stale data is
stale data is contained in a buffer before remote contained in a buffer before remote read access
read access rights are granted to a client (this can rights are granted to a client (this can be done by
be done by zeroing the contents of the memory, for zeroing the contents of the memory, for example).
example).
* 7.4.5 RDMA Read into an RDMA Write Buffer on page 27. * 7.4.5 RDMA Read into an RDMA Write Buffer on page 27.
It is RECOMMENDED that if a server only intends a It is RECOMMENDED that if a server only intends a
buffer to be exposed for remote write access, it set buffer to be exposed for remote write access, it set
the access rights to the buffer to only enable remote the access rights to the buffer to only enable remote
write access. write access.
* 7.4.6 Using Multiple STags to Access One Buffer on * 7.4.6 Using Multiple STags Which Alias to the Same
page 27. It is RECOMMENDED that separate clients not Buffer on page 27. It is RECOMMENDED that separate
be granted write access to the same buffer through clients not be granted write access to the same
different STags. A buffer should be exposed to only buffer through different STags. A buffer should be
one client at a time to ensure that no information exposed to only one client at a time to ensure that
disclosure or information tampering occurs between no information disclosure or information tampering
peers. occurs between peers.
* Denial of Service * Denial of Service
* 7.5.1 RNIC Resource Consumption on page 29. It is * 7.5.1 RNIC Resource Consumption on page 29. It is
RECOMMENDED that the server place the allocation of RECOMMENDED that the server place the allocation of
all scarce resources be placed under the control of a all scarce resources be placed under the control of a
Privileged Resource Manager. Privileged Resource Manager.
* 7.5.2.1 Multiple Streams Sharing Receive Buffers on * 7.5.2.1 Multiple Streams Sharing Receive Buffers on
page 30. If an RNIC Engine provides the ability to page 30. If an RNIC Engine provides the ability to
skipping to change at page 48, line 43 skipping to change at page 46, line 20
If a server allows the client to influence CQ entry If a server allows the client to influence CQ entry
resource allocation, then it is RECOMMENDED that the resource allocation, then it is RECOMMENDED that the
CQ be isolated to Streams within a single Protection CQ be isolated to Streams within a single Protection
Domain (i.e. streams that share Partial Mutual Domain (i.e. streams that share Partial Mutual
Trust). Trust).
It is RECOMMENDED that the Local Peer implement a It is RECOMMENDED that the Local Peer implement a
mechanism to ensure that the Completion Queue can not mechanism to ensure that the Completion Queue can not
overflow. overflow.
* 7.5.2.4 RDMA Read Request Queue on page 34. It is * 7.5.2.4 Attacking the RDMA Read Request Queue on page
RECOMMENDED that access to interfaces that allocate 35. It is RECOMMENDED that access to interfaces that
RDMA Read Request Queue entries be restricted to a allocate RDMA Read Request Queue entries be
trusted Local Peer, such as a Privileged Resource restricted to a trusted Local Peer, such as a
Manager. Privileged Resource Manager.
It is RECOMMENDED that RDMA Read Request Queue It is RECOMMENDED that RDMA Read Request Queue
resource consumption be controlled such that resource consumption be controlled such that
RDMAP/DDP Streams which do not share Partial Mutual RDMAP/DDP Streams which do not share Partial Mutual
Trust do not share RDMA Read Request Queue resources. Trust do not share RDMA Read Request Queue resources.
* 7.5.3 Resource Consumption by Idle Applications on * 7.5.3 Resource Consumption by Idle Applications on
page 35. Refer to Section 7.5.1. page 36. Refer to Section 7.5.1.
* 7.5.5 RI an STag Shared on Multiple Streams on page * 7.5.5 Remote Invalidate an STag Shared on Multiple
36. If RDDP Streams do not share Partial Mutual Trust Streams on page 37. If DDP/RDMAP Streams do not share
(i.e. the client may attempt to invalidate the STag Partial Mutual Trust (i.e. the client may attempt to
prematurely), it is NOT RECOMMENDED that the server invalidate the STag prematurely), it is NOT
allow an STag to be valid across multiple Streams. RECOMMENDED that the server allow an STag to be valid
across multiple Streams.
12 Appendix B: Summary Table of Attacks 12 Appendix B: Summary Table of Attacks
Issue: Finish Summary table of Attacks/Trust Models <TBD> Below is a summary of implementation requirements for the RNIC:
<editor: This section is under construction, and will be * 7.3.1 Buffer Overrun - RDMA Write or Read Response
completed in a future version of this document>
Rows are the attack (grouped into categories) * 7.4.8 Controlling Access to PTT & STag Mapping
Columns are the: * 7.5.1 RNIC Resource Consumption
* Sec - Section the attack is discussed * 7.5.2.1 Multiple Streams Sharing Receive Buffers
* Attack Name - short name for the attack * 7.5.2.2 Local Peer Attacking a Shared CQ
* Threat - threat type (Spoof (Spoofing), Tamp (Tampering), * 7.5.2.3 Remote Peer Attacking a Shared CQ
ID (Information Disclosure), and DOS (Denial of Service))
* SH ū Does the threat assume there are shared resources * 7.5.2.4 Attacking the RDMA Read Request Queue
(yes/no/NA ū not applicable)?
* TR ū Does the threat assume there is Partial Mutual Trust * 7.5.4 Exercise of non-optimal code paths
between Streams (MT), no trust between Streams (NT), or
is this parameter not applicable (NA)?
12.1 Spoofing * 7.6 Elevation of Privilege
+--------+---------------------------------------------+-----+--+--+ Below is a summary of implementation requirements for the
| Sec | Attack Name |Sh|TR| application above the RNIC:
+--------+---------------------------------------------+-----+--+--+
| 7.2.1 | Impersonation |NA|NA|
+--------+---------------------------------------------+-----+--+--+
| 7.2.2 | Stream Hijacking |NA|NA|
+--------+---------------------------------------------+-----+--+--+
| 7.2.3 | Man in the Middle Attack |NA|NA|
+--------+---------------------------------------------+-----+--+--+
| 7.2.4 | Using an STag on a Different |Y |NT|
+--------+---------------------------------------------+-----+--+--+
12.2 Tampering * 7.2.4 Using an STag on a Different Stream
+--------+---------------------------------------------+-----+--+--+ * 7.3.2 Modifying a Buffer After Indication
| 7.3.1 | Buffer Overrun - RDMA Write or Read Response|NA|NT|
+--------+---------------------------------------------+-----+--+--+
| 7.3.2 | Modifying a Buffer After Indication |NA|NT|
+--------+---------------------------------------------+-----+--+--+
| 7.3.3 | Multiple STags to access the same buffer |Y |NT|
+--------+---------------------------------------------+-----+--+--+
| 7.3.4 | Network based modification of buffer content|NA|NA|
+--------+---------------------------------------------+-----+--+--+
12.3 Information Disclosure * 7.4.2 Using RDMA Read to Access Stale Data
+--------+---------------------------------------------+-----+--+--+ * 7.4.3 Accessing a Buffer After the Transfer
| 7.4.1 | Probing memory outside of the buffer bounds |NA|NT|
+--------+---------------------------------------------+-----+--+--+
| 7.4.2 | Using RDMA Read to Access Stale Data |
+--------+---------------------------------------------+-----+--+--+
| 7.4.3 | Accessing a Buffer After the Transfer |
+--------+---------------------------------------------+-----+--+--+
| 7.4.4 | Accessing Unintended Data With a Valid STag |
+--------+---------------------------------------------+-----+--+--+
| 7.4.5 | RDMA Read into an RDMA Write Buffer |
+--------+---------------------------------------------+-----+--+--+
| 7.4.6 | Using Multiple STags to Access One Buffer |
+--------+---------------------------------------------+-----+--+--+
| 7.4.7 | Remote Node Loading Firmware onto the RNIC |
+--------+---------------------------------------------+-----+--+--+
| 7.4.8 | Controlling Access to PTT & STag Mapping |
+--------+---------------------------------------------+-----+--+--+
| 7.4.9 | Network based eaves dropping |
+--------+---------------------------------------------+-----+--+--+
12.4 Denial of Service * 7.4.4 Accessing Unintended Data With a Valid STag
+--------+---------------------------------------------+-----+--+--+ * 7.4.5 RDMA Read into an RDMA Write Buffer
| 7.5.1 | RNIC Resource Consumption |
+--------+---------------------------------------------+-----+--+--+ * 7.4.6 Using Multiple STags Which Alias to the Same Buffer
| 7.5.2.1| Multiple Streams Sharing Receive Buffers |
+--------+---------------------------------------------+-----+--+--+
| 7.5.2.2| Local Peer Attacking a Shared CQ |
+--------+---------------------------------------------+-----+--+--+
| 7.5.2.3| Remote Peer Attacking a Shared CQ |
+--------+---------------------------------------------+-----+--+--+
| 7.5.2.4| RDMA Read Request Queue |
+--------+---------------------------------------------+-----+--+--+
| 7.5.3 | Resource Consumption by Idle Applications |
+--------+---------------------------------------------+-----+--+--+
| 7.5.4 | Exercise of non-optimal code paths |
+--------+---------------------------------------------+-----+--+--+
| 7.5.5 | RI an STag Shared on Multiple Streams |
+--------+---------------------------------------------+-----+--+--+
Figure 2 - Summary Attacks and Trust Model Table * 7.5.2.2 Local Peer Attacking a Shared CQ
* 7.5.5 Remote Invalidate an STag Shared on Multiple
Streams
13 Appendix C: Partial Trust Taxonomy 13 Appendix C: Partial Trust Taxonomy
Partial Trust is defined as when one party is willing to assume Partial Trust is defined as when one party is willing to assume
that another party will refrain from a specific attack or set of that another party will refrain from a specific attack or set of
attacks, the parties are said to be in a state of Partial Trust. attacks, the parties are said to be in a state of Partial Trust.
Note that the partially trusted peer may attempt a different set Note that the partially trusted peer may attempt a different set
of attacks. This may be appropriate for many applications where of attacks. This may be appropriate for many applications where
any adverse effects of the betrayal is easily confined and does any adverse effects of the betrayal is easily confined and does
not place other clients or applications at risk. not place other clients or applications at risk.
skipping to change at page 53, line 47 skipping to change at page 48, line 47
level is determined based on whether the Local Peer of a level is determined based on whether the Local Peer of a
specific RDMAP/DDP Stream partially trusts the Remote specific RDMAP/DDP Stream partially trusts the Remote
Peer of the Stream (see the definition of Partial Trust Peer of the Stream (see the definition of Partial Trust
in Section 3 Introduction). in Section 3 Introduction).
Not all of the combinations of the trust characteristics are Not all of the combinations of the trust characteristics are
expected to be used by applications. This paper specifically expected to be used by applications. This paper specifically
analyzes five application Trust Models that are expected to be in analyzes five application Trust Models that are expected to be in
common use. The Trust Models are as follows: common use. The Trust Models are as follows:
1. NS-NT - Non-Shared Local Resources, no Local Trust, no Remote * NS-NT - Non-Shared Local Resources, no Local Trust, no
Trust - typically a server application that wants to run in Remote Trust - typically a server application that wants
the safest mode possible. All attack mitigations are in place to run in the safest mode possible. All attack
to ensure robust operation. mitigations are in place to ensure robust operation.
2. NS-RT - Non-Shared Local Resources, no Local Trust, Remote * NS-RT - Non-Shared Local Resources, no Local Trust,
Partial Trust - typically a peer-to-peer application, which Remote Partial Trust - typically a peer-to-peer
has, by some method outside of the scope of this application, which has, by some method outside of the
specification, authenticated the Remote Peer. Note that scope of this specification, authenticated the Remote
unless some form of key based authentication is used on a per Peer. Note that unless some form of key based
RDMA/DDP session basis, it may not be possible be possible authentication is used on a per RDMA/DDP Stream basis, it
for man-in-the-middle attacks to occur. See section 8, may not be possible be possible for man-in-the-middle
Security Services for RDDP on page 37. attacks to occur. See section 8, Security Services for
RDMA and DDP on page 38.
3. S-NT - Shared Local Resources, no Local Trust, no Remote * S-NT - Shared Local Resources, no Local Trust, no Remote
Trust - typically a server application that runs in an Trust - typically a server application that runs in an
untrusted environment where the amount of resources required untrusted environment where the amount of resources
is either too large or too dynamic to dedicate for each required is either too large or too dynamic to dedicate
RDMAP/DDP Stream. for each RDMAP/DDP Stream.
4. S-LT - Shared Local Resources, Local Partial Trust, no Remote * S-LT - Shared Local Resources, Local Partial Trust, no
Trust - typically an application, which provides a session Remote Trust - typically an application, which provides a
layer and uses multiple Streams, to provide additional session layer and uses multiple Streams, to provide
throughput or fail-over capabilities. All of the Streams additional throughput or fail-over capabilities. All of
within the local application partially trust each other, but the Streams within the local application partially trust
do not trust the Remote Peer. This trust model may be each other, but do not trust the Remote Peer. This trust
appropriate for embedded environments. model may be appropriate for embedded environments.
5. S-T - Shared Local Resources, Local Partial Trust, Remote * S-T - Shared Local Resources, Local Partial Trust, Remote
Partial Trust - typically a distributed application, such as Partial Trust - typically a distributed application, such
a distributed database application or a High Performance as a distributed database application or a High
Computer (HPC) application, which is intended to run on a Performance Computer (HPC) application, which is intended
cluster. Due to extreme resource and performance to run on a cluster. Due to extreme resource and
requirements, the application typically authenticates with performance requirements, the application typically
all of its peers and then runs in a highly trusted authenticates with all of its peers and then runs in a
environment. The application peers are all in a single highly trusted environment. The application peers are all
application fault domain and depend on one another to be in a single application fault domain and depend on one
well-behaved when accessing data structures. If a trusted another to be well-behaved when accessing data
Remote Peer has an implementation defect that results in poor structures. If a trusted Remote Peer has an
behavior, the entire application could be corrupted. implementation defect that results in poor behavior, the
entire application could be corrupted.
Models NS-NT and S-NT above are typical for Internet networking - Models NS-NT and S-NT above are typical for Internet networking -
neither Local Peers nor the Remote Peer is trusted. Sometimes neither Local Peers nor the Remote Peer is trusted. Sometimes
optimizations can be done that enable sharing of Page Translation optimizations can be done that enable sharing of Page Translation
Tables across multiple Local Peers, thus Model S-LT can be Tables across multiple Local Peers, thus Model S-LT can be
advantageous. Model S-T is typically used when resource scaling advantageous. Model S-T is typically used when resource scaling
across a large parallel application makes it infeasible to use across a large parallel application makes it infeasible to use
any other model. Resource scaling issues can either be due to any other model. Resource scaling issues can either be due to
performance around scaling or because there simply are not enough performance around scaling or because there simply are not enough
resources. Model NS-RT is probably the least likely model to be resources. Model NS-RT is probably the least likely model to be
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/