Internet Draft                            James Pinkerton
draft-ietf-rddp-security-07.txt
draft-ietf-rddp-security-08.txt             Microsoft Corporation
Category: Standards Track                 Ellen Deleganes
Expires: October, 2005 September, 2006                    Intel Corporation
                                          Sara Bitan
                                            Microsoft Corporation
                                          April 2005
                                          March 2006

                           DDP/RDMAP Security

Status of this Memo
   By submitting this Internet-Draft, each author represents that
   any applicable patent or other IPR claims of which he or she is
   aware have been or will be disclosed, and any of which he or she
   becomes aware will be disclosed, in accordance with Section 6 of
   BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time. It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

Abstract
   This document analyzes security issues around implementation and
   use of the Direct Data Placement Protocol(DDP) and Remote Direct
   Memory Access Protocol (RDMAP). It first defines an architectural
   model for an RDMA Network Interface Card (RNIC), which can
   implement DDP or RDMAP and DDP. The document reviews various
   attacks against the resources defined in the architectural model
   and the countermeasures that can be used to protect the system.
   Attacks are grouped into spoofing, tampering, information
   disclosure, denial of service, and elevation of privilege.
   Finally, the document concludes with a summary of security
   services for DDP and RDMAP, such as IPsec.

   J. Pinkerton, et al.     Expires October, 2005 September, 2006                 1
   Table of Contents

   1    Introduction.................................................4
   2    Architectural Model..........................................6
   2.1  Components...................................................7
   2.2  Resources....................................................8  Resources....................................................9
   2.2.1  Stream Context Memory......................................8 Memory......................................9
   2.2.2  Data Buffers...............................................8 Buffers..............................................10
   2.2.3  Page Translation Tables....................................9 Tables...................................10
   2.2.4  STag Namespace.............................................9  Protection Domain (PD)....................................11
   2.2.5  Completion Queues..........................................9  STag Namespace and Scope..................................11
   2.2.6  Completion Queues.........................................12
   2.2.7  Asynchronous Event Queue..................................10
   2.2.7 Queue..................................12
   2.2.8  RDMA Read Request Queue...................................10
   2.2.8 Queue...................................13
   2.3  RNIC Interactions.........................................10
   2.2.8.1 Interactions...........................................13
   2.3.1  Privileged Control Interface Semantics.................10
   2.2.8.2 Semantics....................13
   2.3.2  Non-Privileged Data Interface Semantics................11
   2.2.8.3 Semantics...................13
   2.3.3  Privileged Data Interface Semantics....................11
   2.2.9 Semantics.......................14
   2.3.4  Initialization of RNIC Data Structures for Data Transfer..11
   2.2.10 Transfer..14
   2.3.5  RNIC Data Transfer Interactions..........................13 Interactions...........................16
   3    Trust and Resource Sharing..................................14 Sharing..................................17
   4    Attacker Capabilities.......................................15 Capabilities.......................................18
   5    Attacks and Countermeasures.................................16 That Can be Mitigated With End-to-End Security......19
   5.1  Tools for Countermeasures...................................16  Spoofing....................................................19
   5.1.1  Protection Domain (PD)....................................16  Impersonation.............................................19
   5.1.2  Limiting STag Scope.......................................17  Stream Hijacking..........................................20
   5.1.3  Access Rights.............................................18
   5.1.4  Limiting the Scope  Man-in-the-Middle Attack..................................20
   5.2  Tampering - Network based modification of the Completion Queue................18
   5.1.5  Limiting the Scope buffer content....21
   5.3  Information Disclosure - Network Based Eavesdropping........21
   5.4  Specific Requirements for Security Services.................21
   5.4.1  Introduction to Security Options..........................22
   5.4.2  TLS is Inappropriate for DDP/RDMAP Security...............22
   5.4.3  ULPs Which Provide Security...............................23
   5.4.4  Requirements for IPsec Encapsulation of an Error............................18
   5.2  Spoofing....................................................19
   5.2.1  Impersonation.............................................19
   5.2.2  Stream Hijacking..........................................19
   5.2.3  Man in the Middle Attack..................................20
   5.2.4 DDP...............23
   6    Attacks from Remote Peers...................................25
   6.1  Spoofing....................................................25
   6.1.1  Using an STag on a Different Stream.......................20
   5.3  Tampering...................................................21
   5.3.1 Stream.......................25
   6.2  Tampering...................................................26
   6.2.1  Buffer Overrun - RDMA Write or Read Response..............22
   5.3.2 Response..............27
   6.2.2  Modifying a Buffer After Indication.......................22
   5.3.3 Indication.......................27
   6.2.3  Multiple STags to access the same buffer..................23
   5.3.4  Network based modification of buffer content..............23
   5.4 buffer..................28
   6.3  Information Disclosure......................................23
   5.4.1 Disclosure......................................28
   6.3.1  Probing memory outside of the buffer bounds...............23
   5.4.2 bounds...............28
   6.3.2  Using RDMA Read to Access Stale Data......................23
   5.4.3 Data......................28
   6.3.3  Accessing a Buffer After the Transfer.....................24
   5.4.4 Transfer.....................29
   6.3.4  Accessing Unintended Data With a Valid STag...............24
   5.4.5 STag...............29
   6.3.5  RDMA Read into an RDMA Write Buffer.......................24
   5.4.6 Buffer.......................29
   6.3.6  Using Multiple STags Which Alias to the Same Buffer.......25
   5.4.7  Remote Node Loading Firmware onto the RNIC................25
   5.4.8 Buffer.......30
   6.3.7  Controlling Access to PTT & STag Mapping..................25
   5.4.9  Network based eavesdropping...............................26
   5.5 Mapping..................30
   6.4  Denial of Service (DOS).....................................26
   5.5.1 (DOS).....................................31
   6.4.1  RNIC Resource Consumption.................................26
   5.5.2 Consumption.................................31
   6.4.2  Resource Consumption by Idle ULPs.........................32
   6.4.3  Resource Consumption By Active ULPs.......................27
   5.5.2.1 ULPs.......................32
   6.4.3.1   Multiple Streams Sharing Receive Buffers...............27
   5.5.2.2   Local ULP Attacking a Shared CQ........................29
   5.5.2.3   Local or Buffers...............33
   6.4.3.2   Remote or Local Peer Attacking a Shared CQ.............29
   5.5.2.4 CQ.............34
   6.4.3.3   Attacking the RDMA Read Request Queue..................32
   5.5.3  Resource Consumption by Idle ULPs.........................33
   5.5.4 Queue..................37
   6.4.4  Exercise of non-optimal code paths........................34
   5.5.5 paths........................37
   6.4.5  Remote Invalidate an STag Shared on Multiple Streams......34
   5.5.6 Streams......38
   6.4.6  Remote Peer attacking an Unshared CQ......................34
   5.6 CQ......................38
   6.5  Elevation of Privilege......................................35
   6    Security Services for RDMAP and DDP.........................36
   6.1  Introduction to Security Options............................36
   6.1.1  Introduction to IPsec.....................................36
   6.1.2  Introduction to SSL Limitations on RDMAP..................38
   6.1.3  ULPs Which Provide Security...............................38
   6.2  Requirements for IPsec Encapsulation of DDP.................39 Privilege......................................38
   7    Security considerations.....................................40    Attacks from Local Peers....................................40
   7.1  Local ULP Attacking a Shared CQ.............................40
   7.2  Local Peer Attacking the RDMA Read Request Queue............40
   8    IANA Considerations.........................................41    Security considerations.....................................41
   9    References..................................................42
   9.1    IANA Considerations.........................................42
   10   References..................................................43
   10.1   Normative References........................................42
   9.2 References......................................43
   10.2   Informative References......................................42
   10 References....................................43
   11   Appendix A: ULP Issues for RDDP Client/Server Protocols.....43
   11 Protocols.....45
   12   Appendix B: Summary of RNIC and ULP Implementation
   Requirements.....................................................47
   12
   Requirements.....................................................49
   13   Appendix C: Partial Trust Taxonomy..........................49
   13   Author's Addresses..........................................51 Taxonomy..........................51
   14   Acknowledgments.............................................52   Author's Addresses..........................................53
   15   Acknowledgments.............................................54
   16   Full Copyright Statement....................................53 Statement....................................55

   Table of Figures

   Figure 1 - RDMA Security Model....................................7

1  Introduction

   RDMA enables new levels of flexibility when communicating between
   two parties compared to current conventional networking practice
   (e.g. a stream-based model or datagram model). This flexibility
   brings new security issues that must be carefully understood when
   designing Upper Layer Protocols (ULPs) utilizing RDMA and when
   implementing RDMA-aware NICs (RNICs). Note that for the purposes
   of this security analysis, an RNIC may implement RDMAP and DDP,
   or just DDP. Also, a ULP may be an application or it may be a
   middleware library.

   The document first develops an architectural model that is
   relevant for the security analysis - it details components,
   resources, and system properties that may be attacked in Section
   2. The document uses Local Peer to represent the RDMA/DDP
   protocol implementation on the local end of a Stream. The local
   Upper-Layer-Protocol (ULP) is used to represent the application
   or middle-ware layer above the Local Peer. The document does not
   attempt to differentiate between a Remote Peer and a Remote ULP
   (an RDMA/DDP protocol implementation on the remote end of a
   Stream versus the application on the remote end) for several
   reasons: often the source of the attack is difficult to know for
   sure; and regardless of the source, the mitigations required of
   the Local Peer or local ULP are the same. Thus the document
   generically refers to a Remote Peer rather than trying to further
   delineate the attacker.

   The document then defines what resources a local ULP may share
   across Streams and what resources the local ULP may share with
   the Remote Peer across Streams in Section 3.

   Intentional sharing of resources between multiple Streams may
   imply some level of trust between the Streams. However, some
   types of resource sharing have unmitigated security attacks which
   would mandate not sharing a specific type of resource unless
   there is some level of trust between the Streams sharing
   resources.

   This document defines a new term, "Partial Mutual Trust" to
   address this concept:

        Partial Mutual Trust - a collection of RDMAP/DDP Streams,
        which represent the local and remote end points of the
        Stream, which are willing to assume that the Streams from
        the collection will not perform malicious attacks against
        any of the other Streams in the collection.

   ULPs have explicit control of which collection of endpoints is in
   a Partial Mutual Trust collection through tools discussed in
   Section 5.1 Tools for Countermeasures on page 16. 13 Appendix C: Partial Trust Taxonomy.

   An untrusted peer relationship is appropriate when a ULP wishes
   to ensure that it will be robust and uncompromised even in the
   face of a deliberate attack by its peer. For example, a single
   ULP that concurrently supports multiple unrelated Streams (e.g. a
   server) would presumably treat each of its peers as an untrusted
   peer. For a collection of Streams which share Partial Mutual
   Trust, the assumption is that any Stream not in the collection is
   untrusted. For the untrusted peer, a brief list of capabilities
   is enumerated in Section 4.

   The rest of the document is focused on analyzing attacks and
   recommending specific mitigations to the attacks. First, the
   tools for mitigating attacks Attacks are listed (Section 5.1),
   categorized into attacks mitigated by LLP mechanisms, attacks
   initiated by Remote Peers, and then a
   series of attacks on components, resources, or system properties
   is listed in the rest of Section 5. initiated by Local Peers.
   For each attack, possible countermeasures are reviewed.

   ULPs within a host are divided into two categories - Privileged
   and Non-Privileged. Both ULP types can send and receive data and
   request resources. The key differences between the two are:

        The Privileged ULP is trusted by the local system to not
        maliciously attack the operating environment, but it is not
        trusted to optimize resource allocation globally. For
        example, the Privileged ULP could be a kernel ULP, thus the
        kernel presumably has in some way vetted the ULP before
        allowing it to execute.

        A Non-Privileged ULP's capabilities are a logical sub-set of
        the Privileged ULP's. It is assumed by the local system that
        a Non-Privileged ULP is untrusted. All Non-Privileged ULP
        interactions with the RNIC Engine that could affect other
        ULPs need to be done through a trusted intermediary that can
        verify the Non-Privileged ULP requests.

   If all recommended mitigations are in place the implemented usage
   models, the RDMAP/DDP protocol can be shown to not expose any new
   security vulnerabilities.

2  Architectural Model

   This section describes an RDMA architectural reference model that
   is used as security issues are examined. It introduces the
   components of the model, the resources that can be attacked, the
   types of interactions possible between components and resources,
   and the system properties which must be preserved.

   Figure 1 shows the components comprising the architecture and the
   interfaces where potential security attacks could be launched.
   External attacks can be injected into the system from a ULP that
   sits above the RNIC Interface or from the network.

   The intent here is to describe high level components and
   capabilities which affect threat analysis, and not focus on
   specific implementation options. Also note that the architectural
   model is an abstraction, and an actual implementation may choose
   to subdivide its components along different boundary lines than
   defined here. For example, the Privileged Resource Manager may be
   partially or completely encapsulated in the Privileged ULP.
   Regardless, it is expected that the security analysis of the
   potential threats and countermeasures still apply.

   Note that the model below is derived from several specific RDMA
   implementations. A few of note is [VERBS-RDMAC], [VERBS-RDMAC-
   Overview], and [INFINIBAND].

          +-------------+
          |  Privileged |
          |  Resource   |
 Admin<-+>|  Manager    |     ULP Control Interface
        | |             |<------+-------------------+
        | +-------------+       |                   |
        |       ^               v                   v
        |       |         +-------------+   +-----------------+
        |---------------->| Privileged  |   |  Non-Privileged |
                |         | ULP         |   |  ULP            |
                |         +-------------+   +-----------------+
                |               ^                   ^
                |Privileged     |Privileged         |Non-Privileged
                |Control        |Data               |Data
                |Interface      |Interface          |Interface
RNIC            |               |                   |
Interface       v               v                   v
=================================================================

              +--------------------------------------+
              |                                      |
              |               RNIC Engine            | <-- Firmware
              |                                      |
              +--------------------------------------+
                                ^
                                |
                                v
                             Internet

                     Figure 1 - RDMA Security Model

2.1  Components

   The components shown in Figure 1 - RDMA Security Model are:

       *   RDMA Network Interface Controller Engine (RNIC) - the
           component that implements the RDMA protocol and/or DDP
           protocol.

       *   Privileged Resource Manager - the component responsible
           for managing and allocating resources associated with the
           RNIC Engine. The Resource Manager does not send or
           receive data. Note that whether the Resource Manager is
           an independent component, part of the RNIC, or part of
           the ULP is implementation dependent.

       *   Privileged ULP - See Section 1 Introduction for a
           definition of Privileged ULP. The local host
           infrastructure can enable the Privileged ULP to map a
           data buffer directly from the RNIC Engine to the host
           through the RNIC Interface, but it does not allow the
           Privileged ULP to directly consume RNIC Engine resources.

       *   Non-Privileged ULP - See Section 1 Introduction for a
           definition of Non-Privileged ULP.

   A design goal of the DDP and RDMAP protocols is to allow, under
   constrained conditions, Non-Privileged ULP to send and receive
   data directly to/from the RDMA Engine without Privileged Resource
   Manager intervention - while ensuring that the host remains
   secure. Thus, one of the primary goals of this document is to
   analyze this usage model for the enforcement that is required in
   the RNIC Engine to ensure the system remains secure.

   The host interfaces that could be exercised include:

       *   Privileged Control Interface - A Privileged Resource
           Manager uses

   DDP supports both untagged and tagged buffers.  Tagged buffers
   allow the RNIC Interface Data Sink ULP to be indifferent to what order (or in
   what messages) the Data Source sent the data, or what order
   packets are received in.  Typically tagged data can be used for
   payload transfer, while untagged is best used for control
   messages.  However each upper layer protocol can determine the
   optimal use of tagged and untagged messages for itself.  This
   document will discuss when Data Source flexibility is of benefit
   to applications

   Note that in DDP there are two mechanisms for transferring data.
   These two data transfer mechanisms are also enabled through
   RDMAP, with additional control semantics:

       *   Untagged data transfer - the Data Source payload simply
           consumes the first buffer in a queue of buffers that are
           in Data Sink specified order (commonly referred to as the
           Receive Queue), and

       *   Tagged data transfer - the Data Source explicitly states
           which destination buffer is targeted, through use of an
           STag. STag based transfers allow the Data Sink ULP to be
           indifferent to what order (or in what messages) the Data
           Source sent the data, or what order packets are received
           in.

   For DDP the two forms correspond to Untagged and Tagged DDP
   Messages, respectively. For RDMAP the two forms correspond to
   Send Type Messages and RDMA Messages (either RDMA Read or RDMA
   Write Messages), respectively. Typically tagged data can be used
   for payload transfer, while untagged is best used for control
   messages.  However each upper layer protocol can determine the
   optimal use of tagged and untagged messages for itself. See
   [APPLICABILITY] for more information on application applicability
   for the two transfer mechanisms.

   The host interfaces that could be exercised include:

       *   Privileged Control Interface - A Privileged Resource
           Manager uses the RNIC Interface to allocate and manage
           RNIC Engine resources, control the state within the RNIC
           Engine, and monitor various events from the RNIC Engine.
           It also uses this interface to act as a proxy for some
           operations that a Non-Privileged ULP may require (after
           performing appropriate countermeasures).

       *   ULP Control Interface - An ULP uses this interface to the
           Privileged Resource Manager to allocate RNIC Engine
           resources. The Privileged Resource Manager implements
           countermeasures to ensure that if the Non-Privileged ULP
           launches an attack it can prevent the attack from
           affecting other ULPs.

       *   Non-Privileged Data Transfer Interface - A Non-Privileged
           ULP uses this interface to initiate and to check the
           status of data transfer operations.

       *   Privileged Data Transfer Interface - A superset of the
           functionality provided by the Non-Privileged Data
           Transfer Interface. The ULP is allowed to directly
           manipulate RNIC Engine mapping resources to map an STag
           to a ULP data buffer.

       *   Figure 1 also shows the ability to load new firmware in
           the RNIC Engine. Not all RNICs will support this, but it
           is shown for completeness and is also reviewed under
           potential attacks.

   If Internet control messages, such as ICMP, ARP, RIPv4, etc. are
   processed by the RNIC Engine, the threat analyses for those
   protocols is also applicable, but outside the scope of this
   document.

2.2  Resources

   This section describes the primary resources in the RNIC Engine
   that could be affected if under attack. For RDMAP, all of the
   defined resources apply. For DDP, all of the resources except the
   RDMA Read Queue apply.

2.2.1  Stream Context Memory

   The state information for each Stream is maintained in memory,
   which could be located in a number of places - on the NIC, inside
   RAM attached to the NIC, in host memory, or in any combination of
   the three, depending on the implementation.

   Stream Context Memory includes state associated with Data
   Buffers. For Tagged Buffers, this includes how STag names, Data
   Buffers, and Page Translation Tables (see section 2.2.3 on page
   9) 2.2.3)
   interrelate. It also includes the list of Untagged Data Buffers
   posted for reception of Untagged Messages (commonly called the
   Receive Queue), and a list of operations to perform to send data
   (commonly called the Send Queue).

2.2.2  Data Buffers

   There

   As mentioned previously, there are two different ways to expose a
   local ULP's data buffer; buffers for data transfer; Untagged Data
   Transfer - a buffer can be exposed for receiving RDMAP Send Type
   Messages (a.k.a. DDP Untagged Messages) on DDP Queue zero - or
   Tagged Data Transfer - the buffer can be exposed for remote
   access through STags (a.k.a. DDP Tagged Messages). This
   distinction is important because the attacks and the
   countermeasures used to protect against the attack are different
   depending on the method for exposing the buffer to the network.

   For the purposes of the security discussion, for Tagged Data
   Transfer a single logical Data Buffer is exposed with a single
   Stag on a given Stream. Actual implementations may support
   scatter/gather capabilities to enable multiple physical data
   buffers to be accessed with a single STag, but from a threat
   analysis perspective it is assumed that a single STag enables
   access to a single logical Data Buffer.

   In any event, it is the responsibility of the Privileged Resource
   Manager to ensure that no STag can be created that exposes memory
   that the consumer had no authority to expose.

   A data buffer has specific access rights. The local ULP can
   control whether a data buffer is exposed for local only, or local
   and remote access, and assign specific access privileges (read,
   write, read and write) on a per Stream basis.

   For DDP, when an STag is advertised, the Remote Peer is
   presumably given write access rights to the data (otherwise there
   was not much point to the advertisement). For RDMAP, when a ULP
   advertises an STag, it can enable write-only, read-only, or both
   write and read access rights.

   Similarly, some ULPs may wish to provide a single buffer with
   different access rights on a per-Stream basis. For example, some
   Streams may have read-only access, some may have remote read and
   write access, while on other Streams only the local ULP/Local
   Peer is allowed access.

2.2.3  Page Translation Tables

   Page Translation Tables are the structures used by the RNIC to be
   able to access ULP memory for data transfer operations. Even
   though these structures are called "Page" Translation Tables,
   they may not reference a page at all - conceptually they are used
   to map a ULP address space representation (e.g. a virtual
   address) of a buffer to the physical addresses that are used by
   the RNIC Engine to move data. If on a specific system a mapping
   is not used, then a subset of the attacks examined may be
   appropriate. Note that the Page Translation Table may or may not
   be a shared resource.

2.2.4  STag Namespace

   The DDP specification defines  Protection Domain (PD)

   A Protection Domain (PD) is a 32-bit namespace for local construct to the STag.
   Implementations may vary in terms RDMA
   implementation, and never visible over the wire. Protection
   Domains are assigned to two of the actual number resources of concern, Stream
   Context Memory and STags
   that are supported. In any case, this is associated with Page Translation Table
   entries and data buffers. A correct implementation of a bounded resource
   Protection Domain requires that
   can come under attack. Depending upon STag namespace allocation
   algorithms, the actual name space resources which belong to attack may a given
   Protection Domain can not be significantly
   less than 2^32.

2.2.5  Completion Queues

   Completion Queues are used in this document on a resource belonging to conceptually
   represent how
   another Protection Domain, because Protection Domain membership
   is checked by the RNIC Engine notifies the ULP about the
   completion of the transmission of data, or the completion of the
   reception of data through the Data Transfer Interface. Because
   there could be many transmissions or receptions in flight at prior to taking any
   one time, completions are modeled as a queue rather than action involving such
   a single
   event. An implementation may also use the Completion Queue resource. Protection Domains are therefore used to
   notify the ULP of other activities, for example, the completion
   of a mapping of ensure that
   an STag can only be used to a access an associated data buffer on
   one or more Streams that are associated with the same Protection
   Domain as the specific ULP buffer. Completion
   Queues may be shared by a group of STag.

   If an implementation chooses to not share resources between
   Streams, or may it is recommended that each Stream be designated associated with
   its own, unique Protection Domain. If an implementation chooses
   to handle a specific Stream's traffic.

   Some implementations may allow this queue to be manipulated
   directly by both Non-Privileged and Privileged ULPs.

2.2.6  Asynchronous Event Queue

   The Asynchronous Event Queue resource sharing, it is a queue from the RNIC recommended that Protection
   Domain be limited to the
   Privileged Resource Manager collection of bounded size. It is used by the
   RNIC to notify the host of various events which might require
   management action, including protocol violations, Stream state
   changes, local operation errors, low water marks on receive
   queues, and possibly other events.

   The Asynchronous Event Queue is a resource Streams that can be attacked
   because Remote have Partial
   Mutual Trust with each other.

   Note that a ULP (either Privileged or Local Peers and/or ULPs Non-Privileged) can cause events to
   occur which
   potentially have the potential multiple Protection Domains. This could be used,
   for example, to ensure that multiple clients of overflowing a server do not
   have the queue.

   Note that an implementation is at liberty ability to implement corrupt each other. The server would allocate
   a Protection Domain per client to ensure that resources covered
   by the
   functions of Protection Domain could not be used by another (untrusted)
   client.

2.2.5  STag Namespace and Scope

   The DDP specification defines a 32-bit namespace for the Asynchronous Event Queue STag.
   Implementations may vary in a variety of ways,
   including multiple queues or even simple callbacks. All
   vulnerabilities identified are intended to apply regardless terms of the implementation actual number of the Asynchronous Event Queue. For example, STags
   that are supported. In any case, this is a callback function bounded resource that
   can come under attack. Depending upon STag namespace allocation
   algorithms, the actual name space to attack may be viewed as simply a very short queue.

2.2.7  RDMA Read Request Queue significantly
   less than 2^32.

   The RDMA Read Request Queue scope of an STag is the memory that holds state
   information for one or more RDMA Read Request Messages that have
   arrived, but for set of DDP/RDMAP Streams on which the RDMA Read Response Messages have not
   yet been completely sent. Because potentially more than one RDMA
   Read Request can be outstanding at one time, the memory
   STag is
   modeled as a queue of bounded size. Some implementations may
   enable sharing of a single RDMA Read Request Queue across
   multiple Streams.

2.2.8  RNIC Interactions

   With RNIC resources and interfaces defined, it valid. If an STag is now possible to
   examine the interactions supported by valid on a particular DDP/RDMAP
   Stream, then that stream can modify the generic RNIC functional
   interfaces through each of buffer, subject to the 3 interfaces - Privileged Control
   Interface, Privileged Data Interface, and Non-Privileged Data
   Interface.

2.2.8.1  Privileged Control Interface Semantics

   Generically,
   access rights that the Privileged Control Interface controls stream has for the RNIC's
   allocation, deallocation, and initialization of RNIC global
   resources. This includes allocation and deallocation of Stream
   Context Memory, Page Translation Tables, STag names, Completion
   Queues, RDMA Read Request Queues, and Asynchronous Event Queues.

   The Privileged Control Interface is also typically used for
   managing Non-Privileged ULP resources (see Section 2.2.2
   Data Buffers for the Non-Privileged ULP
   (and possibly additional information).

   The analysis presented in this document assumes two mechanisms
   for limiting the Privileged ULP as well). This includes
   initialization and removal scope of Page Translation Table resources,
   and managing RNIC events (possibly managing all events Streams for which the
   Asynchronous Event Queue).

2.2.8.2  Non-Privileged Data Interface Semantics STag is valid:

           *   Protection Domain scope. The Non-Privileged Data Interface enables data transfer (transmit STag is valid if used on
               any Stream within a specific Protection Domain, and receive) but does
               is invalid if used on any Stream that is not allow initialization a member
               of the Page
   Translation Table resources. However, once the Page Translation
   Table resources have been initialized, the interface may enable a
   specific Protection Domain.

           *   Single Stream scope. The STag mapping to be enabled and disabled by directly
   communicating with is valid on a single
               Stream, regardless of what the RNIC, or create an STag mapping for Stream association is
               to a
   buffer that has been previously initialized Protection Domain. If used on any other Stream,
               it is invalid.

2.2.6  Completion Queues

   Completion Queues (CQ) are used in this document to conceptually
   represent how the RNIC Engine notifies the RNIC.

   For RDMAP, ULP data can be sent by using RDMAP Send Type
   Messages, RDMA Read Responses, and RDMA Writes. ULP data about the
   completion of the transmission of data, or the completion of the
   reception of data through RDMAP the Data Transfer Interface
   (specifically for Untagged Data Transfer - Tagged Data Transfer
   can not cause a completion to occur). Because there could be done by receiving Send Type
   Messages into buffers that have been posted on the Receive Queue many
   transmissions or Shared Receive Queue. It can receptions in flight at any one time,
   completions are modeled as a queue rather than a single event. An
   implementation may also be done by receiving RDMA
   Write and RDMA Read Response Messages into buffers that have
   previously been exposed use the Completion Queue to notify the
   ULP of other activities, for external write access through
   advertisement example, the completion of a mapping
   of an STag. Additionally, STag to cause a specific ULP data to buffer. Completion Queues may be
   pulled (read) across the network, RDMAP uses an RDMA Read Request
   Message (which only contains RDMAP control information necessary
   to access the ULP buffer to
   shared by a group of Streams, or may be read), to cause an RDMA Read
   Response Message designated to be generated that contains the ULP data.

   For DDP, transmitting data means sending DDP Tagged or Untagged
   Messages. For data reception, for DDP it can receive Untagged
   Messages into buffers that have been posted on the Receive handle a
   specific Stream's traffic. Limiting Completion Queue association
   to one, or Shared Receive Queue. It can also receive Tagged DDP Messages
   into buffers that have previously been exposed for external write
   access through advertisement a small number of an STag.

   Completion RDMAP/DDP Streams can prevent
   several forms of data transmission or reception generally entails
   informing attacks by sharply limiting the ULP scope of the completed work
   attack's effect.

   Some implementations may allow this queue to be manipulated
   directly by placing completion
   information on the Completion Queue.

2.2.8.3 both Non-Privileged and Privileged Data Interface Semantics ULPs.

2.2.7  Asynchronous Event Queue

   The Privileged Data Interface semantics are Asynchronous Event Queue is a superset of the
   Non-Privileged Data Transfer semantics. The interface can do
   everything defined in the prior section, as well as
   create/destroy buffer to STag mappings directly. This generally
   entails initialization or clearing of Page Translation Table
   state in queue from the RNIC.

2.2.9  Initialization of RNIC Data Structures for Data Transfer

   Initialization of the mapping between an STag and a Data Buffer
   can be viewed in to the abstract as two separate operations:

       a.  Initialization
   Privileged Resource Manager of bounded size. It is used by the allocated Page Translation Table
           entries with
   RNIC to notify the location host of the Data Buffer, various events which might require
   management action, including protocol violations, Stream state
   changes, local operation errors, low water marks on receive
   queues, and

       b.  Initialization of possibly other events.

   The Asynchronous Event Queue is a mapping from an allocated STag name resource that can be attacked
   because Remote or Local Peers and/or ULPs can cause events to a set
   occur which have the potential of Page Translation Table entry(s) or partial-
           entries. overflowing the queue.

   Note that an implementation may not have a Page Translation Table
   (i.e. it may support a direct mapping between an STag and a Data
   Buffer). In this case threats and mitigations associated with is at liberty to implement the
   Page Translation Table
   functions of the Asynchronous Event Queue in a variety of ways,
   including multiple queues or even simple callbacks. All
   vulnerabilities identified are not relevant.

   Initialization intended to apply regardless of
   the contents implementation of the Page Translation Table can Asynchronous Event Queue. For example,
   a callback function may be done by either the Privileged ULP or by the Privileged
   Resource Manager viewed as simply a proxy for the Non-Privileged ULP. By
   definition the Non-Privileged ULP very short queue.

2.2.8  RDMA Read Request Queue

   The RDMA Read Request Queue is not trusted to directly
   manipulate the Page Translation Table. In general the concern is memory that the Non-Privileged ULP may try to maliciously initialize the
   Page Translation Table to access a buffer holds state
   information for which it does not one or more RDMA Read Request Messages that have permission.

   The exact resource allocation algorithm
   arrived, but for which the Page Translation
   Table is outside the scope of this document. It may be allocated
   for a specific Data Buffer, or be allocated as a pooled resource
   to be consumed by RDMA Read Response Messages have not
   yet been completely sent. Because potentially multiple Data Buffers, or more than one RDMA
   Read Request can be
   managed in some other way. This document attempts to abstract
   implementation dependent issues, and group them into higher level
   security issues such outstanding at one time, the memory is
   modeled as resource starvation and a queue of bounded size. Some implementations may
   enable sharing of
   resources between a single RDMA Read Request Queue across
   multiple Streams.

   The next issue is how an STag name

2.3  RNIC Interactions

   With RNIC resources and interfaces defined, it is associated with a Data
   Buffer. For now possible to
   examine the case interactions supported by the generic RNIC functional
   interfaces through each of an Untagged the 3 interfaces - Privileged Control
   Interface, Privileged Data Buffer, there is no wire
   visible mapping between an STag Interface, and the Non-Privileged Data Buffer. Note that
   there may,
   Interface. As mentioned previously in fact, be an STag which represents the buffer, if an
   implementation chooses Section 2.1 Components,
   there are two data transfer mechanisms to internally represent Untagged Data
   Buffer using STags. However, because the STag by definition is
   not visible on the wire, this is a local host implementation
   specific issue which should be analyzed in the context of a local
   host implementation specific security analysis, examined - Untagged
   data transfer and thus is
   outside the scope of this document.

   For a Tagged Data Buffer, either the data transfer.

2.3.1  Privileged ULP, Control Interface Semantics

   Generically, the Non- Privileged ULP, or Control Interface controls the Privileged Resource Manager acting on
   behalf RNIC's
   allocation, deallocation, and initialization of the Non-Privileged ULP may initialize a mapping from an
   STag to a RNIC global
   resources. This includes allocation and deallocation of Stream
   Context Memory, Page Translation Table, or may have the ability to
   simply enable/disable an existing Tables, STag to Page Translation Table
   mapping. There may names, Completion
   Queues, RDMA Read Request Queues, and Asynchronous Event Queues.

   The Privileged Control Interface is also be multiple STag names which map to a
   specific group typically used for
   managing Non-Privileged ULP resources for the Non-Privileged ULP
   (and possibly for the Privileged ULP as well). This includes
   initialization and removal of Page Translation Table entries (or sub-
   entries). Specific security issues with this level of flexibility
   are examined in Section 5.3.3 Multiple STags to access the same
   buffer on page 23.

   There are a variety of implementation options resources,
   and managing RNIC events (possibly managing all events for the
   Asynchronous Event Queue).

2.3.2  Non-Privileged Data Interface Semantics

   The Non-Privileged Data Interface enables data transfer (transmit
   and receive) but does not allow initialization of the Page
   Translation Table entries and mapping an STag to a group
   of resources. However, once the Page Translation
   Table entries which resources have security
   repercussions. This includes support for separation of Mapping an been initialized, the interface may enable a
   specific STag versus mapping a set of Page Translation Table entries, to be enabled and
   support for ULPs disabled by directly manipulating
   communicating with the RNIC, or create an STag to Page Translation
   Table entry mappings (versus requiring access through mapping for a
   buffer that has been previously initialized in the
   Privileged Resource Manager).

2.2.10 RNIC Data Transfer Interactions

   RNIC Data Transfer operations can be subdivided into send
   operations and receive operations. RNIC.

   For send operations, there is typically a queue that enables the RDMAP, ULP to post multiple operation requests to send data (referred to
   as the Send Queue). Depending upon the implementation, Data
   Buffers used in the operations may or may not have Page
   Translation Table entries associated with them, and may or may
   not have STags associated with them. Because this is a local host
   specific implementation issue rather than a protocol issue, the
   security analysis can be sent by one of threats and mitigations is left to the host
   implementation.

   Receive operations are different for Tagged Data Buffers versus previously
   described data transfer mechanisms - Untagged data transfer or
   Tagged Data Buffers. If more than Transfer. Three RDMAP data transfer mechanisms are
   defined, one using Untagged Data Buffer data transfer (Send Type Messages),
   and one using Tagged data transfer (RDMA Read Responses and RDMA
   Writes). ULP data reception through RDMAP can be posted done by the ULP, the DDP specification requires
   receiving Send Type Messages into buffers that they be
   consumed in sequential order. Thus have been posted
   on the most general
   implementation is that there is Receive Queue or Shared Receive Queue. Thus a sequential queue of receive Receive
   Queue or Shared Receive Queue can only be affected by Untagged
   data transfer. Data Buffers (Receive Queue). Some implementations may reception can also support sharing of the sequential queue between multiple
   Streams. In this case defining "sequential" becomes non-trivial -
   in general the be done by receiving RDMA
   Write and RDMA Read Response Messages into buffers for a single Stream are consumed from the
   queue in the order that they were placed on the queue, but there
   is no consumption order guarantee between Streams.

   For receive have
   previously been exposed for external write access through
   advertisement of an STag (i.e. Tagged Data Buffers, at some time prior data transfer).
   Additionally, to cause ULP data
   transfer, the mapping of the STag to specific Page Translation
   Table entries (if present) and be pulled (read) across the mapping from
   network, RDMAP uses an RDMA Read Request Message (which only
   contains RDMAP control information necessary to access the Page
   Translation Table entries ULP
   buffer to be read), to cause an RDMA Read Response Message to be
   generated that contains the Data Buffer must ULP data.

   For DDP, transmitting data means sending DDP Tagged or Untagged
   Messages. For data reception, for DDP it can receive Untagged
   Messages into buffers that have been
   initialized (see section 2.2.9 for interaction details).

3  Trust and Resource Sharing posted on the Receive Queue
   or Shared Receive Queue. It is assumed can also receive Tagged DDP Messages
   into buffers that in general have previously been exposed for external write
   access through advertisement of an STag.

   Completion of data transmission or reception generally entails
   informing the Local and Remote Peer are
   untrusted, and thus attacks ULP of the completed work by either should have mitigations placing completion
   information on the Completion Queue. For data reception, only an
   Untagged data transfer can cause completion information to be put
   in
   place.

   A separate, but related issue is resource sharing between
   multiple Streams. If local resources are not shared, the
   resources Completion Queue.

2.3.3  Privileged Data Interface Semantics

   The Privileged Data Interface semantics are dedicated on a per Stream basis. Resources are
   defined in Section 2.2 Resources on page 8. The advantage superset of not
   sharing resources between Streams is that it reduces the types of
   attacks that are possible.
   Non-Privileged Data Transfer semantics. The disadvantage of not sharing
   resources is that ULPs might run out of resources. Thus there interface can
   be a strong incentive for sharing resources, if do
   everything defined in the security
   issues associated with prior section, as well as
   create/destroy buffer to STag mappings directly. This generally
   entails initialization or clearing of Page Translation Table
   state in the sharing RNIC.

2.3.4  Initialization of resources RNIC Data Structures for Data Transfer

   Initialization of the mapping between an STag and a Data Buffer
   can be mitigated.

   It is assumed viewed in this document that the component that implements abstract as two separate operations:

       a.  Initialization of the mechanism allocated Page Translation Table
           entries with the location of the Data Buffer, and

       b.  Initialization of a mapping from an allocated STag name
           to control sharing a set of Page Translation Table entry(s) or partial-
           entries.

   Note that an implementation may not have a Page Translation Table
   (i.e. it may support a direct mapping between an STag and a Data
   Buffer). In this case threats and mitigations associated with the RNIC Engine resources is
   Page Translation Table are not relevant.

   Initialization of the Privileged Resource Manager. The RNIC Engine exposes its
   resources through contents of the RNIC Interface to Page Translation Table can
   be done by either the Privileged Resource
   Manager. All Privileged and Non-Privileged ULPs request resources
   from ULP or by the Privileged
   Resource Manager (note that by definition both as a proxy for the Non-
   Privileged and Non-Privileged ULP. By
   definition the Privileged application might Non-Privileged ULP is not trusted to directly
   manipulate the Page Translation Table. In general the concern is
   that the Non-Privileged ULP may try to greedily
   consume resources, thus creating a potential Denial of Service
   (DOS) attack). The Resource Manager implements resource
   management policies maliciously initialize the
   Page Translation Table to ensure fair access to resources. a buffer for which it does not
   have permission.

   The
   Resource Manager should be designed to take into account security
   attacks detailed in this document. Note that exact resource allocation algorithm for some systems the
   Privileged Resource Manager Page Translation
   Table is outside the scope of this document. It may be implemented within the
   Privileged ULP.

   All Non-Privileged ULP interactions with the RNIC Engine that
   could affect other ULPs MUST allocated
   for a specific Data Buffer, or be done using the Privileged
   Resource Manager allocated as a proxy. All ULP pooled resource allocation requests
   for scarce resources MUST also
   to be done using a Privileged
   Resource Manager.

   The sharing of resources across Streams should consumed by potentially multiple Data Buffers, or be under the
   control of the ULP, both
   managed in terms of the trust model the ULP
   wishes some other way. This document attempts to operate under, as well as the abstract
   implementation dependent issues, and group them into higher level of
   security issues such as resource sharing
   the ULP wishes to give local processes. For more discussion on
   types of trust models which combine partial trust starvation and sharing of
   resources, see Appendix C: Partial Trust Taxonomy on page 49.
   resources between Streams.

   The Privileged Resource Manager MUST NOT assume different Streams
   share Partial Mutual Trust unless there next issue is a mechanism to ensure
   that the Streams do indeed share Partial Mutual Trust. This can
   be done in several ways, including explicit notification from the
   ULP that owns the Streams.

4  Attacker Capabilities

   An attacker's capabilities delimit how an STag name is associated with a Data
   Buffer. For the types case of attacks that
   attacker an Untagged Data Buffer, there is able to launch. RDMAP no wire
   visible mapping between an STag and DDP require that the
   initial LLP Stream (and connection) Data Buffer. Note that
   there may, in fact, be set up prior to
   transferring RDMAP/DDP Messages. This requires at least one
   round-trip handshake an STag which represents the buffer, if an
   implementation chooses to occur.

   If internally represent Untagged Data
   Buffer using STags. However, because the attacker STag by definition is
   not visible on the Remote Peer that created the initial
   connection, then the attacker's capabilities can wire, this is a local host implementation
   specific issue which should be segmented
   into send only capabilities or send and receive capabilities.
   Attacking with send only capabilities requires the attacker to
   first guess analyzed in the current LLP Stream parameters before they can
   attack RNIC resources (e.g. TCP sequence number). If this class context of attacker also has receive capabilities, they are typically
   referred to as a "man-in-the-middle" attacker, local
   host implementation specific security analysis, and they have a
   much wider ability to attack RNIC resources. The breadth of
   attack thus is essentially
   outside the same as that scope of an attacking Remote
   Peer (i.e. this document.

   For a Tagged Data Buffer, either the Remote Peer that setup Privileged ULP or the initial LLP Stream).

5  Attacks and Countermeasures

   This section describes the attacks that are possible against the
   RDMA system defined in Figure 1 - RDMA Security Model and
   Privileged Resource Manager acting on behalf of the
   RNIC Engine resources defined in Section 2.2. The analysis
   includes Non-
   Privileged ULP may initialize a detailed description of each attack, what is being
   attacked, and mapping from an STag to a description of Page
   Translation Table, or may have the countermeasures that can be
   taken ability to thwart the attack.

   Note that connection setup and teardown is presumed simply
   enable/disable an existing STag to Page Translation Table
   mapping. There may also be done in
   stream mode (i.e. no RDMA encapsulation multiple STag names which map to a
   specific group of the payload), so there Page Translation Table entries (or sub-
   entries). Specific security issues with this level of flexibility
   are no new attacks related to connection setup/teardown beyond
   what is already present examined in Section 6.2.3 Multiple STags to access the LLP (e.g. TCP or SCTP). Note,
   however, that RDMAP/DDP parameters may be exchanged in stream
   mode, and if they same
   buffer.

   There are corrupted by an attacker unintended
   consequences will result. Therefore, any existing mitigations for
   LLP Spoofing, Tampering, Repudiation, Information Disclosure,
   Denial a variety of Service, or Elevation implementation options for initialization
   of Privilege continue Page Translation Table entries and mapping an STag to apply
   (and are out a group
   of scope Page Translation Table entries which have security
   repercussions. This includes support for separation of this document). Thus the analysis in
   this section focuses on attacks that are present regardless Mapping an
   STag versus mapping a set of
   the LLP Stream type.

   The attacks are classified into five categories: Spoofing,
   Tampering, Information Disclosure, Denial of Service (DoS)
   attacks, Page Translation Table entries, and Elevation of Privileges. Tampering is any
   modification of
   support for ULPs directly manipulating STag to Page Translation
   Table entry mappings (versus requiring access through the legitimate traffic (machine internal or
   network). Spoofing attack
   Privileged Resource Manager).

2.3.5  RNIC Data Transfer Interactions

   RNIC Data Transfer operations can be subdivided into send
   operations and receive operations.

   For send operations, there is typically a special case of tampering where
   the attacker falsifies an identity of queue that enables the Remote Peer (identity
   can be an IP address, machine name,
   ULP level identity etc.).

5.1  Tools for Countermeasures

   The tools described in this section are the primary mechanisms
   that can be used to provide countermeasures post multiple operation requests to potential attacks.

5.1.1  Protection Domain (PD)

   A Protection Domain (PD) is a local construct send data (referred to
   as the RDMA
   implementation, and never visible over Send Queue). Depending upon the wire. Protection
   Domains are assigned to two of implementation, Data
   Buffers used in the resources of concern, Stream
   Context Memory and STags associated with operations may or may not have Page
   Translation Table entries associated with them, and data buffers. A correct implementation of a
   Protection Domain requires that resources which belong to a given
   Protection Domain can may or may
   not be used on a resource belonging to
   another Protection Domain, because Protection Domain membership have STags associated with them. Because this is checked by a local host
   specific implementation issue rather than a protocol issue, the RNIC prior
   security analysis of threats and mitigations is left to taking any action involving such
   a resource. Protection Domains the host
   implementation.

   Receive operations are therefore used to ensure that
   an STag can only be used to access an associated different for Tagged Data Buffers versus
   Untagged Data Buffers (i.e. Tagged data buffer on
   one or transfer vs. Untagged
   data transfer). For Untagged data transfer, if more Streams that are associated with than one
   Untagged Data Buffer can be posted by the same Protection
   Domain as ULP, the specific STag.

   If an implementation chooses to not share resources between
   Streams, it is recommended DDP
   specification requires that each Stream they be associated with
   its own, unique Protection Domain. If an consumed in sequential order
   (the RDMAP specification also requires this). Thus the most
   general implementation chooses
   to allow resource sharing, it is recommended that Protection
   Domain be limited to the collection of Streams that have Partial
   Mutual Trust with each other.

   Note that there is a ULP (either Privileged or Non-Privileged) can
   potentially have sequential queue of
   receive Untagged Data Buffers (Receive Queue). Some
   implementations may also support sharing of the sequential queue
   between multiple Protection Domains. This could be used, Streams. In this case defining "sequential"
   becomes non-trivial - in general the buffers for example, to ensure that multiple clients of a server do not
   have single Stream
   are consumed from the ability to corrupt each other. The server would allocate
   a Protection Domain per client to ensure queue in the order that resources covered
   by they were placed on
   the Protection Domain could not be used by another (untrusted)
   client.

5.1.2  Limiting STag Scope

   The key to protecting a local data buffer queue, but there is no consumption order guarantee between
   Streams.

   For receive Tagged data transfer (i.e. Tagged Data Buffers, RDMA
   Write Buffers, or RDMA Read Buffers), at some time prior to limit the scope
   of its STag to the level appropriate for the Streams which share
   Partial Mutual Trust. The scope of data
   transfer, the STag can be measured in
   multiple ways.

       *   Number mapping of Connections and/or Streams on which the STag is
           valid. One way to limit specific Page Translation
   Table entries (if present) and the scope of mapping from the STag is Page
   Translation Table entries to limit the connections and/or Streams Data Buffer must have been
   initialized (see section 2.3.4 for interaction details).

3  Trust and Resource Sharing

   It is assumed that in general the Local and Remote Peer are allowed to use
   untrusted, and thus attacks by either should have mitigations in
   place.

   A separate, but related issue is resource sharing between
   multiple Streams. If local resources are not shared, the STag. As noted
   resources are dedicated on a per Stream basis. Resources are
   defined in Section 2.2 Resources. The advantage of not sharing
   resources between Streams is that it reduces the previous section, use types of
           Protection Domains appropriately attacks
   that are possible. The disadvantage of not sharing resources is
   that ULPs might run out of resources. Thus there can limit be a strong
   incentive for sharing resources, if the scope of security issues
   associated with the STag. The analysis presented sharing of resources can be mitigated.

   It is assumed in this document assumes
           two mechanisms for limiting that the scope component that implements
   the mechanism to control sharing of Streams for
           which the STag RNIC Engine resources is valid:

           *   Protection Domain scope.
   the Privileged Resource Manager. The STag is valid if used on
               any Stream within a specific Protection Domain, RNIC Engine exposes its
   resources through the RNIC Interface to the Privileged Resource
   Manager. All Privileged and
               is invalid if used on any Stream Non-Privileged ULPs request resources
   from the Resource Manager (note that is not a member
               of by definition both the Protection Domain.

           *   Single Stream scope. The STag is valid on a single
               Stream, regardless of what Non-
   Privileged and the Stream association is Privileged application might try to greedily
   consume resources, thus creating a Protection Domain. If used on any other Stream,
               it is invalid.

       *   Limit the time an STag is valid. By Invalidating an
           advertised STag (e.g., revoking remote potential Denial of Service
   (DOS) attack). The Resource Manager implements resource
   management policies to ensure fair access to resources. The
   Resource Manager should be designed to take into account security
   attacks detailed in this document. Note that for some systems the
           buffers described by an STag when done
   Privileged Resource Manager may be implemented within the
   Privileged ULP.

   All Non-Privileged ULP interactions with the
           transfer), an entire class of attacks can RNIC Engine that
   could affect other ULPs MUST be eliminated.

       *   Limit done using the buffer Privileged
   Resource Manager as a proxy. All ULP resource allocation requests
   for scarce resources MUST also be done using a Privileged
   Resource Manager.

   The sharing of resources across Streams should be under the STag can reference. Limiting
   control of the
           scope ULP, both in terms of an STag access the trust model the ULP
   wishes to just operate under, as well as the intended portion level of resource sharing
   the ULP buffers to be exposed is critical wishes to prevent
           certain forms of attacks.

       *   Allocating and/or advertising STag numbers in an
           unpredictable way. If STags are allocated/advertised
           using an algorithm which makes it hard for the attacker
           to guess which STag(s) are currently in use, it makes it give local processes. For more difficult for an attacker discussion on
   types of trust models which combine partial trust and sharing of
   resources, see Appendix C: Partial Trust Taxonomy.

   The Privileged Resource Manager MUST NOT assume different Streams
   share Partial Mutual Trust unless there is a mechanism to guess ensure
   that the correct
           value. As stated Streams do indeed share Partial Mutual Trust. This can
   be done in several ways, including explicit notification from the RDMAP specification [RDMAP], an
           invalid STag will cause
   ULP that owns the Streams.

4  Attacker Capabilities

   An attacker's capabilities delimit the types of attacks that
   attacker is able to launch. RDMAP and DDP require that the
   initial LLP Stream to (and connection) be
           terminated. For the case of [DDP], set up prior to
   transferring RDMAP/DDP Messages. This requires at a minimum it must
           signal an error least one
   round-trip handshake to occur.

   If the ULP. This permits attacker is not the ULP to
           detect such attempts, and take countermeasures. Commonly, Remote Peer that created the ULP will cause initial
   connection, then the DDP Stream to attacker's capabilities can be immediately
           terminated.

5.1.3  Access Rights

   Access Rights associated with a specific advertised STag segmented
   into send only capabilities or
   RDMAP/DDP Stream provide another mechanism for ULPs send and receive capabilities.
   Attacking with send only capabilities requires the attacker to limit
   first guess the current LLP Stream parameters before they can
   attack capabilities RNIC resources (e.g. TCP sequence number). If this class
   of attacker also has receive capabilities and the Remote Peer. The local ULP can control
   whether a data buffer is exposed for local only, or local and
   remote access, and assign specific access privileges (read,
   write, read and write) on a per Stream basis.

   For DDP, when an STag is advertised, the Remote Peer is
   presumably given write access rights ability to pose
   as the data (otherwise there
   was not much point receiver to the advertisement). For RDMAP, when a ULP
   advertises an STag, it can enable write-only, read-only, or both
   write sender and read access rights.

   Similarly, some ULPs may wish the sender to provide the receiver,
   they are typically referred to as a single buffer with
   different access rights on "man-in-the-middle" attacker
   [RFC3552]. A man-in-the-middle attacker has a per-Stream basis. For example, some
   Streams may have read-only access, some may have remote read and
   write access, while on other Streams only the local ULP/Local
   Peer much wider ability
   to attack RNIC resources. The breadth of attack is allowed access.

5.1.4  Limiting essentially
   the Scope same as that of an attacking Remote Peer (i.e. the Completion Queue

   Completions associated with sending and receiving data, or
   setting up buffers for sending and receiving data, could Remote
   Peer that setup the initial LLP Stream).

5  Attacks That Can be
   accumulated in a shared Completion Queue for a group of RDMAP/DDP
   Streams, or a specific Mitigated With End-to-End Security

   This section describes the RDMAP/DDP Stream could have a dedicated
   Completion Queue. Limiting Completion Queue association attacks where the only
   solution is to one,
   or implement some form of end-to-end security. The
   analysis includes a small number detailed description of RDMAP/DDP Streams each attack, what is
   being attacked, and a description of the countermeasures that can prevent several
   be taken to thwart the attack.

   Some forms of Denial of Service attacks, by sharply limiting attack involve modifying the scope of RDMAP or DDP payload
   by a network based attacker or involve monitoring the attack's effect.

5.1.5  Limiting traffic to
   discover private information. An effective tool to ensure
   confidentiality is to encrypt the Scope data stream through mechanisms
   such as IPsec encryption. An effective tool to ensure the remote
   entity is who they claim to be as well as ensuring that the
   payload is unmodified as it traverses the network is the use of an Error

   To prevent a variety
   authentication protocols such as IPSec Authentication.

   Note that connection setup and teardown is presumed to be done in
   stream mode (i.e. no RDMA encapsulation of attacks, it the payload), so there
   are no new attacks related to connection setup/teardown beyond
   what is important already present in the LLP (e.g. TCP or SCTP). Note,
   however, that an RDMAP/DDP implementation parameters may be robust exchanged in the face of errors. If stream
   mode, and if they are corrupted by an
   error on a specific Stream can cause other unrelated Streams attacker unintended
   consequences will result. Therefore, any existing mitigations for
   LLP Spoofing, Tampering, Repudiation, Information Disclosure,
   Denial of Service, or Elevation of Privilege continue to
   fail, then a broad class apply
   (and are out of scope of this document). Thus the analysis in
   this section focuses on attacks that are enabled against present regardless of
   the
   implementation.

   For example, an error on a specific RDMAP LLP Stream should not cause type.

   Tampering is any modification of the RNIC to stop processing incoming packets, legitimate traffic (machine
   internal or corrupt network). Spoofing attack is a
   receive queue for special case of
   tampering where the attacker falsifies an unrelated Stream.

5.2  Spoofing

   Spoofing attacks can be launched by identity of the Remote Peer, or by a
   network based attacker. A
   Peer (identity can be an IP address, machine name, ULP level
   identity etc.).

5.1  Spoofing

   Spoofing attacks can be launched by the Remote Peer, or by a
   network based attacker. A network based spoofing attack applies
   to all Remote Peers.

   Because the RDMAP Stream requires an LLP Stream to be fully
   initialized (e.g. for [TCP] it is in This section analyzes the ESTABLISHED state),
   certain various types of traditional forms of wire
   spoofing attacks do not apply -
   - an end-to-end handshake must have occurred applicable to establish the RDMAP Stream. So, the only form of spoofing that applies is one
   when an attacker can both send and receive packets. Yet even with
   this limitation the Stream is still exposed to the following
   spoofing attacks.

5.2.1 & DDP.

5.1.1  Impersonation

   A network based attacker can impersonate a legal RDMAP/DDP Peer
   (by spoofing a legal IP address), and establish address). This can either be done as a
   blind attack (see [RFC3552]) or by establishing an RDMAP/DDP
   Stream with the victim. End-to-end authentication (i.e. IPsec,
   SSL or ULP authentication) provides Because an RDMAP/DDP Stream requires an
   LLP Stream to be fully initialized (e.g. for [TCP] it is in the
   ESTABLISHED state), existing transport layer protection
   mechanisms against this
   attack. blind attacks remain in place.

   For additional information see Section 6, Security
   Services for RDMAP and DDP, on page 36.

5.2.2  Stream Hijacking

   Stream hijacking happens when a network based attacker eavesdrops
   the LLP connection through blind attack to succeed, it requires the Stream establishment phase, and
   waits until attacker to inject
   a valid transport layer segment (e.g. for TCP it must match at
   least the authentication phase (if such 4-tuple as well as guess a phase exists) is
   completed successfully. The attacker then spoofs sequence number within the IP address
   and re-direct
   window) while also guessing valid RDMAP or DDP parameters.  There
   are many ways to attack the Stream from RDMAP/DDP protocol if the victim transport
   protocol is assumed to its own machine. be vulnerable. For example, for Tagged
   Messages, this entails guessing the STag and TO values. If the
   attacker wishes to simply terminate the connection, it can do so
   by correctly guessing the transport & network layer values, and
   providing an invalid STag. Per the DDP specification, if an
   invalid STag is received, the Stream is torn down and the Remote
   Peer is notified with an error. If an attacker wishes to
   overwrite an Advertised Buffer, it must successfully guess the
   correct STag and TO. Given that the TO often will start at zero,
   this is straightforward. The value of the STag should be chosen
   at random, as discussed in Section 6.1.1 Using an STag on a
   Different Stream. For Untagged Messages, if the MSN is invalid
   then the connection may be torn down. If it is valid, then the
   receive buffers can be corrupted.

   End-to-end authentication (e.g. IPsec or ULP authentication)
   provides protection against either the blind attack or the
   connected attack.

5.1.2  Stream Hijacking

   Stream hijacking happens when a network based attacker eavesdrops
   the LLP connection through the Stream establishment phase, and
   waits until the authentication phase (if such a phase exists) is
   completed successfully. The attacker then spoofs the IP address
   and re-directs the Stream from the victim to its own machine. For
   example, an attacker can wait until an iSCSI authentication is
   completed successfully, and then hijack the iSCSI Stream.

   The best protection against this form of attack is end-to-end
   integrity protection and authentication, such as IPsec (see
   Section 6, Security Services for RDMAP and DDP, on page 36), IPsec, to
   prevent spoofing. Another option is to provide physical a physically
   segregated network for security. Discussion of physical security
   is out of scope for this document.

   Because the connection and/or Stream itself is established by the
   LLP, some LLPs are more difficult to hijack than others. Please
   see the relevant LLP documentation on security issues around
   connection and/or Stream hijacking.

5.2.3  Man in the Middle

5.1.3  Man-in-the-Middle Attack

   If a network based attacker has the ability to delete, inject
   replay, delete or modify
   packets which will still be accepted by the LLP (e.g., TCP
   sequence number is correct) then the Stream can be exposed to a man in the middle
   man-in-the-middle attack. One style of attack is for the man-in-the-middle man-in-
   the-middle to send Tagged Messages (either RDMAP or DDP). If it
   can discover a buffer that has been exposed for STag enabled
   access, then the man-in-the-middle can use an RDMA Read operation
   to read the contents of the associated data buffer, perform an
   RDMA Write Operation to modify the contents of the associated
   data buffer, or invalidate the STag to disable further access to
   the buffer.

   The best protection against this form of attack is end-to-end
   integrity protection and authentication, such as IPsec (see
   Section 6 Security Services for RDMAP and DDP on page 36), IPsec, to
   prevent spoofing or tampering. If Stream or session level authentication and integrity protection
   protections are not used, then physical protection must be employed, lest a
   employed to prevent man-in-the-middle
   attack occur, enabling spoofing and tampering. attacks.

   Because the connection/Stream itself is established by the LLP,
   some LLPs are more exposed to man-in-the-middle attack than
   others. Please see the relevant LLP documentation on security
   issues around connection and/or Stream hijacking.

   Another approach is to restrict access to only the local
   subnet/link, and provide some mechanism to limit access, such as
   physical security or 802.1.x. This model is an extremely limited
   deployment scenario, and will not be further examined here.

5.2.4  Using an STag on a Different Stream

   One style of attack from the Remote Peer is for it to attempt to
   use STag values that it

5.2  Tampering - Network based modification of buffer content

   This is not authorized to use. Note that if actually a man in the Remote Peer sends an invalid STag to middle attack - but only on the Local Peer, per
   content of the
   DDP and RDMAP specifications, buffer, as opposed to the Stream must be torn down. Thus man in the threat exists if an STag has been enabled for Remote Access
   on one Stream middle attack
   presented above, where both the signaling and a Remote Peer content can be
   modified. See Section 5.1.3 Man-in-the-Middle Attack.

5.3  Information Disclosure - Network Based Eavesdropping

   An attacker that is able to use it eavesdrop on an unrelated
   Stream. If the attack is successful, the attacker could
   potentially be able to perform either RDMA Read Operations to network can read the contents
   content of the associated data buffer, perform RDMA
   Write Operations all read and write accesses to modify the contents of a Peer's buffers. To
   prevent information disclosure, the associated read/written data
   buffer, or to invalidate the STag to disable further access to must be
   encrypted. See also Section 5.1.3 Man-in-the-Middle Attack. The
   encryption can be done either by the buffer.

   An attempt ULP, or by a Remote Peer protocol that
   can provide security services to access RDMAP & DDP (e.g. IPsec).

5.4  Specific Requirements for Security Services

   Generally speaking, Stream confidentiality protects against
   eavesdropping. Stream and/or session authentication and integrity
   protection is a buffer with an STag on counter measurement against various spoofing and
   tampering attacks. The effectiveness of authentication and
   integrity against a
   different Stream in specific attack depend on whether the same Protection Domain may or may not be
   an attack depending on whether resource sharing
   authentication is intended (i.e.
   whether the Streams shared Partial Mutual Trust machine level authentication (such as IPsec),
   or not). For some
   ULPs using an STag on multiple Streams within the same Protection
   Domain could be desired behavior. For other ULPs attempting to
   use an STag on a different Stream could be considered to be an
   attack. Since this varies by ULP, a ULP typically would need to
   be able authentication.

5.4.1  Introduction to control the scope of the STag.

   In the case where an implementation does not share resources
   between Streams (including STags), this attack Security Options

   The following security services can be defeated by
   assigning each Stream to a different Protection Domain. Before
   allowing remote access applied to an RDMAP/DDP
   Stream:

   1.  Session confidentiality - protects against eavesdropping
       (section 5.3).

   2.  Per-packet data source authentication - protects against the buffer, the Protection Domain of
   the
       following spoofing attacks: network based impersonation
       (section 5.1.1), Stream where hijacking (section 5.1.2), and man in
       the access attempt was made is matched middle (section 5.1.3).

   3.  Per-packet integrity - protects against
   the Protection Domain tampering done by
       network based modification of the STag. If the Protection Domains do
   not match, access to the buffer content (section 5.2)

   4.  Packet sequencing - protects against replay attacks, which is denied, an error is generated,
   and the RDMAP Stream associated with
       a special case of the attacking above tampering attack.

   If an RDMAP/DDP Stream is
   terminated.

   For implementations that share resources between multiple
   Streams, it may not be practical subject to separate each impersonation attacks,
   or Stream into its
   own Protection Domain. In this case, the ULP can still limit the
   scope of any of the STags to a single Stream (if hijacking attacks, it is enabling
   it for remote access). If the STag scope has been limited to a
   single Stream, any attempt to use recommended that STag on a different Stream
   will result in an error, and the RDMAP Stream is terminated.

   Thus for implementations that do not share STags between Streams,
   each Stream MUST either be in a separate Protection Domain or the
   scope of an STag MUST be limited
   authenticated, integrity protected, and protected from replay
   attacks; it may use confidentiality protection to a single Stream.

   An RNIC MUST ensure that a specific protect from
   eavesdropping (in case the RDMAP/DDP Stream in traverses a specific
   Protection Domain can not access an STag in public
   network).

   IPsec is a different
   Protection Domain.

   An RNIC MUST ensure that if an STag protocol suite which is limited in scope used to a
   single Stream, no other Stream can use the STag.

   An additional issue may be unintended sharing of STags (i.e. a
   bug in secure communication
   at the ULP) or a bug in network layer between two peers. The IPsec protocol suite
   is specified within the Remote Peer which causes an off-
   by-one STag to be used. For additional protection, an
   implementation should allocate STags in such a fashion that it IP Security Architecture [RFC2401], IKE
   [RFC2409], IPsec Authentication Header (AH) [RFC2402] and IPsec
   Encapsulating Security Payload (ESP) [RFC2406] documents. IKE is
   difficult to predict
   the next allocated STag number, key management protocol while AH and also
   ensure that STags ESP are reused at as slow a rate as possible. Any
   allocation method which would lead used to intentional or
   unintentional reuse protect
   IP traffic. Please see those RFCs for a complete description of an STag by
   the peer should be avoided
   (e.g. a method which always starts with a given STag and
   monotonically increases it respective protocols.

   IPsec is capable of providing the above security services for each new allocation, or a method
   which always uses IP
   and TCP traffic respectively. ULP protocols are able to provide
   only part of the same STag above security services.

5.4.2  TLS is Inappropriate for each operation).

5.3  Tampering

   A Remote Peer DDP/RDMAP Security

   TLS [RFC 2246] provide Stream authentication, integrity and
   confidentiality for TCP based ULPs. TLS supports one-way (server
   only) or a network mutual certificates based attacker can attempt to tamper
   with the contents of data buffers on a Local Peer authentication.

   There are at least two limitations that have been
   enabled make TLS underneath RDMAP
   inappropriate for remote write access. DDP/RDMA security:

   1.  The types of tampering attacks
   that are possible are outlined in the sections that follow.

5.3.1  Buffer Overrun - RDMA Write or Read Response

   This attack is an attempt maximum length supported by the Remote Peer to perform an RDMA
   Write or RDMA Read Response to memory outside of TLS record layer protocol
       is 2^14 bytes - longer packets must be fragmented (as a
       comparison, the valid maximum length
   range of the data buffer enabled an Untagged DDP Message is
       roughly 2^32).

   2.  TLS is a connection oriented protocol. If a stream cipher or
       block cipher in CBC mode is used for remote write access. This
   attack bulk encryption, then a
       packet can occur even when no resources are shared across
   Streams. This issue be decrypted only after all the packets preceding
       it have already arrived. If TLS is used to protect DDP/RDMAP
       traffic, then TLS must gather all out-of-order packets before
       RDMAP/DDP can also arise if place them into the ULP has a bug.

   The countermeasure for this type buffer. Thus one of attack must the
       primary features of DDP/RDMAP - enabling implementations to
       have a flow-through architecture with little to no buffering,
       can not be in achieved if TLS is used to protect the RNIC
   implementation, leveraging data
       stream.

   If TLS is layered on top of RDMAP or DDP, TLS does not protect
   the STag. When RDMAP and/or DDP headers. Thus a man-in-the-middle attack can
   still occur by modifying the local ULP specifies RDMAP/DDP header to incorrectly
   place the RNIC the base address and data into the number of bytes in wrong buffer, thus effectively corrupting
   the
   buffer that data stream.

   For these reasons, it wishes to make accessible, the RNIC must ensure is NOT RECOMMENDED that the base and bounds check are applied TLS be layered on
   top of RDMAP or DDP.

5.4.3  ULPs Which Provide Security

   ULPs which provide integrated security but wish to any access leverage
   lower-layer protocol security should be aware of security
   concerns around correlating a specific channel's security
   mechanisms to the
   buffer referenced authentication performed by the STag before the STag is enabled ULP. See
   [NFSv4CHANNEL] for
   access. When an RDMA data transfer operation (which includes an
   STag) arrives additional information on a Stream, a base and bounds byte granularity
   access check must be performed promising approach
   called "channel binding". From [NFSv4CHANNEL]:

        "The concept of channel bindings allows applications to ensure
        prove that the operation accesses
   only memory locations within end-points of two secure channels at
        different network layers are the buffer described same by binding
        authentication at one channel to the session protection at
        the other channel.  The use of channel bindings allows
        applications to delegate session protection to lower layers,
        which may significantly improve performance for some
        applications."

5.4.4  Requirements for IPsec Encapsulation of DDP

   The IP Storage working group has spent significant time and
   effort to define the normative IPsec requirements for IP Storage
   [RFC3723]. Portions of that STag.

   Thus specification are applicable to a
   wide variety of protocols, including the RDDP protocol suite. In
   order to not replicate this effort, an RNIC implementation MUST ensure
   follow the requirements defined in RFC3723 Section 2.3 and
   Section 5, including the associated normative references for
   those sections. Note that a Remote Peer this means that support for IPSEC ESP
   mode is not normative.

   Additionally, since IPsec acceleration hardware may only be able
   to access memory outside handle a limited number of the buffer specified when the
   STag was enabled active IKE Phase 2 SAs, Phase 2
   delete messages may be sent for remote access.

5.3.2  Modifying a Buffer After Indication

   This attack can occur if idle SAs, as a Remote Peer attempts to modify means of keeping
   the
   contents number of active Phase 2 SAs to a minimum. The receipt of an STag referenced buffer by performing an RDMA Write
   or
   IKE Phase 2 delete message MUST NOT be interpreted as a reason
   for tearing down an RDMA Read Response after the Remote Peer has indicated DDP/RDMA Stream. Rather, it is preferable to
   leave the Local Peer or local ULP (by a variety of means) that Stream up, and if additional traffic is sent on it, to
   bring up another IKE Phase 2 SA to protect it. This avoids the STag
   data buffer contents are ready
   potential for use. This attack can occur
   even when no resources are shared across Streams. continually bringing Streams up and down.

   Note that a bug
   in a Remote Peer, or network based tampering, could also result
   in this problem. there are serious security issues if IPsec is not
   implemented end-to-end. For example, assume the STag referenced buffer contains ULP
   control information as well if IPsec is implemented as ULP payload, and a
   tunnel in the ULP sequence middle of operation is to first validate the control information network, any hosts between the Peer
   and
   then perform operations on the control information. If IPsec tunneling device can freely attack the unprotected
   Stream.

6  Attacks from Remote
   Peer can perform an additional Peers

   This section describes remote attacks that are possible against
   the RDMA Write or system defined in Figure 1 - RDMA Read Response
   (thus changing the buffer) after the validity checks have been
   completed but before the control data is operated on, the Remote
   Peer could force Security Model and the ULP down operational paths that were never
   intended.
   RNIC Engine resources defined in Section 2.2. The local ULP can protect itself from this type analysis
   includes a detailed description of attack by
   revoking remote access when the original data transfer has
   completed each attack, what is being
   attacked, and before it validates the contents a description of the buffer. The
   local ULP countermeasures that can either do this by explicitly revoking remote access
   rights for be
   taken to thwart the STag when attack.

   The attacks are classified into five categories: Spoofing,
   Tampering, Information Disclosure, Denial of Service (DoS)
   attacks, and Elevation of Privileges. As mentioned previously,
   tampering is any modification of the legitimate traffic (machine
   internal or network). A spoofing attack is a special case of
   tampering where the attacker falsifies an identity of the Remote
   Peer indicates (identity can be an IP address, machine name, ULP level
   identity etc.).

6.1  Spoofing

   This section analyzes the operation
   has completed, various types of spoofing attacks
   applicable to RDMAP & DDP. Spoofing attacks can be launched by
   the Remote Peer, or by checking to make sure a network based attacker. For
   countermeasures against a network based attacker, see Section 5
   Attacks That Can be Mitigated With End-to-End Security.

6.1.1  Using an STag on a Different Stream

   One style of attack from the Remote Peer
   invalidated is for it to attempt to
   use STag values that it is not authorized to use. Note that if
   the Remote Peer sends an invalid STag through to the Local Peer, per the
   DDP and RDMAP Remote Invalidate
   capability (see section 5.5.5 Remote Invalidate specifications, the Stream must be torn down. Thus
   the threat exists if an STag Shared on
   Multiple Streams on page 34 has been enabled for a definition of Remote
   Invalidate), Access
   on one Stream and if a Remote Peer is able to use it did not, the local ULP then explicitly
   revokes on an unrelated
   Stream. If the STag remote access rights.

   The local ULP SHOULD follow attack is successful, the above procedure attacker could
   potentially be able to protect the
   buffer before it validates perform either RDMA Read Operations to
   read the contents of the buffer (or uses
   the buffer in any way).

   An RNIC MUST ensure that network packets using the STag for a
   previously advertised buffer can no longer associated data buffer, perform RDMA
   Write Operations to modify the buffer
   after contents of the ULP revokes remote access rights for associated data
   buffer, or to invalidate the specific STag.

5.3.3  Multiple STags STag to disable further access the same buffer

   See section 5.4.6 Using Multiple STags Which Alias to
   the Same
   Buffer on page 25 for this analysis.

5.3.4  Network based modification of buffer.

   An attempt by a Remote Peer to access a buffer content

   This is actually with an STag on a man
   different Stream in the middle same Protection Domain may or may not be
   an attack - but only depending on whether resource sharing is intended (i.e.
   whether the
   content of Streams shared Partial Mutual Trust or not). For some
   ULPs using an STag on multiple Streams within the buffer, as opposed same Protection
   Domain could be desired behavior. For other ULPs attempting to
   use an STag on a different Stream could be considered to be an
   attack. Since this varies by ULP, a ULP typically would need to
   be able to control the man in scope of the middle attack
   presented above, where both STag.

   In the signaling and content case where an implementation does not share resources
   between Streams (including STags), this attack can be
   modified. See Section 5.2.3 Man in the Middle Attack on page 20.

5.4  Information Disclosure

   The main potential source for information disclosure is through defeated by
   assigning each Stream to a
   local buffer that has been enabled for different Protection Domain. Before
   allowing remote access. If access to the
   buffer can be probed by a Remote Peer on another Stream, then
   there is potential for information disclosure.

   The potential attacks that could result in unintended information
   disclosure and countermeasures are detailed in buffer, the following
   sections.

5.4.1  Probing memory outside Protection Domain of
   the buffer bounds

   This is essentially Stream where the same attack as described in Section
   5.3.1, except an RDMA Read Request access attempt was made is used to mount matched against
   the attack.
   The same countermeasure applies.

5.4.2  Using RDMA Read to Access Stale Data Protection Domain of the STag. If a the Protection Domains do
   not match, access to the buffer is being used for a combination of reads and writes
   (either remote or local), and denied, an error is exposed to generated,
   and the Remote Peer RDMAP Stream associated with
   at least remote read access rights, the Remote Peer attacking Stream is
   terminated.

   For implementations that share resources between multiple
   Streams, it may not be able practical to examine the contents of the buffer before they are initialized
   with the correct data. separate each Stream into its
   own Protection Domain. In this situation, whatever contents were
   present in the buffer before the buffer is initialized can be
   viewed by the Remote Peer, if the Remote Peer performs an RDMA
   Read.

   Because of this, case, the local ULP SHOULD ensure that no stale data
   is contained in the buffer before remote read access rights are
   granted (this can be done by zeroing still limit the contents
   scope of any of the memory,
   for example).

5.4.3  Accessing STags to a Buffer After the Transfer single Stream (if it is enabling
   it for remote access). If the Remote Peer has remote read access to a buffer, and by
   some mechanism tells the local ULP that the transfer STag scope has been
   completed, but the local ULP does not disable remote access limited to
   the buffer before modifying the data, it is possible for the
   Remote Peer a
   single Stream, any attempt to retrieve use that STag on a different Stream
   will result in an error, and the new data.

   This RDMAP Stream is similar to the attack defined terminated.

   Thus for implementations that do not share STags between Streams,
   each Stream MUST either be in Section 5.3.2 Modifying a Buffer After Indication on page 22. The same countermeasures
   apply. In addition, the local ULP SHOULD grant remote read access
   rights only for separate Protection Domain or the amount
   scope of time needed to retrieve the data.

5.4.4  Accessing Unintended Data With a Valid an STag

   If the ULP enables remote access MUST be limited to a buffer using single Stream.

   An RNIC MUST ensure that a specific Stream in a specific
   Protection Domain can not access an STag that
   references the entire buffer, but intends only in a portion of the
   buffer to be accessed, it different
   Protection Domain.

   An RNIC MUST ensure that if an STag is possible for the Remote Peer limited in scope to
   access the a
   single Stream, no other parts of the buffer anyway.

   To prevent this attack, the ULP SHOULD set Stream can use the base and bounds STag.

   An additional issue may be unintended sharing of STags (i.e. a
   bug in the buffer when ULP) or a bug in the Remote Peer which causes an off-
   by-one STag is initialized to expose only the data to be retrieved.

5.4.5  RDMA Read into an RDMA Write Buffer

   One form used. For additional protection, an
   implementation should allocate STags in such a fashion that it is
   difficult to predict the next allocated STag number, and also
   ensure that STags are reused at as slow a rate as possible. Any
   allocation method which would lead to intentional or
   unintentional reuse of disclosure an STag by the peer should be avoided
   (e.g. a method which always starts with a given STag and
   monotonically increases it for each new allocation, or a method
   which always uses the same STag for each operation).

6.2  Tampering

   A Remote Peer or a network based attacker can occur if attempt to tamper
   with the access rights contents of data buffers on the
   buffer a Local Peer that have been
   enabled remote read, when only for remote write access was
   intended. If the buffer contained ULP data, or data access. The types of tampering attacks
   from a
   transfer on Remote Peer are outlined in the sections that follow. For
   countermeasures against a network based attacker, see Section 5
   Attacks That Can be Mitigated With End-to-End Security.

6.2.1  Buffer Overrun - RDMA Write or Read Response

   This attack is an unrelated Stream, attempt by the Remote Peer could retrieve
   the data through to perform an RDMA
   Write or RDMA Read operation. Note that an RNIC
   implementation is not required Response to support STags that have both
   read and memory outside of the valid length
   range of the data buffer enabled for remote write access. This
   attack can occur even when no resources are shared across
   Streams. This issue can also arise if the ULP has a bug.

   The most obvious countermeasure for this type of attack is to not grant
   remote read access if the buffer is intended to must be write-only.
   Then in the Remote Peer would not be able to retrieve data
   associated with RNIC
   implementation, leveraging the buffer. An attempt STag. When the local ULP specifies
   to do so would result in
   an error the RNIC the base address and the RDMAP Stream associated with number of bytes in the Stream would be
   terminated.

   Thus if a ULP only intends a
   buffer to be exposed for remote
   write access, that it MUST set wishes to make accessible, the RNIC must ensure
   that the base and bounds check are applied to any access rights to the
   buffer to only
   enable remote write access.  Note that this requirement is not
   meant to restrict referenced by the use of zero-length RDMA Reads. Zero-length STag before the STag is enabled for
   access. When an RDMA Reads do not expose ULP data. Because they are intended to
   be used as data transfer operation (which includes an
   STag) arrives on a mechanism Stream, a base and bounds byte granularity
   access check must be performed to ensure the operation accesses
   only memory locations within the buffer described by that STag.

   Thus an RNIC implementation MUST ensure that all RDMA Writes have been
   received, and do not even require a valid STag, their use Remote Peer is
   permitted even if a not
   able to access memory outside of the buffer has only been specified when the
   STag was enabled for write remote access.

5.4.6  Using Multiple STags Which Alias to the Same

6.2.2  Modifying a Buffer

   Multiple STags which alias After Indication

   This attack can occur if a Remote Peer attempts to modify the same
   contents of an STag referenced buffer at the same time
   can result in unintentional information disclosure if the STags
   are used by different, mutually untrusted, performing an RDMA Write
   or an RDMA Read Response after the Remote Peers. This
   model applies specifically Peer has indicated to client/server communication, where
   the server is communicating with multiple clients, each of which
   do not mutually trust each other.

   If only read access is enabled, then
   the Local Peer or local ULP has complete
   control over information disclosure. Thus (by a server which intended
   to expose variety of means) that the same STag
   data (i.e. buffer) to multiple clients by
   using multiple STags to the same buffer creates contents are ready for use. This attack can occur
   even when no new security
   issues beyond what has already been described in this document. resources are shared across Streams. Note that if a bug
   in a Remote Peer, or network based tampering, could also result
   in this problem.

   For example, assume the server did not intend to expose STag referenced buffer contains ULP
   control information as well as ULP payload, and the same data ULP sequence
   of operation is to first validate the clients, it should use separate buffers for each client (and
   separate STags).

   When one STag has remote read access enabled control information and a different STag
   has remote write access enabled to
   then perform operations on the control information. If the same buffer, it is
   possible for one Remote
   Peer to view can perform an additional RDMA Write or RDMA Read Response
   (thus changing the contents that buffer) after the validity checks have been
   written by another Remote Peer.

   If both STags have remote write access enabled and
   completed but before the two Remote
   Peers do not mutually trust each other, it control data is possible for one operated on, the Remote
   Peer to overwrite could force the contents ULP down operational paths that have been written were never
   intended.

   The local ULP can protect itself from this type of attack by
   revoking remote access when the other Remote Peer.

   Thus a original data transfer has
   completed and before it validates the contents of the buffer. The
   local ULP with multiple Remote Peers which can either do not share Partial
   Mutual Trust MUST NOT grant write this by explicitly revoking remote access to
   rights for the STag when the same buffer
   through different STags. A buffer should be exposed to only one
   untrusted Remote Peer at a time to ensure that no information
   disclosure or information tampering occurs between peers.

5.4.7  Remote Node Loading Firmware onto indicates the RNIC

   If operation
   has completed, or by checking to make sure the Remote Peer can cause firmware to be loaded onto
   invalidated the RNIC,
   there is an opportunity for information disclosure. See Elevation
   of Privilege in Section 5.5.6 for this analysis.

5.4.8  Controlling Access to PTT & STag Mapping

   If a Non-Privileged ULP is able to directly manipulate through the RNIC
   Page Translation Tables (which translate from RDMAP Remote Invalidate
   capability (see section 6.4.5 Remote Invalidate an STag to Shared on
   Multiple Streams for a host
   address), definition of Remote Invalidate), and if
   it is possible that did not, the Non-Privileged local ULP could point then explicitly revokes the Page Translation Table at an unrelated Stream's or ULP's
   buffers and thereby be able to gain STag remote
   access rights.

   The local ULP SHOULD follow the above procedure to information protect the
   buffer before it validates the contents of the
   unrelated Stream/ULP.

   As discussed in Section 2 Architectural Model on page 6,
   introduction of a Privileged Resource Manager to arbitrate the
   mapping requests is an effective countermeasure. This enables the
   Privileged Resource Manager to ensure a local ULP can only
   initialize the Page Translation Table (PTT)to point to its own
   buffers.

   Thus if Non-Privileged ULPs are supported, buffer (or uses
   the Privileged
   Resource Manager buffer in any way).

   An RNIC MUST verify ensure that network packets using the Non-Privileged ULP has the
   right to access a specific Data Buffer before allowing an STag for which a
   previously advertised buffer can no longer modify the buffer
   after the ULP has revokes remote access rights to be associated with a
   specific Data Buffer. This can be done when for the Page Translation
   Table is initialized specific STag.

6.2.3  Multiple STags to access the Data Buffer or when same buffer

   See section 6.3.6 Using Multiple STags Which Alias to the STag Same
   Buffer for this analysis.

6.3  Information Disclosure

   The main potential source for information disclosure is initialized to point to through a group of Page Translation Table
   entries, or both.

5.4.9  Network based eavesdropping

   An attacker
   local buffer that is able to eavesdrop on the network can read the
   content of all read and write accesses to a Peer's buffers. To
   prevent information disclosure, the read/written data must be
   encrypted. See also Section 5.2.3 Man in has been enabled for remote access. If the Middle Attack on
   page 20. The encryption
   buffer can be done either by the ULP, or probed by a
   protocol Remote Peer on another Stream, then
   there is potential for information disclosure.

   The potential attacks that provides security services to could result in unintended information
   disclosure and countermeasures are detailed in the LLP (e.g. IPsec
   or SSL). Refer to section 6 for discussion of security services
   for DDP/RDMA.

5.5  Denial following
   sections.

6.3.1  Probing memory outside of Service (DOS)

   A DOS the buffer bounds

   This is essentially the same attack as described in Section
   6.2.1, except an RDMA Read Request is one of used to mount the primary security risks of RDMAP. This attack.
   The same countermeasure applies.

6.3.2  Using RDMA Read to Access Stale Data

   If a buffer is because RNIC resources are valuable being used for a combination of reads and scarce, writes
   (either remote or local), and many ULP
   environments require communication with untrusted Remote Peers.
   If is exposed to the Remote Peer with
   at least remote ULP can be authenticated or encrypted, clearly, read access rights, the
   DOS profile can Remote Peer may be reduced. For able
   to examine the purposes contents of the buffer before they are initialized
   with the correct data. In this analysis, it
   is assumed that situation, whatever contents were
   present in the RNIC must buffer before the buffer is initialized can be able to operate
   viewed by the Remote Peer, if the Remote Peer performs an RDMA
   Read.

   Because of this, the local ULP SHOULD ensure that no stale data
   is contained in untrusted
   environments, which the buffer before remote read access rights are open to DOS style attacks.

   Denial
   granted (this can be done by zeroing the contents of service attacks against RNIC resources are not the
   typical unknown party spraying packets at a random host (such as memory,
   for example).

6.3.3  Accessing a TCP SYN attack). Because Buffer After the connection/Stream must be fully
   established, Transfer

   If the attacker must be able Remote Peer has remote read access to both send a buffer, and receive
   messages over by
   some mechanism tells the local ULP that connection/Stream, or be able the transfer has been
   completed, but the local ULP does not disable remote access to guess a valid
   packet on an existing RDMAP Stream.

   This section outlines
   the potential attacks and buffer before modifying the
   countermeasures available data, it is possible for dealing with each attack.

5.5.1  RNIC Resource Consumption the
   Remote Peer to retrieve the new data.

   This section covers attacks that fall into is similar to the general category
   of attack defined in Section 6.2.2 Modifying
   a local ULP attempting to unfairly allocate scarce (i.e.
   bounded) RNIC resources. Buffer After Indication. The same countermeasures apply. In
   addition, the local ULP may be attempting to
   allocate resources on its own behalf, or on behalf SHOULD grant remote read access rights
   only for the amount of time needed to retrieve the data.

6.3.4  Accessing Unintended Data With a Remote
   Peer. Resources that fall into this category include: Protection
   Domains, Stream Context Memory, Translation and Protection
   Tables, and Valid STag namespace. These can be due

   If the ULP enables remote access to attacks by
   currently active local ULPs or ones a buffer using an STag that allocated resources
   earlier,
   references the entire buffer, but are now idle.

   This type of attack can occur regardless of whether or not
   resources are shared across Streams.

   The allocation intends only a portion of all scarce resources MUST the
   buffer to be placed under accessed, it is possible for the
   control Remote Peer to
   access the other parts of a Privileged Resource Manager. This allows the
   Privileged Resource Manager to:

       * buffer anyway.

   To prevent a local ULP from allocating more than its fair
           share this attack, the ULP SHOULD set the base and bounds of resources.

       *   detect if a Remote Peer
   the buffer when the STag is attempting initialized to launch a DOS
           attack by attempting expose only the data
   to create be retrieved.

6.3.5  RDMA Read into an excessive number RDMA Write Buffer

   One form of
           Streams (with associated resources) and take corrective
           action (such as refusing disclosure can occur if the request access rights on the
   buffer enabled remote read, when only remote write access was
   intended. If the buffer contained ULP data, or applying network
           layer filters against data from a
   transfer on an unrelated Stream, the Remote Peer).

   This analysis assumes that Peer could retrieve
   the Resource Manager data through an RDMA Read operation. Note that an RNIC
   implementation is responsible
   for handing out Protection Domains, not required to support STags that have both
   read and RNIC implementations will
   provide enough Protection Domains write access.

   The most obvious countermeasure for this attack is to allow not grant
   remote read access if the Resource Manager buffer is intended to be write-only.
   Then the Remote Peer would not be able to assign retrieve data
   associated with the buffer. An attempt to do so would result in
   an error and the RDMAP Stream associated with the Stream would be
   terminated.

   Thus if a unique Protection Domain for each
   unrelated, untrusted local ULP (for only intends a bounded, reasonable number
   of local ULPs). This analysis further assumes that buffer to be exposed for remote
   write access, it MUST set the Resource
   Manager implements policies access rights to ensure that untrusted local ULPs
   are not able the buffer to consume all of the Protection Domains through a
   DOS attack. Note that Protection Domain consumption cannot result
   from a DOS attack launched by a Remote Peer, unless a local ULP
   is acting on the Remote Peer's behalf.

5.5.2  Resource Consumption By Active ULPs

   This section describes DOS attacks from Local and Remote Peers
   that are actively exchanging messages. Attacks on each RDMA NIC
   resource are examined and specific countermeasures are
   identified. only
   enable remote write access.  Note that attacks on Stream Context Memory, Page
   Translation Tables, and STag namespace are covered in Section
   5.5.1 RNIC Resource Consumption, so are not included here.

5.5.2.1  Multiple Streams Sharing Receive Buffers

   The Remote Peer can attempt to consume more than its fair share
   of receive data buffers (i.e. Untagged buffers for DDP are or
   Send Type Messages for RDMAP) if receive buffers are shared
   across multiple Streams.

   If resources are not shared across multiple Streams, then this
   attack requirement is not possible because the Remote Peer will not be able
   to consume more buffers than were allocated to the Stream. The
   worst case scenario is that the Remote Peer can consume more
   receive buffers than the local ULP allowed, resulting in no
   buffers being available, which could cause the Remote Peer's
   Stream to the Local Peer to be torn down, and all allocated
   resources to be released.

   If local receive data buffers are shared among multiple Streams,
   then the Remote Peer can attempt
   meant to consume more than its fair
   share of restrict the receive buffers, causing a different Stream to be
   short use of receive buffers, thus possibly causing the other Stream
   to be torn down. For example, if the Remote Peer sent enough one
   byte Untagged Messages, they might be able to consume all local
   shared receive queue resources with little effort on their part.

   One method the Local Peer could use is to recognize that a Remote
   Peer is attempting to use more than its fair share of resources
   and terminate the Stream (causing the allocated resources to be
   released). However, if the Local Peer is sufficiently slow, it
   may be possible for the Remote Peer to still mount a denial of
   service attack. One countermeasure that can protect against this
   attack is implementing a low-water notification. The low-water
   notification alerts the ULP if the number of buffers in the
   receive queue is less than a threshold.

   If all of the following conditions are true, then the Local Peer
   or local ULP can size the amount of local receive buffers posted
   on the receive queue to ensure a DOS attack can be stopped.

       *   a low-water notification is enabled, and

       *   the Local Peer is able to bound the amount of time that
           it takes to replenish receive buffers, and

       *   the Local Peer maintains statistics to determine which
           Remote Peer is consuming buffers.

   The above conditions enable the low-water notification to arrive
   before resources are depleted and thus the Local Peer or local
   ULP can take corrective action (e.g., terminate the Stream of the
   attacking Remote Peer).

   A different, but similar attack is if the Remote Peer sends a
   significant number of out-of-order packets and the RNIC has the
   ability to use the ULP buffer (i.e. the Untagged Buffer for DDP
   or the buffer consumed by a Send Type Message for RDMAP) as a
   reassembly buffer. In this case the Remote Peer can consume a
   significant number of ULP buffers, but never send enough data to
   enable the ULP buffer to be completed to the ULP.

   An effective countermeasure is to create a high-water
   notification which alerts the ULP if there is more than a
   specified number of receive buffers "in process" (partially
   consumed, but not completed). The notification is generated when
   more than the specified number of buffers are in process
   simultaneously on a specific Stream (i.e., packets have started
   to arrive for the buffer, but the buffer has not yet been
   delivered to the ULP).

   A different countermeasure is for the RNIC Engine to provide the
   capability to limit the Remote Peer's ability to consume receive
   buffers on a per Stream basis. Unfortunately this requires a
   large amount of state to be tracked in each RNIC on a per Stream
   basis.

   Thus, if an RNIC Engine provides the ability to share receive
   buffers across multiple Streams, the combination of the RNIC
   Engine and the Privileged Resource Manager MUST be able to detect
   if the Remote Peer is attempting to consume more than its fair
   share of resources so that the Local Peer or local ULP can apply
   countermeasures to detect and prevent the attack.

5.5.2.2  Local ULP Attacking a Shared CQ

   DOS attacks against a Shared Completion Queue (CQ) can be caused
   by either the local ULP or the Remote Peer if either attempts to
   cause more completions than its fair share of the number of
   entries, thus potentially starving another unrelated zero-length RDMA Reads. Zero-length
   RDMA Reads do not expose ULP such
   that no Completion Queue entries data. Because they are available.

   A Completion Queue entry can potentially intended to
   be maliciously consumed
   by used as a completion from the Send Queue or mechanism to ensure that all RDMA Writes have been
   received, and do not even require a completion from the
   Receive Queue. In the former, the attacker valid STag, their use is
   permitted even if a buffer has only been enabled for write
   access.

6.3.6  Using Multiple STags Which Alias to the local ULP. In Same Buffer

   Multiple STags which alias to the latter, same buffer at the attacker is same time
   can result in unintentional information disclosure if the STags
   are used by different, mutually untrusted, Remote Peer.

   A form of attack can occur Peers. This
   model applies specifically to client/server communication, where
   the local ULPs can consume
   resources on the CQ. A local ULP that server is slow to free resources
   on the CQ by communicating with multiple clients, each of which
   do not reaping mutually trust each other.

   If only read access is enabled, then the completion status quickly enough
   could stall all other local ULPs attempting to use that CQ.

   For these reasons, an RNIC MUST NOT enable sharing a CQ across
   ULPs that do not share Partial Mutual Trust.

5.5.2.3  Local or Remote Peer Attacking ULP has complete
   control over information disclosure. Thus a Shared CQ

   For an overview of server which intended
   to expose the shared CQ attack model, see Section
   5.5.2.2.

   The Remote Peer can attack a shared CQ by consuming more than its
   fair share of CQ entries same data (i.e. buffer) to multiple clients by
   using one of multiple STags to the following methods:

       *   The ULP protocol allows same buffer creates no new security
   issues beyond what has already been described in this document.
   Note that if the Remote Peer server did not intend to cause expose the
           local ULP same data to reserve
   the clients, it should use separate buffers for each client (and
   separate STags).

   When one STag has remote read access enabled and a specified number of CQ entries,
           possibly leaving insufficient entries different STag
   has remote write access enabled to the same buffer, it is
   possible for other Streams
           that are sharing one Remote Peer to view the CQ.

       * contents that have been
   written by another Remote Peer.

   If both STags have remote write access enabled and the two Remote Peer, Local Peer, or local ULP (or any
           combination) can attack
   Peers do not mutually trust each other, it is possible for one
   Remote Peer to overwrite the CQ contents that have been written by overwhelming
   the CQ
           with completions, then completion processing on other
           Streams sharing that Completion Queue can be affected
           (e.g. the Completion Queue overflows and stops
           functioning).

   The first method of attack can be avoided if the Remote Peer.

   Thus a ULP does with multiple Remote Peers which do not
   allow a share Partial
   Mutual Trust MUST NOT grant write access to the same buffer
   through different STags. A buffer should be exposed to only one
   untrusted Remote Peer to reserve CQ entries or there is a trusted
   intermediary such as at a Privileged Resource Manager. Unfortunately
   it is often unrealistic time to ensure that no information
   disclosure or information tampering occurs between peers.

6.3.7  Controlling Access to not allow PTT & STag Mapping

   If a Remote Peer Non-Privileged ULP is able to reserve CQ
   entries - particularly if directly manipulate the number of completion entries RNIC
   Page Translation Tables (which translate from an STag to a host
   address), it is
   dependent on other possible that the Non-Privileged ULP negotiated parameters, such as could point
   the amount Page Translation Table at an unrelated Stream's or ULP's
   buffers and thereby be able to gain access to information of buffering required by the ULP. Thus an implementation MUST
   implement
   unrelated Stream/ULP.

   As discussed in Section 2 Architectural Model, introduction of a
   Privileged Resource Manager to control arbitrate the mapping requests is
   an effective countermeasure. This enables the allocation
   of CQ entries. See Section 2.1 Components on page 7 for a
   definition of Privileged Resource Manager.

   One way that
   Manager to ensure a Local or Remote Peer local ULP can attempt only initialize the Page
   Translation Table (PTT)to point to overwhelm its own buffers.

   Thus if Non-Privileged ULPs are supported, the Privileged
   Resource Manager MUST verify that the Non-Privileged ULP has the
   right to access a CQ specific Data Buffer before allowing an STag
   for which the ULP has access rights to be associated with completions a
   specific Data Buffer. This can be done when the Page Translation
   Table is by sending minimum length RDMAP/DDP Messages initialized to cause as access the Data Buffer or when the STag
   is initialized to point to a group of Page Translation Table
   entries, or both.

6.4  Denial of Service (DOS)

   A DOS attack is one of the primary security risks of RDMAP. This
   is because RNIC resources are valuable and scarce, and many completions (receive completions for the ULP
   environments require communication with untrusted Remote
   Peer, send completions for the Local Peer) per second as
   possible. Peers.
   If it is the Remote Peer attacking, and we assume that can be authenticated or the Local Peer's receive queue(s) do not run out of receive
   buffers (if they do, then this is a different attack, documented
   in Section 5.5.2.1 Multiple Streams Sharing Receive Buffers on
   page 27), then it might ULP payload
   encrypted, clearly, the DOS profile can be possible for reduced. For the Remote Peer to
   consume more than its fair share
   purposes of Completion Queue entries.
   Depending upon the CQ implementation, this could either cause the
   CQ to overflow (if analysis, it is not large enough assumed that the RNIC must be
   able to handle all operate in untrusted environments, which are open to DOS
   style attacks.

   Denial of service attacks against RNIC resources are not the
   completions generated) or for another Stream
   typical unknown party spraying packets at a random host (such as
   a TCP SYN attack). Because the connection/Stream must be fully
   established (e.g. a 3 message transport layer handshake has
   occurred), the attacker must be able to not both send and receive
   messages over that connection/Stream, or be able to
   generate CQ entries (if guess a valid
   packet on an existing RDMAP Stream.

   This section outlines the potential attacks and the
   countermeasures available for dealing with each attack.

6.4.1  RNIC had flow control Resource Consumption

   This section covers attacks that fall into the general category
   of a local ULP attempting to unfairly allocate scarce (i.e.
   bounded) RNIC resources. The local ULP may be attempting to
   allocate resources on generation its own behalf, or on behalf of CQ entries a Remote
   Peer. Resources that fall into the CQ). In either case, the CQ will stop
   functioning correctly this category include: Protection
   Domains, Stream Context Memory, Translation and any Streams expecting completions on
   the CQ will stop functioning. Protection
   Tables, and STag namespace. These can be due to attacks by
   currently active local ULPs or ones that allocated resources
   earlier, but are now idle.

   This type of attack can occur regardless of whether all of the Streams
   associated with the CQ are in the same Protection Domain or not
   resources are
   in different Protection Domains - the key issue is that the
   number shared across Streams.

   The allocation of Completion Queue entries is less than all scarce resources MUST be placed under the number
   control of all
   outstanding operations that can cause a completion.

   The Local Peer can protect itself Privileged Resource Manager. This allows the
   Privileged Resource Manager to:

       *   prevent a local ULP from this type of attack using
   either allocating more than its fair
           share of the following methods: resources.

       *   Size the CQ to the appropriate level, as specified below
           (note that   detect if the CQ currently exists, and it needs to be
           resized, resizing the CQ a Remote Peer is not required attempting to succeed in
           all cases, so the CQ resize should be done before sizing
           the Send Queue launch a DOS
           attack by attempting to create an excessive number of
           Streams (with associated resources) and Receive Queue on the Stream), OR

       *   Grant fewer resources than take corrective
           action (such as refusing the Remote Peer requested (not
           supplying request or applying network
           layer filters against the number of Receive Data Buffers requested).

   The proper sizing of Remote Peer).

   This analysis assumes that the CQ Resource Manager is dependent on whether the local
   ULP(s) responsible
   for handing out Protection Domains, and RNIC implementations will post as many resources
   provide enough Protection Domains to allow the various queues as the
   size of the queue enables or not. If the local ULP(s) can Resource Manager
   to be
   trusted able to post assign a unique Protection Domain for each
   unrelated, untrusted local ULP (for a bounded, reasonable number
   of resources local ULPs). This analysis further assumes that is smaller than the
   size of the specific resource's queue, then a correctly sized CQ
   means Resource
   Manager implements policies to ensure that the CQ is large enough untrusted local ULPs
   are not able to hold completion status for consume all of the outstanding Data Buffers (both send and receive
   buffers), or:

            CQ_MIN_SIZE = SUM(MaxPostedOnEachRQ)
                          + SUM(MaxPostedOnEachSRQ)
                          + SUM(MaxPostedOnEachSQ)

   Where:

           MaxPostedOnEachRQ = the maximum number of requests which
                  can cause Protection Domains through a completion
   DOS attack. Note that will be posted on Protection Domain consumption cannot result
   from a
                  specific Receive Queue.

           MaxPostedOnEachSRQ = the maximum number of requests which
                  can cause DOS attack launched by a completion that will be posted on Remote Peer, unless a
                  specific Shared Receive Queue.

           MaxPostedOnEachSQ = local ULP
   is acting on the maximum number Remote Peer's behalf.

6.4.2  Resource Consumption by Idle ULPs

   The simplest form of requests which
                  can cause a completion that will be posted on DOS attack given a
                  specific Send Queue.

   If the local ULP must be able to completely fill the queues, or
   can not be trusted fixed amount of
   resources is for the Remote Peer to observe create a limit smaller than the queues, RDMAP Stream to a
   Local Peer, and request dedicated resources then do no actual
   work. This allows the CQ must be sized Remote Peer to accommodate be very light weight (i.e.
   only negotiate resources, but do no data transfer) and consumes a
   disproportionate amount of resources at the maximum number Local Peer.

   A general countermeasure for this style of
   operations that it attack is possible to post at any one time. Thus the
   equation becomes:

            CQ_MIN_SIZE = SUM(SizeOfEachRQ)
                          + SUM(SizeOfEachSRQ)
                          + SUM(SizeOfEachSQ)

   Where:

          SizeOfEachRQ = monitor
   active RDMAP Streams and if resources are getting low, reap the maximum number of requests which
                  can cause a completion
   resources from RDMAP Streams that can ever are not transferring data and
   possibly terminate the Stream. This would presumably be posted
                  on a specific Receive Queue.

          SizeOfEachSRQ = under
   administrative control.

   Refer to Section 6.4.1 for the maximum number analysis and countermeasures for
   this style of requests which
                  can cause a completion that can ever be posted attack on a specific Shared Receive Queue.

          SizeOfEachSQ = the maximum number of requests which
                  can cause a completion following RNIC resources: Stream
   Context Memory, Page Translation Tables and STag namespace.

   Note that can ever be posted
                  on some RNIC resources are not at risk of this type of
   attack from a specific Send Queue.

   Where MaxPosted*OnEach*Q Remote Peer because an attack requires the Remote
   Peer to send messages in order to consume the resource. Receive
   Data Buffers, Completion Queue, and SizeOfEach*Q varies on RDMA Read Request Queue
   resources are examples. These resources are, however, at risk
   from a per Stream
   or per Shared Receive Queue basis.

   If the local ULP is sharing a CQ across multiple Streams which do not
   share Partial Mutual Trust, that attempts to allocate resources, then goes
   idle. This could also be created if the ULP MUST implement a
   mechanism negotiates the
   resource levels with the Remote Peer, which causes the Local Peer
   to ensure that consume resources, however the Completion Queue Remote Peer never sends data to
   consume them. The general countermeasure described in this
   section can not overflow. be used to free resources allocated by an idle Local
   Peer.

6.4.3  Resource Consumption By Active ULPs

   This section describes DOS attacks from Local and Remote Peers
   that are actively exchanging messages. Attacks on each RDMA NIC
   resource are examined and specific countermeasures are
   identified. Note that it is possible attacks on Stream Context Memory, Page
   Translation Tables, and STag namespace are covered in Section
   6.4.1 RNIC Resource Consumption, so are not included here.

6.4.3.1  Multiple Streams Sharing Receive Buffers

   The Remote Peer can attempt to consume more than its fair share CQs even if the Remote Peers
   accessing the CQs
   of receive data buffers (i.e. Untagged buffers for DDP are untrusted or
   Send Type Messages for RDMAP) if either of the above two
   formulas receive buffers are implemented. shared
   across multiple Streams.

   If resources are not shared across multiple Streams, then this
   attack is not possible because the ULP can Remote Peer will not be trusted able
   to not post consume more buffers than MaxPostedOnEachRQ, MaxPostedOnEachSRQ, and
   MaxPostedOnEachSQ, then were allocated to the first formula applies. If Stream. The
   worst case scenario is that the Remote Peer can consume more
   receive buffers than the local ULP can
   not be trusted to obey the limit, then allowed, resulting in no
   buffers being available, which could cause the second formula
   applies.

5.5.2.4  Attacking Remote Peer's
   Stream to the RDMA Read Request Queue

   If RDMA Read Request Queue Local Peer to be torn down, and all allocated
   resources to be released.

   If local receive data buffers are pooled across shared among multiple Streams, one attack is if
   then the local ULP attempts Remote Peer can attempt to unfairly
   allocate RDMA Read Request Queue resources for consume more than its Streams. For
   example, fair
   share of the receive buffers, causing a local ULP attempts different Stream to allocate all available resources
   on a specific RDMA Read Request Queue for its Streams, thereby
   denying be
   short of receive buffers, thus possibly causing the resource other Stream
   to ULPs sharing the RDMA Read Request Queue.
   The same type of argument applies even be torn down. For example, if the RDMA Read Request
   is not shared - but a local ULP attempts Remote Peer sent enough one
   byte Untagged Messages, they might be able to allocate consume all of the
   RNIC's local
   shared receive queue resources when with little effort on their part.

   One method the queue Local Peer could use is created.

   Thus access to interfaces recognize that allocate RDMA Read Request Queue
   entries MUST be restricted to a trusted Local Peer, such as a
   Privileged Resource Manager. The Privileged Resource Manager
   SHOULD prevent a local ULP from allocating Remote
   Peer is attempting to use more than its fair share of resources.

   Another form of attack is resources
   and terminate the Stream (causing the allocated resources to be
   released). However, if the Local Peer is sufficiently slow, it
   may be possible for the Remote Peer sends more RDMA Read
   Requests than the depth to still mount a denial of
   service attack. One countermeasure that can protect against this
   attack is implementing a low-water notification. The low-water
   notification alerts the RDMA Read Request Queue at ULP if the
   Local Peer. If number of buffers in the RDMA Read Request Queue
   receive queue is less than a shared resource,
   this could corrupt the queue. threshold.

   If all of the queue is not shared, following conditions are true, then the worst case Local Peer
   or local ULP can size the amount of local receive buffers posted
   on the receive queue to ensure a DOS attack can be stopped.

       *   a low-water notification is that enabled, and

       *   the current Stream Local Peer is no longer functional
   (e.g. torn down). One approach able to solving bound the shared RDMA Read
   Request Queue would be amount of time that
           it takes to create thresholds, similar replenish receive buffers, and

       *   the Local Peer maintains statistics to those
   described in Section 5.5.2.1 Multiple Streams Sharing Receive
   Buffers on page 27. A simpler approach determine which
           Remote Peer is consuming buffers.

   The above conditions enable the low-water notification to not share RDMA Read
   Request Queue arrive
   before resources among Streams are depleted and thus the Local Peer or enforce hard limits of
   consumption per Stream. Thus RDMA Read Request Queue resource
   consumption MUST be controlled by local
   ULP can take corrective action (e.g., terminate the Privileged Resource Manager
   such that RDMAP/DDP Streams which do not share Partial Mutual
   Trust do not share RDMA Read Request Queue resources.

   If Stream of the issue
   attacking Remote Peer).

   A different, but similar attack is a bug in if the Remote Peer's implementation, but
   not Peer sends a malicious attack, the issue can be solved by requiring
   significant number of out-of-order packets and the
   Remote Peer's RNIC has the
   ability to throttle RDMA Read Requests. By properly
   configuring use the Stream at ULP buffer (i.e. the Untagged Buffer for DDP
   or the buffer consumed by a Send Type Message for RDMAP) as a
   reassembly buffer. In this case the Remote Peer through can consume a trusted
   agent,
   significant number of ULP buffers, but never send enough data to
   enable the RNIC can ULP buffer to be made completed to the ULP.

   An effective countermeasure is to create a high-water
   notification which alerts the ULP if there is more than a
   specified number of receive buffers "in process" (partially
   consumed, but not transmit RDMA Read Requests
   that exceed completed). The notification is generated when
   more than the depth specified number of buffers are in process
   simultaneously on a specific Stream (i.e., packets have started
   to arrive for the RDMA Read Request Queue at buffer, but the Local
   Peer. If buffer has not yet been
   delivered to the Stream ULP).

   A different countermeasure is correctly configured, and if the Remote
   Peer submits more requests than for the Local Peer's RDMA Read
   Request Queue can handle, RNIC Engine to provide the requests would be queued at
   capability to limit the Remote Peer's ability to consume receive
   buffers on a per Stream basis. Unfortunately this requires a
   large amount of state to be tracked in each RNIC on a per Stream
   basis.

   Thus, if an RNIC Engine provides the ability to share receive
   buffers across multiple Streams, the combination of the RNIC until previous requests complete. If
   Engine and the
   Remote Peer's Stream is not configured correctly, Privileged Resource Manager MUST be able to detect
   if the RDMAP
   Stream Remote Peer is terminated when attempting to consume more RDMA Read Requests arrive at the
   Local Peer than its fair
   share of resources so that the Local Peer or local ULP can handle (assuming the prior
   paragraph's recommendation is implemented). Thus an RNIC
   implementation SHOULD provide a mechanism apply
   countermeasures to cap detect and prevent the number of
   outstanding RDMA Read Requests. The configuration attack.

6.4.3.2  Remote or Local Peer Attacking a Shared CQ

   For an overview of this limit
   is outside the scope of this document.

5.5.3  Resource Consumption by Idle ULPs shared CQ attack model, see Section 7.1.

   The simplest form of a DOS Remote Peer can attack given a fixed amount shared CQ by consuming more than its
   fair share of
   resources is for CQ entries by using one of the following methods:

       *   The ULP protocol allows the Remote Peer to create a RDMAP Stream cause the
           local ULP to reserve a specified number of CQ entries,
           possibly leaving insufficient entries for other Streams
           that are sharing the CQ.

       *   If the Remote Peer, Local Peer, and request dedicated resources or local ULP (or any
           combination) can attack the CQ by overwhelming the CQ
           with completions, then do no actual
   work. This allows completion processing on other
           Streams sharing that Completion Queue can be affected
           (e.g. the Completion Queue overflows and stops
           functioning).

   The first method of attack can be avoided if the ULP does not
   allow a Remote Peer to reserve CQ entries or there is a trusted
   intermediary such as a Privileged Resource Manager. Unfortunately
   it is often unrealistic to not allow a Remote Peer to be very light weight (i.e.
   only negotiate resources, but do no data transfer) and consumes a
   disproportionate amount of resources at reserve CQ
   entries - particularly if the Local Peer.

   A general countermeasure for this style number of attack completion entries is to monitor
   active RDMAP Streams and if resources are getting low, reap
   dependent on other ULP negotiated parameters, such as the
   resources from RDMAP Streams that are not transferring data and
   possibly terminate amount
   of buffering required by the Stream. This would presumably be under
   administrative control.

   Refer ULP. Thus an implementation MUST
   implement a Privileged Resource Manager to Section 5.5.1 for control the analysis and countermeasures allocation
   of CQ entries. See Section 2.1 Components for
   this style a definition of attack on the following RNIC resources: Stream
   Context Memory, Page Translation Tables and STag namespace.

   Note
   Privileged Resource Manager.

   One way that some RNIC resources are not at risk of this type of
   attack from a Local or Remote Peer because an attack requires can attempt to overwhelm a CQ
   with completions is by sending minimum length RDMAP/DDP Messages
   to cause as many completions (receive completions for the Remote
   Peer to
   Peer, send messages in order to consume completions for the resource. Receive
   Data Buffers, Completion Queue, Local Peer) per second as
   possible. If it is the Remote Peer attacking, and RDMA Read Request Queue
   resources are examples. These resources are, however, at risk
   from a local ULP we assume that attempts to allocate resources,
   the Local Peer's receive queue(s) do not run out of receive
   buffers (if they do, then this is a different attack, documented
   in Section 6.4.3.1 Multiple Streams Sharing Receive Buffers),
   then goes
   idle. This could also it might be created if the ULP negotiates the
   resource levels with possible for the Remote Peer, which causes the Local Peer to consume resources, however more
   than its fair share of Completion Queue entries. Depending upon
   the Remote Peer never sends data to
   consume them. The general countermeasure described in CQ implementation, this
   section can be used could either cause the CQ to free resources allocated by an idle Local
   Peer.

5.5.4  Exercise overflow
   (if it is not large enough to handle all of non-optimal code paths

   Another form the completions
   generated) or for another Stream to not be able to generate CQ
   entries (if the RNIC had flow control on generation of DOS CQ entries
   into the CQ). In either case, the CQ will stop functioning
   correctly and any Streams expecting completions on the CQ will
   stop functioning.

   This attack can occur regardless of whether all of the Streams
   associated with the CQ are in the same Protection Domain or are
   in different Protection Domains - the key issue is to attempt to exercise data paths that the
   number of Completion Queue entries is less than the number of all
   outstanding operations that can consume cause a disproportionate amount completion.

   The Local Peer can protect itself from this type of resources. An
   example might be if error cases are handled on a "slow path"
   (consuming attack using
   either host or RNIC computational resources), and an
   attacker generates excessive numbers of errors in an attempt the following methods:

       *   Size the CQ to
   consume these resources. Note the appropriate level, as specified below
           (note that for most RDMAP or DDP errors, if the attacking Stream will simply CQ currently exists, and it needs to be torn down. Thus for this form
   of attack
           resized, resizing the CQ is not required to succeed in
           all cases, so the CQ resize should be effective, done before sizing
           the Send Queue and Receive Queue on the Stream), OR

       *   Grant fewer resources than the Remote Peer needs to exercise data
   paths which do not cause requested (not
           supplying the Stream number of Receive Data Buffers requested).

   The proper sizing of the CQ is dependent on whether the local
   ULP(s) will post as many resources to be torn down.

   If an RNIC implementation contains "slow paths" which do not
   result in the tear down various queues as the
   size of the Stream, it queue enables or not. If the local ULP(s) can be
   trusted to post a number of resources that is recommended smaller than the
   size of the specific resource's queue, then a correctly sized CQ
   means that an
   implementation provide the ability CQ is large enough to detect hold completion status for
   all of the above condition outstanding Data Buffers (both send and allow an administrator to act, including potentially
   administratively tearing down the RDMAP Stream associated with receive
   buffers), or:

            CQ_MIN_SIZE = SUM(MaxPostedOnEachRQ)
                          + SUM(MaxPostedOnEachSRQ)
                          + SUM(MaxPostedOnEachSQ)

   Where:

           MaxPostedOnEachRQ = the Stream exercising data paths consuming maximum number of requests which
                  can cause a disproportionate
   amount completion that will be posted on a
                  specific Receive Queue.

           MaxPostedOnEachSRQ = the maximum number of resources.

5.5.5  Remote Invalidate an STag requests which
                  can cause a completion that will be posted on a
                  specific Shared Receive Queue.

           MaxPostedOnEachSQ = the maximum number of requests which
                  can cause a completion that will be posted on Multiple Streams

   If a Local Peer has enabled an STag for remote access,
                  specific Send Queue.

   If the Remote
   Peer could attempt local ULP must be able to remote invalidate the STag by using completely fill the
   RDMAP Send with Invalidate queues, or Send with SE and Invalidate
   Message. If the STag is only valid on
   can not be trusted to observe a limit smaller than the current Stream, queues,
   then the only side effect CQ must be sized to accommodate the maximum number of
   operations that it is possible to post at any one time. Thus the
   equation becomes:

            CQ_MIN_SIZE = SUM(SizeOfEachRQ)
                          + SUM(SizeOfEachSRQ)
                          + SUM(SizeOfEachSQ)

   Where:

          SizeOfEachRQ = the maximum number of requests which
                  can cause a completion that can ever be posted
                  on a specific Receive Queue.

          SizeOfEachSRQ = the Remote Peer maximum number of requests which
                  can no longer use cause a completion that can ever be posted
                  on a specific Shared Receive Queue.

          SizeOfEachSQ = the STag; thus there are no security issues. maximum number of requests which
                  can cause a completion that can ever be posted
                  on a specific Send Queue.

   Where MaxPosted*OnEach*Q and SizeOfEach*Q varies on a per Stream
   or per Shared Receive Queue basis.

   If the STag ULP is valid sharing a CQ across multiple Streams, then the Remote
   Peer can prevent other Streams from using that STag by using the
   remote invalidate functionality.

   Thus if RDDP Streams do not share Partial Mutual Trust (i.e. the
   Remote Peer may attempt to remote invalidate the STag
   prematurely), multiple Streams which do not
   share Partial Mutual Trust, then the ULP MUST NOT enable an STag which would be
   valid across multiple Streams.

5.5.6  Remote Peer attacking an Unshared CQ

   The Remote Peer implement a
   mechanism to ensure that the Completion Queue can attack an unshared CQ not overflow.
   Note that it is possible to share CQs even if the Local Peer does
   not size Remote Peers
   accessing the CQ correctly. For example, CQs are untrusted if either of the Local Peer enables above two
   formulas are implemented. If the CQ ULP can be trusted to handle completions of received buffers, not post
   more than MaxPostedOnEachRQ, MaxPostedOnEachSRQ, and
   MaxPostedOnEachSQ, then the receive
   buffer queue is longer than first formula applies. If the Completion Queue, then an
   overflow ULP can potentially occur. The effect on the attacker's
   Stream is catastrophic. However if an RNIC does
   not have be trusted to obey the
   proper protections in place, limit, then an attack to overflow the CQ second formula
   applies.

6.4.3.3  Attacking the RDMA Read Request Queue

   The RDMA Read Request Queue can also cause corruption and/or termination of an unrelated
   Stream. Thus an RNIC MUST ensure that be attacked if a CQ overflows, any
   Streams which do not use the CQ MUST remain unaffected.

5.6  Elevation of Privilege

   The RDMAP/DDP Security Architecture explicitly differentiates
   between three levels Remote Peer
   sends more RDMA Read Requests than the depth of privilege - Non-Privileged, Privileged,
   and the Privileged Resource Manager. If a Non-Privileged ULP is
   able to elevate its privilege level to a Privileged ULP, then
   mapping a physical address list to an STag can provide local and
   remote access to any physical address location on RDMA Read
   Request Queue at the node. Local Peer. If a
   Privileged Mode ULP the RDMA Read Request Queue
   is able to promote itself to be a Resource
   Manager, then it is possible for it to perform denial of service
   type attacks where substantial amounts of local resources shared resource, this could
   be consumed.

   In general, elevation of privilege is a local implementation
   specific issue and thus outside corrupt the scope of this document.

   There is one issue worth noting, however. queue. If the RNIC
   implementation, by some insecure mechanism (or implementation
   defect), can enable a Remote Peer or un-trusted local ULP to load
   firmware into queue
   is not shared, then the RNIC Engine, it worst case is possible to use that the RNIC current Stream is
   no longer functional (e.g. torn down). One approach to
   attack solving
   the host. Thus, an RNIC implementation MUST NOT enable
   firmware shared RDMA Read Request Queue would be to create thresholds,
   similar to those described in Section 6.4.3.1 Multiple Streams
   Sharing Receive Buffers. A simpler approach is to not share RDMA
   Read Request Queue resources among Streams or enforce hard limits
   of consumption per Stream. Thus RDMA Read Request Queue resource
   consumption MUST be loaded on controlled by the RNIC Engine directly from an
   untrusted local ULP or Remote Peer, unless they are properly
   authenticated (by Privileged Resource Manager
   such that RDMAP/DDP Streams which do not share Partial Mutual
   Trust do not share RDMA Read Request Queue resources.

   If the issue is a mechanism outside bug in the scope of this document.
   The mechanism presumably entails authenticating that Remote Peer's implementation, but
   not a malicious attack, the remote
   ULP has issue can be solved by requiring the right
   Remote Peer's RNIC to perform throttle RDMA Read Requests. By properly
   configuring the update), and Stream at the update is done
   via Remote Peer through a secure protocol, such as IPsec (See Section 6 Security
   Services for RDMAP and DDP on page 36).

6  Security Services for RDMAP and DDP

   RDMAP and DDP are used to control, read and write data buffers
   over IP networks. Therefore, trusted
   agent, the control and RNIC can be made to not transmit RDMA Read Requests
   that exceed the data packets depth of
   these protocols are vulnerable to the spoofing, tampering and
   information disclosure attacks listed in Section 7.

   Generally speaking, Stream confidentiality protects against
   eavesdropping. RDMA Read Request Queue at the Local
   Peer. If the Stream and/or session authentication and integrity
   protection is a counter measurement against various spoofing and
   tampering attacks. The effectiveness of authentication correctly configured, and
   integrity against a specific attack, depend on whether if the
   authentication is machine level authentication (as Remote
   Peer submits more requests than the one
   provided by IPsec and SSL), or ULP authentication.

6.1  Introduction to Security Options

   The following security services Local Peer's RDMA Read
   Request Queue can handle, the requests would be applied to an RDMAP/DDP
   Stream:

   1.  Session confidentiality - protects against eavesdropping
       (section 5.4.9).

   2.  Per-packet data source authentication - protects against queued at the
       following spoofing attacks: network based impersonation
       (section 5.2.1),
   Remote Peer's RNIC until previous requests complete. If the
   Remote Peer's Stream hijacking (section 5.2.2), and man in is not configured correctly, the middle (section 5.2.3).

   3.  Per-packet integrity - protects against tampering done by
       network based modification of buffer content (section 5.3.4)

   4.  Packet sequencing - protects against replay attacks, which RDMAP
   Stream is terminated when more RDMA Read Requests arrive at the
   Local Peer than the Local Peer can handle (assuming the prior
   paragraph's recommendation is implemented). Thus an RNIC
   implementation SHOULD provide a special case mechanism to cap the number of
   outstanding RDMA Read Requests. The configuration of this limit
   is outside the above tampering attack.

   If an RDMAP/DDP Stream may be subject to impersonation attacks,
   or Stream hijacking attacks, it scope of this document.

6.4.4  Exercise of non-optimal code paths

   Another form of DOS attack is recommended to attempt to exercise data paths
   that the Stream can consume a disproportionate amount of resources. An
   example might be
   authenticated, integrity protected, and protected from replay
   attacks; it may use confidentiality protection to protect from
   eavesdropping (in case the RDMAP/DDP Stream traverses if error cases are handled on a public
   network).

   Both IPsec "slow path"
   (consuming either host or RNIC computational resources), and SSL are capable an
   attacker generates excessive numbers of providing errors in an attempt to
   consume these resources. Note that for most RDMAP or DDP errors,
   the above security
   services attacking Stream will simply be torn down. Thus for IP and TCP traffic respectively. ULP protocols are
   able to provide only part this form
   of attack to be effective, the above security services. The
   next sections describe the different security options.

6.1.1  Introduction Remote Peer needs to IPsec

   IPsec is a protocol suite exercise data
   paths which is used to secure communication
   at the network layer between two peers. The IPsec protocol suite
   is specified within do not cause the IP Security Architecture [RFC2401], IKE
   [RFC2409], IPsec Authentication Header (AH) [RFC2402] and IPsec
   Encapsulating Security Payload (ESP) [RFC2406] documents. IKE Stream to be torn down.

   If an RNIC implementation contains "slow paths" which do not
   result in the tear down of the Stream, it is recommended that an
   implementation provide the key management protocol while AH and ESP are used ability to protect
   IP traffic.

   An IPsec SA is a one-way security association, uniquely
   identified by detect the 3-tuple: Security Parameter Index (SPI),
   protocol (ESP/AH) above condition
   and destination IP address. The parameters for allow an IPsec security association are typically established by a key
   management protocol. These include administrator to act, including potentially
   administratively tearing down the encapsulation mode,
   encapsulation type, session keys and SPI values.

   IKE is RDMAP Stream associated with
   the Stream exercising data paths consuming a two phase negotiation protocol based disproportionate
   amount of resources.

6.4.5  Remote Invalidate an STag Shared on Multiple Streams

   If a Local Peer has enabled an STag for remote access, the modular
   exchange of messages defined Remote
   Peer could attempt to remote invalidate the STag by ISAKMP [RFC2408],and using the IP
   Security Domain of Interpretation (DOI) [RFC2407]. IKE has two
   phases,
   RDMAP Send with Invalidate or Send with SE and accomplishes Invalidate
   Message. If the following functions:

   1.  Protected cipher suite and options negotiation - using keyed
       MACs and encryption and anti-replay mechanisms.

   2.  Master key generation - via Diffie-Hellman calculations.

   3.  Authentication of end-points (usually machine level
       authentication).

   4.  IPsec SA management (selector negotiation, options
       negotiation, create, delete, and rekeying).

   Items 1 through 3 are accomplished in IKE Phase 1, while item 4 STag is handled in IKE Phase 2.

   IKE phase 1 defines four authentication methods; three of them
   require both sides to have certified signature or encryption
   public keys; only valid on the fourth requires current Stream, then
   the only side to exchange out-of-band
   a secret random string - called pre-shared-secret (PSS).

   An IKE Phase 2 negotiation effect is performed to establish both an
   inbound and an outbound IPsec SA. The traffic to be protected by
   an IPsec SA that the Remote Peer can no longer use
   the STag; thus there are no security issues.

   If the STag is determined by a selector which has been proposed
   by valid across multiple Streams, then the IKE initiator and accepted Remote
   Peer can prevent other Streams from using that STag by using the IKE Responder. The IPsec
   SA selector can be a "filter" or traffic classifier, defined as
   remote invalidate functionality.

   Thus if RDDP Streams do not share Partial Mutual Trust (i.e. the 5-tuple: <Source IP address, Destination IP address,
   transport protocol (e.g. UDP/SCTP/TCP), Source port, Destination
   port>. The successful establishment of a IKE Phase-2 SA results
   in
   Remote Peer may attempt to remote invalidate the creation of two uni-directional IPsec SAs fully qualified
   by STag
   prematurely), the tuple <Protocol (ESP/AH), destination address, SPI>. ULP MUST NOT enable an STag which would be
   valid across multiple Streams.

6.4.6  Remote Peer attacking an Unshared CQ

   The session keys for each IPsec SA are derived from a master key,
   typically via a MODP Diffie-Hellman computation. Rekeying of Remote Peer can attack an
   existing IPsec SA pair is accomplished by creating two new IPsec
   SAs, making them active, and then optionally deleting unshared CQ if the older
   IPsec SA pair. Typically Local Peer does
   not size the new outbound SA is used immediately,
   and CQ correctly. For example, if the old inbound SA is left active to receive packets for some
   locally defined time, perhaps 30 seconds or 1 minute. Optionally,
   rekeying can use Diffie-Hellman for keying material generation.

6.1.2  Introduction Local Peer enables
   the CQ to SSL Limitations on RDMAP

   SSL and TLS [RFC 2246] provide Stream authentication, integrity handle completions of received buffers, and confidentiality for TCP based ULPs. SSL supports one-way
   (server only) or mutual certificates based authentication.

   There are at least two limitations that make SSL underneath RDMAP
   less appropriate than IPsec for DDP/RDMA security:

   1.  The maximum length supported by the TLS record layer protocol receive
   buffer queue is 2^14 bytes - longer packets must be fragmented (as a
       comparison, than the maximal length of Completion Queue, then an IPsec packet
   overflow can potentially occur. The effect on the attacker's
   Stream is
       determined by catastrophic. However if an RNIC does not have the maximum length
   proper protections in place, then an attack to overflow the CQ
   can also cause corruption and/or termination of an IP packet).

   2.  SSL is unrelated
   Stream. Thus an RNIC MUST ensure that if a connection oriented protocol. CQ overflows, any
   Streams which do not use the CQ MUST remain unaffected.

6.5  Elevation of Privilege

   The RDMAP/DDP Security Architecture explicitly differentiates
   between three levels of privilege - Non-Privileged, Privileged,
   and the Privileged Resource Manager. If a stream cipher or
       block cipher in CBC mode Non-Privileged ULP is used for bulk encryption,
   able to elevate its privilege level to a Privileged ULP, then
   mapping a
       packet physical address list to an STag can be decrypted only after all provide local and
   remote access to any physical address location on the packets preceding
       it have already arrived. node. If SSL a
   Privileged Mode ULP is used able to protect DDP/RDMA
       traffic, then SSL must gather all out-of-order packets before
       RDMAP/DDP can place them into the ULP buffer, which might
       cause promote itself to be a significant decrease in its efficiency.

   If SSL Resource
   Manager, then it is layered on top of RDMAP or DDP, SSL does not protect
   the RDMAP and/or DDP headers. Thus a man-in-the-middle attack can
   still occur by modifying the RDMAP/DDP header to incorrectly
   place the data into the wrong buffer, thus effectively corrupting
   the data stream.

6.1.3  ULPs Which Provide Security

   ULPs which provide integrated security but wish possible for it to leverage
   lower-layer protocol security should perform denial of service
   type attacks where substantial amounts of local resources could
   be aware consumed.

   In general, elevation of security
   concerns around correlating privilege is a local implementation
   specific channel's security
   mechanisms to the authentication performed by issue and thus outside the ULP. See
   [NFSv4CHANNEL] for additional information on a promising approach
   called "channel binding". From [NFSv4CHANNEL]:

        "The concept scope of channel bindings allows applications to
        prove this document.

7  Attacks from Local Peers

   This section describes local attacks that the end-points of two secure channels at
        different network layers are possible against
   the same RDMA system defined in Figure 1 - RDMA Security Model and the
   RNIC Engine resources defined in Section 2.2.

7.1  Local ULP Attacking a Shared CQ

   DOS attacks against a Shared Completion Queue (CQ - see Section
   2.2.6 Completion Queues) can be caused by binding
        authentication at one channel to either the session protection at local ULP or
   the other channel.  The use of channel bindings allows
        applications to delegate session protection Remote Peer if either attempts to lower layers,
        which may significantly improve performance for some
        applications."

6.2  Requirements for IPsec Encapsulation cause more completions than
   its fair share of DDP

   The IP Storage working group has spent significant time and
   effort to define the normative IPsec requirements for IP Storage
   [RFC3723]. Portions number of entries, thus potentially
   starving another unrelated ULP such that specification no Completion Queue
   entries are applicable to available.

   A Completion Queue entry can potentially be maliciously consumed
   by a
   wide variety completion from the Send Queue or a completion from the
   Receive Queue. In the former, the attacker is the local ULP. In
   the latter, the attacker is the Remote Peer.

   A form of protocols, including attack can occur where the RDDP protocol suite. In
   order local ULPs can consume
   resources on the CQ. A local ULP that is slow to free resources
   on the CQ by not replicate this effort, reaping the completion status quickly enough
   could stall all other local ULPs attempting to use that CQ.

   For these reasons, an RNIC implementation MUST
   follow NOT enable sharing a CQ across
   ULPs that do not share Partial Mutual Trust.

7.2  Local Peer Attacking the requirements defined in RFC3723 Section 2.3 and
   Section 5, including RDMA Read Request Queue

   If RDMA Read Request Queue resources are pooled across multiple
   Streams, one attack is if the associated normative references for
   those sections.

   Additionally, since IPsec acceleration hardware may only be able local ULP attempts to handle a limited number of active IKE Phase 2 SAs, Phase 2
   delete messages may be sent unfairly
   allocate RDMA Read Request Queue resources for idle SAs, as its Streams. For
   example, a means of keeping
   the number of active Phase 2 SAs local ULP attempts to allocate all available resources
   on a minimum. The receipt of an
   IKE Phase 2 delete message MUST NOT be interpreted as a reason specific RDMA Read Request Queue for tearing down an DDP/RDMA Stream. Rather, it is preferable to
   leave its Streams, thereby
   denying the Stream up, and if additional traffic is sent on it, to
   bring up another IKE Phase 2 SA resource to protect it. This avoids ULPs sharing the
   potential for continually bringing Streams up and down.

   Note that there are serious security issues RDMA Read Request Queue.
   The same type of argument applies even if IPsec the RDMA Read Request
   is not
   implemented end-to-end. For example, if IPsec is implemented as shared - but a
   tunnel in the middle local ULP attempts to allocate all of the network, any hosts between the Peer
   and the IPsec tunneling device can freely attack
   RNIC's resources when the unprotected
   Stream.

7 queue is created.

   Thus access to interfaces that allocate RDMA Read Request Queue
   entries MUST be restricted to a trusted Local Peer, such as a
   Privileged Resource Manager. The Privileged Resource Manager
   SHOULD prevent a local ULP from allocating more than its fair
   share of resources.

8  Security considerations

   This entire document is focused on security considerations.

8

9  IANA Considerations

   IANA considerations are not addressed by this document.  Any IANA
   considerations resulting from the use of DDP or RDMA must be
   addressed in the relevant standards.

9

10 References

9.1

10.1 Normative References

   [RFC2828] Shirley, R., "Internet Security Glossary", FYI 36, RFC
       2828, May 2000.

   [DDP] Shah, H., J. Pinkerton, R.Recio, R. Recio, and P. Culley, "Direct
       Data Placement over Reliable Transports", Internet-Draft Work
       in Progress draft-ietf-rddp-ddp-04.txt, December 2004.

   [RDMAP] Recio, R., P. Culley, D. Garcia, J. Hilland, "An RDMA
       Protocol Specification", Internet-Draft Work in Progress
       draft-ietf-rddp-rdmap-03.txt, December 2004.

   [RFC3723] Aboba Aboba, B., et al, "Securing Block Storage Protocols
       over IP", Internet draft (work in progress), RFC3723, April
       2004.

   [SCTP] Stewart, R. Stewart et al., "Stream Control Transmission
       Protocol", RFC 2960, October 2000.

   [TCP] Postel, J., "Transmission Control Protocol - DARPA Internet
       Program Protocol Specification", RFC 793, September 1981.

9.2

10.2 Informative References

   [APPLICABILITY] Bestler, C. , Coene, L. "Applicability of Remote
       Direct Memory Access Protocol (RDMA) and Direct Data
       Placement (DDP)", Internet-Draft Work in Progress draft-ietf-
       rddp-applicability-05.txt, December 2005.

   [IPv6-Trust] Nikander, P., J.Kempf, E. Nordmark, "IPv6 Neighbor
       Discovery Trust Models and threats", Informational RFC,
       RFC3756, May 2004.

   [NFSv4CHANNEL] Williams, N., "On the Use of Channel Bindings to
       Secure Channels", Internet-Draft draft-ietf-nfsv4-channel-
       bindings-02.txt, July 2004.

10

   [VERBS-RDMAC] "RDMA Protocol Verbs Specification", RDMA
       Consortium standard, April 2003.
       http://www.rdmaconsortium.org/home/draft-hilland-iwarp-verbs-
       v1.0-RDMAC.pdf

   [VERBS-RDMAC-Overview] "RDMA enabled NIC (RNIC) Verbs Overview",
       slide presentation by Renato Recio, April 2003.
       http://www.rdmaconsortium.org/home/RNIC_Verbs_Overview2.pdf

   [RFC3552] "Guidelines for Writing RFC Text on Security
       Considerations", Best Current Practice RFC, RFC3552, July
       2003.

   [INFINIBAND] "InfiniBand Architecture Specification Volume 1",
       release 1.2, InfiniBand Trade Association standard.
       http://www.infinibandta.org/specs. Verbs are documented in
       chapter 11.

11 Appendix A: ULP Issues for RDDP Client/Server Protocols

   This section is a normative appendix to the document that is
   focused on client/server ULP implementation requirements to
   ensure a secure server implementation.

   The prior sections outlined specific attacks and their
   countermeasures. This section summarizes the attacks and
   countermeasures that have been defined in the prior section which
   are applicable to creation of a secure ULP (e.g. application)
   server. A ULP server is defined as a ULP which must be able to
   communicate with many clients which do not necessarily have a
   trust relationship with each other, and ensure that each client
   can not attack another client through server interactions.
   Further, the server may wish to use multiple Streams to
   communicate with a specific client, and those Streams may share
   mutual trust. Note that this section assumes a compliant RNIC and
   Privileged Resource Manager implementation - thus it focuses
   specifically on ULP server (e.g. application) implementation
   issues.

   All of the prior section's details on attacks and countermeasures
   apply to the server, thus requirements which are repeated in this
   section use non-normative "must", "should", "may". In some cases
   normative SHOULD statements for the ULP from the main body of
   this document are made MUST statements for the ULP server because
   the operating conditions can be refined to make the motives for a
   SHOULD inapplicable. If a prior SHOULD is changed to a MUST in
   this section, it is explicitly noted and it uses upper-case
   normative statements.

   The following list summarizes the relevant attacks that clients
   can mount on the shared server, by re-stating the previous
   normative statements to be client/server specific. Note that each
   client/server ULP may employ explicit RDMA operations (RDMA Read,
   RDMA Write) in differing fashions. Therefore where appropriate,
   "Local ULP", "Local Peer" and "Remote Peer" are used in place of
   "server" or "client", in order to retain full generality of each
   requirement.

       *   Spoofing

           *   Sections 5.2.1 5.1.1 to 5.2.3. 5.1.3. For protection against many
               forms of spoofing attacks, enable IPsec.

           *   Section 5.2.4 6.1.1 Using an STag on a Different Stream on
               page 20. Stream. To
               ensure that one client can not access another
               client's data via use of the other client's STag, the
               server ULP must either scope an STag to a single
               Stream or use a unique Protection Domain per client.
               If a single client has multiple Streams that share
               Partial Mutual Trust, then the STag can be shared
               between the associated Streams by using a single
               Protection Domain among the associated Streams (see
               section 6.1.3 5.4.3 ULPs Which Provide Security on
               page 38 for
               additional issues). To prevent unintended sharing of
               STags within the associated Streams, a server ULP
               should use STags in such a fashion that it is
               difficult to predict the next allocated STag number.

       *   Tampering

           *   5.3.2   6.2.2 Modifying a Buffer After Indication on page 22. Indication. Before the
               local ULP operates on a buffer that was written by
               the Remote Peer using an RDMA Write or RDMA Read, the
               local ULP MUST ensure the buffer can no longer be
               modified, by invalidating the STag for remote access
               (note that this is stronger than the SHOULD in
               section 5.3.2). 6.2.2). This can either be done explicitly by
               revoking remote access rights for the STag when the
               Remote Peer indicates the operation has completed, or
               by checking to make sure the Remote Peer Invalidated
               the STag through the RDMAP Invalidate capability, and
               if it did not, the local ULP then explicitly revoking
               the STag remote access rights.

       *   Information Disclosure

           *   5.4.2   6.3.2 Using RDMA Read to Access Stale Data on page
               23. Data. In a
               general purpose server environment there is no
               compelling rationale to not require a buffer to be
               initialized before remote read is enabled (and an
               enormous down side of unintentionally sharing data).
               Thus a local ULP MUST (this is stronger than the
               SHOULD in section 5.4.2) 6.3.2) ensure that no stale data is
               contained in a buffer before remote read access
               rights are granted to a Remote Peer (this can be done
               by zeroing the contents of the memory, for example).

           *   5.4.3   6.3.3 Accessing a Buffer After the Transfer on page
               24. Transfer. This
               mitigation is already covered by section
               5.3.2 6.2.2
               (above).

           *   5.4.4   6.3.4 Accessing Unintended Data With a Valid STag on
               page 24. STag.
               The ULP must set the base and bounds of the buffer
               when the STag is initialized to expose only the data
               to be retrieved.

           *   5.4.5   6.3.5 RDMA Read into an RDMA Write Buffer on page 24. Buffer. If a peer
               only intends a buffer to be exposed for remote write
               access, it must set the access rights to the buffer
               to only enable remote write access.

           *   5.4.6   6.3.6 Using Multiple STags Which Alias to the Same
               Buffer on page 25.
               Buffer. The requirement in section 5.2.4 6.1.1 (above)
               mitigates this attack. A server buffer is exposed to
               only one client at a time to ensure that no
               information disclosure or information tampering
               occurs between peers.

           *   5.4.9   5.3  - Network based eavesdropping on page 26. Based Eavesdropping. Confidentiality
               services should be enabled by the ULP if this threat
               is a concern.

       *   Denial of Service

           *   5.5.2.1   6.4.3.1 Multiple Streams Sharing Receive Buffers on
               page 27. Buffers. ULP
               memory footprint size can be important for some
               server ULPs. If a server ULP is expecting significant
               network traffic from multiple clients, using a
               receive buffer queue per Stream where there is a
               large number of Streams can consume substantial
               amounts of memory. Thus a receive queue that can be
               shared by multiple Streams is attractive.

               However, because of the attacks outlined in this
               section, sharing a single receive queue between
               multiple clients must only be done if a mechanism is
               in place to ensure one client cannot consume receive
               buffers in excess of its limits, as defined by each
               ULP. For multiple Streams within a single client ULP
               (which presumably shared Partial Mutual Trust) this
               added overhead may be avoided.

           *   5.5.2.2   7.1 Local ULP Attacking a Shared CQ on page 29. CQ. The normative
               RNIC mitigations require the RNIC to not enable
               sharing of a CQ if the local ULPs do not share
               Partial Mutual Trust. Thus while the ULP is not
               allowed to enable this feature in an unsafe mode, if
               the two local ULPs share Partial Tutual Mutual Trust, they
               must behave in the following manner:

               1) The sizing of the completion queue is based on the
               size of the receive queue and send queues as
               documented in 5.5.2.3 Local or 6.4.3.2 Remote or Local Peer Attacking
               a Shared CQ on page 29. CQ.

               2) The local ULP ensures that CQ entries are reaped
               frequently enough to adhere to section 5.5.2.3's 6.4.3.2's
               rules.

           *   5.5.2.3 Local or   6.4.3.2 Remote or Local Peer Attacking a Shared CQ on
               page 29. CQ.
               There are two mitigations specified in this section -
               one requires a worst-case size of the CQ, and can be
               implemented entirely within the Privileged Resource
               Manager. The second approach requires cooperation
               with the local ULP server (to not post too many
               buffers), and enables a smaller CQ to be used.

               In some server environments, partial trust of the
               server ULP (but not the clients) is acceptable, thus
               the smaller CQ fully mitigates the remote attacker.
               In other environments, the local server ULP could
               also contain untrusted elements which can attack the
               local machine (or have bugs). In those environments,
               the worst-case size of the CQ must be used.

           *   5.5.2.4   6.4.3.3 The section requires a server's Privileged
               Resource Manager to not allow sharing of RDMA Read
               Request Queues across multiple Streams that do not
               share Partial Mutual Trust, for a ULP which performs
               RDMA Read operations to server buffers. However,
               because the server ULP knows best which of its
               Streams share Partial Mutual Trust, this requirement
               can be reflected back to the ULP. The ULP (i.e.
               server) requirement in this case is that it MUST NOT
               allow RDMA Read Request Queues to be shared between
               ULPs which do not have Partial Mutual Trust.

           *   5.5.5   6.4.5 Remote Invalidate an STag Shared on Multiple
               Streams on page 34.
               Streams. This mitigation is already covered by
               section 5.3.2 6.2.2 (above).

11

12 Appendix B: Summary of RNIC and ULP Implementation Requirements

   This appendix is informative.

   Below is a summary of implementation requirements for the RNIC:

       *   3 Trust and Resource Sharing

       *   5.2.4   6.1.1 Using an STag on a Different Stream

       *   5.3.1   6.2.1 Buffer Overrun - RDMA Write or Read Response

       *   5.3.2   6.2.2 Modifying a Buffer After Indication

       *   5.4.8   6.3.7 Controlling Access to PTT & STag Mapping

       *   5.5.1   6.4.1 RNIC Resource Consumption

       *   5.5.2.1   6.4.3.1 Multiple Streams Sharing Receive Buffers

       *   5.5.2.2   7.1 Local ULP Attacking a Shared CQ

       *   5.5.2.3 Local or   6.4.3.2 Remote or Local Peer Attacking a Shared CQ

       *   5.5.2.4   6.4.3.3 Attacking the RDMA Read Request Queue

       *   5.5.6   6.4.6 Remote Peer attacking an Unshared CQ on page 34. CQ.

       *   5.6   6.5 Elevation of Privilege 35 38

       *   6.2   5.4.4 Requirements for IPsec Encapsulation of DDP

   Below is a summary of implementation requirements for the ULP
   above the RNIC:

       *   5.2.4   6.1.1 Using an STag on a Different Stream

       *   5.3.2   6.2.2 Modifying a Buffer After Indication

       *   5.4.2   6.3.2 Using RDMA Read to Access Stale Data

       *   5.4.3   6.3.3 Accessing a Buffer After the Transfer

       *   5.4.4   6.3.4 Accessing Unintended Data With a Valid STag

       *   5.4.5   6.3.5 RDMA Read into an RDMA Write Buffer

       *   5.4.6   6.3.6 Using Multiple STags Which Alias to the Same Buffer

       *   5.4.9   5.3  - Network based eavesdropping Based Eavesdropping

       *   5.5.2.2   7.1 Local ULP Attacking a Shared CQ
       *   5.5.5   6.4.5 Remote Invalidate an STag Shared on Multiple
           Streams

12

13 Appendix C: Partial Trust Taxonomy

   This appendix is informative.

   Partial Trust is defined as when one party is willing to assume
   that another party will refrain from a specific attack or set of
   attacks, the parties are said to be in a state of Partial Trust.
   Note that the partially trusted peer may attempt a different set
   of attacks. This may be appropriate for many ULPs where any
   adverse effects of the betrayal is easily confined and does not
   place other clients or ULPs at risk.

   The Trust Models described in this section have three primary
   distinguishing characteristics. The Trust Model refers to a local
   ULP and Remote Peer, which are intended to be the local and
   remote ULP instances communicating via RDMA/DDP.

       *   Local Resource Sharing (yes/no) - When local resources
           are shared, they are shared across a grouping of
           RDMAP/DDP Streams. If local resources are not shared, the
           resources are dedicated on a per Stream basis. Resources
           are defined in Section 2.2 - Resources on page 8. Resources. The advantage of
           not sharing resources between Streams is that it reduces
           the types of attacks that are possible. The disadvantage
           is that ULPs might run out of resources.

       *   Local Partial Trust (yes/no) - Local Partial Trust is
           determined based on whether the local grouping of
           RDMAP/DDP Streams (which typically equates to one ULP or
           group of ULPs) mutually trust each other to not perform a
           specific set of attacks.

       *   Remote Partial Trust (yes/no) - The Remote Partial Trust
           level is determined based on whether the local ULP of a
           specific RDMAP/DDP Stream partially trusts the Remote
           Peer of the Stream (see the definition of Partial Trust
           in Section 1 Introduction).

   Not all of the combinations of the trust characteristics are
   expected to be used by ULPs. This document specifically analyzes
   five ULP Trust Models that are expected to be in common use. The
   Trust Models are as follows:

       *   NS-NT - Non-Shared Local Resources, no Local Trust, no
           Remote Trust - typically a server ULP that wants to run
           in the safest mode possible. All attack mitigations are
           in place to ensure robust operation.

       *   NS-RT - Non-Shared Local Resources, no Local Trust,
           Remote Partial Trust - typically a peer-to-peer ULP,
           which has, by some method outside of the scope of this
           document, authenticated the Remote Peer. Note that unless
           some form of key based authentication is used on a per
           RDMA/DDP Stream basis, it may not be possible be possible
           for man-in-the-middle attacks to occur. See section 6,
           Security Services for RDMAP and DDP on page 36.

       *   S-NT - Shared Local Resources, no Local Trust, no Remote
           Trust - typically a server ULP that runs in an untrusted
           environment where the amount of resources required is
           either too large or too dynamic to dedicate for each
           RDMAP/DDP Stream.

       *   S-LT - Shared Local Resources, Local Partial Trust, no
           Remote Trust - typically a ULP, which provides a session
           layer and uses multiple Streams, to provide additional
           throughput or fail-over capabilities. All of the Streams
           within the local ULP partially trust each other, but do
           not trust the Remote Peer. This trust model may be
           appropriate for embedded environments.

       *   S-T - Shared Local Resources, Local Partial Trust, Remote
           Partial Trust - typically a distributed application, such
           as a distributed database application or a High
           Performance Computer (HPC) application, which is intended
           to run on a cluster. Due to extreme resource and
           performance requirements, the application typically
           authenticates with all of its peers and then runs in a
           highly trusted environment. The application peers are all
           in a single application fault domain and depend on one
           another to be well-behaved when accessing data
           structures. If a trusted Remote Peer has an
           implementation defect that results in poor behavior, the
           entire application could be corrupted.

   Models NS-NT and S-NT above are typical for Internet networking -
   neither local ULPs nor the Remote Peer is trusted. Sometimes
   optimizations can be done that enable sharing of Page Translation
   Tables across multiple local ULPs, thus Model S-LT can be
   advantageous. Model S-T is typically used when resource scaling
   across a large parallel ULP makes it infeasible to use any other
   model. Resource scaling issues can either be due to performance
   around scaling or because there simply are not enough resources.
   Model NS-RT is probably the least likely model to be used, but is
   presented for completeness.

13

14 Author's Addresses

   James Pinkerton
   Microsoft Corporation
   One Microsoft Way
   Redmond, WA. 98052 USA
   Phone: +1 (425) 705-5442
   Email: jpink@windows.microsoft.com

   Ellen Deleganes
   Intel Corporation
   MS JF5-355
   2111 NE 25th Ave.
   Hillsboro, OR 97124 USA
   Phone: +1 (503) 712-4173
   Email: ellen.m.deleganes@intel.com

   Sara Bitan
   Microsoft Corporation
   Email: sarab@microsoft.com

14

15 Acknowledgments

   Allyn Romanow
   Cisco Systems
   170 W Tasman Drive
   San Jose, CA 95134 USA
   Phone: +1 408 525 8836
   Email: allyn@cisco.com

   Catherine Meadows
   Naval Research Laboratory
   Code 5543
   Washington, DC 20375
   Email: meadows@itd.nrl.navy.mil

   Patricia Thaler
   Agilent Technologies, Inc.
   1101 Creekside Ridge Drive, #100
   M/S-RG10
   Roseville, CA 95678
   Phone: +1-916-788-5662
   email: pat_thaler@agilent.com

   James Livingston
   NEC Solutions (America), Inc.
   7525 166th Ave. N.E., Suite D210
   Redmond, WA 98052-7811
   Phone: +1 (425) 897-2033
   Email: james.livingston@necsam.com

   John Carrier
   Adaptec, Inc.
   691 S. Milpitas Blvd.
   Milpitas, CA 95035 USA
   Phone: +1 (360) 378-8526
   Email: john_carrier@adaptec.com

   Caitlin Bestler
   Broadcom
   49 Discovery
   Irvine, CA  92618
   Email: cait@asomi.com

   Bernard Aboba
   Microsoft Corporation
   One Microsoft Way
   Redmond, WA. 98052 USA
   Phone: +1 (425) 706-6606
   Email: bernarda@windows.microsoft.com

15

16 Full Copyright Statement

   Copyright (C) The Internet Society (2005). (2006).

   This document is subject to the rights, licenses and restrictions
   contained in BCP 78, and except as set forth therein, the authors
   retain all their rights.

   This document and the information contained herein are provided
   on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
   REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND
   THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES,
   EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY
   THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY
   RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
   FOR A PARTICULAR PURPOSE.

   Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be
   claimed to pertain to the implementation or use of the technology
   described in this document or the extent to which any license
   under such rights might or might not be available; nor does it
   represent that it has made any independent effort to identify any
   such rights.  Information on the procedures with respect to
   rights in RFC documents can be found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the
   use of such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR
   repository at http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention
   any copyrights, patents or patent applications, or other
   proprietary rights that may cover technology that may be required
   to implement this standard.  Please address the information to
   the IETF at ietf-ipr@ietf.org.

   Acknowledgement

   Funding for the RFC Editor function is currently provided by the
   Internet Society.