draft-ietf-nfsv4-rfc5666bis-10.txt   draft-ietf-nfsv4-rfc5666bis-11.txt 
Network File System Version 4 C. Lever, Ed. Network File System Version 4 C. Lever, Ed.
Internet-Draft Oracle Internet-Draft Oracle
Obsoletes: 5666 (if approved) W. Simpson Obsoletes: 5666 (if approved) W. Simpson
Intended status: Standards Track DayDreamer Intended status: Standards Track Red Hat
Expires: August 12, 2017 T. Talpey Expires: September 28, 2017 T. Talpey
Microsoft Microsoft
February 8, 2017 March 27, 2017
Remote Direct Memory Access Transport for Remote Procedure Call, Version Remote Direct Memory Access Transport for Remote Procedure Call, Version
One One
draft-ietf-nfsv4-rfc5666bis-10 draft-ietf-nfsv4-rfc5666bis-11
Abstract Abstract
This document specifies a protocol for conveying Remote Procedure This document specifies a protocol for conveying Remote Procedure
Call (RPC) messages on physical transports capable of Remote Direct Call (RPC) messages on physical transports capable of Remote Direct
Memory Access (RDMA). It requires no revision to application RPC Memory Access (RDMA). It requires no revision to application RPC
protocols or the RPC protocol itself. This document obsoletes RFC protocols or the RPC protocol itself. This document obsoletes RFC
5666. 5666.
Requirements Language Requirements Language
skipping to change at page 1, line 44 skipping to change at page 1, line 44
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 12, 2017. This Internet-Draft will expire on September 28, 2017.
Copyright Notice Copyright Notice
Copyright (c) 2017 IETF Trust and the persons identified as the Copyright (c) 2017 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 44 skipping to change at page 2, line 44
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Remote Procedure Calls On RDMA Transports . . . . . . . . 3 1.1. Remote Procedure Calls On RDMA Transports . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Remote Procedure Calls . . . . . . . . . . . . . . . . . 4 2.1. Remote Procedure Calls . . . . . . . . . . . . . . . . . 4
2.2. Remote Direct Memory Access . . . . . . . . . . . . . . . 7 2.2. Remote Direct Memory Access . . . . . . . . . . . . . . . 7
3. RPC-Over-RDMA Protocol Framework . . . . . . . . . . . . . . 9 3. RPC-Over-RDMA Protocol Framework . . . . . . . . . . . . . . 9
3.1. Transfer Models . . . . . . . . . . . . . . . . . . . . . 9 3.1. Transfer Models . . . . . . . . . . . . . . . . . . . . . 9
3.2. Message Framing . . . . . . . . . . . . . . . . . . . . . 10 3.2. Message Framing . . . . . . . . . . . . . . . . . . . . . 10
3.3. Managing Receiver Resources . . . . . . . . . . . . . . . 11 3.3. Managing Receiver Resources . . . . . . . . . . . . . . . 11
3.4. XDR Encoding With Chunks . . . . . . . . . . . . . . . . 13 3.4. XDR Encoding With Chunks . . . . . . . . . . . . . . . . 13
3.5. Message Size . . . . . . . . . . . . . . . . . . . . . . 18 3.5. Message Size . . . . . . . . . . . . . . . . . . . . . . 19
4. RPC-Over-RDMA In Operation . . . . . . . . . . . . . . . . . 22 4. RPC-Over-RDMA In Operation . . . . . . . . . . . . . . . . . 22
4.1. XDR Protocol Definition . . . . . . . . . . . . . . . . . 22 4.1. XDR Protocol Definition . . . . . . . . . . . . . . . . . 23
4.2. Fixed Header Fields . . . . . . . . . . . . . . . . . . . 27 4.2. Fixed Header Fields . . . . . . . . . . . . . . . . . . . 27
4.3. Chunk Lists . . . . . . . . . . . . . . . . . . . . . . . 29 4.3. Chunk Lists . . . . . . . . . . . . . . . . . . . . . . . 29
4.4. Memory Registration . . . . . . . . . . . . . . . . . . . 32 4.4. Memory Registration . . . . . . . . . . . . . . . . . . . 32
4.5. Error Handling . . . . . . . . . . . . . . . . . . . . . 33 4.5. Error Handling . . . . . . . . . . . . . . . . . . . . . 33
4.6. Protocol Elements No Longer Supported . . . . . . . . . . 36 4.6. Protocol Elements No Longer Supported . . . . . . . . . . 36
4.7. XDR Examples . . . . . . . . . . . . . . . . . . . . . . 37 4.7. XDR Examples . . . . . . . . . . . . . . . . . . . . . . 37
5. RPC Bind Parameters . . . . . . . . . . . . . . . . . . . . . 38 5. RPC Bind Parameters . . . . . . . . . . . . . . . . . . . . . 38
6. Upper Layer Binding Specifications . . . . . . . . . . . . . 40 6. Upper Layer Binding Specifications . . . . . . . . . . . . . 40
6.1. DDP-Eligibility . . . . . . . . . . . . . . . . . . . . . 40 6.1. DDP-Eligibility . . . . . . . . . . . . . . . . . . . . . 40
6.2. Maximum Reply Size . . . . . . . . . . . . . . . . . . . 41 6.2. Maximum Reply Size . . . . . . . . . . . . . . . . . . . 42
6.3. Additional Considerations . . . . . . . . . . . . . . . . 42 6.3. Additional Considerations . . . . . . . . . . . . . . . . 42
6.4. Upper Layer Protocol Extensions . . . . . . . . . . . . . 42 6.4. Upper Layer Protocol Extensions . . . . . . . . . . . . . 43
7. Protocol Extensibility . . . . . . . . . . . . . . . . . . . 43 7. Protocol Extensibility . . . . . . . . . . . . . . . . . . . 43
7.1. Conventional Extensions . . . . . . . . . . . . . . . . . 43 7.1. Conventional Extensions . . . . . . . . . . . . . . . . . 43
8. Security Considerations . . . . . . . . . . . . . . . . . . . 43 8. Security Considerations . . . . . . . . . . . . . . . . . . . 44
8.1. Memory Protection . . . . . . . . . . . . . . . . . . . . 43 8.1. Memory Protection . . . . . . . . . . . . . . . . . . . . 44
8.2. RPC Message Security . . . . . . . . . . . . . . . . . . 44 8.2. RPC Message Security . . . . . . . . . . . . . . . . . . 45
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 47 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 48
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 48 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 49
10.1. Normative References . . . . . . . . . . . . . . . . . . 48 10.1. Normative References . . . . . . . . . . . . . . . . . . 49
10.2. Informative References . . . . . . . . . . . . . . . . . 49 10.2. Informative References . . . . . . . . . . . . . . . . . 50
Appendix A. Changes Since RFC 5666 . . . . . . . . . . . . . . . 51 Appendix A. Changes Since RFC 5666 . . . . . . . . . . . . . . . 52
A.1. Changes To The Specification . . . . . . . . . . . . . . 51 A.1. Changes To The Specification . . . . . . . . . . . . . . 52
A.2. Changes To The Protocol . . . . . . . . . . . . . . . . . 51 A.2. Changes To The Protocol . . . . . . . . . . . . . . . . . 52
Appendix B. Acknowledgments . . . . . . . . . . . . . . . . . . 52 Appendix B. Acknowledgments . . . . . . . . . . . . . . . . . . 53
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 53 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 53
1. Introduction 1. Introduction
This document specifies the RPC-over-RDMA Version One protocol, based This document specifies the RPC-over-RDMA Version One protocol, based
on existing implementations of RFC 5666 and experience gained through on existing implementations of RFC 5666 and experience gained through
deployment. This document obsoletes RFC 5666. deployment. This document obsoletes RFC 5666.
The new specification clarifies text that is subject to multiple The new specification clarifies text that is subject to multiple
interpretations, and removes support for unimplemented RPC-over-RDMA interpretations, and removes support for unimplemented RPC-over-RDMA
skipping to change at page 4, line 22 skipping to change at page 4, line 22
must be fully described if RPC implementations are to interoperate. must be fully described if RPC implementations are to interoperate.
RDMA transports present semantics different from either UDP or TCP. RDMA transports present semantics different from either UDP or TCP.
They retain message delineations like UDP, but provide reliable and They retain message delineations like UDP, but provide reliable and
sequenced data transfer like TCP. They also provide an offloaded sequenced data transfer like TCP. They also provide an offloaded
bulk transfer service not provided by UDP or TCP. RDMA transports bulk transfer service not provided by UDP or TCP. RDMA transports
are therefore appropriately viewed as a new transport type by RPC. are therefore appropriately viewed as a new transport type by RPC.
In this context, the Network File System (NFS) protocols as described In this context, the Network File System (NFS) protocols as described
in [RFC1094], [RFC1813], [RFC7530], [RFC5661], and future NFSv4 minor in [RFC1094], [RFC1813], [RFC7530], [RFC5661], and future NFSv4 minor
verions are all obvious beneficiaries of RDMA transports. A complete versions are all obvious beneficiaries of RDMA transports. A
problem statement is presented in [RFC5532]. Many other RPC-based complete problem statement is presented in [RFC5532]. Many other
protocols can also benefit. RPC-based protocols can also benefit.
Although the RDMA transport described herein can provide relatively Although the RDMA transport described herein can provide relatively
transparent support for any RPC application, this document also transparent support for any RPC application, this document also
describes mechanisms that can optimize data transfer even further, describes mechanisms that can optimize data transfer even further,
when RPC applications are willing to exploit awareness of RDMA as the when RPC applications are willing to exploit awareness of RDMA as the
transport. transport.
2. Terminology 2. Terminology
2.1. Remote Procedure Calls 2.1. Remote Procedure Calls
skipping to change at page 15, line 27 skipping to change at page 15, line 27
sender's Payload stream, transferred via separate operations, and sender's Payload stream, transferred via separate operations, and
then re-inserted into the receiver's Payload stream to form a then re-inserted into the receiver's Payload stream to form a
complete RPC message. complete RPC message.
Each chunk consists of one or more RDMA segments. Each RDMA segment Each chunk consists of one or more RDMA segments. Each RDMA segment
represents a single contiguous piece of that chunk. A requester MAY represents a single contiguous piece of that chunk. A requester MAY
divide a chunk into RDMA segments using any boundaries that are divide a chunk into RDMA segments using any boundaries that are
convenient. The length of a chunk is the sum of the lengths of the convenient. The length of a chunk is the sum of the lengths of the
RDMA segments that comprise it. RDMA segments that comprise it.
The RPC-over-RDMA Version One transport protocol does not place a
limit on chunk size. However, each Upper Layer Protocol may cap the
amount of data that can be transferred by a single RPC (for example,
NFS has "rsize" and "wsize", which restrict the payload size of NFS
READ and WRITE operations). The responder can use such limits to
sanity check chunk sizes before using them in RDMA operations.
3.4.4.1. Counted Arrays 3.4.4.1. Counted Arrays
If a chunk contains a counted array data type, the count of array If a chunk contains a counted array data type, the count of array
elements MUST remain in the Payload stream, while the array elements elements MUST remain in the Payload stream, while the array elements
MUST be moved to the chunk. For example, when encoding an opaque MUST be moved to the chunk. For example, when encoding an opaque
byte array as a chunk, the count of bytes stays in the Payload byte array as a chunk, the count of bytes stays in the Payload
stream, while the bytes in the array are removed from the Payload stream, while the bytes in the array are removed from the Payload
stream and transferred within the chunk. stream and transferred within the chunk.
Individual array elements appear in a chunk in their entirety. For Individual array elements appear in a chunk in their entirety. For
skipping to change at page 39, line 16 skipping to change at page 39, line 16
[RFC1833], which associates an RPC Program number with a service [RFC1833], which associates an RPC Program number with a service
address. This policy is no different with RDMA transports. However, address. This policy is no different with RDMA transports. However,
a different and distinct service address (port number) might a different and distinct service address (port number) might
sometimes be required for Upper Layer Protocol operation with RPC- sometimes be required for Upper Layer Protocol operation with RPC-
over-RDMA. over-RDMA.
When mapped atop the iWARP transport [RFC5040] [RFC5041], which uses When mapped atop the iWARP transport [RFC5040] [RFC5041], which uses
IP port addressing due to its layering on TCP and/or SCTP, port IP port addressing due to its layering on TCP and/or SCTP, port
mapping is trivial and consists merely of issuing the port in the mapping is trivial and consists merely of issuing the port in the
connection process. The NFS/RDMA protocol service address has been connection process. The NFS/RDMA protocol service address has been
assigned port 20049 by IANA, for both iWARP/TCP and iWARP/SCTP. assigned port 20049 by IANA, for both iWARP/TCP and iWARP/SCTP
[RFC5667].
When mapped atop InfiniBand [IB], which uses a Group Identifier When mapped atop InfiniBand [IB], which uses a Group Identifier
(GID)-based service endpoint naming scheme, a translation MUST be (GID)-based service endpoint naming scheme, a translation MUST be
employed. One such translation is defined in the InfiniBand Port employed. One such translation is defined in the InfiniBand Port
Addressing Annex [IBPORT], which is appropriate for translating IP Addressing Annex [IBPORT], which is appropriate for translating IP
port addressing to the InfiniBand network. Therefore, in this case, port addressing to the InfiniBand network. Therefore, in this case,
IP port addressing may be readily employed by the upper layer. IP port addressing may be readily employed by the upper layer.
When a mapping standard or convention exists for IP ports on an RDMA When a mapping standard or convention exists for IP ports on an RDMA
interconnect, there are several possibilities for each upper layer to interconnect, there are several possibilities for each upper layer to
skipping to change at page 40, line 50 skipping to change at page 41, line 5
directly in the receiver's memory. directly in the receiver's memory.
An XDR data item should be considered for DDP-eligibility if there is An XDR data item should be considered for DDP-eligibility if there is
a clear benefit to moving the contents of the item directly from the a clear benefit to moving the contents of the item directly from the
sender's memory to the receiver's memory. Criteria for DDP- sender's memory to the receiver's memory. Criteria for DDP-
eligibility include: eligibility include:
o The XDR data item is frequently sent or received, and its size is o The XDR data item is frequently sent or received, and its size is
often much larger than typical inline thresholds. often much larger than typical inline thresholds.
o If the XDR data item is a result, its maximum size must be
predictable in advance by the requester.
o Transport-level processing of the XDR data item is not needed. o Transport-level processing of the XDR data item is not needed.
For example, the data item is an opaque byte array, which requires For example, the data item is an opaque byte array, which requires
no XDR encoding and decoding of its content. no XDR encoding and decoding of its content.
o The content of the XDR data item is sensitive to address o The content of the XDR data item is sensitive to address
alignment. For example, pullup would be required on the receiver alignment. For example, pullup would be required on the receiver
before the content of the item can be used. before the content of the item can be used.
o The XDR data item does not contain DDP-eligible data items. o The XDR data item does not contain DDP-eligible data items.
skipping to change at page 43, line 50 skipping to change at page 44, line 10
example of a conventional extension to RPC-over-RDMA Version One is example of a conventional extension to RPC-over-RDMA Version One is
the specification of backward direction message support to enable the specification of backward direction message support to enable
NFSv4.1 callback operations, described in NFSv4.1 callback operations, described in
[I-D.ietf-nfsv4-rpcrdma-bidirection]. [I-D.ietf-nfsv4-rpcrdma-bidirection].
8. Security Considerations 8. Security Considerations
8.1. Memory Protection 8.1. Memory Protection
A primary consideration is the protection of the integrity and A primary consideration is the protection of the integrity and
privacy of local memory by an RPC-over-RDMA transport. The use of confidentiality of local memory by an RPC-over-RDMA transport. The
RPC-over-RDMA MUST NOT introduce any vulnerabilities to system memory use of an RPC-over-RDMA transport protocol MUST NOT introduce
contents, nor to memory owned by user processes. vulnerabilities to system memory contents nor to memory owned by user
processes.
It is REQUIRED that any RDMA provider used for RPC transport be It is REQUIRED that any RDMA provider used for RPC transport be
conformant to the requirements of [RFC5042] in order to satisfy these conformant to the requirements of [RFC5042] in order to satisfy these
protections. These protections are provided by the RDMA layer protections. These protections are provided by the RDMA layer
specifications, and in particular, their security models. specifications, and in particular, their security models.
8.1.1. Protection Domains 8.1.1. Protection Domains
The use of Protection Domains to limit the exposure of memory regions The use of Protection Domains to limit the exposure of memory regions
to a single connection is critical. Any attempt by an endpoint not to a single connection is critical. Any attempt by an endpoint not
skipping to change at page 44, line 48 skipping to change at page 45, line 9
is done, and before the RPC consumer is allowed to continue execution is done, and before the RPC consumer is allowed to continue execution
and use or alter the contents of a memory region. and use or alter the contents of a memory region.
An RPC transaction on a requester might be terminated before a reply An RPC transaction on a requester might be terminated before a reply
arrives if the RPC consumer exits unexpectedly (for example it is arrives if the RPC consumer exits unexpectedly (for example it is
signaled or a segmentation fault occurs). When an RPC terminates signaled or a segmentation fault occurs). When an RPC terminates
abnormally, memory regions associated with that RPC should be abnormally, memory regions associated with that RPC should be
invalidated appropriately before the regions are released to be invalidated appropriately before the regions are released to be
reused for other purposes on the requester. reused for other purposes on the requester.
8.1.4. Denial of Service
A detailed discussion of denial of service exposures that can result
from the use of an RDMA transport is found in Section 6.4 of
[RFC5042].
A responder is not obliged to pull Read chunks that are unreasonably
large. The responder can use an RDMA_ERROR response to terminate
RPCs with unreadable Read chunks. If a responder transmits more data
than a requester is prepared to receive in a Write or Reply chunk,
the RNICs typically terminate the connection. For further
discussion, see Section 4.5. Such repeated chunk errors can deny
service to other users sharing the connection from the errant
requester.
An RPC-over-RDMA transport implemention is not responsible for
throttling the RPC request rate, other than to keep the number of
concurrent RPC transactions at or under the number of credits granted
per connection. This is explained in Section 3.3.1. A sender can
trigger a self denial of service by exceeding the credit grant
repeatedly.
When an RPC has been canceled due to a signal or premature exit of an
application process, a requester may invalidate the RPC's Write and
Reply chunks. Invalidation prevents the subsequent arrival of the
responder's reply from altering the memory regions associated with
those chunks after the memory has been reused.
On the requester, a malfunctioning application or a malicious user
can create a situation where RPCs are continuously initiated and then
aborted, resulting in responder replies that terminate the underlying
RPC-over-RDMA connection repeatedly. Such situations can deny
service to other users sharing the connection from that requester.
8.2. RPC Message Security 8.2. RPC Message Security
ONC RPC provides cryptographic security via the RPCSEC_GSS framework ONC RPC provides cryptographic security via the RPCSEC_GSS framework
[RFC7861]. RPCSEC_GSS implements message authentication, per-message [RFC7861]. RPCSEC_GSS implements message authentication
integrity checking, and per-message confidentiality. However, (rpc_gss_svc_none), per-message integrity checking
integrity and privacy services require significant movement of data (rpc_gss_svc_integrity), and per-message confidentiality
on each endpoint host. Some performance benefits enabled by RDMA (rpc_gss_svc_privacy) in the layer above RPC-over-RDMA. The latter
two services require significant computation and movement of data on
each endpoint host. Some performance benefits enabled by RDMA
transports can be lost. transports can be lost.
8.2.1. RPC-Over-RDMA Protection At Lower Layers 8.2.1. RPC-Over-RDMA Protection At Lower Layers
Note that performance loss is expected when RPCSEC_GSS integrity or Performance loss is expected when RPCSEC_GSS integrity or privacy
privacy is in use on any RPC transport. Protection below the RDMA services are in use on any RPC transport. Protection below the RPC
layer is a more appropriate security mechanism for RDMA transports in transport is often more appropriate in performance-sensitive
performance-sensitive deployments. Certain configurations of IPsec deployments, especially if it, too, can be offloaded. Certain
can be co-located in RDMA hardware, for example, without any change configurations of IPsec can be co-located in RDMA hardware, for
to RDMA consumers or loss of data movement efficiency. example, without change to RDMA consumers and little loss of data
movement efficiency. Such arrangements can also provide a higher
degree of privacy by hiding endpoint identity or altering the
frequency at which messages are exchanged, at a performance cost.
The use of protection in a lower layer MAY be negotiated through the The use of protection in a lower layer MAY be negotiated through the
use of an RPCSEC_GSS security flavor defined in [RFC7861] in use of an RPCSEC_GSS security flavor defined in [RFC7861] in
conjunction with the Channel Binding mechanism [RFC5056] and IPsec conjunction with the Channel Binding mechanism [RFC5056] and IPsec
Channel Connection Latching [RFC5660]. Use of such mechanisms is Channel Connection Latching [RFC5660]. Use of such mechanisms is
REQUIRED where integrity and/or privacy is desired and where REQUIRED where integrity or confidentiality is desired and where
efficiency is required. efficiency is required.
8.2.2. RPCSEC_GSS On RPC-Over-RDMA Transports 8.2.2. RPCSEC_GSS On RPC-Over-RDMA Transports
Not all RDMA devices and fabrics support the above protection Not all RDMA devices and fabrics support the above protection
mechanisms. Also, per-message authentication is still required on mechanisms. Also, per-message authentication is still required on
NFS clients where multiple users access NFS files. In these cases, NFS clients where multiple users access NFS files. In these cases,
RPCSEC_GSS can protect NFS traffic conveyed on RPC-over-RDMA RPCSEC_GSS can protect NFS traffic conveyed on RPC-over-RDMA
connections. connections.
skipping to change at page 45, line 47 skipping to change at page 46, line 49
As part of the ONC RPC protocol, protocol elements of RPCSEC_GSS that As part of the ONC RPC protocol, protocol elements of RPCSEC_GSS that
appear in the Payload stream of an RPC-over-RDMA message (such as appear in the Payload stream of an RPC-over-RDMA message (such as
control messages exchanged as part of establishing or destroying a control messages exchanged as part of establishing or destroying a
security context, or data items that are part of RPCSEC_GSS security context, or data items that are part of RPCSEC_GSS
authentication material) MUST NOT be reduced. authentication material) MUST NOT be reduced.
8.2.2.1. RPCSEC_GSS Context Negotiation 8.2.2.1. RPCSEC_GSS Context Negotiation
Some NFS client implementations use a separate connection to Some NFS client implementations use a separate connection to
establish a GSS context for NFS operation. These clients use TCP and establish a GSS context for NFS operation. These clients use TCP and
the standard NFS port (2049) for context establishment. However the standard NFS port (2049) for context establishment. To enable
there is no guarantee that an NFS/RDMA server provides a TCP-based the use of RPCSEC_GSS with NFS/RDMA, an NFS server MUST also provide
NFS server on port 2049. a TCP-based NFS service on port 2049.
8.2.2.2. RPC-Over-RDMA With RPCSEC_GSS Authentication 8.2.2.2. RPC-Over-RDMA With RPCSEC_GSS Authentication
The RPCSEC_GSS authentication service has no impact on the DDP- The RPCSEC_GSS authentication service has no impact on the DDP-
eligibity of data items in an Upper Layer Protocol. eligibity of data items in an Upper Layer Protocol.
However, RPCSEC_GSS authentication material appearing in an RPC However, RPCSEC_GSS authentication material appearing in an RPC
message header can be larger than, say, an AUTH_SYS authenticator. message header can be larger than, say, an AUTH_SYS authenticator.
In particular, when an RPCSEC_GSS pseudoflavor is in use, a requester In particular, when an RPCSEC_GSS pseudoflavor is in use, a requester
needs to accommodate a larger RPC credential when marshaling Call needs to accommodate a larger RPC credential when marshaling Call
skipping to change at page 46, line 32 skipping to change at page 47, line 32
and a Reply chunk in the same RPC-over-RDMA header to convey a Long and a Reply chunk in the same RPC-over-RDMA header to convey a Long
call and provision a receptacle for a Long reply. More frequent use call and provision a receptacle for a Long reply. More frequent use
of Long messages can impact transport efficiency. of Long messages can impact transport efficiency.
8.2.2.3. RPC-Over-RDMA With RPCSEC_GSS Integrity Or Privacy 8.2.2.3. RPC-Over-RDMA With RPCSEC_GSS Integrity Or Privacy
The RPCSEC_GSS integrity service enables endpoints to detect The RPCSEC_GSS integrity service enables endpoints to detect
modification of RPC messages in flight. The RPCSEC_GSS privacy modification of RPC messages in flight. The RPCSEC_GSS privacy
service prevents all but the intended recipient from viewing the service prevents all but the intended recipient from viewing the
cleartext content of RPC arguments and results. RPCSEC_GSS integrity cleartext content of RPC arguments and results. RPCSEC_GSS integrity
and privacy are end-to-end. They protect RPC arguments and results and privacy services are end-to-end. They protect RPC arguments and
from application to server endpoint, and back. results from application to server endpoint, and back.
The RPCSEC_GSS integrity and encryption services operate on whole RPC The RPCSEC_GSS integrity and encryption services operate on whole RPC
messages after they have been XDR encoded for transmit, and before messages after they have been XDR encoded for transmit, and before
they have been XDR decoded after receipt. Both sender and receiver they have been XDR decoded after receipt. Both sender and receiver
endpoints use intermediate buffers to prevent exposure of encrypted endpoints use intermediate buffers to prevent exposure of encrypted
data or unverified cleartext data to RPC consumers. After data or unverified cleartext data to RPC consumers. After
verification, encryption, and message wrapping has been performed, verification, encryption, and message wrapping has been performed,
the transport layer MAY use RDMA data transfer between these the transport layer MAY use RDMA data transfer between these
intermediate buffers. intermediate buffers.
skipping to change at page 47, line 12 skipping to change at page 48, line 12
bytes, but they don't have to be. Thus reducing DDP-eligible items bytes, but they don't have to be. Thus reducing DDP-eligible items
affects the result of message integrity verification or encryption. affects the result of message integrity verification or encryption.
Therefore a sender MUST NOT reduce a Payload stream when RPCSEC_GSS Therefore a sender MUST NOT reduce a Payload stream when RPCSEC_GSS
integrity or encryption services are in use. Effectively, no data integrity or encryption services are in use. Effectively, no data
item is DDP-eligible in this situation, and Chunked Messages cannot item is DDP-eligible in this situation, and Chunked Messages cannot
be used. In this mode, an RPC-over-RDMA transport operates in the be used. In this mode, an RPC-over-RDMA transport operates in the
same manner as a transport that does not support direct data same manner as a transport that does not support direct data
placement. placement.
When RPCSEC_GSS integrity or privacy is in use, a requester provides When a RPCSEC_GSS integrity or privacy service is in use, a requester
both a Read list and a Reply chunk in the same RPC-over-RDMA header provides both a Read list and a Reply chunk in the same RPC-over-RDMA
to convey a Long call and provision a receptacle for a Long reply. header to convey a Long call and provision a receptacle for a Long
reply.
8.2.2.4. Protecting RPC-Over-RDMA Transport Headers 8.2.2.4. Protecting RPC-Over-RDMA Transport Headers
Like the base fields in an ONC RPC message (XID, call direction, and Like the base fields in an ONC RPC message (XID, call direction, and
so on), the contents of an RPC-over-RDMA message's Transport stream so on), the contents of an RPC-over-RDMA message's Transport stream
are not protected by RPCSEC_GSS. This exposes XIDs, connection are not protected by RPCSEC_GSS. This exposes XIDs, connection
credit limits, and chunk lists (but not the content of the data items credit limits, and chunk lists (but not the content of the data items
they refer to) to malicious behavior, which could redirect data that they refer to) to malicious behavior, which could redirect data that
is transferred by the RPC-over-RDMA message, result in spurious is transferred by the RPC-over-RDMA message, result in spurious
retransmits, or trigger connection loss. retransmits, or trigger connection loss.
In particular, if an attacker alters the information contained in the In particular, if an attacker alters the information contained in the
chunk lists of an RPC-over-RDMA header, data contained in those chunk lists of an RPC-over-RDMA header, data contained in those
chunks can be redirected to other registered memory regions on chunks can be redirected to other registered memory regions on
requesters. An attacker might alter the arguments of RDMA Read and requesters. An attacker might alter the arguments of RDMA Read and
RDMA Write operations on the wire to similar effect. The use of RDMA Write operations on the wire to similar effect. If such
RPCSEC_GSS integrity or privacy services enable the requester to alterations occurs, the use of RPCSEC_GSS integrity or privacy
detect if such tampering has been done and reject the RPC message. services enable a requester to detect unexpected material in a
received RPC message.
Encryption at lower layers, as described in Section 8.2.1, protects Encryption at lower layers, as described in Section 8.2.1, protects
the content of the Transport stream. To address attacks on RDMA the content of the Transport stream. To address attacks on RDMA
protocols themselves, RDMA transport implementations should conform protocols themselves, RDMA transport implementations should conform
to [RFC5042]. to [RFC5042].
9. IANA Considerations 9. IANA Considerations
Three assignments are specified by this document. These are A set of RPC "netids" for resolving RPC-over-RDMA services is
unchanged from [RFC5666]: specified by this document. This is unchanged from [RFC5666].
o A set of RPC "netids" for resolving RPC-over-RDMA services
o Optional service port assignments for Upper Layer Bindings
o An RPC program number assignment for the configuration protocol
These assignments have been established, as below.
The new RPC transport has been assigned an RPC "netid", which is an The RPC-over-RDMA transport has been assigned an RPC "netid", which
rpcbind [RFC1833] string used to describe the underlying protocol in is an rpcbind [RFC1833] string used to describe the underlying
order for RPC to select the appropriate transport framing, as well as protocol in order for RPC to select the appropriate transport
the format of the service addresses and ports. framing, as well as the format of the service addresses and ports.
The following "Netid" registry strings are defined for this purpose: The following "netid" registry strings are defined for this purpose:
NC_RDMA "rdma" NC_RDMA "rdma"
NC_RDMA6 "rdma6" NC_RDMA6 "rdma6"
The "rdma" netid is to be used when IPv4 addressing is employed by
the underlying transport, and "rdma6" for IPv6 addressing. The netid
assignment policy and registry are defined in [RFC5665].
These netids MAY be used for any RDMA network satisfying the These netids MAY be used for any RDMA network satisfying the
requirements of Section 2.2.2, and able to identify service endpoints requirements of Section 2.2.2, and able to identify service endpoints
using IP port addressing, possibly through use of a translation using IP port addressing, possibly through use of a translation
service as described above in Section 5. The "rdma" netid is to be service as described in Section 5.
used when IPv4 addressing is employed by the underlying transport,
and "rdma6" for IPv6 addressing.
The netid assignment policy and registry are defined in [RFC5665].
As a new RPC transport, this protocol has no effect on RPC Program The use of the RPC-over-RDMA protocol has no effect on RPC Program
numbers or existing registered port numbers. However, new port numbers or existing registered port numbers. However, new port
numbers MAY be registered for use by RPC-over-RDMA-enabled services, numbers MAY be registered for use by RPC-over-RDMA-enabled services,
as appropriate to the new networks over which the services will as appropriate to the new networks over which the services will
operate. operate.
For example, the NFS/RDMA service defined in [RFC5667] has been For example, the NFS/RDMA service defined in [RFC5667] has been
assigned the port 20049, in the IANA registry: assigned the port 20049 in the IANA registry. This is distinct from
the port number defined for NFS on TCP, which is assigned the port
nfsrdma 20049/tcp Network File System (NFS) over RDMA 2049 in the IANA registry. NFS clients use the same RPC Program
nfsrdma 20049/udp Network File System (NFS) over RDMA number for NFS (100003) when using either transport [RFC5531].
nfsrdma 20049/sctp Network File System (NFS) over RDMA
The RPC program number assignment policy and registry are defined in [RFC5666] was listed as the reference for the nfsrdma port
[RFC5531]. assignments. This document updates [RFC5666], but neither this
document nor [RFC5666] specifies these port assignments. Therefore
this document should not be listed as the reference for the nfsrdma
port assignments.
10. References 10. References
10.1. Normative References 10.1. Normative References
[RFC1833] Srinivasan, R., "Binding Protocols for ONC RPC Version 2", [RFC1833] Srinivasan, R., "Binding Protocols for ONC RPC Version 2",
RFC 1833, DOI 10.17487/RFC1833, August 1995, RFC 1833, DOI 10.17487/RFC1833, August 1995,
<http://www.rfc-editor.org/info/rfc1833>. <http://www.rfc-editor.org/info/rfc1833>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
skipping to change at page 53, line 21 skipping to change at page 54, line 14
Charles Lever (editor) Charles Lever (editor)
Oracle Corporation Oracle Corporation
1015 Granger Avenue 1015 Granger Avenue
Ann Arbor, MI 48104 Ann Arbor, MI 48104
USA USA
Phone: +1 248 816 6463 Phone: +1 248 816 6463
Email: chuck.lever@oracle.com Email: chuck.lever@oracle.com
William Allen Simpson William Allen Simpson
DayDreamer Red Hat
1384 Fontaine 1384 Fontaine
Madison Heights, MI 48071 Madison Heights, MI 48071
USA USA
Email: william.allen.simpson@gmail.com Email: william.allen.simpson@redhat.com
Tom Talpey Tom Talpey
Microsoft Corp. Microsoft Corp.
One Microsoft Way One Microsoft Way
Redmond, WA 98052 Redmond, WA 98052
USA USA
Phone: +1 425 704-9945 Phone: +1 425 704-9945
Email: ttalpey@microsoft.com Email: ttalpey@microsoft.com
 End of changes. 32 change blocks. 
79 lines changed or deleted 126 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/