draft-ietf-rddp-applicability-00.txt   draft-ietf-rddp-applicability-01.txt 
Remote Direct Data Placement C. Bestler Remote Direct Data Placement C. Bestler
Working group L. Coene Working group L. Coene
Expires: December 11, 2003 Expires: April 12, 2004
Applicability of Remote Direct Memory Access Protocol (RDMA) and Applicability of Remote Direct Memory Access Protocol (RDMA) and
Direct Data Placement (DDP) Direct Data Placement (DDP)
draft-ietf-rddp-applicability-00.txt draft-ietf-rddp-applicability-01.txt
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
skipping to change at page 1, line 33 skipping to change at page 1, line 33
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at http:// The list of current Internet-Drafts can be accessed at http://
www.ietf.org/ietf/1id-abstracts.txt. www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on December 11, 2003. This Internet-Draft will expire on April 12, 2004.
Copyright Notice Copyright Notice
Copyright (C) The Internet Society (2003). All Rights Reserved. Copyright (C) The Internet Society (2003). All Rights Reserved.
Abstract Abstract
This document describes the applicability of Remote Direct Memory This document describes the applicability of Remote Direct Memory
Access Protocol (RDMAP) and the Direct Data Placement Protocol Access Protocol (RDMAP) and the Direct Data Placement Protocol
(DDP). It contrasts the different transport options over IP that DDP (DDP). It contrasts the different transport options over IP that DDP
can use, compares use of DDP with direct use of the supporting can use, compares use of DDP with direct use of the supporting
transports, and compares DDP over IP transports with non-IP transports, and compares DDP over IP transports with non-IP
transports that support RDMA functionality. transports that support RDMA functionality.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Direct Placement . . . . . . . . . . . . . . . . . . . . . . . 6 3. Direct Placement . . . . . . . . . . . . . . . . . . . . . . 6
3.1 Fewer Required ULP Interactions . . . . . . . . . . . . . . . 6 3.1 Fewer Required ULP Interactions . . . . . . . . . . . . . . 6
3.2 Direct Placement using only the LLP . . . . . . . . . . . . . 6 3.2 Direct Placement using only the LLP . . . . . . . . . . . . 6
4. Tagged Messages . . . . . . . . . . . . . . . . . . . . . . . 8 4. Tagged Messages . . . . . . . . . . . . . . . . . . . . . . 8
4.1 Order Independent Reception . . . . . . . . . . . . . . . . . 8 4.1 Order Independent Reception . . . . . . . . . . . . . . . . 8
4.2 Reduced ULP Notifications . . . . . . . . . . . . . . . . . . 8 4.2 Reduced ULP Notifications . . . . . . . . . . . . . . . . . 8
4.3 Simplified ULP Exchanges . . . . . . . . . . . . . . . . . . . 9 4.3 Simplified ULP Exchanges . . . . . . . . . . . . . . . . . . 9
4.4 Order Independent Sending . . . . . . . . . . . . . . . . . . 10 4.4 Order Independent Sending . . . . . . . . . . . . . . . . . 10
4.5 Tagged Buffers as ULP Credits . . . . . . . . . . . . . . . . 11 4.5 Tagged Buffers as ULP Credits . . . . . . . . . . . . . . . 11
5. RDMA Read . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5. RDMA Read . . . . . . . . . . . . . . . . . . . . . . . . . 13
6. LLP Comparisons . . . . . . . . . . . . . . . . . . . . . . . 14 6. LLP Comparisons . . . . . . . . . . . . . . . . . . . . . . 14
6.1 Multistreaming Implications . . . . . . . . . . . . . . . . . 14 6.1 Multistreaming Implications . . . . . . . . . . . . . . . . 14
6.2 Out of Order Reception Implications . . . . . . . . . . . . . 14 6.2 Out of Order Reception Implications . . . . . . . . . . . . 14
6.3 Header and Marker Overhead . . . . . . . . . . . . . . . . . . 14 6.3 Header and Marker Overhead . . . . . . . . . . . . . . . . . 14
6.4 Data Integrity Implications . . . . . . . . . . . . . . . . . 15 6.4 Middlebox Support . . . . . . . . . . . . . . . . . . . . . 14
6.5 Non-IP Transports . . . . . . . . . . . . . . . . . . . . . . 15 6.5 Processing Overhead . . . . . . . . . . . . . . . . . . . . 15
6.6 Other IP Transports . . . . . . . . . . . . . . . . . . . . . 15 6.6 Data Integrity Implications . . . . . . . . . . . . . . . . 15
7. Local Interface Implications . . . . . . . . . . . . . . . . . 17 6.6.1 MPA/TCP Specifics . . . . . . . . . . . . . . . . . . . . . 15
8. Security considerations . . . . . . . . . . . . . . . . . . . 18 6.6.2 SCTP Specifics . . . . . . . . . . . . . . . . . . . . . . . 16
8.1 Connection/Association Setup . . . . . . . . . . . . . . . . . 18 6.7 Non-IP Transports . . . . . . . . . . . . . . . . . . . . . 16
8.2 Tagged Buffer Exposure . . . . . . . . . . . . . . . . . . . . 18 6.7.1 No RDMA Layer Ack . . . . . . . . . . . . . . . . . . . . . 16
8.3 Impact of Encrypted Transports . . . . . . . . . . . . . . . . 18 6.8 Other IP Transports . . . . . . . . . . . . . . . . . . . . 17
References . . . . . . . . . . . . . . . . . . . . . . . . . . 19 6.9 LLP Independent Session Establishment . . . . . . . . . . . 17
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 19 6.9.1 RDMA-only Session Establishment . . . . . . . . . . . . . . 18
Full Copyright Statement . . . . . . . . . . . . . . . . . . . 21 6.9.2 RDMA-Conditional Session Establishment . . . . . . . . . . . 18
7. Local Interface Implications . . . . . . . . . . . . . . . . 20
8. Security considerations . . . . . . . . . . . . . . . . . . 21
8.1 Connection/Association Setup . . . . . . . . . . . . . . . . 21
8.2 Tagged Buffer Exposure . . . . . . . . . . . . . . . . . . . 21
8.3 Impact of Encrypted Transports . . . . . . . . . . . . . . . 21
References . . . . . . . . . . . . . . . . . . . . . . . . . 22
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 22
Full Copyright Statement . . . . . . . . . . . . . . . . . . 24
1. Introduction 1. Introduction
Remote Direct Memory Access Protocol (RDMAP) and Direct Data Remote Direct Memory Access Protocol (RDMAP) and Direct Data
Placement (DDP) work together to provide application independent Placement (DDP) work together to provide application independent
efficient placemenet of application payload directly into buffers efficient placemenet of application payload directly into buffers
specified by the Upper Layer Protocol (ULP). specified by the Upper Layer Protocol (ULP).
The DDP protocol is responsible for direct placement of received The DDP protocol is responsible for direct placement of received
payload into ULP specified buffers. The RDMAP protocol provides payload into ULP specified buffers. The RDMAP protocol provides
skipping to change at page 3, line 52 skipping to change at page 3, line 52
o RDMAP consolidates ULP notifications, thereby minimizing the o RDMAP consolidates ULP notifications, thereby minimizing the
number of required ULP interactions. number of required ULP interactions.
o RDMAP defines RDMA Reads, which allow remote access to advertised o RDMAP defines RDMA Reads, which allow remote access to advertised
buffers. This document will review the advantages of using RDMA buffers. This document will review the advantages of using RDMA
Reads as contrasted to alternate solutions. Reads as contrasted to alternate solutions.
Some non-IP transports, such as InfiniBand, directly integrate RDMA Some non-IP transports, such as InfiniBand, directly integrate RDMA
features. This document will review the applicability of providing features. This document will review the applicability of providing
RDMA services over ubiquitous IP transports as opposed to the use of RDMA services over ubiquitous IP transports as opposed to the use of
customized transport protocols. customized transport protocols. Due to the fact that DDP is defined
cleanly as a layer over existing IP transports, DDP has simpler
ordering rules than some prior RDMA protocols. This may have some
implications for application designers.
The full capabilities of DDP and RDMAP can only be fully realized by The full capabilities of DDP and RDMAP can only be fully realized by
applications that are designed to exploit them. The co-existence of applications that are designed to exploit them. The co-existence of
RDMAP/DDP aware local interfaces with traditional socket interfaces RDMAP/DDP aware local interfaces with traditional socket interfaces
will also be explored. will also be explored.
Finally, DDP support is defined for at least two IP transports: SCTP Finally, DDP support is defined for at least two IP transports: SCTP
and TCP. The rationale for supporting both transports is reviewed, and TCP. The rationale for supporting both transports is reviewed,
as well as when each would be the appropriate selection. as well as when each would be the appropriate selection.
skipping to change at page 6, line 30 skipping to change at page 6, line 30
desirable, it is critical for high speed connections. The burst desirable, it is critical for high speed connections. The burst
packet rate for a high speed interface could easily exceed the host packet rate for a high speed interface could easily exceed the host
systems ability to switch ULP contexts. systems ability to switch ULP contexts.
Content access applications are primary examples of applications with Content access applications are primary examples of applications with
both high bandwidth and high content to required ULP interaction both high bandwidth and high content to required ULP interaction
ratios. These applications include file access protocols (NAS), ratios. These applications include file access protocols (NAS),
storage access (SAN), database access and other application specific storage access (SAN), database access and other application specific
forms of content access such as HTTP, XML and email. forms of content access such as HTTP, XML and email.
3.2 Direct Placement using only the LLP
Direct data placement can be achieved without RDMA. Pre-posting of Direct data placement can be achieved without RDMA. Pre-posting of
receive buffers could allow a non-RDMA network stack to place data receive buffers could allow a non-RDMA network stack to place data
directly to user buffers. directly to user buffers.
3.2 Direct Placement using only the LLP
The degree to which DDP optimizes depends on which transport is being The degree to which DDP optimizes depends on which transport is being
compared with, and on the nature of the local interface. Without compared with, and on the nature of the local interface. Without
RDMAP/DDP pre-posting buffers requires the receiving side to RDMAP/DDP pre-posting buffers requires the receiving side to
accurately predict the required buffers and their sizes. This is not accurately predict the required buffers and their sizes. This is not
feasible for all ULPs. By contrast, DDP only requires the ULP to feasible for all ULPs. By contrast, DDP only requires the ULP to
predict the sequence and size of incoming untagged messages. predict the sequence and size of incoming untagged messages.
An application that could predict incoming messages and required An application that could predict incoming messages and required
nothing more than direct placement into buffers might be able to do nothing more than direct placement into buffers might be able to do
so with a properly designed local interface to SCTP or TCP. Doing so so with a properly designed local interface to SCTP or TCP. Doing so
skipping to change at page 7, line 13 skipping to change at page 7, line 13
byte). byte).
The LLP can also be used directly if ULP specific knowledge is built The LLP can also be used directly if ULP specific knowledge is built
into the protocol stack to allow "parse and place" handling of into the protocol stack to allow "parse and place" handling of
received packets. Such a solution either requires interaction with received packets. Such a solution either requires interaction with
the ULP, or that the protocol stack have knowledge of ULP specific the ULP, or that the protocol stack have knowledge of ULP specific
syntax rules. syntax rules.
DDP achieves the benefits of directly placing incoming payload DDP achieves the benefits of directly placing incoming payload
without requiring tight coupling between the ULP and the protocol without requiring tight coupling between the ULP and the protocol
stack. However, "parse adn place" capabilities can certainly provide stack. However, "parse and place" capabilities can certainly provide
equivalent services to a limited number of ULPs. equivalent services to a limited number of ULPs.
4. Tagged Messages 4. Tagged Messages
This section covers the major benefits from the use of Tagged This section covers the major benefits from the use of Tagged
Messages. Messages.
A more critical advantage of DDP is the ability of the Data Source to A more critical advantage of DDP is the ability of the Data Source to
use tagged buffers. Tagging transfers allows the Data Source to use tagged buffers. Tagging messages allows the Data Source to
choose the ordering and packetization of its payload deliveries. choose the ordering and packetization of its payload deliveries.
With direct data placement based solely upon pre-posted receives, the With direct data placement based solely upon pre-posted receives, the
packetization and delivery of payload must be agreed by the ULP packetization and delivery of payload must be agreed by the ULP peers
peers. Even if there is an encoding of what is being transferred, as in advance. Even if there is an encoding of what is being
is common with middleware solutions, this information is not transferred, as is common with middleware solutions, this information
understood at the application independent layers. The directions on is not understood at the application independent layers. The
where to place the incoming data cannot be accessed without switching directions on where to place the incoming data cannot be accessed
to the ULP first. DDP provides a standardized 'packing list' which without switching to the ULP first. DDP provides a standardized
can be interpreted without requiring ULP interaction. Indeed, it is 'packing list' which can be interpreted without requiring ULP
designed to be implementable in hardware. interaction. Indeed, it is designed to be implementable in hardware.
4.1 Order Independent Reception 4.1 Order Independent Reception
Tagged messages are directed to a buffer based on an included Tagged messages are directed to a buffer based on an included
Steering Tag. Additionally, no notice is provided to the ULP for Steering Tag. Additionally, no notice is provided to the ULP for
each individual Tagged Message's arrival. Together these allow each individual Tagged Message's arrival. Together these allow
tagged messages received out-of-order to be processed without tagged messages received out-of-order to be processed without
intermediate buffering or additional notifications to the ULP. intermediate buffering or additional notifications to the ULP.
4.2 Reduced ULP Notifications 4.2 Reduced ULP Notifications
skipping to change at page 14, line 23 skipping to change at page 14, line 23
SCTP provides for preservation of message boundaries. Each DDP SCTP provides for preservation of message boundaries. Each DDP
segment will be delivered within a single SCTP packet. The segment will be delivered within a single SCTP packet. The
equivalent services are only available with TCP through the use of equivalent services are only available with TCP through the use of
the MPA adaptation layer. the MPA adaptation layer.
6.1 Multistreaming Implications 6.1 Multistreaming Implications
SCTP also provides multi-streaming. When the same pair of hosts have SCTP also provides multi-streaming. When the same pair of hosts have
need for multiple DDP streams this can be a major advantage. A need for multiple DDP streams this can be a major advantage. A
single SCTP association carries multiple DDP streams, consolidating single SCTP association carries multiple DDP streams, consolidating
connection setup and flow control. connection setup, congestion control and acknowledgements.
Completions are controlled by the DDP Source Sequence Number (DDP- Completions are controlled by the DDP Source Sequence Number (DDP-
SSN) on a per stream basis. Therefore combining multiple DDP Streams SSN) on a per stream basis. Therefore combining multiple DDP Streams
into a single SCTP association cannot result in a dropped packet into a single SCTP association cannot result in a dropped packet
carrying data for one stream delaying completions on others. carrying data for one stream delaying completions on others.
6.2 Out of Order Reception Implications 6.2 Out of Order Reception Implications
The use of unordered Data Chunks with SCTP guarantees that the DDP The use of unordered Data Chunks with SCTP guarantees that the DDP
layer will be able to perform placements when IP datagrams are layer will be able to perform placements when IP datagrams are
skipping to change at page 14, line 50 skipping to change at page 14, line 50
single Data Chunk and never spread over multiple IP datagrams. single Data Chunk and never spread over multiple IP datagrams.
6.3 Header and Marker Overhead 6.3 Header and Marker Overhead
MPA and TCP headers together are smaller than the headers used by MPA and TCP headers together are smaller than the headers used by
SCTP and its adaptation layer. However, this advantage can be SCTP and its adaptation layer. However, this advantage can be
considerably reduced by the insertion of MPA markers. In any event considerably reduced by the insertion of MPA markers. In any event
the different in ULP payload per IP Datagram is not likely to be a the different in ULP payload per IP Datagram is not likely to be a
signifigant factor. signifigant factor.
Even with the MPA adaptation layer, DDP traffic will appear to all 6.4 Middlebox Support
network traffic as a normal TCP connection. In many environmenets
there may be a requirement to use only TCP connections to satisfy Even with the MPA adaptation layer, DDP traffic carried over MPA/TCP
existing network elements and/or to facilitate monitoring and control will appear to all network middleboxes as a normal TCP connection.
of connections. In many environmenets there may be a requirement to use only TCP
connections to satisfy existing network elements and/or to facilitate
monitoring and control of connections. While SCTP is certainly just
as monitorable and controllable as TCP, there is no guarantee that
the network management infrastructure has the required support for
both.
6.5 Processing Overhead
A DDP stream delivered via MPA/TCP will require more processing A DDP stream delivered via MPA/TCP will require more processing
effort than one delivered over SCTP. However this extra work may be effort than one delivered over SCTP. However this extra work may be
justified for many deployments where full SCTP support is unavailable justified for many deployments where full SCTP support is unavailable
in the intermediate network. in the intermediate network.
6.4 Data Integrity Implications 6.6 Data Integrity Implications
Both the SCTP and MPA/TCP adaptation provide end-to-end CRC32c Both the SCTP and MPA/TCP adaptation provide end-to-end CRC32c
protection against data corruption, or its equivalent. protection against data corruption, or its equivalent.
A ULP that requires a greater degree of protection may add it own. A ULP that requires a greater degree of protection may add it own.
However, DDP and RDMAP headers will only be guaranteed to have the However, DDP and RDMAP headers will only be guaranteed to have the
equivalent of end-to-end CRC32c protection. A ULP that requires data equivalent of end-to-end CRC32c protection. A ULP that requires data
integrity checking more thorough than an end-to-end CRC32c should integrity checking more thorough than an end-to-end CRC32c should
first invalidate all STags that reference a buffer before applying first invalidate all STags that reference a buffer before applying
their own integrity check. their own integrity check.
6.5 Non-IP Transports 6.6.1 MPA/TCP Specifics
It is mandatory for MPA/TCP implementations to implement CRC32c, but
it is NOT mandatory to use the CRC32c during an RDMA connection. The
activating or deactivating of the CRC in MPA/TCP is an administrative
configuration operation at the local and remote end. The
administration of the CRC(ON/OFF) is invisible to the ULP.
Applications SHOULD trust that this administrative option will only
be used when the end-to-end protection is at least as effective as a
transport layer CRC32c. Applications SHOULD NOT apply additional
protection as a guard against this administrative option being turned
on inadvertently.
Administrators MUST NOT enable CRC32c suppression unless the end-to-
end protection is truly equivalent.
If the CRC is active/used for one direction/end , then the use of the
CRC is mandatory in both directions/ends.
If both ends have been configured NOT to use the CRC, then this is
allowed as long as an equivalent protection(comparable or better
than/to CRC) from undetected errors on the connection is provided.
6.6.2 SCTP Specifics
SCTP provides CRC32c protection automatically. The adaptation to
SCTP provides for no option to suppress SCTP CRC32c protection.
6.7 Non-IP Transports
DDP is defined to operate over ubiquitous IP transports such as SCTP DDP is defined to operate over ubiquitous IP transports such as SCTP
and TCP. This enabled a new DDP-enabled node to be added anywhere to and TCP. This enabled a new DDP-enabled node to be added anywhere to
an IP network. No DDP-specific support from middle-boxes is an IP network. No DDP-specific support from middle-boxes is
required. required.
There are non-IP transport fabric offering RDMA capabilities. There are non-IP transport fabric offering RDMA capabilities.
Because these capabilities are integrated with the transport protocol Because these capabilities are integrated with the transport protocol
they have some technical advantages when compared to RDMA over IP. they have some technical advantages when compared to RDMA over IP.
For example fencing of RDMA operations can be based upon transport For example fencing of RDMA operations can be based upon transport
level acks. Because DDP is cleanly layered over an IP transport, any level acks. Because DDP is cleanly layered over an IP transport, any
explicit RDMA layer ack must be separate from the transport layer explicit RDMA layer ack must be separate from the transport layer
ack. ack.
There may be deployments where the benefits of RDMA/transport There may be deployments where the benefits of RDMA/transport
integration outweigh the benefits of being on an IP network. integration outweigh the benefits of being on an IP network.
6.6 Other IP Transports 6.7.1 No RDMA Layer Ack
DDP does not provide for its own acknowledgements. The only form of
ack provided at the RDMAP layer is an RDMA Read Response. DDP and
RDMAP rely almost entirely upon other layers for flow control and
pacing. The LLP is relied upon to guarantee delivery and avoid
network congestion, and ULP level acking is relied upon for ULP
pacing and to avoid ULP buffer overruns.
Previous RDMA protocols, such as InfiniBand, have been able to use
their integration with the transport layer to provide stronger
ordering guarantees. It is important that application designers that
require such guarantees to provide them through ULP interaction.
Specifically:
There is no ability for a local interface to "fence" outbound
messages to guarantee that prior tagged messages have been placed
prior to sending a tagged message. The only guarantees available
from the other side would be an RDMA Read Response (coming from
the RDMAP layer) or a response from the ULP layer. Remember that
the normal ordering rules only guarantee when the Data Sink ULP
will be notified of untagged messages, it does not control when
data is placed into receive buffers.
Re-use of tagged buffers must be done with extreme care. The fact
that an untagged message indicates that all prior tagged messages
have been placed does not guarantee that no later tagged message
have. The best strategy is to only change the state of any given
advertised buffers with with untagged messages.
As covered elsewhere in this document, flow control of untagged
messages MUST be provided by the ULP itself.
6.8 Other IP Transports
Both TCP and SCTP provide DDP with reliable transport with TCP Both TCP and SCTP provide DDP with reliable transport with TCP
friendly rate control. As currently DDP is defined to work over friendly rate control. As currently DDP is defined to work over
reliable transports and implicitly relies upon some form of rate reliable transports and implicitly relies upon some form of rate
control. control.
DDP is fully compatible with a non-reliable protocol. Out-of-order DDP is fully compatible with a non-reliable protocol. Out-of-order
placement is obviously not dependent on whether the other DDP placement is obviously not dependent on whether the other DDP
Segments ever actually arrive. Segments ever actually arrive.
skipping to change at page 17, line 5 skipping to change at page 17, line 44
application would be only limited by the link layer rate, almost application would be only limited by the link layer rate, almost
inevitably resulting in severe network congestion. inevitably resulting in severe network congestion.
RDMAP encourages applications to be ignorant of the underlying RDMAP encourages applications to be ignorant of the underlying
transport PMTU. The ULP is only notified when all messages ending in transport PMTU. The ULP is only notified when all messages ending in
a single untagged message have completed. The ULP is not aware of a single untagged message have completed. The ULP is not aware of
the granularity or ordering of the underlying message. This approach the granularity or ordering of the underlying message. This approach
assumes that the ULP is only interested in the complete set of assumes that the ULP is only interested in the complete set of
messages, and has no use for a subset of them. messages, and has no use for a subset of them.
6.9 LLP Independent Session Establishment
For an RDMAP/DDP application, the transport services provided by a
pair of SCTP Streams and by a TCP connection both provide the same
service (reliable delivery of DDP Segments between two connected
RDMAP/DDP endpoints).
6.9.1 RDMA-only Session Establishment
It is also possible to allow for transport neutral establishment of
RDMAP/DDP sessions between endpoints. Combined, these two features
would allow most applications to be unconcerned as to which LLP was
actually in use.
Specifically, the procedures for DDP Stream Session establishment
discussed in section 3 of the SCTP mapping, and section 13.3 of the
MPA/TCP mapping, both allow for the exchange of ULP specific data
("Private Data") before enabling the exchange of DDP Segments. This
delays can allow for proper selection and/or configuration of the
endpoints based upon the exchanged data. For example, each DDP
Stream Session associated with a single client session might be
assigned to the same DDP Protection Domain.
To be transport neutral, the applications should exchange Private
Data as part of session establishment messages to determine how the
RDMA endpoints are to be configured. One side must be the Initiator,
and the other the Responder.
With SCTP, a pair of SCTP streams can be used for sequential
sessions. With MPA/TCP each connection can be used for at most one
session. However, the same source/destination pair of ports can be
re-used sequentially subject to normal TCP rules.
6.9.2 RDMA-Conditional Session Establishment
It is sometimes desirable for the active side of a session to connect
with the passive side before knowing whether the passive side
supports RDMA.
This style of session establishment can be supported with either TCP
or SCTP, but not as transparently as for RDMA-only sessions. Pre-
existing non-RDMA servers are also far more likely to be using TCP
than SCTP.
With TCP. a normal TCP connection is established. It is then used
by the ULP to determine whether or not to convert to MPA mode and use
RDMA. This will typically be integral with other session
establishment negotiations.
With SCTP, the establishment of an association tests whether RDMA is
supported. If not supported, the application simply requests the
association without the RDMA adaptation indication.
In key difference is that with SCTP the determination as to whether
the peer can support RDMA is made before the transport layer
association/connection is established while with TCP the established
connection itself is used to determine whether RDMA is supported.
7. Local Interface Implications 7. Local Interface Implications
Full utilization of DDP and RDMAP capabilities requires a local Full utilization of DDP and RDMAP capabilities requires a local
interface that explicitly requests these services. Protocols such as interface that explicitly requests these services. Protocols such as
Sockets Direct Protocol (SDP) can allow applications to keep their Sockets Direct Protocol (SDP) can allow applications to keep their
traditional byte-stream or message-stream interface and still enjoy traditional byte-stream or message-stream interface and still enjoy
many of the benefits of the optimized wire level protocols. many of the benefits of the optimized wire level protocols.
8. Security considerations 8. Security considerations
skipping to change at page 19, line 32 skipping to change at page 22, line 32
Statement", RFC 3257, April 2002. Statement", RFC 3257, April 2002.
[6] Recio, R., "An RDMA Protocol Specification", draft-ietf-rddp- [6] Recio, R., "An RDMA Protocol Specification", draft-ietf-rddp-
rdmap-00 (work in progress), February 2003. rdmap-00 (work in progress), February 2003.
[7] Shah, H., "Direct Data Placement over Reliable Transports", [7] Shah, H., "Direct Data Placement over Reliable Transports",
draft-ietf-rddp-ddp-00 (work in progress), February 2003. draft-ietf-rddp-ddp-00 (work in progress), February 2003.
[8] Stewart, R., "Stream Control Transmission Protocol (SCTP) Remote [8] Stewart, R., "Stream Control Transmission Protocol (SCTP) Remote
Direct Memory Access (RDMA) Direct Data Placement (DDP) Direct Memory Access (RDMA) Direct Data Placement (DDP)
Adaption", draft-stewart-rddp-sctp-02 (work in progress), Adaptationn", draft-ietf-rddp-sctp-00 (work in progress),
February 2003. September 2003.
[9] Culley, P., "Marker PDU Aligned Framing for TCP Specification", [9] Culley, P., "Marker PDU Aligned Framing for TCP Specification",
draft-culley-iwarp-mpa-02 (work in progress), February 2003. draft-ietf-rddp-mpa-00 (work in progress), October 2003.
Authors' Addresses Authors' Addresses
Caitlin Bestler Caitlin Bestler
1241 W. North Shore 1241 W. North Shore
# 2G # 2G
Chicago, IL 60626 Chicago, IL 60626
USA USA
Phone: +1-773-743-1594 Phone: +1-773-743-1594
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/