draft-ietf-nfsv4-sess-01.txt		draft-ietf-nfsv4-sess-02.txt
	INTERNET-DRAFT Tom Talpey		INTERNET-DRAFT Tom Talpey
	Expires: August 2005 Network Appliance, Inc.		Expires: December 2005 Network Appliance, Inc.
	Spencer Shepler		Spencer Shepler
	Sun Microsystems, Inc.		Sun Microsystems, Inc.
	Jon Bauman		Jon Bauman
	University of Michigan		University of Michigan
	February, 2005		July, 2005
	NFSv4 Session Extensions		NFSv4 Session Extensions
	draft-ietf-nfsv4-sess-01		draft-ietf-nfsv4-sess-02
	Status of this Memo		Status of this Memo
	By submitting this Internet-Draft, I certify that any applicable		By submitting this Internet-Draft, each author represents that any
	patent or other IPR claims of which I am aware have been disclosed,		applicable patent or other IPR claims of which he or she is aware
	or will be disclosed, and any of which I become aware will be		have been or will be disclosed, and any of which he or she becomes
	disclosed, in accordance with RFC 3668.		aware will be disclosed, in accordance with Section 6 of BCP 79.
	Internet-Drafts are working documents of the Internet Engineering		Internet-Drafts are working documents of the Internet Engineering
	Task Force (IETF), its areas, and its working groups. Note that		Task Force (IETF), its areas, and its working groups. Note that
	other groups may also distribute working documents as Internet-		other groups may also distribute working documents as Internet-
	Drafts.		Drafts.
	Internet-Drafts are draft documents valid for a maximum of six		Internet-Drafts are draft documents valid for a maximum of six
	months and may be updated, replaced, or obsoleted by other		months and may be updated, replaced, or obsoleted by other
	documents at any time. It is inappropriate to use Internet-Drafts		documents at any time. It is inappropriate to use Internet-Drafts
	as reference material or to cite them other than as "work in		as reference material or to cite them other than as "work in
	skipping to change at page 2, line 45		skipping to change at page 2, line 45
	2.3.2. Negotiated RDMA Connection Model . . . . . . . . . . . 24		2.3.2. Negotiated RDMA Connection Model . . . . . . . . . . . 24
	2.3.3. Automatic RDMA Connection Model . . . . . . . . . . . 24		2.3.3. Automatic RDMA Connection Model . . . . . . . . . . . 24
	2.4. Buffer Management, Transfer, Flow Control . . . . . . . 25		2.4. Buffer Management, Transfer, Flow Control . . . . . . . 25
	2.5. Retry and Replay . . . . . . . . . . . . . . . . . . . . 28		2.5. Retry and Replay . . . . . . . . . . . . . . . . . . . . 28
	2.6. The Back Channel . . . . . . . . . . . . . . . . . . . . 28		2.6. The Back Channel . . . . . . . . . . . . . . . . . . . . 28
	2.7. COMPOUND Sizing Issues . . . . . . . . . . . . . . . . . 30		2.7. COMPOUND Sizing Issues . . . . . . . . . . . . . . . . . 30
	2.8. Data Alignment . . . . . . . . . . . . . . . . . . . . . 30		2.8. Data Alignment . . . . . . . . . . . . . . . . . . . . . 30
	3. NFSv4 Integration . . . . . . . . . . . . . . . . . . . . 31		3. NFSv4 Integration . . . . . . . . . . . . . . . . . . . . 31
	3.1. Minor Versioning . . . . . . . . . . . . . . . . . . . . 32		3.1. Minor Versioning . . . . . . . . . . . . . . . . . . . . 32
	3.2. Slot Identifiers and Server Duplicate Request Cache . . 32		3.2. Slot Identifiers and Server Duplicate Request Cache . . 32
	3.3. COMPOUND and CB_COMPOUND . . . . . . . . . . . . . . . . 35		3.3. COMPOUND and CB_COMPOUND . . . . . . . . . . . . . . . . 36
	3.4. eXternal Data Representation Efficiency . . . . . . . . 36		3.4. eXternal Data Representation Efficiency . . . . . . . . 37
	3.5. Effect of Sessions on Existing Operations . . . . . . . 36		3.5. Effect of Sessions on Existing Operations . . . . . . . 37
	3.6. Authentication Efficiencies . . . . . . . . . . . . . . 37		3.6. Authentication Efficiencies . . . . . . . . . . . . . . 38
	4. Security Considerations . . . . . . . . . . . . . . . . . 38		4. Security Considerations . . . . . . . . . . . . . . . . . 39
	4.1. Authentication . . . . . . . . . . . . . . . . . . . . . 40		4.1. Authentication . . . . . . . . . . . . . . . . . . . . . 40
	5. IANA Considerations . . . . . . . . . . . . . . . . . . . 41		5. IANA Considerations . . . . . . . . . . . . . . . . . . . 41
	6. NFSv4 Protocol Extensions . . . . . . . . . . . . . . . . 41		6. NFSv4 Protocol Extensions . . . . . . . . . . . . . . . . 42
	6.1. Operation: CREATECLIENTID . . . . . . . . . . . . . . . 41		6.1. Operation: CREATECLIENTID . . . . . . . . . . . . . . . 42
	6.2. Operation: CREATESESSION . . . . . . . . . . . . . . . . 46		6.2. Operation: CREATESESSION . . . . . . . . . . . . . . . . 47
	6.3. Operation: BIND_BACKCHANNEL . . . . . . . . . . . . . . 51		6.3. Operation: BIND_BACKCHANNEL . . . . . . . . . . . . . . 52
	6.4. Operation: DESTROYSESSION . . . . . . . . . . . . . . . 53		6.4. Operation: DESTROYSESSION . . . . . . . . . . . . . . . 54
	6.5. Operation: SEQUENCE . . . . . . . . . . . . . . . . . . 54		6.5. Operation: SEQUENCE . . . . . . . . . . . . . . . . . . 55
	6.6. Callback operation: CB_RECALLCREDIT . . . . . . . . . . 56		6.6. Callback operation: CB_RECALLCREDIT . . . . . . . . . . 57
	6.7. Callback operation: CB_SEQUENCE . . . . . . . . . . . . 56		6.7. Callback operation: CB_SEQUENCE . . . . . . . . . . . . 57
	7. NFSv4 Session Protocol Description . . . . . . . . . . . . 58		7. NFSv4 Session Protocol Description . . . . . . . . . . . . 59
	8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . 64		8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . 65
	9. References . . . . . . . . . . . . . . . . . . . . . . . . 64		9. References . . . . . . . . . . . . . . . . . . . . . . . . 65
	9.1. Normative References . . . . . . . . . . . . . . . . . . 64		9.1. Normative References . . . . . . . . . . . . . . . . . . 65
	9.2. Informative References . . . . . . . . . . . . . . . . . 65		9.2. Informative References . . . . . . . . . . . . . . . . . 66
	10. Authors' Addresses . . . . . . . . . . . . . . . . . . . . 67		10. Authors' Addresses . . . . . . . . . . . . . . . . . . . . 68
	11. Full Copyright Statement . . . . . . . . . . . . . . . . . 67		11. Full Copyright Statement . . . . . . . . . . . . . . . . . 69
	1. Introduction		1. Introduction
	This draft proposes extensions to NFS version 4 [RFC3530] enabling		This draft proposes extensions to NFS version 4 [RFC3530] enabling
	it to support sessions and endpoint management, and to support		it to support sessions and endpoint management, and to support
	operation atop RDMA-capable RPC over transports such as iWARP.		operation atop RDMA-capable RPC over transports such as iWARP.
	[RDMAP, DDP] These extensions enable support for exactly-once		[RDMAP, DDP] These extensions enable support for exactly-once
	semantics by NFSv4 servers, multipathing and trunking of transport		semantics by NFSv4 servers, multipathing and trunking of transport
	connections, and enhanced security. The ability to operate over		connections, and enhanced security. The ability to operate over
	RDMA enables greatly enhanced performance. Operation over existing		RDMA enables greatly enhanced performance. Operation over existing
	skipping to change at page 25, line 20		skipping to change at page 25, line 20
	: :		: :
	: Create Clientid(nfs_client_id4) :		: Create Clientid(nfs_client_id4) :
	: ------------------------------> :		: ------------------------------> :
	: : Prepost		: : Prepost
	: Clientid reply(clientid, ...) : receive		: Clientid reply(clientid, ...) : receive
	: <------------------------------ :		: <------------------------------ :
	Prepost : :		Prepost : :
	receive : Create Session(clientid, size S, :		receive : Create Session(clientid, size S, :
	: maxreq N, RDMA ...) :		: maxreq N, RDMA ...) :
	: ------------------------------> :		: ------------------------------> :
	: : Prepost <=N' receives		: : Prepost <=N'
	: Session reply(sessionid, size S', : of size S'		: Session reply(sessionid, size S', : receives of
	: maxreq N') :		: maxreq N') : size S'
	: <------------------------------ :		: <------------------------------ :
	: :		: :
	: <normal operation> :		: <normal operation> :
	: ------------------------------> :		: ------------------------------> :
	: <------------------------------ :		: <------------------------------ :
	: : :		: : :
	2.4. Buffer Management, Transfer, Flow Control		2.4. Buffer Management, Transfer, Flow Control
	Inline operations in NFSv4.1 behave effectively the same as TCP		Inline operations in NFSv4.1 behave effectively the same as TCP
	skipping to change at page 31, line 28		skipping to change at page 31, line 28
	all go into the determination of a maximal NFSv4 request size and		all go into the determination of a maximal NFSv4 request size and
	therefore minimal buffer size. The client must select its offered		therefore minimal buffer size. The client must select its offered
	value carefully, so as not to overburden the server, and vice-		value carefully, so as not to overburden the server, and vice-
	versa. The payoff of an appropriate padding value is higher		versa. The payoff of an appropriate padding value is higher
	performance.		performance.
	Sender gather:		Sender gather:
	|RPC Request|Pad bytes|Length| -> |User data...|		|RPC Request|Pad bytes|Length| -> |User data...|
	\------+---------------------/ \		\------+---------------------/ \
	\ \		\ \
	\ Receiver scatter: \--------------+- ...		\ Receiver scatter: \-----------+- ...
	/-----+----------------\ \ \		/-----+----------------\ \ \
	|RPC Request|Pad|Length| -> |FS buffer| -> |FS buffer| -> ...		|RPC Request|Pad|Length| -> |FS buffer| -> |FS buffer| -> ...
	In the above case, the server may recycle unused buffers to the		In the above case, the server may recycle unused buffers to the
	next posted receive if unused by the actual received request, or		next posted receive if unused by the actual received request, or
	may pass the now-complete buffers by reference for normal write		may pass the now-complete buffers by reference for normal write
	processing. For a server which can make use of it, this removes		processing. For a server which can make use of it, this removes
	any need for data copies of incoming data, without resorting to		any need for data copies of incoming data, without resorting to
	complicated end-to-end buffer advertisement and management. This		complicated end-to-end buffer advertisement and management. This
	includes most kernel-based and integrated server designs, among		includes most kernel-based and integrated server designs, among
	skipping to change at page 33, line 21		skipping to change at page 33, line 21
	When the client issues a new request, it selects a slotid in the		When the client issues a new request, it selects a slotid in the
	range 0..N-1, where N is the server's current "totalrequests" limit		range 0..N-1, where N is the server's current "totalrequests" limit
	granted the client on the session over which the request is to be		granted the client on the session over which the request is to be
	issued. The slotid must be unused by any of the requests which the		issued. The slotid must be unused by any of the requests which the
	client has already active on the session. "Unused" here means the		client has already active on the session. "Unused" here means the
	client has no outstanding request for that slotid. Because the		client has no outstanding request for that slotid. Because the
	slot id is always an integer in the range 0..N-1, client		slot id is always an integer in the range 0..N-1, client
	implementations can use the slotid from a server response to		implementations can use the slotid from a server response to
	efficiently match responses with outstanding requests, such as, for		efficiently match responses with outstanding requests, such as, for
	example, by using the slotid to index into a outstanding request		example, by using the slotid to index into a outstanding request
	array.		array. This can be used to avoid expensive hashing and lookup
			functions in the performace-critical receive path.
	The sequenceid, which accompanies the slotid in each request, is		The sequenceid, which accompanies the slotid in each request, is
	important for a second, important check at the server: it must be		important for a second, important check at the server: it must be
	able to be determined efficiently whether a request using a certain		able to be determined efficiently whether a request using a certain
	slotid is a retransmit or a new, never-before-seen request. It is		slotid is a retransmit or a new, never-before-seen request. It is
	not feasible for the client to assert that it is retransmitting to		not feasible for the client to assert that it is retransmitting to
	implement this, because for any given request the client cannot		implement this, because for any given request the client cannot
	know the server has seen it unless the server actually replies. Of		know the server has seen it unless the server actually replies. Of
	course, if the server replied, the client would not need to		course, if the client has seen the server's reply, the client would
	retransmit!		not retransmit!
	Therefore, a sequenceid is transmitted along with the slotid in		The sequenceid must increase monotonically for each new transmit of
	each request. The minimum rule is that the sequenceid must change		a given slotid, and must remain unchanged for any retransmission.
	for each new transmit of a given slotid, and must remain unchanged		The server must in turn compare each newly received request's
	for retransmission. This will ensure that the server can detect		sequenceid with the last one previously received for that slotid,
	any new requests by simply comparing the sequenceid with that		to see if the new request is:
	previously issued for the slot. However, it is more useful and
	logical, particularly for tracing and avoiding possible		o A new request, in which the sequenceid is greater than that
	implementation errors, to simply increment the sequenceid for each		previously seen in the slot (accounting for sequence
	new request on any slotid. The sequenceid, in any case, is never		wraparound). The server proceeds to execute the new request.
	interpreted by the server for anything but a test for equality to
			o A retransmitted request, in which the sequenceid is equal to
			that last seen in the slot. Note that this request may be
			either complete, or in progress. The server performs replay
			processing in these cases.
			o A misordered duplicate, in which the sequenceid is less than
			that previously seen in the slot. The server must drop the
			incoming request, which may imply dropping the connection if
			the transport is reliable, as dictated by section 3.1.1 of
			[RFC3530].
			This last condition is possible on any connection, not just
			unreliable, unordered transports. Delayed behavior on abandoned
			TCP connections which are not yet closed at the server, or
			pathological client implementations can cause it, among other
			causes. Therefore, the server may wish to harden itself against
			certain repeated occurrences of this, as it would for
			retransmissions in [RFC3530].
			It is recommended, though not necessary for protocol correctness,
			that the client simply increment the sequenceid by one for each new
			request on each slotid. This reduces the wraparound window to a
			minimum, and is useful for tracing and avoidance of possible
			implementation errors.
			The client may however, for implementation-specific reasons, choose
			a different algorithm. For example it might maintain a single
			sequence space for all slots in the session - e.g. employing the
			RPC XID itself. The sequenceid, in any case, is never interpreted
			by the server for anything but to test by comparison with
	previously seen values.		previously seen values.
	The server may thereby use the slotid, in conjunction with the		The server may thereby use the slotid, in conjunction with the
	session id and sequence id, within the SEQUENCE portion of the		session id and sequence id, within the SEQUENCE portion of the
	request to maintain its duplicate request cache (DRC) for the		request to maintain its duplicate request cache (DRC) for the
	session, as opposed to the traditional approach of ONC RPC		session, as opposed to the traditional approach of ONC RPC
	applications that use the XID along with certain transport		applications that use the XID along with certain transport
	information [RW96].		information [RW96].
	Unlike the XID, the slotid is always within a specific range; this		Unlike the XID, the slotid is always within a specific range; this
	has two implications. The first implication is that for a given		has two implications. The first implication is that for a given
	session, the server need only cache the results of a limited number		session, the server need only cache the results of a limited number
	of COMPOUND requests. The second implication derives from the		of COMPOUND requests. The second implication derives from the
	first, which is unlike XID indexed DRCs, the slotid DRC by its		first, which is unlike XID-indexed DRCs, the slotid DRC by its
	nature cannot be overflowed. Through use of the sequenceid to		nature cannot be overflowed. Through use of the sequenceid to
	identify retransmitted requests, it is notable that the server does		identify retransmitted requests, it is notable that the server does
	not need to actually cache the request itself, reducing the storage		not need to actually cache the request itself, reducing the storage
	requirements of the DRC further. These new facilities makes it		requirements of the DRC further. These new facilities makes it
	practical to maintain all the required entries for an effective		practical to maintain all the required entries for an effective
	DRC.		DRC.
	The slotid therefore takes over the traditional role of the port		The slotid and sequenceid therefore take over the traditional role
	number in the server DRC implementation, and the session replaces		of the port number in the server DRC implementation, and the
	the IP address. This approach is considerably more portable and		session replaces the IP address. This approach is considerably
	completely robust - it is not subject to the frequent reassignment		more portable and completely robust - it is not subject to the
	of ports as clients reconnect over IP networks. In addition, the		frequent reassignment of ports as clients reconnect over IP
	RPC XID is not used in the reply cache, enhancing robustness of the		networks. In addition, the RPC XID is not used in the reply cache,
	cache in the face of any rapid reuse of XIDs by the client.		enhancing robustness of the cache in the face of any rapid reuse of
			XIDs by the client.
	It is required to encode the slotid information into each request		It is required to encode the slotid information into each request
	in a way that does not violate the minor versioning rules of the		in a way that does not violate the minor versioning rules of the
	NFSv4.0 specification. This is accomplished here by encoding it in		NFSv4.0 specification. This is accomplished here by encoding it in
	a control operation within each NFSv4.1 COMPOUND and CB_COMPOUND		a control operation within each NFSv4.1 COMPOUND and CB_COMPOUND
	procedure. The operation easily piggybacks within existing		procedure. The operation easily piggybacks within existing
	messages. The implementation section of this document describes		messages. The implementation section of this document describes
	the specific proposal.		the specific proposal.
	In general, the receipt of a new request using any valid slotid is		In general, the receipt of a new sequenced request arriving on any
	an indication that the previous DRC contents of that slot may be		valid slot is an indication that the previous DRC contents of that
	discarded. In order to further assist the server in slot		slot may be discarded. In order to further assist the server in
	management, the client is required to use the lowest value slotid		slot management, the client is required to use the lowest available
	when issuing a new request. In this way, the server may be able to		slot when issuing a new request. In this way, the server may be
	retire additional entries.		able to retire additional entries.
	However, in the case where the server is actively adjusting its		However, in the case where the server is actively adjusting its
	granted maximum request count to the client, it may not be able to		granted maximum request count to the client, it may not be able to
	use receipt of the slotid to retire cache entries. The slotid used		use receipt of the slotid to retire cache entries. The slotid used
	in an incoming request may not reflect the server's current idea of		in an incoming request may not reflect the server's current idea of
	the client's session limit, because the request may have been sent		the client's session limit, because the request may have been sent
	from the client before the update was received. Therefore, in the		from the client before the update was received. Therefore, in the
	downward adjustment case, the server may have to retain a number of		downward adjustment case, the server may have to retain a number of
	duplicate request cache entries at least as large as the old value,		duplicate request cache entries at least as large as the old value,
	until operation sequencing rules allow it to infer that the client		until operation sequencing rules allow it to infer that the client
	skipping to change at page 44, line 46		skipping to change at page 45, line 29
	{ id_arg, *, old_principal_arg, clientid_ret, TRUE }		{ id_arg, *, old_principal_arg, clientid_ret, TRUE }
	Since the value of the clientdesc.id subfield of each client		Since the value of the clientdesc.id subfield of each client
	record must be unique, there is no modification of the		record must be unique, there is no modification of the
	server's state, and NFS4ERR_CLID_INUSE is returned to indicate		server's state, and NFS4ERR_CLID_INUSE is returned to indicate
	the client should retry with a different value for the		the client should retry with a different value for the
	clientdesc.id subfield of CREATECLIENTID4args.		clientdesc.id subfield of CREATECLIENTID4args.
	This scenario may also represent a malicious attempt to		This scenario may also represent a malicious attempt to
	destroy a client's state on the server		destroy a client's state on the server. For security reasons,
	For security reasons, the server MUST NOT remove the client's		the server MUST NOT remove the client's state when there is a
	state when there is a principal mismatch.		principal mismatch.
	4) Replay		4) Replay
	If the server has the following unconfirmed record then this		If the server has the following unconfirmed record then this
	request is likely the result of a client replay due to a		request is likely the result of a client replay due to a
	network partition or some other connection failure.		network partition or some other connection failure.
	{ id_arg, verifier_arg, principal_arg, clientid_ret, FALSE }		{ id_arg, verifier_arg, principal_arg, clientid_ret, FALSE }
	Since the response to the CREATECLIENTID request that created		Since the response to the CREATECLIENTID request that created
	this record may have been lost, it is not acceptable to drop		this record may have been lost, it is not acceptable to drop
	skipping to change at page 65, line 23		skipping to change at page 66, line 23
	9.2. Informative References		9.2. Informative References
	[BW87]		[BW87]
	B. Welch, "The Sprite Remote Procedure Call System",		B. Welch, "The Sprite Remote Procedure Call System",
	University of California Berkeley Technical Report CSD-87-302,		University of California Berkeley Technical Report CSD-87-302,
	ftp://sunsite.berkeley.edu/pub/techreps/CSD-87-302.html		ftp://sunsite.berkeley.edu/pub/techreps/CSD-87-302.html
	[CCM]		[CCM]
	M. Eisler, N. Williams, "The Channel Conjunction Mechanism		M. Eisler, N. Williams, "The Channel Conjunction Mechanism
	(CCM) for GSS", Internet-Draft Work in Progress,		(CCM) for GSS", Internet-Draft Work in Progress,
	http://www.ietf.org/internet-drafts/draft-ietf-nfsv4-ccm-03		http://www.ietf.org/internet-drafts/draft-ietf-nfsv4-ccm
	[CJ89]		[CJ89]
	C. Juszczak, "Improving the Performance and Correctness of an		C. Juszczak, "Improving the Performance and Correctness of an
	NFS Server," Winter 1989 USENIX Conference Proceedings, USENIX		NFS Server," Winter 1989 USENIX Conference Proceedings, USENIX
	Association, Berkeley, CA, Februry 1989, pages 53-63.		Association, Berkeley, CA, Februry 1989, pages 53-63.
	[DAFS]		[DAFS]
	Direct Access File System, available from		Direct Access File System, available from
	http://www.dafscollaborative.org		http://www.dafscollaborative.org
	[DCK+03]		[DCK+03]
	M. DeBergalis, P. Corbett, S. Kleiman, A. Lent, D. Noveck, T.		M. DeBergalis, P. Corbett, S. Kleiman, A. Lent, D. Noveck, T.
	Talpey, M. Wittle, "The Direct Access File System", in		Talpey, M. Wittle, "The Direct Access File System", in
	Proceedings of 2nd USENIX Conference on File and Storage		Proceedings of 2nd USENIX Conference on File and Storage
	Technologies (FAST '03), San Francisco, CA, March 31 - April		Technologies (FAST '03), San Francisco, CA, March 31 - April
	2, 2003		2, 2003
	[DDP]		[DDP]
	H. Shah, J. Pinkerton, R. Recio, P. Culley, "Direct Data		H. Shah, J. Pinkerton, R. Recio, P. Culley, "Direct Data
	Placement over Reliable Transports",		Placement over Reliable Transports", Internet-Draft Work in
	http://www.ietf.org/internet-drafts/draft-ietf-rddp-ddp-03		Progress, http://www.ietf.org/internet-drafts/draft-ietf-rddp-
			ddp
	[FJDAFS]		[FJDAFS]
	Fujitsu Prime Software Technologies, "Meet the DAFS		Fujitsu Prime Software Technologies, "Meet the DAFS
	Performance with DAFS/VI Kernel Implementation using cLAN",		Performance with DAFS/VI Kernel Implementation using cLAN",
	http://www.pst.fujitsu.com/english/dafsdemo/index.html		http://www.pst.fujitsu.com/english/dafsdemo/index.html
	[FJNFS]		[FJNFS]
	Fujitsu Prime Software Technologies, "An Adaptation of VIA to		Fujitsu Prime Software Technologies, "An Adaptation of VIA to
	NFS on Linux",		NFS on Linux",
	http://www.pst.fujitsu.com/english/nfs/index.html		http://www.pst.fujitsu.com/english/nfs/index.html
	[IB] InfiniBand Architecture Specification, Volume 1, Release 1.1.		[IB] InfiniBand Architecture Specification, Volume 1, Release 1.1.
	available from http://www.infinibandta.org		available from http://www.infinibandta.org
	[KM02]		[KM02]
	K. Magoutis, "Design and Implementation of a Direct Access		K. Magoutis, "Design and Implementation of a Direct Access
	skipping to change at page 66, line 30		skipping to change at page 67, line 32
	Proceedings of 2002 USENIX Annual Technical Conference,		Proceedings of 2002 USENIX Annual Technical Conference,
	Monterey, CA, June 9-14, 2002.		Monterey, CA, June 9-14, 2002.
	[MIDTAX]		[MIDTAX]
	B. Carpenter, S. Brim, "Middleboxes: Taxonomy and Issues",		B. Carpenter, S. Brim, "Middleboxes: Taxonomy and Issues",
	Informational RFC, http://www.ietf.org/rfc/rfc3234		Informational RFC, http://www.ietf.org/rfc/rfc3234
	[NFSDDP]		[NFSDDP]
	B. Callaghan, T. Talpey, "NFS Direct Data Placement",		B. Callaghan, T. Talpey, "NFS Direct Data Placement",
	Internet-Draft Work in Progress, http://www.ietf.org/internet-		Internet-Draft Work in Progress, http://www.ietf.org/internet-
	drafts/draft-ietf-nfsv4-nfsdirect-01		drafts/draft-ietf-nfsv4-nfsdirect
	[NFSPS]		[NFSPS]
	T. Talpey, C. Juszczak, "NFS RDMA Problem Statement",		T. Talpey, C. Juszczak, "NFS RDMA Problem Statement",
	Internet-Draft Work in Progress, http://www.ietf.org/internet-		Internet-Draft Work in Progress, http://www.ietf.org/internet-
	drafts/draft-ietf-nfsv4-nfs-rdma-problem-statement-02		drafts/draft-ietf-nfsv4-nfs-rdma-problem-statement
	[RDDP]		[RDDP]
	Remote Direct Data Placement Working Group charter,		Remote Direct Data Placement Working Group charter,
	http://www.ietf.org/html.charters/rddp-charter.html		http://www.ietf.org/html.charters/rddp-charter.html
	[RDDPPS]		[RDDPPS]
	A. Romanow, J. Mogul, T. Talpey, S. Bailey, Remote Direct Data		A. Romanow, J. Mogul, T. Talpey, S. Bailey, Remote Direct Data
	Placement Working Group Problem Statement, Internet-Draft Work		Placement Working Group Problem Statement, Standards Track
	in Progress, http://www.ietf.org/internet-drafts/draft-ietf-		Informational RFC, http://www.ietf.org/internet-drafts/draft-
	rddp-problem-statement-05		ietf-rddp-problem-statement
	[RDMAP]		[RDMAP]
	R. Recio, P. Culley, D. Garcia, J. Hilland, "An RDMA Protocol		R. Recio, P. Culley, D. Garcia, J. Hilland, "An RDMA Protocol
	Specification", Internet-Draft Work in Progress,		Specification", Internet-Draft Work in Progress,
	http://www.ietf.org/internet-drafts/draft-ietf-rddp-rdmap-03		http://www.ietf.org/internet-drafts/draft-ietf-rddp-rdmap
	[RPCRDMA]		[RPCRDMA]
	B. Callaghan, T. Talpey, "RDMA Transport for ONC RPC"		B. Callaghan, T. Talpey, "RDMA Transport for ONC RPC"
	Internet-Draft Work in Progress, http://www.ietf.org/internet-		Internet-Draft Work in Progress, http://www.ietf.org/internet-
	drafts/draft-ietf-nfsv4-rpcrdma-01		drafts/draft-ietf-nfsv4-rpcrdma
	[RFC2203]		[RFC2203]
	M. Eisler, A. Chiu, L. Ling, "RPCSEC_GSS Protocol		M. Eisler, A. Chiu, L. Ling, "RPCSEC_GSS Protocol
	Specification", Standards Track RFC,		Specification", Standards Track RFC,
	http://www.ietf.org/rfc/rfc2203		http://www.ietf.org/rfc/rfc2203
	[RW96]		[RW96]
	R. Werme, "RPC XID Issues", Connectathon 1996, San Jose, CA,		R. Werme, "RPC XID Issues", Connectathon 1996, San Jose, CA,
	http://www.cthon.org/talks96/werme1.pdf		http://www.cthon.org/talks96/werme1.pdf
	skipping to change at page 68, line 9		skipping to change at page 69, line 9
	535 W. William St. Suite 3100		535 W. William St. Suite 3100
	Ann Arbor, MI 48103 USA		Ann Arbor, MI 48103 USA
	Phone: +1 734 615-4782		Phone: +1 734 615-4782
	Email: baumanj@umich.edu		Email: baumanj@umich.edu
	11. Full Copyright Statement		11. Full Copyright Statement
	Copyright (C) The Internet Society (2005). This document is		Copyright (C) The Internet Society (2005). This document is
	subject to the rights, licenses and restrictions contained in BCP		subject to the rights, licenses and restrictions contained in BCP
	78 and except as set forth therein, the authors retain all their		78, and except as set forth therein, the authors retain all their
	rights.		rights.
	This document and the information contained herein are provided on		This document and the information contained herein are provided on
	an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE		an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
	REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND		REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND
	THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES,		THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES,
	EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT		EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT
	THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR		THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR
	ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A		ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A
	PARTICULAR PURPOSE.		PARTICULAR PURPOSE.
End of changes.
This html diff was produced by rfcdiff 1.24, available from http://www.levkowetz.com/ietf/tools/rfcdiff/